Inhibitions of Nucleus Accumbens Neurons Encode a Gating Signal for Reward-Directed Behavior

Abstract

The nucleus accumbens (NAcc) is critical in the control of goal-directed behavior. Pharmacological studies suggest that the NAcc may act in both instructive and permissive modes; however, previous electrophysiological studies in behaving rats have reported firing patterns consistent with an instructive, but not permissive, role for NAcc neurons. We now report that a subset of NAcc neurons shows a long-lasting inhibition in firing rate whose onset precedes initiation of goal-directed sequences of behavior and terminates at the conclusion of the sequence. Together with data from previous behavioral studies, this firing pattern suggests that, when active, these neurons tonically inhibit appetitive and consummatory behaviors and that, when inhibited, these neurons permissively gate those behaviors.

Introduction

The nucleus accumbens (NAcc) is a brain region that is critical for motivated behavior. The NAcc has been implicated in the generation of goal-directed behavioral responding, but the neural firing that underlies this function is uncertain. In broad terms, NAcc motor encoding could be permissive, instructive, or both. A permissive signal originating in the NAcc would act as a go/no-go gating signal that does not specify the specific behavior to be executed. Instructive signals, in contrast, would participate in the selection of a particular behavioral response.

How might permissive and instructive signaling be encoded in the firing of individual NAcc neurons? Instructive signaling requires that NAcc neural activity facilitate selection of a single behavior from among competing alternatives. Models of instructive NAcc signaling propose that small ensembles of NAcc neurons encode competing behaviors and that inhibitory interactions between neural activity in these ensembles mediate behavioral selection (Pennartz et al., 1994). Thus, these models predict that the firing of subgroups of neurons should be correlated with specific behaviors. Neural encoding related to a permissive signal would not be expected to be correlated with a specific action; rather, it should be modulated in the same way during many different appetitive behaviors, independent of the specific behavior performed.

Electrophysiological studies of NAcc neural firing in behaving animals provide abundant evidence of neural activity correlated with reward-directed behaviors. NAcc neuron firing is typically correlated with very specific behaviors, suggestive of instructive encoding. NAcc neurons discharge in a manner correlated with lever pressing for reward (Chang et al., 1994), as well as nose poking (Nicola et al., 2004a) and locomoting toward a response lever (Peoples et al., 1998). These firing patterns, in which small subsets of neurons encode information related to very specific motor elements of a goal-directed behavior, are most consistent with an instructive role for the NAcc in regulating behavioral selection.

Given the pharmacological evidence for a permissive function of the NAcc, the predominance in electrophysiological studies of firing patterns with specific behavioral correlates is unexpected. To our knowledge, there are few, if any, reports of NAcc neural activity related to a flexible sequence of motivated actions, independent of the particular motor action performed. We report here that a subset of NAcc neurons are inhibited during both learned and spontaneous motivated behavior. We propose that these neurons act as a go/no-go “gate” for motivated behavior, with inhibitions permitting and tonic excitation suppressing the initiation and maintenance of reward-directed behavior.

Materials and Methods

Experimental subjects and surgical procedures. Male Long–Evans rats (n = 12; Charles River Laboratories, Wilmington, MA) were used for electrophysiological recordings. Rats were food restricted to maintain 90% ad libitum body weight. After behavioral training, rats were stereotaxically implanted bilaterally with eight-wire electrode arrays (NB Labs, Denison, TX) directed at the NAcc (anterioposterior, 1.2–1.6; mediolateral, 0.7–1.2; dorsoventral, 7.2–7.5). Arrays were composed of blunt-cut stainless steel 50 μm wires arranged in a 2 × 4 array.

Behavioral paradigms. Six rats were trained in a delayed-response task in an operant chamber (42 × 53 × 40 cm) equipped with an audio stimulus generator, house lights, and a central nosepoke (NP) hole flanked by identical reward receptacles (RRs). Trials were initiated by the presentation of a white noise cue, after which the animal could initiate a response at any time by executing a sustained nosepoke (see Fig. 1 A). One of two instructive tones (3 or 7 kHz at 90 dB, chosen randomly) was delivered 200–350 ms (randomized) after the initiation of the nosepoke, cueing the animal to respond to either the left or the right RR for delivery of a sucrose reward. Rats were required to withhold responding until tone offset. Tone duration was randomly varied (950–1550 ms). Tone–response direction pairings remained constant for each rat throughout training and recording sessions but were counterbalanced across rats. Successful completion of each trial required that the nosepoke be maintained for the duration of the tone presentation and that the subsequent response be directed to the appropriate RR. Thus, each successful trial consisted of three successive behavioral intervals: a delay period (beginning with the nosepoke and ending with the nosepoke termination or nosepoke break); a movement period (beginning with the nosepoke break and ending with RR entry); and a reward period (beginning with RR entry and ending with RR exit). Successful trials resulted in delivery of 100 μl of 10% sucrose into the instructed RR. During a 4 s interval immediately after RR exit during sucrose delivery, new trials could not be initiated. Thereafter, white noise cue delivery signaled the start of the next trial.

Figure 1.

Schematic of delayed-response paradigm and recording locations. A, A white noise cue (“Trial start signal”) signified the start of each trial; thereafter, rats could initiate a response by executing a nosepoke within a nosepoke port. One of two instructive tones (3 or 7 kHz, chosen randomly) was delivered at short variable latency after nosepoke onset, cueing the rat to respond to either the left or right reward receptacle for a sucrose reward. Rats were required to maintain the nosepoke until tone offset, at which point a movement to the instructed reward receptacle resulted in delivery of 0.1 ml of 10% sucrose. Thus, each successful trial involved a delay period (in which the nosepoke was maintained), a subsequent movement period (in which the rat moved to the correct RR), and a final reward period, in which sucrose was consumed. B, The location of the reconstructed position of each recording array is shown in coronal sections. Distance anterior from bregma is shown to the right of each section (in millimeters).

Termination of the nosepoke before tone offset constituted an incorrect response, as did responding to the noninstructed RR. Errors of either kind terminated the ongoing trial and were signaled by extinguishing the house lights for a period of 10 s, during which sucrose rewards could not be earned and additional trials could not be initiated. Rats were trained in the delayed-response task until they completed >50 successful trials in two consecutive behavioral sessions.

An additional six rats were trained in a simple sucrose consumption task. Rats were trained to lick a spout to receive brief (1.5 s average) access to either 0 or 10% sucrose solution (randomized). After training, we performed simultaneous electrophysiological recording and videotaping of these animals to (1) identify neurons that were inhibited during reward-directed consummatory behavior and (2) correlate firing in these neurons with locomotor bouts that followed reward receptacle exit. Visual inspection was used to identify locomotor bouts after reward receptacle exit, which were time stamped using a digital video timer.

Single-unit recording and discrimination. Neural signals were recorded with a unity head stage amplifier, amplified 10,000-fold and captured digitally using commercial hardware and software (Plexon Instruments, Dallas, TX). Discrimination of individual units was performed off-line using principal component analysis of waveform shape. Single units were identified by constancy of waveform shape, autocorrelogram, and interspike interval.

Analyzing neural response properties. For the delayed-response task, we first identified neurons that were inhibited during consummatory behavior, a firing pattern described previously (Taha and Fields, 2005). Significant inhibitions were identified using the Mann–Whitney rank sum test (p < 0.05), comparing firing during correct RR visits with that occurring during a baseline period that occurred 1 s before the white noise cue, signaling trial start. To identify long-lasting inhibitions, we identified neurons within this group in which inhibition of firing continued in successive behavioral intervals that preceded RR visit (i.e., during movement or during delay plus movement intervals). Only those neurons in which the inhibition began before the RR visit and persisted from the time of initiation through the end of the RR visit interval were included in the following analysis.

Onset and offset of inhibition in firing were calculated for each neuron by extracting, from a smoothed (boxcar filter with 0.5 s kernel) histogram of mean firing, the time at which the absolute rate of change of firing was maximal. A similar technique was used to identify the time of the offset of inhibition; the sign of the rate of change of firing was negative for inhibition onsets and positive for inhibition offsets.

Histograms of mean firing were plotted by averaging all of the individual neuron histograms constructed around each behavioral event of interest. Because inhibitions were sustained for the duration of a single epoch of goal-directed behavior, we identified “terminal” behavioral events, which defined the end point of a goal-directed behavioral sequence. These terminal behavioral events were those not followed by additional receptacle entries or nosepokes within 2 s. These included the following: RR Exit (see Fig. 3A, last panel); Wrong RR Exit (see Fig. 3B, last panel); Spontaneous RR Exit (see Fig. 4 A, second panel); premature nosepoke breaks (see Fig. 4 D); and RR₂ Exit (see Fig. 4 F, last panel). Conversely, behavioral events associated with persistent reward seeking were those that were followed by receptacle entries at short latency (within 0.5 s) and included premature nosepokes followed by movement to an RR (see Fig. 4C) and RR₁ Exit (see Fig. 4 F, first panel).

Figure 3.

A subset of NAcc neurons is inhibited for the duration of motivated behavior in the delayed-response task. A, Population histogram shows that inhibition onset occurred before the nosepoke that initiated the delay period (second panel) and persisted for the duration of performance in the delayed-response task. Graph conventions are identical to those used for perievent histograms in Figure 2. NPB, Nosepoke break. B, Firing in this group of neurons was not affected by reward, because inhibition occurred even during unrewarded error trials. C, Firing was significantly inhibited during correct performance in the delayed-response task (gray bars: delay, move, reward periods) but not before (baseline) or after (after reward). Firing was also inhibited during incorrect reward receptacle entries during error trials (black bar), which were never reinforced. The broken gray line indicates baseline firing rate in all graphs. n = 30 neurons for all graphs; asterisks indicate significant inhibition relative to baseline firing (p < 0.05; ANOVA on ranks; Dunn's post hoc test). Error bars represent SEM.

Figure 4.

Inhibition of NAcc neurons occurs during goal-directed behavior outside the delayed-response task. A, Population histogram shows that NAcc neurons were inhibited during spontaneous visits to RRs, which occurred outside of the delayed-response task. B, Mean firing was significantly inhibited during spontaneous RR visits but not during the second before (left bar) or after (right bar) these visits. C, In most error trials in which the delay period was prematurely terminated, rats followed the delay period with a movement to the instructive reward receptacle, despite delivery of an error signal. In these trials, mean activity remained inhibited during the movement that followed the delay period. D, Occasionally, rats failed to move to the reward receptacle after premature delay termination and error signal delivery. In these trials only, firing abruptly returned to baseline firing levels. Because delay periods were very short in these trials (0.41 ± 0.05 s), the onset of inhibition is apparent to the left of time 0. E, Inhibition of firing occurred in all prematurely terminated delays but was sustained thereafter only in trials in which the delay period was followed by movement to the reward receptacle (Move). F, G, After correct trials, inhibition was maintained during persistent reward seeking. Spont., Spontaneous; NPB, nosepoke break. Error bars represent SEM.

Average firing rates were calculated for the following behavioral intervals: baseline (which occurred 1 s interval before white noise trial start signal; delay (interval from nosepoke to nosepoke break); movement (nosepoke break to RR entry); reward (correct RR entry to RR exit); no reward (incorrect RR entry to RR exit); and spontaneous RR (spontaneous RR entry to RR exit). Analysis intervals occurring before or after behavioral events (e.g., “after reward”) (see Fig. 3C) were always 1 s in duration and ended or began with the behavioral event, respectively. For statistical analysis of firing rates, significant decreases in firing were identified by comparing firing during the baseline period with firing in the behavioral interval of interest using a repeated-measures ANOVA, with Dunn's test used to compare firing against baseline activity. For pairwise comparisons of firing rates (i.e., firing rate during leftward vs rightward movement), Wilcoxon's signed rank test was used.

For the sucrose consumption task, neurons with inhibitions occurring during consummatory behavior were identified by nonparametric one-way ANOVA, comparing baseline firing with that occurring during 0 and 10% sucrose consumption. Firing in this population of neurons was subsequently analyzed during a 1 s interval starting at the onset of locomotion away from a lick spout after sucrose consumption. Locomotor bouts were identified by inspection of videotaped behavioral sessions; a digital video counter with 10 ms time resolution was used to timestamp each locomotor bout.

Histology. Rats were deeply anesthetized, and recording sites were marked by passing 20 μA current for 20 s through each electrode. Rats were perfused with a solution of 10% formaldehyde and 3% potassium ferricyanide to mark sites of iron deposition. Brains were cryoprotected, section, and stained with cresyl violet, and recording sites were identified under a light microscope.

Results

We studied a total of 149 neurons in six rats trained in the delayed-response task (Fig. 1A) at recording sites that included both the core and the shell of the nucleus accumbens (Fig. 1B). Of the recorded neurons, a fraction (53 of 149, 36%) possessed firing that was inhibited during consummatory behavior (i.e., during rewarded receptacle visits), consistent with previous reports (Nicola et al., 2004b; Taha and Fields, 2005).

A subset of these NAcc neurons (30 of 149, 20%) showed a long-lasting inhibition that began far in advance of consummatory behavior. These inhibitions were equally likely to be found in the NAcc core (20 neurons) or shell (10 neurons; p = 0.28, z test). Figure 2 shows typical firing patterns from two example neurons that occurred during correct performance of the delayed-response task. It is striking that the inhibition of activity that occurred in these neurons started even before the NP response, which initiated the delay period. The firing rate in these neurons remained low throughout the subsequent movement and reward periods and only recovered to baseline levels after the rats exited the reward receptacle after consuming the sucrose reinforcer.

Figure 2.

Example firing patterns from two nucleus accumbens neurons showing long-lasting inhibition during the delayed-response task. A, Perievent raster (top) and histogram (bottom) are shown for a single NAcc neuron. For each raster and paired histogram, the x-axis shows the time (in seconds) relative to the behavioral event around which the graphs were constructed. The y-axis shows the firing rate for each neuron. Labeled arrows (above middle panels) show the behavioral intervals occurring before and after each event. Labels below each graph show the behavioral event around which the graphs were constructed. NPB, Nosepoke break. Only correct trials are included in these graphs. Filled circles in the raster show white noise cue presentation. B, A second NAcc example neuron. Graph conventions are identical to those shown in A. In both example neurons, firing rate was significantly inhibited relative to baseline (left panels) for the duration of performance in the delayed-response task (during delay, movement, and reward intervals) and recovered to baseline levels only after exit from the RR.

In this class of neurons, the mean onset time of inhibition preceded the NP that initiated the delay period (0.45 ± 0.15 s before the NP; mean ± SEM) (Fig. 3A, population histogram shown). Neural activity in this population was significantly reduced during successive delay, movement, and reward consumption intervals (Fig. 3C) (all p < 0.05 relative to baseline). Firing in this class of neurons returned to baseline levels of activity shortly after rats finished sucrose consumption and exited the RR (0.47 ± 0.08 s after RR exit). This firing pattern was unrelated to the instructive cues that indicated required response direction, because it began before instructive cue presentation and terminated after the offset of this cue. The magnitude of the inhibition occurring during delay and movement periods was very similar (delay, 0.62 ± 0.11 Hz; movement, 0.66 ± 0.15 Hz; p = 0.79), demonstrating that modulation in these neurons was not related to the presence or absence of movement per se. Nor did firing differ as a function of the movement direction (left, 0.83 ± 0.21 Hz; right, 0.91 ± 0.2 Hz; p = 0.52, data not shown). Although inhibition occurred during sucrose consumption, it was not dependent on reward delivery. Significant inhibition of firing occurred in these neurons even during unrewarded error trials in which rats moved to the uninstructed RR (Fig. 3B) (rewarded, 0.83 ± 0.17; unrewarded, 0.99 ± 0.23; p = 0.98).

Thus, inhibition of firing in these neurons was sustained for the duration of performance in the delayed-response task, and firing was not modulated by instructive cues, movement parameters, or reward consumption. However, this inhibition of firing did not occur solely during performance of the delayed-response task. Four additional observations suggest that inhibition occurred in this class of neurons during all goal-directed behavior, independent of the context in which that behavior occurred.

First, inhibitions occurred during spontaneous RR visits, which occurred outside the delayed-response task. Although an NP was required to initiate the delay period, rats sometimes made visits to an RR without a preceding NP. Firing rate was significantly reduced during these spontaneous, unreinforced RR visits (Fig. 4A,B) (0.89 ± 0.21 Hz during RR visit; p < 0.05 relative to baseline).

Second, rats often prematurely terminated the delay period before the offset of the instructive cue. These errors resulted in the immediate delivery of an error signal. Significant inhibition of these neurons occurred during these prematurely terminated delays (0.70 ± 0.25 Hz; p < 0.05 relative to baseline). Rats usually followed even prematurely terminated delay periods with immediate movement to a RR; in these cases, inhibition persisted for the duration of the subsequent movement (Fig. 4C) (0.70 ± 0.17 Hz). Occasionally, however, rats abruptly terminated goal-seeking behavior after delivery of the error signal and did not visit an RR receptacle (Fig. 4D). In these trials only, inhibition abruptly ceased and returned to baseline firing (Fig. 4E) (1.5 ± 0.38 Hz within 1 s after delay termination).

Third, after correct task performance, following consumption of sucrose reward and RR exit, animals sometimes moved immediately to the other (unrewarded) RR (RR₂). These visits were highly stereotypic, in that they occurred at short latency after RR exit and usually occurred in only one direction for a given rat. We speculate that this behavior arose through the training schedule, in which sucrose was delivered to both RRs on each trial during initial training. During this sustained reward-seeking behavior, inhibition persisted during movement between the RRs and during the subsequent unrewarded RR visit (Fig. 4F,G) (move, 1.1 ± 0.28; RR₂, 1.1 ± 0.23 Hz; both p < 0.05 relative to baseline). This persistent inhibition contrasted sharply with trials in which rats did not engage in persistent reward seeking; in these trials, activity returned to baseline levels shortly after rats exited the first RR [compare Figs. 3A (last panel), 4F (first panel); both histograms are constructed around first RR exit].

Finally, these inhibitions occurred only during reward-directed behaviors and not during all movement sequences. Locomotor bouts (toward the nosepoke port) preceding each trial were qualitatively similar to locomotion bouts occurring after sucrose consumption (away from the reward receptacle). However, inhibition onset occurred only during the former and not during the latter. To study this, six additional rats were trained in a simple sucrose consumption paradigm, and simultaneous electrophysiological recording and videotape analysis was performed. Ten of 50 neurons recorded from these rats showed significant inhibition during sucrose consumption (0.75 ± 0.33 Hz) relative to baseline firing (2.0 ± 0.56 Hz; p < 0.05). However, during locomotor bouts that followed sucrose consumption (identified through videotape analysis), firing was not inhibited. In contrast, firing in these neurons during these locomotor bouts was significantly elevated (3.39 ± 0.98, p < 0.05, paired t test). These data provide evidence that inhibitions in these neurons are specific to reward-directed behavioral sequences.

Discussion

We found that inhibition in a subset of NAcc neurons occurred when rats engaged in goal-directed behaviors, both during and outside of a delayed-response task. This inhibition did not reflect a response to task-relevant sensory cues, task parameters, specific motor actions, or reward delivery. Rather, these inhibitions were sustained during the performance of a sequence of reward seeking and consumption, independent of the context in which these behaviors occurred.

It is significant that the onset of these inhibitions occurred just before the initiation of goal-directed behavior. The great majority of NAcc neurons are medium spiny projection cells that contain the inhibitory neurotransmitter GABA, and therefore inhibition of neural activity in these neurons would be expected to disinhibit target regions. Thus, the data we present are consistent with a role for the sustained inhibition of these NAcc neurons in permissively gating and maintaining a sequence of appetitive behaviors.

Our electrophysiological data are correlative and do not prove that these inhibitions are causal to appetitive behavior. However, there is extensive behavioral evidence demonstrating that inhibition of NAcc neurons gates reward-directed behavior through disinhibition of target brain regions (Stratford and Kelley, 1997, 1999; Zhang et al., 2003). Pharmacological inactivation of the medial shell subregion of the NAcc results in a short-latency, intense hyperphagia (Stratford and Kelley, 1997), which requires neural activity in a projection target of the NAcc, the lateral hypothalamus (Stratford and Kelley, 1999). Furthermore, inactivation of the NAcc shell causes widespread increases in c-fos expression in LH neurons, consistent with disinhibition of neural activity (Zheng et al., 2003; Baldo et al., 2004).

In a previous publication, we showed that a large population of NAcc neurons was inhibited during consummatory behavior and that the magnitude of this inhibition was directly correlated with the frequency with which consummatory behavior (licking) occurred (Taha and Fields, 2005). We suggested that inhibition of these neurons might underlie feeding caused by NAcc muscimol infusion. The current results extend those findings by showing that inhibition in many of these NAcc neurons is not confined to the time in which consummatory behavior takes place but also occurs during the preceding appetitive behaviors. This finding is consistent with behavioral studies that demonstrate increased appetitive behavior after NAcc ablation (Bowman and Brown, 1998).

An extensive amount of literature provides evidence that a dopamine-dependent mechanism in the NAcc (in particular, the NAcc shell) gates appetitive behavior. Infusion of the psychostimulant and indirect dopamine agonist amphetamine into the NAcc shell increases operant responding to a reward-associated cue (Wyvell and Berridge, 2000) and increases breakpoint in a progressive ratio task (Zhang et al., 2003). These studies suggest that dopamine signaling in the NAcc can regulate the magnitude of operant responding in a variety of experimental paradigms. Although the electrophysiological effects of amphetamine are complex, direct iontophoresis of the drug causes inhibition in a large majority of NAcc neurons (Kiyatkin and Rebec, 1997). Thus, we speculate that amphetamine effects on response magnitude may be mediated, in part, by pharmacological inhibition of the class of NAcc neurons with firing that is normally inhibited during appetitive behavior.

In summary, neural activity in a subset of NAcc neurons has sustained inhibition during goal-directed behavioral responding, consistent with a role for these neurons in gating appetitive and consummatory behaviors through disinhibition of target brain regions. Evidence from behavioral studies supports this conclusion, because increasing either GABAergic or dopaminergic signaling, both of which typically inhibit neural activity, in the NAcc shell increases reward-directed behavior. Understanding the possible role of this neural gate for appetitive and consummatory behavior may be critical in advancing our understanding of the neural changes that underlie disorders of motivated behavior, such as obesity and addiction.