The Ventral Striatum in Off-Line Processing: Ensemble Reactivation during Sleep and Modulation by Hippocampal Ripples

Abstract

Previously it has been shown that the hippocampus and neocortex can spontaneously reactivate ensemble activity patterns during post-behavioral sleep and rest periods. Here we examined whether such reactivation also occurs in a subcortical structure, the ventral striatum, which receives a direct input from the hippocampal formation and has been implicated in guidance of consummatory and conditioned behaviors. During a reward-searching task on a T-maze, flanked by sleep and rest periods, parallel recordings were made from ventral striatal ensembles while EEG signals were derived from the hippocampus. Statistical measures indicated a significant amount of reactivation in the ventral striatum. In line with hippocampal data, reactivation was especially prominent during post-behavioral slow-wave sleep, but unlike the hippocampus, no decay in pattern recurrence was visible in the ventral striatum across the first 40 min of post-behavioral rest. We next studied the relationship between ensemble firing patterns in ventral striatum and hippocampal ripples-sharp waves, which have been implicated in pattern replay. Firing rates were significantly modulated in close temporal association with hippocampal ripples in 25% of the units, showing a marked transient enhancement in the average response profile. Strikingly, ripple-modulated neurons in ventral striatum showed a clear reactivation, whereas nonmodulated cells did not. These data suggest, first, the occurrence of pattern replay in a subcortical structure implied in the processing and prediction of reward and, second, a functional linkage between ventral striatal reactivation and a specific type of high-frequency population activity associated with hippocampal replay.

Introduction

Memory consolidation is a multifaceted process that can last up to many years in humans. Most work on memory consolidation has been focused on declarative-episodic memory, which strongly depends on the integrity of the hippocampus and adjacent structures, at least for an initial period after behavioral experience (Scoville and Milner, 1957; Winocur, 1990; Squire and Zola-Morgan, 1991; Kim and Fanselow, 1992). Their time-limited role in memory formation has led researchers to hypothesize that, eventually, memory traces must be stored outside the hippocampus. With its vast computational capacities, the neocortex has traditionally been considered the main final repository of context-dependent memory (Squire and Zola-Morgan, 1991; McNaughton, 1998; but see Nadel and Moscovitch, 1997).

Around the same time that experimental studies began to suggest a role for sleep in memory consolidation (Fishbein, 1971; Pearlman and Greenberg, 1973; but see Siegel, 2001), Marr (1971) proposed that, during periods such as sleep, when the brain is not occupied with the processing of ongoing events (i.e., “off-line”), spontaneous reactivation of memory traces in the hippocampus may initiate and coordinate a process of information retrieval and repeated transfer within the neocortex, leading to the stabilization and refinement of the cortical memory trace. Contemporary studies have supported the notion of spontaneous, temporally specific replay of firing patterns in hippocampal area CA1 during postexperiential sleep and rest states (Pavlides and Winson, 1989; Wilson and McNaughton, 1994; Skaggs and McNaughton, 1996; McNaughton, 1998; Kudrimoti et al., 1999; Louie and Wilson, 2001). Pattern recurrence in the hippocampus appears especially strong during high-frequency (∼200 Hz) EEG events called ripples, which are coupled to sharp waves (Buzsáki, 1986; Kudrimoti et al., 1999). Off-line reactivation might serve to strengthen synaptic connections within hippocampal cell assemblies storing a representation but may also be hypothesized to help coordinate consolidation in target structures.

Hippocampal-neocortical interactions, however, are unlikely to account for the entire spectrum of memory consolidation processes. Formation of nondeclarative or procedural memory critically depends on the basal ganglia but not the hippocampus (Mishkin et al., 1984; Knowlton et al., 1996; White, 1997). This type of memory has been suggested to improve after sequences of slow-wave and rapid eye movement (REM) sleep (Karni et al., 1994; Gais et al., 2000; Stickgold et al., 2000). Associations between discrete sensory cues and reinforcers are formed in many types of learning but do not require an intact hippocampus, at least under some circumstances (Selden et al., 1991; McDonald and White, 1993).

These considerations prompted us to investigate whether reactivation also exists in the ventral striatum (VS), a basal fore-brain structure that receives a strong input from hippocampal area CA1 and subiculum (Groenewegen et al., 1987; Pennartz et al., 2001). This structure has been implicated in the guidance and direction of behavior on the basis of the motivational value of environmental stimuli (Schultz et al., 1992; Pennartz et al., 1994; Cardinal et al., 2002; Hernandez et al., 2002). In addition to investigating reactivation in the VS, we asked how its dynamics are in comparison with hippocampal reactivation and whether hippocampal ripples, which reflect reactivation in this structure, are in any way coupled with ventral striatal reactivation.

Materials and Methods

Subjects

Subjects were three adult male Fisher 344 rats (Charles River Laboratories, Wilmington, MA) weighing 320-340 gm. Animals were housed individually, weighed and handled daily, and kept on a 12 hr light/dark cycle with lights off at 10 A.M. throughout the training and recording period. Water and food (rat chow, 4% mouse/rat diet 7001, Harlan Teklad, Madison WI) were available ad libitum until 8-4 d before surgery, when shaping on the task began. During the training and recording periods, rats were maintained at 80% of their weights as observed under ad libitum conditions and had access to water at all times. All procedures were performed following National Institutes of Health guidelines for the use of vertebrate animals in research.

Behavioral apparatus and procedures

A wooden T-maze was used for training and recording sessions. It consisted of a T-stem (77 × 6 cm) and two arms (each 39 × 6 cm) placed orthogonally with respect to the stem. According to their geographic position, these arms were labeled the “south” and “north” arm, respectively. The maze was placed on a platform 1 m above the floor, in the center of a dimly illuminated room (3.9 × 3.9 × 2.5 m) with black walls, black ceiling, and various visual cues at the periphery. Food rewards could be placed in two cups (diameter, 1 cm; height, 2 cm) placed at a distance of 4 cm from the outer edges of the south and north arms. This food consisted of a soft, easily edible mix of rat chow and apple sauce.

On the eighth day before surgery, all three animals were subjected to the following behavioral procedure once per day. First, the rat was placed on a “nest” consisting of a towel folded into a 25-cm-wide, deep plate on a flower pot, which was placed on the center of the T-maze. The rat was allowed to sleep or rest in this nest for 15-30 min (REST1 period). Next, the nest and flower pot were removed, and the rat was placed in the T-stem, which marked the onset of the active behavioral period (MAZE) lasting 10-20 min. At this stage, a wooden block placed at the end of the T-stem merging into the south and north arms restricted the rat's movements to positions within the stem, and the rat was allowed to explore this nonrewarded space for a few minutes. Subsequently the wooden barrier was removed, and the rat entered either the south or the north arm. The wooden block was reinstated after the rat made a choice. In this initial choice trial, food reward was allocated to the arm designated as having a high reward probability. When failing to encounter a reward in the chosen arm, the rat was allowed to explore the other, rewarded arm. After this choice phase, the rat shuttled back and forth between the two arm ends without entering the T-stem and collected rewards that were allocated according to a probabilistic pseudorandom schedule (high reward probability: 75%; low reward probability: 25%). A total of 30-80 shuttle trials were given per session. Thus, the rat was not certain to obtain a reward after any given shuttle trial but encountered a 3:1 difference in reward probability between the two arms. After maze running, the rat was placed into the nest again and slept or rested for 15-40 min (REST2). In addition to this behavioral protocol, each rat was subjected to a series of choice trials in a few sessions not intended for studying reactivation but for assessing the learned preference of the rat to choose the arm with high reward probability. In accordance with the results from single choice trials, these additional sessions confirmed the development of a strong preference for the high-reward arm (percentage of total choices: 87 ± 8%; n = 6; p < 0.05 relative to random performance). After reversing the reward probabilities between the arm ends, the rat's preference was observed to shift across a few sessions. Although the present data do not yield precise insights into the relationship between reactivation and the degree of novel learning, it is noteworthy that a previous study in the hippocampus found such a relationship (Gerrard, 2002).

Both the REST1 and REST2 phases can be denoted as off-line periods in the sense that they are not characterized by active, reward-seeking behavior in the task. The use of the term off-line should not be taken to imply that the hippocampus, ventral striatum, or other brain structures would be cognitively or neurophysiologically inactive or would not process relevant information.

The procurement of food reward by deprived animals implies a change in motivational status during performance on the T-maze. In principle, such a change might influence reactivation measures; however, there are three main arguments against this possibly confounding factor. First, the relatively short MAZE period was not sufficient to remove the intense drive to seek food in the rats tested. In each of the rats, we incidentally tested whether they were able to run the T-maze task two times on the same day, with intermittent rest-sleep periods. This was the case in all three rats. Second, if motivational-deprivation status significantly affected the results, one would expect to see a difference in reactivation measures between the sleep-rest periods and the first versus second half of the MAZE period, because the latter two are marked by a decreasing degree of food deprivation; however, no marked differences in explained variance (EV) and reverse explained variance (REV) values (see below) for the first and second half of MAZE were found. Third, a change in motivational status would be expected to have a generalized effect on firing activity and cross-correlations, unlike the reactivation effects reported here.

Surgery and electrophysiology

Tetrodes were used to obtain stable, parallel recordings of many well isolated single units (McNaughton et al., 1983; Recce and O'Keefe, 1989; Gray et al., 1995). A multi-electrode microdrive containing 12 tetrodes and 2 single reference electrodes was implanted unilaterally above the ventral striatum of each rat (Gothard et al., 1996). Briefly, tetrodes (diameter, ∼30 μm) were made by wiring four nichrome electrodes (H. P. Reid, Palm Coast, FL) together in a microbundle. Electrode tips were gold plated to reduce their impedance to ∼0.5-1.0 MΩ. They were mounted on 14 independently movable microdrives having an action range of 9 mm from the cortical surface. These microdrives were arranged along the perimeter of a circle and descended at a 30° angle curving toward the center of the drive array, so that the tetrodes converged to a 1.5-mm-wide, circular exit grid at its bottom. Twelve tetrodes were used for unit recordings in the ventral striatum, whereas two electrodes were placed in the neocortex to serve as a reference and to record neocortical EEG traces. In addition, two electrodes (insulated stainless steel, 50 μm diameter twisted and glued together with a 500 μm tip separation) were implanted in the hippocampus to record local EEG activity differentially between the stratum oriens and stratum lacunosum.

Animals were anesthetized with sodium pentobarbital (40 mg Nembutal/kg body weight, i.p.) and mounted in a Kopf stereotaxic frame. The exit port of the microdrive assembly was centered on 1.7 mm anterior and 1.3 mm lateral to bregma, above the mid-portion of the ventral striatum in the anterior-posterior direction (Paxinos and Watson, 1986). Hippocampal EEG electrodes were placed in the contralateral hemisphere (3.8 mm posterior, 2.0 mm lateral to bregma) and lowered into the pyramidal cell layer and stratum radiatum under electrophysiological guidance. Whereas the latter electrodes were directly placed at their target positions and fixed to the skull, the 12 tetrodes for unit recording were gradually lowered toward the ventral striatum in the course of the first 6 d after surgery. During the first 5-6 d after surgery, rats were allowed to recover under ad libitum feeding conditions and then re-exposed to the behavioral procedure described above.

Recording sessions were initiated as soon as all tetrodes were estimated to have entered the ventral striatum, provided that the animal had regained its presurgery performance level. On consecutive recording days, individual tetrodes could be lowered farther, depending on cell yield. The percentage of tetrodes being moved in between the reported recording sessions amounted to 68.3 ± 6.3%, indicating that most of the units were changing from session to session.

Each lead of a tetrode was connected to a unity-gain field-effect transistor preamplifier, the output of which was relayed via insulated, multiwire cables to a 64 channel commutator and then fed into an 8 channel amplifier (Neuralynx, Tucson, AZ) for amplification (5000 times) and bandpass filtering (0.6-6.0 kHz). Amplifier output was digitized at 32 kHz using a Cheetah ADC interface (Neuralynx) and stored on a Sun Solaris workstation. A 1 msec data sample was taken whenever the signal exceeded a preset voltage threshold, thus capturing 32 data points distributed across the width of an extracellularly recorded spike. The signal was also sent to a speaker and an oscilloscope for auditory and visual inspection during recording. Neocortical and hippocampal EEG signals were amplified 500 times, bandpass filtered between 1 and 3000 Hz, and sampled at 2.5 kHz. The rat's head position was monitored with the use of light-emitting diodes fixed on the headstage and a video camera mounted 2 m above the center of the T-maze. Video images were relayed to a television monitor, video recorder, and the Cheetah ADC interface for digitization. Thus, all behavioral phases of the experiment could be studied off-line using analog videotape as well as digitized position records. The effective spatial resolution of these position records was 1.5 cm per bin (see Fig. 1).

Figure 1.

Spatial distribution of firing rates of 18 representative individual single units recorded simultaneously in the ventral striatum during performance of the T-maze task. Each panel represents the firing pattern of a single unit. Time-stamped spikes of each single unit were attributed to a spatial bin on the T-maze, each of which was 1.5 cm² in size. The local firing rate was computed by dividing the number of spikes for each bin by the total amount of time spent in that bin. A smoothing procedure was used by which the firing rate estimate for each bin was calculated by averaging the value of the bin and those of its four neighboring bins. Color-coded firing rates increase from dark-red color to progressively “hotter” colors (orange, yellow, white), with maximal firing rates (maxfr) corresponding to the numbers plotted atop each panel. For each panel, the cell number (e.g., 11.1) is denoted by the tetrode number (11) followed by the cluster number (0.1) on the tetrode. In the top left panel, “S” and “N” denote the south and north arm ends of the T-maze, respectively.

Data analysis

Unit identification. Single units were discriminated off-line using established cluster-cutting methodology (Glaser and Ruchkin, 1976; Mizumori et al., 1989; Gray et al., 1995). Several spike parameters recorded on each lead of the tetrode were used to generate multidimensional scatter plots, including peak amplitudes, area under the spike waveform, and spike width. Spike interval histograms, autocorrelograms, and average waveforms were checked visually before a putative cluster of spikes was accepted as belonging to one unit. Care was taken to separate noise from spike activity and confirm unit stationarity across the recording period (i.e., REST1, MAZE, and REST2). A distinction was made between putative fast-spiking interneurons (Kawaguchi, 1993) and other cells (mostly medium spiny neurons). These putative interneurons were identified by (1) their low values of spike width (≤0.31 msec peak-to-valley time) and (2) a spike waveform with similar shapes and widths of peak and valley phases (mirror symmetry). The 11 neurons satisfying these criteria fired at average rates >5 Hz. The exclusion of these cells from the ensemble did not markedly alter the mean values of our measures of reactivation.

Assessment of sleep-wake states and ripple detection. Hippocampal and neocortical EEG traces were used to characterize sleep-wake states and detect hippocampal ripples-sharp waves. During REST1 and REST2 episodes, four different states were classified: slow-wave sleep (SWS), REM sleep, quiet wakefulness, and active wakefulness. This classification was based on a combination of EEG and behavioral criteria, first assessed on-line by one or two investigators watching the television monitor and EEG computer display, and subsequently assessed off-line by a third observer using the videotapes and software for EEG display. This off-line software displayed raw EEG traces next to 100-300 Hz bandpass-filtered traces revealing hippocampal sharp-wave-ripple complexes and other characteristic EEG events. SWS was defined by (1) large-irregular activity (LIA) in the hippocampal EEG, with intermittent ripple-sharp wave activity; absence of hippocampal theta rhythm; (2) immobility of the rat, assuming a sleeping posture with eye lids closed; and (3) an onset often marked by neocortical spindle activity, although the latter was not an obligatory condition. REM sleep was recognized by (1) a dominant 6-12 Hz (theta) rhythm in hippocampal EEG, with LIA and ripples-sharp waves being absent; (2) complete immobility of the rat, after an earlier period of sleeping that lasted at least several minutes; and (3) a desynchronized neocortical EEG. Because the amounts of REM sleep were insufficient for reactivation analysis, dominant theta rhythmicity was assessed by visual inspection, and the resulting temporal segments were used merely to exclude them from further analysis on the occurrence of reactivation during different sleep-wake states. Quiet wakefulness was characterized by predominant lack of whole-body movement of the rat, having its eyes open for most of the time and either lying down or assuming a still, attentive posture. In this state, neocortical and hippocampal EEG traces were devoid of spindles and theta rhythm, respectively, but ripples were typically present. Active wakefulness was marked by behavioral activity, including body turns, grooming, and short bouts of locomotion on the nest; some of these periods were accompanied by hippocampal theta rhythm (Vanderwolf et al., 1975).

Ripples were automatically detected by rectifying voltage values of the 100-300 Hz bandpass-filtered EEG trace recorded from the hippocampal pyramidal cell layer and applying a preset threshold (3× the SD of the trace). Care was taken to remove artifacts from the list of threshold crossings. Peri-ripple time histograms (PRTHs) were constructed by aligning single-unit spike trains to ripple onset. The statistical significance of ripple modulation of VS firing rate was assessed by applying a one-way ANOVA test.

Assessment of covariation in firing rate and reactivation. Across each episode (REST1, MAZE, and REST2), spike trains from all simultaneously recorded ventral striatal neurons were binned into T intervals to obtain, for each cell i, a sequence of spike counts per consecutive bin (f_i). For reactivation analysis we used the last 10 min of REST1, immediately preceding the behavioral phase, as well as the first 10 min of REST2, immediately after the behavior, unless noted otherwise. Because both of the rest-sleep segments were temporally adjacent to the behavioral phase, differences in pattern similarity between REST1-MAZE and REST2-MAZE cannot be ascribed to a mere time-dependent, nonspecific drift of firing patterns. A bin size of 50 msec was chosen unless noted otherwise. Temporal correlations in firing patterns of ACB neurons were determined by computing a Pearson's correlation coefficient (PCC) using the following equation (Kudrimoti et al., 1999):

1

where r_ij is the correlation coefficient between neurons i and j, f_i,n is the spike count for cell i in bin n, μ_i is its mean spike count, and n is an index for bin number. When using bin sizes of 10-50 msec, this measure of firing correlations is especially sensitive to near-synchronous firing and much less so to temporal relationships with fairly long delays or advances. The use of this range of bin sizes does not take into account aspects of temporal coding manifested at a still finer resolution, which does not imply, however, that such aspects would be irrelevant for the study of reactivation. Compared with the “temporal bias” method (Skaggs and McNaughton, 1996), which can operate on bins of a few milliseconds, the current method uses a coarser time resolution. Especially for cells with low firing rates, however, this conveys the advantage that spike counts are summated across a longer period (Eq. 1), improving statistical robustness. Conversely, coarse binning (∼1 sec) (Louie and Wilson, 2001) has been used as an important, complementary method for studying reactivation.

Thus, a PCC value (also referred to as “correlation” here) was obtained for each cell pair and for each of the three phases of the experiment. All PCC values of a particular rest-active episode were assembled into a single matrix (labeled “R-matrix” by convention), and the similarity between the three R-matrices was determined by computing a correlation coefficient, this time defined at the level of two matrices (REST1 vs MAZE, REST1 vs REST2, and MAZE vs REST2) (McNaughton, 1998) as follows:

2

where r_MAZE,R1 is the correlation coefficient for the MAZE and REST1 pattern, m_i and n_i are the correlation coefficients for matrix element i during MAZE and REST1, respectively, μ and ν are the corresponding means, and K is the total number of matrix elements. Thus, the following similarity measures were obtained for the three consecutive episodes: r_MAZE,R1, r_R2,R1, and r_MAZE,R2. In principle, reactivation can now be assessed by testing whether the similarity between REST2 and MAZE is larger than for REST1 and MAZE; however, an important refinement in this approach, using partial correlation methods (Kleinbaum et al., 1998), was to compute the REST2-MAZE EV (Kudrimoti et al., 1999). This statistical measure gauges the extent to which the variance in the correlation pattern of REST2 can be explained by the pattern in MAZE while factoring out any preexisting influence from REST1. This correction is important because ensemble firing patterns in ACB are characterized by a fair amount of basal correlations that remain constant throughout the entire experiment (including REST1), which may overshadow the recruitment of novel correlations during learning and their reactivation during subsequent sleep or rest. The explained variance was computed as follows:

3

where R1 and R2 represent REST1 and REST2, respectively. EV equals the square of the partial correlation coefficient and is bounded between 0 and 1. To be able to assess the statistical significance of EV, we designed a novel control measure that could be applied to the same animal and the same experimental session as was used for extracting the EV value. Previously, the square of the correlation coefficient, r_MAZE,R1, was used as a control (Kudrimoti et al., 1999), but the disadvantage of this measure is that it does not correct for basal correlations. By definition, reactivation must follow the arrow of time, because pattern recurrence follows but never precedes a novel behavioral experience. Our design of a control measure was based on the idea that the same R matrices and the same matrix correlation coefficients, r_MAZE,R1, r_R2,R1, and r_MAZE,R2, can be used as for the EV, but in reverse temporal order. Thus, an REV was computed by switching around R1 and R2 in Equation 3 as follows:

4

REV measures the extent to which the variance in the correlation pattern of REST1 can be explained by the pattern in MAZE while factoring out any influence from REST2. Our basic assumption is that reactivation should result in a higher similarity of the correlation patterns in MAZE and REST2 than in MAZE and REST1, so that on average EV > REV. If EV and REV are not significantly different, one should accept the null hypothesis that any difference between R2-MAZE and R1-MAZE pattern similarity should be attributed to stochastic changes in correlations.

In summary, we measured to what extent cell pairs in the recorded ensemble fired “together” (covariation in firing rate) or not by computing a Pearson's correlation coefficient (Eq. 1). After all correlations had been computed and assembled in separate matrices for REST1, MAZE, and REST2, the similarities between these correlation matrices were determined by computing the three matrix correlation coefficients, r_MAZE,R1, r_R2,R1, and r_MAZE,R2 (Eq. 2). Subsequently, these three correlation coefficients were used to compute the explained variance (Eq. 3) and reverse explained variance (Eq. 4) as a measure of ensemble reactivation and a control measure, respectively. The explained variance expresses which fraction of the variance in the REST2 pattern can be accounted for by the MAZE pattern, correcting for “basal” similarities as already expressed during REST1. The value of the reverse explained variance as a control measure especially relates to the fact that it is computed in exactly the same way as the explained variance, but with REST1 and REST2 switched around.

Because EV and REV can be determined for the same session, we used a pairwise test to assess the statistical significance of differences (Wilcoxon's matched-pairs signed rank test; this test was used throughout this paper unless noted otherwise). In Results, correlations between all active, stable, and well isolated neurons are presented; when we removed correlations between cells recorded on the same tetrode, no significant deviation was noted. In addition to the EV-REV analyses and related tests, we undertook a temporal bias analysis to test the possibility of reactivation of spike sequences (Skaggs and McNaughton, 1996). Although the results of this analysis did not confirm a significant sequence effect, it should be noted that, unlike the previous experiment, the rat's behavior on the T-maze was characterized by overlapping trajectories traversed in opposite directions, and thus cells with behavioral correlates were not activated in a fixed, often repeated sequence (i.e., when the rat visited two locations A and B in the task, a cell with a behavioral correlate at A was not activated exclusively before a cell firing at location B, because the sequence B-A also occurred in the same session). This may explain the apparent lack of a sequence effect. All numerical values in the text represent mean ± SEM unless noted otherwise.

Histology

During the lowering of tetrodes into the brain, a “descent profile” characteristic of tetrodes heading toward the ventral striatum was documented. Briefly, this profile consisted of the penetration of neocortical layer V, with intense unit activity of high amplitude, of the corpus callosum, marked by an overall quiescence, followed by a number of encounters with dorsal and ventral striatal units as described by Callaway and Henriksen (1992), Lavoie and Mizumori (1994), and Shibata et al. (2001) (see also below). At the end of an experiment the final positions of the tetrodes in the VS were marked by passing a 10 sec, 25 μA current through one lead of each tetrode. The rats were transcardially perfused with 100 ml of 0.9% saline solution followed by 500 ml of 4% paraformaldehyde and 0.05% glutaraldehyde in 0.1 m phosphate buffer, pH 7.4. Brain sections were cut (40 μm) using a freezing microtome and Nissl stained to reconstruct tetrode tracks and their final positions. These end points could be linked to the descent profiles, leading to the exclusion of some extrastriatal tetrodes. The tetrode tip position in the ventral striatum at the time of recording was estimated on the basis of the distance traveled until termination of the experiment. A minority of end points (one to three per rat) was located in the striatal parts of the olfactory tubercle or islands of Calleja, which are considered part of the ventral striatum although not of the nucleus accumbens proper (Heimer et al., 1985). The relative distribution of recordings across different ventral striatal regions was as follows: 15% in rostral pole, 17% medial shell, 19% ventral shell, 44% core, and 5% ventromedial caudate-putamen. Placement of the hippocampal EEG electrodes in stratum pyramidale and stratum lacunosum moleculare was guided by electrophysiological hallmarks (Buzsáki, 1986; Gothard et al., 1996; Kudrimoti et al., 1999) and verified by applying a lesion current of 25 μA (10 sec) and subsequent Nissl staining.

Results

The data in this study are based on a total of 340 single-unit recordings from the ventral striatum, obtained from 21 recording sessions in three rats. The number of well isolated, stable, and sufficiently active single units recorded per session ranged from 14 to 19 (mean ± SEM: 16.3 ± 0.35). The precise nature of the behavioral correlates of VS firing patterns in the T-maze task falls outside the scope of this study but generally agrees with previous reports on rats and monkeys (Schultz et al., 1992; Lavoie and Mizumori, 1994; Shibata et al., 2001; Setlow et al., 2003). Briefly, behavioral correlates were analyzed using rate maps (plotting the firing rate of a single unit as a function of two-dimensional maze position) (Fig. 1), peristimulus time histograms, and time-position maps of firing rate (in which the firing rate was plotted as a function of time and position along one maze dimension, e.g., the north-south axis). Correlates of reward consumption and reward approach, or a combination of these two, were encountered frequently. Correlates of motor behavior, e.g., whole-body turns or head movements, were also found, albeit less frequently. Some units fired specifically during a particular phase of a maze trial, such as the waiting period in the T-stem, before a choice could be made. Only rarely did we find units with a firing rate response that could be attributed to the rat's position on the maze alone. These results will not be presented here in further detail.

Modulation of firing rate by rest-active state

In addition to the fine-grained behavioral correlates identified during T-maze performance, we observed a modulation of firing rate by the rest-active state of the animal going through the three main episodes of the experiment. Individual cells showed a large variability in their firing rate changes after rest-active transitions, with some cells increasing and other cells decreasing their firing rate during the maze running as compared with the prebehavioral or postbehavioral sleep-rest period. On average, however, the firing rate was significantly higher during maze running than during the rest phases (mean ± SEM during REST1: 2.18 ± 0.26 Hz; MAZE: 2.62 ± 0.28 Hz; REST2: 2.17 ± 0.27 Hz; p < 0.005 for REST1 vs MAZE; p < 0.00005 for REST2 vs MAZE; REST1 vs REST2: NS; n = 340). Postbehavioral firing rates of hippocampal cells, which were first activated by exposure to their place fields, can be enhanced with respect to prebehavioral firing rates (Pavlides and Winson 1989), but apparently only if the previous waking episode was restricted to a small space encompassing the place field (Wilson and McNaughton, 1994). This relatively subtle effect was not found in the mean firing rates of VS populations in the rats of the current study, whose behavioral experience (during MAZE) was substantially more diverse; however, the possibility of such an effect in subsets of VS cells showing particular behavioral correlates during maze running cannot be excluded.

Reactivation in ventral striatal ensembles of neurons

For each episode and each pair of neurons recorded throughout the three episodes, a linear correlation coefficient (PCC) was computed, representing the degree to which the neurons were coactivated, using a time resolution of 50 msec. After a correlation matrix containing all PCC values had been composed for each episode (Fig. 2), the EV and REV were calculated as a measure of reactivation and a control measure, respectively. The EV was larger than REV in 15 of 21 sessions, with means of 10.1 ± 2.3 and 2.8 ± 0.8%, respectively (p < 0.01; n = 21). The difference between EV and REV (7.3%) represents the portion of the post-experiential correlation patterns that is explained by correlations during T-maze behavior, factoring out linear influences from the pre-experiential correlation pattern and correcting for basal similarity patterns. Bins of 10, 20, 100, and 250 msec yielded (EV-REV) differences similar to the standard bin size of 50 msec.

Figure 2.

Example of reactivation in a session recording 19 units. For each main episode, cell numbers are plotted along the perimeter of a circle. If a correlation for any given cell pair was significant (p < 0.001), a straight line was plotted between those two cells (color scale: yellow-red is positive, blue is negative). Line thickness is proportional to the absolute correlation magnitude (dark red bar corresponds to a maximal correlation value of +0.16). During REST1, relatively strong correlations are found between (5,15), (6,8), (6,11), (6,17), (6,18), (7,14), (8,12), and (11,18). Some of these correlations, such as (6,11), (6,18), and (11,18), are present throughout all episodes and thus do not contribute to specific REST1-MAZE similarity. During MAZE, some correlations from REST1 disappear, such as (5,15), (6,17), and (7,14), and new correlations appear [e.g., (3,4), (8,11) and (8,14)]. During REST2, some correlations newly recruited during MAZE reappear [e.g., (3,4), (8,11), (8,14)]. Some correlations are present only during REST1 and REST2 [e.g., (8,12) and (10,17)] and do not specifically influence reactivation measures. Overall, the similarity between REST2 and MAZE patterns was larger than between REST1 and MAZE patterns.

Whether the EV measure of reactivation decays back toward zero in the course of postbehavioral rest-sleep, as has been described for the hippocampus (Kudrimoti et al., 1999), was tested next. Surprisingly, however, no marked decrease in EV was found when four subsequent 10 min blocks of REST2 were analyzed, whereas REV values for the two blocks of REST1 were both low (Fig. 3) (repeated measures ANOVA across six time blocks: p < 0.05; EV values for REST2 blocks were not significantly different from each other, whereas REV values for REST1 blocks were significantly lower than EV values for corresponding REST2 blocks) (Fig. 3).

Figure 3.

Lack of decay in reactivation of ventral striatal units across the first 40 min of REST2. When the EV was plotted in 10 min blocks as a function of the time elapsed since MAZE, no decay of EV was apparent across the first 40 min of REST2. The EV for the four blocks making up REST2 was calculated using the last 10 min of REST1 (“-10”) as control period. The two control values at -10 and -20 min refer to the first and second 10 min block before MAZE, taking the first 10 min block of REST2 as control period (REV) (Eq. 4). Statistical significance: ^*p < 0.05 for blocks +20 versus -20 min; ^**p < 0.01 for blocks +10 versus -10 min (n = 21 sessions).

Dependence of reactivation on sleep-wake state

Depending on the nature of the learning task, memory consolidation has been suggested to occur primarily during particular sleep phases. For instance, consolidation of spatial memory may especially require SWS but not REM sleep (Plihal and Born, 1999), whereas a procedural visual discrimination task seems to require both SWS and REM sleep (Gais et al., 2000; Stickgold et al., 2000). Therefore, the question was addressed whether ventral striatal reactivation depends on specific sleep-wake states occurring during the REST2 phase: quiet wakefulness, active wakefulness, and SWS. In 17 of 21 sessions, all three of these states were represented well. REM sleep periods were generally not sufficiently long to be taken into account in this analysis. Most REST2 phases contained 0.5-2 min of REM sleep, yielding spike samples much smaller in size than the other three types of state analyzed.

Significant reactivation was present only throughout epochs of SWS but not active or quiet wakefulness (Fig. 4A). The EV values of SWS and of the overall REST2 period were highly similar and significantly different from REV (p < 0.01 and p < 0.02, respectively), whereas those for the wakeful epochs were lower and nonsignificant, indicating that a high proportion of the overall reactivation during postexperiential rest can be attributed to SWS. Indeed, the EV for SWS exceeded the EV for both quiet and active wakefulness at p < 0.05. The total number of spikes sampled for each phase was quantified to investigate whether these results might arise from sampling differences. The mean firing rates, however, did not differ significantly among the three states (Fig. 4B). Moreover, the mean number of spikes per session during SWS was similar to that during quiet wakefulness (NS) and lower than for active wakefulness (p = 0.0503) (Fig. 4C). Although it can be concluded that reactivation occurs prominently during SWS, we note that the mean EV values for quiet and active wakefulness were considerably higher than the corresponding REV values (Fig. 4 A), leaving open the possibility that reactivation during those periods does occur; however, our statistics were not robust enough to reveal this. In summary, these data suggest that ventral striatal reactivation is prominent during SWS but do not exclude its occurrence during other off-line phases.

Figure 4.

Ventral striatal reactivation is particularly prominent during slow-wave sleep. A, EV and REV plotted as a function of sleep-wake categories in the REST2 period. “All” refers to the complete REST2 period, AW to active wakefulness, QW to quiet wakefulness, and SWS to slow-wave sleep. The difference between EV and REV in SWS was slightly more significant (^**p < 0.01) than for All (^*p < 0.02; n = 17 sessions). The EV for All was significantly higher than for AW (p < 0.05) but not for QW. For SWS the EV was significantly higher (p < 0.05) than for both AW and QW. B, No significant differences were found in the mean firing rates for each phase. C, Mean total spike counts for the three different phases of REST2.

Modulation of ventral striatal firing patterns in close temporal association with hippocampal ripples

Considering the important role the hippocampus plays in spatio-contextual memory formation, one may hypothesize that this structure acts as an initiator and generator of pattern reactivation and may drive its target structures into a state of off-line reprocessing. Instrumental in the empirical testing of this hypothesis is the finding that hippocampal replay is most prominent during ripples (Kudrimoti et al., 1999), massive high-frequency (∼200 Hz) bursts of pyramidal cell activity conjoined with dendritic sharp waves and intervening, local GABAergic inhibition (Buzsáki, 1986; Ylinen et al., 1995). The hypothesis predicts that hippocampal events associated with reactivation can be propagated to target structures and orchestrate their population activity to initiate pattern replay as well. This prediction was tested first by examining whether ventral striatal firing rate was modulated in close temporal association with hippocampal ripples. Across 21 sessions (340 unit recordings), a subset of the ensemble recorded during REST2 indeed showed significant firing rate increments (n = 64; 18.8%) (Fig. 5A), decrements (n = 18; 5.3%), or sequences of an increment and a decrement (n = 3; 0.9%) around and especially after ripple onset (one-way ANOVA: p < 0.05; these cells are henceforth referred to as being “ripple modulated”). In a majority of cells showing decrements, we noted that these decrements were already visible before ripple onset. When segregating the complete REST2 period into temporal segments of SWS versus non-SWS, ripple modulation of ventral striatal firing was found both during and outside episodes of SWS. Generally, firing rates were not sufficiently high to examine whether a specific temporal relationship existed between VS discharge patterns and a particular phase of ripple cycling. Likewise, it was not feasible to determine the exact peak or trough latencies of firing responses relative to ripple onset, because onset measurement depends on the moment that the waxing and waning oscillation constituting the ripple trespasses a threshold defined relative to the background noise of the EEG signal.

Figure 5.

Modulation of ventral striatal firing rates in close temporal association with hippocampal ripples. A, Example of six cells from one session showing a statistically significant ripple modulation (p < 0.05; one-way ANOVA). This session comprised a total of 19 single-unit recordings, and the examples were taken from REST2. Peri-ripple time histograms were constructed taking the ripple onset as the synchronizing time point (t = 0.0 sec) and plotting firing rate as a function of time relative to ripple onset (bin size: 50 msec). In some of the graphs, modulation of the average firing rate is observed in advance of ripple onset. B, Average peri-ripple time histogram for all 340 units during REST2. Time bins significantly different from the average baseline firing rate (computed across -1.0 to -0.5 sec) are indicated by asterisks (p < 0.001) or plus signs (p < 0.005). The modest trough in firing rate in advance of the peak was significantly different from baseline for three time points. The average population histogram for REST1 was similar to REST2, in both size and shape.

Across the entire REST2 period, the subsets comprised on average 4.0 ± 0.3 cells per session (range, 1-7; fractional size per session: 25.4 ± 2.1%, i.e., number of units showing ripple modulation divided by total number of units in that session). The size of these subsets during REST2 was modestly but significantly larger than during REST1 (3.2 ± 0.5 cells; fractional size: 19.9 ± 3.2%; p < 0.05). Correspondingly, the overall fraction of cells nonmodulated during REST1 but modulated during REST2 (15.1%) was larger than the fraction of cells modulated during REST1 but not during REST2 (9.8%; fraction of cells nonmodulated during REST1 and REST2: 65.2%; fraction of cells modulated during both phases: 9.9%). When viewing the time course of VS firing rate averaged across the entire population of units recorded in this study, we observed a marked peak shortly after ripple onset, which was significantly different from baseline (p < 0.001) (Fig. 5B). Moreover, a “trough” in average firing rate preceded ripple onset, significantly deviating from baseline (p < 0.001), although for less time points than the peak. Thus, hippocampal ripples are closely associated in time with significant transient changes of firing rate in a subpopulation of VS neurons.

Reactivation is associated with ripple modulation of firing rate

A second test of the prediction that hippocampal reactivation-associated events coordinate reactivation in the VS was to examine whether ripple-modulated neurons in the VS showed a higher degree of reactivation than nonmodulated neurons. It should be noted at the outset, however, that the EV-REV measures of reactivation are highly sensitive to the number of cells (n) contributing to the correlation matrix. This sensitivity relates directly to the fact that the number of correlations increases with the number of cells as n(n - 1)/2, and the probability of finding highly correlated cell pairs increases accordingly. In the absence of high correlations, EV and REV become strongly subject to influences of correlational “noise,” i.e., small stochastic changes in correlations. Despite this, a first indication in favor of the hypothesis was that the EV of ripple-modulated VS subsets (26.6 ± 7.1%) was significantly higher than that of nonmodulated ensembles (5.3 ± 1.8%; p < 0.01; we included 14 of 21 sessions showing ripple-modulated subsets of at least four neurons).

In a second type of analysis undertaken to address the same question, we first pooled all of the correlation (PCC) values across sessions and rats and then computed the difference (EV-REV) for each subset (i.e., one group of pairs of cells, both of which were ripple modulated vs a group in which none were ripple modulated; mixed ripple-modulated/nonmodulated pairs were not included). The difference (EV - REV) in the ripple-modulated subset (28.8%) was much higher than in the non-modulated subset (0.7%). To assess the mean and variance of the distributions of EV and REV, we applied a bootstrapping procedure, resampling the dataset constituted by all pooled PCC values (Hoffman and McNaughton, 2002). The statistics of these distributions revealed, first, that the difference (EV - REV) was significantly higher than zero in the ripple-modulated subset (p < 0.01) but not in the nonmodulated subset (p > 0.05), indicating a significant reactivation in ripple-modulated cells but not in nonmodulated ones. Second, the (EV - REV) distribution of the nonmodulated subset differed significantly from that of the ripple-modulated subset (Student's t test: p < 0.005), indicating a stronger reactivation in ripple-modulated versus nonmodulated cells.

Conceivably, these results could be biased by the fact that we determined whether cells were ripple modulated or not during the REST2 phase; perhaps unknown physiological or measuring constraints bring about an unforeseen relationship between the detection of ripple modulation and pattern similarities between REST2 and MAZE versus REST1 and MAZE. Therefore we also performed the pooling and bootstrapping procedures described above for subsets distinguished on the basis of ripple modulation during REST1. These subsets yielded results very similar to those of the REST2-based subsets, arguing against an artifact derived from a temporally biased selection procedure. Finally, we tested whether the incidence rate of ripples across REST1, MAZE, and REST2 was globally similar and found values of 0.31 ± 0.02, 0.29 ± 0.03, and 0.34 ± 0.03 Hz, respectively. Despite the significance of the modest difference between the REST1 and REST2 values (p < 0.05), it is deemed improbable that reactivation should be ascribed to such a minor difference in occurrence of ripples. Namely, a mere ∼10% increase of ripple incidence rate is unlikely to explain a highly specific partial recurrence of patterned information during REST2. The incidence of ripples during the MAZE period can be ascribed to bouts of awake immobility, drinking, eating, and grooming and is in line with previous observations (Buzsáki, 1986).

Discussion

This study showed that a statistical measure of ensemble reactivation (EV) in the VS was significantly elevated above a control measure (REV), indicating a larger correlation pattern similarity between the behavioral period and postbehavioral rest than behavior and prebehavioral rest. Reactivation did not notably decay during 40 min of the postbehavioral period and was especially prominent during SWS but not significant during episodes of wakefulness. Subsets of neurons were modulated in close temporal association with hippocampal ripples. Strikingly, ripple-modulated neurons showed an enhanced reactivation relative to nonmodulated cells.

A critical consideration of the enhancement of EV is in place. It may be argued that this finding could be caused by an effect of arousal in structures afferent to the VS, arising during active behavior and lingering during postbehavioral rest. Arousal, however, would be expressed by a global modulation of excitability in many cells, which contrasts with the highly specific recurrence of temporal correlations between cell pairs in the ensemble (Fig. 2) and with the reversibility of the overall firing rate enhancement during behavior relative to rest periods. That EV varied as a function of the postbehavioral sleep-wake phase (Fig. 4) but did not decay across ∼40 min (Fig. 3) also speaks against a mere lingering of acquired information. A lingering effect would not be expressed in a specific sleep state (i.e., SWS), but should extinguish gradually against a background of (re)processing of older or newer information. Rather, the postbehavioral period appears composed of successive reactivation-enriched (SWS) and reactivation-poor (awake) episodes. The present data, however, do not rule out that some reactivation may occur during wakeful episodes.

The selective reactivation in VS subsets that were modulated in association with hippocampal ripples suggests a temporal relationship between VS reactivation and a specific brain event that may be the ripple itself, a ripple-triggering event originating elsewhere, or a ripple-associated convergence of connected structures afferent to the VS into an overall attractor state (Pennartz et al., 1994). Before accepting this suggestion, however, one may object that the high EV in ripple-modulated cells may be caused by the excitation of many cells during REST2 around ripple onset (Fig. 5), possibly enhancing neural synchrony, PCC values, and similarity to MAZE patterns. A first argument against this objection is that in the absence of circuit modifications, any ripple-driven neural synchrony during REST2 should also be expected during REST1, thus affecting EV and REV in the same way. This point is corroborated by the similar occurrence of ripples during REST1 and REST2. It is thus difficult to explain why any similarity between ripple-driven synchrony during REST2 and synchrony during MAZE should be larger than the synchrony during REST1 and MAZE, unless a memory effect were involved. Second, EV remained significantly enhanced when ripple-modulated cells were identified from their response patterns during REST1 instead of REST2, arguing against a detection bias.

Although alternative interpretations cannot be ruled out entirely, we conclude that the enhanced EV is thus likely to reflect a partial recurrence of patterns acquired during preceding behavior and that this reactivation effect is particularly clear for ripple-modulated neurons.

Ripple modulation and off-line reactivation: a hypothesis

Although the present study does not prove that ripples directly modulate VS firing rates, the global range of short response latencies (Fig. 5) argues in favor of a close temporal and functional linkage. Furthermore, unit responses resembled firing patterns observed in VS after electrical stimulation of the fimbria-fornix or CA1-subiculum (Pennartz and Kitai, 1991; Boeijinga et al., 1993; Mulder et al., 1998). Recently, negative field potential shifts in rat hippocampus under anesthesia were shown to correlate to transitions to the “up” state in VS neurons (Goto and O'Donnell, 2001), also supporting a strong functional linkage. A similar relationship has been observed between hippocampal ripples and the transition of neocortical ensembles from lower to higher states of excitability (Battaglia et al., 2001). Possibly, the modest pre-ripple trough in firing rate represents a temporarily lowered state of excitability corresponding to a “down” state as seen in intracellular recordings. In concert with CA1, the subiculum and entorhinal cortex may well contribute to ripple modulation of VS firing rate (Chrobak and Buzsáki, 1996).

Combined with previous results, our findings suggest the following hypothesis of VS reactivation. Because of the putative auto-associative and ripple-generating properties of area CA3 (Buzsáki, 1986; Ylinen et al., 1995), we assume that spontaneous reactivation arises in CA3, perhaps as a result of decreased neocortical input and activity changes in pontine nuclei (Kirk, 1997; Battaglia et al., 2001). Ripples are then rapidly induced in area CA1, subiculum, and deep entorhinal cortex, instigating associative pattern regeneration in these areas (Treves and Rolls, 1994) and possibly exciting additional target structures, including basolateral amygdala and neocortex. Activation of glutamatergic afferents converging onto VS medium-sized spiny neurons may then switch a subset of these neurons into a different, predominantly excited state, extending well beyond the duration of the ripple (Fig. 5) (Goto and O'Donnell, 2001). Thus, ripple-induced cascades of cortical and amygdaloid activity may reinstate a distributed ensemble pattern in VS, capturing relevant aspects of previous behavioral experiences. By way of lateral GABAergic inhibition between striatal principal cells (Tunstall et al., 2002; Taverna et al., 2004), this pattern may be sharpened to accentuate activity differences between ensembles. Assuming that ripples, as massive ∼200 Hz network oscillations, can elicit long-term potentiation (Buzsáki et al., 1987; Staubli and Lynch, 1987), they may not only act as vehicles carrying reactivated information to extra-hippocampal structures, but might also induce long-lasting synaptic changes in glutamatergic inputs to the VS (Pennartz et al., 1993; Kombian and Malenka, 1994) or within afferent limbic structures. One of the many elements remaining to be investigated is whether, in rats, the prefrontal and cingulate cortex, midline thalamic nuclei, and basolateral amygdala coreactivate with the hippocampus and VS. Notably, VS reactivation did not decay visibly across the first 40 min of REST2, in contrast to hippocampus (Kudrimoti et al., 1999). This may suggest, first, a more prolonged replay in VS than can be obtained with a drive or “trigger” from area CA1 alone, which would imply the existence of an extra-hippocampal substrate to sustain this prolonged reactivation. Indeed, a causal role for the hippocampus in inducing ventral striatal reactivation remains to be demonstrated. Alternatively, prolonged hippocampal replay may take place after specific tasks such as those used in the present study, which differed substantially from those for which hippocampal reactivation was reported. Finally, the statistics of the present data do not exclude the possibility that some decay may have occurred.

Functional implications

Spontaneous reactivation has been observed previously in hippocampus and neocortex (Wilson and McNaughton, 1994; Qin et al., 1997; Kudrimoti et al., 1999; Hoffman and McNaughton, 2002). Although such reactivation is consistent with current canonical concepts of declarative memory consolidation (Squire and Zola-Morgan 1991; McNaughton, 1998), the present data imply a much more widespread process and suggest that consolidation of memories, whether declarative, procedural, or otherwise, may well involve subcortical routes, including basal ganglia structures. As a central component of the emotional motor system, the VS has functions in signaling and predicting reward (Schultz et al., 1992; Lavoie and Mizumori, 1994; Setlow et al., 2003), control of consummatory behaviors (Stratford and Kelley, 1999), and control of choice and action patterns on the basis of behaviorally relevant stimuli (Pennartz et al., 1994; Cardinal et al., 2002). When reactivated in VS, motivationally charged information streams may be directed toward target structures, including lateral hypothalamus, substantia nigra, ventral tegmental area, and ventral pallidum (Pennartz et al., 1994; Groenewegen et al., 1999). Thus, it is likely that VS reactivation affects information processing in downstream areas and cortico-basal ganglia-thalamic loops (Alexander et al., 1990; Groenewegen et al., 1999). It may be speculated that the ventral striatum contributes a “value” and reward-expectancy component to the overall process of procedural or episodic memory consolidation, which may help select memories with high storage priority.

The present data also have implications for our thinking of VS as a limbic-motor interface. Traditionally, investigators have taken this concept as meaning that the VS is a structure in which “motivation gets translated into action” (Mogenson et al., 1980). Increasing support has been gained for complementary cognitive functions of the VS, most notably in instrumental conditioning, reversal learning, and spatial memory (Stern and Passingham, 1995; Setlow and McGaugh, 1998; Hernandez et al., 2002; Sargolini et al., 2003). Three recent studies indicated that post-training VS injections of receptor antagonists or a protein synthesis inhibitor impaired retention in spatial memory and inhibitory avoidance tasks (Setlow and McGaugh, 1998; Sargolini et al., 2003) and instrumental learning (Hernandez et al., 2002). The present study confirms the relevance of post-training reprocessing and illuminates a potential neurophysiological substrate of this process, as expressed in ensemble activity. Taken together, these studies call for a broadening of the concept of the VS as a structure not only converting limbic input into skeletomotor, endocrine, or autonomous output, but also supporting nonmotor cognitive functions such as memory consolidation. The differentiated firing patterns during rest periods and the dependence of firing rate on distinct brain states during sleep and resting (Callaway and Henriksen, 1992) testify to the plausibility of this concept. Whether VS reactivation supports spatial-episodic, procedural, or other mnemonic functions remains to be examined.