Changes in reward contingency modulate the trial-to-trial variability of hippocampal place cells

Abstract

Pyramidal cells in the rodent hippocampus often exhibit clear spatial tuning. Theories of hippocampal function suggest that these “place cells” implement multiple, independent neural representations of position (maps), based on different reference frames or environmental features. Consistent with the “multiple maps” theory, previous studies have shown that manipulating spatial factors related to task performance modulates the within-session variability (overdispersion) of cells in the hippocampus. However, the influence of changes in reward contingency on overdispersion has not been examined. To test this, we first trained rats to collect food from three feeders positioned around a circular track (task₁). When subjects were proficient, the reward contingency was altered such that every other feeder delivered food (task₂). We recorded ensembles of hippocampal neurons as rats performed both tasks. Place cell overdispersion was high during task₁ but decreased significantly during task₂, and this increased reliability could not be accounted for by changes in running speed or familiarity with the task. Intuitively, decreased variability might be expected to improve neural representations of position. To test this, we used Bayesian decoding of hippocampal spike trains to estimate subjects' location. Neither the amount of probability decoded to subjects' position (local probability) nor the difference between estimated position and true location (decoding accuracy) differed between tasks. However, we found that hippocampal ensembles were significantly more self-consistent during task₂ performance. These results suggest that changes in task demands can affect the firing statistics of hippocampal neurons, leading to changes in the properties of decoded neural representations.

the ubiquity of spatial tuning in the rodent hippocampus is well established (O'Keefe and Dostrovsky 1971; O'Keefe and Nadel 1978; Redish 1999). In an impressive range of experimental settings, the best behavioral correlate of hippocampal pyramidal cell discharge is the animal's position within the environment. The spatial tuning of hippocampal “place cells” is robust enough to allow accurate estimation of position based solely on hippocampal ensemble activity (Brown et al. 1998; Wilson and McNaughton 1993; Zhang et al. 1998), and spatial firing patterns are manifest even as rats perform tasks that are not interrupted by hippocampal lesions (reviewed in Redish 1999).

In contrast to the marked spatial selectivity of place cells, their temporal variability can be great. To quantify the “intertrial” variability of place cells, Fenton and Muller (1998) examined the number of spikes place cells fired on consecutive passes through their place fields. Although only behaviorally identical passes through each field were compared, the number of spikes emitted by a place cell during a given pass fluctuated greatly, exceeding even the variance of inhomogeneous Poisson model predictions (Fenton and Muller 1998). Data characterized by such unexplained variability are described as “overdispersed,” and Fenton and Muller (1998) suggested that place cell overdispersion results from modulation of firing rate by internal, cognitive processes (Johnson et al. 2009; Olypher et al. 2002).

The overdispersed firing patterns (Fenton and Muller 1998; Fenton et al. 2010; Jackson and Redish 2007) and several other properties of place cell activity challenge the notion that the hippocampal representation of space is a unified, Cartesian map of the environment (reviewed in Eichenbaum et al. 1999). Instead, it has been suggested that place cells participate in multiple, distinct submaps constructed to subserve the behavioral task the animal must solve (McNaughton et al. 1996; Redish and Touretzky 1997; Samsonovich and McNaughton 1997; Touretzky and Redish 1996). In the “multiple maps” framework, the within-session variability of place cells reflects switching between submaps on a fast timescale (∼1 s or less; Olypher et al. 2002; Jackson and Redish 2007). The multiple maps hypothesis predicts that the nature of a behavioral task should influence the overdispersion of hippocampal place cells. When no submap or reference frame is particularly well-suited to performing a given task, overdispersion should remain high as the active hippocampal map cycles between equally useful submaps. In contrast, a task that forces subjects to rely on one particular reference frame or set of cues should increase the stability of the appropriate submap, and hippocampal neurons participating in that submap should exhibit decreased intertrial variability (reviewed in Redish 1999). In support of the map-switching explanation of overdispersion, requiring rats to navigate toward a goal location reduces overdispersion relative to undirected foraging for food scattered randomly in the same environment (Fenton et al. 2010; Jackson and Redish 2007; Olypher et al. 2002). Furthermore, tasks that encourage preferential use of a subset of multiple, concurrently available cues also decrease the overdispersion of place cells (Fenton et al. 2010).

Demonstrations of behavioral control of overdispersion have been achieved thus far through manipulation of spatial factors. Imposing a goal requirement on the cylinder foraging task (Jackson and Redish 2007; Olypher et al. 2002) introduces a spatial contingency to what was previously a randomly rewarded task. Likewise, requiring subjects to use one particular cue set encourages the use of one particular spatial reference frame (Fenton et al. 2010; Kelemen and Fenton 2010).

Here we examined how changing the reward contingency of a behavioral task affects neural representation by hippocampal cells, paying particular attention to the within-session variability of place cell spiking and the quality of reconstructed ensemble representations. Subjects were first trained to collect food from each of three feeder sites around a circular track. After rats achieved proficiency, the order in which feeder sites dispensed pellets was modified, to change the reward contingency subjects experienced. Because overt behavior remains largely similar after the switch, this task allows us to assess how a covert, cognitive change induced by the new task contingency impacts hippocampal neural activity.

EXPERIMENTAL PROCEDURES

Subjects.

Male Fisher-Brown Norway hybrid rats (n = 5; Harlan, Indianapolis, IN) were maintained on a 12:12-h light-dark cycle, with behavioral sessions occurring at the same time daily, ±1 h. Rats were handled for at least 7 days before beginning behavioral training, during which time they were acclimated to the 45-mg food pellets earned during task performance (Test Diet; Richmond, IN). Subjects were food deprived to no less than 80% of their free-feeding weight, and water was always freely available in the home cage. All experimental and animal care policies complied with National Institutes of Health guidelines for animal care and were approved by the Institutional Animal Care and Use Committee at the University of Minnesota.

Training and surgery.

Subjects ran clockwise laps around an elevated circular track with a diameter of 80 cm (Fig. 1). Running direction was enforced by physically blocking attempts to run backwards, although in practice this rarely occurred. Three equally spaced feeder sites were located on platforms extending from the main circular track. At the end of each platform, automated food dispensers (Med Associates; St. Albans, VT) were positioned to deliver pellets into small cups. Extramaze cues were present in the experimental room and remained stable throughout all experiments. Behavioral sessions occurred daily and lasted 30 min. During presurgical training, rats ran laps around the track for food reward, receiving two food pellets as they arrived at each feeder site. Once rats ran >35 laps for at least 3 consecutive sessions, they were implanted with a 14-tetrode microdrive (Kopf, Tujunga, CA) directed toward the CA1 region of the right hippocampus (−3.8 mm anteroposterior, 3.0 mm lateral from bregma). Surgical procedures have been described in detail previously, as have perfusion and histological methods (Johnson and Redish 2007; Schmitzer-Torbert and Redish 2004). After experiments were complete, current was passed through each tetrode (10 μA for ∼5 s) to mark its final location, and recording locations were subsequently verified to lie within CA1 hippocampus.

Fig. 1.

Rats ran laps around an elevated circular track (diameter = 80 cm). Salient extramaze cues were available around the maze and remained stable throughout all experimental sessions. In task₁, rats obtained food from each feeder site on every lap. In contrast, task₂ manipulates reward contingency by rewarding visits to every other feeder along the path. The active feeder site consequently precesses around the track.

Behavioral task.

To examine how changes in a behavioral task's reward structure influence hippocampal neural activity, we designed two task variants that preserved similar behavior but allowed us to manipulate reward contingency. Task₁ was identical to training sessions; each feeder site was baited on every visit. Within task₂, however, every other feeder site was baited. For example, after food is received at feeder 1, feeder 2 is inactive, so the subject should proceed directly to feeder 3 to receive food. From feeder 3, feeder 1 is unrewarded, so the subject should run to feeder 2 to collect food (Fig. 1, bottom). Although subjects were not explicitly punished for pausing at unrewarded feeders, feeder sites were extended off the main track to increase the cost of investigation, encouraging subjects to skip unrewarded locations after the switch to task₂. Rats ran task₁ sessions after implantation, as tetrodes were advanced toward CA1 hippocampus. Once tetrodes were in position, neural ensembles were recorded as subjects performed task₁. Subjects continued to perform task₁ for at least five recording sessions and then switched to task₂. The switch was entirely uncued, forcing subjects to learn the new reward contingency by trial and error.

Neural recordings.

Action potentials were recorded by a 64-channel Neuralynx (Bozeman, MT) Cheetah system. The voltage of each tetrode channel was monitored at 32 kHz and filtered between 600 Hz and 6 kHz. When the voltage on any channel of a tetrode exceeded a user-defined threshold, a 1-ms voltage sample from all channels was saved to disk and time-stamped with microsecond resolution. Spikes were sorted off-line with MClust 3.5 (Redish et al., http://redishlab.neuroscience.umn.edu/MClust/MClust.html).

Behavioral analysis.

All behavioral and neural analyses were carried out with Matlab (MathWorks, Natick, MA). Position tracking data were linearized by conversion to polar coordinates, and we detected whether or not subjects investigated a feeder site by measuring their excursion from the center of the track; when rats fully explored a site, they proceeded farther from the center than when they passed by without investigating. The maximum distance from center for each pass by a feeder site was z-scored, and a boundary of z = −2 reliably distinguished feeder investigation from feeder skipping. To assess learning during task₂ sessions, each pass by a feeder was scored. Bypassing an unrewarded site or collecting food from a rewarded location was counted as a correct pass, while investigating an unrewarded location or failing to collect food at a rewarded site was scored as an error.

Place fields.

Units that fired <100 spikes during task performance (<0.05-Hz session mean firing rate) were excluded from place field analyses, as were units with a session mean firing rate >5 Hz (putative interneurons). Spikes that occurred when the animal's speed was <5 cm/s were excluded from place field analysis. To find the one-dimensional place field(s) of a unit, the linearized track was divided into ∼5-cm bins, and the unit's firing rate within each bin was computed over the session. Contiguous bins in which the firing rate was elevated >15% of the cell's session maximum firing rate were considered place fields. Fields separated by <3 bins were merged (Gupta et al. 2010).

Firing rate.

Firing rates of neurons recorded during task₁ and task₂ performance were compared in two ways. First, the mean task firing rate was computed by averaging the firing rates of all neurons with place fields separately for task₁ and task₂ sessions. To test whether average firing rates differed at specific locations in space across task₁ and task₂, the track was divided into 64 bins, each ∼5 cm in size. The mean firing rate of each bin was calculated over all cells. For each bin, the mean task₁ firing rate was plotted against that bin's mean task₂ firing rate, and the points were fit with a linear model using Matlab's “regress” function.

Calculating overdispersion.

The overdispersion of each place field was calculated after Fenton et al. (2010). Briefly, the expected number of spikes for each pass through a place field was determined by multiplying the average firing rate at each position by the amount of time spent in that position. For each pass, we calculated z:

$$z={\text{Spikes}}_{\text{observed}}-{\text{Spikes}}_{\text{expected}}/\sqrt{{\text{Spikes}}_{\text{expected}}}$$

The overdispersion of each field is the variance of this distribution of z-scores. Because z is calculated in standard deviation units, distributions can be combined for multiple cells, to test ensemble overdispersion. Because the tasks differed in reward contingency, we expected rats' behavior to vary somewhat, especially near feeder sites. Thus, to ensure that neural analyses were restricted to epochs with consistent behavior, place fields with centers located within 10 cm of a feeder site were excluded.

Bayesian decoding and decoding quality.

One-step Bayesian decoding (Zhang et al. 1998) was applied to the entire ensemble of cells recoded during each session, using a ∼5-cm spatial bin size, a time step of 200 ms, and a uniform spatial prior. This method returns a probability distribution of the rat's location over space for each decoded time step. The general pattern of decoding results was the same at different bin sizes, and the statistical significance of our analyses was not changed (data not shown).

To quantify local probability, we assigned the subject's mean position during each decoding time step to a spatial bin and then averaged the collection of decoded position probability distributions within each bin. For each session, mean decoded distributions were plotted against actual position, and the average value of pixels along the diagonal of this plot (i.e., the amount of decoded probability matching the subject's actual location) was computed. Because this value depends on the size of the ensemble used for decoding (Schmitzer-Torbert and Redish 2008; van der Meer et al. 2010; Zhang et al. 1998), the mean local probability of each session was plotted against the number of cells recorded that day. Task₁ and task₂ sessions were plotted separately, and each collection of points was fit with a linear model.

To measure decoding accuracy, the subject's decoded position was taken to be the peak of the probability distribution for each decoded time step. Decoding error was then computed as the absolute value of the difference between the subject's actual and decoded positions for each time step. Like local probability, decoding accuracy varies as a function of the number of cells available for decoding (Schmitzer-Torbert and Redish 2008; van der Meer et al. 2010; Zhang et al. 1998), so the mean decoding error for each session was plotted as a function of ensemble size and linear regression was performed separately for each task.

i

Ensemble self-consistency was computed following the development of Jackson and Redish (2003). The goal of this measure is to quantify how much neural activity recorded during task performance differs from neural activity predicted given the decoded position of the animal and the tuning curves of cells in the ensemble. If representations are self-consistent, the difference between actual and predicted activity should be small (Jackson and Redish 2003; Johnson et al. 2008). Briefly, for each time step, actual activity packets were constructed by summing ensemble tuning curves weighted by the firing rates of cells during that time step. Expected activity packets were constructed similarly but were weighted by the expected firing rates of cells, obtained by evaluating the tuning curves at the position decoded during that time step. The difference between actual and expected activity packets for each time step was measured as the root mean squared error (RMSE) (Jackson and Redish 2003).

RESULTS

Behavior.

Across both tasks, rats ran increasingly more laps per session (Fig. 2, top). Accordingly, subjects' mean speed increased across task₁ sessions, eventually stabilizing during task₂ performance (Fig. 2, middle). In task₂ sessions, each pass by a feeder site was scored as correct or incorrect. Retrieving food from a rewarded feeder and avoiding an unrewarded site were counted as correct responses, while skipping a rewarded site and checking an unrewarded feeder were marked as errors. Subjects learned to skip unrewarded feeders in task₂ sessions, asymptoting to ∼90% correct responses (Fig. 2, bottom), demonstrating sensitivity to the new reward contingency.

Fig. 2.

The task performance curve (top) shows that rats ran increasingly more laps across the experimental sequence. Subjects' mean running speed showed a similar increase across sessions (middle). To construct the task₂ learning curve (bottom), skipping an inactive site and collecting food from a baited location were counted as correct passes. Across task₂ sessions, rats learned to skip unrewarded feeders.

Firing rates.

As subjects performed the tasks, ensembles of neurons were recorded from the CA1 region of hippocampus. Consistent with previous work, many hippocampal units displayed spatial tuning (O'Keefe and Dostrovsky 1971; O'Keefe and Nadel 1978; Redish 1999). Because neural variability can vary with firing rate (Barlow and Levick 1969; Rodieck 1967), we first looked for gross changes in the firing rates of neurons between tasks. The mean firing rate of cells recorded during the tasks (0.90 Hz and 0.91 Hz, task₁ and task₂, respectively) did not differ (P = 0.38, rank-sum test). To assess whether the firing rates of cells differed systematically at different positions on the track, we divided the track into 64 spatial bins and computed the average firing rate for each bin across all rats, sessions, and cells for each task. We then plotted the bin's mean rate during task₁ against its task₂ mean rate (Fig. 3). Uniform spatial firing across tasks would result in points distributed along a diagonal line of unity slope and y-intercept of 0. We fit points with a linear model and found that the slope and y-intercept of the regression line were not statistically distinguishable from 1 and 0, respectively (slope CI_95%: 0.79 to 1.08; y-intercept CI_95%: −0.01 to 0.14), indicating that the distribution of spiking across space did not vary significantly as a function of task.

Fig. 3.

The distribution of spiking did not vary across tasks. The mean firing rate within each of 64 spatial bins (across all cells) in task₁ sessions is plotted against the task₂ average firing rate for the same spatial bins. A perfect correspondence of firing rates across tasks would result in points distributed along the diagonal (solid line). The linear regression line fit to firing rate data (dashed line) did not statistically deviate from the diagonal.

Overdispersion.

To compare the intertrial variability of place cells between tasks, we computed the overdispersion of place cells (Fenton and Muller 1998; Fenton et al. 2010). Consistent with previous work, we found that place cells exhibited highly variable firing patterns during task₁ sessions (σ² = 5.36, n = 1,577 passes, Fig. 4). In contrast, cells recorded during task₂ sessions (following the switch in reward contingency) displayed significantly decreased within-session variability (σ² = 2.99, n = 1,810 passes, F-test, P < 10⁻⁶, F = 1.79). The decrease in overdispersion was observed for all subjects (Fig. 4, right). Place cell overdispersion was relatively stable both before and after the task switch, exhibiting a sharp, steplike transition following the change in reward contingency (Fig. 5).

Fig. 4.

Place cell within-session variability decreased during task₂ performance. Overdispersion was high during task₁ but was significantly reduced in task₂ sessions (left; F-test, P < 10⁻⁶). This effect was observed in all 5 subjects (right).

Fig. 5.

Overdispersion was stable during task₁ and task₂ sessions. The mean within-session variability (collapsed across subjects, error bars = SD) is plotted for task₁ and task₂ performance. Place cell variability was stable during task₁ and task₂ performance but dropped sharply after the switch between tasks (vertical dashed line).

Other factors besides the switch in reward contingency could account for the decreased trial-to-trial variability of place cells recorded during task₂. For instance, running speed (Fig. 2, middle) is known to influence place cell discharge (McNaughton et al. 1983; Wiener et al. 1989). To test the relationship of running speed and overdispersion, we regressed place cell overdispersion onto subjects' running speed for each session (Fig. 6, top). We found a significant correlation between mean running speed and place cell overdispersion (P < 0.001; r² = 0.1583; Fig. 6, top). However, when we performed separate regressions for task₁ and task₂ sessions (Fig. 6, top), no significant correlations between place cell variability and speed were observed (P_task1 = 0.12; P_task2 = 0.73). We also examined place cell overdispersion in sessions in which running speed was similar but the task differed. Regression analysis was restricted to sessions in which mean running speed was between 60 and 70 cm/s, the region of greatest overlap of running speeds between task₁ and task₂ sessions (Fig. 6, top). For sessions matched by rat running speed, the correlation between speed and overdispersion was not significant (P = 0.40; Fig. 6, middle). Finally, we examined the rate at which running speed and overdispersion changed across sessions. We plotted the difference in running speed and overdispersion between successive sessions over the course of the experiment (Fig. 7). Running speed increased by an approximately constant amount throughout most of the experiment, eventually reaching an asymptote during the final few task₂ sessions (Fig. 7, top). Overdispersion, on the other hand, changed little within task₁ sessions, decreased sharply during the first several task₂ sessions, and then stabilized for the remainder of the experiment (Fig. 7, bottom).

Fig. 6.

Running speed and task familiarity do not account for decreased within-session place cell variability. We plotted mean running speed (top) and session number (bottom) against the measured place cell overdispersion for all sessions and fit the distribution with a regression line (dashed line). Both speed and session number were significantly correlated with overdispersion (r² = 0.1583, 0.2498, respectively; P < 0.001 for both speed and session number). However, in both cases, correlations were not significant when regression was performed separately for task₁ (solid line) and task₂ (dotted line) sessions. When the analysis was restricted to behavioral sessions with similar running speeds (middle), the correlation between speed and overdispersion was not significant (P = 0.40).

Fig. 7.

To examine the dynamics of running speed and overdispersion across sessions, we plotted the difference in each variable across consecutive sessions (i.e., current session value − previous session value). Individual subject data are plotted in gray and the mean in black. The switch between tasks (between sessions 5 and 6) is marked with a vertical line. Running speed (top) increases throughout the experiment by an approximately constant amount, reaching an asymptote toward the end of task₂ performance (Δrunning speed approaches 0). In contrast, overdispersion is stable during task₁ performance (Δoverdispersion is 0) but drops sharply after the switch to task₂ before stabilizing again.

Increasing familiarity with the experimental conditions across days or increasing proficiency at performing the task could also influence the firing statistics of hippocampal neurons. Such an effect would manifest as a significant correlation between overdispersion and session number, so to test this idea we plotted overdispersion against session number for all subjects and performed regression analysis (Fig. 6, bottom). Session number and overdispersion were significantly correlated (P < 0.001, r² = 0.2498), but this correlation was not significant when the regression was performed within sessions of each task (P_task1 = 0.78; P_task2 = 0.17).

Differences in the reward contingencies of the two tasks give rise to differences in trajectories that could affect place cell intertrial variability. In task₁ sessions, because subjects were rewarded at every feeder site, the majority of passes through a given place field were initiated and terminated at consistent track locations (Fig. 8, top). In task₂, however, because subjects learned to skip unrewarded feeders, passes through a particular place field were more likely to begin and end at multiple, unique positions on the track (Fig. 8, top). To control for this effect, we categorized passes through each place field during task₂ sessions as proximal (initiated from the nearest feeder preceding the field) or distal (initiated from the feeder preceding the “proximal” site; Fig. 8, top). We calculated overdispersion separately for task₂ proximal and distal passes (Fig. 8, bottom). Compared with task₁ passes (all of which are “proximal” because subjects did not skip feeder sites), the overdispersion of both proximal and distal task₂ passes was significantly lower (task₂ proximal: σ² = 2.81, n = 351 passes, F-test, P < 10⁻⁶, F = 1.91; task₂ distal: σ² = 2.97, n = 393 passes, F-test, P < 10⁻⁶, F = 1.80). In contrast, the overdispersion of proximal and distal task₂ passes was not statistically different (F-test, P = 0.60; F = 0.95).

Fig. 8.

In task₂ sessions, because subjects skipped unrewarded feeders, they encountered place fields at different points in their trajectory between feeder sites (top). Proximal passes began from the feeder site nearest the place field (represented by gray shading) in question, while distal passes originated from 1 feeder farther back in the sequence. In contrast, all task₁ sessions were proximal passes because subjects did not skip rewarded feeders. We split passes through place fields during task₂ performance into distal or proximal based on where the subject began his trajectory and computed the overdispersion of these passes separately (bottom). Overdispersion was not significantly different between distal and proximal task₂ passes (P = 0.60, F-test), but both types of task₂ passes were less overdispersed than task₁ passes (P < 10⁻⁶ for both proximal and distal passes, F-test).

Ensemble decoding.

To examine whether the change in reward contingency between tasks affected the content or quality of hippocampal neural representations, we used one-step Bayesian decoding (Zhang et al. 1998) to estimate subjects' positions during task performance and assessed the quality of decoded position representations with three measures: local probability, decoding accuracy, and ensemble self-consistency (Jackson and Redish 2003; Johnson et al. 2008). Together, these three measures provide a detailed characterization of hippocampal ensemble representations.

Local probability.

We first asked how much probability decoded to the rat's actual location during each session. In Fig. 9, the mean decoded position probability distribution (averaged over all rats, separated by task) is plotted for each location on the track. For both tasks, probability was concentrated along the diagonal, indicating a strong correspondence between decoded position and actual location. Ensemble size affects decoding accuracy (van der Meer et al. 2010; Zhang et al. 1998), so to quantitatively compare local probability in task₁ and task₂ sessions, we averaged the probability along the diagonal of the position-averaged decoded probability plot for each session. We plotted this session mean local probability against ensemble size (Fig. 10). This relationship shows how the quality of distributed representations is affected by ensemble size, providing a measure of decoding efficiency. As expected, local probability and ensemble size were positively correlated for both task₁ and task₂ (Fig. 10). We performed linear regressions on these distributions separately for task₁ and task₂ sessions. The 95% confidence intervals on the slopes of regression lines fit to these plots overlapped completely (task₁ slope CI_95%: 0.00093–0.002; task₂ slope CI_95%: 0.00059–0.0019; task₁ y-intercept CI_95%: 0.004–0.010; task₂ y-intercept CI_95%: 0.008–0.029; see shading in Fig. 10).

Fig. 9.

Average decoded probability distributions for position are plotted for each position along the track, with greater probability density indicated by “hotter” colors. Probability is concentrated along the diagonal for both tasks.

Fig. 10.

For each session, the mean local probability is plotted as a function of ensemble size. Local probability was higher for sessions with a greater number of simultaneously recorded cells. Regression lines were fit to task₁ (solid line) and task₂ (dashed line) distributions. The 95% confidence intervals on the regression lines (red shading = task₁, blue shading = task₂) overlapped almost completely.

Decoding accuracy.

To estimate how well each ensemble predicted location, we took the peak of the decoded probability distribution for each time step to be the animal's decoded position, and then quantified error as the absolute value of the distance between estimated and actual position. Like local probability, this measure depends on ensemble size. Accordingly, Fig. 11 plots the mean decoding error as a function of the number of cells recorded during that session. As with local probability, we fit points from task₁ and task₂ sessions with linear models to look for differences in decoding error as a function of task. We found that the regression lines for each session were not statistically distinguishable (task₁ slope CI_95%: −1.242 to −0.3959; task₂ slope CI_95%: −1.112 to −0.3975; task₁ y-intercept CI_95%: 35.93–44.17; task₂ y-intercept CI_95%: 32.29–44.17; see shading in Fig. 11).

Fig. 11.

The mean decoding error for each session is plotted against the number of simultaneously recorded cells. The 95% confidence regions of regression lines fit to task₁ (solid line) and task₂ (dashed line) sessions overlapped extensively (red shading = task₁; blue shading = task₂).

Self-consistency.

Finally, we compared the self-consistency of neural ensembles recorded during the tasks. Bayesian decoding returns a position estimate for each decoded time step but makes no claims about the underlying probability distribution from which the estimate is derived. Self-consistency quantifies how well cells in the ensemble “agree” with the decoded estimate, based on their tuning curves. Self-consistency does not depend on how well decoded and actual position match; instead, self-consistency assesses whether the network state at a given moment is consistent with the corresponding decoded representation. Unlike the previously described measures of representation quality, ensemble self-consistency was not correlated with the number of simultaneously recorded cells (P = 0.52, r² = 0.008). Self-consistency was, however, significantly correlated with hippocampal overdispersion (P < 0.01, r² = 0.355). Comparing self-consistency across tasks (Fig. 12), we found that hippocampal ensembles were significantly more self-consistent during task₂ performance (task₁ session mean RMSE: 5.77, task₂ session mean RMSE: 2.79; P = 0.038, rank-sum test).

Fig. 12.

Session mean root mean squared error (RMSE, a measure of ensemble self-consistency) was significantly lower for task₂ sessions, indicating that hippocampal ensembles were significantly more self-consistent as subjects performed task₂ (P = 0.038, rank-sum test).

DISCUSSION

Here we have shown that after a change in the reward contingency of a behavioral task the overdispersion of hippocampal place cells decreased significantly. Our findings extend previous work investigating behavioral modulation of place cell spiking variability (Fenton and Muller 1998; Fenton et al. 2010; Jackson and Redish 2007; Lansky et al. 2001; Olypher et al. 2002), demonstrating that, like changes in spatial factors associated with a task, changes in the task's reward structure can also influence the reliability of place cell firing patterns. These results are consistent with several possible interpretations, discussed below.

Multiple maps and overdispersion.

Our findings mesh well with the multiple map theory of hippocampal function (McNaughton et al. 1996; Redish and Touretzky 1997; Samsonovich and McNaughton 1997; Touretzky and Redish 1996). In this framework, overdispersion is thought to reflect the hippocampus cycling between representational submaps. Because whether a cell fires depends upon both the organism's behavior and the currently active map, switching between submaps introduces variability to hippocampal cell spiking.

Analyses of large ensembles of simultaneously recorded neurons have demonstrated that map switches in the hippocampus are networkwide events that occur on the order of a second or less (Jackson and Redish 2007). Behavioral tasks that render one hippocampal submap selectively advantageous to the organism are thought to result in increased stability of that submap within the hippocampus. Consequently, place cells participating in the stabilized submap would exhibit less variability on consecutive passes through their place fields. In this way, the multiple map theory can accommodate behavioral effects on hippocampal overdispersion. Previous work has shown that navigation toward a goal decreases levels of overdispersion compared with random foraging tasks (Jackson and Redish 2007; Olypher et al. 2002). Similarly, other experiments have shown that forcing rats to attend to one subset of the array of concurrently available cues decreases hippocampal overdispersion (Fenton et al. 2010). Such findings are consistent with reduced overdispersion resulting from increased stability of a behaviorally relevant spatial submap.

Our finding that a change in reward contingency reduces overdispersion can be accommodated in the multiple maps framework. The switch between tasks requires subjects to shift their strategy from approaching every feeder site to skipping unrewarded locations. Previous work has shown that strategy shifts can affect hippocampal representations (Johnson et al. 2009; Kentros et al. 2004; Markus et al. 1995). In task₁, location in the environment was entirely predictive of reward delivery, and subjects might have solved the task by using a spatial strategy. During task₂, however, subjects had to run past a particular sequence of feeders for food to be delivered. In this case, proximity to a feeder site was not in itself enough to predict reward delivery. Because decreased overdispersion during task₂ performance suggests increased hippocampal submap stability, it is possible that a hippocampal submap was formed in association with the newly acquired strategy to facilitate task₂ performance. A hippocampal submap devoted to the sequential feeder-sampling strategy subjects adopted would be selectively stabilized during task₂ sessions, resulting in the decreased overdispersion we measured.

Task familiarity and proficiency.

It is possible that other differences between the tasks caused the change in overdispersion we observed. For instance, because subjects performed task₁ sessions first, slow changes operating over the course of both tasks might have caused decreased overdispersion during task₂ sessions simply because they occurred later. However, the stability of place cell variability before and after the switch in tasks argues against this idea (Fig. 5). Overdispersion decreases in a stepwise manner after the task switch, which suggests that some attribute of the new task is responsible for the differences in firing statistics of the hippocampal ensembles we recorded. Furthermore, although session number was correlated with overdispersion (Fig. 6), this correlation did not hold when computed separately within task₁ and task₂ sessions, again suggesting that overdispersion differs between tasks but is stable within a task. Together, these observations argue that general familiarity with the task and increasing task proficiency do not fully account for changes in overdispersion.

Running speed.

Subjects ran more laps across sessions throughout the experiment, and consistent with this, their mean speed increased across sessions (Fig. 2). Since running speed can affect the firing rate of place cells, it is possible that this difference explains the change in overdispersion we observed. To minimize this issue, we excluded place fields located at or near feeder sites, as we expected subjects to travel through these areas at different speeds during task₁ and task₂. To further address this issue, we asked whether mean running speed was systematically related to hippocampal overdispersion (Fig. 6). We found a correlation between running speed and overdispersion, but again this correlation disappeared when the regression was evaluated separately for the two tasks and when sessions were matched by running speed. Examining the dynamics of running speed and overdispersion across sessions (Fig. 7) revealed that running speed changed steadily across the experiment, but overdispersion was stable before and after the task switch but dropped sharply at the switch between tasks. Changes in speed alone thus cannot account for the increased place cell reliability during task₂.

Trajectories.

The simple spatial reward contingency of task₁ requires only that subjects run directly from feeder to feeder, collecting food at each site (Fig. 1). The modified reward contingency of task₂, however, encourages multiple trajectories across the track. Consequently, during task₂ sessions, subjects passed through a place field early in their trajectory in some cases and in other instances passed through the same field later in the trajectory (Fig. 8). To address this difference, we separated passes through each place field in task₂ sessions based on how near to the place field the subject was when beginning the trajectory. We found that distal and proximal passes during task₂ sessions showed similar levels of overdispersion, suggesting that differences in the length of motion sequences within task₂ are not related to overdispersion in a systematic way. However, both categories of task₂ passes were significantly less variable than task₁ “proximal” passes (Fig. 8).

Other explanations for changes in overdispersion.

Our results show that some difference between the two behavioral tasks caused a persistent decrease in the within-session variability of place cells. We have argued that this decrease reflects a strategy shift between tasks with spatial and sequential reward contingencies, but other interpretations are possible. For instance, during task₂ sessions, subjects might have used dead reckoning (Redish 1999) to keep track of the distance between reward deliveries. This way, instead of determining where to go by learning the new reward contingency, subjects instead might have simply traveled the appropriate “interreward” distance to arrive at the next active site. It has been suggested that an attentional process governs the rate of hippocampal map-switching (Fenton et al. 2010). Greater attention to self-motion information during task₂ sessions might have decreased map-switching, which could account for the decrease in overdispersion. Similarly, the hippocampus has been implicated in cognitive control-related processes (Kelemen and Fenton 2010), such as coordinating between multiple sensory input channels. If task₂ sessions increased the relevance of self-motion information, the extent to which this input stream influenced place cell firing might have increased in parallel. Both of these explanations are consistent with the findings presented here.

Temporal variability and decoding quality.

Interestingly, during task₂ performance we detected a large decrease in place cell temporal variability but did not observe changes in two measures of the quality of decoded neural representations. Although this finding seems counterintuitive, it underscores that the Bayesian decoding method we utilized here is unlikely to be an algorithm implemented by neurons in the brain. For instance, Bayesian decoding does not utilize information that may be present in networkwide correlation phenomena such as overdispersion (Zhang et al. 1998), but neural information processing algorithms might be able to do so (Averbeck et al. 2006). Because we rarely have a complete understanding of how the brain extracts and utilizes the information contained in the firing patterns of neurons, our results highlight the utility of subtle measures of neural activity (such as intertrial variability or self-consistency) that can be highly sensitive to changes in neural dynamics not necessarily detected by other analyses (decoding error, local probability).

Comparison to dorsal striatum.

The findings we report here contribute to a growing body of work highlighting the fundamentally different computations carried out by the hippocampus and dorsal striatum (Barnes et al. 2005; Berke et al. 2009; Schmitzer-Torbert and Redish 2008). Schmitzer-Torbert and Redish (2008) showed that representations in rat dorsal striatum were influenced by changes in the spatial contingency of reward. In that study, when food delivery was contingent on arrival at a feeder site, subjects' location could be decoded from dorsal striatal ensembles with considerable accuracy (see also van der Meer et al. 2010). However, when rats were rewarded for traveling past a fixed number of feeder sites (in a task topologically similar to task₂ used here), position was no longer decodable from dorsal striatal ensembles (Schmitzer-Torbert and Redish 2008). Thus, in the dorsal striatum, spatial information is available in neural representations as rats perform a spatially rewarded task but not after a change in reward contingency. In contrast, our results showed that hippocampal representations contained spatial information regardless of the reward contingency, and in fact show decreased within-session variability during the arguably less spatially contingent task₂ condition.

Summary.

In conclusion, we have shown that when rats adapt their strategies to changes in the reward structure of a behavioral task, hippocampal cells reflect the switch with a decrease in intertrial variability, resulting in more temporally reliable representations of position. We showed that this decrease in variability manifested as increased ensemble self-consistency without impacting other measures of Bayesian decoding quality. This work supports models of the hippocampal function based on multiple, interacting representations of space (McNaughton et al. 1996; Redish and Touretzky 1997; Samsonovich and McNaughton 1997; Touretzky and Redish 1996), influenced by behavioral processes such as attention (Fenton et al. 2010) or cognitive control (Kelemen and Fenton 2010).

GRANTS

This work was supported by National Institutes of Health Grants 5T90-DK-070106-05 (A. M. Wikenheiser), T32-DA-008234 (A. M. Wikenheiser), and R01-MH-083018 (A. D. Redish).

DISCLOSURES

No conflicts of interest, financial or otherwise, are declared by the author(s).