Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2017 Dec 7.
Published in final edited form as: Neuron. 2016 Nov 17;92(5):975–982. doi: 10.1016/j.neuron.2016.10.028

Interplay between hippocampal sharp wave ripple events and vicarious trial and error behaviors in decision making

Andrew E Papale 1, Mark C Zielinski 2, Loren Frank 3, Shantanu P Jadhav 4, A David Redish 5,*
PMCID: PMC5145752  NIHMSID: NIHMS824277  PMID: 27866796

Summary

Current theories posit that memories encoded during experiences are subsequently consolidated into longer-term storage. Hippocampal sharp-wave ripple (SWR) events have been linked to this consolidation process during sleep, but SWRs also occur during awake immobility, where their role remains unclear. We report that awake SWR rates at the reward site are inversely related to the prevalence of vicarious trial and error (VTE) behaviors, thought to be involved in deliberation processes. SWR rates were diminished immediately after VTE behaviors and an increase in the rate of SWR events at the reward site predicted a decrease in subsequent VTE behaviors at the choice point. Furthermore, SWR disruptions increased VTE behaviors. These results suggest an inverse relationship between SWRs and VTE behaviors, and suggest that awake SWRs and associated planning and memory consolidation mechanisms are engaged specifically in the context of higher levels of behavioral certainty.

Introduction

The hippocampus has been implicated in many aspects of cognition including navigation (O'Keefe and Nadel, 1978; Redish, 1999), imagination (Buckner and Carroll, 2007; Hassabis and Maguire, 2011) and the consolidation of memories (Buzsáki, 1989; Carr et al., 2011). The working hypothesis of the hippocampal field is that hippocampal sequences underlie these cognitive processes (Skaggs et al., 1996; Foster and Wilson, 2007; Wikenheiser and Redish, 2015a). These sequences occur during two largely distinct hippocampal states (Vanderwolf, 1971; O'Keefe and Nadel, 1978): theta (a 6–10 Hz continuous oscillation, occurring most prominently during movement and attentive states) and sharp wave ripple complexes (SWR, a 100–200 ms transient 200 Hz oscillation, occurring during slow wave sleep and awake stillness). When rats pause at decision points, theta sequences alternate between options (Johnson and Redish, 2007). This behavior (vicarious trial and error, VTE) occurs during high uncertainty (Gardner et al., 2013; Schmidt et al., 2013; Redish, 2016), which would necessitate the exploration of future options. VTE has thus been suggested to provide a measure of uncertainty, and theta sequences during VTE may provide a neural substrate for the internal exploration of those possible future options (Johnson and Redish, 2007; Amemiya and Redish, 2016; Redish, 2016).

In contrast, SWR sequences occur both during sleep and waking. SWRs during sleep have been linked to consolidation processes (Ego-Stengel and Wilson, 2010; Girardeau and Zugaro, 2011; de Lavilléon et al., 2015) wherein repeated reactivation of sequences is thought to engrain representations in distributed hippocampal-cortical networks (Alvarez and Squire, 1994; Sutherland and McNaughton, 2000; Nadel et al., 2012; Silva et al., 2015). SWRs during waking are important for learning (Jadhav et al., 2012) and occur most often during periods of rest at rewarded locations (Karlsson and Frank, 2009; Redish, 1999). The specific role that awake SWRs play in memory processes remains unclear. There is evidence that, like theta sequences, SWR sequences contribute to internal exploration of possible future options (Pfeiffer and Foster, 2013; Singer et al., 2013), but SWR sequences also encode paths unrelated to immediate options (Davidson et al., 2009; Gupta et al., 2010) or even to the immediate task at hand (Jackson et al., 2006; Karlsson and Frank, 2009), and the prevalence of SWRs at reward locations suggests a potential role in consolidation processes that could link a previous experience to its outcome (Foster and Wilson, 2006; Singer and Frank, 2009; Buzsáki, 2015).

We report here an inverse relationship between SWRs at reward sites and VTE at a choice point. VTE became less frequent and SWRs more frequent as animals learned to exploit a rule. VTE at a choice point was associated with a decreased rate of SWR events at the subsequent reward site, and a prevalence of SWR events at a reward site was associated with decreased VTE on the subsequent lap. Furthermore, on a second task, we found that selective interruption of SWRs during learning led to an increased prevalence of VTE. These dynamics imply a complex interaction between VTE, theta sequences, and SWR sequences, suggesting an inverse relationship between SWRs and VTE.

Results

In rats, monkeys, and humans, learning often entails a transition from attentive to more automated processes (O'Keefe and Nadel, 1978; Squire, 1987; Hikosaka et al., 1999; Redish, 2013). One intriguing possibility is that the two phenomena (VTE and SWRs) occur at different times in this transition. To directly examine the relationship between VTE, SWRs, navigational planning, and behavioral flexibility, we examined the interplay between VTE and SWRs on two decision tasks, one which included a within-session transition from flexible to more automated behaviors (Papale et al., 2012), and the other which combined across-session development of environmental familiarity with the learning of a complex decision-rule (Karlsson and Frank, 2009; Jadhav et al., 2012).

The first task was the spatial adjusting delay discounting (DD) task (Papale et al., 2012), a neuroeconomic task with a single decision point at a T-intersection (Fig. 1A). In this task, rats are faced with a choice between a small reward delivered quickly (1x 45mg unflavored pellet delivered after 1s, smaller-sooner) or a larger reward delivered after a delay (3x food pellets delivered after Ds, larger-later). Larger-later and smaller-sooner sides were counterbalanced between sessions, as was the starting delay (initial range 1–30s). The delay D was adjusted based on the rat's decisions: choosing larger-later increased D by 1s, while choosing smaller-sooner decreased D on the larger-later side by 1s. Thus, alternating between the two sides leaves the delay unchanged.

Figure 1. Behavior on the spatial delay discounting task.

Figure 1

Open in a new tab

A: Task layout. One side provides a larger reward (3x) after a delay D; the other side provides a smaller reward (1x) after 1s. The delay is adjusted as a function of the animal's decisions. B: Delay on the adjusted delay side by lap. Red indicates laps to the delayed side, increasing D by 1s; blue indicates laps to the non-delayed side, decreasing D by 1s. Small dots show LR and RL laps; circles show LL or RR laps. Behavior reveals three phases: investigation, titration, and exploitation. B1: upward titration; B2: downward titration. C: Gray dots show all sampled positions in a given session, with a single lap in red. VTE can be measured quantitatively with zIdphi (see supplemental methods). C1: VTE pass (zIdphi =6.05); C2: non-VTE pass (zIdphi =−0.13). D: zIdphi distributed in a skewed manner, but can be separated into VTE events and non-VTE events. The threshold between VTE and not was set at zIdphi =0.5, the point where the observed distribution diverged from the expected normal distribution (see supplemental methods). E: VTE decreased in the exploitation phase. Bars show interquartile range, line shows median, notch shows standard error of the median. F: Calculated distance between paths (see supplemental methods). Paths became more stereotyped with time. Early laps are distant from each other as well as from the later laps. Later laps are more stereotyped, marked by a lack of distance between the paths.

Behavior on this task typically proceeds through three separable phases in each daily session (Papale et al., 2012): Investigation — rats alternated sides to identify the delayed side and the initial starting delay D. Titration — rats adjusted the delay by preferentially selecting one side over the other. Exploitation — rats alternated sides, holding the delay at a preferred indifference point (Fig. 1B) (Papale et al., 2012; Bett et al., 2015; Breton et al., 2015; Mazur, 1997).

On a proportion of passes through the T choice, rats paused and re-oriented towards each option (vicarious trial and error, VTE, Fig. 1C1) and on a proportion of passes rats ran ballistically through (non-VTE. Fig. 1C2). We quantified VTE with a z-scored measure of the integrated angular velocity (zIdphi, see supplemental methods). Consistent with previous experiments, zIdphi was highly skewed (skewness of the histogram of all laps =2.15, median skewness per session was 2.0 ± 0.07 SEM). VTE laps were defined to be those with zIdphi > 0.5. (See supplemental methods and Fig. 1D)

Behavioral analyses indicated that exploitation was marked by a decrease in VTE relative to the other two phases (ANOVA, overall effect, df=2, n=12058, F=256, p<10−51, post-hoc Tukey test exploitation relative to investigation and titration phases, p<0.0001 each, Cohen's D=0.2) (Fig. 1E). 33% of the laps in the titration phases showed VTE, and 25% of the laps in the investigation phases showed VTE, while only 14% of the laps in the Exploitation phase showed VTE. Exploitation was also marked by an increase in stereotyped alternation laps as evidenced by an increase in the consistency of the rats’ paths of travel (Fig. 1F). Whereas non-VTE laps were almost all alternation laps (93% of non-VTE laps were alternation), VTE laps were evenly divided between alternation and adjustment laps (46% of VTE laps were alternation). These findings suggest that exploitation is a period in which the decision about where to go was made earlier, likely at the exit from the reward site, rather than at the T choice. In contrast, on VTE laps, the decision of where to go seems to be made at the T choice itself.

To determine what information was represented while the rat was at the T (Fig. 2A), we divided the maze into three regions (reward sites, choice point, the rest of the maze, Fig. 2B) and decoded the animal's location from the spiking activity (see supplemental methods). We measured the mean of the posterior of the decoded representation. On average, the posterior decoded probability was primarily local, even when the rat was at the choice point (Fig. 2C,I), and the small amount of posterior decoded probability that distributed non-locally to the feeders while the rat was at the choice point was primarily towards the choice the rat would subsequently choose (Fig. 2D, Wilcoxon ranksum test, z=46, n=95766, p<10−100, Cohens D=0.2).

Figure 2. Hippocampal representations during VTE.

Figure 2

Open in a new tab

While rats were at the choice point (A), we measured the mean Bayesian posterior probability over the choice point (green), the reward sites (blue), and the rest of the maze (brown) (B). The data in this figure come from all three phases. C: While the rat was at the choice point, most of the posterior remained local. D: The small portion of the posterior that distributed to the reward sites was significantly more distributed to the side that would subsequently be chosen. E: Decoding to the reward sites increased during VTE, matched (F) by a decrease in the decoding locally at the choice point. G: Decoding on the rest of the maze remained unchanged. H: On non-VTE laps, the non-local decoding to the reward sites was preferentially distributed towards the chosen side, whereas it was more evenly distributed on VTE laps. I: Average decoding probabilities. White boxes show the regions used to calculate the decoded posterior probability for the reward sites. Laps in which the rat went left have been flipped around the midline for display purposes. On both VTE and non-VTE passes, the majority of the decoded posterior remained at the choice point; however, the small amount of decoded posterior at the reward sites was different under VTE and non-VTE conditions. J: The average decoded posteriors of all VTE laps z-scored against the mean and standard deviations found in the non-VTE laps. K: Theta cycle representations encoded one side or the other but not both. For each decoded sample, we measured the proportion of the posterior assigned to each goal side (larger-later or smaller-sooner). On both VTE and non-VTE laps, when there was increased posterior to one side or the other, there was no increased posterior to the other side. This implies that the theta sequences were alternating between options serially, not simply spreading out ahead of the animal. Boxplots (C-H) show IQR (box), median (line), and standard error of the median (notch).

This analysis revealed a systematic shift in representation during VTE laps (Fig. 2H,J, ANOVA [n=143663], effect of region [df=2, F=83040, p<10−100, η2=0.3], effect of VTE [df=1, F=22, p<10−100, η2=10−4], interaction [df=2, F=112, p<10−100, η2=10−3]). VTE laps were marked by increased decoding of the reward location (Fig. 2E, Wilcoxon ranksum test [z=9, p<10−20, Cohens’ D=0.1]), and a concomitant decrease in decoding of the animal's actual location at the T (Fig. 2F, Wilcoxon ranksum test [z=−8, p<10−15, Cohen's D=0.1]). There was no significant change in decoding on the rest of the maze (Fig. 2G, Wilcoxon ranksum, [z=1.8, p=0.07]). Consistent with previous experiments (Johnson and Redish, 2007; Amemiya and Redish, 2016), there was decoding to both sides during VTE events, but preferentially to the chosen side on non-VTE laps (Fig. 2H, ANOVA [n=95775], effect of region (chosen/unchosen) [df=1, F=322, p<10−72, η2=10−3], effect of VTE [df=1, F=72, p<10−17, η2=10−3], interaction [df=1,F=142, p< 10−32,η2=10−3]). This suggests that on laps in which the rats ran ballistically through the choice point (not showing VTE), they knew their target goal before arriving at the choice point, and the hippocampal sequences preferentially reflected only the chosen goal (Wikenheiser and Redish, 2015b; Amemiya and Redish, 2016). However, consistent with previous work (Johnson and Redish, 2007; Amemiya and Redish, 2016), on VTE laps, the hippocampal sequences were more equally divided between the two options. These neurophysiological findings suggested that the rat was still deciding where to go at the T on VTE laps, but that on non-VTE laps, it already knew where it was going to go before it arrived at the T.

We also examined the decoded representation on the timescale of the 6-10 Hz theta rhythm. Consistent with previous work (Johnson and Redish, 2007; Gupta et al., 2012; Amemiya and Redish, 2016), theta cycles represented each side serially, even during VTE events. During any given theta cycle, the sequences were preferentially towards one side or the other, and tended not to distribute simultaneously across both sides. This was true for both VTE and non-VTE laps (Fig. 2K).

There are three potential explanations for the low (but significant) increased decoding to reward sites during VTE events. (1) The entire assembly could transiently jump to the reward site, with the low probability due to our inability to decode at fast enough time scales due to ensemble size. (2) A sequence could run from the current location to the goal, with the low probability due to our misalignment of decoding time bins with the timing of the sequence. (3) The representation could stretch to include both the current location and the reward site. We cannot differentiate these possibilities from the data here; however, other work looking directly at sequences (Wikenheiser and Redish, 2015b; Wang et al., 2014; Gupta et al., 2012; Feng et al., 2015) suggest that the second alternative is the most likely.

Hippocampal functional connectivity, information processing, and neural activity patterns differ between θ and the hippocampal state in which SWRs occur (LIA, marked by a more broad-spectral local field potential with power in 2–4 Hz [δ]). To determine whether these two events (passes through the choice point, VTE or not, and departure from a reward site) occur during similar or different hippocampal network states, we measured the local field potentials from the hippocampal pyramidal layer and analyzed their spectral components. Over the entire session, there was increased power in both these frequency ranges (Fig. S1B). Because SWRs are transient events, they vanish in the averaging process of the PSD, but they can be revealed by measuring auto-coherence plots, which measure correlations across frequencies (Masimore et al., 2004). These plots revealed transient events in the 180–200 Hz range, with cross-spectral power in lower frequencies (Fig. S1A). While these SWR transients were present in the 3s prior to departing the reward zone (Fig. S1C), they were not present during passes through the T (Fig. S1D), even during VTE events (Fig. S1E). Supporting this distinction, δ showed stronger power than θ in the 3s prior to departure from the reward site (Fig. S1F), but θ showed greater power than δ at the choice point (Fig. S1G), even during VTE (Fig. S1H). The difference can be seen by subtracting the auto-coherence plots (Fig. S1I) and the PSDs (Fig. S1J). These two situations (leaving the feeder site and VTE events) were marked by different hippocampal network states, as can be seen in the different peri-event aligned spectrograms (Fig. S1K,L,M).

To examine the interaction between SWRs and VTE events, we measured the rate of SWRs during the pause time at the reward site, after getting food and prior to leaving the reward site. We compared SWR rates across task phase (investigation/titration/exploitation) and with regards to the presence or absence of VTE during the lap approaching the reward site (Fig. 3A) or after the reward site visit (Fig. 3B). SWR rates increased during exploitation relative to the investigation and titration phases. The SWR rate was significantly lower when VTE occurred on the lap preceding the feeder visit in question (ANOVA [n=11260], effect of Rat [df=5, F=124, p<10−100, η2=0.04]; effect of Session [continuous, F=2911, p<10−100, η2=0.2]; effect of Phase [df=2, F=8.0, p=0.0003, η2=0.001]; effect of VTE [df=1, F=5.25, p=0.02, η2=10−4]). There were minor differences between individual rats (Fig. S4) and an increase in SWR rate across sessions (Fig. S2A), so we included rat and session in the ANOVA model as separate factors. Significances did not change if we included lap instead of phase. Even with these components, VTE on the previous lap remained a significant explanatory variable in the ANOVA model. Adding in the speed of the lap did not change the significance of preceding VTE as an explanatory variable.

Figure 3. Sharp waves were more common before and after non-VTE laps.

Figure 3

Open in a new tab

A: We divided reward site experiences based on whether a VTE (zIdphi > 0.5) event occurred as the animal passed the choice point approaching the reward site experience. There was a lower rate of SWR events at the reward site after a VTE lap than after a non-VTE lap. B: If we divided reward site experiences based on whether a VTE (zIdphi > 0.5) event occurred on the subsequent lap (following the reward site experience), there was a lower rate of SWR events before a VTE lap than a non-VTE lap. Boxplots show IQR (box), median (line), and standard error of the median (notch).

SWR rates in the pause time at the reward site also predicted the prevalence of VTE on the following lap, suggesting bi-directional interactions. There were significantly fewer SWR events preceding VTE events on the subsequent lap (ANOVA [n=11268], effect of Rat [df=5, F=124, p<10−100, η2=0.04]; effect of Session [continuous, F=2914, p<10−100, η2=0.2]; effect of Phase [df=2, F=7.9, p=0.0004, η2=0.001]; effect of VTE [df=1, F=5.22, p=0.02, η2=10−4]). Subsequent VTE remained a significant explanatory variable in the ANOVA model, even with the inclusion of rat, session, and phase in the model. Using lap instead of phase and adding in the speed of the lap did not change the results. Similar effects could be seen if we measured zIdphi as a function of a median split on SWR rate.

SWR events were more likely to occur when rats took a ballistic path through the T, and higher SWR rates increased the likelihood of taking a ballistic path through the next pass. These results are consistent with previous experiments showing a negative correlation between SWR rate and behavioral variability (Jackson et al., 2006) on familiar environments, although other experiments have found increased SWR rates during novel experiences (O'Neill et al., 2006; Cheng and Frank, 2008).

All animals had extensive experience on the DD task before recording, including at least one month of training before implantation of the recording electrodes, as well as at least two weeks of training after implantation; nevertheless, we observed large changes in VTE and the rate of SWRs, as well as differences in behavioral regularity across the 30-day experiment. The zIdphi measure of VTE decreased significantly over sessions (Fig. S2B), driven primarily by a decrease in VTE during the exploitation phase (ANOVA [n=11268], effect of rat [df=5, F=12, p<10−100, η2=0.005], effect of phase [df=2, F=18, p<10−100, η2=0.003], effect of session [df=continuous, F=118, p<10−100, η2=0.01], interaction between phase and session [df=2, F=3.8, p=0.02, η2=10−3]). In parallel, the rate of SWR events increased over sessions (Fig. S2A), driven primarily by an increase in SWR rates during the exploitation phase (ANOVA [n=11268], effect of rat [df=5, F=125, p<10−100, η2=0.05], effect of phase [df=2, F=0.73, p=0.48], effect of session [df=continuous, F=1656, p<10−100, η2=0.1], interaction between phase and session [df=2, F=7.0, p=0.0009, η2=10−3]). These changes occurred along with an increase in the efficiency on the task (Fig. S2C) (ANOVA [n=11268], effect of rat [df=5, F=1.8, p=0.11, η2=0.06], effect of session [df=continuous, F=5.2, p=0.025, η2=0.03]). To measure the changes themselves, we measured the slopes of each, see Supplemental Table S1).

We also found that the cumulative number of SWR events emitted within a session up to a given lap was negatively correlated with the presence or absence of a VTE event on that lap, even after controlling for the fact that VTE tended to occur on earlier laps and earlier phases of the task (ANOVA [n=11629], effect of phase, [df=2, F=45, p<10−100, η2=0.01]; effect of lap, [df=continuous, F=130, p<10−100, η2=0.01]; effect of cumulative number of SWR events, [df=continuous, F=5, p<0.02, η2 =10−4]; three-way interaction, [df=3, F=6, p=0.005, η2=0.001], even after including effects of rat, [df=5, F=18, p<10−100, η2=0.01] and session [df=continuous, F=378, p<10−100, η2=0.03]). This observation is consistent with a role for awake SWRs in consolidation and the firming-up of a map within a session (Buzsáki, 1989; O'Neill et al., 2006; Carr et al., 2011; Gupta et al., 2010). Moreover, it predicts that SWR disruption might influence VTE behavior.

To determine whether SWRs causally influence VTE behavior, we reanalyzed data from a spatial memory task in which SWRs were disrupted by stimulating the ventral hippocampal commissure when they were detected (Jadhav et al., 2012). In this task, rats were trained to alternate on a W-shaped maze (Fig. 4A). Jadhav et al. found that disrupting SWRs led to an increase in working memory-dependent (outbound) errors, but not simple (inbound) errors. Outbound errors were defined as returning to the previous arm on the outbound journey instead of alternating. Inbound errors were defined as the animal not returning to the center arm on the return journey, whether incorrectly proceeding to the other outer arm or turning around and repeating a visit to the outer arm the animal was already in.

Figure 4. SWR disruption increases VTE behavior in a spatial alternation task.

Figure 4

Open in a new tab

Data re-analyzed from Jadhav et al. (2012). A: Rats ran out from the center to a side arm (A1, outbound) then returned to the center (A2, inbound). The following trial required visiting the alternate side arm (A3) before again returning to center (A4). B: VTE behavior during outbound trajectories was quantified by lnIdphi. (See supplemental methods.) There were more high lnIdphi choice-point passes (VTE) in the SWR disruption animals as compared to the control group. Line and shaded area shows mean and SEM.

We hypothesized that the SWR disruptions would increase VTE events. We tested this hypothesis by comparing lnIdphi scores between SWR-disrupted and control rats. lnIdphi does not normalize within rat, allowing better comparison across groups of rats. (See supplemental methods.)

We found that disrupting SWRs produced a dramatic increase in VTE, as evidenced by an increase in choice-point passes with high lnIdphi scores (Fig. 4B, rank-sum test between distributions, [n=1461 control, 1719 disruption] z = −17, p<10−67, Cohen's D=0.6, z-proportion test for fraction of VTE events, z = 12, p<10−100). The increase in VTE due to SWR disruption could not be explained solely by the increase in error trials in the disruption animals (Fig. S3 A,B; ANOVA [n=3180]: control vs. disruption, [df=1, F=138, p<10−100, η2=0.04], correct vs. error trials, [df=1, F=67, p<10−100, η2=0.02], interaction, [df=1, F=11, p<0.001, η2=0.003]). (see Fig. S3C). We also found that this difference in VTE behavior between disruption and control animals was seen both during initial learning and during performance in later days (Fig. S3D). The SWR disruption occurred throughout learning and disrupted the rate of learning on the outbound decisions of the W-task.

Discussion

The working hypothesis of the hippocampal field is that information processing underlying cognitive processes depends on sequential representations expressed during theta cycles (including during VTE events) and SWR events. We examined the interplay between VTE and SWR events and found that SWR rates were diminished following VTE events and that VTE events were diminished following increased SWR rates. We also found a negative relationship between the number of SWR events emitted within a single session and the number of VTE events in that session, and that a disruption of SWR events led to an increase in VTE events.

While older theories of SWR function suggested a primarily offline role in consolidation of recent memories (Buzsáki, 1989; Sutherland and McNaughton, 2000), newer observations have found SWR sequences related to immediately-available future options (Diba and Buzsáki, 2007; Pfeiffer and Foster, 2013; Singer et al., 2013), as well as backward paths (Foster and Wilson, 2006; Davidson et al., 2009; Gupta et al., 2010; Wikenheiser and Redish, 2013), novel paths (Gupta et al., 2010; Dragoi and Tonegawa, 2011; Ólafsdóttir et al., 2015), and other environments (Jackson et al., 2006; Karlsson and Frank, 2009; Silva et al., 2015).

An intriguing possibility is that the non-theta hippocampal states in which SWRs occur are similar to introspective representations that may parallel the default network in humans (Buckner et al., 2008), and that theta states may parallel executive function in humans (Garavan et al., 2002). One theory is that SWRs reflect processes exploring the cognitive space of the task to find connections (Samsonovich and Ascoli, 2005). This hypothesis would suggest that SWRs may concentrate on areas of particular interest and importance, which would be consistent with the small increase in representation of future plans (Diba and Buzsáki, 2007; Pfeiffer and Foster, 2013; Ólafsdóttir et al., 2015), recent experiences (Wilson and McNaughton, 1994; Jackson et al., 2006; Singer and Frank, 2009), and novel paths in an environment (Gupta et al., 2010; Dragoi and Tonegawa, 2011). It would also be consistent with the observation that on tasks where it is important to maintain non-recently experienced portions of the maze, it is those non-recently experienced portions that are more represented (Gupta et al., 2010). Thus a syncretic hypothesis would be that SWRs play a role in establishing, sustaining, and exploring the cognitive map, which is a form of schema development, and very much in line with a generalization of consolidation theories.

This syncretic hypothesis is supported by our data. It suggests that a disruption of SWR events should lead to increased confusion and deliberation, and that increased SWR events would decrease confusion in both the short and long-term. Moreover, our data that the presence of VTE diminishes the subsequent rate of SWR events suggests that in familiar environments, SWRs primarily play this role after stable behaviors are established.

Both theta and SWR sequences play roles in various hippocampal functions. We found that these two processes interacted, shifting from variable behaviors that included a preponderance of VTE events to less variable behaviors marked by an increase in SWR events. The occurrence of VTE at a decision reduced the number of subsequent SWRs at a reward site, and a preponderance of SWRs on a given lap diminished the likelihood of subsequent VTE events. A selective interruption of SWRs during learning led to an increased prevalence of VTE. These dynamics imply a complex interaction between theta and SWR sequences, suggesting that while VTE sequences may reflect an immediate decision-making process, SWR sequences may reflect ongoing consolidation and planning processes that depend on and predict uninterrupted behavior.

Supplementary Material

Acknowledgements

This work was supported by NIH grants MH080318, MH090188, MH100284, and a Sloan Fellowship to SPJ.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Author Contributions. The DD experiment was designed by AEP and ADR. AEP collected the DD data and analyzed it with ADR. SPJ and LF designed the W-maze experiment and collected the sharp-wave disruption data, which was analyzed by MCZ and SPJ. All five authors co-wrote the paper.

Contributor Information

Andrew E. Papale, Graduate Program in Neuroscience, University of Minnesota, papalae04@gmail.com

Mark C. Zielinski, Graduate Program in Neuroscience, Brandeis University, mcz@brandeis.edu

Loren Frank, HHMI, Kavli Institute for Fundamental Neuroscience, Department of Physiology and Center for Integrative Neuroscience, UCSF, loren@phy.ucsf.edu.

Shantanu P. Jadhav, Neuroscience Program, Department of Psychology and Volen Center for Complex Systems, Brandeis University, shantanu@brandeis.edu

A. David Redish, Department of Neuroscience, University of Minnesota, redish@umn.edu.

References

  1. Alvarez P, Squire LR. Memory consolidation and the medial temporal lobe: A simple network model. Proceedings of the National Academy of Sciences, USA. 1994;91:7041–7045. doi: 10.1073/pnas.91.15.7041. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Amemiya S, Redish AD. Manipulating decisiveness in decision making - effects of clonidine on hippocampal search strategies. Journal of Neuroscience. 2016;36:814–827. doi: 10.1523/JNEUROSCI.2595-15.2016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Bett D, Murdoch LH, Wood ER, Dudchenko PA. Hippocampus, delay discounting, and vicarious trial-and-error. Hippocampus. 2015;25:643–654. doi: 10.1002/hipo.22400. [DOI] [PubMed] [Google Scholar]
  4. Blumenthal A, Steiner A, Seeland KD, Redish AD. Effects of pharmacological manipulations of NMDA-receptors on deliberation in the Multiple-T task. Neurobiology of Learning and Memory. 2011;95:376–384. doi: 10.1016/j.nlm.2011.01.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Breton YA, Seeland KD, Redish AD. Aging impairs deliberation and behavioral flexibility in inter-temporal choice. Frontiers in aging neuroscience. 2015;7 doi: 10.3389/fnagi.2015.00041. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Buckner RL, Andrews-Hanna JR, Schacter DL. The brain's default network. anatomy, function, and relevance to disease. Annalys of the New York Academy of Sciences. 2008;1124:1–38. doi: 10.1196/annals.1440.011. [DOI] [PubMed] [Google Scholar]
  7. Buckner RL, Carroll DC. Self-projection and the brain. Trends in Cognitive Sciences. 2007;11:49–57. doi: 10.1016/j.tics.2006.11.004. [DOI] [PubMed] [Google Scholar]
  8. Buzsáki G. Two-stage model of memory trace formation: A role for “noisy” brain states. Neuroscience. 1989;31:551–570. doi: 10.1016/0306-4522(89)90423-5. [DOI] [PubMed] [Google Scholar]
  9. Buzsáki G. Hippocampal sharp wave-ripples: A cognitive biomarker for episodic memory and planning. Hippocampus . 2015 doi: 10.1002/hipo.22488. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Carr MF, Jadhav SP, Frank LM. Hippocampal replay in the awake state: a potential substrate for memory consolidation and retrieval. Nature Neuroscience. 2011;14:147–153. doi: 10.1038/nn.2732. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Cheng S, Frank LM. New experiences enhance coordinated neural activity in the hippocampus. Neuron. 2008;57:303–313. doi: 10.1016/j.neuron.2007.11.035. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Davidson TJ, Kloosterman F, Wilson MA. Hippocampal replay of extended experience. Neuron. 2009;63:497–507. doi: 10.1016/j.neuron.2009.07.027. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. de Lavilléon G, Rondi-Reig MML, Benchenane K. Explicit memory creation during sleep demonstrates a causal role of place cells in navigation. Nature Neuroscience. 2015;18:493–495. doi: 10.1038/nn.3970. [DOI] [PubMed] [Google Scholar]
  14. Diba K, Buzsáki G. Forward and reverse hippocampal place-cell sequences during ripples. Nature Neuroscience. 2007;10:1241–1242. doi: 10.1038/nn1961. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Dragoi G, Tonegawa S. Preplay of future place cell sequences by hippocampal cellular assemblies. Nature. 2011;469:397–401. doi: 10.1038/nature09633. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Ego-Stengel V, Wilson MA. Disruption of ripple-associated hippocampal activity during rest impairs spatial learning in the rat. Hippocampus. 2010;20:1–10. doi: 10.1002/hipo.20707. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Feng T, Silva D, Foster DJ. Dissociation between the experience-dependent development of hippocampal theta sequences and single-trial phase precession. The Journal of Neuroscience. 2015;35:4890–4902. doi: 10.1523/JNEUROSCI.2614-14.2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Foster DJ, Wilson MA. Reverse replay of behavioural sequences in hippocampal place cells during the awake state. Nature. 2006;440:680–683. doi: 10.1038/nature04587. [DOI] [PubMed] [Google Scholar]
  19. Foster DJ, Wilson MA. Hippocampal theta sequences. Hippocampus. 2007;17:1093–1099. doi: 10.1002/hipo.20345. [DOI] [PubMed] [Google Scholar]
  20. Garavan H, Ross TJ, Murphy K, Roche RAP, Stein EA. Dissociable executive functions in the dynamic control of behavior: Inhibition, error detection, and correction. NeuroImage. 2002;17:1820–1829. doi: 10.1006/nimg.2002.1326. [DOI] [PubMed] [Google Scholar]
  21. Gardner RS, Uttaro MR, Fleming SE, Suarez DF, Ascoli GA, Dumas TC. A secondary working memory challenge preserves primary place strategies despite over-training. Learning and Memory. 2013;20:648–656. doi: 10.1101/lm.031336.113. [DOI] [PubMed] [Google Scholar]
  22. Girardeau G, Zugaro M. Hippocampal ripples and memory consolidation. Current Opinion in Neurobiology. 2011;21:452–459. doi: 10.1016/j.conb.2011.02.005. [DOI] [PubMed] [Google Scholar]
  23. Gupta AS, van der Meer MAA, Touretzky DS, Redish AD. Hippocampal replay is not a simple function of experience. Neuron. 2010;65:695–705. doi: 10.1016/j.neuron.2010.01.034. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Gupta AS, van der Meer MAA, Touretzky DS, Redish AD. Segmentation of spatial experience by hippocampal theta sequences. Nature Neuroscience. 2012;15:1032–1039. doi: 10.1038/nn.3138. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Hassabis D, Maguire EA. The construction system in the brain. In: Bar M, editor. Predictions in the Brain: Using our Past to Generate a Future. Oxford University Press; 2011. pp. 70–82. [Google Scholar]
  26. Hikosaka O, Nakahara H, Rand MK, Sakai K, Lu X, Nakamura K, Miyachi S, Doya K. Parallel neural networks for learning sequential procedures. Trends in Neurosciences. 1999;22:464–471. doi: 10.1016/s0166-2236(99)01439-3. [DOI] [PubMed] [Google Scholar]
  27. Jackson JC, Johnson A, Redish AD. Hippocampal sharp waves and reactivation during awake states depend on repeated sequential experience. Journal of Neuroscience. 2006;26:12415–12426. doi: 10.1523/JNEUROSCI.4118-06.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Jadhav SP, Kemere C, German PW, Frank LM. Awake hippocampal sharp-wave ripples support spatial memory. Science. 2012;336:1454–1458. doi: 10.1126/science.1217230. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Johnson A, Redish AD. Neural ensembles in CA3 transiently encode paths forward of the animal at a decision point. Journal of Neuroscience. 2007;27:12176–12189. doi: 10.1523/JNEUROSCI.3761-07.2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Karlsson MP, Frank LM. Awake replay of remote experiences in the hippocampus. Nature Neuroscience. 2009;12:913–918. doi: 10.1038/nn.2344. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Masimore B, Kakalios J, Redish AD. Measuring fundamental frequencies in local field potentials. Journal of Neuroscience Methods. 2004;138:97–105. doi: 10.1016/j.jneumeth.2004.03.014. [DOI] [PubMed] [Google Scholar]
  32. Mazur J. Choice, delay, probability and conditioned reinforcement. Animal Learning and Behavior. 1997;25:131–147. [Google Scholar]
  33. Nadel L, Hupbach A, Gomez R, Newman-Smith K. Memory formation, consolidation and transformation. Neuroscience and Biobehavioral Reviews. 2012;36:1640–1645. doi: 10.1016/j.neubiorev.2012.03.001. [DOI] [PubMed] [Google Scholar]
  34. O'Keefe J, Nadel L. The Hippocampus as a Cognitive Map. Oxford; 1978. [Google Scholar]
  35. Ólafsdóttir HF, Barry C, Saleem AB, Hassabis D, Spiers HJ. Hippocampal place cells construct reward related sequences through unexplored space. Elife. 2015;4:e06063. doi: 10.7554/eLife.06063. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. O'Neill J, Senior T, Csicsvari J. Place-selective firing of ca1 pyramidal cells during sharp wave/ripple network patterns in exploratory behavior. Neuron. 2006;49:143–155. doi: 10.1016/j.neuron.2005.10.037. [DOI] [PubMed] [Google Scholar]
  37. Papale A, Stott JJ, Powell NJ, Regier PS, Redish AD. Interactions between deliberation and delay-discounting in rats. Cognitive, Affective, and Behavioral Neuroscience. 2012;12:513–526. doi: 10.3758/s13415-012-0097-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Pfeiffer BE, Foster DJ. Hippocampal place-cell sequences depict future paths to remembered goals. Nature. 2013;497:74–79. doi: 10.1038/nature12112. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Redish AD. Beyond the Cognitive Map: From Place Cells to Episodic Memory. MIT Press; 1999. [Google Scholar]
  40. Redish AD. The Mind within the Brain: How we make decisions and how those decisions go wrong. Oxford; 2013. [Google Scholar]
  41. Redish AD. Vicarious trial and error. Nature Reviews Neuroscience. 2016;17:147–159. doi: 10.1038/nrn.2015.30. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Samsonovich AV, Ascoli GA. A simple neural network model of the hippocampus suggesting its pathfinding role in episodic memory retrieval. Learning and Memory. 2005;12:193–208. doi: 10.1101/lm.85205. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Schmidt BJ, Papale AE, Redish AD, Markus EJ. Conflict between place and response navigation strategies: Effects on vicarious trial and error (VTE) behaviors. Learning and Memory. 2013;20:130–138. doi: 10.1101/lm.028753.112. [DOI] [PubMed] [Google Scholar]
  44. Silva D, Feng T, Foster DJ. Trajectory events across hippocampal place cells require previous experience. Nature Neuroscience. 2015;18:1772–1779. doi: 10.1038/nn.4151. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Singer AC, Carr MF, Karlsson MP, Frank LM. Hippocampal SWR activity predicts correct decisions during intial learning of an alternation task. Neuron. 2013;77:1163–1173. doi: 10.1016/j.neuron.2013.01.027. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Singer AC, Frank LM. Rewarded outcomes enhance reactivation of experience in the hippocampus. Neuron. 2009;64:910–921. doi: 10.1016/j.neuron.2009.11.016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Skaggs WE, McNaughton BL, Wilson MA, Barnes CA. Theta phase precession in hippocampal neuronal populations and the compression of temporal sequences. Hippocampus. 1996;6:149–173. doi: 10.1002/(SICI)1098-1063(1996)6:2<149::AID-HIPO6>3.0.CO;2-K. [DOI] [PubMed] [Google Scholar]
  48. Squire LR. Memory and Brain. Oxford; 1987. [Google Scholar]
  49. Steiner A, Redish AD. Orbitofrontal cortical ensembles during deliberation and learning on a spatial decision-making task. Frontiers in Decision Neuroscience. 2012;6:131. doi: 10.3389/fnins.2012.00131. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Stott JJ, Redish AD. A functional difference in information processing between orbitofrontal cortex and ventral striatum during decision-making behavior. Philosophical Transactions of the Royal Society B. 2014;369:20130472. doi: 10.1098/rstb.2013.0472. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Sutherland GR, McNaughton BL. Memory trace reactivation in hippocampal and neocortical neuronal ensembles. Current Opinion in Neurobiology. 2000;10:180–186. doi: 10.1016/s0959-4388(00)00079-9. [DOI] [PubMed] [Google Scholar]
  52. van der Meer MAA, Redish AD. Expectancies in decision making, reinforcement learning, and ventral striatum. Frontiers in Neuroscience. 2010 doi: 10.3389/neuro.01.006.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Vanderwolf CH. Limbic-diencephalic mechanisms of voluntary movement. Psychological Review. 1971;78:83–113. doi: 10.1037/h0030672. [DOI] [PubMed] [Google Scholar]
  54. Wang Y, Romani S, Lustig B, Leonardo A, Pastalkova E. Theta sequences are essential for internally generated hippocampal firing fields. Nature Neuroscience . 2014 doi: 10.1038/nn.3904. [DOI] [PubMed] [Google Scholar]
  55. Wikenheiser A, Redish AD. The balance of forward and backward hippocampal sequences shifts across behavioral states. Hippocampus. 2013;23:22–29. doi: 10.1002/hipo.22049. [DOI] [PMC free article] [PubMed] [Google Scholar]
  56. Wikenheiser AM, Redish AD. Decoding the cognitive map: ensemble hippocampal sequences and decision making. Current Opinion in Neurobiology. 2015a;32:8–15. doi: 10.1016/j.conb.2014.10.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  57. Wikenheiser AM, Redish AD. Hippocampal theta sequences reflect current goals. Nature Neuroscience. 2015b;18:289–294. doi: 10.1038/nn.3909. [DOI] [PMC free article] [PubMed] [Google Scholar]
  58. Wilson MA, McNaughton BL. Reactivation of hippocampal ensemble memories during sleep. Science. 1994;265:676–679. doi: 10.1126/science.8036517. [DOI] [PubMed] [Google Scholar]
  59. Zhang K, Ginzburg I, McNaughton BL, Sejnowski TJ. Interpreting neuronal population activity by reconstruction: Unified framework with application to hippocampal place cells. Journal of Neurophysiology. 1998;79:1017–1044. doi: 10.1152/jn.1998.79.2.1017. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

NIHMS824277-supplement.pdf (140.8KB, pdf)

RESOURCES