The role of replay and theta sequences in mediating hippocampal-prefrontal interactions for memory and cognition

Abstract

Sequential activity is seen in the hippocampus during multiple network patterns, prominently as replay activity during both awake and sleep sharp-wave ripples (SWRs), and as theta sequences during active exploration. Although various mnemonic and cognitive functions have been ascribed to these hippocampal sequences, evidence for these proposed functions remains primarily phenomenological. Here, we briefly review current knowledge about replay events and theta sequences in spatial memory tasks. We reason that in order to gain a mechanistic and causal understanding of how these patterns influence memory and cognitive processing, it is important to consider how these sequences influence activity in other regions, and in particular, the prefrontal cortex, which is crucial for memory-guided behavior. For spatial memory tasks, we posit that hippocampal-prefrontal interactions mediated by replay and theta sequences play complementary and overlapping roles at different stages in learning, supporting memory encoding and retrieval, deliberative decision making, planning and guiding future actions. This framework offers testable predictions for future physiology and closed-loop feedback inactivation experiments for specifically targeting hippocampal sequences as well as coordinated prefrontal activity in different network states, with the potential to reveal their causal roles in memory-guided behavior.

Introduction

The hippocampus is critical for rapid encoding, storage and retrieval of episodic memory. Our ability to learn, form memories of past experiences, and use these memories to guide ongoing behavior, depends on the hippocampus and its interactions with other brain regions. The physiological mechanisms underlying these roles are most-often studied in rodent spatial memory models, and these models have proven highly tractable in linking physiology to behavior. Particularly, sequential activity of place cells found in rodent hippocampus has gained increasing attention in recent years. This fast time-scale sequential activity is prevalent during two network patterns: theta oscillations (6–12 Hz) and sharp wave ripples (SWRs; 150–250 Hz). Hippocampal activity is dominated by theta oscillations during active exploration, and within each theta cycle, there are sequences of hippocampal spikes ordered according to the locations of their place fields during behavior (i.e., “theta sequences”). On the other hand, during inactive rest, such as waking immobility or slow wave sleep (SWS), short, transient bursts of high frequency oscillations occur in the hippocampus, called SWRs. Like theta sequences, hippocampal place fields are sequentially reactivated during SWRs (i.e., “hippocampal replay”). Clearly, hippocampal replay and theta sequences are generated during two different states as the animal engages (theta) and detaches (SWRs) from action-perception cycles (Pezzulo et al., 2017). Various mnemonic and cognitive roles have been proposed for these two types of hippocampal sequences; however current evidence remains primarily phenomenological.

Understanding the function of hippocampal sequences in memory-guided behavior will require investigating how these sequences relate to activity in brain regions outside the hippocampus. In addition to the regions within the medial temporal lobe memory system, interactions of the hippocampus with other regions, such as the frontal cortical regions, the striatum, and the amygdala, are crucial for cognitive functions. In particular, the prefrontal cortex (PFC, including the anterior cingulate, prelimbic and infralimbic areas) has an essential role in memory formation and retrieval, with related executive roles in working memory, decision making and cognitive flexibility (Eichenbaum, 2017b; Eichenbaum and Cohen, 2001; Genzel et al., 2014; Shin and Jadhav, 2016; Tse et al., 2011). The hippocampus forms strong connections with the PFC through direct and indirect pathways. Recent studies on the hippocampus and the PFC have emphasized the importance of their interactions and coordination through various anatomical pathways for memory-guided behaviors (Battaglia et al., 2011; Eichenbaum, 2017b; Euston et al., 2007; Preston and Eichenbaum, 2013; Shin and Jadhav, 2016). Especially, many of these studies have found that prefrontal activity is engaged during both hippocampal theta (Benchenane et al., 2010; Fujisawa and Buzsaki, 2011; Guise and Shapiro, 2017; Jones and Wilson, 2005) and SWRs (Jadhav et al., 2016; Peyrache et al., 2009; Tang et al., 2017; Wang and Ikemoto, 2016; Wierzynski et al., 2009). However, what still remains to be investigated is how this PFC activity is coordinated with the information conveyed by hippocampal sequences during different network states in support of cognitive functions.

Here, we aim to provide an overview of the mnemonic functions of replay and theta sequences, with a focus on the role of hippocampal-prefrontal interactions during these events. We summarize current physiological findings and causative experimental evidence that point towards various cognitive roles of replay and theta sequences. We suggest that investigating how these sequences mediate hippocampal-prefrontal interactions can promote an integrated and comprehensive understanding of their role in different stages of learning and performance, and propose future physiological and perturbation experiments to causally establish these functions.

Sleep Replay and Memory Consolidation

It is a long-standing idea for many decades (Marr, 1971) that reactivation of neural activity that was previously used to encode behavioral experience consolidates memory of that experience by selectively strengthening its memory traces. The finding of “hippocampal replay” as physiological evidence was an important breakthrough in supporting this notion. During active behavior, hippocampal pyramidal cells fire at specific locations in the environment (i.e., place cells; O’Keefe, 1976; O’Keefe and Dostrovsky, 1971), with sequential activation of groups of place cells depicting the animal’s trajectory. When an animal is at rest after behavior (sleep or immobility), the same sequences of place cells that are active during behavior are reactivated at a faster timescale during SWRs (Figure 1A, B). Early demonstration of this phenomenon of hippocampal replay came from a number of sleep studies (see Girardeau and Zugaro, 2011 for review): after waking behaviors, reactivation of place cell activity was observed during subsequent sleep in both rodents (Lee and Wilson, 2002; Wilson and McNaughton, 1994) and humans (Peigneux et al., 2004). This reactivation occurs predominantly during SWRs in SWS (rarely in rapid-eye-movement or REM sleep), with the time window between individual spikes suitable for synaptic plasticity. Reactivation is most prevalent during the first hours after learning when the memory is newly formed and requires consolidation (Eschenko et al., 2008; Kudrimoti et al., 1999), and is correlated with subsequent memory performance (Dupret et al., 2010; Ramadan et al., 2009). While evidence linking sleep replay with memory consolidation is compelling, only recently has their causative relationship been demonstrated in studies that showed that perturbing SWR activity during post-task sleep in rats impaired learning in a hippocampal-dependent spatial memory task (Ego-Stengel and Wilson, 2010; Girardeau et al., 2009).

Figure 1. Hippocampal replay and prefrontal reactivation during SWRs.

(A) Spatially selective firing of hippocampal place cells in the W-maze spatial alternation task. These cells are successively activated as the animal runs through their place fields (represented as color-coded ellipses; left). Along a certain linearized trajectory, place cells typically show a peak firing rate at the most “preferred” location (right).

(B) An example of a hippocampal replay event. Sharp wave ripples occur when the animal is at rest, and are associated with spikes from the place cells (same color code as in A) reactivated in the same order as in exploration but on a faster timescale (left). Typically, replay events are detected using Bayesian decoding (right). Estimated decoded positions during the replay event describe a reactivated trajectory (dashed line). Color bar indicates probability of position reconstruction. Panels adapted from Tang et al. (2017). This replay was observed across 8 simultaneously recorded CA1 place cells in awake rats.

(C) Schematic representation of PFC reactivation during SWRs. Left: When the animal runs through the environment, spatially-selective PFC neurons are co-activated with the hippocampal neurons that have overlapping spatial maps. Some PFC cells are more active when the animal runs through the maze (e.g., red and magenta cells), whereas other cells are more active at the reward well when the animal is immobile (e.g., black cell). Right: During SWRs, the PFC neurons (schematized as red) with spatial firing patterns overlapping with the CA1 place fields are preferentially reactivated (SWR-excited), whereas PFC neurons that encode immobility are suppressed (SWR-inhibited; Jadhav et al., 2016).

(D) More structured CA1-PFC reactivation during awake SWRs than sleep SWRs. Tang et al. (2017) measured the relationship between the similarity of spatial map pairs (spatial correlation) and the strength of their co-activation during SWRs (SWR co-firing). Sleep reactivation was examined during slow-wave sleep prior (pre-task; grey) and after (post-task; black) behavior. For both CA1-CA1 and CA1-PFC pairs, the co-activation of cell pairs during exploration (measured as spatial correlation) more accurately predicts their co-activation during SWRs in awake state (purple) as compared to sleep state. This high-fidelity reactivation of current behavioral experiences in the CA1-PFC network during awake SWRs is also seen in ensemble measures, and is ideally suited for accurate memory storage related to current experiences.

(E) CA1-PFC reactivation is enhanced during initial learning. As animals learned the W-track task over multiple days, spatial reactivation was measured as the correlation between SWR co-firing and spatial correlation. Note that this correlation for CA1-PFC pairs is highest on the first (novel exploration) day when the animal begins to learn the task, and progressively decreases on the latter days when performance starts to stabilize, suggesting a reduction in spatial reactivation over learning. Panels adapted from Tang et al. (2017).

Memory consolidation processes crucially depend on interactions between the hippocampus and cortical regions. Models posit that initial memory traces encoded in the hippocampus are repeatedly reactivated for long-term storage in distributed hippocampal-cortical networks. This involves either a transfer of representations from the hippocampus to cortex, or strengthening of existing representations in the hippocampal-cortical network during consolidation (“two-stage” and “systems consolidation” models; Born and Wilhelm, 2012; Buzsaki, 1989; Diekelmann and Born, 2010; Marr, 1971). Consistent with this view, global coordination between hippocampus and cortical regions is seen at the level of field potentials. During SWS, hippocampal SWRs associated with replay events are coordinated with cortical spindles (12–18 Hz), delta (1–4 Hz) and slow oscillations, promoting widespread synchronization throughout the brain (Inostroza and Born, 2013; Logothetis et al., 2012; Siapas and Wilson, 1998; Sirota et al., 2003; Staresina et al., 2015). At the level of neuronal activity, cortical units show transient increases in spiking, co-firing of cell pairs, and ensemble reactivation during sleep SWRs (Ji and Wilson, 2007; Peyrache et al., 2011; Rothschild et al., 2016; Tang et al., 2017; Wang and Ikemoto, 2016; Wierzynski et al., 2009).

When considering hippocampal-cortical interactions for memory consolidation, the prefrontal cortex (PFC) has a privileged role given its involvement in memory and executive functions. The PFC is required for memory retrieval (Genzel et al., 2014; Tse et al., 2011; Wiltgen et al., 2004); during memory-guided behavior, PFC is also thought to be crucial for rapidly updating cognitive information from other brain regions for decision-making. Contextual representations from hippocampus are directly sent to PFC (Navawongse and Eichenbaum, 2013), which is further reciprocally connected to the multiple brain regions. Further, PFC is also important for generalizing representations during consolidation for formation of schemas in hippocampal-dependent tasks (Tse et al., 2011), possibly for building semantic memories (Genzel et al., 2014). These functional roles indicate that PFC activity should be coordinated during sleep hippocampal reactivation. Indeed, it has been found that a decrease in hippocampal-prefrontal functional connections correlated with a decrease of overnight memory consolidation (Genzel et al., 2015a; Genzel et al., 2015b). At the physiological level, SWR-modulated spiking, reactivation of cell assemblies, and coordinated field potentials during sleep SWRs have also been observed in PFC by several studies (Peyrache et al., 2011; Siapas and Wilson, 1998; Sirota et al., 2003; Wierzynski et al., 2009). Recent evidence also demonstrates SWR coordination in hippocampal-prefrontal neurons during sleep that recapitulates coactivity seen in previous behavior (Tang et al., 2017). Supporting this notion, Maingret et al. (2016) found that enhancing coordination between hippocampal SWRs and prefrontal delta waves and spindles during sleep resulted in higher recall performance, providing important causal gain-of-function evidence for the role of sleep SWR-related hippocampal-prefrontal interactions in memory consolidation.

Awake Replay vs. Sleep Replay

Hippocampal replay doesn’t occur exclusively during sleep, but also during SWRs in the awake state, prominently during immobility and consummation. Following pioneering studies that demonstrated hippocampal pairwise reactivation (Kudrimoti et al., 1999) and reverse replay (Foster and Wilson, 2006) during awake SWRs, an increasing number of studies have focused on awake SWRs and associated replay events (reviewed in Buzsaki, 2015; Carr et al., 2011; Colgin, 2016; Foster, 2017; Roumis and Frank, 2015). Awake and sleep hippocampal replay share some common features: replay in both states is associated with SWRs and both consist of sequences of place cell firing related to previous experiences (Ambrose et al., 2016; Davidson et al., 2009; Foster and Wilson, 2006; Lee and Wilson, 2002), and thus both reactivate behavioral experiences. Despite these similarities, a recent study has found that the mechanisms that generate sleep vs. awake ripples during quiescence are different, indicating that these two forms of replay events may have different functions (Yamamoto and Tonegawa, 2017). In fact, previous studies have noted many unique features of awake replay. First, awake SWRs are up-regulated by novelty and reward (Cheng and Frank, 2008; Foster and Wilson, 2006). Second, awake replay is a more structured representation of ongoing experiences than sleep replay (Grosmark and Buzsáki, 2016; Karlsson and Frank, 2009; Roumis and Frank, 2015). Third, sleep replay mostly occurs in a forward direction (Carr et al., 2011), whereas reverse replay is more prevalent in the awake state (Ambrose et al., 2016; Davidson et al., 2009; Diba and Buzsáki, 2007; Foster and Wilson, 2006; Grosmark and Buzsáki, 2016). This difference of replay orders in awake versus sleep states was found to be fundamental after ruling out methodological differences or variability in animals (Wikenheiser and Redish 2013). In addition, similar to awake SWRs, such reverse replay in awake states is also modulated by novelty and reward (Ambrose et al., 2016; Foster and Wilson, 2006); this unique nature of awake replay implies that it could play an evaluation role by linking the reactivated activity to current task demands, such as reward and goals (Ambrose et al., 2016; Foster, 2017). Notably, it was found that disrupting awake SWRs led to selective impairment of working memory-dependent performance (outbound), but not the simple performance dependent on reference memory (inbound) (Jadhav et al., 2012). These findings lead to the hypothesis that blocking awake SWRs may not just interrupt consolidation alone, but can also interrupt the process that brings the reactivated spatial trajectories into the present context, highlighting the possibility that replay may play an additional role in retrieval and/or planning. In favor of this idea, it has been found that reactivated place cell activity during awake SWRs represents future behavioral paths at specific times when memory retrieval is required (Pfeiffer and Foster, 2013; Singer et al., 2013). Another recent study by Wu et al. (2017) has found that hippocampal replay represents paths towards a shock zone following by shock zone avoidance behavior, suggesting that awake replay plays a role in the recall of the fear memory due to its behavioral salience in the task. Interestingly, awake replay not only reflects ongoing experiences, but is also capable of generating novel replay sequences in an order that has never been experienced (Gupta et al., 2010; Pfeiffer and Foster, 2013). Taken together, these findings lead to the hypothesis that the role of awake replay may not be confined to memory consolidation, it may also serve as “bridges” or “explorations” of a cognitive map that enables retrieval and planning in order to guide ongoing behavior (Carr et al., 2011; Foster, 2017).

Broadly, awake and sleep states are dramatically different in many aspects: they have different levels of neuromodulators (Diekelmann and Born, 2010); and behavioral and internal contexts are distinct in these two states (Carr et al., 2011; Roumis and Frank, 2015) as awake states have more constraints built from current sensory inputs and task demands, suggesting that awake and sleep replay should have corresponding differences (also reviewed in Roumis and Frank, 2015). Especially, as noted above, the coordination of cortical delta and spindle oscillations with SWRs is thought to be a key feature of information exchange between the hippocampus and cortical regions during sleep reactivation (Battaglia et al., 2011; Inostroza and Born, 2013; Maingret et al., 2016; Peyrache et al., 2011; Sirota et al., 2003), and it has been proposed that hippocampal replay may reach cortex only during sleep when such global connections are favorable (Genzel and Robertson, 2015). Hippocampal-cortical reactivation should therefore also be expected to show differences for awake and sleep states.

A recent study investigated this very question during learning in a spatial alternation task, and indeed found striking differences in the two forms of reactivation (Tang et al., 2017). It had previously been shown that activity of prefrontal neurons is modulated by hippocampal SWRs in awake state (Jadhav et al., 2016; Wang and Ikemoto, 2016), supporting coordinated reactivation in hippocampal-prefrontal networks (Jadhav et al., 2016). During awake SWRs, PFC neurons with spatial maps overlapping with CA1 place fields are preferentially reactivated (SWR-excited), thus reinstating the co-activity during behavior, whereas PFC neurons that encode immobility are suppressed (SWR-inhibited; Jadhav et al., 2016). Since awake SWRs occur primarily during periods of immobility, this pattern of modulation enables a switch from current behavioral coding to coordinated spatial reactivation when SWRs occur (Figure 1C). Furthermore, when comparing PFC modulation patterns during awake and sleep SWRs, a marked difference was revealed. Individual PFC neurons showed distinct modulation patterns of excitation vs. inhibition during awake vs. sleep SWRs, and a bias towards excitation of neurons during sleep (Tang et al., 2017). Temporal coordination of hippocampal-prefrontal activity was stronger during awake SWRs, and spatial reactivation was also significantly stronger during awake as compared to sleep SWRs (Figure 1D). At the ensemble level, reactivation of synchronized cell assemblies (~ 50–100 ms time scale) across the CA1-PFC network was observed during SWRs, similar to synchronized cell assemblies described within both the PFC (Peyrache et al., 2011) and CA1 networks (van de Ven et al., 2016). Finally, CA1-PFC reactivation was found to be strongest during initial learning, and was especially enhanced for awake SWRs in novel environments (Figure 1E), indicating a role in early learning.

This difference between awake and sleep reactivation is remarkable, given that both processes are thought to reactivate behavioral experiences. What could be the functional roles of coordinated reactivation during awake vs. sleep SWRs? One important observation is that stronger awake reactivation was seen despite lack of coordination between SWRs and cortical delta-spindles events, a prominent feature of sleep reactivation (Tang et al., 2017). During sleep, the nested and coordinated oscillations promote the spread of synchronous activity across large-scale networks, which is well-suited to reactivate the representations of multiple related experiences. Sleep reactivation may thus serve to integrate memories during consolidation, in favor of its hypothesized roles of generalization, schema building, and semantic knowledge (Battaglia et al., 2012; Lewis and Durrant, 2011; Marshall and Born, 2007; Penagos et al., 2017; Roumis and Frank, 2015; Tse et al., 2011). On the other hand, the high-fidelity reactivation of current behavioral experiences in the hippocampal-prefrontal network during awake SWRs is ideally suited for accurate memory storage related to current experiences, as well as for retrieval and planning related to ongoing behaviors.

Theta Sequences

Theta cycles seen during explorative behavior are considered to underlie cognitive chunks of information (Lisman and Redish, 2009; Mizuseki et al., 2009), which organize information processing in wide-spread circuits, including the hippocampus and associated cortical areas. For example, in an experiment with prior exposure to two distinct environments, the firing of place cells during individual theta cycles represented only one environment or the other following an illusory instantaneous switch between environments, suggesting that any relevant circuit computations may be discretized by theta cycles (Jezek et al., 2011). In addition to the sequential activation of place cells at the behavioral time-scale, temporally compressed theta sequences within theta cycles form another tightly coordinated and repetitive temporal relationship (Figure 2). The mechanistic generation of theta sequences is still an active question, with the original assumption that they are a passive extension of independently phase precessing neurons, in contrast to the recent view that that they serve as a distinct unit of patterned hippocampal ensemble activity along with SWR associated replay (Jensen and Lisman, 1996; Lisman and Redish, 2009; Lisman and Jensen, 2013; though for conflicting models, see Chadwick et al., 2015).

Figure 2. Decoded position during a theta sequence.

(A) Example of a hippocampal theta cycle, and corresponding place cell spiking during this theta cycle illustrated and color coded in the raster below.

(B) Schematic of place fields for cells shown in panel A, mapped as a function of their position on the track. Note highly overlapping representations.

(C) Decoded theta sequence from theta cycle and place field spiking in panel A. Animal’s position and choice point are indicated by the dashed white and magenta lines, respectively. Decoded information denotes probability of linear position (y axis) on the track (indicated by heat) over successive sliding windows covering the entirety of the theta cycle (x axis). Note that this particular theta sequence begins directly in front of the animal, and sweeps ahead of the animal through the choice point and onto the future trajectory indicated by the place fields in B.

(D) Schematic of decoded positions as a function of their location during the theta sequence; note their compressed spatial and temporal nature.

The coordinated temporal relationship between spikes from multiple hippocampal cells during theta sequences implicates them as a possible means of synaptic strengthening between units to encode experience relevant to behavior along with awake SWR sequences (Foster and Wilson, 2007). These sequences are not merely reflections of ongoing sensory experience; when sensory cues such as animal head direction and visual landmarks are decoupled from movement direction, as in the case of backward travel, theta sequences reverse their order to predict the ongoing reverse trajectory with similar relative timing between spikes (Cei et al., 2014). When sensory cues are intact but an animal has yet to experience a task, such as the first lap of a novel linear track, there is an absence of distinct theta sequences, though single trial phase precession is prominent throughout, suggesting experience is necessary for theta sequence formation (Feng et al., 2015). Statistically, the sequence of place cell spiking seen during theta cycles is patterned more precisely than expected if generated by phase precession alone (Dragoi and Buzsáki, 2006; Foster and Wilson, 2007; Itskov et al., 2008; Maurer et al., 2006). Additionally, theta sequences represent the environment in a segmented manner, which could not be explained by the spatial representations, even after taking non-uniformity of place fields into account (Gupta et al., 2012). Interestingly, theta sequences represent the current spatial environment, not just as a uniform spatial representation, but occasionally sweeping ahead or emerging from behind the animal and clustering around behaviorally relevant elements of a task, such as choice points and reward locations (Gupta et al., 2012; Johnson et al., 2007; Wikenheiser and Redish, 2015a). Different forms of sequences within theta cycles are also seen to be associated with low and high gamma periods (Zheng et al., 2016). Further, theta sequences within individual theta cycles can represent information related to the animal’s current location as well as non-local information such as upcoming goals (Wikenheiser and Redish, 2015b).

Do theta sequences indeed form cognitively relevant chunks of information, and what mnemonic roles do they play? To establish a broader context for the roles of theta sequences in cognition and their meaning to downstream areas, unambiguous periods of distinct cognitive behavior are needed. One such cognitive behavior that has been shown to involve the PFC is that of deliberation before making a choice, which generally is composed of the identification of viable options and their subsequent evaluation before executing a decision. Rodents have been known to have their own version of deliberative decision-making behavior first described by Tolman in the 1940s in which animals pause at a decision point and look back and forth prior to making a decision, a behavior deemed vicarious trial and error (VTE) (Tolman, 1948). In a pioneering study, theta sequences have been shown to be a prominent signature of these VTE events, sweeping ahead of the animal at a choice point serially over potential options (Johnson and Redish, 2007). Given that PFC is crucial for deliberative decision-making, it may play an important role in VTE by interacting with hippocampal theta sequences for cognitive relevance (Colgin, 2013; Redish, 2016).

There is indeed a wealth of evidence that the hippocampus and prefrontal cortex become coupled via theta band synchrony in spatial memory tasks (Battaglia et al., 2011; Shin and Jadhav, 2016; Spellman et al., 2015). Results suggest both phase locking of prefrontal unit firing to hippocampal theta oscillations and theta band coherence between the two structures as possible substrates for the coordination required for retrieval and memory guided decision making (Jones and Wilson, 2005; Siapas et al., 2005). In support of behavioral roles for theta band synchrony, it has been shown that theta coherence peaks at the choice point most strongly after acquisition of novel rules on a Y-maze task, with prefrontal cell assemblies additionally shifting their firing to the hippocampal theta trough over acquisition, indicating a role for theta phase (Benchenane et al., 2010). Deficiencies in this synchrony and spatial working memory are seen in animal models of schizophrenia (Sigurdsson et al., 2010). Synchrony between the two structures may also play an active role in shaping decision making behavior. Prefrontal neurons that modulate their firing with specific behavioral variables such as running direction alternate between theta entrained and non-theta entrained firing corresponding to these behavioral epochs (Hyman et al., 2005). This theta entrainment may also be a reflection of decision confidence or certainty, with prefrontal units preferentially phase locking to hippocampal theta oscillations during correct trials in the sample and test phases of a delayed non-match to sample (DNMS) task without changing their firing rates (Hyman et al., 2010).

Despite this evidence describing hippocampal-prefrontal coordination during theta oscillations, the correspondence between theta sequence representations and related prefrontal representations in support of cognitive processing is not yet clear. During VTE behaviors that are prevalent during initial task learning (Papale et al., 2016), theta sequence representations sweep ahead of the animal representing prospective choices (also reviewed in (Redish, 2016; Wikenheiser and Redish, 2015a)). It stands to reason that prefrontal representations should also represent these choices within theta cycles in coordination with hippocampal sequence representation (Figure 3A), with the directionality of such interactions also an open question. As animals learn the task and transition to more ballistic, high-certainty trajectories, increases in theta coherence and prefrontal phase-locking is also observed at choice points. Theta sequences in this case sweep along the direction of intended choice (Papale et al., 2016), and have also been suggested to represent upcoming goals (Wikenheiser and Redish, 2015b). We posit that prefrontal representations should align with these local and remote theta sequence representations on a cycle-to-cycle basis, sub serving the cognitive processes of future planning and intended actions (Figure 3). How these theta-cycle associated interactions relate to SWR replay interactions during learning and memory-guided behavior is one of the major questions that needs to be addressed by future studies.

Figure 3. Schematic illustrating proposed theta sequence decoding and corresponding activity in PFC over learning.

(A) Early in task acquisition during periods of high uncertainty, theta sequences are seen to decode upcoming paths at the choice point in an alternating manner during VTE behavior. Note proximal corresponding representation in PFC.

(B) Theta sequence corresponding to panel A as a function of linear position and theta cycle phase, showing local decoding sweeping ahead of the animal in both leftward and rightward representations in individual theta cycles.

(C) When a task is learned, theta sequences sweep ahead to behavioral locations indicating the upcoming choice, with PFC representing this behavioral space.

(D) Corresponding theta sequence decode in CA1.

(E) Schematic theta sequence decoding during performance of a well-learned task, showing highly non-local decoding in CA1 sweeping from an animal’s current location to the goal arm. Note PFC representing this remote goal as well.

(F) Schematic of corresponding theta sequence decoding in a well-trained task showing highly non- local representation of upcoming goal arm, as in (Wikenheiser and Redish 2015b).

It is also worthwhile to point out that theta oscillations are not limited to awake and active exploration, much like awake vs. sleep associated SWRs. Prominent theta oscillations of a lower characteristic frequency than awake theta occur during the REM phases of sleep (Montgomery et al., 2008). Whether theta sequences also occur during REM sleep is largely unexplored, with some evidence showing that temporally sequenced CA1 activation patterns from active behavior are correlated with similar reactivation patterns seen during REM sleep (Louie and Wilson, 2001). Changes in theta coherence between the hippocampus, the amygdala, and the prefrontal cortex in REM sleep correlate with changes in fear conditioning, extending the roles of sleep reactivation to memory consolidation in hippocampally associated cortical areas (Popa et al., 2010). CA1 firing rate decreases in REM are concurrent with increases in neural synchrony measures during non-REM sleep, suggesting a mechanism for the strengthening and refinement of relevant synaptic connections, combined with the pruning of non-relevant connections (Grosmark et al., 2012; Miyawaki and Diba, 2016). The alternation of REM and non-REM episodes in sleep, and the possible alternation of characteristic behavioral sequences during these sleep stages, are compelling mechanisms for refining, regulating, and strengthening behaviorally relevant synaptic weights offline through both theta and replay sequences (Diekelmann and Born, 2010). Ultimately, closed-loop, real-time loss- and gain- of function experiments are needed to tease apart the causal relationships between the two kinds of sequences during both behavior and sleep, and their relevance to other brain areas.

The Role of Replay and Theta Sequences in Spatial Memory

Spatial memory tasks provide an important opportunity to understand the role of coordinated hippocampal-prefrontal representations during replay and theta sequences. Yet spatial learning is not a singular process. Initially, the brain needs to encode task-relevant information and form representations of new experiences, and the newly encoded information is then stabilized through memory consolidation. Later on, relevant memories need to be retrieved based on current context and task demands. Therefore, the role of hippocampal-prefrontal coordination may be different as learning progresses.

While various behavioral tasks have been used to investigate spatial learning, it should be noted that the dependence of the PFC during spatial learning varies based on the task engagement and demands. For example, for contextual fear conditioning tasks, the engagement of PFC during initial learning is very low (Kitamura et al., 2017). In contrast, a recent study involving a spatial reversal-learning task with rule switching found that inactivation of PFC didn’t affect learning of simple spatial alternations but impaired switching between rules (Guise and Shapiro, 2017). In addition, disrupting hippocampal-prefrontal functional connections impairs spatial alternation with delay periods, tasks that require working memory (Floresco et al., 1997; Spellman et al., 2015; Wang and Cai, 2006). Consistent with these findings, it has been proposed that PFC plays a critical role in using context, such as a set of task rules, to guide the retrieval of memory in the hippocampus (Eichenbaum, 2017a; Eichenbaum, 2017b; Preston and Eichenbaum, 2013). According to this hypothesis, in some tasks (such as contextual fear conditioning), the newly formed memory is particularly strong and has little memory competition, and PFC may not be crucial during initial learning here (Kitamura et al., 2017). However, PFC becomes active in learning tasks that require reconciling memory conflicts, such as working memory tasks (Lara and Wallis, 2015; Postle, 2006) and rule-switching tasks (Guise and Shapiro, 2017).

If hippocampal-prefrontal interactions play a role in reconciling competing memory traces as proposed (Eichenbaum, 2017a; Eichenbaum, 2017b; Preston and Eichenbaum, 2013), such context-appropriate retrieval must be supported by the formation of associations between spatial representations and outcomes or rules through information flow between these two regions (Figure 4A). Subsequently, these context-appropriate associations can become accessible to the animal through different physiological mechanisms for accurate retrieval during task performance (Figure 4B). Coordinated hippocampal-prefrontal reactivation during SWRs seems ideally suited to form such associations and the development of stable representations of task selectivity that support learning. There is evidence that disrupting either awake (Roux et al., 2017) or sleep SWRs (van de Ven et al., 2016) (but see Kovács et al., 2016) affects place field stability, suggesting a role of SWRs in the stabilization of hippocampal spatial map. The finding of enhanced hippocampal-prefrontal reactivation during initial learning in a novel spatial alternation task (Tang et al., 2017) (Figure 1F) raises the intriguing possibility that this “representational stabilization” role might extend to the PFC. In particular, the prevalence of awake SWRs at reward locations indicates a potential role in linking spatial experience with rewards and outcomes, presumably through concomitantly reactivating spatial-selective ensembles in the hippocampus and “rule”-selective ensembles in PFC (Figure 4A). Indeed, disrupting awake SWRs during learning of a spatial alternation task results in an increase of VTE behaviors, indicating high uncertainty and a shift to theta sequence representations during deliberative decision making (Papale et al., 2016). One thing to note is that synchronization of cell assemblies in hippocampal-prefrontal networks, rather than sequential activity, may better describe these joint representations, as timing information for sequences is not seen to be preserved across the CA1-PFC network during reactivation (Tang et al., 2017) (but also see Euston et al., 2007, where timing information is preserved during PFC reactivation in a well-learned task). This leads us to an interesting question about how hippocampal sequence information is transformed, or functionally coordinated, with a synchronized cell assembly in PFC, possibly for both replay and theta sequences, which needs to be addressed in future studies. Together, these findings suggest that hippocampal-prefrontal reactivation during awake SWRs may support spatial learning by developing task-selective patterns and associations in the hippocampus and PFC, and these stable representations may subsequently be accessed through various physiological mechanisms to support accurate memory performance. While sleep SWRs also undoubtedly play a role in this learning process, coordinated reactivation during sleep may also play a unique role in integration of memories of related experiences during consolidation (Figure 4C, D), as discussed above.

Figure 4. The functional role of hippocampal-prefrontal interactions in spatial memory.

(A) During initial learning, the animal gradually learns the task by updating associations among rewards, actions and outcomes. The place fields of hippocampal cells (red) overlap with the spatial map of one prefrontal cell (orange), but not the other (magenta). The hippocampal and prefrontal cells with overlapping spatial maps are co-activated as the animal traverses the portion of the W-maze. During SWRs, the reactivation of these cells recapitulates the coactivity observed during exploration (dashed box). This coordinated activity in the hippocampal-prefrontal network is thus repeatedly reactivated during SWRs, which provides a potential learning mechanism to form associations between task-selective neuronal ensembles and to stabilize memory traces in the two regions. Disrupting SWRs during learning leads to an increase in VTE behaviors, indicating a shift to theta associated representations for deliberating upcoming choices.

(B) After learning, these stable task-specific representations in the hippocampal-prefrontal network become accessible to the animal during task performance (exploitation), which can be retrieved during SWRs or during theta oscillations to guide correct choices.

(C) In a novel environment, new associations between hippocampal and prefrontal ensembles are formed and reactivated during SWRs.

(D) During subsequent sleep, especially slow-wave sleep, related new and old memories are reactivated and integrated during consolidation.

What mechanisms could allow access to task-relevant representations to support accurate performance during memory-guided behavior? The physiological mechanisms that enable access to related functional representations across regions (in hippocampal-prefrontal networks in this context) during behavioral performance still remain a focus of intense investigations, and evidence suggests that both awake SWR replay and theta associated interactions could play this role. While the role of SWRs during initial learning and novelty is emphasized in some studies, others have revealed that awake SWRs are also prevalent in well-learned tasks, with replay events reflecting future or novel paths (Diba and Buzsáki, 2007; Dupret et al., 2010; Gupta et al., 2010; Karlsson and Frank, 2009; Papale et al., 2016; Pfeiffer and Foster, 2013). In addition, when such tasks involve rule or goal changes, there is enhanced reactivation of goal-related hippocampal assemblies during awake SWRs (Dupret et al., 2010; Pfeiffer and Foster, 2013). This suggests that during the exploitation phase of spatial learning after rule learning, coordinated hippocampal-prefrontal replay during awake SWRs may reinstate activity in the two regions through similar mechanisms used to build the cross-regional associations (Figure 4B). Awake replay may thus enable prospective route-priming to support memory-guided decisions, in addition to its retrospective role during memory consolidation, and may also serve as a mechanism for continuous evaluation of changes in task demands to maintain flexibility and to implement adaptive behavioral strategies. In another view, coordinated oscillations are hypothesized to play a major role as the underlying mechanism (Eichenbaum, 2017b; Shin and Jadhav, 2016). In particular, as we noted in the previous section, coherence and phase-locked spiking during theta oscillations has been proposed to support memory retrieval in hippocampal-prefrontal networks in spatial tasks (Benchenane et al., 2010; Jones and Wilson, 2005). The exact role of theta sequences in these interactions has yet to be investigated, but given the evidence for complex interplay of SWR replay and theta sequences in memory-guided behaviors (Papale et al., 2016), this raises interesting questions about prefrontal activity during theta sequences and its putative role in retrieval and guiding future actions.

Disruption and Manipulation Experiments

Findings linking replay and theta sequences to memory processes provide an essential foundation for our understanding, though many of these studies are correlative or observational in nature. Solidifying the functions and relevance of these sequences and their causal roles in memory and cognition requires precise and selective experimental manipulation targeting the hippocampus and closely associated regions using real-time detection and closed-loop feedback inactivation. Both careful elimination or interruption of these network states, and manipulation of sequence content or readout by other areas, will ultimately clarify their roles.

The first experiments to ascribe a causal role for SWRs selectively detected and eliminated SWRs during sleep immediately following the learning of hippocampally dependent spatial memory tasks (Ego-Stengel and Wilson, 2010; Girardeau et al., 2009). By stimulating the ventral hippocampal commissure (vHC) after SWR detection, transient inactivation of the entire hippocampal formation ensured interruption of SWR-related activity and its propagation (Zugaro et al., 2005). These disruption experiments revealed learning impairments as a result of SWR disruption, thus establishing a role of sleep replay in memory consolidation. A similar approach was used for the first demonstration of the causal role of awake SWRs in spatial memory tasks (Jadhav et al., 2012). Selective disruption of awake SWRs and associated replay activity during learning of a spatial alternation task was achieved using real-time detection and closed-loop triggering of vHC stimulation. This lead to a selective deficit in learning the spatial working memory component of the task which required integration across space and time. Place field activity and sleep reactivation was unaffected by this perturbation, indicating that memory deficits were solely due to loss of awake replay activity.

These initial studies establish the behavioral relevance of awake and sleep hippocampal SWR activity, providing the foundation to causally probe the next set of questions: how downstream targets read-out or influence the associated replay information, the physiological effects of disrupting replay activity, and selectively eliminating specific replay sequences to probe the effect on memory. For sleep SWRs, experiments targeting associated regions by timing manipulations with the onset of SWRs have already revealed interesting features. In one study, the association of hippocampal SWRs and prefrontal delta-spindle oscillation complexes was enhanced by electrically stimulating cortex during SWRs (Maingret et al., 2016). This gain-of-function manipulation led to an improvement in memory in an object-displacement task, suggesting an enhancement of memory consolidation by improving the refinement of mPFC network connections required for memory performance. In another experiment, the SWR triggered stimulation of locus coeruleus (LC) in post task sleep similarly caused the blockade of ripple associated cortical spindles, leading to deficiencies in reference memory accuracy and learning speed (Novitskaya et al., 2016). Further, stimulating selective dopaminergic release through ventral tegmental area (VTA) stimulation during sleep SWR reactivation of specific place cells led to explicit memory creation for the associated place field location (McNamara et. al., 2014).

Manipulating activity in downstream and upstream regions during SWRs is thus a powerful tool, although this has yet to be fully utilized for awake SWRs. Manipulating prefrontal activity at different learning and task stages will be a particularly effective tool to test the myriad hypotheses regarding the cognitive relevance of awake reactivation. Examining the physiological effects of SWR disruption has also been very revealing. In a task which featured new goal locations to be learned daily, awake SWR silencing of CA1 pyramidal cells at goal locations impaired stabilization of place cell representations (Roux et al., 2017). Sleep SWR disruption following exploration of a novel environment was not seen to affect place cell stability (Kovács et al., 2016), although another study reported that optogenetic silencing of SWRs impairs stabilization of novel environment representation in the hippocampus (van de Ven et al., 2016). As we noted in previous sections, given that coordinated hippocampal-prefrontal reactivation is enhanced during initial learning, it is possible that the stabilization role of SWRs may not just be limited to the hippocampus, but also extend to PFC. Future studies that combine closed-loop inactivation with multisite recordings may be able to test these hypotheses. Finally, detecting replay sequence information in real-time and selectively perturbing specific sequences and associated prefrontal activity has the potential to clearly prove the necessity of sequence information in memory processes, and the requisite tools are currently being developed (Deng et al., 2016; Deng et al., 2015; Kloosterman et al., 2014).

In comparison to SWRs and replay, selective disruption of theta sequences and associated processing in other regions has not been clearly explored. However, evidence from manipulation of theta oscillations, theta phase-locked spiking, and indirect manipulation of theta sequences, underlines the role of this network pattern in memory processes. The theta rhythm has long been known to also play a significant role in spatial task performance, with lesions to the medial septum (MS) impairing spatial memory due to a loss of theta oscillations and impairment in spatial representations in the medial temporal lobe (Brandon et al., 2011; Koenig et al., 2011; Mitchell et al., 1982; Mizumori et al., 1990; Shen et al., 1996; Wang et al., 2015). Reversible disruption of theta by medial septum inactivation has also been shown to disrupt putative internally generated theta sequences and firing fields formed during stationary running in a treadmill task without affecting replay sequences, with a corresponding impairment in learning (Pastalkova et al., 2008; Wang et al., 2016). More temporally and spatially specific manipulations of the theta rhythm have been attempted, both in awake and asleep states. In one particular study, REM specific silencing of MS eliminated theta and impaired fear conditioned place memory, as well as novel object place recognition performance (Boyce et al., 2016). In a study designed to illustrate the importance of theta-phase specific computations, hippocampal activity was perturbed at specific phases of theta using real-time detection and optogenetic inactivation. Inhibition of hippocampal activity at the peak of theta enhanced performance during the encoding segment of a task, while inhibition of hippocampus at the theta trough enhanced performance during the retrieval segment of a task (Siegle and Wilson, 2014). The specific effects on encoding and retrieval point to the possibility that phase-specific prefrontal interactions may be important for sub serving these functions. Indeed, in a spatial alternation task, optogenetic inhibition of ventral hippocampal projections to PFC impaired encoding of spatial cues in mice, but did not affect retrieval and maintenance of working memory (Spellman et al., 2015). Future experiments need to combine these approaches of real-time detection and targeting of specific connections to reveal the circuit basis of cognitive functions.

Summary

Results from physiology and disruption experiments have begun to illuminate potential roles for theta and replay sequences, with many overlapping features. Could these patterns share fundamental building blocks or serve shared memory mechanisms in a complementary manner? Possible support for this is seen in awake sharp wave ripple disruption, which has been shown to cause an increase in vicarious trial and error behavior characteristic of theta sequences (Papale et al., 2016). Many of these questions can only be addressed by improving our understanding of prefrontal interactions during these network patterns at different learning and task phases. Further, in addition to closed-loop inactivation of the hippocampus, targeting of prefrontal activity or specific connections is critically required. Closed-loop feedback inactivation experiments that specifically target hippocampal-prefrontal coordination during SWRs and theta oscillations will allow us to dissect their distinct functions and clarify the inter-relationships between the two network patterns. This also has the potential to shed new light on the pathophysiology of disorders related to hippocampal and prefrontal activity, including memory disorders and schizophrenia.