Sharp-wave ripples as a signature of hippocampal-prefrontal reactivation for memory during sleep and waking states

Abstract

It is widely believed that memories that are encoded and retrieved during waking behavior are consolidated during sleep. Recent studies on the interactions between the hippocampus and the prefrontal cortex have greatly advanced our understanding of the physiological bases of these memory processes. Although hippocampal-prefrontal network activity differs in many aspects during waking and sleep states, here we review evidence that hippocampal sharp-wave ripples (SWRs) emerge as a common neurophysiological pattern in both states, facilitating communication between these two regions via coordinated reactivation of stored memory information. We further consider whether sleep and awake reactivation mediate similar memory processes or have different mnemonic functions, and the mechanistic role of this cross-regional dialogue in learning and memory. Finally, we provide an integrated view of how these two forms of reactivation might work together to support spatial learning and memory.

1. Introduction

Our memories are a record of life experiences, and give us the great capability to learn and adapt to ongoing demands. The memory system of our brain retains this enduring information through three major processes. First, new experiences are encoded to form an internal representation. Then, this initial representation is consolidated for long-term storage; and later on, when future experiences are associated with or require this memory, the stored internal representations can be retrieved to guide behavior and updated for generalization in novel situations. Decades of research have pointed towards sleep as a pivotal state for memory consolidation (Frankland and Bontempi, 2005; Diekelmann and Born, 2010; Rasch and Born, 2013; Genzel et al., 2014; Sara, 2017). In sleep states, new memories are thought to be gradually strengthened and integrated into preexisting representations (Frankland and Bontempi, 2005; Diekelmann and Born, 2010; Rasch and Born, 2013; Genzel et al., 2014). In waking states, encoding and retrieval of memories take place most effectively, supporting rapid behavioral performance (Diekelmann and Born, 2010; Ackermann and Rasch, 2014; Roumis and Frank, 2015; Feld and Born, 2017). As a result, both sleep and waking states together support memory processes (Roumis and Frank, 2015; Feld and Born, 2017; Mizuseki and Miyawaki, 2017).

However, what are the neural substrates that give rise to these proposed functions for sleep and waking states? A major breakthrough in understanding the physiological basis of memory formation came from the discovery of memory reactivation during fast oscillations in the hippocampus, called sharp-wave ripples (SWRs or ripples; also reviewed by Buzsáki, 2015). Hippocampal neural activity that represents previous behavioral experiences has been found to be reactivated during SWRs, suggestive of an important mechanism contributing to memory. SWRs primarily occur during non-rapid eye movement (NREM) sleep (sleep SWRs), and in the awake state during consummation and immobility (awake SWRs). Classically, SWRs have drawn special attention in sleep research, as sleep SWRs are intricately interlinked with various cortical oscillations, which bear the potential to synchronize activity across different brain areas and redistribute memories for updating and for long-term storage (Battaglia et al., 2011; Rasch and Born, 2013; Genzel et al., 2014; Buzsáki, 2015). Among all the brain regions interacting with the hippocampus, the prefrontal cortex (PFC) has been identified by many imaging and inactivation studies as playing a crucial role in processing long-term memory (Frankland and Bontempi, 2005; Tse et al., 2011; Kitamura et al., 2017). Interactions between the hippocampus and PFC therefore have been the focus of much investigation, and in particular, the link between hippocampal-prefrontal reactivation during sleep SWRs and memory consolidation is of interest (Frankland and Bontempi, 2005; Diekelmann and Born, 2010; Battaglia et al., 2011). Notably, the prefrontal cortex is also known to be important for working memory, decision making and attentional selection of task-relevant information (Miller and Cohen, 2001; Euston et al., 2012; Preston and Eichenbaum, 2013; Eichenbaum, 2017). These functions of PFC have been highlighted by recent studies of awake SWRs and associated hippocampal-prefrontal reactivation (Carr et al., 2011; Jadhav et al., 2012; Roumis and Frank, 2015; Jadhav et al., 2016; Shin and Jadhav, 2016; Foster, 2017; Tang et al., 2017). Evidence from such studies has revealed that awake SWRs, like sleep SWRs, involve memory reactivation in the hippocampal-prefrontal network, but this reactivation has been linked to memory retrieval and behavioral planning.

How do we reconcile and integrate these observations on hippocampal-prefrontal reactivation in light of the well-known functional roles of sleep and waking states in memory? In this article, we review recent data on SWR reactivation in the hippocampal-prefrontal networks during both sleep and waking states. We consider how information can be communicated between the hippocampus and PFC during SWRs in these two states and how this dialogue contributes to learning and memory. Finally, we aim to provide an integrated view of different mnemonic roles of awake and sleep reactivation, which can serve to further our understanding of memory processing.

2. Communication between hippocampus and prefrontal cortex during sleep

The representations of new experiences must be consolidated into long-lasting memory traces in order to be remembered for guiding future behaviors. Over the past decades, the “two-stage” hypothesis has risen to be a dominant model of consolidation theory (Marr, 1971; Buzsáki, 1989; Sutherland and McNaughton, 2000; Diekelmann and Born, 2010). Within this framework, information is rapidly encoded in the hippocampus and associated regions during behavior. Later on, in the sleep state, fast oscillations called sharp-wave ripples (SWRs; 150-250 Hz), coincident with synchronized neuronal population activity, predominate in the hippocampus resulting in plasticity in the hippocampal output regions, such as the PFC. This process is thought to enable transfer of hippocampal memory information to the PFC for long-term storage, or to strengthen distributed representations of the initial memory trace (i.e., system consolidation; Buzsáki, 1989, 1998; Frankland and Bontempi, 2005; Diekelmann and Born, 2010). Ever since this proposition, much attention has been focused in determining the role of sleep and the physiological basis of hippocampal-prefrontal communication in memory consolidation.

The role of two major stages of mammalian sleep, rapid eye movement (REM) sleep and NREM sleep, in memory is an active area of investigation (Diekelmann and Born, 2010; Rasch and Born, 2013; Genzel et al., 2014; Sara, 2017). In these two different stages, brain activity, neuromodulator levels, and hippocampal-prefrontal interaction patterns are quite distinct. Here, we summarize current knowledge about the influence of hippocampal-prefrontal interactions during different sleep stages on memory consolidation, with a particular focus on physiological mechanisms underlying reactivation in the hippocampal-prefrontal network.

2.1. NREM sleep

One of the noticeable differences between NREM and REM sleep is that hippocampal SWRs, which are thought to be crucial for memory consolidation (Buzsáki, 2015), occur predominantly in NREM stage (i.e., sleep SWRs), but rarely in REM stage (Fig. 1A; Kudrimoti et al., 1999; Eschenko et al., 2008; Tang et al., 2017). Since the early days of sleep research, a large number of studies have investigated the role of NREM sleep in memory consolidation. An important finding was the discovery of hippocampal replay during sleep SWRs (Wilson and McNaughton, 1994; Kudrimoti et al., 1999; Lee and Wilson, 2002). It is known that hippocampal place cells show selective firing at specific locations in an environment (i.e., place fields), and when an animal runs through the environment, different place cells fire in a sequence along the animal’s behavioral trajectory. Such sequential firing patterns of place cells are found to be reactivated coinciding with SWRs during subsequent NREM sleep (“replay” or “reactivation”). This repeated reactivation of new memories during SWRs is well-suited as a candidate physiological mechanism that stabilizes and consolidates these memories (Marr, 1971; Buzsáki, 1989, 1998). Consistent with this idea, it has been shown that SWR occurrence rates and replay events increase during the first few hours of sleep following a training session (Kudrimoti et al., 1999; Eschenko et al., 2008), and is correlated with subsequent memory performance (Ramadan et al., 2009; Dupret et al., 2010). Selectively disrupting SWRs during NREM sleep results in performance impairment in hippocampal-dependent memory tasks (Girardeau et al., 2009; Ego-Stengel and Wilson, 2010), suggesting that sleep SWRs have a causal role in memory consolidation.

Figure 1. Hippocampal-prefrontal coordination during sleep SWRs.

(A) Distinct hippocampal-prefrontal activity patterns during NREM and REM sleep. Simultaneously recoded spiking activity from the hippocampal area CA1(black ticks) and PFC (blue ticks) is shown along with corresponding CA1 LFP traces (red line: theta-delta ratio; green line, ripple-filter LFP). REM sleep (red shading) is accompanied by enhanced theta (6-12 Hz) power (characterized by theta-delta ratio), and theta-modulated firing of CA1 pyramidal cells (horizontal stripes in the raster plot). In NREM sleep (grey shading), large-amplitude ripples (green line) were observed, coincident with population synchrony of CA1 spikes (note the vertical stripes in the raster plot). Top panel shows an expanded view of a representative CA1 synchronous reactivation event during sleep SWRs (orange shading).

(B) Hippocampal-prefrontal oscillatory coupling during NREM sleep. Example traces of simultaneously acquired LFPs from two CA1 (top) and two PFC tetrodes (bottom) in NREM stage are shown. Note that CA1 ripples (green) are closely followed by delta waves (1-4 Hz; orange) and spindles (12-18 Hz; red).

(C-E) Coordinated hippocampal-prefrontal reactivation during sleep SWRs. The coordinated reactivation of hippocampal-prefrontal ensembles can be measured using reactivation strength (Peyrache et al., 2009; Peyrache et al., 2010). This method uses principle components analysis (PCA) to detect neuronal ensembles, within which spiking activity is strongly co-activated among individual neurons of the ensemble during behavioral tasks. The contributions of single neurons to the co-activation are measured as their corresponding PC weights (D; color-coded to highlight neurons with high weights). The reactivation strength (RS) measures the correlation between the activity patterns during behavior characterized by PC weights and those during SWRs. If this synchronous activation of the detected ensembles during behavior is also strongly reactivated during sleep periods, the RS will have a high value. The RS of a representative CA1-PFC ensemble during sleep is shown in C. Notably, the RS is higher during post-task sleep (bottom) than pre-task sleep (top), suggesting enhanced reactivation after experience and learning. Also, the increase of RS during post-task sleep is prominent during NREM sleep (grey shadings), but not REM sleep (red shadings), suggesting stronger CA1-PFC reactivation in NREM stage. (E) The peaks of RS (top) represent synchronous events of CA1-PFC ensembles (bottom), which occurred predominantly during SWRs (same color code used in D).

Panels adapted from Tang et al. (2017).

Besides the prevalence of SWRs, NREM sleep is also characterized by other electrical field potential rhythms in neocortex: namely slow oscillations (SO; < 1 Hz), delta waves (1-4 Hz) and sleep spindles (12-18 Hz) generated in the thalamo-cortical network (Fig. 1B). These network rhythms are also known to play a role in memory consolidation. For example, previous studies have found that spindles can trigger short-term and long-term potentiation in cortical pyramidal cells (Contreras et al., 1996); and boosting either spindles (Lustenberger et al., 2016) or SO (Marshall et al., 2006) during NREM sleep improves memory consolidation. Notably, these different rhythms and SWRs in the hippocampal-prefrontal network show precise temporal relationships and appear to interact with each other to support memory consolidation (also reviewed in Inostroza and Born, 2013). In the NREM stage, cells in the neocortex show active “UP” (depolarized) and silent “DOWN” (hyperpolarized) states corresponding to the cortical SO. Hippocampal SWRs preferentially occur during cortical UP states, especially at the transitions between DOWN and UP states (Sirota et al., 2003; Battaglia et al., 2004). Similarly, sleep spindles are often enhanced during the transitions to cortical UP states and closely follow delta waves (Fig. 1B; Peyrache et al., 2011; Phillips et al., 2012; Genzel et al., 2014; Maingret et al., 2016). The fine temporal relationship of these rhythms is further demonstrated by the oscillatory coupling between hippocampal ripples and cortical SO/delta/spindles during NREM sleep in both rodents (Siapas and Wilson, 1998; Sirota et al., 2003; Wierzynski et al., 2009; Battaglia et al., 2011; Peyrache et al., 2011; Maingret et al., 2016; Tang et al., 2017) and humans (Clemens et al., 2007; Staresina et al., 2015). For example, cortical delta waves and spindles generally follow hippocampal ripples (Fig. 1B; Siapas and Wilson, 1998; Buzsáki, 2015; Maingret et al., 2016). In some cases, hippocampal ripples are selectively locked to the troughs of cortical spindles during NREM sleep (Siapas and Wilson, 1998; Clemens et al., 2011; Inostroza and Born, 2013; Staresina et al., 2015). Interestingly, a new study reports that the ripple oscillations exist in cortical regions as well, especially in the PFC and parietal cortex. These cortical ripples are coupled with hippocampal ripples, which are further nested with cortical delta/spindle/SO activity during NREM sleep (Khodagholy et al., 2017), although the mechanisms of generating these cortical ripples are currently unclear.

Some dominant memory models posit that such oscillatory coupling permits the reactivated memory information to spread from the hippocampus to cortex, resulting in memory consolidation (Buzsáki, 1989, 1998; Diekelmann and Born, 2010; Carr et al., 2011; Genzel et al., 2014; Buzsáki, 2015; Shin and Jadhav, 2016). In addition, NREM sleep has a unique neuromodulatory environment enabling memory consolidation, as the low levels of cortisol (or glucocorticoids) and acetylcholine can facilitate communication across brain regions (Gais and Born, 2004; Kelemen et al., 2014; Mitra et al., 2016). Consistent with this proposal, coupling between hippocampal ripples and spindles, as well as cortical ripples, increases during NREM sleep following learning (Molle et al., 2009; Khodagholy et al., 2017). Furthermore, there is recent casual evidence that disrupting PV interneurons phase-locked to ripples and spindles in either the hippocampus or PFC results in learning impairments and elimination of learning-induced ripple-spindle coupling between these two regions (Ognjanovski et al., 2017; Xia et al., 2017). In a critical gain-of-function study, boosting ripple-spindle coupling by triggering prefrontal spindle activity contingent on hippocampal ripples has been shown to improve spatial memory performance, even though the coupled events only contributed to a very small percentage of overall ripple and spindle events (Maingret et al., 2016).

This oscillatory coupling thus presumably supports memory reactivation in the hippocampal-prefrontal network. Indeed, PFC neurons are modulated by SWRs exclusively in NREM sleep but not in REM sleep (Wierzynski et al., 2009), and reactivation of behavioral representations within PFC was observed when sleep spindles follow hippocampal SWRs (Peyrache et al., 2009). A recent study elaborated and extended this observation to coordinated reactivation in the hippocampal-prefrontal network during SWRs (Tang et al., 2017). Using simultaneously recorded neuronal spiking activity from the hippocampal and prefrontal ensembles, this study found that hippocampal-prefrontal ensembles that were co-activated during a hippocampal-dependent spatial task were subsequently reactivated during sleep SWRs (Fig. 1C-E), which may serve to incorporate hippocampal memory information into prefrontal representations (Tang et al., 2017).

In summary, there is a vast amount of experimental evidence supporting the role of NREM sleep in memory consolidation, which essentially involves communication between the hippocampus and PFC. Reactivation in the hippocampal-prefrontal network during NREM sleep seems to preferentially occur during sleep SWRs, which are coupled with various cortical oscillatory patterns. The temporally coordinated oscillations between the hippocampus and PFC during NREM sleep thus provide ideal conditions for information exchange between these two regions via reactivation of neural patterns related to memory (Battaglia et al., 2011; Genzel et al., 2014; Colgin, 2016).

2.2. REM sleep and NREM-REM sequences

REM sleep, originally named paradoxical sleep owing to similar properties as waking stage such as the acetylcholine level and strong theta oscillations (Fig. 1A; 8-12 Hz), however has several salient differences from waking states. REM sleep has lower levels of norepinephrine and serotonin compared to waking, and it is also thought that information is routed differently in REM sleep and waking states (Mizuseki and Miyawaki, 2017); for example, hippocampal area CA1 receives different inputs from hippocampal CA3 and entorhinal cortex (EC) during these two stages: CA1 pyramidal cells respond most effectively to CA3 input during waking, but preferentially respond to input from EC layer 3 during REM sleep (Schomburg et al., 2014; Fernandez-Ruiz et al., 2017). In addition, many oscillatory patterns, including theta, show reduced coherence between hippocampal-prefrontal and thalamo-cortical circuits during REM sleep than waking and NREM sleep (Cantero et al., 2003; Axmacher et al., 2008), suggesting that hippocampal and prefrontal regions become more decoupled during REM sleep (Diekelmann and Born, 2010). Indeed, reactivation of memory information in hippocampal-cortical circuits has been observed almost exclusively during NREM sleep as we discussed above, but rarely during REM sleep (Fig. 1C; Kudrimoti et al., 1999; Peyrache et al., 2009; Wierzynski et al., 2009; Mizuseki et al., 2011; Genzel et al., 2015; Tang et al., 2017; but see Poe et al., 2000; Louie and Wilson, 2001). While reactivation during NREM sleep aids memory consolidation, it has been proposed that REM sleep may contribute to other functions, such as synaptic homeostatic regulation and consolidation of non-declarative types of memory. As the role of REM sleep in synaptic homeostatic regulation is beyond the scope of this review (see Diekelmann and Born, 2010 and Cirelli, 2017 for review), here we discuss its role in supporting procedural/ non-declarative memory.

Regarding the role of REM sleep in non-declarative memory, one influential account is the “dual process hypothesis”, which proposes that NREM sleep facilitates declarative, hippocampus-dependent memory, whereas REM sleep supports procedural and emotional aspects of memory (i.e., non-declarative memory; Maquet, 2001). Early studies using a night-half paradigm (i.e., learning a memory task following the first or last night-half of sleep) found that hippocampal-dependent memory benefits from the first night-half, NREM-rich sleep, whereas emotional memory benefits from the last night-half, REM-rich sleep (reviewed in Ackermann and Rasch, 2014). However, the interpretation of the night-half findings should be treated with caution: although NREM and REM stages dominate in the first and second half of the night respectively, both stages and associated neural activity occur in each half, and these two halves also differ dramatically in many aspects besides the composition of NREM/ REM stages (e.g., hormone levels and test timing related to recent learning; reviewed by Genzel et al., 2014; Genzel et al., 2015). Recent studies, which more accurately targeted REM and NREM sleep, have revealed some important evidence challenging the “dual process” view. Indeed, there is evidence that REM sleep is associated with memory processing related to amygdala, which is essential for emotional memory (reviewed by Genzel et al., 2015). For example, strong activation has been found in the amygdala, PFC and hippocampus during human REM sleep (reviewed by Nir and Tononi, 2010). Additional evidence has been reported that bidirectional changes in fear memory are selectively correlated with the changes of theta coherence between the amygdala and PFC, as well as the hippocampus, during REM sleep (Popa et al., 2010). However, a recent study investigated the interactions between the hippocampus and the amygdala during sleep using a spatial task combining an aversive component (Girardeau et al., 2017). They found that while the firing rates of amygdala pyramidal cells increased during REM sleep, emotional memory in the hippocampus-amygdala system was reactivated during SWRs in NREM sleep, instead of REM sleep (Girardeau et al., 2017), indicating that NREM sleep could be beneficial to emotional memory. Along with this finding, many studies have found that NREM and associated sleep spindles play a critical role in consolidation of motor skills (Walker et al., 2003; Gulati et al., 2014; Ramanathan et al., 2015; Gulati et al., 2017), which suggests NREM sleep is also important for non-declarative memory. On the other hand, another study targeted hippocampal theta rhythm during REM by selectively inhibiting GABAergic neurons of the medial septum, and found that this manipulation of REM sleep patterns resulted in impairment of hippocampal-dependent spatial and contextual memory consolidation (Boyce et al., 2016). Therefore, the functions of NREM and REM sleep in memory may not be mutually exclusive for different memory types, but rather overlapping and complementary. Such a view has been proposed as the “sequential hypothesis”, which argues that the optimum benefits of sleep for consolidation of both declarative and non-declarative memory occur when NREM and REM stages take place in succession (Giuditta et al., 1995; Diekelmann and Born, 2010; Giuditta, 2014; Sara, 2017). Consistent with this proposal, it has been found that the number of NREM-REM transitions are positively correlated with memory performance in a two-way active avoidance task (Langella et al., 1992). Moreover, a short nap for humans only improves memory if it contained REM sleep in addition to NREM sleep (Mednick et al., 2003). Interestingly, it has been shown that the incidence of sleep spindles and SWRs during NREM sleep is correlated with firing rate changes in the hippocampus during subsequent REM sleep, suggesting that the synaptic homeostatic regulation implemented during REM sleep may be initiated by SWRs and spindles during preceding NREM sleep (Grosmark et al., 2012; Miyawaki and Diba, 2016).

Collectively, the above-described findings support the notion that sequences of NREM and REM stages optimize memory consolidation. During NREM-REM cycles, fast repeated reactivation of neuronal ensembles related to different types of memories predominately occur during SWRs in NREM sleep. This memory reactivation during NREM sleep may lead to plasticity in many key regions that are part of memory circuits, such as the PFC, and further initiate subsequent memory processes during following REM sleep (Frankland and Bontempi, 2005; Mizuseki and Miyawaki, 2017). In this way, different sleep stages could operate synergistically to support memory consolidation, with sleep SWRs and associated reactivation acting as a key linkage of the NREM-REM sleep chain.

3. Hippocampal-prefrontal reactivation during the waking state

While many studies have investigated the role of sleep SWR-associated communication between the hippocampus and PFC in memory as discussed above, only recently has the role of awake SWRs begun to be elucidated. Awake SWRs occur prominently during pauses in exploratory behavior. Initially, many studies explored the role of awake SWRs in memory by examining or disrupting the associated physiological activity in the hippocampus. These studies show that hippocampal neurons during awake SWRs repeatedly reactivate ongoing experiences in both forward and reverse orders (Foster and Wilson, 2006; Diba and Buzsáki, 2007; Wikenheiser and Redish, 2013; Ambrose et al., 2016), suggesting that awake replay could also play a role in memory consolidation. In favor of this idea, a key study by Jadhav and colleagues showed that disrupting awake SWRs leads to a selective impairment of working memory-dependent performance, without effects on post-task sleep reactivation (Jadhav et al., 2012). In another study, disrupting awake SWRs led to destabilization of spatial representations in the hippocampus (Roux et al., 2017). These studies provide direct evidence for a causal role of awake SWRs in memory formation. Furthermore, hippocampal awake replay events often predict future trajectories and generate novel sequences that the animal has never been experienced (Davidson et al., 2009; Gupta et al., 2010; Pfeiffer and Foster, 2013; Singer et al., 2013), highlighting an additional role of awake SWRs in memory retrieval and planning (Carr et al., 2011; Foster, 2017).

Despite these studies, evidence for hippocampal-cortical communication during awake SWRs is scarce. This is probably due to the concern that CA1 ripples are strongly localized (Chrobak and Buzsaki, 1996; Csicsvari et al., 2003), which may only weakly affect neocortical neurons via multi-synaptic pathways (Buzsáki, 2015). As we discussed above, sleep hippocampal SWRs are coordinated with neocortical spindles and slow oscillations, and such coordination may promote ‘effective connectivity’ that enables long-range activity correlations favorable for global information exchange (Fries, 2005; Inostroza and Born, 2013; Genzel et al., 2014; Igarashi, 2015). However, field potential (or EEG) rhythms are different during waking, with a lack of cortical SO and spindles (Fig. 2A, middle; Watson and Buzsáki, 2015; Tang et al., 2017). Therefore, it has been proposed that reactivation of neural activity during SWRs is perhaps more global in sleep states, but becomes more local in waking states (Genzel and Robertson, 2015).

Figure 2. Hippocampal-prefrontal network oscillatory patterns differ during awake and sleep SWRs.

(A-B) SWR-triggered power spectrograms in CA1 (A) and PFC (B) during pre-task NREM sleep (left), wake (middle), and post-task NREM sleep (right). The spectrograms were Z-scored by the average power of each frequency in a given session. Note that there is enhanced delta (1-4 Hz) and spindle (12-18 Hz) power in the CA1 and PFC during sleep SWRs, but not during awake SWRs.

(C-D) Averaged cross-correlograms of CA1 ripple power versus PFC spindle (C) and delta (D) power. Note that there is stronger spindle-ripple and delta-ripple coupling during NREM sleep (purple) than waking (black). In addition, consistent with previous studies (Siapas and Wilson, 1998; Phillips et al., 2012), there is an asymmetry between positive and negative time lags in sleep correlograms, indicating an overall tendency for ripples to precede spindle-ripple/delta-ripple episodes.

Panels adapted from Tang et al. (2017).

To investigate the brain-wide impact of SWRs during waking, a key study combined electrophysiological recording with fMRI and found that SWRs in non-human primates during waking quiescent states were associated with robust BOLD signals in multiple brain regions including PFC, indicating widespread activation similar to sleep states (Logothetis et al., 2012). Consistent results have been found in rodent studies that demonstrated modulation of PFC spiking by awake SWRs (Jadhav et al., 2016; Wang and Ikemoto, 2016; Tang et al., 2017), suggesting that the hippocampus and PFC interact during awake SWRs. In fact, it has recently been shown that reactivation in the hippocampal-prefrontal network more accurately recapitulates previous experience during awake SWRs than sleep SWRs (Tang et al., 2017). This structured awake reactivation in the hippocampal-prefrontal network may thus support accurate memory storage and memory-guided behavior. Importantly, despite the high fidelity of awake reactivation, the strength of cortical spindles and delta oscillations and their coordination with hippocampal SWRs significantly decreased in the waking state as compared to the NREM sleep state (Fig. 2; Tang et al., 2017). Therefore, given that network patterns are quite different in waking and sleep states, hippocampal-prefrontal communication during SWRs may depend on different underlying mechanisms that route flow of information in these two states. Consistent with this idea, it has been found that while hippocampal area CA2 has a trigger role for both sleep and awake SWRs, CA2 cells contributed more strongly to awake SWRs (Oliva et al., 2016). Additional evidence has been found that medial entorhinal cortical (MEC) inputs affect CA1 ripples during waking states, but not NREM sleep (Yamamoto and Tonegawa, 2017). This raises the possibility that CA1 ripples in the waking state may be triggered by entorhinal inputs conveying external stimuli, such as top-down signal from cortical areas; whereas hippocampal circuits may internally initiate SWR reactivation which is routed to cortical areas during NREM sleep for memory consolidation (Yamamoto and Tonegawa, 2017).

What are the underlying neural circuits and pathways that support hippocampal-prefrontal reactivation? Anatomical evidence indicates the possibility that the activity of PFC is modulated by direct excitation from hippocampal awake SWRs. PFC receives monosynaptic projection from ventral CA1 (Jay et al., 1989; Hoover and Vertes, 2007; Buzsáki, 2015; Eichenbaum, 2017), and recent evidence has also revealed a direct projection from dorsal CA1 to PFC (Maharjan et al., 2016; Ye et al., 2017). The strong excitation gain during SWRs can depolarize the monosynaptic target regions of CA1, such as PFC (reviewed by Buzsáki, 2015). In addition, it has been found that hippocampal inputs can directly excite cortical interneurons, at least in PFC (Tierney et al., 2004). Therefore, synchronous activity during SWRs may potentially enable memory information to be routed from the hippocampus to PFC. On the other hand, PFC may, in turn, also influence hippocampal SWRs. It has been proposed that the top-down signal from PFC could trigger hippocampal replay involved in planning upcoming choices (Yu and Frank, 2015; Shin and Jadhav, 2016). However, direct return connections from PFC to hippocampus are rare (Vertes et al., 2007; Eichenbaum, 2017; Ito, 2017), although a recent study has reported sparse projections from anterior cingulate cortex (a major part of medial PFC in rodents) to the hippocampus in mice (Rajasethupathy et al., 2015). Therefore, the impact of PFC on awake SWRs remains unclear, but it is likely that the information from PFC to the hippocampus may be transferred by indirect pathways that involves other cortical and subcortical structures.

In all, these data point to the importance of awake reactivation during SWRs in memory consolidation, retrieval and planning. Although waking and sleep states differ in many aspects, such as neuromodulatory tone, network activity patterns, and behavioral and internal contexts (Diekelmann and Born, 2010; Carr et al., 2011; Roumis and Frank, 2015), hippocampal SWRs appear as a common neurophysiological pattern in these two states for reactivating memory information across hippocampal-prefrontal circuits. Of note, the differences in sleep and awake reactivation, particularly related to ongoing experience and behavioral planning, imply that specific mnemonic functions of these two forms of reactivation may be different, a discussion of which we turn to in the last section.

4. Plasticity during SWR reactivation for learning

The studies discussed in the preceding sections indicate the importance of hippocampal-prefrontal reactivation for memory. Notably, hippocampal-prefrontal interactions do not represent a static process, but rather evolve during task learning (reviewed by Igarashi, 2015; Eichenbaum, 2017), and plasticity in hippocampal-prefrontal networks could be a key mechanism that supports learning. Here, we suggest that the plasticity driven by SWR reactivation plays a central role in learning.

Since large populations of neurons across multiple brain regions fire synchronously during SWRs, this network synchronization has been hypothesized to be ideal for triggering plastic changes that facilitate learning (Csicsvari and Dupret, 2014; Buzsáki, 2015). In fact, SWRs and the content of hippocampal activity associated with SWRs show many learning- and experience-related changes. Previous work has found that SWR rates are enhanced for novel experience (Foster and Wilson, 2006; Cheng and Frank, 2008; Eschenko et al., 2008) and modulated by rewards (Foster and Wilson, 2006; Singer and Frank, 2009; Ambrose et al., 2016). The content of hippocampal activity during SWRs also changes with experience: the replay of place cell activity reflecting previously traversed trajectories dominates during post-experience sleep, but not pre-task sleep (Wilson and McNaughton, 1994; Lee and Wilson, 2002; Silva et al., 2015); such replay events increase during the first few hours of post-experience sleep and decrease with time (Kudrimoti et al., 1999; Eschenko et al., 2008). The amount and content of replay events can also be modulated by reward and dopaminergic signal (Dupret et al., 2010; McNamara et al., 2014; Ambrose et al., 2016), as well as sensory cues presented during sleep (Bendor and Wilson, 2012; Barnes and Wilson, 2014; Rothschild et al., 2016). Furthermore, disruption studies have revealed that SWRs can promote learning-related plasticity and stabilization of the hippocampal spatial map (Dupret et al., 2010; Schoenenberger et al., 2016; van de Ven et al., 2016; Roux et al., 2017). In particular, disrupting SWRs impairs only the consolidation of hippocampal assemblies reflecting a novel environment, but not the consolidation of familiar memories (van de Ven et al., 2016). Together, these results point towards plasticity during SWRs and associated neural activity as a mechanism contributing to learning.

Although there is strong evidence that PFC interacts with the hippocampus during SWR events as we discussed above, few studies have addressed whether the hippocampal-prefrontal interactions during SWRs change over learning. Recently, there have been some clues regarding this question. Benchenane and colleagues studied the learning-related changes in PFC using a rule-shifting Y-maze task (Benchenane et al., 2010). They found that strongly co-activated PFC cell ensembles emerged at the choice point upon learning, and these ensembles were selectively reactivated during subsequent sleep SWRs. This suggests that reactivation of PFC activity associated with newly learned experiences occurs during hippocampal SWRs, at least in sleep states. Consistent with this idea, previous studies also found that the coupling between hippocampal ripples and PFC delta and spindle waves increased during NREM sleep following training on spatial memory tasks (Maingret et al., 2016; Khodagholy et al., 2017; Xia et al., 2017). However, do hippocampal-prefrontal neuronal interactions during SWRs evolve as learning progresses? A recent study addressed this very question by examining the PFC and CA1 activity over the course of learning (Tang et al., 2017). CA1-PFC reactivation during both sleep and awake SWRs was found to peak during initial learning in a novel environment and gradually diminished towards the end of learning. These findings therefore indicate that coordinated reactivation in hippocampal-prefrontal ensembles during SWRs may play a critical role in spatial learning.

5. The functional roles of hippocampal-prefrontal reactivation across sleep-waking cycles

The evidence reviewed here suggests that, while sleep and waking are two distinct states, SWRs provide a common biophysical signature of memory reactivation in the hippocampal-prefrontal network during both states. However, this is not to say that awake and sleep reactivation serve the same mnemonic functions. As we mentioned above, there are two crucial and unique features of awake reactivation in the hippocampal-prefrontal network (Tang et al., 2017): First, it provides a clearer recapitulation of ongoing experience as compared to sleep reactivation, without requiring coordination of network oscillations seen during sleep. Second, awake reactivation is strongest during initial learning. Such unique features could be beneficial to some certain memory functions.

Since awake reactivation is a more accurate representation of current behavioral contexts, it raises the possibility that it supports rapid memory-guided performance in action. Consistent with this idea, a previous study has also found that the activity replayed during awake SWRs in the hippocampus was more closely correlated with the activity during running on the maze compared with sleep replay (Karlsson and Frank, 2009). Further, it has been reported that hippocampal activity during awake SWRs can predict subsequent decisions (correct or incorrect; Singer et al., 2013), and the content replayed during awake SWRs is biased towards spatial trajectories that start from the animal’s current position to upcoming goals (Pfeiffer and Foster, 2013) or an avoided shock zone (Wu et al., 2017). These findings collectively demonstrate an important relationship between neuronal activity during awake SWRs and ongoing behavior. On the other hand, the dynamics of awake reactivation over the course of learning suggest that it has a critical role in spatial learning and memory. It has been found that awake reactivation predominantly occurs at reward locations and is modulated by reward (Foster and Wilson, 2006; Diba and Buzsáki, 2007; Singer and Frank, 2009; Ambrose et al., 2016). Initially, when animals start to learn tasks, strong awake reactivation at the reward locations may thus have an important role in linking spatial experience encoded by hippocampal ensembles with rewards and outcomes encoded by PFC ensembles (illustrated as a pair of PFC neurons in Fig. 3, left and middle; Euston et al., 2012; Hyman et al., 2013; Insel and Barnes, 2015; Pinto and Dan, 2015). Reward representations of PFC ensembles in different contexts could potentially overlap; however, trajectories leading to reward in these contexts are expected to have unique representations in the hippocampal-prefrontal network. The representations associated with actions leading to reward can therefore be strongly reactivated during awake SWRs at reward locations. This process is repeatedly implemented during awake SWRs when animals incrementally learn the task, and can facilitate stabilization of task-selective representations in the two regions. Consistent with this, it has been found that disrupting awake SWRs results in unstable spatial maps in the hippocampus (Roux et al., 2017). Whether or not the PFC is also involved in this stabilization process remains to be determined.

Figure 3. The functional roles of hippocampal-prefrontal reactivation across sleep-waking cycles.

In spatial learning tasks, animals learn by trial and error that particular sequences of locations (paths) will be rewarded (left top). As animals traverse the environment, many hippocampal neurons (blue triangles) selectively respond to certain locations in the environment (i.e., place fields, represented by blue ellipses). Upon reward receipt, awake SWRs occur at the reward location (left; SWRs shown in green), which serve as a potential neural mechanism to linking spatial experience encoded by hippocampal ensembles with rewards and outcomes encoded by PFC ensembles (red triangles; left bottom). When animals perform the same task in a different environment (middle), different action-outcome associations (green and red triangles) form in the hippocampal-prefrontal network during awake SWRs. During subsequent sleep SWRs (right), repeated hippocampal-prefrontal reactivation strengthens memory traces. These sleep SWRs are further coupled with prefrontal delta waves (orange line) and spindles (red line), which allows active systems consolidation and local cortical processes. During these processes, the overlapping prefrontal memory traces form connections with each other, creating an integrated representation (red triangles and connecting lines).

Subsequent to these processes during waking states, reactivation during NREM sleep after behavior can support further consolidation of these memories. In particular, it has been proposed that different related memories need to be integrated into “memory schema” to represent associative structures among these memories, which crucially relies on hippocampal-prefrontal interactions (Frankland and Bontempi, 2005; Tse et al., 2007; Lewis and Durrant, 2011; Tse et al., 2011; Battaglia et al., 2012; Genzel et al., 2017; Schlichting and Frankland, 2017). Building on the idea that overlapping reactivation of learned information promotes building memory schema during sleep (Lewis and Durrant, 2011; Feld and Born, 2017), it is tempting to speculate that hippocampal sleep reactivation, followed by prefrontal spindles and delta waves, facilitates the integration of several distinct experiences in the PFC, which therefore manifests as “noisy” reactivation of the most recent experience (Battaglia et al., 2012; Roumis and Frank, 2015; Tang et al., 2017). Consistent with this, an important study has shown that when rats learned place-flavor associations for several weeks, new associations were learned much more rapidly, presumably because they developed a “memory schema” of these associations (Tse et al., 2007). Additionally, pharmacological disruption of PFC impairs the rapid learning of new associations, as well as retrieval of remote memories (Tse et al., 2011). Likewise, an interesting prediction can be made that sequential training in an integration task (e.g., training with the same rules in two different environments or similar associations as in Tse et al., 2007) may result in coordinated reactivation of these distinct but related experiences in the hippocampal-prefrontal network during sleep, and the overlapping reactivation in PFC could further facilitate schema formations (Fig. 3, right). We can therefore hypothesize that related contexts can be reactivated in a correlated manner (either simultaneously or sequentially; Lewis and Durrant, 2011) during subsequent sleep, whereas the reactivation will be less correlated or even independent when animals learn different rules in distinct environments.

Further studies are still needed to determine the proposed functional roles of sleep and awake reactivation. Previous studies using causal manipulations, such as optogenetic techniques, have provided important insights by perturbing SWRs and associated oscillatory and neuronal activity in these two regions. However, we argue that this activity pattern exhibits considerable changes as learning evolves. Therefore, in order to specify the role of hippocampal-prefrontal reactivation in memory, it is important to show how such dynamics are related to learning. This question may be pursued by specifically perturbing this activity at different learning and behavioral stages. Furthermore, since the hippocampus and PFC interact via multiple direct and indirect pathways, it is of great interest to understand how communication of memory information occurs in these pathways during different behavioral stages. New techniques that permit projection-specific optogenetic perturbation will hopefully reveal the circuit basis for memory and learning. Ultimately, addressing these questions might allow us to tackle cognitive deficits in many neurological disorders, such as Alzheimer’s disease and schizophrenia, in which the involvement of the hippocampal-prefrontal network is strongly implicated (Colgin, 2011; Gordon, 2011; Spellman et al., 2015; Gillespie et al., 2016).

Highlights.

• SWRs mediate hippocampal-PFC reactivation during both waking and sleep.

• Reactivation and oscillatory coupling during sleep SWRs are key to consolidation.

• Awake reactivation is more structured and does not show oscillatory coupling.

• Hippocampal-PFC reactivation is enhanced during initial learning.

• A model of how sleep and awake reactivation together support learning is proposed.