How our understanding of memory replay evolves

Abstract

Memory reactivations and replay, widely reported in the hippocampus and cortex across species, have been implicated in memory consolidation, planning, and spatial and skill learning. Technological advances in electrophysiology, calcium imaging, and human neuroimaging techniques have enabled neuroscientists to measure large-scale neural activity with increasing spatiotemporal resolution and have provided opportunities for developing robust analytic methods to identify memory replay. In this article, we first review a large body of historically important and representative memory replay studies from the animal and human literature. We then discuss our current understanding of memory replay functions in learning, planning, and memory consolidation and further discuss the progress in computational modeling that has contributed to these improvements. Next, we review past and present analytic methods for replay analyses and discuss their limitations and challenges. Finally, looking ahead, we discuss some promising analytic methods for detecting nonstereotypical, behaviorally nondecodable structures from large-scale neural recordings. We argue that seamless integration of multisite recordings, real-time replay decoding, and closed-loop manipulation experiments will be essential for delineating the role of memory replay in a wide range of cognitive and motor functions.

INTRODUCTION

Memory, as a fundamental function of the brain, refers to the physiological processes of acquiring, storing, maintaining, and later retrieving information. Memory uniquely defines a sense of self-identity and includes all “who,” “what,” “when,” and “where” information in our past and present life experiences, both remote and recent. Among the many forms of memories, episodic and semantic memories are the two main subcategories of declarative memory (“knowing that,” in contrast to procedural memory, or “knowing how”) that are explicit and can be consciously recalled (1). The hippocampus, neocortex, and amygdala are involved in explicit memory, whereas the prefrontal cortex (PFC) is also critical for various (e.g., spatial, verbal) forms of working memory (also known as short-term memory). Reactivation or replay of these memories in the brain during awake and sleep states has been implicated in navigational planning, recall and spatial working memory, learning and generalization, and memory consolidation (2). In contrast to episodic and semantic memory, motor skills reflect one aspect of procedural memory that is responsible for knowing how to do things. Reactivation or replay of procedural memories has also been found in the motor cortex of songbirds, rodents, and humans. Technological advances in large-scale electrophysiology (EP), neuroimaging, and optogenetics technologies (3–7) as well as the development of analytic and computational modeling methods have enabled us to delineate the content and functional significance of these events. Importantly, combination of experimental and computational techniques has constantly reshaped our understanding of memory replay for a wide range of cognitive and motor tasks. It is beneficial to revisit these analytic methods and understand their strengths and limitations.

First, it is necessary to distinguish the term “replay” from “reactivation.” Replay and reactivation are both referred to as a process of strengthening past experiences during an offline, postencoding period. However, replay, also known as sequential reactivation, specifically refers to a spatiotemporal event that typically corresponds to a predefined behavioral template (with a specific “sequenceness” or similarity measure defined at the neural, spatial, or abstract state space) (2, 8). Replay of neuronal assemblies can appear as a form of substantiated neural sequence or template, and this notion can further generalize beyond the cellular resolution. In contrast, reactivation is more generally defined by the similarity of activity patterns without a specific temporal relationship (often defined by zero-lag correlation). Importantly, a true replay event does not necessarily have a high reactivation score at the single time moment; reactivation score also heavily depends on the temporal scale. Replay analyses focus on identifying content-specific behavioral correlates (Supplemental Fig. S1), whereas reactivation analyses do not. In human memory studies, memory recall and memory retrieval both refer to a process of accessing or recovering previously stored information (9). Memory retrieval is usually triggered by a partial or complete cue. Reactivation and replay do not automatically imply successful recall or retrieval, but it is commonly implied that successful recall is accompanied by replay. In other words, reinstatement of dynamic memories requires replay of neural patterns that unfold in time in a predictable manner. We note that reactivation and replay have sometimes been used interchangeably in the literature; in the remainder of this article, we try to distinguish these two terms as explicitly as possible.

The past few decades have witnessed a continuing wave of new discoveries and deeper mechanistic understanding of memory replay. New hypotheses have been proposed, while some older theories have also been revisited thanks to discoveries of new data. Several reviews on replay have been written but have primarily focused on the animal literature (2, 8, 10–14), especially on the rodent hippocampus. An excellent review on human memory replay can be found in Ref. 9. The literature on memory replay has a long list and has continued to grow substantially (Supplemental Fig. S2). Two sets of questions are relevant to our understanding of replay. The first set of questions is centered on “what,” “when,” and “where” questions about replay events, and the second set of questions is centered on how to identify, decode, and assess replay. In this “Now and Then” review, we first present a historical overview of representative studies in memory replay across brain areas and multiple species (i.e., the first set of questions) (Table 1). Next, we systematically review the existing analytic methods for reactivation/replay analyses based on spatiotemporal patterns of neuronal ensembles or mesoscopic brain activity (i.e., the second set of questions). Along the way, we also provide some guiding thoughts and discuss the outlook for the development of new analytic methods. Our article shares some overlap with a few other complementary methodology reviews on replay (50–52) but features a much broader discussion on new ideas, analysis paradigms, and unresolved questions. Specifically, we focus more attention on various types of population decoding approaches as well as emerging analytic methods to examine spatiotemporal structures from large-scale neural recordings, including EP, calcium imaging, magnetoencephalography (MEG), electroencephalography (EEG), electrocorticography (ECoG), and functional magnetic resonance imaging (fMRI). We advocate the unsupervised learning paradigm to assess “behaviorally nondecodable” replay. Finally, we discuss some outstanding questions critical to improved understanding of memory replay.

Table 1.

Representative experimental studies of memory replay and reactivation

Reference	Brain State	Brain Area	Neural Recording	Methods	Reactivation or Replay	Significance Assessment
Wilson and McNaughton (1994) (15)	Sleep	Rat HPC	EP	Correlation	Reactivation	Pearson cross-correlation P < 0.05
Skaggs and McNaughton (1996) (16)	NREM	Rat HPC	EP	Correlation	Reactivation	Pearson cross-correlation P < 0.05
Nádasdy et al. (1999) (17)	NREM	Rat HPC	EP	Template matching	Replay	Pearson cross-correlation P < 0.05, random shuffling
Dave and Margoliash (2000) (18)	NREM	Zebra finch (songbird) sensorimotor	EP	Correlation	Replay	Cross-correlation P < 0.02
Hoffman and McNaughton (2002) (19)	Rest	macaque monkey cortex	EP	Correlation	Reactivation	Cross-correlation, P < 0.05, EV
Louie and Wilson (2001) (20)	REM	rat HPC	EP	Template matching	Replay	Bin/column/swap/shift shuffle, correlation P < 0.05
Lee and Wilson (2002) (21)	NREM	rat HPC	EP	Sequence matching	Replay	Theoretical z score, approximate P value
Pennartz et al. (2004) (22)	Rest	Rat ventral striatum	EP	Correlation	Reactivation	Cross-correlation, P < 0.05, EV
Ribeiro et al. (2004) (23)	NREM, REM	Rat forebrain	EP	Correlation	Reactivation	Cross-correlation, P < 0.05
Foster and Wilson (2006) (24)	Awake	Rat HPC	EP	Ordered sequence	Replay	Cell order shuffle
Diba and Buzsaki (2007) (25)	Awake	Rat HPC	EP	Sequence correlation	Replay	Shuffle surrogate
Ji and Wilson (2007) (26)	NREM	Rat HPC and V1	EP	Ordered sequence	Replay	Combinatorial method [Lee and Wilson (155)]
Euston et al. (2007) (27)	Sleep	Rat PFC	EP	Correlation, template matching	Reactivation	Cross-correlation, EV
Lansink et al. (2009) (28)	Sleep	Rat HPC and ventral striatum	EP	Correlation	Reactivation	Cross-correlation, EV, REV, bootstrap [EV-REV], P < 0.05
Davidson et al. (2009) (29)	Awake	Rat HPC	EP	Bayesian decoding	Replay	Column/unit/pseudoevent shuffle
Karlsson & Frank (2009) (30)	Awake	Rat HPC	EP	Bayesian decoding	Replay	Random shuffle
Pfeiffer and Foster (2013) (31)	Awake	Rat HPC	EP	Bayesian decoding	Replay	Random shuffle
Gulati et al. (2014) (32)	NREM	Rat motor	EP	PCA	Reactivation	Method [Peyrache et al. (65, 151)]
Ramanathan et al. (2015) (33)	NREM	Rat motor	EP	PCA, temporal sequence	Reactivation	Pearson correlation
Jafarpour et al. (2014) (34)	Awake	Human	MEG	MVPA (SVM decoding)	Replay	n/a
Kurth-Nelson et al. (2016) (35)	Awake	Human	MEG	MVPA (LASSO logistic regression)	Replay	“Sequenceness” measure
Malvache et al. (2016) (36)	Awake	Mouse HPC	Calcium imaging + EP	Sequence	Replay	Spearman correlation P < 0.05, random shuffle
Olafsdottir et al. (2016) (37)	Rest	rat HPC and mEC	EP	Bayesian decoding	Replay	Place cell rate map shuffle
O’Neil et al. (2017) (38)	Rest	Rat HPC and mEC	EP	Bayesian decoding	Replay	Spike time shuffle
Zhang et al. (2018) (39)	Rest and NREM	Human	iEEG	MVPA, RSA	Replay	Spearman correlation surrogate
Liu et al. (2019) (40)	Rest	Human MTL	MEG	MVPA	Replay	“Sequenceness” measure
Stella et al. (2019) (41)	Awake	Rat HPC	EP	Bayesian decoding	Replay	Spike jittering, spike identity reassignment, place field rotation, position shuffle, reactivation speed shuffle
Schuck and Niv (2019) (42)	NREM	Human HPC	fMRI	SVM decoding	Replay	“Sequenceness” measure
Kaefer et al. (2020) (43)	Awake	Rat HPC and PFC	EP	Population decoding	Replay	Rate map shuffle
Eichenlaub et al (2020) (44)	Rest	Human motor cortex	EP	Template matching	Replay	2-D cross-correlation
Vaz et al. (2020) (45)	Awake	Human MTL	EP + ECoG	Sequence similarity	Replay	Correlation P < 0.05, rank-order correlation, permutation
Elmaleh et al. (2021) (46)	Sleep	Songbird premotor	EP	Template matching	Replay	Random shuffle
Grosmark et al. (2021) (47)	Awake	Mouse HPC	Calcium imaging	Bayesian decoding, ICA	Reactivation	Random shuffle, random re-reordering
Wittkuhn and Schuck (2021) (48)	Awake	Human visual cortex	fMRI	MVPA	Replay	Sequentiality metric
Rubin et al. (2022) (49)	Sleep	Human motor	EP + EEG	Unsupervised decoding (KF)	Replay	Random pseudotemplate

2-D, two-dimensional; ECoG, electrocorticography; EEG, electroencephalography; EP, electrophysiology; EV, explained variance; fMRI, functional magnetic resonance imaging; HPC, hippocampus; ICA, independent component analysis; iEEG, intracranial EEG; KF, Kalman filter; mEC, medial entorhinal cortex; MEG, magnetoencephalography; MTL, medial temporal lobe; MVPA, multivariate pattern analysis; n/a, not available; NREM, non-rapid eye movement; PCA, principal component analysis; PFC, prefrontal cortex; REM, rapid eye movement; REV, reversed EV; RSA, representational similarity analysis; SVM, support vector machine.

HISTORICAL OVERVIEW OF REPRESENTATIVE MEMORY REPLAY STUDIES

Hippocampal Replay in Animal Studies

Replay studies originated from the rodent hippocampus, where behavioral sequences are defined by spatial trajectories as animals navigate in a maze or perform spatial memory tasks. The majority of hippocampal pyramidal cells have place-specific tunings (i.e., “place cells”) (53). During the online state, hippocampal place cells fire in sequence in relation to one (1-D)- or two (2-D)-dimensional spatial position (“spatial sequence”) or in relation to a specific episodic event (“nonspatial sequence”). During the offline state [such as immobility and slow-wave sleep (SWS)], typically accompanied by hippocampal sharp-wave ripples (SWRs), synchronous oscillatory electrical events in hippocampal local field potentials (LFPs) (54), subsets (10–20%) of hippocampal place cells fire in a temporally compressed manner but with similar sequence order, a phenomenon referred to as “hippocampal replay” (Fig. 1A).

Figure 1.

Memory reactivation and replay in animal studies. A: an example of rat hippocampal replay. The first 2 traces show the raw hippocampal local field potential (LFP) and filtered signal within the ripple band. Ticks show action potentials. Time 0 denotes the onset of putative replay event. Inset shows a 170-ms decoded replay trajectory, and red color denotes high posterior probability. B: illustration of pairwise correlation strengths among hippocampal place cells during run, prerun sleep, and postrun sleep. Edge width is proportional to the Pearson’s correlation. For clarity, only positive correlation > 0.05 is shown [Wilson and McNaughton (15). C: heat map of mean posterior probabilities (left; cyan line indicates temporal sequence), movement (top center), and slow-gamma oscillation (bottom center) for a representative trajectory event. Right: average spiking probability as a function of fast oscillations (20–50 Hz, 10° bin size) for all trajectory events [modified with permission from Pfeiffer and Foster (55); copyright AAAS). D: rat visual cortical (left) and hippocampal (right) neuronal firing sequences during RUN and POST sleep replay event. Cortical firing sequence during RUN and POST sleep. Each row represents a cell, and each tick represents a spike. Cells were assigned numbers 0, 1, etc. and then arranged (01234567) from bottom to top according to the order of their firing peaks (vertical lines). Each colored curve represents the average firing rate of a cell. Triangles and circles, event start and end times, respectively [Ji and Wilson (26)]. E, top: spiking of spatially selective rat medial prefrontal cortex (mPFC) cells (n = 23 cells) during an example trial. The black line denotes the real position of the animal and the gray shaded area an event where trajectory replay has been detected. The dashed box zooms into the trajectory replay event. Bottom: hippocampal (HPC; blue) and mPFC (red) trajectory replay independently. At the goal, the trajectory replay rate in the hippocampus (left) positively correlated with the number of trials required to switch to the new rule, and therefore negatively correlated with rule-switching performance. Conversely, the mPFC trajectory replay rate (right) negatively correlated with number of trials required to switch [modified with permission from Kaefer et al. (43); copyright Elsevier].

In one landmark review paper, Buzsáki (56) proposed a two-stage model for memory consolidation, which starts with a waking “theta” state followed by a “ripple” state, wherein information is transferred from the hippocampus and then stored in the cortex as long-lasting memory traces. Accordingly, the ripple state provides a window for replaying the theta state. Ever since its proposal, this theory has motivated generations of researchers to examine the change in hippocampal neuronal firing across different brain states. Pavlides and Winson (57) were among the first to ask how hippocampal neuronal firing changes from awake behavior to postbehavior sleep, but their analysis was limited to the mean firing rate. In the very first memory reactivation study, Wilson and McNaughton (15) reported that the activity of rat hippocampal place cells that were coactive during spatial navigation exhibited an increased probability of cofiring during postrun sleep (Fig. 1B). Based on correlation methods, they concluded that reactivation of hippocampal assemblies reflects the animal’s prior spatial episodic memory. This seminal study was followed by several other detailed studies (16, 58, 59). Most analyses were based on firing rate and firing rate correlation and did not fully account for temporal relationships (except that temporal bias of cross-correlation was considered in Ref. 16); therefore, the phenomena were referred to as reactivation. It was Nádasdy and colleagues (17) who first applied a template-matching method to discover recurring neural sequences in the hippocampus; they also first applied four different spike shuffling procedures to assess the significance of these replayed spike sequences occurring at a faster timescale. In subsequent studies, Louie and Wilson (20) analyzed hippocampal replay (using a generalized template-matching method) during rapid eye movement (REM) sleep following run behavior; they found that rat hippocampal ensembles repeatedly reactivated in a temporally consistent order during postrun sleep. Next, Lee and Wilson (21) developed the combinatorial rank-order analysis and examined hippocampal replay during non-REM (NREM) sleep or SWS. Since REM sleep is interleaved by NREM sleep, the firing rate correlation in REM sleep was not related to the preceding familiar run experience, and memory replays occurred more frequently for remote run experiences. These findings of sleep-associated hippocampal reactivation and replay supported the idea that the hippocampus consolidates the memory of awake experience.

Independent of hippocampal replay in sleep, awake hippocampal replay during immobility was also reported from multiple laboratories (24, 25, 29,30, 60). Specifically, population bursts of spiking activity occur during single or multiple events of SWRs. Depending on the firing sequence order of place cells or decodable content of the population activity, hippocampal replay in a simple linear environment can be categorized as “forward” or “reverse” replay, referring to reactivated hippocampal sequences in relation to the run trajectory. The forward or reverse replay can start in the nearby or remote position with respect to the animal’s actual position (24, 25). Furthermore, in a larger 1-D environment, extended hippocampal replay may occur during trains of ripple events, linking shorter subsequences to express memories of prolonged experience (29).

In 2-D environments, studies of hippocampal replay were empowered by a large number of unit yields, typically 200–400 neurons from 32–40 tetrodes (31, 41, 55). Pfeiffer and Foster (31) applied Bayesian decoding to identify putative replay trajectories associated with rat hippocampal SWRs, which did not always represent a perfect replay of prior behavior but instead a broad array of spatial paths. These sequences predicted animals’ immediate future paths that were strongly biased to a remembered goal location (31). Furthermore, the replay trajectories showed discontinuity between immobility and rapid movement, suggesting a fundamental discretization in hippocampal memory retrieval (55). The reactivation alternation between sharpening and transition is time locked to a fast oscillation around 25–50 Hz (Fig. 1C), where the oscillations may originate from the concatenated ripples (61). Importantly, the replay event statistics can be different from the statistics of actual spatial trajectories (41, 60), suggesting that hippocampal circuits can generate representations of various behavioral outcome combinations, a role important for learning, generalization, and planning.

The majority of rodent hippocampal studies have relied on electrophysiology. However, large-scale ultrafast (up to 60 Hz) calcium imaging has become increasingly popular to study hippocampal assemblies and memory during chronic recordings (36, 47). While examining the CA1 region of awake mice during runs on a treadmill, Malvache and colleagues (36) reported that anatomically intermingled yet functionally orthogonal hippocampal assemblies reactivated discrete temporal segments of run-associated neuronal sequences; furthermore, temporal binding of these assemblies into longer chains revealed temporally ordered replay. Thanks to the calcium imaging capability of tracking mouse hippocampal place cells over 2 wk of online spatial learning behavior and offline resting, Grosmark and colleagues (47) reported that offline hippocampal reactivation of spatial experiences predicted the future representational stability of place cells before their online reinstatement. Furthermore, the representations of reward-adjacent locations were stable across multiple days (47).

Extrahippocampal, Cortical, and Subcortical Replay in Animal Studies

In parallel to studies of hippocampal replay, replay studies in cortical or subcortical areas (either separately or jointly with the hippocampus) have been conducted. During NREM sleep, the hippocampus and neocortex mutually interact through temporal coordination of neuronal activity in the UP state, in which hippocampal ripples and neocortical sleep spindles and slow waves are temporally coupled or nested (62, 63). Together, they play a crucial role in memory consolidation. Presumably, cortical replay co-occurs with hippocampal replay (26, 27, 58, 64). The first memory reactivation finding was reported in the rat hippocampocortical and corticocortical ensembles based on cross-correlation analysis of simultaneously recorded units from the rat CA1 and posterior parietal cortex (PPC) (58). Furthermore, when songbirds learned a correspondence between vocal-motor output and auditory feedback during development, Dave and Margoliash (18) found that temporally precise activity patterns of single neurons elicited by the playback of songs during sleep matched the activity during daytime singing, suggesting a form of “song replay.” These neurons were coordinated and exhibited simultaneous bursting during sleep, and different burst patterns for each neuron predicted the same sequences of syllables. Subsequently, reactivation was reported in the cortex of the macaque monkey (19) as well as in the rat barrel cortex (23) and ventral striatum (22). It should be noted that the majority of memory studies in the cortex, such as the rat medial PFC (mPFC) (43, 65) and rat primary motor cortex (M1) (32, 33), have centered on reactivation analysis.

Additionally, many studies have focused on hippocampal-cortical/subcortical recordings and studied separate hippocampal and cortical representations of memories. In one study of rat hippocampus and primary visual cortex (V1) (26), it was found that CA1-V1 neuronal firing sequences evoked by awake experience were replayed in the UP state during SWS; these replay events were coordinated to reflect the same experience (Fig. 1D), suggesting that simultaneous replay of coherent memory traces in these two areas reflects or contributes to memory consolidation. In another study in the rat hippocampal and ventral striatum, Lansink and colleagues (28) found that hippocampal-striatal ensembles reactivated together to represent place-reward information during sleep and reactivation was dominated by hippocampal place cell pairs, with hippocampal firing preferentially before the striatal reward-related neurons. This result suggests that hippocampal memory consolidation can lead to reactivation in a projection area, whereas in another study of rat hippocampus and basolateral amygdala (BLA), Girardeau and colleagues (66) found coordinated hippocampal-amygdala reactivations during posttraining NREM sleep; furthermore, reactivation was stronger for the hippocampus-BLA correlation patterns representing the run direction that involved the air puff than for those representing the “safe” direction, suggesting that the consolidation of contextual emotional memory occurs during reactivation of hippocampal-amygdala circuits. In a study of the rat hippocampus and auditory cortex (AC) when animals learned a sound-guided task, it was found that there is a rapid cortico-hippocampal-cortical loop of information flow around the times of SWRs during sleep. The patterned activation in AC preceded and predicted the subsequent content of hippocampal activity during SWRs, and hippocampal patterns during SWRs predicted the subsequent AC activity (67).

Furthermore, hippocampal-prefrontal replay has been reported in multiple lines of studies (43, 64, 68, 69). Specifically, in a goal-directed spatial learning task, a coherent CA1-PFC reactivation distinguished correct past and future behavioral trajectories during awake immobility, with the reverse replay supporting retrospective evaluation and the forward replay supporting prospective planning (64). A coherent CA1-PFC reactivation is defined as a CA1 replay trajectory that is coincident with a strongly reactivated CA1-PFC ensemble (e.g., associated with a large reactivation strength). This study showed that there is a learning shift from hippocampal reverse replay to forward replay, associated with prefrontal readout of memory-guided paths for decision-making. In another hippocampal-prefrontal study in which rats performed a rule-switching task, it was shown that hippocampal and mPFC trajectory replay occurred independently and were not coordinated (Fig. 1E); more specifically, the mPFC replays trajectories of generalized position, and the occurrence of mPFC replay positively correlated with rule-switching performance (43).

Additionally, memory replays have been reported in the hippocampal-entorhinal cortex (37, 38). Specifically, grid cells recorded from the deep layers (layers V/VI) of medial entorhinal cortex (mEC) showed spatially coherent representations with the CA1 place cells during hippocampal replay, and the coherence was stronger for forward replay than reverse replay (37). On the other hand, entorhinal cell assemblies recorded from the superficial layers of mEC replayed trajectories independently of the hippocampal replay during SWRs (38). These results suggest that representations of different mEC layers may be coupled or decoupled with hippocampal memory trace reactivation, suggesting in turn that the hippocampal-entorhinal network may have multiple mechanisms for memory replay (70). Similar to the mEC, the retrosplenial cortex (RSC) is a major relay of hippocampal formation output to other neocortical areas and exhibits experience-dependent spatial sequence coding (71). In a study of the mouse hippocampus-RSC circuit, it was found that the hippocampal ripples activated RSC interneurons and inhibited RSC pyramidal cells during SWS; moreover, optogenetic stimulation of the hippocampus-RSC pathway activated and suppressed RSC putative inhibitory and excitatory neurons, respectively (72). Independently, it was reported that optogenetic reactivation of memory-related neural ensembles in the RSC (i.e., engram neurons active naturally during initial learning to represent a contextual fear memory) could produce a shift to older remote memories in sleeping brains of mice (73). The anterior thalamic nucleus structure is a key relay of neural signals within the limbic system. Head-direction (HD) cells of the anterior thalamic nuclei preserved stable directions and homogeneous coupling to SWRs during NREM sleep, whereas non-HD cells showed diverse couplings to the hippocampus (74).

In the zebra finch, song-related sparse premotor sequences within the forebrain nucleus HVC generated precise sleep replay of song activity; furthermore, these sleep replay events did not require the thalamic input, as they were unperturbed by thalamic lesion and continued to cross syllable boundaries (46).

Together, there seem to exist two contradictory observations from the animal data: 1) hippocampal and cortical replay or reactivations are coordinated (e.g., Refs. 26, 37, 64, 66) and 2) cortical replay is independent of hippocampal replay (e.g., Refs. 38, 43). While these results differ according to the task, brain state, and cortical structures or layers, a full understanding of all mechanisms that determine the replay coordination remains unclear. One of the mechanisms may be related to brain development (75). Additionally, methodological caveats in the analytic methods may create obstacles in interpretation of results, such as detection of spurious coordinated replay (76).

Replay in Human Brains

Studies of human memory during sleep have a long history (77–79), but the investigation of memory replay only became feasible in the past decade with the refinement of experimental design and analytic techniques (34, 35, 39, 40, 42, 44, 45, 48, 49, 80–84). Several comprehensive reviews of human memory reactivation and human memory recall have been reported (9, 85). Measurements of brain activity from specific brain regions rely on both invasive recordings in patients (e.g., multielectrode array and intracortical EEG) and noninvasive recording in healthy subjects (e.g., MEG and fMRI). Studies of human memory reactivation and replay can be categorized into two kinds: reactivation during memory retrieval (i.e., initiated by an internal or external cue) (86, 87) and reactivation during resting state (e.g., working memory and sleep) (e.g., Refs. 35, 40, 42, 88). The time of onset of the potential reactivation or replay is known in studies that use a cue but not in resting-state studies.

One working hypothesis of working memory is that it involves periodic reactivation of the maintained information coordinated by neural oscillations at theta and gamma bands reflected in MEG or EEG recordings (82, 89–91). Under this theory, “reactivation” refers not to firing of single neurons but rather to neural patterns producing similar classifier outputs at specific times; sequential reactivation of working memory items was also conveniently referred to as generalized “replay.” For instance, it was found that replay was coordinated by the phase of theta oscillations in human MEG recordings, and the degree of theta coordination correlated with working memory performance (90), consistent with the rodent LFP data (92). Additionally, there is an established information flow between the hippocampus and cortex during working memory replay (93).

During memory retrieval, a number of reactivation-associated oscillatory signatures have been identified, varying from low frequency (e.g., theta band) to medium or high frequency (beta, gamma, and ripple bands) (94–97). Content-specific reactivations have been reported in the human temporal cortex, hippocampus, entorhinal cortex, and visual cortex. For instance, Staresina and colleagues (86) used representational similarity analysis (RSA) to investigate the similarity of neural patterns from intracranial EEG (iEEG) between encoding and retrieval of word-color associations in the human hippocampus. Furthermore, in an associative memory task, the same research group showed that hippocampal neuronal firing rates were elevated from 500 to 1,500 ms after the cue onset during successful retrieval. Concurrently, the retrieved target object could be decoded from spike patterns of neuronal population in the entorhinal cortex but not in the hippocampus, and hippocampal firing rate predicted the strength of entorhinal reactivation. Together, these results suggest that the hippocampus coordinates reactivation of the encoded information in the entorhinal cortex (87).

Reactivation or replay of single-unit spiking activity has also been reported during human memory retrieval (98–100). In a recent study, Vaz and colleagues (45) combined a macro-iEEG grid and a microelectrode array to record LFP and single-unit spiking activity from the human medial temporal lobe (MTL) structures including the parahippocampal gyrus and entorhinal cortex. They found that bursts of single-unit spiking activity accompanying ripple oscillations were organized into trial- or memory-specific sequences during successful memory formation, demonstrating that human episodic memory was encoded by specific sequences of neural activity and that memory recall involved reinstating a temporal order similar to that of this activity (Fig. 2A) (45).

Figure 2.

Memory retrieval and replay in human studies. A, left: 2 examples of human cortical spiking sequences for burst events during 2 separate correct encoding trials. Units were colored according to the ordering in the first sequence (left) to demonstrate rearrangement of units to form the second sequence (right). Right: memory encoding (left) and retrieval (right) spike rasters of the corresponding trial during a paired-associates verbal memory task. Inset text indicates the study pair to be memorized (CROW JEEP), the test probe (JEEP), and the verbalized response (CROW). The sequence replay event had a sequential similarity value of 0.42 (P < 0.002) for this encoding-retrieval sequence pair (all panels are modified with permission from Vaz et al. (45); copyright AAAS). B, left: illustration of human memory replay that generalizes to novel experiences. Top right: illustration of magnetoencephalography (MEG) functional localizer and training a decoding model (consisting of a set of weights over sensors) to recognize each stimulus. Bottom right: illustration of human memory replay follows a previously learned rule but not the visually experienced sequence [all panels are modified with permission from Liu et al. (40); CC-BY license]. C: decoding human memory replay with functional magnetic resonance imaging (fMRI). Top left: participants made age judgments of either faces or houses for a sequence of overlaid face-house images. Task rules required keeping in mind the age and judged category of the current and previous trial, called task states. A classifier was first trained to classify the 16 task states from on-task hippocampal fMRI data and then applied to fMRI data recorded during wakeful rest in the same participants to decode potentially replayed sequences of task states (bottom left). Sequences of decoded task states were related to the sequential structure of the task (center) by counting how many steps separated every 2 consecutive decoded states in the true task structure. Top right: schematic illustration of replay analysis procedure. The trained classifier produced a sequence of classifier labels, which was further transformed into a transition matrix that summarized the frequency of paired task states appearing consecutively. Bottom right: classification accuracy during task performance was significantly higher in hippocampal data than in a permutation test. Average distance to the hyperplane for classified states during rest was lowest in the NOISE, followed by PRE and POST conditions [all panels are modified with permission from Schuck and Niv (42); copyright AAAS].

Liu and colleagues (40) recorded MEG signals from participants who were trained on a rule defining an ordering of objects and then presented a novel set of objects in a random order. They found that human replay events occurred in sequences faster than the actual experience, and that the frequency of reverse trajectories greatly increased for rewarded sequences. Furthermore, replay can represent a sequence implied by learned abstract knowledge beyond the visual experience, thereby going beyond sensory representations of the relevant objects (Fig. 2B) (40). In an investigation of whether local replay coincided with spontaneous patterns of whole brain activity, it was found that replay sequences were packaged into transient bursts occurring selectively during activation of the default model network and parietal alpha networks, both of which may support attention and inhibit bottom-up sensory processing (83).

Recently, Schuck and Niv (42) examined the human hippocampus and orbitofrontal cortex (OFC), using fMRI recordings of subjects trained on a carefully designed nonspatial decision-making task. The task consisted of 16 task states, each of which is associated with specific brain patterns. Not only did they report that sequences of patterns decoded from the hippocampus during rest reflected the order of previous experiences, they also identified temporally compressed sequential replay based on the premise that the consecutively decoded task states were closer in the abstract task-state diagram (Fig. 2C) (42). Additionally, hippocampal “sequentiality” correlated with the fidelity of task representations in the OFC as well as with better task performance.

In comparison to the spatial navigation task in animal studies, human memory studies often involve more object items and rules. Furthermore, the majority of human replay studies have focused on awake replay. However, in the few human sleep replay studies (42, 49), the comparison between presleep and postsleep replay was lacking. Bridging the knowledge gap between species remains a top priority in this research field.

Representations and Functions of Hippocampal and Cortical Replay

Hippocampal replay can appear in the form of both forward and reverse replay. A long-standing view of the field is that forward replay is linked to consolidation and planning whereas reverse replay is more related to reward learning (2, 8, 25). Early experiments have shown that the content of hippocampal replay reflects behavioral demands of the task and includes never or rarely experienced shortcut trajectories (60). Hippocampal replay may also reflect future paths to remembered goals or flexible rerouting while rats perform goal-directed navigation in either an open arena or an environment with barriers (31, 101). Similarly, in goal-directed spatial tasks on the radial eight-arm maze, hippocampal replay may predict an animal’s learned behavior or preferentially encode trajectories of the next visited arm (102). These findings therefore suggest that mental replays enable flexible navigation and decision-making (the planning hypothesis). This theory, however, is not completely settled. A recent study has shown that hippocampal replay reflects specific past experiences rather than a plan for subsequent choice (103). In other words, replay is better suited to storing and updating representations of past experiences (e.g., previous goals and non-recently visited locations), supporting memory maintenance and long-term storage rather than guiding immediately subsequent behavior. One possible source of such inconsistency may be the fact that behavioral tasks in some experiments did not include spatially and temporally remote or specific, nonpreferred options, which do not allow the possibility of assessing the effects of those options on replay content (103). Furthermore, reward, navigation history, and learning can affect replay representations (104–108), which also creates additional ambiguity in interpreting these results. Specifically, reward can not only modulate reverse replay (105) but also increase the fidelity of forward replay, while increasing the rate of reverse replays for all possible trajectories heading toward a rewarding location (106). Replay representations can also occur in the context of inhibitory avoidance behavior during fear memory retrieval (109). Overall, these findings suggest that hippocampal forward and reverse replays contribute to establishing a “value map” associated with spatial navigation (2, 24). Additionally, replay ceases to predict forthcoming paths during the course of learning (110). Notably, reward-encoding dopamine neurons from the rat ventral tegmental area (VTA) coordinate their sparse spiking activity with awake hippocampal memory reactivation of reward-approaching paths. However, the coordination of VTA neurons with the hippocampus is greatly reduced during sleep, suggesting that waking memories and sleep-associated memories serve different purposes (111). Furthermore, waking hippocampal-neocortical replay is linked to skill consolidation in human motor learning (112).

Sleep replay may have a functional role different from awake replay. The current view is that awake replay is a potential contributor to memory consolidation and retrieval (113), whereas sleep replay is a key ingredient for sleep-dependent memory consolidation (13, 79, 114). An interplay of sleep-specific oscillations is believed to regulate the hippocampal-neocortical dialog (115, 116). Similar to awake replay, sleep replay during NREM sleep is also associated with hippocampal SWRs, although hippocampal ripples during sleep are additionally coordinated with cortical K-complexes, slow delta waves, and sleep spindles. During REM sleep, there are strong theta rhythms in the rodent hippocampus, reflecting a theta power correspondence with running behavior (20). REM sleep may also be responsible for consolidating emotional memories (117). There is some experimental evidence that sleep reactivation or replay events are less frequent or less structured than during the waking state, supporting the idea that hippocampal-prefrontal reactivation is better suited to support memory integration across experiences during consolidation (69). fMRI recordings from human odor-cued memory experiments have shown that hippocampal and posterior cortical regions were activated during NREM sleep, whereas reactivation during wakefulness primarily activated prefrontal cortical areas (118). A comparative decoding analysis of hippocampal replay events also showed that the degree of “sequenceness” was lower during NREM sleep than wakefulness (119). Furthermore, it has been reported that hippocampal engram cells are orchestrated to constitute a memory trace, with subensembles preferentially reappearing during postlearning sleep and being more likely to be reactivated during memory retrieval (120). Memory reactivation and replay during sleep have been implicated in improving skill learning (32, 33, 46) and boosting creative problem-solving (121). To date, various targeted memory reactivation (TMR) strategies have used auditory or olfactory cue stimulations to strengthen or weaken sleep-related memory reactivation (9, 122–124). Furthermore, selective enhancement of memory has been demonstrated through targeted sleep reactivation (78). However, it is still not completely clear whether replay of preexperience memory during sleep is truly necessary for better postsleep memory and precisely what aspect of memory processing is enhanced.

Diversity and Origin of Hippocampal Ripples

It is important to point out the electrophysiological and functional diversity of hippocampal ripples (54). In rats, it has been shown that SWR bursts emerged in the CA3a and b subregions and spread to CA3c before propagating to the CA1 area (125, 126). Recently, it has found that the CA2 subregion is an initiation zone for SWRs, as the activity of superficial CA2 neurons precedes the activity surge in CA3-CA1 phasic cells and deep CA2 neurons gradually increase their activity before ripples and are suppressed during the population bursts of CA3-CA1 neurons (127). Although hippocampal SWRs can arise from either the CA3 or CA2 subregion, their functional roles are rather different. Specifically, CA2 ripples and ensembles of CA2 pyramidal neurons are important for reactivation of social memory (128). In macaque monkeys, concurrent electrophysiological and fMRI recordings have also shown different subtypes of SWRs with distinct electrophysiological properties (129). A complete delineation of mechanisms and properties of hippocampal SWRs at multiple spatial scales will be vital for understanding of memory replay.

Computational Models for Hippocampal Replay

Complementary to experimental investigations, rapid progress in computational modeling has also contributed to our improved understanding of memory replay. In the early 1970s, Marr (130) suggested that daytime memories may be replayed in the hippocampus during sleep. Models of replay can be traced back to some early concepts of two learning systems (131). In a series of follow-up papers, computational models for spontaneous reactivation, sequence learning, and compressed recall were proposed (132–135). Hasselmo (136) developed a large-scale spiking network of place cells, grid cells, and HD cells to model temporally structured replay of place cell activity during simulated REM sleep, where the population of place cells first activates the HD cells and further causes frequency shifts within the population of grid cells to update a phase representation of location, followed by a new cycle of sequential activation. Jahnke and colleagues (137) proposed a computational model consisting of leaky integrate-and-fire neurons for both learning and replay. Foster and Wilson (24) proposed a feedforward model of replay based on spatial gradients of excitation combined with time-varying inhibition that was able to produce features of both forward and reverse replay. A similar model had been previously developed to explain theta phase precession and theta sequence generation (138). Káli and Dayan (139) proposed a model of neocortical-hippocampal interactions, predicting that offline hippocampal replay helps maintain and retrieve declarative memories in the cortex and that the absence of replay makes consolidated memories fragile in the presence of cortical plasticity. Sequential decision tasks require the brain to evaluate candidate actions for forward planning, where actions can be evaluated by integrating multiple intermediate experiences from pieces potentially never experienced together. Mattar and Daw (140) proposed a normative theory and a computational model of priority memory access to memories of locations sequentially ordered by utility or reward, which help explain multiple functions of replay such as planning, learning, and consolidation.

Spontaneous replay can be a mechanism of consolidation. For instance, it has been found that replay of assemblies can be evoked by sensory-like cues or can emerge spontaneously from the activity fluctuations of a spiking neuronal network with excitation-inhibition (E/I) balance, suggesting that global modulation of neuronal excitability (e.g., wake vs. sleep) may switch between network states that favor retrieval and consolidation (141). Recently, Milstein and colleagues (142) proposed a recurrent spiking network model of hippocampal area CA3 that imposes constraints on network dynamics (such as population sparsity, stimulus selectivity, rhythmicity, and spike rate adaptation); the network was capable of reproducing emergent properties, including online neuronal sequences on the timescale of theta rhythms and offline forward/reverse neuronal sequences that resemble memory replay. Computer simulations have also shown that chainlike recurrent excitatory interactions during learning can determine replay content and bidirectional replay requires the interplay of temporally asymmetric plasticity rule and neuronal adaptation (143). Similarly, internal theta sequences and compressed replay sequences can emerge from the spiking neural network models (144, 145). Additionally, computer simulations have shown that sensory stimulation during the DOWN states of SWS selectively enhances memory replay and consolidation (146).

Together, these modeling results suggest that replay is an emergent property of E/I-balanced spiking networks with proper recurrent connectivity and synaptic plasticity (141–143, 147). Therefore, biologically realistic computational models not only offer new insights into the replay generation mechanism but also provide experimentally testable hypotheses.

METHODOLOGICAL REVIEW OF REPLAY ANALYSIS

Generic Analytic Methods for Memory Replay

The choice of analytic methods for reactivation or replay analysis depends on the signal modality and on the associated temporal and spatial resolution. Nearly all existing analytic methods for memory replay are based on the common premise that there is a similar structure shared by neural data between wakeful behavioral experiences (STATE₁) and a target state (STATE₂, e.g., rest or sleep). Structure may refer to consistent patterns in cell-pair correlation, neuronal firing sequence order, manifold topology, or representational content, etc. However, the structures in the two states need not be completely identical, as the structure in the target state may be distorted, reversed, permuted, compressed, and even randomized with respect to the behavioral experience state. Therefore, to assess the similarity between structures, a common analysis procedure consists of three steps: 1) identify and uncover the structures in both STATE₁ and STATE₂; 2) measure the similarity between the two structures and compute a score; and 3) assess the statistical significance of the score. All steps are method dependent.

Single-unit spiking analysis was often used in studies of memory retrieval or recollection in wakeful states, where the number of simultaneously recorded neurons was small (87, 98–100). In these studies, single-unit spiking was temporally associated with specific objects or virtual location and then reoccurred during successful memory retrieval. This type of analysis, however, is mostly based on the mean firing rate of units and conditioned on the cue presentation.

Pairwise correlation, first used in hippocampal reactivation analysis (15, 16), can be defined by the correlation coefficient between neurons’ firing rates at specific timescales (Fig. 1B). Compared to Pearson correlation, partial correlation assesses how much additional variance in the post-WAKE state can be explained by values in WAKE state while taking into consideration pre-WAKE structure (19, 58, 59). By using the temporal bias, correlation has been used to generate hippocampal sequence-like insights (16). Despite their simplicity, correlation-based methods have several caveats: correlation does not account for the effect of the change in mean firing rate and may be quite sensitive to “noise correlation,” an effect that has been noticed in hippocampal spatial representations (148). Furthermore, correlation does not fully consider spatiotemporal neural patterns and may be subject to artifactual contamination (21, 149).

Template matching can be viewed as a generalized version of correlation and compares the similarity between a mean pattern template of STATE₁ and a candidate pattern of STATE₂ (17, 20, 23, 27, 150). Each template can consist of multiple neurons and multiple time bins, subject to temporal smoothing and warping. However, similar to correlation, template matching is sensitive to the choice of bin size and sensitive to noise or outliers, especially when the number of neurons is small.

Subspace methods such as principal component analysis (PCA) and independent component analysis (ICA) extend pairwise correlation by considering activities of all neurons. The PCA method consists of three steps (65, 151): 1) Compute an N × N correlation matrix from a z-scored population firing rate matrix collected from N neurons for STATE₁; 2) diagonalize the correlation matrix and extract the dominant eigenvalue and eigenvectors; and 3) in the testing phase, project the instantaneous N-dimensional z-scored firing rate vector onto the dominant eigensubspace and compute the reactivation strength within a moving window. The nonnegative reactivation strength has been used as a metric for memory reactivation. However, it is positively correlated with the quadratic power of the z-scored temporal firing rate of individual neurons (see a toy example in Ref. 152). This is often problematic, as the reactivation strength metric can be rather sensitive to outliers in the firing rate vector, especially considering the long-tailed lognormal law in neuronal firing (153).

ICA generalizes PCA by utilizing high-order moment statistics to extract statistically independent components from non-Gaussian signals (154). PCA and ICA methods have been adopted in many memory reactivation analyses (32, 33, 65). However, these two subspace methods have several common limitations. First, they are linear methods and cannot capture statistical dependencies under nonlinear transformations. Second- and high-order moment statistics are estimated based on large wide-sense stationary data (i.e., with time shift-invariant mean and correlation functions), but snapshots of events with a few hundred milliseconds duration can contain outliers (e.g., intrinsic bursting and noise) that skew these statistics. Second, the static mixing and statistical independence assumptions are rarely fulfilled in practice, but the sparseness of reactivations of neural assemblies can be exploited to relax the independence assumption. The meaning of the loadings/weights is poorly interpretable given the sign ambiguity in ICA. Third, the notion that a high reactivation strength is associated with memory reactivation may lead to a false positive. The relative value of reactivation strength does not reveal the quality of memory reactivation or content of replay, nor does it provide a bounded value with respect to the number of observed or activated neurons.

Sequence matching and rank-order analysis is aimed at matching two sequences defined by neuronal firing (e.g., by the peak firing or the first spike). A naive criterion is to count how many neurons are consistent with the rank order. However, this will not account for possibly missing neurons. Instead, the analysis procedure computes the match probability by converting neuronal firing order into a word, compares the match probability between two words (one in STATE₁ and the other in STATE₂), and determines the statistical significance of the match (17, 21, 155). The probability of a repeated sequence of a smaller subset of neurons can be assessed by a set probability, and the chance of a repeated sequence for a long sequence length is very small (156). Sequence or rank-order analysis is based on the premise that the firing order contains the primary source of encoded information. This analysis has been applied in memory reactivation for hippocampal place cells but may not be valid for cortical neuronal firing. The sequence matching analysis is sensitive to spike timing (e.g., first spiking vs. peak spiking) and neuronal firing rates and therefore is insufficient to deal with neurons with multimodal receptive fields.

Population decoding analysis constructs an encoding model for neuronal firing (e.g., Gaussian place cell tuning curve) based on certain statistical assumptions about the neuronal population’s joint spike activity (e.g., independent Poisson firing) and subsequently uses likelihood or Bayesian inference to decode the representation based on the newly observed spiking activity of the population (29, 119, 157). Decoding analyses can be supervised or unsupervised. In the supervised approach, the estimation of the encoding model requires an a priori definition of the “content” or “behavioral correlate” (Fig. 3A), such as the animal’s location in the case of rodent hippocampus (158, 159). Whereas traditional decoding methods depend on clustered spikes, scalable clusterless decoding techniques can facilitate hippocampal decoding fidelity or accuracy based on high-density tetrode or silicon probe recordings (160–165). In clusterless decoding methods, however, spike amplitude is treated uniquely, without accounting for bursting or amplitude shift during behavioral change (e.g., wake-to-sleep transition). In some cases, such as when the animal’s behavior is not well defined or appears high dimensional, the unsupervised approach based on latent variable analysis [such as the hidden Markov model (HMM); Fig. 3B] can be employed (166–170). This approach has also been generalized from sorted spiking activity to unsorted spike multiunit activity (MUA), as well as to ultrahigh frequency (>300 Hz) amplitude derived from field potentials (171) and to calcium fluorescence activity (172) (Fig. 3C). Let p(x_k) denote a prior probability of the decoded state x_k; Bayesian decoding analyses produce a posterior probability map p(x_1:t|y_1:t) of the reconstructed trace or state sequence x_1:t using a Bayes’ rule

$$p\left(x_{1:t}|{\mathbf{y}}_{1:t}\right)={\prod }_{k=1}^{t}p\left(x_k\mathrm{|}{\mathbf{y}}_k\right)\propto {\prod }_{k=1}^{t}p\left({\mathbf{y}}_k\mathrm{|}{x}_k\right)p\left(x_k\right)$$

where p(y_k|x_k) denotes the likelihood model of observed data given the decoded state x_k. Depending on the nature of the behavioral task, the latent state sequences can be either defined a priori or completely undefined. The unsupervised HMM decoding analysis is built upon the spatial correlation structure of populational firing, while imposing a temporal prior on latent state transition.

Figure 3.

Decoding analysis for large-scale electrophysiological and calcium imaging data. A: illustration of Bayesian decoding analysis of hippocampal replay and shuffling analysis. Position was decoded from the temporally binned (20 ms) ensemble activity using the spatial tuning curves obtained during RUN, producing an estimated probability distribution function (PDF) over position at each time bin. The replay sequence was determined as a temporal shift of the peak in the PDFs. A Radon transform was applied to detect linear replay trajectories. The line with the highest score was selected as the putative replay trajectory, and its score was reported as the “replay score” for the event. The statistical significance of a putative replay was assessed by repeating the scoring procedure on 3 shuffled versions of the same data. The true replay score (red line) was compared with each of these distributions, and the largest of the 3 resulting Monte-Carlo P values is conservatively reported as the significance level for the event [Davidson et al. (29)]. B: illustration of a hidden Markov model (HMM) for unsupervised decoding analysis. Neural observations {y_t} are fed to the HMM to infer a latent state sequence {S_t}, from which the state transition matrix can be derived. The spatial topology can be constructed from the state transition matrix, and the state sequence is correlated with the animal’s position to construct a one-to-one “state-position” correspondence map. The HMM is then applied to decode a new sequence from neural observations of a putative replay event. The neural observations may appear in many forms, such as the sorted spikes, unsorted spike multiunit activity (MUA), or ultrahigh frequency (>300 Hz) amplitude derived from field potentials and calcium fluorescence activity. C: illustration of end-to-end decoding based on deep learning such as a convolutional neural network (CNN) [modified with permission from Frey et al. (173); CC-BY license]. The CNN input is fed into image frames. Each image frame can appear as a time-frequency power map computed from the broadband field potential or as a calcium imaging frame.

End-to-end decoding analysis has also been developed based on deep learning approaches (173–175). In these decoding approaches, deep neural networks were first trained on a large number of paired labeled {input, output} samples, where the input could be raw spiking data, frequency-specific features extracted from electrode arrays, or high-dimensional calcium imaging traces (Fig. 3). These methods have been developed for animals’ run behaviors but have not been tested in memory replay analyses.

In rodent hippocampus studies, Bayesian decoding methods have some advantages for several reasons: they are relatively unbiased and insensitive to individual neuronal firing or noisy spikes; they can accommodate unsorted spikes, MUA, and other LFP-derived features; they can deal with neurons with multimodal receptive fields; and they are flexible and can incorporate prior information such as the behavioral sampling bias or preferred position. Additionally, it is possible to integrate multiple sources of information (such as MUA and LFP) to derive the joint posterior probability p(x_t|y_t) of the target variable (171). As well, multimodal decoding analysis can be derived from either a single probability map or multiple probability maps, each weighted by the prior confidence of its respective modality (176, 177). Practically, to alleviate the small-binning effect (i.e., spikes can fall into the same bin or 2 neighboring bins depending on the boundary), one can consider using overlapping bins to obtain a smoother decoded trajectory or estimating a predictive representation p(x_t|y_t, y_{t – 1}) based on the conjunctive temporal information (178). This intuition of the latter idea is justified by the empirical observation of hippocampal prospective representations during rodent run behavior (179). Bayesian decoding methods are not completely bulletproof; they can be biased by behavioral and cell sampling as well as experience-dependent neural plasticity in receptive fields (52).

Analytic Methods Tailored for Human Memory Replay

In recent years, there have been an increasing number of studies of human memory replay derived from MEG, ECoG, EEG, and fMRI recordings. MEG and ECoG signals have a relatively higher signal-to-noise ratio (SNR) compared with EEG. With intracranial or intracortical recordings, template- or correlation-based methods may be applied to single-unit or multiunit activity for the analysis of memory retrieval or replay. With noninvasive recordings, multivariate pattern analysis (MVPA) and representational similarity analysis (RSA) are two widely used methods in the study of human memory replay. A comparison of these two methods and practical guideline can be found in Ref. 180.

MVPA is a multiclass pattern classification (“item decoding”) method that aims to identify one out of N items in time to form an item sequence (e.g., “A-B-C-D”). This type of analysis depends on the choice of supervised machine learning classifier and the choice of features. Usually, selection of features is more important than selection of classifiers. At the first stage of memory coding, classifiers are trained on repeated trials of sequences. At the second stage of testing memory replay, classifiers are used to examine possible reactivation of specific states. A sequential reactivation of a neural state path suggests a replay of learned experiences. Amplitude and phase represent two independent information carriers of neural oscillatory patterns. In the wakeful state alpha, beta, and gamma frequency bands are commonly examined, whereas in the sleep state lower frequencies such as delta, theta, and spindle bands are also considered. To date, MVPA has been used to examine human spatiotemporal EEG patterns across a wide range of frequency bands during sleep and to test whether the activity of previously learned materials was reactivated (181, 182). A similar supervised decoding strategy was also used to examine forward or reverse replay during human working memory. Specifically, the classifier produced one of three visual stimulus categories (plus the null category) but reported no replay in sequential order (183). The major limitation of supervised decoding analyses is the requirement of preidentified “templates” in training sets. The testing performance depends on the number of training samples; it is also necessary to test various window sizes during testing to accommodate the uncertainty in replay onset and duration.

RSA is a multivariate method that extracts information about high-dimensional patterns of neural representations across the brain based on human fMRI and EEG data (184, 185). Briefly, RSA tries to assess the similarity between neural patterns between two conditions and measure the similarity based on a predefined criterion (86) (Fig. 4A). It can be viewed as a generalized version of template matching. Applications of RSA in memory replay analysis all share one common pitfall: sensitivity to the SNR. If neural patterns from one condition are defined by the trial average while the neural patterns from another condition are defined by a single trial, the unbalanced trial numbers between conditions may lead to a large degree of variability across trials (even comparing leave-one-out similarities within the first condition group). Therefore, the result of RSA can be only statistically meaningful if the between-condition similarity is significantly greater than the within-condition similarity. Furthermore, RSA is still susceptible to the pitfalls of any correlation-based methods (180).

Figure 4.

Multivariate analysis for memory replay based on intracranial electroencephalography (EEG) and magnetoencephalography (MEG) recordings. A: schematic overview of memory reinstatement analysis based on human intracranial EEG recordings. For each trial, retrieval and encoding patterns were correlated via a sliding 400-ms window encompassing relative power changes from 2 to 100 Hz. Each instance of correlating a “frequency × time” encoding pattern with a retrieval pattern gave rise to a single correlation bin in a trial-specific reinstatement map [reproduced with permission from Staresina et al. (86); CC-BY license]. B and C: sequential replay analysis pipeline based on temporally delayed linear modeling (TDLM). During training, multivariate decoding models were constructed for each task stimulus with MEG sensor data. During testing, peak accuracy decoders were applied to neural data from the rest scan to derive a decoded [time × state] reactivation matrix. Finally, a 2-step lagged regression approach was applied to quantify the evidence of sequential replay [reproduced with permission from Nour et al. (188); CC-BY license]. GLM, general linear model.

The “sequenceness” or “sequence strength” is an important measure for detecting sequential reactivation of a task structure. It also characterizes the representational power of predicting one state from a previously visited state at a specific time lag (i.e., speed of replay). For sequenceness analysis, temporally delayed linear modeling (TDLM) has become an increasingly popular method used in assessment of human memory replay (40, 186–188). As a multiple linear regression technique, TDLM uses each state’s reactivation time course as a dependent variable and the historical (time lagged) reactivation time sources of all states as predictor variables (188) (Fig. 4B). Once the model is identified, a lag-specific transition matrix is constructed from the task transition structure; the structural sequence can be either forward or reverse direction. For statistical testing, the rows and columns of the forward/backward predictor matrix are randomly shuffled to generate an empirical null distribution. Meanwhile, the TDLM method has also limitations. For instance, it is based on linear predictivity, and the predictive power depends on the model complexity. A modified version of the ordinary least squares regression algorithm that imposes sparsity or structural constraints may further improve the result interpretation.

Because of the lack of cellular resolution, exposition of human replay measures defined by either specific spatiotemporal patterns or sequences of various oscillatory patterns misses a mechanistic link to the underlying neuronal ensemble level, creating possible confounds of these measures and their interpretative power. In principle, these analytic methods used for human memory studies can also be applied to animal studies (e.g., Ref. 189), but there is no systematic effort yet in comparing the efficacy of these methods on animal data. However, large-scale brainwide recordings in mice may provide a good starting point to test these methods (190). Therefore, cross-species or cross-modality validation of these methods will help us understand the link or limitation of specific analytic methods for replay analysis. Furthermore, regardless of analytic methods, replay analysis in animal or human studies will assume sufficient temporal resolution at the replay timescale (typically 20–50 ms); therefore, methods applied to calcium imaging and fMRI recordings with insufficient sampling frequency should be used with caution.

Comparison of Supervised and Unsupervised Methods in Replay Analysis

The analytic methods reviewed thus far can be categorized into supervised and unsupervised learning approaches or a hybrid form of both. Supervised analyses require a behavioral template defined a priori, whereas unsupervised analyses can more or less relax such an assumption. Although the results of both approaches depend on behavioral sampling or data sample size, supervised learning can suffer severely from overfitting. Borrowing ideas from other neural population decoding applications (191, 192), semisupervised or self-supervised methods may potentially incorporate unlabeled or cross-modal data to improve replay decoding analysis. When data allow, it is recommended to conduct both supervised and unsupervised replay analysis and compare their results. Despite the lack of ground truth in replay, we often a get a sense of a “good model” if multiple approaches lead to a similar conclusion. When there is no behavioral reference point while applying unsupervised learning methods, a good model can be validated by its good predictive power (e.g., predictive likelihood), consistency (generalization across sessions and subjects), and exploratory structure discovery.

Challenges for Assessing Statistical Significance of Replay Events

Identification of memory replay from candidate events is often formulated as a hypothesis-testing problem. A null hypothesis, examined at a chosen level of significance, assumes that no statistical relationship and significance exists between the observed data and replay. Because of the lack of ground truth and limited statistical power, detection of memory replay is subject to both false positives (type I error) and false negatives (type II error). There is a fundamental trade-off between these two types of errors. A standard practice is to quantify a putative replay with a goodness-of-fit metric and assess the statistical significance of the event by comparing the metric with those derived from randomly shuffled events. In the case of rodent hippocampal memory replay studies, goodness-of-fit metrics that have been proposed to characterize the fidelity of replay include linear fit R-squared (29), weighted correlation (193), and unweighted or weighted distance correlation (152). To create diversified and independent random events, possible permutation procedures may include neuronal/electrode identity shuffling, spike jittering or temporal shuffling, encoded state/position shuffling, place field shuffling, or channel shuffling (e.g., Fig. 3A; see review in Ref. 8). Determination of putative replay events may heavily depend on the statistical metrics and shuffle procedures (51). Mostly likely, any statistics used for identifying replay will have limitations and create confounds in result interpretation. Although this continues to be an active research topic, a rigorous statistical criterion in practice and community consensus can help replicate results across different recordings, tasks, and species.

These proposed goodness-of-fit metrics are based on the premise that replay patterns are repeated stereotypes of prior behavioral experiences. However, this assumption is not always valid. In general, the observed neural events may have gone through a series of warping and transformations in spacetime because of behavioral variability (Fig. 5A). Postbehavior replay can be viewed as an example of a covert, cognitive event identified by clear structure within extraneous spiking (194). Both behavioral sampling and neuronal subsampling biases impose challenges in assessing the significance of replay events (52). A recent in-depth study showed that rat hippocampal replay patterns represent Brownian motion-like trajectories that were dissimilar to the run trajectories in an open field arena (41) (Fig. 5B). Furthermore, when rats navigated in a linear or circular track, empirical data also showed that some decoded trajectories were better described by a piecewise linear curve than a linear line (Fig. 5C; see also Ref. 165), calling into question the validity of linearity-based measures. This observation suggests that some replayed trajectories had variable speed (e.g., animals often slow down their run speed when approaching a reward site); consequently, such behavioral variability induces time-warping in the neural space.

Figure 5.

Challenges and misconceptions in hippocampal replay. A: flowchart of behavioral-neural transformation and relationship. At the stage of behavioral warping, new random trajectories (orange) can emerge from stereotyped trajectories (blue). At the neural level, the spike activity of the neuronal population can be subject to compression, stretching, or nonlinear warping. During replay, subsets of N recorded neurons are fired, leading to various versions of subsampling in an N-dimensional neural space. By pooling and stitching multiple detected replay events, a neural manifold can be inferred from a model-based method. In a low-dimensional space, the correspondence between neural trajectories in the neural manifold and behavioral trajectories in the behavioral manifold can be established. B: sorted mean (±SD) reactivation speed of sharp-wave ripples (SWRs) and shuffled counterparts for an example session. An equal number of original and randomized events are displayed. Inset: lognormal distribution (dashed line) fits the distribution of mean reactivation speeds for the original events [reproduced with permission from Stella et al. 2019 (41); copyright Elsevier]. C: illustration of constant and variable speed (top) and time warping (middle) in memory replay. Bottom: 2 examples of rat hippocampal replay events, where the trajectories are better fit by piecewise linear lines (M. A. Wilson, unpublished data).

Another common assumption is that replay representations are based on recent wakeful experiences. However, this assumption is overconstrained since the representations may reflect all physically available trajectories within the environment (60) or replay remote (in time or space) experiences. In many of these cases, the putative replay may appear “undecodable” because of the lack of a well-matched template, which can directly influence the interpretation of results. Although the literature has clearly demonstrated the experience dependence of hippocampal replay, there have been reports of biased activity before experience described as hippocampal “preplay” (195–197). The analysis of replay phenomena is dependent on the subset of cells that participate in place coding in a given context (typically 30–50% of the population), the pattern of activity across those cells (typically 1–5% at any given time) providing the place specificity of the coding, and the sequential expression of states or locations over time that reflects the topology of the space that is linked to behavior and experience. The selection of cells that will be active during experience and will contribute to the formation of cognitive maps can be subject to preexisting biases that could be detected during prerun periods (198). The place-specific responses that involve the correlated activity across subsets of those cells cannot simply be the result of biased excitability and would require additional biased connectivity. Synaptic plasticity resulting from prior experience could provide this bias and may contribute to contextual interactions that may reflect important coding functions (199). Robust replay analyses typically involve the statistical characterization of sequential state trajectories that could not result from simple cell selection or pairwise correlation biases that can introduce confounds (21, 149, 200).

In Bayesian decoding, a strong premise is that the place fields constructed in run behaviors are approximately invariant in replay events. Thus far this assumption has been shown to be approximately correct during immediate postbehavior sleep (<4 h), but it is still unclear whether the receptive fields will still be preserved or remapped as sleep progresses further (>4 h). The timescale is a key factor, since sleep may promote sleep-dependent synaptic and neural plasticity (201–203). To reduce reliance on these assumptions, one possible approach is to apply unsupervised decoding methods to identify interpretable latent sequential structures and evaluate the significance of such structures based on permutation tests (i.e., “structure first, content later”) (50, 152). Without the access to labels, these unsupervised decoding methods can still stitch together information from multiple isolated putative replay events and reconstruct the neural dynamics without access to the behavioral ground truth (204). Latent variable approaches and rigorous statistical criteria can help identify replay events during hippocampal SWRs based on state-space analysis (170, 205).

It is noted that changes in brain state can also affect the reactivation or replay measures. For instance, the increased synchrony of sleep dynamics can trivially explain the increased reactivation strength measure. For instance, slow-wave activity exhibits strong infraslow fluctuations (∼40- to 120-s periods) that correlate with altered neuronal synchronization in NREM sleep (206). In vivo and in vitro recordings have also shown that spontaneous cortical activity may preserve repeated motifs or temporally compressed sequential reactivation (207). Therefore, it is important to control the nonspecific bias of reactivation/replay measure and to demonstrate the specificity of replayed patterns.

In animal intracortical and human intracranial recordings, there are also methodological challenges to detecting ripple events because of the lack of a gold standard when the recording site is outside the hippocampal CA1 subfield. One critical issue is to distinguish ripples from high-frequency oscillations or MUA since using different criteria may lead to inconsistent interpretation of results (208). A community consensus and effort would help overcome this obstacle.

Additionally, technical and methodological challenges remain for decoding mental states based on the EEG or fMRI activity. EEG-based decoding analysis is restricted by its limited spatial resolution based on source localization. Additionally, because of limited spatial resolution in fMRI, decoder patterns are often spatially imprecise; uncareful preprocessing and insufficient shuffle control may lead to spurious results or overstated decoding performance (209).

Finally, results of decoding analysis may vary depending on the dimensionality of neural observables (e.g., number of cells, MEG channels, fMRI voxels). In the presence of high dimensionality of observables, it would be important to conduct cell/channel/voxel subsampling analysis to assess the sensitivity of model parameters and gain more confidence and interpretability of the results.

Online Manipulation and Decoding of Hippocampal Replay

Hippocampal ripple oscillations are prominent neural signatures directly associated with memory replay (54, 210, 211), but how do hippocampal ripples contribute to replay? The invention of optogenetics has provided some answers to this question. At the behavioral level, disruption or suppression of hippocampal ripples via electrical or optogenetic stimulation impairs spatial learning and memory (212–215). Methods, results, and caveats of ripple disruption approaches were summarized in a recent review (216). On the other hand, closed-loop optogenetic prolongation of hippocampal SWRs improves spatial working memory (217) and promotes social memory (128). At the neural level, recently formed cell assembly patterns are consolidated by hippocampal offline reactivation, and optogenetic silencing of SWRs impairs consolidation of patterns of a novel but not a familiar place (218). During sleep, SWR-triggered stimulation of prefrontal deep layers enhanced temporal coupling between hippocampal and cortical events and led to a reorganization of prefrontal cortical networks as well as subsequent high performance in a spatial memory task (219).

Since ripples are not always accompanied by replay, it is crucial to know whether online manipulation of ripples can causally affect replay-induced behaviors and hippocampo-cortical representational contents. In other words, we seek answers to not only an “ON or OFF” question but also a “what and how” question, whereas the majority of these manipulation studies did not address how optogenetic manipulation affects the content of replay (220). Recently, however, Fernández-Ruiz and colleagues (217) have discovered that the spike content of optogenetically prolonged ripples was biased by the ongoing, naturally initiated neuronal sequences, whereas the spike content of randomly induced ripples was similar to short-duration spontaneous ripples and contained little spatial information, implying differential roles between different ripple-associated replay. Accordingly, optogenetic prolongation of ripples has been shown to improve memory performance (217). Notably, the opto-induced ripples contain the same cell population patterns as natural and spontaneous ripples (220, 221). We note that the study of Fernández-Ruiz et al. did not manipulate ripples based on the replay content; future experiments that allow replay content-specific (e.g., forward vs. reverse replay; local vs. remote replay) manipulations should provide further insight into the replay function for learning.

To date, technological advances in neural decoding methods have enabled us to examine the content of hippocampal replay in a real-time setting (163, 164, 171). Furthermore, computationally efficient and scalable methods have been developed to enable real-time significance assessment of hippocampal replay events (164) (Fig. 6A) (164, 171). In a replay-specific manipulation during sleep, Gridchyn and colleagues (222) combined online hippocampal replay content decoding with selective optogenetic disruption of hippocampal replay; they reported impairment of specific spatial memory represented by disrupted patterns but no impairment of undisrupted memories (Fig. 6B) (222). This study is the first closed-loop brain-machine interface (BMI) experiment that demonstrated replay-specific memory manipulation.

Figure 6.

Online replay analysis and closed-loop experiments. A: online event detection analysis. Top: unsorted hippocampal ensemble spikes and replay burst detection based on the hippocampal multiunit activity (MUA) and a predetermined threshold (horizontal dashed line). The marked replay onset (vertical lines) was identified after 3 consecutive time bins that crossed the threshold. Middle: starting from the candidate event onset, spatial position was reconstructed from unsorted ensemble spike activity at each time bin (20 ms). The ongoing decoded “spatial trajectory” was assessed based on the weighted distance correlation using online shuffling statistics. Bottom: estimated P value for the online-evaluated replay (black). An accumulative score (red) was computed as the assessment was continuously updated. Replay was claimed to be detected when the accumulative score reached a threshold, followed by a rest of the accumulative score [Hu et al. (164)]. B: hippocampal firing responses during and after the assembly detection periods. A laser pulse was triggered to disrupt the hippocampal firing pattern unless the decoding algorithm could identify with high confidence that the control environment was detected. Top: illustration of a sharp-wave ripple (SWR) within the time windows before and after the event detection. Raster plots show the expected firing of control and target environment-encoding cells in the respective conditions. Bottom: mean z-scored firing rates before and after the assembly detection time (100 ms) in control and target-decoded putative events [reproduced with permission from Gridchyn et al. (222); copyright Elsevier]. C, left: TR_D and TR_I denote 2 sets of direct and indirect M1 neurons monitored during the initial learning, respectively. The magnitude of spike-spike coherence (SSC) for TR_D-TR_D and TR_D-TR_I pairs showed distinct changes during sleep before and after skill acquisition, reflecting a change of functional connectivity induced by the microstructure of sleep reactivation. Right: learning curves from two brain-machine interface (BMI) sessions in the same rat with and without optogenetic inhibition (denoted by OPTO_UP and OPTO_OFF sessions, respectively) during sleep sessions [reproduced with permission from Gulati et al. (223); copyright Springer Nature].

BMI systems in animals and humans have become an increasingly useful tool to study the relationship between replay and skill learning (44, 49, 223). Based on neuroprosthetic learning, Gulati and colleagues (223) found that M1 neuronal reactivations during NREM sleep may help resolve the credit assignment problem (i.e., establishing neural patterns that produce a targeted behavioral outcome) by downscaling task activations, whereas cortical UP-state-triggered optogenetic inhibition decoupled M1 spiking and prevented consolidation (Fig. 6C) (223), suggesting a causal sleep reactivation-behavior relationship. Based on chronic intracortical microelectrode array implants in patients with tetraplegia, Eichenlaub and colleagues (44) used a motor BMI cursor control device to demonstrate awake replay of learned neural firing rate sequences that caused cursor movements. The repeated sequences were different from the control sequences, and specific sequence correlations increased from pre- to posttask rest, suggesting learning-related replay in the motor cortex (44). In the traditional BMI system, the decoding algorithm and feedback control are based on a fixed strategy or policy. Future generations of active and adaptive BMI systems that optimize every step of the closed loop may facilitate future replay studies (224).

DISCUSSION AND OUTSTANDING QUESTIONS

In this “Now and Then review,” we have revisited popular methods for the analysis of memory replay, as well as how experimental and computational approaches jointly have improved our understanding of replay. Being advocates for new research directions in replay, we would like to provide some perspectives on new theories and analytic methods for replay research.

Theories of Memory Replay

Theories are the useful tools by which scientists can make sense of observations and make predictions (225). According to this reasoning, a theory of memory replay can promote high-level abstraction and generate experimentally testable predictions. Stoianov and colleagues (226) proposed a computational theory of the hippocampal formation as a hierarchical generative model entailing characterization of hippocampal reactivations as generative replay. That is, hippocampal sequences are understood as an offline resampling of fictive sequences from the latent generative model, rather than as a verbatim replay of previous experiences (226). This generative view is also conceptually in line with the notion of a hippocampal predictive map (227) and the notion of a state-dependent hierarchical generative model for learning and inference (228).

Theurel and colleagues (229; Theurel D, Chen ZS, Wilson MA, unpublished observations) cast hippocampal planning or goal-directed navigation as an optimal control problem and proposed that forward and reverse replay of spatial trajectories are two mental simulation approaches to solving the control problem (i.e., “Pontryagin’s two-point boundary value problem”), analogous to the forward and reverse shooting methods (i.e., “forward through time from initial boundary” or “backward through time from end boundary”). This control view also allows us to make many experimentally testable predictions.

Multiple Timescales of Brain Dynamics

The structure of replay events is sensitive to the organization of underlying neural activity by brain dynamics at various timescales, which are often coordinated with one another. Within the hippocampus, the SWR events are organized by lower-frequency delta waves and slow oscillations and can be coordinated with sleep spindles (63). Additionally, lower-frequency infraslow oscillations can also impact hippocampal processing during sleep (231). Understanding the role of these interacting dynamics and incorporating them into replay analyses on various neural recordings [(e.g., electrophysiology, calcium imaging, and fMRI blood oxygen level-dependent (BOLD) signals] will be essential to our understanding of the computations and functions of these events. For instance, correlation-based replay analysis may be influenced by correlated firing modulated by various intrinsic rhythms during sleep (such as breathing) (232).

Emerging Analytic Methods for Replay Analysis

To date, Bayesian decoding analyses have been the primary methods for analyzing memory replay of spatial trajectories, whereas MVPA and RSA have been the dominant methods for analyzing human memory replay. Here we outline several promising research directions.

State-space analysis.

The majority of supervised decoding methods follow an “encode-decoding” paradigm, whereas unsupervised decoding methods that relax the “encoding model” assumption represent a new way of thinking about replay. Dynamic latent variable models, such as the HMM and Kalman filter, have been used in replay analysis for the rodent hippocampus (169, 170, 233) and human motor cortex (49). Importantly, once the latent-variable model is identified (in either supervised or unsupervised manner), state-space analysis can enable us to decode and evaluate replay continuously. The discrete-state HMM not only can discover the spatial topology of environments from hippocampal ensemble spikes and reconstruct the behavioral template from sleep-associated neural data (166, 167, 204) but also can identify the latent states of a set of canonical resting-state networks from the spontaneous activity of whole brain activity (83). Based on a continuous-state linear dynamical system, Rubin and colleagues (49) used a real-time steady-state Kalman filter to decode a two-dimensional output from intracranial recordings and compared the decoded output with the predecoded templates of cursor coordinates (Fig. 7A), demonstrating that learned motor patterns are replayed in the human motor cortex during sleep.

Figure 7.

Emerging methods for analysis of large-scale neural spatiotemporal patterns. A: detecting reactivation of memory traces in motor cortex during sleep. Left: template target trajectories in two 1-dimensional coordinates. KF, Kalman filter. Center: one putative replay event occurring during overnight sleep following the motor task completion. 1st row: relative power in low gamma (40–125 Hz) power band from the medial precentral gyrus (PCG) array. 2nd row: multiunit activity (MUA) spike raster. 3rd row: Kalman filter output demonstrating replay of 2-dimensional (2-D) trajectory associated with the target task. 4th row: the instantaneous correlation between the Kalman filter output and successful target trial template. Blue and red lines indicate X and Y positions, respectively, and dashed horizontal lines indicate the “match” threshold. Right: cross-correlation between the Kalman filter template matching signal and the number of channels observed to have ripple activity in a ±5-s window around each putative replay event. Mean cross-correlation for the X dimension (red) and Y dimension (blue) ±95% confidence interval (CI) is shown for each replay speed. For each putative replay event, the time of the peak in the sum of the cross-correlograms for both the X and Y dimensions is shown as gray circles. The sleep replay occurred at a speed of 1- to 4-fold faster than the actual task speed. The median time to the peak is 2.2 s [all panels are modified with permission from Rubin et al. 2022 (49); CC-BY license]. B, left: analysis procedure of representation-to-theta-phase-clustering in sliding representational similarity analysis (RSA). For each cue, 1 neural representation vector (NRV_i) was extracted during cue presentation. For each trial, the changing similarity was calculated between NRV_i and all neural vectors (NV_1−n) during the retrieval period. Right: exemplary trials depicting representation-to-theta-phase-clustering. The circle at top left depicts the preferred theta phases from 8 different trials (1 for each cue representation). This circle is unfolded at top right [all panels are modified with permission from Kunz et al. (91); CC-BY-NC license]. RS, representational similarity; sRS, sliding representational similarity. C: illustration of traveling analysis of 2-D spatiotemporal patterns. Each 2-D neural pattern is preprocessed at a specific frequency band and visualized as an image frame. Principal component analysis (PCA) and singular value decomposition (SVD) can be used to extract the dominant modes from the image frames. Arrows denote vector fields. Temporal activation traces associated with PCA or SVD modes are further identified. During replay analysis, the similarity between these dominant traces and the detected reactivation trace is calculated [all panels are modified with permission from Townsend and Gong (241); CC-BY license]. a.u., arbitrary units. D: illustration of the spline parameterization for unsupervised decoding (SPUD) method. 1st panel: the spline is parameterized by assigning coordinates along its length. The coordinates represent the values of an internal (latent) state. 2nd panel: moment-by-moment decoding of the internal state is done by reading out the parameterization value at the point on the spline closest to the data point. 3rd panel: parameterization of the spline by coordinate α. 4th panel: coloring of neural states via the unsupervised latent variable estimate (i.e., α) [all panels are modified with permission from Chaudhuri et al. (251); copyright Springer Nature].

Traveling wave analysis.

In the brain, traveling waves are spatiotemporal oscillations with steady phase shifts as a function of space and have been widely reported in the hippocampus (234–237) and other cortical areas (see Ref. 238 for a comprehensive review). Traveling waves recorded from the hippocampal formation are state dependent, appearing in the form of theta oscillations during wake and REM sleep (234, 235, 239) or in the form of MUA at the ultra-high frequency band (171). During NREM sleep, hippocampal ripples are also organized as a propagating wave (240), but no relationship has been established between these state-dependent traveling wave patterns in different brain states. A recent study has suggested that human hippocampal theta phases organize the reactivation of large-scale electrophysiological representations of objects during a virtual goal-directed navigation task (Fig. 7B) (91). Although the exact function of hippocampal and cortical traveling waves remains elusive, we can use them to study neural pattern similarity by comparing the dominant wave patterns in both STATE₁ (e.g., memory encoding) and STATE₂ (e.g., memory retrieval). A high degree of variability among wave patterns in STATE₁ is expected. Once high-density neural signals are recorded from two-dimensional arrays, low-rank estimation methods, such as PCA, singular value decomposition (SVD), and nonnegative matrix factorization (NMF), can be employed to extract features from the amplitude or phase information (Fig. 7C) (171, 241).

Temporal sequence analysis.

Identifying precise temporal structures from single-trial high-dimensional neural spike trains is an active research topic (242–244). For high-density LFP or iEEG recordings, detecting temporal sequences can be transformed into a problem of identifying consistent phase shift at certain oscillatory frequency bands. Along these lines, human sleep replay was identified from the peak activation order of high gamma activity based on spatiotemporal “motifs” (81). Temporal sequence structures are also noticeable in resting-state fMRI activity (245) and in calcium imaging data (71, 246), suggesting similar or new temporal sequence detection methods beyond TDLM. However, applications of these advanced methods to replay analysis remain largely unexplored, thereby presenting new opportunities in the future.

Manifold learning and analysis.

Rodent hippocampal population responses are known to represent the spatial topology of the maze during run and replay (167, 193, 247). In contrast, responses of cortical neurons are complex and multiplex (i.e., exhibiting multiselectivity). Because of the high variability of cortical neuronal responses, direct decoding of complex behavioral variables is often difficult. An alternative view of “neural-behavior” mapping is to visualize high-dimensional cortical population responses in a low-dimensional neural subspace, in which low-dimensional neural trajectories are driven by either well-defined or possibly undefined behaviors. Nonlinear dimensionality-reduction techniques, such as manifold learning (248–250), can help extract topological features of low-dimensional manifolds or decode internal neural states (Fig. 7D) (251). Low-dimensional manifolds can encode both physical and abstract variables or learned knowledge in the hippocampus (252). The neural manifold may also change with learning or brain states (251, 253). Several profound questions still remain: Can we identify interpretable structures through neural manifolds during rest or sleep and further decode the associated latent variables? Can properties of these manifolds be used to predict properties of learning such as transferability or generalization?

“Template-free” analysis.

The key benefit of template-free methods is that they allow researchers to examine replay without the generation of a behavioral template (51). Unsupervised learning approaches relax the assumption of predefined templates and aim to discover consistent and interpretable latent sequential structures of high-dimensional neural data (169). Similar to the above-reviewed unsupervised population decoding method, other template-free methods may include unsupervised clustering for spatiotemporal patterns and similarity analysis between two observed patterns (254), or multiview clustering (255) across multiple regions, multiple modalities, and multiple canonical frequency bands, as well as unsupervised deep learning for single-trial neural spiking data (256, 257).

Bidirectional predictability analysis.

Predictability provides a quantification of information flow and Granger causality between brain regions. Multisite and concurrent electrophysiological recordings have enabled us to assess the correspondence of evoked or spontaneous spatiotemporal activities between a large body of brain structures (258). For instance, conditioned on the onset of hippocampal ripples, one research question is whether neural patterns of downstream structures of hippocampus or even brainwide interactions can be predicted from hippocampal neural patterns before, during, and after ripples, or vice versa (190, 259, 260). One can also ask whether the predictability of neural patterns between two brain regions observed during memory encoding is preserved during replay or working memory (67, 93). To address these questions, analytic methods designed for investigating statistical relationships between concurrent neural recordings of different modalities and resolutions (e.g., LFP in conjunction with wide-field calcium imaging or LFP in conjunction with EEG) are urgently needed. Borrowing ideas from motor control and subspace identification (261), we propose an unsupervised state-space analysis paradigm to address the bidirectional predictability problem. Briefly, let x_t denote a continuous-valued latent state variable that drives neural activity y_t (e.g., spikes, LFP, or other modalities) of one brain region Y and neural activity z_t of another brain region Z; we assume that the dynamics of variables {x_t, y_t, z_t} are characterized by a linear dynamical system:

$$\{\begin{array}{c}{\mathbf{x}}_{t+1}=\mathit{A}{\mathbf{x}}_t+{\mathbf{w}}_t\\ {\mathbf{y}}_t={\mathit{C}}_{\mathit{y}}{\mathbf{x}}_t+{\mathbf{v}}_t\\ {\mathbf{z}}_t={\mathit{C}}_{\mathit{z}}{\mathbf{x}}_t+{\text{ϵ}}_t\end{array}$$

where {w_t, v_t, ϵ_t} denote the Gaussian noise variables. The latent state vector x_t = [x_1,t, x_2,t] is decomposed into two components such that dim(x_t) = dim(x_1,t) + dim(x_2,t), where x_1,t is assumed to drive the dependent variable and x_2,t is not. Projecting future neural activity z_t (denoted by Z_f) onto past neural activity y_t (denoted by Y_p) would enable us to assess Y → Z predictability, whereas projecting future neural activity y_t (denoted by Y_f) onto past neural activity z_t (denoted by Z_p) would enable us to assess Z → Y predictability. Once the model parameters and latent subspace are identified by closed-form solutions, one can use the Kalman filter to predict the dependent variable Z_f (or Y_f) based on the predictor variable Y_p (or Z_p) (261). Additionally, this generalized state-space analysis paradigm can be used for inferring the latent state x_t based on either complete or partial measurements from {Y, Z} to examine replay of abstract behavioral variables.

Outlook

Thus far we have reviewed how concepts of memory replay have been transformed with the development of new technologies, computational modeling, and analytic methods. We argue that we will reach the highest-level understanding of memory only if we are capable of controlling and predicting reactivation or replay patterns (“controllability” hypothesis). To achieve this goal, seamless integration of multisite recordings, real-time replay decoding, and closed-loop manipulation experiments will be pivotal to delineate the role of memory replay for cognitive and motor functions. It is likely that the controllability hypothesis is rather different between awake and sleep states. To help control behavioral variability, designs of virtual reality experiments may be useful for studying awake replay in BMI experiments. Furthermore, bidirectional control of replay (enhancement vs. impairment) would enable us to validate the causal role of sleep replay in any type of postsleep learning.

Another important research direction is to understand the translational impact of memory replay and how different characterizations of replay distinguish a pathological brain from a healthy control. It was found in the mouse model of schizophrenia that hippocampal ripple-associated replay was impaired (262). Recent work also showed evidence of deficits in memory replay from a mouse model of Alzheimer’s disease (263). On the human research side, a recent neuroimaging study has shown that schizophrenia patients tend to have abnormal neural replay of memories (188). Additionally, measuring speed of mental replay of movies may give insight into how healthy subjects versus posttraumatic stress disorder (PTSD) patients access their memories (264).

Furthermore, an improved understanding of memory replay may have a direct impact on many other questions in cognitive neuroscience, including REM sleep dreaming (50, 265–267), ponto-geniculo-occipital waves (PGO) waves (268), insight (“aha” moment), and creativity (121, 269). Equally important, neuroscientific knowledge may be translated to artificial systems and machine learning, such as reinforcement learning (227, 270–273) and continual learning (274).

Finally, we conclude the review with a list of outstanding questions that might motivate and guide future research on replay.

• What is the replay generation mechanism? Is replay internally generated by the hippocampus or externally generated by the cortex? Or are there multiple mechanisms for memory replay? There is experimental evidence that mEC inputs are critical to hippocampal CA1 awake replay (275), but a general answer to the question for hippocampal sleep replay or cortical replay is still missing. Cholinergic inputs can impact hippocampal ripples (215), but it is still unclear what differential roles synaptic plasticity, E/I balance, and neurotransmitters may play in awake and sleep replay.

• When and where do representations of hippocampal replay and neocortical replay become temporally coupled or independent? What is the underlying control mechanism? The same question can equally apply to subcortical structures, as brainwide activity is often coordinated with respect to the onset of hippocampal ripples (190, 276). However, it is still unclear whether the contents of subcortical representations provide independent or conditionally conjunctive meanings on cortical representations.

• Can hippocampal replay predict representations of neocortical replay? Or can neocortical representations predict the content of hippocampal replay? Some evidence has shown that sound-based auditory cortical neuronal ensembles predict reactivations of hippocampal CA1 firing during SWR (67). The abovementioned bidirectional predictability analysis could provide quantitative answers to these questions.

• Can all subcortical structures have replay? The amygdala in the limbic structure is functionally linked to emotional memories. Early work has reported coordinated reactivations between the amygdala and hippocampus during NREM sleep (66), but sequential reactivations have not yet been documented. In the case of thalamus, the findings vary highly depending on the specific thalamic nuclei. The rat anterior dorsal thalamus can represent the animal’s HD, and multiple lines of evidence have shown coactivation of neuron pairs and neuronal sequences during both SWS and REM sleep (277, 278). In the rat mediodorsal (MD) thalamus, the MD activity was suppressed during hippocampal awake ripples and NREM spindle-uncoupled ripples, but the MD activity increased during NREM spindle-coupled ripples (279). Additionally, in the rat midline thalamus (nucleus reuniens), sparse thalamic activations were reported in coordination with hippocampal ripples, cortical delta activity, and spindles during NREM sleep (280), but no sequential sleep replay has been documented yet. Future technological advances may enable us to investigate replay across a wide range of subcortical circuits for diverse forms of memories.

• Can hippocampal replay represent another subject’s experiences relative to self? Can the hippocampus replay social memories? This question naturally arises since rat hippocampal place cells can encode another animal’s position (281), but no result has been reported on those hippocampal place cells during SWRs. There is also some experimental evidence that hippocampal CA2 neurons active during social exploration are reactivated during NREM sleep (128), whereas CA2 neurons that receive projections from hypothalamic supramammillary nucleus (SuM) neurons are highly active during REM sleep, and optogenetic silencing SuM → CA2 impairs social memory consolidation (282).

• Do the replay structures during NREM sleep that follow immediately after wakefulness differ from those during NREM sleep that follow immediately after REM sleep? Answers to these questions may enable us to assess the differential roles of replay in NREM and REM sleep and to investigate whether they have complementary functions.

• What determines the speed of replay? What are the external factors that control the compression factor for hippocampal and cortical replay? Can the nature of tasks or the context of brain states modulate the compression factor? Does the replay speed affect brain functions? Answers to this set of questions will provide more guidelines to identify and search for replay events given specific conditions of task nature and brain state.

• Is the segmented structure of extended replay events across multiple SWRs relevant to the communication with extrahippocampal structures, through synchronization with brain rhythms such as alpha, sigma, and beta oscillations? Is there a longer-timescale replay structure that extends beyond multiple SWR events? Within a single SWR, hippocampal CA1 and CA3 neurons are coordinated via gamma synchronization to enable memory reactivation with the hippocampal network (283), but it is unclear whether other oscillations can play a similar coordinating role in a larger brain network.

• Can replay appear in a hierarchically organized manner as in the hierarchical structure of complex decision-making with spatial or nonspatial task components (e.g., Refs. 284, 285)? In analog to the hierarchy of natural language (ranging from “words,” “phrases,” “clauses” to “sentences”), memory can appear in a hierarchical form with composition of independent segments. Recent evidence has suggested that coherent activations of the default mode network during rest support a hierarchy of representations of episodic memories or replay cascades, forming a cascaded memory systems model (286). Answers to this and the previous set of questions depend on our ability to assess extended memory replay at a longer timescale and further assess the replay content under multiple contexts or reference frames (i.e., the hierarchy has different roots, each is represented by a distinct environment or task but also shares some common components such as the same spatial topography or a common task state).

• Are motor, movement, or action state representations coordinated with hippocampal replay events? Can we use virtual reality (VR) to decouple self-motion movement from action-state representations? Rodent hippocampal replay reflects a trajectory with locomotion, which is associated with velocity-transformed and discrete action state (such as stop, turn, speed up or down). It has been well known that hippocampal CA1 neurons can represent a continuous spatial representation across different states of motion (movement and immobility). During SWRs, separate representations of experiences encoded by different motion-associated activity patterns are reactivated (287). However, it is unclear whether cortical or subcortical representations (e.g., RSC or basal ganglia) also coordinate hippocampal representations in encoding these action states. The answer to this question can reveal mechanistic insight into sensory-motor representations during replay and its link to reinforcement learning.

• Can characterizations of memory replay be used as a biomarker for diagnosis of memory deficit or dementia? In light of recent rodent and human findings (188, 262–264), answers to these questions may establish the mechanistic link between replay and a wide spectrum of brain disorders.

In summary, we believe that integration of multisite recordings, custom analytic methods, real-time replay analysis, and closed-loop experiments will help fully uncover circuit and computational mechanisms of memory replay, revealing their functional significance for cognitive and motor functions. With incremental advances in some or all of these techniques, our understanding of memory replay and the above outstanding questions will continue to evolve.

SUPPLEMENTAL MATERIALS

Supplemental Figs. S1 and S2: https://doi.org/10.5281/zenodo.7552954.

GRANTS

This work was funded by the National Institutes of Health (MH118928 to Z.S.C. and M.A.W.; DA056394 to Z.S.C.).

DISCLOSURES

No conflicts of interest, financial or otherwise, are declared by the authors.

AUTHOR CONTRIBUTIONS

Z.S.C. and M.A.W. conceived and designed research; Z.S.C. analyzed data; Z.S.C. interpreted results of experiments; Z.S.C. prepared figures; Z.S.C. drafted manuscript; Z.S.C. and M.A.W. edited and revised manuscript; Z.S.C. and M.A.W. approved final version of manuscript.