Deciphering Neural Codes of Memory during Sleep

Abstract

Memories of experiences are stored in the cerebral cortex. Sleep is critical for consolidating hippocampal memory of wake experiences into the neocortex. Understanding representations of neural codes of hippocampal-neocortical networks during sleep would reveal important circuit mechanisms on memory consolidation, and provide novel insights into memory and dreams. Although sleep-associated ensemble spike activity has been investigated, identifying the content of memory in sleep remains challenging. Here, we revisit important experimental findings on sleep-associated memory (i.e., neural activity patterns in sleep that reflect memory processing) and review computational approaches for analyzing sleep-associated neural codes (SANC). We focus on two analysis paradigms for sleep-associated memory, and propose a new unsupervised learning framework (“memory first, meaning later”) for unbiased assessment of SANC.

Memory, Sleep and Neural Codes

Memory is referred to the capacity of an organism to encode, store, retain and retrieve information. It can be viewed as a lasting trace of past experiences that influences current or future behavior. Memory uniquely defines a sense of self-identity and includes all information of ‘who’, ‘what’, ‘when’, and ‘where’ of our life experiences in the past and present, remote or recent. The time span over which information in memory remains available varies from seconds (short-term memory) to years (long-term memory). Long-term memory is often divided into two types: explicit or declarative memory (“knowing what”) and implicit or procedural memory (“knowing how”). Declarative memory also includes episodic memory (see Glossary), semantic memory (knowledge) and autobiographical memory.

Episodic memory stores details of specific events in space and time, each associated with unique multimodal, multi-dimensional information content. The hippocampus plays a pivotal role in spatial and episodic memory [1]. Sleep is important for learning and memory [2–6]. On average, the human being spends about one third of their lifetime during sleep, whereas rodents sleep 12–14 hours a day. Memory consolidation occurs in sleep, during which a short-term memory can be transformed into a long-term memory. Sleep deprivation deteriorates performance in memory tests and negatively affects attention, learning, and many other cognitive functions [6,7]. A fundamental task in the study of memory is to understand the representation of sleep-associated neural codes (SANC) that support memory processing. Simply put, how can we read out memory during sleep? Since sleep-associated memory is influenced by WAKE experiences, how do we identify and interpret memory-related neural representations during sleep in an unbiased way?

To address these questions, neuroscientists record neuronal ensemble activity from the hippocampus and neocortex in sleep sessions before and after a behavioral session. In animal studies, “neural codes” are acquired by implanting multieletrode arrays to record in vivo extracellular neuronal ensemble spike activity [8–12]. In human studies, measurements of brain signals are acquired through non-invasive EEG or fMRI recordings [13–16]. For the purpose of this article, we will review important work in both research areas, with more focus on rodent studies.

At the neuronal ensemble level, the computational task of identifying memory-related neural representations of population codes (i.e., neural activity patterns that reflect memory processing) in sleep remains challenging for several important reasons: First, although local field potentials (LFPs) reveal important information of circuits at a macroscopic scale, they lack the cellular resolution to reveal sleep memory content. Second, sleep-associated ensemble spike activities are sparse (low occurrence) and fragmental in time. Third, the magnitude of neural population synchrony, measured as the spiking fraction of all recorded neurons during each network burst, follows a lognormal distribution: strongly synchronized events are interspersed irregularly among many medium and small-sized events [17]. Finally, the lack of ground truth makes the interpretation and assessment of memory-related neural representations difficult. In the past two decades, although a number of systematic studies have examined memory content in SLEEP compared to WAKE, many memory-related research questions remained elusive. In the next section, we review some experimental and computational strategies to answer these questions.

Hippocampal-Neocortical Circuits in Sleep

During sleep, the brain is switched into an “off-line” state that is distinct from wakefulness at both microscopic (spike timing) and macroscopic (e.g., neocortical EEG oscillations) levels. In different stages of sleep, such as slow wave sleep (SWS) and rapid eye movement (REM) sleep, brain activity varies and the cerebral cortex exhibits a wide range of oscillatory activities (Box 1) [18]. During SWS, the neocortex is known to oscillate between UP and DOWN states [19]. During neocortical UP states, increased population synchrony of pyramidal cells in hippocampal-neocortical networks is accompanied by hippocampal sharp wave-ripples (SWRs, Box 1, Figure 1b) [20,21]. Most animal studies on memory and sleep use the rodent model. A widely adopted spatial memory paradigm is to let rodents freely forage in a closed environment. During active exploration, many hippocampal pyramidal neurons show localized spatial tuning, or place receptive fields (RFs) [22]. Notably, many hippocampal pyramidal neurons are also responsible for non-spatial sequence coding [23,24], as well as conjunctive coding of both spatial and non-spatial memories [25]. During sleep, in the absence of external sensory input or cues, the hippocampal network is switched to a state that is mainly driven by internal computations.

Box 1. Brain rhythms in sleep.

Slow oscillation (0.5–1 Hz)

During SWS, neocortical activity displays synchronized slow waves between 0.5 and 1 Hz, which are associated with alternation between widespread hyperpolarization and reduced neuronal firing during the DOWN state, and UP states which are associated with widespread depolarization and increased neuronal firing. The cortical slow oscillations also reach and impact hippocampal and thalamic circuits.

Delta wave (1–4 Hz)

High amplitude brain wave with frequency of oscillation between 1 and 4 Hz. It is prominent during SWS.

Theta oscillation (4–9 Hz)

During REM sleep, the rodent hippocampus exhibits theta oscillations similar to those seen during wakeful exploration.

Spindle oscillation (9–15 Hz)

During SWS, the thalamus and neocortex exhibit brief bursts of EEG oscillations between 9 and 15 Hz, typically lasting 0.5–2 seconds. Sleep spindles often occur in the neocortical UP state and are temporally aligned with hippocampal ripples.

Gamma oscillation (35–120 Hz)

During SWS, human and rodent EEG recordings show gamma oscillations in low (35–50 Hz) and high (60–120 Hz) frequency bands.

Hippocampal sharp wave-ripples (SWRs, 150–300 Hz)

The SWR complex consists of large amplitude sharp waves in the hippocampal LFP and associated fast LFP oscillatory activity filtered between 150 and 300 Hz, typically lasting 50–100 milliseconds. Bursts of SWRs may last up to 400 milliseconds.

FIGURE 1. Study of Rodent Hippocampal Memory and Sleep.

(a) A standard study paradigm for rodent hippocampal memory consists of pre-RUN sleep, RUN/behavior, and post-RUN sleep. (b) Classification of sleep stages from EMG, cortical LFP (Delta power), hippocampal ripple power, and cortical theta/delta power ratio [21]. (c) Rodent hippocampal population spike activity during RUN on a linear track. (d) Rodent hippocampal LFP and SWRs during post-RUN SWS, and the associated spatiotemporal spike pattern that shows a similar temporal order (“replay”) (reproduced with permission [18]).

In a seminal study, Pavildes and Winson [8] first reported that the activity of rat hippocampal place cells in the awake state influenced the firing characteristic (e.g., firing rate and burst rate) in subsequent sleep episodes. Wilson and McNaughton [9] extended the first-order to second-order statistical analysis and demonstrated that rat hippocampal place cells that were co-active during spatial navigation exhibited an increased tendency to fire together during subsequent sleep, whereas neurons that were active but had non-overlapping place RFs did not show such increase. This effect declined gradually during each post-RUN sleep session. Kudrimoti et al. [11] and Nádasdy et al. [12] further studied spike patterns involving multi-neuron patterns (e.g., triplet) during sleep. These studies revealed the temporal relationship between hippocampal replays and SWRs [12], as well as the memory trace decay time [11]. Additional studies also revealed that rodent hippocampal spatiotemporal patterns in SWS reflected the activation patterns or temporal order in which the neurons fired during spatial navigation [10,12,26,27]. Specifically, subsets of hippocampal neurons fire in an orderly manner at a faster timescale within SWRs, with either the same or reverse order as in active navigation. In a linear track environment, such population burst events, depending on their contents, can be categorized as “forward” or “reverse” replay—referred to as reactivated hippocampal sequences of the run trajectory (Figure 1c). Such hippocampal replay events are prevalent in SWS [26], quiet wakefulness [28,29], and “local sleep” (also known as “microsleep” —a phenomenon that neurons go offline in one cortical area but not others in an awake yet sleep-like state) [30], although the functional roles in each of those states are most likely to be different. The engagement of the replay process, the frequency of activation, and the time during which replay occurs can affect subsequent performance on behavioral tasks or learned skills. In a series of studies [26,31,32], researchers have found that following RUN experiences, hippocampal place cells reactivated in a temporally precise order repeatedly in SWS and REM sleep. Unlike SWS, the firing-rate correlation in REM sleep was not related to the preceding familiar RUN experience (possibly due to the trace decay during the interleaving SWS) [11], and the memory replays occurred more frequently for remote yet repeated RUN experiences [31]. These findings suggest that reactivated hippocampal sequences in post-RUN sleep consolidate memory of RUN experiences, and that SWR-associated hippocampal activity may contribute to this process.

A central hypothesis of memory consolidation is that the hippocampus and neocortex interact with each other through the temporal coordination of neuronal activity in the form of slow oscillations, SWRs, and sleep spindles [33–39]. While memory reactivation during sleep has been mainly reported in rodents, including the rat primary visual cortex (V1) [36], the barrel cortex [40], the posterior parietal cortex [41], the medial prefrontal cortex (mPFC) [42,43], the primary motor cortex (M1) [44,45] and the medial entorhinal cortex (MEC) [46]; general phenomena of neocortical memory reactivation were also reported in the other species, such as in the song bird during sleep [47] and in the macaque monkey during rest [48]. The assumption of hippocampal-neocortical interactions during sleep would naturally suggest examining the interactions of simultaneously recorded hippocampal-neocortical ensembles [36,38,41,46]. Comparing the spatiotemporal neural patterns in each area during both WAKE and SLEEP would leverage our knowledge of hippocampal spatial coding and further our understanding of the role of hippocampal-neocortical memory processing during sleep. In one study of rodent hippocampal-visual circuits [36], researchers found that memory reactivation in the V1 was temporally coordinated with memory reactivation in the hippocampus during SWS (Figure 2a,b). In another study [37], researchers found that auditory cues associated with neural activity during learning enhanced replay of the same neural patterns if the same auditory cues were presented during sleep. Although the auditory stimuli did not affect the number of replay events, the replay content was biased by the respective sounds (Figure 2c), suggesting mechanisms of selective memory enhancement in sleep. In another recent report on a similar study [38], researchers simultaneously recorded ensemble spikes from the rat auditory cortex and hippocampus while presenting task-related sounds during sleep (Figure 2d), and found that the patterned activation in auditory cortex preceded and predicted the subsequent content of hippocampal activity during SWRs (Figure 2e), while hippocampal patterns during SWRs also predicted subsequent auditory cortical activity. Consistently, delivering sounds during sleep biased the auditory cortical activity patterns, and sound-based auditory cortical patterns predicted subsequent hippocampal activity. Among many neocortical structures, the MEC is an important neocortical circuit that sends input to the hippocampus, and plays an important role in spatial navigation and memory processing. Two recent rodent experimental findings have showed that there was coordinated replay between hippocampal (CA1) place cells and grid cells at deep MEC layers (L4/5) during rest [49]; however, the cell assemblies at superficial MEC layers replayed trajectories independently of the hippocampal reactivation rest or sleep, suggesting that the superficial MEC can trigger its own replay events and initiate recall and consolidation processes independent of hippocampal SWRs, whereas deep MEC layers are directly influenced by hippocampal replay [46].

FIGURE 2. Dissection of Hippocampal-Neocortical Memories during Sleep.

(a,b) Neuronal firing sequences in rat V1 (a) and hippocampus (b) during RUN and POST-RUN SWS episodes. Lap: population neuronal firing pattern during a single running lap on the left-to-right trajectory. Each row represents a cell and each tick represents a spike. Avg: template firing sequence obtained by averaging over all laps on the trajectory. Each curve represents the average firing rate of a cell. Cells were assigned to numbers 0, 1, etc. and then arranged (01234567) from bottom to top according to the order of their firing peaks (vertical lines). Frame: the same population firing patterns in a POST-RUN SWS episode. Triangles and circles denote the onset of UP and DOWN states, respectively. Seq: firing sequence in the frame. Spike trains were convolved with a Gaussian window and cells were ordered (0132567) according to the peaks (vertical lines) of the resulted curves [36]. (c) Auditory sound (L, in red, indicating a left turn) biased the hippocampal reactivation during SWS [37]. In the raster plot, spikes from place cells with place fields on the right side of the track are blue, and left-sided place fields in red. Place fields are ordered from top to bottom by their location on the track (right → left side). Prior to sleep onset, the rat was resting in the sleep chamber. The reactivation event in the green dashed box is shown to the right. (d) Sound-biased auditory cortical neuronal ensembles (green) predict reactivations of hippocampal neurons (orange) during SWRs. Pink bars indicate sounds; cyan bars indicate detected SWRs. Top black trace is ripple-filtered LFP in hippocampal CA1 [38]. (e) Quantification of prediction gain of using sound-based pre-SWR auditory cortical (AC) ensemble spike patterns to predict hippocampal CA1 firing. Data is significantly different from the shuffled statistics (n=96) [38]. All figures are reproduced with permission.

Overall, these findings suggest that the neocortex communicates with the hippocampus about “when” and “what” to reactivate memory during sleep, and the activation of specific cortical representations during sleep influences the consolidated memory contents. Nearly all reported findings are correlation-based observations. The first direct causal evidence of hippocampal-cortical coupling in memory consolidation during sleep was demonstrated physiologically and behaviorally in [39]. Importantly, it was found that reinforcing the endogenous coordination between hippocampal SWRs, cortical delta waves and spindles by timed electrical stimulations resulted in a reorganization of the mPFC network, along with subsequent increased prefrontal task responsivity and high recall post-sleep performance [39].

In addition to considering the specific ensembles that participate in reactivated memory patterns, the temporal structure of memory patterns can also vary by brain state [25]. The reactivated patterns during SWRs closely resembled the compressed structure of encoded memory observed within individual cycles of the theta rhythm during awake behavior in the hippocampus [12,50]. During SWS, the hippocampal-neocortical memory reactivation occurred at a faster time scale, with reported time compression factors of 9–10 in the rodent hippocampus [26], and compression factor of 6–7 in the rodent mPFC [42], although there was also inconsistent report on no evidence of time compression or expansion in other rodent brain regions [40]. In REM sleep, the speed of hippocampal replay is close to or slightly faster than the actual run speed [31]. Notably, spatial memory was impaired by selective suppression or disruption of SWRs by electrical or optogenetic stimulations [51–53], suggesting the causal role of SWRs for hippocampal replays during the off-line state.

In contrast to animal research (most exclusively in rodents), human studies have provided more limited access to the content of sleep-associated memory at the neuronal ensemble level. Nevertheless, memory study of human subjects, such as H.M. [54], provides a unique and valuable perspective far beyond rodent studies. For healthy or diseased human subjects, semi-invasive ECoG recording or noninvasive EEG/MEG recordings and fMRI imaging have been widely used in sleep studies [13–16]. However, none of them directly measure single neuronal activity, which therefore poses great challenges in studying sleep’s memory content. When single units are available, different cortical areas display distinct yet localized spatiotemporal spike and LFP patterns [55]. In a remarkable study, researchers used fMRI and machine learning tools to decode (or more precisely, “classify”) visual imagery of brain patterns in the visual cortex (V1, V2 and V3 areas) during REM sleep, as compared to spatiotemporal brain patterns of fMRI imaging during wakeful state [56]. This provided the first clue about the content of human dreams (Figure 3). In a sleep study on epilepsy patients, it was reported that single-unit spike activity in the MTL was modulated around REM onsets, which was similar in REM sleep, wakefulness and controlled visual stimulations, suggesting that REM during sleep rearranged discrete epochs of visual-like processing as during awake vision [57].

FIGURE 3. Decoding the Content of Visual Imagery during Human REM Sleep.

(a) fMRI data were acquired from sleeping participants simultaneously with polysomnography. Participants were awakened during sleep stage 1 or 2 (red dashed line) and verbally reported their visual experience during sleep. The fMRI data immediately before awakening (9 s) were used as the input for main decoding analyses (sliding time windows were used for time course analyses). Words describing visual objects or scenes (red letters) were extracted. The visual contents were predicted using machine-learning decoders trained on fMRI responses to natural images. (b) During the training phase, words describing visual objects or scenes were first mapped onto synsets of the WordNet tree [a dictionary of nouns, verbs, adverbs, adjectives, and their lexical relations]. Synsets were grouped into “base synsets” located higher in the tree. Visual reports (participant 2) are represented by visual content vectors, in which the presence or absence of the base synsets in the report at each awakening is indicated by white or black, respectively. Examples of images used for decoder training are shown for some of the base synsets. During the testing phase, a pairwise or multi-label decoder is applied to awakening event for predicting the visual object label (reproduced with permission [56]).

Despite rapid progress in experimental investigations and growing knowledge of hippocampal-neocortical circuit mechanisms, answers to many research questions remain completely or partially unknown. Since most “content” questions are driven by statistical analyses of SANC, it is imperative to develop computational paradigms to investigate the representation of sleep-associated memory.

Computational and Statistical Methods: Strengths and Limitations

In WAKE, how do we interpret the representation (“meaning”) of neural codes? This is formally established by the neural encoding problem. Given the measured sensory input or motor behavior associated neural responses, we can identify the meaning of neural spike patterns in a supervised manner. In SLEEP, the essential computational question is: what and how much information can we read out from memory-related neural representations during sleep? Since the representation of an experience is sparse, the answer to this question is nontrivial. To date, several computational methods (Box 2) have been developed to analyze SANC derived from hippocampal-neocortical circuits. However, most of methods cannot identify the “meaning” (content) of memory other than merely establishing significant “similarity” (by correlation or matching) of spike activities between WAKE and SLEEP. In other words, they can reveal the presence of memory replay, but not necessarily the content of replay. As a general principle of deciphering sleep-associated memory content, it is critical to develop statistical methods that allow studying memory without first having to establish how brain activity encodes behavioral variables such as spatial locations or movement kinematics. During sleep, the brain is normally disconnected from the external sensory world, although sensory stimulation may induce physiological changes in sleep-associated memory [37,38,70]. The content of sleep memory lacks behavioral readout; therefore it is preferred to use computational methods that do not require behavioral measurements a priori.

Box 2. Methods for analyzing sleep-associated spike activity.

Correlation Analysis

Correlation analysis computes the strength of Pearson correlation between two neurons based on their firing activities in WAKE and SLEEP; the strength of zero-lag, co-activation of pairwise cell firing determines the similarity between neural firing patterns in WAKE and SLEEP [9]. The “explained variance” method assesses how much additional variance in post-SLEEP correlation can be explained by values in WAKE, while taking into consideration of pre-SLEEP structure [11].

Template Matching

Template matching compares two spike count matrices (arranged in cell-by-time) that are temporally binned and smoothed [12,31,42], and assesses whether the reactivation in pairwise activity is coherent across neuronal ensembles. The outcome of template matching is sensitive to temporal bin size, and its correlation strength varies between different compressed timescales.

Sequence Matching

Sequence matching is a combinatorial method for examining sequential firing patterns of population spike activity. It computes the match probability by converting neuronal firing orders into a word, and compares the match probability between two words (one in WAKE and the other in SLEEP), and determines the statistical significance of match [26,32]. The sequence matching method is sensitive to spike timing (and consequently to spike detection and sorting) and the number of activated cells in SLEEP.

Principal Component Analysis and Independent Component Analysis

Principal component analysis (PCA) extends the correlation method and assesses the similarity between two correlation matrices between WAKE and SLEEP [43,58]. It computes the reactivation strength between two templates and provides an instant-by-instant resemblance measure between WAKE and SLEEP. A large value of reactivation strength indicates a good similarity (Fig. 4a). However, the reactivation strength is positively correlated with the neuronal firing rate and does not directly reveal the memory content of ensemble firing patterns. The PCA method assumes that the correlation statistic is stationary within both WAKE and SLEEP, which is the strongest limitation in the presence of nonstationary neuronal spiking data. Independent component analysis (ICA) extends the PCA method and finds a linear projection space that separates statistically independent sources. The ICA method is conceptually similar to the PCA method except that there is an additional ICA step followed by PCA [59]. Both PCA and ICA belong to the linear subspace method, therefore they cannot capture any nonlinear transformation, and their reactivation strengths are positively correlated with the quadratic power of temporal firing rate per se.

Topology Analysis

Algebraic topology is a mathematical tool that was borrowed to study hippocampal neuronal coding for spatial topology [60–62]. It is aimed to compute abstract topological properties from the derived topological object and use those to derive a group relationship within neurons.

Population Decoding

Population decoding is a computational approach that uses statistics or information theory to extract quantitative information from neural ensemble spike activity [63]. The population-decoding approach makes certain statistical assumptions about the population spike activity (e.g., independent Poisson assumption) and employs likelihood or Bayesian inference to decode the content of population codes. One class of decoding approach is supervised, which requires the receptive field information about individual neurons [64,65]; another class of decoding approach is unsupervised, which requires no receptive field or behavior measure [66–68] (Fig. 4b). Systematic comparisons of these two types of population-decoding methods in a sleep-associated hippocampal memory study are reported in [69].

Here we would like to discuss two quantitative approaches for the analysis of SANC. In the first approach, the principal component analysis (PCA) method [43,58] (Box 2, Figure Ia) does not explicitly define the neuronal RF. Instead, it computes the correlation matrix of cell assemblies in a TEMPLATE epoch and then further compares it with another spatiotemporal population spike matrix from the MATCH epoch—moving the population spike vector in time would allow us to assess the time-varying reactivation strength. The basic statistical assumption is that the spatiotemporal patterns of a specific behavior can be well characterized by the correlation matrix of ensemble spiking. Conceptually, the choice of TEMPLATE and MATCH is arbitrary and this analysis can be applied to both directions (WAKE➔ASLEEP or SLEEP➔WAKE). However, the limitation of linear subspace methods, including both PCA and independent component analysis (ICA) [59,53], is that they assume a stationary correlation statistic during the complete TEMPLATE or MATCH period, which is untrue in the presence of distinct or complex behaviors that drive the state-dependent neuronal responses. Furthermore, the derived reactivation strength from these methods does not identify the “meaning” of memory; instead it is positively correlated with the quadratic power of temporal firing rate in the neuronal ensemble.

Box 2 FIGURE I. Unbiased Assessment of Sleep-Associated Neuronal Population Codes.

(a) Principal component analysis (PCA) for computing the similarity of two templates of correlation matrices from population spike counts (WAKE and SLEEP) and assessing the reactivation strength during sleep (reproduced with permission, [43]). In WAKE, {λ,₁,p1} are associated with the dominant principal component (PC) extracted from PCA. In SLEEP, time-varying reactivation strength is computed. (b) Unsupervised population decoding using a finite-state hidden Markov model (HMM). Specifically, the spatial environment is represented by a finite discrete state space. Trajectories across spatial locations (“states”) are associated with consistent hippocampal ensemble spike patterns, which are characterized by a state transition matrix. From the state transition matrix, a topology graph that defines the connectivity in the state space is inferred [69]. In these two methods, no assumption is made about neuronal RF, and the bin size in Post-SLEEP is independent on the bin size used in WAKE. Since the order of WAKE and SLEEP can be switched, and one can apply these methods to SLEEP data first and then examine their meanings in the WAKE behavior; therefore they both fall into the new paradigm (“memory first, meaning later”).

The second approach is a population-decoding method. Unlike the traditional supervised or RF-based decoding methods [64,65], an unsupervised population-decoding method [66–69] has been developed for recovering hippocampal spatial memory with the assumption of place RFs (Box 2, Figure Ib). This is achieved by associating spatiotemporal spiking patterns with unique latent states without defining meanings of those states a priori. Such an approach is conceptually appealing since it requires no assumption of explicit behavioral measures. In the case of rodent navigation example, the latent states may represent animal’s spatial locations. Statistically, the latent states are assumed to follow a Markovian or semi-Markovian transition dynamics. Trajectories across spatial locations (“states”) are associated with consistent hippocampal ensemble spike patterns. In other non-spatial tasks, the latent states may also accommodate non-spatial features of experiences or distinct behavioral patterns that cannot be measured directly. The connection between latent states and spatiotemporal spiking patterns can be established from statistical inference, hypothesis testing, and Monte Carlo shuffled statistics [66–68]. Furthermore, additional features (such as spiking synchrony or LFP features in terms of power or instantaneous phase) can be incorporated into the statistical model for further disassociating distinct latent states. Since this model-based approach is built upon a generative model, model fitting is therefore strongly dependent on the probability distributions that describe the data generation process. If there is a model mismatch, this approach may yield poor performance.

The standard paradigm for memory is to first figure out how the brain encodes information during WAKE, and then determine if those coded patterns appear later, during either SLEEP or subsequent behavioral memory testing — thereby “meaning first, memory later”. In contrast, the new framework allows us to shift the paradigm and look at memory first (by decoding intrinsic structure in neural codes), and then determine the meaning later (i.e., how that structure might correlate with subsequent behavior), thereby “memory first, meaning later” [69]. The main differences between these two paradigms are their assumptions and analysis order (independent of the chronological order). The unsupervised approach is unbiased in that it avoids predefining neural activity patterns in WAKE associated with a specific task or behavior, and also enables us to seek structures that are either not explicitly defined or simply indefinable. Therefore, this unbiased approach may potentially provide us opportunities to discover hidden structures in brain activity, which may represent well-defined WAKE experiences or may reflect some undefined processes (e.g., creative thoughts and imagination). More importantly, this approach may suggest outstanding research questions for experimental investigations. For instance, how can we distinguish the memory in sleep related to previous navigating experiences in two or more distinct spatial environments? How can we decipher non-spatial hippocampal episodic memory [23,71–74] in sleep?

From a data analysis perspective, several technical challenges are worth consideration. First, the sleep episodes have short epochs, sparse and sporadic firing (reduced firing rate compared to wake), and compressed timescale. Dealing with these issues often involves unsubstantiated assumptions (e.g., temporal independence, homogeneity) in data analysis. Second, our empirical studies using synthetic sleep spike data [69] have demonstrated that the number of active hippocampal pyramidal cells is critical for reliable representation of the space as well as detection of spatiotemporal reactivated patterns in SWS. Since only a small fraction (~10–15%) of hippocampal neurons that are active during WAKE is reactivated at any given time during SWS, a reliable investigation of sleep-associated population codes would require simultaneous recording of hundreds of neurons in WAKE. Third, there is large diversity among hippocampal pyramidal neurons for their contributions to the sequence replay [75]. Furthermore, a small percentage of hippocampal pyramidal neurons have no significant spatial tuning but may still fire during sleep. It is unclear whether their firing activities represent other non-spatial episodic memory components in the memory space, and how we can identify their statistical significance. Similar challenges would also apply to the neocortex [76,77].

Future Directions

Neural population recording

Recent advances in neural recordings have greatly expanded our capability to investigate neuronal population codes [78–80]. According to the newest technology in multi-electrode recording (personal communication, Professor M. Roukes at Caltech), it is predicted that by year 2020 neuroscientists would be able to simultaneously record 10,000–100,000 hippocampal neurons from rats (based on new development of stacked nanoprobes [81]). As a result, the statistical power of SANC analysis would increase significantly by ~100 fold. In addition, calcium imaging is another emerging technique for measuring large-scale activity of neuronal populations, which has been successfully used for chronic recordings from the rodent hippocampus [82–85] and cortex [86]. Since calcium signals are merely indirect measurements of neuronal spiking, the precise relationship between calcium signals and spiking is not fully identifiable and is also susceptible to biophysical variations. Therefore, improving the temporal resolution (>500 Hz) and light sensitivity for fluorescence images would potentially enable us to examine large-scale population codes at faster timescales. Combining electrophysiology and cell-type-specific imaging techniques would be an important future direction due to their complementary strengths. In human/non-human primate studies, a new tool that integrates electrophysiological and fMRI (known as neural-event-triggered fMRI) recordings [87] has proven valuable for examining the spatial mapping of a priori defined local brain patterns. Developing wireless multi-electrode recording techniques [88] is also crucial for chronic neural recording from non-human primates in a naturalistic sleep environment.

REM sleep

While NREM sleep has been strongly implicated in the reactivation and consolidation of memory traces, the exact function of REM sleep remains elusive [89,90]. Unlike NREM sleep, in REM sleep there is no UP state or population synchrony associated with hippocampal SWRs, resulting in a decrease in neuronal firing and an increase in synchrony, both of which are correlated with the power of theta oscillations [91]. This implies that the ensemble spike activity is even more sparse and unstructured. Moreover, there is some experimental evidence that in REM sleep rat hippocampal neurons exhibit gradual phase shift from the novel (theta peak) to the familiar (theta trough) firing-phase pattern [92]. Such experience-dependent phase reversal suggests that hippocampal circuits may be selectively restructured during REM sleep by selectively strengthening recently acquired memories and weakening remote ones—an idea consistent with the original Crick-Mitchison’s hypothesis of “reversal learning” in REM sleep [93]. Experimentally, the total REM sleep duration is much shorter than the NREM sleep duration for rodents and human adults. Most animal experiments have primarily targeted on waking behaviors, thereby limiting the recording period of REM sleep. To increase the length of REM sleep or the probability of transition into REM from NREM sleep, optogenetic manipulations of specific neural circuits have been considered in rodents [94–96]. Alternatively, one can investigate rodent infants or other specifies that have longer REM sleep episodes. Recent single-unit recordings in human MTL suggested that eye movements during REM sleep might reflect a change of the visual imagery in dreams [57]. With ever-accumulating “BIG neural data”, an ultimate goal is to decipher the animal’s dreams during REM sleep in reference to WAKE experiences— a demanding task still requiring extensive experimental and computational investigations.

Contextual memory

All memories are context-specific, whether being spatial, temporal or emotional, leading to the concept of sequence coding or trajectory coding. As the hippocampal network is connected with the amygdala— a specific brain area responsible for emotions and memory modulation, episodic memories are often associated with emotions, such as happiness, fear, and anxiety. This may occur in memory recall and dream experiences. Notably, sleep consolidates or reshapes emotional memories [97]. One hypothesis is that emotional or contextual memory can be strengthened or weakened in the hippocampus during REM sleep theta activity [98,99]. Recent causal evidence showed that temporally precise attenuation of the theta rhythm impaired fear-conditioned contextual memory [99]. However, how to read out contextual episodic memories embedded with distinct emotions is still a big puzzle. Development of new computational approaches for deciphering hippocampal-amygdala population codes will be an extended research direction.

Creativity and insight

Creativity involves the forming of associative elements into novel associations that are useful for future task behaviors (e.g., planning, problem solving). Such new association patterns might not occur frequently, and shall not be confused with the “preplay” events [100]. Insight is defined as a neural restructuring process that leads to a sudden gain of explicit knowledge leading to qualitatively changed behavior [101]. Human sleep studies suggested that REM sleep promotes creativity and insight because of the changes in cholinergic and noradrenergic neuromodulation [102], which allow neocortical structures to reorganize associative hierarchies and reinterpret the hippocampal information. Computationally, how to detect such new associations of spatiotemporal patterns across a large hippocampal-neocortical network remains unknown. Future simultaneous recordings from multiple targeted brain areas would enable us to examine high-dimensional spatiotemporal spike patterns and evaluate their probabilities of coincident reactivations at different brain states.

Manipulation of memory

To date, neuroscientists have relied on many powerful engineering or genetic tools, such as the virtual environment [103,104] and optogenetics [53,105–108], to manipulate hippocampal memory during wakeful experiences. In virtual environments, rodent hippocampal neurons exhibited different spike firing patterns from real environments. However, it remains unclear how such firing patterns would be affected in sleep. False memories play a significant role in human mental health and legal practice [109]. In a series of groundbreaking experiments [105,106], researchers stimulated or suppressed memories with optogenetics to manipulate engram-bearing neurons in the mouse hippocampus. Their findings suggested that optogenetic reactivation of memory engram-bearing cells was not only sufficient for the behavioral recall of that memory, but also served as a conditioned stimulus for the formation of an associative memory. Techniques of selective enhancement of desired memories and indirect suppression of unwanted memories might find potential translational applications in treating traumatic memories in post-traumatic stress disorder (PTSD) patients. Similarly, it remains unknown how these manipulations affect memory during sleep. Among all experimental manipulations, one key research goal is to study their sleep-associated memory contents and use them to further predict future behavior.

Closed-loop neural interface

Brain-machine interfaces provide not only potential therapies for animals and humans, but also new tools for studying memory processing during sleep [44,53,110,111]. Combining various invasive (e.g., electrical) or non-invasive (e.g., optical, acoustic) closed-loop stimulation techniques [39,112–115], we can test the causal functions of neural circuits or sleep for memory processing in a real-time manner. For instance, coupling spontaneous reactivation of a place cell during sleep to a reinforcing stimulation of the medial forebrain bundle (MFB) induced a place preference during subsequent wake, providing another evidence that place cells encode the same spatial information during sleep and wakefulness [116].

Concluding Remarks

In summary, accumulative experimental evidence has pinpointed the critical role of sleep in consolidating hippocampal-neocortical memories. With advances in large-scale neural population recordings and imaging techniques, it is imperative to develop computationally relevant methods to provide unbiased assessment of memory-related SANC. Despite rapid progress in the last two decades, many outstanding questions still remain. Furthermore, contributions of many other subcortical circuits to various sleep-associated memories remain to be investigated, such as the ventral striatum [117,118] and the anterior thalamus [119,120]. Combinations of experimental and computational investigations will be a crucial step forward for improving our understanding of this exciting and important research field. Future dissection of memory during sleep will shed light on neural mechanisms of dreaming, creativity, contextual or emotional memories, and will provide further insights into memory-related neurological and psychiatric disorders.

Trends Box.

• The thalamus (a subcortical structure) plays an important role in sensory gating, arousal regulation, and generating thalamo-cortical sleep spindles. To fully dissect sleep-associated memory, it is critical to understand three-way communications among hippocampal-neocortical, thalamo-cortical, cortico-thalamic circuits in sleep.

• Combining electrophysiology, imaging, virtual reality, and optogenetics in experimental investigations can significantly expand our understanding of neural codes underlying memory and sleep.

• Optogenetics has proven powerful to test the causal role of neural circuits in memory consolidation and valuable to create false memories. Finding effective means for consolidating false memories may have a significant impact on future behavior.

• Bridging the research gaps between rodents and non-human/human primates in sleep studies is the key to dissect circuit mechanisms in consolidating various forms of memories, and to provide further insights into treatment of neurological and psychiatric diseases.

Outstanding Questions.

• WHAT: representation—the content of sleep-associated memory in hippocampal-neocortical network. Does sleep-associated spike activity have any significant representation and how to assess their significance? Does the content of sleep-associated memory in one brain region help decipher the content of sleep-associated memory in another region?

• WHEN: temporal coordination—the timing of memory reactivation (e.g., coincident or non-coincident ripple and spindle events) and their distinct functional roles. How does the hippocampal-neocortical coordination evolve in different sleep stages?

• WHERE: Episodic memories consist of spatiotemporal sequences in behavioral experiences, including spatial trajectory coding and non-spatial sequence coding. How can we distinguish the content of spatial vs. non-spatial memories in sleep? Can we read out contextual or emotional memories in sleep?

• To what extent can we identify the content of hippocampal-neocortical population codes during REM sleep?

• What’s the principled way to systematically investigate creativity and insights in sleep?

• Do the NREM and REM sleep play different roles in consolidating declarative memory versus procedure memory?

• What are the circuit mechanisms that allow external factors (e.g., reward, sensory cue) to bias the content of sleep-associated memory? Are they top-down or bottom-up?

• How can we effectively manipulate sleep-associated memory to improve the performance of post-sleep cognitive functions?

• Are false memories consolidated in the same way as true memories during sleep? What are the effective ways to enhance or suppress them?

• Can investigations of sleep-associated memory reveal new discovery between normal and aging/diseased brains, or even between ordinary and genius brains?

Glossary

Episodic Memory

is made of associations of several elements, such as objects, space and times. The associations are encoded by chemical and physical changes in neurons, as well as by modifications to synapses between neurons

Hippocampus

a brain structure within the medial temporal lobe (MTL) that is important for episodic memory, spatial learning and associative recollection. It consists of CA1, CA2, CA3 and dental gyrus, and is connected to various brain structures, including the prefrontal cortex (PFC), entorhinal cortex and amygdala

Memory Consolidation

a process that converts and stabilizes information from short-term memory into long-term storage. The hippocampal-neocortical memory consolidation involves transferring hippocampal episodic memory into the neocortex during the off-line (such as sleep) process after waking experiences in memory acquisition

Population Codes

are referred to neuronal ensemble spike activity that represents and transmits information. Spikes are the basic neuronal language for information and communication. Depending on specific neural circuits, different statistical assumptions are made about the computational principle or information carrier, such as spike count, spike timing, independent or correlation codes

UP and DOWN States

are defined as periods (~a few hundred milliseconds) of synchronized population firing and widespread depolarization, and periods of relative silence and hyperpolarization, respectively. The DOWN states alternate between the UP states during slow wave sleep

Local Field Potential (LFP)

is considered to represent the aggregate subthreshold activity of a local population of neurons in a spatially localized area near the recording electrode and can be viewed as the input information in that area. Spectral analysis of the broadband LFP signal can reveal significant oscillatory activity at specific frequency bands

Rapid Eye Movement (REM) Sleep

a sleep stage characterized by quick, random movements of the eyes and low muscle tone. REM sleep occurs in cycles of about 90–120 minutes in night and accounts for 20–30% sleep time in adult humans. Most human dream activity occurs in REM sleep. In rodents, REM sleep is accompanied by theta oscillations

Slow Wave Sleep (SWS)

a sleep stage also known as non-REM (NREM) sleep or deep sleep, accounting for ~75% of total sleep time, is characterized by synchronized EEG activity of slow waves with frequency below 1 Hz and relatively high amplitude. Sleep spindles (9–15 Hz) occur during SWS

Place Receptive Field (RF)

a property of localized spatial tuning exhibited prominently in hippocampal pyramidal neurons of rodents and bats. The RF defines the firing property of hippocampal place cells with respect to specific spatial location. On a linear track, the rodent hippocampal place RF is often directionally dependent

False Memory

refers to recall of an event or observation that did not actually occur. Internally generated stimuli can get associated with concurrent external stimuli, which can lead to the formation of false memories