Selection of experience for memory by hippocampal sharp wave ripples

Abstract

Experiences need to be tagged during learning for further consolidation. However, neurophysiological mechanisms that select experiences for lasting memory are not known. By combining large-scale neural recordings in mice with dimensionality reduction techniques, we observed that successive maze traversals were tracked by continuously drifting populations of neurons, providing neuronal signatures of both places visited and events encountered. When the brain state changed during reward consumption, sharp wave ripples (SPW-Rs) occurred on some trials, and their specific spike content decoded the trial blocks that surrounded them. During postexperience sleep, SPW-Rs continued to replay those trial blocks that were reactivated most frequently during waking SPW-Rs. Replay content of awake SPW-Rs may thus provide a neurophysiological tagging mechanism to select aspects of experience that are preserved and consolidated for future use.

Some episodic events experienced during the day are further consolidated during sleep, whereas others are discarded (1). Remembering events depends on processes that occur both immediately after encoding and thereafter during sleep (2–4). However, no unified solutions have yet emerged for brain mechanisms that assign which experiences are selected for storage and which are eliminated. A molecular candidate for synaptic tagging has been proposed (5) as an eligibility trace (6), but no neurophysiological mechanism has been identified for online selection of experience. Awake sharp wave ripples (SPW-Rs) in the hippocampal system (3, 7–8) are a potential candidate for selecting particular aspects of experience for future use (9). In support of this hypothesis, salient features of the experience, such as novelty and reward magnitude, facilitate replay of multiple aspects of waking experience and enhance memory (2, 9, 10). To hold confounding external biasing factors (such as reward or novelty) constant and identify whether SPW-Rs assign credit to particular events, we examined which events within a session were selected by waking SPW-Rs and continued to be replayed repeatedly during sleep SPW-Rs. Recordings taken from many hundreds of neurons simultaneously in the dorsal CA1 region of the hippocampus (n = 4469 cells from six mice) with dual-side silicon probes (Fig. 1A) (11) and the application of advanced analysis tools allowed us to visualize and decode perpetually changing spike contents of successive trials during experience and relate them to spike sequences during both awake and sleep SPW-Rs.

Fig. 1. Trial block identity can be decoded from the distinct temporal evolution of neuronal population activity.

(A) Illustration of two (out of four) shanks of the dual-sided probe. (B) The figure-eight maze task, where mice alternated between left and right arms for water reward (blue droplets). The mouse’s trajectory along maze corridors is color coded by its linearized position. (C) Trials 1 (right arm traversal) and 2 (left arm traversal) during the figure-eight maze task. (Top) The linearized position of the mouse. Red, right traversal; green, left traversal. (Bottom) A raster plot of 422 pyramidal cells that were simultaneously recorded from the right dorsal CA1 region of a mouse’s hippocampus, sorted by seqNMF, and color coded according to the linearized position. This follows the same color scheme as in (B). (D) Trials 3 to 31 [of 70 trials in total (fig. S1)] of an example session. E, error trials. (E) (Left) Running trajectory of a mouse in the figure-eight maze. (Right) UMAP embedding of population activity. Each point corresponds to the low-dimensional representation of one-binned spiking data. Both are colored according to the mouse’s position, as in (B). (F) The same session as (E), but the running trajectory and neural manifold are colored by trial block number [see color key in (H)]. (G) Neural manifold of the same session as in (E) and (F) made by using semisupervised UMAP trained on blocks of five trials. The manifold is colored by linearized position. (H) Same as (G), but the neural manifold is colored by trial block number. (I) Confusion matrix of trial block decoding from the original high-dimensional space [same session as (A) to (H)]. (J) Trial block identity decoding errors from across all sessions using UMAP, PCA, Bayesian decoding, decoding from original high-dimensional space with Euclidean distance (EUD), and cosine similarity (COS) as distance metrics. Each dot in the violin plot indicates one session, pooled across n = 26 sessions from six animals. Decoding error was measured in units of trial block in which five trials were binned to one trial block.

To extract the sequential structure embedded in the spike data, we used sequence non-negative matrix factorization (seqNMF) (12). seqNMF identified robust patterns that matched the behavioral events of mice in the figure-eight maze (Fig. 1, B to D, and fig. S1). To further examine the differences in the sequences between different events, we used uniform manifold approximation and projection (UMAP) to embed the same high-dimensional data in a low-dimensional space (13). As expected, qualitative UMAP visualization (Fig. 1E) revealed that population activity of the hippocampus corresponded to a latent space that topologically resembled the physical environment (14, 15). Notably, a prominent progression of states that corresponded to trial sequences was observed after color-labeling the manifold with trial block numbers (Fig. 1F) (16). This observation was consistent across different maze types and rodent species (fig. S2) (17).

Within-session temporal evolution of neuronal population activity

To quantify the trial sequence information (18) present in the state space, we tested if trial block membership can be accurately decoded from the population activity (13, 19). Successive five trials composed a trial block, and a specific label was assigned to each trial block (Fig. 1, G and H). Decoding was performed using a k-nearest neighbor (kNN) decoder, followed by 10-fold cross-validation in the original high-dimensional space (fig. S7B) (materials and methods). Trial block membership could be accurately decoded from the original high-dimensional space (Fig. 1I) as well as from the low-dimensional UMAP embedding (fig. S3, C and E). These results were further confirmed by using principal component analysis (PCA) for dimensionality reduction or by using Bayesian decoding (Fig. 1J and fig. S3), although these classical methods did not generate intuitive visualization for the initial hypothesis generation process. Notably, we confirmed that UMAP embedding in the low-dimensional space preserved the trial block membership of the data compared with that in the original high-dimensional space, yielding consistent trial-decoding results between the low- and high-dimensional spaces (fig. S4). Nevertheless, the main summary statistics (across the entire dataset) through-out the paper were performed in the original high-dimensional space.

Intertrial variation of population activity is not random

To probe the contribution of the individual neurons to the aforementioned neuronal population features, we first plotted the tuning curves of individual neurons across trials. We found diverse patterns of intertrial variability: within-place field rate remapping (fig. S6, A and B) (20), place fields emerging on later trials (bottom, Fig. 2A) (21), and shifting place fields across trials (top, Fig. 2A, and fig. S6C) (22–24). In principle, decodability of trial block membership could be explained by either stochastic or structured variation across trials. To identify the source of variability, we built a model that generated stochastic fluctuation in firing rate across trials. Each simulated cell’s spiking activity was generated through a Poisson process based on the tuning curve of real neurons (Fig. 2B) (materials and methods). We passed the simulated spike trains through the same dimensionality-reduction pipeline as we did for the real data. The manifold of the simulated data also reflected the topology of the maze (Fig. 2C). By contrast, trial block identity information across trials vanished (Fig. 2, D and E). As an alternative to the standard 10-fold cross-validation method (Fig. 1I and fig. S7, B and D), we implemented a targeted validation method (a “leave one trial out” procedure) that was expected to yield high decoding accuracy only if the state space of the data changed in a structured way, evolving along one axis according to the sequence of trial events. Because the training set did not contain any data that shared the trial block membership with the test set, the test data could decode only to the next closest trial block (not the same trial block) in the state space (fig. S7, A and C). Indeed, we observed that only the real data were decoded correctly to the preceding or succeeding trials, occupying the space immediately next to the diagonal of the confusion matrix (1.20 and 7.84 mean error for real and simulated data for one example session) (Fig. 2F and fig. S7, C and E to G). The trial decoding accuracy of the real data was significantly higher than that of the simulated or trial-shuffled data (P < 10⁻¹⁰; 1.51, 4.07, and 5.34 mean error for real, simulated, and shuffled data, respectively) (Fig. 2G and fig. S8). These findings indicate that intertrial variation of population activity (drift) cannot be captured by random fluctuations at the single neuron level. By using four different decoding methods, we confirmed that the systematic trial-dependent variation of the neuronal population activity could not be explained by electrode drift (figs. S5, S6, and S9) (25).

Fig. 2. Contribution of single cell and subpopulation of cells to the trial block identity coding.

(A) Trial-by-trial firing rate of an example neuron during left-arm (top) and right-arm trials (bottom) in the figure-eight maze. Color indicates the trial block number. (B) Raster plot of an example neuron (top) and a simulated neuron (bottom), generated from a Poisson process model based on the across-trial mean firing rate of the real neuron (materials and methods). (C) (Top) UMAP manifold generated from the population activity of all the neurons in one example session. (Bottom) UMAP embedding from the simulated population activity. Both are colored by linearized position. (D) Same as (C) but colored by the trial block number [see color key in (E)]. (E) Neural manifold of the same session as (C) and (D) made by using the semisupervised UMAP trained on blocks of five trials. (F) Confusion matrix of trial block decoding results for the real (top) and simulated population (bottom). For decoding results from other decoding methods, see fig. S7, E to G. (G) Trial block decoding error of real (purple) and simulated (green) data across all sessions (decoded from the original high-dimensional space) (***P < 10⁻¹⁰, unpaired t test; n = 26 sessions from six animals). The dashed line indicates the chance level from trial-shuffled data (fig. S8). (H) Neural manifold generated from place cells (left) and nonplace cells (right) from the same session as (A) to (F) by using semisupervised UMAP trained on blocks of five trials. (I) Trial block decoding error of all cells (red), place cells (PC), and its size-matched control from all cells (SM-PC, purple), as well as nonplace cells (NPC) and their size-matched control (SM_NPC, blue). Decoding error was measured in units of trial block in which successive five trials were binned to one trial block. (N.S., not significant; unpaired t test; n = 26 sessions from six animals.) (J) Decoding error when downsampling the cells in the example session to subsamples of varying cell numbers (from 50 to 450 cells). Error bar indicates the standard error of the mean (SE) across 1000 subsamples. Decoding error was measured in units of trial block in which successive five trials were binned to one trial block). The red line shows the fitted exponential curve.

Both place cells (PC) and nonplace cells (NPC) contributed to trial block membership decoding (Fig. 2H) [mean decoding errors were 0.1, 0.42, 0.49, 0.53, and 0.59 for all cells, PCs, NPCs, and the size-matched controls of PCs and NPCs, respectively (Fig. 2I)]. The neural manifolds of different trials were aligned better with place cells than with nonplace cells. Decoding accuracy deteriorated rapidly when downsampled to <100 neurons in a session, indicating that trial block identity is encoded in the population (Fig. 2J and figs. S10 and S11).

Waking SPW-Rs replay current experience

Place and trial sequence encoding is associated with theta frequency (6 to 9 Hz) oscillations (“theta state”) (26) as animals actively navigate in an environment. When animals stop consuming the water reward, theta activity gives way to synchronous population events, SPW-Rs (Fig. 3, A and B). Because spike sequences within SPW-Rs are known to replay place field sequences (27, 28), we asked whether trial block identity could also be decoded in replay events. A candidate SPW-R event was classified as a significant replay if both its distance to manifold and trajectory length were significantly shorter than those of shuffled distributions (Fig. 3, C and D, and figs. S12 to S14) (materials and methods). The majority of SPW-Rs occurred in the reward area (fig. S15A), and about 33% of the awake SPW-Rs were significant replays (Fig. 3E). From the original high-dimensional space, we decoded not only the spatial trajectory but also the trial block identity of the significant replay events. We validated the decoding results using four different methods, including decoding from the original high-dimensional space with different distance metrics and decoding from the low-dimensional space with PCA and UMAP. Different methods yielded consistent decoding results (Fig. 3, F to G, and figs. S16 and S18, A to D). The within-event coherence of trial replay was high (different time bins within the same events coherently decoded to the same trial block) but decreased rapidly as we downsampled the number of cells included for decoding (fig. S18, E to G). Next, we examined whether the SPW-Rs replayed the past, future, or current trial block. The spike content of SPW-Rs decoded most reliably to the present trial block (Fig. 3H) (4, 28–30). The strongest predictor of the trial block distribution pattern for maze replay was the immobility time (fig. S19B).

Fig. 3. Maze SPW-Rs replay of trial block identity.

(A) Spiking activity of an example trial in the figure-eight maze. (Top) Linearized position of the mouse. (Bottom) Raster plot of neuronal spiking activity sorted by seqNMF. Purple stars, SPW-Rs. (B) Zoomed-in display of a replay event. (Top) Local field potential (LFP) filtered in the ripple band. (Bottom) Raster plot of neurons belonging to a sequence factor with significant reactivation strength (materials and methods). Neurons were sorted in the same order as in (A). (C) All waking SPW-R events (red) in this example session were embedded with the neural manifold data during navigation (light gray), and the noise cloud consisted of negative samples (dark gray). Two clusters of SPW-R replay events were distinguished: off- and on-manifold events (materials and methods) (figs. S12 and S13). (D) SPW-R replays were classified as significant if they were (1) close to the maze manifold and (2) their trajectory length along the manifold was short in comparison to shuffled data. (E) Percentage of significant replays in the maze (number of significant maze replay events/total number of maze SPW-R events) compared with shuffled data. (***P < 10⁻⁴, unpaired t test; n = 26 sessions from six animals). (F) UMAP embedding and the decoded position for the same event as in (B). (Left) The neural manifold during maze running (“position manifold,” colored by position). (Right) The position content of each replay time bin was decoded to a position bin along the maze trajectory according to the position label of its nearest neighbor on the manifold. The black triangle represents the physical location of the mouse when the replay event took place, and the pink-purple dots represent the neural embedding of seven successive time bins of a SPW-R replay event (each time bin was 20 ms). (G) (Left) The same event was embedded with the “trial manifold” and was colored by trial block number. (Right) Trial content of each replay time bin was decoded as the trial block label of its nearest neighbor on the manifold (each trial block corresponds to five trials). (H) Distribution of differences between the actual trial block of SPW-Rs events and their decoded trial block identity across all sessions.

Postexperience sleep SPW-Rs replay events selected by waking SPW-Rs

To examine the relationship between awake SPW-Rs and those during postexperience sleep, we compared the population activity of awake SPW-Rs in the maze with that during postexperience sleep in the home cage (Fig. 4, A to E). The distribution patterns of trial block identity during postexperience sleep were highly correlated with that during maze replay, but not with preexperience sleep replay (R = 0.86, P < 10⁻³⁶ for correlation between postexperience sleep replay and maze replay) (Fig. 4, F to H, and figs. S17 and S19A). We compared which factor best explained the trial block distribution pattern during postexperience sleep by using a mixed-effect linear regression analysis. All decoding methods consistently showed that the strongest predictor for the trial block distribution pattern during postexperience sleep was that of the SPW-R replay in the maze (P < 10⁻²⁶, P = 0.06, P < 10⁻⁴, P = 0.89, and P = 0.87 for in-maze replay, theta power, theta cycle count, presleep replay, and trial-shuffled data, respectively) (Fig. 4I and fig. S19C) (32).

Fig. 4. Replay of trial block identity and maze segments during sleep can be predicted from waking SPW-Rs in the maze.

(A) Spiking activity in the figure-eight maze. (Top) Linearized position of the mouse. (Bottom) Raster plot of neuronal spikes, sorted by seqNMF. Purple stars represent SPW-Rs. (B) Zoomed-in display of an awake replay event in the maze. The raster plot contains neurons belonging to the sequence factor with significant reactivation strength (materials and methods). Neurons were sorted in the same order as in (A). (C) Decoded position and trial block identity of successive 20-ms bins (one to six bins) of the same SPW-R replay event in (B). The black triangle represents the physical location of the mouse when the replay event took place. (D) Raster plot of a SPW-R replay event during sleep in the home cage. (E) Decoded position and trial block identity of successive 20-ms bins of the same SPW-R replay event in panel (D). (A), (B), and (D) were colored according to the linearized position of the mouse; (C) and (E) were colored according to trial block number. (F) Percentage of significant replays during pre- and postexperience sleep (presleep and postsleep, respectively) in the home cage compared with shuffled data. (Presleep versus postsleep, ***P < 10⁻⁵; postsleep versus shuffled data, ***P < 10⁻⁸; presleep versus shuffled data, ***P < 10⁻⁸; unpaired t test; n = 26 sessions from six animals). (G) Distribution of the trial blocks decoded from the population spike content of SPW-Rs in an example session. (Top) Maze and postsleep replay trial block distribution pattern. (Bottom) Maze and presleep replay trial block distribution pattern. (H) Correlation between trial block distributions across trial blocks during maze and postsleep SPW-Rs replay (decoded from the original high-dimensional space; Pearson correlation coefficient, R = 0.86; P < 3.1 × 10⁻³⁴; n = 16 sessions from five animals) (results from all four different decoding methods, see fig. S18). (I) The predictive relationship between trial block distribution patterns of postsleep SPW-Rs and other candidate variables, including the trial block distribution patterns of theta cycle, theta power, presleep, and trial-shuffled data (***P < 10⁻²³ for awake replay; ***P < 10⁻³ for theta cycle number; the relative predictive power of a given metric was considered nonsignificant when it overlapped with zero; n = 16 sessions from five animals) (for results from all four different decoding methods, see fig. S18). (J) Proportion of maze segment replays (left arm and right arm) during awake and postexperience sleep SPW-Rs in an example session. (K) Proportion of sessions with same rank order between maze segments during awake and post-maze sleep SPW-R replays, compared with shuffle data. (*P < 0.05, chi-square test; n = 13 sessions from five animals).

We examined whether replays of the left versus right arms during waking and postsleep SPW-Rs were correlated, exploiting the natural variability in the replay distribution patterns across different arms in different sessions (Fig. 4J and fig. S15B). The correlation of maze-arm replays between wake and sleep SPW-Rs was significantly higher than in the shuffled data (Fig. 4K). The difference in replay proportion across left and right arms could not be explained by the difference in the decodability (measured by spatial information) or the difference in coverage (measured by number of visits) between the two arms (fig. S20).

Waking SPW-Rs weigh and select the experience, and sleep SPW-Rs consolidate it

Exploration of the environment is a regular alternation between ambulation and rest phases, enabling brain state changes (33–35), and replaying aspects of experience during SPW-Rs (27, 28, 30). We hypothesize that waking SPW-Rs represent a natural credit assignment (tagging) mechanism of experiences (3, 7–9). They tag selected neuronal patterns, possibly by comparing them to previous experience and relevance to the animal, and the tagged patterns are reactivated numerous times during SPW-Rs of postexperience sleep to consolidate the selected experience and combine it with the existing knowledge base of the brain (4, 29, 36, 37). Waking SPW-Rs may trigger molecular mechanisms to induce long-term synaptic changes (5, 38). At the functional level, they may create a brain state–dependent attractor. Consequently, when the hippocampal network returns to a similar brain state, such as non–rapid eye movement (NREM) sleep, it continues generating patterns set forth by the waking SPW-R attractor.

Previous observations support this scenario. Cues relevant to future behavioral success can bias the neuronal trajectories of SPW-Rs (39–41). Perturbation of SPW-Rs prevents place field and experience stabilization (7, 8, 42–48). In humans, items for which electrophysiological brain patterns are not reactivated during non-REM sleep are forgotten. Replay can occur after a single experience and the number of SPW-R after learning predicts subsequent memory performance (3, 31, 46, 49–51).

Salient features of the experience, such as novelty (9, 52, 53), repetition (9), and reward (10), increase the incidence of SPW-Rs and replays (54, 55). In our experiments, we could exclude differences in external factors that could shape sleep replay patterns by demonstrating distinctly changing neuronal population activity patterns within the same maze and session. Selective tagging of trial events by waking SPW-Rs could occur because the perpetually evolving population activity patterns during the theta state serve as a temporal scaffold to organize experiences (56) and differentiate successive events. The sequence of theta state–dependent turnover of neuronal assemblies, their selection by waking SPW-Rs, and the numerous repetitions during sleep SPW-Rs may represent a hippocampal network mechanism of selective episodic memory consolidation.

Our observations suggest that rewards provide an affordance for shifting brain states and that the waking SPW-R serves as a neurophysiological mechanism for memory selection (49, 56). These findings establish a neurophysiological framework for multiple domains of systems neuroscience, including credit assignment (28, 37), representational drift (57), “event remapping” (18), time coding (16, 58–60), and memory editing (61). Our work also links a shared principle of memory processing important for both biological and artificial learning. In particular, it relates to importance sampling (62, 63) in machine learning, which enables faster acquisition (64) and more robust generalization (65).

Data and materials availability:

All data are available in the main text and supplementary materials or are deposited at Zenodo (66, 67). Notated neurophysiological data are available at https://dandiarchive.org/dandiset/000552/0.230630.2304, https://buzsakilab.nyumc.org/datasets/GrosmarkAD/Achilles/Achilles_10252013/, and https://buzsakilab.nyumc.org/datasets/HainmuellerT/TH11_210605/.