Consolidation of human skill linked to waking hippocampo-neocortical replay

SUMMARY

The introduction of rest intervals interspersed with practice strengthens wakeful consolidation of skill. The mechanisms by which the brain binds discrete action representations into consolidated, highly temporally resolved skill sequences during waking rest are not known. To address this question, we recorded magnetoencephalography (MEG) during acquisition and rapid consolidation of a sequential motor skill. We report the presence of prominent, fast waking neural replay during the same rest periods in which rapid consolidation occurs. The observed replay is temporally compressed by approximately 20-fold relative to the acquired skill, is selective for the trained sequence, and predicts the magnitude of skill consolidation. Replay representations extend beyond the hippocampus and entorhinal cortex to the contralateral sensorimotor cortex. These results document the presence of robust hippocampo-neocortical replay supporting rapid wakeful consolidation of skill.

Graphical Abstract

In brief

Substantial jumps in performance immediately following short rest periods account for early motor skill learning. In this study, Buch et al. report waking hippocampo-neocortical replay during these rest periods that is temporally compressed by 20-fold relative to behavior, selective for the trained skill, and predicts the magnitude of rapid consolidation of skill.

INTRODUCTION

The popular idiom “practice makes perfect” emphasizes the importance of intense practice in acquiring and perfecting skill (Dayan and Cohen, 2011). However, the same amount of practice results in significantly different skill depending on the presence or absence of waking rest within the training schedule, a phenomenon termed the spacing effect (Kornmeier and Sosic-Vasic, 2012). Consolidation of skill is superior when frequent rest periods are interspersed with practice blocks (distributed practice) than when the same total amount of practice is performed over longer continuous blocks (massed practice) (Cepeda et al., 2006; Gerbier et al., 2015; Kornmeier and Sosic-Vasic, 2012; Song et al., 2012; Toppino et al., 2009). Thus, the presence of waking rest interleaved with practice is a crucial determinant of skill memory consolidation.

Consistent with this notion, it has been proposed that “much, if not all” skill learning occurs offline during rest rather than during actual practice (Robertson, 2019). For example, performance improvements while acquiring a new skill accumulate almost exclusively during waking rest periods interleaved with practice (Bönstrup et al., 2019). These micro-offline gains indicate a rapid form of skill memory consolidation that develops over a much shorter timescale than previously thought (Bönstrup et al., 2019, 2020; Jacobacci et al., 2020), and that is approximately 4-fold greater in magnitude than classically studied overnight consolidation requiring sleep (Bönstrup et al., 2019).

Skill learning involves forming new memories through binding, a process that hierarchically links simple action elements (e.g., a single piano key press) to construct representations of complex spatiotemporal sequences (e.g., a refrain within a classical piano concerto) (Karni, 1995; Karni et al., 1998; Song and Cohen, 2014; Walker et al., 2003). How the brain binds sequences of discrete action representations into consolidated, temporally precise skills during waking rest is not known (Gerbier et al., 2015; Jacobacci et al., 2020; Kornmeier and Sosic-Vasic, 2012). One possible candidate mechanism is neural replay, the temporally compressed reactivation of neural activity patterns representing behavioral sequences during rest (Genzel et al., 2020). Previous work in rodents has shown that hippocampal replay constructs representations of navigational maps through relational binding of discrete spatial locations into trajectories (Carr et al., 2011). Furthermore, hippocampal activity has been documented during rest periods of early learning in humans (Albouy et al., 2015; Jacobacci et al., 2020). Thus, it is possible that waking replay, through offline recapitulation of prior practice (Carr et al., 2011; Diba and Buzsáki, 2007; Foster and Wilson, 2006; Singer and Frank, 2009), promotes wakeful consolidation of skill, an issue that has not been investigated in humans or animal models (Carr et al., 2011; Ramanathan et al., 2015).

In this study, we posited a contribution of neural replay in wakeful memory consolidation. We tested this by applying a multivariate analytical approach to magnetoencephalography (MEG) brain activity data acquired during skill learning to decode spontaneous, waking replay during the periods of rest in which sequences of simple discrete actions learned during interspersed practice are bound into a consolidated skill.

RESULTS

Wakeful consolidation of skill was evaluated using a keypress sequence task (Karni et al., 1995; Walker et al., 2003) that shows rapid memory strengthening over a single session (Bönstrup et al., 2019). Subjects were instructed to type the sequence 41324 on a response pad as fast and accurately as possible during each 10-s practice period. Thirty-six 10-s practice periods (trials) were interleaved with 10-s rest periods (Figure 1). Skill was measured as the correct sequence typing speed (sequences [seq]/s). Early learning, encompassing the set of trials within which 95% of total learning occurred, was explained by micro-offline, but not micro-online, gains (Figure 1D). That is, skill improvement occurred during waking rest, but not during active practice (Figures 1C and D).

Figure 1. Skill learning task and behavioral performance.

(A) Keypress sequence motor skill task. Subjects acquired a novel motor skill over a single training session. They were instructed to repeatedly type a trained sequence, 41324, with the left non-dominant hand as fast and as accurately as possible. Keypress 4 was performed with the left index finger, keypress 3 with the left middle finger, keypress 2 with the left ring finger, and keypress 1 with the left little finger. Following task instructions, a 5-min MEG waking rest recording was acquired prior to commencement of training (pre-training rest). Subjects then performed the task over 36 individual practice periods lasting 10 s each. Practice periods were interleaved with 10-s waking rest intervals (inter-practice rest). MEG was acquired continuously during training (12 min). A second, 5-min waking rest MEG recording was acquired after training concluded (post-training rest).

(B) Performance curve. Skill was measured as the correct sequence typing speed (sequences [seq]/s). Mean average performance (mean ± SEM) increased rapidly during early learning (the set of trials within which 95% of total learning occurred, trials 1–11; vertical dashed line).

(C) Instantaneous correct sequence typing speed (group mean ± SEM shown) was used to quantify micro-online performance gains during practice (cyan) and micro-offline gains during interleaved rest (magenta) for the early learning period.

(D) Micro-online (cyan), micro-offline (magenta), and total early learning (green) for each subject. Note that skill increases occur during intervening periods of waking rest, and not during active practice. Thick horizontal lines and associated boxes indicate the group mean and 95% confidence interval, respectively.

Fast replay of skill sequences is prominent during waking rest interspersed with practice

Next, we examined parcellated, source-space MEG brain activity data for evidence of spontaneous replay of the trained sequence during waking rest (Figure 2). We used radial basis support vector machine decoders to interrogate resting-state MEG data for individual replay events (see STAR methods for details). Practice data segments centered on individual keypress events were used to train four one-versus-all keypress state decoders (i.e., one decoder per finger). We then applied these decoders to waking rest MEG data to detect replay events (Figure 2A) that occurred during pre-training, inter-practice, and post-training rest periods. Since replay is known to be temporally compressed to different degrees relative to behavior, we looked for replay over a range of durations (25–2,500 ms) corresponding to different compression magnitudes (approximately 0.4–40×) relative to the acquired skill (the average correct sequence duration by the end of early learning was 1,037.7 ± 61.7 ms; mean ± SEM; Figure 1B).

Figure 2. Replay detections.

(A) Replay detection analysis pipeline. Raw MEG data were pre-processed, source-localized, and parcellated into 216 brain regions using the Brainnetome Atlas (Fan et al., 2016). Practice data segments centered on individual keypress events were used to train four one-versus-all radial basis support vector machine keypress state decoders (i.e., one decoder per finger). We then used the trained keypress state decoders to interrogate resting-state MEG data (preceding training, during rest periods interspersed with practice, and after the end of training) at each time point, T, for replay of keypress state sequences over a range of durations (25–2,500 ms) corresponding to temporal compression ranges (0.4–40×) previously reported (Davidson et al., 2009; Marchesotti et al., 2016; Papaxanthis et al., 2002). Decoder performance evaluation (Figure S1) was used to threshold minimum detectable sequence replay probabilities. Permutation testing against probabilities for all possible sequences (n = 1,024) at time point T was used to determine statistically significant detection of keypress sequence replay events.

(B) Neural replay was detected during waking rest. The maximum forward trained sequence replay rate was observed for events of 50-ms duration. This was consistent with observed replay for the reverse trained sequence (Figure S2A). Shaded regions indicate the 95% confidence interval of the group mean (thick lines).

(C) Waking replay rates (50-ms duration) for the forward trained sequence more than tripled during inter-practice rest (2.44 ± 0.29, mean ± SEM) relative to pretraining rest (0.75 ± 0.21). Similar magnitude rate changes were also observed for the reverse trained sequence (Figure S2B). Replay rates then receded during post-training rest (0.88 ± 0.21) to levels similar to pre-training. Thick horizontal lines and associated boxes indicate the group mean and 95% confidence interval, respectively.

(D) Within individuals, waking replay of the trained sequence increased more prominently (one-way ANOVA_{[forward, reverse, control]}: F_2,87 = 6.42; p = 0.002; forward versus control: p = 0.0062, Bonferroni-corrected; reverse versus control: p = 0.0099, Bonferroni-corrected; see also Figure S2C) for the trained sequence than for a control sequence (33433; right). This control was selected on the basis that it was the only one out of 1,022 possible alternate sequences sharing no common ordinal position or transition structure with the forward (41324) and reverse (42314) trained sequence. Pre-training rest replay rates did not significantly differ between these sequences (F_2,53.94 = 2.431; p 0.098). Of note, this same trend was observed even after expanding these criteria to consider additional sequences sharing either a single common ordinal position or a single common transition with the practice sequence (Figure S2C).

Neural replay of the full trained sequence was identified in all waking rest periods examined. However, detection rates (i.e., the number of replay events detected per second) varied depending on replay duration (i.e., compression magnitude; Figure 2B). Forward replay rates for the trained sequence (41324) were most prominent for events lasting 50 ms and progressively decreased for longer event durations (Figure 2B; see Figure S2A for reverse replay of the trained sequence, 42314). Thus, the modal neural replay representation of the skill was temporally compressed approximately 20-fold relative to the actual behavior. Figure 2C highlights the large difference in forward replay rates across the different waking rest states (F_2,58 = 15.26, p = 0.000018). Specifically, forward replay events during inter-practice rest occurred approximately 3-fold more frequently (2.44 ± 0.29 replays/s) relative to pre-training (0.75 ± 0.21 replays/s, p = 0.00528) or post-training rest (0.88 ± 0.21 replays/s, p = 0.00089, Figure 2C), which were both comparable (p = 0.2972). Reverse replay of the trained sequence exhibited similar features (Figure S2B). Elevated replay rates were maintained across all individual inter-practice rest periods (Figure S3A). Within each of these rest periods, replay events were likely to occur in clusters (Figures S3B and S3D) that were uniformly distributed across the entire 10 s (Figures S3B and S3C). Finally, mean replay rates during early learning (initial 11 trials) were significantly higher than during late learning (last 11 trials; t₂₉ = 2.66, p = 0.006, one-tailed).

We next asked whether the increase in replay rate induced by practice was selective to the trained sequence. This question was addressed by interrogating resting-state MEG data for replay of an unpracticed sequence (control; 33433), which was selected on the basis that it is the only unpracticed sequence combination (out of 1,022 possible alternatives) that shares no common ordinal position or transition structure with forward and reverse replay of the practiced sequence. After practice onset there was a greater increase for replay of the trained sequence relative to this control (one-way ANOVA_{[forward, reverse, and control]}: F_2,87 = 6.42; p = 0.002; forward versus control: p = 0.0062; Bonferonni-corrected; Figures 2D and S2C). Thus, wakeful replay is selective for, but not exclusive to, the trained motor skill sequence.

Replay of the learned skill is represented within a mediotemporal-sensorimotor network

We also examined the spatiotemporal features of replay event dynamics. After averaging all 50-ms duration forward replay event epochs detected by the decoding pipeline during inter-practice rest, we applied principal-component analysis (PCA) to extract low-dimensional networks with co-varying source-space power changes during fast replay. Two orthogonal networks explained 86.6% of the total signal variance during an average replay event. The first network (PC1 explaining 67.8% of total variance) featured sensorimotor, entorhinal, and hippocampal regions (Figure 3). It also included the precuneus, one of the main outputs of the hippocampus to the neocortex (Buzsáki and Tingley, 2018; Margulies et al., 2009), shown to be active in previous fMRI work during rest periods of early learning (Jacobacci et al., 2020). The second network (PC2) included a similar set of regions except for the precuneus (Figure S4). Thus, replay of the skill is represented predominantly in sensorimotor, entorhinal, and hippocampal regions.

Figure 3. Spatial features of detected replay events.

Principal-component analysis (PCA) was used to characterize networks with covarying power changes in parcellated MEG source-space during group average replay events. Together, PC1 and PC2 explain 86.6% of the total power-related variance during replay. PC1, shown here, explains 67.8% of the total variance alone and is characterized by a positively correlated sensorimotor-entorhinal-hippocampal network. PC2 displays a similar set of regions (Figure S4). Surface maps are thresholded to display the highest loading parcels (see STAR methods).

Replay of the trained sequence correlates with rapid skill consolidation

Finally, we asked whether wakeful consolidation in individual participants could be predicted by the rate of replay events detected for the trained sequence during inter-practice rest intervals. We observed significant correlations (Figure 4) between cumulative micro-offline gains over early learning and replay of the trained sequence (Figure 4A; r = 0.451, p = 0.012). Alternatively, replay of the control sequence (33433; see STAR methods) did not correlate with behavior (Figure 4B; r = 0.165, p = 0.38). In summary, greater replay rates related to the trained sequence predicted greater rapid wakeful skill consolidation across individuals.

Figure 4. Forward and reverse neural replay of the trained sequence predict rapid wakeful skill consolidation.

The scatterplots depict inter-individual replay rates over all early learning inter-practice rest periods (abscissa) relative to cumulative micro-offline gains (ordinate) over the same time window (trials 1–11). Shaded regions indicated the 95% confidence interval.

(A) A significant correlation was observed for forward replay of the trained sequence (r = 0.451, p = 0.012). A similar finding was observed for reverse replay of the trained sequence (r = 0.521, p = 0.003). Thus, higher rates of replay related to the trained sequence predicted greater micro-offline gains and rapid wakeful skill consolidation in individuals.

(B) Conversely, replay during inter-practice rest of the control sequence (33433) did not display a significant correlation with rapid wakeful skill consolidation (r = 0.165, p = 0.38) during early learning.

DISCUSSION

In this study, we address the question of how the brain consolidates memory during waking rest. Positing a role for neural replay, we first document replay in medial temporal and sensorimotor brain regions during wakeful human memory consolidation. We then demonstrate that replay is temporally compressed by approximately 20-fold relative to the practiced behavior. Finally, we show that forward and reverse waking replay rates increase rapidly after the onset of practice, display selectivity for the trained sequence, and predict consolidation of skill.

The presence of waking rest interspersed with practice influences successful learning. For example, skill memory is enhanced when practice events are separated by rest rather than massed in immediate succession, a phenomenon known as the “spacing effect.” This observation, initially made by Ebbington in 1885 (Ebbinghaus, 2013), has been repeatedly documented across cognitive domains (Cepeda et al., 2006; Gerbier et al., 2015; Kornmeier and Sosic-Vasic, 2012; Song et al., 2012; Toppino et al., 2009). Waking rest also contributes to skill acquisition. Early learning (a period where performance improves exponentially) is largely accounted for by micro-offline gains that occur between, rather than during, practice periods (Bönstrup et al., 2019). Thus, a form of wakeful consolidation of skill develops over much shorter timescales (i.e. seconds and minutes) than previously thought (Bönstrup et al., 2020). Importantly, this form of wakeful memory consolidation is approximately 4-fold greater in magnitude than classically studied overnight consolidation, which requires sleep (Bönstrup et al., 2019; Jacobacci et al., 2020) and is preserved even when practice experience is reduced by half (Bönstrup et al., 2020). In this study, we investigated how the brain binds, during waking rest, sequences of discrete motor actions learned during previous practice, leading to successful consolidation of skill (Liu et al., 2019).

Waking replay, displaying robust temporal features, was decoded during the same rest periods in which rapid consolidation occurs. Replay of full practiced sequences had a modal duration of approximately 50 ms, representing a 20-fold temporal compression of keypress sequence representations within each replay event relative to the duration of the actual skill behavior. This is consistent with the range of compressions previously reported in animal models as well as in humans (Davidson et al., 2009; Liu et al., 2019), and it is too fast to be explained by mental rehearsal or motor imagery processes, which share a veridical duration with executed behavior (in this case, 1,000 to 2,500 ms) (Marchesotti et al., 2016; Papaxanthis et al., 2002).

Immediately following the initiation of practice, replay rates during interspersed rest periods increased more than 3-fold beginning with the first one (Figure S3A). This finding is reminiscent of the increase in waking replay rates in rodents upon introduction into a novel maze environment (Carr et al., 2011; Fernández-Ruiz et al., 2019). We also found a high likelihood for temporal clustering, as more than half of all detected replay events were paired with a second event within a window of <200 ms (Figure S3D). Such replay clustering (Liu et al., 2019) has been linked to memory consolidation (Carr et al., 2011). Following practice, replay rates receded to pre-training levels rather rapidly contrary to the slow decrease reported in rodents (Karlsson and Frank, 2009), a difference that could be attributed to the explicit awareness of training termination in humans (likely to be absent in rodents) or to the presence of reward contingencies in rodent behavioral paradigms (absent in this study) (Singer and Frank, 2009).

The increase in waking replay (Diba and Buzsáki, 2007) rates for the trained sequence was larger than for the alternative unpracticed sequences examined (Figures 2D and S2C). Replay of sequences not yet experienced has been extensively reported in the literature. First, studies of navigation memory in rodents have repeatedly shown that hippocampal cell assemblies generate plausible behavioral trajectories or sequences that have not yet been explored (Dragoi and Tonegawa, 2011, 2013; Gupta et al., 2010). It is possible that such replay events reflect more generalizable abstract knowledge structures (i.e., cognitive maps) important for flexible skill adaptation and transfer (Behrens et al., 2018). The keypress sequence representations we also observed during pre-training rest after subjects had received task instructions, but before they initiated practice (Figures 1A and 2B), can be similarly explained. Evidence of “pre-play” in rodents indicates that these reactivation patterns could be generated from intrinsic neural dynamics supporting broad sequence representations that are subsequently tuned through experience (Dragoi and Tonegawa, 2011; Farooq et al., 2019). Alternatively, it is also possible that semantic knowledge about the task requirements was enough to elicit replay of keypress sequences in preparation of the impending training (Liu et al., 2019).

Learning this skill requires binding individual discrete keypress actions into a fast, precise spatiotemporal sequence (Wong and Krakauer, 2019). While neural representations of individual keypresses—stereotypical, overlearned finger movements—are highly invariant and maintained within the primary motor cortex (Yokoi and Diedrichsen, 2019), keypress sequence skills are spatially represented throughout a distributed frontoparietal brain network that reorganizes with learning (Albouy et al., 2015; Jacobacci et al., 2020; Wong and Krakauer, 2019; Yokoi and Diedrichsen, 2019). Waking neural replay events characterized herein were encoded in regions that differed from those reported for practice sequences (Yokoi and Diedrichsen, 2019) in the proportion of hippocampal-entorhinal engagement. The most parsimonious explanation for this difference is that during the rest periods within which rapid consolidation occurred, synchronization of hippocampus with engaged neocortical regions potentiated the skill memory to a greater extent than during practice itself (Dudai, 2004; Squire and Zola-Morgan, 1991). For individual replay events, which were detected on average between 20 and 30 times during each 10-s rest period, the hippocampus and entorhinal cortex could engage memories encoding abstract knowledge about the action structure of the skill sequence, while the sensorimotor cortex may engage memories containing information about the kinematics and dynamics of the elemental finger movements. In this way, replay initiated by the hippocampus could promote the binding of abstract knowledge about the action structure of the skill with detailed sensory predictions important for execution and evaluation of the skill (Alvarez and Squire, 1994; Gonzalez Castro et al., 2011; Whittington et al., 2020). This view is consistent with evidence of hippocampal involvement during rest periods of early learning (Albouy et al., 2015; Jacobacci et al., 2020). Taken together, our data indicate that frequent, fast waking replay reinforces hippocampus and neocortical associations learned during prior practice, a process relevant for improving subsequent performance and wakeful consolidation of skill. Still, while the correlation between replay of the trained sequence and micro-offline learning is strongly suggestive of a direct contribution to consolidation of skill (Bönstrup et al., 2019), causality remains to be established in either animals or humans (de Sousa et al., 2019; Eldar et al., 2020).

We conclude that robust hippocampal and neocortical replay predicts micro-offline learning and is a possible binding mechanism supporting wakeful memory consolidation.

Limitations of the study

As with any classifier-based analysis, the ability to discriminate between accurate detection of biological phenomena and mislabeling artifacts is an important challenge. While the thresholding and significance testing procedures embedded within the current replay detection pipeline aim to mitigate this issue (i.e., false-positive sequence replay detections), it cannot be entirely eliminated. One limitation of our approach is that it jointly considers the independent classification of keypress states at each ordinal position in the sequence. Thus, sequences with shared ordinal structure are more likely to be confused. For example, since forward and reverse replay of the practiced sequence share the same keypress states at the first, third, and fifth ordinal positions, mislabeling of the other two positions could conceivably result in incorrect detection of replay direction. The average likelihood of this type of misclassification can be estimated by calculating the product of the respective average ordinal probabilities for 41324 (true positive detection of forward replay) and 42314 (false positive detection of reverse replay) during practice or rest (Figures S1B–S1F) and comparing them. Based on these comparisons, we estimate that approximately 1 in between 6 and 11 detected forward replay events for the practiced sequence could have the direction incorrectly classified (i.e., labeled as reverse). Further investigation could help refine this approach for detecting replay in MEG data.

STAR★METHODS

RESOURCE AVAILABILITY

Lead contact

Further information and requests for resources should be directed to and will be fulfilled by the lead contact, Ethan R Buch (ethan.buch@nih.gov).

Materials availability

This study did not generate new materials.

Data and code availability

Data (subject to participant consent) used to generate the findings of this study will be made freely available upon request to the Lead Contact, Ethan R Buch (ethan.buch@nih.gov). Custom code generated during this study are available at https://github.com/hcps-ninds/Replay.

EXPERIMENTAL MODEL AND SUBJECT DETAILS

Participants

Thirty-three naive right-handed healthy participants (16 women, mean ± SEM age 26.6 ± 0.87) with a normal neurological examination gave their written informed consent to participate in the project, which was approved by the Combined Neuroscience Institutional Review Board of the National Institutes of Health (NIH). Active musicians were excluded from the study (Abraham et al., 2014). The sample size was determined a priori via a power analysis of prior skill learning data collected in our research group using the same task (Censor et al., 2014a, 2014b). Two participants were excluded before the start of analysis because of technical problems with the MEG scanner during the recording sessions. One additional participant was excluded due to large movement artifacts in the MRI scan required for source modeling.

METHOD DETAILS

Behavior

Task

Participants performed a novel explicit motor skill sequence task where they repetitively typed a 5-item numerical sequence displayed on a computer screen (41324) as quickly and as accurately as possible. Keypresses were performed with the participant’s non-dominant, left hand and applied to a response pad (Cedrus LS-LINE, Cedrus Corp). Keypress 1 was performed with the little finger, keypress 2 with the ring finger, keypress 3 with the middle finger and keypress 4 with the index finger. The thumb was not used to respond. Individual keypress times and identities were recorded for behavioral data analysis. Participants practiced the task for thirty-six (36), 10 s duration trials. 10 s rest periods were interleaved between trials. During practice, participants were instructed to fixate on the five-item sequence displayed for the full duration of the trial. Small asterisks appeared above a sequence item when a keypress was recorded, providing feedback to the participant about their current location within the sequence. After completion of a full iteration of the sequence, the asterisks were removed. The asterisk display did not provide error feedback since they appeared for both correct and incorrect keypresses. During the 10 s interleaved rest periods, the sequence was replaced with a string of five “X” symbols, which participants were instructed to fixate on. Visual stimuli and task instructions were presented, and keypress responses recorded using a custom script running in E-Prime 2 (Psychology Software Tools, Inc.).

Magnetic resonance imaging (MRI)

MRI acquisition

Structural MRI scanning was performed on a 3T MRI scanner (GE Excite HDxt and Siemens Skyra) with a standard 32-channel head coil. T1-weighted high-resolution (1mm³ isotropic MPRAGE sequence) anatomical images were acquired for each participant.

Magnetoencephalography (MEG)

MEG acquisition

Continuous MEG was recorded at a sampling frequency of 600Hz with a CTF 275 MEG system (CTF Systems, Inc., Canada) while participants were seated in an upright position. The system is composed of a whole head array of 275 radial 1^st-order gradiometer/SQUID channels housed in a magnetically shielded room (Vacuumschmelze, Germany). Two of the gradiometers were malfunctioning and were not used. A third channel was removed (MLT16–1609) after visual inspection of the data indicated the presence of high noise levels for all recordings, resulting in 272 total channels of MEG data. Synthetic 3rd order gradient balancing was used to remove background noise online. A TTL trigger sent from the task computer to the MEG data acquisition computer was used to temporally align the behavioral and MEG data. Head position indicator (HPI) points were monitored in the scanner coordinate space using head localization coils (HLCs) placed on the nasion, left, and right pre-auricular locations of the participant’s head. These coil positions were also recorded in the subject’s T1-MRI coordinate space during the session using a stereotactic neuronavigation system (Brain-Sight, Rogue Research Inc.).

QUANTIFICATION AND STATISTICAL ANALYSIS

Behavior

Skill measure calculation

Instantaneous correct sequence speed (skill measure) was quantified as the inverse of time (in seconds) required to complete a single iteration of a correctly generated full 5-item sequence. Individual keypress responses were labeled as members of correct sequences if they occurred within a 5-item response pattern matching any possible circular shifts of the 5-item sequence displayed on the monitor (41324). This approach allowed us to quantify a measure of skill within each practice trial at the resolution of individual keypresses and identify all possible types of keypress response errors (i.e., – omission, replication, or substitution). Performance within each trial was summarized as the median instantaneous correct sequence speed over the full trial.

Determination of the early learning trial cutoff

Here, early learning corresponds to the set of trials between trial 1 and a cutoff trial, T, on which 95% of the performance gains are first achieved at the group level. We determined T as follows. First, we fit the trial-by-trial group average correct sequence speed data with an exponential model of the form:

$$L\left(t\right)=C_1+C_2\cdot \left(1-e^{-\text{λ}t}\right),$$ (1)

where L(t) represents the group average learning state on practice trial, t, C₁ and C₂ control the pre-training performance and asymptote (i.e., – extent of learning), and λ controls the learning rate. Parameters C₁ (boundary constraints = [0,5]), C₂ ([0,15]) and λ ([0,2]) were estimated using a constrained nonlinear least-squares method (MATLAB’s lsqcurvefit, trust-region-reflective algorithm). Next, we accumulated L(t) (Equation 1) over all t and divided each trial’s cumulative sum by the area under the curve to obtain a trial-by-trial cumulative percentage of learning. Finally, we set T to the first trial where 95% of the learning had been achieved (T = 11 in this study).

Calculation of micro-online and micro-offline gains during early learning

We then assessed early learning changes at the individual trial level by quantifying performance improvements occurring during individual practice and rest periods. Micro-online learning was defined as the difference in instantaneous correct sequence speed between the beginning and end of a practice trial. Micro-offline learning was defined as the absolute difference in correct sequence speed between the end of a practice period and the beginning of the subsequent practice period. Cumulative micro-online and micro-offline learning scores measured over the first 11 trials for each participant were used to assess their respective contribution to total early learning (i.e., - change in performance between Trial 1 and Trial 11) (Bönstrup et al., 2019).

Magnetoencephalography (MEG)

MEG data preparation and pre-processing

Each participant’s MEG data were organized into the following samples: pre-training rest (single segment of 5 min), inter-trial practice rest (36 segments of 10 s), practice (36 segments of 10 s) and post-training rest (single segment of 5 min). All MEG samples were band-pass filtered between 1–100Hz, notch-filtered at 60Hz and subjected to Independent Component Analysis (ICA). ICA was fit using MNE Python’s (Gramfort et al., 2013) implementation of FastICA (Fan et al., 2016) after pre-whitening with Principal Component Analysis (set to retain the top 15 components). Spatial topographies and time series of all components were visually inspected and those determined to be strongly associated with known electrocardiogram, eye movement and blinks, and head movement artifacts were removed from the MEG sample.

MEG source reconstruction and parcellation

Source reconstruction of the MEG data was performed using the standard pipeline in MNE Python (Gramfort et al., 2014). A single forward solution and inverse solution was calculated for each MEG sample. A boundary element model (BEM) required for generation of the forward solution was constructed for individual subjects from inner-skull and pial layer surfaces obtained by segmentation of the subject’s T1-MRI volume in FreeSurfer (Fischl, 2012). Hippocampal surface labels were obtained by parcellating the participant’s MRI according to the coordinates in the Brainnetome atlas (Fan et al., 2016) (https://atlas.brainnetome.org/). Neocortical and hippocampal source spaces were obtained by sampling the corresponding surfaces on a recursive octahedral grid with 4.9mm between dipoles, and spatially aligned to the MEG sensor data with the co-registration data obtained from BrainSight. The BEM was then used to generate the forward solution at each source dipole location. The Linearly Constrained Minimum-Variance (LCMV) beamformer operator was used to compute the inverse solution from the recorded MEG sensor space data. The entire duration of each MEG sample was used to calculate the inverse solution data covariance matrix. All inverse solutions used the same sample noise covariance matrix, which was calculated over the first 20 s of the pre-training rest MEG sample for individual subjects.

MNI-space transformations obtained from the FreeSurfer segmentation pipeline were then used to spatially register the resulting source reconstruction time series to the Brainnetome atlas, enabling quantitative estimation of millisecond-resolved brain activity for 212 neocortical and 4 hippocampal parcels (i.e., − 216 total parcels). Parcel activity was estimated by averaging the time series of all source dipoles falling within a given parcel boundary. A process called mean-flipping was used to prevent within-parcel source cancellation when averaging. That is, the sign was flipped for all sources with a sign that differed from that of the average source. Since the sensor space data had been bandpass filtered with a cut-off frequency of 100Hz, the source data were downsampled from 600Hz to 200Hz to increase computation speed in subsequent processing steps. All remaining analyses were performed directly on the parcellated source-space MEG time series.

Training and evaluation of keypress decoders

Individualized keypress decoders were trained for each participant using Python’s Scikit-learn library (Pedregosa et al., 2011). The training data of a subject’s decoder was assembled as follows. We time-averaged the parcellated source for each correct keypress in all 36 practice MEG samples between t ± ∆t ms, where t is the keypress timestamp and ∆t is treated as a cross-validated hyperparameter described in detail below. The timestamp for each keypress was obtained from behavioral data files output by E-prime. This produced the predictor matrix X_{[keypresses × parcels]} and the target array y_{[keypresses × 1]} where keypresses varied between subjects (some typed more sequences than others) and parcels = 216 as earlier described.

We used X and y to train four one-versus-all Radial-basis Support Vector Machines (Python Scikit-learn’s SVC) with a pipeline parameterized with a feature centering/scaling policy, a time window for averaging the MEG source around a keypress (∆t), a regularization term (C) and a kernel coefficient (γ). We solved for these (hyper)parameters by running 5-fold cross-validated grid-search hyperparameter optimization three times, one for each of the following optimality criteria:

$$\overline{\text{F}1}=Averag{e}_{keys}\left(\text{F}1\left(key\right)\right)\text{where F}1\left(key\right)=2\cdot \frac{Precision\left(key\right)\cdot Recall\left(key\right)}{Precision\left(key\right)+Recall\left(key\right)},$$

$$\text{BA}\left(\text{Balanced Accuracy}\right)=\sum _{key=1,2,3,4}Accuracy\left(key\right),$$

$$\text{WBA}\left(\text{Weighted Balance Accuracy}\right)=\sum _{key=1,2,3,4}Weight\left(key\right)\cdot Accuracy\left(key\right),$$

where Weight(key) = [2/7, 2/7, 2/7, 1/7] for keys 1, 2, 3 and 4, respectively, and Accuracy(key) = percentage correct classification of key. For the second and third optimality criteria, classification was done by picking the key that maximized the probability estimated by the SVCs, rather than using the decision function. The search grid used was: C = [0.1, 0.5, 1, 10, 50, 100, 1000], gamma = [0.05, 1/(2 × keypresses), 0.1, 0.5, 1], center policy = [True (will remove median), False (will not remove median)], standardization policy = [True (will divide by inter-quartile range (IQR)), False (will not divide by IQR)] and ∆t = [5, 10, 25] – that is a total of 420 configurations. SVCs were trained with a target error of 10⁻³ or less.

Three cross-validated grid searches were run for each participant, one for each of the three criteria:$\overline{\text{F1}}$, BA and WBA. Grid search under a particular criterion worked as follows. First, the training set was divided into 5 mutually exclusive stratified partitions so that the proportions of all keypresses were preserved as in the original data (since the training sequence was 41324, each folder would have approximately twice as many keys = 4). Next, we used the random oversampler in Python’s Imbalanced-learn toolbox (Lemaitre et al., 2017) (with a “not majority” sampling policy) to independently oversamples keypresses 1, 2 and 3 within each training partition. Note that test partitions were never oversampled, and training partitions did not share common oversampled patterns as this would lead to double-dipping and ultimately overfitting. Finally, we performed the same 5-fold cross-validation procedure on each of the 420 configurations in the grid. Each configuration produced 5 estimates, resulting in a total of 2100 estimates. We then selected the decoder configuration with the best cross-validated performance (data in test partitions only, and performance under the considered optimality criterion) for each individual participant. To evaluate the decoder’s performance, we created a null performance set by permuting the test labels (i.e., –identities of all keypresses) 1000 times and calculating the decoder’s performance after each permutation, leading to a p value = (correct classifications + 1) / (1000 + 1). We also calculated the decoder’s cross-validated confusion matrix, that is, the confusion matrix calculated over the data in the cross-validated test partitions. This confusion matrix played an important role in our replay detection pipeline as described later.

Estimation of decoder performance on practice sequence detection

The goal of the present step was to select the best decoder for a participant based on their estimated performance on full practiced sequences. Recall that keypress decoder training resulted in three decoder options for each participant (i.e., −$\overline{\text{F}1}$, BA and WBA) with associated cross-validated performances, p values and cross-validated confusion matrices. We used the following process to select the optimal decoder for each participant. First, we discarded any decoder with a p value > = 0.05. This step resulted in at least one viable decoder for all participants. Next, we reshaped each participants’ predictor matrix X^practice_{[keypresses × parcels]} back into the correctly typed sequences they originated from, resulting in an associated array X^practice_{[5 × parcels × N]} for each subject, where N is the number of correctly typed sequences of 4-1-3-2-4.

We then evaluated the ability of the decoders to correctly identify each typed practice sequence from j = 1…N using the following procedure:

1. Scored the probability, p_seq[j], of X^practice_{[:, :, j]} being identified as sequence 4-1-3-2-4 as the product of the probabilities of each of the sequence’s constituent keys, p_key( ·), in the correct order. Probabilities p_key( ·) were obtained from the keypress SVC decoders for each subject.

2. Repeated Step 1 for all other possible sequences (i.e., – all other 1023 5-digit combinations of 1,2,3 and 4). Combined, these sequences form the set of null probabilities, Ω.

3. From Ω, we calculated the p value for detection of the j-th practiced sequences as p-value = 1 – percentile^Ω(p_seq [j]/100 and counted a detection of typed sequence j if the p value < 0.05.

Finally, we obtained the sequence detection rate of a decoder by dividing the number of detections by the total number of practiced sequences, N. We then selected the decoder option with the top detection rate for sequences observed in the practice data. Figure S1B shows the distribution of top detection rates across participants. Figure S1C shows keypress probabilities first averaged at each ordinal position of detected practice sequences and then averaged over subjects. Note that the expected key at each position is always the keypress with the top detection probability. All participants had at least one decoder with a non-zero detection rate. The selected decoder was the one used to interrogate waking rest MEG data samples for replay events.

Sequence replay detection and replay rate calculation

Given a rest period MEG sample, X^rest _{[T × parcels]}, of T time-points x parcels recorded from a given subject, we estimated the probability that a given sequence, seq = ABCDE, was replayed at a time 0 ≤ t < T within that sample to be the product of the probabilities of each of the sequence’s constituent keys A, B, C, D and E ∈ [1, 2, 3, 4] being observed both in the right order and ending at time t, that is:

$$p_{seq}=ABCDE\left[t\right]=p_{key=A}\left({\mathbf{X}}^{\text{resampled}}\left[t-4,:\right]\right)\cdot p_{key=B}\left({\mathbf{X}}^{\text{resampled}}\left[t-3,:\right]\right)\cdot \cdot \cdot p_{key=E}\left({\mathbf{X}}^{\text{resampled}}\left[t,:\right]\right),$$ (2)

Where X^resampled = resample (X^rest[t − 5·tscl + 1 : t, :] is an MEG sample segment extracted from t − 5·tscl + 1 to t at all 216 parcels and compressed down to 5 time-points. Probabilities of different replay durations were obtained by varying the timescale compression factor, tscl, where tscl∈ [1,…, 10; 15; 20; 30; 40; 80; 100].

For example, since our time resolution is 5ms, we set tscl = 1 to look for 25ms replay durations, tscl = 2 for 50ms replay durations, and up to as high as tscl = 100 for 2500ms durations. In terms of implementation, we obtained X^resampled using Python’s scipy resample with default arguments and num = 5. Again, probabilities p_key( ·) were obtained from the keypress SVC decoders selected for each subject. Scikit-learn implements a multiclass version of Platt’s scaling technique to obtain probability distributions from classifications over more than two classes (Wu et al., 2004).

We only assessed replay likelihood at time-points where p_{seq = ABCDE}[t] exceeded a threshold, θ_seq, set to the product of the maximum off-diagonal detection rates (i.e., false positive rates) of the keypress decoder’s cross-validated confusion matrix. For each time-point, t, such that p_{seq = ABCDE}[t] > θ_seq, we then tested the associated statistical significance as to “how rare” p_{seq = ABCDE}[t] would be under the null distribution of probabilities for all other 1,023 possible decodable sequences (i.e., – the Ω set) resulting in a p – value = 1 – percentile^Ω(p_seq[t])/100. All p values were FDR-corrected (Benjamin-Hochberg) to account for multiple comparisons using an α = 0.05 with the number of tests set to the size of Ω. By pre-selecting which time-points to test, we mitigated underpowering effects of correcting for multiples comparisons. Note that the selection threshold θ_seq was fully determined from practice data (cross-validated confusion matrices) and never from test data on which statistical testing was carried out, therefore eliminating the possibility of circular analysis or double-dipping. Statistically significant replay detections were marked for all p values[t] < 0.05. Since it was possible that overlapping X^resampled[t] could result in multiple detections of the same putative replay event, each set of multiple detections occurring (within the same tested replay duration) was replaced with a single detection at the median time instant of the set. Finally, we scored the replay rate of seq during the given rest period MEG sample as the number of detections per the sample’s duration in seconds:

$$repla{y}_{seq}=counts\left(p_{seq}\left[t\right]<\alpha \right)\cdot fs/T,t\in \text{Ω}$$ (3)

where counts( ·) is the number of detections of seq along the timeline of the MEG sample, and fs is the sampling rate of the MEG sample (i.e., – fs/T is the inverse of the duration of the sample in seconds).

Replay network characterization

While use of a nonlinear kernel optimized the performance of the keypress state decoders, it precludes direct assessment of the spatial representations of these states (and sequence dynamics) from the decoder weights. However, since the output of the replay detection pipeline specifies the time-point at which individual replay events are initiated and what their duration is, it is possible to gain insight into replay network dynamics by interrogating these event epochs. After averaging all 50ms forward replay (i.e., − 41324) event epochs, we performed principal component analysis (PCA) to extract low-dimensional networks displaying strong covariation in source power during replay. We mapped the membership coefficients for cortical regions onto the FreeSurfer average brain, and the four hippocampal subregions onto a separate surface representation. The thresholded maps display the top 10% of PC loading coefficient magnitudes (i.e., – absolute value of signed coefficients).

Correlation of replay rates with behavior

We performed Pearson correlation analyses of the average forward and reverse trained sequence replay rates detected during the first eleven 10 s waking rest periods with cumulative micro-offline learning over the same period. We also performed a control analysis by the assessing the correlation between replay of a single alternative sequence (33433) with cumulative micro-offline learning over trials 1–11. This is the only alternative sequence (1 out of 1022) which does not share any common ordinal position or transition structure with the forward and reverse of the trained sequence.

KEY RESOURCES TABLE.

REAGENT or RESOURCE	SOURCE	IDENTIFIER
Deposited data
MEG Data	This paper	N/A
Software and algorithms
MATLAB	MATLAB	https://www.mathworks.com
Python	Python	https://www.python.org
E-prime 2	Psychology Software Tools, Inc., Sharpsburg, PA, USA	https://pstnet.com/products/e-prime/
FreeSurfer		https://surfer.nmr.mgh.harvard.edu/
MNE		https://mne.tools/stable/index.html
Custom code and algorithms	This paper	https://github.com/hcps-ninds/Replay
Other
CTF MEG 275 system	CTF MEG International Services LP. Coquitlam, BC, Canada	https://www.ctf.com
Cedrus LS-LINE	Cedrus Corporation. San Paolo, CA, USA	https://www.cedrus.com/lumina/
GE Discovery MR750 3T scanner system	GE Healthcare. Chicago, IL, USA	https://www.gehealthcare.com/products/magnetic-resonance-imaging/3–0t/discovery-mr750
Siemens Skyra	Siemens Healthineers. Erlangen, Germany	https://www.siemens-healthineers.com/magnetic-resonance-imaging/3t-mri-scanner/magnetom-skyra
NIH HPC Biowulf Cluster	NIH, Bethesda, MD, USA	https://hpc.nih.gov/

Highlights.

• Waking replay bursts occur during short rest periods interspersed with practice

• Replay is temporally compressed by approximately 20-fold relative to the practiced skill

• Replay activity is distributed across the hippocampus and neocortex

• Replay rates correlate with rapid wakeful skill consolidation in individuals