Neural ensemble reactivation in rapid eye movement and slow-wave sleep coordinate with muscle activity to promote rapid motor skill learning

Abstract

Neural activity patterns of recent experiences are reactivated during sleep in structures critical for memory storage, including hippocampus and neocortex. This reactivation process is thought to aid memory consolidation. Although synaptic rearrangement dynamics following learning involve an interplay between slow-wave sleep (SWS) and rapid eye movement (REM) sleep, most physiological evidence implicates SWS directly following experience as a preferred window for reactivation. Here, we show that reactivation occurs in both REM and SWS and that coordination of REM and SWS activation on the same day is associated with rapid learning of a motor skill. We performed 6 h recordings from cells in rats' motor cortex as they were trained daily on a skilled reaching task. In addition to SWS following training, reactivation occurred in REM, primarily during the pre-task rest period, and REM and SWS reactivation occurred on the same day in rats that acquired the skill rapidly. Both pre-task REM and post-task SWS activation were coordinated with muscle activity during sleep, suggesting a functional role for reactivation in skill learning. Our results provide the first demonstration that reactivation in REM sleep occurs during motor skill learning and that coordinated reactivation in both sleep states on the same day, although at different times, is beneficial for skill learning.

This article is part of the Theo Murphy meeting issue ‘Memory reactivation: replaying events past, present and future’.

1. Introduction

A growing body of evidence suggests that neural activity during rest and sleep is related to awake cognition and behaviour in a reciprocal relationship, although the details of this relationship are poorly understood. One example of this relationship between awake and resting neural activity is the reactivation of recent experience during sleep. In the rodent hippocampus, sequences of place cell activity that are experienced during behaviour are reactivated (or replayed) during subsequent sleep [1,2]. Since its discovery, reactivation has been observed in other brain areas, including the neocortex [3,4]. Notably, the hippocampus and neocortex are critical for memory processing, suggesting an important role for reactivation in the consolidation of memory. Different terms related to memory reactivation are now common in the literature, thus for clarification, the terms pre-/reactivation typically refer to coincident activation of task-related ensembles before or after the task epoch (as described in this study), while the terms pre-/replay typically refer to matching a sequential pattern of cell firing before or after the task epoch, although these terms are sometimes used interchangeably during general discussion of the phenomenon.

Behavioural evidence suggests that memory reactivation improves memory performance [5–10]. The disruption of reactivation in the hippocampus by using electrical impulses during sharp-wave- ripples impairs memory recall [11,12]. In addition, growing evidence suggests that the enhanced processing of memory during sleep can improve behaviour performance. The association of neutral sensory stimuli such as tones during learning can improve memory if the stimuli are played during sleep [13], a process that appears to work by biasing replay content during sleep [14].

Still missing from the evidence is a more detailed understanding of how the different stages of sleep might reactivate neural activity experienced during behaviour, and how different kinds of memory—declarative (memory for times and places, hippocampus-dependent) versus non-declarative (includes motor skills)—might be reactivated. Most data supporting reactivation have been sampled from a relatively short time window of rest following behaviour (typically less than 1 h). Since rodents experience a sleep structure that is grossly similar to humans, i.e. cycles of slow-wave sleep (SWS) and rapid eye movement (REM) sleep, with the sleep immediately following behaviour rich in SWS, most evidence for reactivation has come from SWS. One study has demonstrated reactivation in REM sleep, but it is notable that most of the observed reactivation actually occurred in sleep periods prior to behaviour [15], suggesting a possible pre-activation of upcoming experience [16].

We have sampled neural activity from 6 h sleep periods in rats across several weeks of training on a motor skill task. We show that coordinated reactivation between SWS and REM sleep is correlated with the fastest skill acquisition. Reactivation is also coordinated with muscle twitches during sleep, suggesting a functional link of reactivation. Finally, we observed that particularly strong periods of reactivation persist for a brief time upon waking.

2. Results

(a). Skill learning and sequential cell firing in motor cortex

Single units were recorded from the forelimb region of the primary motor cortex of five rats while they were trained on the single-pellet reaching task, as well as during 3 h pre- and post-task rest periods (figure 1a,b). Rats were naive to the task initially, except for brief testing of paw preference prior to surgery. Training occurred daily during a 30 min period, typically including 60 trials, and continued until asymptotic performance was achieved. Learning rates varied between animals, taking between 10 and 20 days to reach maximum success rates of 60–85%. Although performance was variable, two rats exhibited a more rapid acquisition of the skill, showing major performance gains from Day 3 to Day 4, followed by a more gradual increase in performance thereafter (figure 1c, Rats 1 and 2). Three rats exhibited a gradual increase in success over several days that has been reported previously using Long Evans rats (figure 1c, Rats 3–5) [17–19]. Because of this broad difference in the learning rate, we applied the terms rapid and gradual learners to distinguish between the two groups of animals.

Figure 1.

Behaviour and motor cortex cell firing during reaching. (a) Daily recording procedure. (b) Front view of reaching cage. (c) Performance on the single-pellet reaching task. (d) Example raster plots of six cells showing diverse firing patterns during reaches. Blue lines indicate start of reach (R) and grasp (G) times. (e) Six successive reach trials illustrating variability in sequential firing pattern of single units. Solid blue lines indicate trial boundary (start of reach), and dashed lines indicate grasp time. Trial duration is 1200 ms. (Online version in colour.)

Individual cells typically fired at similar times with respect to the start of the reach across trials, although the time of peak firing varied considerably over the population of simultaneously recorded units (figure 1d). When units were sorted according to the time of maximum firing, a sequential pattern of activation became clear during 1200 ms windows surrounding reaches (figure 1e). Although there was trial-to-trial variability in firing, the underlying sequential pattern was evident. We also compared ensembles for successful versus failed reaches and found them to be similar, with correlation coefficients in the range 0.4–0.8 (see electronic supplementary material, figure S4). Similarity between success and fail ensembles typically increased as training progressed (electronic supplementary material, figure S4). When the whole reach was divided into preparatory, grasp and return segments, the preparatory and return segments showed the largest increases in similarity between success and fail trials, whereas the grasp segment did not show as much of an increase.

Because of the sequential nature of unit firing during the task, we considered the possibility that the sequential pattern (figure 1e) would be replayed during rest, as previously reported for spatial learning tasks in both the cortex and hippocampus. Averaging unit activity across reach trials consistently yielded a clear template (figure 2a); however, we observed only modest matching of the template in either Rest 1 or 2 (figure 2a,b). The distribution of standardized match scores pooled from the rest periods of all animals lacked strong matches compared to task matches (figure 2c). Breaking down sleep into REM and SWS showed that match strength in REM and SWS was similar (figure 2c), and neither SWS nor REM showed a clear difference in match strength between Rest 1 and 2, suggesting that no sequentially organized reactivation was detected either in SWS or in REM sleep (figure 2c). We also considered the possibility that replay was temporally compressed with respect to the task; however, the examination of compression factors from 8× to 0.125× did not reveal evidence of faster or slower replay (figure 2d). Considering the clear sequential activity patterns during the task (figures 1e and 2a, left) and previous evidence of sequential replay in the hippocampus and neocortex, these negative results were rather surprising. However, they are consistent with a similar study in which they did not report sequential replay following reach training [20].

Figure 2.

Template matching of sequential unit firing during sleep. (a) Template of sequential firing during reaches and example matches from task, SWS and REM. (b) Distributions of template match scores pooled from all animals for 1 h of Rest 1 and 2 sleep and 30 min of reaching task. Strong matching is evident during the task, but not during Rest 1 or 2. (c) Group data for template matching (no temporal compression). Overall distributions of match strength during sleep are weak compared to task. REM and SWS match strength was similar and neither REM nor SWS was significantly different between Rest 1 and 2. (d) Template matching strength for faster (8×) or slower (0.12×) replay compression. The strongest matches occur when no compression is applied. Error bars are s.e.m. of n = 5 rats. (Online version in colour.)

(b). Principal component activation during sleep

Lacking strong evidence for sequential replay, we next considered the possibility that the coincident activation of ensemble activity during rest was more important for skill learning, as this has been reported recently [20]. Based on the established methods [21], principal component (PC) analysis was performed on a task activity matrix constructed from concatenated 1200 ms reach segments (as shown in figure 1e). The first PC, which showed consistently strong activation at reach times (figure 3a), was used to construct an activation time series for each rest epoch, which was then segmented into REM and SWS epochs for analysis.

Figure 3.

PCs of reaching activity are activated during REM and SWS. (a) Reach-triggered average of PC activation shows PC1 is related to reach behaviour. Shaded region is mean ± s.e.m. of randomly sampled time points in task. (b) Example time course of PC activation in Rest 1 and 2 against the background hippocampal LFP spectrogram. HNPC, hippocampus; LFP, local field potential. (c(i)) Group average (n = 5 rats) of activation strength from sleep from all training days. REM activation is stronger than SWS activation (t₄ = 3.5, p < 0.05). REM activation is stronger in Rest 1 (t₄ = 2.8, p < 0.05), and SWS activation is stronger in Rest 2 (t₄ = 2.8, p < 0.05). (c(ii)) Distributions of activation strength in individual REM and SWS epochs pooled from all animals and all training days (***p < 0.001, Kolmogorov–Smirnov test). (d(i)) Rats slept more in Rest 1 than Rest 2 (t₄ = 6.7, p < 0.01). (d(ii)) There was strong tendency for a greater allocation of REM sleep in Rest 1 (t₄ = 2.65, p = 0.056). (e) The net REM activation was significantly greater in Rest 1 (t₄ = 4.4, p < 0.05), whereas the net SWS activation was similar in Rest 1 and Rest 2. (Online version in colour.)

PC activation during sleep revealed a dynamic difference between REM and SWS. Similar to previous reports for declarative and procedural tasks [20,21], there was an increased activation strength in Rest 2 SWS (figure 3b). Additionally, we observed strong activation during REM sleep that was more apparent in Rest 1 (figure 3b). Calculating the average activation strength for REM and SWS for all training days revealed that REM sleep activation was significantly stronger than SWS activation (t₄ = 3.5, p < 0.05) (figure 3c(i)). Furthermore, the activation in REM was significantly stronger in Rest 1 compared to Rest 2 (t₄ = 2.8, p < 0.05), whereas SWS activation was significantly stronger in Rest 2 (t₄ = 2.8, p < 0.05) (figure 3c(i)). Previous results have shown that the distribution of PC activation strength in SWS is particularly long-tailed [21]. To compare distributions of REM and SWS activation strength, we pooled average activation scores from individual REM and SWS epochs for all days and all rats. Comparing the SWS distributions showed that the average increase in Rest 2 activation strength was driven mainly by an increase in the upper tail of the distribution (similar to previous reports of SWS activation [21]). Comparing the REM distributions showed that the stronger Rest 1 activation was owing to an overall increase in activation (figure 3c(ii)).

We also examined how changes in sleep architecture might impact the net amount of activation that occurred during the rest periods. The quality of sleep was better during Rest 1 compared to Rest 2; the rats spent more time sleeping (figure 3d(i); t₄ = 6.7, p < 0.01) and the sleep was less fragmented in Rest 1 compared to Rest 2 (t₄ = 6.7, p < 0.01, data not shown). Additionally, there was a strong tendency for a greater allocation of REM sleep in Rest 1 and SWS in Rest 2, although they did not reach significance (figure 3d(ii); t₄ = 2.65, p = 0.056 for both REM and SWS since total sleep is the sum of REM and SWS). The combination of more REM in Rest 1 with the stronger activation strength of REM in Rest 1 resulted in a greater net activation in REM for Rest 1 (figure 3e; t₄ = 4.4, p < 0.05). Although there was stronger SWS activation in Rest 2, the net SWS activation was similar in Rest 1 and Rest 2 (figure 3e; t₄ = 0.3, p = 0.80) because the stronger activation was offset by a reduced sleep time in Rest 2.

(c). Functional role of principal component activation during sleep

To gain a better understanding of the relationship between PC activation in sleep and learning, we examined when the strongest activation occurred during training. From the distributions of average activation scores for individual REM and SWS epochs, we selected those above the 95th percentile for each animal and named these epochs big-REM and big-SWS, respectively, and those below threshold as reg-REM and reg-SWS. The occurrence of big-REM and big-SWS was then plotted on the performance curve for each rat (figure 4a). In line with the observed activation strength in Rest 1 and Rest 2, big-REM tended to occur more frequently in Rest 1 (figure 4a, top row, red markers above the performance curve) and big-SWS tended to occur in Rest 2 (figure 4a, middle row, green markers below the performance curve). Furthermore, both big-REM and big-SWS showed a non-random allocation across training days, which was confirmed by significant χ² tests for all rats (figure 4a). For the two rapid learners (rats 1 and 2), there was a clear allocation of big-REM and big-SWS to days around the large gain in performance, whereas the allocation in gradual learners (rats 3–5) was more distributed across days. We next examined the co-occurrence of big-REM and big-SWS. Because big-REM occurred more in Rest 1 and big-SWS occurred more in Rest 2, we analysed the co-occurrence of big-REM in Rest 1 and big-SWS in Rest 2. We generated control distributions of the number of co-occurring Rest 1 big-REM and Rest 2 big-SWS by randomly assigning big-REM and big-SWS across training days. Days for which the actual number of co-occurring Rest 1 big-REM and Rest 2 big-SWS exceeded the expected number were indicated on the performance curve (figure 4a, bottom row, yellow marker). We found that the two rapid learners showed a clear allocation of co-occurring big-REM and big-SWS on the 2 days prior to their large performance gain. For the gradual learners, it was not easy to relate co-occurrence with performance improvement because these animals did not exhibit clear performance gain on a single day.

Figure 4.

PC activation during rest is related to rapid skill learning. (a) Distribution of big-REM and big-SWS epochs across training days. Red and green dot size indicates the percentage of big-REM and big-SWS, respectively; dots above the performance curve are Rest 1, below is Rest 2. χ² statistic indicates significant non-uniform distribution of big epochs across days and yellow dots indicate days for which the observed number of big epochs was greater than expected by chance. Bottom row indicates days with significant joint occurrence of big-REM and big-SWS; for rapid learners, significant joint activation of REM and SWS occurred just prior to large performance gains. (b) Correlation of PC activation in REM or SWS and daily change in performance in rapid and gradual learners. The amount of activation (either net or average) is plotted against the daily change in performance (Day (n) to Day (n + 1)). The net activation is the sum of activation and average is the net activation divided by time. Yellow background indicates significant Pearson correlation. (Online version in colour.)

To explore the relationship between sleep activation and learning further, we analysed the relationship between the daily change in performance (Day (n) to Day (n + 1)) and the amount of activation in REM or SWS during Rest 1 or Rest 2. When data from all rats were pooled together, no significant relationship was observed. When rapid and gradual learners were separated, however, significant correlations were observed in rapid learners, but not gradual learners (figure 4b). We compared two measures: net activation (sum of activation strength in either REM or SWS) and average activation (net activation divided by duration of REM or SWS). In rapid learners, there was a significant correlation between the net REM activation in Rest 1, but not the average REM activation, and performance change. Since the average measure factors out time, the correlation between net activation and performance change was likely owing to the observed changes in sleep architecture, i.e. the increase in overall sleep time in Rest 1 (figure 3d(i)) as well as the increased allocation of REM in Rest 1 (figure 3d(ii)). Rapid learners also exhibited a significant correlation between SWS activation in Rest 2 and performance change. Both net and average activation measures were significantly correlated with learning, indicating that the correlation was driven by the increase in activation strength in Rest 2 SWS compared to Rest 1. The SWS activation in Rest 2 also exhibited a time dependence relative to the task, whereas activation in REM or Rest 1 SWS did not. Activation in Rest 2 SWS was strongest immediately following the task and decreased significantly within the first hour following the task (electronic supplementary material, figure S5), which is consistent with the previously observed time course of memory reactivation in rat medial prefrontal cortex [22].

We found further evidence for a functional role of sleep activation by examining its relationship with electromyographic (EMG) activity. For each animal, we performed a cross-correlation of the activation signal with the rectified EMG signal for all days and then calculated the average cross-correlation for each animal. After calculating the average cross-correlation across rats, we observed significant relationships in both REM and SWS (figure 5). In Rest 1, REM activation was significantly correlated with EMG activity within a narrow window from −100 to 100 ms (compared to time-shuffled EMG; ANOVA F_1,4 = 12.1, p < 0.05). No significant correlation between REM activation and EMG activity was observed in Rest 2. In SWS, a reciprocal relationship was observed: no relationship was observed in Rest 1, but a significant relationship was observed in Rest 2 within the same time window of −100 to 100 ms (ANOVA F_1,4 = 7.8, p < 0.05). To verify the relationship between sleep activation and EMG, we performed two additional analyses. First, we calculated the triggered average of sleep activation at times of significant EMG activity and found that the pattern of triggered activation corroborated the cross-correlation analysis (electronic supplementary material, figure S6). Second, to rule out the possibility that the activation–EMG relationship was not a by-product of brief arousals, we calculated the triggered average of firing rate at times of significant EMG activity. Although firing rate did increase after EMG twitches, it was similar between Rest 1 and 2, whereas PC activation was not (electronic supplementary material, figure S7).

Figure 5.

Activation of task activity during sleep is related to muscle activity. Activation of PC1 during sleep is related to muscle activity. Significant cross-correlations between PC1 activation and EMG activity occurred in Rest 1 REM (ANOVA F_1,4 = 12.1, p < 0.05) and Rest 2 SWS (ANOVA F_1,4 = 7.8, p < 0.05; green is time-shuffled EMG, shaded areas are s.e.m. of n = 5). (Online version in colour.)

In summary, we found evidence of REM pre-activation and SWS reactivation coexisting in the motor cortex during skill learning. We also found evidence that strong REM pre-activation and strong SWS reactivation are functionally linked to skill improvement, as the co-occurrence of REM and SWS activation was temporally linked to performance improvements, specifically for the two rapid learners. Furthermore, both REM pre-activation and SWS reactivation were coordinated with EMG activity.

(d). Activation surrounding big-REM and big-SWS

To examine the relationship between REM and SWS activation on a finer time scale, we constructed triggered averages of the activation time course centred on the start of REM. We constructed separate time courses for big- and reg-REM to test whether the stronger REM activation was preceded by stronger SWS activation. In Rest 1, there was no difference between big- and reg-REM during the 2 min of SWS preceding the REM epoch (figure 6a). In Rest 2, however, there was evidence of a coordination between SWS and REM activation, as big-REM activation was preceded by significantly stronger activation in SWS (ANOVA F_1,4 = 13.2, p < 0.05). Because most REM epochs frequently end with waking, we also analysed the activation in the subsequent wake periods following REM. Interestingly, big-REM activation in Rest 1 persisted upon waking, whereas reg-REM activation returned to pre-REM levels upon waking. Although the waking-state activation following big-REM was somewhat variable, it was significantly stronger than the activation following reg-REM in the 2 min wake period (ANOVA F_1,4 = 10.4, p < 0.05). A similar persistence of big-REM into wake was not observed in Rest 2. One possible explanation for the apparent persistence of big-REM activation into wake is a difference in behaviour upon waking, i.e. greater movement upon waking. To test this possibility, we constructed a similar time course using the EMG signal (electronic supplementary material, figure S8), but there was no difference between EMG activity for the big- and reg-REM time courses. Therefore, a persistent activation into wake following big-REM was not induced by behaviour, at least not behaviours associated with neck-muscle EMG, but rather it indicates a tendency for neural activity in big-REM to be sustained during the subsequent wake.

Figure 6.

Strong activation in REM and SWS persists into wake. (a) Time course of activation strength for big-REM and reg-REM epochs that were preceded by SWS and followed by wake epochs. In Rest 1, activation was significantly stronger in the 2 min wake following big-REM compared to reg-REM. In Rest 2, activation was significantly stronger in the 2 min of SWS preceding big-REM epochs compared to reg-REM epochs (ANOVA F_1,12 = 13.2, p < 0.05). (b) Time course of activation strength for big-SWS and reg-SWS epochs that were preceded by and followed by wake epochs. In Rest 2, activation was significantly stronger in the 90 s of wake preceding (ANOVA F_1,4 = 8.4, p < 0.05) and following (ANOVA F_1,4 = 9.6, p < 0.05) the big-SWS epochs in Rest 1 as well as Rest 2, compared to reg-SWS. (Online version in colour.)

We also performed a similar analysis for big-SWS activation, examining the activation time course from wake to SWS and back to wake (figure 6b). In Rest 1, there was no difference between big- and reg-SWS activation time courses in either the wake preceding or the wake following the SWS epoch. However, in Rest 2, the big-SWS activation time course was significantly stronger than reg-SWS, for both the 90 s of wake preceding (ANOVA F_1,4 = 8.4, p < 0.05) and the 90 s of wake following (ANOVA F_1,4 = 9.6, p < 0.05) the SWS epoch. We also verified that these differences in activation levels were not a result of differential EMG activity (electronic supplementary material, figure S8).

In summary, we found evidence of a coordinated SWS and REM activation in Rest 2. We also found evidence of a coordinated sleep and wake activation: big-REM activation was extended into the subsequent wake in Rest 1 and big-SWS was preceded and followed by strongly activated wake in Rest 2.

(e). Activation in sleep spindles

While analysing the time course of REM activation, we noted that activation had already started rising approximately 30 s prior to the start of the REM epoch for both big- and reg-REM (end of SWS epoch in figure 6a). It has also been reported that PC activation during SWS is associated with spindle oscillations [20,21]. Given this evidence, we investigated PC activation within spindles, focusing on the spindle-rich pre-REM period. We confirmed that the average activation strength within spindles was significantly stronger compared to non-spindle SWS (figure 7a). Further analysis of the activation time course within spindle oscillations revealed a dynamic difference between Rest 1 and Rest 2. Compared to spindles in Rest 1, there was significantly stronger activation in the early part of spindles in Rest 2, suggesting that task activation occurs preferentially early in the spindle oscillation (figure 7b,c).

Figure 7.

PC activation in SWS occurs preferentially at the start of sleep spindles. (a) Activation strength is significantly stronger in spindles compared to non-spindle SWS (t₄ = 4.9, p < 0.01). (b) Time course of activation within spindles in Rest 1 and 2. For each spindle, activation was binned into 100 ms bins for the duration of the spindle. If the spindle duration was less than 1500 ms, unfilled bins were assigned NaNs (‘not-a-number’) and excluded from the average. Activation is normalized to 1000–1500 ms in Rest 1, and error bars are s.e.m. of five rats. (c) When averaged into 500 ms bins, Rest 2 activation was significantly stronger up to 1000 ms, but not thereafter (p < 0.05, Bonferroni-corrected). (d) Pre-REM spindle power increases prior to reg-REM epochs, but the increase prior to big-REM is significantly greater compared to reg-REM for Rest 1 (ANOVA F_1,4 = 19.8, p < 0.05) and Rest 2 (ANOVA F_1,4 = 17.0, p < 0.05). (e) Summary figure showing pre-REM spindle power and pre-REM activation strength. In Rest 1, activation does not increase with spindle power, but does increase with spindle power in Rest 2. (Online version in colour.)

We further analysed spindle power and activation strength in the spindle-rich pre-REM period to explore a possible difference between big-REM epochs and reg-REM epochs. An increase in LFP spindle power occurred prior to both big-REM and reg-REM epochs (figure 7d); however, the increase was significantly greater for big-REM epochs in both Rest 1 (ANOVA F_1,4 = 19.8, p < 0.05) and Rest 2 (ANOVA F_1,4 = 17.0, p < 0.05). Even though there was a difference in pre-REM spindle power prior to reg- and big-REM epochs in Rest 1 and Rest 2, activation strength in the pre-REM period was only stronger prior to big-REM in Rest 2. Figure 7e summarizes the activation strength data from figure 6b, and the spindle power from figure 7d to illustrate that pre-REM activation strength leading into big-REM is only greater than reg-REM in Rest 2, whereas spindle power leading into big-REM is greater than reg-REM in both Rest 1 and Rest 2.

3. Discussion

Many studies have demonstrated memory trace reactivation in SWS following training on a declarative memory tasks, but evidence has been lacking for memory reactivation during REM sleep or in the context of procedural tasks. We have demonstrated the activation of task-related ensembles during both REM and SWS in rats learning a motor skill task. Activation in SWS was strongest following task training in Rest 2 and was associated with spindles, whereas REM activation predominated in Rest 1 and occurred along with a change in sleep architecture towards more REM sleep. REM and SWS activation were functionally linked to skill learning, as Rest 1 REM and Rest 2 SWS activation were correlated with performance gains in the two rapid learning rats. Additionally, in all rats, there was a significant correlation with EMG activity for Rest 1 REM and Rest 2 SWS activation. The strongest activation during REM in Rest 1 and SWS in Rest 2 also had the intriguing property of persisting into waking following the sleep epoch.

(a). Rest 1 rapid eye movement activation

Our results demonstrated strong activation of reach-related activity during Rest 1 REM sleep. The lack of prior evidence for reactivation during REM sleep may in part be owing to limited sampling, as REM sleep comprises only approximately 20% of sleep and most reactivation studies record 30–60 min of sleep. Notably, one of the earliest reports of hippocampal memory replay provided evidence of REM replay using a template matching procedure [15]. Although it was termed ‘replay’, most significant template matches in that study [15] were detected in pre-task sleep. This observation suggests that the REM activation they observed is a form of pre-activation of task activity, similar to our results. Preplay of sequential place cell activity has been reported in the hippocampus and explained either as an allocation of pre-existing circuits [16,23] or a combination of pre-existing and novel circuits [24]. The concept of preplay has been questioned, however [25]. Comparisons between our results and those of Louie & Wilson [15] are limited by several differences: hippocampus versus motor cortex recording, maze running versus motor skill task, template matching versus PC activation. In addition, it is important to note the difference of time scales. Louie & Wilson [15] used templates based on a 1 s binsize and further smoothed them with a 1.5 s Gaussian kernel, whereas a 50 ms binsize resolution was used in our study. This represents a significant difference in time scale. Thus, although we both observed significant pre-activation during REM, whether or not the two effects reflect similar processes is questionable.

The strong Rest 1 REM activation was accompanied by an increase in REM sleep, caused by an increase in overall sleep time as well as increased allocation of sleep to REM. The combined effect led to an increase in total activation during REM that was correlated with the rate of learning on the reaching task for the two rapid learners. The change in sleep structure was similar to changes described in early studies of learning and REM sleep. In these studies, rats trained on a conditioned avoidance task exhibited a significant increase in SWS immediately following the training session, and then exhibited a transition to a more prolonged increase in REM sleep 4–6 h after training [26]. If the delayed increase in REM sleep was prevented by sleep deprivation, then learning was impaired [27]. A recent study using a task similar to ours did not report an increase in reach accuracy following a 2 h sleep period, although an improvement in reach kinematics and ‘neural speed’ (shift in peri-event time histogram (PETH) peak) was observed [20]. One possible explanation is that the motor cortex circuits had undergone refinement and induced kinematics and neural speed improvement, but that the 2 h rest period, which was SWS-rich, was too short to include enough REM to improve reach accuracy. It is possible that further sleep, which included more REM, would allow for activation in the newly refined circuits and result in a performance improvement.

(b). Slow-wave sleep activation

In agreement with a previous report [20], our results demonstrate strong activation of reach activity in post-task SWS. Previous studies have demonstrated that training on the reach task results in synaptic strengthening in the motor cortex, suggesting that motor cortex circuits are undergoing refinement following training, possibly during SWS [28–30]. Supporting this idea, recent imaging of dendritic spine dynamics during reach training has shown that new spines are formed during sleep following reach training [31]. Furthermore, REM sleep ultimately refines these newly formed spines, pruning some and stabilizing others [32]. Under these conditions of circuit refinement, it is not expected that a template of sequential activity based on previous behaviour would continue to match for very long in sleep. Indeed, this refinement possibly explains the decay in SWS activation we observed during the first hour of Rest 2, as well as the lack of sequential template matching that we and others [20] observed.

If the purpose of the circuit refinement is to improve reach accuracy, however, then the activity in REM, which mostly reflects the newly refined circuits, might be expected to pre-activate the upcoming and presumably more successful reaching behaviour. In summary, our data, together with previous behavioural and anatomical results, suggest that both SWS and REM activation are important for skill learning, although at different times relative to behaviour. Ideally, a continuous 24 h recording would provide a complete picture of the time course of SWS and REM activation.

The activation that extended into waking following a bout of strong REM activation in Rest 1 and strong SWS activation in Rest 2 is an interesting observation. Whether or not it is necessary for the consolidation process is unclear, and future experiments will need to test this by selectively disrupting/supressing motor cortex activity during this time.

(c). Functional implications of rest activation

We observed a significant correlation of sleep activation and muscle activity in Rest 1 REM and Rest 2 SWS, suggesting a direct link between sleep activation and muscle activity. Evidence from neonatal rats indicates that muscle twitches during sleep are not random events, but instead exhibit spatio-temporal structure that undergoes refinement [33]. Furthermore, these early twitches are coordinated with spindle oscillations, the earliest patterned activity in the neocortex, implying a functional relationship between muscle twitches and brain activity during development [34]. It is possible that a similar process continues in adulthood, serving to shape motor circuits in order to support new skills. Muscle twitching in REM sleep is well documented and evidence suggests that they are not random events or temporary lapses in REM sleep paralysis. Twitching in REM develops throughout the duration of the REM epoch, increasing in frequency such that longer REM epochs exhibit more twitches [35]. Our data support a continued cooperation between muscle activity and sleep activation in adults that facilitates motor skill learning.

We also observed a correlation between activation and daily performance improvements in the two rapid learning rats. The net activation in Rest 1 REM and both the net and average activation in Rest 2 SWS correlated with performance gains. This pattern of correlation was similar to the correlation of EMG with Rest 1 REM and Rest 2 SWS activation, providing further support that activation in both SWS and REM are important for skill learning. This hypothesis is further strengthened by the observation that most of the co-occurring big-REM and big-SWS took place just prior to the large performance gains for the two rapid learners. It is unclear why the correlation between activation and daily performance was not observed in the gradual learners, but one possibility is that the record of task performance is noisy [19,30,36,37], thus the smaller changes in performance in the gradual learners may have been affected by measurement noise. A more detailed behavioural analysis, as is often done in stroke models [38,39], may reveal a correlation with sleep activation.

(d). Slow-wave sleep and spindle activation

The importance of spindle oscillations in memory consolidation is supported by numerous studies, for both declarative [40–43] and procedural [44–46] tasks. In agreement with these findings, we observed an increase in spindle density following training. Associated with this increase in spindle density was an increase in activation strength. This is also in agreement with previous studies using PC analysis to measure memory activation, in both declarative [21] and procedural [20] tasks. We further identified the early part of the spindle oscillation as the critical window in which memory activation occurs. Since spindles are frequently associated with down- to up-state transitions [43], our results implicate activity directly following an up-state transition as important windows for memory consolidation.

We also identified pre-REM spindles as especially important for memory activation. Although this pre-REM period of enhanced spindle activity was identified decades ago [47–49], little is known about its functional significance. We have found that pre-REM spindles are enhanced prior to REM epochs that display particularly strong activation strength. Furthermore, this pre-REM period itself displays enhanced activation in Rest 2, raising the possibility that strong pre-REM activation and strong activation in the following REM epoch are coordinated and promote memory consolidation. Interestingly, many serotonin-specific reuptake inhibitors (SSRIs), commonly used to treat depression, suppress REM sleep to varying degrees [50–52], yet this suppression does not affect cognition or behaviour. It has even been reported that motor skill memory is improved following REM suppression with SSRI treatment [53]. Although most studies of the effects of antidepressants on REM sleep do not take into account pre-REM sleep, two have reported intriguing effects on the pre-REM sleep period. One study reported an increase in pre-REM sleep time that paralleled the decrease in the duration of REM sleep caused by SSRI treatment [52]. Another study that used a norepinephrine-specific reuptake inhibitor found a reduction in the amount of REM and pre-REM sleep and noted that the reduction in pre-REM was highly correlated with memory deficits [54]. Thus, the role of REM sleep in memory consolidation may come from an interaction between pre-REM and REM, rather than REM sleep per se.

In summary, we have demonstrated the activation of task-related ensembles during REM and SWS as rats were trained daily on a skilled reaching task. REM sleep activation was strongest in Rest 1 sleep and was accompanied by an increase in the amount of REM sleep. SWS activation was stronger in Rest 2, in particular during spindles and the spindle-rich pre-REM period. Both REM and SWS activation was correlated with performance improvements and EMG activity. Thus, our data support a coordinated role of REM and SWS activation during motor skill learning.

4. Methods

(a). Animals and surgery

Five adult male Fisher-Brown Norway rats (Rattus norvegicus) were used in these experiments. The rats were between four and nine months old at the time of surgery and weighed between 350 and 450 g. Animals were housed in a reverse light-cycle room (10.00 lights off, 22.00 lights on) and experiments were conducted during the dark cycle. Animals were allowed free access to food and water except during training on the reaching task, when they were food restricted to 85% body weight. All procedures were performed in accordance with the Canadian Council for Animal Care guidelines as well as University of Lethbridge guidelines.

After brief behavioural testing to determine paw preference on the single-pellet reaching task (described below), a hyperdrive containing an array of 12 tetrodes and 2 reference electrodes [55] was implanted at the surface of the forelimb region of the primary motor cortex contralateral to the preferred paw (coordinates: 1.0 mm anterior, 2.5 mm lateral to bregma; see electronic supplementary material, figure S1). Additionally, a twisted bipolar electrode (Teflon insulated stainless steel: 75 µm conductor) was implanted in the dorsal hippocampus to record the local field potential (LFP; coordinates: 3.8 mm posterior, 2.6 mm lateral, tip separation 0.6 mm). A pair of wires (Teflon insulated multi-strand stainless steel) was inserted into the neck muscle to record EMG activity. The entire assembly was cemented to the skull with dental acrylic and jeweller screws. Following surgery, the rats received 3 days of analgesic injections (Metacam) and 5 days of antibiotics (Baytril). They were allowed one week to recover before recording began.

(b). Recording and behavioural training

Recording was done with digital Cheetah SX data acquisition software (Neuralynx, Boseman, Montana). For spike recording, the signal was bandpass filtered (600–6000 Hz) and sampled at a rate of 32 kHz. One electrode in the drive was lowered to the white matter below the cortex and used as a reference for the tetrode recordings. LFP and EMG signals were bandpass filtered (0.1–1000 Hz), sampled at 2 kHz and referenced to a skull screw above the cerebellum. During the habituation period prior to training, tetrodes were slowly lowered over the course of one to two weeks to deep layers of the motor cortex. During the training period, small adjustments were made if necessary at the end of daily recording to maximize cell yield.

Prior to training, the rats were food restricted to 85% of their free-feeding weight. The recording procedure each day occurred during the animal's dark cycle in a dimly lit room. The procedure involved a 3 h pre-task rest epoch (Rest 1), a 30 min task epoch and another 3 h post-task rest epoch (Rest 2, figure 1a). During the rest epochs, the rat rested in a flower pot lined with towels, and the behaviour was recorded with an infrared security camera. During the task epoch, the rat was placed into a polycarbonate box designed for the single-pellet reaching task. The front of the box contained a 1.5 cm slot opening through which the rat reached to retrieve a sugar pellet (45 mg, Bio-Serv, Frenchtown, NJ, USA). The pellet was positioned in a well that was 1.5 cm from the opening and on a shelf that was 3 cm high (figure 1b). A high-speed infrared camera was used to record the task training epoch at 200 fps (Prosilica GigE). Additionally, an infrared beam across the front of the cage detected reaches, and this timing information was logged in the cheetah recording system. Before training on the reaching task began, there was a habituation period of one week. During habituation, the daily recording procedure was the same except that during the 30-min task period, the door to the shelf remained closed and the rat did not reach for pellets. During this time, the experimenter occasionally scattered pellets on the floor of the cage for the rat to eat. This habituation period ensured that the rat experienced all environmental factors except for reaching into the slot.

The rats were naive to the skill training except for the initial testing to determine paw preference. Skilled reach training typically consisted of 60 trials that were separated by 20–30 s and continued daily until asymptotic performance was achieved for three consecutive days (less than 5% increase in performance across 3 days; figure 1c). Between trials, a sliding door blocked access to the slot, and this allowed control over the time between trials and the total number of trials completed each day. The same experimenter did the training each day and recorded the performance as a percentage of successful reaches. A successful trial was one on which the rat grasped the pellet, retrieved it and ate it. All other trials were considered unsuccessful, including misses and attempts where the pellet was grasped but dropped during retrieval. The performance and reach timing were later scored more accurately using the high-speed video and this time was used for analysis.

At the end of the experiment, one electrode tip from each tetrode was marked by passing DC current through it (5 µA for 10 s). After Nissl staining, electrode tip locations were confirmed in deep layers of motor cortex (see electronic supplementary material, figure S1).

(c). Sleep classification and spindle detection

Classification of sleep into REM and SWS epochs was based on a combination of EMG and hippocampal LFP activity. The power of the EMG signal was thresholded visually to segment the recording into epochs of rest or movement. Within rest epochs, the classification of REM was based on a ‘REM’ signal calculated by dividing hippocampal theta power by hippocampal delta power and EMG power. Theta power was calculated by filtering the LFP (6–10 Hz), squaring the result and smoothing with a 1 s moving average. Delta power was calculated similarly except for filter frequency (1–5 Hz). EMG power was calculated by squaring the raw recording and smoothing with a 1 s moving average. The REM signal was calculated as theta/delta/EMG. For each rat, this signal was thresholded (greater than 3 s.d.) to obtain start/end timestamps for REM. The remaining rest period was classified as SWS or quiet wake based on the strength of upper delta power (2–6 Hz, greater than 2 s.d.). Automated sleep classification was confirmed by visual inspection and refined manually if necessary. Manual refinement was done by plotting REM and SWS epochs as coloured patches on top of the hippocampal LFP spectrogram. The experimenter could then click on a REM or SWS epoch to adjust the start or end time. Manual refinement was necessary mainly to resolve small overlaps at SWS–REM transitions. The same experimenter performed all sleep structure analysis.

Spindles were detected from the LFP of one of the tetrode channels. Since spindles in the rodent can be dissociated into two types, low-voltage spindle (LVS) and high-voltage spindle (HVS), we detected both forms of spindle, but then focused our analysis on LVSs because they are related to memory reactivation, whereas HVSs are not [43]. For LVS detection, the LFP was filtered between 10 and 20 Hz and then squared to obtain a power signal. Peaks in the power signal that were greater than 1.5 s.d. were expanded down to 0.75 s.d. to give a list of start and end timestamps. Gaps in timestamps less than 100 ms were merged together and a minimum duration of 200 ms was required. Spindle frequency was calculated as the mean of the peak-to-peak frequency within the oscillation. HVSs are characterized by a lower frequency range and higher amplitude compared to LVSs. They were calculated similarly to LVS except that the LFP was filtered between 6 and 10 Hz first and peaks in the power signal had to exceed 3 s.d. For K-complex detection, the LFP was filtered in the delta band (1–5 Hz) and then a time-shifted (35 ms) difference signal was calculated. The time-shifted signal emphasized large amplitude fluctuations on the time scale of the K-complex and peaks in this signal (mean + 3 s.d.) were considered detected K-complexes. Example detection of LFP events can be seen in electronic supplementary material, figure S3. The period just prior to REM sleep is characterized by a marked increase in LVS activity [56]. To quantify this pre-REM spindle power, the average value of the Hilbert transformed spindle LFP (10–20 Hz) was calculated in 10 s bins around the start of REM epochs.

(d). Spike sorting

Spikes were sorted using automated clustering (Klustakwik, K. Harris) followed by manual refinement (MClust). Several features were used to assess the quality of sorted units: the inter-spike interval histogram (less than 0.2% spikes in 2 ms refractory period), cross-correlation with other units (no peak in refractory period), waveform shape with low variance and consistent firing rates/patterns over the whole recording. Quality was further assessed by calculating the L-ratio [57]. The majority of units had an L-ratio less than 0.1 (99th percentile: 0.09) and units with L-ratios greater than 0.12 were excluded ([57]; electronic supplementary material, figure S4).

(e). Principal component activation

PC activation was calculated according to the method described by Peyrache et al. [21]. A task activity matrix was constructed from binned spikes (ncell × nbin, 50 ms bin size) by concatenating 1200 ms reach segments together (as in figure 1e). Each row of the activity matrix was z-scored by subtracting its mean and dividing by its standard deviation, and the PC analysis was performed. Determination of signal PCs was based on the theoretical distribution of eigenvalues (Marcenko–Pastur distribution) as described previously [21]. The first PC, which was consistently related to reaching (figure 3a), was used to construct a projector operator that was compared against activity in Rest 1 and 2. Only the first component was analysed because it was the only one consistently above signal threshold across all rats and days (the number of signal PCs and the variance captured by the first PC are shown in electronic supplementary material, figure S2). Each column vector of the z-scored rest activity matrix was left and right multiplied against the projector operator to form a time series of activation strength. This raw activation measure and a z-scored measure derived from a control distribution of activation time series were analysed. To generate the control distribution, the analysis was repeated on surrogate rest activity matrices (n = 500) constructed from Poisson spike trains based on the real firing rates of the individual cells.

The PC activation strength was quantified by taking the average value in each REM and SWS epoch. The individual epoch measures were then used to calculate the average REM or SWS activation strength in Rest 1 and 2. Additionally, for each animal, epochs from all days were pooled to form a distribution of REM and SWS activation strengths. From this distribution, the 95th percentile was used to identify strong REM and SWS activation epochs (termed big-REM and big-SWS).

(f). Template matching

Replay of sequential activity was analysed using methods described previously [4,22]. A task template was constructed by averaging the 1200 ms reach segments of the activity matrix (described above). Replay strength was calculated as the correlation coefficient of the template and an equally sized segment from the rest activity. A time series of replay strength was calculated by advancing the rest segment one bin at a time. The raw match score was z-scored by generating a distribution of match scores (n = 500) based on Poisson spikes similar to the PC activation score. Temporal compression of replay was assessed by varying the bin size of the rest activity from 6.25 to 400 ms, which corresponds to 8× (faster)–0.125× (slower) temporal compression.

Reported p-values are based on paired t-tests or repeated-measures ANOVA unless otherwise noted (*p < 0.05, **p < 0.01, ***p < 0.001).

Ethics

All procedures were performed in accordance with the Canadian Council for Animal Care guidelines as well as University of Lethbridge guidelines (approved protocol nos 1307 and 1709).

Data accessibility

Data and code are available by contacting the author or by download from doi:10.5061/dryad.7d7wm37rn.

Authors' contributions

M.J.E. designed and conducted experiments, analysed data and wrote the manuscript. B.L.M. contributed analysis and wrote the manuscript. M.T. designed the experiment, contributed analysis and wrote the manuscript.

Competing interests

We declare we have no competing interests.

Funding

Supported by DARPA (grant no. HR0011-18-2-0021; B.L.M.), Canadian Institutes of Health Research (grant no. PJT 156040; B.L.M.), NSF Award no. 1631465 (B.L.M.), Natural Sciences and Engineering Research Council Discovery Grant no. 06109 (M.T.).