Cognitive Replay of Visuomotor Learning at Sleep Onset: Temporal Dynamics and Relationship to Task Performance

Abstract

Study Objectives:

Studies of neural activity in animals and humans suggest that experiences are “replayed” in cortical and hippocampal networks during NREM sleep. Here, we examine whether memory reactivation in sleeping humans might also be evident within reports of concomitant subjective experience (i.e., dreaming).

Design:

Participants were trained on an engaging visuomotor learning task across a period of one or more days, and sleep onset mentation was collected at variable intervals using the “Nightcap” home-monitoring device. Verbal reports of sleep onset mentation were obtained either at the beginning of the night, or following 2 h of initial sleep.

Setting:

Data were collected in participants' home environments, via the Nightcap monitoring system, and at The Center for Sleep and Cognition, Beth Israel Deaconess Medical Center, Boston MA.

Participants:

43 healthy, medication-free college students (16 males, age 18-25 years).

Interventions:

N/A

Measurements and Results:

The learning task exerted a powerful, direct effect on verbal reports of mentation during light NREM sleep (stages 1 and 2). On post-training nights, a full 30% of all verbal reports were related to the task. The nature of this cognitive “replay” effect was altered with increasing durations of sleep, becoming more abstracted from the original experience as time into sleep increased.

Conclusions:

These observations are interpreted in light of memory consolidation theory, and demonstrate that introspective reports can provide a valuable window on cognitive processing in the sleeping brain.

Citation:

Wamsley EJ; Perry K; Djonlagic I; Babkes Reaven L; Stickgold R. Cognitive replay of visuomotor learning at sleep onset: temporal dynamics and relationship to task performance. SLEEP 2010;33(1):59-68.

A SUBSTANTIAL BODY OF EVIDENCE NOW DEMONSTRATES THAT RECENT EXPERIENCES ARE “REPLAYED” IN CORTICAL AND HIPPOCAMPAL NETWORKS during sleep.^1–4 Rodent studies utilizing single-cell recordings have clearly established that neuronal activation patterns seen during waking exploration are later re-expressed in sleeping animals.^1,2,4 Functional imaging studies in humans complement this work.^3,5 The observation that memories are reactivated during sleep suggests that “replay” of memory in the sleeping brain could be accompanied, at least in humans, by related subjective experience (i.e., dreaming or sleep mentation).^6–8 In fact, several key features of neural-level memory reactivation in sleep exhibit strong parallels with emerging cognitive data on the appearance of recently encoded memories in reports of sleep mentation:

1. Neural-level reactivation of recent experience has most often been observed during NREM sleep. Similarly, subjective reports elicited from NREM are more likely to contain episodic memory sources than reports from REM sleep.⁹

2. Within NREM sleep, the strength of neural-level memory replay decays quickly across time.^2,4 Similarly, sleep mentation may be most strongly related to recent memories early in the sleep phase.^10,11

3. Recent experience is not replayed veridically during sleep. Neuronal firing sequences established during wake are re-expressed only intermittently during rodent NREM sleep, with relatively low fidelity and on a faster time scale than the original experience.^12,13 Similarly, only intermittent fragments of recent episodic memories appear in sleep mentation, intermingled with remote and semantic material.¹⁴

We propose that reports of subjective experience may provide a valuable methodology for exploring the characteristics of memory traces recruited for reactivation during sleep. Though little empirical work has thus far addressed this question, some preliminary evidence is at least suggestive of a link between sleep mentation and recent learning. Smith and Hanke,¹⁵ for example, report evidence of sleep mentation related to a pre-sleep learning task, present to a greater degree in participants cued during sleep using a task-associated sound. Meanwhile, Fiss et al.¹⁶ report that dreaming symbolically of stories presented to participants prior to sleep is associated with enhanced morning recall, suggesting that task-related mentation could have a beneficial effect on memory performance.

Stickgold et al.¹¹ have previously demonstrated that an engaging, visually salient learning task (the popular video game Tetris®) exerts a direct influence on subsequent sleep onset mentation. The present study extends this work by asking how specific features of this cognitive-level memory reactivation change across time, and by examining task performance as a predictor of this replay effect. To approach these questions, we employed a highly engaging visuomotor video game (downhill skiing arcade game Alpine Racer™ II) as an intensive learning task, and observed the effects of this training on mentation during subsequent periods of light NREM sleep (stages 1 and 2).

It is well known that a general correspondence exists between waking life experience and dream content, in that predominant persons, concerns, and activities in an individual's waking life also are consistently evident in their his or her sleep mentation reports.^17,18 Yet historically, controlled laboratory studies have met with little success in experimentally manipulating sleep mentation through the use of specific pre-sleep interventions.¹⁹ This difficulty has likely arisen, in part, from reliance on passive visual stimulation as a pre-sleep stimulus, the effects of which are often overshadowed by the much more salient experience of spending the night in a sleep laboratory.²⁰ Unlike the passive stimuli used in many prior studies, the task we employed here actively engages participants in a multi-sensory learning experience, within an immersive 3D-style virtual environment. Furthermore, the confounds of the sleep laboratory environment were eliminated by recording sleep and mentation in the home.

Although dream research has most often focused on REM sleep, periods of early NREM sleep may actually provide the ideal opportunity to examine the effects of a learning task on subsequent mentation. Accumulating behavioral and neurophysiologic data suggest that consolidation of recent memory is supported by NREM-specific physiology, including sleep spindles^21–24 and hippocampal sharp wave ripples.^25,26 Furthermore, memories of recent episodes are proportionally greater in mentation from NREM sleep, and are particularly prevalent during the sleep onset period.⁹ Taken together with the observation that neural-level memory replay is strongest immediately following training, as noted above,¹ the available data suggest that cognitive-level replay of recent learning may be most apparent during periods of early NREM sleep. In the present study, we therefore focused on collecting mentation samples from light NREM sleep, shortly after sleep onset (stages 1 and 2).

Though considerable evidence supports the notion that memories are consolidated during sleep, we still have a relatively poor understanding of the processes controlling which recent experiences are “selected” for offline reactivation and consolidation. By studying the incorporation of recent waking experience into dreams, we can gain valuable insight into the processes controlling the activation of memory networks during sleep. As prior research suggests that task difficulty and/or initial skill level may be important mediators of sleep dependent memory processing,^27–29 here we asked whether Alpine Racer performance at baseline would predict the extent of cognitive-level task replay observed in subsequent sleep mentation. Furthermore, we examined how the incorporation of recent experience into sleep mentation is altered across time. Though the appearance of recent events in dreams has been reported to decline across the course of a night,¹⁰ the processes accounting for this effect remain obscure. Temporal proximity to waking experience provides a potential explanation, but it could also be that reactivation of memory on a cognitive level is incrementally transformed across time due to consolidation/processing occurring during intervening sleep. In the present experiment, we approached this question by independently manipulating the clock time of report collection and the position of mentation reports within the sleep phase.

METHODS

Participants (n = 43) played the downhill skiing game Alpine Racer II across a period of one or more days, during which time sleep onset mentation was collected using the Nightcap home-monitoring device. All participants were undergraduate students (16 males, age 18-25 years), by self-report free of mental disorders, sleep disorders, and medications known to affect sleep architecture. Participants did not have prior experience playing Alpine Racer. Twenty-seven of the 43 participants had at least some prior experience with real-life downhill skiing.

Alpine Racer(Figure 1) was chosen for the present research due to the visually salient and highly engaging nature of the game, as well as its strong interactive and kinesthetic components. During this simulated downhill skiing game, participants learn to control an on-screen “skier” through the movements of their legs. The game features 4 different modes of play, including 2 difficulty levels of downhill racing and 2 difficulty levels of “gate racing,” in which participants must accurately navigate through pairs of flagpoles (i.e., “gates”) on their way to the course end goal. During each game session, participants repeatedly cycled through these 4 modes of play in a fixed order (easy downhill racing, followed by difficult downhill racing, easy gate racing, and, finally, difficult gate racing). One “trial” of play is defined as one cycle through all 4 modes.

Figure 1.

In the arcade game Alpine Racer II, the player stands on a pair of “skis” and controls an onscreen character through the use of leg movements.

Performance Measures

Measures of Alpine Racer performance were calculated based exclusively on gate racing mode, which provided the most quantifiable and reliable performance data. Gate racing mode tracks both the total number of gates that the participant attempted during a downhill run, and the number of these gates which were successfully completed. For the purposes of this study, 2 primary measures of performance were used. (Due to technical difficulties, performance data were not recorded for 5 participants.)

1. Initial Performance Score quantified participants' baseline skill level at the beginning of training, as assessed during the first trial of the study. Initial Performance Score was calculated as the average of gates attempted and gates completed across both the “easy” and “difficult” courses, during the first trial of the study.

2. Improvement Score quantified the change in game performance across the entire duration of the study. These scores reflect the change in performance across three days of training in Experiment 1, but across only the single day of training in Experiment 2. Improvement Score was calculated as the difference between performance on the first trial of the study (Initial Performance,above) and performance on the last trial of the study. This difference score was then divided by the number of days across which the participant played Alpine Racer. Calculations were carried out separately for attempted gates and completed gates on the easy and difficult courses. Scores were z-transformed in order to facilitate cross-study comparisons. As with Initial Performance Scores, Improvement Scores were also averaged across attempted and completed gates, and across the easy and difficult courses.

Collection of Sleep Mentation

Participants slept in their home environment, where sleep was monitored using the Nightcap home sleep recorder, which reliably distinguishes wake, REM, and NREM sleep.³⁰ Sleep onset reports were collected as previously described.³³ During the first hour of sleep, the Nightcap automatically prompted participants to dictate detailed verbal reports of their mental experience following 0 (pre-sleep wakefulness), 15, 45, 75, 120, 180, or 300 sec of sleep. Pre-sleep reports were collected after 3, 6, and 9 minutes of continuous wake; Following sleep onset, the sleep duration at which reports were collected was pseudorandomly selected as either 15, 45, 75, 120, 180, or 300 sec. Report collection terminated 1 h following sleep onset, or after a maximum of 10 prompts had been delivered. Subjects were instructed that upon hearing a prompt, they were to report all mental activity experienced just prior to the prompt. A “mentation report” included all verbal material provided in response to a single prompt. Participants awoke and successfully dictated reports for 98% of computer prompts delivered, indicating high compliance with experimenter instructions. To enable expected cell-counts high enough to facilitate valid approximations of the χ² statistic, Nightcap conditions were collapsed into the following categories: ≤ 15 sec, 45-75 sec, and ≥ 120 sec. In addition to this nocturnal report collection procedure, participants also reported any dream content recalled immediately upon awakening each morning, and using a handheld recorder carried throughout the day, reported any daytime thoughts or imagery related to Alpine Racer.

Protocol Detail for Experiment 1

In the initial protocol, participants (n = 16) played Alpine Racer across 3 consecutive days. On the night prior to their first game session, baseline mentation data were collected according to the procedures outlined above. On the first day of game exposure, participants completed two 45-min game sessions between 08:00–13:00, and a third session between 15:00–20:00. Across the next 2 days, participants played one 45-min session between 08:00–13:00, and a 2nd session between 15:00–20:00. As described above, during each 45-min session, participants repeatedly played all 4 game play modes, in a fixed order.Participants completed a visual analogue scale describing their degree of engagement in the task for each day of Alpine Racer play. On each night of the study, mentation reports were collected as described above. In order to assess whether incorporation of the task into sleep mentation required active participation in the game, a small set of additional participants (n = 3), each observed another participant playing Alpine Racer, but did not themselves play the game.

Protocol Detail for Experiment 2

Participants in this follow-up study (n = 24) played Alpine Racer across a single day. Again, baseline mentation data were collected on the night prior to game exposure. Participants then played two 45-min sessions of Alpine Racer early the following day and a third session later that day, following the same procedures described above in Experiment 1. As in Experiment 1, all participants completed a visual analogue scale describing their level of engagement with the task. Mentation reports were collected on the subsequent night. Report collection was identical to Experiment 1, with the exception that the timing of reporting was altered. Sixteen of the 24 participants were randomly assigned to report sleep onset mentation on a Delayed schedule, in which participants slept undisturbed for the first 2 hours of the night, and were then awakened to begin the sleep onset report collection procedure following these hours of intervening sleep (Figure 2). Comparison of reports from these 16 participants with those collected during Experiment 1 (where participants followed an Early Immediate schedule) provided a manipulation of the placement of report collection within the sleep phase, while holding the duration of prior wakefulness constant (Figure 2). An additional group of participants (n = 8) were assigned to report mentation according to a Late Immediate schedule. These participants went to sleep 2 h later than their normal bedtime, after which report collection took place during the initial sleep onset period. Comparison of these Late Immediate participants with participants assigned to the Delayed schedule provided a manipulation of the placement of report collection within the sleep phase, while holding the time of night constant (Figure 2).

Figure 2.

Timing of Early Immediate, Late Immediate, and Delayed reporting schedules, presuming a habitual bedtime of midnight.

Protocol for Experiment 3

To confirm the reliability of Nightcap-assisted sleep scoring, sleep data were collected from an additional group of participants in the General Clinical Research Center at Beth Israel Deaconess Medical Center (n = 23,10 males), using simultaneous polysomnography (PSG) and Nightcap recording. To ensure similar conditions, these participants also played three 45-min sessions of Alpine Racer during the day prior to PSG recording, following the same procedures described above. PSG data were collected using a Medcare Embla N7000 digital acquisition system, employing a standard EEG (C3-A2, C4-A1, O1-A2, O2-A1), EMG, and EOG montage. Participants were awoken for reports using the Nightcap following either an Early Immediate (as in Experiment 1) or Delayed (as in Experiment 2) awakening schedule. We hypothesized that sleep stage distributions would be similar across the Immediate and Delayed awakening schedules.

We expected that sleep stage distributions would be similar in the Immediate and the Delayed awakening schedules. Firstly, the EEG power spectrum in NREM sleep, particularly slow wave activity, does not vary as a function of circadian time.^31,32 Thus, the circadian phase in which sleep was initiated would not be expected to influence the latency to stage 2 and 3 NREM in the present study. Secondly, although the frequency content and distribution of NREM stages is modulated by the duration of prior wakefulness,³³ this varied by only about 10% in the present study.

Scoring of Mentation Reports

Reports from Experiments 1 and 2,along with a set of 200 Controlreports, were transcribed and then scored by 3 raters, blind to report set, awakening condition, and performance data. Control reports were randomly selected from a prior study of Nightcap-collected sleep onset mentation,³⁴ which employed identical report-collection procedures as the present study, but in which no learning task was administered prior to sleep. Raters were extensively trained on the scoring system prior to use. Each report was first classified as either containing some description of a mental experience, or as containing no content at all (i.e. “I can't remember,” “Umm, there was nothing,” etc.). The proportion of prompts that yielded mental content was similar on baseline and post-training nights (80.3% vs. 79.6%; P > 0.8), as was mean report length (78.6 vs. 76.4 words; P > 0.8). Reports containing content were then scored for imagery or thoughts related to Alpine Racer. Game-related images and thoughts were classified as being either direct (unambiguous representations of the game specifically, or of skiing in general) or indirect (containing sensations, persons, objects, locations, or themes related to Alpine Racer). Game-related imagery was further scored as (a) visual and/or (b) kinesthetic, or (c) of “indeterminate/other” modality. Reports were also scored for the presence of content related to the experimental setting.

Reports were initially scored by 2 raters, blind to report set, awakening condition, and performance data. Mean percentage agreement across scoring categories was 89% ( ± 4% SD) for the 2 raters. Disagreements were resolved through scoring by a third blind rater. Two-thirds agreement was reached for all scores, with the exception of residual 3-way disagreements for 0.5% of imagery scores and 0.2% of thought scores. In these cases, reports were assigned scores of “indeterminate.” Otherwise, final scores were assigned based on 2/2 or 2/3 agreement.

Statistical Analyses

On post-training nights, Alpine Racer-related content was prevalent during periods of pre-sleep wakefulness as well as during Nightcap-identified sleep. In fact, no sub-category of task-related content differed significantly between Nightcap-identified wakefulness and sleep (P > 0.2 for all tests). This homogeneity may result from imprecision inherent in the Nightcap monitoring system. The Nightcap provides an excellent overall approximation of polysomnographically defined sleep stage, but fine-grained category distinctions are not strongly reliable. Many “Wake” Nightcap reports, for example, are taken from early stage 1 sleep (Figure 6). Furthermore, there is reason to suppose that the neurobiological processes of interest in the present study are common across all phases of the sleep onset period, including awakenings from sleep, and from quiescent pre-sleep wakefulness. These include a) the activation of recently altered memory networks in the sleeping brain and b) the expression of recent memories within the content of conscious mentation. For example, in animal models, the neural-level reactivation of recent memories initially observed during sleep has also been reported to occur during periods of quiescent, resting wakefulness.^35,36 In addition, the contents of mentation during quiet waking, elicited under conditions of reduced sensory input, are surprisingly similar to reports of dream mentation collected from sleep.^37,38 For all of these reasons, in the following analyses, Nightcap Wake mentation reports are grouped together with those elicited from Nightcap-identified sleep, except where otherwise noted.

Figure 6.

Polysomnographically defined sleep stage was similar across Early Immediate and Delayed awakening schedules

As recruitment and procedures were largely identical across Experiments 1 and 2, data were combined wherever possible, in order to increase the available sample size. Unless otherwise indicated, the following analyses always include the combined data from Experiments 1 and 2. Each experiment is treated separately only when necessary to test specific hypotheses, and this is noted in the text.

RESULTS

Task-Related Mentation on Post-Learning Nights

Alpine Racer exerted a profoundly strong influence on sleep onset mentation. The impact of game play was most evident on the first post-training night of Experiment 1, on which 47% of all reports with content (35% of all reports) were game-related. In contrast, only 1 of 78 morning dream reports collected across Experiments 1 and 2 were related to the task. Representation of the game within sleep onset mentation primarily took the form of sensory imagery, as opposed to thought (91 vs. 23 reports), and most often bore a direct, unambiguous relationship to the game (90 vs. 24 reports; see Table 1 for example reports, and Tables 2–3 for incidence of task-related reporting). On post-learning nights across Experiments 1 and 2, 114 of 386 content-filled reports (30%) contained either thoughts (23 reports, 6%) or imagery (91 reports, 24%) related to Alpine Racer. The majority of game-related imagery (75%) was classified as directly related to Alpine Racer, and this imagery was most often experienced in the visual modality, although kinesthetic game imagery was present in a third of these reports (Table 2).

Table 1.

Examples of Alpine Racer-related mentation

	Related Imagery	Related Thought
Examples of Direct Incorporation	“I get like flashes of that… game in my head, virtual reality skiing game… downhill umm race, in my head. Umm, there's this one particular corner that I haven't quite been able to master, and every time I get flashes of it, it's like that corner that umm I keep crashing into in my head.”	“I was just thinking about how tonight's my last night doing this experiment umm, and how I did pretty well in today's ahh afternoon session, last, my last session with the Alpine Racer game.”
	“I once again, saw the, the game, it was smooth at first, and then it went into the cave, and then it just stopped like abruptly, like the game turned off”	“I'm thinking about the game, the game that I should have used my knees more, like bent my legs, and that would have been better.”
Examples of Indirect Incorporation	“I do have this weird sensation – it's almost like when you've been on a small boat for a while and you step onto the dock, you have the rocking sensation – well, I feel like I'm on the, the Alpine Racer at times, even when I'm lying in bed, just the backward and forward motion”	“I was thinking about, um I was about to say a downhill banana (laughs), but uh I don't know exactly what that means.”
	“I was in a race. Um, like a running race, um… sort of like through San Francisco. It was kind of hilly and it wasn't difficult. Like, I was just kind of coasting through the race I think.”	“I was just falling asleep and thinking about the Olympics and some of the events that went on there.”

Table 2.

Task-related mentation on post-learning nights in experiments 1 and 2

	number of reports	% of reports	% of reports with content
Reports Containing Related Imagery	91	19%	24%
Type of Imagery
Directly-Related	68	75% a
Indirectly-Related	23	25% a
Modality of Imagery
Visual	57	63% a
Kinesthetic	34	37% a
Indeterminate/Other	23	25% a
Reports Containing Related Thoughts	23	5%	6%
Type of Thought
Directly-Related	22	96% b
Indirectly-Related	1	4% b

Notes. Across Experiments 1 and 2, participants reported at least some content from 620 (79.4%) of 781 total sleep onset reports. In cases where mentation was reported, reports were an average of 83.1 (± 51.1 SD) words long. On post-learning nights, 386 (78.9%) of 489 reports contained at least some content. These data do not include daytime reports, morning reports, and reports from n = 3 control “observers” (see methods).

^a% of all reports containing related imagery; modality is not a mutually exclusive category

^b% of all reports containing related thoughts

Table 3.

Percent of mentation reports related to Alpine Racer across report sets (all nights)

	Control Report Set	Baseline Nights	Experimental Nights
Task-Related Imagery
Direct Incorporation	0% (0%)	0.4% (0.3%)	17.6% (13.9%)
Indirect Incorporation	1.3% (1%)	3.8% (3.1%)	6.0% (4.7%)
Task-Related Thought
Direct Incorporation	1.3% (1%)	2.6% (2.1%)	5.7% (4.5%)
Indirect Incorporation	0% (0%)	0% (0%)	0.3% (0.2%)

Notes. Alpine Racer play primarily affected sleep onset imagery, as opposed to thought, and this imagery was most often directly related to the task. Data are first presented as the % of content-filled reports with task-related mentation, followed by the % of all reports elicited which contained related mentation, in parentheses (). Control reports are from a previous, unrelated study of sleep onset mentation (see methods).

Task-Related Imagery on Baseline vs Post-Learning Nights

Raters judged only 2% of all Control sleep onset reports (taken from an unrelated study, see methods) as related to Alpine Racer, establishing that the scoring methods employed here yielded a very low rate of false-positive identification. In contrast, 6% of baseline night reports across Experiments 1 and 2 were scored as having incorporated thought or imagery related to the game, an incidence of task-related content significantly greater than that seen in the Control report set (x²_{1, N = 386} = 5.01, P = 0.02, Cramer's V = 0.11). Alpine Racer-related content during baseline data collection thus appears to reflect genuine anticipation of future game play, as participants had been told that they would be playing a skiing game the following day. Interestingly, in contrast to post-training reports, game-related imagery on baseline nights tended to be indirectly, rather than directly, related to Alpine Racer(x² = 9.25, P = 0.002, Cramer's V = 0.38).

Alpine Racer play appeared to primarily affect sleep onset imagery, rather than thought. Table 3 presents the incidence of task-related content across baseline, post-training, and control report sets. Participants were substantially more likely to report game-related imagery following task exposure, as compared to during baseline data collection. On baseline nights, 22% of participants provided at least one report with related imagery, increasing to 65% on the firstnight following Alpine Racerexposure (McNemar χ²_{N = 37}, P = 0.001). In contrast, the number of persons reporting task-related thoughts did not increase significantly from baseline (P > 0.2). On a per-report basis, the percentage of content-filled reports with related imagery increased by 19% (from 4.2% at baseline to 23.6% on the first experimental night), while task-related thought increased only 3% from baseline to experimental nights (from 2.6% to 6.0%). Conversely, the proportion of participants' reports related to the general experimental context decreased substantially from baseline to experimental nights (from 31% to 18% of content-filled reports; F_1,35 = 9.05, P = 0.005, n_p² = 0.2), perhaps reflecting habituation to the recording/reporting procedures.

Game-related Imagery Becomes Increasingly Abstracted Across Longer Durations of Sleep

The form which Alpine Racer-related imagery took (directly vs. indirectly related to the game) was strongly dependent on Nightcap latency (x²_{2, N = 60} = 10.27, P = 0.006, Cramer's V = 0.29; Figure 3). While the probability that directly-related imagery would be present in a report decreased with increasing durations of sleep (x²_{2, N = 275} = 6.78, P = 0.03, Cramer's V = 0.11; Figure 3a), indirectly related imagery exhibited a near-significant increase at longer sleep durations (x²_{2, N = 275} = 4.87, P = 0.08, Cramer's V = 0.09; Figure 3b). At Nightcap latencies ≤ 15 sec, a full 25% of content-filled reports contained imagery unambiguously representing either the game itself, or skiing experiences in general (see Table 1). After 120 sec of sleep, however, this directly related game imagery was reduced by more than half (Figure 3a). At the same time, longer bouts of sleep were increasingly associated with imagery more loosely associated with the game, including sensations, objects, locations, and activities which were similar (but not identical) to those seen in Alpine Racer(Figure 3b). When collapsing across direct and indirect incorporation, the presence of game-related content did not vary as a function of Nightcap latency (P > 0.6, N = 275, χ² test of independence).

Figure 3.

The form which Alpine Racer-related imagery took (directly vs. indirectly related to the game) was strongly dependent on Nightcap latency (x²_{2, N = 60} = 10.27, P = 0.006, Cramer's V = 0.29). While incidence of imagery directly related to Alpine Racer decreased with increasing time since sleep onset, indirectly-related imagery conversely increased with greater sleep durations.

An identical pattern of results was observed when “wake” condition reports were excluded from the ≤ 15 sec Nightcap latency category. Again, the form of Alpine Racer imagery (directly vs. indirectly related) was strongly dependent on Nightcap time (x²₂ = 9.01, P = 0.01), with directly-related imagery decreasing across time (x²₂ = 6.45, P = 0.04), as indirectly-related imagery increased, now significantly (x²₂ = 7.55, P = ).02). Finally, when disregarding ≤ 15 sec reports altogether, > 120 sec reports still contained relatively greater amounts of indirect vs. direct imagery than 45-75 sec reports (x²₂ = 3.65, P = 0.05). These findings, therefore, do not appear to depend on the inclusion of wake or very early stage 1 reports.

Timing of Report Collection

Immediate vs Delayed Awakening Schedules

Imagery directly related to Alpine Racer was reduced when awakenings were scheduled later in the night. In contrast, indirectly related imagery remained relatively constant across awakening schedules (2 [Image Type] × 3 [Awakening Schedule] ANOVA interaction: F_1.36 = 4.67, P = 0.02, n_p² = 0.21; Figure 4). Relative to Early Immediate awakenings, imagery directly related to the game was reduced in both the Delayed (P < 0.05) and Late Immediate conditions (P = 0.001; Fisher's LSD tests; Figure 4). In contrast, there was no difference in direct Alpine Racer imagery between the Late Immediate and Delayed schedules. Thus, directly related imagery declined substantially across time, but contrary to our hypotheses, was not significantly altered by intervening sleep when time of night was held constant.

Figure 4.

Delayed vs. Immediateawakening schedules. Imagery directly-related to Alpine Racer decreased substantially across time, independent of the presence of sleep, while declining non-significantly across sleep when time of night was held constant. The dependent variable was proportion of participants' content-filled mentation reports that contained at least one Alpine Racer-related image. Prior to analysis, data were log-transformed (ln(x+1)) in order to normalize distributions.

Changes in Mentation Within and Across Experimental Nights

During post-training nights, Alpine Racer-related content decreased across successive awakenings. Reports containing content related to Alpine Racer tended to occur during the earlier awakenings of the night (mean ordinal position of awakening (i.e., 1st, 2nd, 3rd…) for reports containing Alpine Racer content = 4.9 ± 3.0 SD, as compared to 5.8 ± 3.1) for reports unrelated to Alpine Racer; t₂₅₆ = 2.31, P = 0.02, d = 0.31). In Experiment 1, the mean number of Alpine Racer-related reports per subject declined linearly across the 3 nights of the study (M = 2.2, 1.8, and 1.3 task-related reports, across the 3 nights of the study; F_2,30 = 5.71, P = 0.03, n_p² = 0.28). There was, however, a simultaneous drop in the proportion of awakenings from which participants recalled any content, and hence the proportion of content-filledreports with task related mentation did not change significantly across nights (P > 0.1).

Game Performance and Task-Related Mentation

Across Experiments 1 and 2, Initial Performance Score predicted the report of task-related mentation during subsequent daytime wakefulness (reports dictated during the day, see Methods; r₃₅ = 0.41, P = 0.02; data from Experiment 1 and 2 combined). In addition, among participants who played Alpine Racer across 3 consecutive days (Experiment 1 only), there was a trend for those with superior Initial Performance Scores to report more game-related sleep onset mentation (number of task-related reports from all 3 nights combined; r₁₆ = 0.41, P = 0.11).

The present studies were designed primarily to examine the impact of learning on subsequent sleep mentation, rather than the effect of task-related mentation on next-day improvement. However, exploratory analyses did not reveal a significant relationship between the presence of task-related mentation reports and Improvement Score (P > 0.2; data from Experiments 1 and 2 combined). Note that in Experiment 2, no re-test was administered following the experimental night—thus, a relationship between sleep mentation and improvement might not be expected to occur in this study. However, in Experiment 1, where change in performance was tracked across 3 days of play, task-related mentation was also unrelated to Improvement Score (P > 0.3). Nor did task-related mentation predict improvement across any individual night in Experiment 1 (i.e., improvement from the last trial of Day 1 to the first trial of Day 2).

Task Engagement and Prior Skiing Experience

The degree to which participants were engaged with the game and/or participants' prior experience with downhill skiing could be important mediators of the subsequent appearance of task-related mentation. However, participants' emotional engagement VAS ratings did not significantly correlate with any measure of task performance or task-related mentation. Similarly, whether or not subjects had prior downhill skiing experience was unrelated to the reporting of task-related mentation.

Control Observers

Interestingly, in Experiment 1, the small group of control “observers” (n = 3), who merely watched others play Alpine Racer, reported game-related sleep onset mentation at rates similar to participants who actively played the game. 19% of observers' experimental night reports contained game-related imagery, compared to 24% for players. Thus, the physical sensation of actual game play did not appear to be necessary for substantial task-related sleep onset mentation to occur.

Correspondence Between Nightcap Awakenings and PSG Data

In the sleep laboratory, 359 Nightcap-initiated awakenings were performed with simultaneous polysomnographic recording. Nightcap condition was a strong predictor of polysomnographically-defined sleep stage (x²_{10, N = 359} = 80.40, P < 0.001; Figure 5). Furthermore, as anticipated, sleep stages were similar across the Immediate and Delayed awakening conditions (P > 0.4; Figure 6).

Figure 5.

Polysomnographically-defined sleep stage for ≤ 15 sec Nightcap Reports (A), 45-75 sec Nightcap reports (B), and > 75 sec Nightcap reports (C).

DISCUSSION

Accumulating evidence demonstrates that recent experiences are “reactivated” in the sleeping brains of animals and humans. Yet little empirical work has explored the possibility that memory “replay” during sleep might be observed not only in measures of neural activity, but in reports of subjective experience as well. As reported previously for the video game Tetris,¹¹ Alpine Racerdirectly influenced subjective experiences during sleep, establishing that such intensive pre-sleep learning experiences are cognitively replayed during early periods of NREM sleep. However, the strength of this effect, in which 24% of post-training reports contained task-related imagery, far exceeds that observed in previous research. Historically, consistent and direct incorporation of experimental stimuli into dream reports has been exceedingly rare,¹⁹ and even in the aforementioned Tetris sleep onset study,¹¹ only 7% of post-learning reports contained task-related imagery. In cases where blind raters and appropriate controls have been employed, statistically significant effects of pre-sleep manipulation have typically been seen within measures of qualitative dream features (i.e., emotion, vividness), ^39,40–42 rather than in the frequency of directly or indirectly related dream content.

Alpine Racer challenges participants with an intense, interactive, multi-sensory learning experience played out in an engaging 3D environment. We speculate that these task features produced the exceptionally powerful effects that we observed. Task incorporation, however, did not require sensations of physical movement or perceptual/motor learning during active game play. Control “observers” in Experiment 1 incorporated Alpine Racer into their mentation at rates similar to those who actively engaged in the game, suggesting that the motor-activity aspect of the task was not critical for cognitive-level reactivation to occur. Instead, it appears that the novel, engaging experience of viewing others progress through the game—which in itself could be described as a substantive learning experience—was sufficient to induce subsequent task-related sleep mentation.

Although participants were not explicitly informed of the goals of the research, the possible influence of demand characteristics must be considered. There are several reasons, however, that a major impact of such characteristics is unlikely. First, rather than giving the obvious “replay” report of standing on the virtual skis and playing the game, participants consistently described stereotyped abstractions of the game, never describing the skis, the game controls, or the arcade machine. In this respect, the reports are strikingly similar to those collected in the earlier Tetrisstudy,¹¹ in which both controls and densely amnesiac patients (for whom demand characteristics could not have been at work, as they had no memory of the game) reported this same, stereotyped, abstracted imagery. Finally, although the present estimate of incorporation prevalence could be inflated due to demand characteristics, it is unlikely that this could explain the more nuanced patterns of results regarding report timing and task performance, discussed below.

Although Alpine Racer-related content increased substantially following training, task-related themes were present even prior to game exposure. As game-related baseline reports were more prevalent than in a matched set of Control reports, it seems that this content reflects a genuine anticipation of playing the game in the future, rather than a mere artifact of false positive identification. A growing body of literature in the cognitive neurosciences suggests than an important function of human memory and cognition is to think prospectively about potential future events, in preparation for flexible response to novel situations.^43,44 Alpine Racer-related mentation observed prior to task exposure could represent such offline prospective memory processing, or “future projection.”^43,44 Thus, the sleeping brain may not only reiterate past experience, but also may activate information related to impending future events.

Imagery directly related to the game quickly dropped off with increasing sleep durations, across both shorter and longer time scales. On a time scale of minutes, increasing sleep time was associated with progressively less veridical and more abstract representations, increasingly drawing on the incorporation of related remote and semantic memory material. Participants therefore initially reported straightforward cognitive-level replay of the event (e.g., “I get like flashes of that… game in my head, virtual reality skiing game…”), and later reported imagery which was only loosely connected to the game:

“I was picturing stacking wood this time… I felt like I was doing it at… at a ski resort that I had been to before, like five years ago maybe.”

Although a decline in directly related game imagery could result from a simple time-dependent decay process, known shifts in systems-level activation patterns during the transition into stage 2 NREM sleep also provide a compelling explanation for our observations. Sleep is thought to transform memory traces in part by slowly “interleaving” neural representations of recent experience into existing remote and semantic cortical networks.^6,45 Here, we observed that representation of Alpine Racer was not restricted to the isolated, veridical incorporation of game images, but instead consisted of fragments of the Alpine Racer experience “interleaved”⁴⁵ with other material. The increasing abstraction of game-related imagery seen across time may reflect a shift in the relative engagement of hippocampal and cortical memory systems, as participants enter further into sleep, progressing from wakefulness through stage 2. Indeed, recent functional imaging data demonstrate that patterns of brain activity are dramatically altered across the first minutes of sleep, even within classically defined stage 1 NREM sleep, with large and rapid increases in hippocampal activity, together with decreases in several cortical regions seen shortly before the transition from stage 1 to stage 2 NREM sleep.⁴⁶ Stickgold et al. have previously demonstrated that the hippocampus is not required for the near-veridical cognitive replay of memories from recent events during the first minutes of sleep. However, as the hippocampus comes online later in the sleep phase,⁴⁶ indirectlyrelated imagery may be facilitated, as the sleeping brain begins to slowly integrate recent memories into cortical networks.

Directly related game imagery also decreased across longer time scales, declining with each successive awakening during the first hour of the experimental night, and being attenuated in the both the Delayed, and Late Immediate awakening conditions. These findings suggest that the mere passage of time, even in the absence of any intervening sleep, can lead to a decline in directly related game imagery across these longer time scales.

Finally, task performance at baseline predicted the extent to which cognitive-level task replay would later emerge. This was most clearly evident in the relationship between baseline performance and game-related cognition during daytime wakefulness. In Experiment 1, there was also a non-significant trend for baseline performance to predict task-related mentation at sleep onset. These observations suggest that features of task acquisition may determine the extent of later offline memory reactivation. Indeed, several previous studies have shown that baseline performance and/or task difficulty modulates sleep-dependent memory processing.^27–29

CONCLUSIONS

Here we report strong cognitive evidence that recently-modified memory networks are “reactivated” during the sleep onset period. Such spontaneous offline activity is thought to be critical for the stabilization, consolidation, and reorganization of memory traces.^45,47 The nature of this replay effect changes rapidly across time, consistent with data from rodent studies showing that the strength of neural-level memory reactivation is quick to decay.^1,2 Although the present data do not provide direct evidence of a function for task-related sleep mentation, these time-dependent shifts in the content of mental experiences during sleep could represent the progressive integration of recent experiences into pre-existing related cortical networks.⁴⁵

DISCLOSURE STATEMENT

This was not an industry supported study. Dr. Stickgold has received research support and consulted for Merck, Takeda, Actelion, and Sepracor; has consulted for Jazz and Epix; and is on the advisory board of Merck. The other authors have indicated no financial conflicts of interest.