Sleep and the Extraction of Hidden Regularities: A Systematic Review and the Importance of Temporal Rules

Summary

As part of its role in memory consolidation, sleep has been repeatedly identified as critical for the extraction of regularities from wake experiences. However, many null results have been published as well, with no clear consensus emerging regarding the conditions that yield this sleep effect. Here, we systematically review the role of sleep in the extraction of hidden regularities, specifically those involving associative relations embedded in newly learned information. We found that the specific behavioral task used in a study had far more impact on whether a sleep effect was discovered than either the category of the cognitive processes targeted, or the particular experimental design employed. One emerging pattern, however, was that the explicit detection of hidden rules is more likely to happen when the rules are of a temporal nature (i.e., event A at time t predicts a later event B) than when they are non-temporal. We discuss this temporal rule sensitivity in reference to the compressed memory replay occurring in the hippocampus during slow-wave-sleep, and compare this effect to what happens when the extraction of regularities depends on prior knowledge and relies on structures other than the hippocampus.

Introduction

The last two decades have seen the accumulation of substantial amount of evidence suggesting sleep facilitates the consolidation of memory in humans [1]. Beginning with demonstrations of improved abilities in simple declarative and procedural tasks (e.g., [2]), data soon began to support the notion that sleep also facilitates higher cognitive functions, including rule learning, language acquisition, gist extraction, problem solving, spatial navigation, insight and creativity [3–10]. Two sleep stages, Slow-Wave-Sleep (SWS) and Rapid-Eye-Movement (REM) sleep, were often noted for their involvement in these processes, with findings commonly linking a specific sleep stage with facilitation of specific cognitive faculties [1].

One of the high-level cognitive functions that sleep was reported to improve is the incidental extraction of hidden regularities within newly encoded stimuli. In these studies, participants are asked to perform a simple task, which can be easily accomplished by following a given set of instructions; however, unknown to participants, the stimuli in the task embed some hidden regularities that, if discovered (either implicitly or explicitly), can lead to a marked improvement in performance. Sleep, and particularly SWS, was shown in several key studies to facilitate such incidental discovery more than simple time passed in wake [4,5,9–14]. The ability to extract novel regularities is believed to originate in the hippocampus and form the initial basis of generalization [15]. This is of particular importance because unlike some other high cognitive functions that sleep was shown to facilitate, generalization of this sort is considered to be well characterized by contemporary theoretical models of sleep and memory.

Two main mechanisms have been proposed. First, based on evidence showing that hippocampal memories are reactivated during SWS in coordination with cortical activity (otherwise known as “memory replay”; [16,17]), the “Active System Consolidation” approach suggests that memory reactivation supports the transformation of hippocampally-dependent episodic memories into cortically-dependent semantic ones [18]. Through this process, regularities embedded within the encoded memories are slowly extracted, avoiding catastrophic interference, and then distributed within existing knowledge structures for long-term memory storage [18,19]. Second, based on data suggesting that SWS leads to a net reduction in synaptic strength within the hippocampus and cortex, sleep may act to maintain stable levels of synaptic strength (known as synaptic homeostasis) by reducing and even eliminating excessive connectivity created during wake. Such homeostasis has the potential to improve signal to noise ratio and maintain the important common aspects of memories while reducing the salience of their less relevant idiosyncratic features, thus creating generalized representations of the individual experiences (e.g., [20,21]). These two basic mechanisms may also act in concert. As suggested by the ‘information overlap to abstract’ (iOtA) model [22], replay-induced strengthening of related memories followed by non-specific downscaling may lead to a net strengthening of only the overlapping features of those memories while eliminating the rest, thus supporting abstraction of gist information and eventually the formation of cognitive schemata.

Nevertheless, whether or not the behavioral tasks used in those human studies actually tap the higher cognitive functions they presume to support is not often considered. Instead, when a new study finds evidence that sleep facilitates performance in a specific behavioral task, the cognitive processes assumed to underlie performance in this task are often given a catchy label (e.g., gist extraction; insight; relational learning; cognitive schemata), leading to the misperception that sleep is doing exactly that – inspiring insight, extracting gist – rather than having a specific kind of influence in a specific task that does not necessarily represent the general process attached to it. In place of this rather sensational approach, it might be more useful to disentangle the particular types of influence that sleep exerts by carefully deconstructing the tasks that it impacts upon (for a related approach, see [23]).

In this systematic review, we take a deeper look at human studies that investigated the effects of sleep on the extraction of hidden regularities. We focus on tasks in which the regularities depend on associative relations between newly learned stimuli rather than those relying on previous knowledge or on motor/perceptual skills. Such tasks are often considered to involve the hippocampus and directly relate to the two sleep-dependent generalization mechanisms described above (though note that the hippocampus likely contributes to other tasks as well; e.g., [24]). To that end, we compare the main findings according to the cognitive function studied, the behavioral task, the nature of the hidden regularities that characterized the task, the experimental approach taken, and the measurements used to test performance. In particular, based on our findings, we will make the case that: a) There are no general experimental designs that are inherently more likely to produce sleep effects than others, but the specific behavioral task chosen has a strong impact on the outcome; b) when considering the effects of sleep on explicit (rather than implicit) extraction of hidden regularities, there is a fundamental distinction between regularities based on a temporal pattern and those with a non-temporal pattern, with sleep almost exclusively facilitating detection of the former ones. We conclude by embedding those findings within current experimental and modeling approaches, and compare them with results coming from sleep studies employing tasks that are less hippocampally-oriented. We also suggest that at least in some cases, “high-level” cognitive processes that sleep is claimed to facilitate can, in fact, be explained by local memory-strengthening mechanisms.

Methods

Identification of articles was carried out in four stages. We first assembled an initial list of 22 papers based on our familiarity of the literature, all of which were well-cited studies from the last two decades that were relevant for the topic of this review (see list in Supplementary Materials). We then identified all relevant studies cited in those papers, and added them to the list. The same step was repeated with the extended list, again and again, until no more new relevant articles could be found. Next, we searched PubMed and Google Scholar for review papers on the topic. From these review papers, several new relevant articles were identified and added to the list. On the third stage, we searched both PubMed and Google Scholar for relevant papers with the keywords “Sleep”, “Nap”, “Consolidation” or “Offline” in their title + keywords containing task names such as “Serial Reaction Time Task” or “Transitive Inference” in the body text, or the terms “abstraction”, “gist”, “rule extraction”, “generalization” or “regularities” in the abstract, and added any new relevant articles to the list. Finally, we conducted a similar search using the same keywords in the title + the names of each of the authors appearing in any of the papers in the existing list.

Studies were included for review if they were human experiments (of any age group) examining the behavioral or physiological effects of sleep on performance in tasks requiring the identification of hidden regularities, specifically regularities that depend on associative relations within newly learned stimuli. Exclusion criteria were: a) examination of memory recall alone, without any measurement of regularity extraction; b) the use of unhidden regularities (i.e., participants knew they should be looking for patterns at any point before sleep); c) dependence of the regularities on previous knowledge (e.g., semantic knowledge) or heavily relying on motor or perceptual skills; d) highly-complex tasks requiring a variety of skills that, while possibly containing an element of regularity extraction, this element is undistinguishable from the other skills through the performance measure used (e.g, playing computer games, [25]; solving the Iowa Gambling Task, [26]; navigation ability in Virtual Maze Tasks, [7]). Please see Supplementary Materials for an additional exclusion criterion regarding studies using the Serial Reaction Time Task (SRTT).

In studies that contained more than one experiment (or condition), some of which adhering to our criteria above and some that don’t, only those passing the criteria were addressed. The only exception was in cases where an association between sleep physiology and behavior was found with conditions/experiments other than the ones we focused on, in which case we still noted the association (e.g., a correlation of simple memory recall with SWS or REM sleep despite a lack of generalization effects). Furthermore, if a study included several relevant conditions but only some of which yielded a sleep effect, we considered the study as showing an effect and report it as such.

To define whether a reported effect should be considered explicit or implicit, we used the distinctions appearing in the papers themselves. Specifically, explicit measurements were considered as such if they required subjects to state the learned rule explicitly, generate new examples of that rule, or if they produced an effect on reaction time (RT) so dramatic that it can only be explained by rule learning (relevant only to the Number Reduction paradigm; see later). Other measurements, such as standard incremental effects on RT, accuracy measures in forced choice categorization, or familiarity testing were all considered implicit (cf. [27]). We note three exceptions to this rule: Two papers used explicit measurements that were reminiscent of a generation test, though not perfectly fitting to its definition [28,29]. We still considered them as explicit, but noted their unique status (see Table 1). One additional paper, which only measured performance implicitly, attempted to apply indirect methods to “extract” hints for explicit knowledge out of those implicit measurements (i.e., strategy analysis of accuracy responses; [30]); we did not consider those true explicit measurements.

Table 1:

Summary of studies reviewed in the systematic analysis.

Learning Type	Task	Rule Type	Study	N	Subjects	Design	Sleep Time	Implicit Effect	Explicit effect	Sleep correlates
Insight	NRT	T	Wagner et al., 2004 [9]	106	Adults	Sleep vs. Wake	Overnight	No	Yes	-
	NRT	T	Yordanova et al., 2008 [31]; 2009a [90], 2009b [91], 2010 [92], 2012 [12]	55	Adults	SWS-rich vs. REM-rich	Overnight	No	Yes	SWS
	NRT	T	Debarnot et al., 2017 [93]	74	Adults, Elderly	Sleep vs. Wake Young vs. Old	Overnight	-	Yes (young) No (elderly)	-
	Virtual Map Insight	T	Lerner & Gluck, 2018 [32]	20	Adults	Single Sleep Group	Multiple Nights	Yes	No	SWS
	Surveillance	T	Lerner et al., 2019 [33]	24	Adults	Cue vs. sham	Overnight	Yes	Indirect a	nREM
Visuo-Motor Sequence Learning	SRTT	T	Maquet et al., 2000 [37]; Peigneux et al., 2003 [94]	31	Adults	Sleep vs. Wake b Structured vs. Random	Overnight	Yes	No	REM
	SRTT	T	Robertson et al., 2004 [34] (implicit instructions condition)	36	Adults	Sleep vs. Wake	Overnight	No	No	-
	SRTT	T	Robertson et al., 2005 [95]	44	Adults	Sleep vs. Wake	Overnight	Yes (some conditions) c	No	-
	SRTT	T	Spencer et al., 2006 [76], (implicit instructions condition)	50	Adults	Sleep vs. Wake	Overnight	Yes (some conditions) d	No	-
	SRTT	T	Spencer et al., 2007 [96], (implicit instructions condition)	16	Elderly	Sleep vs. Wake	Overnight	No	No	-
	SRTT	T	Ertelt et al., 2012 [97]	37	Adults	Sleep vs. Wake	Overnight	Yes	No	-
	SRTT	T	Pace-Schott & Spencer, 2013 [98]	112	Adults, Elderly	Sleep vs. Wake	Overnight	Yes (young) No (elderly)	No	-
	SRTT	T	Wilhelm et al., 2013 [39]	161	Children, Adults	Sleep vs. Wake	Overnight	-	Yes	SWS
	SRTT	T	Song et al., 2014 [99] (unintentional group)	40	Adults	Sleep vs. Wake	Overnight	No	No	-
	SRTT	T	Cousins et al., 2014 [100]	32	Adults	Cueing before vs. during sleep	Overnight	Yes	Yes	SWS, Spindles
	SRTT	T	Kirov et al., 2015 [101]	53	Adults	Single sleep group	Overnight	-	Yes	nREM-REM tr.
	SRTT	T	Diekelmann et al., 2016 [40]	36	Men, Women	Cue vs. sham	Overnight	No	Yes (men) No (women)	SWS
	SRTT	T	Cousins et al., 2016 [64]	22	Adults	Single sleep group, cued vs. uncued target	Overnight	Yes	Indirect e	SWS, REM
	SRTT	T	Zinke et al., 2017 [102]	25	Children	Single sleep group	Overnight	No	Yes	Spindles
	SRTT	T	Yordanova et al., 2017 [103]	53	Adults	Single sleep group	Overnight	No	Yes	Spindles
	SRTT	T	Lutz et al., 2018 [104]	117	Adults	Sleep vs. Wake	Overnight	Yes	Yes f	-
	Ocular SRTT	T	Albouy et al., 2006 [105]	49	Adults	Sleep vs. Wake	Overnight	Yes	No	-
	ASRT	T	Song et al., 2007 [36]	36	Adults	Sleep vs. Wake	Overnight	No	No	-
	ASRT	T	Nemeth et al., 2010 [41]	25	Adults, Elderly	Sleep vs. Wake	Overnight	No	No	-
	ASRT	T	Hallgato et al., 2013 [106]	102	Adults	Sleep vs. Wake	Overnight	No	No	-
	ASRT	T	Csabi et al., 2014 [107]	34	Adults	OSA vs. control	Overnight	No	No	-
	High-order SRTT	T	Fischer et al., 2006 [11]	20	Adults	Sleep vs. Wake	Overnight	No	Yes	-
	High-order SRTT	T	Fischer et al., 2007 [35]	25	Children, Adults	Sleep vs. Wake	Overnight	No (children) Yes (adults)	-	-
	High-order SRTT	T	Drosopoulos et al., 2011 [81]	40	Adults	Sleep vs. SD	Overnight	-	Yes	-
Statistical Learning	Tone Learning	T	Durrant et al., 2011 [14]	36	Adults	Sleep vs. Wake	Overnight	Yes	-	SWS
	Tone Learning	T	Durrant et al., 2012 [49], 2016 [50]	48	Adults	Sleep vs. Wake	Nap	Yes	-	SWS
	Tone Learning	T	Hennies et al., 2017 [108]	42	Adults	Cue at sleep vs. wake	Overnight	Yes	-	SWS
	WPT	S	Djonlagic et al., 2009 [28]	99	Adults	Sleep vs. Wake	Overnight	Yes (some conditions)	Indirect (some conditions) g	REM
	WPT	S	Barsky et al., 2015 [47]	51	Adults	Sleep vs. Wake	Nap	Yes	-	REM
	WPT	S	Kemeny & Lukacs, 2016 [109]	44	Adults	Sleep vs. Wake	Overnight	No	-
	WPT	S	Lerner et al., 2016 [48] (experiment 2)	20	Adults	Single sleep group	Multiple Nights	Yes	-	SWS, N1/N2
Artificial Grammar Learning (AGL)	AXB words	T	Gomez et al., 2006 [5]	48	Infants	Sleep vs. Wake	Nap	Yes	-	-
	AXB words	T	Hupbach et al., 2009 [61]	48	Infants	Sleep vs. Wake	Nap	Yes	-	-
	AXB words	T	Frost & Monaghan et al., 2017 [62]	72	Adults	Sleep vs. Wake	Overnight	Yes	-	-
	Syllable regularity	S h	Gaskell et al., 2014 [45]	38	Adults	Single cued sleep group	Nap	Yes	No	SWS
	“Reber”-like Grammar 1	T	Nieuwnhuis et al., 2013 [110]	81	Adults	Sleep vs. Wake	Overnight	Yes	-	-
	“Reber”-like Grammar 2	T	Kemeny & Lukacs, 2016 [109]	45	Adults	Sleep vs. Wake	Overnight	No	-	-
	“Reber”-like Grammar 3	S g	Batterink & Paller, 2017 [44]	44	Adults	Single cued sleep group	Nap	Yes	No	SWS
	Segmentation	T	Simon et al., 2017 [46]	37	Infants	Sleep vs. Wake	Nap	Yes	-	SWS
Relational Memory	Transitive Inference	S	Ellenbogen et al., 2007 [4]	56	Adults	Sleep vs. Wake	Overnight	Yes	No	-
	Transitive Inference	S	Werchen & Gomez, 2013 [42]	64	Adults	Sleep vs. Wake	Overnight	Yes	No	-
	Associative Inference	S	Lau et al., 2010 [13]	31	Adults	Sleep vs. Wake	Nap	Yes	-	SWS
	Associative Inference	S	Alger & Payne, 2016 [43]	58	Adults	Sleep vs. Wake	Nap	Yes	-	REM
Information-Integration	Frequency-Orientation Associations	S	Maddox et al., 2009 [30]	49	Adults	Sleep vs. SD	Overnight	Yes	-	-
	Visual-Audio Associations	S	Hennies et al., 2014 [29]	52	Adults	Sleep vs. Wake	Overnight	No	No i	-
	Tone-Density Associations	S	Ashton et al., 2018 [51]	95	Adults	Sleep vs. Wake	Overnight	Yes j	No	-
Generalization of Categorical Learning	Prototypes (AB task)	S	Maddox et al, 2011 [58]	18	Adults	Sleep vs. SD	Overnight	No	-	-
	Objects	S	Werchan & Gomez, 2014 [60]	27	Infants	Sleep vs. Wake	Nap	No	-	-
	Dot Pattern	S	Graveline & Wamsley, 2017 [56]	73	Adults	Sleep vs. Wake	Overnight	Yes	-	-
	Satellites	S	Schapiro et al., 2017 [59] (novel items condition)	193	Adults	Sleep vs. Wake	Overnight, Nap	-	No	REM k
	Abstract images	S	Lutz et al., 2017 [57]	28	Adults	Sleep vs. Wake	Overnight	No l	-	REM k
	Movies	T	Sandoval et al., 2017 [55]	79	Children	Sleep vs. Wake	Nap	Yes	-	-
	Novel objects	S	Friedrich et al., 2015 [52]	90	Infants	Sleep vs. Wake	Nap	Yes	-	Spindles
	Novel objects	S	Friedrich et al., 2017 [53]	107	Infants	Sleep vs. Wake	Nap	Yes	-	Spindles, N2
	Novel objects	S	Friedrich et al., 2019 [54]	30	Infants	Single sleep group	Nap	Yes	-	Spindles

Note: T and S in the ‘Rule type’ column refers to Temporal and Stationary regularities, respectively; ASRT, Alternating Serial Response Time task; nREM, non-REM; NRT, Number Reduction Task; OSA, Obstructive sleep apnea; REM, Rapid Eye Movement; SD, Sleep Deprivation; SRTT, Serial Reaction Time Task; SWS, Slow Wave Sleep; tr, transitions; WPT, Weather Prediction Task;

^aCueing during sleep was correlated to implicit measures, which themselves were strongly influenced by explicit rule detection;

^bWake and sleep groups were compared in order to identify brain regions showing higher activation during REM sleep; then, those regions were used to detect correlations with performance improvement for a sleep group that trained on structured sequences compared to a group that trained on random sequences;

^csleep was shown to prevent the obstruction of regularities extraction caused by Transcranial Magnetic Stimulation (TMS) delivered following training; without TMS, however, no sleep effect was found;

^dsleep effect found only in the ‘contextual’ condition where additional hidden hints regarding the regularity were present;

^eno effects of cueing on explicit behavioral measures, but existing correlations between explicit measures and brain measures;

^fExplicit effects found for confidence ratings of a generation task (‘triplet completion’), though not for generation accuracy;

^gexplicit rule knowledge not examined, but sleep was shown to facilitate explicit evaluation of simple stimulus-response probabilities when conditions prevented pre-sleep training performance to approach ceiling levels;

^hstimuli were words presented visually that were also read aloud; thus, the hidden rule could be discovered based on extraction of either stationary or temporal regularities;

ⁱexplicit measurements were based on metrics from three additional tests. Although none of these tests directly required to state the hidden two-dimensional rule, one was reminiscent of a generation test (i.e., requiring to generate new examples of the rule) as defined in Methods;

^jno effects immediately following sleep, but sleep affected the ability to relearn the stimuli a week later;

^kREM sleep was associated to performance in memory tests but not in generalization tests (which were absent);

^lEffects were not found immediately following sleep; some sleep effects were found a year later, but at that point the generalization items were not totally novel.

Results

The common experimental design

Studies explored sleep effects by training subjects on a particular task and then testing their performance – and sometimes their explicit knowledge of the task regularities – after an interval that included sleep or, in a control group, only wake. Usually, the interval was either 12 hours long (in overnight studies) or a few hours (in nap studies). Many studies also examined the correlations between the behavioral performance in the sleep group and several recorded sleep variables that are known to index memory consolidation, such as time spent in SWS or REM, slow-wave or spindle power in the sleep EEG, and others. Another line of studies employed targeted-memory-reactivation (TMR) techniques, in which auditory (or olfactory) stimulation was associated to relevant task stimuli during wake and then presented again during sleep (under the assumption it reactivates the associated memories and strengthen the sleep effect). The outcome of such cueing on a subsequent test performance was then measured, often as a function of the sleep stage the cueing was administered in (usually SWS vs. REM).

Overview of the main findings in each class of tasks

The main findings of each study are reviewed below according to the type of task used, and summarized in Table 1 (note that in cases where several papers used the same data, they were grouped together in the Table). Effects are reported only if their significance level was 0.05 or better. Statistical trends were not considered.

Number Reduction Task (NRT) and other insight learning paradigms

Perhaps the most cited paradigm used to suggest that sleep inspires insight, in the Number Reduction Task subject perform computations on a series of digit pairs in succession. For each pair (comprised of the digits 1, 4, and 9), they need to produce a third digit based on a simple pre-taught rule. In each trial, eight digits are presented, and subjects are required to go over them serially by first applying the rule to the first two digits; then applying the rule to their response together with the third digit; then to their new response and the fourth digit, and so on. Subjects thus produce a total of seven digits one after the other throughout each trial by continually employing the rule, with the final digit considered the ultimate answer for that trial. Subjects are told, however, that if they happen to realize what the last digit will be before having gone through all seven computations, they can respond with that answer immediately and end the trial early. Indeed, unrevealed to the subjects, there is a hidden rule that governs the required responses and which, if detected, allows the subjects to predict the last digit prematurely: The inputs are organized such that, for a given trial, the last three required responses always mirror the preceding three responses (e.g., 4, 9, 4, 1, 1, 4, 9). If subjects recognize this regularity, they can predict the final answer for the trial as soon as they compute the second response and thus considerably reduce their RT for that trial. Studying the effects of sleep on performance, it is regularly found that sleep dramatically increases the probability of subjects explicitly discovering the hidden rule (evident by both a large decrease in RTs and in stating the relation between the 2^nd and 7^th response in a follow-up questionnaire [9,31]), with some studies linking the effect specifically to SWS [12,31]. Implicit effects of sleep (i.e., gradual reduction of RTs to each of the three predictable responses before the insight occurs), in contrast, are rarely found. Recently, two studies from our lab examined the effect of sleep on insight using different paradigms. In one of our studies [32], subjects navigated through a virtual winding corridor and were then required to choose one of five marked doors to exit. Untold to participants, the marking of the correct door corresponded to the shape of the corridor just traversed (from a bird’s eye view). Realizing this consistency negated the need to locate the door by trial and error. It was found that the average time spent in SWS over multiple nights negatively correlated with the number of errors, demonstrating a facilitatory effect of sleep on implicit rule learning. Sleep, however, was not correlated with explicit rule learning. Conversely, in our second study [33] subjects were required to surveil targets passing by in a 3D immersive environment and respond to each by clicking a button. Unbeknownst to subjects, the order of appearance abided a hidden regularity reminiscent of the NRT rule. It was found that TMR administration during non-REM sleep facilitated reaction time for the predictable targets and this facilitation corresponded with explicit rule discovery.

Serial Reaction Time Task (SRTT)

By far the most studied paradigm used to examine the effects of sleep on pattern extraction, subjects in the SRTT are exposed to a series of successive cues, appearing in one of four (or, sometimes, six) possible locations. They are asked to respond to each cue, as fast and as accurate as possible, by pressing one of four (six) corresponding buttons (or, in an ocular version, by looking at the cue location). A hidden regularity governs the location of the cues, with different studies using a different type of regularity. In many cases, the location of the cues followed a fixed and repetitive sequence in some blocks, and random in the rest (e.g., [34]). In other studies, every two successive locations probabilistically predicted the location of the next one (“high-order SRTT”; e.g., [11,35]). And in yet other studies, a fixed sequence was integrated within random locations (“Alternating Serial Reaction Time” – ASRT; e.g., [36]). Subjects who recognize the regularity can potentially predict where the next cue will appear, thereby reducing their RTs considerably – and sometimes even generate or explicitly state the hidden sequence. Effects of sleep on performance were generally found to be dependent on the type of task: Using a fixed sequence, implicit effects of RT reduction are often apparent and sometimes show an association to REM sleep (e.g., [37]), though some studies suggest that a passive passage of time also contributes to these results (e.g., [34,38]). Explicit effects using a fixed sequence are sometimes found as well, in which case a relation to SWS, or non-REM sleep in general, is often found (e.g., [39,40]). Using the high-order SRTT, both explicit and implicit effects are regularly present (e.g., [11]); and using the ASRT, neither implicit nor explicit effects are found (e.g., [36,41]).

Relational Learning

Relational learning refers to the ability to infer indirect associations between stimuli after learning their direct relations to each other. In a highly cited study, Ellenbogen and colleagues ([4]; and see replication by [42]) used the transitive inference task to examine how sleep affects this ability. On each trial, subjects were presented with a pair of abstract images and asked to choose between them, after which they received feedback for their choice. Through trial and error, subjects needed to discover which image in each pair should be preferred over the other. The images were chosen from 6 stimuli with a hidden rule governing the preferences hierarchy: A>B>C>D>E>F. Only adjacent pairs were presented during training (e.g., AB, BC, CD,..), in random order. At test, subjects needed to once again choose, without further feedback, the preferred stimuli from the learned pairs, but also from unlearned “inference” pairs (e.g., BD, CE, BE) for which the correct answer follows the same hierarchy rule (e.g., B > E). Results showed that sleep facilitates performance for the inference pairs in an implicit way (i.e., more correct answers than in the wake condition), but does not benefit explicit recognition of the hidden rule ([4, 42], feedback condition). In another study of relational learning using a different task, Lau and colleagues [13] trained subjects to associate between sets of images, A and B, as well as between another set, B and C. Then, the strength of the associations was tested with the originally learned pairs as well as with novel pairs of the form A-C. It was found that an afternoon nap facilitates both the direct (A-B, B-C) and indirect (A-C) associations, with the facilitation of indirect associations correlating to the amount of SWS during the nap. These results were later replicated in a study that used both neutral and emotional images, though effects were found to correlate with REM sleep rather than SWS [43].

Artificial Grammar Learning (AGL)

A set of studies investigated the detection of grammatical rules in an artificial language. In these tasks, subjects are exposed to sentences made of gibberish words or syllables. A hidden grammatical rule governs these sentences and restricts the order of words such that not all possible combinations are allowed (e.g., 3-word sentences in which the first word always determines the identity of the third word). In some studies, often employed when investigating learning in infants, the rules are simple and deterministic; in other studies, using adults, more complex, probablistic rules are applied. During training, subjects are presented with several example sentences adhering to the hidden rule, with the words usually delivered auditorily, one by one (though sometimes visual delivery with simultaneous presentation of the words was preferred, or the delivery included both visual and auditory presentation). During testing, subjects’ ability to recognize new sentences adhering to the same rule is examined. Sleep, in most cases, was shown to increase recognition of new rule-based sentences [5], with some studies also showing the effect is related to SWS [44–46]. Explicit awareness of the grammatical rule could not be examined in infant studies; however, with adult subjects, two studies failed to find such sleep-dependent explicit effects [44,45]. Interestingly, those two studies included visual presentation of the stimuli, with or without concurrent auditory delivery.

Statistical Learning

Statistical learning refers to the ability to detect regularities that are repeated probabilistically rather than deterministically. One paradigm employing statistical learning is the Weather Prediction Task (WPT), in which subjects are presented with abstract images and asked to learn, by trial and error, whether they predict Sun or Rain. Various combinations of 1,2 or 3 images (out of possible 4) are displayed on different trials, with a complex and probabilistic relation linking each combination to the correct answer. Subjects can improve performance above chance even if not fully realizing the complex rule, by developing simple strategies that take under consideration only some of the images. Using the WPT, Djonlagic and colleagues ([28]; replicated later by [47]) found sleep enhances performance in the task as long as ceiling effects are avoided during training, with effects correlated to REM sleep. In addition, under the same conditions, sleep was shown to enhance an indirect measure of explicit knowledge of the complex rule, namely, the ability to estimate the probabilities by which single images predict specific responses. Conversely, when examining performance improvement in the task over a week, a relation between performance and trait-like baseline levels of SWS was found [48]. Another, quite different paradigm that was used to explore sleep and statistical learning had subjects exposed to a series of structured tones in which each tone (or 2 consecutive tones) predicted the next one in the sequence in a probablistic manner. Sleep was found to enhance recognition of new instances of the structured tones compared to random ones in a subsequent test, with effects correlated to SWS [14,49,50]. Finally, note that two other tasks, the probablistic versions of the SRTT and AGL mentioned earlier, could also be classified as belonging to statistical learning.

Information-Integration

In this paradigm, subjects are exposed to a set stimuli differing on two dimensions, both of which could be visual (e.g., a grating pattern differing on orientation and frequency), or one auditory and one visual (e.g., location of an image and an accompanying tone). The stimuli are differentiated to two groups based on a linear decision bound in the 2D stimuli space, such that information from both dimensions need be taken under consideration simultaneously for optimal performance. Three studies examined how sleep following training affects the categorization of new exemplars in this task, with mixed results. One study [30] found sleep enhances categorization performance. A second study [51] found sleep did not enhance performance immediately, but enhanced the effects of retraining on the same rule following sleep. An explicit test of rule knowledge (using a generation task) showed no sleep effects. A third study [29] found no facilitatory effects of sleep at all.

Generalization of Categorical Learning

In tests of category generalizations, subjects learn to classify a group of exemplars to two or more categories based on instructions or through trial and error; and are subsequently tested on their knowledge of the categories when required to classify new exemplars, or the never-seen category prototypes, without feedback. Studies that examined the effects of sleep in these tasks varied greatly by the choice of stimuli, from simple objects (used in infant studies) to abstract shapes, dot patterns, or multi-feature cartoonish characters (used with adults). Results were highly polarized. For both infants/children and adults, approximately equal number of studies showed that sleep enhances correct categorization of new exemplars and prototypes [52–56] and that sleep has no effect [57–60]. One study [57] found no short-term benefits of sleep, but a facilitating effect on prototype recognition a year later, when comparing subjects who slept immediately after training and those who didn’t (though those results are questionable given that by that time the prototypes were already familiar and so the test was no longer examining generalization per se). The only study that tested explicit measures (using a generation test) did not find a sleep effect [59]. No study found direct relation to SWS. Two studies showed REM sleep to be associated with memory of the trained exemplars but not with generalization [57,59], and three studies in infants, all using the same paradigm, showed a relation to spindles and N2 sleep [52–54].

Main trends across studies

Influence of the specific task

Overall, findings tended to be replicated across studies that used the same task (Figure 1A). Almost all studies employing the NRT found facilitatory effects of sleep on explicit recognition of the hidden rule; the two studies examining transitive inference showed the same implicit but not explicit effects of sleep; three of five experiments using the WPT found sleep-dependent facilitation of performance and in two of them the effect was associated with REM sleep; all three studies using the same tone-learning task found implicit sleep effects related to SWS; and findings in the SRTT were pretty well grouped according to the version of the task used, as detailed earlier. On the other hand, experiments studying grammar, categorization and information-integration learning varied greatly in the tasks employed, and the resulting sleep-dependent effects varied accordingly. Yet even then, in experiments that used the very same task (e.g., [52–54] in object categorization; [5,61,62] in grammar rule learning), the same effects of sleep and sleep stage associations were found. This trend suggests that the specifics of the tasks have a great deal of influence on whether sleep affects performance, potentially far more than the general cognitive processes assumed to underlie subjects’ behavior (though, admittedly, it is also true that the same groups of researchers tend to apply the same tasks, which contributes to the likelihood of replication in and off itself; and it is often easier to publish replication of previous results than contradicting findings, a confirmation bias that further contributes to similar tasks producing similar results).

Figure 1:

Comparison of studies. Note that for studies that included several relevant conditions leading to differentiated sleep effects, each condition was treated as a separate study for the purpose of summary statistics as long as different sets of subjects were used in each condition. A: Replication of results by task, for implicit (“Im.”) and explicit (“Ex.”) effects. Only tasks that were repeated in at least two studies are presented. Each icon represents one study; green icons for studies reporting a sleep effect and red icons for null effects. ASRT, Alternating Serial Response Time task; NRT, Number Reduction Task; SRTT, Serial Reaction Time Task; WPT, Weather Prediction Task. Refer to Table 1 for more information about the tasks. B: Comparison between study designs in terms of likelihood of finding a sleep effect. Y-axis: Ratio between the number of studies finding a sleep effect out of the total number of studies with the corresponding design. The specific numbers are indicated above each bar. Designs compared: Time of sleep (overnight/nap); Age of participants (adults/children under 12); Experimental design (sleep vs. wake/sleep vs. sleep deprivation/targeted memory reactivation (TMR) during sleep/single sleeping group); Type of hidden regularity (Temporal/Stationary); Type of hidden regularity only in studies that measured explicit effects (Temporal/Stationary). C: Classification of studies based on whether an implicit or explicit effect was found and the type of regularity used. Each circle represents a study. Color of circle indicates whether any effects in the study were associated with specific physiological sleep measurements (green: effects associated to non-REM sleep, particularly SWS or sleep spindles; blue: effects associated with REM sleep; green/blue: relation to both SWS and REM, or transitions between them; grey: no reported association to any physiological sleep measure, either because none were found or because sleep physiology was not recorded). The Explicit category includes studies that either found explicit effects alone, or both implicit and explicit effects; Circles located in between the Explicit and Implicit-only categories represent studies in which an implicit effect was found together with an indirect or partial explicit effect. See text for details.

No simple effects of experimental design

No single experimental approach seemed to have had a decisive influence on whether a sleep effect, either implicit or explicit, was found (Figure 1B, left to middle). The likelihood of finding a sleep effect when the study employed overnight sleep was somewhat lower compared to a nap (33 of 53 studies vs. 13 of 15 studies, respectively). An even smaller difference was evident in the likelihood of finding an effect when participants were children (9 of 11 studies, including infants, toddlers and elementary-school kids) compared to adults (38 of 57 studies). Experimental design was more difficult to compare due to the low number of studies employing certain type of designs; nevertheless, all experimental designs had a relatively high likelihood of finding effects: studies contrasting sleep and wake groups showed facilitation effects in almost identical ratios to those contrasting sleep and sleep deprivation groups (28 of 46 versus 2 of 3, respectively), whereas studies employing TMR or those examining correlations between performance and physiological measurements in a single-group design almost always yielded effects (7 of 8 and 6 of 6, respectively).

Sleep facilitates explicit detection of temporal, not stationary regularities

Whereas finding a facilitatory effect of sleep on implicit detection of a hidden regularity was common to all paradigms, finding an effect of sleep on the ability to explicitly recognize the hidden regularity was relatively rare, and almost exclusively evident in only two types of tasks: The NRT, and some versions of the SRTT. Studies employing the NRT regularly yielded a facilitatory effect of sleep on explicit rule recognition (3 of 4). Similarly, both of the studies examining explicit effects of sleep using the high-order SRTT, which is considered to rely heavily on the hippocampus [63], yielded an effect (though note that the regularity used in those studies was probablistic, which is sometimes taken to indicate lower rather than higher likelihood of hippocampal involvement; e.g., [36]). Other SRTT versions produced effects in 8 of the 18 studies. Altogether, 14 of the 30 studies examining explicit detection of the rule in those and closely related paradigms found an effect: 12 studies found a behavioral effect of the main manipulation (i.e., facilitation of performance by sleep compared to wake, effect of cueing memory during sleep versus not cueing, or correlations between sleep and performance), and two additional studies found a partial effect (i.e., correlations between explicit detection of the regularity and sleep biomarkers, despite lack of a direct cueing effect [33,64]). Moreover, among the 14 studies that found an effect, 11 studies made use of polysomnographic recordings of sleep and no less than 9 of them have found a relation to non-REM sleep (6 to SWS, two to spindles, and one to SWS-REM transitions; see Table 1), while none have found an exclusive relation to REM sleep.

As exhibited in Table 1, the common theme for both the NRT and SRTT is the use of a hidden regularity with a temporal (or sequential) nature: event x on time t predicts event y happening a few seconds later. The only other task to show a sleep-related effect on explicit detection of a hidden rule, the surveillance task, employed a similar type of regularity [33]. Beyond these tasks, the other paradigms with a temporal rule were statistical tone learning (e.g., [14]) and some grammar learning tasks (e.g., [5]), but, unfortunately, none of the studies using these tasks have investigated explicit detection of the rule (note that some of those in the latter category were infant studies).

The above observation stands in clear contrast to studies examining sleep-dependent explicit recognition of non-temporal (“stationary”) rules. None of the 9 experiments examining such influence found sleep to facilitate explicit rule knowledge, despite employing paradigms as diverse as the WPT, Artificial Grammar Learning, Relational Learning, Information-Integration and Categorization (Table 1). Only one study found, under some conditions, facilitation of a simplified rule [28], which was associated with REM sleep. Note that this difference between tasks employing temporal and stationary rules is eliminated when considering both implicit and explicit effects together: 31 of 45 studies using a temporal rule, and 15 of 22 using a stationary rule, found some kind of a sleep effect, either implicit or explicit (Figure 1B).

To summarize, studies examining the effect of sleep on the extraction of a temporal rule were relatively likely to find sleep was facilitating the explicit detection of this rule when using the appropriate tasks, and this effect was often related to non-REM sleep in general or SWS in particular. In contrast, studies using a stationary rule were highly unlikely to find an effect of sleep on the explicit detection of this rule no matter the task, and in the one case where a partial effect was found – it was related to REM sleep (Figure 1B, right, and Figure 1C).

Discussion

We found two major patterns across studies examining the ability of sleep to facilitate incidental detection of hidden regularities within newly learned stimuli. First, effects are highly task-dependent, with both behavioral and electrophysiological results often replicated across studies using the very same task. Second, sleep-dependent explicit detection of the regularities appears almost exclusively when the hidden regularity has a temporal nature, and tends to be associated with non-REM sleep measures. The implications and possible explanation for these effects are discussed below.

Task-dependency

The major predictor of whether a study produced a sleep effect or not, and which sleep stage was associated with this effect, was found to be the particular task used. In contrast, when results from different tasks are grouped according to the processes they are assumed to engage (e.g., category learning; grammar learning; statistical learning; information-integration), behavioral effects and the sleep stages attached to them were mixed. This result is generally consistent with a previous meta-analysis of sleep and associative processes that grouped together various different types of tasks and, despite finding an overall sleep effect, emphasized the heterogeneity in the results [65]. Such heterogeneity poses challenges to qualitative descriptions of sleep as supporting general high-level processes such as “reorganization” or “assimilation“ (e.g., [66]), as well as to simplistic portraits of sleep as facilitating broadly-defined cognitive faculties such as “insight” (e.g., [9]). Indeed, studies that attempt to tap the effect of sleep on the same declared processes but using very different tasks, often find null effects (e.g., [67,68]). Instead, it appears that the behavioral tasks that are used in sleep studies may contain various properties, yet to be clarified, that affect whether they are sensitive to sleep regardless of the cognitive processes they are presumed to target. Reaching a similar conclusion, a recent critical review of contemporary models of sleep and memory summarized: “when conceiving sleep-memory studies…. all designs should thoroughly consider the kind of procedure, task and instructions provided, with the aim of controlling for awareness, intentionality, individual learning abilities and learning strategies” [23].

Sleep-Dependent Explicit Detection of Temporal Regularities – the Temporal Scaffolding Hypothesis

One property that was common among all studies revealing a clear sleep-dependent explicit effect is the use of a task with temporal regularities. Why would sleep, and particularly SWS, play a special role in such tasks? One possibility is that temporal regularities were simply easier to learn explicitly than stationary ones, independent of sleep (either because such regularities are easier to learn in general, or because of poor choice of tasks with stationary regularities). In that case, sleep could in fact be facilitating all types of explicit learning, but the effect is diminished due to the difficulty in learning the stationary regularities (though see [34] for an example of a task with stationary regularities that yielded explicit learning but no sleep-related effects). Another possibility, however, is that there is an actual bias towards sleep-dependent explicit detection of temporal regularities. One mechanism explaining such putative bias, “temporal scaffolding”, was recently suggested by us [69]. Our model asserts that explicit detection of regularities within newly learned stimuli, often assumed to originate in the hippocampus (e.g., [15]), might be less likely in case the regularities are unexpected and temporal in nature. According to standard models (e.g., [18,70]), ongoing experiences during wake are registered in the hippocampus as episodic memories before they are transferred into permanent storage in the neocortex. Hebbian learning mechanisms influence this process, causing experiences that share common features to be encoded in a correlative manner that reflects those commonalities and support generalization and pattern completion (at least in subareas CA3 and CA1 of the hippocampus; [71,72]). For example, if we repeatedly observe that all individuals coming out of a certain restaurant are wearing suits, we might associate between that attire and this restaurant such that the next time we notice an individual exiting the restaurant, we could predict she will be wearing a suit. However, when regularities have a temporal nature that depends on information occurring over several seconds or more, the typical timescale of Hebbian mechanisms (approximately 50-200ms; [73]) may not be sufficient to create the necessary associations in real time. This is especially true when the regularities are unexpected and learning is incidental, thus discouraging any active attempt to keep ongoing experiences in working memory for search of commonalities. To continue our example, if a truck partially blocks the view of the people coming out from the restaurant such that we can see their head/face as they come out but only notice their attire a few seconds later once they walked away, the regularity would likely be missed. Nevertheless, despite the failure to extract temporal regularities, experiences are still encoded in hippocampus in the form of sequences, and may later be replayed during SWS [73]. One critical feature of memory replay in the hippocampus during SWS is that it does not occur in the same rate of the original experience; in fact, it is time-compressed, by a factor of up to ×20 of the original speed (in rodents; the compression factor in humans is unknown; [1]). Consequently, temporal regularities, which have been originally separated by several seconds, would now fall within typical Hebbian timescales and allow their offline detection just like non-temporal regularities are detected during waking. The result of this process may be the formation of new representations during SWS, which, come the next waking period, could be used by cortical networks to explicitly recognize the hidden temporal regularities, thus explaining the pattern of results summarized above. It is, however, important to note that since the evidence for time-compressed memory replay almost exclusively relies on rodent research, the temporal scaffolding model remains speculative until further corroboration from human studies.

Is detection of regularities a high-level cognitive process?

Many of the studies that demonstrated effects of sleep on extraction of regularities presented these effects as examples of sleep facilitating a high-level learning process (compared to simple memorization; see [66]). While it is true that generalization effects can be potentially complex, the nature of the tasks used to support this claim may not always serve as evidence for the involvement of high-level learning. One example is the effects of sleep on transitive inference [4,42], which are often presented as reflecting learning of the hidden hierarchical rule governing the task. However, the actual findings were that subjects failed to explicitly recognize the hidden rule following sleep, and only sleep-related implicit improvement in generalization was found. This type of implicit effects, in turn, has been criticized in the past as resulting from local strengthening of individual memories that do not reflect true learning of the hierarchies [74,75] (see Figure S1 for a demonstration of this claim using a simple neural network simulation). Similar local strengthening of individual memories could potentially explain other implicit effects of sleep in some of the studies covered here, especially those related to relational memory.

Unlike implicit effects, studies showing explicit effects more likely reflect true high-level learning of generalized rules, given that they require the ability to state these rules or self-generate examples of them; however, even those do not necessarily prove the case that sleep facilitates high level learning since a period of waking after sleep is often required for those effects to kick in (e.g., [9]). In other words, it may be that sleep only contributes to the facilitation of simple associative processes that serve as a substrate for more complex learning, occurring after waking up. More work is required to settle between those two alternatives.

Rule extraction that potentially use different memory systems

While the current review focused on the ability of sleep to facilitate incidental extraction of hidden regularities characterizing newly learned stimuli, it is interesting to compare these results with a line of studies investigating sleep and regularity extraction under different conditions – for example, when the regularities are not completely hidden (i.e., when the instructions given to participants encourage looking for regularities, or when the task demands divert attention to the relevant features in which the regularities are embedded). These include studies using the SRTT (e.g. [34,76] – “explicit instructions” condition); melody learning tasks [77,78]; and tasks investigating relational memory [79,80]. With one exception [80], those studies find a facilitatory effect of sleep on implicit recognition of regularities and, often, also an explicit effect that is associated to SWS. Whereas most of the tasks used temporal regularities, at least one study showed an explicit effect on stationary regularities as well [79]. This, however, may not be surprising: As was shown by Drosopoulos et al. [81], awareness of the possible existence of regularities significantly increases the probability of finding them after a retention interval, both with and without sleep, as learning is no longer incidental and therefore most likely incorporates significant involvement of brain structures other than the hippocampus.

Another line of studies examines extraction of regularities that depend on prior knowledge. Studies of this nature almost always rely on participants employing semantic/linguistic memory to improve performance. They include: (a) studies using the false-memory paradigm, in which subjects learn a list of words with a common theme – for example, pillow, bed, sheets, night – and are then tested for recall of the never-presented common “gist” word, sleep (e.g., [6,82,83]); (b) Studies using the Remote Associate task, which taps the ability to find a common concept linking three presented words (e.g., tooth, ring, age – linked by gold; [3,84]); and (c) studies examining the ability to associate stimuli that have common visual features with concepts with similar meanings [85–87]. The majority of those studies demonstrate a facilitatory effect of sleep on performance, which is often explicit. Importantly, the regularities used in those tasks are non-temporal, suggesting that unlike the studies covered in this review, explicit rule extraction involving non-temporal rules is likely – as long as the regularity is based on prior (semantic) knowledge. Those studies also commonly show either the involvement of REM sleep [3, 85] or, in one case, a negative correlation to SWS [6], suggesting that rather than the SWS-dependent active system consolidation, a different mechanism might be in play – a somewhat expected result given the strong dependency of semantic memory on cortical rather than hippocampal regions [111]. This conclusion is further supported by additional works showing a relation between REM sleep and other tasks that involve semantic knowledge and associative thinking (e.g, Anagram solving, [10]; semantic priming, [88]), as well as by tasks tapping previous knowledge that is non-semantic (e.g., [89]). Note, however, that at least one class of semantic tasks, involving the integration of new stimuli into existing semantic structure (e.g., [8]), often shows a relation to markers of non-REM sleep, such as sleep spindles.

Conclusion

In the current review, we showed that the reported facilitatory effects of sleep on the extraction of hidden regularities within newly encoded stimuli strongly depend on the specifics of the tasks used, and should not be misinterpreted as suggesting a broader effect of sleep on general cognitive processes. Moreover, we questioned whether in some cases effects actually demonstrate sleep-dependent high-level rule extraction rather than local strengthening of connections not unlike the one characterizing basic memory consolidation. One common theme that did emerge in our review was that sleep-dependent explicit extraction of regularities occurs when the regularities have a temporal nature, a result that could be explained if compressed memory replay during SWS implemented a temporal scaffolding mechanism. In that respect, future studies employing temporal rule learning – for example, artificial grammar learning in adults – should add explicit measurements of rule extraction to verify the generality of this conclusion. Nevertheless, the association between sleep and explicit learning of temporal regularities seems to apply only when the regularities are fully encapsulated within the newly encoded stimuli; when prior knowledge (particularly semantic knowledge) is involved, a sleep-dependent explicit effect can be demonstrated even with stationary regularities, though it probably involves different mechanisms, which are related to REM sleep.

Practice Points.

1. There are no general experimental designs that are substantially more likely to produce sleep-induced regularity extraction than others, but the specific behavioral task chosen has a strong impact on the outcome.

2. When considering the effects of sleep on explicit (rather than implicit) extraction of hidden regularities within newly formed memories, sleep is much more likely to facilitate detection of hidden temporal regularities than non-temporal regularities, with slow-wave-sleep, in particular, strongly involved.

3. When extraction of hidden regularities depends on former knowledge that is likely not hippocampally-dependent, sleep does seem to facilitate the explicit extraction of both temporal and non-temporal regularities, with rapid-eye-movement sleep mostly involved.

4. At least in some cases, extraction of regularities that sleep is claimed to facilitate can, in fact, be explained by local memory-strengthening mechanisms.

Research Agenda.

1. Sleep studies would do well to avoid characterizing their findings based on the presumed underlying general cognitive processes, and, rather, stick with the particular nature of the behavioral task used.

2. It would be beneficial to include both implicit and explicit measures of regularity extraction in future sleep studies whenever appropriate. In particular, measures of explicit rule learning should be included when examining the effects of sleep on Artificial Grammar Learning in adults.

Glossary of Terms

AGL:

artificial grammar learning

ASRT:

Alternating Serial Reaction Time

EEG:

electroencephalogram

NRT:

Number Reduction Task

REM:

Rapid-Eye-Movement

RT:

reaction time

SRTT:

Serial Reaction Time Task

SWS:

Slow-Wave-Sleep

TMR:

targeted-memory-reactivation

WPT:

Weather Prediction Task