Memories replayed: reactivating past successes and new dilemmas

Abstract

Our experiences continue to be processed ‘offline’ in the ensuing hours of both wakefulness and sleep. During these different brain states, the memory formed during our experience is replayed or reactivated. Here, we discuss the unique challenges in studying offline reactivation, the growth in both the experimental and analytical techniques available across different animals from rodents to humans to capture these offline events, the important challenges this innovation has brought, our still modest understanding of how reactivation drives diverse synaptic changes across circuits, and how these changes differ (if at all), and perhaps complement, those at memory formation. Together, these discussions highlight critical emerging issues vital for identifying how reactivation affects circuits, and, in turn, behaviour, and provides a broader context for the contributions in this special issue.

This article is part of the Theo Murphy meeting issue ‘Memory reactivation: replaying events past, present and future’.

1. Introduction

Our minds are constantly active. Even once an experience, such as trying to recall the location of a lost set of car keys, has ceased, it continues to be processed ‘offline’, which enables inspiration to strike––and those keys to be found––at the most unlikely of times. Offline processes have reliable effects upon our memories. For example, they enhance our memories during sleep, so that performance on a skill learnt one day is improved by as much as 25–30% the next day ([1–4]; for reviews see [3–5] and also in the current issue [6]). These and other memory changes have been attributed to specific processes, including the concept that a memory is reactivated or replayed offline after a memory has been formed. Such memory reactivation may lead to synaptic strengthening [7], weakening ([8]; please see in this current issue [9]), structural changes (please see in this current issue [10]) and perhaps more generally to the reorganization of a memory [11,12]. This special issue of the Philosophical Transactions of the Royal Society focuses upon the topic of memory reactivation and follows on from a 2-day meeting entitled ‘Memory reactivation: replaying events past, present and future’ held at Chicheley Hall in May, 2019.

The meeting brought together those working on rodents, non-human primates and humans using a variety of techniques from optogenetics, to computational approaches to behavioural analysis. This special issue has a similar diversity of approaches and views. Within this diversity, there is a unity of ambition for creating a deeper understanding of reactivation and its importance for memory processing. Here we discuss the challenges to developing such an understanding, and specifically what makes offline processes, such as reactivation, even more difficult to study and explore than other more traditional brain–behaviour relationships. Addressing these challenges may benefit from developing criteria for identifying and defining reactivation (please see the Consensus Statement in this current issue [13]).

2. Exploiting events to understand the brain–behaviour relationship

Establishing the connection between mental state and behaviour is challenging. Experimental work has risen to this challenge in part by focusing on brain activity before or after an event (i.e. event-related design). That event could be the presentation of a stimulus, the initiation, or the inhibition of a movement, or the encoding of a memory. Brain activity before or after these events has been recorded across many species from rodents, to non-human primates, to humans using a diverse array of techniques from single-unit recording to functional imaging such as, functional magnetic resonance imaging and magneto- or electro-encephalography (MEG/EEG). Fortunately, there is also a diverse array of techniques that can be embedded within event-related designs to modify brain activity with astonishing temporal precision. From the use of optogenetics in rodents where a pulse of light can be used to modify, or even create brain activity, to transcranial magnetic stimulation (TMS) in humans where a single pulse of magnetic stimulation can modify brain activity [14,15]. Together, these techniques provide complementary perspectives by measuring activity around an event, such as the encoding of a memory, and manipulating that activity to determine the critical importance of that activity for behaviour.

Yet, different techniques may be identifying, or manipulating different processes (please see, in the current issue [16–18]). For example, single-unit recording, which predominately measures cell body firing, could be identifying a very different type of reactivation, performing a different computation than identified by EEG and local field potentials, which predominately measures dendritic activity. Alternatively, despite occupying different biological compartments (cell body versus dendrite), the measured activity could be different aspects of the same biological process. Thus, the plethora of available techniques for measuring reactivation presents the easy to state, but difficult to solve, problem of piecing together evidence from across studies, and determining whether the same or different reactivation processes are being measured. This though is not the only challenge. Experimental design also presents a challenge, which cannot easily be overcome by the enormously powerful event-related design.

3. The challenges of investigating the offline brain

For the offline brain, there is no event. There is no single, discrete point in time in which offline processing is initiated, or subsequently ends. It can potentially start once a memory has been formed, and continue for the subsequent hours, or perhaps days across different brain states (wakefulness versus sleep). Across those many hours, there may be a single event that underlies a change in a formed memory––where a ‘needle’ of memory change needs to be uncovered among the ‘haystack’ of other changes in neuronal activity. Alternatively, a succession of events perhaps relying upon transitions from one brain state to another may be necessary for an offline change in a memory (please see in the current issue [19–21]). As a consequence, identifying how offline processing is achieved and leads to behavioural change cannot benefit from using the strengths of event-related designs, which have been so vital to our understanding of other aspects of cognition. Instead, a new experimental approach is required.

Many different experimental techniques have been developed to identify reactivation. All seek to record brain activity during memory formation and then compare that against activity recorded during subsequent rest. The comparison can take many forms; the pattern of activity, its variance and similarity in principal components are all examples of the types of brain activity comparison that have been made ([5]; please also see in the current issue [16,17,22]; figure 1). All of these techniques essentially rely upon what has become the defining feature of reactivation: the same brain activity during memory formation being found during subsequent rest. However, this approach is very poorly constrained. For example, the period of rest following memory formation is vast extending for hours and perhaps days, which makes it possible that a pattern of activity resembling that during memory formation may arise by chance during rest. Fortunately, other features of reactivation including its link to learning can perhaps provide a source of valuable constraint to help in its reliable and robust identification.

Figure 1.

Methods to measure reactivation that use supervised-learning techniques. (a) Pairwise correlation showing the correlations between neuron pairs and how they change from pre-sleep (top), to during the task (middle) and post-sleep (bottom; for example, see [23]). Each dot on the circle represents a neuron and the line thickness indicates correlation strength. (b) Sequence replay in which each line represents the activity of one neuron, thus the sequence of activity during the task (left) can be found in a time-compressed replay during subsequent rest (right; for example, see [24]). (c) Similar time compression can also be found in template matching techniques, for which the actual sequence between neurons is not critical (for example, see [25]). (d) Finally, dimensionality reduction techniques such as principle component analysis (PCA) can also be used to identify cell groups and then be used to track the cell group activity across time (for example, see [26,27]). For examples of unsupervised-learning machine learning techniques that are used for memory reactivation analysis please see [17] in this issue. (Online version in colour.)

4. Reactivation induced by learning

Learning should lead to subsequent reactivation. For example, reactivation is present within the motor cortex of rodents following the learning of a skill memory [25,28]. However, learning is not unique in being able to induce reactivation. Performing even simple tasks can lead to reactivation [23,29,30]. Even in the absence of a specific behaviour or task to perform, the structured patterns of activity present during wakefulness can re-emerge during sleep [31]. It is perhaps important to distinguish this type of reactivation, which relates to experience, from reactivation that emerges specifically owing to learning and leads to memory changes, such as enhancement, stabilization and reorganization [32].

Reactivation induced through learning may simply differ quantitatively from that induced by experience. For example, there may be more reactivation events following the learning as opposed to the performance of a skill. Novel events, such as learning a new skill, have been linked to increased firing in the ventral tegmental area and an increase in hippocampal reactivation [33–35]. However, there may also be qualitative differences between these reactivation events. For example, the information content of the reactivation events, or the brain areas participating in the reactivation may differ (please see in the current issue [19,20,36]). Distinguishing between how reactivation is induced may provide powerful insights into how reactivation drives circuits, which leads to either a maintenance of performance following a routine experience, or a change in performance following a novel learning event.

There are different changes in the connectivity of brain circuits following the performance of a movement compared to the learning of a new motor skill [37]. Although the movements performed are similar (i.e. whole arm reaching movements), the changes in large-scale connectivity are different, which may be driven by different forms (either qualitatively or quantitatively) of reactivation. Structured patterns of activity arise spontaneously when behaviour has ceased, some of these are reactivation events, and only some of those are likely to be related to any prior learning.

5. Reactivation content

The information encoded at memory formation should also be present during reactivation. Yet very few, if any, studies have tested this aspect of reactivation; instead, it has largely been assumed that during reactivation the information being processed is somehow related to the information encoded at memory formation. It is easy to understand what has led to this assumption.

Firstly, such an assumption is extremely seductive when finding similar patterns of brain activity at memory formation and subsequently offline during reactivation events. It seems natural to assume that a similar pattern of activity should indicate that a similar type of information is being processed. However, just because an event such as memory formation elicits a spatio-temporal pattern of neural activity does not mean that every time that same or similar activity pattern is observed then that memory is being formed or processed. This is similar to the fallacy that the activation of a specific brain area; for example, the medio-temporal lobe is always attributable to the processing of a specific type of information (for example, declarative or episodic information; i.e. reverse inference is not valid).

Secondly, there are very reasonable pragmatic reasons for making the assumption that the same information encoded at memory formation is also processed during reactivation events. The challenges in relating a pattern of brain activity to a particular information source are substantial. For example, demonstrating that activity within the visual system is linked to the processing of a specific aspect of a viewed face, which is being used to determine an individual's identity, is only starting to be carried out (please see in the current issue [22]). In principle, a similar experimental approach would enable a particular information source such as lip position (smiling or not) to be tracked through brain circuits from initial presentation to memory formation. At present though, we lack an established means to link a pattern of offline brain activity to a particular type of information. Being able to do so is not simply essential for testing a key aspect of reactivation––that information encoded at memory formation is processed again offline––it would also allow detection of what specific information is being reactivated.

Only a subset of the information encoded during memory formation may be reactivated. For example, different aspects of a skill memory are enhanced over different brain states––the goal is enhanced over sleep, while the action is enhanced over wakefulness––and this state-dependent dissociation may be owing to only a critical subset of information acquired at skill formation being reactivated [38]. This state-dependent dissociation may be because different forms of replay take place over different brain states. Following spatial learning the replay during wakefulness is a less faithful (i.e. veridical) version than during sleep of the pattern of activity during memory formation [39]. These different forms of replay may arise because different information is being processed, or a different type of processing is taking place during these different states [11,12,26,40].

Identifying the type of information being processed during reactivation events could provide a mechanistic explanation for the nature of offline memory changes [41]. It is ironic and frustrating, perhaps in equal measure, that information content of a memory, while so central to many theories and descriptions of memory, is profoundly difficult to measure [42–44]. Potentially, the information content of reactivations may affect when they occur (rapid eye movement (REM) versus non-REM (NREM) versus wakefulness) and how they occur (i.e. the importance of spindles, ripples and up-states during NREM figure 2; please see in the current issue [18,20,21,36,45–49]). Yet at present, the information being processed during reactivation can at best be deduced based upon the subsequent offline changes in a memory.

Figure 2.

Different oscillations that have been linked to memory reactivations. (a) The slow oscillation (SO) is caused by global on-off states during NREM sleep—which is visible on the surface EEG as a K-complex (0.5–1.5 Hz). (b) Slow wave activity (SWA, or delta waves, 1–4 Hz), which are owing to local on-off states occurring mainly during deep or slow wave sleep. (c) The sleep spindle (12–16 Hz). This is present throughout all NREM sleep. (d) The sharp-wave ripple (SWR) of the hippocampus. The SWR is comprised two different components—the ripple and the sharp wave—that are seen on different electrode sites. The ripple occurs in the pyramidal layer while the sharp wave occurs in the input layer.

6. Reactivation and memory changes

Reactivation is frequently linked to memory changes. The performance of a newly formed memory, such as a skill is enhanced offline during wakefulness, or over sleep, and this improvement in performance may be owing to memory reactivation (for example, see [3,4,50,51]). Other changes also occur to a memory offline following its formation. From the stabilization of a memory making it resistant to disruption and interference in the hours after its formation, to memory reorganization leading to the extraction of common features (for model please see in the current issue [36]; for example, the common meaning across a list of words, or the common structure within a sequence of different events (for a review, see [52])). These memory changes have all been attributed to reactivation. However, much of the evidence linking offline memory changes and reactivation is circumstantial. Both occur following memory formation.

Nonetheless, some work has established a direct link between reactivation and offline memory changes. One approach has been to correlate reactivation events with subsequent offline memory changes. For example, reactivation within the motor cortex of rats following the acquisition of a motor skill is correlated with subsequent offline improvements in that skill ([25]; for a review, see [28]; please also see in the current issue [9]).

Other studies have sought to make a causative connection between reactivation and memory changes. One approach has been to disrupt brain activity when and where reactivation events occur. For example, reactivation occurs within the motor cortex after learning, and applying TMS to this site, and at this time disrupts subsequent offline performance improvements [53,54]. However, this approach lacks specificity. It is not just the reactivation events that are disrupted but also the function of an entire large-scale brain network. A more specific approach has been to disrupt sharp-wave ripples (SWR), which are high-frequency physiological events that are closely associated with subsequent reactivation events (figure 2). Disrupting these prevents subsequent offline memory changes [55]. Despite the elegance of this approach it too lacks some specificity because rather than directly targeting reactivation it disrupts an event closely related to reactivation (i.e. SWRs). Recent beautiful experimental work using optogenetics has shown that reactivation is critical for subsequent offline changes in a skill memory [41]. Subsequent work elegantly showed that specifically disrupting the reactivation of a memory modified subsequent performance of that memory, while the performance on other memories whose reactivation was not disrupted was not affected [56]. At least in principle, optogenetics could allow reactivation not only to be disrupted but also to be modified in a multiplicity of ways. The number of reactivation events, the speed of those events could all be modified providing insight into not only the importance of reactivation but also how it drives offline memory changes.

7. Reactivation and its mechanistic link to memory changes

What remains unclear is how reactivation leads to memory changes. Intuitively the notion of reactivation is appealing because it seems to provide an offline period of additional practice or training. Within this framework, reactivation leads to exactly those memory changes that would be provided by prolonged practice. Many different strands of evidence are consistent with this viewpoint.

A skill can be enhanced through practice, and enhancement also occurs offline where it is correlated to reactivation events [3,4,25,28]. Equally, a memory can be unstable, susceptible to disruption following its formation, but through prolonged practice it becomes stable and resistant to disruption ([57], for a review, see [58]). This same transformation from an unstable, vulnerable memory to a stable memory can be achieved offline over many hours (i.e. greater than 2 h; for example, see [59,60]). Even more qualitative changes, such as becoming aware of an underlying pattern in a sequence of movements is achieved through practice, and also offline [61,62]. Reactivation is envisaged to drive changes within neural circuits and across ensembles that resemble those achieved during practice, and so similar memory changes are created during practice and offline. Reactivation then is not qualitatively different from processing during practice; it may simply be the residue of events that could not be completed during practice ([63]; see also [64]). It would be triggered during practice and subsequently continues offline perhaps multiplexed in with the other patterns of neural activity that are supporting current behavioural performance. Yet, there are some important problems with this perspective.

The same or at least broadly similar circuits would be expected to be critical for the acquisition of a memory and its subsequent offline processing. After all, practice and reactivation during subsequent offline processing are being envisaged as essentially the same process. However, while the hippocampus may not be critical for the formation of some types of memory, it is absolutely vital for their subsequent offline processing [65,66]. This demonstrates that different circuits are being driven during practice and offline processing, which implies that distinct mechanisms operate during practice and offline processing, and consequently, reactivation is not simply a continuation of mechanisms engaged during practice.

Reactivation is also not identical to the patterns of activity during memory formation. Reactivation events generally take place over a smaller period of time than the original pattern of activity present during memory formation (i.e. they are time compressed). Within any mechanistic framework, there needs to be an explanation for reactivation events being time compressed. For example, it has been proposed that time compression could enhance Hebbian plasticity [7,67].

Time compression is not a universal feature of reactivation. While frequently reported in rodents, it is not found in songbirds. When learning their song, the pattern of neural activity during practice is almost exactly reactivated during subsequent rest and critically takes place over the same period of time. The birds are learning a highly stereotyped motor performance––there is little or no variation in their song. Highly stereotyped behaviours may be improved, or supported though high-fidelity reactivation, which may depend upon songbirds not having high-frequency SWR [68].

Yet, by contrast, the lack of fidelity associated with time compressed reactivation may allow more flexible behaviours; perhaps owing to the SWR, which is unique to mammals. For example, discovering that a mathematical problem involving a set of iterative steps can be solved more quickly because the answer at one of the earliest iterations is always the final solution too [69]. This solution that ‘short-circuits’ many iterative steps to quickly arrive at an answer is dependent upon sleep, and perhaps reactivation during sleep. Another solution to arriving at an answer faster, which some participants use, is simply to increase the speed at which each iterative step is completed. Both of these strategies may depend upon reactivation; however, they may rely upon qualitatively different types of reactivation.

Time compression may allow an entirely novel and flexible approach to the problem with iterative steps being ‘short-circuited’; whereas, simply enhancing the speed of each step may be dependent on higher fidelity reactivation akin to that observed in songbirds. This suggests that at the very least time-compressed reactivation may make unique contributions to memory processing, which cannot be achieved through practice.

One such is the creation of generalizable knowledge allowing performance to be applied flexibly to different situations [70]. This has been linked to reactivation––perhaps specifically time compressed reactivation––and to the offline instability of a memory following its formation [71–73]. These mechanisms––reactivation and memory instability––need not be mutually exclusively or even inextricably linked; it seems likely that there would be multiple distinct ways in which generalization, and the creation of broad concepts could be achieved. Thus, reactivation and other offline processes including time-compressed reactivation may drive circuits and lead to memory changes that are distinct and complementary to that achieved during practice. However, this leads to challenging questions about what reactivation is doing––in terms both of biological mechanism and computational function––because it can no longer be described simply as covert practice that is lingering on after memory formation.

Fortunately, we are perhaps on the brink of understanding the importance of time compression. A memory can be reactivated using optogenetics, and at least in principle, it may be possible to manipulate the number of cells activated, and their synchrony (or otherwise) across multiple ensembles to determine how the properties of reactivation––including time compression––determine offline memory changes.

8. The past success and future challenge

Reactivation can no longer be dismissed as the mere ‘echo’ of earlier memory formation. It is correlated with and also critical for the development of offline memory changes (for example, see [25,41,55,56]). However substantial challenges remain. For instance, how reactivation drives cellular changes (synaptic to myelination), how this alters function within and across circuits, and in turn changes memory peformance remains poorly understood. Memory changes such as an increase in skill that occur during practice can also be driven by reactivation during subsequent offline processing; equally, other memory changes are perhaps the unique product of reactivation during offline processing ([3,4,25] cf. [70–73]). These different memory changes––some the same as during practice others unique to offline processing––may be driven by an equally diverse set of flavours of reactivation (for example, high fidelity versus substantially time compressed). This diversity may be owing to the information content being different for different reactivation episodes. Yet, at present we lack a clear means to measure the information contained within a memory at its formation, far less during its reactivation. Addressing these challenges offers the promise of a complete understanding of how the now undeniable link between reactivation and memory change is created.

Data accessibility

No dataset was generated by this work.

Authors' contributions

E.M.R. wrote and edited the work; L.G. edited the work and provided the figures.

Competing interests

We declare we have no competing interests.

Funding

We received support from the Royal Society in the form of a Theo Murphy meeting grant, which provided the basis for this special issue.