Oscillatory patterns in temporal lobe reveal context reinstatement during memory search

Abstract

Psychological theories of memory posit that when people recall a past event, they not only recover the features of the event itself, but also recover information associated with other events that occurred nearby in time. The events surrounding a target event, and the thoughts they evoke, may be considered to represent a context for the target event, helping to distinguish that event from similar events experienced at different times. The ability to reinstate this contextual information during memory search has been considered a hallmark of episodic, or event-based, memory. We sought to determine whether context reinstatement may be observed in electrical signals recorded from the human brain during episodic recall. Analyzing electrocorticographic recordings taken as 69 neurosurgical patients studied and recalled lists of words, we uncovered a neural signature of context reinstatement. Upon recalling a studied item, we found that the recorded patterns of brain activity were not only similar to the patterns observed when the item was studied, but were also similar to the patterns observed during study of neighboring list items, with similarity decreasing reliably with positional distance. The degree to which individual patients displayed this neural signature of context reinstatement was correlated with their tendency to recall neighboring list items successively. These effects were particularly strong in temporal lobe recordings. Our findings show that recalling a past event evokes a neural signature of the temporal context in which the event occurred, thus pointing to a neural basis for episodic memory.

The pivotal distinction between memory for facts (semantic memory) and memory for episodes or experiences (episodic memory) has been argued to reflect, at least in part, the reinstatement of a gradually changing context representation that reflects not only external conditions, but also an ever-changing internal context state (1, 2). According to this view, the unique quality of episodic memory is that in remembering an episode, we partially recover its associated mental context, and that this context information conveys some sense of when the experience took place, in terms of its relative position along our autobiographical time line.

A number of laboratory memory tasks rely on episodic memory, including experimenter-cued tasks (e.g., item recognition and cued recall) and self-cued tasks (e.g., free recall). Performing these episodic memory tasks requires distinguishing the current list item from the rest of one's experience. According to early theories of episodic memory (e.g., 3, 4), context representations are composed of many features that fluctuate from moment to moment, gradually drifting through a multidimensional feature space. These contextual features may reflect environmental cues, recently studied items, participants’ internal mental states, or may evolve randomly over time. During recall, the context representation forms part of the retrieval cue, enabling us to distinguish list items from nonlist items. Understanding the role of context in memory processes is particularly important in tasks such as free recall, where the retrieval cue is “context” itself.

Recent neurocomputational models of episodic memory (5, 6) suggest that contextual reinstatement underlies the contiguity effect: people's tendency to successively recall items that were presented in nearby positions on a studied list (7). Behavioral studies of memory show that, for a given class of memories, the contiguity effect can span many other intervening memories (8–10). This result is difficult to explain according to the view that contiguity arises from direct item-to-item associations that are established within a few seconds, as suggested by other classes of psychological and neurobiological theories (11–14). The contiguity effect is an example of temporal clustering, which is perhaps the dominant form of organization in free recall.

Although this behavioral evidence provides indirect support for context-based theories of memory, there is no direct neurophysiological evidence for contextual reinstatement. To test the context reinstatement hypothesis, we studied 69 neurosurgical patients who were implanted with subdural electrode arrays and depth electrodes during treatment for drug-resistant epilepsy. As electrocorticographic (ECoG) signals were recorded, the patients volunteered to participate in a free recall memory experiment, in which they studied lists of common nouns and then attempted to recall them verbally in any order following a brief delay.

Results

The recorded ECoG signals simultaneously sample local field potentials throughout the brain, and can be analyzed in terms of specific time-varying oscillatory components of neural activity. Such components have been implicated in memory encoding and retrieval processes (15–20) and in the representations of individual stimuli (21). For each study and recall event, we analyzed these oscillatory components across all recording electrodes (Fig. 1 A and B). We constructed a matrix containing, for each electrode, measurements of mean oscillatory power in five frequency bands (δ: 2–4 Hz, θ: 4–8 Hz, α: 8–12 Hz, β: 12–30 Hz, and γ: 30–99 Hz) during each study event (200–1,600 ms relative to each word's appearance on screen) and recall event (−600 to 200 ms relative to vocalization). We then used principal components analysis (PCA) to distill these highly correlated neural features into a smaller number of orthogonal components (Fig. 1C). In this way, each component reflects a linear combination of the power in each frequency band across all recording electrodes, such that the pairwise neural similarities between events are preserved.

Fig. 1.

Illustration of behavioral and electrophysiological methods. (A) After studying a list of 20 words and performing a brief distraction task, a participant recalls as many words as he can remember, in any order. (B) During each study presentation and just before each recall event, we calculate the z-transformed oscillatory power at each recording electrode in each of five frequency bands (δ: 2–4 Hz, θ: 4–8 Hz, α: 8–12 Hz, β: 12–30 Hz, and γ: 30–99 Hz). (C) We use PCA to find a smaller number of orthogonal components that jointly account for a large proportion of the variation in the data shown in B. We select those components that show significant positive autocorrelation (a defining feature of temporal context) during the study phase of the experiment. We then compute the similarity (normalized dot product) between the feature vectors of each recall event (e.g., “nose”) and the feature vectors associated with the corresponding study event (lag = 0), as well as the similarity of the recall event to surrounding study events with varying lags.

Context-based models conceive of context as a representation that integrates incoming information with a long time constant (22), leading to the prediction that the representation of temporal context evolves gradually as the experiment progresses (23). We asked whether the neural recordings supported a gradually changing representation of context by regressing, for each participant, the mean similarity between the neural vectors (in principal component space) on their positional distance in the studied list (Fig. 2). The similarity in recorded activity during each pair of word presentations decreased with the positional distance between the presentations [t test on distribution of t values from the regressions: t(68) = −9.31, P < 10⁻¹⁰], indicating that the ECoG recordings evolve gradually over the course of the studied lists. Whereas this gradually changing neural representation is consistent with context-based models, such a result would also be expected to arise as a result of other autocorrelated neural processes that lack the rich dynamics implied by context-based theories of memory. To determine whether this gradually changing neural representation reflects the contexts in which list items were studied, we selected PCA-derived features from study events that showed significant positive autocorrelations (Materials and Methods), a defining feature of temporal context, for further analysis. In the remainder of this paper, we refer to the set of autocorrelated principal components as feature vectors.

Fig. 2.

Evolution of ECoG activity as participants study lists of words. Mean neural similarity is shown as a function of study distance (difference in serial position) between pairs of presented words. Error bars denote ±1 SEM.

To test whether the gradually changing neural representation we identified is reinstated during recall, we compared feature vectors recorded during each study and recall event. First, we identified the serial position (on the studied list) of each correctly recalled word. If neural activity during study is reinstated during recall, then the neural activity recorded during a given recall event should be more similar to activity recorded during the study event for the same word than during study events for other words (Fig. 3 E and F). This finding would not be expected if the neural activity we measured did not contain content or context information (Fig. 3D). For each correctly recalled word (e.g., “nose” in Fig. 1A), we calculated the similarity between the feature vector associated with the recall event and the feature vectors associated with each of the studied items (e.g., ant, tape, nose, vase, sheep), where similarity is defined as the normalized dot product between the feature vectors (the vectors were normalized to have unit length before the dot product was performed). We assigned each studied item a lag (positional distance) relative to the recalled item (e.g., vase has a lag of +1 to “nose”, ant has a lag of −2 to “nose”, and nose has a lag of 0 to “nose”). We found that the mean neural similarity at lag = 0 was significantly greater than the mean neural similarity at other lags [paired-sample t test across 39 participants with at least 5 autocorrelated features: t(38) = 3.10, P = 0.004; Fig. 4A]. This result would arise if the signal represents either content (the list words themselves) or context (the cues surrounding the items).

Fig. 3.

Predicted neural similarity as a function of lag according to three models. (A–C) Pattern of activations for a simulated 20-neuron neural network as a 15-item list is studied. Events 1–15 of each matrix show activations after each item is presented. Events 16–20 show activations as distracting items are presented. Events 21–26 show activations as items 15, 10, 1, 2, 4, and 3 are recalled (in that order). In each simulation, a single neuron is activated during each experimental event. Once activated, a neuron's activity decays gradually; thus, multiple neurons may be active at a given time. (A) For the autocorrelated noise simulation, each experimental event activates a random neuron, irrespective of which item is being presented or recalled. (B) For the content reinstatement simulation, each neuron is activated by a single item or distractor (neurons 1–15 represent items and neurons 16–20 represent distractors). Only content information (specific to a single item) is reinstated during recall. (C) The context reinstatement simulation is similar to that shown in B, but here we simulate context reinstatement during recall. (D–F) Average expected neural similarity between the pattern of activity during study and recall as a function of lag. Each simulation used the same presented and recalled items that were included in our data analyses (Fig. 4). Further details on the simulations are presented in SI Materials and Methods.

Fig. 4.

A neural signature of temporal context reinstatement. (A) Neural similarity between the feature vector corresponding to recall of a word from serial position i and study of a word from serial position i + lag [black dot denotes study and recall of the same word (i.e., lag = 0)]. (B) Participants tend to successively recall neighboring study items (the contiguity effect). Here, we show the probability of recalling an item from serial position i + lag immediately following an item from serial position i, conditional on the availability of an item in that list position for recall. Error bars in A and B denote ±1 SEM. (C) Participants exhibiting greater context reinstatement also exhibited more pronounced contiguity effects. Here, the t value associated with the regressions in A serves as a measure of the degree context reinstatement for each participant. (Only the regressions for negative lags were used, because the regressions for positive lags are not expected to distinguish between content and context reinstatement; Fig. 3). The temporal clustering score measures the degree to which responses were clustered on the basis of their temporal contiguity at study (SI Materials and Methods, Analysis Methods).

To distinguish between content and context reinstatement, we compared the feature vectors associated with each recall event with the feature vectors associated with the neighbors of the recalled word in the study sequence. Context-based models predict that similarity between feature vectors should decrease as a function of absolute lag in both the forward (positive) and backward (negative) directions (22). For each participant, we regressed the mean neural similarity between feature vectors on lag separately for positive and negative lags (two regressions were performed for each participant). Each regression yielded a t value associated with the slope (β coefficient) of the fitted line. Consistent with the context-reinstatement hypothesis, t tests on the distributions of t values across participants indicated that similarity decreased with absolute lag in both the positive [t(38) = −3.63, P = 0.0008] and negative [t(38) = −2.42, P = 0.02] directions (Fig. 4A).

We conducted three neural network simulations to contrast the predictions of three models of the observed feature vector dynamics (Fig. 3 and Fig. S1). In the autocorrelated noise model (Fig. 3 A and D), neural activity evolves randomly over time, irrespective of what is happening in the experiment. In the content reinstatement model (Fig. 3 B and E), each neuron represents a different word; a neuron is activated if its associated word is presented or recalled. In the context reinstatement model (Fig. 3 C and F), each neuron also represents a different word. We simulate context reinstatement by activating not only the neuron associated with the word being recalled, but other neurons that were active at the time the recalled word was studied. Of these three simulations, only the context reinstatement model predicts that neural similarity will decrease substantially with absolute lag in both the positive and negative directions, as observed in the neural data (further details and discussion are provided in SI Materials and Methods).

The decrease in neural similarity with absolute lag mirrors the contiguity effect: people's striking tendency to make transitions to neighboring items rather than remote ones, as seen in behavioral data for the same participants (Fig. 4B). However, the forward asymmetry in the contiguity effect (Fig. 4B) is evident in neither the neural data (Fig. 4A) nor our simulation of context reinstatement (Fig. 3F). Models in which recall of an item retrieves both context and content information (e.g., 24) account for the forward asymmetry by virtue of the content information providing a boost in similarity to subsequently studied items (Fig. 3E). In this way, if our feature selection framework filters out content information, one would not expect to see asymmetry in the neural data (further discussion is provided in SI Materials and Methods).

Consistent with the hypothesis that the contiguity effect arises due to context reinstatement (5, 6, 24), participants with stronger neural signatures of context reinstatement exhibited more pronounced contiguity effects than did participants with weaker reinstatement effects (r = 0.42, P = 0.007; Fig. 4C). In recalling a list item, people not only reinstate that item's representation, as has been recently documented (25, 26), but they also revive the brain activity associated with neighboring items. Further, the degree of this neural context reinstatement effect predicts the tendency of an individual participant to recall neighboring list items successively during memory search.

Having identified a neural signature of context reinstatement, we next asked whether this phenomenon could be localized to one or more brain regions. For example, recent work has given rise to the hypothesis that the medial temporal lobe (23, 27–32) and prefrontal cortex (22, 32, 33) are critically involved in the maintenance and updating of temporal context. To test for regional specificity of context reinstatement, we repeated our test for neural context reinstatement using electrodes from each of the following regions of interest (Fig. 5A): temporal lobe (including the hippocampus and medial temporal lobe), frontal lobe (including prefrontal cortex), parietal lobe, and occipital lobe. We found that neural activity recorded from temporal lobe electrodes exhibited a decrease in similarity with increasing absolute lag in both the positive and negative directions [positive: t(20) = −2.20, P = 0.04; negative: t(20) = −2.82, P = 0.01; Fig. 5B]. As in the whole-brain analysis, the neural signature of context reinstatement in the temporal lobe was significantly correlated with the temporal clustering of participants’ recalls (r = 0.48, P = 0.03; Fig. 5C). The frontal lobe exhibited a weak neural signature of context reinstatement that trended toward significance [positive: t(20) = −2.85, P = 0.01; negative: t(20) = −1.54, P = 0.14]. However, this frontal signature of context reinstatement was not correlated with temporal clustering of participants’ recalls (r = −0.08, P = 0.73). Our findings in the parietal and occipital lobes were inconclusive due to insufficient data.

Fig. 5.

Evidence for context reinstatement in the temporal lobe. (A) Each dot marks the location of a single electrode from our dataset in Montreal Neurological Institute space. We divided our dataset into four regions of interest: temporal lobe (blue, 1,815 electrodes), frontal lobe (red, 1,737 electrodes), parietal lobe (yellow, 512 electrodes), and occipital lobe (green, 138 electrodes). (B and C) Same format as in Fig. 4 A and C but reflecting data from temporal lobe electrodes only. Of the temporal lobe electrodes, 13.9% were located in hippocampus, 27.0% were located in medial temporal lobe structures (excluding hippocampus), and the remaining 59.1% were located in other temporal lobe regions.

Our inability to find neural signatures of context reinstatement in extratemporal brain regions does not necessarily mean that those regions are not involved in context reinstatement. Furthermore, our analysis does not distinguish between structures contained within the regions of interest we examined. Thus, an important goal of future work will be to more precisely localize the neural machinery underlying context reinstatement.

Discussion

The preceding analyses demonstrate that when recalling an item, the pattern of neural activity exhibits graded similarity to the neural activity measured during the encoding of items studied in neighboring list positions. Furthermore, the strength of this neural contiguity effect tracks the behavioral contiguity effect in free recall: Participants who exhibit a stronger tendency to make transitions among neighboring items during recall also exhibit a stronger relation between neural similarity and absolute lag. This pattern of results is exactly what one would predict on the basis of retrieved context theories of episodic memory (1, 5, 6, 24). These theories posit that a gradually changing contextual state becomes associated with each experienced event, and that recalling an event revives the contextual state associated with the original experience. This retrieved context, in turn, activates other memories that were associated with similar contexts, producing the contiguity effect seen in recall tasks (Fig. 4B). The present findings provide critical neurobiological evidence in support of context reinstatement by showing that remembering an item reinstates the patterns of distributed oscillatory activity associated with surrounding (contextual) items from the original study episode. This neural signature of context reinstatement was observed both for the whole-brain analysis and for recordings taken only from the temporal lobe.

Retrieved context models are one of a broader class of episodic memory models providing insight into our finding that patterns of neural activity are reinstated prior to recall. For example, by rehearsal-based models, words are rehearsed after they are presented, and more recently presented words are more likely to be rehearsed than temporally distant words. Rehearsal-based models have been shown to be difficult to distinguish from context-based models (34, 35), likely because a context-based mechanism is necessary to explain the pattern of rehearsals made in a free-recall task. If associations are formed between items that are rehearsed successively, activating the representation of an item before recall would be expected to activate the representations of other items rehearsed nearby in time (consistent with the neural signature of context reinstatement we observe).

To assess the extent to which variability in rehearsal strategies across participants might explain the observed correlation between the neural and behavioral contiguity effects (Fig. 4C), we performed an analysis of the neural correlates of the primacy effect in our data. It has been well established that rehearsal is associated with enhanced recall for early list items [i.e., the primacy effect (36–40)]. Thus, if our basic findings were driven by rehearsal, one might expect that participants exhibiting strong neural contiguity should also show a strong primacy effect. We observed no significant correlation between primacy and neural contiguity (r = 0.13, P = 0.42; SI Materials and Methods, Analysis Methods and Fig. S2), suggesting that rehearsal during study, per se, is unlikely to account for our findings. Although rehearsal is one of the hypothesized mechanisms underlying primacy, we recognize that other factors, such as enhanced attention to early list items (39, 41), may also contribute to primacy. Nonetheless, it is clear that the mechanisms underlying the primacy effect are unrelated to the neural contiguity effect we observe.

Modern psychological and neuroscientific investigations are still grappling with basic questions regarding how the human brain establishes continuity in a rapidly changing environment, and how our memory system revives prior states of the world. Recent neurocomputational models of human memory (1, 5, 6) posit that continuity is provided by a context representation that changes gradually over time as a consequence of the integration of present and past events. The current state of context is assumed to become associated with each newly experienced event, such that reminders of the event retrieve the event's associated context. This notion is consistent with the contention of Tulving (2) that episodic memory retrieval is like mental time travel, in that when we remember the past, many details of the prior experience are retrieved along with the desired material. By showing that a component of the neural activity retrieved during memory search shows graded similarity to the brain states observed during the study of neighboring stimuli, we provide neural evidence for temporal context reinstatement in humans.

Materials and Methods

Participants.

We tested 69 patients with drug-resistant epilepsy who had arrays of subdural and/or depth electrodes surgically implanted for 1 to 4 wk to localize the sites of seizure onset (Table S1). The clinical team determined the placement of these electrodes with the goal of localizing suspected epileptogenic foci and identifying functional regions to be avoided in surgery (details on recording methods are provided in SI Materials and Methods, Recording Methods and Table S2). Our research protocol was approved by the appropriate institutional review boards, and informed consent was obtained from the participants and their guardians. Data were collected as part of a long-term multicenter study, with previously published articles describing separate analyses conducted on subsets of these data (19, 42, 43).

Behavioral Methods.

Participants studied lists of 15 or 20 high-frequency nouns for a delayed free recall task. Following a fixation cue, the computer displayed each word for 1,600 ms, followed by an 800- to 1,200-ms blank interstimulus interval. Each word was displayed at most once within a single testing session. For 18 s following list presentation, participants solved a series of single-digit addition problems of the form A + B + C = X. Participants were then given 45 s to recall list items in any order. Vocal responses, digitally recorded during the trial, were scored for subsequent analysis. Participants recalled 22.7 ± 1.0% (mean ± SEM) of the studied words. Repetitions and incorrect recalls (32.4 ± 2.6% of all responses) were excluded from our analyses, as were responses that occurred within 1 s of a prior vocalization.

Data Analysis.

We measured oscillatory power in the ECoG recordings by applying a Hilbert transform to the Butterworth band-passed signal in each of five frequency bands (Results). To reduce edge artifacts, we computed power at each frequency for the entire recording session before parsing the recordings into experimental events. Before applying PCA to the frequency × electrode matrices (Fig. 1B), we z-transformed power values relative to the distribution of all events in the recording session (the z-transformation was performed independently for each frequency-electrode pair). We used the Kaiser criterion to choose, for each participant, the principal components that explained a substantial proportion of the variance (44). We identified principal components that changed gradually during the study period (SI Materials and Methods, Analysis Methods and Fig. S3). Features that met these criteria were identified in 132 (of 144) recording sessions. We further excluded an additional 68 sessions in which fewer than five candidate context features were identified. This threshold was chosen to balance two factors. First, we wanted to ensure that the feature vectors were of high enough dimensionality that it would be possible to observe neural signatures of context reinstatement (Fig. 4A) for each participant. Second, we wanted to maximize the amount of data included in our analysis. We report the mean number of features selected for each participant in Table S1. To prevent selection bias, recall events were not used in the feature selection process.