Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2019 Mar 4.
Published in final edited form as: Curr Biol. 2017 Nov 9;27(22):3499–3504.e4. doi: 10.1016/j.cub.2017.09.057

Event Boundaries Trigger Rapid Memory Reinstatement of the Prior Events to Promote Their Representation in Long-Term Memory

Ignasi Sols 1,2, Sarah DuBrow 3, Lila Davachi 3, Lluís Fuentemilla 1,2,4,5,*
PMCID: PMC6398599  NIHMSID: NIHMS1009043  PMID: 29129536

SUMMARY

Although everyday experiences unfold continuously over time, shifts in context, or event boundaries, can influence how those events come to be represented in memory [1–4]. Specifically, mnemonic binding across sequential representations is more challenging at context shifts, such that successful temporal associations are more likely to be formed within than across contexts [1, 2, 5–9]. However, in order to preserve a subjective sense of continuity, it is important that the memory system bridge temporally adjacent events, even if they occur in seemingly distinct contexts. Here, we used pattern similarity analysis to scalp electroencephalographic (EEG) recordings during a sequential learning task [2, 3] in humans and showed that the detection of event boundaries triggered a rapid memory reinstatement of the just-encoded sequence episode. Memory reactivation was detected rapidly (~200–800 ms from the onset of the event boundary) and was specific to context shifts that were preceded by an event sequence with episodic content. Memory reinstatement was not observed during the sequential encoding of events within an episode, indicating that memory reactivation was induced specifically upon context shifts. Finally, the degree of neural similarity between neural responses elicited during sequence encoding and at event boundaries correlated positively with participants’ ability to later link across sequences of events, suggesting a critical role in binding temporally adjacent events in long-term memory. Current results shed light onto the neural mechanisms that promote episodic encoding not only for information within the event, but also, importantly, in the ability to link across events to create a memory representation of continuous experience.

In Brief

Event boundaries influence how experience is carved and represented in memory. Using scalp EEG pattern similarity analysis in humans, Sols et al. show that event boundaries trigger an online memory reinstatement of the just-encoded sequential episode that links adjacent events to create a long-term memory representation of continuous experience.

RESULTS AND DISCUSSION

The current study was designed to examine whether bridging across adjacent events may be supported by the reinstatement of the prior event at the start of a new event. This mechanism would be beneficial for several reasons. First, the rapid reactivation of a recent event could serve to promote the strengthening, or chunking, of that past event unit in memory. Second, the reactivation of an event contemporaneously with the experience of a subsequent adjacent event could theoretically result in the co-activation of the past and present events, promoting the binding of sequential events in their temporal order [10]. Thus, we tested the hypothesis that context shifts might trigger the rapid reinstatement of the just-encoded sequential episode.

In this study, we used a modified version of a task [2, 3] that required participants to encode a series of pictures that were arranged into “events” while recording their brain EEG signals (see STAR Methods), because EEG is ideally suited to measure with precision neural responses time-locked to sequences of stimuli that are close in time. Each list contained 34 object and face pictures that were arranged into sequential events (Figure 1A). Specifically, embedded in the encoding list there were sequences of seven different pictures from the same category (episodic condition) and sequences of seven repeated items (non-episodic condition), alternated with pairs or triads of same-category items, thus providing two experimental conditions with the same sequence length that differ in their degree of episodic content during encoding. Critically, the end of each sequence or “event” was always marked by the occurrence of a picture from the other category (e.g., a face event ends with the occurrence of an object picture and vice versa).

Figure 1. Task Design and Behavioral Results.

Figure 1.

Open in a new tab

(A) Participants were presented with sequences of objects and faces. There were two main conditions of interest: episodic (trains of same-category items) and non-episodic (repetitions of the same item, with the same length as the episodic condition). The first item after a category switch was a boundary item.

(B) During the recency discrimination task, participants were asked for the temporal order in which pairs of items appeared during the encoding phase. There were two main conditions of interest: lag-3 No-Switch condition, where the tested pair of items belonged to the same episodic event and were separated by three intervening items (therefore, there had been no contextual switch between them during the encoding phase), and the lag-3 Switch condition (there had been two contextual switches between the pair of items during the encoding phase).

(C) During the sequential memory test, participants were asked to recall the image category of the last item in an event sequence, cued either by the preceding item (within-episode) or by the boundary item (across-episode) following that image within that specific sequence.

(D) Participants showed better temporal order memory for items within the sequence, where there was no category switch.

(E) Participants’ memory for the category information of the pre-boundary item in the episodic condition.

(F) Participants’ accuracy in the across-episode condition of the sequential memory test predicted their temporal order memory accuracy in the No-Switch condition in the recency discrimination test. Solid line represents the line of best fit, and dashed lines represent 95% confidence intervals.

For all boxplots in (D) and (E), the central mark is the median, the edges of the box are the 25th and 75th percentiles, and the whiskers extend to the most extreme data points.*p < 0.05.

After each encoding list, we assessed several components of temporal memory. First, past studies examining recency discrimination on pairs of items within and across picture categories have demonstrated that category boundaries affect this discrimination ability [2, 3] (Figure 1B). We replicated those past findings by showing that temporal order memory was better preserved for items from the same event category (No-Switch condition) than those that were separated by category switches during encoding (Switch condition) (t(33) = 2.45, p = 0.01) (Figure 1D) and that this behavioral effect was independent of the participants’ confidence judgments (repeated-measures ANOVA, F(1,33) < 1). Critically, this effect was observed even though actual distance between the pairs of tested items was the same, indicating that shared picture category provides mnemonic support for the encoding of sequences after a single experience. Second, we tested for whether adjacent events had become successfully associated in memory, which was assessed by asking participants to recall the image category of the last item in the sequence cued by the boundary item following that image within that specific sequence (across-episode condition). In an attempt to assess temporal order memory for that sequence in the same test, we also asked the participants to recall the image category cued by the preceding item from that sequence (within-episode condition) (Figure 1C). Note that, however, the two conditions required participants to answer based on two different recall strategies, namely backward (across-episode) and forward (within-episode) recall, which could explain the behavioral performance differences between them [11]. Thus, in the context of the current study, although behavioral performance in each of these two conditions informs about participants’ abilities to link the last sequence item to either the following boundary item or the previous sequence item, differences between the two conditions should be taken cautiously, because they may not be attributed to similar temporal order memory processes. The results of this task showed that participants’ performance was higher in the within- than in the across-episode condition (t(33) = 3.89, p < 0.01) (Figure 1E). Interestingly, we found that, across participants, performance in the across-episode condition, but not performance in the within-episode condition, of the sequential memory test correlated positively with temporal order memory accuracy in the recency test (across-episode condition with No-Switch: r = 0.42, p = 0.014 [Figure 1F], and with Switch: r = 0.48, p = 0.006; within-episode condition with No-Switch: r = 0.33, p = 0.055, and with Switch condition: r = 0.17, p = 0.33). Altogether, these findings suggest that across-episode memory test was sensitive to temporal order memory processes underlying the preservation of sequence information within, as well as to adjacent episodes from the encoded sequence.

Next, we sought to investigate whether memory reinstatement of the just-encoded sequence is evident at context boundary images. To address this question, we adopted a spatiotemporal pattern similarity (STPS) analysis that queried the degree of similarity between two sets of EEG patterns [12]. Specifically, STPS was used to assess the degree to which EEG patterns elicited during the encoding of each image within a sequence correlated with EEG patterns elicited at the subsequent boundary item (see STAR Methods), in essence an index of memory reinstatement that occurs at the boundary between events. In the context of the current study, we examined STPS differences at boundary items preceded by episodic and non-episodic encoding conditions. This analysis revealed greater neural similarity for the episodic condition in frontal and posterior scalp regions in a time window from ~200 to ~800 ms after the boundary onset (Figure 2 and S2). Taken together, these results suggest that our findings do not generalize to all types of context novelty, but rather only to contextual shifts that follow episodic content.

Figure 2. Spatiotemporal Pattern Similarity (STPS) Results.

Figure 2.

Open in a new tab

(A) The 29 electrodes were grouped into six regions. To obtain more stable spatial patterns, we included the electrodes in the border between these regions.

(B) SPTS, expressed in averaged z values, between EEG neural activity time-locked to the onset of the boundary item and averaged EEG neural activity time-locked to the appearance of the previous episodic and non-episodic sequence items. The x axis represents time, and the y axis represents the spatial locations.

(C) The statistics of contrasting STPS in the episodic versus non-episodic conditions. White boxes mark the cluster showing significant effects (p < 0.05) after correction for multiple comparisons.

See also Figures S1 and S2.

An important issue that remained to be investigated is the extent to which the increase in neural similarity at the boundary item in the episodic condition was driven by the reinstatement of the full sequence, the reinstatement of a subset of items from the previously encoded sequence, or the reinstatement of the item presented immediately before the boundary. To address this issue, we extracted the averaged STPS from the spatiotemporal bins identified in the main analysis (i.e., Figure 2C) derived from each separate item from the sequence in the episodic condition and contrasted them with the corresponding ones for the non-episodic condition. A repeated-measures ANOVA, including condition (episodic versus non-episodic) and item order as within-subject factors, revealed a significant main effect of condition (F(1,33) = 10.33; p = 0.003) but no significant condition × item order interaction (F(6,198) = 0.65; p = 0.69) (Figures 3A and S3), indicating that STPS differences at item boundaries were explained by an overall increase in STPS between the boundary item and the entire sequence of items from the episodic condition.

Figure 3. STPS throughout Sequence Encoding and Correlation with Memory Accuracy.

Figure 3.

Open in a new tab

(A) Participants’ averaged STPS values over the similarity cluster between each item from the sequence and the boundary item in the episodic and in the non-episodic condition.

(B) Participants’ averaged STPS values over the similarity cluster averaged over different groups of items of the sequence. Results indicate that STPS values were similar throughout sequence encoding and only differed at the boundary item when comparing episodic and non-episodic encoding conditions.

(C) Mean cluster STPS in the episodic condition for the boundary item correlated significantly and positively with behavioral accuracy across participants in the test linking prior sequence information to the boundary item (extracted from the sequential memory test). Solid line represents the line of best fit, and dashed lines represent 95% confidence intervals.

In (A) and (B), error bars indicate SEM. *p < 0.05. See also Figure S3 and Table S1.

An alternative possibility is that this significant STPS effect, suggestive of picture category shift-induced memory reinstatement, is not unique to the boundary items but rather may reflect other encoding-dependent memory processes that are present throughout the encoding of picture sequences that contain similar information (e.g., [12]). To test whether this neural similarity effect was unique to the boundary item, we ran two separate analyses. First, we examined the extent of reinstatement during the fifth, sixth, and seventh items. Specifically, we quantified the degree of similarity between EEG patterns elicited in the first half of the sequence (i.e., first four pictures) and EEG activity in response to the following picture in the same sequence (i.e., the fifth picture). To track the extent to which neural similarity evolved as a function of sequence encoding, we repeated this analysis throughout the sixth and seventh items in the event sequence and then compared the results with those obtained at the boundary item. With the aim of providing a quantitative assessment of associative memory maintenance during sequence encoding in the episodic condition, we also ran the same analysis in the non-episodic condition. The results of this analysis confirmed that neural similarity only differed between the two conditions at the boundary item (repeated-measures ANOVA interaction number of items × condition: F(1,33) =3.93, p = 0.02; paired t test comparison between boundary items, t(33) = 3.75, p = 0.001) (Figure 3B). A paired t test confirmed that the interaction found in our previous analysis was explained by similarity changes at the boundary item in the episodic condition, because similarity at the boundary was greater when compared with the averaged similarity values in the fifth, sixth, and seventh items (t(33) = 2.32, p = 0.03). The same contrast was not significant in the non-episodic condition (t(33) = 1.34, p = 0.18). These findings suggest that memory reinstatement takes place specifically during event boundaries preceded by a sequence of items with episodic content. In addition, to further investigate the specificity of memory reinstatement at event boundaries in the episodic condition, we extracted the averaged STPS from the spatiotemporal bins identified in the main analysis between items 2–7 and the boundary item and compared it to the STPS obtained from correlating items 1–6 to item 7, only in the episodic condition. Notably, although this analysis is limited in terms of matching for serial position, it controls for possible image novelty processes inherent to the encoding of event boundaries in the episodic and in the non-episodic condition. In line with the previous findings, we found that STPS between items 2–7 and the boundary item were significantly higher than STPS derived from items 1–6 and item 7 (t(33) = 2.08; p = 0.045), thereby supporting the notion that the increase in neural similarity was specific to the event boundary in the episodic condition.

Next, we reasoned that if memory reactivation at event boundaries supported the binding of the new episode to the previously encoded episode, then neural similarity should be predictive of participants’ ability to link them behaviorally in the across-episode condition of the sequential memory test. Confirming this prediction, we found a highly significant positive correlation between these two measures (r = 0.46, p = 0.006; Figure 3C). In addition, we also found that the degree of neural similarity was also predictive of temporal order memory accuracy for items within the just-encoded event (measured by the No-Switch condition in the recency discrimination task: r = 0.45, p = 0.007). Notably, we did not find a significant correlation between the degree of neural similarity and participants’ memory accuracy in the Switch condition of the recency discrimination task (r = 0.17, p = 0.34), which may rely more on item strength than on sequence memory [2]. Thus, the correlation findings were specifically related to order memory within a context rather than general temporal order memory performance. As such, the degree of memory reinstatement at boundary items is likely a neural signature of the ability to form an integrated memory representation of adjacent events, which may also promote encoding the temporal order of the previous episode into long-term memory.

The current results support the notion that rapid memory reinstatement of the preceding sequence of events is elicited during context shifts to promote the formation of a bound memory representation across episodes. This reinstatement occurred during the first 1,000 ms after the boundary onset, which fits well with onset timing of memory formation in human hippocampal recordings [13, 14]. Instances of memory reactivation for past episodes have also been shown to occur very rapidly during recollection (at ~500 ms after stimulus onset), as measured by hippocampal recordings [15] and magnetoencephalographic recordings in humans [16]. Our findings, however, provide novel evidence that memory reactivation can also take place during online encoding of new information and that it may represent a very early stage of how sequential episodes are bound together into long-term memory.

The present results constitute the first experimental evidence in humans, bridging the gap with previous animal findings [17–20], that rapid, online memory reinstatement may be a critical neural mechanism supporting the long-term retention of just-encoded episodes. Previous human studies have shown neural signatures of memory persistence of just-encoded item pairs [21] and implicate the hippocampus in encoding individual episodes into memory following event offsets [22]. Our results offer novel insights into the neural mechanisms that promote episodic memory encoding not only for information within an event, but also, critically, in the ability to link across events to create a long-term memory representation of continuous experience.

Single-cell recordings from the rodent hippocampus during navigational tasks have shown that memory replay of episodic sequences preserves the temporal structure of the sequential information in the encoded episode [17–20]. Such sequential memory structure has been shown to be rapidly replayed either in a forward or in a backward manner, presumably underlying neural mechanisms related to future planning and memory formation or consolidation of recently encoded episodic information, respectively [19]. Our findings are blind, however, to whether memory reinstatement at event boundaries relies on memory replay of a temporally preserved structure of the encoded sequence (see Figure S3). Further attempts to investigate the sequential nature of memory replay at event boundaries should take into account the possibility that memory replay may be temporally compressed [19] and may be observed in brain signals arising from the gamma range (e.g., [23]).

An open question is which specific neural mechanisms trigger memory reinstatement at event boundaries. Event segmentation theory (EST) [1, 24] posits that a prediction error is driven by the mismatch between the prediction generated by the previous experience and the current input. According to EST, it is through such a prediction error that memory systems can separate two event episodes online. On the contrary, the integrated encoding mechanism suggested by Shohamy and Wagner [25], as well as the literature on inferential learning in humans [26], argues that a mismatch between the current input and the prediction derived from the previous experience triggers the memory systems to build a bound representation of the two episodes. In the context of the current study, therefore, one possibility is that memory reinstatement is triggered by this prediction error or mismatch signal and that, in doing so, the two events (previous and new) become linked for the long term.

Our findings show that neural similarity between EEG patterns elicited during episodic sequence and event boundary encoding were mostly located in frontal and posterior regions. Although EEG recordings from the scalp suffer from low resolution of the precise sources of brain activity, these regions fit well with previous fMRI studies in humans that found frontal and parietal regions that showed higher neural responses during successful temporal order memory judgements in sequential memory tasks [3, 27]. Recent fMRI findings have also shown a prominent role for the hippocampus in exhibiting memory-predictive activity time-locked to the offset of short, narrative audiovisual episodes [22], as well as being sensitive to event inputs that signaled transitions from previously learned sequences of pictures [28]. Thus, while we cannot be certain as to the neural sources that contributed to our findings derived from scalp EEG activity, the findings suggest that the detection of event boundaries during the encoding of novel episodic sequence information recruits specific mechanisms from a distributed neural network that reinstates the just-encoded event.

Memory reinstatement at event boundaries may also help to preserve a subjective sense of continuity, because this helps memory systems to bridge temporally adjacent events during encoding. Classic models of working memory [29] had already proposed the need to incorporate into the working memory system a mechanism that explains our ability to establish temporal continuity from the very recent past to the present, as well as to maintain and integrate complex incoming information into a unified experiential field. Previous work has shown that memory reinstatement could be a mechanism that supports working memory maintenance of complex relational information [30]. Thus, together with the current findings, it is tempting to speculate that memory reinstatement could also have a specific role in working memory by supporting a subjective sense of continuity that would resolve the computational tension given by the detection of an event boundary and the need to preserve a temporal order of the experience in long-term memory.

STAR★METHODS

CONTACT FOR REAGENT AND RESOURCE SHARING

Further information and requests should be directed to and will be fulfilled by the Lead Contact, Lluís Fuentemilla (llfuentemilla@ub.edu)

EXPERIMENTAL MODEL AND SUBJECT DETAILS

Forty-nine Spanish speakers (46 right-handed, 35 females, age range 18–27, mean = 22) participated for pay (10€/h). Participants were recruited from the University of Barcelona and the broader community. All participants were healthy and did not consume psychoactive substances. Informed consent was obtained from participants in accordance with procedures approved by the Ethics Committee of the University of Barcelona. Seven participants were excluded from further analysis due to the chance-level performance on the recency discrimination task (i.e., overall performance < 60%; binomial test). We set this criterion because pilot studies with this design indicated that the recency discrimination test was easier than the sequential memory test. Thus, this criterion allowed us to, at least, exclude participants with a lack of attention during the experiment. We additionally excluded two participants for excessive eye blinks during the task and six for technical problems during the EEG recording. Thus, the final sample of participants included in the study was thirty-four, which is similar to the sample size included in our previous studies using a similar task design [2, 3]).

METHOD DETAILS

Experimental Design

The experiment consisted of sixteen separate blocks, each one comprising an encoding phase that was immediately followed by a short distractor task, a recency discrimination task and a sequential memory test. Task timing and visual stimulus presentation were under the control of commercially available software (Presentation, Neurobehavioral Systems). Before starting the experiment, participants had to complete a shorter practice block. There were two stimulus categories, objects and faces: 224 photos of objects (taken from the Stark lab set of stimuli, freely available at (http://faculty.sites.uci.edu/starklab/mnemonic-similarity-task-mst/) and 224 photos of worldwide celebrities and local Spanish celebrities.

Encoding phase:

Each encoding phase consisted of a sequence of 34 items, consisting of faces and objects. We used different list structures adapted from the paradigm developed by DuBrow and Davachi [2, 3]. These lists included several same-category trains of items (i.e., seven different objects, or faces, one after the other; episodic condition), alternated with pairs or triads of same category items and the repetition of the same face or object item seven times consecutively (i.e., repeated condition). Each list contained seven boundary items, that is, items for which the category of the image, and thus the category-specific judgment, switched from the preceding trial (i.e., a face appearing after several objects). The lists structure, regarding item category, was the same for all participants, with face and object positions counterbalanced across lists. The order of specific item appearance was randomized for each participant. Participants were told to try to memorize the order of item appearance by using an associative encoding strategy based on making narratives with the items that appeared subsequently. Vivid imagery was also encouraged as an additional strategy. Each item was presented for 2500 ms and participants were asked to make a category-specific judgment at its offset for up to 1500 ms (big/small for objects relative to the computer screen, man/woman for faces). The inter-trial interval was 2000 ms. The encoding phase was followed immediately, without pause, by a 45 s odd-even distractor task that preceded the recency discrimination task and the sequential memory test detailed below.

Recency discrimination task.

All items appearing in the recency discrimination task of a given block had appeared immediately before, during the study phase of that block. Five item pairs were tested in each block. On each trial, the prompt “Which was more recent?” appeared below the pair of items. Participants were instructed to press a left or a right keyboard button with the index and middle finger of the right hand, depending on which image they thought was the more recent one, and had up to 8 s to answer. 300 ms later, a confidence probe appeared. Participants were then asked for their level of confidence in their answer (“very high, high, low”) and had up to 6 s to answer. A prompt requesting a faster answer appeared if the participants exceeded the stipulated timing, both in the recency and confidence trials. Responses made after the deadline were not included in behavioral analyses.

There were two main conditions of interest. In both conditions a pair of images was tested, either two faces or two objects. During the study phase, these images had always appeared separated by three intervening items (lag-3). In the first condition the three intervening items were of the same category train as the pair of tested items (lag-3 No-Switch). In the second condition, the pair of tested items had been separated during the encoding phase by two boundaries (lag-3 Switch). The study lists were designed so that the positions of the two conditions of interest in the sequences were counterbalanced across lists. There was a third condition in which the pair of tested items was an object and a face that had been presented across boundaries during the encoding phase (across category condition). This condition was added so as not to discourage sequential binding whenever there was a category switch during the study phase, which might otherwise occur if the items in each pair were always of the same category. The order in which the pairs of items appeared was randomized, as was the right-left position. Each condition of interest was tested twice in each block. The across category condition was tested one time in each block.

Sequential memory test.

All items appearing in the sequential memory test of a given block had appeared immediately before, during the study phase of that block. Participants were asked to indicate the category (face/object) of the pre-boundary item of the episodic condition (7th item of the sequence) for each study list. Category information for the same pre-boundary item was tested twice by cueing it with the boundary item and with the item that just preceded the pre-boundary item. The probe item always appeared in the center of the screen. The target item position was indicated with a white question mark on a red background. The trial presentation order was randomized for each sequence test.

EEG Recording

EEG was recorded using a 32-channel system at a sampling rate of 500 Hz, with an online band-pass filter set at 0.1–100 Hz, using a BrainAmp amplifier and tin electrodes mounted in an electrocap (Electro-Cap International) located at 29 standard positions Fp1/2, Fz, F7/8, F3/4, Fc1/2 Fc5/6, Cz, C3/4, T3/4, Cp1/2, Cp5/6, Pz, P3/4, T5/6, PO1/2, O1/2) and at the left and right mastoids. An electrode placed at the lateral outer canthus of the right eye served as an online reference. EEG was re-referenced offline to the average of all channels. Vertical eye movements were monitored with an electrode at the infraorbital ridge of the right eye. Electrode impedances were kept below 3 kΩ. EEG was low-pass filtered offline at 30 Hz. We applied the Parks-McClellan Notch filter using the toolbox ERPLAB (http://www.erpinfo.org/erplab).

QUANTIFICATION AND STATISTICAL ANALYSIS

Behavioral Data Analysis

The skewness of the behavioral data from the recency discrimination (No-Switch: mean = −0.09, STD = 0.40; Switch: mean = −0.001, STD = 0.40) and sequential memory task (across-episode: mean = −0.32, STD = 0.40; within-episode: mean = −0.50, STD= 0.40) indicated that they obey the normal distribution and therefore allowed the use of parametric testing such as Student’s t test to compare them statistically. Thus, a paired Student’s t test was used to compare participants’ performance (measured in percentage) between conditions in the recency discrimination task and in the sequential memory task. Repeated-measures ANOVA was used to statistically assess differences between participants’ performance when they included more than two variables. Statistical significance threshold was set at p < 0.05.

Participants’ Accuracy throughout the Task in the Sequential Memory Test

One possible concern is whether participants were using a non-mnemonic strategy to produce correct responses in the sequential memory task, as target items in this test were always from the same category when tested in the within-episode condition and were always from the other category when tested in the across-episode condition. Notably, if this was the case, we would expect participants’ accuracy to go from floor (i.e., 50% correct) to ceiling (i.e., 100%) in their task performance. To address this concern, we quantified the participant’s proportion of correct responses during the course of 4 consecutive blocks in the task (each including an average of 4 encoding runs) separately for the within and the across-episode condition. A repeated-measures ANOVA, including condition (2 levels: across-episode and within-episode) and block (4 levels) as within subject factors, confirmed these observations statistically, as it revealed a significant main effect of condition (F(1,33) = 20.08, p < 0.001) and a significant main effect of block (F(3,99) = 3.25, p = 0.025) but no significant interaction between the two (F(3,99) = 1.98, p = 0.12). All in all, these findings lend support against the possibility that participants used a no-mnemonic strategy to correctly respond to the sequential memory test. Additionally, a subsample (n = 17) of the participants were also asked, at the end of the experiment, to indicate whether they used any particular non-mnemonic strategy to respond either to “same” or “different” category in this test. None of them reported doing so. In addition, overall accuracy in the sequential memory test was well above chance (mean = 0.75, SD = 0.11, where chance would be 0.5) and participants’ accuracies were significantly above chance in both the across-episode (t(34) = 1.83, p = 0.038, one-sided) and in the within-episode (t(34) = 3.84, p < 0.001, one-sided) conditions. Furthermore, no differences were observed between the overall accuracy between the two picture categories neither in the within-episode condition (t(33) = 0.20, p = 0.841) nor in the across-episode condition of the sequential source memory test (t(33) = 0.81, p = 0.424).

EEG Data Analysis

The continuous sample EEG data were then epoched into 1500 ms segments (0 to 1500 ms relative to trial onset), and the pre-stimulus interval (−100 to 0 ms) was used as the baseline for the baseline removal procedure. For each participant, we obtained epoch trials that were separately cataloged as belonging to the two main conditions of interest: episodic and non-episodic sequence conditions. Thus, for each condition we extracted EEG epochs for items within the sequence, namely an EEG epoch for the 1st, 2nd, 3rd, 4th, 5th, 6th and 7th image. EEG epoch trials for boundary items following each of the sequences were also obtained. For each participant, EEG epochs for items of the same condition and sequential position were grouped into matrices, which were then used to run the representational similarity analysis detailed below.

Spatiotemporal Pattern Similarity Analysis (STPS)

Spatiotemporal vectors were constructed from the epoch data for each item of interest, adapting the approach developed by Lu et al. [12] and implemented in MATLAB (Mathworks). The spatial features were scalp voltages from one of the six regions for better spatial specificity (Figure 2A), and the temporal features were selected using a 100 ms (50 time points) sliding window from the epoched EEG data. Specifically, EEG epochs included in the analysis were 1500 ms long starting at the onset of the item picture. This resulted in a 751 time-point vector (given the 500Hz EEG recording sampling rate) for each trial. Then, data was smoothed by averaging each time point with the next 50 time points (100 ms). The averaging stopped at sample point 701 given that it was the last one that could include averaged data from the next 50 time points. Therefore, although our feature vector had 701 time points, equivalent to 1400 ms, it resulted from EEG data from up to 1500 ms after stimulus onset. Finally, we grouped the data into 20 ms bins, resulting in the 70 time points. In order to have bins of the same size, we did not include the last of the 701 time points. Then, a vector containing both the spatial and temporal features was formed for each trial. Similarity between trials was calculated using Pearson correlation coefficients. The correlation coefficients were then converted to Fisher’s z scores for subsequent statistical analysis. See Figure S1 for an illustrative schematic of the STPS analysis.

Individual STPS vectors for the episodic and non-episodic sequence conditions were calculated by first correlating EEG epoch data for each item (1st, 2nd, 3rd, 4th, 5th, 6th and 7th) separately with the EEG epoch data for the corresponding boundary item and then averaging across correlation values for each condition and individual. For each condition, STPS was calculated separately for sequences of objects and faces, and then correlations were averaged. For each spatial region and EEG epoch time bin of the boundary item, we averaged the correlation values obtained across the 70 EEG epoch time bins from the image sequence, thereby resulting in a matrix in which spatial and temporal information was preserved at the boundary item level. Paired t test (two tailed) was used to statistically assess differences between conditions.

Nonparametric Cluster-Based Permutation Test

To correct for multiple-comparison problems that may potentially result in false positive results in this study, we employed a nonparametric statistical method based on cluster-level randomization testing to control the family-wise error rate [31]. This method is implemented in the open source software Fieldtrip (http://www.fieldtriptoolbox.org/). For the STPS analysis, statistics were computed for every time point, and the time points whose statistical values were larger than a threshold (p = 0.05) were selected and clustered into connected sets on the basis of temporal and spatial adjacency. The observed cluster-level statistics were calculated by taking the sum of the statistical values within a cluster. Then, condition labels were permuted 10,000 times based on their exchangeability to simulate the null hypothesis and the maximum cluster statistics over all six regions were chosen to construct a distribution of the cluster-level statistics under the null hypothesis. The nonparametric statistical test was obtained by calculating the proportion of randomized test statistics that exceeded the observed cluster-level statistics.

Event-Related Potentials (ERPs) to Sequence and Boundary Items

ERPs were investigated starting at 100 ms before each item to 1400 ms after each item’s onset. The analysis of sequence items focused on the P1 (a positive deflection, approximately 100 ms post-stimulus onset), which reflects early stages of visual processing and has previously been shown to be modulated by different types of attention [32–34]. Attentional modulation at the P1 is apparent in parietal and occipital EEG regions. The analysis of ERP to boundary items centered on the P3 ERP component, which has been shown to provide a suitable neural coding response for the degree of unexpectedness or surprise of an event [35, 36]. P3 was obtained by averaging amplitudes within a window of 400–800 ms from item boundary onset and is maximally represented in parietal EEG regions. ERP trials exceeding 100 μV/s were automatically rejected offline. ERPs were then averaged separately for each individual and condition. The resulting ERPs were then averaged from sensors included in regions 5 and 6 (parietal and occipital EEG regions). The analysis of the P1 amplitude showed a main effect of encoding condition (F(1,33) = 4.91, p = 0.04), thereby indicating P1 amplitudes were higher in the episodic than in the non-episodic condition. However, we did not find any significant P1 amplitude changes across item sequence positions (F(6,198) = 0.94, p = 0.48) nor any significant interaction between P1 across items and encoding condition (F(6,148) = 0.21, p = 0.97), suggesting that the successive presentation of same-category items — either repetitions of the same item, or different items — did not elicit different patterns of neural suppression during sequence encoding. The analysis of the P3 amplitude revealed that it was greater for the boundary item of the Non-episodic condition than for the boundary item of the episodic condition (t(33) = 4.51, p < 0.001).

DATA AND SOFTWARE AVAILABILITY

All MATLAB scripts, and pre-processing scripts for analysis of the EEG data, are available upon request by contacting the Lead Contact, Lluís Fuentemilla (llfuentemilla@ub.edu).

Supplementary Material

Supplemental Information

KEY RESOURCES TABLE

REAGENT or RESOURCE SOURCE IDENTIFIER
Software and Algorithms
MATLAB Mathworks https://es.mathworks.com/
SPSS Statistics 21 IBM https://www.ibm.com/analytics/us/en/technology/spss/
Presentation Neurobehavioral Systems https://www.neurobs.com/
Open in a new tab

Highlights.

  • We used EEG pattern similarity to study physiological mechanisms at event boundaries

  • Event boundaries trigger the reinstatement of the just-encoded episode

  • Pattern similarity at boundaries correlates with temporal order memory

ACKNOWLEDGMENTS

This research study was supported by grants to L.D. (NIH RO1 MH074692) and to L.F. (PSI2013–46057-P and PSI2016–80489-P).

Footnotes

SUPPLEMENTAL INFORMATION

Supplemental Information includes three figures and one table and can be found with this article online at https://doi.org/10.1016/j.cub.2017.09.057.

REFERENCES

  • 1.Zacks JM, Speer NK, Swallow KM, Braver TS, and Reynolds JR (2007). Event perception: a mind-brain perspective. Psychol. Bull 133, 273–293. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.DuBrow S, and Davachi L (2013). The influence of context boundaries on memory for the sequential order of events. J. Exp. Psychol. Gen 142, 1277–1286. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.DuBrow S, and Davachi L (2014). Temporal memory is shaped by encoding stability and intervening item reactivation. J. Neurosci 34, 13998–14005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Polyn SM, Norman KA, and Kahana MJ (2009). A context maintenance and retrieval model of organizational processes in free recall. Psychol. Rev 116, 129–156. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Ezzyat Y, and Davachi L (2011). What constitutes an episode in episodic memory? Psychol. Sci 22, 243–252. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Ezzyat Y, and Davachi L (2014). Similarity breeds proximity: pattern similarity within and across contexts is related to later mnemonic judgments of temporal proximity. Neuron 81, 1179–1189. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Zwaan RA (1996). Processing narrative time shifts. J. Exp. Psychol. Learn. Mem. Cogn 22, 1196–1207. [Google Scholar]
  • 8.Heusser AC, Ezzyat Y, Shiff I & Davachi L Perceptual boundaries cause mnemonic trade-offs between local boundary processing and across-trial associative binding. PsyArXiv, 10.17605/OSF.IO/Z3TSD. https://psyarxiv.com/z3tsd. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Davachi L, and DuBrow S (2015). How the hippocampus preserves order: the role of prediction and context. Trends Cogn. Sci 19, 92–99. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Jensen O, and Lisman JE (2005). Hippocampal sequence-encoding driven by a cortical multi-item working memory buffer. Trends Neurosci. 28, 67–72. [DOI] [PubMed] [Google Scholar]
  • 11.Howard MW, and Kahana MJ (2002). A distributed representation of temporal context. J. Math. Psychol 46, 269–299. [Google Scholar]
  • 12.Lu Y, Wang C, Chen C, and Xue G (2015). Spatiotemporal neural pattern similarity supports episodic memory. Curr. Biol 25, 780–785. [DOI] [PubMed] [Google Scholar]
  • 13.Fernández G, Effern A, Grunwald T, Pezer N, Lehnertz K, Dümpelmann M, Van Roost D, and Elger CE (1999). Real-time tracking of memory formation in the human rhinal cortex and hippocampus. Science 285, 1582–1585. [DOI] [PubMed] [Google Scholar]
  • 14.Fell J, Klaver P, Lehnertz K, Grunwald T, Schaller C, Elger CE, and Fernández G (2001). Human memory formation is accompanied by rhinal-hippocampal coupling and decoupling. Nat. Neurosci 4, 1259–1264. [DOI] [PubMed] [Google Scholar]
  • 15.Staresina BP, Michelmann S, Bonnefond M, Jensen O, Axmacher N, and Fell J (2016). Hippocampal pattern completion is linked to gamma power increases and alpha power decreases during recollection. eLife 5, e17397. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Jafarpour A, Fuentemilla L, Horner AJ, Penny W, and Duzel E (2014). Replay of very early encoding representations during recollection.J. Neurosci 34, 242–248. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Foster DJ, and Wilson MA (2006). Reverse replay of behavioural sequences in hippocampal place cells during the awake state. Nature 440, 680–683. [DOI] [PubMed] [Google Scholar]
  • 18.Diba K, and Buzsáki G (2007). Forward and reverse hippocampal place-cell sequences during ripples. Nat. Neurosci 10, 1241–1242. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Carr MF, Jadhav SP, and Frank LM (2011). Hippocampal replay in the awake state: a potential substrate for memory consolidation and retrieval. Nat. Neurosci 14, 147–153. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Karlsson MP, and Frank LM (2008). Network dynamics underlying the formation of sparse, informative representations in the hippocampus. J. Neurosci 28, 14271–14281. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Tambini A, and Davachi L (2013). Persistence of hippocampal multi-voxel patterns into postencoding rest is related to memory. Proc. Natl. Acad. Sci. USA 110, 19591–19596. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Ben-Yakov A, and Dudai Y (2011). Constructing realistic engrams: post-stimulus activity of hippocampus and dorsal striatum predicts subsequent episodic memory. J. Neurosci 31, 9032–9042. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Heusser AC, Poeppel D, Ezzyat Y, and Davachi L (2016). Episodic sequence memory is supported by a theta-gamma phase code. Nat. Neurosci 19, 1374–1380. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Kurby CA, and Zacks JM (2008). Segmentation in the perception and memory of events. Trends Cogn. Sci 12, 72–79. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Shohamy D, and Wagner AD (2008). Integrating memories in the human brain: hippocampal-midbrain encoding of overlapping events. Neuron 60, 378–389. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Schlichting ML, and Preston AR (2015). Memory integration: neural mechanisms and implications for behavior. Curr. Opin. Behav. Sci 1, 1–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Jenkins LJ, and Ranganath C (2010). Prefrontal and medial temporal lobe activity at encoding predicts temporal context memory. J. Neurosci 30, 15558–15565. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Hsieh LT, Gruber MJ, Jenkins LJ, and Ranganath C (2014). Hippocampal activity patterns carry information about objects in temporal context. Neuron 81, 1165–1178. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Baddeley A (2000). The episodic buffer: a new component of working memory? Trends Cogn. Sci 4, 417–423. [DOI] [PubMed] [Google Scholar]
  • 30.Fuentemilla L, Penny WD, Cashdollar N, Bunzeck N, and Düzel E (2010). Theta-coupled periodic replay in working memory. Curr. Biol 20, 606–612. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Maris E, and Oostenveld R (2007). Nonparametric statistical testing of EEG- and MEG-data. J. Neurosci. Methods 164, 177–190. [DOI] [PubMed] [Google Scholar]
  • 32.Rugg MD, Milner AD, Lines CR, and Phalp R (1987). Modulation of visual event-related potentials by spatial and non-spatial visual selective attention. Neuropsychologia 25 (1A), 85–96. [DOI] [PubMed] [Google Scholar]
  • 33.Hillyard SA, Vogel EK, and Luck SJ (1998). Sensory gain control (amplification) as a mechanism of selective attention: electrophysiological and neuroimaging evidence. Philos. Trans. R. Soc. Lond. B Biol. Sci 353, 1257–1270. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Valdes-Sosa M, Bobes MA, Rodriguez V, and Pinilla T (1998). Switching attention without shifting the spotlight object-based attentional modulation of brain potentials. J. Cogn. Neurosci 10, 137–151. [DOI] [PubMed] [Google Scholar]
  • 35.Duncan-Johnson CC, and Donchin E (1977). On quantifying surprise: the variation of event-related potentials with subjective probability. Psychophysiology 14, 456–467. [DOI] [PubMed] [Google Scholar]
  • 36.Fuentemilla L, Cucurell D, Marco-Pallarés J, Guitart-Masip M, Morís J, and Rodríguez-Fornells A (2013). Electrophysiological correlates of anticipating improbable but desired events. Neuroimage 78, 135–144. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplemental Information
NIHMS1009043-supplement-Supplemental_Information.pdf (465KB, pdf)

Data Availability Statement

All MATLAB scripts, and pre-processing scripts for analysis of the EEG data, are available upon request by contacting the Lead Contact, Lluís Fuentemilla (llfuentemilla@ub.edu).

RESOURCES