Episodic sequence memory is supported by a theta-gamma phase code

Abstract

The meaning we derive from our experiences is not a simple static extraction of the elements, but is largely based on the order in which those elements occur. Models propose that sequence encoding is supported by interactions between high and low frequency oscillations, such that elements within an experience are represented by neural cell assemblies firing at higher frequencies (i.e. gamma) and sequential order is coded by the specific timing of firing with respect to a lower frequency oscillation (i.e. theta). During episodic sequence memory formation in humans, we provide evidence that items in different sequence positions exhibit relatively greater gamma power along distinct phases of a theta oscillation. Furthermore, this segregation is related to successful temporal order memory. These results provide compelling evidence that memory for order, a core component of an episodic memory, capitalizes on the ubiquitous physiological mechanism of theta-gamma phase-amplitude coupling.

INTRODUCTION

While many aspects of our cognition and behavior, including language processing, spatial navigation and episodic memory, share the requirement to represent, encode and retrieve temporal sequences of events, how the brain accomplishes this is still not well understood. It has been known for decades that stimuli that activate interconnected neurons can result in long-term potentiation (LTP) of the bridging synapse ¹, thus supporting an association between the two, and compelling new research has provided causal evidence for a link between LTP and associative memory formation ². However, the model of long-term potentiation in its current form is not viable for stimuli (or neurons) whose activation is separated by more than ~300 ms, and unarguably, much of what we encode and remember is separated by temporal gaps at least an order of magnitude larger than this. Thus, a central, unresolved issue is how the brain can bridge and relate stimuli encountered across seconds or minutes.

To deal with this conundrum, mechanistic models of sequence encoding posit that temporal coding of sequences can be supported by neural oscillations ^3–7, or rhythmic fluctuations in neuronal excitability (see ⁸ for review). One influential model of sequence encoding ^3,5 hypothesizes that individual items represented by largely non-overlapping neural cell assemblies, when activated, fire in higher frequency bands (i.e. gamma: >30 Hz), while the sequential order of those items is coded in a temporally segregated manner along the phase of an underlying slower rhythm (i.e. theta: ~3–8 Hz). This ‘phase coding’ model posits that during sequence encoding, the current item is represented and encoded by transient higher frequency activity, while the relative position of each item in a sequence may be coded by the relative phase of a lower frequency oscillation. Theoretically, phase coding allows the temporal segregation of activity supporting individual items that are encountered at different times across an experience, and critically may also permit temporally extended experiences to be represented in a time-compressed manner ⁶.

There is ample evidence that modulation of gamma power by theta phase (i.e. phase-amplitude coupling or ‘PAC’) is important for learning and memory ^9–13. However, to date, there is little evidence that theta phase coding is a mechanism underlying temporal aspects of human memory formation. We set out to test this fundamental question by presenting participants with six-item sequences each consisting of pictures of trial-unique objects that were embedded on a repeating background colored frame (Fig. 1a) while recording brain activity using magnetoencephalography (MEG). We later tested participants’ ability to recover temporal details of the presented sequence from memory. Thus, we could ask whether gamma power associated with each item in a sequence (positions 1–6) is biased towards distinct phases of theta, and whether theta phase coding is behaviorally relevant by examining it during both successful and unsuccessful temporal order encoding.

Figure 1.

Theta-gamma model fitting analysis. (a) Schematic of the paradigm. Participants viewed a series of trial-unique objects (36 per block) embedded on a colored frame which periodically switched every 6 trials. After each block, temporal order memory was probed by presenting two items studied within the same color and asking which of the two occurred earlier in the sequence. (b) Model of theta-gamma phase coding hypothesis. In this model, items (represented in gamma) encountered in the same color are concatenated along the theta phase. At switches in color, item representations are hypothesized to be removed. (c) Expected pattern of theta-gamma coupling measure (MI: modulation index) across sequence positions derived from simulated hypothesis. (d) Group-level topographic statistical map (n=17, t-values thresholded at t(16)>2.1, p<.05, cluster corrected using bootstrapping procedure) representing fit of model to across-trial theta-gamma coupling estimates. Two significant clusters of sensors emerged (p<.05, cluster corrected using permutation procedure): 1) a left lateralized cluster and 2) a left posterior cluster. (e) Group-level source space statistical map (t-values) representing fit of model to across-trial theta-gamma coupling estimates. Coronal (left), axial (middle) and sagittal (right) views are shown (n=17, source space statistical map is thresholded at t(16)>3.22, p<.005). uncorrected. PHG = parahippocampal gyrus. FG = fusiform gyrus.

RESULTS

Sequence memory performance

During each encoding-retrieval block, participants encoded six 6-item sequences (for a total of 36 consecutive object stimuli) before being tested for the temporal order of pairs of object stimuli from each presented sequence. Behaviorally, temporal order memory for pairs of object stimuli studied within a sequence was well above chance (mean=.74, SD=.12, chance was .5) but still in a range that allowed us to compare successful to unsuccessful encoding of sequences.

Theta and gamma power and coupling during sequence encoding

Before testing the critical hypotheses concerning whether theta-gamma interactions are related to successful sequence memory formation, we first characterized the distribution of spectral power in the MEG data in a broad range of frequency bands (1–100 Hz) during stimulus encoding (0 to 2.5 seconds), averaging power over time, trials, sensors, and subjects. This provides a global measure of the frequency content of the signal and allows identification of reliable peaks in the power spectra, since spectral peaks are necessary for a meaningful estimate of phase - and ultimately a reliable estimate of cross-frequency coupling ¹⁴. We found distinct peaks in both the theta (3–8 Hz) and high gamma (70–100 Hz) bands during each trial presentation compared to a baseline period (Supplementary Fig. 1). Using these spectral power peaks to constrain frequencies of interest, we tested whether any reliable relationship between theta and gamma oscillations were present in our data (see Methods for details). We identified significant theta-gamma PAC in a number of MEG sensors distributed across the scalp (One-sample t-test: thresholded at t>3.96, p<.001, Supplementary Fig. 2).

Modulation of theta-gamma coupling by sequence position

Next, we asked whether theta-gamma PAC was modulated by sequence position. If items are temporally coded by gamma power biased towards distinct phases of theta, then theta-gamma PAC may be parametrically modulated as a function of an item’s position within a sequence. We hypothesized that an item in the initial part of the sequence should be associated with a tight theta phase - gamma amplitude relationship because a single item would be represented at a singular phase on repeating cycles of theta ^4,6. However, as subsequent items are encoded, the addition of gamma cycles (representing additional items) would result in the widening of the distribution of gamma power over the phase of theta (see Fig. 1b for schematic of the hypothesized modulation by position). This would result in an overall reduction in our measure of theta-gamma coupling as more items are added into the sequence. To quantify this hypothesis, we ran a simulation where for each trial, additional gamma cycles (representing additional items in the sequence) were concatenated along the phase of a theta oscillation. PAC was then estimated on the simulated data. This verified our intuition that if our hypothesis is correct, our PAC measure should linearly decrease across trials when estimated separately for each trial within a sequence (Fig. 1c).

Based on the results of the simulation, we examined the MEG data for this pattern of decreasing theta-gamma PAC across each 6-item sequence. Then, using the pattern estimated by our simulated hypothesis as a predictor variable (Fig. 1c), we performed a linear regression on the actual PAC estimates to identify sensors that displayed the predicted pattern of results. This analysis identified two clusters of sensors (left lateral and posterior) that reliably fit our model at the group-level (Fig. 1d; One-sample t-test: thresholded at t(16)>2.1, p<.05; cluster-corrected using permutation test procedure, see Methods). To verify that this result was driven by our expected pattern, we extracted PAC estimates from sensors showing a significant fit to our model (p<.05, cluster corrected using permutation procedure). Visual inspection confirms a linearly decreasing PAC across the sequence, consistent with the simulation (Supplementary Fig. 3). Furthermore, regressing out the power values (both theta and gamma) had virtually no effect on the PAC model fits (see Supplementary Fig. 4; Two-way ANOVA with repeated measures: F(1,5332)=.002, p=.96) and thus do not interact with the PAC effects. In summary, we found that theta-gamma PAC decreased across items of a sequence, consistent with the idea that sequence encoding may be supported by the concatenation of gamma cycles at distinct, consecutive phases of theta for each additional item in the sequence.

Source localization of theta-gamma coupling effects

Theoretical models ^4,15 and empirical research in humans ^16–21 and rodents ^22,23 all point towards the hippocampus as being a structure that is critical for the formation of temporal associations among items. In humans, fMRI and intracranial EEG recordings with a spatial resolution on the order of millimeters have allowed for the precise characterization of hippocampal signals. While MEG is well suited for fine temporal analysis of brain activity, recent advances in source localization methods have made it possible to localize sources of MEG activity to a spatial resolution on the order of single centimeters, or even millimeters ²⁴. Additionally, a growing number of papers ^25–27 and simulations studies ^28,29 argue that source localization of MEG signals is possible to subcortical structures such as the hippocampus. However the parameters required for accurate source localization are still actively debated in the literature. Nonetheless, we applied best practice and performed source localization on our MEG data. We estimated trial-level time series in source space using a linearly constrained minimum variance beamformer approach (see Methods). Then, we computed PAC estimates for each trial and each source-space point (whole brain) and then fit the trial-wise PAC estimates to the model generated by our simulation (i.e. decreasing PAC estimates across a 6-item sequence). This analysis was nearly identical to the sensor-level analysis, but performed on source-space time-series estimates throughout the whole brain. Remarkably, the only region to reliably emerge from this analysis was a region centered in the left hippocampus (Fig. 1e, One-sample t-test: thresholded at t>3.19, p<.005; see Supplementary Fig. 5 for other thresholds) and extending posteriorly to a region of the parahippocampal and fusiform gyrus. This is consistent with the suspected role of the hippocampus and surrounding medial temporal lobe cortical regions in sequence encoding ^20,30 and with the idea that sequence memory may be supported by an interaction between theta and gamma activity in the left hippocampus.

Theta phase coding supports temporal sequence encoding

While the sensor-level and source-space analyses described thus far establish that theta-gamma PAC is modulated by sequential position in a manner consistent with a theta-gamma phase code, they do not directly demonstrate that result. Indeed, one could imagine alternative scenarios where the decreasing theta-gamma PAC pattern across sequence positions could emerge - a simple example being that the response pattern represents accumulating item representations without the orderly representation of their temporal order.

Thus, we conducted analyses to directly test our central hypothesis that objects encountered in different positions within a sequence are coded at distinct and consecutive phases of theta. To that end, we extracted raw MEG data from the two clusters of sensors that displayed a significant fit to the model (p<.05, cluster corrected using permutation procedure) derived from our simulated data (Fig. 1d). Separately for these two clusters of sensors, we binned gamma power by the phase of theta (18 phase bins) to generate an individual histogram for each trial (see Methods for details). We then averaged the theta-binned gamma distributions for each cluster across all subjects, but separately for each position and by subsequent temporal order memory.

Examining successfully encoded sequences first, in the left posterior cluster of sensors, we found a main effect of position on the distribution of gamma power over theta phase (Watson-Williams Test: F(5,96)=16.04, p=1.81e-11) and, critically, the gamma power distributions reflected the relative order in which the objects were encoded (Fig. 2a; Supplementary Fig. 6). By contrast, for sequences where order memory was later incorrect, we observed significant differences in mean phase angle by position (Watson-Williams Test: F(5,96)=5.26, p=1.35e-04), but the order of the phase angles was scrambled relative to the actual order that the items were experienced (Fig. 2b). Critically, the memory by position interaction was also significant (Harrison-Kanji Test: F(5,196)=15.79, p=.01). Together, these results suggest that objects in distinct ordinal positions within a sequence may be coded in gamma band activity biased towards distinct, and ordered, phases of a theta oscillation.

Figure 2.

Phase analysis of theta-gamma coupling during sequence encoding plotted by position and subsequent temporal order memory for left posterior cluster of sensors. (a) Theta-binned gamma distributions during successful sequence encoding. Each color represents a distinct sequence position and the number above the bin represents the sequence position with the highest gamma power at that bin. The inset shows the relative pattern over sequence positions for a single phase bin. (b) Theta-binned gamma distributions during unsuccessful sequence encoding. (c) Mean angle of theta-gamma coupling for each sequence position for successful sequence encoding. The main effect (i.e. strong bias of gamma at all positions to the trough of theta) is removed to highlight the relative phase biases. (d) Mean angle of theta-gamma coupling for each sequence position for unsuccessful sequence encoding.

Another way to visualize this effect is to plot the phase biases in polar coordinates. Thus, we transformed the theta-binned gamma distributions into polar coordinates (after removing the main effect of coupling across positions). We see that, for successfully encoded sequences, the order of the angles across positions mirrors the order in which the objects were encoded (Fig. 2c), whereas for incorrect sequences, the order of the angles is scrambled relative to the actual encoding order (Fig. 2d).

We next averaged the theta-binned gamma distributions for the 6-sequence positions into 3 bins (1&2, 3&4, 5&6) after subtracting out the main effect of coupling. A statistical contrast of each position bin relative to the average of the other two bins shows that gamma power was preferentially higher at distinct phase bins of theta (Fig. 3b; Paired-samples t-tests: early > middle and late time bins: t1: t(16)=2.38, p=.01, t2: t(16)=3.00, p=.004; middle>early and late time bins: t1: t(16)=3.04, p=.004, t2: t(16)=2.16, p=.023; late>early and middle time bins: t1: t(16)=2.58, p=.01, t2: t(16)=4.28, p=.0003). However, for sequences where the temporal order was later incorrectly remembered, the relative distributions for each position bin were not different from one another (Fig. 3c; t’s > 2.1, p’s < .05). Together, these data strongly support the idea that successful sequence encoding is accompanied by a theta-gamma phase coding mechanism, whereby gamma power associated with each sequential item is biased toward a distinct, consecutive phase of an underlying theta oscillation.

Figure 3.

Relative biases in gamma power over theta phase by sequence position and subsequent memory. (a) Schematic representing our binning strategy. We binned trials into early (1&2), middle (3&4), and late (5&6). (b) The distribution of gamma power over theta phase after removing the main effect of coupling for each sequence bin when temporal order was correct (paired-samples t-test, n=17: early > middle and late time bins: t1: t(16)=2.38, p=.01, t2: t(16)=3.00, p=.004; middle>early and late time bins: t1: t(16)=3.04, p=.004, t2: t(16)=2.16, p=.023; late>early and middle time bins: t1: t(16)=2.58, p=.01, t2: t(16)=4.28, p=.0003; t1=first significant time bin; t2=second significant time bin). (c) The same as B, but for sequences where order was incorrect. Stars represent phase bins where the statistical contrast (for example, early vs. the average of middle and late) was significant at p<.05. Error bars indicate standard error of the mean.

Ruling out alternative explanations

While the results reported here are consistent with a theta phase coding account of sequence memory formation, it is important to rule out alternative explanations for the data. Below, we outline a number of control analyses aimed to test whether other features of the data could explain the effects.

Temporal dynamics of gamma power and theta phase locking

We considered the possibility that variance in gamma power or theta phase locking across sequence positions or memory conditions could explain the phase coding effects. To test this, we analyzed the time course of gamma power and theta phase locking separately by sequence position (binned 1&2, 3&4 and 5&6) and by subsequent memory in the left posterior cluster of interest that displayed phase coding. Transient stimulus-evoked gamma power and theta phase locking stabilize by 500 ms post stimulus onset (Supplementary Figs. 7 and 8). There were no differences in theta phase locking by position (One-way ANOVA with repeated measures: 0–500 ms: F(2,34)=.78, p=.46, 500–2500 ms: F(2,34)=.5, p=.61) and no differences by subsequent memory (Two-way ANOVA with repeated measures: 0–500 ms: F(1,34)=1.17, p=.29; 500–2500 ms: F(1,34)=.04, p=.96). Gamma power also did not significantly vary by sequence position (One-way ANOVA with repeated measures: 0–500 ms: F(2,34)=.24, p=.78; 500–2500ms: F(2,34)=.13, p=.87) and there was a trend for greater gamma power in the forgotten sequences compared to the remembered sequences early (0–500 ms) but not later (500–2500 ms) (Two-way ANOVA with repeated measures: 0–500 ms: F(1,34)=3.12, p=.09; 500–2500 ms: F(1,34)=.95, p=.34). It’s important to highlight that absolute gamma power likely did not play a role in the phase coding effects we see here, because for each trial, after computing the distribution of gamma power over theta phase, the resulting distribution was normalized (to sum to 1). This minimizes the likelihood that the phase coding effects were somehow driven by gamma power differences across conditions.

Removing the stimulus evoked response

One major concern when performing cross-frequency coupling analyses is that the neural response to the presentation of a stimulus could simultaneously result in a burst of high-frequency activity and low frequency phase alignment over trials. Thus, an effect that looks like a phase-amplitude interaction may actually be driven by two independent processes with a common driver. It is worth noting that a burst of high-frequency activity at stimulus onset would need also be accompanied by (1) a predictable shift in theta phase across each position in a sequence and (2) only do so on successfully encoded sequences to fully account for our results. Nonetheless, to rule out the possibility that our effects are in some way driven by an evoked stimulus response, we reanalyzed the data without the time window (first 500 ms) containing the evoked potential.

We found that both the decreasing PAC over sequence positions (Supplementary Fig. 9; One-sample t-test: including first 500ms: t(16)=6.99, p= 3.07e-06, excluding first 500ms: t(16)=6.10, p= 1.53e-05), as well as the phase coding by subsequent order memory effects (Supplementary Fig. 10; excluding first 500ms - temporal order correct: Watson William’s Test: F(5,96)=9.10, p=1.05e-05; temporal order incorrect: Watson William’s Test F(5,96)=6.42, p=2.50e-02; memory by sequence position interaction: Harrison-Kanji Test: F(5,196)=11.03, p=.025) were still present and statistically robust.

Phase coding in the inter-trial interval

As a further test to rule out an evoked response confound, we ran the same analyses on the time periods between each trial, the inter-trial intervals (ITI; 2500–5000 ms post stimulus onset), when participants were presumably maintaining prior items and reinforcing associations between items. Interestingly, we found that theta-gamma PAC was in fact stronger during the ITI than the stimulus presentation interval (see Supplementary Fig. 11a) and the phase coding by subsequent memory effects were still present and robust (temporal order correct: Watson William’s Test F(5,96)=9.10, p=1.07e-05; temporal order incorrect: Watson William’s Test F(5,96)=9.09, p=4.03e-04; position x memory interaction: Harrison-Kanji Test: F(5,196)=8.02, p=.07). However, the decreasing PAC by sequence position effect was no longer present (Supplementary Fig. 11b). This very intriguing result highlights that phase coding is persistent even after the stimulus was removed from the screen, thus ruling out the possibility that the evoked response is in someway responsible for the phase coding pattern we observed.

Theta phase shift and phase symmetry controls

Finally, we examined whether systematic shifts in theta phase by sequence position or systematic changes in theta phase symmetry may contribute to our findings. The results do not reveal any evidence for a systematic theta phase shift across sequences (One-way ANOVA with repeated measures: F(4,84)=.33, p>=.85; Supplementary Fig. 12) and theta waveforms were equally symmetric over sequence positions (Watson-Williams test: F(5,101)=.23, p=.94), but varied in power, which is evident by increasing area of the polar phase distributions by sequence position (Supplementary Fig. 13).

DISCUSSION

Temporal coding models ³¹ converge on the idea that the brain may utilize the precise timing of neuronal firing to encode information. Theta phase coding models ^3,4,6,32,33 specifically predict that the order in which a sequence of events occurred in the external world may be represented internally in the brain by the timing of neural ensemble firing (in the gamma frequency) with respect to the phase of an underlying theta wave. Our results provide compelling evidence in humans that a peak in gamma power for each successive item shifts along the phase of an underlying theta rhythm during successful, but not unsuccessful, sequence encoding. These findings suggest that by associating the ordinal position of a gamma-coded item representation with a particular theta phase, the brain may preserve the order in which a sequence occurred.

In this study, we assess whether gamma power during the encoding of items in distinct sequence positions is preferentially biased towards distinct and consecutive phases of theta. Critically, using this paradigm, we are able to separately examine whether phase coding is evident during both successful and unsuccessful temporal order encoding. First, we find that for all six sequence positions, there is a strong main effect of gamma power over theta phase (Fig 2a). Second – in line with our primary predictions – gamma power shifts progressively later in the theta cycle for each item in the sequence, and the relative peak gamma power reflects the order in which the items were encountered (Figs. 2 and 3). Critically, this ordinal phase coding effect is only present during the encoding of sequences later correctly remembered. The specificity of this effect to successfully encoded sequences provides strong support for the notion that the brain codes recent elements in memory by leveraging relative phase differences among distinct items in a sequence. The specificity to remembered sequences also helps to rule out concerns that the effects are somehow derivative only of the visual and task structure, as the timing of stimulus onset and motor responses are identical within sequences later remembered or forgotten. Further, the memory-related phase coding effects are present after removing the early stimulus-evoked activity (<500ms), and remarkably, even persist into the inter-trial interval when the stimulus is no longer on the screen. These results provide strong evidence for the longstanding theory that theta-gamma phase coding supports temporal sequence memory.

In our initial set of analyses, we find that theta-gamma PAC is modulated by the position of the object within a sequence. Specifically, using a computational model-driven linear regression approach (see Methods), we find that theta-gamma PAC decreases over items in a sequence in left lateralized and left posterior MEG clusters (Fig. 1d). While perhaps counterintuitive, this pattern of decreasing PAC is predicted by a computational simulation of the theta-gamma hypothesis (Fig. 1c) and we show is due to a broadening of the gamma distribution over a theta cycle (see Fig. 2). One remaining question is: what is the nature of the mechanism that is driving the model fits in our initial set of analyses (i.e. Fig. 1d)? One possibility is that the broadening of gamma power across sequential positions is the result of the accrual and maintenance of all prior items within a sequence, each nested in a distinct phase of an underlying theta oscillation, similar to the model proposed by Lisman and Idiart (1995) and formalized in our computational simulation (see Fig. 1b & c). It follows then that the representation of additional items would result in a broader distribution of gamma power over a theta cycle.

A second possibility, however, is that the effect is driven by a progressive shift in gamma power along the phase of theta while items are being encoded (in the absence of gamma broadening related to the accrual of item representations). On its own, this pattern would not modulate theta-gamma coupling as we measured it. This is because the measure of coupling we utilize is sensitive to the width of the gamma distribution over theta, and a mere shift in gamma power would not modulate the width of the distribution. However, a forward shift in gamma power by sequence position -- in conjunction with the strong main effect of coupling we observe in the data – could, in fact, account for the pattern of decreasing PAC by sequence position that we observe. For instance, if items in early sequence positions are preferentially locked to the trough of theta (i.e. where gamma power is highest for all sequence positions in our data), this pattern would result in an exaggerated measure of theta-gamma coupling for early positions. If items later in a sequence are preferentially coded at the theta power peak (i.e. where gamma is lowest for all sequence positions), this would result in an attenuated measure of theta gamma coupling. While a subtle, but very important point, our initial analyses are actually agnostic to the relative likelihood of either of these two possibilities.

Our data are more consistent with the latter model where a position-related forward shift in gamma power along the phase of theta supports sequence memory encoding, in the absence of accruing gamma-coded item representations. There are two major data points that led us to this conclusion. First, if additional items are added into the sequence representation, one might expect overall measures of gamma power to increase across the sequence as well. Contrary to this expectation, gamma power is relatively flat across the sequence (Supplementary Fig. 3). Secondly, one might also expect, that the width of the gamma power distribution over theta should broaden as a function of sequence position (as more item representations are accrued and maintained). Contrary to this hypothesis, we did not see evidence that the width (standard deviation) of the distribution increased across sequence positions (One-way ANOVA with repeated measures: F(1,15)=.20, p=.64). Thus, we believe that the best explanation for our results is that sequence encoding is supported by a temporal shifting of gamma power (representing each item) along the phase of an underlying theta oscillation.

With this conclusion in mind, it is important to point out that in order to examine gamma power associated with each sequential position, we examine a time window that is coincident with each item presentation – i.e. the time period when that stimulus was presented, as well as the immediately following inter-trial interval. In its original conception, the Lisman and Idiart model offered a proposed mechanism to support the maintenance of multi-item sequences in working memory - after items had been encountered and were presumably being actively maintained. If gamma power associated with each item representation in a sequence is ordered along an underlying theta wave, then we should be able to measure this effect while these items are being encoded. Thus, the current results do not preclude the possibility that, during working memory, if one could track the gamma power associated with each item into subsequent time periods, that they would still be associated with the same phase of theta.

Furthermore, the maintenance of multiple items is likely to be dependent on the task and specifically whether items are being ‘actively’ retained in working memory. Our task is not necessarily an ‘active’ working memory task in the sense that participants are not required to rehearse and maintain the sequence of items during a delay period as in classical working memory paradigms. Rather, participants are instructed to remember the temporal order of the sequence by forming associative links between neighboring items, which likely involves some working memory maintenance along with other associative encoding operations. Thus, our current results are agnostic as to whether one would see the continued maintenance of phase coding effects during working memory rehearsal. Thus, it is important to point out that these results are not necessarily in conflict with the Lisman and Idiart model, and may in fact be parsimonious. Future studies should test whether there is a relationship between theta-gamma phase coding during sequence memory encoding and theta-gamma activity that has been observed during working memory maintenance ⁹.

The gamma power effects we report here peak between ~70–100 Hz. Critically, recent work using simultaneous MEG and iEEG suggests that gamma power <100 Hz is reliably detectable ³⁴. Interestingly, recent work in rodents suggests that there is a functional distinction between high (~60–100 Hz) and low (25–55 Hz) gamma in the hippocampal circuit, such that lower frequency theta-locked gamma ‘sweeps’ may represent the future spatiotemporal trajectory of the animal, while fast theta-locked gamma may support the coding of ongoing trajectories in real-time ³⁵. Possibly related to this dissociation, in the current study, where subjects are encoding trial-unique sequences (i.e. sequences did not repeat, and thus could not be ‘predicted’), we observe our memory-related phase coding effects in high gamma. Future work could test whether theta phase – low gamma power interactions emerges with repeated sequences, possibly indexing the forward prediction of upcoming items. While its important to note that our experimental design as well as the signals that we record are admittedly very different than the rodent study discussed above, it is nonetheless intriguing to consider parallels between the datasets.

While gamma power is relatively stable over positions, we find an increase in theta power across the sequence, consistent with prior studies showing theta power increases during memory encoding ^36,37 and working memory maintenance ^38–42. One prior study observed theta power increases specifically during delay period maintenance of temporal order information as compared to the maintenance of just the items themselves ³⁸. One possibility is that theta power increases reflect control processes related to relational encoding ^27,36,38. Consistent with this idea, we see that sustained theta power increases occur predominantly over frontal regions (see Supplementary Fig. 1), possibly reflecting prefrontal control processes involved in representing temporal relations among items ⁴³.

There is a rapidly growing literature linking theta-gamma coupling to human memory ^9,11,13,44. These studies all converge on the idea that theta-gamma phase-amplitude coupling plays a role in the maintenance, as well as the long-term retention of mnemonic information. While these prior studies have been critical to advancing our understanding of the temporal dynamics of episodic memory formation, the fundamental and important question of whether theta phase coding supports sequence encoding has not been tested. To our knowledge, these data provide the first empirical evidence in humans that memory for event sequences is supported by the precise timing of item-related gamma activity with respect to an underlying theta oscillation.

Prior work in rodents has leveraged the spatial specificity of ‘place cells’ ⁴⁵ to show that sequences of hippocampal place cells representing a ‘movement trajectory’ not only fire during an experience, but also later ‘replay’ in a similar order on subsequent cycles of a theta oscillation ^46–48. The current result, by analogy, demonstrates that the encoding of an episodic ‘object trajectory’ or temporal sequence of items is also supported by a theta phase code. Given these two sets of findings, it is possible that the sequential coding of information along the phase of a theta oscillation represents an all-purpose mechanism in the brain that allows for temporally separated information to be associated via long-term potentiation. These results fill a critical gap in the literature by providing empirical evidence from a well-controlled and characterized behavioral paradigm in humans that theta-gamma phase-amplitude coupling and more specifically, theta phase coding, supports object sequence memory.

METHODS

Subjects

Twenty healthy right-handed native English speakers (4 males, age range: 21–35, mean age: 28) recruited from New York University and the greater New York metropolitan area participated in the MEG experiment. The study was approved by the University Committee on Activities Involving Human Subjects and all participants gave informed written consent. We excluded two subjects whose performance on the order memory test was not statistically different from chance (50%, using binomial test) and one subject who did not complete the study due to drowsiness. We excluded these participants, leaving 17 subjects for the MEG analyses. No statistical methods were used to pre-determine sample sizes but our sample sizes are similar to those reported in ⁴⁹.

Experimental procedures

Materials

Stimuli consisted of 576 gray-scale pictures of objects collected from various online sources. Some examples of stimuli can be seen in the main text (Fig. 1). Colored borders for the objects were generated by selecting 24 unique colors from a color continuum ranging from [0,0,0] to [255,255,255] RGB values. Colors were manually chosen to be maximally distinct from each other. Color backgrounds were pseudo-randomized and stimulus order was randomized across subjects.

Design and Procedure

Encoding

During encoding, participants made pleasantness decisions on trial-unique objects that were paired with a colored background frame. Specifically, participants were instructed to imagine each object in the color of the background frame and press a button to indicate whether or not the combination was pleasant. We chose to use this encoding task to encourage participants to associate the color and object, since attention to the context (i.e. color) was critical to our hypothesis. To promote successful temporal order memory, we additionally instructed participants to associate neighboring objects together by imagining them interacting with each other. Subjects were instructed to perform this task irrespective of the color changes between some of the items. Pilot data indicated that this instructional manipulation was critical to achieving above chance temporal order memory performance in a majority of our participants.

During the encoding task, the background color frame remained the same for 6 consecutive trials (i.e. a ‘sequence’) and then switched to a new color. There were 6 sequences (totaling 36 objects) in each encoding block and 16 encoding-test blocks across the experiment. Each object was on the screen for 2.5 seconds, followed by a 2 second inter-trial interval (ITI) and a .5 second fixation period. The timing of the task was fixed (i.e. not jittered). During the ITI and fixation period, the color frame remained on the screen.

Temporal Order Memory Test

After each study block, we tested temporal order memory. We used this temporal order test as a proxy for probing intact sequence memory. In this test, two previously studied objects were presented side by side (with the previously colored background frame now gray). Participants were asked to indicate which of the two objects appeared first (earlier) in the sequence and rate their confidence using a 4 alternative forced choice design. Thus, there were four possible responses during the test: high confidence correct order, low confidence correct order, high confidence incorrect order and low confidence incorrect order. The tested objects always occurred in the 2^nd and 6^th position within a sequence and all tested object pairs were separated by 3 intervening trials during encoding. The test was self-paced with a mandatory .5 second fixation period between each test trial.

MEG recordings and data processing

MEG data were recorded using a 157-channel whole-head axial gradiometer system (KIT, Kanazawa, Japan). Three reference channels seated above the MEG system were also recorded and used to remove ambient electromagnetic environmental noise from the data. MEG data were acquired in DC with a sampling rate of 1000 Hz, a low pass filter at 200 Hz and a notch filter at 60 Hz to remove line noise. To measure head position, 5 electromagnetic coils were attached to a participant’s head during recording. Coil locations were determined by registering scalp coil positions with 3D digitized head shape data (software: Source Signal Imaging, Inc.; hardware: Polhemus, Inc.), which was collected before MEG recording. The anatomical locations used to register the coils with the head shape data were the nasion and the left and right periauricular points. The coils were localized to the MEG sensors at the beginning and end of the experiment.

MEG data were preprocessed as follows: raw MEG data were loaded into MATLAB (version 7.10, Mathworks) and malfunctioning channels (average per subject: ~2) were immediately removed and interpolated with the average of its nearest neighbors. Then, data were denoised using a time-shifted principal components analysis approach (temporal shift parameter = 100ms), which removed ambient environmental noise using 3 reference channels ⁵⁰. The remaining preprocessing steps utilized the Fieldtrip MEG and EEG software package ⁵¹ and custom MATLAB scripts. The data were band-pass filtered (default settings in eegfilt.m) from 1–100 Hz. Then, the data were epoched from −4 to 4 seconds surrounding trial onset to assure adequate time for spectral estimation of both pre- and post-stimulus activity. The epochs were downsampled to 500 Hz to speed processing time in later steps. Finally, to facilitate interpretation of topographic plots, we transformed the MEG data from axial to planar gradient. One advantage of this linear transformation is that planar signal amplitude is typically largest directly above the source, whereas axial signal amplitudes are typically maximal on either side of the neural source of the signal. This transformation was performed for all topographic analyses, but not the source-space analyses.

After preprocessing, the MEG data were examined for artifacts. The artifact rejection approach we took was three-fold: First, excessively noisy trials and channels were removed using Fieldtrip’s visual artifact rejection ‘trial summary’ feature. Specifically, channels and trials for which the across-trial variance exceeded 3 standard deviations from the mean were identified and removed from the analysis. Then, independent component analysis was implemented to remove components related to eye blinks, eye movements, and heartbeat-related artifacts. Finally, remaining trials were visually inspected and epochs containing any remaining artifacts were removed from the dataset. The group average proportion of trials removed due to artifacts was ~8.6%. Data collection and analysis were blind to the conditions of the experiment.

Statistics

All statistical tests are two-sided unless otherwise noted. Data distributions were assumed to be normal, but this was not formally tested. Details for the specific statistical analyses can be found in the subsections below.

Time-Frequency Power Analyses

A time-frequency analysis was performed for each epoch (−4 to 4 seconds, 50 ms sliding window, zero-padded) using a morlet wavelet approach (number of cycles=6), estimating spectral power from 1 to 100 Hz in steps of 1 Hz. This analysis resulted in time-frequency spectrograms representing oscillatory power for each time-frequency-sensor point for each trial and each subject. This relatively long epoch window allowed us to analyze data during the ‘stimulus on’ period (0 to 2.5 seconds) as well as the inter-trial periods (−2.5 to 0 seconds) while avoiding edge artifacts particularly in the low frequencies.

Cross-Frequency Coupling Analyses

Phase-amplitude cross-frequency coupling (PAC) was estimated as follows for each trial for each sensor and subject. The algorithm to compute the PAC ‘modulation index’ (referred to as MI or coupling values or estimates) was taken from ⁵². First, to compute gamma amplitude, raw MEG time-series (for each trial and sensor) was filtered from 70–100 Hz, which was determined by frequencies that showed a peak in the spectral power distribution (Supplementary Fig. 1). The envelope was then computed by taking the absolute value of the Hilbert Transform of the filtered time-series. To compute theta phase, the raw data was filtered from 3 to 8 Hz in steps of 1 Hz resulting in 5 filtered time-series (i.e. 3–4, 4–5, etc.). We filtered in steps of 1 Hz instead of the range of the band (3–8 Hz) because we wanted to minimize the possibility that changes in the peak frequency of the theta band could explain our findings. Phase was computed for each of the 5 filtered time series by taking the angle of the Hilbert transform of the filtered signal. Then we binned gamma power by theta phase (18 bins) during stimulus presentation (0–2.5 seconds), now averaging across the 5 theta sub bins, resulting in a single theta-binned gamma histogram for each trial. We then normalized the distributions, such that the power of each histogram summed to one. Finally, we computed Kullback-Leibler divergence for each theta-binned gamma distribution and divided by the log(18) i.e. the number of phase bins.

In order to determine statistical significance of the coupling values, we employed a phase scrambling permutation procedure as outlined in ⁴⁴. For each trial, we recomputed the coupling analysis described above, but circularly shifted the time-series of phase values by a random interval greater than 500 samples (i.e. the sampling rate, 1 second). For each trial (and theta phase sub-bin), we repeated this phase-shifting process 100 times to derive a null distribution of coupling values. We then converted the coupling estimates to statistical values (z-statistics). To calculate within-subject statistics, we computed a t-statistic across trials (averaging across theta-sub bin, i.e. one value per trial) for each sensor and subject. Then, to calculate group-level statistics, we computed a t-statistic across subjects for each sensor.

Fitting Theta-Gamma Coupling Estimates to the Model

Our initial hypothesis was that as items are maintained in and integrated into working memory, additional gamma cycles would be concatenated along the phase of theta ⁵. To simulate this hypothesis and test for this pattern in our data, we generated theta (4 Hz) sinusoidal waves (10 cycles to mimic our 2.5 second stimulus presentation), adding 1 to 6 cycles of a gamma rhythm (85 Hz) per theta cycle. The 6 different simulations represented our hypotheses for the 6 positions within a sequence. We then computed the predicted coupling score (i.e. modulation index) for each of these simulations. Due to an increase in the width of the distribution of gamma over theta as function of sequence trial position, our simulation predicted linearly decreasing coupling values across sequence positions (see Fig. 1b,c for visual depiction).

To test for this pattern of broadening gamma over theta in our data, we performed a linear regression on the trial-wise estimates of theta-gamma coupling. The predictor (independent) variable was our simulated hypothesis (described above) and the dependent variable was a vector of observed coupling estimates. To construct the vectors of coupling values, for each trial, sensor, and subject, we averaged together the coupling estimates across theta sub-bins so that there was a single coupling value per trial. We then performed a separate linear regression analysis (independent: model-predicted coupling value, dependent: trial-wise coupling estimate) for each sensor and subject, resulting in subject-level topographic statistical maps (t-values) representing the fits of our model to the empirically observed coupling estimates. Then to compute group-level statistics, we computed a t-statistic across subjects for each sensor (see Fig. 1d). To correct for multiple comparisons, we derived a statistical threshold based on the size of a cluster of sensors compared to what one might expect by chance. For each sensor and subject, we shuffled the trial labels (1000 times) and then refit the model. Then, we recomputed cluster sizes on each iteration to build a null distribution of maximum cluster sizes expected by chance and only retained clusters exceeding p<.05 of the null distribution of cluster sizes. Only clusters of sensors that exceeded this threshold were further analyzed.

To control for power differences across a sequence as a potential confounding factor for the analyses described above, we performed a second regression analysis where first, a general linear model with theta and gamma power (as separate predictor variables) was constructed and regressed against the coupling estimates. We then refit our model (i.e. decreasing coupling by sequence position) to the residuals of this model. Thus, any explanatory power that theta or gamma power had on theta-gamma coupling was removed (Supplementary Fig. 4).

Theta-Gamma Model Source Localization Analysis

A linearly-constrained minimum-variance beamformer analysis ⁵³ was performed to estimate neural sources of the decreasing theta-gamma coupling by sequence position. Briefly, this technique utilizes an adaptive spatial filtering algorithm designed to estimate sources of neural activity originating from a spatial location in the brain given a particular topographic distribution of MEG activity by applying a unit gain constraint to the spatial location of interest and minimizing the contribution of all other sources. First, each subject’s data was registered to a canonical structural MPRAGE brain from the FSL software package ⁵⁴. This was achieved by aligning anatomical landmarks (nasion, left and right periauricular points) from digitized head shape data to the structural brain image for each subject. To estimate source-space time-series data, we used a combination of source analysis script from the Fieldtrip software package as well as custom MATLAB scripts. A semi-realistic head model was constructed following methods described by ⁵⁵. Then, using the MEG data from all trials (irrespective of sequence position), a common spatial filter was estimated for each point in a three dimensional grid representing potential neural source locations with 1 centimeter spacing between points. The result of this analysis was a vector of spatial weights (1 × 157, i.e. the total number of MEG sensors) mapping the contribution of each sensor’s activity to a particular grid (i.e. brain) location. Then, to derive source-space time series for each trial (−1 to 3.5 seconds), the matrix of sensor-level time series (157 sensors × 2251 time points for each trial) was multiplied by the spatial weight matrix, resulting in a single time course for each source location for each trial.

Once the all the sensor-level data was projected into source space, we performed an analysis very similar to the sensor-level theta-gamma model fit analyses described above. For each trial (during stimulus presentation, 0 to 2.5 seconds) and source-space location, we bandpass filtered the data in the high gamma band (70–100 Hz; using default filter settings of eegfilt.m) and computed gamma amplitude by taking the absolute value of the Hilbert transform of the filtered signal. Then, we derived the theta phase time course by bandpass filtering the data in the theta frequency range (3–8 Hz; using default filter settings of eegfilt.m) and then computed the angle of the Hilbert transform of the filtered signal. The remainder of the analysis was identical to the sensor-level theta-gamma model fitting analyses described above. Briefly, for every subject, we computed theta-gamma coupling for each trial and source-space point and fit the theta-gamma coupling source-space point to our model of decreasing theta-gamma coupling as a function of sequence position. Then, to compute group-level statistics, we computed t-statistics across subjects for every source-space point. The final product was a 3D source-space statistical map of t-values representing the group reliability of the fit of theta-gamma coupling to our model of decreasing coupling across sequence positions. Note that there is typically a center bias for beamformer source localization when a source is localized without respect to a baseline (i.e. pre-stimulus period or another condition). Given that the regression analysis we ran is a linear contrast across sequence positions, this potential confound is likely not a contributing factor.

Phase Analyses

We hypothesized that for items in different sequence positions, gamma power would be biased to different phases of theta, particularly when temporal order was successfully encoded. To test this, for each trial, we computed histograms of gamma power binned by theta phase (18 bins; power and phase computations are described above in Cross-frequency Coupling Analyses section) for each sensor. We then averaged across clusters of sensors that displayed a significant fit to our model (i.e. decreasing coupling by sequence position, p<.05, cluster-corrected using permutation procedure), which resulted in two clusters (a left lateral cluster and a left posterior cluster), and then sorted sequences by subsequent temporal order memory. To test whether the theta-binned gamma distributions differed by sequence position, we used a Watson-Williams multi-sample test for equal means implemented from the Circular Statistics Toolbox for MATLAB ⁵⁶. This test is effectively a one-way ANOVA for circular-linear data. To compute significance of the interaction between position and sequence memory, we used a Harrison-Kanji Test, a circular implementation of a two-way ANOVA. It should be noted that all phase analyses described in this section were statistically tested using circular-linear tests unless otherwise specified.

In a follow up analysis, we then averaged the trials into 3 bins by sequence position: 1&2 (early), 3&4 (middle) and 5&6 (late). Finally, we averaged the data across subjects. The result of this analysis was 6 histograms (for each cluster of interest) of gamma power binned by theta phase for early, middle and late sequence trials and for sequences where later temporal order was correct and incorrect (Fig. 3).

To compute theta phase locking (also known as ‘inter-trial’ phase coherence), we followed methods outlined in ⁵⁷ (see Fig. 3c in the referenced paper). Briefly, for each trial, we filtered the data at the theta frequency. Then, we computed the Hilbert transform and normalized the resulting complex vectors to remove the amplitude component. Finally, for each time point, we computed the phase locking value by averaging the normalized vectors across trials for a given condition for each subject.

Testing for systematic shifts in theta phase by sequence position

Another possible alternative explanation for our result is that gamma power remains temporally fixed with respect to the onset of the stimulus, but there is a systematic shift in theta phase as a function of sequence positions, perhaps due to a reset in the phase of theta caused by the stimulus presentation. To test for a mechanism of this nature, we simulated sinusoidal time series at the theta frequency, where the phase of theta systematically shifted over the sequence, and then computed the cross-correlation of the simulated time series for each pair of sequence positions (cross-correlation between 1–2, 1–3, 1–4, etc.). If a systematic theta phase shift is present in the data, then the peak cross-correlation should also systematically increase with increasing lag between items. An analysis of this ‘toy’ example concretized our intuition that the temporal lag of the peak correlation would increase with distance between sequence positions (Supplementary Fig. 12). We used this framework to test for a systematic phase shift in our data at the level of each sequence, and then averaging the peak lag values over sequences and over subjects. We performed this analysis specifically for correct sequence, as that is where we observed the phase coding effects. The results do not reveal any evidence for a systematic theta phase shift across sequences (F(4,84)=.33, p>.5; Supplementary Fig. 12). Thus, the phase-amplitude relationship is more likely to be driven by a shift in gamma across sequence positions rather than a phase shift in theta.

Testing for systematic changes in theta phase symmetry

Another possible explanation for PAC modulation by sequence position is that the shape of the theta waveform could systematically change, becoming more or less asymmetric over a sequence of items. While the exact pattern of expected results would vary based on the phase dynamics, oscillations with symmetric, sinusoidal phase dynamics will map to a circular distribution in polar coordinates, while asymmetric oscillations will map to a non-circular distribution (see Supplementary Fig. 13 for a visualization of this pattern). To test whether an interaction between phase symmetry and sequence position could explain these effects, we filtered each trial in the theta band. Then, we computed group-averaged phase-amplitude distributions in polar coordinates, averaging separately over each sequence position. The results suggest that the theta waveforms were equally symmetric over sequence positions (Watson-Williams test: F(5,101)=.23, p=.94), but varied in power, which is evident by increasing area of the polar phase distributions by sequence position (Supplementary Fig. 13). This replicates the prior finding that theta power systematically increases over sequence positions. Thus, differences in phase symmetry are not likely to explain this pattern of results.

Data availability

The data that support the findings of these studies are available from the corresponding author upon request.

Code availability

The code that support the findings of these studies are available from the corresponding author upon request.