Human intracranial high-frequency activity maps episodic memory formation in space and time

Abstract

Noninvasive neuroimaging studies have revealed a network of brain regions that activate during human memory encoding; however, the relative timing of such activations remains unknown. Here we used intracranially recorded high-frequency activity (HFA) to first identify regions that activate during successful encoding. Then, we leveraged the high-temporal precision of HFA to investigate the timing of such activations. We found that memory encoding invokes two spatiotemporally distinct activations: early increases in HFA that involve the ventral visual pathway as well as the medial temporal lobe and late increases in HFA that involve the left inferior frontal gyrus, left posterior parietal cortex and left ventrolateral temporal cortex. We speculate that these activations reflect higher-order visual processing and top-down modulation of attention/semantic information, respectively.

1 Introduction

Among those experiences that enter the focus of our attention, some are encoded in a manner that can easily support subsequent recollection while others are not. This variability in goodness of memory encoding has been the subject of considerable psychological research over the last century (Kahana, 2012), yet only in the last decade or so have we begun to uncover its physiological basis. In the laboratory, one can investigate the neural basis of goodness of encoding by recording brain signals from participants while they engage in a learning task and then correlating specific features in the signal with subsequent memory performance (Paller & Wagner, 2002).

Using this approach, often referred to as the subsequent memory (SM) paradigm, functional magnetic resonance imaging (fMRI) studies have identified several brain regions involved in memory encoding; the left prefrontal cortex (Blumenfeld & Ranganath, 2007), posterior parietal cortex (Uncapher & Wagner, 2009), medial temporal lobe (Henson, 2005), and fusiform cortex are among those areas most consistently activated during successful encoding (Spaniol et al., 2009; Kim, 2011). However, fMRI studies lack the temporal resolution required to identify the temporal sequence of activations underlying memory encoding. This, in turn, has limited our understanding of how these regional activations interact to form functional memory encoding networks (Rugg et al., 2002).

To investigate the spatiotemporal properties of this memory encoding network, it is necessary to use a brain signal with millisecond temporal resolution, such as intracranially recorded high-frequency activity (HFA). HFA refers to fast fluctuations in neuro-electrophysiological recordings that manifest as increases in spectral power at frequencies above 60-70 Hz. The neural mechanism that gives rise to such fast activity is a topic of on-going research: HFA has been linked to asynchronous “shot-noise” related to increased multi-unit activity (Milstein et al., 2009; Manning et al., 2009; Miller et al., 2009; Ray & Maunsell, 2011), the superposition of multiple high-frequency oscillations (Crone et al., 2011; Gaona et al., 2011), as well as a combination of these two processes (Scheffer-Teixeira et al., 2013). Despite its unclear neural origin, however, an increasing number of studies have leveraged HFA as a marker of underlying neural activation (Crone et al., 2011; Lachaux et al., 2012), similar to the blood-oxygen-level-dependent (BOLD) signal. Indeed, HFA has been directly correlated with BOLD activity (Mukamel et al., 2005; Conner et al., 2011), further suggesting that HFA represents a marker of general neural activation.

As a marker of general activation, HFA has been used to functionally map areas involved in motor activity (Leuthardt et al., 2007), auditory perception (Crone et al., 2001), language processing (Sinai et al., 2005), tactile sensation (Chang & Cheung, 2012), among others. Here, using intracranial recordings from neurosurgical patients in the SM paradigm, we leverage HFA to functionally map areas of the brain responsible for episodic memory formation. Whereas previous work has established that HFA increases during successful memory encoding in the SM paradigm (Sederberg et al., 2007, 2007b), such work has interpreted HFA strictly through an oscillatory framework. However, if HFA instead represents a more general metric of neural activation, the information conveyed by this signal should be reflected in the exact time and spatial location in which it is active. By collecting data from a very large number of patients (ninety-eight), we were able to overcome the limited spatial sampling of human intracranial electrophysiology and use HFA to map memory encoding in both space and time. This approach revealed a dynamic spatiotemporal activation of functional networks that mediate encoding, as described in this report.

2 Material and methods

2.1 Participants

Participants with medication-resistant epilepsy underwent a surgical procedure in which grid, strip, and depth electrodes were implanted so as to localize epileptogenic regions. Data were collected over a 14 year period as part of a multi-center collaboration with neurology and neurosurgery departments across the country. Our research protocol was approved by the institutional review board at each hospital and informed consent was obtained from the participants and their guardians. Our final participant pool consisted of 98 patients (86 left-language dominant patients; see Supplementary Table S1).

2.2 Free Recall Task

Each patient participated in a delayed free-recall task. In each trial of this task, participants are instructed to study a list of 15 or 20 words and are then asked to freely recall as many words as possible. Words were presented sequentially and remained on the screen for 1600 ms, followed by a randomly jittered 800-1200 ms blank inter-stimulus interval (ISI). Immediately following the final word in each list, participants were given a distraction task (arithmetic problems; minimum 20 sec) and were then given 45 sec to recall as many words as possible from the list in any order. Words that were presented during the encoding period and successfully retrieved during the recall period are considered successfully encoded (Paller & Wagner, 2002).

2.3 iEEG recordings

Clinical circumstances alone determined electrode number and placement. Subdural (grids and strips) and depth contacts were spaced 10 mm and 8 mm apart, respectively. iEEG was recorded using a Bio-Logic, DeltaMed (Natus), Nicolet, Grass Telefactor, or Nihon-Kohden EEG system. Depending on the amplifier and the discretion of the clinical team, the signals were sampled at 200, 256, 400, 500, 512, 1000, 1024, or 2000 Hz. Signals were converted to a bipolar montage by differencing the signals between each pair of immediately adjacent contacts on grid, strip, and depth electrodes; the resulting bipolar signals were treated as new virtual electrodes (henceforth referred to as electrodes throughout the text), originating from the midpoint between each contact pair (Burke et al., 2013). Signals were re-sampled at 256 Hz; a notch filter was applied at 60 Hz or 50 Hz. Analog pulses synchronized the electrophysiological recordings with behavioral events. Contact localization was accomplished by co-registering the post-op CTs with the MRIs using FSL Brain Extraction Tool (BET) and FLIRT software packages. The resulting contact locations were mapped to both MNI and Talairach space using an indirect stereotactic technique. To identify whether a particular anatomical area exhibited task-related changes in power, we grouped spatially similar electrodes from different participants by segregating Talairach space into 53,471 overlapping 12.5 mm radius spheres spaced every 3 mm. Only spherical regions that had electrodes from 5 or more patients were included in analyses.

2.4 Spectral Power

We convolved clips of iEEG (1000 ms before item onset to 2900 ms after onset, plus a 1000 ms flanking buffer) with 30 complex valued Morlet wavelets (wave number 10) with center frequencies logarithmically spaced from 2 to 95 Hz (Addison, 2002). We squared and log-transformed the wavelet convolutions, and then averaged the resulting log-power traces into 500 ms epochs with 490 ms overlap, yielding 341 total temporal epochs surrounding each word presentation. For the low-temporal resolution analysis in Figure 2, we averaged the continuous time power trace into a single time epoch from 0 to 2000 ms after word presentation. Power was then averaged into a high-frequency activity (HFA) band (64 to 95 Hz), which was used to create the topographic activation maps. We z-transformed power values separately for each session (Burke et al., 2013). For every electrode and for every temporal epoch, we assessed the difference in spectral power during memory formation by calculating a parametric t-statistic on the distributions of average power values during successful and unsuccessful encoding. In Figures 2B-C and Figure 4A, t-statistics comparing power during successfully and unsuccessfully encoded words were averaged across all electrodes from each patient in a particular region.

Figure 2.

Power changes during encoding (0-2000 ms post word presentation). All spherical regions that exhibited a significant (permutation procedure; FDR corrected) change in power during successful memory encoding are displayed on a standardized three-dimensional brain. Increases (Rec > NRec) and decreases (Rec < NRec) in power during encoding are shown in red and blue, respectively. The horizontal dashed line on the sagittal views corresponds to the level of the axial cut in the third panel. Color and grayscale renderings represent the percentage of nearby regions exhibiting significant effects and containing more than 5 patients, respectively. Radiological slice view is shown with right (R) and left (L) hemispheres labeled.

Figure 4.

Timing comparisons of HFA between regions-of-interest (ROIs). A: In each panel, the difference in HFA across time during the presentations of recalled (REC) vs. not recalled (NREC) words is shown for each ROI. Width of lines represent ±1 SEM across patients. Green line represents time of maximum activation. Panels are arranged top-to-bottom, left-to-right by time of peak activation. B: t-statistics comparing differences in peak activation times between all 21 ROI-ROI pairs are colored-coded and displayed in each cell of the matrix. Positive t-statistics (green) indicated that the region on the corresponding y-axis (R1) activated later than the region on the corresponding x-axis (R2). Negative t-statistics (purple) indicated the R1 region activated before the R2 region. Multiple comparisons were corrected using a permutation procedure. Significant pairs (p < 0.05) are located inside of the yellow boxes. PHA: para-hippocampal area. PPC: posterior parietal lobe. IFG: Inferior frontal gyrus. VLTC: ventrolateral temporal cortex.

2.5 Statistical Procedure

For the anatomical plots in Figures 2 and 3, we assessed whether changes in spectral power were significant across participants for a given ROI or spherical voxel using a non-parametric permutation procedure. We calculated a t-statistic on the distribution of log-power values during successful and unsuccessful encoding during a single temporal epoch for every electrode and from each participant. We then permuted the labels for the conditions 10,000 times to generate a distribution of 10,000 shuffled t-statistics. We averaged the true and permuted t-statistics across all electrodes within each spherical region for each participant. For each region, we then summed the true and permuted averaged values across all participants (Sederberg et al., 2007; Burke et al., 2013). To generate a p-value for changes in spectral power for a given region, we determined the position of the summed true t-statistics in the distribution of summed permuted values. To correct for multiple comparison across space (Figure 2) and time (Figure 3), we used a false discovery rate (FDR) procedure (Genovese et al., 2002, q=0.05).

Figure 3.

Power changes during encoding for multiple time windows. For six selected time windows, anatomical regions exhibiting a significant change in HFA during the presentation of all subsequently recalled (Rec) and not-recalled (NRec) items are displayed on a standardized three-dimensional brain. Increases (Rec > NRec) and decreases (Rec < NRec) in power during encoding are shown in red and blue, respectively. Radiological slice view is shown with right (R) and left (L) hemispheres labeled. Brain plots rendered identically as in Figure 2.

2.6 Topographic plots

To plot spatial changes in spectral power, we identified spherical regions that exhibited a statistically significant (FDR corrected) increase or decrease in power across participants. At each spherical region, we calculated the percentage of other regions within 12.5 mm that exhibited identical encoding-related effects. We translated these percentages to color saturation and rendered these values onto cortical and subcortical topographical plots using a standard MNI brain with information from the WFU PickAtlas toolbox (Maldjian et al., 2003). Colored values were smoothed using a three-dimensional Gaussian kernel (radius = 12.5 mm; σ = 3 mm). The maximal color saturation in either direction corresponded to 50% of adjacent spherical regions. All regions with fewer than five patients were colored black and were not analyzed. Grayscale rendering in other regions represented the percentage of spherical regions surrounding a given location with at least five patients, and thus represented regions that were analyzed but that did not exhibit significant effects. For anatomical plots collapsed across time (Figures 2 and 5), only contiguously statistically significant regions (spherical regions flanked by other significant regions in all dimensions) were visualized in order to identify regions with concentrated clusters of activity.

Figure 5.

Regions clustered by temporal activation profile. A: All regions showing a significant change in HFA were clustered into two categories, using a k-means algorithm, which yielded an early (left panel; average temporal profile peaked at 490 ms) and a late (right panel; average temporal profile peaked at 1110 ms) cluster. B: Regions in each cluster were re-projected back onto the standard brain. Colors represent regions belonging to the early (purple) and late (green) clusters. Brain plots rendered identically as in Figures 2 and 3.

2.7 Pairwise ROI timing

In order to quantitatively assess timing interactions of regional activations, we selected regions-of-interest (ROIs; Figure 4). So as not to bias these timing comparisons, ROIs were selected using the single time epoch analysis in Figure 2. We focused on the left hemisphere in order to minimize the number of pairwise comparisons, and also because the majority of significant neocortical regions were located on the left. ROIs were manually selected based on clusters of significant spherical regions in the L. inferior frontal gyrus (L. IFG) and L. posterior parietal cortex (PPC). For posterior regions, it was difficult to distinguish between ROIs based on separation in Talairach space (see Figure 2). We therefore defined ROIs as any region that showed a statistically significant modulation of HFA during encoding and also fell within predefined Brodmann areas (BA; Visual cortex: BA-17,18; Fusiform cortex: BA-37; Parahippocampal Area: BA-34,35,36; Ventral lateral temporal cortex: BA-21). In addition, recognizing its fundamental role in episodic encoding, a clinician experienced in neuro-anatomical localization manually reviewed post-OP CT and MRI images to accurately identify all depth contacts located within the hippocampus. A bipolar pair was categorized into the hippocampal ROI if at least one contact within the pair was determined to lie within the hippocampus (Burke et al., 2013).

Within each ROI and each patient, we found the time point of the maximum t-statistic comparing power during successfully and unsuccessfully encoded words. In order to remove spurious peaks from patient’s that did not show an electrophysiological response, patient’s with maximum t-statistics < |1| were discarded for this analysis. Independent sample t-tests were used to compare the resulting across subject distribution of maximum time-points between each ROI pair. To correct for multiple comparisons across the 21 ROI-ROI comparisons, we used a permutation procedure to empirically set the false-positive rate such that less than one pair was labeled significant during 1,000 shuffled iterations.

2.8 Temporal Clusters

To summarize the regions showing early vs. late timing (Figure 5), we first found all regions showing a statistically significant modulation of HFA for any time epoch. We treated each such region as a vector defined by the values of its across patient t-statistics for each time epoch (341 dimensional space). We then z-scored each vector to its own mean and standard deviation to remove the effect of raw amplitude on the clustering algorithm. Next, we forced all such vectors into two clusters using a k-means clustering algorithm. The average waveforms in Figure 5 reflect the mean t-statistics across all regions in each cluster.

3 Results

We recorded intracranial electroencephalography (iEEG) from 98 neurosurgical patients (86 left-language dominant) while they engaged in a free recall task. We first characterized the overall pattern of power changes across all frequencies during successful encoding for a large time interval (2000 ms) surrounding word presentation. Figure 1A depicts averaged raw power from a single electrode in the left temporal cortex for 30 frequencies logarithmically spaced from 2 to 95 Hz. Figure 1A shows that, during word presentation, there is a broadband increase in high-frequencies and a broadband decrease in low-frequencies relative to baseline, and these changes are amplified during successful encoding. In Figure 1B these same data are shown using a comparison statistic. Specifically, a t-statistic was calculated at each frequency comparing the distributions of power in the successful and unsuccessful encoding conditions (red and blue lines in Figure 1A). The statistic more clearly demonstrates that successful encoding is associated with more power at higher frequencies and less power at lower frequencies relative to unsuccessful encoding.

Figure 1.

Power changes during encoding (0-2000 ms post word presentation). A: For a single electrode in the left temporal cortex, averaged raw power (y-axis) during the presentation of items that were later recalled (Rec; successful encoding; red line), not-recalled (NRec; unsuccessful encoding; blue line), and all baseline events (gray line) are plotted for all wavelet frequencies (x-axis). Errorbars represent 95% CI across items or baseline events. B: For the same data in (A), t-statistics (y-axis) comparing successfully and unsuccessfully encoded events are shown for all wavelet frequencies (x-axis). C,D: Averaged t-statistics across patients are shown for electrodes (C) in the left temporal cortex and (D) all electrodes. Errorbars reflect ±1 SEM across patients. Yellow asterisks mark significant increases/decreases in power during encoding (t-test; p < 0.05; Bonferroni corrected). In panel (D), 28.7 Hz refers to the mid-way point between adjacent frequency bins on the x-axis.

We similarly plotted these changes across all left-language dominant patients with electrodes in the left temporal lobe (59 patients; Figure 1C) and all patients, all electrodes (98 patients; Figure 1D). We found that human iEEG during successful encoding typically displays increases in high-frequency activity (HFA) and decreases in low-frequency activity (LFA) (Figures 1A-D). This pattern is reliable at the at the individual electrode level (1A-B), within a particular anatomical area (1C), and across all regions sampled from the intracranial electrodes (1D). More specifically, across our entire dataset, 28.7 Hz delineated the point above which spectral power tended to increase during encoding, and below which spectral power tended to decrease (Figure 1D).

We hypothesized that the pattern of a combined increase in HFA and decrease in LFA reflected a general process of neural activation that co-varied with successful encoding (see discussion). Thus, HFA should yield a similar fMRI “activation” map when comparing words that were successfully versus unsuccessfully encoded. We confirmed this hypothesis by investigating where, in the spatial distribution of intracranial electrodes (Figure S1), HFA increased during successful encoding using a large time window (2000 ms) surrounding word presentations, analogous to the temporal resolution of fMRI studies (Figure 2). Specifically, we grouped high-frequencies into an HFA frequency band (64-95 Hz) and assessed where in the brain such activations occur during memory formation. All subsequent anatomical analyses were performed exclusively on the 86 left-language dominant patients in the study. We found increases in HFA in the left inferior frontal gyrus (L. IFG), left posterior parietal cortex (L. PPC), left ventrolateral temporal cortex (L. VLTC), as well as the bilateral medial temporal lobe (MTL), fusiform cortex, and visual areas. These regions agree with recent meta-analyses of analogous functional imaging studies (Spaniol et al., 2009; Kim, 2011).

Having recapitulated the main fMRI subsequent memory effects using HFA, we next probed the timing of these activations. In particular, we plotted the difference between HFA during successful vs. unsuccessful encoding as a function of anatomical region (similarly to Figure 2) for several, shorter time windows. Time windows were 500 ms in length and were incremented every 10 ms from 1000 ms before word-onset to 2400 ms after word onset (yielding 341 such windows). The resulting 341 activation maps were sequentially concatenated into two movies at 50 and 25 frames per second (equivalent to 50% and 25% of real-time speed, respectively), which are included in the on-line material (50% real-time speed movie links here, see caption heading Movie S1; 25% real-time speed movie links here, see caption heading Movie S2). Using these movies, it is possible to visualize regional activations as the brain encodes a new memory.

Figure 3 displays six time windows from this movie that capture key electrophysiological stages of memory formation. First, during the pre-stimulus interval (−750 to −250 ms before item presentation), there is relatively little HFA that correlates with successful encoding. Of note, however, there is a small region in the right posterior inferior temporal area that showed an increase in activity during the pre-stimulus period. During the period immediately after word presentation (0 to 500 ms after item presentation), there is a bilateral activation of visual areas and fusiform cortex. Next, in the 400 to 900 ms time epoch, HFA is observed more anteriorly; the L. VLTC and bilateral MTL (including both hippocampi) show HFA activation for successfully encoded items during this time. In the 800-1300 ms time epoch, however, MTL and visual areas are no longer activated and almost all HFA is focused in the left neocortex. Specifically, the L. VLTC, L. IFG and L. PPC come on-line during this period and stay active through the 1200-1700 ms time-epoch (as the word is removed from the computer screen). Finally, HFA in these three neocortical regions gradually decreases after the word is removed, leaving only residual activation in L. VLTC during the 1600 to 2100 ms time epoch and beyond.

In the remaining analyses, we used two approaches to evaluate the consistency, across patients, of the timing of regional activations observed in Figure 3. First, in Figure 4, we defined regions-of-interest (ROIs) to directly test whether the timing of peak increases in HFA were reliably different across anatomical regions. ROIs were chosen based on the activation map in Figure 2 (which contained no timing information) so as not to bias the temporal characteristics of each ROI. Once the ROIs were identified (see Experimental procedures), we next examined the activations across time for all ROIs in the left hemisphere (Figure 4A). We then tested the difference in peak activation times between each of these regions across patients. The resulting 21 pair-wise t-statistics are displayed using a color code in Figure 4B (further details of all 21 statistical tests are summarized in Supplementary Table S2). In Figure 4B, a positive t-statistic (green color) indicates that the ROI along the y-axis activated later than the associated ROI along the x-axis (and vice versa for negative t-statistics; purple color). The results confirmed the qualitative findings in Figure 3 and showed that the fusiform and parahippocampal areas activated reliably before the left neocortical areas. The hippocampus, in turn, activated after these ventral visual areas, as can be seen in Figure 3 (compare 0-500 ms epoch with 400-900 ms epoch). The lack of reliable across subject differences involving the visual (Figure 4B) cortex most likely reflects the small number of patients who have electrodes in this area, which decreases the statistical power of such comparisons.

Finally, to obtain a visual summary of HFA temporal dynamics during encoding, we clustered every significant region based on its temporal activation profile using a k-means algorithm. Specifically, we forced all significant regions into two clusters based on normalized changes in HFA across time. Early (490 ms post-onset) and late (1110 ms post-onset) activations naturally emerged from these clusters (Figure 5A). Furthermore, when the regions in each cluster were re-projected back into anatomical space (Figure 5B), they segregated into an early ventral visual region (bilateral visual, fusiform, and MTL) and a late left neocortical region (L. IFG, L. PPC, and L. VLTC), confirming the qualitative timing results in Figure 3.

4 Discussion

We recorded iEEG from 98 neurosurgical patients as they participated in a verbal free recall task. Spectral activity during the presentation of words that were later recalled exhibited a prominent increase in high-frequency activity (HFA) and a decrease in low-frequency activity (LFA) compared to words that were later forgotten (Figures 1 and 2). We interpreted this activity as reflecting underlying neural activation, which we leveraged to map the sequence of regional activations during memory encoding across both space and time (Figure 3).

Before stimulus onset, we found a localized increase in HFA in the right posterior inferior temporal region (Figure 3, −750 to −250 ms panel). After stimulus onset, we found that successful encoding invoked a particular spatiotemporal pattern of neural activation. This pattern began in posterior perceptual regions, including the fusiform and visual cortex as well as posterior parahippocampal regions (Figure 3, 0-500 ms panel). Such activation gradually migrated anteriorly, culminating in the hippocampus as well as the ventrolateral temporal cortex (VLTC; Figure 3, 400-900 ms panel). Following these initial activations, there was a marked shift in HFA to the left neocortex, including the inferior frontal gyrus (IFG) and posterior parietal cortex (PPC) as well as continued activation in the L. VLTC (Figure 3, 800-1300 ms panel). These three regions remained active throughout item presentation (Figure 3, 1200-1700 ms panel) with the L. VLTC maintaining activation until the presentation of the next word (Figure 3, 1600-2100 ms panel). In summary, the time courses of these activations (Figure 4) suggest to two major anatomical pathways associated with successful encoding (Figure 5A). An early stage marked by bilateral activation of the ventral visual pathway culminating in the hippocampus and a later stage comprised of a three node network in the left neocortex consisting of the IFG, PPC and VLTC (Figure 5B).

4.1 HFA: A general activation signal

Our results suggest that memory formation co-varies with a pattern of spectral activity characterized by a skew in power toward higher frequencies at the expense of lower frequencies. We speculate that this pattern represents general neural activation, similar to the BOLD signal in haemodynamic studies. Evidence in support of this claim stems from three lines of research. First, electrophysiological studies investigating spectral changes during behaviors other than memory formation have reported a similar pattern of results (for example see Crone et al., 1998a, 1998b, 2001; Miller et al., 2007; Hipp et al., 2011). These findings suggest that the skew of power toward higher frequencies is a general electrophysiological activation signal, and is not a memory specific phenomenon per se. Second, HFA has been directly correlated with multi-unit activity (MUA; Manning et al., 2009; Ray & Maunsell, 2011). Thus, the increased HFA in our data suggests that large neuronal populations become generally more active during memory formation. Third, many studies that have directly correlated the haemodynamic response with spectral fluctuations in local field potentials have found a direct relation between HFA (also referred to as gamma activity) and the BOLD signal (Logothetis et al., 2001; Mukamel et al., 2005; Niessing et al., 2005; Goense & Logothetis, 2008; Scheeringa et al., 2011; Conner et al., 2011). A subset of these studies additionally report that LFA is inversely related to the BOLD signal (Goldman et al., 2002; Niessing et al., 2005; Mukamel et al., 2005; Conner et al., 2011; Scheeringa et al., 2011) suggesting that the pattern of activation observed in Figure 1 and the haemodynamic response represent overlapping information.

Despite this evidence, there are important caveats linking the spectral changes in Figure 1 to a general activation signal. For example, the positive correlation between MUA and HFA is not absolute; the degree of underlying inter-neuronal correlation (Nir et al., 2007) and the phase of low-frequency oscillations (Rasch et al., 2008) have been shown to influence the MUA/HFA relation. These findings may partly explain why some neurons display an inverse relation between HFA and MUA (Manning et al., 2009). Additionally, some research suggests that LFA is positively correlated with BOLD (Goense & Logothetis, 2008; Ekstrom et al., 2009), that the relation between LFA and BOLD may vary as a function of anatomical region (Ekstrom, 2010; Conner et al., 2011), and that LFA co-varies with specific features of BOLD activation (Magri et al., 2012). Thus, although we find that relative decreases in LFA are the most dominant electrophysiological pattern observed during encoding (Figure 1), there may be more complex and nuanced LFA activation patterns embedded within this result. In summary, the meaning of spectral changes accompanying memory formation is a topic of on-going research. However, converging evidence suggest that increases in HFA are best conceptualized as a cognitive “activation signal” that can be used to map memory formation (Lachaux et al., 2012).

4.2 Comparison with haemodynamic studies of memory formation

If HFA reflects general neural activation, then there should be a strong correspondence between the anatomical areas implicated in memory formation using both HFA and the BOLD signal in the subsequent memory (SM) paradigm (Paller & Wagner, 2002). Although previous studies have shown such memory activation maps using intracranially recorded HFA (Sederberg et al., 2007, 2007b), the current study represents the largest intracranial electrophysiological study of SM, which allowed us to map HFA across a larger percentage of the brain (Figure S1) and with higher spatial resolution. Using a time window for HFA that is roughly equivalent to the temporal resolution in fMRI studies (see Figure 2), it is thus possible to compare, on a large-scale, the effect of SM on HFA and the BOLD signal. To facilitate this comparison, the reader is referred to a recent meta-analysis of 74 haemodynamic studies implementing the SM paradigm (see Figures 1 and 2A Kim, 2011). In terms of similarities, both the L. IFG and the bilateral fusiform areas exhibit strong HFA and BOLD SM effects (see also Paller & Wagner, 2002). Additionally, there is overlap in many smaller areas of activation including the bilateral PPC (although the HFA map shows a stronger activation in left temporal parietal junction; see also Uncapher & Wagner, 2009; Cabeza et al., 2012), the R. IFG and pre-motor cortex, the bilateral MTL (see also Henson, 2005; Ranganath, 2010), as well as the L. VLTC or middle temporal gyrus (see also Staresina & Davachi, 2006; Blumenfeld et al., 2011). One inconsistency with the Kim et al. meta-analysis, however, is that that subsequent forgetting (SF) effects were much less frequent in the HFA maps, perhaps due to the fact that the regions commonly showing SF effects (especially the precuneus region) are rarely sampled using intracranial electrodes.

4.3 Timing information

Using HFA as a marker of subsequent memory allowed us to go beyond previous functional imaging studies and provide direct information regarding the temporal sequence of regional activations during encoding (Figures 3-5), which yielded three novel findings.

First, prior to stimulus presentation, increased HFA in the right posterior inferior temporal cortex predicted subsequent recall (Figure 3, top panel). Pre-stimulus activity that co-varies with successful encoding of upcoming items is theoretically important because it indicates that memory formation is influenced by item-independent activity and fluctuations in on-going cognitive states (Linkenkaer-Hansen et al., 2004; Wyart & Tallon-Baudry, 2009). The role of pre-stimulus activity in the memory system is speculative, but converging evidence suggests that such activity reflects the expected reward-value of the upcoming item (Adcock et al., 2006; Gruber et al., 2013). Previous haemodynamic studies have identified pre-stimulus effects that predict future successful encoding (Adcock et al., 2006; Park & Rugg, 2010). Thus, one would expect that HFA, which has been directly correlated with the BOLD signal, would be similarly increased during the pre-stimulus interval. However, previous electrophysiological studies examining pre-stimulus memory effects have exclusively found an increase in theta/alpha activity before word presentation (Guderian et al., 2009; Fell et al., 2011; Addante et al., 2011; Fellner et al., In Press), although most of this work was done non-invasively making HFA diffcult to detect. Thus, the current results are the first to show that pre-stimulus effects extend into the HFA frequency range (> 60 Hz) and provide a key link between haemodynamic and electrophysiological findings.

Second, we found that the L. IFG activated relatively late during memory formation, reliably after the fusiform and parahippocampal regions. This finding can be seen in the activation maps in Figure 3 and was directly tested and confirmed by comparing the time to peak activations for these regions (Figure 4). The finding that the L. IFG activates after the L. fusiform and parahippocampal areas helps constrain the hypothesized role of these two regions during memory formation. In particular, activity in the L. IFG has been hypothesized to reflect pre-hippocampal “content-processing” (Buckner et al., 1999; Wagner et al., 1999; Kim, 2011), top-down modulation of posterior regions (Paller & Wagner, 2002), control-processes during memory encoding (Buckner, 2003), and the selection and organization of mnemonic information stored in other anatomical regions (Blumenfeld & Ranganath, 2007). Similar to the L. IFG, the fusiform and parahippocampal areas have also been hypothesized to play a role in pre-hippocampal “content-processing” of items (Kim, 2011). However, our findings suggest that the L. IFG and the fusiform/parahippocampal regions activate with very different relative time courses and thus likely participate in different functional networks during encoding. Specifically, because the L. IFG activates very late during memory formation alongside the L. PPC and the L. VLTC (Figure 5B), it is unlikely that the L. IFG act as as a pre-hippocampal “content-processing” buffer. Instead, we speculate that the L. IFG organizes and controls semantic processing in the L. VLTC and internal attentional processes in the L. PPC (see Uncapher & Wagner, 2009) during late stage memory encoding (Blumenfeld & Ranganath, 2007; Badre & Wagner, 2007).

Third, the timing of hippocampal HFA warrants particular discussion; despite the fact that the temporal profile of hippocampal HFA clustered with regions associated with early visual activation (Figure 5B), peak hippocampal activity occurred reliably after peak activity in early ventral visual ROIs (Figure 4B). These seemingly opposing results simply reflected that the underlying timing of hippocampal HFA fell between early visual activations and late left neocortical activations (see 400-900 ms panel in Figure 3). More quantitatively, Figure 4A shows that ROIs along the ventral visual pathway had peak activations near 500 ms, ROIs in the L. neocortex had peak activations near 1000 ms, and the hippocampal ROI peaked between these two temporal epochs. Despite these differences, the hippocampus clustered with the early ventral visual regions (Figure 5B) because, in both regions, there was not a prominent HFA component after the word was removed from the screen, unlike late L. neocortical activations (Figure 4A). From a theoretical standpoint, the fact that hippocampal activation bridges activity in early visual and late left neocortical areas may suggest that the hippocampus plays a role in the transition from visual perception to semantic retrieval of the item; however, further research is needed to investigate this speculation.

4.4 Conclusions

We have shown how timing data provided by HFA can help to categorize regional activations during memory encoding. Specifically, the aforementioned early- and late-stage anatomical pathways (Figure 5B) may represent separate networks, each with distinct cognitive functions, that mediate encoding. HFA thus provides a framework for understanding encoding as a dynamic activation of functional networks as opposed to a static map of regional activations.

Supplementary Material

02

Supplemental Movie S1. High-temporal resolution movie of HFA during memory encoding (50% real-time speed). For all 341 time windows, all spherical regions that exhibited a significant (permutation procedure; 10,000 iterations; FDR corrected for multiple comparisons; q = 0.05) change in power with successful memory encoding are displayed on a standardized three-dimensional brain. Increases and decreases in power are shown in red and blue, respectively. The color scale reflects the percentage of nearby ROIs exhibiting identical encoding-related effects. The horizontal dashed line on the sagittal views corresponds to the level of the axial cut in the third panel (the z = −13.0 plane in Talairach space). Grayscale rendering represents the percentage of spherical regions surrounding a given location with at least five patients. Time windows were sequentially concatenated at a frame rate of 50 frames-per-second to create a movie corresponding to 50% of real-time speed.

03

Supplemental Movie S2. High-temporal resolution movie of HFA during memory encoding (25% real-time speed). Identical to Supplemental Movie S1, however the frame rate was set to 25 frames-per-second, which corresponds to 25% of real-time speed.

Highlights.

• Human memory formation involves early and late neural activations

• Early activations included higher-order visual areas culminating in the hippocampus

• Late activations included left neocortical regions involved in semantic processing

Abbreviations

HFA

High-frequency activity

SM

Subsequent memory

LFA

Low-frequency activity

IFG

Inferior frontal gyrus

VLTC

Ventrolateral temporal cortex

PPC

Posterior Parietal Cortex