A human intracranial study of long-range oscillatory coherence across a frontal–occipital–hippocampal brain network during visual object processing

Abstract

Visual object-recognition is thought to involve activation of a distributed network of cortical regions, nodes of which include the lateral prefrontal cortex, the so-called lateral occipital complex (LOC), and the hippocampal formation. It has been proposed that long-range oscillatory synchronization is a major mode of coordinating such a distributed network. Here, intracranial recordings were made from three humans as they performed a challenging visual object-recognition task that required them to identify barely recognizable fragmented line-drawings of common objects. Subdural electrodes were placed over the prefrontal cortex and LOC, and depth electrodes were placed within the hippocampal formation. Robust beta-band coherence was evident in all subjects during processing of recognizable fragmented images. Significantly lower coherence was evident during processing of unrecognizable scrambled versions of the same. The results indicate that transient beta-band oscillatory coupling between these three distributed cortical regions may reflect a mechanism for effective communication during visual object processing.

Partial occlusion, camouflage, and low lighting are just a few of the everyday environmental circumstances that can cause visual information to be incomplete. The ability to identify objects accurately, despite such situations where limited information is available, is a fundamental and rather remarkable aspect of human object-recognition. The term “perceptual closure” has been used to refer to the neural processes responsible for the filling-in of missing information under such partial viewing conditions (1).

Line drawings of common objects that are systematically fragmented to varying degrees have been used in a number of studies to assess the timing of the neural processes responsible for perceptual closure (2, 3). Previously, we characterized a robust event-related potential component that we termed the “N_CL” (for “negativity associated with closure”), which tracked these closure related processes. This component was manifest as a relative negativity over bilateral occipitotemporal scalp and occurred in the 230- to 400-ms time frame (2). By combining high-density event-related potential (ERP) recordings with functional magnetic resonance imaging (fMRI), we also determined that the major generators of the N_CL reside in a tract of cortex known as the lateral occipital complex (LOC) (4). The LOC is a cluster of regions that includes the dorsolateral-occipital lobe close to area MT/V5 and the ventral fusiform area (5). Functional imaging studies have also strongly implicated the LOC in visual object recognition (4–6). Although the LOC clearly plays a substantial role in visual object processing, it does not do so in isolation. Regions of the frontal lobes and also of middle temporal lobe (MTL) structures have been variously implicated in these processes (7, 8). These regions may be part of a large-scale neurocognitive network for object recognition. The mechanisms of interplay between the nodes of this putative network are of considerable interest and are at the basis of a number of models of visual object processing (4, 9).

Here, we took advantage of the relatively rare opportunity to record intracranially from humans performing a perceptual closure task while electrodes were implanted both in and over regions of this hypothesized network. We asked whether closure-related activity is evident in intracranial recordings from the LOC, frontal cortex, and hippocampal formation (a major deep component of the MTL) and whether oscillatory activity is coherent across the network, as it works to “close” fragmented images. The timecourse of activities observed in previous high-density scalp ERP recordings of closure has revealed periods of concurrent activity in the posterior visual processing streams (LOC) and prefrontal cortical regions (2, 4, 10). Prefrontal regions have been shown to be involved in processes such as guessing and hypothesis generation (11). These observations led us to propose a convergent model for closure processing (4) whereby feed-forward processes conveying perceptual information from posterior sensory areas converge with feed-back projections from frontal areas, limiting the number of possible matches with a set of stored representations (9). Although the timecourse of scalp ERPs lends support to such a model, the exact mechanism for the convergence of information and the role of deep MTL structures in the processing of representations (8, 12) remain to be properly delineated.

Recently, there has been renewed interest in the possible involvement of MTL structures in visual perception (13). Several nonhuman primate studies have revealed a critical role for perirhinal cortex in perception when stimuli contain a high degree of feature ambiguity (12). Indirect evidence for MTL involvement in perceptual processes also comes from studies in patients with MTL lesions who show deficits in recognizing complex features (14). Yet this brain region has not been nearly as extensively explored for its role in perceptual processing as it has for its primary role in memory storage processes (15, 16).

Characterization of the spatiotemporal dynamics of MTL structures, using scalp recorded ERPs, is not trivial, because linear interactions of voltages on the scalp and the relative distance of scalp electrodes from deeper brain structures make it particularly difficult to tease apart the contributions from these deeper generators. Functional imaging studies, of course, afford excellent spatial resolution and provide valuable information regarding the functions of these midline structures, but lack the temporal resolution of ERPs. Intracranial recordings closer to current sources, when possible, resolve many of these problems. In the present study, local neural responses were recorded directly from the surface of the cortex (subdurally) over both LOC and prefrontal cortical regions and from depth electrodes implanted stereotactically in the hippocampal formation. The first goal was to examine the presence and relative onsets of closure-related activity. Synchronous high frequency oscillatory activity has been proposed as a mechanism providing a functional link between spatially segregated cortical areas (17–23). Therefore, the second goal was to determine whether oscillatory activity in the LOC, prefrontal cortex, and deep MTL structures was synchronized.

Results

Inspection of averaged responses to each of the unscrambled (images that could be closed) and randomly scrambled stimuli at electrode sites lying over the visual system revealed the classic visual evoked potential components P1 and N1 (Fig. 1). The responses to the unscrambled and scrambled conditions did not differ during the P1 or the N1, but showed clear divergence in the timeframe of the N_CL (see Fig. 1). The P1 and the N1 were visible in two adjacent electrode sites in DM and BL and in one site in MV. The generators of N_CL proved to be highly localized among the lateral occipital sites that we recorded from, with significant divergence appearing in only a single one of these electrode sites for each of the subjects. These differences reached statistical significance beginning at 228, 294, and 410 ms for MV, DM, and BL, respectively, and continued to diverge significantly throughout the remainder of the analyzed epoch for DM and BL (i.e., to 700 ms). The amplitude of the N1 was greatest at the electrode site at which the N_CL was observed, consistent with a shared underlying neural generator in the LOC. Our data also revealed a significant differential ERP response to the unscrambled condition with respect to the scrambled condition in the hippocampal formation. The onset of the significant divergence was found to be at 570, 250, and 364 ms in the case of MV, DM, and BL, respectively (Fig. 1). Over frontal regions, the response to the scrambled and unscrambled conditions showed a slow-going positivity that diverged between the conditions, but this did not reach significance by our criteria.

Fig. 1.

Event-related potentials. (Upper) Responses over lateral occipital cortex (LOC). ERP responses to the unscrambled and scrambled stimuli recorded from three neighboring sites within lateral occipital cortical region in participants DM (Left), BL (Center) and MV (Right). [Note that 3D reconstruction of MV's brain was not possible; the positions of the electrodes in this case are therefore presented on a standard Talairach-transformed brain (63)]. The electrode site represented in red displays a significant divergent ERP response at 294 ms in DM (Talairach: X 44, Y −80, Z 2), at 410 ms in BL (Talairach: X 29, Y −74, Z −13) and at 228 ms in MV (Talairach: X 44, Y −66, Z −15). The ERP response to the unscrambled stimulus condition (in red) was relatively more negative when compared with the scrambled condition (in green). Significant differential responses are within the dotted box. (Lower) Responses in the hippocampal formation. ERP responses to the unscrambled and scrambled stimuli recorded from the hippocampal electrode site represented as a red dot, in the coronal MRI slice of participants DM (Left) (Talairach: X 31, Y −27, Z −6), BL (Center) (Talairach: X 32, Y −12, Z −15), and MV (Right) (Talairach: X 23, Y −12, Z −19). The electrode site shows a significant divergent ERP response at 250 ms in DM, 364 ms in BL, and 570 ms in MV. Significant differential responses are within the dotted box.

The functional role of the large scale network of brain regions posited to be involved in closure processing was evaluated by assessing significant differences between the coherence in one experimental condition versus the other. Significant (P corrected <0.05) differential coherence (effect of condition) was observed from 150 to 225 ms in the frequency range of 20–26 Hz for MV, 200 to 300 ms in the frequency range of 14–18 Hz for DM, and 400 to 475 ms in the frequency range of 18–24 Hz for BL in all three pairwise comparisons, i.e., LOC-hippocampal, LOC-frontal, frontal-hippocampal (Figs. 23–4). Table 1 shows the coherence values for the above time-frequency windows in each condition along with the corresponding baseline coherence values. These baseline values were not significant and did not differ significantly between the two conditions. A window of 200-ms duration centered around these respective time windows revealed significant (P corrected <0.05) beta coherence (including the time frequency window where significant differential coherence was observed) and gamma coherence, compared with the baseline, in each condition, for each pairwise combination of sites. In contrast to the differential beta coherence, no significant difference was observed between conditions in gamma coherence.

Fig. 2.

Coherence across the object-recognition network. The figures display the coherence measures for LOC-hippocampal, LOC-frontal and, hippocampal–frontal electrode pairs for the unscrambled and the scrambled conditions in each individual. The plots at the center, display the time-frequency window where for each pairwise comparison significant (P corrected <0.05) differences in beta-band coherence measures were observed in each individual. The functional topology described here represented a spatiotemporal pattern where all of the three regions showed a coherence pattern that was significant in the same time-frequency window for each condition, and showed a common differential coherence pattern, which temporally corresponded with the onset of the N_cl (blue dotted line). The Talairach positions for BL are as follows: LOC (X 40, Y −73, Z −6), hippocampus (X 32, Y −12, Z −15), and frontal (X 48, Y 26, Z 13).

Fig. 3.

Coherence across the object-recognition network. (see legend for Fig. 2 for more details.) The Talairach positions for DM are as follows: LOC (X 44, Y −80, Z 2), hippocampus (X 31, Y −27, Z −6), and frontal (X 54, Y 18, Z 33). The corresponding time-frequency windows are reported in Table 1.

Fig. 4.

Coherence across the object-recognition network. (See legend for Fig. 2 for more details.) The Talairach positions for MV are as follows: LOC (X 44, Y −66, Z −15), hippocampus (X 23, Y −12, Z −19), and frontal (X 42, Y 22, Z 43). The corresponding time-frequency windows are reported in Table 1.

Table 1.

Baseline coherence values in each subject, the time-frequency window where significant (P corrected < 0.05) differences in coherence were observed, the significant (P corrected < 0.05) coherence values for each condition, and the number of trials in each condition

Subject	Baseline coherence	Unscrambled coherence				Scrambled coherence				Time-frequency window
		LOC-frontal	LOC-hippocampal	Hippocampal-frontal	Trials, no.	LOC-frontal	LOC-hippocampal	Hippocampal-frontal	Trials, no.
DM	0.01–0.09 (0.05 ± 0.04)	0.40 ± 0.05	0.50 ± 0.10	0.38 ± 0.08	113	0.16 ± 0.08	0.24 ± 0.12	0.18 ± 0.12	118	200–300 ms; 14–18 Hz
BL	0.01–0.07 (0.04 ± 0.03)	0.27 ± 0.05	0.36 ± 0.06	0.50 ± 0.05	129	0.08 ± 0.02	0.14 ± 0.04	0.24 ± 0.03	138	400–475 ms; 18–24 Hz
MV	0.12–0.22 (0.17 ± 0.05)	0.40 ± 0.02	0.40 ± 0.06	0.52 ± 0.02	116	0.16 ± 0.04	0.19 ± 0.05	0.25 ± 0.03	106	150–225 ms; 20–26 Hz

The event-related desynchronization measure (ERD) revealed a relative suppression of power within a large frequency band (8–40 Hz) in a time frame corresponding to the closure-related processes. This suppression (ERD) of power was restricted to the sites over lateral occipital cortex (most evident in patient BL and DM), and did not differ significantly between the scrambled and unscrambled stimulus conditions. At the time-frequency window where significant coherence was observed, no significant (P corrected <0.05) differences in power were found between the conditions [supporting information (SI) Fig. 6]. Likewise, for each condition, no significant increase in power was observed at these time-frequency windows.

Discussion

These intracranial data provide direct electrophysiological evidence in humans for the involvement of a widely distributed cortical network in perceptual closure processes. They suggest that three major nodes of this network reside in the hippocampal formation, occipitotemporal cortex, and lateral prefrontal cortex (Figs. 2–4) and that a major mode of communication between these three regions relies on long-range synchronous beta-band oscillatory activity.

Oscillatory synchronized activity has long been proposed as a mechanism providing functional links between nodes of distributed cortical networks (10, 17, 18, 20, 24). Synchronized activity within neural networks has been reported in different frequency ranges including 30–100 Hz gamma (25), 14–30 Hz beta (26), 9–13 Hz alpha (27), and 5–9 Hz theta/gamma (28). It was suggested in ref. 29 that neural networks perform specialized functions within a preferred frequency range (e.g., beta oscillations during motor imagery). For example, synchronized oscillatory activity in the beta range between distinct ventral visual-stream areas has been reported when subjects were required to perform a delayed match-to-sample task, suggesting a role in working memory maintenance (20). A recent MEG study (10) demonstrated synchronous activation in the alpha/low-beta bands between prefrontal cortex and ventral visual stream areas when subjects identified briefly presented masked visual stimuli. Recently, we reported increased beta-band activity in both sensory and frontal regions during multisensory integration of audiovisual inputs (30), suggesting that beta might be one mode of communication between the separate sensory systems during multisensory object-formation. The amplitude of this beta activity also correlated with average reaction times in the task.

The present study supports and extends these findings, providing direct evidence for the synchronous engagement of prefrontal cortex, the LOC, and the hippocampus during perceptual closure. The beta-band synchrony involving the hippocampal formation suggests a process that is distinct from the previously characterized theta oscillatory behavior of this region (14), a frequency band optimal for bidirectional modulation of long-term potentiation (LTP), and long-term depression (LTD) of synaptic strength (31). Our data provide evidence for the involvement of beta-band synchrony in the MTL in the mechanism of engagement of the distributed cortical network at the onset of the “comparator” function and suggests a central role for this region in the process of object recognition under ambiguous stimulus conditions.

Whether beta band synchronies also selectively enhance long-term synaptic plasticity involved with either learning and/or recall remains to be determined. The observed simultaneous generation of beta and gamma synchrony (and perhaps theta in different tasks) could allow both functional binding of structures together and segregation of different processing and memory recall tasks by multiband frequency multiplexing.

In our data, we see that the onset of N_CL activity in the lateral occipital cortex was accompanied by significant N_CL-like differential activity in the hippocampal formation. This suggests to us that a matching process between sensory inputs and mnemonic representations may be produced by sustained interactions between the hippocampal formation and LOC during the N_CL timeframe. Studies in nonhuman primates have demonstrated the presence of bidirectional pathways between the inferotemporal (IT) cortex, the functional homologue of LOC in humans, and MTL (32).

MTL structures are more typically associated with long-term memory processes (33). The interaction between MTL and neocortex has been found to play a role in processes that bind the distributed cortical sites that represent a memory (15). Recent studies, however, suggest that the MTL may also play a prominent role in processes involving short-term memory (34) and higher order perception (8, 12, 35, 36), challenging the idea that MTL exclusively subserves long-term memory functions. Regions of the hippocampus, in particular circuits of the CA3 region, have been postulated to be well suited to perform a “comparator” function, where pyramidal neurons receive converging indices of current sensory input and of pointers to memory patterns stored in neocortex (37).

One model of MTL function suggests that the hippocampus might play a role in representing conjunctions of spatial information (12), a process that could be useful for recognizing spatial relations among disconnected/occluded elements of an object like those used here. MTL has also been implicated in the representation of complex conjunctions of features for objects with a high degree of ambiguity (8). According to this model, the MTL is engaged when temporal and/or environmental context needs to be learned and recognized later. The present findings provide further support for the theory that the hippocampus is involved in binding of perceptual components in object recognition, perhaps by supporting relational and working memory (34) mechanisms that are particularly relevant in the context of perceptual closure.

An assessment of the relative timing of activation onsets across the nodes of this network raises some intriguing possibilities about how closure processes might be initiated and what role the various nodes of the network may play. In two of the three subjects, the onset of N_CL-like divergence occurred earliest in hippocampus (Fig. 1), some 40–50 ms before N_CL divergence was seen in the LOC. In these two subjects, significant differential coherent oscillatory beta coupling occurred in the time period between the start of N_CL-like hippocampal activity and subsequent N_CL divergence in LOC. In the third subject, significant differential oscillatory coherence also preceded N_CL divergence in LOC, but unlike the other two subjects hippocampal N_CL-like divergence did not reach statistical significance until considerably later. This could simply represent individual differences in how the object-recognition network is engaged across individuals or could be due to suboptimal placement of the hippocampal electrode in this one subject. If the earliest closure-related activity is seen in the hippocampal formation, this would suggest that it plays a significant role in mediating a bidirectional dialogue with key neocortical regions such as the LOC, which results in oscillatory phasic binding of the structures in this network. We suggest that hippocampal-LOC coherence enables a process by which precise neurons and synapses become selected for activation that is phase-locked to the oscillations and that this selection underlies closure in a process similar to annealing or settling to a stable state that matches more complete stored patterns to current incomplete sensory input.

Nonhuman primate studies have shown that frontal regions are involved in memory retrieval of visual object representations in inferotemporal cortex (IT) (38), and that there are direct connections from IT and area TE of the temporal lobe to prefrontal cortex (39). In addition, bidirectional pathways exist connecting the prefrontal cortex with the hippocampal formation and parahippocampal cortex (40) and are associated with functions involving both working memory and semantic information processing (7). It has been suggested that frontal regions play a role in hypothesis generation for the purpose of object recognition (9). The coherent activation between the LOC, hippocampus, and prefrontal regions observed here in the time-frequency domain suggests the existence of a perceptual binding mechanism that functionally integrates these regions at the onset of closure processing and may be necessary for closure to occur.

One important issue that remains unresolved in these data is the nature of the relationship between the two indices of closure processing that are measured, i.e., the low-frequency N_CL response and the considerably higher-frequency beta oscillations. Recent results from intracranial recordings in the auditory cortex of nonhuman primates may shed some light on this issue. Lakatos et al. (41) found a striking hierarchical organization of the various prominent oscillatory bands of the EEG during processing of simple auditory tonal stimuli, such that the amplitude of high-frequency gamma activity (30–50 Hz) was modulated by the phase of the lower frequency theta (4–19 Hz) activity, which was in turn modulated by the phase of delta (1–4 Hz) activity. Similarly, Canolty et al. (42) found phase-relationships between theta and high-gamma (80–150 Hz) oscillations during human intracranial recordings over frontal and temporal regions and suggested that transient coupling between the low and high frequency oscillations was a means by which distributed cortical regions communicated during stimulus processing. Although not explicitly tested here, our results suggest that the significant changes in beta-coherence seen during closure occur subsequent to the initiation of slow N_CL-like activity in hippocampus. In light of the Lakatos and Canolty findings, the obvious implication is that higher-frequency beta-band activity may be phase modulated by the lower-frequency N_CL activity, but this remains a speculation here, and clearly further studies are called for to elucidate the relationship of these brain processes.

In closing, although we show clear coherent activity across three well established nodes of the visual object-recognition circuit here, it is also important to note that these three regions are almost certainly part of a larger network, with additional regions that were not sampled from with the available electrode coverage here.

Materials and Methods

A detailed description of the methods is provided in SI Methods.

Subjects.

Intracranial recordings were obtained from three patients (BL, 35 years old; DM, 41 years old, and MV, 47 years old) with intractable epilepsy, where pharmacological approaches had proven insufficient to control seizures. The three patients presented here were of particular interest because, in addition to large arrays of subdural grid electrodes, depth electrodes were also placed stereotactically in the hippocampal formation. No seizure activity was observed in the structures investigated here. All three participants were on antiseizure medications at the time of recording. All neuropsychological tests were within normal limits, and language was left lateralized as determined by Wada testing (43). The participants provided written informed consent, and the procedures were approved by the Institutional Review Boards of both the Nathan Kline Institute and Weill Cornell Presbyterian Hospital.

Stimuli and Experimental Design.

Fragmented line drawings, here referred to as “unscrambled pictures” (1), and their scrambled versions, serving as control stimuli, were presented to the participants (see Fig. 5 and SI Methods). Previous studies have shown that at this level of fragmentation participants identify the unscrambled pictures 73% of the time. It is also important to note that, even in cases where the incomplete object is not identified, participants experience the objectness of the image, and the N_CL is still generated (2, 4). A total of 400 unique images, 200 unscrambled (incomplete), and 200 scrambled, were presented over eight consecutive runs. The images were randomly distributed with equal distribution in each run. Each image was presented for 750 ms, followed by a blank screen for 800 ms. A “Y/N” (yes/no) response prompt subsequently appeared for 200 ms and was followed by another blank screen for 2,200 ms. The participants' response windows extended for 2,300 ms from the onset of the Y/N prompt. The duration of each run was ≈3.25 min. After the presentation of the Y/N prompt, participants were instructed to press the left mouse button when they recognized the image as an object and the right mouse button when they did not.

Fig. 5.

Stimulus paradigm. Examples of the stimuli. Unscrambled (incomplete) line-drawings are shown at Left, and scrambled line-drawings are shown at Right. The figures bell (Top), axe (Middle), and arrow (Bottom).

Data Acquisition.

Continuous EEG from intracranial electrodes was acquired by using the BrainVision amplifier system. These electrodes are highly sensitive to local field potentials (LFP) generated within ≈4.0 mm² area and are much less sensitive to distant activity (44, 45). A frontally placed intracranial electrode served as the reference. The data were bandpass-filtered online from 0.05 to 100 Hz and digitized at 1,000 Hz. Data were analyzed offline (46), using a ±600 μV artifact rejection criterion. The ERP recordings were not obtained immediately before or after seizures. High-resolution presurgical MRIs were coregistered with postsurgery MRIs for the calculation of the electrode coordinates and reconstruction into 3D images.

Data Analysis.

Epochs of continuous EEG extending from −300 ms before stimulus to 700 ms after stimulus for both unscrambled and scrambled stimulus conditions were used to compute the visual evoked potential. The baseline was defined as the 200-ms prestimulus interval. To obtain a measure of robustness of the responses at the intracranial electrode sites for each participant, we measured the variance across the single-trials (47). Differences were considered meaningful if there was no overlap of the evoked potentials within the range of one standard error (48) for at least 10 consecutive data points (≥ 20 ms at a 500-Hz digitization rate) (49). To obtain the envelope amplitude and the phase of a specified frequency band as a function of time (46, 50, 51), time-frequency transformation was performed by using complex demodulation (50) for frequencies of 4–40 Hz in the time-window between −300 and 700 ms. Frequencies were sampled in 2-Hz steps; latencies were sampled in steps of 25 ms. This corresponds to a time-frequency resolution of ±2.83 Hz and ±39.4 ms at each time-frequency bin (full width at half maximum). Baseline was defined as −200 ms to stimulus onset. From this time-frequency transformation, event-related (de)synchronization measures, also termed temporal spectral evolution (TSE) were derived, defined as the relative power change at a time-frequency bin compared with the mean power over the baseline epoch for that frequency (24). In the next step of the analysis, the functional role of the large scale network of brain regions posited to be involved in closure processing was evaluated. In this step, we investigated significant differences between the coherence, a measure of the cross-correlation between two signals in the frequency domain, as an index of functional connectivity (52, 53), in one experimental condition versus the other (54). In each condition, coherence estimates were computed (55) for pairwise combination of sites, i.e., LOC-hippocampal, LOC-frontal, frontal-hippocampal (see SI Methods for a formal description of the measure of coherence).

Before the analysis of TSE and coherence, we removed the event-related signal from each single-trial (see SI Methods) to analyze induced activity only (i.e., activity that is not phase-locked to the stimulus) and to ensure stationarity (56, 57). Statistics in the time-frequency domain were performed in two steps: First, uncorrected P values were obtained for the parameters of interest (TSE, coherence). In the second step, these values were then corrected for multiple testing.

Step 1.

TSE values were tested for significance by means of a Studentized bootstrap method as described in ref. 58. To test for significant differences of TSE values between two conditions, a two-sided permutation test was performed. To determine the probability that coherence at a particular time-frequency sampling point is significantly higher than what is expected from random fluctuations was investigated based on an approach suggested by ref. 59. To investigate the probability that the coherence in one condition differed significantly from the coherence in the other condition, we followed the approach given in refs. 54 and 60.

Step 2.

Correction for multiple testing was performed on the P values obtained by the above-mentioned methods. A modification of the Bonferroni correction developed by Simes (61) was used as suggested for time-frequency analysis by Auranen (62). The corrected significance level α was set to 0.05. A detailed description of the analyses is provided in SI Methods.