Skip to main content
NCBI home page
The Journal of Neuroscience logoLink to The Journal of Neuroscience
. 2007 Jun 6;27(23):6234–6242. doi: 10.1523/JNEUROSCI.4627-06.2007

Enhanced Category Tuning Revealed by Intracranial Electroencephalograms in High-Order Human Visual Areas

Eran Privman 1, Yuval Nir 3, Uri Kramer 2,5, Svetlana Kipervasser 2,4, Fani Andelman 5, Miri Y Neufeld 2,4, Roy Mukamel 6, Yehezkel Yeshurun 1, Itzhak Fried 2,5,7, Rafael Malach 3,✉
PMCID: PMC6672161  PMID: 17553996

Abstract

The functional organization of human sensory cortex was studied by comparing intracranial EEG (iEEG) recordings of local field potentials in neurosurgical patients with functional magnetic resonance imaging (fMRI) obtained in healthy subjects. Using naturalistic movie stimuli, we found a tight correlation between these two measures throughout the human sensory cortex. Importantly, the correlation between the iEEG and fMRI signals was site-specific, exhibiting neuroanatomically specific coupling. In several cortical sites the iEEG activity was confined strictly to one object category. This site selectivity was not limited to faces but included other object categories such as houses and tools. The selectivity of the iEEG signals to images of different object categories was remarkably higher when compared with the selectivity of the corresponding fMRI signals. A plausible interpretation of the fMRI and iEEG results concerns cortical organization in which object categories are organized in a mosaic of narrowly tuned object-selective clusters.

Keywords: electrocorticography, ECoG, intracranial, EEG, fMRI, visual selectivity, object recognition

Introduction

Functional mapping of human ventral stream visual areas via functional magnetic resonance imaging (fMRI) (Malach et al., 1995, 2002; Kanwisher et al., 1997; Haxby et al., 2001; Grill-Spector, 2003) has extended earlier electrophysiological studies (Halgren et al., 1994, 2000; Klopp et al., 2000) concerning the functional selectivity and shape-processing properties of these areas.

The issue of selectivity for object category is of importance to models of object representations, which are at the core of the controversy regarding distributed (Haxby et al., 2001), “complex” (Malach et al., 1995), or specialized domains of object representations (Kanwisher et al., 1997). Because of the relatively low spatial resolution of conventional fMRI techniques, this issue is difficult to resolve. Our previous study (Avidan et al., 2001), which used the fMRI adaptation paradigm (Grill-Spector and Malach, 2001), suggested the existence of “hot spots” within face-selective cortex in which the selectivity is narrower than that measured with conventional fMRI. In inferotemporal cortex of behaving monkey it was reported that large clusters of neurons show high selectivity specifically for faces (Tsao et al., 2006). It thus seems that the degree of selectivity for object category is inconsistent across studies. We hypothesized that this inconsistency might be related to the discrepancy between human studies, which rely on the blood oxygenation level-dependent (BOLD) signal, and the electrophysiological recordings in primates.

At the most fundamental level it is still unclear to what extent the BOLD signal reflects electrical neuronal activity. Early studies of fMRI and intracranial EEG (iEEG) (Puce, 1995) have established the relation of BOLD to local field potentials (LFPs) in humans. Important progress in this field has been obtained via simultaneous recordings of LFPs and hemodynamic signals in animals (Logothetis et al., 2001; Niessing et al., 2005). Recently, using movie stimuli (Hasson et al., 2004) that evoked highly reproducible activity, we have demonstrated a tight coupling among neuronal firing rates, high-frequency LFP, and BOLD in human auditory cortex (Mukamel et al., 2005).

In the present study we examined the issue of selectivity to visual object categories via a combination of iEEG and fMRI. In the first stage we examined the extent to which these two techniques reflect similar neuronal processes. We recorded electrical neuronal activity by using iEEG in neurosurgical patients (Allison et al., 1999). This neuronal activity then was compared with fMRI signals measured in healthy individuals under identical stimulation conditions (Mukamel et al., 2005). Our results show a highly significant and localized coupling between iEEG and fMRI activations across the human occipital and temporal cortices.

In the second stage we compared the tuning characteristics as reflected in the iEEG signal with those measured with fMRI in the corresponding regions. Our results show that iEEG recordings reveal a significantly sharper neural tuning than that reported in fMRI studies. Importantly, this result was not limited to face and house stimuli but also was found for the category of manmade objects. Thus our results confirm and extend the findings of strong selectivity to object category in human ventral stream object areas.

Materials and Methods

Data acquisition

Intracranial EEG

Recordings of electrical activity with the use of intracranial subdural electrodes were obtained from seven patients with medically intractable epilepsy who were evaluated for possible surgery (Dewar et al., 1996). In total, 275 electrode recording sites were obtained. Electrode location was based solely on clinical criteria by which each patient was implanted with subdural electrode arrays containing 40–80 contact electrodes (Adtech, Racine, WI). The spatial coverage of the recordings was different in each subject but typically included temporal (70%), occipital (8%), parietal (7%), and frontal (15%) cortices. Electrodes were arranged in one-dimensional strips or in two-dimensional grids placed directly on the cortical surface. Each electrode was 2 mm in diameter, with 8 mm spacing between adjacent electrodes. We used monopolar recordings referenced to an extracranial electrode. Eye movements were not measured.

Patients provided written informed consent to participate in the experiment. The Tel Aviv Sourasky Medical Center Committee for Activities Involving Human Subjects approved the experimental protocol. All sessions were conducted at the patient's quiet bedside while the patient was sitting upright in bed. Stimuli were presented via a standard laptop screen and speakers. The visual viewing angle was 15° for the movie experiment and 8° for the object category experiment (see below). Intracranial EEG data were filtered electronically between 1 and 70 Hz and were sampled at a rate of 200 Hz by the clinical equipment (Grass Technologies, West Warwick, RI). Stimulus-triggered electrical pulses were recorded along with the iEEG data for precise synchronization of the stimuli with the electrical responses. All experimental sessions were conducted after periods of at least 12 h without any identifiable seizures. Recording sites exhibiting signs of ictal epileptiform activity (supplemental Table 3, available at www.jneurosci.org as supplemental material) were excluded from the analysis. Electrode locations were determined by coregistration of computed tomographic (CT) scans after electrode implantation with preoperative MRI, using iPlan Stereotaxy software (BrainLAB, Westchester, IL).

Functional MRI

Five healthy subjects participated in the experiments. All subjects provided written informed consent. The Tel Aviv Sourasky Medical Center Committee for Activities Involving Human Subjects approved the experimental protocol. Subjects were scanned in a 1.5 T scanner using a standard head coil. BOLD contrast was obtained with a gradient echo echo-planar imaging sequence [echo time (TE) = 55; flip angle = 90°; field of view, 24 × 24 cm2; matrix size, 80 × 80; slice thickness of 4 mm and 1 mm gap]. In the movie experiment nine slices and repetition time (TR) of 1000 ms were used, whereas for the object category experiment (separate scan) 17 slices and TR of 3000 ms were used.

Experimental design

Naturalistic movie experiment

iEEG setup.

Nine minutes of an audiovisual movie clip from the popular western The Good, the Bad, and the Ugly were presented twice to six patients (Fig. 1a). The subjects' task was simply to follow the plot. Between the two presentations there was a break of 30 min during which other stimuli were shown to the patients. For details regarding the audiovisual movie, see Mukamel et al. (2005).

Figure 1.

Figure 1.

Open in a new tab

iEEG experimental design. a, Naturalistic movie experiment. The patients were presented twice with the same 9 min audiovisual movie segment from The Good, the Bad, and the Ugly. b, Object category experiment. Stationary pictures from three categories (faces, houses, and tools) were presented for 250 ms, with a 750 ms intertrial interval of fixation.

fMRI setup.

The same movie clip presented to the iEEG patients was presented once to five healthy subjects. These data are identical to those used in Mukamel et al. (2005).

Object category experiment

iEEG setup.

Stationary gray scale pictures from three categories of faces, houses, and tools were presented for 250 ms, followed by a 750 ms interval of a gray screen. A small red fixation dot was superimposed on the pictures. Examples of the stimuli are shown in Figure 1b. The patient's task was to fixate on the central fixation dot and to perform covertly a one-back memory task.

fMRI setup.

Images were identical to those presented to the iEEG patients, but images were presented consecutively in a block design rather than in an interleaved manner. Thus images of each category were presented in blocks of 9 s after a 6 s rest period. For more details, see Hasson et al. (2003).

Data analysis

iEEG data were analyzed by using custom-developed MatLab software (MathWorks Natick, MA), and the fMRI data were analyzed with BrainVoyager (Brain Innovation, Masstricht, The Netherlands) and custom-developed MatLab software.

Naturalistic movie experiment: iEEG

The methodology we used in analyzing the movie experiment followed our previous studies (Hasson et al., 2004; Mukamel et al., 2005). The 50 Hz electrical interference was removed off-line by using a zero-phase second-order Butterworth notch filter (MatLab, MathWorks). For the continuous recordings of the movie experiment in which standard rejection of noisy epochs was not possible, we performed global noise filtering by subtracting the averaged signal of all of the electrodes from each electrode. This procedure discards non-neuronal contributions related to the extracranial reference electrode, which may have affected all intracranial channels.

Electrodes responding to the movie stimuli were identified by examining the reproducibility of response across the two movie presentations. Electrodes were classified as reproducible in a given frequency band if the correlation between band-limited power (BLP) changes in that band across the two runs exceeded r = 0.2 [following Mukamel et al. (2005)]. BLP changes were extracted by performing (1) bandpass filtering (α, 8–13 Hz; β, 13–30 Hz; γ, 30–70 Hz) with the use of a zero phase second-order Butterworth filter (MatLab), (2) computing the root mean square (RMS) of the filtered signal with the use of a sliding window of 50 ms, and (3) smoothing the result with a Gaussian of full width at half-maximum (FWHM) = 300 ms.

To find the correlations of iEEG activity during movie presentation across patients (when recorded in similar cortical fields), we compared, in pairs, the gamma BLP with those recorded from all electrodes of the other patients, with the threshold set at r > 0.2 similar to that used for reproducibility.

Comparison of iEEG and BOLD signals

We calculated the predicted fMRI response by convolving the gamma BLP of the iEEG signal, averaged across the two movie presentations, with a standard canonical hemodynamic response function (HRF) (Boynton et al., 1996). We used the HRF as described by Boynton and colleagues with fixed parameters (τ = 1250; δ = 2500). These parameters also were used by Mukamel et al. (2005) and are implemented as the default HRF in fMRI software packages (BrainVoyager). We then used the convolved iEEG signal as a regressor in a general linear model framework to identify the fMRI voxels that best correlated with it. The threshold was set at p < 0.05 after correction for multiple comparisons and was such that the area of the correlated patch was smaller than 300 mm2.

Object category experiment: iEEG

Preprocessing of the object category experiment data included removal of 50 Hz electrical interference as described above. In line with standard visual-evoked potential (VEP) analysis methods (Allison et al., 1999) we used the full frequency bandwidth of the signal for data analysis.

Averaged VEP

The average VEP was used as a first step to identify electrodes that were selective to object categories. The VEPs of each electrode triggered by the presentation of a particular object category were averaged (Allison et al., 1999; Tsao et al., 2006), yielding an averaged VEP. For reproducibility validation we also averaged odd and even trials separately and compared their waveforms.

Single trial analysis

VEPs also could be identified in the raw data without averaging. For every trial we searched for negative local minima in the VEP during the period of 50–250 ms after image presentation onset. The threshold was defined separately per electrode at approximately −3 SD so that the number of trials exceeding it was the same as the number of trials per category. The plots in Figures 4c and 6 were obtained by concatenation of raw signal segments corresponding to periods of 50–250 ms after stimulus onset.

Figure 4.

Figure 4.

Open in a new tab

Exemplar selectivity for object category in iEEG responses to stationary images. a, Mean VEPs of a face-selective electrode (located near the fusiform face area) while the patient was presented with various object images. In each category odd trials are plotted separately from even trials. b, Raw data plots of 20 consecutive face trials from the same face-selective electrode as in a. Note the highly consistent timing of the negative peak at ∼140 ms (arrow). c, A concatenation of raw data of all trials (for each trial we segmented a raw signal interval of 50–250 ms after stimulus onset) (see Materials and Methods); crosses mark the expected negative peak of the VEP 140 ms after stimulus onset. Red, Faces; green, houses; blue, tools. d, The estimated location of the electrode (red dot) as seen on a ventral view of the brain. L, Left; R, right; A, anterior; P, posterior.

Figure 6.

Figure 6.

Open in a new tab

Single-presentation VEP responses of category-selective iEEG electrodes. Shown are single-trial response profiles of four category-selective electrodes grouped by category. Similar to Figure 4c, each plot presents all of the trials of a category (of 54 trials) that elicited negative VEPs above threshold level (see Materials and Methods); signals were inverted for presentation purpose. Each row corresponds to a different electrode, and each column corresponds to a different object category (faces, tools, houses). a, b, Activities of two face-selective electrodes. c, Activity of a tool-selective electrode. d, Activity of a house-selective electrode. Note the striking object selectivity exhibited by each electrode.

Object selectivity index

For all selectivity analyses we defined a category selectivity index (CSI) (Tsao et al., 2006) as the following:

graphic file with name zns02307-3377-m01.jpg

A CSI index of 0 indicates no selectivity, i.e., an equal response to all object categories, whereas a CSI index of 1 indicates maximum selectivity (i.e., a significant response to one category and no response to all other categories). We defined the category responses in the formula as follows.

iEEG RMS.

This is the average RMS of all trials per category at the interval of 50–250 ms poststimulus onset minus baseline RMS as measured 50 ms before stimulus onset.

iEEG convolved with HRF.

To simulate the fMRI block design paradigm, we concatenated nine events (within category), performed RMS with a sliding window of 50 ms, and convolved the result with a HRF (Boynton et al., 1996). We then subtracted the baseline RMS, also convolved with a HRF.

fMRI category response.

This is the average signal change in category blocks relative to baseline signal as measured 1 TR before block onset.

Single trial category response.

This is the number of trials that exceeded the threshold of the electrode as defined above.

fMRI data

Preprocessing of functional scans included three-dimensional motion correction, filtering out of low frequencies (slow drift), and spatial smoothing with a Gaussian kernel of FWHM = 4 mm (object category experiment) or 8 mm (naturalistic movie experiment). Data were aligned to the Talairach coordinate system and averaged across five subjects.

Statistical analysis

Significance values associated with Pearson correlation coefficients (reproducibility of iEEG signal) were computed by using degrees of freedom adjusted conservatively to the temporal resolution of the convolution function.

The p values related to the significance of the correlations between the iEEG predictor and the fMRI data were corrected for multiple comparisons by using the false discovery rate (Benjamini and Yekutieli, 2001) and were corrected for autoregression in the regressors (consecutive data points of the regressor may not be statistically independent because of the nature of the hemodynamic smoothing process).

For comparing the selectivity index of iEEG and fMRI measurements, we applied a two-tailed two-sample unequal variance t test in which each electrode and fMRI subject served as an independent measurement. For comparison between different iEEG frequency bands (their degree of correlation with the fMRI signal) we applied a two-tailed two-sample unequal variance t test in which each electrode exhibiting reproducible activity at a particular frequency band served as an independent measurement.

Results

The results reported here are based on two types of stimulation paradigms: a continuous segment of a movie and a discrete presentation of images belonging to different object categories. Stimuli from both these paradigms were presented, on the one hand, to patients undergoing presurgical recording with the use of intracranial electrodes and, on the other hand, to five healthy subjects that underwent fMRI scans. Figure 1 illustrates the two experimental paradigms as presented to the patients. In the movie experiment the patients were exposed twice to 9 min of the popular western The Good, the Bad, and the Ugly. In the object category experiment the patient viewed stationary images from three categories: faces, houses, and tools (see Materials and Methods).

We first searched for electrodes that were activated robustly by the movie. To that end we identified activation time courses that exhibited significant reproducibility in their BLP changes during repeated presentations of the movie stimuli (Mukamel et al., 2005) (for details, see Materials and Methods). When comparing the level of reproducibility in different frequency bands, we found that gamma band activity tended to exhibit higher reproducibility as compared with other frequency bands, but this bias was not statistically significant in all cortical regions. Moreover, an examination of the power distribution (supplemental Fig. 4, available at www.jneurosci.org as supplemental material) revealed that power decreased at higher frequencies, indicating that reproducibility in different frequency bands did not correlate with power.

Figure 2 provides an example of the gamma power changes in a movie-driven electrode, which was located on the occipital pole of the right hemisphere (Fig. 2a). The reproducibility of responses in the gamma BLP of the iEEG signal can be seen in Figure 2c (r = 0.31; p < 10−11). Generally, 38 of 275 recorded electrodes (14 ± 4% SD across patients) exhibited reproducibility in their gamma power changes above the minimal criterion (r > 0.2). The mean reproducibility value across the two presentations was r = 0.31 ± 0.11 (mean ± SD) (for the full distribution, see supplemental Fig. 6, available at www.jneurosci.org as supplemental material). In the rest of this work we refer to electrodes with reproducibility levels of r > 0.2 as reproducible or responsive to the movie stimuli. Electrodes that did not exceed this level of reproducibility were excluded from additional analysis.

Figure 2.

Figure 2.

Open in a new tab

Reproducibility of iEEG signal and correlation to fMRI during movie stimuli. a, The anatomical location of the iEEG electrode (marked in red; the rest of the electrodes are green) as seen in a coronal MRI slice of patient's brain (y = −87). b, The fMRI voxels that correlated best to the iEEG signal from the patient (p < 0.05, corrected), as seen from an occipital view. The white circle denotes the estimated anatomical location of the electrode. L, Left; R, right; IPS, intraparietal sulcus. c, Plots of the iEEG gamma BLP recorded from the electrode shown in a during two presentations of the movie clip. d, The iEEG gamma BLP smoothed with a HRF (red) superimposed on the actual fMRI signal sampled from the anatomical location of the electrode (blue).

That the signals were reproducible across runs attests to the fact that they were evoked by the movie stimuli, rather than reflecting spontaneous activity or noise. However, the precise identification of the stimulus parameters that were responsible for this correlation in each electrode requires exhaustive “reverse correlation” (Hasson et al., 2004) analysis, which is beyond the scope of this report. The fact that the correlations of the iEEG responses to the fMRI were localized anatomically (see below) argues against a global attention or arousal effects as the main source of such correlations.

A comparison of the correlations of iEEG activity during movie presentation across patients (when placed in similar cortical fields) revealed 42 correlated electrodes (15% of all electrodes and 30 of the 38 electrodes that were found reproducible between the two movie runs). Correlations across patients were found in retinotopic visual cortex, in ventral and lateral high-order visual cortex, and in auditory cortex (for details, see supplemental Table 2, available at www.jneurosci.org as supplemental material) (also see Materials and Methods).

iEEG to fMRI correlation

After establishing that our stimuli elicited reproducible responses in several iEEG recording sites, we set out to compare the activity of iEEG BLPs to the fMRI signals of healthy subjects viewing the same movie clip. To examine this issue in detail, we extracted the iEEG BLPs, applied convolution with a canonical HRF (Boynton et al., 1996), and then correlated the iEEG BOLD predictor signal with the actual fMRI signal sampled from the estimated anatomical location of the electrode (see Materials and Methods). An example of such a comparison is depicted in Figure 2d, in which the gamma BLP iEEG predictor is superimposed on the actual fMRI signal (r = 0.33; p < 10−8).

We examined this correlation for power changes in several frequency bands of the iEEG. Throughout the human sensory cortex the iEEG signal component that best correlated to the fMRI signal during movie stimulation was the gamma BLP (r = 0.2 ± 0.18; mean ± SD; p < 0.002; t test across bands).

Importantly, the location of the iEEG electrodes (as determined by coregistering the postoperative CT scan with a preoperative MR scan) accurately overlapped the region showing highest correlation to the fMRI signal (Fig. 2a,b) (for details, see Materials and Methods). Considering that these two measures were recorded from different individuals, this is a clear indication of the colocalization of the fMRI and iEEG gamma BLP responses.

Furthermore, such colocalization was not an isolated incident. Figure 3 shows a map summarizing the results obtained from 10 electrodes that represent the cortical regions in which we found the highest temporal correlations between the iEEG BLP and the fMRI signal. As can be seen, correlations were highly localized to specific cortical regions. Note the consistent overlap between the anatomical location of an iEEG electrode (Fig. 3, white symbols) and its corresponding correlated fMRI patch (Fig. 3, colored patches). A presentation of these results on a flat cortical surface is shown in supplemental Figure 1 (available at www.jneurosci.org as supplemental material). Overall, 23 of 38 (60%) of all electrodes that showed significant reproducibility in their gamma band response were colocalized with the corresponding fMRI activation.

Figure 3.

Figure 3.

Open in a new tab

Spatial correlation between iEEG and fMRI responses to movie clips. a, A lateral view of iEEG electrodes and their corresponding fMRI activations (p < 0.05, corrected). White symbols with colored dots denote the anatomical location of electrodes on the patient's cortex as determined by coregistration of the postoperative CT to the preoperative MRI. Each color patch represents fMRI voxels that were correlated to a different electrode of the corresponding color. The different white symbols (circle, square, and triangle) correspond to the different patients. b, Ventral view, with conventions as in a. Gray contours mark approximate borders of cortical visual regions. Note the consistent correspondence between the estimated location of the electrodes and the corresponding fMRI activations at various visual and auditory regions. L, Left; R, right; A, anterior; P, posterior; COS, collateral sulcus; STS, superior temporal sulcus.

Category selectivity: comparing iEEG and fMRI

Our ability to find corresponding cortical regions in which the iEEG and fMRI signals could be evaluated allowed us to compare the object category selectivity of the fMRI response with that of the local iEEG recordings. Thus for each visually driven iEEG electrode and for each corresponding fMRI region in the visual cortex we calculated the CSI (ranges from zero to one) (for details, see Materials and Methods) as measured during the object category experiment. This CSI reflects the degree to which a specific cortical site responds exclusively to a certain category of object images.

To identify the electrodes selective for object category, we used the standard mean VEP method (Fig. 4a) (see Materials and Methods). Six electrodes (of 16) that responded differently to various object categories were found to be highly selective to one category. Next we measured the CSI of each selective electrode by averaging the RMS of all of the trials compared with the baseline (see Materials and Methods). The CSI of the six selective electrodes was 0.77 ± 0.15 (mean ± SD), which is equivalent to a selectivity ratio of 1:8. To assess the fMRI selectivity corresponding to a specific electrode, we first defined the fMRI region that was highly correlated to the response profile of the electrode from the movie experiment. Next we calculated the selectivity index of this cortical region as reflected in the fMRI signals obtained during the fMRI object category experiment (see Materials and Methods) (also see supplemental Fig. 2, available at www.jneurosci.org as supplemental material). The fMRI CSI was found to be 0.33 ± 0.06 (mean ± SD).

It could be argued that the reduced selectivity found in fMRI was attributable to the lack of precise spatial correspondence between the iEEG and the fMRI voxels. To examine this possibility, we searched in each subject for the most object-selective fMRI voxel for each object category in the entire hemisphere. However, even when we applied this stringent criterion, the fMRI selectivity index was found to be CSI = 0.40 ± 0.07 (mean ± SD), well below the iEEG CSI of 0.77 ± 0.15. This difference was highly significant (p < 10–3; t test). Importantly, this difference was maintained when convolving the iEEG signal with a HRF (see Materials and Methods): CSI of convolved iEEG = 0.69 ± 0.09 (p < 10–3; t test). Thus our results show that responses to object categories revealed via iEEG recordings consistently showed higher object category selectivity than any measure of selectivity in the fMRI responses (see Fig. 7).

Figure 7.

Figure 7.

Open in a new tab

Comparison of selectivity between iEEG and fMRI. Shown is a comparison of the selectivity index as evaluated by several methods. Bars (left to right) mark the following: (1) fMRI best voxels (average CSI of signal change of the most selective voxels per category), (2) iEEG RMS (average CSI of RMS change of all trials per category at the interval of 50–250 ms poststimulus onset), (3) iEEG convolved HRF [average CSI of nine iEEG trials convolved with HRF (to simulate fMRI block)], and (4) iEEG peaks count [CSI of the number of trials in which the negative peak (at 50–250 ms after onset) exceeded the threshold of the electrode (∼3 SD) (see Materials and Methods)]. All three iEEG CSIs are significantly higher than the fMRI CSI (t test; p < 10–3). Error bars indicate SEM. Asterisks indicate significant difference.

Single trial selectivity for object category

Importantly, the signal-to-noise ratio of our intracranial EEG recordings was high enough to allow for a quantitative analysis of the response profile to each individual picture that was presented. Figure 4c depicts the signal recorded in one electrode while the subject viewed images of objects from several categories. Using a detection threshold of 3 SD below baseline, we found 83% of face stimuli to produce supra-threshold responses, whereas only 2% of house or tool stimuli produced such responses. This profile corresponds to a selectivity index of CSI = 0.96 (which is equivalent to a ratio of 1:45 or D′ = 3.06). The single-trial response CSI of all six selective electrodes was 0.85 ± 0.08 (mean ± SD).

Figure 5a shows the 50 pictures producing the highest amplitude responses in a face-selective electrode, arranged in descending order. As can be seen, only one of these 50 pictures was an image of a house, and only one was an image of a tool. Similar galleries corresponding to all six object-selective electrodes are shown in supplemental Figure 3 (available at www.jneurosci.org as supplemental material), providing a striking illustration of the object selectivity manifested in human object cortex. Interestingly, in several cases the non-optimal objects had some shape attributes common with the optimal category, such as the two round tools that evoked the strongest response of the face-selective electrode (Fig. 5).

Figure 5.

Figure 5.

Open in a new tab

Gallery of pictures that evoked the strongest VEP. Images are presented in descending order (strongest response is shown in the top left corner). a, b, Two galleries corresponding to activity patterns of two adjacent face-selective electrodes (8 mm distance) recorded in the same patient. In a, only a single house and a single tool produced activations in the top 50 pictures that were examined. Note that the shape of the two tools that evoked the highest response is round, and note the difference in specific face exemplars between the two face-selective electrodes. c, A gallery of a house-selective electrode.

To examine the single trial object-selective responses in more detail, we grouped the VEP responses by object category. This procedure is illustrated for four (of six) object-selective electrodes in Figure 6, in which rows a and b represent face-selective electrodes, row c represents a tool-selective electrode, and row d represents a house-selective electrode. Interestingly, in two patients we found two adjacent electrodes (8 mm distance) selective to faces. Although these electrodes were both face-selective, the pictures that elicited the strongest responses were different in each electrode (supplemental Fig. 3, available at www.jneurosci.org as supplemental material). This result suggests that the underlying neuronal selectivity is quite localized in ventral stream object areas (also see Discussion).

Discussion

Correlation between iEEG and fMRI responses

The first significant aspect of the present results is the tight correlation we observed between high-frequency iEEG and fMRI activity. Careful analysis of the electrodes that were driven most consistently by the movie showed that high-frequency gamma activity (30–70 Hz) manifested the most significant correlation to the fMRI signal (r = 0.34 ± 0.08; mean ± SD) (also see supplemental Table 1, available at www.jneurosci.org as supplemental material). These results are compatible with our previous study of human auditory responses (Mukamel et al., 2005), as well as studies in area V1 of anesthetized monkeys and cats (Logothetis et al., 2001; Niessing et al., 2005) and studies comparing scalp EEG to fMRI (Sehatpour et al., 2006). The current results extend these findings to the occipitotemporal human cortex and to iEEG recordings.

The correlation between iEEG and fMRI was particularly significant in terms of regional colocalization (Figs. 2, 3). Importantly, although obtained in different individuals, the location of 60% of the highly active electrodes overlapped with an fMRI patch that was found to be best correlated to their activity patterns.

Moreover, the fact that these correlations were site-specific demonstrates a high level of local functional specialization in human sensory areas and consistency in functional organization across human subjects. Thus the localized nature of the highly correlated fMRI patches strongly suggests that natural movie stimuli, despite their rich and multidimensional nature, nevertheless activated each individual iEEG electrode and its corresponding fMRI region in a regionally selective manner. The localization of functional profile was also evident in the visual object category experiment in which adjacent electrodes could show strikingly different category selectivity as well as different selectivity profiles within a certain category (supplemental Fig. 5, available at www.jneurosci.org as supplemental material).

Selectivity for object category in the human visual system

The second significant aspect of the results is that iEEG in human object areas shows an extremely sharp tuning for object category, significantly sharper than that measured with standard fMRI. This conclusion was based on comparing the selectivity measured with iEEG and fMRI in different individuals. It may be argued that this result was attributable to a lack of precise spatial correspondence between the iEEG electrode and the fMRI measurements. However, even when we applied the most conservative criteria by searching for the most object-selective fMRI voxels in the entire hemisphere, the iEEG recordings nonetheless exhibited a significantly higher object selectivity (CSI = 0.77 for iEEG category-selective electrodes as compared with CSI = 0.40 at the most selective voxels of fMRI; p < 10–3; t test). It also could be argued that comparing the fMRI block design data with the event-related EEG data is problematic. However, several event-related fMRI studies (Grill-Spector et al., 2004; Schwarzlose et al., 2005) reported similar degrees of selectivity to those found in our fMRI blocked design experiment. Taken as a whole, the iEEG category-selective electrodes showed significant activations to one object category and essentially no detectable responses to any of the other categories, and this was not the case with fMRI.

These results confirm recent findings from behaving monkeys that similarly show extremely high selectivity to faces (CSI = 0.89) in inferotemporal cortex (Tsao et al., 2006). However, this comparison should be interpreted cautiously, because a different number of object categories was used in the different studies.

Our results extend previous reports (Allison et al., 1999) first by demonstrating sharp face selectivity in human ventral stream areas, via single exemplar analysis. Second, we show that these single trial results also hold true for other object categories such as houses and tools. Thus the organization of high-order object areas into complex domains of finely tuned object categories appears to be a general principle of the primate brain rather than a specific feature of face-selective regions.

The organization of object-selective clusters in human cortex

An additional indirect implication of this study concerns the putative size of the category-selective regions in the human visual cortex. Note that our results reveal a consistently reduced selectivity for fMRI (CSI = 0.40) as compared with the high selectivity ratios (CSI = 0.77) of iEEG and those found in monkeys with single-unit studies (CSI = 0.89) (Tsao et al., 2006). What could be the source of such difference?

Our recordings were obtained by using electrodes of 2 mm in diameter placed on the cortical surface. Although the localization power of such recordings is certainly better than scalp EEG, its exact boundaries are still not certain and should be interpreted with caution. Nevertheless, recent studies that compared the selectivity of extracellular LFP with that of multi-unit spiking activity found strong correspondence within an area of ∼3 mm (Kreiman et al., 2006; Liu and Newsome, 2006). Our results strengthen the notion that changes in the LFP are related to local neuronal processes by demonstrating that electrodes separated by 8 mm exhibit markedly different selectivity profiles (supplemental Fig. 5, available at www.jneurosci.org as supplemental material).

A plausible source for the relatively broader tuning of conventional fMRI could be a better spatial resolution of the iEEG measurements. Thus a likely interpretation of the reduced fMRI selectivity is that in conventional fMRI the measured voxel receives additional contributions from neighboring clusters selective to different object categories. This interpretation is compatible with models in which object categories are represented by a complex mosaic of functionally tuned patches (Malach et al., 1995; Puce et al., 1999). In addition, it is in line with our previous study (Avidan et al., 2001), which proposed a “mosaic” organization of object-selective clusters based on the fact that non-optimal categories nevertheless produced adaptation effects in high-order object areas. However, it should be emphasized that the present results are also compatible with other models, for example, an organization in which a highly selective cluster is surrounded by clusters showing much broader object selectivity levels. Furthermore, the source of the broader fMRI tuning may be an as yet unknown consequence of the different temporal characteristics of the fMRI and iEEG signals.

What could be the principle along which these clusters are organized? Extensive work regarding the functional properties of these areas derived from fMRI research (Grill-Spector and Malach, 2004), as well as previous iEEG studies (Puce et al., 1999), suggests that high-level visual representations likely include holistic aspects of the objects rather than low-level features (Hasson et al., 2001; Kourtzi and Kanwisher, 2001; Lerner et al., 2002). Thus it is unlikely that the selectivity found in the iEEG responses is attributable to shared low-level visual features. A more likely possibility is that the common aspect of the neurons in such a cluster is a high-level object category, either a “prototype” similarity (Edelman et al., 1998) or some abstract high-level aspect (Gauthier et al., 2000; Kanwisher, 2006). Furthermore, similar to Tsao et al. (2006), we also found examples in which the non-face images that activated the face-selective patches shared common holistic shape attributes with faces (Fig. 5), indicating that the clustering principle likely follows visual shape rather than more abstract semantic categorizations.

Single-trial iEEG responses

The signal-to-noise level of our recordings allowed us to extend previous research via the analysis of single-trial iEEG responses to visual stimulation in the human cortex. Thus it was possible to examine the detailed profile of the neuronal response to individual pictures. Our results show that the neuronal selectivity was not confined to a small number of face images: 58 of the 70 face images presented to the patients (83%) elicited a significant response. Given that the iEEG electrode likely samples from many thousands of neurons, this broad response may not be surprising and in fact could be derived indirectly from our fMRI data (Levy et al., 2004). However, in contrast to this broad response profile within a single category, our results also show a sharp tuning across object categories. Here we found that, in sixobject-selective electrodes, 90% of the images that elicited the highest peaks of activity were of the optimal category, whereas <10% were of the non-optimal categories (Fig. 5) (also see supplemental Fig. 3, available at www.jneurosci.org as supplemental material). Given the large neuronal population sampled by the iEEG electrode, such tight selectivity is indeed remarkable and provides an indication that object selectivity is a powerful organizing principle of high-order visual areas.

Because we have examined only three object categories, our results certainly cannot provide a detailed picture of the selectivity profile of the entire neuronal population. However, they do demonstrate conclusively the striking tendency of neurons in high-order object areas to cluster according to some aspect of a category-related dimension.

Whereas electrophysiological studies of object selectivity have focused primarily on face-selective responses (Farah et al., 1998; Rolls, 2000; Sigala et al., 2002), fMRI studies in humans also have drawn attention to house-selective responses (Epstein and Kanwisher, 1998) as well as other categories (Hasson et al., 2003). To the best of our knowledge our results provide the first electrophysiological evidence for the existence of neuronal clusters selective for house and tool images in the human visual system (Fig. 6, bottom panels). These findings extend single-unit studies that found additional category-specific visual responses in the human medial temporal lobe (Fried et al., 1997; Kreiman et al., 2000; Kraskov et al., 2007) as well as iEEG results showing selectivity to letter and digit forms (Puce et al., 1999; Gaillard et al., 2006).

Footnotes

This work was funded by the Israel Science Foundation Center of Excellence Program and the Dominique and Benozyo Center (to R.M.), Minkowski Minerva Center for Geometry (to Y.Y.), and United States–Israel Binational Science Foundation (to R.M. and I.F.). We thank the patients for their cooperation in participating in this study. We also thank David Yossef, Sari Nagar, Rivi Cohen, and the technicians of the EEG lab at Sourasky Medical Center, Tel Aviv, Israel, for technical assistance; Michal Harel for support with fMRI data analysis; and Amos Arieli, Sharon Gilaie-Dotan, Hagar Gelbard-Sagiv, and Yulia Golland for helpful discussions and feedback.

References

  1. Allison T, Puce A, Spencer DD, McCarthy G. Electrophysiological studies of human face perception. I. Potentials generated in occipitotemporal cortex by face and non-face stimuli. Cereb Cortex. 1999;9:415–430. doi: 10.1093/cercor/9.5.415. [DOI] [PubMed] [Google Scholar]
  2. Avidan G, Hendler T, Zohary E, Malach R. Analysis of the neuronal selectivity underlying low fMRI signals. Soc Neurosci Abstr. 2001;27:11–8. doi: 10.1016/s0960-9822(02)00872-2. [DOI] [PubMed] [Google Scholar]
  3. Benjamini Y, Yekutieli D. The control of the false discovery rate in multiple testing under dependency. Ann Stat. 2001;29:1165–1188. [Google Scholar]
  4. Boynton GM, Engel SA, Glover GH, Heeger DJ. Linear systems analysis of functional magnetic resonance imaging in human V1. J Neurosci. 1996;16:4207–4221. doi: 10.1523/JNEUROSCI.16-13-04207.1996. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Dewar S, Passaro E, Fried I, Engel J., Jr Intracranial electrode monitoring for seizure localization: indications, methods and the prevention of complications. J Neurosci Nurs. 1996;28:280–284. 289–292. doi: 10.1097/01376517-199610000-00002. [DOI] [PubMed] [Google Scholar]
  6. Edelman S, Grill-Spector K, Kushnir T, Malach R. Toward direct visualization of the internal shape representation space by fMRI. Psychobiology. 1998;26:309–321. [Google Scholar]
  7. Epstein R, Kanwisher N. A cortical representation of the local visual environment. Nature. 1998;392:598–601. doi: 10.1038/33402. [DOI] [PubMed] [Google Scholar]
  8. Farah MJ, Wilson KD, Drain M, Tanaka JN. What is “special” about face perception? Psychol Rev. 1998;105:482–498. doi: 10.1037/0033-295x.105.3.482. [DOI] [PubMed] [Google Scholar]
  9. Fried I, MacDonald KA, Wilson CL. Single neuron activity in human hippocampus and amygdala during recognition of faces and objects. Neuron. 1997;18:753–765. doi: 10.1016/s0896-6273(00)80315-3. [DOI] [PubMed] [Google Scholar]
  10. Gaillard R, Naccache L, Pinel P, Clemenceau S, Volle E, Hasboun D, Dupont S, Baulac M, Dehaene S, Adam C, Cohen L. Direct intracranial, fMRI, and lesion evidence for the causal role of left inferotemporal cortex in reading. Neuron. 2006;50:191–204. doi: 10.1016/j.neuron.2006.03.031. [DOI] [PubMed] [Google Scholar]
  11. Gauthier I, Tarr MJ, Moylan J, Skudlarski P, Gore JC, Anderson AW. The fusiform “face area” is part of a network that processes faces at the individual level. J Cogn Neurosci. 2000;12:495–504. doi: 10.1162/089892900562165. [DOI] [PubMed] [Google Scholar]
  12. Grill-Spector K. The neural basis of object perception. Curr Opin Neurobiol. 2003;13:159–166. doi: 10.1016/s0959-4388(03)00040-0. [DOI] [PubMed] [Google Scholar]
  13. Grill-Spector K, Malach R. fMR-adaptation: a tool for studying the functional properties of human cortical neurons. Acta Psychol (Amst) 2001;107:293–321. doi: 10.1016/s0001-6918(01)00019-1. [DOI] [PubMed] [Google Scholar]
  14. Grill-Spector K, Malach R. The human visual cortex. Annu Rev Neurosci. 2004;27:649–677. doi: 10.1146/annurev.neuro.27.070203.144220. [DOI] [PubMed] [Google Scholar]
  15. Grill-Spector K, Knouf N, Kanwisher N. The fusiform face area subserves face perception, not generic within-category identification. Nat Neurosci. 2004;7:555–562. doi: 10.1038/nn1224. [DOI] [PubMed] [Google Scholar]
  16. Halgren E, Baudena P, Heit G, Clarke JM, Marinkovic K, Clarke M. Spatio-temporal stages in face and word processing. I. Depth-recorded potentials in the human occipital, temporal and parietal lobes [corrected] J Physiol (Paris) 1994;88:1–50. doi: 10.1016/0928-4257(94)90092-2. [DOI] [PubMed] [Google Scholar]
  17. Halgren E, Raij T, Marinkovic K, Jousmaki V, Hari R. Cognitive response profile of the human fusiform face area as determined by MEG. Cereb Cortex. 2000;10:69–81. doi: 10.1093/cercor/10.1.69. [DOI] [PubMed] [Google Scholar]
  18. Hasson U, Hendler T, Ben Bashat D, Malach R. Vase or face? A neural correlate of shape-selective grouping processes in the human brain. J Cogn Neurosci. 2001;13:744–753. doi: 10.1162/08989290152541412. [DOI] [PubMed] [Google Scholar]
  19. Hasson U, Harel M, Levy I, Malach R. Large-scale mirror-symmetry organization of human occipito-temporal object areas. Neuron. 2003;37:1027–1041. doi: 10.1016/s0896-6273(03)00144-2. [DOI] [PubMed] [Google Scholar]
  20. Hasson U, Nir Y, Levy I, Fuhrmann G, Malach R. Intersubject synchronization of cortical activity during natural vision. Science. 2004;303:1634–1640. doi: 10.1126/science.1089506. [DOI] [PubMed] [Google Scholar]
  21. Haxby JV, Gobbini MI, Furey ML, Ishai A, Schouten JL, Pietrini P. Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science. 2001;293:2425–2430. doi: 10.1126/science.1063736. [DOI] [PubMed] [Google Scholar]
  22. Kanwisher N. Neuroscience. What's in a face? Science. 2006;311:617–618. doi: 10.1126/science.1123983. [DOI] [PubMed] [Google Scholar]
  23. Kanwisher N, McDermott J, Chun MM. The fusiform face area: a module in human extrastriate cortex specialized for face perception. J Neurosci. 1997;17:4302–4311. doi: 10.1523/JNEUROSCI.17-11-04302.1997. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Klopp J, Marinkovic K, Chauvel P, Nenov V, Halgren E. Early widespread cortical distribution of coherent fusiform face selective activity. Hum Brain Mapp. 2000;11:286–293. doi: 10.1002/1097-0193(200012)11:4&#x0003c;286::AID-HBM80&#x0003e;3.0.CO;2-R. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Kourtzi Z, Kanwisher N. Representation of perceived object shape by the human lateral occipital complex. Science. 2001;293:1506–1509. doi: 10.1126/science.1061133. [DOI] [PubMed] [Google Scholar]
  26. Kraskov A, Quiroga RQ, Reddy L, Fried I, Koch C. Local field potentials and spikes in the human medial temporal lobe are selective to image category. J Cogn Neurosci. 2007;19:479–492. doi: 10.1162/jocn.2007.19.3.479. [DOI] [PubMed] [Google Scholar]
  27. Kreiman G, Koch C, Fried I. Category-specific visual responses of single neurons in the human medial temporal lobe. Nat Neurosci. 2000;3:946–953. doi: 10.1038/78868. [DOI] [PubMed] [Google Scholar]
  28. Kreiman G, Hung CP, Kraskov A, Quiroga RQ, Poggio T, DiCarlo JJ. Object selectivity of local field potentials and spikes in the macaque inferior temporal cortex. Neuron. 2006;49:433–445. doi: 10.1016/j.neuron.2005.12.019. [DOI] [PubMed] [Google Scholar]
  29. Lerner Y, Hendler T, Malach R. Object-completion effects in the human lateral occipital complex. Cereb Cortex. 2002;12:163–177. doi: 10.1093/cercor/12.2.163. [DOI] [PubMed] [Google Scholar]
  30. Levy I, Hasson U, Malach R. One picture is worth at least a million neurons. Curr Biol. 2004;14:996–1001. doi: 10.1016/j.cub.2004.05.045. [DOI] [PubMed] [Google Scholar]
  31. Liu J, Newsome WT. Local field potential in cortical area MT: stimulus tuning and behavioral correlations. J Neurosci. 2006;26:7779–7790. doi: 10.1523/JNEUROSCI.5052-05.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Logothetis NK, Pauls J, Augath M, Trinath T, Oeltermann A. Neurophysiological investigation of the basis of the fMRI signal. Nature. 2001;412:150–157. doi: 10.1038/35084005. [DOI] [PubMed] [Google Scholar]
  33. Malach R, Reppas JB, Benson RR, Kwong KK, Jiang H, Kennedy WA, Ledden PJ, Brady TJ, Rosen BR, Tootell RB. Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex. Proc Natl Acad Sci USA. 1995;92:8135–8139. doi: 10.1073/pnas.92.18.8135. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Malach R, Levy I, Hasson U. The topography of high-order human object areas. Trends Cogn Sci. 2002;6:176–184. doi: 10.1016/s1364-6613(02)01870-3. [DOI] [PubMed] [Google Scholar]
  35. Mukamel R, Gelbard H, Arieli A, Hasson U, Fried I, Malach R. Coupling between neuronal firing, field potentials, and fMRI in human auditory cortex. Science. 2005;309:951–954. doi: 10.1126/science.1110913. [DOI] [PubMed] [Google Scholar]
  36. Niessing J, Ebisch B, Schmidt KE, Niessing M, Singer W, Galuske RA. Hemodynamic signals correlate tightly with synchronized gamma oscillations. Science. 2005;309:948–951. doi: 10.1126/science.1110948. [DOI] [PubMed] [Google Scholar]
  37. Puce A. Comparative assessment of sensorimotor function using functional magnetic resonance imaging and electrophysiological methods. J Clin Neurophysiol. 1995;12:450–459. doi: 10.1097/00004691-199509010-00004. [DOI] [PubMed] [Google Scholar]
  38. Puce A, Allison T, McCarthy G. Electrophysiological studies of human face perception. III. Effects of top-down processing on face-specific potentials. Cereb Cortex. 1999;9:445–458. doi: 10.1093/cercor/9.5.445. [DOI] [PubMed] [Google Scholar]
  39. Rolls ET. Functions of the primate temporal lobe cortical visual areas in invariant visual object and face recognition. Neuron. 2000;27:205–218. doi: 10.1016/s0896-6273(00)00030-1. [DOI] [PubMed] [Google Scholar]
  40. Schwarzlose RF, Baker CI, Kanwisher N. Separate face and body selectivity on the fusiform gyrus. J Neurosci. 2005;25:11055–11059. doi: 10.1523/JNEUROSCI.2621-05.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Sehatpour P, Molholm S, Javitt DC, Foxe JJ. Spatiotemporal dynamics of human object recognition processing: an integrated high-density electrical mapping and functional imaging study of “closure” processes. NeuroImage. 2006;29:605–618. doi: 10.1016/j.neuroimage.2005.07.049. [DOI] [PubMed] [Google Scholar]
  42. Sigala N, Gabbiani F, Logothetis NK. Visual categorization and object representation in monkeys and humans. J Cogn Neurosci. 2002;14:187–198. doi: 10.1162/089892902317236830. [DOI] [PubMed] [Google Scholar]
  43. Tsao DY, Freiwald WA, Tootell RB, Livingstone MS. A cortical region consisting entirely of face-selective cells. Science. 2006;311:670–674. doi: 10.1126/science.1119983. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from The Journal of Neuroscience are provided here courtesy of Society for Neuroscience

RESOURCES