Abstract
The constituent elements and dynamics of the networks responsible for word production are a central issue to understanding human language. Of particular interest is their dependency on lexical category, particularly the possible segregation of nouns and verbs into separate processing streams. We applied a novel mixed-effects, multilevel analysis to electrocorticographic data collected from 19 patients (1942 electrodes) to examine the activity of broadly disseminated cortical networks during the retrieval of distinct lexical categories. This approach was designed to overcome the issues of sparse sampling and individual variability inherent to invasive electrophysiology. Both noun and verb generation evoked overlapping, yet distinct nonhierarchical processes favoring ventral and dorsal visual streams, respectively. Notable differences in activity patterns were noted in Broca's area and superior lateral temporo-occipital regions (verb > noun) and in parahippocampal and fusiform cortices (noun > verb). Comparisons with functional magnetic resonance imaging (fMRI) results yielded a strong correlation of blood oxygen level-dependent signal and gamma power and an independent estimate of group size needed for fMRI studies of cognition. Our findings imply parallel, lexical category-specific processes and reconcile discrepancies between lesional and functional imaging studies.
Keywords: electrocorticography, fMRI, functional mapping, language, naming
Introduction
Psycholinguistic, lesional (Caramazza and Hillis 1991; Damasio and Tranel 1993; Tranel et al. 2001), and functional imaging studies (Soros et al. 2003; Sahin et al. 2006; Liljestrom et al. 2008, 2009) are generally unable to resolve the fine time scale of processes underlying language production. Core issues such as serial versus parallel order in word production, and distinctions (or lack thereof) between cortical regions and processes intrinsic to the production of particular lexical categories have eluded definitive answers due to the lack of experimental strategies to tackle them. Lesional analyses of patients with selective deficits in noun or verb naming have led to suppositions that these grammatical classes reside in distinct brain regions, yet functional imaging studies have not reached a broad consensus on the differences between them.
Intracranial electrocorticographic (ECoG) data collected using subdural macroelectrodes possess spatio-temporal characteristics uniquely suited to evaluating hi-speed, transient interactions between the disseminated neural modules involved in language generation (Engel et al. 2005; Liljestrom et al. 2008; Chang et al. 2011; Conner, Ellmore, Pieters, et al. 2011). Yet, despite their remarkable spatial and temporal resolution, analyses of phonologic and semantic processes using ECoG are restricted to small cortical regions (Sahin et al. 2009; Edwards et al. 2010; Chang et al. 2011; Mesgarani and Chang 2012) and unable to provide a complete description of disseminated network activity, limiting their broader utility to cognitive neuroscience. The reasons for this include the problem of spatial variability and limited electrode coverage per individual (a “sparse sampling” issue), varied epileptogenic networks (and implied aberrant activity) in each individual, and an inability to analyze these data as a group. Simple approaches used in the past—averaging across individuals in a common space, followed by corrections using a false detection rate (Vidal et al. 2010)—unfairly assume equivalent intrasubject and negligible intersubject variability and disregard the sparse sampling problem. Given the sparse sampling and interindividual variability in cognitive processing modules (Ojemann et al. 1989; Xiong et al. 2000), large numbers of subjects are needed for these studies, as is a statistically robust analytic technique to interpret grouped ECoG data.
Our novel approach optimizes the colocalization of ECoG data, specifically accounting and adjusting for intra- and intersubject variabilities, to provide a precise estimate of the effect of interest. This strategy, termed mixed-effects, multilevel analysis (MEMA), was recently developed for the analysis of functional magnetic resonance imaging (fMRI) data (Chen et al. 2011) and is particularly well suited to grouped statistical analyses of datasets with varied coverage. A large cohort of individuals were studied during the visually cued retrieval of common nouns and verbs (Damasio and Tranel 1993; Price et al. 2005; Liljestrom et al. 2008, 2009), tasks that were chosen as patient performance on them mirror the accuracy and timing seen in healthy volunteers. Additionally, to abrogate concerns that ECoG data collected in patients with epilepsy may not completely reflect normal processes (unlikely given varied seizure loci and normal response parameters), we also compared the fMRI activation in the patient group with the similar sized group of normal human volunteers.
Materials and Methods
Data were collected from 19 patients (12 females, mean age 33 ± 12 years) undergoing left hemispheric subdural electrode (SDE) implants for localizing seizure onset sites and from 14 healthy volunteers. Informed consent was obtained following study approval by our institution's committee for the protection of human subjects. Patients were selected to be able to perform the task within normal response parameters and possessed an average or higher Intelligence quotient (IQ) (mean IQ—98 ± 11). A total of 1942 individual SDEs were implanted. Of these, 313 electrodes were excluded due to proximity to sites of seizure onset (162), interictal spikes (89), or 60 Hz noise (62); the remaining 1629 SDEs were analyzed. There was excellent coverage over all canonical language cortex, including lateral frontal, lateral temporal, and ventral occipito-temporal cortices (Figs 1 and 2). The extent of coverage allowed for meaningful grouped analysis of the temporal order in language processing. The methods involved in data acquisition and preprocessing are similar to those detailed previously (Conner, Ellmore, Disano, et al. 2011; Conner, Ellmore, Pieters, et al. 2011; Swann et al. 2012).
Language Tasks
All patients and healthy volunteers performed 3 language tasks (Kaplan et al. 1983)— naming of visually presented common nouns (Boston Naming Test), visual depictions of actions, and scrambled images (generated from the noun and verb stimuli). Stimuli consisted of simple line drawings, akin to those found in Snodgrass and Vanderwart (1980). During noun generation, subjects responded with single word descriptions of the object presented, and during verb naming, they responded with a single action word such as “cooking” or “walking.” In response to the scrambled images, subjects articulated “scrambled,” which provided us with a high-level control condition. During ECoG recordings, patients verbally articulated their response, while during fMRI acquisition both patients and healthy volunteers were asked to internally (covertly) vocalize and respond with a button press recorded by the stimulus presentation software (Fig. 1).
MR Data Acquisition
Imaging data acquisition was performed with a 3-T whole-body MR scanner (Philips Medical Systems, Bothell, WA, USA) equipped with a 16-channel SENSE head coil. MR data were acquired prior to surgery. A magnetization-prepared 180° radio-frequency pulses and rapid gradient-echo sequence with 1-mm thick sagittal slices and an in-plane resolution of 0.938 × 0.938 mm and functional MRI volumes (33 axial slices, 3-mm slice thickness, 2.75 in-plane resolution, 30-ms time echo, 2015-ms time repetition, 90° flip angle) were collected. Language stimuli were presented in a block design (Salmelin et al. 1994). For each task (noun and verb generation), 2 runs of fMRI data were collected. Each run comprised of 8 blocks (136 volumes per run), each block comprised of 10 task stimuli and 7 scrambled stimuli. 160 individual noun and verb, and 224 scrambled stimuli, were presented. Each stimulus was presented at the onset of TR using the Presentation software (version 11, Neurobehavioral systems) and a screen positioned above the eyes (IFIS, Invivo, Gaineville, FL, USA), for 1500 ms with a 515-ms interstimulus interval (Fig. 1).
Image Analysis
MRI realignment, spatial normalization transformation, and group analysis were performed in AFNI (Cox 1996). Surface reconstructions of the pial surface were generated using FreeSurfer v4.5 (Dale et al. 1999). The aligned 4-dimensional dataset was spatially smoothed with a 3-mm Gaussian filter, then processed using multiple regression at each voxel to contrast the 2 tasks (noun and verb generation) with the control condition (scrambled naming). Both the effect estimates (regression coefficients) and their corresponding t-values were used for group MEMA.
Laterality Estimates
To verify left hemispheric language dominance, language lateralization indices were calculated for each individual using the language fMRI data (Conner, Ellmore, Pieters, et al. 2011). Activations in Brodmann areas 44 and 45 in each hemisphere were extracted using masks constructed from a standard atlas (Amunts et al. 1999). The number of significant voxels (P < 0.001) during the 2 tasks, verb and noun naming, was computed for the mask in each hemisphere. The laterality index used was equal to (#L − #R)/(#L + #R), where #L and #R are the number of significant voxels in the left and right hemispheres, respectively. In the 17 patients with fMRI data, all were left hemisphere lateralized for language function; of the 14 healthy volunteers, all were left hemisphere lateralized. Additionally, 12 patients underwent intracarotid injection of sodium amytal (the Wada procedure; Wada and Rasmussen 2007) and were found to be left hemispheric dominant for language. Finally, all but one underwent language localization using cortical stimulation mapping and were found to have eloquent language in the left hemisphere (Tandon 2012).
Electrocorticography
ECoG recordings were performed using arrays of subdural platinum–iridium electrodes (PMT Corporation, Chanhassen, MN, USA) with a top hat design (4.5- and 3-mm diameter contact with cortex) embedded in silastic sheets (10 mm center-to-center spacing), using standard neurosurgical techniques (Tandon 2012; Pieters et al. 2013). SDEs were localized using postoperative computed tomography sans and in-house software, onto a cortical surface model (Pieters et al. 2013). Stimulus presentation was carried out using identical stimuli and Presentation software as used for fMRI. In all patients, >50 trials of noun, verb, and scramble naming were performed. Each image was displayed on a 15″ LCD screen positioned at an eye level for 1500 ms with an interstimulus interval of >3000 ms. A transistor–transistor logic pulse triggered by the Presentation software at stimulus onset was recorded as a separate input during the ECoG recording to time lock all trials. Audio recording of each ECoG session was used to accurately measure the onset of articulation and to compute reaction time (RT). Only trials in which the patient responded correctly in <2 s were included in further analysis. ECoG data were also visually inspected for interictal epileptiform discharges and electrical noise. For 17 patients, ECoG data were collected at 1000 Hz using the NeuroFax software (Nihon Kohden, Tokyo, Japan). The other 2 patients underwent ECoG data collection at 2000 Hz using the NeuroPort recording system (Blackrock Microsystems, Salt Lake City, UT, USA). To avoid including any brain regions with potentially abnormal physiology, all electrodes that showed interictal activity (spikes) or that were involved with seizure onset were excluded from all further analysis. All electrodes with >10 dB of noise in the 60-Hz band were also excluded.
For the individual measures of activation during task performance, spectral analysis using the Hilbert transform and analytic amplitude were used to estimate power changes in different frequency bands. The raw ECoG data were bandpass filtered (IIR Elliptical Filter, 30 dB sidelobe attenuation) into 7 bands: Delta (0–4 Hz), theta (4–8 Hz), alpha (8–12 Hz), beta (12–30 Hz), low gamma (30–60 Hz), mid gamma (60–120 Hz), and high gamma (120–240 Hz). A Hilbert transform was applied, and the analytic amplitude was smoothed (Savitzky-Golay FIR, 2nd order, frame length of 255 samples) to derive the time course of power in each band. The percent change and t-value at each time point were calculated by comparing power with the prestimulus baseline (−700 to −200 ms). The epoch from 50 ms after stimulus onset to mean RT minus 1 sd was selected for further analysis in order to minimize the effects of articulation on the ECoG. The composite t-value and effect size for this time interval of ECoG data were then computed (metafor package ver 1.4 in R; Viechtbauer 2010).
Volumetric Representation of ECoG Data
The time-integrated ECoG activity was transformed into volumetric data for each subject individually (Conner, Ellmore, Pieters, et al. 2011) to reflect the cortical regions that the recordings likely originated from. This transformation also enables grouped analysis and minimizes errors in coregistration. The Euclidean distance from each electrode to each voxel in image space was computed, and then this distance was scaled using 3-dimensional (3D) Gaussian filters (SD = 6 mm). This transformation was chosen because it maximizes agreement with fMRI results (Conner, Ellmore, Pieters, et al. 2011) and concurs with our limited understanding of ECoG signal sources (Acar et al. 2011). The net activity at each voxel was defined as the weighted sum of all SDEs that contributed to it. Individual volumes of activity were then constructed for noun, verb, and scrambled naming. This transformation produced a 3D blur of the original, point estimate data provided by the SDEs. Additionally, an SDE coverage map was constructed for each individual subject—all voxels within 10 mm (equal to the spacing between individual SDE electrodes) of an SDE were given a value 1, and all other voxels were set to 0. This binary map represents the volume of approximate SDE coverage for each subject and was summed across all 19 individuals to obtain a total group coverage map (Fig. 2), thresholded values of which were then used to constrain the group fMRI results. This is essential as the fMRI data are “whole brain,” while the ECoG data, even for the large group used here, provide data for only parts of the cortex.
Spatial Normalization
For the grouped analysis, the datasets (both ECoG and fMRI) for each subject were aligned to the MNI-N27 brain. This alignment was performed by first computing the transform of the individual's anatomical MRI to the N27 anatomical MRI. The 12 parameter affine transformation of the individuals anatomical MRI was then applied to each individual's fMRI and volumetric ECoG data. In this manner, all of the ECoG datasets (n = 19), the patient fMRI data (n = 17), and the healthy volunteer fMRI data (n = 14) were all transformed into the MNI-N27 imaging space.
Population-Level Analysis of ECoG and fMRI Data
Two methods were adopted in our group analysis of fMRI and ECoG data. The traditional approach for performing group analysis [e.g. t-test and 2-way, mixed-effects analysis of variance (ANOVA)] assumes that the effect estimates across subjects have the same reliability (or variance). In contrast, the MEMA approach takes both effect estimates (percent change for ECoG and regression coefficient for fMRI) and their variances for each individual to estimate the cross-subject variability using restricted maximum-likelihood function based on each subject's precision information of effect estimate (Chen et al. 2011).
Statistical Corrections
To correct for multiple comparisons, clustering analysis was applied to both fMRI and ECoG group analyses (to each ANOVA and MEMA test). An initial threshold of P < 0.05 (uncorrected) was applied to select voxels of interest, and then grouped to get the number of contiguous voxels in each cluster. To determine the minimum size of a significant cluster, samples of white noise with the same dimensions and smoothness of the datasets were generated. Only clusters greater than the minimum size (359 voxels) at the corrected P < 0.05 were visualized (Figs 3, 4, and 7).
Conjunction Analysis
To assess the difference between noun and verb naming, a conjunction analysis was applied to the verb versus scramble and noun versus scramble conditions (Fig. 3). These maps were individually thresholded at P < 0.05 (corrected), binarized and consolidated to identify the regions of coactivation, and areas only involved in 1 task (Fig. 4). This comparison between verb and noun highlights regions that may not otherwise be considered significant (Friston et al. 1999).
Time Series Analysis
To estimate the average group time course for different regions, loci with the greatest divergence over the entire response epoch in activity between verb, noun, and scrambled naming were identified from the 3D, volumetric group analysis (Figs 3 and 4). A total of 12 regions were used for this analysis. The coordinates at the center of mass for each region were then used to select SDEs from each individual that lay within 8 mm of those co-ordinates (Table 1). For each region, percent change in the mid-gamma band was averaged across the electrodes in that region from −500 to 2000 ms for each of the 3 tasks (Figs 5 and 6).
Table 1.
ECoG | ||
Inferior temporal | −43, −39, −26 | BA36 |
Fusiform | −29, −43, −19 | BA20 |
Parahippocampal | −6, −53, 0 | BA18 |
Precuneus | −1, −57, 16 | BA23 |
Lateral occipital (SLOC) | −43, −75, 26 | BA39 |
Supramarginal | −60, −47, 23 | BA40 |
Superior temporal | −64, −35, 14 | BA22 |
Superior frontal | −34, 46, 32 | BA9 |
Pars orbitalis | −54, 34, 3 | BA45/47 |
Pars triangularis | −56, 19, 19 | BA45 |
Premotor | −41, 3, 52 | BA6 |
Primary motor | −61, −9, 28 | BA4 |
fMRI | ||
SFG (deactivation) | −27, 34, 40 | BA8/9 |
Lateral temporo-occipital | −53, −65, 1 | BA37/19 |
IFG (pars orbitalis) | −47, 27, 4 | BA45/47 |
IFG (pars triangularis) | −48, 25, 16 | BA45 |
Premotor | −44, 2, 44 | BA6 |
Superior occipital gyrus | −38, −75, 28 | BA19 |
Fusiform | −38, −45, −15 | BA37 |
Comparison Between Group fMRI and Group ECoG
The results of grouped fMRI and ECoG datasets were compared with a voxel-based approach derived using the beta coefficients from the MEMA. Both sets of beta coefficients (7 values in the ECoG dataset, one corresponding to each spectral band; and a single value for the fMRI), along with the ECoG coverage map, were utilized. Correlations were made only for voxels with >5 patients contributing to the ECoG (see coverage map—Fig. 2) and were rerun for voxels with increasing coverage (greater numbers of patients contributing to the data) to a maximum of 15 to model goodness of fit. To evaluate the effect of fMRI group size and correlation with ECoG, a bootstrapping analysis was run using the individual fMRI datasets. The group of 17 subjects with both fMRI and ECoG data was resampled with replacement (500 resamples) for different fMRI group sizes (3, 5, 7, 9, 11, 13, and 15) and then each new group was reanalyzed with MEMA. The resulting beta coefficient maps for each resample were correlated with the ECoG group results (using all 19 subjects). Only voxels that were sampled in ECoG for at least 10 patients were used to ensure for an adequate number of subjects. The distribution of resamples was used to model the correlation of fMRI to ECoG at the different group sizes. Differences between group size distributions were performed using sign tests (Bonferroni corrected P < 0.001).
Results
We applied MEMA to represent and view grouped ECoG signals and to evaluate distinctions between visually cued noun and verb generation. This analysis was carried out on ECoG data (n = 19), fMRI data for the same patients, and a normative group (n = 14). Robust changes in the ECoG signal in the mid gamma range (60–120 Hz) occurred early (100–400 ms) in visual and ventral occipito-temporal cortices, followed by concurrent activation of Broca's area, lateral premotor cortex, and primary face/mouth cortex (M1; 300–1200 ms). Verb generation led to overall greater neural activation, and both lexical categories involved vastly overlapping networks with distinct regional differences in magnitude.
Behavioral Results
Behavioral responses were collected for all patients (n = 19) during ECoG recordings and most patients during fMRI acquisition. During ECoG, the mean RTs were 1377 ms (SD = 274 ms) for noun naming, 1479 ms (SD = 262 ms) for verb naming, and 1210 ms (SD = 285 ms) for scrambled images. RTs for verb and noun naming were significantly longer than for the scrambled controls (paired t-test, P < 0.01). As expected, verb naming had a significantly longer RT than noun naming (P = 0.03; Szekely et al. 2005). Mean accuracy for all tasks was >90%, although only correct naming trials were included in the analysis. During fMRI, the mean RTs were 952 ms (SD = 115 ms) for noun naming, 1082 ms (SD = 225 ms) for verb naming, and 736 ms (SD = 178 ms) for scrambled images. As in the ECoG, verb and noun naming were both significantly longer than the scrambled naming (P < 0.01), and the latency of verbs was also greater than nouns (P = 0.03). To verify that this difference was not due to word frequency, the frequencies of verb and noun stimuli (using the SUBLEX word frequency database) were compared and found not to be significantly different (Wilcoxon rank sum, P = 0.2; Brysbaert and New 2009). Given that response time during MRI acquisition was measured using a button press, the difference between the 2 recording modalities likely reflects the delay due to voice onset time in ECoG and the button press instead of overt articulation in the fMRI condition.
Grouped ECoG Analysis
MEMA of visual naming of both nouns and verbs compared with scrambled picture naming (epoch from stimulus onset to just before articulation was considered a single block) revealed strong, high-frequency power increases for both categories over the inferior frontal gyrus (IFG) and basal temporal, precuneate, premotor, and M1 cortices (Figs 3 and 4). Categorical distinctions (corrected P < 0.05) were obvious in the posterior parahippocampal gyrus (PHG), which was more active during noun generation, while pars orbitalis (PO) of the IFG, inferior parietal lobule, and superior lateral-occipital cortex (SLOC) were significantly more active during verb generation (Fig. 4 and Table 1). Cortical regions were always more active (in the gamma band—30–240 Hz) during naming of real nouns or verbs relative to the scrambled images except for the anterior superior frontal sulcus.
Grouped ECoG Time Series Analysis
The time series of all electrodes across all individuals, located in cortical regions with significant differences in the MEMA results, revealed prominent early activity (100–200 ms) in the ventral occipito-temporal region—the posterior inferior temporal gyrus, the fusiform gyrus, and the posterior PHG (Figs 5 and 6). Activity in these areas continued to be significantly elevated from baseline well past the onset of articulation. Early activity was also was noted in Broca's area, lateral premotor cortex, and M1 mouth beginning around 300 ms, peaking at 700 ms, and lasting till past the onset of articulation. These timelines imply nonhierarchical interactions between these regions, contradicting previously proposed models of serial order in language processing (Indefrey and Levelt 2004). We also note that activity in pars triangularis starts to decline well prior to the onset of articulation and is almost back to baseline during response articulation. This suggests that Broca's area is not directly involved in articulation per se, but is more integral to response selection (Badre and Wagner 2007). To confirm that the patterns of activity in the group analysis reflected individual data well, single-subject time series analyses were also performed (Fig. 5). These single electrodes were chosen to overlap with the regions used in the grouped analysis and corroborate those findings.
When ventral occipito-temporal electrodes were closely analyzed, a medio-lateral gradient favoring noun-to-verb naming was noted, with nouns production resulting in the greatest activation overall, but especially so over the posterior PHG where a late response (500 ms) was greatest for nouns relative to verb and scramble conditions. This response followed an earlier peak corresponding to the N100 response over the primary visual cortex. An SLOC also showed strong, early (300 ms) activation, greater and more sustained during verb naming than during noun and scramble conditions, in keeping with the MEMA results. Activity in motor cortex and superior temporal gyrus (STG; primary auditory cortex) was vastly similar across conditions, implying minimal differences in articulatory difficulty or length between these conditions.
An intriguing response was noted in PO, where all of the naming tasks produced an initial net decrease in gamma power followed by a significant increase during verb generation. The time series analysis also clearly demonstrated that the superior frontal sulcus deactivations noted for both verb versus scrambled and noun versus scrambled contrasts are not due to a decrease in power during the task condition, but rather due to an absolute power increase during the scrambled condition starting at 400 ms and peaking at 600 ms.
Grouped fMRI Analysis
fMRI analysis of the nouns and verb production elicited broadly similar patterns of activation as revealed by the MEMA of the ECoG data (Fig. 3). In addition, verb generation led to greater amplitude and spatial distribution of activation overall. For both lexical categories, clear increases in activity were noted in the ventral occipito-temporal cortex, specifically centered in the anterior fusiform gyrus—as revealed in many other prior studies (Luders et al. 1991; Kanwisher 2010), the lateral temporo-occipital junction (Goodale and Milner 1992), Broca's area, and M1 mouth. Significantly, no increase in activity in the lateral temporal neocortex was noted during either of these naming tasks, in agreement with prior studies (Wise et al. 1999). A strong focus of deactivation was noted at the temporo-parieto-occipital junction, corresponding to greater activity in the control than the task condition. Finally, a small but consistent focus of deactivation in the SFG was noted for both tasks relative to scrambled naming, similar to that seen during ECoG.
MEMA comparing verb with noun generation of the fMRI data performed on both the patient and the healthy controls revealed vastly similar results (Fig. 7). There was greater activation for verbs than nouns in PO, anterior pars triangularis, M1 mouth, and lateral temporo-occipital cortex. The PHG was more active for noun rather than verb generation, whereas the inferior temporal gyrus was more active in verb generation, recapitulating the lateral-to-medial bias of verbs to nouns seen with ECoG. While the middle temporal gyrus appeared to be more active in noun rather than verb generation, it was not active in either one of these conditions when compared with baseline (Price et al. 2005).
Comparisons Between ECoG and fMRI
The relationship between the blood oxygen level-dependent (BOLD) or hemodynamic response, and the underlying electrophysiology, the local field potential (LFP), is an area of intense interest. We evaluated the LFP-BOLD coupling (LBC) function (akin to Conner, Ellmore, Pieters, et al. 2011) for all SDEs per individual using ECoG activity at each electrode in 7 frequency bands as regressors against the fMRI data around each electrode. The overall correlation (Pearson's r) for all 7 bands was 0.18, and just as in the prior analysis, only 2 bands were significant in the regression: Beta (12–30 Hz, regression coefficient = −0.22, P < 10−13) and mid gamma (60–120 Hz, coefficient = 0.15, P < 10−8). When the regression was rerun after reduction to include only these 2 frequency bands, the correlation coefficient was 0.17, and the results for each band were consistent: Beta (coefficient = −0.19, P < 10−16) and mid gamma (coefficient = 0.19, P < 10−16). A final model was fitted using mixed-effects analysis using beta and mid gamma as fixed effects. An additional fixed effect, location of the SDEs defined by lobe, and a random effect, the patient that the SDEs were implanted in, were added to the model. Beta (coefficient = −0.14, F1,2540 = 194.3), mid gamma (coefficient = 0.17, F1,2540 = 407.9), and location (F3,2540 = 87.6) were all significant fixed effects.
Comparison between ECoG and fMRI results at the group level allowed us to evaluate the effect of ECoG group size on the correlation between these 2 measures. Each voxel was assigned a value from the group ECoG MEMA for each frequency band (Fig. 3), from the group fMRI MEMA, and the number of subjects in whom that voxel was sampled (Fig. 2). By varying the minimum level of coverage (i.e. the numbers of patients that had electrodes in that region), the correlation for each LFP band was computed for different ECoG group sizes (Fig. 8). Therefore, with increasing numbers of subjects contributing to a particular voxel, the number of voxels used in the correlation diminishes. Beyond 15 patients, this number is too small to support meaningful analysis. In keeping with the per-subject analysis, we found strong correlation between ECoG and fMRI in the mid- (60–120 Hz) and high-gamma bands (120–240 Hz) (maximum Pearson's r of 0.47 and 0.32, respectively) and a weaker correlation with low gamma (30–60 Hz). This correlation increased with decreasing voxel numbers (and increasing numbers of patients who had coverage in those particular voxels) until the group sizes of 15 subjects.
An important question in the fMRI of cognitive processes is: How many individual datasets does it take to achieve reliable estimates of the underlying process? We evaluated this by measuring the LBC function with varying fMRI population sizes. The full cohort of ECoG samples (n = 19) was used as the benchmark, and the fMRI datasets for these subjects were resampled with replacement for several group sizes (Fig. 8). The voxel-wise correlations of the group fMRI results were computed against all 7 ECoG bands. Voxels were included if they had at least 10 subjects in the group ECoG analysis. The distribution of the 500 resamples for each fMRI group size revealed an improving correlation as the number of subjects in the fMRI increased (2-sided sign test, Bonferroni corrected P < 0.001). This trend stabilized for group sizes >13 subjects, implying that fMRI studies with >13 patients reflects near optimum overlap with electrophysiological measures of function. An important caveat to this analysis is that prior work on a smaller subset of this data has shown that the relationship between BOLD fMRI and the ECoG signal varies over the cortex (Conner, Ellmore, Pieters, et al. 2011). It stands to reason that this estimate of group size may be different for each subregion and could also change based on task demands.
Discussion
In a large cohort with broad field intracranial electrode recordings, we applied a novel, statistically rigorous method of group ECoG to reveal 2 similar, but nonidentical networks that involve components of the ventral and dorsal visual streams during the retrieval of distinct grammatical classes. Saliently, this work definitively shows that language is not a compilation of segregated, serial processes, as some have previously suggested (Levelt et al. 1999). Analysis of timelines across brain regions reveals that disparate areas are active concurrently, implying parallel, nonhierarchical network processing. Distinctions in the spatial extent of activity seen for different tasks are reflected in conditionally dependent dissociations in temporal patterns of activity in those regions. While both the parahippocampal and lateral temporo-occipital regions are active in both noun and verb generation, the early PHG activity (400–500 ms) in the gamma band is greater during noun generation, whereas the SLOC is more active in the same time window during verb generation. These findings validate the critical roles of the PHG in visually cued naming (Luders et al. 1991) and of the SLOC during action perception (Goodale and Milner 1992; Martin et al. 1995) and action decoding (Rizzolatti and Matelli 2003). Taken together, the patterns of ventral and lateral temporo-occipital activities provide unambiguous validation for the dual visual stream hypothesis—a dorsal stream (SLOC) prominently involved in the naming of verbs (Damasio and Tranel 1993), and a ventral stream (PHG) preferentially active for nouns (Ungerleider and Haxby 1994).
Group ECoG MEMA provides high-resolution maps of these network processes underlying language production and the time line of neural activity across disseminated constituents of this network. Robust electrophysiological activity was seen in the ventro-lateral prefrontal cortex (BA45/47) during verb generation (500–1000 ms) as would be expected given the greater role of this region in action perception (Rizzolatti and Matelli 2003) and in syllabification (Indefrey and Levelt 2004). This greater activation of left IFG and premotor cortex is in keeping with prior fMRI and MEG studies contrasting noun and verb production using visual (Soros et al. 2003; Liljestrom et al. 2008, 2009) and auditory cues (Edwards et al. 2010). A novel, conditionally dependent divergence in activity was noted in PO (Figs 3 and 6), where a definitive initial deactivation immediately after stimulus onset in all conditions was followed by a selective increase in gamma power specific for verb naming. This early deactivation suggests that PO may play a higher-order role in the phonologic retrieval process, likely analogous to the role that the right IFG exerts over motor rather than phonologic processes (Swann et al. 2009, 2012). The initial deactivation in PO likely represents a suppression of top–down processes to facilitate uninterrupted local computation in Broca's area (Martin and Chao 2001; Thompson-Schill et al. 2005). Following the initial decrease, activity in PO was noted to reach baseline levels during scrambled and noun naming, but increased markedly and significantly during verb generation. This coincides with the increase in activity that occurs in pars triangularis at the same time (favoring verbs > nouns) and likely reflects the increased semantic and syntactic load during verb generation. Given that prior fMRI data imply a ventro-dorsal gradient in the IFG for syntactic–semantic processes (Hagoort 2005), these time series might be the signature of distinct linguistic processes in these regions. Further insights into the dynamics of this region may be gleaned by applying network analytic strategies to these data.
Activities in other parts of the frontal lobe—caudal IFG, premotor and motor cortices, were broadly identical between the 2 categories of lexical retrieval. The timing of activity across these regions was also roughly similar, suggesting a broad range of interregional cross-talk during the selection, execution, and monitoring of the articulatory plan (Hickok 2012). Finally, auditory responses (monitoring) in the STG were also identical across tasks, suggesting that the scrambled control condition used in this study elicited similar input (in visual and auditory cortices) and output (premotor and motor) responses, allowing for the visual recognition and lexical retrieval processes to be highlighted. Viewed another way, the processing streams for these 3 language processes start and end at similar locations, but have divergent, intermediary paths. Finally, the activation of primary motor and auditory cortices also serves to validate our electrode localization, spatial normalization, and ECoG processing schema.
In dorso-lateral prefrontal cortex (DLPFC), a significant deactivation was noted during both noun and verb generation relative to the scrambled naming condition in both the ECoG and fMRI results. The grouped time series analysis clearly reveals that this “task-related deactivation” truly reflects increased activation of this region during scrambled naming compared with the other conditions. The onset of this activity is during the lexical access portion of task performance (overlapping with activity in the SLOC and PHG), suggesting that early-on, DLPFC processes information regarding the fact that the stimulus is a not an image associated with a particular lexical category, but rather with a rule—to articulate “scrambled”. The DLPFC is integral to task switching, cognitive control, monitoring of behavior (MacDonald et al. 2000; Petrides 2000), and higher-level working memory (du Boisgueheneuc et al. 2006), and several of these processes are in operation here.
The patterns of activity noted for the noun and verb generation were also confirmed using fMRI, both in the patients undergoing ECoG and in a separate group of healthy volunteers, validating the reliability of these data despite the presence of epilepsy. Direct comparisons of the fMRI data with the ECoG analysis showed that activity in the mid-gamma band correlates best with the fMRI signal change and reveals that group sizes of at least 13 subjects are needed in fMRI studies to make reliable interpretations regarding the underlying neural processes. Indeed, we found no lateral temporal activation during any of these tasks, by either modality, except related auditory responses following articulation, even though the importance of this region in the visual naming of nouns has been established in lesion studies (Caramazza and Hillis 1991; Damasio and Tranel 1993). This dissonance might be explained several different ways—the weak activations seen in the lateral temporal cortex in both methodologies are possibly due to the variability in functional organization, which is lower in primary cortices than in higher-order brain regions (Xiong et al. 2000). One possible cause of the difference in variability between these 2 types of the cortex is the level of distributed activity within them (Nir et al. 2007). Cell assemblies representing each lexical concept are not nearly as well delineated as somatotopic or retinotopic cortical columns (Pulvermuller 2005). Ignition and maintenance of cell assembly activation in association cortex is less likely to generate measurable gamma oscillations or BOLD signal change. Alternately, lesions typically affect both cortex and white matter, and connections between occipital/temporo-occipital cortex (inferior fronto-occipital fasciculus and inferior longitudinal fasciculus) and the IFG. It is plausible that the relatively selective impact of temporal lobe lesions on noun generation is consequential to a disconnection of these parahippocampal, posterior fusiform, and inferior temporal gyrus substrates from frontal lobe sites. Importantly, this works shows that the lack of lateral temporal activity during naming is not due to a lack of sensitivity of functional imaging or due to susceptibility artifacts in this area.
In summary, overlapping yet distinct neural substrates in the ventral and dorsal visual streams operate in parallel to result in noun and verb generation, respectively. Grouped MEMA of ECoG data can provide unique insights into cognitive processes. Its correlations with fMRI data provided an independent estimate of group size needed for studies of cognition.
Funding
This work was supported by NIH Center for Clinical and Translational Sciences (KL2 RR0224149 to N.T.), the Vivian Smith Foundation for Neurologic Research, and the Keck Center of the Gulf Coast Consortia, on the Training in Theoretical and Computational Neuroscience, National Institute of Biomedical Imaging and Bioengineering (T32EB006350 to C.R.C.).
Notes
Conflict of Interest: None declared.
References
- Acar ZA, Palmer J, Worrell G, Makeig S. Electrocortical source imaging of intracranial EEG data in epilepsy. Conf Proc IEEE Eng Med Biol Soc. 2011;2011:3909–3912. doi: 10.1109/IEMBS.2011.6090971. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Amunts K, Schleicher A, Burgel U, Mohlberg H, Uylings HB, Zilles K. Broca's region revisited: cytoarchitecture and intersubject variability. J Comp Neurol. 1999;412:319–341. doi: 10.1002/(sici)1096-9861(19990920)412:2<319::aid-cne10>3.0.co;2-7. [DOI] [PubMed] [Google Scholar]
- Badre D, Wagner AD. Left ventrolateral prefrontal cortex and the cognitive control of memory. Neuropsychologia. 2007;45:2883–2901. doi: 10.1016/j.neuropsychologia.2007.06.015. [DOI] [PubMed] [Google Scholar]
- Brysbaert M, New B. Moving beyond Kucera and Francis: a critical evaluation of current word frequency norms and the introduction of a new and improved word frequency measure for American English. Behav Res Methods. 2009;41:977–990. doi: 10.3758/BRM.41.4.977. [DOI] [PubMed] [Google Scholar]
- Caramazza A, Hillis AE. Lexical organization of nouns and verbs in the brain. Nature. 1991;349:788–790. doi: 10.1038/349788a0. [DOI] [PubMed] [Google Scholar]
- Chang EF, Rieger JW, Johnson K, Berger MS, Barbaro NM, Knight RT. Categorical speech representation in human superior temporal gyrus. Nat Neurosci. 2011;13:1428–1432. doi: 10.1038/nn.2641. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chen G, Saad ZS, Nath AR, Beauchamp MS, Cox RW. fMRI group analysis combining effect estimates and their variances. Neuroimage. 2011;60:747–765. doi: 10.1016/j.neuroimage.2011.12.060. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Conner CR, Ellmore TM, Disano MA, Pieters TA, Potter AW, Tandon N. Anatomic and electro-physiologic connectivity of the language system: a combined DTI-CCEP study. Comput Biol Med. 2011;41:1100–1109. doi: 10.1016/j.compbiomed.2011.07.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Conner CR, Ellmore TM, Pieters TA, Disano MA, Tandon N. Variability of the relationship between electrophysiology and BOLD-fMRI across cortical regions in humans. J Neurosci. 2011;31:12855–12865. doi: 10.1523/JNEUROSCI.1457-11.2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cox RW. AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. Comput Biomed Res. 1996;29:162–173. doi: 10.1006/cbmr.1996.0014. [DOI] [PubMed] [Google Scholar]
- Dale AM, Fischl B, Sereno MI. Cortical surface-based analysis. I. Segmentation and surface reconstruction. Neuroimage. 1999;9:179–194. doi: 10.1006/nimg.1998.0395. [DOI] [PubMed] [Google Scholar]
- Damasio AR, Tranel D. Nouns and verbs are retrieved with differently distributed neural systems. Proc Natl Acad Sci USA. 1993;90:4957–4960. doi: 10.1073/pnas.90.11.4957. [DOI] [PMC free article] [PubMed] [Google Scholar]
- du Boisgueheneuc F, Levy R, Volle E, Seassau M, Duffau H, Kinkingnehun S, Samson Y, Zhang S, Dubois B. Functions of the left superior frontal gyrus in humans: a lesion study. Brain. 2006;129:3315–3328. doi: 10.1093/brain/awl244. [DOI] [PubMed] [Google Scholar]
- Edwards E, Nagarajan SS, Dalal SS, Canolty RT, Kirsch HE, Barbaro NM, Knight RT. Spatiotemporal imaging of cortical activation during verb generation and picture naming. Neuroimage. 2010;50:291–301. doi: 10.1016/j.neuroimage.2009.12.035. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Engel AK, Moll CK, Fried I, Ojemann GA. Invasive recordings from the human brain: clinical insights and beyond. Nat Rev Neurosci. 2005;6:35–47. doi: 10.1038/nrn1585. [DOI] [PubMed] [Google Scholar]
- Friston KJ, Holmes AP, Price CJ, Buchel C, Worsley KJ. Multisubject fMRI studies and conjunction analyses. Neuroimage. 1999;10:385–396. doi: 10.1006/nimg.1999.0484. [DOI] [PubMed] [Google Scholar]
- Goodale MA, Milner AD. Separate visual pathways for perception and action. Trends Neurosci. 1992;15:20–25. doi: 10.1016/0166-2236(92)90344-8. [DOI] [PubMed] [Google Scholar]
- Hagoort P. On Broca, brain, and binding: a new framework. Trends Cogn Sci. 2005;9:416–423. doi: 10.1016/j.tics.2005.07.004. [DOI] [PubMed] [Google Scholar]
- Hickok G. Computational neuroanatomy of speech production. Nat Rev Neurosci. 2012;13:135–145. doi: 10.1038/nrn3158. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Indefrey P, Levelt WJ. The spatial and temporal signatures of word production components. Cognition. 2004;92:101–144. doi: 10.1016/j.cognition.2002.06.001. [DOI] [PubMed] [Google Scholar]
- Kanwisher N. Functional specificity in the human brain: a window into the functional architecture of the mind. Proc Natl Acad Sci USA. 2010;107:11163–11170. doi: 10.1073/pnas.1005062107. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kaplan E, Goodglass H, Weintraub S. The Boston Naming Test. Philadelphia: Lea and Febiger; 1983. [Google Scholar]
- Levelt WJ, Roelofs A, Meyer AS. A theory of lexical access in speech production. Behav Brain Sci. 1999;22:1–38. doi: 10.1017/s0140525x99001776. discussion 38–75. [DOI] [PubMed] [Google Scholar]
- Liljestrom M, Hulten A, Parkkonen L, Salmelin R. Comparing MEG and fMRI views to naming actions and objects. Hum Brain Mapp. 2009;30:1845–1856. doi: 10.1002/hbm.20785. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Liljestrom M, Tarkiainen A, Parviainen T, Kujala J, Numminen J, Hiltunen J, Laine M, Salmelin R. Perceiving and naming actions and objects. Neuroimage. 2008;41:1132–1141. doi: 10.1016/j.neuroimage.2008.03.016. [DOI] [PubMed] [Google Scholar]
- Luders H, Lesser RP, Hahn J, Dinner DS, Morris HH, Wyllie E, Godoy J. Basal temporal language area. Brain. 1991;114(Pt 2):743–754. doi: 10.1093/brain/114.2.743. [DOI] [PubMed] [Google Scholar]
- MacDonald AW, III, Cohen JD, Stenger VA, Carter CS. Dissociating the role of the dorsolateral prefrontal and anterior cingulate cortex in cognitive control. Science. 2000;288:1835–1838. doi: 10.1126/science.288.5472.1835. [DOI] [PubMed] [Google Scholar]
- Martin A, Chao LL. Semantic memory and the brain: structure and processes. Curr Opin Neurobiol. 2001;11:194–201. doi: 10.1016/s0959-4388(00)00196-3. [DOI] [PubMed] [Google Scholar]
- Martin A, Haxby JV, Lalonde FM, Wiggs CL, Ungerleider LG. Discrete cortical regions associated with knowledge of color and knowledge of action. Science. 1995;270:102–105. doi: 10.1126/science.270.5233.102. [DOI] [PubMed] [Google Scholar]
- Mesgarani N, Chang EF. Selective cortical representation of attended speaker in multi-talker speech perception. Nature. 2012;485:233–236. doi: 10.1038/nature11020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nir Y, Fisch L, Mukamel R, Gelbard-Sagiv H, Arieli A, Fried I, Malach R. Coupling between neuronal firing rate, gamma LFP, and BOLD fMRI is related to interneuronal correlations. Curr Biol. 2007;17:1275–1285. doi: 10.1016/j.cub.2007.06.066. [DOI] [PubMed] [Google Scholar]
- Ojemann G, Ojemann J, Lettich E, Berger M. Cortical language localization in left, dominant hemisphere. An electrical stimulation mapping investigation in 117 patients. J Neurosurg. 1989;71:316–326. doi: 10.3171/jns.1989.71.3.0316. [DOI] [PubMed] [Google Scholar]
- Petrides M. The role of the mid-dorsolateral prefrontal cortex in working memory. Exp Brain Res. 2000;133:44–54. doi: 10.1007/s002210000399. [DOI] [PubMed] [Google Scholar]
- Pieters TA, Conner CR, Tandon N. Recursive grid partitioning on a cortical surface model: an optimized technique for the localization of implanted subdural electrodes. J Neurosurg. 2013;118:1086–1097. doi: 10.3171/2013.2.JNS121450. [DOI] [PubMed] [Google Scholar]
- Price CJ, Devlin JT, Moore CJ, Morton C, Laird AR. Meta-analyses of object naming: effect of baseline. Hum Brain Mapp. 2005;25:70–82. doi: 10.1002/hbm.20132. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pulvermuller F. Brain mechanisms linking language and action. Nat Rev Neurosci. 2005;6:576–582. doi: 10.1038/nrn1706. [DOI] [PubMed] [Google Scholar]
- Rizzolatti G, Matelli M. Two different streams form the dorsal visual system: anatomy and functions. Exp Brain Res. 2003;153:146–157. doi: 10.1007/s00221-003-1588-0. [DOI] [PubMed] [Google Scholar]
- Sahin NT, Pinker S, Cash SS, Schomer D, Halgren E. Sequential processing of lexical, grammatical, and phonological information within Broca's area. Science. 2009;326:445–449. doi: 10.1126/science.1174481. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sahin NT, Pinker S, Halgren E. Abstract grammatical processing of nouns and verbs in Broca's area: evidence from fMRI. Cortex. 2006;42:540–562. doi: 10.1016/s0010-9452(08)70394-0. [DOI] [PubMed] [Google Scholar]
- Salmelin R, Hari R, Lounasmaa OV, Sams M. Dynamics of brain activation during picture naming. Nature. 1994;368:463–465. doi: 10.1038/368463a0. [DOI] [PubMed] [Google Scholar]
- Snodgrass JG, Vanderwart M. A standardized set of 260 pictures: norms for name agreement, image agreement, familiarity, and visual complexity. J Exp Psychol Hum Learn Mem. 1980;6:174–215. doi: 10.1037//0278-7393.6.2.174. [DOI] [PubMed] [Google Scholar]
- Soros P, Cornelissen K, Laine M, Salmelin R. Naming actions and objects: cortical dynamics in healthy adults and in an anomic patient with a dissociation in action/object naming. Neuroimage. 2003;19:1787–1801. doi: 10.1016/s1053-8119(03)00217-9. [DOI] [PubMed] [Google Scholar]
- Swann N, Tandon N, Canolty R, Ellmore TM, McEvoy LK, Dreyer S, DiSano M, Aron AR. Intracranial EEG reveals a time- and frequency-specific role for the right inferior frontal gyrus and primary motor cortex in stopping initiated responses. J Neurosci. 2009;29:12675–12685. doi: 10.1523/JNEUROSCI.3359-09.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Swann NC, Cai W, Conner CR, Pieters TA, Claffey MP, George JS, Aron AR, Tandon N. Roles for the pre-supplementary motor area and the right inferior frontal gyrus in stopping action: electrophysiological responses and functional and structural connectivity. Neuroimage. 2012;59:2860–2870. doi: 10.1016/j.neuroimage.2011.09.049. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Szekely A, D'Amico S, Devescovi A, Federmeier K, Herron D, Iyer G, Jacobsen T, Arevalo AL, Vargha A, Bates E. Timed action and object naming. Cortex. 2005;41:7–25. doi: 10.1016/s0010-9452(08)70174-6. [DOI] [PubMed] [Google Scholar]
- Tandon N. Mapping of human language. In: Yoshor D, editor. Clinical brain mapping. New York, NY: McGraw Hill; 2012. [Google Scholar]
- Thompson-Schill SL, Bedny M, Goldberg RF. The frontal lobes and the regulation of mental activity. Curr Opin Neurobiol. 2005;15:219–224. doi: 10.1016/j.conb.2005.03.006. [DOI] [PubMed] [Google Scholar]
- Tranel D, Adolphs R, Damasio H, Damasio AR. A neural basis for the retrieval of words for actions. Cogn Neuropsychol. 2001;18:655–674. doi: 10.1080/02643290126377. [DOI] [PubMed] [Google Scholar]
- Ungerleider LG, Haxby JV. ‘What’ and ‘where’ in the human brain. Curr Opin Neurobiol. 1994;4:157–165. doi: 10.1016/0959-4388(94)90066-3. [DOI] [PubMed] [Google Scholar]
- Vidal JR, Ossandon T, Jerbi K, Dalal SS, Minotti L, Ryvlin P, Kahane P, Lachaux JP. Category-specific visual responses: an intracranial study comparing gamma, beta, alpha, and ERP response selectivity. Front Hum Neurosci. 2010;4:195. doi: 10.3389/fnhum.2010.00195. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Viechtbauer W. Conducting meta-analyses in R with the metafor package. J Stat Softw. 2010;36:1–48. [Google Scholar]
- Wada J, Rasmussen T. Intracarotid injection of sodium amytal for the lateralization of cerebral speech dominance. 1960. J Neurosurg. 2007;106:1117–1133. doi: 10.3171/jns.2007.106.6.1117. [DOI] [PubMed] [Google Scholar]
- Wise RJ, Greene J, Buchel C, Scott SK. Brain regions involved in articulation. Lancet. 1999;353:1057–1061. doi: 10.1016/s0140-6736(98)07491-1. [DOI] [PubMed] [Google Scholar]
- Xiong J, Rao S, Jerabek P, Zamarripa F, Woldorff M, Lancaster J, Fox PT. Intersubject variability in cortical activations during a complex language task. Neuroimage. 2000;12:326–339. doi: 10.1006/nimg.2000.0621. [DOI] [PubMed] [Google Scholar]