Dynamics of large-scale cortical interactions at high gamma frequencies during word production: Event related causality (ERC) analysis of human electrocorticography (ECoG)

Abstract

Intracranial EEG studies in humans have shown that functional brain activation in a variety of functional-anatomic domains of human cortex is associated with an increase in power at a broad range of high gamma (> 60 Hz) frequencies. Although these electrophysiological responses are highly specific for the location and timing of cortical processing and in animal recordings are highly correlated with increased population firing rates, there has been little direct empirical evidence for causal interactions between different recording sites at high gamma frequencies. Such causal interactions are hypothesized to occur during cognitive tasks that activate multiple brain regions. To determine whether such causal interactions occur at high gamma frequencies and to investigate their functional significance, we used event-related causality (ERC) analysis to estimate the dynamics, directionality, and magnitude of event-related causal interactions using subdural electrocorticography (ECoG) recorded during two word production tasks: picture naming and auditory word repetition. A clinical subject who had normal hearing but was skilled in American Signed Language (ASL) provided a unique opportunity to test our hypothesis with reference to a predictable pattern of causal interactions, i.e. that language cortex interacts with different areas of sensorimotor cortex during spoken vs. signed responses. Our ERC analyses confirmed this prediction. During word production with spoken responses, perisylvian language sites had prominent causal interactions with mouth/tongue areas of motor cortex, and when responses were gestured in sign language, the most prominent interactions involved hand and arm areas of motor cortex. Furthermore, we found that the sites from which the most numerous and prominent causal interactions originated, i.e. sites with a pattern of ERC “divergence”, were also sites where high gamma power increases were most prominent and where electrocortical stimulation mapping interfered with word production. These findings suggest that the number, strength and directionality of event-related causal interactions may help identify network nodes that are not only activated by a task but are critical to its performance.

1. Introduction

Complex human cognitive tasks, especially word production tasks, require the operation of large-scale cortical networks during which causal interactions are likely to occur between network components or modules that are functionally specialized for different aspects of task processing (Levelt, 2001; Marinkovic, 2004; Mesulam, 1990). In each moment of a cognitive task, causal influences may be responsible for the selection of network components for activation and subsequent interactions with other components across a large-scale cortical network (Bressler and Tognoli, 2006; Mainy et al., 2008; Marinkovic, 2004; Wennekers et al., 2006). Thus, to fully understand the neurophysiological dynamics of language and other complex cognitive tasks, it may be necessary not only to measure the location and timing of functional brain activation in detail, but also to measure how different cortical network components causally influence one another.

The patterns of causal influences among network components under different functional conditions or perturbations have been referred to as “effective connectivity” (Friston, 1994; Sporns, 2007). This concept adds a potentially important dimension to functional mapping because it suggests that the functional role of any given cortical site is defined not only by its activation during a particular set of tasks and/or task conditions, but also by its pattern of causal interactions with other sites, i.e. its effective connectivity (Bressler and Tognoli, 2006). Indeed, activation of a cortical site may not necessarily indicate a key role in task processing if it does not also have an impact on processing in other cortical sites engaged by a task. This distinction could be particularly relevant when attempting to distinguish participating vs. necessary sites among those activated during a cognitive task. In addition, because causal interactions are expected to contribute to network function on a timescale relevant to that of cognitive tasks, i.e. tens to hundreds of milliseconds, effective connectivity is arguably best studied in electrophysiological recordings such as ECoG.

Many previous studies have suggested a role for gamma oscillations in the integration of cortical processing across large-scale brain networks, including those responsible for complex cognitive functions (Bressler and Kelso, 2001; Engel et al., 2001; Jerbi et al., 2009; Palva et al., 2002; Rodriguez et al., 1999; Varela et al., 2001). Most of these studies have focused on the cross-correlation of gamma oscillations within a conceptual framework of parallel processing and functional integration across cortical networks. Studies adopting this framework have provided insights into the functional connectivity of cortical networks, but because they are agnostic with respect to causal influences between network components, they do not address the effective connectivity of these networks. Functional activation of cortex is associated with an increase in signal energy at a wide range of gamma frequencies. Recent studies using intracranial EEG have found that functional responses occur most consistently at gamma frequencies higher than 60 Hz (Crone et al., 2006; Crone et al., 2011).

These so-called ”high gamma” responses have been observed in every major functional-anatomic domain of human cortex, and although they are best recorded with invasive EEG recordings, they can also be observed noninvasively in scalp EEG (Ball et al., 2008; Darvas et al., 2010; Lenz et al., 2008) and MEG (Dalal et al., 2008; Kaiser and Lutzenberger, 2005). Regardless of where and how these responses have been recorded, however, they appear to be highly specific for the location and timing of cortical processing. The neural mechanisms generating these responses are not known in detail, but recent recordings of local field potentials and multi-unit activity in animals have indicated that the magnitude and timing of these responses are highly correlated with increased firing rates in local cortical neurons (Ray et al., 2008a). Although this would suggest a potential impact on downstream cortical processing, there is little direct empirical evidence for causal influences at high gamma frequencies between distantly separated cortical sites. To test whether these causal interactions occur and can be used to image the effective connectivity of cortical networks serving language function, we recorded intracranial EEG in a patient undergoing surgery for intractable epilepsy and estimated the timing and directionality of event-related causal interactions at high gamma frequencies during complementary word production tasks, i.e. picture naming and auditory word repetition.

A popular approach for measuring causal influences in electrophysiological recordings, and thus the effective connectivity of large-scale cortical networks, is to use multivariate autoregressive (MVAR) models (Astolfi et al., 2007b; Cadotte et al., 2008; Cadotte et al., 2009; Chen et al., 2009; Dauwels et al., 2010; Eichler, 2006; Gow and Segawa, 2009; Pereda et al., 2005; Schlogl and Supp, 2006; Supp et al., 2007; Takahashi et al., 2007). One advantage of this approach over others (e.g. (Hinrichs et al., 2008; Liang et al., 2001; Tononi and Sporns, 2003)) is that it is capable of determining the frequencies at which causal interactions occur. The directed transfer function (DTF), for example, has been widely used to estimate the spectral characteristics, as well as the strength and directionality, of causal influences (Kaminski and Blinowska, 1991) under a variety of normal and pathological conditions. DTF and related methods have also been employed to investigate causal influences in fMRI data (Deshpande et al., 2008; Deshpande et al., 2009; Deshpande et al., 2006; Hamilton et al., 2010; Hinrichs et al., 2006; Sato et al., 2009; Wilke et al., 2009), but electrophysiological recordings such as EEG, MEG, and ECoG are better suited to investigate the dynamics of causal influences on timescales relevant to the timing of cognitive tasks, i.e. evolving over hundreds of milliseconds.

To capture the temporal evolution of task-related causal influences, i.e. during different stages of functional tasks, various modifications of MVAR model fitting have been made (Astolfi et al., 2007a; Astolfi et al., 2007b; Astolfi et al., 2009; Giannakakis and Nikita, 2008; Wilke et al., 2007; Wilke et al., 2009). One of these (Ding et al., 2000) is the short-time directed transfer function - SDTF (Blinowska et al., 2010; Ginter et al., 2001; Ginter et al., 2005; Kaminski et al., 2005; Kus et al., 2006; Kus et al., 2008; Philiastides and Sajda, 2006), which uses multiple trials of a cognitive task to estimate causal influences in short time windows. Subsequent modifications of this approach have included the short-time direct directed transfer function (SdDTF), which selectively estimates direct causal influences between recording sites, i.e. not mediated by other sites (Korzeniewska et al., 2003), as well as event-related causality – ERC, which tests the statistical significance of task-related (event-related) changes in SdDTF (Korzeniewska et al., 2008). When the latter approach was used in a previous study of auditory word repetition, we found spatial and temporal patterns of causal interactions at high gamma frequencies that were generally consistent with putative task demands (Price, 2000). However, because the effective connectivity of language cortex and the dynamics of causal interactions during language tasks are not well established, additional experimental confirmation is needed, preferably under circumstances where more concrete experimental predictions can be made.

Intracranial EEG recordings in a clinical subject with normal hearing who was skilled in American Signed Language (ASL), offered a rare and valuable opportunity to test our main hypothesis, i.e. that causal interactions occur at high gamma frequencies, with reference to a predictable pattern of causal interactions, i.e. that perisylvian language cortex would have causal interactions with the more or less distinct areas of sensorimotor cortex responsible for spoken vs. signed responses. We also compared the results of our ERC analyses with the timing and magnitude of high gamma responses, and with the results of routine electrocortical stimulation mapping (ESM) during the same or similar language tasks.

2. Materials and Methods

2.1 Event-Related Causality Analysis

To evaluate the spatial-temporal patterns of causal interactions between recordings sites of multichannel ECoG data, a multivariate autoregressive (MVAR) model is fitted to the recorded signals (as detailed in (Korzeniewska et al., 2008)). The K analyzed signals are treated as a vector output of a multivariate stochastic process and can be expressed as:

$$\mathbf{x}(t)=-{\displaystyle \sum _{j=1}^p{\mathbf{A}}_j\mathbf{x}(t-j)+\mathbf{e}(t)}$$ (1)

where the values of components of vector x at a time t are linear combinations of p previous values and the zero mean uncorrelated residual noise vector e(t). To determine the optimal value of the model order p, the Akaike Information Criterion was used (Akaike, 1974). The matrix coefficients of the model can be calculated by solving Yule-Walker equations (r=1,…, p):

$${\displaystyle \sum _{j=0}^p{\mathbf{A}}_j\mathbf{R}(-r+j)=0}$$ (2)

where A₀ is equal to the KxK identity matrix, A_j is a KxK MVAR coefficients matrix, and R(n) is a covariance matrix of x at lag n. For a single realization of the stochastic process, the unbiased estimate of the covariance matrix can be computed from (T denotes vector transposition):

$$\mathbf{R}(n)=\frac{1}{N-n}{\displaystyle \sum _{t=1}^{N-n}\mathbf{x}(t){\mathbf{x}}^T(t+n)}$$ (3)

Where x̄ is the vector of recorded data, and N is the number of samples. To follow the temporal course of brief changes in signal propagation between different brain regions, an algorithm introduced by Ding et al. (Ding et al., 2000) was used for MVAR coefficient estimation. Multiple repetitions of the same task can be treated as multiple realizations of a stochastic process with locally stationary segments. This allows one to estimate the covariance matrices R as the average of L estimators computed for each trial. To evaluate spectral properties of the analyzed signals, equation (2) is transformed to the frequency domain (Marple, 1987)

$$\mathbf{H}(f)={\left({\displaystyle \sum _{j=0}^p{\mathbf{A}}_j{e}^{-i2\pi jf\Delta t}}\right)}^{-1}$$ (4)

where H(f) is the transfer function of the multi-channel system, f is frequency, and Δt is the sampling interval. The element h_kl of the matrix H(f) describes the transfer function between the l-th input and the k-th output of the system. If the element h_kl of H is equal to zero, the hypothesis that x_l(t) causes x_k(t) can be rejected. The matrix is not symmetric, meaning that transfer from channel k to channel l is different from that in the opposite direction. In this way the directional properties of a multi-channel system may be obtained and interpreted as causal relationships (in the Granger causality sense), signal flows, or activity transfers. Note that a value of H(f) significantly different from zero does not indicate that the causal relationship between recording sites is direct; the interaction may be mediated by other sites. To distinguish direct causal relationships from indirect ones, we have multiplied elements h_kl(f) of matrix H by partial coherence X_kl(f) (Korzeniewska et al., 2008; Korzeniewska et al., 2003). A nonzero value of partial coherence indicates that the relationship between channels k and l is a direct one at the given frequency. Since partial coherence describes direct relationships between signals, and the transfer matrix H(f) measures the directed relationships between them, a combination of the two measures results in a measure of directed flows of activity between recording sites not confounded by indirect interactions. The resulting function normalized to the 0–1 range, called short-time direct directed transfer function (SdDTF), is defined by the formula:

$${\zeta }_{\mathit{\text{kl}}}(f)=\frac{|h_{\mathit{\text{kl}}}(f)\Vert X_{\mathit{\text{kl}}}(f)|}{\sqrt{{\displaystyle \sum _f{\displaystyle \sum _{\mathit{\text{kl}}}|h_{\mathit{\text{kl}}}(f){|}^2|X_{\mathit{\text{kl}}}(f){|}^2}}}}$$ (5)

The non-zero values of SdDTF may be interpreted as causal interactions or flows of activity from one channel to another one, i.e. ζ_kl(f)>0 indicates flow of activity from channel l to channel k (l→k). The temporal evolution of these estimates is obtained by calculating SdDTF in short windows shifted in time.

To test whether causal interactions occurring after stimulus onset are statistically different from those occurring in the pre-stimulus baseline, and thus whether they are event- or task-related, SdDTF values in each post-stimulus window are compared with SdDTF values in each baseline window. Smoothing is conducted using semiparametric regression based on penalized thin-plate splines (Korzeniewska et al., 2008; Ruppert et al., 2003; Crainiceanu et al., 2005) and correction for testing multiplicity is done via the Bonferroni correction. This statistical procedure does not assume stationarity of the signal either in the pre- or post-stimulus period and compares the smoothed SdDTF value of each baseline window with the smoothed SdDTF value of each post-stimulus window. We have employed a bivariate smoothing model in time-frequency space, defined as:

$$Y_{\mathit{\text{ft}}}=g(f,t)+{\epsilon }_{\mathit{\text{ft}}}$$ (6)

where Y_ft is the empirical average of SdDTF over all trials, g(f,t) is an unspecified function representing the actual SdDTF function and ε_{f t} are independent $N(\mathit{0},{\sigma }_{\epsilon }^{\mathit{2}})$ random noise residual. The estimation is based on the Restricted Maximum Likelihood (REML) criterion, which is based on the maximum likelihood principle exploiting the link between the penalized criterion and mixed models - for technical details see Ruppert et al. (Ruppert et al., 2003).

To test for every frequency f, and for every baseline/stimulus pair of time windows (t,T), whether g(f,t)=g(f,T), the implicit null hypothesis is that

$${\mathrm{H}}_{0,\mathrm{f},\mathrm{T}}:g(f,t_1)=g(f,T)\text{or}g(f,t_2)=g(f,T)\text{or}\dots \text{or}g(f,t_n)=g(f,T)$$ (7)

where f₁, …, f_m are the frequencies, m is the number of analyzed frequencies, t = t₁, …, t_n is the time index corresponding to one window in the baseline, n is the total number of baseline windows, T=T₁, …, T_k is the time index corresponding to one window in the post-stimulus period, and k is the total number of post-stimulus windows. To test these hypotheses we constructed a joint 95% confidence interval for the differences g(f, t)-g(f, T) for t=t₁, …, t_n. Let ĝ(f, t), σ̂_g(f, t) be the penalized spline estimator of g(f,t) and its associated estimated standard error in each baseline time window. Similarly, let ĝ(f,T), σ̂_g(f, t) be the penalized spline estimator of g(f,T) and its associated estimated standard error in each post-stimulus time window. Since residuals are independent at points well separated in time, the law of large numbers applies, and we can assume that for every baseline/stimulus pair of time windows (t,T), the null distribution is well approximated by a standard normal distribution. A joint confidence interval with at least 95% coverage probability for g(f, t)-g(f, T) is

$$\hat{g}(f,t)-\hat{g}(f,T)\pm m_{.95}\sqrt{{\hat{\sigma }}^2(f,t)+{\hat{\sigma }}_g^2(f,T)}$$ (8)

where m.₉₅ is the 97.5% quantile of the distribution

$$\begin{array}{ccc}\mathit{\text{MAX}}(t_n,T_k){=}_{t_1\le t\le t_n,}& \underset{T_1\le T\le T_k,}{\text{max}}& {}_{f_1\le f\le f_m}|N_{t,T,f}|\end{array}$$ (9)

where N_t,T,f are independent N(0,1) random variables. This test rejected H_0,f,T if zero was not contained in any of the corresponding confidence intervals. The t-test was designed for the null hypothesis of zero differences between the SdDTF means. The normalizing standard error is the standard deviation of the estimated mean difference. To account for multiple testing, the family wise error rate (FWER) was controlled using the Bonferroni correction. The number of tests for correction was calculated as a product of n, k, m, and the number of pairs of channels i.e. K*(K-1) This ensured that our statistical approach was conservative.

Event-Related Causality (ERC) corresponds to significant differences in smoothed SdDTF between baseline and post-stimulus periods. Thus, ERC is designed to estimate the intensity, directionality, spectral characteristics, and time course of statistically significant event-related changes in causal interactions or activity flows between recording sites. In this study the threshold for statistical significance was set as p=0.05 (95% confidence interval) after Bonferroni correction for number of tests (K=23, f_m=51, t_n=43,T_n=250, 268, 523 - depending on the task). For a more intuitive visualization of effective connectivity, the ERC results integrated over relevant frequency bands were depicted as arrows connecting recording sites. Only statistically significant increases in directed causal interactions (ERC flows) were used for this visual representation. The width of the arrows and their color represented the magnitude of these ERC flows.

ERC is a linear method and does not give any information about the nature (linear or not) of the causality. Nevertheless, a few points are worth noting in this regard: (1) linear methods may be sensitive to both linear and nonlinear causal interactions (Chavez et al., 2003; Freiwald et al., 1999; Gourevitch et al., 2006), (2) the detection of dependencies by linear methods does not necessitate that the dependencies are of linear nature (Freiwald et al., 1999), (3) multivariate autoregressive models can be used to describe output from nonlinear systems (Franaszczuk and Bergey, 1999), (4) linear techniques are more robust for systems with noise, e.g. electrophysiological data, which is never free of the presence of additive noise (Netoff et al., 2004), and models with a higher degree of non-linearity do not seem to provide any significant advantages over linear ones (Barbero et al., 2009), and (5) functions similar to SdDTF (directed transfer function (DTF), direct directed transfer function (dDTF), directed coherence and partial directed coherence (PDC)) appear to correctly identify linear linkages even if the autoregressive parts are nonlinear (Gourevitch et al., 2006), and return similar estimates of cortical connectivity from neuroelectric measurements (Astolfi et al., 2009). Thus, although the ERC method cannot determine whether the observed activity flow changes are due to linear vs. nonlinear dynamics, it may, nevertheless, detect them.

2.2 Matching Pursuit

Event-related time-frequency energy analysis began with the matching pursuit (MP) algorithm (Boatman-Reich et al., 2010; Cervenka et al., 2011; Mallat and Zhang, 1993; Ray et al., 2003; Ray et al., 2008a; Ray et al., 2008b). MP is an iterative procedure that decomposes a signal x(t) into a linear combination of members of a specified set of functions a_γm(t) called atoms,

$$x(t)={\displaystyle \sum _{m=0}^{M-1}\langle D^{m}x,a_{\gamma m}\rangle a_{\gamma m}(t)+D^{M}x(t)}$$ (10)

where <x,a> denotes the inner product of functions x and a, D^mx is the residue after m steps of iteration, and D⁰x=x. In each step the matching pursuit algorithm choosesthe a_γm that maximizes the inner product <D^mx,a>. The dictionary of functions a_γm contains cosine-modulated Gaussians (Gabor functions), cosines, and Dirac delta functions. Gabor functions provide the best compromise between time and frequency resolution. Cosines are added to the dictionary for better representation of rhythmic components with constant amplitude, and a Dirac delta is added to represent very short transients. In this study we used a dyadic Gabor dictionary with sub-sampling in time in the lowest two octaves and sub-sampling in frequency in the two highest octaves with functions defined as:

$$a_{\gamma }(t)=F(\gamma )e^{-\pi {\left(\frac{t-u}{s}\right)}^2}\text{cos}(2\pi \mathit{\text{ft}}+\phi )$$ (11)

where F(γ) is factor normalizing a_γ,_φto unity, γ=(s,u,f) is an index in the dictionary, s is scale, u is translation in time, and f is modulation frequency of atom a_γ. This resulted in improved time resolution for small scale atoms and improved frequency resolution for large scale atoms. Time-frequency plots for each trial were obtained by summing the Wigner-Ville distributions of individual atoms:

$$E_x(t,f)={\displaystyle \sum _{m=0}^M{\left|\langle D^{m}x,a_{\gamma m}\rangle \right|}^2{\mathit{\text{Wa}}}_{\gamma m}(t,f)}$$ (12)

where Wa denotes the Wigner distribution of the atom a. This approach is especially useful because it does not include cross-terms. Atoms representing common artifacts (e.g. power line 60 Hz and its harmonics) are excluded from the summation.

To find task-related changes in the time-frequency distribution of energy we compared the energy distribution after stimulus onset with the baseline period of 1 second preceding stimulus onset. This assumed that the ECoG signal was governed by a random process characterized by its time-frequency energy distribution E(t,f). Furthermore, we assumed that within the baseline segment of the EEG preceding the onset of the stimulus at time t_S, the average of energy over trials was not dependent on time. To make the distribution symmetric we performed a log transform of these energy density estimates, i.e. log(E(t,f)). To reduce the variance due to noise and jitter of the response we applied 2D (two dimensional) Gaussian smoothing to the energy distribution using the following kernel:

$$G(t,f)=\frac{1}{2\pi {\sigma }_t{\sigma }_f}{e}^{-\frac{1}{2}\left(\frac{t^2}{{\sigma }_t^2}+\frac{f^2}{{\sigma }_f^2}\right)}$$ (13)

where σ_t and σ_f represent the width of the gaussian in time and frequency, respectively. In this study we used the same σ in both dimensions, equal to one resolution unit, i.e. 2 msec and 0.488 Hz respectively. In numerical computations we used σ_t = σ_f = 1 resulting in a normalized kernel matrix of size 7×7 resolution units. The estimate of baseline energy E_B(f) was obtained by averaging over time in the baseline interval, as well as over all included trials:

$$E_B(f)=\frac{1}{T_{B}L}{\displaystyle \sum _{l=1}^L{\displaystyle \sum _{B=1}^{T_B}\text{log}(E_l(t_B,f))}}$$ (14)

where T_B was the number of time points in the baseline (i.e. t_B <t_S), and L was the number of trials. For the post-stimulus signal we computed the energy density estimate E_S (t,f) by averaging El over L trials for each t>t_S:

$$E_S(t,f)=\frac{1}{L}{\displaystyle \sum _{l=1}^L\text{log}(E_l(t,f))}$$ (15)

Event-related changes were represented by a difference E_S(t,f)-E_B(f). Because the estimates were obtained using log transformed data, this difference was proportional to the logarithm of the energy ratio and was measured in dB as a relative increase or decrease of power in frequency f at time t. We were interested in finding times and frequencies for which significant changes in the spectrum of the signal occurred. For a given frequency f and time t>t_S, we tested the null hypothesis that E_S(t,f)= E_B(t,f). A natural way to test this hypothesis was using the Z statistic:

$$Z=\frac{E_S(t,f)-E_B(f)}{\mathit{\text{SE}}(t,f)}$$ (16)

where SE(t,f) was a pooled standard error estimator. Because the number of trials was usually ≥100, the averages of log energy were well approximated by a standard normal distribution (central limit theorem) and the Z statistics followed the Student t distribution when the null hypothesis was true. The Jarque-Bera test confirmed the normality of the distribution of E_B in our data. We calculated Z for N_f frequencies and N_t time points after the stimulus; therefore, in assessing the significance of task-related changes after stimulus onset, we made a correction for multiple (N_f * N_t) tests using a false discovery rate procedure (FDR) (Benjamini and Hochberg, 1995; Benjamini and Yekutieli, 2001). For this analysis we use a false discovery rate (p=0.05) to correct for the number of tests (N_f =1025, 2049, N_t=2048, 4096 - depending on the task).

2.3 Subject and Experiment

Our subject was a 47-year-old, right-handed woman with a history of medically refractory complex partial seizures that had begun at 39 years of age and were occurring on average once or twice per week. Her hearing, vision, and history of motor and language development were normal. She was a native speaker of American English and had been a professional sign language interpreter since 31 years of age. Her brain MRI was normal. Her full scale IQ was 116. Left intracarotid amobarbital injection suppressed speech and sign language, and right injection mildly suppressed speech but not sign language. Video-EEG monitoring with scalp electrodes failed to satisfactorily lateralize her seizures. Monitoring with depth electrodes in bilateral orbitofrontal cortices, amygdalae, and hippocampi recorded interictal discharges in left hippocampus and twelve seizures with onset in the left hippocampus. Because the onset of ictal activity followed the onset of ictal behavior in several seizures, the patient subsequently underwent implantation of grids over the left hemisphere to rule out a neocortical focus. Four seizures were recorded with ictal activity maximal at B15 and F63 (Fig 1). Stimulation of these sites provoked pain that prevented language testing. Given the potential risk of language impairment, no surgical resection was performed. There were no post-operative complications, and there was no improvement in the patient’s seizures.

Fig. 1.

Anatomical locations of subdural electrodes and montages used for ECoG recording and ERC analyses. White discs indicate electrodes implanted for clinical purposes. Numbered discs (except B15) indicate the subset of recording sites chosen for analysis with ERC based on prior analysis of event-related high gamma augmentation. Left panel - all electrodes analyzed, middle panel - electrodes of montage (a), right panel - electrodes of montage (b). Letter B indicates basal temporal-occipital strip, F - frontal-temporal grid, and P frontal-parietal grid. Central sulcus marked by black dots.

Electrode positions were determined through co-registration of pre-implantation MRI with post-implantation CT, and cortical gyral anatomy was obtained through surface rendering of the pre-implantation MRI (Sinai et al., 2005). Subdural ECoG was digitally recorded (1 kHz sampling) during two simple single word production tasks: auditory word repetition and picture naming. The word repetition task used 116 spoken word stimuli detailed previously (Crone et al., 2001b). The naming task used a set of 84 pictures of objects derived from the Boston Naming Test. The subject was asked to repeat the words aloud or to name the pictured object as soon as possible. Both tasks were performed twice: once with spoken responses and once with signed (American Sign Language) responses. ECoG was also recorded during a visual-motor task during which the subject performed an isometric muscle contraction in response to a visual cue for the body part lasting 3 seconds. The muscle contraction consisted of either tongue protrusion or fist clenching (41–50 trials). Because the recording apparatus at the time was limited to simultaneous recordings of only 64 channels, two different montages were used in order to obtain ECoG data in all implanted electrodes. The picture naming task was recorded with both montages, allowing coverage of either (a) perisylvian and frontoparietal regions, including hand area of sensorimotor cortex, or (b) perisylvian and basal temporal regions (Fig. 1). Time limitations permitted recordings of the word repetition task with only the montage (b). Electrode sites were selected for ERC analyses based in part on the patterns of event-related high gamma power augmentation during the same tasks, reported previously (Crone et al., 2001b).

2.4 Data

To obtain reference-independent spatial maps of spectral measures, and to de-emphasize more global activity unrelated to the functional task without introducing systematic biases from the reference electrode (Crone et al., 2001a), a common average reference was used (Ludwig et al., 2009; Yao et al., 2007; Yao et al., 2005). ECoG recordings were visually inspected, and channels and trials with artifacts were removed. One of the electrodes where ictal activity was maximal (B15), was not included, and one (F63) was included because of its anatomic location. We did not expect causal interactions due to epilepsy to occur with a consistent temporal correspondence to trials of the word production task, i.e. to consistently occur as an event-related change from baseline, and no event-related high gamma augmentation or prominent language-related ERC flows were observed at this site. Selection of ECoG channels for SdDTF and matching pursuit (MP) analyses was based on the results of previous spectral analyses (Crone et al., 2001b) demonstrating event-related power augmentation in high gamma frequencies (80–100 Hz) during word production tasks in the same patient.

Preliminary MP analysis showed that event-related activity extended no higher than to 180 Hz. Thus, for ERC analysis the raw recordings were downsampled to a sampling frequency of 333.33 Hz and band-pass-filtered (using Finite Impulse Response - FIR - filter) to avoid including artifacts identified in full band MP analysis. Signals in each channel were normalized by subtracting the means and dividing by the variance over time and trials (Ding et al., 2000; Korzeniewska et al., 2008; Oya et al., 2007). For each dataset the common multivariate autoregressive (MVAR) model order was determined using AIC criterion, and the coefficients were computed for 360 msec windows, shifted by 0.015 msec over 4 seconds (including pre- and post-stimulus epochs) for all selected trials of the experiment. The values of SdDTF were then calculated according to eq. 5 for a high gamma frequency band between 70 and 115 Hz (to avoid artifacts of 60 Hz and 120 Hz), and SdDTF estimates in the pre-stimulus baseline were compared with those in consecutive stages of task performance.

For MP analyses signals were down sampled to a sampling frequency of 500 Hz. The MP decomposition was performed on signals of length 2048 samples (4.096 sec, for repetition and picture naming tasks) or 4096 samples (8.192 sec, for motor tasks, where a part of the task was a 3 sec fixation, and responses were longer), yielding 2048×1025 or 4096×2049 arrays of time-frequency values (with a time resolution of 2 msec, and frequency resolution of 0.24 Hz or 0.12 Hz, respectively). Energy in the time-frequency plane was calculated from atoms for each trial according to eq. 12. Line noise (60 Hz and harmonics) is represented in large part by long Gabor atoms (spread along the time axis) with energy concentrated around 60 Hz and its harmonics. Similarly, a sinusoidal stimulus artifact is represented by long atoms with energy concentrated at the stimulus frequency and its harmonics. Such atoms were excluded from energy computations and further analysis. This procedure does not eliminate all contributions of line and stimulus artifacts but is less destructive than traditional notch filtering because it preserves atoms, and thus energy, at these frequencies that may have physiologic origin.

3. Results

The opportunity to record ECoG in a subject skilled in signed language was an unusual one and allowed us to investigate dynamic cortical network interactions during word production in two different output modalities: verbal (speech) and manual (signing). Because the clinical circumstances for this patient did not require electrode coverage of occipital cortex or posterior fusiform gyrus, we could not study in detail the cortical interactions associated with the perceptual aspects of the picture naming task, i.e. visual object recognition. Nevertheless, there was extensive coverage of perisylvian cortex and sensorimotor cortex, as well as sampling of basal temporal-occipital cortices (Fig. 1). Many, but not all, of the subdural electrodes from which our ECoG recordings and ERC analyses were made, were also tested with electrocortical stimulation mapping (ESM) (Fig. 2). Because the patient’s clinical circumstances did not require ESM at all implanted electrodes, some of the sites where ERC analyses indicated involvement during language and motor activation, were not tested with ESM.

Fig. 2.

Results of electrocortical stimulation mapping (ESM). Colored bars indicate the effects of bipolar stimulation between pairs of electrode sites: Red - involuntary tongue movement, Pink - involuntary hand movement, Dark purple - spoken picture naming and auditory sentence comprehension (modified Token Test) impaired, Light purple - signed picture naming impaired. Green electrodes - ESM had no effect on naming. White electrodes - motor function and naming not tested with ESM. Numbered discs indicate the subset of recording sites used for ESM and subset chosen for analysis with ERC.

3.1 Picture naming: spoken vs. signed responses

The ERC method estimates the directions and magnitudes of statistically significant, event-related changes in direct activity propagation between brain sites, as a function of frequency. Thus, we often describe our results here using a conceptual shorthand in which we refer to event-related increases in directed causal interactions as “ERC flows”. Decreases of ERC were not taken into consideration because the physiological interpretation of flow decreases during task performance is less clear. In all illustrations of ERC results (e.g. Figure 3), the aforementioned ERC flows at high gamma frequencies are represented by colored arrows. Arrows indicate the directions of ERC flows (in comparison to the pre-stimulus baseline) between recording sites, and the intensities of these flows (integrated over the frequency range of 70–115 Hz). The intensities are coded by the color (from yellow to red – color scale at the left) and the widths of arrows. In each figure, the magnitudes of statistically significant ERC flows are integrated over behaviorally relevant time intervals during which ERC flows appeared to be relatively stable. For language tasks these intervals were defined by stimulus duration and/or response latency. For example, in Figure 3 depicting picture naming, the left panel shows ERC flow integrals likely corresponding to object recognition and word retrieval, i.e. from stimulus onset to median response onset; the right panel shows ERC flow integrals most likely corresponding to word production, i.e. integrated over 750 msec following the median response latency (patient responses rarely lasted more than 750 msec).

Fig. 3.

Picture naming task with spoken (top panels) vs. signed responses (bottom panels), recorded with montage (a). Integrals of ERC flows at high gamma frequencies (70–115 Hz) were calculated for two sequential time intervals. Left panels correspond to object recognition and word retrieval stages (i.e. from stimulus onset to median response onset). Right panels correspond to the word production stage (i.e. 750 msec following the median response latency). Arrows indicate the directions and intensities of statistically significant ERC flow increases between recording sites. The width and color of each arrow both represent linearly the magnitude of its integral ERC flow. Color scale (at the left) is of the same range for all ERCs (Figs. 3–8), scaled from maximal to minimal integral of ERC magnitude. 10% off the smallest ERCs are not shown.

3.1.1 Picture naming with spoken responses

Prior to spoken responses (Fig. 3 top-left), site F31 in posterior left inferior frontal gyrus (LIFG), at the junction of ventral premotor cortex (BA6 – Brodmann’s area 6) and pars opercularis of LIFG (BA44), was the source of many ERC outflows (flows directed from F31 into other sites). The targets of these ERC flows included a variety of perisylvian sites, including other sites in posterior LIFG and sites in middle temporal gyrus (MTG), superior temporal sulcus (STS), and superior temporal gyrus (STG). ESM at some of these sites interfered with spoken picture naming (F45 and F53). Targets of ERC flows from F31 also included sites in ventral sensorimotor cortex, including sites where ESM interfered with tongue movement.

During spoken responses to the pictured objects (Fig. 3 top-right), several prominent ERC outflows were directed across the sylvian fissure (e.g. into F53, F54, and F48). In addition, during both task stages (Fig. 3 top panels) there was a significant ERC flow from LIFG (F31) to dorsal sensorimotor cortex (P8). Although ESM at sites adjacent to P8 interfered with hand motor function, ESM at P8, located in post-central parietal cortex, did not interfere with motor function but did interfere with signed picture naming. We speculate that the subject’s long experience with simultaneous word production in both spoken and signed language may have been responsible for this ERC flow.

3.1.2 Picture naming with signed responses

Prior to signed responses to the picture naming task (Fig 3 bottom-left), there were also significant ERC flows from F31 in posterior LIFG, but, they were less prominent than when responses were spoken. The most prominent ERC flows were directed from superior parietal cortex near hand area of sensorimotor cortex (P7), to a variety of perisylvian regions, especially into MTG (F62) and posterior STG (F64, likely BA22 in Wernicke’s area). ESM at P7 interfered with picture naming only when responses were signed. ESM at several adjacent sites interfered with hand function.

During signed responses to the picture naming task (Fig. 3 bottom-right) prominent ERC flows from the hand area of sensorimotor cortex (P7) persisted, and some of them were even greater, e.g. into LIFG. Although the patient was instructed not to move her tongue or mouth while signing her responses, we observed ERC flows involving mouth/tongue areas during signed responses, analogous to the ERC flows involving hand areas during spoken responses. The absence of tongue and mouth movements was confirmed during picture naming, as much as possible, by review of video recordings of her performance.

As a contrast with the word production tasks, ERC analysis was also performed for the following visually cued motor tasks: tongue protrusion (Fig. 4 top), and fist clenching (Fig. 4 bottom). Whereas spoken picture naming (Fig. 3 top) had prominent ERC flows from F31 in posterior LIFG, the most prominent ERC flows during tongue protrusion (Fig. 4 top) originated in F32, in ventral pre-central gyrus, likely mouth/tongue premotor/motor cortex. Whereas the most prominent ERC flows during signed picture naming originated from P7 in dorsal post-central gyrus, likely hand somatosensory cortex, the most prominent ERC flows during fist clenching originated from pre-central gyrus (P12), a site where ESM interfered with hand motor function. These results suggest that the strong ERC flows originating in F31 and P7 during picture naming were specific to word production.

Fig. 4.

Visually cued motor tasks: tongue protrusion (top) and fist clenching (bottom), recorded with montage (a). Integrals of ERC flows at high gamma frequencies (70–115 Hz) are illustrated with the same color scale as in Fig. 3.

3.1.3 Picture naming: causal interactions with basal temporal-occipital (BTO) cortex

The picture naming task was also recorded with montage (b), which included sites in basal temporal-occipital cortex (BTO) (Fig. 1). ERC flows recorded with this montage are illustrated in Fig. 5 (spoken response) and Fig. 6 (signed response). This montage allowed us to investigate task-related causal interactions between a few sites in the ventral object processing stream and perisylvian language cortex during word production. Because recordings of this subject were limited to 64 channels at a time, the electrode grid (P) over hand area of sensorimotor cortex that was recorded in montage (a), was not recorded in montage (b) (Fig. 1). Instead, the electrode strip over BTO (B) was included.

Fig. 5.

Picture naming task with spoken responses, recorded with montage (b). Integrals of ERC flows at high gamma frequencies (70–115 Hz) were calculated for three time intervals. Panel a - object recognition (0 to 500 msec after visual stimulus onset). Panel b - word retrieval (500 msec to median response onset). Panel c - spoken word production (750 msec following the median response latency). Color scale - same as in Fig. 3.

Fig. 6.

Picture naming task with signed responses, recorded with montage (b). Integrals of ERC flows at high gamma frequencies (70–115 Hz) during three task stages, as in Figure 5. Color scale - same as in Fig. 3.

Removing or adding signals in a multivariate autoregressive (MVAR) model is expected to change the interactions among the remaining signals, particularly if the signals that are added or removed participate in strong interactions (Eichler, 2005; Korzeniewska et al., 2008). Fewer changes are expected when the substituted signals are less important for the recorded system (Korzeniewska et al., 2008). However, in our patient several of the electrode substitutions across the two montages involved sites with prominent ERC flows. Consistent with the importance of these flows, the patterns of ERC flows observed with the two montages were different. Nevertheless, many statistically significant flows survived these substitutions, albeit with different relative magnitudes. It is also worth mentioning here that the aforementioned differences could have also been due in part to the effects of repeating the same language tasks while recording the two montages at different times. The recordings with montage (a), described above (Fig. 3), were made on the day following the recordings with montage (b). Our patient had somewhat shorter response latencies each time the tasks were repeated. This could have compressed the temporal course of cortical activation and ERC flows, affecting their significance and magnitude. Likewise, we cannot rule out other learning effects, such as repetition suppression, i.e. less activation with repeated exposure, which has been seen in fMRI experiments on comparable time scales (van Turennout et al., 2000).

At an early stage of word retrieval during picture naming, from 0 to 500 msec after appearance of the visual stimulus, there were prominent bottom-up ERC flows from B10 and other sites in fusiform gyrus to sites in perisylvian cortex, regardless of whether responses were spoken (Fig. 5 left) or signed (Fig. 6 left). In contrast, during later stages of word retrieval, i.e. after activation of BTO sites but before response onset (Fig. 5 middle), the most prominent ERC outflows originated at F31 in posterior LIFG, similar to those observed with montage (a) (Fig. 3 top-left). However, as mentioned above, replacing seven electrodes did change the relative strengths of the observed ERC flows. In particular, ERC outflows from F31 were observed with both montages, but the flows with greatest intensities had different targets. These differences could have been due in part to the fact that in both early (Fig. 5 left) and late (Fig. 5 middle) stages of word retrieval, prominent ERC flows involved sites (e.g. B7, B10, and B16) that were not recorded at all in montage (a).

Note that ERC analyses of picture naming recorded with montage (a) revealed flows (Fig. 3) that were relatively stable during two time intervals corresponding to word retrieval (between stimulus onset and median response onset) and word production (750 msec after the median response onset). However, ERC analyses of picture naming recorded with montage (b) (Fig. 5, Fig. 6) revealed very early and brief (0–500 msec) flows originating from BTO (B10)(left-most panels in Figs. 5 and 6) These flows included prominent bottom-up flows from BTO sites in fusiform gyrus to perisylvian cortex that may have reflected access to lexical and phonological representations during and/or after object recognition in the ventral visual object processing stream (Borowsky et al., 2007; Oliver et al., 2009). Cortical activation indexed by high gamma responses had its earliest onset and briefest duration at BTO sites, consistent with their involvement in visual object recognition, which is the earliest processing requirement for word retrieval and subsequent production during naming. The early, brief bottom-up ERC flows temporally overlapped this activation. These flows could not be observed with montage (a), which did not include the BTO electrode. All ERC flows observed with montage (a) were relatively stable during the whole interval of word retrieval, between stimulus onset and response onset. It is possible that some of the dynamics during this larger interval analyzed for montage (a) corresponded to dynamics in either the first or the second, or both, smaller intervals analyzed for montage (b).

3.2 Auditory word repetition: spoken vs. signed responses

3.2.1 Auditory word repetition: spoken responses

ERC integrals at high gamma frequencies (70–115 Hz) during the auditory word repetition task are illustrated in Fig. 7 (spoken responses) and Fig. 8 (signed responses). For the aforementioned reasons, these tasks were only recorded with montage (b) (Fig 1). When responses were spoken (Fig. 7), many ERC flows originated in auditory association cortex in STG (e.g. F45, F55) at times corresponding to auditory word perception, i.e. from stimulus onset to stimulus offset (Fig. 7 left). The targets of these flows included sites in posterior LIFG, posterior MTG, and mouth/tongue area of sensorimotor cortex. There were also prominent ERC flows within STG, between STG and BTO, between MTG and BTO, and between LIFG and BTO. ERC outflows were also observed from F31 in LIFG, where many outflows had originated during word retrieval for picture naming (see above). Many of these flows involved sites where ESM interfered with spoken picture naming (F45, F55), or with tongue movement (F37).

Fig. 7.

Auditory word repetition task with spoken responses, recorded only with montage (b). Integrals of ERC flows at high gamma frequencies (70–115 Hz) during three task stages: a - auditory word perception (from stimulus onset to median stimulus offset), b - word retrieval and response preparation stage (stimulus offset to median response onset latency), and c - spoken response (750 msec following the median response latency). Color scale - same as in Fig. 3.

Fig. 8.

Auditory word repetition task with signed responses. Organization of plots as in Fig. 7. Color scale - same as in Fig. 3.

During the interval between the mean stimulus offset and the mean response onset, presumably corresponding to word retrieval and response preparation (Fig. 7 middle), the most prominent ERC flows originated in LIFG (F28 and F31), presumably including parts of Broca’s area (BA44/45) and ventral premotor cortex (BA6). These ERC flows were more intense than during the previous stage and included new flows, for example into F32. The targets of many of these ERC flows (F32, F39, F48) likely corresponded to ventral sensorimotor cortex supporting articulation. Although ESM was not done at F32, ESM at F39 and F48 interfered with mouth/tongue movements. Prominent ERC flows were also observed from F31 in LIFG to posterior STG and MTG, possibly reflecting preparation of receptive speech areas with a forward model or “efference copy” of the upcoming articulatory program (Flinker et al., 2010; Rauschecker, 2011; Ventura et al., 2009).

During the spoken response itself, (Fig. 7 right), ERC flows were still observed within ventral sensorimotor cortex, and from other perisylvian sites. Nevertheless, like the same time interval during picture naming, the ERC flows observed during the response itself were less numerous and prominent than those observed earlier in the task, prior to the mean response onset latency, possibly reflecting a greater demand for network interactions during lexical retrieval and preparation for articulation than during the motor realization of speech.

3.2.2 Auditory word repetition (translation): signed responses

ERC analyses of auditory word repetition with signed responses (translation from spoken to signed words) revealed fewer and less prominent ERC flows (Fig. 8) compared with the same task with spoken responses (Fig. 7). The most prominent causal interactions during this task would be expected to occur between auditory cortex and hand area of sensorimotor cortex, and perhaps superior parietal cortex, as observed during picture naming with signed responses (see above). However, the montage (b) used for recording this task did not cover these areas. Nevertheless, the results for this task offer an important contrast for the ERC flows observed during auditory word repetition with spoken responses. In particular, the relative absence of flows observed with montage b during this translation task and during picture naming with signed responses, supports the task specificity of the observed ERC flows. In addition, the flows observed during this task were consistent with its putative processing demands. During auditory word perception (Fig. 8 left), for example, ERC outflows from STG were observed, as they were during word repetition with spoken responses.

3.3 Comparison of ERC flows with event-related energy changes at high gamma frequencies

We noticed that the cortical sites participating in prominent causal interactions according to ERC analysis often had the greatest event-related power augmentation at high gamma frequencies according to time-frequency analysis with matching pursuits (MP). The latter analysis was not intended to investigate functional interactions between cortical sites, but merely functional activation of individual sites during the task. Thus, direct comparisons of ERC flows to event-related power changes (calculated from MP) were not possible. We did, however, endeavor to make a qualitative comparison between these two electrophysiological measures of brain dynamics. At each ECoG recording site, we summed the integrated magnitude of ERC outflows from that site, and compared it to the time-frequency pattern of power changes (MP) at that site during the same task stages (Fig. 9). The largest sum of ERC outflows, marked as the largest circle, occurred at site F31 during the word retrieval stage of the task (Fig. 9 top-left). The most prominent event-related high gamma power augmentation (obtained with MP) was observed at the same site (Fig. 9 bottom). This power augmentation occurred primarily during the word retrieval stage of the task, which was in agreement with the integral ERC outflows during the same timeframe (Fig. 3a top-left).

Fig. 9.

Comparison of event-related causal interactions vs. functional activation during picture naming with spoken responses. Top panels illustrate the relative magnitudes of ERC outflows from each site during two sequential time intervals (as in Fig. 3 top): object recognition and word retrieval stages (i.e. from stimulus onset to median response onset), word production stage (i.e. 750 msec following the median response latency). The radius of each circle is proportional to normalized sum of statistically significant event-related increases in causal interactions directed outwardly from the site (sum of ERC flow arrows originating at a site). Bottom panel illustrates the results of MP analysis of event-related ECoG power changes in the time-frequency plane for the same task, for each channel labeled in the top panels. Vertical axis of each time-frequency plot is frequency (0–250 Hz); horizontal axis is time in seconds. Vertical marker (r) indicates the median response latency, which divides the two time intervals for which ERC outflows are illustrated in panels a and b. Visual stimulus onset occurs at 0 seconds (labeled s). Magnitudes of statistically significant ECoG power changes (in decibels) are plotted with color scale to the right of the plots. Orange-red colors indicate event-related power augmentation (usually in high gamma frequencies), and green-blue colors indicate event-related power suppression, usually in alpha and beta bands. The black background indicates no significant change in power. Straight black lines connecting panels a and b with panel c indicate sites with significant high gamma augmentation at time intervals (task stages) when prominent ERC outflows are observed.

A similar comparison of summed ERC outflows and event-related high gamma augmentation during picture naming with signed responses (Figure 10) also revealed a correspondence between these two measures of cortical dynamics for both stages of the task, most prominently at the site P7, but also at adjacent sites. As noted before, ESM testing at P7 interfered with signed picture naming. In addition, there was a correspondence between summed ERC outflows at F31 and event-related high gamma augmentation, primarily during the word retrieval stage of the task.

Fig. 10.

Comparison of event-related causal interactions vs. functional activation during picture naming with signed responses. Sums of ERC outflows (purple circles, see also Fig. 3 bottom) vs. ECoG power changes (MP analysis) are illustrated as in Fig. 9. Plots of ECoG power changes are shown for sites with prominent high gamma augmentation (orange-red).

Analysis of auditory word repetition with spoken responses, (Figure 11), also demonstrated a correspondence between summed ERC outflows and event-related high gamma augmentation. Both measures were highly dependent on the temporal stage of task performance. For example, during perception of the auditory word stimuli, the largest sums of ERC outflows, marked as the largest circle, occurred at the sites F31, F45 and F55 (Fig. 11 left). The most prominent event-related high gamma power increases were observed at the same sites. During response preparation, both methods emphasized the role of site F31 (Fig. 11 middle). During the spoken response itself, site F40 was emphasized (Fig. 11 right); ESM at this site interfered with tongue movement. Site F31 was more active, according to both ERC and MP, during stimulus perception and response preparation stages (Fig. 11 left and middle) than during the response itself (Fig. 11 right). A similar effect was observed for spoken and signed picture naming (Fig. 9 left and middle, 10 left and middle). Together, these findings suggest that site F31 played a key role in lexical retrieval and/or articulatory programming in anticipation of the spoken response (see below for further discussion).

Fig. 11.

Comparison of event-related causal interactions vs. functional activation during auditory word repetition with spoken responses. Sums of ERC outflows (purple circles, see also Fig. 7) and plots of MP results are illustrated as in Figs 9 and 10. Three panels illustrate ERC outflows during three sequential time intervals: auditory word perception (stimulus onset to median stimulus duration, left panel), word retrieval and response preparation (between median stimulus offset and median response onset, middle panel), and spoken response (750 msec following the median response latency, right panel). Time-frequency plots of event-related ECoG power changes are shown for sites with prominent high gamma augmentation. Stimulus onset is at 0 seconds. Vertical markers indicate the times for median stimulus offset (o) and the median response onset (r).

Figure 12 plots a quantitative comparison of values of event-related functional activation vs. causal interactions, shown in Figs. 9–11, for all analyzed sites. Small changes in ECoG energy (X-axis) coincided with small ERC outflows (Y-axis), and there was no apparent correlation between their magnitudes. However, for high gamma energy changes of greater intensity, e.g. logarithm of the change > 0.09 (marked by vertical line), the relationship appeared to have linear characteristics. Although the threshold of 0.09 varied, prominent event-related high gamma augmentation appeared to be correlated with a divergent pattern of ERC outflows. Note that this relationship did not appear to depend on the task or the task stage.

Fig. 12.

Comparison of values of event-related functional activation vs. causal interactions. X axis represents mean ECoG power changes in the frequency range 70–115 Hz (chosen to match the range used for ERC), for each site, and each task stage, as depicted in Figs. 9–11. Y axis represents mean magnitude of ERC outflows (70–115 Hz) from each site, for each task and stage (Figs. 9–11). Both X and Y axes are in logarithmic scale.

4. Discussion

4.1 Objectives and approach

Recent intracranial EEG studies in humans have found that functional activation of cortex is associated with power augmentation in a wide range of “high gamma” frequencies, typically at frequencies greater than 60 Hz and without a well-defined peak in the power spectrum. These “high gamma” responses have been observed during functional activation in a variety of functional-anatomic domains, including cortical systems supporting motor (Ball et al., 2008; Brovelli et al., 2005; Brunner et al., 2009; Crone et al., 1998; Dalal et al., 2008; Miller et al., 2007; Ohara et al., 2000; Pfurtscheller et al., 2003), auditory (Chang et al., 2010; Crone et al., 2001a; Edwards et al., 2005; Edwards et al., 2009; Kaiser and Lutzenberger, 2005; Ray et al., 2003; Steinschneider et al., 2008; Trautner et al., 2006), visual (Lachaux et al., 2005; Siegel et al., 2007; Tallon-Baudry et al., 2005; Tanji et al., 2005), and language (Brown et al., 2008; Canolty et al., 2007; Crone et al., 2001b; Crone et al., 2006; Edwards et al., 2010; Jung et al., 2008; Lachaux et al., 2007; Mainy et al., 2008; Sinai et al., 2005; Tanji et al., 2005; Towle et al., 2008) functions. Although this high gamma augmentation has also been observed noninvasively in MEG (Dalal et al., 2008; Gunji et al., 2007; Kaiser and Lutzenberger, 2005) and scalp EEG recordings (Ball et al., 2008; Darvas et al., 2010; Lenz et al., 2008), it is observed with higher fidelity in intracranial EEG recordings, including subdural ECoG and depth recordings (Ball et al., 2009; Jerbi et al., 2009).

Recent animal studies have confirmed the functional specificity of high gamma augmentation, and have shown that it is highly correlated with mean population firing rates (Belitski et al., 2008; Liu and Newsome, 2006; Manning et al., 2009; Ray et al., 2008a), as well as with regional cerebral blood-flow (Brovelli et al., 2005; Logothetis et al., 2001). In humans, they are highly specific for the location and timing of functional activation at individual cortical sites and have been used to study the functional specialization of local cortical populations activated by different tasks and task components (Crone et al., 2006). However, to date there has been little empirical evidence that high gamma activity can be used to identify causal influences of one cortical region on another (Korzeniewska et al., 2008; Marinkovic, 2004). Such causal interactions are hypothesized to occur during complex human cognitive tasks that activate widely distributed brain regions and in which processing in one region has an influence on processing in another region. Based on the tight correspondence between high gamma responses and local cortical processing, we hypothesized that event-related causal interactions between different cortical areas would be observed at high gamma frequencies during word production tasks, and that the number, magnitude, timing and directionality of these interactions would depend on the functional demands of different word production tasks, e.g. their stimulus and response modalities. To measure event-related causal interactions, we used a newly developed method, called event-related causality (ERC), based on Granger causality and multivariate autoregressive (MVAR) modeling. This method allowed us to capture event-related causal influences at high gamma frequencies that were consistent across trials of each task, and we compared the patterns of causal influences with the patterns of single-site cortical activation at the same frequencies. Although ERC analyses allowed us to examine causal interactions at a resolution of hundreds of milliseconds, the most prominent ERC flows typically spanned several hundred milliseconds (Korzeniewska et al., 2008), suggesting quasi-stable network states corresponding to different temporal stages of task-related network activity. Based on this observation, we integrated the observed ERC flows over time intervals derived from behavioral task parameters (stimulus duration, response latency).

An ideal test of our hypotheses would start with experimental predictions based on a priori knowledge of the patterns of event-related causal influences responsible for the language tasks we used. However, the conceptual framework of effective connectivity and the methods used to probe it are quite new, and the dynamics of causal influences between different cortical regions have not been previously explored at comparable temporal and spatial scales. Furthermore, the functional anatomy of language and the dynamics of its activation can vary substantially across individuals. We therefore chose to test our hypotheses against the most basic and conservative prediction that the patterns of causal interactions measured with ERC would depend on the functional demands of tasks using different stimulus modalities (visual vs. auditory) and response modalities (speech vs. gesture). A clinical subject who had normal hearing but was fluent in sign language provided a rare and valuable opportunity for this test.

In addition to demonstrating their dependence on stimulus and response modalities, we also expected event-related causal influences to be consistent with existing models of the functional anatomy and dynamics of word production. This aspect of the interpretation of our results was necessarily more speculative due in part to the evolving and complex nature of neuroanatomically constrained models (Golfinopoulos et al., 2010; Hickok, 2009; Hickok and Poeppel, 2007; Humphreys et al., 1999; Indefrey and Levelt, 2004; Levelt, 1999, 2001; Petkov et al., 2009; Price, 2000; Rauschecker and Scott, 2009; Shalom and Poeppel, 2008) and the lack of specific predictions with respect to effective connectivity in many of these models. Psycholinguistic models of word production have proposed interactions between network nodes at different levels of representation, including implicit or explicit causal influences of one level on another (Levelt, 1999), but the variety, complexity, and incomplete anatomical specification of these models complicates their application to our results. Another important caveat is that the ECoG recordings of picture naming in our subject had limited sampling of regions in the ventral visual object processing stream that play an important role in models of picture naming and likely have prominent causal influences on perisylvian language regions. Although the electrode coverage in our patient offered more comprehensive coverage of the important functional-anatomic components of auditory word repetition, this task was not recorded with montage (a) and thus we could not compare ERC flows during spoken vs. signed responses.

The ERC flows we observed during both word production tasks appeared consistent with existing models of these tasks derived from lesion analyses, psychophysiological studies, electrophysiological studies, and from functional neuroimaging. However, we also interpreted the ERC flows we observed by comparing their spatiotemporal patterns with the magnitude and timing of functional activation at individual ECoG recording sites, as well as with the results of electrocortical stimulation mapping (ESM), a method based on interference with cortical function that is still widely considered a clinical gold standard.

4.2 Dependence of causal interactions on response modality

The dependence of event-related causal interactions at high gamma frequencies on response modality was best observed in recordings using montage (a), in which we could compare the effective connectivity of mouth/tongue vs. hand areas of sensorimotor cortex. When responses were spoken, we observed stronger ERC outflows from a site (F31) in posterior left inferior frontal gyrus. Although ESM was not done at this site, it was adjacent to sites where ESM interfered with mouth/tongue motor function (F31). When responses were signed, we observed prominent ERC outflows from a site (P7) where ESM interfered with picture naming with signed responses but not with hand movement. This site was, however, adjacent to sites where ESM interfered with hand movement. Because montage (b) included only mouth/tongue areas of sensorimotor cortex, the dependence on response modality could only be deduced from very strong ERC outflows during spoken responses vs. very weak ERC outflows during signed responses.

Overall, the observed patterns of effective connectivity were in agreement with our experimental prediction of dependence on response modality. It may appear incongruent that ERC flows involving mouth/tongue areas were sometimes observed during signed responses, and that ERC flows involving hand area were sometimes observed during spoken responses. However, our subject was a professional sign language interpreter and had routinely produced words with simultaneous spoken and signed output. Thus word retrieval could have involved obligatory activation of regions serving both kinds of word production. In this case, it would not be surprising for both areas of sensorimotor cortex to interact with perisylvian areas responsible for processing lexical-semantic and phonological representations. Furthermore, because spoken and signed words may have been routinely translated into one another, it may not be surprising that the two regions of premotor cortex responsible for their respective motor realizations interacted with each other during both spoken and signed word production.

A particularly interesting finding was the predominance of ERC outflows at high gamma frequencies from the aforementioned regions, particularly from F31 and P7, during word retrieval, i.e. before the onset of spoken and signed responses, respectively. These outflows were directed to a variety of sites in and around perisylvian language regions where semantic, lexical, and phonological representations for words have been posited in neuroanatomically constrained models of word production (Price, 2000). Given the proximity of F31 and P7 to sensorimotor cortex responsible for articulation and manual control, these outflows may have reflected top-down causal influences from sites mediating articulatory or gestural output, onto sites responsible for semantic, lexical, and/or phonological representations of the target words (see 4.5 below for further discussion). Most anatomically constrained models of spoken word production, however, imply that the predominant directionality of the flow of processing is from these intermediate representations to sites coordinating articulation (Indefrey and Levelt, 2004). We speculate that the observed top-down causal influences may have reflected an executive control mechanism that monitors, maintains, and/or reinforces the activation of these intermediate representations arising from bottom-up influences from perceptual processing of the visual object or spoken word stimuli. Feed-forward influences to F31 and mouth/tongue sensorimotor cortex were more evident during spoken word repetition than during picture naming (Fig. 7), in which response latencies were more variable and in which important sources of these influences, e.g. from temporal-occipital cortex (see below), were likely missing.

Cognitive models of word production developed from psychophysiological studies have proposed processing networks with nodes at conceptual/semantic, lexical, and phonological levels of representation (Levelt, 1999, 2001). Activation typically spreads through these networks in a cascading fashion, and there are bidirectional interactions between the aforementioned levels of representation. In Dell’s interactive lexical network model (Dell, 1986), for example, lexical selection is driven by a syntactic frame for the utterance and results in additional activation of lexical and phonological nodes. Although picture naming does not require sentence-level syntactic processing, it does require selection of a single noun from other potential lexical targets with similar semantic and phonological features. Posterior LIFG has been activated in neuroimaging studies during syntactic processing (Price, 2000), and the top-down ERC flows we observed from F31 in posterior LIFG could have reflected such a selection process. In addition, lexical selection could have been constrained by representations for phonological output codes, which have also been localized to LIFG (Indefrey and Levelt, 1999), or by representations for articulatory gestures, perhaps facilitated by the proximity of F31 to sites where ESM interfered with mouth/tongue movement. The dynamics of ERC outflows we observed from P7 were similar to those we observed from F31, and we speculate that these reflected a selection process driven by upper extremity gestural codes for signed responses analogous to the articulatory gestural codes for spoken responses.

4.3 Dependence of causal interactions on stimulus modality

Although the clinical circumstances of our subject did not require electrode coverage of early visual cortex in the occipital lobe, we were able to record from several electrodes over fusiform gyrus and adjacent BTO cortex. These areas are part of the ventral visual object processing stream and have been shown in previous studies to be activated by tasks requiring discrimination or identification of visual symbols and objects (Halgren et al., 1994; Nobre et al., 1994; Tanji et al., 2005). They are also part of the basal temporal language area, which has been extensively studied in epilepsy patients (Luders et al., 1986; Moore and Price, 1999; Pawlik et al., 1990). In recordings with montage (b), which included these sites, we observed stronger and more numerous bottom-up ERC flows during an early stage of picture naming than during a comparable stage of auditory word repetition. During object naming, 0 to 500 milliseconds after onset of the visual stimulus, there was prominent event-related high gamma power augmentation at some of these sites, regardless of whether responses were spoken (Fig. 5 left) or signed (Fig. 6 left). In both cases, we also observed prominent ERC flows that originated at the site B10 over fusiform gyrus, as well as other less prominent ERC flows to and from BTO. When responses were to be spoken, this ERC flow was directed to the temporal-parietal junction (F64), likely in BA22 and within Wernicke’s area (Fig. 5 left), possibly reflecting access to phonological representations for the target words. In contrast, when responses were to be signed, this ERC flow was directed to the LIFG e.g. F37 (Fig. 6 left). This is in agreement with functional studies reporting BTO to be involved in both receptive and expressive functions of language and to have strong functional connectivity with perisylvian cortex (Ishitobi et al., 2000; Matsumoto et al., 2004). We expected to observe a prominent ERC flow from B10 to hand area of sensorimotor cortex when responses were to be signed, but we were not able to record this subject with coverage of both areas at the same time.

The dependence of functional interactions on stimulus modality was also evident in the patterns of ERC flows observed between auditory association cortex in posterior STG and other perisylvian sites recorded with both montages. Posterior STG is well known to be robustly activated during perceptual processing of auditory stimuli, especially speech (Crone et al., 2001a; Edwards et al., 2005; Edwards et al., 2009; Fukuda et al., 2010), and is responsible for constructing sound-based representations of speech during both speech perception and speech production (Hickok and Poeppel, 2000). Thus, causal interactions involving this area are expected when linguistic stimuli and/or responses are spoken. During the earliest (perceptual) stages of auditory word repetition (Fig. 7 and Fig. 8), there were many ERC flows from auditory cortex in posterior STG to other perisylvian sites, including Broca’s area in LIFG, MTG, ventral premotor (vPM) cortex, and inferior sensorimotor cortex. As noted above, these flows were more evident when responses were to be spoken (Fig. 7 left) than when they were to be signed (Fig. 8 left), probably because the latter involved superior parietal sites not recorded during this task. Nevertheless, the dominant sources of ERC flows were in STG.

Prominent ERC flows were also observed among sites within posterior STG (BA41/42/22) whenever word production was cued by an auditory stimulus, i.e. during perceptual processing of spoken words. These flows occurred regardless of whether the responses to these stimuli were to be spoken or signed (Fig. 7 left and Fig. 8 left). The directionalities of these ERC flows suggested a predominantly feed-forward flow of processing within auditory association cortex from low level (BA41/42) to higher level representations of the acoustic stimulus in nearby cortical regions. Similar flows were not observed during picture naming, probably because electrode coverage in visual cortex was much more limited. When responses to auditory words were to be spoken, ERC flows also extended from auditory cortex to sites in LIFG and inferior sensorimotor cortex responsible for word retrieval and articulatory programming, respectively. In contrast, when word production was cued by visual stimuli, auditory cortex (BA41/42/22) was not activated at stimulus onset, i.e. no significant event-related high gamma augmentation, and was not a prominent source for event-related causal interactions (Fig. 3 left, Fig. 5 left, and Fig. 6 left). Instead, early interactions with temporal lobe (e.g. F62, B7, and B16) primarily involved sites outside of BA41/42 and only rarely involved BA22.

4.4 Correspondence between ERC and the expected location and timing of cortical activation

Although we could not compare our results with previously established patterns of causal interactions, we could compare the causal interactions we observed with the location (Binder et al., 1997; Price, 2000) and timing of cortical activation that has been reported in previous studies of similar word production tasks. Perhaps the most detailed model of both the location and timing of brain activation during word production, however, is that of Indefrey and Levelt (2004). This model is largely consistent with classical models of the functional anatomy of language (Shalom and Poeppel, 2008), but also specifies a temporal sequence for different processing stages in different brain regions. Based on their meta-analysis of 82 prior neuroimaging experiments, Indefrey and Levelt proposed five regions and time ranges for core representations/processes in word production: retrieval of a lexical semantic concept represented in a widely distributed semantic network, retrieval of a target lemma (abstract word form without its specific phonological form) in the middle temporal gyrus, retrieval of the lexical phonological output code in posterior middle and superior temporal gyri, formulation of a syllabified phonological output in posterior inferior frontal gyrus, and implementation of the articulatory command in inferior precentral and postcentral gyrus with self-monitoring in superior temporal gyrus. Even though the representations at some of the aforementioned stages may depend on representations at earlier stages, it is not necessary for each stage to be completed before another stage begins, and thus parallel and/or cascading flows may be observed at the level of neural populations.

Consistent with other clinical subjects undergoing epilepsy surgery, our subject’s spoken responses were slower than those of the normal subjects, on which the model of Indefrey and Levelt were based. If this difference is taken into account, the first stage in Figs 5 and 6 (labeled object recognition) may very roughly correspond to the first two stages of the Indefrey and Levelt model, the intermediate stage (word retrieval) to the next two stages, and the last stage (spoken response) to the last stage. Note that the first stage (word retrieval) in Figure 3 represents a combination of the first two stages in Figs 5 and 6 because the montage did not include sites likely responsible for object recognition. The ERC flows from BTO cortex during the first stage in Figs 5 and 6 may thus correspond to the retrieval of the target lemma and/or the lexical phonological output code in middle and superior temporal gyri, respectively, which may be causally influenced by the results of visual object processing and retrieval of the object’s lexical semantic representation. On the other hand, the ERC flows during the second stage in Figs 5 and 6 from posterior LIFG (BA44) and inferior sensorimotor cortex to posterior superior temporal gyrus may correspond to the process of syllabification and phonetic encoding of articulatory gestures in LIFG, already mentioned above (see 4.2).

Our results were also generally consistent with the timing and location of cortical activation reported in previous studies of word production using ECoG high gamma as an index of cortical activation, including studies of picture naming (Edwards et al., 2010), word repetition (Brown et al., 2008), and auditory cued movement (Nagasawa et al., 2010).

4.5 Relationship between event-related causal influences and the magnitude of cortical activation

We found that when the number and magnitude of outflows indexed by ERC were integrated over task-relevant time windows, the individual ECoG sites with the most dominant ERC outflows also demonstrated the greatest activation indexed by event-related high gamma augmentation. This correspondence was not trivial because the methods to obtain these results are based on very different assumptions. ERC is a multichannel method, emphasizing causal interactions within an investigated system of sites potentially involved in task processing. In contrast, time-frequency analysis with matching pursuit treats each site separately, revealing any functional activation at a particular site, regardless of its relationship to activation at other sites. Across the different language tasks performed by our subject, both methods consistently pointed to the same sites as crucially involved in word production. Moreover, the temporal dynamics of functional activation and functional interactions were consistent with each other. For example, the site in vPM and likely in Broca’s area (F31) was more active and was the source of greater ERC outflows during word retrieval than during word production (Fig. 3 top-left and Fig. 9 top-left, Fig. 3 bottom-left and Fig. 10 left, Fig. 7 left and middle, and Fig. 11 left and middle). Likewise, sites in auditory association cortex (F45 and F55) were more active and were sources of greater ERC outflows during early perceptual processing of acoustic stimuli (Fig. 7 left and Fig. 11 left). A site in motor cortex likely responsible for articulation (F40) demonstrated a similar correspondence during spoken responses (Fig. 3 top-right and Fig. 9 top-right, Fig. 7 right and Fig. 11 right). In contrast, P7 near hand-related motor cortex was active and was the source of prominent ERC outflows during both word retrieval (Fig. 3 bottom-left and Fig. 10 left) and during spoken responses (Fig. 3 bottom-right and Fig. 10 right). Here again, ERC and MP were in agreement.

Although quite preliminary, our finding that prominent high gamma augmentation coincides with a divergent pattern of ERC outflows, suggests that cortical sites with high gamma augmentation are more likely to have a causal influence on other sites. Figure 12 illustrates the relationship between these two measures on a log-log scale. Although all the depicted data points in this figure represent statistically significant changes in both measures, the relationship between them is not a simple one. For event-related power changes of lower magnitude, there is no clear relationship with the magnitude of ERC outflows, but for power changes of higher magnitude (e.g. for logarithm of energy change higher than 0.09, vertical line in Fig. 12), there is a positive correlation. The neurophysiological mechanisms responsible for high gamma activity during functional activation are not well understood, but recent studies in animals (Belitski et al., 2008; Liu and Newsome, 2006; Manning et al., 2009; Ray et al., 2008a) have shown that the timing and magnitude of high gamma responses are highly correlated with increases in overall cortical population firing rates. Although it is conceivable that one cortical site can have a causal influence on another site without an increase in its overall firing rate, i.e. through changes in exact spike timing (Fries, 2005), such causal influences may also be facilitated by an increase in the firing rates of cortical projection neurons, in turn resulting in an increase in high gamma power.

4.6 Relationship between causal influences and functional interference

The aforementioned comparison of ERC outflows and high gamma responses (MP) suggested a prominent role in language processing for several sites, of which some were tested with ESM during language tasks. At P7, ESM specifically interfered with signed naming, but not with spoken naming, and both MP and ERC suggested a prominent role for this site in signed naming (Fig. 10). ESM at F45 and F55 in STG interfered with both spoken naming and with auditory sentence comprehension (modified Token Test), but not with signed naming (auditory word repetition was not tested with ESM). MP and ERC analyses indicated much greater involvement of these sites during auditory word repetition than during spoken naming, consistent with the critical role of STG in perceptual processing of spoken speech stimuli. One potential explanation for ESM interference with spoken naming at these sites is that the interference occurred during access to lexical-phonological representations during word retrieval or during monitoring of auditory feedback from spoken responses. Site F40 was tested only for motor function because involuntary motor responses to ESM interfered with language testing. Nevertheless, both MP and ERC results at this site suggested a role for this site in the articulation of spoken responses.

In addition to the sites where there was strong agreement between ERC, MP, and ESM results, there were several sites where these results were inconsistent. For example, F62 in MTG was the recipient of many prominent ERC inflows. However, ESM at this site did not interfere with the language tasks, and except for a small augmentation of high gamma power during word repetition, MP analyses indicated no significant task-related activation at this site. This suggests that although sites with numerous and/or prominent ERC outflows may correspond to sites that have prominent ECoG high gamma augmentation and functional interference with ESM, sites with prominent ERC inflows may not. One may speculate that at a site with many ERC outflows, ESM is more likely to interfere with performance of the same task because of the site’s effective connectivity with other sites on which task performance depends. Unfortunately, the small number of sites tested with all three methods did not allow a definitive analysis of the relationship between ERC, MP, and ESM results. For example, although both MP and ERC indicated prominent functional involvement of sites F31 and B10, these sites were not tested with ESM because they were not a priority for surgical planning. Nevertheless, our results do raise intriguing questions and suggest that maps of effective connectivity resulting from ERC analyses not only provide information about the structure of large-scale cortical networks supporting complex cognitive functions, but may also help identify nodes in these networks that are critical to function.

4.7 Potential clinical utility

All functional mapping techniques that rely on passive measurement of functional activation, including fMRI, EEG, MEG, and ECoG, suffer from the same fundamental limitation: It is difficult to discriminate between activation at sites that are participating in a task, i.e. along for the ride, vs. critical to task performance, i.e. without which task performance would be impaired. An important motivation, therefore, for studying effective connectivity, is to better define the functional importance of individual sites to overall task performance.

To map human brain function with passive ECoG, a variety of electrophysiological indices of local cortical activation have been used, including event-related potentials (Sahin et al., 2009) and event-related spectral changes in different frequency bands (Crone et al., 2006; Jerbi et al., 2009). Of the latter, power augmentation in high gamma frequencies has been particularly promising, but its clinical utility is still under investigation. When ECoG mapping has been compared with ESM, the best results have been obtained in motor cortex and in sensory association cortex, where functional specialization is relatively predictable and densely organized (Brunner et al., 2009; Crone et al., 1998; Leuthardt et al., 2007; Miller et al., 2007). However, the results have been more mixed for mapping the functional-anatomy of language (Brown et al., 2008; Crone et al., 2001b; Edwards et al., 2010; Sinai et al., 2005; Sinai et al., 2009; Tanji et al., 2005; Towle et al., 2008). Because many common language tasks such as picture naming likely require the cooperative activity of large-scale networks of cortical sites, it may be difficult to determine the functional role and thus the relative importance of any single cortical site based on its functional activation. This is where information about task-specific effective connectivity could potentially make the greatest contribution.

The current study found substantial agreement between local cortical activation (MP), effective connectivity (ERC), and ESM. Moreover, the agreement between ERC and MP results was observed using independent methods of analysis. Additional studies will be needed to verify our findings in more subjects and to compare ERC and MP not only with ESM, but also with the postoperative functional outcomes of patients with and without resection of sites with prominent effective connectivity. Nevertheless, the results of this in-depth case analysis suggest that ERC-based maps of effective connectivity may provide a different and valuable perspective on cortical function that complements ECoG measures of local cortical activation.

4.8 Limitations of this study

We cannot exclude the possibility that our results were affected by the patient’s history of medically refractory seizures. However, this concern was part of the motivation for the event-related design of our ERC analyses, which emphasized task-related increases in causal interactions. Causal interactions due to epilepsy are not expected to occur with a consistent temporal correspondence to trials of the word production task, i.e. to consistently occur as an event-related change from baseline. Interestingly, one of the two sites with maximal ictal activity was included in both MP and ERC analyses but did not have significant high gamma augmentation or prominent language-related ERC flows. In addition, the spatial-temporal patterns of event-related gamma in our subject were qualitatively similar to the patterns observed in other subjects performing the same or similar tasks (Brown et al., 2008; Edwards et al., 2010; Korzeniewska et al., 2008). Finally, it is important to appreciate that if our patient did not have intractable epilepsy, this study would not have been possible. Indeed, most studies benefiting from the superior signal quality afforded by intracranial EEG share the aforementioned limitations.

The coverage of cortical areas activated during our language tasks was not comprehensive, and at the time this subject was tested, only 64 channels could be recorded at a time. This required us to repeat the same experiments using two different montages, albeit overlapping in some respects. Intracranial EEG studies almost always suffer from inadequate sampling of all brain regions of potential interest, and the present study is no exception.

5. Conclusions

The effective connectivity of large-scale cortical networks potentially provides a unique perspective on the role of individual cortical sites in overall task performance. This perspective may be particularly valuable for mapping the cortical networks responsible for complex human cortical functions such as language. Here we demonstrate the use of new methods of analyzing event-related changes in Granger causality in passive ECoG recordings. We used these methods to measure the timing, magnitude, and directionality of task-related causal interactions, as well as whether any given site had an overall divergent (many outgoing interactions), or convergent (many incoming interactions), pattern of causal interactions with other sites. Accurately imaging the temporal-spatial dynamics of causal cortical influences may facilitate a growing paradigm shift from models of language function based on the activation of local processing modules to models based on dynamically engaged cortical networks.

Glossary

ASL

American signed language

BA

Brodmann’s area

BTO

basal temporal-occipital cortex

DTF

directed transfer function

ECoG

electrocorticography

ERC

event-related causality

ESM

electrocortical stimulation mapping

LIFG

left inferior frontal gyrus

MP

matching pursuit

MTG

middle temporal gyrus

MVAR

multivariate autoregressive model

SdDTF

short-time direct directed transfer function

STG

superior temporal gyrus

APPENDIX

As described in the Methods section, SdDTF estimates the intensity and direction of causal interactions. Thinking of causality, we usually assume a time delay between related events. This often raises a question: does SdDTF always find an interaction between any two regions that are activated in a task, but with some latency difference in their activation profile? The answer is no. The activation of the two regions must not only be offset in time, but also related in a Granger causality sense.

SdDTF is related to the Granger causality formulation (Granger, 1969), which says that for two signals (time series):

$$x_1,x_2,\dots ,x_n,\dots \text{and}{y}_1,y_2,\dots ,y_n,\dots$$

the observed time series y(t) causes series x(t), if knowledge of y(t)'s past significantly improves prediction of x(t). If we predict values of x(t) using only a combination of its own past values:

$$x_n=a_1{x}_{n-1}+\dots +a_m{x}_{n-m}+\epsilon$$

the noise component ε represents the error of this prediction. If we predict values of x(t) using also values of the past of y(t):

$$x_n=b_1{x}_{n-1}+\dots +b_k{x}_{n-k}+c_1{y}_{n-1}+\dots +c_k{y}_{n-k}+\eta$$

the noise component η is a new prediction error. If the variance of η is smaller than the variance of ε,the second prediction will be a better description of signal x(t), and it can be said that series x(t) is caused by series y(t) in a Granger sense. Another important thing to understand is the nature of the “noise component”. Perhaps because EEG and ECoG signals have well known oscillatory features, and we often think of their task-relevant components as pure sine waves (sinusoids) with some phase shift between brain regions, and we assume that all other components of the signal is noise. However, an oscillation is a repetitive variation, and rhythmic brain activity is better explained as a stochastic time series with short time correlations (as used in MVAR models).

Consider the situation of “noise” flowing from one region to another region. If we record signals from both regions, both of the recorded signals will contain components of the same noise, and the signal recorded in the second region can be described by the past of the noise present in first region. Thus, it is important to remember that non-sinusoidal, or even non-oscillatory, components of recorded signals are not the same as the components ε and η used above, or the component e(t) of equation 1, called noise components in the sense of Granger causality or MVAR model. Components ε, η and e(t) are prediction errors, responsible only for the part of signal which cannot be explained by its own past, or past of other signals included in MVAR model.

Coming back to the question: does ERC always find an interaction between any two regions that are activated in a task, but with some latency difference in their activation profile, we can say that an interaction will be detected only if the “activation profile” of one region is well explained, or predicted, by the activation profile of the second region. The latency difference in their activation profiles may be only a component of this prediction.

The above discussion can be illustrated by a simple simulation of signals in three different channels changing over time. A schematic of the simulation is presented in the top of Fig. A.1, and the results are shown below; cross-correlation in the middle panel, SdDTF in the bottom panel. The signals were simulated in such a way that there were flows from channel 1 to 2 (1→2), and from 2 to 3 (2→3), during the first second. During the next second there were flows from 1 to 2 (1→2), and from 1 to 3 (1→3), however with different latencies, and during the last second flows 1→3 and 2→3 were simulated, also with different latencies.

Fig. A.1.

Cross-spectra and SdDTFs of three-channel MVAR model of simulated signals. Top pannel - schematic of simulated model of activity flows. Horizontal axis represents time. Numbers within circles represent channels (recording sites). Arrows represent direct flows only. Delays between channels are placed near adequate arrow. Middle panel - cross-spectra for pairs of channels (one of them is named above and the other is named to the left of the time-frequency plot). Bottom panel - activity flows measured by SdDTF. Each plot shows SdDTF for direct flows from the channel named above the plot to the channel named to the left of the plot. In each plot the horizontal axis represents time in seconds (relative to the beginning of the shifted window), the vertical axis represents frequency, and the color scale (at right of each set of plots) represents the value of calculated functions from minimum (blue) to maximum (red).

All signals were created to have a combination of deterministic (sinusoidal) phase-locked components, and a “noisy” component. These signals were created as follows. During the first second, when flows 1→2 and 2→3 were simulated, channel 1 contained 3 sinusoids of frequency between 86 and 96 Hz, as well as white noise. Channel 2 contained the signal used in channel 1, albeit shifted later by 3 samples, and white noise of different mean and variance was added to it. Channel 3 contained the signal used in channel 2, albeit shifted later by 1 sample, with additional white noise of different mean and variance than was added to all of the previously used signals.

During the next second, when flows 1→2 and 1→3 were simulated, channel 1 contained the same signal of 3 sinusoids used during the first epoch, and white noise of different mean and variance was added to it. Channel 2 contained the signal used in channel 1, albeit shifted later by 1 sample, and white noise of different mean and variance was added to it. Channel 3 contained the signal used in channel 1, albeit shifted by 2 samples, with additional white noise of different mean and variance than was added to all of the previously used signals.

During the last second, when flows 1→3 and 2→3 were simulated, channel 1 contained the same signal of 3 sinusoids used during the first second, and white noise of different mean and variance was added to it. Channel 2 contained the same signal of 3 sinusoids albeit shifted later by 3 samples, and white noise of different mean and variance was added to it. Channel 3 contained both signals used in channel 1 and channel 2, shifted by 5 samples, with additional white noise.

In the cross-spectra shown in Fig. A.1 (middle panel), there is activity around 90 Hz during all three seconds for each pair of channels. SdDTF plots (Fig. A.1, bottom panel) show only direct flows 1→2 and 2→3 during the first second epoch, only direct flows 1→2 and 1→3 during the next epoch, and only direct flows 1→3 and 2→3 during the last epoch (thin patches between epochs are edge effects from concatenated signals).

Although, the simulation was designed to have latency differences for all pairs of channels, SdDTF did not show interactions for all of them. During the first epoch, the latency shift between the deterministic (sinusoidal) part of the signals in channels 1 and 2 was 3 samples, between channels 2 and 3, by 1 sample, and between channels 1 and 3, by 4 samples. However, the latency between channels 1 and 3 did not produce a direct activity flow, as measured by SdDTF, because the signal in channel 3 was better explained by the past of the signal in channel 2, than by the past of the signal in channel 1.

During the next epoch, the latency shift between the deterministic (sinusoidal) components of signals in channels 1 and 2 was 1 sample. In channels 1 and 3 it was 2 samples, and in channels 2 and 3 it was 1 sample. Also in this case, a latency shift between channels 2 and 3 was not a sufficient condition for it to be detected as a direct activity flow.

Similarly, during the third epoch, the latency of 3 samples between channels 1 and 3 did not cause it to be detected as an activity flow.

The effects cannot be explained by, for instance, selective sensitivity for noisy components because the flows depicted in Fig. A.1 are of frequency around 90 Hz. Although weak flows were detected in other frequencies, these were likely caused by the “noisy” components of signals, which were also flowing between channels.

The simulation may also help to understand that in a case when oscillating signals of the same frequency, are present in many regions, with different latencies, they are not necessarily detected as causal interactions between all of the regions.