Modulation of Orthographic Decoding by Frontal Cortex

Abstract

Opinions are divided on whether word reading processes occur in a hierarchical, feedforward fashion or within an interactive framework. To critically evaluate these competing theories, we recorded electrocorticographic (ECoG) data from 15 human patients with intractable epilepsy during a word completion task and evaluated brain network dynamics across individuals. We used a novel technique of analyzing multihuman ECoG recordings to identify cortical regions most relevant to processing lexical information. The mid fusiform gyrus showed the strongest, earliest response after stimulus onset, whereas activity was maximal in frontal, dorsal lateral prefrontal, and sensorimotor regions toward articulation onset. To evaluate interregional functional connectivity, ECoG data from electrodes situated over specific cortical regions of interest were fit into linear multivariate autoregressive (MVAR) models. Spectral characteristics of the MVAR models were used to precisely reveal the timing and the magnitude of information flow between localized brain regions. This is the first application of MVAR for developing a comprehensive account of interregional interactions from a word reading ECoG dataset. Our comprehensive findings revealed both top-down and bottom-up influences between higher-level language areas and the mid fusiform gyrus. Our findings thus challenge strictly hierarchical, feedforward views of word reading and suggest that orthographic processes are modulated by prefrontal and sensorimotor regions via an interactive framework.

SIGNIFICANCE STATEMENT Word reading is a critical part of everyday life. When the ability to read is disrupted, it can lead to learning disorders, as well as emotional and academic difficulties. The neural mechanisms underlying word reading are not well understood due to limitations in the spatial and temporal specificity of prior word reading studies. Our research analyzed data recorded from sensors implanted directly from surface of human brains while these individuals performed a word reading task. Our research analyzed these recordings to infer how brain regions communicate during word reading. Our original results improve upon current models of word reading and can be used to develop treatment plans for individuals with reading disabilities.

Introduction

Reading is a crucial part of the establishment of modern human life. Yet, the mechanisms by which the human brain decodes written information are poorly understood. Although it is clear from functional imaging and lesional analyses that regions situated in the left hemisphere (Fiez and Petersen, 1998; Price, 2012) function together to transform orthographic information into spoken output, the spatiotemporal dynamics of how these regions interact with each other is a matter of active debate (Carreiras et al., 2014).

Opinions remain divided on whether word reading is achieved through feedforward or interactive processes (Price and Devlin, 2011; Carreiras et al., 2014). Feedforward theorists (Coltheart and Curtis, 1993; Jobard et al., 2003; Levy et al., 2009) assert that single word reading is achieved through sequential stages: that orthographic processing precedes phonological and semantic processing. These arguments are largely derived from behavioral effects observed during tasks involving lexical and prelexical interactions (e.g., the word superiority effect). As feedback connections are deemed unnecessary (and later lexical processes proposed to not “benefit” from interacting with orthographic stages), the hierarchical framework has been deemed the most plausible explanation (Norris et al., 2000). Connectionists (Seidenberg, 2005) posit an alternate, interactive framework in which word reading is achieved via a multilayered network with bidirectional connections between orthographic, phonological, and semantic nodes.

Both theoretical frameworks have gained empirical support (Jobard et al., 2003; Levy et al., 2009; Sahin et al., 2009; Carreiras et al., 2014; Schurz et al., 2014; Woodhead et al., 2014; Klein et al., 2015). fMRI investigations in favor of feedforward processing have localized anatomical regions associated with the proposed sequential stages and strongly suggest the mid fusiform gyrus (mFuG) is strictly involved in prelexical processes (Jobard et al., 2003). However, the only ways of assessing information flow for any language process (most of which occur in the 10 s of milliseconds time range) is by using MEG, hd-EEG, and intracranial EEG data (for MEG limitations, see Jerbi et al., 2009; for scalp EEG restrictions, see Lachaux et al., 2012). The need for a comprehensive account of word reading that includes empirical estimates of information flow has motivated this effort (Price, 2012).

We collected electrocorticographic (ECoG) data in 15 humans during a word-stem completion task. The exceptional spatiotemporal resolution of human ECoG recordings (Jacobs and Kahana, 2010; Lachaux et al., 2012; Kadipasaoglu et al., 2015) allows for studies of cognitive function in single patients (Crone et al., 2001; Swann et al., 2012) as well as in groups (Mainy et al., 2008; Edwards et al., 2010; Conner et al., 2014; Kadipasaoglu et al., 2014). Recent methodological developments have extended the ability of ECoG analyses to quantify the direction and magnitude of interregional interactions (Korzeniewska et al., 2008, 2011; Vidal et al., 2012). One such technique, the Short-time direct Directed Transfer Function (SdDTF) (Korzeniewska et al., 2008, 2011; Flinker et al., 2015), has been recently applied to the analyses of single and multipatient data and is especially well suited to investigate processes engaged by widespread networks crucial for word reading.

We present the first ever application of SdDTF to multihuman ECoG word reading data to quantify the amplitude and timing of directed information flow. Such a data-driven approach should allow us to: (1) delineate the time course of cortical interactions; and (2) evaluate how such interactions agree with current word reading frameworks. Connectivity analyses will evaluate for feedforward and bidirectional information flow, with a focus on information originating in the mFuG, a region well known to be crucial to reading words and word-like stimuli (Dehaene et al., 2002; Dehaene and Cohen, 2011). The existence of predominantly bottom-up communication originating in mFuG would support evidence (Jobard et al., 2003; Levy et al., 2009) for feedforward models of word reading. In contrast, the finding of bidirectional exchanges between orthographic, phonological, and semantic components of the network would align with the interactive frameworks (Price and Devlin, 2011; Carreiras et al., 2014).

Materials and Methods

Fifteen patients with intractable epilepsy (5 male; average 32 ± 10 years; average IQ of 97 ± 11; 1 left-handed person, proven to be left hemisphere dominant for language by intracarotid amytal testing) scheduled to undergo surgical implantation of subdural electrodes for seizure localization were enrolled in the study after informed consent and study design approval by the University of Texas Health Science Center's committee for protection of human subjects.

Imaging data collection and analysis.

Before the implantation surgery, a 3T whole-body MR scanner (Philips Medical Systems) furnished with a 16-electrode SENSE head coil was used to collect the anatomical MRI scans with magnetization-prepared 180 degree radio-frequency pulses and rapid gradient-echo (MP-RAGE) sequence. The scanner was optimized for gray-white matter contrast with 1-mm-thick sagittal slices and an in-plane resolution of 0.938 × 0.938 mm (Ellmore et al., 2009). Following scans, cortical surface representations were reconstructed with FreeSurfer software (version 5.1) (Dale et al., 1999) and then imported to SUMA (Saad and Reynolds, 2012) for our analysis.

Behavioral data collection.

Several days after electrode implantation, we collected ECoG data during a word-stem completion task while these 15 patients viewed 80–100 partial words (stems) (e.g., “‘ru_”) and were instructed to say aloud, as quickly as possible, the first action word that came to mind (e.g., “running”) (Table 1). Stimuli were presented on a 15-inch LCD monitor positioned 2 feet away, at eye level and consisted of black, lowercase text (font height of 100 pixels, Calibri font type) centered on a 1300 × 800 pixel white background. Time from stimulus offset to the next stimulus onset was randomly varied between 1.5 and 8.5 s. Transistor-transistor logic pulses initialized by the task presentation software (Python version 2.7 software was used for Patients 11, 12, 13, and 15; Presentation software, version 11, Neurobehavioral Systems, was used for the others) were used to mark stimulus onset times on concurrent ECoG traces.

Table 1.

Stimulus list^a

avoi	bak	ben	blen	boi	brea	bru	brus	bu	buil
cal	catc	chas	che	cho	clea	clim	cli	coun	craw
danc	div	dra	drea	dres	drin	driv	dum	ea	eras
fal	fee	fl	giv	gro	hea	hel	hol	hun	jum
kee	kis	knee	knoc	lau	lea	lear	lis	loo	lov
mov	pai	pla	pou	pra	prac	punc	pus	qui	rea
reac	rid	ru	se	shak	sho	si	slee	smil	smok
spea	stan	stea	sto	stretc	swi	tak	thro	tou	tur
visi	wast	wav	wea	wor	wri

^aWord-stem stimuli were randomly drawn from this list each time the task was administered.

Audio traces from each ECoG recording session were reviewed to exclude trials with false starts and incorrect responses (Table 2). Additionally, trials containing interictal activity or 60 Hz noise were excluded from the analysis. Initial estimates for the articulation onsets in the remaining trials were then determined using in-house software (written for MATLAB 2013a, The MathWorks) to identify the first time point following stimulus onset in every trial at which the baseline amplitude was exceeded by 50% in each patient's audio trace. Audio recordings were then systematically reviewed to precisely mark articulation onset times, which were then used to derive the reaction time (RT) for each trial.

Table 2.

Patient behavioral and task performance data^a

Patient no.	Age (yr)	IQ	Total trials	Incorrect/noisy/false-start trials	RT <600 ms or >1800 ms	Remaining trials	RT, mean (ms)	RT, SD (ms)	Recording frequency (Hz)
1	37	89	80	33	2	45	979	283	1000
2	21	97	80	27	3	50	949	231	1000
3	39	100	80	35	8	37	1102	242	1000
4	38	96	80	34	2	44	1138	205	1000
5	18	67	80	45	4	31	1032	211	1000
6	30	100	80	23	26	31	1432	200	1000
7	20	97	80	19	3	58	997	248	1000
8	42	107	80	32	18	30	1431	184	1000
9	28	97	80	31	3	46	1066	254	1000
10	51	92	80	39	13	28	1394	200	2000
11	24	105	100	33	2	65	936	245	2000
12	21	101	100	21	36	43	790	154	2000
13	34	97	100	37	11	52	1288	272	2000
14	45	93	80	27	4	49	1275	272	1000
15	31	124	100	31	1	68	956	219	2000
Total	—		1280	467	136	677	1088	295	—

^aFrom left to right: patient number, age at the time of surgery, postsurgery IQ score, incorrect and noisy trials, number of trials removed for RT, remaining trails, mean RT, SD RT, and frequency at which the intracranial recordings were sampled.

It is important to note that a natural occurrence in human subject data is the existence of variability both within and across individuals in RT. To minimize the effect of this confound, we first eliminated all trials with RTs <600 ms or >1800 ms. We then performed all ECoG analyses twice, aligning data in two different ways: (1) aligning trials to stimulus onset, and then (2) aligning trials to articulation onset. In this fashion, we could ensure that neural processes engaged during both stages of analysis (i.e., following stimulus onset and leading up to articulation) were more likely to be similar across both epochs and individuals.

ECoG data collection and initial processing.

Fifteen patients, with a total of 1684 subdural electrodes (PMT; top-hat design; 3-mm-diameter contact with cortex), were implanted following methods previously published by Tandon (2008) (Fig. 1A). ECoG data were collected at 1000 Hz using NeuroFax software (Nihon Kohden) with a bandwidth of 0.15–300 Hz or at 2000 Hz using NeuroPort NSP (Blackrock Microsystems) with a bandwidth of 0.1–750 Hz (Table 2). Data from every electrode were rereferenced to a common average chosen by excluding those with 60 Hz noise or interictal activity when initially referenced to an artificial 0 V (Conner et al., 2011; Swann et al., 2012; Kadipasaoglu et al., 2014). Electrode localization on cortical surfaces was performed using previously published standard techniques (Pieters et al., 2013).

Figure 1.

Group coverage maps. A, A total of 1684 subdural electrodes were implanted in 15 patients. Electrodes are represented on a standard N27 surface using surface-based transformation to optimize the fit to cortical topology. Electrodes in black (539 total) were excluded because of excessive noise or proximity to epileptic tissue. B, The surface recording zones for the remaining 1145 electrodes, thresholded at a minimum value of 3 patients, are shown (for more details, see Kadipasaoglu et al., 2014).

Subsequent analyses were performed using broadband gamma activity (BGA, 60–120 Hz) following bandpass filtering (IIR Elliptical Filter, 30 dB sidelobe attenuation) of individual ECoG recordings and spectral decomposition (Conner et al., 2014; Kadipasaoglu et al., 2014). BGA was selected because this frequency range provides a robust task-specific indicator of human cognitive function (Crone et al., 2001; Mainy et al., 2008; Edwards et al., 2010; Lachaux et al., 2012; Conner et al., 2014; Kadipasaoglu et al., 2014).

Surface-Based Mixed-Effects Multilevel Analysis (SB-MEMA).

To provide statistically robust and topologically precise estimates of BGA, surface-based representations of BGA at the individual and population-level were created using an SB-MEMA (Kadipasaoglu et al., 2014). SB-MEMA was developed to overcome challenges in grouped ECoG analyses due to the following: (1) difficulties in the accurate localization and spatial transformation of ECoG (Miller et al., 2007; Conner et al., 2014; Esposito et al., 2013); (2) integration of data across subjects while accounting for the sparse sampling typical of ECoG coverage of the cortex and intersubject anatomical variability (Lachaux et al., 2003; Anticevic et al., 2008; Jerbi et al., 2009; Oosterhof et al., 2011; Saad and Reynolds, 2012); and (3) statistical analyses of grouped datasets, arising from sparse sampling, outlier inferences, as well as intrasubject and intersubject variability that led to violations in the assumptions of traditional statistical models (e.g., ANOVA, t tests) (Chen et al., 2012; Conner et al., 2014; Kadipasaoglu et al., 2014). By integrating spatially transformed BGA with surface-based coregistration and a mixed-effects multilevel group analysis (Kadipasaoglu et al., 2014), SB-MEMA enables the generation of multihuman brain activity maps of cognitive functions, such as reading.

After bandpass filtering raw ECoG data from all 1684 electrodes (Fig. 1A,B) into the BGA frequency band, the data were analyzed using SB-MEMA and integrated across two poststimulus epochs: 50–600 ms after stimulus onset (for stimulus-aligned trials), and −600 ms to articulation onset (for articulation aligned trials). Within each epoch, ROI were identified using cortical areas recorded from at least 5 patients and demonstrated at least 5% power increase from prestimulus baseline BGA (p = 0.05, uncorrected). From this initial selection, only those ROI showing positive changes from baseline and containing >300 continuous nodes (to correct for multiple comparisons) were used in further analysis (Kadipasaoglu et al., 2014).

Time series analysis of BGA.

Time series traces were computed using BGA from electrodes localized over ROI identified from SB-MEMA to identify event-related regional dynamics. To compute the traces, ECoG data were Hilbert transformed and the analytic amplitudes were subsequently smoothed to obtain BGA power (Savitzsky-Golay FIR, fifth order, frame length of 155 samples, MATLAB 2013b, The MathWorks). Percentage change in BGA power was computed for each subdural electrode, with respect to baseline (−700 to −200 ms prestimulus window) and then averaged over trials. To obtain population-level time-series representations, percentage change in BGA was then averaged across all electrodes in all individuals contributing to each ROI (p ≤ 0.01 Wilcoxon sign rank test corrected for multiple comparisons with false discovery rate [FDR]). Similar to SB-MEMA, the BGA traces were analyzed across two poststimulus aligned to stimulus and articulation onset.

Connectivity analysis.

Individual and grouped estimates of directed information flow were derived using task-relevant BGA from electrodes in each ROI as follows. Data were aligned to stimulus and articulation onset and divided into overlapping windows (length 100 ms, shift 50 ms). Windows covered the same poststimulus epochs used for SB-MEMA. Trial averages and SDs were computed for each electrode (at each time point, the average signal across all trials was subtracted from each time series; the result was divided by the SD across all trials at each time point). Next, time averages and SDs over each window were computed (the data within each window then had its corresponding average subtracted and was divided by its SD) (Ding et al., 2000). Data from the i^th electrode were then expressed as xⁱ ∈ ℝ^n_s×n_t, where n_s denotes the number of data points and n_t is the number of trials. The number of electrodes for a single patient across all ROI was denoted n_e. For each patient, at fixed time j and trial k, these electrodes were compiled as X_j,k = {x_j,k¹, x_j,k², …, x_j,k^n_e} ∈ ℝ^n_e. Patient-specific multivariate time series were fit to a predictive multivariate autoregressive (MVAR) model on each window via the following:

The model order m, gives the number of time points used to forecast the time series at time j and was selected as the minimizer of the Bayesian Information Criterion (Ding et al., 2006). The unknown convolution matrices, A_i^(m) ∈ ℝ^n_e×n_e, i = 1, …, m, and covariance of the model fit error term, E_j,k^(m) ∈ ℝ^n_e, were then computed (Morf et al., 1978).

Causal influences were determined using SdDTF to describe how relationships between ROI change over time (Korzeniewska et al., 2008). SdDTF estimates of the magnitude of direct influence from time series j and i at frequency f_w within time window τ_k take values along the interval 0 ≤ SdDTF_k,w(i, j) ≤ 1. In this fashion, we quantified the timing, frequency, direction, and magnitude of information flow between brain ROI for each patient. It is important to note that SdDTF is a causal measure, which is mathematically distinct from what is measured by the BGA. That is, SdDTF_k,w(i, j) ≥ 0 implies an increase in the ability of signal j to predict signal i, which can occur without inducing large changes in BGA.

SdDTF values were computed for pairwise interactions between all electrodes in each ROI for each patient and integrated over the BGA frequency band. The resulting frequency-integrated, interregional averaged SdDTF values between ROIs across all patients will be referred to as “information flow” (IF; → indicates unidirectional IFs; ↔ indicates bidirectional IFs). Event-related significance was evaluated with a Wilcoxon signed rank test performed on the percentage change of all poststimulus IFs versus baseline IFs (−700 to −450 ms prestimulus window) (p ≤ 0.01, FDR-corrected).

We note here that a different baseline window was selected for the connectivity analysis to address stationarity assumptions of the MVAR models (Ding et al., 2000). It is important to note that, although SdDTF values from all 15 patients were included in grouped IFs, the presence of an IF between any two regions can only be derived from patient(s) with electrodes recording from both of those regions simultaneously.

Clustering analysis.

A clustering analysis was performed to objectively characterize temporal patterns of grouped IFs to derive network-wide transitions. For stimulus/articulation-aligned trials, IFs were evaluated using k-means clustering, where the squared Euclidean norm was selected as the distance metric, and silhouette values were used to measure fit of each cluster size (evalclusters function, MATLAB 2013b, The MathWorks). Once the number of optimal clusters was determined (via the size that minimized average silhouette value), the time windows corresponding to each cluster were identified.

Results

The average RT of all patients across all included trials was 1088 ± 295 ms (Table 2). We note that this falls well within the range of reported response times of healthy individuals (mean RT = 1311 ms) recorded during a comparable task (Badgaiyan and Posner, 1997). The SB-MEMA, BGA time series, and SdDTF connectivity results from a single representative patient (Patient 2) and then from the entire group are presented below under the associated subheadings.

Representative single patient

In the individual surface-based ECoG representation (Fig. 2B), distinct activation patterns were observed across left hemispheric brain regions (percentage changes from baseline, −700 to −200 ms prestimulus window). Seven electrodes showed especially prominent task-related changes (Fig. 2A). These electrodes were located in distinctly separate anatomical locations: mFuG, pars opercularis (pOp), pars triangularis (pTr), middle frontal gyrus (MFG), subcentral gyrus (sCG), postcentral gyrus (PoCG), and precentral gyrus (PrCG).

Figure 2.

Single-patient example for trials aligned to stimulus onset. A, Spectrograms show the full spectrum (0–200 Hz, linear frequency axis) of percentage change of power during the 0–1000 ms poststimulus period versus baseline for seven representative electrodes. Horizontal black bars represent 60 and 120 Hz. Vertical bars represent 50 and 600 ms. B, Upon averaging across the broadband gamma frequencies (encompassed within the horizontal black lines), and from 50 to 600 ms (enclosed in the vertical black lines), individual surface-based representations are obtained. The views are of the left hemisphere, with the lateral surface at top and ventral surface at bottom. Spectrograms (A) and surface representation (B) are on a scale from −100% to 100%. C, BGA time traces of the percentage changes from prestimulus baseline are shown with each region as an individual trace. D, Percentage changes of information flows (IFs; vs baseline) were determined for the analysis windows. Horizontal axis labels are the first point of each 100 ms window. The plot in the i^th row, j^th column represents IFs from the j^th region to the i^th region. No statistical corrections were performed for this illustrative example.

BGA time series (Fig. 2C) highlight activity that occurred in distinct, overlapping intervals. The earliest response was seen in mFuG, which showed a 350% change from prestimulus baseline ∼180 ms, a signal indicative of prelexical visual processing (Dehaene et al., 2002). At ∼180 ms, the profiles of MFG and pOp also became positive, suggesting the engagement of short-term memory and phonological processes (Paulesu et al., 1993; Bokde et al., 2001; Barbey et al., 2013). pTr was the last subregion to become engaged (∼360 ms), and activity here peaked over 100% ∼660 ms. Power in PrCG, sCG, and PoCG increased throughout the majority of the epoch, also peaking ∼660 ms, at ∼150%, 175%, and 275%, respectively.

Connectivity estimates for this single individual, derived from the frequency integrated SdDTF values (Fig. 2D), illuminated how and when regions communicated with each other throughout the analysis windows. As expected given its early, notable changes in BGA power, mFuG was involved in numerous significant interactions in the early interval from ∼0 to 250 ms. These included individual IFs each of varying amplitude: pTr → mFuG (∼200 ms), PoCG → mFuG (∼200 ms), MFG → mFuG (∼250 ms), and pOp → mFuG (∼250 ms). In the reverse direction, mFuG → sCG (∼150 ms) and mFuG → PoCG (∼200 ms) were prominent.

In the later interval beyond 250 ms, prominent IFs included PoCG → MFG (∼250 ms), MFG → pOp (∼350 ms), and pTr → MFG (∼300 ms), and bidirectional interactions between MFG and pOp (∼350 ms). Near 400 ms, several IFs simultaneously peaked: PoCG → mFuG, sCG → mFuG, and an especially prominent pTr → sCG. Additionally, PrCG was involved in multiple interactions with most of the neighboring sensorimotor cortex from ∼250 ms to 400 ms: PoCG → PrCG, PrCG → sCG, and sCG → PrCG. These single patient data are broadly consistent with the grouped results that follow.

SB-MEMA ECoG maps

The synthesis of individual surface-based representations into grouped maps using SB-MEMA revealed distinct and prominent early activations in mFuG and PrCG during the 50–600 ms time interval locked to stimulus onset (Fig. 3A), as might be expected during lexical processing (Nobre et al., 1994; Pammer et al., 2004; Thesen et al., 2012) and prearticulatory activity (Pammer et al., 2004; Wheat et al., 2010). In contrast, SB-MEMA computed time locked to articulation onset (Fig. 3B) showed much greater frontal activity (in pOp, pTr, and MFG) (consistent with Dhond et al., 2001; Levy et al., 2008; Thesen et al., 2012; Bedo et al., 2014). Notably, the strong BGA responses across the group revealed in pTr, pOp (concordant with Jobard et al., 2003; Mechelli et al., 2005; Mainy et al., 2008; Sahin et al., 2009), dorsal lateral prefrontal cortex (consistent with Paulesu et al., 1993; Jobard et al., 2003; and Barbey et al., 2013), and the prearticulatory responses concentrated in PrCG, PoCG, and sCG (Bouchard et al., 2013) are consistent with existent literature.

Figure 3.

Surface-based ECoG representations for stimulus- and articulation-aligned trials. SB-MEMA maps were generated for the grouped BGA across all 15 patients. Initially, areas that showed significant activation (p ≤ 0.05, uncorrected threshold) were included. Of these regions, areas active in at least 5 patients that showed more than a 5% change from baseline (−700 ms to −200 ms prestimulus window) were used. A correction for multiple comparisons was performed to include regions smaller than a fixed threshold; regions that showed positive percentage change were ultimately included. The resulting ROIs are represented on a standard N27 surface for two time periods (lateral surface, top row; ventral surface, bottom row; left hemisphere). A, Time series were aligned to stimulus onset and integrated over 50–600 ms. B, Time series were aligned to articulation onset and integrated over the 600 ms epoch before articulation.

BGA time-frequency analysis results

A total of 366 electrodes were situated over the seven task-relevant, anatomical ROIs noted in the SB-MEMA results (Tables 3, 4; Fig. 4A). Significant (p = 0.01, FDR-corrected), stimulus-aligned responses began in mFuG (Fig. 4B) starting at ∼120 ms and peaking at 100% ∼260 ms (consistent with Nobre et al., 1994; Pammer et al., 2004). PrCG was active next ∼280 ms (Wheat et al., 2010), and activity here increased until ∼580 ms (peaking at ∼58%). From ∼400 ms onward, significant changes from baseline were observed in PoCG (∼400 ms), pOp (∼440 ms), pTr (∼470 ms), sCG (∼490 ms), and MFG (∼580 ms).

Table 3.

Distribution of task-relevant electrodes^a

Region	Patient number															Total
	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15
mFuG	5	6	2	4	0	3	2	4	2	0	0	3	3	5	5	44
pOp	2	9	2	3	3	2	3	5	2	3	4	5	3	5	4	55
pTr	5	4	5	5	4	5	1	5	2	3	3	4	5	3	0	54
PoCG	2	3	3	1	5	4	2	5	4	4	3	5	3	2	3	49
PrCG	5	5	6	2	5	8	6	7	4	4	7	7	7	5	6	84
sCG	2	1	4	5	6	2	3	3	3	4	0	2	1	1	2	39
MFG	2	3	3	2	4	2	2	2	0	4	3	2	6	3	3	41
Total	23	31	25	22	27	26	19	31	17	22	20	28	28	24	23	366

^aA total of 366 electrodes were identified with SB-MEMA as task-relevant and examined further with time-frequency analysis of BGA and connectivity analyses with the SdDTF. Electrodes are organized by patient (columns) and region (row).

Table 4.

Spatial coordinates for regions of interest^a

Region	Mean	SD
mFuG	−25, −42, −19	6, 14, 11
pOp	−56, 13, 23	5, 8, 3
pTr	−55, 23, 14	4, 10, 9
PoCG	−54, −15, 42	3, 8, 6
PrCG	−52, −3, 40	5, 5, 8
sCG	−62, −10, 19	9, 4, 12
MFG	−43, 24, 39	1, 6, 7

^aMean and SD (in mm) of x, y, z coordinates in Talairach space of electrodes located within each of the seven anatomical locations identified through the SB-MEMA.

Figure 4.

Grouped BGA time traces. A, A total of 366 electrodes were situated over ROIs and are represented on the standard N27 surface (lateral view at top, ventral view at bottom; left hemisphere). B, The average ±1 SE of percentage change in BGA from baseline (−700 ms to −200 ms prestimulus window) for stimulus-aligned trials averaged across all electrodes and all subjects are shown, with each ROI as a trace. Horizontal bars at the bottom of represent times where percentage change in power of each ROI significantly differed from 0 (Wilcoxon sign rank, FDR corrected for multiple comparisons, p = 0.01). C, The average ±1 SE of percentage change in BGA from baseline (−700 to −200 ms prestimulus window) for articulation-aligned trials averaged across all electrodes and all subjects, with each ROI as a trace. Horizontal bars at the bottom indicate times where percentage change in power of each ROI significantly differed from 0 (Wilcoxon sign rank, FDR corrected for multiple comparisons, p = 0.01).

Distinct patterns of activity were seen in the articulation-aligned analysis (Fig. 4C). The earliest response was seen in the mFuG (−700 ms to −440 ms, ∼55%, losing significance ∼−180 ms). Power in the MFG increased to a smaller amount (20%–40% range between −700 ms and −100 ms). Activity in frontal regions (pTr and pOp) increased from −700 ms until ∼−240 ms. In sensorimotor cortex (sCG, PoCG, and PrCG), gamma power was observed to increase monotonically, beginning ∼−540 ms and reached 60%–100% at articulation onset, consistent with their participation in the articulatory network (Hickok and Poeppel, 2007).

Connectivity results

The availability of a large multisubject dataset with high spatial and temporal resolution allowed us to progress beyond identifying when and where areas are active to assess interregional interactions. As anticipated based on the SB-MEMA and BGA results, the most prominent interactions revealed by the SdDTF involved the mFuG. In the stimulus onset-aligned analysis, IFs to mFuG from ALL the other cortical regions were very notable (Fig. 5), suggesting top-down modulation of lexical processing. The largest, most significant (p = 0.01, FDR-corrected) IFs to the mFuG originated in MFG, PoCG, and sCG, and these peaked rather early (between 200 and 350 ms). Additionally, prominent concurrent feedforward IFs originating in the mFuG, directed to the PoCG (∼50–600 ms) and the dorsolateral prefrontal cortex, MFG (from ∼100 to 500 ms) were present. The MFG in turn then also showed prominent interactions: MFG → pTr (∼50–200 ms), MFG → sCG (∼50–200 ms and ∼400–600 ms), and MFG → PrCG (from ∼200 ms onward). The presence of these IFs despite the absence of a prominent increase in gamma power in the MFG suggests that these inputs are likely the precursor to the subsequent activation seen in the later time window (Fig. 3B).

Figure 5.

Information flows (IFs) aligned to stimulus onset. Percentage changes of IFs derived from SdDTF compared with baseline during time windows following stimulus onset. Markers in gray represent values that did not achieve significance (Wilcoxon signed rank, p = 0.01, FDR-corrected for multiple comparisons). Markers (not in gray) are shown in the color corresponding to the region receiving the flow (see Fig. 4). Horizontal axis labels are the first point of each 100 ms window. The plot in i^th row, j^th column represents IFs from the j^th region to the i^th region.

Given the large and monotonic rise in power in sensorimotor cortex (sCG, PrCG, and PoCG) leading up to articulation (Fig. 4), we expected to see a rise in articulation-aligned interactions involving these regions. Indeed, these three areas became highly coupled in the time period leading up to articulation onset (Fig. 6). Consistent and significant IFs between prefrontal (PFC) cortex and sensorimotor cortex were seen: MFG → PrCG (∼−550 ms onward), MFG → sCG (∼−450 ms onward), and pTr → sCG (∼−400 ms onward). Pars opercularis sent inputs to sensorimotor cortex as well. PoCG → pOp (∼−600 ms to −200 ms) was a significant IF from sensorimotor cortex to lateral premotor cortex (pOp). Articulation-aligned mFuG interactions were prominent just before articulation and included the bilateral PoCG ↔ mFuG from −600 ms to articulation. Within the sensorimotor region itself, several interactions were noted: PrCG ↔ sCG from −450 ms to −300 ms; PoCG ↔ PrCG from −200 ms to articulation; and PoCG ↔ sCG from −100 ms to articulation.

Figure 6.

Information flows (IFs) aligned to articulation onset. Percentage changes of IFs derived from SdDTF compared with baseline during time windows leading up to articulation onset. Markers in gray represent values that did not achieve significance (Wilcoxon signed rank, p = 0.01, FDR-corrected for multiple comparisons). Markers (not in gray) are in the color corresponding to the region receiving the flow (Fig. 4). Horizontal axis labels are the first point of each 100 ms window (i.e., the −600 ms point refers to flows computed across the −600 ms to −500 ms window before articulation onset). The plot in i^th row, j^th column represents IFs from the j^th region to the i^th region.

Clustering results

Interregional dynamics derived from the IFs were characterized with k-means clustering to allow for unbiased comparison to the BGA time trace patterns. Six overlapping clusters (or states) arose from the network's IFs (Fig. 7). Following stimulus onset (Fig. 7), State A emerged from 50 to 200 ms, State B from 150 to 350 ms, State C from 300 to 450 ms, and State D from 400 to 600 ms. In evaluating the data time locked to articulation (Fig. 8), they occurred during two time periods: State E from −600 ms to −450 ms, and State F from −400 ms to 0 ms. Notably, transitions between clusters appeared to separate epochs of distinguishable patterns in the regional BGA time traces.

Figure 7.

Stimulus-onset aligned clusters. Stimulus-aligned information flows (IFs) were analyzed with k-means clustering to identify network-level transitions. Each cluster is shown in a different shade of gray. Top, Arrows indicate time integrals of IFs for the four cluster timeframes shown. Color and width of arrows are proportional to the magnitude of the normalized integrated information. State A exists from 50–200 ms, State B from 150–350 ms, State C from 300–450 ms, and State D from 400–600 ms. Bottom, BGA time traces are overlaid on top of shaded clusters (periods of overlap are not depicted). The clusters were derived from all IFs values, not just those that achieved significance. This figure should be viewed along with Figure 8, which shows articulation onset aligned clusters. Similar IFs and regional broadband gamma activity can be seen in both State D time-locked to stimulus onset and State E (in Fig. 8) that is time-locked to articulation onset.

Figure 8.

Articulation-aligned clusters. Articulation-aligned information flows (IFs) were analyzed with k-means clustering to identify network-level transitions. Each cluster is shown in a different shade of gray. Top, Arrows indicate time integrals of IFs for the two cluster timeframes shown. Color and width of arrows are proportional to the magnitude of the normalized integrated information. States A–D are shown in Figure 7 and described in the analysis time locked to stimulus onset; State D and E are similar, though derived from distinct concatenations of the time series. State E is from 600–450 ms prior to articulation, and State F from 400 ms to onset of articulation. Bottom, BGA time traces are overlaid on top of shaded clusters (periods of overlap are not depicted). The clusters were derived from all IFs values, not just those that achieved significance.

State A was marked by the increase in BGA power in mFuG and relatively weak connectivity involving this region. State B showed increasing mFuG power, the onset of increasing activity in PrCG, and the strengthening of interregional networks involving mFuG. After mFuG peaked in power ∼300 ms, the network transitioned to a brief (∼150-ms-long) State C. This was marked by decreasing IFs to mFuG, increasing power in PrCG and a gradual decrease in power in mFuG's toward baseline. Prominent IFs to mFuG, sCG, PoCG, and PrCG marked the onset of State D. A continuing decrease in mFuG's power and an increase in power was seen across PrCG, pTr, PoCG, pOp, and sCG during this state.

State E encapsulated the period when mFuG's power continued falling while the regional BGA in PrCG, pTr, pOp, PoCG, and sCG were rising. Incoming IFs (from all other regions to mFuG) and outgoing IFs (mFuG → MFG, PrCG, pOp, and PoCG) were observed during this state. In these ways, similar patterns in both IFs and regional gamma activity can be seen between State D and State E, suggesting that this time window may be similar or overlapping between the two.

As MFG reached its peak power, the network evolved to State F. Throughout this state, mFuG dropped back to baseline, in contrast to the sensorimotor cortex (PrCG, PoCG, and particularly sCG), in whom gamma power peaked at articulation onset. Prominent information flow into sensorimotor cortex (pTr → sCG, MFG → PrCG, pOp → sCG, and PrCG → sCG) drove this increase in local activity.

Discussion

This work addresses a long-standing debate about visual word reading regarding whether or not orthographic information from the mFuG feeds into lexical representations in a predominantly hierarchical fashion or whether higher level regions involved in phonology and articulation exert a top-down influence to modulate early orthographic processing (Carreiras et al., 2014). We critically evaluated competing theories of word reading through precise descriptions of brain dynamics derived from the novel application of grouped ECoG analysis techniques (Price and Devlin, 2003; Carreiras et al., 2014). Our results reveal the timing and magnitude of parallel, bidirectional connectivity between frontal (pTr, pOp, and MFG) and sensorimotor (PrCG, sCG, and PoCG) regions with mFuG during epochs aligned to word stimulus presentation and the onset of the spoken response. Our empirical evidence shows that orthographic processes in the mFuG closely interact with phonological, semantic, and short-term working memory processing regions throughout the word-stem completion task (Paulesu et al., 1993; Jobard et al., 2003; Mechelli et al., 2005; Mainy et al., 2008; Price, 2012; Barbey et al., 2013).

SB-MEMA aligned to stimulus onset and articulation onset allowed us to localize cortical substrates using 366 electrodes situated over specific loci of activation common across the group (Fig. 4A). As expected (Nobre et al., 1994; Dehaene et al., 2002; Mainy et al., 2008; Thesen et al., 2012), the strongest stimulus locked response, a near doubling of gamma power activity, was observed in mFuG soon after stimulus onset (Figs. 3A, 4B); concurrent with mFuG's peak response were lagged activations in all frontal and sensorimotor regions. The greatest articulation-aligned responses were seen in PoCG, sCG, and PrCG at the onset of spoken response (Fig. 4C); other articulation-aligned responses were noted in pTr, pOp, and MFG (Fig. 3B). Together, these patterns of activity implicate the mFuG as a likely network hub for mediating cortical interactions during the early periods following stimulus onset, whereas sensorimotor regions would be anticipated to show strongest interactions closer to articulation onset.

Notably, despite extensive temporal and parietal coverage in our subject cohort (Fig. 1B), these regions were not seen to be active via the SB-MEMA. This absence of activity may be due to the fact that these regions are more engaged during semantic integration than during reading (Price and Devlin, 2011; Carreiras et al., 2014; Pattamadilok et al., 2015), while the word-stem completion task relies more heavily on phonologic, syntactic, and articulatory processes. The active cortical regions identified by SB-MEMA are consistent with the early PET and fMRI studies of word stem completion (Dhond et al., 2001; Palmer et al., 2001), as well as a recent fMRI meta-analysis (Martin et al., 2015). Our findings thus challenge models that propose grapho-phonological conversion occurs solely in the middle temporal and lateral parietal areas (Jobard et al., 2003; Levy et al., 2009) and instead offer support for the notion that the mFuG is involved in two functions: both grapho-phonological conversion as well as whole-word recognition (Martin et al., 2015). The cortical substrates (mFuG, MFG, PrCG, sCG, PoCG, pTr, and pOp) identified here using SB-MEMA prominently overlap a large meta-analyses of fMRI data from the past 20 years (Price, 2012), with the added benefit that we were able to evaluate strictly feedforward versus interactive interregional pathways, for which purpose time series analyses and interregional connectivity measures were performed (Price and Devlin, 2011; Carreiras et al., 2014).

The interregional connectivity analysis using grouped SdDTF clarified the directionality and strength of interactions. Saliently, informational flow analyses revealed that all regions were significantly linked with mFuG following stimulus onset, primarily through feedback inputs (Fig. 5). This finding is consistent with reports by others that stronger inflows are positively correlated with an increase in neural activity (and vice versa) (Jiao et al., 2011). This result challenges empirical evidence in favor of strictly feedforward flow of information following stimulus onset reported by others; namely, mFuG → lateral parietal regions and mFuG → inferior frontal cortices were found (Jobard et al., 2003; Levy et al., 2009), yet these studies did not find flows in the opposite direction to be present during word-reading tasks. Stimulus-locked connectivity from frontal (pTr, pOp, and MFG) and sensorimotor cortices to mFuG (Schurz et al., 2014) provides clear evidence in support of orthographic-phonological modulation during early processing periods (Bokde et al., 2001; Jobard et al., 2003; Mechelli et al., 2005; Mainy et al., 2008; Sahin et al., 2009). These connectivity patterns show that mFuG's role is likely more complex than feedforward, orthographic conversion (Dehaene et al., 2002; Jobard et al., 2003). In accordance with interactive theories of word reading, these top-down processes were likely engaged to minimize error and facilitate task performance (Price and Devlin, 2011; Carreiras et al., 2014). Interestingly, prominent interactions were also seen involving MFG, pTr, and sensorimotor regions during the time intervals before periods when their regional gamma responses showed significance. We propose that these interregional interactions likely reflect inhibitory processing, which enabled the network to efficiently perform the earliest orthographic-phonological processes.

During articulation-aligned timeframes, a highly prominent finding was the increase in interactivity involving sensorimotor cortex (Fig. 6). Consistent and significant communication from MFG and pTr to sensorimotor cortex was observed, in line with anterior-to-posterior connectivity patterns reported recently by others (Voytek et al., 2015). Together, these findings suggest that decision-making processes were engaged during the selection of an appropriate response for each word stem. The exchanges observed between these between pOp and sensorimotor regions are also in agreement with their role in the articulatory network (Hickok and Poeppel, 2007) and likely reflect the assemblage of appropriate phonological codes before spoken response. Finally, as the network progressed closer to articulation, strong intraregional connectivity among the PrCG, PoCG, and sCG was observed, suggesting the coupling of these regions accompanied the execution of the phonological codes. Of note during this epoch, strong feedback connectivity to mFuG from these prefrontal regions continued to be observed, even as the BGA in the mFuG decreased back toward baseline. These later inputs to mFuG would therefore likely include inhibitory signals from sensorimotor cortex (Flinker et al., 2015).

In sum, we have provided compelling support for mFuG's relevance to word reading processes realized through top-down communication pathways with pOp, pTr, MFG, sCG, PoCG, and PrCG. Our work directly addresses long standing debates, all of which have indicated the need for comprehensive descriptions of interregional connectivity during word reading processes (Pammer et al., 2004; Price and Devlin, 2011; Carreiras et al., 2014). Importantly, we circumvent confounds of past behavioral, fMRI, MEG, and PET research with increased temporal and spatial specificity of multihuman word reading data paired with a quantifiable metric of interregional connectivity. We improved upon prior methods by quantifying directionality (Mainy et al., 2008; Thesen et al., 2012), magnitude (Bedo et al., 2014), and precise timing of grouped, interregional interactions during lexical processing. Our results are consistent with the notion that word reading occurs within an integrative framework between the mFuG, PFC, and sensorimotor cortex. The k-means cluster analysis suggests that the networks delineated here proceed through six stages, supporting arguments that cognitive processes emerge from distinct stages of processing along a distributed network, rather than from a serial process that engages substrates in stepwise fashion (Voytek et al., 2015).

Limitations

Despite the extensive coverage offered across all 15 patients (Fig. 1), some task-relevant regions remained undersampled in this study. Coverage in occipital cortex was particularly limited, leaving us unable to explore some grapho-phonological pathways, such as those originating in occipital cortex (Levy et al., 2009; Seghier et al., 2012; Simos et al., 2013; Schurz et al., 2014).

Furthermore, our work is limited by the relative difficulty of word-stem completion compared with low-level paradigms often studied. Dehaene and Cohen (2011) suggest that top-down connectivity is seen with more demanding tasks; hence, our findings would benefit from replication during low-level conditions, such as covert and overt word reading, or phonological/semantic decision making. This will be the focus of future work from our group.

Finally, concerns might be expressed that our analysis was performed on data from epilepsy patients. Thorough care was taken to review patients' IQ scores and to ensure they were capable of performing the task. Furthermore, we followed standard protocols to remove trials and electrodes with interictal activity (Jerbi et al., 2009). We have previously shown that fMRI activation patterns in patients with epilepsy do not significantly differ from those in a healthy population (Conner et al., 2014); the overlap of cortical substrates localized with ECoG in patients with those localized by fMRI in a healthy cohort during reading further validate the relevance of these results. (Price, 2012).

In conclusion, we have begun to untangle open questions in current word reading models (Mechelli et al., 2005; Mainy et al., 2008; Bedo et al., 2014) by elucidating the spatiotemporal, interregional dynamics of multidimensional networks from grouped ECoG data recorded during a word-stem completion task. Our results are not aligned with the notion that word reading occurs in a modular, feedforward fashion. Rather, our results support complex, interactive framework as the more biologically plausible explanations for word reading processes.