Brain network dynamics in the human articulatory loop

Abstract

Objective

The articulatory loop is a fundamental component of language function, involved in the short-term buffer of auditory information followed by its vocal reproduction. We characterized the network dynamics of the human articulatory loop, using invasive recording and stimulation.

Methods

We measured high-gamma activity_{70-110 Hz} recorded intracranially when patients with epilepsy either only listened to, or listened to and then reproduced two successive tones by humming. We also conducted network analyses, and analyzed behavioral responses to cortical stimulation.

Results

Presentation of the initial tone elicited high-gamma augmentation bilaterally in the superior-temporal gyrus (STG) within 40 ms, and in the precentral and inferior-frontal gyri (PCG and IFG) within 160 ms after sound onset. During presentation of the second tone, high-gamma augmentation was reduced in STG but enhanced in IFG. The task requiring tone reproduction further enhanced high-gamma augmentation in PCG during and after sound presentation. Event-related causality (ERC) analysis revealed dominant flows within STG immediately after sound onset, followed by reciprocal interactions involving PCG and IFG. Measurement of cortico-cortical evoked-potentials (CCEPs) confirmed connectivity between distant high-gamma sites in the articulatory loop. High-frequency stimulation of precentral high-gamma sites in either hemisphere induced speech arrest, inability to control vocalization, or forced vocalization. Vocalization of tones was accompanied by high-gamma augmentation over larger extents of PCG.

Conclusions

Bilateral PCG rapidly and directly receives feed-forward signals from STG, and may promptly initiate motor planning including sub-vocal rehearsal for short-term buffering of auditory stimuli. Enhanced high-gamma augmentation in IFG during presentation of the second tone may reflect high-order processing of the tone sequence.

Significance

The articulatory loop employs sustained reciprocal propagation of neural activity across a network of cortical sites with strong neurophysiological connectivity.

Introduction

Humans begin to imitate speech and environmental sounds during infancy (Darwin, 1877; Lenneberg et al., 1965), and acquire the ability to accurately hum pitches often through singing activities at school (Trollinger, 2003). Vocal reproduction requires perception and analysis of auditory stimuli, preparation or planning of vocalization/phonation of sounds, followed by execution of vocalization. Auditory perception and analysis are believed to be performed primarily in the superior-temporal gyrus (STG), whereas execution of vocalization relies primarily on sensorimotor cortex (Binder et al., 1996; Boatman and Miglioretti, 2005; Zatorre et al., 2007; Leff et al., 2009). Based on the ‘articulatory loop model’ (Baddeley and Hitch, 1974; Baddeley, 1986), accurate vocal responses to speech or non-speech sounds are secured by ‘short-term buffering of stimuli via sub-vocal rehearsal’ during motor preparation. Functional imaging studies have inferred that either STG, precentral gyrus (PCG), or inferior-frontal gyrus (IFG) may be involved in such short-term buffering of auditory stimuli (Paulesu et al., 1993; Rauschecker and Scott, 2009; Price, 2010).

The relatively under-reported neurobiological aspects of the articulatory loop include the temporal dynamics of cortical activations in STG, PCG, and IFG during listening to sound stimuli. This is partly because of the limitations of non-invasive neuroimaging techniques. Functional MRI (fMRI) has a temporal resolution on the order of seconds and is an indirect measure of neuronal activity at best. Noninvasive electrophysiology recordings, such as EEG and magnetoencephalography (MEG) have excellent temporal resolution, but tend to suffer from unwanted artifactual signals during overt repetition tasks (Yuval-Greenberg et al., 2008; Ball et al., 2009; Carl et al., 2012). In the present study using intracranial electrocorticography (ECoG), we investigated the temporal relationships between STG, PCG, and IFG activations when patients listened to tone sequences, with or without subsequent vocal reproduction (humming). We used augmentation of high-gamma activity at 70-110 Hz as a summary measure of in situ neural activation (Tanji et al., 2005; Crone et al., 2006; Lachaux et al., 2012; Kojima et al., 2013).

The first question in this study was “When do PCG and IFG initially respond to the given sounds?” The initial response within these regions can be taken to signify the steps within the articulatory loop whereby sensory information is entered into a short-term buffer for guiding sound reproduction. Based on hierarchical models of language (Geschwind, 1979), one may hypothesize that PCG and IFG, primarily responsible for speech production, would be involved immediately before and/or during the execution of overt responses. An alternative hypothesis is that these structures are involved during initial processing of auditory stimuli. The latter hypothesis finds support in previous fMRI observations that attention to non-speech auditory stimuli, even without overt responses, induces hemodynamic activation of bilateral PCG and IFG, inferring that these structures exert function beyond the mere execution of overt responses (Schubotz et al., 2003; Bangert et al., 2006). Several ECoG studies have reported that neural activation may take place in frontal lobe sites during listening to human voices (Brown et al., 2012b; 2014; Cogan et al., 2014; Potes et al., 2014; Cheung et al., 2016).

We subsequently sought to determine if and when sound-listening-related high-gamma augmentation in PCG and IFG is further enhanced by the request to reproduce the perceived sounds. Here, patients were instructed to listen to two successive tone pitches with and without subsequent imitation of the given pitches. The former task condition effectively imposed ‘short-term buffer of auditory stimuli’ not required in the latter condition. We predicted that greater high-gamma augmentation during the sound-imitation task would help localize sites involved in the short-term buffering of auditory stimuli as part of the motor planning process.

We then performed two independent analyses of cortical connectivity among the regions involved in the articulatory loop. We specifically measured the causal flow of high-gamma activity between STG, PCG, and IFG using event-related causality (ERC) analysis (Korzeniewska et al., 2011; Flinker et al., 2015). Moreover, we tested the hypothesis that single pulse (i.e.: no faster than 1 Hz) stimulation of high-gamma STG sites would elicit cortico-cortical evoked potentials (CCEPs; Matsumoto et al., 2004; Matsuzaki et al., 2013), an index of strong functional connectivity, at PCG and IFG sites, and vice versa. We predicted that these analyses would provide direct evidence for strong functional connectivity among the sites most active in the articulatory loop, as well as task-related neural propagation between these sites. Based on the common behavioral phenomenon that irrelevant sounds can disrupt short-term memory performance (Macken et al., 2009), we predicted that frontal lobe sites participating in the short-term buffering of auditory stimuli would interact with those participating in auditory perception during sound presentation.

Finally, we tested the hypothesis that vocalization would be transiently impaired by stimulation with a train of pulses (i.e.: 50 Hz) at frontal lobe sites (PCG and IFG) showing rapid high-gamma augmentation during sound presentation. This hypothesis was based on the theory that non-speech sound stimuli are directly and rapidly processed by sub-vocal rehearsal (motor function) for short-term buffering of perceived acoustic representations prior to their vocal reproduction (Baddeley and Hitch, 1974; Baddeley, 1986). Alternatively, if stimulation induced auditory hallucinations or altered auditory perception, the stimulated site would be interpreted as part of the perceptual system.

Methods

Patients

The inclusion criteria included: (i) chronic ECoG recording as part of clinical management of drug-resistant seizures at Children's Hospital of Michigan in Detroit (Asano et al., 2009), (ii) ECoG sampling from STG, PCG, and IFG, and (iii) completion of the auditory tasks described below. The exclusion criteria included: (i) brain malformations or seizure onset zone involving STG, PCG or IFG, (ii) history of hearing impairment, and (iii) severe cognitive dysfunction reflected by verbal comprehension index of <70. We studied 10 right-handed children (age range: 11-17 years; average age: 14 years; seven females) who satisfied both inclusion and exclusion criteria. ECoG was sampled from the left hemisphere in five patients and from the right in the remaining five (Table 1). The study was approved by the Institutional Review Board at Wayne State University, and written informed consent was obtained from the guardians of all patients. Written assent was obtained from children above 13. Oral assent was obtained from younger children.

Table 1. Patient profiles.

Patient Number	Age (years)	Gender	Antiepileptic drugs	Sampled hemisphere	CCEPs	SOZ	Resection area	ILAE seizureoutcome**
1	11	F	LEV; OXC	Left	Not performed	Temporal*	Temporal	1
2	11	F	LEV; OXC	Right	Performed	Parietal	Parietal	1
3	11	F	LEV; OXC	Right	Performed	Frontal	Frontal	1
4	12	M	VPA	Left	Performed	Temporal	Temporal	3
5	13	F	LEV	Left	Performed	Temporal	Temporal	1
6	14	F	LAC; LEV	Right	Not performed	Parietal*	Parietal	3
7	14	F	OXC	Left	Not performed	Temporal*	Temporal	1
8	15	M	LAC	Right	Not performed	Frontal	Frontal	1
9	17	F	LAC; LTG	Left	Performed	Temporal*	Temporal	1
10	17	M	LAC; LTG	Right	Not performed	Occipital	Occipital	1

F: Female. M: Male. SOZ: Seizure onset zone. CCEPs: Cortico-cortical evoked potentials. LAC: Lacosamide. LEV: Levetiracetam. LTG: Lamotrigine. OXC: Oxcarbazepine. VPA: Valproate. Five patients failed to undergo the measurement of CCEPs in the present study, due to the insufficient ECoG monitoring period.

^*SOZ was localized at the medial or inferior surface, not visible on the lateral view.

^**Patient 2 underwent focal resection involving the right post-central gyrus and had persistent mild left hemiparesis. None of the other patients developed permanent deficits in motor or speech function.

Subdural electrode placement, ECoG recording, and imaging process

Platinum macro-electrodes (10 mm center-to-center distance) were placed in the subdural space generously over the affected hemisphere following frontotemporal craniotomy (Figure 1), to satisfactorily determine the boundaries between the epileptogenic zone and eloquent areas (Asano et al., 2009; Nakai et al., 2017). The location of electrode placement was determined purely based on clinical needs, and we do not place electrodes more than clinically indicated (Nonoda et al., 2016).

Figure 1. Location of subdural electrodes.

Red circles: Superior-temporal gyrus (STG). Blue circles: Superior precentral gyrus (sPCG). Yellow circles: Inferior precentral gyrus (iPCG). Green circles: Inferior-frontal gyrus (IFG). Black squares: Seizure onset zone excluded from data analysis. Oblique slashes: Channels contaminated with artifacts and excluded from time-frequency ECoG analysis.

A three-dimensional surface image was created with the location of electrodes directly defined on the brain surface (von Stockhausen et al., 1997; Muzik et al., 2007; Alkonyi et al., 2009; Nakai et al., 2017). The spatial normalization of each individual electrode site was performed using FreeSurfer scripts (http://surfer.nmr.mgh.harvard.edu; Desikan et al., 2006; Chan et al., 2011; Ghosh et al., 2010; Matsuzaki et al., 2015). Automatic parcellation of cortical gyri was performed on both individual and spatially normalized brain surfaces (Supplementary Figure S1), and all electrode sites were assigned anatomical labels.

ECoG recordings were obtained for three to six days with a sampling frequency at 1,000 Hz and amplifier band pass at 0.016-300 Hz. ECoG signals were then re-montaged to a common average reference (Korzeniewska et al., 2011; Kojima et al., 2013). Electrodes overlying seizure onset zones (Asano et al., 2009) or structural lesions were excluded from further analysis (Jacobs et al., 2009; Zijlmans et al., 2012). It has been inferred that high-frequency oscillations time-locked to physiological events likely reflect physiological phenomena, whereas those spontaneously generated by the seizure onset zone are likely to be pathologic (Matsumoto et al., 2013).

The regions of interest (ROIs) included: STG, sPCG (defined as the precentral gyrus at 4-5 cm above the sylvian fissure), iPCG (precentral gyrus at <4 cm from the sylvian fissure), and IFG (summation of pars opercularis and pars triangularis). We have divided the PCG into two ROIs, since previous fMRI and ECoG studies suggested the presence of two distinct laryngeal motor areas closely corresponding to sPCG and iPCG (Brown et al., 2008a; Bouchard et al., 2013). The functional dynamics of these distinct laryngeal motor areas during sound replication have remained uncertain.

Task

We measured high-gamma activity during ‘sound-listening alone’ and ‘sound-listening and humming’ (Figure 2), using a method similar to that previously reported (Brown et al., 2008b; Fukuda et al., 2008; Toyoda et al., 2014). Patients were initially instructed to covertly listen to a total of thirty 1,000 ms auditory stimuli. Each 1,000 ms sound stimulus consisted of two successive, distinct tone pitches of 500 ms duration each. Five to ten minutes after the completion of the ‘sound-listening alone’ task, the ‘sound-listening and humming’ task was assigned. Patients were instructed to overtly imitate provided sound stimuli by humming; thereby, the pitches of vocalized sounds simply depend on the shape and tension of the vocal folds, with minimal mouth movement. In this study focusing on the articulation loop system, the tasks were designed so that patients would be minimally required to process semantic, syntactic, or phonological information of a given sound. The response time was defined as the period between the offset of a sound stimulus and the onset of the humming response.

Figure 2. Tasks.

(A) ‘Sound-listening alone’ task. (B) ‘Sound-listening and humming’ task. Each 1,000-ms sound stimulus consisted of two successive, distinct tone pitches (for example, ‘E major’ followed by ‘A major’) of 500-ms duration and 70-dB intensity. The frequency of each 500-ms tone pitch was either 147 Hz (equivalent to ‘D major’), 165 Hz (‘E major’), 175 Hz (‘F major’), 196 Hz (‘G major’), 220 Hz (‘A major’), or 247 Hz (‘B major’). Each type of 500-ms tone pitch was used 10 times during each task, and the order of presentation of sound stimuli was pseudo-randomized. Inter-trial interval randomly ranged from 4 to 8 seconds. Both tasks consisted of 30 trials.

Measurement of event-related high-gamma modulation

We determined the spatial-temporal dynamics of high-gamma amplitudes_{70-110 Hz} time-locked to the onset of the stimulus presentation and humming response. Each ECoG trial was transformed into the time-frequency domain using a complex demodulation technique (Papp and Ktonas, 1977; Hoechstetter et al., 2004; Kambara et al., 2017). Time-frequency transformation was performed for frequencies between 70 and 110 Hz and latencies between -1,000 and +1,500 ms time-locked to stimulus onset as well as between -1,000 and +500 ms time-locked to response onset, in steps of 5 Hz and 10 ms. At each time-frequency bin, we analyzed the percent change in high-gamma amplitude_70-110Hz (averaged across trials) compared to the mean amplitude in a reference period at -1000 to -200 ms relative to stimulus onset. We subsequently determined whether the degree of event-related high-gamma augmentation at each 10 ms window at each channel reached significance using studentized bootstrap statistics followed by Bonferroni correction (for multiple time-frequency bins over time). The level of significance was set at corrected p=0.05. We determined whether (i) high-gamma augmentation would initially involve STG, and subsequently PCG and IFG; (ii) high-gamma augmentation at PCG and IFG would take place before stimulus offset rather than immediately before the humming responses; and (iii) such high-gamma augmentation at PCG and IFG would be indeed larger during the ‘sound-listening and humming’ task compared to during the ‘sound-listening alone’ task.

Event-related causality (ERC) analysis

We determined the dynamics of interactions at high-gamma range between cortical sites within the articulatory loop system, using the ERC method identical to that previously reported (Korzeniewska et al., 2011; Flinker et al., 2015). We presented the direction, intensity, and temporal courses of high-gamma activity propagation among cortical sites causally influencing other given sites. The ERC method is a multichannel extension of the Granger causality concept, which states that an observed time series x(t) causes another series y(t), if knowledge of x(t)'s past significantly improves prediction of y(t). ERC gives an estimate of changes in the intensity and direction of neural activity propagation between recording sites as a function of frequency, in comparison to baseline activity. A greater ERC value _X→Y infers that propagation of high-gamma activity from site X causally influences site Y by Granger criteria (Korzeniewska et al., 2008). For each patient, ERC values time-locked to stimulus or response onset were compared to baseline epochs preceding each trial, using penalized spline (Crainiceanu et al., 2005). Thereby, trials containing an interictal spike were removed so that the potential influence of epileptic activity on the network was diminished (van Diessen et al., 2013; Lopes et al., 2014).

For group analysis, ERC values reaching significance (p=0.05 Bonferroni corrected for the number of time-frequency points analyzed) were pooled-averaged across patients. This approach was taken to reassure equal contribution of each patient in the group average, whereas the numbers of electrodes in each ROI differed across patients. To obtain the pooled-average across patients, causal flows were averaged initially at an individual level. Subsequently, individual means of ERC flows were grand-averaged across all patients. The sites included in the group ERC analyses were STG, sPCG, iPCG, and IFG. We determined if these ROIs would reciprocally interact with each other during sound presentation as well as before and around the onset of overt responses.

Measurement of cortico-cortical evoked potentials (CCEPs)

The general principles of data acquisition and analysis were previously described (Matsuzaki et al., 2013). Trains of electrical stimuli were delivered to a contiguous pair of subdural electrodes at a frequency of 1 Hz for 40 s. Each electrical stimulus consisted of a square wave pulse of 0.3 ms duration, 5 mA intensity, and biphasic polarity. ECoG signals were averaged time-locked to the onset of each electrical stimulus with a time window of -100 to +500 ms with a low-frequency filter of 5.0 Hz. We determined if stimulation of electrode pairs including the earliest high-gamma STG site would elicit larger CCEPs at high-gamma PCG and IFG sites, compared to at non-high-gamma sites within a given gyrus. The size of CCEPs was quantified using the area under the curve of ECoG signals between 11 and 250 ms, by taking into account the effect of stimulation artifacts ranging between 0 and 10 ms (Matsumoto et al., 2004). The analysis included ECoG signals recorded at least 2 cm away from the stimulus site, by considering the potential effects of stimulation artifacts on immediately surrounding sites (Swann et al., 2012). We likewise determined if stimulation of electrode pairs including the earliest high-gamma PCG site would elicit larger CCEPs at high-gamma compared to non-high-gamma sites within STG.

Direct cortical stimulation

Cortical mapping by electrical stimulation was performed, as part of presurgical evaluation (Kojima et al., 2012; Kumar et al., 2012; Nakai et al., 2017). A pulse-train of stimuli was delivered to neighboring electrode pairs, with frequency of 50 Hz, pulse duration of 300 μsec, and train duration ranging up to 5 s. Stimulus intensity was initially set to 3 mA, and increased up to 9 mA until either clinical responses or after-discharges were observed. During each period of stimulation, patients were asked to answer brief spoken questions. Sites at which stimulation reproducibly resulted in auditory perceptual changes, speech arrest, changes in voice, failure to verbalize correct responses, or sensorimotor symptoms involving the mouth or throat, were determined by at least two investigators, including at least one neuropsychologist (Kojima et al., 2012). Additional tasks such as syllable repetition, humming, counting, reciting ABC's, or picture naming were performed, when possible, to determine the specific role of each cortical site at which stimulation resulted in a clinical symptom. We determined if stimulation of high-gamma PCG and IFG sites, on either hemisphere, would transiently impair vocalization or alter auditory perception.

Results

Behavioral data

All patients successfully completed both ‘sound-listening alone’ and ‘sound-listening and humming’ tasks; sound stimuli were imitated in 29.8 out of 30 trials on average across patients. None of the patients reported difficulty in imitating given sounds. The median response time was 941.3 ms (standard deviation [SD]: 407.9 ms).

The spatial-temporal profiles of high-gamma augmentation on animation movies

As best presented in Videos S1 and S2, high-gamma augmentation involved the posterior STG immediately following stimulus onset, and subsequently sPCG and IFG sites, during both ‘sound-listening and humming’ and ‘sound-listening alone’ tasks (Figure 3). High-gamma activity subsequently involved a large extent of Rolandic cortex including iPCG particularly around the onset of the humming response. The left and right hemispheres shared the afore-mentioned spatial-temporal profiles of high-gamma augmentation during both tasks (Figure 4).

Figure 3. The spatial-temporal characteristics of high-gamma activity during the auditory tasks in patient #5.

The upper and lower rows show the snapshots of high-gamma activity during ‘sound-listening and humming’ and ‘sound-listening alone’ tasks (Video S1). The percent change in high-gamma amplitude compared to that during the reference period is presented (Fukuda et al., 2008; Toyoda et al., 2014). High-gamma augmentation involved the left STG (light blue arrows), and the PCG particularly during ‘sound-listening and humming’ task (red arrow). High-gamma augmentation persisted after stimulus offset (i.e.: >1000 ms after stimulus onset) only during the ‘sound-listening and humming’ task, and involved a larger extent of PCG immediately prior to and during the humming response (blue arrow).

Figure 4. The spatial-temporal characteristics of high-gamma activity in patient #3 (Video S2).

During both tasks, high-gamma augmentation involved the right STG (light blue arrows), sPCG (red arrows), and IFG (yellow arrows). High-gamma augmentation in sPCG persisted after stimulus offset only during ‘sound-listening and humming’ task, and involved a larger extent of PCG (blue arrow) immediately prior to and during the humming response.

High-gamma augmentation in the superior-temporal gyrus (STG)

In all 10 patients, significant high-gamma augmentation occurred in at least one STG site during the time interval between stimulus onset and offset. The earliest high-gamma augmentation was observed within the posterior half of the STG (Figure 5A). Group analysis of STG sites, with the earliest activation in each subject, revealed that the onset of high-gamma augmentation was 40 ms following stimulus onset (Figure 5B). High-gamma activity was augmented prominently following onset of the first tone pitch (maximum percent change: 85% on average), and less prominently following onset of the second one (58%). High-gamma amplitude during the first pitch was larger than that during the second (p=0.04 on studentized bootstrap test).

Figure 5. The temporal dynamics of high-gamma activity.

(A) The locations of the earliest high-gamma STG sites are denoted on the FreeSurfer average brain. Sites are color-coded for each patient. (B) Mean high-gamma activity across these 10 STG sites is plotted between -1,000 and +1,500 ms relative to stimulus onset. (C) The mean of high-gamma activity relative to response onset. (D-F) Ten earliest high-gamma sPCG sites, and the mean high-gamma activity. (G-I) Seven earliest high-gamma iPCG sites. (J-L) Five earliest high-gamma IFG sites.

High-gamma augmentation in the superior precentral gyrus (sPCG)

In all 10 patients, significant high-gamma augmentation involved at least one precentral site during sound presentation. The earliest precentral high-gamma augmentation was observed 4-5 cm superior to the Sylvian fissure (defined as sPCG in the present study; Figure 5D). Group analysis of these sPCG sites, with the earliest activation in each subject during sound presentation, revealed that high-gamma augmentation took place within 160 ms following stimulus onset and was sustained until stimulus offset (Figure 5E). High-gamma amplitude during the ‘sound-listening and humming’ task was larger than that during the ‘sound-listening alone’ task (maximum percent change: 37% vs 26%; p=0.02 on studentized bootstrap test). Only the ‘sound-listening and humming’ task was associated with sPCG high-gamma augmentation continuing after stimulus offset, and high-gamma augmentation peaked immediately after response onset (Figure 5F).

High-gamma augmentation in the inferior precentral gyrus (iPCG)

In seven patients, significant high-gamma augmentation involved at least one iPCG site (Figure 5G). Group analysis, employed on the seven earliest high-gamma iPCG sites, revealed that high-gamma augmentation began 250 ms after stimulus onset (Figure 5H). High-gamma amplitude during the ‘sound-listening and humming’ task was larger than that during the ‘sound-listening alone’ task (maximum percent change: 25% vs 12%; p=0.03 on studentized bootstrap test). High-gamma augmentation in these iPCG sites subsided before stimulus offset during the ‘sound-listening alone’ task, but increased over time until response onset during the ‘sound-listening and humming’ task (Figure 5I).

High-gamma augmentation in the inferior-frontal gyrus (IFG)

In five patients, significant high-gamma augmentation involved at least one IFG site (Figure 5J). Group analysis, employed on the five earliest high-gamma IFG sites, revealed that the onset of high-gamma augmentation was 160 ms. In contrast to STG (Figure 5B), high-gamma activity was augmented only modestly during the first tone pitch, but rather prominently during the second pitch (maximum percent change: 25% during the first and 43% during the second pitch; Figure 5K); thereby, we found that the temporal change (i.e.: slope) of high-gamma amplitude from the first to the second pitch differed between IFG and STG (p=0.02 on studentized bootstrap test). The maximum percent change in high-gamma activity was similar between ‘sound-listening and humming’ and ‘sound-listening alone’ tasks (Figure 5K). High-gamma augmentation in IFG subsided following stimulus offset during both tasks and resumed after response onset (Figure 5L).

Results of event-related causality (ERC) analysis

ERC group analysis revealed the dynamics of reciprocal interactions among STG, PCG, and IFG high-gamma sites, during the ‘sound-listening and humming’ task (Videos S3 and S4). Figure 6A presents the ERC flows, averaged across patients, time-locked to stimulus onset. Consecutive plots (from left to right) show ERC measures as a function of time window: ‘0 to 0.25’, ‘0.25 to 0.5’, ‘0.5 to 0.75’, ‘0.75 to 1’, and ‘1 to 1.25 s’. Immediately following stimulus onset (0 to 0.25 s), strong local interactions occurred between STG high-gamma sites, with less intense reciprocal propagations noted between STG and sPCG. Subsequently (0.25 to 0.5 s), reciprocal propagations took place between IFG and sPCG. During presentation of the second tone stimulus (0.5 to 0.75 and 0.75 to 1 s), the most pronounced were propagations from IFG to sPCG and from sPCG to STG. Figure 6B presents the averaged ERC flows time-locked to response onset. Immediately before response onset (-0.2 to 0 s), strong neural propagation occurred from iPCG to sPCG, and from sPCG to STG, and within sPCG sites. The strength of such neural interaction was smaller at ‘-0.55 to -0.2 s’ prior to response onset.

Figure 6. Results of event-related causality (ERC) analysis.

ERC analyses on each of the groups of five patients with electrodes placed over left or right hemispheres revealed similar patterns of network processing; therefore, we present the data averaged across all 10 study patients on the left hemisphere template. (A) Pooled averaged ERC neural flows time-locked to stimulus onset. Both width and color (thin-yellow: weak; thick-red: strong) of arrows represent intensity of causal flows. Linear arrow: neural flow between regions of interest (ROIs). Circular arrow: flow within each ROI. ERC flows, averaged across hemispheres, are presented on the left hemisphere template. (B) Pooled averaged ERC neural flows time-locked to response onset.

Cortico-cortical evoked potentials (CCEPs)

Measurement of CCEPs confirmed the presence of reciprocal and effective connectivity between high-gamma STG, PCG, and IFG sites. Figure 7 shows the results of an individual patient. Group analysis was employed on CCEPs elicited by stimulation of the 10 earliest high-gamma STG sites, and it suggested that the areas under the curve at the high-gamma PCG and IFG sites were much larger (by 70% and 108%, respectively) compared to those at non-high-gamma sites within a given gyrus (p<0.001 on studentized bootstrap test). Likewise, stimulation of the earliest high-gamma sPCG sites elicited large CCEPs at high-gamma STG and IFG sites. Group analysis was employed to these earliest high-gamma sPCG sites, and it suggested that the areas under the curve of CCEPs at high-gamma STG and IFG sites were much larger (by 53% and 88% on average) compared to those at non-high-gamma sites within a given gyrus (p<0.001). As best appreciated in Video S5, CCEPs noted at distant high-gamma sites could not be attributed to volume conduction from the stimulated site (Shimada et al., 2017). Stimulation confined to high-gamma IFG sites was available only in patients #3 (Figures 7E and 7I) and #9.

Figure 7. Cortico-cortical evoked potentials (CCEPs) in patient #3.

(A) The locations of stimulated pairs. (B) The dynamic change of high-gamma activity. CCEPs elicited by stimulation of (C and G) STG, (D and H) sPCG, (E and I) IFG, and (F and J) iPCG pairs.

Functional role clarified by cortical stimulation

Regardless of hemispheres, stimulation of PCG (but not IFG) high-gamma sites induced either speech arrest, inability to control vocalization, or forced vocalization. Figure 8 shows the results of an individual patient. The probability of such impaired vocalization induced by stimulation of the earliest high-gamma PCG site was 0.70 on average across patients (Supplementary Table S1). Stimulation of PCG sites showing high-gamma augmentation at any time point induced impaired vocalization with a probability of as large as 0.54. These were both greater than 0.26, the chance probability of impaired vocalization resulted from stimulation of all precentral sites (p≤0.03 on Wilcoxon Signed Rank test). Stimulation of a high-gamma IFG site impaired vocalization in only one out of the five patients; rather, stimulation of high-gamma IFG sites induced ‘naming error’ with a probability of 0.60. No auditory symptoms were induced by stimulation of PCG or IFG sites in any of our study patients, whereas auditory hallucination was induced by stimulation of a high-gamma STG site in one of the 10 patients.

Figure 8. Functional role of the right sPCG high-gamma site.

(A) The results of cortical stimulation mapping are summarized. Stimulation of B29-B21 pair at sPCG impaired vocalization without sensorimotor symptoms in patient #3. (B) The dynamic changes of high-gamma activity at eight sites are presented. Only B29 showed significant high-gamma augmentation during ‘sound-listening and humming’ task. (C) Sound spectrograms during stimulation mapping is presented. Rhythmic vocalization and humming are reflected by rhythmic augmentation of sound amplitude. Stimulation of B29-B21 constantly and specifically resulted in transient inability to control vocalization.

Discussion

The role of precentral gyrus in subvocal rehearsal and short-term buffering of auditory information

We observed high-gamma augmentation in both left and right sPCG, beginning 160 ms after stimulus onset and sustained until the overt response. This observation is consistent with those in previous fMRI and ECoG studies reporting that PCG can be activated together with STG even during sound listening tasks not requiring responses of any kind (Schubotz et al., 2003; Bangert et al., 2006; Potes et al., 2014). The temporal dynamics of task-related cortical activations provide an important clue to determine the function of the activated regions; yet, the temporal order of high-gamma augmentation per se does not clarify whether such PCG activations reflect perceptual or motor function, or both. Here, we predicted that functional interference by electrical stimulation could address this issue and rule out the possibility that an un-sampled structure such as the thalamus transfers the same signals to two activated regions at different time points. Indeed, our CCEPs and ERC analyses suggested that PCG can directly receive feed-forward signals from STG. Cortical stimulation revealed that the earliest high-gamma sPCG sites have a critical functional role in vocalization. Furthermore, imposing ‘short-term buffering of auditory stimuli’, by asking patients to reproduce tone stimuli, enhanced the degree of PCG high-gamma augmentation during sound presentation (Figures 5E and 5H). These combined findings constitute strong evidence that portions of PCG rapidly initiate and maintain subvocal rehearsal to buffer auditory information before a wider region of PCG initiates vocalization.

The aforementioned statement is consistent with the concept of an ‘articulatory loop’ proposed by Baddeley (1986). We found that non-speech auditory information was rapidly processed at sPCG, spatially concordant with the upper border of the laryngeal motor cortex (Brown et al., 2008a; Bouchard et al., 2013). Rapid and bilateral involvement of such laryngeal motor cortices was reported in several ECoG studies including our own (Brown et al., 2012b; 2014; Cogan et al., 2014) but was not emphasized in the articulatory network proposed in the dual-stream models of auditory language processing (Hickok and Poeppel, 2007; Rauschecker and Scott, 2009), in which auditory information is transferred to the left IFG and the left premotor region anterior to the laryngeal motor cortex via the dorsal pathway. The lack of emphasis of the role of PCG in these prior models might be attributed to the experimental designs in previous fMRI studies, which typically did not require overt vocalization. Emphasis of the left hemisphere in the dual-stream models may be attributed to the types of tasks employed in previous studies, which often consisted of repetition of words and syllables (Hickok and Poeppel, 2007); such tasks likely require higher-order language processes widely known to be biased to the left hemisphere. Conversely, non-speech pure tone stimuli were utilized in the present study, allowing us to focus on more rudimentary pathways that may not necessarily be biased to one hemisphere or the other.

Our data do infer that iPCG sites also participate in motor preparation during sound listening. These portions were suggested to have primary motor function for the larynx (Brown et al., 2008a; Bouchard et al., 2013; Toyoda et al., 2014). High-gamma augmentation was likewise noted at iPCG, though the percent change in high-gamma amplitude was somewhat modest during sound presentation.

An alternative explanation for rapid and sustained sPCG high-gamma augmentation is that this region may be involved in active maintenance of working memory. Previous neuroimaging studies showed that greater working memory loads in auditory and visual domains were associated with greater hemodynamic activation including the frontal lobes involving regions such as the sPCG (Courtney et al., 1996; D'Esposito et al., 1998; Martinkauppi et al., 2000; de Fockert et al., 2001). Our recent ECoG study, using an auditory working memory task, reported that high-gamma augmentation involved the PCG with left hemispheric dominance during the short-term maintenance period and that such pre-central high-gamma augmentation was larger with greater memory loads (Kambara et al., 2017). Such a theory would explain how the sPCG sites with high gamma augmentation in this study remain activated following stimulus offset when patients were asked to hum the sounds that were heard.

Roles of STG and IFG in sound imitation

The earliest high-gamma STG sites showed reduced degree of neural activation to the second pitch (Figure 5B), whereas IFG showed increased degree of high-gamma augmentation when the second pitch was presented (Figure 5K). Relatively reduced neural activation to repetition of a similar sound (also known as repetition suppression) in the posterior STG has been reported in a number of studies using fMRI, scalp EEG, MEG, and ECoG (Bergerbest et al., 2004; Garrido et al., 2009; Vaden et al., 2010; Boutros et al., 2011; Todorovic et al., 2011). Repetition suppression is an automatic process initiated at the level of low-order sensory cortex, resulting from habituation induced by the first stimulus, neural fatigue associated with the first response, or top-down perceptual expectation to the second stimulus (Grill-Spector et al., 2006; Todorovic et al., 2011; Matsuzaki et al., 2012). The requirement of subsequent vocal reproduction minimally altered the degree of sound-related high-gamma augmentation in these regions. CCEPs and ERC analyses suggested that subsets of the high-gamma STG sites were densely connected with high-gamma IFG sites in our patient cohort and interacted with each other during sound presentation. These findings are consistent with the notion that STG and IFG exert low- and high-order processes on auditory stimuli, respectively.

Plausible high-order processes of the IFG, showing enhanced high-gamma augmentation during the second pitch, would include (i) merging two pitches as a single sound for subsequent imitation and (ii) analysis and judgment of differences between the first and second pitches as well as the externally-delivered and self-vocalized sounds. An fMRI study reported that hemodynamic IFG activation was increased when participants were given a musical chord not harmonically related to the preceding prime context (Tillmann et al., 2003). Another fMRI study reported that successful retrieval of brief musical phrases was associated with increased hemodynamic activation in the left IFG but not in STG (Watanabe et al., 2008).

Clinical implications

Our multimodal study helps clarify the role of the non-dominant PCG in subvocal rehearsal, and can explain apparent speech impairments following right hemisphere injury. Previous studies reported embolic stroke involving the right middle cerebral artery territory frequently resulted in impairment of comprehension and production of prosody (i.e.: aprosodia) including intonation, melody, pauses, timing, stress, and accent (Gorelick and Ross, 1987; Darby, 1993). Cerebral infarction restricted to the right PCG can induce anarthria (Sugishita et al., 1987; Tanji et al., 2001), a deficit in linguistic prosody (Kuschmann and Lowit, 2012), or even Broca's aphasia (Trojanowski et al., 1980). Right frontal lobectomy for drug-resistant epilepsy was reported to result in amusia, a profound deficit involving musical abilities (McChesney-Atkins et al., 2003). FMRI studies have reported that tasks involving singing or prosodic comprehension elicit hemodynamic activation rather predominantly in the right hemisphere (Riecker et al., 2000; Meyer et al., 2002; Mitchell et al., 2003). A study using transcranial magnetic stimulation reported that excitability of the right motor cortex increased during both overt singing and humming, whereas that of the left motor cortex increased during overt speech (Sparing et al., 2007). Recent studies showed that high-frequency stimulation of the PCG on either hemisphere often resulted in transient speech arrest (Tate et al., 2014; Nakai et al., 2017). Further studies using larger samples are warranted to determine the effect of right-hemispheric resection on prosodic and musical function of patients who undergo epilepsy surgery.

Methodological issues

All participants had been diagnosed with drug-resistant epilepsy, but we employed strict inclusion-exclusion criteria so that the effect of epilepsy on high-gamma measurements became minimal. Patients whose seizure onset zones involved STG, PCG or IFG were excluded from the present study. We previously reported that interictal spikes arising from the seizure onset zone can transiently suppress task-related high-gamma augmentation, but such unwanted effects were confined to the seizure onset zone and proximal cortex (Brown et al., 2012a).

Due to the lack of ECoG sampling from the opposite hemisphere, our ECoG analyses per se were not designed to determine the language dominant hemisphere or to assess neural interaction across hemispheres. Previous studies have suggested that the vast majority of patients with right-hemispheric language dominance had left-handedness associated with early left-hemispheric neocortical lesions such as cortical dysplasia or encephalomalacia (Rasmussen and Milner, 1977; Akanuma et al., 2003; Möddel et al., 2009; Kojima et al., 2013). None of our patients had these clinical and imaging profiles that would increase the likelihood of right hemispheric language dominance. Electrical stimulation mapping in the left hemisphere indeed localized essential language areas in the frontal and temporal lobes without exception. Thus, all of our patients were estimated to have either left-hemispheric dominant or bilateral language representation. We still do not have strong evidence that our observations deviated significantly from those of the general population, taking into account that a substantial proportion of healthy individuals, including typically developing children, have bilaterally symmetric hemodynamic activations during language tasks on fMRI (Gaillard et al., 2000; Szaflarski et al., 2006).

Electrical stimulation, performed as part of the presurgical evaluation of our patients, was not time-locked to the period when high-gamma augmentation took place. Thus, impaired vocalization during electrical stimulation could have been due to transient loss of either motor planning or execution (or both). Time-selective stimulation, in the future, may further dissect the causal significance of early and delayed high-gamma augmentation at relevant sites. Limitations of electrical stimulation also include insensitivity in localization of eloquent cortex in children with focal epilepsy (Chitoku et al., 2001). Particularly, stimulation of association cortex including IFG may not easily reveal the underlying function (Schevon et al., 2007; Kojima et al., 2012). Furthermore, subdural disk electrodes did not effectively stimulate the cortex facing a sulcus or fissure such as the planum temporale. Thus, the lack of induced symptoms does not necessarily suggest the absence of function but rather should be interpreted as failure to disclose the function of stimulated sites. Our recent study of 100 patients with focal epilepsy reported that auditory hallucinations were elicited by stimulation of STG with a probability of 0.19 (Nakai et al., 2017).

The present study constitutes an excellent example where two independent analytic methods, CCEPs and ERC, can help mitigate each others' potential weaknesses. ERC is designed to determine the naturalistic information flow between two sites during a given task, based on temporal relationship and specificity (Korzeniewska et al., 2008). CCEPs provide experimental evidence (Matsumoto et al., 2004; Asano et al., 2013) for the dense connectivity between two sites, and can help rule out the possibility of a third focus simply transferring the same information to two sites at different time points. One may assume that the latency of frontal high-gamma augmentation represents the temporal summation of multiple processes including (i) neural signals initially reaching STG, (ii) analysis of acoustic features at STG, and (iii) propagation of relevant neural signals from STG to a given frontal lobe site. If the STG-frontal propagation period is equivalent to the latency of frontal CCEPs elicited by STG stimulation, further studies may be needed to estimate the moment when STG generates neural discharges to be propagated toward each of frontal lobe high-gamma sites.

The present study was not designed to determine when and where lexical-semantic or syntactic systems work in the human brain. Another ECoG study with control tasks involving syllables, words, and non-speech sounds may determine if speech sounds are processed preferentially within an articulatory loop system of the left hemisphere, as proposed in the dual-stream model (Hickok and Poeppel, 2007). Future studies of patients who undergo measurements of high-gamma activity elicited by distinct control tasks are expected to segregate them from the articulatory loop system and further refine the model of language processing.

Conclusion

Taking our ECoG and stimulation data together with the existing literature, we propose a dynamic neurobiological model of the articulatory loop. According to this model (Figure 9), the articulatory loop is critically present in both left and right hemispheres without clear hemispheric predominance. The STG initiates acoustic analysis of auditory stimuli at approximately 40 ms. Processed auditory information is directly transferred to sPCG and IFG at approximately 160 ms. sPCG initiates subvocal rehearsal for short-term buffering of auditory information to be used in guiding vocal reproduction; thereby feedback signals are transferred to STG. By interacting with the PCG, the IFG exerts higher-order analysis of sequential auditory stimuli. Larger areas of the PCG execute overt vocalization by receiving feed-forward signals from the aforementioned cortical structures. We believe that our model contributes to a better understanding of the mechanisms of speech impairment following cortical injury, including that of the right hemisphere.

Figure 9. Our proposed model of the articulatory loop system.

Feedforward flows from STG are dominant following sound presentation. Distant cortical regions frequently send feedback signals during sound listening and motor preparation. Inward flows to both sPCG and iPCG are dominant immediately prior to response.

Supplementary Material

Fig S1 & Table S1

Supplementary Figure S1 (will be available on the website alone): Automatic parcellation of the cerebral cortex.

Video S1

Video S1 (on the website alone): High-gamma augmentation during the two auditory tasks in patient #5. The percent change in high-gamma amplitude is presented on the individual's brain surface image (left image: ‘sound-listening and humming’ task. right: ‘sound-listening alone’ task). Animation movies are presented time-locked to the onset of initial tone presentation. High-gamma augmentation involved the left STG, during both tasks, immediately following the onsets of the initial (0 ms) and second tone stimuli (+500 ms). Specifically during ‘sound-listening and humming’ task, high-gamma augmentation began to involve the sPCG by +300 ms and remained enhanced during and after the sound presentation. The subsequent movie is presented time-locked to the onset of the humming response. High-gamma augmentation intensely involved the iPCG around the onset of the humming response.

Video S2

Video S2 (on the website alone): High-gamma augmentation during the two auditory tasks in patient #3. The percent change in high-gamma amplitude is presented (left image: ‘sound-listening and humming’ task. right: ‘sound-listening alone’ task). Animation movies are presented time-locked to the onset of initial tone presentation. High-gamma augmentation intensely involved the right STG sites, during both tasks, immediately following the onsets of the initial (0 ms) and second tone stimuli (+500 ms). The subsequent movie is presented time-locked to the onset of the humming response. High-gamma augmentation intensely involved the PCG sites immediately prior to and during the response.

Video S3

Video S3 (on the website alone): Pooled averaged ERC neural flows. Data time-locked to stimulus onset. Both width and color (thin-yellow: weak; thick-red: strong) of arrows represent intensity of causal flows. Linear arrow: neural flow between regions of interest (ROIs). Circular arrow: flow within each ROI. ERC flows, averaged across hemisphere, are presented on the left hemisphere template.

Video S4

Video S4 (on the website alone): Pooled averaged ERC neural flows. Data time-locked to response onset.

Video S5

Video S5 (on the website alone): Cortico-cortical evoked potentials (CCEPs) in patient #3. The amplitude of CCEP is presented, sequentially as a function of time, at each electrode site on the individual's brain surface image (Fukuda et al., 2008). The animation movies include CCEPs following stimulation of (i) STG, (ii) sPCG, (iii) IFG, and (iv) iPCG pairs.

Highlights.

• -Tone sounds elicit early high-gamma (HG) augmentation in the precentral gyrus.
• -Stimulation of precentral HG sites in either hemisphere impairs vocalization.
• -Connectivity analyses reveal dynamic, reciprocal interactions across the articulatory loop network.