Bidirectional Connectivity Between Broca's Area and Wernicke's Area During Interactive Verbal Communication

Abstract

Aim:

This investigation aims to advance the understanding of neural dynamics that underlies live and natural interactions during spoken dialogue between two individuals.

Introduction:

The underlying hypothesis is that functional connectivity between canonical speech areas in the human brain will be modulated by social interaction.

Methods:

Granger causality was applied to compare directional connectivity across Broca's and Wernicke's areas during verbal conditions consisting of interactive and noninteractive communication. Thirty-three pairs of healthy adult participants alternately talked and listened to each other while performing an object naming and description task that was either interactive or not during hyperscanning with functional near-infrared spectroscopy (fNIRS). In the noninteractive condition, the speaker named and described a picture-object without reference to the partner's description. In the interactive condition, the speaker performed the same task but included an interactive response about the preceding comments of the partner. Causality measures of hemodynamic responses from Broca's and Wernicke's areas were compared between real, surrogate, and shuffled trials within dyads.

Results:

The interactive communication was characterized by bidirectional connectivity between Wernicke's and Broca's areas of the listener's brain. Whereas this connectivity was unidirectional in the speaker's brain. In the case of the noninteractive condition, both speaker's and listener's brains showed unidirectional top-down (Broca's area to Wernicke's area) connectivity.

Conclusion:

Together, directional connectivity as determined by Granger analysis reveals bidirectional flow of neuronal information during dynamic two-person verbal interaction for processes that are active during listening (reception) and not during talking (production). Findings are consistent with prior contrast findings (general linear model) showing neural modulation of the receptive language system associated with Wernicke's area during a two-person live interaction.

Impact statement

The neural dynamics that underlies real-life social interactions is an emergent topic of interest. Dynamically coupled cross-brain neural mechanisms between interacting partners during verbal dialogue have been shown within Wernicke's area. However, it is not known how within-brain long-range neural mechanisms operate during these live social functions. Using Granger causality analysis, we show bidirectional neural activity between Broca's and Wernicke's areas during interactive dialogue compared with a noninteractive control task showing only unidirectional activity. Findings are consistent with an Interactive Brain Model where long-range neural mechanisms process interactive processes associated with rapid and spontaneous spoken social cues.

Introduction

Natural and purposeful verbal interaction is generally considered a foundation of human social and cultural activities, and has been an active topic of investigation for over a century (Hickok and Poeppel, 2007; Keenan et al., 1977). However, the neural dynamics necessary to understand the interactive process of talking and listening between dyads has emerged as a central question in the new “neuroscience of two.”

We have previously demonstrated that neural responses during talking and listening with and without interaction revealed increased activity during listening (Wernicke's area), but not talking (Broca's area), during the interactive condition (Hirsch et al., 2018). A canonical human language model includes specialized units for transmission of speech in the left inferior prefrontal region (Broca's area) and for reception/comprehension of auditory signals in the left temporoparietal region (Wernicke's area; Price, 2012).

These frontal and temporoparietal regions constitute essential speech-processing networks interfacing the articulatory motor and phonological sensory functions (Hickok and Poeppel, 2007). It is assumed that the natural flow of live and interactive verbal communication includes the sending and receiving of information in rapid succession that includes simultaneous functions such as motor planning, articulation, auditory perception, and comprehension of the spoken words.

However, it is not known if these pathways are uni- or bidirectional such as might occur in top-down or bottom-up pathways or if these processes occur in parallel. In the former case, a directional connectivity analysis would determine the information flow between the canonical language areas and distinguish between neural processes during interactive and noninteractive communication.

We applied Granger causal measures (Barnett and Seth, 2014; Seth et al., 2015) to investigate the signature of directional connectivity structures across Broca's and Wernicke's language systems during interactive and noninteractive verbal communication. Granger causality determines the effective or causal relationship between pairs of signals. Since its first application to local-field potentials (Bernasconi and KoÈnig, 1999), Granger causality analysis has been utilized to determine the leader/follower relationship in pairs of functional brain signals such as functional magnetic resonance imaging (fMRI), electrophysiological data sets (Seth et al., 2015), and functional near-infrared spectroscopy (fNIRS; Zhang et al., 2014; Im et al., 2010).

This analysis was performed on data acquired to test the hypothesis that neural activity during interactive dialogue would be greater than (and possibly different from) neural activity during the same task but performed without interaction (De Jaegher et al., 2016; Di Paolo and De Jaegher, 2012; Hirsch et al., 2018).

The present study used a two-person hyperscanning paradigm utilizing fNIRS as the neuroimaging technology to acquire brain signals from naturally interacting pairs of individuals (dyads). This neuroimaging method is based on changes in spectral absorbance of both oxyhemoglobin (OxyHb) and deoxyhemoglobin (deOxyHb) detected by surface-mounted optodes (Cui et al., 2011; Ferrari and Quaresima, 2012; Scholkmann et al., 2014; Strangman et al., 2002; Villringer and Chance, 1997).

These hemodynamic signals serve as a proxy for neural activity similar to fMRI blood-oxygenation level-dependent signals (Boas et al., 2014; Ferrari and Quaresima, 2012; Noah et al., 2015; Ogawa et al., 1990). One advantage of fNIRS is the tolerance to limited head movement as the optical probes are tightly fixed on the surface of the head (Ono et al., 2014; Zhang et al., 2017). This experimental setup enables an open and naturalistic experimental environment that approximates the actual situation of interpersonal verbal interaction.

Although understanding the underlying neural circuitry associated with human speech production and perception has been an intense focus of neuroscience research, progress has been challenged by the absence of a neural imaging technology that enabled neural observations during speaking tasks. For example, conventional fMRI neuroimaging limits investigations to single subjects who perform proxies of speaking tasks using “silent” or “internal” speech while lying in the bore of a loud and confining scanner. In addition to the contraindication for actual speaking due to head movements, the opportunities for natural dyadic interaction are vanishingly small from within a single MRI bore. However, adaptations of fNIRS for dyadic interactions have partially resolved these conventional technical obstacles, and participants are able to face each other in upright and natural conditions and to converse in a typical dialogue.

Recent investigations of the underlying neural systems engaged during live and natural verbal exchanges between dyads confirm the impact of this breakthrough technology. Not only do these “real-life” two-person verbal interactions confirm prior findings of canonical language areas and their basic functions, the models have also been expanded to embrace constructionist approaches that integrate the language systems with social, perceptual, and cognitive systems (Descorbeth et al., 2020; Hirsch et al., 2021).

These naturalistic language investigations are guided by theoretical frameworks that include the functional neural anatomy of language during dynamic and interactive tasks and the functional neural anatomy of live social interactions (Hasson and Frith 2016; Hasson et al., 2012; Redcay and Schilbach, 2019; Schilbach et al., 2013). The emerging fused theoretical frameworks for interactive functions and language functions have focused on temporal/parietal mechanisms showing specializations for interactive functions such as real face viewing (Kelley et al., 2021; Noah et al., 2020) and real interactive language exchanges (Hirsch et al., 2018; Jiang et al., 2012). However, beyond the receptive and comprehensive components of language and interaction, frontal mechanisms are thought to underlie productive functions.

One prominent model of the cortical organization of speech processing proposes a “dual stream” that segments processing of comprehension and acoustic signals (Hickok and Poeppel, 2007). Using Granger causality within this context of an interactive and live language task, we expand a limited version of the “dual-stream” model between the canonical language regions. The hypothesis is that the interactive condition will be associated with bidirectional information streams reflecting the increased functional load of dynamic and unscripted talking and listening.

Participants were seated across a table from each other during the experiment with an occluder between the participants that prevented viewing the face of their partner to exclude confounds of face processing. Talking and listening turns were structured and cued as 15-sec epochs for two conditions: object naming and description with interaction and without. See the Materials and Methods section below and the previous study (Hirsch et al., 2018).

In applying Granger causality analysis to the fNIRS signals recorded in the real-world settings, we performed realistic simulations to confirm the detectability of directional interactions. The region-wise, intrinsic variation in hemodynamic delay has been raised as a potential confound that may alter the estimated temporal relationship between remote hemodynamic signals measured with fMRI (Deshpande et al., 2010; Schippers et al., 2011; Smith et al., 2012). We compared the causal structures estimated from simulated hemodynamic signals that were sampled with a range of time resolutions, a major factor determining the detection accuracy of Granger causal relationship (Deshpande et al., 2010), to investigate whether the superior time resolution of fNIRS could improve the detectability of directional connectivity.

In the present study, we hypothesize that interactive verbal communication will alter the directional connectivity of canonical language areas in the listener's brain as suggested by previous contrast findings that showed interactive effects during listening and not talking (Hirsch et al., 2018). This hypothesis also takes into account the active effort of the listener during interaction as simultaneous functions of interpretation, coding, and response planning are components of live interaction. Neurophysiological evidence supporting this specific hypothesis would further contribute to the more general Interactive Brain Hypothesis that proposes a broad theoretical framework for two-person social neuroscience (De Jaegher et al., 2016; Di Paolo and De Jaegher, 2012). This hypothesis predicts that interpersonal social behavior and cognitive processes involve specific neural mechanisms of two-way interactions within dyads.

Materials and Methods

Participants

All participants provided written consent according to the guidelines established by the Yale Institutional Review Board, as specified in protocol number 1501015178. The fNIRS data used in this study were taken from our companion study (Hirsch et al., 2018), which focused on intensity changes of neural activity and two-brain synchronizations during verbal social interactions, but not on the directional connectivity within-brains. Thirty-four pairs of healthy adult participants were included and no individual participated in more than one dyad.

Since fNIRS signal quality could be easily affected by individual anatomical characteristics such as skull thickness and fat deposits (Cui et al., 2011; Okada and Delpy, 2003; Owen-Reece et al., 1999), participants were initially screened before the experiment for signal quality using a right-hand finger-thumb-tapping task and passive viewing of a reversing checkerboard visual stimulus. Participants who demonstrated expected fiducial response patterns in the left sensorimotor area and occipital visual area, respectively, were eligible to participate. One dyad was excluded from the analysis as the fNIRS signals from one partner did not meet the stationarity criteria required for Granger causality analysis. We therefore analyzed fNIRS data of 33 dyads (n = 66, mean age: 24.8 ± 6.5 years, 56% female, 97% right-handed; Oldfield, 1971).

fNIRS data acquisition

Participants sat in a chair ∼140 cm across a table from each other with 42-channel fNIRS probes (Shimadzu LABNIRS, Kyoto, Japan) placed on each of their heads using an optode cap. Participants were instructed to look at a screen positioned at eye height and ∼50 cm in front of each of them throughout the experiment. Participants did not have a view of their partner. Video and audio recordings were acquired on all sessions and confirmed compliance with instructions. Figure 1 illustrates the distribution of 42 channels over both hemispheres of the scalp. Channel distances were based on cap size to fit the head of the subject, and the channel separations were either 2.75 cm for small heads or 3.0 cm for large heads (Noah et al., 2015). Relative concentration changes of OxyHb and deOxyHb were continuously measured with a sampling frequency of 37 Hz (temporal resolution of 27 ms) using a standard protocol described elsewhere (Hirsch et al., 2017, 2018).

FIG. 1.

fNIRS channel layout. Forty-two fNIRS channels were assigned with each participant over both hemispheres of the scalp. Channel distances were based on cap size to fit the head of the subject, and the channel separations were either 2.75 cm for small heads or 3.0 cm for large heads (Dravida et al., 2017). The optode positions were digitized from each participant and the average channel positions of all participants were projected onto the normalized brain image. fNIRS, functional near-infrared spectroscopy.

The anatomical locations of optodes were determined for each participant immediately after the experiment. The standard head landmarks, including inion; nasion; Cz in the international 10–20 layout; and left and right tragi, were acquired with optode locations using a Patriot 3D Digitizer (Polhemus, Colchester, VT). Linear transform techniques were applied as previously described (Eggebrecht et al., 2012; Ferradal et al., 2014; Okamoto and Dan, 2005) to obtain Montreal Neurological Institute (MNI) coordinates for the fNIRS channels and their corresponding anatomical locations using the statistical parametric mapping for near-infrared spectroscopy (NIRS-SPM) software (Ye et al., 2009) with MATLAB (MathWorks, Natick, MA).

Interactive and noninteractive task

The verbal task was based on the well-established Object Naming and Description task frequently used for clinical applications using fMRI where mapping of the human language system is the goal for neurosurgical planning purposes (Hart Jr et al., 2007; Hirsch et al., 2000). However, unlike in fMRI applications, the fNIRS paradigm allows participants to actively speak instead of silently rehearsing the response (Zhang et al., 2017).

The neural regions observed using this task have been validated by intraoperative recordings of both Broca's and Wernicke's regions (Hirsch et al., 2000) and motivate the application of this task in the novel investigation of live two-person verbal communication. The well-established regions activated by this task are consistent with classical models of overt speech, including Broca's and Wernicke's regions located in inferior frontal and superior temporal gyri, respectively (Geschwind, 1974).

The stimulus pictures used for the two-person adaptation of the Object Naming and Description task were of common and unrelated objects selected for clarity and familiarity. The time series is shown in Figure 2a. Participant roles of “speaker” and “listener” switched every 15 s when a new picture was automatically presented on the two screens. This exchange between talking and listening continued for 3 min and was repeated twice. The onset of each block was cued by the appearance of a new picture viewed by both participants. See Hirsch and colleagues (2018) for further details.

FIG. 2.

Diagram of experimental design (a) including original (b) and shuffled (c, d) fNIRS signals for GC analysis. Dyads of participants performed an object naming and description task alternately. Background colors of gray, green, and yellow indicate corresponding epochs. The Granger causality was calculated with original and shuffled data. GC, Granger causality.

The present study consisted of noninteractive (monologue) and interactive (dialogue) person-to-person communication. In the “monologue” condition, the “talking participant” named and described the picture without intention to communicate with the partner. The “listening participant” heard the narrative, but did not respond. The “dialogue” condition was identical except that each speaker and listener intended to communicate with the partner. The speaker responded to the comments of the previous speaker before describing the new picture, and the listener heard both parts and responded during the upcoming epoch when he/she was the speaker.

Participants were instructed to change topics from “your partner's comments about the previous picture” to “comments about your new picture” near the middle of the epoch. The exact time of the topic switch was not specified to assure that communication flowed as naturally as possible. There was no evidence for a difference in the number of words spoken during the monologue and dialogue conditions, and auditory recordings confirmed compliance.

fNIRS data preprocessing

Although presented previously, these methods are briefly described here to provide a self-contained report. Baseline drift of raw fNIRS data was removed using the wavelet detrending method with a default setting specified in NIRS-SPM toolbox provided by Ye and colleagues (2009). Global systemic effects due to blood pressure changes and respiration (Kirilina et al., 2012; Tachtsidis and Scholkmann, 2016) were further removed from the baseline-corrected fNIRS signal using a spatial filter method based on principle component analysis (Zhang et al., 2016, 2017).

The following fNIRS signal analyses were applied to the preprocessed data set described as above. Since fiducial markers previously acquired by fMRI associated with talking and listening task have also been observed using the fNIRS deOxyHb signal (Hirsch et al., 2018; Scholkmann et al., 2013; Zhang et al., 2017), we analyzed deOxyHb signals with the following general linear model (GLM) analysis for determining cortical regions showing task-related activation.

Determination of regions of interest using GLM

Group analysis contrasting regional brain activity between talking and listening tasks was performed with a beta-value distribution of the preprocessed fNIRS deOxyHb data estimated by a GLM. This analysis was used to localize the talking and listening related regional brain activity. Briefly, beta-values were calculated with the GLM approach for each channel, and the distribution of the beta-values on the 3753 2 × 2 × 2 mm voxels on the cortical surface (depth up to 1.8 cm) of the standard MNI brain was estimated with linear interpolation using functions provided by NIRS-SPM.

The voxel-wise approach used computational tools conventionally applied to fMRI, and provided the most precise spatial locations of activity using fine-grained interpolation between the channels. Second-level group analysis was performed using SPM8 with the voxel-wise data sets. Findings based on this approach are reported at p < 0.01 (uncorrected) for descriptive representations of anatomical correlates that are consistent with the findings established for the neurosurgical planning applications. The relatively liberal statistical threshold was set to define locations of the regions of interest (ROIs) that could be collectively applied to most of the participants for further effective connectivity analysis described below.

Effective connectivity analysis using Granger causality

We applied the following Granger causality analysis to the deOxyHb data, in which the expected talking and listening related activity patterns within Broca's and Wernicke's areas in the left hemisphere were obtained (Hirsch et al., 2018). Granger causality is a connectivity analysis algorithm that can explore the directionality of leader/follower relationships between pairs of signals (Barnett and Seth, 2014). The Granger causality analysis calculates the autoregressive model of the time-course of the data, assuming that the data are stationary and the present state of the data could be estimated by the history of the data, as shown in Eq. (1).

Here p is the model order (how much past is referred to estimate present state) and ${\epsilon }_x\left(t\right)$ is residual. The main idea of Granger causality from data Y to data X, $G_{Y\to X}$, is how much the estimate of $X\left(t\right)$ could be improved if the regression model includes the past of $Y\left(t\right)$ or not. Therefore, we calculate two regression model Eqs. (1) and (2) and compare the estimation error (residual) to determine the Granger causality from Y to X, as in Eq. (3).

$$G_{Y\to X}\equiv ln\frac{\epsilon {\prime }_x\left(t\right)}{{\epsilon }_x\left(t\right)}$$ (3)

This two-variable model can be expanded to a multivariate model, in which there are more than two data sets to determine the conditional effective connectivity between each pair under the common contribution of the past of other data sets (Barnett and Seth, 2014). We used the multivariate Granger causality (MVGC) Toolbox (Barnett and Seth, 2014) to determine MVGC relationships between each pair of hemodynamic signals measured from two canonical language systems of Broca's and Wernicke's areas from two persons. The strengths of effective connectivity between all pairs of two language systems' responses of two persons were determined as Granger causality indices according to the toolbox output for each pair of participants, which were further assigned to the group-level statistical analysis between original and shuffled data sets.

To determine the hemodynamic signal time-courses of the two canonical language systems, the deOxyHb data of specific channels were selected based on the individual anatomical locations of channels in the normalized MNI space. We selected the channels localized in the left Broca's area as channels of interest of motor speech area. The channels selected for Wernicke's area included those localized in the left supramarginal and angular gyri. If there was no channel localized in these two motor language areas due to the sparse arrangement of the optodes, we included the channels localized to the left superior temporal gyrus. The deOxyHb signals in each ROI (Broca's and Wernicke's areas) were averaged within participants if there were multiple channels selected. The average number of channels selected for each ROI was 1.95 (ranging from 1 to 3) and 2.19 (ranging from 1 to 4) in Broca's and Wernicke's areas, respectively.

The 180-s single run of fNIRS data averaged within the ROI was divided into twelve 15-s blocks, in which one participant was talking and the other was listening (Fig. 2a). Data from two runs were combined, and therefore, there were 12 extracted data sets for the same talking/listening condition (such as “participant 1 talks and participant 2 listens”) for each pair and each monologue/dialogue condition.

The data were further downsampled to 3.7 Hz according to our previous fNIRS connectivity studies (Hirsch et al., 2017, 2018) to avoid spurious connectivity measures related to higher frequency noise component such as heart beats. Model order for Granger causality analysis was fixed to 15 (equivalent to 4.05 s) based on the results of model order estimation using the Akaike Information Criteria, implemented in the toolbox, and the previous findings of delayed synchrony of fNIRS signals between the speaker and listener during natural verbal communication (Liu et al., 2017). To confirm the validity of the Granger causality model, we calculated the square summability and stability (covariance stationarity) of the coefficients of an estimated autoregressive model (Barnett and Seth, 2014). Residual covariances and 1-lag residual covariances are also checked for positive-definitiveness to exclude ill-conditioned autoregressive modeling. We used the default criteria for these validations implemented in the MVGC Toolbox.

As mentioned in the Participant section, the data of one dyad were excluded from the analysis as the fNIRS signals from one partner did not meet the stationarity criteria. The data were demeaned and whitened before the Granger causality analysis to avoid potential confound caused by different power of signals in the ROIs (Bastos and Schoffelen, 2016).

The Granger causality between within-brain and across-brain canonical language systems was calculated with original, surrogate, and shuffled data, in which the latter two were adopted to validate the statistical significance of causal strength found in the original data. In case of analyzing original data, the averaged deOxyHb signals from Broca's area were paired with those from Wernicke's area of the same block (Fig. 2b). In case of analyzing surrogate data, the original data were phase-randomized before Granger causality analysis, while the origin of signals and order of blocks were preserved for each individual. The Granger causality indices derived from the surrogate data serve as a valid ‘null case’ for statistical inference (Prichard and Theiler, 1994).

Strengths of the causal relationships of the original data are expected to be larger than those of the surrogate data if the time series changes of the hemodynamic responses in one language area influence those in another. The empirical p value of the Granger causality indices derived from original data was determined from distribution of 500 permutated surrogate data sets.

The statistical inference of the original Granger causality indices was further tested with two types of shuffled data sets, where the phase information of the signal was preserved, but the signals from different blocks were paired in either a within-brain or an across-brain manner. In case of analyzing within-brain Granger causality in shuffled data, signals from Broca's area were paired with those from Wernicke's area taken from different blocks of the same individual (Fig. 2c). In case of analyzing across-brain Granger causality in shuffled data, the pair of signals from Broca's and Wernicke's areas was preserved in the same block with each individual, while those were combined with different blocks of their partners (Fig. 2d). We used these two shuffled data sets to manipulate either within-brain or across-brain combinations of the data structure while preserving the other combination relationship since MVGC takes the contribution of the causal relationships between all possible combinations of the data subsets into account.

The Granger causality indices from 265 combinations for each shuffled data set, which are the all possible combinations of shuffled order without overlap with the original combination of blocks for each run, were averaged and compared with those obtained from original data using a paired t-test. Strengths of causal relationship of original data are expected to be larger than those of shuffled data if the hemodynamic response patterns of communicating brain regions covary with the contents and compositions of the spoken words that are specific to the corresponding block. We confirmed the statistical independence of the surrogate and shuffled data sets, indicating that the two statistical analyses are based on different null hypotheses described above (see Supplementary Data for the details and Supplementary Figs. S1 and S2). Therefore, we defined a valid causal relationship as one that met both statistical criteria from surrogate and shuffled data analyses. We defined p value <5% as a statistically significant difference.

Validation of Granger causality model with simulated fNIRS signals with potential hemodynamic delay

We performed realistic simulations to investigate whether the directional interactions detected in the present data could be identified from fNIRS data despite the possible hemodynamic confounds. We assumed four ROIs in two brains in which directional connectivity was assumed independently in each brain ($\left\{x_1,\dots ,x_4\right\}$ in Fig. 3a). The autocorrelated time series data were first modeled from x₁ to x₂ and from x₃ to x₄ using Eq. (4) as in Schippers and associates (2011), followed by adding the receiver signals with a fixed neuronal transmission delay (4.05 s, gray lines at x₂ and x₄ in Fig. 3b). The duration of neuronal delay was empirically determined from the model estimation result of the original fNIRS data set. The causal influence between the signals was set to 0.15 so that the mean Granger causality index estimated from the simulation data set would be comparable with that obtained from the original data set.

FIG. 3.

Simulation summary investigating detectability of Granger causal relationship with fNIRS data. (a) Causal relationship was assumed between two ROIs in each of two brains. (b) Example of simulated fNIRS signals. Signals at the receiver ROIs (gray lines in x₂ and x₄) were assumed to have neuronal delay from those at the sender ROIs (x₁ and x₃) for information transmission. The signals at the receiver ROIs were further shifted back to opposing direction of the neuronal delay (black lines in x₂ and x₄) to simulate the worst-case scenario leading to the underestimation of causal relationship. Gaussian noise was further added to these simulated signals for the GC analysis. (c) Heat maps showing the ratio of correctly (blue panels) or incorrectly (orange panels) detected Granger causal relationship with varied sampling Fs of the data. The number in each cell indicates the percentage of detecting valid GC among 100 trials of simulation. At the sampling rate of the original fNIRS GC analysis (Fs = 3.7 Hz), the assumed directional connectivity was perfectly detected, while the false detection was scarce. (d) Changes of mean detection rates for true and false connectivity with different sampling frequencies. The mean detection rates were calculated as the average occurrence ratio of valid GC over true (from x₁ to x₂ and from x₃ to x₄: filled circles) or false (the other pairs of ROIs: open triangles) combinations of signals. Fs, frequencies; ROIs, regions of interest.

$$\begin{array}{cc}\left[\begin{array}{c}\begin{array}{c}{x}_1\left(i\right)\\ x_2\left(i\right)\\ \end{array}\\ \begin{array}{c}{x}_3\left(i\right)\\ x_4\left(i\right)\\ \end{array}\\ \end{array}\right]& =\left[\begin{array}{cccc}0.9& 0& 0& 0\\ 0.15& 0.9& 0& 0\\ 0& 0& 0.9& 0\\ 0& 0& 0.15& 0.9\\ \end{array}\right]\left[\begin{array}{c}\begin{array}{c}{x}_1\left(i-1\right)\\ x_2\left(i-1\right)\\ \end{array}\\ \begin{array}{c}{x}_3\left(i-1\right)\\ x_4\left(i-1\right)\\ \end{array}\\ \end{array}\right]\\ & +\left[\begin{array}{c}\begin{array}{c}{\epsilon }_1\left(i\right)\\ {\epsilon }_2\left(i\right)\\ \end{array}\\ \begin{array}{c}{\epsilon }_3\left(i\right)\\ {\epsilon }_4\left(i\right)\\ \end{array}\\ \end{array}\right]\end{array}$$ (4)

Here $x_n\left(i\right)$ and ${\epsilon }_n\left(i\right)\left(n=1,\dots ,4\right)$ represent fNIRS signal and noise, respectively, at ROI $n$ at time i.

The variations in hemodynamic delay were introduced by convoluting the autocorrelated time series data with two types of hemodynamic response function, which are different in onset peak time. We assumed the hemodynamic delay of 2.5 s, its empirical upper limit length (Deshpande et al., 2010; Handwerker et al., 2004), opposing the direction of neural information flow (black lines of x₂ and x₄ in Fig. 3b). The hemodynamic response functions were generated by built-in function of SPM8 software (spm_hrf.m). In this case, the signals of the sender and the receiver are temporally overwrapped and may cause underestimation of sender-to-receiver causal relationship. We considered this worst-case scenario rather than estimating the hemodynamic delay from the present fNIRS data set, which may be affected by the potential trial-to-trial variation in the hemodynamic response pattern due to the nature of natural speaking and listening task. Gaussian noise of 100% signal strength was added to represent physiological noise in the hemodynamic response.

Since detection accuracy of Granger causal relationship in fMRI data strongly depends on the sampling rate (Deshpande et al., 2010), we investigated detection rates of true and false connectivity with varied sampling rates of 0.25, 0.33, 0.5, 1, 2, 3, and 3.7 Hz, including the sampling rate of conventional fMRI data (0.33–1 Hz) and that of downsampled fNIRS data presently used for original Granger causality analysis (3.7 Hz). The significance of Granger causality strength was validated by calculating z-score of Granger causality index with the original simulation data relative to the empirical null distributions of Granger causality indices obtained from 1000 times of Granger causality analysis with phase-randomized data as surrogate data sets. The z-score larger than three was recognized as a significant Granger causal relationship (Deshpande et al., 2010). We performed 100 z-score calculations and determined the detection rate of Granger causality for all possible combinations of ROIs and sampling rates.

Results

GLM analysis

Figure 4 and Table 1 demonstrate the results of voxel-wise GLM contrast maps showing cortical activity between talking and listening in dialogue conditions. The representative contrast map for the dialogue condition confirmed the findings in the canonical language regions, as reported previously (Hirsch et al., 2018) in accordance with the conventional models of language processing (Price 2012).

FIG. 4.

GLM contrast map of deOxyHb signals between talking and listening tasks in dialogue condition. Red/yellow indicates regions with larger activity in talking relative to listening, and blue/cyan indicates those with larger activity in listening relative to talking. Please also refer to Table 1 for detailed anatomical locations of the activities. deOxyHb, deoxyhemoglobin; GLM, general linear model.

Table 1.

Voxel-Wise General Linear Model Contrast Comparisons (Deoxyhemoglobin Signals, Contrast Threshold p < 0.01)

Contrast	Peak voxel
	MNI coordinates [mm]			t value	Anatomical regions in area	BA	Prob.
	X	y	z
Dialogue
Talk > listen	−56	2	40	3.31	Premotor and supplementary motor cortex	6	0.71
					Dorsolateral prefrontal cortex	9	0.22
	−66	−6	10	3.25	Superior temporal gyrus	22	0.34
					Premotor and supplementary motor cortex	6	0.21
					Subcentral area	43	0.17
	−62	4	−2	2.98	Superior temporal gyrus	22	0.40
					Middle temporal gyrus	21	0.35
Listen > talk	−66	−48	18	−3.88	Superior temporal gyrus	22	0.56
					Supramarginal gyrus part of Wernicke's area	40	0.31
	−70	−36	−8	−3.01	Middle temporal gyrus	21	0.81
					Superior temporal gyrus	22	0.12

Clusters on the left hemisphere are shown.

BA, Brodmann area; MNI, Montreal Neurological Institute; Prob., probability of inclusion.

Increased activity was found in Broca's area, the left inferior frontal gyrus, and the ventral premotor area in case of talking relative to listening task in both conditions, confirming the involvement of the articulatory network during talking. Increased activity in the left superior/middle temporal gyrus and supramarginal gyrus was observed in case of listening relative to talking, supporting the involvement of the speech perception and comprehension centers (posterior middle temporal and parietal areas) during listening. These patterns were consistent with the results obtained from the monologue and combined monologue and dialogue conditions (p < 0.05). The ROIs were selected based on these results for the following effective connectivity analysis using Granger causality. We specifically focused on Broca's area and Wernicke's area considering their fundamental role of the motor/sensory interaction (Hickok and Poeppel, 2007; Price, 2012) in the verbal communication.

Validation of Granger causality model

Changes in detection rates for true and false connectivity of the simulation data are shown in Figure 3c and d. As expected, the true connectivity was more likely to be detected with a higher sampling rate. The causal relationship was completely determined from simulated hemodynamic data with a sampling rate of 1 Hz or above. Although the false detection rate was negligible (<5% of occurrence) at any combination of the ROIs and sampling frequency, the number of connections showing false-positive connectivity was also decreased with higher time resolution of the data (>0.5 Hz). These results demonstrate that the Granger causality analysis could reliably interpret the directional interactions identified in the original data analysis under present conditions of assumed neuronal delay and sampling rate of fNIRS data.

Granger causality connectivity

Changes in the effective connectivity were observed between the Broca's and Wernicke's areas relative to the surrogate or shuffled control data set within single brain, but not for across brains, in both cases of “monologue” and “dialogue” conditions. Table 2 shows the summary of Granger causality strengths derived from original, surrogate, and within-brain shuffled data sets. The original fNIRS signals showed valid Granger causality indices relative to surrogate signals bidirectionally between Broca's area and Wernicke's area in the listener's brain and unidirectionally from Broca's area to Wernicke's area in the speaker's brain regardless of the conditions of communication. In comparison with the shuffled data sets, original fNIRS signals showed significantly larger Granger causality indices bidirectionally between Broca's and Wernicke's areas in the listener's brain in “dialogue” condition, while only unidirectional connectivity from Broca's area to Wernicke's area was confirmed in the “monologue” condition.

Table 2.

Strengths of Within-Brain Effective Connectivity (Granger Causality Indices) Between Two Language Systems

	Listener		Speaker
	Broca to Wernicke	Wernicke to Broca	Broca to Wernicke	Wernicke to Broca
Dialogue
Original	0.0370 ± 0.0016a,b	0.0398 ± 0.0021a,b	0.0433 ± 0.0038a,b	0.0379 ± 0.0018
Surrogate	0.0349 ± 0.0014	0.0350 ± 0.0016	0.0357 ± 0.0016	0.0355 ± 0.0016
Shuffled	0.0320 ± 0.0025	0.0332 ± 0.0017	0.0336 ± 0.0037	0.0335 ± 0.0015
Monologue
Original	0.0454 ± 0.0026a,b	0.0378 ± 0.0016a	0.0430 ± 0.0037a,b	0.0355 ± 0.0014
Surrogate	0.0350 ± 0.0015	0.0347 ± 0.0015	0.0364 ± 0.0016	0.0361 ± 0.0016
Shuffled	0.0332 ± 0.0025	0.0338 ± 0.0016	0.0330 ± 0.0036	0.0331 ± 0.0013

Values show mean ± standard error for original and shuffled data sets and mean ± standard deviation for surrogate data sets, respectively. We used standard deviation to describe the distribution of surrogate data since the standard error is affected by the arbitrarily determined number of generated data sets.

^a Significantly larger than surrogate control.

^b Significantly larger than shuffled control (p < 0.05).

A unidirectional causal relationship from Broca's area to Wernicke's area was found in the speaker's brain regardless of the conditions of communication. These results indicate that the detected effective connectivity networks are characterized by both the time series and the context-dependent structure of the cortical activities between the motor and sensory language areas.

Figure 5 summarizes effective connectivity changes between language regions of listeners and speakers' in “dialogue” and “monologue” conditions. Directional connectivity relationships that meet both criteria from surrogate and shuffled controls are illustrated. Both listener and speaker showed a statistically significant increase in directional connectivity from Broca's area to Wernicke's area regardless of the conditions. On the contrary, an increase in the directional connectivity from Wernicke's area to Broca's area was found only in the listener's brain during the dialogue condition.

FIG. 5.

Schematic representation of Granger causal connectivity between canonical speech areas during two-person dialogue with interaction (a, b) and monologue talking and listening (c, d). The directional connectivity that showed statistical inference over both surrogate and shuffled controls was illustrated in the figure. For the speaking task (b, d), the observed direction of connectivity is from Broca's area to Wernicke's area regardless of existence or inexistence of interaction. However, for the listening task, the connectivity is bidirectional including both bottom-up and top-down directions under interaction (a), whereas it is unilateral without interaction (c). The nodes were visualized with the BrainNet Viewer (Xia et al., 2013).

Discussion

We apply a novel brain imaging technology, fNIRS, to capture the dynamics of cortical signal processing during live two-person verbal communication. Production of speech was characterized by increased activity in Broca's area, located in the left hemisphere prefrontal cortex, and receiving verbal information (listening) was associated with the increased activity in the left temporoparietal region, Wernicke's area.

The directional connectivity analysis using Granger causality between the two language areas revealed that both listening and speaking showed augmented unidirectional connectivity from Broca's area to Wernicke's area regardless of the existence of interaction. In addition, however, interactive verbal communication was characterized by bidirectional cortical connectivity between these two language areas in the listener's brain, while the structure of effective connectivity in the speaker's brain remained unchanged as in the noninteractive condition.

This finding is consistent with a dual-stream model for language processing (Hickok and Poeppel, 2007) within the context of a natural interactive task. Bidirectional anatomical connectivity between these two language areas has previously been demonstrated by studies using diffusion tensor MRI (Catani and Jones, 2005), electrocorticography (ECoG; Matsumoto et al., 2004), and lesion studies (Hickok and Poeppel, 2007). Our results further confirm this effective connectivity between these two language areas in the novel application of live, verbal partnered communication.

This dyadic interaction revealed different dynamics depending on the interactive/noninteractive aspect of the communication. The leader/follower relationship from Broca's area to Wernicke's area was commonly observed in both listener and speaker during interactive as well as noninteractive communication, even though their behavior was different (sending vs. receiving information). In the case of the speaker, activation of Broca's area, which is assumed to play a role in articulating the sentence, is hypothetically supported by a function of Wernicke's area to confirm the spoken sounds. The flow of information corresponds to the previously described earlier gamma band firing pattern in Broca's area related to Wernicke's area observed in presurgical patients who engaged in an interpersonal conversation task under ECoG monitoring (Towle et al., 2008). In the case of listener, activation of Broca's area may arise from the anticipation of the upcoming sentences from the speaker (Pickering and Garrod, 2007), and again which would be confirmed as auditory information processed by Wernicke's area.

Interactive verbal communication further added feedback-style connectivity from Wernicke's area to Broca's area, and may represent additional cognitive activity relating interpreted sentences from the speaker to the sentences that the listener prepares for the forthcoming turn to speak. Indeed, this hypothesis is supported with our previous observation of the enhanced average signal intensity in Wernicke's ROI in the listener's brain during the interactive relative to noninteractive communication (Hirsch et al., 2018). The difference in Granger causality indices between the actual and surrogate/shuffled data set further suggest that hemodynamic responses within these canonical language-related areas were responsive to the conditions of interpersonal interaction.

Our previous functional connectivity analysis using wavelet coherence (Hirsch et al., 2018) found augmented hemodynamic coherence between the superior temporal gyrus (part of Wernicke's area) and the subcentral area of the dyads. These regions are associated with verbal reception and encoding as well as face/mouth motor activity, respectively, during the interactive dialogue condition compared with noninteractive monologue condition. The complementary findings in the present within brain study illustrate the natures of the two connectivity analysis methods.

Coherence analysis used in the cross-brain study determines functional connectivity by timing-synchronized activity of two signals, while Granger causality analysis used in the single-brain analysis determines effective connectivity by calculating the involvement of the history of other signal(s) to estimate the signal of interest. The connectivity found between canonical language areas in Granger causality analysis but not in the coherence analysis may be due to a delay in neural information processing between the motor and sensory language areas.

Some support for this hypothesis is provided by previous ECoG studies utilizing word-repeating tasks that have demonstrated that gamma activity in the frontal language area precedes the actual voice-onset time to speak a word, but that in the temporal language area arises close to the voice-onset time (Towle et al., 2008). Our results suggest that Granger causality analysis is well fit to understand leader/follower relationships of the language areas within a brain, and is a promising analysis tool to investigate the strength of causal relationships depending on the social context such as interactive and noninteractive verbal communication.

Since activity in the subcentral area was also found in the interpersonal communication using a different social modality of eye-contact (Hirsch et al., 2017), it might also play a role in the coordination of social exchanges regardless of the sensory modality. Although the present analysis focused on the canonical language areas and did not analyze causal relationship between these areas and the subcentral area, our results suggest that the multiple neural network activities based on the functional and effective connectivity were involved in the person-to-person language communication. Incorporating these additional ROIs found in the GLM analysis into the functional and effective connectivity analyses could further expand our understanding of the neural definition of ‘interactive’ verbal communication.

Brain connectivity studies of verbal communication including overt speaking tasks have been limited due to the constraints of muscle, electrical, and motion artifacts related to speaking. These artifacts can have deleterious effects on neuroimaging data measured by functional brain imaging techniques such as electroencephalogram and fMRI. A relative tolerance of fNIRS to these artifacts permits more naturalistic experimental environments, including active speaking between two people. However, fNIRS data can be also be affected by systemic blood flow changes that do not represent neural effects. In this experiment, we removed these using a spatial filter method (Zhang et al., 2016).

Our simulation study clearly showed another important advantage of fNIRS in analyzing a causal relationship from hemodynamic signals. The results confirmed previously reported confounds in detecting Granger causality structures between remote cortical regions derived from differences in a hemodynamic delay (Smith et al., 2012), especially when the sampling frequency of the data is lower than 0.5 Hz in the present case. However, most conventional fNIRS systems, including ours, provide hemodynamic signals with higher sampling rates, and the causal relationship found from the corresponding data was sufficiently accurate. The present results are encouraging in the sense that the higher sampling rate of fNIRS over fMRI could be further utilized to understand causal networks between cortical hemodynamic signals. However, the applicability of Granger causality analysis could also depend on factors other than sampling rates such as neuronal delay, noise level, and causal strength. Validation of connectivity analysis with realistic simulations specific to the research paradigm would contribute to understanding the causal structure of neuronal signals.

It should be noted, however, that the present simulation is valid under the assumed neuronal delay parameter, which was in fact much longer than that required for electrophysiological signal transmission between ROIs. Differences in hemodynamic responses of two regions (up to a couple of seconds) in the direction opposite to the flow of information could effectively flip the direction of the detected connectivity if we assume the neuronal transmission delay to only account for the time required for hard-wired transmission of the electrical signals between regions (i.e., up to tens of milliseconds).

The effect of the hemodynamic delay is likely to be minor relative to the length of the presently estimated neuronal delay, according to fMRI studies showing synchronized activation patterns in the sensory and motor language areas during natural sentence listening tasks (Wildgruber et al., 2002). However, these studies lack precise time resolution, which motivated us to validate our Granger causality model with the fNIRS time resolution under the worst-case scenario of the hemodynamic delay. The longer neuronal delay parameter found in this study suggests additional within-region, local-field information processing for transmission to higher functional processes (Wildgruber et al., 2002).

The statistical power of the GLM analysis was relatively weak despite the large number of participants incorporated. A possible explanation would be that the trial-to-trial variations in the hemodynamic responses in the present natural speech task may not fit well with the general, block-design model of the hemodynamic response functions that we adopted for the activity analysis. Since the participant received a different topic every trial, the content and composition of the spoken sentences would also have trial-by-trial variations. The time-course of the hemodynamic response could have consequent trial-by-trial variations, possibly resulting in smaller beta-value activations under the GLM analysis that assumes constant activity for every trial. Incorporating speech parameters such as the amplitude envelope of the spoken words into the model might improve the statistical power; however, this is beyond the scope of the present study.

The statistical power of GLM analysis does not affect the main findings, since the Granger causality analysis uses raw fNIRS signals without hemodynamic modeling and is capable of detecting causal relationships between ROIs that commonly exist across trials regardless of the variation in the time-course of hemodynamic responses.

Conclusion

We applied directional connectivity analysis to the hemodynamic data of dyads who communicated with and without verbal interaction. Realistic simulations demonstrated the advantage of higher time resolution of fNIRS data in accurately detecting causal relationships between remote hemodynamic signals. Our results show that interactive communication engages an adaptive language network associated with the receptive function of listening. Noninteractive verbal communication (monologue) was associated with unidirectional connectivity from Broca's to Wernicke's areas in both speaker's and listener's brains. On the contrary, interactive verbal communication (dialogue) involved bidirectional language-related flow of information between the two canonical language areas in the listener's brain. Together, directional connectivity analysis reveals bidirectional dynamic flow of neuronal information during the listening epochs of two-person verbal interaction.

Authors' Contributions

J.H. and Y.O. devised the project and the main conceptual ideas. J.A.N., S.D., and X.Z. carried out the experiments and performed the data analysis for the original article: (Hirsch J, Noah JA, Zhang X, Dravida S, Ono Y. 2018. A cross-brain neural mechanism for human-to-human verbal communication. Soc Cogn Affect Neurosci 13:907–920. DOI: 10.1093/scan/nsy070). Y.O. analyzed the data and took the lead in writing the article. All authors contributed to the interpretation of the results and provided critical feedback that shaped the research, analysis, and the article.

Disclaimer

The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. All data reported in this article are available upon request from the corresponding author.

Author Disclosure Statement

There are no competing financial interests or conflicts.

Funding Information

This research was partially supported by the National Institute of Mental Health of the National Institutes of Health under award numbers R01MH107513 (PI J.H.), R01MH119430 (PI J.H.), R01MH111629 (PIs J.H. and J. McPartland), NIH R37HD090153 (PI J.H. subcontract for imaging), NRSA 1F30MH116626 (PI S.D.), NIH Medical Scientist Training Program Training Grant T32GM007205, and the Japan Society for the Promotion of Science grants in Aid for Scientific Research (KAKENHI) JP19H03985 (PI Y.O.).

Supplementary Material

Supplementary Data

Supplementary Figure S1

Supplementary Figure S2