Brainstem-cortical functional connectivity for speech is differentially challenged by noise and reverberation

Abstract

Everyday speech perception is challenged by external acoustic interferences that hinder verbal communication. Here, we directly compared how different levels of the auditory system (brainstem vs. cortex) code speech and how their neural representations are affected by two acoustic stressors: noise and reverberation. We recorded multichannel (64 ch) brainstem frequency-following responses (FFRs) and cortical event-related potentials (ERPs) simultaneously in normal hearing individuals to speech sounds presented in mild and moderate levels of noise and reverb. We matched signal-to-noise and direct-to- reverberant ratios to equate the severity between classes of interference. Electrode recordings were parsed into source waveforms to assess the relative contribution of region-specific brain areas [i.e., brainstem (BS), primary auditory cortex (A1), inferior frontal gyrus (IFG)]. Results showed that reverberation was less detrimental to (and in some cases facilitated) the neural encoding of speech compared to additive noise. Inter-regional correlations revealed associations between BS and A1 responses, suggesting subcortical speech representations influence higher auditory-cortical areas. Functional connectivity analyses further showed that directed signaling toward A1 in both feedforward cortico-collicular (BS→A1) and feedback cortico-cortical (IFG→A1) pathways were strong predictors of degraded speech perception and differentiated “good” vs. “poor” perceivers. Our findings demonstrate a functional interplay within the brain’s speech network that depends on the form and severity of acoustic interference. We infer that in addition to the quality of neural representations within individual brain regions, listeners’ success at the “cocktail party” is modulated based on how information is transferred among subcortical and cortical hubs of the auditory-linguistic network.

INTRODUCTION

In natural listening environments, noise and reverberation hinder the successful extraction of speech information (for review, see White-Schwoch and Kraus, 2017). Although both are acoustic interferences, each has a distinct effect on speech signals (for review, see Bidelman, 2017; Helfer and Wilber, 1990; Nabelek and Dagenais, 1986). Noise is caused by the addition of external competing sound(s) that masks target speech. Contrastively, reverberation is the persistence of reflected acoustic energy in the sound field caused by internal room acoustics (Kinsler et al., 2000). Reverberation produces an overlap between direct and indirect sounds that “smears” a signal’s spectrum. Natural reverberation also acts to low-pass filter speech compared to the overall flattening of modulations produced by noise. Conveniently, the degree of noise and reverberation superimposed on a target signal can be quantified by similar metrics. For noise, the relative contribution of “noise” and “signal” are quantified via the signal-to- noise ratio (SNR). Similarly, the proportions of acoustic energy attributable to signal and reverberant energy are characterized (in dB) by the direct-to-reverberant energy ratio (D/R), or less commonly, “wet- to-dry” ratio (von Békésy, 1938; Zahorik, 2002).

While both acoustic stressors hinder intelligibility, behavioral studies reveal that human listeners show a differential sensitivity when perceiving signals in noise vs. reverberation (Larsen et al., 2008; McShefferty et al., 2015; Zahorik, 2002). That is, even when the relative intensities between signal and interference are matched in severity (i.e., SNR≈D/R), noise and reverberation can impact speech perception in different manners. While each interference reduces speech understanding by ~15–20%, vowel confusion patterns can differ under these two acoustic backdrops (Nabelek and Dagenais, 1986). Interestingly, reverb also induces less listening effort than noise during speech comprehension, even at similar levels of behavioral performance (Picou et al., 2016). This suggests that while there is a comparable tax on speech intelligibility, noise and reverberation might uniquely impact the underlying neural representations for speech (cf. White-Schwoch and Kraus, 2017). To our knowledge, this possibility has not been formally tested.

It is now well-established that speech-in-noise (SIN) understanding is determined by more than audibility or peripheral hearing status (Middelweerd et al., 1990; Song et al., 2011) (but see Humes and Roberts, 1990). The fact that SIN performance is not reliably predicted from the audiogram (Killion and Niquette, 2000) and varies among even normal-hearing listeners (Song et al., 2011) has led to the notion that central brain mechanisms play a critical role in supporting successful cocktail listening (e.g., Alain et al., 2014; Bidelman and Howell, 2016; Billings et al., 2009; Billings et al., 2010; Billings et al., 2013; Parbery-Clark et al., 2011; Song et al., 2011). In this regard, electrophysiological studies have been important in elucidating these central factors of speech sound processing.

The brain’s neuroelectric response to speech reflects an aggregate of activity generated from both brainstem and cortical structures. By exploiting properties of each response (e.g., spectral bandwidth), one can isolate their contributions within the scalp EEG and examine sound encoding within various structures of the auditory hierarchy (Bidelman et al., 2013). The cortical ERPs are composed of several “waves” (e.g., P1-N1-P2), reflecting activation of auditory thalamus, cortex, and associative areas (Picton et al., 1999). ERPs are sensitive to the acoustic features of speech (Agung et al., 2006; Chang et al., 2010; Kraus and Cheour, 2000; Sharma and Dorman, 1999) and correlate with listeners’ perceptual skills (Bidelman et al., 2014b; Ross and Tremblay, 2009; Tremblay et al., 2001). The subcortical component, or frequency-following response (FFR), is a sustained potential emitted dominantly from the upper brainstem that closely mirrors acoustic stimuli with high fidelity (Bidelman, 2018; Krishnan, 2007; Skoe and Kraus, 2010). FFR activity similarly correlates with listeners’ SIN perception (Anderson et al., 2010; Parbery-Clark et al., 2009a; Song et al., 2011). Yet, few studies have examined FFRs to reverberant speech (Bidelman and Krishnan, 2010; Fujihira and Shiraishi, 2015), and we unaware of any directly contrasting the effects of noise and reverb on speech FFRs. Moreover, while a number of studies have investigated the independent contributions of brainstem (e.g., Bidelman, 2016; Bidelman and Krishnan, 2010; Billings et al., 2013; Krishnan et al., 2010; Parbery-Clark et al., 2009a; Song et al., 2011) and cortical neurophysiology (e.g., Alain et al., 2014; Bidelman and Howell, 2016; Billings et al., 2010; Billings et al., 2013; Shtyrov et al., 1998) to degraded speech processing, examining these functional levels simultaneously (within individual listeners) would offer a more comprehensive, systems-level characterization of the biological mechanisms underlying cocktail party listening in different acoustic scenarios and possible interplay between stages of the neuroaxis.

To this end, our recent studies have championed the use of concurrent FFR-ERPs recordings to examine hierarchical auditory processing (e.g., Bidelman, 2015; Bidelman and Alain, 2015b; Bidelman et al., 2013; Bidelman et al., 2014b) (see also Bellier et al., 2015; Slugocki et al., 2017). Dual FFR-ERP paradigms have provided important insight into how lower vs. higher tiers of the neuroaxis code complex sounds and interact during early perception when object-based representations of speech are still in their nascent stages (e.g., Bidelman et al., 2013). Translational applications have further shown how brainstem vs. cortical function is uniquely strengthened by plasticity (Bidelman and Alain, 2015a; Bidelman et al., 2014a; Musacchia et al., 2008), are differentially compromised by hearing loss (Bidelman et al., 2014b), and are altered in neurocognitive disorders (Bidelman et al., 2017). This approach emphasizes a growing body of work that advocates speech processing as an integrative and dynamic network (Kraus and White-Schwoch, 2015; Obleser et al., 2007; Scott et al., 2009) which includes possible interactions and/or signal transformations between brainstem-cortex (Gao and Suga, 1998; Suga et al., 2002) and auditory-linguistic brain areas downstream (Du et al., 2014). Given that early brainstem-cortical and later auditory-linguistic interplay can presumably vary on an individual basis, we hypothesized these individual differences might modulate cocktail party perception. Here, we exploited dual FFR-ERPs to further investigate the neural encoding of impoverished (noisy and reverberant) speech and better define the functional connectivity between subcortical and cortical hubs of the auditory system.

The aims of the present study were thus twofold. First, we directly compared how subcortical and cortical levels of the auditory system encode different forms of degraded speech information. While previous reports have investigated relations between brainstem and cortical auditory processing (e.g., Bidelman et al., 2013; Bidelman et al., 2017; Krishnan et al., 2012; Musacchia et al., 2008; Slugocki et al., 2017), studies have focused exclusively on scalp (electrode-level) recordings and therefore, can only infer contributions of sub- and neo-cortex from volume conducted mixtures of EEG signals. Here, source analysis of brainstem FFRs and cortical ERPs allowed us to parse region-specific activity with higher granularity and more definitively reveal how neural processing within each tier of the neuroaxis coordinate during speech processing. Functional connectivity analysis evaluated the directed, causal signaling between brainstem and cortical regions and how inter-regional neural communication might predict listeners’ speech perception skills. We also measured source FFR/ERPs elicited by noisy and reverberant speech. This allowed us to directly assess how neural speech representations are affected by different acoustic stressors common to the auditory scene. To anticipate, our data reveal that degraded speech perception is governed by intra- and inter- regional brainstem-cortical activity including important cortico-collicular (brainstem-cortical) and cortico- cortical (frontotemporal) signaling.

METHODS

Participants

Eleven adults (age: 24.7 ± 2.7 years) participated in the experiment. All had obtained a similar level of formal education (at least an undergraduate degree), and were monolingual speakers of American English. Musical training is known to enhance auditory evoked responses (e.g., Bidelman et al., 2011; Musacchia et al., 2008; Zendel and Alain, 2009) and improve degraded speech-listening skills (Bidelman and Krishnan, 2010; Parbery-Clark et al., 2009a; Parbery-Clark et al., 2009b). Hence, all participants were required to have minimal formal musical training (1.3 ± 1.8 years) and none within the past 5 years. Audiometric screening confirmed normal hearing (i.e., thresholds < 25 dB HL) at octave frequencies (250–8000 Hz). All participants reported no history of neuropsychiatric disorders. Participants gave written informed consent in compliance with a protocol approved by the University of Memphis Institutional Review Board and were compensated monetarily for their time.

Stimuli

Brainstem and cortical auditory ERPs were elicited by a 300 ms/vCv/speech token/ama/(Bidelman and Howell, 2016) (Figure 1). The pitch prosody fell gradually over the token’s duration (F0= 120 Hz to 88 Hz). Vowel formant frequencies (F1–F3) were 830, 1200, and 2760 Hz, respectively.

Figure 1. Speech stimuli.

Example time waveforms and spectrograms for the speech stimulus/ama/presented in clean (no noise), noise-degraded (+5 dB SNR), and reverberant (+5 dB D/R) tokens.

Noise-degraded speech

In addition to this “clean” stimulus (SNR = ∞ dB), noise-degraded stimuli were created by adding four-talker babble noise (Killion et al., 2004) to the clean token at SNRs of +10 and +5 dB. SNR was manipulated by changing the level of the masker rather than the signal to ensure SNR was inversely correlated with overall sound level (Binder et al., 2004). To mimic real-world acoustic scenarios, babble was presented continuously so that it was not time-locked to the stimulus presentation (e.g., Alain et al., 2012).

Reverberant speech

Reverberant speech was simulated via convolution reverb. We used the XFX1 (v1) reverb plugin in Sound Forge 9 (Sony Inc.) with the “Rich Hall” preset to simulate acoustics of a realistic room (RT₆₀: 500 ms; early reflection style: 16 ms; predelay: 30 ms; filter attenuation: 20– 12000 Hz; early reflection attenuation: −20 dB). We then exported the “wet” (reverberant) portion of the signal (i.e., 0% mix of the “dry” speech) to extract only the “interference” component of the reverberant signal. Effectively, this resulted in a simulation as if the speech was produced in a reverberant environment but with only indirect (reflected) energy containing no direct sound. We then mixed the original (clean) speech token with this reverberation to create reverb-degraded speech at mild and medium direct-to-reverberant ratios (D/R) of +10 and +5 dB, respectively. Importantly, this approach allowed us to precisely control and equate the overall level of severity (i.e., SNR and D/R) between noise- and reverb-degraded stimuli. This avoided potential confounds of having different degrees of acoustic “noisiness” (i.e., signal figure-ground quality) between noise and reverb conditions and allowed us to focus on differences in sound quality produced by each acoustic stressor¹. Acoustic waveform and spectrograms are illustrated for the clean, +5dB SNR noise-degraded, and +5 dB D/R reverberant speech in Figure 1.

Procedure

Listeners heard 2000 trials of each condition (passive listening) presented with fixed, rarefaction polarity using a jittered interstimulus interval (400–700 ms; 20-ms steps, uniform distribution). Stimulus presentation was controlled by MATLAB® 2013b (The MathWorks) routed to a TDT RP2 interface (Tucker-Davis Technologies). Tokens were delivered binaurally at 81 dB SPL through ER-30 insert earphones (Etymotic Research). ER-30 earphones feature an extended acoustic tubing (20 ft) that allowed us to place their transducers outside the testing booth. Spatially separating the transducer from participants avoided the possibility of electromagnetic stimulus artifact from contaminating neural responses (Aiken and Picton, 2008; Campbell et al., 2012). The low-pass frequency response of the headphone apparatus was corrected with a dual channel 15 band graphical equalizer (dbx EQ Model 215s; Harman) to achieve a flat frequency response through 4 kHz. Stimulus level was calibrated using a Larson–Davis SPL meter (Model LxT) measured in a 2-cc coupler (IEC 60126). The entire experimental procedure, including electrophysiological and behavioral testing (described below) took ~2 hrs.

Behavioral speech-in-noise task

Before EEG testing, we measured listeners’ speech reception thresholds in noise using the QuickSIN test (Killion et al., 2004). Participants were presented two lists of six sentences with five key words per sentence embedded in four-talker babble noise. Sentences were presented at 70 dB SPL via insert earphones (bilaterally) using pre-recorded signal-to-noise ratios (SNRs) that decreased in 5 dB steps from 25 dB (very easy) to 0 dB (very difficult). Listeners scored one point for each key word correctly repeated. “SNR loss” (computed in dB) was determined as the SNR required to correctly identify 50% of the key words (Killion et al., 2004). We averaged SNR loss from two lists for each listener. As there is no reverb analog to the QuickSIN, this test functioned as a (normative) behavioral assay of degraded speech listening skills.

Electrophysiological recordings

EEG acquisition and preprocessing

Electrophysiological recording procedures were identical to our previous report (Bidelman and Howell, 2016). Neuroelectric activity was recorded from 64 sintered Ag/AgCl electrodes at standard 10-10 scalp locations (Oostenveld and Praamstra, 2001). EEGs were digitized at a sampling rate of 5000 Hz (SynAmps RT amplifiers; Compumedics Neuroscan) using an online passband of DC-2500 Hz. Electrodes placed on the outer canthi of the eyes and the superior and inferior orbit were used to monitor ocular activity. During online acquisition, all electrodes were referenced to an additional sensor placed ~1 cm posterior to Cz. Data were re-referenced off-line to a common average reference for subsequent analyses. Contact impedances were maintained ≤ 5 kΩ.

Subsequent pre-processing was performed in Curry 7 (Compumedics Neuroscan). Ocular artifacts (saccades and blinks) were first corrected in the continuous EEG using a principal component analysis (PCA) (Picton et al., 2000). Cleaned EEGs were then epoched (−200–550 ms), baseline-corrected to the pre-stimulus period, and averaged in the time domain to obtain compound evoked responses (containing both brainstem and cortical activity) for each stimulus condition per participant.

Source waveform derivations

Scalp potentials (sensor-level recordings) were transformed to source space using discrete inverse models. Fitting was carried out in BESA® Research v6.1 (BESA, GmbH). We used a four-shell volume conductor head model (Berg and Scherg, 1994; Sarvas, 1987) with BESA default settings, i.e., relative conductivities (1/Ωm) of 0.33, 0.33, 0.0042, and 1 for the head, scalp, skull, and cerebrospinal fluid, respectively, and sizes of 85 mm (radius), 6 mm (thickness), 7 mm (thickness), and 1 mm (thickness) (Herdman et al., 2002; Picton et al., 1999). From grand averaged responses, we seeded dipoles in prominent hubs of the speech network, including brainstem (BS), bilateral primary auditory cortex (A1), and bilateral inferior frontal gyrus (IFG). Although a simplistic speech circuit, these three regions of interest (ROIs) have been shown to predict SIN processing in several neuroimaging studies (Adank et al., 2012; Bidelman and Krishnan, 2010; Bidelman and Dexter, 2015; Bidelman and Howell, 2016; Binder et al., 2004; Coffey et al., 2017; Du et al., 2014; Scott and McGettigan, 2013; Song et al., 2011) and allowed specific hypothesis testing of the relation between auditory brainstem-cortical connectivity and degraded speech perception. Sparse source models have also been shown to better resolve subcortical sources in M/EEG than denser dipole configurations (Krishnaswamy et al., 2017), further justifying a restricted dipole count. While EEG inverse solutions are non-unique, we have confirmed the biological plausibility of this brainstem-cortical configuration (clean spatial separation, spatiotemporal dynamics) using identical modeling assumptions in our recent source imaging and computational studies (Bidelman, 2018). Overall, the model provided an excellent fit to the grand averaged data across stimuli and listeners (goodness of fit: 82 ± 4.5%). This dipole configuration was then used to guide source derivations at the single-subject level.

To extract each individual’s source waveforms within each ROI, we transformed listener’s scalp recordings into source-level responses using a virtual source montage (Scherg et al., 2002). This digital re-montaging applies a spatial filter to all electrodes (defined by the foci of our dipole configuration). Relative weights are then optimized to image activity within each brain ROI while suppressing overlapping activity stemming from other active brain regions (for details, see Scherg and Ebersole, 1994; Scherg et al., 2002). This allowed us to reduce each listener’s electrode recordings (64-channels) to 5 source channels, each of which estimated activity generated within a single ROI (i.e., left/right IFG, left/right A1, BS). For each participant, the model was held fixed and was used as a spatial filter to derive their source waveforms (Zendel and Alain, 2014), reflecting the neuronal current (in units nAm) as seen within each anatomical ROI. Source waveforms were then bandpass filtered into high- (80 – 1500 Hz) and low- (0.5 –20 Hz) frequency bands to isolate the periodic brainstem FFR vs. slower cortical ERP waves from each listeners’ compound evoked potential (Bidelman et al., 2013; Musacchia et al., 2008). Comparing FFR and ERP source waveforms allowed us to assess the relative contribution of brainstem and cortical generators to degraded speech processing. Identical model assumptions for the inverse solution (e.g., volume conductor, conductivities) were used for brainstem and cortical ROIs waveforms; only the anatomical location of the dipole differed. Importantly, EEG provides more consistent SNR for both deep (brainstem) and superficial sources (cortex) and between focal and distributed models compared to other neuroimaging modalities (e.g., MEG; Goldenholz et al., 2009).

FFR/ERP source waveform quantification

FFRs and ERP source waveforms were quantified based on their amplitude (FFR: RMS amplitude; ERP: N1 amplitude). Previous studies have shown correspondence between these measures and degraded speech perception (Bidelman and Howell, 2016; Billings et al., 2013; Parbery-Clark et al., 2011). The cortical N1 was taken as the negative-going deflection between 100 and 150 ms based on the expected noise-related delay in the ERPs (Bidelman and Howell, 2016; Billings et al., 2009) after factoring in the acoustic delay of the headphones based on the speed of sound in air [17.8 ms = (20 ft)/(1125 ft/s)].

Distributed source imaging (CLARA)

We used Classical Low Resolution Electromagnetic Tomography Analysis Recursively Applied (CLARA) [BESA (v6.1)] to estimate the neuronal current density underlying the sensor data recorded at the scalp. Distributed analyses were applied only to cortical activity (i.e., 0.5–20 Hz filtered data). The CLARA technique models the inverse solution as a large collection of elementary dipoles distributed over nodes on a mesh of the cortical volume. The aggregate strength of current density in each voxel can then be projected spatiotemporally onto the neuroanatomy, akin to a functional map in fMRI. CLARA estimates the total variance of the scalp-recorded data and applies a smoothness constraint to ensure current changes little between adjacent brain regions (Michel et al., 2004; Picton et al., 1999). CLARA renders highly focal source images by iteratively reducing the source space during repeated estimations. On each step (x10), a spatially smoothed LORETA solution (Pascual-Marqui et al., 2002) is recomputed and voxels below a 1% max amplitude threshold are removed. This provided a spatial weighting term for each voxel of the LORETA image on the subsequent step. Ten iterations were used with a voxel size of 5 mm in Talairach space and regularization (parameter accounting for noise) set at 0.01% singular value decomposition. The spatial resolution of CLARA is estimated at 5–10 mm (Iordanov et al., 2014; Iordanov et al., 2016), which is smaller (better) than the physical distances between PAC and IFG, and between BS and PAC (~35–40 mm) (Mazziotta et al., 1995). We have previously used this approach to image electrophysiological speech processing (e.g., Alain et al., 2017; Bidelman, 2018). Source activations were visualized on BESA’s adult brain template (Richards et al., 2016).

We used cluster-based permutation tests (Maris and Oostenveld, 2007) implemented in BESA Statistics (2.0) to contrast the distributed source images and identify anatomical locations in the brain volume (and over time) that distinguished noise- vs. reverb-degraded speech. For each voxel and time point, a paired samples t-test was conducted contrasting the two mild forms of acoustic interference (i.e., +10 dB SNR noise vs. +10 D/R reverb). This allowed us to determine where and when neural activity showed significant differences (p<0.05) between interferences. Cluster values were derived based on the sum of all t-values of data points within a given cluster. Significant differences were determined by generating and comparing surrogate clusters from n=1000 resamples of the data permuting between stimulus conditions (e.g., Oostenveld et al., 2011). This identified contiguous clusters of voxels over time (i.e., statistical contrast maps) where stimulus conditions were not interchangeable (i.e., noisy speech ≠ reverberant speech; p < 0.05). Importantly, BESA corrects for multiple comparisons across the aggregate of all voxels and time points by controlling the familywise error rate through this clustering process. The same procedure was then repeated to contrast the two medium forms of interference (i.e., +5 dB SNR noise vs. +5 D/R reverb). In the present study, distributed source imaging was used to identify empirically, the most relevant cortical regions that distinguish noise and reverberant speech. Source montages, based on discrete dipole modeling, were then used to quantitatively investigate the time course of speech-evoked responses within each cortical ROI and assess their relation to brainstem activity.

Functional connectivity

We measured causal (directed) information flow between nodes of the brainstem-cortical speech network using phase transfer entropy (PTE) (Lobier et al., 2014). TE is a non-parametric, information theoretic measure of directed signal interaction. It is ideal for measuring functional connectivity between regions because it can detect nonlinear associations between signals and is robust against the volume conducted cross-talk in EEG (Hillebrand et al., 2016; Vicente et al., 2011). TE was estimated using the time series of the instantaneous phases of the signals, yielding the so-called phase transfer entropy (PTE) (Hillebrand et al., 2016; Lobier et al., 2014). PTE was computed between ROIs according to Eq. 1:

$${\mathit{PTE}}_{X\to Y}=\mathrm{\sum }p(y_{t+\tau },y_t^m,x_t^n){log}_2\left[\frac{p(y_{t+\tau }\mid y_t^m,x_t^n)}{p(y_{t+\tau },y_t^n)}\right]$$ (Eq.1)

where X and Y are the ROI signals and the log(.) term is the conditional probabilities between signals at time t+τ for sample m and n. The probabilities were obtained by building histograms of occurrences of pairs of phase estimates in the epoch (Lobier et al., 2014). Following Hillebrand et al. (2016), the number of histogram bins was set to e^{0.626+0.4ln(Ns − τ − 1)} (Otnes and Enochson, 1972). The prediction delay τ was set as (N_s x N_ch)/N±, where N_s and N_ch are the number of samples and ROI sources, respectively, and N± the number of times the phase changes sign across time and ROI signals (Hillebrand et al., 2016). PTE cannot be negative and has no upper bound. Higher values indicate stronger connectivity, whereas PTE_X→Y =0 implies no directed signaling.

Intuitively, PTE can be understood as the reduction in information (units bits) necessary to encode the present of ROI_Y if the past of ROI_X is used in addition to the past of ROI_Y. In this sense, it is similar to the definition of Granger Causality (Barnett et al., 2009), which states that ROI_X has a causal influence on the target ROI_Y if knowing the past of both signals improves the prediction of the target’s future compared to knowing only its past. Yet, PTE has several important advantages over other connectivity metrics: (i) PTE is more robust to realistic amounts of noise and linear mixing in the EEG that can produce false-positive connections; (ii) PTE relaxes assumptions about data normality and is therefore model-free; (iii) PTE is asymmetric so it can be computed bi-directionally between pairs of sources (X→Y vs. Y→X) to infer causal, directional flow of information between interacting brain regions. Computing PTE in both directions between pairwise sources (e.g., BS↔A1 and A1↔IFG) allowed us to quantify the relative weighting of information flowing between subcortical and cortical ROIs in both feedforward and feedback directions.

RESULTS

Sensor-level data

Scalp-recorded cortical ERPs and brainstem FFRs (electrode data) to noise and reverberant speech are shown in Figures 2–3. Visual inspection of scalp maps suggested weaker cortical responses with decreasing SNR for noise-degraded speech (Fig. 2). Weaker N1-P2 is presumably due to the decrease in inter-trial response coherence (i.e., increased jitter) caused by noise (Koerner and Zhang, 2015). This amplitude reduction was not observed for reverberant speech where ERPs showed little apparent change with increasing severity of reverb. Similarly, scalp FFRs (Fig. 3), reflecting phase-locked brainstem activity, showed a reduction in amplitude with increasing levels of noise but an apparent enhancement for speech in reverberation. This was most apparent when directly comparing mild (+10 dB SNR noise vs. mild reverb) and moderate interferences (+5 dB SNR noise vs. med. reverb). Despite matching noise SNR and reverb D/R, reverberant speech produced larger responses, particularly at the voice fundamental frequency (F0) (Fig. 3B). The volume conducted nature of sensor recordings did not allow for the separation of underlying sources. Hence, subsequent analyses were conducted in source space.

Figure 2. Cortical ERPs (electrode data).

Grand average responses at frontal and temporal channels (T7/8, Fz) to speech as a function of noise SNR (top traces) and reverb D/R (bottom traces). Responses appear as a series of obligatory waves within the first 200 ms of speech characteristic of the auditory P1- N1-P2 signature. Topographies illustrate the distribution of activity across the scalp. Blue map colors refer to negative voltage, red colors indicate positive voltage. Reverberation is less detrimental t (and in some cases facilitates) the neural encoding of speech compared to additive noise.

Figure 3. Brainstem FFRs (electrode data).

(A) Grand average time waveforms and (B) response spectra to noise-degraded and reverberant speech. F0, voice fundamental frequency. Spectra illustrate enhancements in speech encoding (particularly at F0) with reverberation compared to speech presented in equivalent amounts of interference. Otherwise as in Fig. 2.

Source waveform data

CLARA distributed source maps contrasting cortical activity (0.5–20 Hz) to noise- and reverb- degraded speech are shown in Figure 4. Contrasts between the +10 dB SNR noise and mild reverb conditions revealed responses to these mild forms of interference were differentiated in bilateral, but especially right, auditory cortex (Fig. 4A). The maximal effect (i.e., reverb > noise) occurred 80 ms after stimulus onset. Similarly, contrasts between the +5 dB SNR noise and medium reverb conditions revealed that responses to moderate interferences were differentiated ~30 ms later (108 ms), primarily within left IFG (Fig. 4B). These data suggest a distributed cortical network for speech consisting of auditory (A1) and linguistic (IFG) brain regions that is differentially engaged, intra- and inter-hemispherically, depending on the severity of acoustic interference. While these distributed source analyses provided a whole-brain view of degraded speech processing, they do not allow specific hypothesis testing within the network. Consequently, we extracted source waveforms within cortical A1 and IFG as well as brainstem ROIs to investigate how noise and reverb modulate the subcortical-cortical encoding of speech. Following our previous report (Bidelman and Howell, 2016), left and right hemisphere responses were collapsed for subsequent analyses. This reduced the dimensionality of the data and provided increased SNR for source- level analysis. Pooling hemispheres is also motivated by the fact that even monaural stimulus presentation produces bilateral auditory cortical activity (Schonwiesner et al., 2007).

Figure 4. Statistical contrasts (t-maps) of source responses to noise and reverberant speech.

Data filtered 0.5–20 Hz (i.e., cortical activity). Functional data projected on the brain volume are displayed on the BESA brain template (Richards et al., 2016). (A) Contrast of mild forms of acoustic interference (+10 dB SNR noise vs. mild reverb). (B) Contrast of moderate forms of acoustic interference (+5 dB SNR noise vs. medium reverb). Maps are masked (p<0.05) and corrected for multiple comparisons (see text). Bottom traces show the running t-value (noise > reverb) of the cluster marker in the brain volume by crosshairs. Cool colors denote locations where reverb > noise. Degraded speech is processed in bilateral A1 and IFG. For mild acoustic interferences (A), noisy and reverberant speech are differentiable mainly in right A1. This contrasts speech in moderate interferences (B), which becomes distinguishable ~30 ms later in left IFG.

Brainstem (BS) and cortical (A1, IFG) source waveforms are shown in Figures 5A and B, respectively. BS sources illustrate the periodic frequency-following response (FFR) of the auditory midbrain (Bidelman, 2018). Response spectra (lowest panels) illustrate phase-locked activity to the voice pitch (F0≈100 Hz) and harmonic structure of speech. A1 and IFG waveforms track the time course of speech encoding in primary auditory and linguistic brain regions and modulations with noise and reverberation. We used a mixed model, 3-way ANOVA (ROI x interference type x interference level; subject=random factor) to analyze response amplitudes. Initial diagnostics revealed that amplitudes were leptokurtic (i.e., high peakedness) in their distribution. Hence, amplitudes measures were asinh(.) transformed to improve normality and homogeneity of variance assumptions necessary for parametric statistics. The ANOVA revealed significant main effects of interference type [F_1,140=7.01, p=0.009] and ROI [F_2,140=8.45, p=0.0003]. Planned contrasts revealed linear effects of interference level on BS [t₄₀ =3.34, p=0.0018] and IFG [t₄₀ =2.62, p=0.0124] amplitudes. That is, relative to clean speech, FFR amplitudes steadily increased with additional noise but were also larger for reverberant compared to noisy speech (i.e., Fig. 5C and 5E; +10 dB < +5 dB < mild < medium). This noise/reverb-related modulation was not observed in A1 [linear effect: t₄₀ =0.64, p=0.53] (Fig. 5D). Within IFG, we also found a significant effect of interference type [F_1,40=7.91, p=0.0076]. That is, despite matched level of severity, speech in mild reverb elicited larger amplitudes than speech in mild noise (p=0.0129). The facilitation effect of reverb on cortical IFG responses parallels the reverb facilitation observed in the sensor-level data (Figs. 2–3).

Figure 5. Brainstem FFR and cortical ERP source responses to speech in noise and reverberation.

Source waveforms (collapsed across hemispheres) extracted from cortical (IFG, A1; 0.5–20 Hz) and brainstem (BS; 80–1500 Hz) dipole sources. (A) Responses in noise, (B) Responses in reverb. Bottom traces below BS-FFR waveforms show the spectra of phase-locked activity from the brainstem source. Gray trace, stimulus waveform. (C–E) Changes in IFG, A1, and BS source amplitudes across interferences. Shading and errorbars = ±1 s.e.m.

Relations between brainstem and cortical speech responses

Pairwise Pearson’s correlations examined associations between brainstem and cortical neuroelectric responses (Fig. 6). We found that BS-FFRs were positively correlated with A1-ERP amplitudes (r=0.74, p=0.0089) such that stronger brainstem speech encoding was associated with larger auditory cortical responses (Fig. 6A). This suggests that neural activity along the ascending auditory pathway (BS→A1) might drive responses downstream. No associations were observed between BS and IFG amplitudes (Fig. 6B; r=0.37, p=0.27) nor between A1 and IFG (Fig. 6C; r=0.20, p=0.56). The lack of correlation between A1→IFG (and BS→IFG) implies that speech representations might become more independent/abstract at higher levels of the neural hierarchy.

Figure 6. Correlations between BS, A1, and IFG source activity during degraded speech processing.

(A) BS-FFRs are positively correlated with A1-ERP amplitudes; stronger brainstem encoding of speech is associated with larger responsivity in auditory cortex. No associations were observed between BS and IFG amplitudes (B) nor between A1 and IFG (C) suggesting speech representations become more independent at higher levels of the speech hierarchy beyond lemniscal auditory regions. Solid lines, significant correlations; dotted lines, n.s. correlations. **p < 0.01

Brain-behavior relations

We used a generalized linear mixed effects model (GLME) to evaluate relationships between neural responses to acoustically-degraded speech and behavioral performance (Fig. 7A). We considered the combination of all neural amplitude measures (BS_FFR, A1_ERP, IFG_ERP; pooled across noise/reverb conditions) as well as their interactions as predictors of listeners’ QuickSIN scores. Participants were modeled as a random term [i.e., QuickSIN ~ FFR*A1*IFG + (1|subject)]. Dependent variables were mean centered to reduce multicollinearity (Afshartous and Preston, 2011). We found that the overall model was significant [F_1,3=11.56, p=0.037; null hypothesis coefficients=0] accounting for 81% of the variance in QuickSIN scores (adjusted-R²= 0.81). Scrutiny of individual model terms revealed significant neural predictors in BS (p=0.0241) and IFG (p=0.011) amplitudes. In addition to these focal responses, we observed significant A1*IFG (p=0.018) and BS*IFG (p=0.011) interactions. No other terms reached significance. Collectively, these results suggest that in addition to intra-regional activity within brainstem and cortical areas (BS, IFG), cortico-collicular (BS-A1) and cortico-cortical (A1-IFG) interactions are strong predictors of degraded speech perception abilities.

Figure 7. Degraded speech perception performance is predicted by intra-regional activations and inter-regional connectivity.

(A) BS, A1, and IFG activity and their inter-regional interactions (pooled across all interference conditions) account for 81% of the variance (adj-R²) in behavioral QuickSIN scores. Cortico-collicular (BS-A1) and cortico-cortical (A1-IFG) neural interactions are also strong predictors of perception. Lines connecting ROIs show pairwise source terms that interacted in the GLME (solid lines, significant; dotted lines, n.s.) Statistical flags within ROIs reflect the main effect of that source in the GLME. (B) QuickSIN scores for “good” (n=6) vs. “poor” (n=5) SIN performers (median split of scores). Despite all having normal hearing, certain listeners obtain better (~1 dB lower) SIN thresholds indicating individual differences in SIN perception. (C) Functional connectivity in the brainstem-cortical speech network that differentiates “good” vs. “poor” SIN performers. Values represent the magnitude of connectivity computed via phase transfer entropy (Lobier et al., 2014). Statistical flags represent group contrasts at each connection. Feedforward (BS→A1) and feedback (IFG→A1) towards primary auditory cortex (A1) is stronger in “good” relative to “poor” perceivers. ^†p < 0.1; *p < 0.05; **p ≤ 0.01

Functional connectivity

Phase transfer entropy, quantifying the feedforward (afferent) and feedback (efferent) functional connectivity between sources, is shown in Figure 7C. To investigate the behavioral relevance of PTE connectivity measures, we divided our cohort in two groups based on a median split of listeners’ QuickSIN scores. This resulted in two subgroups, “good” (n=6) vs. “poor” (n=5) SIN performers who differed in their SIN scores (Fig. 7B). Subgroups were otherwise matched in age (t₉=1.61, p=0.14), musical training (t₉= 0.25, p=0.81), and gender (Fisher’s exact test: p=1). Despite all having normal hearing, good listeners obtained better (~1 dB lower) SIN thresholds [t₉= -2.47, p=0.035], confirming individual differences in SIN perception (Fig. 7B) (e.g., Song et al., 2011). We then analyzed each of these group’s PTE values to assess whether or not directed, inter-regional neural connectivity (i.e., BS↔A1 and A1↔IFG) could distinguish “good” vs. “poor” SIN performers (Fig. 7C). An ANOVA conducted on PTE values revealed a significant group x region interaction in the strength of functional connectivity [F_3,27=4.79, p=0.0084]. Identical results were also obtained with a Freidman non-parametric ANOVA (group x region: F_3,27=4.79, p=0.0084). Tukey-Kramer contrasts revealed that this interaction was driven by stronger connectivity in “good” compared to “poorer” perceivers between IFG→A1 (p=0.0041) and marginally stronger connectivity between BS→A1 (p=0.07) (Fig. 7C). No other regional pairs showed group differences in connectivity. These results confirm both bottom up (BS→A1) and top down (IFG→A1) communication directed toward A1 is important for degraded speech perception.

DISCUSSION

By measuring source-level FFR and ERPs to noise- and reverb-degraded speech, the present study helps elucidate how central neural representations for communicative signals are affected by different acoustic stressors common to the auditory scene. Collectively, our results relate to three primary findings: (i) reverberation is less detrimental to (and in some cases facilitated) the neural encoding of speech compared to additive noise matched in acoustic severity (i.e., signal figure-ground quality); (ii) listeners’ degraded speech perception skills are predicted by intra- and inter-regional brainstem-cortical activity including cortico-collicular (brainstem-cortical) and cortico-cortical (frontotemporal) interactions; (iii) functional connectivity directed toward A1 from BS (feedforward) and IFG (feedback) differentiates “good” from “poor” perceivers on behavioral SIN tests.

Noise and reverberation differentially impact the neural encoding of speech

Direct comparisons between acoustic stressors demonstrated that reverberation causes lesser interruption to the brain’s speech representations than comparable levels of noise. These findings cannot be attributed to trivial differences in “noisiness” (i.e., acoustic figure-ground quality) as noise and reverb were matched in their severity (i.e., SNR≈D/R). At both brainstem and cortical levels, reverberant speech produced larger evoked responses compared to clean speech, a facilitation that was most apparent in BS and IFG sources. Similarly, whole-brain analyses indicated that cortical activations were stronger for mild and moderate reverberation relative to comparable noise levels in both right A1 and left IFG (Fig. 4). The lesser impact of reverb than noise we find in electrophysical responses is consistent with behavioral data that demonstrates human speech perception is remarkably resilient to reverberation (Bradley et al., 1999; Culling et al., 2003; Nielsen and Dau, 2010; Watkins, 2005) and the fact that listeners notice fewer signal changes in reverberation than in noise (Larsen et al., 2008; McShefferty et al., 2015; Zahorik, 2002).

Weaker cortical responses in noise could reflect neural desynchronization caused by lower inter- trial phase coherence (i.e., increased jitter) of responses in the presence of acoustic interference (Koerner and Zhang, 2015). But why is a similar desynchronization to speech not observed in reverberation? Animal recordings (Sayles and Winter, 2008) and previous human FFR studies (Al Osman et al., 2017; Bidelman and Krishnan, 2010) show that speech cues are more easily maintained at the neural level in 15 reverb compared to noise (for review, see Bidelman, 2017). Some investigators have also postulated that neurons in the brainstem inferior colliculus—the putative generator of scalp FFRs (Bidelman, 2018; Smith et al., 1975)—might perform a neural compensation that mitigates the negative effects of reverb and target signal representations (Slama and Delgutte, 2015). For a subset of collicular neurons, Slama and Delgutte (2015) found that the temporal coding of amplitude envelope was better for reverberant than for anechoic stimuli having the same modulation depth at the ear, indicating a robustness in auditory signal processing to reverb. The strong resilience of speech coding we find across brain levels may reflect population-level dynamics of these early neural compensatory mechanisms (Slama and Delgutte, 2015) that ultimately increase the tolerance of auditory processing in reverberation (Bradley et al., 1999; Culling et al., 2003; Nielsen and Dau, 2010; Watkins, 2005).

Alternatively, the resilience of neural responses to reverberation could reflect a refinement of auditory signal processing aided by corticofugal (cortico-collicular) efferent pathways. Cortical neurons can enhance the SNR of their own thalamo-collicular inputs via corticofugal gain but only when their receptive fields can accurately estimate task-relevant stimulus features—termed “egocentric selection” (Suga et al., 2000). However, why corticofugal effects would be specific (and/or larger) for reverberant compared to noise-degraded speech is unclear. Whether such top-down modulations can occur under strictly passive listening tasks (as used here) also remains speculative. That said, efferent modulations from the lower brainstem to the cochlea (olivocochlear bundle) that provide an “antimasking” function (Winslow and Sachs, 1987) are observed under passive scenarios and are highly predictive of SIN skills at the behavioral level (Bidelman and Bhagat, 2015; Mishra and Lutman, 2014). However, our data show that afferent (BS→A1) rather than efferent (A1→BS) brainstem-cortical signaling differentiated “good” from “poor” listeners on the QuickSIN (Fig. 7C).

Remarkably, these brain-behavior associations were observed between passive neural encoding of relatively simple speech sounds (i.e., vCv in noise) and more complex speech recognition (i.e., QuickSIN), corroborating findings of recent brainstem and cortical ERP studies (Bidelman and Dexter, 2015; Song et al., 2011). Still, most real-world “cocktail party” scenarios require more than mere passive figure-ground (cf. target-noise) analysis as assessed here. Indeed, listeners must often parse multiple streams of competing speech information in a multi-talker environment, a process itself enhanced by attention (Alain, 2007; Ding and Simon, 2012; Hill and Miller, 2010; Puvvada and Simon, 2017; Xiang et al., 2010). Future studies are needed to clarify the role of afferent/efferent interplay during speech processing and the impact of active vs. passive listening paradigms. Furthermore, while we find acoustically matched noise SNR and reverb D/R are processed differentially by the auditory system, it is still possible these interferences differ in their perceptual equivalency. Thus, an interesting extension to the present study would be to examine speech coding and brainstem-cortical connectivity for noise and reverb stimuli first titrated for perceptual difficulty. On this point, Picou et al. (2016) have recently shown that reverb induces less listening effort than noise during speech comprehension, which is consistent with our neurophysiological findings.

Hierarchical neural encoding of degraded speech

Cross-level comparisons between functional tiers revealed that noise- and reverb-related changes in neural responses were generally larger in brainstem compared to cortex. While FFRs exhibited systematic amplitude increases with increasing noise/reverb, A1 responses showed more muted changes. This suggests acoustic interferences have a differential effect on subcortical vs. cortical tiers of the auditory hierarchy during speech processing (i.e., BS > A1). To our knowledge this has not been previously reported and is only made apparent by the dual FFR-ERP paradigm used here.

The lesser effect of noise/reverb at the cortical compared to brainstem level might be accounted for by differences in how non-auditory, compensatory brain regions are marshalled during degraded speech processing. In cortex for instance, top-down control of auditory A1 responses from frontal regions can occur pre-attentively and independent of attention (Doeller et al., 2003). Top-down influences are also generally stronger at cortical relative to subcortical levels (Galbraith and Kane, 1993; Hillyard and Picton, 1979; Okamoto et al., 2011; Woods and Hillyard, 1978). Thus, the smaller interference-induced changes we find in A1 relative to brainstem (midbrain) could be due to stronger “top-down” influences from higher-order, non-auditory brain areas (e.g., IFG) to primary auditory cortex than those from corticofugal modulations between brainstem and A1. This notion is consistent with recent studies showing strong engagement of IFG (linguistic areas) in addition to A1 (primary auditory areas) during passive (Bidelman and Dexter, 2015; Bidelman and Howell, 2016) and active (Binder et al., 2004; Du et al., 2014; Scott and McGettigan, 2013) SIN processing. Indeed, our data here demonstrate that IFG is actively involved during speech processing in both distinguishing noise from reverberant speech (Fig. 4) and in predicting behavioral QuickSIN scores (Fig. 7). Our results corroborate the notion that neural activity in frontal brain regions outside conventional auditory system are robust predictors of SIN perception (Adank et al., 2012; Bidelman and Dexter, 2015; Bidelman and Howell, 2016; Binder et al., 2004; Wong et al., 2008). Nevertheless, the strong interactions we find between BS and A1 do indicate that degraded speech processing also depends on the functional interplay between subcortical and cortical hubs of the auditory hierarchy—perhaps in maintaining a veridical depiction of the speech signal during its neural ascent.

Feedforward/feedback to A1 from BS and IFG is critical to degraded speech perception

In this regard, our GLME and PTE results help reveal important patterns of functional connectivity during degraded speech processing. We found intra-regional activity within auditory BS and IFG were themselves strong predictors of listeners’ QuickSIN performance, in agreement with previous neuroimaging studies (cf. Bidelman and Howell, 2016; Binder et al., 2004; Coffey et al., 2017; Du et al., 2014). Similarly, BS and A1 activity was strongly correlated, suggesting that neural representations in lower auditory relays (which dominate FFRs) might feed-forward to influence sound encoding at the cortical level. These results corroborate findings of Coffey et al. (2017) who suggested that superior feed532 forward (brainstem-cortical) neural encoding during noise-degraded speech processing provides better information to A1. We replicate and extend these results by showing brainstem-cortical reciprocity is also critical for speech processing in other acoustic stressors (i.e., reverb) but that behavior also depends on links between auditory and linguistic cortical regions. In addition to cortico-collicular (BS-A1) associations (Bidelman and Alain, 2015a; Coffey et al., 2017), we found that cortico-cortical (A1-IFG) interactions were a strong predictor of degraded speech perception abilities (cf. Adank et al., 2012; Binder et al., 2004). The observation that BS and IFG responses were largely independent (Figs. 6 and 7A) further suggests speech undergoes several neural transformations from brainstem, through the auditory cortices, in route to linguistic brain regions during the analysis of speech.

Functional connectivity corroborated these regression results by demonstrating the direction of inter-regional signaling that drives degraded speech perception. In particular, we found that bottom up (BS→A1) and top down (IFG→A1) directed communication towards primary auditory cortex (A1) is critical in determining successful degraded speech perception and differentiating “good” vs. “poorer” listeners (Fig. 7). Our connectivity findings broadly agree with notions that speech is processed in a dynamic, and distributed fronto-temporal network (Bidelman and Dexter, 2015; Bidelman and Howell, 2016; Du et al., 2014; Obleser et al., 2007; Scott and McGettigan, 2013; Scott et al., 2009) whose engagement is differentially recruited depending on signal clarity, intelligibility, and linguistic experience (Adank et al., 2012; Bidelman and Dexter, 2015; Scott and McGettigan, 2013).

Our results reveal that a series of hierarchical computations are involved in degraded auditory processing, whereby neural correlates for speech are maintained in successively more abstract forms as the signal traverses the auditory system. This parallels similar hierarchical processing observed for non- speech sounds (Bidelman and Alain, 2015b). It is thought that information relayed from lower levels of the pathway are successively pruned so as to allow easier readout of signal identity in higher brain areas responsible for generating percepts (Chechik et al., 2006). Indeed, multi-unit (Perez et al., 2013) and ERP studies (e.g., Bidelman and Alain, 2015b; Bidelman et al., 2013; Bidelman et al., 2014b) directly comparing responses in brainstem and early sensory cortex help bolster the notion of a continued abstraction in the neural code (see also Sinex et al., 2005; Sinex et al., 2003). It is possible that the current data reflect a similar pruning operation where speech cues coded in lower structures (brainstem) are faithfully carried forward into cortex (A1) but are then recast upon arrival in IFG into more abstract, invariant representations that are more robust to surface-level stimulus manipulations like noise (Adank et al., 2012; Bidelman and Howell, 2016; Binder et al., 2004; Du et al., 2014).

Collectively, findings of the present study point to fundamental differences in how noise and reverberation affect the subcortical-cortical neural encoding of speech. A differential neural coding between interferences may help account for the challenges observed by hearing impaired listeners in certain acoustic environments (but not others) as well as the unique types of perceptual confusions listeners experience in noise compared to reverberation (Nabelek and Dagenais, 1986).

Highlights.

• Examined source-level brainstem/cortical ERPs to speech in noise an reverb

• Reverb less detrimental than severity-matched noise to neural encoding

• BS→A1 and IFG→A1 functional connectivity predicted speech-in-noise perception

• “Cocktail party” success driven by bottom-up and top-down communication toward A1