Effects of better-ear glimpsing, binaural unmasking, and spectral resolution on spatial release from masking in cochlear-implant users

Abstract

Bilateral cochlear-implant (BICI) listeners obtain less spatial release from masking (SRM; speech-recognition improvement for spatially separated vs co-located conditions) than normal-hearing (NH) listeners, especially for symmetrically placed maskers that produce similar long-term target-to-masker ratios at the two ears. Two experiments examined possible causes of this deficit, including limited better-ear glimpsing (using speech information from the more advantageous ear in each time-frequency unit), limited binaural unmasking (using interaural differences to improve signal-in-noise detection), or limited spectral resolution. Listeners had NH (presented with unprocessed or vocoded stimuli) or BICIs. Experiment 1 compared natural symmetric maskers, idealized monaural better-ear masker (IMBM) stimuli that automatically performed better-ear glimpsing, and hybrid stimuli that added worse-ear information, potentially restoring binaural cues. BICI and NH-vocoded SRM was comparable to NH-unprocessed SRM for idealized stimuli but was 14%–22% lower for symmetric stimuli, suggesting limited better-ear glimpsing ability. Hybrid stimuli improved SRM for NH-unprocessed listeners but degraded SRM for BICI and NH-vocoded listeners, suggesting they experienced across-ear interference instead of binaural unmasking. In experiment 2, increasing the number of vocoder channels did not change NH-vocoded SRM. BICI SRM deficits likely reflect a combination of across-ear interference, limited better-ear glimpsing, and poorer binaural unmasking that stems from cochlear-implant-processing limitations other than reduced spectral resolution.

I. INTRODUCTION

In a typically functioning auditory system, binaural hearing offers distinct advantages over monaural hearing by improving sound localization, speech intelligibility, and the organization of auditory scenes (Blauert, 1997; Hawley et al., 2004). Similarly, bilateral cochlear implants (BICIs) provide benefits exceeding those of unilateral cochlear implants (CIs) for sound localization (Litovsky et al., 2009), speech understanding in quiet (Müller et al., 2002), and especially speech understanding in the presence of noise or competing talkers (van Hoesel and Tyler, 2003; Litovsky et al., 2006; Loizou et al., 2009; Rana et al., 2017; Buss et al., 2018). Nonetheless, there remains a gap in the binaural benefits accessible to BICI listeners. Relative to listeners with clinically normal hearing (NH), BICI listeners show considerably less spatial release from masking (SRM), defined as the improvement in target speech understanding when maskers are spatially separated compared to when they are co-located with the target (Culling and Lavandier, 2021).

The BICI SRM deficit is particularly pronounced when maskers with envelope fluctuations that are independent from the target (and from each other) are placed symmetrically to either side of a frontal speech target. In this scenario, the ears are the same distance and angle relative to the target and each of the respective masker sources. This configuration produces no long-term target-to-masker ratio (TMR) advantage at either ear (Rana et al., 2017; Hu et al., 2018). The specific mechanism underlying the BICI SRM deficit is unclear. Hu et al. (2018) proposed that the deficit stems largely from reduced ability to extract and utilize momentary fragments of target-dominated energy in either ear in a process described as “better-ear glimpsing” (Brungart and Iyer, 2012). Alternatively, the deficit could stem from an inability to take full advantage of interaural time differences (ITDs) and interaural level differences (ILDs) to improve signal detection (a process referred to as “binaural unmasking”; e.g., Van Deun et al., 2011). Binaural unmasking improves performance by suppressing masker effectiveness, thus improving the effective internal TMR (e.g., Durlach, 1963; Culling, 2007), or by allowing for the perceptual segregation of sources into different perceptual streams (Ihlefeld and Shinn-Cunningham, 2008; Middlebrooks and Waters, 2020) based on perceived spatial separation. BICI listeners typically obtain very little binaural unmasking, which can be measured by comparing bilateral performance in a spatial target-masker configuration where interaural difference cues are available to performance in a configuration where interaural difference cues are not available (such as when listening is restricted to the ear with the long-term TMR advantage; Loizou et al., 2009). On the other hand, the improvement from bilateral compared to unilateral listening in co-located configurations (known as redundancy) is similar to NH peers for some BICI listeners (Litovsky et al., 2006). Therefore, the BICI binaural unmasking deficit likely reflects a limitation in comparing information across the ears.

The following sections outline proposed mechanisms of SRM in asymmetric compared to symmetric masker configurations. A study addressing potential deficiencies in BICI SRM related to glimpsing ability and binaural unmasking is then presented. In the study, glimpsing ability was manipulated through idealized monaural stimuli that collapsed all the better-ear glimpses into one ear, essentially performing better-ear glimpsing on behalf of the listener. Any SRM improvement that occurred in idealized glimpsing compared to natural symmetric masker conditions was interpreted as evidence of reduced glimpsing ability in the natural symmetric case. Binaural unmasking was manipulated by presenting all the material rejected from the idealized monaural stimuli to the contralateral ear, which restored some within-channel binaural cues. Therefore, any SRM improvement in this condition (where binaural cues were present) compared to the idealized glimpsing condition (where they were not) was interpreted as binaural unmasking. The study assessed three types of distortions in CI processing that could contribute to the limited better-ear glimpsing or binaural unmasking hypothesized to underlie SRM deficits: (1) reduced spectral resolution (e.g., Croghan and Smith, 2018), (2) lack of temporal fine-structure (TFS) encoding (e.g., Ihlefeld and Litovsky, 2012), and (3) asymmetric inputs (e.g., Laback et al., 2015). These distortions were assessed by comparing BICI SRM to that of NH listeners with unprocessed stimuli (for distortion types 1, 2, and 3) and vocoded stimuli with a variable number of channels (for distortion type 1).

A. Head shadow and binaural contributions to SRM with asymmetric maskers

For sources away from the midline, the head acts as an acoustic obstruction. Especially for asymmetric spatial configurations, head shadow creates an ILD that produces a better long-term TMR at the ear farther away from the masker (Culling et al., 2012; Sheffield et al., 2020). Relative to the co-located case, this better long-term TMR improves the speech-reception threshold (the TMR corresponding to, for example, 50% correct performance). The head shadow effect results in similar speech-reception threshold improvements (about 6 dB) for NH (e.g., Hawley et al., 2004) and BICI listeners (e.g., Loizou et al., 2009). The range of speech-reception-threshold improvement due to head shadow, from 4 to 8 dB (Culling and Lavandier, 2021), depends on the azimuth of the noise (Bronkhorst and Plomp, 1988; Zurek, 1993) and number of maskers (Hawley et al., 2004). However, this advantage is not available in the case of symmetrically placed maskers that each have the same long-term level and spectrum; because neither ear has a better long-term TMR, listeners cannot benefit by simply listening to the better ear.

For NH listeners, SRM also includes taking advantage of the different ITDs and ILDs associated with the target and masker to improve signal detection. In the case of a noise masker that produces mainly energetic masking (i.e., masker energy exists in the same auditory filters as target energy), this process may be mediated by a binaural equalization-cancellation mechanism (Durlach, 1963; Culling, 2007; Wan et al., 2014) that allows the central auditory system to detect the target speech at a lower TMR. By taking advantage of head shadow and binaural cues, NH listeners typically obtain 7–11 dB of SRM in asymmetric configurations [see Bronkhorst (2000) for a review; see also Dieudonné and Francart (2019) for a discussion of differentiating binaural contributions to SRM from contributions arising from information redundancy].

In the case of a speech masker that also produces informational masking (i.e., the inability to segregate and selectively attend to one stream in the presence of competing streams; Kidd et al., 2008), NH SRM can be as large as 10 dB (for three talkers; Hawley et al., 2004) or even 18 dB (for target and masker speech in different spectral bands; Arbogast et al., 2002). It is possible that the different ITDs and ILDs from the different source locations contribute to release from informational masking by facilitating stream segregation. The stream segregation process might involve central mechanisms different from those required for localization (Middlebrooks and Waters, 2020).

B. Contributions to SRM with symmetric maskers for NH listeners

Symmetric spatial configurations, where two speech maskers are placed to either side of a frontal target, offer a unique scenario to examine binaural factors and glimpsing because listeners cannot simply rely on attending to an ear with a better long-term TMR. Unlike situations with one speech masker, with two speech maskers, the time-frequency intervals containing the most favorable TMR can vary rapidly across ears, depending on which speech masker is more intense at a given moment. Depending on the speech corpus, NH listeners can obtain anywhere from 5 to 15 dB of SRM for symmetrically placed maskers relative to the co-located case where the target and both maskers are in front (Marrone et al., 2008; Brungart and Iyer, 2012; Glyde et al., 2013; Hu et al., 2018).

A potential contributor to speech understanding in the presence of symmetrically placed maskers is the ability to take advantage of brief spectro-temporal epochs of target-dominated energy in either ear, perhaps by rapidly switching between the ears to integrate better-ear glimpses. Brungart and Iyer (2012) examined this possibility by processing speech mixtures (modified rhyme-test target sentences and masker discourse from The Wealth of Nations (Smith, 2003) with an ideal binary mask (IBM) (Wang, 2005) that performed the better-ear glimpsing and integration on behalf of the listener. The unprocessed target-masker mixtures in each ear were partitioned into time-frequency tiles, and for each tile, the “better ear” with the relatively lower masker energy (and thus higher TMR) was identified. The resulting streams of better-ear tiles were summed to create one ideal monaural better-ear mask (IMBM) stimulus. This process was assumed to represent the ideal case where the better-ear time-frequency tiles were perfectly extracted and combined across ears, frequency, and time. In a condition with symmetrically placed maskers at ±60°, the IMBM did not provide a statistically significant advantage over the unprocessed case, which suggests that NH listeners might perform ideal better-ear glimpsing and across-ear integration on their own.

To probe the potential contribution of binaural unmasking to SRM with symmetrically placed maskers, Brungart and Iyer (2012) also included a “hybrid” condition that presented the better-ear time-frequency tiles to one ear (IMBM stimulus) and the worse-ear tiles (i.e., those tiles with the lower TMR) to the other ear. Compared to the IMBM stimulus, the hybrid stimulus added binaural cues (i.e., the original ITD and ILD information was preserved within each time-frequency tile, albeit sometimes left-right reversed) but limited energetic-masking benefit since all the worse-ear time-frequency tiles had a lower TMR than their corresponding better-ear tiles. The hybrid stimulus only increased SRM by a non-significant amount of 1 dB relative to the IMBM condition, suggesting that SRM reflected mainly better-ear glimpsing and not binaural unmasking. However, this study examined a condition with relatively low informational masking. For high informational-masking conditions, subsequent studies (Glyde et al., 2013; Best et al., 2015, Hu et al., 2018) have found an IMBM stimulus to produce less SRM than for a natural condition with symmetrically placed maskers. This suggests that additional binaural processing factors beyond better-ear glimpsing likely contribute to SRM for NH listeners. This conclusion corroborates modeling studies that have identified an important role for both better-ear glimpsing and binaural unmasking in explaining SRM across a variety of symmetric masker conditions (e.g., Culling and Mansell, 2013; Collin and Lavandier, 2013; Ewert et al., 2017; Biberger and Ewert, 2019; Vicente et al., 2020).

C. Contributions to SRM with symmetric maskers for BICI listeners

For listeners with BICIs, SRM is severely limited for symmetric masker configurations (Rana et al., 2017; Hu et al., 2018; Yun et al., 2021). Evidence that better-ear glimpsing limitations contribute to SRM deficits is provided by the finding that BICI listeners obtain a relatively large advantage from IMBM processed speech compared to symmetric speech maskers, even in the case of relatively low informational masking (Hu et al., 2018). This result contrasts with NH (Brungart and Iyer, 2012) and hearing-impaired (HI) listeners (Best et al., 2015), who show similar performance between IMBM and natural symmetric masker conditions. BICI listeners have also been shown to lack binaural-unmasking benefits (e.g., Loizou et al., 2009). An exception is some high-performing listeners presented with stimuli with potentially high levels of informational masking in configurations that maximized perceived spatial separation (Bernstein et al., 2016). These results imply that BICI listeners are generally unable to perform better-ear glimpsing and binaural unmasking on their own.

The better-ear glimpsing and binaural unmasking processes, whose failure is hypothesized to be responsible for BICI deficits, could be mediated by several different distortions associated with CI processing. First, CI spectral resolution is restricted by the broad spread of excitation associated with electrical stimulation that depends on electrode placement and neural degeneration (Kalkman et al., 2015; DeVries et al., 2016), among other factors. A possible role for limited spectral resolution is evidenced by the fact that HI listeners also show reduced SRM in symmetric maskers compared to NH listeners (Best et al., 2015), and signal processing simulations of widening auditory filters were found to have a small but significant impact on SRM for high informational masking conditions (Glyde et al., 2013). Further evidence is provided by studies that measured the impact of spectro-temporal resolution on intelligibility during monaural glimpsing (Li and Loizou, 2008; Conroy et al., 2020). When listening to a stimulus based on ideal time-frequency segregation (a measure of glimpsing ability), the frequency resolution used during time-frequency segregation had a significant impact on NH speech recognition.

Reduced access to TFS information is an additional factor that could limit BICI SRM. For NH listeners, TFS (auditory-nerve phase locking to fast fluctuations on the order of hundreds to thousands of Hz; Moore, 2008) is thought to facilitate binaural unmasking by encoding ITDs between target and masking speech and producing perceived source separation through ITD-based spatial-difference cues (Swaminathan et al., 2016). Most CI processing strategies discard TFS, but because CI listeners show poor monaural pulse rate discrimination and ITD discrimination above 300–500 Hz (Kong et al., 2009; Ihlefeld and Litovsky, 2012; Johnson et al., 2021), providing TFS may not produce a binaural benefit. Finally, asymmetric auditory inputs may further disrupt the precision of binaural cues. While BICI listeners are nominally symmetric in that both ears receive electrical stimulation, in practice, most BICI listeners will experience some asymmetry in the peripheral input due to differences in electrode placement between the ears (Bernstein et al., 2021), differences in the preservation of neurons available for CI stimulation between the ears, or differences in automatic gain control between the two CI processors (Bakal et al., 2021; Archer-Boyd and Carlyon, 2019, 2021). Any of these asymmetries may degrade binaural processing.

The purpose of the current study was to gain insight into the possible sources of BICI SRM deficits that occur for symmetrically separated maskers. Listeners had NH or BICIs. NH listeners were designated as NH VOC when presented CI simulations through vocoder processing and NH control when presented with unprocessed stimuli. Experiment 1 employed the IMBM paradigm of Brungart and Iyer (2012) to assess the possible contributions of glimpsing ability and binaural unmasking to SRM. Experiment 1 also examined whether SRM deficits might reflect asymmetric auditory inputs (by contrasting performance between BICI and NH VOC listeners who have relatively symmetric auditory inputs) or lack of TFS (common to BICI and NH VOC processing). Experiment 2 examined the possible dependence of glimpsing ability and binaural unmasking on spectral resolution by repeating the IMBM experiment for NH VOC listeners while varying the number of vocoder channels.

II. EXPERIMENT 1: SRM AND IDEALIZED BETTER-EAR LISTENING

A. Listeners and equipment

Postlingually deafened BICI listeners (N = 11; mean = 59.4 years, range = 47–71 years) and NH listeners (N = 10; mean = 22.0 years, range = 18–32 years) participated in this experiment. The BICI listeners had at least 2 years of experience with both of their CIs and used 24-electrode nucleus CIs (N24, Freedom, or CI512). CI listener demographic information is shown in Table I. NH listeners had hearing thresholds that were ≤20 dB hearing level measured at octave frequencies between 250 and 8000 Hz and had ≤10 dB of asymmetry in their thresholds at any frequency.

TABLE I.

CI listener demographic information, hearing history, and device information.

Listener	Gender	Age (yr)	Experience (yr) left/right	External device left/right	Onset of deafness	Etiology
S1	F	66	15/8	Freedom/Freedom	Childhood	Unknown
S2	F	61	4/7	Freedom/Freedom	Adult	Hereditary
S3	M	71	8/2	Freedom/N5	Adult	Noise-induced, possibly hereditary
S4	F	66	12/7	Freedom/N5	Childhood	Unknown
S5	F	57	1/5	N5/N5	Adult	Unknown
S6	M	65	2/11	Freedom/Freedom	Childhood	Unknown
S7	F	47	1/4	N5/Freedom	Adult	Otosclerosis
S8	F	57	4/7	N5/Freedom	Adult	Progressive
S9	F	48	4/0.5	N5/N5	Adult	Unknown
S10	F	52	2/9	N5/N5	Childhood	Unknown
S11	F	63	7/10	Freedom/N5	Childhood	Hereditary

BICI listeners were presented with speech signals through Freedom TV/HiFi cables (Cochlear Ltd., Sydney, Australia) into the direct audio input of their clinical sound processors. They used their clinical processors set to their clinically fit everyday program with automatic gain control activated. In cases where the listener had an N5 sound processor, an adaptor was used so that they could use the Freedom TV/HiFi cable.¹ NH listeners were presented with speech signals over open-back circumaural headphones (Sennheiser HD650; Hanover, Germany).

All listeners were tested in a standard double-walled, sound-attenuating booth at the University of Maryland, College Park. Experiments were controlled by a personal computer running matlab (MathWorks, Natick, MA); stimuli were presented through a sound card (UA-25 EX Edirol/Roland Corp., Los Angeles, CA) and an amplifier (D-75A, Crown Audio, Elkhart, IN). Even though there were two listener groups (BICI and NH), the NH listeners were tested with both vocoded and unprocessed stimuli. For convenience, the term listener modality is used to differentiate between the three combinations of listener groups and stimulus processing: the BICI listeners, the NH listeners tested in the vocoded condition (NH VOC), and the NH listeners tested with unprocessed stimuli (NH control).

B. Conditions

Target and speech maskers were spatialized using pinna-related head-related transfer functions (HRTFs), similar to Brungart and Iyer (2012), to create five conditions where the target speech was always at 0° azimuth and the maskers were either co-located with the target or spatially separated symmetrically at ±60°. The original pinna-related HRTFs from Brungart and Iyer (2012) were used instead of those from behind-the-ear microphones as would occur in a CI to better be able to compare against the previous study, and this is also similar to the approach used in Loizou et al. (2009). Behind-the-ear microphone HRTFs would change the ILDs and glimpses (Culling et al., 2012; Mayo and Goupell, 2020; Gray et al., 2021). In some conditions designed to assess hypotheses related to better-ear glimpsing and binaural unmasking, IMBM signal processing (Brungart and Iyer, 2012) was applied to automatically extract better-ear (or poorer-ear) glimpses in each time-frequency unit. The conditions were:

• (1)Co-located: Where target and maskers were presented binaurally at 0° azimuth. This served as a baseline control condition from which to evaluate SRM.
• (2)Symmetric: Where maskers were placed at ±60°. This served as the natural symmetric masker conditions to which SRMs in the idealized conditions were compared.
• (3)Monaural: Same as the symmetric condition except with the combined target and masker stimulus presented to one ear only (randomly the left or right ear on each trial). Target and masker energy in the other ear was set to zero. This served as a control condition to test for any possible long-term head shadow advantage in the symmetric condition while eliminating any possible benefit of redundancy (the same information presented to both ears) that would be available in the co-located condition.
• (4)IMBM: A better-ear monaural signal was presented diotically, effectively carrying out the better-ear glimpsing process on the listener's behalf. Separate time-frequency matrices were generated for the spatialized left and right ear mixtures. These matrices entailed partitioning the mixture into 128 frequency bins using an ERB_N (equivalent rectangular bandwidth of a NH auditory filter; Glasberg and Moore, 1990) scale from 80 to 5000 Hz and then dividing the signals into 10-ms time windows. The root-mean-square energy for each masker was computed for each time-frequency unit, referred to as a “tile.” The level of the combined masker energy was then compared across the ears. For each tile, the ear with masking energy resulting in a higher TMR was designated as TMR↑, and the ear with the lower TMR was designated as TMR↓. The IMBM was created by combining the right-ear TMR↑ and left-ear TMR↑ tiles. Thus, the IMBM contained, for each tile, the signal from the ear with the better TMR, while the tiles from the TMR↓ ear were discarded.
• (5)Hybrid: The IMBM was presented to one ear, while a signal created by combining the previously discarded, “worse-ear” TMR↓ tiles was presented to the other ear. By preserving the magnitudes of the ILDs and ITDs within each tile, this condition had the potential to restore (relative to the IMBM condition) some of the binaural cues that would ordinarily occur in the symmetric stimulus, with the caveat that the direction of the ITD for some tiles could be reversed, possibly causing portions of the perceived source locations to swap to the opposite side of the head. In this case, the target would have a stable position, but the masker locations have the potential to become blurred. Because the “worse-ear” signal had a lower TMR than the “better-ear” signal within each tile, it was assumed that the energetic-masking or information redundancy benefits provided by the hybrid condition were limited. Therefore, SRM improvement in the hybrid compared to the IMBM stimulus was likely to be driven by binaural unmasking due to the availability of ILDs and ITDs. The validity of these assumptions is addressed in Sec. IV.

Via pairwise comparisons of the magnitude of SRM between various combinations of these conditions, the experiment tested three main hypotheses regarding the source of the SRM limitations for BICI listeners.

• (H1)Glimpse ability advantage (IMBM vs symmetric): We hypothesized that listeners in BICI and NH VOC modalities would obtain significantly more benefit from the IMBM stimulus than NH controls. If this hypothesis is confirmed [as it was in Hu et al. (2018)], it will potentially indicate that NH listeners are better at extracting and integrating glimpses than BICI listeners.
• (H2)Binaural unmasking advantage (hybrid vs IMBM): We hypothesized that listeners in BICI and NH VOC modalities would obtain less benefit from the addition of binaural cues to the IMBM stimulus than NH controls. If this hypothesis is confirmed, it will provide support for binaural unmasking failure as a contributor to SRM limitations.
• (H3)Symmetric input advantage (VOC vs BICI): Benefits from better-ear glimpses or spatial cue comparisons across the ears are hypothesized to be partially limited by asymmetric auditory inputs. Assuming that most BICI listeners experience some form of peripheral asymmetry, BICI listeners are predicted to obtain greater benefit than NH VOC listeners from the IMBM vs the symmetric stimulus (indicating poorer glimpsing ability) and less benefit than NH VOC listeners from the hybrid vs IMBM stimulus (indicating poorer binaural unmasking).

Two additional results were expected based on previous literature: (1) based on evidence for reduced SRM for BICI listeners (Rana et al., 2017; Hu et al., 2018), SRM in the symmetric condition was expected to be larger for NH controls than for NH VOC or BICI listeners; and (2) based on the idea that no long-term TMR advantage should be available in a symmetric masker condition for any listener group, performance in the monaural compared to the co-located condition was not expected to differ significantly between the three listener modalities (NH control, NH VOC, BICI). Finally, hypotheses regarding the contributions of spectral resolution or TFS processing to the binaural unmasking deficit are addressed in experiment 2.

C. Stimuli

The target speech consisted of randomly selected sequences of three digits from the triple-digit corpus spoken by a female talker (Wilson et al., 2006). Target sentences were all approximately the same length, with a mean duration of 1750 ms and a standard deviation (SD) of 130 ms. The small set size of the triple-digit stimuli made it possible to test at relatively lower TMRs than previous studies (e.g., Hu et al., 2018). Masker speech consisted of two segments of text of the same duration as the target speech, each spoken by a female talker, and chosen on each trial with a different random starting point (without regard to word boundaries) from a long-duration recording of The Wealth of Nations (Smith, 2003). Compared to stimuli with similar speech corpora [i.e., the CRM corpus used for target and masker speech in Marrone et al. (2008)], the distinct target and masker corpora used in this study reduced the influence of informational masking. While different-gendered target and masker speech would likely have reduced informational masking even further for the NH listeners, it is unclear if this would have influenced the results for the CI listeners; gender differences did not improve performance in speech-on-speech masking conditions in Bernstein et al. (2016), but CI listeners appear to be able to use sex differences to obtain masking release in other studies [see Richter et al. (2021) for a review]. On each trial, the two concurrent maskers started before and ended after the target speech, with the onset and offset delay dependent on the duration of silence before and after the target speech in each individual triple-digit recording (onset delay: mean = 339 ms, SD = 51 ms; offset delay: mean = 338 ms, SD = 74 ms). At the end of each three-digit presentation, the listener used the keyboard or mouse to respond.

For the NH listeners, the stimuli were processed with an eight-channel noise vocoder in half the trials and were unprocessed in the other half. The vocoding procedure followed Hopkins et al. (2008), except that noise carriers were used instead of tone carriers (Bernstein and Brungart, 2011). The stimuli were first passed through a bank of finite impulse-response filters to partition the signal into eight frequency channels with band frequency edges from 100 to 10 000 Hz constant bandwidth on the ERB_N scale according to Glasberg and Moore (1990) and a stop band of −65 dB. The filter order was variable such that the transition band between the −6 dB cutoff frequency defined by the ERB_N scale and the stop band was approximately 0.25 octaves wide. The amplitude envelope of each channel was extracted by taking the magnitude of the Hilbert transform of the bandpass-filtered signal. The amplitude envelopes were used to independently modulate white-noise carriers, and the resulting signals were passed through the original filterbank. The narrowband noise carriers were independently generated for the right and left ears, meaning they were interaurally uncorrelated. The level of each channel was normalized to equal the root-mean-squared level of the unprocessed signal in the same band, and delays associated with the filtering process were removed. The final broadband vocoded sound was created by summing the signals across channels. The vocoder processing occurred after any processing for the IMBM and hybrid stimuli; therefore, the conditions presented to the NH VOC listeners were primarily limited by the spectral resolution of a simulated CI. This is in contrast to Hu et al. (2018), who extracted the IMBM after the envelope extraction.

D. Procedure

An interleaved adaptive procedure was used to first adjust the TMR to the point where listeners achieved 50% correct performance in the co-located condition and then to test the other four conditions at that same TMR. Setting the TMR to ensure that performance was the same across all listeners in this baseline condition then allowed the performance, measured in percentage correct in each of the three spatially separated conditions (symmetric, IMBM, and hybrid), to act as a proxy for SRM. The three digits presented on each trial were independent, and the task was to have the listener enter the three digits they heard by selecting them from a grid on the computer screen using the keyboard or mouse.

Each listening session was divided into blocks that consisted of 120 trials. The first 20 trials were “tracking” trials, intended to adjust the TMR into the range where listeners were achieving 50% correct in the baseline co-located condition. At the beginning of the block, the tracked TMR value was set to 0 dB (NH control) or +8 dB (BICI and NH VOC). In the tracking trials, each listener heard a pair of trials in the co-located condition, one presented at 1 dB above the tracked TMR and one presented at 1 dB below the tracked TMR. The order of these two trials was randomized within each pair. Other than the fact that the two trials included the same set of stimulus parameters and TMRs that were 2 dB apart, the two trials were independent from each other—each contained a different set of three digits and random segments of the maskers, and listeners were required to identify the three digits before the next trial was presented. After the pair of trials, the tracked TMR was adjusted down 1 dB for each correct response in the pair and up 1 dB for each incorrect response in the pair. The tracked TMR could be adjusted anywhere from +6 dB (all six digits incorrect) to –6 dB (all six digits correct) following the two trials. These 20 tracking trials were excluded from the subsequent data analysis.

After the tracking trials, the adaptive procedure switched to testing trials. The TMR was adjusted, and performance was continuously tracked throughout the experiment to produce 50% correct in co-located trials, allowing for ongoing refinement of the tracked TMR. In addition, the other four conditions were tested at this same TMR. To accomplish this, the 100 testing trials were divided into ten sets of ten trials each, with the TMR for all ten trials in the set based on the tracked TMR value obtained in the previous set. Within each set, two trials were collected for each of the five test conditions, one at +1 dB and one at –1 dB relative to the tracked TMR. At the end of each set of 10 trials, the tracked TMR value was adjusted up 1 dB for each incorrect answer and down 1 dB for each correct answer in the two co-located trials. Correct/incorrect performance in the other eight trials in the set (i.e., those for the other test conditions) did not influence the tracked TMR. There were a total of two blocks of 120 trials for each CI listener. The NH listeners completed two blocks for each processing condition (NH control and NH VOC) for a total of four blocks.

E. Results

Figure 1(A) shows the average accuracy in percent correct for each of the three listener modalities and five spatial conditions. Performance in the co-located condition was near 50% in all three modalities as expected, with very little variability across listeners. For the NH control modality, performance was similar between the symmetric and IMBM conditions. For the NH VOC and BICI modalities, there was a substantial increase in performance for the IMBM compared to the symmetric condition. Hybrid processing resulted in an improvement in performance relative to IMBM for the NH control modality but a decrease in performance for the NH VOC and BICI modalities. None of the BICI listeners and only one NH VOC listener obtained greater accuracy in the hybrid than the IMBM condition. The TMRs that the adaptive procedure tracked, computed by averaging the presentation TMR across all of the testing trials, were –19.6 ± 0.7 dB (standard error, SE) for the NH control modality, –5.7 ± 0.3 dB for the NH VOC modality, and –3.1 ± 1.0 dB for the BICI modality [Fig. 1(B)].

FIG. 1.

(A) Experiment 1 accuracy across spatial conditions for NH control, NH VOC, and BICI modalities. (B) Tracked TMRs required for 50% correct performance in the co-located condition for each modality. Error bars represent ±1 standard error (SE) of the across-subject mean. Some conditions have error bars that are smaller than the symbol.

The effects of spatial condition and listener modality on accuracy were analyzed with a linear mixed model using the lme4 package (Bates et al., 2015) in R version 4.0.3 (R Core Team, 2020). A mixed model was chosen to account for the within-subject correlations due to repeated measures and the correlations due to the same NH participants tested in the NH control and NH VOC modalities. We followed a “best-path” procedure (Barr et al., 2013) to determine if inclusion of random slopes improved model fit as assessed by the “anova” function of the lmerTest package (Kuznetsova et al., 2017). The Satterthwaite correction was applied to obtain p-values based on likelihood ratio tests. The initial model included a random effect of listener and fixed effects of spatial condition (five levels), listener modality (three levels), and the condition × listener modality interaction. Random intercepts for different listeners were used, but random slopes for condition did not significantly improve the model fit [χ²(14) = 11.44, p = 0.652]. In other words, the model assumed a similar strength of the effect of condition on accuracy across listeners but allowed for different baseline levels of accuracy.

Through visual inspection of model residuals, we verified that normality and homoscedasticity assumptions for the analysis were reasonably met. In the final model, there was a significant main effect of spatial condition [F(4,118.73) = 185.19, p < 0.001], a significant main effect of listener modality [F(2,34.47) = 10.17, p < 0.001], and a significant condition × listener modality interaction [F(8,118.73) = 13.34, p < 0.001].

Table II presents results of the mixed model from experiment 1. For the model results, the co-located condition and NH control modality served as the reference. Therefore, the main effects reflect SRM for the NH controls. A large SRM was evident in all three spatialized conditions (symmetric, IMBM, and hybrid) as expected. The interaction terms reflect the differences between listening modalities for each test condition. Given that performance in the co-located condition was equalized across listener modalities, the interaction terms represent the difference in SRM between the NH control and NH VOC or BICI modalities in each condition. Larger SRM was observed for the NH control compared to the BICI or NH VOC modalities in the symmetric (14%–22% more SRM) and hybrid conditions (13%–14% more SRM). No significant differences in listening modality were observed for the IMBM condition for the NH control vs NH VOC (p = 1.0) or NH control vs BICI (p = 0.168) comparisons. Furthermore, no significant differences in listening modality were observed for the monaural condition (p = 1.0) as expected. Releveling the mixed model such that the BICI listeners served as the reference for post hoc tests showed that there were no significant SRM differences between BICI and NH VOC modalities for any condition (p > 0.3 for all comparisons), which argues against the hypothesis that reduced SRM in BICI listeners is due to asymmetric inputs (H3).

TABLE II.

Model output of mixed model analysis for experiment 1. Listener modality main effects are reported relative to the NH control mean across all conditions. All other fixed effects and interactions are reported relative to the NH control co-located level (intercept) and thus reflect SRM relative to NH control modality. Significant (p < 0.05, corrected for 15 comparisons) differences are in bold.

	Accuracy
Fixed effects	Estimates	SE	t	p (Bonferroni-adjusted)
Intercept (NH control co-located)	51.08	1.95	26.26	<0.001
Monaural	−7.67	2.54	−3.02	0.012
Symmetric	25.67	2.54	10.10	<0.001
IMBM	23.42	2.54	9.21	<0.001
Hybrid	31.91	2.54	12.56	<0.001
CI	−1.06	2.75	−0.39	1.0
VOC	0.83	2.54	0.33	1.0
Monaural × CI	0.17	3.59	0.05	1.0
Symmetric × CI	−21.98	3.59	−6.11	<0.001
IMBM × CI	8.31	3.59	2.31	0.168
Hybrid × CI	−13.62	3.59	−3.79	0.002
Monaural × VOC	4.92	3.59	1.37	1.0
Symmetric × VOC	−14.25	3.59	−3.96	0.001
IMBM × VOC	1.00	3.59	0.28	1.0
Hybrid × VOC	−13.41	3.59	−3.73	0.002
Random effects	Variance	SD
Residual	32.30	5.68
By-listener intercepts	5.55	2.36

Paired-sample t-tests were used to assess whether accuracy differed between conditions associated with two principal hypotheses: IMBM vs symmetric (hypothesis H1: glimpsing ability) and hybrid vs IMBM (hypothesis H2: binaural unmasking) for each listener modality (Table III). The p-values were Bonferroni-adjusted for six comparisons. IMBM processing yielded a significant performance improvement relative to the symmetric condition for the NH VOC and BICI modalities, supporting the hypothesis that the IMBM conferred additional benefit beyond any better-ear glimpsing that these listeners were able to perform on their own. In contrast, the small accuracy difference between the IMBM and symmetric conditions for the NH control modality was not significant, suggesting that the automated glimpsing provided no additional benefit. There was also significant improvement in the hybrid relative to IMBM conditions for the NH control modality, suggesting a benefit from the added interaural difference cues. No such improvement was observed for the NH VOC and BICI modalities, and in fact, performance was significantly poorer in the hybrid condition, suggesting binaural speech interference.

TABLE III.

Glimpsing ability and binaural unmasking comparisons. Mean differences in accuracy are reported between the IMBM and symmetric (Sym) conditions (to test hypothesis H1: glimpse ability) and between the hybrid (Hy) and IMBM conditions (to test hypothesis H2: binaural unmasking) for each listener modality. The accuracy comparisons serve as a proxy for SRM comparisons because accuracy in each spatialized condition (Sym, IMBM, and Hy) was measured at the TMR that tracked 50% correct performance in the co-located condition. The p-values (corrected for multiple comparisons) and standard errors for the comparisons are also reported. Significant (p < 0.05) differences are in bold.

Modality	Contrast	Estimates	SE	t	p
NH control	H1 IMBM−Sym	−2.2	2.81	−0.80	1.0
	H2 Hy−IMBM	8.5	2.22	3.82	0.025
NH VOC	H1 IMBM−Sym	13.0	3.04	6.38	<0.001
	H2 Hy−IMBM	−5.9	2.03	−2.92	0.103
BICI	H1 IMBM−Sym	28.04	3.83	7.32	<0.001
	H2 Hy−IMBM	−13.4	2.0	−6.74	<0.001

Pairwise comparisons (Bonferroni-corrected for three comparisons) were also conducted to determine whether the TMRs [Fig. 1(B)] differed significantly between listener modalities. A paired-sample t-test was used for the NH control vs NH VOC comparison, and a two-sample t-test assuming unequal group variance was used for the NH control vs BICI and NH VOC vs BICI comparisons. TMR was significantly different between NH control and BICI modalities [–16.5 dB difference, t(15.58) = –13.48, p < 0.001] and between the NH control and NH VOC modalities [–13.9 dB difference t(9) = –21.26, p < 0.001], but not between the NH VOC and BICI modalities [–2.6 dB difference, t(10.3) = –2.44, p = 0.102].

In summary, the results from experiment 1 support the idea that listeners in BICI and NH VOC modalities show less glimpsing ability (i.e., more benefit from the IMBM relative to the symmetric condition, hypothesis H1) and are less able to take full advantage of binaural unmasking cues (i.e., less benefit from hybrid compared to IMBM processing, hypothesis H2) than NH controls. The general patterns in SRM, glimpsing ability, and binaural unmasking were similar between NH VOC and BICI modalities (contrary to the asymmetric input hypothesis H3).

Finally, it is important to note that there was one major difference that might have also contributed to the different pattern of results: the TMR required to yield 50% correct performance in the baseline co-located condition was significantly lower for the NH control than for the other two modalities (16.5 dB lower than NH VOC and 13.9 dB lower than BICI). Monaural glimpsing has been shown to be sensitive to TMR (Bernstein and Grant, 2009), and therefore it is unclear whether the different patterns of results directly reflect the processing differences or instead reflect the different TMRs at which the groups were tested. This issue will be addressed in the analysis of the next experiment.

III. EXPERIMENT 2: SRM AND IDEALIZED BETTER-EAR GLIMPSING AS A FUNCTION OF NUMBER OF VOCODER CHANNELS

The BICI and NH VOC modalities showed a very similar pattern of results in experiment 1, which differed from the pattern for NH controls. There are at least two signal distortions associated with BICI and NH VOC listening that could impact the processes of glimpsing, integration, and binaural stream segregation that underlie speech understanding in the presence of symmetric, spatially separated fluctuating maskers: (1) a lack of encoding of TFS information and (2) poor spectral resolution.

In experiment 1, the effect of spectral resolution was not systematically controlled. The BICI listeners likely varied in how well their CIs encoded spectral information, and the NH listeners were only tested with an eight-channel vocoder. To assess the extent to which spectral resolution played a role in the reduced SRM, glimpsing ability, and binaural unmasking observed in experiment 1, experiment 2 examined the effect of varying the number of vocoder channels in the simulations presented to NH listeners. If spectral resolution shows little influence on these abilities, this would suggest that other aspects of CI and VOC processing, such as a lack of TFS encoding, are likely responsible for the deficits.

Specifically, we hypothesized:

• (H4)Resolution × glimpse ability advantage: Increased number of vocoder channels would significantly improve glimpsing ability, evidenced by a decrease in the performance difference between the IMBM and symmetric conditions.
• (H5)Resolution × binaural unmasking advantage: Increased spectral resolution would significantly improve binaural unmasking, evidenced by an improvement in performance in the hybrid relative to the IMBM condition. This hypothesis was based on the idea that narrow auditory filters (as opposed to the broad representation, which typically occur in CIs due to factors such as current spread) result in better use of within-channel interaural difference cues that are necessary for binaural unmasking.

A. Methods

Eleven NH listeners participated in this experiment, one of whom had also participated in experiment 1. The audiometric inclusion criteria for the NH listeners were the same as in experiment 1, as were the equipment, conditions, and procedures, except that listeners were presented with speech processed by noise vocoders with two changes. First, the number of channels was parametrically varied. There were a total of seven spectral-resolution conditions, including a vocoder with 4, 8, 16, 32, 64, or 128 channels, and an unprocessed control. Second, the vocoder boundaries in this experiment were adjusted to match the bandwidth of the target and interfering speech (100–5000 Hz). While not a perfect CI representation, the use of the noise-vocoder in this experiment allowed for control over the spectral resolution of a CI-like signal. Listeners completed one block of 120 trials for each of the seven spectral-resolution conditions.

B. Results

Figure 2 shows the average accuracy data for the five test conditions as a function of spectral resolution (increase in the number of vocoder channels; the unprocessed condition serves as a control). Figure 2(F) shows that the threshold TMR decreased with increasing spectral resolution, as expected. The data generally show similar trends to those observed in experiment 1. Performance in the binaural co-located condition was at the tracked value of 50% (horizontal dashed line) as expected [Fig. 2(A)], while monaural performance was at or slightly below this value [Fig. 2(B)]. Performance was above 50% for the symmetric condition [Fig. 2(C)], indicating the presence of SRM, while performance was generally higher in the IMBM condition [Fig. 2(D)], indicating that automatically performing the binaural glimpsing and integration on the listeners' behalf improved performance. Finally, performance in the hybrid condition was roughly similar to the IMBM condition [Fig. 2(E)]. These trends were mostly unaffected by the number of vocoder channels, although there were exceptions to this general trend that are discussed in the subsequent analyses.

FIG. 2.

Accuracy for each spatial condition (A)–(E) as a function of spectral resolution in experiment 2. (F) TMRs tracked by the adaptive procedure. Error bars represent ±1 SE. Some conditions have error bars that are smaller than the symbol.

The accuracy data were analyzed using a linear mixed model. Fixed effects were spectral resolution (seven levels: Unprocessed, 4, 8, 16, 32, 64, 128 vocoder channels) and spatial condition (five levels). The initial model included random intercepts for listener. Adding random slopes for spectral resolution significantly improved the model fit according to a likelihood ratio test between models with and without random slopes [χ²(27) = 111.47, p < 0.001], so the final model retained the random slopes. This means that the final model accounted for different baseline levels of accuracy and variation in the slope of accuracy across spectral resolution conditions for different listeners. Model diagnostics did not reveal major departures from normality or homoscedasticity assumptions.

There was a significant main effect of spatial condition [F(4,290) = 414.37, p < 0.001], a significant main effect of spectral resolution [F(6,10.61) = 3.65, p = 0.03], and a significant spatial condition × spectral resolution interaction [F(24,290) = 6.99, p < 0.001]. From Fig. 2, the significant interaction was likely driven by the precipitous drop in accuracy for the four-channel vocoder in the hybrid condition as well as the differences in accuracy for unprocessed vs vocoded trials (especially in the symmetric condition).

Table IV presents results of the linear mixed model for experiment 2. The model reference conditions were the co-located spatial condition and four-channel vocoder processing. The significant main-effect coefficients for spatial condition indicate that for a four-channel vocoder, there was significant SRM (i.e., performance was better than in the co-located case) in the symmetric, IMBM, and hybrid conditions, consistent with experiment 1. As in experiment 1, performance in each of the three spatial conditions serves as a proxy for SRM given that performance in the co-located condition was tracked to be 50% correct. Thus, the interaction coefficients in Table IV can be interpreted as reflecting the change in SRM relative to the four-channel vocoder. For example, a significant IMBM× 8Ch interaction would indicate that SRM for the IMBM condition was different for eight channels compared to four channels. The only significant interaction terms involved the hybrid and symmetric conditions. For the hybrid condition, SRM for the four-channel vocoder was significantly lower than for the eight-channel, 128-channel, and unprocessed cases. Releveling the analysis using different spectral resolution levels as a reference did not reveal significant SRM differences for any other pairwise comparisons in the hybrid condition. For the symmetric condition, SRM for the four-channel vocoder was significantly lower than for the unprocessed control. Releveling the analysis showed that SRM was significantly larger for the unprocessed control than for each of the vocoded conditions (p < 0.01), regardless of the number of channels, but there were no differences between any of the vocoded conditions. This result demonstrates that when unprocessed trials were removed, the significant effect of spectral resolution was limited to the hybrid condition (less SRM for four channels than for eight or 128 channels).

TABLE IV.

Linear mixed model output for the effects of spatial condition and spectral resolution on accuracy (experiment 2). Significant (p < 0.05) differences are in bold. The p-values were Bonferroni-adjusted for 23 multiple comparisons.

Fixed effects	Accuracy
	Estimates	SE	t	p (Bonferroni-adjusted)
Intercept (4Ch co-located)	51.29	1.67	30.70	<0.001
Monaural	−1.51	2.16	−0.70	1.0
Symmetric	8.48	2.16	3.93	<0.001
IMBM	21.36	2.16	9.90	<0.001
Hybrid	12.20	2.16	5.65	<0.001
8Ch	0.04	2.55	0.02	1.0
16Ch	−1.36	2.98	−0.46	1.0
32Ch	−0.45	2.58	−0.18	1.0
64Ch	0.98	2.73	0.36	1.0
128Ch	−0.68	2.79	−0.24	1.0
Unprocessed	2.20	2.62	0.84	1.0
Monaural × 8Ch	1.32	3.05	0.43	1.0
Monaural × 16Ch	−2.12	3.05	−0.69	1.0
Monaural × 32Ch	−1.21	3.05	−0.40	1.0
Monaural × 64Ch	−2.27	3.05	−0.75	1.0
Monaural × 128Ch	2.95	3.05	0.97	1.0
Monaural × unprocessed	−9.17	3.05	−3.00	0.101
Symmetric × 8Ch	4.50	3.05	1.47	1.0
Symmetric × 16Ch	−0.15	3.05	−0.05	1.0
Symmetric × 32Ch	−0.45	3.05	−0.15	1.0
Symmetric × 64Ch	−3.86	3.05	−1.26	1.0
Symmetric × 128Ch	4.47	3.05	1.46	1.0
Symmetric × unprocessed	15.00	3.05	4.91	<0.001
IMBM × 8Ch	1.17	3.05	0.38	1.0
IMBM × 16Ch	4.32	3.05	1.41	1.0
IMBM × 32Ch	0.53	3.05	0.17	1.0
IMBM × 64Ch	−0.68	3.05	−0.22	1.0
IMBM × 128Ch	5.15	3.05	1.69	1.0
IMBM × unprocessed	−3.71	3.05	−1.22	1.0
Hybrid × 8Ch	11.47	3.05	3.76	<0.001
Hybrid × 16Ch	9.47	3.05	3.10	0.07
Hybrid × 32Ch	8.26	3.05	2.70	0.25
Hybrid × 64Ch	8.03	3.05	2.63	0.314
Hybrid × 128Ch	14.09	3.05	4.62	<0.001
Hybrid × unprocessed	12.95	3.05	4.24	<0.001
Random effects	Variance	SD
Residual	23.30	4.83
By-listener intercept	4.60	2.15
By-listener slope for 8Ch	18.18	4.26
By-listener slope for 16Ch	42.31	6.51
By-listener slope for 32Ch	19.93	4.46
By-listener slope for 64Ch	28.0	5.29
By-listener slope for 128Ch	31.15	5.58
By-listener slope for unprocessed	21.91	4.68

The effect of spectral resolution (excluding unprocessed trials) was also analyzed for conditions associated with hypothesis H4 (spectral resolution × glimpsing ability advantage) and hypothesis H5 (spectral resolution × binaural unmasking advantage). Spectral resolution had a significant effect on the binaural unmasking advantage (hybrid vs IMBM accuracy) [F(5,55) = 5.20, p < 0.001], but not glimpsing ability (IMBM vs symmetric).

One possible confound in both these results and those of experiment 1 is that the TMR varied considerably as a function of spectral resolution [Fig. 2(F)]. Because the ability to glimpse speech signals in a fluctuating masker is known to depend on TMR (Bernstein and Grant, 2009), it was important to determine whether TMR differences, rather than spectral resolution per se, might be responsible for some of the observed effect of number of vocoder channels in the hybrid condition. To assess the relationship between TMR and performance in the hybrid condition, we used the repeated-measures correlation package rmcorr (Bakdash and Marusich, 2017) to account for non-independent observations in computing the repeated-measures correlation (R_m). The rmcorr method uses analysis of covariance to estimate the common regression slope among listeners and assesses how well this slope fits individual listeners' data given random intercepts. The approach is similar to a mixed model that assumes random intercepts but parallel slopes.

TMR was not correlated with glimpse extraction [represented by performance in the IMBM condition; R_m = 0.04, p = 0.784; Fig. 3(B)]. In contrast, TMR was moderately correlated with SRM in the hybrid condition [R_m = –0.32, p = 0.017; Fig. 3(A)] and the binaural unmasking advantage, represented by the difference between the hybrid and IMBM conditions [R_m = –0.37, p = 0.005; Fig. 3(C)]. The greatest binaural unmasking advantage occurred at negative TMRs, and the greatest binaural unmasking deficit occurred at positive TMRs. However, it is important to note that the observed significant relationships were driven by the four-channel vocoder condition. The steep negative slopes in Figs. 3(A) and 3(C) would not be present if the four-channel condition was removed from the analysis. In the case of a four-channel vocoder, we cannot rule out the possibility that TMR rather than vocoder processing per se contributed to the reduced binaural unmasking. For higher numbers of channels, the similar results for hybrid accuracy [Fig. 3(A)] and binaural unmasking [Fig. 3(C)] across vocoder conditions, despite differences in TMR, contradict the idea that binaural unmasking deficits are dependent on TMR changes, at least for the –15 to –5 dB range.

FIG. 3.

(Color online) Correlation between TMR and binaural advantage metrics. [(A) and (B)] Accuracy for hybrid or IMBM stimuli as a function of TMR. (C) Binaural unmasking as a function of TMR. The fitted line represents the slope associated with the repeated-measures correlation (R_m) for the data. See text for details.

In summary, the results of experiment 2 largely mirrored the BICI and NH VOC results from experiment 1, regardless of the number of vocoder channels. The primary result was that glimpse ability was minimally affected by spectral resolution. Therefore, the hypothesis that spectral resolution would affect glimpsing ability (hypothesis H4) was not supported. Furthermore, there was no evidence of any effect of spectral resolution on binaural unmasking between 8 and 128 channels. Due to the possible TMR confound, the four-channel results are inconclusive with respect to the hypothesis that spectral resolution moderates the binaural unmasking advantage (hypothesis H5). It could be that binaural unmasking was limited by poor spectral resolution, or it could be that binaural unmasking is generally smaller for TMRs near 0 dB than for lower TMRs.

IV. GENERAL DISCUSSION

Two experiments investigated the possible contributions of deficits in glimpsing ability and binaural unmasking to the challenges faced by BICI listeners in obtaining SRM for symmetrically placed maskers. Experiment 1 evaluated glimpsing ability by comparing performance in an IMBM condition (which performed better-ear glimpsing on the listener's behalf) to a natural symmetric masker condition. In contrast to the NH control modality where no difference between the IMBM and natural symmetric maskers was observed, listeners in the BICI or BICI-simulated (NH VOC) modalities showed a benefit from IMBM, consistent with the idea that deficiencies in glimpsing ability contributed to reduced SRM (hypothesis H1).

Experiment 1 also evaluated the ability to benefit from binaural cues reintroduced in each time-frequency bin of the IMBM stimulus by presenting all of the poorer-TMR tiles to the second ear. In contrast to the NH control modality where hybrid presentation provided a benefit relative to the IMBM condition, BICI and NH VOC modalities showed poorer performance in the hybrid condition. One interpretation for the lack of benefit from hybrid presentation is that reduced binaural unmasking contributed to reduced SRM (hypothesis H2). The assumption that the hybrid vs IMBM comparison mainly reflects binaural unmasking requires scrutiny. An alternative view is that the addition of worse-ear tiles provided an information redundancy benefit in some cases (especially when the short-term TMR of the better-ear and worse-ear tiles were similar). This viewpoint is weakened by the observation that adding glimpses with a lower short-term TMR to glimpses with a higher short-term TMR had little effect on intelligibility according to Tang (2022). Moreover, the unexpectedly degraded performance in the hybrid compared to IMBM processing suggests that across-ear interference may have occurred. This implicates a deficit in stream segregation and is discussed further below.

Finally, experiment 1 also examined whether asymmetric inputs across the ears contributed to the observed deficits in glimpsing ability and binaural unmasking. There were no significant differences in the pattern of results for the NH VOC compared to the BICI modality. We interpret this result as providing no support for the idea that SRM is limited by a lack of symmetric auditory inputs (hypothesis H3). Another possibility is that significant differences between BICI and NH VOC listeners were obscured due to large variation in electrode-to-neural interfaces across BICI listeners or a lack of important processing features in the vocoder (e.g., non-synchronized N-of-M channel selection). Furthermore, factors beyond the periphery (i.e., amount of experience listening through a CI) may interact with peripheral factors in determining sensitivity to binaural cues that are important for SRM (Thakkar et al., 2020). In any case, the fact that the BICI and NH VOC modalities produced similar results motivates the consideration of distortions common to BICI and NH VOC processing as possible culprits for limited glimpsing ability and binaural unmasking. Two possibilities, reduced spectral resolution and lack of TFS encoding, were addressed in experiment 2.

Experiment 2 employed vocoder simulations to examine whether glimpsing ability (hypothesis H4) or binaural unmasking (hypothesis H5) would be impacted by changes in spectral resolution. For glimpsing ability, the number of vocoder channels had no significant effect. Binaural unmasking decreased only in the most severe condition tested—reducing the number of vocoder channels to four—although this effect could be ascribed to spectral resolution or the higher TMR at which listeners were tested in this condition. Overall, there was little evidence that SRM, binaural unmasking, or glimpsing ability were affected by spectral resolution, suggesting that lack of TFS encoding or some other common aspect of CI and VOC processing was responsible for these deficits.

A. SRM deficits related to glimpsing ability

The results of experiment 1 showing an advantage from IMBM processing in the BICI and NH VOC modalities [Fig. 1(A)] support the hypothesis H1 that CI and VOC processing limit glimpsing ability, similar to findings from Hu et al. (2018). Our finding of no significant SRM difference between IMBM and natural symmetric masker conditions for NH controls is also in agreement with previous literature (Brungart and Iyer, 2012; Best et al., 2015; Hu et al., 2018). Together, these IMBM results suggest that BICI listeners have reduced glimpsing ability compared to NH listeners, which contributes to the reduced SRM they experience with symmetric maskers.

To compare the magnitude of SRM and the effect of IMBM processing to the previously reported data from Hu et al. (2018) requires conversion of the estimates reported in Fig. 1, expressed in percentage points, to decibels. The slope of the psychometric function was estimated by comparing mean percentage-correct performance for the half of the trials collected with a TMR offset of +1 dB to the half collected with an offset of –1 dB from the tracked TMR. For the symmetric condition, the NH listeners had a slope of 4.2%/dB for unprocessed stimuli and 4.4%/dB for vocoded stimuli, while the BICI listeners had a slope of 5.5%/dB. Thus, the 26% (NH control), 11% (NH VOC), and 8% (BICI) of SRM observed in the symmetric condition (Fig. 1) translate to about 6, 3, and 1 dB, respectively, whereas Hu et al. (2018) reported 10, 0, and 0 dB. Likewise, the 23%, 24%, and 32% of SRM observed in the IMBM condition translate to about 6, 6, and 6 dB for the NH control, NH VOC, and BICI modalities, respectively, whereas Hu et al. (2018) reported 6, 5, and 3 dB. Considering that the target speech corpus, masker characteristics, TMRs, and procedures differed between the two studies, these values appear to be in reasonable agreement.

If glimpsing ability is important for SRM, then it is worthwhile to consider the acoustic criteria necessary for successful glimpsing with two auditory inputs. The signal degradation of a CI has important ramifications for the information that could be successfully used to extract and integrate glimpses across the ears. Modeling studies varying the glimpse duration, frequency resolution, and short-term TMR for defining better-ear glimpses may yield further insight into BICI SRM deficits.

Modeling studies that process speech-in-noise mixtures using an IBM framework have been used to predict monaural speech intelligibility (e.g., Cooke, 2006; Gibbs and Fogerty, 2018) and SRM for NH listeners (Vicente et al., 2020). These studies have identified the optimal acoustic criteria (i.e., the short-term TMR, glimpse duration, or time-frequency resolution) for defining glimpses in the IBM framework that best explain variance in speech intelligibility. While monaural glimpses are not as predictive of intelligibility for CI-processed stimuli (Fogerty et al., 2016), there is evidence that long-duration glimpses [500ms (Nelson et al., 2003) for CI listeners; 200ms (Fogerty et al., 2016) for NH-vocoded listeners] are important for intelligibility with a CI. An important unanswered question is whether long-duration glimpses support intelligibility for binaural maskers. If, for example, BICI SRM is best predicted using a 500-ms window for monaurally defining a glimpse at either ear (using the IMBM framework), this would implicate a sluggish monaural glimpse extraction mechanism as a factor limiting SRM. Future work should also assess how the short-term TMR and time-frequency region for monaurally defining a glimpse impact intelligibility predictions. These modeling results should yield insight regarding the critical acoustic information needed before BICI listeners could reasonably employ better-ear glimpsing.

It should also be noted that successful SRM may require more than an ability to effectively extract and integrate glimpses. In Best et al. (2015), HI listeners obtained equivalent SRM in IMBM compared to natural symmetric masker conditions, yet they remained at a significant disadvantage for SRM with symmetric maskers. It is noteworthy that the stimulus set in Best et al. (2015) was relatively large and open compared to the small, closed set of digits in the present study, which could have led to a larger effect of spectral resolution on SRM. The contributions of other sources of SRM deficits such as binaural unmasking also warrant discussion.

B. SRM deficits related to binaural unmasking

The results of experiment 1—showing that NH controls benefitted from hybrid processing but that no such benefit was observed for BICI and NH VOC modalities—support hypothesis H2 that CI and VOC processing limit binaural unmasking. Binaural cues (in addition to better-ear glimpsing) have been shown to play a critical role in facilitating NH SRM. Ewert et al. (2017) analyzed SRM with IMBM stimuli as well as stimuli designed to preserve different spatial cues. Based on model predictions, better-ear glimpsing and preservation of ILD cues (which also provided monaural spectral coloration cues) contributed equally to explaining variance in the data, with ITD cues contributing less. Ewert et al. (2017) also found that SRM was better for an idealized stimulus that combined IMBM with infinite ILD processing than for a stimulus that only provided infinite ILDs. This latter result was replicated in Hu et al. (2018) for BICI and NH listeners. Hu et al. (2018) concluded that SRM for BICI listeners may be aided by approaches that accomplish better-ear glimpsing on behalf of the listener as well as enhance perceived spatial separation.

While we hypothesized that relative to NH controls, the benefit from hybrid processing would be diminished for BICI and NH VOC listener modalities due to impoverished binaural cues, surprisingly, the hybrid condition resulted in a SRM decrease compared to IMBM processing [Figs. 1(A) and 2(E)]. In other words, adding a signal dominated by masker energy to the second ear reduced performance. This effect mirrors the results for a comparable condition presented to NH and HI listeners in Best et al. (2015). When the target and masker stimuli were highly confusable, hybrid presentation decreased performance relative to the IMBM condition. When the target and masker sentences were more easily distinguishable, hybrid presentation improved performance. Best et al. (2015) suggested that their result may have arisen due to “blurred apparent locations” of the masker that made them more confusable with the target. This blurring is related to the fact that the hybrid condition preserves the magnitude of binaural cues within each time-frequency unit, but not the direction. A related finding was reported by Sheffield et al. (2019), whereby localization cues were degraded and inconsistent across channels while keeping within-channel binaural-difference cues intact; this processing reduced speech understanding for highly confusable, symmetric, multi-talker maskers but not for speech-shaped-noise maskers.

Thus, mere masker suppression (i.e., through equalization-cancellation; Culling 2007; Wan et al., 2014) might not be sufficient to guarantee SRM for symmetrically placed talkers. Accurate localization and, more broadly, stream segregation (Ihlefeld and Shinn-Cunningham, 2008; Schoenmaker et al., 2016; Middlebrooks and Waters, 2020) seem important for SRM in such conditions. While the current study used target and masker stimuli that were designed to be readily distinguishable, the lack of pitch cues in BICI and vocoder processing might have rendered the target and masker more confusable (Qin and Oxenham, 2006). Therefore, successful SRM might have been more reliant on perceived spatial location (Freyman et al., 2001). Deficiencies in better-ear glimpsing and binaural unmasking may be related in BICI listeners. Both could stem from a sluggish monaural glimpsing mechanism. Future studies should address the extent to which stream segregation facilitates binaural unmasking, independent of better-ear glimpsing in BICI listeners.

Finally, the detrimental effect of adding poorer-ear time-frequency tiles to the IMBM may be related to susceptibility to contralateral interference for asymmetric CI listeners. In addition to providing binaural cues that promote stream segregation, the end result of the hybrid processing was to present a signal dominated by the target speech to one ear and a signal dominated by the masking speech to the other ear. It could be that listeners had a difficult time ignoring the interfering speech associated with the lower TMR in the contralateral ear (which would not have been present in the baseline co-located condition). Several recent studies have found that in cases of asymmetric CI hearing, adding an interfering voice to a demonstrably poorer ear can cause inferior performance relative to listening with only the better ear [for adult BICI and single-sided deaf CI listeners: Goupell et al. (2018), Bernstein et al. (2020); for NH-vocoded listeners: DeRoy Milvae et al. (2021), Goupell et al. (2021)]. However, this explanation is partially weakened by the fact that the CI subjects in the current study did not show the dramatic asymmetries in monaural speech understanding or duration of deafness that are thought to underlie this phenomenon (Goupell et al., 2018) and that the NH listeners were presented with symmetric vocoder processing, which has not been shown to produce the interference effect (DeRoy Milvae et al., 2021; Goupell et al., 2021). It is possible that differences in the stimuli and procedures could explain the inconsistencies. For example, the fact that the stimulus conditions were interleaved meant that listeners would not have had an opportunity to become accustomed to the hybrid presentation. If the conditions had been separately blocked instead, listeners might have had the opportunity to learn to ignore the salient but irrelevant masker-dominated speech in the contralateral ear, reducing the interference effect.

C. The limited effect of spectral resolution on SRM mechanisms

In contrast to hypothesis H3, experiment 2 (Fig. 2) found that manipulating NH VOC spectral resolution had a limited effect on SRM, glimpsing ability, and binaural unmasking. The only condition where a significant effect was identified was in the case of a four-channel vocoder where binaural unmasking appeared to be reduced; however, this result might reflect TMR rather than spectral-resolution effects.

The lack of an effect of spectral resolution on SRM is consistent with a study of simulated spectral-resolution impairment in NH listeners by Glyde et al. (2013). Broadening IMBM filters to simulate HI auditory filters had no significant effect on SRM for maskers that produced low informational masking, although this manipulation had a small but significant effect for maskers with high informational masking. Even more pertinent to BICI listeners, Garadat et al. (2009) found that increasing the number of vocoder channels had no significant effect on any of the primary speech-perception advantages associated with having two ears (head shadow, squelch, or redundancy).

A potentially important confounding factor in this study is that the TMR was adjusted to produce 50% accuracy in the baseline co-located condition and varied across listener modalities [see Fig. 1(B)] and spectral resolution [see Fig. 2(F)]. In addition to the importance of TMR for monaural glimpsing (Bernstein and Grant, 2009), SRM for vocoded stimuli is larger for negative TMRs and negligible for positive TMRs (Freyman et al., 2008). Therefore, some of the SRM differences between listener modalities in this study could have been due to different testing TMRs. This point is particularly pertinent for binaural unmasking variability given that increasing TMR was associated with greater contralateral interference in Goupell et al. (2018). However, while an analysis of TMR effects showed that the four-channel vocoder, which produced TMRs near 0 dB, was associated with poorer binaural unmasking, this pattern was not observed for the higher numbers of vocoder channels where TMRs spanned −15 to −5 dB (Fig. 3). Recall that TMRs for listeners in the NH control and NH VOC modalities for experiment 1 were approximately −20 and −6 dB, respectively. Unless significant binaural unmasking differences exist between TMRs of −20 and −15 dB (a possibility we cannot rule out), it is unlikely that the binaural unmasking discrepancies between listener modalities in experiment 1 were due to TMR differences. Likewise, while the older age of BICI compared to NH listeners is a potential confound for SRM (Gallun, 2021), the similar performance between listeners in BICI and NH VOC modalities mitigates this concern.

D. Potential contribution of TFS to SRM

The lack of evidence for a role for asymmetric auditory inputs (experiment 1) or spectral resolution (experiment 2) in limiting SRM, glimpsing ability, and binaural unmasking for BICI listeners suggests that other factors inherent to CI processing underlie these deficits. Although not examined directly here, the lack of TFS encoding in BICI and NH VOC processing is a likely contributor to the BICI-NH SRM discrepancy. This point is underscored by the observation that SRM in the symmetric condition was significantly higher for unprocessed vs vocoded stimuli with as many as 128 channels [Fig. 2(C)]. These two conditions produced similar spectral detail, but a 128-channel vocoder replaced natural TFS with interaurally uncorrelated noise.

Given that access to TFS ITDs has been shown to be important for SRM in NH listeners (Swaminathan et al., 2016), it is reasonable to expect reduced TFS ITDs to explain the limited SRM in listeners with impaired hearing. Support for this interpretation is limited at present. While Strelcyk and Dau (2009) found some measures of binaural TFS sensitivity to be correlated with SRTs in HI listeners, results from two more recent studies (King et al., 2017; Baltzell et al., 2020b) failed to identify such a correlation. Using tone vocoding to manipulate ITDs in the TFS of correlated noise carriers, King et al. (2017) found that the correlation between monaural TFS sensitivity and SRM was not significant in HI listeners after controlling for audiometric thresholds below 1.5 kHz. Similarly, when examining SRM across a range of interaural TFS correlation, Baltzell et al. (2020b) found that SRM was generally larger for younger NH compared to younger HI listeners, even when the degree of interaural TFS correlation was manipulated such that ITD thresholds were similar between listeners. Moreover, low-frequency audiometric threshold explained more of the SRM variance than ITD sensitivity.

Direct evidence for the effect of TFS on binaural advantages promoting SRM in CI listeners is also lacking. A unique approach to investigating whether access to TFS afforded SRM advantages in CI-simulated listeners was undertaken by Garadat et al. (2009) by manipulating the order of vocoder and HRTF processing. When HRTFs were applied after the vocoding stage, speech TFS was eliminated, but TFS for binaural cues was preserved. When HRTF was applied before the vocoding stage, both speech TFS and binaural TFS were eliminated. Garadat et al. (2009) found no significant effect of HRTF processing order on any spatial advantage with asymmetric maskers, though there was a trend of reduced binaural squelch with the HRTF-before-vocoder processing that eliminated binaural TFS. The authors noted individual variability in SRM, which may have tempered the observed effect of binaural TFS preservation.

The issue of individual variability is pertinent in our study as considerable variability in binaural unmasking advantage was observed, most notably as a function of increasing spectral resolution in experiment 2, with a relatively heterogeneous group of NH listeners [Fig. 3(C)]. The mechanisms responsible for this variability remain unexplained. One possibility is that variability could be related to susceptibility to contralateral interference (Goupell et al., 2018). Another possibility is that given sufficient spectral resolution, some listeners can adopt a weighting strategy that enhances the use of binaural cues at perceptually salient points such as speech onsets (e.g., Hu et al., 2017; Stecker, 2018; Baltzell et al., 2020a). Further investigation is warranted to determine how cue weighting strategies contribute to SRM and are potentially related to listener variability in performance that occurs with changes in spectral resolution and susceptibility to contralateral interference.

V. SUMMARY AND CONCLUSIONS

Consistent with previous results, BICI listeners showed a SRM limitation compared to NH controls for symmetric masking conditions where the long-term head-shadow provides little benefit, a limitation that was mirrored in NH listeners presented with vocoded stimuli. Whereas NH controls received no SRM benefit from an idealized better-ear stimulus (IMBM) compared to simply listening to symmetric maskers, listeners in BICI and BICI-stimulated NH (VOC) modalities received substantial benefit from the IMBM, suggesting that for natural symmetric stimuli they were unable to perform the bilateral glimpsing and integration operation on their own. Adding binaural cues in the hybrid condition resulted in further SRM increases for NH controls but surprisingly resulted in a performance decrease for BICI and most NH VOC listeners. This suggests that not only were listeners in BICI and NH VOC modalities impaired in their ability to benefit from binaural unmasking cues, but they also experienced across-ear interference that further degraded performance. There was little evidence that reduced spectral resolution or a lack of symmetric auditory inputs was responsible for these impairments, suggesting that another aspect of CI and vocoder processing—such as a lack of TFS encoding—underlies the observed deficits in binaural glimpsing and unmasking.