Lateralization of interaural time differences with mixed rates of stimulation in bilateral cochlear implant listeners

Abstract

While listeners with bilateral cochlear implants (BiCIs) are able to access information in both ears, they still struggle to perform well on spatial hearing tasks when compared to normal hearing listeners. This performance gap could be attributed to the high stimulation rates used for speech representation in clinical processors. Prior work has shown that spatial cues, such as interaural time differences (ITDs), are best conveyed at low rates. Further, BiCI listeners are sensitive to ITDs with a mixture of high and low rates. However, it remains unclear whether mixed-rate stimuli are perceived as unitary percepts and spatially mapped to intracranial locations. Here, electrical pulse trains were presented on five, interaurally pitch-matched electrode pairs using research processors, at either uniformly high rates, low rates, or mixed rates. Eight post-lingually deafened adults were tested on perceived intracranial lateralization of ITDs ranging from 50 to 1600 μs. Extent of lateralization depended on the location of low-rate stimulation along the electrode array: greatest in the low- and mixed-rate configurations, and smallest in the high-rate configuration. All but one listener perceived a unitary auditory object. These findings suggest that a mixed-rate processing strategy can result in good lateralization and convey a unitary auditory object with ITDs.

I. INTRODUCTION

For normal-hearing (NH) listeners, binaural hearing enables localization of sound sources on the horizontal plane and improves their speech understanding in noise. The ability of listeners to localize sounds is a product of the ability of the binaural system to utilize spatial cues. These spatial cues are created due to the physical differences in the intensity and time of arrival of a sound between a listeners' two ears, and as a result, allows a listener to perceive a horizontal location of a sound relative to the head (Blauert and Butler, 1985; Wallach, 1938). Two spatial cues are interaural level differences (ILDs) and interaural time differences (ITDs); sensitivity to which is frequency dependent. In the freefield, NH listeners are most sensitive to ILDs at high frequencies and ITDs at low frequencies (Feddersen et al., 1957; Klingel and Laback, 2022; Macpherson and Middlebrooks, 2002; Middlebrooks et al., 1989; Mills, 1959; Wightman and Kistler, 1992). They are also sensitive to ITDs in high-frequency stimuli when modulated by a low-frequency envelope (Henning, 1974; Middlebrooks and Green, 1990).

In contrast to NH listeners, bilateral cochlear-implant (BiCI) listeners do not reach the same level of sound localization performance and may rely primarily on ILDs when locating sound sources in free field (Aronoff et al., 2010; Grantham et al., 2007). One reason for the gap in performance is that when listening through clinical processors, BiCI listeners are unable to access ITDs for low-rate signals, which is typically conveyed through the carrier frequencies of a modulated speech signal, or “temporal fine structure” (TFS) of an acoustic signal (Dennison et al., 2022). Most clinically available strategies have an envelope-extraction process that does a poor job of encoding the across-ear delays in the TFS in the electrical signal (Gray et al., 2021; Laback et al., 2015). Clinical strategies typically use high-rate pulsatile stimulation (≥900 pulses/s) to accurately represent the envelope. Thus, the lack of TFS encoding, combined with the high-rate stimulation, effectively obliterates usable low-frequency ITDs (van Hoesel and Tyler, 2003; Kan and Litovsky, 2015). While there have been attempts to provide TFS using various other strategies (Hochmair et al., 2006; Zirn et al., 2016), there is little evidence that the encoded TFS has translated to useful ITD sensitivity for broadband signals (Fischer et al., 2021). A second reason is that BiCI listeners may have poor neural survival in one or both ears leading to across-ear asymmetries which may affect sensitivity to ITDs (Anderson et al., 2019b; Ihlefeld et al., 2015). A third reason is the known difficulty in ensuring that electrode arrays in the two ears are placed in the same location during surgery. For instance, spiral ganglia of the auditory nerve that are excited at the same position on the electrode array could be mismatched in the neural populations that are adjacent to the array, leading to a mismatch in frequency information. Relatively large insertion depth differences have been shown to result in poorer sensitivity to ITD, for single-electrode (Goupell et al., 2013; Kan et al., 2013, 2015b) as well as multi-electrode stimulation (Kan et al., 2019). Work using computed-tomography scans has shown that the differences in insertion depths between the two ears are relatively small (Goupell et al., 2022; Sokolov et al., 2020). Goupell et al. (2022) showed that the median interaural mismatch in BiCI users was approximately 1.3 mm. Interaural mismatch of 1.3 mm. Hence, interaural mismatch may only be a significant problem in CI users with relatively large mismatches. Finally, there is also a lack of communication between the processors in the two ears which leads to little opportunity to accurately encode ITDs in the pulse timing of the electrical signal, ultimately reducing sensitivity to ITDs (Dennison et al., 2022; Gray et al., 2021).

The limitations regarding the lack of communication between the processors can be partly overcome in a laboratory setting using synchronized research processors. Prior work has shown that even with synchronization, BiCI listeners with adult-onset of deafness show decreasing sensitivity to ITDs when stimulation rate of a constant amplitude pulse train approaches the 400–800 pulses/s (pps) range (Anderson et al., 2019a; van Hoesel et al., 2009; Laback et al., 2015) and can achieve good sensitivity at lower stimulation rates (Kan and Litovsky, 2015). ITD sensitivity in BiCI listeners with adult-onset of deafness typically ranges from 50 to 1000 μs when presented with low rates of stimulation (∼100–300 pps) through synchronized research processors that deliver coordinated stimulation of binaural cues (van Hoesel, 2008a; van Hoesel et al., 2009; Kan and Litovsky, 2015; Laback et al., 2015; Thakkar et al., 2020). These findings, along with prior work that shows that synchronization alone is not sufficient for improvements in localization (Dennison et al., 2022), highlight the need for a strategy that conveys coordinated binaural cues across the two implants.

Previous work has sought to overcome the fine-structure and ITD limitations of current clinical processors by creating strategies that include different stimulation rates across the electrode array (i.e., “mixed” rates). These strategies attempt to convey ITDs in the timing of electrical pulses on some channels by firing them at low stimulation rates. These pulses are usually timed to a certain feature of the acoustic stimulus. In one such strategy [“peak-derived timing” (PDT)], (van Hoesel, 2007), an electrical pulse is presented whenever there is a peak in the bandpass filtered acoustic signal at low-frequency channels. For high-frequency channels, high stimulation rates are still used to better encode speech information. However, benefits of this approach have not yet been shown for binaural hearing tasks (van Hoesel and Tyler, 2003). Churchill et al. (2014) developed a similar strategy that attempted to deliver ITD cues using either low rates, mixed rates, or high rates, but only found improvements in ITD sensitivity and lateralization when only low rates were presented on all channels, which is not ideal for speech representation. Similar to PDT, FS4, or Fine Structure stimulation at the 4 apical-most electrodes, is a strategy designed for the MED-EL cochlear implant that aims to relay TFS information by introducing a pulse at each positive-going zero crossing in the bandpass filter output of a channel, leading to a repetition rate low enough to follow the instantaneous TFS frequency. These pulses are then delivered to the four apical-most channels of the electrode arrays (Hochmair et al., 2006). While the Hochmair et al. (2006) study has shown some improvements in pitch perception, the findings on improvements in ITD sensitivity are mixed. In particular, the FS4 strategy has yielded ITD thresholds ranging from 2.2 to 3.3 ms when compared to high definition continuous interleaved sampling (HDCIS), a high rate-only strategy that yielded no measurable ITD thresholds (Zirn et al., 2016). At least four of the 12 listeners tested in Zirn et al. (2016), showed an improvement from a non-measurable just-noticeable difference (JND) in the all-high strategy, to an ITD JND threshold below 1.85 ms when listening with the FS4 strategy. However, these ITD JNDs were still greater than the largest physiologically relevant ITDs for human sound localization (about 700–760 μs) (Feddersen et al., 1957). This implies that FS4 does not seem to be able to provide ITD cues that are usable for real-world situations.

More recently, a temporal limits encoder (TLE) strategy has been proposed (Meng et al., 2015, 2016) as a way to improve pitch discrimination and Mandarin tone recognition in CI listeners (Zhou et al., 2022). The TLE strategy down-transposes band limited mid-frequency channel information to a lower frequency, resulting in slower envelope modulations. A by-product of this strategy when used bilaterally is that ITD cues are theoretically encoded within the down-transposed envelope modulations, implying a possible use for relaying ITDs to BiCI listeners (Kan and Meng, 2021). The TLE strategy attempts to encode TFS without relying on changes to the stimulation rate or encoding of TFS using the peaks or zero crossings of an acoustic signal. However, psychophysical listening tests have only shown a modest bilateral benefit of the TLE strategy in BiCI listeners.

A limitation of the aforementioned strategies is that only the apical-most electrodes were used to convey ITDs in the pulse timing. As shown by Kan et al. (2015a, 2016), best ITD sensitivity is not restricted to the apical-most electrodes in BiCI listeners; in fact, thresholds were often lowest for basal-most stimulation. Thakkar et al. (2018) showed a similar effect when presenting some electrodes with low-rate stimulation, and other electrodes with high-rate stimulation. Better ITD sensitivity was typically observed when the low-rate stimulation was presented in either the middle or basal regions of the electrode array. In addition, only a small number of low-rate electrodes (one to three) were needed to have ITD sensitivity comparable to that of stimuli with only low stimulation rates presented across the electrode array.

A number of prior studies have shown that while BiCI listeners can reliably detect changes in ITD (Baumgärtel et al., 2017; Ehlers et al., 2017; Fitzgerald et al., 2015; Kan et al., 2019; Litovsky et al., 2012), they may not be able to map ITDs to the full extent of right-to-left intracranial space. This demonstrates that measuring lateralization is important for ascertaining information about the spatial mapping of perceived azimuth to highly controlled cues, rather than just determining a listener's sensitivity to these cues. Further, it is unknown whether multi-electrode, mixed-rate stimulation with a single ITD will induce the perception of a unitary auditory object or multiple auditory objects; this will be important for future studies in which a mixed-rate strategy might be used in the presence of multiple sound sources. A potential consequence of perceiving a unitary auditory object from a multi-electrode stimulus is that it may interfere with the spatial mapping of a real sound source in space. Prior work in NH listeners has suggested that perceptual fusion is difficult to measure but that the inability to process ITDs in a single frequency band, in the presence of activity in other frequency bands, is a consequence of auditory object formation or fusion; also known as “interference” (Woods and Colburn, 1992). Woods and Colburn (1992) modeled object formation by manipulating an ITD in a single frequency band (600 Hz) and presented an ITD of 0 μs at adjacent bands (400 and 800 Hz) in a harmonic complex. Overall, the findings suggested that “interference” can serve as a framework or proxy for auditory object formation. Further, it suggests that reduced ITD sensitivity to a tone with multiple frequency components may increase the opportunity for hearing a single auditory object, yet still result in interference. While the current study did not aim to investigate the impact of varying ITDs across the electrode array, the study by Woods and Colburn (1992) points to the possibility that a multi-electrode mixed-rate stimulus may be influenced by auditory object formation (or lack thereof), ultimately impacting overall ITD sensitivity. Therefore, it is important to determine whether listeners explicitly perceived a single, unitary percept in one spatial location for a mixed-rate stimulus.

The feasibility of a mixed-rate strategy has previously been demonstrated using an ITD discrimination task in our lab (Thakkar et al., 2018). Here, we aimed to expand our understanding of whether listeners with BiCIs stand to benefit from multi-electrode, mixed-rate stimulation for spatial hearing using a lateralization task. Perceived intracranial lateralization was measured to study the functional utility of the ITD cue by measuring how well listeners could map the physical ITD cue to a perceived location in their head. We hypothesized that if low-rate, multi-electrode stimulation of ITDs is in fact superior to a stimulus configuration with only high rates, then we would observe improvements in lateralization range when more low-rate channels are systematically added into a mixed-rate stimulus. Additionally, if a mixed-rate, multi-electrode stimulus negatively impacts auditory object formation, we would expect to see an increasing percentage of trials heard as “two sources.” Taken together, findings from the current study will inform us about the utility of the mixed-rate strategy and whether listeners can map ITD cues to an intracranial lateral location, determining the benefit of mixed rates of stimulation.

II. METHODS

A. Subjects

Eight postlingually deafened BiCI listeners who had participated in the Thakkar et al. (2018) study and had previously demonstrated sensitivity to ITDs were recruited for this study. Listeners traveled to the University of Wisconsin-Madison for 3–5 days for testing and were paid a stipend for their participation. Listener demographics are displayed in Table I. All listeners had Cochlear Ltd. (Sydney, Australia) implants (CI24 and CI512 family of implants). These devices have 24 electrodes (22 intra-cochlear and two ground electrodes). Electrodes are numbered such that 22 is the apical-most electrode and 1 is the basal-most electrode. All listeners used electrode arrays that have a variable spacing between electrodes. The spacing varies from about 0.8–0.4 mm from basal to apical ends. The electrode arrays are designed in this way such that electrodes along the array will have approximately constant angular spacing when properly inserted inside the cochlea. All experimental procedures followed the regulations set by the National Institutes of Health and best practices for direct stimulation studies (Litovsky et al., 2017), and were approved by the University of Wisconsin-Madison Health Science Institutional Review Board.

TABLE I.

Listener demographics and etiology. Table lists age at testing, sex, years of experience with a CI, etiology, and the electrodes used for testing.

Listener ID	Age	Sex	Electrode (left/right)					Years of CI experience (left/right)	Implant type (left/right)	Etiology
			Apex	Mid-apex	Mid	Mid-basal	Base
IBF	64	F	21/21	16/17	12/12	8/10	4/5	8/9	CI24RE/ CI24RE	Hereditary; unknown onset of hearing loss
IBK	75	M	18/22	15/16	14/13	11/10	6/6	12/6	CI24R (CS)/ CI24RE	Hereditary, noise exposure; unknown onset of hearing loss
IBY	51	F	20/18	16/14	12/12	8/11	4/7	7/3	CI24RE/CI512	Unknown; unknown onset of hearing loss
ICB	64	F	18/18	15/14	12/12	8/9	4/4	12/12–15 *listener was re-implanted in the right ear after 3 years	CI24RE/ CI24R (CA)	Hereditary; unknown onset of hearing loss
ICD	57	F	20/18	16/14	12/10	8/6	4/2	6/7	CI24RE/ CI24R (CS)	Hearing loss at 3 years old
ICI	57	F	20/20	12/14	8/10	4/8	2/4	6/5	CI24RE/CI24RE	Hearing loss at 31 years, etiology unknown
ICJ	65	F	20/16	16/15	12/10	8/8	4/6	4/4	CI512/CI512	Hearing loss at 13 years, perhaps due to illness
ICP	52	M	20/20	16/18	12/14	7/11	4/8	6/3	CI24RE/CI24RE	Hearing loss at 3 years old

B. Stimuli and equipment

Stimuli were delivered via direct stimulation using bilaterally synchronized research processors (RF GeneratorXS, Cochlear, Sydney, NSW, Australia). Custom-written matlab (Mathworks, Natick, MA) software was used to generate the stimuli and communicate with the research platform through the Nucleus Implant Communicator (NIC) libraries (version 3 for RF Generator). Stimuli were checked using an oscilloscope before beginning experiments; a standard best practice for CI research (Litovsky et al., 2017). Listeners entered responses to stimuli on a touchscreen monitor connected to the same computer. All stimuli were 300 ms duration constant amplitude pulse trains presented via monopolar stimulation. Each pulse had a 25 μs phase duration and 8 μs inter-phase gap (except for listener ICP who required a phase duration of 75 μs phase duration and 8 μs inter-phase gap). It should be noted that Nucleus implants are designed for continuous interleaved sampling and unable to stimulate multiple electrodes at the same time. In our multi-electrode stimulus, the pulses across electrodes were presented in an apical-to-basal temporal order with a 70 μs inter-pulse timing offset.

C. Loudness mapping

Prior to testing with multi-electrode stimulation, comfortable loudness levels were determined for single interaural pairs of electrodes, using a 300 ms, 100 pps stimulus. These loudness-balanced maps were created to ensure that all mixed-rate configurations would elicit equal loudness and a centered image of the auditory object when the ITD was set to zero. However, it should be noted that auditory object formation was not directly assessed during the mapping process. The map defines the range of current units that generates an audible percept between threshold (T) and maximum comfortable (M) loudness levels at each electrode. Within this range, the experimenter measured a most comfortable (C) level, which we defined as the stimulation level that the listener is willing to be tested at for an extended period. Then, using a single across-ear pitch-matched pair (described in the next paragraph), the experimenter adjusted the C level in each ear until the listener perceived the sound as equally loud and coming from the center of the head.

The five electrode pairs tested here were selected using a similar approach as previous studies, whereby pairs of electrodes with the closest percept of pitch were identified at five regions along the length of the electrode array (Kan et al., 2015a,b; Litovsky et al., 2010). Prior work from our lab has used pitch matching of electrodes as a proxy for selecting electrodes in the two ears that stimulate similar interaural place of stimulation. The assumption is that similar interaural place-of stimulation would yield an electrode pair that can produce a fused auditory object and acceptable binaural sensitivity. Recent work has shown that pitch-matching does not necessarily find an across-ear electrode pair that yields the highest binaural sensitivity due to potential biases that arise from the task and procedure (Goupell et al., 2019; Jensen et al., 2021). Still, prior research shows that pitch-matching procedures typically yield an electrode pair that is within ±4 electrodes of a pair with sufficient binaural sensitivity (Kan et al., 2015b). We note that approaches, such as the binaural interaction component (Gordon et al., 2012; He et al., 2010; Hu and Dietz, 2015), computed-tomography scans (Goupell et al., 2022), or simply matching electrode pairs based on best ITD sensitivity (Bernstein et al., 2021), as opposed to pitch-matching procedures, can also be used for interaural matching of electrode pairs. To be consistent with our prior work, the present study selected five pitch-matched electrode pairs that had been used in prior studies by the authors to measure ITD sensitivity (Kan et al., 2015a; Thakkar et al., 2018, 2020).

Pitch-matched electrode pairs were identified using a direct pitch comparison task whereby listeners compared the perceived pitch of the stimulus in the left ear to the perceived pitch of the stimulus in the right ear; all stimuli were 300 ms, 100 pps pulse trains. On each trial, the left ear electrode was held constant, and the right ear electrode was varied trial by trial in a two-interval, five-alternative forced choice task. The five reference electrodes in the left ear were chosen such that they would be distributed along the electrode array at the following locations: base, mid-base, mid, mid-apex, and apex. The first interval was always the reference electrode in the left ear and the stimulus in the second interval was: “much higher,” “higher,” “same,” “lower,” or “much lower” in pitch compared to the reference electrode. In each block, the reference and test electrodes were chosen randomly for each trial. From the direct pitch comparison task, the pair that yielded the highest number of “same” responses was chosen as the pitch-matched pair for further testing (see Litovsky et al., 2012, for further details on methodology). The five pitch-matched pairs determined for each listener are shown in Table I.

Next, multi-electrode loudness maps were created. Using the five pitch-matched electrode pairs, seven multi-electrode, mixed-rate configurations were created (see Fig. 1). Two of these configurations had a uniform rate across all five electrodes; “High5” presented 1000 pps on all electrodes and “Low5” presented 100 pps on all electrodes. Three other configurations had 1000 pps on four electrode pairs, with the low-rate electrode pairs 100 pps presented at either the apex, middle, or base electrodes (configurations labeled “Apex1,” “Mid1,” and “Base1,” respectively). Two other configurations had 100 pps at three electrode pairs, either the three apical-most or spread across the array (configurations labeled “Apex3” and “Spread3,” respectively), with the remaining two electrode pairs presented at 1000 pps.

FIG. 1.

(Color online) A schematic depicting the seven testing configurations used in the study. The multi-electrode configurations reflect their pitch-matched electrodes of a cochlear array, where lower numbered electrodes (e.g., electrode 4) are basal and higher numbered electrodes (e.g., electrode 21) are apical. The label for each testing configuration is listed above each picture. The legend shows red pulse trains as the high rate (1000 pps) and the dark blue and dark red pulse trains as the low rate (100 pps).

C levels across all configurations were adjusted so that each mixed-rate multi-electrode configuration map was matched in loudness to a reference “High5” map. Multi-electrode configurations were loudness-balanced using the following steps: (1) The experimenter determined C levels unilaterally at single electrodes for 1000 pps at all pitch-matched electrodes. (2) All C levels in the five electrodes within one ear were reduced by 15% of the listener's dynamic range and played simultaneously to ensure that stimulus in each ear alone was not too loud, and no one electrode was noticeably louder than the others. The stimuli in the two ears were then played together to ensure that the bilateral multi-electrode stimulation was comfortably loud and perceived as coming from the center of the head. Thus, mapping began with two High5 (all high maps) that were first unilaterally loudness-balanced across electrodes and then bilaterally centered across the ears. Once the High5 maps were determined for each ear, the C levels for each of the five electrode pairs were no longer adjusted. (3) Mixed-rate maps, where only one electrode had a low stimulation rate, were created by substituting an electrode from the High5 map with a 100 pps stimulation in one ear. To ensure that the mixed-rate map with the introduced 100 pps electrode was equal in loudness to a High5 map, the loudness of the mixed-rate map was compared to the loudness of the High5 map played in the opposite ear. The two comparison maps were unilaterally played one after the other with an inter-stimulus interval of 100 ms. Listeners indicated whether the ear with the mixed-rate map was either softer or louder than the ear with high rates only. If a loudness difference was perceived, only the electrode with 100 pps stimulation was adjusted in ±3 current unit (CU) incremental steps. When the mixed-rate map was created in one ear, the same procedure was used to create the corresponding mixed-rate map in the contralateral ear. (4) Once the mixed-rate maps for each ear had been found, the two maps were played bilaterally to ensure the listener perceived a centered auditory object. If an adjustment was needed to center the auditory object, only the C levels of the 100 pps electrode was adjusted. (5) Mixed-rate maps with more than one low-rate electrode were created from maps found using steps (3) and (4) by repeatedly introducing a new low-rate electrode, one at a time.

D. Procedure

For the task, listeners reported the perceived intracranial location(s) on an image of a face shown on a computer monitor (see Fitzgerald et al., 2015, for a picture of the user interface). Once the stimulus was played, a blue bar spanning the image of the face appeared, and listeners were then asked to click on any point in the bar to indicate the perceived intracranial location. If only one sound was perceived, the listener was only required to indicate the location of the object. If a listener perceived multiple auditory objects, they could select whether they perceived two or three auditory sources or “objects,” and multiple bars were shown on the face. When more than one auditory object was perceived, listeners were instructed to rank each object from most- to least-dominant by indicating the location of the most “dominant” auditory object in the top bar and working their way downwards in terms of dominance. The term “dominant” was defined to be the most robust, easily heard, and localized auditory object. On each trial, listeners could repeat the stimulus as many times as they needed to make a response. Typically, listeners only required one or two repetitions to complete their response. The task was a single-interval lateralization task and used the method of constant stimuli.

During testing, all stimuli had ITDs consisting of ±100, ±200, ±400, and ±800 μs and were presented in a randomized order. In some instances, larger ITDs, such as ±1000 or ±1600 μs, were added to measure a complete range of right-to-left lateralization percept. Testing was blocked by configuration type. For each of the seven configurations, each ITD magnitude was tested 40 times, with 20 left-leading and 20 right-leading trials. The 40 trials for each ITD magnitude were randomized and divided into smaller blocks of five repetitions for each configuration. The different configuration types were tested one at a time but the order of presentation of configuration was pseudorandomized and counter-balanced across different listeners. For example, block one could have five repetitions of all ITDs for the Low5 configuration and block two could have five repetitions of all ITDs for the Apex3 configuration. This method of pseudo-randomization continued until all seven configurations were tested for 40 repetitions of each ITD.

E. Data analysis

To quantify the range of perceived lateral locations, analysis methods similar to Ehlers et al. (2016) were employed. Listeners' responses were assigned a value between 0 and 1, with 0 corresponding to the far left, 0.5 to the center, and 1 to the far right. The lateralization responses were then fit with a four-parameter logistic function [see Eq. (1) with a non-linear least square (NLS) curve fitting procedure in R (version 3.4.4) using the minipack.lm package (version 1.2–1)],

$$p\left(\mathit{ITD}\right)=\mathit{Floor}+\frac{\mathit{Range}}{1+e^{(-m*\hspace{0.17em}\mathit{ITD}+\mathit{shift})}}.$$ (1)

The p(ITD) in Eq. (1) represents the perceived lateralization, the Range refers to the extent between the upper and lower bounds (values were constrained between 0 and 1), and the Floor represents the lower bound of the function, or the lower asymptote. The upper asymptote is derived using the Range estimate after the model fit is complete. The slope is represented by m and the shift represents the bias; see nlsLM (non-linear least squares Levenberg–Marquardt) R documentation for further details (Elzhov et al., 2016). Equation (1) uses a standard logistic function with a natural e exponential to model the entire response curve. The value of the slope is an index of the listener's perceptual mapping of the cues to the response range and refers to the natural log change in the listener's lateralization responses as a function of ITD. Once the model fit was complete, the upper and lower asymptote was estimated by inputting the largest tested ITD into Eq. (1) and computing the estimated lateralization curve. For example, the largest ITDs tested for listener IBK was ±1600 μs. Therefore, –1600 μs and +1600 μs were used to estimate the lower and upper asymptotes, respectively. The difference between the asymptotes is the lateralization range estimate used in the following analyses. The range estimate approximates how well a listener maps the most extreme ITDs to the far-left and far-right sides of the head. Overall, a narrower range of perceived lateralization indicates a less robust mapping of ITDs to lateral positions.

Another metric of performance was determined based on the methods for estimating JNDs as described in Litovsky et al. (2010). First, an arc-sine transformation was applied to all lateralization responses; similar to a logit transformation, an arc-sine transformation extends the lateralization values to the ends of the distribution (Sokal and Rohlf, 1995). Next, lateralization responses were sorted by each ITD magnitude into their own distributions. Mean and standard deviations were calculated for each distribution and compared for left-ward and right-ward responses. For example, the mean and standard deviation of responses lateralized for +100 μs ITD was compared to responses lateralized for −100 μs ITD. The d prime for each ITD magnitude was calculated using the equation

$$d\hspace{0.17em}\mathit{prime}=\frac{{y_1-y}_2}{s_{\rho }},$$ (2)

where y₁ and y₂ represents the means of the left-ward and right-ward responses, respectively, and s_ρ represents the pooled standard deviation of the left-ward and right-ward response distributions. Using the d prime value calculated for each ITD magnitude, a JND was estimated by fitting the d prime values with a straight line passing through zero and taking the value where the line passed through d prime =1. These JND estimates were taken to be the smallest ITD that would lead to a discriminable difference between two ITD locations across the midline. ITDs that were presented at values greater than ±800 μs were removed from this analysis because many listeners showed saturation of lateralization responses for ITDs of larger magnitudes.

III. RESULTS

A. Lateralization range

Individual lateralization functions for all listeners, for each stimulus configuration (High5, Low5, Apex1, Mid1, Base1, Spread3, and Apex3) are shown in Fig. 2. Results from all listeners exhibited a sigmoidal shape in at least one mixed-rate configuration suggesting that listeners could map ITDs to the range of lateral positions in the head. It should be noted that all listeners exhibited poorest lateralization in the All-High condition. A Friedman's test conducted to examine whether lateralization range was different between configurations revealed a significant difference [χ²(6) = 30.64, p < 0.001] in lateralization ranges across all listeners.

FIG. 2.

(Color online) Response lateralization for all configurations and all listeners. Solid colored lines represent the model fit and the solid gray lines represent average lateralization response for each ITD. Closed symbols illustrate the primary auditory object reported, while open symbols depict the secondary auditory object. The values 0, 0.5, and 1 correspond to left, center, and right, respectively.

Of the eight listeners, only one listener (IBK) reported perceiving more than one auditory object in some trials where an ITD was applied. Even for that listener, the percentage of responses reported as having a secondary percept in each configuration was relatively low (Apex1: 10.5%; Apex3: 5.4%; Spread3: 1.3%; Low5: 13%). Despite perceiving a secondary auditory object, the lateralization of the primary auditory object followed that of the applied ITD.

Figure 3 shows the ranges calculated from the four-parameter logistic fit [using Eq. (1) for each tested configuration]. The High5 and Apex1 configurations elicited the smallest lateralization mean ranks. A post hoc Wilcoxon signed-rank test was conducted to evaluate whether the pairwise comparisons between configurations were significant. Using the “signrank” function in matlab, a z-statistic was determined. The z-statistic specifies the direction of the comparison between any two configurations. Each z-statistic for each pairwise comparison is shown in Table II. All mixed-rate configurations yielded lateralization ranges greater than the High5 configuration. Additionally, several mixed-rate configurations were not statistically different from the Low5 condition. The exception was Apex1, which has lateralization ranges that vary widely across participants (see Fig. 3).

FIG. 3.

(Color online) Lateralization ranges per listener and configuration; ranges are estimated from the data fit with Eq. (1). Range of lateralization was calculated using the upper and lower bounds of the model fit. A larger range is associated with a greater extent of laterality. Lower hinge of each box plot represents the 25% quartile (Q1), the upper hinge represents the 75% quartile (Q3), the lower whisker represents the smallest observation ≤Q1–1.5*(Q3–Q1), and the upper whisker represents the largest observation ≤Q3 + 1.5*(Q3–Q1).

TABLE II.

Z-statistics for each pairwise comparison of lateralization range data; asterisks represent whether the pairwise comparisons were significantly different. Negative values indicated that ranges for the configuration in the left column were significantly smaller than the comparison indicated in the top row; this has been denoted with an asterisk (p < 0.05).

	High5	Low5	Apex1	Base1	Mid1	Apex3	Spread3
High5	–	−2.52*	−2.38*	−2.52*	−2.52*	−2.52*	−2.52*
Low5		–	2.38*	−1.26	0.70	−0.70	−1.40
Apex1			–	−2.52*	−2.52*	−2.52*	−2.52*
Base1				–	2.10*	0.70	0.14
Mid1					–	−0.70	−1.4
Apex3						–	−0.98
Spread3							–

Effect sizes, calculated from the output of the Wilcoxon-signed rank test, for all comparisons between groups are shown in Table III. Using Cohen's scale for interpreting effect sizes (Cohen, 1988), where: 0.10 (small effect, explains 1% of the total variance), 0.30 (medium effect, accounts for 9% of the total variance), and 0.50 (large effect, accounts for 25% of the variance), values in Table III indicate a large effect size for the difference in range between each mixed-rate configuration and the High5 configuration. A small effect size is observed between Mid1, Apex3, and Low5 configurations, suggesting these configurations had lateralization ranges that were similar.

TABLE III.

Effect sizes for each pairwise comparison across High5, Low5, and all mixed-rate configurations.

	High5	Low5	Apex1	Base1	Mid1	Apex3	Spread3
High5	–	0.63	0.60	0.63	0.63	0.63	0.63
Low5		–	0.60	0.32	0.18	0.18	0.35
Apex1			–	0.63	0.63	0.63	0.63
Base1				–	0.53	0.18	0.04
Mid1					–	0.18	0.35
Apex3						–	0.25

B. Estimated JNDs

Data from the lateralization responses were further used to estimate JNDs (see Fig. 4) as a measure of the smallest ITD that would lead to a discriminable difference between two conditions with equal ITD values across the midline (e.g., ±100, ±200, etc.). A Friedman's test found that estimated JNDs across all listeners yielded a significant effect of configuration, χ²(6) = 24.79, p < 0.001. Post hoc Wilcoxon signed-rank tests revealed that mean ranks of each configuration, ordered from highest to lowest, were: High5, Apex1, Apex3, Mid1, Low5, Base1, and Spread3.

FIG. 4.

(Color online) Estimated ITD JNDs. Closed circles represent measurable JNDs. The upper limit of the human physiological range (∼800 μs) is indicated by the red dashed line. The open triangles are values greater than 1600 μs and were considered “Not Determinable” JNDs. Lower hinge of each box plot represents the 25% quartile (Q1), the upper hinge represents the 75% quartile (Q3), the lower whisker represents the smallest observation ≤Q1–1.5*(Q3–Q1), and the upper whisker represents the largest observation ≤Q3 + 1.5*(Q3–Q1).

The data suggest that the High5 configuration resulted in the poorest JNDs for pairs of ITDs across the midline. JND estimates showed that the Base1 and Spread3 configurations resulted in the best performance. Pairwise comparisons revealed that all mixed-rate configurations, apart from Apex1, were statistically different from the High5 configuration (Table IV). However, unlike the lateralization ranges, there was a significant difference in performance between the Low5 and Apex3, suggesting that when all electrodes are stimulated with low rates, there may be greater variability in left-right perception than for a configuration with only one or three electrodes stimulated with low rates.

TABLE IV.

Z-statistics for each pairwise comparison of estimated JND data; asterisks represent whether the pairwise comparisons were significantly different. Negative values indicated that the configuration in the left column were greater than the configuration indicated in the top row. A significant difference between any pairwise comparisons is denoted with an asterisk (p <0.05).

	High5	Low5	Apex1	Base1	Mid1	Apex3	Spread3
High5	–	2.20*	2.20*	2.20*	2.20*	1.99*	2.20*
Low5		–	−2.20*	0.10	−1.78	−1.99*	1.15
Apex1			–	2.20*	1.99*	1.99*	1.99*
Base1				–	−2.20*	−1.36	0.73
Mid1					–	0.73	1.99
Apex3						–	2.20*
Spread3							–

IV. DISCUSSION

The present study investigated whether improvements in the lateralization range of BiCI listeners are observed when more low-rate channels are systematically added into a mixed-rate stimulus. Further, this study identified whether BiCI listeners are capable of perceiving a unitary auditory object that can be systematically mapped to a range of intracranial lateral locations using a mixed-rate stimulus. In a previous study, we used similar mixed-rate configurations to demonstrate that mixed-rate strategies, when compared to an all high-rate strategy, can be used for improving ITD sensitivity in BiCI listeners. While ITD JNDs provide insight into sensitivity to binaural cues, the lateralization measures shown here are arguably more directly linked to sound localization, which is worse in BiCI listeners than in NH listeners. Ultimately, our goal is to introduce a new approach for a mixed-rate CI sound processing strategy capable of delivering usable ITD information to the auditory system.

A significant finding from the current study was that good auditory object formation (i.e., perception of one fused sound source) was found amongst most of our listeners using the mixed-rate configurations; all but one listener reported perceiving a unitary auditory object in all conditions. This finding is important because it suggests that the low-rate channels were perceptually grouped with the high-rate channels, with the latter conveying insufficient ITD information. Thus, despite insufficient ITD information in the high-rate channels and a rate difference across channels, most BiCI listeners in the present study still experienced a unitary auditory object. This means that for future studies investigating speech-in-noise with mixed rates, the speech signal and location cue have the potential to be associated together in a noisy environment.

It is not clear why listener IBK heard a secondary auditory object, but it did not seem to interfere with this subject's ability to spatially map the dominant auditory object as a function of ITD in the mixed-rate configurations. It was noted that the secondary auditory object was only perceived in configurations that included the most apically tested electrode pair stimulated at a low rate, when an ITD was applied, and that it was only ever on the right-hand side. One interpretation is that the apical electrode in the right-ear implant elicited an auditory artifact when that became clearly audible for the mixed-rate configurations. This artifact may have been a consequence of various factors. One of these factors could be that listener IBK's clinical maps had a difference in dynamic range (C-level minus T-level) of 20 CUs between the two ears, with the right-ear MAP having a larger dynamic range. However, prior research has not shown a consistent relationship between dynamic range differences across the ears and ITD sensitivity (Todd et al., 2017). Further, there are currently no data to suggest that differences in dynamic range might negatively affect auditory object formation. Another factor could be that this listener has an insertion of their apical-most electrodes in two different scala, resulting in greater interaural mismatch (Goupell et al., 2022). Hence, the listener may have been reporting multiple auditory objects due to one or both confounds.

Another hypothesis is that listener IBK was attempting to fuse two disparate auditory objects in the High5 condition. When listening to the Low5 and Apex3 configurations, this may have induced some “release from interference” allowing the listener to successfully segregate the object with the low-rate ITD from the secondary image created by the high-rate electrodes; this would be in line with findings from Best et al. (2011). However, to fully understand this, it would require revisiting the centering procedure during mapping to ensure that listener IBK's loudness maps were accurately centered for the most dominant auditory object. It is also possible that the artifact observed in listener IBK was a product of binaural interference. Prior work in NH listeners has shown that binaural interference, or a reduction in ITD sensitivity when ITDs of different magnitudes are presented to different frequency regions of the cochlea, is a consistent phenomenon in the typically developing binaural system (Dye et al., 1996, 2005; Stellmack and Dye, 1993). This interference could be related to monaural grouping cue across competing frequency regions. Similarly, in BiCI listeners, Best et al. (2011) showed that when ITDs of different magnitudes were presented to different electrode pairs with wide tonotopic separation, BiCI listeners experienced reduced lateralization to the ITD in the target electrode pair. Further, when the ITD in the non-target electrode pair was inserted into an ongoing stream, the ability of BiCI listeners to lateralize the target ITD improved. The Best et al. (2011) study demonstrates that across-electrode interference can impact lateralization ability in BiCI listeners, as well as impacting stream and source segregation. Unlike in Best et al. (2011), the same ITD was presented across electrodes in the current study. Here, the across-electrode variation was a rate difference rather than an ITD difference. While there is no evidence for a direct relationship between binaural interference and source segregation due to rate differences, it is certainly possible that listener IBK experienced some type of interference due to the mixed-rate configuration, preventing formation of a unitary auditory object.

Prior work in our lab has found that age of onset of hearing loss in adults is not a strong predictor of ITD sensitivity in adult BiCI listeners (Thakkar et al., 2020). While the impact of etiology, age, and years of experience with electric hearing were not explicitly studied, it should be noted that both listeners ICD and ICJ experienced hearing loss at a very young age (see Table I) and still showed a bimodal distribution of lateralization responses for the High5 configuration, albeit a very restricted bimodal distribution for listener ICD. Further, both listeners' lateralization ranges increased with an increasing number of low-rate electrodes. Conversely, listener ICI showed greater variability in their responses, even for the Low5 and Apex3 configurations, yet experienced hearing loss as an adult. The variability observed for listener ICI and the bimodal distribution observed for listener ICJ in the Low5 configuration could explain why the estimated ITD JNDs for Low5 were significantly higher than for Apex3.

Findings from this study also suggest that the lateralization performance of all eight listeners significantly increased with configurations that had mixed rates compared to a configuration that had high rates only. The data showed that lateralization ranges increased for configurations where mixed-rates were used, specifically when the low-rate stimulation was introduced at either the basal electrode pair, middle electrode pair, or spread across the electrode array. We observed a similar outcome when looking at the estimated JNDs, specifically that the apical region of the array may not be the best place to present low-rate information in a mixed-rate strategy. These findings are consistent with our prior study investigating ITD sensitivity with mixed-rate stimulation (Thakkar et al., 2018) and demonstrate that the utility of mixed rates applies not only to discrimination but also to intracranial lateralization of ITDs.

The current study sheds light on the importance of low-rate ITD cues conveyed via the pulse timing in the presence of high-rate pulse trains, whereby the temporal information in at least one electrode pair is sufficient for relaying ITDs if conveyed at a low rate. It is possible that the salience of this cue could be one of the major reasons a mixed-rate stimulus elicits good ITD lateralization (van Hoesel, 2008b). Evidence for this notion can be seen by the fact that all listeners exhibited increases in lateralization range in all mixed-rate configurations, regardless of which electrode carried the low-rate pulse train, except the Apex1 configuration. This suggests that BiCI listeners can take advantage of a low-rate ITD such that good lateralization will be achieved.

Our results may also be indirectly related to several findings from studies with CI listeners. For example, Ihlefeld et al. (2015) found that that the place-of-stimulation with lowest ITD JND was predictive of ITD JNDs from multi-electrode stimulation. While we did not compare performance between configurations that varied the number of electrodes, the poorer performance observed for some listeners in the Low5 and Apex3 configurations compared to the Base1 or Spread3 configurations, suggests that poor sensitivity in some of the low-rate electrodes may have contributed to the reduced lateralization range and increased JNDs in these configurations. Additionally, work by Carlyon et al. (2007) suggested that cochlear implant listeners struggle with sound source segregation even when a rate difference exists across electrodes. Though that study did not directly measure object formation, it instead used a temporal pitch discrimination task which required listeners to segregate dissimilar stimuli across multiple electrodes. This finding from Carlyon et al. (2007) potentially reveals why many of the listeners in our study still heard a unitary auditory object.

More direct applications of the current data come from prior work published in our lab. First, ITD sensitivity was significantly better in the Low5 than the High5 configuration, consistent with single electrode studies (e.g., van Hoesel et al., 2009). Second, applying low rates of stimulation at the apical-most channels in a multi-electrode stimulus does not guarantee best ITD sensitivity, consistent with findings from Laback et al. (2015), Kan et al. (2015a, 2016, 2019), and Thakkar et al. (2018, 2020). Third, multi-electrode low-rate ITDs are sufficient to yield good lateralization while keeping object formation uninterrupted, consistent with other studies (Best et al., 2011; Egger et al., 2016; Francart et al., 2015; Kan et al., 2016). It should be noted, however, that the current study limited the number of electrodes to five in each ear; thus, if bilaterally synchronized devices are to harness the full electrode arrays, special consideration will be needed regarding spread of excitation due to monopolar stimulation; it is currently unknown how this might disrupt the perception of mixed-rate ITDs. Fourth, significant improvements in ITD sensitivity and lateralization can be achieved with only one low-rate channel, consistent with results of Thakkar et al. (2018). Future work should take these issues under consideration when designing stimulation approaches intended for clinical adoption.

V. CONCLUSIONS

The data from this study support our ongoing premise that mixed-rate stimulation can be used to improve spatial hearing outcomes in BiCI listeners. We found that ITDs presented via mixed-rate stimulation are sufficient to elicit improved lateralization range compared to a configuration with high-rate only stimulation. Further, auditory object formation is maintained despite different stimulation rates being used on different channels. Our findings suggest that mixed rates of stimulation could have the ability to improve spatial hearing outcomes for BiCI listeners.