Binaural jitter improves interaural time-difference sensitivity of cochlear implantees at high pulse rates

Abstract

Interaural time difference (ITD) arises whenever a sound outside of the median plane arrives at the two ears. There is evidence that ITD in the rapidly varying fine structure of a sound is most important for sound localization and for understanding speech in noise. Cochlear implants (CIs), neural prosthetic devices that restore hearing in the profoundly deaf, are increasingly implanted to both ears to provide implantees with the advantages of binaural hearing. CI listeners have been shown to be sensitive to fine structure ITD at low pulse rates, but their sensitivity declines at higher pulse rates that are required for speech coding. We hypothesize that this limitation in electric stimulation is at least partially due to binaural adaptation associated with periodic stimulation. Here, we show that introducing binaurally synchronized jitter in the stimulation timing causes large improvements in ITD sensitivity at higher pulse rates. Our experimental results demonstrate that a purely temporal trigger can cause recovery from binaural adaptation. Thus, binaurally jittered stimulation may improve several aspects of binaural hearing in bilateral recipients of neural auditory prostheses.

Interaural time difference (ITD) arises whenever a sound source outside the median plane arrives at the two ears and provides important information on the sound's lateral position. ITD occurs both in the rapidly varying fine structure and in the slowly varying envelope of the signal. There is evidence that ITD in the fine structure of a sound is most important for sound localization (1, 2) and for understanding speech in noise (3, 4). Listeners bilaterally supplied with cochlear implants (CIs) have been shown to be sensitive to fine structure ITD, but their sensitivity disappears at a pulse rate of a few hundred pulses per second (5–8). This is in contrast to normal hearing (NH) listeners, who are sensitive to ITD in the fine structure up to much higher frequencies (9, 10). The CI listeners' limitation in the ability to process fine structure ITD information at higher rates is disadvantageous with respect to speech coding where such high rates are required.

In this study, we present a method of electric stimulation that improves CI listeners' sensitivity to fine structure ITD at higher pulse rates. The method is based on previous findings on the limitation in ITD perception at higher modulation rates in NH listeners.

In studies with NH listeners, it has been observed that the sensitivity to ITD information degrades with increasing modulation rate of a high-frequency carrier signal (11, 12). By using filtered pulse trains, it was shown (11, 13) that, as pulse rate increases, increasing the stimulus duration yields a smaller improvement of ITD sensitivity than would be expected from a model based on optimum integration of ITD information across time (11). This effect has been referred to as binaural adaptation. Binaural adaptation has such a strong effect on ITD perception at higher pulse rates that the onset of a sound receives maximum perceptual weight, whereas the ongoing signal contributes little (14, 15). Going one step further, studies with NH listeners have shown that introducing a change in the ongoing signal (a trigger) causes a recovery from binaural adaptation (15, 16). As a consequence, the portion of the signal after the trigger becomes more important and this results in improved ITD sensitivity.

Based on the results of these studies, we assumed that the decreasing ITD sensitivity with increasing pulse rate in CI listeners is a form of binaural adaptation and that the introduction of a trigger in the signal can cause a recovery from the adaptation. A consequence of this recovery would be an increase in ITD sensitivity. We used a trigger similar to the one used by Hafter and Buell (16), which is a temporal change in the interpulse interval (IPI). In contrast to Hafter and Buell (16), who used acoustic stimulation, we used electric stimulation. By testing CI listeners at a single interaural electrode pair, we were able to change solely the temporal properties of the stimulus and not the spectral properties. Additionally, to multiply the recovery effect caused by one trigger, we attempted to trigger on every pulse by randomly varying (jittering) the IPI. To preserve the ITD in the fine structure, the jitter was synchronized between the two ears and is referred to as binaural jitter. The effect of binaural jitter on ITD-based left/right discrimination of a pulse train, a measure of ITD sensitivity, was tested at different pulse rates with five binaurally implanted CI listeners.

Methods

Stimuli.

Stimuli were 300-ms trains of biphasic electric pulses with trapezoidal amplitude modulation (Fig. 1a). An amplitude modulation of this type has been successfully used in a previous study (5). This signal attempts to roughly approximate the characteristics of real-world signals, in particular speech. The stimuli were presented binaurally and had an ITD in the entire waveform. Thus, any improvement in ITD sensitivity due to binaural jitter must occur despite the availability of envelope ITD information.

Fig. 1.

Experimental stimuli. (a) Schematics of the stimulus presented to each ear. For clarity, only three of the four trapezoids are shown, and the fine structure characteristic is shown in one trapezoid only. The ramps slope down to the absolute threshold of each subject. Between the trapezoids, the amplitude was set to zero. (b) Schematics of the stationary portion of a periodic pulse train (Upper) and of a binaurally jittered pulse train (Lower). For clarity, in b, only the positive phase is shown even though the pulses are biphasic. Note that the binaural jitter preserves the interaural time difference (marked with arrows).

The periodic pulse trains had a constant IPI (Fig. 1b Upper), the nominal IPI. In contrast, jittered pulse trains had randomly varied IPIs (Fig. 1b Lower). To preserve the ITD in the fine structure, the jitter was synchronized between the two ears. This is apparent from the constant length of the arrows in Fig. 1b. For jittered stimuli, the IPIs were chosen so that the average value over the stimulus duration exactly represented the nominal IPI. The jitter followed a rectangular distribution, where the parameter k defines the width of the distribution and therefore the amount of jitter. The parameter k is defined relative to the nominal IPI and ranges from 0 (periodic condition, no jitter) to 1 (maximum jitter). A jittered pulse train was “constructed” pulse by pulse. For each pulse added, its temporal position was varied within the interval IPI·(1 − k) to IPI·(1 + k). Thus, for k = 1, the largest possible IPI is twice the nominal IPI and the smallest possible IPI is 0. Each stimulus repetition had a new random jitter manifestation.

The independent variables were k (0, 0.125, 0.25, 0.5, 0.75, and 0.9), the pulse rate (400, 800, 938, 1,182, and 1,515 pps), and the ITD (100, 200, 400, and 600 μs). The stimuli were presented at an interaural electrode pair, which was chosen to elicit equal pitch on both sides. The respective electrode pairs for each subject are specified in the last column of Table 1, where electrodes are numbered from apex to base. For each pulse rate, current levels were determined to evoke a centralized auditory image at a comfortable level (5). The stimuli were created on a laboratory computer and directly transmitted to the CIs via an interaurally synchronized research interface (RIB, developed at the University of Innsbruck, Innsbruck, Austria) with an interaural timing accuracy of 2.5 μs.

Table 1.

Information about the subjects

Subject	Etiology	Age, years	Age at implantation, years		Duration of deafness		Binaural electrical stimulation experience, years	Test electrodes
			L	R	L	R
CI3	Meningitis	24	21	21	2 months	2 months	3	4/3
CI8	Osteogenesis imperfekta	44	41	39	3 years	12 years	3	7/5
CI10	Sudden hearing loss	54	44	48	9 years	6 years	6	7/8
CI11	Temporal bone fracture	28	22	22	2 years	2 years	6	2/3
CI12	Sudden hearing loss	40	35	34	8 years	3 years	5	2/2

The last column specifies the test electrodes on the left (L) and right (R) sides, respectively. The electrodes are numbered from apex to base.

Subjects and Procedure.

Five listeners, bilaterally implanted with Combi 40+ CIs (manufactured by MED-EL, Austria), participated in the experiment. All listeners were postlingually deafened, had high speech recognition scores, and had at least 3 years of binaural CI experience at the time of the tests. Subject data are provided in Table 1.

A target stimulus containing ITD was compared with a preceding reference stimulus with zero ITD in a left/right discrimination task. The reference stimulus was always periodic and had a k of 0. Visual response feedback was provided after each trial. Listeners were trained on the task for a couple of hours before starting formal data collection. Stimulus conditions corresponding to combinations of the independent variables (k, pulse rate, and ITD) were presented in a balanced design. Because of limited availability of the subjects, not all combinations of k and pulse rate were tested for each subject. Each condition was repeated 100 times. Inspection of the distribution of the left/right judgments for each listener revealed sufficient symmetry so that an adjustment of the percent correct scores to remove response bias was not required.

Statistical Analysis.

Repeated-measures ANOVA was used to test the effects of the parameters k and pulse rate on the percentage of correct left/right discrimination (P_c). Tukey's post hoc tests were used to compare the factor levels of k. For all statistical analyses, the P_c scores were transformed by using the rationalized arcsine transform (17) to not violate the assumption of homogeneity of variance required for ANOVA.

Results

The complete set of results for the individual listeners is provided in supporting information (SI) Fig. 6.

Sensitivity as a Function of Pulse Rate.

Fig. 2 shows P_c averaged over subjects as a function of the pulse rate, presenting the results for different ITDs in separate graphs. The values of the parameter k were pooled into large jitter (k = 0.75 and 0.9), small jitter (k = 0.125, 0.25, and 0.5), and the periodic condition without jitter (k = 0). The effect of jitter is very similar for the different ITD values, despite shifts in overall performance that are most easily seen between the ITD values of 100 and 200 μs. Because of this similarity, Fig. 3 summarizes the data further by averaging across the ITD values 200, 400, and 600 μs and across subjects. At the lowest pulse rate tested [400 pulses per second (pps)], P_c is generally high and does not differ between the conditions with binaural jitter and the periodic condition (P = 0.98). However, at the higher pulse rates (>400 pps), there is a large difference between the results for the periodic condition and the binaurally jittered conditions. For the periodic condition, P_c decreases sharply with increasing pulse rate (P < 0.0001) and even approaches chance performance. In contrast, the conditions with binaural jitter show large improvements compared with the periodic condition (P < 0.0001). For large jitter, the performance remains constantly high up to 1,182 pps and declines at 1,515 pps, even though still significantly above the periodic condition (P = 0.0003). For small jitter, the improvements are about one-half of those for large jitter but still significant (P < 0.0001).

Fig. 2.

Percentage of correct scores for left/right discrimination as a function of the pulse rate, averaged over the five subjects. The results for different interaural time difference (ITD) values are presented in separate graphs. The periodic condition without binaural jitter (k = 0) is depicted by the squares, the condition with small jitter (k = 0.125, 0.25, and 0.5) is depicted by the triangles, and the condition with large jitter (k = 0.75 and 0.9) is depicted by the circles. The error bars represent 95% confidence intervals.

Fig. 3.

Percentage of correct scores for left/right discrimination as a function of the pulse rate. The data are averaged over the five subjects and the interaural time difference values, 200, 400, and 600 μs. The periodic condition without binaural jitter (k = 0) is depicted by the squares, the condition with small jitter (k = 0.125, 0.25, and 0.5) is depicted by the triangles, and the condition with large jitter (k = 0.75 and 0.9) is depicted by the circles. The error bars represent 95% confidence intervals.

Sensitivity as a Function of ITD.

Fig. 4 shows the effect of binaural jitter as a function of the magnitude of the ITD, each graph showing the results for a different pulse rate, averaged across subjects. For pulse rates >400 pps, the periodic condition shows low values of P_c and the overall effects of binaural jitter are approximately similar apart from overall shifts in performance. Thus, Fig. 5 presents the data averaged over these pulse rates. For the periodic condition (k = 0), P_c is low at all ITD values. However, for the conditions with binaural jitter, P_c increases monotonically with the ITD. The improvements from binaural jitter are significant at the smallest ITD of 100 μs (both large and small jitter: P < 0.0001) and increase further with increasing ITD. The improvements reach a maximum of 28% at 600 μs for large jitter (P < 0.0001) and of 14% at 400 μs for small jitter (P < 0.0001). Binaural jitter improves the performance even for ITDs that approach or exceed one-half of the IPI and thus contain ambiguous ongoing fine structure ITD cues (5). For example, the ITD of 400 μs is ambiguous at all pulse rates from 800 to 1,515 pps (within one-quarter to three-quarters of the IPI), and still binaural jitter significantly improves the performance (both large and small jitter: P < 0.0001).

Fig. 4.

Percentage of correct scores for left/right discrimination as a function of the interaural time difference, averaged over the five subjects. The results for different pulse rates are presented in separate graphs. The periodic condition without binaural jitter (k = 0) is depicted by the squares, the condition with small jitter (k = 0.125, 0.25, and 0.5) is depicted by the triangles, and the condition with large jitter (k = 0.75 and 0.9) is depicted by the circles. The error bars represent 95% confidence intervals.

Fig. 5.

Percentage of correct scores for left/right discrimination as a function of the interaural time difference. The data are averaged over the five subjects and the pulse rates, 800, 938, 1,182, and 1,515 pps, for which the periodic condition has a low P_c. The periodic condition without binaural jitter (k = 0) is depicted by the squares, the condition with small jitter (k = 0.125, 0.25, and 0.5) is depicted by the triangles, and the condition with large jitter (k = 0.75 and 0.9) is depicted by the circles. The error bars represent 95% confidence intervals.

Discussion

The decline in ITD sensitivity with increasing pulse rate for the periodic condition is consistent with previous studies (5–8). At 400 pps, performance seems to be less affected by rate limitation mechanisms compared with higher rates and hence less improvement by applying binaural jitter can be expected. However, at pulse rates ≥800 pps, performance seems to be severely reduced by rate limitation mechanisms. As expected, ongoing envelope ITD appears to have contributed little to ITD sensitivity. The results clearly show that introducing binaural jitter makes CI listeners sensitive to fine structure ITD up to rates for which NH listeners show sensitivity to ITD in pure tones (9, 10), even though the absolute performance of the CI listeners is considerably lower. Therefore, binaurally jittered stimulation resolves the discrepancy in the rate limitation between CI and NH listeners. The finding of large improvements by adding binaural jitter at higher pulse rates (≥800 pps), but not at lower rates, is consistent with the hypothesis that an excessive form of binaural adaptation limits fine structure ITD sensitivity at higher pulse rates. Thus, introducing ongoing temporal changes in the stimulus seems to cause a recovery from binaural adaptation in CI listeners.

Possible reasons for the excessive form of the binaural adaptation effect could be the high degree of phase locking and across-fiber synchrony in the neural response to electric stimulation (18–22). Introducing artificial randomness into the stimulus may reduce the amount of periodicity in the neural response and consequently avoid binaural adaptation. According to this explanation, binaural jitter holds the binaural system “awake” over the duration of the stimulus and thus improves access to the ITD information. Furthermore, the beneficial effect of binaural jitter may be interpreted in terms of a generally better neural representation of temporal information. Neural models as well as experimental results suggest that restoring stochastic responses in electric stimulation enhances the neural representation of stimulus timing (23, 24). Thus, jittering the IPI may be expected to improve also rate pitch perception in electric hearing, which is limited to pulse rates up to ≈300 pps (25). Chen et al. (26) studied the effect of jitter on monaural pitch discrimination in three CI listeners. They tested only small amounts of jitter and found no effect on pitch discrimination besides a deterioration at low pulse rates. They did not test larger amounts of jitter for which we observed the largest improvements in ITD sensitivity. However, such amounts of jitter would likely smear the pitch cue, counteracting the potential benefit of jitter to rate pitch perception. Thus, there is currently no indication that jittering the IPI improves the neural representation of temporal information in a way that is advantageous for temporal pitch perception.

It is intriguing that binaural jitter improves the performance even for ITDs that approach or even exceed one-half of the IPI and thus contain ambiguous ongoing fine structure ITD cues (5). This result could be explained by a model in which the auditory system resolves the ambiguity in ongoing fine structure ITD by picking out interaural pulse pairs with a large IPI to adjacent pairs. This corresponds to a so-called multiple looks model (27), where the auditory system stores samples or “looks” of the signal in memory and accesses and processes them selectively.

The results of this study indicate that purely temporal changes in the ongoing signal can cause recovery from binaural adaptation. This finding extends the conclusion of Hafter and Buell (16) on the recovery from binaural adaptation in acoustic hearing by inserting a trigger (a temporal gap or a brief sound) into a pulse train. They attributed the recovery effect to the spectral changes induced by the trigger. Our results with electric stimulation show that a recovery is possible without spectral changes, only temporal.

The findings of our study have important implications for stimulation strategies aiming to transmit fine structure ITD information to listeners supplied with bilateral neural auditory prostheses such as cochlear, brainstem, or intraneural implants. Commonly used periodic or near-periodic stimulation limits the perception of fine structure ITD to a few hundred pulses per second. Introducing a binaurally synchronized variation to the interpulse interval removes this limitation. Consequently, fine structure ITD information can be transmitted at higher pulse rates, which are important for the coding of speech signals in cochlear implants (28). The enhancement of the sensitivity to fine structure ITD information promises improvements in the localization of sound sources and in the understanding of speech in noise.