Sensitivity to Envelope Interaural Time Differences at High Modulation Rates

Abstract

Sensitivity to interaural time differences (ITDs) conveyed in the temporal fine structure of low-frequency tones and the modulated envelopes of high-frequency sounds are considered comparable, particularly for envelopes shaped to transmit similar fidelity of temporal information normally present for low-frequency sounds. Nevertheless, discrimination performance for envelope modulation rates above a few hundred Hertz is reported to be poor—to the point of discrimination thresholds being unattainable—compared with the much higher (>1,000 Hz) limit for low-frequency ITD sensitivity, suggesting the presence of a low-pass filter in the envelope domain. Further, performance for identical modulation rates appears to decline with increasing carrier frequency, supporting the view that the low-pass characteristics observed for envelope ITD processing is carrier-frequency dependent. Here, we assessed listeners’ sensitivity to ITDs conveyed in pure tones and in the modulated envelopes of high-frequency tones. ITD discrimination for the modulated high-frequency tones was measured as a function of both modulation rate and carrier frequency. Some well-trained listeners appear able to discriminate ITDs extremely well, even at modulation rates well beyond 500 Hz, for 4-kHz carriers. For one listener, thresholds were even obtained for a modulation rate of 800 Hz. The highest modulation rate for which thresholds could be obtained declined with increasing carrier frequency for all listeners. At 10 kHz, the highest modulation rate at which thresholds could be obtained was 600 Hz. The upper limit of sensitivity to ITDs conveyed in the envelope of high-frequency modulated sounds appears to be higher than previously considered.

Introduction

The ability to determine the location of the source of a sound relies on detecting differences in the timing and the intensity of the sound arriving at the two ears—interaural time and level differences (ITDs and ILDs), respectively. ITDs, especially, are the subject of intense investigation, in part due to the exquisite temporal performance limits achieved by human listeners: for sound frequencies below about 1,300 Hz, ITDs of just a few tens of microseconds are discriminable at the behavioral level. For frequencies above this limit, where sound waves increasingly interact with the head, ILDs are considered the more important localization cue. Nevertheless, this dichotomy between low-frequency ITD processing, and high-frequency ILD, is not strict: as well as being sensitive to ITDs conveyed in the temporal fine structure (TFS) of low-frequency sounds, listeners are also sensitive—sometimes highly so—to ITDs conveyed in the modulated envelopes of high-frequency sounds (Henning, 1974; McFadden & Passanen, 1976). While this suggests similarities in temporal processing across the tonotopic gradient in hearing, the extent to which sensitivity to, and mechanisms underpinning, envelope ITDs and ITDs conveyed in the TFS are similar remains a matter of debate, not least because of differences in how temporal information is conveyed by the cochlea and the auditory nerve.

Colburn and Esquissaud (1976) suggested that differences in sensitivity to ITDs conveyed in low- and high-frequency sounds might be the result of differences in peripheral processing. If this is indeed the case, it may be that the binaural brain is equally sensitive to temporal information conveyed at high- and low-spectral frequencies, so long as temporal information conveyed by the peripheral auditory system is equivalent. Transposed tones were designed by van de Par and Kohlrausch (1997) to do just this—conveying temporal information in the modulated envelopes of high-frequency modulated sounds approximating the half-wave rectified output of the cochlea for the same low (spectral) frequency. Using transposed tones, Bernstein and Trahiotis (2002) demonstrated that, for the appropriate envelope waveforms, human listeners can be as sensitive to ITD cues conveyed in the modulated envelope of high-frequency sounds as they are to ITDs conveyed in the TFS of low-frequency pure tones. For example, ITD discrimination thresholds are identical for pure tones with a frequency of 128 Hz and for transposed tones with a modulation rate of 128 Hz and a 4 kHz carrier frequency. However, unlike pure tones, for which ITD discrimination thresholds improve with increasing tone frequency up to ∼1,000 Hz (Klumpp & Eady, 1956; Zwislocki & Feldman, 1956), thresholds for modulated high-frequency sounds show a marked decline in performance for modulation rates above a few hundred Hertz (Henning, 1974; McFadden & Passanen, 1976; Nuetzel & Hafter, 1981) a decline in performance that is also apparent for transposed tones (Bernstein & Trahiotis, 2002; Oxenham, Bernstein, & Penagos, 2004). To account for this disparity between sensitivity at high- and low-spectral frequencies, Bernstein and Trahiotis (2002) suggested a central limitation in the ability of the auditory system to follow envelope fluctuations at rates greater than about 150 Hz. This form of low-pass characteristic has been observed monaurally for temporal-modulation-transfer functions (Ewert & Dau, 2000; Kohlrausch, Fassel, & Dau, 2000; Viemeister, 1979; Zwicker, 1952) as well as for modulation rate discrimination in both acoustic and electric hearing (see Discussion section).

Bernstein and Trahiotis (2002) also reported a reduction in temporal performance with increasing carrier frequency: As carrier frequency increased from 4 kHz to 6 and 10 kHz, listeners were less able to discriminate envelope ITDs for the same modulation rate, suggesting that the low-pass modulation filter becomes increasingly low-pass with increasing spectral frequency. Based purely on the bandwidths (in Hertz terms) of cochlear filters—which increase with increasing frequency—temporal performance might have been expected to have improved with increasing carrier frequency, as the spectral side-bands of the stimuli are less attenuated by the edges of the filter, resulting in a higher modulation depth at higher modulation rates.

While thresholds do appear to increase for modulation rates above 150 Hz, some participants are still sensitive to envelope ITDs at modulation rates exceeding this. For example, although two (of four) participants in Bernstein and Trahiotis’ (2002) study were unable to perform the ITD-discrimination task at a modulation rate of 512 Hz (the highest tested in their study) for a 4-kHz carrier, the remaining two subjects were at least somewhat sensitive, with one listener attaining discrimination thresholds below 100 μs. All four participants were experienced binaural listeners. A similar range of abilities was reported by Goupell, Laback, and Majdak (2009), who found that one of three experienced listeners was able to lateralize a pulse train presented at a rate of 600 pulses per second (pps) and band-pass filtered around 4.6 kHz, with an accuracy of 85% when an ITD of 150 μs was applied, while another showed accuracy of 73% for an ITD of 400 μs. One experienced listener and a further three inexperienced listeners were unable to lateralize pulse trains above 70% performance at this pulse rate, even when an ITD of 600 μs was applied. Of five experienced listeners tested by Majdak and Laback (2009), four achieved thresholds below 400 μs for a 500 pps band-pass filtered pulse train, and one showed thresholds <200 μs for a pulse rate of 600 pps at a center frequency of 4.6 kHz. Majdak, Laback, and Baumgartner (2006) also assessed listeners’ performance for pulse rates of up to 938 pps and reported that the highest rate for which ITD thresholds could be obtained was between 400 and 600 pps. This was based on the average of four normal-hearing (NH) listeners. Individual performance was not reported for NH listeners, but these authors reported little intersubject variability. At least for experienced listeners, then, it appears that a significant proportion is still able to detect or discriminate envelope ITDs presented at high-modulation rates. For such sensitive listeners, the upper limit above which useful ITD information may no longer be extracted from modulated sounds has not been explored.

Notwithstanding methodological differences between these studies, the prospect that envelope ITDs might be less limited than previously considered is important, particularly for hearing-impaired individuals who use bilateral cochlear implants. While envelope ITDs may have limited practical value for NH listeners—ILDs dominate localization at high frequencies Macpherson and Middlebrooks (2002), and envelope ITDs are more susceptible to distortion from reverberation (Monaghan, Krumbholz, & Seeber, 2013)—interest in their utility for spatial hearing has increased with the advent of bilateral cochlear implantation (CI). Most signal processing strategies in CIs remove information about the TFS of sounds and transmit information only about the temporal envelope. As a result, ITDs conveyed in modulated envelopes are the only ITD cues available to CI users.

Performance varies considerably between CI participants but, typically, ITD sensitivity decreases for pulse rates above about 100 pps (Laback et al., 2005; Majdak et al., 2006; van Hoesel & Tyler, 2003). Majdak et al. (2006) found two of the four CI users tested were sensitive to ITDs at pulse rates of 800 pps, although maximum lateralization discrimination accuracy was ∼60% (chance level for lateralization discrimination was 0%, corresponding to 50% correct responses) for an ITD of 250 μs. The remaining two CI listeners were only marginally sensitive to ITDs at pulse rates of 400 pps, with lateralization discrimination accuracy of 30% and 40% for the 250 -μs ITD. The NH listeners tested were sensitive to ITDs only for pulse rates up to 600 pps, with a maximum lateralization accuracy of ∼30% for an ITD of 250 μs. Laback et al. (2007) also reported sensitivity to ongoing ITDs for pulse rates of 800 pps for one of the four CI users they assessed, with their just-noticeable-difference being ∼200 μs. For the other three participants, thresholds were either above 1,000 μs or unmeasurable. At a pulse rate of 400 pps, thresholds were obtained for three out of four CI users, and just-noticeable-differences ranged from 250 to 600 μs, a pulse rate for which NH listeners were unable to do the task. Majdak et al. (2006) suggested two (related) explanations for the apparently higher performance of CI compared with NH listeners for high rate pulse trains: that cochlear filtering, present for NH but not for CI listeners, smears temporal envelope information, limiting the availability of robust ITD information or, that the higher degree of phase locking in electric, compared with acoustic, hearing generates more temporally precise input to binaural neurons sensitive to ITDs.

Here, we assess the performance limits for discriminating ITDs conveyed in transposed envelopes of high-frequency modulated sounds in NH listeners. We find that, for some listeners, discrimination thresholds can indeed remain low (<100 μs) at modulation rates well in excess of 500 Hz for carrier frequencies up to 10 kHz. Peripheral filtering appears to be the main factor limiting performance at 4-kHz carrier frequency and likely contributes to the reduced performance at higher frequencies. Nevertheless, even for listeners with excellent performance at high (>500 Hz) modulation rates, performance degrades more rapidly with increasing modulation rate as carrier frequency is increased. This is consistent with the low-pass filter operating on interaural temporal information becoming increasingly low pass with increasing carrier frequency.

Methods

To aid comparison with previous studies, aside from some minor differences, the stimuli and procedure we employed were identical to those employed by Bernstein and Trahiotis (2002). The differences were the following: (a) the low-pass masking noise used here is uncorrelated in order to provide a slightly greater degree of masking than that afforded by a correlated noise, (b) the stopping procedure for ending a run if the threshold was likely to prove unobtainable was slightly less severe, and (c) thresholds were obtained with a three-down one-up tracking procedure rather than the two-down one-up procedure employed by Bernstein and Trahiotis. Consequently, the thresholds reported here reflect the 79% point rather than 71% as in the previous study.

Participants

Five subjects (two female) took part in the experiment. All subjects had normal hearing, as determined audiometrically, at octave frequencies from 250 Hz to 8 kHz. Subjects 1 (the first author) and 2 were siblings. Subjects 1 to 3 were experienced binaural listeners. Subject 4 had taken part in monaural psychoacoustic experiments previously and Subject 5 was a naïve listener. Subjects 1 and 2 had previously taken part in a pilot experiment for the study.

Stimuli

Transposed tones were generated by half-wave rectifying low-frequency tones, low-pass filtering in the frequency domain at 2 kHz, and multiplying by a high-frequency carrier tone. ITDs were applied to the envelope by applying a phase delay to all components in the frequency domain before multiplication with the diotic carrier tone. Transposed tones of 0.3 s duration were presented at 75 dB(A) and ramped on and off (20-ms cosine-squared ramps) diotically to reduce onset ITD cues. Low-pass masking noise (<1,300 Hz) was presented continuously at 60 dB sound pressure level (29 dB spectrum level) to mask low-frequency distortion products. A schematic of the stimulus is plotted in Figure 1. Thresholds were measured for carrier frequencies of 4, 6, 8, and 10 kHz for modulation frequencies of up to 900 Hz.

Figure 1.

Schematic of stimuli showing low-pass masking noise and a transposed tone with a modulation frequency of 512 Hz and a carrier-frequency of 4 kHz.

Apparatus

The stimuli were generated in Matlab with a sampling rate of 48 kHz and converted using a Fireface RME soundcard. The stimuli were presented via headphones (Sennheiser HDA200) to the participant, who was seated in a double-walled, sound-attenuating booth.

Procedures

ITD thresholds were measured for ITDs conveyed in the modulated envelopes of the transposed tones, using a four-interval two-alternative forced-choice task. The ITD to be discriminated (always left leading) was presented in either the second or third interval. In all other intervals, an ITD of zero was presented. Participants indicated which interval contained the sound heard furthest to the left. Feedback was given.

ITD discrimination thresholds were obtained using an adaptive three-down one-up procedure, returning the ITD that yields 79.4% correct responses (Levitt, 1971). The tracking algorithm employed used logarithmic step sizes, the initial ITD was 300 μs, and the initial step size was a factor of 1.584 in ITD, decreasing to a factor of 1.122 after two reversals. Threshold estimates were determined by averaging over the last 10 of 12 reversals once this minimum step size was reached. If the tracking procedure increased the ITD above 1,000 μs, a counter was initiated and if the ITD increased three further times without being decreased, the track was stopped and the threshold considered unobtainable. If the ITD was decreased at any point, the counter was reset. All participants ran through each experimental condition at least once as practice. Participants 4 and 5 had additional practice with 128 and 256 Hz modulation frequency transposed tones with a carrier frequency of 4 kHz until their thresholds were stable. Three threshold estimates were obtained for each condition from each listener. The data are available for download at http://dx.doi.org/10.5258/SOTON/382673.

At each carrier frequency, thresholds for each participant were first obtained for modulation frequencies of 128, 256, and 512 Hz, in random order. If a threshold could be obtained at 512 Hz, the participant was tested at progressively higher modulation frequencies of 600, 700, 800, and 900 Hz.

Results

We assessed the ability of listeners to discriminate ITDs conveyed in the envelopes of transposed tones for carrier frequencies of 4, 6, 8, and 10 kHz and modulation frequencies up to 900 Hz. Figure 2 plots ITD-discrimination thresholds for five individual subjects as a function of modulation rate (Figure 2(a)–(d)). Consistent with Bernstein and Trahiotis (2002), the three experienced subjects showed low thresholds (good performance) for modulation rates up to 256 Hz when the carrier frequency was 4 kHz (Figure 2(a)). In addition, all were able to discriminate ITDs at a modulation frequency of 512 Hz. At higher modulation rates, performance was more variable between these subjects. Subject 3 was unable to obtain a threshold at a modulation rate of 600 Hz, while Subjects 1 and 2 were able to perform the task even at modulation rates of 700 Hz, with thresholds of 184 µs and 115 µs, respectively. Only Subject 2 was able to complete the discrimination task when the modulation rate was 800 Hz. The less-experienced participants (Subjects 4 and 5) had higher thresholds at 128 and 256 Hz and neither could perform the task at 512 Hz modulation rate.

Figure 2.

ITD discrimination thresholds for transposed tones as a function of modulation rate at carrier frequencies of (a) 4 kHz, (b) 6 kHz, (c) 8 kHz, and (d) 10 kHz. Data from five participants are shown together along with predictions from the normalized-correlation model of Bernstein and Trahiotis (2002) with (dotted line) and without (solid line) a 150 Hz low-pass filter. Error bars show ± 1 geometric standard deviation from the geometric mean. Conditions for which thresholds below 1,000 μs could not be obtained are indicated by points plotted above the axis break. The shaded region indicates the region of frequency-ITD space in which the IPD is between 90° and 180°, the region for which the lateralization percept may be ambiguous. Beyond this region, reversals in laterality may occur (Sayers, 1964).

To confirm that distortion products were not contributing to the generation of low thresholds observed for some listeners, at high modulation frequencies (where sensitivity to the ITDs carried by the distortion products would be greatest), additional thresholds were recorded for Subject 1 for a carrier frequency of 4 kHz and a modulation frequency of 700 Hz in the presence of 80 dB sound pressure level (49 dB spectrum level) low-pass masking noise. Thresholds were virtually identical to those recorded with the 60 dB noise (114 vs. 115 µs).

The influence of carrier frequency on envelope-ITD discrimination performance as a function of modulation frequency has previously been reported. Bernstein and Trahiotis (1994) found that for two-tone complexes and for sinusoidally amplitude-modulated tones, the highest modulation frequency for which the thresholds below 2,000 µs could be obtained by the majority of subjects declined from ∼400 Hz for a center frequency of 4 kHz to 128 Hz for a center frequency of 12 kHz. Similarly, Bernstein and Trahiotis (2002) reported a decline in discrimination performance for transposed tones with increasing carrier frequency, with thresholds increasing for modulation rates of 128 and 256 Hz, as the carrier was increased from 4 kHz to 6 kHz or 10 kHz, with some subjects apparently insensitive to envelope ITDs for modulation rates beyond 128 Hz at 10 kHz. Although Majdak and Laback (2009), employing bandpass filtered pulse trains, reported no such decline in performance in their subjects, this observation was explicitly refuted by Bernstein and Trahiotis (2014) employing apparently identical stimuli and procedures. Interestingly, the effect of center frequency on the low-pass filter is not apparent from studies of monaural listening. Kohlrausch et al. (2000) measured TMTF using SAM tones at center frequency of 1, 3, 5, 8, and 10 kHz and Moore and Glasberg (2001) at 1, 2, and 5 kHz, while Joris and Yin (1992) measured TMTF in auditory nerve fibers of the cat, also employing SAM tones. In none of these studies was it reported that TMTFs became more low-pass with increasing center frequency.

In the current study, we found discrimination performance at higher carrier frequencies to be worse for all subjects than it was at 4 kHz (Figure 2), including those subjects with the best performance across all carrier frequencies and modulation rates. The highest modulation rate for which envelope-ITD thresholds could be obtained using our method was consistently lower at 6 kHz compared with 4 kHz, with Subject 3 unable any longer to perform the experiment for a modulation rate of 512 Hz, and Subjects 1 and 2 unable to do so for modulation rates of 700 Hz and 800 Hz, respectively. Given the high overall performance of these listeners, it would seem unlikely that the decline in performance could be attributed to generally poor sensitivity to envelope-ITD cues. Thresholds for the less experienced Subjects 4 and 5 were very similar at 6 kHz and 4 kHz. Thresholds increased still further for the 8-kHz carrier (Figure 2c). In this case, the highest modulation rate at which Subject 2 could perform the task was 512 Hz. For Subjects 1, 3, and 4, the upper limits of performance (in terms of modulation rate) at 8 kHz were similar to those at 6 kHz. Subject 5 showed the greatest decline in performance for a modulation rate of 256 Hz. Here, a threshold could only be obtained on two out of three repeats. Threshold ITDs for the highest carrier frequency—10 kHz—(Figure 2(d)) were elevated still further. For this carrier frequency, Subject 2 was no longer able to complete the task at a modulation rate of 512 Hz, but Subject 1 was still sensitive to ITDs, with a threshold of 353 µs at a modulation rate of 600 Hz. Subjects 3 and 5 could still perform the task for a modulation rate of 256 Hz, but the average threshold for Subject 5 was >1,000 µs.

For the 4-kHz carrier frequency, the thresholds for Subjects 1 and 2 are consistent with those suggested by Bernstein and Trahiotis’ (2002) normalized correlation model, but without the need to employ the low-pass filter they applied to account for the data. These data, therefore, suggest the existence of a relatively steep modulation filter–thresholds are still low, only 100 Hz below the cutoff of 800 Hz for Subject 1–but at a much higher frequency than suggested by Bernstein and Trahiotis (2002). For Subjects 3 to 5, however, the data are more consistent with the model incorporating a 150 Hz low-pass filter.

A two-way, repeated-measures ANOVA was performed on the data obtained from Participants 1 to 4 (Participant 5 was excluded because they were only able to perform the task on two of three occasions at 256 Hz for a carrier frequency of 8 kHz) for modulations rates of 128 and 256 Hz at all carrier frequencies. Besides being the only modulation rates for which thresholds could be obtained by all of these four participants at all carrier frequencies, these are the only modulation rates for which modulation depth in the on-frequency channel was not greatly reduced by peripheral filtering at any carrier frequency (see below). There was a significant main effect of carrier frequency on ITD thresholds, F(1.44,4.32) = 8.14, p = .038, but no significant effect of modulation rate, F(1,3) = 6.24, p = .090]. There was a significant interaction between carrier frequency and modulation rate, F(3,9) = 4.43, p = .036, indicating that thresholds increased more with carrier frequency for a modulation rate of 256 Hz than they did for a modulation rate of 128 Hz (see Figure 3(a) and (b)).

Figure 3.

ITD discrimination thresholds for all subjects as a function of carrier frequency for modulation rates of (a) 128 Hz and (b) 256 Hz. Thresholds increase with carrier frequency, particularly at 256 Hz. (c) Highest modulation rate for which each subject showed a threshold below 1000 μs, plotted as a function of carrier frequency.

Consistent with the findings of Bernstein and Trahiotis (2002, 2014), however, and inconsistent with those of Majdak and Laback (2009), the upper modulation rate at which ITD-discrimination thresholds below 1,000 µs could be obtained decreased as a function of carrier frequency (Figure 3c). Because the bandwidths of cochlear filter increase with center frequency, the effect of peripheral filtering cannot wholly account for limitations associated with increasing modulation rate.

Effects of Peripheral Filtering

To account for the effect of cochlear filtering on sensitivity to ITDs conveyed in transposed tones, we modeled in MATLAB the output of a series of cochlear filters using a fourth-order gammatone filter-bank (Hohmann, 2002; Patterson, Nimmo-Smith, Holdsworth, & Rice, 1987) with filter widths equal to the equivalent rectangular bandwidth (Glasberg & Moore, 1990) and a filter density of 10 filters per equivalent rectangular bandwidth. The Hilbert envelopes of the filter outputs were first calculated, and the modulation depth computed as the ratio of the difference between the maximum and minimum magnitude of the envelope and their sum. Additionally, first derivatives of the Hilbert envelopes were calculated and the maximum value of the envelope gradients determined. The root-mean-squared (RMS) level at the output of each filter is plotted in Figure 4(a)–(d) for transposed tones with carrier frequencies of 4, 6, 8, and 10 kHz. Note that, for a carrier frequency of 4 kHz, the magnitude of filter outputs at frequencies above and below the on-frequency filter varies between maximum and minimum values for modulation rates above 256 Hz. This is due to the sidebands being attenuated by the edges of the filter. The effect of increasing the modulation rate is also to reduce the magnitude of the output of the on-frequency filter. With increasing carrier frequency, however, the bandwidths of the peripheral filters broaden, with the consequence that attenuation of the spectral sidebands is apparent only at increasingly higher modulation rates. In terms of modulation depth (middle panels of Figure 4), as modulation rate increases, and the sidebands are attenuated in the on-frequency filter, the modulation depth of the Hilbert envelopes decreases. For a given modulation rate, modulation depth increases as carrier frequency increases. For filters centered at frequencies approximately half the modulation rate above and below the filter corresponding to the carrier frequency, the depth of modulation is, once more, maximal. Additional maxima in the magnitude of the modulation depth are found for filters equidistant from the remaining sidebands. However, because the level of the sidebands decreases with increasing distance from the carrier frequency, not all of the information conveyed in the highly modulated filters is likely to be equally beneficial for ITD discrimination. Assuming the salient envelope ITD requires sufficient acoustic energy to evoke a neural response, and sufficient modulation depth to generate neural phase locking, including potentially synchronized across neurons, we computed an index of envelope energy termed the “effective modulation depth” as the modulation depth multiplied by the RMS level of the gammatone filter output (Figure 4(i)–(l)). From these, it can be seen that for a carrier frequency of 4 kHz, the most salient temporal information is found in the on-frequency channel only for the lowest modulation rates (128 and 256 Hz). For higher modulation rates, certain off-frequency channels provide more salient information than is found in the on-frequency channel. However, as center frequency increases, the peripheral filters broaden, with the consequence that temporal information remains more salient in the on-frequency than off-frequency channels at increasingly higher modulation rates. With a carrier frequency of 10 kHz, off-frequency channels are optimal only for modulation frequencies of 700 Hz and above.

Figure 4.

RMS level of stimuli at the output of a gammatone filter as a function of filter center frequency for transposed tones with carrier frequencies of (a) 4 kHz, (b) 6 kHz, (c) 8 kHz, and (d) 10 kHz. Modulation depth of stimulus envelopes for transposed tones with carrier frequencies of (e) 4 kHz, (f) 6 kHz, (g) 8 kHz, and (h) 10 kHz. Modulation depth × RMS level for transposed tones with carrier frequencies of (i) 4 kHz, (j) 6 kHz, (k) 8 kHz, and (l) 10 kHz. Maximum gradient of stimulus envelopes for transposed tones with carrier frequencies of (m) 4 kHz, (n) 6 kHz, (o) 8 khz, and (p) 10 khz.

Because filters widen with increasing frequency, spectral components of the transposed tones become increasingly less resolved. For a carrier frequency of 4 kHz, the RMS level decreases with increasing modulation rate, as the sidebands are increasingly attenuated by the filter skirts. Because thresholds for transposed tones are level dependent (Dietz et al., 2013a), this would tend to increase thresholds. When listening in neighboring, off-frequency channels, thresholds would also be expected to be higher because the RMS level is lower within filters distant from that of the carrier frequency. Additionally, modulation depth in the on-frequency channel decreases with increasing modulation rate, and envelope ITD thresholds are also known to increase as modulation depth is reduced (Henning, 1974). Nevertheless, as is evident from the plots in Figure 4(e)–(h), an off-frequency channel always exists in which the modulation depth is high (>0.9). Bernstein and Trahiotis (2011) demonstrated that their data were best accounted for by a model which permitted off-frequency listening in channels with a more favorable modulation depth. The benefit of exploiting off-frequency listening may, therefore, explain some of the large intersubject variability observed for this task because participants may be employing different approaches to the discrimination task (i.e., listening through different filters).

The reduction in the detrimental effects of peripheral filtering with increasing carrier frequency can also be seen in the temporal domain (see Figure 5). Before cochlear filtering, increasing the modulation rate increases steepness, but reduces off-time. For a 4-kHz carrier (filtered by a 4-kHz centered cochlear filter), steepness initially increases with increasing modulation rate, but then starts to decrease at higher modulation rates as spectral components become resolved (see Figure 4(m)). In comparison, while for 128-Hz modulations, steepness increases only slightly as carrier frequency (CF) increases (Figure 4(m)–(p)), it increases substantially for higher modulation rates, such that the steepness of the cochlear filtered enveloped at 512-Hz modulation rate is a factor of three greater at 10-kHz than it is at 4 kHz, and the off-time is also relatively greater (due to a greater number of spectral components within the filter). Both of these factors would be expected to improve performance (Klein-Hennig, Dietz, Hohmann, & Ewert, 2011; Monaghan et al., 2013) at 10 kHz compared with 4 kHz but, in fact, performance is relatively poorer at 10 kHz CF. We would note that in very few instances, even at 10 kHz, does the filtered transposed signal actually replicate the presumed output of a low-frequency tone of the same rate.

Figure 5.

Transposed tones with carrier frequencies of (a) 4 kHz and (b) 10 kHz following filtering in the on-frequency filter and the off-frequency filter generating the highest value of modulation depth × RMS level for modulation rates of 256, 512, and 800 Hz.

Nevertheless, despite the potential for maintaining discrimination performance by listening through off-frequency channels, the potential benefits of off-frequency listening are likely to decline with increasing carrier frequency as, by 10 kHz, modulation rates below 600 Hz will be represented in the outputs of on-frequency filters (due to the increased bandwidth of the cochlear filters). Overall, then, although at 4 kHz, it is not possible to exclude the effects of peripheral filtering on ITD thresholds, the decline in performance with increasing carrier frequency cannot be explained by peripheral filtering, which would, all other factors being equal, predict an increased tolerance to increases in modulation rate with increasing carrier frequency. Coupled with the decline in performance with increasing carrier frequency, even at lower modulation rates where sidebands are less likely to be resolved at any of the frequencies assessed (e.g., 256 Hz), this strongly suggests the operation of an increasingly low-pass filter for processing interaural temporal information as carrier frequency is increased.

Discussion

We assessed the ability of listeners to discriminate ITDs conveyed in the modulated envelopes of high-frequency sounds, employing the transposed stimulus developed by van de Par and Kohlrauch (1997), and used by Bernstein and Trahiotis (2002) to explore the upper limit—in terms of modulation rate—at which ITD discrimination is possible in NH listeners. We also determined the extent to which envelope-ITD discrimination declined with increasing carrier frequency. Similar to Bernstein and Trahiotis (2002), we found ITD discrimination thresholds increased with increasing modulation frequency above 128 Hz, although the upper limit of performance appears to be higher than they reported. For a carrier frequency of 4 kHz, all three experienced subjects in our study showed good sensitivity (i.e., low thresholds) for a modulation rate of 512 Hz and two of the three were able to perform the task for a modulation rate of 700 Hz, and one at 800 Hz.

All of our subjects showed declining performance as carrier frequency was increased, both in terms of the lowest thresholds obtained, and in the maximum rate at which envelope ITDs were discriminable. The highest modulation frequency for which thresholds could be obtained also decreased with carrier frequency for all listeners, consistent with the findings of Bernstein and Trahiotis (2002, 2014).

Intersubject Variability in Envelope-ITD Discrimination

It is unclear which factors determine the ability to exploit ITDs at high modulation frequencies, although it appears that some listening experience is necessary, although not sufficient, to generate high performance or low thresholds. Note that Goldsworthy and Shannon (2014) found that 32 hr of training improved rate difference limens in CI listeners by a factor of two, suggesting that training in the task might be an important factor. One limitation for correct lateralization with increasing modulation rate is that the limiting value of half the period of the modulation will start to encroach on the range of potential ITD thresholds. For low frequency pure tones, Sayers (1964) demonstrated, that as the interaural phase difference (IPD) was increased, the extent of laterality increased, peaking around 90° and then decreasing to a minimum around 180°. For IPDs greater than 180°, a reversal of laterality was observed. Majdak et al. (2006) found similar results for high-frequency band-pass filtered pulse trains: ITD sensitivity decreased for IPDs above 90°, and for a phase difference of 180°, the lateralization percept was entirely ambiguous. For a modulation rate of 512 Hz, an IPD of 180° corresponds to 977 μs ITD, and a phase difference of 90° corresponds to 488 μs. At a rate of 800 Hz, an IPD of 180° corresponds to 625 μs and an IPD of 90° corresponds to 313 μs. Thus, for the poorer performers, allowing the ITD in the threshold tracking procedure to enter this ambiguous region may render thresholds unattainable, as listeners may not be able to recover sufficiently from this ambiguity to obtain a threshold recording. The region of frequency-ITD space where the IPD is between 90° and 180° is plotted as the shaded region on Figure 2. It can be seen that Participants S02 and S03 were both able to obtain thresholds in this region for a center frequency of 4 kHz. One repeat for S03 was 771, above the 180° limit. Although it was assumed that the participants were selecting the interval with the left-most sound, the task did not, in fact, require the participants to correctly lateralize the sound. Participants may have equally selected a target interval where the sound was right-leading or more interaurally diffuse than the reference intervals. Reduced extent of laterality may still have affected performance, however, particularly for the less experienced listeners, who may have relied more on the lateralization to perform the task (as the task instructions indicated they should).

The Rate Limit and Cochlear Implants

What are the consequences of an increasingly low-pass (as a function of carrier frequency) modulation filter for users of bilateral cochlear implants? With envelope fluctuations the only temporal information available, the existence of a low-pass filter has potentially serious repercussions for the ability of CI users to extract temporal information from electrical hearing. The high carrier rates (>1,000 pps; Wilson, 2004) considered necessary for good speech perception with CIs presents a problem when employing electrical pulses to convey ITD information because CI users are not sensitive to ITDs for pulse rates above (maximally) 800 pps (Majdak et al., 2006). One approach employed (Laback & Majdak, 2008) to try to overcome this limitation was to jitter the timing of the intervals between pulses, while maintaining their timing interaurally. Although the proposed reasons for the observed improvement in performance are subject to considerable debate (Brown & Stecker, 2011; Hancock, Chung, & Delgutte, 2012; van Hoesel, 2008), the application of jitter allows for an ITD of 100 μs to be discriminated with an accuracy of about 70% for (average) carrier rates of 1,200 pps. Similarly, acoustically at high carrier frequencies, adding bilateral jitter enabled NH listeners to lateralize band-filtered pulse trains with rates of 1,200 pps with an accuracy of >70% for an ITD of 400 μs (Goupell et al., 2009). Leaving aside the precise mechanism by which this apparent improvement arises, the temporal jitter applied equally to the pulse trains in each ear may distort the stimulus envelope. As speech information in CIs is conveyed largely through modulations of the stimulus envelope, temporally jittering electrical trains in an attempt to improve binaural hearing may inadvertently degrade speech processing.

Peripheral filtering does not appear to be the limiting factor in processing ITD information at high modulation rates. A central mechanism would be expected to affect both CI and NH listeners equally. In this respect, the apparent higher sensitivity for the best performing CI compared with NH participants reported in the literature is puzzling. Nevertheless, performance by CI listeners may more often be summarized according to the data obtained for the highest performing listeners (e.g., Majdak et al., 2006), whereas data from NH participants are usually summarized according to the median performance (e.g., Bernstein & Trahiotis, 2002). With considerable variability among highly trained NH participants—some NH listeners can show sensitivity to ITDs conveyed at rates as high as 800 Hz, in line with the highest performing CI listeners—it may be the case that CI users are generally more sensitive to envelope ITDs conveyed at high pulse (modulation) rates than are NH listeners, with the higher degree of peripheral phase locking in electric compared with acoustic hearing (Hartmann, Topp, & Klinke, 1984) providing a representation that is more robust to central low-pass (postsynaptic) filtering by binaural neurons.

A rate limit is also seen monaurally for CIs, with the majority of CI users being unable to discriminate changes in rate above 300 pps (Carlyon, Deeks, & McKay, 2010), although some users can discriminate pitch changes at far higher rates, up to 800 pps (Kong & Carlyon, 2010). Carlyon and Deeks (2002) also reported a limitation in the ability of NH listeners to discriminate changes in rate when acoustic pulse trains were band-pass filtered so that all of the harmonics were unresolved. For pulse trains band-pass filtered between 7,800 and 10,800 Hz, listeners were unable to discriminate changes in rate above 600 pps, consistent with temporal sensitivity (to envelope modulations) declining at increasingly higher carrier frequencies. Ihlefeld, Carlyon, Kan, Churchill, and Litovsky (2015) addressed the issue of whether temporal performance in monaural and binaural tasks is limited by the same underlying mechanism. They measured rate discrimination and ITD discrimination in the same bilateral CI listeners for pulse rates in the range 100 to 500 Hz and found performance in the ITD task to be correlated with performance in the rate-discrimination task in the worse ear. They concluded that the same, or similar, mechanisms might be involved in determining the rate limit in both tasks. Furthermore, this common factor appears to lie beyond the auditory nerve. Carlyon and Deeks (2013) measured both electrically evoked compound action potentials (eCAPs) and performance in a rate-discrimination task in the same CI listeners to determine whether limitations in following high rates could be explained by a failure of the auditory nerve to convey this information. They found no evidence of any rate limitation in neural encoding of high rates, as determined by eCAP measures, and concluded that the rate limitation was not evident at the level of the auditory nerve.

Other Lines of Evidence for a Rate Limitation in Temporal Processing

Stecker (2014) suggested that despite evidence that peripheral filtering might explain the precedence effect at lower CFs and smaller inter-click intervals (e.g., Bianchi, Verhulst, & Dau, 2013; Xia & Shinn-Cunningham, 2011), for higher CFs, where filter “ringing” is less pronounced, central effects must contribute to rate limitations, citing several physiological and human brain-imaging studies (Dietz et al., 2013b; Dietz et al., 2014; Remme et al., 2014) in support of this notion. In this study, Stecker (2014) measured temporal weighting functions for click trains at center frequencies from 1 to 8 kHz, and reported that onset clicks dominated localization judgements consistently across center frequencies, but that this dominance was reduced, albeit modestly, with increasing center frequency. A model incorporating a low-pass filter of 150 Hz provided a better fit to the data than did a model containing peripheral filtering alone, indicating that peripheral effects were not sufficient to explain onset dominance. However, particularly at higher center frequencies, the low-pass filter model underestimated the weight ascribed to ongoing envelope cues, consistent with the current study.

The rate limit was also assessed indirectly by Klein-Hennig et al. (2011) who examined the influence on ITD discrimination thresholds of the duration of pauses between pulses. They found that thresholds increased with decreasing pause duration but that, for a pause duration of 0 ms (for which, in terms of the duration between successive modulation maxima and the overall waveform shape, the stimulus was identical to a 400-Hz SAM tone), mean thresholds were still <500 μs. Dietz et al. (2013c) compared the effect of pause duration for the same stimuli employed by Klein-Hennig et al. (2011) and Laback et al. (2011). They found that for the smallest pause duration of 1 ms—where pause duration was defined as the time between points on successive pulses at which the amplitude was 0.25 times the maximum level—median thresholds were <500 μs for pulses with sine-squared onset or offset and ∼1,500 μs for those with exponential onset or offset. Measurable thresholds were obtained in over half of the six runs obtained for each of five subjects. By comparison, pause durations for 512-Hz and 600-Hz modulation rates in the current study—when defined in the manner employed by Dietz et al. (2013c)---are 1.1 and 0.97 ms, respectively.

Neural Mechanisms Accounting for Sensitivity to Envelope ITDS

One possible explanation for reduced temporal performance with increasing carrier frequency is a reduction in the temporal coding properties of brainstem neurons themselves. Employing in vitro recordings and modeling techniques, Remme et al. (2014) demonstrated intrinsic electrical properties of neurons in the high-frequency limb of the lateral superior olive (LSO) that are characterized by slower, more integrative responses. This was compared with neurons in the medial superior olive, and the lateral, low-frequency limb of the LSO, where the intrinsic electrical properties exhibited fast temporal performance. This is consistent with the expression pattern of Kv1.1-mediated potassium conductances responsible for temporally precise neural spiking across the tonotopic axis of the LSO, which declines from the low-frequency lateral limb to the high-frequency medial limb, rendering neurons less temporally precise (Barnes-Davies, Barker, Osmani, & Forsythe, 2004). Such an explanation also accords with ethological considerations: signal energy in both the spectral and modulation domains declines with increasing frequency (the “1/f” law), and in order to encode this information reliably, it makes sense to integrate energy over a longer time window, avoiding erroneous coding of faster, but perhaps spurious, temporal fluctuations. This would have the effect of reducing the upper modulation rate at which ITD information conveyed in the temporal envelope could be extracted. The postsynaptic location of this temporal filter in binaural neurons also allows for the possibility that binaural and monaural processing of envelopes is subject to different temporal filters.

Conclusions

Some listeners show good (and even excellent) thresholds for transposed tones with modulation frequencies of 512 Hz and above, but absolute thresholds and upper limits to performance in terms of modulation rate vary across listeners. At 4 kHz, low-ITD thresholds could be obtained for modulation rates up to 700 Hz, and for one listener, thresholds could still be obtained at 800 Hz. For carrier frequencies above 4 kHz, thresholds could no longer be attained at 700 Hz by any listener, but thresholds could still be attained at 600 Hz for one subject up to a carrier frequency of 10 kHz. Considering all subjects, the cutoff in modulation rate above which the task could not be performed spans the range found for CI users, from 256 to 800 Hz. Overall, the data provide additional evidence that the use of envelope ITD cues at high frequencies is limited by a low-pass filter which becomes increasingly low-pass with increasing carrier frequency. A plausible explanation for this decline with carrier frequency lies in an increasing limitation to the extraction of temporal cues by central auditory neurons.

Off-frequency listening may be important in generating low thresholds. This may affect the ability to perform this task in “real-world” situations where competing sources or noise may limit “off-frequency” listening.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: EPSRC grant EP/K020501/1, the European Union under the Advancing Binaural Cochlear Implant Technology (ABCIT) grant agreement (No. 304912) and Medical Research Council Grant G1002267 to D. M.