Time-Varying Distortions of Binaural Information by Bilateral Hearing Aids

Abstract

In patients with bilateral hearing loss, the use of two hearing aids (HAs) offers the potential to restore the benefits of binaural hearing, including sound source localization and segregation. However, existing evidence suggests that bilateral HA users’ access to binaural information, namely interaural time and level differences (ITDs and ILDs), can be compromised by device processing. Our objective was to characterize the nature and magnitude of binaural distortions caused by modern digital behind-the-ear HAs using a variety of stimuli and HA program settings. Of particular interest was a common frequency-lowering algorithm known as nonlinear frequency compression, which has not previously been assessed for its effects on binaural information. A binaural beamforming algorithm was also assessed. Wide dynamic range compression was enabled in all programs. HAs were placed on a binaural manikin, and stimuli were presented from an arc of loudspeakers inside an anechoic chamber. Stimuli were broadband noise bursts, 10-Hz sinusoidally amplitude-modulated noise bursts, or consonant–vowel–consonant speech tokens. Binaural information was analyzed in terms of ITDs, ILDs, and interaural coherence, both for whole stimuli and in a time-varying sense (i.e., within a running temporal window) across four different frequency bands (1, 2, 4, and 6 kHz). Key findings were: (a) Nonlinear frequency compression caused distortions of high-frequency envelope ITDs and significantly reduced interaural coherence. (b) For modulated stimuli, all programs caused time-varying distortion of ILDs. (c) HAs altered the relationship between ITDs and ILDs, introducing large ITD–ILD conflicts in some cases. Potential perceptual consequences of measured distortions are discussed.

Introduction

For individuals with bilateral hearing loss, bilateral hearing aids (HAs) can restore audibility to both ears and thus offer to restore access to binaural cues for sound source localization and segregation (for review, see Akeroyd, 2014; Boymans, Goverts, Kramer, Festen, & Dreschler, 2008; Byrne & Noble, 1998; Markides, 1977). However, implementation of common HA processing algorithms can distort (i.e., alter relative to unaided conditions) interaural level and time difference cues (ILDs and ITDs, respectively). Studies employing a wide range of tasks and stimuli have demonstrated that binaural performance is generally not improved, and can even be degraded, by HA amplification or simulations thereof (e.g., Drennan, Gatehouse, Howell, Van Tasell, & Lund, 2005; Keidser et al., 2006; Marrone, Mason, & Kidd, 2008; Noble & Byrne, 1990; Picou, Aspell, & Ricketts, 2014; Schwartz & Shinn-Cunningham, 2013; Van den Bogaert, Klasen, Moonen, Van Deun, & Wouters, 2006; Wiggins & Seeber, 2011; but see Ahlstrom, Horwitz, & Dubno, 2009). For example, Van den Bogaert et al. (2006) demonstrated that bilateral HA users localized a variety of sound sources (low- and high-frequency narrowband noises, a ringing telephone) more accurately without than with their HAs, after compensating for target sensation level. The authors explicitly related degradations in localization performance to distortions of ILDs and ITDs caused by the HAs, as measured through a binaural manikin using broadband noise (Van den Bogaert et al., 2006; see also Keidser et al., 2006; Picou et al., 2014).

Different HA program settings cause different types of binaural cue distortions. Several studies have examined effects of nonlinear wide dynamic range compression (WDRC) on binaural information or on performance in binaural tasks, including effects of individual compression parameters such as compression ratio, attack time, and release time (e.g., Musa-Shufani, Walger, von Wedel, & Meister, 2006; Ou, Bentler, & Goodman, 2015; Wiggins & Seeber, 2011; Wiggins & Seeber, 2012; Schwartz & Shinn-Cunningham, 2013). A majority of these studies have been conducted in normal-hearing subjects using simplified compression schemes rather than with actual bilateral HAs. In general, dynamic-range compression exerts negative but relatively modest effects on sound localization (Musa-Shufani et al., 2006; Ou et al., 2015; Wiggins & Seeber, 2011) and segregation (Schwartz & Shinn-Cunningham, 2013), primarily due to compression-related distortions of source ILDs (see Wiggins & Seeber, 2011, 2012). Effects are worst when the compression attack time is fast, the stimulus onset is slow, the stimulus amplitude envelope fluctuates over time, or when the stimulus spectrum is predominantly high frequency (e.g., Musa-Shufani et al., 2006; Wiggins & Seeber, 2011). When reliable ITD cues are available—either in low-frequency fine structure or in the envelope of high-frequency stimuli—perceptual effects of ILD distortion are reduced, and localization is relatively accurate (Musa-Shufani et al., 2006; Wiggins & Seeber, 2011). Notably, both localization and segregation deficits may be ameliorated by linking gain control between the left and right ears, a feature available in some modern HAs (see Ibrahim, Parsa, Macpherson, & Cheesman, 2013; Schwartz & Shinn-Cunningham, 2013; Wiggins & Seeber, 2013).

Apart from WDRC, other commonly implemented HA processing algorithms can also affect binaural performance. For example, Picou et al. (2014) measured sound localization and sentence recognition ability in bilateral HA users fit with three different types of directional processing algorithms: mild—roughly omnidirectional filtering, moderate—traditional adaptive directional filtering, and strong—beam-forming achieved via a proprietary (wireless bilateral) algorithm. Although moderate and strong directional processing improved sentence recognition at the group level in a moderately reverberant room, localization accuracy was significantly degraded by strong directional processing for laterally positioned loudspeakers. Localization deficits were reduced with presentation of a concomitant visual cue, but localization deficits would be expected to persist for more eccentric targets or in other scenarios where visual cues would be limited or unavailable (e.g., in the dark). Indeed, measurement (through a binaural manikin) of ITDs and ILDs provided by the three different types of directional processing using speech-shaped noise demonstrated that the strong processing eliminated (changed to zero) ITDs at all azimuths (a feature of the beam-forming algorithm), but also reduced ILDs in an azimuth-dependent manner, with little or no reduction for forward targets, but several-decibel reductions for lateral targets.

Existing data thus collectively suggest that bilateral HAs can distort binaural information in a manner that reduces performance in binaural tasks, particularly sound source localization. However, it is unclear that acoustic degradations of binaural information reported hereto are commensurate with limitations on binaural behavioral performance observed in psychophysical tests (e.g., Drennan et al., 2005; Keidser et al., 2006; Noble & Byrne, 1990; Picou et al., 2014; Van den Bogaert et al., 2006) or self-reported by many HA users (e.g., Harkins & Tucker, 2007). Binaural and spatial hearing difficulties in many bilateral HA users may of course arise from a combination of exogenous (device/acoustic) and endogenous (patient) factors, i.e., auditory and nonauditory factors such as impaired ITD sensitivity or selective attention, both of which, like audiometric thresholds, tend to worsen with age (Grose & Mamo, 2010; Maylor & Lavie, 1998). However, within-subjects studies in HA users (e.g., Van den Bogaert et al., 2006) and in normal-hearing users tested with simulated HA-processed signals (e.g., Musa-Shufani et al., 2006; Van den Bogaert, Doclo, Wouters, & Moonen, 2008; Wiggins & Seeber, 2011) strongly suggest that some binaural deficits are directly attributable to acoustic cue distortions caused by the HAs themselves.

Previous analyses of binaural distortions by HAs have generally (a) considered cues in a broadband sense, that is, without considering differential effects across frequency (but see Van den Bogaert et al., 2008), and have also (b) considered cues in a time-averaged sense, that is, without considering fluctuations in cues over time (but see Wiggins & Seeber, 2011, 2012). Broadband, time-averaged analysis of binaural information can reveal consistent biasing of cues but is inherently limited in its ability to reveal fluctuations of ITD or ILD across frequency and time, that is, across the dimensions in which the auditory system operates. This is particularly problematic when trying to extrapolate reported measurements, often made using steady-state noise, to signals that fluctuate in both frequency and time (e.g., amplitude- and frequency-modulated signals like speech). Wiggins and Seeber (2011, 2012) made measurements of time-varying ILDs (within a running 10-ms window) for a variety of signals including steady-state noise, 4-Hz amplitude-modulated noise, pulse trains, and speech (a spoken sentence), as produced by independently operating left and right compressors. In addition to a significant reduction of mean ILDs (relative to control), the authors demonstrated increased variance in computed ILDs for amplitude-modulated signals (Wiggins & Seeber, 2011), with clear ILD fluctuations over time (Wiggins & Seeber, 2012). Increased variance in ILD was further shown to correlate with increased perception of “moving/gradually broadening” images by normal-hearing listeners presented with the compressed stimuli (Wiggins & Seeber, 2011). However, these measurements were made with simulated HA compression rather than actual HAs and thus also in the absence of other processing typically implemented in tandem with compression.

The present report describes analyses of ITD and ILD across frequency and over time produced by modern digital behind-the-ear HAs. The HAs were fit with three clinically common custom programs. All programs included WDRC. Two programs included nonlinear frequency compression—an algorithm intended to compress high-frequency information to lower, more audible frequencies for patients with high-frequency hearing loss. Nonlinear frequency compression and other frequency-lowering algorithms can yield modest improvements in speech reception for some patients, but provide no benefit in other patients (reviewed in Simpson, 2009; see also Picou, Marcrum, & Ricketts, 2015; Souza, Arehart, Kates, Croghan, & Gehani, 2013), and can significantly degrade subjective sound quality (Parsa, Scollie, Glista, & Seelisch, 2011). Effects of frequency compression on binaural information, which can be critical for speech understanding in many listening environments, have not to our knowledge been evaluated. Finally, we evaluated a program that implemented bilateral beam-forming technology, designed to enhance the perception of forward-directional stimuli by outputting the same signal from left and right devices, in tandem with nonlinear frequency compression. We report that all program settings caused marked time-varying distortions of ILDs for amplitude-modulated signals (cf. Wiggins & Seeber, 2011, 2012), and that frequency compression in particular caused time-varying distortions of envelope ITDs and, relatedly, reduced interaural coherence.

Materials and Methods

Apparatus

All measurements were completed inside a walk-in anechoic chamber (2.7 m × 4.6 m, Industrial Acoustics Company, Bronx, NY). A semicircular hoop of 15 loudspeakers (Orb Mod1, Orb Audio LLC, New York, NY) was suspended from the ceiling near the center of the chamber. Five of these loudspeakers, located at 0, +13, +39, +64, and +90° (i.e., from center to directly right) were used to assess binaural information in the present study.

A Knowles Electronic Manikin for Acoustic Research (KEMAR; G.R.A.S. Sound & Vibration A/S, Holte, Denmark) was positioned at the center of the loudspeaker array. A height-adjustable stand was used to bring the ears in vertical alignment with the loudspeakers, resulting in a uniform 1 -m source-to-head distance. Left and right microphone signals (40AO ½”, G.R.A.S. Sound & Vibration A/S, Holte, Denmark) were preamplified (Type 26AC, G.R.A.S. Sound & Vibration A/S, Holte, Denmark), conditioned (12AR, G.R.A.S. Sound & Vibration A/S, Holte, Denmark), and recorded via a Tucker-Davis Technologies RX8 (Alachua, FL) at a sampling rate of 97.65625 kHz for off-line analysis.

Stimuli

Stimuli consisted of broadband (white) noise bursts and 10-Hz sinusoidally amplitude-modulated (SAM) noise bursts designed in MATLAB (Mathworks, Natick, MA), and the consonant–vowel–consonant word, “purse” (/p3rs/), spoken by a female talker. All stimuli were synthesized at a sampling rate of 97.65625 kHz using TDT System 3 hardware (RX8) and delivered to one of the five loudspeakers inside the anechoic chamber through a multichannel mixer (Ultralink Pro MX882, Behringer, Dusseldorf, Germany). Control (unamplified) recordings were made at a target level of ∼80 dB SPL, while HA recordings were made at a target level of ∼65 dB. All stimuli were ∼1 s in duration, repeated five times (noise tokens were frozen) from each loudspeaker.

Hearing aids

HAs were digital behind-the-ear HAs manufactured by Phonak (Naida Q90-SP) fit to NAL-NL2 prescriptive targets for a severe high-frequency sloping hearing loss characterized by an audiogram declining from 40 dB HL at 250 Hz to 90 dB HL at 8,000 Hz (−10 dB HL/octave). We characterized binaural distortions for three different custom programs. For all three programs, WDRC was implemented with an attack time of 1 ms and a release time of 50 ms. The compression ratio varied by frequency, from 1.7:1 at 0.29 kHz to 3.5:1 at 1.5 kHz and back to 2.3:1 at 6.6 kHz. The kneepoint also varied across frequency but was in the range of 23 to 36 dB HL across the frequencies considered in our analyses and thus ∼20 to 40 dB less than (i.e., well below) the target signal intensity in all cases. Noise reduction (NoiseBlock) was also enabled and set to moderate in all programs. The first program, labeled “Basic HA” throughout the article, implemented no other features. The second program, labeled “HA + NFC,” implemented one additional feature, nonlinear frequency compression (SoundRecover). When enabled, nonlinear frequency compression compresses each octave range of frequencies above a kneepoint according to a set compression ratio. In the present study, the kneepoint was set to 3.2 kHz and the compression ratio was set to 2.1:1, as recommended by the manufacturer for the programmed hearing loss. Thus, for example, signal components within the octave from 3.2 to 6.4 kHz were compressed into the range 3.2 to 4.5 kHz. Finally, a third program, termed “HA + NFC & BF,” implemented both nonlinear frequency compression and Phonak’s bilateral beam-forming technology (StereoZoom). StereoZoom is a proprietary algorithm, with the following description of its implementation provided by a company white paper (Phonak, 2013): “… the input signals of both microphones in both HAs are used to calculate a standard dual microphone system. The respective output signal of [each] microphone system is sent to the contralateral side using wireless transmission to cover the full bandwidth of the audio data. It is then processed together with the output signal of the ipsilateral dual microphone system using a weighting function.” Signals processed by the StereoZoom algorithm are thus intended to be highly forward directional (Phonak, 2013, p. 2) and carry ITDs near 0μs (see Picou et al., 2014).

Data Processing and Analysis of Binaural Information

All analyses were completed using custom-written scripts in MATLAB. Recordings were first digitally bandpass filtered from 20 Hz to 20 kHz to remove noise and extraneous signal components beyond the limits of human hearing using fourth-order low- and high-pass Butterworth filters. The five repetitions for each stimulus were then averaged to further improve the signal-to-noise ratio (noise was added to the recordings both by the room and by the in-ear microphone preamplifiers). Finally, stimuli were bandpass filtered into ∼1/3-octave bands with four different center frequencies (1, 2, 4, and 6 kHz). In each of these frequency bands, the signal envelope was also calculated by taking the absolute value of the Hilbert transform of the signal waveform. Binaural analyses were then completed, as described below.

Binaural information was analyzed in terms of ITDs, ILDs, and interaural coherence. ITDs were computed via interaural cross-correlation. Specifically, the ITD was taken as the peak of the cross-correlation function of the left- and right-ear signals. ITDs were calculated for the waveform in the 1-kHz frequency band and for the envelope in 2-, 4-, and 6-kHz frequency bands. Cross-correlation was calculated in a normalized sense, given by,

$$\gamma (\tau )=\frac{{\int }_0^{T}xL(t)xR(t+\tau )\text{d}t}{\sqrt{{\int }_0^T{x}_L^2(t)x_R^2(t+\tau )}\text{d}t}$$ (1)

where x_L is the signal at the left ear, x_R is the signal at the right ear, T is the signal duration, and τ defines an interval of delay between the left- and right-ear signals; τ was limited in the present study to the range of −1500 ≤ τ ≤ 1500 µs, sampled in 10 -µs steps. Note that this range subsumes the complete physiologic range of ITDs for humans, which is roughly −700 ≤ τ ≤ 700 µs (e.g., Kuhn, 1977). Conventionally, interaural coherence is defined as the maximum value of γ(τ), that is, coherence Γ = max(γ), and the ITD is given by the value of τ at Γ. Waveform (i.e., fine structure) coherence varies over the range −1 ≤ γ ≤ 1, while envelope coherence (for Gaussian noise) varies over the range ≈π/4 ≤ γ ≤ 1, as shown by Aaronson and Hartmann (2010).

In addition to computing ITD and interaural coherence for the complete left- and right-ear signals x_L and x_R, these values were computed within a running temporal window, given by,

$$\gamma w(\tau ,ti)=\frac{{\int }_{ti}^{ti+W}xL(t)xR(t+\tau )\text{d}t}{\sqrt{{\int }_{ti}^{ti+W}{x}_L^2(t)x_R^2(t+\tau )\text{d}t}}$$ (2)

where t_i denotes the beginning of the ith temporal window and W is the window duration. In the present analysis, the starting value of t_i was 0, the window size (W) was 50 ms, and the ending value of t_i was T − W (i.e., the final bin included the final 50 ms of the signal. In practice, γ_W was computed iteratively, with a 10-ms increment to t_i after each iteration. Thus, Γ_W and ITD_W are vectors of interaural coherences and ITDs, respectively, for a running 50-ms temporal window with 10-ms resolution. As test signals were ∼1 s in duration, time-varying binaural differences were generally characterized for ∼100 windows, with 40 ms of overlap between adjacent windows.

ILDs were computed by comparing the root-mean-square (RMS) value of the left- and right-ear pressure waveforms according to the standard formulation,

$$\text{ILD}=20\text{log}10(\frac{\sqrt{\frac{1}{T}{\int }_0^T{x}_R^2(t)\text{d}t}}{\sqrt{\frac{1}{T}{\int }_0^T{x}_L^2(t)\text{d}t}})$$ (3)

where the numerator and denominator yield the RMS of the right ear (x_R) and left ear (x_L) signals, respectively. By construction, larger amplitudes in the right ear lead to ILD > 0 and larger amplitudes in the left ear lead to ILD < 0.

As for ITDs and interaural coherence, ILDs were also computed in a time-varying sense, within a running temporal window, given by,

$$\text{ILD}W(ti)=20\log 10(\frac{\sqrt{\frac{1}{W}{\int }_{ti}^{ti+W}{x}_R^2(t)\text{d}t}}{\sqrt{\frac{1}{W}{\int }_{ti}^{ti+W}{x}_L^2(t)\text{d}t}})$$ (4)

where t_i and W are defined as in Equation (2). ILD_W thus gives a vector of ILDs across the signal duration, again with 50-ms bins, giving ∼100 ILD estimates across the signal duration. We also computed time-varying ILDs using shorter duration bins (e.g., 10 ms, cf. Wiggins & Seeber, 2011, 2012); changing bin size did not change the overall trajectory of ILDs across time, though bin-to-bin variance was slightly increased. Finally, for both time-varying ILD and ITD calculations, values of ITD and ILD within each frequency band were given normalized weights according to the average binaural level (ABL) in terms of RMS level per bin, such that the highest ABL bin within each frequency band had a weight of 1. These weights were applied to reduce the influence of bins with low signal energy in the calculation of the mean ITD and ILD (Figures 3–7).

Figure 3.

Calculation of time-varying ITD. (a) Normalized cross-correlation of left and right signal waveform at 1 kHz, computed according to Equation (2), for a 50-ms window running from the beginning to the end of a 10-Hz sinusoidally amplitude modulated noise presented from the +64° loudspeaker. Within each panel, the bold point along each trace indicates the estimated ITD for that temporal window, taken as the maximum value of the cross-correlation function (see text). In all panels, the vertical dashed line indicates the time-averaged ITD estimated in the Control condition (black). Scale bar indicates a change in correlation of 1.0. (b) As in (a), but for the signal envelope at 4 kHz. Scale bar indicates a change in correlation of 0.25. Note. ITD = interaural time difference.

Figure 4.

Frequency compression distorts envelope ITD. Each panel depicts the right- and left-ear signals for the 10-Hz SAM noise stimulus presented at +39°, within a given frequency band and recording condition (control and three HA programs). The calculated time-varying envelope ITD is given above each signal trace, with color intensity weighted according to the average binaural level of the corresponding signal bin (see text). Dashed black lines give the physiologic range of ITD. Note. ITD = interaural time difference; SAM = sinusoidally amplitude-modulated; HA = hearing aid.

Figure 5.

Hearing aids cause time-varying fluctuations of ILD. Each panel gives right- and left-ear signals for the 10-Hz SAM noise stimulus presented at ±64. The signal trace is wrapped to the period of the modulation cycle, 100 ms. Time-varying ILD, computed according to Equation (4), is given above each signal trace, with the size of points and color intensity weighted according to the average binaural level of the corresponding signal bin (see text). The solid colored line gives the weighted mean ILD across all bins, while the colored dashed lines give the weighted standard deviation. Note. ILD = interaural level difference; SAM = sinusoidally amplitude-modulated.

Figure 6.

Binaural cue distortions evident in noise are also evident in speech signals. (a) Right- and left-ear signals for the CVC word “purse” presented from the speaker at 0° azimuth as measured through 2, 4, and 6 kHz filters. (b) Comparison of HA signal energy in the 4 kHz frequency band without (blue) and with (green) frequency compression enabled. Frequency compression reduces signal amplitude by 9 dB but also introduces notable asymmetries in the left and right signal envelopes (black arrows). (c) Time-varying ITD (upper row) and ILD (middle row) within the 4-kHz frequency band for the word purse presented from the speaker at �imuth across conditions. Note. HA = hearing aid; ILDs = interaural level differences; ITD = interaural time difference.

Figure 7.

Binaural distortions introduce spatial conflict and spread of ITD and ILD cues. (a) Each panel schematically depicts the KEMAR manikin in the test arena. ILD (outer arc) and ITD (inner arc) were mapped onto azimuth by interpolating measured control ITD and ILD values across azimuth. Note that because of the limited number of speakers used to measure the cues, the interpolation is coarse, and the projection does not capture expected nuances such as nonmonotonicity in the ILD across azimuth. The spatial location and extent of each colored arc depends on the weighted mean and standard deviation of the computed cues. Cues are depicted for the 4 kHz band of the 10-Hz SAM noise with the source at 0 azimuth. (b) As in (a), but for �imuth. Data extending into the rear hemifield are plotted to maintain a sense of measured variance but faded out to convey that the front-back ambiguity inherent in pure ITD and ILD values limits their interpretation. Note. ILD = interaural level difference; ITD = interaural time difference; SAM = sinusoidally amplitude-modulated.

Results

Figure 1 presents time-averaged values of ILD (Figure 1(a)), ITD (Figure 1(b)), and interaural coherence (Figure 1(c)) within three different frequency bands (rows) for the steady-state broadband noise stimulus. All HA programs (colors) reduced ILDs, most especially at high frequencies (4 and 6 kHz) and at lateral source angles. Waveform ITDs (at 1 kHz) were preserved by both the basic (“Basic HA,” blue) program and by the frequency compression (“HA + NFC,” green) program, while the bilateral beam-forming program (“HA + NFC & BF,” red) program yielded waveform ITDs near 0 µs (cf. Picou et al., 2014). The same pattern of results was observed at 0.5 kHz, suggesting that waveform ITDs were preserved, except in the HA + NFC & BF case, across their useful frequency range (0.5 kHz data not shown). Results for envelope ITD were quite different, with the basic program preserving envelope ITDs, but both programs implementing frequency compression clearly distorting envelope ITDs at 4 and 6 kHz. Correspondingly, while waveform interaural coherence was preserved at 1 kHz for all programs, envelope interaural coherence was dramatically reduced at 4 and 6 kHz, approaching π/4, the theoretical lower limit of envelope interaural coherence for Gaussian noise (dashed line; see Materials and Methods section). To provide additional insight on this observation, the measured waveform interaural coherence is also given (Panel C, insets). Values of coherence approached 0, demonstrating that the waveforms in the two ears were nearly independent, in turn giving rise to nearly independent envelopes when passed through narrowband filters. These patterns are generally duplicated in Figure 2, which presents time-averaged data for 10-Hz SAM noise. Mean ILDs (Figure 2(a)) were somewhat larger than for steady-state noise, while envelope ITDs were again distorted (Figure 2(b)) and interaural coherence reduced (Figure 2(c)) by frequency compression. ITD and ILD distortions are examined in greater detail in the following sections using time-varying analyses.

Figure 1.

Hearing aids distort binaural cues in steady-state noise. (a) Whole-stimulus ILDs computed using Equation (3) for control recordings and recordings from three different HA programs (colors; see legend and text). Note that no data were measured between 64° and 90° azimuth, where the maximum of the ILD function is expected; thus the connector line is not meant as a meaningful interpolation (dashes). (b) Whole-stimulus waveform (top panel) and envelope (lower panels) ITD computed using Equation (1). ITD values exceeding the axes limits (and the physiologic range of ITD) are not shown, indicated by dotted lines. (c) Whole-stimulus waveform (top panel, insets) and envelope (lower two panels) interaural coherence, computed using Equation (1). Dashed lines indicate the theoretical lower limit of coherence (0 for waveform coherence; π/4 for envelope coherence—see text). Note. HA = hearing aid; ILDs = interaural level differences; ITD = interaural time difference.

Figure 2.

Hearing aids distort binaural cues in 10-Hz sinusoidally amplitude modulated noise. (a–c) Legend as in Figure 1. Note that the π/4 limit of envelope coherence is an approximation in this case, as the noise is no longer strictly Gaussian.

Time-Varying Distortion of ITD by Frequency Compression

Figure 3 provides an example of the manner in which time-varying ITDs were computed. Each panel shows a waterfall plot of normalized cross-correlation of left and right signal envelopes for a 50-ms running temporal window (as given by Equation (2)). Relative delay (from −1500 to +1500 µs) is given on the abscissa of each panel, and signal time (from 0 to 1 s) is given on the ordinate. In each case, the stimulus was a steady-state broadband noise burst presented from the loudspeaker at +64°, expected to produce a time-invariant ITD of roughly 500 µs based on control measurements. Figure 3(a) gives cross-correlation of the waveform within the 1-kHz frequency band, while Figure 3(b) gives cross-correlation of the envelope within the 4-kHz frequency band. The peak of each cross-correlation function, which defines the estimated ITD for the given temporal window, is demarcated by a bold point. The dashed line in each panel depicts the mean ITD across time as computed for the control data (black). Although the ITD reliably falls near the expected value at 1 kHz for all HA programs and at 4 kHz for the basic program, ITD values fluctuate widely over time in the 4-kHz band for both programs implementing frequency compression. Interestingly, the HA + NFC & BF condition produced cross-correlation functions at 1 kHz with a major peak (ITD) at 0 µs, as expected from the beam-forming algorithm, but a secondary peak (side lobe) near the veridical ITD.

Figure 4 provides further insight on time-varying ITD distortion caused by frequency compression. In this case, the stimulus was the 10-Hz SAM noise stimulus presented from the speaker at +39°. Data are given for 2-, 4-, and 6-kHz frequency bands. In each panel, the right- and left-ear signals are shown, with the time-varying ITD plotted above each signal. Variations in the color intensity of plotted ITDs represent the variations in the signal intensity (see Materials and Methods section), with ITD bins roughly centered above the signal segment from which they are derived. Although ITD values were stable across time and near expected values for all HA programs at 2 kHz, large fluctuations of ITD occurred in the 4- and 6-kHz frequency bands for the programs implementing frequency compression. Such fluctuations were not due to a lack of signal energy in high-frequency bands (e.g., computation of spurious ITD values from background noise); although frequency compression did act to reduce high-frequency signal energy, signals in the 4-kHz frequency band, for example, were only attenuated ∼5 to 6 dB in each ear (relative to “Basic HA” levels). Indeed, gross asymmetries that would be expected to precipitate large fluctuations of ITD are readily visible upon inspection of signal envelopes at both 4 and 6 kHz for both programs implementing frequency compression. Notably, these distortions are not limited to noise signals, also occurring for speech sounds (see related subsection below).

Time-Varying Distortion of ILD

As demonstrated in Figures 1(a) and 2(a), and as expected on the basis of previous reports, mean ILDs carried by HA-processed signals were reduced in magnitude relative to control across frequency and azimuth. The largest disparities occurred at high frequencies, where the ILD was naturally largest (due to the frequency dependence of head-shadow effects; e.g., Kuhn, 1977), and HA amplification was greatest (due to the sloping audiogram that was fit to the test HA). For steady-state noise in the 6-kHz frequency band, for example, ILDs were reduced by more than 50% for all three HA programs at lateral azimuths. Interestingly, HA-processed and control mean ILDs were somewhat more similar for amplitude-modulated noise (Figure 2(a)) than for steady-state noise (Figure 1(a)). Evaluation of ILDs over time provided key insight on this difference.

Figure 5 plots right- and left-ear signals and resultant ILD values as a function of time (computed according to Equation (4)) within the 4-kHz frequency band for the 10-Hz SAM noise source at +64°. To enable easier visualization of signal amplitudes and variance in ILD over the modulation cycle, the signal is “wrapped” on the 100-ms period of the envelope. Each filled dot represents the computed ILD for a given bin, with bins again plotted over the segment of the signal from which they are derived, with color intensity and size scaled by the average intensity of that segment. Colored solid and dashed lines represent the weighted mean and standard deviation, respectively, of ILD across all windows. The increased variance in ILD over time in the HA conditions is striking, with HA-processed ILDs initially exceeding those of the control condition, then decreasing over tens of milliseconds through and then below the (veridical) control value. This sloping trajectory occurs on each modulation cycle for the amplitude-modulated noise, but only at the overall signal onset for the steady-state noise (not shown). While the mean ILD is thus greater (nearer the control value) for SAM noise than for steady-state noise due to the large values of ILD occurring at each cycle onset, variance in ILD is also greater for the modulated signal (cf. Wiggins & Seeber, 2012). Time-varying distortions of ILD for the SAM noise stimulus are summarized in Table 1; potential perceptual consequences of time-varying ILD are considered further in the Discussion section.

Table 1.

Time-Varying ILD for 10-Hz SAM Noise.

	0° Azimuth		13° Azimuth		39° Azimuth		64° Azimuth		90° Azimuth
	Mean ILD (dB)	St. dev. (dB)	Mean ILD (dB)	St. dev. (dB)	Mean ILD (dB)	St. dev. (dB)	Mean ILD (dB)	St. dev. (dB)	Mean ILD (dB)	St. dev. (dB)
CONTROL
1 kHz	0.0	0.2	2.3	0.5	7.3	0.9	8.3	0.9	7.0	0.9
2 kHz	0.0	0.5	3.2	0.8	7.9	1.4	10.7	1.6	6.2	1.2
4 kHz	0.0	0.5	2.9	0.5	10.9	0.6	12.6	1.1	14.9	1.0
6 kHz	0.0	0.2	4.5	0.4	13.8	0.6	14.5	1.3	19.9	1.6
BASIC HA
1 kHz	0.0	0.5	3.4	0.9	9.5	1.8	6.4	2.6	7.2	2.4
2 kHz	0.0	0.3	2.8	1.0	7.6	2.7	11.7	4.3	7.9	3.2
4 kHz	0.0	0.4	2.2	1.0	8.8	3.3	11.2	4.5	10.5	4.2
6 kHz	0.0	0.5	1.8	0.9	7.6	3.0	8.3	3.5	13.4	5.3
HA + NFC
1 kHz	0.0	0.5	3.1	1.0	9.1	1.9	6.1	2.5	6.9	2.4
2 kHz	0.0	0.3	2.6	1.3	7.4	2.8	11.1	4.4	7.7	3.4
4 kHz	0.0	0.9	1.6	1.4	7.3	3.1	11.6	4.4	11.4	4.5
6 kHz	0.0	2.5	1.9	2.4	5.6	3.8	8.6	3.9	9.2	5.1
HA + NFC&BF
1 kHz	0.0	0.6	1.2	0.7	5.9	1.8	3.7	1.1	4.2	1.9
2 kHz	0.0	0.3	1.1	0.8	5.2	1.2	4.3	1.4	7.4	1.6
4 kHz	0.0	1.5	4.4	1.9	7.5	1.7	5.9	1.8	5.4	2.1
6 kHz	0.0	2.2	−0.6	2.8	8.2	3.4	6.3	3.1	7.4	3.2

Note. ILD = interaural level difference; HA = hearing aid; SAM = sinusoidally amplitude modulated. Means and standard deviations of ILD (in dB) computed within a sliding 50 ms window (using Equation (4)) are given for all device conditions across the four frequency bands we evaluated (rows) and for all five speaker locations (columns).

Distortion of ITD and ILD for Speech Signals

While synthetic signals such as noise are useful for the interrogation of audio systems, HAs are primarily designed for the amplification of speech. It was thus desirable to ascertain that signal distortions observed using noise signals could be reproduced using speech signals. Figure 6 gives measurements for one such signal, the CVC word “purse”, spoken by a female talker (similar measurements were produced for other speech sounds, not shown). Figure 6(a) shows the recorded signal in 2-, 4-, and 6-kHz frequency bands across conditions. As expected, the signal was significantly attenuated in the 4- and 6-kHz frequency bands for the programs implementing frequency compression (Re: “Basic HA”). Significant energy was still present, however, with the signal in the HA + NFC condition attenuated ∼9 dB in the 4-kHz frequency band (Figure 6(b)) (Note: energy in the 4- and 6-kHz frequency bands was associated with the phoneme/s/). Figure 6(c) depicts the signal in the 4-kHz frequency band with the source shifted to +39° azimuth. Time-varying ITD and ILD are also given, plotted as in previous figures. Whereas ITD was relatively static over time in both the control and “Basic HA” condition, both programs implementing frequency compression introduced large fluctuations of ITD, with little evidence of the veridical ITD in the HA + NFC condition, and fluctuation around 0 µs in the HA + NFC & BF condition. ITD distortions were even more severe in the 6-kHz frequency band (not shown), although signal energy was also lower. ILDs in the 4-kHz frequency band (Figure 6(c), middle row) were better preserved in all conditions, but still fluctuated substantially, with up to ∼5 dB standard deviations over the signal duration (which also occurred in 2-kHz and 6-kHz bands; cf. Table 1).

ITD–ILD Conflict Produced by HAs

In the preceding sections, we demonstrated that commonly applied HA program settings could distort both ITDs and ILDs. The magnitude and type of distortion depended on the HA programs, frequency bands, and signals under consideration. In general, WDRC distorted (reduced) ILDs, while frequency compression, when implemented, distorted envelope ITDs. Low-frequency waveform ITDs were mainly preserved, although the beam-forming program brought ITDs to near 0 µs while producing nonzero ILDs. Importantly, all of these scenarios disrupted the natural relationship between ITD and ILD across sound source azimuth. When the auditory system is presented with such ITD–ILD conflict, one cue is often weighted more heavily than the other (e.g., Macpherson & Middlebrooks, 2002; Rakerd & Hartmann, 1985; Wightman & Kistler, 1992). Factors affecting the perceptual weighting of ITD versus ILD include the frequency range under consideration (e.g., Wightman & Kistler, 1992), temporal (e.g., modulation) characteristics of the signal (Macpherson & Middlebrooks, 2002), and the stability of each binaural difference (e.g., rapidly fluctuating ITDs may not receive much perceptual weight, even at low frequencies where ITDs typically dominate localization judgments; Rakerd & Hartmann, 1985; cf. Wightman & Kistler, 1992). Significant individual differences in binaural cue weighting have also been demonstrated (e.g., Macpherson & Middlebrooks, 2002). Some evidence suggests that ITDs at both low and high frequencies may be particularly important for localization among HA users, as ILDs are significantly distorted by dynamic range compression (Wiggins & Seeber, 2011).

Toward a more intuitive representation of the conflicting binaural information that HA users can be confronted with, Figure 7 provides a schematic diagram of the occurrence of ITD–ILD conflict for the three tested HA programs. For simplicity, only one stimulus condition is depicted: the 4-kHz frequency band of the 10-Hz SAM noise (similar representations could be generated for other stimuli). Each panel schematically depicts the KEMAR manikin in the test arena. ILD (outer arc) and ITD (inner arc) were mapped onto azimuth by interpolating the control ITD and ILD values as a function of azimuth. The resultant spatial location and extent of each colored arc depends on the weighted mean and standard deviation of the computed interaural differences. Figure 7(a) depicts measurements for a 0° source, whereas Figure 7(b) depicts measurements for a +39° source.

The ITDs and ILDs available after HA processing (colors) clearly no longer correspond to the ITDs and ILDs associated with the veridical sound source location (black radius in each panel). In the best case (“Basic HA”), the effective spatial width of the ILD is increased and pulled toward the midline, while the ITD is relatively preserved. In the worst case, the spatial extent of the ITD nearly fills an entire hemifield, while the ILD is still reduced with a standard deviation spanning at least 10° of azimuth. Such ITD–ILD conflicts occurred in other frequency bands (especially 6 kHz, but also 2 kHz), and for other source azimuths and stimuli. While conflicts were generally largest at high frequencies, where ILD and in some cases envelope ITD distortions were largest, the beam-forming program also introduced the interesting conflict of an ITD reliably pointing to 0° at low frequencies with concomitant nonzero ILDs (cf. Figures. 1–3).

Discussion

People with hearing loss experience difficulty localizing and segregating sound sources (e.g., Akeroyd, 2014; Harkins & Tucker, 2007; Markides, 1977). While bilateral HAs improve audibility in both ears, HAs do not necessarily improve binaural outcomes and may in some cases actually reduce performance in binaural tasks (see Introduction). Here, following on the work of others (e.g., Keidser et al., 2006; Van den Bogaert et al., 2006), we have quantified distortions of binaural information caused by the HAs themselves, analyzing distortions of ITD and ILD across both time and frequency. Our analyses demonstrated substantial distortions of ILD across all of the program settings we evaluated, and large time-varying distortions of high-frequency envelope ITD (and corresponding reductions in interaural correlation) for programs implementing nonlinear frequency compression. The settings we evaluated reflect those typically implemented in patient-worn devices, consisting primarily of the manufacturer’s recommended settings (except where otherwise specified), with gain set using a common prescriptive method (NAL-NL2).

Nonlinear Frequency Compression Causes Envelope ITD Distortion

Previously, reports have suggested relatively modest distortions of ITDs by bilateral HAs, on the order of tens of microseconds shift to either side of the veridical (unaided) ITD (e.g., Keidser et al., 2006; Van den Bogaert et al., 2006), with the exception of beam-forming algorithms that effectively zero the ITD (by design) by outputting nearly identical signals bilaterally (e.g., see Picou et al., 2014). Our analysis of ITDs yielded similar data at low frequencies, with generally well-preserved ITDs for all HA programs, except the beam-forming program, where ITDs were, as expected, near 0 µs. Data were very different at high frequencies, however, in both programs that implemented nonlinear frequency compression (including program HA + NFC, which differed from “Basic HA” only in its implementation of nonlinear frequency compression). Envelope ITDs calculated at 4 and 6 kHz varied erratically over time for both speech and noise signals. Correspondingly, values of interaural envelope correlation/coherence were very low, with no prominent peaks in cross-correlation functions from which to draw well-defined ITDs.

Nonlinear frequency compression was implemented with a cutoff frequency of 3.2 kHz and a compression ratio of 2.1:1, as recommended by the manufacturer for the programmed hearing loss. Because spectral components below the cutoff frequency are unaffected by the SoundRecover algorithm, it is unsurprising that ITDs in the 1- and 2-kHz frequency bands were undistorted. However, ITDs in the 4- and 6-kHz frequency bands (above the cutoff) were severely distorted. Distortions were demonstrated both with noise signals and with the speech sound /s/ in the word “purse” (which carried most of its energy above the implemented 3.2 kHz cutoff; cf. Figure 6); similar distortions can be expected for other speech sounds containing high-frequency energy, for example, /ʃ/, /f/, /z/.

Nonlinear frequency compression is only one type of frequency lowering (see Simpson, 2009 for review). As of this writing, the SoundRecover algorithm is enabled by default on Phonak HAs, including the HAs used in this study. Frequency compression is also available on some Sivantos (formerly Siemens), GN ReSound, Unitron, and Hansaton HAs and is thus used by a substantial population of HA wearers. Although frequency compression (unlike, e.g., frequency transposition) does not affect signal components below the lowered frequency band, processing of components above the cutoff frequency evidently occurs independently in the two ears, leading to asymmetry in left and right signal envelopes (also quantified by the interaural envelope coherence, Figures 1 and 2). The asymmetry and consequent distortion of envelope ITD cues were most severe at high frequencies and for lateral source angles (Figures 1–6). While the SoundRecover algorithm is proprietary, one possible explanation for the origin of asymmetry is that the acoustic head shadow leads to greater attenuation of high-frequency components at the ear further from the source. A proportionately greater high-frequency signal thus remains at the ear nearer the source, with more energy shifted to lower frequencies via the NFC algorithm; energy that would have been shifted by the NFC algorithm at the shadowed ear is instead attenuated by the head. This explanation cannot account for reduced coherence for sources at 0° azimuth, however, so other aspects of the NFC algorithm must also lead to asymmetry. Interestingly, NFC-related reductions in coherence also appeared to disrupt the effectiveness of the beam-forming algorithm in the HA + NFC & BF program for components above the NFC cutoff frequency, as reliable ITDs near 0 µs were no longer observed. It would be interesting to explore the possibility of bilaterally linked frequency compression, which could theoretically preserve, or at least reduce the variance of, interaural envelope differences. It would also be interesting to evaluate the temporal/ITD distortions precipitated by other frequency-lowering algorithms, such as frequency transposition/translation, as any algorithm that differentially affects the signals in the two ears could introduce interaural decorrelation and ITD distortion.

ILD Distortion

HAs that implement nonlinear amplification (e.g., WDRC) distort ILDs for laterally displaced sources by amplifying the signal at the head-shadowed ear relatively more than the signal at the ear nearer the source (e.g., Keidser et al., 2006; Wiggins & Seeber, 2011, 2012). The degree and spectral extent of ILD distortion is expected to depend on a host of factors including the audiogram to which the devices are fit, compression parameter settings, and, of course, characteristics of the stimulus under consideration, including spectral content and intensity. In some respects, nonlinear compression mimics the function of nonlinear outer hair cell amplification, which is typically compromised in the impaired ear. However, particularly for modulated signals, the attack and release of WDRC can lead to large fluctuations of the ILD available to the HA user (e.g., Figure 5; Table 1): At signal onset, HA-processed ILDs may approach or even exceed unaided ILDs. Over the course of the modulation cycle, the relative difference in amplitude between the two ears decreases, such that—for a static source that should produce a fixed ILD—the ILD traverses a range of values and does so repeatedly (for each cycle of modulation). Our analysis of this phenomenon was focused on 10-Hz SAM noise, a modulation rate within the range of speech envelope modulations (e.g., Rosen, 1992) and on a representative CVC word. Similar effects were reported in a HA simulation study Wiggins and Seeber (2011, 2012) for 4-Hz amplitude-modulated noise and indeed also for speech. The present measurements underscore that real HAs (implementing other common processing algorithms in addition to WDRC) can cause significant time-varying distortions of ILDs for amplitude-modulated signals, including speech. We note that bilaterally linked compression can act to reduce ILD distortion, potentially improving binaural outcomes (see Ibrahim et al., 2013; Schwartz & Shinn-Cunningham, 2013; Wiggins & Seeber, 2013), although spatial deficits can persist even with linked compression. Ibrahim et al. (2013), for example, showed that bilaterally linked compression improved HA users’ sound localization performance, but did not their performance in a improve hearing-in-noise test, although it is not clear that listeners relied on ILD cues for these tasks (the availability/use of specific cues was not evaluated).

Implications of Measured Binaural Distortions for Perception

As described in the Introduction section, the perceptual consequences of HA-related ILD distortions have been studied considerably (although many studies have used HA simulations and normal-hearing listeners rather than actual HA users). Overall reductions and time-varying fluctuations in ILD can exert negative effects on auditory spatial acuity, characterized in one simulation study by the perception of a “moving/gradually broadening” image (Wiggins & Seeber, 2011). Relatedly, ILD distortion via dynamic range compression appears to reduce the separability (segregation) of nearby sources (Schwartz & Shinn-Cunningham, 2013; cf. Marrone et al., 2008). However, a key finding of these studies has been that the negative effects of ILD distortion may be ameliorated by simultaneously available and relatively undistorted ITDs (e.g., Musa-Shufani et al., 2006), including high-frequency envelope ITDs (Wiggins & Seeber, 2011), which are sometimes considered as less important than other cues for sound localization (Wightman & Kistler, 1992). Thus, given ILDs already distorted by WDRC, the distortion of envelope ITDs by NFC (in the frequency bands above but near the NFC cutoff that are expected to remain audible to the user) could prove particularly deleterious for binaural performance. Low-frequency fine-structure ITDs, which we found to be well preserved for all tested stimuli and programs (except the beam-forming program, as discussed previously), would seem essential for binaural benefit under these conditions. For stimuli containing primarily high-frequency energy or otherwise lacking informative low-frequency ITD cues (e.g., sirens, high-frequency speech sounds), loss of reliable envelope ITDs could be very detrimental to binaural performance indeed (cf. Monaghan et al., 2013). Future psychophysical studies should evaluate the perceptual consequences of nonlinear frequency compression and other frequency-lowering algorithms in tandem with standard dynamic range compression to establish whether distortion of envelope ITDs further degrades performance in binaural tasks. In addition to degraded localization, the spatial spread of binaural cues (see Figure 7) might be expected to limit source segregation and render broadband signals like background noise extremely diffuse.

Limitations of the Present Study and Summary

Measurements reported in the present study were made using a single HA fit to a single audiogram. Similar measurements were made with a second HA fit to a mild high-frequency sloping loss audiogram; these measurements largely recapitulated those reported and were omitted for brevity. A high-frequency sloping audiogram configuration was selected for the study because it is the most common clinically (e.g., Ciletti & Flamme, 2008). We assessed a single set of WDRC parameters, with fairly fast attack/release times reflective of only some HAs on the market. Variance in ILD for amplitude-modulated stimuli would be expected to be somewhat less with slower attack/release times, although mean reductions would be expected to be similar (dependent on overall gain settings). The primary novelty of our investigation was the evaluation of effects of nonlinear frequency compression on envelope ITD and interaural coherence. We found that nonlinear frequency compression distorted envelope ITDs and reduced the interuaral coherence of spectral components above the cutoff frequency. In tandem with time-varying distortions of ILD, these distortions severely impoverish binaural information at high frequencies. However, the perceptual consequences of these distortions remain to be tested.

Declaration of Conflicting Interests

The authors declare no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Institute on Deafness and Other Communication Disorders (NIH Grant Nos. F32-DC013927 [A.D.B.], R00-DC010206, R01-DC014948 [M.J.G.], R01-DC011555 [D.J.T.], and P30-DC004664 [University of Maryland Center of Comparative Evolutionary Biology of Hearing core grant]).