Temporal weighting functions for interaural time and level differences. VI. Effects of pulse rate on modulated envelopes

Abstract

Temporal weighting functions (TWF) were measured for binaural lateralization of amplitude-modulated trains of narrowband clicks. When the envelope was one rise-fall cycle of a 20-Hz sinusoid and the interclick interval (ICI) was 2 ms, TWFs revealed strong onset dominance in the form of maximal weight on the initial click, despite its low amplitude (−36 dB) relative to the envelope peak. Onset dominance was greater for lateralization based on interaural time differences (ITD) compared to interaural level differences (ILD), and was also reduced when the ICI was lengthened to 5 ms. At that rate, ILD weights appeared to follow the sound envelope, suggesting a level-dependent contribution of ILD that was not seen for ITD. The overall results support the view that binaural cues are accessed via a brief temporal readout window aligned to sound onsets and other onset-like events, i.e., rising segments of the amplitude envelope. The process cannot be re-triggered for at least a few milliseconds, so that binaural processing of high-rate sounds is dominated by the overall sound onset, whereas for low-rate sounds, each pulse or cycle serves as its own onset.

I. INTRODUCTION

The relative dominance of binaural cues associated with sound onsets and other rising fluctuations of a sound’s envelope (“onset–like events”) has been studied under numerous guises since at least the 1940s (see Stecker et al., 2021, for review). A key observation of this work is that the processing of such cues is highly dependent on the temporal rate of onset-like events. At rates below 100 Hz (intervals of 10 ms or more), each event contributes strongly to binaural perception. But at higher rates, above 300–500 Hz (i.e., intervals of 2–3 ms), successive events fail to contribute and the overall onset dominates—a pattern that is also seen for continuous unmodulated tones.¹ This rate dependence has serious implications for the processing of binaural cues in periodic stimuli such as electrical pulse trains delivered by bilateral cochlear implants (CI).

Some work on this problem has investigated the binaural temporal “readout” window—the brief period during which interaural time- and level-difference (ITD and ILD) cues can be effectively accessed following an onset-like event. Understanding the timing and shape of this window is an important step to understanding how listeners access binaural cues in modulated sounds. In particular, it is not well understood whether the readout window is fixed (Zurek, 1980) or adapts depending on characteristics of the stimulus such as cochlear place / carrier frequency (Bernstein and Trahiotis, 2002), electrical pulse rate (Hu et al., 2017), or envelope shape (Klein-Hennig et al., 2011). For example, Hu et al. (2017) asked listeners to detect ITDs embedded within an amplitude-modulated (AM) train of filtered pulses. Varying the pulse rate altered the apparent pattern of binaural read-out: at high rates (600 pulses per second [pps]), listeners were most sensitive to interaural delays in the early rising segment of the AM envelope. At 200 pps, detection was better for ITDs delivered near the AM peak, i.e. later in time. CI listeners exhibited the same peak-dominant pattern at 200 pps but were not tested at other rates. Interpreting the difference between 600 and 200 pps as a temporal shift of the readout window, the authors concluded that the position of the binaural readout window varies with test condition and subject group. How can we reconcile that conclusion with numerous reports of a fixed readout window aligned to onset-like events (e.g., Harris et al., 1963; Houtgast and Plomp, 1968; Saberi, 1996; Zurek, 1980)? The rate-dependence of onset dominance itself suggests an alternative explanation. In this paper, we use established methods to measure temporal weighting functions (TWF) for lateralization of stimuli fundamentally similar to those of Hu et al. (2017). Strong onset dominance at short (2-ms) ICI was reduced at 5-ms ICI, supporting the view that at low rates, each pulse serves as its own onset-like event, triggering multiple readout events rather than shifting a single readout window in time.

A. Onset dominance and binaural readout

Originally suggested by Harris et al. (1963), the binaural readout window was quantified by Zurek (1980). In that study, listeners detected ITD or ILD cues embedded at different moments within a longer noise stimulus. The data were well accounted by a brief temporal window of binaural sensitivity, lasting approx $700\mu s$ and triggered by the overall sound onset. This was followed by a period of binaural insensitivity lasting 2–3 ms, recovery from which was not complete until roughly 10 ms post-onset. Both the data and theory were highly consistent with many aspects of the well-studied “precedence effect” (Wallach et al., 1949), widely believed to play a key role in echo suppression (see Brown et al., 2015, for review). But Zurek also connected this mechanism to a broader aspect of binaural hearing: namely, the need to avoid processing ambiguous interaural delays as sound wavelengths approach the binaural head size. A readout window of $700\mu s$ (on the order of the maximum physical ITD for direct sounds) is well suited to this purpose, allowing ecologically valid ITDs to be extracted following each onset event. The subsequent insensitivity ensures that slightly longer delays—naturally present in periodic and reverberant sounds but unlikely to indicate real sources—are not processed by the binaural system. Thus, the readout window briefly interrupts binaural processing and emphasizes the binaural cues most closely aligned to sound onsets (i.e., the least ambiguous cues). The width of the window may also help to explain human listeners’ profound insensitivity to fine-structure ITD above 1400 Hz (cf. Brughera et al., 2013; Füllgrabe, 2013; Goupell et al., 2024). According to Zurek (1980), the benefit of avoiding such ambiguities transcends the effects of echoes, applying to all manner of binaural sensitivity, including even pure tones untouched by echoes or reverberation.² In our lab, Stecker and Bibee (2014) tested that hypothesis, using fine-structure ITD cues carried during the onset and later portions of a brief 500-Hz tone. Consistent with Zurek’s theory, we found strong onset dominance in the form of (a) greater sensitivity to dynamic ITD near sound onset and (b) shallow performance improvement in static-ITD detection as a function of tone duration. However, unlike Zurek (1980), we found no evidence of binaural recovery later in the sound. Instead, binaural cues seemed tonically inaccessible during the later ongoing portions of the tone.

A potential explanation for the difference in results of Zurek (1980) and Stecker and Bibee (2014) may be sought in the work of Hafter and colleagues (Hafter and Dye, 1983; Hafter et al., 1983), who measured how binaural discrimination improves with the duration of a high-frequency click train. Strong onset dominance can be inferred by shallow performance-duration functions (i.e., indicating little benefit of later ongoing sound), whereas optimal integration of the ongoing cues results in steep functions—with an ideal slope of $-1/\sqrt{N}$ (Houtgast and Plomp, 1968). Hafter and colleagues found these slopes to be strongly dependent on the click rate: nearly flat at 1–2 ms ICI and close to optimal at 10 ms ICI. Those values are suspiciously consistent with the time course suggested by Zurek (1980): total insensitivity to successive clicks occurring within 1–2 ms, but full sensitivity when ICI exceeded 10 ms. Yet the short-ICI data reported by Hafter and colleagues did not indicate any gradual recovery of binaural sensitivity, even when duration greatly exceeded 10 ms. Why not? It now appears that Zurek’s choice to use noise carriers was fortuitous; Stecker (2018) and Brown and Stecker (2011) demonstrated that binaural sensitivity to ongoing cues benefits from temporal fluctuation of the amplitude envelope within each critical band. For noise and other spectrally dense signals, these fluctuations include onset-like events at slow rates (> 10 ms intervals) which can drive binaural readout and thus reveal ongoing binaural sensitivity (Pastore and Braasch, 2019; Reed et al., 2016; Tobias and Schubert, 1959). High-rate periodic sounds and pure tones lack such fluctuations and are instead dominated by the overall onset (Stecker et al., 2021).

B. Extended envelopes and the ongoing precedence effect

The importance of sound onsets to reliable spatial representations has been noted through decades of research on the precedence effect (Brown et al., 2015; Wallach et al., 1949), in which spatial perception of a sound and its echoes is dominated by the first arriving wavefront (i.e., the direct sound). But many real-world sounds lack clear onsets and are instead characterized by ongoing fluctuations. Luckily, it turns out that obscuring or removing the overall onset has little impact on this phenomenon. Dizon and Colburn (2006) removed the overall onsets of delayed noise pairs (“lead” and “lag”) and found that lateralization varied with lead-lag delay in much the same way as when onset interaural delays were preserved. Freyman et al. (2010) used rapid $(500/s)$ trains of lead-lag noise bursts to demonstrate the same effect (now termed the “ongoing precedence effect”), at least when the noise samples fluctuated from moment to moment. When Freyman et al. (2010) altered the stimulus by using identical (“frozen”) noise bursts in all positions—so that the trains were in fact periodic—the overall onset dominated perception even when it opposed the ongoing lead location. Following Hartmann (1983), Freyman and colleagues attributed the difference to greater spectral density of the aperiodic trains, which supports localization of ongoing sound via larger and more variable envelope fluctuations within each frequency band. TWFs measured for similar stimuli demonstrated clear enhancement of perceptual weights at moments when envelopes fluctuated due to presentation of new (“fresh”) noise tokens (Stecker, 2018), which did not occur for repetitions of identical frozen noise bursts. Together with similar observations when randomizing the envelopes of narrowband sounds (Brown and Stecker, 2011; Laback and Majdak, 2008), these results lend further support to the role of envelope fluctuations in triggering binaural readout.

The ongoing precedence effect thus suggests that the binaural readout window is triggered not only by the overall sound onset, but also by onset-like events in the envelope of the ongoing sound. The results of Hafter and Dye (1983), Stecker et al. (2013), and others suggest that to be successful, the triggers must occur at least 5–10 ms apart. But how does the time course described by Zurek (1980) apply during later segments of an ongoing sound? Dietz et al. (2013) presented listeners with tones that were synchronously modulated in amplitude and interaural phase, at rates of 4–64 Hz. By varying the relative timing of the modulations, the authors determined that listeners reliably lateralized the sounds according to the interaural phase (i.e. the ITD) within a brief window early during the rising segment of the amplitude envelope: roughly 40–115° or 5–14 ms after the envelope minimum, depending on modulation rate.³

Hu et al. (2017) adopted a similar method to study ITD sensitivity in electrical and acoustical stimuli at a rate of 200 pps, filtered (for acoustic listeners) to center frequencies of 200 and 4000 Hz. A separate condition tested 600 pps acoustic stimuli filtered at 600 Hz. ITD cues applied during the rising, peak, or falling segment of an 8 or 20 Hz SAM envelope imposed on the stimulus. As in Dietz et al. (2013), a relatively long duration (500 ms) was employed to present multiple cycles (4 or 10) of the modulation period. Typical-hearing (TH) listeners exhibited better ITD sensitivity during the rising phase of the 600 Hz stimulus, but relatively greater sensitivity during the peak phase for the 200 pps stimuli, regardless of filter frequency (200 or 4000 Hz). CI users were also most sensitive to ITD conveyed in the peak phase of the SAM envelope, but were only tested at 200 pps.

Hu et al. (2017) concluded that “the position of the ITD read-out window in the modulation cycle depends strongly on the test condition as well as the subject group” [p. 1871] but were not able to reconcile the similarity of results across low- and high-frequency 200-pps conditions with the very different pattern observed at 600 Hz. Citing Klein-Hennig et al. (2011), they considered the possibility that at 200-pps, each pulse might be treated as an independent event, but they did not connect that idea to the body of literature on this topic discussed in Section I A. As a consequence, the authors were left to conclude that the rising-envelope emphasis at 600 Hz might be a specialization of binaural processing in the 400–800 Hz ITD “dominance region” (Bilsen and Raatgever, 1973; Folkerts and Stecker, 2022; Goupell et al., 2024) available to TH listeners but not CI users (who cannot access ITD cues in that frequency region).

Here, we aim to close the loop between measures of the binaural readout window with slow envelopes (e.g., Dietz et al., 2013; Hu et al., 2017; Stecker, 2018) and studies of rate-dependent onset dominance (Hafter and Dye, 1983; Saberi, 1996; Stecker et al., 2013) which imposed only rapid onsets. By directly measuring TWFs across rate and envelope conditions drawn from both sets of studies, and particularly by testing both rates at the same carrier frequency, we disambiguate the effects of pulse rate and cochlear place reported by Hu et al. (2017). Our results clearly support the “independent event” hypothesis at 5-ms ICI. Whereas onsets are dramatically overweighted at 2-ms ICI, each click contributes significantly at the lower rate. Higher-amplitude clicks received greater weight at 5 vs 2-ms ICI, giving the appearance of greater sensitivity at the envelope peak without shifting the readout window in time.

II. METHODS

The study was conducted at Vanderbilt University Medical Center (Nashville, TN). All procedures, including recruiting, consenting, and testing of human subjects, followed guidelines of the Vanderbilt University Human Research Protections Program and were reviewed and approved by the cognizant Institutional Review Board.

A. Participants

Participants were 8 adults (5 female) aged 24–46 years with self-reported typical hearing. Pure-tone air conduction thresholds < 15 dB HL were confirmed at octave frequencies 250–8000 Hz, and differed by less than 10 dB between ears. Participants provided written informed consent and were paid for their participation.

B. Stimuli and apparatus

Stimuli (shown in Fig. 1) were trains of Gabor clicks, each with carrier frequency of 4000 Hz and duration of 0.52 ms full-width half-maximum $(221\mu s/\sigma )$ as in Stecker et al. (2013). The inter-click interval (ICI) was either 2 or 5 ms. Click amplitudes were modulated 100% by a 20-Hz sinusoid. The overall duration of each stimulus was nominally 50 ms, matching the period of the sinusoidal envelope. The number of clicks in each stimulus thus differed across ICI: either 24 clicks at 2 ms ICI or 9 clicks at 5 ms ICI, with click 1 presented one ICI (2 or 5 ms) following the initial envelope minimum. The amplitude of Click 1 was thus 20 dB below the peak at 5-ms ICI, and 36 dB below the peak at 2-ms ICI. Stimuli were presented at a peak-equivalent level of 70 dB SPL.

FIG. 1.

Stimuli were trains of 4000-Hz Gabor clicks, sinusoidally amplitude modulated at 20 Hz. Interclick interval was either 2 ms (a, blue) or 5 ms (b, red). Individual click levels (c, in dB SPL, peak-equivalent) ranged by 36 dB (2 ms) or 20 dB (5 ms) relative to the envelope peak at 70 dB SPL_pe.

The binaural configuration of each click was adjusted randomly from click to click and trial to trial as in Stecker et al. (2013). Separate conditions tested ITD and ILD cues. In the ITD condition, each trial presented a “base” ITD of $-600\mu s$ (leading to the left ear), −300, 0, +300 (right leading), or $+600\mu s$. Each click was presented with an ITD equalling the base value plus a random deviation (“jitter”) drawn from a uniform distribution spanning $\pm 300\mu s$. Jitter was independently applied to each click in the train, on each trial. ITDs were applied as whole-waveform delays to each click. In the ILD condition, the base value was −6 dB (left ear more intense), −3, 0, +3 (right-favoring), or +6 dB and jitter values were drawn from a uniform distribution spanning ± 3 dB.

C. Procedure

Aside from differences in the stimuli themselves, the procedure was basically identical to that used previously to measure TWFs for binaural lateralization (Stecker et al., 2013). Testing took place in a double-walled sound attenuating booth. On each trial, a single stimulus was presented over headphones (Stax 370, Saitama Japan) and participants were instructed to indicate the apparent lateral position using a touchscreen interface (Apple iPad Mini 2, Cupertino CA) which displayed a diagram of the listener’s head and ears, with midline indicated by a vertical line. Each run presented 75 trials (15 repeats at each of 5 base values of ITD or ILD). Participants completed 4 such runs (300 trials total) for each combination of cue type (ITD or ILD) and ICI (2 or 5 ms). Conditions were tested in random order, with participants completing one run of each combination before continuing to the next set of runs.

D. Analysis

Lateralization responses were transformed to ranks per run (Stecker et al., 2013) and then subjected to multiple linear regression to fit a weighting model for each combination of participant and condition:

$${\hat{\theta }}_R=\sum _{i=1}^N{\beta }_i{\theta }_i+k,$$ (1)

where $i$ indicates the click number (of $N$ total clicks), ${\theta }_i$ the ITD or ILD applied to the click, and ${\theta }_R$ the rank-transformed lateralization response. Aside from the constant term $k$, regression coefficients ${\beta }_i$ were transformed to normalized weights $w_i$ by scaling to the sum of absolute values across coefficients:

$$w_i=\frac{{\beta }_i}{\sum _{j=1}^N\left|{\beta }_j\right|},$$ (2)

as in Stecker et al. (2013).

The set of weights $w_i$ plotted against click number $i$, constitutes the temporal weighting function (TWF) for a given participant and condition. A normalized weight of $w_i=0$ indicates no (linear) influence of the click ITD or ILD on a listener’s lateralization response, whereas a high weight $\left(w_i\to 1\right)$ indicates a relatively dominant influence, at the expense of other clicks. A reasonable null-hypothesis expectation would be for weights to scale with the energy or amplitude of the click. Past studies employing equal-energy clicks thus compared TWFs to an equal-weighting prediction, $w_i=1/N$ (dashed line in Fig. 2). Here, amplitude modulation precludes such a simple interpretation, but the weights may be compared to the amplitude of the AM envelope, which is included for reference in Figs. 2–3.

FIG. 2.

Mean temporal weighting functions (TWFs) for each of the tested conditions. Lines and error bars plot across-participant normalized mean weights and bootstrapped 95% confidence intervals (ordinate) against click number (abscissa). Grey line plots the shape of the sinusoidal envelope, normalized to sum to 1 (as were the TWFs), for reference. Simple correlation $\left(r\right)$ of the mean TWF to the sinusoidal envelope is shown in each panel. Asterisks (*) indicate individual weights which significantly exceeded the value predicted by the click’s relative amplitude $(p<.03)$.

FIG. 3.

Individual temporal weighting functions (TWFs; colored lines) for each of the tested conditions. Other formatting as in Fig. 2.

After fitting equations 1 and 2 for each combination of participant and condition, we computed mean weights ${\bar{w}}_i$ by averaging the normalized weights $w_i$ across participants, then re-scaling by $1/\sum \left|{\bar{w}}_j\right|$ (as in Eq. 2) to ensure that mean TWF values correspond to proportion of total weight. We estimated 95% confidence intervals on each mean weight by 1000-fold bootstrap estimation (Efron and Tibshirani, 1986).

As in previous studies, we quantified the non-uniformity of TWFs by calculating the average ratio, $AR$ (Saberi, 1996). $AR$ is the ratio of a single [typically, onset] click weight relative to the mean of the other weights. Here, we modified the approach of Stecker et al. (2013) to compute $AR$ separately for the onset click (click 1), the peak click (maximum amplitude click), and the offset click (click $N$). We hypothesized that strong onset dominance at 2-ms ICI would produce a large onset $AR$ in that condition that would significantly exceed $AR$ calculated for peak or offset clicks. Increasing the ICI to 5-ms was hypothesized to reduce the magnitude of onset dominance, leading to smaller differences between $AR$ for onset, peak, and offset clicks. Conversely, if the readout window were maintained but shifted to the envelope peak (Hu et al., 2017) at lower ICI, greater $AR$ should be observed for peak clicks in that condition, with a magnitude similar to the onset $AR$ at 2-ms ICI. To test these predictions, we computed across-participant mean and bootstrapped 95% confidence intervals on $AR$ values in each condition. Pairwise comparisons of $AR$ across ICI, Cue type, and $AR$ type (onset, peak, offset) were conducted using repeated-measures paired-difference bootstrap tests (i.e., resampling the $AR$ differences across participants). For these, $p$-values were computed as the proportion of bootstrapped difference scores falling at or below 0. A more detailed description of the procedure is given by Stecker et al. (2013).

III. RESULTS AND DISCUSSION

Fig. 2 plots mean TWFs (normalized mean weight versus click number) in each of the tested conditions (panels). At 2-ms ICI (upper panels), wide confidence intervals—reflecting the large number of coefficients—hinder precise click-to-click comparisons, but the results are sufficiently clear for our purposes. In both ITD and ILD conditions, click 1 carried the largest mean weight. Later clicks received less weight, particularly so in the ITD condition where weights were significantly greater than zero only for clicks 1, 2, and 19. Relatively higher weights on post-onset clicks were observed in the ILD condition, consistent with prior reports of greater sensitivity to ongoing ILD than ITD cues (Brown and Stecker, 2010; Stecker and Brown, 2010).

To assess the role of click amplitude on the weights, we computed a simple envelope-weighted prediction by normalizing the amplitudes via the same procedure as for the weights themselves (Eq. 2). The predictions, i.e., a scaled plot of the sinusoidal envelope, appear for reference in each panel of Figs. 2 and 3. Significant departures from the envelope-weighted prediction can be identified by comparing the mean-weight confidence intervals to the envelope level. Weights which significantly exceeded the prediction (based on the lower edge of the confidence interval, i.e., $p<.025$) are denoted by asterisks (*) in Fig. 2. At 2-ms ICI, these significant weights were confined to the initial clicks (1 and 2). The interested reader may also compare the confidence intervals to zero (solid black line) or to the equal-weighting prediction (dashed line), leading to broadly similar conclusions.

When ICI was extended to 5 ms (lower panels), a different pattern emerged, with more equal ITD weighting over the stimulus duration, and ILD weights which more closely scaled with click amplitude. In both cases, a local weight elevation was noted on the onset click, which significantly exceeded the relative-amplitude prediction only for ITD. Note that prior studies have tended to show moderate onset dominance at 5-ms ICI, compared to strong onset dominance at 2-ms ICI and negligible onset dominance at or beyond 10-ms ICI (Brown and Stecker, 2010; Stecker and Hafter, 2002; Stecker et al., 2013). Thus the significant onset weight in the 5-ms ITD condition is not surprising. There does not appear any evidence that the narrow region of high weight observed at 2-ms ICI shifted to a later segment of the stimulus at 5-ms ICI. In no case did click weights near the AM peak exceed the relative-amplitude prediction. Rather, the longer ICI increased weights throughout the duration of the stimulus, while preserving a local maximum on the earliest (onset) click. For ILD in particular, later weights appeared to follow the amplitude of the stimulus when the ICI was 5 ms. To quantify that observation, we computed the simple correlation (Pearson’s $r$) between the amplitude envelope and each mean TWF. The highest correlation $(r=0.83)$ was observed in the 5-ms ILD condition, indicating greater weight on more intense clicks. Negative correlation $(r=-0.18)$ in the 2-ms ITD condition indicates greater weighting of lower-amplitude clicks (e.g., click 1). The other conditions exhibited small positive correlations (2-ms ILD: $r=0.16$; 5-ms ITD: $r=0.10$) consistent with their flatter TWFs.

TWFs for individual participants are shown in Fig. 3. Not surprisingly, epochs of low mean weight were characterized by significant individual variability, with many weights falling near or below zero. Consistent epochs of significant weights were observed near sound onset at 2-ms ICI, and during later more intense segments of the 5-ms ICI stimulus. Individual TWFs appear consistent with the group means shown in Fig. 2, suggesting that regions of high and low weight represent consistent trends—i.e. broadening of the TWF rather than sharp but variable weighting of individually preferred segments.

Figure 4 plots the $AR$ values calculated for onset, peak, and offset clicks (colored bars) in each condition. Error bars indicate bootstrapped 95% confidence intervals, which appear assymetric in some cases due to the logarithmic scale of the ordinate. The reference of $AR=1$ is shown as a dotted line to indicate the expectation when all clicks receive equal weight. This value was significantly exceeded only for onset $AR$ at 2-ms ICI (for ITD, $AR=17.8$; for ILD, $AR=3.1$). Significant pairwise differences are denoted by horizontal lines. Black lines compare $AR$ values of the same type, across ICI and cue type. Onset $AR$ for the 2-ms ITD condition significantly exceeded the onset $AR$ values obtained in all other conditions ($p<.001$ in each case). For ILD, significantly greater onset $AR$ was observed for 2-ms than for 5-ms ICI $(p<.02)$. Gray lines compare $AR$ values across type (onset/peak/offset) at each combination of ICI and cue. Onset $AR$ was significantly greater than $AR$ for peak-amplitude clicks in both 2-ms conditions $(p<.03)$, and significantly greater than offset $AR$ in both ITD conditions (2-ms ICI, $p<.01$; 5-ms ICI, $p<.04$). Only in the 5-ms ILD condition did peak $AR$ exceed the other types (peak > onset, $p<.005$; peak > offset, $p<.04$). Peak $AR$ in that condition was also significantly greater than peak $AR$ in the equivalent 5-ms ITD condition $(p<.001)$.

FIG. 4.

Mean average ratio ($AR$, ordinate) by condition (abscissa) and $AR$ type (Onset (blue): Click 1 versus clicks $2-N$; Peak (red): maximum-amplitude click versus others; Offset (yellow): Click $N$ versus others. Error bars indicate bootstrapped 95% confidence intervals on $AR$ values; some extend below $AR=0$ and appear extended on the logarithmic axis used here. Horizontal lines indicate significant differences (pairwise bootstrapped difference test, $p<.05$): dark lines indicate significant effects of ICI and cue type (ITD versus ILD). Lower gray lines indicate significant differences between AR for onset, peak, and offset clicks. Dotted line indicates equal weighting $(AR=1)$, for reference.

IV. GENERAL DISCUSSION

1. Temporal weighting functions for interaural time and level differences

This paper is the sixth in a series which measured TWFs for ITD and ILD carried in trains of clicks or noise bursts. The first two publications measured TWFs for ITD and ILD discrimination in 4-kHz click trains. Brown and Stecker (2010) documented strong onset dominance for both cues which was greater for shorter ICI (1–5 ms vs 5–10 ms), consistent with earlier work (Hafter and Dye, 1983; Hafter et al., 1983; Saberi, 1996). Brown and Stecker (2011) demonstrated significant reductions in onset dominance when click times were altered via binaurally synchronous temporal jitter (Laback and Majdak, 2008). Both papers also documented greater sensitivity to ongoing ILD cues as compared to ITD cues, helping to explain cue-dependent differences in the precedence effect (Brown and Stecker, 2013) and in the importance of sound offsets for localization under some conditions (Stecker and Brown, 2010, 2012; Stecker and Hafter, 2009).

The third paper (Stecker et al., 2013) employed lateralization measures to study TWFs for ITD and ILD presented separately and in combination, confirming abrupt rather than monotonic decline in weights following onset (Houtgast and Aoki, 1994) and relatively greater effectiveness of ILD cues near the end of the sound (Stecker and Hafter, 2009). The fourth paper (Stecker, 2014) measured TWFs as a function of carrier frequency 1–8 kHz, confirming rate-dependent onset dominance at all frequencies which thus does not reflect a specific limitation of binaural processing above 2 kHz (Bernstein and Trahiotis, 2002). Additional offset-specific effects observed at 1–2 kHz were consistent with models of temporal smearing in the auditory periphery.

The fifth paper (Stecker, 2018) applied the TWF method to trains of noise bursts whose envelope fluctuations were controlled by changes between “fresh” and “frozen” noise tokens, or by imposition of sinusoidal AM. That study was motivated in part by observations of (Freyman et al., 2010) that onset dominance was greater for trains of frozen noise bursts compared to fresh bursts characterized by greater spectral density— and thus, within-band envelope fluctuations. Stecker (2018) measured TWFs for similar stimuli, reporting transient weight increases which aligned with the timing of such fluctuations (onset-like events). Weighting of binaural cues aligned with those events helps to explain the greater sensitivity to ongoing cues (i.e., reduced onset dominance) in spectrally dense signals such as noise (e.g., Pastore and Braasch, 2019; Reed et al., 2016; Tobias and Schubert, 1959) and provides a temporal account of how spectral sparsity leaves some signals particularly susceptible to onset-dominant phenomena (Freyman et al., 2010; Hartmann and Rakerd, 1989). TWF measures for AM noise-burst trains (e.g., Fig. 6 of Stecker, 2018) directly motivated the current work, by demonstrating the dominance of binaural cues carried by the initial click of the sinusoidal AM envelope, despite it having the lowest amplitude.

2. Onset dominance with slow envelopes

Compared to sounds with abrupt onset and constant amplitude (as used in the bulk of prior TWF studies), the current study and its predecessor (Stecker, 2018) described a somewhat different pattern of “onset” dominance, but one which strengthens the primary conclusions of earlier work. First, despite the gradual onset of the sinusoidal envelope, both studies found for 2-ms ICI the highest weights occurred reliably on the very first click having non-zero amplitude—even through its level was up to 36 dB below the envelope peak. This suggests that the trigger for readout must occur at an even lower level, and/or involve highly precise sampling of the initial onset event (cf. Heil, 2001).

Despite clear onset dominance for these stimuli, the shape of the TWF is indeed altered by the sinusoidal envelope. Flat-envelope TWFs tend to show an abrupt drop in weight on the second click, suggesting that onset dominance is immediate and absolute. Here and in Stecker (2018), TWFs declined roughly monotonically over the initial envelope segment. The significant weight observed on click 2 (taken as an example) could be interpreted as widening of the readout window, but more likely (in our view) reflects the statistical probability of eliciting a response at that time rather than on the initial click. Since TWFs cannot be estimated for single trials, we cannot say whether they reflect a reliable but non-synchronous mass action (e.g., differently timed sampling across multiple fibers) or a synchronous action that varies from trial to trial. Either way, the shallow envelope slope at least appears to extend the readout window somewhat toward the envelope peak, particularly so as the envelope rate declines (Stecker, 2018, Fig. 6b).

By computing the envelope phase of the maximum weight, we can estimate the position of the initial readout window across conditions. As mentioned above, greatest weight was found on click 1 in all conditions of the current study, other than 5-ms ILD, and at the equivalent position for all AM rates tested by Stecker (2018). Naturally, these correspond to the ICI in each case (2 or 5 ms), which can be expressed as 14° or 34° of the 20-Hz envelope tested here; the values for Stecker (2018) were all 2 ms, i.e., 22.5° at 31 Hz, 45° at 64 Hz, and 90° at 125 Hz. In comparison, the estimates of (Dietz et al., 2013) were much higher: 62° (11 ms) at 16 Hz, 83° (7 ms) at 32 Hz, and 115° (5 ms) at 64 Hz. If the highest weights had been aligned to the readout window positions reported by Dietz et al. (2013), the 2-ms ICI TWFs in Fig. 2 would have peaked around click 6, not around click 1.

Differences in basic stimulus features (500 Hz tones vs 4000 Hz or broadband clicks) or procedure (closed-loop adjustment vs open-loop lateralization) are unlikely to explain differences in timing of the readout window found here and by Dietz et al. (2013), based on past studies that explored these factors (Stecker, 2010; Stecker and Bibee, 2014). The overall duration, however, could be a factor. Each trial of the current study presented a single 50-ms sound (one cycle of the sinusoidal envelope), compared to 500 or 750 ms of sound consisting of multiple envelope cycles in Hu et al. (2017) and Dietz et al. (2013).⁴ It is possible that repeated exposure to the periodic envelope alters the temporal weighting of binaural cues. The matter is taken up in the following:

Adaptation or “buildup” across multiple envelope cycles

A curious observation noted by Stecker (2018) was that TWFs for sounds containing more than one cycle of the sinusoidal envelope seemed to favor different weights in later cycles than in the initial cycle. For example, 2 cycles of a 62.5 Hz envelope produced the largest weight on the initial click (of the first cycle) as described above, but the local maximum weight within the second cycle was reliably shifted one click later, i.e., 2-ms toward the envelope peak. Note this would position the readout window roughly 4 ms after the envelope minimum, quite close to the 5-ms position reported by Dietz et al. (2013) for 64 Hz AM. When 4 cycles of a 125-Hz envelope were presented, the highest weight again favored the initial click of the first cycle, but local maxima in the second through final cycles shifted 1–2 clicks toward the envelope peak. Could a similar shift have occurred in the experiment described by Dietz et al. (2013), such that judgments of the ongoing/repeated stimulus more strongly favored later segments? Unfortunately, Dietz et al. (2013) did not test single-cycle presentations suitable for comparison, nor did Stecker (2018) test more than one cycle at rates lower than 62.5 Hz.

To investigate this possibility, we conducted a brief followup experiment [see Appendix A] using stimuli with four cycles of the 20 Hz envelope. Other methods were identical to those of the experiment described above. A significant drawback of this approach is the large number of weights which must be estimated relative to the limited amount of data, resulting in highly variable TWFs and strongly limiting the conclusions that can be drawn. Nevertheless, the results are suggestive that TWFs in some conditions follow the envelope (i.e. favor clicks near the peak) more strongly as the sound proceeds. The results are consistent with the observation made by Stecker (2018) and perhaps with the later readout windows reported by Dietz et al. (2013).

3. Stronger onset dominance for ITD than ILD

TWF measurements have reliably demonstrated stronger rate-dependent onset dominance for ITD than for ILD (Brown and Stecker, 2010), a difference that likely contributes to the greater ability of ILD change to disrupt the precedence effect (Brown and Stecker, 2013; Krumbholz and Nobbe, 2002) and its greater contribution to binaural processing at sound offset (Stecker and Hafter, 2009; Stecker et al., 2013). The current results are wholly consistent with that difference. At 2-ms ICI, onset dominance (i.e., onset $AR$) was significantly greater for ITD than ILD. At 5-ms ICI, TWFs were less onset-dominated overall and ILD functions more closely followed the amplitudes of the imposed envelope. Both features suggest greater access to ongoing ILD than ITD cues, consistent with de novo extraction of ILD via more central brain mechanisms which may not be subject to binaural readout (Stecker and Hafter, 2009).

4. Clinical implications

Hu et al. (2017) were motivated to compare binaural readout across pulse rates for the purpose of better understanding the binaural-processing limitations of CI users. In particular, because many outcomes (e.g., speech) benefit from high pulse rates, clinical progress is frustrated by the apparent inability of many CI users to access binaural cues much beyond 200 pps (Van Hoesel, 2008). Hu et al. (2017) suggest a role of cochlear place, in that TH listeners seem to perform best when sounds contain energy around 600 Hz, but CI users cannot access those cues. It is for this reason that the study aimed primarily to compare performance between each group’s best frequencies: TH at 600 Hz and CI at 200 pps. Yet results of the current study and its precursors demonstrate that TH listeners—like CI users—are unable to access ongoing binaural cues at rates much beyond 200 pps. This limitation applies regardless of cochlear place. Thus we conclude that the differences noted by Hu et al. (2017), much like the rate effects noted by Van Hoesel (2008), reflect only the normal limitations of the binaural system and not a specific handicap associated with CI use. Given these known limits of the binaural system, attempts to transcend the rate limitation for CI users seem likely to fail—except by mimicking the natural situations that support access to ongoing binaural cues in TH listeners, such as aperiodic fluctuations (Brown and Stecker, 2011; Laback and Majdak, 2008).

V. SUMMARY AND CONCLUSIONS

1. Temporal weighting functions (TWFs) were measured for the lateralization of high-frequency click trains presented at 2 or 5-ms ICI and amplitude modulated by a slower 20-Hz sinusoid. Lateralization by interaural time and level differences (ITD and ILD) was assessed in separate conditions.

2. Onset dominance was observed in the form of maximal weight on the initial click, despite its low amplitude 20 to 36 dB below the envelope peak. Onset dominance was greater for 2- than 5-ms ICI, and greater for ITD than ILD conditions.

3. Contrary to suggestions in the literature, lowering the click rate from 2-ms to 5-ms ICI did not shift the high-weight segment of TWFs toward the envelope peak. Rather, it reduced onset dominance and flattened the TWFs overall.

4. For ILD conditions tested at the lower rate, this revealed a clear dependence of click weights on their amplitudes.

5. Some aspects of the data, and a brief followup experiment, appear consistent with a gradual shift toward amplitude-dominated TWFs over repeated presentations of the periodic envelope.

APPENDIX A: MULTIPLE ENVELOPE PERIODS

A brief followup experiment was conducted to examine whether TWFs adapt to repeated presentation of the sinusoidal envelope. Stimuli were identical to those of the main experiment, but with duration of 200 ms (four periods of the 20 Hz sinusoidal envelope). The task and analytical procedures were identical to the main experiment. Participants were 7 of the 8 individuals who completed the main study (one participant was not able to return for followup testing). As in the main experiment, each completed 4 runs of 75 trials per condition.

Due to the large number of regression coefficients required to be estimated (39–99 click weights) relative to the number of trials (300 per condition), weight estimates were more variable than in the main experiment. For this reason, individual-participant TWFs were smoothed using a 10-ms rectangular window (i.e., spanning 5 * 2-ms ICIs [6 click weights] or 2 * 5-ms ICIs [3 click weights]) prior to computing the mean and bootstrapped confidence intervals for plotting in Fig. 5 and further analysis.

FIG. 5.

Mean smoothed TWFs for stimuli presenting 4 cycles of the 20 Hz modulator (line). Violet line indicates the mean TWF, computed from individual TWFs each smoothed with a 10-ms rectangular window. Yellow patch indicates the bootstrapped 95% confidence interval on the mean weights. Gray line indicates the AM envelope for reference.

To assess whether TWFs more closely followed the envelope during later cycles, we computed the simple correlation between the mean smoothed TWF and the sinusoidal envelope, separately for each of the 4 envelope periods. Correlation values are given in Table I. Positive correlation indicates similarity to the envelope, i.e., greater weighting of higher-amplitude clicks. Negative correlations indicated greater weighting of lower-amplitude clicks, e.g., local onset or offset clicks within each envelope period. Both of the 2-ms ICI conditions showed greater positive TWF-envelope correlation in the final period than in the initial period (which was negatively correlated in both cases). Results for the 5-ms ICI conditions were less clear, with ITD showing greater correlation in Period 1 and negative correlation in Period 4. The 5-ms ILD condition showed the opposite pattern, ending with very tight correlation $(r=.94)$ of the TWF to the envelope in the final period. TWFs of the corresponding condition in the main experiment also correlated to the envelope ($r=0.83$, see Fig. 2).

TABLE I.

Simple correlation (Pearson’s $r$) between the smoothed mean TWF and the sinusoidal envelope, calculated separately for each of the 4 envelope periods (center columns), and over the full duration of the stimulus (right column)

Condition	Period: 1	2	3	4	Full
ITD, 2 ms	−. 93	. 03	−. 57	. 53	−. 35
ILD, 2 ms	−. 30	. 46	−. 18	. 57	. 11
ITD, 5 ms	. 84	−. 39	−. 41	−. 24	. 03
ILD, 5 ms	. 61	−. 33	. 85	. 94	. 54

The results appear consistent with the trend noted by Stecker (2018): weights in short-ICI conditions shifted toward the envelope peaks over multiple periods of a sinusoidal envelope. The effect can be qualitatively compared to the “buildup of precedence effect” Freyman et al. (1991), in which precedence effects appear to strengthen, and/or extend to longer lead-lag delays, with repeated presentation of the lead-lag stimulus (see also Brown and Stecker, 2013; Brown et al., 2015). The buildup of precedence effect enhances rather than diminishes onset dominance over time, predicting a reduction in correlation values from Period 1 to Period 4, as greater onset dominance shifts the weight away from the envelope peak and toward its local onset. Such an effect was observed in the 5-ms ITD condition, a logical place to look for it given that 5-ms lead-lag delays fall near the sweet spot for precedence buildup effects (i.e. just beyond the “echo threshold”), and that such effects appear especially robust for ITD, but not ILD, cues (Brown and Stecker, 2013). Shifts toward the envelope peak in other conditions suggest a different impact of stimulus history on binaural-cue weighting.

APPENDIX B: CENTRAL VS LATERAL PRESENTATION

Some past studies have suggested differences in cue weighting for sounds presented near or far from the interaural midline. For example, Stecker et al. (2026) reported higher-frequency spectral weighting for sounds near 0 μs versus 200 μs ITD. Freyman et al. (2010) reported weaker onset dominance when an ongoing ITD was fixed at 0 μs versus 500 μs.⁵ Both results suggest potentially greater access to high-rate ongoing cues (i.e., weaker onset dominance) near 0 μs. To investigate that possibility, we re-computed TWFs for subsets of trials as a function of base ITD (0, ±300, or ±600 μs) or ILD (0, ±3, or ±6 dB) using the method described in Section IID. Mean normalized TWFs are plotted in Fig. 6. Despite noisier weight estimates due to the limited number of trials in each subset, key features appear to match those of Fig. 2 and did not differ across base values. In particular, onset weights appear equally large for trials with 0 μs base ITD and for lateral trials. Inset tables show $AR$ values computed from the normalized mean TWFs (not directly comparable to individual $AR$ values in Fig. 4), which corroborate that observation. The result is not consistent with reduced onset dominance near 0 μs ITD.

FIG. 6.

Comparison of TWF across different values of base ITD or ILD. Lines plot mean normalized TWFs computed for subsets of trials with base value of 0 μs or 0 dB (“0,” black lines), ±300 μs or ±3 dB (“±3,” blue), or ±600 μs or ±6 dB (“6,” violet) in each experimental condition (panels). Note that because each TWF is separately normalized, averaging the three curves will not produce the overall mean TWFs from Fig. 2. Tables show the $AR$ onset values computed from the mean TWF for each condition and subset. For reference, gray lines plot the AM envelope and dashed lines plot the equal weighting value $(1/N)$.