The fMRI Data of Thompson et al. (2006) Do Not Constrain How the Human Midbrain Represents Interaural Time Delay

Abstract

This commentary provides an alternate interpretation of the fMRI data that were presented in a communication to the journal Nature Neuroscience (Thompson et al., Nat. Neurosci. 9: 1096–1098, 2006 ). The authors argued that their observations demonstrated that traditional models of binaural hearing which incorporate “internal delays,” such as the coincidence-counting mechanism proposed by Jeffress and quantified by Colburn, are invalid, and that a new model for human interaural time delay processing must be developed. We argue that the fMRI data presented do not strongly favor either the refutation or the retention of the traditional models, although they may be useful in constraining the physiological sites of various processing stages. The conclusions of Thompson et al. are based on the locations of maximal activity in the midbrain in response to selected binaural signals. These locations are inconsistent with well-known perceptual attributes of the stimuli under consideration, as is noted by the authors, which suggests that further processing is involved in forming the percept of subjective lateral position.

Introduction

The human binaural system has been the object of extensive study for many decades. Rayleigh (1907) first identified interaural time delays (ITDs) and interaural intensity differences (IIDs) of a binaural signal presented through headphones as the primary physical cues mediating the subjective lateral position of the image of the sound, at least for lower frequencies. The binaural system is very sensitive to small changes in these cues and can discriminate ITDs on the order of 20 μs and IIDs of about 1 dB at low frequencies (e.g., Hershkowitz and Durlach 1969). Many researchers since have attempted to determine how these cues combine to form a uniform subjective image, as reviewed and discussed in Durlach and Colburn (1978).

A number of models have been proposed over many decades to describe and predict many types of binaural phenomena, as described in several comprehensive reviews (e.g., Colburn and Durlach 1978; Stern and Trahiotis 1995, 1996; Colburn and Kulkarni 2005). Most of these models were inspired by the prescient hypothesis of Jeffress (1948) who proposed that ITD could be extracted by a coincidence mechanism. Specifically, he suggested that stimulus ITDs could be coded by central units that recorded coincidences in arrivals of neural impulses stemming from peripheral auditory fibers of the same characteristic frequency (CF), one from each ear, after a fixed internal interaural delay that varied over the population. With a large number of such coincidence-counting units, these operations could be thought of as a computation of the interaural cross-correlation function of the stimuli to the two ears, after the frequency-dependent processing of the peripheral auditory system. (Our internal interaural delay is typically called characteristic delay in the physiological literature, and our models are presently based on units that are assumed to exhibit zero characteristic phase.)

This paper provides an alternate interpretation of fMRI data that were presented in a communication to the journal Nature Neuroscience (Thompson et al., Nat. Neurosci. 9: 1096–1098, Thompson et al. 2006). The study of Thompson et al. was designed to test the hypothesis that the internal delays of the putative coincidence-counting units proposed by Jeffress are limited to approximately half a period at a given characteristic frequency. Thompson et al. (2006) and others have referred to this hypothesis as “the π-limit,” and the interpretation of the data by Thompson et al. (2006) has been widely cited for more than a decade as confirmation of the existence of that limit. While we do not dispute the fMRI data themselves, we disagree with the interpretation of the data as expressed in Thompson et al. (2006) and with their consequential conclusions about the validity, at least in principle, of the Jeffress model. Specifically, we believe that an alternate interpretation of the data that is at least equally plausible leads to the conclusion that the fMRI data can be explained without assuming a π-limit in the distribution of internal delays. In the following sections, we discuss first some of the history of how the original function describing the distribution of internal delays was developed based on psychoacoustical data. We then describe the specific psychoacoustical data that motivated the fMRI experiment by Thompson et al. We then summarize the fMRI study itself, and finally contrast our interpretation of the resulting data to that of Thompson et al. (2006), arguing that the experimental data could either support or refute the existence of the π-limit, depending on how they are interpreted.

The Distribution of Interaural Delays in Models Based on Coincidence Detection

Jeffress (1948) did not specify a distribution for the internal delays of the coincidence-counting units, although he did suggest that the distribution would emphasize delays of relatively small magnitude (e.g., Jeffress et al. 1956). Colburn (1969, 1973, 1977) developed a quantitative formulation of the Jeffress model that included formal specifications of models of auditory-nerve activity and the coincidence-counting mechanism. As part of this effort, Colburn (1969) estimated the function p(τ, f) which characterizes the distribution of fiber pairs with respect to internal delay, τ, and CF, f. He did so by assuming a distribution of internal delays that was independent of CF, and fitting the form of the function to describe accurately the dependence of the ratio of N0Sπ to NπS0 binaural detection thresholds as a function of target frequency.

A subsequent model by the present authors and other colleagues, called the position-variable model, describes a method for deriving an estimate of the subjective lateral position from the centroid along the internal-delay axis of the activity of the coincidence-counting units after weighting by a pulse-shaped function along the internal-delay axis that represents the effects of IID (Stern and Colburn 1978). The function p(τ, f) was later modified somewhat by Stern and Shear (1996) to describe a wider variety of psychoacoustical data. This form of the p(τ, f) function is depicted in Fig. 1a. As can be seen, while most coincidence detectors are assumed to have internal delays of small magnitude, a small number of units are assumed to have delays of much greater magnitude. (Fig. 1b will be discussed below.)

Fig. 1.

The function p(τ, f) which describes how the coincidence-counting units are distributed with respect to internal delay and CF. a The function as proposed by Colburn (1969) and modified by Stern and Shear (1996). Maxima are observed at internal delays close to zero. b The function p(τ, f) as quantified by Hancock and Delgutte (2004). We have converted IPD to ITD and plot the sum of two left-right symmetric densities based on the specification of Hancock and Delgutte. Maxima are observed around internal delays equal to approximately one-quarter period at a given best frequency. The solid white curves in panel b indicate the so-called π-limit

One particularly interesting set of results concerns the lateralization of bandpass noise as a function of bandwidth when presented with an ITD equivalent to three quarters of the period of the center frequency. As can be seen in Fig. 2a, such a signal is perceived to be on the side of the head receiving the signal that is lagging in time at small bandwidths, but on the opposite side of the head for larger bandwidths. This phenomenon can be explained by considering the patterns in Fig. 2b. Those panels depict the combination of internal delay and CF that elicit maximum response by the coincidence-counting units as a function of internal delay and CF for the two extreme bandwidths of 50 Hz (panel b) and 400 Hz (panel c). It was suggested by Stern et al. (1988) that narrowband noise is lateralized toward the left because the shapes of the trajectories of maxima are not salient for narrowband signals, and the locus of maximal activity at 500 Hz is closer to the left side of the midline. We refer to the greater weighting paid to components of the cross-correlation function that are closer to the midline as centrality. In contrast, noise with greater bandwidths, such as 400 Hz, is perceived toward the right because the “straight” trajectory that represents the true ITD is located on the right side.

Fig. 2.

a Perceived laterality of bandpass noise with a center frequency of 500 Hz, ITD 1.5 ms, and IPD 0 degrees as a function of bandwidth. b The loci of maximum activity of binaural coincidence-counting units in response to the signal of panel a with a bandwidth of 50 Hz. c Same as b, but with a bandwidth of 400 Hz. (Panel a reproduced from Stern et al. (1988), J. Acoust. Soc. Am. 84:1285–1293, with the permission of the Acoustical Society of America)

It was further suggested by Stern et al. (1988) that the lateralization of broadband noise could be accounted for by straightness weighting, which provides emphasis to the outputs of the interaural coincidence counters that provide consistent ITD estimates in their response over a range of frequencies. Stern and Trahiotis (1992, 1998) proposed subsequently that this straightness weighting could be accomplished by a mechanism that makes use of a second layer of units that record coincidences in activity of the original Jeffress-Colburn units at a particular internal delay over a range of CFs. These formulations, with modifications over the years, have been successful in describing many binaural lateralization, discrimination, and detection phenomena. We note that many other lateralization phenomena are described equally well without straightness weighting.

In recent years, McAlpine and others (e.g., McAlpine et al. 1996, 2001) have made a series of physiological measurements in attempts to determine, empirically, the distribution of the internal delays of the coincidence-counting units, in several species of small mammals. For a given CF, the preponderance of the internal delays that McAlpine and colleagues observed appear to take on values less than approximately half a period from the midline (ITD of zero), which (as noted above) McAlpine et al. refer to as the “π-limit.” McAlpine et al. (1996, 2001), Marquardt and McAlpine (2007), and others typically report that the observed internal delays tend to cluster around 0.125 cycles or π/4 rad IPD at any given CF. Hancock and Delgutte (2004) modeled similar physiological findings using a distribution of internal delays that was the product of two density functions, one depending on the “best” interaural phase difference (IPD) and the other depending on characteristic frequency. Figure 1b is a replotting of the function p(τ, f) as described by Hancock and Delgutte, converting IPD into ITD and adding together two symmetric distributions to represent total response from the two hemispheres. The axes in Fig. 1b are consistent with those of Fig. 1a. The putative π-limit is represented in Fig. 1b by the solid curve at f = 1/2τ. Since that time, other researchers have described consistent phenomena in other species including measurements in humans using magnetoencephalography (MEG) and functional magnetic resonance imaging (fMRI) (e.g., von Kriegstein et al. 2008; Salminen et al. 2018). Other researchers have constructed models of binaural interaction that incorporate the π-limit and a formulation based on overall excitation that address the data of Stern et al. (1988) directly or indirectly without the use of straightness weighting (e.g., Hancock 2007; Dietz et al. 2009), although predictions are described only qualitatively in the latter example.

As many have observed, the π-limit is in conflict with the relatively broad distribution of internal delays described in Fig. 1a that we have found to be needed to describe the psychoacoustical detection and lateralization data (e.g., Blodgett et al. 1956; Mossop and Culling 1998; van der Heijden and Trahiotis 1999). It is also inconsistent with an even broader distribution of coincidence-counting units with respect to internal delay that was proposed by Shackleton et al. (1992) and that predicts the lateralization results in Fig. 2a without resorting to any explicit straightness-weighting mechanism. We assume that the distribution of delays that mediate binaural perception is a consequence of multiple mechanisms that, in principle, represent processing at every level at the auditory system. They need not (and should not) be assumed to reflect processing at any particular peripheral level of the system.

The Study by Thompson et al. (2006)

The communication addressed here by Thompson et al. (2006) described a study using functional magnetic resonance imaging (fMRI) that measured the physiological response to sound within a region in and close to the inferior colliculus in the human brainstem. Specifically, Thompson et al. (2006) measured the fMRI response to noise with a center frequency of 500 Hz and a bandwidth of 400 Hz presented with five different ITDs, including the ITDs of − 0.5 ms and − 1.5 ms (1/4 and 3/4 of the period of the signal at the center frequency, respectively). The measured maximal fMRI responses for these two stimuli were found to be on opposite sides of the head. In contrast, it is easy to verify that both of these signals are perceived on the same side of the head by humans, with the signal presented with the ITD of − 1.5 ms appearing at an intracranial location that is farther from the midline than the signal with the − 0.5-ms ITD (e.g., Trahiotis and Bernstein 1986; Stern et al. 1988). The human psychoacoustical observations were confirmed in a replication using inexperienced listeners by Yost et al. (2007).

Thompson et al. (2006) interpreted their data as providing phenomenological support for physiological data that indicate that the internal delays of the binaural coincidence counters fall within the range proscribed by the π-limit, suggesting that the nature of the fMRI data are consistent with that hypothesis. Specifically, the curves in panels a and b of Fig. 3 (replotted from Thompson et al. 2006) describe the relative response of Jeffress-like coincidence-counting units as a function of internal delay, including only internal delays up to the π-limit at any given CF, for the signals with ITDs of − 0.5 ms and − 1.5 ms, respectively. The smooth curves above the panels represent the sum of the predicted curves across frequency, assuming a uniform distribution of fiber pairs within the π-limit. Thompson et al. noted that the maxima of these two curves are observed to be on opposite sides of the midline, consistent with the loci of maximal response observed in the fMRI data.

Fig. 3.

Cross-correlograms used to model the putative neural response to ITDs of − 0.5 ms (left column) and − 1.5 ms (right column). a, b Cross-correlogram restricted to the putative π-limit, as plotted by Thompson et al. (2006). c, d Cross-correlograms including centrality and putative straightness weighting from the extended position-variable model using the original p(τ, f) function, also as plotted by Thompson et al. (2006). e, f Cross-correlogram using the original p(τ, f) function including centrality but not straightness weighting, plotted by the present authors. Details in text. (Panels a through d reprinted by permission from Macmillan Publishers Ltd., Thompson SK et al. Nature Neuroscience 9:1096–1098, copyright 2006)

Thompson et al. contrasted the predictions of panels a and b with those of panels c and d, also replotted from the same paper. The curves in these two panels represent the putative response of the Jeffress model with straightness weighting to the same stimuli, incorporating the effects of a p(t, f) function similar to that in Fig. 1a and straightness weighting (Stern et al. 1988; Stern and Trahiotis 1992, 1998). They noted that the maxima of the sums of these curves across frequency fall on the same (left) side, contrary to the fMRI observations (but consistent with the actual perceived lateral position by humans). This inconsistency between predictions of the Jeffress model and the fMRI data was interpreted by Thompson et al. as confirmation of the existence of the π-limit, and they concluded that “the data require a new model for human ITD processing.”

Discussion

As we have noted above (and as Thompson et al. acknowledge), the two binaural signals discussed in the Thompson et al. (2006) paper are unambiguously perceived to be lateralized on the same side of the head, so the observations of Thompson et al. (which describe the corresponding fMRI responses to be on opposite sides of the head) clearly provide, at best, only a partial explanation for the lateralization process. Thompson et al. did not speculate on how the percepts are formed, beyond noting that our present understanding is incomplete, with which we agree.

An equally plausible interpretation for the nature of the fMRI data of Thompson et al. is that these data represent the response of Jeffress-type units with the original function p(τ, f) depicted in Fig. 1a but not including straightness weighting. The putative response of Jeffress-like units including centrality weighting by the original p(τ, f) function, but not including straightness weighting, is depicted by the curves in panels e and f of Fig. 3, which were not included in the paper by Thompson et al. It is seen that the maxima of the sums of these curves across frequency also are on the opposite sides of the head for the two signals, consistent with the fMRI data. (The midline in panels e and f is denoted by the heavy vertical line.) We also note that the total amount of hemispheric activity is greater on the right side than on the left side for the predictions of panel f as well, following the observations in FIG. 6 of Stern and Trahiotis (1992) which indicate that the predicted lateralization of stimuli with ITD 1.5 ms and bandwidth of 400 Hz is slightly to the side that receives the signal that is leading in time (which would be the right side for FIG. 3 of the Thompson et al. paper).

While the straightness-weighting mechanism described in Stern and Trahiotis (1992, 1998) was originally motivated by the study of Takahashi and Konishi (1986) describing measurements made in the inferior colliculus of the barn owl, we are not assuming that the human auditory system functions in exactly the same fashion. For example, the straightness weighting in humans could occur at a more central location including the cortex. In any case, it is clear that some further processing must occur after the site of the fMRI measurement because the actual subjective lateral positions of the two signals are unambiguously on the same side of the head. Hence, we argue that the observed fMRI data are as supportive of the predictions of the Jeffress model with centrality but no straightness weighting as they are of the formulation by Thompson et al. that assumes a π-limit in the distribution of the coincidence counters with straightness weighting.

In summary, there is no compelling reason to infer from the fMRI data of Thompson et al. that “a new model of sound localization is required to relate the perception of lateralization to its neural instantiation.” Specifically, while the fMRI data may be consistent with the assumption of the existence of a π-limit in the distribution of the internal delays, the actual perceived lateral positions of the signals are not consistent with the locations of maximal response to ITDs. An alternate explanation of the fMRI data that is at least equally plausible and that is consistent with the actual perceptual phenomena is that (1) the fMRI data reflect the response of Jeffress-like coincidence counters distributed according to Colburn’s original p(τ, f) function and (2) the recordings simply measure processing before the straightness weighting. In other words, the experimental fMRI data of Thompson et al. (2006) could either support or refute the existence of a π-limit, depending on how they are interpreted. It is neither compelling nor obvious to the present authors, at least, that a new model for human ITD processing is necessary.

Author Contributions

H.S.C. developed the original theory of binaural interaction based on auditory-nerve data, including the original formulation of thep(τ, f) function. R.M.S. extended this model to describe subjective lateral position (including straightness weighting), modified thep(τ, f) function, and also performed the calculations for the present paper. L.R.B. and C.T. provided many useful discussions over decades and significant editorial contributions to the present manuscript.

Funding Information

This work was supported by the National Science Foundation (Grant IIS-10916918) for Richard Stern and by the National Institutes of Health (Grant R01 DC000100) for H. Steven Colburn. Leslie R. Bernstein and Constantine Trahiotis are supported by the Office of Naval Research (ONR Award No. N00014-15-1-2140).

Compliance with Ethical Standards

Conflict of Interest

The authors declare that they have no conflict of interest.