Creating a sense of auditory space

Abstract

Determining the location of a sound source requires the use of binaural hearing – information about a sound at the two ears converges onto neurones in the auditory brainstem to create a binaural representation. The main binaural cue used by many mammals to locate a sound source is the interaural time difference, or ITD. For over 50 years a single model has dominated thinking on how ITDs are processed. The Jeffress model consists of an array of coincidence detectors – binaural neurones that respond maximally to simultaneous input from each ear – innervated by a series of delay lines – axons of varying length from the two ears. The purpose of this arrangement is to create a topographic map of ITD, and hence spatial position in the horizontal plane, from the relative timing of a sound at the two ears. This model appears to be realized in the brain of the barn owl, an auditory specialist, and has been assumed to hold for mammals also. Recent investigations, however, indicate that both the means by which neural tuning for preferred ITD, and the coding strategy used by mammals to determine the location of a sound source, may be very different to barn owls and to the model proposed by Jeffress.

Binaural hearing – a historical context

The ability to localize the source of a sound has obvious survival value for prey and predator alike. For over 50 years, a single model has dominated thinking on how the brain determines the position of a sound source in space. The coincidence detection model, proposed by Lloyd Jeffress in 1948 (Jeffress, 1948), suggests that the brain transforms information about the relative time of arrival of a sound at the two ears into a ‘place code’ – a map of auditory space. The Jeffress model, illustrated in Fig. 1, consists of an array of coincidence-detector neurones that fire maximally when action potentials arrive simultaneously from each ear. This array of coincidence detectors is fed by a series of delay lines, axons of different length from each ear. Action potentials generated by a sound arriving earlier at one ear are delayed in time so that those from the other ear can catch up, arriving at the neurone simultaneously. The neurone signals the position of the sound source by virtue of its maximum rate. Different neurones in the array have axon lengths from the two ears that differ in a systematic fashion. Thus, each neurone in the array encodes for a different interaural (between the ears) time difference, or ITD, and by extension a different spatial position. Elegant work in the barn owl – a species that hunts at night using only hearing to locate its prey – appears to confirm this model (Knudsen & Konishi, 1978a; Moiseff & Konishi, 1981; Konishi et al. 1985; Takahashi & Konishi, 1986; Proctor & Konishi, 1997). The brain of the barn owl contains a systematic array of neurones sensitive to different ITDs, forming a map of auditory space (Carr & Konishi, 1988, 1990; Carr & Boudreau, 1993). Because of its inherent elegance, the Jeffress model is well known beyond auditory neuroscience, and has been influential in engineering and psychophysics (Colburn, 1973; Colburn et al. 1990), where problems such as detecting speech in a background of noise have relied on its ‘engineering systems’ design to explain how human spatial hearing is realized, and how automatic sound source separation and speech recognition in noisy listening environments might be employed (Breebaart et al. 2001). However, recent studies have called into question the means by which ITD is encoded in the mammalian brain (McAlpine et al. 2001; Brand et al. 2002; Shackleton et al. 2003; Hancock & Delgutte, 2004), and the means by which neurones are rendered sensitive to a preferred ITD (Brand et al. 2002).

Figure 1. The Jeffress model of binaural coincidence detection.

The model consists of an array of coincidence detectors – neurones that respond maximally when inputs arrive from the two ears simultaneously – innervated from each ear by a series of delay lines – axons of variable path length. The purpose of this arrangement is to create a map of auditory space from information about the relative time of arrival of a sound at the two ears.

The duplex theory of binaural hearing

Traditionally, sound localization cues that rely on binaural (two-eared) hearing have been divided into low- and high-frequency processes. In their classic description of human pure-tone localization in the free field, Stevens & Newman (1936) observed that human subjects performed best (i.e. showed fewest localization errors for sound sources in azimuth) for frequencies below 1.5 kHz, and above 5 kHz. Localization errors were most common for mid-frequency sounds, peaking around 3 kHz. This indicates that humans, and probably other species with acoustic sensitivity spanning a broad frequency range extending down into the sub-kilohertz range, tap into two different mechanisms when localizing sound sources in the horizontal plane, one operating at low frequencies, and one operating at high frequencies. This dichotomy, referred to as the duplex theory of binaural hearing and first proposed by Lord Rayleigh (1907), arises due to the physical dimensions of the acoustic stimulus. The head can create an acoustic shadow at the ear furthest from the sound source, with the effect that the sound is less intense at that ear. These interaural level differences (ILDs) were, even in the late nineteenth century, widely recognized as a cue for localizing a sound source in space. However, equally, it was recognized that the magnitude of ILDs depends on the spectral content of the stimulus. In particular, the low-pass filter characteristics of the head, whereby low-frequency sounds pass relatively easily around the head by virtue of their longer wavelength, result in ILDs becoming negligible with decreasing sound frequency; the smaller the head size, the higher the cut-off frequency at which this occurs. At that time it was considered that differences in the time of arrival of the sound at the two ears, brought about by the incidence angle of the sound source relative to the head, were too small to be detectable, being, at most, a few hundreds of microseconds. Rayleigh's confirmation of the early work of Thompson (1877, 1878) demonstrating that small temporal differences between the ears were indeed detectable, and could provide for the ability to localize low-frequency sound sources, marked a critical juncture in the development of a theory of spatial hearing, and the duplex theory is now an established tenet of binaural processing.

Comparative binaural hearing

The textbook versions of binaural hearing are most often drawn from studies carried out in the barn owl – a binaural hearing specialist. Unlike mammals, however, barn owls localize only very poorly for frequencies below 3 kHz and above 10 kHz (Knudsen & Konishi, 1979; Coles & Guppy, 1988), but their localization abilities are exceptionally well developed between these limits. Consistent with this, physiological recordings in a variety of barn owl auditory nuclei routinely report ITD-sensitive neurones tuned, in terms of their sensitivity to sound frequency, in the ecologically relevant (to the barn owl at least) range 5–9 kHz (e.g. Cohen & Knudsen, 1994). This ability to utilize temporal information in the stimulus fine-structure at such high frequencies must be underpinned by a unique specialization in the processing within their cochleae – and auditory nerve fibres in the barn owl are able to signal the fine-structure of the sound waveform in the timing of their discharges (phase-locking) up to 8 or 9 kHz (Koppl & Carr, 1997). Typically, in mammals, phase-locking in the auditory nerve declines above 3 kHz or so, and is completely absent by 4 or 5 kHz (Palmer & Russell, 1986). Consistent with this, ITD sensitivity in mammals is low-pass with respect to sound frequency; physiological recordings in a range of mammalian species demonstrate few instances of neural sensitivity to ITDs for frequencies above 2 kHz.

Neural basis of sensitivity to ITDs

The dichotomy of binaural hearing into low- and high-frequency processes is strengthened by the existence of apparently dedicated lower-brainstem pathways subserving it. The medial superior olive (MSO) shows a relative over-representation of low-best frequency (BF) neurones and, in many mammals, appears specialized for processing ITD information (Goldberg & Brown, 1969; Yin & Chan, 1990), whilst the lateral superior olive (LSO), with a relative over-representation of high- ( > 2 kHz) BF neurones, processes ILD information. The tonotopic organization at subsequent auditory centres in the auditory pathway maintains this separation (Adams & Warr, 1976; Beckius et al. 1999; Ramachandran et al. 1999), at least in mammals (an anatomical dichotomy also exists in the barn owl, but here reflecting a division into pathways encoding interaural time and level differences spanning the same frequency range, Konishi et al. 1985).

Binaural coincidence detection

Strong evidence exists for a process of binaural coincidence detection underlying sensitivity to ITDs. Physiological observations indicate that temporal information, in the form of action potentials phase-locked to the carrier stimulus waveform (Fig. 2A), converges from each ear onto single neurones in the brainstem to generate interaural delay sensitivity (Goldberg & Brown, 1969; Yin & Chan, 1990; Spitzer & Semple, 1995). Responses of ITD-sensitive neurones are qualitatively similar between different mammalian species, and between mammals and birds. Probing the response of such neurones to pure tones over a range of ITDs reveals cyclic input–output functions (Goldberg & Brown, 1969; Kuwada & Yin, 1983), with response maxima at multiple periods of the stimulus waveform (see Fig. 2B). The cyclic nature of the functions has been taken to indicate that binaurally responsive neurones are sensitive to interaural phase differences (IPDs) rather than to interaural time differences per se. It is also well established that such neurones often show an ITD at which the relative magnitude of the response is independent of stimulus frequency (Rose et al. 1966; Yin & Kuwada, 1983), presumably reflecting the difference in axonal conduction time from the two ears to the binaural coincidence detector.

Figure 2. Phase-locking and neural sensitivity to interaural time differences.

A, illustration of the concept of phase-locking. In response to low-frequency (< 4 kHz) sounds, auditory nerve fibres preferentially fire at the same stimulus phase. Such phase-locking is a requirement of neural sensitivity to interaural time differences (ITDs). B, response of a neurone in the inferior colliculus of the guinea pig to interaurally delayed pure tones of various frequencies. Note the cyclic nature of the response for each frequency, and the alignment at just one ITD of a peak response across all the functions.

Ethological considerations

Barn owls, utilizing ITDs at high frequencies, are faced with the challenge that although a sound source might hold a single position in azimuth, its location may not be inferred from the output of a single ITD channel in the barn owl brainstem (Fig. 3). At some frequencies, multiple cycles of the stimulus waveform arrive at the nearer ear before the first arrives at the farther ear. The action potentials phase-locked to each cycle provide binaural coincidence detection across a range of ITDs encompassing the barn owl's head-width. This proves problematic for barn owls when they attempt to localize narrow-band sound sources and, predictably, they are susceptible to the perception of phantom auditory images and consequent errors in localization (Saberi et al. 1999). This problem is overcome in the barn owl by means of a systematic map of ITDs running orthogonal to the primary tonotopic map. Vertical electrode penetrations through the frequency laminae in the central nucleus of the inferior colliculus (IC) of barn owls reveal neurones within those penetrations to be similarly tuned in terms of their preferred ITD tuning (Wagner et al. 1987). Thus, each iso-frequency lamina in the barn owl auditory pathway contains a full representation of the Jeffress model, and broad-band signals activate multiple frequency channels across a single ‘ITD column’. Neurones in the external nucleus of the IC integrate the output of this ITD column, such that the frequency-dependent side peaks cancel, to leave a single peak at the ‘true’ ITD (Knudsen & Konishi, 1978b).

Figure 3. ITD tuning sharpness and sound-frequency tuning.

A, hypothetical frequency tuning curves of a mammalian (red) and a barn owl (blue) auditory neurone. The sound frequency range over which ITDs are processed is an order of magnitude higher in barn owls as compared to mammals. B, hypothetical ITD functions for mammalian (red) and barn owl (blue) binaural neurones. The high frequency range over which barn owls process ITDs as compared to mammals renders their ITD functions much more sharply tuned relative to the physiological range of ITDs they can experience (denoted by line with arrowheads on abscissa) as compared to mammals with a similarly sized head.

Mammals are faced with the opposite problem to that faced by barn owls. For low-frequency sounds, the long wavelength relative to the width of the head means that it is highly unlikely that more than one cycle of the stimulus waveform will arrive at the nearer ear before the first cycle reaches the farther ear. Phase ambiguities are therefore less likely to occur, and unlikely ever to be critical for animals with interaural distances less than about 12 cm – approximately the wavelength of a 1500 Hz sine wave – and that use ITDs for exclusively low-frequency localization. The problem facing any species using ITDs to localize low-frequency sound sources is that, simply by virtue of their wavelength, low-frequency sounds can provide for only a very coarse representation of ITD based on the peak responses of neurones. For a hypothetical low-frequency neurone tuned to zero ITD, neural output is largely unmodulated, being essentially maximal, over a wide range of ITDs (Fig. 3). ITD-sensitive neurones in mammals are necessarily broad relative to the size of the head whereas those in barn owls are, necessarily, sharp.

The neural representation of ITDs

The vast majority of ITD functions reported in the barn owl literature are tuned to ITDs within the physiologically plausible range determined by the width of the head (approx. ± 200 μs). They often lie close to zero ITD, indicating a relatively high density of neurones encoding spatial positions around midline. This accords with anatomical evidence from the barn owl nucleus laminaris (the avian homologue of the mammalian MSO) suggesting a full compliment of Jeffress-like delay lines within each frequency channel (Carr & Konishi, 1990). In contrast, physiological evidence throughout the mammalian auditory pathways reveals that the distribution of ITDs to which mammalian delay-sensitive neurones are tuned lies approximately in the range +200 to +300 μs (Fig. 4). Positive ITDs correspond to sounds leading at the ear on the opposite side of the head to the brain centre being recorded. However, this range lies outside the physically plausible range of many small mammals, and is apparently independent of head width (Palmer et al. 1992). Furthermore, convincing anatomical evidence for a systematic arrangement of delay lines in the mammalian brainstem is lacking, despite the attempts of several studies to demonstrate the existence of such an arrangement (Smith et al. 1993; Beckius et al. 1999). These empirical observations notwithstanding, a tacit assumption in many models of low-frequency binaural hearing in mammals is that an identical (and relatively broad) range of peak ITDs is fully represented within each low-frequency channel of the tonotopic gradient, for frequencies up to approximately 2 kHz. By analogy with the barn owl, it is has been suggested that there might exist a topological map of peak ITDs running orthogonal to the tonotopic axis (Yin & Chan, 1990). It comes as some surprise, therefore, that when the representation of ITD is examined as a function of the tonotopic gradient, a systematic relationship is observed, but one in which the representation of ITD runs parallel to the tonotopic axis (McAlpine et al. 2001; Brand et al. 2002; Shackleton et al. 2003; Hancock & Delgutte, 2004). This is illustrated in Fig. 4. Neurones with the lowest BFs, and therefore with the broadest ITD functions tend to have peak responses at relatively long ITDs, whilst neurones with the highest BFs tend to have peak responses at relatively short ITDs (Fig. 4A). The consequence of the BF dependence of the peak ITD is that the steepest region of the function relating discharge rate to ITD falls close to midline for all neurones irrespective of BF. Furthermore, the extent of the representation within each frequency band is not equivalent across BF, with a much wider range of ITDs producing maximum firing in lower-BF than in higher-BF neurones. A corollary of this relationship is that both the value and the range of the interaural phase difference at the peak within each frequency band are essentially invariant as a function of BF. Peak responses are distributed around 45 deg IPD (Fig. 4B), regardless of neural frequency tuning. These observations do not accord with current models of low-frequency binaural processing, and are incompatible with the accepted physiology of the barn owl auditory system. Given the relationship between BF and peak ITD, two obvious questions are: (1) how might such a representation of ITD be constructed in the central auditory nervous system, and (2) how does neural activity of this form contribute to the localization of low-frequency sound sources?

Figure 4. Preferred tuning for ITDs is dependent on neural CF.

A, responses of 6 typical inferior colliculus neurones to interaurally delayed noise (the damped side peaks of the responses are hidden to aid visual inspection). The neurone with the lowest sound-frequency tuning shows the widest function which peaks at the longest ITD. Neurones with tuning to progressively higher sound frequencies show systematically narrow functions and peak responses that lie closer to zero ITD. The grey shaded area indicates the physiological range of ITDs (approx. ± 180 ms) in the guinea pig. B, location of peak responses for a population of neurones recorded in the inferior colliculus of the guinea pig. The positions of the peak responses of the 6 neurones in A are indicated by the appropriately coloured dots. The line indicates the ITD equivalent to 45 deg interaural phase difference. The grey shaded area indicates the physiological range.

The role of glycinergic inhibition

Although a process of binaural coincidence detection underpinning sensitivity to ITDs is well established, this does not preclude a role for additional or alternative neural mechanisms in the generation of ITD sensitivity coding strategies adopted to account for behavioural sensitivity to ITDs. Recent evidence from recordings in the MSO of the gerbil, which like many mammals including humans uses ITDs to localize low-frequency sounds (< 1500 Hz; Heffner & Heffner, 1988), suggests that precisely timed inhibition plays an important role in determining the tuning of binaural neurones for their preferred ITD. MSO neurones are known to receive inhibitory (glycinergic) inputs from the brainstem medial (Cant & Hyson, 1992) and lateral (Kuwabara & Zook, 1992; Grothe & Sanes, 1993) nuclei of the trapezoid body as well as bilateral excitatory inputs. Following a period of developmental refinement, these glycinergic inputs synapse almost exclusively on the somata of MSO principal neurones (Kapfer et al. 2002). These glycinergic inputs are more than an adjunct to the binaural cross-correlation process that permits ITD sensitivity in the auditory nervous system in the first place. Their critical role in ITD processing was demonstrated most vividly by iontophoresis of the glycinergic antagonist strychnine through one barrel of a multibarrel pipette electrode, whilst simultaneously recording the responses of single MSO neurones to pure tones with ITDs imposed on them (Brand et al. 2002). During blockade of the glycinergic input, two related effects were observed. First there was an increase in the discharge rate compared with control conditions, but in an ITD-specific manner. Second, the ITD that evoked the maximum response shifted from outside to inside the physiological range of the gerbil, to peak at, or very close to, zero ITD. The implication from this experiment is that glycinergic inhibition determines a neurone's tuning for ITD; without it the axonal conduction delay is effectively zero. A computer simulation using an established MSO model with the value of the internal delay set to zero confirmed this view; in the model, adding a fast inhibitory conductance shifts the peak ITD to longer and longer values as the magnitude of the inhibitory conductance is increased. Note, though, that although the glycinergic inhibition is phase-locked, it need not be ITD sensitive. Rather, its specificity is a result of its timing relative to the excitatory inputs. How this specificity is brought about is currently the subject of considerable investigation.

Reading the code – converting ITD to spatial position

The classic Jeffress model suggests a labelled-line code, where a particular neurone firing maximally indicates the azimuthal position of the source. Physiological data from mammals, however, appear unequivocal in demonstrating that such a mechanism is unlikely to exist in mammals. How then might the position of a sound source be encoded? One possibility, first suggested by the Nobel Prize winner Georg von Bekesey (see van Bergeijk, 1962), but never fully explored, is that the spatial position of a sound source is represented in the form of a rate code mediated by broadly tuned spatial channels. If the broad neural representation of ITDs in each side of the brain acts as a hemispheric channel, a change in activity brought about by a change in azimuthal position in one channel is accompanied by a change in activity, of opposite sign, in the other hemispheric channel. Thus, as activity in one channel increases, activity in the other decreases, and vice versa. Thus a relative difference between the two binaural channels could correctly be interpreted as a change in spatial position of a sound source.

Importantly, this model does not compromise the hypothesis that mammalian ITD sensitivity incorporates a process of binaural cross-correlation. All evidence from the responses of single ITD-sensitive neurones suggests that it is. Nevertheless, the existence of the binaural cross-correlation process does not necessitate the existence of an array of delay-lines representing all azimuths. Binaural cross-correlation exists within the broader confines of models of coincidence detection, and can be considered without reference to the more specific model in which the neural representation of auditory spatial co-ordinates is systematically arranged within a topological map.

Optimal coding of ITDs

Consistent with the Jeffress model, coding sound-source position in the barn owl resembles most closely a form of local coding; different spatial positions of a sound source are represented by peak activity of individual neurones. In mammals, however, it follows that the relationship between spatial position and peak ITD tuning cannot be a form of local coding or, indeed, any form of direct representation since, for many small mammals, the majority of neurones are ‘tuned’ to ITD values that can never be experienced. They do not represent a unique spatial position by virtue of their ITD tuning. To this end, recent studies have generated considerable debate as to the nature of any neural code for ITDs, as the data they report appear to contradict the basic tenets of the Jeffress model. At their core, these studies question whether a single unifying model can ever account for ITD sensitivity in the wide range of species in which such sensitivity is observed. A recent model, however, has attempted to reconcile the very different data sets whilst satisfying a basic principle of neural computation called efficiency of coding. An assumption of efficient coding is that the brain maximizes the accuracy with which it performs a particular task, in this case localizing the position of a sound source in space. In order to maximize information, the brain will use whichever strategy best serves this purpose. Clearly, how the brain achieves this may not actually conform to the Jeffress model, which was developed before any anatomical or electrophysiological studies of binaural hearing had been carried out.

The results of the model (see Harper & McAlpine, 2004, and Fig. 5B) suggest that the very different sound frequency ranges over which mammals and barn owls make use of ITDs in sound localization tasks, as well as the different head sizes (and thus the range of ITDs that is experienced) of different species, are major factors in determining the form of any efficient code the brain might adopt in solving a sound localization task. The critical feature of these data is that positioning the sensitive slopes of ITD functions within the physiological range of ITDs maximizes ITD coding accuracy over this range (here measured as Fisher information – the dotted curves in Fig. 5B). In order to achieve this, the peaks of the broadly tuned ITD functions in the gerbil (Fig. 5B, top) are positioned outside the physiological range, whilst the peaks of the sharply tuned ITD functions in the barn owl (Fig. 5B, bottom) are positioned within the physiological range. Thus, purely physical factors appear to have a major impact on how neurones go about constructing a perception of sound space. By making use of efficient coding, the model is able to predict some of the more puzzling features long observed in recordings from the brains of small mammals, e.g. that many neurones respond maximally to ITDs beyond the range that their head size can create under natural listening conditions. In addition, it accounts for the established data recorded over several decades in the barn owl brain, as well as data from more recent recordings from the barn owl that also appear inconsistent with the Jeffress model (Bala et al. 2003).

Figure 5. Modelling binaural hearing.

A, hemispheric channels model of spatial hearing. See text for further details. B: top, ideal distributions of tuning curves for coding ITDs (here plotted as interaural phase differences, IPDs) for 500 Hz tones in gerbil for left (red) and right (blue) subpopulations. Dotted lines show a measure of coding efficiency – Fisher information – which is maximal over the physiological range (shaded grey area); bottom, ideal distribution for coding IPDs of 5000 Hz tones in barn owl (sample only shown). Red lines show tuning curve (continuous line) and scaled Fisher information (dotted lines) for one neurone (modified from Fig. 2 in Harper & McAlpine, 2004).

In summary, the new model provides a unifying principle for understanding how the brain encodes the location of a sound source using ITDs. It also accounts for a wide range of data sets, and is consistent with requirements in the field of computational neuroscience, which were developed after the Jeffress model was first published over half a century ago.