Relearning Auditory Spectral Cues for Locations Inside and Outside the Visual Field

Abstract

Previous research has demonstrated that, over a period of weeks, the auditory system accommodates to changes in the monaural spectral cues for sound locations within the frontal region of space. We were interested to determine if similar accommodation could occur for locations in the posterior regions of space, i.e. in the absence of contemporaneous visual information that indicates any mismatch between the perceived and actual location of a sound source. To distort the normal spectral cues to sound location, eight listeners wore small moulds in each ear. HRTF recordings confirmed that while the moulds substantially altered the monaural spectral cues, sufficient residual cues were retained to provide a basis for relearning. Compared to control measures, sound localization performance initially decreased significantly, with a sevenfold increase in front–back confusions and elevation errors more than doubled. Subjects wore the moulds continuously for a period of up to 60 days (median 38 days), over which time performance improved but remained significantly poorer than control levels. Sound localization performance for frontal locations (audio-visual field) was compared with that for posterior space (audio-only field), and there was no significant difference between regions in either the extent or rate of accommodation. This suggests a common mechanism for both regions of space that does not rely on contemporaneous visual information as a teacher signal for recalibration of the auditory system to modified spectral cues.

INTRODUCTION

Our ability to localize sound sources in the space around us relies largely on subtle acoustic cues at each ear that are in turn dependent on the precise shape and location of the ears on the head (e.g. Shaw 1974). The filter properties of the outer ear vary as a function of the location of the source relative to the ear and are referred to as the head-related transfer functions (HRTFs) (e.g. Wightman and Kistler 1989). These spectral variations are perceptually significant (e.g. Carlile and Pralong 1994) and provide important monaural cues for the accurate localization of sound sources (for recent review, see Carlile et al. 2005). As we develop and age, the morphology of the head and ears changes (e.g. Clifton et al. 1988; Otte et al. 2013), which then requires the auditory system to recalibrate over time by mapping the evolving spectral profiles to specific locations in space. Such a recalibration in adult humans was first demonstrated experimentally more than a decade ago by producing subtle (reversible) distortions of the spectral cues using pinna moulds and examining the rate and extent of accommodation through periodic testing of auditory localization performance (Hofman et al. 1998).

Van Opstal and colleagues manipulated the spectral cues of one ear (Van Wanrooij and Van Opstal 2005) or both ears (Hofman et al. 1998) for periods of 19 to 39 days. Localization performance was tested for locations within the audio-visual region of space (±30 °: Hofman et al. (1998) and ±75 °: Van Wanrooij and Van Opstal (2005) in azimuth and elevation) using eye or nose pointing, respectively. Bilateral moulds produced an initial disruption of vertical, but not horizontal, localization acuity—a finding which is consistent with the view that these moulds substantially disrupted monaural spectral cues but had little effect on binaural cues. Following accommodation, subjects approached control performance in vertical localization and there appeared to be no after-effect once the moulds were removed (Hofman et al. 1998). Inserting the moulds in one ear only reduced vertical localization performance ipsilateral to the mould with a decreasing effect as the sound source location moved into the contralateral (unmodified) space. Over time (7 to 49 days), vertical localization improved in those subjects whose spectral cues had been substantially changed by the moulds, where as individuals whose moulds caused relatively small acoustic changes exhibited less adaptation and smaller initial localization bias.

More recent work using unilateral earplugs to perturb binaural rather than monaural (spectral) cues has demonstrated the capacity for moderate levels of accommodation in mature ferrets (Kacelnik et al. 2006) and humans (Kumpik et al. 2010). While this work has also suggested that training over a period of days can accelerate accommodation to changes in auditory localization cues, this does not seem to involve a recalibration of the binaural cues (Kumpik et al. 2010).

The notion of accommodation to changes in auditory input is not new and considerable behavioural and neurophysiological work has examined these processes (for reviews, see Wright and Zhang 2006; King 2009). Much of this previous work has focussed on developmental adaptive changes and on the alignment of auditory and visual representations at the neuronal level (e.g. ferret: King et al. 1988; King and Carlile 1993; barn owl: Knudsen and Brainard 1991). In general, there is a significant amount of evidence for interactions between the visual and auditory modalities that could point to visual inputs as a functional basis for auditory recalibration (see King 2009 for review). The ventriloquist effect (e.g. Alais and Burr 2004) and so-called reverse-ventriloquism (Recanzone 1998 recent review: Recanzone 2009) are well-known cross-modal interactions to contemporaneous audio-visual stimuli. Wearing distorting lenses for 2–3 days also produces compensatory changes in the auditory localization behaviour to maintain audio-visual alignment (Zwiers et al. 2003). Furthermore, there is a growing body of neurophysiological evidence of the modulation of auditory cortical activity by visual input (Schroeder and Foxe 2005; Bizley and King 2008, 2009).

It has been largely assumed that available visual information can be used in the process of recalibrating auditory spectral cues to location. Notably, in both previous studies of accommodation to altered spectral cues, the spatial region over which auditory localization was tested corresponded to the frontal visual field so, presumably, in the course of their normal daily activities, subjects were continually being apprised of any potential spatial mismatch between the spectral cues and the visual location of sounding objects.

While the cross-modal data are compelling, significant questions remain unanswered. What capacity does the auditory system have for remapping new spectral cues that point to regions outside the visual field. In regions where it is impossible for visual input to provide a direct ‘teacher signal’ for recalibration, what is the driver for this process. In the present study, we extend our investigations beyond the audio-visual confines of previous studies to examining the auditory systems ability to learn new spectral cues for regions from all around the body. Our aims in doing so were threefold: (1) to determine if the auditory system was even able to recalibrate to spectral cues for regions where no contemporaneous audio-visual stimulation is possible, and if it was, (2) to ascertain whether there were any differences in the extent of recalibration or (3) in the time course of that recalibration for the AV region of space compare to the auditory-only region. On the one hand, any differences in accommodation between these regions could provide clues as to the respective processes involved. On the other hand, a lack of difference would have significant implications for the sort of teacher signal that the auditory system utilises to drive this recalibration.

METHODS

Eight subjects participated in this study (age, 20 to 47 years; three females) drawn from the postgraduate and research members of the laboratory. All subjects had normal hearing determined using pure tone audiometry and two had previous experience in auditory localization psychophysics. Subjects were first trained in the auditory localization procedure, and control performance was measured and verified as falling within the normal range (see below). Small moulds were custom made for each ear, and the head-related transfer functions (HRTF) were recorded both before and after the bilateral fitting of the moulds. As a control measure, auditory localization performance was tested prior to fitting the moulds and then at least twice a week over the course of adaptation to the chronically worn moulds (average 40.5 days; range 28–62 days). Following the adaptation period, localization performance was also tested immediately after removal of the moulds to determine whether the inserts had any impact on the subjects' ability to revert to the use of their normal spectral cues. Additionally, after a ‘mould-free’ period of 7 days or more localisation acuity while wearing the moulds was again tested to investigate how long the newly learned spectral cues could be retained, even in the absence of chronic exposure to them. All procedures were approved by the Human Research Ethics Committee (HREC) of the University of Sydney.

Design of the moulds

Due to the curing time of the mould material, it was not practical to manufacture the inserts in situ. Rather, this was done using casts of the outer ears made for each subject from dental grade polyvinylsiloxane (Extrude Wash type 3 by SDS Kerr). This compound was used to create positive and then negative models of the outer ear. The negative mould was then a life-like replica of the individual's outer ear and was used to create the occlusive moulds by carefully pouring prosthetic grade silicone (Barnes Prosil 20) into the negative cast to the depth required to produce a custom fit mould (Fig. 1). The moulds were further trimmed and shaped using a scalpel so that they partially filled the base of the concha and completely filled the cymba concha with care being taken to ensure that the external auditory meatus remained unobstructed by the mould. This procedure not only produced substantial changes in the volume of the outer ear cavities that gave rise to significantly altered HRTFs (see below) but also produced a snug fit that ensured that the mould remained firmly in place during the course of daily activities over the accommodation period.

FIG. 1.

An example of the finished mould in a subject's left ear.

The design objectives of the mould were to produce a substantial change in the spectral transfer functions of the outer ear while ensuring that the residual transfer functions were sufficiently location dependent as to provide adequate replacement spectral cues for localization. There was also an attempt to match the changes in each ear to minimize abnormal binaural differences. These objectives were sometimes achieved iteratively by measuring the spectral transfer functions with the moulds in place, modifying the mould slightly following analysis of the recordings and then measuring the functions again until the design goals were met.

Measurement and analysis of HRTFs

The methods for measuring the HRTF have been described in detail elsewhere (Jin et al. 2004) but are briefly summarized here. We utilized the so-called blocked ear canal recording technique which involved embedding a small recording microphone (AuSIM miniature microphone) in an earplug located just interior to the distal end of the ear canal. The output of the microphone was delivered to two preamplifiers (Sound Devices MP-1) which also provided the phantom power for the microphones. The signal was routed to an analogue to digital converter (TDT System II) and recorded to hard disk.

The HRTF recordings were performed inside an anechoic chamber with the subject placed at the centre of the chamber. A loudspeaker, mounted on a robot arm, delivered the stimuli from positions locations around the head of the subject on an imaginary sphere of radius 1 m. The automated procedure resulted in 393 HRTFs for the right and left ear recorded for locations evenly distributed on the sphere, from −45 ° to 90 ° in elevation. Head stability was ensured throughout by monitoring head position using an inertial tracking device (Intersense: Inertia Cube 3). Impulse responses were measured using Golay code pairs 1,024 bits in length sampled at 80 kHz. The directional transfer function (DTF) used in the analysis to follow was calculated from the measured HRTF by removing the location-independent component (according to Middlebrooks 1992).

Of particular interest to this study was how the DTF varied around the so-called cone of confusion. Due to the symmetry of the position of the ears on either side of the head, a given binaural interval will only specify the surface of a cone centred on the interaural axis rather than a unique location in space. The spectral cues, however, vary systematically around this cone of confusion; as a consequence, when a sound is sufficiently broadband, integration of the binaural and monaural spectral cues uniquely specifies the location of the sound source. To examine how the spectral cues varied about the cones of confusion, the DTFs were visualized by plotting all the polar angles lying on a cone corresponding to particular lateral angle (Fig. 2A). The spectrum amplitudes at intervening locations were calculated using a combination of principal components analysis and spherical thin plate spline described in detail elsewhere (Carlile et al. 2000 and see also Wahba 1981; Kistler and Wightman 1992; Middlebrooks and Green 1992). To gain insight into the relevant features likely to be encoded by the auditory system, the pattern of acoustical changes was also passed through a cochlear filter model prior to interpolation (Fig. 2B; Glasberg and Moore 1990; Carlile and Pralong 1994).

FIG. 2.

A Filter functions of the left ear of one subject are plotted for the midline cone of confusion before and B after passing through a cochlear filter model. Filter functions for the left ear of a different subject are plotted for the midline (D,E,F Azimuth 0, cf. red line in (C)) and 40 ° off the midline (G,H,I Azimuth −40, ; cf. blue line in (C)) are plotted without moulds (D and G) and with moulds (E and H). The data have been smoothed, as above, using the cochlear filter model. The differences between the bare and mould conditions for both lateral angles are plotted in (F) and (I).

To better quantify the extent of the modification produced by the pinna mould, the similarity index (SI) was calculated from the DTFs derived from the HRTFs measured with and without the mould. We applied the method of van Opstal et al. (see Van Wanrooij and Van Opstal 2005) so as to enable a direct comparison with the SI calculated for the data in the two earlier studies. In summary, those authors calculated the SI from the DFTs obtained from 25 locations at different elevations located on the anterior midline (−50 ° to +50 ° re audio-visual horizon) by computing the correlation matrix for each individual DTF recorded from the bare ear compared to every DTF recorded from the mould ear. This resulted in a 25 × 25 matrix and the SD of the each row was computed and the average standard deviation for all rows was taken as the SI. In this study, the same calculation was applied to the 16 locations recorded on the frontal midline (−45 to +60 re the audio-visual horizon) after cochlear filtering and band limiting between 4 and 16 kHz.

Localization performance

Our procedures for assessing localization accuracy have been described in detail elsewhere (Carlile et al. 1997) but are summarized below. Prior to the experiment, subjects underwent a training regime to acclimatize them to the test environment, acquaint them with the test procedure and practice using the nose-pointing response method. This final point was particularly emphasized in order to overcome the natural tendency to ‘capture’ targets located at extreme elevations with the eyes, rather than properly extending the neck so as to precisely orient the nose toward the sound source. During training, a subject stood in the middle of the anechoic chamber wearing a cap fitted with an electromagnetic-tracking device (Polhemus: IsoTrak) that measured the orientation of the head. To initiate a trial, the subject aligned their head with the stimulus coordinate system with the aid on an LED compass driven by the head tracker and pressed a response button to indicate readiness. A 150-ms burst of broadband noise (300 Hz–16 kHz with a 10-ms cosine ramp) was emitted at one of 76 positions located 1 m from the head of the listener by a loudspeaker mounted on a robotic arm. In response to this stimulus, the subject turned to face the perceived location of the sound source, pointed her nose toward it, and pressed a handheld button to register this response position. Feedback as to the accuracy of the nose pointing was delivered in two stages. (1) After the initial localization attempt, an LED mounted on the loudspeaker was illuminated. As this provided participants with the exact location of the sound source, it allowed them to correct any gross errors (e.g. front–back confusions) which arose from misperception. After registering this corrected response, the LED went off and was followed by (2) auditory only feedback. In this instance, a noise burst was repeated from the speaker at a rate inversely proportional to the pointing error (that is, the smaller the localization error, the higher the repetition rate of the noise burst). Subjects were encouraged to achieve the maximum repetition rate, thereby gaining experience in fine tuning their pointing technique. Subjects typically completed between 150 and 200 training trials in the dark, with the experimenter coaching them through the procedure during the first 10 to 20 trials with the anechoic chamber lights on. The experimental phase used the same stimulus parameters, presentation and response methodologies as did the training, but no feedback was provided in any of the tests.

Localization accuracy was first quantified using the percentage of front–back confusions. Responses were classified as front–back confusions when the actual and perceived locations were in opposite hemisphere but not within plus or minus 25 ° of the interaural axis. These errors were then removed from the data to allow a more fine grained analysis of performance using mean unsigned lateral and polar angle errors. Polar angle errors were corrected using the cosine of the lateral angle to account for variations in the circumference of the circle of the sphere described by each cone of confusion. A global measure of localization performance was also obtained, using the spherical correlation coefficient (SCC) between the actual and perceived locations. The analysis methods are described in detail in Leong and Carlile (1998).

Localization performance without the moulds served as the control measures for the experiment (denoted by ‘C’ in the Figures), and comprised a minimum of two test blocks of 76 locations. The acute impact of the spectral cue distortion was measured by retesting localization acuity immediately after mould insertion (accommodation day 0: ‘A0’). Once inserted, the moulds were worn during all waking hours over the course of the accommodation period (average 40.5 days, range 28–62 days). In order to gain some insight into the pattern and rate of accommodation to the new spectral cues, subjects then completed two blocks of trials at least twice a week until performance gains plateaued, at which point the accommodation period was deemed complete. Just prior to the removal of the moulds (last day of accommodation: ‘An’), localization performance was again tested to characterise the extent to which the auditory system had accommodated to the new spectral cues. Additional testing following removal of the moulds (post-accommodation: ‘Post’) indicated whether accommodation to the new spectral cues had in any way disrupted a subject's ability to access and use their ‘native’ spectral cues (i.e. to check for any accommodation after-effect). Finally, approximately a week after the end of the accommodation period, the moulds were reinserted and a further two blocks of localization testing was performed. The aim of this final test was to measure the extent to which accommodation to the new spectral cues had extinguished (extinction: ‘Ext’) in the absence of chronic exposure to the mould.

To test for differences in treatments each performance measure was subjected to a one-way non-parametric Kruskal–Wallis test (Matlab V7.1, The MathWorks). Where overall significant differences were demonstrated, differences between conditions were examined post hoc using Tukey's honestly significant difference criteria (multcompare, Matlab) and/or a paired t test both with Bonferroni corrections.

RESULTS

Acoustic effects of the moulds

A detailed analysis of the acoustic and acute localization performance effects of pinna moulds is presented elsewhere (Carlile et al. in prep) but a brief description is provided below. The changes to the acoustics of the outer ear produced by the moulds were examined both qualitatively using the lateral angle plots of the midline DTFs and 40 ° off the midline and quantitatively by calculating the SI as outlined above. Figure 2A shows a representative plot of the DTFs on the midline cone of confusion for the left ear of one subject. In Figure 2B, the same data has been smoothed using a cochlear filter model.

A number of typical location-dependent features are evident in both plots of the DTFs recorded from the bare ear. For instance, the elevation location on the frontal midline (front) can be determined from the centre frequency of a deep spectral notch progressing from around 5–6 kHz for sounds below the horizon to around 14 kHz for locations directly above the head (top). This overhead location is also associated with a gain feature centred on 8 kHz and a notch at 16 kHz that moves progressively down in frequency to around 9 kHz as the source moves from above to behind the head (Fig. 2A: ‘Top’ to ‘Back’). The cochlear excitation plots indicate that these features are sufficiently robust that they are likely to be passed though the cochlear and encoded in the pattern of neural activity (Fig. 2B).

Figure 2D,E,F,G,H,I shows the cochlear filtered data for another subject at two lateral angles and clearly illustrate the impact of the moulds. Figure 2D depicts the bare ear DTFs for the left ear at an azimuth of 0 °; Figure 2E displays the DTFs for the same lateral angle with the mould inserted. Figure 2F plots the difference between the two (the same pattern is followed for Fig. 2G,H,I for a lateral angle of −40 °). The deep notch extending from 5 to 12 kHz in the frontal midline for this subject (Bare ear Az 0, demarcated by the red line in in Fig. 2C) has been significantly attenuated by the mould and shifted down in frequency resulting mainly from the appearance of a large gain feature centred on 8 kHz (at the front). The peak frequency of this gain feature in the moulds DTF appears to move up in frequency as elevation is increased from −45 ° below the midline to around 60° above the horizon. In addition to the weak notch at lower frequencies this variation in the frequency peak could provide a spectral cue to elevation. The mould also significantly changed the cues for the posterior midline locations virtually eliminating gain features at 8 and 16 kHz which in the absence of the mould could also aid in front–back disambiguation. In the plots of the transfer functions for the lateral angle 40 ° left of the midline (indicated by the blue line in Fig. 2C), it is evident that the mould produced much the same patterns of change. However, as this cone of confusion includes the acoustic axis of the pinna, the overall gain—particularly for the frontal hemisphere—is greater.

While the specific details vary from subject to subject, all DTFs from bare ears demonstrated to some degree the spectral notch cue for variation in elevation in the frontal midline. There was more variation across subjects in the elevation cues for the posterior midline where, in some cases, notches and in others spectral peak frequencies demonstrated elevation-dependent changes. In all cases, the moulds used were sufficiently disruptive that the frontal midline notch was significantly reduced and/or shifted to lower frequencies and in most cases, a gain feature at 8 kHz was evident for locations around the frontal horizon. In many cases, moulds also reduced gain features on the posterior midline and/or shifted the frequencies of idiosyncratic notches for this region of space. Very similar spectral changes were reported in the two previous studies using moulds (Hofman et al. 1998; their Fig. 1 and Van Wanrooij and Van Opstal 2005 their Fig. 5), particularly in terms of the disruption of the mid to high frequency notch for the frontal midline. The effects of the mould inserts were also assessed quantitatively by calculating the similarity index (SI) between the DTF obtained for the bare ears and those obtained with moulds (see ‘METHODS’ section and Table 1). The SI averaged across both ears ranged from 0.21 to 0.39 with a mean of 0.29 (SD 0.05) across the eight subjects.

FIG. 5.

A The changes in localization performance (SCC, FB confusions, PAE) over the course of accommodation are shown for one subject. Solid line: linear fit for the first 30 days accommodation. Large filled circle: control (bare ear) performance. B The changes shown in (A) are plotted at each test block as a percentage (%) of the difference between the mean performance immediately on insertion of the moulds (large filled circle in (A)) compared to the control performance. C Box plots of the slopes of the linear fits across all subjects for each of the performance parameters (see text for details).

TABLE 1.

Acoustic similarity measures for mould V's free ears and days accommodation

Subject	SI–RE	SI–LE	Accommodation
1	0.28	0.26	44
2	0.22	0.35	57
3	0.35	0.30	38
4	0.19	0.31	38
5	0.19	0.23	39
6	0.28	0.40	32
7	0.31	0.26	35
8	0.19	0.31	38

Localization performance

Auditory localization performance was assessed (1) before the moulds were inserted (C); (2) immediately after the moulds were inserted (A0) and then every few days following insertion of the moulds; at the end of the accommodation period both (3) before (An) and (4) after (Post) removal of the moulds; as well as without the mould (5) a week after the end of the accommodation with moulds to test for any extinction to the new cues (Ext). The group mean data are shown in Figure 3 along with the standard errors of the mean. These data show number of interesting patterns.

FIG. 3.

Group mean data for eight subjects where localization performance was assessed using the spherical correlation coefficient (A), the percentage of front–back confusions (B), the polar angle error (C) and the lateral angle error (D). C control pre-accommodation, A0 performance with acute exposure to the mould, An performance with the moulds on the last day of accommodation, Post performance immediately post-accommodation without the moulds, Ext performance a week or more post-accommodation with the moulds. Significant differences (P < 0.05) between conditions are indicated by an asterisk ‘*’.

The control data in Figure 3 are consistent with previous studies in our lab (for example, see Carlile et al. 1997) and were characterised by a SCC around 0.9 and a low percentage of front–back confusions (~2 %). The small standard errors in the control condition reflect the relatively uniform performance across our group of subjects. The acute impact of the moulds on localization performance is evident when comparing C and A0 for each of the measures. In particular, the percentage of front–back confusions increased more than sevenfold and polar angle error more than doubled in magnitude—effects that are largely indicative of perturbation of the spectral cues to the sound's location. By contrast, the lack of significant change in the lateral angle error indicates that the moulds had little if any effect on the binaural cues to location.

With the exception of one subject, all subjects wore their moulds chronically for more than 30 days, and localization performance was tested every 2 to 3 days. The extension of the accommodation period beyond 30 days was determined principally by the willingness of the subject to on-going testing (see Table 1). Localization was tested again at the end of the accommodation period (An) and while values had moved towards the control values, the group means and SEM suggested that, in spite of such a lengthy accommodation period, the SCC and polar angle error while wearing the moulds remained significantly different from control.

Immediately after the moulds were removed, localization was again tested (Post) and performance was found to overlap the control performance (Fig. 3: C compared to Post for all panels). This indicates that, despite accommodating to the new spectral cues produced by the pinna moulds, the auditory system retained its ability to use the subject's original spectral cues. Furthermore, there was no correlation between localization performance in the post-accommodation condition and the length of the accommodation period, i.e. this capacity was evenly retained by subjects despite the fact that the longest accommodation period, which constituted more than 2 months of chronic mould wear was double that of the shortest.

To examine whether the new cues would extinguish following cessation of chronic wear, all but one subject returned after about a week to again test localization performance. In this case, the moulds were inserted into the ears immediately before the test began to ensure that the subject gained as little exposure to environmental sounds with the moulds in place as possible. With the exception of lateral angle error, localization performance at this stage (Ext) was virtually identical (n.s.d. on any parameter) to the performance demonstrated at the end of the accommodation period (An). This finding indicates that, in spite of the chronic re-exposure to the original spectral cues during the intervening week, the accommodated spectral cues were still available to provide information as to the stimulus sources location.

The effects of treatments were examined using a one-way ANOVA and were found to be significant for spherical correlation coefficient (SCC: F = 40.8; P < <0.001), front–back confusions (%FB: F = 17.8; P < < 0.001) and polar angle error (PAE; F = 25.6; P < <0.001) but not for lateral angle error (LAE; F = 2.21, P = 0.09). Pair wise comparisons confirmed the differences highlighted in the text above and are indicated on Figure 3.

Localization performance in audio-visual and audio-only regions

An important question in this study was whether the localization performance differed between regions of space that were also immediately accessible to the visual system (i.e. in the audio-visual region ) and those areas for which there was no contemporaneous visual input available (audio only). In this analysis, we recalculated the performance measures described above separately for each region and compared the performance for AV sound locations (defined as the region ±70 ° centred on the frontal midline and audio-visual horizon) with performance for the remaining locations situated outside this visual field (Fig. 4).

FIG. 4.

Localization performance plotted separately for audio-visual (filled bars) and audio-only (open bars) regions of space. All other details as per Figure 3.

In the control, condition performance on all of the parameters was marginally better for locations within the AV regions of space (filled bars) compared to the A-only region. This is consistent with other localization data which indicate a slightly better performance for locations in the frontal hemisphere compared to the posterior hemisphere (see Carlile et al. 1997) which no doubt reflects both errors associated with performance on the task as well as perceptual resolution. Most importantly, however, the general patterns of change as a function of the condition are seen in both the audio-visual region and the audio-only data. One notable difference is in the percentage of front–back confusions, i.e. when the moulds were first inserted, the rate of back-to-front confusions was far greater than the front-to-back confusions (Fig. 4: top right, ‘A0’) although the size of the error bars points to substantial variability between subjects. However, by the end of the accommodation period (An), this discrepancy had so reduced that with the relative differences between AV and A-only matched the pattern seen in the control condition.

The similarity in the levels of accommodation to the new spectral cues is of particular interest as it suggests that the accommodation process is not dependent on direct visual input as a conditioning or training signal in the mapping of the new acoustic cues to particular regions of space.

Patterns of accommodation

While the extent of accommodation appears to be similar for both AV and A-only regions of space, we were also interested in determining whether there were any differences in the patterns of accommodation in each over time, as such a distinction might indicate regional differences in the mechanism(s) driving accommodation. Previous work has indicated that accommodation does not necessarily proceed in a linear fashion. When a mould was inserted in one ear, Van Wanrooij and van Opstal (2005) reported a sinusoidal pattern in the changes of perceived elevation for the AV region with a period of between 6 and 10 days. While we observed some evidence of non-linear patterns in some subjects, no clear sinusoidal pattern was evident in any of the performance measures we used here. This may have been due to the relatively long intervals between our test sessions (2–4 days) which could have resulted in a sampling period that was relatively low compared to the oscillatory behaviour reported previously.

Nonetheless, these data do indicate an overall and progressive improvement in performance over time for all of the subjects in our cohort. As an example, Figure 5A shows the data from the subject who wore the moulds for the longest period. The large filled circle at Days Accommodation 0 indicates performance in the control condition, i.e. without the mould. Of note, although this subject was unavailable for testing for 9 days in the middle of the accommodation period, this does not seem to have had any impact on the rate of accommodation. This suggests that, in the absence of any performance feedback, the process of accommodation is not driven by repeated exposure to the testing itself (c.f. Kacelnik et al. 2006). The vertical dotted line indicates the 30th day of accommodation; the solid line is the linear regression line fitted only to the data for the first 30 days of accommodation. While this is a somewhat arbitrary period over which to fit the linear regression, our data provide two main reasons to do so. Firstly, as illustrated by the data in Figure 5A, the accommodative changes for the SCC and PAE in particular appear to saturate so that changes in performance after 30 days fall progressively further below or above the line, respectively. Secondly, although the majority of subjects accommodated beyond 30 days, this is the period for which we have accommodation data for nearly all our subjects (see Table 1). The data for the %front–back confusions (Fig. 5A, middle panel) is somewhat noisier but the general trend is that the data is increasingly above the line after 30 days. This is representative of the patterns seen in other subjects.

As noted earlier, the acute effects of the moulds varied to some extent across our cohort, so to facilitate comparison across subject, we have recalculated the slopes calculated for each individual as the change in localization performance produced by the mould. Figure 5B shows the extent of accommodation achieved by the same subject (Fig. 5A) over the first 30 days—that is, only the days' data which contribute to the linear fit—expressed as a percentage of the difference between the control performance and acute impact of the mould (A0). Note that these plots show the negative performance effects of the moulds progressively declines over the course of the accommodation period. Having transformed the mould effects on performance in this way, the slope of the percent change over the accommodation period can be calculated for all subjects and the means and distributions plotted (Fig. 5C). The box plot in Figure 5C defines the 25th and 75th percentiles, and the bar within the box indicates the median of each group of scores. All the slopes are statistically different from zero (Kruskal–Wallis: Chi-Sq = 23.09; df = 3; P < 0.001) indicating that our population of subjects demonstrated a progressive improvement in these performance parameters over time. This is consistent with the mean data for ‘A0’ and ‘An’ conditions shown in Figures 3 and 4. A comparison of the mean ranks of the slopes for front–back confusions and polar angle error indicates that these are significantly different (P < 0.05; see Hochberg and Tamhane 1987) although neither is significantly different from the rate of change of the SCC. These differences suggest that the processes underlying front–back confusion resolution and local polar angle error may accommodate to the changed spectral cues at different rates. The similarity in the medians for the SCC and PAE is probably not that surprising given the relative dependence of the SCC on PAE.

Of importance to the present study were the rates of accommodation for the AV and A-only regions of space. Whilst the data in Figure 4 demonstrates a similar extent of accommodation between both regions, it may not necessarily reflect a common underlying process but rather may simply indicate a ceiling produced by other limits such as the quality of available cues or the resolution of processing. To examine if there was evidence of differences in the rates of accommodation, we calculated the slopes of the accommodative changes in localization performance for AV and A-only regions, meaned for all subjects, and compared those to the overall rates of change (Fig. 6A).

FIG. 6.

A The pooled slopes of the linear fits to the % change in SCC (left), %FB (centre) and Polar Angle Errors are shown for all locations (All), locations in the audio-visual region of space (AV) and audio-only regions of space (A-only). B The FB confusion data for two subjects are shown for audio-visual (red) and audio-only (green) regions of space. The pooled data for the subjects demonstrating large increases in FB confusions on acute exposure to the mould (left panel, red) are plotted for all locations and for the AV and A-only locations.

There were no significant differences in the rates of accommodation for the SCC and PAE for the AV v's A-only regions (Fig. 6A left and right panels, respectively). The rate of accommodation shown for FB confusions was statistically different between the regions (Chi-sq = 10.19, df = 2, P = 0.006; Fig. 6A centre panel) but close inspection of the individual data indicated that two different patterns of response were evident. Half of the subjects showed very low initial rates of front–back confusions, despite high rates of back-to-front confusion. As a consequence, for these subjects, accommodative change was coming from a very low baseline and was reflected in the relatively flat regression slopes (Fig. 6B, middle panel, red line). This is evident in the relatively wide range of the 25th and 75th percentiles for the %FB in the AV region data (Fig. 6A, middle pane) and presumably indicates that the moulds for these subjects failed to disrupt the spectral cues normally available to provide this discrimination. The other half of the subjects demonstrated relatively high levels of front-to-back confusions with the moulds and showed a sharply negative slope in the rate of accommodation (Fig. 6B, left panel, red line). When the analysis was restricted to only those subjects who did show a large initial increase in front-to-back confusion rates there was no difference in the rate of accommodation for the AV or A-only regions of space (Fig. 6B, right panel; Chi-sq = 0.5, df = 2, P = 0.78).

DISCUSSION

Recalibration to modified spectral cues

These results show clearly that the recalibration of the auditory system to altered spectral cues is not dependent on contemporaneous input from the visual field. That is not to say that visual input does not have a role to play, but rather at the time a sound occurs at locations outside the visual field, there is no contemporaneous visual information to indicate any potential mismatch between the actual location and the location indicated by a specific spectral cue.

The potent influence of the visual system on auditory localization has been known for a very long time (see Pick et al. 1969) and has been exploited by performers and others who have practiced ventriloquism. Somewhat more recently, reverse ventriloquism has been described (e.g. Alais and Burr 2004) and suggests an optimal integration of auditory and visual cues that relate to their fidelity or reliability. Importantly, such influences have also been demonstrated to persist—the so-called ventriloquist after-effect (see Radeau and Bertelson 1974; Recanzone 1998)—with recent work showing this effect even after very brief audio-visual interactions (Wozny and Shams 2011). Wearing distorting lenses for several days has also been shown to produce compensatory changes in auditory localization (Zwiers et al. 2003). While much neurophysiological research has been focussed on the multi-modal integration of stimulus location cues in the superior colliculus (for review, see King 2004) a growing number of studies have shown visual and somatosensory convergence in auditory cortex and an increasing understanding of the functional interactions of these inputs (King and Walker 2012). It has been suggested that this convergence of visual and auditory information serves to increase the precision of auditory localization given the greater spatial precision of visual (and somatosensory) systems (for review and discussion, see Schroeder and Foxe 2005). Consistent with this is the finding that visual inputs to auditory cortex increase the spatial information encoded by a population of single neurons in the anterior dorsal field (e.g. Bizley and King 2008).

Much previous work has also had a focus on the role of visual inputs in guiding the development of auditory spatial representations (e.g. King et al. 1988; Knudsen and Knudsen 1989; Knudsen and Knudsen 1990; Brainard and Knudsen 1998; Knudsen 2002). Largely because of the convenience of a topographical representation of auditory space, much of this work has exploited the superior colliculus of the mammal or the barn owl. Perturbing the visual input during development has been shown to produce compensatory changes in the auditory map of space so as to maintain topographic alignment between the two. More recent studies have shown that, under certain conditions, such effects can also be demonstrated in the mature animal (Brainard and Knudsen 1998; Zwiers et al. 2003; Bergan et al. 2005; Linkenhoker and Knudsen 2002; Woods and Recanzone 2004) indicating that this influence is not fundamentally restricted to some limited sensitive period of development.

On the one hand, given this wealth of evidence of audio-visual interaction in the central nervous system, the lack of any difference in the rate or extent of accommodation for the AV and A-only regions is surprising. On the other hand, Kacelnik et al. (2006) report that visual input was not required for ferrets to accommodate to altered binaural cues to location produced by a unilateral earplug; however, the processes activated in response to binaural perturbation may differ from those at play in accommodating to altered monaural (spectral) cues. In addition, there are of course many reports of the capacity for auditory localization in blind people and, leaving aside the question of normal or supranormal performance (e.g. Roder et al. 1999), the existence and maintenance of this performance in the blind demonstrates that visual input per se is not required. We return to this issue in the concluding section below.

Acoustic effects of the moulds

A key feature of the moulds used in this and related work is that they produce perceptually significant change to the spectral cues but retain sufficient location-dependent information which the auditory system can effectively learn to exploit. While there has been no systematic study of what constitutes ‘sufficient’ residual cues for adaptation, previous studies have shown that retaining some elevation dependent change in spectral features, such as the mid frequency spectral notch or high (8 to 16 kHz) spectral grains, results in accommodation over a period of weeks of exposure to the mould (Hofman et al. 1998). Similar mould effects can be seen in van Wanrooij and van Opstal (2005) where the bare ear on one side of the head was compared to the other ear wearing a mould. In the current study, DTFs measured with the moulds in place clearly indicate that elevation-dependent changes remain in the mid frequency notch and the high frequency gain as does a substantial mid frequency gain in the 3 to 6 kHz region (Fig. 2B). Similar effects are evident for locations off the midline (e.g. Fig. 2G) particularly for lateral angles associated with the pinna axis over the mid to high frequencies at lateral angles between 30 ° and 60 ° (e.g. Middlebrooks et al. 1989).

So as to compare the acoustic effects produced by our moulds with data obtained in their work, we have calculated the SI according to van Wanrooij and van Opstal (2005). In that study, the SI was calculated for six subjects using a selection of frontal midline locations and varied between 0.22 to just under 0.5. The magnitude of the SI was correlated with the acute effects of the mould on elevation localization performance (their Fig. 6) and on the subsequent rate and extent of accommodation to the unilateral mould. Significant effects were seen when the subjects SI between the bare ear and the mould ear was below 0.3. The SI calculated for our group of subjects ranged around 0.21 to 0.39 (mean = 0.29, SD = 0.05) with five of the eight subjects falling below an SI of 0.3. Taken together, the qualitative and quantitative analysis of the effects of the moulds indicate that the spectral cue modifications in this study fell within the range previously reported to have produced the acute effects on localization performance (as demonstrated in our Fig. 3) while still providing an adequate basis for subsequent accommodation to the new cues.

Acute effects of the moulds on localization performance

The control localization performance fell well within the range of normal performance we have reported previously (e.g. Carlile et al. 1997). In particular, front–back confusions were on the order of 2 % (Fig. 3) with very little difference in the proportion of back-to-front versus front-to-back confusions (Fig. 4). On insertion of the mould, the front–back confusion rates increased nearly sevenfold with a greater proportion of these being from back-to-front. This large increase in reversals is still slightly less than percentage back-to-front reversals reported by Oldfield and Parker (1984b) who also found a greater proportion of front-to-back confusions. This may be a consequence of a more complete filling of the pinna in the Oldfield and Parker study than in the current study which would have resulted in a more complete elimination of the spectral cues. Such a result is also consistent with neurophysiological studies that removed all of the pinna cues (e.g. Carlile and King 1994). In Van Wanrooij and Van Opstal's (2005) experiments using monaural pinna perturbation, subjects who demonstrated substantial changes in localization performance while wearing the mould had front–back reversal rates ranging between 2 and 50 % for locations ipsilateral to the mould (Van Wanrooij and Van Opstal 2005) although at the stimulus levels similar to those used here, the percentage of front–back reversals were closer to the percentages found in this study. Of interest, the wide range of front–back confusions reported by that group is consistent with the wide range of front-to-back confusions in the present study (Fig. 6B) where, on acute exposure to the mould, about half the subjects demonstrated a very low level of front-to-back confusions but a high level of back-to-front confusions. In the Van Wanrooij and van Opstal study, no stimuli were presented from the back so all front–back confusions correspond to our front-to-back confusions.

The doubling of the polar angle error (PAE) and the very significant fall in the spherical correlation coefficient (SCC) was qualitatively quite similar to that found previously (Hofman et al. 1998; Van Wanrooij and Van Opstal 2005) although different quantitative measures were used. Likewise in this study, there was no statistical effect on the perception of lateral angle, which is consistent with the idea that the moulds have almost no effect on the binaural cues to localization—a finding that is universally reported (e.g. Oldfield and Parker 1984b; Hofman et al. 1998; Van Wanrooij and Van Opstal 2005).

Accommodation to the new spectral cues

At the end of the accommodation period, all of the performance measures (with the exception of the lateral angle error) trended towards the control measures (Fig. 3); however, the group means were still poorer than the controls. Looking at the individual performance data, we found only one subject who fell within the normal control range for all three parameters. Interestingly, this subject had a 38-day accommodation period which was the median for the group. Since longer accommodation periods did not correlate with increased performance gains, it is unlikely that duration of accommodation was a limiting factor in this study.

Again, these results accord with Hofman et al. (1998) in which subjects using bilateral moulds accommodated to around 50 to 75 % of their control values (their Fig. 3). In the monaural mould study conducted by Van Wanrooij and Van Opstal (2005), adaptation was slightly less and rarely extended into the whole of the ipsilateral hemisphere affected by the mould (their Figs. 8 and 9). In the latter case, it is also possible that the contralateral (bare ear) was having an increasing effect on the perception of elevation over the accommodation period, potentially through some relative reweighting of the cues (c.f. Van Wanrooij and Van Opstal 2007; Kumpik et al. 2010). Behavioural studies using owls have also demonstrated that the mature auditory system is able to accommodate to changes in the auditory input produced by clipping the facial ruff (Knudsen et al. 1994) though, interestingly, this manipulation is likely to mostly affect the orientation of the iso-ILD lines rather than some specific monaural cue (Campenhausen and Wagner 2006).

Following adaptation and removal of the mould, localization performance was not statistically different from the pre-accommodation control tests (Figs. 3 and 4). This confirms the previous findings (Hofman et al. 1998; Van Wanrooij and Van Opstal 2005) although those studies did not report any specific statistical test. These data, however, extended this general observation by demonstrating that the new or accommodated cues also persist in the absence of on-going exposure to the mould. In this study, the majority of subjects were tested with their moulds a week or so after the accommodation period to determine if the new cues had been retained in the absence of on-going exposure (Figs. 3 and 4; Ext). Again, while there were some small differences in the SCC and PAEs compared to performance on the last day of accommodation, performance was statistically indistinguishable. On the one hand, it is quite remarkable that several weeks of chronic exposure to the mould seems to have no impact (or after-effect) on performance with bare ears. It is even more remarkable that following exposure of only a few weeks, the new cues persist in the absence of any on-going reinforcement. How long these cues persist remains to be seen, but other work in our laboratory suggests that this could exceed 70 days post-accommodation. If calibration to on-going changes in the shapes of—and therefore spectral cues produced by—the ears is a continuous process, the reason for the persistence of any one instance of the cues is not immediately apparent. It is unlikely that retaining this cue information is of some ecological utility but may instead reflect some epiphenomenal aspects of the encoding of these cues. If true, this would also likely set this sort of accommodation apart from other forms of cognitively directed learning.

Possible Mechanisms of Recalibration

While these data demonstrate that contemporaneous visual input is not necessary as a teacher signal in any process of recalibration, they do not speak to the issue of what other inputs might have that role. The possibility that other sensorimotor mechanisms maybe involved in the accommodation to altered spectral cues was raised when these observations were first made (Hofman et al. 1998). Prior to the current experiment, however, there have been no studies that have explicitly looked at the capacity to accommodate to new spectral cues for locations outside the visual field, let alone to the possible role of sensorimotor mechanisms in this process (see however, Zahorik et al. 2006; Parseihian and Katz 2012). In this study, we have shown that the extent of accommodation (Fig. 4) does not differ for locations that are within the visual field compared to locations that are outside the visual field and the rates at which accommodation occurs are also the same for both regions of space (Fig. 6). From a computational perspective, some recent models have been developed whereby auditory spatial representations can be created by unsupervised sensorimotor learning where receiver motion is integrated with acoustic information (Aytekin et al. 2008; Bernard et al. 2012). Whether the motion signal is of motor (efference copy) or sensory origin (proprioception) is not known. However, behavioural work has demonstrated that repetitive exposure to motor and subsequent auditory stimulus, in a behaviourally meaningful context, appears to be important in auditory training with perturbation of binaural inputs (Bergan et al. 2005; Kacelnik et al. 2006). Whether these are important issues for understanding the accommodation to monaural spectral cues remains the subject of further research.