Eye position and cross-sensory learning both contribute to prism adaptation of auditory space

Abstract

Optical prisms shift visual space, and through adaptation over time, generate a compensatory realignment of sensory-motor reference frames. In humans, prism-induced lateral shifts of visual space produce a corresponding shift in sound localization. We recently reported that sound localization shifts towards eccentric eye position, approaching ~40% of gaze over several minutes. Given that eye position affects sound localization directly, prism adaptation may well reflect contributions of both eye position and sensory adaptation; while the visual world is shifted by the prisms, the eyes must also shift simply to gaze ahead. To test this new concept of prism adaptation, 10 young (18–27 year) adults localized sound targets before and after 4h of adaptation to base-right or base-left prisms that induced an 11.4° shift left or right, respectively. In separate sessions subjects were exposed to: (1) natural binaural hearing; (2) diotically presented inputs devoid of meaningful spatial cues; or (3) attenuated hearing to simulate hearing loss. These preliminary results suggest that the prism adaptation of auditory space is dependent on two independent influences: (1) the effect of displaced mean eye position induced by the prisms, which occurs without cross-sensory experience; and (2) true cross-sensory learning in response to an imposed offset between auditory and visual space.

Introduction

Optical prisms shift visual space, and this in turn induces an adaptive realignment of sensory-motor reference frames (Redding et al., 2005). For auditory—visual coordination in particular, prism-induced lateral shifts of visual space produce a corresponding shift in sound localization in the barn owl (Knudsen and Knudsen, 1985; Brainard and Knudsen, 1995). Although prism adaptation of auditory space has also been studied in humans, it is unclear whether the resultant shift in spatial localization can be attributed to an adaptive response in the auditory system (Lackner, 1976). Recently, we reported that sound localization shifts by simply maintaining eccentric eye position. This shift is robust, time-dependent, and spatially broad; it develops exponentially over minutes in the direction of ocular deflection and approaches ~40% of eye eccentricity (Razavi et al., 2007).

Given that eye position itself affects sound localization, prism adaptation of auditory space may well reflect contributions of both eye position and cross-sensory adaptation. Previous examinations of auditory–visual coherence did not take into account the effect of optical prisms on ocular deviation. Specifically, while the visual world is shifted in the direction dictated by the prisms, the eyes must also shift in the same direction simply to fixate the same field of targets. Even though the eyes are free to move, average eye position will shift over time, and this alone, apart from any shift due to auditory–visual interactions, will cause a corresponding shift in sound localization.

More formally, we hypothesize that the prism adaptation of auditory space is comprised of two components: (1) a physiologic adaptation of auditory space in direct response to a new average eye position; and (2) cross-sensory adaptive plasticity, an experience-dependent learning phenomenon that recalibrates auditory to visual space over time. In this preliminary experiment, we revisited the paradigm of prism adaptation to test this hypothesis directly and to quantify the efficacy of these two components. The effect of eye position was examined in isolation by eliminating meaningful binaural localization cues normally present in auditory signals while wearing prisms, thereby disrupting cross-sensory re-calibration. By comparing this adaptation paradigm to that with normal binaural hearing, the contribution of eye position to the overall phenomenon was quantified, while the remaining difference presumably constitutes a cross-sensory learning effect.

Methods

Subjects

Ten normal human subjects (4 male, 6 female; 18–27 years old) participated in this study.

Apparatus and stimulus

Subjects sat in a dark, echo-attenuated room facing the centre of a cylindrical screen of black speaker cloth at 2 m distance. The head was aligned with the horizontal plane using Reid’s baseline, and fixed in place using a personalized bite-bar. Acoustic targets were presented using an 8-cm-diameter two-way coaxial speaker mounted on a two-axis servo-controlled robotic arm hidden behind the screen. The setup enabled rapid positioning of the speaker in cylindrical coordinates and provided an unlimited array of possible targets over the range of ±65° azimuth (Az) and ± 25° elevation (El). Spatially diffuse Gaussian white noise (65 dB sound pressure level, SPL), delivered through two stationary loudspeakers, masked potential predictive positional cues during speaker movements between localization trials. Auditory targets consisted of 150 ms bursts (10 ms rise–fall time) of broadband (0.1–20 kHz) Gaussian noise (equalized to compensate speaker frequency response), repeating at 5 Hz, and randomly varied between 70 dB and 75 dB SPL from trial-to-trial (Razavi et al., 2007).

Experimental paradigm and response measures

Subjects manually localized stationary auditory targets using a red laser-LED pointer mounted on a 2-axis cylindrical joystick, aligning its beam with the perceived sound locations. For each target presentation, subjects registered response endpoint with a key press, and the target and pointer positions were recorded. Subjects were instructed to localize quickly but accurately. Auditory localization was studied before, during, and after 4 h of adaptation to either base-right (R) or base-left (L) prisms (20 prism-diopters) that induced an 11° shift L or R, respectively. For each session, a normal baseline of sound localization was first established without prisms or other devices. Subjects then donned the prisms for 4 h, during which time they engaged in normal active behaviour in and around the University of Rochester Medical Center (noise level ≤ 90 dB SPL), and returned to the laboratory for repeat testing after 1 and 4 h. Testing occurred without prisms or other devices (always removed or restored while subjects were on the bite-bar with eyes closed). In separate sessions, subjects were exposed to: (1) natural binaural hearing; (2) diotic hearing (portable Lavalier microphone-amplifier presented the same monaural signal to both ears through earphones); and (3) sound-attenuation (44 ± 5dB SPL) using earmuffs and earplugs (near-deafness). Twenty-six randomly distributed target locations were tested (repeated measures design; Fig. 1a). To complete all three acoustic conditions under both prism conditions (L and R), six sessions on different days at least 2 days apart were required per subject.

Fig. 1.

(a) Auditory target distribution. (b) Mean change in Az localization accuracy (shift magnitude) for both diotic and binaural hearing conditions following 1 and 4h of prism adaptation (R and L combined), binned in 10° intervals of target Az. (c) Average shift magnitude across subjects, pooled for all target locations (error bars are SDs).

To effectively separate eye position effects from those of cross-sensory interaction, it is important to control for the availability of auditory localization cues. Control sessions examined the efficacy of the lavalier microphone–amplifier (diotic condition) and the earplugs–earmuffs (near-deafness condition) in eliminating effective auditory spatial cues. Linear regression was performed to quantify spatial gain (slope) of response relative to target positions across horizontal space. As would be expected following the elimination of useable cues, spatial gains in Az fell to near zero for both diotic (0.05 ± 0.07) and near-deafness (0.03 ± 0.21) conditions, in contrast to a normal binaural baseline of 1.19 (± 0.12, or a 19% overshoot of target positions).

Data analysis

Data were sorted by prism direction (L or R shift), auditory input (binaural, diotic, or near-deafness), and adaptation duration (baseline, 1, and 4h). Accuracy, the error between response and target locations, was normalized to baseline localization for each target. The effect of prism adaptation was quantified as shift magnitude (Δ accuracy in Az) between R and L prism directions across all targets.

Results

Visual prisms consistently shifted sound localization in the direction of visual deviation. The localization accuracy at 1 h differed significantly from baseline for both diotic and binaural conditions (p<0.01). The shift magnitude (Δ accuracy in Az; R minus L) in the normal binaural condition increased over time between 1h (5.92 ± 2.32°, mean ± SD) and 4h (10.4 ± 2.60°,or ~ half the visual shift) of prism adaptation (ANOVA, p<0.01). In contrast, sound localization also shifted under diotic conditions, but did not significantly increase between 1 h (4.62 ± 2.65°) and 4h (6.52 ± 2.94°; p = 0.1) of prism adaptation. Interestingly, the small augmentation in shift magnitude between the two time points of the diotic condition was restricted to the central region of head-centred auditory space, demonstrating a spatial selectively not present in the binaural condition (Fig. 1b). Note that the shift magnitudes were similar between the two conditions after 1 h of prism adaptation (p = 1.00), but differed significantly after 4h (p = 0.02; Fig. 1b, c). Additionally, the shift magnitude in the near-deafness condition (3.89 ± 3.39°) was comparable to that of both diotic (p = 1.00) and binaural (p = 0.63) conditions following 1 h of prism adaptation.

Six subjects (2 male, 4 female; 21–27 years old) also participated in an alternating fixation paradigm (Razavi et al., 2007). In this paradigm, head-fixed subjects maintained ocular fixation on one of three red laser-LED spots projected on the screen (centre, L20°, or R20° Az; 0° El), and used peripheral vision to guide the pointer to localize auditory targets. Two sessions (on different days) of 161 trials were parsed into 5 separate but contiguous epochs (Fig. 2). Sessions began and ended with an epoch of central fixation, interjected by a sequence of three epochs of ocular eccentricities that reversed order between sessions. The paradigm measured the shift’s dynamics in response to a ± 20° (e.g., epochs 1→2, 4 →5) as well as a ±40° (e.g., epochs 2→3, 3→4) change in eye position. The time course and amplitude of the shift in sound localization in response to a change in eye position was parameterized using the first-order exponential equation: y(t)=y₀ + a(1−e^−1/τ). Interestingly, results demonstrated that the amplitude of the shift (a, from starting point to asymptote) for a given change in eye position correlated closely with the 1 h shift magnitude in the diotic condition under prisms (0.88; p=0.02).

Fig. 2.

Sound localization accuracy (error) across target locations in one subject. Fixation (solid trace) started at centre and then alternated between R and L 20° in Az. The dashed trace reflects the exponential model. Mean shift across epochs (including session 2) measured 7.96° in this subject.

Discussion

The phenomenon of visual prism adaptation was re-evaluated in a context that includes a newly described phenomenon of a gaze-dependent auditory localization shift (Razavi et al., 2007). Preliminary findings support the notion that the prism adaptation of auditory space is dependent on two independent influences: (1) the effect of displaced mean eye position induced by the prisms, which occurs without cross-sensory experience; and (2) true cross-sensory learning in response to an imposed offset between auditory and visual space.

We have shown previously that the shift in sound localization in response to eye position increases exponentially over time at a highly variable rate among subjects, eventually approaching ~40% of gaze in a matter of minutes. Because the temporal dynamics of the gaze-dependent responses are unknown beyond 30–40 min, we are uncertain whether the small increase in the diotic shift magnitude for longer periods is attributable to additional drift in sound localization towards gaze or the change in shape across space (central progression only). Nevertheless, at 4 h, adaptation of auditory space under binaural conditions showed continued wide-field growth far exceeding that for diotic conditions, suggesting that cross-sensory learning accounts for the additional magnitude and progression of adaptation, beyond that for diotic hearing. Additional support for our hypothesis stems from the close correlation between the shift in sound localization after 1 h of prism adaptation and the shift amplitude during the alternating fixation paradigm. The latter constitutes a pure assessment of the auditory spatial shift induced by ocular eccentricity.