The power of vision: calibration of auditory space after sight restoration from congenital cataracts

Abstract

Early visual deprivation typically results in spatial impairments in other sensory modalities. It has been suggested that, since vision provides the most accurate spatial information, it is used for calibrating space in the other senses. Here we investigated whether sight restoration after prolonged early onset visual impairment can lead to the development of more accurate auditory space perception. We tested participants who were surgically treated for congenital dense bilateral cataracts several years after birth. In Experiment 1 we assessed participants' ability to understand spatial relationships among sounds, by asking them to spatially bisect three consecutive, laterally separated sounds. Participants performed better after surgery than participants tested before. However, they still performed worse than sighted controls. In Experiment 2, we demonstrated that single sound localization in the two-dimensional frontal plane improves quickly after surgery, approaching performance levels of sighted controls. Such recovery seems to be mediated by visual acuity, as participants gaining higher post-surgical visual acuity performed better in both experiments. These findings provide strong support for the hypothesis that vision calibrates auditory space perception. Importantly, this also demonstrates that this process can occur even when vision is restored after years of visual deprivation.

1. Introduction

It is widely accepted that blind individuals can partially compensate for their visual impairment through other sensory modalities (sometimes even outperforming sighted individuals in auditory or haptic tasks [1–4]). For instance, congenitally blind participants can localize single sounds along the azimuth as well as, or even better than typically sighted controls [3,4]. Such compensatory behaviours are associated with a reorganization of the visual cortex that becomes responsive to non-visual stimuli (e.g. [5–7]).

However, recent studies have also reported some distinct deficits in blind individuals in some auditory, tactile and proprioceptive tasks [8–17]. For instance, early blind individuals perform poorly when localizing sounds along the vertical axis, rather than the horizontal [8,9], especially for lower elevations [10]. Deficits are particularly evident in tasks requiring the ability to build and update complex representations of space [11–20]. For example, blind children and adults are impaired in auditory space bisection tasks, in which they are presented with three consecutive, laterally separated sounds and have to report whether the second sound is spatially closer to the first or the third sound [18–23]. This task appears particularly challenging because, unlike canonical localization tasks (e.g. [7]) in which participants can estimate the position of a single sound with respect to themselves (i.e. using an egocentric reference frame), it requires the ability to estimate and compare the relative position of sounds in an allocentric reference frame [19].

It has been hypothesized that such deficits emerge as a consequence of atypical cross-sensory calibration processes (e.g. [24,25]): according to the cross-calibration hypothesis, information in one sensory modality for a specific task is used as the (dominant) reference during development for calibrating the other modalities [13,25–29]. For instance, vision typically conveys the most accurate information about space [30] and may therefore be used for calibrating space in the other modalities. Both vision and audition provide information on distal cues. However, while vision provides high-resolution spatial information coded topographically, auditory space perception is indirect and needs to be inferred from a range of cues [31]. It may therefore also be more prone to errors (i.e. less accurate) than visual space. When visual deprivation occurs, proper calibration of the auditory space cannot take place, and auditory spatial maps would consequently be compromised. Support for this hypothesis comes from neurophysiological studies in animals describing altered auditory spatial maps after modified or absent visual input [32,33]. For instance, rearing owls with prism goggles inducing a displacement of the visual field distorts the owls’ auditory spatial maps, which become aligned to the distorted visual field [33]. In humans, non-permanent analogous effects have been reported (e.g. [34,35]). Electrophysiological studies showed that, while in sighted individuals solving the auditory space bisection task activates the visual cortex, such activation is absent in blind individuals [22,23]. This finding suggests that the construction of auditory spatial maps for understanding spatial relationships among sounds might rely on the visual maps of the occipital cortex. The lack of visual calibration of the auditory space in blind individuals might also explain why sound localization is particularly poor for elevation (i.e. along the vertical mid-sagittal plane): while people can rely on binaural cues to localize sounds along the azimuth, vision might be crucial to calibrate the auditory spectral pinna cues related to elevation, where the contribution of binaural cues is less efficient [8,36].

A question that is still unanswered is whether there is a sensitive period for cross-calibration to occur. It has been hypothesized that cross-calibration might be particularly important in the first years of life with the body growing rapidly and the sense organs developing. Consequently, the different sensory systems have to be continuously recalibrated (e.g. [13,14,17,18,24,27,28,37]). Indications that the visual calibration of the auditory space might develop early in life come from studies showing that early but not late blind individuals are impaired in localizing auditory stimuli in the lower part of space [10]. Similarly, early blind are more strongly impaired than late blind in auditory space bisection tasks (although the performance of late blind becomes poorer with prolonged blindness [38]). However, so far, it is not known whether structured visual input in the first years of life is essential for developing the ability to flexibly use vision for cross-calibration, or whether visual experience gained only later in life can still contribute to such development. Studies in adult owls seem to suggest the former, as their auditory spatial maps can be modified by adaptation to a prismatic visual displacement only if the owls have been previously exposed to such visual distortion as juveniles [39] (although comparisons with humans should be done with caution).

Individuals suffering from congenital bilateral cataracts who undergo surgical treatment several years after birth offer the opportunity to investigate whether particular functions can be acquired after cataract removal or whether such functions require early structured visual input to develop. Studying these individuals has highlighted the existence of multiple sensitive periods for different visual skills, which can be permanently altered by abnormal visual experiences during development. This is the case for visual acuity, contrast sensitivity, global motion and global form perception and holistic face processing (see [40–42] for reviews). Similarly, a short period of congenital cataracts can lead to altered multisensory interactions at both neural and behavioural levels (e.g. [43,44]). However, recent studies also describe surprisingly rapid improvements of visual and multimodal abilities in late cataract-treated individuals. For instance, although their visual acuity does not reach the level of the sighted population, it still significantly improves after surgery [45]. Similarly, early visual deprivation does not completely impair the ability to combine vision with other sensory inputs [46,47] once vision is restored, and such multisensory integration abilities can gradually develop following surgery [48]. These findings suggest that the critical period for the development of some visual and cross-modal abilities might extend beyond early childhood.

In the present study, we investigated whether sight restoration after several years of early onset visual impairment might lead to the construction of more accurate auditory space perception, or whether there is a sensitive period which impairs late development of cross-calibration. Moreover, we assessed whether, in cases where such ability can develop, cataract-treated participants reach performance levels comparable to that of sighted individuals in auditory tasks. To this end, we investigated space perception in Ethiopian children and adolescents who received surgical treatment for congenital dense bilateral cataracts several years after birth. We compared their performance with that of individuals tested before surgery, blind participants, and typically developing sighted controls. In Experiment 1, we used the auditory space bisection task to investigate participants' ability to understand spatial relationships among sounds. In Experiment 2, we tested participants’ ability to localize auditory and visual stimuli in a large two-dimensional space presented along participants' frontal planes.

2. Experiment 1

(a) . Methods

(i) . Participants

Fifty-nine participants in four age-matched groups participated in Experiment 1. Sixteen Ethiopian children and adolescents with congenital dense bilateral cataracts were tested after surgery, which took place 8–15 years after birth (‘Post-op’: females/males = 8/8, one left-handed, age = 12.13 ± 2.77 years (mean ± s.d.), time since surgery = 1.31 ± 0.7 years). A second group of seven children was tested before surgery (‘Pre-op’: f/m = 4/3, all right-handed, age = 11.74 ± 2.1 years, see electronic supplementary material and electronic supplementary material, table S1). Among them, two participants were also tested after surgery. Unfortunately, we did not have the chance to test the other participants both before and after surgery. We assessed their spatial visual acuity before (when possible) and after surgery by measuring their contrast sensitivity function (CSF) cut-off frequency (electronic supplementary material and [48]). The CSF cut-off frequency was on average 0.85 ± 1.14 cycle per degree, cpd, in pre-op and 5.52 ± 3.48 cpd in post-op (electronic supplementary material, table S1). Before surgery, all participants had light perception and some residual vision, and according to the measured CSF cut-off frequency, most of them were classified as suffering from legal blindness or severe low vision (electronic supplementary material). After surgery, their visual acuity significantly improved and most participants transitioned out of the category of legal blindness (electronic supplementary material, figure S1).

Since individuals suffering from cataracts typically show some residual vision before surgery (e.g. light perception), we included a group of eight Ethiopian congenitally blind participants (i.e. without light perception; ‘Blind’, f/m = 4/4, all right-handed, age = 11.63 ± 1.41 years). This allowed exploring whether such residual vision could have already contributed to some calibration of the auditory space. While some participants were blind due to congenital peripheral damage (N = 5), we have no medical records regarding the cause of blindness of the remaining participants. However, their parents reported they were completely blind since birth.

Finally, a group of 30 typically sighted individuals (Sighted, nine Ethiopians, 21 Germans, f/m = 12/18, two left-handed, age = 12.38 ± 2.42 years) participated. As the performance of the German and Ethiopian individuals did not differ, data from the two samples were aggregated.

Patients were treated at the Hawassa Referral Hospital. Ethiopian participants performed the task at the hospital, in the Shashamane School for the blind or the Sebeta Blind School. German participants were tested in primary and secondary schools. The procedure was approved by the ethics committee of the University of Bielefeld (EUB 2015-139). Parents or legal guardians gave their written consent to participate in the study.

(ii) . Procedure

Participants took part in an auditory space bisection task and in a single sound localization task, in counterbalanced order across participants (cf. [18]). They sat blindfolded at a table with their head resting on a chin-rest. The set-up consisted of a semicircle printed on fabric, where the experimenter could manually place either one (single sound localization task) or three (auditory space bisection task) speakers. The semicircle was divided into sectors, each being the size of the speaker (5 cm per side), and placed at 57 cm from the head of the participant (cf. [19] and figure 1a).

Figure 1.

Auditory space bisection. (a) Apparatus. (b) Psychometric function in one representative participant for each group. Curves are fitted on the proportion of ‘probe closer to the right sound’, plotted against the probe's position (with negative/positive values indicating probe closer to the left/right sound, respectively). (c) JNDs for the auditory bisection task in the different groups (eight Blind, seven Pre-op, 16 Post-op, 30 Sighted), as estimated from the GLMM on the whole sample, with lower JNDs indicating better performance. (d) Factors potentially contributing to task performance in Post-op (visual acuity, time since surgery, age at test). Individual JNDs are estimated with a GLM in each participant. Asterisks represent statistical significance (at α = 0.05). (Online version in colour.)

In the auditory space bisection task, participants had to spatially bisect three consecutive spatially distributed sounds by reporting whether the second sound was closer to the first or the third one. The first sound was always presented to the left of the participant, the third one to the right, while the second sound (i.e. the probe) at one position between the two, varying across trials (see below). In each trial, the locations of the first and third sounds could be presented at one of four possible combinations, while the distance between the two was kept constant and covered 40° of visual angle: −30° and +10°, −10° and +30°, −22° and +18°, −18° and +22°, where negative and positive values indicate locations to the left or right of the participant, respectively. The probe could be presented at any of eight possible locations between the other two (figure 1a). The stimuli were three animal sounds: a chicken, a horse and a dog. Each auditory stimulus lasted 2 s, in which the animal sound was repeated three times. In each trial, the three animal sounds were presented successively at 500 ms intervals. Previous studies used either brief pink and white noise bursts or pure tones as auditory stimuli [18–23]. However, here we chose animal sounds to make the experiment more engaging and easier to understand, especially since we relied on interpreters for translating the instructions.

Participants verbally reported whether the probe animal (second sound) sounded as if it was spatially closer to the animal on the left or the right. After a few practice trials, participants took part in two blocks of 16 trials. In each block, the probe was always the same animal (e.g. dog). In the next block, the probe was changed to another animal (e.g. horse). The types of animals in each block and their order were randomized across participants. In each block, the probe was presented twice in each of the eight positions, yielding a total of 32 trials. The first participants were presented with three blocks (one for each animal sound). However, since the task was long and tiring for the children, we reduced the task to two blocks, after checking, in the first participants,–that the specific type of animal sound used as a probe did not affect performance.

In the single sound localization task, we investigated sound localization along the azimuth within the same set-up. This task assessed whether any possible deficit shown by any of the groups in the auditory space bisection task could be ascribed to a simpler deficit in localizing single sounds rather than in understanding spatial relationships among them (cf. [18]). All groups were able to accurately localize single sounds along the azimuth. Details on this task are provided in electronic supplementary material.

(iii) . Statistical analyses

We fitted the probability of responding ‘probe closer to right sound’ with a generalized linear mixed model (GLMM) with position of the second sound (i.e. probe's relative position with respect to the other two sounds), group (Blind, Pre-op, Post-op and Sighted) and the interaction between them as fixed effect predictors. A Probit link function was applied. To account for the heterogeneity among different participants, we included the random intercept alone or both the random intercept and slope, with and without interaction, as random effects predictors. We compared the models by means of the Akaike information criterion (AIC). The best models according to AIC included only the random intercept as a random-effect predictor. We used maximum-likelihood estimation to estimate the parameters.

(b) . Results

Figure 1b depicts the performance of one representative participant for each group. Figure 1c shows the just noticeable differences (JNDs) resulting from the GLMM (with lower JNDs indicating better performance). The GLMM revealed that Blind and Pre-op exhibited a similar performance (JND, calculated at the 84th percentile ± standard error, Blind: 19.62 ± 6.35°, Pre-op: 20.52 ± 4.37°, t₁₉₀₆ = 0.15, p = 0.88). Both groups performed significantly worse than Sighted (4.13 ± 0.32°, Blind versus Sighted, t₁₉₀₆ = 9.91, p < 0.0001, Pre-op versus Sighted, t₁₉₀₆ = 10.44, p < 0.0001). Importantly, Post-op executed the task significantly better (11.13 ± 1.73°) than both Pre-op (t₁₉₀₆ = 3.02, p = 0.0025) and Blind (t₁₉₀₆ = 2.66, p = 0.008). The finding that Post-op perform better than Pre-op is further confirmed by the fact that the two participants who were tested both right before and around four months after surgery strongly reduced their JNDs in the post-surgical test (9.59° and 10.85°) compared with the pre-surgical assessment (13.42° and 42.29°, respectively). Thus, while blind and cataract participants tested before surgery performed poorly in this task, confirming previous evidence in the blind [18–20], cataract-treated individuals show some rapid visual calibration of the auditory space, occurring already within months after surgery. However, despite this improvement, Post-op did not reach the performance level of the Sighted, who showed significantly lower JNDs (t₁₉₀₆ = 8.37, p < 0.0001).

To investigate which factors could have mediated the improvement shown by Post-op participants at the individual level, we calculated their individual JNDs by fitting the proportion of ‘closer to right sound’ in each participant with a generalized linear model (GLM) with second sound position as a fixed effect predictor (see electronic supplementary material, figure S2 for individual JNDs). Since two Post-op participants were completely unable to perform the task, we conservatively assigned them an arbitrary JND of 40° (i.e. equal to the distance between the external speakers, cf. [18]). We ran a multiple regression analysis on these JNDs with the factors age at test, visual acuity and time since surgery (all log-transformed; those three factors were not correlated among them, all p-values > 0.15). The JND tended to be affected by both visual acuity (t(12) = 1.92, p = 0.079) and time since-surgery (t₁₂ = 2.05, p = 0.06, figure 1d): the higher the visual acuity and the longer the time since surgery, the smaller the JND. Instead, age at test did not affect the performance (t₁₂ = 0.006, p = 0.99, figure 1d).

Overall, these findings show that before surgery, individuals suffering from congenital bilateral cataracts are as impaired as blind participants in the auditory space bisection task. Their poor performance cannot be merely ascribed to a generic impairment in localizing single sounds, since they localize single sounds along the azimuth as well as sighted controls (electronic supplementary material, cf. [18]). Importantly, after surgery participants show signs of visual calibration of the auditory space, and such recovery appears to be mediated by visual acuity and time since surgery.

3. Experiment 2

As for sound bisection, previous studies have reported poor performance in the blind also for sound localization in elevation (e.g. [8,9]), with specific impairments occurring in the lower frontal plane [10]. In Experiment 2 we used a set-up similar to that of [10] to investigate whether participants with bilateral cataracts, before and after surgery, could accurately localize sounds in a large two-dimensional space presented along their frontal plane. As participants with cataracts typically have light perception even before surgery, we also assessed their ability to localize light sources. Comparison of the ability to localize visual stimuli before and after surgery offered us the possibility to assess the functionality of the improved visual acuity post-surgery. Since some of the cataract-treated participants were tested only a few days after surgery, we wanted to ascertain that their visual recovery would translate into a fast improvement in performing spatial visual tasks. That is, we wanted to exclude that a possible lack of post-surgical improvement in sound localization could simply be ascribable to the inability to use vision right after surgery, which would in turn prevent the visual calibration of the auditory space.

(a) . Methods

(i) . Participants

We tested 21 Ethiopian cataract-treated participants (‘Post-op’: f/m = 12/9, one left-handed, age = 11.80 ± 2.96 years, time since surgery = 1.09 ± 0.92 years, visual acuity = 3.69 ± 3.16 cpd,). Five of them were tested also right before surgery (‘Pre-op’: all right-handed, age = 10.83 ± 3.52 years, time before surgery = 3.20 ± 2.7 days, visual acuity = 0.49 ± 0.53 cpd). Eighteen of them took part also in Experiment 1 at the same or at a different time since surgery (see electronic supplementary material, table S1). A control group of 29 typically sighted German individuals (Sighted, f/m = 23/6, two left-handed, age = 13.1 ± 2.3 years) also participated. Among them, eight took part also in Experiment 1.

(ii) . Procedure

Participants sat in front of the set-up, consisting of a circle printed on fabric (∅ = 1 m) hanging from the ceiling (cf. [10], electronic supplementary material and electronic supplementary material figure S3 for further details). The centre of the circle was positioned halfway between participants' eye and ear level. The task consisted of two blocks, in counterbalanced order across participants: in one block participants localized visual stimuli (visual condition), and in the other auditory stimuli (auditory condition). The experimenter presented each stimulus by hand on the circular set-up. After stimulus presentation, the participant reached for the location where they believed the stimulus came from, by directly touching the fabric with the index finger of their dominant hand. To make the stimuli easy to reach, the diameter of the circle in which the stimuli could be presented was 60 cm for the shorter (typically younger) participants and 90 cm for the taller (typically older) participants, leading to an average diameter of 75 cm (electronic supplementary material, figure S3). In each block (auditory, visual), 24 trials were delivered. Given that the task was tiring for the younger participants, for around half of the participants we presented only 16 trials per block (electronic supplementary material).

Participants were blindfolded before approaching the set-up. They were seated in front at a comfortable reaching distance. In the auditory condition, the experimenter held a small speaker (5 cm cube) and placed it onto the target circle at the desired location. A metronome sound (single 500 Hz pulse intermittent sound at 180 bpm as in [10]) was played for 3 s. When the sound stopped, the experimenter removed the speaker and the participants had to reach for the location where they believed the sound came from. Participants did not receive any feedback on their reaching performance.

In the visual condition, the blindfold was removed and at the beginning of each trial participants were required to keep their eyes closed. The experimenter positioned a small LED flashlight behind the set-up and asked participants to open their eyes. The experimenter left the light (a high-contrast disc of around 2 cm in diameter) on for around 1 s or slightly longer if Pre-op participants required more time to orient in the direction of the light. When the light was off, participants closed their eyes and reached for the location where they believed the light was. A numbered grid was printed on the fabric, so that the experimenter could take note of actual and reported target locations (both azimuth and elevation). For each trial we calculated the absolute localization error as the linear distance between target and participant's response.

(iii) . Statistical analyses

We first investigated whether the three groups differed in their absolute localization error (cf. [10]). For each condition (audition, vision) we compared such error across the groups via a linear mixed effect model (LMME) with group (Pre-op, Post-op and Sighted) as a fixed effect predictor. Next, since previous evidence reported impaired sound localization in the lower part of the space in blind individuals [10], we assessed participants' accuracy at different heights. For that, we divided the set-up into three heights along the vertical axis: upper, middle and lower (figure 2a). As the diameter of the target circle was 75 cm on average across participants, the middle height was 25 cm (12.5 below and above the centre line), while the upper and lower heights included all target locations above and below that middle height, respectively (figure 2a). For each condition, we compared the previous model, which included only group as a fixed effect, with a series of LMMEs including also height (upper, middle and lower) as a fixed effect, with and without a possible interaction with group.

Figure 2.

Auditory and visual localization in the frontal plane. (a) Performance in the auditory localization task (five Pre-op, 21 Post-op, 29 Sighted). For graphical representations, results are summarized by averaging targets' positions and participants’ endpoint locations along azimuth (x-axis) and elevation (y-axis) in nine different regions of the set-up (defined by the dashed lines). The nine regions were obtained by dividing the space for the analyses into nine equally sized regions: the central region was 25 cm wide and high (i.e. 12.5 cm in each direction from the centre). Since the stimuli were presented on average in a 75 cm diameter circle, each region was around 25 cm wide and high. The dark grey grid connects the averaged actual targets' positions. The magenta grid connects the averaged location of participants’ responses. The black arrows indicate the direction of the localization errors. The horizontal dashed lines indicate the boundaries of the different heights: upper (Up), middle (Mid) and lower (Low) (see electronic supplementary material for further analyses). (b) Results of the visual localization task for the three groups. (c) Absolute localization error (as linear distance between target and response) in the auditory condition averaged across participants in each group. (d) Correlation between visual acuity and localization error in the auditory condition in Post-op. (e) Absolute localization error in the visual condition averaged across participants in each group. (f) Correlation between visual acuity and localization error in the visual condition in Post-op. (g) Absolute localization error in the auditory condition in the different heights averaged across participants in each group. (h) Absolute localization error in the visual condition in the different heights averaged in each group. Error bars represent SEM. Asterisks represent statistical significance. (Online version in colour.)

All models included either the random intercept alone or both the random intercept and random slope, with and without interaction, as random effect predictors. According to AIC, most models included only the random intercept, unless specified otherwise in the Results section. Maximum-likelihood estimation was used.

(b) . Results

Performance in the auditory and visual conditions is summarized in figure 2a,b individually for each group. Overall, Pre-op showed larger localization errors compared with Post-op and Sighted in both the visual and auditory conditions. In the auditory condition, the LMME showed that Pre-op were significantly less accurate than the other groups: their absolute error (14.13 ± 0.69 cm) was greater than that shown by Post-op (11.11 ± 0.46 cm, t₉₉₇ = 3.79, p < 0.001) and Sighted (12.48 ± 0.37 cm, t₉₉₇ = 2.1, p = 0.036). Instead, Post-op did not differ from Sighted (t₉₉₇ = 1.15, p = 0.25; figure 2c). However, when taking the three different heights into account, the model including group, height and their interaction as fixed effects and the random intercept and slope as random predictors provided a better fit. As expected, Pre-op were less accurate in the lower height, as compared to the upper (17.45 ± 1.68 cm versus 11.47 ± 0.98 cm, respectively, t₉₉₁ = 2.23, p = 0.026). Their performance in the middle height was in-between the one for the other two: the error did not differ from that in the upper height (13.70 ± 0.84 cm, p = 0.441; figure 2g, left panel), while it showed a trend for being smaller than for the lower height (t₉₉₁ = 1.91, p = 0.056). Importantly, compared to Pre-op, Post-op reduced the error in the lower (13.04 ± 1 cm, (t₉₉₁ = 2.19, p = 0.029) and middle heights (9.92 ± 0.53 cm, t₉₉₁ = 2.08, p = 0.038). Instead, their absolute error in the upper heights did not differ from that shown by Pre-op (11.11 ± 0.94 cm, t₉₉₁ = 0.002, p = 0.99). While Pre-op were less accurate at lower elevations, the accuracy of Post-op was comparable among the three heights of space (all p-values > 0.21; figure 2g, central panel), as it was in Sighted (lower: 13.55 ± 0.82 cm, middle: 12.19 ± 0.49 cm, upper: 11.95 ± 0.73 cm, all p-values > 0.33; figure 2g, right panel). Crucially, Post-op and Sighted did not differ in any height of space (all p-values > 0.09). However, despite such improvements in their absolute error, Post-op still presented a bias along the vertical axis (i.e. a systematic pointing error toward the centre of the set-up), especially for lower heights, analogous to that of the Pre-op group (figure 2a and electronic supplementary material, figure S4). Also Sighted showed this bias, but to a lesser degree.

The LMME in the visual condition showed that the groups performed the task differently from each other: Pre-op presented a larger absolute error (5.68 ± 0.47 cm) than Post-op (2.07 ± 0.22 cm, t₉₆₈ = 8.53, p < 0.0001) and Sighted, who performed the task near-perfectly (0 ± 0 cm, t = 10.83(968), p < 0.0001). Although Post-op were more accurate than Pre-op, they performed worse than Sighted (t₉₆₈ = 5.62, p < 0.0001; figure 2e). When including also the different heights in the analysis, the best fitting model included group, height and their interaction as fixed effects, and independent random effect terms for intercept and slope. Pre-op were significantly more accurate in the upper heights (4.10 ± 0.60 cm), compared with the lower (6.56 ± 1.05 cm, t₉₆₂ = 2.87, p = 0.004) or middle ones (6.26 ± 0.72 cm, t₉₆₂ = 2.11, p = 0.035; figure 2h, left panel). Post-op showed better performance than Pre-op in the lower (2.36 ± 0.76 cm, t₉₆₂ = 3.53, p = 0.0004) and middle heights (1.85 ± 0.21 cm, t₉₆₂ = 3.53, p₉₆₂ = 0.0004), but not in the upper heights (2.22 ± 0.3 cm, t₉₆₂ = 1.68, p = 0.10; figure 2h, central panel). Although Post-op showed better performance than Pre-op, they were still less accurate than Sighted for all heights (all p-values < 0.001, figure 2h, right panel). Moreover, while Pre-op participants showed a bias along elevation analogous to that found in audition, Post-op participants reduced such a bias (electronic supplementary material, figure S4).

The same post-surgical improvement was evident in the five participants who were tested both before and after surgery (electronic supplementary material, table S1): although their post-surgical assessment took place only a few days after cataract removal, participants already reduced the absolute localization error in both the auditory and visual conditions (9.38 ± 2.25 cm and 2.82 ± 0.93 cm, respectively), as compared with the pre-surgical test (14.10 ± 0.57 cm, and 5.75 ± 1.30 cm, respectively).

To investigate which factors may have mediated the improvement in Post-op, we ran a multiple regression analysis for each condition on the absolute error with the factors age at test, visual acuity and time since surgery (all log-transformed; those three factors were not correlated among each other, all p-values > 0.07). Outliers above or below 2 s.d. from the group mean were excluded from the analysis. This led to the exclusion of one participant in the visual condition, and two in the auditory condition. In vision, the absolute error was significantly affected by visual acuity (t₁₆ = 3.55, p = 0.0026 figure 2f): as expected, the higher the visual acuity the smaller the error, thus better performance. Instead, time since surgery and age at test did not affect task performance (t₁₆ = 0.11, p = 0.92 and t₁₆ = 0.64, p = 0.53, respectively). Importantly, the absolute error was affected by visual acuity also in the auditory condition (t₁₅ = 2.84, p = 0.012, figure 2d), with higher visual acuity leading to smaller sound localization error. Time since surgery and age at test did not influence the performance in the auditory condition (t₁₅ = 1.52, p = 0.15 and t₁₅ = 1.01, p = 0.33, respectively).

These findings show that participants tested after surgery localized visual and auditory stimuli in the frontal plane better than those tested before, especially in the lowest tested locations. This improvement appeared mediated by the enhanced visual acuity gained after surgery, with better visual acuity leading to better performance in both vision and audition. However, despite this improvement, specific localization biases along elevation are still present after surgery.

4. General discussion

We investigated whether sight restoration after prolonged early onset visual impairment due to congenital bilateral cataracts could lead to the visually guided calibration of the auditory space. Cataract-treated participants had a better performance in an auditory bisection task (Experiment 1) and in an auditory and visual localization task (Experiment 2) compared with untreated cataract participants. However, they often did not reach the performance levels of the sighted controls. Note that this is not surprising in vision, as post-surgical visual acuity does not usually reach the normative range [40,45]. Crucially, the improvement shown by cataract-treated participants in both experiments correlated with post-surgical visual acuity: higher visual acuity was not just related to better performance in the visual but also in the auditory tasks.

These findings strongly support the cross-calibration hypothesis, which suggests that early sensory deprivation often results in severe deficits also in the sensory modalities that are typically calibrated by the damaged modality for a given task (e.g. [17,18,25–27,49]; for examples of non-visual modalities calibrating the others, see [13,26,27,50]). In the present study, we showed that sight restoration can calibrate audition, leading to more accurate auditory spatial perception. Remarkably, this happens even when restoration takes place after prolonged early onset visual impairment.

However, with the present study we probably cannot fully appreciate the potential contribution of time (and thus experience) after surgery to the development of spatial calibration: in Experiment 2 we did not observe a significant correlation between sound localization and time since surgery, probably because cataract-treated participants reduced their absolute error rapidly in localizing single sounds, showing an absolute error analogous to that of the sighted controls already within days or weeks following surgery. Instead, in Experiment 1 we found a trend for participants to perform better in the auditory bisection with longer time past surgery. However, this correlation seems to be mainly driven by one participant, incidentally the only one we could test several years after surgery. It is important to note that all the other participants we managed to assess in both experiments were tested in a narrow temporal window following surgery (between 3 months and around 1 year and a half). Although they quickly improved in the auditory tasks after surgery, only a few of them reached the performance of the sighted controls in the investigated timeframe. At the moment, whether cataract-treated individuals might reach performance levels of the sighted controls if provided with some more years of post-surgical experience is still an open question. In principle, the improved visual acuity gained after surgery could lead to further improvements with time past surgery, by providing cataract-treated individuals with opportunities to better interact with the environment. Such further visual, multisensory and visuomotor experience might in turn further contribute to the development of the auditory spatial representations.

In line with the cross-calibration hypothesis, it is possible to hypothesize that sight restoration might have led to the construction of visual–spatial maps that, in turn, could have been used to build more accurate auditory spatial maps. Support for this hypothesis comes from electrophysiological evidence. In auditory bisection tasks, the visual cortex is recruited in sighted but not in blind individuals, suggesting that the construction of the auditory spatial maps, which are needed to understand spatial relations among sounds, might rely on visual maps and might be mediated by visual experience [22,23]. Blind individuals seem to hinge on temporal representations, rather than spatial representations, to solve the auditory spatial bisection task. Indeed, when also provided with temporal cues, they tend to answer whether the probe sound is closer to one of the other two sounds in time rather than space [21]. Moreover, their occipital responses correlate with the position suggested by the temporal delay of the sound, rather than with its physical position [51].

Future studies investigating the activation of the visual cortex in cataract-treated individuals during auditory bisection tasks could give further support to the cross-calibration hypothesis. In this case, when participants are provided with both temporal and spatial cues (as in [21,51]), we should observe a ‘shift’ in brain activations: instead of reflecting temporal processes, as in the blind [51], occipital activations following surgery should gradually start reflecting spatial processing, as in the sighted [22,23]. Similarly, at the behavioural level, cataract-treated participants should not be biased as much as blind participants when provided with temporal cues that are in conflict with the spatial cues [21].

The fact that structured vision gained late in life can lead to more accurate auditory space perception questions the existence of a critical period in early childhood for cross-calibration processes. The lack of a narrow critical period for cross-calibration in early life offers some advantages. Indeed, during development, all perceptual systems involved in spatial processes require constant recalibration to account for physical growth [24,37]. Such systems might keep cross-calibrating each other during development, with the most accurate sensory system for a specific task calibrating performance for that task in the other modalities. The lack of an early critical period might guarantee more flexibility for these processes. However, we cannot exclude that delayed pattern visual input might have further extended a possible sensitive period. For instance, previous evidence has shown that contrast sensitivity can also develop when sight is restored after what is typically considered the age of critical periods [45].

Our findings suggest that structured vision gained late in life can lead to more accurate auditory space perception, showing that cataract-treated participants still preserve considerable plasticity despite many years of visual impairment.

Ethics

The procedure was approved by the ethics committee of the University of Bielefeld (ref. no.: EUB 2015-139). Parents or legal guardians gave their written consent to taking part in the study.

Data accessibility

The full datasets including all the experimental results for Experiment 1 (auditory bisection) and Experiment 2 (auditory localization; visual localization) have been deposited in the Dryad Digital Repository: https://doi.org/10.5061/dryad.w3r2280tq [52].

The data are provided in electronic supplementary material [53].

Authors' contributions

I.S.: conceptualization, data curation, formal analysis, investigation, methodology, writing—original draft; S.P.: investigation, writing—review and editing; M.G.: conceptualization, methodology, writing—review and editing; M.E.: conceptualization, funding acquisition, methodology, resources, supervision, writing—review and editing.

All authors gave final approval for publication and agreed to be held accountable for the work performed therein.

Conflict of interest declaration

We declare we have no competing interests.

Funding

This study was supported by the Deutsche Forschungsgemeinschaft (DFG) German–Israel cooperation DIP-Grant awarded to M.E. (grant no.: ER 542/3-1).