Do you hear what I see? How do early blind individuals experience object motion?

Abstract

One of the most important tasks for 3D vision is tracking the movement of objects in space. The ability of early blind individuals to understand motion in the environment from noisy and unreliable auditory information is an impressive example of cortical adaptation that is only just beginning to be understood. Here, we compare visual and auditory motion processing, and discuss the effect of early blindness on the perception of auditory motion. Blindness leads to cross-modal recruitment of the visual motion area hMT+ for auditory motion processing. Meanwhile, the planum temporale, associated with auditory motion in sighted individuals, shows reduced selectivity for auditory motion. We discuss how this dramatic shift in the cortical basis of motion processing might influence the perceptual experience of motion in early blind individuals.

This article is part of a discussion meeting issue ‘New approaches to 3D vision’.

1. Introduction

Understanding the motion of objects is critical to the ability of individuals to track and interact with objects in the 3D environment. In sighted individuals, these tasks are almost exclusively carried out using visual information. Most of us experience little to no disorientation weaving our way through a moving crowd to rapidly exit an overly noisy rock concert, but would be understandably reluctant to cross a busy intersection with our eyes closed. Indeed, the word ‘blind’ is commonly used as a synonym for confusion or a general lack of situational awareness—as her irate professor approached, she was blind to the very real danger …

Yet, blind individuals are by no means ‘blindly’ and helplessly stumbling about, incapable of crossing roads. Instead, blind individuals, especially those who become blind early in life, show remarkable fluency navigating a world designed by, and for, sighted people. This navigation relies primarily on auditory information; among the four remaining senses, audition is the only modality that provides information about distant space. This switch to audition as the primary sense that mediates the representation of 3D space requires remarkable sensory adaptations, including the development of novel specialized responses within deprived occipital cortex—cross-modal plasticity, and the development of hyper-expertise within existing areas. However, the mechanisms underlying this cortical plasticity are only just beginning to be understood.

Here, we focus on how early blind individuals perceive auditory motion. We begin by comparing what is known about visual versus auditory motion processing in sighted individuals. Then we discuss how blindness early in life leads to cross-modal plasticity, whereby hMT+, an area known for its selectivity for visual motion in sighted individuals, shows novel responses to auditory motion stimuli. By contrast, the planum temporale (PT), associated with auditory motion in sighted individuals, shows reduced selectivity to auditory motion as a result of blindness early in life. Finally, we discuss the effects of blindness on the perception of auditory motion.

Understanding how early blind individuals perceive auditory motion in space provides important insights both into the neuroanatomical basis of cortical plasticity, and into how the computations underlying our perception of 3D space are optimized to match the available sensory information.

2. Auditory versus visual spatial resolution

One important factor to consider when comparing the computational mechanisms underlying visual and auditory motion processing is the difference in the precision of spatial localization [1] between these two modalities. Humans are capable of detecting visual spatial offsets as small as 2–5 arcsec [2,3], and thresholds for detecting positional displacements for successively presented visual stimuli can be as low as 5 min [4,5]. By contrast, free-field localization of auditory click trains produces discrimination thresholds on the order of 1° [6,7].

The visual and auditory systems also represent spatial location very differently. Throughout early visual cortex, retinal position is directly encoded with remarkable fidelity: neural tuning incorporates information about retinal position, with receptive fields that represent less than 1° within the fovea [8]. In comparison, auditory spatial tuning is broad, within both the primary auditory cortex and PT [9,10], and neural tuning to sound-source locations seem to be represented by an opponent process, based on differences in the activity of two broadly tuned channels formed by contra- and ipsilaterally preferring neurons [9,11,12].

By contrast to the robust retinotopic organization observed in all early visual areas [13,14], no map exists for auditory location (or for proxy cues, such as interaural level or phase differences) in mammalian auditory cortex [15]. The reason for this is not clear, but it may be related to the fact that that retinotopic location and auditory frequency are directly encoded within the retina and the cochlea. Early visual area maps do not represent spatial position [16], which must be computed by combining retinal signals, head position and eye position. Similarly, early visual area maps do not reflect depth, which is inferred using a combination of cues such as stereopsis and motion parallax. Like spatial location, and depth, sound location must be inferred through a combination of cues, including interaural level differences, interaural time differences, pinna shadows and Doppler shift. Although these cues support extremely high temporal sensitivity (e.g. gap detection on the order of 1.5–3 ms [17]), spatial resolution is poor [1].

Finally, visual information, in the absence of occlusion, is continuous, whereas auditory information is sporadic. Take two examples—watching a basketball player, and moving through a crowded airport terminal without bumping into anyone. In both scenarios the visual motion information projected from the 3D world consists of continuous object trajectories with only brief periods of occlusion. By contrast, the auditory signals projected from the 3D world are sporadic: when the player is standing still without the ball she effectively disappears. Similarly, when she is heavily guarded by the opposition, there is no way of spatially resolving her from the other players—she rapidly becomes the auditory equivalent of a sardine in a school of fish. In the airport, one can listen for footsteps, conversations or the sounds of a rolling bag, but the vast majority of objects around one remain acoustically invisible or only sporadically visible. Thus, tracking auditory motion essentially requires inferring continuously moving objects from occasional auditory ‘clues’.

3. The neural basis of visual motion processing in sighted individuals

In sighted individuals, the hub of visual motion processing is hMT+ (human middle temporal cortex) [18]. hMT+ consists of regions analogous to macaque areas MT/MST/FST, along with some possible additional areas tuned for complex types of motion [19–21]. The ecological importance of MT and its surrounding motion areas cannot be overestimated: hMT+ is the largest ‘specialized’ area in the visual hierarchy. In non-human primates, MT neurons have exquisite selectivity for motion direction [22] and speed [23], as well as more complex motion patterns [24]. Neurons in MT and MST also play a crucial role in the perception of 3D space [25], including tracking objects moving in depth [26,27], inference of 3D structure from motion [28,29], motion parallax [30,31] and optic flow [32–35]. Importantly, responses in both MT and MST are associated with the conscious perception of motion. Given a stimulus with an ambiguous direction of motion, the responses of MT and MST neurons correlate with the ‘perceptual choice’ of the animal on a trial-by-trial basis [36,37], and lesions of area MT cause selective impairments of motion perception [19]. The human analogue, the hMT+ complex, has similarly been shown to have selectivity to perceived motion direction [38] and responds to complex motion patterns [39,40] and motion in depth [41,42]. Patients with damage in brain regions overlapping with hMT+ can report akinetopsia or ‘stroboscopic vision’ in which moving objects appear as frames of a cinema reel, and have great difficulties parsing moving objects [43,44].

Responses in MT neurons are classically described in terms of non-separable spatio-temporal tuning [45]. Figure 1a left panel shows a spatio-temporal description of a vertical bar moving to the right. As described by Adelson & Bergen [45], if we ignore the y-dimension (since a vertical bar is unchanging along that dimension), we can represent this stimulus using only the x–t plane (figure 1a, right panel). The moving bar becomes a slanted strip, and detecting motion is equivalent to detecting its spatio-temporal orientation. In other words, any 1D motion signal can be conceptualized as a diagonally oriented template in the space–time plane, where the orientation of the template represents velocity, v = x/t, with a steeper slant representing tuning for lower speeds.

Figure 1.

(a) Spatio-temporal depiction of a vertical bar moving to the right, redrawn from Adelson & Bergen [45]. In the left panel all three dimensions are shown, and in the right panel only the x–t slice is shown. (b) Classic separable (upper panel) and non-separable (lower panel) spatio-temporal templates in a space–time plot. Bright and dark regions correspond to excitatory and inhibitory inputs to the template, respectively. Both templates are selective for rightward motion. Responses to six spatio-temporal stimuli are shown for each template.

The upper panel of figure 1b represents a classic spatio-temporally separable template; defined as separable because it is constructed as the outer dot product of separate spatial and temporal tuning functions (shown with dotted grey lines). The grey bars show predicted responses from this template to a variety of moving stimuli. Although this template is clearly responsive to rightward motion, the strongest responses are elicited for a discrete flash on the left followed by one on the right. Consequently this type of template is considered to be tuned to discrete changes in position over time rather than continuous motion. The final three bars represent responses to analogous spatio-temporal stimuli moving leftward—all three produce suppressive responses.

The lower panel of figure 1b shows a classic non-separable template, typical of MT tuning. The largest response is to a stimulus whose velocity matches the template. Weaker responses are obtained for a stimulus whose speed is lower than the ideal stimulus for the template. It is worth noting that discrete flashes elicit strong responses, as observed by Adelson & Bergen [45]. Spatio-temporal stimuli moving leftward produce suppressive responses.

This computational specialization for visual motion in MT is evolutionarily ancient—analogues to MT can be seen in ancestral predecessors to primates across over 50 million years [46]. Both retinotopic organization and motion selectivity in MT appears to be at least partially ‘innate’—simple selectivity for visual motion can be observed in infant monkeys as soon as measurements can be made [47]. Thus, it seems highly likely that the non-separable spatio-temporal selectivity of MT for motion processing reflects innate mechanisms designed to support the evolutionarily critical task of processing self-motion and the movement of objects in space.

4. The perception of visual motion in sighted individuals

Non-separable spatio-temporal tuning is also observed in human behavioural measurements of visual motion sensitivity. For example, using a psychophysical reverse correlation paradigm, Neri [48] estimated the templates underlying visual motion detection in sighted observers. Participants discriminated a horizontal bar moving vertically up or down (figure 2a, left panel) which was embedded in noise that consisted of stationary bars that appeared in random locations in space and time with luminance randomly drawn from a Gaussian distribution (right panel). The shape of the perceptual templates was inferred by contrasting the noise present in ‘hit’ trials (where the bar direction was correctly identified) versus ‘miss’ trials (where the bar direction was not correctly identified). The resulting estimated visual motion templates (figure 2b) resemble the non-separable spatio-temporal template of the lower panel in figure 1b—selectivity that mirrors the classic model of MT neurons [45].

Figure 2.

Visual motion templates estimated psychophysically using a reverse correlation technique, replotted from Neri [48, fig. 2l]. (a) Observers were asked to discriminate an upward versus downward moving horizontal bar (signal), a single frame is shown (left panel). Spatio-temporal noise was added to the moving signal bars, a single frame is shown (right panel). (b) Because the signal did not vary across the horizontal dimension of space (x), it can be described as a two-dimensional space–time plot, illustrated by the yellow diagonal overlaid on the estimated perceptual visual motion template. The templates were constructed by sorting trials into four categories based on the direction of the signal direction (up (q = 1) or down (q = 0)) and participants' response (correct (z = 1) or not (z = 0)). The resulting template (F) is derived: $\mathit{F}=\hspace{0.17em}{\mathit{N}}^{[1,1]}(\mathit{x},\mathit{t})-{\mathit{N}}^{[1,0]}(\mathit{x},\mathit{t})+{\mathit{N}}^{[0,1]}(-\mathit{x},\mathit{t})-{\mathit{N}}^{[0,0]}(-\mathit{x},\mathit{t})$, where ${\mathit{N}}^{[\mathit{q},\mathit{z}]}$ denotes the noise sample over space (x) and time (t), and $⟨\hspace{0.17em}⟩$ represents the average across trials. The spatial axis is mirror-inverted (−x) for noise samples with downward-signals to align the template orientations. Figure from [49].

5. The neural basis of auditory motion processing in sighted individuals

In striking contrast to the clear evidence for specialized visual motion mechanisms, it is not yet clear whether or not genuine selectivity for auditory motion exists in the human brain.

In non-primate animal models, direction selectivity for auditory motion has been observed in inferior colliculus neurons, but this directional selectivity seems to reflect spatial adaptation rather than spatio-temporal tuning [50–52]. Within auditory cortex, the large majority of directionally selective neurons show separable spatio-temporal tuning (figure 1b, upper panel) [53–56], such that their maximal response would occur for a sound burst occurring at two discrete locations in space and time. In general, directionally selective neural responses in auditory cortex [57] have been shown to reflect a combination of a preference to the spatial location of sounds at the onset/offset, and binaural interactions [58]. For example, a neuron that shows selective responses to motion might have a strong preference for a sound onset in the contralateral ear and for a simultaneous decrease and increase in sound intensity at the contralateral and ipsilateral ears, respectively, rather than being tuned for motion per se.

Numerous studies point to PT as the cortical region that processes auditory motion in humans and primates. In primates, BOLD auditory responses to motion stimuli in this area cannot simply be predicted on the basis of a linear combination of location preference and spectrotemporal tuning [59], but it remains ambiguous whether this reflects separable or non-separable tuning. In humans, PT shows stronger responses to coherent auditory motion as compared to stationary stimuli appearing at one location in space [60–62], responses in this area can be used to classify direction of motion [63,64], and cortical damage in this region can lead to selective impairments in auditory direction perception [65]. However, whether the computation in PT is specialized for continuous auditory motion versus discrete spatial changes over time remains an open question. BOLD responses in PT are not statistically different in magnitude (and fail to show selective event-related adaptation) for genuine auditory motion versus stationary 1 s noise bursts that vary in location [66,67]. Similarly, although ERP data in humans show distinct neural signatures for moving versus stationary stimuli [68], the patterns are similar for smooth motion, random scattered changes in sound location and abrupt displacement to a contralateral location over time [69].

6. The perception of auditory motion in sighted individuals

Human behavioural data similarly suggest a heavy reliance on separable spatio-temporal tuning. A variety of psychophysical studies support a ‘snapshot’ model of auditory motion in which the perception of a moving sound is constructed from successive, temporally integrated frames [70–72]. Consistent with this model, it is easier to discriminate stimuli of similar speeds but different durations than it is to discriminate stimuli of similar durations but different speeds [73], suggesting that duration and distance cues dominate human auditory speed perception.

A direct measurement of perceptual auditory motion templates also suggests the use of separable filters [49]. Here, using a psychophysical reverse correlation paradigm directly analogous to that Neri [48] used to characterize visual motion templates (figure 2), participants were asked to discriminate left versus right auditory motion embedded in spatio-temporal noise (figure 3a). By contrast to the result for visual motion, measured auditory motion templates were space–time separable (figure 3b, left panel), with large weights for sound locations at the onset and offset.

Figure 3.

Auditory motion templates for sighted and early blind individuals estimated using a psychophysical reverse correlation technique that was a direct analogue to Neri [48]. (a) Early blind (n = 8) and age and musical experience matched sighted participants (n = 8) were asked to discriminate the direction of signal motion (left panel; broadband noise, 500–14 000 Hz, left versus right, leftward motion depicted). Spatio-temporal noise bursts (right panel; amplitudes drawn from a Gaussian distribution) was added to the moving signal. (b) Estimated perceptual auditory motion templates for sighted (left panel) and blind (right panel) individuals. Intensity in each cell in (a) represents sound amplitude (scale = [0 1], from black to white). Note that the temporal and spatial scales were designed to maximize the perception of audition motion [70] and are very different from those used to measure visual motion templates by Neri [48]. Figure from [49].

Why do the templates underlying auditory motion processing differ so substantially from those for visual motion, as measured using closely analogous paradigms? An ideal observer analysis, figure 4, suggests that spatio-temporally separable templates may be optimal, given the poor spatial resolution of the auditory input. When predicting performance for these analogous motion tasks using a variety of templates, a narrowly tuned non-separable template (light grey curve), which resembles classic visual motion detectors, predicted the best performance (lowest signal amplitude threshold), especially at low internal noise levels. However, if one assumes that the spatial tuning of the filter is broader than the width of the stimulus or high levels of internal noise, then the separable template (green curve) performs better than all non-separable templates with broader spatial tuning (darker grey curves). Thus, given the poor spatial selectivity of auditory signals, the optimal solution for auditory motion discrimination may be a spatio-temporally separable template.

Figure 4.

An ideal observer analysis, simulating performance in a task analogous to Neri [48] using a set of non-separable templates with varying tuning width (grey curves) and a single separable template (green curve). On each trial, the stimulus (signal + random external spatio-temporal noise, as described in the main text) was passed through each template. Internal noise was added to the template response. The model assumes a correct response when the appropriately oriented template (e.g. leftward) had a larger response than the opposite (e.g. rightward) tuned template. Signal amplitude was varied to find the predicted perceptual threshold as a function of internal noise. The separable template (green curve) performed better than broadly tuned non-separable models (darker grey) and comparably to the most narrowly tuned non-separable template (light grey) except for very low levels of internal noise.

7. The neural basis of auditory motion processing in early blind individuals

Numerous neuroimaging studies have shown selectivity for auditory motion in hMT+ as a result of early blindness [74–77]. One of the earlier studies, Saenz et al. [78] compared the location of auditory motion responses in typically sighted and two sight recovery participants (who lacked vision for most of their life, and had some vision restored in adulthood, figure 5). Both sight recovery individuals showed overlapping visual and auditory motion responses in hMT+, suggesting that auditory cross-modal responses coexisted with regained visual responses within the visual cortex. These auditory responses in hMT+ were specific for motion; the region lacked sensitivity to other complex auditory stimuli, including frequency sweeps and forward versus reversed speech. This finding has been replicated in numerous studies in early blind individuals [75–77,79–81]. This recruitment of hMT+ seems to require blindness during development; directional tuning has been observed in both early blind and sight recovery individuals, but not in late blind or typically sighted participants [76,81].

Figure 5.

Surface maps of auditory motion responses and hMT+ in two sight recovery (SR) participants. Auditory motion was generated by manipulating interaural level differences. Yellow regions responded more to moving versus stationary auditory white noise. Green and blue regions show hMT+ location (green, hMT+ overlapped by auditory motion responses; blue, hMT+ not overlapped by auditory motion responses). Note the near-complete overlap (very little blue), indicating co-localization of auditory responses with their visually defined hMT+. Figure from [78].

It has been suggested that hMT+ may be inherently multisensory, capable of representing auditory as well as visual motion signals [82]. However, auditory motion does not modulate spiking responses in marmoset MT/MST [83], and the majority of studies have failed to find auditory BOLD responses within individually localized hMT+ in sighted individuals [64,75,84] (although auditory motion responses are found in a nearby region [75]). One recent study did show successful classification of up–down versus left–right auditory motion in hMT+ in sighted individuals [85]. However, in the same dataset, neither up-versus-down nor left-versus-right motion could be classified. One concern is that up–down (performance between 74% and 78%) was much harder than left–right (96%) discrimination, so successful classification of horizontal versus vertical motion directions could have been driven by arousal mediated by task difficulty. Another possibility is that successful classification was driven by differences in the spatial spread of cross-modal spatial attention as the participant switched their attention between vertical versus horizontal motion trajectories [86].

Curiously, the recruitment of hMT+ for auditory motion processing in early blind individuals seems to be accompanied by a loss of selectivity for auditory motion [64], as demonstrated by a reduced ability to classify motion direction based on the pattern of BOLD responses in right PT [75,87]. Almost all examples of cortical sensory plasticity in the current literature, whether at the cellular (e.g. [88,89]) or the map (e.g. [90–92]) level can be interpreted as being the result of competition between inputs. For example, competition between inputs can explain responses to auditory motion within hMT+ in early blind individuals [75,79,81]—auditory inputs to hMT+ (whether top-down or bottom-up) benefit from a lack of competition with visual input. However, competition between inputs cannot explain the loss of selectivity in right PT, whose typical auditory input remains unperturbed. This finding suggests that competition between cortical areas may also play a significant role in developmental cortical specification: with the recruitment of hMT+ competitively reducing the role of right PT in auditory motion processing (figure 6).

Figure 6.

In early blind individuals the altered cortical locus of auditory motion processing seems to be driven both by competition between inputs and by competition between areas for cortical role.

8. The perception of auditory motion in early blind individuals

The effects of blindness on auditory motion perception are still not entirely clear. Early blind individuals perform worse than sighted individuals in a speed judgement task [93]; however, they outperform sighted individuals on a simple auditory motion discrimination task—the minimum detectable audible movement angle [94]. Early blind individuals also show an enhanced ability to understand complex auditory motion under tasks designed to mimic more ecologically valid circumstances. For example, in a complex auditory motion task involving integration of both frequency and motion information, designed to replicate the experience of crossing a road at a busy intersection, early blind individuals outperformed their sighted counterparts when asked to report the overall direction of perceived motion [79] (figure 7).

Figure 7.

(a) Stimuli consisted of eight ‘noise bands’. Each of the eight noise bands was a different frequency (generated by filtering white noise in the Fourier domain). For each noise band we simulated motion from right to left or vice versa, with motion simulated using naturalistic interaural time differences, interaural level differences and Doppler shift. The frequency range of each noise band remained constant as it travelled from left to right (or vice versa), apart from the frequency shift induced by simulating the Doppler effect. (b) For the data shown here six noise bands travelled from left to right, and two travelled from right to left. Participants were asked to report the overall direction of perceived motion. (c) Early blind participants are significantly better at determining the direction of auditory motion. Replotted from Jiang et al. [79,87]. (Online version in colour.)

These mixed findings for the effects of blindness on auditory motion processing resemble differences between early blind and sighted individuals in spatial localization. Blind individuals outperform sighted individuals when judging the relative location of two sounds sources [95,96], but are less accurate at auditory spatial bisection [97]. One possibility is that the ability to encode auditory spatio-temporal cues is enhanced in blind individuals, enhancing performance on some spatial and motion tasks [79,94], but 3D spatial calibration is impaired in the absence of vision [98,99], resulting in poorer performance in spatial and motion tasks that require a precise encoding of Euclidean auditory space [94,97].

Given the known spatio-temporal selectivity for processing visual motion in hMT+ discussed above, one intriguing possibility is that enhanced abilities for perceiving auditory motion observed in early blind individuals for some tasks might be the result of recruiting non-separable spatio-temporal mechanisms innate to hMT+. However, the perceptual templates estimated for early blind individuals using the analogue of Neri [48] are also spatio-temporally separable [49], showing only subtle differences from the templates observed in sighted individuals (figure 3b, right panel). This result suggests that if auditory motion is processed in hMT+ in early blind individuals, then this must result in a shift from non-separable to separable templates within this region—a significant modification of hMT+ normal computational operations.

One possibility is that the signals within MT are not motion signals per se, but reflect spatial and temporal signals from either subcortical (perhaps recruited superior colliculus [100]), or cortical [101] areas. If so, the ability to classify direction of auditory motion in hMT+ in early blind individuals might be mediated by its retinotopic organization (if, for example, neural responses were biased towards the sound onset) rather than by its spatio-temporal tuning. It remains possible that these signals in hMT+ nonetheless play an important role both in mediating the perceptual experience of motion and for propagating motion information to other areas in the brain. hMT+ has reciprocal projections [102] to a variety of sensorimotor areas including parietal V6 and V6A (object motion recognition and control of reach-to-grasp movements [103,104]), intraparietal AIP (visuo-motor transformations for grasp [105]), MIP (coordination of hand movements and visual targets [106]), LIP (saccadic target selection) and frontal A4ab (motor cortex), prefrontal A8aV (frontal eye fields) and A8C (premotor).

Given that it is not possible to classify motion direction based on the pattern of BOLD responses in PT in early blind individuals [75,87], it seems unlikely that the signals in hMT+ come from PT. Indeed, the loss of motion selectivity in PT as a result of early blindness seems to have no discernable perceptual consequence. This suggests that simple motion discrimination may be mediated subcortically; PT's responses to auditory motion in sighted individuals may not reflect the computation of auditory motion per se, but rather functions analogous to higher level areas of hMT+ [107,108], including the segregation, identification and direction of attention to moving objects [109,110].

9. How does our qualitative experience of motion arise?

Over a wide range of conditions we perceive visual signals as motion (rather than changes in space and time), even when the underlying stimulus consists of discrete signals [111]. This likely reflects both physiological constraints (such as a lack of sensitivity to high spatial frequencies) within the computational machinery used to compute motion within MT [112], as well as an ecological prior that objects move continuously in space. In either case, hMT+ seems to play a crucial role, with damage to this area causing ‘stroboscopic’ vision [43,44]. This ecological prior seems not to require statistical experience with visual motion. Sight recovery participant MM, who became blind at 3.5 years, had no visual memories and had sight restored in his forties. However, after sight recovery he seemed to have no difficulty understanding visual motion, with no evidence of ‘stroboscopic’ motion perception, in striking contrast to his deeply and permanently impaired 3D form processing [113,114]. He did not subjectively report his first experience of visual motion as a ‘novel sensation of motion’ (as he did for colour) and he could perform tasks like biological motion recognition, which relies heavily on continuous motion perception. Similar immediate fluency with visual motion is also reported in other sight recovery studies, some involving congenitally blind individuals [115,116].

In the auditory domain, for both sighed and early blind individuals, there is a similarly strong bias towards interpreting auditory signals that change in space and time as representing motion. It seems unlikely that this prior is generated via a ‘bottom up’ interpretation of experienced auditory signals, given that audition rarely provides continuous motion information. In sighted individuals visual information might ‘scaffold’ the interpretation of auditory signals, but this does not explain why early blind individuals also have an understanding of motion. One possibility is that signals from hMT+ may be ‘innately’ interpreted as motion by other regions of the brain, so blind individuals' understanding of motion may be the result of the propagation of signals from hMT+. Another possibility is that, even in the absence of vision, an understanding of motion can be mediated by higher level amodal knowledge about object permanence.

10. Conclusion

It is tempting to think of auditory and visual motion as being direct analogues of each other—representing corresponding information about the movement of objects in space, albeit in different modalities. Another perspective, however, is to appreciate that both auditory and visual motion signals represent the movement of objects in the world, but accept that they are deeply incommensurate in terms of the information that is carried about the 3D world, the quality of that information, and how this information is represented neurally. Visual information, in the absence of occlusion, is continuous, whereas auditory information is sporadic. The quality of spatial information also differs substantially across the two modalities. Visual motion is directly encoded by the changes in firing over time in retinal cells with remarkable spatial precision [8], and is carried by ‘labelled lines’ through most of the early visual hierarchy [13,14]. By contrast, sound location must be inferred through a combination of indirect cues [1]. It seems plausible that the difference in how auditory and visual motion is represented neurally is driven by differences in the nature of the input.

In early blind individuals the templates underlying motion discrimination are very similar to those of sighted individuals [49]. This similarity is somewhat surprising, given the significant neural reorganization that occurs as a result of early blindness, wherein hMT+ is recruited for auditory motion [75–81], and there seems to be a reduced selectivity for motion in right PT [75,79]. One possibility is that auditory motion is processed within hMT+ in early blind individuals, there is a substantial change in hMT+ tuning, from non-separable to separable, and the similarities between early blind and sighted templates simply reflect homologies of function—driven by the statistics of auditory input. A second possibility is that the templates underlying auditory motion perception reflect separable computations either prior or feedback to hMT+. If so, the recruitment of hMT+ may reflect its selectivity for retinotopic location rather than its neural spatio-temporal tuning. The recruitment of hMT+ may still be functionally important, given that this area provides the motion information necessary for navigating and interacting with the 3D world to numerous cortical areas [102]. The recruitment of hMT+ may also enhance the perceptual experience of continuous motion from discontinuous auditory input.

In conclusion, blind individuals make their way through the world with limited and noisy information, using neural motion representations that are heavily constrained by limitations in the sensory input. Navigating our way through a 3D world is an impressive feat of inference for sighted individuals. In the case of blind individuals, it represents a genuine triumph of cortical plasticity that is only just beginning to be understood.

Ethics

Previously unpublished data included in this article were approved by the University of Washington's Institutional Review Board, and was carried out in accordance with the Code of Ethics of the Declaration of Helsinki.

Data accessibility

Data and code used in generating figures 1, 3, and 4 are provided at https://github.com/VisCog.

Authors' contributions

I.F.: conceptualization, funding acquisition, writing—original draft, writing—review and editing; W.J.P.: conceptualization, writing—original draft, writing—review and editing.

All authors gave final approval for publication and agreed to be held accountable for the work performed therein.

Conflict of interest declaration

We declare we have no competing interests.

Funding

NIH National Eye Institute Grant R01EY014645 (to I.F.) and a Weill Neurohub Postdoctoral Fellowship (to W.J.P.).