Using superior colliculus principles of multisensory integration to reverse hemianopia

Abstract

The diversity of our senses conveys many advantages; it enables them to compensate for one another when needed, and the information they provide about a common event can be integrated to facilitate its processing and, ultimately, adaptive responses. These cooperative interactions are produced by multisensory neurons. A well-studied model in this context is the multisensory neuron in the output layers of the superior colliculus (SC). These neurons integrate and amplify their cross-modal (e.g., visual-auditory) inputs, thereby enhancing the physiological salience of the initiating event and the probability that it will elicit SC-mediated detection, localization, and orientation behavior. Repeated experience with the same visual-auditory stimulus can also increase the neuron’s sensitivity to these individual inputs. This observation raised the possibility that such plasticity could be engaged to restore visual responsiveness when compromised. For example, unilateral lesions of visual cortex compromise the visual responsiveness of neurons in the multisensory output layers of the ipsilesional SC and produces profound contralesional blindness (hemianopia). The possibility that multisensory plasticity could restore the visual responses of these neurons, and reverse blindness, was tested in the cat model of hemianopia. Hemianopic subjects were repeatedly presented with spatiotemporally congruent visual-auditory stimulus pairs in the blinded hemifield on a daily or weekly basis. After several weeks of this multisensory exposure paradigm, visual responsiveness was restored in SC neurons and behavioral responses were elicited by visual stimuli in the previously blind hemifield. The constraints on the effectiveness of this procedure proved to be the same as those constraining SC multisensory plasticity: whereas repetitions of a congruent visual-auditory stimulus was highly effective, neither exposure to its individual component stimuli, nor to these stimuli in non-congruent configurations was effective. The restored visual responsiveness proved to be robust, highly competitive with that in the intact hemifield, and sufficient to support visual discrimination.

1. Introduction

Each of our senses processes a different type of energy, and each can function independently of the others. But, because important events are often registered by more than one sense, the senses often operate in concert, and do so cooperatively. Because the brain has the capability to synthesize the information provided by its different senses, perceptual and behavioral performance can be enhanced far beyond that which would be expected by the sum of their individual action (see reviews in Stein and Meredith, 1993; Naumer and Kaiser, 2010; Murray and Wallace, 2012; Stein, 2012). The synthetic process is known as “multisensory integration,” and has a profound impact on our daily lives. It is commonly engaged in a variety of functional domains, from increasing our ability to detect and localize stimuli to understanding the speech of a friend in a noisy airport (Stein et al., 1996; Ernst and Banks, 2002; Lovelace et al., 2003; Alais and Burr, 2004; Shams et al., 2005; Ross et al., 2007; Jaekl and Harris, 2009).

The process depends, in large part, on the action of individual multisensory neurons that are able to transform their different sensory inputs into an integrated multisensory product. Multisensory neurons are widely distributed in the nervous system and are embedded in a variety of diverse circuits which serve different functional roles. They are even found within structures that are generally identified as “unisensory” (Meredith and Stein, 1983; King and Palmer, 1985; Meredith et al., 1987; Binns and Salt, 1996; Wallace and Stein, 1996; Bell et al., 2001; Wallace et al., 2004; Ghazanfar and Schroeder, 2006; Jain and Shore, 2006; Nagy et al., 2006; Avillac et al., 2007; Bizley et al., 2007; Lakatos et al., 2007; Romanski, 2007; Winkowski and Knudsen, 2007; Bizley and King, 2008; Reches and Gutfreund, 2009; Zahar et al., 2009; Fetsch et al., 2012; Lippert et al., 2013; Reig and Silberberg, 2014; Ishikawa et al., 2015; Costa et al., 2016; Felch et al., 2016; Kardamakis et al., 2016; Bieler et al., 2017; Truszkowski et al., 2017).

However, the functional properties of multisensory neurons have been most extensively studied in the cat superior colliculus (SC) (Meredith and Stein, 1983, 1986a; Stein et al., 1983, 1986b; Wallace et al., 1993, 1998; Rowland et al., 2007b; Alvarado et al., 2009; Pluta et al., 2011; Miller et al., 2017). This midbrain structure receives visual, auditory, somatosensory inputs, and is involved in detecting, locating, and orienting to external events (Stein and Clamann, 1981; McHaffie and Stein, 1982; Benedetti, 1995; Burnett et al., 2004, 2007). The integrated multisensory responses of these neurons can be significantly greater than expected based on responses to modality-specific stimuli (Stanford et al., 2005; Rowland et al., 2007b; Miller et al., 2015, 2017). Furthermore, repeatedly presenting a neuron with the same cross-modal (e.g., visual-auditory) stimulus complex enhances its sensitivity to it, and to its individual modality-specific component stimuli (Yu et al., 2009, 2013a). This has obvious translational applications for circumstances in which the effectiveness of one modality has been compromised by a neurological disorder or brain injury. Hemianopia is one such disorder.

Large lesions of visual cortex on one side of the brain induce blindness in contralesional space (hemianopia) (Zhang et al., 2006; Goodwin, 2014; Frolov et al., 2017). Although the SC is far from the site of the physical damage, its multisensory neurons lose their visual responsiveness despite the presence of visual tectopetal afferents from other sources. Apparently, the effectiveness of these remaining visual afferents is compromised by the cortical lesion (Jiang et al., 2015). Because repeated visual-auditory stimulation enhances the visual responsiveness of multisensory SC neurons in normal animals, there was reason to suspect that it might also restore visual responsiveness in the SC of hemianopic animals and thereby restore some contralesional visual function. Indeed, it did so (Jiang et al., 2015). While the specific mechanisms of this recovery remain unknown, the constraints on the effectiveness of the technique appear to be the same as those governing multisensory integration, which is unlikely to be coincidental (Dakos et al., 2019b, 2019a). Thus, understanding how the process of multisensory integration operates is not only of significant value in understanding normal brain function, but also in developing more effective therapeutic strategies to mitigate the effects of brain damage.

2. SC multisensory integration

As stated earlier, the SC is a midbrain structure involved in detection, localization and orientation functions (Fig. 1) (Jay and Sparks, 1987a, 1987b; Stein et al., 1989; Munoz and Wurtz, 1995a, 1995b; Groh and Sparks, 1996a, 1996b; Harrington and Peck, 1998; Lomber et al., 2001; Burnett et al., 2004, 2007; Rowland et al., 2007a). Its intermediate and deeper layers are well-endowed with multisensory neurons responsive to visual, auditory and somatosensory stimuli individually and in various combinations. Each of these sensory representations is organized in map-like (topographic) fashion and the various maps are in spatial register (Stein et al., 1975; Middlebrooks and Knudsen, 1984; Meredith and Stein, 1990; Meredith et al., 1991; Benedetti, 1995). This is most obvious in the case of the visual and auditory modalities, which both deal with stimuli in extra-personal space, but extends to the somatosensory map as well (Stein et al., 1975). Neurons in different regions of the SC represent different regions of space regardless of sensory modality. Neurons in the rostral SC represent forward space (i.e., face, and central visual and auditory space), whereas rearward body parts and peripheral visual and auditory space are represented by neurons in the caudal SC. Upper sensory space is represented in the medial SC, and lower space is represented in the lateral SC. These maps are formed by the receptive fields (RFs) of individual neurons, and the different RFs of a multisensory neuron overlap one another (Gordon, 1973; Clemo and Stein, 1984; Meredith and Stein, 1990; King et al., 1998; Kadunce et al., 2001). Although discussed here are data obtained primarily from the cat SC, its organization is reflective of a general mammalian scheme (see Stein and Stanford, 2008).

Fig. 1.

Operational schematic of the SC and its behavioral roles. Left: The SC (illustrated here in a cut-away view of a cat brain) mediates detection, localization, and orientation to salient environmental events. Its intermediate and deep layers contain overlapping topographic maps of somatosensory, auditory, and visual “space.” An external event registered by more than one of these senses will generate cross-modal signals that converge onto common multisensory neurons whose receptive fields include the event’s location. Depicted here is an illustrative visual-auditory neuron (stars on the maps show its location in the SC and its receptive fields are plotted on polar map of space). Right: A sensitive behavioral assay of SC multisensory integration involves detecting and orienting towards brief low-intensity visual (V) and auditory (A) stimuli. When the stimuli are presented in spatiotemporal concordance (VA) as in a perimetry device shown here, the accuracy in detecting/localizing them is significantly enhanced. % enhancement at each location tested in an exemplar animal is shown on bar graphs above the perimetry device.

As would be expected in such a scheme, the inputs from cross-modal stimuli that are in spatial and temporal concordance (typical of stimuli derived from the same event) converge onto common multisensory SC neurons. In the neurotypic adult, SC multisensory neurons transform these unisensory signals into a synthesized multisensory product in which physiological responses are faster, more reliable, and more robust than those elicited by either individual stimulus (Corneil and Munoz, 1996; Goldring et al., 1996; e.g., Bell et al., 2001; e.g., Rowland et al., 2007b). Cross-modal stimuli that are non-congruent either elicit weaker responses or no interaction (Meredith and Stein 1986a,b; see also Perrault et al., 2005; Stanford et al., 2005).

These physiological principles are predictive of corresponding SC-mediated behaviors: congruent cross-modal stimuli elicit contralateral detection/orientation responses that have lower thresholds, are faster, more reliable, and more accurate than responses to the individual component stimuli (e.g., Stein et al., 1989; Wilkinson et al., 1996; Burnett et al., 2004; Gingras et al., 2009). That there are parallels in multisensory physiology and behavior is not surprising: deep layer SC neurons send projections to structures in the brainstem and spinal cord that directly mediate the motor components of these behavioral responses (Sprague and Meikle, 1965; Stein et al., 1976, 1989; Stein and Clamann, 1981; Munoz et al., 1991; Pare et al., 1994 ; Lomber et al., 2001; Burnett et al., 2004; Guillaume and Pelisson, 2006; Rowland et al., 2007a; Gingras et al., 2009). More recently activity the SC has also been linked to spatial attention and perceptual awareness (Krauzlis et al., 2013, 2018), and other studies have provided evidence linking multisensory enhancement in these behaviors to the SC in humans (Calvert et al., 2001; Fort et al., 2002; Bertini et al., 2008; Leo et al., 2008; Maravita et al., 2008).

3. SC multisensory plasticity

Despite the impact of these multisensory physiological and behavioral effects, and the obvious importance for normal behavior, multisensory enhancement is not an inherent characteristic of the SC circuit. Animals lacking visual-auditory experience (reared in the dark, reared with masking noise, or reared with alternating visual and auditory stimuli) do develop visual-auditory neurons, but these neurons cannot synthesize their cross-modal inputs to enhance their responses (Fig. 2) (Wallace and Stein, 1997, 2001, 2007; see also Carriere et al., 2007; Xu et al., 2012, 2014, 2015; Yu et al., 2013, 2019). Rather, their visual-auditory multisensory responses are no more robust than the most effective of its individual unisensory responses, and in some cases are weaker, reflecting a default intersensory competition (Yu et al., 2019). During normal development, as the circuit gathers congruent experience with cross-modal stimuli, it crafts its mature functional configuration, one which is capable of integrating those stimuli to enhance physiological and behavioral performance (Stein et al., 2014).

Fig. 2.

Multisensory enhancement requires experience with cross-modal stimuli. Depicted are results from exemplar neurons tested with visual (V), auditory (A), and spatiotemporally concordant visual-auditory (VA) stimuli after rearing animals in one of four conditions. Impulse rasters indicate the responses to each stimulus and stimulus combination (overlying traces indicate stimulus timing), each dot represents an impulse. Summary histograms on the right indicate average response magnitudes and the proportionate multisensory enhancement (ME). The first row of rasters and summary bar histograms show the responses of a multisensory SC neuron in a normally-reared animal. Note that the VA response was enhanced. The second, third and fourth rows depict neuronal responses of animals reared in darkness, omnidirectional sound, or with random appearing visual or auditory cues. In each of these conditions, the absence of multisensory experience resulted in the absence of SC neurons capable of multisensory enhancement. Components adapted from (Yu et al., 2010; Xu et al., 2012, 2014).

However, this developmental process is not limited to early life. SC neurons restricted of multisensory experience in early life are able to develop multisensory integration capabilities if given appropriate experience as adult (e.g., Yu et al., 2010; Xu et al., 2014; see Stein et al., 2014 for review). This can be accomplished by a multisensory exposure protocol in which they are repeatedly exposed to invariant, spatiotemporally congruent, cross-modal (e.g., visual-auditory) stimulus pairs over the course of minutes to hours. The exposure protocol does not require training with explicit rewards or even an alert brain: it is effective even when the animal is anesthetized during the exposure period. But, the protocol is ineffective if it involves only a modality-specific component stimulus. Placing the animals in the normal environment for the period required for normal multisensory development (i.e., 6 months) is also ineffective (Xu et al., 2017). Apparently, cross-modal events that are encountered in natural environments have far less impact on the adult circuit.

Both multisensory (but not unisensory) development and multisensory-induced plasticity are dependent on the functional integrity of a specific source of tectopetal afferents: those from the anterior ectosylvian sulcus (AES). This region has three adjacent subregions that are primarily unisensory: a visual area (AEV), an auditory area (FAES), and a somatosensory area (SIV) (Mucke et al., 1982; Stein et al., 1983; Clemo and Stein, 1984, 1986; Norita et al., 1986; Meredith and Clemo, 1989; Wallace and Stein, 1994; Scannell et al., 1996; Harting et al., 1997; Jiang et al., 2001). Although the AES contains many multisensory neurons, especially in the transition zones between its subregions, the neurons that project to the SC are unisensory (Wallace et al., 1993). These unisensory AES efferents converge on the same SC neurons, matching the convergence patterns of its inputs from other, primarily ascending, sources. Thus, visual-auditory neurons receive two sets of converging unisensory afferents, one from AEV and FAES, and one from non-AES sources. The AES inputs are special, and are critical for SC neurons to exhibit multisensory enhancement and to support enhanced performance on SC-mediated behavioral tasks (Wallace and Stein, 1994; Jiang et al., 2001, 2002; Stein et al., 2002; Alvarado et al., 2007b, 2007a, 2009). Ablation of AES in the neonate or temporarily deactivating AES while multisensory experience is acquired (in the naïve adult or neonate) precludes the development of multisensory integration capabilities (Jiang et al., 2006; Yu et al., 2013; Rowland et al., 2014). This is, perhaps, not surprising. Ablation or deactivation of AES in the neurotypic adult also eliminates multisensory enhancement (Jiang et al., 2001, 2002; Alvarado et al., 2007a, 2009). These observations suggest that AES is the portal by which multisensory experience impacts the SC in ways that allow it to craft and maintain its multisensory integration capabilities (see Stein et al., 2014). Presumably, it does so via a type of Hebbian-based learning in SC neurons, whereby the impact of the cross-modal inputs from AES are enhanced by their repeated co-occurrence and activation of their common target neuron (see (Cuppini et al., 2011, 2012) see also (Bi and Poo, 2001).

Hebbian principles have been shown to be applicable to other examples of SC multisensory plasticity. For example, if a visual and an auditory cue are repeatedly presented in tandem, but on the margin of a neuron’s temporal window for integrating cross-modal cues, the response to the first stimulus increases in magnitude and duration (Yu et al., 2009). This is consistent with the effects of a temporally asymmetric Hebbian learning rule. Furthermore, as noted above, when congruent visual-auditory stimuli are presented together in a multisensory exposure protocol, the sensitivity of the responses to both individual stimuli increases within minutes (Fig. 3) (Yu et al., 2013a). The effect is not specific to the particular stimuli being presented, but to the sensory channel. Hence, a variety of visual stimuli would now be more effective. The most striking demonstration of this change in unisensory sensitivity is in the case of s0-called “covert” multisensory neurons. These are neurons in which overt responses (neuronal discharges) are elicited by stimuli in only one modality, but the magnitude of those responses can be modulated by stimuli from another modality. The multisensory exposure protocol enhances the sensitivity of the neuron to its covert input, rendering it effective in eliciting overt responses (Yu et al., 2013a). Repeated exposure to modality-specific stimuli, or cross-modal stimuli that are at disparate spatial locations, does not produce this effect (see Fig. 5) (see Fig. 4).

Fig. 3.

SC multisensory plasticity in the neurotypic adult. A: Impulse rasters depicting multisensory (left), visual (middle), and auditory (right) responses obtained before (top row) and after (bottom row) this visual-auditory neuron was given a multisensory exposure protocol (middle row). All response magnitudes were enhanced. B, C: Scatter plots show unisensory response magnitudes before (x-axis) and after (y-axis) the multisensory exposure protocol for the population. Note the positive deviation from the line of unity for both visual (B) and auditory (C) response. Insets indicate the average magnitudes across neurons. Adapted from (Yu et al., 2013a).

Fig. 5.

Visual orientation capabilities of 4 hemianopic animals. Each circle in the polar plot represents 10% correct responses, and visual orientation and approach performance are shown in green. A: A normal animal at the start of testing, after the cortical lesion, and after rehabilitation. Note that the normal visual field (left schematic) was compromised by a right cortical lesion and vision was lost in the left hemifield (middle), but recovered after auditory- visual (AV) training (right). B: Training with auditory cues alone failed to produce this result, but subsequent AV training reinstated vision. C: The reinstated vision was lost following removal of ipsilesional AES cortex. Adapted from (Jiang et al., 2015). (For interpretation of the references to colour in this figure legend, the reader is referred to the Web version of this article.)

Fig. 4.

Representative lesion producing hemianopia. Labeled areas: LAT = lateral sulcus, SUPS = suprasylvian sulcus, PECS = posterior ectosylvian sulcus, LGN = lateral geniculate, SC = superior colliculus). Gray regions show areas that have been removed on the left side of the brain, resulting in a complete blindness on the right side of space. Adapted from (Dakos et al., 2019).

4. Implications for hemianopia

These observations suggested that a multisensory exposure protocol in the hemianopic animal might not only restore the capacity for SC visual responses that were lost, but by doing so also restore some contralesional visual capabilities. They also suggested that the operational features of SC multisensory integration would predict the constraints of the rehabilitative paradigm. This issue was then explored as described below.

However, it is first important to note that there is a puzzling aspect to the permanent hemianopia induced by lesions of visual cortex (Fig. 4; see also Sprague, 1966a,b; Sherman, 1974; Wallace et al., 1990; Zihl, 1995; Romano, 2009; Das and Huxlin, 2010). A good deal of the visual brain is spared by the lesion, in particular, the circuitry of the SC. Although the SC has lost inputs from visual cortex, and these are important for some of the selective responses of SC neurons (see Ogasawara et al., 1984), it retains significant visual inputs from a number of other structures. These include retina and thalamus (see Edwards et al., 1979; Berson and McIlwain, 1982; Huerta and Harting, 1982; Rhoades et al., 1989; May, 2006). Yet these inputs appear to be incapable of supporting SC-mediated visually-guided behaviors in contralesional space.

Recently, recordings from SC neurons of hemianopic animals provided some clarity on this issue by examining the physiological effect of the lesion. It was quite selective, compromising the visual responses of neurons in the multisensory layers of the structure where visuomotor transformations take place, but not the responsiveness of neurons in its purely visual superficial layers (albeit, some of the selective properties of these neurons were likely to have been altered by visual cortex lesions – see Ogasawara et al., 1984). The loss of responsiveness in SC neurons after visual cortex lesions is thought to result from an alteration in the balance between the excitatory visual inputs that remain and the inhibitory inputs coming from the opposite side of the brain via the intercollicular commissure (e.g., Sprague, 1966a,b; Sherman, 1974; Wallace et al., 1989, Wallace et al., 1990; Durmer and Rosenquist, 2001). This effect was not only layer-specific, but modality-specific. The auditory and somatosensory responses of SC neurons in the multisensory layers were retained (Jiang et al., 2015). Many of them were likely visually responsive multisensory neurons prior to the lesion, and now appeared to be unisensory (auditory or somatosensory), or bisensory (auditory-somatosensory).

4.1. Using a multisensory exposure protocol to rehabilitate vision

It is in this context that the multisensory exposure paradigm, so effective in changing the physiological responses of SC neurons, becomes of translational interest. A series of studies in humans and cats provided support for the ability of this simple, non-invasive multisensory exposure paradigm to reverse hemianopia (Bolognini et al., 2005; Frassinetti et al., 2005; Làdavas, 2008 ; Passamonti et al., 2009; Dundon et al., 2015a, 2015b; Jiang et al., 2015; Purpura et al., 2017; Tinelli et al., 2017).

In this paradigm, hemianopic subjects are repeatedly exposed to spatiotemporally congruent visual-auditory pairs in the contralesional (i.e., hemianopic) field, at either a fixed location or randomized locations. Visual, auditory, and visual-auditory probe trials are conducted in both ipsilesional and contralesional space to assess overall sensory responsiveness and track the restoration of visual responsiveness. Each trial begins by having subjects fixate on a central point within a perimetry apparatus with light emitting diodes (LEDs) and speakers at each 15 of eccentricity (see Fig. 1). They are instructed or trained to make saccades or overt orientation responses to visual stimuli wherever they appear. There are several differences in the paradigms that have been used for cats and humans (discussed below); however, the rehabilitative results are generally consistent.

This procedure works within weeks and persists long after exposure (“training”) ceases (see also Bolognini et al., 2005). The return of visuomotor behavior is paralleled by physiological changes in the multisensory layers of the SC (Fig. 5; Jiang et al., 2015). The exposure paradigm is believed to be effective because the repeated bouts of convergent signals it provides to SC neurons engage Hebbian-like mechanisms to potentiate their individual modality-specific channels. This makes multisensory neurons more likely to respond to their visual inputs. Such a mechanism of action makes several predictions regarding the neural targets of the exposure-induced effect and constraints that would be imposed on the rehabilitation paradigm. First, multisensory neurons, but not neighboring unisensory neurons, should have their visual responses restored, since only the former have the potential to engage multisensory plasticity. Second, behavioral recovery should depend on the same underlying circuit (e.g., AES-SC influences) required for multisensory plasticity in these neurons. Third, effective rehabilitation should require the same stimulus configurations that generate enhanced multisensory responses in these neurons.

Evidence for the first prediction comes from the finding that the vast majority of SC neurons that became visually-responsive after the multisensory exposure paradigm were demonstrably visual-auditory (Jiang et al., 2015). This proportion of such neurons proved to be higher than that found in the normal SC, presumably because the visual channels of many normally covert multisensory neurons had become overt. Second, as predicted, behavioral rehabilitation required an active AES. Deactivating AES reinstated hemianopia in recovered animals (Fig. 5; Jiang et al., 2015). Visually-responsive SC neurons in recovered animals were also more sensitive to the AES input than is typical (Fig. 6). Evidence for the third prediction comes from a recent study in which hemianopic cats were exposed to different visual-auditory configurations. This requires some explanation (see Fig. 8) (see Fig. 7).

Fig. 6.

Deactivation of AES cortex modulates the visual responsiveness of deep layer SC neurons in rehabilitated animals. Depicted are the responses of two deep layer SC neurons (columns) on the ipsilesional side in an animal rehabilitated after unilateral lesion of visual cortex. The top row depicts an impulse raster of the response of each neuron to a moving bar of light (ramp). The middle row indicates the response of each neuron to the same stimulus after ipsilateral AES has been deactivated: note the decrease in magnitude. The bottom row illustrates the responses after AES cortex has been reactivated. From Jiang et al., 2015.

Fig. 8.

Visual choice preference in rehabilitated animals. Each animal (F1–F3) received cortical lesions resulting in hemianopia followed by a multisensory exposure protocol with congruent VA stimuli that restored vision. In this illustration, the formerly compromised side of space is drawn on the right. Top row: When two equally-rewarded and equally-intense visual stimuli are presented in opposite hemifields, two of the animals more often respond to the stimulus in the formerly-blind hemifield, while only one prefers the hemifield that was always intact. Percentage and arrow thickness indicate the proportion of responses to each stimulus. Middle row: This preference was sensitive to stimulus intensity, and the choice preference could be eliminated by lowering the intensity of the preferred stimulus. Bottom row: Choice was re-instated when intensity was again equilibrated, but at the lower level. This indicates that the animals’ preference was for the stimulus, not simply the side, and existed at multiple levels of intensity. Thus, visual processing in the formerly-compromised hemifield is robust after rehabilitation, and resilient even to competition from other stimuli.

Fig. 7.

Multisensory rehabilitation of hemianopia: the impact of cross-modal congruence. Top row (pre lesion): Performance of three animals (F1–F3) in detecting and localizing visual stimuli at all tested locations in the central 90° of space was near-perfect. There was also near-perfect performance on No-Go (NG) trials. Second row (post lesion): After removing all contiguous areas of left visual cortex animals were hemianopic in the contralesional (right) hemifield. Near-perfect performance in localizing visual stimuli in ipsilesional space, and on No-Go trials, was retained. Third row: Following 4 weeks of daily exposure to non-congruent (spatially or temporally disparate) visual-auditory stimuli visual performance was unchanged. Fourth row: After approximately two weeks of daily exposure to congruent visual-auditory stimuli, the hemianopia was resolved in all animals, and they detected and localized visual stimuli on both sides of space with near-perfect accuracy. Adapted from (Dakos et al., 2019).

4.2. Violating the principles of multisensory integration in the rehabilitation paradigm

As described above, each animal was trained to stand at the “go” point and fixate directly ahead at the LED illuminated at 0°. When the fixation light was turned off, another LED (the target stimulus) was illuminated. The cat was required to orient to and approach that target stimulus. Once the animal learned the task, all contiguous regions of visual cortex were ablated in one hemisphere. This resulted in massive degeneration of the ipsilesional lateral geniculate nucleus (LGN) and a profound neglect of all visual stimuli in the contralesional hemifield. This was immediately obvious in casual observation, as animals would ignore hand movements, threatening gestures, even food held out by the experimenter in contralesional space. Consistent with degeneration of the LGN, they did not show any evidence of “blindsight” in the contralesional hemifield (Cowey and Stoerig, 1995; Cowey, 2010; Schmid et al., 2010; Leopold, 2012). But, they responded briskly to such stimuli in ipsilesional space and to auditory and tactile stimuli on both sides of space as previously noted (Jiang et al., 2015). After a three month wait to ensure that the defects were stable, animals were re-tested in the perimetry apparatus. They remained profoundly hemianopic.

The initial series of multisensory rehabilitation exposure trials were designed to examine whether visual-auditory stimulus configurations that do not elicit SC multisensory integration could reverse hemianopia. Several visual-auditory configurations were used, and in all cases animals were rewarded for approaching the visual stimulus, but none ameliorated hemianopia. In the spatially-disparate paradigm, the auditory and visual stimuli were activated simultaneously, but the auditory stimulus was at 15° and the visual at 45° in the blinded hemifield. In the temporally-disparate paradigm, the stimuli both appeared at 45° in the blinded hemifield, but the auditory stimulus always preceded the visual by 300 ms. Although these configurations were chosen to be beyond the general spatial and temporal overlaps required to initiate multisensory enhancement, they could have facilitated behavior in other ways. In the temporal disparity paradigm the auditory stimulus could serve as a “prompt” or “cue” that could attract exogenous attention to the location of the visual stimulus. However, this paradigm proved to be ineffective in reversing hemianopia (Fig. 7). In the spatially disparate paradigm, the auditory stimulus could serve as a reliable predictor of the location of the visual stimulus as it was always 30 temporal to it. Because animals often missed the exact location of the brief auditory exposure stimulus, they would get rewarded for any miss that brought them to the location of the visual stimulus. After a number of such “errors” they began reliably orienting to the visual stimulus location (i.e., 45°). Nevertheless, when tested with the visual stimulus alone, they could not respond to it. They continued to be hemianopic.

The second round of attempted rehabilitation trials involved congruent multisensory exposure protocols. Visual and auditory pairs were presented either in spatiotemporal concordance at 45° in the blinded hemifield or in a unique spatially disparate configuration with the visual stimulus at 45° and the auditory stimulus at 75°. Even though in this “unique” configuration the stimuli are physically disparate in space, because SC neurons representing peripheral space have very large receptive fields, this stimulus configuration can also provide convergent excitatory signals that enhance the activity of their common SC target neurons and enhance behavioral responses in normal animals (Rowland et al., 2007a). Therefore, the configuration serves as an important test of the mechanistic hypothesis.

Within several weeks of training with this protocol, animals began responding to visual stimuli in contralesional space. Their hemianopia was resolving. Within a week of the first-identified visual responses to stimuli in the previously blind hemifield the animals were responding to stimuli across its entire extent. They had recovered, and their recovery times were very similar to those of animals that did not receive a first round of (ineffective) non-congruent multisensory exposure.

4.3. What is recovered?

All of these observations are consistent with the hypothesis that, as long as AES influences are available to the ipsilesional SC convergent excitatory multisensory inputs are both necessary and sufficient to restore visual function in hemianopia. Nevertheless, it is not known what specific pattern of activity must be generated in the remaining tectopetal afferents to achieve this visual restoration. Nor is it known whether tectopetal inputs from other structures (e.g., basal ganglia, see Harting et al., 1988; Wallace et al., 1990; McHaffie et al., 1993, 2005; Ciaramitaro et al., 1997) must also be altered by the exposure paradigm. Perhaps most importantly, the extent of the restored visual capabilities of these animals remains unknown. In the absence of visual cortex these animals would be expected to have only rudimentary vision in the contralesional hemifield so that visual stimuli in that hemifield should be at a competitive disadvantage with stimuli in its intact counterpart. To examine this possibility, an experimental series was begun that provided competitive visual cues in homologous locations in the two hemifields of rehabilitated animals. The cues were identical and responses to either were equally rewarded.

Not only did the previously blind hemifield of the rehabilitated animals show no competitive disadvantage in these tests, but 2 of the 3 animals tested preferred stimuli in that hemifield. Their preference was so strong that the relative visual intensity of cues in that hemifield had to be markedly reduced to eliminate it (Fig. 8). These observations suggest that the presence of visual cortex does not help in detecting and localizing visual cues – at least the simple cues that were used here. This finding also suggests that the issue of “balance” between excitatory inputs from cortex and inhibitory influences from the opposite hemisphere, discussed above, does not fully capture the nuances in these relationships and the functional potential of these circuit components. Furthermore, rehabilitated animals are capable of rudimentary pattern discrimination involving the previously-blind field (Fig. 9). This observation, combined with the very strong preferences and robust responses that animals exhibited, as well as their responses to manually presented stimuli, were difficult to reconcile with the idea that the animal responds reflexively, and without conscious perception, to contralesional stimuli.

Fig. 9.

Visual discrimination performance in a rehabilitated animal. The animal was trained to discriminate vertical and horizontal gratings at different spatial frequencies (green bars). After unilateral ablation of visual cortex and subsequent rehabilitation by multisensory exposure, the animal could perform the visual discrimination task at low but not high spatial frequencies (gray bars). Adapted from (Jiang et al., 2015). (For interpretation of the references to colour in this figure legend, the reader is referred to the Web version of this article.)

Yet, despite the impressive capabilities of the reinvigorated visual circuit, the loss of visual cortex must certainly have compromised many aspects of vision that were not tested here, including but not limited to processing fine details of visual patterns and the use of recent and past experience to complete and disambiguate visual sensory information. These functions have long been associated with different areas of visual cortex, and in their absence it is certain that even rehabilitated subjects do not have a “normal” visual perceptual experience.

It has been reported that rehabilitated human patients do not have awareness of contralesional stimuli when required to maintain fixation and inhibit responses to them. This is a standard clinical assay of visual perception, and neurotypic subjects have no difficulty in detecting visual stimuli under this constraint. Yet it is puzzling given the results described above suggesting that the salience of contralesional visual stimuli has not been compromised in rehabilitated cats. It is possible that procedural differences in the multisensory exposure paradigms used in the human and animal preparations produce different functional outcomes; e.g., differences in the contralesional exposure locations (fixed in the animal vs. variable in the human), physical differences in the stimuli used and/or frequency and duration of the experimental sessions. It is also possible that there are important differences in the site and extent of the lesions. However, the more likely possibility is based on the different task instructions provided. When central fixation was maintained in experiments with human subjects, there was likely a substantial inhibition of caudal SC (responsive to peripheral space) imposed by the rostral SC (e.g., see Rensink et al., 1997; Simons and Levin, 1997; Meredith and Ramoa, 1998). In the absence of cortical visual areas to supplement ipsilateral visual function, this suppression may be sufficient to suppress caudal visual responses to an extent that they are insufficient to generate detection of the peripheral visual stimulus. This remains an open research question.

In summary, despite likely changes in the perceptual experience of rehabilitated animals, their restored visual capabilities are quite impressive. Their visual capabilities appear to exceed those normally attributed to “blindsight,” as they can do far more than guess at the presence or absence of visual stimuli. They can discriminate rudimentary patterns and often choose the stimulus in the previously blind hemifield over a competitor in the intact field. Although perceptual deficits will certainly remain, these may be mitigated in part by their ability to properly orient to, and focus on, a visual event in contralesional space, thereby bringing at least part of that event into the intact field.

As a final note, although the principles of multisensory integration, development, and plasticity are helpful in understanding why and how the multisensory exposure paradigm is effective in this context when normal experience is not, it is unclear at this time whether the rehabilitative mechanism is dependent on the synthesis of cross-modal signals (i.e., multisensory integration) per se, or that the conditions necessary for its engagement are simply the same. Multisensory integration at the single neuron level requires that cross-modal signals converge onto target neurons within reasonably narrow windows of time. In the neurotypic animal, signals convergent in this way (which are typically derived from spatiotemporally congruent stimuli) lead to amplified responses. However, Hebbian mechanisms postulated to underlie multisensory plasticity do not necessarily require amplified responses, only that there is post-synaptic activation that follows pre- synaptic activation. Certainly in other cases in which these Hebbian mechanisms are believed to be engaged pre-existing multisensory integration capabilities are not required as in the case of the initial development of visual-auditory integration capabilities in the neonate or in the adult that was previously prevented from obtaining visual-auditory experience.