Skip to main content
PMC home page
Cerebral Cortex (New York, NY) logoLink to Cerebral Cortex (New York, NY)
. 2019 Dec 4;30(4):2030–2041. doi: 10.1093/cercor/bhz220

Using the Principles of Multisensory Integration to Reverse Hemianopia

Alexander S Dakos, Huai Jiang 1, Barry E Stein 1, Benjamin A Rowland 1,✉
PMCID: PMC7175010  PMID: 31799618

Abstract

Hemianopia can be rehabilitated by an auditory-visual “training” procedure, which restores visual responsiveness in midbrain neurons indirectly compromised by the cortical lesion and reinstates vision in contralesional space. Presumably, these rehabilitative changes are induced via mechanisms of multisensory integration/plasticity. If so, the paradigm should fail if the stimulus configurations violate the spatiotemporal principles that govern these midbrain processes. To test this possibility, hemianopic cats were provided spatially or temporally noncongruent auditory-visual training. Rehabilitation failed in all cases even after approximately twice the number of training trials normally required for recovery, and even after animals learned to approach the location of the undetected visual stimulus. When training was repeated with these stimuli in spatiotemporal concordance, hemianopia was resolved. The results identify the conditions needed to engage changes in remaining neural circuits required to support vision in the absence of visual cortex, and have implications for rehabilitative strategies in human patients.

Keywords: cortical blindness, rehabilitative training, superior colliculus

Introduction

A common consequence of unilateral damage to visual cortex is hemianopia: a profound blindness in the contralesional visual hemifield (Sprague 1966; Sherman 1974; Wallace et al. 1990; Zihl 1995; Romano 2009; Das and Huxlin 2010). This defect is generally permanent despite the physical sparing of visual circuits elsewhere in brain, particularly those in the midbrain superior colliculus (SC). Nevertheless, despite being far from the site of the lesion, SC neurons typically become unresponsive to visual stimuli consequent to these lesions, presumably because of the lesions’ excitotoxic effects on this efferent target mediated by N-methyl-D-aspartate (NMDA) receptors (Jiang et al. 2009). This renders SC neurons incapable of supporting visual orientation behaviors.

Recent work shows that hemianopia can be rehabilitated in both cats and humans (Bolognini et al. 2005; Frassinetti et al. 2005; Làdavas 2008; Passamonti et al. 2009; Dundon et al. 2015a, 2015b; Jiang et al. 2015; Purpura et al. 2017; Tinelli et al. 2017) using a simple noninvasive procedure in which spatiotemporally concordant pairs of auditory-visual stimuli are repeatedly presented in contralesional space over multiple training sessions. After weeks of this multisensory training, subjects become capable of detecting and localizing visual stimuli in the previously blind hemifield. This recovery of vision persists even after cessation of training (Bolognini et al. 2005). Furthermore, rehabilitated cats have been shown to be capable of performing rudimentary visual pattern discrimination tasks in the formerly hemianopic field (Jiang et al. 2015).

The success of this training paradigm is believed to depend on restoring the visual sensitivity of multisensory neurons in the ipsilesional SC through mechanisms of multisensory plasticity (Frassinetti et al. 2005; Yu et al. 2013; Dundon et al. 2015a; Jiang et al. 2015). Multisensory neurons in the intermediate and deep layers of the SC receive visual, auditory, and somatosensory inputs in various crossmodal combinations (Stein and Meredith 1993). One of the characteristic features of these neurons is their ability to integrate spatiotemporally concordant crossmodal inputs to enhance their responses (Meredith and Stein 1986; Meredith et al. 1987; Stein and Meredith 1993). It has also been noted that repeated presentation of such crossmodal stimuli increases their sensitivity to the individual modality-specific component cues (Yu et al. 2013). Thus, the convergence of multiple sensory inputs can have two complementary effects in these neurons: producing multisensory responses that are significantly more robust than those to the individual component inputs (Meredith et al. 1987; Stein and Meredith 1993) and, via Hebbian mechanisms, significantly enhancing unisensory responsiveness to these component inputs (Yu et al. 2013).

If multisensory training paradigms restore visual responsiveness of SC neurons to reverse hemianopia using similar mechanisms, the principles governing SC multisensory integration and plasticity should also govern the effectiveness of the multisensory training procedure. The aim of the present study was to test this assumption by providing cortical lesion-induced hemianopic cats with training paradigms involving different visual-auditory stimulus configurations. Some of these configurations provide convergent excitatory inputs onto common SC target neurons (i.e., they are “congruent”), and some of them do not. Presumably, the former configurations would rehabilitate the hemianopia and the latter would not (Table 1).

Table 1.

Summary of training paradigms

First (incongruent) training Second (congruent) training
F1 V15°/A45° V45°/A75°
F2 V15°/A45° V45°/A45°
F3 A45°(300 ms)V45° V45°/A45°
Open in a new tab

Each animal was first engaged in a training paradigm with incongruent multisensory cues (i.e., those not conforming to the spatial and temporal multisensory principles). Then, each cat was further trained with congruent cues. All degree measurements represent locations in the blinded hemifield.

Methods

Three adult mongrel cats (one male and two female), 2–6 years of age were used. Animals were obtained from a USDA licensed commercial animal breeding facility (Liberty Labs, Waverly, NY) and are referred to here as F1, F2, and F3. All procedures were performed in compliance with the 8th Edition of the ‘Guide for the Care and Use of Laboratory Animals’ (National Research Council of the National Academies 2011) and approved by the Institutional Animal Care and Use Committee at Wake Forest School of Medicine. One animal was sacrificed in order to confirm accuracy of the visual cortex lesion.

Prelesion Sensorimotor Evaluation

To ensure that each animal had normal visual and auditory fields, and was suitable for behavioral training, it was first evaluated in the apparatus shown at the left in Figure 1. The animal was gently restrained by the experimenter and required to stand at the start point and fixate on a food object presented in a hole at 0° in the forward wall of the apparatus. Next, a ping-pong ball on a steel wand was manually lowered into the visual field from behind an opaque curtain at various points between −105° and 105°. The experimenter was guided by premarked degree measurements on the top of the apparatus in order to ensure accuracy. Cats orient almost reflexively to this stimulus and rapidly approach it, ensuring easy observation of detection. Food rewards were given to all correct orientation/approach responses, and also on the rare occasion that the animal moved directly ahead to the visible food at 0°. Additionally, on some trials, the ball remained hidden behind the opaque curtain and was tapped on the side of the apparatus to produce an auditory stimulus. The animal was rewarded for approaching this auditory stimulus. All three animals rapidly learned to orient toward the visual and auditory cues when presented independently. They then advanced to the next stage of training.

Figure 1.

Figure 1

Open in a new tab

The apparatuses used for visual field assessment and multisensory training. (Left) A schematic of the perimetry device used to screen animals for their suitability for behavioral training. Fixation was induced by food presented through the hole at the center of the front wall, and visual orientation and approach were initiated by a ping-pong ball (on a metal wand) thrust into the visual field from behind an opaque curtain. (Right) A schematic of the perimetry device used for rehabilitative training and quantitative assessment of visual function. LEDs (red) and speakers (gray) were arranged along the horizontal axis and separated by 15°. The device was housed in a lighted room but shielded from direct overhead lighting by a flat ceiling. The animal was trained to fixate on the central LED at 0° before each trial (only the most central (i.e., nasal) LED in each 3-LED display was used as a visual target). Trials consisted of V alone or a crossmodal (A-V) combination. The animal’s task was to orient to and approach the location of V whether presented alone or in combination with A.

Visual Localization Training

Each animal was then trained to detect, localize, and approach illuminated light emitting diodes (LEDs, Lumex Opto/Components; model 67-1102-ND) within a perimetry apparatus with complexes of LEDs and speakers placed from −90 to 90° in 15° intervals (Fig. 1, right) (Gingras et al. 2009; Rowland et al. 2014). First, animals were trained to fixate on an illuminated LED at 0° (fixation stimulus) to receive the food reward (fixation was verified via a mirror above the forward wall of the apparatus). Next, animals were trained to localize and approach a briefly illuminated LED (100 ms) at randomly selected locations between −45° and 45° following termination of the fixation stimulus. Occasionally, trials containing more eccentric stimuli (generally ±60°) were randomly interleaved with those containing stimuli at the training locations to ensure that animals would respond to stimuli in more peripheral space.

Each trial was initiated by the experimenter pressing a foot pedal, which illuminated the fixation stimulus. Once the animal fixated on that stimulus, a second press of the foot pedal by the experimenter extinguished the fixation stimulus and triggered the target stimulus after a 500 ms delay. Animals were required to approach the location of the target visual stimulus within 3 s to receive a food reward (175 mg food pellet, Hill’s Science Diet). As a precaution, the experimenter wore headphones and averted his eyes from the array when the stimulus was triggered to ensure that the location of the stimulus was not known until after the animal’s response was scored. Catch trials were interleaved, in which no stimulus was delivered. On these, reward was delivered if the animal continued to hold fixation and remained at the starting position (No Go). No crossmodal stimuli were presented during the training period. Immediately after each training session animals were allowed to eat to satiation in the testing apparatus. Their weight was maintained at 85% or more of baseline. The criterion for the completion of training was achieved when an animal would correctly orient/approach a visual stimulus on 85% or more trials at each stimulus locations and maintain its fixation and position on 85% or more of the catch trials. Animals were not explicitly trained with multisensory cues before lesioning beyond what is naturally experienced in the environment.

Visual Cortex Lesions and the Hemianopic Defect

Once training and testing were complete, a unilateral visual cortex lesion was performed using sterile procedures as previously described (Jiang et al. 2015). Briefly, each animal was anesthetized with acepromazine/buprenorphine (0.02–0.05/0.005–0.01 mg/kg, IM) and sodium pentobarbital (22–30 mg/kg/iv), its head was fixed in a stereotaxic apparatus, it was intubated through the mouth, and its visual cortex was exposed by a craniotomy. All contiguous areas of left visual cortex were targeted for removal via aspiration (Fig. 2). This included most of the posterior lateral and suprasylvian gyrus, portions of the posterior ectosylvian sulcus (sparing the anterior region) and the cortical area above the splenial sulcus, posterior to the cruciate gyrus. All animals were given identical lesions in the same theater by the same surgeon. Postsurgical cefazolin (20 mg/kg, IM), buprenorphine (0.005–0.01 mg/kg, IM), and saline were administered. Following recovery from surgery, animals exhibited characteristic ipsiversive circling and showed no visual responsiveness to a laser pointer or threatening gestures, and variable responses to auditory and tactile snapping and poking stimuli in contralesional space (Sprague 1966; Wallace et al. 1990; Jiang et al. 2015). The circling behavior resolved within 1–3 days and tactile and auditory deficits progressively lessened, disappearing by the end of the next week. Animals remained unresponsive to contralesional visual stimuli for 2.5–3 months, at which point the hemianopic deficit was judged to be stable. Animals were then engaged in rehabilitation training.

Figure 2.

Figure 2

Open in a new tab

Lesion schematic. (Left) The extent of the hemianopia-inducing lesion is shown by gray shading on a tracing of the brain of animal F2 shown in a dorsal view. (Right) Four traced coronal sections illustrate the extent of the lesion at locations indicated by dashed lines. Gray shading represents removed tissue. Note that the damage is contained to visual cortex of the left hemisphere, while auditory cortex as well as the right hemisphere remain intact.

Rehabilitation Training

Each training/testing session included crossmodal (visual-auditory) stimuli repeatedly presented in the contralesional hemifield. Each animal received two “rounds” of training and was always rewarded for approaching the location of the visual stimulus regardless of the stimulus configuration. In the first round of training, the crossmodal stimulus configuration was designed to be noncongruent; that is, it would not produce concurrent crossmodal excitatory inputs to the same SC target neurons (see (Meredith and Stein 1986; Meredith et al. 1987; Stein and Meredith 1993; Miller et al. 2015). This was achieved by configuring the visual and auditory cues to be simultaneous but spatially disparate (two animals, F1 and F2), or spatially concordant and temporally disparate (one animal, F3). In the spatially disparate configuration, the visual stimulus was placed at 45° in the contralesional hemifield and the auditory stimulus was placed at 15°. This spatial offset exceeds the window for multisensory enhancement in localization behavior (Stein et al. 1989; Jiang et al. 2002) and, in multisensory neurons, often results in at least one of the crossmodal stimuli being located outside their respective receptive fields (Middlebrooks and Knudsen 1984; Meredith and Stein 1986; Meredith et al. 1987; Kadunce et al. 1997, 2001; Rowland et al. 2007). In the temporally disparate configuration, both stimuli were placed at 45° in contralesional space, but the visual stimulus onset was 300 ms after the auditory stimulus onset. This temporal offset exceeds the normal temporal window for multisensory integration in SC neurons (Meredith et al. 1987; Stein and Meredith 1993; Miller et al. 2015). Smaller delays are likely to lead to rehabilitation because they will produce convergent inputs (Meredith et al. 1987). The choice to present the auditory stimulus first was made in order to give the animal the best chance at rehabilitation in the case that attention would be pulled toward the auditory stimulus. In all configurations, visual and auditory stimulus durations were 100 ms, visual intensities were 6 mcd in a lighted room, and auditory intensities were 64 dB against a background of 51 dB at the ear. The duration was chosen to be long enough to be detectable by the animal but short enough to avoid the animal searching for the location/presence of the stimulus. In crossmodal trials, the auditory stimulus was clearly audible and served as a cue that a response to 45° (the location of the contralesional visual stimulus) would be rewarded. This allowed animals to repeatedly orient to the location of the visual stimulus even when they could not detect it (as confirmed on visual-only trials).

In the second round, congruent spatiotemporal configurations were selected that would elicit convergent crossmodal inputs to the SC. The animal’s task was the same as described above. In two animals (F2 and F3), the visual and auditory stimuli were simultaneously presented at 45° in contralesional space. In the third animal (F1), the visual stimulus was presented at 45°, and the auditory stimulus was simultaneously presented at 75°. Although it involves cues that are physically disparate, this configuration reliably elicits enhanced multisensory responses from normal animals at both the single SC neuron and behavioral levels (see Meredith and Stein 1986; Stein et al. 1989; Rowland et al. 2007). This enhancement likely reflects the structure of sensory map alignment, wherein auditory receptive field with centers at 45° (and aligned with visual receptive fields at 45°) often have lateral borders extending well beyond 75°. As a result, crossmodal stimuli in this configuration are within a single neuron’s respective receptive fields and send convergent excitatory inputs it, thereby rendering the stimuli functionally congruent. Other stimulus features in this round were the same as in the first.

In each round, 50 crossmodal trials were presented each day (for 5 days/week) with randomly interleaved No Go trials and visual-only trials in the ipsilesional hemifield to maintain animal conditioning (30% of trials). Immediately following each rehabilitative training session, animals were tested once again for the possible return of contralesional vision with 2–4 visual-only trials at random locations within ±60° of fixation. Criterion performance for rehabilitation was the same as in the prelesion training condition. If no rehabilitative effects were noted in these trials after 1.5 months of training in the first round (see Jiang et al. 2015), rehabilitation was considered a failure, and the second round was initiated.

Evaluation of Rehabilitative Progress

For each tested eccentricity within the affected hemifield (i.e., 15°, 30°, 45°, and 60°), the first replicable correct response to a visual-only stimulus in the hemianopic field was noted (two correct trials within one session), and the progression of responses to visual-only stimuli at each location tested in that field was tracked over several weeks. Following complete recovery of responses to the LED generated stimuli in that hemifield, visual recovery was further assessed in the apparatus initially used to evaluate sensorimotor function (Fig. 1, left) and with a variety of manually presented stimuli.

Statistical Significance

Significant responses to visual stimuli at each location were statistically determined as an increase in the probability of orienting and approaching the location when stimuli were present versus when they were not. Animals rarely oriented to locations in the absence of stimuli, but there was no pattern to these errors across locations in space (χ2 test F1: P = 0.91, F2: P = 0.79, and F3: P = 0.97). Thus, animals were said to be capable of detecting a visual stimulus at a particular location in space if their probability of an accurate localization response was above chance, evaluated with one-tailed binomial tests (H0: P = 0.125). Patterns in No-Go versus localization errors were evaluated for stimulus-containing trials using X2 tests. Responses within a hemifield were highly consistent with one another, and thus summary statistical testing was also performed by pooling responses on each side of space.

Histology

At the end of data collection, the animal was sedated with ketamine hydrochloride (20–30 mg/kg, i.m.) and acepromazine (0.1 mg/kg, i.m.) and, following loss of pinna reflexes, was injected with a lethal dose of pentobarbital (50–100 mg/kg, i.v.). The animal was then exsanguinated by transcardial perfusion with 0.9% saline followed by 4% paraformaldehyde fixative. The brain was removed, photographed, cut on a cryostat and processed using routine histological procedures. The extent of the cortical lesion on the dorsal surface of the brain was then drawn before charting the damage in serial coronal sections from postfixation photographs. The reconstructed lesion then was referenced to standard physiological and anatomical maps (Rosenquist 1985).

Results

Prelesion Training and Testing

Animals rapidly learned the initial sensorimotor task used to assess visual and auditory detection and localization capabilities. All had normal visual and auditory responses across both hemifields. They then were trained to orient and approach a visual stimulus in the LED apparatus that would later be used for rehabilitative training (Fig. 1). They reached criterion performance at all tested locations across both visual hemifields within 2–3 months (see Fig. 3), after which the contiguous areas of visual cortex on one side of the brain were removed.

Figure 3.

Figure 3

Open in a new tab

Prelesion visual field assessment. Plotted here in bar graphs is the performance of each animal on 20–30 visual-only trials after training at each location within the central 90° of visual space. The % correct on the vertical axis of the bar graphs refers to correct orientation/approach responses at that location. Vertical lines represent standard error *P < 0.01.

Hemianopic Defect

After the unilateral ablation of visual cortex, each of the animals was unresponsive to visual stimuli in contralesional space. This visual defect was categorized operationally as permanent when it persisted for a minimum of 2.5 months. The first confirmation of this hemianopia was via qualitative tests, in which the animals failed to respond to laser generated spots of light moved within, or into, contralesional space. They were similarly unresponsive to the appearance of food objects and threatening stimuli in this hemifield. The hemianopia was then confirmed with quantitative tests in the LED perimetry apparatus (see Fig. 4), in which animals failed to orient to or approach visual stimuli in the contralesional hemifield, but responded as they did in prelesion tests to stimuli in the ipsilesional hemifield. Performance on qualitative tests of visual performance in ipsilesional space was also similar to the results of prelesion tests and to the performance of animals without cortical lesions. This included approach, investigation and ingestion of food presented in that hemifield, and defensive reactions to threat.

Figure 4.

Figure 4

Open in a new tab

Postlesion visual field assessment. The accuracy of visual orientation/localization responses for each animal was quantified 2.5 months or more after unilateral visual cortex ablation. Bar graphs indicate the % correct on the final 40–80 visual-only trials at each location after the animals returned to their prelesion performance on the ipsilesional side of space. Note the absence of responses to V stimuli in contralesional (right) space. The bar graph next to the animal indicates performance on No Go ‘catch’ trials. Conventions are the same as in Figure 3.

Following completion of the postlesion visual assessment tests, the animals began the first round of rehabilitation training. Tests were conducted after each training session to assess visual localization performance on both sides of space.

Multisensory Training with Spatially Nonconcordant Stimuli Failed to Rehabilitate Hemianopia

Animals F1 and F2 were trained with the spatially disparate crossmodal configuration, in which the visual and auditory stimuli were simultaneous, but the auditory stimulus was presented at 15° and the visual stimulus was presented at 45°. Animals initially responded to the crossmodal trials with No Go or approach responses to locations that appeared to be randomly selected.

After several sessions, they appeared to adopt a strategy by which the auditory stimulus triggered an orientation response to 45° (the location of the contralesional visual stimulus), where they would receive a food reward. However, they continued failing to respond to contralesional visual stimuli at this location when presented alone. Hemianopia failed to be resolved in both animals even after 1.5 months of training with 50 crossmodal trials per day (Fig. 5). There were a few responses by animal F1 to visual stimuli near the fixation point on a small number (<10%) of test trials; these rare occurrences were observed randomly with no apparent pattern throughout the evaluation period.

Figure 5.

Figure 5

Open in a new tab

Spatially disparate cross modal stimuli failed to ameliorate hemianopia. Data are plotted for the final 40 visual-only trials at each location for each of these animals after 1.5 months of multisensory training. During training the A was 30° central to V. Both animals had a total of 320 exposures over this time period. *P < 0.01, n.s. = nonsignificant. Conventions are the same as in preceding figures.

Multisensory Training with Temporally Nonconcordant Stimuli Failed to Rehabilitate Hemianopia

Animal F3 was trained with the temporally nonconcordant configuration, in which the stimuli were spatially concordant at 45°, but the auditory stimulus onset was 300 ms before the visual stimulus onset. This animal also received 50/trials/day for 1.5 months and it too began responding on crossmodal trials to the 45° location, but failed to respond to the visual stimulus when it was presented alone at that location or at any other location in contralesional space. Its hemianopia had not resolved (see Fig. 6). Like animal F1, there were rare and unpredictable occurrences (<10% of trials) throughout the evaluation period in which this animal responded to the visual stimulus location near fixation.

Figure 6.

Figure 6

Open in a new tab

Temporally disparate multisensory training failed to ameliorate hemianopia. After 1.5 months of crossmodal exposure to the temporally disparate stimuli (A 300 ms before V) vision had not returned in the contralesional hemifield (320 visual probes). Conventions are the same as in previous figures *P < 0.01, n.s. = nonsignificant.

Subsequent Multisensory Training with Spatiotemporal Concordant Stimuli Rehabilitated Hemianopia

Two of the animals (F2, F3) that failed rehabilitation on the first round of crossmodal training were subsequently given 7 weeks of training with spatiotemporally concordant (i.e., congruent) visual-auditory stimuli at 45° in the hemianopic field (50 trials/day). Both were rehabilitated. Figure 7A illustrates their restored visual localization accuracy at all tested locations. Performance levels in contralesional and ipsilesional space were indistinguishable in both animals (P > 0.05 with the Fisher’s exact test at each location). Figure 7B plots the number of the session, in which the animal first responded to a visual stimulus at each of the tested eccentricities. Consistent with previous reports (Jiang et al. 2015), visual responsiveness first appeared to stimuli at central locations and then expanded to more peripheral locations. Additionally, the previous temporally asynchronous stimulation to which animal F3 was exposed did not appear to facilitate this animal’s eventual rehabilitation using spatiotemporally concurrent cues, given that this animal required the greatest number of sessions to show signs of full rehabilitation.

Figure 7.

Figure 7

Open in a new tab

Visual responses following rehabilitation of hemianopia with crossmodal stimuli in physical concordance. (A) Both animals showed localization capabilities to prelesion levels following 7 weeks of multisensory training with spatiotemporally concordant V and A stimuli at 45° in the blind hemifield. These training, or exposure, trials were provided after rehabilitation failed with nonconcordant stimuli. Both animals now showed similar performance to visual stimuli in the ipsilesional (−15 to −45°) and contralesional (15–45°) hemifields during V probe trials (380 in F2 and 336 in F3). *P < 0.01, n.s. = nonsignificant. (B) Below, the line plots show the number of sessions that preceded the onset of responding to V at a particular eccentricity in the contralesional hemifield. Note that both animals began responding first to stimuli at the most central location and then to progressively more peripheral locations. NG = No Go trials. Conventions are the same as in previous figures.

The last to be trained in a second round of crossmodal exposure trials was animal F1. In this case, rather than using the spatiotemporally concordant stimulus configuration that was used with animals F2 and F3, it was trained for 7 weeks with a configuration in which the crossmodal stimuli were simultaneous, but physically disparate. The visual stimulus was at 45° and the auditory was at 75°. As noted in Methods, the same degree of physical disparity was used in the first round of training with animals F1 and F2, but now the auditory stimulus was peripheral, rather than central, to the visual stimulus. This disparate configuration has been shown to be functionally equivalent to the spatiotemporally concordant configuration. It also provides convergent crossmodal excitatory inputs to SC neurons, thereby yielding enhanced multisensory responses both physiologically and behaviorally (Meredith and Stein 1986; Stein et al. 1989; Rowland et al. 2007). Thus, it was operationally defined as functionally congruent.

As illustrated (Fig. 8), this configuration was as effective at restoring visual responsiveness as was the spatiotemporally concordant paradigm described above. As in that paradigm, the recovery proceeded from more central to more peripheral locations. Rehabilitation progress speed was not increased in comparison with that observed in previous studies, suggesting that the earlier incongruent stimuli had no effect (Jiang et al. 2015). A summary of rehabilitation progress is provided (Fig. 9). Visual performance in this animal was even better at peripheral locations than was the recovered function in animals F2 and F3 (Fig. 10).

Figure 8.

Figure 8

Open in a new tab

Visual responses following rehabilitation of hemianopia with crossmodal stimuli in functional, but not physical, concordance. This animal received multisensory training with V at 45° and A at 75°, a physically disparate configuration that normally produces multisensory enhancement in single SC neurons and in tests of overt orientation behaviors. The bar graph plots % correct orientation responses in 320 trials after 7 weeks of multisensory training. Complete recovery was present at all tested locations in the contralesional hemifield, and performance was equally robust in both hemifields. The line graph shows that, as in the spatiotemporal concordant training paradigm (see Fig. 7), the onset of visual recovery was first evident at central locations and then at progressively more peripheral locations. NG = No Go trials. *P < 0.01, n.s. = nonsignificant. Conventions are the same as in previous figures.

Figure 9.

Figure 9

Open in a new tab

Summary of rehabilitation progress. Comparisons of the rehabilitation progress for all animals in the first and second rehabilitation sessions.

Figure 10.

Figure 10

Open in a new tab

Rehabilitation from hemianopia was apparent in the visual assessment apparatus. Rehabilitated animals readily detected, localized, and approached the ping-pong ball at eccentricities well beyond the 45° limit tested in the LED apparatus. Note, however, there was some fall-off in performance at the most peripheral locations in the contralesional hemifield. Conventions are the same as in previous figures.

Recovery from hemianopia was also evident from tests in the sensorimotor evaluation apparatus shown on the left in Figure 1. All rehabilitated animals oriented briskly and accurately to the introduction of the ping-pong ball from behind the curtain and showed no performance asymmetries across hemifields for most locations (Fig. 10). The exception to this observation was one animal that showed a slight performance deficit at contralesional 75° and another that showed a deficit at 60° with no responsiveness to further peripheral stimuli (the animals had not previously been trained to orient to more peripheral stimuli). Qualitative assessments in each animal revealed no hemifield asymmetries in following a laser generated spot between the two sides of space, and no hemispheric specific differences in reaction to manually presented stimuli.

Sensorimotor Responses to Contralesional Visual Stimuli Failed Rehabilitate Hemianopia

It is interesting that orientation to the visual stimulus location itself did not lead to rehabilitation in the first round of training. As noted above, the animals were rewarded when they oriented to 45° on crossmodal trials (where the visual stimulus was located). They did so even while they could not perceive the visual stimulus when presented alone at that location, or any other location in contralesional space. Apparently, each animal learned to respond to the crossmodal cue by approaching the 45° location to obtain a reward. But, as illustrated in the left column of Figure 11, even after months of reliably orienting to that location during the first round of training, their hemianopia had not resolved. Thus, neither the motor act itself, nor its association with a visual target, was sufficient to restore visual behavior. In contrast, after those animals were switched to congruent multisensory training (Fig. 11, right column), their hemianopia rapidly resolved and they showed no facilitation as a result of those previous orientation behaviors: there was no apparent relationship between the total number of such responses in the first round of training and the speed of the recovery induced by the second round of training. During this training period, errors directed to the auditory stimulus (in the conditions where the auditory stimulus appeared at a different location than the visual stimulus) were numerous but quickly decreased due to this behavior not being rewarded. During the final visual probes, whenever animals can reliably detect stimuli, both types of errors are very low; whenever error rates are high, no-go responses are overwhelmingly the most common. The dispersion of the localization errors (often calculated from only a few trials) did not change in a consistent way during the experiment. These data are summarized in Table 2.

Figure 11.

Figure 11

Open in a new tab

Overt sensorimotor responses to the visual stimulus location did not rehabilitate hemianopia. Plotted is the percentage of orientation/approach responses to the visual stimulus location (solid), auditory stimulus location (dashed), as well as the percentage of no overt response (No Go, dotted) during multisensory training trials. Each animal’s data are shown in one row. (Left) (first round of training): Animals F1 and F2 (spatial disparity) rapidly learned not to approach the auditory stimulus location (no reward was obtained there), but rather to approach 45° (the visual stimulus location) to obtain a reward—despite being unable to respond to that visual stimulus during visual-only trials. In the case of animal F3 (temporal disparity), the visual and auditory stimulus locations were the same. This “visuomotor” response did not resolve the hemianopia; responses to visual stimuli did not rise above chance during visual-only trials in any animal. (Right) (second round of training): Arrows indicate when responses to contralesional visual stimuli were first observed in each animal in unisensory testing sessions, and when its hemianopia had resolved at all tested locations.

Table 2.

Summary of errors

NoGo/localization (standard deviation)
Ipsilateral Contralateral
Prelesion 1–5%/3–4% (21–26°) 2–3%/1–5% (8–10°)
Postlesion 2–4%/2–3% (11–27°) 82–88%/13–18% (32–35°)
First (incongruent) training 3–4%/3% (9–28°) 91–93%/4–9% (27–33°)
Second (congruent) training 2–5%/2–3% (14–32°) 3–4%/1–3% (11–26°)
Open in a new tab

Percentages of responses on visual-only probe trials which were NoGo and localization errors (standard deviation of localization errors indicate in parentheses). Ranges of error rates across all animals and locations in ipsilateral and contralateral space are reported at four time points: before the lesion (prelesion), 3 months after the lesion (postlesion), after the first round of rehabilitation using noncongruent stimuli, and after the second round of rehabilitation using congruent stimuli.

Discussion

The present study demonstrated that there are specific spatial and temporal relationships, which are crucial for using crossmodal exposure to rehabilitate hemianopia: crossmodal configurations which were functionally congruent proved to be effective in this regard, but those that were not proved to be ineffective. Presumably, rehabilitation required that excitatory neural signals from different modalities converge onto their target neurons within a narrow temporal window in which they can interact. This is the same requirement for SC neurons to engage in multisensory integration and to produce enhanced multisensory responses (Meredith and Stein 1986; Meredith et al. 1987; Stein and Meredith 1993). However, at the moment it is unclear whether the integrative capability, or enhanced multisensory responses (physiologically or behaviorally), are also required. Evidence from hemianopic human subjects suggests that visual stimuli may (Leo et al. 2008) or may not (Ten Brink et al. 2015) be sufficient to significantly enhance localization of auditory stimuli in the blinded field at the behavioral level.

Nevertheless, that the neural architecture must be capable of supporting some form of cooperative crossmodal interaction, even at the earliest stages of training, seems highly likely. Training with modality-specific cues is ineffective, which suggests that multisensory circuits must be engaged at some level. Another alternative, that the auditory stimulus acted in a nonspecific alerting capacity to initiate attention to the visual stimulus in the present experiments (i.e., a “priming” effect, see Buchtel and Butter 1988; Spence and Driver 1997) was obviated by its ineffectiveness when positioned to best serve this purpose (i.e., 300 ms before the visual). In addition, although the auditory stimulus was ineffective in rehabilitation when simultaneous and central to the visual stimulus, it was highly effective when simultaneous and peripheral to it despite identical physical disparities in the two conditions. This latter disparate configuration is known to produce enhanced SC multisensory responses and overt SC-mediated behavior (Meredith and Stein 1986; Stein et al. 1989; Rowland et al. 2007). Thus, in each of the configurations employed here, the rule of thumb was: those configurations which normally induce SC multisensory enhancement will also rehabilitate hemianopia.

These crossmodal stimulus requirements for SC multisensory integration also match the requirements for engaging simple Hebbian mechanisms to strengthen the effectiveness of unisensory signals (Yu et al. 2013; Cuppini et al. 2018). Because the auditory signals drive postsynaptic responses in SC neurons at the same time, convergent visual inputs are arriving at those same neurons, the unisensory visual inputs can be strengthened (as can their auditory counterparts). This can ultimately render them more effective in driving the postsynaptic target neurons on their own. If such a mechanism were engaged here on those tectopetal visual afferents that survived the lesion but were initially too weak to evoke visual responses, they could, over time, become strong enough to once again activate their target SC neurons and support contralesional visual behaviors. This possibility is consistent with previous studies showing that repeated exposure to congruent visual-auditory stimuli, very much like those used here, enhanced the responsiveness of multisensory SC neurons to the visual component stimulus (Yu et al. 2013) and also restored the visual responsiveness of multisensory SC neurons lost after a hemianopia-inducing cortical lesion (Jiang et al. 2015). It is tempting to speculate that the SC is responsible for the recovery. However, whether recovery from hemianopia in the present studies reflected a direct potentiation of the remaining tectopetal visual afferents as hypothesized, and/or the action of changes elsewhere in the functional “loop” architecture between the SC and cortex (e.g., see McHaffie et al. 2005), remains to be determined. An attractive candidate for supporting the restored visual function would be a visual “loop” architecture traveling from the superficial layers of the SC through posterior thalamus, to association cortex, and then back to the intermediate/deep layers of the SC. Future studies selectively deactivating/lesioning these areas may provide evidence for their roles. Recovery from hemianopia not only leads to new detection abilities for the animal, congruent with blindsight, but also an ability for the animal to distinguish stimulus features. Animals rehabilitated using a similar paradigm were shown to be able to differentiate between vertical and horizontal lines as well as different triangle orientations presented simultaneously in both hemifields (Jiang et al. 2015). Animals here can distinguish between stimuli presented in the contralesional field and no-stimulus “catch” trials, which surpasses the capabilities of blindsighted animals (Cowey and Stoerig 1995; Weiskrantz et al. 2002).

A puzzling feature of the present results and those of the previous study of Jiang et al. (2015), is that recovery of vision, once initiated, proceeded in a central to peripheral progression. Yet, the exposure stimulus was always at a single peripheral location. Although this recovery pattern likely reflects differences in the density of the visual map in the SC, one that becomes progressively lower at more eccentric positions in the visual field, the dynamics of this relationship remain unclear (Hilgetag et al. 1999). What is clear, however, is that it is not necessary to train or sensitize each position in the hemianopic field for vision to return throughout that hemifield. Furthermore, it is not at all clear that training at multiple locations in the hemianopic field would speed this recovery process. The variation in the location of the stimuli that would be induced by such a training paradigm might actually retard the Hebbian mechanism alluded to above and its ability to induce visual recovery. But, this possibility remains to be tested.

It is particularly interesting to note the contrast between the effectiveness of the exposure paradigms used here in inducing recovery from hemianopia and the general ineffectiveness, in this context, of the cues experienced in the normal housing environment. Exposure to crossmodal stimulus pairs is presumably common in an animal’s home environment, involving a variety of natural events, some of which are likely to be of high significance. Yet these crossmodal experiences generally have little impact on hemianopia, albeit they may help explain occasional cases of “spontaneous” recovery and/or residual function that are attributed to incomplete damage to visual cortex (e.g., Fendrich et al. 1992).

The most obvious differences between the crossmodal experiences in the training paradigm and the crossmodal events encountered in more “natural” circumstances are the simplicity of the stimuli presented here, their consistency, and their regularity of appearance. Natural events produce visual and auditory stimuli with varying spatial and temporal relationships, varying features, varied backgrounds, and little consistency in rate of occurrence. Presumably, by ensuring that the training experience involved regular and consistent exposure to the identical configuration in the identical background condition, the effectiveness of a statistically-based Hebbian-like learning rule was maximized. If this assumption is correct, the regularity of the exposure (training) rather than the characteristics of the cues themselves would be the strongest predictive factor of the effectiveness of a multisensory training paradigm. A critical test of this hypothesis would involve systematically varying the physical properties of the crossmodal cues and/or their regularity during training.

If, as postulated above, the underlying mechanism engages Hebbian-like learning rules to enhance visual drive, other features of the rehabilitation paradigm such as orienting and approaching the location of the stimulus may not be crucial for its success. Indeed, these sensorimotor responses had little effect on rehabilitation in the present experiments. During crossmodal spatial disparity, training animals did learn to respond to the stimulus by approaching the 45° location, where the visual stimulus was located, to obtain reward regardless of where the auditory stimulus was. Yet, they still failed to show any capability of detecting the visual cue alone at that location, or at any other location tested in the contralesional hemifield. Furthermore, they showed no “savings” in the speed with which their hemianopia was ameliorated when switched to training with the spatiotemporal concordant crossmodal stimulus. Recovery was even slower than that found in the Jiang et al. (2015) experiments where animals were trained only with spatiotemporally concordant crossmodal stimuli.

Nevertheless, the absence of an effect of sensorimotor involvement in this particular paradigm does not obviate the possibility that it would affect recovery in other paradigms. Research with human hemianopic patients has shown facilitation effects when patients were trained to make successively larger saccades into the hemianopic field to capture progressively more eccentric visual stimuli (Zihl 1995; Nelles et al. 2001). Furthermore, in an important series of experiments with human hemianopic patients, Làdavas and colleagues showed that the active maintenance of fixation interfered with detecting and responding to visual cues in the hemianopic field following crossmodal rehabilitative training (see Bolognini et al. 2005). One likely possibility is the action of the competing intracollicular circuits shown to exist between regions representing fixation and peripheral targets (Meredith and Ramoa 1998). While the precise mechanisms of this recovery have not yet been conclusively determined, previous observations, combined with those presented here, provide increasingly stronger evidence that the multisensory circuitry of the SC and its multisensory principles are critical.

Funding

National Eye Institute at the National Institutes of Health (R01EY026916 and F31EY027686 to A.S.D.), the Tab Johnson Family Foundation and the Johnson Family Foundation.

Notes

We thank Nancy London for technical and editorial assistance. Conflict of Interest: None declared.

References

  1. Bolognini N, Rasi F, Coccia M, Làdavas E. 2005. Visual search improvement in hemianopic patients after audio-visual stimulation. Brain J Neurol. 128:2830–2842. [DOI] [PubMed] [Google Scholar]
  2. Buchtel HA, Butter CM. 1988. Spatial attentional shifts: implications for the role of polysensory mechanisms. Neuropsychologia. 26:499–509. [DOI] [PubMed] [Google Scholar]
  3. Cowey A, Stoerig P. 1995. Blindsight in monkeys. Nature. 373:247–249. [DOI] [PubMed] [Google Scholar]
  4. Cuppini C, Stein BE, Rowland BA. 2018. Development of the mechanisms governing midbrain multisensory integration. J Neurosci. 38:3453–3465. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Das A, Huxlin KR. 2010. New approaches to visual rehabilitation for cortical blindness: outcomes and putative mechanisms. Neuroscientist. 16:374–387. [DOI] [PubMed] [Google Scholar]
  6. Dundon NM, Bertini C, Làdavas E, Sabel BA, Gall C. 2015a. Visual rehabilitation: visual scanning, multisensory stimulation and vision restoration trainings. Front Behav Neurosci. 9:192. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Dundon NM, Làdavas E, Maier ME, Bertini C. 2015b. Multisensory stimulation in hemianopic patients boosts orienting responses to the hemianopic field and reduces attentional resources to the intact field. Restor Neurol Neurosci. 33:405–419. [DOI] [PubMed] [Google Scholar]
  8. Fendrich R, Wessinger C, Gazzaniga M. 1992. Residual vision in a scotoma: implications for blindsight. Science. 258:1489–1491. [DOI] [PubMed] [Google Scholar]
  9. Frassinetti F, Bolognini N, Bottari D, Bonora A, Làdavas E. 2005. Audiovisual integration in patients with visual deficit. J Cogn Neurosci. 17:1442–1452. [DOI] [PubMed] [Google Scholar]
  10. Gingras G, Rowland BA, Stein BE. 2009. The differing impact of multisensory and unisensory integration on behavior. J Neurosci. 29:4897–4902. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Guide for the Care and Use of Laboratory Animals: Eighth Edition 2011. Washington, D.C.: National Academies Press. [PubMed] [Google Scholar]
  12. Hilgetag C-C, Kötter R, Young MP. 1999. Chapter 8. Inter-hemispheric competition of sub-cortical structures is a crucial mechanism in paradoxical lesion effects and spatial neglect In: Progress in brain research. Amsterdam; New York: Elsevier, pp. 121–141. [DOI] [PubMed] [Google Scholar]
  13. Jiang H, Stein BE, McHaffie JG. 2009. Cortical lesion-induced visual hemineglect is prevented by NMDA antagonist pretreatment. J Neurosci. 29:6917–6925. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Jiang H, Stein BE, McHaffie JG. 2015. Multisensory training reverses midbrain lesion-induced changes and ameliorates haemianopia. Nat Commun. 6:7263. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Jiang W, Jiang H, Stein BE. 2002. Two corticotectal areas facilitate multisensory orientation behavior. J Cogn Neurosci. 14:1240–1255. [DOI] [PubMed] [Google Scholar]
  16. Kadunce DC, Vaughan JW, Wallace MT, Benedek G, Stein BE. 1997. Mechanisms of within- and cross-modality suppression in the superior colliculus. J Neurophysiol. 78:2834–2847. [DOI] [PubMed] [Google Scholar]
  17. Kadunce DC, Vaughan JW, Wallace MT, Stein BE. 2001. The influence of visual and auditory receptive field organization on multisensory integration in the superior colliculus. Exp Brain Res Exp Hirnforsch Expérimentation Cérébrale. 139:303–310. [DOI] [PubMed] [Google Scholar]
  18. Làdavas E. 2008. Multisensory-based approach to the recovery of unisensory deficit. Ann N Y Acad Sci. 1124:98–110. [DOI] [PubMed] [Google Scholar]
  19. Leo F, Bolognini N, Passamonti C, Stein BE, Làdavas E. 2008. Cross-modal localization in hemianopia: new insights on multisensory integration. Brain J Neurol. 131:855–865. [DOI] [PubMed] [Google Scholar]
  20. McHaffie J, Stanford T, Stein B, Coizet V, Redgrave P. 2005. Subcortical loops through the basal ganglia. Trends Neurosci. 28:401–407. [DOI] [PubMed] [Google Scholar]
  21. Meredith MA, Nemitz JW, Stein BE. 1987. Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors. J Neurosci. 7:3215–3229. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Meredith MA, Ramoa AS. 1998. Intrinsic circuitry of the superior colliculus: pharmacophysiological identification of horizontally oriented inhibitory interneurons. J Neurophysiol. 79:1597–1602. [DOI] [PubMed] [Google Scholar]
  23. Meredith MA, Stein BE. 1986. Spatial factors determine the activity of multisensory neurons in cat superior colliculus. Brain Res. 365:350–354. [DOI] [PubMed] [Google Scholar]
  24. Middlebrooks JC, Knudsen EI. 1984. A neural code for auditory space in the cat’s superior colliculus. J Neurosci. 4:2621–2634. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Miller RL, Pluta SR, Stein BE, Rowland BA. 2015. Relative unisensory strength and timing predict their multisensory product. J Neurosci. 35:5213–5220. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Nelles G, Esser J, Eckstein A, Tiede A, Gerhard H, Diener HC. 2001. Compensatory visual field training for patients with hemianopia after stroke. Neurosci Lett. 306:189–192. [DOI] [PubMed] [Google Scholar]
  27. Passamonti C, Bertini C, Làdavas E. 2009. Audio-visual stimulation improves oculomotor patterns in patients with hemianopia. Neuropsychologia. 47:546–555. [DOI] [PubMed] [Google Scholar]
  28. Purpura G, Cioni G, Tinelli F. 2017. Multisensory-based rehabilitation approach: translational insights from animal models to early intervention. Front Neurosci. 11:430. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Romano JG. 2009. Progress in rehabilitation of hemianopic visual field defects. Cerebrovasc Dis. 27:187–190. [DOI] [PubMed] [Google Scholar]
  30. Rosenquist AC. 1985. Connections of visual cortical areas in the cat. Cereb Cortex. 3:81–117. [Google Scholar]
  31. Rowland B, Jiang W, Stein B. 2014. Brief cortical deactivation early in life has long-lasting effects on multisensory behavior.. J Neurosci Off J Soc Neurosci. 34:7198–7202. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Rowland B, Stanford T, Stein B. 2007. A Bayesian model unifies multisensory spatial localization with the physiological properties of the superior colliculus. Exp Brain Res Exp Hirnforsch Expérimentation Cérébrale. 180:153–161. [DOI] [PubMed] [Google Scholar]
  33. Sherman SM. 1974. Monocularly deprived cats: improvement of the deprived eye’s vision by visual decortication. Science. 186:267–269. [DOI] [PubMed] [Google Scholar]
  34. Spence C, Driver J. 1997. Audiovisual links in exogenous covert spatial orienting. Percept Psychophys. 59:1–22. [DOI] [PubMed] [Google Scholar]
  35. Sprague JM. 1966. Interaction of cortex and superior colliculus in mediation of visually guided behavior in the cat. Science. 153:1544–1547. [DOI] [PubMed] [Google Scholar]
  36. Stein BE, Meredith MA. 1993. The merging of the senses, Cognitive neuroscience series. Cambridge (Mass): MIT Press. [Google Scholar]
  37. Stein BE, Meredith MA, Huneycutt WS, McDade L. 1989. Behavioral indices of multisensory integration: orientation to visual cues is affected by auditory stimuli. J Cogn Neurosci. 1:12–24. [DOI] [PubMed] [Google Scholar]
  38. Ten Brink AF, Nijboer TCW, Bergsma DP, Barton JJS, Van der Stigchel S. 2015. Lack of multisensory integration in hemianopia: no influence of visual stimuli on aurally guided saccades to the blind Hemifield. PLoS One. 10:e0122054. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Tinelli F, Cioni G, Purpura G. 2017. Development and implementation of a new telerehabilitation system for audiovisual stimulation training in hemianopia. Front Neurol. 8:621. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Wallace SF, Rosenquist AC, Sprague JM. 1990. Ibotenic acid lesions of the lateral substantia nigra restore visual orientation behavior in the hemianopic cat. J Comp Neurol. 296:222–252. [DOI] [PubMed] [Google Scholar]
  41. Weiskrantz L, Cowey A, Hodinott-Hill I. 2002. Prime-sight in a blindsight subject. Nat Neurosci. 5:101–102. [DOI] [PubMed] [Google Scholar]
  42. Yu L, Rowland BA, Xu J, Stein BE. 2013. Multisensory plasticity in adulthood: cross-modal experience enhances neuronal excitability and exposes silent inputs. J Neurophysiol. 109:464–474. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Zihl J. 1995. Visual scanning behavior in patients withhomonymous hemianopia. Neuropsychologia. 33:287–303. [DOI] [PubMed] [Google Scholar]

Articles from Cerebral Cortex (New York, NY) are provided here courtesy of Oxford University Press

RESOURCES