Do Cross-Modal Projections Always Result in Multisensory Integration?

Abstract

Convergence of afferents from different sensory modalities has generally been thought to produce bimodal (and trimodal) neurons (i.e., exhibit suprathreshold excitation to more than 1 sensory modality). Consequently, studies identifying cross-modal connections assume that such convergence results in bimodal (or trimodal) neurons that produce familiar forms of multisensory integration: response enhancement or depression. The present study questioned that assumption by anatomically identifying a projection from ferret auditory to visual cortex Area 21. However, electrophysiological recording within Area 21 not only failed to identify a single bimodal neuron but also familiar forms of multisensory integration were not observed either. Instead, a small proportion of neurons (9%; 27/296) showed subthreshold multisensory integration, in which visual responses were significantly modulated by auditory inputs. Such subthreshold multisensory effects were enhanced by γ-aminobutyric acid antagonism, whereby a majority of neurons (87%; 20/23) now participated in a significant, multisensory population effect. Thus, multisensory convergence does not de facto result in bimodal (or trimodal) neurons or the traditional forms of multisensory integration. However, the fact that unimodal neurons exhibited a subthreshold form of multisensory integration not only affirms the relationship between convergence and integration but also expands our understanding of the functional repertoire of multisensory processing itself.

Introduction

How the brain deals with events that are simultaneously transduced by different sensory modalities has been of considerable recent interest, as evidenced by the burgeoning number of multisensory investigations over the last half-decade. For multisensory processing to occur, the 1st, requisite step is for afferents from different sensory modalities to converge onto individual neurons (neuronal convergence) within the brain. In this manner, different sensory modalities can have access to each other such that one can influence processing in another. Evaluation of this gateway to multisensory processing has historically fallen into the realm of connectivity studies, which used neuroanatomical techniques to identify neural areas that receive converging inputs from parts of the brain subserving different sensory modalities (areal convergence), such as the superior temporal sulcus (STS; e.g., Seltzer and Pandya 1994) and superior colliculus (SC; e.g., Harting et al. 1992). In these regions, the physical and conceptual gap between “areal” and “neuronal” convergence seemed moot as both areas were populated by neurons easily identifiable as multisensory. These neurons showed direct evidence of multisensory convergence because they exhibited suprathreshold excitatory responses to the different stimuli represented by the afferent modalities (e.g., are bimodal or trimodal). Furthermore, when the stimuli from different sensory modalities were combined, these bimodal (and trimodal) neurons integrated their responses to the different inputs. Specifically, when compared with the response elicited by the most effective single-modality stimulus, a statistically significant increase (response enhancement) or decrease (response depression) in response evoked by combined-modality stimuli were defined as evidence of multisensory integration (Meredith and Stein 1983, 1986; reviewed in Stein and Meredith 1993). Thus, the notion that multisensory convergence results in multisensory integration robustly applies for bimodal neurons in the SC, the STS, and numerous other brain regions. Recently, there have been a number of studies that have used neuroanatomical tracers to demonstrate areal multisensory convergence in traditionally nonmultisensory areas, including portions of primary visual cortex in monkeys (Falchier et al. 2002; Rockland and Ojima 2003; Cappe and Barone 2005). A widely accepted conclusion from these anatomical studies has been that cross-modal projections into “unimodal” areas result in multisensory integration (Bavelier and Neville 2002; Falchier et al. 2002; Ghazanfar and Schroeder 2006). The present experiments were initiated to directly test this assumption by identifying cross-modal projections from auditory to visual cortices in ferrets and then physiologically examining the auditory-recipient visual areas for evidence of multisensory neurons and integration.

The ferret is becoming an effective model for neurophysiological investigations, and recent efforts by several labs have identified multiple auditory and visual regions as well as mapped their sensory properties. As shown in Figure 1 (adapted from Bizley et al. 2005), the auditory cortices are found on or near the middle ectosylvian gyrus and contain primary (A1), anterior auditory field (AAF), anterior dorsal field (ADF), anterior ventral field, posterior pseudosylvian field (PPF), and posterior suprasylvian field (PSF) areas. Each of these areas is distinct from the others based on physiological response features (Bizley et al. 2005). The visual cortices are found at the occipital pole and the primary and secondary visual areas form vertical bands that progress anteriorly, as also shown in Figure 1 (adapted from Manger et al. 2002). Using these auditory and visual cortical areas, the present study sought to determine if anatomical projections between ferret auditory and visual cortices occur and, if so, evaluate if these cross-modal projections correlate with the presence of bimodal visual–auditory neurons that generate multisensory integration. An abstract of preliminary findings for this work has been presented (Bittencourt-Navarrete et al. 2006).

Figure 1.

Visual and auditory areas of the ferret cerebral cortex. The inset shows a lateral view of the ferret cortex with the boxed region enlarged below. Cradled within the bend of the suprasylvian sulcus are the auditory cortices, which include the AAF, ADF, anterior ventral field (AVF), A1, PPF, and the PSF (after Bizley et al. 2005). Progressing anteriorly from the occipital pole are the retinotopically organized visual Areas 17, 18, 19, and 21 (after Manger et al. 2002).

Materials and Methods

All procedures were performed in compliance with the Guide for Care and Use of Laboratory Animals (National Institutes of Health publication 86-23) and the National Research Council's Guidelines for Care and Use of Mammals in Neuroscience and Behavioral Research (2003) and were approved by the Institutional Animal Care and Use Committee at Virginia Commonwealth University. The procedures are similar to those used for studies of cat cortex by this laboratory (Dehner et al. 2004; Meredith et al. 2006; Clemo et al. 2007). For this study, a total of 20 adult ferrets were used, as detailed below.

Anatomical Studies

Surgical Procedures

For tract tracing studies, a total of 6 ferrets were anesthetized with sodium pentobarbital (40 mg/kg intraperitoneally [i.p.] with supplemental doses as necessary) and their heads placed in a stereotaxic frame. Sterile techniques were used to perform a craniotomy to expose either visual or auditory cortical regions. To make tracer injections, a modified electrode carrier was used to support a 5-μl Hamilton syringe and its needle (31 gage). Biotinylated dextran amine (BDA; molecular weight = 10 000; lysine fixable; 10% in 0.1 M phosphate buffer) was pressure injected at a rate of 15 μl/min. To trace orthograde projections from auditory to visual cortices, 3 of the 6 animals received injections in primary auditory cortex (A1). In an attempt to cover this large cortical area, 3 injections, each in different parts of A1, were made in each animal (0.4- to 0.8-μl per injection). To identify retrogradely labeled neurons in auditory cortex, the other 3 (of 6) animals were injected around the posterior end of the lateral sulcus, corresponding to visual Area 21 (according to Innocenti et al. 2002). Following all injections, the cortex was covered with gel foam, the skin around the wound sutured closed, and standard postoperative care was provided.

Histological Procedures

After a postinjection survival period of 7–10 days, the animals were given a sodium pentobarbital overdose and perfused intracardially with heparinized saline followed by fixative (4.0% paraformaldehyde and 0.5% glutaraldehyde). The brains were blocked stereotaxically and stored in 0.1 M phosphate buffer (PB) with 25% sucrose (for cryoprotection) at 4 °C until sunk. Coronal sections (50 μm thick) were cut on a freezing microtome and collected serially. Sections, at 200- to 250-μm intervals, were processed for visualization of BDA using the avidin–biotin peroxidase method, according to the protocol of Veenman et al. (1992) with nickel–cobalt intensification. The sections were mounted on glass slides and coverslipped without counterstain.

Data Analysis

Neuronal labeling was visualized using a light microscope and the data plotted using a PC-driven digitizing stage controlled by Neurolucida software (Microbrightfield Biosciences, Inc., Williston, VT). A calibrated tracing of each section outline with the border between gray and white matter, the position of the injection site, labeled neurons, and axon terminals was produced. Injection sites were defined as the large aggregate of densely labeled cell bodies, dendrites, and axons at the terminus of the injection needle track. BDA-labeled neurons generally were sharply black throughout their soma and dendrites. BDA-labeled axon terminals appeared as sharp, black swellings at the end of thin axon stalks or as symmetrical varicosities along the course of an axon. Tissue outlines and injection sites were plotted at ×40 magnification. Labeled neurons and boutons were plotted at ×200 magnification and the Neurolucida software kept a count of numbers of identified neurons and boutons. Labeled cell bodies were plotted in the region of auditory cortex, whereas labeled axons and boutons were sought throughout the occipital pole of the cortex. Gyral and sulcal landmarks were used to identify labeled regions and to correlate them with functionally distinct regions of cortex, as mapped by Innocenti et al. (2002) and by Bizley et al. (2005). The plotted sections from each animal case were arranged serially and then graphically displayed using Photoshop software (Adobe Systems, Inc., San Jose, CA).

Physiological Studies

Surgical Procedures

For single-unit recording procedures, a total of 8 adult ferrets were anesthetized with sodium pentobarbital (40 mg/kg i.p.) and their heads were placed in a stereotaxic frame. Under aseptic surgical conditions, a craniotomy was performed to expose the visual cortices. In 3 of the 8 animals, BDA was injected into the auditory cortices (in a manner identical to that described for orthograde projections, above) to determine if subsequent recording penetrations sampled the zone of termination for this cross-modal projection. Anatomical data (i.e., the number of labeled neurons and boutons) were not tabulated for these 3 animals. A stainless steel recording well was secured to the animal's head with screws and dental acrylic. The scalp was then sutured around the implant and routine postoperative care was provided.

Recording Sessions

Three to 5 days following implantation, recording sessions began by anesthetizing the animals (35 mg/kg ketamine; 2 mg/kg acepromazine intramuscularly [i.m.]) and securing the implanted well to a supporting bar. The animals were intubated through the mouth and maintained on a ventilator with expired carbon dioxide at ∼4.5%. Heart rate and body temperature were monitored and a heating pad was used to maintain temperature at 38 °C. Fluids, supplemental anesthetics (8 mg/kg/h ketamine; 0.5 mg/kg/h acepromazine i.p.), and a muscle relaxant (to prevent spontaneous movements; pancuronium bromide 0.3 mg/kg initial dose; 0.2 mg/kg/h supplement i.p.) were continuously administered using an infusion pump. The recording well was then opened to expose the visual cortices surrounding the posterior section of the lateral sulcus.

For recording, a glass-insulated tungsten electrode (tip exposure ∼20 μm, impedance <1.0 MΩ) was inserted into the cortex and advanced using a hydraulic microdrive. Neuronal activity was amplified, displayed on an oscilloscope, and played on an audiomonitor. Neurons, identified at 125- or 250-μm intervals along a recording penetration, were isolated by their spontaneous activity and/or their responses to manually presented moving bars of light or dark stimuli. Each neuron was also tested using manually delivered auditory (clicks, claps, hisses, etc.) and somatosensory (air puffs, brushes, or taps with a fine paintbrush, muscle compression, and/or joint rotation) stimuli. After this qualitative evaluation, “quantitative” sensory tests consisting of computer-triggered visual and auditory stimuli, alone or in combination, were presented. The stimulus presentations were separated by 7 s and each condition was presented 25 times in an interleaved fashion. Combined stimuli were presented with visual onset preceding auditory by 50 ms to compensate for the latency discrepancy between these modalities. Visual cues were light bars, whose movement direction, velocity, and amplitude across the visual receptive field were computer controlled and projected on a translucent hemisphere (92 cm diameter). Free field auditory cues were electronically generated white noise bursts (40- to 50-ms duration, 70-dB sound pressure level [SPL]) delivered by a hoop-mounted (88 cm diameter) speaker positioned in spatial register with the visual receptive field. Neuronal responses were routed to a computer for storage and later analysis. For each data file, Spike2 software was used to examine the templates for the recorded action potentials and only reliably isolated waveforms were subjected to further analysis. For a given spike waveform (i.e., a single neuron), a peristimulus time histogram was constructed for each of the test conditions (visual only = V, auditory only = A, and visual–auditory combined = VA), from which the response duration was determined and the mean spikes per trial (and standard deviation) was calculated. For each neuron, these data were recorded and tabulated to enable calculation of population effects.

The results were statistically evaluated at 2 different levels (note: no suprathreshold responses to auditory stimuli were observed, consequently values for auditory activity did not have to be considered). 1) Individual neurons: for each neuron, the mean spikes/trial from the 25 trials each of visual (V) and combined stimuli (VA) were compared (paired t-test, P < 0.05). 2) Population: the average population response (mean spikes/trial) for each category (i.e., V and VA) were compared (paired t-test, P < 0.05).

Microiontophoresis

A total of 6 additional adult ferrets were prepared for single-unit recording in the manner described above. On the day of the recording experiment, they were anesthetized (35 mg/kg ketamine; 2 mg/kg acepromazine, i.m.), the recording well was opened, and the dura reflected to admit the microiontophoresis pipette/electrode assembly. The microiontophoresis/recording assembly was either obtained from Carbostar-3 (Kation Scientific, Minneapolis, MN) or made from a standard glass-insulated recording electrode (impedance 0.6–1 MΩ), to which was glued (cyanoacrylic, supported by dental acrylic) a double-barreled pipette (4 in. long, 1.5-mm-outer diameter capillary glass with microfilament; #6310, A-M Systems, Carlsborg, WA) with tips pulled and trimmed to reveal an internal diameter opening of ∼10 μm (pipette tips positioned within 40–50 μm of the electrode tip). In either case, 1 barrel of the pipette was filled with bicuculline methiodide (BIC, 10 mM, pH 3.5), whereas the other contained normal saline. The assembly, supported by a stereotaxic carrier, was manually positioned over Area 21 through which it was slowly advanced using a hydraulic microdrive. During all sensory tests, a microiontophoresis current programmer (WPI, Inc., Sarasota, FL model 260) was used to deliver holding/ejection currents. For BIC, a negative holding current of 2–5 nA was used, whereas a positive current of the same value was delivered through the saline channel to balance the net current flow. Once a neuron was identified, the following battery of tests was presented. A quantitative separate- and combined-modality (V, A, and VA; n = 25 presentations of each stimulus condition, interleaved, 7-s interstimulus interval) test was presented and the neuronal responses stored on computer. This sensory stimulation paradigm was presented 1st prior to the ejection of BIC and then 60 min after its iontophoresis into the tissue (ejection current was positive 10–20 nA, continuous for 3–5 min, balanced by a negative current of the same magnitude through the saline channel). Neuronal responses to each of the sensory stimulation conditions (A, V, A) were determined for the control (pre-BIC) and post-BIC conditions. The responses to the sensory stimuli were then compared for the control and post-BIC treatment conditions to determine 1) if an excitatory response to auditory alone stimulation could be elicited in the presence of BIC and/or 2) if visual responses might be facilitated by the presence of an auditory cue in the presence of BIC. These data for each neuron were then tabulated and graphed, so that the BIC effect could be assessed for the entire sample. Statistical evaluation of neuronal and population responses were conducted using paired t-tests (P < 0.05).

Recording Site Reconstruction

The depth of each identified neuron within a penetration was noted and correlated with the results of the quantitative sensory tests. At the conclusion of each recording penetration, a small electrolytic lesion (0.3 mA for 0.5 s) was made to facilitate histological reconstruction of the data. When the recording was finished, the animals received a barbiturate overdose and were perfused intracardially with saline followed by 10% formalin (4% paraformaldehyde in the cases in which BDA was injected). The brains were then blocked stereotaxically, removed, and postfixed in 25% sucrose/10% formalin (4% paraformaldehyde in the cases in which BDA was injected). Frozen sections (50 μm) were cut in the coronal plane through the recording sites, processed using standard histological procedures and counterstained. A projecting microscope was used to trace sections and reconstruct the recording penetrations.

Results

Anatomical Connections

Orthograde Projections from Auditory Cortex

For the examination of orthograde projections from auditory to visual cortex, sulcal and gyral landmarks were used to estimate the location of the auditory cortices (according to the map of Bizley et al. 2005) and multiple injections were centered upon this region. Upon reconstruction, these injections not only filled portions of primary auditory cortex (A1) but also included portions of its adjoining neighbors of AAF, ADF, PPF, and PSF as well as some of the underlying white matter.

Examples of labeled auditory boutons in visual Area 21 are depicted in the micrographs shown in Figure 2A. In these and the other cases, filled axons were often interrupted by symmetrical swellings or boutons in passage that occurred with such periodicity that some examples looked like microscopic strings of beads; labeled boutons at the ends of short axon stalks, or terminal boutons, were less frequently observed. A total of 27 565 labeled boutons were plotted from 18 coronal sections from the 3 animals. In each animal, boutons labeled from the auditory cortex were consistently found along the posterior end of the lateral sulcus (see Figure 3), in a region that generally corresponded with the area of the horizontal meridian of the visual field representation of Area 21 (Manger et al. 2002), although the most posterior extent of the labeling may have encroached upon Area 19. Qualitatively, this projection was rather modest in terms of its overall density, which was progressively reduced toward the posterior end of the sulcus. It preferentially targeted the supragranular layers of Area 21, with an average 77:23 suprainfragranular ratio suggestive of a feedback-type projection.

Figure 2.

Orthogradely labeled auditory boutons in visual Area 21 (A) and retrogradely labeled neurons in auditory cortex (B). In (A), the micrograph shows a network of labeled axons with terminal as well as boutons in passage in visual Area 21. These orthogradely labeled processes resulted from tracer injections focused on auditory area A1. In (B), tracer injection into visual Area 21 produced retrogradely labeled pyramidal neurons in auditory cortex. Scale bars = 10 μm.

Figure 3.

Orthograde projections from auditory to visual cortex. In each of 3 cases, tracer injection into and around primary auditory cortex produced labeled axons and boutons (each dot = 1 bouton) in the visual area surrounding the lateral suclus (LS) designated Area 21 (Manger et al. 2002). The insets show lateral views of the ferret cortex and the sites of injection for each case; vertical lines indicate location of the coronal sections expanded to the right. The coronal sections, arranged serially from anterior (left) to posterior (right) display the injection sites (gray areas) as well as the locations of labeled boutons. In each case, the density of labeled boutons was highest anteriorly and diminished at more posterior levels. The last section to the right indicates the most posterior level at which labeled boutons were found. Scale bar = 1 mm.

Retrograde Labeling from Visual Cortex

To confirm the auditory cortical source of the cross-modal projections to visual cortex, tracer was injected into Area 21 and retrogradely labeled neurons were identified in tissue sections through the auditory cortex. These labeled neurons had a morphology that was typical of cortical pyramidal neurons (see Fig. 2B). As depicted in Figure 4, almost all retrogradely labeled neurons were located within the posterior aspects of the auditory cortices, especially within the dorsal and posterior aspects where areas A1 and PSF approach the suprasylvian sulcus. Only a few retrogradely labeled neurons were found within the gyral area corresponding to A1 as it is presently defined (Bizley et al. 2005) or in sulci of auditory areas outside this posterior portion of auditory cortex.

Figure 4.

Auditory cortical source of projections to visual cortex. The inset shows the lateral view of the ferret cortex with the auditory cortical divisions outlined (dashed lines) and the visual cortical injection site (gray area) in and surrounding the lateral sulcus. Vertical lines indicate the levels from which the expanded coronal sections (anterior-left to posterior-right) are taken. Each black dot represents the location of 1 neuron retrogradely labeled from the Area 21 injection site (gray area on far right). Retrogradely labeled neurons were primarily found in association with the dorsal and posterior aspects of A1. SS, suprasylvian sulcus; LS, lateral sulcus. Scale bar = 1 mm.

Single-Neuron Sensory Physiology

A total of 19 recording penetrations traversed the banks of the lateral sulcus corresponding to the lower field representations of Area 21. These penetrations are summarized in Figure. 5. Recording sites were spaced at 125-μm (or 250 μm) intervals through the full thickness of the cortical mantle to facilitate an unbiased sampling (gaps indicate unresponsive sites). Neurons encountered at each location were subjected to a thorough battery of qualitative, manually delivered sensory tests that included several varieties of visual, auditory, and somatosensory stimuli (see Materials and Methods), which have been used effectively to identify multisensory neurons in other studies (Dehner et al. 2004; Meredith et al. 2006; Clemo et al. 2007). Of the 416 neurons evaluated, 296 were responsive to visual stimuli, 120 were unresponsive to all sensory (visual, auditory, and somatosensory) stimulation. No bimodal (i.e., visual–auditory) neurons were identified.

Figure 5.

Recording sites through visual Area 21. Schematic (far right) of the lateral view of the ferret cortex shows the levels (anterior-left to posterior-right) from which the coronal sections containing 19 recording penetrations were taken. Along a given penetration, each dash indicates a recording site where visual neurons were found and tested using qualitative and quantitative single- and combined-modality sensory stimulation. LS, lateral sulcus. Scale bar = 1 mm.

Figure. 6 shows a recording penetration in an animal that received an auditory cortical BDA injection at the time of implantation of the recording well. During the subsequent recording experiment, a penetration sampled 7 visual neurons in Area 21 at a location just lateral to the lateral suclus. Following the experiment, histological processing of sections containing Area 21 revealed the labeled axons and boutons that projected from auditory cortex as well as the electrode track. As shown in Figure 6, the recording penetration passed through the projection field from auditory cortex, as evidenced by the labeled axons and boutons visible on either side of the electrode track. Histological reconstruction of the recording penetration revealed that all but 1 of the recording sites was within 125 μm of a labeled bouton. Thus, it appears that the neurons recorded in Area 21 were in the appropriate location to have received inputs from auditory cortex.

Figure 6.

Cross-modal projections terminated in the area of recording penetrations. In an animal that received an auditory cortical tracer injection prior to recording, orthogradely labeled axons terminals and boutons (fine black wavy lines and dots, respectively) were present in the same cortical location as a recording electrode track (long black line overlying tissue damage). Visual neurons were found at each dash on the recording penetration. Note that labeled boutons were found on both sides of the recording track, indicating that the penetration occurred in close proximity to the termination of the cross-modal projection. A and B refers to recording sites for the neuronal responses shown in the next figure (Fig. 7).

A total of 296 visually responsive neurons were subjected to additional sensory tests using electronically controlled separate- (A, V) and combined-modality (VA) stimuli and quantitative analysis. Included in this sample were 2 neurons whose recording sites were located among labeled auditory boutons depicted in Figure 6 (neurons A and B). The responses of these neurons to separate- and combined-modality tests are shown in Figure 7. These neurons responded vigorously to the presentation of a visual stimulus, showed no response to an auditory stimulus presented alone, and the response to the visual stimulus was unaffected by the presence of the auditory cue. The same lack of auditory responsiveness was observed in the vast majority of the 296 neurons tested in this fashion. Thus, at the neuronal level, there was no evidence of suprathreshold nonvisual inputs. The possibility of “subthreshold” effects on multisensory processing, as observed in other cortical areas (Meredith 2002; Dehner et al. 2004; Meredith et al. 2006; Allman and Meredith 2007) was also evaluated. When the responses to combined visual–auditory stimuli were statistically compared (paired t-test) with that elicited by the visual stimulus alone, combined-modality stimulation produced a subtle but significant response change indicative of multisensory integration (Meredith and Stein 1983, 1986) in 9% (n = 27/296) of Area 21 neurons. Of these, 3.3% showed a significant increase (n = 10/296; average +28%), whereas 5.7% revealed a significant decrease (n = 17/296; average −30%).

Figure 7.

Responses of Area 21 visual neurons to visual, auditory, and combined visual–auditory stimuli. For 2 representative neurons (marked A and B in Fig. 6), responses to a visual stimulus (ramp labeled “V”), auditory stimulus (square wave labeled A), and combined stimuli (VA) are shown in the rasters (dot = 1 spike; each row = 1 trial) and histograms (10-ms time bins). For both neurons, the visual stimulus elicited a robust response, whereas the auditory stimulus was ineffective. When these same visual and auditory stimuli were combined, the responses of these neurons were not significantly changed from the visual alone condition, as shown in the respective bar graphs (mean spikes/trial ± SD). Spontaneous activity is indicated by the dashed line (Sp).

Given the presence of subthreshold multisensory effects at the neuronal level, the data set was reexamined to assess whether a multisensory effect would be apparent at the population level. For each neuron, the response to the visual stimulus alone was plotted against its response to visual–auditory combined. In the resulting graph (see Fig. 8A), nearly equal proportions fell slightly above (48%; n = 142/296) or slightly below (52%; n = 154/296) the line of unity. Furthermore, the population average response to the visual (10.4 ± 0.5 mean spikes/trial) and to the visual–auditory (10.3 ± 0.6 mean spikes/trial) combinations were not significantly different (P = 0.38 paired t-test). Collectively, these results indicate that multisensory effects were not evident at the population level in Area 21.

Figure 8.

Comparison of responses of Area 21 neuron population to visual and combined visual–auditory stimuli. (A) For the 296 neurons examined, this graph plots the relationship of neuronal responses (mean spikes/trial) to the visual stimulus alone (V; x-axis) to those evoked by the combined auditory and visual stimuli (VA; y-axis). Most responses fell on or near the line of unity, with a similar number either slightly above (48%; 142/296) or below (52%; 154/296), suggesting that the auditory stimuli had no net effect on the population. (B) The bar graph shows the population (n = 296) response average of the mean spikes/trial (±SEM) to the visual alone (V) and to the combined visual–auditory stimulation (VA), which were not significantly different (P = 0.38, paired t-test).

Microiontophoresis and Single-Neuron Responses

The presence of an auditory cortical projection to Area 21, coupled with physiological evidence for subthresold cross-modal effects in 9% of Area 21 neurons, suggests that auditory influences in this visual region may be weak and/or strongly regulated by local inhibitory processes. To examine the latter possibility, an additional 23 neurons were tested using the same sensory testing and recording techniques as used before but now in combination with microiontophoretic application of the γ-aminobutyric acid A (GABA-A) antagonist BIC. Figure 9 depicts the responses of a neuron to the single- (V, A) and combined-modality (VA) stimuli before application of BIC (control) and following its ejection (post-BIC). In the control condition, this neuron failed to respond to an auditory stimulus presented alone but was vigorously activated by the visual cue. In addition, the response of this same neuron to the combined visual–auditory stimulation (2.3 ± 1.8 mean spikes/trial) was significantly reduced compared with the visual stimulation alone (3.9 ± 2.1 mean spikes/trial, P < 0.05), indicative of cross-modal subthreshold inhibition. In the presence of BIC, this same neuron showed an increase in spontaneous activity and an increased response during both visual alone and combined visual–auditory stimulus conditions. Furthermore, and despite the continued lack of response to the auditory cue alone, the response to the combined visual–auditory stimulation was now significantly greater than the visual stimulation alone (V = 5.9 ± 3.2 vs. VA = 8.0 ± 3.8, P < 0.05). Within the sample of neurons tested with BIC, a larger proportion showed significant levels of subthreshold facilitation in the post-BIC condition (n = 4/23) than before BIC administration (n = 0/23).

Figure 9.

Effect of blockade of local inhibition on an Area 21 visual neuron in response to visual, auditory, and combined visual–auditory stimulation. (A) The neuron responded (rasters: dot = 1 spike; each row = 1 trial; histograms: 10-ms time bins) to a visual stimulus (ramp labeled V) but not to an auditory stimulus (square wave labeled A). When the 2 stimuli were combined (VA), the visual response of the neuron was noticeably reduced. These responses are summarized in the bar graphs (far right), where it is evident that the response (mean spikes/trial ± SD) to the combined response was significantly reduced (paired t-test, P < 0.05, *). (B) The identical tests were repeated in the same neuron, but this time after the ejection of the GABAergic antagonist BIC. When GABAergic inhibition was blocked, this neuron remained unresponsive to the auditory stimulus but showed significant levels of response facilitation to the combined visual–auditory stimulation (paired t-test, P < 0.05, *). Spontaneous activity is indicated by the dashed line (Sp).

For the population tested with BIC (n = 23), when the combined and visual responses for each neuron were plotted for control and post-BIC conditions, 87% (20/23) now exhibited responses that fell above the line of unity. In contrast, in the control condition, 52% (12/23) of the responses plotted above the line of unity (see Figure 10A), as did a similar proportion (48%) in the original sample (142/296, Fig. 8A). In addition, the blockage of local inhibition resulted in a significant increase for the population average response to the combined-modality stimulus over that evoked by visual stimulation alone. As shown in Figure 10B, control responses to visual and to visual–auditory stimulation showed no significant difference (as did the original sample of 296 neurons; see Fig. 8B) but in the post-BIC condition responses to combined visual–auditory stimulation (6.9 ± 1.1) significantly exceeded those evoked by visual cues alone (6.1 ± 1.0; P < 0.001). Collectively, these comparisons indicate that inhibitory mechanisms can strongly regulate the expression of subthreshold multisensory integration in Area 21.

Figure 10.

Comparison of responses of Area 21 neurons to visual and combined visual–auditory stimuli before and after inhibitory blockade with BIC. (A) For the 23 neurons tested, this graph plots the responses (mean spikes/trial) to the visual stimulus alone (V; x-axis) versus those evoked by the combined auditory–visual stimuli (VA; y-axis) under control (light gray symbols) and post-BIC ejection conditions (black symbols). For the control condition, most responses fell on or near the line of unity (dashed line). However, when inhibition was blocked, 4/23 neurons showed significant levels of response facilitation and 87% (20/23) had responses that fell above the line of unity. Regression lines are plotted for the control and post-BIC conditions. (B) The bar graph shows the population (n = 23) response average of the mean spikes/trial (±SEM) to the V and combined condition VA for control (gray) and post-BIC conditions (black). Note that when GABAergic inhibition was blocked, there was a significant difference in the response to visual and combined visual–auditory stimulation (paired t-test, P < 0.001, *).

Discussion

The major findings of this study show that cross-modal projections do not necessarily result in the generation of suprathreshold (bimodal or trimodal) types of multisensory neurons. However, cross-modal projections do result in multisensory integration, which, in this case, took the form of subthreshold multisensory effects that were substantially controlled by local inhibition.

Cross-Modal Anatomical Projections

The present experiments document the presence of a cross-modal projection from auditory cortex to ferret visual Area 21 (likely homologue of primate V4; Manger et al. 2002). Injections centered on primary auditory cortex consistently labeled axon terminals along the banks and fundus of the lateral sulcus, corresponding to the representation of region of the horizontal meridian of the visual field in Area 21 (Manger et al. 2002). Furthermore, tracer injected into Area 21 retrogradely labeled neurons in sulcal regions surrounding the posterior aspects of the auditory cortex. Although cross-modal projections between primary auditory and visual areas have been documented in monkeys (Falchier et al. 2002; Rockland and Ojima 2003), the present study identified only a few retrogradely labeled neurons in gyral locations that may correspond to A1. In addition, the present study, which used the same tracer as Rockland and Ojima (2003), observed that the cross-modal auditory cortical projection primarily terminated in a portion of visual Area 21. Although some labeled boutons may have also fallen within the anterior part of Area 19, few labeled auditory boutons were observed in visual cortices posterior to this. Regardless of the possible connectional differences between species, the basic question addressed here is whether demonstrated cross-modal connections can be assumed to produce a specific form of multisensory neuron and, thereby, multisensory integration.

Cross-Modal Convergence and Suprathreshold (Bimodal) Neurons

The suprathreshold (bimodal or trimodal) form is the most evident form of multisensory neuron and has the longest history. Among the 1st neurophysiological studies of multisensory phenomena was the demonstration of the excitatory responses to independent visual, auditory, and somatosensory stimulation on SC neurons (Horn and Hill 1966). Since then, many examples of bimodal (and trimodal) neurons have been reported in other well-known multisensory areas, such as the STS, intraparietal sulcus, and prefrontal cortex.

Because bimodal neurons have essentially been the currency of multisensory studies, it should be of little surprise that, when cross-modal connections have been identified, the “assumed” result of that convergence is the presence of bimodal neurons. In support of this assumption, studies of nonvisual responses in visual cortex, conducted 3–4 decades ago using unanesthetized, paralyzed animals, (Horn 1965; Murata et al. 1965; Bental et al. 1968; Spinelli et al. 1968; Morrell 1972; Fishman and Michael 1973) have been cited (e.g., Bavelier and Neville 2002). Because these reports fundamentally conflict with the lack of bimodal neurons in the present study, a close examination of the earlier studies seems justified.

In cat visual cortex (Areas 18/19), Morrell (1972) reported that 41% of the neurons sampled showed suprathreshold activation by acoustic stimulation, that every one of the acoustically responsive neurons had its auditory receptive field in alignment with the accompanying visual receptive field, and that the observed auditory receptive fields lacked inhibitory surrounds. However, although no measure of auditory response latency was provided, response latencies averaging 118 ms can be calculated from the scales in the published figures. These values are in stark contrast to the average response latencies in cat primary auditory cortex (12.5 ± 2.4 ms; Phillips and Irvine 1982), AAF (10–15 ms; Phillips and Irvine 1982), dorsal zone of auditory cortex (22 ms; Stecker et al. 2005), posterior auditory field (30.8 ± 10.7 ms; Phillips and Orman 1984), and the field of the anterior ectosylvian sulcus (FAES; 12.6 ± 2.3 ms; Meredith and Clemo 1989), none of which had any examples showing a latency in excess of 75 ms. Even if the projection from auditory to visual cortices is feedback in nature (Falchier et al. 2002; Rockland and Ojima 2003), a conduction time along this pathway in excess of 100 ms is not physiological. Furthermore, the spatial dimensions of the reported auditory receptive fields are uncharacteristically small: the receptive fields depicted in Figure 2 of Morrell (1972) were only about 7.5 degrees in azimuth. Nowhere else in the cortex have such restricted auditory receptive fields been identified in contralateral auditory space. Therefore, a skeptical review of these early data would not strongly support the designation of the reported auditory-evoked activity in visual cortex as representative of an auditory sensory response.

The auditory-evoked activity described by these early studies more likely correlate with other physiological processes that were not sufficiently understood at that time to be considered. For example response latencies in the 100-ms range and activation fields in correspondence with accompanying visual receptive fields closely match reports of sensory-evoked movement-related activity in the SC (Mays and Sparks 1980; Jay and Sparks 1987; Groh and Sparks 1996) and cortex (e.g., Bruce and Goldberg 1985) and may be indicative of corollary discharges related to the suppression of the retinal image slip that occurs during saccadic eye movements (Adey and Noda 1973; Duffy and Burchfiel 1975; Mohler and Cechner 1975). In addition, because eye movements can be electrically evoked by stimulation of the visual cortex (McIlwain 1988) and the fact that the long-latency auditory-evoked activity in visual cortex has not been replicated in anesthetized animals supports the notion that Morrell's recordings were from visual cortical neurons receiving information related to the programing of (attempted) movements to foveate the stimulus. If this notion is true, when the movement (attempt) is blocked (e.g., by anesthesia), true sensory responses should remain regardless of the modality of the stimulation. In fact, Murata et al. (1965) reported that “responses evoked by acoustic and somesthetic stimuli both disappeared…” with low doses of anesthetic, whereas visual responsiveness remained. At the very least, these early reports of suprathreshold auditory activity in visual cortex should be regarded with tempered skepticism.

Alternative Possibilities for Lack of Bimodality

The presence of a cross-modal projection to Area 21 without the identification of bimodal neurons seems to contradict the generally accepted notions about the basis for multisensory processing; yet, there might be methodological factors that could account for the absence of this neuron type from the present sample. One such possibility is the presence of anesthesia, which eliminates the effects of attention, affect, movement, and other dynamical influences on sensory responses. However, the present ketamine/acepromazine anesthetic regimen seems to uncover, not block, multisensory activity, especially in structures like the SC which exhibits broad motor-related sensory suppression in awake, behaving animals (Bell et al. 2003). In addition, studies of sensory cortices have shown that responses under light gas anesthesia, but not barbiturates, maintain many of the properties of awake responses (reviewed in Moshitch et al. 2006). In fact, the present anesthetic regimen has successfully revealed bimodal responses in other cortical areas as well as subtle subthreshold multisensory effects (Wallace et al. 1992, 1993; Dehner et al. 2004; Meredith et al. 2006; Bizley et al. 2007; Clemo et al. 2007). Therefore, unless Area 21 is differentially sensitive to ketamine/acepromazine, it does not seem likely that this drug combination selectively blocked the activation of bimodal neurons. Obviously, however, “how” the “unanesthetized” Area 21 uses cross-modal information to guide behavior and perception is a completely different issue from whether that information is present or not.

Alternatively, an entire subpopulation of multisensory neurons may have gone unobserved due to electrode bias. This situation, in fact, has been examined and resolved for another region of cortex: somatosensory Area SIV (Dehner et al. 2004). In that study, although auditory stimuli presented alone were ineffective, somatosensory responses in nearly 70% of the neurons were significantly suppressed by concurrent auditory stimulation. This effect was based upon auditory inputs synapsing on somatosensory inhibitory interneurons, whose activity went undetected unless combined auditory–somatosensory stimuli were presented or pharmacological treatment with a GABA antagonist blocked the suppressive effect. Cross-modal suppressive effects have also been observed in the auditory FAES (Meredith et al. 2006). Because similar methods were used in the present study, the possibility of missing a subgroup of neurons seems likely for those Area 21 neurons that showed cross-modal suppression. In these cases, it would appear that auditory inputs to visual inhibitory interneurons went undetected, but their indirect effect was to suppress the activity of 5.7% of the identified principal visual neurons. However, the population of Area 21 as a whole showed no overall cross-modal effect. Given the well-known and extensive local interconnections of cortical neurons, if a sizeable population of bimodal neurons were present, it seems unlikely that their indirect effect on their neighbors would go undetected in the remaining 94.3% of the sample.

Yet another possibility is that the auditory stimuli used, which were white noise bursts (shaped rise–fall, 70-dB SPL, 50-ms duration), might be inadequate to activate the auditory projection to visual Area 21. As shown by Bizley et al. (2005), many ferret auditory cortical neurons are exquisitely tuned to specific tones, especially at low intensities (e.g., 20- to 30-dB SPL). Nevertheless, some auditory neurons were activated by a broad frequency ranges at 70- to 80-dB SPL (see Bizley et al. 2005, Figs 3, 5, and 6). In addition, these same white noise stimuli were effective in producing both suprathreshold as well as subthreshold multisensory effects in other cortical regions (Dehner et al. 2004; Meredith et al. 2006; Clemo et al. 2007). Furthermore, in the presence of the GABA antagonist BIC, approximately 80% of the Area 21 neurons showed increased responses to combined auditory–visual stimulation. Therefore, it seems unlikely in this case that white noise stimuli were inadequate to elicit an effect.

Finally, that the observed multisensory interactions were so minimal because the size of the projection was so small correspond with data from Falchier et al. (2002), who estimated the size of auditory cortical connections to primate visual cortex to represent less than 1% of the total projection to the region. Furthermore, given that visual responses have been recently identified in ferret auditory cortex (Bizley et al. 2007), it might be possible that the observed projection may merely convey visual information back toward its source. This last possibility, however, is not supported by the results of bicuculline application, which ultimately revealed subthreshold auditory influences on visual responses of Area 21 neurons.

Measures of Multisensory Processing

Traditionally, multisensory processing effects at the neuronal level have been measured in terms of spike counts (or some variation thereof) and multisensory integrative effects have been defined as significant changes between unimodal and multisensory responses. The present study adheres to that established paradigm. Yet, the problem remains that the demonstrated cross-modal projection yielded few examples that met the criterion for multisensory integration. Related problems have also been reported recently for multisensory responses in cat rostral suprasylvian cortex, where bimodal neurons generally failed to integrate multisensory cues (Clemo et al. 2007). These observations suggest that subtle response changes may occur but pass undetected by traditional spikecounting measures. Accordingly, changes in temporal response features or changes in information content have recently been tested for multisensory responses in auditory cortices (see Bizley et al. 2007). Furthermore, cross-modal projections can result in phase resetting of neuronal activity, such as observed in primate auditory cortex (Lakatos et al. 2005, 2007). Collectively, these and other similar studies present a convincing argument that cross-modal projections can lead to multisensory processing effects that fall outside the measurement criteria established for suprathreshold, bimodal effects.

Cross-Modal Convergence and Subthreshold Multisensory Integration

The notion that convergence of inputs from different sensory modalities onto individual neurons results in multisensory integration has received considerable support since its initial documentation in the 1980s (e.g., Newman and Hartline 1981; Meredith and Stein 1983, 1986; King and Palmer 1985). Multisensory integration at the neuronal level has been defined as the significant response change evoked by combined-modality stimulation when compared with that elicited by effective single-modality stimuli (Meredith and Stein 1983, 1986). This criterion for multisensory integration was met by the 9% of visual Area 21 neurons that were significantly influenced by the presence of an auditory stimulus. Furthermore, when local inhibition was pharmacologically blocked, approximately 87% of Area 21 neurons participated in a significant change in population response evoked by combined-modality stimulation. Thus, the cross-modal projections from ferret auditory cortex to visual Area 21 apparently do result in multisensory integration, albeit in a subthreshold, not suprathreshold form. The term subthreshold is necessary for this class of multisensory neurons because although 1 modality of inputs fails to elicit suprathreshold responses when presented alone, that modality significantly influences spiking activity when other effective inputs are present. Subthreshold forms of multisensory processing have been reported for other unimodal cortices, whereby auditory stimulation suppressed responses in 66% of the neurons from somatosensory area SIV (Meredith 2002; Dehner et al. 2004), somatosensory stimulation suppressed responses in 25% of the neurons in the auditory FAES (Meredith et al. 2006), auditory cues facilitated responses of 16% of visual neurons in the posterolateral lateral suprasylvian visual area (Allman and Meredith 2007), and visual stimulation facilitated responses of neurons in approximately 4–47% of the neurons in a variety of auditory cortices (Bizley et al. 2007). In addition, subthreshold multisensory integration has also been observed in traditionally multisensory areas such as the optic tectum (Newman and Hartline 1981), prefrontal cortex (Sugihara et al. 2006), and the STS (Barraclough et al. 2005).

Given the presence of subthreshold multisensory neurons and the robustness of their processing effects, it seems possible that they represent a part of a continuum of cross-modal phenomena between purely unimodal and suprathreshold (bimodal or trimodal) integrative responses. Unlike the dramatic levels of response activity produced by the suprathreshold forms of multisensory integration that have been shown to underlie detection and orientation behaviors (for review, see Stein and Meredith 1993), it seems likely that subthreshold multisensory processing mitigates effects at the modulatory end of the multisensory spectrum, perhaps in ways that finely adjust the quality of perceptions. In this manner, subtle but consistent cross-modal modulations may serve to enhance signal-noise functions in networks that would be overwhelmed (or disrupted) by the stronger response gains (or losses) mediated by well-known suprathreshold integrative effects. From another perspective, a network of cross-modal subthreshold projections may also provide a viable and immediately available substrate for the reorganization of cortical sensory representations observed following unimodal sensory loss or deprivation (Rauschecker et al. 1992; Rauschecker and Korte 1993; Rauschecker 1995; Bavelier and Neville 2002).

Conclusion

These results support the premise that multisensory convergence leads to multisensory integration, but the neuronal substrate for such an effect may not be that which is currently assumed or expected. Despite the presence of cross-modal projections from auditory cortex to Area 21, the traditional form of multisensory neuron, the bimodal neuron, was not observed. Instead, subthreshold multisensory neurons were identified through combined-modality stimulation and by pharmacological blockade of local inhibition. Therefore, in this projection, multisensory convergence leads to subthreshold multisensory integration, an observation that expands our understanding of the range and effects of multisensory processing itself.

Funding

National Institutes of Health Grant (NS-39460 to M.A.M. and NIAAA R01-13023 to A.E.M); Natural Sciences and Engineering Research Council of Canada (to B.L.A.); Coordenação de Aperfeiçoamento de Pessoal de Nível Superior of Brazilian Ministry of Education (to R.E.B.N.).