Skip to main content
PMC home page
The Journal of Neuroscience logoLink to The Journal of Neuroscience
. 2010 May 19;30(20):6991–6998. doi: 10.1523/JNEUROSCI.5723-09.2010

Interactions between Stimulus-Specific Adaptation and Visual Auditory Integration in the Forebrain of the Barn Owl

Amit Reches 1, Shai Netser 1, Yoram Gutfreund 1,✉
PMCID: PMC6632650  PMID: 20484641

Abstract

Neural adaptation and visual auditory integration are two well studied and common phenomena in the brain, yet little is known about the interaction between them. In the present study, we investigated a visual forebrain area in barn owls, the entopallium (E), which has been shown recently to encompass auditory responses as well. Responses of neurons to sequences of visual, auditory, and bimodal (visual and auditory together) events were analyzed. Sequences comprised two stimuli, one with a low probability of occurrence and the other with a high probability. Neurons in the E tended to respond more strongly to low probability visual stimuli than to high probability stimuli. Such a phenomenon is known as stimulus-specific adaptation (SSA) and is considered to be a neural correlate of change detection. Responses to the corresponding auditory sequences did not reveal an equivalent tendency. Interestingly, however, SSA to bimodal events was stronger than to visual events alone. This enhancement was apparent when the visual and auditory stimuli were presented from matching locations in space (congruent) but not when the bimodal stimuli were spatially incongruent. These findings suggest that the ongoing task of detecting unexpected events can benefit from the integration of visual and auditory information.

Introduction

Stimulus-specific adaptation (SSA) is the ability of neurons to adapt to high probability stimuli while maintaining a relatively strong response to low probability stimuli. This phenomenon has been suggested as a neural correlate for the detection of unexpected stimuli (i.e., change detection, also known as novelty detection) (Ulanovsky et al., 2003; Nelken and Ulanovsky, 2007; Malmierca et al., 2009). SSA is common in the brain and has been observed in visual (Müller et al., 1999), auditory (Pérez-González et al., 2005; Reches and Gutfreund, 2008), and somatosensory (Katz et al., 2006) pathways, yet none of the studies cited has involved more than one modality.

The everyday environment provides an influx of auditory and visual information that is often congruent and complementary. In humans and other animals, visual and auditory information is integrated in the brain to facilitate the perception of unitary events that can be both seen and heard (Frassinetti et al., 2002; Jiang et al., 2002; Narins et al., 2005; Whitchurch and Takahashi, 2006). This facilitation has been correlated with the responses of multisensory neurons in midbrain structures involved in attentive and orienting behaviors (King and Palmer, 1985; Stein et al., 1988; Stein and Meredith, 1993). It is reasonable to assume that cross-modal integration can also facilitate the ability to specifically detect and respond to unexpected events. Thus, we predicted a link between cross-modal facilitation and change detection, two themes that have each been widely explored separately. Such a link is expected to be manifested in the modulation of SSA by bimodal stimuli.

In the present study, experiments were performed in the entopallium (E), the forebrain recipient of the avian tectofugal pathway (Karten and Hodos, 1970). The tectofugal pathway is a visual pathway that exists in all mammals and birds (Karten, 1969; Shimizu and Bowers, 1999), and it is suggested to play a role in bottom-up attention control and visual saliency mapping (Robinson and Petersen, 1992; Shipp, 2004; Marín et al., 2007). Recently, it has been reported in barn owls that neurons in the E can respond to auditory as well as visual signals (Reches and Gutfreund, 2009). Thus, the E provides an interesting opportunity to explore the multisensory SSA hypothesis. Our results show that the SSA effect, defined as the difference between the response to a stimulus when it is rare and the response to the same stimulus when it is frequent, is markedly enhanced when both visual and auditory streams deviate simultaneously. We further demonstrate that this effect requires the bimodal sequence to be congruent in space as well as in time. These findings are consistent with the hypothesis that the ongoing task of detecting deviants benefits from the integration of visual and auditory information.

Materials and Methods

Five barn owls (Tyto alba) were used in this study. All owls were hatched in captivity, raised, and kept in a large flying cage equipped with perching spots and nesting boxes. The owls were provided for in accordance with the guidelines of the Technion Institutional Animal Care and Use Committee.

Electrophysiological measures.

Owls were prepared for repeated electrophysiological experiments in a single surgical procedure. A craniotomy was performed, and a recording chamber was cemented to the skull. At the beginning of each recording session, the owl underwent anesthesia using isoflurane (2%) and nitrous oxide in oxygen (4:5). Once anesthetized, the animal was positioned in a stereotaxic apparatus at the center of a double-wall sound-attenuating chamber. The head was bolted to the stereotaxic apparatus and aligned using retinal landmarks (Wathey and Pettigrew, 1989). Within the chamber, isoflurane was removed and the bird was maintained on a fixed mixture of nitrous oxide and oxygen (4:5). A glass-coated tungsten microelectrode (1–2 MΩ; NAN Biosystems) was driven into the recording chamber using a motorized manipulator (Narishige SM-191). An MPC Plus-8 System (Alpha Omega) was used to amplify and filter the signal (350–4000 Hz). Multiunit recordings were obtained by manually setting a threshold and consistently selecting the largest unit waveforms. The highly bursty activity characterizing the E precluded the isolation of single-unit templates (Reches and Gutfreund, 2009). Therefore, we could not tell in confidence to what extent the recorded activity reflected a single neuron or a small cluster of neurons near the tip of the electrode.

Targeting of the E.

To target the E, we determined the position in the optic tectum (OT) corresponding to the visual receptive field of zero azimuth and zero elevation (directly in front). From this point, the electrode was advanced 2 mm rostrally, 0.4 mm laterally, and 6.5–7 mm dorsally. In a previous study, anatomical reconstructions have confirmed the correspondence of these stereotaxic coordinates and of the typical bursting activity with recording sites within the borders of the E (Reches and Gutfreund, 2009).

Stimuli presentation.

Auditory stimuli generated on a computer (custom MATLAB programs) were converted to analog (48 kHz sampling rate, RP2; Tucker Davis Technologies) and transduced by a pair of matched miniature earphones (Knowles ED-1914). The earphones were placed in the center of the ear canal ∼8 mm from the tympanic membrane. The amplitude and phase spectra of the sound were equalized within ±2 dB and ±2 μs by computer adjustment of the stimulus waveform. Sound levels were controlled by two independent attenuators (PA5; Tucker Davis Technologies). Stimuli consisted of broadband (2–12 kHz) noise presented at an interstimulus interval (ISI) of 1 s and a duration of 300 ms. The average binaural sound intensity was maintained at 10–15 dB above the threshold of the units.

Visual stimuli were generated using the MATLAB Psychophysics Toolbox extensions (Brainard, 1997; Pelli, 1997) and projected (refresh rate, 72 Hz; XD400U; Mitsubishi) on a calibrated screen inside the sound attenuating chamber (screen size, 150 × 115 cm, 1.5 m away from the owl). The projector was positioned outside the chamber, projecting the image through a double-glass window. The visual stimulus was a moving dark dot on a gray background (1.9° diameter; luminance of background screen was 6.16 cd/m2, and luminance of dot was 1.02 or 2.74 cd/m2). The stimulus appeared on the screen at onset and moved a short distance of ∼4° upward for a duration of 300 ms and disappeared.

Oddball stimulation protocol.

For each test, two visual, two auditory, and two bimodal stimuli were first selected. The visual stimuli differed only in the azimuth of the horizontal position. The stimuli were set 20° apart, both within the contralateral field. The positions of the first and second stimuli were between 0–10° and 20–30° contralateral to the recording side, respectively. The elevation was held constant at near-zero elevation. The two auditory stimuli differed in the interaural time difference (ITD). The ITDs were set to match the horizontal positions of the visual stimuli: ITD (in μs) = 2.5 * azimuth (in degrees) (Brainard and Knudsen, 1993). The interaural level difference was maintained at +5 dB (right ear louder) to match zero elevation (Olsen et al., 1989). In the bimodal condition, a combination of one of the visual stimuli and one the auditory stimuli was presented simultaneously. Bimodal stimulation was either congruent (i.e., the visual stimulus was presented together with its spatially matched auditory stimulus) or incongruent (i.e., the visual stimulus was presented together with the spatially non-matched auditory stimulus).

Most oddball tests consisted of six blocks: two visual, two auditory, and two congruent bimodal blocks (see Fig. 1A). A block consisted of a sequence of 100 stimuli, each stimulus with a duration of 300 ms and an ISI of 1 s. In each block, one of the stimuli was defined as the rare stimulus and the other as the common stimulus (frequent). The probability of occurrence of the rare and frequent stimulus was 15 and 85%, respectively. In the next experimental block, the roles of the frequent and rare stimuli were reversed. During the experiments, blocks were randomly interleaved and each was presented twice. At the end of the test, we compared the neural responses to the rare occurrences of the stimuli with the responses to the frequent occurrences of the same stimuli. In 32 tests, an additional two incongruent bimodal blocks were added.

Figure 1.

Figure 1.

Open in a new tab

Comparison between responses to rare and frequent presentations. A, Outline of the bimodal oddball test. Each line represents a sample sequence from a single oddball block. Squares designate the auditory stimuli, circles designate the visual stimuli, and filled squares designate the bimodal stimuli. Gray filled shapes specify the peripheral position (Loc 1), and black shapes specify the central position (Loc 2). Bold arrows point to examples of rare occurrences of auditory (blue), visual (red), and bimodal (cyan) stimuli. Light arrows point to corresponding examples of frequent occurrences. B–G, PSTHs of responses to rare occurrences (green lines) are shown together with the corresponding responses to frequent occurrences (blue lines). Responses to the two auditory stimuli are shown in B,C, the two visual stimuli in D,E, and the two bimodal stimuli in F,G. Results from stimuli at the peripheral location are presented on the left, and results from stimuli at the center are presented on the right. Gray filled squares mark stimulus onset and duration. H–K, The PSTHs of responses to the two bimodal stimuli (cyan lines) are overlaid with the PSTHs of responses to their visual (red) and auditory (blue) components. Comparison is performed twice, once for rare occurrences (H, J) and once for frequent occurrences (I, K).

Data analysis.

For each stimulus in the sequence, spikes were registered in a time window starting 200 ms before stimulus onset and ending 500 ms after stimulus onset. The number of spikes in the prestimulus window (200 ms before the stimulus) multiplied by 2.5 was subtracted from the number of spikes in the poststimulus window (0–500 ms). The resulting value was averaged across all the rare and all the frequent stimuli in a single block to give the responses to rare and to frequent, respectively. Single-test poststimulus time histograms (PSTHs) with 2-ms-wide bins were smoothed for display purposes (running average window of 9 bins) (see Figs. 1B–K, 4A–H, 5A–D). To demonstrate the population difference between the responses to rare and frequent presentations of stimuli, we calculated the average novelty response: we first normalized the PSTHs in each experiment relative to the maximal value achieved in the experiment. The normalized PSTHs of the rare and the frequent responses were then averaged across the entire population and subtracted from one another [novelty_response(t) = deviant(t) − frequent(t)] (see Fig. 4I).

Figure 4.

Figure 4.

Open in a new tab

Results from incongruent bimodal oddball tests. A–H, An example from a single recording site. PSTHs of responses to rare occurrences (green lines) are shown together with the corresponding responses to frequent occurrences (blue lines). Results are presented for auditory (A, B), visual (C, D), congruent bimodal (E, F), and incongruent bimodal (G, H) stimuli. Left column shows average responses to the visual stimuli in the peripheral location (Loc 1), and right column shows responses to the visual stimuli in the central location (Loc 2). I, The novelty response of visual (red), congruent bimodal (cyan), and incongruent bimodal (purple) stimulation. J, A histogram showing the distribution of all differences between the congruent bimodal NIs and the visual NIs. K, Same as J but showing results from the incongruent bimodal tests.

Figure 5.

Figure 5.

Open in a new tab

Comparison between responses to strong and normal visual stimuli. A–D, An example from a single recording site. PSTHs of responses to rare occurrences (green lines) are presented together with the corresponding responses to frequent occurrences (blue lines). Results are presented for normal visual stimulation (A, B) and strong visual stimulation (C, D). E, Scatter plot of EIs (computed as the enhancement of responses to strong visual stimuli over the responses to normal visual stimuli) measured from frequent occurrences (x-axis) versus the corresponding EIs measured from rare occurrences (y-axis). The diagonal line is the line of equality. The inset compares the population average PSTHs for the responses to rare occurrences of the strong visual stimuli (red trace) and the normal visual stimuli (blue trace). F, The distribution of the differences between the NI of the strong visual stimuli and the NI of the normal visual stimuli for all sites. Loc 1, Responses to the visual stimuli in the peripheral location; Loc 2, responses to the visual stimuli in the central location.

Results

SSA in unimodal and bimodal stimuli

We recorded neural responses from 60 sites in the E. At each site, an oddball paradigm was used to characterize SSA of auditory, visual, and congruent bimodal stimuli. An example from one recording site in the right E is shown in Figure 1B–G. In this example, the visual stimuli were positioned one at left 20° (Loc 1, contralateral stimulus) and the other at 0° (Loc 2, central stimulus). The ITDs of the two auditory stimuli were set to match the locations of the two visual stimuli (−50 μs left ear leading and 0 μs, respectively). The two bimodal stimuli consisted of the spatially matching pairs of auditory and visual stimuli (−50 μs with left 20°, 0 μs with 0°). In this example, the recording site did not respond to the two auditory stimuli regardless of whether they were rare or frequent (Fig. 1B,C). The same site did respond to both rare and frequent visual occurrences (Fig. 1D,E) (t test, p < 0.05). The responses to the rare occurrences, however, were noticeably stronger than the responses to the same stimuli when frequent (compare green lines with blue lines). Thus, SSA to the location of the visual stimulus was apparent in this site. The results from the bimodal oddball blocks are shown in Figure 1, F and G. Here, once again, a robust SSA effect was observed. Interestingly, however, the difference between the rare and frequent responses was clearly larger in the bimodal test than in the visual test (Fig. 1, compare F,G with 1D,E). Thus, the visual SSA effect was enhanced by the presence of congruent auditory stimuli.

Figure 1H–K shows the same average responses (PSTHs) shown in Figure 1B–G, compared in a different manner. The bimodal stimuli (cyan) are shown together with the average responses to their corresponding unimodal components (visual in red and auditory in blue). This display demonstrates that, despite the fact that there were no apparent responses to the rare auditory occurrences in this recording site, the responses to the rare bimodal stimuli were larger than the responses to the rare visual stimuli (Fig. 1H,J). As a result of the SSA, the visual and bimodal responses to frequent stimuli were markedly reduced (Fig. 1I,K) but were not completely abolished (both visual and bimodal stimuli induced significant responses to frequent occurrences; t test, p < 0.05). Interestingly, multisensory enhancement was not apparent in the responses to the frequent occurrences. Thus, the multisensory enhancement depended on the context of the stimulus. Such context dependency implies that the multisensory SSA is stronger than the unimodal SSA, as is demonstrated in Figure 1B–G (for a mathematical support of this statement, see supplemental data, available at www.jneurosci.org as supplemental material).

A summary of the results of the oddball tests from all sites is shown in Figure 2. For each condition (auditory, visual, and bimodal), the two stimulus indices (SI1 and SI2) and a single neural index (NI) were computed (Ulanovsky et al., 2003). SI1 is defined as SI1 = (r1 − f1)/(r1 + f1), where r1 and f1 are the responses to stimulus 1 when presented as rare and frequent, respectively. SI2 is the same as SI1 but for the other stimulus. A positive SI indicates a stronger response to the deviant than to the frequent. The NI is defined as NI = (r1 − f1 + r2 − f2)/(r1 + r2 + f1 + f2). This index combines the results from the two stimuli, quantifying the tendency of the neurons to better respond to rare occurrences. The left column in Figure 2 presents scatter plots of SI1 versus SI2 calculated from the auditory (Fig. 2A), the visual (Fig. 2C), and the congruent bimodal (Fig. 2E) responses. A point appearing in the top left quadrant (within the two dashed lines) implies stronger responses to the rare occurrence of both stimuli. However, as noted by Ulanovsky et al. (2003), any point above the diagonal line implies a tendency for SSA. In the auditory condition, the dot distribution was not significantly biased above the diagonal line (Fig. 2A) (sign test, p > 0.05). The low level of auditory SSA in the population is not surprising considering the relatively weak auditory responses observed in most cases. Results from the visual stimuli, conversely, were significantly distributed above the diagonal line (Fig. 2C) (sign test, p < 0.01), showing a clear population tendency for SSA. The results from the bimodal tests were also distributed above the diagonal line (Fig. 2E) (sign test, p < 0.01). In this case, however, almost all dots appeared within the top left quadrant (dashed lines) and further above the diagonal line, indicating a stronger net SSA effect in the bimodal condition compared with the visual-only condition. This result was also apparent in the NI distributions (Fig. 2, right column). No significant tendency to positive values (t test, p > 0.05) was observed in the auditory condition (Fig. 2B), whereas the NI distribution of the visual condition (Fig. 2D) was significantly biased to positive values (t test, p < 0.01). The distribution of the bimodal condition (Fig. 2F) was further shifted toward positive values compared with the distribution of the visual condition (Mann–Whitney test, p < 0.01). We analyzed the differences between the NIs of the bimodal condition and the NIs of the visual condition within the same sites. The distribution of the differences was significantly shifted to the positive side (Fig. 3A) (one tailed t test, p < 0.01). Thus, in most recording sites, the visual SSA was enhanced by the presence of auditory stimuli.

Figure 2.

Figure 2.

Open in a new tab

Summary of results from all recording sites. A, A scatter plot showing SI1 versus SI2 measured from the responses to the auditory sequences. The dashed lines mark the first quadrant, in which both SIs are positive. The diagonal line marks the nonspecific adaptation line (see Results). B, A histogram showing the distribution of the NIs measured from all auditory oddball tests. The dashed vertical line indicates NI = 0. C, D, Results from all visual oddball tests. E, F, Results from all bimodal oddball tests. Format as in A and B.

Figure 3.

Figure 3.

Open in a new tab

Interaction between SSA and multisensory integration. A, The difference between the bimodal NI and the visual NI was measured for each site. The histogram shows the distribution of all differences. B, Scatter plot of EIs measured from frequent occurrences (x-axis) versus the corresponding EIs measured from deviant occurrences (y-axis). The diagonal line shows the line of equality.

Multisensory enhancement in rare versus frequent occurrences

In the single-site example shown in Figure 1, it is clear that the phenomenon of multisensory enhancement depended on the context of the stimulus. To assess the context dependency of the multisensory enhancement at the population level, we calculated the enhancement indices (EIs) (Stein and Meredith, 1993), defined as EI = (AV − max(A,V))/max(A,V), where AV is the response to the bimodal stimulus, A is the response to the auditory component, and V is the response to the visual component. The EIs were calculated for the two bimodal stimuli, once when rare and once when frequent. Figure 3B summarizes these results, with each dot designating the EI of the rare stimuli (y-axis) versus the EI computed from the same stimuli when frequently presented (x-axis). The majority of the dots appeared above the diagonal line (92 of 120; sign test, p < 0.01), indicating a significant tendency for stronger multisensory enhancement in rare occurrences relative to frequent occurrences. Multisensory enhancement at the population level was significant in rare occurrences (distribution of dots along the y-axis; sign test, p < 0.01) but nonsignificant in frequent occurrences (distribution of dots along the x-axis; sign test, p > 0.05).

Because our recordings were multiunit (see Materials and Methods), a concern is that the multisensory enhancement identified in this study arises from electrode summation of independent visual and auditory streams (as would be expected if the visual and auditory responses are carried by distinct intermixed populations of neurons). However, the population multisensory SSA was enhanced despite the fact that the neurons showed no auditory SSA (Fig. 2), a result that cannot be explained based on summation of responses from distinct populations of neurons. Moreover, as can be seen in the single-site example, the rare congruent bimodal response (Fig. 1H,J) exceeded the sum of the unimodal responses. Thus, the enhancement most likely reflects interactions in multisensory neurons (for more evidence for nonlinear multisensory interactions in the E, see Reches and Gutfreund, 2009).

Spatial incongruency

The response to a rare stimulus in SSA has been related to the amount of change from the expected (frequent) sensory scene (Nelken and Ulanovsky, 2007). The multisensory enhancement of SSA shown above may accordingly reflect the additional change imposed on the multisensory scene in which both visual and auditory streams deviate simultaneously. If it is only the amount of deviation from the expected that leads to enhanced SSA, an effect should occur whether the change is spatially congruent (i.e., visual and auditory stimuli shifting together from one location to another) or not. To test this, we presented the three oddball blocks described in Figure 1A (auditory, visual, and congruent bimodal) together with a fourth incongruent bimodal oddball block. This test was repeated in 32 recording sites. Similar to the congruent block, the rare event differed from the frequent by a change in the locations of the auditory and visual stimuli, yet to opposite locations. An example from a single recording site is presented in Figure 4A–H. Consistent with the results presented above (Figs. 1, 2), the congruent bimodal condition (Fig. 4E,F) gave rise to a larger SSA effect than the SSA effect obtained in the visual (Fig. 4C,D) or auditory (Fig. 4A,B) conditions. Conversely, the incongruent bimodal condition did not achieve the same SSA effect as the congruent condition (Fig. 3, compare G,H with E,F); results of the incongruent bimodal condition more closely resembled results from the visual-only condition.

The benefit of bimodal congruency was evident at the population level as well. The novelty response (the difference between the average response to the deviant and the frequent occurrences) of the congruent bimodal condition (Fig. 4I, cyan curve) was substantially enhanced over the visual novelty response (Fig. 4I, red curve), whereas the novelty response of the incongruent bimodal condition (Fig. 4I, purple curve) was only slightly enhanced. It is interesting to note that the average peak of both bimodal conditions appeared before the average peak of the visual condition. This was consistent with a previous study showing significant shortening of the times-to-peak of the bimodal responses in the E (Reches and Gutfreund, 2009). Figure 4, J and K, shows the differences between the bimodal and visual NI for all tests. In the congruent bimodal condition (Fig. 4J), the distribution was significantly shifted to positive values (t test, p < 0.01). Conversely, the distribution of the differences from the incongruent bimodal test (Fig. 4K) was not significantly biased from zero (t test, p > 0.05), indicating no net effect of auditory stimuli on the visual SSA.

SSA in strong versus weak visual stimuli

The enhanced SSA in bimodal conditions may be explained by an adaptation mechanism that is sensitive to the effectiveness of the stimulus: stimuli that give rise to stronger neural responses (for example, the congruent bimodal stimulus in our case) consequently give rise to stronger adaptation, independent of the modality of the input. Therefore, it is interesting to examine the effect of the intensity of the unimodal stimulus on SSA. For this purpose, we performed an additional set of experiments using a new oddball test composed of four randomly interleaved blocks. Here we tested only visual stimuli: two blocks displayed visual stimuli identical to the stimuli used in the previous tests (normal stimuli), and two blocks displayed the same two stimuli but at a higher contrast (strong stimuli; background, 6.16; and foreground, 1.02 cd/m2). A single-site example is shown in Figure 5A–D. The responses to the frequent stimuli were clearly suppressed relative to the responses to the rare stimuli in both normal and strong conditions. Despite the larger responses evoked by the rare strong stimuli (Fig. 5C,D, green lines compared with A,B, green lines), the SSA did not differ substantially between the two conditions. The NI of the normal and strong conditions was 0.44 and 0.45, respectively. This experiment was repeated in 56 sites. In each site, the enhancement indices of the responses to the higher contrast stimuli (strong stimuli) over the responses to the normal stimuli were calculated (Vs − Vn/Vn, where Vs is the response to the strong stimulus and Vn is the response to the normal stimulus from the same location). Figure 5E shows the EI of the rare stimuli versus the EI of the same stimuli when presented frequently. As expected, the strong rare stimuli gave rise to responses that tended to be above the responses to the normal rare stimuli (see inset). This tendency is statistically expressed in the distribution of the rare EIs, which was positively shifted (Fig. 5E, distribution of dots along the y-axis) (sign test, p < 0.01). Note that this effect is qualitatively comparable with the multisensory enhancement obtained in the previous experiment (Fig. 3B, distribution of dots along the y-axis); however, the median of the distribution here was 0.16, somewhat smaller than the median of the multisensory EI distribution of 0.25. We were not able to obtain additional enhancement of visual-only responses, most likely because of response saturation. To quantitatively compare between the SSAs under the two contrast levels, we analyzed the differences between the NIs of the strong condition and the NIs of the normal condition within the same sites. The distribution of the differences was not significantly shifted from zero (Fig. 5F) (t test, p > 0.05). This result was markedly different from the result obtained using the bimodal stimuli (compare Figs. 5F, 3A). Thus, our results do not support an SSA that is sensitive to the general effectiveness of the stimulus.

Discussion

Saliency mapping and the tectofugal pathway

We have shown SSA, which is considered to be a neural correlate of change detection (Nelken and Ulanovsky, 2007), to visual stimulation in the E. Change detection has been linked with the ability of an animal to abruptly attend to events that differ from their background (i.e., saliency mapping) (Tiitinen et al., 1994; Itti and Koch, 2001; Knudsen, 2007). Thus, this result may not come as a surprise considering the suggested involvement of the avian tectofugal pathway in visual saliency mapping (Marín et al., 2007). Recently, auditory responses have been identified in the barn owl's E (Reches and Gutfreund, 2009), raising the possibility that it is engaged in modality-independent processing. Here, we showed that the response properties of the E were not symmetrical between the two modalities: visual-only responses were consistently stronger and showed clear SSA effects. Auditory responses, conversely, were weak and at the population level showed no significant SSA effect (Fig. 2A,B). In some cases, auditory responses were not observed at all and were only exposed in bimodal conditions (for example, see Fig. 1). This asymmetry suggests that the E in barn owls is mainly involved in visual and not auditory processing. Because we did not systematically survey the E, we cannot rule out the possibility that it is functionally divided into subregions with different modality preferences, consistent with the idea that the tectofugal pathway is functionally segregated (Benowitz and Karten, 1976; Mpodozis et al., 1996; Karten et al., 1997).

Despite the relatively weak influence of auditory signals alone, congruent auditory stimuli presented simultaneously with the visual stimuli substantially enhanced the visual SSA effects. This intriguing result leads us to hypothesize that visual–auditory integration provides a mechanism for improving visual change detection when additional information from the auditory modality is made available.

The origin of this effect, shown in the E, is unknown. Recently, we have reported in tectal neurons that, in a monotonic sequence of sensory events, the first, less expected event usually gives rise to stronger multisensory enhancement compared with the subsequent events (Zahar et al., 2009). In addition, Bergan and Knudsen (2009) have reported enhanced modulation of auditory responses in the inferior colliculus of barn owls by rare visual stimuli (presumably via visual signals from the OT). These studies agree with the multisensory oddball effects reported here and thus suggest the possibility that what we identified in the E reflects computations occurring in the OT. Supporting this is the recent finding that the origin of the auditory signals in the E is the OT (Reches and Gutfreund, 2009). Accordingly, information about temporal saliency may be first computed in the OT (or the superior colliculus in mammals) to initiate rapid gaze responses. This information, however, should somehow be made available to the forebrain for the control of various covert orienting responses (Sokolov, 1963; Müller et al., 2005; Boehnke and Munoz, 2008). Our results, together with the study by Marín et al. (2007) that addressed spatial saliency, support the hypothesis that information regarding saliency is distributed to the forebrain through the tectofugal pathway. It remains an open question whether the saliency information is further refined along the tectofugal pathway.

Mechanisms of interaction between SSA and bimodal integration

In principle, two equally feasible models can explain the enhanced SSA observed in bimodal conditions. In the first model, the visual–auditory integration occurs before the specific adaptation (Fig. 6A). Here, we assume that, at lower levels, responses are enhanced each time congruent visual and auditory stimuli are presented together (multisensory enhancement). Because of this enhancement, the inputs to the higher levels, in which the specific adaptation takes place, are stronger in the bimodal condition compared with the visual condition. We assume that the auditory input to the bimodal integrator is weak, by itself unable to produce substantial responses at the output level. In this model, the simplest way to achieve the enhanced SSA in bimodal conditions would be through an adaptation mechanism that is sensitive to the strength of the response; stronger responses (bimodal condition in our case) would lead to stronger adaptation. Similarly, incongruent conditions that lead to smaller responses would induce smaller SSAs. One of the classical models of synaptic depression precisely obeys such a rule (Tsodyks and Markram, 1997). Conversely, a recent experimental study on adaptation in the thalamus and cortex of rats reported the contrary, i.e., adaptation that decreased with the intensity of the stimulus (Ganmor and Lampl, 2009).

Figure 6.

Figure 6.

Open in a new tab

Schematic representation of the two basic models. A, Visual and auditory inputs are integrated at the first stage. At a later stage, the bimodal responses undergo through an SSA filter. B, Each input channel, auditory or visual, is filtered separately through an SSA channel. The filtered responses are then fed to the multisensory integrator. The bars represent a sequence of frequent events (dark bars) with one rare event (light bar). The height of each bar represents the hypothetical responses at the various stages of the model.

The other possible mechanism is that SSA takes place at unimodal levels before the multisensory integration (Fig. 6B). Again, the auditory inputs are assumed to be weak. In this model, to account for our results, rare, non-adapted inputs should give rise to stronger multisensory enhancement compared with frequent, adapted inputs. This requirement seemingly opposes the common observation that less effective inputs give rise to stronger multisensory enhancement [i.e., “the principle of inverse effectiveness” (Meredith and Stein, 1986b; Perrault et al., 2005; Stanford et al., 2005)]; rare inputs are more effective in driving the neurons, yet they provide stronger enhancement (Figs. 1H–K, 3B). However, the inverse effectiveness principle has been demonstrated with stimuli that were less effective because of their low signal-to-noise ratio (Stanford et al., 2005). Here, the effectiveness of the stimuli was reduced as a result of adaptation. There may be a conceptual difference between the two that is manifested in the outcome of multisensory integration. How this differential integration may be achieved at the cellular level remains unknown. One possibility is that the adaptation weakens the auditory presynaptic inputs to a degree that renders the postsynaptic neurons practically unisensory for frequent responses but not for rare responses. The absence of the effect in the incongruent condition would be simply because of the lack of multisensory enhancement in these conditions (Reches and Gutfreund, 2009).

Both basic phenomena, unisensory SSA and enhancement of responses to congruent bimodal stimuli, have been reported in tectal neurons of the barn owl (Reches and Gutfreund, 2008; Zahar et al., 2009). Therefore, on the basis of the available physiological data, it is difficult to distinguish between the two models (described above and in Fig. 6). However, in the last experiment (Fig. 5), we report that higher contrast visual stimuli, which induced stronger responses at the population level, did not produce an SSA distribution that was significantly different from the SSA distribution obtained using the less effective visual stimulus (Fig. 5). This result is not consistent with SSA that is scaled with the effectiveness of the stimulus and therefore does not support the first possibility (Fig. 6A). It should be noted, however, that the response enhancement obtained through bimodality was, on average, larger than the response enhancement obtained through increasing the visual contrast (see Results). We therefore cannot rule out that yet higher responses will demonstrate stronger adaptation rates.

Context dependency of multisensory integration

To integrate visual and auditory information in an adaptive manner, it is important to distinguish between bimodal stimuli that are aligned in space (congruent events) and bimodal stimuli that are misaligned in space (incongruent events). Congruent bimodal stimuli are likely to arise from the same object and therefore provide complementary information, whereas incongruent bimodal stimuli are not and thus provide competing information. Consistent with this is the common observation that the presentation of congruent audiovisual stimulation leads to enhancement over the unimodal responses, whereas incongruent stimuli lead to no enhancement or even suppression compared with the unimodal responses (King and Palmer, 1985; Meredith and Stein, 1986a; Wallace et al., 1992; Reches and Gutfreund, 2009; Zahar et al., 2009). If the multisensory integration reported here plays a role in increasing the probability of detecting deviant objects, we expect the enhancement to appear when the information is likely to be complementary and to be absent when the information is not complementary. Consistent with this, the bimodal incongruent condition did not provide a net advantage over the visual-only condition. Conversely, it did not suppress the visual SSA either. Thus, auditory inputs affected the visual SSA only when the auditory stimulus was spatially congruent with the visual stimulus; otherwise, the auditory inputs had no apparent influence on the response properties of the neurons. This result again points to the dominance of visual processing in this brain region.

Multisensory enhancement is considered a hallmark property of cross-modal integration (Stein and Stanford, 2008). The majority of the studies in this field, however, use unitary presentations of bimodal stimulation, overlooking the effect of stimulus history. Here, we show that the history of the stimulus can strongly influence the level of its multisensory enhancement (Figs. 1H–K, 3B). This result highlights the importance of the context of the stimulus in the integration of visual and auditory information. Such a context-dependent multisensory effect, which is predicted by several EEG studies in humans (Schröger and Widmann, 1998; Besle et al., 2005), has significant implications for interpreting and modeling multisensory integration. An important consequence of this effect is that SSA can be markedly enhanced in multisensory settings, providing a neural mechanism for a link between change detection and multisensory integration.

Footnotes

This work was supported by the Focal Initiatives in Research in Science and Technology (FIRST) grant from the Israel Science Foundation and by the Joel Elkes grant from the National Psychobiology Institute in Israel (founded by the E. Smith family). We thank Prof. Israel Nelken for discussions and Felix Milman for technical support.

References

  1. Benowitz LI, Karten HJ. Organization of the tectofugal visual pathway in the pigeon: a retrograde transport study. J Comp Neurol. 1976;167:503–520. doi: 10.1002/cne.901670407. [DOI] [PubMed] [Google Scholar]
  2. Bergan JF, Knudsen EI. Visual modulation of auditory responses in the owl inferior colliculus. J Neurophysiol. 2009;101:2924–2933. doi: 10.1152/jn.91313.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Besle J, Fort A, Giard MH. Is the auditory sensory memory sensitive to visual information? Exp Brain Res. 2005;166:337–344. doi: 10.1007/s00221-005-2375-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Boehnke SE, Munoz DP. On the importance of the transient visual response in the superior colliculus. Curr Opin Neurobiol. 2008;18:544–551. doi: 10.1016/j.conb.2008.11.004. [DOI] [PubMed] [Google Scholar]
  5. Brainard DH. The psychophysics toolbox. Spatial Vision. 1997;10:433–436. [PubMed] [Google Scholar]
  6. Brainard MS, Knudsen EI. Experience-dependent plasticity in the inferior colliculus: a site for visual calibration of the neural representation of auditory space in the barn owl. J Neurosci. 1993;13:4589–4608. doi: 10.1523/JNEUROSCI.13-11-04589.1993. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Frassinetti F, Bolognini N, Làdavas E. Enhancement of visual perception by crossmodal visuo-auditory interaction. Exp Brain Res. 2002;147:332–343. doi: 10.1007/s00221-002-1262-y. [DOI] [PubMed] [Google Scholar]
  8. Ganmor E, Lampl I. Early stages of sensory processing counterbalance effects of short term synaptic depression during adaptation. Soc Neurosci Abstr. 2009;35:173–5. [Google Scholar]
  9. Itti L, Koch C. Computational modeling of visual attention. Nat Rev Neurosci. 2001;2:194–203. doi: 10.1038/35058500. [DOI] [PubMed] [Google Scholar]
  10. Jiang W, Jiang H, Stein BE. Two corticotectal areas facilitate multisensory orientation behavior. J Cogn Neurosci. 2002;14:1240–1255. doi: 10.1162/089892902760807230. [DOI] [PubMed] [Google Scholar]
  11. Karten HJ. Organization of avian telencephalon and some speculations on phylogeny of amniote telencephalon. Ann N Y Acad Sci. 1969;167:164–179. [Google Scholar]
  12. Karten HJ, Hodos W. Telencephalic projections of nucleus rotundus in pigeon (Columba livia) J Comp Neurol. 1970;140:35–51. doi: 10.1002/cne.901400103. [DOI] [PubMed] [Google Scholar]
  13. Karten HJ, Cox K, Mpodozis J. Two distinct populations of tectal neurons have unique connections within the retinotectorotundal pathway of the pigeon (Columba livia) J Comp Neurol. 1997;387:449–465. [PubMed] [Google Scholar]
  14. Katz Y, Heiss JE, Lampl I. Cross-whisker adaptation of neurons in the rat barrel cortex. J Neurosci. 2006;26:13363–13372. doi: 10.1523/JNEUROSCI.4056-06.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. King AJ, Palmer AR. Integration of visual and auditory information in bimodal neurones in the guinea-pig superior colliculus. Exp Brain Res. 1985;60:492–500. doi: 10.1007/BF00236934. [DOI] [PubMed] [Google Scholar]
  16. Knudsen EI. Fundamental components of attention. Annu Rev Neurosci. 2007;30:57–78. doi: 10.1146/annurev.neuro.30.051606.094256. [DOI] [PubMed] [Google Scholar]
  17. Malmierca MS, Cristaudo S, Pérez-González D, Covey E. Stimulus-specific adaptation in the inferior colliculus of the anesthetized rat. J Neurosci. 2009;29:5483–5493. doi: 10.1523/JNEUROSCI.4153-08.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Marín G, Salas C, Sentis E, Rojas X, Letelier JC, Mpodozis J. A cholinergic gating mechanism controlled by competitive interactions in the optic tectum of the pigeon. J Neurosci. 2007;27:8112–8121. doi: 10.1523/JNEUROSCI.1420-07.2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Meredith MA, Stein BE. Spatial factors determine the activity of multisensory neurons in cat superior colliculus. Brain Res. 1986a;365:350–354. doi: 10.1016/0006-8993(86)91648-3. [DOI] [PubMed] [Google Scholar]
  20. Meredith MA, Stein BE. Visual, auditory, and somatosensory convergence on cells in superior colliculus results in multisensory integration. J Neurophysiol. 1986b;56:640–662. doi: 10.1152/jn.1986.56.3.640. [DOI] [PubMed] [Google Scholar]
  21. Mpodozis J, Cox K, Shimizu T, Bischof HJ, Woodson W, Karten HJ. GABAergic inputs to the nucleus rotundus (pulvinar inferior) of the pigeon (Columba livia) J Comp Neurol. 1996;374:204–222. doi: 10.1002/(SICI)1096-9861(19961014)374:2<204::AID-CNE4>3.0.CO;2-6. [DOI] [PubMed] [Google Scholar]
  22. Müller JR, Metha AB, Krauskopf J, Lennie P. Rapid adaptation in visual cortex to the structure of images. Science. 1999;285:1405–1408. doi: 10.1126/science.285.5432.1405. [DOI] [PubMed] [Google Scholar]
  23. Müller JR, Philiastides MG, Newsome WT. Microstimulation of the superior colliculus focuses attention without moving the eyes. Proc Natl Acad Sci U S A. 2005;102:524–529. doi: 10.1073/pnas.0408311101. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Narins PM, Grabul DS, Soma KK, Gaucher P, Hödl W. Cross-modal integration in a dart-poison frog. Proc Natl Acad Sci U S A. 2005;102:2425–2429. doi: 10.1073/pnas.0406407102. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Nelken I, Ulanovsky N. Mismatch negativity and stimulus-specific adaptation in animal models. J Psychophysiol. 2007;21:214–223. [Google Scholar]
  26. Olsen JF, Knudsen EI, Esterly SD. Neural maps of interaural time and intensity differences in the optic tectum of the barn owl. J Neurosci. 1989;9:2591–2605. doi: 10.1523/JNEUROSCI.09-07-02591.1989. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Pelli DG. The VideoToolbox software for visual psychophysics: transforming numbers into movies. Spat Vis. 1997;10:437–442. [PubMed] [Google Scholar]
  28. Pérez-González D, Malmierca MS, Covey E. Novelty detector neurons in the mammalian auditory midbrain. Eur J Neurosci. 2005;22:2879–2885. doi: 10.1111/j.1460-9568.2005.04472.x. [DOI] [PubMed] [Google Scholar]
  29. Perrault TJ, Jr, Vaughan JW, Stein BE, Wallace MT. Superior colliculus neurons use distinct operational modes in the integration of multisensory stimuli. J Neurophysiol. 2005;93:2575–2586. doi: 10.1152/jn.00926.2004. [DOI] [PubMed] [Google Scholar]
  30. Reches A, Gutfreund Y. Stimulus-specific adaptations in the gaze control system of the barn owl. J Neurosci. 2008;28:1523–1533. doi: 10.1523/JNEUROSCI.3785-07.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Reches A, Gutfreund Y. Auditory and multisensory responses in the tectofugal pathway of the barn owl. J Neurosci. 2009;29:9602–9613. doi: 10.1523/JNEUROSCI.6117-08.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Robinson DL, Petersen SE. The pulvinar and visual salience. Trends Neurosci. 1992;15:127–132. doi: 10.1016/0166-2236(92)90354-b. [DOI] [PubMed] [Google Scholar]
  33. Schröger E, Widmann A. Speeded responses to audiovisual signal changes result from bimodal integration. Psychophysiology. 1998;35:755–759. [PubMed] [Google Scholar]
  34. Shimizu T, Bowers AN. Visual circuits of the avian telencephalon: evolutionary implications. Behav Brain Res. 1999;98:183–191. doi: 10.1016/s0166-4328(98)00083-7. [DOI] [PubMed] [Google Scholar]
  35. Shipp S. The brain circuitry of attention. Trends Cogn Sci. 2004;8:223–230. doi: 10.1016/j.tics.2004.03.004. [DOI] [PubMed] [Google Scholar]
  36. Sokolov EN. Higher nervous functions; the orienting reflex. Annu Rev Physiol. 1963;25:545–580. doi: 10.1146/annurev.ph.25.030163.002553. [DOI] [PubMed] [Google Scholar]
  37. Stanford TR, Quessy S, Stein BE. Evaluating the operations underlying multisensory integration in the cat superior colliculus. J Neurosci. 2005;25:6499–6508. doi: 10.1523/JNEUROSCI.5095-04.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Stein BE, Meredith MA. The Merging of the senses. Cambridge, MA: Massachusetts Institute of Technology; 1993. [Google Scholar]
  39. Stein BE, Stanford TR. Multisensory integration: current issues from the perspective of the single neuron. Nat Rev Neurosci. 2008;9:255–266. doi: 10.1038/nrn2331. [DOI] [PubMed] [Google Scholar]
  40. Stein BE, Huneycutt WS, Meredith MA. Neurons and behavior: the same rules of multisensory integration apply. Brain Res. 1988;448:355–358. doi: 10.1016/0006-8993(88)91276-0. [DOI] [PubMed] [Google Scholar]
  41. Tiitinen H, May P, Reinikainen K, Näätänen R. Attentive novelty detection in humans is governed by pre-attentive sensory memory. Nature. 1994;372:90–92. doi: 10.1038/372090a0. [DOI] [PubMed] [Google Scholar]
  42. Tsodyks MV, Markram H. The neural code between neocortical pyramidal neurons depends on neurotransmitter release probability. Proc Natl Acad Sci U S A. 1997;94:719–723. doi: 10.1073/pnas.94.2.719. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Ulanovsky N, Las L, Nelken I. Processing of low-probability sounds by cortical neurons. Nat Neurosci. 2003;6:391–398. doi: 10.1038/nn1032. [DOI] [PubMed] [Google Scholar]
  44. Wallace MT, Meredith MA, Stein BE. Integration of multiple sensory modalities in cat cortex. Exp Brain Res. 1992;91:484–488. doi: 10.1007/BF00227844. [DOI] [PubMed] [Google Scholar]
  45. Wathey JC, Pettigrew JD. Quantitative analysis of the retinal ganglion cell layer and optic nerve of the barn owl Tyto alba. Brain Behav Evol. 1989;33:279–292. doi: 10.1159/000115936. [DOI] [PubMed] [Google Scholar]
  46. Whitchurch EA, Takahashi TT. Combined auditory and visual stimuli facilitate head saccades in the barn owl (Tyto alba) J Neurophysiol. 2006;96:730–745. doi: 10.1152/jn.00072.2006. [DOI] [PubMed] [Google Scholar]
  47. Zahar Y, Reches A, Gutfreund Y. Multisensory enhancement in the optic tectum of the barn owl: spike count and spike timing. J Neurophysiol. 2009;101:2380–2394. doi: 10.1152/jn.91193.2008. [DOI] [PubMed] [Google Scholar]

Articles from The Journal of Neuroscience are provided here courtesy of Society for Neuroscience

RESOURCES