Impact of response duration on multisensory integration

Abstract

Multisensory neurons in the superior colliculus (SC) have been shown to have large receptive fields that are heterogeneous in nature. These neurons have the capacity to integrate their different sensory inputs, a process that has been shown to depend on the physical characteristics of the stimuli that are combined (i.e., spatial and temporal relationship and relative effectiveness). Recent work has highlighted the interdependence of these factors in driving multisensory integration, adding a layer of complexity to our understanding of multisensory processes. In the present study our goal was to add to this understanding by characterizing how stimulus location impacts the temporal dynamics of multisensory responses in cat SC neurons. The results illustrate that locations within the spatial receptive fields (SRFs) of these neurons can be divided into those showing short-duration responses and long-duration response profiles. Most importantly, discharge duration appears to be a good determinant of multisensory integration, such that short-duration responses are typically associated with a high magnitude of multisensory integration (i.e., superadditive responses) while long-duration responses are typically associated with low integrative capacity. These results further reinforce the complexity of the integrative features of SC neurons and show that the large SRFs of these neurons are characterized by vastly differing temporal dynamics, dynamics that strongly shape the integrative capacity of these neurons.

the superior colliculus (SC) is a mammalian midbrain nucleus well recognized for its role in the generation of coordinated eye and head movements (Marino et al. 2008, 2012a, 2012b; Munoz and Guitton 1985, 1989; Sparks and Mays 1983). In addition, the SC is a watershed site for the convergence of sensory information, with visual, auditory, and somatosensory inputs terminating in its intermediate and deep layers (Edwards et al. 1974; Huerta and Harting 1984; Mucke et al. 1982; Tortelly et al. 1980). As a result of this convergence, many neurons in the SC are multisensory, receiving inputs from two and even three different sensory modalities (Meredith and Stein 1983, 1986a, 1986b; Wallace et al. 1993). These neurons do far more than passively reflect these different inputs, with many actively integrating them in order to give rise to dramatically transformed outputs (Meredith and Stein 1983, 1986a, 1986b, 1996; Meredith et al. 1987; Stein and Meredith 1993; Wallace et al. 1993). The presumptive importance of multisensory integration lies in the close ties between the SC and behavior, where changes in the firing characteristics of SC neurons are likely to be important for the facilitations that can be observed in saccadic, gaze-related, and orientation behaviors (Corneil et al. 2002; Frens et al. 1995; Frens and Van Opstal 1998; Goldring et al. 1996; Hughes et al. 1994).

In addition to neurons that are overtly responsive to multiple sensory cues, there is an additional population of SC neurons that are responsive to cues in only a single sensory modality (i.e., as indexed by spiking responses) but whose responses are strongly modulated under multisensory conditions (Carriere et al. 2008; Royal et al. 2009). The role of these modulated neurons in multisensory processing remains unresolved. Regardless of whether neurons are frankly responsive or modulatory, the nature by which they combine their different sensory inputs has been shown to be strongly dependent upon the physical characteristics of these inputs (Meredith and Stein 1986a, 1986b, 1996; Meredith et al. 1987). Thus stimulus factors such as space, time, and relative effectiveness are key determinants in dictating the final integrative product.

One characteristic feature of SC neurons is their large receptive fields (Kadunce et al. 1997, 2001; Krueger et al. 2009; Meredith and Stein 1990; Stein and Meredith 1993). Although classically treated as simply bounded areas within which sensory responses can be evoked, recent work has revealed a surprising degree of heterogeneity in the responses seen within these receptive fields (Carriere et al. 2008; Krueger et al. 2009; Royal et al. 2009). As a means of examining this heterogeneity, these prior studies used the construct of a “spatial receptive field” (SRF), which represents the profile of neuronal responses for a series of stimulus locations. Marked differences in response were seen as a function of location, with firing rates varying by four- to fivefold with changes in stimulus location. More importantly, these studies showed that these differences in neuronal responsiveness were an important factor in the integrated multisensory response, with SRF locations showing the weakest unisensory (i.e., visual alone, auditory alone) responses having the greatest capacity for multisensory enhancements.

Along with highlighting the importance of spatial location within the SRF in dictating response effectiveness, these prior studies (Ghose et al. 2010) also illustrated differences in temporal response dynamics that are also likely to be important factors in multisensory integration. Thus these prior studies, along with others (Rowland et al. 2007a), found that changes in both response latency and duration were key components in the enhanced multisensory response, with shorter latency and longer duration typically accompanying multisensory conditions. In the course of this work, we also began to see distinctions in response durations as a function of spatial location, such that certain locations within the SRF appeared to show short-duration responses, whereas other locations were characterized by much longer-duration responses. Such differences in response dynamics are likely to have strong implications for the integrated multisensory response, with the hypothesis that shorter-duration responses may be coupled to the largest multisensory gains (because of inverse effectiveness). Alternatively, longer-duration responses may be associated with greater integrative potential due to a lower overall firing rate, thus allowing for greater amplification. The present study set out to test between these competing hypotheses by systematically examining the temporal dynamics of response in a population of multisensory SC neurons and linking the temporal characteristics of response to multisensory integration.

METHODS

General procedures.

Experiments were conducted in adult cats (n = 2) raised under standard housing conditions. All experiments were done in an anesthetized and paralyzed semichronic preparation and consisted of single-unit extracellular recordings from the SC. Experiments were run on a weekly basis on each animal. All surgical and recording procedures were performed in compliance with the Guide for the Care and Use of Laboratory Animals at Vanderbilt University Medical Center, which is accredited by the Association for Assessment and Accreditation of Laboratory Animal Care. All procedures were approved by the Vanderbilt Institutional Animal Care and Use Committee.

Implantation and recording procedures.

For surgical anesthesia, animals were induced with ketamine hydrochloride (20 mg/kg im) and acepromazine maleate (0.04 mg/kg im). For implantation of the recording chamber over the SC, animals were intubated and artificially respired. A stable plane of surgical anesthesia was achieved with inhalation isoflurane (1–3%). Body temperature, expiratory CO₂, blood pressure, and heart rate were continuously monitored (VSM7, Vetspecs/SCIL), recorded, and maintained within ranges consistent with a deep and stable plane of anesthesia. A craniotomy was made to allow access to the SC, and a head holder was attached to the cranium with stainless steel screws and orthopedic cement to hold the animal during recording sessions without obstructing the face and ears. Postoperative care (antibiotics and analgesics) was done in close consultation with veterinary staff.

For recording experiments, animals were anesthetized with ketamine (20 mg/kg im) and acepromazine maleate (0.04 mg/kg im) and maintained throughout the procedure with constant-rate infusion of ketamine (5 mg·kg⁻¹·h⁻¹ iv) delivered through a cannula placed in the saphenous vein. Although the effects of ketamine anesthesia on multisensory processes are the subject of some debate (i.e., see Populin 2005; Populin and Yin 2002; Stanford et al. 2005; Stanford and Stein 2007) we have seen very little difference in receptive fields or the integrative capacity of multisensory neurons in the SC when comparing data from ketamine-anesthetized and awake preparations (Wallace et al. 1998). The head holding system was then used to maintain the animal in a comfortable recumbent position. To prevent ocular drift (which can impact the mapping of receptive fields), animals were paralyzed with pancuronium bromide (0.1 mg·kg⁻¹·h⁻¹ iv) and artificially respired for the duration of recording. Before paralysis was induced, a stable plane of anesthesia was verified in each animal. To achieve this, in an initial session a continuous infusion of ketamine was delivered and adjusted while a number of key physiological parameters indicative of anesthetic state (heart rate, ECG, temperature, blood pressure) were monitored. The basal rate of infusion for future recording sessions was thus determined. In addition, before the paralytic was introduced during recording sessions, these procedures were once again carried out prior to paralysis in order to ensure adequate depth of anesthesia. The rate of infusion was adjusted throughout the experiment depending on the established physiological parameters to ensure a stable plane of anesthesia. Since during recording there are no wounds or pressure points, with careful monitoring and adjustment based on vital signs ketamine is able to provide a sufficient sedation level. Parylene-insulated tungsten electrodes (Z = 2–5 MΩ) were advanced into the SC with an electronically controlled mechanical microdrive. Single-unit neural activity (signal-to-noise ratio ≥ 3:1) was recorded (Sort client software, Plexon), amplified, and routed to an oscilloscope, audio monitor, and computer for performing online and off-line analysis. At the end of the recording session (∼8–10 h), paralysis was reversed and the animal was weaned from the ventilator. Anesthesia was discontinued, and upon return of stable respiration and locomotion the animal was returned to its home cage. Animals were given 60–100 ml of lactated Ringer solution subcutaneously in order to facilitate recovery.

Stimulus presentation and search strategy.

Extracellular single-unit recordings targeted visual-auditory (VA) multisensory neurons in the deep layers of the SC. A multisensory neuron was defined as one in which the response in the multisensory condition (mean number of spikes/trial) was statistically different from the best unisensory response (mean number of spikes/trial) as determined by the Wilcoxon rank test (P < 0.05). Multisensory neurons were further divided into two categories. Frank or overt multisensory neurons were those that showed an overt response to both visual and auditory stimuli. Modulatory multisensory neurons were those in which the response to the driving modality was modulated by a stimulus in the other modality. Once a neuron was isolated, the borders of its receptive field were coarsely mapped. Visual stimuli consisted of the illumination of stationary light-emitting diodes (LEDs; 100-ms duration), while auditory stimuli were delivered through speakers and consisted of 100-ms-duration broadband noise (20 Hz–20 KHz) with an intensity of 67 dB SPL. Both the LEDs and speakers were mounted on a hoop 0.6 m away from the center of the animal's head, with locations spanning azimuthal space from 0° to 90° on either side of the midline. Stimulus location typically varied by 10° (azimuth and elevation) for each tested position. The hoop could be rotated along different elevations. This stimulus configuration allowed for the sampling of numerous locations within and just outside the coarsely delimited receptive fields, creating a SRF for each of the effective modalities as well as for the multisensory condition. The physical characteristics of the stimuli were always identical in all respects except for spatial location. Visual and auditory stimuli were presented in a randomized interleaved manner at multiple azimuthal locations along a single elevation at a time. Multisensory combinations always consisted of visual and auditory stimuli presented at the same spatial location (i.e., spatial coincidence). A minimum of 60 trials (20 visual, 20 auditory, 20 multisensory) were collected for any given stimulus location. Consecutive stimulus presentations were separated by a minimum of 1.5 s to avoid response habituation.

Data acquisition and analysis.

A custom-built PC-based real-time data acquisition system controlled the structure of the trials and the timing of the stimulus (LabVIEW, National Instruments). Analog waveforms were transferred to a Plexon MAP system (Plexon), where they were digitized at 40 KHz. Single units were isolated online with Sort Client software (Plexon) and also stored for further off-line analysis. Neuronal responses were characterized through construction of peristimulus time histograms (PSTHs) for each condition [visual (V) only, auditory only (A), visual-auditory (VA)] for each location tested within the SRF. Response baseline was calculated as the mean firing rate during the 500 ms immediately preceding the stimulus onset for each of the three conditions. Thresholds for the PSTHs were set at 2 SD above the respective baselines to delimit the stimulus-evoked response. After stimulus onset, the time at which the PSTH crossed above the 2 SD line (and remained so for at least 30 ms) was noted as the response onset. Response offset was the time at which the PSTH fell below the 2 SD line and stayed below this line for ≥30 ms. Response duration was defined as the time interval between response onset and response offset. Mean stimulus-evoked response was calculated as the average number of spikes elicited per trial during the defined response duration interval. Mean spontaneous firing rate was always subtracted.

Measures of multisensory integration.

Two measures were used to quantify multisensory integration. The first was the interactive index (ii), which measures how the multisensory response differs from the best unisensory response. The magnitude of this change was calculated as [(CM − SM_max)/SM_max] × 100 = % interaction, where CM is the mean response evoked by the combined-modality stimulus and SM_max is the mean response evoked by the most effective single-modality stimulus (Meredith and Stein 1983, 1986). Statistical comparisons between the mean stimulus-evoked responses of the multisensory condition and the best unisensory condition were done with a nonparametric Wilcoxon rank test. The second measure used was mean statistical contrast (msc). This metric evaluates the multisensory response as a function of the response predicted by the addition of the two unisensory responses. Multisensory contrast is calculated with the formula ∑[(SA − A) − (V − VA)]/n, where SA is spontaneous activity, A is auditory response, V is visual response, VA is multisensory response, and n is the number of trials. The model assumes independence between the visual and auditory inputs and uses additive factors logic to distinguish between subadditive (contrast < 0), additive (contrast = 0), and superadditive (contrast > 0) modes of response (Perrault et al. 2003, 2005; Stanford et al. 2005; Stanford and Stein 2007). Significant differences from a contrast value of 0 were determined by the Wilcoxon rank test.

Temporal epoch analysis.

For a subset of neurons (those with long discharge durations) the total response was divided into three equivalent temporal epochs: early, middle, and late. Both the ii and msc values were calculated for each of these epochs to determine how the integrative abilities of these multisensory neurons changed over time.

RESULTS

Multisensory SC neurons exhibit distinct firing modes.

A total of 54 multisensory (visual-auditory) neurons (n = 21 for animal 1 and n = 33 for animal 2) were isolated from the intermediate and deep layers of the SC (below stratum opticum) and held for the duration of the extensive analyses that comprise this study (1–2 h). Of these, 30 neurons were classified as frank/overt (i.e., overtly responsive to both visual and auditory stimuli) while 24 neurons were modulatory (i.e., only driven by a single modality; see methods for definitions of frank and modulatory neurons). No differences were noted in these distributions between the two animals. Individual SC neurons exhibited a wide range of response duration in response to both unisensory and multisensory stimuli. Slightly less than 20% of the recorded neurons (10/54) exhibited only short-duration (i.e., <250 ms) responses for all locations tested within their SRF under both best unisensory and multisensory conditions (Fig. 1). In contrast, slightly more than 80% (44/54) of the neurons examined exhibited a response exceeding 250 ms in duration for at least one tested location (Fig. 2). Nonetheless, typically in these neurons the majority of the locations (mean = 72%) within the SRF exhibited much shorter-duration discharge patterns. A systematic analysis of the duration of the multisensory response for all neurons and all locations is shown in Fig. 3.

Fig. 1.

Representative example of a single neuron recorded from the intermediate or deep layers of the superior colliculus (SC) that shows short-duration discharges at 4 of the representative locations tested within its receptive field. In fact, all the locations tested within the spatial receptive field of this neuron displayed short discharge duration. The spatial receptive field of the neuron is shown by a shaded rounded rectangle, top. Letters represent the locations for which poststimulus time histograms are shown below and bar graphs quantify the firing rates at each of the 3 stimulus conditions; interactive index (ii) values are also depicted. For the location represented by A within the receptive field there is significant response depression (*P = 0.016) and ii = −55.36%. For the location represented by B ii = 83%, which is also statistically significant (*P = 0.004). For the location represented by C ii = 60.45%, which is statistically significant (*P = 0.02 as determined by Wilcoxon rank test). The location represented by D exhibits significant interaction as expressed by ii = 80.61% (*P = 0.004). V, visual only; A, auditory only; VA, visual-auditory.

Fig. 2.

Representative example of a single neuron recorded from the intermediate or deep layers of the SC that shows a dual mode of discharge. Two of the representative locations show short response durations, while the rest show long response durations. The spatial receptive field of the neuron is shown by a shaded rounded rectangle, top. Letters represent the locations for which poststimulus time histograms are shown below and bar graphs quantify the firing rates at each of the 3 stimulus conditions; ii values are also depicted. For the location represented by A ii = 75.23% and is statistically significant (*P = 0.0001). For the location represented by B ii = 130.81% and is statistically significant (*P = 0.0001). For the location represented by C ii = 11.52% and is statistically nonsignificant. The same is true for the location represented by D, where ii = 6.4% and P = 0.67.

Fig. 3.

Multisensory neurons in the SC exhibit different response durations: short discharge durations and long discharge durations. Short discharge duration is associated with high integrative abilities (mean ii = 92.43%), while long discharge duration is associated with lower integrative abilities (mean ii = 34.35%, R = −0.19, P < 0.00001). Solid black line represents the trend of the data set (y = −0.0828x + 87.474).

Analysis of response latencies revealed no apparent differences based on the duration of response. Thus, with the arbitrary division of 250 ms as a means to divide responses into short and long duration, the mean visual latency was ∼75 ms for both groups (Student's t-test, P = 0.7552). Similarly, the mean auditory latency for both short- and long-duration responses was 23 ms (Student's t-test, P = 0.8965).

Influence of temporal discharge patterns on integrative abilities of multisensory SC neurons.

These different response modes and temporal discharge patterns were found to be associated with significant differences in multisensory integrative capacity. Thus there is an inverse relationship between response duration and ii (Fig. 3). Again, in order to better clarify the relationship between discharge duration and multisensory integration, we used the arbitrary duration criterion of 250 ms to divide the population into short and long response. When divided in this way, the average gain in response relative to the better of the two unisensory responses (i.e., ii) was 92% for short-duration responses versus 34% for long-duration responses, a significant difference (Student's t-test, P = 2.015 × 10⁻¹⁹).

One striking finding in the data was that there were significant differences in response duration between the best unisensory and multisensory conditions, the nature of which depended on the type of integration (Fig. 4). Thus for response enhancements the duration of response in the multisensory condition was significantly greater than for the best unisensory condition (Fig. 4A), while for response depressions the response duration in the multisensory condition was significantly lower than for the best unisensory condition (Fig. 4B). Under conditions in which there were no significant interactions, the response durations did not differ (Fig. 4C).

Fig. 4.

Relationship between best unisensory and multisensory response durations. A: for response enhancements, multisensory response duration was significantly longer (mean = 225.08 ms) than the best unisensory condition (mean = 135.79 ms) as measured by the Wilcoxon signed-rank test (P < 0.00001). Solid black line represents the trend of the data set, while dashed black line represents the slope of 1 (y = x). B: for response depressions, the multisensory duration was significantly lower (mean = 82.55 ms) than the best unisensory duration (mean = 180.94 ms) (P < 0.00001). Solid black line represents the slope of the data, which is <1, while dashed line has a slope of 1 (y = x). C: for no interactions, the dashed line representing a slope of 1 and the trend of the data set represented by the solid black line overlap and the durations do not differ between the best unisensory (mean = 245.49 ms) and multisensory (mean = 251.22 ms) conditions (P = 0.06).

Reinforcing the role of discharge duration in determining integrative magnitude, within individual mixed-response neurons (i.e., the neurons exhibiting both short and long discharge durations) the largest interactions were invariably associated with locations at which short-duration responses were evoked. To exemplify this, a subset of 10 neurons are shown in Fig. 5 that exhibited both short- and long-duration discharges within their SRF. Locations at which short-duration responses were elicited invariably exhibited large gains in response under multisensory conditions, whereas those in which long-duration responses were elicited showed little gain. This pattern was typical for the entire population of neurons sampled. A comparison of interactive magnitude for the short-duration responses of neurons exhibiting only short-duration responses versus neurons showing both short and long responses revealed both to have large gains. Thus both populations exhibited large gains in ii (neurons with short-duration responses only: mean ii = 141% vs. mixed neurons with both short- and long-duration responses: mean ii = 112%). These differences were not significant between the two groups (P = 0.2908 as determined by a Student's t-test).

Fig. 5.

ii is plotted for locations with short discharge duration and locations with long discharge duration of a single neuron (coded by symbols) for a subset of 10 representative neurons. It can be seen from this graph that the same cell with short response duration exhibits higher integrative abilities than with long response duration when the integrative ability of the neuron is very low.

In addition to the analysis of ii (which uses the largest unisensory response as a referent), msc, which calculates multisensory integration as a function of both unisensory responses (see methods for details), was also determined for each multisensory interaction. With this analysis, multisensory neurons showed a pattern of results similar to that seen with ii. Hence, short-duration responses were typically associated with significant superadditive and subadditive interactions, whereas long-duration responses were mostly associated with nonsignificant interactions (Fig. 6).

Fig. 6.

Mean statistical contrast (msc) as a function of multisensory duration. Locations with short response durations are mostly associated with statistically significant (P < 0.05 as tested by Wilcoxon rank test) superadditive and subadditive interactions (shown by black dots), while locations with long discharge durations are mostly associated with msc values that are statistically not significant (P > 0.05 as tested by Wilcoxon rank test) (shown by gray dots). Solid black line represents the mean msc value (1.34) for short discharge durations, while dashed line represents the mean msc value (0.17) for the longer response durations.

Relationship between firing rate, discharge duration, and integrative abilities.

Since the temporal discharge pattern appeared to play an important role in the integrative abilities of the neuron under study, it was important to examine the relationship between absolute firing rate and discharge duration in these multisensory neurons. Analysis of the population means revealed higher firing rates for responses of shorter duration (41.5 spikes/s) compared with those of longer duration (27.5 spikes/s) (Fig. 7).

Fig. 7.

Relationship between multisensory firing rate and multisensory duration of response. Overall short discharge duration (filled circles) is accompanied by high firing rates, while long discharge durations (open circles) are accompanied by low firing rates (R = −0.39, P = 0.009). Horizontal solid line represents the mean firing rate for short discharge durations (41.5 spikes/s), which is significantly higher than the mean firing rate for long responses (horizontal dashed line, 27.5 spikes/s) (t-test, P = 0.0085).

In an effort to better characterize which temporal aspects of the multisensory response were most closely related to integrative capacity, firing rate as a function of interactive magnitude was also evaluated. As opposed to the strong negative correlation between multisensory duration and ii (R = −0.19, P < 0.000001), mean multisensory firing rate (spikes/s) was not significantly correlated with ii (R = 0.17, P = 0.2). Furthermore, multiple regression analysis revealed that the duration of response was a significant contributor to the magnitude of the multisensory interaction (P = 0.000563), while the contribution of firing rate was nonsignificant (P = 0.69). This analysis helps reinforce the conclusion that the changes in the multisensory integrative abilities of SC neurons are associated with the changes in firing mode (i.e., discharge duration) and are poorly associated with absolute changes in firing rates.

Response dynamics under tonic-mode firing conditions.

In an effort to determine the evolution of multisensory integration during long-duration responses, the multisensory response for neurons exhibiting long-duration discharges was divided into three equivalent epochs: early, middle, and late. The rationale for this division was to create temporal epochs comparable to the short-duration responses and to examine whether interactions happening on shorter timescales were not averaged out as a result of the longer-duration responses. In addition, prior work has highlighted the temporal evolution of multisensory responses in SC neurons (Royal et al. 2009) and has shown that significant interactions often accompany the earliest and latest phases of response (no distinction in this prior work was made between short-duration and long-duration responses).

For this analysis, neurons were further divided on the basis of the presence of either overt responses to stimuli in both the visual and auditory modalities (frank or overt neurons) and or an overt response only in one modality but modulated by the other modality (modulatory neurons). In the vast majority (81%) of the frank/overt neurons, significant multisensory enhancements (i.e., gains in ii) were indeed seen during the earliest response epoch. In contrast, significant interactions were rare in the middle (6%) and late (13%) response epochs. msc revealed a similar pattern, with superadditive interactions being most commonly found in the early response epoch (43%). In contrast, superadditivity was rare in the middle and late epochs of the response (13% and 6%, respectively). In striking contrast, modulatory neurons rarely showed significant interactions in any of the response epochs. Figure 8 shows the contrast measures for early, middle, and late phases of integration for a subset of mixed neurons (both frank and modulatory neurons are included).

Fig. 8.

Phases of integration: contrast measures for the early, middle, and late phases of integration for the long discharge duration of a subset of neurons. The early phase is characterized by superadditive interactions, while the middle and late phases are characterized by additive interactions. Black circles represent statistically significant msc values (P < 0.05), while gray circles represent nonsignificant values as measured by Wilcoxon rank test.

DISCUSSION

In the present study, we show for the first time that multisensory SC neurons exhibit marked heterogeneity in the temporal characteristics of their sensory responses and that one aspect of this heterogeneity (response duration) is intimately tied to integrative capacity. Thus response duration was negatively associated with integrative capacity, such that short-duration responses were strongly associated with high integrative capacity and long-duration responses were associated with lower (or absent) integrative capacity. Although the relationship between discharge duration and multisensory integration appears to be a continuous one (see Fig. 3), we chose to divide the population into short- and long-duration responses in order to best illustrate the relationship between duration and integration. Despite the fact that multisensory SC neurons and their integrative abilities have been the subject of study for over two decades, the heterogeneous nature of the large receptive fields of these neurons have been poorly characterized. Prior work that has focused on this question has largely detailed the spatial heterogeneity of these large receptive fields, showing that responses to the same stimuli can differ by severalfold simply on the basis of their location within the receptive field (Kadunce et al. 2001; Krueger et al. 2009). In the present study we focus on the dimension of time and add an additional layer of description to our understanding of how neuronal response characteristics contribute to the final integrative product. The ultimate goal of this work is to provide a complete description of how the spatiotemporal receptive field shapes the nature of the multisensory response, a description that not only will provide insight into the complex computations carried out by these neurons (and thus provide important clues as to their biophysical basis) but will also provide a more realistic description of how multisensory neurons integrate real-world sensory cues.

Response duration as a determinant of multisensory integration.

Prior work has highlighted that multisensory SC neurons depend critically on a number of stimulus-related factors in determining the integrative product when presented with paired multisensory stimuli. The most salient of these are space, time, and effectiveness, such that the combination of multisensory stimuli that are spatially and temporally coincident, and that are weakly effective when presented on their own, results in the largest multisensory interactions (Meredith et al. 1987; Meredith and Stein 1986a, 1986b, 1996).

More recently, work has expanded these determinants to include “neuron-specific” factors (Perrault et al. 2003, 2005). In these prior studies, it was established that neuronal characteristics such as spontaneous firing rate and dynamic range were important in determining the multisensory capacity of a given neuron, with those having lower spontaneous firing rates and smaller dynamic ranges exhibiting the highest multisensory gains. The present study extends this framework by illustrating an important association between a neuron's temporal response dynamics, specifically discharge duration, and its multisensory integrative capacity.

The present results fit well within the recent emphasis that has focused on better detailing the temporal characteristics of multisensory integration. For example, in addition to work from our own lab (Royal et al. 2009), Rowland and colleagues (Rowland et al. 2007a) have shown that multisensory enhancement is greatest in the initial phase of the multisensory response, a property that these authors have described as “initial response enhancement (IRE).” Concordant with this are the results of the present work, in which the temporal epoch analysis demonstrates that it is the earliest part of the response that is characterized by superadditive interactions.

Placing these results in a behavioral context, early superadditive enhancements in response, coupled with a latency shift (i.e., speeding) under multisensory conditions, could readily provide the initial coding framework that ultimately results in the faster and more accurate gaze shifts that are seen under multisensory (i.e., visual-auditory) situations (Corneil et al. 2002; Frens et al. 1995; Goldring et al. 1996; Hughes et al. 1994). In such a model, these early changes in sensory encoding are ultimately transformed into premotor and motor commands that drive the resulting facilitated orientation response. The present study provides a unique view into the multisensory populations that may contribute to such behavioral gains. Thus, in addition to the short-duration responses, which fit quite readily onto this interpretation, the earliest phase of the longer-duration responses also exhibits superadditive interactions and could also play a role in speeded responses.

In addition to reinforcing the importance of these early multisensory interactions, one of the key findings of the present study is that majority of the multisensory SC neurons also carry a longer-duration response component that is largely additive and whose role in multisensory processing remains unresolved. One possibility for such longer-duration responses is that they are carrying more feature-related information about the multisensory stimulus complex. One candidate for this is motion-related information, given the central role that the SC plays in signaling the location of a stimulus of interest and the strong motion selectivity of its constituent neurons (Dreher and Hoffmann 1973; Stein and Meredith 1993). Such a speculation suggests additional experiments to examine these later response components in the context of manipulations in the structure of the multisensory pair.

These types of distinctions are likely not unique to multisensory systems. Indeed, similar results are seen within the visual system in which phasic or burst mode of firing has been linked with stimulus detection while tonic mode has been linked to stimulus analysis (Guido et al. 1995; Sherman 1996). In addition, phasic firing has been linked to less variability, increased signal-to-noise ratio, and better signal detection in the visual thalamus (Guido et al. 1992; Guido and Sherman 1998). Ongoing studies are testing to see whether response duration is also linked to lower variability in multisensory SC neurons.

In addition, in this study we show that, overall, multisensory response durations are longer than their best unisensory counterparts under conditions of response enhancements and shorter under conditions of response depressions. Also, when no integration occurs response durations between the two conditions do not differ. This finding is important because it implies that not only can multisensory response duration act as a determinant of the amount of multisensory integration (i.e., short response with high integration, long response with low integration) but it can also offer insights into the nature of integration that the neuron engages in. Thus, by knowing the response durations in the best unisensory and multisensory conditions of the neuron, it is possible to determine both the nature and integrative capacity of the neuron. This may prove to be very useful for future modeling studies (see below).

Implications of response modes for modeling multisensory processes.

There has been a great deal of recent interest focused toward the modeling of multisensory integration, largely as an effort to provide more insight into the mechanistic underpinnings of the integrative capacity of multisensory neurons (Anastasio et al. 2000; Anastasio and Patton 2003; Avillac et al. 2005; Diederich and Colonius 2004; Rowland et al. 2007b; Xing and Andersen 2000). Much of this work has been built around the original principles of integration (i.e., space, time, and effectiveness), which provide a good first-order characterization of the integrative abilities of these neurons. However, these principles are incomplete in explaining the behavior of these neurons. Indeed, this incompleteness was the motivation for follow-up studies that began to focus on neuron-specific factors, such as spontaneous activity and dynamic range (Perrault et al. 2005). As models of multisensory integration become increasingly sophisticated (see Cuppini et al. 2010 for a recent incarnation), these stimulus- and neuron-specific factors must be incorporated in an effort to provide the most comprehensive view possible into these processes. The present study provides an important insight into one of the mechanisms (response duration changes) by which the neurons in SC may be engaging in multisensory integration. Consequently, these findings may serve as an important tool for future modeling studies that may be directed toward the development of a more complete model with higher predictive capabilities incorporating the various factors that have been empirically shown to affect the integrative abilities of these multisensory neurons in addition to the original principles of integration.

Functional implications for different response modes in SC multisensory neurons.

The role of the SC in stimulus detection, localization, and orientation behavior has been well documented. Multisensory-mediated improvements in these processes have also been well established, as has the role of the intermediate and deep SC in gating these behavioral improvements (Burnett et al. 2004). Despite the strong correlative links between the activity of multisensory SC neurons and these behavioral facilitations, how (multi)sensory signals are transformed into effective motor commands remains rather poorly understood. Only through a more thorough characterization of the complexities of multisensory neurons and their integrative properties will this understanding be improved to provide a better view into the nature of these important sensorimotor transformations. This knowledge can then be used to tailor the design of experiments in awake and behaving preparations in which the relationship between sensory firing patterns, motor responses, and behavioral outcomes can be assessed. Such studies are becoming increasingly common (Iurilli et al. 2012; Wallace et al. 1998; Wang et al. 2008) but are crucially dependent upon the results of studies in anesthetized animals that allow detailed relationships to be drawn between receptive field architecture, temporal response dynamics, and multisensory integration.

How short and long response modes arise in SC neurons remains unknown. One intriguing possibility is that these differing modes are in some way associated with changes in the nature of the oscillatory inputs to these neurons. Thus it has been shown that stimulus timing plays an integral role in the phase and amplitude of ongoing oscillations and can play a dramatic role in the amplifying (or weakening) neuronal response (Lakatos et al. 2007). Ongoing studies in the lab are analyzing local field potentials (LFPs) in these same neurons in an effort to examine the relationship between response mode, oscillations, and integrative capacity in SC neurons, with the hope of providing a better view into how LFPs may represent the nature of the multisensory encoding.

Finally, the differences in response duration and integrative abilities that are seen within the large receptive fields of these neurons are most likely a reflection of the input architecture of the visual and auditory inputs onto these neurons. The purpose of such heterogeneous receptive fields (some even with multiple “hot spots”) remains unknown, but as alluded to above, such heterogeneity and asymmetry may serve as the substrate for processing of dynamic (i.e., moving) stimulus elements, similar to what has been reported in the different sensory systems (Dreher and Hoffmann 1973; Krueger et al. 2009). In a recent study of motion processing, multisensory benefits are seen more in the periphery than in the center (Macneilage et al. 2012), a result that may be related to the heterogeneous and asymmetric receptive field structure that appears to characterize these complex multisensory neurons.

GRANTS

This work was supported by National Institute of Mental Health Grant MH-63861 and Vanderbilt Kennedy Center Development Funds.

DISCLOSURES

No conflicts of interest, financial or otherwise, are declared by the author(s).

AUTHOR CONTRIBUTIONS

Author contributions: D.G. and M.T.W. conception and design of research; D.G. performed experiments; D.G. and Z.P.B. analyzed data; D.G. and M.T.W. interpreted results of experiments; D.G. prepared figures; D.G. drafted manuscript; D.G. and M.T.W. edited and revised manuscript; D.G. and M.T.W. approved final version of manuscript.