Thalamic influences on multisensory integration

Abstract

In everyday life our brain often receives information about events and objects in the real world via several sensory modalities, because natural objects often stimulate more than one sense. These different types of information are processed in our brain along different sensory-specific pathways, but are finally integrated into a unified percept. During the last years, studies provided compelling evidence that the neural basis of multisensory integration is not restricted to higher association areas of the cortex, but can already occur at low-level stages of sensory cortical processing and even in subcortical structures. In this article we will review the potential role of several thalamic structures in multisensory interplay and discuss their extensive anatomical connections with sensory-specific and multisensory cortical structures. We conclude that sensory-specific thalamic structures may act as a crucial processing node of multisensory interplay in addition to their traditional role as sensory relaying structure.

Several recent human functional imaging studies on multisensory integration provided converging evidence that sensory-specific or even primary cortical areas may be involved in the integration of multisensory stimulus attributes in addition to known integration hubs within higher association cortices, e.g., the superior temporal sulcus (STS),^1,2 parts of the parietal cortex,³ or frontal regions.^4,5 A large number of experiments using functional magnetic resonance imaging (fMRI) reported modulations of neuronal population response in sensory-specific areas due to multisensory stimulation,^1,6–8 in accord with invasive animal studies.^9–12 Moreover, studies using electroencephalography (EEG) in humans reported that earliest sensory-specific event-related potentials (ERP) were modulated by concurrent stimuli in a second modality,^13–15 though others pointed at an alternative explanation for these early effects.¹⁶ However, the exact neural mechanisms of multisensory interplay causing modulations in sensory-specific cortices remain unclear. Recent reviews in reference ^{8, 17} and ¹⁸ suggested several possible neural mechanisms underlying multisensory modulations in sensory-specific cortices:

1. Feedback influences from multisensory convergence zones within higher association cortices to sensory-specific areas.

2. Feedback influences from multisensory convergence zones at the border of two sensory-specific cortices that have been identified just recently (i.e., secondary and tertiary “sensory-specific” cortices).

3. Direct cortico-cortical interconnections between low-level modality-specific areas (including primary sensory-specific areas).

4. Different sensory-specific information could be integrated at subcortical levels including sensory-specific thalamic nuclei and then fed forward to the sensory-specific cortices.

While the first three options have been discussed elsewhere (reviewed in ref. 8, 17 and 18), this paper will focus on the role of complex thalamo-cortical pathways in the integration of multisensory stimuli. In the first part of this review, we will discuss thalamic structures that show multisensory anatomical and functional characteristics in different species and in the second part we will highlight their possible functional role in multisensory interplay, i.e., how the neuronal processing of different sensory information within these nuclei might contribute to integration at the neuronal level, plus to the optimal behavioral responses in multisensory situations.

Anatomical Evidence

Several animal studies performed in various species identified thalamic nuclei which—based on their anatomical connections to structures of different sensory modalities and/or based on multisensory response characteristics of their neurons—might integrate multisensory information, sometimes even before the information has reached neocortical areas. One candidate is the medial geniculate body (MGB) which is a major structure of the sensory-specific auditory pathway and which consists of at least three subdivisions (e.g., cat¹⁹). Whereas the ventral division (MGBv) of the MGB is strongly involved in the processing of auditory information, the dorsal (MGBd) and the medial (MGBm) divisions might be rather regarded as multisensory (reviewed in ref. 17, 20 and 21). For instance, cells within the MGBm (rat^22,23) and MGBd (rat;²⁴ cat²⁵) have been reported to respond to auditory but also to visual, vestibular and somato-sensory stimuli. Furthermore, several neuroanatomical studies report direct anatomical connections between the MGBm/ MGBd and auditory but also other sensory subcortical^26,27 and cortical areas.^26,28,29 These connections are well suited to mediate multisensory interplay at the level of the primary auditory cortex as observed, for instance, by means of invasive electrophysiological recordings in several animal species.^{9,10,12,30–32}

Another important thalamic structure which is also linked with the processing of multimodal information is the pulvinar-posterior complex (pulvinar nucleus in primates). The pulvinar is usually associated with visual processes (e.g., macaque;^33,34 rhesus monkey³⁵), but several electrophysiological studies reported that neurons within the pulvinar can also be activated by other or more than one sensory modality.^36,37 Moreover, the extensive connections between the pulvinar and different sensory-specific cortical areas might be instrumental in multisensory modulations of cortical activity within these areas.^{20,28,38–41} Most recently, Cappe et al.²⁹ injected different retrograde neuronal tracers into the auditory, somatosensory and premotor cortex of the macaque monkey; they found that the pulvinar nucleus exhibited the most extensive overlap of differentially retrogradely labeled neurons and concluded that this nucleus may thus play a key role in multisensory and sensorimotor integration.

Other thalamic nuclei, some of them surrounding the MGB and pulvinar, which show also diverse multisensory responses and which have multiple connections with subcortical and cortical areas of various sensory modalities are, for instance, the suprageniculate (SG), posterior intralaminar (PIN), laterodorsal (LD), lateral and ventral posterior (LP, VP) and posterior thalamic nucleus (Po).^{22,26,28,29,42–44} They may have similar, but yet uncovered functions within the multisensory thalamo-cortical network similar to the medial geniculate and pulvinar nuclei (see also ref. 45 and 46 for additional discussion of specific thalamic cell types that may also subserve multisensory integration).

Functional Role of Thalamic Structures in Multisensory Integration

The functional role of thalamic structures and in particular their relevance for multisensory integration is still debated. Nonetheless, some studies provide evidence of thalamic influence on multisensory information processes in rats⁴⁷ and humans⁴⁸ and others link modulations of neuronal activity in subcortical structures with behavioral consequences like audiovisual speech processing,⁴⁹ audiovisual stimulus onset asynchrony detection⁵⁰ and multisensory attention tasks.⁵¹ Kreifelts et al. reported in humans an enhanced classification accuracy of audiovisual emotional stimuli (relative to unimodal presentation) and linked this increase in perceptual performance to enhanced fMRI-signals in multisensory convergence zones of the cortex (STS) and thalamus.

In another recent human fMRI study we tested how co-occurring sounds modulate the subjects' perceptual sensitivity and the neural responses to visual stimuli of higher or lower intensity.⁵³

We found that a task-irrelevant auditory stimulus increases the sensitivity to low-intensity but not to high-intensity visual targets and that this perceptual enhancement relates to fMRI-signal increases in sensory-specific and multisensory cortical brain regions (reviewed in ref. 1, 9 and 11). Most importantly, modulations of the fMRI-signals were also observed in the sensory-specific visual (lateral geniculate body, LGB) and auditory (MGB) thalamus (Fig. 1). Furthermore, LGB and MGB showed a stronger interregional coupling (psychophysiological interaction⁵⁴) with the STS, and the strength of these functional connections scaled with the subjects' behavioral performance (Fig. 2). These results provide evidence in humans that sensory-specific thalamic structures are involved in multisensory integration processes and resulting behavior performances, here psychophysical detection sensitivity.

Figure 1.

Illustration of fMRI BOLD (blood oxygen level dependent) responses in subject-specific visual and auditory thalamus (adapted from Noesselt et al.⁵³). Top: Brain section depicts visual (LGB, blue) and auditory (MGB, red) thalamus for one illustrative individual subject (for more details see Noesselt et al.⁵³). The bar graphs below of the brain sections depict the height of the fMRI-signal for the experimental conditions deduced from subject-specific ROIs (region of interest; see bar graphs, with grey bars for sound conditions and black for no-sound for the three visual stimuli: high intensity, low intensity, no visual target). An enhanced fMRI-signal was found when a sound was added to a lower-intensity visual target, but no significant change in response when the same sound was added to a higher-intensity visual target in accord with the behavioral findings.

Figure 2.

Brain-behavior relations for coupling of thalamic structures with higher association cortices as a function of behavioral performance. Right: The scatter plot depicts the relation between the size of the behavioral interaction pattern (i.e., the difference in subject's visual detection hit rate for sound minus no-sound conditions being more pronounced for lower-than higher-intensity visual targets; along the y-axis) and the significant changes in LGB- and MGB-seeded (conjunction) interregional coupling-strength (PPI, along the x-axis) with the remote region STS (shown on the left side). This analysis highlights stronger coupling of both LGB and MGB with multisensory STS for subjects with higher behavior benefit than for those with lower behavioral benefit (adapted from Noesselt et al.⁵³).

Due to the known limitations in temporal resolution of the used method (fMRI), conclusions about the functional direction or type of connections (feed-forward or feed-back) could not be made. However, one possible mechanism underlying the detected multisensory integration effects was suggested by Schroeder/Lakatos and colleagues on the basis of experiments on time-frequency relationships of multisensory inputs into the primate cortex.^12,55–57 Lakatos et al.¹² reported enhanced neuronal responses in the auditory cortex of macaques during simultaneous auditory and tactile stimulation which was accompanied by a phase reset of neural oscillations in the auditory cortex. The authors suggested that this phase reset may be mediated by fast feed-forward projections from thalamic structures. Such phase resettings might also help to increase the signal-to-noise ratio between concurrent sounds and low-intensity visual targets (cf. Lakatos et al.⁵⁵ for attention-related phase resetting of visual cortex) and may reflect one role of the thalamus in coupling of “functionally distant” cortical regions.

Taken together, there is now converging evidence that not only sensory non-specific (i.e., nonlemniscal) nuclei but also sensory-specific thalamic nuclei may integrate different sensory stimuli and may influence cortical multisensory processing by means of thalamo-cortical feed-forward connections. Modulations of connection strength of these sensory-specific thalamic nuclei with cortical regions are directly linked to behavioral performance and strongly suggest that a neurobiologically plausible theory of multisensory integration needs to take subcortical and especially thalamic influences into account.