The brain is safely sealed inside the cranium, with virtually no direct interaction with other parts of the body and the outside world. Nevertheless, it constantly processes the information conveyed by several sensory modalities in order to create representations of both body and outer world and to generate appropriates motor responses (Ehrsson et al., 2005; Farnè et al., 2005; Green and Angelaki, 2010). For example, vision can convey information about dangerous stimuli to trigger the generation of appropriate motor response (e.g., escape, avoidance, fight, etc.). Rather than processing sensory inputs in isolation, the brain integrates sensory information (Stein and Meredith, 1993; Fetsch et al., 2012) by forming reliable and robust representation of the external world and body. For example, when both visual and auditory input inform about the same danger, an appropriate motor response is more rapid and efficient (Sereno and Huang, 2006; Laing et al.).
Until a few decades ago, it was strongly believed that sensory (or multisensory) integration occurred only in high-level/associative areas or the cortex (Ghazanfar and Schroeder, 2006; Pavani and Galfano). Recently, several “new” multisensory areas have been discovered (Gobbelé et al., 2003; Pietrini et al., 2004), suggesting that a larger portion of the cortex is engaged in multisensory processing. Additional evidence suggests that multisensory integration also occurs in sub-cortical areas (Kuraoka and Nakamura, 2007; Amad et al., 2014). Finally, and perhaps surprisingly, some studies have demonstrated that multisensory processing occurs in primary sensory areas that were traditionally considered to be uni-sensory (Zangaladze et al., 1999; Murray et al., 2005).
Theories such as “neural reuse” (Anderson, 2010) and “metamodal” organization of the brain (Pascual-Leone and Hamilton, 2001) attempt to provide new paradigms for brain functioning taking into account widespread multisensory integration. The evolutionary advantage of multisensory integration might be the resulting availability of more reliable representations of the external world and body (Elliott et al., 2010; Grüneberg et al.) based on multiple sensory inputs and the resilience to brain injuries and sensory loss (Sarno et al., 2003; Pasqualotto and Proulx, 2012; Brown et al.; Finocchietti et al.). Indeed, multisensory integration has been reported in various experimental tasks including spatial representation (Pasqualotto et al., 2005), object recognition (Woods and Newell, 2004; Harris et al.; Höchenberger et al.; Laing et al.), movement perception (Grüneberg et al.; Imaizumi et al.; Uesaki and Ashida), body representation (Pasqualotto and Proulx, 2015; Pavani and Galfano; Tajadura-Jiménez et al.; Yiltiz and Chen), emotional processing (Miu et al.; Piwek et al.), attentional deployment (Spence, 2002; Depowski et al.), language (Gallese, 2008; Myachykov and Tomlin, 2008; Myachykov et al., 2012; Lam et al.; Shaw and Bortfeld), embodied reasoning (Dumitru, 2014), sensory awareness (Cox and Hong), numerical cognition (Dumitru and Joergensen), auditory perception (Brogaard and Gatzia), and time perception (Homma and Ashida).
The articles included in this special issue offer novel insights about recent developments within the field of multisensory integration, and we believe that they will help understanding the multisensory nature of brain functioning.
Author contributions
AP wrote the first draft of the manuscript. MD and AM provided comments, additions, and further improvements. All authors have approved the final version of the manuscript.
Conflict of interest statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Acknowledgments
MD was supported by a Marie Curie FP7-PEOPLE-IAPP fellowship (grant number 610986).
References
- Amad A., Cachia A., Gorwood P., Pins D., Delmaire C., Rolland B., et al. (2014). The multimodal connectivity of the hippocampal complex in auditory and visual hallucinations. Mol. Psychiatry 19, 184–191. 10.1038/mp.2012.181 [DOI] [PubMed] [Google Scholar]
- Anderson M. L. (2010). Neural reuse: a fundamental organizational principle of the brain. Behav. Brain Sci. 33, 245–266. 10.1017/S0140525X10000853 [DOI] [PubMed] [Google Scholar]
- Dumitru M. L. (2014). Moving stimuli guide retrieval and (in)validation of coordination simulations. Cogn. Process. 15, 397–403. 10.1007/s10339-014-0604-6 [DOI] [PubMed] [Google Scholar]
- Elliott M. T., Wing A. M., Welchman A. E. (2010). Multisensory cues improve sensorimotor synchronisation. Eur. J. Neurosci. 31, 1828–1835. 10.1111/j.1460-9568.2010.07205.x [DOI] [PubMed] [Google Scholar]
- Ehrsson H. H., Holmes N. P., Passingham R. E. (2005). Touching a rubber hand: feeling of body ownership is associated with activity in multisensory brain areas. J. Neurosci. 25, 10564–10573. 10.1523/JNEUROSCI.0800-05.2005 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Farnè A., Iriki A., Làdavas E. (2005). Shaping multisensory action–space with tools: evidence from patients with cross-modal extinction. Neuropsychologia 43, 238–248. 10.1016/j.neuropsychologia.2004.11.010 [DOI] [PubMed] [Google Scholar]
- Fetsch C. R., Pouget A., DeAngelis G. C., Angelaki D. E. (2012). Neural correlates of reliability-based cue weighting during multisensory integration. Nat. Neurosci. 15, 146–154. 10.1038/nn.2983 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gallese V. (2008). Mirror neurons and the social nature of language: the neural exploitation hypothesis. Soc. Neurosci. 3, 317–333. 10.1080/17470910701563608 [DOI] [PubMed] [Google Scholar]
- Ghazanfar A. A., Schroeder C. E. (2006). Is neocortex essentially multisensory? Trends Cogn. Sci. 10, 278–285. 10.1016/j.tics.2006.04.008 [DOI] [PubMed] [Google Scholar]
- Gobbelé R., Schürmann M., Forss N., Juottonen K., Buchner H., Hari R. (2003). Activation of the human posterior parietal and temporoparietal cortices during audiotactile interaction. Neuroimage 20, 503–511. 10.1016/S1053-8119(03)00312-4 [DOI] [PubMed] [Google Scholar]
- Green A. M., Angelaki D. E. (2010). Multisensory integration: resolving sensory ambiguities to build novel representations. Curr. Opin. Neurobiol. 20, 353–360. 10.1016/j.conb.2010.04.009 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kuraoka K., Nakamura K. (2007). Responses of single neurons in monkey amygdala to facial and vocal emotions. J. Neurophysiol. 97, 1379–1387. 10.1152/jn.00464.2006 [DOI] [PubMed] [Google Scholar]
- Murray M. M., Molholm S., Michel C. M., Heslenfeld D. J., Ritter W., Javitt D. C., et al. (2005). Grabbing your ear: Rapid auditory-somatosensory multisensory interactions in low-level sensory cortices are not constrained by stimulus alignment. Cereb. Cortex 15, 963–974. 10.1093/cercor/bhh197 [DOI] [PubMed] [Google Scholar]
- Myachykov A., Tomlin R. (2008). Perceptual priming and syntactic choice in Russian sentence production. J. Cogn. Sci. 9, 31–48. 10.17791/jcs.2008.9.1.31 [DOI] [Google Scholar]
- Myachykov A., Garrod S., Scheepers C. (2012). Determinants of structural choice in visually-situated sentence production. Acta Psychol. 141, 304–315. 10.1016/j.actpsy.2012.09.006 [DOI] [PubMed] [Google Scholar]
- Pascual-Leone A., Hamilton R. (2001). The metamodal organization of the brain. Prog. Brain Res. 134, 427–445. 10.1016/S0079-6123(01)34028-1 [DOI] [PubMed] [Google Scholar]
- Pasqualotto A., Finucane C. M., Newell F. N. (2005). Visual and haptic representations of scenes are updated with observer movement. Exp. Brain Res. 166, 481–488. 10.1007/s00221-005-2388-5 [DOI] [PubMed] [Google Scholar]
- Pasqualotto A., Proulx M. J. (2012). The role of visual experience for the neural basis of spatial cognition. Neurosci. Biobehav. Rev. 36, 1179–1187. 10.1016/j.neubiorev.2012.01.008 [DOI] [PubMed] [Google Scholar]
- Pasqualotto A., Proulx M. J. (2015). Two-dimensional rubber-hand illusion: the Dorian Gray hand illusion. Multisens. Res. 28, 101–110. 10.1163/22134808-00002473 [DOI] [PubMed] [Google Scholar]
- Pietrini P., Furey M. L., Ricciardi E., Gobbini M. I., Wu W. H. C., Cohen L. G., et al. (2004). Beyond sensory images: object-based representation in the human ventral pathway. Proc. Natl. Acad. Sci. U.S.A. 101, 5658–5663. 10.1073/pnas.0400707101 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sarno S., Erasmus L. P., Lipp B., Schlaegel W. (2003). Multisensory integration after traumatic brain injury: a reaction time study between pairings of vision, touch and audition. Brain Inj. 17, 413–426. 10.1080/0269905031000070161 [DOI] [PubMed] [Google Scholar]
- Sereno M. I., Huang R. S. (2006). A human parietal face area contains aligned head-centered visual and tactile maps. Nat. Neurosci. 9, 1337–1343. 10.1038/nn1777 [DOI] [PubMed] [Google Scholar]
- Spence C. (2002). Multisensory attention and tactile information-processing. Behav. Brain Res. 135, 57–64. 10.1016/S0166-4328(02)00155-9 [DOI] [PubMed] [Google Scholar]
- Stein B. E., Meredith M. A. (1993). The Merging of the Senses. Boston, MA: The MITPress. [Google Scholar]
- Woods A. T., Newell F. N. (2004). Visual, haptic and cross-modal recognition of objects and scenes. J. Physiol. 98, 147–159. 10.1016/j.jphysparis.2004.03.006 [DOI] [PubMed] [Google Scholar]
- Zangaladze A., Epstein C. M., Grafton S. T., Sathian K. (1999). Involvement of visual cortex in tactile discrimination of orientation. Nature 401, 587–590. 10.1038/44139 [DOI] [PubMed] [Google Scholar]