Seeing with Your Fingers: A Transcranial Magnetic Stimulation Investigation of Multimodal Sensory Perception

The organization of brain function is often pictured as a simple pyramid, with each sensory modality represented separately at the first level but coming together in “higher” cortical areas to become a unified percept. However, a growing body of work reveals direct and early interactions between sensory areas that were once considered entirely unimodal. For example, recent anatomical studies using retrograde tracers show crossmodal connections in early sensory areas: Falchier et al. (2002) found that the macaque primary visual cortex (V1) receives direct afferents from early auditory cortex and from superior temporal polysensory (STP) cortex, which responds to both audition and touch.

Meanwhile, functional imaging studies have approached this question from another angle. A functional magnetic resonance imaging (fMRI) study by Malcaluso et al. (2000) showed that tactile stimulation increases activity in unimodal visual areas but only when the touch occurs on the same side of the body as the visual stimulus. Likewise, an fMRI study by Merabet et al. (2007) revealed activation in V1 when normally sighted, blindfolded subjects perused a pattern of raised bumps with their fingertips. These two studies provide evidence that the tactile and visual systems interact on a very early level. However, because both are functional imaging studies, they are unable to shed any light on the precise timing of interactions between the two systems.

In their recent Journal of Neuroscience article, Ramos-Estebanez et al. (2007) bridge this gap by taking a novel approach to early visuotactile interactions. The authors asked whether subthreshold sensory stimulation can sum across modalities to produce a reportable percept and, if so, what is the precise timing of the interaction. The study combined transcranial magnetic stimulation (TMS) to V1 with peripheral electrical stimulation (PES) to the left and right index fingers. For many subjects, TMS at sufficient magnitude directly over V1 evokes phosphenes, spots or “sparks” of light in the visual field that do not correlate with any external stimulus. Because some people do not experience phosphenes, only subjects who consistently experienced phosphenes were used in the study. Each subject's phosphene threshold was defined as the TMS level that produced reportable phosphenes in three of five trials. Likewise, each subject's PES threshold was defined as the PES level that produced reportable finger sensations in three of five trials.

TMS and PES levels were set at 80% of the threshold stimulation intensity. TMS over the left occipital lobe targeted V1, corresponding to the right visual field. Each trial consisted of a TMS pulse, presented either alone or immediately after PES to the right or left hand. Because subjects could not feel the subthreshold PES, they had no cues to distinguish PES/TMS trials from TMS alone. Trials were randomly interleaved and were performed with subjects' hands uncrossed and with hands crossed through the midline of the body [Ramos-Estebanez et al. (2007), their Fig. 1 (http://www.jneurosci.org/cgi/content/full/27/15/4178/F1)]. In trials that included PES, tactile stimulation preceded TMS by 40–100 ms interstimulus intervals (ISIs) to allow the experimenters to study the timing of visuotactile interactions. For each trial, subjects reported whether they saw phosphenes during TMS.

Subthreshold TMS to left V1 produced phosphene perceptions in ∼10% of trials. When subthreshold PES to the left hand was added to TMS, there was no significant change in phosphene perception. However, when PES to the right hand was combined with TMS, a dramatic effect emerged: subjects suddenly reported phosphene perceptions up to 50% of trials [Ramos-Estebanez et al. (2007), their Fig. 2 (http://www.jneurosci.org/cgi/content/full/27/15/4178/F2)]. The increase in phosphene perception held true for crossed and uncrossed hand positions and for all ISIs (p < 0.001). This result suggests that the two imperceptible stimuli combine across modalities to produce a salient percept. At no point did the subjects experience reportable sensations in either hand.

The striking effect of stimulation to the right hand persisted whether the hands were crossed or uncrossed. This is what one might expect from “hardwired” connections between the right side of the body and unimodal areas representing the right visual hemifield (Macaluso et al., 2000).

The length of the ISI also significantly affected the extent to which tactile and visual stimuli combined. The most significant crossmodal effect occurred at an ISI of 60 ms [Ramos-Estebanez et al. (2007), their Fig. 2 (http://www.jneurosci.org/cgi/content/full/27/15/4178/F2)]. Given that tactile information takes ∼20 ms to reach the CNS from the periphery (Ramos-Estebanez et al., 2007), that leaves only ∼40 ms for peripheral tactile information to combine with visual stimulation. Because 40 ms is enough time for a limited number of synaptic exchanges in the brain, this suggests that the two modalities are not combining in higher-association cortices as conventionally thought; instead, the interaction takes place at “lower” cortical areas. Interestingly, Foxe and Schroeder (2005) point out that the latency for an auditory stimulus to reach early visual areas via STP is ∼60–70 ms; this is very similar to the 60 ms time course of the tactile interaction with visual cortex reported by the present study. Because STP is known to respond to tactile, auditory, and visual stimuli (Karnath, 2001), perhaps tactile stimulation and auditory stimulation interact with early visual areas via similar paths.

In summary, the 2007 study by Ramos-Estebanez et al. offers insight into cross-sensory interactions by combining TMS and PES. It supplements previous work by showing that even subthreshold sensory stimuli can combine across modalities and that the time course of this interaction occurs within an early, specific temporal range. Many questions remain, because the physiological and anatomical underpinnings of early crossmodal interactions are still being uncovered. However, as our understanding of crossmodal interactions evolves, studies such that by Ramos-Estebanez et al. (2007) may gradually reshape our current concept of brain organization.