Influence of Visual Motion on Tactile Motion Perception

Abstract

Subjects were presented with pairs of tactile drifting sinusoids and made speed discrimination judgments. On some trials, a visual drifting sinusoid, which subjects were instructed to ignore, was presented simultaneously with one of the two tactile stimuli. When the visual and tactile gratings drifted in the same direction (i.e., from left to right), the visual distractors were found to increase the perceived speed of the tactile gratings. The effect of the visual distractors was proportional to their temporal frequency but not to their perceived speed. When the visual and tactile gratings drifted in opposite directions, the distracting effect of the visual distractors was either substantially reduced or, in some cases, reversed (i.e., the distractors slowed the perceived speed of the tactile gratings). This result suggests that the observed visual-tactile interaction is dependent on motion and not simply on the oscillations inherent in drifting sinusoids. Finally, we find that disrupting the temporal synchrony between the visual and tactile stimuli eliminates the distracting effect of the visual stimulus. We interpret this latter finding as evidence that the observed visual-tactile interaction operates at the sensory level and does not simply reflect a response bias.

INTRODUCTION

The various sensory modalities provide partially redundant information about objects and events. Information about the speed of a moving object, for instance, can be conveyed visually, aurally, or tactually. Intermodal interactions have been the focus of an increasing number of studies (Calvert et al. 2004; Spence et al. 2004). A common approach to the study of multimodal perception is to examine the perception of stimuli presented via one modality in the presence and absence of stimuli presented in another modality.

The perception of tactile stimuli has been shown to be affected by input from other modalities. For example, propri-oceptive inputs can affect judgments of temporal order (Craig 2003; Roder et al. 2002; Shore et al. 2002; Yamamoto and Kitazawa 2001). With regard to the effect that visual stimuli have on judgment of tactile stimuli, it has been shown, for example, that the sight of the hands in mirrors can affect tactile localization (Maravita et al. 2002), judgments of tactile temporal order (Gallace and Spence 2005), and reports of touch on amputees (Ramachandran et al. 1995). The localization of a tactile stimulus has also been shown to be affected by the simultaneous presentation of a visual stimulus (Kennett et al. 2002).

In addition to these studies showing intermodal interactions with static stimuli and in recognition of the dynamic nature of much of our real-world perceptions, several recent studies have examined the effects of apparent motion or moving stimuli on the perception of tactile stimuli. Gray and Tan (2002) measured reaction times for localizing tactile stimuli. They found that moving visual stimuli can lower reaction times for tactile localization despite the fact that the visual stimuli do not predict the location of the tactile stimuli. They concluded that moving visual stimuli can direct tactile attention spatially. In another study involving dynamic stimuli, subjects judged the direction of apparent motion, generated on a subject’s finger-pad by stimulating two locations sequentially (Craig 2006). On some blocks of trials, apparent visual motion was generated by activating two lights sequentially. On trials when the visual motion was in a direction opposite to that of the tactile motion, subjects often erred and judged the motion to be in the direction of the visual stimulus rather than that of the tactile stimulus. Under similar conditions, when visual apparent motion affected judgments of auditory apparent motion, this effect has been described by some investigators as an example of “visual capture” (Strybel and Vatakis 2004). Auditory stimuli have also been shown to affect the direction of tactile apparent motion (Soto-Faraco et al. 2004a,b).

One of the limitations in studying dynamic visual-tactile interactions with apparent motion is that, as compared with real motion, it affords only a limited range of motion. If the time between the two stimuli generating optimum apparent motion is increased, the motion may seem to slow down; but too great a temporal separation and subjects perceive the two stimuli as successive and the perception of motion is lost. As the time between two stimuli decreases, the movement may quicken but soon yields a perception of simultaneity. With the limited range of speeds achievable with apparent motion, it is difficult to explore the effects of changes in the speed of tactile motion and the effects that visual stimuli might have on the perception of tactile speed. The properties of apparent motion stimuli are also limited in that they typically consist of single points of stimulation with little to no spatial elaboration.

In the present study, we explored the effect that visual motion has on the tactile perception of speed. One of the strengths of our experimental approach was the way in which the moving tactile patterns were generated. Tactile patterns were presented to the fingerpad using a 400-probe tactile array, which consists of 400 probes arrayed over a 1 × 1-cm area. The center-to-center spacing between probes on the fingerpad is ~500 μm, so the density of the array is greater than the innervation density of the skin (Johnson 2001). By activating successive columns of the array, realistic, continuous motion can be simulated. The speed of this simulated motion can be varied over a considerable range. In the present study, subjects were presented with two tactile drifting sinusoids, one of which (the comparison stimulus) varied in speed. Subjects judged which of the two tactile patterns was moving faster. On some blocks of trials, a visual pattern (also a drifting sinusoid) was presented at the same time as the second of the two tactile patterns. Both the speed and direction of the visual pattern were varied. We wished to determine whether the presence of an irrelevant, moving visual pattern, a visual distractor, would affect subjects’ judgments of tactile speed. Then to the extent that the visual pattern did affect tactile judgments, we wanted to assess the dependence of the observed effect on the parameters of the visual stimulus. Specifically, does changing the wavelength, speed or drift direction of the visual patterns alter their effect on tactile speed discrimination?

The tactile stimuli presented in the present study consisted of drifting sinusoids that varied in their drift speed. With the onset of the tactile patterns, all of the probes were activated simultaneously; that is, the sinusoidal pattern was generated across the entire array and drifted from left to right. This way of generating motion eliminated possible confounds stemming from having the pattern begin at one side of the array and then move across to the other side. In this latter case, subjects could judge the speed of motion by using the difference in time of activation of the columns of probe from one side of the array to the other. One of the reasons for using drifting sinusoids in the present study was to elicit motion percepts with minimal spatial cues.

In the course of examining the effect of visual motion on tactile speed discrimination, we measured the discriminability of (simulated) tactile motion in the absence of visual distrators. Several previous studies of tactile speed discrimination have tested the ability of human subjects to discriminate brushing motions at various velocities. Collins and Roppolo (1980) determined that subjects could discriminate a speed of 132 mm/s from a speed of 182 mm/s on 80% of the trials. In a later study, Essick, Franzen, and Whitsel (1988) examined discriminability for speeds ranging from 15 to 1,400 mm/s. They too used brushing stimuli and obtained Weber fractions ranging from 0.2 to 0.25. These data allowed us to compare the discriminability of drifting sinusoids generated on the 400-probe array to that of brushing stimuli.

The effects of visual motion on tactile speed discrimination were examined in five experiments. In the first experiment, measures of speed discrimination were obtained in the presence and absence of a series of visual distractors varying in their spatial and temporal properties. In the second experiment, subjects judged the relative drift speed of visual as compared with tactile stimuli to probe the relationship between the motion percepts evoked in the two modalities. The objective of the third experiment was to explore the dependence of the observed effect on the parameters of the visual stimuli, specifically on their temporal frequency. In the fourth experiment, the visual distractor drifted in a direction opposite to that of the tactile motion to determine whether the observed visual-tactile interaction was dependent on the relative direction of motion of the visual and tactile stimuli. In the fifth experiment, the temporal synchrony between the visual and tactile stimuli was disrupted to ascertain whether the observed effect was due to perceptual factors or to decisional factors, such as response bias. We discuss plausible neural mechanisms underlying the observed visual-tactile interactions.

METHODS

Subjects

The 60 subjects (24 females and 36 males) were students or employees at Johns Hopkins University. Not all subjects participated in all the experiments: 23 subjects participated in experiment 1 (12 females, 8 males); 22 (7 females, 15 males) subjects participated in experiments 2, 3, and 5; 18 subjects participated in experiment 4 (7 females, 11 males); 6 subjects (1 female, 5 males) participated in the visual-speed-estimation experiment. All subjects were paid for their participation. Consent was obtained from each subject. The Human Subjects Institutional Review Board of Johns Hopkins University approved the experiments.

Apparatus

The tactile stimuli were generated and delivered by means of a dense tactile array, consisting of 400 independently controlled probes arrayed in a 20 × 20 matrix (Pawluk et al. 1998). The tips of the probes, spaced at 0.5 mm, center to center, cover a 1 × 1-cm area. To minimize edge effects, the probe array was surrounded by a metal plate that was flush with the probes. The subject’s finger was pressed against the probes with a force of 100 g using a counterweight mounted on a vertical stage (L.O.T Oriel GmBH and Co., Darmstadt, Germany). This assembly allowed for accurate, repeatable finger positioning on the probe array. The density of the probes is greater than the innervation density on the fingertip, which leads to a smooth motion percept despite the inherent pixilation of the array.

The visual stimuli were generated using OpenGL (version 1.2) and displayed using an MX400 GeForce4 graphics card (NVIDIA, Santa Clara, CA) on a flat screen LCD monitor (MAG Innovision, Irvine, CA). The color depth was set to 24 bits, refresh rate to 60 Hz, and the resolution to 1,024 × 768 pixels. The video driver was configured to synchronize with the vertical refresh of the LCD monitor to maintain smooth animation. The screen was calibrated to obtain a conversion factor with units of pixels/mm. Each stimulus cycle was first converted to pixels and evaluated at 100 evenly spaced points. The stimulus scaled to values between 0 and 1, which were then used as the RGB values (r = g = b for grayscale, 0 = black, 1 = white) for a vertical strip the width of which was 1/100 of a stimulus cycle. A high-resolution timer was used to move the image a given number of pixels for each screen refresh (depending on the desired drift velocity). The screen was interposed between the subject’s head and the probe array. The subject rested his/her head on a chin-rest located 20 cm from the screen of the monitor, which was, in turn, 30 cm from the probe array. The screen, which spanned a visual angle of ~76 × 60°, was placed as close to the finger and as far from the head as the apparatus geometry would allow (Fig. 1). The synchronization between the computers controlling the tactile and visual stimuli was achieved using a private ethernet network (TCP/IP sockets). This synchronization was accurate on the order of 3 ms.

FIG. 1.

Experimental set-up. The distal finger pad of the subject’s left index finger was pressed against the 400-probe array with a force of 100 g using a counter-weight. The computer monitor was interposed between the subject’s finger and his head. The distance between the monitor screen and the subject’s face was 20 cm, and the distance between the probe array and the screen was 30 cm.

In a set of control measurements, we tracked subjects’ eye movements during the task using an eye-tracker (Arrington Research, Scottsdale, AZ) running on a separate computer. The eye tracker consists of a small, black-and-white camera with an infrared light-mitting diode, an analog video-capture card, and commercial software. The software determines eye position by fitting an ellipse to the pupil from the digitized image of the eye. The x and y locations (normalized within the reference frame of the camera) are relayed at 60 Hz to custom software, which communicates with the visual stimulus display computer and synchronizes eye-position data with the stimuli.

Stimuli

The stimuli consisted of visual and tactile spatiotemporal (drifting) sinusoids. The spatiotemporal profile of the sinusoids was given by

$$z(x,t)=A\hspace{0.17em}sin\left(2\pi \left(ft+\frac{x}{\lambda }\right)\right)$$ (1)

where z(x,t) is the depth of indentation (for tactile stimuli) or luminance (for visual stimuli) at position x and time t, f is the temporal frequency of the grating and λ, its wavelength. The drift speed of the grating, V, is then given by

$$V=f\lambda$$ (2)

The wavelength of the tactile stimuli was 8 mm, and their amplitude, A, was 150 μm (0 to peak). The drift speeds are described in the following text. The visual stimuli varied in wavelength and drift speed from experiment to experiment. The luminance varied over the range of achievable grayscale values.

Procedure

MEASUREMENT OF BASELINE PERFORMANCE IN TACTILE SPEED DISCRIMINATION

On each trial, a pair of sinusoids drifting from left to right, each lasting 1 s and separated by a 500-ms interval, was presented to the subject’s left index finger (with no visual stimulus). The first (standard) stimulus always drifted at a speed of 40 mm/s. The second (comparison) stimulus drifted at one of seven velocities: 16, 24, 32, 40, 48, 56, and 64 mm/s (corresponding to frequencies of 2, 3, 4, 5, 6, 7, and 8 Hz, respectively). Comparison stimuli were presented in pseudo-random order. The subject’s task was to indicate, by button click, which of the two stimuli presented on that trial drifted at a greater speed. Subjects made 40 judgments for each comparison stimulus. This baseline measure of performance (from the tactile speed discrimination task with no visual stimulus) was obtained from all the subjects who participated in the study unless otherwise specified.

EXPERIMENT 1

The tactile stimuli and the task were identical to those of the baseline experiment. The only difference was that a visual stimulus, which drifted in the same direction as the tactile stimulus (from left to right), was presented at the same time as the (tactile) comparison stimulus. The onset and offset of the visual stimulus were synchronized with those of the tactile comparison stimulus. In this and subsequent experiments, subjects were instructed to ignore the visual stimuli. The visual distractors varied in their wavelength and drift speed. Three wavelengths were presented at 40 mm/s, namely 5, 10, and 20 mm (corresponding to 1.4, 2.9, and 5.7° per cycle, respectively); another set of distractors, presented in separate blocks, varied in their drift speed (5, 10, 40, 60, and 80 mm/s) but had a constant wavelength of 10 mm. The stimuli presented in this and subsequent tactile speed discrimination (TSD) experiments are summarized in Table 1. In all the TSD experiments, 20 observations were obtained for every combination of tactile comparison stimulus and visual distractor; stimuli in each block were presented in pseudo-random order; each combination of tactile and visual stimulus was presented four times in each of five blocks.

TABLE 1.

Summary of the stimuli presented in each of the four TSD experiments

	λt, mm	Vt, mm/s	λv, mm	Vv, mm/s	fv, Hz	Sync?	Effect
Experiment 1	8	16, 24,	5	40	8	yes	+
		32, 40,	20	40	2		0
		48, 56,	10	5	0.5		0
		64	10	10	1		0
			10	20	2		0
			10	40	4		0
			10	60	6		+
			10	80	8		+
Experiment 3	8	24, 32,	5	10	2	yes	0
		40, 48,	5	20	4		0
		56	5	30	6		+
			5	40	8		+
			20	40	2		0
			20	80	4		0
			20	120	6		+
			20	160	8		+
Experiment 4	8	16, 24,	5	40	8	yes	+
		32, 40,	20	40	2		−
		48, 56,	10	5	0.5		0
		64	10	10	1		0
			10	20	2		0
			10	40	4		−,0*
			10	60	6		0
			10	80	8		+
Experiment 5	8	16, 24,	5	40	8	no	+
		32, 40,	10	40	4		0
		48, 56,	10	60	6		0
		64	10	80	8		0

The first two column headings —λ_t and V_t — refer to the wavelength and speed of the tactile stimuli, respectively. The next three column headings —λ_v, V_v, and f_v — refer to wavelength, speed, and temporal frequency of the visual stimuli. The ’sync?’ column indicates whether or not the onset and offset of the visual and tactile stimuli were (approximately) synchronous. In the effect column, (+) denotes a condition in which the visual stimulus signifi-cantly speeds up the perceived speed of the tactile comparison stimulus; (−) denotes a condition in which the visual stimulus significantly slows down the perceived speed of the tactile stimulus; 0 denotes a condition in which the visual distractor does not have a significant effect on tactile speed discrimination (TSD).

^*This distractor significantly reduced the perceived velocity of the tactile comparison stimulus when blocked with the 5-mm, 40-mm/s and the 20-mm, 40-mm/s distractors, but not when it was blocked with the other 10-mm distractors.

EXPERIMENT 2

On each trial, subjects were presented with a tactile stimulus (λ_t = 8 mm, V_t =40 mm/s) followed 500 ms later by one of the eight visual stimuli used in experiment 1. The subject’s task was to indicate, by button click, which of the two stimuli (the visual or the tactile) drifted faster. Subjects made 40 judgments for each visual comparison stimulus, presented eight times in pseudorandom order in each of five blocks.

EXPERIMENT 3

The procedure was identical to that of experiment 1. The visual distractors were 5-mm gratings drifting at 10, 20, 30, and 40 mm/s and 20-mm gratings drifting at 40, 80, 120, and 160 mm/s. The temporal frequency of the distractors was thus 2, 4, 6 or 8 Hz for both wavelengths (Eq. 2). No baseline measure was obtained from the subjects in this condition because two visual distractors used in experiment 2 were also used in experiment 3 [(5 mm, 40 mm/s) and (20 mm, 40 mm/s)]. Thus subjects’ performance could be compared across experiments without an explicit baseline measurement. Neither the 16- nor the 64-mm/s tactile comparison stimuli was presented as these were shown to be less susceptible to the behavioral effect of the distractors.

EXPERIMENT 4

The procedure was the same as that of experiment 1 except that the visual distractors drifted in the opposite direction from the tactile stimuli.

EXPERIMENT 5

The procedure was similar to that of experiment 1. Four visual gratings were tested as shown in Table 1. However, the onset of the visual distractor was 250 ms before and its offset 250 ms after the onset and offset of the tactile comparison stimulus, respectively, (the total duration of the visual distractor was thus 1,500 ms).

RESULTS

Baseline

To measure the subjects’ speed discrimination thresholds (in the absence of a visual distractor), a simple psychometric function was fit to the data obtained from each subject

$$p(V_t)=\frac{1}{1+e^{-\frac{(V_t-\mu )}{\sigma }}}$$ (3)

where p(V_t) is the proportion of times a comparison of speed V_t was judged faster than the standard (V_t = 40 mm/s), and μ and σ are free parameters. The resulting sigmoid ranges from 0 to 1. The green traces in Fig. 2 show the mean baseline performance of subjects who participated in experiment 1. The estimated point of subjective equality (PSE = μ) was 39.3 ± 0.6 (SE) mm/s. The fact that the PSE almost matched the speed of the standard (40 mm/s) indicated that there was no time-order error: presenting the comparison stimulus after the standard stimulus on each trial did not result in any systematic bias. From the resulting psychometric functions, we could estimate the comparison velocities that would be judged as faster than the standard 25 and 75% of the time. The difference between these two values divided by two is an estimate of the just noticeable difference (jnd), i.e., the increment or decrement in speed that subjects will be able to detect 75% of the time. The mean jnd was 6.5 ± 0.88 mm/s, yielding a Weber fraction (jnd/standard speed) of 0.16, which is similar to though somewhat lower than that measured in a previous study (0.2 in Essick et al. 1988). Essick et al. (1988) used brushing stimuli rather than drifting sinusoids, which may explain the difference in measured jnds: the drifting sinusoids consist of continuous motion while brushing stimuli consist of a single object (the brush) traversing a region of skin. Nonetheless, the high performance level of the subjects in the present study suggests that the stimulator produced a robust motion percept despite its inability to exert tangential forces on the skin.

FIG. 2.

Experiment 1: effect of a visual distractor on tactile speed discrimination. Each plot shows the proportion of times the comparison (λ_t = 8 mm) was judged to be faster than the standard (λ_t = 8 mm, V_t = 40 mm/s) as a function of the drift speed of the comparison. The blue trace shows the subjects’ performance in the presence of a visual distractor, the parameters of which are indicated in the bottom right of each subplot, presented at the same time as the comparison stimulus. The green trace shows baseline performance in the absence of a distractor. Plots shaded in blue show conditions in which the visual distractor significantly increased the perceived speed of the tactile comparison stimulus. Error bars represent ±1 SE.

Experiment 1

Figure 2, top three panels, shows that, of the visual distractors drifting at 40 mm/s, the grating with a wavelength of 5 mm tended to speed up the perceived speed of the tactile gratings, whereas the other two (λ_v = 10 and 20 mm) had little to no effect on tactile speed discrimination. Controlling for differences across subjects, the effect of the 5-mm visual distractor on TSD was statistically significant [F(1,21) = 27.8, P < 0.001] ,whereas the effects of the 10- and 20-mm visual distractors were not [F(1,21) = 0.9 and 1.75, respectively; P ≥ 0.2]. The bottom six panels of Fig. 2 show that, of the visual distractors with a wavelength of 10 mm, only the two fastest visual distractors tended to significantly speed up the perceived speed of the comparison [F(1,21) = 18.4 and 24.5 for the 60-and 80-mm/s visual distractors, respectively, P < 0.001]. Note that the 10-mm, 40-mm/s distractor was presented in both experimental blocks (and had no effect on TSD in either block). In summary then, only three of the 8 visual distractors had an appreciable effect on TSD: the grating of wavelength 5 mm drifting at 40 mm/s and the gratings of wavelength 10 mm drifting at 60 and 80 mm/s.

To further investigate the effect of the visual stimuli on the TSD, we fit the psychometric function described by Eq. 3 to the data obtained from each subject. We then compared the parameter values, μ and σ, fit to data obtained in each experimental condition to those obtained in the baseline condition. The parameter σ corresponds to the slope of the psychometric function, the parameter μ to the position of the function along the x axis. A decrease in σ between the baseline (no visual stimulus) and experimental conditions (with a visual stimulus) would suggest that the visual stimulus reduced the discriminability of the tactile stimuli. A decrease in μ would suggest that subjects tended to perceive the comparison stimulus as faster when it was presented with the visual stimulus. Similarly, an increase in μ would suggest that subjects tended to perceive the comparison stimulus as drifting slower when it was presented with the visual stimulus. We found that fitted values of σ were not significantly different in the baseline and experimental conditions (paired t-test, P > 0.05). In contrast, values of parameter μ fitted to TSD judgments made in the presence of the 5-mm, 40-mm/s (mean: 33.8 ± 1.2 mm/s), 10-mm, 60-mm/s (33.7 ± 1.9 mm/s), and 10-mm, 80-mm/s (31.3 ± 2.3 mm/s) distractors were significantly lower than those obtained from baseline judgments [39.3 ± 0.6; t(19) = 5.1, 3.1 and 3.7, respectively; P < 0.01]. The lower value of μ in the three distracting conditions relative to the baseline condition suggests that subjects tended to perceive the comparison as faster when it was presented with the visual distractor. For example, in the presence of the 5-mm, 40-mm/s distractor, a comparison stimulus drifting at 33.8 mm/s would, on average, be perceived as equal in speed to the 40 mm/s standard stimulus. The lack of a significant effect of the visual distractor on σ suggests that subjects’ ability to discriminate among the tactile stimuli was not impaired. In other words, over the range of tactile speeds tested, the effect of the distractor was to shift the perceived speed by a constant amount regardless of the drift speed of the comparison. By comparing the values of μ in the baseline and experimental conditions, we can estimate the degree to which the perceived speed of the tactile stimulus increased in the presence of the visual distractor. The 5-mm, 40-mm/s distractor sped up the perceived speed of the tactile stimulus by an average of 5.5 mm/s, the 10-mm, 60-mm/s distractor by 5.6 mm/s, and the 10-mm, 80-mm/s stimulus by 8 mm/s.

Experiment 2

The results of experiment 1 showed that only certain visual stimuli had a significant effect on TSD. One possibility is that the interference effect depends on the perceived speed of the visual distractor, relative to that of the tactile standard: those distractors that were perceived as drifting faster than the tactile standard produced an effect whereas the others did not. The perceived speed of a sinusoidal grating has been found to depend on its spatial frequency. Some investigators have found that perceived speed increases (Diener et al. 1976), whereas others have found it to decrease (Smith and Edgar 1990) as spatial frequency increases (at a given drift speed). It is therefore possible that the 5-mm, 40-mm/s visual grating had a significant effect on tactile speed discrimination because it is perceived as drifting faster than the 8-mm, 40-mm/s tactile grating. When we compared the perceived speed of the visual stimuli relative to that of the tactile stimuli (Fig. 3), we found this to be the case: the 5-mm, 40-mm/s distractor was perceived as drifting significantly faster than the tactile standard [t(22) = 3.1, P < 0.01], as were the 10-mm, 60-mm/s and 80-mm/s gratings [t(21) = 4.6 and 9.7, respectively; P < 0.001]; the 10-mm, 40-mm/s grating was perceived as drifting at the same rate as the tactile standard [t(22) = 0.48, P > 0.5]; the 20-mm, 40-mm/s grating was perceived as drifting significantly more slowly than the tactile standard [t(22) = 3.1, P < 0.01] as were the 10-mm, 5-, 10-, and 20-mm/s gratings [t(22) = 12.5, 14.8, and 14.7, respectively, P < 0.01]. One interesting finding in experiment 2 is that subjects could reliably discriminate cross modal drift speeds, judging the relative speeds of visual and tactile stimuli. More importantly, visual distractors that increased the perceived speed of the tactile stimulus were all perceived as drifting faster than the tactile standard (V_t = 40 mm/s). The magnitude of the distracting effect of the visual stimulus may covary with the degree to which it is perceived as drifting faster than the tactile standard.

FIG. 3.

Experiment 2: proportion of times subjects judged the visual comparison to be faster than the tactile standard (λ_t = 8 mm, V_t = 40 mm/s). Error bars represent ±1 SE.

Experiment 3

Another possibility is that the magnitude of the distracting effect of the visual stimulus is a function of its temporal frequency. Indeed, the three distractors that produced a significant effect on TSD performance oscillated at 6 or 8 Hz. We tested the hypothesis that temporal frequency was the relevant parameter by presenting visual distractors that varied in wavelength and drift speed but were equated in temporal frequency. Specifically, we presented 5- or 20-mm gratings that drifted at one of four frequencies (2, 4, 6, or 8 Hz, see Table 1 for stimulus parameters). The effects of these visual distractors on TSD are shown in Fig. 4: The magnitude of the distracting effect of the visual stimulus appears to increase with its temporal frequency. To assess whether temporal frequency alone could account for the pattern of results, we performed an ANOVA on performance with visual wavelength and temporal frequency as factors (controlling for differences across subjects and for the speed of the tactile comparison stimulus). We found that the effect of temporal frequency was highly significant [F(3,694) = 29.3, P < 0.001], whereas that of spatial period was not [F(1,694) = 3.03, P > 0.05]. Thus the effect of the visual distractor on TSD can be accounted for entirely in terms of its temporal frequency.

FIG. 4.

Experiment 3. Effect of the temporal frequency of the visual distractors on tactile speed discrimination (TSD). The effect of the distractor increases as the temporal frequency of the visual distractor increases. TSD performance can be predicted from the temporal frequency and comparison speed; the effect of spatial period is nonsignificant. Error bars represent ±1 SE.

To further probe the effect of the visual distractors on TSD, we fit psychometric functions (Eq. 3) to individual subjects’ data and examined how this effect was modulated by the wavelength and temporal frequency of the visual distractors. Figure 5 shows that the discriminability parameter σ, i.e., the slope of the psychometric function, did not change as the temporal frequency and wavelength of the distractor varied. In contrast, the bias parameter μ decreased monotonically with temporal frequency for both the 5- and 20-mm distractors. As noted, the decrease in μ indicates that, as the temporal frequency of the distractor increases, so does the perceived speed of the tactile comparison stimulus. The same result was obtained when parameters obtained in experiment 1 were replotted against temporal frequency (Fig. 6. Note that the data from experiments 1 and 3 were not combined because they were obtained from different subjects). An ANOVA on data obtained in experiment 3 confirmed that there was no significant effect of wavelength or temporal frequency on σ [F(1,136) = 0.34 and F(3,136) = 0.15, respectively, P > 0.5]. The effect of temporal frequency on μ, however, was significant [F(3,136) = 3.5, P < 0.05], whereas the effect of wavelength was not [F(1,136) = 0.17, P > 0.5]. The effect of the 8-Hz distractors in experiment 3 (λ_v = 5 and 20 mm) was comparable to that observed in experiment 1 (with λ_v = 10 mm, Fig. 6): The perceived speed of the gratings was slower by about 7 mm/s (compared with 8 mm/s in experiment 1, compare Figs. 5 and 6).¹ Thus the degree to which the visual distractor biases the perception of the speed of the tactile stimulus increases with the temporal frequency of the visual distractor and seems to be independent of its wavelength.

FIG. 5.

Effect of visual distractor on TSD plotted against temporal frequency. The parameters of the psychometric functions fit to individual subjects’ data: μ (○) denotes a biasing effect, σ (□) an effect on discriminability (see text). —, parameters fitted to data obtained with a 5-mm visual distractor; · · · , data obtained with a 20-mm distractor. The magnitude of the effect of the visual distractor is a function of its temporal frequency. Error bars represent ±1 SE.

FIG. 6.

Effect of visual distractor on TSD plotted as a function of temporal frequency (data replotted from experiment 1). The parameters of the psychometric functions fit to individual subjects’ data: μ (○) denotes a biasing effect, σ (□) an effect on discriminability (see text). Left: results with 3 different wavelength (λ_v = 20, 10 and 5 mm) when drift speed is constant (V_v = 40 mm/s). Right: results with 6 different drift speeds (V_v = 5, 10, 20, 40, 60, and 80 mm/s) when wavelength is constant (λ_v = 10 mm). Data points plotted at temporal frequency = 0 Hz denote the baseline values for the psychometric parameters. Data from the constant-speed and constant-wavelength conditions are plotted separately as they were run in separate blocks (see DISCUSSION). Error bars represent ±1 SE.

We found then that only visual stimuli that were perceived as drifting faster than the tactile standard stimulus produced a significant effect on TSD. Furthermore, we found that the effect of the distractors was a function of their temporal frequency. One possibility is that the perceived speed of the visual distractors is a function of their temporal frequency. Two results suggest that this is not the case. First, the 5-mm, 40-mm/s distractor is perceived as drifting slower than the 10-mm, 80-mm/s distractor (Fig. 3): the latter is judged as drifting faster than the tactile comparison stimulus significantly more often than is the former [see Fig. 3, F(1,21) = 10.7, P < 0.005]. Thus the perceived speed of a visual grating is not determined by its temporal frequency as the 5-mm, 40-mm/s grating has the same temporal frequency than its 10-mm, 80-mm/s counterpart and yet is perceived as drifting slower. Second, we verified that perceived speed was not solely a function of temporal frequency by presenting subjects with visual gratings identical to those presented in experiments 1 and 3 and having them rate their drift speed in a free magnitude estimation task (10 repeats per stimulus; stimuli presented in pseudorandom order). As was inferred from the results of experiment 2, the perceived speed of the gratings was not determined entirely by their temporal frequency. The results from the magnitude estimation experiment (Fig. 7) show that gratings of different spatial periods but oscillating at the same frequency were not perceived as drifting at the same speed. The magnitude of the distracting effect of the visual stimuli seems to be solely a function of their temporal frequency, and, yet, perceived drift speed is jointly determined by temporal frequency and wavelength. Based on the results of experiments 2 and 3, we conclude that the magnitude of the effect of visual motion on tactile motion perception is a function of the temporal frequency but is not dependent on the perceived (or actual) drift speed of the visual distractors.

FIG. 7.

Magnitude estimation of the drift speed of visual distractors identical to those presented in experiments 1 and 3. Each trace corresponds to the ratings (normalized within subject, averaged over 6 subjects) obtained for visual gratings at a given spatial period. Perceived speed was not solely a function of temporal frequency as evidenced by the fact that the 3 traces do not overlap. Error bars represent ±1 SE.

Experiment 4

In experiment 3, we found that the distracting effect of the visual stimuli could be explained in terms of their temporal frequency. One possibility is that the effect of the visual stimuli is not due to motion per se but rather to the oscillations inherent in these stimuli. In that case, changing the direction of the visual distractors should not alter their effect on TSD. The results from experiment 4 (Fig. 8) show that, in fact, reversing the direction of motion of the visual distractors relative to that of the tactile gratings had a significant effect on TSD. Of the three visual distractors that yielded a significant effect—an increase in perceived speed of the tactile stimulus—in experiment 1, only the10-mm, 80-mm/s grating produced a significant increase in the perceived speed of the comparison stimulus when drift direction was reversed [F(1,17) = 9.3, P < 0.01]. Note that even the effect of this distractor was reduced relative to that observed in experiment 1. Perhaps most significantly, the 10-mm, 40-mm/s and the 20-mm, 40-mm/s distractors actually slowed down the perceived speed of the tactile stimuli [F(1,17) = 14.4 and 8.6, respectively; P < 0.001], an effect never observed in the same-direction condition.

FIG. 8.

Experiment 4: effect of a visual distractor on TSD when the visual and tactile stimuli are drifting in opposite directions. Plots shaded in blue show conditions in which the visual distractor significantly increased the perceived speed of the tactile comparison stimulus; the red shading indicates conditions in which the distractor significantly decreased the perceived speed. Error bars represent ±1 SE.

When the TSD data from experiment 4 were fit to psycho-metric functions (Eq. 3), parameter σ was not significantly different when a visual distractor was presented than when it was not (P > 0.05) in all but one case: the 10 mm, 80 mm/s distractor seems to have significantly impaired subjects’ ability to discriminate between the tactile stimuli [t(17) = 2.49, P < 0.05]. Parameter μ was marginally lower than its baseline value when the 5-mm, 40-mm/s distractor was presented [t(17) = 2.15, P ≈ 0.05] and significantly lower when the 10-mm, 80-mm/s distractor was presented [t(17) = 3.28, P < 0.005]. Furthermore, parameter μ was significantly higher than baseline (i.e., the tactile comparison stimulus was perceived as slower) when the 10-mm, 40-mm/s distractor was presented [t(17) = 2.66, P < 0.05, but only when it was in a block with the 5-mm, 40-mm/s and 20-mm, 40-mm/s distractors] as well as when the 20-mm, 40-mm/s distractor was presented [t(17) = 3.52, P < 0.005]. Thus reversing the direction in which the visual gratings drifted reduced their distracting effect if they had been found to produce such an effect, or actually produced the opposite effect, that is, a decrease in the perceived speed of the tactile gratings under certain circumstances. That the effect of the visual distractors depends on the direction in which they drift suggests that their effect is not simply a result of their oscillations. Paradoxically, then, the magnitude of the effect (when tactile and visual gratings drift in the same direction) is proportional to the temporal frequency of the visual distractor but the effect itself is sensitive to the direction of motion of the distractor.

Experiment 5

A central issue in cross-modal studies is whether the observed effects are the result of perceptual changes or responses biases. In the present study, does the simultaneous presentation of a visual stimulus alter the perceived speed of the tactile stimulus or bias the subjects to respond in particular ways? A single event will typically give rise to a multi-modal sensory experience, each component of which is approximately synchronized in time. In experiment 5, this synchrony was disrupted: the visual stimulus preceded the onset of the tactile comparison stimulus by 250 ms and persisted beyond the offset of the tactile stimulus for 250 ms. We hypothesized that this disruption of the synchrony in the visual and tactile percepts would reduce the tendency for the perceptual system to associate them with a single event, which would in turn reduce the effect of the visual distractor on TSD. In contrast, if the effect of the distractor is due to response bias, then this manipulation should have little to no effect on the observed visual-tactile interaction.

In experiment 5, the three visual distractors that had previously produced a significant increase in the perceived speed of the tactile comparison stimuli (Fig. 2) were tested, along with one distractor (10 mm, 40 mm/s) that had produced no effect. As shown in Fig. 9, the effect of the visual stimulus on TSD was almost completely eliminated in the desynchronized condition: The effect of the 5 mm (V_v = 40 mm/s) distractor was still marginally significant [F(1,22) = 7.0, P < 0.05] in this condition, whereas the effects of the two effectual 10-mm distractors were nonsignificant [F(1,22) = 0.3 and 1.2 for the 60 and 80 mm/s gratings, respectively, P’s > 0.2]. Furthermore, the 10-mm, 40 mm/s distractor, which did not produce an effect in the synchronous condition also did not produce an effect in the asynchronous condition. When the data from experiment 5 were fit to psychometric functions (Eq. 3), the values of parameters μ and σ were not significantly different in the baseline and experimental conditions for all four of the distractors. The dependence of the visual-tactile integration effect on the temporal synchrony between visual and tactile stimuli suggests that the observed visual-tactile interaction reflects a perceptual bias rather than a response bias.

FIG. 9.

Experiment 5: effect of the 3 most effective visual distractors in experiment 1 when the synchrony between tactile and visual stimuli is removed, along with that of a distractor (10 mm, 40 mm/s) that did not produce an effect in experiment 1; the latter distractor still has no effect on TSD; for the former, the effect disappears almost completely and is nonsignificant except for the [5 mm, 40 mm/s] distractor. Error bars represent ±1 SE.

Individual differences

When the visual distractor produced a perceptual bias in the TSD task, it produced this bias in the majority of subjects, and no subject showed the opposite effect to any significant degree. In experiment 1, for instance, 18 of the 20 (90%) subjects yielded a lower μ on runs in which the 5-mm, 40-mm/s distractor was presented than in the baseline condition. For the 10-mm, 60-mm/s and 80-mm/s gratings, 15 (75%) and 16 (80%) of the subjects showed this effect, respectively, (see Fig. 10). However, the magnitude of the effect varied widely from subject to subject. One index of the effect of the visual distractors on TSD is the difference between the value of the psychometric parameter—particularly μ— obtained in the baseline condition and that obtained in the presence of a visual distractor. This difference denotes the magnitude of the bias (in mm/s) in the perception of the speed of the tactile comparison stimulus when a particular visual distractor is presented. In experiment 1, the most efficacious distractor was that with λ_v = 10 mm and V_v = 80 mm/s; the bias obtained in that condition ranged from 3.4 to −35.9 mm/s (i.e., slowed down the perceived speed by 3.4 mm/s for 1 subject and sped it up by 35.9 mm/s for another); the interquartile range was −1.5 to −11.4 mm/s. The ranges of biases observed in experiments 1, 4, and 5 are shown in Fig. 11. This variability in the magnitude of multisensory effects has been observed in previous experiments (e.g., Strybel and Vatakis 2004).

FIG. 10.

Psychometric functions obtained from individual subjects in experiment 1. Left: baseline TSD performance; right: performance in the presence of a 10-mm, 80-mm/s visual distractor. The dark traces show the baseline condition averaged across subjects. Most of the functions in the experimental condition fall to the left of the mean baseline performance, suggesting some degree of bias in most subjects. For some subjects, the bias was much larger in magnitude than for others.

FIG. 11.

Summary of the effects of the visual distractors on TSD in experiments 1, 4, and 5. The ‘effect index’ denotes the value of the psychometric parameter—σ or μ— obtained in the experimental condition minus that obtained in the control condition. Changes in μ suggest a perceptual bias, changes in σ point to a change in the discriminability of the stimuli. The parameters of the visual distractors are shown beneath each plot: [spatial period (mm) drift speed (mm/s)]. The box plots show the median (interior line) and upper and lower quartiles. The whiskers show the range of the data without outliers (>1 interquartile range from median). An outlier is denoted by a plus. The effect of the visual distractors varied considerably across subjects: Some subjects showed no effect whereas, for other subjects, TSD judgments were almost completely “captured” by the visual distractor.

Role of eye movements

The effect of the visual distractors on TSD may reflect not an interaction between the visual and tactile systems but rather an interaction between the motor and tactile systems. Indeed, eye movements may have been evoked during the presentation of the distractor, the frequency of which may have been determined by the temporal frequency of the gratings. It is then possible that the effect of the distractors was due to these eye movements and its dependence on the temporal frequency of the visual gratings might reflect the dependence of eye movements on this stimulus parameter. To investigate this possibility, we tracked subjects’ eye movements during the presentation of the visual distractors as they performed the tactile discrimination task. We presented visual distractors with spatial period 5, 10, and 20 mm, oscillating at 2 or 8 Hz, either drifting from left to right (congruent) or from right to left (incongruent). The resulting eye movements were analyzed using ILAB (Gitelman 2002) to specifically determine the frequency and amplitude of saccades in the various conditions. We used peak velocity as a measure of saccade amplitude as these two quantities have been shown to covary linearly (Carpenter 1988).

We wished to assess, then, whether the frequency (f_s) and peak velocity (v_p) of saccades were significantly modulated by the temporal frequency of the distractors, and by the congruency of the visual and tactile motion. To that end, we performed an ANOVA with subject, congruency of visual and tactile motion, the spatial period, and the temporal frequency of the distractor as factors. The analysis revealed that there was no significant effect of temporal frequency [F(1,6) = 5.0, P > 0.05] or congruence [F(1,6) = 0.04, P > 0.05] on saccade frequency.

Because the eye-tracker was not calibrated for each subject individually, we could not reconstruct the veridical saccade amplitude (or peak velocity) from our eye-tracking data. However, we could compute the relative velocities of all the saccades produced in a given experimental run. To compare the saccade velocities obtained in the two frequency conditions (2 and 8 Hz) across subjects, we computed the normalized peak velocity, i.e., the peak velocity of each saccade divided by the mean peak velocity across all saccades measured in a given experimental run. This normalization precluded any comparison across congruency conditions because the two conditions (congruent and incongruent) were run in separate experimental blocks. However, an ANOVA revealed that there was no significant effect of temporal frequency on v_p in the congruent condition [F(1,7) = 3.9, P > 0.05] or in the incongruent condition [F(1,7) = 0.6, P > 0.05].

Given that the temporal frequency of the visual distractors did not significantly modulate the frequency or amplitude of saccades, it is unlikely that eye movements played a critical role in the observed influence of visual motion on tactile motion perception. However, more data will be required to conclusively rule out eye movements as possible mediators in the observed multi-sensory interaction.

DISCUSSION

The main result of the present study is that visual motion influences the perception of tactile motion. Visually presented drifting sinusoids tended to increase the perceived speed of tactile drifting sinusoids when the two gratings drifted in the same direction and when the temporal frequency of the visual grating was greater than ~2 Hz (Figs. 2 and 4 – 6). The magnitude of the effect of the visual distractor on TSD, that is the degree to which it biased the perception of the tactile grating, was proportional to its temporal frequency and independent of its wavelength (Figs. 4 and 5). The degree of bias, however, could not be explained in terms of the perceived speed of the visual gratings (Figs. 3 and 7). When the visual and tactile gratings drifted in opposite directions, the intermodal interaction was modulated by the stimulus parameters in a complex way: the effect of distractors that significantly sped up the perceived speed of the tactile comparison stimuli in the congruent-direction condition was reduced when the drift-directions were incongruent (Fig. 8). In contrast, visual distractors that had no effect on TSD in the congruent-direction condition slowed down, in some cases, the perceived speed of the tactile comparison stimuli in the incongruent-direction condition (Fig. 8). Thus the relative direction of motion of the visual and tactile gratings modulates the effect of the former on the perception of the latter.

Perceptual or response bias?

The effect of the visual distractors was found to manifest itself as a bias— evidenced by a shift in the psychometric parameter μ (Eq. 3)—such that the perceived speed of the tactile comparison stimuli was higher, under certain conditions, in the presence of the distractors than in their absence. One possibility is that this bias originated in decisional rather than perceptual processes (De Gelder and Bertelson 2003). Indeed an uncertain subject may be inclined to respond that the tactile comparison stimulus drifted faster than the standard if the former is presented with a fast-moving visual stimulus.

The decision-level interpretation of the present findings is problematic for three reasons. First, under the decision bias hypothesis, it is unclear why a slow visual distractor would not bias subjects to respond that the comparison stimulus drifted slower than the standard. Second, it was shown that the magnitude of the effect of visual motion depended directly on temporal frequency but not on the perceived drift speed. One might expect that, in judging tactile speed, a response bias might be more directly sensitive to the perceived speed of the visual distractor. A third, more compelling argument against the decision-level hypothesis is that disrupting the synchrony between the visual and tactile stimuli substantially reduced the magnitude of the effect (experiment 5). Note that the visual distractor was more salient in this than in the other experiments as it started before and finished after the tactile comparison stimulus. The decision bias might thus be expected to be greater in the asynchronous condition than in the synchronous condition. We propose that the distracting effect of the visual stimulus was reduced in the asynchronous condition because the perceptual system was less disposed, in this condition, to associate the visual and tactile stimuli with a single external event and thus to incorporate visual information in the generation of the tactile motion percept. As a result, the tactile motion percept was less susceptible to the distracting effect of the visual stimulus. Our interpretation of the results of experiment 5 is that the observed influence of visual motion on TSD operates at the perceptual rather than the decisional level.

This view that the perceptual association of stimuli from two modalities is critical for intermodal effects has been advanced by other investigators. For example, Sanabria, Soto-Faraco, Chan, and Spence (2004) examined the effect of visual apparent motion on auditory apparent motion. The visual stimuli had a greater effect on the judgments of auditory motion when the perceptual grouping of the visual stimuli favored associating the visual event with the auditory event.

Integration of visual and tactile motion signals

When visual and tactile stimuli drifted in the same direction, the visual distractors increased the perceived speed of the tactile comparison stimulus but they never decreased it. The pattern of results was more complicated when the tactile and visual gratings drifted in opposite directions.

The results from the same-direction condition suggest that visual stimuli oscillating at frequencies greater than ~2 Hz (Figs. 5 and 6) tended to increase the perceived speed of the tactile gratings by a fixed amount. For instance, the presentation of a 10-mm, 80-mm/s grating resulted, on average, in an 8 mm/s increase in the perceived speed of the tactile comparison stimulus, regardless of its speed (which ranged from 16 to 64 mm/s). One interpretation of this pattern of results is that a motion signal originating from the visual system boosts the tactile motion signal by an amount determined by the temporal frequency of the visual stimulus. The low-frequency visual gratings may have had no effect on TSD because the motion signals they evoke are too weak to have an impact on the tactile motion signal. Why the temporal frequency of the visual distractors, rather than their speed, is the relevant stimulus parameter is unclear. We speculate about this result in the following text.

When the visual and tactile gratings drifted in opposite directions (experiment 4), the pattern of visual-tactile interaction was more complex and difficult to interpret. The first, obvious conclusion from this result is that the influence of visual motion information on the tactile motion percept depends on the relative direction of motion of the visual and tactile stimuli; it is not a simple integration of dynamic signals across modalities. Furthermore, the sensitivity of the observed visual-tactile interaction on relative direction of motion implies that subjects were basing their judgments on motion percepts and not simply on differences in stimulus intensity. If subjects were discriminating tactile intensity, which is independent of drift direction, judgments would have been independent of the direction of motion of the visual stimuli.

The difficulty in interpreting the results of experiment 4 may stem from the fact that there are several ways in which a subject might interpret the relationship between the visual and tactile signals evoked by gratings drifting in opposite directions. One possibility is that the sensory percepts are not evoked by the same event because they are incongruous. If the two sensory events are not perceptually bound together, one might expect the visual stimulus to have little to no effect on TSD. To recognize the incongruity between visual and tactile stimuli (that they are moving in opposite directions), however, information about the relative positions of the hand and the head has to be taken into consideration. Given the many degrees of freedom with which the hand can move relative to the head and vice versa, this computation may or may not factor into the process of integration of visual-tactile motion information. If the spatial incongruity is ignored, the two sensory events may then be perceived as stemming from the same event (given their temporal synchrony) and thus bound together. If the two sensory events are perceptually bound, there are at least two ways in which motion information from the two modalities can be combined. One possibility is that visual and tactile speed information is combined as if visual and tactile gratings were drifting in the same direction. In this case, the effect of the visual distractor on TSD would be independent of the relative drift direction of the visual and tactile stimuli. Another possibility is that the two sensory events are combined, but that they offset one another. In other words, the two motion signals are summed, which would thus result in a decrement in the perceived speed of the tactile comparison stimulus. The data do not unambiguously support either of these possibilities.

That there is an inherent ambiguity in the stimuli presented in experiment 4 can be most clearly seen in the effect produced by the 10-mm, 40-mm/s distractor. This stimulus is presented in two separate blocks: It is presented on runs in which the 5-and 20-mm gratings drifting at 40 mm/s are presented; it is also blocked with 10-mm gratings drifting at 5, 10, 20, 60, and 80 mm/s. When paired with the latter, it has no effect on TSD; when paired with the former, it significantly slows down the perceived speed of the tactile comparison stimulus (Fig. 8). Thus the perceived relationship between the 10-mm, 40-mm/s distractor and the tactile comparison stimulus (as gauged by the effect of the former on the perception of the latter) depends on the context in which the two stimuli co-occur. The visual distractors presented on the same block bias the perceptual system toward one or the other interpretations of the relationship (or lack thereof) between the visual and tactile stimuli, although it is unclear why this should be the case. Note that this context-sensitivity is only observed when the two stimuli are drifting in opposite directions. The 10-mm, 40-mm/s grating has the same effect when presented in two different blocks in experiment 1 (compare the middle panel in the top row to the bottom left panel in Fig. 2). There may be an inherent tendency to integrate the visual and tactile motion stimuli. The two stimuli share several features, sinusoidal stimuli moving across a sensory sheet. The fact that the two stimuli are moving in opposite directions, at least in allocentric space, may lead to greater instability in perceiving them as arising from the same event.

Spatial considerations

In the present study, the tactile and visual stimuli could not be presented in the same spatial location because of the geometries of the tactile stimulator and computer monitor. However, according to the binding hypothesis (described previously), stimuli which are spatially congruent (in addition to being congruent in their direction of motion and onset and offset timing) might be even more tightly bound together than stimuli which are congruent only in their direction of motion and onset and offset timing (as was the case in the present study). In general, we expect that, as the congruence between visual and tactile stimuli increases, the degree to which one stimulus affects the perception of the other will increase.

Another way in which the visual and tactile stimuli might be conceived as incongruent is in their spatial dimensions: The tactile stimuli spanned a 1-cm² area, whereas the visual stimuli were much larger (30 × 23 cm). However, the tactile stimuli covered most of the finger pad. Thus enlarging the tactile stimulus would not substantially increase the area over which the tactile stimulus contacts a single fingerpad. Nonetheless, it is likely that a greater degree of binding would be observed if the dimensions of the visual and tactile stimuli were comparable, and the distracting effect of the visual stimuli might as a result increase. Another possibility is that the motion processing mechanisms were overloaded by the large visual stimulus presented in the present study, and that a smaller visual distractor might not interfere with TSD to the same extent.

Spatial incongruities between the visual and tactile stimuli may explain why temporal frequency was the parameter that determined the distracting efficacy of the visual stimuli. Indeed, if both visual and tactile stimuli are perceived as stemming from the same event—a drifting sinusoid with a given spatial period—then their speed will be proportional to their temporal frequency. In other words, speed is defined relative to both a spatial coordinate system and a temporal frame of reference (mm/s), whereas temporal frequency is defined only in the temporal dimension (Hz) and is independent of the spatial frame of reference. Because the visual and tactile stimuli occupied two different regions in space and had different spatial dimensions, visual and tactile motion may have been computed within two separate spatial frames of reference. Within the spatial domain, then, a mapping between the visual and tactile coordinate systems must be effected to integrate motion across these two modalities. However, no such mapping is necessary if the integration operates within the temporal domain. The cross-model integration of motion information may thus have operated within a temporal frame of reference, common to both modalities, an operation for which temporal frequency is the critical quantity.

Neural mechanisms

A promising candidate cortical locus for the integration of visual and tactile motion information is the human middle temporal/V5 area (MT/V5). Indeed, this area, traditionally associated with visual motion processing, has recently been shown to receive tactile input (Blake et al. 2004; Hagen et al. 2002; Moore et al. 2005; Nelson et al. 2005). Hagen et al. (2002) showed that a brush stroke on the forearm produced activation in regions of the middle temporal area that were also activated by visual motion. Blake et al. (2004) showed that touching a moving globe (without seeing it) produced activation in MT/V5. Most recently, Moore and colleagues have demonstrated that vibratory stimuli presented to the hand activate MT/V5, in both humans (Moore et al. 2005) and monkeys (Nelson et al. 2005). However, the role of MT/V5 in tactile motion perception remains unclear. One possibility is that this area plays a role in disambiguating ambiguous visual motion through tactile input (Blake et al. 2004). Another possibility is that the activity observed in area MT/V5 during tactile stimulation reflects visual imagery or other top-down influences. P. Aparicio and C. L. Moore (personal communication) have shown that activity in MT/V5 elicited by tactile stimulation is greater when subjects are judging tactile apparent motion than when they judge the temporal order of two vibratory pulses presented in rapid succession, implicating MT/V5 in tactile motion processing. If, indeed, MT/V5 plays a role in tactile motion perception, then it is plausible that MT/V5 also underlies the visual-tactile interactions observed in the present study.

Another potential, though less promising candidate brain region where the integration of visual and tactile information observed in the present study might take place is the ventral inferior parietal cortex (VIP). Indeed, this area comprises neurons that respond to both visual and tactile stimuli (Duhamel et al. 1998). Furthermore, many of these neurons are sensitive to the direction of motion (the speed sensitivity of VIP neurons was not tested). However, most of the neurons in VIP have receptive fields on or near the face, which seems to exclude this area as a substrate for the inter-sensory phenomenon presented here.

Conclusions

In the present study, we find that visual motion influences the perception of tactile motion. When visual and tactile sinusoids drifting in the same direction are presented simultaneously, the visual stimuli enhance the perceived speed of the tactile stimuli (Fig. 2). The degree of the observed changes in speed judgments is proportional to the temporal frequency of the visual stimuli (Figs. 4 – 6). When visual distractors drift in the opposite direction from the tactile stimuli, their effect is reduced, eliminated, or, in some cases, reversed (Fig. 8). Thus the effect is dependent on the relative direction of motion of the two stimuli. Finally, when the temporal synchrony of the visual and tactile stimuli is disrupted, the distracting effect of the visual motion on TSD is almost completely abolished, suggesting that the effects are perceptual rather than decisional in nature (Fig. 9).