Temporal frequency channels are linked across audition and touch

Summary

Temporal frequency is a fundamental sensory dimension in audition and touch. In audition, analysis of temporal frequency is necessary for speech and music perception [1]; in touch, the spectral analysis of vibratory signals has been implicated in texture perception [2, 3] and in sensing the environment through tools [4–7]. Environmental oscillations impinging upon the ear are generally thought to be processed independently of oscillations impinging upon the skin. Here, we show that frequency channels are perceptually linked across audition and touch. In a series of psychophysical experiments, we demonstrate that auditory stimuli interfere with tactile frequency perception in a systematic manner. Specifically, performance on a tactile frequency-discrimination task is impaired when an auditory distractor is presented with the tactile stimuli, but only if the frequencies of the auditory and tactile stimuli are similar. The frequency-dependent interference effect is observed whether the distractors are pure tones or band-pass noise, so an auditory percept of pitch is not required for the effect to be produced. Importantly, distractors that strongly impair frequency discrimination do not interfere with judgments of tactile intensity. This surprisingly specific crosstalk between different modalities reflects the importance of supramodal representations of fundamental sensory dimensions.

Results and Discussion

Mechanical oscillations in the environment are transduced by specialized receptors in the ear or the skin. Auditory spectral analysis is critical for perceiving auditory objects, especially in the context of music and speech [1]. Spectral analysis of tactile signals in isolation [2] or in conjunction with auditory signals [3, 8–12] plays an important role in texture perception. Despite the frequent co-occurrence of congruent acoustical and vibrotactile signals, the relationship between auditory and tactile spectral analysis has never been systematically studied. In a series of human psychophysical experiments, we report differential influence of auditory distractors on the perceptual readout of the frequency and intensity of tactile stimuli. Both auditory pure tones (aPT) and auditory band-pass noise (aBPN) impaired and biased judgments of perceived tactile frequency, but not perceived intensity, in a frequency-dependent manner.

Tactile frequency discrimination was tested using a two-alternative forced-choice (2AFC) design (Fig. 1A). On each trial, a pair of vibrotactile pure tones (tPT), equated for perceived intensity, was delivered sequentially to the participant’s right index finger; his or her task was to report which of the two tones was higher in frequency (Experiment 1; see Experimental Procedures in Supplemental Data (SD) for a complete description of the Methods). The stimuli were each presented for 1-sec and were separated by a 1-sec inter-stimulus interval. One interval always contained a 200-Hz tPT (standard stimulus); the frequency of the tPT presented during the other interval (comparison stimulus) ranged from 100 to 300Hz in 40-Hz increments. On most trials, an aPT distractor was presented at the same time as the tPT comparison stimulus (In a control experiment, aPT distractors were presented during both the comparison and standard stimuli, yielding similar results (SD; Fig. S2)). Participants were instructed to ignore the auditory distractors. The aPTs ranged in frequency from 100 to 1500Hz and were equated to one another in perceived intensity. On the remaining trials, no auditory distractor was presented in order to establish a baseline against which we could compare performance achieved in the presence of auditory distractors. Great care was taken to ensure that the tactile stimuli were inaudible (SD).

Figure 1.

Tactile frequency discrimination in the presence of auditory pure tone (aPT) distractors (14 and 20 subjects for the 200-Hz and 400-Hz standards, respectively). (A) Experimental design. (B,C) Proportions of trials the comparison was judged higher in frequency, averaged across all participants, as a function of the comparison frequency f_c, tested with the 200- (B) and 400-Hz (C) standards. Each colored trace corresponds to a different aPT distractor (blue denotes baseline performance). Error bars indicate s.e.m. (D) Mean sensitivity (σ) as a function of distractor frequency (filled points indicate means that are significantly different from baseline). The gray dashed line and untethered bars correspond to baseline and s.e.m, respectively. The blue and red vertical dashed lines correspond to the frequency of the standards (200- and 400-Hz respectively). With the 200-Hz standard (blue trace), aPT distractors ranging from 100 to 300Hz significantly reduced sensitivity (i.e. yielded a larger σ) (t(13) = 4.3, P = 0.0009, t(13) = 4.0, P = 0.002, and t(13) = 3.35, P = 0.005, respectively). With the 400-Hz standard (red trace), the 100-, 200-, 300-, 400-, and 1000-Hz aPT distractors significantly decreased sensitivity (t(17) = 4.5, P = 0.0003, t(17) = 3.8, P = 0.001, t(18) = 3.8, P = 0.001, t(18) = 2.9, P = 0.009, and t(19) = 3.0, P = 0.008, respectively). (E) Mean bias (μ) as a function of distractor frequency (conventions as in (D)). With the 200-Hz standard, only the 100-Hz aPT significantly biased perceived tactile frequency (t(13) = 3.58, P = 0.003). With the 400-Hz standard, aPT distractors ranging from 100 to 300Hz significantly biased perceived tactile frequency (t(17) = 4.6, P = 0.0003, t(17) = 3.7, P = 0.002, and t(18) = 3.0, P = 0.007).

We found that auditory distractors differentially affected tactile discrimination performance depending on their frequency (Fig. 1B). We quantified the effect of the distractors by fitting psychometric functions to individual participant’s data. We then compared the sensitivity (σ) and bias (μ) obtained in each distractor condition to that obtained in the baseline condition. Sensitivity is a measure of participants’ ability to discriminate the stimuli (the change in frequency, with respect to the standard, that is discriminable 73% of the time) and bias corresponds to the point of subjective equality (the comparison frequency perceived to be equal to the standard frequency). Auditory distractors reduced participants’ sensitivity to differences in tactile frequency (repeated-measures ANOVA, F_8,104 = 10.9, P < 10⁻⁴; Fig. 1D, blue trace) and did so in a frequency-dependent manner (F_7,91 = 11.4, P < 10⁻⁴). Specifically, the 100-, 200-, and 300-Hz aPT distractors reduced sensitivity (2-tailed paired t-test, all P values < 0.01, with Bonferroni correction), the 100-Hz distractor by a factor of approximately three. The aPT distractors also biased the perceived frequency of the comparison tPT (F_8,104 = 10.1, P < 10⁻⁴; Fig. 1E, blue trace); the magnitude of the bias also depended on distractor frequency (F_7,91 = 10.8, P < 10⁻⁴). In general, the perceived frequency of the tactile stimuli tended to be pulled towards the frequency of the aPT; however, only the bias induced by the 100-Hz distractor achieved statistical significance (P = 0.003). Given that the subjects were instructed to ignore the auditory distractors, the bias seems to reflect an automatic and unconscious influence of auditory stimuli on the perception of their tactile counterparts, an effect loosely analogous to the McGurk or ventriloquism effects [13]. The degree to which auditory distractors affected individual participants’ performance varied (SD; Fig. S3). In summary, perception of tactile frequency is both impaired and biased by low-frequency but not high-frequency aPT distractors.

We then tested whether audio-tactile interference depends on the similarity in frequency of the auditory and tactile stimuli, or whether only low-frequency aPT distractors influence the tactile processing of stimulus frequency. To this end, we repeated the experiment using a 400-Hz tactile standard with comparison stimuli ranging in frequency from 200 to 600Hz (in the presence of the same aPT distractors). In this condition, aPTs impaired and biased perceived tactile frequency over a wider range of distractor frequencies (F_8,152 = 5.2, P < 10⁻⁴ and F_8,152 = 9.9, P < 10⁻⁴, respectively; Fig. 1D,E, red traces). The magnitude of the impairment and bias again depended on distractor frequency (F_7,133 = 3.8, P = 0.001 and F_7,133 = 9.7, P < 10⁻⁴, respectively). Post-hoc tests revealed that aPT distractors at frequencies lower than 600Hz significantly reduced sensitivity to differences in tactile frequency (all P values < 0.01). In addition, distractors at frequencies lower than 400Hz induced significant biases (all P values < 0.01). Thus, audio-tactile interference is not restricted to low frequency auditory distractors, and instead depends on the similarity of the auditory and tactile frequencies.

Importantly, in control experiments, we found that the efficacy of the auditory distractors did not depend on distractor intensity (SD; Fig. S5) nor did it depend on the relative onset and offset times of the tactile and auditory stimuli, so long as the duration of overlap (1 sec) was preserved (SD; Fig. S6).

Impaired performance due to aPT distractors may reflect interference at a sensory level (tactile and auditory inputs converge before pitch is resolved) or at a decisional level (convergence occurs after tactile and auditory pitch have been independently resolved). Whether the experience of auditory pitch is necessary for audio-tactile interference to occur has implications regarding the neural mechanisms underlying the observed effects. Impairment of task performance by aBPN distractors would imply that the interference occurs at the sensory level rather than the decisional level. To test whether audio-tactile interference requires the perception of auditory pitch, we repeated the tactile frequency discrimination experiments using auditory band-passed noise (aBPN) distractors that varied in center frequency (CF = 150, 300, 600, and 1200Hz) and bandwidth (BW = 2/3·CF) (Experiment 2; Fig. 2A). For the 200- and 400-Hz standards (Fig. 2B,C), aBPN distractors reduced sensitivity (F_4,48 = 7.4, P = 0.0001 and F_4,72 = 5.1, P = 0.001, respectively; Fig. 2D) and biased perceived frequency (F_4,48 = 6.8, P = 0.0002 and F_4,72 = 9.7, P < 10⁻⁴, respectively; Fig. 2E). The magnitude of the impairment depended on CF (F_3,36 = 7.2, P < 10⁻⁴ and F_3,54 = 3.7, P = 0.02, for the 200- and 400-Hz standards, respectively). Post-hoc tests revealed that, for both sets of tactile stimuli, the aBPN_CF=150Hz and aBPN_CF=300Hz distractors significantly reduced sensitivity (all P values < 0.01) whereas the other two aBPN distractors did not. The magnitude of the induced bias also depended on CF (F_3,36 = 9.6, P = 0.0001 and F_3,54 = 11.2, P < 10⁻⁴). The aBPN distractors tended to bias perceived tactile frequency in the direction of the distractor CF; only the aBPN_CF=1200Hz distractor induced a significant bias with the 200-Hz standard (P = 0.002) whereas both the aBPN_CF=150Hz and aBPN_CF=300Hz distractors significantly biased perceived tactile frequency with the 400-Hz standard (P = 0.003 and P = 0.008, respectively). Together, these results show that the aPT and aBPN distractors comparably affected tactile frequency discrimination: distractors reduced sensitivity to the extent that the frequency content of the auditory and tactile stimuli was similar and distractors systematically biased perceived tactile frequency. Critically, results with the aBPN distractors demonstrate that interference does not require the elicitation of an auditory pitch percept, and suggests that analysis of tactile frequency is mediated by neural populations sensitive to both aPT and aBPN stimuli.

Figure 2.

Tactile frequency discrimination in the presence of auditory band-passed noise (aBPN) distractors (14 and 19 subjects for the 200-Hz and 400-Hz standards, respectively) (conventions as in Fig. 1). (A) Experimental design. (B,C) Discrimination performance, averaged across participants, obtained with the 200- (B) and 400-Hz (C) standards. (D) With the 200-Hz standard, the aBPN_CF=150Hz and aBPN_CF=300Hz distractors significantly impaired sensitivity (t(12) = 3.1, P = 0.009 and t(12) = 3.3, P = 0.007, respectively). With the 400-Hz standard, the aBPN_CF=150Hz and aBPN_CF=300Hz distractors significantly impaired tactile frequency discrimination (t(18) = 2.9, P = 0.009 and t(18) = 3.4, P = 0.003, respectively). (E) With both standards, there was a general trend for aBPN distractors to bias the perceived tactile frequency towards the distractor CF. With the 200-Hz standard, only the aBPN_CF=1200Hz distractor induced a significant bias (t(12) = 4.0, P = 0.002). With the 400-Hz standard, the aBPN_CF=150Hz and aBPN_CF=300Hz distractors significantly biased perceived tactile frequency (t(18) = 3.4, P = 0.003 and t(18) = 3.0, P = 0.008, respectively).

For both 200- and 400-Hz tactile standards, the greatest interference of tactile frequency processing occurred when the distractor frequencies were lower than the tactile frequencies (this was evident in experiments with aPT distractors). This result is reminiscent of the finding that auditory stimuli more effectively mask (auditory) stimuli at higher frequencies than they do stimuli at lower frequencies [1]. Indeed, psychophysical studies of auditory masking using the notched-noise method revealed asymmetric auditory filter shapes at low center frequencies (100–800Hz), with the lower skirt being shallower than the upper skirt [14]. While this asymmetry is typically attributed to the biomechanics of the basilar membrane, our results raise the interesting possibility that the shape of the auditory filter may reflect the properties of cortical neurons as well. This asymmetry can also be considered in the context of Bayesian models of perception [15]. Within this framework, perceptual decisions about the environment are inferences based on current sensory stimulation and prior knowledge about the environment. This approach has been used to model visual perception of objects, motion, and color constancy, among other phenomena [15]. The distribution of all possible co-occurrences of auditory and tactile stimuli may be skewed such that perceptible tactile frequencies tend to be lower than the frequencies of co-occurring auditory stimuli. For instance, while exploring surfaces tactually, both tactile and auditory inputs contribute to our textural percept [3]. The tactile input may tend to span a lower frequency range than its auditory counterpart. Our perceptual system may then represent prior beliefs that also reflect this asymmetry.

We next determined whether the audio-tactile interference was specific to the perception of tactile frequency, or if auditory distractors affected the perception of vibrotactile stimuli in general. We tested whether the perception of tactile intensity was also subject to auditory interference using a 2AFC design (Experiment 3; Fig. 3A). On each trial, participants were presented sequentially with a pair of tPT stimuli of equal frequency (100 or 200Hz) but differing in amplitude, and reported which of the two was more intense. One interval always contained a tPT at a standard amplitude (A_s) while the other contained a tPT at an amplitude (A_c) ranging from 0.5·A_s to 1.5·A_s. The comparison stimulus was presented either alone or with one of the aPT distractors used in Experiment 1. While auditory distractors interfered with participants’ ability to discriminate the frequency of tactile stimuli, these same distractors did not interfere with participants’ ability to judge tactile intensity (Fig. 3B,C): Distractors neither impaired sensitivity to tactile intensity differences (F_6,24 = 1.7, P = 0.16 and F_6,24 = 2.0, P = 0.1, for the 100- and 200-Hz tPT, respectively; Fig. 3D) nor did they bias perceived tactile intensity (F_6,24 = 0.48, P = 0.82 and F_6,24 = 0.97, P = 0.46; Fig. 3E). Furthermore, aPT distractors that varied in intensity had no effect on perceived tactile intensity (SD; Fig. S7). While tactile stimulation has been shown to influence the perceived intensity of simultaneously presented auditory stimuli [16, 17], we find that the perception of tactile intensity is not affected by the presentation of auditory distractors. Such asymmetry in multisensory interactions has been described previously [13, 18–20].

Figure 3.

Tactile intensity discrimination in the presence of auditory pure tone (aPT) distractors (6 and 5 subjects for the 100-Hz and 200-Hz tPTs, respectively) (conventions as in Fig. 1). (A) Experimental design. (B,C) Proportions of trials the comparison was judged higher in intensity, averaged across all participants, as a function of the comparison amplitude A_c, tested with 100- (B) and 200-Hz (C) tPT. (D) Sensitivity to differences in intensity was not significantly affected by auditory distractors (repeated-measures ANOVA, F_6,24 = 1.7, P = 0.16 and F_6,24 = 2.0, P = 0.1, respectively). (E) Similarly, the perceived intensity of the 100- and 200-Hz tPT was not significantly biased by aPT distractors (F_6,24 = 0.48, P = 0.82 and F_6,24 = 0.97, P = 0.46, respectively).

In sum, we find that the somatosensory and auditory modalities interact in the analysis of stimulus frequency but not intensity. The role of auditory cortices in the spectral analysis of acoustic stimuli is undeniable. Although the cortical representation of “flutter” frequencies (<80Hz) has been extensively studied [21–24], where, how, or even whether the somatosensory cortices process high frequencies is debatable. The perception of flutter and the perception of vibration are likely elaborated along distinct processing streams [24, 25]. The available evidence suggests that, in primary and second somatosensory cortex, information about stimulus frequency in the vibration range (> 80Hz) is not carried in the temporal patterning of the neural response, as the entrainment of individual neurons to vibrations is weak [26]. Whereas the cortical substrate underlying the spectral analysis of high-frequency tactile inputs is beyond the scope of the present study, our results yield clear predictions. Specifically, the putative neural populations (a) are independently driven by tactile and auditory stimulation and (b) are responsive to auditory pure tones and band-pass noise. Indeed, belt areas in the auditory association cortex, notably the caudo-medial area (CM), have been shown to fulfill these requirements. First, the convergence of auditory and tactile signals in these auditory areas has been established in humans [8, 10, 27–29] and non-human primates [30, 31] using a variety of neuroimaging and neurophysiological techniques. Second, these areas have been shown to respond to both aPT and aBPN stimuli [32, 33]. Studies employing anatomical tracers have identified potential thalamic and cortical sources of somatosensory input to CM [34, 35]. While area CM has been ascribed a potential role in spatial localization based on the characterization of neuronal tuning properties [32, 33], audio-tactile interactions have been observed when stimuli were in and out of spatial register [27]. Interestingly, a link between spatial processing and perception of auditory tones has been recently suggested [36].

Our hypothesis, that regions of auditory cortex (like area CM) participate in spectral analysis of high-frequency tactile information as well as auditory information, yields many testable predictions. For instance, we predict that individual cortical neurons should show frequency-tuned responses to auditory and tactile stimuli. Furthermore, we predict that interfering with cortical processing in these areas, using techniques such as transcranial magnetic stimulation (TMS) [37], would severely impair or abolish participants’ ability to discriminate tactile frequency, but not intensity. Similarly, adaptation paradigms [22, 38] can be used to test the frequency-tuning of the tactile responses observed in the auditory areas. Specifically, repeated presentations of the same tactile frequency should result in adaptation of the BOLD signal if these auditory areas subserve tactile frequency processing. Moreover, if the same neuronal populations underlie auditory and tactile spectral processing, one would predict adaptation of the BOLD response to repeated presentations of the same frequency regardless of modality.

That the readout of tactile intensity is unimpaired by auditory processing is compatible with our hypothesis; the representation of tactile stimulus intensity is a function performed by another region of cortex. Our results support the view of a supramodal organization of the brain in which areas are conceptualized as operators that perform specific functions regardless of input modality [39]. Functional subdivisions, based on multisensory responses observed in traditionally unisensory cortices, have been suggested for processing object shape [39–41], motion [42, 43], and microgeometric features [44, 45]. Our results suggest that auditory cortical regions like area CM may serve as supramodal operators for spectral analysis.