Visual effects on tactile stimulation and its perception: A pilot study using near-infrared spectroscopy

Abstract

In rubber hand illusion, visual information affects tactile information, whereas in the mirror box illusion, visual information has the opposite effect. However, its underlying mechanisms are not fully understood. As one of the reasons, non-invasive neuroimaging techniques, such as functional magnetic resonance, positron emission tomography, and electroencephalography, often fail to detect complex and fragile responses in the sensory-motor cortex. Using near-infrared spectroscopy (NIRS), we examined neural activity during tactile tracing on a sine-shaped acrylic board to investigate the effects of (1) visual information and (2) the spatial frequency of the sine shape on brain activity. We used spatial frequencies of 2–3 and 20–30 Hz as low- and high-tactile stimuli, respectively. Two types of experiments, with and without an acrylic board, were conducted. Participants performed the tracing tasks with their index finger at 1 Hz of temporal frequency of a 200 mm length of the acrylic board as main tasks and only space moving without touching as a control task. We show effect of visual information on neural activation, including not only activation intensity but also activation patterns.

• •Testing of mutual effects of vision and haptics.
• •Testing of sensory-motor paradox using NIRS.
• •A high NIRS sensitivity to stimulus-induced hemodynamic change.

Graphical abstract

Specifications table

Subject area:	Neuroscience
More specific subject area:	Neuroimaging
Name of your method:	Near-infrared spectroscopy for detecting neural activity during tactile stimulation
Name and reference of original method:	A. Seiyama, et al., Circulatory basis of fMRI signals: Relationship between changes in hemodynamic parameters and BOLD signal intensity, Neuroimage. 21 (2004) 1204-14. DOI: 10.1016/j.neuroimage.2003.12.002
Resource availability:	FOIRE-3000, Tactile stimuli

Background

In our daily lives, we correctly perceive sensations. It is thought that the addition of visual information to tactile information, or vice versa, leads to more accurate recognition. There are two unique examples called the rubber hand illusion [1,2] and the mirror box illusion [3,4]. The former brings tactile information from visual information [1,2], and the latter brings visual information from tactile information [3,4]. These phenomena indicate that the senses of touch and vision influence each other. However, the detailed mechanisms of these phenomena remain unknown, and phenomenological discussions are ongoing. If the details of brain activity associated with tactile and visual information and their interactions are further clarified in near future, it may potentially improve the quality of rehabilitation for patients with sensory paralysis due to the dementia and aftereffects of cerebral infarction/hemorrhage, as well as to address visual impairment in modern society where touch panels have become widespread. This has the potential to improve the quality of life for people with visual and sensory-cognitive impairments.

Near-infrared spectroscopy (NIRS) has enabled the non-invasive measurement of changes in cerebral blood volume (CBV) and flow (CBF) because biological tissues are relatively transparent to light in the near-infrared region at wavelengths of 700–1000 nm. This technique has recently been applied to develop a brain-computer interface [5] and to monitor CBF in patients in the fields of anesthesiology [6], psychiatry [7], pediatrics [8]; fetuses and infants in the fields of Obstetrics (and Gynecology) [9]; and autonomic nervous function under cognitive and physiological conditions [10].

To date, few reports have been published on the complex changes in brain activity caused by the interactions between tactile and visual responses. As one of the reasons, the sensory motor cortex shows very complex responses to tactile stimulations called “sensory motor paradox” [11] which is unstable for measurements using non-invasive neuroimaging techniques such as functional magnetic resonance imaging (fMRI), positron emission tomography (PET) and electroencephalogram (EEG).

In our previous study, we demonstrated the capacity of multichannel NIRS to detect complex responses [12]. Therefore, in this study, we aimed to verify the neural responses to tactile stimuli by focusing on the following two points using multichannel NIRS as a fundamental study of the sensory interaction between touch and vision: (a) the effect on brain activation when tactile stimulation is accompanied by visual information and (b) the effect of tactile stimulation frequency on brain activation.

Method details

Participants

Six healthy volunteers (22–25 years old; 1 man, 5 women) participated in this study. All the participants were right-handed. While the participants were asked to listen to pink noise to suppress the influence of environmental sounds and a 1 Hz metronome sound as a rhythm indicator, they performed back-and-forth movements from side to side on an acrylic stimulus with a length of 200 mm and a width of 5 mm (Fig. 1A).

Fig. 1.

(A) Acryl tasks used as sensory stimuli. Top, low frequency tactile stimulus with the periodic wavelength was 47 mm (thus, the stimulus frequency = about 2–3 Hz). Bottom, high frequency tactile stimulus with the periodic wavelength was 4.7 mm (thus, the stimulus frequency = about 20–30 Hz). (B) Task design used present study. A block design with one set of ‘10 s of rest + 10 s of task + 10 s of rest’ was used where three tasks, low and high frequency tactile stimuli and only motions corresponding frequency, were used.

Sensory-motor stimulation

To elicit stimulus-induced brain activation, a block design was used for task performance (Fig. 1B), where one set of 10 s of rest + 10 s of task + 10 s of rest and nine sets per session were performed six times (total 54 sets). After each set, the participants rested for a sufficient amount of time so that the previous set did not affect the next set. Three types of tasks were used as tactile stimuli—low-frequency stimulus (periodic wavelength 47 mm, approximately 2–3 Hz); high-frequency stimulus (periodic wavelength 4.7 mm, approximately 20–30 Hz); and motor tasks with no contact with the acrylic plates. Three sets of each were administered in a pseudorandom manner during one session (nine sets per session). During the rest period, participants were instructed not to touch the acrylic plate and to keep the index finger in the air.

Experimental design

As described above, each session consisted of Experiment A (involving tactile information and visual information), conducted while looking at the fixation point (black dot) on the task model (cf. Fig. 1), and Experiment B (involving tactile information without visual information), conducted without looking at the task model but looking at the screen in front of the model. Three pseudo-random sessions were conducted for each experimental condition. Task instructions were provided verbally. The signal to start a task was given by stating the type of task for that set, and the signal to end a task was given by stating the type of task for the next set. Additionally, to suppress eye movement, participants were instructed to fixate on the fixation point at the center of the model in Experiment A and on the fixation point at the center of the screen in Experiment B. In the center of the screen, a symbol (■ during task, ✚ during rest) flashed at a 1 Hz rhythm as a rhythm indicator, and below the symbol the type of task should perform was displayed in the same way as verbal instructions. Before starting the measurements, participants rested sufficiently to avoid the effects of the preceding task.

Near infrared spectroscopy for measuring neural activation

The 14-channel NIRS (FOIRE-3000; Shimadzu Co., Kyoto, Japan) was used. Optical probes comprising three emission and three detection optical fibers were set on the sensory-motor cortex areas in both hemispheres, centered at the C3 and C4 positions of the International 10–20 system of EEG (Fig. 2A). Fig. 2B illustrates the areas covered by individual optical channels (chs) (e.g., chs 1, 3 and 6 for the left motor areas; chs 2, 5 and 7 for the left somatosensory area). The channel 4 (C3) was positioned along the left colonal plane as indicated by the broken line. Stimulus-induced neural activation causes a large increase in oxy-Hb and a small decrease in deoxy-Hb levels after a short delay in the onset of stimulation [12]. Therefore, the amount of change in oxy-Hb is considered an indicator of brain activity [12]. An increase in the set average value of oxy-Hb relative to the grand average can be defined as brain activation [13].

Fig. 2.

(A) Position of optical probes on the sensory motor cortex. The probes were centered at C3 (left) and C4 (right) positions of International 10–20 EEG system. Each probe consisted of three emission and three detection optical fibers gave seven channels in the hemisphere. Black circle dots: incident light-fibers, White circle dots: detection-fibers. (B) Measurement positions and areas on the sensory and motor cortices. Channels (chs) 1, 3 and 6 and chs 8, 10 and 13 are on the left and right motor areas (solid circles), respectively. Chs 2, 5 and 7 and chs 9, 12 and 14 are on the left and right sensory areas (dashed circles), respectively.

Signal processing

Raw NIRS signals contain various types of noise, necessitating the removal of this noise to perform accurate evaluations. Therefore, in this study, we performed the following signal processing on the obtained NIRS raw signals in the following order: (a) a band pass filter between 0.01 and 0.75 Hz was applied to remove the high- and low-frequency noises as well as the heart or pulse rate, (b) separation of each task set (9 sets/1 session) for addition (see below d)), (c) baseline correction for each set by subtracting the average signal value over the initial 10 s of the set, and (d) summation and averaging of each signal for low, high, or motor task conditions in Experiments A and B, respectively. In this study, stimulus-induced changes in oxygenated hemoglobin (oxy-Hb) levels were used as indices of neural activation (Fig. 3).

Fig. 3.

An example of stimulus-induced changes in oxy-Hb (obtained at Ch 8) levels. Mean values and standard deviations of time courses from six participants are shown. Top three figures (low A, high A and motor A) and Bottom three figures (low B, high B and motor B) were results obtained using Experiment A (tactile stimulation + vision) and Experiment B (tactile stimulation only), respectively. The low and high mean the low frequency tactile stimulation (2–3 Hz) and high frequency tactile stimulation (20–30 Hz). The vertical red lines denote the start and stop of the stimulation (10 s).

Statistics

Following the aforementioned signal processing, a generated a grand average by averaging the waveforms for each channel and condition across the six people. Subsequently, we conducted an analysis of variance assuming the null hypothesis that there was no difference in the grand average between the tasks (Low, High, and Motor) for each channel. In cases where channels exhibited a significant difference, we conducted multiple comparisons using the Tukey method, identifying combinations of tasks with significant differences. In the study, p-values below 0.05 and 0.01 were considered statistically significant and denoted as p < 0.05 (*) and p < 0.01 (**), respectively.

Experimental data comparison

Effects of visual information on the sensory-motor perception

Table 1 shows the statistical significance of neural activation in each channel measured in the somatosensory areas, and Fig. 4 shows the results. Comparing the Experiment A (visual information + tactile task) and the Experiment B (only tactile task), at low frequency stimulation, in all measured areas (i.e., left motor (chs 1, 3 and 6), right motor (chs 8, 10 and 13), left somatosensory (chs 2, 5 and 7), and right somatosensory cortices (chs 9 and 12)), the Experiment B gave higher brain activity than that of Experiment A (Fig. 4A). At high-frequency stimulation, Experiment A showed higher brain activation than Experiment B in the left motor (chs 3 and 6) and left somatosensory cortices (chs 2 and 7), whereas in the right motor (ch 8) and right somatosensory cortices (chs 9 and 14), Experiment B showed higher brain activation than Experiment A (Fig. 4B). During exercise alone without tactile stimulation, Experiment B showed higher brain activation in the right motor cortex than Experiment A (Fig. 4C). These results indicate that visual information interferes with tactile information.

Table 1.

Effects of visual information on sensory-motor task perception.

Blue cells indicate statistically significant channels (Exp. B > Exp. A) compared to the ground-averaged NIR signals. Red cells indicate statistically significant channels (Exp. B < Exp. A) compared to the ground-averaged NIR signals. *, p < 0.05; **, p < 0.01.

Fig. 4.

Effects of visual information on sensory-motor task perception. Visualization of Table 1 is shown. The color blue denotes the hinger neural activation in Experiment B than Experiment A, while red denotes the hinger neural activation in Experiment A than Experiment B. The size of triangle denotes the significance of the neural activation (i.e., large (**, p < 0.01); small (*, p < 0.05)).

Effect of frequency of tactile stimulation on sensory-motor perception

The effect of frequency on the sensory-motor cortices was compared between Experiments A and B (Table 2 and Fig. 5). In Experiment A, in which visual information was added to tactile stimulation, with high frequency stimulation significant neural activation was observed in the left motor cortex (chs 1, 3 and 6), right motor cortex (chs 8 and 10) and left somatosensory cortex (ch 2) (Fig. 5A). In Experiment B (only tactile information) with low-frequency stimulation, significant neural activation was observed in the left motor cortex (ch 1), right motor cortex (chs 8 and 13), left somatosensory cortex (ch 2), and right somatosensory cortex (chs 9, 12 and 14) (Fig. 5B). The above results show that brain activation differs depending on the frequency, regardless of the presence or absence of visual information.

Table 2.

Effects of stimulus-frequency on sensory motor task perception.

Blue cells indicate statistically significant channels (Low > High Frequency) compared to the ground-averaged NIR signals. Red cells indicate statistically significant channels (Low < High Frequency) compared to the ground-averaged NIR signals. *, p < 0.05; **, p < 0.01.

Fig. 5.

Effects of stimulus-frequency on sensory-motor task perception. Visualization of Table 2 is shown. The color blue denotes the hinger neural activation with low frequency stimulation (LF) than that with high frequency stimulation (HF), while red denotes the hinger neural activation with HF than that with LF. The size of triangle denotes the significance of the neural activation (i.e., large (**, p < 0.01); small (*, p < 0.05)).

Method validation

In 1995, Ramachandran reported a phenomenon called the phantom limb, in which a body part that was supposed to be lost owing to amputation is perceived as if it still exists [14]. In 1998, Botvinick and Cohen reported that participants, when observing a rubber hand, would feel as if they were being touched at the rubber hand's position when their real hand and rubber hand were stroked or touched synchronously [15]. This phenomenon was later termed the rubber hand illusion. The former involves a tactile sensation generated from a body that no longer exists (e.g., lost hand or leg), while the latter entails a tactile sensation originating from an object (e.g., robber's right hand) distinct from one's own body (e.g., the target's right hand). While these phenomena imply significant interactions between sensory, motor, and visual perception, their detailed mechanisms are still under investigation [[1], [2], [3], [4]]. Moreover, the phenomena called the sensory-motor paradox may have become a long-standing obstacle to investigating human brain function using non-invasive neuroimaging techniques such as fMRI, PET, and EEG [11]. Therefore, in the present study, we aimed to verify neural responses to tactile stimuli using simplified tactile stimulation and a reliable detection system (NIRS) for neuroimaging with sensory-motor stimulation [12].

Characteristics of sensory-motor cortex and the sensory-motor paradox

The sensory-motor paradox among neuroimaging techniques may be related to sensory gain modulation and response inhibition. This complicates understanding the causal role of sensory gain modulation in response inhibition, despite knowledge that incoming information and its effects depend on gain control mechanisms [16], A strong structural anatomical connection exists between the parietal somatosensory cortices (i.e., the primary (SI) and secondary somatosensory cortices (SII)) and the posterior parietal cortex with the motor system [17], as well as prefrontal brain areas [18], potentially involved in response inhibition processes. These feature takes place apparently complex neural responses in the blood oxygenation level (BOLD) of fMRI signal such as BOLD-positive, BOLD-silent and BOLD-negative phenomena in the SI, SII and the super marginal gyrus depending on the activation state in the brain as factors of oxygen metabolic rate, cerebral blood flow rate and cerebral blood volume [12], while NIRS can detect all these phenomena and the brain activation state as a function of oxy-Hb and/or total-Hb [12].

Effects of visual information on the sensory-motor perception

On both left and right sides of the sensory and motor cortices, significantly higher neural activation was induced by low-frequency stimulation without visual information (Experiment B) than with visual information (Experiment A). Brain activation induced by high-frequency stimulation was higher with than without visual information. This result is similar to the rubber hand illusion [1,2], and suggest the presence of visual information intervention for the perception of sensorimotor stimulation. In particular, large neural activation with low-frequency stimulation of the sensory-motor cortices (compared to high-frequency stimulation) suggests that with pure tactile information alone, the perception of the stimulus itself is less certain when receiving low-frequency tactile stimulation. This can be explained by the phenomenon that attention enhances sensory information processing, even under multimodal conditions [19]. This may induce increased attention to the fingertips, resulting in neural activation in the sensorimotor areas, particularly with low-frequency tactile stimulation.

Frequency effects of tactile stimulation on sensory-motor perception

In Experiment A (with visual information), significant neural activation was observed with “high frequency stimulation” compared to that with “low frequency stimulation,” while in Experiment B (with only tactile information) neural activation with “low frequency stimulation” was higher than with “high frequency stimulation.” These results suggest frequency-dependent neural activation in the sensorimotor cortex. Furthermore, the present results suggest that with low-frequency stimulation, participants focused their attention on their fingertips due to the weakness or uncertainty of the stimulus, while with high-frequency stimulation, they recognized the strength of the stimulus from visual information and increased their focus on their fingertips.

Additionally, the observation of the waveforms obtained in this study (Fig. 3), revealed differences in the activation start time and activation peak, and the amount of brain activation determined in this study as well as the method of brain activation showed significant differences. Furthermore, significant neural activation in Experiment A compared to that in Experiment B was observed only in the right motor cortex in the motor condition, which was a control experiment without tactile information (Table 1, Fig. 4). This result is similar to the box-mirror illusion [3,4] and suggests that the illusion of movement of the left hand, which is the dominant area of the right motor cortex, was caused by visual information.

Our study demonstrates that visual information influences tactile perception, and the nature of this influence varies with frequency, as revealed through the use of the non-invasive neuroimaging technique NIRS. NIRS has a lower spatial resolution than fMRI but has a higher temporal resolution and sensitivity to stimulus-induced hemodynamic changes. The application of NIRS to the sensory-motor investigations will progress to the long-standing question of the sensory-motor paradox and neurorehabilitation for sensory paralysis due to the dementia and aftereffects of cerebrovascular disease.

Limitations

Based on the above results, the following four points can be considered as the present limitations and future issues.

• (1)Further investigations of the latency, speed, peak time, and duration of neural activation in addition to brain activation (increase in blood flow),
• (2)Evaluation of the effects of visual stimulation different from the sensorimotor stimulation actually used,
• (3)To verify or compare the present results by testing on visually impaired people and patients with sensory paralysis, and
• (4)To increase the number of participants in order to reach a standardized conclusion

Ethics statements

This study was approved by the Kyoto University Medical Ethics Committee (C1198). All participants were fully informed prior to the experiment, and informed consent was obtained through oral and written consent forms. The work described has been carried out in accordance with The Code of Ethics of the World Medical Association (Declaration of Helsinki) for experiments involving humans.

CRediT authorship contribution statement

Akitoshi Seiyama: Funding acquisition, Investigation, Data curation, Writing – original draft, Writing – review & editing. Tatsuro Miura: Data curation, Formal analysis, Writing – review & editing. Sayaka Okahashi: Methodology, Funding acquisition, Writing – review & editing. Nami Konishi: Writing – review & editing. Monte Cassim: Conceptualization, Funding acquisition, Project administration, Supervision, Writing – review & editing.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Data availability

• Data will be made available on request.