Abstract
Parietal neuronal populations have been found which respond bimodally to visual and somatosensory input regarding one’s own limbs or even perceived haptic input of a false limb (Graziano et al., 2000). Further, neuronal populations have been observed which respond preferentially to visual stimuli presented in spatial congruence with our hands (Graziano 1999). In this study, we examined event-related potentials (ERPs) elicited by laser dots projected onto or above participants’ index and middle fingers during a sustained-attention task. We hypothesized that visual stimuli projected onto the hand would elicit differences in ERP deflections related to sensory gating and categorization in comparison to when projected close to the hand. Participants responded via a footswitch to rare target flashes of light occurring on or directly above the middle finger of the attended hand. We found enhanced amplitudes of the N1 and P3 deflections when the stimuli fell onto the finger tips as opposed to above them. Furthermore, the N1 for unattended stimuli was less suppressed when the lasers were projected onto the fingers. Behaviorally, participants were less accurate to targets when the lasers fell onto the fingers. We conclude that when the lasers appear to “touch” the participants, they act to automatically draw participants’ attention. Thus visual stimuli projected onto the fingers of the ‘unattended’ hand are harder to filter out, leading to decreases in accuracy during task performance.
Keywords: multisensory, visuo-haptic, body schema, ERPs, attention
Introduction
The development of a body schema, the concept of how one’s body relates to external space, is necessary for interacting with our environment through the planning and executing of actions. The integration of multiple sensory representations of body position is critical for the formation of body schema (Maravita and Iriki, 2004). Various mechanisms for this sensory integration of proprioceptive information exist. Graziano et al. (2000) observed maximal response patterns by neurons of monkeys in parietal area 5 to both the realistic visual position of a fake limb and to somatosensory information regarding the position of their real limb. These findings emphasize the existence of parietal cells which preferentially encode visual spatial location and proprioceptive limb position in a bimodal fashion. Similar bimodal neurons have been reported in the premotor cortex and the putamen (Graziano et al., 1994; Fogassi et al., 1996; Duhamel et al., 1998). Several studies have highlighted the discovery of neurons in the premotor cortex which respond preferentially to visual stimuli congruent with the hands (Graziano, Yap, and Gross 1994; Graziano and Gross, 1997; Graziano 1999). This is intuitive since our hands are the primary adaptive appendage we utilize during haptic exploration and interaction with the world.
In agreement with these neurophysiological findings, Spence et al. (1998) reported a cross-modal facilitation of tactile processing when visual events were presented spatially congruent with the location of tactile stimulation. Làdavas et al. (2000) observed that this effect was stronger when the visual stimulus was presented close to the stimulated hand and was much weaker when presented farther away from the hand. In relation to these findings, when participants are allowed to view their stimulated arm, a visual enhancement of tactile processing is often observed (Kennett et al., 2001; Taylor-Clarke et al., 2002). In a systematic approach to studying this effect, Press et al. (2004) observed that this facilitation of tactile processing by the presence of a visual event only occurred under certain task conditions. When the tactile location discrimination task was easy to perform, then viewing the arm was a disadvantage to the participant’s performance in discriminating between sources of tactile stimulation.
Whether advantageous or disadvantageous, there is a clear neural effect of visual input regarding the touch or even the illusion of touch on one’s own limb. We wished to examine whether, during a demanding sustained attention paradigm, participants would demonstrate differential event-related potential (ERP) effects when “touched” on the tips of their fingers by a laser dot in comparison to the laser being shown directly above their fingers. We hypothesized that a flash of light projected onto the finger would be processed as a visual correlate to a tactile input although no true tactile input was presented. Alternately, a flash of light positioned in the immediate vicinity of the fingers would be processed as a visual-only input, with no tactile correlate inferred. In order to examine effects at discreet underlying processing stages for these two conditions, we recorded participants’ event-related potentials (ERPs) while requiring them to respond via a footswitch to target flashes of light projected onto or above their finger tips.
Methods
Participants
Eighteen right-handed participants (10 male; mean age = 20.9 years) completed the experiment after giving informed consent. The experiment was approved by the North Dakota State University ethics committee.
Stimuli and apparatus
Stimuli were four small, focused laser points projected onto the fingers from an adjustable frame positioned above a black desk at which participants were seated (See Figure 1a). The red lasers had a luminance of 100 cd/m2 (600 nm wavelength) and produced no heat or other physical sensations when they touched the participants. Short bursts of light from the lasers were presented in a fast-paced, random sequence. Standard frequent stimuli were aligned with participants’ index fingers in 100 ms bursts of light. Infrequent ‘deviant’ stimuli occurred 15% of the time and were aligned with participants’ middle fingers. Stimuli were presented with an interstimulus interval (ISI) of 270–540 ms (rectangular distribution). A deviant never occurred twice in succession.
Procedure
Participants were seated and had their electroencephalography (EEG) recorded while they performed the experiment. They were instructed to look at a fixation cross on the desk with their hands stretched out in front of them (left/right 45°) (See Figure 1a). Participants’ hands were always visible. There were two experimental ‘laser’ conditions: during the ‘On’ condition, the laser dots were projected onto participants’ index and middle finger tips. During the ‘Off’ condition, participants slid their hands slightly below the laser projections so the dots appeared approximately 2 cm above their finger tips on strips of white medical tape. Since distance of the visual stimulus to the limb does appear to have critical influence over visuo-somatic processing (Làdavas et al., 2000; Farnè et al., 2000), we attempted to control for distance by placing the light falling above the fingers as close to the fingers as possible while not casting any sort of reflection or halo upon them. Further, participants were encouraged to change their body and hand posture as little as possible when they slid their hands beneath the lasers so as to maintain approximate proprioceptive feedback between conditions while performing the task. Participants’ fingers were also wrapped with medical tape so the reflective surfaces were equal between the conditions.
Participants completed 16 blocks, eight of each hand-condition. Blocks were 3.5 minutes each with a net recording time of 56 minutes in length. Participants were instructed to attend only to their left or right hand with attend-left or attend-right blocks interleaved, and to ignore all stimuli occurring on or above their unattended hand. The participants’ task was to respond to the infrequent deviant (i.e., ‘target’) stimulus occurring in alignment with the attended middle finger by pressing a footswitch. Participants were instructed to respond only to deviant stimuli aligned with the attended hand.
Behavioral Data analysis
The percentage of correct target responses (hits) and mean reaction times (RTs) were calculated for target stimuli for each combination of laser condition (On, Off) and attended location (Left, Right). A response was associated with a preceding target if it occurred within 200–800 ms after its onset. A false alarm was defined as a response to any stimulus that was not a target. Analyses of variance (ANOVAs) were run with responses to the target stimuli as the dependent variable in order to examine potential differences in reaction times and accuracy using the within-subject factors of laser condition (On, Off) and attended location (Left, Right).
ERP recordings
The EEG recordings were taken using an Active Two Biosemi Electric System (http://www.biosemi.com; Biosemi, Amsterdam, the Netherlands) from 162 scalp locations. The EOG was recorded from six electrodes located at the outer canthi and above and beneath each eye. The EEG sampling frequency was 512 Hz with a pass-band from DC to 150 Hz. Impedances were kept below 5 kOhms. Data were processed using BESA 5.1.8 (Brain Electric Source Analysis, Gräfelfing, Germany) and visually inspected for blinks and eye-movements, after which automatic artifact rejection criteria of ± 120 μV were applied from −100 to 600 ms post-stimulus onset. Remaining trials were averaged per condition with a baseline of −100 to 0 ms. For analysis and display purposes, data were filtered with a 0-phase-shift 35 Hz IV-order Bessel low-pass filter with a fall-off of 24 dB/octave.
ERP analysis
ANOVAs were run comparing mean amplitudes within specified time windows centered to the peak deflections of interest and referenced to the 100 ms pre-stimulus baseline. Greenhouse-Geisser corrections were applied for violations of sphericity. Partial Eta squared values are reported as a measure of effect size. For the P1 (115–145 ms) and N1 (130–190 ms) latency ranges, mean ERP amplitudes of standard stimuli were subjected to within-subjects ANOVAs with the factors laser condition (On, Off), attention (Attended, Unattended), stimulus location (Left, Right), and electrode cluster (Left, Right). A P3 is typically only elicited for attended target stimuli. Therefore, we subjected the mean P3 peak amplitudes of attended target stimuli within the latency range of (350–450 ms) to a within-subjects ANOVA with the factors laser condition (On, Off), target location (Left, Right), and electrode cluster (Left, Right). We also analyzed the amplitudes at the onset latency of the P3 due to apparent differences at the initiation of the P3 observed in the grand average. We determined the onset of the P3 deflection by taking the first significant sample as compared to baseline in a series of ten significant consecutive sample points showing directional monotonicity (for a review see: Hansen and Hillyard, 1980). Finally, we observed that as the P3 was returning to baseline after reaching peak amplitude, there was a long-standing positivity that appeared to plateau differentially for the On and Off laser conditions with a return to baseline apparently more imminent for the Off condition. We examined this apparent difference in the longstanding plateau following the peak window of the deflection. Thus for the P3 onset (310–330) and late ongoing phase (450–520 ms), mean ERP amplitudes of attended target stimuli were subjected to within-subjects ANOVAs with the same factors as used for the peak analyses.
In order to determine which electrode sites to analyze, deflection topographies in the grand average were visually inspected and analyses were carried out on electrode sites where deflections were most prominent. For the P1 deflection, amplitudes were measured at parieto-occipital sites in two lateral clusters of eight electrodes each. The left cluster consisted of sites P03 to P07 and down to O1 and O3 in the 10–10 system; the right cluster consisted of sites P04 to P08 and down to O2 and O4. A prominently distributed N1 deflection was observed at extended lateral centro-parietal sites; the negative end of the N1 was analyzed at these sites. The N1 amplitudes were measured in two lateral clusters of 16 electrodes each: the left cluster consisted of sites FC4 to FC6 and down to CP4 and CP6; the right cluster consisted of sites FC3 to FC5 and down to CP3 and CP5. For the P3 deflection, amplitudes were measured at fronto-central sites in two lateral clusters of 12 electrodes each: the left cluster consisted of sites FC1 to FC3 and down to CP1 and CP3; the right cluster consisted of sites FC2 to FC4 and down to CP2 and CP4.
Results
Behavioral Data
We found a main effect of laser condition on accuracy: participants were less accurate for the ‘On’ laser condition (F[1, 17] = 8.36; p = .01, h2 = .33; On: M = 88.08%, SE = 1.64; Off: M = 82.05%, SE = 2.52) (See Figure 1b). There were no significant differences in accuracy for left versus right target locations (F[1, 17] = .371; p = .55). There were no significant differences in RT for target location (F[1, 17] = .036; p = .85) or laser condition (F[1, 17] = .34; p = .57).
ERP Data
P1
There was a main effect of laser condition on the P1 amplitude (F[1, 17] = 6; p = .025, h2 = .26): the Off condition elicited a significantly larger P1 to standard stimuli than the On condition. We also found a main effect of attention (F[1, 17] = 19.17; p = .0004, h2 = .53) demonstrating the classic finding that attended stimuli elicit a larger P1 amplitude than unattended stimuli (Näätänen 1992).
N1
There was a main effect of laser condition on the N1 amplitude (F[1, 17] = 8.23; p = .01, h2 = .33) such that the N1 was larger to standard stimuli for the On in comparison to the Off laser condition (See Figure 2a). There was a main effect of attention: attended stimuli elicited larger amplitudes than unattended stimuli (F[1, 17] = 67.62; p = .0000003, h2 = .79). We demonstrated the classic two-way interaction for stimulus location x cluster (F[1, 17] = 53.56, p = .000001, h2 = .76) with larger amplitudes at contralateral sites (Näätänen and Picton, 1987). Finally, there was a three-way interaction of laser condition x attention x stimulus location (F[1, 17] = 8.48; p = .01, h2 = .33). Follow-up tests (with an adjusted p-value of .0125) revealed that for unattended left standard stimuli, the N1 elicited during the On laser condition was significantly larger than during the Off laser condition (F [1, 17] = 13.33, p = .002, h2 = .44). There were no differences between On and Off laser conditions for unattended right standard stimuli (F [1, 17] = 4.56, p = .048); between On and Off laser conditions for attended left standard stimuli (F[1, 17] = 1.28, P = .273); or between On and Off laser conditions for attended right standard stimuli (F[1, 17] = .5.01; p = .039)Visual inspection of the data indicates that the difference between the attended and unattended stimuli for the On condition was smaller than for the Off condition. Post-hoc tests confirm that this effect was apparently modulated by location, suggesting that the attention effect was smallest for stimuli which appeared on the left hand as compared to on the right.
P3
Peak analysis
We examined the extended peak latency of the P3 in order to clarify peak differences between laser conditions. We found a main effect of laser condition (F[1, 17] = 6.54; p = .02, h2 = .28) with larger amplitudes to targets in the On condition (See Figures 2b, 3b). There was also a main effect of cluster with P3 peaks being larger at left electrode sites than at the right cluster (F[1, 17] = 7.2; p = .02, h2 = .29). We demonstrated a two-way laser condition x location interaction (F[1, 17] = 6.26; p = .023, h2 = .27): apparently stimuli falling on the left hand were enhanced in comparison to all other conditions. Finally, we observed a two-way location x cluster interaction (F[1, 17] = 21.85; p = .0002, h2 = .56) suggesting that while left targets elicited equally enhanced peak amplitudes at left and right electrode sites, right targets elicited enhanced amplitudes at left sites and comparatively suppressed amplitudes at right sites.
Onset analysis
We observed differences in the onset of the P3 between laser conditions. We found a main effect of laser condition (F[1, 17] = 6.31; p = .02, h2 = .27). At the onset of the P3, amplitudes were significantly larger for the On condition then when falling above the fingers (See Figure 3a). There was also a main effect of cluster (F[1, 17] = 45.87; p = .00006, h2 = .62) such that the P3 was larger at left electrode sites. Termination analysis. In order to examine differences between laser conditions during the P3 termination latency, we analyzed the termination for attended target stimuli. There was a main effect of laser condition demonstrating that for attended targets the amplitudes to the On condition were larger than to the Off condition (F[1, 17] = 7.23; p = .01, h2 = .29 (See Figure 3c). There was also a main effect of cluster (F[1, 17] = 8.94; p = .01, h2 = .34) such that the amplitudes were larger at the left electrode cluster as compared to the right.
Discussion
Our findings demonstrate that laser dots projected onto the fingers in order to emulate a ‘touch’ effect were processed differentially than the same stimuli projected directly above the fingers with the body position effectively unchanged. Although the stimuli were visual in nature the participants apparently interpreted the lasers as a corollary to a tactile stimulus. This may in part be due to neuronal populations which respond preferentially to visual input presented in spatial congruence with parts of the body (Graziano, Yap, and Gross 1994; Graziano and Gross, 1997; Graziano 1999; Graziano et al., 2000; Maravita and Iriki, 2004), a mechanism which hypothetically enables the formation of body schema. In the case of our sustained attention task the lasers caused an interference effect when ‘touching’ the hands: participants were less accurate during this condition. Examination of the ERPs elicited in response to the stimuli yielded several crucial findings. At the early deflection latencies, amplitudes were consistently more negative when the stimuli fell on the fingers. These early sensory deflections typically index sensory processing and encoding (Näätänen and Picton 1987; Näätänen 1992). This increased negativity, particularly at the N1 latency range, indicates an overall enhancement of the stimuli when they fell upon the fingers. This finding parallels results of behavioral enhancement of actual tactile stimulation in the presence of congruent visual stimulation or during viewing of the limb (Spence et al., 1998; Kennet et al., 2001; Taylor-Clarke et al., 2002). Along with stimuli projected onto the attended hand, stimuli projected onto the to-be-ignored hand were more enhanced as opposed to when they were projected above the fingers. This resulted in a smaller N1 difference between attended and unattended stimuli for either hand location for the On laser condition.
The overall enhancement of stimuli falling on the fingers is particularly demonstrable at the P3 latency range. The P3 is usually elicited by infrequently presented stimuli in a recurring stimulus train and is indicative of short-term memory maintenance and updating (Picton 1992). The first effect we observed in the P3 range was that of differences in amplitude at the onset latency. The P3 amplitude was significantly larger at the initiation of the P3 when targets fell on the fingers, barely breaching 1 μV for the Off laser condition. This suggests that the P3 was initiating earlier and more robustly for the On condition. The P3 amplitude for these targets continued to be enhanced in comparison to when the stimuli fell above the fingers. There was a significant enhancement of the P3 amplitude to target stimuli falling on the fingers during the peak latency range as well. The larger peak amplitude of the P3 to the attended targets falling on the fingers indicates that there may be a processing advantage for visual stimuli perceived as ‘touching’ the hand. Furthermore, at the late ongoing phase of this deflection after the peak, we observed a robust, long-standing positivity with a difference of approximately 3 μV when the lasers fell on the fingers as opposed to above them. The P3 deflection began its return to baseline much more rapidly for stimuli falling above the fingers and was maintained longer for stimuli falling on the fingers.
Although behavioral facilitation is often observed as a result of the visual enhancement effect (Spence et al., 1998; Kennet et al., 2001; Taylor-Clarke et al., 2002), there are cases where viewing the source of a touch was observed to be disadvantageous, depending on the task conditions (Press et al., 2004). In the current study, participants were less accurate in responding to targets when the lasers fell on the fingers. At the early encoding N1 stage of processing, to-be-ignored stimuli were less suppressed for the On laser condition. This suggests that the stimuli falling on the fingers were difficult to ignore even when supposedly unattended. These stimuli apparently had an exogenous attention effect, drawing attention automatically. This may explain why participants were significantly less accurate in responding to targets when the lasers fell on the fingertips instead of directly above them: they were having difficulty filtering out the ‘unattended’ stimuli and thus had more potential responses to choose from.
Of note, we found that our ERP results were in part modulated by stimulus spatial location. Stimuli falling on the left fingers appeared to elicit a larger N1 than stimuli on the right. Also, unattended stimuli on the left hand were the least suppressed, creating a smaller difference between attended and unattended stimuli during the On condition for this location. In accordance with this, the amplitude of the P3 deflection was larger for left target stimuli falling on the fingers as well. These findings combine to suggest that participants had particular difficulty ignoring stimuli falling on their left hands. The explanation for this may be apparent in the interaction observed at the P3 peak amplitude between stimulus location and electrode cluster. Apparently, left spatial locations were processed bilaterally at left and right electrode sites while right locations were processed primarily at left sites. This is not inconsistent with the literature, which suggests that inter-hemispheric connections from right-to-left hemisphere during spatial location tasks have been observed to be stronger than left-to-right inter-hemispheric connections in these same tasks (for a review see: Stephan, Fink, and Marshall, 2007). This is most likely due to right parietal hemispheric dominance in spatial processing. According to Stephan et al. (2007), right parietal sources are likely to be involved in modulating the strength of cortical connectivity during spatial task performance.
Our findings indicate a two-fold effect when it comes to visuo-haptic processing in peri-personal space. At the N1 latency, light flashes projected onto the hands were difficult to filter out even when supposedly ‘unattended.’ This observation appeared to coincide with decreased accuracy in task performance for the On laser condition. However at the later P3 latency, ERPs to target stimuli were enhanced when they fell on the finger tips as compared to above them. Within the attended channel these visuo-haptic stimuli falling on the fingers may cause some sort of enhancement to target processing due to an earlier P3 onset. However, no behavioral advantage was found as a result of this, possibly due to the decreased attention effect of the On laser condition at the earlier N1 stage. One of this study’s limitations is that the stimuli were not actually visuo-tactile in nature and thus we cannot compare our results to what would happen if participants received actual tactile stimulation to their fingertips. Further testing is necessary to investigate and compare the results of a perceived ‘touch’ on the fingers using a projected laser dot against a true visuo-tactile stimulus.
Conclusion
We have demonstrated that laser dots projected onto the fingers appear to interfere with straightforward unimodal visual feature processing, perhaps due to the appearance that they are ‘touching’ the fingers. In the context of our sustained attention task, the stimuli projected onto the fingers during the unattended condition were more difficult to ignore than the stimuli projected above the finger tips. This may have predicated an interference effect, leading to deficits in response accuracy.
Acknowledgments
This project was funded by the National Center for Research Resources (NCRR), National Institutes of Health (NIH), P20 RR020151.
The authors thank Margaret Baune, Malarie Deslauries, Tyler Kurtz, Gabrielle Schreier, and Cassandra Wahl for technical assistance, preparing and testing participants, and recording EEG.
References
- Duhamel J-R, Colby CL, Goldberg ME. Ventral intraparietal area of the macaque: congruent visual and somatic response properties. J Neurophysiol. 1998;79:126–136. doi: 10.1152/jn.1998.79.1.126. [DOI] [PubMed] [Google Scholar]
- Farnè A, Pavani F, Meneghello F, Làdavas E. Left tactile extinction following visual stimulation of a rubber hand. Brain. 2000;123:2350–2360. doi: 10.1093/brain/123.11.2350. [DOI] [PubMed] [Google Scholar]
- Fogassi L, Gallese V, Fadiga L, Luppino G, Matelli M, Rizzolatti G. Coding of peripersonal space in inferior premotor cortex (area f4) J Neurophysiol. 1996;76:141–157. doi: 10.1152/jn.1996.76.1.141. [DOI] [PubMed] [Google Scholar]
- Graziano M, Cooke DF, Taylor C. Coding the location of the arm by sight. Science. 2000;290:1782–1785. doi: 10.1126/science.290.5497.1782. [DOI] [PubMed] [Google Scholar]
- Graziano M. Where is my arm? The relative role of vision and proprioception in the neuronal representation of limb position. Proceedings of the National Academy of Sciences, USA. 1999;96:10418–10421. doi: 10.1073/pnas.96.18.10418. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Graziano M, Gross C. Visual responses with and without neurons in premotor cortex encode spatial locations independently of eye position. Experimental Brain Research. 1997;118:373–380. doi: 10.1007/s002210050291. [DOI] [PubMed] [Google Scholar]
- Graziano M, Yap G, Gross C. Coding of visual space by premotor neurons. Science. 1994;226:1054–1057. doi: 10.1126/science.7973661. [DOI] [PubMed] [Google Scholar]
- Hansen JC, Hillyard SA. Endogenous brain potentials associated with selective auditory attention. Electroencephalogr Clin Neurophysiol. 1980;49:277–290. doi: 10.1016/0013-4694(80)90222-9. [DOI] [PubMed] [Google Scholar]
- Kennett S, Eimer M, Spence C, Driver J. Tactile-visual links in exogenous spatial attention under different postures: convergent evidence from psychophysics and ERPs. J Cogn Neurosci. 2001;13:462–478. doi: 10.1162/08989290152001899. [DOI] [PubMed] [Google Scholar]
- Làdavas E, Farnè A, Zeloni G, di Pellegrino G. Seeing or not seeing where your hands are. Exp Brain Res. 2000;131:458–467. doi: 10.1007/s002219900264. [DOI] [PubMed] [Google Scholar]
- Maravita A, Iriki A. Tools for the body (schema) TRENDS in cognitive science. 2004;8:79–86. doi: 10.1016/j.tics.2003.12.008. [DOI] [PubMed] [Google Scholar]
- Näätänen R. Attention and brain function. Hillsdale; New Jersey: 1992. [Google Scholar]
- Näätänen R, Picton T. The N1 wave of the human electric and magnetic response to sound: a review and an analysis of the component structure. Psychophysiology. 1987;24:375–425. doi: 10.1111/j.1469-8986.1987.tb00311.x. [DOI] [PubMed] [Google Scholar]
- Picton TW. The P3 wave of the human event-related potential. J Clin Neurophysiol. 1992;9:456–479. doi: 10.1097/00004691-199210000-00002. [DOI] [PubMed] [Google Scholar]
- Press C, Taylor-Clarke M, Kennett PH. Visual enhancement of touch in spatial body representation. Exp Brain Res. 2004;154:238–245. doi: 10.1007/s00221-003-1651-x. [DOI] [PubMed] [Google Scholar]
- Spence C, Nicholls MER, Gillespie N, Driver J. Cross-modal links in exogenous covert spatial orienting between touch, audition, and vision. Perception & Psychophysics. 1998;60:544–557. doi: 10.3758/bf03206045. [DOI] [PubMed] [Google Scholar]
- Stephan KE, Fink GR, Marshall JC. Mechanisms of hemispheric specialization: insights from analyses of connectivity. Neuropsychologia. 2007;45:209–228. doi: 10.1016/j.neuropsychologia.2006.07.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Taylor-Clarke M, Kennett S, Haggard P. Vision modulates somatosensory cortical processing. Current Biology. 2002;12:233–236. doi: 10.1016/s0960-9822(01)00681-9. [DOI] [PubMed] [Google Scholar]