Tool use changes multisensory interactions in seconds: evidence from the crossmodal congruency task

Abstract

Active tool use in human and non-human primates has been claimed to alter the neural representation of multisensory peripersonal space. To date, most studies suggest that a short period of tool use leads to an expansion or elongation of these spatial representations, which lasts several minutes after the last tool use action. However, the possibility that multisensory interactions also change on a much shorter time scale following or preceding individual tool use movements has not yet been investigated. We measured crossmodal (visual-tactile) congruency effects as an index of multisensory integration during two tool use tasks. In the regular tool use task, the participants used one of two tools in a spatiotemporally predictable sequence after every fourth crossmodal congruency trial. In the random tool use task, the required timing and spatial location of the tool use task varied unpredictably. Multisensory integration effects increased as a function of the number of trials since tool use in the regular tool use group, but remained relatively constant in the random tool use group. The spatial distribution of these multisensory effects, however, was unaffected by tool use predictability, with significant spatial interactions found only near the hands and at the tips of the tools. These data suggest that endogenously preparing to use a tool enhances visual-tactile interactions near the tools. Such enhancements are likely due to the increased behavioural relevance of visual stimuli as each tool use action is prepared before execution.

Introduction

Tools act as functional extensions of our body, enabling us to reach or to act upon distant objects, and to perform tasks that would otherwise be impossible or harmful. The intuitive idea that tools extend our body has been developed in recent years into the related idea that tools extend the space immediately surrounding parts of our body (i.e., peripersonal space). Peripersonal space is thought to be represented by the brain in certain neuronal sub-populations within intraparietal and premotor cortices (e.g., Duhamel et al. 1998; Graziano et al. 1997). Cells in these areas may have both somatosensory receptive fields (SRFs) and visual receptive fields (VRFs), responding both to somatosensory stimulation applied directly to a certain body part, and to visual stimuli moving nearby or towards the same body part. Importantly, such neurons are thought to encode visual stimuli in body part-centred reference frames, with the possible function of encoding target locations for goal-directed, defensive, or avoidance movements (e.g., Graziano and Cooke 2006). Over the last ten years, evidence has accumulated in support of the idea that tool use leads to an extension or expansion of this representation of nearby space.

Evidence for the extension of multisensory peripersonal space by tool use has come from single unit recordings in macaque monkeys (Iriki et al. 1996a, 2001), spatial modulations of deficits in human neuropsyhological patients (Ackroyd et al. 2002; Berti and Frassinetti 2000; Bonifazi et al. 2007; Farnè et al. 2005a, b, 2007; Farnè and Làdavas 2000; Maravita et al. 2001, 2002a), and spatial modulations of multisensory and sensorimotor behaviour in healthy participants (Longo and Lourenco 2006; Maravita et al. 2002b). Despite this convergence of evidence onto a single explanation, however, when the hypothesis that tool use extends peripersonal space is pitted directly against an alternative hypothesis that tools act as spatial attentional and motor cues, the attentional/motor cueing account has gained more support than the peripersonal space account (Holmes et al. 2004, 2007; Forti and Humphreys 2004; Humphreys et al. 2004). Holmes and colleagues’ studies have shown, for example, that tools serve to ‘highlight’ certain regions of space relevant to the tool use task in healthy participants. Specifically, they found that multisensory spatial interactions are enhanced at the tips, but not in the middles of tools, which is inconsistent with a literal ‘extension’ of peripersonal space (Holmes et al. 2004, 2007). Similarly, several neuropsychological studies have suggested that tools act as salient sensory, motor, or spatial cues, that serve generally to modify spatial aspects of behaviour in neglect patients, rather than specifically to modify a neural representation of peripersonal space (Forti and Humphreys 2004).

Under the hypothesis that tool use changes the multisensory representation of peripersonal space, tool use is usually interpreted as having a relatively long-lasting effect on such a representation. In studies of crossmodal extinction, for example, the effects of tool use were seen to persist for 5-60 minutes or more (Bonifazi et al. 2007; Farnè and Làdavas 2000; Farnè et al. 2005a, b, 2007; Maravita et al. 2002a). In these neuropsychological studies, a consecutive design (i.e., pretest-exposure-posttest) was used in order to examine these relatively long-term changes in peripersonal space following tool use. By contrast, in other experiments testing the hypothesis that tool use modifies peripersonal space, the tool use and the measurement of peripersonal space were performed concurrently (i.e., the tool use motor act itself was measured, Berti and Frassinetti 2000; Holmes et al. 2007; Longo and Lourenco 2006; Pegna et al. 2001; Yamamoto and Kitazawa 2001; Yamamoto et al. 2005), or in an interleaved fashion (Holmes et al. 2004, 2007; Maravita et al. 2001, 2002b). Despite these differences in experimental design (here termed ’consecutive,’ ’concurrent,’ and ’interleaved’ designs), the interpretation of these different studies has typically been within the same theoretical framework (e.g., Holmes and Spence 2004; Holmes et al. 2004; Làdavas 2002; Làdavas et al. 1998; Maravita and Iriki 2004). However, it is not yet clear whether these different experimental approaches are equally applicable to understanding multisensory peripersonal space, or whether instead the choice of experimental design may affect the particular results obtained. In addition, the effects of tool use on more general aspects of visuospatial attention, movement preparation, and sensorimotor cueing, have not, as yet, been systematically investigated in either healthy participants or in patients with crossmodal extinction. Following the results of Forti and Humphreys (2004), such an investigation is clearly important in order to rule out these more general attentional effects as possible explanations for apparent extensions of peripersonal space.

In the present study, we used the third, ‘interleaved’ design, in order to assess whether tool use changes multisensory interactions on a very short timescale of only a few seconds. In previous studies using the interleaved design, a multisensory discrimination task (the crossmodal congruency task) was interleaved with a tool use task (Holmes et al. 2004, 2007; Maravita et al. 2002b). In these studies, the participants were required to use one or two tools after every four trials of the crossmodal congruency task. The tool use task involved crossing or uncrossing the tools over the midline (Holmes et al. 2007; Maravita et al. 2002b), or reaching out with the tool to push distant target buttons (Holmes et al. 2004). In the first two of these studies (Holmes et al. 2004; Maravita et al. 2002b), the effects of tool use were interpreted in terms of relatively long-lasting changes in multisensory processing (i.e., remaining constant throughout, or building up over the course of the experiment). However, in the most recent study (Holmes et al. 2007), the possibility was raised that the preparation of individual tool use movements could lead to significant modulations of crossmodal congruency effects on a trial-by-trial basis. That is, since a tool use movement was required regularly after every four trials of the discrimination task, the participants may have covertly prepared to make the tool use movements during or towards the end of the four trial sequence. Any such movement preparation may have led to systematic changes in either the magnitude or the spatial distribution of crossmodal congruency effects, thus potentially confounding the measurement of peripersonal space with movement-related shifts in spatial or manual attention (see Eimer et al. 2005, 2006, for studies of response-related modulations of multisensory spatial attention).

In order to assess the possible effects of covert (and uninstructed) tool use movement preparation on multisensory interactions in peripersonal space, we therefore manipulated the temporal and spatial predictability of the required tool use task and analysed crossmodal congruency effects on a trial-by-trial basis. The participants were assigned sequentially to two groups: regular tool use, and random tool use. Both groups performed an identical number of trials of the crossmodal congruency task, and an identical number and type of tool use movements. The only difference between the groups was the temporal and spatial predictability of the upcoming tool use task. In the regular tool use group, tool use was required after every fourth trial, and alternated between the left and right hands. In the random tool use group, the tool use was required, on average, after every fourth trial, and half of the time with the left and half with the right hand and tool, but the sequence was pseudorandomised within each block of trials.

If the effects of tool use on crossmodal congruency effects take a relatively long time to develop (i.e., several minutes) and thereafter remain constant, then no trial-by-trial changes should be expected to emerge in either group (the ‘long-lasting modulation’ hypothesis). If tool use movement preparation affects the magnitude or spatial distribution of crossmodal congruency effects, however, then one would expect to find significant effects of trial order and significant differences between the two groups of participants. More specifically, the ‘general movement preparation’ hypothesis predicts that the magnitude of crossmodal congruency effects will increase as a function of trial order in the regular tool use group (since a movement is required only after every fourth trial), while they should be relatively constant in the random tool use group (since a movement could be required unpredictably after any given trial). The ‘spatially-specific movement preparation’ hypothesis predicts that the overall magnitude of crossmodal congruency effects may not vary with trial order or predictability, but that the spatial distribution of these effects will. One version of this hypothesis predicts, following previous results using a very similar task (Holmes et al. 2004), that congruency effects will increase near the hands and at the tips of the tools (but not in the middles of the tools) as a function of trial order and tool use predictability. Alternatively, if tool use leads to an extension of peripersonal space only when one is intending or expecting to move the tool then this hypothesis predicts that the random tool use group should show a peripersonal space-like distribution of crossmodal congruency effects for all trials. By contrast, the regular tool use group should show maximal peripersonal space-like effects on trials immediately preceding a tool use movement - a significant interaction between the spatial and temporal experimental variables.

Methods

Participants

Sixty participants (56 right handed by self-report, 40 female, mean±SE age 22.9±0.9) years, range 18-40 years) were recruited, and assigned alternately to two groups of 30 participants (regular tool use, random tool use). The participants gave their informed consent to take part in the experiments, which were approved by the local research ethics committee, and conducted in accordance with the Declaration of Helsinki.

Apparatus

The participants sat at a table and held a ‘tool’ (a modified toy golf club) in their left and right hands. Two vibrotactile target stimulators (Oticon bone conducting vibrators) were embedded in the handle of each tool, 6cm apart, in an upper (contacted by the participant’s thumb) and lower (forefinger) location. Visual distractor stimuli were 10mm diameter ultrabright red LEDs, positioned 6cm apart in upper and lower locations immediately adjacent to the vibrators (‘near’), at 28cm along the shafts of the tools from the hands (‘middle’), and 56cm along the shafts at the tips of the tools (‘far’). The visual fixation point was indicated with one of three 5mm red LEDs positioned centrally on the tabletop. One fixation LED was illuminated per block of trials, positioned such that the participant’s line of sight passed approximately through the centre of the array of four distractor LEDs at each distance - the four distractor LEDs that were active within each block of trials were therefore presented at approximately the same retinal eccentricity for all conditions and experiments (though note that visual distractor eccentricity varied with distance, see the Discussion for more details). Visual fixation was monitored by the experimenter using a black and white video camera and monitor, and participants were prompted verbally to fixate when necessary.

Design and Procedure

Each trial consisted of three 50ms pulses each of a visual distractor and a vibrotactile target, with individual pulses separated by 20ms gaps. The visual stimulus onset preceded the vibrotactile stimulus by 30ms (see Shore et al. 2006; Spence et al. 2004a). The four vibrotactile target locations (upper and lower on the left and right sides) were presented 6 times each in a block of trials, paired equally with the four possible visual distractor locations (upper and lower, left and right), giving rise to 96 trials per block, which were presented in a pseudorandomised order (i.e., randomly selected without replacement) determined by the computer software at the start of each block of trials. Visual distractor distance (near, middle, far) was manipulated in blocks, with two blocks of 96 trials presented per distance, giving rise to six blocks of trials per participant. Block order was fully counterbalanced across participants and groups, under the constraints that no block type was repeated until the other two types had been presented at least once, and no two adjacent blocks were of the same type. The participants were instructed to respond as rapidly and as accurately as possible to the elevation (‘upper’ vs. ‘lower’) of each vibrotactile target, by lifting one of two foot pedals, one positioned under the heel and the other under the toes. The foot (left or right) used to respond, and the mapping between upper/lower targets and toe/heel responses was counterbalanced across participants and groups.

In the regular tool use group, the tool use task was required after every four trials, and predictably alternating between left and right in sequence, with the side and tool of the first tool use task counterbalanced across participants. In the random tool use group, the tool use task was required, on average every four trials, and half of the time with the left tool on the left side and half with the right tool on the right side, but determined pseudo-randomly by the computer software at the beginning of each block of trials, under the constraints that 24 tool use trials were presented per block, 12 on either side. The tool use task and target location were identical between the experiments, only the predictability of the tool required and the timing of the tool use task differed between groups.

In the tool use task, one of two 10mm diameter red LEDs positioned next to the tool use targets on the left or right was illuminated as a ‘Go’ signal. The participants had to lift the tool with the hand on the indicated side out of its resting position on the table-top, and to use a screw protruding from the tip of the tool to push a button located 100cm in front of the participant, and 67cm above the tabletop. Successful button-pushing extinguished the LED, and the participant returned the tool to its resting position. The instructions to the participants emphasized the accuracy and skilful control of the tool use task over the speed of performance. Typically, each tool use trial took 6-7s to complete. The total number of tool use trials was 144 across the 6 blocks of experimental trials. On average, therefore, the participants spent a total of ∼15min performing the active tool use task. During the discrimination task (approximately 6min per block, ∼36min in total), the participants had to hold the tools firmly in their hands in order to perceive the vibrotactile stimuli. The position of the tools was monitored automatically by a 5V electrical circuit running through the tools and the resting positions on the table - the vibrotactile discrimination task could not proceed until both tools were correctly positioned in their respective place-holders on the table.

Analysis

The data were pooled across the four vibrator positions as is standard practice for this task (since differences between specific hands or digits were not initially of theoretical interest), and were sorted by pooling according to the spatial relationships between the visual and vibrotactile stimuli, and the number of trials since the previous tool use task. Reaction times (RTs) were accepted if 200ms≤RT≤3,000ms. Trials with RTs outside of these limits accounted for 0.5±0.1% and 1.0±0.6% of RTs in the regular and random groups respectively. Statistical outliers were removed per condition per participant following the Modified Recursive procedure reported by van Selst and Jolicoeur (1994), with an initial criterion of 3.5 standard deviations from the mean. This procedure removed a further 3.0±0.3% and 2.2±0.2% of RTs from the regular and random groups respectively. The median RT and proportion of correct responses was determined for each condition and each participant. Inverse efficiency scores (IE = median RT/proportion correct, Townsend and Ashby, 1983) were calculated in order to attempt to correct for any potential speed-accuracy trade-offs in the data, and to provide a single overall measure of performance for the sake of simplicity. Repeated measures analyses of variance (ANOVA) were performed on the IE scores with the within-participants variables of congruency (congruent vs. incongruent visual-tatile stimuli), side (same-side vs. different-sides visual-tactile stimuli), distance (visual distractors at near vs. middle vs. far positions along the tools), and trial order (first vs. second vs. third vs. fourth trial after tool use). Group (regular vs. random) was entered as a between-participants variable. All p-values are reported after Greenhouse-Geisser corrections, significant effects are reported if p≤.05, post-hoc tests were performed using Fisher’s LSD tests, and all numerical values represent the means ± the between-participants standard errors of the means.

Results

Overall performance in the regular tool use group (M±SE=705±30ms) was not significantly different from performance in the random tool use group (691±31ms, F(1,57)<1, n.s.). Performance on congruent trials (661±22ms), was significantly better than on incongruent trials (736±22ms, F(1,57)=71.77, p<.001), but this congruency effect did not differ between the regular and random tool use groups, F(1,57)=2.94, p=.09. The trend in this interaction was towards larger congruency effects in the random compared to the regular tool use group. In the remainder of the results, only the significant effects involving the variable congruency are reported, since only these crossmodal effects are of theoretical interest here. The term ‘crossmodal congruency effect’ refers to the difference between performance on incongruent versus on congruent trials, with congruency referring to the elevation of the visual distractor with respect to the vibrotactile target.

Spatial effects

The effect of congruency between visual distractors and vibrotactile targets interacted significantly with both the distance and the side of the visual distractors. The significant interaction between visual distractor distance and congruency, F(2,114)=9.87, p<.001, revealed that the effect of congruency was strongest near the hands (mean crossmodal congruency effect of 115ms), weakest in the middles of the tools (44ms), and of intermediate magnitude at the far position (67ms). The crossmodal congruency effects were larger near the hands than at the intermediate position (p<.005), and at the far position (p<.001), while the difference between the intermediate and far positions was not significant. The significant interaction between side and congruency, F(1,57)=9.06, p<.005, revealed larger congruency effects for same-side (mean=98ms) as compared to different-sides stimuli (52ms). Importantly, and replicating previous results from our laboratory (Holmes et al. 2004), the interaction between distance, side, and congruency was significant, F(2,114)=3.68, p<.05. This three-way interaction (see Figure 2) was followed up with post-hoc comparisons between same-side and different-sides conditions for the three visual distractor distances separately. The effect of side was significant at the near (p<.001) and far positions (p<.005), but not at the middle position (p=.15). None of the interactions between the spatial variables and the effect of congruency differed significantly between the two groups, indicating that the predictability of tool use did not significantly affect the spatial distribution of crossmodal congruency effects. The RT and error data replicated the above results - all the significant two-way interactions reported above were also significant for RT and error. The three-way interaction was, however, not significant for the RT or error data when considered singly, but both trends were in the same direction as for the IE data.

Figure 2.

Spatial modulation of crossmodal congruency effects during tool use. Columns show the between-participants mean±SE of data from 60 participants. Filled columns - visual distractor and vibrotactile target presented on the same side (same tool). Open columns - visual distractor and vibrotactile target presented on different sides (different tools). The data illustrate the significant interaction between congruency, distance, and side.

The above results contradict the ‘spatially-specific movement preparation’ hypothesis, according to which spatial differences should have been observed between the two tool use groups.

Temporal effects

The effect of crossmodal congruency was significantly affected by trial order, F(3,171)=5.87, p<.005. Crossmodal congruency effects pooled across the two groups increased on each trial from the first (M±SE=48±16ms), to the second (56±9ms), third (84±13ms), and fourth (107±15ms) trials. Significant post hoc comparisons were obtained for the first versus the third and fourth trials, and for the second versus the fourth trials (all ps<.05). Furthermore, this interaction between trial order and congruency differed significantly between the two groups, F(3,171)=2.72, p<.05 (see Figure 3). In the regular tool use group, crossmodal congruency effects were absent on the first trial following tool use (M±SE=9±23ms), but were then significantly larger for each subsequent trial compared to the first (second=61±13ms, p<.05; third=75±18ms, p<.01; fourth=95±21ms, p<.001; main effect of trial for this group, F(3,87)=4.65, p<.005). In the random tool use group, however, the second trial (M±SE=51±13ms) differed significantly from the fourth (119±21ms, p<.005), but no other significant differences were found (main effect of trial order in this group, F(3,87)=2.83, p<.05; first trial=88±23ms, third trial=93±18ms). A linear contrast was performed with the weights (-3, -1, 1, 3) for the first to fourth trials, respectively, for the regular and random tool use groups combined. This linear contrast was significant for the regular tool use group, F(1,58)=8.75, p<.05, but not for the random tool use group, F(1,58)=2.15, p=.15, thus supporting the claim that a linear increase across trials was present only in the regular tool use group. Neither of the spatial variables (distance or side) interacted significantly with the variable trial order (all F’s<1), and neither of these interactions were modulated significantly by the predictability of tool use. Additional analyses of the RT and error data replicated the increase in congruency effects across trials (RTs, p<.01, errors, p=.07), and this effect differed between the groups, linearly increasing for the regular, and remaining relatively constant for the random group (RTs, p<.05, errors, n.s.).

Figure 3.

Crossmodal congruency effects in the regular and random tool use groups: Trial-by-trial analysis. Data points show the between-participants mean±SE of data from 30 participants. Filled black squares - regular tool use group. Open circles - random tool use group. The data illustrate the significant interaction between congruency, trial, and group.

Two additional analyses were performed following requests from our reviewers. First, the IE data from the regular tool use group were reanalysed to examine any spatial effects of the regular alternation between left and right hand tool use trials - i.e., to assess whether there were any spatially-specific movement preparation effects. The crossmodal congruency trials were resorted according to the number of trials since a tool use trial with the same hand as the vibrotactile target stimulus (i.e., trials with vibrotactile targets on the left hand presented five trials after a left hand tool use trial were pooled with trials in which a right hand target was presented five trials after a right tool use trial, and similarly for all eight trial positions). These data were then analysed with the variables congruency, side, and trial, as above, and the variable order (trials 1-4, after tool use with that hand; trials 5-8, before tool use with that hand). The variable order did not significantly affect the overall magnitude of crossmodal congruency effects (order × congruency, F(1,29)<1, n.s.), nor the change in congruency effects across the four trials, F(3,87)=1.74, p=.17.

Second, we assessed the possibility that longer-term changes in the magnitude or distribution of crossmodal congruency effects may have occurred across the six blocks of trials (i.e., similar to the effects reported by Maravita et al. 2002b). The original dataset was divided into two halves, each containing three blocks of trials (near, middle, and far, in counterbalanced order), and the median RT, percentage errors, and IE scores were analysed with a mixed ANOVA as before, except now including the variable experiment half (first half, second half). There was a significant main effect of experiment half for median RTs, F(1,58)=142.02, p<.001, errors, F(1,58)=6.76, p<.05, and for IE scores, F(1,58)=122.67, p<.001. Performance was worse in the first half of the experiment (RT=690±15ms, error=6.9±0.1%, IE=764±24ms) as compared to the second half (RT=615±15ms, error=5.7±0.1%, IE=672±23ms). Despite this general improvement in performance, there were no significant higher-order interactions involving this variable in any performance measure. Importantly, there were no significant interactions with the variable congruency - no changes in the magnitude or spatial distribution of multisensory interactions across the two halves of the experiment.

The temporal modulations of the crossmodal congruency effects support the ‘general movement preparation’ hypothesis, and contradict both the ‘long-lasting modulation hypothesis’, and the ‘spatially-specific movement preparation hypothesis’.

Following the suggestion of one of our reviewers, we also performed an additional experiment in which twenty-four new naïve participants directed a laser pointer at the distant tool use target (which was replaced with a 5mm diameter laser-detecting diode). One laser pointer was embedded in the detached handle of each tool, and the weight of the handle was increased to equal the weight of the whole tool. Throughout the crossmodal congruency task, the tools, LEDs, and tool handles remained in place on the tabletop, exactly as for the main experiments. Laser use movements were required regularly after every fourth crossmodal congruency trial, but unpredictably on the left or right sides. All other experimental parameters were comparable to these reported above.

To assess the spatial effects reported in the above tool use experiments, we performed planned comparisons between same-side and different-sides stimuli for the near, middle, and far visual distractor locations. The near visual stimuli resulted in a significant difference, F(1,23)=10.6, p<.005, while middle, F(1,23)=3.25, p=.08, and far, F(1,23)=0.54, p=.47 did not. This pattern of significant effects replicates previous findings for a non-tool use task (Holmes et al. 2004, ‘handle-use’ experiment). To assess for any temporal modulations, a linear contrast was performed for the effect of trial order on the magnitude of crossmodal congruency effects. This contrast was significant, F(1,23)=4.68, p<.05, revealing that crossmodal congruency effects increased as a linear function of trial order. This additional experiment reveals two interesting results. First that non-tool use tasks result in significant differences between same-side and different-sides crossmodal congruency effects only near the hands, and not in the middles or at the tips of the tools. Second, these results suggest that the general increase in crossmodal congruency effects as a function of trial order reported above is not specific to tool use, but rather may occur for any target-directed manual motor activity, such as reaching and pointing with a laser to a distant visual target.

Discussion

Temporal modulation of the crossmodal congruency effect

The data reported here show a clear effect of tool use movement preparation (i.e., the interaction between congruency and trial order) on the magnitude of multisensory interactions measured along two hand-held tools. This effect of trial order differed between the regular and the random tool use groups: In the regular tool use group, crossmodal congruency effects increased trial-by-trial, from just 9ms on the first trial after tool use, to 95ms on the trial immediately preceding tool use. Conversely, in the random tool use group, crossmodal congruency effects on the first trial after tool use were nearly ten times higher than on the same trial in the regular tool use group (88ms), while the second to fourth trials in this group showed effects similar to those of the regular tool use group. In the control laser-pointing task, the significant linear effect of trial order supports the conclusion that movement-preparation in general, and not specifically tool-use preparation, was driving the significant temporal modulations of the crossmodal congruency effect that we reported.

These data clearly favour the ‘general movement preparation’ hypothesis over both the ‘long-lasting modulation’ and the ‘spatially-specific movement preparation’ hypotheses. The ‘general movement preparation’ hypothesis predicted that crossmodal congruency effects should increase as a function of trial order for the regular tool use group (and for the laser use group). This hypothesis also predicted that crossmodal congruency effects in the random tool use group would remain high and relatively constant across trial order, since the ‘Go’ cue for the tool use task could have been presented at any time after the preceding tool use trial. Both of these predictions were supported by our data. The ‘long lasting modulation’ hypothesis, which predicted no significant trial-by-trial differences, can therefore be rejected. Furthermore, the absence of any significant differences between the first and second halves of the tool use experiments suggests that the effects of tool use on multisensory peripersonal space were either too rapid, transient, weak, or variable to be detected by our experimental design. This result differs from, but is not necessarily incompatible with, the results of Maravita and colleagues (2002b). They showed that repeatedly crossing two tools in opposite directions over the midline (i.e., a different tool use task to the one used here), led to a significant change in spatial modulations following active tool use, specifically in the last four blocks of trials, as compared to the first four blocks.

The relevant behavioural cue informing the participants to begin the tool and laser use tasks was an additional visual target stimulus (an LED) placed 100cm away from the hands, and 67cm above the table surface (60cm above the tools). The illumination of this LED signalled the required time and target side for the upcoming movement. In the regular tool use and laser use group, this visual target information therefore became increasingly imminent (and relevant) as each trial of the four trial sequence passed. By contrast, in the random tool use group, visual information about the tool use task onset and target was always potentially relevant. This behavioural relevance of the target as a function of time offers a clear explanation for the modulation of crossmodal congruency effects as a function of trial and movement predictability. When participants prepared to use a tool or laser in response to a visual cue and target, visual information was behaviourally more relevant, and this relevance increased the magnitude of crossmodal congruency effects prior to the tool use movement.

Interestingly, this increased behavioural relevance of visual stimuli prior to tool use was spatially non-specific, and did not interact significantly with the spatial aspects of the crossmodal congruency effect (i.e., the representation of peripersonal space, as measured by the crossmodal congruency task, was not affected by tool use predictability or trial order). Furthermore, while the laser use experiment resulted in similar temporal effects as those found in the regular tool use experiment (a linear increase across trial order), very different spatial effects emerged (i.e., no significant effect of side in the middles or at the tips of the tools). We therefore suggest that the spatial and temporal modulations of crossmodal congruency effects induced by tool and laser usage are independent, and may provide behavioural indices for different neural or psychological processes, perhaps including but not necessarily limited to multisensory integration, response preparation, peripersonal space, and spatial attention (e.g., see Shore et al. 2006; Soto-Faraco et al. submitted; Spence et al. 2004b).

Our finding of trial-by-trial changes in the effects of tool use and laser use is similar to the findings of Longo and Lourenco (2006), who showed differential effects of four different length tools on line bisections performed at different distances from the body. In their study, the use of the four tools was pseudorandomly interleaved on a trial-by-trial basis. The fact that tool-length differences were found at all suggests that line bisection performance can be modulated on a very short timescale if not immediately upon using the tool (see also Berti and Frassinetti 2000). There is now a pressing need to account for the potentially different neural mechanisms and behavioural processes underlying these immediate (or trial-by-trial) effects, as compared to the longer-lasting effects of tool use training on multisensory interactions measured up to several minutes after tool use (see, for example, Weiss et al. 2000 for a neuroimaging study of line bisection performance).

Spatial modulation of the crossmodal congruency effect

The second result of interest in the present report concerns the spatial modulations of crossmodal congruency effects during tool use. The significant interaction between the distance and the relative side of the visual distractor stimuli replicates a previous finding (Holmes et al. 2004). However, the spatial distribution of congruency effects was unaffected by the predictability of tool use actions. By contrast, in other studies, when the effects of different tool use tasks were compared, the spatial distribution of congruency effects did change significantly (e.g., Holmes et al. 2004; Maravita et al. 2002b; and see also Farnè et al. 2005a). These results strongly suggest that the short-term and longer-term effects of tool use on multisensory interactions are dissociable, and should therefore be studied in isolation in future research. The spatial distribution of multisensory interactions depends upon the specific type of tool use task, while the absolute magnitude of multisensory interactions appears to depend upon the predictability of individual movements made towards visual targets.

One of the reviewers of this manuscript suggested that the lack of any significant difference between crossmodal congruency effects for the middle and far positions for distractors on the same side as the target (the ‘middle’ and ‘far’ filled columns in Figure 2), suggested that there was no real difference in multisensory integration between these two positions. It is indeed possible that the underlying level of multisensory integration was equal at these two locations during the experiments, and the present data cannot provide a definitive statement supporting or refuting this possibility. However, before accepting this apparent equivalence in multisensory integration, several important methodological issues first need to be considered as discussed below.

Interpretation of the crossmodal congruency effects reported here needs to take into account: 1) Comparing crossmodal congruency effects between same-side/hand and different-sides/hands distractors is the most commonly-reported and perhaps most theoretically meaningful way to analyse data obtained using this task (e.g., Holmes et al. 2004, 2007; Maravita et al. 2002b; Pavani & Castiello 2004; Poliakoff et al., 2006; Shore et al. 2006; Spence et al. 2004a; for reviews, see Holmes & Spence, 2006; Spence et al. 2004b); 2) Comparing congruency effects between different distances along the same tool cannot provide any information about hand-centred or tool-centred multisensory mechanisms, which of necessity require comparisons between stimuli presented on different tools; 3) Comparing congruency effects between different distances in the present experimental design also involves comparing the effects of many other, potentially confounding but at present theoretically uninteresting, factors. The tools were not physically designed for these specific comparisons, and since different visual fixation and distractor positions were run in counterbalanced blocks of experimental trials, the following factors all varied systematically with visual distractor distance: The angular size, separation, and eccentricity of the visual distractors with respect to the participants’ head, hands, and eyes, the visual fixation position and head posture adopted by the participant across different blocks, and the visibility of the participants’ hands across blocks. It is not yet known which of the above factors significantly modulate the magnitude or distribution of crossmodal congruency effects. Importantly, however, all the theoretically-motivated spatial comparisons reported in the present manuscript involved visual stimuli presented at the same distance (or collapsed across distances), but on different sides relative to the tactile targets. For this reason, our principal theoretical conclusions are unaffected by the above-mentioned distance-related possible confounds.

Tool use versus laser use: What is the crucial difference

Why is tool use, but not laser use, sufficient to change the spatial distribution of crossmodal congruency effects for visual stimuli presented in extrapersonal space (i.e., at the tips of the tools)? During the experimental trials of the crossmodal congruency task, the tool use and laser use task were identical in terms of the target and distractor stimuli, the task, and the layout of the workspace. Similarly, the tool use and laser use tasks were very similar in that they both required a series of target-directed hand movements towards a target positioned in extrapersonal space. The most salient difference between the tool use and laser use tasks was that, in the tool use task, there was a mechanical connection between the hand of the participant, the distal tip of the tool, and the tool use target. In the laser use task, this connection was established opto-electronically by the laser beam. Since the spatial and attentional requirements of the tool use and laser use tasks were arguably comparable, these factors alone probably cannot account for the observed multisensory spatial differences between the tasks. (We note, however, that a detailed kinematic and eye-movement study of the two tasks may be necessary to rule out such lower-level explanations for the reported task differences.)

We believe that the most likely explanation for the difference between tool use and laser use (or any other non-tool use task) is therefore that actively holding a recently-used or soon-to-be-used tool is accompanied by a shift of attention in depth to the tips of the tools. Such shifts of multisensory attention to the tips of the tools may result in enhanced same-side versus different-sides congruency effects. Endogenously shifting one’s attention to one side or the other has been shown not to change the magnitude or spatial distribution of crossmodal congruency effects (Spence et al. 2004a). However: 1) The trend in those data was towards a greater difference between same-side and different-sides congruency effects when the side was attended (65ms) versus when unattended (40ms); 2) Only 12 participants were studied in the relevant condition, compared to sixty in the present report; 3) Endogenous shifts of attention have been shown to modulate a unisensory vibrotactile version of the congruency task (Soto-Faraco et al. 2004), and; 4) Shifts of attention in depth have not yet been tested using the crossmodal congruency task. Such a test represents an important future research goal, the result of which may help in understanding tool use-related modulations of crossmodal congruency effects.

Response preparation and the neurophysiology of tool use

The significant increase in the overall magnitude of crossmodal congruency effects immediately prior to individual tool or laser use movements is consistent with the qualitative findings of neurophysiological work on tool use. In the original reports on tool use and peripersonal space, Iriki and his colleagues noted speculatively that “the expansion of the visual RF... appeared to be associated with the monkeys’ immediate intention to use [the tool]” (Iriki et al. 1996a>, p. 2328, see also Maravita 2006, for a discussion of the role of motor factors in peripersonal ‘action space’). Indeed, of the 59 ‘multisensory’ neurons studied in their original report, 32 (56%) were reported to show some extension or expansion of their VRFs, while 27 (44%) were not. Although not reported by Iriki and colleagues, whether the VRFs of the remaining 27 cells remained the same size or contracted is clearly an important detail, since an expansion of VRFs in only 32 cells among a population of 59 does not represent a significant deviation from chance variability, assuming approximately normally-distributed variations in repeated VRF size measurements (binomial test, q=0.5, critical values≥36).

Interestingly, of the 32 cells that showed expanded VRFs, 6 were identified in prior testing as responding only when the monkey made active grasping or reaching movements to retrieve a food pellet with its hand. These neurons could not be activated by passive somatosensory stimulation applied by the experimenter (i.e., these cells did not necessarily have a somatosensory receptive field), unlike the prototypical peripersonal neurons reported, for example, by Graziano and colleagues (Graziano et al. 1997). Unfortunately, the number of the other, passively activated, cells that also responded during active movement was not reported by Iriki and colleagues. The reported number of cells responding during active hand and finger movements was, therefore, probably underestimated in their reports.

While the relationship between active movements and peripersonal space is potentially interesting, it is important to note that in certain neurophysiological studies of peripersonal space, active movement-related responses are seen as serious confounding factors when it is the strictly visual response properties of neurons representing peripersonal space that are of most interest, and electromyographic recordings are required to rule out more simple ‘motor’ explanations (e.g., see the discussion in Graziano et al. 1997). Unfortunately, no such controls were reported by Iriki and colleagues, meaning that the role of simple motor factors remains unclear in their reports, although the neurons were reported not to be active during the tool use movements (Iriki et al. 1996b, 2001; Obaysahi et al. 2000). In future neurophysiological studies of tool use and peripersonal space, distinguishing between motor, somatosensory, and visual responses will clearly be important to rule out such possible confounding factors.

Response preparation, spatial attention, and peripersonal space in humans

The ‘active movement’ and ‘intended movement’ responses in the cell population studied by Iriki and colleagues (Iriki et al. 1996a, 2001; Obayashi et al. 2000), if they also exist in humans, might contribute to the effects we report here. Our results, however, while showing a clear general increase in multisensory interactions with movement preparation, do not show any spatial effects that are consistent with the literal extension of peripersonal space, even with our large study sample of sixty participants. This result strongly suggests either that tool use does not literally extend peripersonal space, or that any extension of peripersonal space is too weak to be measured, or cannot be measured reliably with the crossmodal congruency task in healthy participants using the present tool use task.

It may be easier to measure changes in multisensory peripersonal space in patients suffering from crossmodal extinction, and recent studies have indeed shown modulations consistent with a literal extension of peripersonal space (Bonifazi et al. 2007; Farnè et al. 2007). However, while these studies controlled for the effects of goal-directed movement, and the allocation of spatial attention during the tool use task itself (i.e., by comparison with control conditions in which the participants pointed manually to targets for a comparable length of time, but without using the tools, or with a laser pointer), one could reasonably argue that the specific, long-lasting effects of tool use (i.e., effects that last for several minutes following the end of the tool use activity) are also consistent with long-lasting shifts of attention: 1) Towards the right side of space/hand and/or away from the left side/hand, or; 2) Towards the visual modality and/or away from the tactile modality.

On this last point, a very recent study of crossmodal extinction utilizing computer-controlled visual and tactile stimuli (Costantini et al. 2007) showed that a visual stimulus presented near to the left hand decreased the detection of simultaneously-presented tactile stimuli also applied to the left hand. This result provides clear evidence against the hypothesis that a visual stimulus presented near the hand should increase the detection of tactile stimuli on the same hand, but decrease detection of stimuli on the opposite hand. As such, Costantini and colleagues’ results also provide evidence against the involvement of hand-centred multisensory receptive fields in crossmodal extinction, at least for the contralesional hand. Rather, these recent results suggest that at least one form of crossmodal extinction may most clearly reflect a shift of attention to the visual modality, rather than or as well as a shift to the ipsilesional side, perhaps due to a relative dominance of vision over touch under these circumstances. It will be important for future studies to test the effects of tool use on crossmodal extinction using computer-controlled stimuli, and with visual and tactile stimuli presented on both sides of space.

In all but one study of crossmodal extinction and tool use (Maravita et al. 2002a), the tools were held and the visual stimuli were presented on the right side of space, while the (extinguished) tactile stimuli were always presented on the left. Further research is therefore required to distinguish between these alternative accounts for the enhanced visual processing (or suppressed tactile processing) in near space that follows tool use. Such research may require, for example, that the tip of the tool and the hand that holds it are placed on different sides of visual fixation (e.g., see Holmes et al. 2007; Maravita et al. 2002a). An extension of peripersonal space should result in stronger visual-tactile interactions in a hand-and-tool-centred reference frame, whereas a shift in visuospatial attention might result in strongest interactions in an eye-centred reference frame. Crossing the tip of the tool, but not the hand, may dissociate these reference frames. Additionally, the longer-lasting changes in multisensory interactions following a period of tool use may depend upon the participant holding the tool in the same hand during the post-tool use testing phase, as in the tool use phase itself, though this has not yet been tested empirically.

Different experimental designs in studies of tool use

We believe that it is important to distinguish between different experimental approaches in studying the effects of tool use on multisensory interactions in peripersonal space. Our series of tool use experiments have mostly followed what we term the ‘interleaved’ design of tool use experiments - individual tool use movement trials are interleaved with trials of the task used to measure peripersonal space (i.e., the crossmodal congruency task used in the present manuscript, and Holmes et al. 2004; Maravita et al. 2002b; see also Holmes et al. 2007, for different designs). By contrast, the majority of experiments involving crossmodal extinction patients (Bonifazi et al. 2007; Farnè & Làdavas 2000; Farnè et al. 2005a, b, 2007; Maravita et al. 2002a; see Maravita et al. 2001, and below for a different approach) have followed a ‘consecutive’ design in which a period of tool use (typically ∼5 minutes in duration and involving 50 tool use actions) was followed by a period of measurement of peripersonal space (i.e., the crossmodal extinction testing phase). It may be difficult precisely to compare results between studies with different experimental designs. Such comparisons will be made considerably easier, however, by future experiments which test the effects of tool use on crossmodal congruency effects under a consecutive design, and conversely by experiments testing the effects of tool use on crossmodal extinction within the interleaved design.

Interestingly, in one study of crossmodal extinction and tool use, the phase during which crossmodal extinction was tested was also the active tool use phase itself - i.e., this study followed a different, ‘concurrent’ design (Maravita et al. 2001; see also Berti & Frassinetti 2000; Holmes et al. 2007; Pegna et al. 2001). The tool use task in Maravita and colleagues’ experiment involved actively maintaining the tool in position, and checking the tool location repeatedly during the testing period, but between trials. In several other similar studies, the consecutive design was used, and between-condition differences in tool holding during the testing phase were minimised (e.g., Farnè et al. 2005a, b). The effects of task instructions on how participants and patients orient their attention during the testing phase of tool use studies has yet to be investigated, and represents an important question for future research on tool use.

What role for spatial attention in tool use and peripersonal space

What is the general role of spatial attention in the effects of tool use on multisensory interactions in peripersonal space? In a second, less often cited report, Iriki and his colleagues (Iriki et al. 1996b) showed clear ‘attentional’ responses during a visually-cued movement task from neurons in the same area (Brodmann’s area 5V and portions of area 2) as reported in their studies of tool use. In studies of crossmodal extinction and neglect, tool use, and more simply tool holding, have been shown to modulate various aspects of multisensory and sensorimotor performance in these neuropsychological patients. Neglect and extinction are widely regarded to be disorders of spatial attentional orienting (e.g., Danckert and Ferber 2006; Sarri et al. 2006), so one might reasonably conclude that changes in the manifestations of neglect and extinction following tool use might be due, at least in part, to changes in spatial attentional orienting rather than purely or uniquely to changes in peripersonal space (though see Maravita et al., 2002a, Maravita & Iriki, 2004, for the contrary view).

Furthermore, pictures of tools may automatically capture a person’s attention, resulting in enhanced processing of visual stimuli presented near the tool (Handy et al. 2005). These diverse results suggest that it is not yet clear to what extent the reported multisensory effects of tool use are solely due to neural representations of hand-centred peripersonal space per se, or alternatively are due, at least in part, to shifts or biases in visual or multisensory spatial attention (see Holmes et al. 2007, for five experiments investigating this point in more detail). While ‘peripersonal space’ and ‘spatial attention’ may overlap, both conceptually and neurophysiologically, they may be distinguished according to the relevant motor effector involved and the reference frame in which multisensory stimuli are encoded: Hand movements and hand-centred representations for peripersonal space; Eye movements and gaze-centred representations for spatial attention.

Summary

The above considerations suggest that future experiments on tool use in monkeys, neuropsychological patients, and in healthy participants should aim to distinguish between, and assess the relative contributions of, several plausible alternative, but not mutually exclusive, explanations for enhanced visual-tactile interactions during or following tool use: 1) Relatively long-lasting changes in hand-centred multisensory peripersonal space; 2) Relatively long-lasting shifts of multisensory spatial attention; and 3) Relatively brief effects of tool use movement preparation on multisensory interactions.

The present study contributes to the literature on tool use and multisensory peripersonal space by showing that multisensory interactions change on a trial-by-trial basis, depending upon the predictability of the next movement. We conclude that certain effects of tool use on multisensory interactions near hand-held tools may therefore be more ‘motor’ than has previously been considered (see also Maravita 2006, for a similar perspective), and may include strong attentional and movement-preparation components on top of any proposed modulation of purely ‘sensory’ representations of hand-centred space.

Figure 1.

Schematic account of spatial modulations in the crossmodal congruency effect (CCE). The effects are illustrated for a left hand index finger target (the ‘lightning strike’), however the congruency and spatial relationships apply to all four possible target positions. The dashed ellipses schematically represent the approximate proposed shape and size of the ‘peri-hand’ area. The circles represent a ‘congruent’ visual distractor - presented next to the same digit (the same elevation in the task - either ‘upper’ or ‘lower,’ when the hands are holding the tools). The stars represent incongruent visual distractors. Filled symbols represent targets and distractors on the same tool. Open symbols represent target and distractors on different tools. Performance is, on average, worse in incongruent trials with respect to congruent trials. Most importantly, however, this crossmodal congruency effect is larger when target and distractor stimuli are presented on the same side of space (the same hand) relative to when they are presented on different sides (hands).