Visual illusion of tool use recalibrates tactile perception

Abstract

Brief use of a tool recalibrates multisensory representations of the user’s body, a phenomenon called tool embodiment. Despite two decades of research, little is known about its boundary conditions. It has been widely argued that embodiment requires active tool use, suggesting a critical role for somatosensory and motor feedback. The present study used a visual illusion to cast doubt on this view. We used a mirror-based setup to induce a visual experience of tool use with an arm that was in fact stationary. Following illusory tool use, tactile perception was recalibrated on this stationary arm, and with equal magnitude as physical use. Recalibration was not found following illusory passive tool holding, and could not be accounted for by sensory conflict or general interhemispheric plasticity. These results suggest visual tool-use signals play a critical role in driving tool embodiment.

1. Introduction

Tool use is a hallmark of the human species and a ubiquitous part of daily life (Vaesen, 2012). From everyday items, like cutlery, to physical augmentation equipment, such as prosthetics, tool use is often accompanied by a sense of “feeling” the world through the tool (Marasco, Kim, Colgate, Peshkin, & Kuiken, 2011; Yamamoto & Kitazawa, 2001). Indeed, the body and tool fuse into a single functional system during tool use (Maravita & Iriki, 2004). This process, known as tool embodiment, aids in seamless and successful interaction with the environment, and involves rapid recalibration of multisensory representations of the user’s body (Cardinali, Brozzoli, Finos, Roy, & Farnè, 2016; Cardinali et al., 2012; Farnè, Iriki, & Làdavas, 2005; Iriki, Tanaka, & Iwamura, 1996; Maravita, Spence, & Driver, 2002; Sposito, Bolognini, Vallar, & Maravita, 2012). For example, brief tool use modulates a multisensory representation of the arm that structures tactile perception (Canzoneri et al., 2013; Cardinali et al., 2009, 2011; Miller, Longo, & Saygin, 2014).

While tool embodiment has been studied extensively over the past two decades (Maravita & Iriki, 2004), little is known about its boundary conditions. The idea that embodiment would be primarily driven by somato-motor feedback during tool use is intuitive and compelling. Indeed, studies have reported that active use of the tool, as opposed to mere passive holding, is necessary for embodiment (Garbarini et al., 2015; Maravita et al., 2002; Witt, Proffitt, & Epstein, 2005; though, see Baccarini et al., 2014). This suggests that a range of specific motor and kinesthetic factors (Wolpert & Ghahramani, 2000)—such as efference copies and somatosensory feedback—may be critical for the process (Brown, Doole, & Malfait, 2011; Rademaker, Wu, Bloem, & Sack, 2014). Indeed, a recent study failed to find evidence for tool-modulated reaching kinematics in a deafferented patient (Cardinali, Brozzoli, Luauté, Roy, & Farnè, 2016). Here, in contrast, we provide evidence that tool embodiment can be purely driven by the visual experience of tool use.

There is a long tradition in the perceptual sciences of using illusions to illuminate the fundamental machinery of perception (Eagleman, 2001); illusory contours (Murray & Herrmann, 2013) and the rubber hand illusion (Botvinick & Cohen, 1998) are classic examples. We take this approach in the present study to explore the boundary conditions of tool embodiment, as well as its underlying multisensory mechanisms. We explored tool use with a variation of the mirror visual illusion (Ramachandran, Rogers-Ramachandran, & Cobb, 1995), which isolates visual feedback of a body part from concomitant proprioceptive and kinesthetic signals. Several studies have found that this illusion has profound effects on body perception (Romano, Bottini, & Maravita, 2013), such as modulating the conscious awareness of phantom limbs (Hunter, Katz, & Davis, 2003; Ramachandran & Rogers-Ramachandran, 1996; Ramachandran et al., 1995), biasing the felt position of the unseen hand (Holmes, Crozier, & Spence, 2004; Holmes & Spence, 2005; Snijders, Holmes, & Spence, 2007), and altering the perception of action space (Creem-Regehr, Payne, Rand, & Hansen, 2014). To foreshadow our results, we found that a visual illusion of tool use recalibrated tactile perception on a stationary arm that appeared to be using the tool during the illusion. This finding has significant implications for our understanding of the multisensory machinery that constructs body perception and its relation to objects in the external world.

2. Experiment 1: Visual illusion of tool use

In Experiment 1, we used the mirror visual illusion to investigate whether participants could embody visual feedback of a limb using a tool, as measured by a recalibration in tactile perception on a stationary arm that did not use the tool. Further, the stationary arm was placed either behind the mirror (Experiment 1a) or down by the hip (Experiment 1b), allowing us to address whether the magnitude of visual-proprioceptive conflict influences the effect.

2.1. Methods and materials

2.1.1. Participants

Twenty-two participants in total took part in Experiment 1; twelve participants took part in Experiment 1a (10 females; 11 right-handed by self-report; Mean age: 22.34, SD: 2.80) and ten participants took part in Experiment 1b (7 females; all right-handed; Mean age: 21.83 SD: 2.71). All participants had normal or corrected-to-normal vision. The experiment was run under the ethical guidelines of the University of California, San Diego, and all participants gave informed consent before participating in the experiment.

2.1.2. Mirror illusion setup

The setup of the mirror illusion occurred following the first (pre-tool use) block of the tactile task (see Tactile Paradigm and Fig. 1a, below). A long mirror (119 cm in length and 41 cm in height) was placed slightly to the left of the mid-sagittal plane of the participant. In Experiment 1a, the participant’s left arm was placed out-of-sight and palm-down behind the mirror, with the elbow resting 10 cm distally from the start of the mirror’s body. The right elbow was initially placed at the location directly opposite the left elbow so that the mirror image accurately reflected the true location of the left arm during rest. In Experiment 1b, participants instead rested their left arm down by the left hip throughout the course of the illusion. This produced a complete dissociation between the mirror image and the proprioceptively specified location of the left arm.

Fig. 1.

Visual illusion and tactile paradigm. (a) Mirror Visual Illusion: A long mirror was placed slightly to the left of the mid-sagittal plane of the participant. The participant’s left arm was placed out-of-sight and hand palm-down behind the mirror (Exp. 1a) or resting next to the participant’s left hip (Exp. 1b). The illusion produced the experience that the left arm was using the tool, despite remaining completely stationary. (b) Tactile distance judgment task: Two tactile points separated by various distances (blue dots) were applied manually to the arm (target surface) and forehead (reference surface). Participants judged which of the two body parts was touched with the farthest distance between the two tactile points. Each participant’s responses, before and after tool use, were fit with a logistic curve. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

2.1.3. Mechanical grabber

The tool used in the experiment was a mechanical grabber that extended the user’s reach by a maximum of 40 cm (Fig. 1a). The grabber’s pincers had a maximal width of approximately 18 cm apart. When an object was grasped within the pincers it was approximately 34 cm from the user’s hand.

2.1.4. Object interaction task and mirror illusion

After the initial mirror box setup (performed immediately after the first block of the tactile task, described below), a tool was placed in the participant’s right hand as it rested on the table. They were instructed to wrap their fingers around the handle of the tool, but not to move it. The location of the tip of the tool was marked with tape on the table and a foam cube (5 × 5 × 5 cm) was placed 8 cm distal to the midpoint of the tape. Participants used the tool to pick up the foam cube. They were explicitly instructed to only focus on the contents of the mirror image and never look directly at their actual right hand as it used the tool. Their head orientation and gaze was monitored throughout the course of the task by the experimenter. During tool use, they initially started with the grabber’s pincers at the most proximal location of the tape. They then used the tool to pick the cube straight up and place it back down in approximately the same location it was in prior to lifting. They then returned the pincers back to the tape before initiating the next action. This produced visual feedback that the participant’s left arm was using a tool when it was in fact completely stationary (Fig. 1a). The object-interaction task was self-paced, and lasted for a total duration of 8 min.

The mirror illusion procedure produced two forms of sensory conflict: visual-proprioceptive conflict, where there was a mismatch between the seen and felt location of the left arm, and visual-kinesthetic conflict, where there was a mismatch between the seen and felt movements of the left arm. The visual-kinesthetic conflict was likely very similar between Exp. 1a and 1b, because the left arm remained stationary throughout the entirety of the illusion. The visual-proprioceptive conflict, however, differed between the two experiments, with a much larger conflict in Exp. 1b than 1a. Thus manipulating the physical location of the left arm allowed us to investigate whether the magnitude of conflict modulated any embodiment effects measured in Experiment 1.

2.1.5. Tactile paradigm

Tactile perception was measured using a tactile distance judgment task (TDJ; Fig. 1b), a standard 2AFC task for measuring the morphology of body representation (Canzoneri et al., 2013; Longo & Haggard, 2011; Miller, Longo, & Saygin, 2016; Miller et al., 2014; Tajadura-Jiménez et al., 2012; Taylor-Clarke, Jacobsen, & Haggard, 2004). On each trial, two tactile points (wooden posts each tapering to a 1 mm flat point) were manually applied to the left forearm (target surface) and the forehead (reference surface). Tactile stimuli were applied along the arm (proximodistally) and across the forehead (eye-to-eye). The participant’s task was to judge which body part was touched with the two tactile points of the largest distance. The administered distances were 4, 6, or 8 cm, and were combined into five distance combinations (arm/head; 4/8, 4/6, 6/6, 6/4, 8/4). Participants rested their arm flat on a table with their fingers splayed during the experiment. Stimuli were applied to each body surface for approximately one second with an inter-stimulus interval of approximately two seconds. After the second stimulus was administered, participants verbally reported which body part they felt was touched with the largest tactile distance. There were no time constraints for how quickly participants gave their responses. The response was entered into a computer by the experimenter and then the next trial began. Each distance combination was applied eight times for a total of 40 trials. The body part stimulated first (arm or forehead) was counterbalanced across trials. This procedure was performed in two blocks, one before and one after tool use. Psychometric curves were fit to each participant’s responses (see Section 2.1.6 below).

2.1.6. Data analysis

2.1.6.1. Embodiment

We used two methods to identify and quantify embodiment. First, we analyzed the total percentage of responses (collapsing across stimulus levels) made by participants that tactile distances on the arm felt larger than on the head (i.e., “arm larger” responses). This is perhaps the most common method for identifying modulations in tactile distance perception (Bassolino, Finisguerra, Canzoneri, Serino, & Pozzo, 2015; Canzoneri et al., 2013; de Vignemont, Ehrsson, & Haggard, 2005; Tajadura-Jiménez et al., 2012; Taylor-Clarke et al., 2004). Second, we fit logistic functions to each participant’s pre- and post-tool use responses using a maximum-likelihood procedure (Wichmann & Hill, 2001). Psychometric functions were constructed by plotting the percentage of trials on which the arm stimulus was judged as larger as a function of stimulus combination. To quantify tactile distance perception pre and post-tool use, we extracted the point of subjective equality (PSE) from each participant’s psychometric curves. Perceptual recalibration was numerically defined as the difference between the pre and post PSEs (Miller et al., 2014; Tajadura-Jimenez, Tsakiris, Marquardt, & Bianchi-Berthouze, 2015), the magnitude of which was assessed using a paired t-test (all p-values Bonferroni corrected).

We used two statistical methods to compare the different experiments (1a and 1b): First, we assessed whether the recalibration found in each experiment statistically differed using an unpaired t-test. Second, we used Bayes Factors to quantify how much more likely any difference between experiments could be explained by the alternative hypothesis and the null hypothesis. This was done using JASP version 0.7.5.5 (JASP Team, 2016). The Cauchy prior width was kept at the default value in JASP (0.707).

Previous studies have found that using a tool decreases the perceived distance between two tactile points on the arm in the proximo-distal orientation (Canzoneri et al., 2013; Miller et al., 2014). We therefore present our findings in units of compression, calculated by subtracting the post from the pre-tool use PSE. Positive values correspond to compression in tactile distance perception, whereas negative values correspond to expansion.

There is considerable debate about the taxonomy of body representations (e.g., body schema, body image, body model, etc.) and how distinct behavioral tasks map onto these theoretical constructs (de Vignemont, 2010; Kammers, Mulder, de Vignemont, & Dijkerman, 2010; Longo, Azañón, & Haggard, 2010). Which of these representations are modulated by tool use is also currently a matter of debate (Cardinali et al., 2011; Miller et al., 2014). We have therefore decided to remain agnostic as to which representation our task might index. We instead interpret our results in a task-focused manner, discussing them in terms of a recalibration in tactile distance perception.

2.1.6.2. Response bias and compliance

Concerns about response bias and participant compliance are of the utmost importance in psychophysics (Swets, 1961) and should be considered when interpreting any “perceptual” effect. Indeed, it has recently been claimed that many of the supposed top-down effects on perception are actually post-perceptual in nature (Firestone, 2013; though, see Proffitt, 2013). It is therefore worth noting that the current TDJ paradigm is robust to effects of response bias for two reasons. First, temporal two-alternative forced choice tasks are less likely to be contaminated by response bias than those using the method of single stimuli (García-Pérez & Alcalá-Quintana, 2013), where compliance is of great concern. Second, varying the magnitude of the reference stimuli changes across trials, as was the case in the present paradigm (i.e., touch on the forehead), increases the odds that an observed effect is perceptual in nature (Morgan, Melmoth, & Solomon, 2013; Patten & Clifford, 2015). Nevertheless, we aimed to provide statistical evidence against the possibility that our results could be explained solely by non-perceptual factors.

We statistically ruled out any major influences of response bias in two ways. First, it is possible that a modulation in PSE is due to a response strategy on trials of high uncertainty. Specifically, after tool use, the majority of participants may have defaulted to responding that distance on the forehead was larger on trials where tactile distance applied to both body parts was equal. This alteration in response strategy would give the appearance that tool use compressed tactile distance perception when it had not. While it seems unlikely that the majority of participants would decide to adopt an identical strategy, it is worth ruling out. We therefore tested whether modulations were specific to the middle stimulus level—as would be predicted from the above claim—or whether they are also present for the two intermediate levels where there was a 20 mm difference between touch on the hand and forehead. We used paired t-tests to compare the proportion of “arm larger” responses pre and post-illusory tool use. To increase our power, we collapsed across Experiments 1a and 1b.

Second, and relatedly, response biases may manifest itself as an interval effect in our data (García-Pérez & Alcalá-Quintana, 2011). If effects of compression were found when the first tactile stimulus was applied to the arm but not when it was applied to the head (or vice versa), this would suggest a major—if not complete—influence of extra-perceptual factors on our results. We therefore analyzed our data by order of presentation, as recommended by García-Pérez and Alcalá-Quintana (2013). We did so with a 2 × 2 repeated measures ANOVA with the factors initial stimulation location (hand, forehead) and tool use (pre-tool use, post-tool use). Because of the low number of trials per stimulus level (8 in total, 4 per order), we restricted our analysis to the total percentage of “arm larger” responses. Again, we collapsed across Experiments 1a and 1b to increase our power.

2.2. Results

If tool-related motor signals (e.g., proprioceptive and kinesthetic feedback, efferent copies, etc.) were necessary for embodiment (Brown et al., 2011; Cardinali, Brozzoli, Luauté, et al., 2016; Rademaker et al., 2014), there should be no modulation of tactile perception on a stationary arm. In contrast to this prediction, visual feedback produced clear tool embodiment on the stationary left forearm. In Experiment 1a, we found a significant change in the percentage of “arm larger” responses pre and post tool-use (Mean: −8.96, SEM: 1.96; t₁₁ = −4.58, p < 0.01, d_z = 1.32, 95% CI [0.52; 2.10]). Analysis and comparison of the PSE from the psychometric curves (Fig. 2a) revealed a significant 20.64% (SEM: 4.30) compression in tactile distance perception on the forearm post-illusion (t₁₁ = 4.80, p < 0.005, d_z = 1.39, 95% CI [0.57; 2.17]). That is, after the visual illusion two tactile points applied to the arm felt closer together, consistent with the reported effects of actual tool use in previous experiments (Canzoneri et al., 2013; Miller et al., 2014).

Fig. 2.

A visual illusion of tool use recalibrated tactile perception. (a) Group-level psychometric curve for Experiment 1a: Tool use led to a significant rightward shift (p < 0.001) in the post-tool use curve (green) relative to the pre-tool use curve (black). This pattern of results was similar for both Experiment 1a and 2 (not shown). (b) Comparison of arm position (Exp. 1a vs. b): As can be seen, there was no effect of arm position—recalibration occurred on the left arm irrespective of whether it was located behind the mirror (Exp. 1a) or resting down by participant’s left hip (Exp. 1b). (c) Comparison of TDJ intervals: There was no influence of task interval on the observed tool-induced perceptual recalibration. We found an equivalent decrease in the percentage of “arm larger” responses when the arm was stimulated first (left group) and the head was stimulated first (right group). (d) Comparison of illusory and natural tool use (Exp. 1 vs. 2): The magnitude of the perceptual recalibration following illusory tool use (Exp. 1; collapsed across 1a and 1b) was statistically similar to actual tool use (Exp. 2). Data are presented as mean ± SEM; n.s. = not significant. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

It is possible that the observed recalibration in Experiment 1a was due to an initial visual-proprioceptive congruence at the start of each action. Experiment 1b ruled out this possibility by having participants hang their left arm down by their left hip, thus ensuring a large visual-proprioceptive incongruence throughout the entirety of the tool use task. Despite this manipulation, we still found a significant change in the percentage of “arm larger” responses following tool use (Mean: −7.25, SEM: 2.09; t₉ = −3.47, p < 0.01, d_z = 1.10, 95% CI [0.28; 1.88]). This modulation was also found in the pre and post tool-use psychometric curves, corresponding to a significant 15.95% (SEM: 4.28) compression in tactile distance perception on the forearm post-illusion (t₉ = 3.73, p < 0.01, d_z = 1.18, 95% CI [0.34; 1.98]). Crucially, difference between the magnitude of recalibration found in Experiment 1a and 1b was not statistically significant (t₂₀ = 0.77, p > 0.5, d = 0.33, 95% CI [−0.52; 1.17]; Fig. 2b), and was BF₀₁ = 2.10 times more likely to be explained by the null hypothesis. Therefore, in order to increase statistical power, we collapse across the recalibration found in both conditions (Mean: 18.51%, SEM: 3.02; t₂₁ = 6.13, p < 0.001, d_z = 1.31, 95% CI [0.73; 1.87]) for all subsequent comparisons with Experiment 1.

We next explored the potential role of response bias in the above results. We first examined the raw responses for the middle three stimulus levels before and after illusory tool use. We found a significant decrease in the percentage of “arm larger” responses after tool use in all three levels (all p < 0.05, corrected), demonstrating that our effect was not solely driven by a response strategy used by participants when two tactile points applied to the arm and head were equidistant. The interval of stimulus presentation also did not have a significant effect on the percentage of “arm larger” responses after tool use (Fig. 2c). Consistent with our initial analyses, we found a significant main effect of tool use (F_1,22 = 33.81, p < 0.001), demonstrating that tool use modulated tactile distance judgments. Crucially, we did not find a significant interaction of initial stimulation location and tool use (F_1,22 = 0.81, p = 0.38; Fig. 2c). The main effect of initial stimulation location was also not significant (F_1,22 = 0.90, p = 0.36). While non-perceptual influences cannot be completely ruled out, these results strongly suggest that the observed shifts in the PSE largely reflect perceptual recalibration. Therefore, for the remainder of the paper, we treat significant modulations in the PSE as an index of a perceptual effect.

3. Experiment 2: Natural tool use

The pattern of perceptual recalibration found in Experiment 1 for illusory tool use is consistent with the pattern found for non-illusory contexts where tactile perception is measured on the tool-using arm (Canzoneri et al., 2013; Miller et al., 2014). However, the magnitude of the effect between illusory and natural tool use may differ, which would suggest that the visual illusion lacks some crucial components for normal embodiment. This was investigated in Experiment 2.

3.1. Methods and materials

3.1.1. Participants

Eleven new participants took part in Experiment 2 (9 females; 11 right-handed; Mean age: 22.38, SD: 2.44).

3.1.2. Object interaction task

The experimental setup and procedure was identical to Experiment 1a with three exceptions: (1) the mirror’s surface was covered with an occluding board; (2) Participants used the tool while directly staring at their right tool-using arm (i.e., no visual illusion); (3) The target body part in the TDJ was the right arm, which used the tool. As in Experiment 1, the object-interaction task was self-paced and lasted a total of 8 min.

3.2. Results

Consistent with previous studies (Canzoneri et al., 2013; Miller et al., 2014), we found a significant recalibration of tactile perception on the tool-using arm. Tool use significantly modulated the percentage of “arm larger” responses (Mean: −5.91, SEM: 1.63; t₁₀ = −3.63, p < 0.01, d_z = 1.09, 95% CI [0.32; 1.84]). Analysis of the PSEs revealed a significant 15.00% (SEM: 3.50) compression in tactile distance perception on the right forearm post-tool use (t₁₀ = 4.28, p < 0.01, d_z = 1.29, 95% CI [0.46; 2.08]). The magnitude of the recalibration found in Experiment 1 was not statistically different from that found in the present experiment (t₃₁ = 0.71, p > 0.5, d = 0.26, 95% CI [−0.47; 0.99]; Figs. 2d and 4) and was BF₀₁ = 2.38 times more likely to be explained by the null hypothesis.

Fig. 4.

Comparison between Experiment 1 and all other experiments. (a) Perceptual recalibration for all experiments: As can be seen, a significant compression in tactile distance perception was only observed in Experiment 1 and 2, which did not differ from each other. The recalibration following illusory tool use was significantly different than found in all control conditions. Data are presented as mean ± SEM. (b) Forest plot of effect sizes for all experiments: The colored square is centered on the estimated Cohen’s d and the error bars correspond to its 95% confidence interval (see the main text for the actual values). The solid gray line marks the zero point. (c) Forest plot of effect sizes for all comparisons with Experiment 1. n.s. = not significant, **p < 0.01, ***p < 0.005. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

4. Experiment 3: Interhemispheric plasticity

Thus far, our results suggest that both a dynamic visual illusion of tool use (Exp. 1) and natural tool use (Exp. 2) can recalibrate tactile perception on a stationary body part. It is possible that the results of Experiment 1 can solely be explained by interhemispheric transfer of somatosensory plasticity (Calford & Tweedale, 1990), independent of vision. To test this possibility, we ran a third experiment with all experimental procedures identical to those in Experiment 2, except that tactile perception was measured on the stationary left arm (as in Experiment 1). Participants directly viewed their right arm using the tool, while keeping their left arm stationary behind an occluding board.

4.1. Methods and materials

4.1.1. Participants

Eleven new participants took part in Experiment 3 (6 females; 11 right-handed; Mean age: 19.79, SD: 1.02).

4.1.2. Object interaction task

The mirror box setup and object interaction procedure were identical to Experiment 2. Like Experiment 1, the target body part in the TDJ was the stationary left arm.

4.2. Results

In the absence of the mirror visual illusion, no perceptual recalibration was detected on the stationary left arm. We found no change in the percentage of “arm larger” responses (Mean: 0.91, SEM: 1.52; t₁₀ = 0.56, p > 0.5, d_z = 0.17, 95% CI [−0.43; 0.76]). The pre and post curves were also not significantly different (Fig. 3a). Analysis of the PSEs failed to reveal any significant recalibration in tactile perception (Mean: +1.90%, SEM: 3.25; t₁₀ = 0.56, p > 0.5, d_z = 0.17, 95% CI [−0.43; 0.76]). The difference between the magnitude of the recalibration found in the present experiment and Experiment 1 was statistically different (Exp. 1 vs. 3: t₃₁ = 3.42, p < 0.01, d = 1.26, 95% CI [0.46; 2.04]; Fig. 4) and was BF₁₀ = 19.03 times more likely to be explained by the alternative hypothesis. This finding rules out general interhemispheric plasticity as a full explanation for the recalibration we found following illusory tool use.

Fig. 3.

Psychometric curves for control experiments (Exp. 3–5). No significant difference between the pre and post-tool use psychometric curves were found for any control experiment: (a) Experiment 3, when tool use occurred in the absence of a mirror visual illusion; (b) Experiment 4, when participants viewed a mirror reflection of their arm holding a tool stationary; (c) Experiment 5, when participants viewed a mirror reflection of their hand picking up an object. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

5. Experiment 4: Passive tool holding

Using a tool (as opposed to passive handling) has been found to be necessary for tool embodiment (Brown et al., 2011; Farnè et al., 2005; Garbarini et al., 2015; Maravita et al., 2002; Witt et al., 2005), underscoring the importance of use-related sensory and motor signals. In Experiment 4, we tested whether this was also the case for illusory tool use. The procedures of this experiment were identical to Experiment 1, except that participants viewed the mirror reflection of their arm as they held a tool completely stationary.

5.1. Methods and materials

5.1.1. Participants

Twelve new participants took part in Experiment 4 (10 females; 12 right-handed; Mean age: 21.82, SD: 1.73).

5.1.2. Passive holding and mirror illusion

The mirror box setup was identical to Experiment 1a. Participants were instructed to wrap their fingers around the handle of the tool. Unlike Experiment 1, they held this position without ever picking up or moving the tool. They were told explicitly that they would never use the tool during the task, but instead were instructed to focus their attention along the tool body and at the location of the object in the mirror. This task lasted for approximately 8 min. As in Experiment 1, the target body part was the left stationary arm.

5.2. Results

In contrast to the dynamic visual illusion (Exp. 1), passively holding a tool was not sufficient to recalibrate tactile perception on the left arm. We found no change in the percentage of “arm larger” responses (Mean: 1.25, SEM: 1.73; t₁₁ = 0.72, p > 0.4, d_z = 0.21, 95% CI [0.37; 0.78]). The pre and post-tool holding curves were also not significantly different (Fig. 3b). Analysis of the PSEs failed to reveal any significant recalibration in tactile perception (Mean: −3.82%, SEM: 3.26; t₁₁ = −1.17, p > 0.5, d_z = −0.37, 95% CI [−0.91; 0.25]). The difference between the magnitude of the recalibration found in the present experiment and Experiment 1 was statistically different (Exp. 1 vs. 4: t₃₂ = 4.26, p < 0.005, d = 1.53, 95% CI [0.72; 2.31]; Fig. 4) and was BF₁₀ = 390.8 times more likely to be explained by the alternative hypothesis. This finding is consistent with previous studies with natural tool use and demonstrates a central role for the illusion of tool use in our effect.

6. Experiment 5: Sensory conflict

The mirror visual illusion produces conflict between visual and somatosensory signals related to body position (i.e., proprioception) and movement (i.e., kinesthesia). Studies have found that visual-somatosensory conflict alters the perception (Bultitude, Juravle, & Spence, 2016; Folegatti, de Vignemont, Pavani, Rossetti, & Farnè, 2009) and neural processing of touch (Cardini & Longo, 2016). It is therefore possible that the recalibration found in Experiment 1 could be explained by illusion-related conflict between visual and somatosensory signals irrespective of tool use. We ruled this possibility out in Experiment 5. The procedures of this experiment were identical to Experiment 1a, except that participants viewed their right arm in the mirror picking up an object directly (i.e., no tool). This manipulation keeps visual signals in conflict with proprioceptive and kinesthetic feedback from the left stationary arm. If the effects observed in Experiment 1 are due to sensory conflict, they should appear in this experiment; if they reflect vision of tool use, no effects should be found here.

6.1. Methods and materials

6.1.1. Participants

Twelve new participants took part in Experiment 5 (8 females; 12 right-handed; Mean age: 20.91, SD: 1.47).

6.1.2. Object interaction task and mirror illusion

The mirror box setup was identical to Experiment 1a. The object interaction procedure was identical to Experiment 1 with the following exception: participants picked up the foam cube with their right hand instead of a tool. As in Experiment 1, the target body part was the left stationary arm.

6.2. Results

In contrast to Experiment 1, where the visual illusion consisted of tool-object interactions, we found no evidence for recalibration on the stationary left arm in the present experiment. We found no change in the percentage of “arm larger” responses (Mean: 1.88, SEM: 1.77; t₁₁ = 1.06, p > 0.3, d_z = 0.31, 95% CI [−0.28; 0.88]). The pre and post curves were also not significantly different (Fig. 3c). Analysis of the PSEs failed to reveal any significant recalibration in tactile perception (Mean: −4.50%, SEM: 4.81; t₁₁ = −0.94, p > 0.5, d_z = −0.27, 95% CI [−0.84; 0.31]). Further, the direction of this non-significant effect went in the opposite direction (i.e., expansion) as Experiments 1 and 2. The difference between the magnitude of the recalibration found in the present experiment and Experiment 1 was statistically different (Exp. 1 vs. 5: t₃₂ = 4.70, p < 0.005, d = 1.69, 95% CI [0.86; 2.59]; Fig. 4) and was BF₁₀ = 132.5 times more likely to be explained by the alternative hypothesis. This finding demonstrates that the results of Experiment 1 cannot be fully explained by a general visual-somatosensory conflict.

We further compared the perceptual recalibration found in all experiments, with the exception of Experiment 2, using a One-way ANOVA. A significant change in the pre and post PSEs was only observed in Experiment 1 (F(3,53) = 10.97, p < 0.001); post hoc Tukey’s HSD tests demonstrated that the recalibration observed in Experiment 1 differed from all control experiments (all ps < 0.01), and none of the control experiments differed from each other (all ps > 0.5). Fig. 4 displays the Cohen’s d effect size (with 95% confidence intervals) for each experiment (Fig. 4a), as well as the comparisons with Experiment 1 (Fig. 4b). Bayes Factors (BF₁₀) for all possible comparisons between experiments can be seen in Table 1.

Table 1.

Bayes Factors for all possible experimental comparisons.

	E1a	E1b	E2	E3	E4	E5
E1a		0.48	0.55	8.93a	76.06c	20.33b
E1b	0.48		0.40	3.69a	24.59b	8.09a
E2	0.55	0.40		4.36a	37.94c	10.13b
E3	8.93a	3.69a	4.36a		0.66	0.58
E4	76.06c	24.59b	37.94c	0.66		0.38
E5	20.33b	8.09a	10.13b	0.58	0.38

^aSubstantial evidence: BF₁₀: 3–10.

^bStrong evidence: BF₁₀: 10–30.

^cVery strong evidence: BF₁₀: 30–100.

7. General discussion

We used a visual illusion to gain insight into a fundamental principle of tool embodiment, namely its boundary conditions. Participants viewed a mirror reflection of their right arm using a tool to pick up an object, giving the visual impression that the left arm was in fact doing so. This manipulation produced a significant recalibration of tactile perception on their stationary left arm; it is unlikely that these results can be explained by non-perceptual variables, such as changes in response strategy or compliance. Control experiments ruled out (as indicated by the high Bayes Factorswe observed) the possibility that this recalibration was solely due to passive tool holding, sensory conflict, or interhemispheric plasticity.

Natural tool use involves rich somatosensory and motor feedback from the body (e.g., the hand that is wielding the tool), as well as visual input about the interaction between tool and object. In contrast, our study used an illusion such that the user had visual feedback of object-directed tool use, but other tool-related sensory cues from the body part “using” the tool (i.e., the left arm) were absent. From both an experiential and neurocomputational viewpoint, the two tool use situations—natural and illusory—are substantially different, with the latter lacking the seemingly crucial direct somato-motor cues (Brown et al., 2011; Cardinali, Brozzoli, Luauté, et al., 2016; Rademaker et al., 2014). Nevertheless, we found the same pattern of tactile perceptual recalibration for both situations. This suggests that natural tool use (with associated somato-motor signals) and illusory tool use (with vision of tool use as the main feedback signal) modulates mechanisms underlying experience-dependent plasticity of body representations in a similar manner.

The close relationship between tools and the body in the brain (Johnson-Frey, 2004) is a result of ontogenetic development (Quallo et al., 2009), exaptation of neural structures evolved to represent the body (Gallivan, McLean, Valyear, & Culham, 2013; Peeters et al., 2009), as well as mechanisms that enable experience-dependent plasticity (Buonomano & Merzenich, 1998). Tool embodiment is known to be dependent upon the active use of the tool (Brown et al., 2011; Farnè et al., 2005; Garbarini et al., 2015; Maravita et al., 2002; Witt et al., 2005) and our work demonstrates for the first time that the same holds for an illusory situation; passively holding the tool does not result in embodiment (Exp. 4). General hand-object interactions, which in the context of the mirror visual illusion (Exp. 5) produced a visual-somatosensory conflict, are also not sufficient to recalibrate tactile perception. Further, tool embodiment is isolated to the body part (e.g., the arm) that sensory feedback specifies is using a tool; embodiment does not transfer across hemispheres to a non-tool using arm that is blocked from view by an occluding board (Exp. 3).

Dynamic sensory signals of tool use are a crucial component of the embodiment process in both natural and illusory contexts. Whether these tool use signals must be accompanied by active wielding, and not merely passive tool movement is unknown. The former would implicate an important role for motor commands and intentions (Wolpert & Ghahramani, 2000), as well the sense of agency (Gallese, 2000). The role that these phenomena play in tool embodiment is only now beginning to be investigated (Brown et al., 2011; Garbarini et al., 2015) and cannot be addressed by the results of present study. Nevertheless, our results do demonstrate that experience-dependent mechanisms of plasticity are highly sensitive to the contingencies of tool-object interactions conveyed via sensory feedback (e.g., visual biological motion signals, Blake & Shiffrar, 2007). Sensory information about these contingencies may in fact be necessary for embodiment (Canzoneri et al., 2013; Cardinali et al., 2012; Serino, Canzoneri, Marzolla, di Pellegrino, & Magosso, 2015).

The results of our experiments establish an essential role for visual feedback of tool use—in the absence of concurrent motor feedback—in tool embodiment. In Experiment 1, we found that visual signals of tool use recalibrated tactile perception on the stationary arm. The magnitude of the embodiment was quantitatively indistinguishable from that for natural tool use (Exp. 2) and is similar to that reported in a previous study (Miller et al., 2014). Furthermore, this effect was not dependent on the actual spatial position of the arm (i.e., behind mirror, or resting by the hip) during the illusion, as shown by the similar results in Experiments 1a and 1b. This is somewhat surprising given that other body illusions, such as the RHI, show spatial constraints (Lloyd, 2007). It is possible that the magnitude of the present mirror visual illusion was initially sensitive to the degree of visual-proprioceptive mismatch, but that the dominance of visual feedback increased over the course of the eight-minute tool use session (Holmes & Spence, 2005). However, it should be noted that there was a numerical trend towards a difference between the embodiment found in Exp. 1a and 1b. This hints at the possibility that tool embodiment is a graded phenomenon, with stronger embodiment when sensory signals are in greater alignment. Given the ambiguous support for the null hypothesis when comparing Experiments 1a and 1b, as indicated by a low Bayes Factor, this possibility cannot be ruled out and should be investigated in future studies.

What role might the illusory visual feedback have played in the observed tool-induced recalibration? One possibility is that vision ‘captured’ the somato-motor signals from the right, tool-using arm. Indeed, visual capture is known to play a role in the rubber hand illusion (Pavani, Spence, & Driver, 2000) and the mirror visual illusion (Holmes et al., 2004). It is therefore possible that vision binds the entirety of sensory feedback into a multisensory representation of tool use, leading to recalibration. Another possibility is that recalibration was driven by the bottom-up activation of sensorimotor representations of the stationary arm by feed-forward visual signals of tool use. A recent study with patients lacking a corpus callosum found that motor learning during the mirror visual illusion transferred from the mirror-reflected body part to its stationary counterpart (Nojima, Oga, Fukuyama, Kawamata, & Mima, 2013). This finding bolsters support for the second hypothesis, as visual capture of motor signals in the hemisphere contralateral to the patient’s moving limb would have been highly unlikely. It is also possible that this visual feedback facilitated implicit motor imagery of tool use, which has been implicated in tool-induced modulations in motor kinematics (Baccarini et al., 2014)—though notably, not modulations in tactile perception. Indeed, both mirror visual feedback and motor imagery are known to activate sensorimotor brain regions (Jeannerod & Decety, 1995; Nojima et al., 2012). Interestingly, however, mirror visual feedback activates somatosensory regions to a significantly greater degree than motor imagery (Diers, Christmann, Koeppe, Ruf, & Flor, 2010). Future research should focus on the role that each of these three interrelated phenomenon play in tool embodiment.

Our experience of the world is inherently multisensory, whether with our body or in the context of tool use. Representations of the body that structure somatosensory perception are calibrated and refined through myriad interactions between different sensory modalities, particularly vision and touch (Longo et al., 2010; Macaluso & Maravita, 2010). In the present study, we found that visual tool use signals can recalibrate tactile perception, an interplay that highlights the multisensory nature of the embodiment process. The multisensory mechanisms underlying this interplay have still yet to be established. We suggest middle and superior temporal areas coding for tool use-related visual information such as biological motion (Beauchamp, Lee, Haxby, & Martin, 2003; Saygin, Wilson, Hagler, Bates, & Sereno, 2004) play a crucial role in tool embodiment. During tool wielding, these temporal areas may transmit visual tool-use signals, via dense bidirectional white matter connections (Baizer, Ungerleider, & Desimone, 1991), to regions of the posterior parietal cortex that contain multisensory representations of the body (Bolognini & Maravita, 2007; Duhamel, Colby, & Goldberg, 1998; Ro, Wallace, Hagedorn, Farne, & Pienkos, 2004). In addition, these parietal regions are also known to play a role in tool embodiment (Iriki et al., 1996; Quallo et al., 2009), and are functionally coupled with the primary somatosensory cortex (Cooke et al., 2014). Signals from these parietal regions during tool use likely underlie the observed recalibration of body representation, perhaps by modulating receptive field properties of SI neurons through changes in intracortical inhibition (Cardini, Longo, & Haggard, 2011), a mechanism that commonly drives experience-dependent plasticity (Buonomano & Merzenich, 1998).

8. Conclusions

In sum, we used illusory tool use to investigate the boundary conditions of embodiment. Our results demonstrate that a visual illusion of tool use recalibrates tactile perception and strongly suggest that visual signals of tool use are a driving signal for embodiment. This finding represents a major shift in how we view the sensory relationship between bodies, tools and the environment, demonstrating that visual experience of our activities continuously shapes our perception of our body’s dimensions.