Spatial remapping of touch: Confusion of perceived stimulus order across hand and foot

Abstract

The “body schema,” a spatial representation of the body in its environment, has been suggested to be an emergent property of a widespread network of effector-specific frontal and parietal areas, many of which integrate sensory input from the different modalities. On a behavioral level, such multimodality has been shown with temporal order judgment tasks, in which participants decide which of the two hands received a tactile stimulus first. The accuracy of these judgments is influenced by body posture, indicating that tactile stimuli are not simply represented in an anatomical reference frame, but are transformed into external spatial coordinates. So far, these studies have only investigated the hands. It is therefore unclear whether a default remapping of touch into external space is a special feature of visual–manual control or whether all body parts are represented in a common nonanatomically anchored reference frame. In the present study, blindfolded participants made temporal order judgments of stimuli presented to both hands, both feet, or one hand and one foot. The stimulated limbs were held in either a parallel or a crossed posture. Judgments were equally impaired by limb crossing in all limb combinations (hands only, feet only, hand and foot). These results suggest a remapping of tactile location of all body parts into a nonanatomically anchored and, importantly, common reference frame rather than a specific remapping for eye–hand coordination only.

Researchers are still puzzled about how the brain keeps track of where the different body parts are located at any given moment and how individuals are able to distinguish between their own body and the environment. The complexity and importance of this ability is impressively demonstrated in people whose representation or awareness of their own body does not match its real constitution. For example, amputees frequently attribute sensations to their missing limbs (1, 2). Brain damage can lead to the belief of ownership of more than two arms (3) or legs (4) and to the denial of limb ownership (5). Such phenomena have led to the postulation of a “body schema” (e.g., refs. 6 and 7) as a representation encompassing all of the body parts.

Such a body schema may be important not only for the distinction from, but also for the interaction with, the environment. When we perceive a tactile stimulus, it is important to know exactly where on the body it was located. In many situations, it is even more important to orient the senses toward this location to react as promptly and precisely as possible, for example to prevent injury. In humans, tactile localization has frequently been investigated by using the temporal order judgment (TOJ) task (8–13). This task requires participants to judge which of two stimuli, presented in rapid succession to the left and right hand, appeared first. To succeed, participants must presumably localize the two events in space to be able to indicate which one appeared first (8). The just noticeable difference (JND), defined as the stimulus onset asynchrony (SOA) needed to reliably report the correct order of the two stimuli, is ≈40–60 ms in this task. Interestingly, JNDs double or even triple when the hands are crossed (8–12). This finding is surprising insofar as a tactile TOJ would appear to be possible without taking body posture into account: all that a participant needs to decide is which of the two hands was stimulated first, which would seem to be independent of the location of the hands in space. The TOJ impairment with crossed hands has therefore been interpreted as indicating a conflict between two competing reference frames used by the brain to represent tactile events, one anatomical or somatotopic (i.e., anchored to the body surface) and one external (i.e., independent of the body surface). More specifically, this automatic transformation of tactile stimulus sites into external coordinates seems to be induced by the visual system during development, because congenitally blind humans do not show a crossed-hands effect (10). Intriguingly, a crossing effect also occurs in sighted people when the hands are held behind the back (12), suggesting an influence of the visually induced coordinate transformation even for locations that are never seen. That these experiments indeed tap into something like a body schema is further corroborated by the finding that the TOJ crossing effect also occurs when tips of tools held in the hands are stimulated rather than the hands themselves, and that crossing the tools has the same detrimental effect on TOJs as crossing the arms (14–16). This finding has been interpreted to indicate that the body schema is flexibly adjusted to incorporate manipulated tools as part of the body (16).

However, a unified body schema representation for tactile localization and action planning has so far not been identified in the brain. In contrast, a considerable number of recent studies in monkeys and humans imply that, rather than the body being represented as a whole, different effectors are represented in distinct networks in the parietal and frontal lobes (reviewed in refs. 17 and 18). Each of these areas is selectively connected to a specific frontal site, so that a number of parallel, effector-specific frontal–parietal networks seem to exist (17). Typically, neurons in these different networks receive inputs from several sensory systems, predominately visual (19), somatosensory (19), and vestibular (20, 21), but also auditory (22, 23). In part, these areas contain orderly maps of visual space (24), but they also encode spatial locations in a number of different reference frames (23, 25), making it likely that these cortical areas perform coordinate transformations between different sensory systems (25).

Three areas seem of importance to the current study, namely area PE (also referred to as area 5, see ref. 17), the ventral intraparietal area (VIP), and the medial intraparietal area (MIP). A subset of neurons in area PE, the parietal cortex adjacent to the primary somatosensory cortex, has been found to fire in response to specific complex body postures (26, 27). Moreover, some neurons in this area fire in response to vision of a fake arm, but only when it is positioned such that it could indeed be a part of the own body (28), i.e., neurons do not respond when the fake arm is held upside down or to the wrong shoulder. Furthermore, tool use has been shown to lead to an incorporation of the visual space around the tool into the receptive fields of visual–tactile multisensory neurons in the part of PE that extends into the medial bank of the intraparietal sulcus (PEip) (ref. 29; although see ref. 18). There, PE neighbors a number of areas thought to mediate sensory–motor transformations for specific effectors. The caudal part of PE (PEc) comprises neurons with receptive fields on all body parts (30) and is directly connected with area VIP (17). VIP receives inputs from visual, somatosensory, and auditory areas as well. Here, neurons respond to stimuli on the skin and to stimuli near the hands, arms, torso, and head (31, 32). Together with its target area in the ventral premotor cortex (PMv), it has been suggested to represent peripersonal space and movements within or toward this space, possibly for hand–mouth coordination (31) or self-defense (32, 33). It is noteworthy that this representation for peripersonal space has so far only been shown for upper body-related targets and movements. Moreover, neurons in PMv that respond to multiple modalities (the so-called polysensory zone; ref. 32) have receptive fields mainly at the hands and head.

MIP is active during reaching to targets, pointing, and arm movements (34–36). Other areas in the intraparietal sulcus are thought to mediate sensory–motor transformation for other effectors: the lateral intraparietal area (LIP) represents visual targets and saccades (19, 24, 37); the anterior intraparietal area (AIP) is active during hand grasping of 3D objects (38, 39).

Crossing the bridge from these studies back to those investigating the body schema with postural manipulations like hand crossing in humans, Lloyd et al. (40) showed that changes in arm posture during passive somatosensory stimulation resulted in activation changes in parietal areas which they interpreted to be homologous to VIP and MIP, as well as in the premotor cortex; they suggested that their findings provided evidence for a human homologue of the VIP–PMv circuit, which represents peripersonal space. Furthermore, in the same study, when the stimulated hand was crossed over the midline, activation shifted from the ipsilateral to the contralateral hemisphere, indicating a remapping according to the position of the hand in visual space.

In sum, the body and its coordinated use for action seem to be mediated in segmented, multisensory frontal and parietal brain areas. Consequently, it has been suggested that the body in space is not represented in one homogenous, body-schema-like representation, but that it is represented by a network of interconnected areas, each specialized for a certain body part (18, 41). Accordingly, the body schema could be an emergent property of the interactions of these areas (18). However, studies, behavioral, brain imaging, and neurophysiological alike, have investigated only a few body parts (mostly the hands) and thus do not allow any final conclusions about how relations among different body parts are coded, that is, in which coordinates the body as a whole, if at all, is represented. The present study therefore used a TOJ paradigm with different body parts, namely the feet in addition to the hands, while manipulating limb posture. The existence of one body schema comprising all body parts would be suggested by similar crossing effects between homologous and nonhomologous limbs. Such an effect would furthermore suggest that the effector-specific organization in parietal and frontal areas does not reflect separate action systems but parts of a network from whose interaction the body schema may arise.

Results and Discussion

Blindfolded participants made TOJs of two successive tactile stimuli presented either to the two index fingers, the two halluces (i.e., the big toes), or the index finger of one hand and the hallux of the contralateral foot. Limbs were either in an uncrossed position or crossed over the midline (see Fig. 1 Insets). In the uncrossed postures, each stimulated limb was located in the ipsilateral hemifield. In the crossed postures, each stimulated limb was located in the contralateral hemifield (see Fig. 1 Insets).

Fig. 1.

Response curves for TOJ right first (ordinates) of each participant plotted against stimulus onset asynchronies (abscissae) for the different limb combinations (Insets show body postures and stimulation sites). Negative values denote left-first stimulation. Gray/red small symbols and thin lines represent data from uncrossed/crossed conditions, respectively. Each dot represents the probability of right-first responses based on 12–16 judgments. Large symbols and thick-lined curves represent group average data and a group fit using the flip model as described (8). Participants sat in a chair with an arm rest and foot rest and responded with the effector they judged as having been stimulated first.

TOJ accuracy was assessed by calculating the slope of linear regression lines of probit-transformed response probabilities for all conditions (refs. 9 and 42; see Materials and Methods). Only those SOAs in which participants’ judgments were less than perfect were included. In addition, JNDs were derived from these linear regression lines (13, 42) to allow a direct comparison with other studies.^† Note, that better performance is indicated by steeper, i.e., larger slopes, but lower JNDs, as the two measures are inversely related.

If tactile stimuli are mapped into a common, limb-independent reference frame, a similar increase of errors should occur because of limb crossing in homologous (i.e., hand–hand, foot–foot) as well as nonhomologous (i.e., hand–foot) limb conditions, evident in a similar reduction of slope steepness and a similar increase of JNDs in all tested limb combinations. In contrast, if crossing effects reflect a conflict of coordinate systems specific for visual–manual control or within a hand-specific representation, no crossing effects, i.e., no differences of slopes and JNDs between uncrossed and crossed postures, are expected for any condition involving the feet.

Participants responded more accurately in all uncrossed than crossed limbs conditions, as indicated by steeper regression slopes in all conditions involving uncrossed limbs [see Figs. 1 and 2, ANOVA with factors limbs involved (hands only, feet only, right hand and left foot, left hand and right foot) and posture (uncrossed vs. crossed), main effect of posture, F_(1,9) = 48.87, P < 0.001, corrected according to Greenhouse and Geisser (43)]. Performance significantly differed between limb combinations [main effect of limbs involved, F_(3,27) = 4.52, P = 0.03], and posthoc contrasts revealed that this difference was caused by poorer performance in the feet-only condition, whereas all conditions involving at least one hand did not differ from each other (hands only vs. feet only, P = 0.039; feet only vs. right hand and left foot, P = 0.018; feet only vs. left hand and right foot, P = 0.001; all other contrasts, not significant). Crucially, however, the deficit induced by limb crossing was similar in all four tested conditions [interaction of limbs involved and posture, F_(3,27) = 0.03, P = 0.98, not significant, Fig. 2). An identical ANOVA with the JND as dependent variable revealed the same pattern of results [significant effect of posture, F_(1,8) = 44.64, P < 0.001; significant effect of limbs involved, F_(3,24) = 4.37, P = 0.036; nonsignificant interaction, F_(3,24) = 1.96, P = 0.18; see Table 1].^‡ Furthermore, the JNDs in our hands-only condition (uncrossed, 54 ms; crossed, 112 ms) closely match those reported by Shore et al. (9) (uncrossed, 34 ms; crossed, 134 ms), Röder et al. (10) (uncrossed, 47 ms; crossed, 115 ms), and Kóbor et al. (12) (uncrossed, 43 ms; crossed, 119 ms).

Fig. 2.

Comparison of TOJ accuracy (slopes of linear regression lines) for stimulation of the hands, the feet, and each hand with the contralateral foot. White/black bars represent conditions with uncrossed/crossed limbs, respectively. Error bars represent standard errors of the mean. For each participant and condition, judgment probabilities of the part of the curve in which judgments transgressed from left to right (i.e., the nonasymptotic part of the response curves) were transformed by using probit analysis (9). These values were linearly regressed; a steeper slope indicates that participants require a smaller SOA to respond correctly, and therefore greater accuracy. Linear regression slopes were used for statistical analyses using an ANOVA.

Table 1.

JNDs (75% correct responses) of time intervals for TOJs for all four limb combinations in uncrossed and crossed postures

Limbs	Uncrossed posture		Crossed posture
	Mean, ms	Standard error, ms	Mean, ms	Standard error, ms
Hands	52	8	112	19
Feet	64	9	160	29
Right hand, left foot	50	5	99	12
Left hand, right foot	46	6	78	10

See ‡.

In sum, our results thus show that crossing the limbs of two stimulated effectors similarly affects TOJs for homologous and nonhomologous pairs of effectors and thus indicate a remapping of all tactile stimuli into a nonsomatotopically organized reference system. Although clear-cut, these results are surprising from several points of view. Confusion of the two hands has long been known to be evoked by unusual, e.g., crossed arm postures (44, 45). In contrast, one might expect that the decision if a finger or a toe was stimulated first is reached easily because they are far apart on the body surface, and hands and feet are distinctly represented throughout most of the somatosensory system (46, 47).

We therefore interpret the result that response accuracy was largely independent of the limbs involved, but critically depended on their posture even for nonhomologous limbs, as evidence for a body schema-like representation of tactile location, which, rather than being specific for a set of effectors, seems to encompass the whole body and uses external coordinates. How this spatial representation of body parts other than the hands is implemented in the brain will require further investigation. However, several reasons make the parietal areas outlined in the Introduction plausible candidates. Area PE is sensitive for specific body postures, partly involving several limbs including both fore and hind limbs. It has, therefore, been explicitly suggested to be a first level in the construction of a body schema (27). The very selective responsiveness of this area to the sight of plausibly configured body parts (28) strengthens this view. As in PE, visual and somatosensory inputs converge in several areas of the intraparietal sulcus. Moreover, the representation of hand reach plans in eye-centered coordinates in MIP (34) and the presence of both head- and eye-centered as well as intermediate reference frames in VIP (23, 25) clearly indicate that these brain regions play a role in coordinate transformations between the different sensory modalities. Finally, in the functional MRI experiment by Lloyd et al. (40), crossing the hand across the midline resulted in activation changes in some of these intraparietal and their connected frontal premotor areas.

A remapping of somatotopic coordinates into an external, possibly visual reference frame may subserve an effector-independent representation that is read out during action control (48) or allow selection of the most suitable effector for a given action. Thus, the specialization found in the intraparietal sulcus may be more related to function than to specific effectors. For example, arm-reaching or hand-grasping networks may have been labeled with these names because reaching and grasping are usually executed (and experimentally investigated) using these limbs. However, all of these activities are also possible with other body parts, for example, the feet and mouth. Similarly, tool use may be incorporated by adjusting the “effective” coordinates of a prospectively active limb. Initially, the pattern of neuronal responses during tool use had been interpreted as reflecting an incorporation of the whole tool in the visual space of the neuron (16, 29). More recent research, however, has indicated that only that part of the tool that is effectively used to manipulate the environment (i.e., the part that is now the new effector instead of the hand) seems to be represented in a way similar to the body itself (15, 18). Therefore, coordinates for an action goal may be remapped rather than the body schema proper changed (see also ref. 14). Speculatively, then, parietal and frontal cortex may code an action space that is flexibly adjusted according to what is perceived to be reachable or possible to be acted toward. Thus, while research has focused on the arms and hands as the effectors used to manipulate objects, our data indicate that sensory–motor transformation may initially be independent of the limb later chosen to execute an action. In fact, such effector-independent (and, instead eye-centered) spatial coding has been shown in the intraparietal sulcus for hand reaching and was interpreted to subserve eye–hand coordination (34, 49). Possibly, however, such eye-centered coding also ensures maximal flexibility in the choice of effector while necessitating only one external rather than many limb-centered spatial representations (50). Finally, many actions require a coordination of more than one limb (e.g., eating, fighting, self-defense) and may thus profit from a common spatial representation of the limbs involved. The fact that TOJs were less accurate when only the feet were stimulated may indicate a relatively larger representation of the hands because of its more frequent use and functional complexity, for example, in object manipulation.

In sum, the results of the current study imply the existence of a body schema-like representation, which codes tactile events at all body parts in external coordinates. Such a representation may subserve effector-independent action planning.

Materials and Methods

Participants.

Ten participants (six female, eight right-handed, age 21–39 years, mean 27.7 years) took part in the experiment and received either course credit or monetary compensation. All but one were naïve with respect to the aims of the experiment [nonnaïve participants show a comparable TOJ crossed-hands effect (9)]. All had normal or corrected-to-normal vision and hearing, were right-handed, and reported neither any tactile impairments nor any neurological disorder. The experiment was conducted according to the guidelines laid down in the Declaration of Helsinki (51), and all participants gave their informed consent.

Apparatus and Stimuli.

Participants sat on a chair with back, arm, and foot rests. During uncrossed conditions, the arms lay on the arm rests, ≈55 cm apart, and the legs were stretched out in front of the participant, with the feet ≈40 cm apart. The postures of the different conditions were chosen such as to ensure comfortable seating of the participants and to prevent tiredness of the limbs during the course of the experiment. In the arms-crossed condition, the right arm crossed over the left at the elbows and was supported by a pillow between the arms. In the legs-crossed condition, the right leg crossed over the left at the knees and was supported by a foam block (height 12 cm) under the calf. In the arm-leg-crossed conditions, both legs lay uncrossed on one side of the foot rest, and the arm crossed over the thigh (i.e., both the stimulated arm and leg lay in their contralateral hemispace; see Fig. 1 Insets) and was supported by a foam block. The distance between the hand and the foot in the hand–foot conditions was ≈80 cm. The distance between the two stimulated hands, each in its own hemifield, has been shown to affect JNDs (13); however, JNDs differed by only 10 ms between distances of “directly adjacent” vs. 100 cm apart. In a second study, in which the apparent distance between the hands was manipulated with mirrors (i.e., visually, but not proprioceptively), a difference of 6 ms was found between an apparent distance of 6 cm vs. 40/66 cm, but no performance difference at all was found between distances of 40 and 66 cm (52). This finding suggests that the different distances in our experiment did not confound our results (note that the effects caused by crossing in our study are in the order of 80 ms). In addition, within each effector pairing condition, distance between stimulation sites was kept constant for uncrossed and crossed postures, such that any possible, albeit small, influence of effector distance affected both postures alike and therefore did not influence the crossing effect proper.

Tactile stimuli consisted of a metal rod (diameter: 1.5 mm), which was electronically lifted from its resting position by a relay (lift from resting position: 0.5 mm). Relay and rod were contained in small plastic cubes that were attached to the response devices at the hands and feet (see Fig. 3). The relays were controlled by using the software Presentation (Neurobehavioral Systems, Albany, CA). Stimulation was applied to the distal phalanxes of the index fingers and halluces (i.e., big toes) opposite to the finger/toe nail (i.e., to that part of the finger that is used when exploring by touch, and to the equivalent site on the big toe). Stimulus duration was 10 ms. Stimuli were separately adjusted in intensity for hands and feet such that participants reported them as being equally strong by adjusting the voltage used to drive the stimulators’ relays. Electronic measurements of the relays revealed that this adjustment could result in timing differences of maximally 5.5 ms between the hand and foot stimulators. Note that this difference merely displaces the response curves along the abscissa by this amount of time, but does not affect its shape, and therefore neither of our dependent variables, slope and JND. Furthermore, these differences were identical in the uncrossed and crossed conditions and thus did not at all affect the crossed-limb effect reported here.

Fig. 3.

Stimulation/response devices were attached to the effectors (hands and feet) with elastic bands. The front part of the device was attached with a hinge and could be depressed with the fingers or toes. The front part of the response devices could be adjusted such that the plate just touched the hand or foot at rest. Devices were adjusted such that a minimal movement was required for a response to avoid displacement of the stimulators during the experiment. Stimulators were integrated in these response devices and could flexibly be placed under the distal phalanx or the stimulated finger or toe. Stimulated fingers and toes were kept in place by using an elastic band or medical tape wrapped around the stimulated phalanx and the stimulator. Participants judged the equality of stimulation at hands and feet only after the devices were satisfyingly fit to the effectors.

Procedure and Design.

Stimuli were presented at SOAs of 15, 30, 55, 90, 200, 300, 400, 600, 900, and 1,500 ms, each with both the left and the right stimuli leading. Participants made unspeeded responses with the effector that was stimulated first by depressing the response plate attached to the effector. Depression and release of the response devices were acknowledged with centrally presented tones (duration: 100 ms; pitch: 1,000 and 900 Hz, respectively). The random intertrial interval was 1,200–1,600 ms. Participants were blindfolded, and clicks produced by the tactile stimulators were masked by individually adjusted white noise played through headphones. Participants performed two of four parts of the experiment on each of 2 days. The order of these parts was pseudorandomized across participants. During each part, stimuli were presented to one pair of effectors (both hands, both feet, right hand and left foot, and left hand and right foot) in uncrossed and crossed postures. There were two experimental factors, posture (uncrossed vs. crossed) and limbs involved (hands only, feet only, left hand and right foot, right hand and left foot), resulting in eight conditions. Each condition comprised four blocks with four trials of each SOA presented randomly, resulting in 16 trials of each SOA for each posture and effector pairing. Posture was changed every two blocks, and start posture was balanced over participants.

Data Analysis.

Trials were excluded if the reaction time exceeded 3,000 ms (8) or three standard deviations above the mean reaction time. The percentages of “right first” responses for each SOA and limb condition were converted into z scores for each participant, using a cumulative standard normal distribution (probit analysis; refs. 9, 42, and 53). A regression analysis was performed for each set of converted responses per condition and participant. These regressions were calculated from data points between the shortest negative SOA with a 0 probability of a right first response, and the shortest positive SOA with a 1 probability of a right first response.^§ The slopes of these individual regression equations were analyzed with an ANOVA with the two factors, posture (uncrossed, crossed) and limbs involved (hands only, feet only, right hand and left foot, left hand and right foot). A steeper slope indicates that the transition between right first and left first responses takes place in a smaller time interval and therefore denotes better judgment performance.

JNDs were calculated from regression slopes (52) and analyzed with the same ANOVA as slopes (42). Results from the analyses of the two measures were in good agreement and returned equivalent results. For visualization, response curves fitted according to the flip model of Yamamoto and Kitazawa (8) were calculated for the group data of each experimental condition (Fig. 1). The use of this model was justified by a good fit (χ² test, P > 0.98 in all conditions).

Abbreviations

JND

just noticeable difference

SOA

stimulus onset asynchrony

TOJ

temporal order judgment

VIP

ventral intraparietal area

MIP

medial intraparietal area.