Typical action perception and interpretation without motor simulation

Significance

How do we perceive and interpret others’ actions? In recent years, the dominant view on this issue is that efficient perceptual processing of others’ actions cannot be achieved by visual analysis of the movements alone but requires unconscious, covert motor simulation of the observed movements. This idea has developed a large following in many disciplines such as psychology, neuroscience, neurology, psychiatry, and philosophy of mind and has started to influence the study and treatment protocols of patients suffering from conditions affecting the perception and interpretation of action (such as autism, schizophrenia, and various dementias). In this paper, however, we report evidence that efficient perception and interpretation of actions can be achieved without motor simulation.

Abstract

Every day, we interact with people synchronously, immediately understand what they are doing, and easily infer their mental state and the likely outcome of their actions from their kinematics. According to various motor simulation theories of perception, such efficient perceptual processing of others’ actions cannot be achieved by visual analysis of the movements alone but requires a process of motor simulation—an unconscious, covert imitation of the observed movements. According to this hypothesis, individuals incapable of simulating observed movements in their motor system should have difficulty perceiving and interpreting observed actions. Contrary to this prediction, we found across eight sensitive experiments that individuals born with absent or severely shortened upper limbs (upper limb dysplasia), despite some variability, could perceive, anticipate, predict, comprehend, and memorize upper limb actions, which they cannot simulate, as efficiently as typically developed participants. We also found that, like the typically developed participants, the dysplasic participants systematically perceived the position of moving upper limbs slightly ahead of their real position but only when the anticipated position was not biomechanically awkward. Such anticipatory bias and its modulation by implicit knowledge of the body biomechanical constraints were previously considered as indexes of the crucial role of motor simulation in action perception. Our findings undermine this assumption and the theories that place the locus of action perception and comprehension in the motor system and invite a shift in the focus of future research to the question of how the visuo-perceptual system represents and processes observed body movements and actions.

Our social life rests in large part on our capacity to perceive and interpret others’ behavior accurately to anticipate their upcoming actions and adjust our own behavior appropriately. The impairment of this ability in the context of autism, schizophrenia, various types of dementia, or following stroke or head injury has tragic consequences for both patients and society. However, how the human mind and brain supports these abilities remains a major challenge. In this study, we addressed a specific issue regarding the relationship between visual and motor processes in action perception and recognition: Does efficient perception and interpretation of action rely on unconscious, covert motor simulation of the observed movements?

The traditional view on this issue is that efficient perception and interpretation of human movement does not rely on motor simulation but rather on computations occurring in the visuo-perceptual system supported by basic perceptual processes and information extrapolated from perceptual learning (1–5). This view has been challenged by a series of motor simulation theories of perception which, despite their differences, all assume that efficient perceptual processing of others’ actions cannot be achieved by visual analysis of the movements alone but requires unconscious covert imitation—motor simulation—of the observed movements (6–13). In this view, motor simulation of an observed action allows the observer to retrieve knowledge about that action automatically, as if the observer were performing the action herself.

Over the past 20 y, this paradigm-shifting idea—that motor simulation is a core feature of movement perception—has become the dominant neurobiological account of the perception and interpretation of action and has had a great impact in the scientific and medical community (e.g., refs. 14–16). However, to date, the interpretation of the results from neuroimaging, behavioral, neuropsychological, and transcranial magnetic stimulation (TMS) studies that have been cited in support of motor simulation theories of perception remains unsettled (SI Discussion, 1) (17–19).

Here, we investigated the role of motor simulation on the perception and interpretation of action in five individuals who were cognitively and neurologically intact but were born with an extremely rare condition: bilateral upper limb dysplasia, characterized by a congenital absence of upper limbs. The rationale is straightforward. The individuals with dysplasia (IDs) never developed any upper limb motor representations or processes that could be mobilized to execute observed upper limb movements covertly (SI Discussion, 2) (20–22). Thus, the expectation from the motor simulation theories of perception is that the performance of the IDs should differ from that of the control participants in tasks that assess aspects of movement processing requiring motor simulation.

In contrast with that prediction, across eight sensitive tasks (no ceiling or floor effect) assessing different aspects of the perceptual processing of action, we found that individuals born without upper limbs can perceive, anticipate, predict, comprehend, and memorize observed actions with the same accuracy and speed as the controls. Furthermore, in all the tasks, some IDs performed well above the mean of the controls. In addition, and crucially, we also found that the IDs’ perception of upper limb movements was characterized by the same perceptual biases (anticipatory and biomechanical biases) as found in the response profile of control participants—perceptual biases that have been taken as evidence for a role of motor simulation in perception.

This set of findings challenges the central premise of the models of action perception and interpretation centered on motor simulation: They show that the visuo-perceptual system can support efficient perception and interpretation of action unaided by motor simulation.

SI Discussion

1. The Interpretation of the Results Cited in Support of the Motor Simulation Theories of Perception Remains Unsettled.

To the best of our knowledge, there is no conclusive evidence supporting the hypothesis that efficient perceptual processing of others’ actions requires a process of motor simulation. Four main sets of data have been provided in support of the motor simulation theories of perception, but their interpretation remains unclear (17–19, 53). Neuroimaging studies have reported that viewing an action activates parts of the motor system that also are involved in executing that action (54, 55). Behavioral studies have reported that people asked to execute movements do so more slowly or less smoothly when they simultaneously observe different movements executed by someone else (56). These two sets of results show compellingly that the motor system involved in producing an action is also activated when that action is viewed. However, and critically, they do not demonstrate that the involvement of the motor system contributes to perception or interpretation of the action. Within models attributing independent processing levels to action perception and execution, there are two ways by which motor activation is generated from perceived body movements. Either the body movements are recognized as an instance of a given action (i.e., clapping), and then the associated motor representation is retrieved from memory, or body movements are transformed directly into the motor commands that the observer would use to produce the same movements (e.g., ref. 2). In this context, the claim that motor simulation is not necessary for efficient perception and interpretation of action is not in conflict with the finding that observation of an action can influence the production of action (56).

The interpretation of the results of TMS studies that have found that stimulation of the left ventral premotor cortex or of the left inferior frontal gyrus interferes with participants’ performance in tasks involving the recognition of body postures (57) or the processing of visually presented body movements (58, 59) is not easy either. The observation that TMS of the motor system may modulate the processing of observed actions is not evidence that the motor system is required or that the visuo-perceptual system is not sufficient to support the perception and interpretation of action. The claim that motor simulation underlies the perception and interpretation of action is very different from the claim that the state of the motor system may influence how we process actions to solve a task. We see at least two possible mechanisms by which motor processes may contribute to the tasks of assessing action perception and interpretation. First, motor simulation may support some forms of memory of body movements or body postures and therefore may influence performance in tasks requiring some memory without being necessary to the support the perception or interpretation of action. A covert imitation of observed body postures and movements is likely to provide an efficient means of retaining and repeatedly refreshing such information in memory (60). On this assumption, the results of an experiment showing that TMS of the premotor cortex impairs the ability to decide which of two body postures matches a previously seen one is open to alternative interpretations. Such results have been interpreted as support for the hypothesis that the premotor cortex is crucial for the visual discriminations of actions (57). Alternatively, however, the results could indicate that the motor system contributes to maintaining information in active memory. Second, motor representations automatically derived from motion stimuli may provide an additional source of information that can be “read” by the conceptual system or combined with the outcome of conceptual processing to make an optimal decision based on noisy or ambiguous information. It has been observed, for instance, that repetitive TMS of the premotor cortex affects participants’ ability to discriminate point-light displays of human actions embedded in a variable number of similar “noise” dots from “scrambled” versions of the same stimulus (58). In such tasks, a response “action” is produced when the probability that the stimulus is an action exceeds a criterion value. To reach this value, decision processes may be influenced by both information that is essential (e.g., the output of the perceptual stages of processing) and by information that is associated with but ancillary to the task at hand (e.g., the state of the motor system). Therefore, interfering with the state of the motor system can affect participants’ performance.

Finally, it is often argued that more compelling observations in favor of a necessary role for motor simulation can be found in neuropsychological studies that have reported different kinds of difficulties in the perception/comprehension of action in patients suffering from brain damage involving different parts of the motor system in the context of different etiologies such as apraxia, motor-neuron disease, Parkinson’s disease, and cortico-basal degeneration (e.g., refs. 43–45, 61–64). However, in these neuropsychological disorders the brain lesions generally extend outside the motor system, and most patients also present with other cognitive disorders, including conceptual, executive, attentional, and/or visuospatial disorders. Therefore, the impairments reported in these neuropsychological studies cannot be ascribed without ambiguity to motor simulation being hampered. This evidence is all the more inconclusive if we consider that even when trends toward action-processing deficits have been observed in these studies at the group level, they all also reported some patients who, despite equal or more severe motor and/or praxis disorders, did not have any particular difficulty in comprehending actions (43–46, 61–64). In turn, these reports of dissociations between action production and perception have been criticized on the ground that not all parts of the motor system are necessary for motor simulation (11, 19). In addition, the patients in those studies had residual motor abilities that might suffice to support comprehension (e.g., comprehension may be easier than execution and thus may be more resistant to disruption) (19, 50). Thus, patients with a motor production deficit but spared recognition ability may simply be those with damage to components of the motor system that are not involved in motor simulation or whose damage does not sufficiently hamper motor simulation. The experimental approach taken in this study overcomes this ambiguity: The IDs never developed any upper limb motor representations or processes that could be mobilized to execute upper limb movements covertly. The present findings thus constitute clear evidence against theories that assume that motor simulation is necessary for action perception by showing that being completely unable to simulate an observed action motorically does not yield any detectable processing difficulty in perceiving or interpreting actions.

2. The IDs Cannot Covertly Execute the Observed Upper Limb Movements.

Three main arguments support the assumption that a congenital absence of upper limbs prevents IDs from simulating observed upper limb actions to support their perception and interpretation of upper limb actions. First, the existing evidence suggests that the ID’s motor cortex does not contain a representation of the missing limbs (20). Rather, the specific parts of their somatosensory and motor cortices that normally would represent the absent limbs are allocated to the representation of adjacent body parts (21, 22, 65). Beyond this evidence, in any event, it is unclear how a motor representation of hand movement could be formed in individuals who have never had any muscle allowing such movements, and we are not aware of any attempt to describe how and why such a mechanism would operate. Importantly, it has been hypothesized that some aplasic individuals may have at least coarse sensory-motor representations of the absent limbs on the grounds that they felt the presence of “phantoms” of the absent limbs (66, 67). The phantom sensations reported by bilateral upper limb aplasic individuals, however, do not recruit the primary motor cortex (53) and do not show several features that, in the traumatic amputees, are thought to reflect subsisting motor representations such as “forgetting” (68) and rapid “twitch-like” sensations in the phantom following TMS of the motor cortex (20, 69). This evidence suggests that phantoms in dysplasia result from perceptual, not motor, representations of the absent body parts (20, 70). In any case, to avoid this potential source of ambiguity, only IDs who had no experience of phantom limbs were included in the present study.

Second, the existing evidence shows that having upper limb motor representations is not sufficient to simulate an observed action. Motor simulation, in effect, is based not only on motor representations of body parts but also on representations of movements previously executed with these body parts. This property has been proved empirically in studies showing that previous motor experience with observed body movements (i.e., not simply having motor representations of the moving body parts) is critical for motor simulation to occur (39). Because the IDs have no motor representations of movements executed with the upper limbs, it is unclear how they could simulate motorically the observed upper limb actions or movements.

Third, in any event, a motor simulation of observed upper limb movements by the IDs could not support their perception and interpretation of action according to the motor simulation theories of perception. According to this view, motor simulation is indeed necessary but not sufficient to achieve efficient perception of an observed action. The role of motor simulation derives from the fact that it is supposed to allow the retrieving or “triggering” of information about that action gathered through previous motor experience. Motor simulation, for instance, supports the comprehension of action because it allows the retrieving of the goal or intention that the observer had in mind when s/he previously performed similar movements (7–13). Because the IDs obviously have never themselves generated the upper limb actions probed in this study, motor simulation could not be regarded as a possible support for the perception or interpretation of action.

3. The Nature of the Variability in the IDs’ Performance.

There is a marked variability in the IDs’ performance within tasks (e.g., compare ID5 and ID3 in experiments 4 and 5) and across tasks (e.g., compare ID4’s performance in experiments 2 and 3). An important question is whether these discrepant performances reflect the normal/typical variability in performance across typically developed individuals or whether it must be accounted for in terms of the IDs’ type and levels of motor disability (different action capabilities or kinematics, different use of prosthetics, and severity of the dysplasia). To test whether the variance in the IDs’ performance across the tasks was different from the variance in the controls, we ranked the performance (with 1 representing the worst participant and 15 representing the best participant) of the 10 control participants who participated in experiments 1–5 (participants from group 1) and the performance of the five IDs in the same tasks (the performance of ID1 for experiment 4, obtained in the exact same conditions, is taken from ref. 19). For each participant we then computed the variance of his/her rank across the five experiments (six tasks; see Table S5). We found similar variances in both groups [modified t test: t (14) = 0.29, P = 0.77]. Then we compared the variance of the IDs’ and of the controls’ performance in the different experiments. As shown in Table S6, in most experiments (i.e., except in experiment 2) we found either no difference between the variability of performance within the two groups or a smaller variability in the IDs performance (experiment 3). Because the variance of the IDs’ performance in these experiments does not differ from that of the control sample, it probably reflects the normal/typical variability in performance. Moreover, the observed different performances did not correlate with motor performance, or with the use of a prosthetic, or with dysplasia severity, because one of the best IDs (ID4) is totally deprived of upper limbs, has never worn prostheses, and also has a shortened right lower limb. This observation is in line with the numerous studies failing to report any consistent link between brain-damaged patients’ action capabilities and their performance in tasks assessing their perception of action (43, 71). In experiment 2, the variability of the IDs’ performance for the upper limb actions was significantly larger than in the control group. However, the lower performance of three of the IDs cannot be easily accounted for by their being unable to simulate the observed actions motorically, because one of the best IDs (ID4) in this experiment was totally deprived of upper limbs, has never worn prostheses, and also has a shortened right lower limb.

Table S5.

Supplemental results of experiments 1–5

Experiment	ID5	C	C	ID3	C	ID2	C	C	C	C	C	ID4	C	ID1	C
Experiment 1	15	13	9	6.5	9	9	2	3	4	6.5	5	14	1	12	11
Experiment 2	13.5	7.5	11	6	12	1	3.5	7.5	2	15	9.5	13.5	9.5	5	3.5
Experiment 3	9	13.5	15	10	4	6.5	11	3	2	8	12	5	1	6.5	13.5
Experiment 4 (large boxes)	12	13	14	7	5	1	8	2.5	4	6	10	11	9	15	2.5
Experiment 4 (small boxes)	15	13	14	5	3	7.5	2	12	7.5	10	1	4	11	9	6
Experiment 5	14	13	9	1	6	7	5	4	12	3	10.5	10.5	8	2	15
Rank variance	5.2	5.3	7.2	8.6	11.5	12.0	13.0	14.0	15.0	16.8	17.3	18.0	19.6	22.6	28.1

Rank-transformed performance of the five IDs (IDs 1–5) and of the participants from group 1 (C) in experiments 1, 2, 3, 4, and 5 (1 = worst participant) and rank variance. Participants are ordered by rank variance size.

Table S6.

Supplemental results of experiments 1–8: Control participants’ and IDs’ variance in performance

Experiment	Controls’ variance	IDs’ variance	Levene’s statistics
			W	P value
Experiment 1	0.53	0.39	0.07	0.79
Experiment 2
Upper limb	168	757	4.19	0.05
Lower limb	137	12	2.13	0.14
Experiment 3	0.37	0.06	4.32	0.05
Experiment 4
Small boxes	0.15	0.20	<0.01	0.98
Large boxes	0.20	0.23	0.01	0.91
Experiment 5	0.19	0.46	1.50	0.23
Experiment 6
Easy condition (forward bias)	181	81	0.73	0.40
Awkward condition (forward bias)	372	212	0.16	0.69
Experiment 7
Easy condition (forward bias)	98	121	0.05	0.82
Awkward condition (forward bias)	97	271	0.41	0.53
Experiment 8
Easy condition (forward bias)	103	500	2.13	0.16
Awkward condition (forward bias)	132	356	0.63	0.43

4. Can the IDs’ Results Be Explained by Their Simulation of the Movements of the Other Body Parts Involved in the Actions?

Because the IDs have a normal body (except the congenitally severely shortened or absent upper limbs), they can simulate the movements and postures of the actor’s face, body, and lower limbs. We do not think that such partial simulation could account for the IDs’ performance. First, most of the experiments in this study used stimuli that either included movements of only the upper limbs (experiments 3, the small-box condition of experiment 4, and experiments 7 and 8) or were selected to make sure that processing the upper limb movements was necessary for the action to be identified (experiment 2). In experiment 1, although the majority of the stimuli (n = 16) involved only movements of the upper limb(s) (the face was neutral, and the body did not move), four other stimuli (making a basketball free throw, playing golf, throwing a ball, shooting a bow and arrow) involved coarse movements of the actress’ body accompanying the natural movement of the hands. It is unlikely that a simulation of only these body movements can support fast and accurate recognition of upper limb actions. Accordingly the results remained unchanged when these four stimuli were discarded [all IDs’ modified t tests, ts (26) < −0.75, all Ps > 0.45]. In experiment 6, the actor’s shoulder moved slightly, but the use of computer-generated stimuli in experiments 7 and 8 ensures that the only part of the body moving is the actor’s right upper limb (the body, shoulder, and face remained strictly still), ruling out the possibility that this effect may be caused by the IDs simulating some aspects of the body movement. In two experiments—the large-box condition of experiment 4 and experiment 5—movements of the body were present and informative. In experiment 4, the actor bends his legs and body to lift the large box, and the speed of the body movement is clearly informative about whether the actor has been deceived about the weight of the box. In experiment 5, the actor’s whole body is moving, and the movements of both the lower and upper limbs contain information relevant for discriminating between the successful and unsuccessful free shots. However, the hypothesis that the IDs could achieve efficient performance in these two tasks by simulating only these movements confronts two problematic assumptions. The first is that the information about the movements of the upper limbs is not necessary to achieve a high level of performance in these tasks. The second is that motor simulation of the movements of the lower limbs and body can support the IDs’ perception. According to motor simulation theories of perception, the role of motor simulation derives from its allowing the retrieving or triggering of information about that action gathered through previous motor experience (6–13). Because the IDs never have lifted a large box with a similar movement of the body and never have thrown a basketball with their hands or with a similar movement of the body, it is unclear how or why covertly imitating these observed lower limb and body movements could support their efficient perception.

Results

The most frequent functional role attributed to motor simulation is that it underlies efficient (i.e., fast and accurate) comprehension of actions (6–12). Experiment 1 tested this hypothesis. Participants viewed video clips of an actress pantomiming 20 different instrumental actions (e.g., playing a guitar, typing). Only body movements were shown, without any object or context. Participants were asked to name each action at 14 165-ms steps in a gradual unmasking paradigm from 330 ms to 2,640 ms. An item was scored correct at a given level of demasking (from 1 to 14) if it also was identified correctly at all subsequent levels and was scored 15 if not recognized. Contrary to expectations from the motor simulation hypothesis, the IDs were as (or more) accurate and fast as the controls in recognizing pantomimes of action (Fig. 1A and Fig. S1A).

Fig. 1.

Results of experiments 1–5 by individual participant (see also Figs. S1 and S2). IDs are shown in black; control participants are shown in light gray; the mean of the controls is shown in darker gray. We applied two-tailed modified t tests (48) to test whether the performance of each ID was different from the performance of the controls. (A) The IDs were either as efficient [IDs 1–4: all modified t tests, ts (26) < −0.21, all Ps > 0.12] or more efficient [ID5: modified t test, t (26) = −2.36, P < 0.05] than the controls in pantomimes recognition. (B) All the IDs were as accurate as the controls at naming the non-upper limb actions [all modified t tests, ts (26) > 0.11, all Ps > 0.18]. All the IDs [all modified t tests, ts (26) > −1.45, all Ps > 0.08] except ID2 [modified t test, t = −3.72, P < 0.001] were as accurate as the controls in naming upper limb actions. We applied the Bayesian standardized difference test (BSDT) (49) to test whether the discrepancy in the ID’s performance between the two categories of stimuli was significantly different from the discrepancy between the two categories in the control group. The discrepancy in performance between the upper limb and the lower limb actions was larger in IDs 1, 2, and 3 than in the controls (all BSDT P < 0.05) but not in ID 4 and 5 (both BSDT P > 0.5). In comparison with the controls both ID 4 and 5 performed relatively better for upper limb actions (both modified t tests, ts = 1.19) than for lower limb actions (both modified t tests, ts = 0.52). (C) All the IDs were as efficient as the controls in discriminating learned from novel actions [all modified t tests, ts (26) > −0.19, all Ps > 0.45]. (D and E) The IDs were as sensitive as the controls to violation of the actors’ expectations for the large boxes (D) [all modified t tests, ts (17) > −1.74, all Ps > 0.05] and the small boxes (E) (all modified t tests, ts (17) > −0.71, all Ps > 0.05]. (F) IDs 1, 2, 4, and 5 were as sensitive as the controls to the outcome of the shots [all modified t tests, ts (25) > −1.76, all Ps > 0.05], but ID 3 was less sensitive [modified t test, t (25) = −2.15, P = 0.04].

Fig. S1.

Supplemental results of experiments 1–5: analyses by subgroups. The performances of the IDs are superimposed on the bars representing the mean performance of the matched control group (SI Materials and Methods, Materials and Procedures). Error bars indicate the range. All tests were two tailed. We applied the modified t test (48) to test whether the performances of the IDs are different from the performances of the matched control participants. (A) The mean number of steps of demasking needed to recognize the pantomimes did not differ significantly among control groups 1, 2, and 3 [F (2, 24) = 0.13, P = 0.88], and all the IDs performed within the normal range of their respective control group [all modified t tests, ts < −0.29, all Ps > 0.05]. (B) Controls from groups 1, 2, and 3 performed the task with comparable efficiency for both the upper limb actions [F (2, 24) = 0.32, P = 0.73] and the non-upper limb actions [F (2, 24) = 3.27, P = 0.06]. All the IDs performed within the normal range of their respective control group for the non-upper limb actions [all modified t tests, ts > −0.1, all Ps > 0.28]. For the upper limb actions, ID2 named significantly fewer upper limb actions than her control group [modified t test, t (10) = −2.97, P = 0.01], but all the other IDs performed within the normal range of their control group [all modified t tests, ts > −1.86, all Ps > 0.09]. (C) Controls from groups 1, 2, and 3 performed the task with comparable efficiency [F (2, 24) = 0.54, P = 0.59], and all the IDs performed within the normal range of the respective control group [all modified t tests, ts > −0.58, all Ps > 0.57]. (D and E) Control participants of groups 1 and 3 had comparable sensitivity to violations of the actors’ expectations for both the large boxes [t (16) = 0.13, P = 0.9])and the small boxes [(t (16) = −0.29, P = 0.77], and all the IDs performed within the normal range of the respective control group [large boxes: all modified t tests, ts > −1.52, all Ps > 0.17; small boxes: all modified t tests, ts > −0.96, all Ps > 0.05]. (F) Controls from groups 1, 2, and 3 performed the task with comparable efficiency [F (2, 23) = 0.07, P = 0.92], and all the IDs performed within the normal range of the respective control group [all modified t tests, ts > −1.75, all Ps > 0.08].

Another proposed role of motor simulation is that it provides critical complementary information to enhance stimulus identification under adverse perceptual conditions (13, 19, 23, 24). Experiment 2 used point-light animations to test this hypothesis. Participants viewed video clips of an actor reduced to 12 light dots placed on his main joints (center of the head, shoulders, elbows, wrists, center of pelvis, knees, and ankles) performing 20 upper limb actions (e.g., fishing) and 20 non-upper limb actions (e.g., walking backward) and were asked to name the actions. IDs 1, 2, and 3 were significantly less accurate in recognizing upper limb actions than non-upper limb (normal range) actions. However, in contrast to the motor simulation hypothesis, IDs 4 and 5 performed well above the mean of the controls for both upper limb and non-upper limb actions (Fig. 1B and Fig. S1B).

Another possibility is that motor simulation is particularly important in learning to recognize new actions (25, 26). Experiment 3 tested this hypothesis. Participants were asked to memorize 21 video clips (lasting between 2.1 and 2.4 s) of an actress performing meaningless gestures with the upper limb(s). Videos did not include any context or objects, and the actress’s body and face were kept neutral (27). Participants had to recognize the memorized clips from among 21 similar video clips. We calculated the number of correct (hits) and incorrect (false alarms) recognitions of previously learned actions and computed the d′ sensitivity index of each participant. The performance of the IDs was indistinguishable from that of control participants (Fig. 1C and Figs. S1C and S2C).

Fig. S2.

Supplemental results of experiments 1–5: correlation analyses. We carried out correlation analyses over the responses of the IDs and control participants on the items of experiments 1–5. (A) The mean number of steps of demasking needed by the IDs and the control participants to recognize the different pantomimes were strongly correlated [r (20) = 0.84, P < 0.01]. (B) The percentage of participants in both groups who accurately identified the different upper limb actions was strongly correlated [r (20) = 0.82, P < 0.01]. (C) The percentage of participants in both groups who accurately categorized the different actions was strongly correlated [r (42) = 0.79, P < 0.01]. (D and E) The percentage of correct categorization of each stimulus in both groups was strongly correlated for both the large boxes [r (16) = 0.83, P < 0.01] and the small boxes [r (16) = 0.80, P < 0.01]. (F) The percentage of correct categorization of each stimulus in both groups was strongly correlated [r (12) = 0.88, P < 0.01].

Motor simulation has been assumed to support the ability to make inferences based on a fine-grained analysis of others’ kinematics (28, 29). Experiments 4 and 5 examined this hypothesis. In experiment 4, participants viewed video clips of an actor’s arm lifting a small box or of an actor’s whole body lifting a large box containing one of four different weights (small box: 0.05, 0.3, 0.6, or 0.9 kg; large box: 3, 6, 12, or 18 kg) and had to decide after each trial whether the actor knew the correct weight of the box before lifting it (28). We calculated the number of correct (hits) and incorrect (false alarms) identifications of a violation of the actor’s expectation and then computed the d′ sensitivity index for each participant. In experiment 5 participants viewed video clips showing a player shooting a basketball; the clip was interrupted at the instant the ball left the player’s hands, and the participant had to predict the outcome of the shot (29). We calculated the number of correct (hits) and incorrect (false alarms) identifications of the successful free shots and computed the d′ sensitivity index for each participant. The performance of the IDs in these tasks was indistinguishable from that of control participants (Fig. 1 D–F and Figs. S1 D–F and S2 D–F). Thus, the ability to infer the outcome of others’ actions from their kinematics does not require motor simulation.

Still, motor simulation may support more basic perceptual processes underlying our perception of others’ movements. For instance, to process others’ movements in real time despite the informational lag imposed by neural transmission, our perceptual system typically anticipates the outcome of their movements by shifting the perceived position of the effectors forward along their plausible trajectories (30). According to the motor simulation theories of perception, perceptual anticipation (PA) of others’ movements and its tuning to biomechanically plausible movements depends on motor simulation of the observed movements (13, 30). To test this hypothesis, in experiment 6, participants watched series of three sequentially presented pictures inducing the perception of a hand movement that would be either awkward or easy to continue along the same trajectory, and they were asked to decide whether a subsequent probe picture matched the last picture in the series (Fig. 2A). Foils showed the arm shifted slightly backward or forward along its initial trajectory. We compared the number of “same” responses across probe type (forward and backward) and movement condition (easy and awkward). Both IDs and controls showed a significant bias toward forward probes in the easy condition, but this bias was absent or was significantly smaller in the awkward condition (Fig. 2B). Thus, like the controls, the IDs perceive the position of moving hands as shifted slightly forward along their trajectory (action anticipation) but did so only when the anticipated position was not biomechanically awkward. The finding that perceptual anticipation in the IDs is shaped by implicit knowledge of biomechanics shows that the computations underlying this effect are intrinsic to the visual system.

Fig. 2.

Methods and results of experiment 6. (A) Materials and methods. One probe picture is the same as the target, and the others (the foils) show the actor’s hand shifted 33 ms or 66 ms forward (+) or backward (−) relative to its position on the target (depending on the condition). Shown within solid lines is an example of a trial from the easy condition ending with a one-frame-backward foil. In dotted lines is an example of a trial from the awkward condition ending with a two-frames-backward foil. (B) Results. All tests were two tailed. Control participants [paired t test, t (19) = 6.49, P < 0.001] and the five IDs [all χ² (1) > 24, all Ps < 0.001] made more “same” responses for forward than for backward probes in the easy movement condition, but this forward bias was absent (in controls and in IDs 1, 2, and 5) or was significantly smaller [in IDs 3 and 4, both χ² (1) > 20, both Ps < 0.01] in the awkward movement condition. The size of the forward bias in the easy movement condition did not differ significantly between the IDs and the controls [all modified t tests, 0.75 < ts (19) < 1.98, all Ps > 0.05].

A critical issue, however, is whether the PA effect observed in experiment 6 also holds for the perception of continuous movement (31). Therefore in experiment 7 we asked participants to watch a series of video clips showing a computerized 3D actor performing a rotation of the arm away from or arriving at an awkward position and to decide whether a subsequent probe picture matched the position of the hand at the end of the video (Fig. S3A). Again, foils showed the arm shifted slightly backward or forward along its initial trajectory, and we compared the number of “same” responses across probe type (forward and backward) and movement condition (easy and awkward). IDs 1, 2, and 4, like the controls, showed the typical bias toward forward rather than backward foils when the hand moved out of, but not into, an awkward position (Fig. 3A). IDs 3 and 5 did not show the same profile. However, their performance was similar to that of some individual control participants (Fig. S3C), suggesting that their performance reflects variability intrinsic to the task.

Fig. S3.

(A and B) Materials, methods, and supplemental data of experiments 7 (A) and 8 (B). In each panel, the upper inducing video clip shown belongs to the easy movement condition (the actor’s movement could be continued easily along the same trajectory), and the lower video clip belongs to the awkward movement condition (the actor’s movement could not be continued easily along the same trajectory). Both conditions have the same last frame, target, and probe pictures that include the target and two pictures showing the hand displaced slightly forward (+) or behind (−) its position in the target (depending on the condition). (C and D) Supplemental data from experiments 7 (C) and 8 (D). Data are from individual control participants selected for their atypical profiles.

Fig. 3.

Results of experiments 7 and 8 (see also Fig. S3). All tests were two tailed. (A) Results of experiment 7. Control participants produced a significantly larger number of “same” responses for forward than for backward probes in the easy condition [paired t test, t (14) = 4.93, P < 0.001], but this bias was not significant [paired t test, t (14) = 1.59, P = 0.13] and was significantly smaller in the awkward movement condition [repeated measures ANOVA, F (1, 14) = 5.41, P = 0.04]. IDs 1, 2, and 4 showed the same profile, with a significant forward bias in the easy condition [all χ² (1) > 4, all Ps < 0.05] of the same size as that of the controls [all modified t tests, 0.61 < ts (14) < 1.10, all Ps > 0.05], but showed either no bias (IDs 1 and 4) or a significantly smaller bias [ID2: χ² (1) > 16, P < 0.01] in the awkward condition. IDs 3 and 5 did not show any significant bias in the easy condition [both χ² (1) < 1, both Ps > 0.5] but showed either a significant backward bias [ID3, χ² (1) = 9.77, P < 0.01] or forward bias [ID5, χ² (1) > 30, P < 0.05] in the awkward condition. (B) Results of experiment 8. Control participants produced a significantly larger number of “same” responses for forward than for backward probes in the easy movement condition [paired t test, t (14) = 4.64, P < 0.001], but this bias was not significant [paired t test, t (14) = 0.89, P = 0.38] and was significantly smaller in the awkward movement condition [repeated measures ANOVA, F (1, 14) = 5.43, P = 0.03]. IDs 1 and 4 showed the same profile of a significant forward bias in the easy movement condition [both χ² (1) > 6.64, both Ps < 0.01] that was significantly larger than in the awkward movement condition [both χ² (1) > 6, P < 0.01] and was the same size as that of the controls [both modified t tests, −0.3 < ts (14) < 0.93, both Ps > 0.05]. ID3 showed the same trend, although it was not significant [χ² (1) = 1.83, P = 0.1]. ID5 showed a significant backward bias in both conditions [both χ² (1) > 20, Ps < 0.01].

Finally, experiment 8 was carried out to ensure that the results reported in experiments 6 and 7 could be replicated with another body movement. Participants were presented video clips showing a computer-generated actor performing an extension/flexion of the right shoulder and were asked to decide whether a subsequent probe picture matched the position of the hand at the end of the video (Fig. S3B). ID2 was not tested in this experiment because she was able to perform movements of the stumps similar to the presented extension/flexion of the arm. Results from this experiment were analyzed similarly to the results of experiments 6 and 7. IDs 1, 2, and 4, like the controls, showed the typical bias toward forward rather than backward foils when the hand moved out of an awkward position but did not show that bias (controls and IDs 1 and 3) or showed less bias (ID4) when it moved into an awkward condition (Fig. 3B). ID5 did not show the same profile, but his performance did not differ significantly from that of the control participants (see the profiles of control participants 26 and 30 in Fig. S3D).

Discussion

In this study, we tested predictions derived from motor simulation theories of perception (6–13), which assume that efficient perceptual processing of others’ actions cannot be achieved by visual analysis of the movements alone but requires unconscious covert imitation—motor simulation—of the observed movements. The results we report here challenge this view. In the eight experiments, we found that most of the IDs, although they were incapable of simulating observed upper limb movements in their motor system, could perceive, anticipate, predict, comprehend, and memorize upper limb actions as accurately, as rapidly, and with the same sensitivity and biases as typically developed participants.

This evidence cannot be dismissed on the ground that we simply failed to find evidence for a difference of performance between IDs and the control participants because of a lack of statistical power. All the experiments were sensitive, with no ceiling or floor effects and, despite some performance variability (SI Discussion, 3), in all the tasks some of the dysplasic participants performed well above the mean of the controls, including younger controls with a high level of education. In addition, and crucially, we found positive evidence that the IDs’ perception of upper limb movements was characterized by the same perceptual biases (anticipatory and biomechanical biases) found in the response profile of control participants—perceptual biases that have been taken as evidence for a role of motor simulation in perception (7, 13, 30).

Our findings thus considerably extend previous results obtained with IDs or congenital paralysis (19, 32–35). It has been shown that individuals with congenital paralysis of facial muscles recognize facial expressions despite their inability to perform facial movements (32). It also has been found that individuals with upper limb dysplasia are influenced by body biomechanical constraints when asked to judge the laterality of hand drawings (33–35). The present study goes well beyond the scope of this previous evidence by demonstrating that motor simulation contributes neither to the ease (speed) of action recognition nor to its robustness and does not contribute to the ability to anticipate others’ movements perceptually, to predict the outcome of their actions, to read their mental states from their kinematics, or to memorize their body postures and movements. Another distinctive feature of the experimental results reported in this study is that they demonstrate the ability of the visuo-perceptual system, in the absence of motor simulation, to detect, process, and use information about body shape and movement to perceive and interpret observed actions efficiently even when they are presented in extreme, impoverished conditions. IDs performed comparably with controls when actions were to be recognized from only the very first movements and from configurations of the upper limbs (experiment 1) or from the displacement of only four dots representing the actor’s wrists and elbows (experiment 2), when very similar actions had to be discriminated after only a short exposure (experiment 3), when very subtle cues in the kinematics of the actors’ upper limb movements had to be used to make perceptual decisions (experiments 4 and 5), or when subtle perceptual biases were investigated (experiments 6–8).

An alternative interpretation of these findings might be that the IDs’ efficiency in perceiving and interpreting action arises from their “simulation” of the observed upper limb movements and actions with their lower limbs. It is known that in dysplasics the motor system used to execute lower limb actions is activated when they observe hand actions (36, 37). However, this alternative interpretation of the results produces two seemingly insurmountable challenges for current versions of the motor simulation theories (SI Discussion, 4). First, although the effectiveness of simulation in aiding action perception depends on the degree of similarity between the observed action and the observer’s own motor representation of that action (38, 39), the very different skeletal and muscular features and degrees of freedom of the arms and legs and of the hands and feet (40) make it virtually impossible to imitate the observed upper limb actions with the lower limbs. For example, the different mechanical limits of external rotation and of extension/flexion of the shoulder and the hip (40) make it very difficult to attribute the performance of the dysplasics in experiments 6, 7, and 8 as reflecting the effects of simulation of the observed arm movements by the feet. Second, and more importantly, the hypothesis that motor simulation contributes to the perception of action is of interest because the hypothesis assumes that motor simulation operates through a non–cognitively-mediated direct matching process between an observed, uninterpreted movement and the observer’s own motor representation of that movement (18). It is unclear, however, how a mechanism of direct execution–observation matching could operate when there are no motor characteristics in common between the motor plans involved in execution and the observed motor act. Effector-independent motor simulation thus requires the prior categorization of the observed action as an act of a particular type that, once interpreted, can be associated with the motor program of the same action executed with a different effector from the observed one (36, 41, 42).

Our findings thus clearly challenge the central premise of the motor simulation theories of perception: They demonstrate that it is possible to account for efficient action perception and interpretation and, crucially, to explain the kind of performance profiles that have been used so far to support the motor theory of perception (e.g., the anticipatory and biomechanical biases in perception of body movements) without appealing to the concept of motor simulation. The findings reported in this study thus support models of action perception and comprehension that distinguish perceptual, conceptual, and motor stages of processing (e.g., ref. 2). According to these models, when a movement is perceived, a visuo-perceptual analysis of the actor’s body shape and motion provides a visual description of the action, which serves as input to a conceptual system containing conceptual features or attributes of the action (e.g., its typical duration, function, goal, and so forth). As our findings show, these two stages of processing are necessary and sufficient to support the rapid, sensitive, and robust perception and interpretation of actions and to compensate for neural transmission delay through biomechanically tuned perceptual anticipation. According to these models, motor simulation occurs by both a non–cognitively-mediated direct route, supporting effector-specific covert imitation of observed movements occurring in parallel to the conceptual processing, and by a cognitively-mediated indirect route following the conceptual processing (2). However, motor simulation is not required for efficient perception and interpretation of action.

Admittedly, it is possible that the sensitivity to movement kinematics, perceptual anticipation, perceptual tuning toward biomechanically easy movements, and the efficient memory of novel upper limb movements and action recognition found in the IDs are all based on processes and representations different from those supporting the same abilities in the typically developed individuals; that is, the IDs use only visuo-perceptual computations, but the typically developed participants rely on motor simulation as well. Future studies are needed to elucidate this question with the help of neuropsychological studies of patients suffering from brain damage that affects their ability to imitate observed actions covertly, among others. However, several arguments speak against this possibility.

First, we found that the perception and interpretation of action was not only in efficiency but also qualitatively similar in the two groups. An analysis of the qualitative performance of the IDs in experiments 1–5 showed that they made errors on the same stimuli as the typically developed participants (Fig. S2), suggesting that their performance is affected by the same variables. Even more importantly, in experiments 6–8 we observed that the IDs’ perception of upper limb movements is influenced by the same anticipatory and biomechanical biases that affect control participants’ performance. This result demonstrates that the IDs’ and the controls’ perception of body movements cannot be distinguished from each other even at the level of the finest-grained characteristics; thus it is difficult to assume that they are using different strategies in all these tasks.

Second, this conclusion is consistent with computational modeling results showing that action perception and identification can be explained by exclusively visual processes and representations (1). It also is in line with the results from neuropsychological studies showing that damage to various parts of the motor system such as the basal ganglia (43, 44), the inferior parietal lobe (43–45), the inferior frontal gyrus (43, 45, 46), the left premotor cortex (43, 45), the primary motor cortex (43, 45), and the bilateral superior parietal lobule (46) cause motor or praxis disorders (a disorder affecting the capacity to perform actions despite preserved basic motor and somatosensory functions) but do not necessarily hamper action identification.

Third, and crucially, there seems to be no evidence incompatible with this conclusion. Indeed, the interpretation of the results from neuroimaging, behavioral, neuropsychological, and TMS studies, which have been cited in support of motor simulation theories of perception, remains unsettled (SI Discussion, 1). Moreover, one important aspect of our results is that they are based on experiments that used the same materials and procedures used in studies (28–30) that have been interpreted as providing conclusive evidence in favor of motor simulation theories of perception. We showed, for instance, with the same material and procedure as in the seminal paper by Wilson et al. (30) that, in contrast to the initial interpretation, the tuning of action anticipation toward biomechanically easy movements does not require motor simulation. Using the material and procedure from Bosbach et al. (28), we showed that, contrary to the initial interpretation of the results of that study, the ability to detect mismatches between prepared and executed actions and to use such detection to infer the mental state of an actor does not require motor simulation of the observed movements. Finally, using the materials from Aglioti et al. (29), we showed that the ability to predict the outcome of an action efficiently does not require motor simulation. Our results demonstrate that motor simulation is not needed to obtain the performance profiles that have been used to support the motor simulation theory of perception. In fact, within models that do not attribute a causal role to motor simulation in action perception, these results are taken to indicate that motor experience contributes not only to the acquisition of motor representations but also to the acquisition of spatiotemporal knowledge about actions and that the latter, not motor simulation, shapes action perception (47). The observation reported here that IDs 1, 2, and 3 were better at recognizing non-upper limb actions than upper limb actions in experiment 2 (see also ref. 19) and that ID3 was impaired in predicting the outcome of the basketball free shot in experiment 5 also point to this possibility (although this difficulty also could be explained by the IDs having different visual experiences of these actions). Collectively, these results call for the development of fresh hypotheses about how knowledge acquired through motor experience helps shape visually and conceptually based computations of body actions.

In conclusion, several motor simulation theories of perception have proposed that efficient perceptual processing of others’ actions cannot be achieved by the visual analysis of the movement alone but requires a process of motor simulation, that is, a covert execution of the observed movements. Our findings challenge this view and, although the extent to which these results generalize to typically developed participants remains an open question, they underscore the need for a shift in the burden of proof concerning the role of motor simulation in perception. Future research must address the fundamental questions of how the visuo-perceptual and cognitive systems support efficient and effector-constrained perception and interpretation of actions. In this latter framework, fundamental questions remain concerning how the visual system encodes information about the biomechanical constraints of body part movements, how this implicit knowledge constrains action perception, and how knowledge acquired through motor experience helps shape the processing of observed actions (SI Discussion, 1). In addition to its theoretical significance, this set of findings serves as a cautionary note in the application of principles derived from motor simulation theories of perception to the understanding or treatment of neurological and psychiatric conditions (e.g., refs. 14–16).

Materials and Methods

The experimental investigations were carried out from October 2013 to June 2014 in sessions lasting between 60 and 120 min. The study was approved by the biomedical ethics committee of the Cliniques Universitaires Saint-Luc, Brussels, Belgium, and all participants gave written informed consent before the study. During the experiment, participants were seated in front of a computer screen located at a distance of about 60 cm. The experiments were controlled with the E-Prime software (Psychological Software, 2002) and presented on a 15.6-inch Dell Latitude E5530 anti-glare laptop screen set at 1,366 × 768 pixels and 60 Hz.

Participants.

Five individuals born with severely shortened or completely absent upper limbs (see SI Materials and Methods for details) and five groups of control participants were tested (Table S1). No participant had a history of psychiatric or neurological disorder. None of the IDs had history of phantom limb sensations or movements (SI Materials and Methods). Most of the tasks were presented to more than one group of control participants (Table S2) so that we were able to compare each ID with a sample of control participants matched in age, gender, and education. Because no difference in performance was observed across the different control groups for any of the tasks, and because the results obtained when the each of the IDs was compared with the matched control group led to the same conclusions, we decided not to report the results of the different groups in the main text. Results of the different groups and the comparison of each ID with the respective control group are presented in Fig. S1.

Table S1.

Characteristics of participants in control groups

Group	No. of participants	Gender	Age, y (range)	Education, y (range)	Handedness
1	11	All women	48.54 (42; 52)	14.36 (12; 19)	All right-handed
2	9	All men	53.55 (48; 59)	16.88 (14; 19)	All right-handed
3	7	6 women	29.85 (24; 37)	19.85 (17; 22)	6 right-handed
4	15	13 women	22.8 (20; 28)		All right-handed
5	15	11 women	23.86 (20; 27)		All right-handed

Table S2.

Distribution of control groups across experiments

Experiment	Groups
Experiment 1	1, 2, 3
Experiment 2	1, 2, 3
Experiment 3	1, 2, 3
Experiment 4	1, 3
Experiment 5	1, 2, 3
Experiment 6	1, 2
Experiment 7	4
Experiment 8	5

Materials and Procedures.

A detailed description of the materials and procedures of the eight experiments is provided in SI Materials and Methods. The actions pantomimed in experiment 1 are listed in Table S3; the upper limb and non-upper limb actions presented in experiment 2 are listed in Table S4; and supplemental results are given in Tables S5 and S6.

Table S3.

Actions presented as pantomimes in experiment 1 (pantomime naming task with progressive demasking)

Description of the activity	Description of the activity
Opening a jar lid	Using scissors
Shooting a basketball	Eating with a fork
Pouring a liquid	Playing golf
Dealing playing cards	Browsing through a book
Dialing a telephone number	Throwing a ball
Playing the guitar	Writing by hand
Putting on a belt	Typing
Putting on deodorant	Answering a telephone call
Closing a jar lid	Shooting a bow and arrow
Playing the cello	Playing the violin

Table S4.

Upper limb and non-upper limb actions presented in experiment 2 (point-light animations naming task)

Upper limb actions	Non-upper limb actions
Putting on deodorant	Doing cartwheels
Fishing	Climbing down a ladder
Shooting a basketball	Climbing stairs
Writing on a board	Walking backward
Playing golf	Doing a somersault
Drinking	Sitting
Clapping	Lifting something up
Sweeping	Kneeling
Shooting a gun	Moonwalking
Dribbling a basketball	Squatting
Picking up something off the ground	Climbing a ladder
Telephoning	Walking
Putting on a sweater	Running
Shooting a bow and arrow	Doing a half-turn by jumping
Cleaning the windows	Kicking
Putting on a coat	Skipping
Playing a violin	Kicking a football
Splitting logs with an axe	Hopping
Playing tennis	Jumping in place
Digging	Turning 360° while jumping

SI Materials and Methods

Participants.

Participant ID1 was a 52-y-old man at the time of testing (19, 35, 50). He has a master’s degree in psychopedagogy. He has congenital bilateral upper limb dysplasia caused by in utero thalidomide exposure (51). Congenital abnormalities include aplasia of the left upper limb (i.e., the most severe form of dysplasia characterized by a complete absence of arm, forearm, hand, and fingers) and, on the right side, a shortened right arm (±12-cm humerus or ulna) directly fused to a hand composed of fingers 1 and 3 (shoulder, elbow, and wrist joints absent or not functional). Therefore, ID1 can move his right upper extremity only as a whole and by only a couple of centimeters in every direction. His physical and mental development is otherwise normal. ID1 never wore prosthetics, and there is no reported history of phantom limb sensations or movements (see SI Materials and Methods, Materials and Procedures below). ID1 has impaired vision in the left eye (visual acuity = 0.5/10) because of acquired eye injury but has corrected-to-normal vision in the right eye.

ID2 was a 47-y-old woman at the time of testing. She graduated from high school and was in the last year of studying for a bachelor’s degree in psychology. She has congenital bilateral upper limb dysplasia of unknown origin (52). Congenital abnormalities include shortened arms (±30 cm) directly fused to the hands (no forearm, elbow, or wrist) and oligodactyly of the hands (right hand: digits 1, 3, and a shortened thumb with two hypoplastic phalanges; left side: digits 1, 4, and a ±1-cm rudimentary thumb with two hypoplastic phalangeal-like bones). The shoulder joints are anatomically and functionally typical. Therefore, ID2 can move her upper limbs in all directions by the normal range of movement allowed by the shoulder joint. Her physical and mental development are otherwise normal. Vision is normal, and there is no reported history of prosthetic use or phantom limb sensations or movements.

ID3 was a 49-y-old woman at the time of testing. She graduated from high school. She has congenital bilateral upper limb dysplasia of unknown origin. Anatomically and functionally typical hands are positioned a few centimeters below the shoulders (no arms or forearms are present). Therefore, ID3 can move her hands by a couple of centimeters in every direction, and her fingers have a normal range of movement. Her physical and mental development is otherwise normal. Vision is normal. ID3 used bilateral prosthetics composed of pliers between the ages of 3 and 5 y. There is no reported history of phantom limb sensations or movements.

ID4 was a 27-y-old woman at the time of testing. She has a bachelor’s degree in graphic communication. She has congenital bilateral upper limb aplasia (no arm, no forearm, and no hand) and a shortened right lower limb, of unknown origin. Her physical and mental development are otherwise normal. Vision is normal. She uses a below-knee leg and foot prosthesis but has never used prosthetics to compensate for the missing upper limbs. There is no reported history of phantom limb sensations or movements.

ID5 was a 37-y-old man at the time of testing. He has a master’s degree in communication. He presents with congenital bilateral upper limb dysplasia of uncertain origin. Congenital abnormalities include aplasia of the right upper limb (no arm, no forearm, and no hand) and a shortened left arm (±15 cm, no forearm, and no hand) with an anatomically and functionally typical shoulder joint. Therefore, ID5 can move his left arm in every direction by the normal range of movement allowed by the shoulder joint. His physical and mental development are otherwise normal. Vision is normal. ID5 used switch-based right- and left-arm prosthetics as child and still occasionally uses a switch-based right-arm prosthetic as an adult. There is no reported history of phantom limb sensations or movements.

Materials and Procedures.

Phantom limb assessment.

The IDs were asked to answer the following question: “Often after amputation but also sometimes for congenitally absent limbs, people report that they can feel the presence of the missing limb(s) or at least some parts of it. These experiences are known as “phantom” limbs. Phantoms may be extremely vivid and present almost all of the time or may be quite fleeting experiences which occur every few months. The phantom may represent the entire missing limb (a complete forearm for instance) with all phantom parts being equally vivid or may be limited, for example, to the vague sensation of a hand with undifferentiated, ‘finger-like’ projections. The phantoms may manifest as voluntary movements, tingling feelings, or even pain. Have you ever had any phantom limb experiences in/of the missing limb?”

Experiment 1: Pantomime naming task with progressive demasking.

The pantomime naming task was performed by the five IDs and by the participants from control groups 1, 2, and 3. Stimuli were a set of 20 manual actions (listed in Table S3), filmed as pantomimes of object use performed by the same actress. The whole upper body of the actress was visible in all 20 stimuli. In most of the stimuli (n = 16) only the upper limb(s) were in movement (the face was neutral, and the body did not move). In the four other stimuli (shooting a basketball, playing golf, throwing a ball, shooting a bow and arrow) the upper limb movements were accompanied by coarse movements of the body and shoulders. All video clips were sized 978 × 550 pixels and had 30 frames/s. From each original movie we created 14 clips in which the number of frames ranged from 10 (330 ms) to 75 (2,640 ms) in five steps (165 ms).

During the experiment, participants were seated in front of a computer screen located at a distance of about 60 cm and viewed the 14 versions of each video in a row, from the shortest to the longest. Each trial began with the presentation of a black screen for 1,000 ms, followed by the video clip and a screen on which was written the question “What was the action mimed by the actress?” Participants responded orally to the question, and the experimenter wrote down their responses. They were encouraged to provide a response at each step, even if they were not sure. There was no time constraint for responding, but participants were asked not to respond before the end of each video clip.

Experiment 2: Point-light animations naming task.

The point-light animations naming task was performed by the five IDs and by the participants from control groups 1, 2, and 3. The stimuli were chosen from a set of 83 point-light animations of actions created from a motion capture database (asf/amc format obtained from the Carnegie Mellon University Motion Capture Database) with software developed locally. Each point-light animation consisted of 12 white dots corresponding to captors located on the major joints of the actor’s body (center of the head, shoulders, elbows, wrists, center of pelvis, knees, and ankles) presented on a dark gray background. The dots were ∼5 mm in diameter. The point-light actor was about 9 cm in height. These 83 point light displays were shown for naming to two groups of 20 control subjects. Participants in group 1 (mean age = 24.8 y; 10 males) were presented the point light actions and asked to name them. Participants in group 2 (mean age = 23 y; 5 males) were presented the same point light actions as group 1 but with the upper limbs masked (points at elbows and wrist removed) to determine the role of these limbs in the identification of each action. From this preliminary study, 20 manual actions (group1’s mean = 87.75%, group 2’s mean = 8.25%) and 20 nonmanual actions (group 1’s mean = 93.75%, group 2’s mean = 92%) were selected (Table S4).

During the experiment, participants were seated in front of a computer screen located at a distance of about 60 cm and viewed the 40 video clips in a randomized order. Each trial began with the presentation of a black screen for 1,000 ms, followed by the video clip and a screen on which was written the question “What was the action?” Participants responded orally to the question, and the experimenter wrote down their responses. They were encouraged to provide a response for each stimulus, even if they were not sure. There was no time constraint for responding, but participants were asked not to respond before the end of each video clip.

Experiment 3: Novel actions learning task.

The novel actions learning task was performed by the five IDs and participants from control groups 1, 2, and 3. Stimuli, obtained from ref. 27, were 42 video clips lasting between 2.1 and 2.4 s and measuring 960 × 540 pixels that showed an actress performing meaningless gestures with one or both upper limbs. From these video clips, 21 were chosen and used in an encoding phase (encoding set), and the other 21 were used as foils (foil set) during a recognition phase.

Participants were seated in front of a computer screen located at a distance of about 60 cm. In the encoding phase, they viewed the 21 video clips of the encoding set, presented at the center of the screen and separated by a black screen of 1,000 ms. Participants were required to memorize the clips as much as possible and were informed that during a later recognition phase they would be required to select those gestures from similar ones. In the recognition phase, participants viewed the 42 video clips of the encoding and foil sets, presented in the center of the screen in a mixed and randomized order. In this phase, each trial began with the presentation of a black screen for 1,000 ms, followed by the video clip and then by a screen on which was written the question “Was that action part of the first set?” Participants responded orally to the question (yes/no), and the experimenter wrote down their responses. There was no time constraint for responding, but participants were asked not to respond before the end of each video clip.

Experiment 4: Deception detection task.

The deception detection task was performed by the participants of control groups 1 and 3 and by IDs 2–5. The performance of ID 1 in this task had already been reported (19). Stimuli, provided by Simone Bosbach (28), were created by filming a male and a female actor lifting two different types of boxes (large or small) filled by the experimenters with four different weights (small box: 0.05, 0.3, 0.6, or 0.9 kg; large box: 3, 6, 12, or 18 kg). During the filming, the actors lifted each box (two types of box × four weights) several times; the boxes were presented to them in a randomized order, and the actors were always given information about the weight of the box they were to lift. In most of the trials they were correctly informed (no deception), but in some trials they were incorrectly informed about the weight of the box (deception trials). Thus, in the deception trials, there was a mismatch between the weight that the actors planned to lift and the actual weight of the box. For the purpose of this study, one video lasting between 2,320 and 5,360 ms was selected for each condition, resulting in 32 different videos (two actors × two types of box × four weights × two types of information). All the videos in the small-box condition showed the actor’s arm lifting a box from a table and placing it onto a shelf. All the videos in the large-box condition showed the actor going up to a box placed on the ground and lifting it. The actors’ faces were blacked out.

During the experiment participants were seated in front of a computer screen located at a distance of about 60 cm and performed two experimental blocks, one for each type of box; the large-box condition was presented first. In each block, the 16 different videos were presented four times (n = 64) in a pseudorandomized order. The videos were displayed, each in turn, in the center of a computer screen. Each trial began with the presentation of a central fixation cross for 200 ms, followed by a video clip and then by a screen on which was written the question “Was the actor correctly informed about the weight of the box?” Participants responded orally to the question (yes/no), and the experimenter wrote down their responses. There was no time constraint for responding, but participants were asked not to respond before the end of each video clip. As in the original study, each block was preceded by the presentation of eight video clips showing another actor lifting the four possible weights in both types of information conditions, in a randomized order. No feed-back was given during the example. We calculated the number of correct (hits) and incorrect (false alarms) identifications of a violation of the actor’s expectation and then computed the d′ sensitivity index of each participant for both conditions.

Experiment 5: Free shots prediction task.

The free shots prediction task was performed by the five IDs and by the participants of control groups 1, 2, and 3 (except for one participant from control group 1). Stimuli, provided by Salvatore Aglioti and Cosimo Urgesi (29), were initially 12 video clips showing a professional basketball player performing a free basket shot that he either missed (the ball landed out of the basket, n = 6) or made (the ball landed in the basket, n = 6). For the purpose of this study, the duration of each video was limited to the presentation of the 11 first frames, each lasting 66 ms (total = 726 ms). Frame 11 corresponds to the instant the ball leaves the player’s hands. In that way, only the player’s kinematics, not the ball’s trajectory in the air, was available in the videos.

During the experiment, participants were seated in front of a computer screen located at a distance of about 60 cm and viewed each video 10 times for a total of 120 trials, presented in a randomized order. All trials started with the presentation of a black screen for 1,500 ms, followed by the video clip presented at the center of the screen and then by a screen on which was written the question “Will the ball land in the basket?” Participants responded orally to the question (yes/no), and the experimenter wrote down their responses. There was no time constraint for responding, but participants were asked not to respond before the end of each video clip. The experiment was preceded by the random presentation of the 12 video clips to familiarize the participants with the material and by 12 training trials (unlike the procedure in ref. 29). No feedback was provided during these phases. We calculated the number of correct (hits) and incorrect (misses) identifications of the successful free shots and computed the d′ sensitivity index of each participant.

Experiment 6: Movement anticipation task.

The movement anticipation task was performed by the five IDs and by the participants of control groups 1 and 2. Stimuli, provided by Margaret Wilson (30), were created by filming an actor performing an external rotation of the right shoulder with 90° of abduction and 90° of forward flexion (Fig. 2). As shown in Fig. 2, five frames selected from this video clip were used as “inducing pictures.” Inducing picture 3 depicted the hand positioned close to the limit of the external rotation of the shoulder and was used as the target in all the trials (i.e., for both the easy and awkward conditions). Inducing pictures 1 and 2 from the easy condition depicted the arm oriented away from the midsagittal plane and were used to induce the perception of a rotating movement of the arm that would be easy to continue further along the same trajectory beyond the position of the hand in the target frame. Inducing pictures 1 and 2 from the awkward condition depicted the arm oriented toward the midsagittal plane and were used to induce a rotating movement of the arm that, because of the limits of the shoulder joint, would be awkward to continue further along the same trajectory after the position that the hand reached on the target frame.

Five frames were also selected to serve as “probe pictures”: inducing picture 3 (the target) and four other frames that depicted the hand close to its position in the target frame but displaced slightly away from (two frames) or toward (two frames) the midsagittal plane (the foil probes). The amplitude of the displacement of the hand on the foil probes corresponded to the position of the hand 33 ms or 66 ms before or after it had reached its position on the target picture during the original recording session (Fig. 2). A given foil probe was categorized as a forward or backward probe in one condition depending on whether it displayed the hand as having moved slightly forward along the trajectory implied by the inducing stimuli (i.e., in the future) or in the reverse direction (i.e., in the past). For example, the top probe picture in Fig. 2 (easy +66 ms, awkward −66 ms) represents the hand shifted 66 ms toward the midsagittal plane. Accordingly, this probe is considered a forward probe in the easy condition (66 ms in the future) and a backward probe in the awkward condition (66 ms in the past).

During the experiment, participants were seated in front of a computer screen located at a distance of about 60 cm and performed four blocks of 100 trials. In each block, the two conditions and the five different probes were in equal proportion and were mixed in a pseudorandomized order identical for all participants. The first block included familiarization with the four probes and 10 practice trials. All trials started with the presentation of a central cross for 500 ms followed by the presentation of four pictures (inducing pictures 1, 2, and 3 and then the probe picture) for 250 ms each, interleaved with a blank screen for the same duration. The probe picture was followed by a screen that was displayed until a response was recorded and on which was written the phrase “Was the last picture exactly the same as the third one?” and, below it, “If yes, press the green key (k); if no, press the red key (l)” for the control participants; “If yes, press the left pedal; if no, press the right pedal” for ID1; and “If yes, say yes; if no, say no” for IDs 2–5. We calculated the number of “same” responses provided by each participant across the two probe types (forward and backward) and movement conditions (easy and awkward).

Experiment 7: Movement anticipation task—continuous movement.

The continuous movement movement anticipation task was performed by the five IDs and by participants from control group 4. Participants were presented with video clips showing a computer-generated actor (Autodesk MotionBuilder) performing a rotation of the right shoulder with 90° of abduction and 90° of forward flexion of the arm (the actor’s body, shoulders, left arm, and face were kept still) and were asked to decide whether a subsequent probe picture matched the position of the hand at the end of the video (Fig. S3A). Motion was induced by presenting a still frame for 500 ms and then 12 frames for 20 ms each (240 ms in total) interpolated by 5°. The last frame of the video was identical for both the internal and the external rotation conditions and displayed the hand positioned close to the limit of amplitude of the external rotation of the shoulder joint. In this context, the internal rotation induces a rotating movement of the arm that would be easy to continue further along the same trajectory beyond the position of the hand in the target frame (i.e., the easy movement condition), whereas the external rotation induces a rotating movement of the arm that, because of the limits of the should joint, would be awkward to continue further along the same trajectory beyond the position that the hand had reached in the target frame (i.e., the awkward movement condition). The pictures used as probes were the target frame and two foil probes displaying the hand close to its position in the target frame but displaced slightly (4°, 26 ms) away or toward the midsagittal plane. A given foil probe was identified as a forward probe or a backward probe depending on whether it displayed the hand slightly forward along the trajectory implied by the inducing stimuli (i.e., in the future) or in the reverse direction (i.e., in the past).

Participants were seated in front of a computer screen located at a distance of about 60 cm. Participants performed three blocks of 60 trials. In each block, the two conditions and the three different probes were in equal proportion and were mixed in a pseudorandomized order identical for all the participants. The first blocks included familiarization with the four probes and 10 practice trials. Within each block, each trial started with the presentation of a blank screen for 1,000 ms followed by the presentation of a video clip, a 250-ms blank screen, and a probe picture displayed for 250 ms. Then, until a response was recorded, a screen was displayed on which was written the phrase “Was the hand on the picture exactly at the same place as it was at the end of the video clips?” and, below it, “If yes, press the green key (k); if no, press the red key (l)” for the control participants; “If yes, press the left pedal; if no, press the right pedal” for ID1; or “If yes, say yes; if no, say no” for IDs 2–5. We calculated the number of “same” responses provided by each participant across the two probe types (forward and backward) and movement conditions (easy and awkward).

Experiment 8: Movement anticipation task—extension/flexion of the shoulder.

The extension/flexion of the shoulder movement anticipation task involved participants from control group 5 and four of the IDs. (ID2 was not tested in this experiment because the presented extension/flexion of the arm lay within her motor repertoire.) Participants were presented with video clips showing a computer-generated actor (Autodesk MotionBuilder) performing an extension/flexion of the right shoulder (the actor’s body, shoulders, left arm, and face were kept still) and were asked to decide whether a subsequent probe picture matched the position of the hand at the end of the video (Fig. S3B). Motion was induced by presenting a still frame for 500 ms and then 12 frames for 20 ms each (240 ms in total) interpolated by 3°. The last frame of the video was identical for both the internal and the external rotation conditions and displayed the hand positioned close to the limit of amplitude of the external rotation of the shoulder joint. In this context, the flexion of the shoulder induces a rotating movement of the arm that would be easy to continue further along the same trajectory beyond the position of the hand in the target frame (i.e., the easy movement condition), whereas the extension induces a rotating movement of the arm that, because of the limits of the shoulder joint, would be awkward to continue further along the same trajectory after the position that the hand had reached in the target frame (i.e., the awkward movement condition). The pictures used as probes were the target frame and two foil probes displaying the hand close to its position in the target frame but displaced slightly (4°, 16 ms) away from or toward the midsagittal plane. A given foil probe was identified as a forward or backward depending on whether it displayed the hand as having moved slightly forward along the trajectory implied by the inducing stimuli (i.e., in the future) or in the reverse direction (i.e., in the past).

Participants were seated in front of a computer screen located at a distance of about 60 cm. Participants performed three blocks of 60 trials. In each block, the two conditions and the three different probes were in equal proportion and were mixed in a pseudorandomized order identical for all participants. The first blocks included a familiarization with the four probes and 10 practice trials. Within each block, each trial started with the presentation of a blank screen for 1000 ms followed by the presentation of a video clip, a blank screen was displayed for 250 ms, and a probe picture displayed for 250 ms. Then, until a response was recorded, a screen was displayed on which was written the phrase “Was the hand on the picture exactly at the same place as it was at the end of the video clips?” and, below it, “If yes, press the green key (k); if no, press the red key (l)” for the control participants; “If yes, press the left pedal; if no, press the right pedal” for ID1; or “If yes, say yes; if no, say no” for IDs 3–5. We calculated the number of “same” responses provided by each participant across the two probe types (forward and backward) and movement conditions (easy and awkward).