Grasping intentions: from thought experiments to empirical evidence

Abstract

Skepticism has been expressed concerning the possibility to understand others' intentions by simply observing their movements: since a number of different intentions may have produced a particular action, motor information—it has been argued—might be sufficient to understand what an agent is doing, but not her remote goal in performing that action. Here we challenge this conclusion by showing that in the absence of contextual information, intentions can be inferred from body movement. Based on recent empirical findings, we shall contend that: (1) intentions translate into differential kinematic patterns; (2) observers are especially attuned to kinematic information and can use early differences in visual kinematics to anticipate the intention of an agent in performing a given action; (3) during interacting activities, predictions about the future course of others' actions tune online action planning; (4) motor activation during action observation subtends a complementary understanding of what the other is doing. These findings demonstrate that intention understanding is deeply rooted in social interaction: by simply observing others' movements, we might know what they have in mind to do and how we should act in response.

The actions we perform in daily life are usually driven by a prior intention. A current controversy concerns the possibility to understand the intentions of others by simply observing their movements (Jacob and Jeannerod, 2005; Csibra, 2007; Kilner et al., 2007; Jacob, 2008, 2009). A person grasping an apple may grasp it to eat or to hand it to another person (Figure 1). Is it possible to anticipate what he/she is going to do next from the way he/she reaches and grasps the apple? Could an observer understand the person's intention by simulating the observed movement?

Figure 1.

Intention from movement understanding. The same action sequence—reaching toward and grasping an apple (A)—can be performed with different intents: (B) eat the apple, or (C) hand the apple to another person. Is it possible to understand the intention of a person grasping the apple by simply observing her movement?

Questioning motor theories of social cognition, Jacob and Jeannerod (2005) argued against this possibility: motor information might allow an observer to represent what an agent is doing, i.e., his/her intention in action, but will not allow him to understand why the agent is performing that particular action, i.e., his/her prior intention (Searle, 1983). Since the same motor sequence can serve different prior intentions, motor simulation—it has been claimed—might be sufficient to understand the agent's intention in action (e.g., grasping the apple), but it is not sufficient to understand the agent's remote goal in grasping the object. In other words, motor simulation is not sufficient to understand whether the agent is grasping the apple: (1) with the individual intention to eat it; (2) with the social intention to offer the apple to another person; or (3) with the communicative intention to show the apple to another person. To substantiate their claims, Jacob and Jeannerod (2005) proposed the following thought experiment. Consider the case of Dr. Jekyll and Mr. Hyde. The former is a renowned surgeon who performs appendectomies on his anesthetized patients. The latter is a dangerous sadist who performs exactly the same hand movements on his non-anesthetized victims. As it turns out, Mr. Hyde is Dr. Jekyll. Suppose that Dr. Watson witnesses Dr. Jekyll alias Mr. Hyde reaching and grasping for a scalpel. Would it be possible for Dr. Watson to recognize the different social intentions of Dr. Jekyll and Mr. Hyde in grasping the object?

According to Jacob and Jeannerod (2005), this is unlikely. Since the same movement sequence can be at the service of different social intentions, a simply motor equivalence between observed action and its motor representation in the observer's brain, might allow Dr. Watson to represent what the action is (e.g., that's a grasping), but will not allow him to discriminate between Dr. Jekyll and Mr. Hyde's social intentions.

Here we challenge this conclusion by showing how motor information conveyed by visual kinematics may provide a direct access to others' intentions. To illustrate this we focus on one of the most investigated objected-oriented action, grasping. Based on recent empirical findings within the grasping literature, we shall contend that:

1. in contrast to the assumption that the same movement might serve different intentions, the way an object is reached and grasped varies depending on the intention with which the object is grasped;

2. observers are especially attuned to kinematic information and can use early differences in visual kinematics to anticipate others' intentions in grasping an object;

3. during interacting activities, predictions about the future course of others' actions are integrated with pre-planned grasping actions;

4. motor activation during observation of grasping movements is sensitive to intention, and subtends a complementary representation of what the other is doing.

Kinematic specification of intention

The action of grasping an object might be performed with different intents: touch, use, move, throw, or pass. Evidence that prior intention shape action kinematics was first provided by measuring prior-to-contact grasping kinematics for reach-to-grasp movements performed toward a bottle filled with water (Ansuini et al., 2008; see also, Marteniuk et al., 1987; Ansuini et al., 2006; Sartori et al., 2011a). By comparing hand shaping across tasks involving different subsequent actions—pour the water into a container, throw the bottle, move the bottle from one spatial location to another spatial location—the authors were able to demonstrate how the prior intention in grasping the object strongly affected the positioning of the fingers during the reaching and the contact phases of the movement.

Using the same experimental window—reach-to-grasp for an object—variations in the kinematic patterning have been demonstrated for prehensile movements performed with an individual intention and prehensile movements preparing to a subsequent social interaction (Becchio et al., 2010; see also Mason and MacKenzie, 2005; Meulenbroek et al., 2007). Participants reached toward an object and grasped it either to move it from one spatial location to another (individual intention) or to place it into the hand of a partner (social intention). The results revealed a significant decrease in maximal finger aperture and peak grip closing velocity when the object was grasped to be passed to the partner (Becchio et al., 2008a). Similarly, Ferri et al. (2010, 2011) found that when a piece of food was grasped to be placed into the mouth of a human receiver, the final phase of the reaching slowed down compared to when the same action sequence was directed to a mouth-like aperture on the “face” of a human body shape.

Actions such as placing an object into a conspecific's hand or mouth directly affect the behavior of another agent. Communicative acts, in contrast, aims at influencing indirectly the behavior as a consequence of changing the mental state of the recipient. Successful communication relies on the fact that the recipient understands and recognizes the intention of the communicative act (Sperber and Wilson, 1986). To test whether communicative intentions modulate movement kinematics, Sartori et al. (2009a) devised a task in which participants used two spherical objects—a green and blue sphere—to communicate with a partner in a binary code. Different color sequences were associated with specific words. Participants were asked to select a word (and thus a color sequence) and to communicate it to the partner by lifting the spheres in the corresponding order. Relative to the execution of the same action sequence with no-communicative intent—grasp an object and simply lift it—approach movements to the object were more careful and accurate when the lifting action was performed with the scope to show the object to the interacting partner.

Taken together, these findings contradict the assumption that the same movement can serve different prior intentions (e.g., Jacob and Jeannerod, 2005; Csibra, 2007; Kilner et al., 2007; Jacob, 2008, 2009). Intentions influence action planning so that, although the to-be- grasped object remains the same, different kinematic features are selected depending on the remote goal to be achieved.

Intention-from-movement understanding

Is it possible to understand the intentions of other people from observing their movements? The finding that intentions shape action kinematics allows us to refine the question: are observers sensitive to differences in visual kinematics? Are they able to use these differences to understand other people's intentions in grasping an object?

One approach to investigate the contribution of motor information to intention understanding is to use temporal and spatial occlusion methods (Abernethy and Russell, 1987). Sartori et al. (2011a) adopted this approach to investigate how well observers can discriminate between cooperative, competitive, and individual intentions on the basis of movement observation. The experiment consisted of a motion recording phase and an intention discrimination task. First, to assess whether intention information was indeed available in the movement stimuli, they analyzed the kinematics of reach-to-grasp movements performed with the intent to cooperate with a partner, compete against an opponent, or performing an individual action at natural or fast speed. Next, to assess attunement to kinematic information, video clips of the same grasping movements were presented to participants in an intention discrimination task. To ensure that only advance sources of information were made available as to judge the model's intention, videos were temporally occluded at the time the fingers contacted the object so that neither the second part of the movement nor the interacting model, when present, was visible. The results revealed that observers were able to discriminate between cooperative, competitive and individual trials (Sartori et al., 2011a). Discrimination performance was similar for full-body video clips and partially occluded video clips, displaying only the arm and forearm of the model.

Video clips have the advantage of capturing the near normal visual input that is available when watching the motion of others. However, because movement information is provided in conjunction with other sources of information, they do not allow determining the specific role of motion cues. A method employed to isolate the contribution of kinematics is the point-light technique (Johansson, 1973). With this method, the movements of a body are represented by a small number of point-lights indicating the major joints of a moving person. Despite the drastic degradation of the stimulus, observers can easily understanding what an actor is doing in a point-light display (e.g., Dittrich, 1993). From observing a point-light action, they can identify the identity (e.g., Loula et al., 2005), gender (e.g., Kozlowski and Cutting, 1977; see also, Pollick et al., 2005; Richardson and Johnston, 2005), age (e.g., Montpare and Zebrowitz-McArthur, 1988), and emotion of the actor (Atkinson et al., 2004). Moreover, observers can easily discern activities involving two individuals represented through point-light displays (Mass et al., 1971; Manera et al., 2010), being able to use the action of one agent to predict the actions of a second agent (Neri et al., 2006; Manera et al., 2011a,b).

Observers viewing point-light displays of grasping movements can estimate the visual size of the invisible grasped object (Campanella et al., 2011). Manera et al. (2011c) examined whether they can also discern the intention in grasping the object. To this end, they presented participants with point-light displays of grasping movements performed with the intent to cooperate, to compete, or to perform an individual action (Figure 2). Point-light clips were temporally occluded at the time the fingers contacted the object, so that the second part of the movement was not visible.

Figure 2.

Examples of stimuli used in to investigate the contribution of motor information to intention understanding. (A) Single frames extracted from a video clip representing an individual cooperative action sequence. (B) Single frames extracted from a point-light clip representing the same cooperative action sequence. Adapted from Manera et al. (2011c).

The results demonstrated that participants were able to pick up the intention information available in the kinematic patterns: although discrimination was less accurate than under full-light conditions, observers were nonetheless able to discriminate between cooperative, competitive, and individual grasping actions (mean accuracy for the video clips = 76% of correct responses; mean accuracy for the point-light clips = 72% of correct responses). In this study, grasping movements were displayed from a lateral perspective, i.e., the perspective of passive observer. A question for future research is whether discrimination of intention is influenced by the viewpoint, i.e., whether discrimination of intention is facilitated when the action is observed from a view consistent with the observer performing the action or a potential interacting partner performing the action.

Closing the loop: from social intention to social affordance

During social interaction, understanding others' intentions is only part of the story (Frith and Frith, 2010): predictions about the future course of other's actions need to be integrated with pre-planned actions. For instance, when cooperating with a partner to accomplish a task (e.g., building together a tower with wooden blocks), we might use online action and intention prediction to plan an appropriate response to be performed at an appropriate time (Georgiou et al., 2007). But, imagine to be asked to cooperate with a partner who clearly displays the intention to compete, or vice versa. How would the incongruent attitude of the partner influence the implementation of your action? Translating this scenario into an experimental setting, Becchio et al. (2008b) have shown how interacting with a partner displaying an incongruent attitude produces a reversal in the kinematic patterning. Participants were asked to reach and grasp a wooden block either to cooperate with a partner in building a tower or to compete to place their object first in the middle of the working surface. For the congruent trials, the partner—a semiprofessional stage actor—displayed an attitude congruent with the task instructions: cooperative for the cooperative task, competitive for the competitive task. For the incongruent trials, her attitude was manifestly in contrast with the task instructions: competitive for the cooperative task, cooperative for the competitive task. Results revealed a significant reversal of kinematic patterning for incongruent trials: when cooperating with a partner displaying the intention to compete, the agent's kinematic pattern became similar to a competitive pattern; when competing with a partner displaying the intention to cooperate, the kinematic pattern became similar to a cooperative pattern. These findings might indicate that during incongruent trials participants were able to infer the partner's incongruent intention and this led to a re-planning of the action sequence. In this interpretation, reversal in the kinematic patterning would emerge as a consequence of intention attribution. An alternative yet not mutually exclusive explanation is that changes in the agent's kinematics resulted from interpersonal alignment. When people interact, they become aligned at many different levels, from basic motor programs to high-level aspects of meaning (Frith, 2008). Alignment in action, enabled by in-built motor resonance mechanisms (see below), may allow individuals to automatically adjust their actions to those of another person to achieve a common goal (Knoblich and Sebanz, 2008; Marsh et al., 2009). In this view, coordinated resonance might contribute to the emergence of cooperative and competitive patterns.

Beyond eliciting coordination, direct perception of intention in action has been proposed to afford specific action plans (Gangopadhyay and Schilbach, 2011). Raising an empty hand to ask for an object is a prime example. Using a perturbation paradigm, Sartori et al. (2009b) demonstrated that the social affordance of this gesture can be so powerful as to override the initial motor plan. Participants were instructed to grasp an object and then locate it on a platform. On 20% of trials, at the moment the participants started the action toward the objects, a co-experimenter unexpectedly stretched out her arm and unfolded her hand as if to ask for the object. Analysis of the participants' spatial trajectories revealed a significant veering in the arm trajectory occurring 165 ms following the perturbation. Strikingly, in some trials, participants totally disregarded the instructions and handed the object to the co-experimenter. No changes in movement trajectory were observed when the human co-experimenter was replaced by a robotic agent or when the perturbation consisted of a human arm conveying no social intention. Ferri et al. (2011) report a similar perturbation effect for the aperture of the mouth signaling the request to be fed. Exposure to this signal influences both the reaching and the placing phase of action sequences unrelated to feeding, e.g., reach and grasp a sugar lump and touch with it the forehead of the partner. As for hand request gestures (Sartori et al., 2009b), changes in movement parameterization are only observed when the partner's gaze was available, suggesting that gaze—acting a as a strong cue to intention (Becchio et al., 2010)—might be critical to activate an appropriate complementary action plan in the observer.

Taken together, these findings suggest that during social interaction, agents integrate the predictions about the future course of others' actions into their own action planning. As a result, action kinematics does not simply reflect the agent's intention, but also the intentions of others.

Mirroring beyond simulation

What neural mechanisms mediate this ability to extract intention from motion? It has been proposed that an important function of the motor system lies in the prediction of others' actions (Blakemore and Frith, 2005; Wilson and Knoblich, 2005; Prinz, 2006). Observing others' actions activates corresponding representations in the observer's motor system and these representations might be used to generate predictions by running an internal simulation (Wolpert and Flanagan, 2001). This simulation is thought to be carried out by neurons in the premotor and the parietal cortex that are active during both action execution and action observation—the so-called mirror-neuron system (Rizzolatti and Sinigaglia, 2010).

Evidence that in humans activity in mirror areas is sensitive to subtle difference in kinematics conveying intention information is provided by two recent functional MRI studies. Vingerhoets et al. (2010) found that discriminating between an actor's intention to move or use an object based on the visual properties of the movement involves multifocal activations within the intraparietal sulcus, a region also involved in planning of grasp-related actions (Tunik et al., 2007). Becchio et al. (2012) report that in absence of contextual information, observing prehensile movements performed with a social intent relative to prehensile movements performed with an individual intent activates mirror areas, including the inferior frontal gyrus and the inferior parietal lobule (along with the brain areas with no mirror properties, i.e., the temporo-parietal junction and dorsal medial prefrontal cortex, which are normally involved in processing social intentions).

Contrary to the assumption that motor simulation merely reflects what an agent is doing (i.e., grasping), these findings might indicate that simulation processes within the observer motor system integrate why aspects conveyed by action kinematics. An alternative, non-mutually exclusive explanation is that differential activity within mirror areas reflects the activation of a complementary response. Evidence favoring this hypothesis comes from experiments showing that laboratory training (e.g., Catmur et al., 2007) and context (Newman-Norlund et al., 2007; van Schie et al., 2008; Sartori et al., 2011b) can reverse mirror activity. For example, Newman-Norlund et al. (2007) report that merely changing the context in which an action is embedded modulates mirror activity so that activation in the inferior frontal gyrus and inferior parietal lobule is greater during preparation of complementary than of imitative prehensile actions. Along the same lines, Sartori et al. (2011b) demonstrate that depending on the context, motor-evoked potentials to transcranial magnetic stimulation reflect the observed movement or the complementary movement. When an object is present and the observer is implicitly required to act upon the object in response to the observed action, a shift from symmetrical motor resonance to complementary activations of hand muscles is observed. As prehensile movement performed with different intents are likely to evoke in the observer different complementary response, these findings might well explain why activation within mirror areas is modulated by intention. Note than on this account, mirror activation during action observation would not implement a simulation of the observed action, but subtend “a direct intersubjective perception of what the other is doing” (Gallagher, 2008).

Conclusions and future directions

Because kinematics retains specificity to the person's intention in producing a certain action, intention information is potentially available in the human kinematic pattern (Runeson and Frykholm, 1983). In this perspective article we have drawn together studies that highlight the importance of intention-from-movement information in social interaction. By simply observing others' movements, we might know what they have in mind to do. This knowledge is critical to discriminate between movements performed with different intents, predict what others will do next, and plan an appropriate complementary act in response. A challenge for future research will be to clarify how movement kinematics combines with other sources of information in the online prediction of others' actions. There are many situations in which the intention of an observed actor can be unambiguously inferred from goal objects and/or situational constraints. If the scene involves multiple possibilities of action, however, combining movement kinematics with other information might be crucial for action prediction. Stapel et al. (in press) found that in action observation settings observers exploit subtle movement cues rather than the direct visual information about target objects and context to predict how an ongoing action will unfold. Inasmuch as social interaction is rooted in the actions of the interacting agents, we would expect reliance on movement kinematics to be even greater during online social interaction.

Conflict of interest statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.