The Representation of Action

Abstract

The motor-program concept, emphasizing how actions are represented in the brain, helped bring the study of motor control into the realm of cognitive psychology. However, interest in representational issues was in limbo for much of the past 30 years, during which time the focus was on biomechanical and abstract accounts of the constraints underlying coordinated movement. We review recent behavioral and neuroscientific evidence that highlights multiple levels of constraints in bimanual coordination, with an emphasis on work demonstrating that a primary source of constraint arises from the manner in which action goals are represented.

The study of bimanual coordination has played a prominent role in psychological and neuroscientific investigations of action. Motivated by ecological considerations, bimanual tasks were introduced in the late 1970s in response to a motor-control literature built upon simple key-pressing tasks that focused on reaction time. Observation of daily life readily demonstrates that most actions are much more complex and require the integrative coordination of both hands.

Early studies with bimanual tasks pointed to fundamental limitations underlying the planning and execution of complex movements. This work led to models of motor control that emphasized that action planning was not disembodied but, rather, occurred within a set of constraints imposed by a physical system with a particular biomechanical and neural architecture (Turvey, 1990). The role played by representational issues, central to cognitive psychology, was de-emphasized in the study of motor control.

This decade, however, has seen the pendulum swinging back the other direction, with a resurgence of interest in representational issues in action planning and control. This work has emphasized limitations on our ability to concurrently perform independent actions with both hands. Indeed, many of the constraints identified in the bimanual-coordination literature may arise because the experimental tasks do not actually promote coordinated use of the two limbs but, rather, would be performed optimally if the two limbs were independently controlled. Consider the childhood challenge of patting the head with one hand while simultaneously rubbing the stomach with the other. The conflict experienced here is due to cross-talk between the trajectory signals directed to the two hands—the trajectory of each hand becomes more like that of the other, a phenomenon called spatial coupling. Akin to this example, traditional experimental tasks used in bimanual studies highlight that difficulties in producing bimanual actions are due to limitations not in the ability to coordinate the two hands but rather in the ability to control each hand independently.

An important point to emerge from this recent work is that limitations in bimanual coordination are highly sensitive to how action goals are represented. This has fueled a debate concerning whether or not bimanual coordination is constrained purely by perceptual or motor factors (Mechsner, Kerzel, Knoblich, & Prinz, 2001). An intermediate position is developed by considering how coordination reflects the interaction of cognitive, sensory, and motor constraints. Our goal in this article is to highlight behavioral and neuroscientific evidence suggesting that bimanual coordination and interference depend critically on how these actions are represented at multiple levels.

THE IMPORTANCE OF ACTION REPRESENTATION IN UNDERSTANDING MOVEMENT COORDINATION

As we noted, people are quite limited in their ability to produce complex bimanual movements that require asymmetric movements of the two limbs. However, Mechsner et al. (2001) provided a compelling demonstration that people can readily learn to produce bimanual circular movements in which one hand produces four cycles to the other hand’s three cycles. Such movements can be performed when the goal of the action has sensory consequences that entail a simpler representation and the attentional focus is directed to this sensory representation. With vision of their arms precluded, participants rotated cranks that moved visible flags. By using a gear system, the flags rotated at the same speed when the hands maintained a 4:3 ratio. Under these conditions, participants quickly mastered a movement pattern that would be seemingly impossible if the instructions had focused on the movements of the two hands. Similarly, Rosenbaum, Dawson, and Challis (2006) reported that people are able to move their hands with relative independence under external sensory guidance. Participants tracked two objects by lightly placing their fingertips on the objects. Even though the trajectories of the objects were independent, the participants were easily able to perform this task; such movement would have been severely limited had participants been asked to produce them without external guidance.

In a similar vein, Diedrichsen, Hazeltine, Kennerley, and Ivry (2001; also Ivry, Diedrichsen, Spencer, Hazeltine, & Semjen, 2004) found that constraints observed in planning bimanual actions are strongly influenced by how the actions are cued. When simultaneously reaching to two targets, participants are slower to initiate movements when the movements require asymmetric trajectories than they are when the movements require symmetric trajectories (Fig. 1). These effects have been interpreted in light of the idea that movement planning is facilitated when actions are symmetric. However, a critical difference between most laboratory studies of bimanual reaching and actions that occur in real life is that, in the former, the required actions are usually specified symbolically. For example, letters might indicate the target location for each hand, or spatial cues on a computer screen might specify the target locations for reaching movements performed on a table, thus requiring a translation between different reference frames. A series of experiments compared symbolic and spatial cues, to evaluate these factors. For spatial cues, stimuli directly specified target location. The movements themselves were essentially identical for both types of cues. Nonetheless, the reaction-time cost observed on trials requiring incompatible trajectories was dramatically reduced with spatial cues, suggesting that the translation from symbolic cues to their associated responses accounts for most of the cost (Diedrichsen, Grafton, Albert, Hazeltine, & Ivry, 2006; Diedrichsen et al., 2001). In most studies, a small cost is evident on trials requiring incompatible trajectories, regardless of whether they are cued spatially or symbolically (Heuer & Klein, 2006). However, the effect is considerably smaller when actions are cued spatially than it is when actions are cued symbolically.

Fig. 1.

Symbolically and spatially cued reaches performed by participants in a study of one- and two-handed reaching movements. Tasks are shown in panel A. Movements (dotted arrows) were executed forward (F) or sideways (S). The movements were cued by illuminating the target circle directly (spatial cues) or by letters indicating the movement direction (symbolic cues). Participants were tested in unimanual movements (left or right hand), bimanual movements with symmetric trajectories (both forward or both sideways), and bimanual movements with asymmetric trajectories (orthogonal movements). Reaction times for all of the conditions are shown in panel B. Modified from “Goal-Selection and Movement-Related Conflict During Bimanual Reaching Movements,” by J. Diedrichsen, S. Grafton, N. Albert, E. Hazeltine, & R.B. Ivry, 2006, Cerebral Cortex, 16, p. 1730.

In contrast with earlier perspectives that focused on motor programming and execution, recent research emphasizes that a principle source of constraint in bimanual coordination is related to how the task goals are conceptualized. With external sensory guidance or with spatial cues, minimal interference is evident under conditions that would lead to severe cross-talk with internally guided or symbolically cued movements (e.g., simultaneously drawing a U and a C, see Ivry, Diedrichsen, Spencer, Hazeltine, & Semjen, 2004). We propose that the manner by which actions are cued before movement initiation, as well as how they are guided during movement, can lead to radically different representations of the action goals. The pronounced difficulty observed when people produce nonsymmetric movements reflects interference arising from how the objectives of the task goals are conceived. Indeed, this form of interference is quite similar to that observed in traditional dual-task studies, suggesting that a primary source of constraint reflects limitations in processing resources or cross-talk associated with response selection and feedback processing. In this view, response selection and online control of bimanual actions are minimally taxed when the actions are directly specified or conceptualized to focus on a simplified sensory goal.

NEUROPSYCHOLOGICAL EVIDENCE OF A CENTRAL LOCUS OF BIMANUAL COUPLING AND INTERFERENCE

Given our argument that abstract representations of action goals are the prime source of constraint in bimanual coordination, we would expect these constraints to be operative even in extreme situations in which the actual movements or sensory consequences of the movements are absent. Studies with neurological patients have provided a unique opportunity to explore these issues, and they have provided mechanistic evidence in favor of a central locus for bimanual interference and spatial coupling. Franz and Ramachandran (1998) examined “bimanual” coordination in patients with upper limb amputation. Patients had vivid phantom limbs, reporting that they not only sensed the position of the missing arm but were capable of volitionally “moving” it. While drawing a straight line with the intact arm, patients were asked to “move” the phantom arm in a straight line or in a circular motion. As measured by the performance of the intact arm, the amputees showed interference in the spatially incompatible condition.

More recently, Spencer, Ivry, Cattaert, & Semjen (2005) tested patients who had severe loss of sensory input from the arms with minimal disruption of the motor signals. On a bimanual circle-drawing task these patients’ movements remained strongly coupled and exhibited more interference when the movements were asymmetric than they did when the movements were symmetric, even when vision was precluded. Unlike control participants, the patients exhibited large movement variability and asymmetry; for example, the two circles they drew differed in size. However, the interactions between the abstract goals of the two movements, as expressed by the degree of spatial coupling, remained unaffected. Taken together with the amputee study, this work underscores that bimanual interference is not critically dependent on processes that arise from the periphery—an idea consistent with the representational view developed from behavioral studies with healthy individuals.

While the preceding work has focused on excluding possible mechanisms underlying spatial coupling, studies with split-brain patients have helped point to the neural locus of bimanual interference found in neurologically healthy individuals. Remarkably, even with symbolically cued actions, split-brain patients are able to produce spatially incompatible trajectories with no interference (Franz Eliassen, Ivry, & Gazzaniga, 1996). Moreover, the movements of the two hands can be essentially independent (Kennerley, Diedrichsen, Hazeltine, Semjen, & Ivry, 2002). These results indicate that bimanual interference in neurologically healthy individuals arises from interactions involving communication between the cerebral hemispheres via the corpus callosum.

FUNCTIONAL IMAGING INSIGHTS INTO ACTION REPRESENTATION

As we have reviewed, the manipulation of task goals can produce dramatic changes in performance, suggesting that goal conceptualization can lead to qualitatively different forms of action representation. Functional imaging methods have provided converging evidence in support of this hypothesis, revealing that patterns of neural activation can show marked changes as the task goals are varied, even when the actual movements between the tasks are quite similar. One theme has been to compare actions that are internally generated to those that are externally guided. Debaere, Wenderoth, Sunaert, Van Hecke, and Swinnen (2003) asked participants to perform coordinated flexion and extension of the wrists. Participants in the internally generated condition performed the task with their eyes closed, and participants in the externally guided condition received online visual feedback in the form of a single cursor that indicated the degree of coordination between the two limbs. Externally guided movements elicited increased activation in premotor and superior parietal areas. In contrast, internally generated movements elicited increased activations in the basal ganglia, anterior cingulate, and inferior frontal and parietal cortices. Because interference was attenuated when visual feedback was present, the different activation patterns could be related to processes associated with internal or external control or to processes underlying bimanual interference and spatial coupling.

Diedrichsen et al. (2006) adopted a different approach, contrasting symbolically and spatially cued movements. This manipulation allowed the identification of brain activations related to (a) the mapping of symbolic cues onto associated movements, (b) goal-selection conflict that arises when this mapping operation requires the generation of incompatible movement trajectories, and (c) movement-related conflict generated by incompatible movement trajectories, independent of cue type.

In comparison to conditions in which movements were cued spatially, symbolically cued movements were associated with a large increase in activation across the extent of the intraparietal sulcus (IPS), as well as increases in the inferior parietal, premotor, and inferior frontal cortices, all in the left hemisphere (Fig. 2). Interestingly, activation of the middle and posterior aspect of the left-hemisphere IPS was evident even during movements produced with the left hand alone. These results suggest a critical role for the left-hemisphere parietal lobe in actions that require the translation of symbolic stimuli into actions. Similar activation patterns are observed when people manipulate tools with either the left or right hand, observe other individuals using tools, or produce and comprehend abstract gestures (Johnson-Frey, 2004). Thus, for the production of skilled movements, the left-hemisphere’s role may be related to a specialization for representing action goals at an abstract level rather than to the specification of particular movement parameters.

Fig. 2.

Brain regions activated by the execution of bimanual movements in the Diedrichsen, Grafton, Albert, Hazeltine, and Ivry (2006) study. When compared to spatially cued movements, symbolically cued movements elicited increased activity in the intraparietal sulcus (IPS; green region in the lateral view, A), the inferior parietal cortex (IPC), the inferior frontal cortex (IFC), and the premotor cortex (PMC), all lateralized to the left-hemisphere. When compared with symmetric movements, movements with asymmetric trajectories elicited increased activity in the superior parietal lobule (SPL) in both hemispheres. The combination of symbolic cue and asymmetric trajectory elicited increased activity in the anterior cingulate cortex (ACC) and presupplementary motor area (pre-SMA; seen in the medial brain view, B). Figure altered from work originally produced by Patrick J. Lynch and C. Carl Jaffe, licensed through the Creative Commons Attribution 2.5 License 2006.

When the symbolic cues indicated incompatible movements—the condition that leads to goal-selection conflict—an increase in activation in the anterior cingulate cortex and presupplementary motor area was observed. Across a range of tasks, these medial frontal areas have been associated with conflict sensitivity and the modulation of cognitive and motor effort (Paus, 2001). Thus, we again see that constraints associated with bimanual coordination may reflect more general processes instead of being specific to action planning and control.

Movements requiring incompatible trajectories were associated with greater activation in the posterior superior parietal lobule, a region linked to planning spatial aspects of movements. Interestingly, this effect was similar for the symbolic- and spatial-cuing conditions, despite the fact that the reaction-time increase was very small in the latter condition. Thus, the neuroimaging data point to both cognitive and motoric levels of constraint.

MOTOR OUTPUT, WHERE IT ALL CONVERGES

Based on the preceding discussion, one might assume that, when the task goal is appropriately conceptualized, the motor signals themselves impose minimal constraint. However, a body of neurophysiological evidence indicates that there is a bias to simultaneously activate homologous muscles of the two limbs. For example, during unimanual movements, transcranial magnetic stimulation (TMS) of the motor cortex reveals modulation in the excitability of motor pathways of the quiescent hand (Carson, Welsh, & Pamblanco-Valero, 2005).

The reaction-time cost observed in asymmetric bimanual movements may be the sum of independent processes, one associated with low-level interactions associated with motor execution and a second associated with higher-level planning processes. Alternatively, higher-level processes associated with action planning may interact with the motor system, leading to modulations in the degree of interference and coupling. Carson et al. (2005) showed that when a volitional movement with one arm was externally guided by visual feedback, the modulation of excitability of the quiescent arm was reduced compared to when the volitional movement was internally guided.

Based on the studies reviewed here, we postulate that there are two primary levels of interference underlying bimanual movements (see Carson & Kelso, 2004). In the first level, interference occurs when incompatible responses have to be planned and executed. This interference accounts for a small reaction-time cost and is likely due to increased planning demands related to asymmetric target specification represented in the posterior parietal cortex. The second level of interference, which accounts for the majority of the interference and spatial coupling, occurs when independent responses of each hand need to be planned and executed and the goals of the task need to be transformed from abstract codes into movement plans. Based on the finding that the process of translation from symbolic cues to actions plans is lateralized to one (left) hemisphere, we propose that this source of interference reflects the operation of a common processor that is engaged for each of the two actions. Interestingly, it appears that split-brain patients are not subject to this constraint: For these individuals, each hemisphere is capable of performing this translation process in parallel (Hazeltine, Weinstein, & Ivry, in press). This capability either reflects a functional reorganization following callosotomy (severing of the corpus callosum) or the operation of a more conservative strategy in the intact brain, perhaps ensuring that the overall behavior remains coherent (Meyer & Kieras, 1997).

Such constraints are not specific to motor control but, rather, reflect general properties of our cognitive architecture. Consistent with this hypothesis, conditions that produce conflict between the movements of the two hands lead to increased activation in the anterior cingulate cortex, similar to what is observed in many cognitive tasks (Bush, Luu, & Posner, 2000). Anterior cingulate activation would in turn project bilaterally to motor areas in both hemispheres, increasing the degree of bimanual coupling (Carson & Kelso, 2004; Carson et al., 2005). This view is consistent with evidence showing that activity in the anterior cingulate during one-handed and two-handed movements is correlated with the size of the corpus callosum in healthy individuals (Stancak, Cohen, Seidler, Duong, & Kim, 2003).

FINAL COMMENTS

In large part, bimanual coordination emerged as a model system in the field of motor control as an ecologically motivated alternative to the simple laboratory tasks that dominated initial attempts to understand the representational basis of action planning. The work reviewed here suggests that well-studied phenomena such as spatial coupling likely reflect, in large part, more general constraints that limit our ability to perform multiple tasks simultaneously (Ivry et al., 2004). This conclusion is drawn from the integration of behavioral and neuroscientific evidence suggesting a linkage between the difficulties encountered during the production of bimanual tasks and during the performance of dual tasks. This linkage has been made transparent by research manipulating how task goals are represented, even when the actual movements are held constant. Importantly, many of the model tasks employed in the motor-control literature are such that optimal (i.e., constraint-free) performance requires the independent control of the two limbs. However, our everyday behavior generally entails situations in which the hands work together to achieve a common goal even if the two gestures are quite distinct. For example, in opening a jar, one hand stabilizes the object while the other twists the lid. Tasks in which the two hands are used in a cooperative manner have not received sufficient attention in the psychological and neuroscientific literatures. The issues discussed here are likely to prove even more pertinent for these tasks given the unitary nature of the goal representation.