Evidence for a distributed hierarchy of action representation in the brain

Abstract

Complex human behavior is organized around temporally distal outcomes. Behavioral studies based on tasks such as normal prehension, multi-step object use and imitation establish the existence of relative hierarchies of motor control. The retrieval errors in apraxia also support the notion of a hierarchical model for representing action in the brain. In this review, three functional brain imaging studies of action observation using the method of repetition suppression are used to identify a putative neural architecture that supports action understanding at the level of kinematics, object centered goals and ultimately, motor outcomes. These results, based on observation, may match a similar functional anatomic hierarchy for action planning and execution. If this is true, then the findings support a functional anatomic model that is distributed across a set of interconnected brain areas that are differentially recruited for different aspects of goal oriented behavior, rather than a homogeneous mirror neuron system for organizing and understanding all behavior.

1. Introduction: Action hierarchy

A fundamental problem in motor neuroscience is to understand how the nervous system selects and organizes motor elements that, when combined, result in the completion of a temporally distant goal. Achieving this level of behavioral complexity across a broad range of contingencies, irrespective of whether a tool is used, sets humans apart from other animals. This is a key cognitive mechanism that is arguably equivalent to language in importance. How the brain accomplishes action organization remains largely unknown. In this review we argue for the hypothesis that action organization is based on a hierarchical model. Historically, evidence for action hierarchy has been driven by behavioral experiments (Rosenbaum, 1991) and computational principles (Arbib, Iberall, & Lyons, 1985). By using functional brain imaging of action observation and an exciting new methodological approach to experimental design called repetition suppression, we can, for the first time, stratify the understanding of goal-oriented behavior into distinct levels of control at a level of detail previously unobtainable with behavioral methodologies. Critically, the imaging data distinguish brain regions that are recruited for understanding different levels of control for an action. These areas form a functional-anatomic hierarchy that represents increasingly abstract aspects of observed behavior.

The review begins with a brief historic perspective on action hierarchy and related mechanisms based on insights from behavioral studies in normal participants and patients with focal neurological lesions. This is followed by our work on action representation using methods of brain imaging with a focus on three recent experiments that use action observation as an experimental manipulation. Implications for using experiments of action understanding to understand the organization of self-generated actions are then discussed in the framework of the mirror neuron system. Throughout, the emphasis is on the organization of brain functions underlying pragmatic, goal-oriented behavior rather than intangible goals or long-term strategic goals (Shallice & Burgess, 1991).

2. Historic perspective

The modern era for understanding the organization of complex motor behavior can be traced back in part to Nicholai Bernstein (Bernstein, 1996). He was one of the first to recognize a need for integrating evolutionary biology, musculoskeletal form and function, biomechanics and observations of goal driven behavior to explain motor behavior. He emphasized the notion of a control hierarchy spanning multiple levels of the neuroaxis, based on increasing complexity from muscle to spine to brain, with a supraordinate level for action formation at the top of this hierarchy. Others before him had recognized local hierarchies. For example, Sherrington (1906) distinguished upper and lower motor neurons, and Jackson (1875) and Ferrier (1874) referred to higher motor centers controlling simple movements. But it was Bernstein who saw the continuum through all levels of the nervous system. Critically, he also attempted to describe an internal structure at the highest level of the motor hierarchy, that is, in the formation of actions.

“Actions are not simply movements. Most of them are whole sequences of movements that together solve a motor problem. Each such chain consists of different movements that replace each other systematically, leading one to a solution for the problem. All the movements, parts of such a chain, are related to each other by meaning of the problem. If you miss one of the links of the chain, or mix up their order you will fail to solve the problem.” (Bernstein, 1996)

2.1 Essential elements of action

In this quote and later discussions he captured five essential ideas that form the foundations of research in action representation. The first is the notion of chaining. His concept of a chaining structure for movement elements is very different from an earlier theory of chaining proposed by Sherrington (1906). For Sherrington, chaining was a set of sensory driven reflexes, with an emphasis on low level responses generated by the spinal cord. The reflex chains described by Sherrington emerged irrespective of any desire to solve a problem or achieve a desired outcome. Furthermore, there was no consideration for the context of an action. In contrast, Bernstein was considering a series of known movements that could be generated independent of feedback and without any particular sensory reflex that could be combined in a sensible order to attain a goal. As an example, he noted how lighting a cigarette involved 20 distinct motor elements that were linked together to accomplish the final, temporally distal goal.

The second idea for Bernstein's action organization is adaptive variability. Given a novel context, we can adaptively recombine or substitute motor elements to achieve a goal. Lighting a cigarette in the wind demands that a new subset of motor elements are integrated into the chain to achieve the same distal goal.

A third idea implicit in Bernstein's writing can be called recursion. This is literally the ability to run back motor elements. That is, the nervous system has the ability to repeatedly call up or retrieve previously learned elements that form the substrates for generating an action. Movement elements are not formed de novo. They are based on a set of priors or well-learned primitives, subject to further refinement or generalization. Extensive work by Bizzi and colleagues (e.g., d'Avella, Saltiel, & Bizzi, 2003) identifies probable physiologic support for the existence of primitives at the spinal-muscle level of the control hierarchy. A fundamental question is how primitives change across the control hierarchy.

The fourth idea of Bernstein is that actions are performed to achieve a desired goal and to solve a problem. That is, the desired outcome is the organizing center for an action plan. William James had also considered this issue as well in his ideomotor theory. However, James thought that voluntary movement is organized as reflexive actions, and involves “an anticipatory image … of the sensorial consequences of a movement” (James, 1890). This precedes the action and guides performance, acting as a goal or target state that the motor elements should aim towards. This may be the first instance where someone considered the sensory consequences of an action as the referent to which motor planning should take place. However, Bernstein's idea of goal was more general, and involved a representation of the desired physical state of the world, held as a target prior to the action.

The fifth aspect of Bernstein's work is now referred to as chunking, the integration of independent motor elements into a single unit. With chunking there can be an increase of coarticulation and reduced cognitive demands because less elements are organized for a given motor goal (Gobet et al., 2001; Graybiel, 1998). Chunking is also a critical element for automatization to emerge. In summary, Bernstein's framework remains a fertile set of ideas for explaining complex behavior. Although each of these capacities is not unique to humans, when taken together they do reach an extraordinary level of competency in our species that allows for complex behaviors not otherwise observed.

2.2 Flexible cascade of motor elements

Given the above descriptive framework, how does the nervous system use contextual information to recombine motor elements in an optimal chain to achieve a distal goal? For any behavioral goal, such as the example given above of lighting a cigarette, there are simply too many elements and combinations for the brain to use a look-up table as a solution. That is, it is unlikely that we will ever find unique mappings between (1) the context within which the movement is performed, (2) a unique action sequence such as moving the hand and fingers that (3) matches the desired end goal of lighting the cigarette. What Bernstein was trying to identify was a more fluid process for structuring action, based on an internal hierarchical model where elements describing shorter action sequences or motor primitives could be combined. In such a structure a desired outcome is achieved within a cascade of intermediate steps that converge onto a solution. In this situation, the desired outcome is an invariant representation that is held as a reference during planning, when the desired elements are organized. One can readily imagine a learning structure whereby prior experience modifies the likelihood of selecting certain elements, analogous to the use of priors in Bayesian statistics. During execution, when on-line evaluation of performance is used amend movement in the face of noise, error or a changing environment these elements could also be recombined.

3. Behavioral evidence for control hierarchies

3.1 Prehension

Studies of normal prehension have played an essential role in demonstrating modularity in the organization of reach and grasp as separable, but interacting processes. In addition, prehension remains an important experimental paradigm for demonstrating how behavior is shaped in anticipation of future motor outcomes. During a reach and grasp, the arm, hand and digits move toward the desired object in a highly structured behavioral pattern, with kinematic features reflecting the object's size, shape, orientation and position (Jeannerod, 1981, 1984, 1986). Furthermore, the way people pick up an object is determined in large part by how the object will be used (Marteniuk, MacKenzie, Jeannerod, Athènes, & Dugas, 1987), demonstrating the interaction between distal goal and proximate solutions. If no distal action by the object is required, then the grasp configuration is determined in large measure by the optimal biomechanical position of the hand on the object (Rosenbaum, Meulenbroek, & Vaughan, 2001; Rosenbaum, Meulenbroek, Vaughan, & Elsinger, 1999; Rosenbaum, Vaughan, Barnes, & Jorgensen, 1992). However, if the object has a use, then the hand orientation or grip configuration is seamlessly adjusted to reflect this desired action, and not just biomechanical optimality. Examples of this abound from our experience with everyday objects. We pick up tools such as scissors by the center of mass to transport them, and hold them quite differently to use them. In addition, individual motor elements can be adjusted in advance, based on the desired outcome. Gentilucci, Negrotti, and Gangitano (1997) showed that when a person reaches and picks up an object, then places it on a second target, the initial reach velocity and grasp aperture kinematics are modified by the location of the final target. By manipulating the type of target, he could influence the transport kinematics alone. This is consistent with a control mechanism in which the different motor elements are not simply linked together in the correct sequence, but are also tuned individually and linked synergistically based on the final goal, with coarticulation observed as an emergent phenomenon.

3.2 Imitation tasks

Other examples that complex actions are organized in terms of control hierarchies based on outcomes can be observed in imitation tasks, particularly over the course of development. For example, Bekkering, Wohlschlager, and Gattis (2000) showed that when a child imitates another person performing a behavior such as grasping the ear on the same or opposite side of the acting hand, the child tends to copy the goal (the ear being grasped) rather than subgoals (the hand doing the grasping). In this model, interactions with objects and associated outcomes are considered to be higher goals than actions or movement paths (Wohlschlager, Gattis, & Bekkering, 2003). In their model, limitations in these children are not due to impairment of identifying goals, but in developing sufficient memory capacity to represent both goal and preceding motor elements. This distinction is supported by evidence that even very young infants are able to learn the relationships between motor elements and desired effects (Hauf, Elsner, & Aschersleben, 2004; Meltzoff & Moore, 1977). What takes time is the capacity to chain action sequences to achieve goals.

3.3 Prospective control

Prospective control paradigms test how children modify their behavior in a manner that circumvents future problems. A relevant example is the work of McCarty and colleagues (McCarty, Clifton, Ashmead, Lee, & Goubet, 2001; McCarty, Clifton, & Collard, 1999) who studied the emergence of prospective control in children during self-feeding. An infant grasping a spoon full of applesauce will be biased to use the dominant hand and will put whichever end of the spoon is on the thumb side of the hand into the mouth. If the applesauce is on the ulnar side, they can't get the food in the mouth without a new solution. Rotating the hand won't work as a solution because the food will spill. Over the course of development, infants first learn to pass the spoon to the other hand. Thus, they acquire the capacity to perform a two-step sequence. Later, they learn to overcome the dominance bias altogether and grasp the spoon with whichever hand works best relative to the initial orientation of the spoon. This capacity emerges rapidly through trial and error in a setting of learning without instructions. Not only does the child identify intermediate solutions (passing the spoon), they eventually replace this with another behavior (starting with the other hand). This is a powerful experimental model that is motivating the development of computational methods based on unsupervised learning that incorporate chaining based on a distal goal, hierarchical control, delayed reward and some sort of optimality principle that provides a mechanism for replacing one movement with another within and across levels (Brock et al., 2005).

4. Computational models

Given the many observations that actions are organized with respect to distal goals, what is the cognitive or computational framework within which this is achieved? Although the answer to this remains unknown, there are a number of important approaches to consider. A motor program could be played out like a computer algorithm or tape recording. Putative algorithms include feedforward control for sequences of movements such as typing or writing (Keele et al., 1995), action schema, and generalized programs (Schmidt, 1975; Schmidt & Lee, 1999). However, the use of autonomous motor programs misses out on the importance of perceptual information to create a context, to allow for updating and to provide a teaching signal for refining movement. Thus, recent models incorporate sensory information of some form. In a very influential cognitive model, the Theory of Event Coding, the notion of a sensory to motor transformation for guiding behavior is rejected. Instead, this model emphasizes the existence of a common outcome-based frame of reference that is accessible to both perceptual features as well as to motor commands (Hommel, Musseler, Aschersleben, & Prinz, 2001; Mechsner, Kerzel, Knoblich, & Prinz, 2001). This harkens back to Bernstein's proposition that the action is organized in terms of its effect on the external world. This appealing framework has been quite helpful for explaining patterns of interference that emerge in bimanual tasks and hand object interactions. But it is not entirely clear if a common code actually exists, and if so, what the structure or neural instantiation of this common event code really is. It is also limited by a single supraordinate level of representation common to action and perception and thus, a fairly compacted hierarchy.

Though it has been argued that the delays between a sensory event and a motor response severely limit the use of feedback control (Keele, Cohen, & Ivry, 1990), newer behavioral work reveals mechanisms based on the integration of motor commands into estimation of state that can largely mitigate these delays (Desmurget & Grafton, 2000). In related computational models of relatively simple behaviors, motor goals have been defined in terms of a ‘desired trajectory’ (Wolpert & Kawato, 1998), a ‘cost function’ (Hamilton & Wolpert, 2002) or an ‘instruction stimulus’ (Arbib, Billard, Iacoboni, & Oztop, 2000). A major breakthrough has been the integration of forward and inverse internal models for providing a mechanism that can predict both the current and future state of the motor plant (Wolpert & Flanagan, 2001). This approach can be used to reconcile the sensory consequences of actions (Blakemore, Wolpert, & Frith, 2000). An interesting symmetry emerges in these models: Prediction turns motor commands into expected sensory consequences, whereas control turns desired consequences into motor commands. Prediction is readily observed in prehension tasks, where the eyes lead the hand in locating goal targets (Flanagan & Johansson, 2003; Mennie, Hayhoe, & Sullivan, 2006). It has been shown in learning studies that prediction emerges faster than control (Flanagan, Vetter, Johansson, & Wolpert, 2003). The idea of predictive forward models has also been incorporated into a more sophisticated computational framework, where a system of multiple parallel forward – inverse model pairs are able to provide accurate control of action in a variety of contexts, in a model called MOSAIC (Wolpert & Kawato, 1998). In this case, different modules carry different motor solutions. Furthermore, it has been suggested that MOSAIC could be organized in a hierarchical fashion (Haruno, Wolpert, & Kawato, 2003), thus leading towards a model with chaining. Given sufficient levels, this could even provide a means for understanding other people's actions (Wolpert, Doya, & Kawato, 2003). Note that in all these examples the goal is assumed to exist at a level of control above the detailed model. All of these theories envision motor control as a refinement of information processing from a distant goal (‘light the cigarette’) to a more detailed motor plan (‘lift the match, strike the match’) to a precise specification of the reaching and grasping actions required to achieve each motor plan, and finally the activation of specific muscles in a coordinated sequence and the associated coarticulation that emerges at this level of organization. At a behavioral and computational level they provide evidence for a proof of principle, that on-line control, hierarchy, and prediction can all be integrated into a single model, but it is not yet known if or how any of these models are instantiated in the nervous system.

5. Neural evidence for action hierarchy

5.1 Ideomotor apraxia

Neural evidence that there are distinct brain structures for organizing movement in terms of relative hierarchy, including action goals, began with studies of apraxic patients. In building a case for what constituted apraxia versus other clinical syndromes a century ago, Liepmann (1988) argued that distinctions should be made at both a behavior level and in the concomitant localization of lesions in the brain. From the original meaning of Πραττειν, literally to act, that is, to move purposefully, he emphasized that apraxia was due to a loss of purposeful behavior and not just a disruption of movement. Purposefulness implies the presence of a goal, and thus apraxic disorders continue to have the potential for providing clues to the neural mechanisms that underlie action organization. Ever since Liepmann's seminal description, ideomotor apraxia remains the most common and best-understood form of apraxia. In this case, patients cannot perform gestures or pantomimes of common actions from verbal command despite preserved language and motor function.

Much effort has gone into explaining ideomotor apraxia with cognitive models (Rothi, Ochipa, & Heilman, 1997), and into subdividing apraxia into many subtypes (Heilman, 1979). It is useful to keep in mind that the hallmark clinical deficit remains an inability to retrieve or recall a desired movement or set of movements to accomplish a purposeful behavior. Localization according to Liepmann was in the left inferior parietal lobule (IPL). More contemporary lesion studies confirm this localization, but also show action retrieval deficits in posterior left middle frontal gyrus (MFG) (Haaland, Harrington, & Knight, 2000).

5.1.1 Complementary functional imaging studies

Functional imaging studies of gestures of tool use recruit left parietal cortex during task execution (Moll, De Oliveira-Souza, De Souza-Lima, & Andreiuolo, 1998). Using go, no-go paradigms, we have used functional imaging of normal participants during retrieval of common actions (such as a demonstration of how to use a hammer). In this case there is increased activity in IPL and MFG (Johnson-Frey, Newman-Norlund, & Grafton, 2005). Examination of individual participants shows that the IPL localization is consistently within the supramarginal gyrus in nearly all participants, and it is always in the left hemisphere whether the task is to be performed with the left or right hand. Consistent with this, a recent behavioral study in callosotomy patients with the left or right hemisphere dominant for handedness show a decoupling between hand preference and action retrieval, with praxis localized to the left side (Johnson-Frey, Funnell, Gerry, & Gazzaniga, 2005). Piecing this data together, we are in a position to assert that action recall, a critical step in the recursive processing of action elements, requires an intact left supramarginal and middle frontal gyri. Whether the lesion and imaging effects are due to a loss of a memory for motor elements (the engram) or just a deficit of retrieving motor memories stored elsewhere remains uncertain, although the observation that many ideomotor apraxia patients can have modality specific deficits (such as the ability to perform a gesture with the tool in hand but not when they are empty handed) suggests that the core problem is retrieval and not storage.

5.2 Ideational apraxia

Liepmann also described another form of apraxia that he labeled ideational apraxia (Liepmann, 1988). As Liepmann described it, in ideational apraxia, “The simplest motions are always successful and performed well… The disorder only becomes severe when a sequence of motions with different objects is made… The errors consist of partial omissions, the wrong sequence, actions performed ahead of time, the right motion with the wrong object.” In other words, his original definition had two parts: a deficit of chaining motor elements in the proper order to create purposeful movements and a deficit in demonstrating what an object, that is, a tool, is used for. Contemporary clinical studies have shifted attention away from ideational apraxia in terms of chaining errors and focused almost entirely on errors in understanding how to use or choose tools to accomplish a task, also referred to as conceptual apraxia, creating a confusing semiology (Ochipa, Rothi, & Heilman, 1989). With few exceptions (Poeck, 1983), little clinical progress has been made in identifying patients with isolated disorders in the chaining of motor elements to create purposeful movements, in part because of the rarity of this deficit as an isolated lesion. Liepmann placed the lesions that cause ideational apraxia in the posterior parietal-occipital temporal cortex, and contemporary lesion studies of ideational apraxia based on deficits of tool use (and not necessarily chaining errors) usually place the lesion in left tempero-parietal junction (De Renzi & Lucchelli, 1988). Localization for sequencing deficits in patients have been less precise, with lesions in left hemisphere (Haaland & Harrington, 1994), or left posterior hemisphere (Poeck, 1983).

5.3 Functional imaging studies of action sequencing

There have been no functional brain imaging studies of planning purposeful movements that are based specifically on the chaining of motor elements to accomplish a motor goal in normal participants. One related approach varied the complexity of a sequence of finger movements to identify changes of activity as a function of planning complexity. In this case, the left dorsal premotor and parietal areas were engaged to a greater degree for complex motor sequences based on the selection of different fingers, regardless of the performing hand (Haaland, Elsinger, Mayer, Durgerian, & Rao, 2004). It is unknown if this localization generalizes to motor responses other than individuated finger movements. Also lacking in this study is a clear relationship between motor sequence, hierarchy, end goal and adaptive variability. Instead, the design emphasizes working memory for representing sets of individual responses. An alternative approach for studying sequential organization is with the serial reaction time (SRT) task. In this case participants acquire and then perform well-learned movements that are both paced and in a fixed sequential order. Thus, although the task captures sequencing, it does not necessarily characterize chaining, hierarchy or adaptive variability. In addition, the nature of a goal in the SRT task is complex. Depending on training conditions, it is possible to demonstrate that participants are learning the perceptual ordering of stimuli in the task, the discrete motor responses in the task, or the outcome of the responses, that is, the distal goals or consequences of the action (Bapi, Doya, & Harner, 2000; Hazeltine, 2002). With SRT sequence learning there is typically increasing activity in supplementary motor area (SMA) or pre-supplementary motor area (pre-SMA) and the inferior parietal lobule (IPL) (Bapi, Miyapuram, Graydon, & Doya, 2006; Bischoff-Grethe, Goedert, Willingham, & Grafton, 2004; Grafton et al., 1995; Grafton, Hazeltine, & Ivry, 1998). At a single neuron level, coding for sequence order in a very small set of over-learned movements is observed in SMA in humans (Amador & Fried, 2004) and in monkeys within SMA, pre-SMA (Tanji, 1996; Tanji & Shima, 1994) and IPL as well as the motor cortex (Lu & Ashe, 2005) but not in regions associated with reaching (Batista & Andersen, 2001). Interestingly, humans with lesions to the SMA can demonstrate deficits in sequencing and in addition, show impairments in coordinating the reach and grasping components of prehension. Based on this, Gentilucci (2000) proposed that SMA is a key node for the final assembly of action elements into a common motor plan. In this model, the selection and serial ordering of elements required to achieve a distal goal are organized elsewhere, and then compiled in the SMA.

5.4 Neurophysiology of action sequencing

A recent and exciting approach in non-human primates for identifying other brain areas where motor elements might be organized has been to study animals that have learned sets of action sequences in more complex environments or with distal goals as part of the task structure. In one study, freely moving monkeys learned a delayed alternation task in a square room. In the north task, monkeys alternated between feeders: west-north-east-north-west, and so forth. In the south task, another alternation sequence was learned. Neuronal activity was recorded during walking for each of eight possible paths. Neurons in dorsolateral prefrontal cortex were selective for the spatial direction of an ongoing or upcoming response. Selective neuronal activity was tuned through learning and might represent the fundamental units of a planning mechanism. The sustained activity of these neurons suggests an integration between planning and working memory mechanisms (Ryou & Wilson, 2004).

In a different but related study, again recording in lateral prefrontal cortex, Shima, Isoda, Mushiake, and Tanji (2006) identified cells that demonstrated selective activity that was specific for a category of sequences. The animal had to call up a particular sequence according to a specific temporal structure to facilitate higher order planning based on memory. The implication is that there are neural representations capable of storing structured event complexes at a level supraordinate to the effecter. Finally, in a recent study of motor chaining, Mushiake Saito, Sakamoto, Itoyama, and Tanji (2006) measured neuronal responses in prefrontal cortex as monkeys planned multiple steps of a motor behavior using a maze task. The animals had to plan stepwise cursor movements to reach a final goal. During the preparatory period, prefrontal cortex (PFC) neurons reflected forthcoming cursor movements, rather than arm movements. In contrast, activity of neurons recorded in the primary motor cortex primarily reflected arm movements rather than cursor movements during both the preparatory period and during movement execution.

These three studies are an important step towards understanding the physiologic basis for chaining motor elements and suggest that a contribution by prefrontal and premotor areas needs to be considered in parallel human studies. The limitation of these studies is that the animals have over-learned a narrow set of motor sequences, thus they provide only a partial answer. They do not yet demonstrate the relationship between chaining and adaptive variability for selection of motor elements based on a given context and motor goal. In addition, the functional position of these PFC neurons within a larger control hierarchy needs to be characterized. In particular, recordings have not yet been performed in parietal cortex during these goal-sequencing tasks, so the relative contributions of parietal and frontal cortex to the hierarchical control of goal directed actions is not clear from the monkey data.

5.5 Human brain imaging evidence of control hierarchy

Two decades of imaging of the human motor systems using measurements of cerebral blood flow by positron emission tomography and changes of blood oxygen level dependent (BOLD) signals with functional MRI consistently demonstrate a widely distributed set of cortical and subcortical areas involved in volitional movement. As shown in Fig. 1, there is extensive recruitment of parietal, premotor and motor areas for even the simplest of movements, such as visuomotor tracking.

Fig. 1.

The visuomotor brain. All cortical areas showing increased activity during compensatory visuomotor tracking with the right hand relative to rest are shown in color, superimposed on a population based surface atlas of the human brain (PALS dataset and Caret visualization tools (http://brainmap.wustl.edu/caret). Data are from 8 participants, left column is right hemisphere, right column is left hemisphere, upper row is medial surface (inverted).

5.5.1 Preparation versus execution

A long standing idea is that control hierarchies should be reflected by differences in those areas that are recruited for preparation and execution (Passingham, 1993). It has been possible to dissociate neural substrates for planning and execution processes by means of go, no-go tasks (Johnson-Frey, Newman-Norlund, et al., 2005; Kawashima et al., 1996; Tunik, Schmitt, & Grafton, 2007) or instructional delay (Simon et al., 2002; Toni et al., 1999). These studies show greater activity in dorsal premotor and posterior parietal cortex during the planning phase, particularly in tasks where participants are performing motor selection of simple responses based on previously learned sensorimotor associations including choice reaction time tasks (Eliassen, Souza, & Sanes, 2003; Grafton, Fagg, & Arbib, 1998; Toni et al., 1999; Toni, Rushworth, & Passingham, 2001; Toni, Thoenissen, & Zilles, 2001). This is a very narrow range of behavioral complexity for defining a control hierarchy, with a small set of areas demonstrating greater activity for choice of a discrete movement relative to execution. Attempts to identify anatomic hierarchy based on manipulations of more complex behavior have been difficult to perform because of the limitations in what can be accomplished by a participant in the imaging device.

5.5.2 Imagined movement

An alternative approach has been to use imagined movement as a surrogate marker of motor planning, with the idea that imagined movement captures just the supraordinate level relative to execution in a motor control hierarchy (Roland, 1984; Roland, Skinhøj, Lassen, & Larsen, 1980; Tyszka, Grafton, Chew, Woods, & Colletti, 1994). Here too, there are problems because it is not clear that imagined movement is sufficiently similar to natural motor planning to serve as a surrogate and there is a strong overlap between areas recruited for imagined and executed movements, so that a neuroanatomical hierarchy based on imagined action is not sufficient on its own (Grafton, Arbib, Fadiga, & Rizzolatti, 1996; Johnson et al., 2002; Stephan et al., 1995). Imitation tasks could also be used for defining control hierarchies, but in this case there is a challenge of finding a suitable control condition that is equated in terms of working memory load, attention and effort (Buccino, Vogt, et al., 2004; Iacoboni et al., 1999; Koski, Iacoboni, Dubeau, Woods, & Mazziotta, 2002; Leslie, Johnson-Frey, & Grafton, 2004; Tanaka, Inui, Iwaki, Konishi, & Nakai, 2001). Because of these problems in using planning related tasks for defining motor control hierarchies, increasing attention has been given to experiments that localize with alternate strategies, including tasks where participants observe other agents performing different types of action (Decety et al., 1997; Grezes, Armony, Rowe, & Passingham, 2003; Grezes & Decety, 2001).

6. Shared substrates for planning and observation of action

6.1 The mirror neuron system

There is now strong evidence that observing an action by another, such as grasping an object, using a tool, or performing a whole body movement such as dance recruits a widely distributed network of inferior prefrontal, premotor, parietal and superior temporal cortex (Chao & Martin, 2000; Cross, Hamilton, & Grafton, 2006; Grafton et al., 1996; Grafton, Fadiga, Arbib, & Rizzolatti, 1997). Broadly speaking, these areas that are responsive during action observation can be referred to as an action resonance network. Subsets of these areas are also active during motor execution. Areas demonstrating activation for both observation and execution are commonly labeled the “Mirror Neuron System” (MNS) (Rizzolatti & Craighero, 2004). This terminology is motivated by the detection and characterization of mirror neurons in the brain of Macaque monkeys.

Single unit recordings in the inferior frontal cortex (area F5) of the monkey have revealed neurons coding for grasp configuration and object shape (Rizzolatti et al., 1988) and some of these are also responsive to the observation of actions (Di Pellegrino, Fadiga, Fogassi, Gallese, & Rizzolatti, 1992; Gallese, Fadiga, Fogassi, & Rizzolatti, 1996). These are neurons that are active when an animal performs a goal directed action, such as grasping a piece of food, and are also active when the animal observes another agent performing the same action (Rizzolatti et al., 1987). In particular, it has been shown that some cells in area F5 encode the goal of an action, because they respond when an action is inferred to have taken place out of sight (Umilta et al., 2001). As such, mirror neurons are strong evidence for the existence of action representation in terms of a goal that is at a supraordinate level relative to perception or execution. Similarly, single unit recordings in the anterior intraparietal sulcus (AIP) in the monkey have found neuronal coding of object shape and grasp (Sakata, Taira, Murata, & Mine, 1995), and it has been reported recently that neurons in monkey IPL fire when an action sequence is observed (Fogassi et al., 2005). These data have been interpreted in terms of neural coding for performed and observed intentions in the IPL.

6.2 One or many types of mirror neurons?

While the identification of mirror neurons in monkeys clearly demonstrates a representational hierarchy for action that is closely related to hand-object interactions, they also raise many questions as to how the hierarchy is structured. Is there just one type of neuron that is capable of performing supraordinate operations for virtually all types of action and perception including grasping, or are there many types of mirror neurons that vary as a function of the levels of complexity from simple movements like grasping to complex action sequences? A second unsolved issue is how action chaining is organized in this circuit. These are difficult to answer with available evidence because studies of the monkey brain are limited by the fact that in general, only one brain region, and often only one neuron, is tested at a time. This means that it is very difficult to get an overall picture of a motor hierarchy by mapping single unit mirror properties alone. Furthermore, the tasks used in different studies can be quite variable, and do not necessarily separate the different levels of representation. In particular, studies of grasping in monkeys have not always distinguished hand configuration from object identity (Di Pellegrino et al., 1992; Sakata et al., 1995). Finally, while there are homologies between the human and monkey brain, there are also major differences, which are matched by behavioral differences in planning, tool-use, flexibility of action and the ability to infer other people's intentions, all of which are limited or absent in monkeys.

6.3 Imaging the human mirror neuron system: Promise and challenges

Defining a MNS in humans by functional imaging has generated considerable excitement as a strategy for expanding the physiologic recording approach used in the monkey brain (Grezes et al., 2003). If the MNS is recruited similarly for both action execution and observation, then it becomes practical to characterize action hierarchies with observation tasks alone using an enormous variety of action movies and, thus, bypassing the physical limitations of trying to execute complex actions in the scanner. Given this logical leap to action observation as a surrogate method, there is a need for caution and a clear analysis of what assumptions and potential errors are inherent in using functional imaging to define a human MNS. Whether the human MNS actually contains mirror neurons remains unknown, the precise demarcation of the human mirror neuron system remains undefined, and there is no agreement on operational criteria for what observation/execution tasks should be used in humans to locate the human MNS. MNS studies have historically relied on imitation tasks, which is potentially problematic because imitation is a very complex behavior requiring working memory and because imitation was never used to identify mirror neurons in non-human primates in the first place (Arbib et al., 2000; Aziz-Zadeh, Koski, Zaidel, Mazziotta, & Iacoboni, 2006; Buccino, Vogt, et al., 2004; Koski et al., 2003).

6.3.1 Limitations of reverse inference

There is a problem with concluding that any fMRI response in premotor or parietal cortex is related to the MNS, without demonstrating overlap of both observation and execution responses in the same participants (Iacoboni et al., 2005). Nevertheless, it is not uncommon to find declarative statements in the imaging literature concluding that increased activity in a premotor or parietal area (irrespective of what the task is), signifies that the underlying cognitive process must be a mechanism based on mirror neurons (Buccino, Binkofski, & Riggio, 2004; Iacoboni et al., 2005). This is an example of reverse inference, where a cognitive process is inferred by a local activation. The validity of this inference is critically dependent on the consistency of finding the local activation (Poldrack, 2006). The human MNS is located somewhere within premotor, parietal and inferior frontal cortex (Grezes et al., 2003). There is no agreed upon criteria for mapping the MNS in this relatively large expanse of cortex and so reverse inference is particularly problematic. In addition, even if a brain imaging study shows recruitment in a particular area during both action observation and execution, then one cannot automatically conclude that the same population of neurons is generating the fMRI activation under both conditions (Grezes et al., 2003). It is possible that a given brain region, such as ventral premotor cortex, contains one set of neurons for perception and another for execution. In this case there is no population of neurons that is representing both perception and motor execution at a supraordinate level of explanation.

6.3.2 Boundary conditions for generalizing the MNS

It is not clear if all functional imaging studies that identify a brain response that is supraordinate to observation or execution should be called MNS. This enthusiastic use of the MNS as an explanatory principle for any process involving vision of or execution of a behavior has created exuberant generalization. This single population of neurons in the human brain has now been credited for the foundation of action formation, action recognition, language, imitation, learning, theory of mind, intentionality and social cognition (Arbib et al., 2000; Gallese, Keysers, & Rizzolatti, 2004; Miall 2003). It has even been argued that an intact MNS is needed to prevent autism (Burns, 2006; Williams, Whiten, Suddendorf, & Perrett, 2001). The problem of generalization is particularly important in light of a recent criticism of intention coding by the mirror neuron regions (Jacob & Jeannerod, 2005), which argues that actions are not sufficient to determine intentions, and that only low level kinematic representations should be found in motor regions such as the mirror neuron system. The core problem is that we are at a very early stage of knowing the boundary conditions for which the MNS is a useful heuristic for understanding human behavior.

6.3.3 Using action observation to understand action representation

Despite these criticisms of the MNS, it remains an important framework for determining the organization of action representation in humans. A basic question that we have been interested in is whether areas that are active during action observation and part of the MNS operate as a functionally uniform network or whether it is possible to detect functional gradations within the circuit depending on the level of action being processed. Such a gradient across the functional anatomy would suggest that there are multiple levels of representation for action understanding at a cognitive level and similar levels likely exist for performed action organization. This experimental logic can be traced back to Donald Hebb, who was one of the first to argue that if processes can be dissociated on a neuroanatomical level, then they should be considered as cognitively distinct operations (Hebb, 1949). For the action resonance network and the MNS, if we can show that different brain areas are recruited for different kinds of action observation, then the human MNS may need to be broken into component processes at both a cognitive and anatomical level.

Only a limited number of neuroimaging studies have attempted to distinguish different levels of the motor hierarchy based on manipulations of action observation. This is a challenging problem using standard subtraction methodology because any single task involving action observation requires simultaneous processing across all levels of action understanding. For example, if participants observe a video clip of a hand action during fMRI, e.g., (Buccino et al., 2001), brain regions involved in processing visual motion, hand kinematics, goals and intentions will all be activated, and it is hard to design a control condition that is missing any one of these levels without introducing confounds. For example, a few studies have used videos of actions without an object and thus without a clear goal or intention, in comparison to videos of goal directed actions (Pelphrey, Morris, & McCarthy, 2004). However, any differences could be due to low-level perceptual differences rather than differences of intentionality. Similar problems arise in studies using ‘accidental’ or irregular actions versus expected actions as stimuli (Manthey, Schubotz, & Von Cramon, 2003). In this case, confounds related to violations of expectancies or error detection dominate the contrast. Another approach has been to vary the background within which an action is performed (Iacoboni et al., 2005). In this case, differences of context define the goal of the movement. But there are other differences as well, including visual scene complexity, flanker effects, task difficulty, movement kinematics, and saliency. These confounds underscore the need to develop novel approaches other than the traditional subtraction method to distinguish intentions or kinematics across levels in the motor hierarchy.

7. New insights of action hierarchy from repetition suppression

7.1 Repetition suppression

We recently employed a method to distinguish levels of action representation based on a phenomenon called repetition suppression (RS). RS has been extensively used in studies of visual representations (Kourtzi & Kanwisher, 2000; Grill-Spector & Malach, 2001), where it is sometimes referred to as fMRI-adaptation. Repetition suppression is based on reduced physiologic responses to repeated stimuli. Fig. 2 is an example of an RS paradigm from one of our fMRI studies. The phenomenon is not unique to fMRI and is also observed at the level of single neurons. There are three major advantages to the RS approach. First, it allows us to identify changes within a class of stimuli or a level of the hierarchy rather than between classes. In this way, different levels of representation for the same stimulus can be analyzed independently. Second, it can be associated with behavioral correlates, such as reaction time priming (Maccotta & Buckner, 2004; Wig, Grafton, Demos, & Kelley, 2005), although we do not make use of this behavioral consequence here. Third, RS data can usually be interpreted as an effect related to neuronal population coding, because suppression occurs when two successive stimuli are represented in the same neural population, and release from suppression occurs when two successive stimuli are represented in different populations.

Fig. 2.

Example of trial ordering in a series of repetition suppression experiments. In a given sequence, when a stimulus of a given class, such as an object is repeated (Row B), there is reduction of BOLD related responses (Row A). Similar effects apply for different types of objects (Row C). The prediction is that different areas will show RS effects for outcomes of actions (Row D, E) and these can be separated from lower level features such as grip, trajectory and means.

This interpretation of RS data depends on two simple assumptions. First is the assumption of population coding within brain regions, for which there is extensive evidence in many parts of the cortex (Britten, Shadlen, Newsome, & Movshon, 1993; Georgopoulos, Schwartz, & Kettner, 1986). Second, the population response must change when the same stimulus feature is repeated. The precise pattern of this change remains a pattern of debate. It could be due to an overall reduction of neuronal firing, a decrease in firing duration, or a sharpening of neuronal tuning curves (Grill-Spector 2006; Krekelberg, Boynton, & Van Wezel, 2006). Despite these different causes at the neuronal level, the observation of population suppression to repeated stimuli is not in doubt and the principle of measuring RS in order to infer neuronal population coding appears to be sound.

A final issue that arises when using RS in motor studies is the assumption that the phenomenon occurs consistently across the neocortex. The vast majority of studies have examined RS in visual regions such as the lateral occipital complex and fusiform face area (Henson et al., 2003). However, there is also evidence for RS in memory (Buckner et al., 1998) and visual processing tasks (Shmuelof & Zohary, 2005) modulating frontal and parietal regions, and RS studies have been used to examine semantic (Thompson-Schill, D'Esposito, & Kan, 1999), syntactic (Noppeney & Price, 2004) and numeric (Pinel, Dehaene, Riviere, & LeBihan, 2001) representations. This plurality suggests that it should also be effective for studying action representation.

7.2 Constructing an action hierarchy

To identify possible topologies in the human brain corresponding to a motor hierarchy, we developed a library of stimulus sets designed to induce RS based on different observed features. They are listed here in a relative hierarchy from lowest to highest level of complexity.

1. Kinematics

   1. Reach trajectory. RS at this level is related to the detection of how an agent is approaching an object.

   2. Grip configuration. This level is defined by the specific hand-object interaction, such as a power or precision grip.

   3. Means. This encompasses several features, including dynamic interactions based on object weight and the specific transport or manipulation of an object.

2. Goal-object. This is defined by the identity and function of the grasped object

3. Outcome. This is defined by the physical consequences of an action, for example altering the position or configuration of objects in the world

In all our experiments we used a one-back RS design, where each stimulus is defined as novel or repeated relative to the one stimulus before it. This approach is motivated by the fact that RS is largest on a single repeated trial immediately following the prime stimulus and the amount of suppression does not increase after approximately 8 stimuli (Grill-Spector, Henson, & Martin, 2006). Thus, a one-back design provides an efficient and flexible approach to inducing and measuring RS within a single set of stimuli. While this is happening, participants are performing incidental tasks such as monitoring for a target and are unaware of the repetition structure. Thus, the task is well balanced in terms of top-down attention and cognitive set. From a practical standpoint, it becomes difficult to dissociate more than 2 or 3 factors by RS in a given fMRI experiment. The results described below reflect a composite of three recent reports.

7.3 Experiment 1: What and where of grasping

In our initial experiment we measured repetition suppression for which one of two possible objects was grasped, and for the trajectory used to grasp the object. Participants watched brief movie clips of a hand reaching and grasping one of two objects, such as a cookie or a computer diskette as shown in Fig. 2. Each of the objects was positioned to the right or left of midline, so that trajectory could be independently manipulated with respect to which object was grasped. The actor used a similar grip to take each object, then lifted it and transported it to the midline and the trial ended. Thus, there was no change in the grip or in the means or outcome of the hand object interaction between movies. Only the identity of the grasped object was manipulated. In this experiment, the essential goal of the task is grasping the object. Thus, the key RS effect is at the level of a goal as defined by object identity.

The main finding was a strong RS effect in the left anterior intraparietal sulcus (aIPS) when the same object was grasped, irrespective of trajectory, Fig. 3, top right. aIPS was not sensitive to trajectory. Instead, RS effects for trajectory were observed in left lateral occipital sulcus and right superior precentral sulcus, Fig. 3, top left. This result provided clear evidence for an action hierarchy during observation that is based on differences between reach kinematics and the goal of the action defined by the grasped object.

Fig. 3.

Anatomic substrates of action observation hierarchy. Repetition effects for low-level kinematics, such as hand trajectory, grip size and object movement are localized to visual association cortex including inferior occipital cortex and the posterior superior parietal cortex (parietal reach region). RS for object-centered goals strongly modulates the anterior intraparietal sulcus and left ventral premotor cortex. Sensitivity to the outcome of an action overlaps to a certain degree with goal-object areas, but also strongly modulates activity in bilateral inferior parietal lobule and right ventral premotor cortex. Data are adapted from three separate RS experiments (Hamilton & Grafton, 2006; Hamilton & Grafton, in press; Hamilton & Grafton 2007).

Localization of an action goal based on specific objects to aIPS is a generalization of the conventional view that this is an area for grip selection. Numerous fMRI studies comparing reach and grasp identify greater activity in aIPS for grasp (Binkofski et al., 1998; Culham et al., 2003; Frey, Vinton, Norlund, & Grafton, 2005). In addition, isolated lesions to this region disrupt grasp control but not reach kinematics (Binkofski et al., 1998; Frak, Paulignan, Jeannerod, Michel, & Cohen, 2006). This evidence supports the proposal that this region is a homologue to area AIP identified in non-human primates including macaque monkeys (Luppino, Murata, Govoni, & Matelli, 1999). A traditional view of AIP function is that it is a repository of grip apertures generated from object features. Two dimensional features of images projected onto the retina such as object shape, size, and orientation have been found to be encoded not only in early visual areas, but also by neurons in monkey area AIP (Murata, Gallese, Luppino, Kaseda, & Sakata, 2000). Neurons representing 3D shape have been found in the caudal intraparietal sulcus (area CIP) (Sakata, Tsutsui, & Taira, 2005; Tsutsui, Taira, & Sakata, 2005) as well as in an anterior section of the lateral bank of the intraparietal sulcus, area AIP (Sakata et al., 2005) of monkeys. Object specific firing occurs with or without vision of the grasping hand (Murata, Gallese, Kaseda, & Sakata, 1996).

One way to interpret this data on AIP neurons in monkey and aIPS BOLD responses in humans is that this parietal area integrates visual and motor information to represent hand-object interactions at a level of grip selection. An alternative view is that the area is mediating a more abstract representation, related to the goal of the task. One cannot conclude from the first study alone which of these is a more plausible explanation. This is addressed in the next experiment.

7.4 Experiment 2: What and how of grasping

In the second experiment, referred to as the wine drinkers task, participants watched an actor reach out and grasp either a wine bottle or a dumbbell placed on end (Fig. 2). The trial ended after the object was lifted and placed in a new location. Two levels of action representation were localized using RS (Hamilton & Grafton, in press). First, we varied whether they saw a grasp of the dumbbell or wine bottle, thus allowing an independent replication of the goal-object RS effect determined in experiment 1. Second, we manipulated how the hand grasped the object. A wine bottle and dumbbell each have a thin part and thick part. In a given trial, for example, the wine bottle would be grasped either by the body or the neck. With this manipulation we could test if aIPS responses are related to a higher order goal-object process or to local kinematic features of how the hand is interacting with the object, or both.

RS effects for goal-object effects were again localized to left aIPS extending into the adjacent IPL as well as the right aIPS, Fig. 3, right middle. These findings provide a strong replication of experiment 1 (Hamilton & Grafton, 2006), where left aIPS was reported. Using a statistical threshold appropriate for an exploratory analysis, RS for goal-object was also found in left inferior frontal gyrus (IFG). The critical question was whether aIPS was only sensitive to what was grasped, or if it also showed RS effects for how the hand grasped the object or how the object moved.

The main RS effect of how the object was grasped (Fig. 3, left middle) identified three clusters in the inferior and middle occipital regions, a single cluster in the IFG, and a region of the middle IPS. RS for grip was also seen in the SMA and MFG, but there was no evidence of RS for grasp in the anterior IPS or in IPL in either the left or right hemisphere. Interestingly, in a previous fMRI study using a one back detection task where participants observed static pictures of hand-object interactions we were able to identify increased IFG activity when the hand and object had a functional relationship compared to pictures where the hand and object were in an unrelated configuration (Johnson-Frey et al., 2003). These results point to a particular sensitivity of IFG for the detailed kinematic features between the handgrip and object.

Taken together, the results of experiment 1 and 2 provide strong evidence for multiple levels of grasp related action representation in the brain. In particular, the lateral occipital regions contribute to a visual analysis of hand-object kinematics for both how the hand approaches an object, the specific grip on the object and the subsequent movement of the object. In parallel, the inferior frontal region is sensitive to the local kinematic features of how an object is gripped. The relative differences of RS for trajectory and grip are in keeping with a long standing idea that grasp planning and reach control can be represented separately, but are usually linked in prehension tasks (Faillenot, Toni, Decety, Gregoire, & Jeannerod, 1996). Finally, we can assert that the role of aIPS, based on two RS experiments, is at the level of representing an object as a goal for the hand.

7.5 Experiment 3: What and why of grasping

The final RS experiment considered physical outcomes of actions (Hamilton & Grafton, 2007). Outcomes are central to organizing motor behavior and the neural substrates that support outcome representation might also play a role in related social behavior, such as identifying intentionality or mental states in others (Frith & Frith, 2006). To localize outcomes for motor actions in the brain, participants observed actors manipulating objects or tools to accomplish a particular goal. For example, in one trial they might see the actor reach and grasp the sliding top of a wooden box and push or pull the lid. Depending on the starting position of the lid, the outcome of the movement was to either open or close the box. Using RS, we could independently manipulate the outcome (open or close the box) from the means to accomplish the outcome (push or pull the lid). Subjects observed a large battery of movies capturing a broad range of familiar behaviors including turning a stove on or off, switching a light on or off, tying or untying a string, drawing or erasing with a pencil and hammering a nail or a nut by two different means. An RS effect for outcomes was found in the bilateral IPL and the IFG, Fig. 3, bottom right. In all these clusters, the robust response to novel outcomes was suppressed when the same outcome was repeated on a second trial, regardless of the means used to generate the outcome. Analysis of the responses to each of the individual sets of movies indicated that the RS effects for outcome in parietal and frontal areas were not driven by a single action or outcome, but generalized across a wide variety of actions. A striking aspect of the result was the strong shift to right hemisphere parietal and inferior frontal areas for outcomes.

In Experiments 1 and 2, grasping a particular object could be considered a type of low-level outcome in the sense that the final goal-object interaction was the purposeful movement. If so, then there might also be evidence for an RS effect in aIPS for the more complex action outcomes in experiment 3. To test for this, a region of interest analysis was performed in the aIPS using the localization of experiment 1 and 2. Within this region of interest, significant suppression for repeated outcomes was detected, supporting a model in which aIPS is involved in goal representation for tasks spanning a range of complexity.

An analysis of RS effects for means, comparing repeated movements and novel movements irrespective of outcome identified weak effects in left middle intraparietal sulcus, left lateral occipital cortex and left superior temporal sulcus Fig. 3, bottom left. These results show that these visual areas support the analysis of motor parameters. This is consistent with data from experiments 1 and 2 showing RS in left lateral occipital cortex for hand trajectory and grasp and motivates our schema that combines trajectory, grip and means into a common level.

7.5 Potential pitfalls of the RS method

There are three important issues that might affect our interpretation of these RS experiments. First, is the incorrect assumption that the three levels of behavior that we examined for RS effects must be independent of each other for this approach to be informative. This assumption is clearly not true at a behavioral level, as there is much evidence that the kinematics of performed hand actions are altered by the goal of the action (Ansuini, Santello, Massaccesi, & Castiello, 2006; Gentilucci et al., 1997). Furthermore, in RS imaging, experiments there is no requirement that the different levels must be independent parameters of interest. The RS experiments simply identify brain areas most sensitive to one or more levels of control, and this experimental method is biased towards the detection of modularity across functional levels. The results of the RS experiments do not exclude the existence of shared processing across action levels. Obviously, these relative levels of control all interact with each other.

Given the tight coupling between kinematics and goals, a second question is whether RS effects for outcome are actually due to more subtle differences of kinematics rather than the outcome itself. However, if this were the case, we would expect to see similar RS for both the subtle kinematic effects for different outcomes and also for kinematic differences irrespective of outcomes. This was not the case. Instead, across experiments 2 and 3 we independently obtained similar RS effects for studies for kinematic related repetition irrespective of goal or outcome (Hamilton & Grafton, 2006; Hamilton, Wolpert, Frith, & Grafton, 2006). Thus it is implausible to suggest that kinematic representations rather than outcome representations are driving the observed effects at higher levels of the action hierarchy.

The third question that arises from these three experiments is the possibility of an attentional confound. Parietal cortex, in particular on the right, has been associated with spatial attention (Corbetta & Shulman, 2002), so it becomes important to consider whether the RS effects we observed could be due to manipulation of attention. To test we manipulated participant's task related attention during Experiment 2 (Hamilton & Grafton, in press) but did not find any differences in the RS. Furthermore, if attention alone were responsible for the RS we identified, we would expect to see RS in the same ‘attentional’ brain region for every contrast. The fact that we found a distinct set of brain regions for grasps, goals and outcomes is evidence that the RS effects are truly reflecting neuronal population coding in different brain areas, rather than a general attentional phenomenon.

7.6 Implications for understanding the organization of action

The three brain imaging experiments of action observation based on the RS method identify a distributed set of brain regions that are differentially activated as a function of the complexity of motor behavior in relationship to the final outcome of a movement. The range of behavior we studied spanned three levels of behavioral complexity and stopped at the level of action outcome. Our results are particularly relevant for defining a functional-anatomic hierarchy involved in understanding intentionality as defined by watching another person as they manipulate objects. In these experiments, the manipulated object defined the outcome of an action, and was not a tool for achieving a particular outcome. It will be critical in future studies to determine how the incorporation of a tool into an action scheme would be represented in this hierarchy. In addition, it will be interesting to determine if this same functional-anatomic hierarchy is invoked for the understanding of more complex intentions such as human social interaction or in generating a theory of mind.

We found that detection and analysis of the outcome of a movement, such as the opening or closing of a box, recruited a right inferior frontal, biparietal network. This result overlaps closely with a study by Iacoboni et al. (2005) showing activation of these same sites when participants were evaluating action videos for intentionality. Intentionality was determined by the context, such as picking up a cup at the start or end of a meal. This localization was different from the detection of areas sensitive to evaluating goal-object interactions, which were most prominent in the left aIPS. Finally evaluation of lower level kinematics, such as how a hand grasps an object (rather than what is grasped), and arm trajectory including reach and transport, all engaged visual association areas. These areas are all interconnected, and in a hierarchical model such as this it does not make sense to conclude that any one region makes an exclusive contribution towards action understanding. Instead, this set of brain regions can be viewed as supporting a cascade of processing, much like the early visual areas do for vision.

Our current RS experiments have only examined observed behavior, and an equivalent set of studies for imagined or executed behaviors have not yet been conducted. However, if the general principle of mirroring applies throughout the action resonance system, then our results predict that the action control hierarchy should match the action understanding hierarchy. That is, it should be possible to identify regions within the distributed motor control system that differentially encode kinematics, goal-objects and outcomes. Some data support the contention that left aIPS is particularly important for controlling the goals of your own actions (Rice, Tunik, & Grafton, 2006; Tunik, Frey, & Grafton, 2005). These findings suggest that the principle of mirroring may apply at the level of goals in aIPS, but further work will be needed to determine if the whole of the MNS is hierarchical, with neurons encoding increasingly complex attributes of behavior.

There are also limits on our ability to conclude that the MNS is hierarchically organized based on currently available data. Critically, the left inferior frontal cortex did not demonstrate significant RS effects for outcomes or hand-object interactions. Instead, the right IFG and parietal cortex were maximally engaged. In contrast, in imaging studies of goal oriented action planning with pantomime of tool use (Johnson-Frey, Newman-Norlund, et al., 2005) or lesion localization of apraxia patients (Haaland et al., 2000) the left hemisphere premotor and parietal cortex is maximally engaged. This implies a functional-anatomical discontinuity of observation and planning at the outcome level and that is not consistent with the principle of mirroring.

At a lower level, related to goal- object representation, there is consistency of lateralization in parietal and premotor cortex between observation and execution. This would suggest that the overlap for observation and execution, i.e., the MNS, is maximal for mid-level control processes including what is grasped and how the hand grasps the object. The implication is that for representation of distal goals or intentions, there may not be a common neuronal substrate for execution and observation. The way to resolve these conflicting conclusions is to obtain additional studies combining observation and execution tasks in the same participants. The RS method is particularly well suited for this, given the ability of this method to isolate levels of processing while controlling for irrelevant task features such as attention, cognitive strategy and visual features. RS can be applied to action planning where repetition of different aspects within a planning hierarchy could be compared.