Two action systems in the human brain

Abstract

The distinction between dorsal and ventral visual processing streams, first proposed by Ungerleider and Mishkin (1982) and later refined by Milner and Goodale (1995) has been elaborated substantially in recent years, spurred by two developments. The first was proposed in large part by Rizzolatti and Matelli (2003) and is a more detailed description of the multiple neural circuits connecting the frontal, temporal, and parietal cortices. Secondly, there are a number of behavioral observations that the classic “two visual systems” hypothesis is unable to accommodate without additional assumptions. The notion that the Dorsal stream is specialized for “where” or “how” actions and the Ventral stream for “What” knowledge cannot account for two prominent disorders of action, limb apraxia and optic ataxia, that represent a double dissociation in terms of the types of actions that are preserved and impaired. A growing body of evidence, instead, suggests that there are at least two distinct Dorsal routes in the human brain, referred to as the “Grasp” and “Use” systems. Both of these may be differentiated from the Ventral route in terms of neuroanatomic localization, representational specificity, and time course of information processing.

1. Ventro-dorsal and dorso-dorsal substreams

Anatomical studies indicate that extrastriate cortex is composed of at least two segregated but interacting parallel processing streams. Traditionally, the outputs from the primary and secondary visual cortex (V1 and V2) to MT and visual area 4 (V4) are assumed to initiate two anatomically and functionally distinct channels of visual information processing named the dorsal and ventral streams. While MT is specialized for processing motion and depth, V4 is specialized for processing form and possibly color. Newer findings emphasize the role of area V3a in motion processing and its role in the dorsal stream. In general terms, the role of the dorsal stream is to mediate navigation and the visual control of skilled actions directed at objects in the visual world, whereas the goal of the ventral stream is to transform visual inputs into representations that embody the enduring characteristics of objects and their spatial relationships (Milner & Goodale, 2008).

In the monkey, downstream of MT and V3a a large number of interconnected extrastriate cortical areas in the parietal cortex, including medial superior temporal (MST), fundus of the superior temporal (FST), superior temporal polysensory (STP), ventral intraparietal (VIP), lateral intraparietal (LIP), mesial intraparietal area (MIP), anterior intraparietal (AIP) and inferioparietal area PF constitute the dorsal stream. Neuronal processing along the dorsal stream is best characterized by direction of motion and binocular disparity selectivity in MT, more complex motion analysis related to locomotion and pursuit/tracking in areas downstream from MT in the STS (superior temporal sulcus) (MST, FST, and STP), and computations informing target selection for arm and eye movements, object manipulation and visuospatial attention in areas of the intraparietal sulcus (IPS), which divides the IPL and SPL (AIP, MIP, LIP, VIP, and V6a).

There is, however, growing evidence that within the dorsal stream a further anatomical and functional subdivision exists. One of the sources of evidence for the subdivision of the dorsal stream are lesions with numerous neuropsychological consequences affecting visuo-motor function. Dorsal stream lesions affect smooth pursuit eye movements, accuracy of goal directed arm movements, speed discriminations, complex motion perception and the accurate encoding of visual space. The modularity of visuo-motor functions in the posterior parietal cortex (PPC) is also evidenced by the existence of several dorsal sub-streams achieving different visuo-motor transformations (Rizzolatti, Luppino, & Matelli, 1998). The idea of multiple visuo-motor occipito–parieto-frontal pathways has emerged from at least two different backgrounds. First, the theory of independent visuo-motor channels hypothesized that reach-to-grasp movements require independent coding of different object properties (location, size, orientation and shape) (Jeannerod, 1997). Second, anatomical studies have lent support to the idea that the transformation of these properties into appropriate movements of arm, finger and wrist is achieved by separated parieto-frontal pathways controlling the different body segments. For instance, anatomical studies have tended to confirm the existence of separate pathways within the dorsal system (Tanne-Gariepy, Rouiller, & Boussaoud, 2002), especially for reaching (V6a→PMd: Galletti, Fattori, Gamberini, & Kutz, 2004) and for grasping (CIP→AIP→PMv). There have also been neuropsychological reports consistent with this hypothesis. For instance, Binkofski et al. (1998) have reported patients with specific grasping-related impairments after a lesion of the anterior intraparietal sulcus.

Rizzolatti and Matelli (2003) have further detailed the anatomy behind the idea of multiple parallel parieto-premotor circuits, suggesting that parieto-frontal circuits are organized in a dorso-dorsal pathway, running from V3a to V6 to V6a and MIP in the superior parietal lobule (SPL), and from here to the dorsal pre-motor areas (F2vr and F7-non-SEF1); and a ventro-dorsal pathway, running from medial superior temporal area (MT/MST) to the inferior parietal lobule (IPL), and from here to the ventral premotor cortex (AIP – F5 and VIP – F4) (see Fig. 1).

Fig. 1.

Dorso-dorsal stream (A) and ventro-dorsal stream (B) in macaque (adopted from Rizzolatti et al. (1998) and Rizzolatti and Matelli (2003)).

Human neuroimaging data appear consistent with a modular architecture of the parietal lobes (for example Grefkes & Fink, 2005; Rushworth, Behrens, & Johansen-Berg, 2006; Seitz & Binkofski, 2003). The apparent absence of substantial crosstalk between a dorso-dorsal pathway through visual area 6 (V6) and the superior parietal lobule (SPL) and a ventral–dorsal pathway through MT and the inferior parietal lobule (IPL) indicates that the dorsal streammay actually consist of two relatively segregated subcircuits. It has been suggested that these parallel dorsal and ventral pathways maintain segregation all the way into motor-related frontal cortical areas such as the frontal eye field (FEF). Likewise, within the dorsal stream, segregated inputs linking the SPL to dorsal premotor area (PMd) and the IPL to ventral premotor area (PMv) have been shown to exist. Rizzolatti and Matelli (2003) proposed that the two anatomically segregated subcircuits of the dorsal stream might mediate different behavioral goals as well: the dorso-dorsal pathway concerned with the control of action ‘online’ (while the action is ongoing) and the ventral–dorsal pathway for space perception and ‘action understanding’ (the recognition of actions made by others).

While dorsal and ventral streams clearly make up two relatively separate circuits, the anatomical segregation between the two streams is by no means absolute. There is clear evidence of cross-talk between streams, such as the reported connections between V4 and areas MT and LIP, as well as between anterior inferotemporal cortex and inferior parietal area AIP was recently demonstrated in monkey by Borra et al. (2008) and functionally described by Pisella, Binkofski, Lasek, Toni, and Rossetti (2006), Binkofski, Reetz, and Blangero (2007) and Nelissen and Vanduffel1 (2011) (see Fig. 2).

Fig. 2.

Diffusion Tensor Imaging (DTI) analysis of connections between the area aIPS and the area cIPS, the ventral premotor conrtex and the infero-temporal cortex (adopted from Pisella et al. (2006) and Binkofski et al. (2007))

Thus, most connections from the ventral stream reach the ventral part of the dorsal stream, the ventro-dorsal substream. The ventro-dorsal substream seems therefore to constitute an interface between the ventral and the dorsal streams of visual information processing. This way of information exchange between the streams is especially interesting in the context of interaction with objects. It is very likely that both the dorsal and ventral streams are likely to process the same set of visual attributes, but for different behavioral goals. Fig. 3 presents a schematic location of the two dorsal sub-streams and the ventral stream in humans.

Fig. 3.

Shematic relative location of the dorso-dorsal (red) and the ventro-dorsal (green) sub-streams of the dorsal stream, as well as the ventral stream (blue) (from Binkofski and Fink (2005)).

2. Object processing in the dorso-dorsal stream

The dorso-dorsal stream is the most direct (immediate) visual pathway for action. A PET imaging study showed that reaching towards targets with various locations in space and presented through a mirror preferentially engages areas in the dorso-dorsal stream (especially V6a, see Binkofski et al., 2003). The cardinal deficit associated with lesions in the dorso-dorsal stream is optic ataxia (OA), as characterized by misreaching to visual targets that is most flagrant in the peripheral visual field (Balint, 1909; Garcin et al., 1967; Ratcliff, 1990). Indeed, deficits in on-line motor control demonstrated for reaching (Buxbaum & Coslett, 1997; Buxbaum & Coslett, 1998; Grea et al., 2002; Milner et al., 2001; Pisella et al., 2000; Rossetti, Goldenberg, & Rode, 2005; Rossetti, Revol et al., 2005) and more recently for grasping (Tunik, Frey, & Grafton, 2005) in patients with OA highlights the specificity of the superior parietal region and the parieto-occipital junction for direct goal-directed visuo-motor transformations involving short-lived processes (Milner & Goodale, 1995; but see Kroliczak, McAdam, Quinlan, & Culham, 2007). The usual lesion causing OA includes the superior parietal lobule (SPL), the intraparietal sulcus (IPS) and the parieto-occipital sulcus (POS) (Karnath & Perenin, 2005; Perenin & Vighetto, 1988).

The reach and grasp components constitute a first possible factor of dissociation between the dorso-dorsal and ventro-dorsal streams. Two studies have converged toward the anterior part of the IPS (aIPS) as the lesion site causing the distal grasping deficit (Binkofski et al., 1998; Tunik et al., 2005). Conversely, a recent neuro-anatomical study has proposed a more posterior and ventral site as a minimal lesion site causing the misreaching (Karnath & Perenin, 2005): The junction of the two sulci (IPS and POS), designated in another study as the parieto-occipital junction (POJ, Prado et al., 2005). The common zone of lesion overlap in the Karnath and Perenin (2005) study includes the white matter around this area, suggesting that all connections from occipital to parietal are disrupted and the visuo-motor functions therefore markedly disturbed. However, the double-dissociation between reaching and grasping deficits has not yet been described, and most OA patients exhibit deficits on the grasp components as well as misreaching with a posterior parietal (PPC) lesion sparing aIPS. The lack of observable isolated reaching deficits (contrary to the reverse dissociation that seems to emerge from the isolated lesion of aIPS: Binkofski et al., 1998) may simply be due to the combined reach and grasp activities found in the POS (Fattori, Breveglieri, Amoroso, & Galletti, 2004) and/or to the close localization of area cIPS (caudal part of the intraparietal sulcus) with respect to the lesion site revealed for misreaching (Karnath & Perenin, 2005). In studies using event related fMRI, area cIPS has been shown to process information about the spatial orientation of objects (and maybe also other spatial features of objects), which is then forwarded to aIPS. Information in aIPS may be processed to prepare adequate actions on these objects (Shikata et al., 2003, 2008).

One of the more interesting and perplexing findings in studies of OA is that inaccurate reaching may occur with either or both hands, in one or both peripheral hemispaces. In a recent study we identified four bilateral parietal foci, with the two relatively posterior foci showing greater lateralization for contralateral visual stimulation than more anterior ones (Blangero, Menz, McNamara, & Binkofski, 2009). Additionally, the two more anterior foci showed greater lateralization for the use of the contralateral hand than the more posterior ones. We also demonstrated that they are organized along a postero-anterior gradient of visual-to-somatic information integration. Furthermore, from the combination of imaging and lesion data we inferred that a lesion of the three most posterior foci responsible for target-hand integration could explain the hand and field effects revealed in OA reaching behavior (Blangero et al., 2009).

3. Object processing in the ventro-dorsal stream

In contrast to the dorso-dorsal stream, the ventro-dorsal stream appears to underlie processing of sensorimotor information based upon longer-term object use representations. Lesions of the ventro-dorsal stream produce impairments to more overtly “cognitive” aspects of action representation requiring knowledge of skilled object use, including pantomime of object use and use of real objects. Deficits in object-related actions are a hallmark of limb apraxia (LA). Given that OA is regarded as a typical disorder of the dorso-dorsal stream, online motor performance should by definition be preserved in LA. Indeed, a number of studies have shown that reaching and grasping actions in LA are normal when vision of the limb and target are available, but typically degrade when they must be performed “off line”, as when subjects are blindfolded prior to movement execution (Buxbaum, Johnson, & Bartlett-Williams, 2005; Haaland, Harrington, & Knight, 1999; Jax, Buxbaum, & Moll, 2006; Laimgruber, Goldenberg, & Hermsdorfer, 2005). This and other observations (e.g., Dawson, Buxbaum, & Duff, 2010) suggest that patients with LA may be overly reliant on online movement correction due to deficits in anticipatory planning.

A specific example of object use that is of interest here is tool use, in which the ventro-dorsal stream plays a major role. The observation that only patients with left brain damage encounter problems with single familiar tools or tool/object pairs is unequivocal, although a number of studies investigating “naturalistic” multi-step tasks involving several tools and objects like preparing coffee and fixing a cassette recorder has shown that the right hemisphere is also important for these complex functions (Hartmann, Goldenberg, Daumüller, et al., 2005; Schwartz et al., 1998). Specifically, it has been shown that only patients with left brain damage commit errors when asked to match objects to actions demonstrated without an object (Buxbaum, Kyle, & Menon, 2005; Kalénine, Buxbaum, & Coslett, 2010; Vaina, Goodglass, & Daltroy, 1995; Varney, 1978; Vignolo, 1990), to pantomime the action associated with an object (Barbieri & De Renzi, 1988; Goldenberg, Hartmann, & Schlott, 2003; Goodglass & Kaplan, 1963) or to match objects subserving the same purpose (De Renzi, Scotti, & Spinnler, 1969; Rumiati, Zanini, Vorano, & Shallice, 2001; Vignolo, 1990). The ability to infer possible functions from structure in order to apply novel tools linked to their complementary objects by transparent mechanical relationships (Goldenberg & Hagmann, 1998; Heilman, Maher, Greenwald, & Rothi, 1997), or to discover alternative uses of familiar tools (e.g. a coin for screwing, Heilman et al., 1997; Roy & Square, 1985) was also found to be impaired only in patients with left brain damage. Lesion studies have confirmed that the left IPL plays a crucial role in making correct inferences about the function of an object from its structure (Barbieri & De Renzi, 1988). Other tool-responsive regions in the ventro-dorsal pathway (also in the left hemisphere) consist of an anterior portion of the IPS in the IPL and of the PMv (Binkofski, Buccino, Posse et al., 1999; Binkofski, Buccino, Stephan et al., 1999; Boronat et al., 2005; Chao & Martin, 2000; Handy, Grafton, Shroff, Ketay, & Gazzaniga, 2003; Johnson-Frey, 2004; Kellenbach, Brett, & Patterson, 2003).

In summary, there are two distinctive action systems: a bilateral system specialized for online actions directed at currently visible stimuli on the basis of their structure (size, shape, and orientation), subserved primarily by a Dorso-Dorsal system, and a left-lateralized system largely devoted to skilled, functional object-related actions, mediated by a more inferior Ventro-Dorsal stream (Buxbaum, 2001; Fridman, Immisch, Hanakawa, et al., 2006; Glover, 2004; Buxbaum & Kalénine, 2010; Johnson-Frey, 2004; Pisella et al., 2006; Randerath, Goldenberg, Spijkers, Li, & Hermsdörfer, 2010; Vingerhoets, Acke, Vandemaele, & Achten, 2009). Randerath et al. (2010) describe that inappropriate non-functional grasping occurred very rarely in their group of 42 left hemisphere stroke patients, and suggested this could be explained by the contribution of the preserved dorso-dorsal route. Conversely, and also consistent with our account, they additionally noted that support of the preserved dorso-dorsal route seemed not to be equally sufficient for the use of tools. For brevity, these may be characterized as the “Grasp” and “Use” systems, respectively. In the next sections, we consider several important questions about the nature and time course of information processing in each system.

4. The concept of affordances as it relates to the two action systems

Historically as well as in contemporary theory, a central notion in the literature on object-related action is that of “affordance”. Object affordances support interactions of particular types, such as picking up a cup (Gibson, 1979). From a theoretical point of view, the notion of affordance is very important for two reasons: (1) because it demonstrates the close connection between perception and action, and (2) because it provides an understanding of the importance of variability and context. Whether or not an object affords a particular type of interaction is determined jointly by the pragmatic (structural) features of objects (e.g., the size, shape, and rigidity of the handle) and the motor capacities of the biological effector of an organism interacting with the object (e.g., the size, strength, and agility of fingers), as well as by the situation at hand. Indeed, affordances do not represent object features per se, but processes that emerge in the interaction between the object and the effector: Affordances are interactive, variable and dependant on the current action context (e.g., a cup handle affords hefting by a human, but not by a dog).

In the current context, one important question concerns whether perception of (and response to) affordances is a characteristic of the Use system, the Grasp system, or both. Several studies have investigated the Gibsonian idea that affordances do not require an activation of object knowledge, which, in our dichotomy, would suggest that affordances are largely computed by the Grasp system.

In our view, functional manipulation actions contrast with the classic notion of affordances. The latter may be activated without deep semantic processing whereas the former requires access to semantic knowledge. Imagine, for example, that a round object is speeding toward your head. You may instinctively reach up and grasp it accurately based on its shape and size (affordances for grasping), only then discovering that it is a ball. As we will detail below, consistent with this hypothesis the two types of information have different time courses of activation, with activation of structure-based information from visual objects preceding access to functional manipulation information. (There is evidence that the opposite time course may be observed in the case of words, c.f. Bub, Masson, & Cree, 2008.) Clearly, however, under most everyday circumstances, the two types of information must be integrated to enable appropriate interactions with objects. Creem and Proffitt (2001), for example, found that participants grasped everyday objects with handles (e.g. combs and paintbrushes) differently when performing a concurrent semantic interference task, but not during a spatial interference task. This suggests that grasping objects with the intent to use them appropriately requires integration between conceptual knowledge and affordances derived from objects (Buxbaum, Sirigu, Schwartz, & Klatzky, 2003).

5. Differences in processing characteristics of the use and grasp systems

The evidence discussed earlier in this review provides neuropsychological and functional neuroanatomic evidence for the existence of two action systems. In the following section, we review evidence providing additional detail about the types of information each system processes, and the temporal characteristics of this processing.

6. Time course of grasp and use processing

A number of lines of evidence suggest that processing in the Grasp system is relatively evanescent, maintaining information for milliseconds to seconds (Cant, Westwood, Valyear, & Goodale, 2005; Garofeanu, Króliczak, Goodale, & Humphrey, 2004; Milner & Goodale, 2008). The short duration of this information makes sense considering the system’s specialization for grasping objects based on current visual information. The Use system, on the other hand, subserves conceptual knowledge about functional actions (Buxbaum & Saffran, 2002) and maintains information over longer periods of time.¹

The hypothesized differences in the processing characteristics of the two action systems are supported by a recent study that measured participants’ initiation times to act on objects that are picked up and used with different actions (‘conflict objects’, e.g., calculator) or objects that are picked up and used with the same actions (‘non-conflict objects’, e.g., cup) (Jax & Buxbaum, 2010, see also Klatzky, McCloskey, Doherty, Pellegrino, & Smith, 1987). Initiation times for function-based actions were slower for conflict objects than non-conflict objects, implicating interference from structure-based action attributes. For example, as Fig. 4 shows, initiating movement for using a calculator with a “poking” action was slowed by the task-irrelevant activation of the clench action required to grasp the calculator (within-object grasp-on-use interference). In contrast, initiation times for structure-based actions were only slower for conflict-than non-conflict objects when participants had performed function-based actions on the same objects in earlier blocks. In other words, interference from function-based actions upon structure-based actions occurred only when the function-based actions had been activated previously, suggesting a comparatively slower pattern of activation and decay. Thus, the two types of object-related actions differ significantly in their patterns of temporal activation.

Fig. 4.

Initiation times to grasp or indicate hand posture for using common objects as a function of object type and task order. Grasping (left) is slower for conflict objects when preceded by the ‘use’ task than when not, indicating long lasting use representations. Use (right) is slower for conflict objects regardless of task order, indicating obligatory activation of grasp representations that is short-lasting (adapted from Jax and Buxbaum (2010)).

We interpreted these results in terms of a race between the activation of function-and structure-based actions wherein structure-based actions are rapidly elicited by objects but quickly degrading, whereas function-based activations are slower but maintained over an interval of at least several minutes, producing short-term interference effects. To our knowledge, these are the first studies to explore the temporal characteristics of interference between distinctly different, but extremely common, action representations evoked in a naturalistic context by single objects (see Bub et al., 2008 for related work).

These data have several implications. First, they indicate that single objects may evoke more than one type of action representation that may be in conflict; thus, the claim that object perception is associated with activation of motor information must be tempered to specify which type(s) of motor information is at issue. Second, the data show that these two types of action representation have different temporal characteristics, likely as a function of the action system from which they emanate. The long-lasting interference observed with functional representations is similar to that typical of semantic memory (e.g., Damian & Als, 2005). Thus, unlike structural representations, which are rapidly decaying and computed de novo based on the position and size of the object with respect to the viewer, functional use information has characteristics that make it a likely component of distributed object representations.

Our conceptualization of the differences between the representations computed by the two action systems is similar to the concept of stable and variable affordances introduced by Borghi and Riggio (2009). On this account, Stable affordances are considered to emerge from invariant features or properties of objects that can be incorporated into an object representation and stored in memory. For example, we know that cherries are graspable with a precision grip. The same can be true for a pencil, although depending on its orientation and our action intention, it can be grasped with a power grip as well. This does not mean that the property of size is a stable affordance, but that there is a greater probability that size will lead to the emergence of stable affordances than for instance the more variable property of orientation. Variable affordances, in our view, emerge from temporary object characteristics, such as the current orientation of a handle on an object, and are linked to the current actions about to be performed. However, it is important to note that orientation for instance can also lead to the emergence of stable affordances as we typically observe and interact with objects in a given orientation. For example, we observe and use bottles upright rather than upside down. Thus, we do not consider stable and variable affordances as being strictly dichotomous. Rather, we see them as arranged along a continuum.

Thus, the fact that information in the grasp system decays rapidly does not necessarily imply that there is no stored representation (or stable affordance) of how to grasp an object in order to pick it up. Certainly, when asked how to pick up a hammer, one is able to mime a generically-appropriate grasping action even when the object is not in sight. (Of course, the detailed parameters of the size of the grasp aperture and angle or the wrist, etc., must be arbitrarily selected under these “pantomime” conditions.) A more difficult question is whether information about how to pick up an object is an intrinsic aspect of the object’s conceptual representation. One of the ways in which this question has been answered is to assess the “automaticity” of action activations when they are irrelevant to a conceptual task, a topic to which we turn next.

7. The automaticity debate

Many aspects of cognition appear to be grounded in our experiences of the world based on reactivation or resonance of sensory and motor information. Thus, cognition may be “embodied”. Some of the oft-cited evidence in this area comes from experiments in which sensory or motor information is implicitly and automatically activated in tasks in which it is irrelevant, such as object, word, or sentence recognition. For present purposes, an interesting and informative debate concerns the aspects of object-related actions that may be activated “automatically” when we perform such conceptual tasks. Of course, it is quite plausible (and even probable) that different information is activated as a function of stimulus, task, and context, so it is important to be specific.

A number of influential studies have purported to show automatic activation of action information, but have assessed this activation by way of facilitation of or interference with prepared motor responses. For example, in a seminal study, Tucker and Ellis (1998) asked participants to decide whether objects with handles were upright or reversed. They found a compatibility effect between the location of the handle (left or right, not relevant to the task) and the position of the response key (left or right). The claim was that seeing an object with a handle on the left or on the right activated object affordances, leading to the facilitation of responses with the ipsilateral hand. Unfortunately, however, the matter is a bit more complicated. There is evidence that preparing motor responses can direct attention to regions of space congruent with the motor response, a phenomenon known as “motor-visual priming” (e.g., Craighero, Bello, Fadiga, & Rizzolatti, 2002; Craighero, Fadiga, Rizzolatti, & Umiltà, 1999). Thus, preparation of a motor response with the right hand might draw attention to object features on the right side of space, thus facilitating responses to right-sided object features. In a paradigm like that used by Tucker and Ellis (1998), visual object features such as handles may appear to prime a motor response, but actually, the preparation of the motor response (for example, to a right-sided button) facilitates attention to congruently-oriented (in this case, right-oriented) handles, resulting in a compatibility effect (see Botvinick, Buxbaum, Bylsma, & Jax, 2009; Buxbaum & Kalénine, 2010 for discussion). This is quite different than a scenario in which handles “automatically” activate congruent responses.

For the present, one of the most potent debates concerns the action information that may be activated when hearing auditory words. On the one hand, it might be claimed, for example, that hearing and understanding a word such as “hammer” activates all actions associated with hammers, including how they are picked up and swung (i.e., both Use and Grasp actions). At the other extreme, it might be the case that the word “hammer” can be understood without necessarily accessing any of the motor information associated with hammers. A final, hybrid possibility is that only some types of action information are activated.

The last possibility comports well with the 2 Action System account. A number of lines of evidence, noted earlier, indicate that Use action representations tend to be localized more inferiorly in the parietal lobe (in the IPL and MTG) than Grasp actions, which tend to have a predominant focus in the IPS (Binkofski, Buccino, Posse et al., 1999; Binkofski, Buccino, Stephan et al., 1999; Boronat et al., 2005; Chao & Martin, 2000; Handy et al., 2003; Johnson-Frey, 2004; Kellenbach et al., 2003)). In fact, the locus of Use action representations is relatively close, neuroanatomically speaking, to other types of semantic representations concerning manipulable objects, which tend to be localized to the middle and inferior temporal lobes. Moreover, as noted earlier, once activated, Use information is relatively persistent, a characteristic that likens it to other types of semantic information. Based solely on neuroanatomic proximity and persistence of information, then, one possibility is that only Use, but not Grasp actions are intrinsic to words’ conceptual representations.

An elegant series of studies by Bub, Masson, and colleagues used combinations of pictorial and verbal materials to assess function-based and structure-based activations as measured by priming effects; specifically, the degree to which picture or word identification is facilitated by the programming of congruent actions. Subjects heard object names either in isolation or in sentences with non-manipulation verbs. (‘The young scientist looked at the stapler’; ‘Jane forgot the calculator’.) As far as subjects knew, the object names were incidental to the task, which was to grasp a manipulandum as signaled by a particular hand posture cue. Under these circumstances, there was evidence that the words and sentences activated use but not grasp actions (Masson, Bub, & Newton-Taylor, 2008). On the other hand, when object names were presented in sentences with function-based or structure-based verbs (‘picked up’, ‘used’), both function-based and structure-based actions were primed (Bub & Masson, 2010).

Recently, Bub and Masson (2011) demonstrated that structure-based actions may indeed be evoked weakly from verbal materials even without the provision of verbs, but only at specific time points in processing, a point to which we will return in the following section. Note again, however, that their method required the preparation of a manual response, which may bias attention to action-based attributes of the words. Finally, several studies using real or pictured objects, rather than words, have reported task-incidental activation of both function-based and structure-based actions (e.g., Bub et al., 2008; Jax & Buxbaum, 2010), though these studies also required manual object-relevant responses.

Viewed together, the evidence suggests that in the absence of a prepared manual response, function-based actions may be evoked either from object names or visual images. Structure-based actions, on the other hand, are evoked upon visual object presentation, but are not likely to be activated from object names unless an appropriate verb (e.g., “pick up”) is provided or an object-relevant manual response is prepared. If correct, this suggests that word meaning derived solely from the name of a manipulable object (in the absence of a relevant visual stimulus or manual response) may include only function-based but not structure-based actions.

Most interesting, however, is the possibility that context acts to weight action and non-action components of word meaning. Thus, in the context of a manual action task, a heard word may be comprehended in part via activation of action features. In situations without a – congruent manual response task, such as most everyday listening, we may hypothesize that words may be “understood” without necessarily entailing action activation. The challenge of such accounts, of course, is precise specification of the context(s) under which action activations may or may not occur. Although little work has been done to assess the role of context in action activations when the stimuli are words, there is a rich history of study in this area when the stimuli are objects. It is to this topic that we turn next.

8. Role of context

A number of studies have demonstrated that action preparation may facilitate the processing of targets and distractors congruent with that action (e.g., Allport, 1987, 1989; Bekkering & Neggers, 2002; Botvinick et al., 2009; Craighero, Fadiga, Umiltà, & Rizzolatti, 1996; Craighero et al., 1999; Hannus, Cornelissen, Lindemann, & Bekkering, 2005; Pavese & Buxbaum, 2002; van Elk, van Schie, Neggers, & Bekkering, 2010). Consistent with the premotor theory of attention (e.g., Rizzolatti, Riggio, Dascola, & Umiltá, 1987), several recent studies suggest that action preparation results in modulation of visual attention through common neural mechanisms underlying both action and attention (e.g., Gutteling, Kenemans, & Neggers, 2011; Neggers et al., 2007).

In a recent study, we (Lee, Middleton, Mirman, Kalenine, & Buxbaum, 2012) hypothesized that verbal context may act similarly to motor preparation in driving attention to action-relevant features of objects. Action verbs or sentences have been associated with activations in primary and/or pre-motor regions (Buccino et al., 2005; Hauk, Johnsrude, & Pulvermüller, 2004; Jirak, Menz, Buccino, Borghi, & Binkofski, 2010; Pulvermuller, 2005; Raposo, Moss, Stamatakis, & Tyler, 2009 for meta-analysis). Processing object names has been shown to elicit similar motor and pre-motor area activations (Grafton, Fadiga, Arbib, & Rizzolatti, 1997; Rueschemeyer, Lindemann, van Rooij, van Dam, & Bekkering, 2010). Given this, and the findings of Jax and Buxbaum (2010), we wanted to assess whether differing verbal contexts would effect the time course of processing in the Use and Grasp systems. Participants heard either an action verb sentence (‘s/he picked up the…’ ‘s/he used the…’) or an action–neutral sentence (‘s/he saw the…’) followed by a manipulable noun, which served as the target of visual search. A Visual World Paradigm was implemented: targets were presented with distractor objects overlapping in either use or grasp features. We assessed the time course of competition between targets and distractors by measuring eye gaze. Distractors sharing grasp features with the target caused earlier and more short-lived competition than did distractors sharing use features. Moreover, relevant action verb contexts caused significant changes in the patterns of competition between targets and distractors, primarily by moving the competition earlier in processing. Thus, Use or Grasp action contexts serve to shape the time course of our attention to objects in an array (Fig. 5).

Fig. 5.

Time course of gaze fixations on competitor objects (dashed line) and unrelated objects (dotted line) when locating target objects (solid line) in “Saw” and “Picked up” verbal contexts. In the ‘picked up’ context (right), competitor activation is shifted earlier, suggesting facilitation of processing of competitor objects that are picked up similarly to the targets. Adapted from Lee et al. (2012).

9. Conclusion

Our understanding of object-related action has evolved considerably in the last several years. Two parallel streams in the parietal, temporal, and frontal lobes process distinct aspects of object information relevant for action. The dorso-dorsal system, which we have here characterized as the “Grasp” system, processes structural characteristics of particular exemplars of currently-viewed objects (e.g., shape, size, and orientation) for the purposes of prehensile actions. The ventro-dorsal stream, which we characterize as the “Use” system, is concerned with long-term storage of the particular skilled actions associated with familiar objects. Typical everyday actions require precise coordination between the systems. Current research is devoted to uncovering the mechanisms of coordination, the time course of information processing in each system, and the role of context and goals in determining the salience of information processed by each stream.