Moving the gesture engram into the 21st century

Professor Georg Goldenberg is to be commended for providing a detailed and thoughtful summary of much of the body of work relevant to apraxia. His contribution to the field has been sustained and important, raising the profile of a fascinating and sometimes perplexing disorder. In that context, there are a number of areas in which he seems unduly influenced by older “box and arrow” models of cognition and action, perhaps failing to fully appreciate the implications of recent research relevant to the distributed architecture of the action semantic system. In this commentary, I will first summarize two of Goldenberg’s central premises, and then briefly review recent evidence that permits us to reject them.

Goldenberg claims that there are “no fixed associations between tools and the manual actions of their use” (Goldenberg, 2013, p. 126) because there is great variability in performance across instances of use of a given tool, and thus “no firm fundament for the storage of manipulation knowledge…” (p. 124). He notes, however, that there is evidence for “general functional knowledge” that a) includes the typical actions associated with tools, such as knowledge that a hammer’s use “consists of powerful strokes” (Goldenberg, 2013, p. 125), b) takes into account the relative frequency and familiarity of actions associated with tools, c) permits generalization across different variants of a given tool, and d) is used in extended, complex tasks, along with stored scripts or schemas that “represent what is shared by different instances of a multistep action but leave open ‘slots’ for filling in specification of objects or actions that are specific to individual applications of that schema” (p. 143). Goldenberg assumes that motor information plays no role in any of these attributes of “general functional knowledge” but rather seems to imply that this knowledge has a verbal/propositional format. He does acknowledge, however, that motor representations may play a role in “special” cases requiring “specialized patterns of motor coordination like skilled typewriting” (Goldenberg, 2013, p. 125). (This arbitrary distinction between ordinary and special cases of action bears discussion in its own right given that even “simple” tool-use is a learned motor skill, too—see e.g., Kahrs, Jung, and Lockman (2013) – but lack of space precludes that discussion here).

The second questionable premise is that production and recognition of tool-use pantomime relies upon information entirely independent from that required for actual tool-use, because “the range of actions that we know and understand is much larger than the range of actions our motor system can execute” (Goldenberg, 2013, p. 183). Specifically, pantomime requires “selection and combination of distinctive features extracted from a mental image of tool use” based on “some knowledge about the action or, respectively, some memory of having witnessed this action before” (Goldenberg, 2013, p. 183). Again, the format of these “mental images” is not specified, but one can infer that they are meant to be visual and/or verbal (in any event, not motor). With respect to recognition, Goldenberg views as “awkward” the idea that a supplementary mechanism would be needed for understanding actions that are not in our motor repertoire (Goldenberg, 2013, p. 183).

In response to these claims, I’ll briefly review some recent evidence about how the brain organizes semantic information (including stored action knowledge). The data I review are consistent with the basic premises that: 1) knowledge representations are distributed and graded, exhibiting visual, motor, auditory, and/or tactile properties as a function of mode(s) of acquisition, current network states, task demands, and location in the brain (e.g., Plaut, 2002), and 2) brain regions involved in the representation of knowledge are the same regions that were involved in acquiring the information (e.g., Allport, 1985).

A large number of studies show that knowledge of manipulable objects depends in part on spatial and motor processing regions in the frontal and parietal lobes (e.g., Beauchamp & Martin, 2007). Moreover, apraxics with parietal lesions are less accurate than non-apraxics on semantic judgments about tools, but are more accurate than non-apraxics on semantic judgments about animals (Buxbaum & Saffran, 2002). Consistent with this finding, transcranial magnetic stimulation (TMS) of the left inferior parietal lobe (a common site of apraxia-inducing lesions) delays participants’ ability to name manipulable objects but not non-manipulable objects (Pobric, Jefferies, & Lambon Ralph, 2010). Using eyetracking, we have also shown that implicit competition between objects used with similar actions is slower and more attenuated in apraxics than non-apraxics (Lee, Mirman, & Buxbaum, Submitted for publication; Myung et al., 2010). This pattern holds even when the structural (3-dimensional shape) similarity of similarly-used object pairs is deliberately low (e.g., target = spray can, distractor = camera) and matched with the structural similarity of object pairs that are not used similarly (Lee, Middleton, Mirman, Kalenine, & Buxbaum, 2013; Lee, et al., Submitted for publication; and see Campanella & Shallice, 2011; Helbig, Graf, & Kiefer, 2006; Kiefer, Sim, Helbig, & Graf, 2011; Myung, Blumstein, & Sedivy, 2006 for related data from healthy participants).

However, despite our previous demonstrations that functional knowledge is spared (and manipulation/use knowledge impaired) in patients with apraxia (Buxbaum & Saffran, 2002; and see Boronat et al., 2005; Canessa et al., 2008 for related findings), it might be argued that many of these effects derive from what Goldenberg would term “general functional” rather than motoric information. Yee, Chrysikou, Hoffman, and Thompson-Schill (2013) recently reported data that cannot be reconciled with that assertion. Yee et al. showed that semantic judgment and naming tasks with object words and pictures of objects were reliably disrupted by the performance of a concurrent, unrelated motor task (playing a hand-clapping game), and furthermore, that this disruption was modulated by how much experience participants had manipulating those objects. Disruption of the same semantic judgments by a concurrent visual task was not modulated by manipulation experience (see Witt, Kemmerer, Linkenauger, & Culham, 2010, for a similar result). These data add further credence to prior studies that have shown that motor-region activity during access to tool concepts varies with motor experience (e.g., Creem-Regehr, Dilda, Vicchrilli, Federer, & Lee, 2007; Kan, Kable, Van Scoyoc, Chatterjee, & Thompson-Schill, 2006; Kiefer, Sim, Liebich, Hauk, & Tanaka, 2007; Weisberg, van Turennout, & Martin, 2007). The data are also consistent with data from Bub, Masson, and Cree (2008), showing that responding to a manipulable object by making a gesture inconsistent with that object’s typical use action causes interference. Thus, accumulating evidence indicates that the actions associated with manipulable objects are stored in a motor format.

Importantly, this does not imply that motor information is the only aspect of action representations. If I have never used a hammer, but have seen others use them, then my representations of the actions associated with hammers may be primarily visual. If, on the other hand, I have used a hammer (as well as having seen hammering actions), then my hammer-action representations will be distributed across regions subserving manual actions, somatosensory processing, and vision (see Bellebaum et al., 2013; Connolly, Gleitman, & Thompson-Schill, 2007; Hoenig et al., 2011). Moreover, across the many instances of the hammering actions that I see (and/or do), a prototypical hammering representation emerges even if I have never seen or produced a prototypical hammering action. This description of typical exemplars that are shaped by repeated instances of actions is consistent with typicality effects (e.g., Rosch & Mervis, 1975) that emerge in PDP computational models that learn semantic structure on the basis of featural overlap of exemplars (e.g., Rogers & McClelland, 2004). Of course, the multimodality of action representations does not preclude the option of bringing mechanical problem-solving to bear on action tasks, as well. In action tasks, as in other domains, the brain makes flexible use of multiple sources of relevant information (Vingerhoets, Vandekerckhove, Honore, Vandemaele, & Achten, 2011).

Consistent with the claim that tool-related action information (i.e., the gesture engram) is distributed, we recently presented data (Buxbaum, Shapiro, & Coslett, 2013; Buxbaum, Shapiro, & Coslett, Under revision showing that correct performance of the postural (arm and hand positioning) aspects of tool-use pantomimes depends upon the integrity of posterior temporal-occipital regions, an area with known preference for coding tool-use motion (e.g., Kable, Kan, Wilson, Thompson-Schill, & Chatterjee, 2005; Kable, Lease-Spellmeyer, & Chatterjee, 2002) and for recognizing actions presented visually (Kalenine, Buxbaum, & Coslett, 2010). On the other hand, correct performance of the kinematic aspects of the same tool-use actions (movement amplitude and timing) is dependent upon frontoparietal cortex, a region with broad relevance for spatiomotor production, as well as for recognition and prediction of kinematic parameters of body movement (Gallivan, McLean, Valyear, & Culham, 2013; Kalenine, et al., 2010). Far from being “awkward”, representational distribution increases the resilience of representations in the face of brain damage (see Yee, et al., 2013 for a similar argument). Additionally, graded action representations that traverse multiple brain regions and multiple modalities may explain why multimodal information (tactile, kinesthetic, and visual) provided in the case of actual tool-use with recipient objects benefits (but does not completely normalize) apraxic performance, as Goldenberg’s own work attests (e.g., Hermsdorfer, Li, Randerath, Goldenberg, & Johannsen, 2012; Randerath, Goldenberg, Spijkers, Li, & Hermsdorfer, 2011). Rather than a “cognitive” or “motor” disorder, apraxia perfectly reflects the brain’s propensity to encode multimodal, graded representations of action knowledge.