Disembodying cognition

Abstract

The idea that concepts are embodied by our motor and sensory systems is popular in current theorizing about cognition. Embodied cognition accounts come in different versions and are often contrasted with a purely symbolic amodal view of cognition. Simulation, or the hypothesis that concepts simulate the sensory and motor experience of real world encounters with instances of those concepts, has been prominent in psychology and cognitive neuroscience. Here, with a focus on spatial thought and language, I review some of the evidence cited in support of simulation versions of embodied cognition accounts. While these data are extremely interesting and many of the experiments are elegant, knowing how to best interpret the results is often far from clear. I point out that a quick acceptance of embodied accounts runs the danger of ignoring alternate hypotheses and not scrutinizing neuroscience data critically. I also review recent work from my lab that raises questions about the nature of sensory motor grounding in spatial thought and language. In my view, the question of whether or not cognition is grounded is more fruitfully replaced by questions about gradations in this grounding. A focus on disembodying cognition, or on graded grounding, opens the way to think about how humans abstract. Within neuroscience, I propose that three functional anatomic axes help frame questions about the graded nature of grounded cognition. First, are questions of laterality differences. Do association cortices in both hemispheres instantiate the same kind of sensory or motor information? Second, are questions about ventral dorsal axes. Do neuronal ensembles along this axis shift from conceptual representations of objects to the relationships between objects? Third, are questions about gradients centripetally from sensory and motor cortices towards and within perisylvian cortices. How does sensory and perceptual information become more language-like and then get transformed into language proper?

1. Introduction

Perhaps the mind-body problem has collapsed on itself. It is now commonplace to argue that the mind is embodied (Barsalou 1999; Fischer and Zwaan 2008; Glenberg and Robertson 2000). Or put in cognitive neuroscience terms, sensory and motor neuronal activity grounds cognitive processes. Recently, this general framework has been bolstered significantly by the mirror neuron hypothesis (Cattaneo and Rizzolatti 2009; Rizzolatti and Craighero 2004), which could be considered one neural instantiation of embodied cognition. According to the mirror neuron hypothesis much of our perceiving and thinking is embodied in activity within neuronal ensembles that implement motor output. There are several versions of embodied cognition (Clark 1999; Wilson 2002). Some refer to the way that cognitive systems evolved to support actions in specific situations. Others emphasize the integration of perception and action, and how the body interacts with the environment. However, most accounts of embodied cognition in psychology and cognitive neuroscience focus on the role of simulation. Simulation is the process by which concepts re-evoke perceptual and motor states present when perceiving and acting in the world. In this paper I focus on embodied cognition as a form of simulation.

The popularity of embodied cognition as an organizing principle is evident in the dramatic rise of publications since the mid 1990s. This rise can be seen if one searches ‘embodied cognition’ or ‘mirror neurons’ as key words (Figure 1). A quick survey of the titles of papers using these terms suggests that this organizing principle extends not only to different domains within psychology such as language, memory, emotions, time perception, and decision making, but also generalizes to development, social cognition, evolutionary biology, education, robotics, autism, psychopathology, dance, art history, art therapy, and even mind reading.

Figure 1.

These graphs show the results of a search on Science Direct for the number of publications with ‘semantics’, ‘language’, ‘embodied cognition’ and ‘mirror neurons’ between 1995 and 2009. The publication rate for ‘semantics’ has been relatively stable. In the last decade, publication rates have tripled for ‘language’ and have grown exponentially for ‘embodied cognition’ and for ‘mirror neurons.’

Perhaps the widespread endorsement of embodied cognition, bolstered by neuroscience, reflects the fact that the mind sciences have discovered their fixed fulcrum on which deep questions about cognition can be lever-aged. Before accepting this conclusion, a critical look at theories of embodied cognition is warranted. Despite the current wave of enthusiasts, a few such critiques have begun to surface (Dove 2009; Mahon and Caramazza 2008; Talmy in press). The present paper, which might be considered a critique, should be placed in its proper context. The paper is titled disembodying cognition and not disembodied cognition. Debates about whether cognition is embodied or disembodied, in my view, have outlived their usefulness. Those debates were specific to controversies embedded in certain periods of psychology and cognitive science (Barsalou 1999) and had a limited hold within the neurosciences. As Barsalou (1999, 2007) pointed out, some form of embodied cognition has been the historic norm in thinking about mental processes. Investigating the either/or of embodied cognition in my view would be more productively replaced by investigating the when, how much, and in what way of embodied cognition.

My strategy here is to examine behavioral and neuroscientific findings that are used as evidence in support of embodied cognition. I make no attempts to be comprehensive in this examination, which would be a daunting task as suggested by the number of publications in Figure 1 (for recent reviews, see Barsalou 2007; Fischer and Zwaan 2008; Kaschak et al. 2009). Rather, I will select characteristic examples from the literature. Among simulation versions of embodiment, there are ‘strong’ and ‘weak’ versions of embodiment. The thrust of my proposal could be regarded as supportive of a weak embodiment. I will suggest that concepts do not evoke a richly textured recreation of perceptual or motor experiences, but rather highlight very selective and attenuated perceptual and motor attributes. I prefer to frame ‘weak’ embodiment as a form of ‘graded ’ grounding. Referring to graded grounding invites consideration of continua and trade-offs between what is lost and what is gained. Representations by virtue of being less grounded in sensory and motor details lose some of their referential power. But, by virtue of being less grounded they also gain generative and flexible power.

In this discussion, I focus on conceptual systems devoted to knowledge of spatial thought and language. Spatial thought and language serves as an intermediate domain of inquiry between concrete objects and abstract concepts. Embodied cognition accounts lend themselves more easily to conceptualizing concrete objects and face special challenges in conceptualizing abstract ideas (like democracy) (Dove 2009). Spatial thought and language, for reasons that I will describe later, serves as a system that lends itself to asking questions about gradations of sensory-motor grounding in cognition.

In reviewing characteristic findings from the literature, I suggest that knowing what these findings mean is often far from clear. Claims of findings consistent with embodied cognition often lack specificity. Furthermore, an easy acceptance of embodied interpretations often obscures other questions. Following this review of characteristic findings, I will outline work from my lab on the relationship between space and language. This outline again makes no attempt to be comprehensive of the topic in general (Chatterjee 2008; Kemmerer 2006). Other literature will be mentioned where relevant. Discussing these findings raises more questions than it provides answers. The questions raised are about the graded nature of grounding in sensory motor representations and how sensory and motor information might be bleached in the process of thinking more abstractly. I also point out that the influence of analog factors on conceptual processing (often regarded as prima facie evidence for embodiment) does not necessarily arise from typical views of embodiment. Finally, I offer some general functional-anatomic principles to consider in disembodying cognition.

I take as axiomatic that exquisitely developed sensory and motor systems are not sufficient to embody complex human cognitive capacities. Otherwise animals with demonstrably more acute sensory systems and more agile motor systems than humans would be expected to have minds more subtle and sophisticated than those of humans. If the neural substrate for exquisitely well-developed sensory and motor processing is not sufficient as a vehicle for much of human cognition, is it necessary? The questions for embodiment can be framed as: when is it necessary, when is it important, and when is it epiphenomenal? Furthermore, what exactly do we mean by sensory-motor grounding?

2. Behavioral evidence for embodied cognition

Behavioral data in support of embodied cognition accounts emphasize interactions between sensory or motor variables and conceptual processing (Glenberg and Robertson 2000; Barsalou 1999; Fischer and Zwaan 2008). Sensory-motor variables modulate conception, and conception modulates sensory-motor processing. The inference from these findings is that the body in the sense of sensory and motor properties plays a prominent role in structuring concepts.

A typical description of the interaction of perceptual and conceptual representations comes from Richardson and colleagues (Richardson et al. 2003). They found that comprehension of verbs has specific effects on perception. Verbs like lift typically describe movement along the vertical axis and verbs like push describe movement along a horizontal axis. Processing vertical movement verbs affect people’s abilities to discern shapes at the top and bottom of computer screens and processing verbs with horizontal movements affect people’s abilities to discriminate shapes at the left or right of screens. They found similar axis effects in a task in which participants had to remember pictures. Other experiments show that motion words affect participants’ abilities to detect visual motion (Meteyard et al. 2007; Zwaan and Taylor 2006) and analogously processing visual motion in specific directions interferes with processing words (Meteyard et al. 2008) and sentences describing events moving in the same direction (Kaschak et al. 2005).

Studies reporting interactions of motor and conceptual representations follow the same logic as the ones reporting perceptual and conceptual interactions. Objects that predispose a viewer to adopt different hand postures, such as precision or power grips (grape versus hammer), affect motor responses on unrelated tasks (Tucker and Ellis 2004). These and similar studies (Bub et al. 2008; Glover et al. 2004; Symes et al. 2007) suggest that simply viewing objects like cups in different orientations or words describing these objects automatically activate motor systems in a way that simulates interactions with them. Further evidence that simulated actions are part of the meaning of words is found in reports of interference effects by Borghi and colleagues (Borghi et al. 2004). In this study, participants read a sentence like There is a car in front of you and then pressed a middle button on an array of 3 buttons arranged vertically. Pressing this button revealed a target word, roof, wheel or road and the participants decided if the target was part of the object, in this case car. Assuming that interacting with the roof requires moving upward and interacting with the wheel requires moving downward one would expect that responding ‘yes’ would be affected by an interaction between the target being roof or wheel and whether the participant had to move to the upper or the lower button. They would be faster responding to roof if the response key was upper rather than lower, and faster to wheel if the response key was lower rather than upper. This is the pattern of results that they found. Thus, the meaning of the word seems linked to the way we act on objects in the world. Similarly, Glenberg and Kaschak (2002) found that judgments on sentences like Courtney handed you the notebook or You handed Courtney the notebook, were influenced by whether participants moved towards or away from their own body in making their responses. Again, one infers that comprehending these sentences involves simulating the motor behaviors being described. The motor effects in language comprehension can be quite precise in time. Zwaan and Taylor (2006) have shown that motor effects occur precisely when participants encounter action words or relevant adverbial adjuncts as they read sentences and not after. The general view of these kinds of experimental results is that words evoke analog perceptual and motor memories associated with real world referents for these words. This evocation of sensory and motor information is a form of simulation that constitutes the meaning of the word (Kaschak et al. 2009).

The many replications of perceptual and action comprehension compatibility effects across different kinds of experiments (Kaschak et al. 2009) leave little doubt about the veracity of these empirical findings. However, the best explanation for these effects remains in doubt. What does it mean to have quicker responses on the order of 10 to 100 milliseconds? There is a long history of chronometric investigations in experimental psychology (Posner 1986). However, in the kinds of experiment reviewed here, establishing that response delays follow from interference or facilitation at the level of semantic analysis is not so easy. Clearly, no one predicts that asking participants to sit on their hands or confining their limb movements prevents them from comprehending these sentences. If motor implementation is not necessary for understanding, in what sense is it important? Are there alternate explanations for the behavioral results? Perhaps these interference effects producing RT differences might have nothing to do with the conceptualization of actions. Perhaps the actions are conceptualized separately and, as a downstream epiphenomenon, motor systems are activated. Since these tasks require specific movements, the interference producing RT differences might simply be occurring at the level of the response demands of the experiment. Similarly, as argued in detail by Mahon and Caramazza (2008), the interference could be occurring at a decision making level after critical semantic analysis. Again, to be clear, I am not claiming that the sensory or motor interference effects reported are necessarily epiphenomenal. I am simply pointing out that the behavioral findings by themselves do not answer the question of whether sensory and motor contributions to concepts under consideration are necessary, are important, or are epiphenomenal. For a thoughtful discussion of precisely this issue, see Taylor and Zwaan (2009).

3. Neuroscience evidence for embodied cognition

Embodied accounts of cognition are considered especially suited to neuroscience investigations (Simmons and Barsalou 2003). The major neurobiologic support for embodied cognition accounts can be found in Damasio’s time-locked multiregional activation hypothesis (Damasio 1989) and in the mirror neuron hypothesis (Rizzolatti and Craighero 2004). Damasio’s proposal is based on two ideas. The first is that fragments of sensory and motor attributes are recorded in early unimodal sensory and motor cortices. Second, convergence zones, which are often amodal neural structures, coordinate time-locked combinatorial activations of these fragments to be bound into entities or events. These regions (association cortices and convergence zones) have feed forward and feedback reciprocal interconnections. Damasio’s main point is that meaning is not stored in one location. Rather, the meanings of entities and events are bound together by time-locked neural activations that are distributed across geographically distinct cortical regions. Early sensory and motor activations are integral to the neural instantiations of this meaning.

The mirror neuron hypothesis was derived from observations that neurons in the F5 sector of the macaque prefrontal cortex that discharge when the monkey performed an action also discharge when the monkey observed similar actions (Rizzolatti and Craighero 2004; Cattaneo and Rizzolatti 2009). Neurons with similar properties were then identified in the inferior parietal lobule. These neurons may even differentially encode goals of actions, such as grasping to eat versus grasping to place an object (Fogassi et al. 2005). The general idea is that understanding actions of others, whether by observation of actions or through words referring to actions, involves discharging mirror neuron ensembles. Our understanding of actions is implemented in our motor systems.

In general, neuroscience evidence in support of embodied cognition comes from imaging, electrical (ERP and MEG), transcranial magnetic stimulation (TMS) and acquired lesion studies. Again, here I outline representative studies, without attempting to be comprehensive. The general point is that the evidence from neuroscience is not as strong as is often claimed. Furthermore, the data tend not be scrutinized sufficiently.

A characteristic fMRI study cited in support of mirror neuron versions of embodied accounts is by Tettamanti and colleagues (Tettamanti et al. 2005). Tettamanti found that participants listening to action sentences as compared to abstract sentences activated the pars opercularis of the left inferior frontal gyrus, premotor cortices, parts of the insula, as well as the inferior parietal lobule, the intraparietal sulcus and the posterior middle temporal gyrus (pMTG). The authors highlight the idea of action related sentences activating ‘fronto-parietal motor circuits’ in the title of the paper and the discussion emphasizes the frontal and parietal activations as consistent with the mirror neuron hypothesis. They minimize the importance of the pMTG activations. The reason for this bias is to argue that the data are consistent with the mirror neuron hypothesis, an argument made more easily with the fronto-parietal activations than for the pMTG activations. Of course, the pMTG activations might be indicative of visual motion activations, which would be consistent with a general embodied account not restricted to the mirror neuron hypothesis. This is a possibility to which I shall return in mentioning work from my own lab. The relevant point here is that these patterns of results might make one wonder about the relative contributions of visual motion or motor simulation in action concepts. But, rather than raising such questions, the data are taken as reifying the mirror neuron hypothesis version of embodied accounts.

The patterns of neural activity considered compatible with embodied cognition accounts often lack specificity. For example, is there a consequential difference between premotor or motor activation? Are these activations functionally different from activations found within the intraparietal sulcus? Are those activations in turn different from activations in the inferior parietal lobule? Note that Tettamanti and colleagues (Tettamanti et al. 2005) found activity in premotor, intraparietal sulcus and inferior parietal cortices for action sentences (in addition to the insula and pMTG). Are neurons in these different but interconnected regions functionally isomorphic, all doing the same thing, or are they doing different things that in a coordinated fashion give rise to a compound concept? Such basic questions do not seem to be pursued when data are taken as quick endorsements of embodied cognition.

Kemmerer et al. (2008) also looked at neural activation patterns when participants made semantic similarity judgments on 5 different kinds of verbs. These included verbs of running, speaking, hitting, cutting and changes of state. They found different topographic activations for these verb types along motor and premotor cortex. To their credit, they did not confine their investigations to motor variables. They also considered motion as a variable activating postero-lateral temporal cortex. Importantly, they also included an extended consideration of abstract properties of verbs (though these properties were not the subject of the investigation). For this discussion, the relevant point is that they conclude that their results are consistent with a simulation account of embodied cognition. However, a closer look at the data raises doubts about that conclusion. For example, hitting verbs activate areas dorsal to the areas activated by running verbs along the precentral gyrus. On any prediction based on somatotopic organization, one would predict hitting verb activations would be ventral relative to running verbs along motor or premotor cortex. The authors, despite including one of the most theoretically sophisticated treatments of verb semantics to be found in cognitive neuroscience, are willing to accept that the segregated pattern of activations found for these verb classes along premotor cortices is consistent with simulation of these actions, even though it is implausible that the actual use of legs and hands would produce this activation pattern.

Lesion studies might be considered to have greater inferential strength than imaging studies (Chatterjee 2005). Here, a deficit in sensory or motor systems would be expected to produce conceptual deficits. Again, certain obvious examples make clear that the strongest version of this hypothesis cannot be true. Individuals with pure motor hemipareses, a lacunar syndrome that causes paralysis by damaging descending motor fibers within the internal capsule or the pons, do not demonstrate conceptual deficits.

What is the neuropsychological evidence for embodied cognition? Neininger and Pulvermüller (2003) gave patients with right frontal or right temporo-occipital lesions lexical decision tasks using action verbs and visually related nouns. They found a significant group by word type interaction. Patients with right frontal lesions showed greater deficits with action verbs and those with right temporo-occipital lesions showed greater deficits with visually related nouns. They suggest that the lexical decision task draws on semantics, with action verbs drawing on motor information and the visually related nouns on visual information. These observations might be included within the family of category-specific deficits in naming or semantic judgments reported for animate and inanimate objects that have been subsumed under sensory-functional accounts of semantics (e.g. Martin et al. 2000; Warrington 1984). These kinds of data are extremely interesting and informative, but questions still arise. Firstly, the logic of looking at right hemisphere lesion patients is far from clear. This choice is predicated on the view that the right hemisphere instantiates certain aspects of semantics, a view by no means universally accepted. The authors acknowledge that a post-lexical source of the deficit could account for the pattern of behavior. But more notably, more than half of the ‘frontal’ patients also had damage to the posterior inferior parietal lobule and the superior temporal gyrus. Their ‘frontal’ population was by no means confined to patients with only motor or premotor lesions. Perhaps the role of the superior temporal gyrus in visual processing of biological motion could account for these differences. The critical sensory attribute might be different aspects of vision (e.g. biological motion versus shape and color-) and not whether motor or sensory systems are involved at all. A more precise way to test the anatomic hypothesis would be to use voxel-based lesion symptom mapping (Bates et al. 2003) techniques.

Other neuroscience experiments attempt to examine the role of the motor system in conceptual processing more directly. For example, Buccino and colleagues (Buccino et al. 2005) showed that listening to action related sentences modulates the motor system in a somatotopically specific way. Motor evoked potentials (MEPs) from hand areas were modulated by listening to hand action verbs and MEPs for leg areas were modulated by listening to leg action verbs. In both cases, the modulation was a decrease in MEPs. The authors interpret these results as the consequence of interference: simulating the meaning of these words activates the motor system in ways that are specific to the effectors, which then decreases the MEPs in response to TMS stimulation. But again, as argued earlier in describing the behavioral evidence for motor interference effects, how best to interpret these data is not clear. These motor activation interference effects may simply be occurring later at a downstream point after semantic analysis.

Stronger evidence for the role of motor systems in conceptual processing of actions would involve a manipulation of motor systems that produces effects on a conceptual task. With this strategy in mind, Pulvermüller and colleagues (Pulvermüller et al. 2005) applied TMS over left hemisphere motor areas while subjects made lexical decisions about action words that related to the hand (e.g. pick) or to the leg (e.g. kick). In this case TMS was used to facilitate rather than inhibit processing. They found a significant interaction between location of stimulation and reaction times to the kinds of action words on which lexical decisions were being made. These are among the strongest neuroscience data marshaled in support of embodied cognition. However, a closer look raises some doubts about the strength of these data. The ‘leg’ site stimulated is along the dorsolateral convexity of motor cortex, rather than deep in the medial frontal regions as would be expected from the somatotopic organization of the homunculus. In fact, the region described by the authors as the ‘leg area’ is typically considered a watershed zone between the middle and anterior cerebral artery distributions and strokes here produce weakness proximally in both the leg and the arm. Even if we set aside questions of whether the appropriate leg site is accessible by TMS, the data offer at best only partial support for the hypothesis being tested. Reaction times to arm words were virtually identical when the arm or the leg sites were stimulated. And stimulation of the arm site did not interfere with responses to arm words more than to leg words. At best, these data are suggestive of an embodied cognition view. I describe these data at this level of detail because they are cited to argue that the role of the motor system in conceptualizing action words cannot be epiphenomenal (Barsalou 2007). In my view, the data do not support such a strong position.

A comprehensive review of neuropsychological studies that bear on the role of motor systems and action conceptualization is not possible here. The data fall on both sides of the issue. Grossman and colleagues showed that patients with amyotrophic lateral sclerosis (a progressive disorder of motor neurons) have disproportionate difficulties with action words and that this difficulty correlates with atrophy of motor cortex (Grossman et al. 2008). Patients with weakness and lesions involving the motor systems (Moro et al. 2008; Serino et al. 2009) and patients with motor planning deficits (Pazzaglia et al. 2008) may have difficulty recognizing actions. In contrast to these studies that tightly link motor output, perception and conception, classic models of apraxia clearly distinguish production from recognition of skilled motor actions (Rothi et al. 1991). This classical model, according to which patients with apraxia need not have action recognition deficits has been confirmed in more recent studies with a relatively large number of patients (Johnson-Frey 2004; Mahon and Caramazza 2008; Negri et al. 2007).

Even if we allow the most generous interpretation of the neuroscience data supportive of the role of motor systems in conceptualizing actions, what the pattern of results means remains unclear. An alternate hypothesis might be that the role of motor systems contributes to how individuals differ in their understanding of actions, as opposed to representing a core attribute of the semantics of actions. A core attribute for a concept, if such a thing even exists, would be a necessary feature of the concept. Why might an individual differences hypothesis be plausible? Despite not knowing how to swim, I can watch Michael Phelps at the 2008 Olympics and have some understanding and even appreciation of the event. My understanding might be different than that of my 16 year-old competitive swimmer nephew. He might simulate some of the movements as he watches Phelps compete. Instead, I might watch the event and also ponder the social pressures on such a young man. The hypothesis that the degree of motor activation is a result of individual differences in motor experiences is consistent with fMRI data reported by Calvo-Merino and colleagues (Calvo-Merino et al. 2005). In their study, ballet dancers and capoeira experts watched video clips of ballet or capoeira movement segments. Ballet dancers activated pre-motor and intraparietal sulcus areas when watching ballet segments and not capoeira segments, and capoeira experts had the opposite pattern of activations. The authors suggest observations of the actions of others integrate with our own motor repertoire (also see Buccino et al. 2004 for similar conclusions). Note that the claim that motor simulation determines or is a core attribute of the understanding of actions is quite different than the claim that our experience of actions influences how we understand those actions. If one only tests able-bodied participants with actions of which every able-bodied person has frequent experience, like running or kicking or waving or wiping, one could easily conclude that simulation is a core attribute of understanding actions, even though the alternate hypothesis remains equally viable.

Recent studies support the hypothesis that personal experiences influence the neural response in the context of words describing actions. Beilock and colleagues (Beilock et al. 2008; Lyons et al. in press) found that ice hockey players have greater responses in premotor and caudate areas than non-hockey players when presented with sentences describing hockey actions. Similar differences were not seen with everyday actions. The idea that the degree of motor activations depends on different contextual factors is consistent with Taylor and Zwaan’s (2009) suggestion of a multi-variegated system that instantiates action concepts. The richness of the concept is in part determined by one’s experience of these actions. Importantly, in their view the system is tolerant of faults, that is, any one component, such as the activations of one’s own motor systems, need not be essential for comprehension.

In summary, the current popularity of embodied accounts of cognition has generated a wealth of extremely interesting data. Many of the experiments conducted are elegant. However, the best interpretation of these data is not always clear. The popularity of embodied accounts has also had some undesirable side effects. Within neuroscience, the popularity of the mirror neuron hypothesis has emphasized motor effects at the cost of possible perceptual contributions to conceptual understanding. In general, contrasting embodied with disembodied cognition diverts focus away from the question of the nature of this embodiment. Investigators are often permissive in accepting data as confirming embodied accounts. This permissiveness has the danger of obscuring alternate or more nuanced hypotheses.

4. Spatial thought and language

A major challenge for embodied cognition accounts is to address the question of how we abstract. Major proponents of embodied cognition are beginning to accept this challenge (Barsalou 2005; Glenberg et al. 2008) as I will discuss later, but much work remains to be done. Spatial thought and language offer a natural assay within which to examine the disembodying of cognition. Thinking about space gives our mental lives depth and texture by uncovering relational thinking and levels of abstraction inherent in spatial language (Gentner 2003; Gentner and Loewenstein 2002). Verbs establish thematic roles, such as who is doing what to whom in a sentence. Thus the verb ‘push’ implies that someone is doing the pushing and that something is being pushed. By coordinating the argument structure of a sentence, verbs organize a set of possible relations being communicated. Similarly, prepositions describe relationships of two or more objects. For example, the preposition ‘in’ implies two objects in a specific spatial configuration. This shift of focus away from concrete perceptual attributes of objects to their relations delivers enormous cognitive flexibility and generativity (Gentner 2003).

The link between spatial concepts and specific sensory-motor attributes is less clear than it might be for concrete objects. A lion and a child may be running, but which attributes contribute to ‘running?’ Such dynamic events are also transient. The perceptual referent cannot be returned to in the same way that one can return to a static object. Even for static locative relations the problem remains, in so far as the specific objects involved may vary widely. A cup may be on the table, or a monkey may be on a branch. These are very different visual percepts, and yet they share something in common. Mandler (2004: 251) notes, “Achieving this kind of abstract representation, one that ignores the concrete details of the objects involved may be required before spatial relations can be mapped onto language”. The mapping onto language itself involves another shift in level of abstraction. This shift (an analog percept to digital language conversion) is not specific to spatial language per se, but the evanescent nature of spatial events makes the conversion less straight-forward than might apply to making reference to stable concrete objects.

5. Selectivity of simulation

A prediction of strong versions of embodied accounts is that a concept activates the relevant perceptual and motor structures that are engaged by the actual experience of instances of that concept. Thinking about actions would activate the relevant sensory motor regions, such as those for the shape and size of objects engaged in the action, their motor systems as well as visual motion and spatial layouts within which the events occur. Our work on action representations suggests that such a fully articulated simulation is unlikely to occur routinely. Rather, the conceptualization of actions is grounded in highly selective attributes.

In an fMRI study, we presented participants with triads of stimuli and subjects matched one of two target stimuli to a test stimulus based on their similarity. For example, in the action condition, subjects might match a picture of digging to shoveling rather than sewing (Kable et al. 2002). Object trials showed pictures of static objects. Semantic judgments with pictures of actions more than those of objects activated an area bilaterally within posterior temporal-occipital cortex involving inferior and middle temporal gyrus. These areas included visual motion processing areas MT/MST (primarily within BA 37 and anterior BA 19). Thus, when people made semantic judgments of actions, they activated visual motion and adjacent areas even though the stimuli under consideration themselves were not moving. These findings are consistent with other reports that static images of actions activate area MT/MST (Kourtzi and Kanwisher 2000) and this general region appears to be involved in processing meaningful actions (Decety et al. 1997; Martin et al. 1995; Martin and Weisberg 2003). In this study, when people thought about actions, they activated a limited set of perceptual areas from the full ensemble that might have been activated when actually perceiving the same actions.

We interpreted the activation patterns produced by action pictures as being related to visual motion. Support for this view comes from a follow-up fMRI study (Wu et al. 2008) in which we examined whether activity in MT/MST could be related specifically to the manner component of motion events. If the nervous system organizes perception and conception along similar anatomic principles, then the way that language deals with motion might offer a clue to the perceptual parsing of motion as relevant to communication. Languages typically distinguish between manner and path of motion by expressing these attributes with different constituents (Talmy 1985). For example, in English, manner of motion is conveyed primarily by verbs. So, gallop, canter and trot describe different manners of motion. By contrast, path information in English is conveyed primarily by prepositional phrases. So, the horse gallops across the meadow or into the barn or around the track. We reasoned that path information, because of its locative nature, would activate brain regions more dorsally than would manner of motion. By contrast, since manner of motion is primarily related to the biomechanical properties of the object itself, it would be processed more ventrally. We (Wu et al. 2008) tested these hypotheses in an fMRI study in which a ‘star’ figure moved in different manners and along different paths, a technique that has been used in developmental studies (Pruden et al. 2004). Attending to manner or path of motion information produced distinct neural signatures. Greater activation for path than manner was seen bilaterally in the posterior intraparietal sulcus (IPS, BA 39/7) and posterior middle frontal gyrus (pMFG, BA 8/6). Greater activation for manner than path was seen bilaterally within the posterior middle temporal gyrus (pMTG) at the junction of BA 19/37.

In so far as these data are consistent with an embodied cognition account, the selectivity of the grounding in sensations is worth noting. These data suggest that visual motion rather than visual color or luminance or location is the relevant attribute with respect to processing actions. Furthermore, within motion, it is the manner and not the path of motion that seems most relevant to processing actions. While our data gives primacy to manners of motion in action conceptualization, the context in which actions are considered is likely to influence patterns of neural engagement. Thus, investigating the processing of action pictures when segregated by body parts might be more likely to activate premotor areas selectively (Kemmerer and Gonzalez-Castillo in press; Urgesi et al. 2006). Depicting action pictures in a spatial environment so that the path of motion can be inferred in relation to fixed external landmarks might be more likely to activate fronto-parietal cortices. The general point is that conceptualizing actions does not necessarily simulate a richly textured imagined event that recapitulates people’s actual experience of these actions in the world. Rather, selective sensory and motor attributes are highlighted in a multi-variegated (Taylor and Zwaan 2009) fashion that is likely to be dependent on the context in which actions are considered.

6. Disembodying actions

The action data reviewed suggest that the postero-lateral temporal cortex appears to be critical in mediating perception and conception of actions. But do these neural circuits abstract action representations away from the specific agents involved in the actions? An important step towards relational thinking is to shifting focus away from objects (Gentner 1988). Are neural structures within posterolateral temporal cortex designed to extract actions away from actors? To address this question, we conducted an fMRI adaptation (also known as repetition suppression) experiment (Kable and Chatterjee 2006). These experiments exploit the physiologic observation that neural responses in specialized circuits diminish when repeatedly processing features for which the circuit is specialized (Grill-Spector and Malach 2001).

Participants watched action movie clips and judged whether the specific action was common (typically seen at least once a week). They were first familiarized with a subset of the actions. Then four sets of movies were presented: (1) The same set of movies used in the familiarization phase (‘Old Actor, Old Action’), (2) a set of movies with the same people seen in the familiarization phase performing different actions (‘Old Actor, New Action’), (3) a set of movies with different people performing the same actions seen in the familiarization phase (‘New Actor, Old Action’), and (4) a set of movies in which both the people and the actions were different (‘New Actor, New Action’). We also identified the following areas functionally in each subject: The posterior superior temporal sulcus (pSTS), area MT/MST, the extra-striate body area (EBA), the lateral occipital cortex (LO), the fusiform face area (FFA) and the parahippocampal place area (PPA). The pSTS has been implicated in biological motion (Grezes et al. 2001; Grossman and Blake 2002; Martin and Weisberg 2003; Oram and Perrett 1994) and the EBA in processing visual images of human bodies (Peelen and Downing 2007). These regions along with area MT/MST would be candidate regions for processing actions. Area LO is implicated in non-specific static object perception (Kanwisher, Woods, Iacoboni and Mazziotta 1997; Malach et al. 1995), the FFA in face perception (Kanwisher, McDermott and Chun 1997) and the PPA in place and building perception (Epstein and Kanwisher 1998), and our expectation was that these regions would not be involved in action processing. Compared to completely novel events, we found decreases in the fMRI signal for sequences in which the action was repeated, but not the person performing the action, in the pSTS, MT/MST complex and EBA. Similar effects were not seen in LO, FFA or PPA (Kable and Chatterjee 2006). These results suggest that area MT/MST, EBA and pSTS are part of a distributed network that is sensitive to actions as extracted across different actors performing them.

These results are consistent with the general view that the posterolateral temporal cortex mediates action representations and also abstracts these representations away from the actors themselves. With respect to embodiment claims, it should be clear that the circuitry is specifically abstracting a pattern of movement that is divorced from the actors. We suggest that such shedding of object properties is a necessary step in one kind of abstraction, that of establishing relations. Relational thinking in the form of X pushes Y, or X is above Y, gathers flexibility by being referentially promiscuous. There are many possible referents for X and Y. Inserting possible referents for X and Y must mean that relational thinking bleaches out the sensory and motor details of specific actors and objects. Our data suggests that the neural underpinnings of this bleached action representation lies within the posterolateral temporal cortex.

7. Verbal and visual access to action concepts

How does the neural instantiation of action processing triggered by pictures relate to its verbal counterpart? Neuropsychological studies recognize that deficits in naming actions, and possibly conceptualizing actions, can dissociate from naming or conceptualizing objects (Berndt, Haendiges et al. 1997; Berndt, Mitchum et al. 1997; Cappa et al. 1998; Grossman 1998; Marshall et al. 1998). However, systematic analyses of the neural bases for these deficits are scant. In our study of action semantic judgments using picture triads (Kable et al. 2002) we also conducted an experiment using the comparable verb and noun word triads. We found that judgments of verbs compared to nouns activated an area just anterior and dorsal to the area activated by action pictures primarily in the left hemisphere. This area encompassed the pMTG and the pSTS. From the study we inferred that the lateral occipito-temporal lobe mediates different aspects of motion, with a gradient from the concrete apprehension of moving but meaningless stimuli in the inferior occipitotemporal junction (human MT/MST) to the motion implied by action verbs, closer to perisylvian cortex. Also see Pirog Revill et al. (2008) and Saygin et al. (in press) for recent related findings. This inference was predicated on the fact that words did not activate area MT/MST. To confirm that words did not activate MT/MST, we conducted a follow-up fMRI study using verb triads in which the association was based on manners of motion. For example, ‘hopping’ would be matched to ‘skipping’ rather than ‘running.’ Importantly, in this study, all of the choices referred to actions with motion that traverse space (Kable et al. 2005). Comparisons that focused on distinguishing manners of motion would be a more powerful test of the hypothesis that processing the meaning of action verbs would activate area MT/MST. Even with these stringent stimuli, we found that actions words did not activate area MT/MST. We replicated the previous findings of pMTG as well as pSTS activations. Thus, while the areas activated by action words were adjacent to areas specialized in processing visual motion, they were not identical to those areas (also see Fiez et al. 1996; Martin et al. 1995; Perani et al. 1999; Warburton et al. 1996 for related observations).

How do these results about the semantics of actions as accessed by pictures or by words relate to the nature of sensory-motor grounding? The obvious question is why there is a difference between the neural instantiation of action semantics as accessed through words and pictures. Perhaps Paivio’s (1990) dual (visual and verbal) coding hypothesis for semantics might be at play. However, a simulation account would predict similar activation patterns evoked by action pictures and words (regardless of the selectivity of the sensory attribute engaged).

Two differences in the pattern of our results beg explanation. Why do pictures engage posterolateral temporal cortices bilaterally and words mostly unilaterally? And why do words not engage area MT/MST? One possibility for the laterality question might be the neural instantiation of a type-token distinction. Perhaps the right hemisphere is sensitive to tokens of actions and the left to types. To my knowledge, this hypothesis has not been tested for actions. But the observation that the left, but not right, hemisphere responds to invariant views of objects (Simons et al. 2003) suggests a kind of abstraction within the left hemisphere. An invariant representation of an object would not be an evocation of the perceptual experience of seeing an object.

The reason why words do not activate area MT/MST, but do so just anterior to this region of motion processing is also not clear. This adjacency pattern of activation can be taken as support for or against the view that perceptual cortices are being activated by the concept. We proposed the idea that motion attributes are processed along the left lateral temporal cortex with a gradient in which concrete to abstract information is processed along a posterior to anterior (towards and within perisylvian cortex) axis (Chatterjee 2008). See Pirog Revill et al. (2008) and Saygin et al. (in press) for related findings. However, Bedny and colleagues (Bedny et al. 2008) replicated the observation that this area activates verbs more than nouns but found no evidence to support the idea that these activations have anything to do with motion. They speculate that activation in this region might have something to do with the grammatical category of verbs or with conceptual rather than perceptual aspects of event semantics (Wu et al. 2007). Whichever hypothesis turns out to be correct for the functional significance of neural activity in this area, this is the kind of detail at which neuroscience data needs to be examined to be informative of embodied accounts.

8. Relational thinking

As mentioned earlier, an important aspect of spatial language is its relational nature (Gentner 2003). For example, verbs in the context of sentences tell us who is doing what to whom, and prepositions tell us how one thing is related to another in space. Aphasic patients may have comprehension deficits at the level of an event, in which one participant is doing something to another (Berndt, Haendiges et al. 1997; Caplan 1995; Caramazza and Miceli 1991; Kegl 1995; Miceli et al. 1984; Schwartz et al. 1980; Shapiro and Levine 1990; Zingeser and Berndt 1990). They may also have difficulty comprehending locative sentences that describe spatial relationships between objects (Chatterjee and Maher 2000; Frederici 1982; Grodzinsky 1988; Schwartz et al. 1980). Again the neural bases of these deficits have received scant attention. Based on our manner-path studies, we would expect a similar ventral/dorsal division. Lesions based more ventrally along the lateral temporal cortex would be more likely to result in thematic role assignment deficits, and lesions based more dorsally to produce locative deficits. Landau and Jackendoff (1993), adapting the what/where visual processing distinction, speculated that locative prepositions might be processed within parietal cortices. One PET study found that naming locative relations activated inferior parietal cortices (Damasio et al. 2001), and similar findings have been reported for sign language (Emmorey et al. 2002). Tranel and Kemmerer (2004) found that lesions to the left parietal operculum and prefrontal cortices were more likely to produce deficits in knowledge of locative prepositions. For similar results see Amorapanth et al. (in press).

In a series of individuals with aphasia, we examined their relational comprehension abilities in simple sentence-picture matching tasks that described thematic relations (who is doing what to whom) or locative relations. Again, given our functional-anatomic hypotheses, we would expect that thematic role deficits would be associated more closely with posterolateral temporal lesions and locative relations with more dorsally located fronto-parietal lesions. This pattern is what we found (Wu et al. 2007). To determine brain-behavior relationships, we used voxel-based lesion symptom mapping (VLSM) techniques. This method uses a technique in which the level of deficits across groups of subjects is related to whether or not voxels within a standardized space are damaged. Thematic role knowledge deficits correlated with lesions to the middle and superior temporal gyrus. As mentioned earlier, in an fMRI study, Bedny and colleagues (Bedny et al. 2008) found activation in the posterolateral temporal cortex for motion and mental verbs and postulated that this region might instantiate event templates like those for thematic roles (for further evidence on the neural mediation of thematic relations, see Bornkessel et al. 2005 and Shetreet et al. 2007). In our lesion study, the critical location for thematic relation deficits contrasted with locative knowledge deficits, which were associated with posterior parietal, occipito-parietal junction and inferior prefrontal damage. Again, these findings of relational knowledge confirmed our functional anatomic organizational prediction of a ventral-dorsal division between action relational knowledge and locative relational knowledge.

One could speculate that relational thinking is an abstracted category unto itself, removed from the sensory processing of objects and their properties. However, our observations suggest that different kinds of relational thinking, those determined by actions and those determined by locations are treated differently by the nervous system. Their respective neural instantiations follow a general ventral-dorsal organizational form. On a graded view of embodied cognition, the relationships being extracted still retain something, perhaps a schematic representation, of the percepts from which the relationships are derived.

9. Spatial metaphors

Metaphors play a central role in human thought. Lakoff and Johnson (1980) suggested that metaphors are best understood as being ‘grounded’ in their embodied concrete versions. Exactly what is meant by ‘grounded’ in these contexts is far from clear (Gibbs 1996; Gibbs et al. 2004; Murphy 1996), but is hypothesized to be extensions of earlier acquired concrete meanings (Johnson 1987; Lakoff 1987, 1990; Sweetser 1990). We have been suggesting that the lateral temporal cortex processes motion with a gradient of abstraction as one moves anteriorly towards perisylvian cortex (Chatterjee 2008). Would this gradient continue to apply to the figurative use of spatial terms? Talmy (1996) uses the terms ‘factive’ and ‘fictive’ to describe the distinction between literal and certain figurative uses of spatial terms. These terms are used to describe the quality of spatial representations, in contrast to terms like ‘factual’ and ‘fictional’ which describe the truth of things in the world. Factive expressions, like the man runs to the store map palpably to actual movement in the world. By contrast, fictive sentences like the road runs along the river might map onto a spatial aspect the world, but not palpably on to the concrete act of running. Fictive sentences further contrast with metaphoric sentences such as the man runs for office in which no spatial scene is being described.

Matlock (2004) showed that spatial properties of fictive sentences can influence their processing. For example, subjects were given narratives describing travel across large or small distances and then had to judge whether a target sentence like Road 49 crosses the desert applied to the narrative. They found that subjects took longer to judge fictive motion sentences that traversed long over short distances. They also found similar results for fictive motion sentences over difficult as compared to easy terrain. Matlock interpreted these results as indicative that participants are simulating these movements, even though no actual movement is occurring. One issue with this study is that the first line of the narratives instructs the participants to image the scene, like Imagine a desert. Whether these instructions to explicitly imagine the narrative predisposed participants to simulate these fictive statements is not known. More recently, Saygin and colleagues (Saygin et al. in press) reported that attending to fictive sentences activates parts of area MT more so than do static sentences but not as much as sentences of literal motion. These findings do suggest that this figurative use of language describing motion accesses motion attributes perhaps in a graded manner compared to the processing of literal motion.

Much work about the neural bases of metaphoric extensions of spatial terms remains to be done (Schmidt et al. in press). The right hemisphere might be engaged when processing metaphors (see Beeman and Chiarello 1998, and for abstract words Kiehl et al. 1999; Wise et al. 2000; but see Binder et al. 2005; Rapp et al. 2004; Wallentin et al. 2005 for alternate views). Thus, a reasonable prediction would be that homologous regions in the right posterior temporal and parietal cortices would be engaged in metaphoric extensions of spatial events. By contrast, if our hypothesis that the lateral temporal cortex processes motion in increasingly abstract ways, then the left hemisphere within the lateral temporal cortex might be expected to process these kinds of metaphors. Our reasoning was as follows. Action verbs have both concrete sensory attributes as well as more abstract conceptual attributes (Jackendoff 1996). Thus, verbs used literally describe manners of motion, and in the sentential context perhaps the paths or locations of these manners of motion. The conceptual attributes of verbs have to do with notions of source, goals or changes of states. When verbs are used metaphorically, the sensory attributes are attenuated and the conceptual attributes carry the bulk of the meaning. Thus, the man falling under a spell, does not describe physical motion, but does describe arrival at a new state. On this analysis, and given our hypothesized organizational principle for the lateral temporal cortex, we would expect spatial metaphors to be processed more anteriorly within the lateral temporal cortex than sentences that convey literal motion.

In an event-related, functional MRI study, we contrasted spatial metaphors in sentences like the man fell under her spell, to literal motion sentences like the child fell under the slide. Consistent with our predictions, we found greater activation primarily (but not exclusively) in the left lateral temporal lobe and left inferior frontal cortex for spatial metaphors as compared to literal sentences (Chen et al. 2008; see also Wallentin et al. 2005). While considerable work remains to be done regarding predicate metaphors (Schmidt et al. in press), such as the ones using spatial terms as described here, our prediction was confirmed. These findings add further support for the idea that the temporal cortex processes aspects of motion in increasingly abstract ways within neural circuitry that is further removed from posterior perceptual networks. Of course the specific role of the inferior frontal cortex in metaphor processing remains to be worked out.

10. Analog effects on conception not caused by simulation or semantic attributes

As reviewed earlier, analog effects in language or conceptual processing are often considered prima facie evidence in support of embodied cognition. Here, I present evidence that analog effects that might very well be driven by experience in the world need not arise from simulation of that experience or even an attribute relevant to the semantics of the concept.

People often conceive of events proceeding from left-to-right (Chatterjee 2001; Chatterjee, Maher and Heilman 1995). Within this mental representation of events, agents of actions are conceived on the left and the patients or recipients of actions on the right. Evidence for this default representation of the structure of events originated in clinical observations of a highly educated man with aphasia (Chatterjee, Maher, Gonzalez-Rothi et al. 1995; Maher et al. 1995). As mentioned earlier, some people with brain damage have trouble assigning thematic roles in sentences, or determining who is doing what to whom in semantically reversible sentences, such as ‘the girl kisses the boy’ (Chatterjee and Maher 2000; Saffran et al. 1980; Schwartz et al. 1980). The index patient was a college professor who had significant difficulties in producing and comprehending reversible sentences. However, rather than performing randomly, he used a spatial strategy. In describing pictures, he consistently produced sentences in which the participant on the left of the picture was the agent (Maher et al. 1995). Thus, if a picture showed a circle stick figure on the left kicking a square, he accurately stated that the circle was kicking the square. However, if the picture depicted the circle on the right kicking the square on the left, he would say that the square was kicking the circle. Similarly, in comprehension tasks in which he matched sentences to choices of pictures, he was more likely to match ‘the circle kicks the square’ to pictures with the circle on the left regardless of whether the circle was doing the kicking or receiving the kick (Chatterjee, Maher, Gonzalez-Rothi et al. 1995).

We speculated that the subject’s spatial biases reflected a primitive structure of mental representations of events (Chatterjee 2001; Chatterjee, Maher, Gonzalez-Rothi et al. 1995). Hughlings Jackson (Jackson 1932) in the nineteenth century viewed the nervous system as being organized hierarchically, with higher processes inhibiting lower ones. Jackson thought that dissolution of higher functions released more primitive behaviors. Accordingly, the dissolution of this patient’s linguistic abilities by brain damage might have released a primitive prelinguistic representation making explicit an underlying spatial schema that we all might be harboring.

If events are encoded with spatial schemas, then subtle spatial biases might also influence normal subjects’ conception of actions and thematic roles. Several subsequent experiments confirmed this prediction (Chatterjee, Maher and Heilman 1995; Chatterjee et al. 1999). Healthy participants are more likely to draw the circle on the left when asked to draw events like ‘the circle pushes the square’. They also are likely to draw the agent (circle in this example) to the left of where they draw the recipient when asked to draw just the circle or just the square on separate cards. They are more likely to depict horizontal actions with trajectories moving from left-to-right than right-to-left. In reaction time experiments, subjects match sentences to pictures more quickly if the agent is on the left than on the right, and if the action proceeds from left-to-right than from right-to-left. These observations have been replicated and extended by Maass and Russo (2003) in Italian subjects. These subjects also positioned agents to the left of recipients of actions and responded more quickly in sentence picture matching tasks if the action was depicted in a left-to-right direction.

We proposed that a directional schema underlies our representation of events and serves as an intermediary between our perceptions of events in the world and our abilities to communicate information about these events. Why should events be represented with a left-to-right schema? We originally speculated that this bias might follow from the left hemisphere’s propensity to direct spatial attention with a left-to-right vector (Chatterjee, Maher and Heilman 1995) as originally proposed by Kinsbourne (1987). On such an account, one would not expect cultural variables to have much of an influence on the direction of the schema. This prediction has since been disconfirmed (Altmann et al. 2006; Maass and Russo 2003). Maass and Russo (2003) studied Arab along with Italian participants in several tasks of event representation. Arabs use a right-to-left writing system. Arab participants showed a reversal of the directionality effects in many (but not all) instances, when compared to the Italian participants. Similar influences of reading habits have also been reported in other domains that show horizontal axis effects. For example, Arabic-, Hebrew- or Urdu-reading participants show right-to-left directional biases in perceptual exploration, drawing, decoding of facial affect, number representation and aesthetic preferences (Chokron and De Agostini 2002; Dehaene et al. 1993; Maass et al. 2007; Nachson et al. 1999; Padakannaya et al. 2002; Tversky et al. 1991; Vaid et al. 2002). Given the accumulating empirical evidence, the conclusion that reading and writing habits influence directional properties of an underlying event representation is unavoidable.

How might directional selectivity that could be modulated by experience arise in the nervous system? Certainly neurons in area MT and MST (and even in V1 for that matter) respond preferentially to motion in specific directions (Weliky et al. 1996), and MT/MST as well as the intraparietal sulcus is active when subjects are cued to expect motion in a specific direction (Shulman et al. 1999). But these areas are not known to intrinsically harbor more neurons selective for motion in one direction over another. Recent neurophysiological observations suggest a possible mechanism for directional selectivity in motion perception brought about by experience. Visual neurons in ferrets exhibit weak directional biases that lack any spatial coherence. However, training with moving stimuli strengthens the directionally selective responses in individual neurons, induces the emergence of directional selective cortical columns and even reverses the intrinsic bias of some neurons to the trained direction (Li et al. 2008). Of course, there are many steps between the acquisition of visual motion selectivity in the ferret’s visual cortex and the acquisition of directional spatial primitives in humans based on reading and possibly eye-movements. The point is simply that neurophysiological mechanisms that induce directional selectivity based on experience exist. The generalization of similar mechanisms might offer insight into the biology of how experience can influence directional primitives at the interface of perception and conception.

Embodied accounts posit that the structure of our thought is deeply informed by our interactions in the world. As we have reviewed, schematic directional representations of events are influenced by reading habits, and in that sense they are influenced by interactions in the world. However, they are not simulations of actual events as they occur in the world. Nor does the specific directionality capture anything about the semantics of the events themselves. We perceive events from every possible direction and encounter individuals we might conceive of as agents in any number of locations. Certainly, some stylized events like races on tracks, whether run by humans, horses, dogs or cars, typically proceed from left-to-right. But most of us do not spend enough time at racetracks for such events to be ingrained as the prototypic direction for actions.

If these default event schemas are not accounted for by the structure of experienced events, then why do they exist? Here I offer a conjecture. This conjecture is based on three notions: the mental structuring of space, reasoning with mental models, and processing efficiency.

10.1. Spatial schemas structure space

As mentioned earlier, Talmy (1996) proposed that spatial schemas are ‘boiled down’ features of spatial scenes. For example, ‘across’ refers to a schema that describes a specific path of movement. This path is approximately perpendicular to the principle axis of the reference object, as in across a river or across a plank. When a movement proceeds in parallel to the principle axis of the reference object, then ‘along’ is more appropriate. Both ‘across’ and ‘along’ are abstracted from the actual scene. In these schemas, only selective spatial aspects are relevant. Other aspects of the scene, such as whether the referent object is a river or a plank, are not relevant and are not incorporated into the schema. Schemas are topological rather than imagistic and capture only some of the infinite possible spatial configurations. This property seems to be a precondition of communication, in which a wide array of spatial situations need to be described rapidly. Thus, the constituents of schemas are simple geometric forms such as points, lines and planes rather than the rich and infinite possibilities of perception from which they are derived.

10.2. Mental models

This description of spatial schemas bears a striking resemblance to specific elements of mental models used in reasoning (Gentner and Stevens 1983; Johnson-Laird 1996), for which there is a long history (Johnson-Laird 2004). According to Johnson-Laird, perceptions yield models of the world outside of the perceiver, and comprehension of discourse yields models of the world that the speaker describes to the listener. Thinking often involves the manipulation of these mental models in order to anticipate the world and choose among actions (Johnson-Laird 2006). This may not be the only kind of thinking, but it is a critical one, especially when it comes to thinking about relations.

Mental models are schematic (Gentner and Stevens 1983; Johnson-Laird 2006). They are schematic insofar as their parts and relations correspond to the structure of the situation they represent. They underlie perception but are not identical to perceptions. And mental models represent abstractions, in that they attempt to capture what is common in all the ways a certain situation might occur. This description of the iconic elements in mental models is close to Talmy’s description of spatial schemas. For our purposes, the point is that event schemas form iconic elements for thinking about simple events that can then be manipulated or recombined in more complex reasoning efforts.

Thus far, I have suggested that events are schematized in a simple form. As discussed earlier, the location of the agent and the patient and the direction of action are important aspects of this schema. Such schematizing is critical to how we conceptualize space and establish elements used in the construction of mental models for reasoning. But why should it be useful for people to pick a specific location for agents, such as on the left, and a specific direction for actions, such as moving from left-to-right?

10.3. Processing efficiency

The convention of assigning a default direction for actions and a default location for agents provides processing efficiency. Which specific convention is chosen is not nearly as important as the fact that a specific convention is assigned. This point is best made by analogy. There is no logical reason that a society should choose to drive on the left or on the right. However, it is efficient for cultures to choose one or the other convention and adhere to it. Similar observations can be made on busy sidewalks, where norms of where one walks are not established as rigidly as in driving. People may walk on the right or on the left, but when it gets very busy, people organize naturally into streams of movement in ways that maximizes every ones’ ability to move. When schematizing events, we need to establish the sources, directions, causes, and goals of actions. By analogy with driving or walking conventions, in constructing mental models we adopt a default spatial schema that can then be manipulated if there is a lot of mental work to be done. Thus, no mental resources need to be diverted to establishing and remembering the layout of a schematized event and one can get on with the business of thinking.

Whether or not the conjectures offered here are accurate awaits further study. Whatever the best explanation for these directional effects in conceptualizing events, how do they relate to embodied cognition? The reason for reviewing these data in some detail is to point out that analog effects in conception can arise from experience and in that sense might be regarded as embodied. However, these embodied effects are not those that are usually invoked by embodied cognition theorists. These effects have nothing to do with the semantic content of the concept, or sensory and motor experiences with referents of the concept. Here the specific analog properties of event representation might facilitate efficient reasoning, rather than reflect the simulation of actual events and or an attribute intrinsic to the meaning of the event.

11. Concluding comments

Embodied cognition accounts have held great sway in recent theorizing about how humans represent concepts. Many of the experiments designed to demonstrate embodied cognition are elegant and the phenomena described are often striking. Nonetheless, how best to interpret these findings is far from clear. Mahon and Caramazza (2008) incisively point out that more data of the same kind as has accumulated over the last 15 years will not surmount the inferential limits in interpreting these results. A wholesale acceptance of embodied accounts invites permissiveness in the kinds of data that constitute evidence in support of embodied accounts. This permissiveness is accompanied by a tendency to exaggerate the explanatory force of the observations. With respect to neural instantiations of embodied accounts, one often finds a lack of specificity and critical analysis of the actual data generated. We need to design investigations to target the relative contributions of specific sensory or motor attributes. We also need to consider how individual differences in experiences might sculpt the way we understand a concept, as done by Beilock and colleagues (Beilock et al. 2008; Lyons et al. in press).

A challenge for those committed to an embodied account of concepts is to answer the obvious question: how do we abstract? These questions are especially relevant in a view that does not impose impermeable barriers between modal and amodal representations. For example, Talmy (2000) emphasizes the inter-penetrability of perception and conception (which he terms ‘ception’) and suggests that one might focus on identifying common parameters across domains that might otherwise be obscured. I accept the view that language and thought, rather than being completely autonomous cognitive modules, are linked tightly to non-linguistic representations (Landau and Jackendoff 1993; Stanfield and Zwaan 2001; Tanenhaus et al. 1995). While the relationship between percepts, concepts and language may be ‘sketchy’ (Papafragou et al. 2002), it is unlikely to be arbitrary (Bowerman and Choi 2001; Chatterjee 2001; Gennari et al. 2002; Hayward and Tarr 1995; Jackendoff 1987, 1996; Levinson 1996; Mandler 2004; Regier 1995; Talmy 1983, 2000). I suggest that one goal for cognitive neuroscience is to identify and study these non-arbitrary links between perception, conception and language, without collapsing them.

Barsalou (2005) offers some ideas on how abstraction might occur in a way that integrates symbolic operations with experience. He distinguishes between simulations and simulators and suggests that simulators perform symbolic functions like type-token binding, inference, recursion and propositions. This kind of theorizing is what is needed to flesh out embodied cognition accounts. This theorizing also needs to be accompanied by empirical data directed at similar levels of analysis. For example, Glenberg and colleagues (Glenberg et al. 2008) show how repeated motor acts can influence comprehension of the transfer of abstract information (e.g. give responsibility). Within neuroscience, Damasio’s postulates about convergence zones (Damasio 1989), which was an important theoretical advance at the time, needs to be extended. What are the properties of different unimodal and heteromodal association cortices? What is the nature of sensory or motor ‘fragments’ as one moves up the association hierarchy? Referring to control of representations “via association areas in the temporal, parietal and frontal lobes” (Barsalou 2007: 622) lacks the kind of specificity desirable if neuroscience data are to be taken seriously as informative of embodied accounts.

The amodal view of cognition is sometimes referred to as ‘the standard view’ in psychology and cognitive science (Barsalou 2007). However, graded views have been traditional in the neurosciences. This view is traditional in the sense that it has been around a long time in neurology and neuropsychology. In the nineteenth century, Lissauer (1890) in describing object recognition deficits distinguished between apperceptive and associative visual agnosia. Apperceptive agnosias are recognition deficits that arise from sensory integration deficits, whereas associative agnosias arise from more abstract conceptualizations of objects. Discussions since Lissauer focused on whether these syndromes are truly distinct (Farah 1990), but the idea that there is a continuum in which knowledge of an object might be weighted towards the sensory system or towards more abstract conceptions has remained a guiding principle. Similar graded views even apply to a domain that might be considered deeply embedded within the motor system. Liepmann (Goldenberg 2003; Liepmann 1908) argued for the neural instantiation of different forms of knowledge of skilled movements. The idea of distinguishing limb-kinetic from ideomotor from ideational and conceptual apraxias is predicated on increasingly abstract notions of motor representations.

Stepping back from the cognitive theorizing and considering the organization of the nervous system, it is hard to imagine that strong versions of embodied or of disembodied cognition could possibly be correct. The nervous system has unimodal primary sensory cortices, unimodal association cortices, multimodal cortices as well as structures several synapses removed from sensory inputs and motor outputs that they might be considered amodal. This organizational structure must have deep implications for the organization of cognition, as Mesulam (1998) described comprehensively. Geshwind’s seminal paper ‘The disconnection syndromes’ (Geschwind 1965), a paper that set off the modern era of cognitive neurology and neuropsychology, was predicated on the connectedness and coordinated flow of information across different parts of the brain. On these traditional views, a hypothesis that posits deeply divided silos in the brain between perceptual or motor circuitries (embodied) and amodal (disembodied) ones is simply implausible. The question at hand is how do these regions interact and in the service of what kind of representations or processes?

If neuroscience data are to be taken seriously, several questions need to be addressed. Here I suggest three questions organized around three functional-anatomic axes of the brain. How do we incorporate right-left laterality differences in the processing of sensory and motor attributes as they relate to concepts? As suggested earlier, one possibility is that ‘sensory’ cortices in the right hemisphere are more sensitive to richly textured analog representations, and the left to categorical and schematic representations. Schematic representations in particular have not received any scrutiny in cognitive neuroscience. Second, how do we incorporate ventral-dorsal axes in thinking about embodied concepts? In thinking about processing streams, the ventral occipito-temporal cortex seems biased to process objects, and dorsal fronto-patietal cortices to process spatial relations. Manners of motion as related to actions seem to be intermediate between the two. It appears that as one moves along a ventral to dorsal axis, conceptual information sheds its body, shifting from richly textured objects to schematic relationships of the kind conveyed by prepositional relationships. Much more data is needed to establish this gradient both with perception and with language. Third, how do we account for gradients of processing from perceptual and motor cortices centripetally towards peri-Sylvian language cortices? For motion, we have suggested that there is a gradient of processing from low level visual motion, to meaningful motion, to the implied motion as conveyed by action verbs, a gradient represented within area MT/MST and pSTS moving along MTG, as the analog representations get converted into digital ones (Chatterjee 2008). This distancing from sensory attributes towards and within perisylvian cortices might further occur with the figurative use of verbs of motion. Whether there is an analogous organization for paths of motion or static locative relations remains to be worked out.

These three anatomic axes are proposed as a way to organize thinking about how the nervous system might make use of sensory and motor information and then abstract away from that information in a graded manner. It may be that the freedom from specific sensory motor constraints is what allows concepts to extend beyond their referential function to be used flexibly and generatively. Perhaps this freedom is precisely what is needed for our conceptual systems to get off the ground.