Abstract
The relationship between the brain’s conceptual or semantic and sensory-motor systems remains controversial. Here, we tested manual and conceptual abilities of 41 chronic stroke patients in order to examine their relationship. Manual abilities were assed through a reaching task using an exoskeleton robot. Semantic abilities were assessed with implicit as well as explicit semantic tasks, for both verbs and nouns. The results show that that the degree of selective impairment for action word processing was predicted by the degree of impairment in reaching performance. Moreover, the implicit semantic measures showed a correlation with a global reaching parameter, while the explicit semantic similarity judgment task predicted performance in action initiation. These results indicate that action concepts are dynamically grounded through motoric simulations, and that more details are simulated for more explicit semantic tasks. This is evidence for a close and causal relationship between sensory-motor and conceptual systems of the brain.
Keywords: semantics, action, embodiment, language, stroke
Introduction
The relationship between the brain’s conceptual or semantic and sensory-motor systems remains controversial. There has been a steady accumulation of evidence that suggests a sensory-motor basis for concepts (for reviews, see Barsalou, 2008; Gallese & Lakoff, 2005; Kiefer & Pulvermüller, 2011; Meteyard, Cuadrado, Bahrami, & Vigliocco, 2010). This model contrasts with traditional views that see cognition as manipulation of abstract and amodal symbols (Fodor, 1983; Newell & Simon, 1976; Pylyshyn, 1984).
Four major lines of experimental evidence exist for view of grounded cognition: behavioral studies (for a review, see Fischer & Zwaan, 2008), brain activation measures, brain stimulation studies, and patient studies (for reviews, see Binder & Desai, 2011; Meteyard, et al., 2010). The evidence from disruption studies (involving brain damage or stimulation), showing modality-specific semantic impairment resulting from the corresponding sensory-motor impairment, is often considered the strongest, as it can reveal a causal relationship between conceptual processing and sensory-motor systems.
A number of studies have investigated action semantics in populations of patients with motor impairments and reported impairment of action semantics (Bak, O’Donovan, Xuereb, Boniface, & Hodges, 2001; Boulenger, et al., 2008; Buxbaum & Saffran, 2002; Cotelli, et al., 2007; Fernandino, et al., 2012; Fernandino, et al., 2013; Grossman, et al., 2008; Ibanez, et al., 2013; Neininger & Pulvermuller, 2003). No study, however, has shown that the degree of motor impairment is predictive of the degree selective impairment in action concept processing, which can be considered the strongest form of evidence for grounding. A majority of the studies also compare action verbs to nouns, resulting in grammatical class confound, as verbs are syntactically and semantically more complex than nouns (Druks, 2002; Vigliocco, Vinson, Druks, Barber, & Cappa, 2011). Additionally, a complete theory of grounded cognition would necessitate a more detailed account of the nature of concept representations. Two types of important questions can be investigated: (a) are specific conditions, context, or task demands necessary to engage sensory-motor systems in conceptual processing (b) what is the precise nature of this involvement? For example, if actions are understood in terms of abstracted action simulations, what is simulated under various task demands?
The goals for the present study were threefold. A primary goal was to investigate whether motor performance is correlated with processing of action-related words, relative to that of non-action-related words, when controlling for grammatical class. We tested chronic stroke patients with potential impairments in their upper-limb motor functions, through a reaching task implemented in a robotic device that enables detailed measurements of reaching movements. For assessing action language performance, two types of langue stimuli: action verbs and manipulable object nouns. These were compared with non-action verbs and non-manipulable nouns respectively.
A second aim was to examine the effects of task demands that may lead to the involvement of sensory-motor systems in conceptual processing. We used two different tasks: lexical decision (LD) and semantic similarity judgment (SSJ). The LD task typically activates semantics, but does not require access of semantics. It can be used to investigate an early, automatic stage of semantic access. The SSJ task, on the other hand, requires explicit comparison between word meanings, and thus shows effects of controlled semantic processing. The LD task also included a masked priming manipulation, so that we were able to evaluate motor/semantic interactions at three levels of cognitive control: subliminal activation (masked priming), implicit conscious activation (LD), and explicit comparison (SSJ).
Lastly, we aimed to examine the type of motor simulations that may be evoked under different task conditions. The robotic device allowed us to measure a number of different attributes of reaching movements. A strong hypothesis predicts that performance on all aspects of reaching movements will be predictive of action word performance in all tasks, because the action simulations used to for comprehension under subliminal, implicit, or explicit conditions are always very detailed. A dynamic or context sensitive view of semantic representations, on the other hand, would predict that explicit semantics tasks would engage more detailed simulations, while implicit semantic tasks would result in more abstract simulations that would only predict global action performance. Another alternative is that the action simulations that ground action concepts are at a higher level of abstraction, and load on planning complex actions (such as hand-object interactions or tool use). Hence, performance in a relatively simple reaching task would not be predictive of performance on action word processing. Finally, the traditional amodal view states that any activation of the motor system is post-comprehension and peripheral, while the core content is amodal, and hence there would be no causal link between conceptual processing of actions and their performance.
Methods
Patients
Data for both reaching and language tasks was collected from 41 chronic stroke patients (16 females), as part of a larger study. The mean age of the participants was 59 years (SD 11; range 37 to 81 years). Thirty-nine patients had a left hemisphere stroke, while two had brainstem strokes. On average, they were 4.5 years (SD 4; range 10 months to 17 years) post-stroke. Three patients had multiple stroke events, all others had a single event. Prior to the stroke, 34 were right hand dominant while the rest were left hand dominant. Twenty-nine patients were aphasic, while the rest were non-aphasic. All patients were evaluated with a battery of neuropsychological tests, including the Western Aphasia Battery (Kertesz, 1982), and were assigned an aphasia quotient (AQ; a measure of aphasia severity ranging from 0 to 100). The mean AQ for the group was 79.1 (SD 21.9; range 23.6 to 99.6). One patient did not complete the Verb SSJ task (see Language Tasks below); hence data from 40 patients were used for this task. Data from on additional patient was excluded due to below chance performance on the LD task (35% accuracy).
Robotic Apparatus
We assessed motor function of the subjects’ upper limb using the KINARM bilateral endpoint robot (BKIN Technologies Ltd, Kingston, ON, Canada; Fig. 1A). This device permits hand movements in the horizontal plane, monitors and records hand position in real-time, and if required can apply resistive and/or assistive forces to each hand (Scott, 1999; Scott & Dukelow, 2011). The robot is coupled to an augmented reality system that displays computer-generated visual targets in the same plane as the subjects’ arms. This allows subjects to realistically interact with targets, similar to virtual reality, but without completely immersing them into a digital environment that requires complex sensorimotor transformations. Here, we report results from a visually-guided reaching task, described below.
Reaching Task
Motor function was assessed with a visually guided reaching task (Fig. 1, B and C). A full description of the task, outcome measures, and sensitivity and reliability of the various measures has been previously described by Coderre et al. (2010). In brief, subjects were asked to make unassisted reaching movements “quickly and accurately” from a central target (red circle, 1.0 cm radius) to 1 of 8 peripheral targets located 10.0 cm away. A small white curser (0.4 cm radius) provided hand position feedback. Each target was presented once in a randomized block, and eight blocks were collected for a total of 64 trials for each arm. Reaching actions can be described by a number of different attributes, including: a) time of action initiation, b) initial movement planning (feedforward control) c) corrective movements (feedback control), and d) total movement metrics. Although several measurements can be obtained to quantify each attribute, we used one parameter for each attribute for our a priori hypotheses. These measures were previously found to provide the best combination of high reliability and sensitivity (i.e., the ability to reliably detect the presence of an impairment relative to normal behavior) (Coderre, et al., 2010; Dukelow, Herter, Bagg, & Scott, 2012). Time of action initiation was quantified with reaction time (RT), the time from peripheral target onset until movement onset. Initial movement planning was quantified using initial movement direction error (IDE), which is the angular deviation between a straight line from the central to peripheral target and the actual path taken in the initial phase of movement. Corrective movements were quantified by recording the number of speed maxima (velocity peaks) per movement (NSM). The total movement metric used was total movement time (TMT), the time between reaching onset and offset. Two additional post hoc measures were also tested: Path Length Ratio (the ratio of the distance travelled during forward and return movements), and Maximum Speed.
Language Tasks
Four different tasks were used: verb lexical decision (LD), noun LD, verb semantic similarity judgment (SSJ), and noun SSJ.
Materials
Lexical Decision Task
The verb (LD) task used a set of 80 verbs and 80 phonologically plausible pseudowords. Pseudowords were selected from the English Lexicon Project (ELP) database (http://elexicon.wustl.edu; Balota, et al., 2007), such that verbs and pseudowords were matched in number of letters, bigram frequency, orthographic neighborhood size, and LD accuracy. Half of the verbs referred to voluntary hand/arm actions (e.g. to pour, to wave), and the others referred to other sensory or cognitive concepts (e.g. to observe, to notice) not directly associated with actions. Action and non-action verbs were matched in number of psycholinguistic variables (see Table 1), including LD RT and accuracy, and naming accuracy from the ELP database. Additionally, they were also matched on semantic diversity (SemD), a measure of the diversity of contexts that a word appears in (Hoffman, Rogers, & Lambon Ralph, 2011).
Table 1.
Verbs | Nouns | |||||||||
---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||
Action | SD | Non-Action | SD | T-test p | Manip | SD | Non-manip | SD | T-test p | |
NLett | 4.98 | 1.03 | 5.00 | 1.26 | 0.92 | 5.68 | 1.44 | 5.63 | 1.51 | 0.88 |
NPhon | 3.88 | 0.79 | 3.88 | 1.02 | 1.00 | 4.43 | 1.20 | 4.55 | 1.41 | 0.67 |
NSyll | 1.18 | 0.38 | 1.18 | 0.45 | 1.00 | 1.68 | 0.62 | 1.73 | 0.64 | 0.72 |
Log F | 1.29 | 0.60 | 1.23 | 0.60 | 0.68 | 1.08 | 0.51 | 1.15 | 0.47 | 0.52 |
LD RT | 632 | 53 | 638 | 69 | 0.64 | 644 | 72 | 649 | 48 | 0.70 |
LD Acc | 0.97 | 0.05 | 0.96 | 0.04 | 0.84 | 0.97 | 0.05 | 0.96 | 0.04 | 0.64 |
NamingRT | 623 | 52 | 623 | 46 | 0.99 | 627 | 57 | 621 | 47 | 0.63 |
Bigram F | 1559 | 810 | 1539 | 600 | 0.90 | 1659 | 622 | 1540 | 598 | 0.39 |
SemD | 1.65 | 0.31 | 1.66 | 0.23 | 0.89 | 1.57 | 0.21 | 1.52 | 0.22 | 0.34 |
Orth N | 4.38 | 3.70 | 4.93 | 3.98 | 0.52 | 3.75 | 4.89 | 2.85 | 4.19 | 0.38 |
Phon N | 12.55 | 9.89 | 11.18 | 9.49 | 0.53 | 9.38 | 10.95 | 6.78 | 8.46 | 0.24 |
Conc | - | - | - | - | - | 594 | 40 | 586 | 20 | 0.34 |
BOI | - | - | - | - | - | 5.18 | 0.56 | 3.65 | 1.00 | <.001 |
SemD = semantic diversity measure, Conc = concreteness rating, BOI = body-object interaction rating, LD = lexical decision (measures from the ELP database).
Similarly, 80 nouns and 80 pseudowords were used for the noun LD task. Half of the nouns were highly manipulable objects (the phone, the pen), while other half was concrete but non-manipulable objects (the ocean, the stadium). The manipulable and non-manipulable nouns were matched on a number of variables, as shown in Table 1. The body-object interaction (BOI) ratings (Tillotson, Siakaluk, & Pexman, 2008), which assess the ease with which the human body can interact with a word’s referent, were significantly different for the two types of nouns, as expected.
Semantic Similarity Judgment Task
The verb SSJ task used a set of 120 action verbs and a set of 120 non-action verbs. Each set was organized into 40 triplets, such that in each triplet, the target verb was more similar in meaning to one of the two choices (e.g., to thrill, to excite, to harm; to carve, to cut, to catch; bold indicates the target word). The two conditions were matched along a number of psycholinguistic dimensions (Table 2).
Table 2.
Verbs | Nouns | |||||||||
---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||
Action | SD | Non-action | SD | T-test p | Manip | SD | Non-manip | SD | T-test p | |
NLett | 5.13 | 0.97 | 5.10 | 0.83 | 0.87 | 5.70 | 1.67 | 5.68 | 1.54 | 1.00 |
NPhon | 4.08 | 0.77 | 4.09 | 0.92 | 0.93 | 4.51 | 1.34 | 4.62 | 1.55 | 0.48 |
NSyll | 1.29 | 0.40 | 1.36 | 0.37 | 0.44 | 1.58 | 0.62 | 1.66 | 0.75 | 0.30 |
Log F | 1.19 | 0.36 | 1.12 | 0.33 | 0.37 | 0.99 | 0.54 | 1.07 | 0.49 | 0.20 |
LD RT | 655 | 41 | 649 | 42 | 0.55 | 662 | 73 | 656 | 64 | 0.49 |
LD Acc | 0.94 | 0.05 | 0.95 | 0.04 | 0.39 | 0.94 | 0.09 | 0.95 | 0.08 | 0.61 |
NamingRT | 628 | 34 | 627 | 33 | 0.83 | 638 | 63 | 630 | 54 | 0.29 |
Bigram F | 1593 | 480 | 1756 | 378 | 0.10 | 1654 | 746 | 1663 | 800 | 0.91 |
SemD | 1.69 | 0.15 | 1.72 | 0.14 | 0.35 | 1.51 | 0.23 | 1.49 | 0.22 | 0.40 |
Conc | - | - | - | - | - | 582 | 42 | 567 | 49 | 0.38 |
BOI | - | - | - | - | - | 5.14 | 1.00 | 3.91 | 1.30 | <.001 |
Similarly for the noun SSJ task, 120 manipulable and 120 non-manipulable nouns were used (e.g., the chalk, the crayon, the baton; the glacier, the iceberg, the tornado). There was a significant difference (p < 0.001) in the BOI between the two conditions.
Procedure
The procedures were similar to those used by Fernandino et al. (2013). Briefly, the LD task involved presentation of a fixation cross (500 ms), a mask (‘########’, 100 ms), the prime (50 ms), mask (100 ms), followed by the target word. The participants were asked to indicate as quickly and as accurately as possible whether the target was a word or not by pressing one of two response buttons. The participants were given four seconds to respond (which differs from Fernandino et al. (2013), who provided unlimited time for response), after which the next trial started and a missing response was recorded. The prime was the same as the target word/pseudoword in capital letters for half of the stimuli (identity prime), and a consonant string also in capital letters for the other half (consonant string prime) as control. There were 160 trials, divided equally between words and pseudowords, in both the verb and noun LD tasks.
For the SSJ task, in each trial, three words were presented simultaneously in a triangular arrangement, with the target word at the top and the two alternatives at the bottom. Participants were instructed to decide which of the two words on the bottom was most similar in meaning to the target, and indicate their response by pressing one of two response buttons as quickly and as accurately as possible. The participants were given five seconds to respond. There were 80 trials, divided equally between action verbs or manipulable nouns, and non-action verbs or nouns, and presented in random order.
For both tasks, the verbs were preceded by ‘to’ and nouns by ‘the’. Because ourgoal was to assess variation within patients, all patients were tested with identical stimuli, with order of presentation randomized across patients.
Analysis
Accuracy for each condition for each subject was calculated for both LD and SSJ tasks. Missing responses were counted as errors. The degree of priming was calculated as the normalized difference in RT between identity-primed (IP) and consonant-string-primed (CSP) trials, as (CSP-IP)/(CSP+IP). Trials with RT more than 1.5 standard deviations away from the condition mean were removed.
Many patients experience general cognitive decline following a large injury (Glascher, et al., 2009). Because we are concerned with specific effects on action semantics, and not with general cognitive decline, we calculated difference scores as action minus non-action accuracy, and similarly, degree of priming for action minus degree of priming for non-action words. Lower or more negative difference scores indicate selectively worse performance for action semantics, while larger or more positive scores indicate relatively worse performance for non-action semantics.
Each measures obtained from the reaching task was converted to a z-score using age-specific norms and regression models similar to those previously described by Herter et al. (2014). Measures from the left and right arms were combined into a single measure. Three patients who were unable to perform the task with one arm were assigned a z-score of 10 for that arm, giving them the highest rank (indicating the worst performance among the group). Note that the exact value is not critical as long as it is high enough, as we used nonparametric correlation as mentioned below. A correction for multiple comparisons was applied for the post hoc measures, using False Discovery Rate (Benjamini & Hochberg, 1995)
Our a priori hypotheses predicted worse performance on each of the four reaching measures to be correlated with lower difference scores, indicating selective impairment for action semantics. We calculated Spearman’s correlation coefficients between each of the language measures and the reaching measures.
Results
The z scores for the four parameters of the reaching task are summarized in Table 3. A majority of the patients had positive z scores on all four parameters, indicating worse performance than the average control. We also examined the correlation between the variables. All of them were correlated (Table 4), with TMT and NSM exhibiting the highest correlation (rho = 0.78, p < 0.0001). This makes it difficult to distinguish the relative contribution of each.
Table 3.
TMT | NSM | RT | IDE | |
---|---|---|---|---|
Min. | −2.06 | −1.73 | −2.17 | −2.14 |
1st Qu. | 0.58 | 0.79 | 1.12 | 0.90 |
Median | 2.62 | 1.86 | 2.50 | 2.91 |
Mean | 2.70 | 2.78 | 2.73 | 4.09 |
3rd Qu. | 3.76 | 3.71 | 3.96 | 5.32 |
Max. | 9.95 | 11.05 | 11.36 | 17.18 |
TMT = total movement time, NSM = number of speed maxima, RT = reaction time, IDE = initial direction error. (Inability to complete task assigned a z-score of 10 for that arm).
Table 4.
TMT | NSM | RT | IDE | |
---|---|---|---|---|
TMT | 1 | |||
NSM | 0.78 | 1 | ||
RT | 0.59 | 0.53 | 1 | |
IDE | 0.55 | 0.69 | 0.52 | 1 |
For the language tasks, the mean accuracies (SD) and the degree of priming, collapsed across conditions, were as follows: Verb LD 95.5% (7.6%), Verb Priming 2.3% (5.6%), Noun LD 95.2% (6.6%), Noun Priming 2.8% (4.8%), Verb SSJ 69.9% (20.7%), Noun SSJ 74.5% (18.7%). Difference scores (the difference in performance between action- and non-action-related words), which were our main variables of interest, are summarized in Table 5. The mean and median difference scores were close to 0, indicating an approximately equal performance on action and non-action words in all of the tasks across the whole group. We also examined the correlations in difference scores between the tasks, and no significant correlation was found. A trend of correlation between Verb SSJ and Noun SSJ (p < 0.1) was found.
Table 5.
Verb LD | Verb Priming | Noun LD | Noun Priming | Verb SSJ | Noun SSJ | |
---|---|---|---|---|---|---|
Min. | Acc -7.5% | Priming -28.4% | Acc -10.0% | Priming -14.1% | Acc -17.5% | Acc -25.0% |
1st Qu. | 0.0% | −5.2% | −2.5% | −4.6% | −7.5% | −5.0% |
Median | 0.0% | −2.3% | 0.0% | −0.8% | −2.5% | 0.0% |
Mean Median | 1.2% | −2.3% | 1.2% | −1.5% | −1.8% | −0.1% |
3rd Qu. | 2.5% | 1.4% | 5.0% | 1.4% | 2.5% | 5.0% |
Max. | 15.0% | 17.4% | 12.5% | 6.9% | 20.0% | 20.0% |
Results of the Spearman’s correlation between difference scores and the reaching task measures are summarized in Figure 2. Total Movement Time (TMT) was correlated with the difference scores in Verb LD (rho = −0.34, p = 0.015) and Verb Priming (rho = −0.34, p = 0.016). Number of Speed Maxima (NSM) was correlated with Verb LD (rho = −0.27, p = −.041), Verb SSJ (rho = −0.28, p = 0.041), and Noun SSJ (p = 0.039). A trend for Verb Priming was also observed (rho = −0.22, p = 0.079). Reaction Time (RT) was correlated with Verb SSJ (rho = −0.29, p = 0.035) and Noun SSJ (rho = −0.35, p = 0.012). It also showed a trend for correlation with Noun LD (rho = −0.22, p = 0.081). Lastly, Initial Direction Error (IDE) was correlated with difference scores in Verb SSJ (rho = −0.29, p = 0.035). After correction for multiple comparisons, Path Length Ratio and Maximum Speed did not show significant correlation with any language measure. Figure 3 shows the scatterplots for each significant correlation.
Discussion
We aimed to examine whether motor systems of the brain play a causal role in representation of the meaning of action related words, a central question in the contemporary debate regarding the nature of the conceptual system. The results provide an affirmative answer. In language tasks, greater impairment was seen in processing action relative to abstract words, when the impairment in action performance was greater.
The design allowed us to examine semantics at three levels of processing: subliminal, implicit, and explicit. None of the tasks require or encourage artificial mental simulations or visualizations. Any such processes taking place in service of these linguistic tasks, which simply require a word/nonword decision or a semantic comparison, are part of semantics. Remarkably, the results indicate effects of action performance ability at all three levels of semantic processing. The degree of identity priming is reduced for verbs even in the absence of awareness of the prime, indicating the rapid and automatic nature of the spread of activation into action circuits. The LD task, which does not require semantic access but implicitly results in semantic activation, was also affected for verbs, suggesting that explicit or overt use of verb semantics demands is not necessary to induce action-related simulations. Lastly, the explicit semantic task of SSJ was found to predict action performance as well.
Manipulable nouns, relative to concrete non-manipulable nouns, did not show effects in the implicit tasks (LD and priming), where only a trend was observed for correlation with RT. Manipulable nouns are associated with actions, but do not denote actions as such, as opposed to action verbs. We suggest that when actions are one step removed from the meaning in this manner, action semantics may be activated only weakly in task contexts that make no demands on meaning. When a task, such as SSJ, explicitly requires access to meaning, the action-relatedness of the concept comes to the fore, as evidenced by the correlations in the Noun SSJ task.
The differences in correlations with the four reaching parameters are also instructive. For the implicit tasks, a correlation was observed for TMT, which represents a global parameter. Although NSM represents corrective movements and the feedback stage of the action, it was highly correlated with TMT (Spearman’s rho = 0.78, p < 0.0001). Hence, correlations with TMT and NSM for Verb LD and Priming are suggestive of representations that contain global aspects of the action. On the other hand, the SSJ tasks were correlated with RT and NSM, and in the case of verbs, with IDE. This suggests that simulations induced by the task requiring explicit access to, and comparison of, meanings may be somewhat different, and also contain information about action initiation. These results provide evidence not only that conceptual representations contain some form of motoric component that is intrinsic to the concept, but that these representations are context sensitive, and likely contain more details about the action when the semantic demands are higher. We note that these differences, although suggestive, remain speculative, due to the high correlation between all four parameters.
The context sensitivity sensory-motor system activation in processing action concepts has been demonstrated in a number of other studies (Labruna, Fernandez-del-Olmo, Landau, Duque, & Ivry, 2011; Lebois, Wilson-Mendenhall, & Barsalou, 2014; Papeo, Rumiati, Cecchetto, & Tomasino, 2012; Tomasino & Rumiati, 2013; van Dam, van Dijk, Bekkering, & Rueschemeyer, 2011). These are demonstrations that the semantic system is flexible, and concepts are fluid. Sometimes, however, modulation in activation of features is taken as evidence that those features are “peripheral” to the concept, and the central content of the concept is elsewhere. The assumption is that there is a “core” of each concept that is activated quickly and automatically to the same level regardless of task demands or context (Machery, 2007; Mahon & Caramazza, 2008; Whitney, McKay, Kellas, & Emerson, 1985). The current results are compatible with the idea that an action component is part of the core of action-related words, because correlation with at least one parameter of the reaching task was found across widely varying tasks. Concepts certainly have more and less salient features. It is not clear, however, that such immutable cores exist at all. Lebois et al. (2014) review extensive evidence that even classic and robust phenomena, such as the Stroop, SNARC, and Simon effects, perceptual and affective priming, as well as attention cuing, are susceptible to modulation in response to task demands. Of course, even if it is shown that such cores exist, is does not follow that this information is somehow represented with amodal symbols. If the action component of action concepts is peripheral, it raises the obvious questions as to what exactly the separate putative core consists of. For example, a key difference between throw and toss lies in the type of action, and if actions (and related visuospatial information such as speed and trajectory) are not in the core, exactly what allows us to distinguish between these and many other actions? Flexibility and modulation with context appear to a general property of a wide swath of cognitive phenomena, including concept representation and processing. Dynamic aspects of concepts cannot be assumed to be peripheral or somehow less important than some other unspecified parts of concepts. It is not clear, or even likely, that there are any non-dynamic parts of concepts at all.
While the results provide clear support for the idea that motor systems are causally involved in processing of action concepts, strong conclusions cannot be drawn based on the negative findings, namely, the lack of correlation between some of the reaching parameters and language tasks. First, we used a relatively basic task of reaching. It is possible that action representations leverage the higher level planning of complex actions (e.g., pantomiming actions, imitating tool use, interacting with objects). The reaching task, however, does allow us to examine a stronger hypothesis, namely whether action simulations of semantics of single words are affected by impairments in action performance at this relatively simpler level. Secondly, the robotic device requires movements only in the horizontal plane, simplifying the task somewhat, whereas real-world reaching actions typically require movements in three dimensions. While action language-motor correlations were found for the group, several patients in each case did not follow the pattern of the group. Apart from the noise inherent in assessing both language and motor performance, this can be due to at least two other factors. The motor circuit is highly complex, and a number of components are involved even in the simplest tasks. Compromised motor performance can result from impairment to any component of these circuits, all of which many not be equally involved in conceptual processing (e.g., cortical vs. subcortical). On the language side, there are likely individual differences in the nature of conceptual representations. This can only be revealed through even more detailed testing with a variety of motor and language tasks. Finally, all of the language tasks reported here involve processing single words. Effects of linguistic context may be investigated by examining words in sentence (or larger) contexts.
Conclusions
We show that stroke patients’ performance in a reaching task is selectively predictive of semantic processing of action concepts. This holds for semantic processing at subliminal, implicit, and explicit levels. This strongly supports the view that conceptual processing of actions is grounded in embodied simulations. Moreover, task demands appear to modulate the type of simulations, such that more details are simulated with more explicit semantic task demands. These results point to a tight and causal link between conceptual and sensory-motor systems of the brain.
Highlights.
Stroke patients tested on a manual reaching task and on language tasks
Action and abstract verbs and nouns tested in implicit and explicit semantic tasks
Reaching task performance predictive of selective impairment of action words
Suggests causal link between word comprehension and embodied simulations
Simulations are dynamic and change with task demands
Acknowledgments
We thank Leo Fernandino and Carl Brzorad for assistance with stimulus creation and analysis respectively. Supported by R01 DC010783 (RHD) and R01 DC009571 (JF & CR) from NIH/NIDCD.
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- Bak TH, O’Donovan DG, Xuereb JH, Boniface S, Hodges JR. Selective impairment of verb processing associated with pathological changes in Brodmann areas 44 and 45 in the motor neurone disease-dementia-aphasia syndrome. Brain. 2001;124:103–120. doi: 10.1093/brain/124.1.103. [DOI] [PubMed] [Google Scholar]
- Balota DA, Yap MJ, Cortese MJ, Hutchison KA, Kessler B, Loftis B, Neely JH, Nelson DL, Simpson GB, Treiman R. The English Lexicon Project. Behav Res Methods. 2007;39:445–459. doi: 10.3758/bf03193014. [DOI] [PubMed] [Google Scholar]
- Barsalou LW. Grounded cognition. Annu Rev Psychol. 2008;59:617–645. doi: 10.1146/annurev.psych.59.103006.093639. [DOI] [PubMed] [Google Scholar]
- Benjamini Y, Hochberg Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society Series B. 1995;57:289–300. [Google Scholar]
- Binder JR, Desai RH. The neurobiology of semantic memory. Trends in Cognitive Sciences. 2011:1–10. doi: 10.1016/j.tics.2011.10.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Boulenger V, Mechtouff L, Thobois S, Broussolle E, Jeannerod M, Nazir TA. Word processing in Parkinson’s disease is impaired for action verbs but not for concrete nouns. Neuropsychologia. 2008;46:743–756. doi: 10.1016/j.neuropsychologia.2007.10.007. [DOI] [PubMed] [Google Scholar]
- Buxbaum LJ, Saffran EM. Knowledge of object manipulation and object function: dissociations in apraxic and nonapraxic subjects. Brain Lang. 2002;82:179–199. doi: 10.1016/s0093-934x(02)00014-7. [DOI] [PubMed] [Google Scholar]
- Coderre AM, Zeid AA, Dukelow SP, Demmer MJ, Moore KD, Demers MJ, Bretzke H, Herter TM, Glasgow JI, Norman KE, Bagg SD, Scott SH. Assessment of upper-limb sensorimotor function of subacute stroke patients using visually guided reaching. Neurorehabil Neural Repair. 2010;24:528–541. doi: 10.1177/1545968309356091. [DOI] [PubMed] [Google Scholar]
- Cotelli M, Borroni B, Manenti R, Zanetti M, Arevalo A, Cappa SF, Padovani A. Action and object naming in Parkinson’s disease without dementia. Eur J Neurol. 2007;14:632–637. doi: 10.1111/j.1468-1331.2007.01797.x. [DOI] [PubMed] [Google Scholar]
- Druks J. Verbs and nouns - a review of the literature. Journal of Neurolinguistics. 2002;15:289–315. [Google Scholar]
- Dukelow SP, Herter TM, Bagg SD, Scott SH. The independence of deficits in position sense and visually guided reaching following stroke. J Neuroeng Rehabil. 2012;9:72. doi: 10.1186/1743-0003-9-72. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fernandino L, Conant LL, Binder JR, Blindauer K, Hiner B, Spangler K, Desai RH. Parkinson’s disease disrupts both automatic and controlled processing of action verbs. Brain and Language. 2012 doi: 10.1016/j.bandl.2012.07.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fernandino L, Conant LL, Binder JR, Blindauer K, Hiner B, Spangler K, Desai RH. Where is the action? Action sentence processing in Parkinson’s disease. Neuropsychologia. 2013;51:1510–1517. doi: 10.1016/j.neuropsychologia.2013.04.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fischer MH, Zwaan RA. Embodied language: A review of the role of the motor system in language comprehension. The Quarterly Journal of Experimental Psychology. 2008;61:825–850. doi: 10.1080/17470210701623605. [DOI] [PubMed] [Google Scholar]
- Fodor JA. The Modularity of Mind: An Essay on Faculty Psychology. Cambridge, MA: MIT Press; 1983. [Google Scholar]
- Gallese V, Lakoff G. The brain’s concepts: The role of the sensory-motor system in conceptual knowledge. Cognitive Neuropsychology. 2005;22:455–479. doi: 10.1080/02643290442000310. [DOI] [PubMed] [Google Scholar]
- Glascher J, Tranel D, Paul LK, Rudrauf D, Rorden C, Hornaday A, Grabowski T, Damasio H, Adolphs R. Lesion mapping of cognitive abilities linked to intelligence. Neuron. 2009;61:681–691. doi: 10.1016/j.neuron.2009.01.026. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Grossman M, Anderson C, Khan A, Avants B, Elman L, McCluskey L. Impaired action knowledge in amyotrophic lateral sclerosis. Neurology. 2008;71:1396–1401. doi: 10.1212/01.wnl.0000319701.50168.8c. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Herter TM, Scott SH, Dukelow SP. Systematic changes in position sense accompany normal aging across adulthood. J Neuroeng Rehabil. 2014;11:43. doi: 10.1186/1743-0003-11-43. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hoffman P, Rogers TT, Lambon Ralph Ma. Semantic diversity accounts for the “missing” word frequency effect in stroke aphasia: insights using a novel method to quantify contextual variability in meaning. Journal of Cognitive Neuroscience. 2011;23:2432–2446. doi: 10.1162/jocn.2011.21614. [DOI] [PubMed] [Google Scholar]
- Ibanez A, Cardona JF, Dos Santos YV, Blenkmann A, Aravena P, Roca M, Hurtado E, Nerguizian M, Amoruso L, Gomez-Arevalo G, Chade A, Dubrovsky A, Gershanik O, Kochen S, Glenberg A, Manes F, Bekinschtein T. Motor-language coupling: direct evidence from early Parkinson’s disease and intracranial cortical recordings. Cortex. 2013;49:968–984. doi: 10.1016/j.cortex.2012.02.014. [DOI] [PubMed] [Google Scholar]
- Kertesz A. The Western Aphasia Battery. New York: Grune and Stratton; 1982. [Google Scholar]
- Kiefer M, Pulvermüller F. Conceptual representations in mind and brain: Theoretical developments, current evidence and future directions. Cortex; a journal devoted to the study of the nervous system and behavior. 2011;8 doi: 10.1016/j.cortex.2011.04.006. [DOI] [PubMed] [Google Scholar]
- Labruna L, Fernandez-del-Olmo M, Landau A, Duque J, Ivry RB. Modulation of the motor system during visual and auditory language processing. Exp Brain Res. 2011;211:243–250. doi: 10.1007/s00221-011-2678-z. [DOI] [PubMed] [Google Scholar]
- Lebois LA, Wilson-Mendenhall CD, Barsalou LW. Are Automatic Conceptual Cores the Gold Standard of Semantic Processing? The Context-Dependence of Spatial Meaning in Grounded Congruency Effects. Cogn Sci. 2014 doi: 10.1111/cogs.12174. [DOI] [PubMed] [Google Scholar]
- Machery E. Concept empiricism: a methodological critique. Cognition. 2007;104:19–46. doi: 10.1016/j.cognition.2006.05.002. [DOI] [PubMed] [Google Scholar]
- Mahon BZ, Caramazza A. A critical look at the embodied cognition hypothesis and a new proposal for grounding conceptual content. Journal of Physiology-Paris. 2008;102:59–70. doi: 10.1016/j.jphysparis.2008.03.004. [DOI] [PubMed] [Google Scholar]
- Meteyard L, Cuadrado SR, Bahrami B, Vigliocco G. Coming of age: A review of embodiment and the neuroscience of semantics. Cortex; a journal devoted to the study of the nervous system and behavior. 2010;48:788–804. doi: 10.1016/j.cortex.2010.11.002. [DOI] [PubMed] [Google Scholar]
- Neininger B, Pulvermuller F. Word-category specific deficits after lesions in the right hemisphere. Neuropsychologia. 2003;41:53–70. doi: 10.1016/s0028-3932(02)00126-4. [DOI] [PubMed] [Google Scholar]
- Newell A, Simon HA. Computer science as empirical inquiry: Symbols and search. Communications of the ACM. 1976;19:113–126. [Google Scholar]
- Papeo L, Rumiati RI, Cecchetto C, Tomasino B. On-line changing of thinking about words: the effect of cognitive context on neural responses to verb reading. J Cogn Neurosci. 2012;24:2348–2362. doi: 10.1162/jocn_a_00291. [DOI] [PubMed] [Google Scholar]
- Pylyshyn ZW. Computation and Cognition. Cambridge, MA: MIT Press; 1984. [Google Scholar]
- Scott SH. Apparatus for measuring and perturbing shoulder and elbow joint positions and torques during reaching. J Neurosci Methods. 1999;89:119–127. doi: 10.1016/s0165-0270(99)00053-9. [DOI] [PubMed] [Google Scholar]
- Scott SH, Dukelow SP. Potential of robots as next-generation technology for clinical assessment of neurological disorders and upper-limb therapy. J Rehabil Res Dev. 2011;48:335–353. doi: 10.1682/jrrd.2010.04.0057. [DOI] [PubMed] [Google Scholar]
- Tillotson SM, Siakaluk PD, Pexman PM. Body-object interaction ratings for 1,618 monosyllabic nouns. Behav Res Methods. 2008;40:1075–1078. doi: 10.3758/BRM.40.4.1075. [DOI] [PubMed] [Google Scholar]
- Tomasino B, Rumiati RI. At the mercy of strategies: the role of motor representations in language understanding. Front Psychol. 2013;4:27. doi: 10.3389/fpsyg.2013.00027. [DOI] [PMC free article] [PubMed] [Google Scholar]
- van Dam WO, van Dijk M, Bekkering H, Rueschemeyer SA. Flexibility in embodied lexical-semantic representations. Human brain mapping. 2011;33:2322–2333. doi: 10.1002/hbm.21365. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Vigliocco G, Vinson DP, Druks J, Barber H, Cappa SF. Nouns and verbs in the brain: a review of behavioural, electrophysiological, neuropsychological and imaging studies. Neurosci Biobehav Rev. 2011;35:407–426. doi: 10.1016/j.neubiorev.2010.04.007. [DOI] [PubMed] [Google Scholar]
- Whitney P, McKay T, Kellas G, Emerson WA., Jr Semantic activation of noun concepts in context. J Exp Psychol Learn Mem Cogn. 1985;11:126–135. doi: 10.1037//0278-7393.11.1.126. [DOI] [PubMed] [Google Scholar]