Effects of motion speed in action representations

Abstract

Grounded cognition accounts of semantic representation posit that brain regions traditionally linked to perception and action play a role in grounding the semantic content of words and sentences. Sensory-motor systems are thought to support partially abstract simulations through which conceptual content is grounded. However, which details of sensory-motor experience are included in, or excluded from these simulations, is not well understood. We investigated whether sensory-motor brain regions are differentially involved depending on the speed of actions described in a sentence. We addressed this issue by examining the neural signature of relatively fast (The old lady scurried across the road) and slow (The old lady strolled across the road) action sentences. The results showed that sentences that implied fast motion modulated activity within the right posterior superior temporal sulcus and the angular and middle occipital gyri, areas associated with biological motion and action perception. Sentences that implied slow motion resulted in greater signal within the right primary motor cortex and anterior inferior parietal lobule, areas associated with action execution and planning. These results suggest that the speed of described motion influences representational content and modulates the nature of conceptual grounding. Fast motion events are represented more visually whereas motor regions play a greater role in representing conceptual content associated with slow motion.

Introduction

Recent theories of human cognition argue for a tight coupling between perceptual and representational systems. Proposals differ from each other in the exact role they ascribe to perceptual and motor brain regions in conceptual representation (for a review see: Meteyard, Cuadrado, Bahrami & Vigliocco, 2012), but they all share the notion that sensorimotor experiences subserve cognition. On this view, experiential traces stored in sensorimotor brain regions can provide a means for grounding lexical-semantic content (Barsalou, 2008; Glenberg, 1997; Pulvermüller, 1999).

Several lines of evidence support the grounded cognition view. Behavioral studies have provided evidence that language comprehension shares computational processes with perception and action (for reviews see Fischer & Zwaan, 2008; Zwaan & Kaschak, 2008). Similarly, neuroimaging studies have provided evidence that language comprehension involves the recruitment of sensory-motor brain regions. Several studies have shown that the comprehension of action verbs (Hauk, Johnsrude, & Pulvermüller, 2004; Kemmerer, Castillo, Talavage, Patterson, & Wiley, 2008; Van Dam, Rueschemeyer, & Bekkering, 2010), action sentences (Tettamanti et al., 2005; Desai et al., 2009, 2011, 2013; Aziz-Zadeh, Wilson, Rizzolatti, & Iacoboni, 2006) and words denoting manipulable objects (Saccuman et al., 2006; Chao & Martin, 2000; Rueschemeyer, Rooij, Lindemann, Willems, & Bekkering, 2010) reliably activate the cerebral motor system. In a similar vein, neuroimaging studies have provided evidence that comprehension of words semantically related to color (Simmons et al., 2007; Martin, Haxby, Lalonde, Wiggs, & Ungerleider, 1995), odor (Gonzalez et al., 2006) and audition (Kiefer, Sim, Herrnberger, Grothe, & Hoenig, 2008), draw on brain regions relevant for coding corresponding modalities (see Binder and Desai 2011 for a review).

A number of studies on patients with motor impairments corroborate these findings. In two studies with Parkinson’s patients, Fernandino and colleagues (2012, 2013) showed that patients (in contrast to controls) were selectively impaired in automatic and controlled processing of action verbs and sentences. Neininger and Pulvermüller (2003) showed that patients with predominant right frontal lesions were impaired in processing of action verbs, whereas patients with right temporo-occipital lesions showed impairments in processing nouns with strong visual associations. Trumpp and colleagues (2013) showed that damage to the left auditory association cortex led to selective impairments in the processing of sound-related concepts (e.g., “bell”). Bonner & Grossman (2012) demonstrated that reduced gray matter density in patients with auditory association cortex atrophy was correlated with the severity of their deficit in processing sound-related words. Desai et al. (2015) showed that the degree of selective impairment in comprehension accuracy of action-related words was predicted by the degree of impairment in reaching performance in a group of chronic stroke patients. These findings provide strong evidence for a role of sensory-motor brain regions in language comprehension.

Despite the fact that numerous studies have reported sensory-motor activations in conjunction with language, the exact nature of these activations remains unclear. Activation that was specific to the semantic category presented has been documented as early as ~200 milliseconds after word onset (Pulvermüller, Härle & Hummel, 2000; Hauk & Pulvermüller, 2004), when subjects did not attend to a word (Pulvermüller & Shtyrov, 2006; Shtyrov, Hauk, & Pulvermüller, 2004) and for action verbs embedded within abstract sentences (Boulenger et al., 2009). These findings have been taken to support the notion that embodied lexical-semantic effects are automatic and invariant. Recent studies, however, have provided evidence that the overlap observed in modality-specific sensory-motor processing and conceptual processing depends on contextual constraints (Van Dam et al., 2012; Hoenig et al., 2008; Kalénine et al., 2012; Lee et al., 2013; Yee & Thompson-Schill, 2016). The interaction between modality-specific sensory-motor regions and conceptual areas might even be more intricate. It is generally assumed that overlap in the computational mechanisms that underlie conceptual and sensory-motor processing reflect bottom-up sensory-motor effects on higher-order cognition. However, factors like language-mediated categories can constrain sensory perception (Brouwer & Heeger, 2013; Puri, Wojciulik, & Ranganath, 2009; Lee & Noppeney, 2014), suggesting that shared computational processes might partially reflect top-down effects of linguistic priors on perception (Simanova, Francken, de Lange & Bekkering, 2016).

Another crucial issue in theories of embodiment concerns the precise nature of conceptual grounding. Which details of sensory-motor experience are included in, or excluded from these simulations? Several studies have provided evidence that the sensory-motor information activated during language comprehension is fairly specific in nature. Mental simulations seem to encode effector-specific information (Scorolli & Borghi, 2007; Hauk, Johnsrude & Pulvermüller, 2004), the amount of detail in which movement kinematics are specified by a verb (e.g., to wipe vs. to clean: Van Dam, Rueschemeyer, & Bekkering, 2010), the directionality of the action an object affords (Rueschemeyer, Pfeiffer & Bekkering, 2010; Zwaan & Taylor, 2006), implied orientation of an object (Zwaan, Stanfield & Yaxley, 2002), part of the visual field where a described scene would take place (Richardson, Spivey, Barsalou, & McRae, 2003; Bergen, Lindsay, Matlock & Narayanan, 2007) and direction of motion (Zwaan, Madden, Yaxley & Aveyard, 2004). For example, Zwaan and colleagues (2004) had participants decide whether two sequentially presented visual objects were identical or not, while concurrently listening to sentences that implied a movement in a certain direction. Crucially, either the first or second object would be depicted larger and therefore suggesting motion of the object towards or away from the observer. Participants were faster to respond if the direction of the movement implied by the sentence matched the direction suggested by the sequence in which the two pictures were presented. These findings provide evidence that language-induced perceptual and motor simulations contain at least some details about objects and the actions they afford.

The abovementioned demonstrations of strong overlap in the computational mechanisms that underlie conceptual and sensory-motor processing lay at the core of any grounded cognition account. There has also been acknowledgment of the necessity of some degree of abstraction away from sensory-motor processes (Binder, 2016; Barsalou, 2016; Binder & Desai, 2011), and concepts are therefore not entirely reducible to modality-specific sensory or motor representations. In order for a grounded cognition account to be successful and move forward, it is important to detail in which exact ways sensory and motor representations contribute to language processing. Interesting open questions are: How extensive is the overlap in neural pathways involved in modality-specific sensory-motor processing and conceptual processing? Is detailed information incorporated more from certain sensory-motor modalities than others during language-induced simulations?

The role that abstraction away from detailed sensory-motor information plays in sensory-motor grounding can provide us with viable insights in this regard. The mechanism of abstraction might be able explain why in many instances conceptual processing is similar to but does not equate with sensory-motor processing. Hsu and colleagues (2011) demonstrated that the extent to which primary perceptual regions were activated in a color judgment task depended on the degree of perceptual resolution needed by the task. If the task context required retrieval of detailed color knowledge the neural response was highly similar to that observed in color perception, the response in color perception regions was significantly reduced if the task could be accomplished on the basis of categorical (abstracted) knowledge. On the basis of these types of findings, several authors have proposed that conceptual knowledge may be represented at multiple levels of abstraction (Binder & Desai, 2011; Thompson-Schill, 2003). In this view, conceptual processing draws heavily on association areas involved in integration and abstraction in addition to utilizing sensory-motor representations to flesh out particular concepts (Binder 2016; Simmons & Barsalou, 2003). As detailed above, task and linguistic context are likely important factors in determining the extent to which sensory-motor processes are recruited during conceptual processing, and therefore the extent to which representations and simulations mirror objects, actions and events in the real world (see also Van Dam et al., 2012; Hoenig et al., 2008 on the related topic of conceptual flexibility).

An interesting topic in this regard is (1) whether a relatively fine-grained parameter of an action like movement speed is incorporated in language-induced perceptual and motor simulations, and if so, (2) what exact information is activated. Behavioral studies have provided ample evidence that mental simulations during language processing are affected by the motion dynamics of the motor experiences on which they are based. Meteyard and colleagues (2007), for example, showed that listening to verbs that denoted upward or downward motion affected perceptual sensitivity to motion, performance in a motion-detection task and subject’s internal decision criteria. Futhermore, recent behavioral studies have indicated mental simulations for speed during language processing. In a recent study, Speed and Vigliocco (2014) had participants listen to sentences describing fast and slow actions (e.g., The lion dashed to the balloon vs. The lion ambled to the balloon). They found that participants showed slower eye movements and longer looking times towards a concurrent visual scene while listening to slow actions as compared to fast actions. Lindsay and colleagues (2013) presented participants with sentences while concurrently looking at a scene depicting an agent and a path that led to a goal object. Subjects looked more often and longer along the path to the goal if the verb implied a slow manner of motion (e.g., to dawdle), whereas they looked earlier at the goal and less along the path if the verb implied a fast manner of motion (e.g., to dash). These findings suggest that speed-of-motion parameters are incorporated in mental simulations during language comprehension (see also Matlock, 2004).

While no neuroimaging studies have examined the simulation of speed of motion, there is strong neuroimaging evidence that sensory-motor simulations in language comprehension capture motion-related aspects of meaning, there is strong neuroimaging evidence that sensory-motor simulations in language comprehension capture motion-related aspects of meaning (Saygin et al., 2010; Humphreys et al., 2013). Several studies have demonstrated involvement of area MT/V5 in processing motion-related language (Saygin et al., 2010; Rueschemeyer et al., 2010), a region that has shown to be critically involved in low-level motion perception (Beauchamp et al., 2002; Dumoulin et al., 2000; Tootell et al., 1995). Other studies have found that motion-related language activates regions within the posterior STS/pMTG (Kable et al., 2002, 2005; Bedny et al., 2008; Humphreys et al., 2013). These more posterior regions might not be as critical for processing of form and motion in general, but play a more specific role in the processing of biological motion (Vaina & Gross, 2004; Humphreys et al., 2013; Grossman, Batelli, & Pascual-Leone, 2005).

The amount of detail with which speed-of-motion parameters are incorporated in and the sensory-motor regions involved in these simulations remains unclear. Area MT+ is a likely candidate given that it has shown to be critically involved in low-level motion perception (Beauchamp et al., 2002; Dumoulin et al., 2000), and activity in this region has shown to be modulated by the speed of motion (Kawakami et al., 2002; Amano, Kimura, Nishida, Takeda, & Gomi, 2009). It has, however, been shown that the processing of biological motion (i.e., a more specific type of visual motion) relies on a region anterior and superior to low-level motion area MT+. Simulations in comprehension of language that denotes fast vs. slow human movements might therefore rely more strongly on posterior regions of the STS that have similarly shown to be sensitive to speed in biological motion perception (Giese & Lappe, 2002; Jacobs, Pinto, & Shiffrar, 2004). It might, however, be a possibility that speed in biological motion perception is not processed within the posterior STS directly, but relies on low-level signals from motion-sensitive areas (for a discussion see Beintema, Georg, & Lappe, 2006). Given that movement velocity is tightly associated with the amount of effort that needs to be exerted in performing an action, language describing fast and slow human action might similarly draw on regions involved in control and timing of motor movements (Guillot et al., 2009; Michelon, Vettel, & Zacks, 2006).

In summary, there is strong evidence for a role of sensory-motor brain areas in comprehension of motion-related language, while the exact nature of and information recruited in these language-induced simulations is a topic that needs further investigation. Whether sensory-motor simulations during the processing of motion-related language include information about the “speed of motion” and through which neural mechanisms is a question that is addressed here. In the current study, we asked whether motion speed, implied in action sentences, affects representations of sentence meaning by directly testing whether activity in motor and motion processing regions is modulated by whether a linguistic phrase implies a slow or fast movement. We measured changes in the hemodynamic response of participants while they read sentences that either described relatively slow or fast actions. First, if mental simulations during language processing incorporate the exact motion dynamics of the motor experiences on which they are based, then, slow and fast action sentences might differentially recruit regions that have shown to play a role in motion processing, specifically within the posterior superior temporal sulcus (pSTS) and motion perception area MT+ (Rees, Friston & Koch, 2000). As mentioned earlier, the pSTS has shown to play a role in the processing of biological motion (Kourtzi & Kanwisher, 2000; Allison, Puce & McCarthy, 2000; Grossman et al., 2000; Vaina, Solomon, Chowdhury, Sinha, & Belliveau, 2001; Beauchamp, Lee, Haxby, & Martin, 2002). MT+ and surrounding regions are commonly found in accessing conceptual information about motion attributes (Kable et al., 2005; Saygin, McCullough, Alac, & Emmory, 2010; Rueschemeyer, Glenberg, Kaschak, Mueller, & Friederici, 2010). Second, regions that are involved in the execution of motor movements and representation of action plans are also likely to play a role in processing sentences that describe dynamic events, given that movement velocity is strongly linked to the amount of effort exerted in human action performance. Here, we examined the primary motor cortex and the anterior inferior parietal lobule. The latter region has been shown to be involved in the execution of complex motor movements, tool use and the representation of action plans (Brandi, Wohlschläger, Sorg, & Hermsdörfer, 2014; Graziano, Taylor, & Moore, 2002; Lewis, 2006; Rumiati et al., 2004) and is frequently found to be activated by action/tool semantics relative to non-action semantics (Desai et al. 2009, 2011, 2013; Noppeney, Josephs, Kiebel, Friston, & Price, 2005). The directionality of a potential effect of motion dynamics of actions within motor regions is unclear. On the one hand, an increase in effort typically results in a similar increase in speed of movement. However, slower actions can also reflect greater exertion and greater need of control and action planning. Thus, we examined regions involved in visual perception of action/motion, and those involved in their execution, to examine if their activity is modulated by the implied speed of motion described in sentences.

METHODS

Participants

36 individuals participated in the study, all of who were right-handed and between 19 and 27 years of age (M = 22.42, SD = 2.02; 13 males). The data from two participants were excluded because of excessive motion. All participants had normal or corrected-to-normal vision and no history of neurological disorders. Prior to the experiment participants were informed about the experimental procedures, signed informed consent forms according to a protocol sanctioned by the Institutional Review Board of the University of South Carolina. Participants received course credit for their participation.

Stimuli

The three main experimental conditions contained 14 sentences each. They were: (1) Slow Action Sentences (e.g., “The professor sneaked down the corridor”; “Chris shuffled out the classroom”), (2) Fast Action Sentences (e.g., “The professor stormed down the corridor”; “Chris bolted out the classroom”), (3) Abstract Sentences (e.g., “John deceived all of his friends”; “Sophie admired her old history teacher”). Only the main verb differed between Fast and Slow action sentences. The fast, slow, and abstract verbs were rated for motion speed (see below). Additionally, false font sentences were also presented, which we do not use here.

The sentences of all experimental conditions were matched on a number of psycholinguistic variables (see Table 1). A three-way ANOVA ensured that all the main verbs used in these sentences were matched on number of letters, number of phonemes and number of syllables (all ps > 0.05). We used the English Lexicon Project Database (Balota et al., 2007) to ensure that all words were matched on log frequency, lexical decision RT, lexical decision accuracy and naming RT (all ps > 0.05). In addition, seven participants, who did not participate in the imaging experiment, rated the speed implied by a verb on a scale of 1 (word denotes slow speed) to 7 (word denotes a fast speed), with the possibility of indicating ‘no speed’ (coded as 0) as well: Abstract sentences (M = 0.41, SD = 0.69); Slow Action sentences (M = 2.61, SD = 0.44) and Fast Action sentences (M = 5.32, SD = 0.90). Paired-sample t-tests showed that Slow Action sentences had higher speed ratings than Abstract sentences and Fast Action sentences had higher speed ratings than Slow Action sentences (all ps < 0.001).

Table 1.

Psycholinguistic variables

	Abstract sentences	Slow Action sentences	Fast Action sentences
Speed association	0.41 (0.69)	2.61 (0.44)	5.32 (0.90)
# Letters	6.93 (0.83)	7.29 (0.73)	6.93 (0.83)
# Phonemes	5.50(0.94)	5.57(0.76)	5.71(0.73)
# Syllables	1.64(0.50)	1.57(0.51)	1.57(0.51)
Sentence # words	6.21 (0.97)	6.21 (0.97)	6.21 (0.97)
Sentence # letters	31.9 (5.1)	29.0 (3.7)	29.5 (3.7)
Log decision freq	6.56(1.32)	6.12(1.47)	6.12(1.32)
# orthographic neighbours	2.86(1.46)	2.57(1.45)	2.79(1.58)

Procedure

Two blocks were presented for each experimental condition, each block lasting 16 seconds, separated by 16 seconds of rest. Each block consisted of the presentation of seven sentences belonging either to the abstract, slow action, fast action condition. Block order was randomized individually for each participant. Each sentence in the block was presented in its entirety, in a white Arial font on a black background, for 2250 msec. Participants were instructed to read all sentences without making any responses, avoiding motor execution after every trial.

Additionally, a motor localizer task was included. Here, subjects performed a sequence of movements (make fist, open hand, touch pinkie and thumb) with either the right hand, left hand, or rested, in blocks of 16 seconds. Two blocks each for the left and right hand were included. Instructions ‘R’, ‘L’, were displayed for 2 sec before each motor block, ‘rest’ before each resting block, and ‘read’ before sentence blocks.

fMRI data acquisition

Functional images were acquired on a Siemens TRIO 3.0 T MRI system (Siemens, Erlangen, Germany) equipped with a 12-channel head coil. BOLD-sensitive functional images were acquired using a single-shot gradient EPI sequence (echo time = 34 msec, repetition time = 1200 msec, 36 axial slices in ascending order, slice gap = 0.60 mm, field of view = 212 mm, flip angle = 65 degrees, voxel size = 3.3 × 3.3 × 3.6 mm³). High-resolution anatomical images were acquired using an MPRAGE sequence (echo time = 4.15 msec, voxel size = 1 × 1 × 1 mm³, 192 sagittal slices, field of view = 256).

fMRI Data Analysis

Functional data were preprocessed and analyzed with the AFNI software package (Cox, 1996). A standardized preprocessing pipeline involved registration to the anatomy (Saad et al., 2009), co-registration (Cox & Jesmanowicz, 1999), and projection into standard stereotaxic space (Talairach and Tournoux, 1988). An isotropic Gaussian kernel with 6-mm FWHM was applied to the normalized images, and they were converted to percent signal change. The ensuing preprocessed fMRI time series were analyzed on a subject-by-subject basis using an event-related approach in the context of voxel-wise multiple linear regression with regressors for each condition (Fast Action, Slow Action, Abstract, False Font) convolved with a canonical hemodynamic response function. Six motion parameters, the signal extracted from the ventricles, and a regressor coding the instructions before each block were included as covariates of no interest. General linear tests were conducted to obtain the effects of concreteness and speed. While the main contrast of interest was fast vs. slow motion we also examined each sentence type compared to abstract sentences.

In a random effects analysis, group maps were created by comparing activations against a constant value of 0. The group maps were thresholded at voxelwise p < 0.01 and corrected for multiple comparisons to achieve a mapwise corrected p < 0.05. We used 3dFWHMx to calculate the autocorrelation function (ACF) parameters for each individual subject. Subsequently, the median of all subjects was used in running the 3dClustSim program with 10000 iterations (see Cox, Reynolds, & Taylor, 2016). The cluster threshold was determined through Monte Carlo simulations that estimate the chance probability of spatially contiguous voxels exceeding the voxelwise p threshold. The analysis was restricted to a mask that excluded areas outside the brain, as well as deep white matter areas and the ventricles. This mask is based on the probabilistic Desikan-Killiany atlas that contains 35 cortical areas in each hemisphere (Desikan et al., 2006), and the subcortical parcellation provided by FreeSurfer. The mask was created using TT_desai_dkpmaps atlas provided with AFNI.

Regions of Interest

We defined several regions of interest (ROIs) that have shown to be involved in visual-motion-related and motor processing, in order to obtain greater sensitivity in these regions. Given its association with motion processing, we defined a bilateral ROI in motion area MT+, using the mean coordinates provided by Saygin and colleagues (2010). These coordinates were [left MT+:×= −44, y = −72, z = 5] and [right MT+:×= 43, y = −69, z = 3].

Another area central to biological motion perception is posterior superior temporal sulcus (pSTS) (Allison, Puce, & McCarthy, 2000; Grossman et al., 2000; Grosbras, Beaton, & Eickhoff, 2012; Peuskens, Vanrie, Verfaille, & Orban, 2005; Beauchamp et al., 2003). We defined a bilateral ROI within the pSTS on the basis of the Beauchamp et al. (2003) study that investigated neural responses to complex visual motion. Coordinates for the contrast of humans performing wholebody motions versus characteristic natural motion of manipulable objects were used (left pSTS:×= −39, y = −59, z = 15; right pSTS:×= 47, y = −56, z = 15).

The primary motor cortex ROI was defined for each participant individually, on the basis of left minus right hand contrast in the motor localizer task. The left motor cortex was defined by the right > left hand contrast (average left M1:×= −28, y = −25, z = 47), and similarly the right motor cortex by the left > right contrast (average right M1:×= 32, y = −25, z = 53).

Lastly, a ROI was defined in the anterior inferior parietal lobule, a higher-order motor area that has been shown to play a role in hand-object interactions and tool use, and is frequently found in studies of action/tool semantics. The coordinates were taken from a study by Desai and colleagues (2009), from the action > abstract sentence contrast (left aIPL:×= −57, y = −32, z = 32; right aIPL:×= 57, y = −32, z = 32). Spheres of 5 mm in radius were defined around each of these coordinates and parameter estimates for both the Fast and Slow action sentences were extracted from all voxels in the ROI. The mean parameter estimates for the ROIs were entered as dependent variables in a two-tailed paired sample t-test across the group.

Results

The activation maps for the different contrasts are shown in Figure 1. Cluster information, strength and location of peak activations are reported in Table 2.

Figure 1.

Caret software (Van Essen et al., 2001) was used to display the maps on an inflated cortical surface of a representative subject, created through FreeSurfer (Dale, Fischl, & Sereno, 1999). Activations for (a) Fast+Slow-Abstract, (b) Fast-Abstract, (c) Slow-Abstract, (d) Fast-Slow contrasts. Red-orange colors denote positive values, blue-cyan colors denote negative values. Activations are projected on an inflated surface of a brain. Gyri are shown in light gray and sulci in dark gray. L = left hemisphere, R = right hemisphere. F = fast action, S = slow action, A = Abstract.

Table 2.

The volume of the cluster (µl), peak z-score, Talairach coordinates, and the anatomical structures that the clusters overlap are shown. L = left hemisphere, R = right hemisphere, g = gyrus, s = sulcus, ant = anterior, post = posterior, sup = superior, mid = middle, inf = inferior.

Volume	Max	x	y	z	Structure
Fast + Slow Actions > Abstract
11637	4.42	−37	46	11	L mid/inf front g
10584	4.81	−37	−58	35	L angular g
4590	4.49	46	31	26	R mid/inf front g
2160	3.47	−7	−67	35	L precuneus
1728	3.66	−4	28	38	L sup front g
3807	3.70	34	−64	38	R angular g
Abstract > Fast + Slow Actions
10368	−5.19	−55	−4	−3	L sup temp g; sup temp s
5562	−3.89	−34	−61	−6	L occ g, s
2079	−4.31	49	−7	−9	R sup temp s
2052	−3.81	28	40	38	R mid front g
1431	−3.43	40	−76	20	R mid occ g
Fast Actions > Abstract
10368	4.37	−37	−61	38	L angular g
5643	4.18	34	−67	38	R angular g
4995	4.33	−46	22	41	L mid front g
4644	3.87	−40	49	−6	L inf front g (pars orbitalis)
3024	4.17	−4	28	38	L sup front g
2322	3.62	−13	−61	23	L precuneus
2295	3.60	49	28	35	R mid front g
1404	3.12	43	7	50	R mid front g
Abstract > Fast Actions
8586	−4.45	−58	−1	−18	L mid/sup temp g, sup temp s
1458	−3.78	34	34	38	R mid front g
Slow Actions > Abstract
6615	3.90	−40	−49	35	L angular g
5562	4.33	−37	43	8	L mid front g
2700	3.44	−37	−16	2	L insula
Abstract > Slow Actions
9072	−5.04	−55	−4	−3	L sup temp s, mid temp g
6813	−3.75	−37	−64	−6	L inf occ g, s
4482	−3.58	58	−58	2	R mid temp g
3240	−4.40	49	−4	−9	R sup temp s
Fast Actions > Slow Actions
3348	3.56	37	−70	38	R angular g / posterior sup temp s (spreading into mid/sup occ g)

Action (Fast + Slow) - Abstract

The whole brain analysis revealed that areas activated to a greater extent by Action sentences relative to Abstract sentences included the bilateral middle frontal gyri (MFG), bilateral angular gyri (AnG), bilateral inferior frontal gyri (IFG: pars triangularis), and the left superior frontal gyrus (SFG), as well as precuneus. The Abstract sentences relative to the Action sentences led to stronger activation in the bilateral superior temporal sulci (STS), right MFG and middle occipital gyrus (MOG), as well as the left superior temporal gyrus (STG) and occipital gyrus/sulcus (Fig. 1A; Table 2).

Fast Action – Abstract

Fast Action sentences elicited greater levels of activation than Abstract sentences within the bilateral MFG, bilateral AnG, bilateral precuneus, left IFG (pars orbitalis) and SFG. The Abstract sentences elicited greater levels of activation than Fast Action sentences within the left MTG/STG and STS, as well as the right MFG (Fig. 1B; Table 2).

Slow Action – Abstract

Slow Action sentences elicited greater levels of activation than Abstract sentences within the left MFG, AnG and insula. The Abstract sentences relative to the Slow Action sentences led to stronger activation in the bilateral STS, left MTG, as well as the inferior occipital gyrus/sulcus (Fig. 1C; Table 2).

Fast Action – Slow Action

Fast Action sentences elicited greater levels of activation than Slow Action sentences within the right Angular gyrus and pSTS (extending into SOG/MOG). No areas showed stronger activation for Slow Action sentences than Fast Action sentences (Fig. 1D; Table 2).

ROI Analysis

In the right M1, slow action sentences elicited more activation than fast action sentences, t(1,33) = 2.56, p < 0.02. No difference was obtained between slow and fast action sentences within the left M1, t(1,33) = 0.48, p > 0.50. In the right aIPL, slow action sentences elicited more activation than fast action sentences, t(1,33) = 2.83, p < 0.01. In the left aIPL we obtained a marginally significant result, with slow action sentences eliciting stronger activation than fast action sentences, t(1,33) = 1.75, p = 0.09. No difference was obtained between fast and slow action sentences within either the right or left MT+, all ps > 0.50. In the right pSTS, fast action sentences elicited more activation than slow action sentences, t(1,33) = 2.16, p < 0.05. No difference was obtained between fast and slow action sentences within the left pSTS, t(1,33) = 1.08, p > 0.25.

Discussion

The current experiment explored the differences in sensory-motor activation for sentences that describe fast versus slow motion.

Fast sentences elicited greater levels of activation than slow action sentences in the right pSTS both at the whole brain level and in the region of interest analysis. The pSTS has been shown to play a role in the perception of bodies and body movements (Allison, Puce & McCarthy, 2000; Grossman et al., 2000; Puce, Allison, Bentin, Gore, & McCarthy, 1998; Vaina et al., 2001; Beauchamp et al., 2002). Humphreys and colleagues (2013) showed that processing of motion versus static sentences elicited activation in the pSTS, which specifically overlapped with the contrast of human motion > human static (and not object motion > object static), corroborating the idea that this region is sensitive to biological motion. Patients with damage to the STS show impairments in biological motion recognition despite intact form and motion processing (Vaina & Gross, 2004). Similarly, disruption of pSTS activity by transcranial magnetic stimulation (TMS) impairs the perception of biological motion (Grossman, Batelli, & Pascual-Leone, 2005). Furthermore, activity in the pSTS seems to be modulated by the configuration of the stimulus. Grossman and Blake (2002) obtained stronger activation for point-light walkers than for spatially scrambled walkers. Similarly, Grossman and Blake (2001) obtained stronger neural activity within the pSTS for canonical (upright) point-light walkers as compared with inverted (upside-down) point-light walkers. Several studies have shown that speed of motion is a crucial factor that influences biological motion perception. For example, gait speed influences gender recognition (Barclay, Cutting, & Kozlowski, 1978), subjects are able to indicate which speeds appear natural (Giese & Lappe, 2002) and identity discrimination is hampered by unnatural low gait speed (Jacobs, Pinto, & Shiffrar, 2004). Disturbing the phase relations between points in point-light displays also impairs biological motion perception (Bertenthal & Pinto, 1994). These findings provide evidence for a role of global motion and speed of motion information in processing biological motion. It remains unclear whether global motion and speed of motion is processed directly within the STS or relies on input from motion-sensitive areas. Surprisingly, there is no direct evidence that the processing of speed in biological motion perception depends on low-level signals from motion areas (for a discussion see Beintema, Georg, & Lappe, 2006). Furthermore, inactivation or damage to motion processing area MT does not seem to interfere with biological motion perception (Vaina, Lemay, Bienfang, Choi, & Nakayama, 1990; Grossman et al., 2005). These findings suggest that besides stimulus configuration, global motion and orientation information, the STS itself might process speed information.

Stronger activation for fast action than for slow action sentences at the whole brain level within the right pSTS extended into the right SOG/MOG. Fast action sentences as compared to abstract sentences also led to a similar pattern of activation as slow action sentences, with additional activation in the left precuneus. Both the superior/middle occipital regions and precuneus play a role in visual imagery of movements, whereas kinesthetic imagery of the same movements activates motor-related brain regions (Guillot et al., 2009). Desai and colleagues (2009) observed activation of the SOG/MOG for sentences with visual verbs (You see the rope) compared with sentences with abstract verbs (They consider the risk), providing evidence for it’s association with visual semantics and visually guided actions.

Interestingly, the reverse effect was observed for our ROI analyses within fronto-parietal action regions. Slow action sentences elicited greater levels of activation than fast action sentences in the right primary motor area (M1) and anterior inferior parietal lobule (aIPL). The inferior parietal lobule is part of a cortical network involved in the use of tools, performance of complex hand-object interactions and representation of action plans and goals (Lewis, 2006; Rumiati et al., 2004; Hamilton & Grafton, 2006; Ramayya, Glasser, & Rilling, 2010). Damage to aIPL is associated with ideomotor apraxia. This neurological disorder is characterized by impairments in skilled motor performance, difficulty in selecting and carrying out the appropriate action in response to a given object and impairments in imitating gestures (Heilman, Rothi, & Valenstein, 1982; Heilman & Rothi, 1993; Jax, Buxbaum, & Moll, 2006; Haaland, Harrington, & Knight, 2000). The primary motor cortex (M1) is crucial for the execution of motor movements (Graziano, Taylor, & Moore, 2002) and has shown to be activated by linguistic descriptions of actions in some contexts (Pulvermüller, Hauk, Nikulin, & Ilmoniemi, 2005; Hauk et al., 2004; de Lafuente & Romo, 2004; Cacciari et al., 2011; Vigliocco et al., 2006). Furthermore, slow action sentences as compared to abstract sentences led to a similar pattern of activation as fast action sentences, with additional activation in the left insula. While the insula is associated with a wide variety of functions, a recent meta-analysis found that a posterior-dorsal region within the insula is especially associated with somatosensory and sensorimotor tasks, and is functionally connected to the bilateral motor cortex (Chang, Yarkoni, Khaw, & Sanfey, 2013). These findings suggest that conceptual representations of slow actions are grounded through a cortical network involved in the representation of actions and action plans. In the present study, differential recruitment of motor regions in the comprehension of linguistic descriptions of movements at different speeds could reflect the fact that movement velocity bears an intimate relationship with the amount of effort that needs to be exerted in performing an action. On the one hand, an increase in effort typically results in a similar increase in speed of movement. However, slower actions can also reflect greater exertion and greater need of control and action planning. For example, moving a piece of heavy furniture is slow, effortful, and deliberate, while moving a relatively light object such as a pen is faster and less effortful. Thus, we suggest that representations of slower actions reflect their association with greater implied physical effort and therefore more strongly draw on motor simulation processes related to the control and actual timing of movements (Guillot et al., 2009; Michelon, Vettel, & Zacks, 2006). Sentences denoting faster actions, on the other hand, more strongly activated superior and middle occipital regions, which suggests a role for visually based simulation processes in their representation. An additional possibility is that the mental simulation of a slow action takes more time to unfold and therefore affords a greater opportunity for motoric processes to be involved. Fast action simulations are more rapid, and this faster time course only allows for perceptual grounding.

The current study was not designed to test effects of concreteness, and such effects been described in a number of previous reports. Here, abstract and concrete sentences differed in syntax. Concrete sentences included spatial prepositions (e.g, towards, through, along) while the abstract sentences did not. Additionally, there was a repetition effect for concrete sentences, given that fast and slow sentences were identical with the exception of the verb, which was not the case for abstract sentences. While they contained abstract verbs, the agents in the abstract sentences were animate, and likely invoke motion/action associations. Because of these differences, we interpret concreteness effects here with caution. Concrete sentences elicited greater levels of activation in the bilateral IPL extending in to the intraparietal sulcus, the bilateral MFG and IFG (pars triangularis), left SFG and precuneus. The IPL plays a role in the performance of complex hand-object interactions, tool use and the representation of action plans and goals as noted above.

Abstract sentences elicited greater levels of activation than concrete sentences within lateral temporal regions, which suggests context dependent, conceptual association-based processing, and is consistent with a number of previous studies (Wilson-Mendenhall, Simmons, Martin, & Barsalou, 2013; Binder, Westbury, McKiernan, Possing, & Medler, 2005; Desai 2009, 2011; Wang, Conder, Blitzer, & Shinkareva, 2010).

In the present study, no difference was observed between fast and slow action sentences within motion area MT+. In humans it has been shown that, MT+ shows stronger magnetoencephalography (MEG) responses to higher speeds (Kawakami et al., 2002; Amano, Kimura, Nishida, Takeda, & Gomi, 2009). Damage to area MT seems to lead to impairments in low-level motion integration tasks but not in biological motion processing (Vaina et al., 1990). Similarly, other studies have shown that inactivation of area MT does not interfere with biological motion perception (Grossman et al., 2005). Several studies have shown involvement of MT+ in the processing of linguistic descriptions of motions. For example, Saygin and colleagues (2010) showed an involvement of motion-sensitive visual areas (MT+) in the processing of motion (The wild horse crossed the barren field) and fictive motion sentences (The hiking trial crossed the barren field), but not for static sentences (The black horse stood in the barren field). In a similar vein,Rueschemeyer et al. (2010) observed that language modulated activity in area MT+. Other studies, however, have failed to obtain activity within MT+ for linguistic descriptions of human and object motions (Humphreys et al., 2013; Wallentin, Lund, Ostergaard, Ostergaard, & Roepstorff, 2005). Our finding can be grouped with these latter studies that did not find a modulation of motion area MT+ by the speed conveyed in motion sentences. One could argue that such a null effect is in line with the interpretation that the representations that are accessed during comprehension of language that denotes fast and slow motions, may not rely on information stored in low-level visual motion areas (Humphreys et al., 2013). However, this interpretation should be taken with a grain of salt, given that sensitivity to MT+ activations was likely reduced in the current study (as well as several other studies) due to the use of mean coordinates and lack of individual localization. MT+ varies considerably across individuals (Humphreys et al., 2013), and this inter-subject variability might create biases in group-level analyses in standard space (Dumoulin et al., 2000). Studies that have shown positive results in MT+ (Saygin et al., 2010; Rueschemeyer et al., 2010) have typically used a MT+ localizer task to obtain greater sensitivity.

The significant differences between fast and slow sentences were found only in the RH. This is somewhat surprising, given that LH is most closely associated with lexical/sentential semantics. However, a number of findings have pointed towards a unique contribution of the RH to language comprehension (Jung-Beeman, 2005). According to one view, the LH activates the dominant meaning or the few most salient or coarse features, whereas the RH activates more distantly related or fine-grained features (Faust & Chiarello, 1998; Titone, 1998; Faust, Ben-Artzi & Harel, 2008). Verbs in the fast and slow action condition can be thought of as being comprised of multiple features that are more or less central to the concept. For motion verbs, motion itself and associated changing of spatial position may be more central to the meaning. It might be the case that this more basic information is processed by the LH, and that we therefore did not find a difference between fast and slow verbs within the LH. This suggests the possibility that speed of motion is a relatively finer-grained feature and therefore relies more strongly on the RH.

While slow action sentences and fast action sentences differed in terms of the speed of motion that they denote, there might be a difference in terms of how salient the semantic attribute of speed-of-motion is for the fast and slow verbs. It could be argued that the difference observed between fast and slow sentences reflects the fact that speed might be a salient feature of the meaning of fast sentences but not for slow sentences. A post-hoc questionnaire showed that subjects judged speed-of-motion to be a salient feature for both the slow (“The professor sneaked down the corridor”) and fast sentences (“The professor stormed down the corridor”) as compared with sentences for which speed-of-motion wasn’t a salient feature (“The boy moved across the living room”). The ratings of our questionnaire showed that both for fast and slow sentences the semantic attribute of speed-of-motion was a salient feature of the meaning denoted by the sentence. Our questionnaire, however, also pointed towards a small difference in the salience of the speed-of-motion attribute between our fast and slow sentences (i.e., the salience of the speed-of-motion attribute was judged to be slightly higher for fast sentences than for slow sentences). It could therefore be argued that stronger activation for fast as compared to slow sentences within the right posterior STS (SOG/MOG) might partially reflect the fact that “speed-of-motion” was a slightly more salient feature for the fast sentences than for the slow sentences. This possibility, however, is not detrimental to the current findings, given that (1) stronger activation for “fast>slow” within the posterior STS shows that the speed-of-motion is incorporated in language-induced sensory-motor simulations (2) a reversal of the effect (i.e., “slow>fast”) within motor areas cannot be accounted for by an explanation solely in terms of the saliency of a specific semantic attribute to a given concept.

Conclusions

These results show that the speed of motion conveyed by a verb affects the way in which it’s underlying conceptual representation is grounded in sensory-motor regions. When the described actions are relatively fast, simulations appear to be more visual in nature, showing greater activity in pSTS and MOG, areas involved in visual semantics. When sentences describe slow actions, their grounding is more motoric in nature, resulting in greater signal in primary and higher-order motor areas in the RH. Thus, representations accessed during comprehension of language that denotes fast and slow motions seem to incorporate information about the speed of motion. The speed conveyed by motion verbs influences conceptual grounding, which provides us with valuable insights in the exact nature of and information recruited in language-induced sensory-motor simulations.

Figure 2.

Shows the results from the ROI analysis. Red circles show the locations of the sphere of 5 mm radius around the following voxel locations: left M1 (x = −28, y = −25, z = 47); right M1 (x = 32, y = −25, z = 53); left aIPL (x = −57, y = −32, z = 32); right aIPL (x = 57, y = −32, z = 32); left pSTS (x = −39, y = −59, z = 15); right pSTS (x = 47, y = −56, z = 15); left MT+ (x = −44, y = −72, z = 5), right MT+ (x = 43, y = −69, z = 3). Mean percent signal change relative to rest is shown for the fast and slow action conditions. Regions of interest are displayed on an inflated surface of a brain. Gyri are shown in light gray and sulci in dark gray. L = left hemisphere, R = right hemisphere.

Highlights.

• Motion speed influences the nature of conceptual grounding

• If denoted actions are relatively fast, simulations appear to be more visual in nature

• If sentences denote slow actions, their grounding is more motoric in nature