Distinct neural mechanisms underlying conceptual knowledge of manner and instrument verbs

Abstract

Studies on the organization of conceptual knowledge have examined categories of concrete nouns extensively. Less is known about the neural basis of verb categories suggested by linguistic theories. We used functional MRI to examine the differences between manner verbs, which encode information about the manner of an action, versus instrument verbs, which encode information about an object as part of their meaning. Using both visual and verbal stimuli and a combination of whole brain univariate and multivariate pattern analyses, our results show that accessing conceptual representations of instrument class involves brain regions typically associated with complex action and object perception, including the anterior inferior parietal cortex and occipito-temporal cortex. On the other hand, accessing conceptual representations of the manner class involves regions that are commonly associated with the processing of visual and biological motion, in the posterior superior temporal sulcus. These findings support the idea that the semantics of manner and instrument verbs are supported by distinct neural mechanisms.

Introduction

Much work on the organizational principles underpinning the semantic system has focused on nouns. For example, a phenomenon that has been studied extensively in the neuropsychological literature is category-specific semantic impairments of either living or non-living things. Some researchers have suggested that such category specific impairments reflect an underlying evolutionary separation of the neural circuits used for representing concepts from the different domains (e.g., Caramazza & Shelton, 1998, Caramazza & Mahon 2003). Other researchers have instead argued that the different conceptual domains are represented in a single distributed network and that differences between the domains reflect differences in the importance of various types of features to concepts in each domain (e.g., Warrington & Shallice, 1984; Warrington & McCarthy, 1987; Farah & McClelland, 1991). For example, Warrington & Shallice (1984) have argued that concepts of living things rely more strongly on sensory features while the representation of non-living things relies more strongly on functional (possibly motor) features.

Very few studies have examined semantic subclasses of verbs, but similar distinctions have been proposed for the conceptual representations that underlie concrete verbs. The lateral occipito-temporal cortex (LOTC) is especially associated with manner and instrument verbs. Quandt et al. (2015) compared manner verb phrases (e.g., kick the cage) and path prepositions (e.g., into the cage), and found activation in posterior middle temporal gyrus (pMTG) and medial fusiform gyrus for manner verbs, while no regions were activated more for path verbs. Kemmerer et al. (2008) examined five types of verbs (running, speaking, cutting, hitting, and change of state). They found that relative to false fonts, running verbs (a subclass of manner verbs) activated the left posterior superior temporal sulcus (pSTS), and area associated with processing biological motion, while cutting verbs (a subclass of instrument verbs) activated posterior middle and inferior temporal gyri and the lateral occipital cortex. Parris and Weekes (2001) argued that instrument verbs, which encode a typical instrument as part of their meaning (e.g., hammering, chiseling), are more closely associated with noun representations than other types of verbs. In support of their proposal, Parris and Weekes reported cases of patients with patterns of domain-specific semantic impairments with nouns and verbs that aligned with their predictions (see also Bird et al., 2000). A logical extension of this proposal is that the dissociation in patients’ performance on instrument and non-instrument verbs would parallel differences that have been documented for nouns denoting living and non-living things. More specifically, it could be hypothesized that semantic representations of living nouns and non-instrument verbs rely more strongly on sensory features, whereas representations of non-living nouns and instrument verbs rely more strongly on functional features.

Almor and colleagues (2009) investigated this hypothesis by testing Alzheimer’s (AD) patients’ picture naming and picture pointing performance with pictures of living and non-living nouns, and manner and instrument verbs. In contrast to the hypothesized relation between non-living nouns and instrument verbs, they observed category-specific impairments for living nouns and instrument verbs and a correlation between the overall impairment patients showed with nouns over verbs and their relative impairment with instrument over manner verbs. The authors explained these findings in terms of a differential importance of general object knowledge in the representations of instrument and manner verbs. This explanation is compatible with work in linguistics that has argued for a distinction between instrument and manner verbs in terms of the information they lexicalize (Marshall et al., 1996; Rappaport Hovav & Levin, 2008). The meaning of instrument verbs (e.g., hammering, shoveling) is critically dependent on instrument nouns (e.g., hammer, shovel), whose information is incorporated into the meaning of the verb. In contrast, manner verbs (e.g., running, jumping) typically do not lexicalize information about a particular instrument as part of their meaning, but instead encode information about the manner of an action. Rappaport Hovav and Levin (2008) argued that manner verbs are characterized by the fact that they lexicalize non-scalar change (i.e., many co-occurring changes that are not easily quantifiable). For example, a verb like “running” describes a physical activity, which requires a complex combination of an unspecified set of movements. Therefor, it could be argued that instrument verbs might be semantically more restricted than non-instrumental verbs, as proposed by Malyutina and colleagues (2016).

The research reviewed thus far relied on neuropsychological evidence based in patterns of impairments. In the current study, we turned to neuroimaging techniques to assess whether the reported patterns of category-specific verb deficits could indeed reflect a differential importance of object knowledge underlying these conceptual categories in the healthy brain. We measured changes in the hemodynamic response of participants while accessing conceptual representations of instrument and manner verbs. We used both picture and word stimuli in order to assess differences between verb classes across modalities. We test the hypothesis that the instrument class will draw on brain regions that play a role in the representation of action plans and goals, action understanding, storage of action plans, and hand-object interactions, specifically the ventral premotor cortex (vPMC) and anterior inferior parietal lobule (aIPL). The manner class, on the other hand, will elicit activation in processing whole-body movements and biological motion, namely pSTS. Both types of verbs should involve areas involved in general motion processing and body representation at the occipito-temporal junction.

The aim of our study was to focus on the neural underpinnings of the linguistically motivated classification of verbs provided by Levin (1993). Unlike much of the literature on conceptual representations, in which actions and verbs are distinguished on the basis of their conceptual properties, Levin’s verb classification approach is based on the syntactic properties of verbs in terms of which syntactic environments they tend to appear in. Levin acknowledged that manner and instrument verbs are heterogeneous in the sense that they can lexicalize different types of instruments (e.g., instruments for building, eating or used around the house) or different manners of carrying out an action (e.g., nonverbal expression, communication or social interaction). Our study is therefore unique in that it investigates the neural underpinnings of a syntactically- rather than conceptually- driven distinction between the different verb classes.

METHODS

Participants

Seventeen people participated in the study, all of whom were right-handed and between 21 and 33 years of age (M = 24.28, SD = 2.68; 3 males). All participants were native English speakers by self-report, had normal or corrected-to-normal vision, and no history of neurological disorders. Prior to the experiment, participants were informed about the experimental procedures, signed informed consent forms, and were given practice trials according to a protocol sanctioned by the Institutional Review Board of the University of South Carolina. Participants were paid for their participation. Every subject underwent two scans on two different days within a one-week period.

Stimuli

The four main experimental conditions contained 24 stimuli each. They were: (1) Manner verbs (MV, e.g., “running” or “jumping”), (2) Manner verb pictures (MvP, e.g., a picture of a person running/jumping), (3) Instrument verbs (IV, e.g., “hammering” or “shoveling”), and (4) Instrument verb pictures (IvP, e.g., a picture of a person hammering/shoveling). In addition, 16 pictures were presented that were unrealistic or implausible (Implausible picture condition). Scrambled versions of the Manner (MvPScram), Instrument (IvPScram) and Implausible pictures were presented to account for differences in low level visual features between the corresponding conditions. These were created by dividing the original pictures into 10 × 10 pixel tiles and randomly rearranging them. For an example of the picture stimuli see Figure 1. Finally, 16 pseudowords were created using the English Lexicon Project. These pseudowords were formed by taking non-word roots from the English Lexicon Project and adding the –ing, ensuring that the string was still pronounceable and was still not a word. These pseudowords matched the words on syllables and number of letters. See Table 1 for a list of all manner verbs, instrument verbs and pseudowords.

Figure 1.

Example of the picture stimuli. (A) instrument verb picture (pouring); (B) manner verb picture (preaching); (C) implausible picture (girl hovering); (D) scrambled picture

Table 1.

List of Manner and Instrument verbs.

Manner verbs		Instrument verbs		Pseudowords
Coughing	Teaching	Arranging	Pouring	Daffing	Dwilling
Crying	Warning	Blowing	Rolling	Larveling	Halleting
Frowning	Whispering	Cutting	Toasting	Jarking	Grifting
Laughing	Yelling	Grinding	Tossing	Deming	Feging
Panting	Boxing	Hammering	Dusting	Sneeding	Senteeling
Sneezing	Fighting	Knitting	Ironing	Floiding	Varting
Whistling	Hugging	Sculpting	Mopping	Thoping	Ammsing
Yawning	Kissing	Sewing	Raking	Firanging	Bamping
Confessing	Marrying	Baking	Scraping
Preaching	Petting	Cooking	Shovelling
Screaming	Playing	Frying	Sweeping
Singing	Wrestling	Mixing	Vacuuming

The verbs and pictures were a subset of those used in Almor et al. (2009). These consisted of color pictures matched for each verb from various print and on-line sources, or photographed and edited to yield a clearly identifiable picture of the action on a white background. Responses to all words and pictures were made by means of a right thumb press using a MRI-compatible response glove. Each experimental stimulus was presented four times over the course of two sessions across different days, resulting in 96 items in each experimental condition. A word and its corresponding picture were never presented in the same session.

The verbs that corresponded to the picture conditions were matched on a number of psycholinguistic variables: word frequency, verb familiarity, verb imageability, verb action relatedness, verb object relatedness, picture naming accuracy and picture naming latency (see Table 2). As shown in Table 2, manner and instrument stimuli did not differ in any of these variables, except that, as expected, manner verbs were rated as significantly less object-related than instrument verbs. Importantly, image accuracy (a rating of how well participants were able to indicate the verb that corresponded to the picture) was high for pictures of both manner and instrument verbs. Furthermore, the norming agreement of our verbs was 87%, which was calculated as the number of intended target responses divided by the number of semantically correct responses (including synonyms and subordinate classifications as correct responses; see Almor et al., 2009 for details). We therefore ensured that for the English speaking subjects used here, there was a strong association between pictures and particular verbs. We note that there is significant cross-linguistic variability in how pictures or video clips are named, for example, for events of cutting, breaking and opening (Majid & Bowerman, 2007; Majid et al., 2007, 2008), and events of putting and taking (Slobin et al., 2011; Kopecka & Narasimhan, 2012). Hence, the results obtained here would not necessarily generalize to speakers of other languages. How neural signatures change with the characteristics of languages is in itself an important area for future investigations (Kemmerer, 2019).

Table 2.

Manner and Instrument verb characteristics (means, standard deviations and t-tests of differences between the two verb categories). Word frequency was obtained from Francis & Kučera (1982). Familiarity ratings on a scale of 1 to 7 (seven being the most familiar), imageability ratings on a scale of 1 to 7 (seven being the most imageable), and picture naming accuracy and latency were taken from the data reported by Almor et al. (2009). Action relatedness ratings were provided by 15 participants recruited via Mechanical Turk who rated how strongly they thought the verb was associated with an action on a scale of 1 to 7 (seven being strongly associated with an action). Object relatedness ratings were provided by the same 15 participants who indicated how object-related they thought was the action described by each verb on a scale of 1 to 7 (seven being strongly object-related).

	Manner Verbs (n=24)	Instrument Verbs (n=24)	t-test (2 tails)
Word Frequency	57.08 (74.73)	35.79 (49.84)	t(46) = 1.13, p = .26
Familiarity	6.05 (0.59)	5.85 (0.54)	t(46) = 1.23, p = .22
Imageability	5.04 (0.97)	4.74 (0.59)	t(46) = 1.14, p = .26
Action relatedness	4.60 (1.50)	5.13 (0.85)	t(46) < 1.51, p = 0.14
Object relatedness	2.70 (0.79)	5.60 (0.77)	t(46) = 12.68, p < 0.001
Picture naming accuracy	76% (24)	74% (21)	t(46) < 1, p = .82
Picture naming latency (ms)	1167 (325)	1150 (248)	t(46) < 1, p = .84

Procedure

The order of stimulus presentation was randomized individually for each participant. The start of each trial was indicated by a fixation cross. After a variable jitter (0, 200, 400, 600, or 800 milliseconds), the experimental stimulus appeared on screen for 2.0 seconds. The time between offset of the stimulus and onset of the next trial was filled with a blank screen such that each trial lasted 5.5 seconds. In the word blocks, participants were instructed to respond to only pseudowords by pressing a button with their right thumb. This go/no-go lexical decision task was designed to encourage participants to read all words for comprehension while avoiding strong influence of activation related to motor execution on experimental trials. The picture stimuli were presented in separate blocks that included pictures of the instrument and manner verbs, as well as a set of implausible pictures. In these blocks, participants were instructed, again, to respond to only implausible pictures. This go/no-go picture plausibility judgement design was used to encourage participants to activate conceptual representations of the events portrayed in the pictures, while again avoiding motor artefacts in experimental trials. To provide a baseline condition reflecting activation associated with low-level visual features, and also account for any possible differences in low-level visual features between our experimental conditions, scrambled versions of the manner, instrument and implausible pictures were presented in a separate block with a go/no-go feature detection task. Occasionally, a picture contained a 4 × 4 pixel grey square to which participants responded by means of a right thumb button press. The experiment was comprised of two sessions on separate days that were equal in length. The stimulus material was divided in two separate lists that were equal in length and contained exactly half of the pictures and words of each condition, such that a picture and word that denoted the same concept were never presented in the same session. The order of the presentation of the two lists was counterbalanced across subjects. Each session contained 4 runs that consisted of a word, picture and scrambled picture block, the order of presentation of each block within a run was randomized.

fMRI data acquisition

Functional images were acquired on a Siemens TRIO 3.0 T MRI system (Siemens, Erlangen, Germany) equipped with a 12-channel head coil. BOLD-sensitive functional images were acquired using a single-shot gradient EPI sequence (echo time/repetition time = 34/1850 ms), 34 axial slices in ascending order, slice gap = 0.6 mm, field of view = 208 mm, flip angle = 90 degrees, voxel size = 3.25 × 3.25 × 3.60 mm³). High-resolution anatomical images were acquired using a MPRAGE sequence (echo time = 4.15 msec, voxel size = 1 × 1 × 1 mm³, 192 sagittal slices, field of view = 256).

fMRI Data Analysis

Functional data were preprocessed and analyzed with the AFNI software package (Cox, 1996). A standardized preprocessing pipeline involved despiking of the data by fitting a smoothish curve to each voxel time series and registration of functional images to the anatomy (Saad et al., 2009). Subsequently, functional images were co-registered (Cox & Jesmanowicz, 1999) and projected into standard stereotaxic space (Talairach and Tournoux, 1988). The normalized images were smoothed with an isotropic 9-mm FWHM Gaussian kernel, and converted to percent signal change. The ensuing preprocessed fMRI time series were analyzed on a subject-by-subject basis using an event-related approach in the context of voxel-wise multiple linear regression with regressors for each condition (MV, IV, MvP, IvP, MvPScram, IvPScram, Pseudowords, implausible pictures and grey square catch trials) convolved with a canonical hemodynamic response function. Data of the two sessions were concatenated and treated as a single session containing 8 separate runs. In addition, we had regressors for the pseudoword and implausible picture conditions. Six motion parameters and the signal extracted from the ventricles were included as noise covariates of no interest. General linear tests were conducted to obtain the effect of picture category (i.e., MVP vs. IVP); pictures compared to low-level baseline (i.e., MVP vs. MVS and IVP vs. IVS); word category (i.e., MW vs. IW) and words compared to pseudoword baseline (i.e., MW vs. pseudowords and IW vs. pseudowords).

In a random effects analysis, group maps were created by comparing activations against a constant value of 0. The group maps were thresholded at voxelwise p < 0.001 and corrected for multiple comparisons by removing clusters smaller than 832 μl to achieve a mapwise corrected two-tailed p < 0.05. We used AFNI program 3dFWHMx to calculate the autocorrelation function (ACF) parameters for each individual subject. Subsequently, the median ACF value of all subjects was used in running the 3dClustSim program with 10000 iterations (see Cox, Reynolds, & Taylor, 2016). The cluster threshold was determined through Monte Carlo simulations that estimate the chance probability of spatially contiguous voxels exceeding the voxelwise p threshold. The analysis was restricted to a mask that excluded areas outside the brain, as well as deep white matter areas and the ventricles.

Searchlight analysis

In addition to this whole-brain univariate analysis, multivariate searchlight analyses were performed to localize brain regions sensitive to semantic verb category information. Univariate analyses have shown to be more sensitive to between-subject variability in the activation to an experimental variable, whereas MVPA analyses are more sensitive to the voxel-level variability in activation to an experimental variable (Davis et al., 2014). This is a reflection of the fact that in univariate analyses, within-subject variability in the activation to an experimental variable is eliminated by averaging activations over all exemplars within the respective experimental conditions, whereas the variability in the degree of similarity of the patterns of neural activations within and between the exemplars that make up the different experimental conditions provides the main source of information for multi-voxel pattern analyses (MVPA) (Carota et al., 2017). Davis et al. (2014), however, also pointed out that even though MVPA analyses are more sensitive to effects that are distributed over multiple voxels, this does not mean that the dimensionality of the information encoded can simply be inferred from a comparison of univariate and multivariate results. In the current study, MVPA searchlight analyses were therefore used in a complementary way to increase sensitivity in detecting potential differences in the neural representation of manner and instrument verbs, without making any inferences from the potential commonalities and/or differences of the two approaches. Multivariate analyses were conducted on images that were smoothed with an isotropic 5-mm FWHM Gaussian kernel.

For each participant and each voxel, data from a 3-voxel searchlight radius centered at a given voxel were used to identify patterns of brain activity associated with manner and instrument verb representations using the Princeton MVPA Toolbox (https://github.com/princetonuniversity/princeton-mvpa-toolbox). The analysis was restricted to a gray matter mask. Support vector machines were used for classification of picture and word data. One run was left out as test data, while the classifiers were trained on the other 7 runs, resulting in 8 cross-validation folds. Thus each participant had two classification accuracy maps: pictures and words. Chance-level accuracy (.5) was subtracted from obtained classification accuracy maps (Kim et al., 2016) before they were submitted to a random-effects whole-brain group analysis. Initial voxel-level threshold of p = .001 was used for the picture contrasts and p = 0.005 for the word contrasts. Given that the classification accuracy maps of the pictures vs. scrambled pictures were all very high, we subtracted an accuracy of .75 from the obtained accuracy maps before submitting them to a random-effects whole-brain group analysis. An initial voxel-level threshold of p = 0.0002 was used for the pictures vs. scrambled pictures contrasts. Statistical significance was evaluated using permutation tests (Stelzer et al., 2013). The same searchlight analysis procedure as described above was repeated with randomly relabeled trial data for each individual. Resulting individual accuracy maps were submitted to a group analysis and the largest cluster size was obtained. This procedure was repeated to yield 1,000 group maps, representing a null distribution of cluster sizes at the group level. Cluster size threshold corresponding to p = .05 was based on this empirically generated null distribution.

Results

Univariate Results

Manner Pictures-Manner Scrambled

Manner Pictures elicited greater levels of activation than Scrambled Manner Pictures within bilateral lateral occipital/temporal gyrus, right middle occipital gyrus (MOG), precentral gyrus (PrG) (Fig.2A; Table 3).

Figure 2.

(2A) Areas that showed greater activation for the Manner Verb Pictures (MVP) than for the Manner Verb Scrambled Pictures (MVS). (2B) Areas that showed greater activation for the Instrument Verb Pictures (IVP) than for the Instrument Verb Scrambled Pictures (IVS). L = left hemisphere, R = right hemisphere.

Table 3.

The volume of the cluster (μl), peak z-score, Talairach coordinates, and the anatomical structures that the clusters overlap are shown. L = left hemisphere, R = right hemisphere

Volume	Max	x	y	z	Structure
Manner Pictures > Manner Scrambled
5211	4.72	−34	−43	−9	L lateral occipital/temporal gyrus; fusiform gyrus
2187	4.28	40	−79	−3	R middle occipital gyrus
1944	4.87	37	−49	−18	R lateral occipital/temporal gyrus; fusiform gyrus
Instrument Pictures > Instrument Scrambled
14094	4.98	−43	−67	−3	L inferior occipital gyrus, sulcus; middle occipital gyrus
7047	4.96	46	−73	0	R middle occipital gyrus
1053	4.27	49	7	29	R precentral gyrus
1026	3.98	−37	19	−3	L inferior frontal gyrus (pars orbitalis)
999	3.38	−37	−34	38	L post central sulcus; anterior intraparietal sulcus
972	3.86	−55	−19	32	L post central gyrus; supramarginal gyrus
Manner Pictures > Instrument Pictures
1674	3.72	40	−55	23	R angular gyrus; superior temporal gyrus, sulcus
1350	4.12	−40	−73	38	L angular gyrus; superior temporal gyrus, sulcus
1026	3.69	−1	55	26	L superior frontal gyrus
972	3.86	1	−67	29	L/R precuneus
864	3.89	−4	−97	8	L occipital pole
Instrument Pictures > Manner Pictures
9288	5.20	−28	−85	23	L middle occipital gyrus
8370	4.36	−46	−37	44	L postcentral sulcus; supramarginal gyrus
2457	4.42	−37	−7	8	L insula
1485	5.26	−28	−52	−9	L medial occipital/temporal gyrus; lingual gyrus
1242	3.88	31	−82	17	R middle occipital gyrus
1134	3.67	22	−55	−9	R medial occipital/temporal gyrus; lingual gyrus
1188	3.21	22	−49	53	*R superior parietal gyrus
783	3.31	−49	1	23	*L precentral gyrus

^*Activation survives when group map was thresholded at voxelwise p < 0.005, uncorrected.

Instrument Pictures-Instrument Scrambled

Stronger activation was observed for Instrument Pictures relative to Scrambled Instrument Pictures in the bilateral MOG, left inferior occipital gyrus, post central gyrus (spreading into supramarignal gyrus: SMG), postcentral sulcus (extending into anterior intraparietal sulcus: aIPS), PrG (Fig. 2B; Table 3).

Manner Pictures-Instrument Pictures

Areas activated to a greater extent by the Manner than the Instrument Pictures included the bilateral AnG, precuneus, STS, left occipital pole and superior frontal gyrus (SFG) (Fig. 3; Table 3).

Figure 3.

Areas implicated by the contrasts between Manner Verb Pictures (MVP) and Instrument Verb Pictures (IVP). Red-orange colors show greater activation for the MVP condition; blue-cyan colors show greater activation for the IVP condition. L = left hemisphere, R = right hemisphere.

Instrument Pictures-Manner Pictures

Stronger activation was observed for Instrument Pictures relative to Manner Pictures in the bilateral MOG, medial occipital/temporal gyrus, left SMG, postcentral sulcus (extending into the aIPS), insula (Fig. 3; Table 3). At a reduced statistical threshold of p < 0.005; uncorrected, we found greater activation in left inferior ventral premotor cortex (vPMC) and right superior parietal lobule.

Manner verbs-Pseudowords

Manner verbs elicited greater levels of activation than Pseudowords within bilateral PcU, SFG, as well as the AnG. (Table 4).

Table 4.

The volume of the cluster (μl), peak z-score, Talairach coordinates, and the anatomical structures that the clusters overlap are shown. L = left hemisphere, R = right hemisphere

Volume	Max	x	y	z	Structure
Manner Words > Pseudowords
14661	4.78	−1	49	41	L/R superior frontal gyrus
4158	4.92	−37	−70	35	L angular gyrus
2673	3.96	55	−61	20	R angular gyrus
2403	4.28	25	31	50	R superior frontal gyrus
1809	3.76	−7	−43	5	L/R precuneus
Instrument Words > Pseudowords
17496	4.71	−10	64	14	R superior frontal gyrus
3888	4.85	−40	−67	32	L angular gyrus
2430	4.01	55	−61	20	R superior frontal gyrus, sulcus; middle frontal gyrus
2403	3.97	−4	−43	8	L/R precuneus
837	4.19	25	28	47	R superior frontal gyrus; sulcus; middle frontal gyrus

Instrument verbs-Pseudowords

Instrument verbs elicited greater levels of activation than Pseudowords within bilateral PcU, AnG, right MFG, SFG, as well as left anterior cingulate (ACC) (Table 4).

The univariate analyses found no significant differences in the direct contrast between the Instrument and Manner verbs.

Searchlight Results

Searchlight analysis identified the following clusters for the classification of Manner Pictures and Instrument Pictures: the right occipital-temporal lateral fusiform gyrus (FG), MOG, superior parietal lobule, STS, AnG, MTG, PrG, as well as the left ITG, MFG and SMG (Fig. 4A; Table 5).

Figure 4.

(A) Searchlight results for classification of Manner Verb Pictures (MVP) and Instrument Verb Pictures (IVP).

(B) Searchlight results for classification of Manner verbs (MV) and Instrument verbs (IV).

Table 5.

The volume of the cluster (μl), peak z-score, Talairach coordinates, and the anatomical structures that the clusters overlap are shown. L = left hemisphere, R = right hemisphere

Volume	Max	x	y	z	Structure
Manner Pictures vs. Instrument Pictures (p = 0.001)
161703	6.54	−40	−61	−12	L inferior temporal gyrus
		37	−40	−15	R occipito-temporal lateral fusiform gyrus
		−31	−82	20	R middle occ gyrus
		−43	−40	47	L supramarginal gyrus
		22	−91	17	R superior occipital gyrus
		31	−55	53	R superior parietal gyrus
		55	−49	11	R superior temporal sulcus, angular gyrus
1566	4.78	−31	40	8	L middle frontal gyrus
729	4.36	49	13	26	R precentral gyrus
675	5.00	55	−7	−9	R superior temporal sulcus, middle temporal gyrus
Manner words vs. Instrument words (p = .005)
1431	4.14	46	−49	8	R superior temporal sulcus, angular gyrus
1404	4.30	−28	37	23	L middle frontal gyrus, sulcus
1350	4.28	43	−73	26	R inferior parietal gyrus
1080	3.80	31	10	41	R precentral gyrus
459	3.68	37	−34	−21	R occipito-temporal lateral fusiform gyrus

Searchlight analysis identified the following clusters for the classification of Manner verbs and Instrument verbs: the right STS/AnG, inferior parietal lobule, PrG, occipital-temporal lateral FG, as well as the left MFG (Fig. 4B; Table 5).

A crossmodal analysis in which we trained our classifier on pictures and tested on verbs, and vice versa, did not yield any significant results.

Discussion

The current experiment tested if distinct patterns of activation can be observed for verbs that encode the manner of carrying out an action in comparison to verbs that incorporate an instrument in the event they describe. We were interested in the conceptual knowledge that underlies the manner/instrument verb category distinction proposed in the linguistically motivated classification by Levin (1993), irrespective of the domain through which this information was accessed (pictures vs. words). Furthermore, we were interested in how potential differences between the instrument and manner class would parallel the well-documented distinction between living and non-living things and would fit in with the larger neuropsychological literature.

Instrument class

In the current study, instrument pictures elicited greater levels of activation in the bilateral pMTG, MOG, and anterior part of the left inferior parietal lobule (aIPL). At a reduced statistical threshold of p < 0.005; uncorrected, we found greater activation in left inferior ventral premotor cortex (vPMC) and right superior parietal lobule. Activation in the bilateral pMTG, MOG and left inferior parietal lobule fell in close proximity to the regions identified by the searchlight analysis that were able to successfully distinguish between instrument pictures and manner pictures.

The aIPL is a secondary sensory-motor area that is part of a tool use network encompassing the middle/inferior temporal and inferior frontal regions (Ramayya, Glasser, & Rilling, 2009; Lewis, 2006; Rumiati et al., 2004). This region is critical for the representation of action plans and goals and the performance of complex hand-object interactions (Hamilton & Grafton, 2006; Ramayya et al., 2009). Damasio et al. (2001) demonstrated involvement of the aIPL when comparing the naming of actions performed with an implement with the naming of actions performed without an implement. Ideomotor apraxia, a condition characterized by damage to the aIPL/IPL, typically leads to impairments in skilled motor performance, problems in gesture imitation and difficulty in initiating the appropriate action in response to an object (Heilman, Rothi, & Valenstein, 1982; Heilman & Rothi, 1993; Haaland, Harrington, & Knight, 2000; Jax, Buxbaum, & Moll, 2006).

The vPMC is critical for action understanding, imitation, transformation of object properties into hand actions (Rizzolatti, Fogassi, & Gallese, 2002) and storage of general motor plans (Graziano, Taylor, Moore, & Cooke, 2002). The activation in the left pMTG is in close proximity to human motion area MT+ (Rees, Friston, & Koch, 2000; Tootell et al., 1995), a region implicated in accessing conceptual information about motion attributes (Kable et al., 2002, 2005; Saygin et al., 2010). The activation we observed extended into the posterolateral inferior temporal gyrus, a region that has been implicated in the representation of conceptual knowledge about tools (Damasio et al., 1996; Caramazza & Shelton, 1998; Grossman et al., 2002). In addition, instrument pictures led to greater activation than manner pictures within the left postcentral sulcus and the right superior parietal lobule, extending into the anterior intraparietal sulcus (aIPS), a region associated with visually guided grasping, hand-object interactions, and online control of movements (Culham et al., 2003; Tunik, 2005, 2007; Reichenbach, 2014). Thus, overall, areas involved in hand-object interactions, representation of action plans and goals and action understanding were activated more strongly during the processing of instrument pictures than during the processing of manner pictures. These findings suggest that information about which action is appropriate in response to an object and the exact hand-object interactions that are afforded by an object play a role in the processing of instrument pictures.

Instrument pictures elicited stronger activation than manner pictures within a cluster encompassing posterior portions of the ITG/MTG and the MOG. This region is part of the lateral occipitaltemporal cortex (LOTC), and plays a prominent role in action semantics (Lingnau et al. 2015). It plays a role in the processing of the many different dimensions of actions, such as visual motion (Wall, Lignau, Ashida, & Smith, 2008), body detection and perception (Downing & Peelen, 2011; Downing, Jiang, Shuman, & Kanwisher, 2001), action planning and execution (Astafiev, Stanley, Shulman, & Corbetta, 2004), the observation of biological motion (Grezes & Decety, 2001) and the perception and use of tools (Martin, Wiggs, Ungerleider, & Haxby, 1996). Most of these dimensions of actions are shared between manner and instrument verbs, which is reflected in the fact that both manner and instrument pictures strongly activated the LOTC compared to their corresponding scrambled picture conditions. A number of studies have indicated that the LOTC plays an important role in the processing of motion patterns associated with tool use, reading the names of tools and the preparation of gestures and actions associated with tools (Beauchamp, Lee, Haxby, & Martin, 2002; Gallivan, McLean, Valyear, & Culham, 2013; Chao, Haxby, & Martin, 1999), which can explain our finding of stronger LOTC activation when directly comparing instrument with manner pictures.

Within the category of instrument verbs, a further distinction can be made between verbs that refer to actions for which the performance requires a specific object (e.g., sewing, cutting) and verbs that have a direct name relation to a noun (e.g., hammering, mopping). Findings from neuropsychological studies suggest a distinction between verbs that have a name relation to a noun and verbs that don’t. In speakers with anomic aphasia, verb retrieval in action naming was positively influenced by whether a verb had a name relation or not (Jonkers & Bastiaanse, 2007; Kemmerer & Tranel, 2000; Kambanaros & van Steenbrugge, 2006; Kambanaros, 2009). However, a negative effect was observed for the name-relation factor in a word-to-picture matching task that required comprehension without production (Jonkers & Bastiaanse, 2006). Furthermore, their results indicated that the name-relation factor affected fluent vs. non-fluent aphasic speakers differently. Only a relatively small subset of instrument verbs had a namerelation with a noun (7 of the 24 instrument verbs), making it therefore difficult to examine if there is a within-category effect (instrument with vs. without a name relation to a noun). This is a question that needs to be addressed in future research.

Manner class

In the current study, manner pictures elicited greater levels of activation in the anterior part of the bilateral STS, AnG, PcU, left occipital pole (extending into MOG), as well as SFG. Activation in the right STS, AnG and left occipital pole fell in regions identified by searchlight analyses that contained information for successful classification of manner and instrument pictures. Activation in the right STS and AnG fell in close proximity to the regions identified by searchlight analyses that successfully classified manner vs. instrument verbs.

The pSTS plays a role in the processing of biological motion (Beauchamp et al., 2002; Saxe et al., 2004; Grossman, Batelli, & Pascual-Leone, 2005; for a review see Grosbras et al., 2012) and the perception of bodies and body movements (Allison, Puce & McCarthy, 2000; Grossman et al., 2000; Puce et al., 1998; Vaina et al., 2001). Specifically, the pSTS is involved in processing whole-body movements as compared to simple hand actions (Noppeney et al., 2005; Beauchamp et al., 2002). A recent study by Ross (2014) suggests that the pSTS is particularly important for coding motion direction information of whole body movements and not as much for coding body posture information. Patients with damage to the pSTS show deficits in the perception of biological motion despite form and motion processing being intact (Vaina & Gross, 2004). These findings have been corroborated by showing that disrupting activity of the pSTS in healthy subjects by transcranial magnetic stimulation (TMS) impairs perception of biological motion (Grossman et al., 2005). In summary, the pSTS seems to be particularly involved in processing movements that involve multiple body parts (with a complex set of motion trajectories) as compared with movements with simple trajectories (e.g., that mainly involve hand-object interactions). Stronger activation for the manner as compared to instrument class within posterior regions of the STS therefore likely reflects the fact that the manner class encodes information about a complex combination of an unspecified set of movements, whereas the instrument class primarily encodes information about simple hand-object interactions. Indeed, linguists have characterized manner verbs as lexicalizing many co-occuring changes that are not easily quantifiable (Rappaport Hovav & Levin, 2008). The sensory-motor information involved in coding movements denoted by these verbs are therefore relatively unspecified and complex in nature. For example, the verb “fighting” refers to an action that involves a multitude of movement sequences with different motion trajectories that need to be combined in a complex way. Our findings parallel neural distinctions that have been observed for living vs. non-living things, with living things more strongly involving areas associated with visual and biological motion information and non-living things recruiting regions that play a role in the representation of action plans and goals, action understanding and hand-object interactions.

Another area found in the current study to be more strongly activated for the manner as compared to instrument class is the angular gyrus. This region is known to play a role in general semantic processing (Binder et al., 2009). It has been proposed as a higher-level convergence zone where experiential information commonly associated with objects and events is integrated into a coherent representation (Fernandino et al. 2016, Binder et al., 2009; Bonner et al., 2013, Seghier 2013). While both classes are expected to engage AnG, its greater activation may potentially reflect higher demands of conceptual temporal integration for the manner as compared with instrument class. AnG is suggested to especially play a role in integrating across temporally extended set units, which is a characteristic of event concepts (Binder and Desai 2011, Rugg and King 2018, Ramanan et al., 2017). While both the manner and instrument class denote events, the particular sequencing of elements may form a more defining or crucial characteristic for the manner class. Processing of the manner class likely involves the mental simulation of movements of multiple body parts with different motion trajectories. These simulations might require a combination and integration of a rich array of action information. Instrument verbs, on the other hand, are typically built on instrument nouns for which the hand-object interaction is incorporated in their meaning. Processing of these verbs might therefore involve a mental simulation of a relatively well-circumscribed action for which demands on this integrative component are potentially lower. The finding of stronger activation for the manner as compared to instrument class within the left angular gyrus is inline with a number of studies showing that the left angular gyrus is more strongly activated for biological actions (e.g., swimming) versus tool-related actions (e.g., drilling) (Lin et al., 2011; Han et al., 2013). The PcU has similarly been identified as a functional hub region that plays an important role in neural communication and integration (Van den Heuvel & Sporns, 2013; Iturria-Medina et al., 2008). Differential recruitment of the PcU between manner and instrument verbs might similarly reflect differences in integration demands.

Activation in the right AnG and pSTS fell in a region that has often been described as the right temporo-parietal junction (TPJ) and has been argued to play a specific role in Theory of Mind, or the representation of thoughts of others (e.g., desires, emotions, and beliefs; Saxe & Kanwisher, 2003). Beyond the motion and temporal integration aspects mentioned above, a further contributing factor could therefore be a differential role that intentions, beliefs, and desires play in the representation of manner verbs (e.g., fighting, kissing, hugging) as compared with instrument verbs that denote actions that do not typically require the interaction with another agent (e.g., shoveling, knitting, cutting). Thus, manner verbs may require more inference with regards to thoughts of others, while both manner and instrument verbs require action-related simulations.

Instrument versus Manner class

Almor et al. (2009) suggested that a difference in the conceptual representations of the manner and instrument class might parallel the accepted distinction between living and non-living things (often tested by comparing animals vs. tools). The present findings are largely compatible with this hypothesis as the neural distinction that we observed between the manner and instrument class are similar to the differences that have been documented for living and non-living things. Manner class more strongly activated the MOG and pSTS, regions that play a role in visual and biological motion processing and is more strongly activated for living than non-living things (Chao et al., 1999; Caramazza & Mahon, 2003). Instrument class, on the other hand, activated the bilateral aIPL, pMTG and left vPMC, regions that play a role in skilled motor performance and hand-object interactions and have shown to be more strongly activated for non-living than living things (Rumiati et al., 2004; Caramazza & Mahon, 2003; Martin et al., 1996; Cattaneo et al., 2010). These results indicate that the category-specific semantic organization observed for nouns that refer to animals, manmade artifacts and plants, can be extended to verbs that lexicalize different types of information. Our findings indicate that visual and biological motion information plays a role in the semantic representation of manner verbs, whereas information about the action and hand-object interaction that is afforded by an object plays a role in the representation of instrument verbs.

This interpretation is also compatible with neuropsychological studies. For example, a number of patient studies have demonstrated category-specific deficits following brain damage, in which patients showed disproportionate difficulty for semantic judgments on living as compared with non-living things (Mahon and Caramazza, 2011; Basso, Capitani & Laiacona, 1988; Damasio et al., 1996). The reverse pattern in which patients were relatively spared in their performance on living as compared with non-living things has also been documented in a number of studies (Warrington & McCarthy, 1983; Hillis & Caramazza, 1991). Processing of animals typically involves recruitment of the lateral fusiform gyrus (Noppeney et al., 2006; Caramazza & Mahon, 2003), right medial occipital gyrus (Chao et al., 1999) right MOG (Noppeney et al., 2006) and pSTS (Chao et al., 1999; Caramazza & Mahon, 2003).

Processing and naming of tools, on the other hand, commonly activates the left IPL (Mahon et al., 2007; Rumiati et al., 2004), the medial fusiform gyrus (Caramazza & Mahon, 2003; Noppeney et al., 2006), the left MTG (Caramazza & Mahon, 2003; Devlin et al., 2002) as well as the vPMC (Martin et al., 1996; Caramazza & Mahon, 2003; Cattaneo et al., 2010). These findings have been taken as evidence for a category-specific organization of the semantic system with distinct neural mechanisms for conceptual categories like living things (relying on regions involved in visual/biological motion processing) and non-living things (relying on regions involved in skilled motor performance and complex hand-object interactions) (see Capitani et al., 2003 for a review). Jolly (2011) corroborated these findings by showing that a secondary biological motion task interfered with making categorical judgments on living objects, whereas a secondary motor task interfered with making judgments on manipulable non-living objects.

One possible concern is that the observed neural distinction between conceptual representations of manner and instrument classes might merely reflect low-level perceptual differences between the manner and instrument pictures. Two aspects of the data render it implausible. First, searchlights within right STS/AnG, inferior parietal lobule, PrG, occipital-temporal lateral FG and the left MFG contained information to successfully classify manner verbs from instrument verbs. Second, a searchlight analysis in which we tried to classify scrambled manner pictures from scrambled instrument pictures (i.e., and therefore shared low-level features with the manner and instrument pictures, without the semantic content) did not reveal any sensory-motor regions outside the visual cortex.

A limitation of the study is that the results for manner vs. instrument verb comparison were relatively weaker compared to the picture comparison. One possibility is that the lexical decision go/no-go task used for verbs did not pose strong enough semantic demands, leading to shallow activation. The heterogeneous nature of the manner verb class used here (motivated linguistically), which included several verbs that can be considered relatively abstract, may have also resulted in limited activation that is common across different verbs in the class.

We also note that the main comparison of interest here is the manner versus instrument class, and not the comparison of each verb class with a non-semantic baseline. The go/no-go task, which involved a button press for pseudowords but not for words, does not lend itself to a clean comparison. Any action-related activation for words would be eliminated by the strong activation from the button press. Hence, areas common to both instrument and manner verbs, typically seen in left temporal and frontal regions (Kemmerer et al. 2008, Kemmerer et al. 2012, Faroqi-Shah et al. 2018), were not expected to be observed here.

Conclusion

Our results show that the linguistically motivated instrument and manner classes are associated with distinct patterns of neural activation. Accessing conceptual information of the manner class elicited activation in areas associated with processing of visual and biological motion information, areas that have also been shown to play a role in the representation of living things. Accessing conceptual information of the instrument class paralleled activations typically observed for processing non-living things, recruiting regions that play a role in the representation of action plans and goals, action understanding and hand-object interactions. Overall, these findings lend support to theories that argue for a distributed sensorimotor-grounded conceptual representation of objects, actions, and the words that describe them.

• Manner verbs recruit visual and biological motion processing regions

• Instrument verbs recruit regions that play a role in the representation of action plans and goals, action understanding and hand-object interactions.

• Findings lend support to theories that argue for distributed sensorimotor grounding of conceptual representations