Compositionality and the angular gyrus: a multi-voxel similarity analysis of the semantic composition of nouns and verbs

Abstract

The cognitive and neural systems that enable conceptual processing must support the ability to combine (and recombine) concepts to form an infinite number of ideas. Two candidate neural systems for conceptual combination—the left anterior temporal lobe (ATL) and the left angular gyrus (AG)—have been characterized as “semantic hubs” due to both functional and anatomical properties; however, these two regions likely support different aspects of composition. Here we consider two hypotheses for the role of AG in conceptual combination, both of which differ from a putative role for the ATL in “feature-based” combinatorics (i.e., meaning derived by combining concepts’ features). Firstly, we examine whether AG is more sensitive to function-argument relations of the sort that arise when a predicate is combined with its arguments. Secondly, we examine the non-mutually exclusive possibility that AG represents information carried on a verb in particular, whether this be information about event composition or about thematic relations denoted uniquely by verbs. We identified voxels that respond differentially to two-word versus one-word stimuli, and we measured the similarity of the patterns in these voxels evoked by (1) pairs of two-word phrases that shared a noun that was an argument, thus sharing function-argument composition (e.g. eats meat and with meat), in comparison with two-word phrases that shared only a noun, not an argument (e.g., eats meat and tasty meat); and (2) stimulus pairs that shared only an event (operationalized here as sharing a verb; e.g. eats meat and eats quickly), in comparison to both of the above. We found that activity patterns in left AG tracked information relating to the presence of an event-denoting verb in a pair of two-word phrases. We also found that the neural similarity in AG voxel patterns between two phrases sharing a verb correlated with subjects’ ratings of how similar the meanings of those two verb phrases were. These findings indicate that AG represents information specific to verbs, perhaps event structure or thematic relations mediated by verbs, as opposed to argument structure in general.

1. INTRODUCTION

Language owes its infinite expressive capacity to our ability to take simple building blocks, such as words or concepts, and combine them into complex representations. In linguistics, such “semantic composition” refers expressly to the combination of words into complex linguistic expressions, the meanings of which are a function of both the constituent building blocks (words) and the “rules” used to combine them (the grammar). Whether, and how, such grammatical operations might be realized in the brain are still highly debated questions, and the emergence of compositional meaning from units such as morphemes, words, or concepts, is largely a mystery. However, understanding the engine of compositionality in the brain is a fundamental desideratum to any cognitive neuroscientific model of semantics.

1.1 Roles of left anterior temporal lobe and left angular gyrus in semantic composition

The psycholinguistically motivated neuroanatomical models of semantic processing that have emerged in the past few years involve several brain areas which roughly cluster into four main regions: left inferior frontal, left anterior temporal, left posterior temporal, and left temporo-parietal (Ben Shalom & Poeppel, 2007; Binder & Desai, 2011; Binder, Desai, Graves, & Conant, 2009; Lau, Phillips, & Poeppel, 2008; Pallier, Devauchelle, & Dehaene, 2011; Patterson, Nestor, & Rogers, 2007; Price, 2010). Of these regions, there are two – left anterior temporal lobe (ATL) and left angular gyrus (AG) – that are prime candidates to support composition, because both show greater activation for well-formed sentences than for noncompositional lists of words (Pallier et al., 2011, inter alia). In addition, both have been characterized as “semantic hubs”, owing to functional and anatomical patterns that are consistent with multimodal convergence (Binder & Desai, 2011; Lambon Ralph, 2014; Patterson et al., 2007; Seghier, 2012). While the mechanism by which different modalities converge on a single given conceptual representation is still unclear, it is likely that the mechanism that can encode the binding of modality-specific features into a given concept also accomplishes the binding of words into higher-level linguistic constructs (Westerlund & Pylkkänen, 2014). We begin with a brief review of findings relating to composition involving ATL in order to motivate contrasting ideas we will consider in the current study regarding composition in AG.

The ATL is uniquely situated at the end of a caudal-to-rostral stream of information processing feeding from primary sensory and motor areas and association cortex (Binder et al., 2009; J. R. Binder & Desai, 2011; Binney, Parker, & Lambon Ralph, 2012; Felleman & Van Essen, 1991). It is thus located at a prime “convergence zone” for inputs from many different modalities. Moving anteriorly along the temporal lobe, one finds a caudal-to-rostral hierarchy emerge as neuronal responses are more tuned to complex stimuli and more invariant to low-level sensory variation; such a hierarchy has been established along both visual (Felleman & Van Essen, 1991) and auditory (Rauschecker & Scott, 2009) streams. This “graded convergence” may provide a mechanism both for “feature combination” and, in the limit, for maximally invariant amodal, and thus abstract, conceptual representations. The culmination of this graded convergence up the temporal lobe (Rauschecker & Scott, 2009; Stringer & Rolls, 2002) is a basal rostral region of ATL shown to have very limited extra-temporal connectivity and high intra-temporal connectivity (Binney et al., 2012). Such neuroanatomical sequestration is arguably a sine qua non for a region able to represent abstract, modality-invariant semantics. Thus, ATL is a prime candidate for semantic composition.

In one of the first studies investigating the neural correlates of minimal two-word composition, Baron and colleagues (Baron, Thompson-Schill, Weber, & Osherson, 2010) found evidence from fMRI pattern analyses that the left ATL subserved the combination of concepts such that the superimposition of individual patterns of the simplex concepts young and man (as represented by various face stimuli) reliably predicted the activation pattern for the complex concept young man. Consistent with this finding, a magnetoencephalography (MEG) study of visually presented two-word phrases comparing nouns in minimal compositional contexts (red boat) with nouns in non-compositional contexts (in which a non-word letter string was concatenated with a real word, e.g. xkq boat) found increased composition-related activity in left ATL (Bemis & Pylkkänen, 2011). Thus, there is a growing body of functional and tractographic studies to suggest that the left ATL substrate of composition may be multimodal sensorimotor features, and particularly visual features of object-concepts (Coutanche & Thompson-Schill, 2014), corroborating the notion of the left ATL as hub of the so-called ventral “what” pathway.

While the left ATL has recently received much attention as a potential semantic hub, it is not the only region to invite this label. Researchers have also ascribed the role of a semantic hub to the AG, as it lies at the junction between temporal, parietal, and occipital lobes and thus receives a confluence of auditory, somatosensory, spatial, and visual inputs. Left AG has been implicated along with left ATL in conceptual combination studies of the sort described above (Bemis & Pylkkänen, 2012), and several studies demonstrate bilateral AG sensitivity to manipulations whereby well-formed sentences are contrasted with word lists, pseudowords, or scrambled sentences (Bottini et al., 1994; Bavelier et al., 1997; Humphries et al., 2006; Humphries et al., 2007). Left AG also shows greater activity for connected discourse vs. unrelated sentences (Fletcher et al., 1995; Homae, Yahata, & Sakai, 2003; Xu, Kemeny, Park, Frattali, & Braun, 2005). This broad profile of effects has led some to suggest that the AG may play a potentially domain-general role in semantic information integration structured around events (Binder & Desai, 2011; Binder et al., 2009; Lau et al., 2008; but cf. Noonan, Jefferies, Visser, & Lambon Ralph, 2013, and Discussion below for evidence that certain sites within AG are involved in semantic control processes, not representations).

Not all studies investigating conceptual combination find activation in both left ATL and bilateral AG. Upon closer inspection of those stimuli that elicit differential activity in AG but not in left ATL, one finds that the type of composition involved is invariably based on thematic relations rather than feature combination per se. For instance, Graves et al. (2010) compared familiar meaningful noun-noun pairs, such as lake house, with reversed phrases, such as house lake, the meanings of which were not obvious; they found that AG, along with other temporoparietal areas (mostly right-lateralized), showed greater activation for processing the more obviously combinatorial phrases. In characterizing the compositional operation employed in interpreting their particular noun-noun stimuli, the authors noted that most of their noun-noun stimuli were interpreted as denoting thematic relations between head and modifier nouns; that is, most compounds consisted of nouns participating in some spatial relation (as in “a house on a lake”) or event-based relation rather than sharing some common feature (as in, for instance, a nominal compound like cactus carpet, which is more likely to be interpreted as “a carpet that is prickly like a cactus” than as some sort of relational compound, like “a carpet with a cactus placed on it”) (Estes, 2003; Wisniewski & Love, 1998). This raises the question as to whether these stimuli were probing combinatorial semantics in general, or semantic thematic relations in particular.

In another group of studies, experimenters looking at 1-, 2-, and 3-argument verbs (that is, intransitive, transitive, and ditransitive verbs, respectively) found that activation in bilateral angular and supramarginal gyrus (BA 39 and 40) correlated parametrically with the number of thematic roles that can attach to a given verb, even when the verb was presented in isolation (Meltzer-Asscher, Schuchard, Ouden, & Thompson, 2012; Thompson et al., 2007; Thompson, Bonakdarpour, & Fix, 2010). Whereas Graves et al. (2010) indicates AG involvement in processing spatial and event-based relations, broadly construed, the work on verb adicity suggests a more selective sensitivity to verbs’ thematic relations and/or event complexity carried on the verb. While AG has been found to be sensitive to both linguistic event structure and non-verbal events depicted in scenes and mini-movies (Sitnikova, Holcomb, Kiyonaga, & Kuperberg, 2008; Sitnikova, Holcomb, & Kuperberg, 2008), it could be that the verb is the minimal linguistic expression of fundamental thematic relation-based or event-based concepts that AG subserves. This would predict that verb semantics would be particularly privileged in AG semantic space.

1.2 A feature vs. function dichotomy?

Given that both ATL and AG are implicated in semantic composition, we might start with the hypothesis that any kind of semantic similarity between two concepts might influence the similarity of neural (in our case, voxel) patterns evoked by the concepts in these two regions. For instance, regions that encode the meaning of a two-word phrase (such as “eats meat”) ought to elicit a similar neural response to other two-word phrases that share either of these two words as compared to a phrase that shares none of the words. In this study, we go one step further and explore possible restrictions on this prediction. We suggest that whereas the left ATL may be involved in structuring semantic knowledge around commensurate features of (object-) concepts, the AG builds semantic knowledge based on functional/thematic relations between concepts. Of course, this distinction could be operationalized in a number of different ways. In this study, we test two possible dimensions along which the left ATL and AG might cleave “feature-based” and “function-based” composition: one dimension respects whether two concepts share an event (and since verbs can denote events, we operationalize this as two phrases that share a verb), while another dimension concerns whether two concepts share an argument, which we will explain in greater detail below.

1.2.1 Angular gyrus and event-denoting verbs

It is widely agreed that verbs and nouns constitute meaningfully distinct linguistic forms, but it is less clear whether their processing engages different brain areas. In a meta-analysis using hierarchical clustering to identify regions associated with nouns and verbs, Crepaldi et al. (2013) identified several clusters associated with noun or verb processing across all tasks, the left AG among them; however, the left AG was associated with nouns, not verbs (though this might be due to the authors’ inclusion of activation peaks from studies examining nouns not directly contrasted with verbs). The authors conclude from the distribution of their clusters that the neural circuits of noun and verb processing are highly contiguous across a wide network of frontal, parietal, and temporal regions, and that there is little evidence suggesting that verb processing relies primarily on embodied motor representations.

These conclusions contrast with a prominent theory stating that, while not divided by the grammatical class of the noun or verb per se, neural areas are divided by the semantic primitives of objects and actions (Bird, Howard, & Franklin, 2000; Vigliocco, Vinson, Druks, Barber, & Cappa, 2011). Under this account, only prototypical nouns (object nouns) and prototypical verbs (action verbs) dissociate neural areas: specifically, action/verb processing recruits more of the fronto-parietal network while object/noun processing recruits more temporal regions (Cappa, Sandrini, Rossini, Sosta, & Miniussi, 2002; Cappa & Perani, 2003; Damasio & Tranel, 1993; Shapiro, Moo, & Caramazza, 2006). However, several studies fail to support such specific roles for frontoparietal areas and temporal regions in action/verb and object/noun processing, respectively (Crepaldi, Berlingeri, Paulesu, & Luzzatti, 2011; Liljeström et al., 2008; Tranel, Martin, Damasio, Grabowski, & Hichwa, 2005; Tyler, Russell, Fadili, & Moss, 2001). Also, a study by Bedny and colleagues (2008) found that while a certain region of left AG responded more to verbs than nouns, activity in this region did not distinguish between high motion and low motions words, whether nouns and verbs were included together or queried separately. This dissociation between grammatical class and action content, along with evidence for a dissociation between grammatical class and imageability (Bedny & Thompson-Schill, 2006), motivates treating the distinction between most nouns and verbs as a dimension of meaning separate from just action content, particularly when characterizing left AG.

Instead, we propose investigating the combinatory properties of events as a means to probe the semantics of AG. Our verb-centric hypothesis states that if AG represents events, then phrases that refer to similar events should evoke similar patterns of neural activity in AG. Even more specifically, phrases that share a given event-denoting verb (such as eats) should evoke similar patterns of activity in AG; events involving eating should have more similar neural patterns than they do to other kinds of events. Crucially, however, phrase pairs that differ along this dimension, in that one phrase has a verb and the other phrase does not (e.g. eats meat and tasty meat), will not evoke similar patterns of activation in AG, even though these latter two-word phrases also share a content word (meat).

Note that events are necessarily compositional constructs, as they represent not only information about event participants (thematic relations), but also temporal information vis-a-vis tense, aspect, etc. As we have mentioned, AG is more active in conditions which involve tracking narrative and discourse structure as compared with disconnected sentences (Fletcher et al., 1995; Homae, Yahata, & Sakai, 2003; Xu, Kemeny, Park, Frattali, & Braun, 2005). The connective tissue of such narrative and discourse structure is temporal order and temporally mediated causal relations. This suggests that temporal information, particularly that carried on a verb, is critical to engaging AG.

While there are “event” words that are not verbs, such as deverbal nouns and event-denoting nouns like party or hurricane, these lexical items do not often have temporal information. Inflected verbs necessarily carry information on tense and aspect, while nouns generally do not (rare exceptions being nouns that arguably denote a tense, like ex-wife, former champion, husband-to-be, president-elect). There is good reason to expect that event-denoting verbs dissociate from event-denoting nouns in both semantic and neural space: a study crossing grammatical class (nouns vs. verbs) with event denotation (events vs. objects) found that while left posterior middle temporal gyrus responded to both event nouns and verbs over objects, a region at the left temporoparietal junction (inclusive of AG) responded more to verbs than to any nouns, including event-denoting nouns (Bedny, Dravida, & Saxe, 2013).

Therefore, because we focus on events as spatiotemporal denotations, the natural way to examine this is to analyze verb similarity patterns. In this study, we do not directly test which component of event composition – thematic relations (theta roles) or temporal event information – might drive verb sensitivity in AG, but previous evidence implicates both.

1.2.2 Angular gyrus and function-argument composition

Under an alternative hypothesis, AG is sensitive to relational information independent of the verb. The dichotomy between “feature” and “function” has a good deal of traction in more formal theories of semantics, particularly in the distinction between adjuncts and arguments, respectively. The verb is the central predicate of a sentence, and predicates, such as eat, take arguments (like meat) (Heim & Kratzer, 1998; Pylkkänen, Brennan, & Bemis, 2011). While the case of verbs and their direct objects is perhaps the most canonical example of what we will call “argument-type composition,” there are other types of function-argument relations: for instance, the composition of prepositional phrases (e.g. with meat, where the preposition with takes the argument meat). We are particularly interested in the status of function-argument relations because, up until now, studies on two-word minimal composition have focused almost exclusively on another type of composition: adjunct-type composition (e.g. red boat, old man). However, two recent magnetoencephalography (MEG) studies suggest ATL may subserve both adjunct-type and argument-type composition. Linzen, Marantz, & Pylkkanen (2013) found that left ATL is sensitive to verb-argument structure, specifically the subcategorization frames of verbs. Westerlund, Kastner, Kaabi, & Pylkkänen (2015) examined several different instances of both argument- and adjunct-type composition: namely verb-argument (eats meat), preposition-argument (in Italy) and determiner-argument (Tarzan’s vine) composition for function-argument composition and adjective-noun (black sweater), adverb-verb (never jogged) and adverb-adjective (very soft) composition for adjunct-type composition. The authors found that function-argument composition (inclusive of verb phrases, prepositional phrases, and possessives) drove increased activation in left ATL. In our study, we extend this paradigm to examine whether AG responds specifically to argument-type composition.

Rather than rely only on univariate measures of activation for adjunct- vs. argument-based composition across a given brain region, we treat composition as it applies to the multivariate patterns of particular base words (in this case, nouns and verbs). We constructed sets of two-word phrases such that particular pairs in each set would allow us to test our hypotheses. Consider the phrases eats meat, eats quickly, with meat, and tasty meat. Using the logic of Baron et al. (2010), who found that additive superimposition of the voxel patterns underlying simplex concepts like young and man could predict the complex pattern of young man, we ask whether the complex concepts eats meat and with meat might be acting on the base word meat in the same way, as (1) both phrases are instances of function-argument composition and (2) meat is an argument in both (while it is not an argument in, say, tasty meat). On the other hand, if AG represents information carried on the verb (that is, the event, ex hypothesi), then the neural pattern evoked by the complex concept eats meat is expected to be more similar to that of eats quickly than to any other phrase.

1.3 Multi-voxel pattern similarity as a window to various dimensions of compositionality

Until recently, the prevailing approach in the neuroscientific study of concepts was to employ univariate tests of fMRI data, using a brain region’s average metabolic response to discriminate stimulus conditions based on locations of peak activation, potentially at the expense of voxel-level signal variation distributed across a given region. However, increasing use of multivariate methods to harness this voxel-level neural variability has revolutionized the study of object concepts. Current multi-voxel pattern analysis (MVPA) methods are predicated on the idea that information is instantiated in a spatially distributed pattern of neural activity. While some MVPA methods use various classification techniques over voxel patterns to discriminate between stimulus conditions (Boylan, Trueswell, & Thompson-Schill, 2014; Coutanche & Thompson-Schill, 2014; Polyn, Natu, Cohen, & Norman, 2005; inter alia), other methods analyze voxel patterns with respect to the strength of similarities between stimuli within given dimensions (e.g. shape, color, animacy, etc.) (Clarke & Tyler, 2014; Connolly et al., 2012; Fair et al., 2009; Fairhall & Caramazza, 2013; Haxby, 2001; Kriegeskorte, 2008; Weber, Thompson-Schill, Osherson, Haxby, & Parsons, 2009). With the latter approach, neural pattern variation can extend across a more continuous space than is sought in nominal/dichotomous classification techniques. Such MVPA techniques can be remarkably powerful tools, and have been used to query neural patterns using only a few TRs per stimulus event, and with each stimulus event modeled as a single unique regressor in a GLM (the beta values of which enter a correlation matrix or other similarity analysis) (Musz & Thompson-Schill, 2014).

The current study employs MVPA pattern similarity measures to query the relatively high-level semantic similarity space of two-word compositional phrases. We compare fMRI multi-voxel patterns associated with pairs of two-word compositional phrases - e.g. eats meat compared with eats quickly, tasty meat, or with meat – in which we hold constant a single word (here, either meat or eats), but manipulate (1) whether the word shared is a noun or a verb and (2) whether the two compositional phrases share a composition type (both argument-type or adjunct-type composition) or differ in their composition type. As we explain in greater detail below, our strategy is to observe the extent to which neural patterns evoked by two-word phrases are altered by relative isolated manipulations in their content, allowing us a means of inferring the principles that govern neural coding in different cortical regions. This is analogous to how a vision scientist can observe the tuning properties of a neuron by varying a stimulus dimension such as wavelength. Here, the dimensions we manipulate are (1) shared verb (thus shared event) and (2) shared composition type, which allows us to examine (1) how information carried on the verb may be critical to certain regions involved in semantic composition but not to others, and (2) whether the putative grammatical rules that distinguish various two-word phrases are differentially instantiated in regions of the brain implicated in basic semantic composition. We also investigate a corollary to the first hypothesis: if verbs are somehow privileged in certain brain regions (namely AG), then we might expect to see that the neural similarity between voxel patterns associated with phrases sharing a given verb might be predicted by subjects’ ratings on how similar the meanings of these verb-sharing phrases are to one another. We also test whether nouns might likewise drive pattern similarity of noun-sharing phrases in left ATL. 2.

MATERIAL AND METHODS

2.1 Participants

Twenty-one subjects participated in this study. Two participants’ data were excluded due to excessive motion, and one subject was found to have an anatomical anomaly. Data from the remaining eighteen subjects are reported here. Subjects ranged in age from 18 to 28 years, and all were right-handed native speakers of English with normal or corrected-to-normal vision and no reported history of neurologic problems. Subjects gave written informed consent and were provided monetary compensation for their time. The human subjects review board at the University of Pennsylvania approved all experimental procedures.

2.2 Stimuli

2.2.1 Stimuli Design

Crossing type of composition (argument-type vs. adjunct-type) with presence/absence of verb, we chose compositional word phrases that conformed to four different types:

1. +verb_arg: a word phrase that composed via argument-type composition and included a verb, e.g. eats meat

2. −verb_arg: a phrase that composed via argument-type composition, the head of which was a preposition instead of a verb; e.g. with meat

3. +verb_adj: a phrase that composed via adjunct-type composition and included a verb, e.g. eats quickly (note that adjective-noun phrases are not the only type of adjunct-type composition)

4. −verb_adj: a phrase that composed via adjunct-type composition and did not include a verb; e.g. tasty meat

where +verb_arg, −verb_arg, and −verb_adj always had the same noun, and +verb_arg and +verb_adj always had the same verb. These four types of compositional phrases are further illustrated in Table 1.

Table 1. Stimuli Design.

An example set of two-word compositional and one-word non-compositional items sharing a given noun or verb.

	Argument (argument-type) composition	Adjunct (adjunct-type) composition
+verb	eats meat	eats quickly
verb control	eats fghjl / fghjl eats	eats fghjl / fghjl eats
−verb	with meat	tasty meat
−verb (noun) control	meat fghjl / fghjl meat	meat fghjl / fghjl meat

Stimuli consisted of 36 sets of four compositional phrases and two non-compositional items. We implemented the “minimal composition paradigm,” where composition is isolated to two-word phrases and contrasted with one-word non-compositional items consisting of an unpronounceable letter string and a real word (Bemis & Pylkkänen, 2011; Westerlund et al., 2015). Each non-compositional item was presented in one of two possible word orders, for a total possible four one-word items. The format for the non-compositional one-word items ([noun/verb] + [non-pronounceable letter string]) was counter-balanced for the real word being in phrase-initial or phrase-final position.

Our hypothesis concerns verbs that denote events, so we selected verbs that were eventive rather than stative. Of our 36 verbs, we have one traditionally stative verb – love – but this verb is sometimes used in the continuous aspect (I am loving, I was loving), so it is not as strongly stative as have or own. The other 35 verbs are strongly eventive (e.g. I am eating, I am kicking, I am buying, etc.)

A given verb in each compositional set had to be able to compose with a direct object, as in “eats meat,” but, conversely, could not be so strongly transitive as to require a direct object. Therefore, all verbs were chosen to be optionally transitive. This optional transitivity allowed for compositional phrases of the +verb_adj type, as in eats quickly, where there is no direct object. Moreover, all verbs had present tense inflection in order to ensure they were interpreted in the active voice. A verb in a +verb_arg phrase like chews gum, when presented in the past tense – chewed gum – might be read as a passive participle – as in “gum that is chewed.” In such a case, an adjectival participle would compose via adjunct-type rather than argument-type composition.

Similar constraints were placed on the noun stimuli. A given noun had to be able to compose in a variety of contexts (with an adjective, a preposition, and a verb) without requiring extra plural suffixes or determiners (throw stones, throw the stone), the addition of which might involve another type of compositional operation. Therefore, only mass (non-count) nouns were included. Also, since one of our hypotheses concerns the status of events as denoted by verbs, we avoided event-denoting nouns, though marginally event-denoting nouns in our stimuli included the nouns traffic, crime, opera, praise, and pardon.

Note that, due to the constraints of our particular two-word phrase sets (see Table 1), we could not compare two adjunct-type phrases. The nature of our similarity analysis required that a given two-word phrase be compared only to another two-word phrase that shared either a noun or a verb. This is because we are investigating the changes in the voxel pattern of a given base word – e.g. eats or meat – when composed with a function head, an argument, or an adjunct. While it is possible that the operations of argument composition and adjunct composition each have their own stable and distinctive patterns in the brain regardless of what words are composed, it is more likely that the instantiation of the operation is highly dependent on the words composed, and thus highly distributed spatially in the brain. Under this view, a voxel pattern for function-argument composition of eats meat cannot be expected to be the same voxel pattern instantiating the composition of plays guitar. For this reason, we need to make all similarity comparisons relative to an “anchor” word – either a shared noun or a shared verb in this case. Our two types of adjunct-type compositions did not share a word: an event-denoting adjunct-type phrase like eats quickly cannot be directly compared with the non-event-denoting adjunct-type phrase like tasty meat, even though these two phrases are in the same set and can each be compared with an event-denoting argument-type phrase like eats meat. Therefore, we had no means of assessing adjunct-type similarity profiles. However, because a number of previous studies have shown that adjective-noun compositional phrases like tasty meat activate left ATL, we were less interested in replicating such a result than querying other types of compositionality, specifically argument-type pattern similarity (eats meat compared with with meat) and verb-based pattern similarity (eats meat compared with eats quickly). This allowed us to test two different hypotheses about the role of AG specifically: On the one hand, if AG subserves argument-type composition in general (that is, application of arguments to any type of function head; in this case, either a verb or a preposition), then we predict argument-type pairs will elicit highly similar patterns of activation in AG. On the other hand, if AG is preferentially sensitive to event-denoting verbs, then specifically verb-sharing phrases will elicit similar patterns in AG.

Similarly, if left ATL is specialized for feature-based composition, either noun-sharing or verb-sharing phrases might elicit shared activation patterns. Moreover, if left ATL is sensitive to composition-type, we predict conserved patterns of activation for phrases that share either adjunct- or argument-type composition (where only argument-type shared composition is tested here.)

2.2.2 Stimuli norming

All words and word phrases were matched for length except for adverbs (in +verb_adj, e.g. “eats quickly”) compared with nouns (in +verb_arg, e.g. “eats meat”), where adverbs were significantly longer than nouns (M_adv=7.6, s.d. = 1.96; M_noun= 6.2, s.d = 1.97). Only adverbs and nouns were matched on frequency. We also collected imageability ratings on a 1–7 Likert scale (1 being lowest) on one- and two-word items. There was no significant difference between our noun-based stimuli and our verb-based stimuli (not including eats meat stimuli that included both a verb and a noun): M_{noun_items} = 4.74, s.d. = 0.83; M_{verb_items} = 4.39, s.d. = 0.58.

As noted above, all verbs appeared in the present tense in this study. When presented in a non-compositional one-word stimulus (e.g. bvref picks), some of these present tense verbs might be ambiguous between a verb and noun interpretation. However, in all such cases of possible ambiguity, the dominant form of the base word was a verb (assessed using Google Books Ngram Corpus, American English; Lin et al., 2012).

We also normed our two-word phrase pairs’ dissimilarity using a survey posted on Amazon Mechanical Turk (Buhrmester, Kwang, & Gosling, 2011). Instructions were as follows:

“In this survey, you will be asked to indicate how alike two instances of a word in two different contexts are. You will do this using a slider bar. 0 means “exactly the same” and 7 means “substantially changed. For example, a minimal change might be something like “stare at the cash register” and “ask about the cash register.” A moderate change would be the difference between “buys rice” and “grows rice.” In “buys rice”, the rice is likely packaged in a container, while in “grows rice” the rice is in a field on a plant.”

In this way, we were able to extract pairwise dissimilarity scores for phrases that shared a given noun or a given verb, and also average these scores to yield a measure of how much, on average, a given noun or verb changed depending on the word it was composed with. That is, pairwise dissimilarity scores for the phrase pairs (meat, eats meat), (meat, tasty meat), (meat, with meat), (eats meat, with meat), (tasty meat, with meat), and (tasty meat, eats meat) were averaged together to yield a mean dissimilarity score for “meat”; likewise, the pairwise dissimilarity scores for (eats, eats meat), (eats quickly, eats meat), and (eats, eats quickly) were averaged together to yield a mean dissimilarity score for “eats.” We could then compare the pairwise dissimilarity norms with “neural similarity” scores (see below for discussion), where “neural similarity” was the measure of how much a multi-voxel evoked pattern for a given noun/verb changed depending on the word it was composed with (where less change in the patterns indicates greater similarity). We could also analyze the mean dissimilarity norms to look for coarse similarity differences between noun-containing phrases and verb-containing phrases.

After filtering responses for English as a first language, completeness, time to response, fluency, and neurological disorders, between 16 and 40 subjects per each of 9 lists (one list for each phrase pair in a given set) remained. Responses from 16 randomly chosen subjects per list were then analyzed and used to calculate the pairwise and mean similarity norm scores. Interestingly, phrase pairs sharing a noun were rated as significantly (p<0.05) more similar on average (M=3.16, s.d.=0.25; where mean similarity norm on a 0–7 Likert scale, 0 most similar) than phrase pairs sharing a verb (M=3.44, s.d.=0.46).

2.3 Experimental Task and Design

The subject’s task on each trial was to read two simultaneously centrally presented words constituting either a compositional or one-word item. This phrase was presented for 3 seconds. The critical phrase was immediately followed by either a 9-second fixation cross, during which the subject need only passively view the screen, or a two-word phrase probe presented for 3 seconds and followed by a 6-second fixation-cross ISI. This probe was presented in capital letters and terminated with a question mark so as to distinguish it from the preceding critical phrase. If the initial critical phrase was compositional (e.g. asks nicely), then the probe was also compositional (e.g. INQUIRES POLITELY?); otherwise, the probe consisted of a noun/verb and a non-word letter string in the same order as the non-compositional one-word item it followed (e.g. asks xblrdc followed by INQUIRES PCXFDL?; or xblrdc asks followed by PCXFDL INQUIRES?). The subject was instructed to indicate by button press (yes/no) whether the probe phrase (or, in the case of a non-compositional trial, the probe word) was synonymous with the preceding phrase or word. If a trial had no probe, no response was required from the participant. 10% of trials had a probe phrase or word, and 30% of these catch trials had probes that were not synonymous. Probes from catch trials were excluded from analysis. The entire experiment consisted of 9 runs of 24 trials each.

2.4 Image acquisition and pre-processing

FMRI data were collected at the Hospital of the University of Pennsylvania on a 3T Siemens Trio System using a 32-channel multiple-array head coil. Four types of image sequences were collected for each participant: (1) a standard low-resolution anatomic localizer; (2) a high-resolution, T1-weighted sequence for localization of fMRI activity in standard stereotactic space; (3) T2*-weighted images from 9 experimental runs; (4) a B0 field map sequence for subsequent geometric unwarping of T2*-weighted images.

After acquiring T1-weighted anatomical images (TR=1630 ms, TE=3.11 ms, TI = 1100 ms, voxel size = 0.9 mm × 0.9 mm × 1.0 mm, flip angle 15°), we collected T2*-weighted images using a gradient-echo echoplanar pulse sequence (TR=3000 ms, TE=25 ms, voxel size=2 mm × 2 mm × 2 mm, flip angle = 90°, 41 axial slices). Slices were collected at 20° counter-clockwise to the anterior commissure to posterior commissure (AC-PC) plane. This slice orientation was chosen so as to maximize the volume of anterior temporal as well as temporo-parietal cortex within the acquisition, since the former region is particularly prone to signal loss from proximity to sinuses (known as “susceptibility artifact”, Patterson et al., 2007).

FMRI data were pre-processed offline using the AFNI (Cox & Jesmanowicz, 1999) software package. The first four volumes of each functional run were removed so as to allow the signal to reach steady-state magnetization. Functional images were slice-time corrected, and a motion correction algorithm employed in AFNI registered all volumes to a mean functional volume. Images were then unwarped via B0 field maps (using FSL software; http://www.fmrib.ox.ac.uk/fsl) to reduce non-linear magnetic field distortions. We applied a high-pass filter of 0.01 Hz on each run to remove low frequency trends. Images were transformed to Talairach standardized space (Talairach & Tournoux, 1988) and voxels were resampled in the process to 3.5 mm × 3.5 mm × 3.5 mm.

2.5 Analysis

2.5.1 ROIs and image analysis

Using AFNI (Cox, 1996), functional data were registered to the individual subject’s anatomical MRI. Transient spikes in the signal were removed using AFNI’s 3dDespike. Our a priori ROIs were left anterior temporal pole and left angular gyrus, which we delimited using AFNI’s CA_ML_18_MNIA atlas. Our anterior temporal ROI spanned labels “left temporal pole” and “left medial temporal pole”, while our angular gyrus ROI circumscribed only the atlas’s “left angular gyrus” ROI (see Fig. 1).

Figure 1.

ROI boundaries of left ATL and left AG.

For those voxels within a given ROI, multiple regression was used to generate parameter estimates (β) representing each voxel’s activity in each stimulus item condition within subject. Voxels’ βs were calculated by convolving all variables with a gamma-variate hemodynamic response function and entering them into a general linear model (GLM) (AFNI; Cox & Jesmanowicz, 1999). Motion estimates were included as regressors of no interest. After implementing our voxel selection criteria (see below), the per-voxel β values were entered into the similarity analysis.

2.5.2 Voxel selection and similarity analysis

In order to query the similarity space of the various composition conditions in each of our ROIs, we first had to identify which voxels to include in subsequent similarity analyses. For each subject, we selected those voxels which varied the most with respect to the contrast between compositional phrases (e.g. eats meat) and one-word items (fghjl eats), using a GLM at each voxel within bilateral ATL and bilateral AG. Because even a liberal t-threshold on the compositionality contrast revealed no differential activity in right ATL, we did not further analyze this region. The 100 voxels with the highest unsigned (positive and negative; see Discussion for motivation for including both) t-values from the compositionality localizer for each subject for each remaining ROI made up the pattern template for the similarity analysis. Having chosen which 100 voxels would constitute our per-subject, per-ROI vectors, we then modeled each stimulus event as a unique regressor in a GLM, and entered the stimulus item GLM βs for those previously chosen 100 best voxels into vectors of 100 values per condition per ROI per subject (see Fig. 2). We then conducted a correlation analysis over these pattern vectors using Pearson’s r. Our initial regions of interest (ROIs) included bilateral ATL and bilateral AG. Because right AG demonstrated no significant pattern similarity results in any voxel group, only results for left-lateralized ATL and AG are reported below.

Figure 2.

Diagram of similarity analysis. A vector of the N (50, 100, 200, and 500) best voxels’ β values for a given condition (e.g. +verb_arg, or eats meat) was correlated against another vector of β values from the same voxel array for a different condition in the same composition set (e.g. +verb_adj, or eats quickly).

While we report results for the 100-voxel set below, we also used three other voxel sets (50, 200, and 500) to confirm that our results were not idiosyncratic to an arbitrary feature selection criterion. These were entered into the similarity analysis as described for the 100 best voxels, and we report results for these voxel sets in Appendix Tables 1–6.

2.5.3. Neural similarity scores for individual nouns and verbs

In order to have a neural measure to relate to the Amazon Mechanical Turk similarity norms we had calculated for each noun and verb (see Section 2.2.2), we calculated the correlations from the 100 voxels previously chosen for analysis between evoked patterns of every noun-containing phrase and every verb-containing phrase. That is, we calculated multi-voxel pattern correlations for the phrase pairs (meat, eats meat), (meat, tasty meat), (meat, with meat), (eats meat, with meat), (tasty meat, with meat), (tasty meat, eats meat), (eats, eats meat), (eats quickly, eats meat), and (eats, eats quickly). We then correlated these noun- and verb-specific neural similarity scores with the respective pairwise similarity norm scores from the Amazon Mechanical Turk survey. We predicted that the neural similarity score, which was higher the more consistent a given noun or verb pattern was when being composed with other words, would be negatively correlated with the dissimilarity norm score, which was higher for nouns or verbs the meanings of which differed more depending on what words the nouns/verbs were composed with.

3. RESULTS

3.1 Categorical similarity analyses of shared verb and shared composition type

We evaluated the similarity of the multi-voxel patterns evoked by each item across the set of voxels that differentially responded to compositional and non-compositional conditions in a given ROI. That is, we chose the 100 voxels per person with the highest unsigned (positive and negative) t values from the composition-vs-non-composition contrast in each ROI, estimated the beta value for a given item at each voxel, calculated the correlation across the 100 voxels between pairs of items that shared a common concept, and averaged those correlations across the 36 items. Specifically, we contrasted two hypotheses of the role of left AG in two-word composition: (1) that the left AG is specialized for combinations involving argument-type composition, and/or (2) that the left AG is specialized for event/verb semantics.

We compared correlations between pairs like (a) eats meat and with meat, where both a noun is shared and putative composition type (argument saturation) is shared with pairs like (b) “eats meat” and “tasty meat,” where only a noun is shared, and pairs like (c) “eats meat” and “eats quickly,” where the verb is shared. We found a main effect of Condition ((a)shared noun and composition; (b) shared noun only; (c) shared verb only) in left AG (F(2,48) = 6.23, p =0.004). As shown in Figure 3, the shared verb correlations (r = 0.17) are significantly greater than correlations between noun-sharing phrases (r = −0.03; Welch’s t(25.1) = 8.03, p<0.001) or noun+composition-sharing phrases (r = 0.04; t(24.6) = 5.29, p<0.001). The only correlation significantly different from chance in left AG is that between verb-sharing phrases. This pattern was robust across several other voxel selection criteria (see Tables 1 and 2 in Appendix). These findings favor the hypothesis that the left AG is specialized for verb semantics.

Figure 3.

Pairwise correlations between relevant pairs of word phrases in 100 best voxels in left AG. Error bars are 95% confidence intervals.

The same set of comparisons between pairs of two-word phrases in left ATL yields a different pattern from that seen in left AG (see Tables 3 and 4 in Appendix for left ATL comparisons). The noun-sharing phrases and the noun+composition-sharing phrases were significantly greater than both chance (Appendix Table 3) and the verb-sharing condition (Appendix Table 4), though this was not robust across other voxel selection criteria. Prima facie, this might suggest that left ATL is tuned to information carried on the noun and to shared composition type, consistent with Westerlund et al., 2015. However, the overall pattern of ranked correlations was not robust across voxel sizes in left ATL as it was in the case of left AG: that is, the eats meat ~ with meat was not consistently the highest correlation across voxel groups in left ATL, and there was much greater variability in the ordering and magnitude of correlations in left ATL. More importantly, because the current stimuli were not well suited to exploring the full similarity profile of compositionality in left ATL, particularly because we did not have a pair of phrases that shared both adjunct-type composition and either a noun or a verb (see intro), we cannot tell from these data alone whether left ATL is sensitive to composition type, shared noun, or some other dimension of conceptual similarity.

In order to compare the overall similarity structure in left AG with that in left ATL, we performed a 3×2 ANOVA over the Fisher’s z-transformed subject means of Pearson’s correlation values in the 100-voxel group, with factors Condition (shared noun and composition; shared noun only; shared verb only) and ROI (left ATL; left AG). We found a significant interaction between Condition and ROI (F(2,96) = 5.02, p=0.008).

3.2 Continuous similarity analyses between subjects’ pairwise similarity rankings and neural similarity scores for pairs of two-word phrases

Implicit so far in our treatment of the categorical dimensions “+/− shared verb” and “+/− shared composition” is the assumption that the only change between pairs of word phrases in a given set is the words constituting those phrases. However, it might be that the meanings of our two-word phrases differ based on factors related to a syncategorematic “context” of the word phrases. On the one hand, meanings could vary idiosyncratically; for instance, the meaning of “meat” in eats meat vs. with meat might not differ much between these two contexts, but “rice” in grows rice vs. on rice might differ much more from one context to the next (where on rice calls to mind rice in an edible state, while grows rice is more evocative of farming the plant). On the other hand, nouns and verbs might differ systematically in their “changeability” across phrasal contexts; that is, noun-sharing phrases might be more or less variable around a noun than verb-sharing phrases are around a verb.

In order to check for this potential source of similarity structure in our stimuli, we first looked at the pairwise norming scores taken from the Amazon Mechanical Turk survey. We found that there was a significant difference (p<0.05) between noun-sharing pairs and verb sharing pairs (M=3.16 and M=3.44, respectively, on a Likert scale of 0–7, 7 being maximal difference between a pair of word phrases), indicating that, on average, noun-sharing phrases were rated as more similar than verb-sharing phrases. This normed similarity measure was not predictive of neural similarity in either left ATL or left AG at any feature level (50, 100, 200, or 500) when taking verb- and noun-based correlations together, nor was there a main effect of noun- vs. verb-sharing on neural similarity scores across ROIs. However, we found a significant correlation between AMT similarity norms of verb-containing phrases and neural similarity scores in the 100 best voxels in left AG (r = −0.12, p < 0.05; see Tables 5 and 6 of Appendix for AG and ATL correlations across voxel selection criteria).

4. DISCUSSION

While functional neuroimaging studies have made great strides in mapping brain areas involved in language processing, a model of the neural bases of semantic processing is still in its nascence. This may be in part due to the fact that the cognitive neuroscience of semantics does not always utilize linguistic theory. Indeed, it sometimes does not need to. After all, the legacy of model-theoretic semantics has concerned itself primarily with formalizing “the metaphysics of truth in natural language” rather than the various constraints on language processing or the representations of concepts (Seuren, 2009). Cognitive neuroscientists are often more explicitly interested in semantics as it deals with binding sensorimotor features of object-concepts, or how different categories of objects are represented with regard to action-oriented events, e.g., function vs. manipulability (Yee, Drucker, & Thompson-Schill, 2010), etc. For instance, early attempts to define “category-specific” regions of cortex using lesion studies provided evidence that damage to ATL was associated with deficits specific to the knowledge of living things (Gainotti, 1996), while damage to left temporo-parietal junction affected knowledge of man-made artifacts (e.g. wrench, hammer, etc. which, interestingly, are often also verbs) (Tranel, Damasio, & Damasio, 1997). However, such emphasis on accounting for sensorimotor and action-based properties of language may neglect the more abstract significations our language is capable of expressing, and thus miss potential means of generalizing certain embodied aspects of cognition. In this study, we expanded the purview of conceptual semantics from the domain of object concepts and action semantics to more abstract dimensions – here, composition type, argument structure, and event semantics.

We were particularly interested in two regions of the brain – the left ATL and left AG – implicated neuroanatomically as “convergence zones,” and also as “semantic hubs” for their involvement in processing compositional language. We found that the left AG displayed a markedly different pattern-similarity profile from that of left ATL. The only dimension of stimulus similarity that produced a detectable effect on neural similarity in AG was shared verb, and by extension, shared event. Left AG appears to be invariant to composition type, and therefore the level at which AG tracks argument structure may not be as general as that described by “argument-type” composition as denoted above, but rather may explicitly subserve verb argument structure, namely thematic relations. This is an important distinction, as there are many more types of argument structure in language than verbs and their arguments, and these data now behoove us to examine AG’s selective involvement in composing verbs and their arguments.

4.1 Angular gyrus and thematic relations

It is still unclear exactly what information carried on the verb might be engaging AG. Evidence that bilateral AG activity is parametrically modulated by the valency of a verb – that is, the number of arguments a verb can take (Meltzer-Asscher et al., 2012; Thompson et al., 2007, 2010) – suggests that the AG may read out the syntactic complexity of a verb constituent, rather than, or in addition to, the semantic content of the verb itself. Left AG has also been implicated in the detection of syntactic errors (Embick, Marantz, Miyashita, O’Neil, & Sakai, 2000). However, AG is also involved in the processing of connected discourse as opposed to unrelated sentences (Fletcher et al., 1995; Homae, Yahata, & Sakai, 2003; Xu, Kemeny, Park, Frattali, & Braun, 2005), suggesting that AG participates in the construction or analysis of event semantics. Thus it may be that AG acts as an interface between semantic memory and syntactic structure, mapping semantic-thematic relations onto structural constraints surrounding verbs and their arguments. Indeed, electrophysiological and neuroimaging studies support an overlap between (morpho-)syntactic and semantic-thematic verb violations. Kuperberg et al. (2008) compared three different types of verb violations: (1) semantic–thematically violated verbs (e.g. “at breakfast the eggs would eat”) (2) morphosyntactically violated verbs (e.g. “at breakfast the boys would eats”) and (3) real-world violations (e.g. “at breakfast the boys would plant”). They found that, unlike real-world violations, both semantic-thematic and morpho-syntactic violations elicited activity in a frontal/inferior parietal/basal ganglia network, in accord with previous electrophysiological findings that semantic-thematic and syntactic violations evoked P600 event-related potentials highly similar in latency and scalp distribution (Kuperberg, 2007; Hoeks et al., 2004). The authors concluded that this frontal/AG/basal ganglia activity reflected attempts to integrate structural constraints of the verb with semantic properties of the Agent NP argument (Buccino et al., 2001; Chao & Martin, 2000; Damasio et al., 2001; Fogassi et al., 2005).

Evidence from lesion analyses also suggests that such thematic role knowledge is privileged in bilateral AG. The literature on semantic knowledge has long distinguished between so-called taxonomic semantic knowledge, or knowledge of shared/commensurate features, and thematic semantic knowledge, or knowledge of the relations between object-concepts (crucially from different taxonomic categories) that play complementary roles in events. Speakers’ semantic errors tend to reflect either taxonomic fidelity (that is, uttering an incorrect word, but one which has commensurate features, such as when “apple” is named as “pear”) or co-occurrence fidelity (that is, uttering “dog” when “bone” was intended, reflecting the thematic relation between “dog” and “bone”) (Schwartz et al., 2011). Schwartz and colleagues (2011) analyzed the error typologies of 86 individuals with post-stroke aphasia and conducted voxel-based lesion-symptom mapping (VLSM) on each error type separately (with shared variance between error types regressed out). Taxonomic errors were mapped to left ATL lesions, while thematic errors were localized to left AG. This double dissociation between ATL and AG supports the view that the ATL and AG support distinct semantic computations, corresponding roughly to feature-based and relation-based operations, respectively (but cf. Lewis, Poeppel, & Murphy (2015) for evidence that both taxonomic and thematic associations engage AG, while ATL subserves taxonomic associations specifically).

It is interesting to note that word pairs in a thematic error, such as “dog” and “bone,” can be described as related by virtue of some implicit verb/event; in this case “chews” or “buries,” etc. That is, thematic knowledge is precisely knowledge of verbs and their arguments. Our current study provides evidence that verbs in particular, not nouns, and not just any argument-type composition, may indeed be the representational substrate of semantic knowledge in AG.

4.2. Semantic representations or semantic control?

There is some debate, however, as to whether AG is a hub for mapping syntactic and semantic representations, or if it is rather part of an extended regulatory “semantic control” network. Indeed, the functional heterogeneity of bilateral AG apparently defies neat description. While AG activity is most consistently and robustly elicited by tasks involving semantic processing, both in auditory and visual modalities (see Seghier, 2012, for review), AG is also implicated in the default network, where AG is deactivated during goal-oriented tasks (Shehzad et al, 2009); number processing (Dehaene, et al., 2003); attention and spatial cognition, where AG may play a role in shifting attention toward particular stimuli having greater salience in terms of motion, value, emotion, and meaning (Gottlieb, 2007); and verbal working memory retrieval and episodic memory retrieval (Vilberg & Rugg, 2008). Generally, AG activation increases with the amount of semantic information that can be retrieved from a given input, whether exogenously generated or self-generated during mentation (Binder & Desai, 2011; Seghier, 2012).

The ostensible functional heterogeneity of AG in the literature may arise more from ROI definitions that (unintentionally) obscure neuroanatomical divisions within AG. In a meta-analysis of studies comparing semantic tasks with high-vs.-low demands on executive control, Noonan et al., 2013, found a functional divergence between dorsal AG (bilateral, including dorsal/anterior AG and boundaries with superior marginal gyrus (SMG) and inferior parietal sulcus (IPS)) and left mid AG (somewhat closer to PGp than PGa), with respect to executive and representational roles in semantic processing. Dorsal AG showed reliably greater activation in high >low semantic conditions, and was characterized as allocating attention to semantic representations in a task-dependent and goal-driven manner. This characterization is not in itself inconsistent with a model of AG as a site of conceptual combination, as such compositional operations require selective attention to certain properties of events in order to construct higher-order concepts. However, dorsal AG’s role in semantic control was contrasted with mid AG, the activity of which was modulated by the semantic representational content of stimuli even when matched on task demands. While mid AG is associated with the “default mode network” (Raichle et al., 2001), and thus shows more positive activation in the absence of a task, it shows more negative activation for abstract as compared with concrete concepts (Binder, Westbury, McKiernan, Possing, & Medler, 2005; Wang, Conder, Blitzer, & Shinkareva, 2010), and more positive deflection from baseline for semantic as compared with phonological decisions matched on executive demands (Binder et al., 1999, 2009).

The left and right AG ROIs drawn in our study encompassed both dorsal and mid AG regions, but voxel features across subjects were highly dispersed across the ROI in standard space such that we could not determine a difference in pattern similarity profiles between dorsal and middle aspects of AG. It is thus possible that the patterns we report here captured a combination of executive demand and semantic-thematic representation similarities.

However, if there were differences across our dimensions of interest in executive demand, or syntactic/semantic complexity broadly construed, we might expect to have seen evidence for this in univariate contrasts. Yet we found no evidence of significant univariate differences across our dimensions of interest: there were no significant clusters in our ROIs for the noun-based vs verb-based phrase contrast (where eats quickly and eats were verb-based phrases, and tasty meat, with meat, and meat were noun-based phrases), and neither did the argument-vs-adjunct-type contrast reveal any significant differences in either ROI. Nevertheless, we cannot rule out the possibility that the voxel pattern correlations in our AG ROI also reflect semantic control processes.

4.3. Limitations, future directions, and conclusions

Our claim is primarily in regard to composition, rather than lexical effects per se, because we are investigating the changes in the voxel pattern “template” of a given base word – e.g. eats or meat – when composed with a function head, an argument, or an adjunct. The voxel selection criteria we used specifically targeted composition-responsive voxels; that is, voxels the activity of which changes maximally when adding another word to a given base word (e.g. adding the argument meat to eats.) Given that our dependent measure is change (or similarity) in voxel patterns, rather than univariate changes in activity across a cluster or ROI, we chose to include voxels that responded both maximally positively and maximally negatively to an instance of composition. Sampling the ends of both tails allows us to capture a greater range of possible variance in voxel patterns, a range that would be limited if we only looked at positive voxel changes. In addition to facilitating the pattern analysis, the inclusion of composition-negative voxels was motivated by emerging evidence that a region of left AG is part of the so-called “default network,” as discussed above. Seghier et al. (2010), found that the left mid AG was a region within the default network that responded more negatively to perceptual decisions than semantic decisions, though both stimuli elicited a negative divergence from baseline. Mid AG has also been found to be less active for more “difficult” semantic stimuli: e.g. more negative activation for items with longer decision or processing times (Binder et al., 2009; Noonan et al., 2013). Since compositional items might be understood as more semantically “rich,” or more “difficult,” the mid AG region might be expected to index compositionality, but in the composition-negative direction. We did not want to exclude this region when we cast our net over AG voxels, and so we included both composition-positive and composition-negative voxels. It should be noted that analysis of similarity patterns derived from composition-positive voxels alone yield highly similar profiles in both left ATL and left AG, though the shared verb correlation was slightly weaker in left AG.

As mentioned above, the voxels we selected for similarity analysis were highly spatially distributed across subjects, and we were not able to define a particular region of AG (dorsal vs. med) driving similarity patterns. Further study into the functional differences (1) between dorsal and mid AG and (2) between composition-positive and composition-negative voxels across the brain, may clarify whether voxels responding negatively to composition reflect attention-based or representation-based information about verb composition.

It is also important that we discern lexical effects from composition effects when characterizing the role of left AG in verb semantics. When examining voxel pattern similarity across composition-positive and composition-negative voxels, we do not find a correlation between the non-compositional one-word item eats and the compositional phrase eats meat. However, as the voxels were selected to maximize the differences between exactly this contrast, a lack of correlation is not only unsurprising, but expected. Instead of choosing voxels most sensitive to the composition vs. non-composition contrast, we collapsed the two-word compositional and one-word non-compositional conditions together and contrasted this combined “word condition” with the ITI fixation period. We then selected the 100 voxels and the 500 voxels within left AG with the highest positive t statistic for combined word task over fixation baseline. This more agnostic selection criterion allowed us to assess the eats ~ eats meat correlation and also compare it with the eats meat ~ eats quickly correlation. Using the word-vs.-fixation selection criterion, these correlations were neither significantly different from chance nor from one another. This indicates that the word task>baseline contrast is not optimal for testing the substrate of our verb-based effect, and that this effect is indeed driven by composition-sensitive voxels

While this study provides evidence that left AG contains patterns representing information specific to verbs, regardless of whether these verbs are composed with adverbial adjuncts or noun arguments, we cannot entirely rule out the possibility that AG is also involved with argument-type composition in general. In addition to “eats meat”-type verb phrases, the other argument-type compositional phrase included here was the prepositional phrase (e.g. “with meat”). Prepositions have several unique properties. High-frequency, semantically vacuous/impoverished prepositions might have a very different combinatorial effect than adjectives or verbs when composing with nouns (“tasty meat,” “eats meat,” respectively). Indeed, the preposition is little more than a function word, and lacks the semantic content carried on adjectives and verbs. Not only do the prepositions in our stimuli set have the highest average item frequency, but prepositions as a class may also combine with many more surface forms than either nouns or adjectives. This “compositional diversity” may render prepositions, and prepositional phrases, qualitatively different from the other parts of speech used here, and this diversity may make extraction of stable patterns from the prepositional phrase items less likely. Nevertheless, further study is needed to examine whether preposition function heads engage AG in the same way we found verbs do. While this study provides evidence that a shared argument (meat in eats meat and with meat) is not sufficient to drive pattern similarity in AG, it does not query whether a shared preposition (with in with x and with y) is possibly sufficient to drive similarity in the same way a shared verb is (eats in eats meat and eats quickly).

The current study only investigates cases of minimal composition: that of two words isolated from a sentence or discourse. However, it is unlikely that AG is only tracking this level of composition. There is abundant evidence that AG may engage in domain-general event processing in event structures as broad as discourse and in non-verbal depictions of events. Indeed, both ATL and AG are best described as “hubs” at a domain-general level. While the current study did not directly test the manner in which left ATL might subserve feature-based combination, a large body of literature suggests as much. In contrast to ATL, we find increasing evidence that AG is engaged in semantic integration of relation-based event structure, and we must now consider whether this distinction between ATL and AG is ultimately reducible to the well-attested difference between the ventral “what” pathway (the combinatorial hub of which is the ATL) and the dorsal “how/where” pathway (the integrational hub of which is the AG) (Binder & Desai, 2011). The AG is surrounded by the dorsal spatial attention networks, the posterior temporal regions involved in motion perception, and the anterior parietal regions involved in representing action (Kravitz, Saleem, Baker, & Mishkin, 2011). While AG may have originated as a dorsal “where/how” convergence zone of spatial, goal-oriented, and action information, it may have been co-opted by language to represent increasingly abstract relational information. These relations might be learned merely by tracking co-occurrence statistics (“dog” often co-occurs with “bone”; “eggs” often co-occur with “breakfast), or, more likely, these thematic relations are learned part and parcel of hierarchical structures arising in natural language syntax. Thus, the emergence of event and argument structure in thought and language may have been an extension of the already extant dorsal pathways underpinning action and goal understanding.

It is interesting to note, however, that while the fronto-temporal language network may have evolved to be strongly left-lateralized, it is less clear the degree to which right and left AG diverged with regard to processing events and representing thematic relations. Graves et al. (2010) offer a connectionist account of how noun-noun compounds, such as “lake house,” when compared with their less compositional reversals (“house lake”), show differential activity in right AG but not left AG. They suggest that left and right AG can be modeled as attractor networks, where such a network is said to settle into an attractor basin when it optimizes the error space in the mapping between inputs and outputs. Whereas left AG is suggested to have relatively narrow attractor basins, reflecting highly specific and constrained mappings between words and meanings, right AG may contain wider, shallower basins, representing more extensive overlap in meanings. This would accommodate “looser” meanings, and thus enable the interpretation of compounds like “dog bone” that lack the explicit (morpho)syntactic information (i.e. “a bone that a dog chews on”) that would otherwise aid in resolving the relation between the two nouns in the phrase. This attractor network account of the difference between left and right AG accords with Beeman and colleagues' (1994) “coarse semantic coding hypothesis” of the right hemisphere. In this study, “summation primes,” three words weakly related to a target word, were found to better prime a target when the triplet was presented to the left visual hemifield (right hemisphere, RH) than the right visual hemifield (left hemisphere, LH), while the converse was true for “direct primes,” where there was one strongly associated prime flanked by two unrelated primes. This was taken as evidence that RH contains larger semantic fields weakly activating concepts more distantly related to an input word, whereas LH contains smaller semantic fields that conservatively activate concepts highly related to an input word. This distinction might account for why we found evidence of verb-specific pattern conservation in left AG but not right AG. It may be that left AG subserves specifically strong thematic relations, while right AG weakly activates to a wider variety of compositional items. Further study is needed to examine whether such strong vs. weak relations between words in compositional phrases might differentially engage right and left AG.

We suggest that multi-voxel pattern similarity analysis is uniquely suited to address such questions. Our study has demonstrated the sensitivity of this technique to compositional operations even at the level of minimal two-word phrases. We are only just beginning to characterize the AG with respect to its involvement in semantic composition, and this study suggests that the “feature-function” dichotomy may be a fruitful distinction in beginning to operationalize the compositional processes occurring in both ATL and AG.

HIGHLIGHTS.

• We analyze multi-voxel pattern similarity to investigate semantic composition.

• We compare two ways in which angular gyrus (AG) might contain thematic information.

• We find that AG is specifically sensitive to semantic information carried on a verb.

• Study suggests “feature/function” dichotomy between anterior temporal lobe and AG.

Appendix

Table 1. Pearson’s r values for verb-relevant correlations in left AG.

Correlations in left AG between the argument-saturated (eats meat) item with (a) an argument-saturated phrases without a verb (with meat), (b) a phrase sharing only a noun, but not composition type or verb (tasty meat), and (c) a phrase sharing a verb but constituting adjunct phrase (eats quickly) without argument saturation. Pairwise comparisons are reported using the best 50, 100, 200, and 500 voxels with highest (unsigned) t-statistics from the composition -vs.-non-composition contrast in left AG.

	50 voxels	100 voxels	200 voxels	500 voxels
shared noun AND shared composition type (eats meat ~ with meat)	0.027	0.035	0.019	0.018
shared noun (eats meat ~ tasty meat)	0.010	−0.029	−0.0097	0.0036
shared verb (eats meat ~ eats quickly)	0.044	0.17 *	0.056†	0.059 *

^†: 0.05<p < 0.1;

^*: p <0.05

Table 2. T-tests of differences between verb- or noun-based correlations in left AG (p-values reported).

T-tests of pairwise differences between verb-sharing phrase correlations, noun-sharing phrase correlations, and noun+composition phrase correlations using the best 50, 100, 200, and 500 voxels with highest (unsigned) t-statistics from the composition -vs.-non-composition contrast in left AG. Correlations being contrasted are those between the argument-saturated verb phrase “eats meat” and those phrases delineated in Table 1 of Appendix.

	50 voxels	100 voxels	200 voxels	500 voxels
shared verb vs. shared noun	0.092 †	< 0.0001 *	0.052 †	0.034 *
shared verb vs. shared noun +composition	0.48 ns	< 0.0001 *	0.23 ns	0.11 ns
shared noun vs. shared noun + composition	0.54 ns	0.051 †	0.18 ns	0.62 ns

^†: 0.05<p < 0.1;

^*: p <0.05

Table 3. Pearson’s r values for verb-relevant correlations in left ATL.

Correlations in left ATL between the argument-saturated (eats meat) item with (a) an argument-saturated phrases without a verb (with meat), (b) a phrase sharing only a noun, but not composition type or verb (tasty meat), and (c) a phrase sharing a verb but constituting adjunct phrase (eats quickly) without argument saturation. Pairwise comparisons are reported using the best 50, 100, 200, and 500 voxels with highest (unsigned) t-statistics from composition -vs.-noncomposition contrast in left ATL.

	50 voxels	100 voxels	200 voxels	500 voxels
shared noun AND shared composition type (eats meat ~ with meat)	0.030	0.13 *	0.071†	0.026
shared noun (eats meat ~ tasty meat)	0.026	0.088 *	0.052	0.042
shared verb (eats meat ~ eats quickly)	0.027	0.023	0.012	0.037

^†: 0.05<p < 0.1;

^*: p <0.05

Table 4. T-tests of differences between verb- or noun-based correlations in left ATL (p-values reported).

T-tests of pairwise differences between verb-sharing phrase correlations, noun-sharing phrase correlations, and noun+composition phrase correlations using the best 50, 100, 200, and 500 voxels with highest (unsigned) t-statistics from composition -vs.-non-composition contrast in left ATL. Correlations being contrasted are those between the argument-saturated verb phrase “eats meat” and those phrases delineated in Table 3 of Appendix.

	50 voxels	100 voxels	200 voxels	500 voxels
shared verb vs. shared noun	0.96 ns	0.041 *	0.27 ns	0.92 ns
shared verb vs. shared noun +composition	0.91 ns	0.0050 *	0.13 ns	0.66 ns
shared noun vs. shared noun + composition	0.91 ns	0.13 ns	0.74 ns	0.62 ns

^†: 0.05<p < 0.1;

^*: p <0.05

Table 5. Pearson’s r values for correlations of AMT similarity norms with neural similarity scores in left AG split by nouns and verbs.

Comparisons of correlations between similarity norms from AMT survey of pairwise phrase similarity (inclusive of both compositional (e.g. eats meat) and non-compositional (e.g. meat) items) and neural similarity norms calculated by averaging correlations between all pairs of phrases sharing a verb and all pairs of phrases sharing a noun (including both compositional and non-compositional items. Table 5 shows those verb- and noun-based correlations in best (unsigned) 50, 100, 200, and 500 voxels in left AG.

	verbs	100 voxels	200 voxels	500 voxels
verbs	−0.10	−0.12 *	−0.096	−0.14
nouns	−0.016	0.023	−0.039	−0.028

^†: 0.05<p < 0.1;

^*: p <0.05

Table 6. Pearson’s r values for correlations of AMT similarity norms with neural similarity scores in left ATL split by nouns and verbs.

Comparisons of correlations between similarity norms from AMT survey of pairwise phrase similarity (inclusive of both compositional (e.g. eats meat) and non-compositional (e.g. meat) items) and neural similarity norms calculated by averaging correlations between all pairs of phrases sharing a verb and all pairs of phrases sharing a noun (including both compositional and non-compositional items. Table 6 shows those verb- and noun-based correlations in best (unsigned) 50, 100, 200, and 500 voxels in left ATL.

	50 voxels	100 voxels	200 voxels	500 voxels
verbs	−0.013	0.041	0.038	0.026
nouns	−0.090	−0.097	−0.034	−0.048

^†: 0.05<p < 0.1;

^*: p <0.05