Dissociating neural subsystems for grammar by contrasting word order and inflection

Abstract

An important question in understanding language processing is whether there are distinct neural mechanisms for processing specific types of grammatical structure, such as syntax versus morphology, and, if so, what the basis of the specialization might be. However, this question is difficult to study: A given language typically conveys its grammatical information in one way (e.g., English marks “who did what to whom” using word order, and German uses inflectional morphology). American Sign Language permits either device, enabling a direct within-language comparison. During functional (f)MRI, native signers viewed sentences that used only word order and sentences that included inflectional morphology. The two sentence types activated an overlapping network of brain regions, but with differential patterns. Word order sentences activated left-lateralized areas involved in working memory and lexical access, including the dorsolateral prefrontal cortex, the inferior frontal gyrus, the inferior parietal lobe, and the middle temporal gyrus. In contrast, inflectional morphology sentences activated areas involved in building and analyzing combinatorial structure, including bilateral inferior frontal and anterior temporal regions as well as the basal ganglia and medial temporal/limbic areas. These findings suggest that for a given linguistic function, neural recruitment may depend upon on the cognitive resources required to process specific types of linguistic cues.

Despite the great diversity of human languages, the neural basis of language processing has been documented in only a very few languages. To the extent that the neural underpinnings of language are truly universal, the available research may reflect quite accurately what would be found across languages. Alternatively, the fact that the grammars of different languages encode information in different ways may place different processing demands on the neurocognitive systems supporting language (1). For example, in English the order of the words in the sentence John gave his lunch to Mary encodes the grammatical “dependency relationships”—essentially, who did what to whom. Ordering the words differently would convey a different meaning or no meaning at all. In other languages such as German or American Sign Language, word order is less restricted because dependency relationships can be marked by other cues, such as tagging words with inflectional morphemes (e.g., in German, suffixes are added to words within the noun phrase to mark the noun's “case” or role in the sentence, e.g., as “doer” or “receiver” of an action). Whether these different strategies for encoding grammatical information rely on a unitary network of brain regions specialized for processing “grammar” in a broad sense, or whether they impose distinct processing demands relying on nonidentical neural mechanisms, is a fundamental question. It has implications for our understanding of the neurocognition of language, for its relationship to other capacities such as working memory, and for the development of empirically based linguistic theories. However, it is ordinarily challenging to compare morphological and word order systems for encoding the same type of information, as most languages do not provide the option of using either one or the other mechanism, and it is difficult to match stimuli and subjects across languages. Here we present a study using a single natural human language—American Sign Language (ASL)—whose grammar allows us a unique opportunity to directly contrast these two means of marking grammatical roles within the same language and subjects.

ASL is a natural human language used among the deaf community in the United States and parts of Canada, with a grammar that is historically unrelated to, and quite distinct from, that of English. A feature of ASL's grammar that made it ideal for the present study is that whereas languages typically use either word order (e.g., English) or obligatory inflections (e.g., German, Hungarian) to mark grammatical roles, ASL uses both devices. One class of ASL verbs, known as agreeing verbs, can mark grammatical roles by using inflectional morphemes that specify the origin or endpoint and the direction of the movement of the sign (2). However, these morphemes are not obligatory in ASL—the same meaning can be expressed via word order, without inflections (Fig. S1). Similarly, the temporal aspect and number of an event (for example, indicating that an action was repeated or performed daily, rather than performed just once) can be marked in ASL either by inflections on the verb or in an adverbial phrase with uninflected signs. In some cases there are some differences of emphasis or specificity when such inflections are used versus when they are not. For example, sentences with spatially marked (definite) nouns must have agreement morphemes on the verb, whereas comparable sentences without these inflections can be either definite (e.g., if there is a proper noun or a demonstrative noun phrase) or indefinite (3). Overall, however, the meanings of the two types of sentences are highly similar and perhaps as close as it is feasible to achieve within a single language. Thus, using ASL we were able to construct sentences comprising the same major lexical items in the same sequence, with and without complex inflectional morphology. This allowed us to determine whether the processing of grammatical dependency relationships based on these two sources of information involves similar neural systems.

One area that has been consistently implicated in the processing of both inflectional morphology and grammatical word order information is the left inferior frontal gyrus (IFG) (4–15). This could be interpreted as suggesting a fairly general role of the left IFG in grammatical processing consistent with the classical notion of Broca's area (16). On the other hand, “Broca's area” comprises three cytoarchitecturally distinct areas (Brodmann's areas 44, pars opercularis; 45, pars triangularis; and 47, pars orbitalis) and functional dissociations have been found between them: pars orbitalis has been implicated in the selection of semantic information from long-term memory (17, 18) but also in the generation of inflected words (10, 19) and processing of word category information (20); pars opercularis in phonology (18), oro-facial movements (21), and syntactic processing (4, 5, 7, 13, 22); and pars triangularis for both syntax (e.g., syntactic movement, word order) and inflectional morphology (4, 8–11, 13, 23, 24).

Studies in German that manipulated word order found IFG pars triangularis activation increased in a linear fashion with the degree to which sentences violated the canonical ordering rule (4, 5, 9). Other studies report pars triangularis/opercularis activation for syntactic movement (complex sentences in which a syntactic constituent is moved to a noncanonical position, e.g., in relative clauses such as The lawyer that the senator attacked _ was disbarred where the phrase “the lawyer” is moved from the position marked by the underscore) (7, 8, 23). Because moved syntactic constituents must be related to others separated by intervening words, observed brain activation may reflect increased working memory load rather than computations of syntactic relationships per se (25). Accordingly, a number of areas activated in such word order manipulation studies, including the IFG, have all been implicated in strategic processes involved in maintaining sequential information in working memory (26). However, other evidence suggests that IFG activation may be specific to certain types of syntactic operations and not to others that would also impose working memory load (23). As well, an account attributing IFG activation for grammatical processing solely to working memory is inconsistent with the finding of IFG activation for the comprehension (15) and production (10 –12, 14) of single, morphologically inflected words.

Our precision in understanding the functional organization of the IFG is limited, however, because comparisons are typically made between studies. There are a number of limitations to this approach, including the varying approaches used for spatial normalization and statistics across different laboratories, as well as individual variability in the locations of cytoarchitectonic borders with respect to gross anatomical boundaries (27). Thus a within-subjects approach is preferable for attempts to dissociate subtle differences in functional localization within the IFG. Most studies have examined either inflectional morphology or word order, but have not directly compared them within subjects. One study that included violations of both syntactic word order and morphology within subjects found identical foci of activation within the left Brodmann's area (BA) 45, as well as right BA 44/45, for both conditions, although a direct comparison between the two conditions was not reported (13). Thus whereas generally overlapping areas of activation have been reported within BA 44/45 of the IFG for inflectional morphology and word order, a lack of direct comparisons with carefully controlled stimuli limits our ability to conclude that Broca's area is used in a general-purpose way for both types of grammatical processing.

Beyond the IFG, several other brain regions have shown activation in studies of both word order and morphological processing. These include the dorsolateral prefrontal cortex (DLPC) (comprising the inferior frontal sulcus and the middle frontal gyrus immediately dorsal to the IFG), medial prefrontal regions, regions along the superior temporal sulcus (STS) (including the superior and middle temporal gyri), the basal ganglia (particularly the caudate nucleus), and the cerebellum (4, 8 –14, 24, 28, 29). Several of these regions have also been implicated in other language functions, including lexical retrieval and phonology (14, 17, 18, 22, 28, 30–36) and working memory, nonlinguistic memory, and sequencing (26, 37, 38). As noted above, the ability to maintain the words of a sentence in working memory, in the correct sequence, is an essential part of syntactic processing and indeed several models of syntactic processing heavily emphasize this process (e.g., 39, 40).

Thus overall the picture that has emerged is that of a network of brain regions, particularly involving the left IFG but extending into other lateral and medial prefrontal and superior/middle temporal regions and to the basal ganglia, that is engaged in the processing of grammatical information whether conveyed by word order or by inflectional morphology. However, as noted above, no study has directly compared the processing of these two means of conveying grammatical information. Thus although similar, overlapping networks may be engaged by both types of processing, and there may be subtle differences in the relative weighting of activation in different regions, or specialized subareas within these regions, that reflect differential processing demands of word order vs. inflectional morphology. In the present functional (f)MRI study, we contrasted the processing of ASL sentences made up of the same lexical items, but in which the same grammatical dependency relationships were conveyed either through word order cues or by inflectional morphology (Fig. S1 and Movie S1). Deaf native signers performed a semantic monitoring task while viewing digital movies of the sentences, to ensure that participants would fully process the sentences. Brain activation during ASL sentence comprehension blocks was compared to that evoked by a control condition in which each movie comprised three of the ASL sentences, digitally superimposed and played backward, with participants performing the nonlinguistic task of pressing a button every time they noticed two hands that were in the same configuration (e.g., both hands in closed fists, both hands with index finger extended, etc.). Comparing these two conditions allowed for the subtraction of brain activation elicited by features of the stimuli that were not of interest in the present study, such as the perception of facial expression, handshapes, and biological motion, while requiring attention to the stimuli but discouraging attention to residual meaningful aspects of the stimuli.

We predicted activation in an overlapping network of brain regions both for the word order and in the inflected morphology conditions, including all of the areas described above: the left IFG (Broca's area), the caudate nucleus, the DLPC, and the STS. Of interest was the possibility that functionally distinct subregions might emerge within this network, which would suggest nonidentical neural mechanisms across ways of conveying grammatical dependency relationships. If there is a set of brain regions that operate in concert to process grammatical dependency relationships in an abstract way, then identical patterns and levels of brain activation might be predicted in the above network regardless of whether this information was encoded by word order or by inflectional morphology. On the other hand, if the different ways of conveying these grammatical relationships impose distinct processing demands, then a difference in the balance of activation across the above-mentioned regions might be found depending on the availability of different grammatical cues. Because both grammatical processes have been shown to activate the IFG across different studies, we predicted overlapping IFG activation for both word order and inflectional morphology, although we remained open to the possibility that a direct within-subjects comparison might reveal subtle differences in localization. We further predicted that sentences relying heavily on inflectional morphology might preferentially activate the basal ganglia (10, 11, 24) and possibly the middle/posterior STS (4). On the other hand, we hypothesized that sentences relying entirely on word order information would impose greater demands on working memory and so would activate the DLPC, as well as the anterior STS region that has been found activated for manipulations of syntactic word order (22, 34, 35).

Results

Behavioral Data from fMRI Task.

Due to technical failures, behavioral response data were recorded from only eight participants during fMRI scanning, although all participants performed the behavioral task. The fMRI data from one of these participants were unusable, so we were able to analyze the responses from only seven participants. Overall, accuracy was quite high on both the semantic task performed during ASL sentences (mean, two errors per subject across 72 trials; range, zero to six errors) and the bimanual symmetry detection task (one subject made four errors; no other subjects made any errors) performed during backward-layered control stimuli. We performed a repeated-measures ANOVA with sentence type and ASL/backward layered as factors. There was a main effect of ASL/backward layered, F(1, 6) = 8.63, P = 0.026, reflecting greater accuracy for backward-layered stimuli, but no difference between sentence types and no interaction between the two factors.

fMRI Data.

We examined the activation for each sentence type separately, contrasted with its respective backward-layered control condition and thresholded at P < 0.05 (corrected). As can be seen in Fig. 1, whereas an overlapping network of brain regions was engaged, differences can be observed within this network as a function of sentence type.

Fig. 1.

Brain areas involved in processing ASL sentences. Activation is shown for each ASL sentence type compared to its matched backward-overlaid control movie condition, overlaid to show regions of overlap (purple) and nonoverlap (red for word order, blue for inflectional morphology). Z maps were thresholded using clusters determined by z > 2.3 and a (corrected) cluster significance threshold of P = 0.05.

Word order vs. backward-layered word order.

Word order sentences elicited bilateral frontal activation that extended from the DPLC ventrally into the IFG pars triangularis and pars orbitalis, and orbital gyri, as shown in Fig. 1. Whereas the pattern of activation appeared generally symmetrical between the two hemispheres, at the chosen statistical threshold cluster centers appeared in the DLPC bilaterally, the left pars triangularis, and the right pars orbitalis. There was also extensive activation of the medial superior frontal gyrus, including the supplementary motor area (SMA). In the temporal lobes, robust bilateral activation was focused along the middle temporal gyrus (MTG) and STS, extending from the middle portion of the MTG to the temporal pole, as well as medially into the hippocampus and parahippocampal gyri. A cluster of activation was also observed in the left hemisphere (LH) angular gyrus. Details of activation loci and cluster sizes are provided in Table S1.

Inflectional morphology vs. backward-layered inflectional morphology.

An overall similar pattern of activation was observed for inflectional morphology sentences as for word order sentences, as shown in Fig. 1. This pattern included the LH (but not right hemisphere, RH) the IFG/DLPC, medial frontal regions (including SMA), the bilateral MTG/STS, the LH angular gyrus, and medial temporal regions. However, some differences were also observed. In the RH frontal lobe, no activation was found in the IFG pars triangularis or the DLPC, although activation was present in the pars orbitalis and orbitofrontal gyri bilaterally. Further, activation was observed in the RH angular gyrus and in the basal ganglia (LH caudate nucleus and RH globus pallidus) for inflectional morphology sentences that was not present for word order sentences. Finally, there was bilateral cerebellar activation for inflectional morphology but not for word order sentences.

Word order vs. inflectional morphology.

Having characterized the patterns of activation for each sentence type, we next compared the two sentence types statistically. This was a “differences of differences” comparison, in which the activation for each ASL sentence type minus its backward-layered control condition was compared to the other sentence type, again minus its backward-layered control. Due to the smaller size of these differences, results were considered significant at Z > 1.6 (uncorrected), with a minimum cluster size of 0.2 mL (25 voxels). The results of this contrast are shown in Fig. 2, with details in Table S2.

Fig. 2.

Differential activation by word order vs. inflectional morphology sentences. Regions showing greater activation for sentences containing inflectional morphology, compared to word order alone, are shown in blue/green. Regions showing greater activation for word order vs. inflectional morphology are shown in yellow/red. Activations were thresholded at z = 1.6, uncorrected.

Several brain regions were more activated by sentences that included inflectional morphology to convey grammatical dependency relationships. The largest of these areas were in the temporal lobes, with foci in anterior and posterior regions of the MTG/STS, and also in medial structures including the hippocampi, the parahippocampal gyri, and the amygdalae bilaterally. The medial temporal activation clusters were extensive and encompassed midbrain structures including the pallidum and thalamus as well. Other areas that showed greater activation for inflection included the LH IFG pars triangularis, the IFG pars orbitalis/orbital gyrus bilaterally, the LH supplementary motor area, and the cerebellum bilaterally.

Sentences relying exclusively on word order elicited greater activation than those that included inflectional morphology in the DLPC bilaterally, the RH IFG pars triangularis, and the LH angular gyrus.

Assessing Processing Difficulty of Each Sentence Type.

An important question is whether the two sentence types differ in complexity or in other associated dimensions, such as frequency, that might confound our comparison. Whereas both types of sentences are common in everyday ASL signing, no corpus information is available to determine their precise frequencies. The word order sentences contain less redundant marking and therefore might be more difficult to process; or, equally likely, the inflected sentences are more marked and thus might be more difficult. To address this issue, we conducted a follow-up behavioral experiment in which nine deaf native signers (including two who had been in the fMRI experiment) performed a very similar semantic monitoring task, using the same sentence materials, as in the fMRI study. The only difference from the fMRI task was that in the behavioral study, a target occurred in each sentence, and targets were selected to occur late in each sentence to assess the complexity of the whole sentence that preceded it. Reaction times were predicted to reflect sentence complexity on the assumption that, if greater resources were allocated to parsing the sentence, then target detection would be slowed. Similar logic has been used previously in studies of sentence processing (41, 42) and is implicit in eye movement and self-paced studies of reading (43). Although performance on the task does not strictly require processing of the syntactic structure of the sentences, grammatical processing seems to occur automatically (41, 42). The results showed no difference between the word order and inflectional morphology conditions [895 vs. 944 msec, respectively; F(1, 8) = 0.19, P = 0.676, R² = 0.001]. Although indirect, these results suggest that the two types of sentences do not differ significantly in processing complexity.

Discussion

The goal of the present study was to identify the neural systems underlying grammatical dependency relationships. In particular, we wished to determine whether separable subsystems are involved when the same information is conveyed exclusively through the sequence of words or with inflectional morphology. The results indicated a dissociation between the neural substrates of word order and morphosyntactic processing under conditions in which lexical–semantic content was held constant. These findings support the notion that specific parts of the neurocognitive system recruited for grammatical processing are dependent on the type of information that must be processed.

Before turning to these dissociations, we note that overall our results for ASL sentence processing highlight the expected network of neural areas that mediate sentence comprehension. Although identified here in native signers, this network is highly similar to that already described in the literature for spoken languages, encompassing frontal areas including what is traditionally referred to as Broca's area (IFG), the DLPC, large portions of the lateral temporal cortex (STS and MTG), and the temporo-parietal junction including posterior STS and angular gyri (classically referred to as Wernicke's area). In addition, activations were observed in the SMA, a key structure for language production also shown to assist in language comprehension (10, 44), and in medial temporal regions encompassing the parahippocampal and hippocampal gyri, recently proposed to mediate semantic memory access (29) during language processing.

Of primary interest, we found complementary sets of brain regions within this network that were differentially recruited depending on the way grammatical dependency relationships were marked. On the one hand, bilateral posterior and anterior temporal regions, along with the LH pars triangularis, bilateral orbitofrontal (including IFG pars orbitalis), and cerebellar regions and the RH pallidum, were more active when inflectional morphemes conveyed grammatical dependency relationships. On the other hand, bilateral DLPC, the RH pars triangularis, and LH angular gyrus were preferentially engaged during word order processing, although overall the extent of brain area activated more by word order sentences was quite small compared to the areas activated more by inflectional morphology sentences. Overall, the differences in activation between these different sentence types were subtle. Although the results of the comparison between sentence types are presented using statistics uncorrected for multiple comparisons, we are confident that these results are not spurious. First of all, the comparison between conditions was restricted to brain regions that showed robust (corrected for multiple comparisons) activation for at least one ASL sentence type relative to control stimuli. Further, we imposed a minimum cluster size of 25 voxels (0.2 mL) to ensure that the areas identified represented contiguous “blobs” of activation.

One focus of the present study was on the response of the inferior frontal gyri. Previous studies implicated the IFG (primarily in the LH) in both inflectional and word order processing, but these two types of grammatical processing were never directly compared within subjects. Some studies suggested that working memory load in sentence processing was an important determinant of IFG activity (25, 26); however, this region is also activated by single inflected words (10–12, 14, 15). The present study confirmed a role for the LH IFG (BA 45, pars triangularis) in the processing of inflectional morphology, along with more ventral frontal regions including the pars orbitalis and the orbitofrontal gyrus. The pars orbitalis (BA 47) has been associated with the selection of information from semantic memory (17) and is also activated in a variety of studies of grammatical processing, including verb generation and the production of inflected forms (10, 19). This has been hypothesized to be due to the cognitive control demands (17) such as selecting an appropriate inflection among competing alternatives. Following this interpretation, in the present study the bilateral orbitofrontal activation may reflect greater cognitive control demands imposed by the presence of inflectional morphology. In contrast, word order sentences yielded greater activation in the RH IFG pars triangularis. Whereas the LH IFG is most commonly associated with grammatical processing, previous studies have found RH IFG activation for violations of word order and inflectional morphology (13), as well as of less canonical but grammatically acceptable word orders (9), and in other conditions of increased processing load (45, 46). These are in line with the greater activation seen here for sentences that place heavier demands on processing word order. Overall, the results clarify that even for the conveyance of a specific type of grammatical information, dependency relationships, the patterns of brain activation differ according to the type of linguistic structures that carry that information, rather than there being a specific brain area or network devoted to processing “grammar” in a unitary sense. In what follows we discuss other features of these complementary networks.

For the inflectional morphology sentences, one notable finding is the activation in anterior STS/MTG, which is not typically implicated in studies comparing isolated inflected vs. uninflected words. Indeed, we had predicted the opposite pattern, with greater anterior STS activation for word order sentences. This prediction was based on previous results showing greater anterior STS activation for manipulations of word order (34, 35). It has been hypothesized that the anterior STS may have two functionally separable subregions for language processing: building syntactic phrase structures (47, 48) and the integration of syntactic and semantic information (34, 35). Although the design of our experiment does not allow separation of these two functions, a plausible interpretation of the present data is that the anterior STS is recruited more heavily when inflectional morphemes are necessary cues to building syntactic structure, compared to sentences where the syntactic structure is evident from word order alone. An additional, nonexclusive hypothesis is that the complex word forms created by ASL morphology involve the integration of syntactic and semantic information.

The posterior STS was also activated more strongly by inflectional morphology sentences, particularly in the RH. This activation falls within the STSp area identified in several studies of biological motion processing (49–51), in particular the part of this region preferentially activated by hand movements (52, 53). It is also in or adjacent to an area we previously found to be activated by ASL perception in native signers, but not late learners of ASL (54). These data together suggest that the function of the RH posterior STS, which normally possesses sensitivity to biological motion, may be altered by early sign language learning such that it responds preferentially to particular kinds of hand movements that convey grammatical information in native signers.

A somewhat unexpected finding was of hippocampal/parahippocampal activation for inflectional morphology sentences. These regions have not previously been reported for single-word studies of inflectional morphology, suggesting that their activation is related to the role of inflectional cues in processing sentences rather than to morphological complexity per se. As the hippocampal region has been primarily suggested to play a role in semantic encoding and retrieval, its role in resolving grammatical dependency relationships on the basis of cues from inflectional morphology remains to be elucidated. It may be that inflectional morphology increases encoding demands, perhaps simply due to the increased number of morphemes to be encoded.

Other areas activated more by inflectional morphology than by word order sentences included the basal ganglia, the cerebellum, the SMA, and the corpus callosum. The fronto-basal ganglia system has been suggested to be involved in the computation of grammatical constructions, such as inflected forms (29). More broadly, cortico-basal ganglia and cortico-cerebellar circuits are widespread in the brain with targets including several areas activated more by inflectional morphology here (the SMA and inferior and middle frontal gyri), and these circuits participate in a variety of cognitive functions (55). The present data suggest that these regions form an integrated network that is preferentially engaged in processing inflectional morphology. There does not seem to be a single area (e.g., LH IFG) that is solely responsible for this type of grammatical processing. With regard to the corpus callosum, recent work has revealed sensitivity of fMRI measures to callosal activation under conditions of increasing interhemispheric transfer (56). Thus activation in the genu of the corpus callosum here may be indicative of greater information flow between the two hemispheres for inflectional morphology sentences.

It is striking that one of the brain regions most extensively activated when word order, rather than inflectional morphology, conveyed grammatical dependency information was the DLPC. This region has been implicated in the deployment of strategic processes for encoding information (including order information and suprathreshold loads) in working memory (26, 37). This result thus supports our hypothesis that in the absence of morphological information, using word order to resolve syntactic dependency relationships imposes additional working memory demands. This interpretation is further supported by greater activation of the LH angular gyrus by word order sentences; the inferior parietal lobe is another key area in verbal working memory, in both speakers and signers (57).

Throughout this paper we have described the word order and inflectional morphology sentences as two distinct types or categories. In practice, however, sentences and stretches of discourse can include both types of constructions. Moreover, we do not claim that two entirely separate brain systems exist, only one of which processes a given type of sentence. Rather, we designed stimuli for the present study that clearly exhibit one or the other type of device, to discern changes in the patterns of activation within a distributed but highly overlapping network of areas. In normal language processing, it is likely that all of these areas are activated to varying degrees; but the present results suggest that different parts of this network may be responsible for different types of grammatical devices.

Conclusion

To summarize, we exploited a property of American Sign Language, unique among languages thus far studied with neuroimaging, to directly compare the neural systems involved in sentence processing when grammatical information was conveyed through word order as opposed to inflectional morphology. Critically, this comparison was made within subjects, while tightly controlling syntactic complexity and semantic content. Reliance on word order (serial position) cues for resolving grammatical dependencies activated a network of areas related to serial working memory. In contrast, the presence of inflectional morphology increased activation in a broadly distributed bilateral network featuring the inferior frontal gyri, the anterior lateral temporal lobes, and the basal ganglia, which have been implicated in building and analyzing grammatical structure. These dissociations are in accord with models of language organization in the brain that attribute specific grammatical functions to distinct neural subregions, but are most consistent with those models that attribute these mapping specificities to the particular cognitive resources required to process various types of linguistic cues. This result suggests that future investigations into the neural bases of language may find it fruitful to consider the processing requirements of linguistic devices as an important aspect of functional specialization. It will be important to test the dissociations we have reported here in other languages (especially spoken ones), either within another language that allows the same flexibility in conveying grammatical dependency information as in ASL or within bilingual subjects whose two languages use inflectional morphology or word order.

Methods

Please see SI Methods for full details of the methods used.

Participants.

Data from 14 right-handed, congenitally deaf young adults who learned ASL from birth from their deaf parents or caregivers were included in the analyses presented here. In all cases, deafness [90+ dB hearing loss (HL) in each ear] was of peripheral etiology and there was no other known neurological or psychological disease; participants did not report taking any neurotropic or psychotropic drugs. All gave informed consent and were free to terminate participation at any time.

Materials.

A set of 24 ASL sentences, each produced by a native ASL signer (T. Supalla), was recorded in three versions, two of which are discussed here: One contained no grammatical morphology, using word order to convey grammatical information, with all signs produced in the center of signing space and with minimal facial expression; the other used inflectional morphology on verbs and/or adverbs to mark agreement, aspect, or number. The same lexical items were used in both versions of each sentence, except where alterations were required by the condition (e.g., bound morphemes replacing free lexical items). Both sentence types were signed in a natural, colloquial form of ASL. Control stimuli were produced by digitally overlaying clips of three different sentences played backward. The resulting movies had the appearance of three semitransparent versions of the signer moving six arms and three faces simultaneously. Fig. S1 and Movie S1 show examples of the stimuli.

Procedure.

In the MRI scanner subjects performed four scanning runs, each comprising six blocks of sentences (two of each type) and six blocks of backward-layered movies. Each block was separated from the next block by a 15-sec baseline period during which a still image of the signer was displayed. Participants were instructed to press a button whenever they detected a sign from a particular semantic category (in sentences) or whenever they detected a left and a right hand with the same handshape and position (during control blocks).

Data Analysis.

fMRI data processing and analysis was carried out using the FMRI Expert Analysis Tool (FEAT) v. 5.98, part of FMRIB's Software Library (FSL) (www.fmrib.ox.ac.uk/fsl). Each individual fMRI run was analyzed using general linear modeling (GLM) (58 –60). At the second level of the analysis, the coefficients of the [sentence minus backward-layered control] contrasts from each run were entered into a GLM to identify subject-level effects, using a fixed-effects model in FMRIB's Local Analysis of Mixed Effects (FLAME) (61–63). The resulting coefficients were entered into a group-level mixed-effects analysis with subjects treated as a random effect, using FLAME stage 1 and stage 2. Z (Gaussianized T/F) statistic images were thresholded using clusters determined by Z > 2.3 and a (corrected) cluster significance threshold of P ≤ 0.05 (64). Before thresholding, the Z-statistic maps were masked to include only voxels showing positive signal values for the ASL sentence condition (i.e., relative to the low-level baseline of the still image of the signer). To compare brain areas activated more by one ASL sentence type than the other (with the activations for each sentence type's corresponding backward-layered control condition subtracted out), statistical maps were masked before thresholding such that only voxels that showed significantly greater activation for an ASL sentence type vs. its backward-layered control are included. Z maps were thresholded at Z > 1.6 (uncorrected), with a minimum cluster size of 0.2 mL (25 voxels).