Neural systems for sign language production: Mechanisms supporting lexical selection, phonological encoding, and articulation

Abstract

Overt production of ASL signs was evaluated using H₂ ¹⁵O PET to differentiate brain systems that support sign language production at the lexical‐selection and phonological‐articulatory levels. Subjects were 16 right‐handed, congenitally deaf native ASL signers (10 women, six men; age 20 to 29 years). Scans were performed while subjects (1) passively viewed ASL nouns, (2) repeated nouns, (3) generated verbs in response to these nouns, (4) passively viewed videotaped segments depicting transitive actions, and (5) generated a verb to describe these actions. Conjunctions between the two verb‐generation tasks revealed left‐lateralized activation of perisylvian, frontal, and subcortical regions commonly observed in spoken language generation tasks and implicated in processes of semantic feature binding and lexical selection. Analysis of noun repetition minus viewing condition revealed activation of distinct systems supporting phonological encoding and articulation, including bilateral activation of sensorimotor areas and association cortices in the temporal, parietal, and occipital lobes. In addition, lexical‐selection and articulatory processes were associated with activation of different corticostriatal‐thalamocortical circuits: articulation with activation of the motor, and lexical‐selection with activation of the prefrontal circuits, respectively. The results collectively provide insight into dissociable neural systems underlying these psycholinguistic functions. In addition, activation of regions that are typically associated with the auditory system during sign production suggests that these regions may support modality‐independent linguistic processes, or may indicate cross‐modal plasticity within the deaf brain. Hum Brain Mapp 23:156–167, 2004. Published 2004 Wiley‐Liss, Inc.

INTRODUCTION

Signed languages of the deaf are naturally occurring human languages. It has been well established that, despite the differences in form, sign languages possess the same formal linguistic properties that are found in spoken language. Neurolinguistic investigation of sign language structure has provided unique insights into the essential neural systems that underlie human communication: comparing and contrasting languages expressed in different modalities (i.e., oral–aural, manual–visual) has made it possible to explore and distinguish those properties shared by all human languages from those that arise in response to the modality in which the language is expressed.

Neuropsychological case studies of deaf signers who have suffered brain damage convincingly demonstrate that, as is the case with spoken language, sign language aphasia is found after left hemisphere perisylvian damage [Poizner et al., 1990]. Moreover, within the left hemisphere nonfluent and fluent aphasias follow a well‐established anterior versus posterior dichotomy. Frontal perisylvian damage leads to Broca‐like sign aphasia. In these cases, normal fluent signing is reduced to effortful, single‐sign, “telegraphic” output with little morphological complexity. Comprehension is left largely intact. Wernicke‐like sign aphasia, following damage to the posterior third of the perisylvian region, presents with fluent but often semantically opaque output, and difficulties in comprehension. These investigations serve to underscore the importance of left hemisphere structures for the mediation of signed languages and illustrate that sign language breakdown is not haphazard, but rather honors linguistic boundaries [Corina and McBurney, 2001].

The neuropsychological studies have been useful in sketching broad outlines of the cerebral regions important in sign language processing. More recently, a small but growing number of neuroimaging studies have begun to provide a more detailed picture of the neural architecture mediating sign language processing in non‐brain–damaged adults. These studies have consistently revealed that classic left‐hemisphere perisylvian language areas, similar to those implicated in spoken language, mediate processing of sign language in profoundly deaf, life‐long signers [Bavelier et al., 1998; Emmorey et al., 2002, 2003; MacSweeney et al., 2002; McGuire et al., 1997; Neville et al., 1998; Petitto et al., 2000]. In addition, some studies of sign language comprehension have shown greater right hemisphere contributions during sign language processing that are not evident during processing of written language [Bavelier et al., 1998; Neville et al., 1998; Newman et al., 2002]. In contrast, positron emission tomography studies of sign language production have typically observed largely left‐lateralized brain activation. For example, studies of single sign production in British Sign Language [McGuire et al., 1997], Langue des Signes Quebecoise [Petitto, et al., 2000], and ASL [Corina et al., 2003; Emmorey et al., 2003] reported that deaf subjects activated left inferior frontal regions and superior temporal regions similar to those that mediate speech in hearing subjects.1

As functional imaging studies continue to explore the neural correlates of sign processing, the questions being addressed have begun to broaden. Initial studies, using single contrast designs in the context of between group manipulation (e.g., deaf vs. hearing) aimed at illuminating how sign language activation differs from spoken or written language processing [MacSweeny et al., 2002; Neville et al., 1998]. Others studies have explicated changes in a small set of regions, selected on the basis of spoken language processing [Emmorey et al., 2002, 2003; McGuire et al., 1997; Petitto et al., 2000]. More recently, studies have begun to explore properties that are unique to sign languages [Corina et al., 2003; Emmorey et al., 2003; MacSweeney et al., 2002].

Studies of sign language production have differed considerably in methodology. McGuire et al. [1997] contrasted a covert signing sentence task cued by the presence of a videotaped signer producing an adjective to the passive viewing of that signer. Petitto et al. [2000] constructed contrasts between various tasks, including passive viewing of fixation cross, passive watching of non‐signs, passive viewing of real signs; repetition of signs; and verb generation. Emmorey et al. [2003] examined naming faces and animals including both sign and finger spelling, compared to a baseline of a signed yes–no judgment of an inverted face. Emmorey et al. [2002] examined production of a special class of ASL sign classifier that may not have a direct analogue in spoken English. These studies have largely limited the discussion of cortical activation to a predesignated cortical volume. While there is some convergence in patterns of activation seen across these studies, especially in left inferior frontal activation region, more work is needed to carefully document the larger range of cortical and subcortical systems involved and to further establish patterns of cognitive activation during lexical selection that can be distinguished from the task‐specific factors.

The present report seeks to illuminate the neural systems involved in sign language generation with an aim to differentiate processing stages related to lexical selection, phonological encoding and articulation. The present study makes use of two different verb‐generation tasks that are coupled with tests of sign repetition and passive sign viewing. Contrasting these tasks allows us to isolate the neural correlates of psycholinguistically motivated levels of processing.

Psycholinguistic theories of lexical production make an important distinction between those processes involved in the conceptualization and selection of individual lexical items, from those associated with the articulatory output. In Levelt's well‐cited model of word production [Levelt et al., 1999; Levelt, 2001], the intention to produce a given word activates conceptual–semantic information, which in turn activates lexical–semantic representations, resulting in the activation of a word that is consistent with the desired concept. In addition, this lexical semantic representation (or lemma) is thought to encode not only word‐meaning, but associated grammatical information, including part of speech, morphological case, and argument structure.

After selection of a particular lemma, information regarding the form‐base properties of the associated lexical item become available. This is the stage of phonological encoding, selection of the sound form of a word (phonemes, syllable structure, prosodic information) or formational patterns (hand shape, movement, position) of a sign. These, in turn, feed articulatory output processes that specify the articulatory–motor gestures associated with the desired word or sign, providing an acoustic or manual instantiation of the desired concept. The present study sought to differentiate neural systems that support these two distinct stages during lexical sign production. To identify systems involved in lexical semantic processing, we utilize two distinct ASL verb‐generation tasks. The conjunctions, or common features of these tasks, highlight essential regions involved in lexical selection. This analysis allows us to replicate and extend previous findings from studies of spoken and signed languages. Moreover, the use of two verb‐naming tasks provides an opportunity to identify the processes involved in verb generation that are largely independent of the particular constraints of the task. In addition, we wished to demarcate the contributions of the neural systems active during the phonological encoding and complex articulatory movements that underlie single sign production, independent of the requirements of lexical selection. A sign repetition condition provided insights to this question. Taken together, our studies permit the cataloguing of the neural regions active under both lexical‐selection and motor‐articulatory conditions. Particular attention is paid to characterizing not only perisylvian systems but additional cortical and subcortical regions that play a role in sign production in ASL.

SUBJECTS AND METHODS

Sixteen right‐handed, congenitally deaf signers (10 women, 6 men; age 20–29 years of age) were studied. All subjects were children of deaf parents and all were fluent in ASL, having acquired it as a native language around the age of two. Subjects learned English as a second language during the course of primary or secondary education. All participants used standard unmodified ASL (no educational or instructional sign systems were used). Physical examination and medical history were in each case within normal limits except for profound bilateral hearing loss. Subjects were not using neuroactive medications at the time of the study. These studies were conducted under a protocol approved by the NIDCD‐NINDS IRB (NIH 92‐DC‐0178). The risks and procedures of the study were explained to each participant and written informed consent was obtained according to the declaration of Helsinki. Subjects were compensated for participating.

PET Scanning Methods

PET scans were performed on a GE‐Advance 3‐D PET camera (GE Medical Systems, Waukesha, WI), which has an axial and in‐plane resolution of 4.25 mm FWHM; 35 contiguous planes were acquired simultaneously. An initial transmission scan using a rotating Ga/Ge pin source was acquired for attenuation correction. An intravenous line was placed in the forearm so as not to impede movement of the arms, wrists, or obstruct the subjects' use of signing space. A thermoplastic mask was applied to each subject to maintain head position throughout the scanning session. Five injections of 10 mCi of H₂ ¹⁵O were administered intravenously. PET scans were initiated by the arrival of the H₂ ¹⁵O bolus in the brain (approximately 20 s after injection) and continued for 1 min. Scans were separated by intervals of 5 min. Background scans were performed 1 min prior to each H₂ ¹⁵O injection.

Stimuli and PET tasks

Subjects were presented with videotaped recordings of a male deaf actor producing ASL nouns or engaged in a series of transitive actions, and performed a different task in each of the five scan periods. Forty‐one ASL signs for common nouns were executed bimanually; all were non‐mimetic (non‐iconic). Mean duration of signs was 20.5 frames (range 16–26; 30 frames/s). Thirty‐one transitive action sequences were used. Mean duration of these segments was 54.4 frames (range 34–59; 30 frames/s). The Appendix lists the signs and action sequences used. Both nouns and action segments were presented with a SOA of 3 s. The mean ISI for signs was 2,340 ms and for actions 1,218 ms. A uniform blue background was presented between stimuli. Videotapes were played on a PV8451 Panasonic video cassette player and displayed on a CT‐206 13W Panasonic color TV monitor located in front of the PET scanner gantry. Subject's responses were recorded for analysis using a Panasonic Video Camera and an AG DS550 Panasonic video Cassette Recorder. Stimuli presentation, subject's responses, scanning time, and stimuli condition information were recorded simultaneously on the same screen, using a WJ‐420 Panasonic Quad System and a JVC monitor. The scanning time and task condition information were noted using a K‐20 ND Keyboard connected to the video recording system. At the end of each scan, participants were asked a series of yes/no recognition questions to determine if they had been attending to the tasks. PET tasks began 30 s prior to injection of the radiotracer and continued throughout the scanning period. Scans were acquired while subjects were engaged in the following task conditions: (1a) passively viewing ASL nouns; (1b) repeating each noun during the interstimulus intervals; (1c) generating a verb selected to match each noun during the interstimulus intervals; (2a) passively viewing transitive action segments; and (2b) generating a verb to describe each action during the interstimulus intervals. The order of tasks was randomized and counterbalanced. All signs were produced bimanually.

PET data analysis

PET scans were registered and stereotaxically normalized into Talairach coordinate space [Talairach and Tournoux, 1998] using Statistical Parametric Mapping software (Wellcome Department of Cognitive Neurology, London, UK). Individual pairwise contrasts (e.g., verb generation‐noun repetition) were evaluated using the t‐statistic calculated for all voxels; the resulting sets of values were transformed to Z‐scores. PET data were further analyzed using a factorial design in which common activations are evaluated. A modification of the procedure described by Price and Friston [1997] was used and is the same procedure used in Braun et al. [2001]. Conjunctions are, in this case, defined in a Boolean sense as common areas of activation (Z > 3) in a set of task pairs [e.g., (verb generation–noun repetition) + (action naming–transitive action viewing)]. Significant differences between the individual pairwise contrasts [e.g., (verb generation–noun repetition)–(action naming–transitive action viewing)] are then eliminated from the conjunction map (for this purpose, significant differences were defined conservatively as voxels in which Z > 2). The resultant conjunction map is further masked so that only voxels in which significant activations were detected in both of the individual pairwise contrasts (Z > 3) are retained. In using this approach, the main effect actually underestimates the significance of results replicated in two independent contrasts. Therefore, the conjunctions we report should be interpreted as simply depicting, in a more conservative, Boolean sense, shared activations (verb generation minus their respective baselines) that do not significantly differ in magnitude.

RESULTS

Behavior

Behavioral data obtained during performance of the PET tasks indicate that all subjects repeated nouns, generated verbs, and named actions accurately. Following completion of scanning, all subjects scored 80% or better on the recognition tests.

Noun Repetition versus Viewing Signed Nouns

The noun repetition versus viewing contrast (repetition minus viewing; Fig. 1, Table I) was associated with bilateral activation of frontal motor areas including the supplementary motor area (SMA proper) and precentral gyri. Significant increases in activity were observed bilaterally in the dorsal posterior portions of the anterior cingulate cortex (BA24). In post‐Rolandic areas, bilateral activation of parietal regions, including the postcentral gyrus, paracentral lobule (BA 5,7) and the supramarginal gyrus (SMG, BA 40), were observed. Subjects also displayed bilateral activation of extrastriate cortices at the temporo‐occipital junction (BA 19, left greater than right) and left‐lateralized activation of the superior temporal gyrus (BA 22), within the planum temporale. Subcortically, increases in rCBF were detected in the cerebellar hemispheres and vermis and the ventrolateral thalamus bilaterally, and in the left posterior putamen and left midbrain tegmentum.

Figure 1.

Brain map illustrating increases in regional cerebral blood flow (rCBF) during ASL noun repetition (vs. passive viewing of signed nouns). Statistical parametric maps are displayed on a standardized MRI scan, which was transformed linearly into stereotaxic (Talairach) space. Scans are displayed using neurological convention (left hemisphere is represented on the left). Planes of section relative to the bi‐commissural line are indicated. Values are Z‐scores representing the significance of voxel‐wise increases in normalized rCBF. The range of scores is coded in the accompanying color table (right).

Table I.

Regional cerebral blood flow responses in deaf subjects repeating a series of signed nouns

Region	Brod. no.	Left hemisphere				Right hemisphere
		Z‐Score	x	y	z	Z‐Score	x	y	z
Cortical
Frontal motor
SMA proper	6	3.86	−16	−10	52	3.62	2	−8	48
Precentral gyrus	4	5.03	−30	−24	52	5.31	28	−26	52
Cingulate
Posterior dorsal ACC	24	4.06	−4	−6	40	4.02	8	−8	40
Parietal
Postcentral gyrus	3,1,2	3.78	−26	−34	48	4.62	20	−34	48
Paracentral lobule	5,7	3.58	−16	−28	52	3.66	16	−32	52
Supramarginal gyrus	40	3.82	−34	−34	52	3.70	34	−34	52
Temporal
Posterior STG ‐ Planum temporale	22	3.70	−42	−36	20	—	—	—	—
Occipital
Lateral occipital cortex		2.97	−52	−64	−8	2.57	46	−68	−8
Lateral occipital ‐ MT	19	2.97	−48	−76	8	—	—	—	—
Subcortical
Cerebellum
Cerebellar hemisphere		5.03	−22	−48	−20	4.62	22	−64	−20
Midline cerebellum		4.87	−8	−54	−16	4.62	14	−56	−16
Basal Ganglia
Posterior putamen		2.89	−26	−18	8	—	—	—	—
Thalamus
Ventrolateral thalamus		3.45	−18	−22	4	3.05	12	−18	8
Brainstem
Midbrain tegmentum		3.05	−12	−26	−4	—	—	—	—

Regions in which normalized rCBF is greater during repetition than passive viewing of nouns are tabulated along with Z‐scores (representing local maxima) and associated Talairach coordinates.

Generating Verbs versus Repeating Nouns

When the verb generation and noun repetition tasks were contrasted (generation minus repetition; Fig. 2, Table II), we observed strongly left‐lateralized activation of dorsal (BA 44/45) and ventral (BA 47) portions of the frontal operculum and left anterior insula, as well as left‐lateralized activation of both the inferior (BA 46) and superior (BA 9) portions of the dorsolateral prefrontal cortex. The verb generation minus repetition task was also associated with left‐lateralized activation of posterior parietal cortex at the parieto‐occipital junction, including the superior parietal lobule (BA 7) and angular and supramarginal gyri (BA 39, 40), extending into the dorsal portions of BA 19. In the temporal lobe, the left fusiform gyrus (BA 37) was activated, as were both anterior and posterior portions of the left middle temporal gyrus (BA21); in the latter case, activation extended into the superior temporal sulcus. In the occipital lobe, the verb‐generation task was associated with activation of the inferior temporal (BA 19/37) and lingular gyri (BA 19), the latter extending into the cuneus (BA 18/19). Frontal motor regions activated during verb generation minus repetition included the anterior portions of the SMA (pre‐SMA) bilaterally, and the left lateral premotor cortex. Activations were also detected in the anterior dorsal (BA 32/24) and ventral (BA 32) portions of the left anterior cingulate cortex. Subcortically, verb generation was associated with significant increases in rCBF in the right posterolateral cerebellar hemisphere and midline cerebellum, as well as the left caudate nucleus and in the dorsomedial thalamus and midbrain PAG bilaterally.

Figure 2.

Brain map illustrating increases in regional cerebral blood flow (rCBF) during ASL verb generation (vs. repetition of signed nouns). Statistical parametric maps are displayed on a standardized MRI scan as outlined in the legend to Figure 1.

Table II.

Regional cerebral blood flow responses in deaf subjects generating verbs to match a series of signed nouns

Region	Brod. no.	Left hemisphere				Right hemisphere
		Z‐Score	x	y	z	Z‐Score	x	y	z
Cortical
Frontal operculum
Dorsal operculum	44/45	5.04	−50	20	20	—	—	—	—
Ventral operculum	47	4.24	−44	26	−8	—	—	—	—
Insular
Anterior Insula	—	5.12	−38	14	4	—	—	—	—
Prefrontal
Superior DLPFC	9	5.70	−36	6	36	—	—	—	—
Inferior DLPFC	46	4.07	−38	52	16	—	—	—	—
Medial PFC	9	3.54	−2	60	28	—	—	—	—
Parietal
Angular gyrus	39	3.18	−32	−58	36	—	—	—	—
Temporal
Posterior STG/STS	22	3.23	−30	−52	20	—	—	—	—
Middle temporal gyrus	21	3.23	−52	−42	−4	—	—	—	—
Fusiform gyrus	37	3.01	−38	−42	−16	—	—	—	—
Occipital
Lingular gyrus	19	3.18	−12	−66	−4	—	—	—	—
Lateral occipital cortex	19/18	3.54	−24	−94	16	—	—	—	—
Frontal motor
Pre‐SMA	6	4.02	−4	10	48	3.45	2	12	48
Lateral premotor cortex	6	5.08	−46	4	44	—	—	—	—
Cingulate
Ventral ACC	24/32	4.29	−12	22	24	3.40	2	24	24
Anterior dorsal ACC	32	4.64	−8	12	40	4.11	2	14	40
Subcortical
Cerebellum
Cerebellar hemisphere	—	—	—	—	—	5.65	22	−82	−24
Midline cerebellum	—	3.89	−2	−42	−20	3.89	4	−48	−20
Basal ganglia
Caudate	—	3.89	−18	16	16	—	—	—	—
Thalamus
Dorsomedial thalamus	—	3.49	−8	−20	4	3.18	8	−10	4
Brainstem
Midbrain periacqueductal grey	—	3.18	−2	−28	−16	3.54	6	−30	−12

Regions in which normalized rCBF is greater during verb generation than noun repetition are tabulated along with Z‐scores (representing local maxima) and associated Talairach coordinates.

Common Activations: Verb Generation–Repetition and Action Naming

Common patterns of activation for the verb generation contrasts (generation minus noun repetition and naming minus viewing transitive actions) were identified (Fig. 3, Table III) in the dorsal and ventral frontal operculum (BA 44/45, 47), in the anterior insula, and both inferior and superior (BA 46, 9) dorsolateral prefrontal cortices, lateralized to the left hemisphere. Conjunctions were also seen in the left posterior parietal cortex at the parieto‐occipital junction (including both superior and inferior parietal lobules, BA 7, 39, 40), and in the left anterior middle temporal (BA 21), fusiform (BA 37), and lingual (BA 19) gyri. Common activations were also observed in the left lateral premotor cortex and anterior SMA (BA 6, pre‐SMA), and in the rostral portions of the dorsal and ventral anterior cingulate cortices (BA 32). Subcortically, conjunctions included the right posterior and midline cerebellum, the left caudate nucleus, as well as the midbrain PAG and dorsomedial thalamus in both right and left hemispheres.

Figure 3.

Brain maps illustrating conjunctions between activations associated with two verb‐generation tasks: generating verbs to match signed nouns and naming transitive actions versus their respective baselines. Statistical parametric maps are displayed on a standardized MRI scan as outlined in the legend to Figure 1.

Table III.

Conjunctions between rCBF responses in two verb production tasks (generating verbs to match signed nouns and naming transitive actions) versus their respective baselines

Region	Brod. no.	Left hemisphere				Right hemisphere
		Z‐score	x	y	z	Z‐score	x	y	z
Cortical
Frontal operculum
Dorsal operculum	44/45	5.15	−44	18	16	—	—	—	—
Ventral operculum	47	4.82	−38	24	−4	—	—	—	—
Insular
Anterior Insula	—	5.65	−36	16	4	—	—	—	—
Prefrontal
Superior DLPFC	9	5.76	−40	2	40	—	—	—	—
Inferior DLPFC	46	5.02	−36	28	24	—	—	—	—
Parietal
Angular gyrus	39	4.29	−30	−60	32	—	—	—	—
Temporal
Middle temporal gyrus	21	3.52	−56	−44	−4	—	—	—	—
Fusiform gyrus	37	3.55	−36	−48	−16	—	—	—	—
Occipital
Lingular gyrus	19	4.05	−12	−66	−4	—	—	—	—
Frontal motor
Pre‐SMA	6	5.32	−2	10	48	5.21	2	12	48
Lateral premotor cortex	6	6.18	−36	0	44	—	—	—	—
Cingulate
Anterior dorsal ACC	32	5.46	−2	14	40	5.13	2	14	40
Subcortical
Cerebellum
Cerebellar hemisphere	—	—	—	—	—	3.63	24	−84	−24
Midline cerebellum	—	4.96	−2	−70	−20	6.34	6	−76	−20
Basal ganglia
Caudate	—	3.33	−12	4	8	—	—	—	—
Thalamus
Dorsomedial thalamus	—	3.99	−10	−22	8	4.05	10	−16	8
Brainstem
Midbrain periacqueductal grey	—	3.80	−2	−40	−16	4.05	2	−40	16

Regions in which conjunctions are significant (see Subjects and Methods) are tabulated along with Z‐scores (representing local maxima) and associated Talairach coordinates.

Differences Between Verb Generation–Repetition and Action Naming

Contrasts between task pairs [(action naming–viewing)–(verb generation–noun repetition) and (verb generation–noun repetition)–(action naming–viewing)] revealed left lateralized activations unique to action naming in primary motor (Z = 3.31; x = −28, y = −20, z = 52), and primary somatosensory cortices (4.41; −28, −28, 44), supramarginal gyrus (3.62; −30, −34, 44), planum temporale (3.24; −42, −38, 20), and in the dorsal portions of the superior parietal lobule (2.33; −4, −56, 52). Activations unique to action naming were also detected in the cerebellar hemispheres, anterior and superior to the common area of activation (on the right: Z = 2.45; x = 18, y = −52, z = −16; on the left: 4.07; −20, −48, −16) and in the putamen bilaterally (right: 2.26; −24, −8, 4; left: 3.43; −30, −22, 4). Activations unique to the verb generation (minus noun repetition) contrast were detected in the medial prefrontal cortex (2.76; −2, 60, 28) and lateral occipital cortex (2.57; −28, −82, 20).

DISCUSSION

Lexical Selection

The principal aim of this study was to differentiate the brain systems that function at the level lexical selection from those that support phonological encoding and complex articulation during sign language production. We used two distinct but complimentary verb‐generation tasks to identify neural systems involved in lexical selection. This technique has the advantage of minimizing the expected overlap, in order to more clearly delineate the essential systems that underlie verb generation. Common activations were found in cortical regions including left perisylvian (frontal opercular, temporoparietal), prefrontal, mesial temporal, and premotor cortices. Conjunctions also encompassed proisocortical areas (insular and cingulate cortices) as well as subcortical regions including thalamus, basal ganglia, and the right posterior cerebellum.2

Our results confirm that lexical selection during overt production of signs, at least at the level of single words, appears to be associated with robust left‐lateralization of cerebral activity. This pattern is consistent with that reported for production of spoken words [Klein et al., 1999; Petersen and Fiez, 1993; Raichle et al., 1994]. The present study is also consistent with neuroimaging studies of both covert [McGuire, et al., 1997] and overt [Petitto, et al., 2000] sign production. Moreover, a complementary study [Corina et al., 2003] has shown that this pattern of lateralization in the generation versus repeat paradigm is independent of the hands used to sign.

Prefrontal and Associated Subcortical Areas

The verb generation tasks were associated with activation of the entire left frontal operculum, including Brodman's areas 44, 45, and 47. In studies of spoken English, Petersen and coworkers [Petersen et al., 1989; Petersen and Fiez, 1993] similarly reported activation of this region for verb generation, but not for noun repetition. Activation of the operculum has been reported in both covert [McGuire et al., 1997] and overt [Petitto et al., 2000] sign production tasks. Supporting this, cortical stimulation of 44/45 in a deaf signer was associated with disordered motor output of signing [Corina et al., 1999].

The dorsolateral prefrontal cortices, both inferior (BA 46) and superior (BA 9) portions, in each case lateralized to the left hemisphere, were also activated only during the verb‐generation tasks. These findings are consistent with previous studies suggesting that the inferior DLPFC may play a role in lexical selection [Lurito et al., 2000; Petersen et al., 1989]. While much of the neuroimaging literature has focused on the inferior prefrontal regions, the superior portions of the DLPFC, close to the regions we see activated here, have been associated with performance of lexical tasks as well [Phelps et al., 1997].

Left lateralized activation of the caudate, dorsal thalamus, dorsolateral prefrontal cortex, and operculum during both verb‐generation tasks may represent selective activation of the prefrontal corticostriatal‐thalamocortical circuit [Alexander et al., 1986]. Unlike the motor circuit (putamen, ventral thalamus, and SMA proper), which was activated bilaterally during noun repetition, the prefrontal circuit appears to play a role in the timing and sequencing of cognitive, rather than motor, behaviors. Its activation would be consistent with the idea that verb generation engages regions that organize sequential cognitive processes that underlie semantic association and consequent lexical selection in the context of language production.

Post‐Rolandic Association Cortices

The left middle temporal gyrus, activated only during the verb‐generation tasks, has been shown to play a role in semantic processing [Vandenberghe et al., 1996; Wiggs et al., 1999]. Indeed, activation of this region has been detected in almost all spoken language verb‐generation tasks [Martin and Chao, 2001]. In our subjects verb generation was also associated with activation of the posterior superior temporal gyrus extending into the superior temporal sulcus (STS) and the posterior parietal cortex at the parieto‐occipital junction. All of these posterior perisylvian regions have historically been considered constituents of Wernicke's area and are likely to play a role in early lexical access, i.e., in the translation of concepts into signs at the level of the mental lexicon.

Emmorey et al. [2003] examined lexical–semantic retrieval in sign by contrasting naming of faces and animals. In this case, activation of temporal lobe differed with respect to semantic content of the items to be named, with proper naming eliciting more temporal pole and anterior STS activation while naming of animals under some conditions produced activation in temporal‐occipital junctions. Taken together with the present study, these results argue for a distributed representation of lexical semantics within the left temporal lobe in signers that accords well with patterns observed in users of spoken languages.

Extrastriate areas activated during the verb‐generation tasks did not include those that process visual motion (regions that, as outlined below, were active during noun repetition) but instead encompassed basal temporal areas, fusiform and lingual gyri, that have been shown to be involved in attribution of semantic features to exteroceptive stimuli in reading or naming tasks [Buchel et al., 1998; Chao et al., 1999; Moore and Price, 1999]. In this instance, these regions may be activated in top‐down fashion as subjects process semantic information during the early stages of lexical selection. These regions have been shown to be active during sign‐naming tasks for persons and animals, which is suggested to be related to configural analysis of facial information [Emmorey et al., 2003]. In the present study we observe that this region is active across both verb‐generation tasks, even though facial information is not prominent in the transitive actions task.

Motor‐Related Areas

Since the verb generation contrasts appear to highlight regions that are more closely affiliated with lexical–semantic processes, activation of prefrontal, subcortical, and post‐Rolandic heteromodal cortices was not surprising. On the other hand, this contrast was also associated with activation of cortical and subcortical structures that are more typically associated with the motor system. Activation of the posterior portions of the right cerebellar hemisphere was not unexpected. Activation of this region has been consistently observed in previous studies of spoken word production [Petersen and Fiez, 1993], supporting the notion that the cerebellum plays a role in a number of cognitive processes [Ackermann et al., 1998; Cabeza and Nyberg, 2000; Leiner et al., 1993].

Cerebellar activation has been observed in studies of sign production. Petitto et al.'s [2000] data indicate right cerebellar activation during a verb generation versus imitation task. In a study investigating the roles of performing a verb‐generation task with either the left or right hand during signing, Corina et al. [2003] report consistent right cerebellar activation despite the differential motoric demands. Noppeny and Price [2002] have recently suggested that the right lateral cerebellum is part of the network involved in semantic‐executive systems required for the effortful and strategic retrieval of semantic information. As activation is preserved across differing response modes, speech articulation, key‐presses, and now signing, it becomes clear that this activation does not reflect speech‐specific processes.

Both of the verb‐generation tasks were also associated with activation of the pre‐SMA—a region cytoarchtectonically identified as BA 6, anterior to the SMA proper [Zilles et al., 1995]—that may play a role in more cognitive aspects of motor behaviors [Fried et al., 1991], and is a central element of the corticostriatal‐thalamocortical circuit outlined above.

In addition, the left lateral premotor cortex (LPM) was selectively activated during the verb‐generation tasks. Activation of the lateral premotor cortex has been reported in a number of higher level language tasks [Martin et al., 1996; Warburton et al., 1996]. The lateral premotor system may include the insular cortices [Picard and Strick, 2001], which were selectively activated during verb generation as well. It is possible that both lateral and medial premotor systems may be involved in coupling the more abstract lexical representation of an utterance to its motor representation in both spoken and signed language.

The verb‐generation tasks were also associated with activation of the ACC (BA 32, anterior to the region activated during noun repetition). This portion of the ACC is frequently activated in conjunction with the prefrontal cortex [Picard and Strick, 2001] and may play a role in higher order stages of response selection, coupling cognition, and motor control.

Interestingly, anterior cingulate activations for the verb‐generation tasks in each case extended into the anteroventral portions of BA32, which have been shown to be active during performance of stimulus‐response tasks requiring speech utterances as opposed to limb movements [Paus et al., 1993], which represents a potential instance of plasticity within the deaf brain.

Phonological Encoding and Articulation

The results of the sign repetition versus sign viewing contrast permit us to differentiate brain systems associated with articulatory and phonological encoding. Our findings reveal expected activations in primary and sensorimotor corticies but also reveal subcortical contributions that differ systematically from those observed in lexical selection. In addition, activation of certain sites within post‐rolandic association cortices may reflect neuroplastic changes within these regions in deaf subjects.

Rolandic, Frontal Motor, and Associated Subcortical Areas

Activation of somatomotor and sensory areas during phonological encoding and articulation is not unexpected. In keeping with this, the noun repetition task (minus passive viewing of nouns) was associated with activation of the primary motor and somatosensory cortices themselves, regions that constitute the final common pathway for motor control of and proprioceptive or tactile feedback from the articulators. Similarly, noun repetition was associated with bilateral activation of the lateral cerebellar hemispheres (anterior and lateral to the right posterior area activated during verb generation)—consistent with bilateral representation of motor input to and sensory feedback from the articulators in these regions—and with activation of the vermis, which appears to play a role in the execution of complex movements of the distal upper extremities [Sadato et al., 1996].

In the basal ganglia and thalamus, the noun repetition and verb‐generation tasks appear to map different corticostriatal‐thalamocortical circuits [Alexander et al., 1986]. Bilateral activation of the putamen, in association with the ventral thalamus and SMA during the noun repetition‐view contrast may constitute selective activation of the motor circuit, which might be expected to support the timing and sequencing of primary articulatory–motor processes underlying sign production.

The neocortical element of this circuit, the SMA proper, appears to organize complex sequences of oral articulatory movements [Fiez, 2001] and is associated with primary articulatory movements of the limbs. Articulation was associated with activation of the posterodorsal ACC (BA 24) as well, portions of which comprise the cingulate motor area and appear to regulate and regulate movements of the limbs [Dum and Strick, 1991].

Post‐Rolandic Association Cortices

Unexpectedly, the noun repetition‐view contrast resulted in activation of post‐Rolandic visual and auditory cortices. For example, we observed activation of the lateral occipital cortices in both left and right hemispheres during noun repetition. Since subjects' eyes were unoccluded both during baseline (noun viewing) and repetition conditions, this activation must represent an increase in visual cortical activity specifically associated with sign language articulation, rather than visual processing per se. The portions of the occipital cortex that were activated in this instance correspond to area MT/V5, a region associated with visual motion processing [Eden et al., 1996; Watson et al., 1993]. It is conceivable that these regions may be activated by visual self‐monitoring during sign production. While visual motion processing must certainly occur during the baseline sign perception task, monitoring one's own output in intrapersonal (signing) space may place a unique demand upon resources in these extrastriate regions.

The noun repetition task was also associated with activation of the posterior superior temporal gyrus (PSTG), specifically within the region of the planum temporale (PT). No additional activation of this region was detected in the generation minus repeat contrast, suggesting that, in this cohort of deaf subjects, the PT may principally be playing a role in articulatory or phonological processing.

This observation parallels the findings of Petitto et al. [2000], who reported that the PSTG (although not specifically the PT) was activated during the processing of both meaningful and nonsense signs. These authors interpreted this finding as evidence that the PSTG is specifically playing a role in phonological processing. They went on to argue that while this region is generally thought of as a unimodal auditory association area, it appears in this context not to be selectively related to the auditory modality, an idea that is supported by the present results. Beyond this, our observation suggests that the region is not specifically associated with comprehension, as reported by Petitto et al. [2000] but is active for sign production as well.

Previous studies have suggested that the PT or PSTG may play a role in phonological processing of spoken language [Indefrey and Levlet, 2000]. One way to interpret the fact that this region is also activated during what we consider to be a similar linguistic operation in deaf signers would be that the PT in fact plays a more central role in modality‐independent phonological processing of language per se.

On the other hand, if the PT is principally considered an auditory association area, activation during ASL production in the deaf may indeed be related to cross‐modal plasticity following auditory deprivation in these individuals, i.e., normally dedicated auditory cortex may be recruited for novel, non‐auditory functions within the deaf brain. The issue cannot be resolved by the present results.

An imaging study by MacSweeney et al. [2002] suggests that both possibilities may in fact be true: while both deaf and hearing signers activated BA42 and the PT during British Sign Language comprehension, activation was indeed significantly greater in the deaf.

CONCLUSION

This study provides evidence for the importance of left hemisphere involvement in sign language production. Production of ASL verbs during two cognitively distinct generation tasks is characterized by a robust left‐lateralized pattern of frontal, temporoparietal, and subcortical activity and prominent activation of the right cerebellar hemisphere.

The analysis of two complementary conditions permits further insights in to the neural system involved in sub‐processes of sign production, effectively differentiating the anatomy of lexical semantic activation and motor implementation of linguistic codes. Systems related to articulation and lexical selection revealed bilateral activation of sensorimotor areas (during noun repetition minus viewing) and left lateralized activation of heteromodal association areas and right cerebellar hemisphere (during verb generation minus repetition).

In addition, each contrast was associated with activation of a different corticostriatal‐thalamocortical circuit: repetition with activation of the motor and verb generation with activation of the prefrontal circuits, respectively.

This functional dichotomy was not absolute: verb‐generation tasks were additionally associated with activation of motor related areas and noun repetition with activation of association cortices. For example, both sets of contrasts were associated with activation of the temporal gyrus: those highlighting lexical selection with activation of anterior and posterior portions of the left middle temporal gyrus and those highlighting phonological encoding and articulation with left‐lateralized activation of the superior temporal gyrus within the planum temporale. Since the latter is generally considered an auditory association area, this provides some evidence for cross‐modal plasticity in the deaf brain.

APPENDIX 1.

ASL Nouns:

Army, Baby, Ball, Bedroom, Chair, Coffee, Drugs, Fight, Film, Floor, Friend, Group, Husband, Interest, Judge, Key, Knife, Letter, Money, Morning, Mountain, Party, People, Picture, Plant, Project, Race, Secretary, Snow, Store, Story, Street, Table, Tent, Train, Tree.

Action/Object Vignettes:

Bite apple, Blow fire, Break cookies, Brush hair, Brush teeth, Build blocks, Call TTY, Chew gum, Comb hair, Count money, Crush pop‐can, Fold shirt, Hide gun, Kick football, Kiss frog, Lick stamp, Pass basketball, Plug into socket, Pop balloon, Read book, Scan newspaper, Shave razor, Sip water, Sit chair, Smell flower, Spill water, Staple paper, Steal wallet, Sweep broom, Take a picture, Watch movie.