Cortical plasticity for visuospatial processing and object recognition in deaf and hearing signers

Abstract

Experience-dependent plasticity in deaf participants has been shown in a variety of studies focused on either the dorsal or ventral aspects of the visual system, but both systems have never been investigated in concert. Using functional magnetic resonance imaging (fMRI), we investigated functional plasticity for spatial processing (a dorsal visual pathway function) and for object processing (a ventral visual pathway function) concurrently, in the context of differing sensory (auditory deprivation) and language (use of a signed language) experience. During scanning, deaf native users of American Sign Language (ASL), hearing native ASL users, and hearing participants without ASL experience attended to either the spatial arrangement of frames containing objects or the identity of the objects themselves. These two tasks revealed the expected dorsal/ventral dichotomy for spatial versus object processing in all groups. In addition, the object identity matching task contained both face and house stimuli, allowing us to examine category-selectivity in the ventral pathway in all three participant groups. When contrasting the groups we found that deaf signers differed from the two hearing groups in dorsal pathway parietal regions involved in spatial cognition, suggesting sensory experience-driven plasticity. Group differences in the object processing system indicated that responses in the face-selective right lateral fusiform gyrus and anterior superior temporal cortex were sensitive to a combination of altered sensory and language experience, whereas responses in the amygdala were more closely tied to sensory experience. By selectively engaging the dorsal and ventral visual pathways within participants in groups with different sensory and language experiences, we have demonstrated that these experiences affect the function of both of these systems, and that certain changes are more closely tied to sensory experience, while others are driven by the combination of sensory and language experience.

1. Introduction

The brain’s functional architecture shows remarkable reorganization following altered sensory experiences. Owing to the importance of visual motion in signed languages, particularly motion of the hands to convey linguistic information, studies of congenitally deaf individuals who use American Sign Language (ASL) have often focused on changes in the dorsal visual stream associated with visual motion perception (Bavelier et al., 2001; Bosworth & Dobkins, 2002a, 2002b; Fine, Finney, Boynton, & Dobkins, 2005; Finney & Dobkins, 2001; Finney, Fine, & Dobkins, 2001; Neville & Lawson, 1987a). These studies sought to tease apart differences in visual motion perception attributable to experience with ASL from those due to altered early sensory experience (i.e. deafness) by including groups of hearing signers as well as hearing non-signers. Recent studies have shown that although activity in primary visual cortex is comparable across these groups (Bavelier et al., 2001; Fine, Finney, Boynton, & Dobkins, 2005) differences arise in dorsal brain regions associated with visual motion processing and attention (Bavelier et al., 2001). For example, both deaf and hearing participants who acquired ASL early in life showed greater MT/V5 activation in the left than in the right hemisphere, whereas hearing subjects without ASL experience showed the opposite pattern. This finding, together with earlier visual field and ERP studies (Bosworth & Dobkins, 1999; Brozinsky & Bavelier, 2004; Neville & Lawson, 1987a, 1987b), indicates that early acquisition of a visuospatial language can bias cortical lateralization for perception of visual motion (Neville et al., 1987b, Bavelier et al., 2001).

At the same time, other cortical changes seem to stem more from sensory experience than use of ASL. Bavelier and colleagues (Bavelier et al. 2001) reported that area MT/V5 was more active when deaf signers attended to peripherally than to centrally presented visual motion stimuli (see also Bavelier et al., 2000), whereas in hearing groups (both signing and non-signing) area MT/V5 showed a preference for stimuli presented to the central visual field. In addition, regardless of the attended location, posterior parietal and superior temporal cortices showed larger responses in deaf participants than in either hearing group. Such differences were interpreted as driven by sensory experience because hearing signers do not show this pattern of responses.

While these data provide compelling evidence of sensory and language experience-driven plasticity, recently there has been speculation as to whether such plasticity is constrained to the dorsal stream or manifests in the ventral visual pathway as well. Experience-dependent changes in ventral visual cortex would be predicted based on evidence that deafness and/or ASL usage impacts behavioral performance during visual object perception, particularly for faces. Faces serve a unique functional role in deaf communication as deaf individuals are disproportionately dependent on facial cues for decoding affective communicative nuances that are conveyed by prosody and tone of voice for hearing persons. In addition, while visually monitoring the periphery to attend to the hand motions and gestures that comprise ASL, signers (deaf and hearing) attend to faces foveally (De Filippo & Lansing, 2006; Siple, 1978) to distinguish not only affective, but also lexical, semantic, and syntactic information carried by facial expressions involving eyebrow, eye, and mouth movements. Such linguistic facial expressions are unique to sign languages and distinct from facial expressions that code emotional information (see Corina, Bellugi, & Reilly, 1999; Reilly & Bellugi, 1996 for reviews).

Several investigations indicate that deaf participants outperform hearing individuals who have no sign language experience on tests of face recognition and discrimination (Arnold & Murray, 1998; Bettger, Emmorey, McCullough, & Bellugi, 1997; McCullough & Emmorey, 1997; Bellugi et al., 1990) such as the Facial Recognition Test (Benton, Sivan, Hamsher, Varney, & Spreen, 1983). These enhancements appear to be specific to faces, rather than a general improvement in visual processing (Bettger, Emmorey, McCullough, & Bellugi, 1997). In most of these studies deaf and hearing signers showed comparable performance enhancements compared to hearing non-signers, leading to the conclusion that superior face processing abilities appeared to be more related to ASL experience than deafness (Arnold & Murray, 1998; Bettger, Emmorey, McCullough, & Bellugi, 1997; McCullough & Emmorey, 1997). That conclusion is supported by an absence of any face processing advantage in deaf individuals raised with oral communication, which involves speech reading rather than use of signs (Parasnis, Samar, Bettger, & Sathe, 1996; Bettger, Emmorey, McCullough, & Bellugi, 1997).

The brain basis for enhanced performance of face processing in deaf users of sign language is poorly understood. Visual field studies suggested a redistribution of neural resources for face processing in deaf individuals, with a shift toward increased involvement of the left hemisphere (Corina, 1989; Vargha-Khadem, 1983). Thus, the linguistic salience of faces may lead to altered laterality for face processing in deaf signers. Few studies to date have directly examined the physiology of ventral stream function in deaf individuals. One required monitoring colors (Armstrong, Neville, Hillyard, & Mitchell, 2002) and another shapes (Shibata, Kwok, Zhong, Shrier, & Numaguchi, 2001), and both of these reported no differences in brain response between deaf signing and hearing non-signing groups. However, fMRI investigations of face perception by McCullough and colleagues (McCullough et al., 2005; Emmorey & McCullough, 2009) demonstrated differences in cortical activity between hearing and deaf participants in face-selective ventral temporal cortices. In these studies, hearing non-signers demonstrated bilateral activation in the fusiform gyrus, trending towards a rightward asymmetry in response to photographs of faces expressing either linguistic or affective information (contrasted to a gender judgment baseline task). In contrast, deaf subjects showed greater left than right fusiform gyrus activity for both types of facial expressions, indicating a left hemisphere bias for processing facial expressions (McCullough et al., 2005), as in the study by Corina (1989). Hearing native signers did not show the left hemisphere bias in this region (Emmorey & McCullough, 2009), indicating that ASL experience alone could not account for the leftward shift observed in the deaf group.

These reports reveal that early sensory experience not only changes the functional anatomy in dorsal stream regions, which were thought to be particularly malleable, but also induce changes in the ventral visual stream. However, none of these studies of experience-based cortical plasticity in deaf individuals described above have examined changes in both the dorsal and ventral processing streams simultaneously within the same participants. Further, it is unclear whether the ventral pathway differences identified by McCullough et al. (2005) and Emmorey & McCullough (2009) resulted from the communicative and affective nature of the stimuli (i.e. linguistic and emotional expressions), or rather, if the fundamental substrate for face processing was altered. That is, are physiological differences also found in deaf signers for non-expressive faces like those used in behavioral studies of deaf participants (Arnold & Murray, 1998; Bettger, Emmorey, McCullough, & Bellugi, 1997; McCullough & Emmorey, 1997; Bellugi et al., 1990), and like those used in brain imaging studies with hearing individuals (Gauthier, Tarr, Anderson, Skudlarski, & Gore, 1999; Haxby et al., 1994; Haxby et al., 1999; Kanwisher, McDermott, & Chun, 1997; Yovel & Kanwisher, 2004)? Finally, it is unknown if the differences observed by McCullough et al. reflect a more basic change in the physiology of object perception. That is, do similar group differences arise for other, non-face objects, known to activate discrete ventral stream regions during perception and identification, such as houses?

The present study sought to explore functional changes in dorsal and ventral stream tasks (spatial processing and object processing, respectively), and to account for them with regard to sensory experience, language experience, or their interaction. Thus, we examined the claim of disparate plasticity in these two visual pathways. Furthermore, we sought to clarify the determinants of plasticity related to face perception in the ventral visual stream by examining whether non-face objects (i.e. houses) elicit similar differences in ventral temporal cortex as do faces. To accomplish these goals we modified a paradigm previously shown to successfully differentiate the dorsal and ventral visual pathways in hearing populations (Grady et al., 1992; Grady et al., 1994; Haxby et al., 1991; Haxby et al., 1994). During scanning, hearing non-signers, deaf signers, and hearing signers performed tasks that involved either spatial processing (matching images based on their spatial arrangement) or object processing (matching images based on their identity), thereby preferentially tasking the dorsal and ventral visual pathways respectively. Just as in previous studies (e.g. Haxby et al., 1994), the actual stimuli used for engaging these two aspects of the visual system were the same and only the task requirements changed, with subjects attending to the spatial arrangement or the object identity of the stimuli in different blocks. We predicted that dorsal stream regions associated with spatial cognition (e.g. parietal regions) would exhibit greater activity in deaf and possibly hearing signers compared to non-signers during the spatial matching task. Stimuli included pictures of faces and houses, which are object categories known to elicit differential activity within the ventral visual stream (Kanwisher, McDermott, & Chun, 1997; Haxby et al., 1999; Williams, Morris, McGlone, Abbott, & Mattingley, 2004), allowing us to test our second hypothesis: we anticipated group differences for the processing of faces, but not for non-face objects (i.e. houses) in the ventral visual stream because face perception holds special significance for deaf individuals,.

2. Materials and Methods

2.1 Subjects

Ten hearing participants with no sign language experience, 15 congenitally deaf native users of American Sign Language, and 11 hearing native users of American Sign Language participated in the study. The Georgetown University and the Gallaudet University Institutional Review Boards approved all experimental procedures and we obtained written informed consent from all participants. Five deaf and three hearing signers were excluded from data analysis because either they moved excessively (> .20 voxels) during scanning or had poor behavioral performance on the tasks (< 75% correct). Thus, data analysis included ten hearing non-signing (five female), ten deaf signing (six female), and eight hearing signing individuals (seven female). Inclusion of the hearing signing group controlled for sensory experience, which they share with hearing non-signers but not with the deaf group; this group also controlled for visuospatial language experience, which they share with the deaf group, but not with hearing non-signers. All participants were healthy adults with no neurological or psychiatric illness and were medication free at the time of testing. All deaf participants considered their deafness genetic, all were born to two deaf parents, and all had binaural hearing loss with > 80db loss in the better ear. All hearing signers had at least one deaf parent who used ASL with their child from birth, and all had worked or were currently working as interpreters for the deaf community. A live 10-minute interview developed within the laboratory was administered to all signers by an ASL-fluent signer to verify that hearing and deaf signers were equally proficient in ASL and ensure group homogeneity.

Two independent raters (one deaf, one hearing) evaluated the videotaped interview using commonly accepted ASL features as scoring criteria (e.g. spatial or shape classifiers, facial grammatical markers, etc.). The scores from the two raters were significantly correlated (r = .50; p < .001) and revealed that the groups were equally proficient. We administered a battery of neuropsychological tests to all subjects to determine cognitive performance on measures relevant to our experimental tasks. Performance IQ was measured with the Matrix Reasoning and Block Design subtests of the Wechsler Abbreviated Scale of Intelligence (Wechsler, 1999). The Facial Recognition Test (Benton, Sivan, Hamsher, Sivan, Varney, & Spreen, 1983) and the Judgment of Line Orientation Test (Benton, Sivan, Hamsher, Varney, & Spreen, 1983) were included to assess face and spatial processing, respectively (See Table 1). As part of a larger study all subjects participated in additional fMRI and behavioral tests not relevant for the current report.

Table 1.

Summary of participant group characteristics, standardized neuropsychological test scores, and behavioral data collected during scanning¹.

	Hearing Non-Signers	Hearing Signers	Deaf Signers
N	10 (5 female)	8 (7 female)	10 (6 female)
Age	23.2 (2.5)	28.5 (6.3) **	23 (2.9)
Performance IQ	119.8 (7.4)	116.7 (11.5)	112.6 (8.6)
Benton Facial Recognition Test	114.4 (13.1)	119.4 (6.1)	118.7 (10.4)
Benton Judgment of Line Orientation	109.9 (8.0)	109.1 (8.8)	108.3 (4.0)
Reaction Times (ms)
Spatial Matching
Faces	1454 (85.4)	1468 (60.9)	1378 (40.7)
Houses	1462 (93.7)	1449 (50.2)	1392 (48.4)
Object Matching
Faces	1527 (80.0)	1533 (65.1)	1283 (59.9)
Houses	1402 (66.9)	1447 (70.8)	1260 (56.8)
Accuracy (% correct)
Spatial Matching
Faces	95 (1.7)	93 (2.7)	88 (1.5)
Houses	90 (1.9)	91 (1.9)	86 (1.4)
Object Matching
Faces	92 (1.6)	94 (1.7)	91 (2.0)
Houses	90 (2.0)	88 (2.6)	89 (1.4)

^{1 **}Hearing non-signers and deaf signers were slightly younger than hearing signers (p < .01). Group averaged median reaction times and mean accuracy scores were equivalent across groups for all tasks and conditions (F < 2). Participants were slower and more accurate performing matching tasks with Faces than those with Houses (p < .01). Numbers in parenthesis indicate standard error of the mean.

2.2. Stimuli and tasks

During scanning, participants performed a simultaneous match-to-sample task, attending to either spatial location or object identity in separate blocks. Consistent with methods used previously to examine dorsal and ventral stream function (Haxby et al., 1994), we used the same stimuli for these two tasks (with unique exemplars for each task, counterbalanced across subjects). The advantage of this design is that visually, the tasks are identical, and the attended feature (spatial location versus object identity) is the only difference between tasks. The stimuli consisted of three large white squares outlined in black, arranged in a triangular configuration on a grey background (Figure 1). Either the right or left edge of the top square had a thickened black line. In the bottom two squares, the thickened black line always appeared along the bottom edge. Each square contained a black and white photograph of an object (faces or houses, the same category was presented for all three squares). The top square and one bottom square depicted the same exemplar from different views and the third square depicted a different exemplar from the same category.

Figure 1.

Examples of the simultaneous match-to-sample tasks. Blocks of Spatial or Object Matching depicted either Faces (A) or Houses (B) in the embedded squares. During Spatial Matching subjects decided which bottom square contained the photograph in same position relative to the thick line as in the top square. During Object Matching, subjects decided which of the two bottom squares depicted the same object as the sample in the top square, regardless of view.

Spatial Matching

The Spatial Matching task required participants to indicate via a button press which bottom square contained the photograph in the same position, relative to the thick line, as in the top square. Instructions were to locate the thick line in the top square (along the left or right) and mentally rotate the square to decide which bottom square (thick line along the bottom edge) was a match. Faces and houses were depicted in the squares during these trials, but were not relevant to the task.

Object Matching

During the Object Matching task participants indicated via a button press, which bottom square contained a picture that matched the identity of the object in the top square. Objects were faces, with neutral expressions, in some blocks, and houses in others. The thick lines were present, but not relevant to the task.

During Spatial and Object Matching subjects held a one-button response device in each hand and pressed the button corresponding to their choice for each trial. For the ten trials in each block, correct responses were equally divided between left and right, in random order. Each trial remained visible until the participant responded or 2500 ms, whichever was shorter, and was then replaced by an inter-trial interval of 500 msec or longer to yield a total trial length of 3s. Each block contained only pictures of faces or pictures of houses, preceded by a 3s cue instructing participants to attend to either “Object” or “Location”.

Baseline Tasks

During the active baseline blocks, preceded by the cue “Alternate”, subjects viewed three squares, each with a thick line along the bottom edge and each containing identical phase-scrambled images of either a face or a house photograph. The unrecognizable phase-scrambled images were scrambled in the Fourier domain and preserved the contrast, luminance and frequencies of the original images. Active baseline Scrambled blocks contained 10 unique trials, and participants were instructed to alternate left and right button presses for each trial. In addition, a 12s period of Fixation followed each cycle, comprising a resting passive baseline.

Inter-trial intervals for all active conditions depicted capital letters in the center of the screen denoting the current task: an F (Faces) or an H (Houses) during Object matching blocks, an L (Location) during Spatial matching blocks, and an A (Alternate) during the scrambled image blocks. Each eight minute imaging run contained two blocks each of Object Matching with Faces, Object Matching with Houses, Spatial Matching with Faces, and Spatial Matching with Houses, with task and stimulus order counterbalanced across runs and subjects. Stimuli were presented and responses recorded (accuracy and reaction time) using Presentation software (Neurobehavioral Systems, Albany, CA) running in Windows XP. Subjects viewed stimuli via a mirror mounted on the head coil that reflected the image from a screen positioned behind them at the end of the magnet bore.

Prior to scanning, all subjects received instruction is their native language and practiced with stimuli not presented during scanning. The number of practice trials was approximately equivalent to one imaging run. Due to the additional complexity of instructions for the Spatial Matching task, additional practice was given as required to attain criterion performance (> 75% correct).

2.3 Imaging parameters

Participants were scanned on a 3T Siemen’s Trio scanner using gradient echo, echo planar imaging (50 contiguous axial slices, FOV = 209mm, TR = 3s, TE = 40ms, flip angle = 90 degrees, 64 × 64mm matrix, yielding 3 × 3 × 3.25mm voxels). A T1 weighted anatomical scan was collected in the same scanning session (160 contiguous sagittal slices, FOV = 256mm, TR = 1600ms, TE = 4.38ms, flip angle = 15 degrees, 256 × 256mm matrix, yielding 1mm³ voxels).

2.4 Behavioral data analysis

Neuropsychological test scores and demographic measures for the three participant groups were entered into separate one-way analysis of variance (ANOVA) tests. Behavioral data collected during scanning were assessed for each trial type to derive accuracy and median reaction times for each subject. These data were then averaged for each group and entered into a 3 (Group: hearing non-signers, deaf signers, hearing signers,) × 2 (Task: Spatial Matching, Object Matching) × 2 (Category: Face, House) ANOVA.

2.5 Imaging data analysis

All imaging data pre-processing and analysis were performed using AFNI (Cox, 1996). For each participant, functional data were motion corrected, spatially smoothed using a 4.5 mm FWHM 3D Gaussian filter, and converted to percent signal change. Six regressors of interest (Object Matching for Faces; Object Matching for Houses; Spatial Matching for Faces; Spatial Matching for Houses; Scrambled Faces, Scrambled Houses) were convolved with a gamma variate estimate of the hemodynamic response and multiple regression was performed on each voxel’s time series. Cue screens for each task were included in these regressors. Fixation period time points comprised the baseline for the regression model. The beta weights generated by this analysis for each participant for each condition were spatially normalized to standardized space (Talairach & Tournoux, 1988) and entered into random-effects ANOVAs for within- and between-group analyses (each subject’s high resolution anatomical scan was transformed to the TT_27 template provided in AFNI, and this transformation was applied to the functional data). Because we found activity to be stronger than expected for the active baseline condition (Scrambled images), most likely due to the novelty and complexity of the stimuli, we did not focus on this condition as a control, as originally intended. Thus, although the Scrambled image blocks were modeled with a regressor at the individual subject level, all results discussed in this report were derived from direct contrasts between Spatial and Object Matching, or between Face and House Matching.

2.5.1 Within-group analysis

For each of the three groups, we identified brain regions associated with Spatial or Object processing with a random effects analysis at the group level (p < .005) by directly contrasting Spatial versus Object Matching. To ensure that these regions reflected signal increases during the task of interest (rather than decreases from baseline), we masked this contrast by voxels that were more active during the task conditions than during the Fixation passive control condition (i.e. Spatial Matching > Fixation or Object Matching > Fixation, respectively). Finally, to protect against false positives, we performed Monte Carlo simulations to determine the minimum cluster extent surviving a corrected statistical threshold of p < .05 for each effect (AlphaSim software by B. Douglas Ward, part of the AFNI analysis package).

To determine regions differentially responsive to each object category, voxels surviving the analysis described above for the contrast of Object Matching > Spatial Matching were further interrogated to identify those significantly modulated by Category (p < .05 for the contrast of Face Matching versus House Matching).

2.5.2 Between-group analysis

Clusters surviving the within-group random effect analysis described above served as regions of interest (ROIs) for performing between-group statistical analysis. For each contrast of interest, we first examined ROIs representing similar anatomical regions across all three groups, and then examined those specific to the deaf signing or hearing non-signing group. ROIs were considered anatomically similar if, across groups, the clusters overlapped or their local maximas were located within 15mm of each other. Clusters present only in the deaf signing or only in the hearing non-signing group were applied as ROIs to the other group(s). This approach enabled between-group comparisons even in regions that did not emerge in all groups via within-group analysis. For every ROI, we submitted the extracted MRI signal from each individual’s functional scans to a mixed-effects ANOVA as described below.

To address our hypothesis concerning plasticity in the dorsal visual pathway, we entered the data from ROIs that survived statistical thresholding in the Spatial Matching > Object Matching contrast into a 2 (Group: deaf signers, hearing non-signers) × 2 (Task: Spatial Matching, Object Matching) ANOVA comparing task-related activity between the deaf and hearing non-signing groups. For this analysis we collapsed across Category (faces and houses), as our hypothesis concerned group differences during Spatial Matching and object category is not relevant. Regions exhibiting group differences (main effect of Group or interaction with Group) were further interrogated with two additional pair-wise ANOVAs comparing 1) deaf and hearing signers, and 2) hearing signers and non-signers. These latter tests allowed us to determine whether differences between the deaf and hearing non-signing groups were related to sensory or language experience (Bavelier et al., 2001).

To address our hypothesis regarding plasticity in the ventral visual stream, we conducted between-group analyses with independent-samples, two-tailed t-tests, but this time, we examined ROI’s that survived statistical thresholding in the within-group analyses for the Object Matching > Spatial Matching contrast, followed by the comparison of Face Matching versus House Matching. Regions preferentially responsive to faces (i.e. Face Matching > Houses Matching) and those preferentially responsive to houses (i.e. House Matching > Face Matching) were analyzed separately. Again, to examine the influence of sensory and language experience, any region exhibiting differences between the deaf signers and hearing non-signers were subsequently analyzed with pair-wise independent-samples t-tests comparing each of these groups with hearing signers. We report the significance level of all between-group differences surpassing the threshold of p < .05.

3. Results

3.1 Group characteristics and neuropsychological data

Table 1 presents a summary of group characteristics and neuropsychological test scores. Hearing signers were somewhat older than the other two groups (main effect of group, F (2, 25) = 5.14; p < .01; hearing signers vs. hearing non-signers, F = 7.70, p < .05; hearing signers vs. deaf signers F = 8.29, p < .01). All subjects performed in the normal range on the neuropsychological tests (Performance IQ, Facial Recognition Test, and Judgment of Line Orientation Test) with no differences between groups (F < 2 for each measure).

3.2 Behavioral data for Spatial and Object Matching

Analysis of the behavioral performance data collected during scanning for the Spatial Matching and Object Matching tasks indicated that all groups performed equally well (see Table 1; main effect of Group, F < 2 for accuracy and for reaction times). Moreover, neither sensory experience nor language experience interacted with the factors of Task or Category. Although the Spatial and Object Matching tasks proved to be equally difficult (F < 2), a significant main effect of Category (F (1,25) = 9.24, p < .01) revealed that subjects were more accurate when the Object Matching tasks depicted Faces than when they depicted Houses. The reaction time data suggest that this may reflect a speed/accuracy trade-off, as subjects were slower performing matching tasks with Faces than with Houses (main effect of Category, F (1,25) = 10.34, p < .01), especially during Object Matching (Task × Category interaction, F (1,25) = 8.70, p < .01).

3.3 Imaging data

3.3.1 Within-group results

Consistent with previous studies in hearing adults, within-group random effects analysis of the functional imaging data revealed that in each group the Spatial Matching and Object Matching tasks led to increased hemodynamic responses in dorsal and ventral extrastriate cortex, respectively (see Figure 2). Further, Face Matching and House Matching increased responses differentially in the lateral fusiform and parahippocampal/medial fusiform gyri, respectively (see Figure 3). Next we describe these results in detail, first focusing on the brain regions preferentially activated by Spatial Matching compared to Object Matching and vice versa, and then continue with a description of regions demonstrating differential responses to Faces and Houses. Lastly we address between-groups differences for each of these effects.

Figure 2.

Activation patterns in the dorsal visual pathway for hearing non-signers, deaf signers, and hearing signers. Coronal sections depicting clusters of activation from each group’s random effects analysis contrasting Spatial vs. Object matching, overlaid on a single subject’s structural image (Talairach and Tournoux coordinate y = −36). Regions in bilateral inferior parietal lobe (blue) were more active for Spatial than Object Matching (right IPL activity is not visible in this slice for hearing signers). Bilateral fusiform and parahippocampal regions (red) responded more during Object than Spatial matching.

Figure 3.

Axial sections (z = −11) depicting regions that responded differentially to Faces and Houses during Object matching. Bilateral lateral fusiform gyri (red regions) were more active during Face than House matching in all groups, and bilateral parahippocampal gyri (blue regions) were more active during House than Face matching. All regions p < .05, corrected.

Spatial > Object Matching

Regions depicted in blue in Figure 2 (see also Table 2) reflect increased brain activity during the Spatial Matching task relative to the Object Matching task. In all three groups, these areas were largely confined to the dorsal visual pathway, with large clusters of activity located in the bilateral superior and inferior parietal lobules (superior parietal lobule activity not visible in Figure 2). For the deaf group, signal increases were also seen for the Spatial Matching task in bilateral premotor cortex (BA 6) and right cuneus (BA 17/18).

Table 2.

Local maxima for significantly activated clusters in each group for the Spatial Matching vs. Object Matching contrast (p < .05, corrected).

Region	# voxels	x	y	z	t-value
Spatial > Object Matching
Hearing non-signers
R inferior parietal lobule/postcentral gyrus (BA 40/2)	137	52	−22	35	−4.24
L inferior parietal lobule/postcentral gyrus (BA 40/2)	67	−46	−28	29	−3.75
R superior parietal lobule (BA 7)	92	16	−52	50	−7.19
L superior parietal lobule (BA 7)	75	−10	−70	44	−3.73
Deaf signers
R middle frontal gyrus (BA 6)	103	22	4	44	−7.74
L middle frontal/precentral gyrus (BA 6)	35	−22	1	47	−3.8
L inferior parietal lobule/postcentral gyrus (BA 40/2)	187	−52	−22	35	−3.82
R inferior parietal lobule/postcentral gyrus (BA 40/2)	221	43	−25	23	−4.1
L superior parietal lobule (BA 7)	187	−16	−49	47	−3.87
R superior parietal lobule (BA 7)	147	19	−58	41	−5.03
R cuneus/middle occipital gyrus (BA 17/18)	50	22	−79	14	−4.51
Hearing signers
R inferior parietal lobule/postcentral gyrus (BA 40)	13	52	−25	38	−4.4
L inferior parietal lobule/postcentral gyrus (BA 40/2)	43	−52	−31	35	−4.94
L superior parietal lobule (BA 7)	53	−10	−55	47	−4.71
R superior parietal lobule (BA 7)	78	19	−58	44	−6.1
Object > Spatial Matching
Hearing non-signers
R middle frontal gyrus (BA 46)	20	49	43	2	4.47
R inferior frontal gyrus (BA 47)	17	31	28	−6	3.83
L inferior frontal gyrus (BA 47)	10	−28	22	−6	4.13
R inferior/middle frontal gyrus (BA 44/9)	19	34	13	23	4.11
L inferior temporal/fusiform/parahippocampal gyrus (BA 20)	11	−34	−4	−27	3.73
L thalamus	30	−4	−7	2	4.15
R fusiform gyrus (BA 36)	538	34	−13	−24	4
L fusiform gyrus (BA 36/37)	494	−34	−40	−18	4.37
L cerebellum	11	−1	−52	−30	3.72
	10	−4	−73	−33	5.73
L middle occipital gyrus (BA 19)	21	−40	−79	8	3.81
R cerebellum	12	10	−79	30	4.05
Deaf signers
R superior temporal gyrus/sulcus (BA 22)	29	46	−13	−6	3.79
L fusiform/parahippocampal gyrus (BA 35/36)	449	−25	−28	−18	3.77
R fusiform/parahippocampal gyrus (BA 36)	215	31	−34	−18	4.09
R fusiform/inferior occipital gyrus (BA 18/19)	119	37	−73	−9	3.93
Hearing signers
L middle frontal gyrus (BA 10)	18	−28	49	2	4.27
L anterior cingulate gyrus (BA 24/32)	163	−4	34	23	4.33
R orbital gyrus (BA 11)	213	25	28	−12	4.26
R inferior frontal gyrus (BA 45)	18	37	22	17	3.89
L inferior frontal gyrus (BA 44)	50	−37	13	20	4.16
R inferior frontal gyrus (BA 44)	12	55	13	26	3.83
R lentiform nucleus	37	16	4	0	3.75
R middle frontal/precentral gyrus (BA 6)	31	37	4	29	4.01
L lenfiform nuclueus	34	−13	1	0	3.71
R inferior temporal gyrus (BA 20)	11	37	−7	−18	3.85
R precentral gyrus (BA 4)	12	40	−10	38	4.58
R lentiform nucleus	12	28	−19	0	4.26
R parahippocampal gyrus (BA 36)	664	25	−25	−21	3.71
L fusiform/parahippocampal gyrus (BA 35/36)	614	−25	−28	−18	3.91
R inferior parietal lobule (BA 40)	25	37	−49	35	4.84
L cuneus (BA 17)	39	−1	−70	14	4.62
R cerebellum	18	10	−73	−27	4.97

Object > Spatial Matching

In each group, an extensive region spanning the ventral occipitotemporal cortex of both hemispheres exhibited increased activity during Object Matching, compared to Spatial Matching (Figure 2, red regions). Increased task-related activity spanned bilateral fusiform and parahippocampal gyri in all groups (e.g. BA 35, 36, 37, see Table 2 for local maxima in each group). In addition, all groups showed increased activity in occipital regions: in left middle occipital gyrus in hearing non-signers; in right inferior occipital gyrus in deaf signers; and in the left cuneus in hearing signers. Consistent with previous studies, activity for the Object Matching task was not confined to posterior ventral regions (see Table 2). For both hearing groups, additional regions in bilateral inferior or middle frontal gyri (e.g. BA 44), and in the cerebellum responded more to Object than Spatial Matching. Hearing non-signers also showed relatively greater Object Matching activity in a left anterior temporal region bordering the inferior temporal, fusiform, and parahippocampal gyri (BA 20), and in the left thalamus. The deaf group showed an additional cluster spanning the right anterior superior temporal gyrus and sulcus (BA 22). Lastly, hearing signers demonstrated object-related activity increases in the right inferior parietal lobule (BA 40), bilateral lentiform nucleus, right precentral (BA 4) and orbital (BA 11) gyri, and in the left anterior cingulate gyrus (BA 24/32).

Faces vs. House Matching

Consistent with previous reports, the within-group analyses of category-selectivity revealed that, in regions responding more to Object than Spatial Matching, activity was increased for faces, relative to houses, in the lateral fusiform gyrus, bilaterally (BA 19/37; Figure 3, red regions and Table 3). Face-selective responses in brain regions beyond ventral occipitotemporal cortex were not consistently activated across groups: both hearing groups showed heightened activity for faces relative to houses in middle and inferior frontal gyri (non-signers on the right, signers on the left) whereas only hearing non-signers demonstrated face selectivity in the right thalamus (pulvinar). The deaf group alone showed increased face-, compared to house-related activity spanning the right anterior superior temporal gyrus and sulcus (BA 22) near the temporal pole and in the right amygdala.

Table 3.

Local maxima for significantly activated clusters in each group the comparison of Face Matching vs. House Matching (p < .05) within clusters that showed an Object Matching > Location Matching effect (p < .05, corrected).

Region	# voxels	x	y	z	t-value
Face > House Matching
Hearing non-signers
R inferior frontal gyrus (BA 45)	6	49	37	5	3.27
R middle/inferior frontal gyrus (BA 8/9)	7	34	15	23	2.76
L inferior temporal gyrus (BA 20)	4	−34	−4	−27	3.14
R thalamus	8	1	−16	8	3.16
R pulvinar	3	1	−25	1	4.39
L inferior temporal/fusiform gyrus (BA 36/20)	7	−37	−34	−12	2.89
R fusiform gyrus (BA 37)	63	37	−43	−18	4.15
L fusiform gyrus (BA 37)	48	−40	−49	−12	2.36
L fusiform gyrus (BA 19)	3	−43	−67	−15	2.88
R cuneus/middle occipital gyrus (BA 18)	4	19	−100	2	2.47
Deaf signers
R amygdala	4	16	−7	−9	3.85
R superior temporal gyrus/sulcus (BA 22)	5	46	−13	−6	2.38
R fusiform gyrus (BA 37)	5	37	−40	−12	3.1
L fusiform gyrus (BA 37)	12	−37	−40	−18	2.72
L inferior occipital/fusiform gyrus (BA 18)	24	16	−73	−9	2.56
R inferior occipital/fusiform gyrus (BA 18)	4	46	−76	−6	5.77
Hearing signers
L inferior/middle frontal gyrus (BA 8/44)	5	−31	10	26	2.32
R fusiform gyrus (BA 36/37)	23	34	−40	−15	4.1
L fusiform gyrus (BA 37)	15	−37	−40	−18	4.06
L fusiform/inferior temporal gyrus (BA 37)	11	−40	−52	−12	3.81
R fusiform gyrus (BA 37)	17	43	−58	−6	4.69
R precuneus (BA 31)	11	1	−64	23	4.47
House > Face Matching
Hearing non-signers
R parahippocampal gyrus (BA 35)	204	19	−28	−15	−4.58
L parahippocampal/fusiform gyrus (BA 35/36)	82	−31	−28	−15	−4.04
L lingual/parahippocampal gyrus (BA 19)	21	−13	−49	5	−4.5
L inferior temporal gyrus (BA 37)	3	−49	−58	−6	−3.33
R inferior occipital/lingual gyrus (BA 19)	45	25	−82	−6	−2.71
R middle occipital gyrus (BA 19)	4	37	−82	5	−3.05
L fusiform gyrus (BA 18/19)	34	−28	−82	−12	−2.67
Deaf signers
R parahippocampal gyrus (BA 28/36)	127	22	−22	−15	−5.94
L parahippocampal/fusiform gyrus (BA 36/35)	109	−25	−28	−18	−5.84
L fusiform/inferior temporal gyrus (BA 37)	4	−40	−55	−6	−2.62
R fusiform gyrus (BA 19)	50	34	−82	−6	−3.16
L fusiform gyrus (BA 18/19)	84	−31	−82	−12	−2.48
Hearing signers
R middle frontal gyrus (BA 46/10)	4	43	43	−3	−2.55
L middle/inferior frontal gyrus (BA 46)	6	−25	43	11	−2.29
R parahippocampal gyrus (BA 27/28)	211	22	−22	−18	−2.84
	67	10	−34	2	−3.92
L fusiform/parahippocampal gyrus (BA 35/36)	161	−25	−37	−18	−2.88
L parahippocampal gyrus (BA 30)	27	−10	−43	2	−2.3
L middle/superior temporal gyrus (BA 21/22)	5	−49	−49	−3	−3.16
R inferior parietal lobule (BA 7)	9	37	−58	38	−2.56
L fusiform/inferior occipital gyrus (BA 19)	52	−31	−82	−6	−2.37
R lingual/fusiform gyrus (BA 18)	66	19	−88	−9	−2.54

Also consistent with previous findings, houses elicited stronger responses than faces for each group in the medial aspect of the fusiform gyri (BA 18/19/37) and the parahippocampal gyri (BA 28, 35, 36; Figure 3, blue regions and Table 3). In all groups the inferior occipital gyrus (BA 19) also showed a preference for houses. Additional ventral extrastriate regions showing greater activity for houses than faces included the lingual gyrus for both hearing groups (though in opposite hemispheres) and the left inferior temporal gyrus for hearing non-signers and deaf signers. Beyond the ventral visual pathway house-selective responses were seen only in hearing signers, in the right inferior parietal lobule (BA 7), left middle/superior temporal gyrus (BA 21/ 22), and bilateral middle frontal gyri (BA 10, 46).

3.3.2 Between-group results

Spatial > Object Matching

We next examined differences between deaf and hearing non-signers in brain regions that survived the within-group analyses reported above (see Table 2), beginning with the Spatial Matching > Object Matching contrast. Analysis of the four ROIs corresponding to analogous regions identified in deaf signers and hearing non-signers, located in bilateral superior and inferior parietal lobules (see Table 2), revealed a main effect of Task (Spatial > Object Matching, p < .001 in each region) (Figure 4), but no main effect of Group (F < 3 for each region). However, a significant Task × Group interaction revealed that responses differed between the deaf signing and hearing non-signing groups in the left inferior parietal lobule (IPL) and right superior parietal lobule (SPL). In the left IPL, deaf signers exhibited greater activity than hearing non-signers during Spatial Matching (Task × Group interaction, F (1, 18) = 12.38, p < .002) (Fig. 4). This heightened response in the left IPL was also significantly greater in deaf signers than in hearing signers (Task × Group interaction, F (1,16) = 10.30, p < .005). The two hearing groups did not differ from each other (F < 2 for main effect and interaction) in this region.

Figure 4.

Group results in ROIs based on within-group analyses, showing sensory experience-related differences between deaf and hearing groups. Cell means for the deaf group (red bars) were higher than either hearing group (blue and yellow bars) during Spatial Matching in the left IPL (A) and lower than hearing groups in the right SPL (B). In the left IPL (A), pair-wise between-group ANOVAs demonstrated Task × Group interaction (deaf signers vs. hearing non-signers, p < .002; deaf signers vs. hearing signers, p < .005). The hearing groups (signing and non-signing) did not differ from each other in this region. In the right SPL region (B) pair-wise ANOVAs revealed a Task × Group interaction between the deaf group and the hearing non-signers (p < .001), and a main effect of Group when deaf and hearing signers where compared (p < .004). See Table 2 for local maxima for each group.

However, the opposite effect was found in the right SPL, where Spatial Matching elicited greater activity in hearing non-signers than in deaf signers (Task × Group interaction, F (1, 18) = 14.58, p < .001). Hearing signers also activated the right SPL more than deaf signers (main effect of Group (F (1, 16) = 11.43, p < .004) and again, no differences were identified between the two hearing groups (F < 1 for main effect of Group; F < 3, p > .1 for Task × Group interaction). These findings suggest that for visuospatial processing, activity increases in left IPL and decreases in right SPL in deaf, compared to hearing individuals, are related to sensory experience.

Additional dorsal clusters that emerged for the deaf group but not for hearing non-signers in prefrontal cortex and cuneus showed only a main effect of Task (p < .001 in each region).

Face > House Matching

We next determined whether hearing and deaf groups differed in their response to specific object categories by examining ventral stream brain regions that showed increased responses during Object Matching relative Spatial Matching and responded preferentially to Face Matching vs. House Matching. Independent-samples t-tests comparing the responses of deaf signing and hearing non-signing groups in these face-preferring regions were followed-up with pair-wise t-tests with the hearing non-signers where appropriate to separately assess the effects of sensory and language experience.

Of the two clusters located in the left and one in the right fusiform gyrus showing preferential responses to faces during Object Matching (see Table 3), only one of these revealed a significant difference between deaf signers and hearing non-signers (Figure 5). Specifically, deaf individuals exhibited a smaller response than hearing non-signers in the lateral part of the right middle fusiform gyrus (BA 37) (t (1,18) = 2.12, p < .05, two-tailed). Pair-wise t-tests between the two signing groups and between the two hearing groups revealed no further differences (p > .1), suggesting that the reduced response for face recognition in deaf signers cannot be attributed solely to sensory or language experience, but rather, reflects the combined effects of both.

Figure 5.

Group averaged hemodynamic responses in face-selective regions exhibiting group differences between deaf signers and hearing non-signers. The deaf group (red bars) showed a reduced response for faces in the right lateral fusiform gyrus (A) compared to hearing non-signers (blue bars) (p < .05), and an increased response in the right STG/STS region (B) (p < .05). However, comparison of these groups with hearing signers revealed no differences, suggesting that these effects are not specific to either sensory or signing experience alone. The right amygdala (C) was more active in deaf subjects than in hearing non-signers (p < .05) and hearing signers (p < .01), indicating an effect of deafness. Below each histogram are coronal slices depicting the conjunction of each groups’ clusters for the regions described above. Activity for each group, and their conjunctions, are color coded as follows: orange = all groups; purple = deaf signers and hearing non-signers; green = hearing groups; blue = hearing non-signers; yellow = hearing signers; red = deaf signers.

The within group analysis had identified two additional clusters in the hearing non-signing group, located in the left lateral fusiform gyrus of the temporal lobe that were each nearby, but not contiguous with the cluster in BA 37 discussed above (see Table 3). Despite increasing the extent of the left fusiform ROI for hearing non-signers by averaging the signal from these additional voxels, our results were unaltered, still indicating a trend towards a weaker response in the deaf group (t (1,18) = 1.82, p = .08).

In addition to these analogously located ROIs, six additional face-selective clusters emerged only in the hearing non-signers' within- group analysis (reported in Table 3). We found no between-group differences in these regions, indicating that although responses were similar across groups, face-selectivity was not robust enough to survive stringent within-group statistical thresholding in all groups.

Lastly, the within-group analysis identified three regions selectively responsive during Face Matching that survived cluster analysis only in the deaf, but not in the hearing groups. Of these ROIs, those located in the right anterior superior temporal gyrus/sulcus (STG/STS) and the right amygdala revealed greater activity in deaf subjects compared to hearing non-signers, (t (1,18) = 2.28, p < .05 in the STG/STS; t (1,18) = 2.38, p < .05 in the amygdala; see Figure 5). In the right amygdala the response of deaf signers was also greater than that of hearing signers (t (1,16) = 3.19, p < .01), but responses of the two hearing groups were comparable to each other (t (1,16) = 1.07, p > .1), suggesting that this difference is likely related to deafness. In the right STG/STS, there were no differences in task-related responses when comparing hearing signers to either deaf signers (t (1,16) = 1.26, p > .1) or hearing non-signers (t (1,16) = .048, p > .1). Thus, it is less clear whether the source of the heightened response in deaf participants is related to sensory or language experience.

House > Face Matching

Regions common across groups more responsive to houses than faces during Object Matching were all located in the ventral occipitotemporal object processing stream and included the bilateral parahippocampal gyrus, extending into the medial part of the fusiform gyrus, the posterior fusiform gyrus of the occipital lobe (BA 18,19) and the left inferior temporal/fusiform region (BA 37). Pair-wise t-tests revealed no differences between groups in any of these regions. Two additional occipitotemporal regions (BA 19) that survived within-group analysis in hearing non-signers but not in the deaf group failed to reveal any between-group differences. Thus, the response to houses did not differ between the deaf and hearing non-signers in any region preferring house to faces.

4. Discussion

4.1 Overall Findings

To investigate the effects of deafness and visuospatial language experience simultaneously in dorsal and ventral visual pathways, we studied subjects with differing sensory and language experience while they attended to the spatial location or object identity of static face and house photographs. Deaf native signers, hearing native signers, and hearing non-signers all demonstrated the task-specific dissociations in the dorsal and ventral cortical processing streams previously reported for spatial and object processing (e.g. Haxby et al., 1994). In addition, each group showed face and house category-related activation patterns in the ventral visual stream, with greater activity for faces than houses in the lateral fusiform gyri, and greater activity for houses than for faces in the more medial fusiform and parahippocampal gyri (Haxby, et al., 1999; Epstein & Kanwisher, 1998). As predicted, however, we also identified discrete differences between the deaf and hearing groups in both visual processing streams. Specifically, in deaf participants, the Spatial Matching task elicited less activity in the right superior parietal lobe and more activity in the left inferior parietal cortex relative to hearing participants. These differences were also observed when the deaf group was compared with native hearing signers, indicating that deafness is most likely a strong impetus for this cortical reorganization.

We also identified differences between groups during Object Matching in several face-preferring regions. Face Matching elicited less activity in deaf signers relative to hearing non-signers in the right lateral fusiform gyrus, and more activity in the right anterior temporal cortex. However, comparisons between the two signing groups and between the two hearing groups revealed no differences, suggesting that sensory and language experience combine to induce changes in these regions. Finally, activity in the right amygdala was greater in the deaf group than in either hearing group, indicating a substantial influence of deafness. During scanning, behavioral performance on each task was comparable between all three groups, suggesting that the these changes in brain function reflect re-allocation of processing resources rather than task difficulty.

4.2 Cortical plasticity for spatial processing in deaf and hearing signers

All three participant groups activated bilateral parietal cortex during spatial processing, consistent with a large body of animal studies on spatial processing (Colby & Goldberg, 1999), and with clinical (Corballis, 1997) and brain imaging studies in which hearing individuals performed various spatial tasks including line bisection, location judgments, spatial navigation, and mental rotation (Fink et al., 2000; Kohler, Kapur, Moscovitch, Winocur, & Houle, 1995; Parsons, 2003; Shelton & Gabrieli, 2002). In deaf signers we observed greater recruitment of the left inferior parietal lobe and less activity in the right superior parietal lobe, compared with both groups of hearing subjects, differences that likely reflect sensory experience. Our finding is consistent with Bavelier et al.’s (2001) report in that they too attributed changes in dorsal stream function, in their case for visual motion processing, to sensory experience (rather than sign experience). Our data extend their work by showing that differences in deaf participants can be elicited by spatial processing as well as during visual motion perception. These differences may be a consequence of deaf individuals attending differently to the visuospatial world than hearing persons because of the absence of auditory cues signaling changes in the environment.

Studies indicate that the SPL is associated with action organization and attention (Colby & Goldberg, 1999; Culham & Valyear, 2006), and the IPL, while perhaps sharing these functions, is also implicated in motor processes such as eye movements, visual orienting to extra-personal space, mental rotation, distance and location judgments, and spatial navigation (Nobre et al., 2004; Vallar, 2001, Aguirre & D'Esposito, 1997; Fink et al., 2000; Kohler, Kapur, Moscovitch, Winocur, & Houle, 1995; Moscovitch, Kapur, Kohler, & Houle, 1995; Stephan et al., 2003; Vallar et al., 1999). These processes may become affected and their functional anatomy reorganized as a result of visual experiences that accompany deafness. For example, in contrast to hearing individuals who can use auditory cues to monitor extrapersonal space, deaf persons rely almost exclusively on visual monitoring to detect changes in their environment. Such constant and life-long visual vigilance may alter many aspects of spatial attention including orienting to peripheral and extrapersonal space, and judging the relative positions of objects (Bavelier, Dye & Hauser, 2006). Notably, although we instructed our participants to mentally rotate the stimuli during the Spatial Matching task, the task engages additional spatial processes associated with the IPL, including distance and location judgments.

Surprisingly, we found no evidence of dorsal stream reorganization specifically attributable to ASL experience, although the visuospatial nature of the language as well as behavioral studies showing enhanced mental imagery and mental rotation in signers might have predicted such a result (McKee 1987; Emmorey, Klima, & Hickok, 1998; Emmorey & Kosslyn, 1996; Emmorey, Kosslyn, & Bellugi, 1993; Talbot & Haude, 1993). Spatial processing has a prominent role in the perception and expression of ASL, which requires signers to attend to peripheral space to encode the gestures and hand shapes that constitute ASL, to place those gestures appropriately to convey meaning, and at times, to perform spatial transformations in order to decode them. For example, in ASL the spatial relationships among the referents of conversation (e.g. left of, behind) are usually conveyed from an egocentric perspective, and the addressee must make mental transformations in order to decode the proper positions of referents (Emmorey, Klima, & Hickok, 1998; Martin & Sera, 2006).

Evidence that the visuospatial nature of ASL influences the organization of dorsal stream function was provided by Bavelier et al.’s. (2001) investigation of visual motion processing, which found a right-to-left lateralization change in recruitment of MT/V5 in deaf and hearing signers, compared to hearing non-signers. Their finding supported earlier ERP and visual field studies documenting changes in lateralization for visual motion processing in signing groups (Bosworth & Dobkins, 1999;Neville & Lawson, 1987a, 1987b) and argues for an intimate relationship between visual motion and a visuospatial language, resulting in the left hemisphere taking a more active role in non-linguistic motion processing than in users of spoken languages (Neville & Bavelier, 2001). However, our results did not support a strong influence of ASL usage on dorsal spatial processing areas, perhaps because the visuospatial aspects of our task have less overlap with those invoked during ASL. Consistent with the idea that for certain tasks, plasticity in dorsal spatial processing regions is shaped more by sensory, than by language experience, Cattani & Clibbens (2005) reported that during location processing, two groups of deaf individuals (signers and non-signers) showed an opposite visual field preference than did two hearing groups (signers and non-signers).

Our findings in parietal cortex offer evidence of plasticity in the dorsal visual stream for tasks that do not involve visual motion, but rather spatial processing. We suggest that altered recruitment of parietal cortices is associated with the necessity for deaf individuals to detect changes in their environment through constant visual monitoring.

4.4 Cortical plasticity for object recognition in deaf and hearing signers

In the ventral visual stream, we identified face-selective and house-selective clusters of activity for all groups in the lateral middle fusiform gyri and parahippocampal gyri, respectively. Between group comparisons revealed that plasticity in these ventral visual pathway areas was limited to cortex demonstrating a preference for faces (as opposed to houses), indicating a stimulus-specific effect. Specifically, group differences during Face Matching were identified within and beyond ventrotemporal cortex: the right lateral fusiform gyrus showed more activity and right superior temporal gyrus/sulcus showed less activity in deaf signers relative to hearing non-signers. However, we found no differences when we compared those groups with hearing signers. This suggests that effects in face-selective areas cannot be directly attributed to either deafness or ASL experience alone, but more likely, to a combination of both. Deaf signers also demonstrated a heightened amygdala response to faces compared with both hearing groups, indicating that this change is strongly associated with sensory experience.

Over the last decade, a number of studies using a wide variety of tasks have focused on a functionally defined region of the fusiform gyrus in hearing populations that reliably shows increased activity for faces relative to houses (and most other objects) (Haxby et al., 1999; Kanwisher, McDermott, & Chun, 1997). Termed the fusiform face area (FFA), this region is thought to be a node critical for the recognition of facial identity (Haxby, Hoffman, & Gobbini, 2000) within a larger network of regions subserving face processing. Given the evidence supporting a face processing advantage in deaf compared to hearing subjects, (Arnold & Murray, 1998; Bettger, Emmorey, McCullough, & Bellugi, 1997; McCullough & Emmorey, 1997) face-responsive regions (including the FFA) can be considered primary candidates for cortical plasticity in deaf individuals. Our findings of differential activity in these regions in the deaf group bear out that prediction. In contrast, the more medial part of the fusiform gyrus and parahippocampal gyrus are associated with the perception of visual scenes, landmarks and houses (Epstein & Kanwisher, 1998; Haxby et al., 1999). Consistent with the few studies that have examined non-face object processing in deaf individuals (Arnold & Mills, 2001; Arnold & Murray, 1998), we did not identify between-group differences in any regions that preferred houses to faces.

This specificity (i.e. altered brain activity underlying face, but not house processing) is likely related to the salience of faces in deaf communication, but raises the question of why some aspects of the ventral stream processing would be amenable to change, whereas others are more resistant. Accumulated evidence suggests that specific types of visual processes are disposed to plasticity accompanying deafness: those that are visually demanding; require attention to specific task features; and would profit from cross-modal integration in hearing individuals (Bavelier, Dye, & Hauser, 2006). Indeed, recognition of person identity shares all three of these characteristics.

As is often the case in functional brain imaging studies of multiple populations, our results force us to grapple with how to interpret the pattern of activity increases and decreases in our deaf group. Increased skill is frequently associated with expanded representations and increased activity in topographically organized cortices. In association cortices, however, higher cognitive skill and perceptual learning is often reflected by decreased activity as a region becomes functionally more efficient (Kelly & Garavan, 2005). One mechanism thought to underlie this increased neural efficiency is sharper neuronal tuning, such that with increased experience, fewer neurons in a region respond to a particular task or stimulus (Desimone, 1996; Kelly & Garavan, 2005). Such decreases are often accompanied by connectivity changes between nodes of a distributed network (Kelly & Garavan, 2005). Although not directly tested in the current study, within this framework, it is likely that the pattern of increased and decreased activity we identified in face-responsive areas in our deaf group mediates the heightened demand placed on face processing in these individuals. Reduced activity in the right fusiform gyrus may reflect a general increase in face processing efficiency resulting from deaf signers’ experience decoding linguistic and affective facial expressions. Skill acquisition studies in diverse domains including language (Perani et al., 2003), visual skill learning (Kourtzi, Betts, Sarkheil, & Welchman, 2005; Poldrack, Desmond, Glover, & Gabrieli, 1998; Schiltz et al., 1999), and motor skill learning in musicians over many years of training (Haslinger et al., 2004) report similar task-specific activation decreases associated with increased efficiency in association cortices. Further support for the notion that increased processing efficiency leads to reduced activity are reports of studies in non-human primates showing greater selectivity for objects with which they have had extensive perceptual training (Baker, Behrmann, & Olson, 2002; Erickson, Jagadeesh, & Desimone, 2000).

A problem with this interpretation is our failure to observe a behavioral enhancement in face processing performance by our deaf participants relative to the other groups. That is, all three of our groups performed equally well on the Benton Facial Recognition Test and on the Face Matching task performed during scanning. However, the latter, experimental task was intentionally designed to avoid group differences so that we would not have to address such differences in the interpretation of our fMRI data. Studies reporting enhanced performance on the Benton Facial Recognition Test studied larger samples than ours (Bellugi et al., 1990; Bettger, Emmorey, McCullough, & Bellugi, 1997) and as can be seen in Table 1, both of our signing groups scored numerically higher on the Benton Facial Recognition Test than did the hearing non-signers.

The neural efficiency hypothesis is also inconsistent with the notion that signal increases in the FFA reflect greater expertise for faces than for other objects in hearing subjects (Gauthier et al., 1999), a model that would predict a signal increase in deaf, compared to hearing subjects, as deaf individuals presumably have greater experience and expertise with faces. However, an alternative view of FFA as a specialized face processing module (Kanwisher, McDermott, & Chun, 1997), rather than a process-specific region, would predict that the FFA maintains face-selectivity in the deaf group, despite their reduced response, which was the case here. McCullough et al. (2005) reported that for deaf signers matching facial expressions (compared to a gender matching task), extent of activation was greater in the left than in the right fusiform gyrus in deaf signers, whereas hearing non-signers showed equal extent in both hemispheres (with a trend to greater right hemisphere activity). Although we did not find increased activity in left FFA, several important experimental design differences may explain this discrepancy. First, McCullough et al. employed face stimuli in both the baseline and activation conditions, and, second, the activation task used faces selected for maximum expressiveness of emotional or linguistic content. Thus, attention to the communicative aspects of these facial expressions may have contributed to heightened engagement in the left hemisphere in deaf individuals. Our study provides an important baseline for the McCullough study, suggesting that the functional anatomy of face processing in the left ventral stream is not heightened in deaf signers overall, but perhaps specifically for processing communicative aspects of the face, or, as proposed by McCullough and Emmorey (1997), signers’ heightened ability to discriminate local facial features.

In the right amygdala we found greater face-related activity in the deaf group compared to either of the hearing groups. The amygdala is frequently active during face perception, and serves to modulate activity in cortical regions based on the social and emotional significance of stimuli (Phelps, 2006). We suggest that the heightened amygdala response in our deaf participants reflects the increased social salience of faces for deaf individuals.

The third face-responsive region demonstrating plasticity spanned the right anterior superior temporal gyrus and sulcus, just inferior to primary auditory cortex, where deaf signers displayed increased activity relative to hearing non-signers, but hearing signers did not differ from either of these groups. This region, which responds during speech reading in hearing and deaf groups, and during perception of signed languages in deaf individuals, is considered to be multimodal association cortex responsible, in part, for integrating visual and auditory signals (Calvert et al., 1997; Nishimura et al., 1999; Petitto et al., 2000; Capek et al., 2008). Additionally, this region is active in deaf individuals during visual motion and tactile perception (Fine, Finney, Boynton, & Dobkins, 2005; Finney & Dobkins, 2001; Lambertz, Gizewski, de Greiff, & Forsting, 2005; Levanen, 1998). Here, we demonstrate that deaf individuals also recruit this region for discriminating static images of faces. It is possible that in the absence of auditory input, visual neurons co-opt more territory in this multimodal region to support increased demands on the visual system. In this view, recruitment of this region to aid visual processing may lessen the burden on ventral temporal cortex during face processing, allowing for reduced right fusiform gyrus signaling. However, we cannot definitively separate the contributions of deafness versus linguistic experience in this re-mapping and the multimodal nature of this region may be particularly susceptible to influences from both sensory and language experience.

5. Conclusions

In conclusion, we have shown that the well-documented neural dichotomy for spatial and object processing is also observed in deaf individuals. At the same time, we have demonstrated experience-related plasticity within both the dorsal and ventral visual streams that is more closely tied to congenital deafness in some regions, and in others to the interaction between deafness and life-long visuospatial language experience. These findings extend previous reports of changes in the dorsal pathway for visual motion processing (Bavelier et al., 2001; Bavelier & Neville, 2002) and in the ventral visual pathway for discriminating facial expressions (McCullough, Emmorey, & Sereno, 2005). Importantly, our findings of both dorsal and ventral stream plasticity following altered sensory experience were elicited within subjects, using the same, static stimuli for both tasks, demonstrating that deafness leads to more widespread plasticity than previously thought.