Lateralization of face processing in the human brain

Abstract

Are visual face processing mechanisms the same in the left and right cerebral hemispheres? The possibility of such ‘duplicated processing’ seems puzzling in terms of neural resource usage, and we currently lack a precise characterization of the lateral differences in face processing. To address this need, we have undertaken a three-pronged approach. Using functional magnetic resonance imaging, we assessed cortical sensitivity to facial semblance, the modulatory effects of context and temporal response dynamics. Results on all three fronts revealed systematic hemispheric differences. We found that: (i) activation patterns in the left fusiform gyrus correlate with image-level face-semblance, while those in the right correlate with categorical face/non-face judgements. (ii) Context exerts significant excitatory/inhibitory influence in the left, but has limited effect on the right. (iii) Face-selectivity persists in the right even after activity on the left has returned to baseline. These results provide important clues regarding the functional architecture of face processing, suggesting that the left hemisphere is involved in processing ‘low-level’ face semblance, and perhaps is a precursor to categorical ‘deep’ analyses on the right.

1. Introduction

Identifying the nature of asymmetries across the cerebral hemispheres is a key component of understanding functional organization of neural processing. Significant left–right differences have been demonstrated in several domains such as language [1–4], spatial ability [5,6] and neurological disorders [7,8]. Evidence for systematic lateralization in visual processing, however, is quite limited; the posterior to anterior hierarchy appears to be the primary organizing principle [9]. Early visual areas (V1 and V2) in the two sides appear to perform identical functions for the left and right halves of the visual field. Further along the pathway, the distinction between ipsi- and contra-field processing is steadily diminished as neuronal receptive fields in either hemisphere include larger and larger sections of the visual field encompassing both sides of the midline. An implication of this organization is that analogous higher-order visual areas on the left and right are highly redundant. The possibility of such ‘duplicated processing’ is puzzling given that it suggests inefficient neural resource usage. Might there actually be systematic functional differences in processing within late visual areas across the two hemispheres?

Face perception provides an ideal domain in which to frame and examine this question. Previous studies have reported face-selective regions in the fusiform gyri of both the left and right cerebral hemispheres [10–12]. Both of these regions exhibit elevated responses to images of faces relative to non-faces. Qualitatively, therefore, they appear to have similar selectivity properties. However, fusiform activations for faces are often found to be greater in the right than in the left [10,13]. Previous psychophysical investigations with split-brain patients also suggest lateral asymmetry in face processing or encoding [14,15]. It is unclear whether the left and right fusiform gyri process face information in an identical fashion, or have distinct functional roles to process faces at different levels.

Numerous prior studies have measured fusiform activation for faces versus non-faces. However, we argue that the basic face versus non-face contrast might be too coarse a distinction to characterize potentially subtle hemispheric differences in face processing; more precise response functions of these regions are needed. Conceptually, the use of a face versus non-face contrast amounts to sparsely sampling two very different regions in a high-dimensional image space, with one region corresponding to face images and the other to non-faces. Having cortical response values at just these sparse points is insufficient to reliably estimate the form of the response function for a given cortical region. To overcome this problem, we assessed responses of cortical areas in the left and right hemispheres by using a denser, more continuous, sampling from the image space.

Towards this goal, we compiled a novel stimulus set comprising several images that together spanned a range of facial semblance from being very unlike faces to genuine faces. We accomplished this through the use of a computational face-detection algorithm [16]. This detector, on occasion, generates false-alarms—incorrectly declaring a non-face image region to be a face. Typically, this happens when the image region fortuitously has some face-semblance (say, a face-like pattern of light and dark in foliage). We collected 180 such false-alarms. To human observers, some of these images look more like a face than the others. This collection of images was augmented with 60 randomly selected non-face images and 60 genuine faces to yield a stimulus set of 300 images (figure 1).

Figure 1.

Examples of our stimuli. (a) Patterns in the green box were erroneously signalled as faces by a computational face detection system (developed by Pittsburgh Pattern Recognition Systems). Based on human observers' perceptual ratings, these false-alarms were divided into low face-like and high face-like patterns. (b) Sample stimuli that show the range of face-semblance used in the present study. Rows from top to bottom show images with increasing level of similarity, from no face-semblance to genuine faces, as rated by human observers.

This stimulus set provides the foundation for probing the primary question for this study: are brain activations in the left and right hemispheres (specifically the fusiform gyri) redundantly identical or qualitatively different for stimuli along a densely sampled trajectory from non-faces to faces? We adopted a three-pronged approach towards addressing this question.

• — Comparison of activation profiles in the left and right fusiform gyri across the stimulus spectrum.

• — Comparison of response modulation induced by the inclusion of local contextual information in each image of the stimulus set.

• — Comparison of temporal dynamics of responses to the stimulus set in the left and right fusiform gyri.

If the left and right fusiform gyri process faces at different levels (say, the superficial level of image structure and the deeper level of image category), activation patterns in one side should be relatively more stimulus-driven, whereas the other side should be more closely correlated with perceptual judgements. Similarly, one side should be less affected by top-down modulations (e.g. whether contextual information can modulate the activation) than the other side. Finally, if the left and right fusiform gyri process faces serially, temporal dynamics of responses should be sequential rather than simultaneous.

2. Material and methods

(a). Participants

Thirty-six volunteers (aged 17–49 years, 13 females) were recruited from the MIT community to participate in the behavioural experiments. Another six right-handed adults (aged 23–31 years, three females) volunteered for the functional magnetic resonance imaging (fMRI) experiments. All participants had normal or corrected-to-normal visual acuity. All participants gave informed written consent.

(b). Stimuli collection

One hundred and eighty false-alarm images were selected using the Pittsburgh Pattern Recognition face-detection algorithm [16]. Sixty non-face images collected from natural scenes devoid of faces and 60 genuine faces were also added to the stimulus set. Each image was made monochrome and normalized for scale and luminance.

(c). Elo rating

Eighteen observers naive to the purpose of this study were each shown 600 pairs of images and in each case were asked to choose the more face-like of the two images presented. Each image was presented on the screen for 300 ms followed by a 100 ms visual mask. The images were drawn at random from the 300 collected stimuli, such that each image was shown to each observer four times against other images from the stimulus set. Every image was initially assigned a rating of 1000. Following each pairwise comparison, the ratings of both images were updated using the Elo rating algorithm [17],

and

where the subscripts A and B denote the two images being compared, S_A is the outcome of the comparison (1 if A is more face-like and 0 if B is more face-like), and E_A is the expected probability that A would be chosen as more face-like based on prior ratings. Finally, K is a volatility constant that was set to 100 based on simulation results. Based on the ratings, the false-alarm images were divided into a high face-semblance group and a low face-semblance group (referred to as NF_L and NF_H, respectively, in the rest of this paper; the 60 randomly chosen non-faces are called NF₀ and the 60 genuine faces are called F).

(d). Psychophysics classification experiment

Eighteen observers who did not participate in the Elo rating experiment participated in the classification experiment. Each observer was shown 300 images, one at a time. Each image was presented on the screen for 300 ms, after which the observer had to categorize the image as a face or non-face image.

(e). Functional magnetic resonance imaging experiments

Six right-handed adults participated in two sessions of the fMRI experiments (corresponding to two different context conditions as described below) on two separate days. Scanning was performed on a 3.0 T Siemens scanner using a standard head coil at the Martinos Imaging Centre at MIT. A high-resolution T1-weighted 3D-MPRAGE anatomical scan was acquired for each participant (field of view (FOV) 256 × 256, 1 mm³ resolution). To measure BOLD contrast, 33 slices parallel to the anterior and posterior commissure (AC/PC) line were acquired using standard T2*-weighted gradient-echo echoplanar imaging (repetition time (TR) 2000 ms, echo time (TE) 30 ms, flip angle 90°, slice thickness 5 mm, in-plane resolution 3 × 3 mm). Stimuli were rear-projected onto a screen in the scanner bore.

For the ‘without-context’ experiment, 12 scan runs were performed with each participant. Each run began and ended with 12 s of fixation-rest and contained 120 trials of image presentation. Three runs (360 trials) constituted a stimulus presentation cycle, which included 60 no-face-semblance images (NF₀), 90 low face-semblance images (NF_L), 90 high face-semblance images (NF_H), 60 genuine faces and 60 fixation-only trials, all randomly distributed. Each trial was 2 s long and each stimulus image was presented for 300 ms followed by 1700 ms ISI. Four cycles of presenting the stimuli, each with a different random sequence, constituted the 12 runs in total. During the experiments, either the horizontal arms or the vertical arms of the fixation cross randomly changed their length. Participants were instructed to monitor these changes and report accordingly by button presses.

In addition to the above scan runs, two block-design runs were used to functionally localize regions of interest (ROIs). Each of these localizer runs lasted 336 s. In these runs, faces and random non-face images (that were also collected from natural scenes but were not used in the above mentioned experimental runs) were presented in alternating blocks of 16 s each. These blocks were separated by 16 s long fixation periods. Images were presented at a rate of 1 Hz. Each block type was shown five times in a run using different images. A fixation cross was always presented at the centre of the screen. Participants were instructed to perform a change detection task of the fixation cross as described above.

All the six participants came back for the ‘with-context’ experiment on a different day (at least one week later). In this scan session, we used the stimuli with contextual information by including neighbourhoods of the image patches that had been used in the without-context experiment. These ‘context added’ stimuli were presented to observers while all other aspects of the experimental set-up and procedures were the same as in the without-context experiment.

(f). Functional magnetic resonance imaging analysis

All fMRI data underwent three-dimensional motion correction, slice time correction and analysis using SPM2 (http://www.fil.ion.ucl.ac.uk/spm/software/spm2/) and custom-routines in Matlab. Slow drifts in signal intensity were removed by linear detrending, but no other temporal smoothing was applied. ROIs were defined anatomically and functionally. For the anatomically defined ROIs (i.e. calcarine sulcus, fusiform gyrus, parietal and frontal lobes), the Talairach Daemon database was used to generate the ROI masks [18]. The fMRI data of each participant were normalized into Montreal Neurological Institute space for analysis that used these ROIs. Magnetic resonance (MR) signals were averaged across the six participants who each completed four repetitions of each trial that corresponds to each of the 300 images and 60 fixation-only trials. To eliminate the impact of absolute signal magnitude, we used per cent MR signal change to perform further analyses. MR activity at the stimulus onset was used as a baseline to calculate the per cent signal change. As reflected by the per cent haemodynamic signal changes, in total, we had brain activation patterns corresponding to the 300 face/non-face images and 60 fixation-only trials.

To gain better statistical sensitivity, we used multivariate pattern analysis (MVPA) [19] rather than the univariate mean-of-ROI analysis. To minimize any biases of region selection in our analyses, we included the patterns of visually driven voxels across the entire fusiform gyri in both the left and right hemispheres. Specifically, voxels were selected within the left and right anatomically defined fusiform gyri, with the criteria that a voxel only had to have significantly different fMRI activity between the face trials and the fixation-only trials or between the no face-semblance (NF₀) trials and the fixation-only trials (two-tailed t-test, p < 0.05, not corrected). The selected voxels comprised large, roughly equal-sized areas (approx. 16 cm³) in the left and right fusiforms. Note that using more stringent criteria that yielded much smaller (approx. 4 cm³) ROIs did not change our results. This is consistent with the notion that choosing only the maximally selective voxels is not necessary for MVPA [19].

In addition to the anatomically defined ROIs, we also functionally localized fusiform face area (FFA), occipital face area (OFA) and lateral occipital complex (LOC) for each participant. For analysis that used these ROIs, no spatial normalization or smoothing was applied. Activation pattern correlations or magnitudes were calculated across the functionally defined ROIs within each participant and then were averaged across participants. Specifically, to secure an adequate number of voxels to perform the pattern correlation analysis, there were two steps to localize the functionally defined ROIs. (i) General linear model (GLM) analyses were used to contrast face-evoked activation and non-face-evoked activation during the localizer scans. Voxels with peak face-evoked activation greater than non-face-evoked activation were localized in the fusiform gyri and the inferior occipital gyri. (ii) Using these voxels as centres, within a 15 mm radius, significant face-selective voxels (when compared with the fixation period, p < 10⁻⁴) were then localized as FFAs and OFAs, respectively. Similarly, voxels with peak non-face-evoked activation greater than face-evoked activation in the lateral occipital–temporal regions were used as centres to localize the LOC. However, we did not find voxels with non-face-evoked activation significantly greater than face-evoked activation in these regions for one participant. In this case, voxels with peak non-face-evoked activation greater than fixation-evoked activation in these regions were used. On average, there were 71 voxels in the right FFA, 65 voxels in the left FFA, 66 voxels in the right OFA, 73 voxels in the left OFA, 72 voxels in the right LOC and 67 voxels in the left LOC.

Finally, to calculate the activation pattern correlations, within each ROI (both anatomically and functionally defined), fMRI activity of the selected voxels was regressed to the face-evoked fMRI activity. The fMRI activity patterns corresponding to non-face images were compared with those evoked by the 60 face images. Each face-evoked pattern was compared with the other 59 face-evoked patterns excluding itself. After averaging across the 60 or 59 correlation scores corresponding to the 60 or 59 face-evoked activation patterns, there were 300 averaged correlation scores corresponding to the 300 stimuli (60 NF₀, 90 NF_L, 90 NF_H and 60 faces). Further statistical analyses were performed based on these averaged correlation scores.

3. Results

(a). Behavioural results

The Elo rating results of face-semblance are shown in figure 2a. To further validate these perceptually based face-similarity ratings, we employed two popular computational image similarity metrics to compare these 300 images with 80 other face images and 80 other random non-face images. Specifically, we used image-correlation and Gabor-jet similarity metric [20]. Linear regressions showed that both of these computational methods concurred with the observers' face-similarity judgements as quantified by Elo ratings (F > 10, p < 0.001). It is worth pointing out that our choice of the Elo rating system is not a critical aspect of the study. Alternatives like Likert rating scales are also valid. Indeed, in separate studies, we have found that ratings derived by the two approaches on our stimulus set are highly correlated. By contrast, when asked to categorize the 300 images as ‘faces’ or ‘non-faces’, data from 18 observers (who did not participate in the first experiment) show a sharp boundary between non-faces and faces (figure 2b).

Figure 2.

(a,b) Behavioural and (c,d) neural responses to our stimulus set. As reflected by Elo rating, face likeness steadily increases from random non-face, low face-like false-alarms, high face-like false-alarms, to genuine faces (a). In contrast, when asked to make face/non-face judgement, sharp boundary between non-faces and faces was evident. No matter how face-like a non-face image might be, false face report by human observers remained low (b). (c,d) Averaged voxel-by-voxel correlation scores when compared with brain activation patterns corresponding to faces. Error bars represent ±1 s.e.m. At 4 s after the stimulus onset, the correlation score of the left fusiform gyrus (c) increases according to image-level face-similarity change, whereas in the right fusiform gyrus (d), activation pattern correlation reflects perceived face categorization, echoing figure 2b.

(b). Activation in the left and right fusiform gyri across the stimulus spectrum

To quantitatively examine the response profiles of the left and right fusiform gyri, fMRI activity patterns corresponding to non-face images were compared with those evoked by the 60 face images by calculating correlation scores of voxel-by-voxel activation. Each face-evoked pattern was compared with the other 59 face-evoked patterns excluding itself. The averaged correlation scores are plotted in figure 2c,d. A repeated-measures ANOVA reveals that the main effect of face-semblance level is significant, F_3,177 = 12.46, p < 0.0001; the main effect of hemisphere is not significant, F_1,59 = 2.38, p = 0.128. Most interestingly, the interaction between hemisphere and the level of face-semblance is significant, F_3,177 = 3.06, p < 0.05. Note that NF_L and NF_H are the two critical conditions for our study. A planned analysis to compare these two conditions reveals that, in the left fusiform gyrus, the correlation score is significantly higher for NF_H than for NF_L (one tailed t-test, t₁₇₈ = 2.47, p < 0.01), suggesting that high face-semblance images lead to more face-like left fusiform brain activation than do the low face-semblance images. By contrast, the correlation to face-induced brain activation in the right fusiform gyrus is not significantly different between the low and high face-semblance images (one tailed t-test, t₁₇₈ = 0.18, p = 0.43).

For the 180 face-like false-alarms, further linear regression showed that changes of activation patterns in the left fusiform follow the Elo rating of face-semblance of each stimulus (T = 3.45, p < 0.001). By contrast, activation in the right fusiform was independent of face-semblance for all the false-alarm stimuli (T = 0.59, p = 0.56). A comparison of correlated correlation coefficients [21] revealed that the left fusiform gyrus was significantly more than the right fusiform gyrus was correlated with the Elo rating of face-semblance (p < 0.01). Overall, the response profile of activation patterns in the left fusiform (figure 2c) resembles the graded change of the face-similarity ratings (figure 2a). On the other hand, activation patterns in the right fusiform (figure 2d) are consistent with categorical face/non-face judgements (figure 2b), suggesting a role in representing the perceived image category rather than image-level face-semblance. It is worth pointing out that simply comparing responses to F and NF₀ patterns, as has typically been done in the past, would have been insufficient to reveal the graded versus categorical differences between the left and right fusiform gyri.

Several studies have demonstrated an elevated average response in the ‘fusiform face area’ (FFA) to faces relative to non-face objects [10–12]. The right FFA is typically found to exhibit a more robust response than the left FFA [10,13]. An important question to address is whether our current findings can be accounted for simply by the differences in the magnitude of the activity in the FFA. To this end, in separate reference scans, we localized FFA for each participant as the regions of interest (ROIs; see §2). We found that average response magnitudes in the left and right FFAs are statistically indistinguishable (figure 3a). This suggests that our finding of lateral difference in facial processing cannot simply be driven by relative response magnitudes in left and right FFAs. Interestingly, even though the average response magnitudes do not reveal hemispheric processing differences, pattern correlation data do. We calculated pattern correlation of non-normalized brain activation data for the left and right FFA separately for each participant. Linear regression for the 180 false-alarm stimuli (NF_L and NF_H) showed that changes of the pattern of the left FFA activation followed the Elo rating of face-semblance (T = 3.88, p < 0.001), whereas this effect in the right FFA failed to reach significance (T = 1.58, p = 0.12). Again, the left FFA was significantly more than the right FFA was correlated with the Elo rating of face-semblance (p < 0.05).

Figure 3.

(a) Averaged magnitude of responses in left and right fusiform face areas. Univariate analyses of activations in FFA do not reveal categorical selectivity on either the left or the right. (b,c) Pattern correlations in left and right OFA and LOC, respectively. Error bars represent ±1 s.e.m. (a) Blue diamonds with solid line, left FFA; red squares with solid line, right FFA. (b) Blue diamonds with solid line, left OFA, red squares with solid line, right OFA. (c) Blue diamonds with solid line, left LOC; red squares with solid line, right LOC.

Taken together, these results reveal the differential sensitivities of the left and right fusiform regions to face-semblance on the one hand, and face category on the other. Methodologically, the results demonstrate that while a univariate conventional mean-of-ROI analysis might be adequate for separating neural responses to distinct face versus non-face stimuli, the multivariate distributed pattern analysis can be used to more precisely identify neural responses corresponding to categorical face perception. This enhanced sensitivity can be instrumental for detecting differences across populations (e.g. neurotypical versus patient and adult versus pediatric) that might otherwise be difficult to discern using conventional univariate approaches.

To examine whether activity in other cortical areas besides the right fusiform gyrus might also serve as correlates of categorical face perception, we conducted the same pattern correlation analysis in ROIs that have previously been implicated as having an important role in high-level vision. Several studies have suggested that the occipital face area (OFA) in inferior occipital gyri may be involved in face processing [22,23]. Our results are consistent with this notion because the activation pattern in OFA is modulated by faceness of the stimuli (figure 3b). However, the correlations are weaker relative to those in the fusiform gyrus, and we observe no systematic differences between the left and the right OFAs. The LOC constitutes another ROI since it has been reported to be selective to objects [24]. As in the OFA, pattern activation data in the LOC (figure 3c) show a weak correlation across the range of face-semblances and no categorical trends are evident.

To control for the possibility that the observed graded or categorical responses might be artefacts of some low-level properties of the stimulus set used, we also examined responses in anatomically defined primary visual cortex in the occipital lobe and other visually driven areas in the parietal and frontal lobes. We found that unlike the fusiform gyri, activation patterns in the calcarine sulcus and in parietal and frontal lobes do not differentiate across the face/non-face conditions. This suggests that the systematic response patterns observed in the two fusiform gyri (figure 2c,d) are unlikely to be artefactual, driven by some inadvertent regularity in the stimulus set.

(c). Response modulation induced by the inclusion of local contextual information

Contextual information often facilitates perceptual categorization by disambiguating and organizing visual inputs. Previous neuroimaging work has shown that face context alone (i.e. without an actual face) can yield activations in the fusiform gyrus that are comparable to those elicited by actual faces [25]. Here, we tested whether contextual information can modulate the qualitatively different brain activation patterns in the left and right fusiform gyri that are evoked by our stimuli. The 300 images stimulus set described above was augmented with contextual information by including neighbourhoods of the image patches (figure 4a). These ‘context added’ stimuli were presented to observers while we measured their brain activity using fMRI. All other aspects of the experimental set-up and procedures were the same as in the without-context study. Results are shown in figure 4b. In the left fusiform gyrus, contextual information has the effect of reducing pattern correlations to false-alarm images and greatly enhancing those corresponding to genuine faces. This combined reduction and enhancement has the net consequence of transforming graded responses in the left fusiform to categorical ones. In contrast, context-driven modulation on the right is modest with no significant change in the response to false-alarm images and only a small increase for the genuine faces.¹

Figure 4.

(a) A few examples of context-inclusive stimuli. The dashed rectangles indicate the extent of stimuli used in experiment 1. These dashed rectangles are included here for expositional clarity. They were not shown during the actual experiments. (b,c) Pattern correlations in left and right fusiform gyri, respectively, for the with-context stimuli (solid lines) and without-context stimuli (dashed lines). Error bars represent ±1 s.e.m. (b,c) Solid line, with context; dotted line, without context.

(d). Temporal dynamics of responses to the stimulus continuum in the left and right fusiform gyri

The voxel activation values used in the analyses described above correspond to a post-stimulus latency of 4 s. Consistent with the haemodynamic lag of fMRI time course, the average response magnitude is expected to peak at 4 s in the fusiform gyrus. With a rapid event-related design of the kind we have employed, we can probe how the activation pattern changes as a function of post-stimulus time. Figure 5 shows the time-course of changes in correlation scores of the fMRI activation patterns across the left and right fusiform gyri from 2 to 8 s after the stimulus onset. The effect of faceness in the left is noticeable at 2 s, most pronounced at 4 s and weakened by 6 s after the stimulus onset. By contrast, in the right, no trend of the face-induced activation patterns can be observed at 2 s. The effect of faceness is equally large at 4 and 6 s after the stimulus onset. Most interestingly, as long as 8 s after the stimulus onset, face-selectivity persists in the right, whereas activity on the left has returned to baseline. This asymmetry is reliably observed in two experiments for both with and without context stimuli. Table 1 shows planned analyses (one tailed t-tests) to compare face and non-face (NF₀) conditions across the 2–8 s after the stimulus onset in the left and right fusiform gyri, and in both the without and with-context experiments. These results suggest a long-lasting neural activity in the right fusiform gyrus bear interesting similarities to behavioural findings from past studies of split-brain patients that indicate that the right hemisphere is involved in deeper encoding of faces [15,26].

Figure 5.

Pattern correlations across the two fusiform gyri as a function of time post-stimulus. The four panels in the upper row denote responses to each of the stimulus sets at four time points in the left fusiform gyrus. The lower row shows corresponding responses in the right fusiform gyrus at the same time points. Error bars represent ±1 s.e.m. Solid traces represent responses to stimuli with local context. Dashed lines correspond to responses to without-context stimuli.

Table 1.

Results of one-tailed t-tests to compare face and non-face (NF₀) conditions across the 2–8 s after the stimulus onset in the left and right fusiform gyri, and in both the without and with-context experiments.

time after the stimulus onset (s)		2	4	6	8
without context	left fusiform	T = 2.86	T = 4.94	T = 1.42	T = 0.985
		p < 0.01	p < 10−5	p = 0.08	p = 0.16
	right fusiform	T = −0.636	T = 4.67	T = 2.82	T = 1.58
		p = 0.26	p < 10−5	p < 0.01	p = 0.06
with context	left fusiform	T = 2.66	T = 7.27	T = 3.34	T = −4.85
		p < 0.01	p < 10−10	p < 0.001	p = 0.31
	right fusiform	T = −0.411	T = 7.00	T = 5.65	T = 2.09
		p = 0.34	p < 10−10	p < 10−7	p < 0.05

4. Discussion

Our results reveal multiple systematic differences in face responses elicited in the left and right fusiform gyri. First, the results demonstrate that neural activity patterns in the right fusiform gyrus change in a manner consistent with behavioural face/non-face categorical judgements. In contrast, brain activation pattern in the left fusiform gyrus appears to correspond to image-level face similarity. This graded versus categorical response distinction between left and right fusiform gyri clarifies and extends past results that had noted lateral differences in face processing [13,14,27,28]. Interestingly, this hemispheric difference was not revealed in our univariate amplitude analysis. Although the neurophysiological basis of the increased correlation for the face condition is unknown, one possibility is that population coding of faces in these ROIs defines a tighter image sub-space than arbitrary non-faces. The greater self-similarity across activation patterns corresponding to exemplars of the face-set relative to that within the much more heterogeneous conceptual class of ‘non-faces’ would be expected to yield a higher correlation score for the former than the latter.

Second, the results reveal very different susceptibilities in left and right fusiform gyri to contextual modulation. From a theoretical standpoint, models of visual categorization typically have two conceptually distinct stages, one that corresponds to graded evidence accumulation and another that embodies a categorical decision function [29,30]. Our results suggest that the left and right fusiform gyri might approximate these conceptual divisions of a categorization system. The findings of differential contextual modulation and temporal dynamics on the two sides support this conjecture. Response enhancement via evidence accumulation would predict significant influence of context on the left, while the application of a decision function would be expected to mitigate such influences on the right.

Third, responses on the two sides exhibit different temporal dynamics. Interestingly, for both with- and without-context conditions, the categorical responses in the right fusiform persist at least until 8 s after the stimulus onset even though the stimuli are much shorter in duration (each stimulus image was presented for 300 ms followed by 1700 ms ISI). In contrast, activity in the left fusiform gyrus returns to baseline by 8 s after the stimulus onset. In general, responses in the left appear to have an earlier onset and dissolution relative to those on the right. Although not definitive, this evidence is consistent with the notion that the left hemisphere might be involved in rapid processing of face features, whereas the right hemisphere might be involved with ‘deep’ cognitive processing of faces [15,26].

Taken together, our results demonstrate important lateral differences of face processing in the human brain, complementing the presumed hierarchical organization of visual pathways. It remains an open question whether the graded analyses of the left fusiform gyrus and the categorical analyses of the right fusiform gyrus operate in parallel or whether there are functional dependencies between them. This issue can potentially be addressed by conducting the present study with individuals who have recently had corpus callosotomy or via simultaneous single unit recordings that can permit an accurate assessment of response latencies in the two hemispheres. Given this evidence of distinct styles of face processing on the two sides of the brain, it is natural to ask what the genesis of this distinction is. From a developmental perspective, when does this distinction become evident and what causes it to arise? Addressing this question is important for understanding how face processing matures in the course of normal development as well as the kinds of deviations that might occur when the typical developmental trajectory is disrupted by factors such as visual insults [31,32] or neurological disorders [33]. By revealing the functional lateralization of analyses driven by bottom-up image attributes versus by the perceived category, our approach and results further our understanding of, and our ability to probe, the brain mechanisms by which we organize the visual environment into an orderly meaningful world of distinct object classes.