Neural Decoding Reveals Impaired Face Configural Processing in the Right Fusiform Face Area of Individuals with Developmental Prosopagnosia

Abstract

Most of human daily social interactions rely on the ability to successfully recognize faces. Yet ∼2% of the human population suffers from face blindness without any acquired brain damage [this is also known as developmental prosopagnosia (DP) or congenital prosopagnosia]). Despite the presence of severe behavioral face recognition deficits, surprisingly, a majority of DP individuals exhibit normal face selectivity in the right fusiform face area (FFA), a key brain region involved in face configural processing. This finding, together with evidence showing impairments downstream from the right FFA in DP individuals, has led some to argue that perhaps the right FFA is largely intact in DP individuals. Using fMRI multivoxel pattern analysis, here we report the discovery of a neural impairment in the right FFA of DP individuals that may play a critical role in mediating their face-processing deficits. In seven individuals with DP, we discovered that, despite the right FFA's preference for faces and it showing decoding for the different face parts, it exhibited impaired face configural decoding and did not contain distinct neural response patterns for the intact and the scrambled face configurations. This abnormality was not present throughout the ventral visual cortex, as normal neural decoding was found in an adjacent object-processing region. To our knowledge, this is the first direct neural evidence showing impaired face configural processing in the right FFA in individuals with DP. The discovery of this neural impairment provides a new clue to our understanding of the neural basis of DP.

Introduction

The high efficiency associated with human face processing has been attributed to the encoding of face configural information. This includes the representation of the T-shape arrangement of the face parts, and within the T-shape arrangement, better representations of face parts and the spacing between parts (Young et al., 1987; Tanaka and Farah, 1993; Maurer et al., 2002; Yovel et al., 2005). Among the multiple occipital and temporal brain regions exhibiting face selectivity in the human brain, the right fusiform face area (FFA; Kanwisher et al., 1997) is believed to play a central role in face configural processing (Schiltz and Rossion, 2006; Liu et al., 2010; Parvizi et al., 2012; Zhang et al., 2012).

In developmental prosopagnosia individuals (DPs), although behavioral deficits in face part processing (Le Grand et al., 2006; Yovel and Duchaine, 2006) and face pigmentation perception (Russell et al., 2012) have been noted, many studies also reported a deficit in face configural processing. Specifically, behavioral testing has revealed that DPs cannot accurately represent the spacing between face parts or integrate face parts to form holistic face representations (Yovel and Duchaine, 2006; Zhu et al., 2009; Avidan et al., 2011; DeGutis et al., 2012). This implies that DPs may not have a properly developed right FFA capable of supporting face configural processing. Paradoxically, however, in many DPs, the right FFA can be reliably localized with fMRI and shows face selectivity and normal fMRI response adaptation to face identity repetitions (Furl et al., 2011; Avidan et al., 2014). To explain this puzzling finding, some have argued that the impairments seen in DPs are based on the relatively fewer number of voxels, lower face selectivity in the peak voxel, and/or reduced gray matter volume in their right FFAs (Bentin et al., 1999; Behrmann et al., 2007; Garrido et al., 2009; Furl et al., 2011). However, as these abnormalities were not observed consistently across DPs (Avidan et al., 2014), based on impaired functional connectivity among the face processing regions (Thomas et al., 2009; Zhu et al., 2011), others have argued instead that perhaps DPs have an intact right FFA, but an impaired information transmission between the right FFA and other face process regions (Avidan et al., 2014). Critically, although behavioral testing has linked face-processing deficits in DPs to deficits in face configural processing, fMRI investigations of DP have not directly examined this in the right FFA, leaving it unknown whether face configural processing is indeed normal or impaired in DPs' right FFA.

Using fMRI multivoxel pattern analysis (MVPA), a tool that has been widely used in recent fMRI investigations of information representation in the brain (Norman et al., 2006), here we discovered that in DPs, despite the right FFA's preference for faces and it showing differential response patterns to the different face parts, it failed to form distinct neural representations for the intact and the scrambled face configurations. We propose that this face configural processing deficit in the right FFA may play a central role in mediating behavioral face processing deficits in DP.

Materials and Methods

Observers.

In the main experiment, seven right-handed paid DPs (2 females; mean age 27.3 ± 1.4) were tested. One additional DP was tested but excluded from further analysis due to excessive head movements during the scan (>3 mm).

These DPs were characterized in a previously published study and identified through a three-step test among the graduate students in the Chinese Academy of Sciences, Beijing (Zhu et al., 2009). In Step 1, observers self-reported face recognition problems in everyday visual perception through a 21-item questionnaire. In Step 2, observers participated in a 1 h semistructured interview similar to the one developed by Kennerknecht et al. (2007). This interview helped us determine the extent of an observer's face recognition difficulty, whether it started in childhood, and whether the observer had any neurological or psychiatric impairment. In Step 3, the final step, each observer completed a Chinese version of the famous face test (Duchaine and Nakayama, 2005) in which they were asked to identify 30 famous Chinese faces. Correct responses included correctly reporting either the names of the faces or information relevant to the faces, such as professional achievements (for politicians and sports stars) or the movies the individual appeared in (for actors). After the test, observers were shown names of the famous faces used and reported their familiarity with those famous individuals (e.g., whether they recognized the names or whether they knew what these famous people look like). Observers whose low performance in the famous face test was due to their unfamiliarity with the famous people would not be considered as having DP and were excluded from further consideration. For individuals without face recognition difficulties from the same student population, the mean accuracy for our famous face test was 89 ± 12%. Each of the DPs in the present study had a score that was <1.65 SD (95%) of this mean (Table 1). We were able to identify the right FFA in each of the DPs. The size of the right FFA in each DP exceeded an area of ten 3 × 3 × 3 mm voxels.

Table 1.

Demographic information and behavioral performance accuracy

Participant	Gender	Age	Famous faces task
			Accuracy, %	Z-score
DP1	M	29	33.4	−4.48
DP2	M	28	66.7	−1.77
DP3	M	28	36.7	−4.21
DP4	F	28	16.7	−5.83
DP5	M	27	26.7	−5.02
DP6	M	26	43.4	−3.66
DP7	F	25	40.1	−3.94
DP8	M	28	46.7	−3.40
DP mean (SD)	6M/2F	27.4 (1.4)	38.8 (14.8)	−4.04 (1.20)
Control mean (SD)	6M/6F	25.5 (3.8)	91.6 (14.2)	0.25 (1.15)

DP1–DP7 participated in the main experiment; DP1, DP4, DP5, and DP8 participated in the follow-up DP experiment. M, Male; F, female; SD, standard deviation.

In addition to DPs, in the main experiment, 21 right-handed, paid controls were also recruited from Beijing Normal University. Of those, 12 (6 females) were included in the main study based on the following two criteria: (1) observer's head movements were <3 mm during the scan, and (2) observer's right FFA could be reliably identified with more than ten 3 × 3 × 3 mm voxels.

In the follow-up DP experiment, four right-handed, paid DPs (1 female; mean age 28.0 ± 0.8) from the same pool were tested. Three of these DPs also participated in the main experiment.

In the follow-up control experiment, six right-handed, paid Chinese control observers were recruited from Harvard University (2 females). The same selection criteria used in the main experiment were applied here.

Informed consent was obtained from all the observers before the experiments. The experiments were approved by the Institutional Review Boards of Beijing Normal University (the main experiment and the follow-up DP experiment) and Harvard University Committee on the Use of Human Subjects (the follow-up control experiment).

Stimuli.

In the main experiment, to avoid the race-effect in face perception (Tanaka et al., 2004), 20 East Asian male faces were used as stimuli with the external contour of each face image removed. Each face image was then gray-scaled and divided into two parts. The upper face part contained the eyes, and the lower face part contained the nose and mouth. The two face parts were presented together either in the intact face configuration with the eyes above the nose and mouth, or in a scrambled configuration with the eyes below the nose and mouth (see the examples shown in Fig. 1A). Each face part was also presented alone at the same location where it appeared in the intact or the scrambled faces (Fig. 1A). To make the part transition within each image similar and smooth, we added a black bar between the face parts in both the intact and the scrambled conditions. The upper and the lower face parts subtended 4.02° × 12.10° and 8.08° × 12.10°, respectively, and the intact and the scrambled face images both subtended 12.10° × 12.10°.

Figure 1.

An illustration of the face stimuli used in the experiments. A, The stimuli used in the main experiment. These stimuli were created by gray-scaling male East Asian faces and dividing each face into two parts, with the upper part containing the eyes and the lower part containing the nose and the mouth. The two face parts were presented together either in the intact face configuration (the eyes above the nose and the mouth), or in a scrambled configuration (the eyes below the nose and the mouth). Each face part was also presented alone at the same location where it appeared in the intact and the scrambled faces. To make the transition between the two face parts similar and smooth in the intact and scrambled images, a black bar was added between the face parts in both conditions. B, The phase-scrambled face parts used in the follow-up control experiment. These were created by phase scrambling the face parts used in the main experiment to remove any recognizable face features but retain the same low-level visual features present in the intact face parts (such as size, luminance, and spatial frequency distribution profiles).

In the follow-up DP experiment, Caucasian male faces, instead of East Asian male faces, were used. For three of the DP observers, the design of the experiment was identical to that of the main experiment. For one DP observer, when face parts were shown alone, they were all presented at fixation. This prevented us from examining location decoding of the face parts across all the observers. However, because the main experiment showed that location decoding was absent in the right FFA in both DPs and controls, this follow-up experiment still provided us with an opportunity to replicate the results from the main experiment on the decoding of face configurations and face parts in the right FFA.

In the follow-up control experiment, phase-scrambled face parts were presented alone to examine the contribution of low-level feature differences to face part decoding in the intact part conditions in the main experiment. These stimuli were created by phase scrambling the face parts used in the main experiment to remove any recognizable face features but retain the same low-level visual features present in the intact face parts (such as size, luminance, and spatial frequency distribution profile; Fig. 1B). During the experiment, these phase-scrambled parts were presented in the same way as their corresponding intact parts were in the main experiment.

In the localizer experiment, images of faces, indoor and outdoor scenes, everyday objects (e.g., chairs, food, and tools), and scrambled everyday objects were shown. The scrambled objects were created by phase scrambling the object images. Each image subtended 13.5° × 13.5°.

Experimental design and procedures.

Each observer completed a single scan session consisting of 10 experimental runs and two functional localizer runs. Observers maintained central fixation throughout the study with their eye movements monitored by an eye-tracker (EyeLink 1000, SR Research).

In the main experiment, each experimental run lasted 2 min 32 s and contained one block each of the six stimulus conditions (i.e., the intact face, the scrambled face, and 4 single part conditions; Fig. 1A). Each stimulus block lasted 16 s and contained 20 unique images of the same type. Each image was presented for 600 ms at fixation and followed by a 200 ms blank interval. There was an 8 s blank fixation block between each stimulus block. To engage observers' attention on the stimuli, four randomly selected images in each block moved slightly either to the left or right during stimulus presentation. Observers were asked to judge the direction of these movements with key presses. For the intact and the scrambled images, the movement could occur in either face part, encouraging observers to attend to both parts equally.

In the follow-up DP experiment, for three of the DP observers, the experiment design was identical to that of the main experiment. For the one DP observer for whom the individual face parts were shown at fixation, eight instead of 10 experimental runs were collected.

In the follow-up control experiment, each experimental run contained four phase-scrambled face part blocks and lasted 1 min and 44 s.

All observers completed the same localizer experiment. Each localizer run lasted 5 min 4 s, and included sixteen 16 s blocks interleaved by three 16 s fixation blocks inserted at the beginning, middle, and end of the run. Each block contained 20 unique exemplars from a given object category, each presented for 750 ms at fixation and followed by a 50 ms blank interval. To engage observers' attention on the stimuli, they were asked to detect a slight spatial jitter of the image, occurring randomly twice in every block.

fMRI scanning.

Observers in the main experiment and the follow-up DP experiment were tested with a 3T Siemens Trio scanner equipped with an 8-channel phase-arrayed head coil at the Beijing Normal University Imaging Center for Brain Research, Beijing, China. Anatomical images were acquired with MPRAGE, an inversion prepared gradient echo sequence (TR/TE/TI = 2.73 s/3.44 ms/1 s, flip angle = 7°, voxel size 1.1 × 1.1 × 1.9 mm). Functional images were acquired with a T2*-weighted gradient-echo, echo-planar imaging sequence (TR/TE = 2 s/32 ms, flip angle = 90°). Thirty near axial slices (3 mm thick with 20% skip and 3 × 3 mm in-plane resolution), oriented parallel to the temporal cortex and covering the whole brain, were collected.

Observers in the follow-up control experiment were tested at the Harvard University Center for Brain Science with a 3T Siemens scanner of the same model and a head coil of the same type as those in the main experiment. Functional images were also acquired with the same resolution and TR as those in the main experiment.

Data analysis.

Functional data were analyzed with Freesurfer (Cortechs), fROI (http://froi.sourceforge.net), CLOP (Challenge Learning Object Package; http://clopinet.com/CLOP/), and in-house MATLAB codes. Data preprocessing included motion correction and intensity normalization. For the localizer data, spatial smoothing was also applied (Gaussian kernel, 5 mm full width at half maximum). For both the experimental and the localizer runs, to obtain response amplitudes for each voxel in each stimulus condition for each observer, voxel time courses were fitted by a general linear model with each condition modeled by a separate boxcar regressor (matched in stimulus duration) and then convolved with a gamma function (delta = 2.25, tau = 1.25). The resulting β weights were used to characterize the response amplitudes of the different stimulus conditions.

For the localizer data, standard contrasts were used to identify the ROIs in the right hemisphere of each observer. Specifically, the right FFA was defined as the set of contiguous voxels in the right fusiform gyrus (in the volume view) that showed significantly higher responses to faces than to nonface objects. In three observers, two clusters of face selective voxels were found in the right fusiform gyrus and both were included as the right FFA ROI. The right object selective region was defined as the set of contiguous voxels in the right lateral occipital (LO) region that showed higher responses to everyday objects than to scrambled objects.

In an effort to obtain sufficient voxels from each observer to perform the MVPA analyses, we used a more relaxed statistical threshold of p < 0.01 (uncorrected) to define the right FFA instead of the standard threshold of p < 0.001. This resulted in the right FFA voxel number ranging from 11 to 45 for DPs (mean = 21, SD = 13), and from 16 to 67 in controls (mean = 33, SD = 14). Despite this relaxed threshold, the right FFA still showed a high degree of face selectivity in amplitude measures (see Results). Moreover, very similar decoding results were obtained when we redefined the right FFA using the standard threshold (the right FFA voxel number ranged from 7 to 32 for DPs: mean = 17, SD = 11; and from 11 to 53 in controls: mean = 26, SD = 12). However, as <10 right FFA voxels were obtained in some of the observers, it was not clear whether the decoding results obtained would be reliable. In an effort to match the number of voxels between the right LO and the right FFA to produce valid comparisons between these two brain regions, thresholds used to define the right LO varied across observers but were at p < 0.01 or lower.

For all the experimental data, a support vector machine (SVM), a linear classifier, was used to compare the fMRI response patterns evoked by the different stimulus conditions in the predefined ROIs. For each observer, we first extracted, for each voxel in each ROI, the averaged response amplitude in each condition for each run. Using SVM and response patterns from N − 1 of the total N runs, we trained a linear discriminant function to distinguish the fMRI response patterns between two stimulus conditions. The discriminant function could be expressed as follows: g(x) = w_ix_i + w_o, where x_i was the response amplitude of voxel i, w_i was the weight of that voxel, and w_o was the overall bias. After training, response patterns from the Nth run were used to test the classification performance of the discriminant function. Based on the output (either g(x) > 0 or g(x) < 0), the test data were assigned to one of the two stimulus conditions. We performed an N-fold cross-validation by repeating the above procedure N times with each of the N runs serving as the test run and the remaining runs as the training runs. Classification accuracies were averaged across all the training testing procedures for each observer and then averaged across observers (Kamitani and Tong, 2005).

Results

In this study, we examined face configural processing in the right FFA of DPs and controls from Beijing, China. Our DPs all had at least a college education and were identified through a three-step test, including a self-report face recognition questionnaire, a semistructured interview, and the Chinese version of the famous face test (see Materials and Methods; Table 1; Garrido et al., 2009; Zhu et al., 2009; Towler et al., 2012).

Univariate face selectivity measures

In the main experiment, following well established procedures (see Materials and Methods), we showed observers images from different object categories, including faces, scenes, and objects, and successfully identified the right FFA in all seven DPs and 12 controls included in the main study (Fig. 2). To examine face selectivity in the right FFA we used the fMRI response amplitude measures from the localizer. To avoid nonindependence errors, we split the localizer data in half and used half of the data to localize the right FFA and the other half to examine face selectivity (Fig. 3). Replicating previous DP findings (Furl et al., 2011; Avidan et al., 2014), both DPs and controls exhibited strong face selectivity in the right FFA, showing higher responses to faces than objects from other categories (all t values >6.95, p values <0.001) with no interaction between stimulus condition and observer group (F < 1). Thus, with univariate fMRI response amplitude measures, DPs and controls did not differ in face selectivity in the right FFA.

Figure 2.

Face selective brain regions (faces > objects) in the right hemisphere of all seven DPs and all 12 controls shown on inflated brain surfaces. The dashed outline indicates the location of the right FFA on each brain. Activation maps were marked for the eight controls whose right FFA size matched that of DPs and whose data were included in the main MVPA analysis. RH, Right hemisphere; L, lateral; P, posterior; Controls, observers without developmental prosopagnosia.

Figure 3.

Mean fMRI response amplitudes in the right FFA for the different object categories. Both DPs (left graph) and controls (middle and right graphs) showed equally strong face selectivity. The results were nearly identical whether all 12 (middle graph) or only the eight controls with a matching right FFA size were included (right graph). Error bars indicate SEM. Controls, Observers without developmental prosopagnosia.

We noted that, on average, the size of the right FFA was larger in controls than in DPs (t₍₁₇₎ = 1.84, p = 0.08), consistent with what others have reported previously (Behrmann et al., 2007; Garrido et al., 2009; Furl et al., 2011). Because a smaller ROI size can negatively impact MVPA results, to examine whether DPs and controls differed in face configural processing in the right FFA, in any way beyond a size difference, we ranked controls in descending order according to the size of their right FFA and only included the eight observers with the smallest ROI in all further analyses (Fig. 2). This resulted in no difference in right FFA size between the two groups (t₍₁₃₎ = 0.90, p > 0.38). We then repeated our amplitude analysis using only these eight controls and found strong face selectivity in the right FFA (t values >6.46, p values <0.001) that did not differ from when all controls were included or from that of DPs (F values <1; Fig. 3).

MVPA face configuration, part, and location decoding

In behavioral studies, scrambling the relationship among the face parts has been shown to obstruct face configural processing (Tanaka and Farah, 1993; Zhu et al., 2009). Because of the importance of the T-shape arrangement in mediating face configural processing (Maurer et al., 2002) and the involvement of the right FFA in such configural processing (Schiltz and Rossion, 2006; Liu et al., 2010; Parvizi et al., 2012; Zhang et al., 2012), here we examined whether the right FFA in DPs would show the same sensitivity to the T-shape arrangement of the face parts. Specifically, we used 20 unique gray-scaled East Asian male face images and segmented each image horizontally into two parts with the upper part containing the two eyes, and the lower part containing the nose and the mouth. The two face parts were presented together either in the original intact configuration or in a scrambled configuration (Fig. 1A). Each face part was also presented alone at either the upper or the lower location to match the locations at which they appeared in the intact or the scrambled face images (Fig. 1A). During the experiment, different types of face images were shown in different trial blocks and each block contained a sequential presentation of 20 unique images of the same type (e.g., 20 different intact faces or 20 different upper face parts shown above the fixation). Observers maintained central fixation while viewing the face images. To ensure that their attention was on the face stimuli, observers were asked to report the direction of motion when the stimuli moved occasionally.

Using MVPA, we found that, in controls, the intact and the scrambled faces elicited distinctive neural response patterns in the right FFA and produced above chance neural decoding (t₍₇₎ = 5.02, p < 0.01). This indicated that controls were sensitive to the presence of the T-shape arrangement of the face parts. Interestingly, such sensitivity was absent in DPs as neural decoding of the two face configurations was at the chance level (t₍₆₎ = 1.37, p > 0.21; Fig. 4A, left). Critically, face configural decoding performance differed significantly between these two groups of observers (t₍₁₃₎ = 2.84, p = 0.01). Thus, in addition to a difference in the right FFA size, which was marginally significant, our MVPA analysis revealed, for the first time, that DPs and controls also differed in face configural processing in this brain region, even after the right FFA size was matched.

Figure 4.

fMRI MVPA decoding performance in the main experiment for the seven DPs, the eight controls whose right FFA sizes matched those of DPs, and all 12 controls. Overall, very similar results were obtained whether 8 or 12 controls were included in the analyses. A, Right FFA decoding results. Left, Decoding was at chance for the intact and the scrambled faces in DPs, but was significantly above chance in controls, with a significant between group differences. Middle, Decoding for the two face parts shown at the same location was marginally significant in DPs and significant in controls, with no between group differences. Right, Decoding for the same face part shown at different locations was at chance for both DPs and controls. B, Right LO decoding results. Decoding was significant for all three comparisons, with no between group differences. Error bars indicate SEM; †p < 0.10, *p < 0.05, **p < 0.01; NS, nonsignificant; Ctrls, control observers without developmental prosopagnosia.

Could DPs' face configural processing impairment be a result of their inability to form distinct face part representations in the right FFA? To test this, we examined the decoding of individual face parts appearing at the same location. We obtained above chance decoding in controls (t₍₇₎ = 5.26, p < 0.01) and trending toward significantly above chance decoding in DPs (t₍₆₎ = 2.20, p = 0.07; this decoding reached significance in the follow-up DP experiment; Fig. 5A), with no difference seen between the two groups (t₍₁₃₎ = 1.05, p = 0.31; Fig. 4A, middle). DPs, thus, could by and large, form distinct representations for the different face parts in the right FFA, similar to controls.

Figure 5.

fMRI MVPA decoding performance in the two follow-up experiments. Results in the left two columns are from the follow-up DP experiment in which four DPs were tested with Caucasian faces as the stimuli. Results in the right column are from the follow-up control experiment in which 6 controls were tested with phase scrambled face parts. A, Right FFA decoding results. Left, Replicating the results from the main experiment, decoding for face configurations was at chance in DPs. Middle, Again replicating the results from the main experiment, decoding for face parts shown at the same location was significantly above chance in DPs. Right, Decoding for phase-scrambled face parts was at chance in controls. B, Right LO decoding results. Decoding was significantly above chance for all three comparisons. Error bars indicate SEM; *p < 0.05, **p < 0.01.

Interestingly and consistent with a prior report (Golomb and Kanwisher, 2012), decoding was at chance for the same face part shown at the upper and the lower locations in the right FFA of both DPs and controls, with no differences between the groups (t values <1.58, p values >0.16; Fig. 4A, right). Thus, in controls, what enabled the neural decoding of face configuration in the right FFA could not be pure location differences between the face parts; but rather, it had to be how the face parts were configured and whether or not the T-shape arrangement was present. This processing appeared to be impaired in the right FFA of DPs, preventing them from forming distinct representations for the intact and the scrambled faces.

Unlike in the right FFA, in the right LO region, a nearby brain region involved in general object shape processing (Grill-Spector, 2003), face configural decoding was above chance for both DPs and controls (t values >3.97, p values <0.01), with no difference between the groups (t₍₁₃₎ = 0.81, p = 0.43; Fig. 4B, left). Importantly, there was a significant interaction between group and brain region (right FFA vs right LO; F_(1,13) = 5.62, p = 0.03), showing that the impairment seen in DPs was not widely present throughout their right ventral visual cortex. In the right LO, both face part location and identity could be successfully decoded in both DPs and controls (t values >9.73, p values <0.001; with no between group difference in either comparison, t values <1.30, p values >0.22; Fig. 4B, middle and right). This suggests that the success of face configural decoding in the right LO might simply be accomplished by representing which face part appeared where without actually representing face configuration. Given the presence of DPs' behavioral face configural processing deficit, this indicates that face representations formed in the right LO cannot fully compensate for the right FFA's deficiency in face configural processing, providing further support for the right FFA's unique and critical role in face processing.

In an effort to examine whether our MVPA results were robust when more controls were added, we included all 12 controls tested and reran all the analyses. Overall, the results from all 12 controls were almost identical to those from the eight controls with the smallest FFA ROIs (Fig. 4), showing that a difference in the right FFA size did not affect the decoding results. Specifically, in the right FFA, comparison between DPs and all 12 controls revealed that the decoding of the intact and the scrambled face configurations was significantly above chance in controls (t₍₁₁₎ = 4.57, p < 0.001) and was better in controls than in DPs (t₍₁₇₎ = 2.12, p < 0.05); and the decoding of the face parts averaged over the two locations was significantly above chance (t₍₁₁₎ = 7.08, p < 0.001), but not the decoding of the two locations averaged over the two parts (t₍₁₁₎ = 1.82, p = 0.10), with no group difference between DPs and controls in either (t values <1.44, p values >0.16). In the right LO, the decoding of the intact and the scrambled face configurations, the decoding of the two face parts averaged over the two locations, and the decoding of the two locations averaged over the two parts were all significant (t values >2.70, p values <0.03). Controls and DPs did not differ from each other in decoding performance in any of the comparisons in the right LO (t values <1.37, p values >0.19) except that controls showed slightly higher decoding of face parts than DPs (marginally significant, t₍₁₇₎ = 1.84, p = 0.08). However, as DPs also showed above chance decoding for face parts in the right LO (t₍₆₎ = 9.73, p < 0.001), this between group difference was likely quantitative rather than qualitative.

Our main MVPA results were replicated in a follow-up DP experiment with 4 DPs using Caucasian face stimuli (Fig. 5). In the right FFA, DPs again failed to exhibit above chance neural decoding of face configuration (t₍₃₎ = 0.21, p > 0.85), though the decoding of face parts was significantly above chance (t₍₃₎ = 4.19, p < 0.05). In contrast, in the right LO, DPs showed above chance decoding for both face configurations and face parts (t values >3.62, p values <0.05). Thus, even with a reduced sample size, DPs' face configural processing deficit in the right FFA was both prominent and replicable, and could be observed even with faces from a different race.

Although the different face parts could be successfully decoded in both DPs and controls in the right FFA, because the two face parts differed in size and possibly other low-level visual features (such as, luminance and spatial frequency distribution), it is possible that face part decoding in the right FFA relied on the differences between these features rather than the identities of the face parts per se. To examine this possibility, in a follow-up control experiment, we included the four face-part conditions in the main experiment but phase scrambled the face parts to remove the identities of the face parts while retaining the low-level features of the original face parts (Fig. 1B). In a new group of 6 controls, we found that decoding for the phase scrambled face parts was at chance in the right FFA (t₍₅₎ = 1.13, p > 0.30), but above chance in the right LO (t₍₅₎ = 16.10, p < 0.001), with a significant difference between these two brain regions (t₍₅₎ = 20.79, p < 0.001). Thus, although face part decoding in the right LO in the main experiment could have relied on decoding the differences in low-level visual features such as size, luminance, or spatial frequency, face part decoding in the right FFA in the main experiment, however, was unlikely to have relied on anything other than differences in face part identity.

In addition to the right FFA, a number of other occipital and temporal brain regions also exhibit preferences for faces over nonface objects. These include the right and left occipital face area (OFA), which has been shown to participate in the processing of the individual face parts (Gauthier et al., 2000; Liu et al., 2010), the right and left posterior superior temporal sulcus (pSTS), which has been implicated in the processing of the dynamic information related to face perception (Phillips et al., 1997), and the left FFA. Our localizer scans, however, were only able to reveal the right OFA in 3 of 7 DPs and 4 of 8 controls, the left OFA in 4 of 7 DPs and 5 of 8 controls, the left FFA in 3 of 7 DPs (but in all 8 controls), the right pSTS in 5 of 7 DPs (but in all 8 controls), and the left pSTS in 2 of 7 DPs and 5 of 8 controls. We thus did not have sufficient sample sizes and power to contrast face configural processing between DPs and controls in these other face processing brain regions. Future studies with more powerful localizers (such as the dynamic face localizer; Pitcher et al., 2011) and bigger DP sample sizes may help circumvent this difficulty.

Univariate face configuration, part, and location measures

In addition to MVPA, we also examined univariate fMRI response amplitude measures for our face configuration, part, and location manipulations (Fig. 6). Overall, univariate measures produced similar, but much weaker and less consistent results than those from MVPA.

Figure 6.

Mean univariate fMRI response amplitude measures for the different face stimuli used in the main experiment in the right FFA and the right LO. Results were shown for the seven DPs, the eight controls whose right FFA sizes matched those of DPs, and all 12 controls. Overall, the univariate measures produced similar, but weaker and less consistent results compared with those from MVPA. A, Right FFA response amplitudes. B, Right LO response amplitudes. Left, Response amplitudes for the intact and the scrambled face configurations. Middle, Response amplitudes for the two face parts averaged over the two presentation locations. Right, Response amplitudes for the two presentation locations averaged over the two face parts; † p< 0.10, *p < 0.05, **p < 0.01; NS, nonsignificant; Ctrls, control observers without developmental prosopagnosia.

In the right FFA, in both control groups (i.e., the group that included only the eight that matched DPs in their averaged right FFA size and the group that included all 12 controls), responses were higher to the intact than to the scrambled faces (t values >3.52, p values <0.01). This difference, however, was absent in DPs (t₍₆₎ = 1.24, p = 0.26). The interaction between group and stimulus was marginally significant when DPs were compared with the eight controls (F_(1,13) = 4.21, p = 0.06), but not significant when DPs were compared with all 12 controls (F_(1,17) = 1.15, p = 0.30). These results were overall consistent but weaker compared with those obtained with MVPA (Fig. 4).

In the right FFA, response amplitude measures for the two face parts averaged over the two presentation locations revealed no difference between the face parts in either the DPs or controls (t values <1.80, p values >0.10) and no difference between the groups (F values <1). The response amplitude measures for the two presentation locations averaged over the two face parts revealed that, in DPs, responses were marginally significantly higher for the upper than the lower location (t₍₆₎ = 2.05, p = 0.09). This difference was not observed in either group of controls (t values <1). The interaction between group and location was marginally significant when DPs were compared with the eight controls (F_(1,13) = 3.85, p = 0.07) and was significant when DPs were compared with all 12 controls (F_(1,17) = 4.79, p = 0.04). The right FFA's preference for the upper location in DPs has not been reported before and could reflect an imbalance in the processing of the upper and lower face parts in DPs.

In the right LO, response amplitude measures for the intact and scrambled faces revealed no stimulus difference in both DPs and controls (t values <1.69, p values >0.13), and no interaction between stimulus type and group (between DPs and the eight controls, F_(1,13) = 3.19, p = 0.10; and between DPs and the 12 controls, F_(1,17) = 1.90, p = 0.19). The response amplitude measures for the two face parts averaged over the two presentation locations revealed that parts containing a nose and mouth elicited higher responses than those containing eyes in both DPs and controls (t values >4.09, p values <0.01), with no interaction between stimulus type and group (F values <1.01, p values >0.33). This result was expected as parts containing a nose and mouth covered a larger spatial extent than those containing the eyes. The response amplitude measures for the two presentation locations averaged over the two face parts revealed that parts shown at the lower location elicited stronger responses than those shown at the upper location in both DPs (marginally significant, t₍₆₎ = 2.42, p = 0.05) and controls (t values >2.84, p values <0.03) with no interaction between group and location (F values <1, p values >0.34). This result was consistent with a previous report showing that LO exhibited a higher response to the lower than to the upper visual field (Large et al., 2008).

Overall, these analyses indicated that similar results could be obtained in both fMRI univariate and multivariate measures, but the results from univariate measures were weaker and less reliable than those from multivariate measures. Thus, MVPA provided a more sensitive measure than conventional univariate approach in unveiling a face configural processing deficit in the right FFA of DPs.

Discussion

Face processing recruits specialized brain regions such that damage to these regions can lead to face-specific processing deficits (Riddoch et al., 2008). Among the brain regions involved in face processing, the right FFA has been shown to play an important role in face configural processing (Schiltz and Rossion, 2006; Liu et al., 2010; Parvizi et al., 2012; Zhang et al., 2012). Intriguingly, despite the presence of behavioral face configural processing deficits, individuals with DP have been shown to exhibit normal face selectivity and normal fMRI face identity adaptation in their right FFA (Furl et al., 2011; Avidan et al., 2014). These findings promoted researchers to argue that perhaps DPs have an intact right FFA but an impaired information transmission between the right FFA and other face processing regions (Avidan et al., 2014). We noted, however, that fMRI investigations of DP have not directly examined face configural processing in the right FFA, leaving it unknown whether face configural processing is normal or impaired in the right FFA of DPs.

In the present study, using univariate response amplitude measures, we observed normal face selectivity in the right FFA in both DPs and controls. However, using fMRI MVPA, we discovered that, unlike controls, DPs showed chance level neural decoding between the intact and the scrambled face configurations in their right FFA. Although, on average, the size of the right FFA was smaller in DPs than in controls, the face configural processing impairment was present whether or not the size of the right FFA was matched between DPs and controls. This impairment was most clearly seen in fMRI MVPA results, but consistent, though weaker, effects could also be seen in fMRI univariate response amplitude measures.

In DPs, this face configural processing impairment was specifically related to a deficit in how the face parts were conjoined. This was because both DPs and controls showed successful decoding of the identities of the face parts but not the vertical locations of the face parts in the right FFA. Thus, in controls, what enabled neural decoding of face configuration in the right FFA could not be location differences between the face parts in the two face configurations. Instead, decoding appeared to be driven by differences in how the face parts were configured and whether or not the T-shape face part arrangement was present. Such face configural processing seemed to be impaired in the right FFA of DPs, preventing them from forming distinct representations for the intact and the scrambled faces.

Importantly, the impairment seen in DPs was not a broad impairment present throughout the right ventral visual cortex, as decoding of face configuration was successful in the right LO in both DPs and controls. However, we were also able to successfully decode the identity and the location of the face parts in both DPs and controls in this region, unlike that in the right FFA. Thus, the decoding of face configuration in the right LO could simply be accomplished by representing which face part appeared where without actually representing the configuration between the face parts. The presence of a behavioral face configural processing deficit in DPs supports this view and argues that face representations formed in the right LO cannot fully support normal face processing nor compensate for any deficiency in the right FFA's face configural processing in DP.

Previous behavioral studies have reported that many DPs could distinguish between the intact and the scrambled faces equally well as that of the controls (De Haan and Campbell, 1991; de Gelder and Rouw, 2000; Nunn et al., 2001; Duchaine et al., 2003). This is not surprising in light of the present findings. Because brain regions, such as LO, could be recruited to aid the discrimination of the intact from the scrambled faces, this task may be performed successfully without necessarily engaging the face processing neural mechanisms. As such, success in this task alone should not be used as an indicator of the presence of normal face configural processing.

Based on impaired functional connectivity among the face processing regions (Thomas et al., 2009; Zhu et al., 2011), researchers have argued that an impaired information transmission between the right FFA and the other face process regions could contribute to the behavioral face processing deficits associated with DP (Avidan et al., 2014). In light of the present findings, it seemed that impoverished outputs from the right FFA could easily prevent information exchange from being properly established between the right FFA and other face-processing regions during development, thus contributing to the impaired communication observed between these brain regions in DPs. It is also possible that impaired information transmission among the face processing regions and impaired face configural processing in the right FFA constitute two independent deficits associated with DP.

Compared with other object categories, the high efficiency associated with face recognition is believed to be largely due to the presence of configural processing of the face parts. Among the different components of face configural processing (Maurer et al., 2002), the T-shape face part arrangement is believed to be most fundamental as it serves as the basis where other aspects of face configural processing may be carried out. Here we show, for the first time, that in DPs the representation of the T-shape face part configuration is impaired in the right FFA, likely precluding this brain region from supporting other components of face configural processing. Given the importance of face configural processing in face identity representation and the critical role the right FFA plays in mediating such processing (Schiltz and Rossion, 2006; Liu et al., 2010; Richler et al., 2011; Parvizi et al., 2012; Wang et al., 2012; Zhang et al., 2012), we believe that the impairments seen here may account for DPs' behavioral face processing deficits. We propose that the right FFA's face configural processing deficit should therefore be regarded as an important, if not one of the key, neural impairment of DP.