The social evaluation of faces: a meta-analysis of functional neuroimaging studies

Abstract

Neuroscience research on the social evaluation of faces has accumulated over the last decade, yielding divergent results. We used a meta-analytic technique, multi-level kernel density analysis (MKDA), to analyze 29 neuroimaging studies on face evaluation. Across negative face evaluations, we observed the most consistent activations in bilateral amygdala. Across positive face evaluations, we observed the most consistent activations in medial prefrontal cortex, pregenual anterior cingulate cortex (pgACC), medial orbitofrontal cortex (mOFC), left caudate and nucleus accumbens (NAcc). Based on additional analyses comparing linear and non-linear responses, we propose a ventral/dorsal dissociation within the amygdala, wherein separate populations of neurons code for face valence and intensity, respectively. Finally, we argue that some of the differences between studies are attributable to differences in the typicality of face stimuli. Specifically, extremely attractive faces are more likely to elicit responses in NAcc/caudate and mOFC.

INTRODUCTION

Within a single glance of a face, people automatically appraise face attractiveness and make a host of social attributions (Olson and Marshuetz, 2005; Bar et al., 2006; Willis and Todorov, 2006; Rule et al., 2009; Todorov et al., 2009). For example, 33-ms exposure to a face is sufficient for people to make trustworthiness decisions (Todorov et al., 2009). Additional time exposure simply increases confidence in these decisions (Willis and Todorov, 2006). As one of the founding fathers of modern social psychology, Solomon Asch (1948, p. 258), put it, ‘We look at a person and immediately a certain impression of his character forms itself in us. A glance, a few spoken words are sufficient to tell us a story about a highly complex matter. We know that such impressions form with remarkable rapidity and with great ease. Subsequent observations may enrich or upset our view, but we can no more prevent its rapid growth than we can avoid perceiving a given visual object or hearing a melody’.

Recent research confirms Asch's insights (Zebrowitz, 1999; Todorov et al., 2008a,b; Zebrowitz and Montepare, 2008). People rapidly and effortlessly form impressions from facial appearance. Although the validity of such impressions is low (Olivola and Todorov, 2010a), inferences of character and personality have been shown to predict important outcomes in domains ranging from politics (Todorov et al., 2005; Ballew and Todorov, 2007; Olivola and Todorov, 2010b), law (Zebrowitz and McDonald, 1991; Blair et al., 2004; Eberhardt et al., 2006), mate choice (Olivola et al., unpublished data), business (Hamermesh and Biddle, 1994; Rule and Ambady, 2008) and the military (Mazur et al., 1984).

Despite the importance of first impressions for social interactions, research on their neural basis is in its infancy. Researchers began to use social neuroscience methods to investigate this basis only a decade ago (Adolphs et al., 1998; Nakamura et al., 1998; Aharon et al., 2001; Winston et al., 2002). Although a number of neuroimaging studies have been published on the topic, many of the results have been inconsistent (Todorov et al., 2011). The objective of this article is to provide a quantitative summary of the major findings across studies on face evaluation.

The neural basis of face evaluation

Neuroimaging research on the social evaluation of faces has usually focused on evaluations along the trait dimensions of trustworthiness and attractiveness. Although these are separable dimensions, psychometric studies of social judgments from faces show that these judgments are highly inter-correlated with each other, with correlations ranging from 0.60 to 0.80 (Oosterhof and Todorov, 2008; Todorov et al., 2008a,b). For example, principal components analyses show that (i) the first component, which indicates general face valence, accounts for >60% of the variance of judgments; and (ii) trustworthiness and attractiveness judgments are highly correlated with this valence component. Given these behavioral data, one would expect to observe overlapping regions in neuroimaging studies on attractiveness and trustworthiness.

For the purposes of this meta-analysis, we focus on studies on attractiveness and trustworthiness. Typically, such studies present participants with facial stimuli that vary on the respective dimension—either systematically manipulated via computer modeling, or confirmed by independent behavioral ratings—and subsequently report brain activity that shows a linear relationship with changes in facial appearance along that dimension. For example, some studies have observed increased responses in the amygdala for untrustworthy faces (Winston et al., 2002), while other studies have observed increased responses in the nucleus accumbens (NAcc) and medial orbitofrontal cortex (mOFC) for attractive faces (Aharon et al., 2001; O'Doherty et al., 2003). More recent studies have sought to identify regions that show a quadratic relationship between brain activity and changes in attractiveness or trustworthiness. Researchers have observed non-linear responses in the amygdala for both attractive and unattractive faces (Winston et al., 2007), as well as for both trustworthy and untrustworthy faces (Todorov et al., 2011). While there is convergence between the linear and non-linear approaches, there exists the possibility that these analyses are tapping distinct processes, wherein areas that show a linear pattern of activity are coding for face valence, while areas that show quadratic patterns are coding something more like face intensity.

The first objective of this article is to systematically explore the pattern of observed brain activations across published neuroimaging studies on face evaluation as a function of face valence. The second objective is to examine possible dissociations between linear and non-linear responses. The third and final objective is to explore potential differences between trustworthiness and attractiveness studies.

Multilevel kernel density analysis

Meta-analysis is a powerful statistical tool that allows researchers to combine the data sets of a collection of similar studies to provide a more accurate, robust estimate of the effect-size of a given phenomena. This approach is widespread within behavioral research, and in recent years, meta-analyses of neuroimaging studies have become more common (Fox et al., 1998; Phan et al., 2002; Wager and Smith, 2003; Wager et al., 2004; Laird et al., 2005; Nielsen et al., 2005). Meta-analyses of neuroimaging data typically compute how frequently studies examining a given psychological phenomenon report activity in a specific brain area (Kober and Wager, 2010). This approach can be used to confirm the prevailing thinking regarding what brain areas are associated with a particular psychological phenomenon or experience. At the same time, meta-analysis can serve a more exploratory purpose—identifying regions that are consistently activated across a large number of studies of the same psychological phenomenon, but that are not typically associated with that phenomenon.

Indeed, a meta-analysis of the social evaluation of faces has been recently published (Bzdok et al., 2011), and in part, motivated the analyses herein. While we ultimately employed slightly different selection criteria in choosing studies to include, we also sought to perform several more targeted analyses, as noted above. Perhaps more importantly, while Bzdok and colleagues conducted an activation likelihood estimation (ALE) meta-analysis, we use a different statistical procedure.

Specifically, we use a Multi-level Kernel Density Analysis (MKDA), which represents an advance in meta-analytic methods for neuroimaging data, because it accounts for the fact that individual activation peaks are nested within contrast maps (maps of particular comparisons within studies), making these maps the unit of analysis, and not the peaks (Wager et al., 2008). Further, MKDA models contrast maps as a random effect, eliminating the possibility of one contrast dominating the meta-analysis.

We conduct several analyses. First, we analyze activations across all contrasts showing (i) stronger brain responses to negative—untrustworthy and unattractive—than positive—trustworthy and attractive—faces; (ii) stronger responses to positive than negative faces; and (iii) stronger responses to positive and negative faces than to neutral faces. Second, within these contrasts, we also explore potential differences between trustworthiness and attractiveness studies.

METHODS

Data collection

We searched for neuroimaging studies of the social evaluation of faces using the online databases PsycINFO and PubMed, as well as the scholarly article search engine Google Scholar. We limited our search using combinations of keywords including ‘faces’, ‘social evaluation’, ‘social judgment’, ‘fMRI’, ‘trustworthiness’ and ‘attractiveness’. To be included in our meta-analysis, studies had to involve fMRI or PET investigations of healthy adults,¹ report activations in a standard coordinate system—either Talairach or Montreal Neurological Institute (MNI) coordinates, and explicitly state whether their analyses were performed with fixed or random effects. With respect to in-scanner tasks, we only included studies in which subjects either made explicit judgments regarding the trustworthiness or attractiveness of faces, or were presented with faces that varied on one of these two dimensions during an implicit or a passive viewing task, based upon normative ratings, computer modeling or some other form of categorization. In the case of some studies (Hampshire et al., 2011; Pochon et al., 2008; Zaki et al., 2011), relevant contrasts were not originally reported, but were obtained through personal communication with the respective authors.

We excluded studies that did not report specific coordinates arising from relevant contrasts, but instead referred to various ROIs from a functional localizer being more or less active during specific contrasts (Kranz and Ishai, 2006). In some instances, multiple studies were found which presented analyses of the same data sets (Todorov and Engell, 2008; Pinkham et al., 2008b). In these cases, we only included one study's reported coordinates, and this choice was made based upon which version of the study ultimately presented the more relevant analyses. Finally, we excluded some studies whose research questions bordered on ours (for instance, aesthetic judgments of paintings of faces, as in Kawabata and Zeki (2004) or neural responses to faces similar to the self varying in trustworthiness, as in Verosky and Todorov (2010) as they ultimately did not report contrasts that were appropriate for inclusion in our analyses. These choices are not trivial, as they represent some of the differences between our meta-analysis and the one conducted by Bzdok and colleagues (2011), in terms of study selection.

This search yielded 28 published papers comprising 29² neuroimaging studies on the social evaluation of faces. Seventeen of these studies were on attractiveness evaluations and 12 were on trustworthiness or related evaluations (i.e. ‘would you approach or avoid this person’). The latter were included because such approach/avoidance evaluations are highly correlated with trustworthiness evaluations (Todorov, 2008). This set of studies accounted for 52 separate contrasts (Table 1). For contrasts to be included in our database, they had to be representative of neural activity that varied parametrically with either facial attractiveness or trustworthiness, and furthermore, the direction and linearity of this relationship had to be clearly stated. We excluded coordinates derived from complex interaction-based analyses (for instance, stimuli type and gender interactions, as seen in O'Doherty et al. (2003), as well as coordinates arising from analyses that were not relevant to our research questions (e.g. effects of face novelty in Kim et al. (2007). Further, overlapping contrasts are often reported in the articles surveyed. For instance, Aharon and colleagues (2001) report separate contrasts detailing neural activity associated with facial attractiveness for male stimuli, female stimuli, and collapsed across both kinds of stimuli. In these cases, we only included the most general reported contrast—for instance, for Aharon et al. (2001), we used the collapsed contrast. Studies that report separate results for explicit and implicit paradigms presented a unique problem (see Winston et al., 2002; Baas et al., 2008; Chatterjee et al., 2009). On the one hand, both analyses are relevant to our main research question, and favoring one paradigm over the other in these three cases would bias our results in favor of that task design. On the other hand, the contrasts are undeniably non-independent of each other. Ultimately, we chose to run our analyses using both sets of coordinates for these three studies, which were entered into our database as separate contrasts. To confirm that this approach had no demonstrable impact on our results, we ran complimentary analyses that only included one contrast per study (i.e. only the explicit task contrast from the three studies in question). We observed no practical differences in either the size or localization of consistent activations.

Table 1.

Breakdown of Studies Included in the Meta-Analysis

Study	Included contrasts	N	Task naturea			Study type	Valence			ROI?c	Stimulus category
			Implicit	Explicit	Collapsedb		Negative	Positive	Non-Linear
Aharon, et al., 2001	1	6	X			attractiveness		X			extreme
Aron, et al., 2005	1	17	X			attractiveness		X			average
Baas, et al., 2008	4	21	X	X	X	trustworthiness	X	X			average
Blasi, et al., 2009	1	43		X		trustworthiness	X				average
Bray, et al., 2007	2	25	X			attractiveness	X	X			extreme
Chatterjee, et al., 2009	4	13	X	X		attractiveness	X	X			computer generated
Cloutier, et al., 2008	2	48		X		attractiveness	X	X			extreme
Engell, et al., 2007	1	15	X			trustworthiness	X				average
Gordon, et al., 2009	2	6	X			trustworthiness		X	X	X	average
Hampshire, et al., 2011	1	19		X		attractiveness		X			extreme
Iaria, et al., 2008	1	10		X		attractiveness		X			extreme
Kampe, et al., 2001	1	16	X			attractiveness		X			average
Kim, et al., 2007	1	25		X		attractiveness		X			computer generated
Liang, et al., 2010	3	17	X			attractiveness	X	X	X	X	extreme
O'Doherty, et al., 2003	2	25	X			attractiveness	X	X			extreme
Pinkham, et al., 2008	1	12		X		trustworthiness	X			X	average
Pochon, et al., 2008	1	17		X		attractiveness		X			extreme
Said, et al., 2009	3	31		X		trustworthiness	X	X	X		average
Said, et al., 2010	1	24	X			trustworthiness			X		computer generated
Smith, et al., 2010	1	26		X		attractiveness		X			average
Todorov, et al., 2008	2	14	X			trustworthiness	X		X	X	computer generated
Todorov, et al., 2010	2	22		X		trustworthiness	X		X		computer generated
Todorov, et al., 2010	2	22	X			trustworthiness	X		X		computer generated
Tsukiura, et al., 2010a	3	22		X		attractiveness	X	X	X		average
Turk, et al., 2004	1	18		X		attractiveness		X			extreme
VanRijn, et al., 2011	3	18		X		trustworthiness	X	X		X	average
Winston, et al., 2002	2	14			X	trustworthiness	X	X			average
Winston, et al., 2007	2	26			X	attractiveness		X	X		extreme
Zaki, et al., 2011	1	14		X		attractiveness		X			extreme

^a‘Task nature’ categorizes only the contrasts included in our meta-analysis. In some cases, like Winston et al. (2002), both explicit and implicit paradigms were employed, but only collapsed analyses were reported.

^b‘Collapsed’ analyses refer to analyses in which neural activity was aggregated across both explicit and implicit tasks.

^cIn this column, we note if a given study reported coordinates arising from ROI-based analyses. In some cases, these studies only reported such ROI-based analyses (for instance, Pinkham et al., 2008). As such, these studies have only been able to impact our supplementary analyses, which incorporate ROI-based analyses in addition to whole-brain contrasts.

We tabulated the design particulars and parameters of each study, as well as the reported activation points for all relevant contrasts. Specifically, we coded each study in terms of which coordinate system activations were reported in, number of participants, whether a fixed or random effects analysis had been performed, whether activations represented linear or non-linear effects, whether the task was explicit or implicit in nature, and whether the reported activations were the result of a whole-brain or region-of-interest (ROI) analysis. This coding scheme served two purposes. Primarily, this information was fed into the MKDA toolbox and used to determine the proper weighting scheme for the different studies. Secondarily, it served as the basis for contrasting studies against each other on relevant variables. This coding scheme was initially entered by the first author, with subsequent confirmation and complete agreement from the second and third authors. Entered coordinates were checked and re-checked against their original sources numerous times throughout the course of setting up our database.

The studies compiled in our database used a variety of face stimuli. Some studies used computer-generated faces (for instance, Chatterjee et al., 2009), others used standardized photograph sets of volunteer subjects (for instance, Engell et al., 2007), and still others used photographs culled from magazines and newspapers (for instance, O'Doherty et al., 2003). These faces likely differ in terms of their typicality—faces in standardized photographs are more typical than the extreme faces seen in photographs of models and actresses. Given recent work suggesting that face typicality can partially account for the amygdala's response to the valence of face stimuli (Said et al., 2010), it is possible that different types of face stimuli (e.g. extremely attractive faces that are less typical) could lead to different patterns of neural responses. As such, while we did not exclude studies based on the sources of the face stimuli, we did keep track of the source of each study's stimuli. This allowed for the possibility of comparing the more typical faces (computer-generated and standardized sets) against the more atypical faces (photos of models and actors from print media).

It is important to note that contrasts containing ROI-based analyses pose a problem for inclusion in meta-analyses. On the one hand, including coordinates from ROI-based analyses may bias the results by introducing researchers' a priori predictions about which regions are involved in trustworthiness and attractiveness evaluation. On the other hand, such analyses represent theoretically motivated prior research. Further, because some ROIs like the amygdala and NAcc are relatively small and often difficult to image, excluding ROI-based analyses may miss important findings that are consistent across studies. Given that, we chose to run each of our analyses twice—once limited to whole-brain contrasts, and once with ROI-based contrasts included. In the interest of space, we chose to report the whole-brain analyses in the main text, as well as to note whether or not adding ROI-based contrasts substantially affected the results. (In all cases but one, adding ROI-based contrasts did not have a substantial effect on analyses. For the contrast that produced divergent results, we chose to explicitly note in the text how the two approaches differed.) The specific results for the ROI-based analyses are reported in supplemental material. We note that while some studies reported ROI-based analyses side by side with whole-brain analyses (for instance, Van Rijn et al., 2011), there are a small number of studies that reported only ROI-based analyses (for instance, Pinkham et al., 2008a).

Data analysis

Our MKDA of ‘negative’ contrasts comprised all contrasts in which brain activity increases as facial stimuli decrease in either trustworthiness or attractiveness. ‘Positive’ contrasts comprised all contrasts in which brain activity increases as facial stimuli increase in either trustworthiness or attractiveness. Non-linear, quadratic contrasts comprised all contrasts in which brain activity increases as facial stimuli increase or decrease in either trustworthiness or attractiveness relative to faces at the middle of the continuum. Not all studies included in our database reported both negative and positive contrasts. Therefore, neither of our primary MKDAs contains contrasts from every study.

When performing these analyses, the peak coordinates from all relevant contrast maps were first separately convolved with a 10-mm spherical kernel, yielding comparison indicator maps (CIMs). Previous meta-analytic work suggests that this is an appropriate default kernel size (Wager et al. 2003; Salimi-Khorshidi et al., 2009). These CIMs were subsequently weighted based upon the number of participants and what type of analysis was performed in each study, following the same parameters used by Kober and colleagues (2008). Specifically, each map was first weighted by the square root of its study's sample size and subsequently multiplied by an adjustment factor accounting for the type of analysis used in the respective study. Random effects studies were multiplied by an adjustment factor of 1; fixed effects studies were multiplied by an adjustment factor of .75. In this fashion, studies received higher weighting if they had large sample sizes and performed random effects analyses.

Second, the weighted CIMs were averaged together, producing a density map. Each voxel of this density map attains a density statistic, P, which is the weighted proportion of contrasts included in the MKDA that yield activity within 10 mm of that voxel. To identify voxels whose P-statistic exceeds the frequency expected by chance, a Monte Carlo simulation was conducted. Over 5000 iterations of the Monte Carlo simulation, the observed activation blobs (contiguous regions of activation within the CIMs, holding shape constant) from each CIM were randomly shuffled within a gray-matter mask. Following each iteration, we recorded both the maximum whole brain density statistic (P, across all studies) and the largest cluster of contiguous voxels. These values were used to create null-hypothesis distributions for the density statistic and the expected size of clusters, respectively.

Third, the weighted P was subsequently tested against the resulting null-hypothesis P₀ produced by the Monte Carlo simulation. A similar procedure was used to test for the significance of the size of the clusters, allowing us to identify a size threshold at which a certain number of voxels must be activated contiguously for a given cluster to be deemed significant. Hence, we used two types of thresholds—a density height-based threshold and a cluster size threshold, the latter derived from a non-parametric cluster-based thresholding procedure (Nichols and Hayasaka, 2003). For P, the resulting familywise error rate (FWER)-controlled threshold is the proportion of studies reporting activity within 10 mm of a given voxel that exceeds the maximum P-statistic across 95% of the resulting Monte Carlo maps. These voxels appear on resulting maps colored in yellow and will be referred to in our results as exceeding the height-based threshold of the MKDA. For the cluster size threshold, the resulting FWER-controlled threshold is the clusters observed at P < .001 and P < 0.01 whose size exceeds the maximum cluster size across 95% of the Monte Carlo maps. These voxels appear on resulting maps in orange and pink, respectively, and will be referred to as exceeding the extent-based threshold. The thresholded maps were overlaid on a canonical MRI image (colin27.img, the single-subject template in SPM5; http://www.fil.ion.ucl.ac.uk/spm/software/spm5/), which was co-registered to the MNI brain template.

When reporting areas of consistent activation in our tables, we provide information on whether each area withstood height-based thresholding, extent-based thresholding, or both. Some areas of activation were sizable enough to pass extent-based thresholding but not height-based thresholding. Conversely, other areas were highly consistent across the database and passed height-based thresholding, but were not sufficiently large to pass extent-based thresholding. XYZ-coordinates reported in our tables reflect the peak activation foci which withstand height-based thresholding, or, if activations are less consistent, the center of mass of the cluster at the most stringent level of extent-based thresholding. Further, we report the number of voxels in each cluster which withstood height-based thresholding, or if activations are less consistent, the number of voxels at the most stringent level of extent-based thresholding.

We also performed several smaller, more targeted MKDAs exploring differences between trustworthiness and attractiveness studies and performed several additional exploratory analyses based on stimulus typicality. To perform such analyses, a simple subtraction yields the relative difference in the distribution of peaks between the respective contrasts, which is subsequently thresholded as explained above.

RESULTS

Results across negative contrasts

Eleven studies reported 13 negative contrasts—where brain activity increased as attractiveness or trustworthiness decreased—across the whole brain. The MKDA results for these contrasts are summarized in Table 2 and Figure 1. We observed highly consistent activation in right amygdala (withstood height- and extent-thresholding, P < 0.001), as well as activation in left amygdala that survived extent- but not height-thresholding (P < 0.001). Four studies reported ROI-based coordinates for negative contrasts. When we included these additional coordinates in our analysis as well, we continued to observe highly consistent activation in right amygdala (withstood height- and extent-thresholding, P < 0.001), as well as activations which survived extent- but not height-thresholding in left amygdala (P < 0.001), right globus pallidus (P < 0.01) and a large region of consistent activation encompassing right anterior insula, right inferior frontal gyrus (IFG) and right ventrolateral prefrontal cortex (vlPFC, P < 0.01, additional results summarized in Supplementary Table S1).

Table 2.

Areas Consistently Activated During Negative and Positive Face Evaluations

Region	Lat	x	y	z	Vol	%Act
Increased activity for negative evaluations, collapsed across Untrustworthiness and Unattractiveness, whole-brain contrasts only
Basal telencephalon
Amygdala	R	20	−6	−18	1080	0.34	†**
Amygdala	L	−18	−6	−18	1328	0.29	**
Increased activity for positive evaluations, collapsed across Trustworthiness and Attractiveness, whole-brain contrasts only
Basal telencephalon
Caudate/nucleus accumbens/medial orbitofrontal cortex	L	−10	10	−4	1888	0.33	†**
Thalamus	R	14	−16	6	384	0.28	†**
Caudate/right amygdala/anterior insula/inferior frontal gyrus	R	4	22	0	76656	0.22	**
Ventral striatum/thalamus/anterior insula/inferior frontal gyrus/ventrolateral prefrontal cortex	R	14	−2	−2	34688	0.20	*
Frontal/insular cortex
Ventromedial prefrontal cortex	–	0	42	−6	344	0.30	†**
Pregenual cingulate cortex/dorsal anterior cingulate	–	−2	40	8	96	0.25	†**
Pregenual cingulate cortex	–	0	36	2	16	0.25	†**

Note. Stereotactic coordinates representing the areas most consistently activated across full database. We report laterality (Right or Left), XYZ coordinates, number of voxels in each cluster (Vol), and weighted percentage of CIMs which activated each cluster (%Act). †, areas withstanding height-based thresholding. **, areas withstanding extent-based thresholding (p < .001). *, areas withstanding extent-based thresholding (p < .01).

Fig. 1.

Consistently activated areas across negative evaluations, showing consistent right amygdala activation. Yellow voxels withstood height-based thresholding, orange voxels withstood extent-based thresholding (P<.001).

Results across positive contrasts

Twenty-one studies reported 23 positive contrasts—where brain activity increased as attractiveness or trustworthiness increased—across the whole brain. The MKDA results for these contrasts are summarized in Table 2 and Figure 2. We observed highly consistent activation in left caudate extending into NAcc and mOFC, right thalamus, vmPFC and dACC/pgACC (withstood height- and extent-thresholding, P < 0.001), as well as portions of right amygdala right anterior insula, right IFG (P < 0.001), and bilateral vlPFC that survived extent- but not height-thresholding (P < 0.01). Four studies reported ROI-based coordinates for negative contrasts. Including these additional coordinates in our analysis yielded similar results (summarized in Supplementary Table S1).

Fig. 2.

Consistently activated areas across positive evaluations, including pgACC, vmPFC (A), left caudate/NAcc extending into mOFC (A and B), and right amygdala (C). Yellow voxels withstood height-based thresholding, orange voxels withstood extent-based thresholding (P < 0.001), and pink voxels withstood extent-based thresholding (P < 0.01).

Non-linear responses

Nine studies within our database conducted non-linear analyses testing for stronger responses to both negative- (unattractive or untrustworthy) and positive-looking (attractive or trustworthy) faces than to faces at the middle of the continuum. Collapsed across both sets of stimuli, we observed consistent non-linear activation across seven whole-brain contrasts in the right amygdala extending into right putamen (withstood height- and extent-thresholding, P < 0.001, Table 3). Including two additional ROI-based contrasts in the analysis yielded similar results (Supplementary Table S2). We note that given the relatively small number of contrasts documenting non-linear responses, this analysis is underpowered. Nevertheless, five of the seven whole-brain contrasts reported activity in right amygdala.

Table 3.

Consistently Activated Areas Displaying Non-linear Response Profiles

Region	Lat	x	y	z	Vol	%Act
Non-Linear responses, collapsed across Attractiveness and Trustworthiness studies
Basal telencephalon
Amygdala	R	20	−2	−10	1040	0.63	†

Note. Stereotactic coordinates representing the areas most consistently activated across full database. We report laterality (Right or Left), XYZ coordinates, number of voxels in each cluster (Vol), and weighted percentage of CIMs which activated each cluster (%Act). †, areas withstanding height-based thresholding.

We also compared non-linear responses against linear responses, though these comparisons are, by virtue of the smaller number of non-linear contrasts, unavoidably unbalanced. Contrasting negative linear contrasts (13 contrasts) against non-linear contrasts (seven contrasts), we observed a ventral portion of the right amygdala that was more consistently active in negative linear contrasts (withstood height- but not extent-thresholding, Supplementary Table S3), while a more dorsal portion of the right amygdala was more consistently active in non-linear contrasts. Including ROI-based contrasts in the analysis yielded similar results (Supplementary Table S4). (As this contrast is unbalanced, we have provided information regarding the frequency of activation at the peak voxels of those areas that withstood height-thresholding, Supplementary Table S5A)

Contrasting positive linear contrasts (23 contrasts) against non-linear contrasts (seven contrasts), we observed a set of regions that were more consistently active in positive linear contrasts, including bilateral caudate, vmPFC/OFC, dACC/pgACC (withstood height- and extent-thresholding, P < 0.001), right thalamus (withstood height-thresholding, P < 0.001), sgACC and bilateral vlPFC (P < 0.01, Supplementary Table S3). Conversely, we observed a region of right dorsal amygdala extending into right putamen (withstood height- but not extent-thresholding, results summarized in Supplementary Table S3) that was more consistently active in non-linear contrasts. Including ROI coordinates in the analysis yielded similar results (noted in Supplementary Table S4). (Information regarding the frequency of activation at the peak voxels of those areas that withstood height-thresholding is provided in Supplementary Table S5B.)

Negative linear responses in attractiveness and trustworthiness studies

We contrasted negative linear responses in attractiveness (six contrasts) and trustworthiness studies (seven contrasts), observing one activation in the right amygdala that was more consistently active for negative linear responses to trustworthiness than attractiveness (withstood height- and extent-thresholding, P < 0.001). We observed no regions that were consistently more active for negative linear responses to attractiveness than trustworthiness. (Results are summarized in Supplementary Table S5.) Including ROI coordinates (from one unattractiveness study and three untrustworthiness studies) in the analysis yielded similar results (noted in Supplementary Table S6).

Positive linear responses in attractiveness and trustworthiness studies

We contrasted positive linear responses in attractiveness (18 contrasts) and trustworthiness (5 contrasts). We observed activations in left caudate extending into NAcc, vmPFC/OFC and pgACC extending dorsally into dACC (withstood height- and extent-thresholding, P < 0.01) that were more consistent for positive linear responses to attractiveness than trustworthiness. We observed no regions that were consistently more active for positive linear responses to trustworthiness than attractiveness. (Results are summarized in Supplementary Table S5.) Including ROI coordinates (from one attractiveness study and three trustworthiness studies) in the analysis yielded similar results (noted in Supplementary Table S6). (Information regarding the frequency of activation at the peak voxels of those areas that withstood height-thresholding is provided in Supplementary Table S5C.)

Separating attractiveness studies by stimulus type

The differences between trustworthiness and attractiveness studies are interesting but also puzzling given that evaluations on these two dimensions are highly correlated. There were no obvious differences between these two sets of studies (for instance, they were well-balanced between implicit and explicit tasks) except for the nature of the face stimuli used in the studies. Whereas eleven of the attractiveness studies used atypical, extremely attractive faces (culled from magazines and print media, often of models), none of the trustworthiness studies used such faces (typically, these were standardized sets of faces or computer-generated faces).

If the differences between attractiveness and trustworthiness studies are partly due to differences in stimuli, then the regions that differentiate these studies should also appear in contrasts involving the extremeness of faces. We can test this proposition by splitting attractiveness studies into two groups—those that used extremely attractive stimuli and those that used average or computer-generated stimuli. Comparing extreme attractiveness studies to the set of trustworthiness studies should yield areas of consistent activation in NAcc/caudate and mOFC, for example, while there should be fewer differences between average attractiveness and trustworthiness studies.

Indeed, when contrasting extreme attractiveness (11 contrasts) against trustworthiness (8 contrasts), we observed consistent activation in left caudate and NAcc, extending into mOFC, pgACC, and vmPFC (withstood height- and extent-thresholding, p < .01; see Figure 3, results summarized in Supplemental Table 7). Further, we observed a consistent pattern of activation centered in pgACC and extending broadly into both vmPFC and vlPFC that withstood extent-thresholding (p < .01) but not height-thresholding. Contrasting average attractiveness against trustworthiness produced no areas of consistent activation.

Fig. 3.

Consistently activated areas in the contrast ‘Extreme’ Attractiveness > Trustworthiness, (A) NAcc extending into mOFC, (B) pgACC and vmPFC. (The Average Attractiveness > Trustworthiness contrast produces no regions of consistent activation.) Yellow voxels withstood height-based thresholding and pink voxels withstood extent-based thresholding (P < 0.01).

Similarly within the set of attractiveness studies, when contrasting studies that used extremely attractive faces (11 contrasts) against studies that used more typical faces (7 contrasts), we observed consistent activation in left caudate, vmPFC/mOFC, and pgACC/dACC (withstood height- and extent-thesholding, p < .01), while a larger activation extending broadly through mOFC, vmPFC and vlPFC withstood extent-thresholding (p < .01), but not height-thresholding. The reverse contrast produced no areas of consistent activation.

Further, differences due to face stimuli should be apparent in studies that used implicit evaluation paradigms. Because no evaluative dimension is specified in such paradigms, stimulus properties should drive the neural responses. Contrasting implicit paradigm studies that used extremely attractive faces with implicit paradigm studies that used more typical faces produced consistent activation in right amygdala, left caudate extending into NAcc and right inferior frontal gyrus (withstood height- but not extent thresholding; results are summarized in Supplemental Table S7).

DISCUSSION

Using multi-level kernel density analysis, a statistically rigorous method of meta-analysis that treats contrasts as the unit of analysis instead of individual activation peaks, we performed a meta-analysis on 29 neuroimaging studies of the social evaluation of faces. We split these studies by valence into two MKDAs, one focusing on brain responses to negative evaluations like unattractiveness and untrustworthiness, and the other focusing on brain responses to positive evaluations like attractiveness and trustworthiness.

Our negative MKDA revealed the most consistent activation in right amygdala. Less consistent areas of activation were observed in left amygdala, right anterior insula, right IFG, right vlPFC and right globus pallidus. These results are remarkably consistent with previous findings regarding the neural responses to angry faces (Morris et al., 1998; Whalen et al., 2001; Monk et al., 2006; Dannlowski et al., 2007). Amygdala responses to angry faces have been widely observed and characterized (Morris et al., 1998; Whalen et al., 2001; Nomura et al., 2004; Taylor et al., 2006; Dannlowski et al., 2007; Monk et al., 2008; Vrticka et al., 2008). Furthermore, a functional connectivity between the amygdala and vlPFC has been proposed and demonstrated (Nomura et al., 2004; Taylor et al., 2006; Monk et al., 2006, 2008), suggesting that in response to angry faces, the vlPFC may serve to modulate amygdala reactivity, effectively regulating emotional responses. Right IFG (Dannlowski et al., 2007), right insula (Dannlowski et al., 2007; Vrticka et al., 2008) and right globus pallidus (Jackson et al., 2008) have also all been implicated in the neural response to angry faces.

Our positive MKDA revealed highly consistent activations in left caudate extending into NAcc/mOFC, vmPFC, dACC/pgACC, right thalamus as well as less consistent activations in right amygdala, insula, IFG and vlPFC. Once again, this pattern of activations bears a strong resemblance to that associated with a different emotional expression—happiness. Happy faces have been observed to elicit responses in parts of the striatum (Morris et al., 1996; Phillips et al., 1998; Fu et al. 2007; Vrticka et al., 2008), mPFC (Phillips et al., 1998; Kesler-West et al., 2001), ACC (Dolan et al., 1996; Phillips et al., 1998; Kesler-West et al., 2001; Vrticka et al., 2008), thalamus (Dolan et al., 1996), IFG (Dolan et al., 1996) and insula (Lee et al., 2002). Furthermore, a number of studies have observed amygdala responses to happy faces (Breiter et al., 1996; Canli et al., 2002; Pessoa et al., 2002; Yang et al., 2002; Somerville et al., 2004).

The consistent activation in the left caudate nucleus, extending broadly into the nucleus accumbens, suggests that positive evaluation of faces may depend, in part, on the recruitment of structures implicated in reward-processing (Knutson et al., 2001a,b; Haruno et al., 2004). However, we note that consistent activation in this area was almost entirely driven by attractiveness contrasts, and, therefore, may not be part of a general network for face evaluation. Nonetheless, the highly consistent presence of these areas in our meta-analysis suggests that under certain task and stimulus conditions, attractive faces modulate activity in reward-related regions of the brain.

The similarities between the neural correlates of negatively and positively evaluated faces and angry and happy faces, respectively, parallels perceptual similarities between these types of faces. In computer models of facial trustworthiness, extreme untrustworthiness resembles anger and extreme trustworthiness resembles happiness (Oosterhof and Todorov, 2008, 2009; Todorov et al., 2008a,b). Further, behavioral adaptation studies suggest common neural underpinnings for evaluations of trustworthiness and anger/happiness (Engell et al., 2010). These observations are consistent with the emotion overgeneralization hypothesis (Montepare and Dobish, 2003; Todorov et al., 2008a,b; Zebrowitz and Montepare, 2008; Said et al., 2009a,b), according to which evaluative judgments of faces are based on configurations of facial features resembling emotional expressions. In the context of positive and negative evaluation, these configurations signal approach and avoidance behaviors, respectively (Todorov, 2008). Our meta-analysis findings are also consistent with the hypothesis that novel faces are automatically evaluated with respect to their approach/avoidance value.

The role of the amygdala in face evaluation

The amygdala is critical for adaptive social behavior (Adolphs, 2010; Sander et al., 2003) and, possibly, for normal face perception and evaluation (Todorov, 2011). Large meta-analyses of PET and fMRI studies on emotional processing show that faces are one of the most potent stimuli for eliciting responses in the amygdala (Costafreda et al., 2008; Sergerie et al., 2008). The role of the amygdala in face evaluation is also consistent with neurophysiology findings of face selective responses in the amygdala (Nakamura et al., 1992; Rolls, 2000a,b; Gothard et al., 2007). The amygdala receives input from the inferior temporal (IT) cortex and projects back not only to IT cortex but also to extrastriate and striate visual areas (Amaral et al., 1992). The amygdala also has strong interconnections with rACC, OFC, mPFC, basal ganglia and anterior insula. This anatomical position of the amygdala allows for it to serve as an affective hub of information. The current findings, together with the findings of a recent ALE-based meta-analysis of a smaller and only partially overlapping set of 16 studies on face evaluation (Bzdok et al., 2011), further buttress the importance of the amygdala in face perception and evaluation.

Importantly, the amygdala responded not only to negatively evaluated faces but also to positively evaluated faces, consistent with meta-analyses of its responses to the valence of emotional expressions (Sergerie et al., 2008). Interestingly, we observed different loci of activation within the amygdala for linear and non-linear responses (Figure 4). Whereas a ventral portion responded more consistently to negative faces only, a dorsal portion of the amygdala responded more consistently to both negative and positive faces than to neutral faces. This dissociation of linear and non-linear responses in the human amygdala parallels the findings of a high-resolution fMRI study on non-human primates (Hoffman et al., 2007). Hoffman and colleagues observed a linear response in ventral portions of the amygdala (comprising the basolateral amygdala)—specifically, stronger responses to threatening faces and progressively weaker responses to neutral and appeasing faces. However, in a more dorsal portion (comprising the central nucleus and the bed nucleus of the stria terminalis), they observed a non-linear response—stronger responses to both threatening and appeasing faces than to neutral faces. This ventral/dorsal distinction also parallels a distinction made by Whalen and his colleagues (Whalen et al., 2001; Kim et al., 2003, Somerville, et al., 2006; Davis, et al., 2010). They have argued that while the ventral portion of the amygdala is involved in processing valence, the dorsal portion of the amygdala is recruited in determining the value of ambiguous information (e.g. expressions of surprise) in a given context. These authors suggest further that given the dorsal amygdala's response to surprised (Kim et al., 2003), fearful (Whalen et al., 1998, 2001) and happy faces (Breiter et al., 1996; Whalen et al., 1998), it may be tracking the salience of these faces, more generally. This hypothesis is consistent with the current findings.

Fig. 4.

Linear and non-linear response patterns in right amygdala. Blue indicates voxels more consistently active across non-linear contrasts, red indicates voxels more consistently active across negative evaluations, and green indicates voxels consistently active across positive evaluations. Blue and red clusters withstood height-based thresholding, while the green cluster withstood extent-based thresholding (P < 0.001).

These findings open the door to future work along those lines. One possibility is that there exist separate populations of neurons within the amygdala that code for stimulus valence and stimulus salience, respectively. Ultimately, the findings are in line with previous work proposing a shift away from conceptualizing the amygdala as simply a fear or threat module and instead toward an account of the amygdala as also tracking stimulus intensity (Anderson et al., 2003; Small et al., 2003) or motivational salience (Sander et al., 2003; Cunningham et al., 2008; Adolphs, 2010; Todorov, 2011). These findings also serve as an excellent reminder that one of the additional benefits of the meta-analytic method is the possibility of generating new, testable hypotheses for future research.

Faces that are tagged as affectively significant in the amygdala can be further processed in prefrontal regions, which, in turn, can serve to modulate amygdala activity. Prefrontal-amygdala connections have been explored in the vmPFC (Quirk et al., 2003, Heinz et al., 2004), as well as the pgACC (Pezawas et al., 2005; Stein et al., 2007; Zink et al., 2010), both of which were observed as consistently activated across our set of positive contrasts.

Stimulus effects on neuroimaging findings

We also performed several smaller MKDAs to compare between study type, within negative and positive linear responses. These more targeted MKDAs offered evidence that our negative and positive analyses were driven by untrustworthiness and attractiveness, respectively. These two sets of studies were associated with different loci of activations: the right amygdala was more consistently active as facial trustworthiness decreased, while the NAcc/caudate and vmPFC/pgACC were all more consistently active as facial attractiveness increased. This distinction mirrors the results we observed in our primary analyses. In contrast, no brain regions were consistently activated across contrasts where facial attractiveness decreased or facial trustworthiness increased, respectively, even when including ROI coordinates in the analyses. We should note that these contrasts—especially the comparison between positive linear responses in attractiveness and trustworthiness studies—are certainly unbalanced, rendering the results more suggestive than confirmative.

As noted in the introduction, given that attractiveness and trustworthiness judgments from faces are highly correlated, this pattern is puzzling. These differences between trustworthiness and attractiveness studies cannot be explained by researchers' a priori focus on different regions because our results hold even for whole brain analyses that did not include ROI coordinates from individual studies.

The apparent differences in the neural bases of attractiveness and trustworthiness are also puzzling in the context of studies that used the same set of faces to examine responses to facial attractiveness and trustworthiness (Todorov and Engell, 2008). Specifically, Todorov and Engell re-analyzed the data from Engell et al. (2007), using 14 different social judgments of the same set of faces. Most of the brain responses were accounted by a general valence dimension rather than by specific dimensions such as attractiveness and trustworthiness (both of these were highly correlated with this dimension).

What could be driving the differences in neuroimaging studies on attractiveness and trustworthiness? One possibility is that the type of faces used in these studies may lead to different responses. Specifically, a third variable that is correlated with both trustworthiness and attractiveness but could vary across sets of faces may account for such differences. One candidate is face typicality. Recently, Said and colleagues (2010) showed that coding face typicality is a more parsimonious explanation of prior findings of the involvement of the amygdala in face evaluation than coding face valence.

Face typicality could vary across data sets and lead to different results. For example, in many standardized data sets of natural faces, typicality is positively correlated with both attractiveness and trustworthiness judgments (Figure 5A). In studies using these stimuli, the amygdala shows stronger responses to more atypical faces that happened also to be more negative (Todorov and Engell, 2008). In studies using artificial stimuli created by a statistical model, the most atypical faces are faces at the extremes of the dimension. In such studies, the amygdala responds to more atypical faces that happened to be more positive or more negative (Said et al., 2010). The important distinction here is not between real and artificial faces. Judgments of artificial faces that have not been manipulated to exaggerate differences along social dimensions are linearly correlated with their perceived typicality. Finally, in attractiveness studies that use extremely attractive faces (e.g. Aharon et al., 2001), the most attractive faces may be the least typical (Figure 5B). In such studies, the amygdala may respond to both extremely attractive and extremely unattractive faces as observed in Winston et al. (2007).

Fig. 5.

The relationship between perceptions of face typicality and face attractiveness for (A) a standardized set of faces (the Karolinska set) and (B) faces sampled to be extremely attractive from websites of models. The judgments are shown in standardized units. Each point of the plots represents a face.

The typicality hypothesis predicts that faces that systematically differ in their perceived typicality may lead to different neural responses. In fact, in the contrast of attractiveness studies using extreme faces and studies using more typical faces, focusing specifically on studies employing implicit paradigms, we observed consistent patterns of activation in right amygdala and NAcc/caudate. This suggests that when task demands are controlled for, the driving force behind NAcc and caudate activations observed in these studies was the usage of extremely attractive, atypical faces. This result also lends additional support to the suggestion that extreme, atypical faces will drive amygdala activity, regardless of their trustworthiness. Further, the consistent activations in vmPFC and pgACC that were observed across attractiveness contrasts but not trustworthiness contrasts can also be accounted for by face typicality. Contrasting extreme attractiveness and trustworthiness continued to produce consistent activation in these regions, while contrasting more typical attractiveness against trustworthiness did not.

It may be the case that in the context of face evaluation, some of the regions implicated in reward processing are only activated upon the presence of real and extremely attractive faces, or the goal to evaluate face attractiveness, or some combination of stimulus features and task demands. Unfortunately, we do not have a sufficient number of studies to test for more specific effects.

Recommendations for studies using face stimuli

Our findings suggest that the type of face stimuli selected for a particular study matters a great deal. For example, using more ‘extreme’ faces resulted in more consistently observed activation in the NAcc. Given that stimuli are often selected in an ad hoc fashion and rarely shared among research groups, this complicates comparisons across studies. Moreover, it undermines the generalizability of results.

To overcome these problems, researchers need to use a shared set of stimuli, not necessarily the same stimuli but stimuli sampled from a common pool. One approach is to use parametrically manipulated faces generated by an explicitly specified statistical model (e.g. Oosterhof and Todorov, 2008; Todorov and Oosterhof, 2011). This approach has the benefit of providing researchers with a full spectrum of faces—one that is not biased towards one portion of a given dimension. Our laboratory has made a number of such databases available for academic research (http://webscript.princeton.edu/~tlab/databases/). However, artificial faces may not be the best stimuli for many investigators. In this case, it would be best to create a common bank of stimuli that are shared with other research groups. These stimuli could be validated on a number of important variables such as typicality, and these variables could be further used to facilitate comparisons across studies.

SUPPLEMENTARY DATA

Supplementary data are available at SCAN online.

Conflict of Interest

None declared.