Skip to main content
NCBI home page
Social Cognitive and Affective Neuroscience logoLink to Social Cognitive and Affective Neuroscience
. 2013 Aug 15;9(10):1481–1488. doi: 10.1093/scan/nst138

How affective information from faces and scenes interacts in the brain

Jan Van den Stock 1,✉, Mathieu Vandenbulcke 1,2, Charlotte B A Sinke 3,4, Rainer Goebel 3, Beatrice de Gelder 1,3,4
PMCID: PMC4187263  PMID: 23956081

Abstract

Facial expression perception can be influenced by the natural visual context in which the face is perceived. We performed an fMRI experiment presenting participants with fearful or neutral faces against threatening or neutral background scenes. Triangles and scrambled scenes served as control stimuli. The results showed that the valence of the background influences face selective activity in the right anterior parahippocampal place area (PPA) and subgenual anterior cingulate cortex (sgACC) with higher activation for neutral backgrounds compared to threatening backgrounds (controlled for isolated background effects) and that this effect correlated with trait empathy in the sgACC. In addition, the left fusiform gyrus (FG) responds to the affective congruence between face and background scene. The results show that valence of the background modulates face processing and support the hypothesis that empathic processing in sgACC is inhibited when affective information is present in the background. In addition, the findings reveal a pattern of complex scene perception showing a gradient of functional specialization along the posterior–anterior axis: from sensitivity to the affective content of scenes (extrastriate body area: EBA and posterior PPA), over scene emotion–face emotion interaction (left FG) via category–scene interaction (anterior PPA) to scene–category–personality interaction (sgACC).

Keywords: face, scene, emotion, ACC, PPA

INTRODUCTION

Social animals have developed a wide range of communication abilities on which their wellbeing in a society depends. Among the cognitive skills of social species is the ability to monitor each other’s behaviour and to adapt continuously to the social signals of others, whether collaborative or competitive (Darwin, 1872). In line with this, the human brain shows functional specialization to process emotional expressions displayed by conspecifics (Zhu et al., 2013). Although recognition of threatening events in the natural environment is at least equally critical for survival, affective neuroscience has traditionally focussed on perception of facial expressions. There is ample evidence that the brain contains dedicated mechanisms to process faces and facial emotions (Calder et al., 2011). However, recent reports show that the affective meaning of a face is modulated by contextual stimuli, for instance a background scene (de Gelder et al., 2006; de Gelder and Van den Stock, 2011; Van den Stock and de Gelder, 2012; Wieser and Brosch, 2012).

In the present study, we used fMRI to investigate the brain areas that are modulated by the affective valence of the visual background scene in which a face is presented. In previous ERP-studies, we observed an influence of the background scene on the face-responsive N170 in the left hemisphere (Righart and de Gelder, 2006, 2008), presumably originating in the fusiform gyrus (FG) (Pizzagalli et al., 2002; Iidaka et al., 2006). In addition, previous reports of affective context modulation on faces by preceding pictorial (Mobbs et al., 2006) and verbal (Kim et al., 2004) contexts reported activity in the left FG. Based on these findings, our hypothesis is that affective context modulation of facial expressions by background scenes will involve the left FG. Second, in line with our previous findings showing increased EBA activation for neutral bodies against a threatening compared to a neutral background (Van den Stock et al., 2012), we conjecture that affective context modulation for faces may involve face-responsive regions like the occipital (OFA) and fusiform face area (FFA) and superior temporal sulcus (STS) (Haxby and Gobbini, 2011).

Another type of influence on perception of faces is related to the observer. There is evidence that (non-clinical) personality traits like empathy, neuroticism and anxiety have an effect on behavioural (Wieser et al., 2009a,b) and neural processing (Hooker et al., 2010) of social emotional expressions (Fox and Zougkou, 2011). We included a measure of trait empathy to investigate its influence on affective context modulation.

MATERIALS AND METHODS

Participants

Fifteen subjects (six male; age 26.2 ± 5.9 years; all right-handed) participated in the experiment. One subject was excluded due to excessive movement in the scanner.

Stimuli

Backgrounds

Scenes of happy, threatening, neutral, sad or disgusting everyday situations were taken from our own database (Van den Stock et al., 2012). We selected three familiar scene categories (buildings, cars and landscapes) that involve the same objects but with different affective significances. Examples of stimuli are: a house on fire or a holiday cottage, a damaged car in an accident or a shiny new convertible. None of the scenes displayed humans or animals. In a pilot study the scenes were presented one by one for 4000 ms with a 4000-ms inter-stimulus interval. Participants were instructed to categorize as accurately and as quickly as possible the pictures according to the emotion they evoked (anger, fear, happiness, disgust, sadness or neutral). Based on the results, we selected 24 scenes (12 threatening and 12 neutral) for the present experiment (all recognized correctly above 70%). Each category contained four exemplars with a car, four with a building and four landscapes. We created scrambled versions of every scene, by dividing the image in 10 000 (100 × 100) squares and randomly rearranging the squares.

Faces

We selected 24 different face-stimuli (half male; half neutral, half fearful) from the Karolinska Directed Emotional Face database (Lundqvist et al., 1998).

Compound-stimuli

Scenes were combined with faces to create compound stimuli. In the centre of every scene, a face was overlaid. To avoid the impression of a ‘floating’ face, a geometrical torso-like shape was positioned underneath the face. Every scene stimulus was once overlaid with a fearful male face, once with a fearful female face, once with a neutral male face and once with a neutral female face, resulting in 96 realistic compound stimuli. We used the scrambled images as controls for the scenes and triangles as control for the faces. The 24 scrambled scenes were combined once with a fearful face and once with a neutral face, leading to an additional 48 compound stimuli. We also paired every scene with a white triangle (24 intact scenes + 24 scrambled scenes). These scene-triangle combinations were used as a control condition instead of only the scenes in order to maintain the same task in all conditions (see below) and to ensure all stimuli had a clear foreground/background structure. This procedure results in 192 compound stimuli. For examples, see Figure 1.

Fig. 1.

Fig. 1

Open in a new tab

Schematic overview and stimulus examples of the conditions in the 3 (face) × 3 (background) factorial design.

Procedure stimuli were presented in blocks of 9000 ms, separated by fixation blocks of 15 750 ms. Within a block, eight stimuli were presented for 800 ms with an inter-stimulusinterval of 370 ms, during which a grey screen was shown. In fixation blocks, a grey screen with a black fixation cross was presented. We used a 3 (face: fearful, neutral and triangle) × 3 (scene: threatening, neutral and scrambled) factorial design. Participants were given an oddball detection task and instructed to press the response button when a stimulus was shown upside-down. A run lasted 711 s and consisted of 31 experimental blocks and 32 fixation blocks. The order of the blocks was randomized. In 4 of the 31 blocks (13%) an oddball stimulus occurred, while the remaining 27 blocks were divided in 3 blocks of every condition. The experiment consisted of four runs. The methods are also described in Sinke et al. (2012). The study was performed in accordance with the Declaration of Helsinki and was approved by the Ethics Committee of the faculty of Psychology and Neuroscience (ECP Maastricht, the Netherlands).

We used a validated Dutch translation (Lijffijt et al., 2005) of the widely used I7-questionnaire developed by Eysenck and colleagues (1985) as a measure of self-reported empathy. The questionnaire consists of 54 dichotomous items and includes three scales: impulsiveness (19 items), venturesomeness (16 items) and empathy (19 items). One subject did not complete the questionnaire.

RESULTS

Imaging data were analysed using BrainVoyager QX (Goebel et al., 2006). The first two volumes of every functional run were discarded to allow for T1 equilibration. Pre-processing of the functional data included slice scan time correction (cubic spline interpolation), 3D motion correction (trilinear/sync interpolation), temporal filtering [high-pass GLM(General Linear Model)–Fourier of 2 sines/cosines] and Gaussian spatial smoothing (6 mm). Functional data were then co-registered with the anatomical volume and transferred into Talairach space.

The statistical analysis was based on the GLM, with each condition defined as a predictor plus one for the oddball.

Whole-brain analysis

As categorical effects are typically larger than emotional effects, we used a threshold of P < 50−4 (uncorrected) for the categorical comparisons and a threshold of P < 50−3 (uncorrected) for the emotional comparisons (see also Van den Stock et al., 2012). We first examined category selective responses. All face-conditions compared with all triangle conditions revealed activation in the well-known face-selective (FFA, OFA) areas as well as the amygdala. All scene conditions compared with all scramble conditions activated the expected scene areas: parahippocampal place area (PPA) (Epstein and Kanwisher, 1998), retrosplenial cortex (RSC) (Bar and Aminoff, 2003) and the transverse occipital sulcus (TOS) (Hasson et al., 2003). Second, we identified the regions showing within-category emotion effects. Fearful compared with neutral faces triggered activation in the right ventral premotor cortex and bilateral STS, while threatening vs neutral backgrounds activated the posterior PPA and region of the extrastriate body area (EBA) (Downing et al., 2001) in line with previous findings (Sinke et al., 2012; Van den Stock et al., 2012). The reverse contrasts did not reveal any significant activation. The results are displayed in the Supplementary Materials.

Context effects on faces

We identified the brain regions that respond differentially to faces in a threatening compared with a neutral background, controlled for isolated background effects: [(fearful face in threatening background + neutral face in threatening background) > (fearful face in neutral background + neutral face in neutral background)] > [(triangle in threatening background > triangle in neutral background)]. Surprisingly, this revealed no significant results. We also made the reverse comparison to determine the brain regions that respond differently to faces in a neutral compared with a threatening background: [(fearful face in neutral background + neutral face in neutral background) > (fearful face in threatening background + neutral face in threatening background)] > [(triangle in neutral background > triangle in threatening background)]. This revealed activity in the bilateral subgenual anterior cingulate cortex (sgACC) in the ventromedial prefrontal cortex (vmPFC) (Figure 2) and right anterior PPA (Figure 3).

Fig. 2.

Fig. 2

Open in a new tab

Results in the sgACC for face–background interaction effects. The left column shows the sgACC, responding differentially to neutral compared with threatening background containing faces, controlled for isolated background effects [(neutral face in threatening background > neutral face in neutral background) > (fearful face in threatening background > fearful face in neutral background)] (P < 0.005, uncorrected). The scatterplot on the top right shows the correlation with trait empathy scores. The bar plots on the bottom right show the beta-values of the relevant conditions.

Fig. 3.

Fig. 3

Open in a new tab

Results in PPA for object-background interaction effects. The left shows the PPA sub-region in red, responding differentially to neutral compared with threatening background containing faces, controlled for isolated background effects [(neutral face in threatening background > neutral face in neutral background) > (fearful face in threatening background > fearful face in neutral background)] (P < 0.005, uncorrected) with the Beta-values presented in the bar plot on the right. In addition, the activation of the same contrast in similar dataset with bodies instead of faces is shown in purple. The green patch shows the region responding more to threatening than to neutral scenes (P < 0.005, uncorrected), while the black contour outlines the PPA as defined by the contrast scenes vs scrambles (P < 0.0005, uncorrected).

To investigate the face-specificity of these effects, we performed an additional analysis on a dataset obtained with a similar study that used bodies instead of faces (Van den Stock et al., 2012). Note that the stimulus materials and design were exactly the same as in the present study, except for the face stimuli and participants. The results of the contrast [(fearful body in neutral background + neutral body in neutral background) > (fearful body in threatening background + neutral body in threatening background)] > [(triangle in neutral background > triangle in threatening background)] revealed activity in the right PPA, slightly overlapping the region obtained with the faces (8/368 voxels = 2%), but not in the sgACC, even under a more liberal threshold (P < 0.10, uncorrected) (Figure 3).

Summarizing the results from the present and previous (Van den Stock et al., 2012) study for the PPA, we observed:

  • More activation for threatening than for neutral backgrounds in the right posterior PPA (current study and Van den Stock et al., 2012)

  • More activation for neutral than for threatening backgrounds in the left anterior PPA (Van den Stock et al., 2012)

  • More activation for neutral than for threatening backgrounds containing a person (face or body, irrespective of emotion) in the right anterior PPA (current study and Van den Stock et al., 2012)

Context effects on facial expressions

Next, we investigated background effects that are selective for facial expressions of fear. We identified the brain regions that responded more to fearful faces in a threatening compared with a neutral background, controlled for the same background effect on neutral faces: (fearful face in threatening background > fearful face in neutral background) > (neutral face in threatening background > neutral face in neutral background). Note that this contrast also reflects the congruent vs incongruent face–background combinations. This revealed activity in the left FG. We examined whether this region falls within the face-responsive area of the FG by overlaying the smoothed left FFA obtained in another study (Frost and Goebel, 2012). The results are displayed in Figure 4 and show that the activation partly overlaps the posterior region of the face-responsive left FFA.

Fig. 4.

Fig. 4

Open in a new tab

Results of the contrast (fearful face in threatening background > fearful face in neutral background) > (neutral face in threatening background > neutral face in neutral background) in the left FG (in red) with the beta-values presented in the bar plot on the right. The activation is shown on a reconstruction of the average cortically aligned brains of all participants. Gyri are shown in light blue and sulci in dark blue. The purple region outlines the probabilistic face-responsive area obtained in another study (Frost and Goebel 2012). The purple to white colour coding refers to the percentage of subject overlap.

Region of interest analysis

We correlated the beta-differences of all the contrasts described above with the personality scales Empathy, Introversion and Venturesomeness obtained with the I7-questionnaire. This revealed only one significant correlation, i.e. between Empathy and the sgACC activity (r = −0.738, P = 0.004) (Figure 2). Figure 2 shows that participants with the highest scores on Empathy show more activity for faces in a threatening background, while participants with lower Empathy-scores show more activity for faces in a neutral context.

DISCUSSION

The primary research question of the present study was to identify the brain regions that are sensitive to affective valence of the visual background in which a face is presented. The results reveal a nuanced pattern of face–background interactions. First and contrary to our expectations, we found no evidence of brain areas responding more to faces in a threatening background than to faces in a neutral background, controlled for isolated background effects. On the other hand, the sgACC and right anterior PPA responded more to neutral than to threatening backgrounds surrounding faces, but not surrounding geometrical figures (triangles). Additionally, the results show that while similar background effects also obtain on bodies in the right anterior PPA, the effect in sgACC is face-specific and furthermore correlates with the empathic ability of the observer: higher empathy scores are related to smaller background effects on faces. Moreover, empathy modulates the direction of the context effect: a high empathy score is associated with increased activation for threatening compared with neutral backgrounds, while lower empathy scores reveal more activity for neutral than threatening backgrounds. Below, we discuss the main findings point by point.

sgACC

As shown in Figure 2, the activity reported in the sgACC reflects reduced negative activity and not positive activity. This is not an uncommon finding in the medial prefrontal cortex (mPFC). It has been suggested that during baseline condition or ‘rest’, self-reflexive thoughts that activate mPFC are triggered, which may explain reductions in Blood Oxygenation Level Dependent responses for conditions of interest compared with baseline (Gusnard et al., 2001; D'Argembeau et al., 2005; D'Argembeau et al., 2007).

The subgenual portion of the ACC has been associated with emotional processing (Whalen et al., 1998; Bush et al., 2000), also in the context of faces (Bzdok et al., 2012). For instance, a recent report shows that matching facial expressions triggers activity in the sgACC compared with matching geometrical shapes (Ball et al., 2012). The present results add to these findings as they provide evidence that the face-specific sgACC response is modulated by the valence of the visual background scene. This is compatible with the notion that the sgACC processes face-specific information that goes beyond mere decoding of the facial expression and includes processing of contextual information like the visual background, concurrently presented faces or animacy (Ball et al., 2012; Van den Stock et al., 2013). The correlation with personality traits like empathy as we observe here and social anxiety as reported by Ball et al. (2012) is in line with this argumentation. By this logic, one would expect that the sgACC activation is not only modulated by the personality of the observer, but also by his emotional state. Indeed, the sgACC has been implicated in emotion regulation in the normal population (Zald et al., 2002; Kohn et al., 2013), but also in the context of affective disorders (Davidson et al., 2002; Etkin and Wager, 2007). For instance, reductions of both structure and function of the sgACC have been reported in depression (Drevets et al., 1997), while deep brain stimulation of the sgACC is associated amelioration of depressive symptoms (Mayberg et al., 2005). The emotion regulation function of the sgACC also applies to face-related activation, as abnormal sgACC activity to faces has been reported in mood disorders (Lennox et al., 2004; Gotlib et al., 2005; Drevets et al., 2008; Haldane et al., 2008).

A psychological explanation for the present results pertains to the notion that it reflects cognitive mentalizing processes. These presumably occur at the implicit level, as the task (oddball detection) was unrelated to social cognition (Frith, 2012; Frith and Frith, 2012). It may be that faces in a neutral scene elicit cognitive state attributions (‘What is he thinking?’), while faces in threatening contexts primarily activate orienting responses (‘What is going on?’). The observed activity pattern is compatible with this hypothesis and also in line with clinical findings: syndromes characterized by deficits in empathy or other forms of social cognition like autism (Masten et al., 2011), schizophrenia (Koo et al., 2008) and fronto-temporal dementia (Rosen et al., 2005) have been associated with sgACC cortex abnormalities.

An alternative but not incompatible hypothesis relates to top–down mechanisms of object recognition. It has been suggested that perception of objects (for instance faces) triggers context-related processes, inter alia in the mPFC (Bar, 2003, 2004; Kveraga et al., 2011). Possibly, intrinsic contextual associations that are presumably richer for faces than for geometrical figures influence the visual background analysis. The results from the region of interest analysis are consistent with this hypothesis in the sense that faces may trigger stronger contextual associations in more empathic individuals, which may in turn reduce the effect of physically presented visual background. However, processing of contextual associations has been primarily associated with PPA and RSC and we will return to this notion below.

Right anterior PPA

Although the bar plot in Figure 2 reveals that the overall activation is higher for scenes containing triangles than for scenes containing faces, there is no significant effect of the affective valence of the background for triangles, contrary to faces. In other words, threatening backgrounds reduce activation in this area, but only when a face is present. However, it is less likely that this reflects face-specific processes, as this area partly overlapped with the same effect for bodies that we observed in a previous study (Van den Stock et al., 2012).

One possible explanation is that this region primarily sub serves topographic processing (Henderson et al., 2008; Kravitz et al., 2011; Troiani et al., 2012). It may be that the presence of a face triggers non-topographic processes inhibiting the spatial computations as they obtain for scenes without faces and that this is more pronounced when faces are presented against a threatening compared with a neutral background.

Another hypothesis is that context modulation is related to stronger affective contextual associations of faces and bodies, compared with geometrical figures. It has been reported that objects (Bar and Aminoff, 2003), faces (Bar et al., 2008a) and scenes (Bar et al., 2008b) that have elaborate contextual associations activate the PHC more than objects, faces or scenes with fewer contextual associations. In line with this, a recent study shows that the PPA is responsive to the object content of a visual scene (Harel et al., 2012). However, the finding that triangles trigger higher activation in the PPA is not compatible with the notion that faces and bodies presumably have more elaborate and stronger contextual associations than triangles.

The PPA as defined in the present study (all scenes compared with scrambled scenes) occupies a rather extensive area (see Figure 2 and Supplementary Materials). It is unlikely that this region as a whole constitutes a functional unit (Epstein, 2008). Although the PPA has been studied extensively, there have been few reports on functional subdivisions within. The present results add to our previous findings showing a posterior PPA activation for threatening compared with neutral scenes (Van den Stock et al., 2012) and additionally show that the more anterior region shows a reversed pattern as well as sensitivity to object content.

Left FG

One area in the left FG that falls largely within the face selective part of the FG responded to the face emotion–background emotion interaction with increased activity for threatening compared with neutral backgrounds on fearful facial expressions, while the background effect on the neutral faces was in the opposite direction with higher activation for neutral than for threatening backgrounds. This supports the hypothesis that the left FG is involved in processing the affective face–context congruency as has been previously reported for pictorial (Mobbs et al., 2006) and verbal semantic contexts (Kim et al., 2004). It is also consistent with electrophysiological measures of visual background context effects (Righart and de Gelder, 2006, 2008).

Right posterior PPA

A remarkable result of the present study is that nearly all regions (except the left FG) responding to face–background combinations show higher activation for neutral than for threatening backgrounds. This may reflect that threatening information conveyed by the environment is little influenced by categorical information contained by the stimulus like faces or bodies. However, as threatening scenes presumably engage more processes related to attention and emotion (Vuilleumier, 2005) than neutral scenes, we expected that this would be reflected in activation of respective brain areas. The results reveal only two extrastriate visual areas, i.e. the region of the right EBA and posterior PPA (Sinke et al., 2012; Van den Stock et al., 2012). Little is known about how the affective information present in a complex scene is processed, when controlling for the object content of a scene. Our findings provide an indication that at least two areas respond to the affective content of a scene, although an important question concerns the time point of activation. It has been reported that the posterior PPA is activated at different time points and for different task demands during scene perception with an early activation for passive scene perception and later activation during a spatial judgement task (allocentric judgements) (Bastin et al., 2012). Methods with higher temporal resolution like ERP or MEG may provide important insights into the mechanisms underlying content-constrained affective scene perception.

Amygdala

In line with other recent reports (Zhu et al., 2013), we observed face-selective activity in the amygdala (fearful + neutral faces vs triangles), but not for fearful vs neutral faces. We believe that the absence of amygdala activation for fearful vs neutral faces contrast is at least partly due to a methodological factor. It has been reported that amygdala are subject to rapid habituation (Zald, 2003) and we used a blocked design, which is more susceptible to adaptation effects compared with an event-related design. However, ever since the landmark case study by Adolphs et al. (1994), facial emotion research has strongly focused on the amygdala, which may have resulted in a confirmatory publication bias. Although there is general consensus that the amygdala is involved in fear processing, there is an increasing number of reports challenging its presumed critical role (van der Gaag et al., 2007; Tsuchiya et al., 2009; Terburg et al., 2012; Van den Stock et al., 2012; Zhu et al., 2013; Edmiston et al., 2013; Freeman and Luby, 2013). The present results add to this body of evidence and argue for a non-necessary involvement of the amygdala in fear processing.

CONCLUSION AND FUTURE DIRECTIONS

The present findings show that affective face–background congruence is primarily processed in the left FFA. In addition, the results reveal a pattern of complex scene perception with an increasing functional specialization along the posterior–anterior axis. The areas that respond to the affective content conveyed by the background are situated at the posterior extrastriate visual cortex: EBA and posterior PPA. This effect is modulated more anteriorly by the object content of the scene in the anterior PPA. A similar posterior to anterior gradient of object sensitivity has been reported in the medial temporal lobe (Liang et al., 2013) and this has been related to episodic encoding of an event (Preston et al., 2010). This may be particularly relevant for the understanding of the functional organization of occipito-temporal cortex relating to real-world perception. The reports of posterior–anterior gradient relating to object content are paralleled by studies reporting a medial–lateral gradient relating to real world object size, with larger objects represented more medially (Konkle and Oliva, 2012). The present results are in line with this as the anterior PPA region we observe for faces is located laterally to the neighbouring region we observed for bodies. It has been postulated that this functional organization is shaped by a congenital connectivity to similar domain-specific regions (Mahon and Caramazza, 2011). The present results extent this notion by revealing that the object sensitivity gradient is modulated more anteriorly in the sgACC by the personality of the observer, i.e. the empathic ability.

We have previously provided evidence that perception of facial expressions is also modulated by other contexts, for instance bodies (Meeren et al., 2005; Van den Stock et al., 2007; Kret et al., 2013) and these findings have been confirmed by subsequent studies in other labs (Aviezer et al., 2008). In addition to bodies, other face-contexts include voices (de Gelder et al., 2006; de Gelder and Van den Stock, 2011). It would be interesting to investigate whether similar context effects apply. We have recently provided evidence that recognition memory for neutral faces is more influenced by affective bodies than by affective backgrounds (presented during encoding). Considering the face-specificity of the sgACC effect we observe here and the rapid integration of face–body stimuli (Meeren et al., 2005), we hypothesize that this region may respond stronger to body than to background scene modulation of faces. In addition clinical studies in populations with sgACC abnormalities would provide valuable insights in the role of this region in face–context integration.

SUPPLEMENTARY DATA

Supplementary data are available at SCAN online.

Conflict of Interest

None declared.

Supplementary Material

Supplementary Data
supp_9_10_1481__index.html (1KB, html)

Acknowledgments

This work was supported by FWO (Fonds Wetenschappelijk Onderzoek)-Vlaanderen [1.51072.13N to J.V.d.S. and G.0746.09 to M.V.]; Program Financing KU Leuven [PFV/10/008 to M.V.]; and European Commission [FP7-FET-ICT, project Tango and ERC-AdvG to B.D.G].

REFERENCES

  1. Adolphs R, Tranel D, Damasio H, Damasio A. Impaired recognition of emotion in facial expressions following bilateral damage to the human amygdala. Nature. 1994;372(6507):669–72. doi: 10.1038/372669a0. [DOI] [PubMed] [Google Scholar]
  2. Aviezer H, Hassin RR, Ryan J, et al. Angry, disgusted, or afraid? Studies on the malleability of emotion perception. Psychological Science. 2008;19(7):724–32. doi: 10.1111/j.1467-9280.2008.02148.x. [DOI] [PubMed] [Google Scholar]
  3. Ball TM, Sullivan S, Flagan T, et al. Selective effects of social anxiety, anxiety sensitivity, and negative affectivity on the neural bases of emotional face processing. Neuroimage. 2012;59(2):1879–87. doi: 10.1016/j.neuroimage.2011.08.074. [DOI] [PubMed] [Google Scholar]
  4. Bar M. A cortical mechanism for triggering top-down facilitation in visual object recognition. Journal of Cognitive Neuroscience. 2003;15(4):600–9. doi: 10.1162/089892903321662976. [DOI] [PubMed] [Google Scholar]
  5. Bar M. Visual objects in context. Nature Reviews Neuroscience. 2004;5(8):617–29. doi: 10.1038/nrn1476. [DOI] [PubMed] [Google Scholar]
  6. Bar M, Aminoff E. Cortical analysis of visual context. Neuron. 2003;38(2):347–58. doi: 10.1016/s0896-6273(03)00167-3. [DOI] [PubMed] [Google Scholar]
  7. Bar M, Aminoff E, Ishai A. Famous faces activate contextual associations in the parahippocampal cortex. Cerebral Cortex. 2008a;18(6):1233–38. doi: 10.1093/cercor/bhm170. [DOI] [PubMed] [Google Scholar]
  8. Bar M, Aminoff E, Schacter DL. Scenes unseen: the parahippocampal cortex intrinsically subserves contextual associations, not scenes or places per se. The Journal of Neuroscience. 2008b;28(34):8539–44. doi: 10.1523/JNEUROSCI.0987-08.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Bastin J, Committeri G, Kahane P, et al. Timing of posterior parahippocampal gyrus activity reveals multiple scene processing stages. Human Brain Mapping. 2012;34(6):1357–70. doi: 10.1002/hbm.21515. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Bush G, Luu P, Posner MI. Cognitive and emotional influences in anterior cingulate cortex. Trends in Cognitive Sciences. 2000;4(6):215–22. doi: 10.1016/s1364-6613(00)01483-2. [DOI] [PubMed] [Google Scholar]
  11. Bzdok D, Langner R, Hoffstaedter F, Turetsky B, Zilles K, Eickhoff S. The modular neuroarchitecture of social judgments on faces. Cerebral Cortex. 2012;22(4):951–61. doi: 10.1093/cercor/bhr166. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Calder AJ, Rhodes G, Johnson MH, Haxby JV. The Oxford Handbook of Face Perception. New York, NY: Oxford University Press; 2011. [Google Scholar]
  13. D'Argembeau A, Collette F, Van der Linden M, et al. Self-referential reflective activity and its relationship with rest: a PET study. Neuroimage. 2005;25(2):616–24. doi: 10.1016/j.neuroimage.2004.11.048. [DOI] [PubMed] [Google Scholar]
  14. D'Argembeau A, Ruby P, Collette F, et al. Distinct regions of the medial prefrontal cortex are associated with self-referential processing and perspective taking. Journal of Cognitive Neuroscience. 2007;19(6):935–44. doi: 10.1162/jocn.2007.19.6.935. [DOI] [PubMed] [Google Scholar]
  15. Darwin C. The Expression of Emotions in Man and Animals. Oxford, UK: Oxford University Press; 1872. 1998. [Google Scholar]
  16. Davidson RJ, Pizzagalli D, Nitschke JB, Putnam K. Depression: perspectives from affective neuroscience. Annual Review of Psychology. 2002;53:545–74. doi: 10.1146/annurev.psych.53.100901.135148. [DOI] [PubMed] [Google Scholar]
  17. de Gelder B, Meeren HK, Righart R, van den Stock J, van de Riet WA, Tamietto M. Beyond the face: exploring rapid influences of context on face processing. Progress in Brain Research. 2006;155:37–48. doi: 10.1016/S0079-6123(06)55003-4. [DOI] [PubMed] [Google Scholar]
  18. de Gelder B, Van den Stock J. Real faces, real emotions: perceiving facial expressions in naturalistic contexts of voices, bodies and scenes. In: Calder AJ, Rhodes G, Johnson MH, Haxby JV, editors. The Oxford Handbook of Face Perception. New York, NY: Oxford University Press; 2011. [Google Scholar]
  19. Downing PE, Jiang Y, Shuman M, Kanwisher N. A cortical area selective for visual processing of the human body. Science. 2001;293(5539):2470–3. doi: 10.1126/science.1063414. [DOI] [PubMed] [Google Scholar]
  20. Drevets WC, Price JL, Simpson JR, Jr, et al. Subgenual prefrontal cortex abnormalities in mood disorders. Nature. 1997;386(6627):824–7. doi: 10.1038/386824a0. [DOI] [PubMed] [Google Scholar]
  21. Drevets WC, Savitz J, Trimble M. The subgenual anterior cingulate cortex in mood disorders. CNS Spectr. 2008;13(8):663–81. doi: 10.1017/s1092852900013754. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Edmiston EK, McHugo M, Dukic MS, et al. Enhanced visual cortical activation for emotional stimuli is preserved in patients with unilateral amygdala resection. The Journal of Neuroscience. 2013;33(27):11023–31. doi: 10.1523/JNEUROSCI.0401-13.2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Epstein R, Kanwisher N. A cortical representation of the local visual environment. Nature. 1998;392(6676):598–601. doi: 10.1038/33402. [DOI] [PubMed] [Google Scholar]
  24. Epstein RA. Parahippocampal and retrosplenial contributions to human spatial navigation. Trends in Cognitive Sciences. 2008;12(10):388–96. doi: 10.1016/j.tics.2008.07.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Etkin A, Wager TD. Functional neuroimaging of anxiety: a meta-analysis of emotional processing in PTSD, social anxiety disorder, and specific phobia. American Journal of Psychiatry. 2007;164(10):1476–88. doi: 10.1176/appi.ajp.2007.07030504. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Eysenck SBG, Pearson PR, Easting G, Allsopp JF. Age norms for impulsiveness, venturesomeness and empathy in adults. Personality and Individual Differences. 1985;6:613–9. [Google Scholar]
  27. Fox E, Zougkou K. Influence of personality traits on processing of facial expressions. In: Calder AJ, Rhodes G, Johnson MH, Haxby JV, editors. The Oxford Handbook of Face Perception. New York, NY: Oxford University Press; 2011. [Google Scholar]
  28. Freeman TW, Luby J. A five-year-old child with bilateral amygdala loss. The Journal of Neuropsychiatry and Clinical Neurosciences. 2013;25(2):E15–17. doi: 10.1176/appi.neuropsych.12030068. [DOI] [PubMed] [Google Scholar]
  29. Frith CD. The role of metacognition in human social interactions. Philosophical Transactions of the Royal Society of London B: Biological. 2012;367(1599):2213–23. doi: 10.1098/rstb.2012.0123. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Frith CD, Frith U. Mechanisms of social cognition. Annual Review of Psychology. 2012;63:287–313. doi: 10.1146/annurev-psych-120710-100449. [DOI] [PubMed] [Google Scholar]
  31. Frost MA, Goebel R. Measuring structural-functional correspondence: spatial variability of specialised brain regions after macro-anatomical alignment. Neuroimage. 2012;59(2):1369–81. doi: 10.1016/j.neuroimage.2011.08.035. [DOI] [PubMed] [Google Scholar]
  32. Goebel R, Esposito F, Formisano E. Analysis of functional image analysis contest (FIAC). data with brainvoyager QX: from single-subject to cortically aligned group general linear model analysis and self-organizing group independent component analysis. Human Brain Mapping. 2006;27(5):392–401. doi: 10.1002/hbm.20249. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Gotlib IH, Sivers H, Gabrieli JD, et al. Subgenual anterior cingulate activation to valenced emotional stimuli in major depression. Neuroreport. 2005;16(16):1731–4. doi: 10.1097/01.wnr.0000183901.70030.82. [DOI] [PubMed] [Google Scholar]
  34. Gusnard DA, Raichle ME, Raichle ME. Searching for a baseline: functional imaging and the resting human brain. Nature Reviews Neuroscience. 2001;2(10):685–94. doi: 10.1038/35094500. [DOI] [PubMed] [Google Scholar]
  35. Haldane M, Jogia J, Cobb A, Kozuch E, Kumari V, Frangou S. Changes in brain activation during working memory and facial recognition tasks in patients with bipolar disorder with Lamotrigine monotherapy. European Neuropsychopharmacology. 2008;18(1):48–54. doi: 10.1016/j.euroneuro.2007.05.009. [DOI] [PubMed] [Google Scholar]
  36. Harel A, Kravitz DJ, Baker CI. Deconstructing visual scenes in cortex: gradients of object and spatial layout information. Cerebral Cortex. 2012;23(4):947–57. doi: 10.1093/cercor/bhs091. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Hasson U, Harel M, Levy I, Malach R. Large-scale mirror-symmetry organization of human occipito-temporal object areas. Neuron. 2003;37(6):1027–41. doi: 10.1016/s0896-6273(03)00144-2. [DOI] [PubMed] [Google Scholar]
  38. Haxby JV, Gobbini MI. Distributed neural systems for face perception. In: Calder AJ, Rhodes G, Johnson M, editors. The Oxford Handbook of Face Perception. New York, NY: Oxford University Press; 2011. [Google Scholar]
  39. Henderson J, Larson C, Zhu D. Full scenes produce more activation than close-up scenes and scene-diagnostic objects in parahippocampal and retrosplenial cortex: an fMRI study. Brain and Cognition. 2008;66(1):40–9. doi: 10.1016/j.bandc.2007.05.001. [DOI] [PubMed] [Google Scholar]
  40. Hooker CI, Verosky SC, Germine LT, Knight RT, D'Esposito M. Neural activity during social signal perception correlates with self-reported empathy. Brain Research. 2010;1308:100–13. doi: 10.1016/j.brainres.2009.10.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Iidaka T, Matsumoto A, Haneda K, Okada T, Sadato N. Hemodynamic and electrophysiological relationship involved in human face processing: evidence from a combined fMRI-ERP study. Brain and Cognition. 2006;60(2):176–86. doi: 10.1016/j.bandc.2005.11.004. [DOI] [PubMed] [Google Scholar]
  42. Kim H, Somerville LH, Johnstone T, et al. Contextual modulation of amygdala responsivity to surprised faces. Journal of Cognitive Neuroscience. 2004;16(10):1730–45. doi: 10.1162/0898929042947865. [DOI] [PubMed] [Google Scholar]
  43. Kohn N, Falkenberg I, Kellermann T, Eickhoff SB, Gur RC, Habel U. Neural correlates of effective and ineffective mood induction. Social Cognitive & Affective Neuroscience. 2013 doi: 10.1093/scan/nst055. in press. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Konkle T, Oliva A. A real-world size organization of object responses in occipitotemporal cortex. Neuron. 2012;74(6):1114–24. doi: 10.1016/j.neuron.2012.04.036. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Koo MS, Levitt JJ, Salisbury DF, Nakamura M, Shenton ME, McCarley RW. A cross-sectional and longitudinal magnetic resonance imaging study of cingulate gyrus gray matter volume abnormalities in first-episode schizophrenia and first-episode affective psychosis. Archives of General Psychiatry. 2008;65(7):746–60. doi: 10.1001/archpsyc.65.7.746. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Kravitz DJ, Peng CS, Baker CI. Real-world scene representations in high-level visual cortex: it's the spaces more than the places. The Journal of Neuroscience. 2011;31(20):7322–33. doi: 10.1523/JNEUROSCI.4588-10.2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Kret ME, Stekelenburg JJ, Roelofs K, de Gelder B. Perception of face and body expressions using electromyography, pupillometry and gaze measures. Frontiers in Psychology. 2013;4:28. doi: 10.3389/fpsyg.2013.00028. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Kveraga K, Ghuman AS, Kassam KS, et al. Early onset of neural synchronization in the contextual associations network. Proceeding of the National Academy of Sciences of the United States of America. 2011;108(8):3389–94. doi: 10.1073/pnas.1013760108. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Lennox BR, Jacob R, Calder AJ, Lupson V, Bullmore ET. Behavioural and neurocognitive responses to sad facial affect are attenuated in patients with mania. Psychology Medicine. 2004;34(5):795–802. doi: 10.1017/s0033291704002557. [DOI] [PubMed] [Google Scholar]
  50. Liang JC, Wagner AD, Preston AR. Content representation in the human medial temporal lobe. Cerebral Cortex. 2013;23(1):80–96. doi: 10.1093/cercor/bhr379. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Lijffijt M, Caci H, Kenemans JL. Validation of the Dutch translation of the I7 questionnaire. Personality and Individual Differences. 2005;38:1123–33. [Google Scholar]
  52. Lundqvist D, Flykt A, Öhman A. The Karolinska Directed Emotional Faces - KDEF. Stockholm: Karolinska Institutet; 1998. [Google Scholar]
  53. Mahon BZ, Caramazza A. What drives the organization of object knowledge in the brain? Trends in Cognitive Sciences. 2011;15(3):97–103. doi: 10.1016/j.tics.2011.01.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Masten CL, Colich NL, Rudie JD, Bookheimer SY, Eisenberger NI, Dapretto M. An fMRI investigation of responses to peer rejection in adolescents with autism spectrum disorders. Developmental Cognitive Neuroscience. 2011;1(3):260–70. doi: 10.1016/j.dcn.2011.01.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Mayberg HS, Lozano AM, Voon V, et al. Deep brain stimulation for treatment-resistant depression. Neuron. 2005;45(5):651–60. doi: 10.1016/j.neuron.2005.02.014. [DOI] [PubMed] [Google Scholar]
  56. Meeren HK, van Heijnsbergen CC, de Gelder B. Rapid perceptual integration of facial expression and emotional body language. Proceedings of the National Academy of Sciencesof the United States of America. 2005;102(45):16518–23. doi: 10.1073/pnas.0507650102. [DOI] [PMC free article] [PubMed] [Google Scholar]
  57. Mobbs D, Weiskopf N, Lau HC, Featherstone E, Dolan RJ, Frith CD. The Kuleshov Effect: the influence of contextual framing on emotional attributions. Social Cognitive & Affective Neuroscience. 2006;1(2):95–106. doi: 10.1093/scan/nsl014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  58. Pizzagalli DA, Lehmann D, Hendrick AM, Regard M, Pascual-Marqui RD, Davidson RJ. Affective judgments of faces modulate early activity (approximately 160 ms). within the fusiform gyri. Neuroimage. 2002;16(3 Pt 1):663–77. doi: 10.1006/nimg.2002.1126. [DOI] [PubMed] [Google Scholar]
  59. Preston A, Bornstein A, Hutchinson J, Gaare M, Glover G, Wagner A. High-resolution fMRI of content-sensitive subsequent memory responses in human medial temporal lobe. Journal of Cognitive Neuroscience. 2010;22(1):156–73. doi: 10.1162/jocn.2009.21195. [DOI] [PMC free article] [PubMed] [Google Scholar]
  60. Righart R, de Gelder B. Context influences early perceptual analysis of faces–an electrophysiological study. Cerebral Cortex. 2006;16(9):1249–57. doi: 10.1093/cercor/bhj066. [DOI] [PubMed] [Google Scholar]
  61. Righart R, de Gelder B. Rapid influence of emotional scenes on encoding of facial expressions: an ERP study. Social Cognitive & Affective Neuroscience. 2008;3(3):270–8. doi: 10.1093/scan/nsn021. [DOI] [PMC free article] [PubMed] [Google Scholar]
  62. Rosen HJ, Allison SC, Schauer GF, Gorno-Tempini ML, Weiner MW, Miller BL. Neuroanatomical correlates of behavioural disorders in dementia. Brain. 2005;128(Pt 11):2612–25. doi: 10.1093/brain/awh628. [DOI] [PMC free article] [PubMed] [Google Scholar]
  63. Sinke CB, Van den Stock J, Goebel R, de Gelder B. The constructive nature of affective vision: seeing fearful scenes activates extrastriate body area. PLoS One. 2012;7(6):e38118. doi: 10.1371/journal.pone.0038118. [DOI] [PMC free article] [PubMed] [Google Scholar]
  64. Terburg D, Morgan BE, Montoya ER, et al. Hypervigilance for fear after basolateral amygdala damage in humans. Translational Psychiatry. 2012;2:e115. doi: 10.1038/tp.2012.46. [DOI] [PMC free article] [PubMed] [Google Scholar]
  65. Troiani V, Stigliani A, Smith M, Epstein R. Multiple object properties drive scene-selective regions. Cerebral Cortex. 2012 doi: 10.1093/cercor/bhs364. in press. [DOI] [PMC free article] [PubMed] [Google Scholar]
  66. Tsuchiya N, Moradi F, Felsen C, Yamazaki M, Adolphs R. Intact rapid detection of fearful faces in the absence of the amygdala. Nature Neuroscience. 2009;12(10):1224–5. doi: 10.1038/nn.2380. [DOI] [PMC free article] [PubMed] [Google Scholar]
  67. Van den Stock J, de Gelder B. Emotional information in body and background hampers recognition memory for faces. Neurobiology of Learning and Memory. 2012;97(3):321–5. doi: 10.1016/j.nlm.2012.01.007. [DOI] [PubMed] [Google Scholar]
  68. Van den Stock J, de Gelder B, Van Laere K, Vandenbulcke M. Face-selective hyper-animacy and hyper-familiarity misperception in a patient with moderate Alzheimer’s disease. Journal of Neuropsychiatry Clinical Neurosciences. 2013 doi: 10.1176/appi.neuropsych.12120390. in press. [DOI] [PubMed] [Google Scholar]
  69. Van den Stock J, Righart R, de Gelder B. Body expressions influence recognition of emotions in the face and voice. Emotion. 2007;7(3):487–94. doi: 10.1037/1528-3542.7.3.487. [DOI] [PubMed] [Google Scholar]
  70. Van den Stock J, Vandenbulcke M, Sinke CBA, de Gelder B. Affective scenes influence fear perception of individual body expressions. Human Brain Mapping. 2014;35(2):492–502. doi: 10.1002/hbm.22195. [DOI] [PMC free article] [PubMed] [Google Scholar]
  71. van der Gaag C, Minderaa RB, Keysers C. The BOLD signal in the amygdala does not differentiate between dynamic facial expressions. Social Cognitive & Affective Neuroscience. 2007;2(2):93–103. doi: 10.1093/scan/nsm002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  72. Vuilleumier P. How brains beware: neural mechanisms of emotional attention. Trends in Cognitive Sciences. 2005;9(12):585–94. doi: 10.1016/j.tics.2005.10.011. [DOI] [PubMed] [Google Scholar]
  73. Whalen PJ, Bush G, McNally RJ, et al. The emotional counting Stroop paradigm: a functional magnetic resonance imaging probe of the anterior cingulate affective division. Biological Psychiatry. 1998;44(12):1219–28. doi: 10.1016/s0006-3223(98)00251-0. [DOI] [PubMed] [Google Scholar]
  74. Wieser MJ, Brosch T. Faces in context: a review and systematization of contextual influences on affective face processing. Frontiers in Psychology. 2012;3:471. doi: 10.3389/fpsyg.2012.00471. [DOI] [PMC free article] [PubMed] [Google Scholar]
  75. Wieser MJ, Pauli P, Alpers GW, Muhlberger A. Is eye to eye contact really threatening and avoided in social anxiety?–An eye-tracking and psychophysiology study. Journal of Anxiety Disorders. 2009a;23(1):93–103. doi: 10.1016/j.janxdis.2008.04.004. [DOI] [PubMed] [Google Scholar]
  76. Wieser MJ, Pauli P, Weyers P, Alpers GW, Muhlberger A. Fear of negative evaluation and the hypervigilance-avoidance hypothesis: an eye-tracking study. Journal of Neural Transmission. 2009b;116(6):717–23. doi: 10.1007/s00702-008-0101-0. [DOI] [PubMed] [Google Scholar]
  77. Zald DH. The human amygdala and the emotional evaluation of sensory stimuli. Brain Research Brain Research Reviews. 2003;41(1):88–123. doi: 10.1016/s0165-0173(02)00248-5. [DOI] [PubMed] [Google Scholar]
  78. Zald DH, Mattson DL, Pardo JV. Brain activity in ventromedial prefrontal cortex correlates with individual differences in negative affect. Proceeding of the National Academy of Sciences of the Unites States of America. 2002;99(4):2450–4. doi: 10.1073/pnas.042457199. [DOI] [PMC free article] [PubMed] [Google Scholar]
  79. Zhu Q, Nelissen K, Van den Stock J, et al. Dissimilar processing of emotional facial expressions in human and monkey temporal cortex. Neuroimage. 2013;66:402–411. doi: 10.1016/j.neuroimage.2012.10.083. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary Data
supp_9_10_1481__index.html (1KB, html)
supp_nst138_scan-13-139-File006.tif (4.8MB, tif)
supp_nst138_scan-13-139-File007.doc (22.5KB, doc)

Articles from Social Cognitive and Affective Neuroscience are provided here courtesy of Oxford University Press

RESOURCES