Frontolimbic Responses to Emotional Face Memory: The Neural Correlates of First Impressions

Abstract

First impressions, especially of emotional faces, may critically impact later evaluation of social interactions. Activity in limbic regions, including the amygdala and ventral striatum, has previously been shown to correlate with identification of emotional content in faces; however, little work has been done describing how these signals may influence emotional face memory. We report an event-related fMRI study in 21 healthy adults where subjects attempted to recognize a neutral face that was previously viewed with a threatening (angry or fearful) or non-threatening (happy or sad) affect. In a hypothesis-driven region of interest analysis, we found that neutral faces previously presented with a threatening affect recruited the left amygdala. In contrast, faces previously presented with a non-threatening affect activated the left ventral striatum. A whole-brain analysis revealed increased response in the right orbitofrontal cortex to faces previously seen with threatening affect. These effects of prior emotion were independent of task performance, with differences being seen in the amygdala and ventral striatum even if only incorrect trials were considered. The results indicate that a network of frontolimbic regions may provide emotional bias signals during facial recognition.

Introduction

The neural architecture for the recognition of faces has been well described, with multiple studies implicating a network of brain regions including the fusiform gyrus and the superior temporal sulcus (Vuilleumier and Pourtois 2007). Prior research also suggests that the emotional information conveyed by facial affect may have similarities to other emotional stimuli (Britton, et al. 2006b) and be processed by a network of limbic and cortical regions including the amygdala, ventral striatum, and orbitofrontal cortex (Hariri, et al. 2003; Killgore and Yurgelun-Todd 2004; Loughead, et al. 2008). Building on a large literature examining fear conditioning in rodents (LeDoux 2000), multiple functional magnetic resonance imaging (fMRI) experiments have demonstrated that the amygdala responds robustly to threatening faces (Breiter, et al. 1996; Morris, et al. 1996; Phan, et al. 2002). Similarly, the orbitofrontal cortex (OFC) has been found to respond to displays of angry faces, suggesting that it may be important in modulating the emotional context of facial expression (Blair, et al. 1999; Dougherty, et al. 1999), or may be involved in the inhibition of a behavioral response to a perceived threat (Coccaro, et al. 2007). In contrast, the ventral striatum is most often associated with reward-related behavior in both electrophysiological studies in animals (Schultz 2002) and human neuroimaging experiments (Knutson, et al. 2001; Monk, et al. 2008; O'Doherty, et al. 2001). Several imaging studies have demonstrated that the ventral striatum is recruited by the rewarding properties of beautiful or happy faces (Aharon, et al. 2001; Monk, et al. 2008). However, other studies suggest that amygdala (Davis and Whalen 2001; Fitzgerald, et al. 2006) and ventral striatal responses (Zink, et al. 2006; Zink, et al. 2003) are better explained by salience than by specific threat or reward-related properties.

This body of literature has almost uniformly investigated the neural correlates of passive viewing of emotional faces or facial affect identification. Beyond one small study, which found insular activation to the memory of neutral faces initially paired with a description of a disgusting behavior (Todorov, et al. 2007), remarkably little research has investigated the neural correlates of emotional memory of faces (Gobbini and Haxby 2007). The aphorism “You never have a second chance to make a first impression” attests to the importance of the affective content of a face when it is first encountered, as such information inevitably informs the interpretation of subsequent social interactions (Pessoa 2005; Sommer, et al. 2008). Identifying the neural basis of this phenomena is important for further understanding of both social cognition and its dysfunction in psychiatric disorders such as schizophrenia (Gur, et al. 2007; Holt, et al. 2006; Holt, et al. 2005; Kohler, et al. 2000; Mathalon and Ford 2008), post-traumatic stress disorder (Liberzon, et al. 2007), and depression (Fu, et al. 2007; Fu, et al. 2008).

We investigated the neural correlates of emotional face memory using an incidental memory paradigm and event-related fMRI. Subjects made an ‘old’ vs. ‘new’ recognition judgment regarding previously viewed and novel faces. Importantly, the ‘old’ faces had been originally presented with either threatening (angry or fearful) or non-threatening (happy or sad) facial expressions, while in the forced-choice recognition task all faces were shown with neutral expressions. We hypothesized that faces displaying a threatening affect on “first impression” would recruit the amygdala during the recognition task despite their current neutral affect. In contrast, we expected that neutral faces initially displayed with a non-threatening affect would activate the ventral striatum. As described below, these hypotheses were supported. Furthermore, in an exploratory whole-brain analysis, we found that faces previously viewed with threatening expressions recruited the orbitofrontal cortex. These results delineate a frontolimbic network that may provide emotional bias signals to modulate facial memory, integrating critical information learned from an affect-laden first impression.

Experimental Procedures

Subjects

We studied 24 right-handed participants, who were free from psychiatric or neurologic comorbidity. After a complete description of the study, subjects provided written informed consent. Three subjects were excluded for excessive in-scanner motion (mean volume-to-volume displacement or mean absolute displacement greater than two standard deviations above the group average). This left a final sample of 21 subjects (47.6% male, mean age 32.0 years, SD=8.6).

Task

The face recognition experiment was preceded by an emotion identification task similar to those previously reported by our laboratory (Gur, et al. 2002b; Gur, et al. 2002c; Loughead, et al. 2008). In the emotion identification experiment, subjects viewed 30 faces displaying neutral, happy, sad, angry, or fearful affect, and were asked to label the emotion displayed (Figure 1A). Stimuli construction and validation are described in detail elsewhere (Gur, et al. 2002a). Briefly, the stimuli were color photographs of actors (50% female) who had volunteered to participate in a study on emotion. They were coached by professional directors to express facial displays of different emotions. For the purpose of constructing the affect identification task, a subset of 30 intense expressions was selected based on high degree of accuracy of identification by the raters. During the affect identification task, subjects were not instructed to remember the faces or informed of the memory component of the experiment. The emotion identification and the face recognition experiments were separated by a diffusion tensor imaging acquisition lasting 10 minutes. Results from the emotion identification task will be reported elsewhere.

Figure 1.

Experimental Paradigm. A. Encoding task. Subjects initially performed an emotion identification task in which they identified the facial affect displayed. Four emotional labels were available, including two non-threatening affects (happy and sad), two threatening affects (angry and fearful), and neutral. Subjects were not instructed to remember the faces displayed. B. Face recognition task. Following the affect identification task, subjects were asked to make a forced-choice facial recognition judgment. Thirty faces (targets) from the affect identification task and thirty novel faces (foils) were displayed for 2 seconds each. Subjects made a simple ‘old’ vs. ‘new’ judgment as to whether the face had been previously displayed in the affect identification task. Faces were separated by a variable (0-12s) interval of crosshair fixation.

The face recognition experiment (Figure 1B) presented 30 faces from the emotion identification experiment (targets) along with 30 novel faces (foils). Faces were displayed for two seconds and subjects were asked to make a simple ‘old’ vs. ‘new’ judgment using a two button response pad. Target faces initially presented with happy, sad, angry, or fearful affect were displayed with neutral expressions. Foil stimuli not previously seen were also displayed with a neutral expression. Faces shown with neutral expressions in the emotion identification experiment were treated as a covariate of no interest in the face recognition analysis. Interpretation of responses to neutral faces was confounded by their congruent affect in the emotion identification experiment: whereas target threat or non-threat faces were displayed initially with a strong affect in the emotion identification experiment and subsequently re-presented with a neutral affect in the face recognition experiment, neutral faces were displayed with the same expression in both experiments, making neutral trials difficult to compare to other targets. Previous experiments in our laboratory (Gur, et al. 2002b; Loughead, et al. 2008) have conceptualized emotions along the dimension of threat-relatedness. In the current experiment, faces originally displayed with an angry or fearful affect were modeled together as threat; faces originally displayed with a happy or sad affect were modeled as non-threat. This grouping of emotions is suggested by the theories of Gray (Gray 1990), by previous data from our laboratory (Loughead, et al. 2008), as well as other studies (Hariri, et al. 2000; Suslow, et al. 2006). Overall, there were 6 trials per emotion, yielding 12 threat and 12 non-threat trials per subject. Interspersed with the 60 task trials (30 targets, 30 foil) were 60 baseline trials of variable duration (0 - 12 seconds) during which a crosshair fixation point was displayed on a complex background (degraded face). No stimulus was presented twice and total task duration was 4 minutes and 16 seconds.

fMRI Procedures

Participants were required to demonstrate understanding of the task instructions, the response device, and complete one trial of practice prior to acquisition of face memory data. Earplugs were used to muffle scanner noise and head fixation was aided by foam-rubber restraints mounted on the head coil. Stimuli were rear-projected to the center of the visual field using a PowerLite 7300 video projector (Epson America, Inc.; Long Beach, CA) and viewed through a head coil mounted mirror. Stimulus presentation was synchronized with image acquisition using the Presentation software package (Neurobehavioral Systems, Inc., Albany, CA). Subjects were randomly assigned to use their dominant or non-dominant hand; responses were recorded via a non-ferromagnetic keypad (fORP; Current Designs, Inc.; Philadelphia, PA).

Image Acquisition

BOLD fMRI was acquired with a Siemens Trio 3 Tesla (Erlangen, Germany) system using a whole-brain, single-shot gradient-echo (GE) echoplanar (EPI) sequence with the following parameters: TR/TE=2000/32 ms, FOV=220 mm, matrix= 64 × 64, slice thickness/gap=3/0mm, 40 slices, effective voxel resolution of 3 × 3 × 3mm. To reduce partial volume effects in orbitofrontal regions, EPI was acquired obliquely (axial/coronal). The slices provided adequate brain coverage, from the superior cerebellum and inferior temporal lobe ventrally to the hand-motor area in the primary motor cortex dorsally. Prior to time-series acquisition, a 5-minute magnetization-prepared, rapid acquisition gradient-echo T1-weighted image (MPRAGE, TR 1620ms, TE 3.87 ms, FOV 250 mm, matrix 192x256, effective voxel resolution of 1 × 1 × 1mm) was collected for anatomic overlays of functional data and to aid spatial normalization to a standard atlas space (Talairach and Tournoux 1988).

Performance Analysis

Median percent correct and reaction time (RT, in milliseconds) were calculated for threat and non-threat trials separately. Differences in percent correct were evaluated using the Wilcoxon Signed-Rank test; RT between trial types was compared with paired two-tailed t-tests. Performance across all trials was evaluated by calculating the discrimination index Pr and the response bias Br (Snodgrass and Corwin 1988). These were calculated using the following formulas:

• Pr = Correct targets – Incorrect foils

• Br = Incorrect foils / [1 - (Correct targets - False alarms)] – 0.5

The discrimination index Pr provides an unbiased measure of the accuracy of response, with higher values corresponding to a greater degree of accuracy. In contrast, the response bias Br provides an independent measure of the overall tendency of subjects to make ‘old’ or ‘new’ responses regardless of accuracy. Positive values correspond to a familiarity bias (i.e., more likely to say ‘old’ to a new item) whereas negative values correspond to conservative responses with a novelty bias (i.e., more likely to say ‘new’ to an old item).

Image Analysis

fMRI data were preprocessed and analyzed using FEAT (fMRI Expert Analysis Tool) Version 5.1, part of FSL (FMRIB's Software Library, www.fmrib.ox.ac.uk/fsl). Images were slice-time corrected, motion corrected to the median image using a tri-linear interpolation with six degrees of freedom (Jenkinson, et al. 2002), high pass filtered (100s), spatially smoothed (6mm FWHM, isotropic), and grand-mean scaled using mean-based intensity normalization. BET was used to remove non-brain areas (Smith 2002). The median functional and anatomical volumes were coregistered, and then transformed into the standard anatomical space (T1 NMI template, voxel dimensions of 2x2x2 mm) using tri-linear interpolation.

Subject level time-series statistical analysis was carried out using FILM (FMRIB's Improved General Linear Model) with local autocorrelation correction (Woolrich, et al. 2001). The seven condition events (four target emotions, neutrals, foils, and non-responses) were modeled in the GLM after convolution with a canonical hemodynamic response function; temporal derivatives of each condition were also included in the model. Six rigid body movement parameters were included as nuisance covariates.

The contrast map of interest, threat vs. non-threat [i.e., (anger + fear) > (happy + sad)], was subjected to an a priori region of interest (ROI) analysis in two structures implicated in threat and reward processing: the amygdala and ventral striatum. The amygdala ROI was defined using the AAL atlas (Tzourio-Mazoyer, et al. 2002); the ventral striatum was defined from a publication-based probabilistic MNI Atlas used as a binary mask at the threshold of 0.75 probability (Fox and Lancaster 1994; for more information see http://hendrix.imm.dtu.dk/services/jerne/ninf/voi/indexalphabetic). Small volume correction (Friston 1997) was used to identify clusters of at least 10 contiguous 2x2x2 voxels within these ROIs.

In addition to the ROI analysis, two whole-brain mixed effects analyses were performed (Flame2, FMRIB's local analysis of mixed effects): 1) each subject's whole-brain contrast of threat vs. non-threat was entered into a one-sample t-test and 2) each subject's mean activation of the task (all trials) relative to baseline was calculated and entered into a one sample t-test. We corrected for multiple comparisons using a Monte Carlo method with AFNI AlphaSim (R. W. Cox, National Institutes of Health) at a Z threshold of 2.58 and a probability of spatial extent p<0.01. Identified clusters were then labeled according to anatomical regions using the Talairach Daemon database. Coordinates of cluster locations in all analyses are reported in MNI space. To characterize BOLD response amplitudes, mean scaled beta coefficients (% signal change) for threat and non-threat conditions were extracted from significant clusters in the a priori ROI analysis and whole brain analysis.

Finally, we conducted three exploratory analyses to further examine these effects. First, we investigated the effect of individual emotions within significantly activated clusters from the a priori ROIs and certain regions from the whole-brain analysis. To do this, we extracted beta coefficients from each target emotion (happy, sad, angry, fearful) versus a crosshair baseline. Significant differences between individual emotions were assessed with one-tailed paired t-tests. Second, we investigated whether the observed effects were affected by task performance. We modeled seven performance-based regressors at the subject level: threat correct, threat incorrect, non-threat correct, non-threat incorrect, foils, neutrals, and non-responses. Beta coefficients were extracted from above-threshold voxels as described above, and entered into a 2×2 repeated-measures ANOVA (threat vs. non-threat × correct vs. incorrect) to test the main effect of correct response and interaction effects. In this analysis the main effect of threat vs. non-threat was of limited utility as the contrast had already been used to select the voxels under investigation. Third, in order to ascertain if the threat vs. non-threat differences noted in our a priori ROI analysis were present even when subjects did not make a correct response, we used the performance-based regressors and restricted the threat vs. non-threat contrast to incorrect trials only.

Results

Task Performance

The response rate (81.2%) indicated that the task was difficult. Response accuracy was low (median 25% overall among the four target emotions) resulting in a very limited discrimination index (Pr=0.27). The low accuracy was associated with a conservative response bias, whereby subjects were more likely to judge previously presented faces as ‘new’ (Br=-0.29). There were not significant differences between threat and non-threat trials in either response accuracy or response time (Table 1). Although neutral faces and foils were not included in our analysis, accuracy for these trials was significantly higher: 50% for neutral trials and 76% for foils.

Table 1. Performance measures during face recognition task.

Performance measure
Percent Correct	Median	S.D.
Non-threat	33%	19%
Threat	25%	14%
Response Time (msec)
Non-threat	1020	211
Threat	1069	149
Non-threat Correct	1139	173
Non-threat Incorrect	1005	241
Threat Correct	1121	137
Threat Incorrect	1058	152

Task Activation

The analysis of all trials vs. fixation revealed significant activation for the task in a distributed network including bilateral frontal, parietal, medial temporal, thalamic, anterior cingulate, insular, and medial temporal regions (see Supplementary Table and Figure S1). This is consistent with general accounts of recognition memory (Shannon and Buckner 2004; Wheeler and Buckner 2003), as well as studies of face recognition more specifically (Calder and Young 2005; Ishai and Yago 2006).

Region of Interest Analysis

The a priori ROI analysis revealed significant effects in the left amygdala and left ventral striatum (Figure 2A). The left amygdala responded more to threat than non-threat stimuli in a cluster of 28 contiguous voxels surviving small volume correction (peak voxel: 30, -6, -28). The left ventral striatum showed the opposite effect, activating more to faces initially encountered with a non-threatening affect; 28 voxels (peak voxel: -14, 16, -8) survived small volume correction. While significant effects were only seen on the left, the right amygdala and ventral striatum exhibited similar sub-threshold responses. Overall, these results are consistent with our hypotheses, suggesting that during the recall of emotionally-valenced memories, the amygdala is differentially recruited by threat and the ventral striatum by non-threat even when the recognition cues lack emotional content.

Figure 2.

Imaging results. A. Region of interest analysis. The amygdala and the ventral striatum were identified as a priori regions of interest for analysis of target responses. The left amygdala responded more to target faces previously displayed with a threatening (angry or fearful) affect. In contrast, the left ventral striatum (right panels) responded more to target faces previously displayed with a non-threatening (happy or sad) affect. B. Whole-brain analysis. In a voxelwise analysis, the right orbitofrontal cortex (OFC) responded to target faces previously displayed with a threatening affect.

Whole-Brain Analysis

In order to examine brain regions beyond those selected in our a priori analysis, we performed a whole-brain analysis of the threat > non-threat contrast. The right orbitofrontal cortex showed significantly greater BOLD response during threat trials compared to non-threat, corrected at a Z threshold of 2.58 and a probability of spatial extent p<0.01 (90 voxels, peak at 44, 56, -12; see Figure 2B). The left orbitofrontal cortex exhibited a similar response that was not above this threshold. Two left lateral temporal clusters (posterior: 125 voxels, peak at -64, -38, -4; anterior: 96 voxels, peak at -48, -14, -8) were also found to be above threshold.

Exploratory Analysis of Individual Emotions

In order to probe the source of these threat vs. non-threat differences, we examined the effect of individual emotions within the voxels found to be significant in the above analyses. The results indicate that the threat > non-threat difference was not driven by one individual emotion (see Supplementary Figure S2). In the left amygdala, there were significant differences between each of the threat and non-threatening affects: anger > happy [t(40)=2.2, p=0.01], anger > sad [t(40)=2.3, p=0.01], fear > happy [t(40)=1.8, p<0.05], fear > sad [t(40)=1.8, p<0.05]. There was no difference within threat and non-threat affects. The same pattern was seen in the right OFC, with anger > happy [t(40)=2.6, p<0.01] and anger > sad [t(40)=3.5, p<0.001], as well as fear > happy [t(40)=2.5, p<0.01] and fear > sad [t(40)=3.5, p=0.001]. Again, there were not differences within threat or non-threat emotions. Finally, in the ventral striatum, there was a significant differences between sad and fear [t(40=2.0, p<0.05], as well as a trend towards a difference between happy and fear[t(40)=1.4, p=0.07]. There were no other significant differences between emotions in the ventral striatum; anger demonstrated a more intermediate response.

Exploratory Performance-Based Analyses

Next, we examined the effect of recognition performance on the observed threat vs. non-threat differences. Overall, we found that the effects described above were not significantly influenced by performance (Supplementary Figure S3). There was not a main effect of performance in the left amygdala [F(1,20)=0.03, p=0.85], left ventral striatum [F(1,20)=0.004, p=0.95], or right OFC [F(1,20)=2.69, p=0.11]. Furthermore, there were no interactions between performance and trial type in any of the above regions. However, this analysis was likely underpowered due to the limited number of correct trials.

When we restricted our analysis to incorrect trials only, we found that differences between threat and non-threat trials persisted in our a priori ROIs. In the left amygdala, 4 voxels survived small volume correction when only incorrect trials were included; 50 voxels surpassed an uncorrected p=0.05 threshold. In the left ventral striatum, 20 voxels survived small volume correction.

Discussion

This study investigated the impact of an intense facial affect at first presentation on neural activation during a subsequent recognition task. We found that faces previously seen with a threatening expression (fearful or angry) provoked a greater BOLD response in the left amygdala and right orbitofrontal cortex, whereas the left ventral striatum responded more to faces initially viewed with a non-threatening expression. Furthermore, these differences were independent of performance. The implications and limitations of these findings are discussed below.

Amygdala and OFC respond to previously threatening faces

Past experiments have demonstrated the important role of the amygdala in fear conditioning in animals (for a review, see LeDoux, 2000), in humans with amygdalar lesions (Bechara, et al. 1995; LaBar, et al. 1995), and in healthy controls in neuroimaging paradigms (Dolcos, et al. 2004; LaBar and Cabeza 2006). In addition to conditioning paradigms, the amygdala has been critically implicated in emotional learning across a wide variety of tasks (see LaBar and Cabeza 2006). Specifically, amygdalar activity during encoding of emotional stimuli predicts successful recollection (Canli, et al. 2000; Dolcos, et al. 2004). Furthermore, neutral stimuli encoded in emotional contexts have been shown to recruit the amygdala upon retrieval (Dolan, et al. 2000; Maratos, et al. 2001; Smith, et al. 2004; Taylor, et al. 1998). Work by our group (Gur, et al. 2002b; Gur, et al. 2007) and others (Anderson, et al. 2003; Williams, et al. 2004) has demonstrated that the amygdala responds to displays of threatening faces; however, no previous experiment has investigated limbic contributions to facial affective memory.

Consistent with previous studies using non-facial stimulus categories (Dolan, et al. 2000; Maratos, et al. 2001; Smith, et al. 2004) we found that faces initially encountered with a threatening affect elicit amygdalar response upon re-presentation with neutral affect. Gobbini and Haxby (2007) have proposed a core system of facial recognition that is modulated by emotional information through limbic inputs. Our results suggest that modulatory signals from limbic regions reflect not only the current emotional context, but also represent the prior emotional context from a previous exposure. For example, the negative “first impression” created by a threatening facial expression may produce amygdalar activation initially (Gur, et al. 2002b; LeDoux 2000). Later, when the face is re-encountered with a neutral expression, the amygdala is again recruited in response to the emotional context present during encoding. Thus, these results outline a neural system that integrates social information about individuals over time.

The OFC is known to occupy a privileged role at the intersection of emotion and cognition. It has extensive reciprocal connections to medial temporal lobe (Cavada, et al. 2000) and there is ample evidence for its role in the encoding of memories (Frey and Petrides 2002), specifically the aspects of memory associated with rewarding or aversive information (Morris and Dolan 2001; O'Doherty, et al. 2001; Rolls 2000). Two previous studies (Blair, et al. 1999; Lee, et al. 2008) have found that the OFC responds to angry faces; here we found that, like the amygdala, the right OFC responds to neutral faces previously presented with a threatening affect. Like the amygdala and OFC, two left lateral temporal clusters were also found to respond to previously threatening faces. These clusters are situated near the superior temporal sulcus, which has been shown to be involved in numerous functions (Hein and Knight 2008), including face perception (Haxby, et al. 1999) gaze direction (Grosbras, et al. 2005; Hoffman and Haxby 2000), biological motion (Grossman and Blake 2002). Extending findings that the STS may be involved in the processing displays of facial affect (Williams, et al. 2008), our results suggest that lateral temporal regions may also be recruited by previously encountered affect as well.

Ventral striatum responds to previously non-threatening faces

While much research on emotional memory has focused on the amygdala (Phelps 2004), there is increasing evidence that the striatum also plays an important role in the encoding of emotional information (Adcock, et al. 2006; Brewer, et al. 2008; Britton, et al. 2006a; Phan, et al. 2004; Satterthwaite, et al. 2007). Studies by Monk et al. (2008) and Aharon et al. (2001) have demonstrated that the ventral striatum is recruited by displays of happy and beautiful faces, suggesting that pleasant faces are processed as social reinforcers. Like happy faces, sad faces have been theorized to be rewarding because of their pro-social, affiliative properties, as well as demonstrating submission within a social hierarchy (Bonanno, et al. 2008; Eisenberg, et al. 1989; Eisenberg and Miller 1987; Killgore and Yurgelun-Todd 2004; Lewis and Haviland-Jones 2008). Evidence that the human brain is particularly sensitive to affective displays that provide information on social context and hierarchy is accumulating (Britton, et al. 2006a; Carter and Pelphrey 2008; Fliessbach, et al. 2007; Guroglu, et al. 2008; Kim, et al. 2007; Rilling, et al. 2008; Zink, et al. 2008). Also, two prior fMRI studies have found sad faces to activate the striatum (Beauregard, et al. 1998; Fu, et al. 2004). Our results suggest that reward pathways are recruited not just when a non-threatening face is first encountered, but also when it is re-encountered in a neutral context. Thus, the ventral striatum may provide a reward signal that may act in opposition to the amygdala when making threat vs. non-threat distinctions regarding previously encountered individuals.

Threat vs. non-threat differences persist even when faces are not recognized

In an exploratory analysis, we did not find an effect of task performance on brain activation that would account for the differential response of these brain regions. Indeed, when we excluded correct trials and examined amygdalar and striatal responses to incorrect trials only, the results persisted (although at somewhat lower levels of significance). This potentially important finding suggests that successful recognition is not required for facial affective information to be preserved on a neural level. This is relevant to the ongoing controversy as to whether amygdalar responses to emotional information depend on active attention or if they occur automatically (Pessoa, et al. 2005; Vuilleumier and Driver 2007). Several backward-masking fMRI studies have shown that the amygdala responds to fearful faces presented without awareness (Dannlowski, et al. 2007; Whalen, et al. 1998; Williams, et al. 2006), providing evidence for an automatic “bottom-up” response that does not require attention (LeDoux 2000; Phelps and LeDoux 2005). Our task was not designed to isolate such stimulus-driven properties of amygdala response. However, the results do suggest that a “first impression” that is not explicitly remembered may be encoded on a neural level and implicitly influence subsequent social interactions.

Limitations

Two limitations of this study should be acknowledged. First, the grouping of the emotions into threat and non-threat categories may obscure differences between the individual component emotions. For example, while an angry face represents a direct threat indicated by gaze, a fearful face may indicate a more ambiguous environmental threat (Ewbank, et al. 2008), and some studies (Whalen, et al. 2001) have found that that the amygdala may respond more to fearful than angry faces. However, other studies have demonstrated that the amygdala does reliably respond to anger (Beaver, et al. 2008; Evans, et al. 2008; Ewbank, et al. 2008; Hariri, et al. 2000; Nomura, et al. 2004; Sprengelmeyer, et al. 1998; Stein, et al. 2002; Suslow, et al. 2006). In the current study, anger and fear trials produced a similar response in amygdala and OFC clusters, indicating that the results were not driven by one particular emotion. In contrast, the findings in the ventral striatum were more ambiguous, and distinctions other than the threat vs. non-threat effect may have impacted the results. Future studies are necessary to investigate potentially important differences within categories of threatening and non-threatening faces.

Second, the low accuracy in target recognition led our study to be underpowered for certain performance-based comparisons. Performance was likely limited by the general difficulty of face recognition tasks (Calkins, et al. 2005; Ishai and Yago 2006) and by the fact that subjects were not notified that they would be asked to later recognize the faces from the affect identification task. We observed a strong conservative response bias, suggesting low target accuracy was not due to simple guessing. In particular, while we did not find an interaction of performance with the threat vs. non-threat differences reported, this analysis was limited by the number of correct trials in each group. In contrast, the high proportion of incorrect trials allowed us to examine these trials separately in a sub-analysis, finding that the threat vs. non-threat differences are present in the amygdala and ventral striatum even when only incorrect trials were considered. Nonetheless, in the future it will be important to further assess the effects of correct recognition with a design that produces more correct trials and incorporates behavioral metrics of recognition confidence.

Conclusions

In summary, the current study provides the first evidence for frontolimbic responses that could provide emotional bias signals during facial recognition. The amygdala and OFC appear to signal previously threatening faces, while the ventral striatum may oppose this system by responding to previously non-threatening faces. These differences do not appear to depend on accurate recognition of faces, suggesting a process that does not require explicit awareness. Emotional face memory plays an important role in social cognition and these results provide an intriguing first look into the neural correlates of emotional memory of faces. Furthermore, such processes could contribute to deficits in face recognition and social interaction seen in neuropsychiatric disorders such as schizophrenia, mood disorders, and brain injury (Calkins, et al. 2005; Fu, et al. 2008; Gainotti 2007; Holt, et al. 2006; Holt, et al. 2005).