Fear, Faces, and the Human Amygdala

Summary

The amygdala’s historical role in processing stimuli related to threat and fear is being modified to suggest a role that is broader and more abstract. Amygdala lesions impair the ability to seek out and make use of the eye region of faces, resulting in impaired fear perception. Other studies in rats and humans revive earlier proposals that the amygdala is important not for fear perception as such, but for detecting saliency and biological relevance. Debates about some features of this processing now suggest that while the amygdala can process fearful facial expressions in the absence of conscious perception, and while there is some degree of pre-attentive processing, this depends on context and is not necessarily more rapid than cortical processing routes. A large current research effort extends the amygdala’s putative role to a number of psychiatric illnesses.

Introduction

A variety of brain structures implicated in social behavior, reward learning, and emotion have also been implicated in the recognition of emotions from stimuli, such as facial expressions. For instance, ventral regions of the prefrontal cortex are activated in neuroimaging studies of fear and anger; and emotion recognition can be impaired by lesions in this region. Sectors of right parieto-temporal cortex appear important for emotion recognition, especially fear recognition, as well. Some of the most compelling evidence comes from structures that participate in processing reward and punishment: the insula, the ventral striatum, and the amygdala. The insula appears disproportionately important for recognition of disgust (in facial expressions, related to distaste, and also related to moral disgust). Lesions of the ventral striatum can impair anger recognition [1]. And, perhaps most studied of all, lesions of the amygdala can impair recognition of fear from static [2] and dynamic [3] facial expressions. Similarly, amygdala activation can be disproportionate for facial expressions of fear or threat [4•] as well as correlate with correct identification of fear faces [5]. These findings are particularly intriguing given the amygdala’s well-known role in the acquisition and expression of Pavlovian fear conditioning [6], and may be related to its more recently demonstrated role in social transmission of fear information in rats [7•] and humans [8•]. Despite the large literature on this topic, there is considerable discrepancy: for instance, amygdala lesions can also impair recognition of emotions other than fear, and amygdala activation may not be at all specific to fear [9,10], nor to facial expressions of emotion [11]. Here I review the latest developments in the amygdala’s role in fear perception and debates regarding the automatic and non-conscious aspects of such processing, which rely on a number of different tasks (Table 1).

Table 1.

Fear-related processing and tasks to assess it.

Process	Tasks
Fear expression	Psychophysiology, startle, freezing, facial expression
Fear experience	Inferred from fear expression; subjective report in humans
Innate fear	Responses to smell of predators, biologically basic sensory stimuli
Learned fear	Conditioning through experience, observation, or instruction
Fear detection	Reaction time to fear stimuli, or accuracy in discriminating their spatial location of time of occurrence
Fear discrimination	Accuracy in distinguishing fear from other contemporaneous stimuli
Fear classification	Accuracy in distinguishing fear from a particular closed set of other stimuli
Fear recognition	Multivariate judgments related to conceptual knowledge about fear
Rapid	Neurological (EEG, MEG, electrophysiology) latency from stimulus onset Response (behavioral, neural, or psychophysiological) to stimuli of very brief duration
Automatic	Pre-attentive responses to stimuli Pop-out in visual search Attentional effects by fear stimuli on performance related to other stimuli (related in time or space)
Non-conscious	Neurological, behavioral, or psychophysiological responses to stimuli with null-sensitivity in overt detection or discrimination

Is fear processing fast and automatic?

There is good evidence that at least some aspects of social perception can be quite rapid, based on faces shown for 100 ms or less [12]. Judgments of threat, but not other judgments, can even be made with faces that are presented as briefly as 39 ms (backward masked) [13•]. It is conceivable that a rapid initial sweep of stimulus processing detects basic features and gist, such as “threat”, and provides the basis for subsequent biasing of further information processing.

Other studies have argued that threat detection is not only rapid, but automatic, in the sense of being independent of attention. Data supporting this idea have come from visual search tasks, in which threat-related stimuli are reported to pop out [14], arguing that some affective processing precedes (and drives) attention to the stimuli, an idea about the primacy of affective processing that has a long history of debate in psychology. However, attempts to replicate these findings on visual search tasks involving threat-related faces have been inconsistent, and some experiments, perhaps especially with photographic-quality faces that the earlier studies did not use, argue against automatic processing of threat [15]. Fearful faces do appear to facilitate early visual processing in some way, however, since their locations enhance contrast sensitivity discrimination to subsequently shown gratings [16•]. A further complexity is that the extent to which threat processing is automatic depends on individual differences in anxiety [17]. This finding is in line with a rapidly expanding field investigating the role of individual differences in emotional face processing, both at the level of personality traits [18] as well as genetic polymorphisms [19,20]. Attention may come into play not as an essential ingredient for fear detection, but rather for modulating the degree of specificity in such detection: the amygdala responds most specifically to fear when subjects attend to the stimuli, but responds more broadly to threatening stimuli during unattended processing [21].

Studies of neurological patients have provided further ammunition for the idea of threat-related processing in the absence of attention. Patients with spatial hemineglect following unilateral right parietotemporal lesions normally fail to detect stimuli presented contralateral to the side of the lesion (i.e., left visual field) when shown stimuli in both fields. This attentional failure under bilateral stimulus presentation, however, is considerably reduced if the to-be-detected stimulus is a face, and further reduced if the face shows emotional expressions including fear [22]. A possibly related finding is that fearful facial expressions are most efficacious in breaking interocular suppression-- they render the face visible under conditions of continuous flash suppression, when it is typically invisible [23•]. Yet differential responses in the amygdala to threat or fear-related stimuli also appears to be subject to attentional capacity limitations: no such differences are seen when attention is entirely occupied with a difficult distractor task [24,25].

Do threat-related faces “pop out” in search tasks because they signal threat, or because of particular aspects of contrast or other low-level visual feature properties? The only way to definitively resolve this issue is to formally decouple sensory from affective properties experimentally. In one study, visual stimuli (complex but intrinsically neutral visual images) were fear conditioned and it was found that the fear-conditioned stimuli influenced the attentional blink phenomenon [26]. In another study, it was found that strongly affective lexical stimuli (words like “death”, whose association between low-level sensory and affective properties is truly idiosyncratic) also influenced the attentional blink, and moreover that this influence depended on the integrity of the amygdala [27]. These findings argue that fear perception, and the amygdala-dependent processing that underlies it, does incorporate rapid processing that is pre-attentive to some degree, and that it depends both on the acquired emotional meaning of stimuli as well as innate components. This interpretation is also consistent with an fMRI study that found amygdala activation both to emotional facial expressions as such, and an additional activation component due to emotional learning associated with those facial expressions [28].

Conscious and non-conscious processing

The above findings that threat detection can be rapid did not yet address whether it could also be non-conscious. Indeed, one study of rapid threat judgments emphasized their conscious nature, since briefer presentations (going from 39 to 26 ms) led to noticeably less reliable results [13•]. However, there are a number of studies arguing to the contrary: that fear and/or threat processing occurs in the absence of conscious perception. These studies have focused on the amygdala in particular, and build on a large related background literature which argues that the amygdala participates in auditory fear conditioning in part through subcortical inputs, that it modulates responses to emotional stimuli in visual cortices [29], and that it is important for fear recognition [2]. The strong hypothesis is that certain aspects of visual information (contrast, visual motion and low spatial frequency) are conveyed also through a subcortical route comprising a special subset of retinal ganglion cells that project to the superior colliculus, which projects to the pulvinar nucleus of the thalamus, which projects to the amygdala; and that this route can provide the amygdala with rapid information related to threat in the absence of any cortical processing and in the absence of any conscious awareness of the stimulus.

Several different findings provide circumstantial support for the hypothesis [30]. Unilateral lesions to the pulvinar thalamus (and subjacent white matter) can abolish the normal interference of threat-related stimuli on goal-directed task performance and can impair facial fear recognition if encompassing the medial pulvinar [31]. The amygdala is activated differentially by fearful faces in patients with blindsight, who lack conscious visual perception of the stimuli due to damage in early visual cortices [32]. Young infants show behavioral preferences for faces over other visual stimuli, and possess a very immaturely developed visual cortex but relatively more developed subcortical pathway [33]. And fearful faces rendered non-conscious through interocular suppression (as verified by psychophysical assessment) resulted in a BOLD-response in the amygdala essentially indistinguishable in magnitude from that seen with consciously perceived fearful stimuli-- an effect that was not observed in the fusiform gyrus, nor with neutral facial expressions [34••].

Some difficulties with the idea that fear processing in the amygdala can be truly non-conscious comes from contrary findings arguing that amygdala responses to fearful faces require conscious perception [35••], raising important issues about how to establish with certainty the absence of conscious perception. Serious difficulties about the idea of a direct and rapid subcortical route of processing to the amygdala stem from the lack of anatomical and electrophysiological evidence. While the basolateral amygdala does receive direct input from the medial pulvinar [36], the inputs from the superior colliculus terminate in the lateral pulvinar [37]. This makes it necessary to interpose either intrinsic connections within the pulvinar or a collicular-pulvinar-extrastriate cortical route to the amygdala, both of which would be expected to be slower. Finally, there are inconsistencies in the latencies of responses to visual stimuli observed in the amygdala. Some electrophysiological studies in monkeys [38] have reported latencies around 60 ms, and MEG studies in humans reported latencies as short as 30 ms [39] (although these cannot be assigned with certainty to the amygdala given the spatial resolution of the technique). fMRI studies have also argued for a shorter latency of the BOLD response in the amygdala to fearful than to neutral faces [40]. By contrast, other studies in monkeys [41,42] as well as in humans [43–45] have found responses well in excess of 100ms, and as slow or slower than those observed in temporal visual cortex. Additionally, it appears that visual response latencies in the pulvinar are similar to those in visual cortex [46].

The cortical temporal dynamics of emotional face classification were investigated in detail in a recent study that examined ERP responses to particular facial features [47••] (Figure 1). They found that fear is processed relatively faster than other emotions for a simple reason: cortical processing of facial features begins with the eye region of the face, which happens to be the feature most diagnostic for fear. The finding also suggests that the N170, an early cortical potential evoked by faces, reflects both automatic (early processing of the eye region of faces) as well as controlled aspects of processing (termination of processing once features diagnostic for the discrimination of a particular emotion had been reached). While this study does not resolve the debates about a direct subcortical route to the amygdala, it does suggest a mechanism to explain why fearful faces should be processed more rapidly than other expressions.

Figure 1.

Rapid responses to eyes drive fear discrimination. ERPs recorded from right and left (dashed lines) occipitotemporal electrodes are plotted versus time (in ms) relative to the onset of sparsely revealed faces. The peak of this ERP response, corresponding to the classic “N170” is denoted by the blue boxes. Classification images derived by regressing the magnitude of the ERP response (in 4 ms time epochs) onto the randomly sampled location of facial features shown in the sparse stimuli are shown at the bottom. The ERP magnitude is shown by the blue curves; the overall sensitivity of the ERP to facial information is denoted by the red curves. Modified from [45••]; courtesy of Philippe Schyns.

What does the amygdala do?

A large prior literature has implicated the amygdala in facial emotion processing, but the specificity of this finding remains unclear. Curiously, there is no unequivocal evidence that the amygdala is critical for recognizing fear from body postures or movements ([48]; but see [49]), nor from auditory stimuli, although it may be necessary for normal fear recognition from music [50]. Within faces, the eye region of faces is used most prominently when subjects have to discriminate fear from other expressions [51]. Eye gaze direction modulates amygdala responses to fearful faces [52], pupil size modulates amygdala responses [53], and the eye region of fearful faces is most efficacious in breaking conscious suppression of the stimuli [23] and in activating the amygdala [54], even when they are presented very briefly and masked [55]. These findings have argued that perhaps particular features, notably the eyes, depend for their processing on the amygdala.

Bilateral damage to the amygdala results in an inability to fixate the eye region of faces spontaneously, and produces a consequent impairment in utilizing high spatial frequency information from the eyes in order to recognize fear [56] (Figure 2). This abnormal fixation has also been found for real people’s faces, where amygdala damage appears to result in a propensity to fixate the mouth rather than the eyes [57•]. This has led to a revision of the role of the amygdala in emotional face recognition: it does not appear to be specialized for processing threat or fear as such, or perhaps even emotion or reward as such, but rather appears to come into play when stimuli are particularly salient (which could result from them being threatening, rewarding, or unpredictable [58]). One recent study in rats and humans found increased amygdala activity in response to unpredictable compared to predictable tones, an effect that resulted from slower electrophysiological habituation to the unpredictable tones [59••]; another study found increased amygdala activation with sounds that increased in intensity [60]; and a third found amygdala activation depending on how sharp or curved object contours were, a factor that contributes to their perceived preference [61]. It may be that the amygdala is one component of a circuit that is important for processing biological relevance in a broader sense, as has been argued for emotion’s effects on attention more generally [62]. Such processing would be expected to depend on both low-level cues associated with relevance, as well as on contextual and appraisal-related evaluations relative to the circumstances in which a stimulus is encountered, and relative to the perceiver’s goals and capabilities.

Figure 2.

Bilateral amygdala lesions impair the use of the eyes and gaze to the eyes during emotion judgment. Data are from patient SM, who has complete bilateral amygdala lesions and is impaired in recognizing fear [2]. Using sparsely revealed faces to identify face areas used during emotion judgment, patient SM (brain shown in c) differed from controls such that controls exhibited much greater use of the eyes than SM, while SM did not rely more on any area of the face than did controls (a). While looking at whole faces, SM exhibited abnormal face gaze (b), making far fewer fixations to the eyes than did controls. When SM was instructed to look at the eyes (d, “SM eyes”) in a whole face, she could do this, resulting in a remarkable recovery in ability to recognize the facial expression of fear compared to her accuracy prior to this instruction (d, “SM free”). Modified from [54].

Conclusions and Future Directions

Historically, research on the amygdala has emphasized two aspects: its role in social behavior (dating back to the classic studies by Kluver and Bucy in the 1930s), and its role in emotional learning and memory (such as Pavlovian fear conditioning). Both aspects rely in part on privileged allocation of processing resources to stimuli that have particular behavioral relevance, and both incorporate innate and learned components. The recent findings on the amygdala’s role in saliency and unpredictability raise the question of whether such a basic stimulus dimension could also explain other aspects of amygdala function. For instance, is it the inherently greater unpredictability of social stimuli that renders them more relevant and that engages the amygdala? While outside the focus of this review, it is also important to note here that the amygdala is known to play important roles guiding behavioral choice under uncertainty also in the context of reinforcement learning. It has been found to come into play during punishment predictions and prediction errors [63], and expectations of forthcoming reward value computed in the amygdala appear to be relayed to the prefrontal cortex in order to guide choice [64]. The amygdala is also activated during decisions that involve ambiguity (missing information), over and above risk [65]. All these findings from tasks involving decision-making are broadly consistent with the more perceptual/attentional role in processing saliency/unpredictability reviewed above.

One important extension of the amygdala’s role is to psychiatric illnesses in which the behavioral relevance of stimuli is abnormally evaluated. Illnesses ranging from phobias [66] to depression [19] to schizophrenia [67] have shown impaired emotion recognition and/or abnormal amygdala activation. One particularly interesting recent development has been in autism, where it was found that people with autism fail to make normal use of information from the eye region of faces, instead preferring to use the mouth [68], and that duration of eye gaze correlates with amygdala activation [69]. The finding is being extended also to the first-degree relatives of people with autism and may constitute part of an “endophenotype” for impaired amygdala function in autism [70,71], as has been found with respect to other structures, such as anterior cingulate cortex, that are connected with the amygdala [72].