The primate amygdala in social perception - insights from electrophysiological recordings and stimulation

Abstract

The amygdala’s role in emotion and social perception has been intensively investigated primarily through studies using fMRI. Recently, this topic has been examined using single-unit recordings in both humans and monkeys, with a focus on face processing. The findings provide novel insights, including several surprises: amygdala neurons have very long response latencies, show highly nonlinear responses to whole faces, and can be exquisitely selective for very specific parts of faces such as the eyes. In humans, the responses of amygdala neurons correlate with internal states evoked by faces, rather than with their objective features. Current and future studies extend the investigations to psychiatric illnesses such as autism, in which atypical face processing is a hallmark of social dysfunction.

The Primate Amygdala in Health and Disease

The primate amygdala is central for the recognition of and response to social stimuli. Although there is a sizeable literature investigating how the amygdala supports learning and the generation of emotional responses, only recently has this been extended to the study of social perception and social cognition. Crucial insights are being revealed through studies done in a neurosurgical setting, which permits recordings from individual amygdala neurons in awake human subjects engaged in social behavior [71, 98, 99, 114]. Combined with a resurgence of interest in similar studies in non-human primates [36, 43, 72, 73], these approaches are starting to provide circuit-level evidence for the role of the amygdala in one specific aspect of social cognition: the processing of faces. It should be noted that this literature is embedded in a larger set of electrophysiological studies in monkeys that have probed amygdala responses to reward-related stimuli, as well as detailing temporal cortical responses to faces, and a surprisingly large (and growing) literature on human amygdala recordings that spans studies in perception, reward, and emotion (see [76] for a recent detailed review of 47 such studies).

Dysfunction of the amygdala has long been implicated in mood disorders [18], and is now thought to play a role in psychiatric diseases ranging from PTSD to anxiety, borderline personality disorder and autism spectrum disorders. However, we do not know how exactly the amygdala contributes to these diseases, limiting the development of rational intervention and treatment. While there is a large and rapidly growing body of fMRI-studies focused on the amygdala [49] (see below and Box 2 for caveats), it is crucial to investigate the underlying neural mechanisms at the single-neuron and population level: the latter measure neuronal activity directly and provide much higher resolution, and forge an important link to studies in nonhuman primates. Single-neuron recordings in neurosurgical patients with co-morbid psychiatric diseases are a particularly unique opportunity, and this has recently provided crucial new insights into the mechanisms of autism [99]. Simultaneously, there has also been a renewed interest in direct electrical stimulation of the human amygdala for the treatment of psychiatric diseases, with active clinical trials [50] and initial promising case studies [109] in treating difficult psychiatric diseases (see Box 1).

Box 2: Limitations of fMRI studies of the amygdala.

Even aside from the perennial issue of correlation versus causality, and acknowledging the vascular origin of the BOLD signal (see below) measured with fMRI, there are several important limitations to imaging studies of the amygdala. In a nutshell, these revolve around spatial and temporal precision. Spatially, not only is the resolution of fMRI limited (typically to 2–3mm voxels in human studies), but it is also difficult to localize the activation. One source of this problem is the substantial magnetic susceptibility artifact near the amygdala, resulting in signal dropout and geometric distortion. Another source is the spatial complexity of amygdala nuclei. According to some estimates, a good portion of neuroimaging studies that report activations in the amygdala in fact have activations that are certainly outside the amygdala [10]. Probabilistic atlases of amygdala nuclei together with higher-resolution imaging may help with these issues, but ultimately it will be critical to combine loci obtained from neuroimaging studies, with nuclear localization of recording sites from neurosurgical studies (cf. Figure 1).

It is of particular importance to keep in mind that the BOLD signal is a vascular signal because the amygdala is intimately involved in the regulation of both cardiovascular and respiratory activity and in turn blood pressure, among others [20]. Studies are needed to assure that BOLD-fMRI findings can truly be attributed to neuronal changes rather than such indirect effects. It will be key to simultaneously measure variables such as blood pressure to control for such changes, which are in particular accompanying fearful emotions.

Temporal resolution is perhaps an even more acute issue, because many amygdala responses are transient, habituate rapidly, and hence may not be detectable by BOLD-fMRI. Partially this issue might be addressable by utilizing smaller voxels (1–2mm resolution) combined with shorter TRs. Although there have been recent advances in fMRI, such as the development of human 7T, these mostly benefit signal in cortex and not in the amygdala. It thus remains an important challenge for the future how best to optimize BOLD signal in the amygdala.

A long-standing finding in fMRI studies of the amygdala has been their marked variability. This variability is often thought of as arising from individual differences (some attributable to genotypic differences) together with strong context-dependency. However, it also appears that amygdala responses in fMRI may simply be less reliable in terms of test-retest reliability even within the same subject and context [103]. This issue will be important to quantify in more detail, since unreliable fMRI responses (albeit in cortex) have also been highlighted in some disorders in which the amygdala is implicated, such as autism [22], and since this fMRI variability in the amygdala stands in some contrast to the apparent stability of single-unit responses over months [63].

Particularly valuable future studies will combine the causal power of direct electrical (or optogenetic) stimulation with the whole-brain field of view of fMRI. So far, there have been important advances in describing the large-scale networks in which amygdala nuclei participate primarily from correlational analyses of resting-state fMRI (e.g., [67, 94]). However, it is possible in rodents to optogenetically activate very focal neuronal populations while concurrently acquiring fMRI [21], an approach that is feasible in both monkeys and humans by electrical stimulation with concurrent fMRI.

Box 1: Deep-brain stimulation (DBS) of the amygdala for treating psychiatric illness.

There has been a resurgence of interest in utilizing DBS for the treatment of psychiatric illness. For example, DBS of the subgenual cingulate gyrus has yielded promising results for the treatment of depression [47]. One hypothesis is that the efficacy of this approach is due to stimulation of passing fibers that may project to the amygdala. For autism, a single case-study was recently published suggesting amelioration of agression in a patient with autism and self-injurious behavior [109]. Interestingly, the target site for the stimulation in this case was the basolateral amygdala, which is also the target for another psychiatric illness, PTSD (see below).

Extinction of conditioned fear responses appears to depend on the integrity of reciprocal connections between the medial pre-frontal cortex (mPFC) and the basolateral amygdala (BLA) [24]. Human fMRI studies have revealed increased activity in mPFC during fear extinction trials or exposure to trauma triggering stimuli [34]. Further, subjects with PTSD that responded to cognitive therapy demonstrated diminished BLA and increased mPFC activation in comparison to their pre-treatment baseline, while subjects demonstrating the most increased levels of BLA hyperactivity were the least likely to respond to PTSD treatment [112]. This evidence for BLA hyperactivity has led to the development of candidate stimulation paradigms to electrically silence the BLA in humans to treat pharmacologically resistant PTSD with deep brain stimulation (DBS). DBS via chronically implanted electrodes is a proven medical therapy for the treatment of movement disorders [57]. High frequency (>120 Hz) stimulation of nuclei is assumed to result in effective electrical silencing, in effect mimicking ablation, whereas DBS of white matter tracts that consist primarily of projection axons is assumed to mimic stimulation of efferent pathways.

Potential targets for DBS in the treatment of PTSD include basolateral and central amygdala nuclei, as well as regions of medial prefrontal and dorsal anterior cingulate cortex. Hyperactivity of the BLA in PTSD patients as well as its well the large size, defined anatomy, and accessibility of this nucleus make it a particularly attractive target for DBS therapy. The first trial of bilateral BLA DBS in combat veterans suffering from PTSD has recently been initiated [50]. Patients underwent implantation of the Medtronic PC Activa DBS system. Half of the patients are being administered (blind) sham stimulation for 90 days, while the other half are receiving high frequency stimulation starting at 30 days after implantation. Future studies may employ the use of a responsive neural stimulator in which amygdala stimulation is only triggered either by the patient when experiencing a PTSD episode, or when neuronal activity exceed a threshold level of activation. Such novel therapies are already successfully utilized for the treatment of epilepsy [110].

The amygdala, emotion, and social cognition

The crucial role of the amygdala in emotion and social cognition is demonstrated by both lesion [7] and morphometric studies. Structurally, amygdala size co-varies systematically with social group size in humans [13] and macaques [101]. The later study argued for a causal role of group size on amygdala volume, since there was experimental (random) assignment to group; a similar relationship has been found between the volume of the macaque amygdala and social status [78]. When exposing macaques to visual stimuli of other individuals on a computer screen, amygdala volume correlates with the extent to which an individual fixates the eye region of faces [117], and lesions of the monkey amygdala reduce fixations onto eyes in faces [17]. Macaques with experimentally induced amygdala lesions show blunted emotional responses to fear-inducing and novel stimuli [60]. Beyond this specific deficit, such lesions severely disrupt social behavior, including loss of status, social isolation, failure to respond to and initiate social gestures, and disrupted maternal behavior (see for a review [7]). In humans, bilateral amygdala damage is extremely rare, but, in the few cases that have been systematically investigated, manifests with impaired ability to recognize fearful faces [1], abnormal fixation patterns on faces [2], as well as other deficits in social behavior and emotion [3].

Complementary evidence is provided by direct electrical stimulation experiments (see [104] for a recent review). While detailed mapping of the stimulated sub-nuclei is often not available, the majority of studies target the basolateral amygdala (Figure 1). Most prominently, amygdala stimulation can induce negative (unpleasant) emotions such as fear, tension and nervousness [40, 64]. However, positive emotions can be evoked as well, depending on stimulation site [14, 52], mirroring fMRI studies that generally find amygdala activation to unpleasant and well as pleasant stimuli. It is notable that several studies attempted but failed to induce feelings of aggression by amygdala stimulation [40, 64], except in patients who had pathological aggression to begin with. Autonomic arousal, measured as a change in skin conductance (SCR), frequently results as a result of amygdala stimulation [59], especially when the stimulation is accompanied by reports of emotional experience [52]. A frequent symptom of amygdala stimulation is a feeling of deja-vu, often accompanied by spontaneous recall of remote memories or complex visual hallucinations, reinforcing the critical role of the amygdala in memory encoding and retrieval (see below)..

Figure 1. Anatomy and electrode location.

(A) Selected inputs and outputs of the human amygdala, following [6]. (B) Example of electrode locations where face-related responses were recorded in the human amygdala. (b1) MNI-template brain at y=−2 superimposed with estimated location of tip of microwire electrodes. The indicated area (white box) was extracted and the amygdala region from Plate 25 (+4.0 mm) of the Mai atlas (b2) was superimposed, followed by a (b3) two-dimensional non-linear warping. Warping of the atlas plate boundaries was performed by inspection using the optic tract, anterior commissure, medial boundary of the entorhinal cortex and uncus and ventral-lateral boundary of the amygdala as guides. The overlay is for visualization only and is not intended to represent an optimized coregistration of the atlas and MRI data. (C) Histological slide showing nuclear boundaries at 5.4mm relative to the anterior commissure. (D) Probabilistic map of location of different parts of the amygdala (superficial, laterbasal, and centromedia). Figure modified from [8, 32, 99].

These findings are complemented by an enormous number of human neuroimaging studies over the past 20 years. To a large extent these corroborate the above picture, but they probe a much larger range of social and affective processes. There is clear evidence that the amygdala responds, metabolically as measured by the BOLD signal, not only to faces [27], but to a range of visual [105] and nonvisual [95] stimuli as a function of the arousal or emotional salience of the stimuli. Few neuroimaging studies have looked at specific dimensions or features of faces. In the monkey, fMRI has revealed activations that track eye gaze and facial expression in different amygdala subnuclei [43]. In humans, factor-analytic approaches have found evidence that the amygdala may track specific social dimensions in faces (such as their valence or trustworthiness), and that these may correlate with information from particular features of the face, such as the eyes [5].

There has been considerable debate on whether the amygdala serves a role in our conscious awareness of social stimuli. Indeed, amygdala activation to faces has even been used to probe awareness in patients in persistent vegetative state [106]. Whereas some lesion findings show clear evidence that the human amygdala is required for the conscious experience of fear [26], this has been much less clear from neuroimaging studies [54]. There is more consensus, however, that the amygdala’s role in emotion experience (whatever its precise nature) can be modulated in tandem with the volitional regulation of such experience: upregulating or downregulating experience has corresponding effects on amygdala activation [28], through presumptive cortical networks whose identity may vary depending on the exact nature of the emotion regulation [23]. This also brings us back to the amygdala’s importance in psychiatric illness: the leading models propose that emotion dysfunction, and social dysfunction, arises in such diseases through altered connectivity between the amygdala and cortical regions, with resulting dysregulation of emotion and behavior. Some recent meta-analyses have found evidence for a specific mechanism in emotion regulation: semantic representations in temporal cortices are thought to modulate activation of the amygdala [15]. Very recently, single-unit recordings in the human amygdala have tackled this issue, showing that neuronal responses track the subjective judgments of the perceiver (see below).

In conclusion, there is strong evidence that the amygdala is necessary for many aspects of emotion and social perception. However, it remains largely unknown what specifically the amygdala contributes. The resurgence of interest in studying the primate amygdala at the level of single-neurons promises to yield findings that are complementary to the large human neuroimaging literature and add crucial detail. In particular, they highlight the need to carefully differentiate between different types of neurons and nuclei as well as between inputs from other areas and locally generated activity. This requires measuring neural activity directly rather than indirectly as is commonly done with fMRI (see Box 2).

Neuronal responses to faces in the human amygdala

Faces are among the most important social stimuli, providing information about attention through gaze direction, about gender, age, and identity through static features of the face, and about emotional state through dynamic changes particularly around the mouth and eye regions. A productive avenue to study the role of the amygdala in social perception is thus the study of how individual neurons respond to faces and their features.

Several cortical areas of the primate temporal lobe contain neurons that respond to faces and features thereof, such as expression, identity and gaze. While this has been known since the 1980’s, recent combinations of fMRI-targeted electrophysiology have shown that there are specific parts of cortex that appear to be specialized (perhaps exclusively) for the processing of faces [30]. In macaques, the specificity of this finding has been affirmed by directly recording individual neurons in the same areas using either individual acute electrodes [30] or chronically implanted microwire arrays [63]. The latter permits recording from the same individual neuron for up to one year, which has revealed that the tuning of such neurons remains remarkably stable across such long periods of time [63]. The amygdala receives strong inputs from higher visual areas [61], including those found to contain face-selective neurons. The functional role of these inputs to the amygdala remain unknown, but the prominent responses of individual amygdala neurons to faces indicates the importance of this input. Supporting an involvement of the amygdala in face processing, similar fMRI contrasts that were used to identify face-selective cortical areas also activate parts of the amygdala in non-human primates [43] and humans [66]. What differentiates cortical responses from those in the amygdala, and what might be the specific functional role of face representations in the amygdala, remains unclear and will require extensive single-neuron work. Key questions are i) to what degree individual neurons in the amygdala are sensitive to faces and parts thereof, ii) how these responses differ from inferotemporal responses, and iii) how the responses relate to behavior. It is only through a detailed understanding of all three of these points that we can really understand what it is that the amygdala computes as it contributes to face processing.

Neurons in the human amygdala are exquisitely sensitive to faces [31]. In a series of studies with epilepsy patients [4, 98, 99, 114], we found that approximately 50% of all individual neurons changed their firing rate after presentation of a face relative to a scrambled baseline (Figure 2). Such face-sensitive responses were both excitatory and inhibitory (increase and decrease in firing rate vs. baseline) [98]. To investigate the visual tuning to faces and parts thereof in detail, we used a reverse-correlation paradigm during which randomly selected sparse parts of emotional faces were presented (“bubbles” technique [35]). Faces were either happy or fearful and subjects classified them as happy/fearful in each trial. An important advantage of this technique is that both the behavior and neuronal responses can be quantified using a classification image. The classification image has the same size as the displayed stimulus and shows, for every pixel (x,y), the correlation between a behavioral or neuronal variable with whether this pixel was revealed or hidden. We first constructed a behavioral classification image by using the accuracy and reaction time in each trial as the performance metric. This revealed that subjects primarily used information provided by the eye-and mouth region of the stimulus [98, 99]. This has been found in numerous previous studies, confirming that the neurosurgical patients performed the task similar to controls [2, 35]. We subsequently constructed a neuronal classification image for every recorded amygdala neuron by correlating firing rate with whether a pixel was shown or not (Figure 3). We found that nearly 20% of amygdala neurons were sensitive to whether a certain part of a face was shown, such as an eye, or the mouth, but also the eyebrow or wrinkles around the mouth [99]. Thus, these neurons were “part-sensitive”, encoding information about specific features in faces—interesting, in general regions of the face that would move a lot in natural viewing, such as the region around the eyes and the mouth.

Figure 2. Face-responsive neurons in the human amygdale.

(A) Example of emotional faces presented to subjects. Each face was either fearful or happy. (B) Task. Faces or parts thereof (eye, mouth or randomly revealed “bubbled” faces) were shown for 500ms, preceded by a scrambled version of the same stimulus. Subjects indicated with a button press, as quickly as they could, whether the face shown was fearful or happy. (C) Response of 4 example neurons during this task. Note that some neurons respond with an increase (top row), whereas others with a decrease (bottom row) to the onset of the faces but not the scrambled stimulus. (D) Example whole-face neuron. This neuron increased its firing rate only to whole faces (black trials) but not the parts (blue, green) or bubbled (red) faces. Trials are sorted by reaction time, as indicated by the magenta line. Color code is as shown in (A). Figure modified from [98, 114] RT, reaction time.

Figure 3. Face-part response neurons in the human amygdala and impairment by autism.

(A) Example of face-part (mouth) sensitive neuron, identified both using ROIs (blue vs. green) as well as by correlation with the bubbled faces (red trials). The neuronal classification image (bottom left) indicates the z-scored correlation of each pixel and firing rate. A positive (red) value indicates that revealing this picture will increase the firing rate of this neuron. The part of the face to which this neuron reacted significantly is indicated on the face (right). (B–E) Comparison of face-part sensitive neurons in controls (blue) and patients diagnosed with high-functioning autism (ASD, red). (B,C,D) Face part-sensitive neurons clustered around the mouth in ASD (C), whereas neurons normal were more strongly driven by the eye (D). (E) This phenotype was specific to face-part selective neurons (left). In contrast, whole face neurons (see Figure 2) responded comparably to the controls (right). (F,G) Behavioral classification images for controls (F) and ASD subjects indicate which part of the face was used to distinguish happy from fearful faces. This behavioral phenotype mirrors the neuronal phenotype: controls utilized the eyes (F) whereas ASD subjects only utilized the moth (G). Figure modified from [99].

A second set of amygdala neurons responded only when an entire face was present (“whole face neurons”; about 20% of neurons were also of this kind in our study), but not to individual parts [98] (Figure 2). Interestingly, the response of such whole-face neurons was smallest when a face was shown that was almost entirely revealed, with only a small part (such as the tip of the nose) removed. Detailed subsequent analysis revealed that the responses of these neurons could not be explained as a sum of parts, but rather was a holistic response to an entire (whole) face [98].

A third group of cells differentiated between the emotions expressed by the face, regardless of how much of the face was revealed [114]. Following stimulus onset, this subset of neurons (22% in our study) discriminated fearful from happy faces by increasing or decreasing firing rate to happy or fearful faces. Contrary to earlier views that the amygdala is specialized for the processing of fear or otherwise negatively-valenced stimuli [116], a sizeable subset (nearly 10%) of all recorded neurons increased their firing rate specifically only to faces expressing a happy emotion—a finding quite consistent with the results from fMRI of the human amygdala, which also shows responses across all emotional expressions [27] as well as neutral faces [66]. Note that it remains unknown whether the three functional groups of cells described above are truly distinct or whether they are examples of a continuum of response properties.

Aggregate measures of neuronal activity recorded using low-impedance depth electrodes placed in the amygdala provide crucial additional evidence for face-processing in the amygdala. Such intracranial EEG signals are recorded with relatively large electrodes, thus yielding a less localized signal than a typical LFP [100]. Nevertheless, it has been found that such signals from within the amygdala exhibit oscillatory power increases in the theta-and gamma bands specifically to faces and particularly so for eyes as well as threatening faces [65] [102] [79]. Whether these response properties reflect the input or are locally computed cannot be resolved from these studies, but it nevertheless shows that even a coarse aggregate measure of neuronal activity results in face-and eye-specific activity. This provides additional evidence for the prominence of these signals in the human amygdala. Across single-unit and LFP studies of the human amygdala there are also important findings regarding response latencies. In general, single-unit responses to faces in the human amygdala are surprisingly slow [70], and considerably slower than such responses in the monkey amygdala (by approximately 100 ms). Looking across studies that have recorded LFPs, there is a large range of response latencies; one recent synthesis partitioned responses into “early” (50–290ms, with some studies showing LFP-based responses at latencies much shorter than any observed in single-unit recordings), “intermediate” (270–470ms, often novel and task-relevant stimuli and possibly reflecting attentional effects), and “late” (600–1400ms, involving semantic processing and working memory) [76].

One of the most exciting future avenues is the investigation of psychiatric illnesses that are thought to involve the amygdala. There is already a large literature, primarily from PET and fMRI studies, that suggests the amygdala is involved in a number of disorders that show prominent dysfunction in emotion or social behavior, notably including depression, anxiety disorders, PTSD and autism. Remarkably, some of these disorders are now being explored for intervention using direct electrical stimulation of the amygdala (see Box 1). Equally remarkably, one can record single neurons from the amygdala in psychiatric patients—the reason being that epilepsy is often comorbid with such disorders (and, even if it were not, there are of course substantial individual differences across patients that one could capitalize from as well).

In the first such study, we recently recorded from amygdala neurons in two patients with a diagnosis of autism spectrum disorder, and compared their neuronal response selectivity to that of other patients who did not have autism [99]. One important finding of this study was how typical neuronal responses were in the patients with autism: the electrophysiological signatures we verified looked comparable to those of other patients who did not have autism, including spike waveforms, interspike intervals, coefficients of variation, and spontaneous and evoked spike rates. Also, they had similar (and high) proportions of neurons responsive to whole faces as well as to parts of faces. In contrast, the specific features of the face to which the face part-selective cells responded were quite different. Whereas most part-selective neurons respond to the eye region of the face normally, in patients with autism these neurons instead responded to the mouth region of the face (Figure 3). What is remarkable about this finding is not only its specificity, but also the fit with what else we know about face processing in autism. From many studies, it is known that people with autism do not typically fixate the eye region of faces, and fail to make use of eye information from faces, instead focusing on the mouth. Our single-unit recordings thus provide the first window into a neuronal correlate, and suggest that amygdala neurons may encode those regions of the face that are most salient for a particular subject—normally the eyes, but in autism the mouth. Given that autism symptomatology can be thought of as a continuously distributed trait also in the normal population, examining variability in the feature selectivity of amygdala neurons across subjects will constitute a very important future direction. Currently, this approach is limited simply by sample size, but with the accrual of sufficiently large numbers of subjects, that issue can be surmounted.

Neuronal responses to faces in the monkey amygdala

Non-human primates are an excellent model system to investigate facial processing, because they are, like humans, intensively interested in looking at faces and in particular at eyes [33, 58]. Similar to the human amygdala, the non-human primate amygdala contains a substantial proportion of neurons which are sensitive to faces [36, 53, 77, 111] [53, 84] (Figure 4). For example, an early study noted that “it is of interest that the most effective stimuli of about one half of the neurons were images of monkeys” [77]. In contrast, considerably fewer responses could be identified to simple shapes, even when of equal luminance. Predominantly located in the lateral and basolateral nucleus, neurons have been identified that respond only to monkey or human faces [36], only to a specific individual human or monkey [36, 111], only to a specific facial expression such as smiling or threat [36, 111] or to specific gaze and head orientations [73, 111]. Sensitivity to gaze has been demonstrated by Mosher et al., who showed that a subset of eye-contact sensitive cells respond specifically only when the viewer (whose neurons are being recorded) is actively making eye contact with the partner monkey appearing on a screen [73].

Figure 4. Face-response amygdala neurons in macaques.

(A–C) Example neurons that increase their firing rate whenever the animal makes a saccade that lands on an eye of a face displayed on a screen. (C) Eye-specific responses can be seen both for dynamic as well as static stimuli. (D) Neuron responsive to the face of a particular individual regardless of facial expression. (E) Cell responsive to a particular emotional expression, regardless of individual. Figures modified from [36, 73].

It is of note that, similarly to the human studies [98], a substantial proportion of neurons responded with a reduction in firing rate to the onset of stimuli, and such reduction in firing could be equally specifically tuned to faces or features thereof [53, 72]. Given the presumed excitatory nature of the feed-forward inputs from temporal cortex carrying visual information, this salient aspect of amygdala neuronal responses indicates that these neurons receive locally processed information from elsewhere within the amygdala, within which inhibitory connectivity is prominent. This hypothesis has, to our knowledge, not been explored so far and would be important also to relate to the much better known intra-amygdala circuitry studied in rodents.

It has been suggested that the amygdala represents not only the perception of facial expressions of others but also the production of facial expressions in the animal from which the neurons were recorded [55]. This hypothesis is motivated by the strong relationship of amygdala neurons with neurons involved in production of facial expressions, i.e. through a prominent projection to the motor parts of the cingulate cortex that in turn control facial muscles [69]. Through such connections, the amygdala would be ideally suited to monitor self-executed facial expressions [37], together with other areas thought to be involved in the control of facial musculature [39]. Supporting this, in macaques it has been found that many neurons responded immediate before or after onset of facial muscle activity [55]. This hypothesis has so far not been explored in humans, but promises to be a rich avenue for future discoveries.

Responses to other visual stimuli

The activity of some human amygdala neurons discriminates between high-level attributes of visual stimuli, such as whether the image contained an instance of a particular visual category such as an animal, a landscape or a vehicle. Such category-selective neurons can be found throughout the human medial temporal lobe (MTL), including but not limited to the amygdala [31, 42, 51]. A notable specialization of the amygdala may be the proportion of neurons responsive to pictures of animals: a significantly higher proportion of neurons responded selectively to animals in the right amygdala compared to other MTL areas such as the entorhinal cortex, parahippocampal cortex, or hippocampus [71]. It is notable that the neuronal responses to animals were not attributable to the animals being threatening – responses were equally strong for non-threatening pictures such as those of rabbits or horses. This nonspecificity to threat-related stimuli is also found for other stimulus categories and modalities [51, 93], indicating that the amygdala receives and represents highly processed multi-modal sensory inputs regardless of whether they are threatening or not (as is the case for fMRI responses in the amygdala, as we noted earlier). It is notable that the human amygdala also contains neurons which have a very sparse and specific response to individual concepts such as a particular person, a finding which again is valid for many areas in the MTL [93]. What remains unclear is why such visually highly selective neurons are so distributed in the MTL, without any apparent regional specialization.

Amygdala neurons and subjective perception

A key question is whether the response of individual neurons is determined entirely by sensory input or whether these responses are rather representing an internal percept or decision. As we noted earlier, fMRI studies provide strong evidence that sensory responses are modulated by the context and meaning in which they occur, suggesting the possibility that single neurons in the amygdala might similarly show large modulations by other factors and internal states. It has remained challenging to dissect these possibilities, because in the large majority of experiments stimulus-driven responses are indistinguishable from responses that might instead correlate with more flexible internal states evoked by the stimuli. For example, if a neuron increases its firing rate for only fearful faces, is this due to certain visual features present in the face stimulus, or rather because of a decision made by the viewer that this face looks fearful? Studies with humans have started to address this question through protocols where stimuli are sufficiently ambiguous or difficult so that variable behavioral reports (decisions) are made for multiple presentations of the same stimuli [91, 114]. In one such study, we identified facial emotion-selective neurons during a task where patients are asked to classify the emotion of faces as either happy or fearful [114]. We found that during some trials, patients made clearly identifiable errors, i.e. they classified happy faces as fearful and vice-versa. On those trials, we found that emotion-selective neurons did not follow the stimulus but rather represented the subjective (but wrong) decision of the patient: the amygdala neurons encoded the subjective judgment about the emotion, not the objective features shown in the face. We also examined a set of emotion-selective neurons in the hippocampus in the same experiment, and here we found that hippocampal neurons—by contrast—encoded the objective stimulus features, and not the subjective judgment about the emotion.

The subjective nature of visually-evoked responses in the amygdala has also been observed when comparing responses to ambiguous stimuli consisting of morphed versions of faces of two different individuals (A and B). This paradigm yields the interesting behavioral situation where an identical ambiguous picture is perceived as either person A or B on different trials. Examining a neuron tuned to individual A according to the decision that was made revealed that the neuron responded whenever the image was perceived as individual A but not individual B [91].

This line of work contributes to the debate on the role of the amygdala in conscious versus non-conscious perception. For instance, whereas several earlier studies argued that the amygdala was activated by faces even under subliminal presentation [115], later studies argued against this finding [85] and instead suggest that the amygdala plays a more explicit and elaborated role through its interactions with cortical processing [86].

Role of amygdala in learning

The amygdala has long been implicated in learning and memory, often through emotion-dependent modulation of these processes [62, 88]. Such modulation arises from the amygdala’s extensive connectivity with other brain regions, in particular neocortex [86] and the hippocampus. The amygdala is known to play important roles in Pavlovian, instrumental, and episodic forms of learning and memory. Particularly well-studied in animals have been Pavlovian fear conditioning [19] and instrumental learning [75], while in humans there has been study of the emotional modulation of episodic memories [62]. Indeed, the amygdala-facilitated enhancement of memories accompanied by strong emotions is now recognized as a central component of PTSD [107]. For declarative memories, the amygdala influences memory encoding by modulating the strength of plasticity in the hippocampus, a structure with which several amygdala nuclei form strong connections [62, 108]; moreover there is functional evidence of amygdala-hippocampal communication during the encoding of emotional stimuli from fMRI studies in humans [25]. Social situations frequently give rise to strongly emotional episodic memories, and our social behavior is clearly dependent on our ability to retrieve such memories flexibly as we encounter familiar people. While seemingly a separate topic, the amygdala’s role in attention and memory can thus be seen to be of particular importance in social cognition, and consistent with the role in social perception we reviewed above [3].

One of the most important aspects in both social cognition and learning is the rapid identification of novel stimuli. The human amygdala is highly sensitive to stimulus novelty and in particular to novel faces, a finding which has been demonstrated by both single-unit recordings [31, 96] as well as fMRI [9]. This indicates that the role of the amygdala in learning goes beyond memories with a strong emotional component. Similarly, in macaques, many visually responsive neurons respond preferentially to novel or otherwise unfamiliar stimuli, such as food items not experienced before [77]. It is notable that such novelty-dependent responses are present in the complete absence of emotional content, i.e. for neutral faces or neutral scenes such as landscapes or a car [9, 96].

Novelty-dependent processing is a critical component of episodic memories and requires rapid plasticity. Neuronal responses in the human amygdala are highly plastic [96], supporting a role in learning. For example, individual neurons in the human amygdala can modify their response after a single-trial: after a single exposure, the response of novelty-responsive neurons was abolished. While it is unknown where this rapid plasticity occurred, this shows that the amygdala has access to such rapidly modifiable representations. In rodents, theta-oscillations coordinate and modulate plasticity strength in multiple areas [16], including the amygdala [90]. This is also the case in humans: many human amygdala neurons phaselock to local theta oscillations and the strength of such phase-locking is predictive of successful memory formation [97]. Both novelty-dependent responses as well as theta-phase locking are likely crucial for learning in social behavior, a hypothesis that remains to be explored. A particularly intriguing hypothesis is the role of eye movements in this process. It has recently been demonstrated that ongoing theta oscillations in the hippocampus are modulated by eye movements every time a fixation on a stimulus is made [44, 48]. Because theta is principally controlled by a central pacemaker in the medial septum which in turn is modulated by the brainstem [90], it is reasonable to hypothesize that this is also the case in the amygdala. As outlined above, one of the primary means by which primates gather information and interact with each other in social situations is by making saccades onto relevant stimuli, in particular faces. We thus hypothesize that the modulation of ongoing theta oscillations in the amygdala by eye movements plays a crucial role in learning during social situations. This hypothesis remains to be explored experimentally.

A critical component of learning is a representation of reinforcers and the stimuli that predict them. The amygdala is a major component of the brain’s reward system, and amygdala neurons in macaques signal the reward likelihood and value of conditioned stimuli [12] and such responses are learned rapidly. For example, while an animal learns to associate a visual stimulus with a negative or positive reward, some neurons in the amygdala start to respond selectively only to visual stimuli that predict a negative or positive reward and this association reverses rapidly during reversal learning [11, 80]. These results indicate that the amygdala plays a role in associating value with sensory stimuli. Critically, amygdala neurons were found to signal both negative and positive reward-associated stimuli [80], once again indicating a broader role of the amygdala in learning beyond negative reinforcers. Neurons in the human amygdala also carry a value signal: during a task requiring a choice between two food stimuli, individual neurons signaled the value associated with the stimuli presented [46]. Beyond this, nothing yet is known about value encoding in human neurons and whether such neurons can acquire value encoding rapidly through learning. A critical question is whether amygdala neurons can encode value in situations beyond conditioning.

The amygdala and attention

It is possible that many of the functions of the amygdala discussed in this review are ultimately attributable to the amygdala’s role in attentional processing [45]. For instance, it is well known that the amygdala modulates emotional arousal[116]. Aspects of learning, memory, and perception that involve the amygdala may therefore derive from an attention signal: attention towards emotional stimuli enhances their encoding into memory, attention to rewards and punishments enhances learning of instrumental behavior, and attention to faces and their features modulates how we fixate them and how they are processed by the visual system.

While a role of the amygdala in overall attentiveness and arousal has long been recognized, single-neuron findings from non-human primates have revealed that the amygdala also has a crucial role in spatial attention [81, 82]. Responses showed spatial selectivity both in terms of location (e.g., on which side of the visual field a stimulus is presented) and task demands (e.g., sustained responses reflecting spatial attention to a reward) [83]. Remarkably, the variability of the firing rate of amygdala neurons was systematically related to variability in spatial attention as assessed by behavior [83]. This strongly supports a role for the amygdala in spatial attention, a finding which suggests that abnormal face-selective responses in the amygdala seen in patients with autism (see above) could arise from abnormal allocation of attention. Indeed, it is possible that the abnormal attentional signal originates from the amygdala itself, and modulates visual processing that in turn provides input to the amygdala. Future human single-neuron studies combined with explicit instructions to modulate attention are needed to investigate this possibility directly.

Amygdala and prefrontal cortex

The amygdala receives inputs from and provides output to several areas of prefrontal cortex, including bilateral connections with orbital and medial prefrontal cortex, and anterior cingulate cortex, which project especially heavily to subdivisions of the basal nucleus [29]. Recent anatomical studies in monkeys have identified specific circuits by which amygdala neurons target layers of prefrontal cortex [113] and there are strong indirect connections between these two regions via the dorsomedial thalamus [68]. Extinction of Pavlovian fear conditioning relies on prefrontal modulation of amygdala activity [89], and has been a topic of considerable interest in relation to anxiety disorders [38]. Electrical stimulation of medial prefrontal cortex in rodents inhibits CeA neurons [92]. Together, these findings suggest that prefrontal cortex may inhibit amygdala responses, possibly implementing information about task set and context.

Abnormal connectivity between amygdala and prefrontal cortex has been implicated in psychiatric disorders. For instance, both functional coupling of fMRI responses, as well as structural connectivity measured with diffusion MRI, have documented a correlation between connectivity and psychopathology [87]. In humans, some of the most exciting causal links between prefrontal cortex and the amygdala are emerging from rare studies combing lesions or electrical stimulation with fMRI. For instance, amygdala lesions result in abnormal BOLD signal in anterior cingulate cortex during reversal learning [41], and lesions of ventromedial prefrontal cortex result in abnormal activation of the amygdala [74].

Conclusions and future directions

Several important conclusions set the stage for future studies. Overall, a salient finding is that both monkey and human amygdala neurons respond prominently to faces; that a subset of these neurons responds only to whole faces, whereas another subset responds to specific parts of faces; and that some neurons encode specific emotional expressions. The single most important task lying ahead is to delineate the functional role and significance of these findings.

To better understand the functional contribution made by the amygdala, it will be essential to make several kinds of comparisons. One is to compare responses in the amygdala to responses in temporal face-selective regions that provide input to the amygdala, through concurrent recordings in both regions to identical stimuli. This will reveal important information for models describing how amygdala responses are synthesized from the responses of their temporal inputs. Similarly, simultaneous recordings of medial prefrontal cortical areas together with amygdala responses will reveal how mPFC inputs can modulate amygdala responses. Such studies would be extremely important for understanding individual differences and psychiatric illnesses, since it could help to better delineate exactly where in the brain primary dysfunction arises. Is it in the circuitry of the amygdala itself, in other brain structures the project to the amygdala, or even in white matter connectivity between these two?

Other comparisons are across species and across methods. What might be different in responses of the monkey amygdala and responses of the human amygdala, even once we take into consideration the species differences in the meaning of social stimuli? How well can we explain BOLD-fMRI responses from single-unit and LFP responses in the amygdala? For primate V1, detailed work in the non-human primate has revealed a detailed understanding of the relationship between the BOLD signal, the LFP and single-unit response [56]. Similar such comparisons will need to be performed for the amygdala, the lack of which at present limit our understanding of the BOLD signal that originates from the amygdala. From such an analysis, could we isolate components of the BOLD response in the amygdala, in a certain experiment, that might reflect a specific input to the amygdala?

It is important to consider the merits of single-neuron recordings in macaques compared to humans. On the one hand, recordings from humans are a unique opportunity to investigate directly aspects of human social cognition and psychiatric diseases. On the other hand, work with non-human primates are better controlled, allow for extensive recordings over a long time and permit many experimental manipulations not possible in humans. Thus, while experiments with humans will be instrumental and absolutely necessary to begin to answer many of the questions raised in this review, for many questions it will be necessary to perform detailed follow-up studies in non-human primates. The combination of both together, with as similar paradigms as possible, promises to be a very powerful experimental approach.

Finally, we would like to stress the importance of combining electrophysiology with behavior: we will only be able to determine what it is that amygdala neurons represent by linking their responses to both stimuli and behavior. A example of the power of this approach is our recent study (discussed above) that human amygdala responses encode the subjective percept of emotional faces [114]. Other examples, currently underway in several laboratories, are to combine high-resolution eyetracking with electrophysiology: do amygdala neurons respond prior to fixations onto a target, or only once a target is fixated? With the burgeoning interest in single-unit recordings in humans, and the renewed interest of such recordings in monkeys, we are hopeful that many of the above comparisons can be realized in the near future.

Highlights.

• -Individual neurons in the human and non-human amygdala are highly sensitive to faces
• -Sub-populations of neurons respond to specific features such as eyes
• -Stimulation of the amygdala is being explored as a possible treatment for PTSD and autism
• -Comparisons with responses in temporal cortex, and across species are important future directions

Box: Outstanding questions.

• -Can electrical stimulation of the amygdala provide relief from some psychiatric illnesses?
• -Do all amygdala responses contribute to conscious experiences, or might some not—and what would account for the difference?
• -How do individual differences in personality, mood, gender, and genotype relate to possible differences in amygdala responses to faces?
• -What is the difference in what the amygdala contributes to face processing, and what temporal neocortex contributes?
• -What is the difference in what the amygdala contributes to novelty detection and what the hippocampus contributes?
• -Why are the latencies of single-neuron amygdala responses so long in humans—and why do they seem longer than single-neuron responses in monkeys, or LFP responses in humans?
• -What is the contribution of the amygdala to spatial attention?
• -Do amygdala neurons play a pre-motor role in directing gaze? Do they play a pre-motor role in the production of facial expressions?

Glossary

ECOG

electrocorticogram, intracranial EEG

PTSD

post-traumatic stress disorder

ASD

autism spectrum disorder

DBS

deep brain stimulation, a clinical treatment for movement disorders that is also sometimes used to treat psychiatric disorders. Injects high-frequency extracellular stimulation, which is thought to inhibit neural activity in the target area.

MTL

medial temporal lobe, term used to jointly refer to the amygdala, hippocampus and associated cortical regions (parahippocampl, entorhinal, and perirhinal cortex)

Theta oscillation

3–8Hz oscillation in the, prominent in many brain areas and in particular the hippocampus but also the amygdala.

MRI

Magnetic resonance imaging

fMRI

functional magnetic resonance imaging

BOLD

Blood-oxgygen-level dependent

BLA

basolateral nucleus of the amygdala

CE

central nucleus of the amygdala

PFC

prefrontal cortex