Repetition suppression of faces is modulated by emotion

Abstract

Single-unit recordings and functional brain imaging studies have shown reduced neural responses to repeated stimuli in the visual cortex. By using event-related functional MRI, we compared the activation evoked by repetitions of neutral and fearful faces, which were either task relevant (targets) or irrelevant (distracters). We found that within the inferior occipital gyri, lateral fusiform gyri, superior temporal sulci, amygdala, and the inferior frontal gyri/insula, targets evoked stronger responses than distracters and their repetition was associated with significantly reduced responses. Repetition suppression, as manifested by the difference in response amplitude between the first and third repetitions of a target, was stronger for fearful than neutral faces. Distracter faces, regardless of their repetition or valence, evoked negligible activation, indicating top-down attenuation of behaviorally irrelevant stimuli. Our findings demonstrate a three-way interaction between emotional valence, repetition, and task relevance and suggest that repetition suppression is influenced by high-level cognitive processes in the human brain.

The neural signature of stimulus repetition is decreased activation in the cortex, a phenomenon known as repetition suppression. Single-unit recordings in nonhuman primates have shown reduced neural responses to repeated visual stimuli in extrastriate cortex (1, 2). Functional brain imaging studies in humans using various techniques [positron-emission tomography, functional MRI (fMRI), and event-related potentials] have also shown that stimulus repetition results in decreased cortical activation (3-7). Repetition suppression is stimulus-specific, size- and location-invariant, and observed under anesthesia (8). These properties have suggested that repetition suppression is an automatic, intrinsic response of cortical neurons (9, 10).

Repetition suppression has been observed with various classes of visual stimuli, including words, objects, and faces. Emotional stimuli, such as words or pictures with aversive content, may comprise a privileged stimulus category with prioritized processing (11). Behavioral studies have demonstrated that stimuli depicting negative emotions interfere with other tasks more than neutral stimuli (12, 13) and recruit attention more readily (14, 15). Emotional stimuli evoke greater cortical activation than neutral ones (16, 17) and may be processed by a subcortical route to the amygdala that enables the rapid detection of potential danger (18, 19). In an analogy to the role of focal attention in biasing the competition for limited processing resources (20), it has been proposed that the increased activation associated with emotive stimuli may provide the neural basis for their behavioral advantage (21). If indeed emotional stimuli comprise a privileged category, they may be resistant to repetition suppression. Thus, repetition of fearful faces would not be associated with reduced responses. An alternative possibility is that repetition of emotional faces would result in stronger suppression effects. It has been suggested that the reduced neural responses associated with stimulus repetition reflect the shrinkage of the pool of activated neurons. Thus, with repetition, a smaller population of more selective neurons would respond to the stimulus, whereas the nonselective neurons would drop out of the pool (9). In this view, increased repetition suppression to emotional faces would result in a more selective response, leading to faster and more accurate processing of these stimuli.

We used event-related fMRI to test the extent to which repetition of fearful faces is associated with reduced responses. Subjects performed a working memory task in which encoded targets (behaviorally relevant) and distracters (behaviorally irrelevant) were repeated three times, intermixed with novel distracters. Behaviorally, we found shorter response latencies with repetition. Within all face-responsive regions, the repetition of fearful targets was associated with stronger decreased responses. Distracters, regardless of their repetition or valence, evoked negligible activation.

Experimental Procedures

Subjects. Thirteen normal, right-handed subjects (five males and eight females, aged 23 ± 2 yr) with normal vision participated in this study. All subjects gave written informed consent for the procedure in accordance with protocols approved by the National Institute of Mental Health Institutional Review Board.

Stimuli. Because many face stimuli were needed for the current event-related study, we photographed actors who portrayed neutral and fearful expressions. In a behavioral pilot, 13 subjects (not the subjects who participated in the fMRI experiment) rated the fearful faces from 1 (not fearful) to 5 (very fearful), and only faces that received the high fear scores (namely 4 and 5) were included. We used a total of 36 individual faces, each portraying both neutral and fearful expressions. Stimuli were generated by a Macintosh computer (Apple) by using superlab (Cedrus, Wheaton, MD) (22) and were projected with a magnetically shielded liquid crystal display video projector (Sharp, Mahwah, NJ) onto a translucent screen placed at the feet of the subject. The subject viewed the screen by a mirror system. Gray-scale photographs of neutral and fearful faces and phased scrambled pictures of these faces were presented in the center of the screen on a black background.

Task. Subjects performed a face working memory task. In each trial, a target face was presented for 4 sec, followed by 13 faces, each presented for 2 sec. The target and one of the distracters were repeated three times, intermixed with seven novel, neutral distracters. Each run included the following five trial types: neutral targets plus repeated neutral distracters, neutral targets plus repeated fearful distracters, fearful targets plus repeated neutral distracters, fearful targets plus repeated fearful distracters, and scrambled targets plus repeated scrambled distracters. Subjects were instructed to memorize the target face and press a button when detecting it, thereby making the target the behaviorally relevant stimulus. Each of the eight runs included 10 trials (each trial type was presented twice). The order of trial type within each run was randomized and counterbalanced. Thus, our study conformed a 3 × 3 × 2 factorial design. The first factor was repetition (first, second, third). The second factor was emotional valence (neutral versus fearful). The third factor was task relevance (target versus distracter).

Data Acquisition. A 3-T General Electric Signa scanner with a whole head coil (IGC, Milwaukee, WI) was used. Changes in blood oxygen level-dependent T2^*-weighted MRI signal were measured by using a gradient-echo echoplanar sequence (repetition time = 2 sec, echo time = 30 msec, field of view = 24 cm, flip angle = 90°, 64 × 64 matrix). In each time series, 24 contiguous, 5-mm thick axial slices were obtained (voxel size = 3.75 × 3.75 × 5 mm). High-resolution spoiled gradient recalled echo structural images were also collected (n = 124, 1.3-mm thick sagittal slices, time to repeat = 15 msec, time to echo = 5.4 msec, field of view = 24 cm, flip angle = 45°, 256 × 256 matrix). These T1-weighted images provided detailed anatomical information for registration and 3D normalization to the Talairach and Tournoux atlas (23).

Data Analysis. Data were analyzed by using afni Version 2.33a (24, 25). fMRI scan volumes were registered to the single functional image collected closest in time to the high-resolution anatomical images (26) and spatially smoothed in-plane with a 5-mm Gaussian filter.

We used the main effect of faces versus scrambled faces to identify face-selective regions and then examined the main effects and interactions among repetition, valence, and task relevance within these regions. The responses during face perception were analyzed by using a linear convolution model with an assumed hemodynamic response function (27). Voxels were selected that showed a significant effect (P < 0.0001, uncorrected) for the contrast of faces versus scrambled faces. A set of regions of interest was anatomically defined for each subject, including bilaterally the inferior occipital gyrus (IOG), the fusiform gyrus (FG), the superior temporal sulcus (STS), the amygdala, and the inferior frontal gyrus (IFG)/insula. The anatomical locations of these clusters were determined by superimposing the statistical maps on the coplanar high-resolution structural images.

We then estimated the amplitude of the event-related responses during the following event types: target encoding; first, second, and third repetitions of neutral and fearful targets; first, second, and third repetitions of neutral and fearful distracters; and presentation of the novel distracters. We took the average response within each region of interest and refitted the linear convolution model by using seven basis functions to provide a less constrained characterization of the response (a seven-point finite impulse response function with 2-sec bins).

For each subject and each region of interest, a mean time series averaged across activated voxels in the region and across all repetitions of each event type was calculated. These means were used for between-subjects random-effects analyses.

Results

Behavioral Data. The mean accuracies for detecting neutral and fearful targets while subjects performed the task in the scanner were 96% and 98%, respectively. Table 1 indicates the reaction times for first, second, and third repetitions of neutral and fearful targets. Detection of fearful faces was faster than detection of neutral targets (P < 0.0001). The difference in reaction times between the first and third repetitions of both neutral and fearful faces was statistically significant (P < 0.05).

Table 1. Behavioral data.

	First repetition	Second repetition	Third repetition
Neutral targets	767 ± 30	753 ± 28	747 ± 26
Fearful targets	724 ± 30	711 ± 33	704 ± 29

Mean reaction times are expressed in msec ± SEM averaged across 13 subjects.

Imaging Data. Activation evoked by visual perception of faces. Perception of faces, as compared with scrambled faces, significantly activated the IOG, FG, STS, the amygdala, and the IFG/insula (Fig. 1). Within these face-responsive regions, we found bilateral activation in all subjects (see Table 2 for cluster size and Talairach coordinates).

Fig. 1.

A network of face-responsive regions. Shown from left to right are coronal sections illustrating activation in the inferior occipital gyri (IOG, y = -75), FG and STS (y = -55), the amygdala (y = -8), and the IFG/insula (y = 22). The far right image shows vertical lines on whole-brain image indicate location of these sections. Data were averaged across all 13 subjects.

Table 2. Regions activated during visual perception of faces.

	Mean no. of voxels (SEM)	Coordinates
Region		x	y	z
L. IOG	63 (6)	−40	−79	−9
R. IOG	61 (6)	36	−80	−8
L. FG	64 (5)	−40	−55	−15
R. FG	67 (4)	35	−55	−13
L. STS	52 (8)	−55	−43	11
R. STS	81 (11)	50	−49	17
L. amygdala	43 (7)	−18	−7	−9
R. amygdala	46 (8)	15	−7	−8
L. IFG/insula	85 (13)	−38	21	5
R. IFG/insula	93 (17)	30	23	5

Volumes were calculated before spatial normalization. Coordinates are in the normalized space of the Talairach and Tournoux brain atlas. For each region, mean volume and coordinates were averaged across 13 subjects. L., left; R., right.

Response to targets. Having localized the visual activation evoked by faces, we analyzed the amplitude of the response associated with specific events, namely, encoding of targets, repetition of targets, repetition of distracters, and presentation of novel distracters. We first provide the results for activation evoked by the targets. Fig. 2 shows the amplitude of the response evoked by targets. In all regions, a significant, bilateral response was found during target encoding and target repetitions. The activation evoked by encoding of fearful targets was higher than the activation evoked during encoding of neutral targets (P < 0.001 in all regions). In the IOG and the IFG/insula, the first repetition of a target evoked a higher response as compared with target encoding, consistent with findings of “match enhancement” in the monkey (28). These enhanced responses were statistically significant for both neutral and fearful targets (P < 0.0001 in the IOG and P < 0.01 in the IFG/insula). The amplitude of activation during the first repetition of a fearful target was significantly higher than during the first repetition of a neutral target (P < 0.001 in all regions).

Fig. 2.

Activation evoked by targets. The mean amplitudes of the fMRI signal were averaged across all subjects and all repetitions of each event in each subject. In this and subsequent graphs, error bars indicate SEM. Response to neutral and fearful faces is color coded in blue and red, respectively. E indicates target encoding.

Relative to the first repetition of a target, the second and third repetitions of that target resulted in decreased activation. The repetition suppression as manifested by the difference in response amplitude between the first and second, second and third, and first and third repetitions of neutral and fearful targets is shown in Table 3. In all regions except for the left STS, the difference between the first and third repetitions of a target was highly significant. We therefore used the difference between first and third repetitions to test whether the reduction associated with repetition of fearful targets was stronger than that associated with neutral targets. The repetition by valence interaction was significant in all regions, except for the left IFG/insula (Table 3), indicating that repetition suppression of fearful targets was stronger.

Table 3. Target repetition suppression.

Target	Repetition	L. IOG	R. IOG	L. FG	R. FG	L. STS	R. STS	L. AMG	R. AMG	L. INS	R. INS
Neutral	First vs. second	0.01	n.s.	0.0001	0.001	n.s.	n.s.	0.01	n.s.	n.s.	n.s.
Neutral	Second vs. third	n.s.	0.0001	0.01	n.s.	n.s.	n.s.	n.s.	0.01	0.0001	0.001
Neutral	First vs. third	0.001	0.0001	0.0001	0.0001	n.s.	0.05	0.0001	0.001	0.0001	0.0001
Fearful	First vs. second	0.0001	0.0001	0.0001	0.0001	0.01	0.0001	0.0001	0.0001	0.0001	0.0001
Fearful	Second vs. third	0.0001	0.01	0.0001	0.0001	0.0001	0.001	n.s.	0.0001	0.001	0.001
Fearful	First vs. third	0.0001	0.0001	0.0001	0.0001	0.001	0.0001	0.0001	0.0001	0.0001	0.0001
Interaction		0.001	0.05	0.0001	0.0001	0.001	0.0001	0.001	0.0001	n.s.	0.0001

Significant differences (P < n) are indicated. n.s., nonsignificant difference. L., left; R., right; AMG, amygdala; INS, insula.

We then calculated a “suppression index,” namely, the difference in response amplitude between the first and third repetitions divided by their sum, thus providing a normalized estimation of the suppressive effect (Fig. 3). In all regions, repetition of fearful faces resulted in greater decreases than repetition of neutral faces (P < 0.01).

Fig. 3.

Target repetition suppression index in face-responsive regions. The index was calculated by subtracting the response amplitude during the third repetition of a target from its first repetition divided by their sum. L, left hemisphere; R, right hemisphere; AMG, amygdala.

Finally, we calculated the correlations between the mean reaction times and the mean amplitude of the fMRI signal during the repetitions of neutral and fearful targets (Table 4). Within all face-responsive regions, we found a high correlation between the behavioral performance and the fMRI activation.

Table 4. Correlation coefficients (r) between mean reaction times and fMRI signal in all face-responsive regions.

Target	L. IOG	R. IOG	L. FG	R. FG	L. STS	R. STS	L. AMG	R. AMG	L. INS	R. INS
Neutral	0.99	0.81	0.79	0.98	0.98	0.94	0.99	0.81	0.79	0.92
Fearful	0.99	0.99	0.99	0.99	0.95	0.99	0.97	0.98	0.80	0.99

L., left; R., right; AMG, amygdala; INS, insula.

Response to distracters. The amplitude of the response during presentation of novel distracters and during the first, second, and third repetitions of neutral and fearful distracters is shown in Fig. 4. Surprisingly, in all face-responsive regions, these behaviorally “irrelevant” faces evoked weak responses. The mean responses to the novel distracters and the first repeated neutral distracters were not statistically significant from baseline in the left hemisphere, but they were significant in the right (P < 0.001). The mean response to the first repetition of fearful distracters was significantly higher than baseline in both hemispheres (P < 0.0001). In all regions, the mean response to repeated fearful distracters was significantly higher than the mean response to repeated neutral distracters (P < 0.0001 in both hemispheres).

Fig. 4.

Activation evoked by distracters. The mean amplitudes of the fMRI signal were averaged across all subjects and all repetitions of novel, neutral distracters (N, black bars), repeated neutral (blue), and repeated fearful (red) distracters.

We found that in all regions, the difference in response amplitude between the first and third repetitions of either neutral or fearful distracters was not statistically significant. Moreover, the differences between activations evoked by the novel distracters and the first repetition of both neutral and fearful distracters (i.e., before the subjects realized that these were repeated distracters) were not statistically significant.

Comparison between targets and distracters. In all face-responsive regions, the behaviorally relevant face targets evoked significantly higher activation than the behaviorally irrelevant distracters (P < 0.0001 in both hemispheres). To directly compare activations evoked by targets to those evoked by distracters, we plotted the mean fMRI response during their first, second, and third repetitions, averaged across all face-responsive regions (Fig. 5). Targets evoked stronger responses, which were reduced with repetition. Moreover, the mean response to the first repetition of fearful targets was stronger than that of neutral targets, but subsequent repetitions of these fearful targets resulted in stronger decreases. Finally, although fearful distracters evoked stronger responses than neutral distracters, both evoked only weak activation and their repetition was not associated with decreased responses.

Fig. 5.

Comparison between activations evoked by repetition of targets and distracters. Mean amplitudes were averaged across both hemispheres of all face-responsive regions in all subjects. Solid and broken lines indicate targets and distracters, respectively. N, neutral; F, fearful; T, targets; D, distracters.

Discussion

By using event-related fMRI, we investigated the neural responses evoked by repetition of neutral and fearful faces, which were either task relevant (targets) or irrelevant (distracters). Behaviorally, we found facilitation with target repetition, as manifested by shorter reaction times, consistent with the repetition priming literature (e.g., see ref. 29). We found activation in a network of face-responsive regions, namely, the IOG, FG, STS, amygdala, and the IFG/insula. Within these regions, repetition of targets resulted in reduced activation. Although the encoding and initial repetition of fearful targets evoked greater activation than did neutral targets, subsequent repetitions of these fearful targets were associated with stronger suppression. Distracter faces, regardless of their valence or repetition, evoked only negligible activation.

Activation Within the Face-Responsive Regions. The visual response evoked by faces revealed bilateral activation in multiple regions of a distributed neural system for face perception (30). We found activation in the IOG and FG, the extrastriate regions proposed to mediate face recognition (31-37). Moreover, we found activation in the STS, which mediates the processing of cues for social communication, such as the direction of eye gaze (38, 39). Finally, we found activation in the amygdala and insula, regions sensitive to facial expressions, in particular fear, anger, and disgust (40-45). Within all face-responsive regions, we found greater responses to fearful faces. The response during encoding of fearful targets evoked stronger activation than encoding of neutral targets. Additionally, first repetition of fearful targets evoked stronger activation than the first repetition of neutral targets. Finally, fearful distracters, independent of repetition, evoked stronger activation than neutral ones. Although the amygdala is particularly sensitive to emotional faces, we found that all face-responsive regions showed this valence effect, extending previous reports of greater responses to fearful faces than neutral faces in the FG (e.g., see refs. 46 and 47).

It has been suggested that the amygdala conveys valence to cortical regions by virtue of its anatomical projections. In the monkey brain, the lateral nucleus of the amygdala receives input from the inferior temporal cortex (areas TEO and TE) and the STS; this visual information is conveyed to the basal nucleus, which projects back to areas in ventral occipito-temporal cortex (48-52). Thus, assuming the existence of similar anatomical connections in the human brain, the responses to fearful faces in extrastriate cortex may reflect amygdalar modulation.

Repetition of Fearful Faces Resulted in Stronger Suppression. Within all face-responsive regions, we found that repetition of target faces resulted in decreased neural responses, consistent with previous functional brain imaging studies of repetition suppression (3-6). Although the response during the first repetition of a fearful target was stronger than that of a neutral target, subsequent repetitions of fearful faces were associated with stronger reduced responses as compared with the repetition of neutral faces.

It has been suggested that fearful faces might be somewhat resistant to the effects of repetition suppression (53). In that study, repetition of unpleasant faces was associated with less suppression than repetition of neutral faces in occipito-temporal cortex but not in the amygdala. The reduced suppression effect in visual cortex may be explained by the emotional faces used, which were distorted and bizarre rather than expressing natural, negative emotions. These unnatural faces may also explain the lack of a valence effect in occipito-temporal cortex, which is normally observed with natural, negative facial expressions.

It has been proposed that the reduced activation associated with stimulus repetition may reflect the sharpening of its cortical representation. With repeated presentations, neurons that are not well tuned to the features of a stimulus drop out of the pool of responding neurons. Thus, as a result of stimulus familiarity, a smaller, highly selective population of neurons is activated, whereas the responses of other less selective neurons is diminished (9). This model could explain why stimulus repetition results in both reduced neural responses and faster reaction times observed within and across sessions (10). We found that detection of fearful targets was significantly faster than detection of neutral targets. Furthermore, in all face-responsive regions, repetition of fearful faces resulted in stronger suppression effects. Finally, we found high correlations between the mean reaction times and the mean amplitudes of the fMRI signal. The implication of these findings is that emotional faces become more sharply tuned than neutral faces, or at least at a faster rate, presumably reflecting the importance of responding rapidly to biologically significant stimuli.

Our data also support the predictive coding model, according to which learning is reflected by reduced prediction error, as manifested by reduced activation with repetition (54). Consistently, it has been shown that when high-level visual areas can “explain” a stimulus, activation in primary visual cortex is reduced through feedback processes (55). Similarly, the short-term plasticity observed in our study can be attributed to error suppression, as reflected by reduced activation with target repetition.

A recent report by Avidan et al. (56) has shown that high-contrast stimuli, which initially evoked stronger activation than low-contrast stimuli, exhibited a greater reduction in responses with subsequent repetitions. We found that, relative to neutral targets, the first repetition of fearful targets evoked stronger responses that were subsequently reduced to a greater extent. In this view, emotional valence, like high-contrast, may bias the processing of a stimulus in its favor. Although the stronger response and subsequent greater reduction associated with high-contrast stimuli are likely mediated by sensory, bottom-up mechanisms that are intrinsic to visual cortex (see also ref. 57), the valence effect reported here is likely mediated by feedback effects to visual cortex, presumably arising in the amygdala. Attenuation of activity in the amygdala with repeated presentations of emotional faces has been previously observed (40), even in the absence of behavioral consequences. In addition, repetition decreases in occipito-temporal activation have recently been reported for both attended and unattended fearful faces (58). It therefore appears that the sharpening of representations of emotional stimuli within visual processing areas by means of feedback projections from the amygdala could be largely an automatic process.

Weak Activation Evoked by Distracters. We found differences in response amplitudes evoked by targets and nontargets. In all face-responsive regions, the behaviorally relevant target faces evoked significantly stronger activation than the irrelevant distracter faces. Indeed, responses to distracters were greatly attenuated. Although fearful distracters evoked stronger activation than neutral ones, repetition of both was not associated with reduced responses. Previous studies have shown that the mere repetition of an item, be it a target or a distracter, resulted in decreased responses (28, 59). Moreover, it has been proposed that repetition priming is an automatic process in the human brain (60, 61). The weak activation evoked by distracters in the current study suggests an early top-down attenuation of the response to these stimuli. Because no region was found that responded more to distracters than to targets, additional studies will have to determine which brain regions may be the source of such top-down control. Attenuation of the response to distracters may explain the absence of repetition suppression for these stimuli: once filtered out, there may be no need for the brain to tune the representation of the task-irrelevant stimuli.

Other recent studies have also shown that top-down effects, such as stimulus familiarity and task demands, can modulate the decreased responses associated with stimulus repetition (62, 63). Thus, under certain circumstances, the repetition of a stimulus by itself may be insufficient to produce repetition suppression. Cognitive factors, such as task relevance, may have a powerful effect. The task relevance of targets was presumably accompanied by enhanced attention to them. At the same time, successful performance of target detection was accompanied by attenuated responses to intervening distracters, despite their valence and independent of their repetition.

Match Enhancement. In the IOG and the IFG/insula, we found that the first repetition of a target was associated with enhanced responses, as compared with the response during encoding. A similar enhancement effect was previously found for inferior temporal and prefrontal neurons in monkeys performing delayed matching tasks (28, 64). Desimone (9) has suggested that repetition suppression and target enhancement, the two neural mechanisms observed in matching tasks, have complimentary functions: automatic detection of stimulus repetition and maintenance in working memory, respectively. Importantly, these parallel mechanisms are required to bias the competition between multiple objects in typically crowded visual scenes in favor of the behaviorally relevant items. Our findings indicate a strong top-down modulation that depends on the behavioral context: The processing of relevant targets is sharpened, whereas the processing of nontargets is greatly attenuated, even when these irrelevant stimuli are emotional faces.