Skip to main content
PMC home page
Frontiers in Human Neuroscience logoLink to Frontiers in Human Neuroscience
. 2012 Feb 22;6:29. doi: 10.3389/fnhum.2012.00029

Brain responses in evaluating feedback stimuli with a social dimension

Yuan Zhang 1, Xiang Li 1, Xing Qian 2, Xiaolin Zhou 1,2,3,*
PMCID: PMC3284147  PMID: 22371701

Abstract

Previous studies on outcome evaluation and performance monitoring using gambling or simple cognitive tasks have identified two event-related potential (ERP) components that are particularly relevant to the neural responses to decision outcome. The feedback-related negativity (FRN), typically occurring 200–300 ms post-onset of feedback stimuli, encodes mainly the valence of outcome while the P300, which is the most positive peak between 200–600 ms, is related to various aspects of outcome evaluation. This study investigated the extent to which neural correlates of outcome evaluation involving perceptually complex feedback stimuli (i.e., female faces) are similar to those elicited by simple feedback. We asked participants to judge the attractiveness of blurred faces and then showed them unblurred faces as implicit feedback. The FRN effect can be identified in the ERP waveforms, albeit in a delayed 300–380 ms time window, with faces inconsistent with the initial judgment eliciting more negative-going responses than faces consistent with the judgment. However, the ERP waveforms did not show the typical pattern of P300 responses. With the principal component analysis (PCA), a clear pattern of P300 effects were revealed, with the P300 being more positive to faces consistent with the initial judgment than to faces inconsistent with the judgment, and more positive to attractive faces than to unattractive ones. The effect of feedback consistency did not interact with the effect of attractiveness in either the FRN or P300 component. These findings suggest that brain responses involved in processing complex feedback stimuli with a social dimension are generally similar to those involved in processing simple feedback stimuli in gambling or cognitive tasks, although appropriate means of data analysis are needed to reveal the typical ERP effects that may have been masked by sophisticated cognitive (and emotional) processes for complex stimuli.

Keywords: outcome evaluation, performance monitoring, facial attractiveness, ERP, FRN, P300, PCA

Introduction

Over the past decade, there has been increased interest in the neural basis of performance monitoring and outcome evaluation, which plays an important role in decision-making and learning from the environment. Electrophysiological studies on outcome evaluation and performance monitoring have consistently observed two event-related potential (ERP) components (Miltner et al., 1997; Gehring and Willoughby, 2002; Holroyd and Coles, 2002; Nieuwenhuis et al., 2004). The first component, the feedback-related negativity (FRN), is a negative deflection at frontocentral recording sites that typically reaches a maximum amplitude around 250 ms post-onset of the feedback stimulus. It is sensitive to the outcome valence, being more negative-going for negative feedback associated with unfavorable outcomes, such as incorrect responses (Miltner et al., 1997), monetary losses (Gehring and Willoughby, 2002), or violations of expectancy (Heldmann et al., 2008; Wu and Zhou, 2009), than for positive feedback. Another ERP component, the P300, is the most positive peak in the 200–600 ms period post-onset of feedback stimulus and typically increases in magnitude from frontal to parietal sites. The P300 is traditionally believed to reflect processes demanding attentional resources (Polich and Kok, 1995; Polich, 2007) and it has been found to be related to various aspects of outcome evaluation, including the magnitude of reward (Yeung and Sanfey, 2004; Sato et al., 2005), expectancy toward the outcome (Hajcak et al., 2005, 2007; Wu and Zhou, 2009), and the valence of the outcome (Hajcak et al., 2005, 2007; Wu and Zhou, 2009; Leng and Zhou, 2010).

Almost all the previous studies about outcome evaluation and performance monitoring employed relatively simple tasks (e.g., making a gambling choice or estimating time duration elapsed for a simple visual stimulus), with feedback stimuli that demand little perceptual or cognitive processing (e.g., numerals representing monetary reward or symbols indicating the correctness of estimation). However, in our daily life, we often encounter more complex feedback stimuli, which require sophisticated cognitive processing, particularly stimuli that are important in social interactions (e.g., faces). Moreover, we often make judgments or predictions based on limited information. The later outcome evaluation may involve an implicit comparison that checks the previous judgment or expectancy stored in memory against the newly available information. For example, when we think we recognize a friend from distance we might want to approach and greet him. Only when we come closer do we see details of his face and realize that he is not the expected person. During the time between seeing the individual from afar and approaching him, we form expectations toward meeting a particular person based on partial information; the later implicit comparison between the new information and the initial expectation allows us to see whether our initial expectation or judgment was correct. It is not clear to what extent the pattern of brain responses to the new, complex feedback information is affected by the result of comparison (i.e., the consistency between the initial judgment or expectation and new information) and by the neurocognitive processes associated with the complex feedback stimulus itself.

The main purpose of this study is to investigate to what extent the neural correlates of the (implicit) outcome evaluation involving complex feedback stimuli (e.g., faces) are similar to those revealed for simple feedback in gambling or simple cognitive tasks. Answers to this question may open a new avenue for the use of the ERP technique to investigate the neural basis of outcome evaluation and performance monitoring in more complex, including social, situations. To mimic a situation in which limited information is available for decision-making and the feedback stimuli require sophisticated processing, we blurred photos of female faces and asked participants to judge whether a presented female face was attractive or unattractive. Photos of unblurred faces were presented as feedback and ERPs time-locked to the onset of the feedback were measured. Facial attractiveness is a key feature in social interactions, such as peer and mate choice (Etcoff, 1999; Thornhill and Gangestad, 1999; Johnston, 2006) and inferences about that individual's personality (Dion et al., 1972). If participants did make (implicit) comparisons between newly available information and previous judgments, then the feedback faces would carry information concerning the valence of feedback (consistent vs. inconsistent with the initial judgment), in addition to information concerning facial attractiveness. The processing of the social aspect of the feedback faces (i.e., attractiveness) becomes a necessary step in deciding whether the initial judgment or guess was right. A previous study found that the ventral occipital region including the fusiform face area (FFA) shows differential responses to attractive and unattractive faces even when the task is beauty-irrelevant (e.g., in an identity judgment task), indicating that facial attractiveness may be processed automatically (Chatterjee et al., 2009).

If the neural correlates of outcome evaluation for complex feedback stimuli with a social dimension are similar to those revealed for simple stimuli, inconsistent faces (those whose attractiveness is inconsistent with the initial judgment) should show an FRN effect relative to consistent faces. On the other hand, previous studies demonstrated that the P300 is sensitive to the reward valence in monetary gambling tasks (Hajcak et al., 2005, 2007; Wu and Zhou, 2009; Leng and Zhou, 2010), with more positive amplitudes for positive feedback than for negative feedback. If so, the P300 should be more positive to feedback faces consistent with the initial judgment than to faces inconsistent with the judgment. Moreover, previous studies also showed that ERP responses are more positive to attractive faces than to unattractive faces on late positive potential (LPP) or P300 (Johnston and Oliver-Rodriguez, 1997; Oliver-Rodríguez et al., 1999; Werheid et al., 2007; Schacht et al., 2008). We, therefore, predict a P300 or LPP effect for facial attractiveness in this study.

Method

Participants

Sixteen right-handed undergraduate students (eight females, 18–24 years, mean = 21) from Peking University participated in the experiment. All the participants had normal or corrected-to-normal vision, and had no history of neurological, psychiatric, or cognitive disorders. This study was conducted in accordance with the Declaration of Helsinki and was approved by the Academic Committee of the Department of Psychology, Peking University.

Design and materials

This experiment had a 2 by 2 factorial design, with the first experimental factor (consistency: consistent vs. inconsistent) referring to whether the attractiveness of feedback faces were consistent with the initial judgment and the second experimental factor (attractiveness: attractive vs. unattractive) referring to the facial attractiveness of the feedback faces. The experiment included pre-test, EEG test, and post-test. In the pre-test, 170 attractive and 170 unattractive gray-scale photos of female faces were selected from the photo pools of Peking University and the Institute of Psychology, Chinese Academy of Sciences, based on a consistent rating of above five or below three on a seven-point scale by 20 participants who did not participate in the EEG test. The experimental stimuli in the EEG test comprised of five attractive and five unattractive blurred female faces and the 170 attractive and 170 unattractive unblurred female faces used in the pre-test. Given that there were variable ratings as to whether particular male faces were attractive, we did not include male faces in this study. The attractiveness of unblurred female faces was confirmed in which the EEG participants were asked to rate these faces in the same way as the pre-test. Both the attractive and unattractive faces were unfamiliar to the EEG participants to exclude the influence of familiarity.

We adjusted all photos to be approximately 218 pixels in width (SD = 10.3 pixels) and 274 pixels in height (SD = 4.6 pixels), centered on a 260 pixels× 280 pixels black background. The brightness of all photos was adjusted to about 97.8 (SD = 6.0) as indicated in the histogram after undergoing the “auto laves” function on Adobe Photoshop CS2. Five attractive and five unattractive faces were blurred by Gaussian filters with σ = 14 pixels, maintaining all frequencies below four cycles per photo width.

In each trial in the EEG test, participants were instructed to guess the attractiveness of a blurred face, and to simply watch the subsequent unblurred face, which served as implicit feedback to the consistency of the initial judgment. The blurred faces were presented randomly while the 170 attractive and the 170 unattractive feedback faces were presented in pseudo-random orders. For each participant, each of the blurred faces was presented 34 times and each of the attractive or unattractive faces was presented only once, with the restriction that no more than three consecutive feedback faces were from the same category. Different pseudo-random orders were created for different participants. Unknown to the participants, the blurred face in each trial was not the same one as the feedback face. The purpose of this manipulation was to exclude the potential influence of the blurred faces on the perceptual processing of the subsequent feedback faces as well as to make sure that about half of the trials would constitute “consistent” trials.

Procedures

Participants were seated in a sound-attenuated, electrically shielded chamber approximately 1 m from a computer screen. At the start of each trial, at the center of the computer screen, a white fixation cross (0.6° × 0.6° in visual angle) was presented against a black background for 500 ms. Then a blurred black-and-white face photo was presented (6.3° × 4.6°), and remained on the screen until the participant's response (Figure 1). The participants' task was to make a binary attractiveness judgment as quickly as possible, by pressing a key on a joystick using their left or right index finger. Button assignment was counter-balanced between participants. After the response, a fixation cross was presented again for 800 ms. Then a unblurred face photo (6.3° × 4.6°), serving as feedback to the participants' initial judgment, was presented for 800 ms, and participants were asked to simply watch it and wait for the next trial. After the unblurred face, a fixation cross was presented for 700 ms and the screen turned black for 100 ms before the next trial began.

Figure 1.

Figure 1

Open in a new tab

Sequence of events in a single trial. For illustration purpose, the attractive and the unattractive faces in the figure are the ones morphed from several faces used in the experiment.

Before the EEG test, participants completed a practice block consisting of 10 trials to make sure they understood the task. The EEG test consisted of four blocks with 85 trials each. After the EEG test, participants completed a post-test, rating the attractiveness of each feedback face that had appeared in the EEG test. For faces which a participant's rating in the post-test was inconsistent with the predefined attractiveness, the corresponding trial with that face in the formal test was excluded from EEG data analysis.

EEG recording

The EEG was recorded continuously from 62 scalp electrodes mounted on an elastic cap (NeuroScan Inc., Herndon, Virginia, USA) according to the extended 10–20 system with the addition of two mastoid electrodes. Signals were referenced online to the left mastoid and were re-referenced offline to the linked mastoids. Eye blinks and vertical eye movements were monitored with electrodes located above and below the left eye. The horizontal electro-oculogram was recorded from electrodes placed 1.5 cm lateral to the left and right external canthi. The electrode impedance was less than 5 kΩ. The EEG was amplified (bandpass 0.05–100 Hz) and digitized at 500 Hz.

Data analysis

ERPs were computed for each participant over an epoch from 200 ms before to 800 ms after the onset of the feedback faces, with the 200 ms pre-stimulus EEG activity used for baseline correction. Ocular artifacts were corrected with an eye-movement correction algorithm which employs a regression analysis in combination with artifact averaging (Semlitsch et al., 1986). Epochs contaminated by blinks and other movement artifacts were excluded from averaging using an 80 μ V criterion. The EEG data were low-pass filtered at 30 Hz and were baseline-corrected by subtracting the average activity of that electrode during the baseline period from each trial. After excluding trials with artifacts, each participant had at least 46 trials in each condition.

The grand-average ERP waveforms (Figure 2) did not show a typical pattern for ERP responses that were observed for feedback stimuli in gambling or simple cognitive tasks (e.g., no clear P300 component was visible), although it appeared that inconsistent faces elicited negative-going deflections in the 300–380 ms time window. We, therefore, analyzed ERP responses in different conditions in the windows of 150–220 ms (i.e., P200), 300–380 ms (i.e., FRN), and 380–500 ms based on visual inspection. For the purposes of statistical analysis, mean amplitudes for each time window was calculated across 25 electrode locations (F3, F1, Fz, F2, F4, FC3, FC1, FCz, FC2, FC4, C3, C1, Cz, C2, C4, CP3, CP1, CPz, CP2, CP4, P3, P1, Pz, P2, P4) that were chosen to cover scalp areas known from previous studies to be the focus of the FRN and P300. A repeated-measures analysis of variance (ANOVA) was conducted, with attractiveness (attractive vs. unattractive), feedback consistency (consistent vs. inconsistent), anterior-posterior scalp location (frontal, frontocentral, central, centroparietal, parietal), and lateral scalp location (left, left central, midline, right central, right) as four within-subjects experimental factors. The Greenhouse-Geisser correction for violation of the ANOVA assumption of sphericity was applied in all analysis. Bonferroni corrections were used for multiple comparisons.

Figure 2.

Figure 2

Open in a new tab

ERP responses to feedback faces at exemplar electrodes as a function of attractiveness and feedback consistency, with 0 ms corresponding to the onset of feedback faces.

Given that the processing of the feedback faces and their attractiveness was likely to involve sophisticated neurocognitive processes, it is possible that the FRN and the P300 components were not only overlapping in the time course, but also masked by other cognitive (and emotional) processes associated with the complex feedback stimuli. To examine whether the typical P300 effects that were observed in previous studies for various aspects of the outcome evaluation could also be observed for the more complex feedback faces, we performed principal-component analysis (PCA) on the cleaned ERP data (i.e., after preprocessing) in order to disentangle the overlapping and/or masked ERP components. PCA has a wide range of applications in ERP analysis, such as cleaning or filtering noises prior to data analysis, or being used in data exploration as a way to detect and summarize features of the dataset. In this study, we applied PCA on the cleaned ERP data to maximize the possibility that the PCA factors represent interpretable signals (i.e., brain activity due to experimental manipulations) as opposed to noise (i.e., artifacts or background EEG).

PCA is a common approach for decomposing an ERP dataset into its constituent factors by summarizing the relationship between variables (such as microvolt recordings at each time point in temporal PCA or at each electrode in spatial PCA; Dien and Frishkoff, 2005; Dien et al., 2005). This process consists of three main steps: (1) computation of the covariance matrix which captures the interrelationships between temporal/spatial variables; (2) extraction and retention of the PCA factors which extract linear combinations of variables (latent factors) to account for patterns of covariance in the ERP data with the fewest PCA factors; and (3) rotation to simple structure, which is used to restructure the allocation of variables to PCA factors to maximize the chance that each PCA factor reflects a single ERP component. These steps yield two matrices, which are useful in further analysis. The first one is a factor loading matrix, representing correlations between the variables and the factor scores (e.g., describing the time course of each of the PCA factors in temporal PCA). The second one is a factor score matrix that indexes the magnitude of the PCA factors for each of the observations, reflecting the contribution of each PCA factor to ERPs.

In this study, we used the spatiotemporal PCA algorithm implemented in the Matlab ERP PCA toolbox 2.23 (Dien and Frishkoff, 2005; http://sourceforge.net/projects/erppcatoolkit/files). In the spatiotemporal PCA, a spatial PCA was performed first on the ERP data from all the electrodes to capture spatial distribution, with the combination of 501 time points (covering the epoch from –200 to 800 ms post-onset of the feedback stimuli), 16 participants and four experimental conditions. Infomax rotation was used, and two spatial factors were extracted based on the resulting Scree plot. For each spatial factor, this analysis yielded factor scores for each combination, representing the amount of activity in the original data captured by that factor. A temporal PCA was then performed for each of the two spatial factors. Promax rotation was used, and four temporal factors were extracted based on the Scree plot, yielding eight unique factors combinations (i.e., two spatial factors by four temporal factors). The covariance relationship matrix and Kaiser normalization were used for each PCA. The waveforms for each factor combination were reconstructed (i.e., converted to microvolts) by multiplying the factor pattern matrix with the standard deviations. A PCA factor may contain one or more known ERP components, or may contain no well-defined or theoretically interesting ERP components. Following the suggestion of Dien et al. (2005) and based on visual inspection of the waveforms associated with each PCA factor, two factors that contained known ERP components involved in outcome evaluation and facial attractiveness (i.e., P300) were selected for further statistical analysis, with attractiveness (attractive vs. unattractive) and feedback consistency (consistent vs. inconsistent) as two within-subjects experimental factors. Here we focused on electrodes that showed the largest effect for each selected PCA factor. The Greenhouse–Geisser correction was applied for violation of the ANOVA assumption of sphericity.

Results

Behavioral results

In the post-test on the attractiveness of the feedback faces, 377 trials (6.9%) in total were inconsistent with the categorization of attractiveness based on the pretest (i.e., their rating scores were below five for attractive faces or above three for unattractive faces). After excluding the EEG trials corresponding to these faces, the remaining 5063 trials were categorized into four conditions: attractive-consistent (72 trials per participant on average, ranging from 51 to 117 trials over participants), attractive-inconsistent (78 trials on average, ranging from 48 to 100 trials), unattractive-consistent (91 trials on average, ranging from 61 to 117 trials), and unattractive-inconsistent (76 trials on average, ranging from 48 to 107 trials). Overall, the percentage of trials (out of all the available trials) for each condition was 22.6%, 24.5%, 28.9%, and 24.0%, respectively. No statistically significant differences were found between the conditions.

ERP results

ERP waveforms time-locked to the onset of feedback faces and topographic distributions of the differences between conditions (e.g., ERP effects) in three time windows are illustrated in Figures 2 and 3, respectively.

Figure 3.

Figure 3

Open in a new tab

Topographies of ERP effects in three time windows.

For the 150–220 ms time window (Figures 2, 3), repeated-measures ANOVA showed a significant main effect of attractiveness, F(1,15) = 18.39, p = 0.01, with more positive P200 responses to attractive faces (5.95 μ V) than to unattractive ones (4.98 μ V). The interaction between attractiveness and lateral scalp location was significant, F(4,60) = 5.17, p = 0.001, indicating that the size of the P200 effect varied over different scalp locations. No other significant effects were found.

For the 300–380 ms window, ANOVA showed a significant main effect of feedback consistency, F(1,15) = 15.99, p = 0.001, with ERP responses to feedback faces more negative-going following inconsistent judgments (9.97 μ V) than following consistent judgments (11.36 μ V). The main effect of attractiveness was also significant, F(1,15) = 20.48, p < 0.001, with more negative-going responses to unattractive faces (9.56 μ V) than to attractive ones (11.78 μ V). The interaction between attractiveness and feedback consistency was not significant, F(1,15) < 1, indicating that the attractiveness effect was not affected by feedback consistency. The interaction between attractiveness and anterior-posterior scalp location was significant, F(4,60) = 17.66, p < 0.001, as was the interaction between attractiveness and lateral scalp location, F(4,60) = 8.21, p = 0.01. It is clear from Figure 3 that the attractiveness effect in this time window was strongest in the frontocentral regions. No other significant effects were found.

To further verify the feedback consistency effect, we carried out statistical analyses based on peak-to-peak measurements of the feedback consistency responses (Sato et al., 2005; Wu and Zhou, 2009), with the most negative amplitudes in the 300–380 ms window and the most positive amplitudes in the 220–300 ms window as peaks. Consistent with the above analysis, the main effect of feedback consistency was significant, F(1,15) = 15.38, p = 0.001, and this effect did not interact with attractiveness, F(1,15) < 1. The main effect of attractiveness was significant, F(1,15) = 20.51, p < 0.001.

For the 380–500 ms window, ANOVA showed a significant main effect of attractiveness, F(1,15) = 13.31, p < 0.01, with more positive responses to attractive faces (13.62 μ V) than to unattractive ones (12.52 μ V), and a main effect of feedback consistency, F(1,15) = 13.15, p < 0.01, with more negative responses to feedback faces following inconsistent judgments (12.27 μ V) than following consistent judgments (13.87 μ V). However, the interaction between attractiveness and feedback consistency was not significant, F(1,15) < 1, suggesting that the ERP responses in this time window may encode the valence and attractiveness of feedback faces independently.

PCA results

Of the eight factor combinations yielded by the spatiotemporal PCA, two were selected for further statistical analysis based on visual inspection of the shape and the time course of peak responses in the generated waveforms (Figure 4). Either of the selected factors might correspond to the P300 component of the ERP. The first factor, maximal at CPz in terms of amplitude, reached its peak in the 200–300 ms window, accounting for 13.7% of the data variance. The second factor, maximal at FCz in terms of amplitude, contained a positive deflection beginning at about 250 ms and lasting to 800 ms, accounting for 27.8% of the data variance.

Figure 4.

Figure 4

Open in a new tab

ERP responses on the exemplar electrodes converted from factor scores in the spatiotemporal PCA and depicted as a function of attractiveness and feedback consistency. Topographic maps on the right side present the differential responses corresponding to the factors depicted on the left. Sketches in the upper and lower panels correspond to the first and the second selected spatiotemporal factors, respectively.

For the first selected factor (the upper panel of Figure 4), repeated-measures ANOVA based on the mean amplitudes in the 200–300 ms window at CPz revealed a significant main effect of attractiveness, F(1,15) = 29.03, p < 0.001, with more positive responses to attractive faces (6.30 μ V) than to unattractive ones (4.91 μ V). The main effect of feedback consistency was significant, F(1,15) = 6.31, p < 0.05, with faces consistent with the initial judgment eliciting more positive responses (5.87 μ V) than faces inconsistent with the judgment (5.34 μ V). The interaction between attractiveness and feedback consistency was not significant, F(1,15) < 1. In addition, we conducted ANOVA based on the peak amplitude in the 200–300 ms window. Consistent with the above analysis, the main effect of attractiveness was significant, F(1,15) = 27.23, p < 0.001, and this effect did not interact with feedback consistency, F(1,15) < 1. The main effect of feedback consistency was significant, F(1,15) = 6.03, p < 0.05.

For the second selected factor (the lower panel of Figure 4), repeated-measures ANOVA based on the mean amplitudes in the 250–800 ms window at FCz revealed a significant main effect of attractiveness, F(1,15) = 43.00, p < 0.001, with more positive responses to attractive faces (5.37 μ V) than to unattractive ones (3.32 μ V). The main effect of feedback consistency was significant, F(1,15) = 10.38, p < 0.01, with faces consistent with the initial judgment eliciting more positive responses (4.76 μ V) than faces inconsistent with the judgment (3.93 μ V). The interaction between attractiveness and feedback consistency was not significant, F(1,15) < 1.

Discussion

The main purpose of this study was to investigate to what extent brain responses to complex feedback stimuli with a social dimension (e.g., faces) in outcome evaluation and performance monitoring are similar to those revealed for simple stimuli used in monetary gambling or simple cognitive tasks. We asked participants to guess the attractiveness of blurred faces and then showed them the unblurred faces. The implicit comparison between the initial judgment stored in memory and the newly acquired information would indicate whether the feedback stimuli had properties consistent or inconsistent with the initial judgment.

An FRN effect was observed for inconsistent feedback faces in ERP waveforms, in accordance with many earlier studies (Gehring and Willoughby, 2002; Holroyd and Coles, 2002; Yeung et al., 2005; Leng and Zhou, 2010; Zhou et al., 2010; Long et al., 2012), albeit in a delayed 300–380 ms time window. Faces inconsistent with the initial judgment elicited more negative-going responses than faces consistent with the judgment. The delay of the FRN effect was likely due to the complexity of neurocognitive processes involved in facial attractiveness and feedback consistency processing. To know whether the initial judgment or guess was a correct one, the system has to first process the perceptual structure of the feedback face, and evaluate the attractiveness of this face by appealing to experiences or schemas stored in long-term memory, and secondly to compare the result of this evaluation with information concerning the initial judgment stored in working memory. It takes time to accomplish these processes, delaying the appearance of the FRN effect, even though assessing facial attractiveness can be fairly automatic (Chatterjee et al., 2009). A previous study has also shown that the FRN effect was delayed when the sophisticated counter-factual thinking was involved in accessing the outcome valence of gambling choices (Yu and Zhou, 2009).

However, the ERP waveforms did not show the typical pattern of P300 responses; instead, sustained deflections were observed, with significant main effects for attractiveness and feedback consistency. We suspected that the usual P300 responses were masked by the sophisticated cognitive (and emotional) processes involved in processing the feedback faces, and the observed deflections (and possibly the preceding ERP responses) contained P300 components. We, therefore, conducted spatiotemporal PCA, which revealed a clear pattern of P300 effects for both feedback consistency and attractiveness, with more positive responses to those faces consistent with the initial judgment than to faces inconsistent with the judgment, and more positive responses to attractive faces than to unattractive ones. Moreover, the P300 effects centered on centroparietal areas lasted for a short time period while the P300 effects centered on frontocentral area sustained for a long time period, indicating that facial attractiveness and feedback consistency could be processed parallel in different brain areas.

The P300 effect for feedback consistency or valence replicated previous studies using gambling tasks (Hajcak et al., 2005, 2007; Wu and Zhou, 2009; Leng and Zhou, 2010). The P300 effect for facial attractiveness also replicated many previous studies (Johnston and Oliver-Rodriguez, 1997; Oliver-Rodríguez et al., 1999; Werheid et al., 2007; Schacht et al., 2008). As attractive faces activate brain areas dedicated to reward processing such as the orbitofrontal cortex, the nucleus accumbens or the ventral striatum (Aharon et al., 2001; Kampe et al., 2001; O'Doherty et al., 2003; Bray and O'Doherty, 2007; Ishai, 2007; Winston et al., 2007; Cloutier et al., 2008), the attractive feedback faces in this study might be considered to be a form of reward, having stronger motivational significance, and capturing more attentional resources than unattractive faces (Sabatinelli et al., 2004). A large number of studies have already demonstrated that the P300 is sensitive to processes that demand attentional resources (Polich and Kok, 1995; Polich, 2007). The independence between the P300 effect for feedback consistency and the effect for facial attractiveness, while replicating previous studies that simultaneously manipulated different dimensions of feedback stimuli (Hajcak et al., 2005, 2007; Wu and Zhou, 2009; Long et al., 2012), indicate that the evaluative processes for different dimensions of the outcome can be conducted in parallel.

Note that we also found a significant main effect of attractiveness in the 300–380 ms window. However, we do not interpret it as a kind of FRN effect because we believe this effect was very likely due to the spillover of the P300 effects for facial attractiveness.

To conclude, this study provides the first demonstration of electrophysiological responses in outcome evaluation with complex feedback stimuli (e.g., faces) that need complicated neurocognitive processing. We demonstrated that the ERP correlates of processing complex feedback stimuli with a social dimension are generally similar to those involved in processing simplex feedback stimuli; however, appropriate means of ERP data analysis, including PCA, may be needed to uncover the underlying patterns of effects that might be masked by the sophisticated processes associated with the complex stimuli themselves. Consistent with previous studies, the present study demonstrates again that the FRN reflects early assessment of outcome valence while the P300 can encode different attributes of feedback simultaneously in outcome evaluation and performance monitoring.

Conflict of interest statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

This study was supported by National Basic Research Program of China (973 Program: 2010CB833904) and by a grant from the Natural Science Foundation of China (30110972). We thank Mr. Zhiheng Zhou and Ms. Chaohui Guo for their help during the preparation for the experiment, Mr. Stijn Massar, Mr. Stephen Politzer-Ahles, and the two reviewers for their comments on the earlier versions of the manuscript, and Ms. Sarah Dinces for English correction.

References

  1. Aharon I., Etcoff N., Ariely D., Chabris C. F., O'Connor E., Breiter H. C. (2001). Beautiful faces have variable reward value: fMRI and behavioral evidence. Neuron 32, 537–551 10.1016/S0896-6273(01)00491-3 [DOI] [PubMed] [Google Scholar]
  2. Bray S., O'Doherty J. (2007). Neural coding of reward-prediction error signals during classical conditioning with attractive faces. J. Physiol. 98, 3036–3045 10.1152/jn.01211.2006 [DOI] [PubMed] [Google Scholar]
  3. Chatterjee A., Thomas A., Smith S. E., Aguirre G. K. (2009). The neural response to facial attractiveness. Neuropsychology 23, 135–143 10.1037/a0014430 [DOI] [PubMed] [Google Scholar]
  4. Cloutier J., Heatherton T. F., Whalen P. J., Kelley W. M. (2008). Are attractive people rewarding? Sex differences in the neural substrates of facial attractiveness. J. Cogn. Neurosci. 20, 941–951 10.1162/jocn.2008.20062 [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Dien J., Beal D. J., Berg P. (2005). Optimizing principal components analysis of event-related potentials: matrix type, factor loading weighting, extraction, and rotations. Clin. Neurophysiol. 116, 1808–1825 10.1016/j.clinph.2004.11.025 [DOI] [PubMed] [Google Scholar]
  6. Dien J., Frishkoff G. A. (2005). “Introduction to principal components analysis of event-related potentials,” in Event-related Potentials: A Methods Handbook, ed Handy T. (Cambridge, MA: MIT Press; ), 189–208 [Google Scholar]
  7. Dion K., Berscheid E., Walster E. (1972). What is beautiful is good. J. Pers. Soc. Psychol. 24, 285–290 [DOI] [PubMed] [Google Scholar]
  8. Etcoff N. (1999). Survival of the Prettiest: The Science of Beauty. New York, NY: Doubleday [Google Scholar]
  9. Gehring W. J., Willoughby A. R. (2002). The medial frontal cortex and the rapid processing of monetary gains and losses. Science 295, 2279–2282 10.1126/science.1066893 [DOI] [PubMed] [Google Scholar]
  10. Hajcak G., Holroyd C. B., Moser J. S., Simons R. F. (2005). Brain potentials associated with expected and unexpected good and bad outcomes. Psychophysiology 42, 161–170 10.1111/j.1469-8986.2005.00278.x [DOI] [PubMed] [Google Scholar]
  11. Hajcak G., Moser J. S., Holroyd C. B., Simons R. F. (2007). It's worse than you thought: the feedback negativity and violations of reward prediction in gambling tasks. Psychophysiology 44, 905–912 10.1111/j.1469-8986.2007.00567.x [DOI] [PubMed] [Google Scholar]
  12. Heldmann M., Russeler J., Munte T. (2008). Internal and external information in error processing. BMC Neurosci. 9, 33–40 10.1186/1471-2202-9-33 [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Holroyd C. B., Coles M. G. H. (2002). The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. Psychol. Rev. 109, 679–709 [DOI] [PubMed] [Google Scholar]
  14. Ishai A. (2007). Sex, beauty and the orbitofrontal cortex. Int. J. Psychophysiol. 63, 181–185 10.1016/j.ijpsycho.2006.03.010 [DOI] [PubMed] [Google Scholar]
  15. Johnston V. S. (2006). Mate choice decisions: the role of facial beauty. Trends Cogn. Sci. 10, 9–13 10.1016/j.tics.2005.11.003 [DOI] [PubMed] [Google Scholar]
  16. Johnston V. S., Oliver-Rodriguez J. C. (1997). Facial beauty and the late positive component of event-related potentials. J. Sex Res. 34, 188–198 [Google Scholar]
  17. Kampe K. K. W., Frith C. C., Dolan R. J., Frith U. (2001). Reward value of attractiveness and gaze. Nature 413, 589–590 10.1038/35098149 [DOI] [PubMed] [Google Scholar]
  18. Leng Y., Zhou X. (2010). Modulation of the brain activity in outcome evaluation by interpersonal relationship: an ERP study. Neuropsychologia 48, 448–455 10.1016/j.neuropsychologia.2009.10.002 [DOI] [PubMed] [Google Scholar]
  19. Long Y., Jiang X., Zhou X. (2012). To believe or not to believe: trust choice modulates brain responses in outcome evaluation. Neuroscience 200, 50–58 10.1016/j.neuroscience.2011.10.035 [DOI] [PubMed] [Google Scholar]
  20. Miltner W. H. R., Braun C. H., Coles M. G. H. (1997). Event-related brain potentials following incorrect feedback in a time-estimation task: evidence for a “generic” neural system for error detection. J. Cogn. Neurosci. 9, 787–796 [DOI] [PubMed] [Google Scholar]
  21. Nieuwenhuis S., Holroyd C. B., Mol N., Coles M. G. H. (2004). Reinforcement-related brain potentials from medial frontal cortex: origins and functional significance. Neurosci. Biobehav. Rev. 28, 441–448 10.1016/j.neubiorev.2004.05.003 [DOI] [PubMed] [Google Scholar]
  22. O'Doherty J., Winston J., Critchley H., Perrett D., Burt D. M., Dolan R. J. (2003). Beauty in a smile: the role of medial orbitofrontal cortex in facial attractiveness. Neuropsychologia 41, 147–155 10.1016/S0028-3932(02)00145-8 [DOI] [PubMed] [Google Scholar]
  23. Oliver-Rodríguez J. C., Guan Z., Johnston V. S. (1999). Gender differences in late positive components evoked by human faces. Psychophysiology 36, 176–185 [PubMed] [Google Scholar]
  24. Polich J. (2007). Updating P300: an integrative theory of P3a and P3b. Clin. Neurophysiol. 118, 2128–2148 10.1016/j.clinph.2007.04.019 [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Polich J., Kok A. (1995). Cognitive and biological determinants of P300: an integrative review. Biol. Psychol. 41, 103–146 10.1016/0301-0511(95)05130-9 [DOI] [PubMed] [Google Scholar]
  26. Sabatinelli D., Flaisch T. C., Bradley M. M., Fitzsimmons J. R., Lang P. J. (2004). Affective picture perception: gender differences in visual cortex? Neuroreport 15, 1109–1112 [DOI] [PubMed] [Google Scholar]
  27. Sato A., Yausda A., Ohira H., Miyawaki K., Nishikawa M., Kumano H., Kuboki T. (2005). Effects of value and reward magnitude on feedback negativity and P300. Neuroreport 16, 407–411 [DOI] [PubMed] [Google Scholar]
  28. Schacht A., Werheid K., Sommer W. (2008). The appraisal of facial beauty is rapid but not mandatory. Cogn. Affect. Behav. Neurosci. 8, 132–142 [DOI] [PubMed] [Google Scholar]
  29. Semlitsch H. V., Anderer P., Schuster P., Presslich O. (1986). A solution for reliable and valid reduction of ocular artifacts, applied to the P300 ERP. Psychophysiology 23, 695–703 [DOI] [PubMed] [Google Scholar]
  30. Thornhill R., Gangestad S. W. (1999). Facial attractiveness. Trends Cogn. Sci. 3, 452–460 10.1016/S1364-6613(99)01403-5 [DOI] [PubMed] [Google Scholar]
  31. Werheid K., Schacht A., Sommer W. (2007). Facial attractiveness modulates early and late event-related brain potentials. Biol. Psychol. 76, 100–108 10.1016/j.biopsycho.2007.06.008 [DOI] [PubMed] [Google Scholar]
  32. Winston J. S., O'Doherty J., Kilner J. M., Perrett D. I., Dolan R. J. (2007). Brain systems for assessing facial attractiveness. Neuropsychologia 45, 195–206 10.1016/j.neuropsychologia.2006.05.009 [DOI] [PubMed] [Google Scholar]
  33. Wu Y., Zhou X. (2009). The P300 and reward valence, magnitude, and expectancy in outcome evaluation. Brain Res. 1286, 114–122 10.1016/j.brainres.2009.06.032 [DOI] [PubMed] [Google Scholar]
  34. Yeung N., Holroyd C. B., Cohen J. D. (2005). ERP correlates of feedback and reward processing in the presence and absence of response choice. Cereb. Cortex 15, 535–544 10.1093/cercor/bhh153 [DOI] [PubMed] [Google Scholar]
  35. Yeung N., Sanfey A. G. (2004). Independent coding of reward magnitude and valence in the human brain. J. Neurosci. 24, 6258–6264 10.1523/JNEUROSCI.4537-03.2004 [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Yu R., Zhou X. (2009). To bet or not to bet? The error negativity or error-related negativity associated with risk-taking choices. J. Cogn. Neurosci. 21, 684–696 10.1162/jocn.2009.21034 [DOI] [PubMed] [Google Scholar]
  37. Zhou Z., Yu R., Zhou X. (2010). To do or not to do? Action enlarges the FRN and P300 effects in outcome evaluation. Neuropsychologia 48, 3606–3613 10.1016/j.neuropsychologia.2010.08.010 [DOI] [PubMed] [Google Scholar]

Articles from Frontiers in Human Neuroscience are provided here courtesy of Frontiers Media SA

RESOURCES