Neural Response to Action and Reward Prediction Errors: Comparing the Error Related Negativity to Behavioral Errors and the Feedback Related Negativity to Reward Prediction Violations

Abstract

The error-related negativity (ERN) is thought to index an anterior cingulate (ACC) behavioral monitoring system. The feedback ERN (FRN) is elicited to error feedback when the correct response is not known, but also when a choice outcome is suboptimal and to passive reward prediction violation, suggesting that the monitoring system may not be restricted to actions. This study used principal components analysis to show that the ERN consists of a single central component while the reward prediction violation FRN is comprised of central and prefrontal components. A prefrontal component is also present in action monitoring but occurs later, at the Error Positivity latency. This suggests that ACC monitors both actions and events for reward prediction error. Prefrontal cortex may update reward expectation based on the prediction violation with the latency difference due to differential processing time for motor and perceptual information.

Monitoring actions and their outcomes is a critical cognitive function for the effective generation of goal-directed behavior. In the absence of information about the impact of actions and choices on motivational goals an organism cannot modify its behavior for optimal performance. Given the importance of action and outcome monitoring, it seems reasonable that the nervous system would have a neural system dedicated to this function. In the early 1990s, Falkenstein, Hohnsbein, Hoormann, & Blanke (1991) and Gehring, Goss, Coles, Meyer, & Donchin (1993) independently reported an event-related potential (ERP) index of a neural behavioral monitoring system, labeled the Error Negativity (Ne) or Error-Related Negativity (ERN).

The ERN is a medial frontocentral ERP component with an onset at or just before the execution of an overt response, peaking about 100 ms post-response, elicited when the response is erroneous (Falkenstein, Hohnsbein, Hoormann, & Blanke, 1991; Gehring, Goss, Coles, Meyer, & Donchin, 1993). Initial hypotheses about the specific cognitive operation indexed by the ERN focused on the detection of a response error or some post-detection corrective process (Falkenstein, Hohnsbein, Hoormann, & Blanke, 1991; Gehring, Goss, Coles, Meyer, & Donchin, 1993). Source analysis placed the ERN neural source in anterior cingulate cortex (ACC(Dehaene, Posner, & Tucker, 1994). Converging evidence from animal single unit recording (Gemba, Sasaki, & Brooks, 1986) and human hemodynamic neuroimaging (Carter, et al., 1998) are consistent with ACC as a core part of a behavior monitoring neural network. The ACC is densely interconnected with premotor cortex and motivational areas of the brain including the amygdala, ventral tegmentum, and ventromedial prefrontal and orbitofrontal cortex (Denvinsky, Morrell, & Vogt, 1995; Paus, 2001), and thus is well situated to evaluate motor plans in the context of motivational goals.

Holroyd and Coles (Holroyd & Coles, 2002) presented a model of the behavior monitoring system and the ERN based on the reinforcement learning model of the ventral tegmental area (VTA; (Schultz, Dayan, & Montague, 1997), explicitly linking behavior monitoring to the brain’s appetitive motivation system, the mesotelencephalic dopamine (DA) reward system, originating in the VTA with wide-ranging targets in the striatum, ventromedial prefrontal cortex, orbitofrontal cortex, and the ACC. Based on animal self-stimulation studies and assessment of neuron activity and neurotransmitter binding, it was originally theorized that this neural system signaled the presence of items with appetitive motivational value (see reviews in (Spanagel & Weiss, 1999; Wise & Rompre, 1989). Schultz, Dayan, and Montague (Schultz, Dayan, & Montague, 1997) extended this model, reporting that the response of VTA neurons to reward could be conditioned. Using classical conditioning procedures they showed that if an appetitive stimulus was presented in the absence of any predictive cue, VTA neurons showed enhanced firing, the classic reward response. However, if the appetitive stimulus was repeatedly paired with a previously neutral cue, after conditioning the VTA neurons showed enhanced firing to the predictive cue but no longer to the appetitive stimulus itself. Also following conditioning, if the predictive cue was presented but the appetitive stimulus then withheld, VTA neuron response was suppressed below baseline at the time the predicted reward would have been delivered (Schultz, Dayan, & Montague, 1997). Thus VTA neurons do not code reward itself, rather they code whether a delivered reward violates reward prediction, i.e. if there is a reward prediction error, with enhanced firing for outcomes better than predicted and suppressed firing for outcomes worse than predicted. Holroyd & Coles (Holroyd & Coles, 2002) proposed that the ERN reflects input from the DA reward prediction system to the ACC. When an action fails to achieve the expected motivational outcome, the DA reward system produces a reward prediction error signal that is transmitted to ACC where it functions as a ‘learning signal’, biasing the motor planning and production system to acquire new associations. The reduced DA input releases the ACC from inhibition resulting in increased activity and a scalp recorded ERN (Holroyd & Coles, 2002).

There has been a sizable body of work examining the response of the monitoring system to explicit reward expectation violation using an ERP component related to the ERN, the feedback error related negativity (FRN; (Miltner, Braun, & Coles, 1997); reviewed in (Nieuwenhuis, Holroyd, Mol, & Coles, 2004). As originally described, an FRN is elicited when the participant does not know the correct response when making the response, but is subsequently provided with performance feedback. When the participant does not know the correct response then there is no ERN elicited by the action (e.g. the incorrect keypress), rather there is an FRN elicited to the perceptual feedback informing the participant that their response was incorrect. The FRN appears to have the same neural source as the motor-related ERN, suggesting that they are the same component, elicited when the participant can determine the accuracy of the response (Miltner, Braun, & Coles, 1997). As the participant learns the task, the error effect transfers from the feedback-locked FRN to the response-locked FRN, further supporting the unity of cognitive operation indexed by the ERN and FRN (Holroyd & Coles, 2002; Nieuwenhuis, et al., 2002).

Following the initial descriptions of the ERN and FRN as components elicited by action errors, numerous studies have expanded upon the eliciting conditions required for an FRN. The original designs required feedback informing the participant that their response was an explicit error, i.e. a response that violated task instructions, to elicit an FRN. Some subsequent designs have shown that feedback signaling suboptimal trial outcome is sufficient to elicit an FRN. Most of these designs have used some form of monetarily motivated choice option task in which monetary gain or loss on a given trial is the outcome of the participant’s choice (e.g. (Gehring & Willoughby, 2002; Holroyd, et al., 2004; Yeung & Sanfey, 2004). In these gambling-like tasks, an FRN is elicited when the choice outcome is less than the best available given the options. For example, if breaking even is the best available option (i.e. all the other outcomes are monetary losses) then breaking even would not elicit an FRN, however it the other options are all monetary gains, then breaking even would elicit an FRN (Holroyd, Larsen, & Cohen, 2004). To account for these results, the conception of the FRN has been expanded from indexing simple behavioral errors to indexing actions or choice outcomes that fail to meet motivational goals (Nieuwenhuis, Holroyd, Mol, & Coles, 2004).

However, in contrast to the choice outcome evaluation hypothesis, several studies have now reported an FRN in the absence of any overt choice or motor behavior (Donkers, Nieuwenhuis, & van Boxtel, 2005; Martin & Potts, 2004; Potts, Martin, Burton, & Montague, 2006; Yeung, Holroyd, & Cohen, 2005). Some of these experiments have employed designs similar to the choice gambling tasks above, modified so that another player or the computer is making the choice, rather than the participant (Yeung, Holroyd, & Cohen, 2005). Other experiments have used ‘slot machine’ –like designs in which there is no action or choice involved; participants simply observe the stimuli signaling the trial’s monetary outcome for the participant (Donkers, Nieuwenhuis, & van Boxtel, 2005; Martin & Potts, 2004; Potts, Martin, Burton, & Montague, 2006). An FRN is elicited in these designs in the absence of overt action or choice on the part of the participant, when the outcome is suboptimal or worse than expected. Elicitation of an FRN in the absence of an explicit response or choice lends support to the theory that the monitoring system receives input from the VTA reward prediction system, a system that does not require an explicit action to elicit a reward prediction error response, but not that the system monitors only actions and choice outcomes. In the single-unit studies of the reward system, an active choice or action is not required to elicit a response from VTA neurons; in those designs a stimulus predicts the reward (the conditioned stimulus), then the reward itself is either delivered or not (Schultz, Dayan, & Montague, 1997). Withholding the predicted reward is sufficient to elicit a VTA neuronal response; no action by the participant is required. Similarly, in the slot-machine type FRN designs, the failure to deliver a predicted reward is sufficient to elicit an FRN (Donkers, Nieuwenhuis, & van Boxtel, 2005; Potts, Martin, Burton, & Montague, 2006), suggesting that the neural system indexed by the FRN has a function extending beyond the monitoring of actions and choice outcomes.

One difficulty in drawing conclusions about the functional relationships between the ERN and the FRN is that the eliciting contexts are different. While the FRN can be examined during purely perceptual tasks, the ERN, as a response-locked component, cannot. The hypothesis that the stimulus- and response-locked ERNs index the same cognitive operation supported in the same neural system derive primarily from studies that elicited the FRN using feedback that signaled an explicit action error, which does not directly address the relationship between the ERN elicited by action error and the FRN elicited by reward prediction violation. While Miltner, Braun, & Coles (Miltner, Braun, & Coles, 1997) reported that the FRN had the same medial frontal scalp distribution as the ERN, Gehring & Willoughby (Gehring & Willoughby, 2004) directly compared the topography of the ERN elicited by behavioral errors in a flanker task with the FRN elicited by monetary losses in a gambling task and found that the FRN to monetary loss had a more anterior scalp distribution than the error ERN and concluded that the ERN and FRN could not both be due to a single common neural generator. However, that study did not attempt to address what those differences in the neural systems engaged by action errors and monetary loss might be.

The current study investigated to what extent the same neural systems are engaged by response errors and by reward prediction violations by examining whether the ERNs elicited by action errors and reward prediction violations are best described as a single distinct ERP component or if they are better described as one or more distinct sub-components, i.e. whether either the ERN or FRN has a sub-component not shared by the other. We applied difference wave analysis to determine if the scalp topographies of the ERN and FRN were distinct when extracted from perceptual- and motor-related ERP activity, and principal components analysis (PCA) to investigate whether the unique sources of variance contributing to those scalp distributions were the same or different between the action error-related ERN and the reward prediction violation-related FRN, on data from twenty-four participants who participated in both a flanker task and a passive reward prediction design. Difference waves extract the effects distinct between the two conditions but cannot distinguish which condition contributes which effects to the difference wave and can confound amplitude and latency differences. PCA reduces the dimensionality of the data by extracting coherent sources of variance in the ERP across time, space, conditions, and participants (Donchin, 1966; Ruchkin, Villegas, & John, 1964) reviewed in (Donchin & Heffley, 1979).

In the current data, an equivalent ERN and FRN difference wave scalp topography and the same PCA derived factor structure would be consistent with the hypothesis that action errors and reward prediction violations both engage the same neural system. If the ERPs elicited by these two violations have distinctly different scalp topographies and are associated with different factor structures, we would conclude that errors and reward prediction violations activate different cognitive operations, implemented in different neural structures. Distinct topographies with partially overlapping factor structure would indicate that the neural systems and cognitive operations engaged by the two tasks share some subcomponent operations but also have additional operations only engaged by one of the tasks.

Methods

Participants

Twenty-four Rice undergraduate students (15 female, one left-handed, mean age 20.38 (SD 3.17)) were paid for participation, and all participated in both experimental designs. Note that results from subsets of the participants in the reward prediction design have been previously reported in Potts, Martin, Burton, & Montague (Potts, Martin, Burton, & Montague, 2006), and, with fMRI results, in Martin, Potts, Burton, & Montague (Martin, Potts, Burton, & Montague, 2009), thus the results here do not constitute a replication of those findings.

Experimental Designs

Flanker Task

The flanker design was modified from Eriksen & Eriksen (Eriksen & Eriksen, 1979). Stimuli were 5-letter strings consisting of the letters P and R. Participants responded to the center letter in the string in a two-choice forced-alternative manner using the left index finger to respond to one letter and the right to the other (response hand to letter mapping was counterbalanced across participants). Stimulus strings were either congruent (all five letters the same) or incongruent (flanking letters did not match the center letter). A trial consisted of a fixation cross followed by an 800 ms warning asterisk followed by the stimulus string that remained onscreen for 100 ms. Participants had 800 ms to respond and 1000 ms after the stimulus string a feedback screen appeared informing the participant of the outcome of the trial. There were nine blocks of trials with 96 trials/block with stimulus type (P or R central stimulus, congruent or incongruent flankers) chosen randomly and equiprobably within each block for a total of 214 trials of each stimulus combination.

The design contained a motivational aspect in that the center R stimuli were potentially rewarding and the center P stimuli potentially punishing. Participants were given a cash reward for correct responses to R’s with no consequence for errors, and had lost money for incorrect responses to P’s with no reward for correct responses, thus R’s were potentially rewarding and P’s potentially punishing. Participants started each block with $5 in their ‘bank’; correct responses to R’s added $1 and incorrect responses subtracted $1 from the bank. At the end of the experiment participants were paid their winnings on one of the blocks, chosen at random. Since all participants made more correct than error responses, all blocks were ‘winning’ blocks, however participants were told at the beginning of the experiment that they would leave with at least the $5 they started with regardless of performance. The punishment/reward motivation factor had no differential effect on the scalp topography or factor structure of the ERPs so it will not be discussed further here.

Reward prediction design

Stimuli were images of lemons and gold bars in an S1/S2 design in which S1 predicted S2 and S2 delivered or did not deliver a reward. A trial began with 300 ms of fixation followed by S1, an image of either a lemon or a gold bar, remaining onscreen for 500 ms, followed by another 300 ms of fixation, then S2 was presented for 500 ms, then another 300 ms of fixation, then a text screen stating the monetary outcome of the trial and the participant’s current ‘bankroll’, which stayed onscreen for 600 ms. S2 determined the reward amount; if S2 was a gold bar the participant won $1, if it was a lemon the participant won nothing. On 80 % of the trials S1 and S2 were identical leading to four trial types in a two by two design: Prediction (Predicted, Unpredicted) by Reward (Reward, No Reward).

There were eight blocks of trials and participants started each block with $5. Each trial cost $0.25 and that cost was subtracted from and winnings were added to that $5. Each block consisted of 60 trials for 480 trials total. While the overall predicted/unpredicted ratio was fixed at 80/20, slight variation was introduced into each block so the payoff for each block was slightly different. At the end of the experiments participants drew a number from 1 – 8 and were paid for the outcome of that block.

Data Acquisition and Signal Processing

Data were acquired with a 128-channel Electrical Geodesics System 200 (EGI, Eugene, OR). EEG was acquired continuously, sampled at 250 Hz, with a vertex reference. The EEG was digitally filtered at 20 Hz lowpass and segmented from 100 ms pre-response to 400 ms post-response in the flanker task and from 200 ms pre-S2 to 800 ms post-S2 in the reward prediction task. The segments were screened for non-cephalic artifact and the clean segments averaged into error and correct response conditions in the flanker task and into the predicted and unpredicted reward and no reward conditions in the reward prediction task. The subject averages were baseline corrected over the 100 ms pre-response (flanker) or 200 ms pre-stimulus (reward prediction) periods and re-referenced into an average reference representation. The subject averages were averaged together to create grand-average files.

Analyses

Overt responses in the flanker task were analyzed by computing the percent correct responses for each participant in the congruent and incongruent conditions and averaging the individual reaction times for each participant separated by congruent and incongruent conditions and by correct and incorrect response. The percent correct was tested using a paired t-test comparing the congruent and incongruent conditions and the mean reaction times were analyzed in a repeated-measures ANOVA with Condition (Congruent, Incongruent) and Response (Correct, Error) as factors.

To determine if the designs elicited an ERN to error responses and an FRN to reward prediction violation, we extracted the mean amplitude from 0 to 150 ms after the response for the flanker task and from 200 – 350 ms after S2 onset in the reward prediction design from the fronto-central sensor net electrode closest to the frontocentral FCz location. The mean amplitude was analyzed using repeated measures ANOVAS, one ANOVA for each experimental design, with Response (Error, Correct) as the factor in the flanker design and Prediction (Predicted, Not Predicted) and Reward (Reward, No Reward) as the factors in the reward prediction design.

To examine the scalp distribution of the ERN elicited by errors and the FRN from reward prediction violations, separated from the experiment design-specific motor and stimulus ERP effects, we created difference waves in the flanker task (error minus correct) and the reward prediction design (unpredicted no reward (bar lemon) minus predicted reward (bar bar). The subtraction in the reward prediction design confounds effects due to reward delivery and prediction violation but avoids potential confound due differential prediction by holding S1 constant. Interpolated maps were created from these difference waves, and possible differences in scalp distribution were tested parametrically by extracting the difference wave ERN and FRN using the same windows described above (0 – 150 ms post-response in the flanker design, 200 – 350 ms post-S2 in the reward prediction design) at the sensor net midline electrodes corresponding to FPz, Fz, FCz, Cz, CPz, and Pz and using a repeated-measures ANOVA with Experiment Design (Flanker, Reward Prediction) and Electrode (FPz, Fz, FCz, Cz, CPz, Pz) as factors. Effects or interactions with the Electrode factor (the only factor with more than two levels) were corrected for deviation from sphericity with the Greenhouse-Geyser epsilon.

PCA

The ERPs from 100 ms before to 450 ms after the motor response or feedback stimulus for each subject at all electrodes for both error and correct responses in the flanker task and for the predicted reward (bar-bar) and unpredicted no reward (bar-lemon) conditions were submitted to a spatiotemporal PCA (Dien & Frishkoff, 2004; Spencer, Dien, & Donchin, 2001) conducted separately for flanker and reward prediction designs (note that the predicted reward and unpredicted no reward conditions provide the ‘outcome worse than predicted’ contrast relevant to the reward prediction violation theory). Details of the PCA procedures are described in(Dien & Frishkoff, 2004; Spencer, Dien, & Donchin, 2001) as implemented in the Matlab toolbox provided by Dien (v1.23(Dien, submitted)). Briefly, the input to the PCA for each task consisted of 326 observations (2 conditions x 163 data points per epoch) for each participant and electrode. Spatial factors were initially extracted to reveal spatially coherent patterns of variance in the ERP, followed by a temporal PCA performed on each spatial factor to examine when in time each spatial factor had coherent temporal structure. Covariance matrices and Promax rotations without the Kaiser correction option were used in the PCA.

Paired sample t-tests were conducted on the factor scores of each resultant temporal factor to test for differences between experimental conditions. Only those spatiotemporal factors whose factor scores were significantly different between conditions at a level of p=.05 and that accounted for more than 3% of the variance in the total solution are discussed here.

Results

Behavioral

Overall accuracy in the flanker task was 72% (so flanker task errors were slightly more frequent than unexpectedly withheld rewards in the reward prediction design). Participants were more accurate in the Congruous (mean accuracy 84.1 %, SD 13.5) than the Incongruous condition (mean 59.7%, SD 11.7), t(23) = 13.32, p < .001. Participants were also faster in the Congruous (mean RT = 356 ms, SD = 65) than the Incongruous condition (mean = 373 ms, SD = 65), F(1, 23) = 20.38, p < .001. There was no impact on RT of response accuracy.

ERP

The average waveforms across all participants (grandaverage) for the Flanker (correct, error) and Reward Prediction (predicted reward (PR), unpredicted reward (UR), predicted no reward (PNR), unpredicted no reward (UNR)) designs at four midline electrodes in the sensor net approximately equivalent to Fz, FCz, Cz, and Pz are shown in Figure 1.

Figure 1.

Grand average ERP waveform plots from the midline electrodes closest to Fz, FCz, Cz, and Pz in the Sensor Net for the Flanker design showing the correct (thin line) and error (thick line) responses with the ERN analysis window delimited and for the Reward Predictin design showing the predicted (P: dashed lines) and unpredicted (U: solid lines) reward (R: thin lines) and no reward (NR: thick lines) conditions with the FRN analysis window delimited.

Flanker Design

The response-locked ERN was more negative on error responses than correct responses, F(1, 23) = 11.99, p < .0001 (see Figure 1).

Reward prediction design

The S2 stimulus-locked FRN was more negative to an S2 that signaled no reward (lemon) than to stimuli that signaled reward (gold bar), F(1, 23) = 26.44, p < .0001. There was a Reward x Prediction interaction, F(1, 23) = 7.20, p < .05, indicating that FRN to the no reward stimuli was larger when that absent reward was unpredicted, i.e. when a predicted reward was not delivered (see Figure 1).

Difference Wave, Midline Electrodes

There was a main effect for Design, F(1, 23) = 13.01, p < .005, with the ERN in the flanker design larger than the reward prediction FRN. There was also a main effect for Electrode, F(5, 115) = 4.26, ε = .375, p < .05, with the E/FRN largest at FCz and falling off to the anterior and posterior. The Design x Electrode interaction was significant, F(5, 115) = 12.15, ε = .309, p < .005, with the flanker ERN largest at FCz but the reward prediction FRN largest at Fz and FPz (see Figure 2), an interaction that was still significant after normalization to correct for the ‘misallocation of variance’ problem (McCarthy & Wood, 1985), F(5, 115) = 11.18, ε = .394, p < .001.

Figure 2.

a) Maps of the scalp field topography of the error-minus-correct difference wave from the Flanker design and the predicted reward-minus-unpredicted no reward difference wave from the Reward Prediction design. The maps are oriented with the nose at the top of the figure, back of the head at the bottom and the vertex at the center. Darker is more negative. b) Difference wave waveform plots for six midline electrodes in the sensor net approximately equivalent to FPz, Fz, FCz, Cz, CPz, and Pz in the extended 10/20 system showing the electrodes where the behavioral error ERN in the Flanker design (solid lines) or reward prediction violation FRN in the Reward Prediction design (dashed lines) are largest. c) Histrograph of the Design x Electrode interaction effect showing that the difference wave reward prediction violation FRN has a more anterior distribution than the behavioral error ERN.

The amplitude of the error ERN and predicted reward not delivered FRN were not significantly correlated, r = .14, p = ns, but the ERN and FRN differences were positively correlated r = .55, p < .01.

PCA

Only the first three spatial factors extracted from each design contained temporal factors that varied significantly by condition and accounted for more than 3% of the variance, all with midline scalp distributions, one frontal, one central, and one posterior (see Figure 3, top row). The topographic distributions of the spatial factors (i.e. the spatial factor loadings on the electrodes) were highly correlated between designs. The spatial loadings of central factor (spatial factor 1 of the Flanker experiment and spatial factor 2 of the Reward Prediction experiment) were highly correlated, r =.88, p<.01, as were the frontal factor (spatial factor 2 in the Flanker design and spatial factor 3 in the reward prediction design), r =.98, p<.01, and the posterior factor (spatial factor 3 of the Flanker experiment and spatial factor 1 of the Reward Prediction experiment), r =.87, p<.01 (see Figure 3, top row).

Figure 3.

Top row: Topographic maps of the first three spatial factors from the Flanker and Reward prediction designs. Second row: “Virtual ERPs”, showing the temporal course of spatial factors. Third and fourth rows: Temporal factor loadings on time by condition for the temporal factors for each spatial factor that differ significantly by condition

The spatial factor scores were plotted across time points for each condition to create the ‘virtual ERPs’ as described in (Spencer, Dien, & Donchin, 2001) to illustrate the time course of each spatial factor in each condition, and are presented in Figure 3, second row. The temporal factors extracted for each spatial factor that differed significantly between conditions are overplotted by condition in Figure 3, third and fourth rows (no temporal factor beyond factor 6 differed between condition and accounted for more than 3% of the variance in the solution).

In the flanker task, for the central factor (spatial factor one), temporal factor one (SF1TF1) had its highest temporal factor loading and condition difference from 0 – 200 ms peaking at 100 ms, then crossing over at 200 ms to a second peak just past 300 ms (see Figure 3, column 1). This factor accounted for 18% of the variance in the solution, and differed significantly between error and correct responses, t(24) = 8.82, p < .01. For the frontal factor (spatial factor two), temporal factor three (SF2TF3) had its primary temporal factor loadings and condition difference rising through the pre-response period with an abrupt return to baseline immediately following the response (3% variance accounted for) and differed between error and correct responses t(23)= 3.52, p < .01 (see Figure 3, column 3). SF2TF4 also differed by response, t(23)=−3.34, p < .01 (3% variance accounted for), with the temporal factor loadings largest between 200 – 400 ms post-response peaking at about 280 ms. For the posterior spatial factor (SF3), TF1 had an initial large temporal factor loading around response onset and a later positive loading from 200 – 400 ms peaking at about 300 ms (7% variance accounted for) and differed between correct and error responses t(23)= 7.40, p < .01 (See Figure 3, column 5).

In the reward prediction task, for the central factor (which was SF2 in this design), SF2TF1 accounted for 5% of the variance and differed between the predicted reward and unpredicted no reward conditions, t(23) = 3.21, p < .01, temporal factor loading occurring primarily after 350 ms (see figure 3, column 2). SF2TF2 also accounted for 5% of the variance and differed between the conditions, t(23) = 4.61, p < .01, with the temporal factor loading occurring between 200 – 300 ms post-stimulus (see Figure 3, column 2). For the frontal factor (SF3), TF2 accounted for the 5% of the variance and differed between conditions, t(23) = 4.80, p < .01, with the factor loading largest from about 225 – 350 ms (see Figure 3, column 4). For the posterior factor (SF1), TF2 accounted for 7% of the variance and differed between conditions, t(23) = 2.59, p < .05, mostly after 350 ms. SF1TF6 also differed between conditions, t(23) = 2.56, p < .05, with the difference occurring primarily between 180 – 280 ms (see Figure 3, column 6).

Discussion

This study addressed the cognitive function of the purported behavior monitoring system located in the ACC and indexed by the ERN (Dehaene, Posner, & Tucker, 1994; Falkenstein, Hohnsbein, Hoormann, & Blanke, 1991; Gehring, Goss, Coles, Meyer, & Donchin, 1993) by comparing the component structure of the behavioral error elicited ERN with the reward prediction violation elicited FRN. To the extent that the response-related ERN and the feedback stimulus-related FRN index activity in the same neural system performing the same cognitive operation, contrasting the eliciting conditions for the ERN and FRN can help define the actual cognitive operation performed by this monitoring system, whether it is monitoring for explicit behavioral errors, the outcomes of choices, or some more general monitoring function. The current results indicate that the ERN and FRN share a common central factor, likely the ACC generated component (Dehaene, Posner, & Tucker, 1994), but that the FRN contains a second, more anterior factor, perhaps indexing a more frontal medial component. Elicitation of the central component by reward prediction violation lends support to the theory that the error negativity (“error negativity” is used here as a general term to describe the shared aspects of the ERN and FRN) is dependent upon input from the VTA reward prediction system, but elicitation of the component in the absence of explicit action or choice demonstrates that the motor system involvement or overt choice evaluation are not required to engage the cognitive operation indexed by the error negativity.

Both the error minus correct difference ERN and the predicted reward minus unpredicted no reward difference FRN had medial frontal scalp distributions, but the FRN was more anterior than the ERN (see Figure 2), consistent with Gehring & Willoughby (Gehring & Willoughby, 2004), indicating either a more anterior source for the FRN than the ERN or of partially non-overlapping multiple-source configurations for the two components with one FRN source more anterior. The PCA decomposition results indicate the latter, showing that both behavioral errors and reward prediction violations share a medial central error negativity component, a component that fully accounted for the behavioral error elicited ERN, but that the reward prediction violation FRN contains an additional coincident prefrontal subcomponent accounting for its more anterior scalp distribution.

The PCA of the flanker data extracted a single factor (SF1TF1) that had the temporal, spatial, and response characteristics of the ERN: a temporal distribution between 0 – 200 ms peaking at about 100 ms post-response, a medial central scalp distribution, and differed significantly between correct and error responses. This factor accounted for six times as much variance as any other factor that varied by condition in the solution (18 % vs. 3%), suggesting that it reflects the primary ERP index of errors in the flanker task. The “virtual ERP” for the central spatial factor showed a clear correspondence with the ERN waveform (see Figures 1 and 3). While two other factors had partial temporal overlap with this factor (SF2TF3 and SF3TF1), neither had the same time-course as the ERN, indicating that this single medial central component fully accounts for the error negativity elicited to behavioral errors. This component appears to account for the classic ERN, indexing activity in the purported behavior monitoring system in the ACC (Dehaene, Posner, & Tucker, 1994).

The PCA of the reward prediction violation ERP also extracted a factor with a medial central spatial topography with the temporal distribution of the FRN (SF2TF2), from 250 – 350 ms peaking at about 300 ms, that differed significantly between when rewards were predictably delivered and when rewards were unexpectedly withheld. This factor had the same spatial distribution as the ERN factor (SF1TF1) from the flanker task, suggesting that it reflects the same ‘classic ERN’ component, the ERP index of error detection generated in the ACC (Dehaene, Posner, & Tucker, 1994; Falkenstein, Hohnsbein, Hoormann, & Blanke, 1991; Gehring, Goss, Coles, Meyer, & Donchin, 1993). The significant correlation across participants of the difference wave ERN and FRN support the idea of some commonality between the components. In the reward prediction design, however, this error negativity was elicited by the unexpected withholding of a predicted reward in the absence of any action or choice by the participant. If this ERP component reflects the same cognitive operation in the same neural system indexed by the response-locked ERN, then that system’s function cannot be confined to response conflict mediation (Gehring & Fencsik, 2001), behavioral error detection (Falkenstein, Hohnsbein, Hoormann, & Blanke, 1991; Falkenstein, Hoormann, Christ, & Hohnsbein, 2000), or choice outcome monitoring (Hajcak, Moser, Holroyd, & Simons, 2006; Nieuwenhuis, Yeung, Holroyd, Schurger, & Cohen, 2004) since there were no competing response options, behavioral errors, or alternative choices in the reward prediction design. However, if the concept of ‘error’ is expanded beyond the motor domain to include incorrect predictions about the availability of motivationally relevant items in the environment, then this monitoring system may be conceived as performing a more generalized error monitoring function: monitoring for errors of motivational outcome prediction, applied to actions, choices, or environmental events.

In addition to the central factor, the reward prediction violation ERP contained a frontal factor that also had the same time-course as the FRN, SF3TF2, indicating that the reward prediction violation FRN is comprised of two sub-components: one frontal and one central. The frontal factor, like the central factor, distinguished between predicted delivered rewards and when a reward was unexpectedly withheld. While inferring ERP source localization from scalp topography is speculative, the medial anterior inferior scalp distribution of the frontal component suggests that it may emanate from more anterior portions of medial prefrontal cortex, perhaps ventromedial prefrontal cortex (VMPFC). VMPFC, like ACC, receives projections from the VTA, and appears to have an evaluative function in the reward system, integrating reward prediction violation information with information about the environmental context and actions taken, updating estimates of the potential reward values of the current environment and of the current behavioral strategy when a delivered reward exceeds or fails to meet expectation (see reviews in (Bechara, Damasio, & Damasio, 2000; Rolls, 2000). Reward prediction violation has been shown to engage VMPFC using fMRI (Knutson & Cooper, 2005), single-unit recording in monkeys (Schoenbaum, Chiba, & Gallagher, 1998), and depth recording in a human patient (Oya, et al., 2005), and patients with damage to VMPFC are unable to use reward prediction violation information to guide future decisions (Bechara, Damasio, Damasio, & Anderson, 1994), indicating the role or VMPFC in the use of motivation information in the formation of strategic models of the environment. Thus in the reward prediction design there may be simultaneous detection of an event that fails to meet motivational goals, indexed by the central factor, and an updating of those goal representations, indexed by the frontal factor.

There was a frontal factor in the flanker ERP as well that differed between error and correct responses, SF2TF4, however its temporal distribution was later than the central factor, between 200 – 400 ms post-response and peaking about 280 ms post-response, suggesting that the prefrontal evaluative/integrative system was engaged in the flanker task, just later than the central detection system. This later medial frontal factor is in the temporal range of the Error Positivity (Pe; see Figure 3, column 3), which follows the ERN and is more positive on error trials (Falkenstein, Hohnsbein, Hoormann, & Blanke, 1990) see (Arbel & Donchin, in press; Overbeek, Nieuwenhuis, & Ridderinkhof, 2005) for reviews). The cognitive operation indexed by the Pe has received less study than the ERN, however one conception holds that the Pe reflects a more conscious reflection on an error and its consequences, in contrast to the ERN which reflects early detection (Nieuwenhuis, Ridderinkhof, Blom, Band, & Kok, 2001) but see also (Falkenstein, Hoormann, Christ, & Hohnsbein, 2000). Some researchers have suggested that the Pe is simply a P300 to the relatively rare error events (Davies, Segalowitz, Dywan, & Pailing, 2001), while others claim that the Pe appears dissociable into anterior and posterior subcomponents with only the posterior subcomponent reflecting a P300, while the anterior subcomponent is more related to the ERN (Arbel & Donchin, in press). In the current data the SF2TF4 factor may reflect the more anterior subcomponent of the Pe seen in (Arbel & Donchin, in press), perhaps reflecting the same evaluative/integrative function suggested for the frontal subcomponent of the reward prediction violation FRN. The latency difference between the central and frontal components following behaviorally (sequential) and environmentally (simultaneous) signaled errors may be due to substantial error information being available earlier in processing in behavioral errors, in which the motor program is prepared in advance of actual execution, and stimulus representations which can only be formed after presentation.

There were several factors in the flanker and reward prediction violation ERPs outside the temporal range of the ERN and FRN that differed by condition. The current study was not designed to elicit effects outside of the ERN and FRN temporal range, so the following interpretations are post-hoc speculations. A P300 subcomponent of the Pe may be reflected in the current flanker data by the posterior factor SF3TF1 that had a late difference between the error and correct responses. There was also another frontal factor in the flanker task that differed between error and correct responses, SF2TF3, that had most of its temporal distribution prior to the response. This factor may represent motor preparatory activity that differs between preparation or an error and correct response. Premotor cortex can hold multiple competing motor programs prior to execution choice (Cisek & Kalaska, 2002) and the to-be-executed representation gets enhanced and the others suppressed at decision (Cisek & Kalaskaa, 2005). There are ERP components in the pre-response period that may reflect this pre-response neural activity including the Readiness Potential or Lateralized Readiness Potential, a component beginning as early as 800 ms prior to the actual response reflecting motor program preparation, and a bilateral premotor potential possibly related to motor initiation (Deecke, Grözinger, & Kornhuber, 1976; Kornhuber & Deecke, 1965; Kutas & Donchin, 1980). (Gratton, Coles, Sirevaag, Eriksen, & Donchin, 1988) demonstrated that response correctness was predicted by premotor ERP activity. An LRP indicating the to-be-executed response appeared prior to the appearance of the imperative stimulus, thus on trials where the LRP was ipsilateral to the correct response indicated by the imperative stimulus, the response would be an error (Gratton, Coles, Sirevaag, Eriksen, & Donchin, 1988). The pre-response differential between errors and correct responses in this frontal factor may reflect an ERP index of these motor activation/inhibition processes.

In the reward prediction violation ERP there was a central factor, SF2TF1, that appeared to have most of its condition difference at the end of the epoch, perhaps reflecting a P300 response to the unpredicted stimuli (see Figure 3), although it is unclear why the flanker P300 would be more posterior than the reward prediction P300. There were two posterior factors in the reward prediction violation ERP that differed by condition, SF1TF2 and SF1TF6, but both were earlier than the P300. SF1TF2 differed by condition at about 180 ms post-stimulus and SF2TF6 varied between condition peaking at about 220 ms, perhaps reflecting N1 and P2 indices of differential attention to the unexpected stimulus (Eimer, Holmes, & McGlone, 2003; Johannes, Munte, Heinze, & Mangun, 1995) (Mangun, 1995).

Conclusion

The current study indicates that the ERN elicited by behavioral errors and FRN elicited by reward prediction violations consist of partially overlapping components: a medial central component and a medial frontal component. For behavioral errors, the central component is the only one present in the temporal range of the ERN, indicating that the ERN consists of a single component, likely indexing the ACC based error detection system (Dehaene, Posner, & Tucker, 1994; Falkenstein, Hohnsbein, Hoormann, & Blanke, 1991; Gehring, Goss, Coles, Meyer, & Donchin, 1993). However, since this same factor is also present in reward prediction violations, in the absence of explicit motor response or overt choice, the neurocognitive system indexed by this component cannot be linked solely to behavior or choice, as the dominant theories of the ERN and ACC function posit. Rather the system appears to be engaged by a more general class of prediction errors, errors that include actions, choice outcomes, and environmental events that fail to meet motivational goals. In its response to deviance from motivational expectation, the ACC monitoring system, indexed by the ERN, may join the general class of neural deviance responses, from simple sensory mismatch indexed by the mismatch negativity (Näätänen, Simpson, & Loveless, 1982), semantic expectation violation indexed by the N400 (Kutas & Hillyard, 1980), and violation of contextually constrained stimulus expectation indexed by the P300 (Donchin, 1981).

It should be noted that the reward prediction violation FRN elicited here, with its frontal subcomponent, may not be representative of all feedback error-related negativities. In the original (Miltner, Braun, & Coles, 1997) description of the FRN to action error feedback, the FRN had the same scalp distribution as the response-locked ERN. It is the FRN to monetary loss that has the more frontal distribution than the ERN, as shown in the current data and in (Gehring & Willoughby, 2004). Thus the FRNs to performance error feedback and reward prediction violation signal may not be equivalent; the inferior prefrontal subcomponent may only be present in feedback linked to an explicit reward. A study examining the component structure of reward related and performance related FRNs is needed to address that question.