When Errors Are Rewarding

Abstract

For social beings like humans, detecting one's own and others' errors is essential for efficient goal-directed behavior. Although one's own errors are always negative events, errors from other persons may be negative or positive depending on the social context. We used neuroimaging to disentangle brain activations related to error and reward processing, by manipulating the social context (cooperation or competition). Activation in posterior medial frontal cortex (pMFC) was increased for all errors, independent of who made the error or the reward outcome. Conversely, activity in striatum was modulated by reward, independent of whether the action was erroneous or not. The results demonstrate a clear distinction between error and reward processing in the human brain. Importantly, the current study indicates that error detection in pMFC is independent of reward and generalizes beyond our own actions, highlighting its role in optimizing performance in both individual and joint action.

Introduction

“Never interrupt an enemy when he is making a mistake.” This quote by Napoleon Bonaparte not only indicates that we are able to detect others' errors, we also may find others' errors useful, depending on the social context. Here, we study brain activity related to making and observing errors whose reward outcomes are modulated by the social context.

Making errors may result in serious consequences. Hence, successful goal-directed behavior crucially relies on monitoring of actions and the initiation of appropriate adjustments when the outcome deviates from the goal. Functional magnetic resonance imaging (fMRI) consistently implicates posterior medial frontal cortex (pMFC) [including pre-supplementary motor area (pre-SMA) and rostral cingulate zone (RCZ)] in the detection of own errors and in regulating subsequent adaptive behavior (Ridderinkhof et al., 2004). Involvement of pMFC in error detection has additionally been suggested by event-related potential (ERP) studies demonstrating an error-related ERP component known as the error-related negativity (ERN) (Falkenstein et al., 1990; Gehring et al., 1993), thought to have its source in the RCZ (Debener et al., 2005).

Importantly, along with the detection of own errors, we are also perfectly able to detect other people's errors. Recent ERP investigations of observed error detection demonstrated an ERN-like deflection also to be generated after the observation of someone else making an error (Miltner et al., 2004; van Schie et al., 2004). These findings suggest that pMFC may also be involved in the detection of other's errors and possibly relies on similar neural mechanisms.

The reinforcement-learning theory of the ERN in performance monitoring proposes a direct relation between error processing and reward processing in the brain (Holroyd and Coles, 2002). Whenever an error is made, the outcome of the action turns out to be worse than expected. This reward-prediction error is assumed to elicit the ERN upon arrival in RCZ. However, as our own errors are always associated with negative reward-prediction errors (i.e., with losses of anticipated rewards), disentangling whether pMFC activations in these tasks are error- or reward-dependent has proven to be an extremely difficult endeavor.

In contrast to own errors, observed errors may be highly rewarding events depending on the cooperative or competitive context in which the social interaction takes place. To date, it is unknown how these observed errors with different reward outcomes are processed in the brain. As the reinforcement-learning theory predicts pMFC only to be activated by events associated with loss of reward, increased pMFC activation is not expected for observed errors made by an opponent. On the other hand, studies also suggest that pMFC is particularly active when information needed to update representations about action outcome options is gathered (Walton et al., 2004; Rushworth and Behrens, 2008; Jocham et al., 2009). Importantly, such information is also conveyed by observing and detecting other's errors (Fehr and Fischbacher, 2003; Behrens et al., 2008), suggesting that pMFC activations may be increased for all types of errors independent of the associated reward. The current study aimed at using the different aspects of cooperative and competitive contexts to truly disentangle error and reward processes in the human brain.

Materials and Methods

Participants.

We measured brain activations in participants that were engaged in a challenging computer game (Fig. 1A,B) in either a cooperative or a competitive context. Twenty healthy volunteers (9 female; mean age, 26.2 years; SD, 3.2 years) participated in the cooperative task. Fourteen healthy volunteers (7 female; mean age, 24.7 years; SD, 2.4 years) participated in the competitive task. Participants provided written informed consent before participation, and the study was approved by the local research ethics committee.

Figure 1.

Cannon shooting task and pay-off matrix for the two contexts. Examples of a correct action (A) and an incorrect action (B). fMRI was time-locked to the presentation of the feedback, always containing the crucial information about the correctness of the performed or observed action. C, Pay-off matrix for the cooperative and competitive context. Participants started the task with a financial bonus of €10 in the cooperative task and €5 in the competitive task. Own errors were associated with a loss of 10 cents in both contexts. Observed errors in the cooperative context also resulted in a loss of 10 cents, but observed errors in the competitive context were associated with a gain of 10 cents. Correct responses had no effect on the bonus.

Task and design.

Participants performed and observed a computerized task, the cannon shooting game. The aim of the task was to stop a horizontally moving cannon (triangle) by a button press, precisely lining it up with a stationary target to shoot the target (square). For each experimental session, two participants came into the lab and were introduced to each other. They received instructions together, after which one participant went to one scanner and the other participant to the other. Unknown to the real participant, the other participant was a confederate. The computer mimicked actual performance of the real participant with a delay to balance the total amount of own and observed errors throughout the experiment. A mini-block design was used in which participants alternated between performing and observing in blocks of 10 trials each. In total, all participants performed and observed 10 of these mini blocks. The size of the target was dynamically adapted based on the participants' performance such that a mean hit rate of ∼60% was achieved.

The reward manipulations used in the cooperative and competitive tasks (Fig. 1) could be compared with those in a game of doubles tennis. When playing a game of doubles tennis, one is cooperating with their partner, and observing him or her making an error is an event with a negative outcome highly comparable with that of own errors and likewise resulting in a loss of reward. Conversely, observing an opponent on the other side of the net making an error turns out to be a rewarding event resulting in a gain. Thus, in the current paradigm, own errors were associated with a loss of reward in both contexts. Observed errors from a partner in the cooperative context also resulted in a loss of reward, but observed errors from an opponent in the competitive context was associated with a gain. Participants started the task with a financial bonus of €10 in the cooperative task and €5 in the competitive task. Own errors were associated with a loss of 10 cents in both contexts. Observed errors in the cooperative context also resulted in a loss of 10 cents, but observed errors in the competitive context were associated with a gain of 10 cents. Correct responses had no effect on the bonus.

Stimulus presentation was controlled with Presentation software (Neurobehavioral Systems). The onset of the trial was jittered with a blank screen for 0, 250, 500, 750, or 1000 ms. Null events were added to allow the blood oxygenation level-dependent (BOLD) response to go back to baseline. A trial started with the presentation of the target and the cannon. Target location was randomly determined on each trial, whereas the cannon was always horizontally centered. Immediately after presentation, the cannon started moving either to the right or the left for a maximum of 2.5 lengths (3500 ms) in total. An unambiguous feedback signal (thumb up/thumb down) was presented 500 ms after the button press, indicating whether the response resulted in a hit or a miss. Each block started with a cue instructing participants to perform (“You play”) or observe (“You observe”) presented in the center of the screen.

Data acquisition.

Imaging was performed at 3T on a Siemens Trio scanner (Siemens). Twenty-eight functional slices were obtained parallel to the anterior commissure–posterior commissure line using a single-shot gradient echoplanar imaging sequence (repetition time, 2 s; echo time, 30 ms; 64 × 64 pixel matrix; flip angle, 90°; field of view, 192 mm) sensitive to blood oxygen level-dependent contrast. Trials occurred at multiple, systematically offset time points (range, 0–0.5 s) in relation to the image acquisition to jitter the onset of the trials with respect to slice acquisition.

MRI data analysis.

The fMRI data were preprocessed and analyzed in an event-related manner, within the general linear model, with SPM5 software (Wellcome Department of Imaging Neuroscience, London, UK). Preprocessing consisted of spatial realignment, slice-timing correction, normalization, and spatial smoothing with a Gaussian kernel with a full-width at half-maximum of 5 mm. At the first (subject) level, we used a single statistical linear regression model for all our analyses, as follows. Each trial was modeled as a boxcar, of which the onset corresponded to feedback onset, and convolved with a standard hemodynamic response function. Separate regressors were created for the four conditions (agency: own vs observed; response type: correct vs error) and were entered in a full factorial design including the between-subjects factor context (cooperation vs competition). To control for effects of task difficulty, an additional regressor that modeled target size was included in the model. To correct for motion-related artifacts, we modeled subject-specific realignment parameters as covariates of no interest. Linear contrasts of regression coefficients were computed at the individual subject level and then subjected to a group-level random-effects analysis. To avoid any priors on brain localization, we applied whole-brain family-wise error correction for multiple comparisons on the basis of random field theory. We used a corrected voxel threshold of p < 0.05 and cluster threshold of k >100. The regression weights shown were extracted from the local maxima and analyzed using the MarsBar toolbox (the Marseille region of interest toolbox for Statistical Parametric Mapping).

Results

Behavioral results

In the cooperative task, the average hit rate was 63.0% (SD = 4.8%) for the participants and 63.3% (SD = 5.1%) for the computer. The average hit rate in the competitive task was 63.7% (SD = 2.7%) for the participants and 63.4% (SD = 3.1%) for the computer. A repeated-measures ANOVA demonstrated that performance did neither differ between performance and observation (F <1, p = 0.975) nor between cooperation and competition (F <1, p = 0.775).

fMRI results

Our main interest of the current study was to disentangle error mechanisms and reward mechanisms in the brain. Therefore, we focused on two general contrasts. First, we wanted to investigate error-specific brain activations by comparing erroneous actions to correct ones. Second, we were interested in reward-specific activations and contrasted positive reward outcomes with negative ones. As expected, error-specific activity was found in pMFC (pre-SMA and RCZ; local maximum at x = 4, y = 32, and z = 38) (Fig. 2A,C) and anterior insular cortex (Table 1; supplemental materials, available at www.jneurosci.org). Interestingly, the results showed that activity in these areas was increased for all types of errors, independent of agency, outcome, and difficulty of the actions. Hence, even observed errors in the competitive context that resulted in a gain showed a similar pattern of activation as own errors.

Figure 2.

Results of the fMRI analyses showing error-specific and reward-specific brain activations. Maps are thresholded at p < 0.05, corrected for multiple comparisons across the whole brain. A, Sagittal view of pMFC activity increases on error trials compared with correct trials. B, Coronal view of striatal activity increases on positive outcomes compared with negative outcomes. C, D, Beta weights (mean ± SEM) of the local maxima of pMFC (C) and the striatum (D) for own and observed correct and incorrect actions in the two different contexts (cooperation, competition). **p < 0.0001, *p < 0.05, ⁺p = 0.06.

Table 1.

Spatial coordinates of the local maxima of brain regions showing significant effects in the different contrasts

Contrast	Anatomical region	t value	Cluster size	Corrected p value	Stereotactic coordinates (x, y, z)
Error > correct
R posterior medial frontal cortex (BA 6/8/32)		6.97	279	<0.0001	4, 32, 38
R anterior inferior insula (BA 47)		8.24	668	<0.0001	30, 24, −12
L anterior inferior insula (BA 47)		7.92	463	<0.0001	−40, 22, −10
Positive outcome > negative outcome
L ventral striatum		7.46	711	<0.0001	−22, 2, −12
R ventral striatum		7.36	302	<0.0001	28, 0, −14
R dorsal striatum		7.32	214	<0.0001	30, −16, 6
R posterior cingulate (BA 31)		6.84	100	<0.0001	6, −40, 36
Agency × correctness (observed error processing > own-error processing)**
R ventral striatum		11.08	1046	<0.0001	14, 6, −12
L ventral striatum		9.49	970	<0.0001	−16, 6, −10
Posterior cingulate (BA 31)		6.20	202	<0.0001	12, −44, 36

Stereotactic coordinates correspond to the standard Montreal Neurological Institute brain.

**Neither the remaining two-way interactions nor the three-way interaction (agency × correctness × context) showed any significant activations. R, Right; L, left.

Also in line with previous research (Pagnoni et al., 2002; Ullsperger and von Cramon, 2003; Knutson and Cooper, 2005; Klein et al., 2007; O'Doherty et al., 2007), the reward-specific contrast revealed increased activations in the striatum with a maximum in the ventral striatum including the nucleus accumbens for positive reward outcomes compared with negative ones (local maximum at x = −22, y = 2, and z = −12) (Fig. 2B,D). Importantly, these areas were activated independently from agency and correctness.

The interaction contrasts revealed no significant signal changes except for the interaction between agency and correctness (Table 1), which showed increased activations in ventral striatum and posterior cingulate for observed error processing compared with own-error processing.

Discussion

The error-specific activation in pMFC is in line with earlier studies demonstrating increased activations in these areas after own errors. The current findings demonstrate that this area responds to all types of errors, i.e., does not dissociate between who produced the error, oneself or someone else. As such, the current findings support and extend previous ERP findings (Miltner et al., 2004; van Schie et al., 2004) suggesting the involvement of similar neural mechanisms in the detection of both types of errors. Importantly, the error-specific regions also do not dissociate between the reward outcome of the action, be it a relative gain or loss. Consequently, the error-monitoring network was involved even when an observed error resulted in a gain, supporting the error-specificity of these activations.

The areas that responded in a reward-related manner were clustered in the striatum. As in the case of error-specific activations, these reward-specific activations did not distinguish between who actually performed the action. In contrast to the error-specific findings, however, areas in the striatum did not dissociate between correctness of the action performed or observed. As long as the reward outcome of an action turned out to be the most rewarding result for the given situation, activity in these areas was increased. The interaction contrasts only revealed significant increases in striatal and posterior cingulate activations for observed error processing compared with own-error processing. This outcome is driven by the reversed activation patterns for observed errors in the competitive context and is also reflected in the main reward contrast (Fig. 2D). Hence, this interaction contrast reflects the crucial reward manipulation. Importantly, the remaining nonsignificant interactions demonstrate that there are no error-related specific activities that are significantly different between own and observed error processing.

Our error-specific findings do not fit with a strict interpretation of the reinforcement-learning theory of performance monitoring as this theory would predict increased pMFC activations after negative reward outcomes (losses) only. The current study demonstrates that an error is not defined by the personal reward outcome (gain/loss) associated with a given action but by the actual performance of an action. This makes sense in daily life where numerous existing uncertainties in an ever-changing environment would make it extremely difficult to distinguish correct from errors based on reward outcome information alone. Moreover, it is obvious that humans are perfectly capable of detecting highly rewarding errors, e.g., from opponents. Even when the observed error results in winning a grand slam final, for example, one easily detects the other's error and is able to determine that one's opponent performed an incorrect action. The current results show that pMFC is crucially involved in detecting these incorrect actions or motor performances, whereas the striatum is involved in processing the reward associated with the action.

Therefore, our findings are better in line with theories that assume pMFC to be specifically involved in updating action values to predict outcomes and enable potential strategy changes (Rushworth and Behrens, 2008). In real life as well as in the current study, incorrect actions are generally of greater importance and thus more informative than correct responses. Consequently, errors provide higher predictive values and thus activate pMFC to a greater extent (Rushworth, 2008). However, studies using learning paradigms where correct actions in the beginning of learning carry as much information as incorrect ones demonstrated pMFC also to be involved in processing the correct actions associated with a high predictive value (Procyk et al., 2000; Walton et al., 2004). This interpretation also explains why pMFC is often reported to be activated by other sources carrying predictive information, like response conflict (Botvinick et al., 2004) and error likelihood (Brown and Braver, 2005).

To conclude, the current study shows a clear distinction between performance monitoring and reward processing in the human brain. Performance monitoring crucially relies on pMFC independent of the reward outcome, which is in line with an action-value updating function of pMFC. Moreover, the present findings indicate that humans update not only self-performed actions but also observed actions performed by others. As Napoleon Bonaparte already pointed out 200 years ago, observing others making errors is highly relevant for one's own behavior and performance, as it may, for example, show weaknesses of an opponent and thus provide information on where to attack. But, it may also inform that an action, or strategy, is not worth being tried or might even be harmful. The mechanism demonstrated by the current study may thus crucially enable efficient joint actions and optimize social interactions.