Skip to main content
PMC home page
Human Brain Mapping logoLink to Human Brain Mapping
. 2011 May 26;33(7):1621–1633. doi: 10.1002/hbm.21305

Coupling electrophysiological and hemodynamic responses to errors

Nuria Doñamayor 1, Urs Heilbronner 2, Thomas F Münte 1,✉
PMCID: PMC6870069  PMID: 21618663

Abstract

Performance errors are associated with distinct electrophysiological and hemodynamic signatures: a fronto‐central error‐related negativity (ERN) is seen in the event‐related potentials and a network of activations including medio‐frontal, parietal, and insular cortex is revealed by functional magnetic resonance imaging. We used simultaneous electroencephalography and functional magnetic resonance imaging (fMRI) to characterize the relationship between the electrophysiological and hemodynamic responses to errors. Participants performed a modified Flanker task. When analyzed independently, we found the ERN and hemodynamic activations in dorsal anterior cingulate cortex, superior frontal gyrus, precentral gyrus, inferior frontal gyrus, and inferior parietal lobule. fMRI‐informed dipole modeling and joint independent component analysis (ICA) were used to couple electrophysiological and hemodynamic data. Both techniques revealed a temporal evolution of the areas found in the fMRI analysis, with the right hemisphere activations peaking before the left hemisphere. However, joint ICA added information, revealing a number of cortical and subcortical areas that had not been shown with parametric mapping. This technique also uncovered how these areas evolve over time. All together, these analyses provide a more detailed picture of the spatiotemporal dynamics of the processing of performance errors. Hum Brain Mapp 33:1621–1633, 2012. © 2011 Wiley Periodicals, Inc.

Keywords: electroencephalography, functional magnetic resonance imaging, multimodal imaging, error, error‐related negativity

INTRODUCTION

Processing of Errors: Evidence From Electrophysiological and Neuroimaging Studies

Errors are psychologically significant events. From religion to science, from psychoanalysis to behaviorism, what goes on inside our heads when we make a mistake and how we deal with its consequences has occupied people for hundreds of years. About 20 years ago, a number of pioneering studies were published which described how our brains react when we commit an error [Falkenstein et al.,1990,1991]. Since then numerous electroencephalography (EEG) studies [see, e.g., Ehlis et al.,2005; Gehring et al.,1993; Gentsch et al.,2009; Luu et al.,2003; Müller et al.,2005] have described a negative deflection of the event‐related potential (ERP), most prominent in fronto‐central electrodes, which occurs around 50–100 ms after making an erroneous response, and has been labeled error‐related negativity (ERN). The ERN is often followed by a so‐called error‐positivity (Pe), which peaks around 200–450 ms after the incorrect response and has a centro‐parietal scalp distribution. Source modeling has consistently located the neural generator of the ERN in the posterior medial frontal cortex [Dehaene et al.,1994; Herrmann et al.,2004; Keil et al.,2010; Mathewson et al.,2005].

Several theories have tried to explain the significance of the ERN, the most recent ones viewing it is a consequence of the conflict between the mental representation of the correct and the given response [van Veen and Carter,2002; Yeung et al.,2004], or part of a reinforcement‐learning process [Holroyd and Coles,2002]. Lately, the importance of motivational and emotional components has also been highlighted [Hajcak et al.,2004,2005], with studies showing that cognitively more significant errors elicit a greater ERN [Hajcak et al.,2005], or that the ERN can be modulated by inducing short‐term negative affect [Wiswede et al.,2009].

Functional magnetic resonance imaging (fMRI) studies on error processing have used similar tasks as the aforementioned ERP studies and have identified greater hemodynamic activity related to incorrect compared with correct responses in the supplementary motor area (SMA) and anterior cingulate cortex (ACC) (BA 6 and 32), bilateral insula and/or inferior frontal gyrus (IFG), bilateral inferior parietal lobule (IPL), posterior cingulate cortex (PCC), and precuneus [Carter et al.,1998; Hester et al.,2004,2009; Holroyd et al.,2004; Kiehl et al.,2000; Klein et al.,2007; Magno et al.,2006; Menon et al.,2001; Ridderinkhof et al.,2004; Ullsperger and von Cramon,2001]. The fact that these fMRI studies have unveiled a greater number of areas activated after committing an error has led some scientists to suggest that error processing might involve a more distributed neural network than that hypothesized on the basis of ERP studies [Menon et al.,2001].

Coupling Electrophysiological and Hemodynamic Data: Challenges and Achievements

For decades, neuroscientists have been exploring brain function using methods such as EEG and fMRI, which have numerous advantages, but also several drawbacks. When performing electrophysiological recordings, neuronal activity is directly measured with an excellent temporal resolution. However, EEG has a very poor spatial resolution, which is additionally complicated by having to solve the inverse problem [for a review of the properties of EEG see Rippon,2006]. Conversely, fMRI has an excellent spatial resolution, but it images hemodynamic activity, which not only is an indirect measure of neuronal activity, but also has a rather low temporal resolution [for a review of the properties of fMRI see Bandettini,2006]. Likewise, when choosing almost any other noninvasive neuroscientific method, such as magnetoencephalography, positron emission tomography, near infrared spectroscopy, etc., researchers have to make a decision between poor temporal or poor spatial resolution and decide between the lesser of two evils.

For this reason, multimodal imaging has long been a goal for many researchers, so as to exploit the advantages and circumvent the disadvantages of techniques such as EEG and fMRI. Thanks to a number of analysis methods, which have made it possible to remove gradient [Allen et al.,2000; Gonçalves et al.,2007; Koskinen and Vartiainen,2009] and cardioballistic [Debener et al.,2007; Kim et al.,2004; Leclercq et al.,2009; Niazy et al.,2005] artifacts from the EEG data and control the influence of the EEG equipment on the fMRI data [Mullinger et al.,2008], simultaneous acquisition of EEG and fMRI has now been performed successfully for about a decade.

How to couple electrophysiological and hemodynamic data is a more complex question to answer. The probably most straightforward, and also oldest, way of combining these data is to use the anatomical constraints derived from fMRI to optimize current density estimates obtained with EEG, whether it be seeding dipoles in areas hemodynamically active during the performance of the same task [e.g., Heinze et al.,1994; Thees et al.,2003] or performing source analysis of the EEG data over the entire cortical surface but spatially biasing the solution towards areas delimited by fMRI [e.g., Babiloni et al.,2004]. fMRI‐informed source modeling can be used both on data acquired independently or simultaneously. In contrast, EEG parameters, such as amplitude/latency of the ERP of interest, can also be used to modulate fMRI data, but this method does require simultaneous acquisition, since it needs to be done at the single‐trial level [Bénar et al.,2007; Debener et al.,2005,2006; Eichele et al.,2005; Goldman et al.,2009; Mulert et al.,2008; Warbrick et al.,2009]. Other studies have also correlated oscillatory activity from EEG with hemodynamic activity from fMRI, to use changes in the EEG signal as a predictor of the changes in hemodynamic activity recorded simultaneously [e.g., Laufs et al.,2003; Martínez‐Montes et al.,2004].

These methods of analysis focus on influencing one of the modalities with parameters extracted from the other modality. Other researchers, however, have used independent component analysis (ICA) [Makeig et al.,1996] in different ways to extract temporal information from EEG and spatial information from fMRI and combine these features. Joint ICA [Calhoun et al.,2006,2009; Moosmann et al.,2008] uses joint constraints of temporal and spatial independence of the EEG and fMRI data, respectively, to fuse the electrophysiological and hemodynamic results (this method is described in further detail in the Materials and Methods section). A complementary method is parallel ICA [Calhoun et al.,2009; Eichele et al.,2009,2008], which computes the independent components of both EEG and fMRI separately and then matches these by correlating their single‐trial modulation. Alternatively, a number of methods have been developed that are not based on ICA, such as multi‐set canonical correlation analysis [Correa et al.,2010], which decomposes EEG and fMRI based on trial‐to‐trial covariation across modalities, and methods using dynamic Bayesian networks to estimate neural activity from multimodal imaging [Plis et al.,2010].

Processing of Errors, Simultaneous Recordings, and the Current Study

Debener et al. [2005] have previously investigated the processing of errors by combining electrophysiological and hemodynamic results at the single‐trial level. These authors used infomax ICA [Bell and Sejnowski,1995] to isolate the EEG responses to errors and observed that trials with a greater ERN amplitude were associated with stronger BOLD responses in the dorsal ACC (BA 32). Also, by convolving the single‐trial amplitude of the ERN with the hemodynamic response function, Debener et al. [2005] used EEG‐informed fMRI analysis and identified the dorsal ACC as the area most relevant to performance monitoring, separating it from the other error‐related activations present in their conventional fMRI analysis.

In contrast to the aforementioned work, the aim of our study was to explore the association between the averaged hemodynamic and electrophysiological responses to performance errors. To this end, we recorded EEG and fMRI simultaneously.

In the electrophysiological data, we expected to observe a fronto‐centrally distributed ERN component to errors, whereas hemodynamic data were anticipated to show greater activation for errors compared to correct answers in areas such as the ACC/SMA and the insula/IFG bilaterally. By using both fMRI‐informed dipole modeling and joint ICA [Calhoun et al.,2006], we wanted to explore the relationship between the EEG and fMRI responses following performance errors.

MATERIALS AND METHODS

Participants

Data sets from 10 participants (seven women, aged 23–30 years, M = 26.50, SD = 2.27) were analyzed in this study. Data from two additional participants were lost due to artifacts. The study had been approved by the ethics committee of the Otto‐von‐Guericke‐Universität, which was the affiliation of all authors during the execution of the study, and all participants had given written informed consent prior to their participation.

Stimuli and Procedure

A modified version of the Eriksen flanker task [Eriksen and Eriksen,1974] was used. Participants were presented for 100 ms with five white arrowheads in the center of a black screen, which were arranged to be either compatible (>>>>>/<<<<<) or incompatible (<<><</>><>>), with ∼60% incompatible stimuli. A fixation cross was shown between trials. Participants were instructed to make a button press in the direction of the central symbol with the corresponding index finger. Only responses within 900 ms after stimulus‐onset were analyzed. The mean stimulus onset asynchrony was 1,588 ms, jittered between 1,000 and 4,000 ms. The experiment comprised 942 trials, divided into five blocks of 5.5 min. The stimuli were presented using Presentation software (Neurobehavioral Systems, Albany, CA).

EEG Data Acquisition

EEG data were recorded using a 32‐channel MR compatible EEG system (BrainProducts, Munich, Germany). The EEG cap harbored 30 scalp Ag/AgCl electrodes distributed according to the 10–20 system and two additional electrodes, one of which was fixed below the left eye to record the electrooculogram (EOG), and the other was attached on the right side of the left scapula to acquire the electrocardiogram (ECG). The reference and ground electrodes were located at CPz and FCz, respectively. Data were sampled at 5,000 Hz, with a bandpass filter of 0.016–250 Hz and a resolution of 0.1 μV. EEG electrodes had built‐in 15 kΩ resistors, and the ECG and EOG electrodes had built‐in 20 kΩ resistors, therefore impedances were kept below 20 and 25 kΩ, respectively.

fMRI Data Acquisition

Functional MRI images were acquired using a GE 1.5‐T Signa LX scanner (GE Medical Systems, Milwaukee, WI), equipped with a birdcage head coil. The participant's head was immobilized using sponge pads. A standard EPI fMRI sequence was used (voxel dimensions 5 × 5 × 5 mm, 23 slices, 64 × 64 matrix, TE = 35 ms, TR = 2,000 ms, FOV 20, flip angle 80°), and a high resolution 3D T1‐weighted anatomical scan was also acquired in a different session (voxel dimensions 1.5 × 1.5 × 1.5 mm, 124 slices, 256 × 256 matrix, TE = 8 ms, TR = 2,400 ms, FOV 25, flip angle 30°).

EEG Data Analysis

Raw EEG data were processed offline using BrainVision Analyzer 2 (Brain Products, Gilching, Germany). Gradient artifacts were corrected using a modified version of the algorithms proposed by Allen et al. [2000], where a baseline‐corrected sliding average of 20 MR volumes was calculated to correct each volume. The data were subsequently downsampled to 250 Hz and low‐pass filtered at 70 Hz (bandstop at 50 Hz). After gradient artifact correction, QRS complexes were identified semiautomatically, with manual adjustment for misidentified peaks and ballistocardiogram (BCG) artifacts were corrected using Optimal Basis Set (OBS) [Niazy et al.,2005], as implemented in EEGLAB [Delorme and Makeig,2004]. OBS was chosen to remove BCG artifacts over the Average Artifact Subtraction method implemented in BrainVision Analyzer 2, since it has been shown to lead to considerably cleaner data [Warbrick and Bagshaw,2008]. Following BCG artifact correction, data analysis continued with BrainVision Analyzer 2 by re‐referencing the data to the average of TP9 and TP10 and using ICA based on the Infomax principle [Makeig et al.,1996] to correct ocular and muscular artifacts, as well as residual scanner and BCG artifacts. Thirty components were estimated and an average of 23 (SD = 3) were kept for back‐projection (an example of data before and after ICA‐based artifact correction, and grand average ERPs for all channels with and without ICA‐based artifact correction can be found in Supporting Information Figs. 1 and 2). Data were then bandpass filtered between 1 and 20 Hz, segmented into 600 ms response‐locked epochs (−200 to 400 ms postresponse) and inspected for epochs containing a voltage change of more than 50 μV, which were then rejected. The remaining epochs were baselined between −200 and −100 ms, averaged separately for compatible correct, incompatible correct and incompatible erroneous responses, and individual averages collapsed to calculate the grand averages.

Figure 1.

Figure 1

Open in a new tab

(A) Response‐locked ERPs for incompatible erroneous (solid red line), incompatible correct (solid green line), and compatible correct (dashed green line) responses at Fz and Cz, and (B) scalp topography of the difference between incompatible erroneous and incompatible correct responses between 0 and 100 ms postresponse. (C) Cortical regions showing greater activity for incompatible erroneous than incompatible correct responses. Note that all cerebral images in this work are shown in neurological convention. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

Figure 2.

Figure 2

Open in a new tab

(A) Brain slices showing the three regions of greater activity for incompatible erroneous than incompatible correct responses, where the dipoles were seeded (left dorsal ACC in red, right IFG in blue, left IFG in green, right IPL in magenta, and left IPL in tan). (B–D) Response‐locked source waveforms for the incompatible erroneous responses for the three computed solutions: (B) left dorsal ACC, (C) left dorsal ACC and bilateral IFG, and (D) left dorsal ACC and bilateral IPL. Source waveform colors match the colors of the dipoles depicted on the slices. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

A repeated‐measures ANOVA was performed on the averaged amplitude of the ERPs between 0 and 100 ms using the factors condition (incompatible correct/incompatible incorrect) and electrode (Fz/Cz). The scalp topography for the difference between incompatible erroneous and incompatible correct responses in this time window was also computed.

fMRI Data Analysis

fMRI data were processed offline using the software package SPM5 (http://www.fil.ion.ucl.ac.uk/spm/). The first four images of each run were discarded to account for T1 equilibrium effects. Functional images of the five runs were corrected for differences in slice time acquisition, and then spatially realigned to the first image of each run. Subsequently, data were spatially normalized into standard Montreal Neurological Institute space [Friston et al.,1995]. Finally, data were spatially smoothed with an 8 × 8 × 8 mm full width at half‐maximum Gaussian kernel.

An event‐related design matrix was created including the conditions of interest: compatible correct, incompatible correct and incompatible incorrect responses. To accommodate for movement artifacts, realignment parameters were treated as additional predictors in the model. The individual data were analyzed using a stick function corrected for temporal delay of the BOLD response, resulting in the contrast of interest: incompatible incorrect > incompatible correct. Images of the parameter estimates of each participant's contrast were entered into a one‐sample t‐test, and significant activations were identified using an uncorrected threshold of P < 0.0005 and a cluster threshold of 20 contiguous voxels. Functional activation images were projected on the standard single subject brain provided by SPM5.

fMRI‐Informed Dipole Modeling

Source analysis was performed with BESA (MEGIS Software GmbH, Gräfelfing, Germany), using the average ERP and a realistic isotropic head model with a conductivity ratio of 90. Dipoles were seeded in areas which had shown significant activation in the fMRI analysis. One single‐ and two three‐dipole solutions were computed, the source waveforms were obtained for the three dipoles between −200 and 400 ms postresponse, baselined between −200 and −100, and the goodness of fit was calculated for the 1–20 Hz filtered error condition. Dipoles were projected on the fMR images and the standard single subject brain provided by SPM5 using BrainVoyager QX 2.0 (Brain Innovation, Maastricht, The Netherlands).

ERP‐fMRI Data Fusion

The Fusion ICA Toolbox [http://icatb.sourceforge.net/; Calhoun et al.,2006], implemented in Matlab 7.2 (Mathworks, Natick, MA), was used to couple the electrophysiological and hemodynamic data. This toolbox uses an algorithm based on the infomax principle [Bell and Sejnowski,1995], assumes joint spatial and temporal independence of the fMRI and EEG sources, respectively, and computes the shared unmixing matrix and the fused ERP and fMRI sources.

The ERP of the incompatible erroneous responses at Cz and the contrast images for the error predictor the features entered into the joint ICA. The number of independent components of the joint EEG‐fMRI data were estimated to be nine using a method based on the minimum description length criteria [Wax and Kailath,1985]. Independent components were ranked according to their contribution to the time course of the ERP of the erroneous response by first regressing them on the potential. Subsequently, the maximum absolute peak of each component was calculated and components which explained more than one standard deviation of the ERP were used for further analysis. The corresponding fMRI components were scaled to Z values and voxels were colorized if Z ≥ 3.5.

Subsequently, to reveal the dynamic interplay of the hemodynamic and electrophysiological components, twofold spatiotemporal reconstruction was performed [Calhoun et al.,2006]. Firstly, the fMRI components were weighted by their joint ERP time courses and linearly combined, showing an evolution of the fMR images over time. Conversely, to estimate the “ERP time course” of given voxels, the ERP components were weighted by their joint fMRI components and linearly combined. For further details on the algorithms used see Calhoun et al. [2006,2009].

RESULTS

Behavioral Results

On average participants responded correctly to 822 (SD = 60) stimuli and incorrectly to 117 (SD = 60) stimuli. Incorrect responses followed incompatible stimuli (M = 104, SD = 56) significantly more often than compatible stimuli (M = 13, SD = 11), t(9) = 5.37, r = 0.87, P < 0.001. Given that the number of errors following compatible stimuli was too small for a meaningful statistical inference, this condition was excluded from further analysis.

Reaction times were significantly shorter for compatible (M = 389 ms, SD = 50) than incompatible trials (M = 411 ms, SD = 38), t(9) = −3.28, r = 0.74, P < 0.01. Following incompatible stimuli, erroneous responses (M = 371 ms, SD = 42) were made significantly faster than correct responses (M = 450 ms, SD = 36), t(9) = −14.07, r = 0.98, P < 0.001. The average time between two consecutive errors following an incompatible stimulus was 16.01 s (SD = 6.28).

Event‐Related Potentials

ERPs showed a more pronounced negative deflection after incompatible erroneous responses in comparison to both compatible and incompatible correct responses in the analyzed electrodes, Fz and Cz. Incompatible erroneous responses showed a maximal negative peak of −5.6 μV at 48 ms in Fz and of −4.3 μV at 48 ms in Cz. In the time window of the ERN, incompatible correct responses peaked with −1.8 μV at 40 ms in Fz and with 0.3 μV at 36 ms in Cz, whereas compatible correct responses peaked with −2.2 μV at 32 ms in Fz and with −0.01 μV at 24 ms in Cz (Fig. 1A).

A 2 (incompatible correct/incompatible error) × 2 (Fz/Cz) repeated‐measures ANOVA was performed on the 0–100 ms time window. The analysis of the within‐subjects effects showed that the main effects of condition and electrode site were statistically significant, F(1,9) = 25.27, P < 0.005, and F(1,9) = 8.32, P < 0.05, respectively, whereas the interaction between condition and electrode site was not, F(1,9) = 1.08, P > 0.05. The scalp topography of the difference between erroneous and correct responses in this time window showed a fronto‐central distribution (Fig. 1B). Cz was chosen to be introduced in the Fusion ICA Toolbox, since data from only one electrode can be used as an input.

fMRI Activations

Areas that showed a significant increase of hemodynamic activity for incompatible erroneous compared with incompatible correct responses can be seen in Figure 1C. All cortical activations were more pronounced on the right hemisphere, as can be appreciated in the figure. The exact locations and coordinates of the activations can be seen in Table I.

Table I.

MNI coordinates of the fMRI activations of the contrast incompatible error > incompatible correct

BA X Y Z
Left dorsal ACC 32 −4 18 44
Right SFG 6 4 2 66
Right SFG 6 20 −2 72
Left SFG 9 −32 50 32
Right precentral gyrus 6 34 −12 66
Right IFG 47 46 18 2
Left IFG 47 −44 18 4
Right IPL 40 44 −44 44
Left IPL 40 −36 −50 42
Midbrain * 10 −24 20
Open in a new tab

BA, Brodmann's area; ACC, anterior cingulate cortex; IFG, inferior frontal gyrus; IPL, inferior parietal lobule; SFG, superior frontal gyrus.

*No BA available.

fMRI‐Informed Dipole Modeling

Dipoles were seeded in coordinates corresponding to five fMRI activations (Fig. 2A), i.e., left dorsal ACC, bilateral IFG and bilateral IPL. A first solution (Fig. 2B) was calculated using a single dipole seeded in the left dorsal ACC (BA 32; MNI −4, 19, 40), which showed a negative peak of −56.32 nAm at 44 ms postresponse, corresponding with the peak of the ERN. This solution was found to explain 81.09% of the variance of the ERN (0–100 ms).

In a second solution (Fig. 2C), three dipoles were seeded in the left dorsal ACC (BA 32; MNI −4, 19, 40), right IFG (BA 47; MNI 46, 18, 1) and left IFG (BA 47; MNI −44, 18, 2), peaking with −51.13 nAm at 44 ms (left dorsal ACC), −17.41 nAm at 44 ms (right IFG) and −8.77 nAm at 76 ms (left IFG). This three‐dipole model explained 90.51% of the variance around the peak of the ERN (0–100 ms).

A final three‐dipole model (Fig. 2D) was computed with seeds in the left dorsal ACC (BA 32; MNI −4, 19, 40), right IPL (BA 40; MNI 44, −41, 43) and left IPL (BA 40; MNI −36, −47, 41). The peaks of the source waveforms were found at 44 ms with −44.64 nAm for the left dorsal ACC, at 36 ms with −17.85 nAm for the right IPL and at 84 ms with −9.06 nAm for the left IPL. This solution explained 92.13% of the variance of the ERN (0–100 ms).

ERP‐fMRI Data Fusion

Nine independent components of the joint EEG‐fMRI data were estimated using the minimum description length criteria [Wax and Kailath,1985]. Of these, seven EEG components were found to explain over one standard deviation of the ERP of the incompatible erroneous responses at their maximum peak. These and their corresponding fMRI components can be seen in Figure 3. The MNI coordinates of the hemodynamic components can be found in Table II.

Figure 3.

Figure 3

Open in a new tab

ERP time courses (blue line) that were considered for analysis, superimposed on the ERP for incompatible erroneous responses (red line), and their corresponding fMRI independent components. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

Table II.

MNI coordinates of the hemodynamic components obtained with joint ICA [Calhoun et al.,2006]

BA X Y Z
Component 1
 Right ventral ACC 24 2 36 4
 Right SFG 11 32 56 −10
 Right STG 38 42 8 −20
 Left STG 38 −40 10 −20
 Right Culmen * 16 −28 −22
 Left Culmen * −14 −32 −24
Component 2
 Left IFG 9 −56 18 28
 Left Precuneus 19 −34 −80 36
 Left SPL 7 −28 −62 64
 Left STG 22 −54 −82 18
Component 3
 Left dorsal ACC 32 −2 32 −6
 Right SFG 8 24 42 50
 Right Insula 13 44 −10 −8
 Right PCC 30 2 −64 4
 Left PCC 30 −12 −60 6
 Right PHG 35 20 −24 −14
 Right STG 38 42 10 −22
 Left STG 38 −40 6 −20
Component 4
 Right SFG 10 36 62 0
 Left MFG 8 −48 10 42
 Left IFG 45 −48 26 20
 Left IPL 40 −52 −50 52
 Left STG 39 −50 −62 28
 Left Declive * −30 −86 −26
Component 5
 Right IFG * 48 44 0
 Left IPL 40 −62 −30 30
Component 6
 Right ACC 32 2 16 44
 Right IFG 47 58 16 0
 Left STG 38 −48 16 −12
 Right Cuneus 17 10 −100 −8
Open in a new tab

BA, Brodmann's area; ACC, anterior cingulate cortex; IFG, inferior frontal gyrus; IPL, inferior parietal lobule; MFG, middle frontal gyrus; PCC, posterior cingulate cortex; PHG, parahippocampal gyrus; SFG, superior frontal gyrus; SPL, superior parietal lobule; STG, superior temporal gyrus.

*No BA available.

The first EEG component (Fig. 3A) peaked positively at 260 ms and was coupled with hemodynamic activations on right superior frontal gyrus (SFG), right ventral ACC and bilateral superior temporal gyrus (STG), and bilateral cerebellar activations on the culmen.

The second component (Fig. 3B) displayed a positive peak at −16 ms and corresponded to left‐lateralized activity in IFG, precuneus, superior parietal lobule (SPL) and STG.

The third electrophysiological component (Fig. 3C) showed its maximum at 164 ms postresponse and was fused with an increase in activation in right SFG, left dorsal ACC, right parahippocampal gyrus (PHG), bilateral PCC, right insula and bilateral STG.

The fourth EEG component (Fig. 3D) peaked negatively at 122 ms and was matched with an increase in hemodynamic activity in left IFG, left middle frontal gyrus (MFG), right SFG, left IPL and left STG, and cerebellar activity in the left declive.

The fifth component (Fig. 3E) revealed its maximum peak at 204 ms postresponse and was coupled with hemodynamic activations of the right IFG and left IPL.

The sixth and last electrophysiological component (Fig. 3F) largely corresponded with the ERN, peaking at 44 ms, and corresponded to fMRI activations in right IFG, right dorsal ACC, right cuneus, and left STG.

Spatiotemporal Reconstruction

A spatiotemporal chain of images was obtained by linearly combining the hemodynamic components and weighting them by their joint electrophysiological components (see Fig. 4). This revealed hemodynamic activity in the interval between −55 and −20 ms, shortly before the subjects made an erroneous response following an incompatible stimulus. As seen in Figure 4, at −35 ms activations could be found bilaterally in ACC (BA 24, 32), SFG (BA 9) and IFG (BA 13, 47), as well as in right IPL (BA 40), STG (BA 22), and lingual gyrus (BA 17), and left declive.

Figure 4.

Figure 4

Open in a new tab

Reconstruction of the fMRI activations at five points in time (−35, 5, 50, 85, and 200 ms). Depicted on the top row is the ERP for incompatible erroneous responses (red line) and the six components used for the reconstruction (cyan, green, magenta, yellow, orange, and blue lines), on the bottom row are representative slices (z = 45, z = 10, and z = −5) at said points postresponse. All slices at additional time points can be seen in Supporting Information Figure 3. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

The spatiotemporal reconstruction also showed hemodynamic activity in the interval between 5 and 90 ms postresponse. Immediately after the participants responded erroneously (5 ms) fMRI activations could be seen right‐lateralized in ACC (BA 24, 32), IPL (BA 40) and lingual gyrus (BA 17), and bilaterally in IFG (BA 47).

At approximately the peak of the ERN (50 ms), multiple activations could be seen. Most notably, the previous activations became more extensive: bilateral ACC (BA 24, 32), IFG (BA 45, 47), and IPL (BA 40), and right lingual gyrus (BA 18). Additionally, hemodynamic activations were visible in SFG (BA 6), MFG (BA 6, 8, 10), insula (BA 13), PCC (BA 23, 31), SPL (BA 7), inferior temporal gyrus (ITG; BA 20) and thalamus, all bilaterally; right‐lateralized in IFG (BA 46), middle temporal gyrus (MTG; BA 39) and cuneus (BA 19); and left‐lateralized in precentral gyrus (BA 4), MFG (BA 47), IFG (BA 9), and precuneus (BA 7).

Towards the end of the ERN (85 ms), hemodynamic activity was analogous to that observed at 5 ms, with additional activations in bilateral SFG (BA 8) and left insula (BA 13) at 85 ms postresponse.

Finally, some fMRI activations could be seen between 185 and 215 ms postresponse. These occurred right‐lateralized in dorsal ACC (BA 32) and IFG (BA 45, 47) at ∼200 ms postresponse.

The second step in the spatiotemporal reconstruction was to estimate the ERP time course of given voxels (see Fig. 5). In this case it was observed that the reconstructed time courses of areas such as SPL (1.6 μV at −28 ms), IFG (2.7 μV at −32 ms) and both dorsal (2.9 μV at −32 ms) and ventral (1.3 μV at −32 ms) ACC coincided largely with the positive peak preceding the ERN (1.8 μV at −28 ms). To a lesser extent than these, the right STG (0.5 μV at −28 ms) and the left culmen (0.8 μV at −24 ms) also showed a positive deflection preceding the participant's response.

Figure 5.

Figure 5

Open in a new tab

Reconstruction of the ERP time courses at given voxels (axial slices at z = 65, z = 45, z = 30, z = 0, and z = −20). All ERP time courses (blue lines) can be seen superimposed on the ERP for incompatible erroneous responses (red line). [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

The dorsal and ventral ACC and the right IFG showed the most pronounced negative deflections at the time of the ERN (48 ms with −4.3 μV), peaking with −6.2 μV (44 ms), −3.9 μV (48 ms), and −7.2 μV (44 ms), respectively. Additionally, on the left hemisphere, the ERP time courses of the SPL (−2.2 μV at 52 ms), the MFG (−1.1 μV at 64 ms), both the more superior (−1.7 μV at 52 ms) and more inferior (−1.3 μV at 44 ms) portions of the IPL, as well as the culmen (−1.7 μV at 60 ms), and on the right hemisphere, the reconstructed waveform of the STG (−2.8μV at 48 ms) showed important negative deflections corresponding to the ERN.

Finally, several areas displayed considerable positivities at the time of the Pe (208 ms with 1.8 μV). The time courses of dorsal ACC and IFG, with peaks of 2.0 μV at 204 ms and 2.4 μV at 200 ms, respectively, almost fully coincided with the Pe at Cz. Similarly, the reconstructions of the ventral ACC (1.4 μV at 264 ms) and the right STG (1.1 μV at 260 ms) closely matched the positivity, although their appearance was slightly flatter. In contrast, the parietal time courses showed a pronounced peak closely matching the peak of the Pe, with the IPL (MNI −62, −30, 30) peaking at 204 ms with 1.0 μV and the SPL at 212 ms with 1.4 μV. The ERP time courses of left STG and, in the cerebellum, bilateral culmen showed two distinct peaks during the time window of the Pe, the second of which was most pronounced in all three cases, peaking with 1.2 μV (256 ms) for the left STG, with 1.1 μV (256 ms) for the left and with 0.9 μV (260 ms) for the right culmen.

DISCUSSION

In our work we have used simultaneous EEG and fMRI to study the dynamic interplay of electrophysiological and hemodynamic responses to performance errors. As hypothesized, the separate analysis of each modality resulted in the known pattern of electrophysiological and hemodynamic responses to errors. The ERPs show a negative deflection around 50 ms postresponse, which has a central scalp distribution, and is therefore consistent with the ERN [e.g., Falkenstein et al.,1990,1991; Gehring et al.,1993; Luu et al.,2003]. The activity shown by the fMRI reveals maxima in left dorsal ACC (BA 32), bilateral SFG (BA 6, 9), right precentral gyrus (BA 6), bilateral IFG (BA 47), and bilateral IPL (BA 40), consistently with previous reports comparing hemodynamic activity after incompatible erroneous and incompatible correct responses [e.g., Debener et al.,2005; e.g., Hester et al.,2004,2009; Kiehl et al.,2000; Magno et al.,2006].

Given the tight correspondence between electrophysiological and hemodynamic responses [Logothetis et al.,2001; Niessing et al.,2005] and the fact that a more ample network of activations is revealed when using techniques with high spatial resolution like fMRI, we agree with the proposal of authors such as Menon et al. [2001], who suggest that inverse solutions locating the source of the ERN in just one or two brain regions might be masking the complexity of the wider neural network uncovered by fMRI studies. Therefore we used both fMRI informed dipole modeling and joint ICA [Calhoun et al.,2006] to couple the electrophysiological and hemodynamic data, and reveal additional information on the spatiotemporal dynamics of the neural network that underlies the processing of performance errors.

Some authors [Dehaene et al.,1994; Mathewson et al.,2005] have previously used single dipole solutions to locate the source of the ERN. However, when placing a single dipole within the medio‐frontal hemodynamic activation, roughly 19% of the variance is left unexplained, suggesting that more areas of the network revealed by statistical parametric mapping should be used as dipole seeds to give a more physiological solution. Indeed both of the three‐dipole solutions computed by placing two extra dipoles in either the IFG or the IPL, bilaterally, increased the goodness of fit of the model to ∼92%.

Although the differences in the goodness of fit values between both three‐dipole solutions are negligible, a number of interesting observations can be made. In the solution with seeds in left dorsal ACC and bilateral IFG, the source waveforms reveal a temporal evolution of the sources, with the dorsal ACC and right IFG peaking simultaneously in first place, followed by the left IFG around 30 ms later. The bulk of the variance of the ERN seems to be carried by the medial dipole; however, both lateral dipoles, especially the one seeded in the left IFG, appear to contribute greatly to the positivity preceding the ERN. Similarly, with medial and parietal seeds, the bulk of the variance of the ERN still seems to be explained by the dorsal ACC dipole. Regardless, this three‐dipole solution also shows a distinct temporal pattern, with the right IPL peaking slightly before the dorsal ACC, but both around the maximum of the ERN, and the left IPL roughly 40 ms later.

Alternatively, we analyzed our data using joint ICA [Calhoun et al.,2006]. Previous studies have shown that the ERN can be rather well isolated by using ICA [Debener et al.,2005; Gentsch et al.,2009], as can also be observed in our data set. However, by allowing other components in the analysis, which also explained a substantial amount of the variance of the ERP, the current analysis included their coupled fMRI components. Some of these hemodynamic activations had not been revealed by statistical parametric mapping, which is in accord with the suggestion that the statistical analysis methods typically used on fMRI data might actually be underestimating the activity related to the task [Logothetis et al.,2001].

The method proposed by Calhoun et al. [2006] estimates the temporal variation of the hemodynamic signal by weighting the spatial with the temporal components, revealing how the activation of different brain areas evolves over time. Given that the spatiotemporal reconstruction is based on the linear weighting of the components, it must be taken into consideration that the component with the greatest amplitude will have the greatest weight, explaining why the component that carries the bulk of the variance of the ERN is most present in the analysis of performance errors. Still, the spatiotemporal reconstruction reveals that, shortly before the participant makes an erroneous response, the electrophysiological response is already matched with some bilateral hemodynamic activations in the limbic (ACC) and frontal (SFG, IFG) lobes, right‐lateralized activity in IPL, STG and lingual gyrus, and left‐lateralized cerebellar activation in the declive. An especially rich picture is shown coupled with the ERN, with initial right hemisphere activations in ACC, IPL and lingual gyrus, and bilateral activations in IFG, immediately after an erroneous response following an incompatible stimulus. At the peak of the ERN, around 50 ms postresponse, the previous right‐lateralized activations can also be observed in the contralateral hemisphere and are accompanied by more bilateral hemodynamic activity in SFG, MFG, insula, PCC, SPL, ITG and thalamus, as well as activity in right MTG and cuneus, and left precentral gyrus and precuneus. Around the end of the ERN, though, only the initial activations persist, except for some bilateral activity of the SFG and an activation in the left insula.

At the threshold used, a small portion of hemodynamic activation in the right dorsal ACC and IFG is also coupled with the Pe. An increasing amount of research has concerned itself with whether the ERN and Pe represent the same or different processes, with several studies associating the Pe with, e.g., the degree of awareness or the motivational significance of the error [Nieuwenhuis et al.,2001; Overbeek et al.,2005; Ridderinkhof et al.,2009]. While our data does not provide sufficient information to make any functional inference about the Pe, it does seem to indicate that a different set of brain areas are associated to the ERN and the Pe. This altogether is an indicator that, whether or not the ERN and the Pe are part of the same process, these two components are at least anatomically dissociable.

Interestingly, similar to what is observed with joint ICA, the dipoles seeded in the fMRI activations also revealed a staggered activation of both hemispheres, with the right and medial areas peaking towards the beginning of the ERN and the left ones towards the end. But nonetheless there is an important gain in information with joint ICA, since it reveals how hemodynamic activation evolves over time and also, as stated previously, unveils a number of brain areas, whose activity had probably been underestimated by statistical parametric mapping. As expected, given that joint ICA model was computed using the error contrast, all these areas have previously been associated to error processing in numerous studies [Debener et al.,2005; Garavan et al.,2002; Hester et al.,2004; Kiehl et al.,2000; Menon et al.,2001; Ullsperger and von Cramon,2001], and/or with other closely related processes such as response competition [Ullsperger and von Cramon,2001] or inhibition [Braver et al.,2001; Menon et al.,2001]. By weighting the responses from both modalities, joint ICA illustrates how these hemodynamic activations correspond with the different time points of the ERP associated to errors following an incompatible stimulus.

Spatiotemporal reconstruction was also used to obtain the ERP time courses of the hemodynamic components. Once again, in this case, linear weighting provokes that the component with the greater amplitude carries the bulk of the variance in most of the ERP time courses. The reconstructed time courses reveal that the negative deflection corresponding to the ERN is greater in the case of the right IFG, closely followed by the dorsal and ventral ACC, although, in contrast to the dipole waveforms, they all peak practically simultaneously around the maximum of the ERN.

Additionally, joint ICA brought out some subcortical hemodynamic activity, bilaterally located in the thalamus. Once again, the associated time courses show a pronounced negativity coinciding with the ERN. It is well known that subcortical areas such as thalamus, habenula or basal ganglia play an important role in the dopamine system and hence in the reward system, and that these work closely with cortical limbic structures, such as the ones found in our work [Hester et al.,2004; Li et al.,2008; Tsukamoto et al.,2006; Ullsperger and von Cramon,2001,2003]. However, subcortical signals are difficult to detect on the scalp and the temporal properties of hemodynamic activity makes it impossible to discern between phasic and tonic activity. This makes the combination of EEG and fMRI especially relevant, since it is a solid non invasive method able to estimate the temporal evolution of the deep subcortical areas shown by previous fMRI studies.

Summarizing, joint ICA [Calhoun et al.,2006] is an analysis technique, which utilizes the temporal and spatial advantages of EEG and fMRI, respectively. Joint ICA has the main advantage, compared for example to parallel ICA, that it computes the temporal features of EEG and the spatial characteristics of fMRI in a single algorithm, outputting joint spatial and temporal components. However, the basic disadvantage is that the algorithm uses averaged data (i.e., ERPs and statistical maps) from each participant instead of raw activity. Moreover, the data comes from a single electrode, in the case of EEG, and a specific contrast, in the case of fMRI. Therefore, there is a certain amount of information that is lost in the process. It would therefore be interesting to be able to perform joint ICA using information from several electrodes, since these carry different information. In fact, Moosmann et al. [2008] performed a simulation in which, among other things, they computed joint ICA modeling 64 voxels and 64 electrodes. However, to our knowledge, this has never been performed on real data or simulating a realistic number of voxels, which would probably imply modeling a number of extra factors such as, for example, the depth of the sources that contribute to the electrophysiological components.

In conclusion, in our study, we have shown how the electrophysiological and hemodynamic responses to performance errors relate to each other by using methods that combine EEG and fMRI data. We have also shown how results obtained with fMRI‐informed dipole modeling and newer techniques that use ICA are compatible, and how the latter can be used to complete the former. Finally, we feel that using methods such as joint ICA [Calhoun et al.,2006,2009], which exploit the advantages of each modality and circumvent their disadvantages, we will achieve a more profound understanding of the spatiotemporal dynamics of neural processing.

Supporting information

Additional Supporting Information may be found in the online version of this article.

Supplementary Figure 1. (A) Section of continuous EEG from a representative subject after scanner and BCG artifact correction. All EEG channels are shown in black and the EOG channel in blue. (B) Temporal course of the 30 independent components during the same section of continuous EEG. Marked in red are components showing ocular artifacts, residual BCG artifacts as well as noise in the Fpz electrode, which were eliminated from the model. The topographies of these components are shown right and left of the image. All components shown in black were kept for back‐projection. (C) Same section of the continuous EEG after ICA‐based artifact rejection. All EEG channels are shown in black and the EOG channel in blue.

Click here for additional data file. (178.7KB, tif)

Supplementary Figure 2. ERPs for the three conditions of interest computed (A) with and (B) without the use of ICA‐based artifact rejection. It can be seen that the use of ICA for artifact rejection reduces the amount of unrelated noise from the data.

Click here for additional data file. (112.1KB, tif)

Supplementary Figure 3. Reconstruction of the fMRI activations in ∼12 ms intervals between ‐74 and 236 ms post‐response.

Click here for additional data file. (1.5MB, tif)

Acknowledgements

The authors thank Kerstin Moehring and IlonaWiedenhoeft for their assistance with recordings, and Srinivas Rachakonda for his help with data analysis. They also thank the staff of the Center for Advanced Imaging, Magdeburg, for their continuous support.

REFERENCES

  1. Allen PJ, Josephs O, Turner R ( 2000): A Method for Removing Imaging Artifact from Continuous EEG Recorded during Functional MRI. NeuroImage 12: 230–239. [DOI] [PubMed] [Google Scholar]
  2. Babiloni F, Mattia D, Babiloni C, Astolfi L, Salinari S, Basilisco A, Rossini PM, Marciani MG, Cincotti F ( 2004): Multimodal integration of EEG, MEG and fMRI data for the solution of the neuroimage puzzle. Magn Reson Imag 22: 1471–1476. [DOI] [PubMed] [Google Scholar]
  3. Bandettini PA. 2006. Functional Magnetic Resonance Imaging In: Senior C, Russell T, Gazzaniga MS, editors. Methods in Mind. Cambridge, MA: MIT Press; pp. 193–235. [Google Scholar]
  4. Bell AJ, Sejnowski TJ ( 1995): An information‐maximization approach to blind separation and blind deconvolution. Neural Comput 7: 1129–1159. [DOI] [PubMed] [Google Scholar]
  5. Bénar C‐G, Schön D, Grimault S, Nazarian B, Burle B, Roth M, Badier J‐M, Marquis P, Liegeois‐Chauvel C, Anton J‐L ( 2007): Single‐trial analysis of oddball event‐related potentials in simultaneous EEG‐fMRI. Hum Brain Mapp 28: 602–613. [DOI] [PubMed] [Google Scholar]
  6. Braver TS, Barch DM, Gray JR, Molfese DL, Snyder A ( 2001): Anterior cingulate cortex and response conflict: effects of frequency, inhibition and errors. Cereb Cortex 11: 825–836. [DOI] [PubMed] [Google Scholar]
  7. Calhoun VD, Adali T, Pearlson GD, Kiehl KA ( 2006): Neuronal chronometry of target detection: Fusion of hemodynamic and event‐related potential data. NeuroImage 30: 544–553. [DOI] [PubMed] [Google Scholar]
  8. Calhoun VD, Liu J, Adali T. ( 2009): A review of group ICA for fMRI data and ICA for joint inference of imaging, genetic, and ERP data. NeuroImage 45( 1 Suppl 1 ): S163–S172. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Carter CS, Braver TS, Barch DM, Botvinick MM, Noll D, Cohen JD ( 1998): Anterior cingulate cortex, error detection, and the online monitoring of performance. Science 280: 747–749. [DOI] [PubMed] [Google Scholar]
  10. Correa NM, Eichele T, Adali T, Li Y‐O, Calhoun VD ( 2010): Multi‐set canonical correlation analysis for the fusion of concurrent single trial ERP and functional MRI. NeuroImage 50: 1438–1445. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Debener S, Ullsperger M, Siegel M, Fiehler K, von Cramon DY, Engel AK ( 2005): Trial‐by‐trial coupling of concurrent electroencephalogram and functional magnetic resonance imaging identifies the dynamics of performance monitoring. J Neurosci 25: 11730–11737. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Debener S, Ullsperger M, Siegel M, Engel AK ( 2006): Single‐trial EEG‐fMRI reveals the dynamics of cognitive function. Trends Cogn Sci 10: 558–563. [DOI] [PubMed] [Google Scholar]
  13. Debener S, Strobel A, Sorger B, Peters J, Kranczioch C, Engel AK, Goebel R ( 2007): Improved quality of auditory event‐related potentials recorded simultaneously with 3‐T fMRI: Removal of the ballistocardiogram artefact. NeuroImage 34: 587–597. [DOI] [PubMed] [Google Scholar]
  14. Dehaene S, Posner MI, Tucker DM ( 1994): Localization of a neural system for error detection and compensation. Psychol Sci 5: 303–305. [Google Scholar]
  15. Delorme A, Makeig S ( 2004): EEGLAB: An open source toolbox for analysis of single‐trial EEG dynamics including independent component analysis. J Neurosci Methods 134: 9–21. [DOI] [PubMed] [Google Scholar]
  16. Ehlis A‐C, Herrmann MJ, Bernhard A, Fallgater AJ ( 2005): Monitoring of internal and external error signals. J Psychophysiol 19: 263–269. [Google Scholar]
  17. Eichele T, Specht K, Moosmann M, Jongsma MLA, Quian Quiroga R, Nordby H, Hugdahl K ( 2005): Assessing the spatiotemporal evolution of neuronal activation with single‐trial event‐related potentials and functional MRI. Proc Natl Acad Sci USA 102: 17798–17803. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Eichele T, Calhoun VD, Moosmann M, Specht K, Jongsma MLA, Quian Quiroga R, Nordby H, Hugdahl K ( 2008): Unmixing concurrent EEG‐fMRI with parallel independent component analysis. Int J Psychophysiol 67: 222–234. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Eichele T, Calhoun VD, Debener S ( 2009): Mining EEG‐fMRI using independent component analysis. Int J Psychophysiol 73: 53–61. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Eriksen CW, Eriksen BA ( 1974): Effects of noise letters upon the identification of a target letter in a nonsearch task. Percept Psychophys 16: 143–149. [Google Scholar]
  21. Falkenstein M, Hohnsbein J, Hoormann J, Blanke L. 1990. Effects of errors in choice reaction task on the ERP under focused and divided attention In: Brunia CHM, Gaillard AWK, Kok A, editors. Psychophysiological Brain Research. Tillburg, Netherlands: University Press; pp. 192–195. [Google Scholar]
  22. Falkenstein M, Hohnsbein J, Hoormann J, Blanke L ( 1991): Effects of crossmodal divided attention on late ERP components. II. Error processing in choice reaction tasks. Electroencephalogr Clin Neurophysiol 78: 447–455. [DOI] [PubMed] [Google Scholar]
  23. Friston KJ, Ashburner J, Frith CD, Poline JB, Heather JD, Frackowiak RSJ ( 1995): Spatial registration and normalization of images. Hum Brain Mapp 3: 165–189. [Google Scholar]
  24. Garavan H, Ross TJ, Murphy K, Roche RAP, Stein EA ( 2002): Dissociable executive functions in the dynamic control of behavior: Inhibition, error detection, and correction. NeuroImage 17: 1820–1829. [DOI] [PubMed] [Google Scholar]
  25. Gehring WJ, Goss B, Coles MGH, Meyer DE, Donchin E ( 1993): A neural system for error detection and compensation. Psychol Sci 4: 385–390. [Google Scholar]
  26. Gentsch A, Ullsperger P, Ullsperger M ( 2009): Dissociable medial frontal negativities from a common monitoring system for self‐ and externally caused failure of goal achievement. NeuroImage 47: 2023–2030. [DOI] [PubMed] [Google Scholar]
  27. Goldman RI, Wei C‐Y, Philiastides MG, Gerson AD, Friedman D, Brown TR, Sajda P ( 2009): Single‐trial discrimination for integrating simultaneous EEG and fMRI: Identifying cortical areas contributing to trial‐to‐trial variability in the auditory oddball task. NeuroImage 47: 136–147. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Gonçalves SI, Pouwels PJW, Kuijer JPA, Heethaar RM, de Munck JC ( 2007): Artifact removal in co‐registered EEG/fMRI by selective average subtraction. Clin Neurophysiol 118: 2437–2450. [DOI] [PubMed] [Google Scholar]
  29. Hajcak G, McDonald N, Simons RF ( 2004): Error‐related psychophysiology and negative affect. Brain Cogn 56: 189–197. [DOI] [PubMed] [Google Scholar]
  30. Hajcak G, Moser JS, Yeung N, Simons RF ( 2005): On the ERN and the significance of errors. Psychophysiology 42: 151–160. [DOI] [PubMed] [Google Scholar]
  31. Heinze HJ, Mangun GR, Burchert W, Hinrichs H, Scholz M, Münte TF, Gös A, Scherg M, Johannes S, Hundeshagen H, Gazzaniga MS, Hillyard SA. ( 1994): Combined spatial and temporal imaging of brain activity during visual selective attention in humans. Nature 372: 543–546. [DOI] [PubMed] [Google Scholar]
  32. Herrmann MJ, Römmler J, Ehlis AC, Heidrich A, Fallgatter AJ ( 2004): Source localization (LORETA) of the error‐related‐negativity (ERN/Ne) and positivity (Pe). Brain Res Cogn Brain Res 20: 294–299. [DOI] [PubMed] [Google Scholar]
  33. Hester R, Fassbender C, Garavan H ( 2004): Individual differences in error processing: A review and reanalysis of three event‐related fMRI studies using the GO/NOGO task. Cereb Cortex 14: 986–994. [DOI] [PubMed] [Google Scholar]
  34. Hester R, Madeley J, Murphy K, Mattingley JB ( 2009): Learning from errors: Error‐related neural activity predicts improvements in future inhibitory control performance. J Neurosci 29: 7158–7165. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Holroyd CB, Coles MGH ( 2002): The neural basis of human error processing: Reinforcement learning, dopamine, and the error‐related negativity. Psychol Rev 109: 679–709. [DOI] [PubMed] [Google Scholar]
  36. Holroyd CB, Nieuwenhuis S, Yeung N, Nystrom L, Mars RB, Coles MGH, Cohen JD ( 2004): Dorsal anterior cingulate cortex shows fMRI response to internal and external error signals. Nat Neurosci 7: 497–498. [DOI] [PubMed] [Google Scholar]
  37. Keil J, Weisz N, Paul‐Jordanov I, Wienbruch C ( 2010): Localization of the magnetic equivalent of the ERN and induced oscillatory brain activity. NeuroImage 51: 404–411. [DOI] [PubMed] [Google Scholar]
  38. Kiehl KA, Liddle PF, Hopfinger JB ( 2000): Error processing and the rostral anterior cingulate: An event‐related fMRI study. Psychophysiology 37: 216–223. [PubMed] [Google Scholar]
  39. Kim KH, Yoon HW, Park HW ( 2004): Improved ballistocardiac artifact removal from the electroencephalogram recorded in fMRI. J Neurosci Methods 135: 193–203. [DOI] [PubMed] [Google Scholar]
  40. Klein TA, Endrass T, Kathmann N, Neumann J, von Cramon DY, Ullsperger M ( 2007): Neural correlates of error awareness. NeuroImage 34: 1774–1781. [DOI] [PubMed] [Google Scholar]
  41. Koskinen M, Vartiainen N ( 2009): Removal of imaging artifacts in EEG during simultaneous EEG/fMRI recording: Reconstruction of a high‐precision artifact template. NeuroImage 46: 160–167. [DOI] [PubMed] [Google Scholar]
  42. Laufs H, Kleinschmidt A, Beyerle A, Eger E, Salek‐Haddadi A, Preibisch C, Krakow K ( 2003): EEG‐correlated fMRI of human alpha activity. NeuroImage 19: 1463–1476. [DOI] [PubMed] [Google Scholar]
  43. Leclercq Y, Balteau E, Dang‐Vu T, Schabus M, Luxen A, Maquet P, Phillips C ( 2009): Rejection of pulse related artefact (PRA) from continuous electroencephalographic (EEG) time series recorded during functional magnetic resonance imaging (fMRI) using constraint independent component analysis (cICA). NeuroImage 44: 679–691. [DOI] [PubMed] [Google Scholar]
  44. Li C‐SR, Yan P, Chao HH‐A, Sinha R, Paliwal P, Constable RT, Zhang S, Lee T‐W ( 2008): Error‐specific medial cortical and subcortical activity during the stop signal task: A functional magnetic resonance imaging study. Neuroscience 155: 1142–1151. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Logothetis NK, Pauls J, Augath M, Trinath T, Oeltermann A ( 2001): Neurophysiological investigation of the basis of the fMRI signal. Nature 412: 150–157. [DOI] [PubMed] [Google Scholar]
  46. Luu P, Tucker DM, Derryberry D, Reed M, Poulsen C ( 2003): Electrophysiological responses to errors and feedback in the process of action regulation. Psychol Sci 14: 47–53. [DOI] [PubMed] [Google Scholar]
  47. Magno E, Foxe JJ, Molholm S, Robertson IH, Garavan H ( 2006): The anterior cingulate and error avoidance. J Neurosci 26: 4769–4773. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Makeig S, Bell AJ, Jung T‐P, Sejnowski TJ. 1996. Independent component analysis of electroencephalographic data In: Touretzky D, Mozer M, Hasselmo M, editors. Advances in Neural Information Processing Systems 8. Cambridge, MA: MIT Press; pp. 145–151. [Google Scholar]
  49. Martínez‐Montes E, Valdés‐Sosa PA, Miwakeichi F, Goldman RI, Cohen MS ( 2004): Concurrent EEG/fMRI analysis by multiway partial least squares. NeuroImage 22: 1023–1034. [DOI] [PubMed] [Google Scholar]
  50. Mathewson KJ, Dywan J, Segalowitz SJ ( 2005): Brain bases of error‐related ERPs as influenced by age and task. Biol Psychol 70: 88–104. [DOI] [PubMed] [Google Scholar]
  51. Menon V, Adleman NE, White CD, Glover GH, Reiss AL ( 2001): Error‐related brain activation during a Go/NoGo response inhibition task. Hum Brain Mapp 12: 131–143. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Moosmann M, Eichele T, Nordby H, Hugdahl K, Calhoun VD ( 2008): Joint independent component analysis for simultaneous EEG‐fMRI: Principle and simulation. Int J Psychophysiol 67: 212–221. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Mulert C, Seifert C, Leicht G, Kirsch V, Ertl M, Karch S, Moosmann M, Lutz J, Möller H‐J, Hegerl U, Pogarell O, Jöger L. ( 2008): Single‐trial coupling of EEG and fMRI reveals the involvement of early anterior cingulate cortex activation in effortful decision making. NeuroImage 42: 158–168. [DOI] [PubMed] [Google Scholar]
  54. Müller SV, Möller J, Rodriguez‐Fornells A, Münte TF ( 2005): Brain potentials related to self‐generated and external information used for performance monitoring. Clin Neurophysiol 116: 63–74. [DOI] [PubMed] [Google Scholar]
  55. Mullinger K, Debener S, Coxon R, Bowtell R ( 2008): Effects of simultaneous EEG recording on MRI data quality at 1.5, 3 and 7 Tesla. Int J Psychophysiol 67: 178–188. [DOI] [PubMed] [Google Scholar]
  56. Niazy RK, Beckmann CF, Iannetti GD, Brady JM, Smith SM ( 2005): Removal of FMRI environment artifacts from EEG data using optimal basis sets. NeuroImage 28: 720–737. [DOI] [PubMed] [Google Scholar]
  57. Niessing J, Ebisch B, Schmidt KE, Niessing M, Singer W, Galuske RAW ( 2005): Hemodynamic signals correlate tightly with synchronized gamma oscillations. Science 309: 948–951. [DOI] [PubMed] [Google Scholar]
  58. Nieuwenhuis S, Ridderinkhof KR, Blom J, Band GPH, Kok A ( 2001): Error‐related brain potentials are differentially related to awareness of response errors: Evidence from an antisaccade task. Psychophysiology 38: 752–760. [PubMed] [Google Scholar]
  59. Overbeek TJM, Nieuwenhuis S, Ridderinkhof KR ( 2005): Dissociable components of error processing. J Psychophysiol 19: 319–329. [Google Scholar]
  60. Plis SM, Calhoun VD, Weisend MP, Eichele T, Lane T ( 2010): MEG and fMRI fusion for non‐linear estimation of neural and BOLD signal changes. Front Neuroinform 4: 114. [DOI] [PMC free article] [PubMed] [Google Scholar]
  61. Ridderinkhof KR, Ullsperger M, Crone EA, Nieuwenhuis S ( 2004): The role of the medial frontal cortex in cognitive control. Science 306: 443–447. [DOI] [PubMed] [Google Scholar]
  62. Ridderinkhof KR, Ramautar JR, Wijnen JG ( 2009): To PE or not to PE: A P3‐like ERP component reflecting the processing of response errors. Psychophysiology 46: 531–538. [DOI] [PubMed] [Google Scholar]
  63. Rippon G. 2006. Electroencephalography In: Senior C, Russell T, Gazzaniga MS, editors. Methods in Mind. Cambridge, MA: MIT Press; pp. 237–262. [Google Scholar]
  64. Thees S, Blankenburg F, Taskin B, Curio G, Villringer A ( 2003): Dipole source localization and fMRI of simultaneously recorded data applied to somatosensory categorization. NeuroImage 18: 707–719. [DOI] [PubMed] [Google Scholar]
  65. Tsukamoto T, Kotani Y, Ohgami Y, Omura K, Inoue Y, Aihara Y ( 2006): Activation of insular cortex and subcortical regions related to feedback stimuli in a time estimation task: An fMRI study. Neurosci Lett 399: 39–44. [DOI] [PubMed] [Google Scholar]
  66. Ullsperger M, von Cramon DY ( 2001): Subprocesses of performance monitoring: A dissociation of error processing and response competition revealed by event‐related fMRI and ERPs. NeuroImage 14: 1387–1401. [DOI] [PubMed] [Google Scholar]
  67. Ullsperger M, von Cramon DY ( 2003): Error monitoring using external feedback: Specific roles of the habenular complex, the reward system, and the cingulate motor area revealed by functional magnetic resonance imaging. J Neurosci 23: 4308–4314. [DOI] [PMC free article] [PubMed] [Google Scholar]
  68. van Veen V, Carter CS ( 2002): The anterior cingulate as a conflict monitor: fMRI and ERP studies. Physiol Behav 77: 477–482. [DOI] [PubMed] [Google Scholar]
  69. Warbrick T, Bagshaw AP ( 2008): Scanning strategies for simultaneous EEG‐fMRI evoked potential studies at 3 T. Int J Psychophysiol 67: 169–177. [DOI] [PubMed] [Google Scholar]
  70. Warbrick T, Mobascher A, Brinkmeyer J, Musso F, Richter N, Stoecker T, Fink GR, Shah NJ, Winterer G ( 2009): Single‐trial P3 amplitude and latency informed event‐related fMRI models yield different BOLD response patterns to a target detection task. NeuroImage 47: 1532–1544. [DOI] [PubMed] [Google Scholar]
  71. Wax M, Kailath T ( 1985): Detection of signals by information theoretic criteria. IEEE Trans Acoust Speech Signal Process 33: 387–392. [Google Scholar]
  72. Wiswede D, Münte TF, Goschke T, Rüsseler J ( 2009): Modulation of the error‐related negativity by induction of short‐term negative affect. Neuropsychologia 47: 83–90. [DOI] [PubMed] [Google Scholar]
  73. Yeung N, Botvinick MM, Cohen JD ( 2004): The neural basis of error detection: Conflict monitoring and the error‐related negativity. Psychol Rev 111: 931–959. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Additional Supporting Information may be found in the online version of this article.

Supplementary Figure 1. (A) Section of continuous EEG from a representative subject after scanner and BCG artifact correction. All EEG channels are shown in black and the EOG channel in blue. (B) Temporal course of the 30 independent components during the same section of continuous EEG. Marked in red are components showing ocular artifacts, residual BCG artifacts as well as noise in the Fpz electrode, which were eliminated from the model. The topographies of these components are shown right and left of the image. All components shown in black were kept for back‐projection. (C) Same section of the continuous EEG after ICA‐based artifact rejection. All EEG channels are shown in black and the EOG channel in blue.

Click here for additional data file. (178.7KB, tif)

Supplementary Figure 2. ERPs for the three conditions of interest computed (A) with and (B) without the use of ICA‐based artifact rejection. It can be seen that the use of ICA for artifact rejection reduces the amount of unrelated noise from the data.

Click here for additional data file. (112.1KB, tif)

Supplementary Figure 3. Reconstruction of the fMRI activations in ∼12 ms intervals between ‐74 and 236 ms post‐response.

Click here for additional data file. (1.5MB, tif)

Articles from Human Brain Mapping are provided here courtesy of Wiley

RESOURCES