Simultaneous EEG-fMRI Reveals a Temporal Cascade of Task-Related and Default-Mode Activations During a Simple Target Detection Task

Abstract

Focused attention continuously and inevitably fluctuates, and to completely understand the mechanisms responsible for these modulations it is necessary to localize the brain regions involved. During a simple visual oddball task, neural responses measured by electroencephalography (EEG) modulate primarily with attention, but source localization of the correlates is a challenge. In this study we use single-trial analysis of simultaneously-acquired scalp EEG and functional magnetic resonance image (fMRI) data to investigate the blood oxygen level dependent (BOLD) correlates of modulations in task-related attention, and we unravel the temporal cascade of these transient activations. We hypothesize that activity in brain regions associated with various task-related cognitive processes modulates with attention, and that their involvements occur transiently in a specific order. We analyze the fMRI BOLD signal by first regressing out the variance linked to observed stimulus and behavioral events. We then correlate the residual variance with the trial-to-trial variation of EEG discriminating components for identical stimuli, estimated at a sequence of times during a trial. Post-stimulus and early in the trial, we find activations in right-lateralized frontal regions and lateral occipital cortex, areas that are often linked to task-dependent processes, such as attentional orienting, and decision certainty. After the behavioral response we see correlates in areas often associated with the default-mode network and introspective processing, including precuneus, angular gyri, and posterior cingulate cortex. Our results demonstrate that during simple tasks both task-dependent and default-mode networks are transiently engaged, with a distinct temporal ordering and millisecond timescale.

1 Introduction

Internal attentional brain states are challenging to study because their fluctuations are not always event related and can dynamically ebb and flow at multiple timescales. Traditional blood oxygen level dependent (BOLD) functional magnetic resonance imaging (fMRI) enables observation of the average response evoked by stimuli and/or behavioral response. However, task engagement inevitably fluctuates on a single-trial basis throughout the run of an experiment. Even simple target-detection perceptual decisions involve a complex cascade of neural processes including stimulus detection, target recognition/rejection, motor planning, and behavioral response, all of which are associated with evoked responses that vary on a single-trial basis (Philiastides et al., 2007). Many of these processes are time locked to stimulus onset, but others are more closely locked to the behavioral response (Gerson et al., 2005).

The high temporal resolution of scalp EEG allows observation of dynamic neural processes, but activity in deep subcortical structures is difficult to detect. Furthermore, source localization is essentially an ill-posed problem, with skull and cerebral spinal fluid (CSF) impedance resulting in poor spatial resolution. On the other hand, BOLD fMRI can localize both superficial and deep sources of activity with mm-scale resolution, but its temporal resolution is limited due to the slow nature of the BOLD response and the low sampling rate required for acquisition of whole-brain fMRI data.

Perhaps the most well-studied evoked response to a task-related sensory stimuli is the P3 (also called P300), which peaks at approximately 450 ms post-stimulus (Key et al., 2005; Linden, 2005; Polich, 2007). The P3, which is typically measured via electroencephalography (EEG), is known to have amplitude that is modulated by the endogenous state of the subject (Key et al., 2005; Polich, 2007). Extensive evidence from intracranial, lesion, and EEG-fMRI studies shows this response to be generated by a widespread network that includes frontal and temporal-parietal areas (Linden, 2005; Polich, 2007). Frontal activity has been associated with the earlier novelty-related P3a subcomponent, and temporalparietal activity with the later task-related P3b subcomponent, consistent with their EEG scalp topographies (Polich, 2007).

Endogenous brain states that have been linked to inattention to sensory stimuli are often referred to as “resting states.” Both functional connectivity analysis and independent component analysis (ICA) of fMRI data have identified many consistent resting-state networks (De Luca et al., 2006; Fox et al., 2007). These networks are characterized by “infra-slow” fluctuations on the order of 0.01–0.1 Hz (Palva and Palva, 2012) and can be observed by recording fMRI on undirected subjects while they lie in the scanner doing no specific task. The most extensively studied of these is the default-mode network (DMN), a co-activation of the posterior cingulate cortex, medial prefrontal cortex, and angular gyri that was originally defined to represent a baseline brain state (Raichle et al., 2001). It is commonly associated with self-monitoring, auto-biographical, and social functions, reflecting attention to the internal world (Bressler and Menon, 2010). However, recent work has uncovered evidence for a more active role of the DMN related to task performance (Eichele et al., 2008), including its deactivation by cognitive load during task engagement (Esposito et al., 2006).

In this paper we used simultaneously-recorded EEG and fMRI during a simple visual oddball task. We investigate the relationship between neural correlates of processing task-relevant sensory stimuli and brain states reflective of inattention to the task and sensory input. We use the single-trial analysis methodology ofGoldman et al. (2009), whereby machine learning methods are used to find a maximally discriminative projection of the EEG data, and the single-trial variability of that projection is used to construct the BOLD fMRI univariate model. Previous studies of the BOLD correlates of single-trial event-related EEG variability have focused mainly on the P3 and only a few other well-known components at selected stimulus-locked latencies, and have often used an arbitrary selection of electrodes (Benar et al., 2007; Warbrick et al., 2009), and many have studied this coupling without regressing out the effect of the externally-observable reaction time variability.Eichele et al. (2005) were one of the first to investigate the spatio-temporal evolution for BOLD correlates of event-related potential (ERP) components spanning the entire trial. Their approach, applied to an auditory oddball paradigm, used ICA to denoise the EEG from which they then selected a specific subset of electrodes to construct single-trial variability regressors for well defined ERPs (e.g. P2, N2 and P3).

Instead of a priori defining ERP components of interest, we use a purely data-driven approach to identify temporally specific, maximally discriminative task-relevant projections of the EEG data. Specifically, our multivariate discrimination improves identification of task-relevant components in low signal-to-noise ratio environments, such as EEG recorded during MRI acquisition. It also enables us to study the BOLD correlates of continuously-evolving components linked to the task (i.e. defined by trial labels). Since we analyze the BOLD signal by first regressing out the variance linked to observed stimulus and behavioral events (Feige et al., 2005; Goldman et al., 2009), these methods also allow us to investigate the BOLD correlates of modulations in task-engagement that are undetectable with traditional methods, and dissociate them from observable behavioral variability. Furthermore, we investigate the correlates of these modulations only for target trials (i.e. identical stimuli), thus ensuring that the trial-to-trial variation in neural processes is reflecting a latent state. Despite making no prior assumptions about functional connectivity between brain regions and without aiming to study functional networks, we find that for this simple target detection task, regions associated with task-dependent and default-mode networks transiently correlate with the trial-to-trial variability of the EEG discriminating components, and they do so on a millisecond timescale with a distinct temporal ordering.

2 Materials and Methods

2.1 Behavioral Paradigm

Seventeen subjects (6 female, mean 27.7 years, range 20–40) participated in three runs each of a visual oddball paradigm. The 375 (125 per run) total stimuli were presented for 200 ms each with a 2–3 s uniformly-distributed variable inter-trial interval (ITI) and probability of target ⅕. The first two stimuli of each run were constrained to be standards. The target and standard stimuli were, respectively, a large red circle and a small green circle on isoluminant gray backgrounds (3.45° and 1.15° visual angles). The larger target stimuli boosted EEG discriminator performance, and did not confound our final results, since we interpreted only the variability within the target class. Because our study focused on task-related attentional states, subjects were asked to respond to target stimuli, using a button press with the right index finger on an MR-compatible button response pad. Stimuli were presented to subjects using E-Prime software (PST, Pittsburgh, PA) and a VisuaStim Digital System (Resonance Technology, Northridge, CA) 600×800 goggle display. All subjects gave informed consent following the protocol of the Columbia University Institutional Review Board.

2.2 Simultaneous EEG and fMRI Data Acquisition

A 3T Philips Achieva MRI scanner (Philips Medical Systems, Bothell, WA) was used to collect functional echo-planar image (EPI) data continuously with 3 mm in-plane resolution and 4 mm slice thickness. We covered the entire brain by obtaining 32 slices of 64×64 voxels using a 2000 ms repetition time (TR) and 25 ms echo time (TE). We also acquired a single-volume high resolution (2×2×2 mm) EPI image and a 1×1×1 mm spoiled gradient recalled (SPGR) image for each subject for purposes of registration.

We simultaneously and continuously recorded EEG using a custom-built MR-compatible EEG system (Goldman et al., 2009; Sajda et al., 2010), with differential amplifier and bipolar EEG cap. The caps are configured with 36 Ag/AgCl electrodes including left and right mastoids, arranged as 43 bipolar pairs (Supplementary Figure 1). Bipolar pair leads are twisted to minimize inductive pickup from the magnetic gradient pulses and subject head motion in the main magnetic field. This oversampling of electrodes ensures data from a complete set of electrodes even in instances when discarding noisy channels is necessary. To enable removal of gradient artifacts in our offline preprocessing, we synchronized the 1-kHz-sampled EEG with the scanner clock by sending a transistor-transistor logic (TTL) pulse to a field-programmable gate array (FPGA) card (National Instruments, Austin, TX) at the start of each of 170 functional image acquisitions. All electrode impedances were kept below 20 kΩ, including 10 kΩ resistors built into each electrode for subject safety. A comprehensive description of the hardware, along with the preprocessing and analysis methods described throughout the remainder of Section 2, can be found inSajda et al. (2010).

2.3 EEG Data Preprocessing

We performed EEG preprocessing offline using Matlab (Mathworks, Natick, MA). In addition to standard EEG artifacts, electrophysiological signals recorded inside the MRI scanner are contaminated with gradient artifacts and ballistocardiogram (BCG) artifacts due to magnetic induction in the EEG wires. First we removed the gradient artifacts by subtracting the mean artifact across all functional volume acquisitions. We then applied a 10 ms median filter to remove any residual spike artifacts. Because we sync the scanner with the EEG we are able to remove the gradient artifact at the commonly-used 1 kHz sampling rate of stand-alone EEG. Next we removed standard EEG artifacts, using the following digital Butterworth filters: 1 Hz high pass to remove DC drift, 60 Hz and 120 Hz notches to remove electrical line noise and its first harmonic, and 100 Hz low pass to remove high frequency artifacts not associated with neurophysiological processes. These filters were applied together in the form of a linear phase finite impulse response (FIR) filter to avoid distortions caused by phase delays.

BCG artifacts are more challenging to remove, since they share frequency content with EEG activity. Currently-existing BCG removal algorithms cause loss of signal power in the underlying EEG, so we performed single-trial classification (described in section 2.4) on the data prior to BCG artifact removal. This is a justified practice because our classifier identifies discriminating components that are likely to be orthogonal to BCG. In order to compute scalp topographies of these discriminating components, BCG artifacts were removed from the continuous gradient-free data using a principal components analysis (PCA) method (Goldman et al., 2009; Sajda et al., 2010). First the data were low-passed at 4 Hz to extract the signal within the frequency range where BCG artifacts are observed, and then the first two principal components were determined. The channel weightings corresponding to those components were projected onto the broadband data and subtracted out. These BCG-free data were then re-referenced from the 43 bipolar channels to the 34-electrode space to calculate scalp topographies of EEG discriminating components.

We investigated both stimulus-locked and response-locked activity because some cognitive processes are more tightly time-locked to stimulus onset and others to behavioral responses (Gerson et al., 2005; Makeig et al., 2004). We extracted 1000 ms stimulus-locked and response-locked epochs (with baseline removal on the 200 ms prior to stimulus) from both the 43-channel gradient-free dataset and the 34-electrode re-referenced dataset. By visual inspection we discarded trials containing motion or blink artifacts, evidenced by sudden high-amplitude deflections, and also those with incorrect responses, identically for both datasets. This left approximately 95% of the target trials remaining. Because the paradigm does not require a behavioral response to standard stimuli, we randomly assigned reaction times (RTs) to standard trials chosen from the probability distribution of target trial RTs (Goldman et al., 2009). This provided a baseline activity for response-locked single-trial EEG analysis, and it did not affect interpretation of results since we only viewed results derived from the variability within the target class.

2.4 Single-trial analysis of EEG

We discriminated target vs. standard trials by applying a linear classifier to the 43-channel EEG signal amplitude, x_τ(t), using the sliding window method of Parra et al. (2002; 2005). This method is described comprehensively inGoldman et al. (2009) andSajda et al. (2010), and an overview is illustrated in Figure 1. We selected a training window of width 50 ms and varied the window center, τ, across the entire epoch (stimulus-locked 0–1000 ms, response-locked −600–400 ms) in overlapping 25 ms increments. For each of these time windows, we used logistic regression on the 50 ms of data to estimate the linear weighting, w_τ, of the EEG sensors resulting in a projection, y_τ(t), that maximally discriminated the conditions.

$$y_{\tau }(t)={{\mathit{w}}_{\tau }}^T{\mathit{x}}_{\tau }(t)$$ (1)

The classifier output is the distance of each trial from the discriminating hyperplane, representing the classifier’s confidence in its prediction based on the training data and the model. We treat this as a surrogate for the neural confidence of the decision, which for our simple target-detection task is primarily dependent upon the subjects’ instantaneous task-engagement.

Figure 1. Method for constructing fMRI regressors from simultaneously-acquired EEG data.

A. For each trial (i), select a training window of EEG data (X_i) with offset τ from the stimulus (or behavioral response). B. Train linear classifier on EEG data within the time window to estimate a set of spatial weights (w) that maximize discrimination of the two conditions (shown using only 2 EEG channels for visualization purposes). C. In addition to traditional event-related average response (ERAR) and reaction time (RT) regressors, construct single-trial EEG variability (STV) regressors by modulating boxcar height with classifier output (y) for each trial. D. This technique is repeated for multiple window offsets spanning the epoch to view temporal progression of discriminating components spanning the trial.

We assessed classifier performance by means of area under the receiver operating characteristic (ROC) curve, denoted A_z, (Green and Swets, 1966) using leave-one-out (LOO) cross validation (Duda et al., 2001). A_z calculations performed while sliding the window across time enabled observation of the temporal progression of task-relevant components and localization of the stimulus- or response-locked time with maximal discrimination between conditions. To obtain a significance threshold for the A_z values, we used a permutation test in which we randomly permuted the trial labels and ran the classifier using the LOO approach. Repetition 1000 times for each subject generated a distribution of A_z values from which we computed the A_z corresponding to p = 0.01.

We generated forward models using the sensor-space EEG data, in order to view the scalp distribution of discriminating activity for each time window, where X_τ is the matrix comprised of all samples within the 50-ms-wide window from τ − 25 ms to τ + 25 ms.

$${\mathit{a}}_{\tau }={\mathit{X}}_{\tau }{\mathit{y}}_{\tau }/{{\mathit{y}}_{\tau }}^T{\mathit{y}}_{\tau }$$ (2)

This forward model is a normalized correlation between the discriminating component and the data X_τ, and it represents the electrical coupling between them (Goldman et al., 2009; Parra et al., 2002, 2005; Sajda et al., 2010). It allows observation of the progression of the components across the scalp over time.

2.5 fMRI Data Preprocessing

Using FSL (Smith et al., 2004), we performed bias-field correction on all images to adjust for artifacts caused by the EEG wires. We then performed slice-timing correction, motion correction, 0.01-Hz high-pass filtering, and 5-mm full width half max (FWHM) spatial smoothing on the functional data. Motion correction provided motion parameters that were later included as confounds in the general linear model (GLM). Functional and structural images were registered to a standard Montreal Neurological Institute (MNI) brain template following brain extraction, and each subject’s registration was checked manually to ensure proper alignment.

2.6 Traditional fMRI Analysis

We first ran a traditional fMRI analysis, using event-related and RT variability regressors in our GLM. The event-related regressors were comprised of boxcar functions with unit amplitude and onset and offset matching that of the stimuli. RT variability was modeled using unit amplitude boxcars with onset at stimulus time and offset at response time, and these were orthogonalized to the event-related regressors. Orthogonalization was implemented using FSL, which utilizes the Gram-Schmidt procedure (Strang, 2003) to decorrelate the RT regressor from all other event-related regressors. All regressors were convolved with the canonical hemodynamic response function (HRF), and temporal derivatives were included as confounds of no interest. An event-related target vs. standards contrast was also constructed. A fixed effects model was used to model activations across runs, and a mixed effects approach used to compute the contrasts across subjects. Statistical image results for these traditional analyses were thresholded at z > 2.3, and clusters were multiple-comparison-corrected at p = 0.05 (Worsley, 2001).

2.7 EEG-based fMRI Analysis

For the single-trial variability (STV) fMRI analysis, we modeled the variability of the neural response using an additional two regressors – one each for targets and standards. These EEG-based regressors were designed with duration 100 ms, centered on the classifier training window. The STV regressor height was modulated using the output y_τ,i of the EEG discriminator for each trial (i). These regressors were convolved with the HRF and orthogonalized with respect to all traditional regressors, with temporal derivatives included as confounds. It was especially important to regress out the RT variability, since RT is known to be negatively correlated with attention (Eason et al., 1969; Weissman et al., 2006), and our aim was to study variability in task-engagement that cannot be detected using an external measure. This entire analysis was run independently for all stimulus-locked and response-locked EEG training windows exceeding a mean A_z value of 0.75, which is a common psychophysical threshold used in signal detection theory and here represents substantial performance of the classifier. To avoid stimulus-type confounds, we focused on within-class variability, using only the target stimuli STV statistical maps in our results interpretation.

We used a randomization method motivated by deBettencourt et al. (2011) and threshold-free cluster enhancement (Smith and Nichols, 2009) to correct for multiple comparisons, by running a complete STV fMRI analysis after permuting the y_τ=450ms values randomly within each class. 100 permutations were run for each subject. We carried these randomization results through to the group level and thresholded the statistical maps at per-voxel p = 0.005. For each cluster in the resulting null distribution, we summed the negative logarithm of the p-value over each voxel in the cluster. This provided a score for each cluster that, while highly dependent on cluster size, also accounted for some variation in the magnitude of the z-scores within the cluster. We determined a threshold based on the 99^th percentile of these null-distribution cluster scores, or a p-value of 0.01.

We took extra care to investigate the possibility of pulsation artifacts confounding our results, by constructing fMRI regressors from BCG-artifact pulse timing. This timing was determined using a peak detection algorithm on temporal lobe EEG channels, which are the channels most strongly contaminated with BCG artifact. We fit this regressor to the functional data, and thresholded the resulting maps at an uncorrected p = 0.005. We do not report any results that overlap with these pulse correlates.

2.8 fMRI Analysis for Localizing the DMN

Because our single-trial findings included correlates consistent with DMN, we performed ICA on the functional image data to locate the DMN, which is an established method used to study resting-state functional networks not associated with any task (De Luca et al., 2006; Fox et al., 2007). To prevent investigator bias, we used a template-matching algorithm to automatically determine the DMN component for each subject (template obtained from the Neurosynth database of Yarkoni et al., 2011), and we found the group mean DMN component in MNI space. Manual DMN component selection was consistent with our algorithm output.

3 Results

All subjects responded with high accuracy and speed. 98.4% ± 3.1% of targets were correctly detected, with 397.2 ± 38.9 ms RT.

3.1 EEG Analyses

Traditional stimulus-locked event-related potentials (ERPs) displayed a strong visual P2 peak over frontal and posterior sites and a prominent P3 over central sites (Figure 2, left), consistent with previous literature (Hopfinger and West, 2006; Makeig et al., 1999). N1 and N2 responses were also visible. The response-locked ERPs showed a double peak that was most pronounced at central and posterior sites (Figure 2, right), and which was highly replicable across subjects. This matches the post-motor thetaband synchronization described byMakeig et al. (2004).

Figure 2. Stimulus-locked and response-locked ERPs recorded at the Pz electrode.

Our definition of early, middle, and late window ranges (see main text for discussion) are indicated with shading.

We were able to discriminate target vs. standard EEG trials with highly significant accuracy (Figure 3). We surpassed the corrected p = 0.01 value (A_z = 0.66) for all consecutive stimulus-locked training windows from 150–750 ms and for all analyzed response-locked windows centered at −250 ms or later. Maximum discrimination of A_z = 0.86 ± 0.04 was reached at 325 ms for the stimulus-locked classification, with a broad temporal peak. Response-locked classification yielded even higher A_z values, with a double peak shortly after reaction time. A maximum A_z of 0.93 ± 0.04 was reached at 25 ms following RT, and 0.91 ± 0.07 A_z was reached at 150 ms. The window ranges exceeding 0.75 mean A_z (and thus subsequently included in the EEG-based fMRI analysis) were 175 to 600 ms for stimulus-locked and −175 to 375 ms for response-locked discrimination.

Figure 3. Group mean averages and standard errors of single-trial EEG discrimination performance.

Results of both the stimulus-locked (blue) and response-locked (green) analyses are shown, aligned by mean RT. Since we are interested in the BOLD correlates of single-trial EEG variability, we only consider EEG components with discrimination that is both significant (Az > 0.66, p < 0.01) and substantial (Az > 0.75). Windows resulting in significant positive and negative BOLD correlations are indicated with magenta diamonds and orange circles, respectively. Early, middle, and late windows (as grouped for discussion) are indicated with shading.

Discriminator output was significantly (p < 0.01) negatively correlated with RT for multiple stimulus- and response-locked windows. This result demonstrated the need to orthogonalize our STV fMRI regressors to RT-variability regressors, being that our aim was to study residual variance not observable with behavioral response.

The grand mean forward models displayed an early frontal positivity that was strongest at the 250 ms stimulus-locked window. They also revealed a central negativity around 325 ms, which occurred prior to behavioral responses and corresponded to the middle of the broad stimulus-locked discrimination peak. A posterior positivity first appeared near the RT and slowly became more centrally distributed, reaching its highest strength late in the trial. The progression of these discriminating components was consistent with the visual target detection results ofGerson et al. (2005). We show EEG topographies alongside their corresponding BOLD activations in Figures 4, 5, and 6.

Figure 4. Significant clusters correlating with EEG single-trial variability early in the trial.

Shown for stimulus-locked (SL, top) and response-locked (RL, bottom) windows. Corresponding EEG discriminant component scalp projections (forward models) are also shown.

Figure 5. Significant clusters correlating with EEG single-trial variability in middle windows.

These activations occurred near the behavioral response time and in the range of the P3, shown for stimulus-locked (SL, top) and response-locked (RL, bottom) windows. Corresponding EEG discriminant component scalp projections (forward models) are also shown.

Figure 6. Significant clusters correlating with EEG single-trial variability late in the trial.

These activations occurred after the subject has made his/her response, for stimulus-locked (SL, top) and response-locked (RL, bottom) windows. Corresponding EEG discriminant component scalp projections (forward models) are also shown.

3.2 Traditional fMRI Analysis

Event-related BOLD responses to target stimuli were present in multiple brain areas, including bilateral supramarginal gyri, insular cortices, cingulate cortices, angular gyri, precentral gyri, thalamus, cerebellum, and brainstem. Postcentral activations were strong and left-lateralized, consistent with right-handed button press. These activations matched previous visual oddball paradigm results (Laurens et al., 2005; Stevens et al., 2000; Warbrick et al., 2009), and were even stronger and more widespread in the traditional event-related target vs. standard contrast (Supplementary Figure 2). This contrast additionally resulted in right-lateralized clusters in precuneus and middle frontal gyrus. RT-variability statistical maps (Supplementary Figure 3) showed activations in anterior cingulate cortex (ACC), supplementary motor area, and right precentral gyrus.

3.3 Single-trial EEG Variability fMRI Analysis

Our randomization method, which we used to correct for multiple comparisons, determined a p = 0.01 threshold which corresponded to a 60-voxel cluster size (24 voxels for p = 0.05). At the p = 0.01 threshold, the EEG-derived regressors resulted in significant group-level positive and negative correlations for multiple stimulus-locked and response-locked EEG training windows, indicated with diamonds and circles on Figure 3. Interestingly, all significant clusters correlating with single-trial variability in windows prior to the RT were negatively correlated with classifier output. Near the mean RT, we saw a reversal to positive correlations, and this effect lasted approximately 100 ms, after which all significant clusters were negative.

Since our analysis separately identifies EEG discriminating components at different time windows relative to the stimuli and responses, we are able to study the temporal cascade of neural involvement related to modulations of internal attentional states. For simplification and clarity in the following discussion, we divided our results into three stages (indicated in Table 1 and Figures 2 and 3): early (prior to behavioral response), middle (at or near RT), and late (after the response while the subject is waiting for the subsequent stimulus). We based these temporal ranges on timing of ERP components at the Pz electrode (Figure 2) such that the early stage corresponds to the P2 and N2, the middle stage corresponds to the P3, and our late windows are those that occur after the P3.

Table 1. Significant clusters resulting from the stimulus-locked and response-locked EEG-based fMRI analyses.

Dotted lines denote cutoffs between early, middle, and late windows, as per our discussion.

STIMULUS-LOCKED EEG STV fMRI ANALYSIS
window	Az	+/−	#vox	max z	pcluster	MNIx	MNIy	MNIz	hem	region
175ms	0.80	neg	66	3.29	0.0094	32	0	38	R	precentral gyrus, white matter
225ms	0.85	neg	69	3.44	0.0083	30	−70	52	R	superior lateral occipital cortex
250ms	0.85	neg	82	3.25	0.0062	48	32	22	R	middle frontal gyrus, dorsolateral prefrontal cortex
300ms	0.86	neg	89	3.28	0.0052	52	10	28	R	middle frontal gyrus, inferior frontal precentral gyrus precentral gyrus gyrus,
375ms	0.84	pos	192	3.65	0.0008	0	−78	56	L/R	superior sagittal sinus, L precuneus
400ms	0.85	pos	340	3.70	0.0005	−2	−66	62	L/R	superior sagittal sinus
		pos	70	3.20	0.0086	−6	10	68	L	superior frontal gyrus
500ms	0.81	neg	63	3.33	0.0094	8	48	46	R	frontal pole, superior frontal gyrus
525ms	0.79	neg	293	3.59	0.0005	40	6	44	R	middle frontal gyrus
		neg	212	3.29	0.0005	2	46	40	L/R	superior frontal gyrus, frontal pole paracingulate gyrus, R
		neg	109	3.74	0.0029	−52	−66	34	L	superior lateral occipital cortex, angular gyrus
		neg	90	3.28	0.0047	56	−66	32	R	superior lateral occipital cortex, angular gyrus
550ms	0.78	neg	87	3.23	0.0057	0	48	32	L/R	superior frontal gyrus, paracingulate gyrus
RESPONSE-LOCKED EEG STV fMRI ANALYSIS
window	Az	+/−	#vox	max z	pcluster	MNIx	MNIy	MNIz	hem	region
−175ms	0.75	neg	83	3.34	0.0062	8	32	52	R	superior frontal gyrus
−125ms	0.82	neg	63	2.95	0.0104	40	26	26	R	middle frontal gyrus
−100ms	0.84	neg	65	3.33	0.0094	52	−60	30	R	superior lateral occipital cortex, angular gyrus
−75ms	0.86	neg	88	3.25	0.0052	60	−44	10	R	middle temporal gyrus, auditory cortex
−25ms	0.88	neg	86	3.32	0.0049	−42	−18	36	L	postcentral gyrus, precentral gyrus
0ms	0.90	pos	59	3.46	0.0099	16	−58	42	R	white matter, precuneus
50ms	0.93	pos	80	3.33	0.0070	2	−48	4	L/R	posterior cingulate gyrus
		pos	79	3.54	0.0060	20	−48	−48	R	cerebellum VIIIb
		pos	72	3.25	0.0078	40	−20	−18	R	temporal fusiform cortex
		pos	69	3.12	0.0091	4	−70	−18	L/R	cerebellar vermis
75ms	0.91	pos	82	3.39	0.0065	4	−70	−18	L/R	cerebellar vermis
200ms	0.89	neg	184	3.36	0.0010	46	−24	44	R	postcentral gyrus, white matter, hand sensorimotor
		neg	71	3.17	0.0083	−38	−16	36	L	precentral gyrus, white matter

Table 1 contains a complete list of activations exceeding multiple-comparison-corrected p = 0.01 with their corresponding EEG windows.

The early stimulus-locked EEG discriminating components revealed negatively-correlated activations in the precentral gyrus, superior lateral occipital cortex (LOC), and frontal regions including dorsolateral prefrontal cortex (DLPFC). Components close to the response time (but not locked to response) were correlated positively with superior frontal gyrus (SFG) and areas consistent with draining veins. The later stimulus-locked discriminating components showed additional negative correlations with regions in the left and right angular gyrus/LOC, the SFG/paracingulate gyrus, and the right middle frontal gyrus.

The response-locked EEG components revealed an early negative correlate in superior frontal gyrus, but most of the significant BOLD activations appeared in windows close to the RT. These included negative activations in LOC, middle temporal and frontal gyri, and precentral and postcentral gyri. We saw positive activations in cerebellum, posterior cingulate, and precuneus immediately following response time. Later response-locked components revealed additional negative activations in postcentral and precentral gyri. Supra-threshold negative and positive activations also appeared in white matter for our response-locked analysis.

The late stimulus-locked windows showed regions consistent with the default-mode network. Since the DMN is commonly detected using ICA, we confirmed the location of the subject mean DMN using ICA and template matching on the functional data. Our supra-threshold clusters in the medial frontal gyrus and angular gyri overlapped with this ICA-determined DMN, as did a negative 58-voxel activation in the posterior cingulate that achieved a cluster p-value of 0.0125, with max z = 3.185 at MNI coordinates (10, −48, 34) (Figure 7).

Figure 7. DMN determined independently with EEG single-trial variability and ICA.

525 ms stimulus-locked window EEG single-trial variability (STV) negative correlates in medial frontal gyrus, bilateral angular gyri, and posterior cingulate (in blue, with hue representing p-value in the range 0.005−0.001), overlaid with the mean default-mode network (DMN) as determined using ICA (pink). STV results shown are multiple-comparison corrected at p < 0.05. An additional cluster in right middle frontal gyrus can also be seen.

4 Discussion

Based on the difference ERPs and the scalp distributions and timing of our EEG discriminating components, we believe we are tracking variation related primarily to the P2, N2, and P3 responses (Key et al., 2005), so we discuss our results partly in light of these. More specifically, our topographies are consistent with discrimination of the target-related P3b, frontal P3f, and post-motor potential (Pmp) subcomponents of the P3 (Makeig et al., 1999), as expected for our visual oddball task. Since amplitudes of these components are known to modulate with arousal and attention (Key et al., 2005; Luck et al., 2000; Nieuwenhuis et al., 2005), we use the classifier output (i.e. its decision certainty) to index the subjects’ attentional state (i.e. task engagement) during each trial. Our EEG-based fMRI regressors are thus revealing the BOLD correlates of modulations in task engagement.

Electrophysiological activity and BOLD responses are coupled in a complex way that is not yet completely understood. There is evidence to suggest that the sign of the EEG-BOLD correlate is both a function of space and of frequency, and the sign of the BOLD response itself is affected by the excitation-inhibition balance within local neuronal circuits (Logothetis et al., 2008). It has been shown that low-frequency electrophysiological variations, such as those measured by scalp EEG in the 0–20 Hz range, are negatively correlated with the BOLD response (Goldman et al., 2002; Mukamel et al., 2005; Scheeringa et al., 2011). Additionally, commonly-studied fMRI resting state networks have been shown to correlate with specific frequency bands in magneto-encephalography (MEG) studies (Brookes et al., 2011; de Pascuale et al., 2010) and show some (though lesser) frequency-dependency using EEG-fMRI (Mantini et al., 2007). Since there is still some question about the physiological meaning of directionality of electrophysiological BOLD correlates, the following sections do not attempt to interpret the sign of the correlation between the BOLD signal and the STV regressor.

4.1 Early EEG discriminating components reveal BOLD correlations in task-relevant brain areas

Early EEG windows resulted in right-lateralized activations in brain regions associated with taskrelevant neural processes (Figure 4). These included frontal activations in DLPFC and posterior SFG, which are most commonly associated with working memory and higher-level cognitive processing during demanding tasks (Fuster, 2001; MacDonald et al., 2000), and the superior LOC. These areas have been linked to P3 modulations and are discussed further in Section 4.2, which focuses on components in that time range. To the best of our knowledge, BOLD correlates of variability in the early P2 and N2 ERP components have not been studied in the visual domain; however,Eichele et al. (2005) found correlates of auditory oddball N2 variability in both of these areas at similar latencies. Using a visual target detection task,Novitskiy et al. (2011) found these regions correlated with variability in the even earlier N1 component, but their activations were left-lateralized.

4.2 EEG discriminating components during the P3 and around reaction time reveal a period of transition from externally driven task-related to internally-driven processing

Our middle windows, which occurred close to the behavioral response and in the range of the P3, revealed a superposition of task-related and endogenous attention areas (Figure 5). Similar to the early windows, task-related BOLD correlations were found in right middle frontal gyrus and superior LOC. We also found an activation in right inferior frontal gyrus, an area associated with reorienting to salient behaviorally-relevant stimuli and is a main node in the right-hemisphere ventral attention network (VAN) (Corbetta and Shulman, 2002; Corbetta et al., 2008). Right-hemisphere frontal areas have been shown by a number of recent EEG-fMRI studies to modulate with P3 amplitude during visual (Bledowski et al., 2004a) and auditory (Eichele et al., 2005) tasks, and were reported for the latter case at a similar latency.

Our superior LOC correlate is an expected finding, given its common association with visual attention (Murray and Wojciulik, 2003). However, similar regions have also been shown to couple with EEG components in the P3 time range for auditory tasks.Benar et al. (2007) linked the parieto-occipital junction to P3 modulations during an auditory oddball task. In our previous study that used an analogous auditory oddball paradigm and the same EEG discriminating component methods (Goldman et al., 2009), we also detected a correlate in the right LOC in the P3 range, but that cluster was located slightly inferior to our current finding. These collective findings suggest a supra-modal role for the LOC in attention.

An activation also appeared in right temporal fusiform cortex, an area commonly known to be involved in high-level visual processing.Bledowski et al. (2004b) also found that higher visual areas in inferior temporal gyrus contribute to the visual P3b, and their correlate appeared at similar latency. We also detected right-lateralized activations in auditory cortex. This co-activation of auditory and visual processing areas supports the presence of cross-modal attentional modulations, suggesting that modulations in auditory-directed attention (e.g. listening to scanner noise) affect attention to the visual modality.

As expected for EEG windows close to reaction time, we found BOLD correlates in several regions related to the button press. This included the motor areas of the left precentral and postcentral gyri, which is consistent with a right-handed button press and has previously been associated with P3 coupling in visual (Bledowski et al., 2004a, 2004b; Warbrick et al., 2009) and auditory (Benar et al., 2007) tasks. We also saw activations in the cerebellar vermis, which coordinates and monitors movement (Ghez and Fahn, 1985), and right cerebellum area VIIIb, which plays additional roles in motor control, coordination, and accurate timing of movements, and which has been particularly associated with right-handed finger tapping (Stoodley and Schmahmann, 2009; Stoodley et al., 2012).

The precuneus has been associated with a variety of task-relevant cognitive processes, including attentional orienting, visuo-spatial imagery, coordination of motor behavior, success monitoring, and self-referential thoughts (Cavanna and Trimble, 2006; Taylor et al., 2009).Eichele et al. (2005) found precuneus activations correlated with earlier P2 amplitude modulations, though their study used an auditory task. Our reaction-time precuneus activation supports the idea of a transition from the exogenous attentional state required for the visual perceptual decision to motor coordination required for the button press, followed by a reflection on task performance. Activations associated with other forms of internal thought, including posterior cingulate and SFG, were detected both in middle and late windows, and these are discussed in section 4.3.

BOLD fMRI literature historically has reported activations mainly in gray matter, partially due to the common practice of masking white matter during statistical analyses, but growing evidence supports the presence of white matter BOLD signals. This is particularly true when single-trial variability is modeled (Yarkoni et al., 2009). Here, our latent variability feature, the EEG single-trial discriminating component, revealed BOLD correlates in white matter regions. Since these activations were adjacent to sensorimotor regions, including right postcentral and precentral gyri and the left precentral gyrus, they are most closely related to the button press.

Reports of BOLD activations in draining veins are also rare in the literature, due to the high spatial resolution required to differentiate veins from cortical tissue. However,Bianciardi et al. (2011), using a high-resolution study of visual areas at 7 T, found task-related negative BOLD correlates in large cerebral veins. We found a remarkable 340-voxel cluster that is adjacent to the precuneus but appears to track the superior sagittal sinus, a large cerebral vein that drains widespread areas of cortical tissue. The BOLD signal we detected in cerebral veins is likely due to a transient misbalance between metabolic demands and increased oxygenated blood, and may reflect strongly activated cortices that are functionally but not anatomically overlapping across subjects, but which drain into the same veins. Since the superior sagittal sinus is nearly exclusively responsible for draining the cerebrum (Mattle et al., 1990), this interpretation does not allow speculation regarding particular cortices. An alternative idea for the physiological meaning of this activation is that it reflects a global change in cerebral blood flow. Given the care we took with image registration, pulsation artifact investigation, and multiple comparison correction, and the congruency of our cortical results with the literature, we believe these to be true results that should not be ignored. However, these remain preliminary findings that we are currently validating in new datasets using diffusion tensor imaging and magnetic resonance veinography.

4.3 Late EEG discriminating components reveal default mode activity between subject response and the next stimulus

BOLD correlates of EEG discriminating components late in the trial (Figure 6) appeared in a pattern matching the DMN (Figure 7), supporting MEG evidence for a transient nature of functional networks (de Pascuale et al., 2010). Given that we found an earlier activation in a region associated with the VAN, this supports the hypothesis for complementary roles of the DMN and VAN in directed awareness (Boly et al., 2007; Vanhaudenhuyse et al., 2011), and supports the intracranial EEG finding that transient stimulus-related activations in VAN areas occur earlier than those in DMN (Ossandon et al., 2011).

Our finding is in agreement with a number of recent fMRI and combined EEG-fMRI studies that suggest that the DMN plays an active role in task-related processing. Spontaneous DMN fluctuations have been shown to affect visual task performance (Eichele et al., 2008) and during rest have inversely correlated with the frontal theta rhythm (Scheeringa et al., 2008), which is typically associated with cognitive processing. There is also evidence of transient event-related suppression of DMN regions related to increased cognitive load during a visual N-back task (Esposito et al., 2006; 2009a) and during motion discrimination (Singh and Fawcett, 2008) and auditory oddball tasks (Eichele et al., 2005), though in the latter case corresponding to an earlier component. This converging evidence supports the idea that DMN regions modulate selective attention for optimal allocation of attentional resources.

Because our correlate appears after the response, our finding suggests an active role for the DMN in introspective task-relevant processing, such as an active observation of the behavioral response. Such retrospection is not necessarily overtly conscious so it is challenging to investigate experimentally (Schooler, 2011). We do not believe this post-response DMN correlate is reflecting an anticipatory state while subjects await the next stimulus since ERPs (derived from EEG data that were not highpass-filtered) revealed no effect of contingent negative variation (CNV), a slow ERP component with magnitude dependent upon level of expectation of the following stimulus (Palva and Palva, 2012; Scheibe et al., 2010) (data not shown). Previously reported BOLD correlates of CNV did not include the canonical DMN (Scheibe et al., 2010); combined with our result, this loosely suggests that suppression of DMN is not related to increased anticipation. Further evidence for an introspective state late in the trial is an activation in the paracingulate, which has associations with self awareness and theory of mind, and particularly introspection-related activity during visual tasks (Goldberg et al., 2006).

Our bilateral angular gyri clusters, which are subregions of the intra-parietal lobules (IPL) (Udin et al., 2010), are consistent with many reports of IPL in P3 coupling (Linden, 2005). Similar to our late (525–550 ms relative to stimulus) activations, Bledowski et al. (2004a, 2004b) found bilateral IPL correlated with EEG amplitude variability, reported around 540 ms (in 2004b). Our stimulus-locked correlates appear later than the peak amplitude of the P3, which suggests that the IPL is more closely linked to P3 latency variability than amplitude variability. This is consistent with findings that activity in parietal areas correlates with P3 latency but not with amplitude (Warbrick et al., 2009).

4.4 BOLD coupling with the EEG response is captured by reaction-time variability for a subset of brain regions

Our results differ from converging evidence of the literature in that we did not find any EEG variability correlates in the ACC. This region is thought to be involved in both top-down and bottom-up attentional control related to sensory processing (Crottaz-Herbette and Menon, 2006) and has been linked to early EEG component modulations. The N1 has been shown to correlate with ACC activity during auditory discrimination tasks using EEG-fMRI (Mulert et al., 2008) and fMRI-constrained EEG source localization (Esposito et al., 2009b). The latter technique was applied to both visual and auditory oddball data to find correlates in the early N2b and P3a components (Crottaz-Herbette and Menon, 2006). However, we did find ACC correlates of RT variability, which was not regressed out of the single-trial-variability models in these previous investigations. This finding is in accordance withWarbrick et al. (2009), who used a visual target-detection task to discover similar ACC correlates for both RT variability and P3 latency variability, but did not tease these effects apart. We demonstrate that attention-related ACC coupling with EEG components is reflected in the variability of externally-observable behavioral response events.

Our EEG variability results also differ from the many EEG-fMRI reports of P3-variability correlates in the insula (Bledowski et al., 2004a, 2004b; Eichele et al., 2005; Warbrick et al., 2009). However, we did find bilateral insula correlates of the average event-related response, and additional RT-variability correlates (max z-score 3.78 in right insula and 3.38 in left insula) that were slightly too small to pass cluster threshold. Using a visual target-detection task,Warbrick et al. (2009) also found bilateral insula correlates with a traditional analysis, and conversely with P3 amplitude variability but not RT variability. Together with their findings, our results suggest that variance of the BOLD signal within the insula can be explained primarily via the average event-related response, with only minor contributions observable in EEG-component and RT variability.

4.5 Our methods begin to unravel a cascade of neural events associated with endogenous attentional modulations

Precise temporal localizations are difficult in traditional fMRI studies due to the slow, diffuse, and indirect nature of the BOLD measurement and to spatial variations in the HRF. Our method of combining single-trial EEG variability with fMRI was able to circumvent this limitation by finding BOLD correlates of the electrophysiological response at multiple temporal offsets. Because we used EEG STV in our fMRI model design and orthogonalized to event-related regressors, our activations reveal the BOLD correlates of endogenous modulations in attention across trials. This method begins to unravel a complex cascade of neural events, including sensory processing, executive processing, motor planning, and default-mode activity, in high spatial and temporal resolution.

Due to the simplicity of the task, the subjects’ minds were free to wander, and this enabled observation of natural fluctuations in task engagement and the underlying spatial redistribution of attentional resources throughout the duration of each trial. Specifically, our results revealed task-relevant and primarily right-lateralized frontal areas engaged immediately following the stimuli, and default-mode-region activations following the behavioral response. These findings are consistent with the role Polich (2007) proposed for the P3 in rapid inhibition of ongoing neural processes to facilitate the transfer of stimulus information from frontal to temporal-parietal areas. They also provide evidence for a role of the default mode network in task-related processing. Furthermore, our findings demonstrate the utility of our methods for noninvasively investigating the temporal ordering of many widely distributed BOLD activations. We believe these techniques will be important in future investigations of brain function, as the information they provide is complementary to that which can be obtained from intracranial-EEG, MEG, and resting state studies.

Highlights.

*BOLD signal modeled using discriminator-determined trial-to-trial EEG variability.

*Correlates of continuously evolving discriminating components are investigated.

*Methods allow study of processes not observable with behavioral measures.

*Both task-dependent and default-mode regions are transiently engaged.