ABSTRACT
Multimodal imaging studies that combine temporally and spatially precise methods can enhance understanding of reward neurocircuitry by revealing how signals specifically time‐locked to reward processing substages relate to functional network‐level activity. Prior reward studies that require decision making and/or motor responses to obtain rewards may complicate efforts to isolate basic reward responses from higher‐order functions that support reward attainment. Here, we take an integrated, multimodal approach to evaluate anticipatory and consummatory reward processing substages, while minimizing demands on higher‐order cognitive and motor processes. Functional magnetic resonance imaging (fMRI) and electrophysiological (EEG) data were separately recorded from 52 adults playing a simple, slot machine task, with reward outcomes independent of performance. Joint Independent Component Analyses (jICAs) were conducted with EEG‐based event‐related potential (ERP) difference waveforms and fMRI contrast images specific to reward anticipation and outcome processing substages. Resulting joint independent components (JICs) segregated reward processing substages, indicating significant co‐modulation between temporal ERP and spatial fMRI signals (p < 0.001). During Reward Anticipation, a JIC with the temporal signature of the stimulus preceding negativity (SPN) ERP component covaried with fMRI activation in bilateral supplementary motor areas (pre‐SMA/SMA) and inferior fronto‐insular salience network regions implicated in attentional orienting and shifting. During Reward Feedback, JICs with the temporal signature of the reward positivity (RewP) ERP component covaried with fMRI activation in dorsal anterior cingulate cortex (dACC), ventral striatum, SMA, and inferior frontal cortex, extending to the insula. Further, trait reward sensitivity correlated with jICA‐informed Win > Loss brain activations during Reward Feedback (p = 0.016). Our findings demonstrate that temporally precise electrophysiological and spatially rich hemodynamic measures of reward processing converge to map onto specific substages of reward‐related brain processes. ERP and fMRI signaling during reward feedback support covariation of the RewP with dACC‐striatal reward networks while the SPN covaried with fMRI signal in pre‐SMA/SMA and inferior fronto‐insular regions implicated in motor planning, salience, and attention.
Keywords: frontal‐medial negativity, motivation and pleasure, positive valence system, reward anticipation, reward feedback and consumption
Key Points
Spatially rich hemodynamic and temporally precise electrophysiological measures of reward processing provide convergent information that isolates reward anticipation and evaluation processes.
EEG and fMRI signaling during reward feedback support localization of the EEG‐based reward positivity (RewP) event‐related potential component to dorsal anterior cingulate and striatal regions.
In contrast, the reward‐anticipation associated stimulus preceding negativity (SPN) event‐related potential component covaried with fMRI activations in motor planning and salience network regions.
Reward processing is complex and parsing anticipatory from consummatory substages will develop a more specific taxonomy of neural motivation and pleasure systems. Our multimodal data indicate EEG–fMRI covariance linking reward feedback EEG measures to dACC‐striatal regions, while reward anticipation EEG measures covaried with motor planning and salience network fMRI activations.

1. Introduction
Reward‐related brain circuitry plays a central role in driving behavior. Reward processing, however, is complex and linking specific reward subcomponents with their corresponding neurocircuitry is important for understanding reward system functioning in health and disease. One major conceptual distinction separates reward‐related functions into anticipatory “wanting” processes related to reward motivation and goal‐oriented behaviors from consummatory “liking” processes related to reward attainment (Berridge and Kringelbach 2015; Berridge and Robinson 2003).
Most human imaging studies of the neural bases of reward processing are based on a single neuroimaging modality. Anterior cingulate cortex (ACC), ventromedial prefrontal (VMPFC), orbitofrontal cortex (OFC), and ventral striatum (VS) are strongly implicated in single modality functional magnetic resonance imaging (fMRI) studies of reward anticipation and evaluation (Wilson et al. 2018; Clark et al. 2009; Diekhof et al. 2012; Haber and Knutson 2010), with activations in these regions elicited across reward types, while controlling for non‐specific processes, like general arousal and perceptual salience (Haber and Knutson 2010). Because of its millisecond‐level resolution, electroencephalography (EEG)‐based event‐related potentials (ERP) can parse brain activity into constituent anticipatory and consummatory sub‐components with much higher temporal precision than fMRI (Glazer et al. 2018). ERP‐based reward anticipatory measures include components reflecting anticipatory attention such as the stimulus preceding negativity (SPN), a slow‐building negative deflection with a central topography that precedes outcome feedback in the absence of motor preparation (Brunia et al. 2011). The SPN can also show a right‐hemispheric preponderance, particularly when elicited in the context of reward or affective processing. ERP‐based reward consummatory measures that reflect outcome feedback processing (Miltner et al. 1997) include the reward positivity (RewP), a fronto‐central deflection that is positive for wins relative to losses and peaks ~250 ms after external reward outcome feedback (Gehring and Willoughby 2002). The feedback P300 (P3), a maximally centro‐parietal positive deflection that peaks after the RewP, has also been observed but is less well‐studied in the context of reward outcome processing (Glazer et al. 2018; San Martin 2012) than the RewP.
Multimodal imaging studies that combine high temporal resolution methods, like EEG‐based ERPs, with high spatial resolution methods, like fMRI, can enhance current understanding of reward functioning by revealing how signals precisely time‐locked to specific reward substages relate to spatial patterns of network‐level activity. While infrequently implemented, multimodal data integration is a critical need in the field (Glazer et al. 2018; San Martin 2012; Becker et al. 2014), offering insights into the relationship between fMRI and ERP reflections of reward processing and the extent to which these methodologically distinct reward measures converge. While few studies have integrated ERP and fMRI measures of reward processing, those that have demonstrate the benefits of a multimodal approach (Becker et al. 2014; Carlson et al. 2011; Nieuwenhuis et al. 2005). For example, the RewP correlates with fMRI activations during rewarded decision‐making in VS, medial prefrontal cortex (mPFC), caudate, amygdala, and OFC (Carlson et al. 2011). Simultaneously acquired fMRI BOLD activations within VS, midbrain, ACC, and mPFC covary with single trial feedback‐locked RewP amplitudes to correct responses in a time‐estimation task (Becker et al. 2014). Intracranial recordings of 19 patients with refractory epilepsy provide the most direct evidence of RewP localization in the context of reward processing, in which left hemispheric caudal (dorsal) ACC, dorsolateral prefrontal cortex (DLPFC), left frontomedial cortex, and white matter show concordance with simultaneous scalp recordings of the RewP, with the greatest absolute current density activity observed in the dorsal anterior cingulate cortex (dACC) (Oerlemans et al. 2025). A multimodal magnetoencephalography‐fMRI slot machine study also demonstrated meaningful temporal–spatial signal covariation, with theta oscillatory power covarying with increased BOLD signal in widespread frontal‐insular regions when comparing wins to combined losses (Dymond et al. 2014). Less research has examined how anticipatory reward ERPs converge with fMRI activations; though an fMRI‐informed EEG source analysis study of the SPN preceding feedback during a time estimation task identified regions involved in reward expectancy, arousal and salience, and perceptual anticipation of visual stimuli, including insula, OFC, and supplementary motor area (SMA) (Kotani et al. 2015).
The goal of the current study was to integrate fMRI and EEG data to identify functional neuroanatomical correlates of reward processing that covary with temporally precise ERP measures of anticipatory and consummatory sub‐stages of reward processing. A number of prior reward processing studies have used active experimental paradigms that require decision‐making and/or motor response to obtain rewards; though meta‐analysis of single modality fMRI reward processing studies of passive task designs (i.e., passive reward expectancy and outcome evaluation) reveals VS activations during anticipation and VS and vmPFC/OFC activations during evaluation of reward (Diekhof et al. 2012). While performance‐dependent reward paradigms have made valuable contributions to understanding the functional neuroanatomy associated with reward responses, they also pose inherent confounds to isolating neural responses specific to reward responsivity. Tasks in which reward outcomes depend on participant skill or behavior make it difficult to disentangle basic reward responsivity from higher‐order responses necessary to obtain reward, such as an individual's motivation and cognitive effort. Our study utilizes a performance‐independent slot machine task, which allows for an examination of anticipatory and consummatory substages of reward in a context that minimizes demands on higher‐order cognitive processes. Modeling basic reward function is not only relevant in describing healthy reward functions, but can be valuable in developing a more precise understanding of reward dysfunction in populations with impairments in motivation and cognition, such as those diagnosed with neuropsychiatric disorders (Barch et al. 2016; Nusslock and Alloy 2017).
We recorded EEG and fMRI in separate, counterbalanced sessions (n = 52) during a slot machine paradigm. The slot machine is a popular and simple form of gaming, underscoring its ecological validity as a probe of elemental reward responsivity (Barton et al. 2017; Parke and Griffiths 2006; Reid 1996). Both our research group (Fryer et al. 2021) and others (Donkers et al. 2005; Donkers and Van Boxtel 2005) found that RewP and SPN ERPs are readily elicited by EEG slot machine reward tasks; slot machine tasks also recruit reward‐related circuitry measured with fMRI (Clark et al. 2009; Dymond et al. 2014). In addition to wins and full losses, slot machines typically capture “near misses,” which are loss outcomes that appear structurally closer to a win than a “full” loss but have the same economic value as full losses (Reid 1996). To minimize cognitive and motivational task demands that complicate interpretation of performance‐based reward tasks, reward outcome in our slot machine task does not depend on participant performance. We evaluated fMRI–EEG relationships via joint Independent Component Analysis (jICA) (Calhoun et al. 2006; Sui et al. 2012), a data‐driven approach, to identify task‐specific temporal evoked responses in ERP waveforms that covary with spatial fMRI activation patterns. jICA is a blind source separation technique that explains the underlying structure of multimodal data through identifying commonalities or “cross‐information” between the modalities. It extracts “features” which covary the same way across participants. In this way, we can identify specific spatial patterns in the fMRI data that are specifically co‐varying with the higher temporal resolution information in the EEG data (Calhoun et al. 2006; Sui et al. 2012). We predicted we would identify joint components linked to anticipatory and consummatory reward processing substages, with reward outcome ERP responses coincident with the RewP covarying with reward‐related fMRI activations in VS and dACC (Miltner et al. 1997; Oerlemans et al. 2025; Foti et al. 2015; Holroyd and Umemoto 2016). Low frequency ERP components, like the reward anticipation‐sensitive SPN, typically have multiple cortical generators (Brunia et al. 2011). Therefore, we predicted we would identify an anticipatory jICA linked with anticipation‐related activations in distributed regions including insula, OFC, and SMA (Kotani et al. 2015). To aid interpretation of integrated analyses, single‐modality fMRI effects are also presented, whereas single‐modality ERP effects in this sample were previously reported (Fryer et al. 2021). Lastly, jICA‐defined fMRI activations were correlated with individual differences in self‐reported trait reward sensitivity to assess the behavioral relevance of these multimodal measures.
2. Methods
2.1. Subjects
About 52 adult volunteers (19–64 years (mean 33.5 years ±14.6); 78.85% men; 86.54% right‐handed) were recruited as comparison subjects for a larger clinical study. Axis I psychiatric disorders were ruled out with the Structured Clinical Interview for DSM‐IV (SCID‐IV‐TR) (First et al. 2002). Inclusion criteria for study participation were: negative urine toxicology for common substances of abuse (i.e., opiates, cocaine, amphetamines), English fluency, and negative history of head injury, neurological illness, or other major medical illness impacting the central nervous system. Participants were recruited via community advertisements. This study was approved by the Institutional Review Board (IRB) at the University of California, San Francisco (UCSF). All subjects provided written informed consent.
2.2. Task
Participants completed 288 trials of a slot machine task (Fryer et al. 2021) modeled on prior literature (Clark et al. 2009; Dymond et al. 2014; Donkers et al. 2005; Habib and Dixon 2010). To build expectancy, the display consisted of three slot reels (R1, R2, R3), each of which was initially blank and sequentially populated with fruit symbols from left to right over the course of a trial. Reels were populated with one of 12 possible fruit symbols, with stimuli distributed equally among possible outcomes, such that individual fruit symbols carried no predictive information about outcome. The stimuli were presented on a 22‐in. light emitting diode display monitor that was placed 1.32 m in front of the participant. The fruit symbols were each 200 × 200 pixels (or about 5.3 cm × 5.3 cm) with a visual angle of 2.3°.
Trial initiation was self‐paced via participant button press with the participant's dominant hand, after which timing of the slot reels was automated. Each button press triggered an audible animated coin drop and lever press to increase the paradigm's face validity. The spin phase (Reward Anticipation phase) consisted of R1, R2, and R3 populating their respective slots each with a single fruit symbol. After R3 populated, the outcome phase began (Reward Evaluation phase) and a 40 Hz visual checkerboard flicker appeared for ~1000 ms duration followed by outcome text that depicted either “WIN $1.25” or “LOSE,” depending on the trial type. Total trial time post‐participant initiation was 6115 ms per trial. Of note, each reel has a 1.2 s stimulus onset asynchrony, which affords about 3 s of inter‐stimulus interval between the first motor press (at 0 ms) and the measurement of the reward anticipation ERP components, enabling isolation of the SPN component from motor‐related potentials. There were three types of trials: (1) Wins, (2) Near Misses, and (3) Total Misses. Wins occurred when three identical fruit symbols were populated in the slot reels (AAA); Near Misses occurred when symbols in R1 and R2 were identical but the symbol in R3 was incongruent (AAB); Total Miss trials were revealed at R2, indicating no chance of winning on that trial (ABC). To reflect real‐world slot‐machine outcomes, subjects encountered more frequent Total Misses (n = 144, p = 0.50) than Wins (n = 72, p = 0.25) and Near Misses (n = 72, p = 0.25). All Near Misses and Total Misses were $0 payouts while each Win yielded a $1.25 payout. To ensure participants that the opportunity for reward was valid, they were given a $15 monetary bonus reflecting their slot machine winnings. See Figure 1 for task timing details.
FIGURE 1.

Slot machine task timing diagram. First slot reel appears 1266 ms after the participant‐initiated button press. The subsequent slot reels populate at 1200 ms intervals (R2 at 2466 ms; R3 at 3666 ms).
The fMRI version of the task delivered the same trial presentation sequence used for EEG data collection, with slight timing modifications to adapt to the fMRI environment. Task sequencing was optimized for efficiency based on a genetic algorithm (Wager and Nichols 2003) yielding variable time between the end of one trial and the initiation prompt for the next (range: 0 to 10,000 ms, exponentially distributed) to introduce temporal jitter and minimize event collinearity. Trial initiation was self‐paced, as in the EEG task version, with the added constraint that if the participant did not make a button press within 1850 ms of the prompt to play, the trial was then initiated automatically to align with scanner TR timing. Participants completed six task runs (48 trials per run, with trial types randomized within and across runs). To keep trial durations equivalent, additional null time was added at the end of a trial if the participant initiated the trial prior to the 1850 ms response window (yielding per trial additional variable null time of 1850 ms—reaction time of button press).
EEG and fMRI session order was counterbalanced (7.02 ± 10.7 days between sessions).
2.3. fMRI Acquisition and Preprocessing
Structural and functional MRI data were collected using a 3T Siemens Skyra scanner. The structural imaging protocol was a magnetization‐prepared rapid gradient‐echo (MPRAGE) T1‐weighted high resolution imaging sequence: TR = 2300 ms, TE = 2.98 ms, slice thickness = 1.2 mm, Field of View (FOV) = 240 mm, voxel size = 1.0 × 1.0 × 1.2, flip angle = 9°, sagittal orientation, 9:14 min. The fMRI protocol was an AC‐PC aligned echo planar imaging (EPI) sequence: FOV = 220 mm, TR = 2000 ms, TE = 30 ms, flip angle = 77°, number of slices = 30, slice thickness = 4 mm, slice gap = 1 mm, ascending order, matrix = 64 × 64; 8:04 min per run.
Functional images were analyzed with Statistical Parametric Mapping toolbox (SPM8) (www.fil.ion.ucl.ac.uk) running in MATLAB v2009 (www.mathworks.com). Motion correction via affine registration was applied, in which the first image of each run was realigned to the first image of the first run, and then re‐alignment proceeded within each run. Images were then slice‐time corrected to the middle slice. aCompCor, a regression‐based algorithm for denoising BOLD data, was then applied (Behzadi et al. 2007); aCompCor performs principal components analysis (PCA) on time‐series data by deriving white matter and cerebrospinal fluid (CSF) noise regions from segmented high‐resolution T1‐weighted anatomical images co‐registered to functional EPIs. White matter noise regions were derived from FreeSurfer (http://surfer.nmr.mgh.harvard.edu/) segmentation (Fischl 2012). The artifact detection tools package (ART; http://www.nitrc.org/projects/artifact_detect/) identified outlying volumes based on global image intensity values (z > 3) and head motion (> 2 mm translational movement in the x, y, or z planes, or > 0.02° rotation in yaw, pitch, or roll).
2.4. fMRI Single Modality Modeling
A first‐level analysis was performed for each subject using the general linear model (GLM) in SPM8. The trial duration from R1 through outcome feedback was modeled as an epoch with a boxcar function convolved with the canonical hemodynamic response function. Slot outcome events (Wins, Near Misses, Total Misses) were modeled as separate conditions. In addition, the “press to play” and participant response that initiated the trial prior to the reel beginning were modeled as nuisance regressors. A high pass temporal filter of 128 s was applied to remove low‐frequency noise. The first‐level GLM included seven ART motion parameters, consisting of the temporal derivatives of the six motion parameters as well as a composite measure of total motion across translation and rotation. Regressors were also included for (i) data points identified by the ART toolbox as outliers and (ii) statistically significant (p < 0.05) principal noise components from the aCompCor denoising routine from each individual fMRI run. The mean functional image from the motion correction was normalized to standard neuroanatomical space (MNI‐EPI template; http://www.bic.mni.mcgill.ca) with 3 mm3 isotropic voxel dimensions. Normalization parameters were applied to first‐level beta and contrast images, which were then spatially smoothed with a 6 mm full‐width‐half‐maximum Gaussian kernel.
To examine fMRI responses to Reward Anticipation we contrasted Near Miss‐Total Miss (AAB‐ABC). “Total Misses” are losses revealed at R2 prior to final feedback. In trials other than Total Misses, additional reward anticipation builds between R2 and R3. Therefore, by contrasting Near Misses and Total Misses in our task, we identify activations in brain regions associated with reward anticipation which remain after subtracting the loss processes common to both R2 and R3.
To examine fMRI responses to Reward Evaluation we formed two contrasts: Win‐Near Miss (AAA‐AAB) and Win‐Total Miss (AAA‐ABC) to isolate brain responses to winning versus losing at the final reel, and winning relative to losing at R2, respectively.
2.5. EEG Acquisition and Preprocessing
EEG data were recorded on a 64‐channel BioSemi ActiveTwo system (www.biosemi.com) and digitized using a 1024 Hz sampling rate. EEG data were referenced offline to averaged mastoid electrodes, with a 0.1 Hz high‐pass filter applied in ERPLAB (Lopez‐Calderon and Luck 2014). Additional electrodes were placed above/below the right eye and the outer canthi of both eyes to record eye movements and blinks (horizontal and vertical electro‐oculogram, HEOG and VEOG respectively). Data were next subjected to Fully Automated Statistical Thresholding for EEG artifact Rejection (FASTER) using a freely distributed toolbox (Nolan et al. 2010). The method employs multiple descriptive measures to search for statistical outliers (> ±3 SD from mean): (1) outlier channels were identified and replaced with interpolated values in continuous data, (2) outlier epochs were removed from participants' single trial set, (3) spatial independent components analysis was applied to remaining trials, outlier components were identified using the ADJUST procedure (Mognon et al. 2011), data were back‐projected without these components, and (4) outlier channels were removed and interpolated within an epoch. The original FASTER processing approach was modified between steps 2 and 3 to include canonical correlation analysis (CCA). CCA was used as a blind source separation technique to remove broadband or electromyographic noise from single trial EEG data, generating de‐noised EEG epochs. This approach is identical to the CCA methods we have used previously (Fryer et al. 2021).
3. Data Analysis Plan
3.1. Multimodal Analysis of EEG and fMRI Data
To identify neuroanatomical responses associated with anticipatory and outcome substages of reward processing in the ERP data, a joint ICA (jICA) (Calhoun et al. 2006) was implemented using the MATLAB‐based Fusion ICA Toolbox (FIT) (http://mialab.mrn.org/software/fit). FIT uses each subject's EEG (ERP component difference wave time courses) and fMRI (contrast beta maps). Data from both modalities are concatenated (subjects‐by‐time points for EEG and subjects‐by‐voxels for fMRI), across subjects, to identify “fused” ERP‐fMRI joint components that maximize the joint likelihood function. The loading weights for each subject reflect the magnitude of the joint independent component (JIC) for that subject. As part of the jICA pipeline, EEG data were resampled to equate to the number of EEG and fMRI time samples. Sources associated with the two modalities are assumed to covary the same way across subjects (i.e., equal linear covariation) (Calhoun et al. 2006; Sui et al. 2012). We conducted two jICAs, one focused on Reward Anticipation and the other on Reward Evaluation. Each included a temporal input (ERP difference wave) and a spatial input (the first‐level model fMRI contrast image), as described below. Measurements of each ERP component were based on electrodes used in our previous studies with this task (Fryer et al. 2021; Abram et al. 2020) and commonly reported in the literature for the ERP components of interest Cz for SPN (Donkers et al. 2005; Donkers and Van Boxtel 2005), FCz for RewP (Holroyd et al. 2008).
3.1.1. Reward Anticipation jICA ERP Input (AA‐AB)
The R2‐locked difference wave of all Possible Win trials (AA; collapsed across trials eventually revealed as Wins or Near Misses, which are equivalent at R2) minus Total Miss trials (AB; trials revealed as a loss at R2). EEG epochs from electrode Cz were baseline corrected to the −1300 to −1200 ms period preceding R3 (i.e., −100 to 0 ms preceding R2) before averaging, and resulting ERP difference waves comprised activity from −1200 ms to R3 onset. fMRI input: The Near Miss‐Total Miss (AAB‐ABC) contrast beta maps. Unlike the EEG analysis, R2 activity could not be isolated from R3, owing to the relatively poorer temporal resolution of the fMRI signal; therefore, the AAB‐ABC contrast offered the best measure to isolate reward anticipation activity. By subtracting the loss processes common to both Near Misses (AAB) and Total Misses (ABC) in our task, we identify fMRI activations in brain regions associated with reward anticipation which remain.
3.1.2. Reward Evaluation jICA ERP Input (AAA‐AAB)
The R3‐locked difference wave between Wins and Near Misses (AAA‐AAB). Epochs from electrode FCz were baseline corrected using the −100 to 0 ms period prior to R3 before averaging, and resulting ERP difference waves comprised activity from −100 ms before to 900 ms after R3 onset. The fMRI input for Reward Evaluation jICA was the Win‐Near Miss (AAA‐AAB) contrast beta maps.
The RewP is often measured from the ERP win versus loss difference wave as a relative positivity between 228 and 344 ms (reviewed by Proudfit 2015), with contributions from both a positive amplitude defection to wins and a coincident negative deflection to losses (Holroyd et al. 2008; Proudfit 2015). We therefore conducted follow‐up analyses to assess condition specific effects that are not readily interpreted from the Win‐Near Miss difference wave. Accordingly, we conducted follow‐up jICAs with each condition waveform (AAA, AAB) separately contrasted with the ABC loss waveform, which provides no additional outcome feedback at the final reel and therefore serves as a low‐level visual control. These two condition‐specific jICAs were conducted with the same temporal epoch as the main Win‐Near Miss (AAA‐AAB) reward outcome jICA.
Consistent with prior work in our laboratory (Ford et al. 2016; Jacob et al. 2019), the number of estimated jICA components was determined by Minimum Description Length (Rissanen 1983), as implemented in the FIT program (resulting in estimating eight and nine components for the reward anticipation and evaluation jICAs, respectively). To determine which components to analyze further, we used a scree plot of variance accounted for by each component. Only components that accounted for > 15% were retained. For the reward anticipation jICA, only components that showed temporal peaks within 800 ms of R3 were considered for analysis. jICA factor scores from components of interest were subject to formal tests of statistical inference via one‐sample t‐test of the factor scores versus zero and held to a Bonferroni‐corrected p value across the total number of components being tested within each jICA. jICA figures are displayed at z > 1.96, but we note this threshold is for visualization purposes, as statistical inferences about the factors' significance are derived from the one‐sample t‐tests held to p < 0.05, Bonferroni‐corrected for the number of significant components in the jICA.
3.2. Associations Between jICA and Self‐Reported Reward Sensitivity
The reward responsiveness subscale of the Behavioral Inhibition and Behavioral Activation Systems (BAS) Scale was used to measure individual differences in self‐reported reward responsiveness (Carver and White 1994). The relationship between jICA‐informed BOLD activations and self‐reported reward sensitivity was assessed via a linear regression model regressing BAS reward responsiveness level on regions‐of‐interest (ROI) identified in the Reward Anticipation and Reward Evaluation jICAs. Specifically, the jICA maps were used to define ROIs that were applied to extract individual median single modality fMRI activations from the AAB‐ABC and AAA‐AAB contrasts for reward anticipation and evaluation, respectively. Omnibus regression models were held to a Bonferroni‐corrected alpha value of 0.025 (i.e., 0.05/2) to account for type I error across the two regression models.
3.3. Single Modality fMRI Analyses
To aid interpretation of integrated analyses (and provide data for future potential meta‐analyses) single‐modality fMRI effects were also analyzed through focused ROI analyses based on coordinates from a prior meta‐analysis of passive reward studies (Diekhof et al. 2012) and complementary whole‐brain voxelwise analysis. The height threshold for all voxelwise analyses was p < 0.001, two‐tailed, and only clusters surviving family‐wise error (FWE) correction at a p < 0.05 significance level are reported (in Figure S1 and Tables S1 and S2).
4. Results
4.1. EEG–fMRI Multimodality Analysis (jICA)
4.1.1. Reward Anticipation
A significant joint component was detected for the Possible Win‐Known Loss (AA‐AB) ERP waveform with the Near Miss‐Total Miss (AAB‐ABC) fMRI contrast. Temporal weights of this JIC were consistent with the SPN ERP component, slowly building from ~500 ms with peak negative amplitude within the last 200 ms prior to final reel feedback, with spatial weights in bilateral supplementary motor cortex (Brodmann areas [BA] 6,8), bilateral DLPFC (BA 9, 46), and right inferior frontal gyrus (BA 47; IFG), extending into insula and superior parietal lobule (BA 7; SPL). Regions in pre‐SMA and right prefrontal cortex overlapped with salience network regions (Seeley et al. 2007). Negative spatial weights, indicating inverse co‐modulation between ERP and fMRI signals, were observed in regions including the precuneus and posterior cingulate (BA 23, 31), extending into bilateral medial occipital cortex (BA 17, 18), medial superior parietal lobule (BA 7), and bilateral lateral superior frontal cortex (BA 8). One‐sample t‐test of the factor scores from this component indicated a statistically significant effect: t(51) = 5.50, p < 0.001. See Figure 2. A second component with a temporal peak that did not fall within 800 ms of R3 was also detected (See Figure S2 for more details). We present, in the Supporting Information, the single modality EEG grand average waveforms and their corresponding topographical maps during the anticipation phase, as well as correlations of SPN amplitude with horizontal eye movement (HEOG) (See Figures S3 and S4).
FIGURE 2.

Reward Anticipation EEG–fMRI joint‐independent components analysis (jICA). Left: Temporal waveforms showing overlap of EEG input (black line; second‐reel Possible Win‐Known Loss AA‐AB ERP waveform from −1200 to 0 ms, at Cz) and joint independent component (JIC) temporal signature (blue line). Zero ms on the x‐axis indicates final reward feedback. The temporal JIC observed slowly builds from ~500 ms with peak negative amplitude within the last 200 ms prior to final reel feedback. Right: Spatial jICA maps show areas (in red) of positive co‐modulation between fMRI and EEG, including bilateral supplementary motor cortex (pre‐SMA/SMA), dorsolateral prefrontal cortex, and inferior prefrontal cortex, extending to right insula (spatial maps thresholded at z‐score > 1.96 for visual depiction). Pre‐SMA and right inferior prefrontal regions showed overlap (in yellow) with salience network (Shirer et al. 2012) regions (in green), implicated in attentional orienting and shifting.
4.1.2. Reward Evaluation
In the analysis of the Win‐Near Miss (AAA‐AAB) ERP waveform and fMRI contrast, two JICA components were detected that appeared to capture early and later segments of the large positivity evident in the time‐domain Win‐Near Miss loss waveform. See Figure 3.
FIGURE 3.

Reward outcome EEG–fMRI joint‐independent components analysis (jICA). (Panel A) Early feedback. Left: Temporal waveforms show overlap of EEG input (black line; Win‐Near Miss ERP difference waveform from −100 to 900 ms, at FCz) and joint independent component (JIC) temporal signature (blue line). Zero milliseconds on the x‐axis indicates final reward feedback. Temporal JIC weights peaked 259 ms after R3 reward feedback, within a time‐frame similar to the RewP ERP which peaked at 290 ms (Gehring and Willoughby 2002; Sambrook and Goslin 2015). Right: Spatial jICA maps show areas of positive co‐modulation between fMRI and EEG, including the dorsal anterior cingulate cortex (dACC), ventral striatum, pre‐supplementary motor area (pre‐SMA), and inferior frontal cortex, which extended to the insula. Lower Right: Overlap of JIC spatial weights observed in the current study with left caudal (dorsal) ACC coordinates (0x, 24y, 43z) identified as the current source density activity that best accounted for the RewP during simultaneous intracranial and scalp recordings (Oerlemans et al. 2025). (Panel B) Later feedback. A second JIC was observed ~100 ms later (358 ms) and covaried primarily with posterior cortical regions, including left precuneus and left temporo–parietal–occipital junction (including BA 39) and bilateral temporal (middle temporal gyri, BA 37), posterior cingulate (BA 30), and occipital cortices (including cuneus, BA 18, 19). Spatial maps thresholded at z‐score > 1.96 for visual depiction.
4.1.2.1. Early Reward Feedback
The first component had temporal weights within a time frame consistent with the RewP ERP component (Gehring and Willoughby 2002; Sambrook and Goslin 2015), peaking at ~260 ms after R3 reward feedback, with spatial weights in the right VS, and bilateral dorsal (caudal) ACC (BA 32), pre‐SMA (BA 6), mPFC (BA 10), and lateral inferior frontal cortex, which extended medially to the insula. The component also had negative spatial weights in regions including inferior temporal (BA 37), superior parietal (BA 7), and superior frontal (BA 8) cortices. One‐sample t‐test of the factor scores from this component indicated a statistically significant effect: t(51) = 8.29, p < 0.001.
4.1.2.2. Late Reward Feedback
The second evaluation component, which peaked ~100 ms later than the earlier JIC, covaried with primarily posterior cortical regions, including bilateral temporal cortex (middle temporal gyri, BA 37), posterior cingulate (BA 30), bilateral occipital cortex including cuneus (BA 18, 19), left precuneus and left temporo–parietal–occipital junction (including BA 39). The component also had negative spatial weights in right VS, inferior and superior parietal lobules (BA 7, 40), and bilateral occipital cortex (BA 18). One‐sample t‐test of the factor scores from this component (vs. zero) was significant t(51) = 7.29, p < 0.001.
4.1.2.3. Reward Evaluation jICA Follow‐Ups
-
Win‐Total Miss (AAA‐ABC):
Two components were detected in the Win‐Total Miss analysis. The earlier Win condition (AAA‐ABC) component occurs at ~330 ms and showed comodulation with AAA‐ABC fMRI activations in right VS, bilateral inferior frontal gyrus (BA 47), extending to insula, superior frontal gyrus and pre‐SMA (BA 6), and occipital cortex (BA 17, 18). Negative spatial weights were observed in bilateral temporo‐parietal occipital junction (BA 19, 39) and right‐lateralized precuneus and superior frontal/precentral gyri (BA 6,8). The later Win condition (AAA‐ABC) component peaked at ~420 ms and comodulated with fMRI activations in right IFG (BA 9) and ACC (BA 32) extending into BA 6 and 8, left parieto‐occipital cortex (7, 19) and right inferior parietal lobule (BA 40). Negative spatial weights were observed primarily in medial occipital cortex (BA 18). One‐sample t‐test of the factor scores from both of these Win condition components (vs. zero) were significant (early Win component: t(51) = 17.68, p < 0.001; later Win component: t(51) = 13.73, p < 0.001). See Figures 4 and 5.
-
Near Miss‐Total Miss (AAB‐ABC):
A single JIC was observed in the follow‐up analysis for the Near Miss condition (AAB‐ABC), peaking at ~380 ms and comodulating with AAB‐ABC fMRI activations in bilateral dACC (BA 32), VS, and pre‐SMA/superior frontal gyrus (BA 6) and inferior frontal cortex (BA 47), extending to the left insula. Negative spatial weights were observed in bilateral temporo‐parietal occipital (BA 19, 39) and precuneus (BA 7) regions. One‐sample t‐test of the factor scores from the Near Miss condition component (vs. zero) were significant t(51) = 10.06, p < 0.001. See Figures 4 and 5.
FIGURE 4.

Follow‐up jICAs with each condition waveform separated for Wins (AAA‐ABC) and Near Miss Losses (AAB‐ABC). (Panel A) Win condition. Left: Temporal waveforms show overlap of EEG input (black line; Win‐Total Miss ERP difference waveform from −100 to 900 ms) and joint independent component (JIC) temporal signature (blue line). Zero milliseconds on the x‐axis indicates final reward feedback. Temporal JIC weights peaked at 332 ms after R3 reward feedback. Spatial jICA maps show areas of positive co‐modulation between fMRI and EEG, including right VS, bilateral inferior frontal gyrus (BA 47), extending to insula, superior frontal gyrus and pre‐SMA (BA 6), and occipital cortex (BA 17, 18). Middle: A second later component peaked at 421 ms and comodulated with fMRI activations in right IFG (BA 9) and ACC (BA 32) extending into BA 6 and 8, left parieto‐occipital cortex (7, 19) and right inferior parietal lobule (BA 40). (Panel B) Near miss condition. Right: Temporal waveforms show overlap of EEG input (black line; Near Miss‐Total Miss ERP difference waveform from −100 to 900 ms) and JIC temporal signature (blue line). Temporal JIC observed peaked at 379 ms and positively comodulated with fMRI activations in bilateral dACC (BA 32), VS, and pre‐SMA/superior frontal gyrus (BA 6) and inferior frontal cortex (BA 47), extending to the left insula. Spatial maps thresholded at z‐score > 1.96 for visual depiction.
FIGURE 5.

Joint independent component (JIC) signaling in ventral striatum (VS). VS spatial weights (shown on left) corresponding to condition difference score JIC time‐courses (shown on right) for the Win > Near Miss (AAA‐AAB) in red; Near Miss > Win (AAB‐AAA) in blue; Win > Total Miss (AAA‐ABC) in orange; Near Miss > Total Miss (AAB‐ABC) in green. These data highlight the earlier and larger magnitude of fMRI‐ERP win‐related VS signaling, while also highlighting that near misses elicit a multimodal response in VS that is greater than total misses.
Note, all p values reported for jICA analyses in which two components of interest were identified met the Bonferroni‐corrected alpha value of 0.025. There were no analyses in which more than two components of interest were identified.
4.2. Associations Between jICA and Self‐Reported Reward Sensitivity
4.2.1. Reward Anticipation
Self‐reported reward responsiveness was unrelated to jICA‐informed reward anticipation (AA‐AB) fMRI activations (SMA, RIFG/Insula ROIs); the overall model was not significant (F(2,44) = 0.29, p = 0.749 model, R2 = 0.01).
4.2.2. Reward Outcome
When self‐reported reward responsiveness was regressed on jICA‐defined reward outcome (AAA‐AAB) fMRI activation regions (VS, dACC, LIFG, RIFG/Insula ROIs) in a multiple regression, the overall model was significant when held to the Bonferroni‐corrected alpha value of 0.025; (F(4,40) = 3.48, p = 0.016, model R 2 = 0.18), with higher AAA‐AAB activations in VS (standardized beta = 0.313, unstandardized = 3.69, t = 2.21, p = 0.033, partial r = 0.33) and LIFG (standardized beta = 0.513, unstandardized = 3.03, t = 2.80, p = 0.008, partial r = 0.41) regions significantly covarying with higher levels of self‐reported reward approach behaviors (whereas dACC and RIFG/Insula were not significant model predictors, −1.59 < t < 0.071, p > 0.05). See Figure 6 for partial regression plots of the regions showing significant associations between jICA‐informed reward‐related brain activations and self‐report of reward behaviors.
FIGURE 6.

JICA‐defined reward‐related activations and self‐reported trait reward sensitivity. Ventral striatal (VS) and left inferior prefrontal (LIFG) regions significantly covaried with higher levels of self‐reported reward approach behaviors (F(4,40) = 0.016, model R 2 = 0.18), with higher AAA‐AAB activations in VS (t = 2.21, p = 0.033, partial r = 0.33) and LIFG (t = 2.80, p = 0.008, partial r = 0.41). Individual differences in trait reward responsivity were measured with the BIS/BAS scale (Carver and White 1994).
5. Discussion
The primary goal of this study was to apply multimodal analysis to identify covariation between ERP and fMRI activations during distinct anticipatory and consummatory sub‐stages of reward processing. Several key findings emerged. During Reward Anticipation, a JIC with temporal weights consistent with the SPN ERP covaried with regions including bilateral pre‐SMA/SMA, DLPFC, and inferior frontal gyrus extending to the right insula. During Reward Feedback (Consummation), JICs with temporal weights consistent with the RewP ERP covaried with regions including dACC, VS, pre‐SMA, and inferior frontal cortex, extending to the insula. Further, individual differences in self‐reported trait reward sensitivity were significantly associated with jICA‐informed Win > Loss brain activations in the VS and a left prefrontal region of the anterior salience network (Shirer et al. 2012) during reward feedback coincident with the timing of the RewP. Our findings demonstrate that spatially rich hemodynamic and temporally precise electrophysiological measures of reward processing provide convergent information that jointly characterizes substages of reward‐related brain processes. EEG and fMRI signaling during reward feedback support localization of the RewP to dACC‐striatal networks while the SPN covaried with fMRI activations in pre‐SMA regions involved in motor planning (Nachev et al. 2008) and salience network regions implicated in attentional orienting and shifting (Shirer et al. 2012; Menon and Uddin 2010). Follow‐up analyses of the reward outcome jICA isolated Win‐Loss difference waveform components and suggested timing and fMRI–EEG comodulation magnitude differences in Win‐related relative to Near Miss loss outcome processing that vary for VS and dACC regions. Findings are discussed below, by reward processing substage.
5.1. Reward Anticipation
Our findings indicate anticipatory fMRI–EEG co‐modulation coincident with SPN timing and fMRI activations in salience network regions including bilateral pre‐SMA (BA 6/8) and inferior prefrontal cortex (BA 47), extending into the insula on the right (salience network region overlap shown in Figure 2). Despite observing VS anticipatory activations in our single modality fMRI analyses (Figure S1 and Tables S1 and S2), the SPN‐associated JIC was not associated with VS activation, which suggests VS‐mediated reward anticipation was not a major contributor to the pre‐R3 SPN signaling, implicating instead an increase in more cognitive anticipatory processes such as attention and salience detection. Our SPN‐fMRI jICA results converge with a small prior study of outcome feedback during a time estimation task that applied fMRI‐constrained ERP source localization. The authors identified a distributed set of regions for source‐localized SPN, including areas involved in reward expectancy, arousal, and attentional salience, and visual perceptual anticipation, with regions from our JIC directly overlapping with the prior study's right insula/OFC findings (MNI coordinates 34x, 22y, −8z) (Kotani et al. 2015). The concordance of our SPN JIC with salience network regions (Kotani et al. 2015) is in agreement with the putative generator of the SPN in anterior insula (Brunia et al. 2011). The salience network is a functional network with hubs in anterior insula and anterior cingulate and is thought to have a central role in saliency detection, attentional control, and switching, as well as interfacing with key limbic and subcortical structures involved in reward and motivated behavior (Menon and Uddin 2010). Our findings support the interpretation that the SPN, an electrophysiological measure ascribed to anticipatory attention (Brunia et al. 2011) is associated with salience network regions. A logical future direction would be to test this interpretation directly by examining SPN amplitude in multimodal analysis with salience network measures derived from task‐based and/or resting‐state functional connectivity analysis.
5.2. Reward Consummation (Outcome Processing)
5.2.1. Early Reward Feedback
The Win‐Near Miss reward outcome jICA produced two JICs after final reel feedback, a larger, earlier one (JIC peak ~260 ms) corresponding to the RewP ERP peak, and a smaller JIC, 100 ms later. The first JIC covaried with regions including VS and dACC, supporting localizations of EEG signal within the RewP time window to regions involved in processing reward outcomes (Haber and Knutson 2010). Spatial weights in the dACC for the first JIC show direct overlap with left dACC coordinates identified during simultaneous intracranial and scalp recordings of the RewP (Oerlemans et al. 2025) (shown in Figure 3). Taken together, our data support a large literature based on translational models, source‐localized EEG, simultaneous EEG/fMRI, transcranial direct current stimulation, and direct intracranial recordings in humans indicating dACC as the putative generator of the RewP (Miltner et al. 1997; Oerlemans et al. 2025; Foti et al. 2015; Holroyd and Umemoto 2016; Whitton et al. 2023). Importantly, the earlier JIC was directly associated with VS activation, consistent with greater VS responses to Wins than to Near Misses, whereas the later Win‐Near Miss JIC was inversely associated with VS activation, consistent with greater VS responses to Near‐Misses relative to Wins (shown in Figure 5). The involvement of VS and ACC regions in RewP signaling agrees with reinforcement learning models of reward‐seeking behavior (reviewed by (Botvinick et al. 2009)), in which VS‐OFC connections, critical for processing reward outcomes and learning to predict rewards, interact with a DLPFC‐mediated system for execution of goal‐directed behaviors thereby maximizing reward attainment through top‐down biasing of motor planning and output. A hierarchical reinforcement learning theory of ACC function extends these actor‐critic models of motivated behavior, by proposing ACC as a critical intermediary between the dorsal striatal‐DLPFC “actor” and ventral‐striatal‐OFC “critic” circuits (Holroyd et al. 2012). This model poses the RewP as the prediction error/striatal input into the ACC that encodes subjective value and theta oscillatory control signals as the ACC output to DLPFC that motivates behavior based on learned reward valuations (Holroyd and Umemoto 2016; Holroyd et al. 2012).
Given the difficulty of disentangling win from loss effects in time‐domain EEG due to overlapping componentry from the feedback win‐related positivity and the loss‐related N2 negativity, follow‐up jICA analyses were undertaken. These condition‐specific analyses separated Win (p = 0.25) from Near Miss loss (p = 0.25) waveforms by comparing each to a visual control of full‐losses (p = 0.50) for which feedback information was revealed at a previous reel. Upon qualitative observation of joint ICA component latencies, VS activations are elicited by both Win and Near Miss loss outcomes but occur ~50 ms earlier for Wins (and with stronger EEG–fMRI co‐modulation, based on magnitude of the JIC) than for Near Miss loss outcomes (shown in Figure 5). The larger and earlier VS EEG–fMRI comodulation to wins converges with prior observations that variability in the time‐domain win‐loss RewP appears to derive more from winning rather than the loss‐related negativity (with the latter construed as an N2 loss‐related negativity (Holroyd et al. 2008)). Moreover, condition‐specific jICAs segregated VS from dACC win‐related signaling into separate components, with dACC EEG–fMRI co‐modulation occurring later than VS and showing similar timing and JIC magnitude to that observed for the Near Miss losses (Figure 4).
Condition‐specific observations also contribute to a growing literature characterizing Near Miss brain responses. Despite economic valuation as a loss, Near Misses are unique in coming closer to wins than other loss types. Near Misses have been conceptualized within frustrative non‐reward frameworks, in which failure to obtain an anticipated reward invigorates reward‐seeking behavior (Amsel 1958). Prior single modality fMRI studies of slot machine play provide evidence for similar responses to Near Miss and Win outcomes in reward relevant regions including VS (Clark et al. 2009; Sescousse et al. 2016) and insula (Clark et al. 2009) and our own single modality fMRI findings support nearly equivalent Near Miss and Win fMRI responses, with circumscribed Win > Near Miss activation map differences in reward network regions limited to VS (shown in supplement). This indicates that in some respects, Near Miss outcomes can be construed as “almost winning” responses, as has been suggested previously (Dymond et al. 2014). In contrast to the near equivalence of Near Miss and Win fMRI responses, EEG studies from our own group (Fryer et al. 2021) and others (Kreussel et al. 2013; Lole et al. 2013; Luo et al. 2011; Ulrich and Hewig 2014) show clear differences in EEG‐based Win and Near Miss responses. Specifically, the RewP is modulated by reward proximity, demonstrating that “close” outcomes such as Near Misses are processed distinctly from both Win and “Total Miss” events (Kreussel et al. 2013; Lole et al. 2013; Luo et al. 2011; Ulrich and Hewig 2014). For example, we have previously shown in the same sample studied here that the RewP is larger for R3 (Win‐Near Miss) than R2 (Possible Win‐Total Miss), and that Near Miss losses elicit more theta power at R3 than full misses (Fryer et al. 2021). Though Wins and Near Misses show near equivalency, outside of VS, in single modality fMRI responses in our data, ERP components were useful in identifying Win > Near Miss response differences in multimodal analysis, including within the putative dACC generator of the RewP, which did not show single modality fMRI Win versus Near Miss activation differences in our single fMRI modality analyses. Our study's follow‐up analyses comparing Wins and Near‐Misses to Total Loss trials confirm that win‐related signaling mostly accounts for the larger, earlier Win > Near Miss component, as Wins were associated with an earlier, larger magnitude VS activation, whereas Near Misses yielded a JIC associated with a smaller extent of VS activation. In other words, while wins clearly generate the largest ERP and fMRI‐VS multimodal reward response in VS, a somewhat muted and later, but similar, reward response is evident to Near Misses. This suggests that Near Misses appear to also activate a reward response in the VS and likely reinforce ongoing slot machine play during real‐world gambling. Overall, these findings highlight the sensitivity of EEG/ERP measures to reward proximity and Near Miss effects, underscoring the utility of jointly considering EEG with fMRI signals in multimodal analysis.
5.2.2. Later Reward Feedback
A second Win > Near Miss reward evaluation component was identified, which temporally peaked ~100 ms later than the earlier VS‐dACC related JIC. This suggests that the RewP is a multi‐process component, with the peak of the RewP (i.e., the ERP difference wave) occurring when the two components overlap temporally. This later JIC covaried with a distinct set of neuroanatomical regions than the reward‐network focused earlier component; instead, the later component was primarily associated with distributed posterior cortical regions. These included bilateral temporo‐occipital cortex, posterior cingulate, and left parietal regions (precuneus and temporal–parietal–occipital junction) and occipital cortex (cuneus). Others have identified a later, frontocentral feedback‐associated ERP distinct from the RewP; for example, at ~380 ms post‐feedback that was sensitive to behavioral adjustment (Cockburn and Holroyd 2018). Though less well‐studied than the RewP, there is an outcome processing literature describing a feedback‐P3, which is a positive deflection ~300–600 ms post‐feedback, with a centro‐parietal maximum, that is thought to reflect motivational salience of outcome feedback (Glazer et al. 2018; San Martin 2012; Zheng et al. 2017). P3, in the context of reward feedback, has been disassociated from the valence‐sensitive RewP/FRN, with feedback‐P3 apparently sensitive to other aspects of outcome processing such as outcome magnitude (i.e., larger P3 amplitudes are observed in response to larger outcome magnitude) and/or behavioral adjustment (Yeung and Sanfey 2004), though see (San Martin 2012) for important nuances and caveats. We observe this P3‐like JIC in the setting of a probability matched comparison of Win and Near Miss loss events, and so while it is possible the functional significance of this JIC reflects aspects of salience detection during outcome processing, it is not likely to reflect signaling that is sensitive to probability effects. Similarly, the JIC for the Win‐Total Loss contrast showed a second component that temporally overlapped with the Near Miss‐Total Loss JIC, but interestingly, was not associated with VS activation. This suggests that this second later JIC component reflects later processing of the win, possibly including engagement of regions that mediate attentional focus on salient infrequent events. This is consistent with the later positivity captured by the second JIC reflecting a P300‐like response to the winning.
6. Conclusion
Characterizing neurocircuitry specific to sub‐component processes of reward responsivity is important for understanding reward system functioning in health and disease. The convergence between EEG and fMRI measures of reward processing is relatively understudied. A strength of the jICA data‐driven multimodality approach employed here is the ability to isolate temporo‐spatially precise signals that unfold along multistage reward processing pathways. While the binary distinction of anticipatory/consummatory (wanting vs. liking) is a useful heuristic, it is undoubtedly an oversimplification of the complex, interactive processes governing reward approach and receipt behaviors, which limits this work. Future multimodality work incorporating tasks that manipulate reward processing aspects unassessed in our study (including cue processing, effort valuation, reward learning and decision making as well as interactions with the motor planning and execution systems needed to obtain rewards) will help further refine the taxonomy of neural motivation and pleasure systems that govern positive valence and approach behaviors.
The reward‐related ERPs under investigation can be elicited by both rewarded (Gehring and Willoughby 2002) and unrewarded (Miltner et al. 1997) contexts and by a variety of task paradigms (e.g., time estimation (Brunia et al. 2011), monetary incentive delay (Novak and Foti 2015) and the doors task (Proudfit 2015)). For example, the literature includes both performance‐dependent and performance‐independent tasks, such as ours, in which reward outcome is not dependent on participant performance. Recognizing that reward processing occurs in diverse real‐world settings, it is valuable to have different types of reward tasks to comprehensively elicit reward‐related neural processes, and future studies combining different task paradigms, including performance‐dependent and performance‐independent assay of reward processing in the same study would be useful. Our findings are observed using a performance‐independent slot machine task, which facilitates a focused examination of the neural processes related to anticipating and receiving rewards in a context that minimizes task demands, such as decision‐making or motor planning and execution. Although the slot machine task used in this study was specifically selected to effectively isolate basic reward responsivity from higher‐order cognitive processes, it does present with some limitations. Our task design cannot resolve reward anticipation from other aspects of processing, such as frustrative non‐reward, that could be elaborated or distinct to near misses relative to full miss slot machine losses. We also acquired serial, rather than simultaneous, data to favor high‐quality EEG data collection, but our estimates of fMRI–EEG signal co‐variation may have been attenuated by collecting modalities in separate sessions. Another limitation is our likely truncated range of reward responsivity given our healthy adult sample for which we excluded lifetime mental illnesses. Further studies are needed to generalize the fMRI–EEG relationships described here, including the study of children and older adults to assess age‐related variation and extension of these findings to clinical populations characterized by hyper or hypo‐reward responsivity to extend study of these measures along the full phenotypic spectrum of reward sensitivity.
Our observations indicate EEG and fMRI comodulated signal during reward outcome processing that covaried with individual differences in trait reward sensitivity and support localization of the RewP to dACC and striatal regions. In contrast, the SPN covaried with fMRI signal in supplementary motor cortex and inferior frontal/insular regions that overlap with the salience network, and a later, possibly P3‐related, reward outcome JIC showed distinct neuroanatomical correlates involving temporal‐parieto‐occipital regions. Overall, our findings support the value of EEG–fMRI multimodal analysis, both in supporting inferences about the spatial source of EEG‐based reward measures, and conversely to improve attribution of fMRI reward processing signals to specific constituent sub‐processes through their covariance with precisely time locked event‐related potentials from the same task paradigm.
Funding
Research supported by VA CX001028 and CX001980 to Dr. Susanna L. Fryer.
Ethics Statement
This study was conducted in accordance with the Declaration of Helsinki.
Consent
This article reports research involving human subjects. All participants provided written informed consent under procedures approved by the University of California, San Francisco's Institutional Review Board. Recruitment meets scientific requirements & HBM's expectation of inclusivity.
Conflicts of Interest
All authors declare no current or recent disclosures and no conflicts of interest for this study. Drs. Susanna L. Fryer, Samantha Abram, Judith M. Ford, and Daniel H. Mathalon are U.S. Government employees. The content is solely the responsibility of the authors and does not necessarily represent the views of the Department of Veterans Affairs.
Supporting information
Data S1: hbm70466‐sup‐0001‐supinfo.docx.
Lau, K. J. , Roach B. J., Abram S., et al. 2026. “Parsing Reward Processing Substages With Multimodal EEG–fMRI .” Human Brain Mapping 47, no. 3: e70466. 10.1002/hbm.70466.
Data Availability Statement
Data will be made available upon request to Dr. Susanna L. Fryer, susanna.fryer@ucsf.edu. Raw data (EEG, fMRI, behavioral data) can be shared providing VA approves a data use agreement with the requesting party. All data sharing requests, foreign and domestic, will be reviewed for approval by a VA Privacy Officer to ensure compliance and that proper controls are in place for data use. Scripts used for data preparation can be found at: https://github.com/susannafryer/fryer_bieegl_public.
References
- Abram, S. V. , Roach B. J., Holroyd C. B., et al. 2020. “Reward Processing Electrophysiology in Schizophrenia: Effects of Age and Illness Phase.” NeuroImage. Clinical 28: 102,492. 10.1016/j.nicl.2020.102492. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Amsel, A. 1958. “The Role of Frustrative Nonreward in Noncontinuous Reward Situations.” Psychological Bulletin 55: 102–119. [DOI] [PubMed] [Google Scholar]
- Barch, D. M. , Pagliaccio D., and Luking K.. 2016. “Mechanisms Underlying Motivational Deficits in Psychopathology: Similarities and Differences in Depression and Schizophrenia.” Current Topics in Behavioral Neurosciences 27: 411–449. 10.1007/7854_2015_376. [DOI] [PubMed] [Google Scholar]
- Barton, K. R. , Yazdani A., Ayer N., et al. 2017. “The Effect of Losses Disguised as Wins and Near Misses in Electronic Gaming Machines: A Systematic Review.” Journal of Gambling Studies 33: 1241–1260. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Becker, M. P. , Nitsch A. M., Miltner W. H., and Straube T.. 2014. “A Single‐Trial Estimation of the Feedback‐Related Negativity and Its Relation to BOLD Responses in a Time‐Estimation Task.” Journal of Neuroscience 34: 3005–3012. 10.1523/JNEUROSCI.3684-13.2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Behzadi, Y. , Restom K., Liau J., and Liu T. T.. 2007. “A Component Based Noise Correction Method (CompCor) for BOLD and Perfusion Based fMRI.” NeuroImage 37: 90–101. 10.1016/j.neuroimage.2007.04.042. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Berridge, K. C. , and Kringelbach M. L.. 2015. “Pleasure Systems in the Brain.” Neuron 86: 646–664. 10.1016/j.neuron.2015.02.018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Berridge, K. C. , and Robinson T. E.. 2003. “Parsing Reward.” Trends in Neurosciences 26: 507–513. 10.1016/S0166-2236(03)00233-9. [DOI] [PubMed] [Google Scholar]
- Botvinick, M. M. , Niv Y., and Barto A. G.. 2009. “Hierarchically Organized Behavior and Its Neural Foundations: A Reinforcement Learning Perspective.” Cognition 113: 262–280. 10.1016/j.cognition.2008.08.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Brunia, C. H. M. , Hackley S. A., van Boxtel G. J. M., Kotani Y., and Ohgami Y.. 2011. “Waiting to Perceive: Reward or Punishment?” Clinical Neurophysiology 122: 858–868. 10.1016/j.clinph.2010.12.039. [DOI] [PubMed] [Google Scholar]
- Calhoun, V. D. , Adali T., Giuliani N. R., Pekar J. J., Kiehl K. A., and Pearlson G. D.. 2006. “Method for Multimodal Analysis of Independent Source Differences in Schizophrenia: Combining Gray Matter Structural and Auditory Oddball Functional Data.” Human Brain Mapping 27: 47–62. 10.1002/hbm.20166. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Carlson, J. M. , Foti D., Mujica‐Parodi L. R., Harmon‐Jones E., and Hajcak G.. 2011. “Ventral Striatal and Medial Prefrontal BOLD Activation Is Correlated With Reward‐Related Electrocortical Activity: A Combined ERP and fMRI Study.” NeuroImage 57: 1608–1616. 10.1016/j.neuroimage.2011.05.037. [DOI] [PubMed] [Google Scholar]
- Carver, C. , and White T. L.. 1994. “Behavioral Inhibition, Behavioral Activation, and Affective Responses to Impending Reward and Punishment: The BIS/BAS Scales.” Journal of Personality and Social Psychology 67: 319–333. 10.1037//0022-3514.67.2.319. [DOI] [Google Scholar]
- Clark, L. , Lawrence A. J., Astley‐Jones F., and Gray N.. 2009. “Gambling Near‐Misses Enhance Motivation to Gamble and Recruit Win‐Related Brain Circuitry.” Neuron 61: 481–490. 10.1016/j.neuron.2008.12.031. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cockburn, J. , and Holroyd C. B.. 2018. “Feedback Information and the Reward Positivity.” International Journal of Psychophysiology 132: 243–251. 10.1016/j.ijpsycho.2017.11.017. [DOI] [PubMed] [Google Scholar]
- Diekhof, E. K. , Kaps L., Falkai P., and Gruber O.. 2012. “The Role of the Human Ventral Striatum and the Medial Orbitofrontal Cortex in the Representation of Reward Magnitude—An Activation Likelihood Estimation Meta‐Analysis of Neuroimaging Studies of Passive Reward Expectancy and Outcome Processing.” Neuropsychologia 50: 1252–1266. 10.1016/j.neuropsychologia.2012.02.007. [DOI] [PubMed] [Google Scholar]
- Donkers, F. C. , Nieuwenhuis S., and van Boxtel G. J.. 2005. “Mediofrontal Negativities in the Absence of Responding.” Brain Research. Cognitive Brain Research 25: 777–787. 10.1016/j.cogbrainres.2005.09.007. [DOI] [PubMed] [Google Scholar]
- Donkers, F. C. , and Van Boxtel G. J.. 2005. “Mediofrontal Negativities to Averted Gains and Losses in the Slot‐Machine Task: A Further Investigation.” Journal of Psychophysiology 19: 256–262. [Google Scholar]
- Dymond, S. , Lawrence N. S., Dunkley B. T., et al. 2014. “Almost Winning: Induced MEG Theta Power in Insula and Orbitofrontal Cortex Increases During Gambling Near‐Misses and Is Associated With BOLD Signal and Gambling Severity.” NeuroImage 91: 210–219. 10.1016/j.neuroimage.2014.01.019. [DOI] [PubMed] [Google Scholar]
- First, M. , Spitzer R., Gibbon M., and Williams J.. 2002. “Structured Clinical Interview for DSM‐IV‐TR Axis I Disorders, Research Version, Patient Edition. (SCID‐I/P).” New York State Psychiatric Institute.
- Fischl, B. 2012. “FreeSurfer.” NeuroImage 62: 774–781. 10.1016/j.neuroimage.2012.01.021. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ford, J. M. , Roach B. J., Palzes V. A., and Mathalon D. H.. 2016. “Using Concurrent EEG and fMRI to Probe the State of the Brain in Schizophrenia.” NeuroImage. Clinical 12: 429–441. 10.1016/j.nicl.2016.08.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Foti, D. , Weinberg A., Bernat E. M., and Proudfit G. H.. 2015. “Anterior Cingulate Activity to Monetary Loss and Basal Ganglia Activity to Monetary Gain Uniquely Contribute to the Feedback Negativity.” Clinical Neurophysiology 126: 1338–1347. 10.1016/j.clinph.2014.08.025. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fryer, S. L. , Roach B. J., Holroyd C. B., et al. 2021. “Electrophysiological Investigation of Reward Anticipation and Outcome Evaluation During Slot Machine Play.” NeuroImage 232: 117874. 10.1016/j.neuroimage.2021.117874. [DOI] [PubMed] [Google Scholar]
- Gehring, W. J. , and Willoughby A. R.. 2002. “The Medial Frontal Cortex and the Rapid Processing of Monetary Gains and Losses.” Science 295: 2279–2282. 10.1126/science.1066893. [DOI] [PubMed] [Google Scholar]
- Glazer, J. E. , Kelley N. J., Pornpattananangkul N., Mittal V. A., and Nusslock R.. 2018. “Beyond the FRN: Broadening the Time‐Course of EEG and ERP Components Implicated in Reward Processing.” International Journal of Psychophysiology 132: 184–202. 10.1016/j.ijpsycho.2018.02.002. [DOI] [PubMed] [Google Scholar]
- Haber, S. N. , and Knutson B.. 2010. “The Reward Circuit: Linking Primate Anatomy and Human Imaging.” Neuropsychopharmacology 35: 4–26. 10.1038/npp.2009.129. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Habib, R. , and Dixon M. R.. 2010. “Neurobehavioral Evidence for the “Near‐Miss” Effect in Pathological Gamblers.” Journal of the Experimental Analysis of Behavior 93: 313–328. 10.1901/jeab.2010.93-313. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Holroyd, C. B. , HajiHosseini A., and Baker T. E.. 2012. “ERPs and EEG Oscillations, Best Friends Forever: Comment on Cohen et al.” Trends in Cognitive Sciences 16: 192–193. 10.1016/j.tics.2012.02.008. [DOI] [PubMed] [Google Scholar]
- Holroyd, C. B. , Pakzad‐Vaezi K. L., and Krigolson O. E.. 2008. “The Feedback Correct‐Related Positivity: Sensitivity of the Event‐Related Brain Potential to Unexpected Positive Feedback.” Psychophysiology 45: 688–697. 10.1111/j.1469-8986.2008.00668.x. [DOI] [PubMed] [Google Scholar]
- Holroyd, C. B. , and Umemoto A.. 2016. “The Research Domain Criteria Framework: The Case for Anterior Cingulate Cortex.” Neuroscience and Biobehavioral Reviews 71: 418–443. 10.1016/j.neubiorev.2016.09.021. [DOI] [PubMed] [Google Scholar]
- Jacob, M. S. , Ford J. M., Roach B. J., Calhoun V. D., and Mathalon D. H.. 2019. “Aberrant Activity in Conceptual Networks Underlies N400 Deficits and Unusual Thoughts in Schizophrenia.” NeuroImage. Clinical 24: 101960. 10.1016/j.nicl.2019.101960. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kotani, Y. , Ohgami Y., Ishiwata T., Arai J., Kiryu S., and Inoue Y.. 2015. “Source Analysis of Stimulus‐Preceding Negativity Constrained by Functional Magnetic Resonance Imaging.” Biological Psychology 111: 53–64. 10.1016/j.biopsycho.2015.08.005. [DOI] [PubMed] [Google Scholar]
- Kreussel, L. , Hewig J., Kretschmer N., Hecht H., Coles M. G., and Miltner W. H.. 2013. “How Bad Was It? Differences in the Time Course of Sensitivity to the Magnitude of Loss in Problem Gamblers and Controls.” Behavioural Brain Research 247: 140–145. 10.1016/j.bbr.2013.03.024. [DOI] [PubMed] [Google Scholar]
- Lole, L. , Gonsalvez C. J., Barry R. J., and De Blasio F. M.. 2013. “Can Event‐Related Potentials Serve as Neural Markers for Wins, Losses, and Near‐Wins in a Gambling Task? A Principal Components Analysis.” International Journal of Psychophysiology 89: 390–398. 10.1016/j.ijpsycho.2013.06.011. [DOI] [PubMed] [Google Scholar]
- Lopez‐Calderon, J. , and Luck S. J.. 2014. “ERPLAB: An Open‐Source Toolbox for the Analysis of Event‐Related Potentials.” Frontiers in Human Neuroscience 8: 213. 10.3389/fnhum.2014.00213. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Luo, Q. , Wang Y., and Qu C.. 2011. “The Near‐Miss Effect in Slot‐Machine Gambling: Modulation of Feedback‐Related Negativity by Subjective Value.” Neuroreport 22: 989–993. 10.1097/WNR.0b013e32834da8ae. [DOI] [PubMed] [Google Scholar]
- Menon, V. , and Uddin L. Q.. 2010. “Saliency, Switching, Attention and Control: A Network Model of Insula Function.” Brain Structure & Function 214: 655–667. 10.1007/s00429-010-0262-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Miltner, W. H. , Braun C. H., and Coles M. G.. 1997. “Event‐Related Brain Potentials Following Incorrect Feedback in a Time‐Estimation Task: Evidence for a “Generic” Neural System for Error Detection.” Journal of Cognitive Neuroscience 9: 788–798. 10.1162/jocn.1997.9.6.788. [DOI] [PubMed] [Google Scholar]
- Mognon, A. , Jovicich J., Bruzzone L., and Buiatti M.. 2011. “ADJUST: An Automatic EEG Artifact Detector Based on the Joint Use of Spatial and Temporal Features.” Psychophysiology 48: 229–240. 10.1111/j.1469-8986.2010.01061.x. [DOI] [PubMed] [Google Scholar]
- Nachev, P. , Kennard C., and Husain M.. 2008. “Functional Role of the Supplementary and Pre‐Supplementary Motor Areas.” Nature Reviews. Neuroscience 9: 856–869. 10.1038/nrn2478. [DOI] [PubMed] [Google Scholar]
- Nieuwenhuis, S. , Slagter H. A., von Geusau N. J., Heslenfeld D. J., and Holroyd C. B.. 2005. “Knowing Good From Bad: Differential Activation of Human Cortical Areas by Positive and Negative Outcomes.” European Journal of Neuroscience 21: 3161–3168. 10.1111/j.1460-9568.2005.04152.x. [DOI] [PubMed] [Google Scholar]
- Nolan, H. , Whelan R., and Reilly R. B.. 2010. “FASTER: Fully Automated Statistical Thresholding for EEG Artifact Rejection.” Journal of Neuroscience Methods 192: 152–162. 10.1016/j.jneumeth.2010.07.015. [DOI] [PubMed] [Google Scholar]
- Novak, K. D. , and Foti D.. 2015. “Teasing Apart the Anticipatory and Consummatory Processing of Monetary Incentives: An Event‐Related Potential Study of Reward Dynamics.” Psychophysiology 52: 1470–1482. 10.1111/psyp.12504. [DOI] [PubMed] [Google Scholar]
- Nusslock, R. , and Alloy L. B.. 2017. “Reward Processing and Mood‐Related Symptoms: An RDoC and Translational Neuroscience Perspective.” Journal of Affective Disorders 216: 3–16. 10.1016/j.jad.2017.02.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Oerlemans, J. , Alejandro R. J., Van Roost D., et al. 2025. “Unravelling the Origin of the Reward Positivity: A Human Intracranial Event‐Related Brain Potential Study.” Brain: A Journal of Neurology 148: 199–211. 10.1093/brain/awae259. [DOI] [PubMed] [Google Scholar]
- Parke, J. , and Griffiths M.. 2006. “The Psychology of the Fruit Machine: The Role of Structural Characteristics (Revisited).” International Journal of Mental Health and Addiction 4: 151–179. [Google Scholar]
- Proudfit, G. H. 2015. “The Reward Positivity: From Basic Research on Reward to a Biomarker for Depression.” Psychophysiology 52: 449–459. 10.1111/psyp.12370. [DOI] [PubMed] [Google Scholar]
- Reid, R. L. 1996. “The Psychology of the Near Miss.” Journal of Gambling Behavior 2: 32–39. [Google Scholar]
- Rissanen, J. 1983. “A Universal Prior for Integers and Estimation by Minimum Description Length.” Annals of Statistics 11: 416–431. [Google Scholar]
- Sambrook, T. D. , and Goslin J.. 2015. “A Neural Reward Prediction Error Revealed by a Meta‐Analysis of ERPs Using Great Grand Averages.” Psychological Bulletin 141: 213–235. 10.1037/bul0000006. [DOI] [PubMed] [Google Scholar]
- San Martin, R. 2012. “Event‐Related Potential Studies of Outcome Processing and Feedback‐Guided Learning.” Frontiers in Human Neuroscience 6: 304. 10.3389/fnhum.2012.00304. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Seeley, W. W. , Menon V., Schatzberg A. F., et al. 2007. “Dissociable Intrinsic Connectivity Networks for Salience Processing and Executive Control.” Journal of Neuroscience 27: 2349–2356. 10.1523/JNEUROSCI.5587-06.2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sescousse, G. , Janssen L. K., Hashemi M. M., et al. 2016. “Amplified Striatal Responses to Near‐Miss Outcomes in Pathological Gamblers.” Neuropsychopharmacology 41: 2614–2623. 10.1038/npp.2016.43. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Shirer, W. R. , Ryali S., Rykhlevskaia E., Menon V., and Greicius M. D.. 2012. “Decoding Subject‐Driven Cognitive States With Whole‐Brain Connectivity Patterns.” Cerebral Cortex 22: 158–165. 10.1093/cercor/bhr099. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sui, J. , Adali T., Yu Q., Chen J., and Calhoun V. D.. 2012. “A Review of Multivariate Methods for Multimodal Fusion of Brain Imaging Data.” Journal of Neuroscience Methods 204: 68–81. 10.1016/j.jneumeth.2011.10.031. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ulrich, N. , and Hewig J.. 2014. “A Miss Is as Good as a Mile? Processing of Near and Full Outcomes in a Gambling Paradigm.” Psychophysiology 51: 819–823. 10.1111/psyp.12232. [DOI] [PubMed] [Google Scholar]
- Wager, T. D. , and Nichols T. E.. 2003. “Optimization of Experimental Design in fMRI: A General Framework Using a Genetic Algorithm.” NeuroImage 18: 293–309. [DOI] [PubMed] [Google Scholar]
- Whitton, A. E. , Kumar P., Treadway M. T., et al. 2023. “Distinct Profiles of Anhedonia and Reward Processing and Their Prospective Associations With Quality of Life Among Individuals With Mood Disorders.” Molecular Psychiatry 28: 5272–5281. 10.1038/s41380-023-02165-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wilson, R. P. , Colizzi M., Bossong M. G., et al. 2018. “The Neural Substrate of Reward Anticipation in Health: A Meta‐Analysis of fMRI Findings in the Monetary Incentive Delay Task.” Neuropsychology Review 28: 496–506. 10.1007/s11065-018-9385-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Yeung, N. , and Sanfey A. G.. 2004. “Independent Coding of Reward Magnitude and Valence in the Human Brain.” Journal of Neuroscience 24: 6258–6264. 10.1523/JNEUROSCI.4537-03.2004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Zheng, Y. , Li Q., Zhang Y., et al. 2017. “Reward Processing in Gain Versus Loss Context: An ERP Study.” Psychophysiology 54: 1040–1053. 10.1111/psyp.12855. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data S1: hbm70466‐sup‐0001‐supinfo.docx.
Data Availability Statement
Data will be made available upon request to Dr. Susanna L. Fryer, susanna.fryer@ucsf.edu. Raw data (EEG, fMRI, behavioral data) can be shared providing VA approves a data use agreement with the requesting party. All data sharing requests, foreign and domestic, will be reviewed for approval by a VA Privacy Officer to ensure compliance and that proper controls are in place for data use. Scripts used for data preparation can be found at: https://github.com/susannafryer/fryer_bieegl_public.
