Belief updating in uncertain environments is differentially sensitive to reward and punishment learning: Evidence from ERP

Summary

Learning from rewards and punishments relies on prediction errors and belief updating, yet it remains unclear how motivational context reshapes the precision-weighting of prediction errors and neural mechanism while learning in volatile environments. We employed a probabilistic classification task with electroencephalography (EEG) within a hierarchical Bayesian framework to compare reward and punishment learning. Our findings indicate that participants performed better in reward context compared to punishment context. Fitting the hierarchical Bayesian model revealed that punishment drives faster Bayesian belief updates, although these did not translate into improved behavioral outcomes. At the neural level, higher-level precision-weighted prediction error (pwPE₂) was significantly positively correlated with feedback related negativity (FRN) amplitude in the punishment context, and the positive effect of pwPE₂ on P300 amplitude was stronger in the punishment than the reward condition. These results reveal the characteristics of electrophysiological activities under different motivational contexts and highlight differences in the neural mechanisms underlying reward and punishment learning.

Graphical abstract

Highlights

• •A stable “reward advantage” in a reward context but depending on environmental volatility
• •Faster belief updating under punishment did not translate into better behavior
• •The coupling of learning parameters to ERPs was context-dependent

Neuroscience; Sensory neuroscience; Cognitive neuroscience; Techniques in neuroscience

Introduction

Reward and punishment learning are two critical facets of human and animal behavior, enabling successful adaptation to environmental changes by pursuing rewards and avoiding punishments.¹ Reinforcement learning theory provides a theoretical framework for understanding the underlying mechanisms of reward and punishment learning, suggesting that individuals extract patterns from the environment through trial-and-error and adjust their behavior accordingly, highlighting their similarities. But previous studies have found that rewards and punishments have different effects on individuals’ behaviors. Specifically, a reward context promotes individuals’ persistence in current choices to maximize gains, whereas a punishment context increases the likelihood of switching after a negative outcome.^2,3,4 Neuroimaging studies have discovered from a neurophysiological perspective that learning from reward or punishment engages distinct neural substrates. Reward learning relies on mesolimbic dopaminergic projections, whereas punishment learning is associated with serotonergic projections.^5,6,7,8 Previous studies have shown that rewards and punishments are different in neural mechanisms. The current study focuses on contrasting the differences between reward and punishment learning, specifically in terms of their computational and cognitive neural mechanisms. To better differentiate between reward and punishment, we integrated a hierarchical Bayesian model with electroencephalography (EEG) to examine how humans implement reward and punishment learning in dynamic environments.

The Probabilistic Reversal Learning paradigm is a widely used hierarchical learning framework in learning research,^9,10 designed to simulate environments with unstable cue-outcome relationships. To maximize the probability of obtaining rewards or avoiding punishments, participants must learn the statistical associations between cues and outcomes through trial-by-trial learning. When environmental contingencies reverse, previously learned value estimates become obsolete, and the learning rate critically influences behavioral adaptation: a low learning rate may result in sluggish adjustments, while an excessively high learning rate can lead to unstable updates driven by noisy feedback.^11,12 Consequently, this task is able to capture the impact of environmental volatility on learning behavior. Its dynamic nature makes it particularly suitable for studying the cognitive processes underlying reward and punishment learning, leading to its extensive application in both basic learning research and studies of psychiatric disorders such as autism, Parkinson disease, schizophrenia, and anxiety.^13,14,15,16

With respect to Probabilistic Reversal Learning paradigm, studies of reward and punishment learning suggest that individuals exhibit marked asymmetries in task performance and learning strategies when facing reward versus punishment contexts.^{13,14,17,18,19} At the behavioral level, participants demonstrated different flexibility when approaching rewards and avoiding punishments. On the one hand, individuals show less flexibility in avoiding punishment than in tasks that require pursuing rewards¹⁸ and have a higher learning rate under punishment tasks.¹⁷ On the other hand, individuals show a greater emphasis on avoiding punishment than on pursuing rewards,¹⁹ but their flexibility decreases instead. These opposing effects of motivational context become even more pronounced in special populations: Parkinson patients off medication exhibit profound reward-learning deficits that are ameliorated by dopamine agonists at the cost of impaired punishment learning,¹³ and adolescents with conduct disorder show selective impairments in avoiding punishment despite intact reward learning,¹⁴ underscoring that reward and punishment processing depend on at least partially separable neural systems. Nevertheless, purely behavioral measures cannot reveal how individuals dynamically adjust learning rates or encode prediction errors under different motivational demands; integrating computational modeling with EEG is therefore essential to uncover the temporal and mechanistic basis of these behavioral asymmetries.

Computational models capture the dynamic of learning over time—emulating a participant’s experience—and enable better mapping between behavior and neurobiology.^9,20 In this vein, Mathys et al. proposed a hierarchical Bayesian framework (Hierarchical Gaussian Filter [HGF]).^21,22 This model stands out for its robust explanatory power in capturing dynamic estimation of prediction errors and belief updates in uncertain environments. Its key parameters—such as learning rates (LRs; controlling the speed of belief updates), precision-weighted prediction error (pwPE; quantifying the discrepancy between expected and actual outcomes), and inverse temperature parameter (ζ; reflecting decision randomness)—provide a robust framework for quantitative analysis. These parameters not only correlate with behavioral performance (e.g., accuracy)^23,24 but also, when combined with neuroimaging or EEG, reveal the neural underpinnings of prediction errors at different hierarchical levels.^{23,25,26,27,28} For instance, Iglesias et al. utilized a probabilistic reversal learning paradigm with functional magnetic resonance imaging (fMRI) and found that low-level precision-weighted prediction errors were associated with midbrain activation (ventral tegmental area [VTA]/substantia nigra [SN]),²⁶ while high-level prediction errors correlated with activity in the septal region of the cholinergic basal forebrain, indicating the involvement of multiple neural systems in reward and punishment learning. Further, EEG studies have extended the link between HGF parameters and feedback processing. Liu et al. combined HGF with EEG and found that the P300 component independently encodes low-level pwPE₂ (related to specific stimulus-outcome associations) and high-level pwPE₃ (related to environmental volatility), with low-level pwPE₂ also associated with enhanced theta-band (4–8 Hz) power, suggesting theta activity’s role in processing specific prediction errors.²⁷ Similarly, Hein et al. reported that pwPE₂ modulated centroparietal P300 responses in reward learning tasks, further supporting that P300 is sensitive to reward-related prediction errors.²⁵ Collectively, these studies demonstrate that HGF parameters not only quantify behavioral belief adjustments but also link to the multi-level characteristics of neural feedback processing, revealing that learning may rely on distinct internal mechanisms. Accordingly, this study focuses on the differential impacts of reward and punishment frameworks on learning performance, specifically examining whether systematic differences exist in behavioral outcomes and belief-updating mechanisms under volatile environments.

EEG with excellent temporal resolution is able to provide a temporal lens through the roles of feedback-related event-related potential (ERP) components within a two-stage processing model.^{25,27,29,30,31} A key challenge for interpreting feedback-related ERPs is that their amplitudes can be strongly context-dependent. The FRN, peaking around 200–300 ms, is the most extensively studied reward-specific ERP component, serving as a direct marker of reward prediction error (RPE) within reinforcement learning frameworks. But the FRN does not simply scale with the absolute value of an outcome; instead, it is sensitive to the contextual range of possible outcomes and to whether an outcome is better or worse than expected within that range.³² This is particularly relevant for contrasting win- and loss-framed learning, where identical numerical outcomes may carry different relative meanings. In the present study, we test this account using trial-wise precision-weighted prediction errors derived from a hierarchical Bayesian learner. FRN amplitude correlates with the discrepancy between expected and actual outcomes: unexpected rewards elicit a positive deflection (termed reward positivity), while unexpected losses amplify negative deflections.^29,33,34,35 Feedback-locked FRN responses vary based on whether expectations are confirmed or violated,^29,36 with stronger amplitudes when outcomes result from self-initiated actions.³⁷ Westwood and Philiastides demonstrated that an early EEG signal (∼220 ms) differentiates reward and punishment contexts, with its amplitude predicting individual differences in decision-making accuracy.²⁴ This finding may suggest that early feedback-related EEG activity indexes motivational salience or rapid outcome evaluation, although such functional interpretations remain context dependent.

Additionally, the P300 component, occurring between 300 and 600 ms, is associated with attention-driven categorization of salient outcomes and the motivational significance of reward feedback, reflecting emotional and cognitive processing.^38,39 Recent studies have expanded its role to include learning and belief updating in volatile environments.^25,40,41 For instance, Hein et al. demonstrated that P300 amplitude is sensitive to prediction errors in reward learning tasks,²⁵ suggesting a close link between this component and adaptive feedback processing. Consistent with this view, Nassar et al. and Yu et al. reported that feedback-locked P300 activity predicts subsequent learning behavior in changing environments.^40,41 Importantly, Nassar et al. further clarified that the P300 does not merely signal unexpected outcomes but dynamically modulates learning: it facilitates adaptation when environmental changes are relevant and suppresses learning when outcomes reflect incidental noise.⁴⁰ These findings underscore P300’s role in integrating feedback for value updating and strategic adaptation. Taken together, prior work has often interpreted frontocentral FRN activity as reflecting early outcome evaluation and centro-parietal P300 activity as reflecting later attention- and learning-related processing. In the present study, we used this framework as a heuristic account, while acknowledging that FRN and P300 are functionally heterogeneous and that our data do not directly isolate salience versus prediction-error versus belief-updating mechanisms. Whether the parameters of HGF model modulate the FRN and P300 components and whether this modulation is separated between the reward and punishment contexts remain to be seen.

In summary, the current study employs a probabilistic reversal learning paradigm, combined with HGF modeling and trial-by-trial EEG analysis, to examine reward and punishment learning under uncertainty at behavioral, computational, and neural levels. We specifically asked whether reward and punishment contexts differ in behavioral performance, latent belief-updating parameters, and their trial-wise electrophysiological correlates. We put forward the following hypotheses: (1) behaviorally, punishment contexts may decrease behavioral flexibility and enhance learning rates and prediction errors in the HGF framework; (2) neurally, HGF parameters may have different effects on FRN and P300 under reward and punishment contexts, respectively, reflecting the distinct neural mechanisms underlying reward and punishment learning. Specifically, the parameter modulation on FRN amplitude (i.e., detecting early salience) would be stronger in the punishment than the reward context, reflecting enhancing sensitivity to feedback in punishment context. Additionally, the parameter modulation on P300 amplitude (i.e., value updating) would be more prominent in the punishment than the reward context, reflecting faster belief updates in the punishment context.

Results

Thirty-six healthy participants completed a within-subject 2 (context: reward, punishment) × 2 (volatility: stable, volatile) design in a probabilistic categorization learning task. One participant did not complete the experiment, and another was excluded due to excessive EEG artifacts. Thus, the final sample consisted of 34 participants (16 males); each participant performed four blocks (two reward and two punishment blocks; 160 trials per block). In the reward context, correct responses yielded gains (+25 points), whereas incorrect responses yielded neutral outcomes (+0); in the punishment context, correct responses avoided losses (−0), whereas incorrect responses incurred losses (−25). Within each block, a stable phase with fixed cue-outcome probabilities preceded a volatile phase with multiple contingency reversals (Figure 1A). We first report accuracy and post-error switching behavior [P(RuleSwitch)] using generalized linear mixed-effects models. We then compare computational learning models at the group level and summarize key HGF-derived parameters/latent trajectories. Finally, we link trial-by-trial-model-derived prediction-error signals to feedback-locked ERP components (FRN and P300) using linear mixed-effects models to test how motivational context and volatility shape neural encoding of learning signals.

Figure 1.

Schematic illustration of the experiment and model structure

(A) Schematic diagram of the experiment. In a single trial, the fixation point was first presented to the participants, then the stimulus picture was presented, and the participants were required to determine which category the picture belonged to. After the keypress response, a blank screen was presented, and finally the feedback result was presented. Point settings in reward and punishment contexts: in the reward context, correct responses add 25 points, while incorrect responses add 0 points; in the punishment context, correct responses subtract 0 points, while incorrect responses subtract 25 points. Cue-outcome contingency of images over time in the experiment: In the first 160 trials, images S1 and S2 were presented. In the stable environment, the probability of S1 belonging to category A was fixed (0.75/0.25), whereas in the volatile environment, the probability shifted to 0.8/0.2 and reversed four times. The remaining trials followed a similar structure.

(B) Schematic of the model structure. The HGF model contains a three-layer structure (left). In the fit of a single participant (right), at the lowest level, u (blue dots) represents the cue-outcome association for each trial, and y (orange dots) represents the participant’s response (1 for S1 = A & S2 = B; 0 for S1 = B & S2 = A). The red line, s(μ₂), indicates the participant’s estimated cue-outcome tendency x₂, calculated using a sigmoid function. The learning rate for the cue-outcome association at the lowest level is depicted by a black line. At the second level, μ₂ (green line) represents the participant’s estimate of the cue-outcome tendency x₂. At the highest level, μ₃ (blue line) represents the estimate of volatility x₃.

Behavioral results

In the task, the stable environment maintained a fixed dominant-category probability of 0.75 (non-dominant = 0.25), whereas in the volatile environment cue-outcome probabilities alternated between 0.8 and 0.2 across four reversals. For specific details, please refer to the “materials and procedure” section. We first report the behavioral effects of context and environmental type on accuracy and post-error switching.

Accuracy

The fitting results of the GLMM regarding the accuracy are shown in Table S1 (see supplemental information). Participants exhibited superior performance in the reward context compared to the punishment context: under stable conditions, accuracy was significantly lower in the punishment context (β = −0.22, SE = 0.05, z = −4.20, p < 0.001) relative to the reward context. Environmental volatility also impaired performance, with accuracy significantly reduced in volatile environments within the reward context (β = −0.52, SE = 0.05, z = −10.43, p < 0.001). Critically, a significant interaction between context and environment type emerged (β = 0.31, SE = 0.07, z = 4.39, p < 0.001). To further interpret the significant context × volatility interaction, we conducted follow-up simple-effects analyses within each context. In the reward context, accuracy was significantly higher in the stable than in the volatile condition (odds ratio = 1.69, SE = 0.084, z = 10.44, p < 0.0001). A similar effect was also observed in the punishment context, with significantly higher accuracy in the stable than in the volatile condition (odds ratio = 1.24, SE = 0.061, z = 4.44, p < 0.0001). We further compared reward and punishment contexts separately within each environment. In the stable environment, accuracy was significantly higher in the reward than in the punishment context (odds ratio = 1.24, SE = 0.09, z = 2.95, p = 0.003), whereas in the volatile environment the reward-punishment difference was not significant (odds ratio = 0.91, SE = 0.06, z = −1.34, p = 0.18). These results indicate that volatility significantly reduced accuracy in both contexts, although the magnitude of this effect was larger in the reward than in the punishment context (Figure 2A).

Figure 2.

Behavioral results

(A) GLMM analysis of accuracy revealed a significant interaction between environmental type and context. Data are represented as mean ± SD.

(B) Paired-sample t -tests indicated a significant difference in accuracy between reward and punishment contexts. Data are represented as mean ± SEM.

(C) GLMM analysis of post-task switching rate P(RuleSwitch) revealed a significant interaction between environmental type and context. Data are represented as mean ± SD.

(D) Paired-sample t tests indicated a significant difference in P(RuleSwitch) between reward and punishment contexts. Data are represented as mean ± SEM. ∗p < 0.05, ∗∗p < 0.01.

For completeness, we analyzed ΔAccuracy using the paired t test. The analysis results show that, ΔAccuracy values were negative in both conditions, with ΔAccuracy in the reward context (M = −0.08 ± 0.02) being significantly lower than in the punishment context (M = −0.03 ± 0.02), t (34) = −3.01, p = 0.005 (Figure 2B). This pattern suggests that participants were less flexible in avoiding punishment than in pursuing reward, highlighting an asymmetry in adaptive adjustments under different motivational contexts (Figure 2B).

P(RuleSwitch)

GLMM fitting results for the probability of switching rules after error feedback (P(RuleSwitch)) are presented in Table S2. Participants showed a lower propensity to switch in the reward context compared to the punishment context, as evidenced by a significant main effect of task type: under stable conditions, the probability of switching was markedly higher in the punishment context (β = 0.3, SE = 0.06, z = 5.11, p < 0.001) relative to the reward context. Environmental volatility also increased switching, with the probability significantly elevated in volatile environments within the reward context (β = 0.16, SE = 0.06, z = 2.63, p = 0.008). Critically, a significant interaction between task type and environment type emerged (β = −0.23, SE = 0.08, z = −2.82, p = 0.005). Follow-up simple-effects analyses showed that in the reward context, the probability of switching after error feedback was significantly higher in the volatile than in the stable condition (odds ratio for stable vs. volatile = 0.856, SE = 0.051, z = −2.63, p = 0.0084), indicating increased switching under volatility. In contrast, the effect of volatility was not significant in the punishment context (odds ratio = 1.077, SE = 0.060, z = 1.32, p = 0.1856). We similarly further compared reward and punishment differences within each environment. In the stable environment, the probability of switching was significantly lower in the reward than in the punishment context (odds ratio = 0.78, SE = 0.07, z = −2.96, p = 0.003), whereas in the volatile environment the reward-punishment difference was not significant (odds ratio = 0.98, SE = 0.08, z = −0.26, p = 0.79). Thus, volatility significantly increased post-error switching in the reward context but not in the punishment context (Figure 2C).

Similarly, we used the paired sample t test to analyze ΔP(RuleSwitch). The results showed that ΔP(RuleSwitch) in the reward context (M = 0.017 ± 0.01) was significantly higher than in the punishment context (M = −0.01 ± 0.01), t (34) = 2.41, p = 0.022 (Figure 2D). This finding further supports the notion that participants exhibited reduced flexibility in adapting their behavior to avoid punishment compared to pursuing reward.

Model fitting

To characterize latent learning dynamics more directly, we fitted computational models separately for each block and compared three candidate models: the HGF, RW, and SK1 models. The RW model (Rescorla-Wagner model) is the simplest model framework.⁴² Its core idea is that learning driven by expectation error occurs when an individual observes the behavioral outcome. It assumes that expectation error drives belief renewal at a fixed learning rate. The SK1 model incorporates an adaptive learning rate but lacks a hierarchical structure.⁴³ On this basis, the HGF model that has emerged in recent years provides a computational framework of hierarchical concepts for understanding the adaptive learning process in uncertain environments.^21,22 This model integrates the advantages of reinforcement learning and Bayesian learning and can infer the belief update trajectory experienced by an individual from the dynamic interaction process with an uncertain environment (see Figure 1B and method details).

We fitted each model using response data from 34 participants and conducted a group-level comparison of the three models using random-effects Bayesian model selection (BMS).⁴⁴ The log-model evidence (LME) values obtained from model fitting were used as random effects, implemented via the SPM12 toolkit (https://www.fil.ion.ucl.ac.uk/spm/). This approach estimated the expected posterior probabilities (EXP_P), representing the probability that a randomly selected participant’s data are best explained by a given model, as well as the exceedance probability (XP) and protected exceedance probability (PXP), which indicate the probability that a given model outperforms all other models in the model space (see Table 1). The results demonstrated that the HGF model outperformed the other two models, and thus, we selected the HGF model as the optimal model for subsequent analyses.

Table 1.

Model comparison

	HGF	RW	SK1
EXP_P	0.98	0.01	0.01
XP	1	0	0
PXP	1	0	0

Model parameters

For the probabilistic learning rate (learningrate₂, LR₂), the absolute value of the precision-weighted prediction error (|pwPE₂|) and the inverse temperature parameter (ζ, which indexes choice stochasticity in the response model, with larger values reflecting more deterministic choices), we conducted paired t tests. For LR₂, the difference between the reward context (M = 0.937 ± 0.29) and the punishment context (M = 1.148 ± 0.62) was significant, t(33) = −2.35, p = 0.025, and the learning rate was higher in the punishment context. For |pwPE₂|, the probabilistic prediction error in the punishment context (M = 0.48 ± 0.27) was significantly higher than in the reward context (M = 0.39 ± 0.12), t(33) = −2.41, p = 0.022. For the parameter ζ, the difference between the reward context (M = 2.43 ± 1.57) and the punishment context (M = 2.18 ± 0.97) was not significant, t(33) = 1.13, p = 0.27 (Figure 3).

Figure 3.

Model parameter results

(A) Probabilistic learning rate (LR₂).

(B) Probabilistic prediction error (|pwPE₂|).

(C) Inverse temperature parameter (ζ). Data are represented as mean ± SEM in the Figure 3. ∗p < 0.05.

Combined analysis of EEG and model parameters

We compared the results of models with and without parameter modulation, incorporating Context and Environmental volatility (with parameter modulation including the standardized |pwPE₂| model parameter in the LMM) as fixed factors, setting reward and stable conditions as the baseline. A linear mixed-effects model with participants as random intercepts was used, with trial-by-trial data from the electrodes and time windows selected in the original analysis serving as the dependent variable. By comparing the linear mixed models with and without parameters under the two components, we found that the model with parameters was always superior to the one without parameters, which indicates the importance of this parameter for the electroencephalogram component (see Table S3).

Parameter-free modulation

The fixed effects results for the FRN without parameter modulation are presented in Table S4. Compared to the stable environment, the volatile environment significantly reduced FRN amplitude (β = −0.75, SE = 0.21, t = −3.64, p <0 .001), indicating that the volatile environment elicited a more negative FRN compared to the stable environment. Follow-up simple-effects analyses showed that volatility significantly modulated FRN amplitude in both motivational contexts. Specifically, FRN amplitude was significantly larger in the stable than in the volatile condition in the reward context (β = 0.75, SE = 0.21, z = 3.64, p < 0.001) and in the punishment context (β = 1.116, SE = 0.206, z = 5.42, p < 0.001). By contrast, reward and punishment contexts did not significantly differ within the stable environment (p = 0.98), and their difference in the volatile environment was not significant, although trend-level (p = 0.07).

The fixed effects results for the P300 without parameter modulation are presented in Table S5. The P300 amplitude in the volatile environment was significantly reduced compared to the stable environment (β = −1.12, SE = 0.20, t = −5.57, p <0 .001), indicating that the stable environment elicited a more positive P300 amplitude. Further analysis indicates that volatility also significantly reduced amplitude in both the reward context (β = 1.12, SE = 0.20, z = 5.57, p < 0.0001) and the punishment context (β = 1.43, SE = 0.20, z = 7.12, p < 0.0001). No reward-punishment difference was found in the stable environment (p = 0.422), whereas in the volatile environment P300 amplitude was significantly larger in the reward than in the punishment context (β = 0.471, SE = 0.201, z = 2.35, p = 0.019).

To directly explore the relationship between EEG amplitude and task performance, we also constructed a linear mixed-effects model including trial-by-trial accuracy as a predictor. The results showed that accuracy was significantly correlated with both FRN and P300 amplitude. For FRN, the amplitude of correct trials was larger than that of error trials (β = 1.248, SE = 0.307, t = 4.071, p < 0.001). For P300, the accuracy also showed that the amplitude of correct trials was smaller than that of error trials (β = −3.309, SE = 0.296, t = −11.174, p < 0.001). No significant higher-order interactions involving ACC were observed, suggesting that this behavioral effect was robust across task and volatility conditions, and specific fixed effects results are shown in Tables S6 and S7 in the supplemental information.

Parameter modulation

We evaluated the non-modulated model against the modulated model including pwPE₂ using a likelihood ratio test. For FRN, the modulated model significantly improved the fit of the model (χ² = 48.30, p < 0.0001), with lower Akaike Information Criterion (AIC) (154,518 vs. 154,558) and Bayesian Information Criterion (BIC) (154,597 vs. 154,606). Similarly, for P300, the modulated model significantly outperformed the parameter-free model (χ² = 436.7, p < 0.0001), with lower AIC (153,044 vs. 153,472) and BIC (153,123 vs. 153,520) (see Table S3).

For the FRN component, the fixed effects results with the standardized |pwPE2| included as a fixed factor are presented in Table S8. Beyond the effect of environment type, a significant interaction was found between context and |pwPE₂| (β = 0.9, SE = 0.21, t = 4.38, p <0 .001). However, in the reward context, the main effect of |pwPE₂| yielded a β value of only −0.06, indicating that |pwPE₂| did not significantly influence FRN amplitude in the reward context. In contrast, in the punishment context, |pwPE₂| was significantly positively correlated with FRN amplitude (Figure 4).

Figure 4.

The results of the combined analysis of FRN and model parameters

(A) FRN waveform plot; the red box is the selected time window range, and the shaded parts are SEM.

(B) Topographic maps showing the difference between low and high |pwPE₂| for reward (top) and punishment (bottle) conditions.

(C) Interaction effect of context × parameter on FRN amplitude under parameter modulation, with shaded areas representing the 95% confidence interval (CI).

The fixed effects results for the P300 component with the inclusion of the standardized absolute precision-weighted prediction error (|pwPE₂|) are presented in Table S9. In addition to the effect of environment type, we observed a significant main effect of |pwPE₂| (β = 1.50, SE = 0.15, t = 9.81, p <0 .001) and a significant interaction between context and |pwPE₂| (β = 0.54, SE = 0.20, t = 2.73, p = 0.006). The main effect of |pwPE₂| indicates that in the reward context, the P300 effect increased with larger |pwPE₂| values. The significant interaction between context and |pwPE₂| suggests that the positive effect of |pwPE₂| on P300 amplitude was stronger in the punishment context compared to the reward context (Figure 5).

Figure 5.

The results of the combined analysis of P300 and model parameters

(A) P300 waveform plot; the red box is the selected time window range, and the shaded parts are SEM.

(B) Topographic maps showing the difference between low and high |pwPE₂| for reward (left) and punishment (right) conditions.

(C) Interaction effect of context × parameter on P300 amplitude under parameter modulation, with shaded areas representing the 95% confidence interval (CI).

Discussion

This study combined a probabilistic reversal learning task with trial-by-trial EEG and HGF modeling to examine the cognitive mechanisms of reward and punishment learning under volatility. We found that the punishment context exhibited a higher probability of switching rules after negative feedback in stable environments, with this effect attenuated in volatile environments. The HGF revealed higher learning rates and larger precision-weighted prediction errors (|pwPE₂|) in punishment contexts, indicating stronger responses to uncertainty. Analysis of the modulatory effects of the |pwPE₂| on EEG signals showed that FRN was selectively enhanced by punishment feedback, and P300 scaled positively with |pwPE₂|. Critically, |pwPE₂| was associated with FRN only in the punishment context, while it was positively related to P300 in both contexts, with a stronger effect for punishment. These results reveal a functional dissociation between early error signals and late belief-updating signals in reward and punishment learning.

Behavior and computational modeling of Bayesian belief updating in reward and punishment contexts

Behaviorally, the reward advantage was only reflected in the stable environment; under volatility, performance did not differ between reward and punishment. This result indicates that the benefit of reward-based learning depends on environmental uncertainty. Our results are consistent with those of previous studies,^27,45,46,47 which reported better learning in reward than punishment contexts⁴⁶ and longer time to avoid punishment.⁴⁷ We further showed that when the environment shifted from stable to volatile in reward contexts, task performance markedly declined, highlighting the significant impact of environmental volatility on learning. But the interaction between context and environmental volatility revealed that the negative effect of volatility on performance was partially mitigated in punishment contexts, showing a weaker detrimental effect compared to reward contexts.

A similar pattern was observed for the probability of switching rules after error feedback: switching was least frequent in the reward-stable condition, while it significantly increased in the punishment-stable condition, consistent with the possibility that punishment inflates perceived volatility and promotes overexploratory switching. Functionally, this may show that reward contexts appear more vulnerable to volatility in terms of accuracy, while punishment contexts seem to be less affected but at the price of under-adjusting exploration to the true level of environmental change. Our results indicate that participants increased exploratory switching and accepted short-term costs when pursuing rewards but did not adjust switching proportionally when avoiding punishment. In other words, avoidance framing dampened flexibility: participants did not sufficiently increase rule switching as volatility rose and thus maintained a relatively rigid response policy. This rigidity can superficially appear “resistant” to volatility in accuracy terms (a smaller drop in punishment), yet it reflects under-adaptation rather than robustness. These observations are consistent with prior reports of reduced flexibility under aversive motivation.^18,48,49

But beyond that, this pattern may also indicate distinct default response tendencies across motivational contexts, with reward-context-preferred persistence and punishment-context-preferred switching following unfavorable outcomes.^2,3,4 Although participants showed a smaller volatility-related increase in switching in the punishment context, this does not necessarily mean that punishment reduced adaptive capacity per se. An alternative interpretation is that reward-seeking and punishment-avoidance encourage different response biases. Under reward framing, participants may be more likely to persist with the currently preferred option in order to maximize gains, whereas under punishment framing, they may be more likely to switch after unfavorable outcomes in order to avoid further losses.^2,4 From this perspective, greater switching in punishment—especially under stable conditions—may reflect a bias toward abandoning the current rule. This interpretation is also more consistent with computational results. Specifically, punishment contexts were associated with higher learning rates and larger |pwPE₂| values, suggesting that participants assigned greater weight to outcome deviations and updated beliefs more rapidly in aversive contexts. Importantly, faster belief updating does not necessarily translate into greater behavioral flexibility. Instead, it may reflect heightened sensitivity to negative outcomes, which can promote a switch-prone policy even when such switching is not always optimal.

Although we used a Bayesian learner (HGF) to fit the behavior, our ERP findings can be expressed in terms of prediction-error-like teaching signals that have historically been discussed in reinforcement-learning (RL) frameworks.^29,32 RL-style delta-rule learning can be viewed as an approximately Bayesian belief-updating scheme when uncertainty is not explicitly represented (or is effectively treated as fixed), such that outcomes update beliefs with a learning rate that approximates the influence of new evidence.^50,51 The HGF generalizes this idea by providing a Bayesian account of RL-like update equations in which prediction errors are precision-weighted and learning rates vary with inferred uncertainty and volatility.^9,22 Under this unifying view, FRN/P300 modulations by pwPE2 are consistent with a teaching signal whose impact on outcome-evoked neural processing depends on the learner’s moment-to-moment confidence in the underlying beliefs; moreover, feedback-related P300 signals have been shown to track surprise and relate to learning in a way that depends on statistical context.⁴⁰

Our computational modeling results further suggest that the behavioral asymmetry between reward and punishment contexts is accompanied by important differences in computational learning mechanisms. That is, learning rates were higher in punishment contexts compared to reward contexts and similarly, higher in volatile environments compared to stable ones.⁵² Individuals exhibited faster belief updating in negative contexts and assigned greater weight to prediction errors in unstable environments. These findings align with Bayesian learning theory, where learners update beliefs by inferring environmental volatility.^21,22 The elevated punishment-context learning rate accords with neurobiological models emphasizing ACC-mediated error monitoring and avoidance learning.²⁹ Importantly, however, a higher learning rate does not necessarily imply greater behavioral flexibility.⁵³ Rather, in punishment context, it may reflect stronger weighting of unfavorable outcomes and heightened sensitivity to negative feedback, which can bias behavior toward switching or vigilance without always improving adaptive performance.⁵⁴ This interpretation is consistent with accounts suggesting that punishment-related motivation can shape response policies differently from reward-related motivation and that prevention-oriented states are often associated with vigilance and a preference for stability rather than flexible exploration.^3,18

Additionally, increased |pwPE₂| in volatile environments supports a core assumption of hierarchical learning models, namely that individuals adjust learning strategies based on perceived uncertainty. Contrary to frameworks that emphasize reward-driven learning,⁶ the present pattern suggests that punishment contexts may place greater weight on unfavorable outcomes under uncertainty, leading to faster belief updating and stronger trial-wise coupling between computational signals and feedback-related EEG responses. Although this pattern is broadly consistent with theoretical accounts of prioritized threat-related processing, the present study did not directly test an evolutionary explanation.^55,56,57

It is worth noting that we did not find any differences between the reward and punishment in the inverse temperature parameter (ζ). Within the decision model, higher ζ yields more deterministic, policy-consistent choices for a given belief state, whereas lower ζ permits more exploratory or noisy responding. Notably, applying high-definition theta-frequency tACS over right DLPFC during probabilistic reversal learning increased low-level learning rate and uncertainty and amplified low-level pwPEs, yet left the decision model’s temperature parameter unchanged—implicating frontal theta in belief updating rather than in altering choice stochasticity per se.²⁸ This dissociation suggests that motivational framing primarily reconfigured perceptual learning (how beliefs are updated about cue-outcome probabilities) rather than the determinism of the choice rule.⁵⁸ These computational findings provide a bridge to subsequent EEG analyses, suggesting that behavioral differences in reward and punishment learning may stem from distinct neural processing priorities at different stages

EEG signatures of context-dependent precision-weighted prediction errors

Additional analyses further showed that both FRN and P300 amplitudes were significantly associated with trial-wise behavioral accuracy. However, behavioral measures such as accuracy reflect the observable outcome of learning, whereas HGF-derived trial-wise quantities more directly index the latent updating processes that give rise to those outcomes. We therefore regard the EEG-behavior associations as complementary evidence, while retaining computational-neural coupling as the primary mechanistic focus of the present study. First, the follow-up simple-effects analyses provided a clearer view of the parameter-free ERP effects. For FRN, the stable-volatile difference was significant in both reward and punishment contexts, whereas reward-punishment contrasts within each environment were not significant. This suggests that, before computational modulation was considered, FRN was more strongly shaped by environmental volatility than by motivational context itself. For P300, stable-volatile differences were also significant in both contexts; notably, a significant reward-punishment difference emerged in the volatile environment, with larger P300 amplitudes in reward than punishment. Together, these findings suggest that later-stage feedback processing may be more sensitive than FRN to the combined influence of motivational context and environmental uncertainty.

More importantly, the present model-based ERP results revealed that the |pwPE₂| was significantly and positively predicted FRN amplitude only in the punishment context framework, with no significant effect on FRN in the reward context. Previous work has often interpreted the FRN as reflecting early outcome evaluation, particularly in relation to expectancy violation and prediction-error-like processing.^29,33 Intracranial evidence further shows anatomically dissociable prediction-error signals for reward and punishment,⁴⁷ supporting partly segregated evaluative channels. Our finding aligns with the notion that early frontocentral feedback signals are sensitive to outcome deviations under aversive conditions.^29,33 By contrast, although |pwPE₂| also quantifies error magnitude under reward, it did not affect FRN, which may suggest that early error-monitoring mechanisms are preferentially tuned to aversive outcomes.²⁹ But in our study, we did not directly manipulate salience, dissociate signed from unsigned prediction error, or test source-level mechanisms; the present data cannot allow us to conclude that FRN specifically encodes salience or a particular form of prediction error.

Furthermore, our model-based ERP results also demonstrated that the P300 was not only positively influenced by the main effect of the precision-weighted prediction error (|pwPE₂|) but also exhibited stronger effect in the punishment context compared to the reward context. Previous EEG studies have linked the P300 to later-stage attention allocation, context updating, and learning-related processing, but its functional interpretation is not limited to any single construct such as salience or belief updating.^38,39 This ERP amplitude acts as a late feedback stage that integrates unexpected feedback into belief updating while being significantly influenced by motivational salience.⁵⁹ Larger prediction errors were associated with greater P300 amplitudes across contexts, reflecting a general attentional response, but the effect was amplified under punishment; one possible interpretation is that unexpected outcomes in punishment contexts carry greater motivational relevance and therefore engage more extensive later-stage processing.^24,38 Prior work has tied the centroparietal P300 to adjustments in learning rate and belief revision, with its relationship to trial-wise updating depending on whether surprise signals a true change point or noise.⁴⁰ Multimodal EEG-fMRI studies likewise point to separable yet interacting value systems, with later parietal signals reflecting outcome evaluation beyond initial salience detection.⁵⁹ Within hierarchical Bayesian formulations, P300 has been linked to precision-weighted error terms that drive belief updating, a mapping supported by recent EEG-HGF work showing trial-wise associations between pwPEs and late potentials²⁷ and causal modulation of low-level learning dynamics—without changing decision temperature—via theta-frequency stimulation of control circuitry.²⁸ In this framework, |pwPE₂| reflects belief updates under uncertainty, and the enhancement of P300 may correspond to this belief adjustment process, particularly in punishment where motivational salience is higher.²⁴

Collectively, our results indicate that both FRN and P300 components were associated with trial-wise |pwPE₂|, but with different context dependence and temporal profiles. This may be consistent with the view that FRN rapidly detects the salience of unexpected outcomes, while P300 integrates feedback to update beliefs.⁵⁹ Critically, the modulation patterns across these stages differ between reward and punishment contexts. FRN is specifically influenced by |pwPE₂| in the punishment context, reflecting the early system’s prioritized response to negative feedback. In contrast, P300 is regulated in both reward and punishment contexts, but the effect is stronger in the punishment context, indicating that late value updating requires greater cognitive resources in aversive contexts. Prior work aligns with this division of labor: early frontocentral signals encode outcome-related salience/prediction error, while later centroparietal activity indexes the incorporation of those errors into belief states and subsequent learning, with the mapping between EEG and learning itself being conditional on environmental statistics.^40,59 Within HGF-based analyses, pwPEs have been shown to relate both to oscillatory markers and to late evoked activity that scale with hierarchical uncertainty, and causal perturbation of frontal theta selectively shifts learning parameters while sparing decision temperature—supporting a two-stage,^27,28 learning-centric account that our results extend to a reward-versus-punishment comparison. Altogether, the punishment context showed a stronger linkage between |pwPE2| and both early and late feedback-related EEG activity, especially for FRN. Although this pattern is consistent with stage-like accounts of feedback processing, a more conservative interpretation is that punishment contexts amplified the coupling between trial-wise computational signals and EEG responses across multiple processing stages.^24,38,40

Limitations of the study

The current study may have some limitations. First, in this study, the fixed sequence of “stable first, then fluctuating” was adopted in each block in order to reduce task-switching and cognitive load and to obtain a more stable feedback-locked EEG responses. However, this design may still introduce not only sequence-related confounding but also other time-on-task influences that are difficult to distinguish from environmental volatility itself. Particularly, early task adaptation or training effects, as well as later boredom, mental fatigue, and declining motivation, may all influence behavioral performance, model-parameter estimates, and ERP amplitudes over the course of a block. Therefore, some of the observed differences between stable and volatile phases may partly reflect time-related changes in engagement or processing efficiency, rather than volatility alone. Future studies should more strictly address these issues by counterbalancing stable/fluctuating order across subjects, using independent stable and volatile blocks, collecting subjective ratings of boredom/fatigue/motivation, and explicitly modeling trial or block progression in the statistical analyses. This can more strictly examine the causal effects of reward and punishment situations and environmental volatility on belief updating and its neural representation.

In addition, although we used the same operation in the reward and punishment situations, these two situations may have different degrees of arousal to the subjects. Therefore, subsequent studies can increase the synchronous measurement of autonomic arousal, such as physiological indicators such as pupil diameter or heart rate. More generally, although we discussed FRN and P300 within salience- and prediction-error-related frameworks, the present design did not directly dissociate these functional accounts. The present analyses linking HGF parameters to ERP amplitudes were correlational in nature. Therefore, although trial-wise computational quantities such as |pwPE₂| were associated with FRN and P300 amplitudes, no causal direction can be inferred from these relationships. Future research may help clarify this issue by applying methods that can better characterize the directional coupling between belief updating and feedback-related neural activity. Finally, although our prior power analysis supports the current sample size for detecting moderate effects within individuals, the final sample size may still be insufficient to handle smaller interaction effects and more refined neural inference requirements. Therefore, conducting repeated experiments in larger samples will enhance the reliability and generalizability of the current research results.

Resource availability

Lead contact

Requests for further information and resources should be directed to and will be fulfilled by the lead contact, Meng Liu (lium_87@lnnu.edu.cn).

Materials availability

This study did not generate new materials.

Data and code availability

• •All data reported in this paper are available from the lead contact upon request, Meng Liu (lium_87@lnnu.edu.cn).
• •This paper does not report original code.
• •Any additional information required to reanalyze the data reported in this paper is available from the lead contact upon request.

Acknowledgments

We gratefully acknowledge the financial support from the National Natural Science Foundation of China (grant nos. 32400872 and 32400873) and Natural Science Foundation of Liaoning Province (grant nos. 2025-BS-0764 and 2025-BS-0766). We also sincerely appreciate all the volunteers who participated in this study.

Author contributions

L.X. contributed to conceptualization, data curation, methodology, formal analysis, investigation, writing – original draft, and visualization. B.L. contributed to data curation, funding acquisition, and writing – review and editing. M.L. contributed to conceptualization, funding acquisition, writing – review and editing, and supervision. W.L. contributed to funding acquisition, supervision, writing – review and editing, and project administration. All authors reviewed the manuscript.

Declaration of interests

The authors declare no competing interests.

STAR★Methods

Key resources table

REAGENT or RESOURCE	SOURCE	IDENTIFIER
Deposited data
Behavior data	This paper
EEG data	This paper
Software and algorithms
MATLAB (R2023a)	MATLAB software	http://www.mathworks.com/products/matlab/
EEGLAB v2023.1	University College	https://sccn.ucsd.edu/eeglab/
R studio	R studio	https://posit.co/download/rstudio-desktop/
lme4	R studio	N/A
SPM12	University College	https://www.fil.ion.ucl.ac.uk/spm/
HGF software	University College	https://github.com/translationalneuromodeling/tapas/tree/master/HGF

Experimental model and study participant details

Human participants

Based on previous studies and following standard recommendations,^28,60 an a priori power analysis was conducted using G∗Power 3.1.9.2⁶¹ to determine the required sample size. Given a repeated-measures within-subjects design with two factors (2 × 2), we estimated the required sample size to detect a medium effect size (Cohen’s d = 0.5), with a statistical power of 0.80 and an alpha level of 0.05. The analysis indicated that a minimum of 34 participants was required to achieve adequate power. Thirty-six university students participated in this experiment. All participants had normal or corrected-to-normal vision, no hearing or neurological impairments, and no color blindness or color weakness. The study was approved by the Research Ethics Committee of Liaoning Normal University (No.LL2025229), and all subjects gave written informed consent in accordance with the Declaration of Helsinki before participation. One participant did not complete the experiment, and another was excluded due to excessive EEG artifacts. Thus, the final sample consisted of 34 participants, age range: 19-25 years (M = 22.3, SD = 2.56). 16 subjects were males (based on sex assigned at birth). All participants were undergraduate or master’s students and received compensation after completing the experiment. Sex assigned at birth and age were recorded for participant description. Ancestry, race, and ethnicity were not collected because these variables were not included in the approved experimental protocol and were not directly related to the aims of the present study. Sex/gender-related effects were not formally assessed because the study was not designed or statistically powered to test sex/gender differences; this should therefore be considered a limitation of the study.

Method details

Materials and Procedure

In probabilistic classification task, the classification stimuli were eight images selected from the image library provided by Kool et al.⁶² This image library has been used as a classification stimulus in many probabilistic classification tasks.^18,28 In the current study, images were randomly labeled as S1, S2, …, S8 for each participant, constituting 4 pairs of paired images with different colors. To avoid the mutual influence of learning at different content stages and prevent participant fatigue during the experiment, different pairs of images were presented in each block. In each trial, they were shown one of two images and were required to guess whether it belonged to Category A or Category B, with the probabilities of the two images belonging to a given category being complementary.

We employed a within-subject 2 (Context: reward, punishment) × 2 (Environmental volatility: stable, volatile) factorial design. On each of four task blocks, participants completed a probabilistic categorization task based on previous studies.^13,25,27,46 The task was programmed in PsychoPy (version 2024.2.4)⁶³ and run on a PC; participants were seated upright in front of a 1920 × 1080 monitor with a 60 Hz refresh rate and stimuli were presented centrally. On each trial one image from a paired stimulus set was displayed for 1,200 ms, during which participants were required to classify the stimulus as category A or B by pressing the corresponding response key (response window = 1,200 ms). Response-key mappings were balanced across trials to minimize practice-related biases. After the stimulus epoch a blank screen was presented for a jittered interval of 800-1,200 ms, after which trial-wise feedback was shown; following feedback a fixation cross appeared for another jittered 800–1,200 ms before the next trial began (see Figure 1A). Feedback valence and format differed between the reward and punishment contexts⁴⁶: in the reward context a correct response produced positive feedback displayed in green (“+25 points”) while an incorrect response produced neutral feedback in red (“+0 points”); in the punishment context a correct response produced no point loss (displayed as “-0” in green) whereas an incorrect response produced negative feedback in red (“-25 points”). Participants were informed prior to the main experiment that their remuneration consisted of a fixed participation fee (RMB 35) plus a performance-dependent bonus derived from accumulated points, converted at a rate of 100 points = 1 RMB.

In total, each participant completed two reward and two punishment blocks; each block comprised 160 trials. Block order was counterbalanced across participants such that subjects were randomly assigned to a reward-first or punishment-first sequence, while within each block the two environmental conditions (stable, volatile) were implemented according to the task structure. In the stable environment the probability that the paired images belonged to the dominant category was held at 0.75 (non-dominant = 0.25); in the volatile environment cue–outcome associative strength varied over time, with category probabilities switching between 0.2 and 0.8 on four occasions (see Figure 1A). To avoid excessive difficulty and carry-over effects associated with beginning in a volatile context, each block was presented with the stable environment first. Prior to the main experiment participants completed a practice session that included both reward and punishment contexts (20 trials per context) and contained one category reversal so that the practice mirrored the structure of the experimental blocks; the main experiment proceeded after participants confirmed they understood the task. The entire experiment lasted approximately 1hour, with participants taking a brief rest every 60 trials.

Hierarchical Gaussian Filter

We employed the HGF model developed by Mathys et al.^21,22 to estimate each participant’s learning process and belief trajectories. Used for fitting of HGF software comes from (https://github.com/translationalneuromodeling/tapas/tree/master/HGF). To avoid the mutual influence of learning among different blocks, we fitted the HGF separately for each participant and each block (i.e., participant × block), yielding block-specific posterior parameter estimates and latent trajectories (see Figure 1B).

The three-layer model, depicted in Figure 1B, describes the states at each level. The first-layer state, X₁, represents the binary variable of the experimental stimuli: in trial (k), either S1 = A & S2 = B ( X₁^(k) = 1) or S1=B & S2=A(X₁^(k)=0). The second-layer state, X₂, describes the probability of reward occurrence. The third layer, X₃, represents the log-volatility of the second layer. These hidden states (X₂ and X₃) are assumed to evolve via a Gaussian random walk, with their mean at trial k-1 centered on the previous value and variance dependent on the state of the layer above. The learning parameters ω₂ (a constant component of the step size at level 2), ω₃ (a meta-volatility parameter controlling step size at level 3) and ζ (a temperature parameter from the response model) are subject specific. Consistent with prior studies,^25,27 we fixed κ at 1.

The update equations for the HGF for each trial and level i (i = 2, 3) are as follows:

$$\Delta {\mu }_i^{\left(k\right)}={\mu }_i^{\left(k-1\right)}-{\mu }_i^{\left(k\right)}\alpha \frac{{\hat{\pi }}_{i-1}^{\left(k\right)}}{{\pi }_i^{\left(k\right)}}{\delta }_{i-1}^{\left(k\right)}$$

The posterior mean update, $\Delta {\mu }_i^{\left(k\right)}$, represents the difference between the posterior expectations of the previous trial k-1 and the current trial k. The lower-level prediction error, ${\delta }_{i-1}^{\left(k\right)}$, is used to update the belief at the current level. The precision ratio, $\frac{{\hat{\pi }}_{i-1}^{\left(k\right)}}{{\pi }_i^{\left(k\right)}}$, is defined where ${\hat{\pi }}_{i-1}^{\left(k\right)}$ is the expected precision of the lower level, and ${\pi }_i^{\left(k\right)}$ is the expected precision of the current level. In fact, this precision ratio can be understood as a dynamic learning rate.^21,22 Precision, ${\pi }_i^{\left(k\right)}$, is defined as the inverse of the variance of the posterior expectation:

$${\pi }_i^{\left(k\right)}=\frac{1}{{\sigma }_i^{\left(k\right)}}$$

In the HGF model, the precision-weighted prediction error (pwPE) serves as the core driving signal, calculated as the product of the prediction error magnitude and its associated precision. This acts as a “correction signal” used by the brain to update beliefs, widely regarded as a critical neural computational mechanism in perception and learning.

The HGF model employs a unit-square sigmoid observation model to link the posterior estimates of beliefs (what the participant thinks is likely) to the expressed decisions y(k). In this model, the predicted probability m(k) that the presented image belongs to a specific category (e.g., S1=A & S2=B) on trial k is linked to trial-wise category predictions via the softmax (logistic sigmoid) function:

$$\mathrm{p}\left(y=1\right)=\frac{m^{\zeta }}{m^{\zeta }+{\left(1-m\right)}^{\zeta }}$$

where ζ is a subject-specific temperature parameter, interpreted as the inverse of decision noise. This parameter is the only free parameter in the unit-square sigmoid observation model and serves to capture the degree of deterministic association between y and m. Higher ζ values indicate that participants are more likely to choose the option that is more congruent with their current belief. For a more detailed understanding of the specific equations and construction of the HGF model, refer to the works of Mathys et al.^21,22

Model space

We evaluated three computational models of learning. The first was a three-layer HGF model, which integrates key principles of reinforcement learning and Bayesian learning. The other two control models were the Rescorla-Wagner (RW) model and the Sutton K1 (SK1) model. The RW model, one of the most widely used and simplest reinforcement learning models, assumes that expected errors drive belief updates under a fixed learning rate. The SK1 model incorporates an adaptive learning rate but lacks a hierarchical structure. Comparing these two models with the HGF model allows us to assess whether participants engaged in hierarchical learning and adaptively updated their learning processes. Model comparison was conducted using the protected exceedance probability metric from random-effects Bayesian model selection (BMS).⁴⁴

EEG recording and preprocessing

EEG was recorded using a 64-channel Brain products setup, and all electrodes were arranged in accordance with the extended international 10-20 system. The impedance for all electrodes was checked and reduced until it was below 5 kΩ. The FCz electrode was used as the reference electrode while recording, and a 100-Hz low-pass filter was applied. The average value of TP9 and TP10 amplitudes of the left and right mastoid process electrodes was used for re-reference during offline analysis, and EEG signal were submitted to a 0.1-30 Hz band-pass filter, with a sampling rate of 500 Hz. Then, we used independent component analysis (ICA) and the ICLabel plugin in EEGLAB v2023.1^64,65 to automatically classify the components , followed by manual rejection of artifact-related components, including ocular, muscle, and other non-neural sources. Residual artifacts were further screened by visual inspection, and signals exceeding ±75μV in any given epoch were automatically excluded. Following this, continuous recordings were epoched around feedback-locked from -200 to 1000 ms. Other artifacts were removed by visual inspection. For further statistical analysis, approximately 96.2% of the data were retained after excluding artifacts. Based on visual inspection of waveform plots and scalp topography distributions, and with reference to studies such as Jiang et al.⁶⁶ and Liu et al.,²⁷ we selected a 220-320 ms time window with regions of interest (ROIs: F2, F1, Fz, FC2, FC1, FCz, C2, C1, Cz) for analyzing the feedback-related negativity (FRN) component, and a 300-450 ms time window with ROIs (FC1, FC2, FCz, C1, C2, Cz, CP1, CP2, CPz) for analyzing the P300 component.

Quantification and statistical analysis

Behavioral data analysis

Firstly, generalized linear mixed models (GLMM) were conducted to examine the main effects and interactions of Context and Environmental volatility on behavioral metrics(R 4.0.1, lme4 package).⁶⁷ Two binomial GLMMs were constructed to focus on two behavioral metrics respectively: Accuracy and the probability of rule switching after receiving error feedback (i.e., P(RuleSwitch)). Here, accuracy was defined as the probability of selecting the optimal option in the current trial, i.e., the option corresponding to the high-probability association. For each model, Context and Environmental volatility were included as fixed effects, with reward conditions and stable conditions set as baselines, and participants were included as random effects in the model. Accuracy and P(RuleSwitch) were separately incorporated as dependent variables for model fitting.

Additionally, we conducted paired t tests to further analyze the differences in individual behavioral flexibility across the reward-punish conditions. We calculated the behavioral flexibility in Accuracy (ΔAccuracy, the difference between stable and volatile condition) and in the probability of switching rules after error feedback (ΔP(RuleSwitch)), the difference between stable and volatile condition) for each participant, separately for reward and punishment contexts. Paired t tests were then performed on these two difference variables to examine flexibility differences across contexts.

Finally, we tested whether the distinction in reward-punish learning also reflected in the model parameter. Paired t tests were used to test the asymmetry in reward-punish learning on learning parameters: learning rate, prediction error, and temperature parameter from decision model.

EEG data analysis

To investigate the neural correlates of reward and punishment learning processes and their relationships with HGF model parameters, we employed linear mixed models (LMMs) to fit trial-by-trial EEG data with behavioral and computational predictors. Two separate LMMs were constructed to analyze the FRN and P300 components, respectively. The dependent variable in each model was the trial-by-trial EEG amplitude, averaged over specified time windows and regions of interest (ROIs). Context, Environmental volatility, and the absolute value after standardizing the precision weighted prediction error (|pwPE₂|) were included as predictor variables, with participants modeled as random effects. The models were fitted using the restricted maximum likelihood estimation (REML) method to ensure parameter estimation stability. To evaluate the contribution of |pwPE₂|, we compared two LMMs for each EEG component: a baseline model (including only Context, Environmental volatility, and their interaction) and a full model (additionally including |pwPE₂| and its interactions). Model fit was assessed using a likelihood ratio test, where significant improvements indicated the importance of |pwPE₂| in explaining EEG variance.