Exploring when to exploit: the cognitive underpinnings of foraging-type decisions in relation to psychopathy

Abstract

Impairments in reinforcement learning (RL) might underlie the tendency of individuals with elevated psychopathic traits to behave exploitatively, as they fail to learn from their mistakes. Most studies on the topic have focused on binary choices, while everyday functioning requires us to learn the value of multiple options. In this study, we evaluated the cognitive correlates of naturalistic foraging-type decision-making and their electrophysiological signatures in a community sample (n = 108) with varying degrees of psychopathic traits. Reinforcers with different salience were included in a foraging-type decision-making task. Recruitment of various cognitive processes was estimated with a computational model and electrophysiology, and the relationships to psychopathic traits were assessed. Higher Antisocial traits were associated with a bias towards expecting more volatility in the environment when high-salience reinforcers were used. Additionally, higher levels of Interpersonal traits were associated with reduced learning from personalized rewards, as evidenced by reductions in the prediction errors (PEs) about rate of change. Higher Affective traits were associated with lower PEs and aberrant learning from painful punishments. Lastly, the PEs about rate of change were reflected in the trial-wise trajectories of Feedback-Related Negativity event-related potentials. Together, our results point to the importance of volatility processing in understanding aberrant decision-making in relation to psychopathy, demonstrate the relationships between psychopathic traits and learning through reward and punishment, and emphasise the potentially more beneficial effect of personalized rewards and punishment for improving reinforcement-based decision-making in individuals with elevated psychopathic traits.

Introduction

To make decisions effectively, one needs to weigh in the pros and cons of different options. Individuals with elevated psychopathic traits, however, consistently opt for choices that result in harm to themselves and/or others [1]. Psychopathy is a personality construct characterized by four core factors: interpersonal (manipulativeness and pathological lying) and affective deficits (lack of guilt and shallow affect), an erratic lifestyle (impulsivity and sensation-seeking) and a tendency for antisociality [2]. While the prevalence of psychopathic traits is highest among offenders [3, 4], they can also be reliably measured in the general community [5–7]. People with higher psychopathic traits behave in an exploitative manner [8–10], often using aggression to further their own interests [11–13]. While this strategy might offer short-term benefits, in the long term, individuals with higher psychopathic traits are at greater risk of offending [14].

The maladaptive decision-making in psychopathy has been attributed to a reduced ability to learn from past outcomes and adapt behavior appropriately [15, 16]. Bayesian accounts of reinforcement learning (RL) propose that individuals generate predictions about stimulus-outcome associations (i.e. reward and punishment contingencies) and update them on the basis of incoming evidence [17, 18]. Individuals with high psychopathic traits struggle with processing such contingencies, which contributes to the emergence and maintenance of maladaptive behaviors [19–21]. At the mechanistic level, it is unclear how outcomes drive RL in such individuals as there is evidence proposing exaggerated responses to rewards (resulting in disregarding punitive outcomes) [22, 23], normative reward processing with relative insensitivity to punishment [24–26], or a more general deficit in adapting to previously learned contingencies (i.e. volatility tracking) [27–29]. Moreover, individuals with elevated psychopathic traits also generate more uncertain (i.e. less accurate) estimates of how likely it is that the learned associations might change [20, 21]. Recent computational work has additionally demonstrated an atypical updating of predictions, which primes individuals with high psychopathic traits to persevere with their original expectations despite disconfirmatory evidence [19]. Importantly, impaired learning can also be attributed to differences in pre-existing cognitive biases (i.e. priors). A very strong prior expectation of a stable environment, for instance, might lead one to disregard evidence that the environment is changing and result in reduced behavioral adaptation [21]. Finally, in value-based RL, the value of some options might be updated differentially, depending on their salience or personal relevance (i.e. some alternatives might be more desirable than others, leading to a higher value) [30, 31]. Such updates are driven by so-called prediction errors (PEs), reflecting the discrepancy between one’s expectations about outcomes and the observed outcomes [32]. Importantly, findings point out that different types of PEs are generated. Low-level (sensory) prediction errors (PEs) capture discrepancies between the expected and predicted outcomes, while high-level (associative) PEs reflect incorrect expectations about the likelihood of learned contingencies to change [33, 34]. The effect of relevance/salience on RL can be quantified computationally by estimating PEs as a function of individual sensitivity to different types of rewards and punishments [35]. On the one hand, highly salient punishments such as pain should induce higher PEs and improve learning [36]. However, because individuals with high psychopathic traits are relatively insensitive to pain [19, 37, 38], the PEs elicited in response to painful outcomes might be reduced. On the other hand, more personally relevant rewards reduce the RL deficits in individuals with psychopathic traits [30]. Thus, potential differences in the impact that high- vs. low-salient reinforcers may have on decision-making also need to be considered. Finally, exploration might reduce PEs by allowing individuals to gather more information and refine their predictions. Pain, however, disrupts the balance between exploration of new alternatives and exploitation of known options [39]. A relative insensitivity to pain can reduce one’s exploration readiness, biasing how they engage with new options and the generation of PEs. In that sense, pain insensitivity might be a facilitating factor for aberrant learning in individuals with higher psychopathic traits.

In sum, extant evidence links behaviors resulting from altered RL in relation to psychopathy to disturbances in outcome processing (reward hypersensitivity, punishment hyposensitivity, altered value-updating depending on outcome saliency), as well as monitoring and learning from contingency changes (more uncertain predictions, perseveration, atypical learning priors). One important note is that, while disturbances in these cognitive processes are reported across the literature, the extent to which they explain reduced RL appears to be trait-specific. For example, the Affective features of psychopathy are predictive of increased uncertainty in estimates about likelihood of change (environmental volatility), while the Lifestyle features additionally predict higher uncertainty about the rate of change (meta-volatility) [20]. Another consideration is that most of the work on RL in psychopathy was conducted using binary choice paradigms, where two options are compared simultaneously. However, we often need to make series of choices to reach a particular goal, highlighting the importance of studying decisions where alternatives have to be compared serially [40–42]. In such foraging-type paradigms, individuals decide whether to engage with a known option or forgo it and search for better alternatives (e.g. forgoing a job offer in the hopes a better alternative comes along). Strikingly, despite the relevance for everyday functioning, understanding foraging-type decision-making in relation to psychopathy has not been a focus of research. Perhaps this is partly due to additional challenges posed by the higher complexity of this type of decision-making relative to studying binary choices, especially the disentangling of the underlying cognitive correlates. Such processes are unobservable, which makes it challenging to determine the extent to which they explain the poor reinforcement-based decision-making often seen in psychopathy [4].

One solution is to use a versatile computational framework that assumes individuals monitor changes in the associations between events and outcomes (i.e., volatility tracking) [18, 43], in line with prior work on the topic [19, 20, 21]. The Hierarchical Gaussian Filter (HGF) [18, 44] proposes a hierarchical mathematical framework that approximates how individuals track and adapt to volatility on the basis of Bayesian inference, considered the optimal way to update beliefs in probabilistic settings [45]. To understand how cognitive computations such as PEs change on a trial-wise basis, computational models can be combined with approaches sensitive to such fast fluctuations. The high temporal resolution of electrophysiology allows us to estimate fast trial-wise fluctuations in PEs and their neural representations across cortical regions [46]. Prior research points to the feedback-related negativity (FRN) [47, 48] as an electrophysiological index of PEs. The FRN occurs 250 ms following feedback and reflects the divergence of the received reward from the expected reward [49]. Interestingly, the FRN amplitude seems to be scaled by the perceived relevance (i.e. salience) of reinforcers [30, 50], and as such provides insight into altered RL outcome processing in psychopathy. Additionally, while FRN is commonly used as the neural signature of reward PEs to evaluate outcome processing in RL [51], it is unclear whether it reflects both high-level and low-level PEs. The two PEs are modulated by separate systems in the brain [52] and have distinct neural representations [53, 54]. Therefore, the first step is to determine what sort of PE-related information is encoded in the FRN to better understand its role in learning, and then apply these insights to better understand reinforcement-based decision-making in psychopathy.

The overarching goal of the present study was to examine how variability in the level of psychopathic traits in the general community is related to foraging-type decision-making. We focused on the individual psychopathic traits given the evidence for trait-specific associations with RL [20, 55] and moral decision-making [56] in psychopathy. A computational model was built that incorporated all aforementioned cognitive elements linked to disturbed RL in psychopathy. Firstly, we sought to evaluate how variations in levels of psychopathic traits are linked to the ability to make the right choices by learning about environmental change. We expected that at higher levels of psychopathic traits, individuals would demonstrate altered value-updating when learning from pain. Additionally, we predicted that psychopathic traits would be associated with altered priors related to volatility expectations. We also anticipated aberrations in the generation of low-level and high-level PEs with increasing psychopathic traits. We compared the effect of reinforcers with different salience given the evidence of the impact of salience on RL, including painful punishments and monetary losses, and personalized rewards and monetary gains. Based on findings of improved associative learning with salient rewards [30] but aberrant pain learning [19], we hypothesized that the personalized rewards will improve foraging-type decision-making with increasing psychopathic traits, but that the effect of salient punishments (pain) might be reduced for individuals with high levels of psychopathic traits. Finally, using a joint computational and electrophysiological approach [19, 43, 44], we aimed to uncover the neural representation of low-level and high-level PEs, and provide a more nuanced understanding of the neural and cognitive basis of foraging-type decision-making.

Methods

A community sample of 108 healthy adults (age range: 19–54 years, mean ± standard deviation: 29.49 ± 9.05 years, 55.6% women; sample size calculation in Supplement 1) was recruited via social media and the electronic Radboud research participation system. Inclusion criteria were no prior history of psychiatric illness, neurological or chronic pain conditions. The study was approved by the Ethics Committee of the Faculty of Social Sciences, Radboud University (code ECSW-2020-120). All participants provided informed consent prior to participation. All methods were performed in accordance with the relevant guidelines and regulations. Based on visual inspection, we excluded the data of 7 participants due to extreme outliers in the estimated computational parameters (4 in the low-salience, 3 in the high-salience condition).

Psychopathic traits were evaluated with the Self-Report Psychopathy-Short Form (SRP-SF; [57, 58]), which measures Interpersonal, Affective, Lifestyle and Antisocial traits, and provides a total psychopathy score. SRP-SF scores ranged from 30 to 101 (m = 47.9 ± 13.9), similarly to other community sample studies [5, 7]. 9.26% of the current sample had scores above 70, considered to represent elevated psychopathic traits [59].

Participants completed a foraging-type RL decision-making task piloted in a separate subsample of 5 participants (Fig. 1). The task represents an RL-based multi-alternate choice paradigm that combines elements from different conceptual theories of foraging. Individuals make trial-by-trial decisions whether to exploit a known option (whose reward likelihood might deplete with time) or forego it in search of a potentially better alternative. Participants were instructed that outcomes on this task are probabilistic and will vary depending on the condition. Outcomes in the low-salience (LS) condition were monetary gains and losses, and in the high-salience (HS) condition: personalized rewards and valence-matched electrical shocks calibrated to each individual’s pain sensitivity. The order of conditions was fixed with the LS preceding the HS condition, as the LS represents standard RL and can, therefore, serve as a baseline to evaluate the effect of salience. The foraging-type task is described in detail in Supplement 1 (Fig. S1). The analytical pipeline is summarized in Fig. 2.

Fig. 1. Foraging-type decision-making task and reinforcement schedule.

Note. A Learning task: on each trial, participants “stay” with a known option (one grass patch for the blocks’ duration), or “go” explore one of three additional patches. Unbeknownst to participants, each patch is probabilistically associated with a reward (e.g. 80%) which shifts throughout the task. Reward and punishment probabilities were contingent on each other (p(punishment) = 1-p(reward)), and the reward probabilities to the four patches added to 100% on each trial. Outcomes were salient (personalized rewards or electric shocks) and non-salient (monetary gain or loss). Depicted: possible outcomes in the salient condition. B Reinforcement schedule: The maximum reward probability of each patch varied throughout the task (e.g. patch 1 might be associated with a 90% reward probability on some trials, but only 80% on others). The exact reinforcement schedule can be found in Supplement 1 (Fig. S1).

Fig. 2. Analytical pipeline.

Note. The behavioral choice data was processed with a computational model (HGF) to estimate the latent cognitive processes and the trial-wise PEs involved in RL. The psychopathic traits were regressed on the estimated learning parameters in the low- and high-salience condition in a Bayesian Structural Equation Model (BSEM). The trial-wise PEs trajectories were included in a General Linear Mixed Effect Model (GLMM1) as predictors for the single-trial FRN amplitudes extracted from the EEG data (red: outcome = loss, blue: outcome = win). In another GLMM (GLMM2), we assessed the effect of the individual psychopathic traits on the trial-wise high-level PEs (red: outcome=loss; blue: outcome = win; hi-sal: high-salience condition; lo-sal: low salience condition).

Computational model

A three-level mean-reverting HGF model [18, 44] was built, informed by prior research [19]. It estimates several learning processes: individual differences in pre-existing cognitive biases ${\mathrm{\mu }}_2^{\left(0\right)}$ (higher ${\mathrm{\mu }}_2^{\left(0\right)}$ implies an initial, pre-learning expectation that the environment is likely to change) and ${\mathrm{\mu }}_3^{\left(0\right)}$ (higher ${\mathrm{\mu }}_3^{\left(0\right)}$ values reflect initial expectations that these changes occur often, i.e. rate of environmental change). Additionally, the model quantifies how much post-learning beliefs drift away from the initial expectations, or the magnitude of belief updating about volatility m₂ and meta-volatility m₃. Finally, it generates an estimate k₂ of how closely coupled beliefs about volatility and meta-volatility are (i.e. does learning about meta-volatility impact learning about volatility). PEs are generated on a trial-by-trial basis following feedback, depending on these computational processes. The model is described in more details in Supplement 1.

The perceptual mean-reverting model is combined with a SoftMax response model that quantifies exploration readiness β (higher values represent closer belief-to-action mapping, or decreased exploration readiness), as well as sensitivity to rewards λ-win and sensitivity to punishments λ-loss. In addition to the latent parameters, we also extracted the single-trial trajectories of ε₂ (low-level PEs) and ε₃ (high-level PEs).

The model was fit to the low-salience and high-salience condition data separately with the same priors (Supplement 1, Table S1). Data was simulated with the model to assess if simulated choices reflected the true choices. Additionally, we evaluated if we could reliably recover the true underlying parameters. The full recovery procedure and the model’s update equations are detailed in Supplement 1. The results from the recovery procedure and the simulations are reported in Supplement 2 (Figs. S1, S2, Table S1).

ERP acquisition and data processing

Electroencephalography (EEG) was recorded using 32 active electrodes (Acticap, Brain Products GmbH, Germany) arranged to an extended version of the 10–20 system (impedance <10 kΩ, 500 Hz sampling frequency). Data was pre-processed offline with BrainVision Analyzer software [60] and re-referenced to the linked earlobes. Ocular artifacts were removed using Independent Component Analysis [61], and data was filtered with a 0.2–20 Hz bandpass filter and notch (50 Hz) filter. The data was segmented into epochs (200–700 ms post-feedback) according to condition (non-pain/pain) and outcome type (win/loss), rejecting trials with amplitudes exceeding −75 or 75 mV. For each individual, the trial-averaged post-feedback waveforms were inspected visually to define a custom time window for the FRN detection within the 200–350 ms range most commonly reported in prior research [30, 62–64]. We identified the FRN as the difference between the most negative peak in that window and the preceding positive peak (P200) at electrode Cz1 [30, 47, 63–65].

Statistical analyses

Prior to assessing the computational model results, we evaluated task performance across conditions. The behavior analysis methods are reported in Supplement 1, and the results—in Supplement 2 (Fig. S3, Tables S2, S3). To test the relationship between psychopathic traits and the latent cognitive processes, we used Bayesian Zero-Order correlations and BSEMs regressing the psychopathic traits on the learning parameters in each condition in Mplus [66]. A Bayesian estimator (PX1) and Markov Chain Monte Carlo (MCMC) sampling with four Markov chains and 75,000 iterations (first half discarded as burn-in) were used to estimate the correlation coefficients and standardized regression weights and the corresponding 95% credibility intervals (CI). Model fit was determined based on the posterior predictive p-value (PPP-value) and posterior predictive check using χ2 testing [19–21]. A good model fit is indicated by a PPP-value approaching 0.5 and a 95% CI interval for the posterior predictive check that includes 0 [20, 66, 67]. Significance for both the correlations and the regression coefficients was determined based on the 95% CIs.

The trial-wise dynamics of PEs were investigated with three general linear mixed effect models (GLMM). In GLMM1, single-trial FRN amplitudes were regressed on the absolute values of low-level PEs |ε₂| and high-level PEs ε₃. The absolute value for ε₂ was chosen as, at this level of the hierarchy, the PEs’ sign is arbitrary and by converting it to absolute value, we can make inferences about the magnitude of the surprise at the outcome [46, 68]. Statistical tests (ANOVA) demonstrated that the low-level PEs made no contribution to the model fit or the single-trial FRN amplitudes, so ε₂ was removed. As a confirmatory post-hoc step, in GLMM1a, we regressed the choices on the next trial (stay or switch) on the single-trial FRN amplitudes and the low-level and high-level PEs values on the current trial. Finally, in GLMM2, the trial-wise PEs encoded by the FRN were regressed on the four psychopathic traits. The high-level PEs values were transformed via the cube-root transform, which handles negative values naturally and preserves the sign [69], and has been successfully used in prior research [70]. Control variables were outcome, condition, and trial number.

For all GLMMs, the random-effects structure comprised a subject intercept, and by-subject slopes for condition, trial and outcome. The residual plots were inspected visually for obvious deviations from homoscedasticity or normality. Significance of the predictors was determined based on the estimated 95% CI via the profile method (which must not contain 0), as well as the p-values provided by lmerTest. A post-hoc analysis with the emmeans package in R [71] was conducted on significant interactions which provides the pairwise contrasts of the estimated marginal means. 95% CIs and p-values with a multivariate correction for multiple comparisons are reported for the contrasts.

Results

Latent learning processes and psychopathic traits

The only zero-order correlation observed was between higher Antisocial traits and higher initial expectations about meta-volatility in the HS condition, ${\mathrm{\mu }}_3^{\left(0\right)}$, r = 0.19, 95% CI [0.001 to 0.37]2. The remainder of the correlations can be found in the Supplement 2 (Table S4).

The first BSEM inspected the associations between the learning parameters in the LS condition and the four psychopathic traits. Model fit was excellent, ppp = 0.427, χ² 95% CI [−20.82 to 25.83] but no statistically significant paths were found. The model fit for the BSEM including the HS parameters also demonstrated an excellent fit, ppp = 0.428, χ² 95% CI [−21.77 to 27.25]. There was a significant path from increased prior beliefs about the meta-volatility ${\mathrm{\mu }}_3^{\left(0\right)}$ to higher Antisocial traits, β = 0.25, 95% CI [0.03 to 0.45].

Neural representations of hierarchical PEs

In GLMM1 we evaluated which of the two hierarchical PEs are reflected in the trial-wise dynamics of the FRN signal. Results suggested that only high-level PEs predicted the changes in the single-trial FRN amplitudes, b = −0.52, 95% CI [−0.74 to −0.30], p < 0.0013. A two-way interaction between high-level PEs and the type of outcome was also present, b = 1.02, 95% CI 0.74 to 1.30], p < 0.001, showing more negative FRN amplitudes post-losses with higher high-level PEs, and more positive FRN amplitudes post-wins with higher high-level PEs. GLMM1a validated the effect of the hierarchical PEs on choice behavior by demonstrating that single-trial FRN amplitudes, and low-level and high-level PEs explained choice behavior on the next trial independently from each other. Results for both GLMM1 and GLMM1a can be found in Supplement 2 (Tables S5 and S6).

Generation of high-level PEs and psychopathic traits

No fixed effects of the individual psychopathic traits on the trial-wise high-level PEs were found (GLMM2; Table 1). However, there were significant three-way interactions between psychopathic traits, condition, and outcome (Fig. 3). Both Interpersonal (b = −0.38, 95% CI [−0.43 to −0.33], p < 0.001) and Affective traits (b = 0.58, 95% CI [0.52 to 0.64], p < 0.001) moderated the interaction between condition and outcome (Fig. 4). Individuals with higher Interpersonal traits elicited higher PEs after more salient losses (HS loss vs LS loss, b = 0.14, p = 0.02, 95% CI [0.02 to 0.27]), but lower PEs after more salient wins (HS win vs LS win, b = −0.24, p < 0.001, 95 CI [−0.36 to −0.11]. At higher levels of Affective traits, PEs after low-salience losses were higher (HS loss vs LS loss, b = −0.30, p < 0.001, 95% CI [−0.45 to −0.16]), but the PEs after high-salience wins were higher (HS win vs LS win, b = 0.28, p < 0.001, 95% CI [0.13 to 0.43]). Lifestyle traits also showed a significant three-way interaction, but the post-hoc analyses revealed no significant contrasts.

Table 1.

GLMM results: predicting single-trial high-level Pes.

Dependent variable: single-trial ε3
Predictor	b	95% CI	SE	T	P
Outcome	0.87*	[0.51 to 1.24]	0.18	4.74	<0.001
Condition	−0.01	[−0.08 to 0.06]	0.04	−0.25	0.80
Trial	0.01	[−0.01 to 0.03]	0.01	1.36	0.17
Interpersonal traits	−0.04	[−0.28 to 0.20]	0.12	−0.36	0.72
Affective traits	0.15	[−0.14 to 0.43]	0.14	1.03	0.31
Lifestyle traits	0.03	[−0.22 to 0.28]	0.13	0.24	0.81
Antisocial traits	−0.03	[−0.23 to 0.17]	0.10	−0.27	0.79
Outcome x condition	−0.03	[−0.07 to 0.004]	0.02	−1.74	0.08
Outcome x Interpersonal	0.14	[−0.38 to 0.65]	0.26	0.523	0.60
Condition x Interpersonal	0.14*	[0.04 to 0.24]	0.05	2.80	<0.01
Outcome x Affective	−0.28	[−0.89 to 0.33]	0.31	−0.91	0.37
Condition x Affective	−0.30*	[−0.42 to −0.18]	0.06	−4.97	<0.001
Outcome x Lifestyle	−0.21	[−0.74 to 0.33]	0.27	−0.76	0.45
Condition x Lifestyle	0.09	[−0.02 to 0.20]	0.05	1.68	0.1
Outcome x Antisocial	0.12	[−0.31 to 0.56]	0.22	0.56	0.58
Condition x Antisocial	−0.02	[−0.11 to 0.06]	0.04	−0.5	0.62
Outcome x Condition x Interpersonal	−0.38*	[−0.43 to −0.33]	0.03	−14.32	<0.001
Outcome x Condition x Affective	0.58*	[0.52 to 0.64]	0.03	18.53	<0.001
Outcome x Condition x Lifestyle	−0.06*	[−0.12 to −0.01]	0.03	−2.28	0.02
Outcome x Condition x Antisocial	0.01	[−0.03 to 0.05]	0.02	0.42	0.67

b: regression coefficient, SE: standard error, T: t-value, P: P-value. Significant predictors (95% CI don’t contain 0) denoted in bold and with an asterisk.

Fig. 3. Post-hoc analysis of interactions between psychopathic traits, condition and outcome.

Note. Moderating effect of Interpersonal (right), Affective (middle), Lifestyle (left) traits on the interaction between condition (low-salience vs high-salience) and outcome (win vs loss) on high-level PEs.

Fig. 4. Graphical summary of main findings.

Note. The main findings from each analysis are denoted by a thick blue arrow and a blue box. Only the GLMM2 results for Affective traits are displayed visually in the figure.

Discussion

The overarching goal of the present study was to uncover the (neuro)cognitive underpinnings of foraging-type decisions and their relationship to psychopathic tendencies. Firstly, higher Antisocial traits predicted inflated priors about environmental meta-volatility. Additionally, we discovered reduced learning from personalized rewards with higher Interpersonal traits, and reduced learning from pain with higher Affective traits, both represented by lower associative PEs following feedback. Finally, we demonstrated the neural signature of these high-level associative PEs by showing the magnitude of the trial-wise FRN signal scaled with the magnitude of the PEs (Fig. 4).

Having inflated initial expectations about meta-volatility represents a cognitive bias: the belief that learned stimulus-outcome associations change rapidly. Such a bias might make individuals with higher Antisocial traits primed to see the world as more unstable. In turn, unexpected outcomes are less surprising, resulting in diminished learning and reduced behavioral adaptation [72]. Behaviorally, the meta-volatility bias in the present study was associated with reduced lose-switch and increased win-stay rates. Perceiving the world as unstable might make individuals prefer to stick with choices yielding predictable outcomes, regardless of their valence. This perseveration might be beneficial at first, but when a true reversal in associations occurs, individuals will struggle to learn the new contingencies. In turn, outcomes become increasingly more unpredictable, driving further perseveration. Importantly, failing to accurately predict change results in more uncertain (i.e. less precise) predictions. More uncertain predictions are also reported in computational work with youths at a high risk of developing antisocial personality disorder [20]. Our findings of a meta-volatility bias, therefore, provide a cognitive-level explanation for the maladaptive perseveration reported in relation to psychopathy [73–75], and connect it to findings of contingency reversal insensitivity [28, 29] and increased uncertainty [20].

Additionally, our results on the link between psychopathic traits and the generation of PEs provide a more nuanced understanding of the learning deficits in foraging-type decision-making. The hierarchical framework we employed assumes that individuals generate predictions about outcomes at the low (sensory) level, and predictions about change in learned associations at the high (associative) level. Crucially, we demonstrated that the trial-wise dynamics of the FRN reflect violations of predictions (i.e. prediction errors) at the high level only. These high-level PEs essentially represent the discrepancies between the predicted and actual change in associations and drive meta-volatility learning as a discrepancy indicates that changes occur at an unexpected rate. Our results build onto findings of distinct neural signatures of hierarchical PEs [53, 54]. While FRN is considered to reflect outcome evaluation [48, 62, 76], research has linked it to PEs stemming from negative outcomes [48], motivationally salient outcomes [77], unexpected outcomes regardless of valence or motivational value [78, 79], as well as negative outcomes regardless of expectancy [63]. One key difference between these studies and ours is that our hierarchical learning framework allows us to disentangle the low-level PEs related to the expectancy of the outcome (similar to [63], from the high-level ones linked to expectancies about stimulus-outcome mappings. In general, our finding that the FRN encodes high-level associative PEs only supports the view that this signal is related to violations of predictions about the speed with which associations change, rather than expectancy, valence of salience of outcome.

Finally, our findings also provide the first direct evidence that psychopathic traits are linked to variability in the generation of high-level associative PEs, but not low-level sensory PEs. These results can be seen to support the possibility that RL disturbances in psychopathy are primarily associated with higher-level problems with tracking and adjusting to changes in learned stimulus-outcome contingencies through time [28, 80, 81], and that the presence of lower-level issues with processing reward and punishment is more dependent on contextual factors (e.g. sample type, experimental paradigm). Our findings indicate that updating both stimulus-reward and stimulus-punishment associations are altered in individuals with higher psychopathic traits, thus affecting how much is learned in each trial. This converges with prior computational work showing that elevated psychopathic traits and conduct disorder (a risk factor for psychopathy) are associated with reductions in learning rate [82], particularly when learning from punitive outcomes [80, 83, 84]. Our results extend these findings by demonstrating that reduced punishment learning from pain, but not from monetary losses, co-occurs with higher Affective traits. The atypical pain learning can be interpreted in the context of a reduced sensitivity to pain associated with psychopathy [37, 38] and especially with high callous-unemotional traits which map onto the Affective facet [81]. Pain usually drives fast behavioral adaptation due to its motivational salience [85–87]. Lower pain insensitivity, however, might make painful outcomes less salient. Reduced learning from pain with increased psychopathic traits is reported in other computational work and seems to be specifically mediated by pain insensitivity [19]. This insensitivity to pain potentially also explains the current findings in individuals with higher Affective traits.

While individuals with higher Affective traits generated reduced associative PEs to painful punishment associations, salient rewards resulted in increased associative PEs. This falls in line with the suggestion that naturalistic rewards might improve the RL deficits found in psychopathy [30], and could perhaps also accommodate computational work demonstrating aberrant punishment but intact reward learning in psychopathy [83, 84]. Importantly, these results show the dissociation between stimulus-punishment and stimulus-reward learning occurs particularly at higher levels of Affective traits.

Interestingly, at higher levels of the Interpersonal traits, high-level associative PEs after losses were higher in the high-salience condition, indicating adaptive learning from pain. However, higher associative PEs were elicited in response to monetary rewards. This suggests those individuals learned faster from low-salience monetary compared to high-salience naturalistic rewards. Potentially, individuals with higher Interpersonal traits might present with a limited experience of hedonic value or reduced reward sensitivity to naturalistic rewards. Essentially, this reduced sensitivity might lead them to struggle to form and update associations as effectively. While certain psychopathic facets (i.e. the impulsive-antisocial) are associated with increased sensitivity to reward, such an effect is not found for the interpersonal-affective domains [23]. In a post-hoc analysis we report in Supplement 2, Interpersonal traits correlated negatively with reward responsiveness. However, as the pattern extended to all SRP traits, we cannot conclude that this is the mechanism driving the reductions in PEs to subjective rewards. Nevertheless, future research into RL should explore the possibility that higher Interpersonal traits might present with a reduced reward sensitivity.

These results hold important implications, not only for the mechanisms underlying aberrant learning and decision-making in psychopathy, but also for potential rehabilitation strategies. Our results build upon prior findings of punishment insensitivity [22], reduced punishment learning [84], atypical punishment prediction errors [88] and reduced pain learning [19] in relation to psychopathy by demonstrating that individuals with high Affective traits struggle when learning about changes based on negative painful outcomes, but not from personalized rewards. At the same time, our findings highlight the importance of considering personalized negative outcomes particularly when it comes to improving RL in individuals with higher Interpersonal traits.

A key limitation is that we could only explore certain latent processes to how reliable certain computational parameters were. Thus, we could not evaluate the path from Antisocial traits to meta-volatility bias in the low-salience condition. Since the parameters were well-recovered in the HS condition, we are confident in the robustness of our findings. However, future research should specifically investigate the role of the motivational salience of feedback. Additionally, given the economic toll of psychopathy-related crime [89] and the prevalence of psychopathic traits in offenders, exploring foraging-type decision-making in an offender sample can further validate these results, and perhaps provide valuable insights for rehabilitation and treatment development.

In conclusion, we demonstrated that elevated Antisocial traits are associated with a cognitive bias related to a prior overestimation of how volatile the environment is. Additionally, Interpersonal and Affective traits moderated how different types of reinforcers and outcomes are incorporated to update predictions about the rate at which stimulus-outcome contingencies change. Finally, we showed that the FRN signal encodes violations of predictions about stimulus-outcome contingencies and drives learning about meta-volatility, and that there are different patterns of relationships with variation in individual psychopathic traits. These results provide the first evidence for foraging-related alterations in the generation of associative, rather than sensory prediction errors at higher levels of psychopathic traits and demonstrate the cognitive mechanisms underlying the persistent maladaptive decision-making in psychopathy.

Author contributions

DVA, JMO and IAB conceived the presented idea. DVA and IAB designed the experiment. DVA and VIM collected the data. DVA performed the computations and analysis with support from IAB, CM and AOD. CM and AOD verified the analytical methods and contributed to the interpretation of the results. DVA wrote the manuscript with input from all authors. All authors provided critical feedback and helped shape the research, analysis and manuscript.

Data availability

The data and code described in this study are openly available at https://osf.io/ztkcv/?view_only=91e9d90df4ec40a2a1ae4f185919c687.

Competing interests

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Supplementary information

The online version contains supplementary material available at 10.1038/s41398-025-03245-2.