Directed exploration is reduced by an aversive interoceptive state induction in healthy individuals but not in those with affective disorders

Abstract

Elevated anxiety and uncertainty avoidance are known to exacerbate maladaptive choice in individuals with affective disorders. However, the differential roles of state vs. trait anxiety remain unclear, and underlying computational mechanisms have not been thoroughly characterized. In the present study, we investigated how a somatic (interoceptive) state anxiety induction influences learning and decision-making under uncertainty in individuals with clinically significant levels of trait anxiety. A sample of 58 healthy comparisons (HCs) and 61 individuals with affective disorders displaying elevated anxiety symptoms (iADs; i.e., anxiety and/or depression) completed a previously validated explore-exploit decision task, with and without an added breathing resistance manipulation designed to induce state anxiety. Computational modeling revealed a significant group-by-condition interaction, such that information-seeking (i.e., directed exploration) in HCs was reduced by the anxiety induction (Cohen’s d=.47, p=.013), while no change was observed in iADs. The iADs also showed slower learning rates than HCs across conditions (Cohen’s d=.52, p=.003), suggesting more persistent uncertainty. These findings highlight a complex interplay between trait anxiety and state anxiety. Specifically, state anxiety may attenuate reflection on uncertainty in healthy individuals, while familiarity with anxious states in those with high trait anxiety may create an insensitivity to this effect.

Introduction

Persistent uncertainty and maladaptive avoidance are key maintenance factors in anxiety disorders and a major focus of psychotherapy (1, 2). While considerable progress has been made in understanding the neural and cognitive mechanisms associated with anxiety, underlying computational processes have only been examined in a limited number of studies to date (3, 4). For example, subclinical levels of trait anxiety have been associated with reduced flexibility in learning rates when there is a change in the stability of environmental statistics (5), and individuals with anxiety disorders show elevated learning rates in general (6), suggesting the belief that the environment is volatile or that action-outcome contingencies often change unexpectedly. Both depression and anxiety have also been associated with elevated learning from punishment in particular (7). More recently, trait anxiety has been linked to a greater tendency to infer changes in the underlying causes of aversive outcomes during extinction learning – which may facilitate return of fear and reduce the long-term efficacy of behavioral therapies (8).

While such studies provide important insights into the learning processes that may contribute to depression and anxiety disorders, they do not fully account for avoidance or other behaviors driven by intolerance of uncertainty. There could also be multiple computational mechanisms underlying avoidance behavior, each representing distinct hypotheses and possible treatment targets. For example, individuals must often seek information to learn that a feared situation is tolerable. Avoidance prevents such learning, which can in turn maintain avoidance. This speaks to the explore-exploit dilemma (9, 10), which has begun to receive attention in psychiatry and substance use research (11–13). This dilemma reflects the need to judge whether one has sufficient information to maximize reward, or whether one should first seek more information. In avoidance, an individual may hold the confident belief that avoided situations are dangerous, while, in fact, exploration would allow them to learn otherwise. Yet, there are multiple types of exploration, and factors that could deter exploration, which have not been thoroughly evaluated.

To date, only a few studies have examined the relationship between affective disorders and information-seeking. For example, one study found that ‘directed’ exploration (DE) in a community sample was lower in those with higher negative affective symptoms (14), while another study associated lower DE with greater somatic anxiety in particular (15). This type of exploration is strategically directed toward situations and actions for which an individual has had fewer past experiences. Further studies examining effects of traumatic/unpredictable childhood environments (16, 17) – which often correlate with affective disorders (18–20) – also suggest negative effects on DE. Other supportive work has shown that: higher stress/anxiety is associated with less exploration in a virtual-reality plus-maze (21), agoraphobia and anxiety sensitivity are associated with less exploratory behavior (22), and increases in cortisol in response to an acute stressor and scores on a chronic stress questionnaire are both associated with under-exploration in foraging tasks (23). Aversive arousal states are also generally known to reduce cognitive control network activity in neuroscience studies (24–28), consistent with reduced cognitive reflection tendencies seen in those who display lower DE (14). Notably, a distinct exploratory strategy – ‘random’ exploration (i.e., where choices become less reward-driven as a means of gaining information) – has not shown such associations.

Past research therefore suggests that negative affect may selectively reduce DE, consistent with maintained avoidance. Yet, these studies are largely correlational and have tended to focus on trait anxiety, and they have primarily investigated sub-clinical symptoms in community samples. There are also some reasons to expect relationships in the oppositive direction. Namely, the cognitive aspects of anxiety involve elevated uncertainty and worry, which could promote over-exploration. Indeed, intolerance of uncertainty could drive over-exploration as a means of continually attempting to reduce it. This is consistent with some work instead suggesting greater DE in those with higher trait anxiety (29, 30) and increased exploratory behavior in depression (30, 31). Combined with the work reviewed above, this suggests the possibility that state anxiety might reduce exploration, while uncertainty- or worry-related trait anxiety could instead promote it.

Here we sought to build on this previous work by manipulating state anxiety with a somatic anxiety induction while individuals performed an explore-exploit task. By comparing exploration and learning with vs. without this induction, while also gathering information about trait anxiety and other clinical, cognitive, and affective dimensions, we aimed to disentangle the role of state- and trait-related anxiety. By doing this in both healthy comparison (i.e., low-negative affect) and clinical (i.e., high-negative affect) groups, we also sought to clarify potential differences between subclinical variation in anxiety and depression and that associated with psychopathology. We hypothesized that state anxiety induction would reduce DE. This would follow from the possibility that anxious somatic (high arousal) states plausibly disrupt reflective cognition and attention to uncertainty (e.g., see 15, 24, 25, 26). Given the mixed results described above with respect to trait anxiety, we did not have a confident directional hypothesis regarding group differences. As secondary aims, we also sought to reproduce the aforementioned relationships found between DE and both cognitive reflectiveness (14) and early adversity (16, 17), as these associations could offer additional explanatory power if present in our sample.

Methods

Participants

Data were collected as part of a larger, multi-visit study to investigate the cognitive and neural correlates of psychiatric disorders at the Laureate Institute for Brain Research (LIBR), with participants recruited from the community in and surrounding Tulsa, OK, USA. Clinical diagnoses were assessed by a licensed clinician according to the Mini International Neuropsychiatric Interview 7 (MINI; (32)). Healthy comparisons (HCs) were not previously diagnosed with, or treated for, any mental health disorder and had a score of < 8 on the Overall Anxiety Severity and Impairment Scale (OASIS; (33)). The clinical group consisted of individuals with one or more affective disorders displaying elevated anxiety symptoms (iADs). Due to high comorbidity rates between depression and anxiety diagnoses (34, 35), those in this group were permitted to have either a current or lifetime diagnosis of anxiety and/or depressive disorders and were required to have a score ≥ 8 on the OASIS in a pre-visit screening. For a full comorbidity breakdown in iADs, see Supplementary Table S1. Any iADs who were taking psychiatric medications (e.g., SSRIs, SNRIs, etc.) were required to have a stable dosage for at least 6 weeks prior to study entry. Participants were asked to abstain from other medications (e.g., benzodiazepines), marijuana, and alcohol in the 48 hours preceding participation and were required to pass a drug panel during their study visit. Antipsychotic and stimulant medications were not allowed (a detailed list of included/excluded medications is provided in Supplementary Materials). The following diagnoses were also not permitted to minimize confounding influences on primary outcome measures: bipolar disorder, personality disorders, substance use disorders, eating disorders, schizophrenia, or obsessive-compulsive disorder. Recruitment aimed to match participants in the two groups by age, sex, and education level.

Seven individuals withdrew from participation due to discomfort with the anxiety induction paradigm (2 HCs, 5 iADs, all female; details in Supplementary Materials), leading to a final sample of 119 participants: 58 HCs, 61 iADs (see Table 1). A post-hoc power analysis (using the wp.rmanova function in the WebPower package in R (36)) indicated that this sample size would afford 80% power to detect a medium effect size of ${\eta }_p^2=.063$ for the effect of anxiety induction on task behavior. While no prior studies are available (to our knowledge) reporting effects of breathing-based anxiety induction on decision-making, we note that some previous work has assessed effects of stress induction on potentially related cognitive tests (e.g., executive function batteries; ³⁷). Here, effect sizes for stress-induced changes in test scores have tended to be large (Cohen’s d>.88). Thus, results of the above power analysis for the present study suggest a sufficiently low false negative rate, assuming similarity to effects reported in this prior work.

Table 1.

Demographic and clinical characteristics (mean and SD).

Measure	Healthy Comparisons (N=58)	Affective Disorders (N=61)	Statistic
Age	35.41 (13.08)	33.44 (10.97)	t(117) = 0.89, p = .374
Sex	72% Female	75% Female	χ2(1) = 0.03, p = .870
PHQ	1.66 (2.27)	9.46 (5.15)	t(117) = 10.60, p < .001
OASIS	1.07 (1.52)	8.41 (3.02)	t(117) = 16.62, p < .001
QIDS	2.17 (1.77)	8.77 (4.01)	t(117) = 11.52, p < .001
STAI Trait	29.17 (7.4)	51.87 (9.06)	t(117) = 14.92, p < .001
CTQ Total*	34.44 (10.36)	48.07 (16.19)	t(112) = 5.31, p < .001
Physical Neglect	6.18 (2.14)	8.41 (3.79)	t(112) = 3.82, p < .001
Physical Abuse	6.67 (2.33)	7.95 (3.26)	t(112) = 2.39, p < .019
Emotional Neglect	8.76 (3.88)	13.03 (5.02)	t(112) = 5.05, p < .001
Emotional Abuse	6.89 (2.98)	10.59 (4.59)	t(112) = 5.07, p < .001
Sexual Abuse	5.93 (3.05)	8.08 (5.01)	t(112) = 2.75, p = .007
CRT	3.33 (2.18)	2.18 (1.93)	t(117) = 3.05, p = .003

Note. While OASIS scores at screening were required to be ≥ 8 for inclusion in the Affective Disorders group, values shown here are at the later date of study participation. As OASIS scores are rated with respect to the prior week, some scores represented here were < 8 in this group. Legend: PHQ = Patient Health Questionnaire-9 (depressive symptoms); OASIS = Overall Anxiety Severity and Impairment Scale; STAI = State-Trait Anxiety Inventory, CTQ = Childhood Trauma Questionnaire; CRT = Cognitive Reflection Test (scores correspond to number of correct answers).

^*This measure was collected as part of the larger study at an earlier date (35 days prior to participation on average).

It is important to note here that data from the healthy sample in this study was previously used for comparison to a different clinical population (methamphetamine use disorders) (38). Here, we instead focus on differences between this group and iADs.

Measures

Primary measures were chosen to assess state and trait anxiety (State-Trait Anxiety Inventory [STAI]; (39)) and to account for clinical symptoms of depression (Patient-Health Questionnaire [PHQ]; (40)). Our secondary aim of replicating prior work motivated inclusion of measures to test links between DE and both childhood adversity (Childhood Trauma Questionnaire [CTQ]; (41)) and cognitive reflectiveness (Cognitive Reflection Test [CRT]; (14, 16, 17)) – the tendency to ‘think things through’ before responding based on intuition. Detailed descriptions of each measure are included in Supplementary Materials. Descriptive information for all participants is shown in Table 1, as well as preliminary group comparisons. Additional sample demographics are included in Supplementary Table S2.

Somatic Anxiety Induction and Sensitivity Assessment

To manipulate state anxiety, we utilized a previously established interoceptive (breathing-based) anxiety induction paradigm (42–44). In this paradigm, participants are asked to breathe through a silicon mask attached to a valve (Figure 1a) that allows application of different levels of inspiratory resistance, creating air hunger-related sensations that induce anxiety. To provide participants a chance to familiarize themselves with the paradigm, and assess baseline sensitivity, participants were first exposed to a series of resistances in ascending order (0, 10, 20, 40, 60, and 80 cmH2O/L/sec) applied for 60 seconds each. Participants were asked to rate their anxiety level after each exposure from 0 (“no anxiety”) to 10 (“maximum possible anxiety”). Other questions relating to subjective breathing difficulty, arousal, and other affective states were also presented (see Supplementary Materials).

Figure 1.

a) Equipment used for anxiety induction including silicon mask with adjustable straps and single breathing port, and an example resistor used to create resistance on inhalation. b) Graphical depiction of the Horizon Task. Shown are the two horizon conditions (H1 and H6) at the last forced choice (red box) and the first free choice (green boxes). The H1 example also shows an equal information trial (2 outcomes for each choice), while the H6 example shows an unequal information trial (3 forced choices on the right, 1 on the left). c) Table outlining each of the trial type combinations, counterbalanced across the task.

This induction was later applied using a moderate resistance level (40 cmH2O/L/sec) during performance of one run of the behavioral task. This level of resistance was chosen because of prior work demonstrating its effectiveness and feasibility (45); we also confirmed that it induced moderate (roughly 5 out of 10) levels of anxiety within our clinical sample (Supplementary Figure S2).

Behavioral Task

Participants completed two runs of the Horizon Task (46), a previously validated task used to measure explore-exploit decision-making. Each task run consists of 80 independent games. In each game, players are presented with two new options (slot machines) and asked to repeatedly choose between them. They are told that the average reward values for options on one game are unrelated to those on other games. Half of the games include 5 sequential choices, while the other half include 10 sequential choices. The first four choices in all games are “forced”, meaning the player is told which option to choose (see Figure 1b). The remaining choices are “free”, where either option can be chosen. Trials with one free choice (H1) and those with six free choices (H6) appear an equal number of times throughout the task.

Forced-choice patterns create two different information conditions: equal, in which each option is chosen twice; and unequal, in which one option is chosen three times and the other is chosen only one time. The goal of the task is to maximize the number of points won by repeatedly choosing the option expected to provide higher point values on average. For each option, point values were sampled (rounded to the nearest integer) from a Gaussian distribution with a standard deviation of 8. One of the options in each game always had a mean reward value of either 40 or 60. Then, for each game, the other option had a mean point value difference from this first option of +/− 4, 8, 12, 20, or 30. The same pseudorandom sequences of forced-choice outcomes were shown to each participant, consistent with the generative means. All game dimensions (length, information type, mean difference, and better/worse side) were counterbalanced across the task (Figure 1c). For maximal comparability, each of the two task runs was kept identical in terms of mean differences, forced-choice outcomes, and the associated game order.

Participants wore the mask for each task run. In the no-resistance run, resistance was not added (0 cmH2O/L/sec). In the resistance run, a moderate resistance (40 cmH2O/L/sec) was consistently applied throughout the run. The order was counterbalanced across participants in each group. The first six participants played a version of the task with different outcome values than the rest in a small number of games (but still sampled from the same underlying distributions). To ensure this did not confound results below, this difference in task version was evaluated as a potential covariate.

Computational Model and Model Fitting

Here, we followed the same modeling and parameter estimation approach used in previous studies with this task (e.g., ^{47, 48}). This approach is presented in detail in Supplementary Materials. Briefly, the model assumes participants start each game by assigning an initial expected reward value to the two new options presented (here fixed at a neutral value of 50). These expected reward values are then updated after each forced-choice outcome based on the resulting prediction error, moderated by an evolving (uncertainty-dependent) learning rate. Finally, the resulting expected reward value differences between options are used to make the first free choice, moderated by two parameters (i.e., information bonus and decision noise) promoting different forms of exploration. Specific definitions of each computational parameter are provided in Table 2. As each game in the task is independent (i.e., with two new options presented, along with a new set of forced-choice outcomes), the model assumes no learning carries over between games or task runs. Independent parameter values were therefore estimated for each task run for each participant. This estimation step was carried out simultaneously for both groups and conditions with an established hierarchical fitting procedure using a Markov Chain Monte Carlo method (see Supplementary Materials).

Table 2.

Descriptions of model parameters.

Model Parameter	General Description
Information Bonus (IB)	IB parameters (separated by horizon condition to yield IB1 and IB6) quantify an individual’s tendency to choose the option that maximizes information gain. This parameter only affects decisions in the unequal information condition, where selecting the option that was chosen only once during the forced choices provides more information about its average reward value. Directed exploration (DE) is derived from these parameters (IB6 minus IB1). This is because information gain in H1 cannot guide future choices, while information gain in H6 can guide future choices. Thus, DE can be seen as the tendency to increase information-seeking when it would be useful.
Decision Noise (DN)	The DN parameters (separated by horizon condition to yield DN1 and DN6) quantify an individual’s tendency to choose the option with a lower observed mean reward. Random exploration (RE) is derived by subtracting DN1 from DN6 in the equal information condition. Thus, RE can be seen as the tendency to make less reward-sensitive choices when information gain would be useful.
Spatial Bias	These parameters (one per horizon/information condition) account for the possibility that an individual prefers to choose one option over the other simply because of which side it is on.
Initial Learning Rate (α0)	The initial value of the learning rate before making the first forced choice.
Asymptotic Learning Rate (α∞)	The learning rate an individual would converge to if the game were played indefinitely (i.e., when uncertainty about mean reward values would be minimal).

Statistical Analyses

Resistance sensitivity

To verify that the anxiety induction was effective, we examined affective responses to each resistance level in the initial exposure protocol. To this end, we estimated linear mixed effects models (LMEs) with self-reported anxiety (i.e., the single anxiety question on a scale from 0 to 10) as the outcome variable and resistance level and group as regressors (and their interaction).

Primary model-based analyses

Before addressing our primary hypotheses, we checked for outliers using an iterative Grubbs method (threshold: p<.01; using grubbs.test from the outliers package in R (49)). This resulted in one data point being removed for IB1 (in the resistance condition), two for DN6 in the equal information condition (one in each resistance condition), and two for RE (one in each resistance condition). Then, we estimated LMEs that included group, resistance condition, and their interaction as sum-coded regressors for each model parameter (HCs=−1, iADs=1; no-resistance=−1, resistance=1). Given that the two learning rates were significantly positively correlated in both task runs (rs>.34, p<.001), models with ${\mathit{\alpha }}_{\mathrm{\infty }}$ as the outcome variable were also tested when accounting for ${\mathit{\alpha }}_0$ values to assess whether unique relationships with asymptotic learning rate were present even after accounting for its shared variance with initial learning rate. To rule out possible alternative explanations, we also re-ran these models including covariates to confirm that any group or resistance effects could not be explained by age, sex (Male=−1, Female=1), or task version (new=−1, old=1).

We then assessed continuous relationships between computational parameters and affective symptoms. In each group separately, we tested linear models of DE in each resistance condition using PHQ scores and STAI Trait scores as regressors (also accounting for age). However, given the high inter-correlations between PHQ and STAI, problematic multicollinearity issues were possible. We calculated variance inflation factors (VIFs; using the ols_vif_tol function from the olsrr package in R (50)) for each model and, in cases where VIFs surpassed a threshold of 4, we planned to use ridge regressions designed to minimize problematic multicollinearity effects (using the linearRidge function from the ridge package (51)).

Model-free task performance

To better interpret observed effects on task behavior and assess relationships between task performance and model parameters, we performed additional supportive analyses examining overall accuracy on free choices (as measured by the number of times participants chose the option with the higher underlying mean reward). We first restricted analyses to first free choice and estimated an LME with accuracy as the outcome variable and horizon, information condition, resistance, and group as regressors. We also included possible three-way interactions between group, horizon, and information condition, and between group, horizon, and resistance (and associated two-way interactions). This allowed us to explore whether groups might show greater differences depending on horizon, information condition, or anxiety induction. As commonly observed in this task, we expected accuracy in H1 trials would be higher than in H6 trials (i.e., reflecting random exploration).

A subsequent model was run associating accuracy with free choice trials in H6 (choices 5–10). We also tested three-way interactions between group, free choice number, and information condition, and between group, free choice number, and resistance (and associated two-way interactions) to explore whether groups might differ in the slope of accuracy improvement over time depending on information condition or anxiety induction.

To evaluate whether some parameter values might be considered more optimal than others, we then tested LMEs associating parameters with accuracy on H6 free choice trials, excluding the first free choice to which model parameters were fit. As IB is only calculated in the unequal information condition, accuracy only on the relevant trials was the outcome for these models, with resistance condition, group, free choice number, IB for H6 (IB6), and the interaction between free choice number and IB6 as regressors. Analogous models were used to test relationships between accuracy and the other model parameters within the relevant trial types (i.e., decision noise for H6 with equal information condition accuracy; learning rates with accuracy in both information conditions).

Secondary replication analyses

Additional LMEs were run to accomplish our secondary aim of replicating prior relationships found between DE and both early adversity (16, 17) and cognitive reflectiveness (14). First, separate LMEs were run on DE based on each subscale of the CTQ, while also including group, resistance condition, and their interaction as regressors (while controlling for known effects of age). Analogous LMEs were run instead using CRT scores as regressors.

Results

Model Validation

We first assessed parameter recoverability. Specifically, participants’ fitted parameter values were used to simulate behavior under the model, and this simulated behavior was then used for parameter estimation. The correlation between generative and estimated parameter values in these simulations provides a measure of parameter recoverability. All correlations were significant and positive (rs>.52, ps<.001), suggesting moderate-to-good recoverability (see Supplementary Figure S1), however IB1 was somewhat lower in recoverability than the other parameters. This was likely because observed values for this parameter tend to show a restricted range near zero, as the H1 condition is designed to minimize the value of information. Recoverability for DE itself was notably higher (r=.78) and correlated more strongly with IB6 values (r=.78), suggesting the IB1 estimates acted more like stable floor values.

We then confirmed that parameter estimates correlated with descriptive measures of task behavior in expected directions (e.g., percentage of trials in which the high-information and low-mean option was chosen, points won). These results were all in expected directions (detailed in Supplementary Materials). In particular, IB parameters were positively associated with the percentage of trials in which the high-information option was chosen (rs≥.86, ps<.001 for both resistance conditions), and DN parameters were positively associated with the percentage of trials in which the low-mean option was chosen (rs≥.29, ps≤.002). Slower learning rates were also associated with a greater number of trials in which the low-mean option was chosen (rs≤−.43, ps<.001) and worse task performance in general (i.e., fewer points won on the task; rs≥.45, ps<.001).

The Breathing Resistance Protocol Successfully Induced Anxiety During Task Performance

Anxiety ratings during the initial breathing resistance sensitivity protocol are shown in Supplementary Figure S2. Briefly, anxiety increased as resistance level increased, anxiety ratings were higher in iADs than HCs, and iADs showed greater increases in anxiety compared to HCs as resistance level increased. LMEs for self-reported anxiety scores at baseline and during both task runs showed the same pattern (Figure 2). See Supplementary Materials for full results.

Figure 2.

Self-reported anxiety and STAI State at baseline and during each run of the Horizon Task (i.e., with and without the added breathing resistance of 40 cmH2O/L/sec). Results showed greater anxiety in iADs, greater anxiety with higher resistance level, and greater increases in anxiety in iADs than HCs when resistance was added. Boxplots show median and quartile values along with individual datapoints.

Directed Exploration was Elevated in Affective Disorders Patients, but Reduced by State Anxiety Induction

Plots depicting parameter values by group and condition are shown in Figure 3. Descriptive statistics for model parameters are reported in Supplementary Table S4. In an initial LME for DE, there was a significant Group × Resistance Condition interaction (F(1,117)=3.97, p=.049, ${\eta }_p^2=.03$), such that HCs and iADs showed similar DE in the no-resistance condition, DE in HCs was lower in the resistance condition (estimated marginal mean [EMM]= 4.28) than in the no-resistance condition (EMM=5.27; t(117)=−2.52, p=.013), and DE in iADs did not change between conditions (t(117)=0.27, p=.786). Further exploration suggested this was driven primarily by differences in IB6 values within the resistance (see top-right panel of Figure 3). Main effects of group and resistance were nonsignificant. In a subsequent model accounting for age, sex, and task version, this interaction remained significant (F(1,117)=3.97, p=.049, ${\eta }_p^2=.03$). There was also a negative association with age (b=−0.06; F(1,114)=5.65, p=.019, ${\eta }_p^2=.05$).

Figure 3.

Individual data points and density plots by group and resistance condition for each parameter. In HCs, the anxiety induction led to reduced levels of DE. In contrast, iADs were insensitive to this effect and showed stable DE values in both conditions. For illustrative purposes, the individual information bonus (IB) values underlying DE for each horizon condition (H1 and H6) are shown in the top-right (recall that DE corresponds to the difference in information bonuses between horizon conditions; DE = IB6 – IB1). As can be seen, only IB6 values in the resistance condition show differences between groups. RE did not differ by group or condition (bottom left). Learning rates tended to show lower values in iADs than HCs across conditions. As visible in the lower middle panel, ${\mathit{\alpha }}_0$ showed a bimodal distribution, which motivated the cluster-based analysis approach described in the text. Asymptotic learning rates (${\mathit{\alpha }}_{\mathrm{\infty }}$) were significantly lower in iADs than HCs in both conditions (bottom right). Note that stars and Cohen’s d effect sizes correspond to the interaction between group and resistance for DE (upper left) and for the group difference in ${\mathit{\alpha }}_{\mathrm{\infty }}$ across resistance conditions (bottom right) in post-hoc comparisons.

No significant effects were found in LMEs for RE (Fs≤3.83, ps≥.053).

Because of bimodality in the distributions for ${\mathit{\alpha }}_0$ (see Figure 3), we used k-means clustering (52) across resistance conditions to divide participants into groups with high vs. low values and treated this as a 2-level categorical variable (low=−1; high=1). In logistic regression models using membership in the high vs. low ${\mathit{\alpha }}_0$-value group as the outcome variable (using glmer in the lme4 package; (53)), there were no significant effects before or after adding covariates (zs≤|1.53|, ps≥.125). Complementary nonparametric models treating ${\mathit{\alpha }}_0$ as a continuous outcome variable (using the Rfit package in R (54)) revealed a nonsignificant trend (b=−0.03, p=.052) suggesting potentially higher ${\mathit{\alpha }}_0$ values in HCs than iADs. A main effect of sex (b=−0.06, p=.003) was also found, indicating higher ${\mathit{\alpha }}_0$ values in male participants.

The LME for ${\mathit{\alpha }}_{\mathrm{\infty }}$ (accounting for ${\mathit{\alpha }}_0$) revealed a significant main effect of group (F(1,117)=7.72, p=.006, ${\eta }_p^2=.07$), such that HCs (EMM=.33) had higher learning rates than iADs (EMM=.28, t(117)=−2.78, p=.006). The expected positive association with ${\mathit{\alpha }}_0$ was also observed (F(1,199)=16.31, p<.001, ${\eta }_p^2=.08$; b=.11). Both effects remained significant when accounting for covariates, and no additional effects were observed (Fs≤1.74, ps≥.189).

We also evaluated within-group associations with symptom severity to explore potentially differential effects of depression and anxiety. We noted that PHQ and STAI Trait scores were highly correlated (rs=.86; see Supplementary Figure S4); yet, all VIFs<4, suggesting multicollinearity did not threaten the validity of these models. Within HCs, a linear regression with DE in the no-resistance condition as the outcome variable revealed no significant relationships (Fs≤1.95, ps≥.169). However, there was a significant positive effect of STAI Trait in the resistance condition (b=.193, F(1,55)=4.17, p=.046) that became marginal after adding covariates (b=.167, F(1,53)=2.93, p=.093). PHQ showed no associations with and without covariates. In iADs, no significant effects were observed in equivalent analyses. These analyses were also carried out for other model parameters (detailed in Supplementary Results). Briefly, these suggested 1) opposing relationships with ${\mathit{\alpha }}_{\mathrm{\infty }}$ for depression (negative) and anxiety (positive) symptoms in HCs in the no-resistance condition that flipped in the resistance condition (i.e., positive relationship with depression and negative with anxiety), 2) potential negative associations between trait anxiety and ${\mathit{\alpha }}_0$ in HCs in the resistance condition, and 3) a possible relationship between RE and depression (negative) for iADs in the no-resistance condition.

Higher Levels of Directed and Random Exploration are Associated with Greater Task Performance

In LMEs for H6 accuracy (on choices 6–10) using each model parameter, we found interactions between free choice number and IB6, DN6, and ${\mathit{\alpha }}_0$ (Table 3). Specifically, the interaction between IB6 and free choice number indicated that, in unequal information trials, those with higher IB6 values showed steeper increases in accuracy from early to later choices. Similar results were found for DN6 when examining accuracy on equal information trials and for ${\mathit{\alpha }}_0$ on unequal information trials.

Table 3.

Models of free choice accuracy (separated by information condition) testing associations with model parameters.

	Parameter	Resistance	Group	Free Choice Number	Parameter*Free Choice
IB6 (unequal IC)	F(1,987)=1.71, p=.191	F(1,1078)=.14, p=.711	F(1,117)=1.63, p=.204	F(1,1067)=91.96, p<.001; b=0.017	F(1,1067)=7.35, p=.007; b=0.001
DN6 (equal IC)	F(1,1173)=3.16, p=.076	F(1,1059)=2.40, p=.122	F(1,117)=5.69, p=.019	F(1,1057)=113.00, p<.001; b=0.013	F(1,1057)=6.28, p=.012; b=0.003
α0 Cluster (equal IC)	F(1,1170)=2.03, p=.154; b=−.014	F(1,1067)=256, p=.110	F(1,117)=5.67, p=.019	F(1,1067)=114.48, p<.001; b=.025	F(1,1067)=1.66, p=.198
α0 Cluster (unequal IC)	F(1,1159)=5.63, p=.018	F(1,1067)=.01, p=.919	F(1,117)=1.67, p=.199	F(1,1067)=91.67, p<.001; b=0.016	F(1,1067)=11.34, p=.001;b=0.016
α∞ (equal IC)	F(1,1172)=.65, p=.419	F(1,1067)=2.37, p=.124	F(1,119)=5.43, p=.022	F(1,1067)=114.59, p<.001; b=.03	F(1,1067)=.66, p=.418
α∞ (unequal IC)	F(1,1165)=.42, p=.518	F(1,1067)=.036, p=.850	F(1,118)=1.69, p=.196	F(1,1067)=91.13, p<.001; b=0.033	F(1,1067)=2.97, p=.085

Note. For each significant effect of group, contrasts indicated that accuracy in iADs (EMM=0.80) was lower than in HCs (EMM=0.85; p<.05). The models of accuracy including α_∞ also showed positive effects of α₀ (Fs≥8.83, ps≤.003, bs≥0.055). IC=information condition.

We also tested an analogous model of free choice accuracy using DE as a regressor and saw comparable results to those seen for IB6 (i.e., main effect of free choice number and a significant interaction with DE; Fs≥6.11, ps≤.014). Similarly, results when using RE in a model of equal information accuracy corroborated those found for decision noise (main effect of free choice: F(1,1062)=114.45, p<.001; RE × free choice: F(1,1062)=8.92, p=.003; main effect of group: F(1,117)=5.60, p=.020).

Task Performance was Reduced in Affective Disorders in Specific Conditions

Secondary analyses of free choice accuracy indicated high performance across both groups (HCs: M=.84, SD=.13; iADs: M=.80, SD=.16; see Supplementary Figure S5). LMEs for first free choice accuracy (Supplementary Table S5) revealed significant effects of horizon type (greater accuracy in H1; F(1,823)=278.81, p<.001) and information condition (greater accuracy in equal information trials; F(1,823)=72.08, p<.001). The Group × Horizon × Information Condition interaction was also significant (F(1,823)=4.18, p=.041), indicating that HCs had significantly greater accuracy than iADs in H1/equal information trials and in H6/unequal information trials only (see post-hoc contrasts). Subsequent LMEs for accuracy across free choices in H6 showed significant effects of choice number (greater accuracy in later choices; F(1,2727)=201.30, p<.001), information condition (greater accuracy in equal information trials; F(1,2727)=51.37, p<.001), and a Group × Information Condition interaction (greater accuracy in HCs only in the equal information condition; F(1,2727)=8.93, p=.003).

Replication Analyses Confirm Associations with Early Adversity and Cognitive Reflectiveness

CTQ scores were greater in iADs than HCs (t(112)=5.31, p<.001). In separate LMEs for DE with each subscale of the CTQ, group, resistance condition, and their interaction as regressors (controlling for age), and after correcting for multiple comparisons (p≤.01), there was a significant negative association with physical abuse (b=−.76; F(1,109)=6.96, p=.010, ${\eta }_p^2=.06$) such that exploration was lower in those who experienced greater physical abuse. This remained significant in iADs alone (b=−1.05; F(1,56)=7.27, p=.009, ${\eta }_p^2=.11$). No effects were observed for any other parameter or CTQ subscale.

CRT scores were lower in iADs than HCs (t(117)=3.05, p=.003). In an LME assessing the effect of CRT on DE across all participants, there was a significant positive association (b=.80; F(1,114)=6.46, p=.012, ${\eta }_p^2=.05$). The effect of CRT was directionally the same in iADs alone (and stronger in effect size), but only marginally significant due to smaller sample size (b=.77; F(1,58)=3.89, p=.053, ${\eta }_p^2=.05$). There were no significant effects of CRT on RE or ${\mathit{\alpha }}_{\mathrm{\infty }}$ in either the full sample or in the clinical group alone (ps>.053). Higher CRT scores were associated with membership to the cluster with high ${\mathit{\alpha }}_0$ values in the full sample (z=−2.72, p=.007; log-odds=−.85, CI=[-1.46,-0.24]), but not in iADs alone (z=−1.61, p=.108; log-odds=−.77, CI=[-1.71,0.17])). Complementary nonparametric models using ${\mathit{\alpha }}_0$ as a continuous outcome variable also revealed a significant positive association with CRT scores in both the full sample (b=0.08, p<.001) and in iADs alone (b=0.07, p=.003).

Discussion

In this study, we compared decision-making behavior on an explore-exploit task in participants with and without clinically significant levels of anxiety and depression. A somatic (interoceptive) anxiety induction was used to dissociate the influences of state vs. trait anxiety. We hypothesized that state anxiety induction would reduce directed exploration (DE) as a potential mechanism promoting avoidance behavior. Results were mixed, offering only partial support for our primary hypotheses. First, as predicted, DE was reduced by the anxiety induction in HCs, suggesting reduced reflection on uncertainty. This is consistent with other correlational work in non-clinical samples linking anxiety to reduced DE (14, 15), as well as work showing that stress reduces exploration (21–23). Notably, while choice accuracy did not differ between conditions, higher levels of DE were associated with steeper improvements in task performance over time, suggesting reductions in DE during the anxiety induction could be viewed as maladaptive.

In contrast, DE in iADs remained stable between conditions. One interpretation of this result is that, as iADs experience greater state anxiety on a regular basis, they may have developed compensatory strategies that led them to be insensitive to this effect. This is consistent with other work suggesting compensatory processes in anxious individuals that allow them to maintain good performance (e.g., see 55, 56). On the other hand, these stable DE levels across conditions also suggest iADs had greater uncertainty during the anxiety induction about the average reward value of each choice in the task, or greater sensitivity to the difference in their uncertainty between options. This appears consistent with a few recent studies suggesting greater DE in those with higher levels of cognitive anxiety or worry (e.g., ^{29, 30}).

We also found that iADs showed slower learning rates across conditions, which may offer some additional explanatory insights. First, it is worth noting that these learning rates were also positively associated with task performance. This therefore suggests that slower learning may have contributed to worse performance in iADs. Here, slower learning implies that beliefs remained closer to uninformative prior values; thus, confidence in the better choice would increase more slowly (similar to less flexible learning rates previously associated with higher anxiety; (5)). Within this task, slower learning rates might thus support a type of persistent uncertainty about the estimated values of each bandit, with suboptimal effects on decision-making. Interpreting this result in light of previous findings is subtle, however, as faster learning rates are theoretically linked to greater uncertainty, and prior work has linked anxiety to both faster learning (6) and a greater tendency to infer changes in context (8). On the other hand, the type of uncertainty in these studies pertains to volatility, or how frequently environmental contingencies are expected to change. Thus, if anxious individuals believe the world is ever-changing, then learning rates should be high.

However, if “uncertainty” instead pertains to the estimated stochasticity of (i.e., noise in) the mapping from underlying states to observations, learning rate should instead be low, so as not to overfit beliefs to random outcomes (57). Thus, in the present task, slower learning in iADs could represent greater uncertainty about the informativeness of each outcome when inferring the underlying reward mean. This could offer a complementary means by which uncertainty is maintained in negative affect, consistent with the previous findings reviewed above. It could also relate to another recent study showing that individuals who experienced greater early adversity, itself a predictor of subsequent affective disorders (58–60), also showed slower learning rates (16). It should be kept in mind, however, that our sample includes individuals with affective disorders and used an anxiety induction, while most prior work has examined sub-clinical levels of anxiety and depression using correlational approaches. Thus, some results between studies may not be fully comparable. This was suggested by our follow-up results in which DE in HCs showed positive associations with trait anxiety in the resistance condition (i.e., even accounting for age and depression), while no such relationships were found in the clinical sample. These results support specific associations with trait anxiety, but also highlight how prior results in non-clinical samples (reviewed in ⁶¹) may not generalize to clinical samples (or perhaps suggest ceiling effects).

In line with our secondary aims, we were also able to successfully replicate prior results showing higher DE in those with greater cognitive reflectiveness (14) and lower DE in those who had experienced greater childhood adversity (i.e., physical abuse; ^{16, 17}). These findings suggest that less reflection on uncertainty and greater exposure to unpredictable/harsh early environments could each contribute to clinical differences seen here in exploration and learning. It is also worth noting that cognitive reflectiveness has been shown to improve with training (62–64). Future studies could therefore examine whether improving reflectiveness also optimizes uncertainty estimation or reduces affective symptoms. Another consideration is that multiple previous studies have shown strong relationships between physical abuse in childhood and later avoidance behavior (65, 66). Our results thus suggest differences in DE could contribute to this avoidance behavior in those recovering from childhood trauma, highlighting DE as a possible treatment target.

There are important limitations to consider. First, state anxiety levels were higher in iADs than HCs in both the no-resistance and resistance conditions. Thus, effects of state anxiety level may not be fully comparable between groups (e.g., if non-linear relationships are present). Additionally, the sample size was only moderate and may not have afforded sufficient power to detect some effects within the clinical group alone. Future research will also be needed to see whether results generalize to other explore-exploit tasks as well as tasks designed to distinguish learning rates in relation to volatility vs. stochasticity (57).

Conclusions

The results of this study suggest that somatic anxiety induction reduces directed exploration in healthy individuals, while those with affective disorders may be insensitive to this effect – possibly due to learned compensatory strategies (55, 56). They also suggest those with affective disorders display slower learning rates generally. Results further confirm lower levels of directed exploration in those displaying less cognitive reflection and in those who have experienced greater early adversity. These findings highlight potential computational mechanisms underlying the influence of anxiety on uncertainty estimation and maladaptive avoidance. If confirmed in future work, this could suggest potential benefits of treatments aimed at optimizing reflection on uncertainty, levels of information-seeking, and belief testing in relation to current affective states, as well as adjusting beliefs about the reliability of new experiences in revising expectations.

Data and code availability

All data used in the analyses described in this paper are available in Supplementary Materials. Model fitting code is publicly available in Zajkowski, Kossut (47) and was used here with minimal modification.