Reinforcement-learning signals support dynamic adaptive control during language switching

Abstract

Language switching exemplifies a real-world model of adaptive control, as bilinguals select languages in response to continuously changing contextual demands. Although the Adaptive Control Hypothesis (ACH) highlights context-dependent language control in bilinguals, how these strategies are learned remains unclear. Drawing on reinforcement learning (RL) theory, we examined whether reward prediction error (RPE) drives adjustments in voluntary language switching. Chinese-English bilinguals performed a voluntary picture-naming task with probabilistic (i.e., high, medium, and low) reward feedback that generated RPE depending on the switch decisions. Computational modeling revealed that RPE dynamically updated an abstract, generalizable switch policy. Representational similarity analysis (RSA) suggested the learned value of the control policy was represented in the MTL, whereas RPE representations emerged in the dorsolateral prefrontal cortex (dlPFC). Furthermore, connectome-based predictive modeling (CPM) revealed a multi-stage network process. A distributed, cross-network pattern initially supported early exploratory learning, which then transitioned into a centralized, hub-centric network for policy exploitation. Finally, the network specialized into a localized circuit for policy automation, coupled with a globally distributed network for monitoring unexpected errors. Together, these findings establish adaptive language control as a value-based RL process. This provides a neurocomputational framework for strategy learning, extending RL principles from simple stimulus-response to high-level cognitive control.

Language control is not driven by static costs or preferences, but by the dynamic learning of an abstract, generalizable action policy.

Introduction

Human cognition is fundamentally adaptive in that individuals constantly modify behavior to meet changing goals and environmental demands^1,2. This dynamic adaptability relies on the brain’s capacity to flexibly allocate cognitive resources, a set of operations known as high-level cognitive control^3,4. In real-world environments, demands on cognitive control are not static, but rather, they fluctuate rapidly and often unpredictably. The human mind operates as a dynamic predictor that continuously adapts to complex, volatile contexts. The present study adopts voluntary bilingual language switching as a naturalistic paradigm for testing dynamic adaptive control, wherein individuals must make real-time decisions about which language to use based on feedback. By embedding this task within a reinforcement learning (RL) framework, we examine how high-level control strategies are dynamically shaped by trial-wise feedback, offering a mechanistic account of adaptive behavior in a more ecologically grounded setting.

A naturalistic and cognitively demanding instance of high-level control can be found in bilingual language switching. Bilingual individuals manage two language systems, selecting the appropriate language in a given context and often switching between languages when the situation requires—a common behavior known as language switching^5–8. Regulating two languages in one mind to prevent interference and ensure fluent communication places significant demands on cognitive control processes. These demands are met by a set of operations known as language control, i.e., the ability to monitor, select, and inhibit linguistic information to meet communicative goals. Because the target language often changes unpredictably, effective language control must itself be adaptive, such that the amount of control applied from moment to moment is finely tuned (e.g., control increases when cross-language interference is high and decreases when interference is stable or low). As such, adaptive language control describes how these control mechanisms are regulated over time and how they adapt to various contexts.

The most influential account of such dynamic regulation is the adaptive control hypothesis (ACH)⁹, which proposes that bilinguals allocate language control according to the linguistic context in which they find themselves, applying minimal control in single-language settings, and heightened control in dual-language contexts where switching is frequent. Although the ACH captures context-level adjustments, the framework itself remains largely descriptive. For instance, in its current form, it does not specify the mechanism by which bilinguals update control on a trial-by-trial basis, nor does it offer a computational framework for learning from ongoing feedback. The ACH asserts that control is adaptive in the long run, but it does not specify how such adaptation occurs on a moment-to-moment basis. Moreover, formalized computational models quantifying how bilinguals learn and adapt based on ongoing environmental feedback are needed to extend the ACH. These issues are critical because the dynamics of language switching remain poorly understood in real-world contexts, where bilinguals must adapt their language strategies based on situational demands^5,9–12. Crucially, we still lack models capturing how bilinguals learn optimal language choices from trial-wise feedback.

Current paradigms used in bilingual control studies face limitations in capturing the adaptive nature of real-world language use. Voluntary switching designs can improve ecological validity by granting autonomy, but many do not have a clear motivational framework or interactive feedback to support bilinguals' language selection behavior, leaving them prone to habits rather than strategic adaptation. The present study advances this line of work by embedding voluntary switching within a goal-driven RL framework. In our experiment, switching is not a neutral option but a goal-directed choice, guided by probabilistic reward feedback. This innovation enhances ecological validity in two ways: it provides motivation by making language choice purposeful, and it fosters dynamic adaptation through trial-by-trial feedback in an uncertain environment. As such, we reframe language switching from a simple act of adaptive control into a learned decision-making process, thus more closely reflecting natural settings.

RL is an influential theoretical framework in cognitive neuroscience that provides a computational account of how individuals learn optimal behaviors through trial and error by updating their actions based on the rewards or penalties they receive. A key concept in RL is RPE, the difference between expected and actual outcomes, which serves as a teaching signal to drive learning, to iteratively adjust behavioral strategies, and to enhance adaptability across diverse contexts^13–17. The brain utilizes RPE to iteratively adjust behavioral strategies, continuously optimizing decision-making policies^17–19. In recent years, a theoretical framework has emerged that interprets cognitive control with principles of associative learning²⁰. This principle has been successfully demonstrated in the domain of task switching, where studies have shown that the action of switching tasks can be selectively reinforced, influencing subsequent voluntary behavior^21,22. However, two crucial questions remain. First, while this principle holds for domain-general task sets, it is unclear whether it extends to bilingual language switching, a unique form of high-level control requiring the dynamic modulation of lexical-semantic systems. Second, the neural mechanisms underlying the learning of such an abstract policy have not been specified.

The current study investigates the neural mechanisms of dynamic adaptive language control by examining how RL signals guide language control, using a combination of computational modeling and functional neuroimaging. Specifically, we use a computational RL paradigm to simulate a dynamic language context, providing cues and rewards for language selection. We examine whether bilingual speakers learn to optimize their language switching behavior through feedback, and how such learning is manifested in the brain.

To test the hypothesis that language switching is guided by an error-driven adaptive mechanism, we operationalized the induction of RPEs through the manipulation of reward feedback. Specifically, we created distinct reward conditions (high, medium, and low) based on the outcome of the preceding trial. By serving as proxies for positive, neutral, and negative RPEs, these conditions may dynamically modulate subsequent language switching behavior. In our behavioral analyses, we first used generalized linear mixed-effects models (GLMM) to test how the reward condition influenced the likelihood of a subsequent language switch. To quantify the underlying computational signals, we then applied a hierarchical Bayesian implementation of the Rescorla–Wagner model to generate trial-by-trial estimates of participants’ expectations and model-derived RPE¹⁵. At the neural level, we first identified brain regions sensitive to our feedback manipulation by contrasting activation across the different reward conditions. To pinpoint the neural representation of the learning signal itself, we then identified brain regions where activation covaried parametrically with the trial-by-trial model-derived RPE magnitude. Finally, we investigated how individual differences in functional connectivity might predict behavioral adaptability, revealing network-level mechanisms of adaptive language control.

Here, we show that reinforcement-learning signals support dynamic adaptive control during voluntary language switching. Behavioral and computational modeling results indicate that bilingual speakers use reward feedback to update an abstract, generalizable control policy rather than relying on static language preferences or fixed switching costs. Neuroimaging and representational analyses reveal that medial temporal regions encode the learned value of this control policy, whereas the dorsolateral prefrontal cortex dynamically tracks reward prediction errors as learning progresses. Connectome-based predictive modeling further identifies a multi-stage network process supporting exploration, policy exploitation, automation, and error monitoring. Together, these findings provide a neurocomputational account of adaptive language control as a value-based reinforcement-learning process.

Results

Positive reward condition elicits more language switching behavior

Our primary GLMM tested the effect of the previous reward condition (high reward, medium reward, low reward) on the switch probability  in the current trial. For this analysis, the medium reward condition was set as the reference level for model contrasts. The results revealed a significant main effect of reward condition, F = 31.86, p < 0.001. Post-hoc comparisons showed that participants were significantly more likely to switch languages following a high reward condition (mean switch probability M_High = 0.83, SD_High = 0.37) compared with both the low (M_Low = 0.54, SD_Low = 0.50; t_High-Low = −14.64, p_High-Low < 0.001) and the medium reward condition (M_Medium = 0.66, SD_Medium = 0.47; t_High-Medium = −9.46, p_High-Medium < 0.001).

Reinforcement-learning models support generalizable policy learning

We compared our primary AV model against a baseline non-learning model (Simulation model) in Fig. 1, demonstrating how the predicted value of the next action (Vt_switch) is updated based on the RPE and the subject-specific learning rate, while the choice probability (Pt_switch) is influenced by the inverse temperature parameter.

Fig. 1. Behavioral and computational validation of the AV model: experimental vs. simulated data.

A Trial-by-trial learning trajectories. Panel A shows the evolution of the estimated action value (value of sequence, black line, left y-axis) and the model-predicted choice probability (switch probability, red line, right y-axis) across 50 trials within 4 blocks. The experimental dataset exhibits a clear upward learning trend, indicating that participants gradually acquired the underlying rule. In contrast, the simulated dataset fluctuates randomly around zero, showing no learning progression. Shaded regions represent the SEM. B Comparison of trial-by-trial latent variables. Violin plots with overlaid individual data points demonstrate that the experimental group generated significantly higher mean RPE and mean VS compared with the simulated group. Boxplots represent the median and interquartile ranges. (*** indicates p < 0.001, t-test). C Distributions of estimated computational parameters. Density plots compare the parameter profiles of the two groups. The left panel shows the distribution of the learning rate, and the right panel shows the inverse temperature parameter. Dashed vertical lines indicate the group means. The experimental dataset (red) exhibits a distinctively higher inverse temperature compared to the simulated dataset (blue), confirming that human participants utilized a deliberate, value-guided decision strategy rather than random responding. (*** indicates significant group differences).

When fitted to the experimental data, the AV model revealed the learning trajectory. As shown in Fig. 1A, participants’ switching behavior was accompanied by higher positive values for the next trial, indicating gradual identification of the underlying reward action rule. In contrast, the simulation group showed no such learning, with reward predictions remaining unstable and centered around zero. The experimental group (t = −28.30, p < 0.001) (M = 0.302, SD = 0.355) generated a significantly higher VS than the simulation group (M = −0.006, SD = 0.630). Furthermore, the experimental group obtained significantly more (t = 5.71, p < 0.001) positive prediction errors (M = 0.065, SD = 0.642) than the simulation group (M = −0.004, SD = 0.476) (see Fig. 1B).

Group-level posterior distributions of parameters were compared. Bayesian parameter estimation showed higher inverse temperature in the experimental group than the simulation group (Fig. 1C), with a 95% CI for the mean difference (simulation-experimental) of  [−2.95, −1.85]. The 95% CI excluded zero, with a near 100% posterior probability that the experimental group’s parameter was larger (P_{exp > sim} = 99.88%), confirming high credibility. The difference in the learning rate between the two groups was not robust (M = 0.145, 95% CI = [−0.02, 0.46], P_{sim > exp} = 89.03%). Human subjects’ decision-making was highly targeted (high inverse temperature), using learned values to guide actions, whereas simulated data updated values similarly but sampled actions randomly, unable to convert values into a strategy. This contrast confirms participants’ behavior was nonrandom and that the AV model captured real, feedback-driven learning.

The comparison results (Supplementary Table S1) identified the dynamic learning models, specifically the LVGP-Asym model and the AV model, as the top-performing models. Stacking analysis assigned slightly higher weight to LVGP-Asym (0.49) than to AV (0.40). Both dynamic learning models were significantly better than the static “bias” model (AVc model) and the rational comparison model (LVGP-Sym model).

Our first question was whether participants learned an abstract, generalizable policy or stimulus-specific associations. To test this, we compared the AV model against the SV model. The AV model fit better than the SV model, with a substantial difference in expected log pointwise predictive density (ΔELPD_WAIC = −333.1, comparing SV ELPD_WAIC = −2358.04 relative to AV ELPD_WAIC = −2024.92) (see Supplementary Table S1). The SV model received near-zero stacking weight (<0.001), while policy-based models dominated. This confirms that participants learned a generalized “non-switch vs. switch” policy transferable across stimuli, rather than stimulus-specific values (see Fig. 2A, B, and Supplementary Table S1).

Fig. 2. Bayesian model comparison.

A WAIC model comparison. Panel A shows the ELPD difference for each candidate model relative to the best model based on the WAIC. The top-performing LVGP-Asym model serves as the reference point (difference = 0). Negative values on the x-axis indicate worse out-of-sample predictive accuracy. Error bars represent the standard error of the ELPD differences. B Model stacking weights. Panel B displays the relative predictive weights for each model (constrained to sum to 1). C Prediction accuracy by AV and LVGP-Aysm model. D The distribution of learning rates. The selected AV model (red) is sharply peaked near zero, suggesting a robust, conservative learning strategy. In contrast, the LVGP-Asym model (orange) captures a more dynamic learning rate (α ≈ 0.14), differing markedly from the broad distribution of the symmetric model (brown). E The distribution of inverse temperature. This parameter controls choice randomness. Most models show a consistent high inverse temperature range (β ≈ 2.5–3.0 indicating stable and deterministic choice behavior. The exception is the pure language-value model (green), which exhibits extremely low β values (peaking < 1.0), reflecting a failure to capture choice. F Model-specific parameters: left panel (bias): the static bias parameter in the AVc model (purple) shows a small positive deviation rather than a null effect; however, this minor bias was insufficient to justify the added complexity compared to the simpler AV model. Right panel (weight w): the LVGP-Asym model (orange) clusters around a negative value w ≈ −1.0), implying that high language value facilitates switching (i.e., success-induced flexibility). Conversely, the LVGP-Sym model (brown) clusters around a small positive value, thereby failing to capture the dominant mechanism of flexibility. Model comparison and parameter estimates were based on n = 23 independent participants.

Having established abstract learning, we asked whether it involved actions (non-switch/switch) or merely language preferences (L1/L2). We compared the AV model against the LV model. The AV model was significantly better than the LV model (ΔELPD_WAIC = −954.8, comparing LV ELPD_WAIC = −2979.74 relative to AV ELPD_WAIC = −2024.92) (see Supplementary Table S1). This indicates behavior was driven by dynamic learning of switching actions (i.e., assessing switch worthiness) rather than simple language value updates (see Fig. 2A, B).

To understand dynamic gating, we tested its mechanism by comparing the Asymmetric model (evaluating only the current language) against the Symmetric model (rationally comparing V_L1 vs. V_L2). The LVGP-Sym model performed worse (ΔELPD_WAIC = −995.3, comparing LVGP-Sym ELPD_WAIC = −2982.41 relative to the LVGP-Asym ELPD_WAIC = −1987.04), ranking last among policy models (see Supplementary Table S1 and Fig. 2A, B). This suggests language switching is not a rational maximization process where participants constantly weigh the benefits of L1 vs. L2. Instead, it is a success-induced control process.

Given action-policy driven behavior, we examined if it was augmented by static language bias or a dynamic mechanism. We compared AV, AVc (learning + static bias), and LVGP-Asym (learning + dynamic gating). The AVc model performed worse than the AV model (ΔELPD_WAIC = −19.4, comparing AVc ELPD_WAIC = −2044.34 relative to AV ELPD_WAIC = −2024.92), indicating static bias added complexity without explanatory value. Although the LVGP-Asym model (ELPD_WAIC = −1987.04; stacking_weight = 0.49) exhibited a slightly better fit than the AV model (ELPD_WAIC = −2024.92; stacking_weight = 0.41), the difference was not statistically significant (i.e., ΔELPD < 2SE) (see Fig. 2A, B, and Supplementary Table S1). Additional analyses confirmed the predictive performance and behavioral validity of the AV model (Fig. 2C and Supplementary S1). Thus, according to the principle of parsimony, we selected the AV model as the primary regressor for the main fMRI analysis.

Subject-level ELPD differences revealed heterogeneity: roughly half favored LVGP-Asym, half AV (Supplementary Fig. S2). This explains near-equal stacking weights. For predictive performance and behavioral validation of AV, see Supplementary Materials S1 section “Reinforcement models reliability and fit”).

The group-level posterior parameter distributions across the six models were presented in Fig. 2D–F. AV model exhibited a sharper, lower distribution of learning rates than complex models (e.g., LVGP-Asym) (Fig. 2D). This suggests that the AV model’s higher predictive accuracy may stem from a more conservative and updating strategy that resists overfitting to trial-by-trial noise. The inverse temperature distributions (Fig. 2E) were consistent across top models (higher values, indicating deterministic choice), but poor performers like LV showed low values (random decisions; Fig. 2E). For LVGP models’ control weight (w), LVGP-Sym trended positive (promoting perseverance), while LVGP-Asym estimated negative weights (group means w < 0; Fig. 2F).

In our framework, this negative weight w means higher current language values decrease staying odds, facilitating switches. This negative-weight mechanism (LVGP-Asym) is significantly better than the positive-weight counterpart (LVGP-Sym), which provides evidence that language control is dynamically gated by success to prevent perseveration.

Positive reward enhances switch-related brain activity

The results of the full factorial analyses indicated significant main effects of language sequence and reward condition (see Supplementary Fig. S3 and S4, Supplementary Tables S2 and S3). A significant interaction between language sequence and reward condition was reflected in the right precentral gyrus (PreCG), left middle temporal gyrus (MTG, BA37_L area), right superior temporal gyrus (STG), BA20_L area (left inferior temporal gyrus, ITG_L), and left cerebellum (CB) (Fig. 3A). Further ROI analyses showed that switch trials elicited stronger activity than non-switch trials after high reward conditions in the right PreCG, left MTG, right STG, BA20_L, and left CB (see Fig. 3B and Supplementary Table S4), while no such differences were observed after low reward conditions. In contrast, after the medium reward condition, neural activity significantly decreased more on switch trials compared to non-switch trials (see Fig. 3B and Supplementary Table S4).

Fig. 3. Reward-related activity and neural representations of computational decision variables.

A The significant brain regions reflected by an interaction between language sequence × reward condition. B The simple effects show differences between conditions. Group-level fMRI analyses were based on n = 23 independent participants. C Neural representation of value in sequences (VS). Results from the whole-brain searchlight RSA using partial Spearman correlation, controlling for temporal distance between trials (Analysis A). Significant clusters where neural pattern similarity correlated with value similarity were identified in the right parahippocampal gyrus/fusiform gyrus and left inferior temporal gyrus. The map is threshold at p < 0.001 (corrected) with a cluster extent threshold of k ≥ 10 voxels. The color bar represents the t-statistics of the RSA effect. Note that controlling temporal distance was critical to isolate value-specific signals from hemodynamic aliasing in the rapid event-related design. D Dynamic emergence of RPE representations. Results from the learning-stage analysis (Analysis B), identifying regions where RPE representational strength exhibited a significant linear increase across the four learning blocks. The glass brain shows a significant cluster in the dlPFC and supplementary motor area (threshold at p < 0.001, k ≥ 10). Bar plot extracting the mean Fisher z-transformed correlation coefficients from the peak dlPFC cluster for each learning block. Error bars represent the standard error of the mean (SEM). The trend illustrates that RPE representations in the prefrontal cortex became progressively more distinct as learning proceeded. RSA group-level statistics and bar plots were based on n = 23 independent participants.

Neural representations track value and RPE

Using the trial-by-trial VS and RPE derived from the AV model, we conducted a partial correlation RSA. Prior to the main analysis, we assessed the degree of collinearity between our model variables and the temporal structure of the task. As expected in learning tasks, the VS showed a correlation with the block structure (mean Spearman r = 0.42 between model RDM and temporal/block RDM), reflecting the accumulation of value over time. In contrast, the PE signal showed negligible correlation with temporal distance (mean r = 0.02), indicating that RPE magnitudes were independent of trial timing. Thus, we used the partial correlation method to isolate real neural representations from temporal aliasing.

Using the partial correlation RSA (controlling for temporal distance), we identified significant clusters where local neural patterns encoded the similarity structure of the chosen option’s value. Our control method revealed highly specific representational effects in the medial temporal lobe. Significant clusters were observed in the right parahippocampal gyrus (peak MNI: 21, −42, 9; k = 41, t = 4.82, p < 0.001), and fusiform gyrus (peak MNI: 27, −45, −6; k = 15; t = 5.51, p < 0.001, corrected) and the left inferior temporal gyrus (peak MNI: −45, −6, −27; k = 12, t = 5.09, p < 0.001) (see Fig. 3C).

The analysis of learning dynamics revealed RPE had a significant positive linear trend in the left dorsolateral prefrontal cortex (dlPFC/middle frontal gyrus) (peak MNI: −39, 18, 39; k = 21, t = 4.94, p < 0.001) (see Fig. 3D). This finding indicates that the multivariate encoding of RPE in the dlPFC became progressively more distinct as learning proceeded from Block 1 to Block 4.

Learning-phase-specific network patterns predict individual differences

We use CPM to predict computational parameters (learning rate, inverse temperature, and prediction error), which are derived from our AV model. This approach revealed a dynamic, multi-stage neural progression that uniquely tracks distinct computational demands as learning evolves across blocks.

In the early phase (Block 2), as participants actively updated their internal models based on feedback, we identified a network of positively correlated edges that significantly predicted participants’ learning rate (r = 0.43, p = 0.039) (see Fig. 4). A higher learning rate indicates a faster, more sensitive integration of recent feedback into value updates. The predictive neural substrate was characterized by a distributed architecture (38 connections) predominantly driven by between-network interactions. Network 1 served as the primary topological hub, with a single high-degree node (Node 64, degree = 19) orchestrating extensive cross-network connections, particularly with Networks 4, 3, and 5. The negative model for this parameter was not significant (r = −0.18, p = 0.42).

Fig. 4. Neural predictive network for learning rate during the early learning phase (Block 2).

A Circle plot visualizing the 38 positive edges that significantly predicted higher learning rates. Edges are color-coded by the functional networks they connect. B Anatomical representation of the predictive network. Red nodes represent hubs with high degree centrality. C Hub analysis highlights Node 64 (Network 1, degree = 19) as a primary connector, orchestrating interactions with Networks 3, 4, and 5. D Sagittal and axial views of the functional connections. E Scatter plot illustrates the significant correlation between the observed learning rate and the values predicted by the positive network (r = 0.432, p = 0.039). Shaded areas represent 95% confidence intervals. F Adjacency matrix of the identified positive edges. CPM prediction analyses were based on n = 23 independent participants; each point in the scatter plot represents one participant, with the same sample size and data representation applying to Figs. 5–7.

By Block 3, a distinct neural pattern emerged to predict the choice policy parameter, inverse temperature. The strength of a positive network significantly predicted higher inverse temperature values (r = 0.49, p = 0.017), reflecting a shift towards a more deterministic, exploitative decision-making strategy. Unlike the globally distributed networks seen earlier, this predictive network was uniquely centralized around a massive hub, Node 6 within Network 3, which accounted for the majority of the connections (degree = 55) (see Fig. 5). This specific hub anchored long-range connections to multiple large-scale systems (Networks 4, 5, 6, and 8), suggesting that mid-stage policy exploitation is globally broadcasted from a core Network 3 hub.

Fig. 5. Hub-centric network predicting inverse temperature in the mid-learning phase (Block 3).

A, B The positive predictive network (81 connections) is characterized by a dense concentration of edges anchored by a specific hub. C Topological analysis identifies Node 6 (Network 3) as a massive hub (degree = 55), mediating long-range connections to Networks 4, 5, 6, and 8. This centralized architecture reflects a shift towards exploitative decision-making. D Sagittal and axial views of the functional connections. E Leave-one-out cross-validation results show a significant positive correlation between the network strength and inverse temperature (r = 0.491, p = 0.017). F Adjacency matrix of the identified positive edges.

In the final phase of the task (Block 4), as learning stabilized, the neural substrates diverged into two highly specific functional architectures tracking choice policy (inverse temperature) and model accuracy.

First, we found that a negative model emerged for the inverse temperature (r = 0.45, p = 0.03) (Fig. 6). In contrast to the hub-centric network in Block 3, this late-phase predictive network for inverse temperature was highly specialized and restricted. All consistently identified predictive edges were entirely localized within Network 5 (Nodes 60, 92, and 202), consisting exclusively of within-network connections. The positive model for this phase was not significant (r = 0.28, p = 0.20).

Fig. 6. Localized within-network connectivity predicts inverse temperature in the late learning phase (Block 4).

A, B In contrast to the distributed networks in earlier blocks, the predictive features for late-stage policy tuning are highly sparse and localized. C The significant negative network consists exclusively of connections within Network 5 (Nodes 60, 92, and 202). D Sagittal and axial views of the functional connections. E Scatter plot showing that the strength of this negative network significantly predicts individual differences in inverse temperature (r = 0.446, p = 0.033). Note that the positive model was not significant. F Adjacency matrix of the identified negative edges.

Concurrently, a positive network emerged to predict RPE in Block 4. The positive model significantly predicted observed RPE (r = 0.48, p = 0.02) (Fig. 7). To support this late-stage error monitoring, the brain recruited a massive, globally distributed architecture consisting of 209 connections. This widespread cross-network communication was anchored by distinct high-degree hubs across Network 2 (Node 184), Network 7 (Node 214), and Network 4 (Node 113). The negative model showed an opposite-direction association and therefore did not meet the criterion for a successful CPM prediction  (r = -0.46, p = 0.03).

Fig. 7. Large-scale distributed architecture supports RPE processing in the late learning phase (Block 4).

A, B A widespread positive network comprising 209 edges emerged as a significant predictor of RPE. C The network topology reveals a globally distributed system anchored by multiple high-degree hubs, including Node 184 (Network 2), Node 214 (Network 7), and Node 113 (Network 4). D Sagittal and axial views of the functional connections. E CPM results indicate a significant correlation between the positive network strength and observed RPE (r = 0.48, p = 0.02). F Adjacency matrix of the identified positive edges.

Discussion

Our study provides a multi-level account of how RL principles guide language control during language switching among bilinguals. Manipulating trial-wise RPEs revealed that bilingual speakers exploit reinforcement signals to update language choices in real time. Hierarchical Bayesian reinforcement-learning modeling corroborated that this adaptive control was governed by an RL process that dynamically learns the value of an abstract, generalizable action policy. Specifically, the choice between “switch” and “non-switch” was characterized by low learning-rate and high inverse-temperature parameters, suggesting that participants operated in an exploration-heavy regime rather than simple incremental learning. This computational mechanism is supported by a multi-level neural architecture. At the neural level, we found that brain responses to language control were modulated by RPEs, with enhanced activation in regions including the left MTG, right PreCG, among others, highlighting the sensitivity of adaptive control networks to feedback. At the representation level, AV model parameters, such as VS and RPE, were found to be encoded in the MTL and dlPFC. At the network level, we identified a dynamic, multi-stage process of whole-brain functional connectivity that predicted individual differences in RL parameters as subjects transitioned from early, exploratory learning to late, exploitative decision-making. Together, these results provide a model-based account of how bilinguals learn and implement adaptive control strategies.

A central principle of cognitive control theories, such as the expected value of control framework²³, is that the allocation of control is continuously adjusted based on a cost-benefit analysis. Our computational modeling results provide evidence that bilingual language switching operates under such a framework. The model with a fixed bias parameter (AVc model) was worse than the purely dynamic learning model (AV model), suggesting that the switch cost is more like a dynamic opportunity cost, learned implicitly through trial-by-trial reinforcement.

Our step-by-step model comparison further specified what is being learned. The better performance of the AV model over both the SV and LV models provides a clear finding that RL in this context targets an abstract cognitive policy. Participants were not merely forging stimulus-response mappings, nor were they simply updating a general preference for L1 or L2. Instead, they learned the instrumental value of the act of switching itself, a policy that could be generalized across novel stimuli. It suggests a learning mechanism operating at the level of cognitive strategy, grounding the abstract concept of control allocation in a computationally tractable and generalizable learning process.

Finally, our findings offer an explanation of how this learned policy is deployed. The performance of the symmetric model, which assumes a rational comparison between the values of both languages, argues against an exhaustive maximization process. Instead, the success of the asymmetric gating mechanism suggests a more computationally efficient, heuristic strategy. This aligns with principles of bounded rationality²⁴, whereby the cognitive system avoids the computational overhead of continuously evaluating all possible options. We interpret this as a success-induced flexibility mechanism; the system primarily monitors the utility of the current state. A policy shift is triggered not by a rational calculation of global optima, but by the perceived inadequacy of the current state or the high learned value of the alternative switch policy. It is worth noting that while our primary neuroimaging analyses were based on the more parsimonious AV model, the comparable performance of the LVGP-Asym model and the observed inter-subject heterogeneity suggest that individuals may differ in their reliance on such gating heuristics, a promising avenue for future investigation.

Contrary to traditional psycholinguistic models that often conceptualize language switching through the lens of static switch costs or fixed language proficiencies, our model comparisons demonstrated that the bias parameter (as tested in the AVc model) added complexity without explaining additional behavioral variance. Instead, bilingual decision-making in our task was best captured by purely dynamic valuation processes (AV and LVGP-asymmetric models). This suggests that the expected value of engaging cognitive control (i.e., making a switch) is actively and sufficiently updated through experience via RPEs. When a language switch leads to a high reward, the positive RPE updates the expected value of the switch policy, making it more probable in subsequent trials.

In addition, this computational mechanism provides an extension to the ACH⁹. The ACH posits that bilinguals flexibly adjust their cognitive control based on the demands of different interactional contexts (e.g., from a single-language to a dual-language context). However, the precise mechanism driving this adaptation has remained unknown. Our findings suggest that context adaptation is achieved through trial-by-trial RL of an abstract control policy. By continuously evaluating the expected value of switch versus non-switch, the bilingual can dynamically infer the adaptive control for the current environment. Thus, our model grounds the flexible contextual adaptations proposed by the ACH into a computationally value-driven learning process.

Our neuroimaging results provide a neural basis for the computational model. The findings demonstrated that a network of brain regions is selectively tuned to positive feedback during strategic adjustments. Our ROI analyses revealed that positive RPEs were associated with heightened neural responses during language switch trials compared to non-switch trials across multiple regions, including the right PreCG, left MTG, right STG, BA20_L, and left cerebellum. This finding suggests that the language control system is sensitive to reward feedback, particularly when a language switch yields a better-than-expected outcome. These regions may collectively support adaptive adjustments by integrating language selection (MTG, STG, and BA20) with action planning and execution (PreCG). Collectively, this pattern of neural modulation by positive RPEs provides a potential neural basis for how bilinguals dynamically adjust language control strategies, reinforcing beneficial switching behaviors.

Having established a computational model of abstract policy learning, we use RSA to examine  neural representation of its computational variables. Our findings localize the representation of the chosen action’s value (VS) to the MTL, specifically the parahippocampal and fusiform gyri. The MTL is a well-established hub for both episodic memory and the representation of value associated with specific states and actions²⁵. The presence of VS representations in this region may suggest that the abstract, learned value of the switch vs. non-switch policy is stored and retrieved via the brain’s core memory and valuation systems. The MTL may maintain the memory of how rewarding a language control policy has been in the past, providing a stable value signal to guide future decisions.

Our analysis suggested the dynamic emergence of RPE representations within the left dlPFC. Rather than being a static feature, the fidelity of RPE encoding in the dlPFC progressively increased as learning unfolded across the task in a dynamic pattern. Initially, when the internal model of the task is weak, RPE signals may be noisy and less informative. As learning proceeds and the policy model becomes more refined, the dlPFC appears to become increasingly adept at utilizing RPE signals to drive meaningful policy updates. This aligns with the established role of the dlPFC in higher-order cognitive control, working memory, and strategy adaptation^26–28. The dlPFC acts not as a passive recipient of error signals, but as an active learning system that leverages these signals with increasing efficiency to refine the abstract control policy. The ACH theory features the prefrontal cortex as a critical hub for updating control and adapting to new linguistic demands. Our RSA results specify what this region is computing: it progressively tunes into trial-by-trial prediction errors to drive policy updating. Taken together, these RSA findings suggest a memory-based system in the MTL represents the accumulated value of the language control policy, while a dynamic, executive system in the dlPFC actively utilizes RPE to update and optimize this policy online. This dissociation between value storage and active learning provides a neural foundation for the RL mechanisms identified in our behavioral modeling.

Our CPM results revealed a dynamic, multi-stage network reconfiguration that aligns with the evolving cognitive demands of switch policy learning. In the early phase of learning, we found that a distributed, cross-modular network predicted individual differences in learning rates. This network is suited for the initial, exploratory stage of learning, where the primary cognitive challenge is to integrate diverse information from multiple systems (e.g., reward, control) to build a coherent model of the task environment. Individuals with stronger functional integration across these networks were better able to use feedback to rapidly update their policy values, reflecting a neural mechanism optimized for information gathering and model building.

As learning progressed to a mid-stage, the neural architecture underlying decision-making underwent a shift. A highly centralized, hub-centric network emerged to predict the choice inverse temperature parameter, which governs the transition from exploration to exploitation. The emergence of a centralized dominant hub suggests a shift towards policy consolidation, where a core network begins to broadcast the learned, high-value policy to downstream motor and executive systems. This centralization facilitates a stable and consistent application of the discovered strategy, marking a transition from flexible learning to action implementation. In the final learning phase, the neural network showed greater functional specialization. On one hand, the network predicting choice determinism became highly localized, consisting of within-module connections. This may suggest a process of policy automation, where the well-learned strategy is implemented within a specialized, efficient local circuit. On the other hand, a massive, globally distributed network emerged to predict RPE.

While the ACH characterizes how brains differ across communicative contexts, our CPM findings suggested the temporal trajectory of adapting to a new context. We show that transitioning into a proficient adaptive state requires shifting from a globally distributed network to a highly efficient, localized network, supplemented by error monitoring. This mechanism ensures both efficient performance and the capacity for rapid adaptation should the environment change, providing a neural mechanism for balancing stability and flexibility in learned behavior.

In conclusion, our study provides a multi-level account of the computational and neural mechanisms underlying adaptive language control. Our findings reveal that this process is not driven by static costs or preferences, but by the dynamic learning of an abstract, generalizable action policy. This computational framework is instantiated in the brain through a complementary neural mechanism; the MTL represents the learned value of the control policy, while the dlPFC dynamically tracks RPE to guide learning. In addition, we found a multi-stage network process that supports this learning, transitioning from a distributed network for initial exploration to a hub-centric architecture for policy exploitation, and finally to a specialized system of localized circuits and global error monitoring. These findings reveal a computational process framing adaptive control as a dynamic process of learning, implementing, and monitoring abstract policies, aligning with theories of dynamic cognitive control, demonstrating that feedback-based learning signals can fine-tune language control processes in voluntary naming tasks.

Method

Participants

We conducted an a priori power analysis using G * Power 3.1²⁹ to determine the required sample size for a 2 × 3 within-subjects ANOVA. The analysis was based on an expected effect size of f = 0.25 (medium effect size), α level of 0.05, desired statistical power of 0.80, and six repeated measures per participant. The correlation among repeated measures was set to 0.5, and the nonsphericity correction was set to 1. The analysis indicated that a minimum sample size of 19 participants was required to achieve sufficient statistical power (actual power = 0.8198). Given this estimate, we recruited 31 unbalanced Chinese-English (L1–L2) bilingual participants. All participants were right-handed, had normal or corrected-to-normal vision, and reported no history of neurological, psychological disorders, or psychological impairments. The fMRI data analyses excluded 8 participants, three due to excessive head movements during scanning, and five for excessive missing responses, resulting in a final sample of 23 participants (11 males, M ± SD = 21.55 ± 2.14 years) for analysis. Sensitivity analyses indicated that effects at the ROI level (dz ≥ 0.58), voxel-level (dz ≥ 0.79), and CPM correlations (r ≥ 0.56) can be reliably detected with this sample size (see Supplementary Materials Section 2 for the detailed power analysis)¹ (The sample size (N = 23), while adequately powered for detecting our primary behavioral and medium-to-large neural effects within a within-subjects design, is somewhat small by current neuroimaging standards. This is particularly relevant for the connectome-based predictive modeling analyses, as brain-behavior correlations typically require larger samples to yield stable and generalizable predictive models. Although our use of permutation testing ensures the statistical validity of our findings within this sample, the specific predictive edges identified should be interpreted with some caution. Future studies should consider larger sample sizes to test the replicability of the study and to potentially extend these network-level findings).

The experiment was approved by the Research Center of Brain and Cognitive Neuroscience at Liaoning Normal University (no. LL2025109), and all volunteers provided their written informed consent prior to participating in the study. All ethical regulations relevant to human research participants were followed. The participants’ mean age of L2 acquisition and self-ratings of L1 and L2 skills are shown in Supplementary Table S5. The self-ratings were based on a scale of 1–6, with “1” meaning “no proficiency” and “6” indicating “highly proficient”. Paired-sample t-tests revealed that the participants’ L1 proficiency was significantly higher than their L2 for listening, t(22) = −14.45, p < 0.001, speaking, t(22) = 22.38, p < 0.001, reading, t(22) = −24.22, p < 0.001, and writing, t(22) = −7.91, p < 0.001.

Materials

The materials for the voluntary language switching task consisted of 52 black-and-white line drawings from Snodgrass and Vanderwart’s picture set (standardized also by Zhang and Yang)^30,31. Of these, 48 images were used in the formal experiment, and 4 were included in the practice trials. Twenty age-matched bilinguals who were independent of this study rated the subjective familiarity of the pictures’ names in Chinese and English on a 5-point scale (1 = “very unfamiliar”, 5 = “very familiar”). T-tests indicated no significant difference in familiarity between the two languages (M_L1 = 4.79, SD_L1 = 0.12, M_L2 = 4.81, SD_L2 = 0.10), t(47) = −1.48, p > 0.05. We also consulted a word frequency database for Chinese³² and English³³. T-tests revealed no significant difference between the frequency of the L1 and L2 names of the pictures (M_L1 = 77.53, SD_L1 = 114.24, M_L2 = 104.23, SD_L2 = 128.39), t(47) = −1.54, p > 0.05.

In language switching tasks, trials are classified as either switch trials or non-switch trials based on whether the language used in the immediately preceding trial was different or the same, respectively. Switch trials typically require greater cognitive control as participants must inhibit interference from the previously used language while (re-)activating the lexical and syntactic rules of the new language. Non-switch trials, on the other hand, demand less cognitive effort since participants can continue using the same language without the need for heightened cognitive control. In the present study, the language sequence was not predetermined but was calculated based on the language used by participants during voluntary naming of pictures across trials.

Design and procedure

As shown in Fig. 8A, in the experiment, a 75% probability of presenting a token of 9 (high reward outcome) and a 25% probability of a token of 5 (medium reward outcome) were set for switch trials, while a 75% probability of a token of 1 (low reward outcome) and a 25% probability of a token of 5 (medium reward outcome) were set for non-switch trials. This reward structure was designed to implicitly encourage participants to switch languages, and as such, they were not informed of the underlying reward rules. Language switching is a key skill for bilinguals, as it enables them to adapt to different contextual demands. Frequent language switching allows us to examine how RPEs influence language control, helping to shed light on the adaptability of language selection in dynamic environments.

Fig. 8. Overview of the experiment.

Experimental design (A) and trial procedure (B). “中文” means “Chinese”. The token image shown at the beginning of the trial timeline is a schematic for the generic, non-informative motivational cue, which was identical on all trials. The token image at the end of the trial timeline represents the actual reward outcome, which varied based on performance.

The experiment was programmed in Psychtoolbox (www.psychtoolbox.org). Before undergoing the formal experiment in the MRI machine, participants were required to complete a practice experiment consisting of 25 trials to familiarize themselves with the experimental procedures. The procedure for the practice experiment was the same as for the formal experiment, except that the former did not include reward rules or images that were used in the latter to avoid practice effects.

After completing the practice experiment, participants entered the MRI for the formal experiment. The experiment included a total of 200 trials, presented in 4 blocks of 50 trials. We considered the first two trials in each block (8 total trials) as warm-up trials and thus, excluded them from the data analyses. The formal experiment consisted of 48 images, with each image appearing four times, for a total of 192 trials. Each block lasted approximately 7.5 min. The total duration for the functional imaging scans was 30 min, with an additional 5 min allocated for structural imaging. The order of functional and structural imaging was balanced between subjects.

Prior to the experiment, participants were instructed to maximize their rewards (as represented by tokens). As illustrated in Fig. 8B, each trial began with a fixation cross for 500 ms. This was followed by a standard, non-informative motivational cue (a generic token image, displayed for 1000 ms), which was identical on all trials and served only as a general attentional alert. Subsequently, a picture appeared for 2000 ms, during which participants named it either in their L1 or L2. Due to strict head movement restrictions, participants were advised to name the pictures aloud softly. The participants then made a language choice by pressing “1” for L1 Chinese or “3” for English. These response keys were counterbalanced across participants. Asking participants to softly name their responses was intended to ensure active lexical retrieval, but the vocalizations themselves were not recorded during the fMRI scan. The button press was treated as the sole and definitive response for each trial. Accordingly, the reinforcement schedule was linked exclusively to the button press, which determined whether the trial was coded as a switch or non-switch and which reward was subsequently delivered. The actual reward outcome received was then displayed based on the language sequence for 2000 ms. Finally, inter-trial intervals varied randomly between 1000, 2000, and 3000 ms. This jittered, rapid event-related design was chosen to maximize trial counts and statistical power for effect detection³⁴.

Because the fMRI experiment did not record participants’ vocal naming responses, we conducted an independent supplementary behavioral experiment with a separate group of 20 participants. The paradigm and stimuli were identical to the fMRI task, but vocal responses were recorded for offline analysis. This allowed us to verify (1) whether the language actually produced by participants matched the one indicated by their button press and (2) whether the naming response itself was correct. Results showed low error rates: the speech-key mismatch rate was 0.76% (SD = 0.73%), with each participant below 5% (range: 0.00–2.60%); the naming error rate was only 0.26% (SD = 0.52%), also with all participants below 5% (range: 0.00–2.09%). The two error types were uncorrelated (r = 0.25, p = 0.30), indicating they arise from independent sources and thereby supporting the validity of the button-press data. In addition, computational modeling (AV and LVGP-Asym models) applied to both datasets revealed no significant differences in any key parameters (learning rate, inverse temperature, or language weight) between the fMRI and behavioral experiments, confirming that the cognitive mechanisms engaged during scanning were fully preserved in the behavioral validation task. Detailed statistics, individual-level results, and model comparisons are provided in the Supplementary Materials Section 3 “Behavioral experiment.”

fMRI data acquisition

The fMRI data were acquired using a GE Discovery MR750 3 T magnetic resonance scanner at Liaoning Normal University. Functional images were obtained using T2-weighted echo planar imaging (EPI) with a voxel size of 3.5 × 3.5 × 4.2 mm³, an interslice gap of 0.7 mm, a repetition time (TR) of 2000 ms, an echo time of 30 ms, and a flip angle set at 90°. Each whole-brain acquisition scan consisted of 33 slices with a thickness of 3.5 mm, an image matrix size of 64 × 64, and a field of view (FOV) of 224 × 224 mm. There were four scanning sessions in total, each comprising 450 time points.

Structural images were acquired using a T1-weighted 3-D MPRAGE sequence, with a TR of 6.652 ms, an echo time of 2.928 ms, a flip angle of 12°, a slice thickness of 1 mm, a slice spacing of 1 mm, an image matrix of 256 × 256, an FOV of 256 × 256 mm, and 192 slices in total. The voxel size was 1 × 1 × 1 mm³. Participants were instructed to make their utmost effort to keep their heads still during the entire scanning session.

Data analysis

We used a generalized GLMM with language sequence and previous trial reward condition (high reward, medium reward, low reward) as fixed effects, as well as the by-subject and by-item random intercepts, to examine the behavioral effect of reward condition on language sequence³⁵. This approach effectively controls potential confounding variables, such as the language sequence and different types of reward conditions, allowing us to isolate the specific error-driven adaptive effects of reward conditions on language switching behavior.

To model participants’ learning behavior in the voluntary language switching task, we used a Bayesian hierarchical RL algorithm^15,36 to model experimental and simulated data. Experimental data refer to the real behavioral data collected from participants, while simulated data represents data generated computationally under the same task structure but without RL mechanisms. For trial-by-trial latent variables (e.g., VS, RPE), frequentist statistics (Welch’s t-tests) were applied to account for unequal variances. For computational model parameters derived from the Bayesian Hierarchical Model, group differences were evaluated directly by computing the difference between their posterior distributions, assessing whether the 95% highest density interval excluded zero. This approach maximizes the utility of the Bayesian framework by fully accounting for parameter uncertainty.

Furthermore, RL modeling addresses potential feedback loops between behavior and model-derived RPE. This model enables us to clarify the causal relationship between positive model-derived RPE and switching behavior. Specifically, it distinguishes between behavioral feedback and reward feedback, allowing us to determine whether the reward mechanism is truly driving language switching behavior, or if it is a consequence of the behavior itself. Through these complementary analyses, we can ensure that our findings reflect the true impact of RPEs on language control, independent of potential confounding factors.

A whole-brain activation analysis was conducted to identify the influence of RL on language control. To further explore the neural mechanisms by which RPE influences adaptive language control, we performed Connectome-based predictive modeling (CPM) from brain mappings to behaviors. We used CPM to explore how brain functional connectivity patterns can predict individual behavior during language control tasks, especially in relation to how RPE signals influence adaptive language control. The goals of CPM are: (1) to predict behavioral performance: By using brain network functional connectivity data, a predictive model is constructed to assess how changes in brain connectivity relate to changes in behavior during language control tasks; (2) to uncover the adaptive influence of RPE on language control: By integrating behavioral indices, such as value of sequence (VS) and value of RPE derived from RL models, the analysis aims to understand how RPE modulate brain network functional connectivity and enhance language control adaptability; and (3) to identify brain regions and networks associated with language control, feature selection and correlation analyses allow us to identify brain connections that are significantly related to behavioral indices (VS/RPE).

Reinforcement learning modeling

The RL model was built using data from 23 participants using the hBayesDM and RStan packages in R. After removing the warm-up trials and all non-response trials, each block was trimmed for the modeling analysis. Each participant completed 48 trials in each of the 4 blocks, resulting in 192 total trials. In our analysis, we used the 48 trials from each block for every participant.

The hBayesDM package offers state-of-the-art hierarchical Bayesian modeling in which both individual and group parameters are estimated simultaneously in a mutually constraining fashion³⁷. RStan is the R interface to the Stan C++ package, a high-performance statistical computation platform used for modeling^38–40. Our analysis was designed to dissect the learning strategy by specifying a model space of six competing models. These models formalize distinct hypotheses about the learning process. The core learning mechanism for all adaptive models was the Rescorla–Wagner update rule¹⁵.

The standard Rescorla–Wagner model of RL posits that the value of an action changes when new information indicates that the expected outcome of a decision is inconsistent with the actual outcome. Specifically, for each trial of a language sequence (switch or non-switch), the model estimates the predicted VS, i.e., V_(switch) or V_(non-switch), with the initial predicted value of the language sequence set to zero. For example, the predicted value of an individual’s reward for the switch at trial t is noted as V_t(switch). The reward prediction value V_t + 1 for the next trial is updated through a functional relationship between the reward prediction value V_t(switch) for the current trial, the learning rate α, and the reward prediction error RPE_t(switch), as shown in Eq. 1.

$$\mathrm{Value}\mathrm{update}:V_{t+1\left(\mathrm{switch}\right)}=V_{t\left(\mathrm{switch}\right)}+\alpha \times {\mathrm{RPE}}_{t\left(\mathrm{switch}\right)}$$ 1

The reward prediction error RPE_t(switch) is defined as a function of the difference between the actual reward R_t(switch) obtained by switching languages on trial t (positive reward prediction error is 1, neutral reward is 0, negative reward prediction error is −1) and the reward prediction value V_t(switch) for that trial, as shown in Eq. 2.

$$\mathrm{Reward}\mathrm{prediction}\mathrm{error}:{\mathrm{RPE}}_{t\left(\mathrm{switch}\right)}=R_{t\left(\mathrm{switch}\right)}-V_{t\left(\mathrm{switch}\right)}$$ 2

The degree to which the RPE updates the predicted value is determined by the learning rate α, which ranges between 0 and 1. The learning rate α controls the degree to which the current expectation is updated with new information. A lower learning rate indicates that the reward prediction error has less effect on updating the reward prediction value for the next trial^{15,19,36,41–43}. The inverse temperature parameter β controls participant noise, that is, the degree to which a participant decides to choose the most valuable option rather than to explore actions that may or may not be more valuable. The softmax function estimates the trial-by-trial probability of each action by the ratio of the expected values weighted by the inverse temperature parameter. A lower inverse temperature parameter β indicates greater randomness of behavior, and a higher β indicates greater consistency of choice and a stronger tendency to make decisions with high payoffs^{15,19,36,41–43}. W estimated the parameters α and β as shown in Eq. 3.

$$\mathrm{softmax}:P_t\left(\mathrm{switch}\right)=\frac{\exp \left[\beta V_t\left(\mathrm{switch}\right)\right]}{{\sum }_{a^{\prime }}\exp \left[\beta V_t\left(a^{\prime }\right)\right]}$$ 3

To formally adjudicate between different learning strategies, we employed a two-phase computational modeling approach. We first sought to validate that participants’ behavior was driven by a real learning process. We then used a competitive model comparison to identify the specific cognitive mechanism underlying participants’ choices. We elaborate on these two steps below.

To create a null-hypothesis baseline, we generated a simulated dataset from a random-choice agent, which served as a pure null model with no learning. In this model, the agent chose to switch languages or not on every trial with 50% probability, irrespective of previous reward outcomes. We simulated 23 agents, each performing a task with 4 blocks (50 trials each), mirroring the experimental structure. On each trial, the agent’s choice was random (50% probability), and it received rewards based on the same probabilistic schedule as in the human experiment. This produced a full dataset that matched the task and reward environment of the experimental data but relied on non-adaptive, random decision-making.

This simulated dataset served as a formal baseline for interpreting model parameters. We fitted the AV model to both the human experimental data and this non-learning simulation. By doing so, we established expected parameter ranges (e.g., learning rate, inverse temperature) for a non-learning agent. Comparing these to estimates from human participants tested whether human behavior reflected real learning.

Having confirmed learning in participants, our second phase identified their learning strategy. We fitted and compared six competing models with distinct hypotheses (see Fig. 9): an Action-Value (A_V) model assuming a generalizable, abstract switching policy; a Stimulus-Value (S_V) model assuming non-generalizable, stimulus-specific learning; a Pure Language-Value (L_V) model based on learning the intrinsic value of each language; an Action-Value with Control (A_Vc) model integrating policy learning with a static language preference; an Asymmetric Language-Value-Guided Policy (LVGP-Asym) model positing a state-dependent gating mechanism driven by the current language’s value; and a Symmetric Language-Value-Guided Policy (LVGP-Sym) model assuming a rational maximization based on the relative value difference between languages. Detailed model descriptions are in Supplementary materials Section 4 “Reinforcement learning computational models.”

Fig. 9. Schematic diagrams of the six candidate computational models.

All diagrams represent Bayesian Hierarchical models in which plates denote the data structure at the participant level (“Subject-s”) and the trial level (“Trial-t”). Solid-line nodes represent observed data, RPE is the reward prediction error received in each trial, and action is the participant’s observed action (e.g., switch or non-switch). Dashed-line nodes represent latent variables estimated by the model, including trial-specific values (V) and probabilities (P), as well as subject-specific parameters for learning rate (α), inverse temperature (β), bias (Bias_Languages), and cognitive control weight (w). Arrows indicate directional influence. The models are AV model: learns a single, abstract policy value for the switching action independent of stimuli. SV model: learns distinct action values conditional on the specific stimulus presented, lacking generalizability. LV model: learns the intrinsic value of choosing a specific language (Vt_L1) rather than an action policy. AVc model: extends the AV model by including a static, subject-specific bias parameter that directly influences choice probability to control for baseline preferences. LVGP-asymmetric model: a comprehensive model integrating language-value learning with a cognitive control mechanism. It dynamically learns the value of the current language (Vt_current) and maps this state evaluation to the switch probability via a control parameter (w), representing a gating mechanism without evaluating the alternative language. LVGP-symmetric model: assumes a rational maximization where the agent actively tracks and compares the learned values of both languages (Vt_L1, L2). The relative difference between the alternative and current language drives the decision to switch, weighted by the control parameter (w).

We assessed out-of-sample predictive accuracy using WAIC and LOO-CV (prioritizing WAIC⁴⁴, with consistent results (Supplementary Fig. S2A). Furthermore, we computed Bayesian stacking weights to quantify each hypothesis’s relative contribution to the data.

fMRI data preprocessing

Preprocessing of the fMRI data was conducted using the Data Processing and Analysis for Brain Imaging (DPABI) software⁴⁵. Initially, the collected EPI DICOM files were converted into NIFTI format. To stabilize the signal, images from the first ten time points of each instrument run were discarded. Second, slice timing correction was performed using the middle slice as a reference, and all scanned slices were realigned to the first scan to correct for head motion. Third, the origin was set at the anterior commissure for registration of each participant’s structural images with their functional images, which were then normalized to the Montreal Neurological Institute (MNI) template. Fourth, all voxels were resampled to 3 × 3 × 3 mm³. Finally, all functional images were smoothed using a 6 mm full-width at half maximum (FWHM) Gaussian kernel.

Whole-brain activation analysis

We employed statistical analyses and group analyses using SPM12 software for individual subjects. Through the specified 1st-level function, we established a general linear model (GLM) with two factors during the voluntary language switching task: language sequence (switch, non-switch) and reward condition (high reward, medium reward, low reward). The duration of the image presentation (2000 ms) served as the analysis window. Language sequence and reward condition were treated as regressors, along with head motion parameters from each participant’s scan. These vectors were convolved with the hemodynamic response function (HRF) for BOLD signal modeling and estimation, using a double-gamma HRF. In the second-level analysis, six contrast files were obtained for full-factorial analysis during voluntary language switching, placing the six NII files in corresponding condition slots. We conducted multiple comparisons by applying the Gaussian random field (GRF) correction, a family-wise error rate (FWE) correction method. We implemented correction with a voxel-level threshold of p < 0.001 and a cluster-corrected threshold of p < 0.05. We only considered clusters significant if they comprised more than 50 voxels.

Analyses on brain ROIs were conducted with the MarsBaR tool, which utilizes the peak coordinates of GRF-corrected brain regions. The centroid MNI coordinates were used as centers of spherical ROIs, and all radii were 6 mm. To examine interactions, we extracted the beta values of significant clusters by averaging all voxels within a cluster. These beta values were determined by the location of the activated region. Post-hoc comparisons were performed on the extracted beta values. We also used generalized linear mixed-effects models in R (R version 4.1.1), utilizing the lme4 package and lmerTest package (lme4 version 1.1-23) with the beta values treated as the dependent variable, language sequence, and reward condition as fixed effects, and subjects as random effects.

Representational similarity analysis

To decode the neural basis of language control associated with the parameters of RL models, and understand how the brain integrates language control at the trial level, we performed a whole-brain searchlight representational similarity analysis (RSA; see Fig. 10). This method identified brain regions where multivariate activity patterns reflected the similarity structure of computational variables (VS and RPE).

Fig. 10. Trial-wise searchlight RSA pipeline with temporal confound control.

For each participant, single-trial neural activity patterns were estimated using a GLM applied to preprocessed (normalized, unsmoothed) fMRI data. Each trial onset was modeled as a delta function convolved with canonical HRF. A whole-brain searchlight analysis (sphere radius = 6 mm) was performed on the resulting 4D beta image (61 × 73 × 61 × ~N trials). Within each searchlight, local multivariate patterns were extracted for every trial (voxel-wise beta amplitudes shown as spatial profiles for Trial 1 and Trial 2 in the example). Neural representational dissimilarity matrices (Neural RDMs) were constructed by computing the correlation distance (1 − Pearson r) between all pairs of trials (repeated for every trial pair; only the upper triangle was used for subsequent analyses). Two model RDMs were generated for each participant: VS RDMs (chosen-sequence value signal, blue) and RPE RDMs (prediction error, green). Model RDMs were defined as absolute differences d_{_{i,j}} = |θ_Trial·i − θ_Trial·j|, where θ represents the trial-wise parameter (VS or RPE). Temporal distance RDM (gray) was used as a confound matrix  and was defined as c_{_{i,j}} = |onset_{_i}  − onset_{_j}| (in seconds). Analysis A (primary, all trials): partial Spearman rank correlation between the neural RDM and each model RDM (VS or RPE), controlling for the temporal distance confound. This was implemented by (1) rank-transforming all three vectors, (2) regressing out the ranked temporal distance vector from both ranked neural and model vectors via OLS, and (3) computing Pearson correlation on the residuals (equivalent to partial Spearman correlation). Analysis B (complementary, block-wise): The same partial Spearman procedure was performed independently within each of the four learning blocks, followed by Fisher z-transformation and averaging across blocks. The resulting whole-brain maps (one per participant and method) were subsequently smoothed (6 mm FWHM), submitted to group-level one-sample t-tests, and corrected for multiple comparisons using cluster-extent thresholding.

First, we estimated trial-specific neural activation patterns via trial-wise beta estimation. For each participant, we built a GLM with every trial as a separate regressor, convolved with the canonical HRF. This produced a 4D series of beta maps (one 3D volume per trial). We used a spherical searchlight (radius = 6 mm) to traverse the entire brain. Within each sphere, we computed a neural representational dissimilarity matrix (Neural RDM) using correlation distance (1 − Pearson r) between the beta patterns of all trial pairs. Model RDMs were simultaneously constructed for each parameter (VS and RPE) as the absolute difference between parameter values for every trial pair (d_ij = |θ_i − θ_j|).

To account for hemodynamic overlap and temporal autocorrelation between adjacent trials, we used partial Spearman correlation. For each searchlight sphere, we correlated the model RDM with the neural RDM while partialing out a temporal distance RDM, which captured absolute time differences between trial onsets (|onset_i − onset_j|). This removed variance due to temporal proximity (e.g., hemodynamic aliasing or scanner drift), ensuring RSA effects stemmed from computational content rather than temporal structure.

Since RL involves evolving internal models, we examined whether neural representation fidelity changed over time. For RPE (which drives learning updates), we analyzed the linear trend of representational strength across the four learning blocks. Resulting correlation maps were Fisher z-transformed and smoothed (6 mm FWHM). Group-level inference used one-sample t-tests against zero, with multiple-comparison correction via cluster-extent thresholding (voxel-wise p < 0.001, cluster size k ≥ 10), a standard method for detecting spatially distributed representational effects^46,47.

Connectome-based predictive modeling

To investigate whether whole-brain functional connectivity patterns predicted individual differences in learning-modulated language control, we implemented a CPM approach using a LOOCV⁴⁸. The detailed analysis of the procedure is described in  Supplementary Materials Section 5, Connectome-based predictive modeling.

The model utilized functional connectivity matrices from the Shen 268-node atlas as predictors. The behavioral outcomes we aimed to predict were the VS, RPE, learning rate, inverse temperature parameter, all derived from our RL model. Within each fold of the LOOCV, feature selection was performed on the training data by identifying functional connections (edges) that were significantly correlated with the behavioral measure of interest (p < 0.001).

Separate linear models were then constructed based on the sum of edge strengths for positively and negatively correlated connections. These models were used to predict the behavioral score of the single left-out participant. The overall predictive performance of the model was assessed by calculating the Pearson correlation between the predicted and actual scores across all participants.

Finally, to determine the statistical significance of our model, we conducted a 1000-iteration permutation test. In each iteration, the behavioral data were randomly shuffled, and the entire CPM pipeline, including the cross-validated feature selection, was repeated. This process generated a null distribution of prediction-outcome correlations, against which the significance of our observed results was evaluated.

Statistics and reproducibility

Sample-size estimation, participant recruitment, and exclusion criteria are described in the Participants section. The final fMRI sample included 23 participants. The supplementary behavioral experiment recruited 20 participants, all of whom were included in the final analysis. The participant was treated as the biologically independent unit of analysis. Trials, blocks, runs, and repeated measurements within the same participant were treated as repeated observations rather than independent biological replicates. Statistical analyses, including behavioral mixed-effects models, reinforcement-learning modeling, whole-brain fMRI analyses, region-of-interest analyses, representational similarity analysis, and connectome-based predictive modeling, were conducted as described in the corresponding “Methods” sections. Exact sample sizes, statistical tests, thresholds, and correction procedures are reported in the Results, Figure legends, Methods, and Supplementary Information where applicable. Numerical source data underlying all main and Supplementary Figs., together with figure-generation scripts and related derived outputs, are available in the OSF repository described in the “Data availability and Code availability” sections.

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.

Author contributions

M.Z. conceived the study, designed the experiments, developed the methodology, collected the behavioral and fMRI data, performed the behavioral and neuroimaging analyses, interpreted the results, created the figures, wrote the original draft, and revised the manuscript. H.L. acquired funding, supervised the project, contributed to conceptualization, designed the experiments, developed the methodology, and interpretation of the results, wrote the original draft, and revised the manuscript. L.Y.L. provided methodological guidance on reinforcement learning modeling. J.W.S. contributed to the interpretation of the findings and revised the manuscript. Y.J.W. contributed to the interpretation of the findings and revised the manuscript.

Peer review

Peer review information

Communications Biology thanks the anonymous reviewers for their contribution to the peer review of this work. Primary handling editor: Jasmine Pan.

Data availability

Numerical source data underlying all main and Supplementary Figs., together with related derived analysis outputs, are available in the OSF repository⁴⁹. fMRI data and additional anonymized data supporting the findings are available from the corresponding author upon reasonable request, subject to participant consent, institutional ethics approval, and institutional data-sharing regulations.

Code availability

Figure-generation scripts and related analysis scripts used to generate the main and Supplementary Figs. are available in the OSF repository⁴⁹. Additional analysis code is available from the corresponding author upon reasonable request, subject to institutional and data-sharing regulations.

Competing interests

The authors declare no competing interests.

Supplementary information

The online version contains Supplementary material available at https://doi.org/10.1038/s42003-026-10448-w.