Synaptic-state-dependent plasticity routes learning signals in recurrent circuits

Abstract

Synapses differ in molecular and ultrastructural state, but the computational role of this heterogeneity remains unclear and computational models do not account for it. Local Hebbian-like plasticity is well supported experimentally, whereas learning complex behaviors most likely requires non-local credit-assignment signals. Recent cortical electron microscopy data suggest that strong and weak synapses occupy distinct structural states, with large synapses often containing a spine apparatus that can alter calcium dynamics and plasticity. Here, we assigned different computational roles to different morphological synaptic types: we asked how networks with state-dependent plasticity operate and whether the mechanisms and structural features they develop are consistent with biological observations. We built a plasticity-switching recurrent neural network (psRNN) in which synaptic state is modeled by recurrent-weight thresholds. Synapses in the low-weight state use a local Hebbian-like rule, whereas synapses in the high-weight state receive an idealized non-local credit-assignment signal implemented with backpropagation through time. psRNNs learned cognitive tasks with temporal and working-memory demands in fewer trials than fully backpropagation-trained RNNs, despite routing BP-based updates to fewer recurrent weights. The advantage arose from three interacting mechanisms: 1) Hebbian-like updates exposed the credit-assignment component to several nearby parameter configurations, 2) the network dynamically built a task-relevant recurrent initialization, and 3) synapses switched into the credit-assignment subset before Hebbian growth destabilized learning. Structurally, trained psRNNs produced more anti-symmetric weights, lower effective rank, and stronger feedforward structure than control RNNs. Our results shed light on the computational role of heterogeneous synapses, which may regulate distinct types of learning signals, and we generate falsifiable predictions for connectomic analyses of recurrent cortical circuits.

2. Introduction

Synapses are not interchangeable learning sites. Their size, molecular composition, and organelle content vary across cortical circuits, and these differences can change the plasticity rules available to individual synapses^1-4. This heterogeneity creates an opportunity for computation: instead of applying one learning rule uniformly across a circuit, the brain could route different learning signals to synapses in different states.

In the past decades, appreciation for the various computational mechanisms of the brain has been growing considerably. For example, large bodies of work have focused on the heterogeneity of cell types and their distinct functional roles; computational models followed this trend, providing key insights into how neural circuits compute distributively ^5-12. In recent years, new observations more clearly pointed out distinct synaptic states¹³ likely undergoing different plasticity mechanisms³. However, computational models have yet to explore the functional, computational, and structural consequences of synaptic heterogeneity.

Recent cortical connectomic data suggest a possible substrate for selective routing of learning signals. In the MICrONS phase 1 connectome, cortical synapses between pyramidal neurons are well described by two structural states: small synapses that typically lack a spine apparatus and larger synapses that often contain one (Fig. 1a)¹³. The spine apparatus, a smooth endoplasmic reticulum-derived organelle, acts as an internal calcium reservoir and can influence synaptic plasticity dynamics^1-3. Plasticity pathways that depend on internal calcium stores, including IP3-dependent Ca²⁺ release, can therefore differ between synapses with and without endoplasmic reticulum⁴. These observations raise a concrete computational hypothesis: synaptic structural state may regulate which synapses use local, relatively simpler, plasticity and which synapses receive non-local, relatively complex, credit assignment signals.

Figure 1: Synaptic states motivate a plasticity-switching recurrent network.

a Transmission electron microscopy images of example synapses with and without a spine apparatus (top). Cartoon illustration of the two synaptic states (bottom). The presynaptic boutons and spine heads are contoured in yellow and orange, respectively. The postsynaptic density is marked by green arrow head. The spine apparatus is marked by pink arrow head. The cartoon includes a schematic of receptors, with a highlighted Ca²⁺ channel. The spine apparatus (pink) provides an additional, internal Ca²⁺ source. b Control RNN architecture (vanilla). All parameters (in pink) are trained using backpropagation through time (BP). c Recurrent weights distribution at initialization. Recurrent weights are sampled from a uniform distribution with a scale that sets the spectral radius at initialization to ~ 1. d Spectrum of the recurrent weight matrix at initialization. Dashed line indicates the unit circle. e Switching RNN architecture. Parameters in pink receive BP updates, parameters in blue are updated using a local Hebbian-like rule, and the switching mask in yellow is updated based on synaptic strength throughout training. f Recurrent weights at initialization same as vanilla model (c). Switching thresholds are indicated by the dashed lines (green: hebb → BP, black: BP → hebb). g Spectrum of the recurrent weight matrix at initialization same as vanilla model (d). h Example inputs (green, blue, orange, and gray lines) for two selected tasks, “DelayGo” and “MultiDelayDM”. Shaded areas represent the time period where the loss function is computed. i Recurrent activity during the example trials for both model types for each task. j Target outputs (black lines) and model responses (green lines) for the example trials for both model types for each task.

Most experimentally observed synaptic plasticity rules are local, relying on the activity of pre- and post-synaptic neurons^14-19. Local rules are biologically plausible and computationally efficient, and they can support memory and associative learning^14,15,20-25. Complex behavior, however, also requires credit assignment: error or value information must influence the synapses that contributed to a behavioral outcome^26-31. Proposed mechanisms for credit assignment in the brain use non-local information to update synapses while preserving biological plausibility, including feedback alignment³², dendritic predictive coding^33,34, and three-factor learning rules^29,35,36. Such alternatives to the strong, but biologically-implausible, credit assignment that backpropagation offers often fail to scale to more complex tasks³⁷. Here, we propose that only a subset of synapses need to access non-local information for successful learning, thus removing a substantial part of the credit assignment computational burden without hurting task learning.

Established computational models addressing synaptic states do not capture the computational role of synaptic heterogeneity. Instead, they typically develop neurally-inspired algorithms that improve learning or memory. For example, such previous models have limited dynamics and learning capacity, generally lacking any type of credit assignment and focusing on distinct synaptic update timescales only³⁸, improved learning mechanisms by loading individual synapses with several learning rules ^39,40, or hand-designed specific fixed modules of the network to solve different aspects of the task at hand⁴¹. Our work is built around connectomic observations that individual synapses may have distinct plasticity states that depend on prior experience of the organism, thus being able to switch between states as needed to maintain a balance between flexible learning and efficient energy consumption.

Our assumptions are also motivated by theory. Individual gradient-based synaptic learning rules typically produce unimodal weight distributions^42,43, whereas cortical excitatory synapses can show bimodal structure ¹³. A bimodal synaptic distribution could therefore reflect more than one learning process operating within the same recurrent circuit. The unresolved question is whether such heterogeneity merely reflects biological detail or whether it changes what and how a circuit can learn, how a circuit computes, and how it affects the circuit’s structure.

We tested this question with a recurrent neural network in which synapses switch between learning rules based on their strength. Weight magnitude is an operational proxy for synaptic state in this model, not a one-to-one mapping onto spine apparatus or any single organelle, and the specific switching thresholds are hyperparameters rather than fitted biological boundaries. Synapses in the low-weight state followed a local Hebbian-like rule, whereas synapses in the high-weight state received an idealized non-local credit-assignment signal implemented with backpropagation through time. This model treats backpropagation through time as a computational proxy for strong credit assignment, not as a proposed biological mechanism. It asks whether state-dependent synaptic switching can reduce the number of recurrent weights that receive BP-based updates while preserving the computational benefits of credit assignment. We found that plasticity-switching RNNs learned cognitive tasks faster than fully backpropagation-trained RNNs, and that switching produced recurrent structural signatures that can be tested in connectomic data. These results provide a set of falsifiable hypotheses for neural circuit structure and suggest a computational role for synaptic structural states: they may control how local plasticity and credit assignment are distributed across a recurrent circuit. Our work, directly motivated by the recently quantified synaptic heterogeneity, provides new insights into how the brain learns using variable components and how these components alter the circuit structure.

3. Methods

3.1. Network Architecture

We used recurrent neural networks (RNNs) because they provide a simple model of temporal circuit dynamics. The models combine two classes of synaptic updates: local Hebbian-like plasticity and an idealized non-local credit-assignment signal implemented with backpropagation through time (BPTT; for brevity, BP; Fig. 1b,e). We simulated all networks in discrete time (Supplementary Methods). All parameters in the control models (vanilla) were trained using BP. Recurrent weights were initialized from a uniform distribution scaled to set the spectral radius at initialization to 1 (Fig. 1c,d), using a scale of $\sqrt{3∕N}$ where $N$ is the number of hidden units. This initialization is known to provide a good starting point for artificial networks ^44-48. Unless otherwise specified, vanilla RNNs had N = 256 hidden units and used the tanh transfer function for the hidden layer and a linear transfer function for the output layer.

Our plasticity-switching RNN (psRNN) shares the same computation and connectivity graph as the vanilla model, but the recurrent weights differ in how they learn (Fig. 1e). We divided recurrent weights into two subsets based on their strength and applied a different learning rule to each subset. A binary dynamic mask M switches synaptic learning states between local Hebbian-like plasticity and BP-based credit assignment as synapse weights cross state thresholds.

In line with simple synapses generally having smaller spine heads than spine-apparatus synapses¹³, synapses with small absolute weights use Hebbian-like plasticity, whereas synapses with large absolute weights receive BP updates. This threshold rule is a modeling abstraction. It captures the hypothesis that synaptic state can gate access to different learning signals, but it does not specify the molecular pathway by which a biological synapse would switch state. The threshold values are model hyperparameters and were not fitted to measured synapse-size distributions. Recurrent weights were initialized identically to the vanilla model (Fig. 1f), using the same random seeds for matched comparisons, with switching thresholds set at ${\theta }_{BP\to Hebb}$ = 0.5 and ${\theta }_{Hebb\to BP}$ = 0.8 in units of the network scale $\sqrt{3∕N}$ (Fig. 1f, dashed lines). These thresholds made psRNNs start with about 50% of recurrent weights in each subset. The two thresholds differ to account for overlap between synaptic states and to prevent unstable rapid switching. All reported results used the same thresholds unless otherwise noted, although we explored a range of switching thresholds (Supplementary Methods). The spectrum of the recurrent weight matrix at initialization was identical for both model types (Fig. 1g). Unless otherwise specified, psRNNs had N = 256 hidden units and used the same transfer functions as the vanilla model.

3.2. Tasks

We evaluated both model types on cognitive tasks from the set developed by Yang et al.⁴⁹ for studying task representations in recurrent neural networks. These tasks are widely used in computational neuroscience^50-54 and require temporal integration, working memory, and context-dependent responses. They therefore provide a useful setting for asking how local Hebbian-like plasticity interacts with non-local credit assignment. We focus on two representative tasks, delayed response (DelayGo)⁵⁵ and multi-sensory delayed decision making (MultiDelayDM)⁵⁶ (Fig. 1h,j; see also Fig. S1a,d for alternative input-target visualizations consistent with prior work). Results for additional non-human-primate-inspired cognitive tasks are reported in Tab. 1, Tab. 2, Tab. S1, and Tab. S2.

Table 1: Best performance across tasks and BP learning rates.

Mean ± standard deviation of averaged validation loss across 5 random initializations. For robustness, validation loss was averaged across last 10 epochs of training across all validation trials. Vanilla models results are shown in gray and switching models results are shown in black. For each task, the best performance across all BP learning rates is shown in bold. All bold values are from switching models. Green stars link to the corresponding experiments in Fig. 2, highlighted by the pink (vanilla) and blue (switching) boxes. Tasks are ordered by increasing difficulty, i.e. best performance across our trained models, from left to right.

Table 2: Learning speed across tasks and BP learning rates.

Mean ± standard deviation of learning speed across 5 random initializations. Learning speed is defined as the area under the curve of validation loss throughout training. All other details are the same as in Tab. 1. Tasks maintain same order as in Tab. 1.

The DelayGo task (moderate difficulty) requires the network to maintain a spatial location in working memory during a delay period and produce a response at the remembered location after a go cue⁵⁵. The MultiDelayDM task (high difficulty) extends this challenge by requiring comparison between multiple stimuli presented sequentially with varying delay periods, placing additional demands on working memory (extension of the task from Raposo et al.⁵⁶). Both tasks feature distinct temporal phases (stimulus presentation, delay, and response) that provide natural opportunities for Hebbian plasticity to operate on correlated activity patterns while simultaneously requiring precise credit assignment for successful performance.

3.3. Training

All parameters shown in pink in Fig. 1b and Fig. 1e receive BP updates. Parameters shown in blue in Fig. 1e are updated using a local Hebbian-like rule with decay δ and parameter-specific BP-trainable learning rate Λ. The update rule is given by ${\mathbf{W}}_{Hebb}\leftarrow \delta \cdot {\mathbf{W}}_{Hebb}+\Lambda \odot \Delta {\mathbf{W}}_{Hebb}$, where ⊙ denotes element-wise multiplication. The Hebbian update $\Delta {\mathbf{W}}_{Hebb}$ is computed as $\Delta {\mathbf{W}}_{Hebbi\ne j}$ = $\frac{1}{P(T-1)}{\sum }_{p=1}^P{\sum }_{t=1}^{T-1}{\mathbf{h}}_{p,i,t+1}\cdot {\mathbf{h}}_{p,j,t}$, where $\mathbf{h}\in {\mathbb{R}}^{P\times N\times T}$ (number of trials included in Hebbian plasticity $P$ , hidden units $N$, time steps $T$) is the hidden activity over the last $P$ trials. To avoid unstable self-amplification, we set the diagonal of the Hebbian update to zero, $\Delta {\mathbf{W}}_{Hebbi,i}$ = 0. To allow BP to account for Hebbian updates, we set the Hebbian period $P$ smaller than the batch size $B$ and took multiple Hebbian steps during one BP batch. Unless otherwise specified, psRNNs took 2 Hebbian steps per BP batch, with $P$ = 128 = $B$/2 trials per period.

We explored several regimes for the Hebbian learning rate. In the default configuration, the Hebbian learning rate, Λ, was BP-trained independently for each Hebbian synapse. This choice gives the vanilla and switching models the same number of BP-trained parameters, making the comparison more conservative. BP-training individual synaptic learning rates is a computational simplification. In biological circuits, heterogeneous plasticity parameters could arise from evolutionary, developmental, or active selective control processes rather than gradient-based optimization^57-61. We therefore treated psRNN variants with fixed Hebbian learning rates and many fewer trained parameters as an important control on the role of meta-optimized plasticity; these variants also perform well (Fig. S2b,c). We tested several choices for the decay parameter δ, including heterogeneous and BP-trained variants, but these did not substantially affect performance. We therefore set δ = 0.9999 throughout all experiments. Λ was initialized at 0 for all Hebbian synapses to remove any initial Hebbian bias.

We verified that both RNNs and psRNNs produce similar dynamics after training (Fig. 1i). Internal dynamics are projected to the output space via the output weights and biases, which are BP-trained parameters in both model types. Both model types eventually solve the task and produce behavior that follows the target (Fig. 1j). For alternative visualizations of the hidden activity and responses, consistent with prior work using these tasks, see Fig. S1b,c.

4. Results

4.1. State-dependent plasticity improves learning despite reduced synaptic credit assignment

We first asked whether a recurrent network can learn well when many recurrent weights do not receive BP-based updates. We trained vanilla RNNs and psRNNs on DelayGo and MultiDelayDM using the best BP learning rate for each model type and task, selected from a sweep across a range of values (Fig. S2a). By design, many recurrent weights in psRNNs use only the Hebbian-like rule until, if ever, they cross into the BP subset. Nevertheless, psRNNs reached lower training and validation loss than vanilla RNNs across tasks (Fig. 2a,b; Tab. 1). They also learned faster, as measured by the area under the validation loss curve (Fig. 2a,b; Tab. 2). These results generally held across network sizes (Tab. S1, Tab. S2) and transfer functions (Tab. S3). Behavioral performance, measured as validation accuracy, was also higher for psRNNs even though accuracy was not directly optimized (Fig. 2c; see Supplementary Methods). These comparisons represent model-level replication across random initializations and hyperparameter sweeps, and we report uncertainty across seeds in figures and tables.

Figure 2: Switching plasticity improves learning while reducing the recurrent synapses receiving credit assignment.

a Training loss as a function of training epochs for both model types and tasks. Shaded areas represent standard error of the mean (s.e.m.) across 5 random initializations. Green stars link to the corresponding experiments in Tab. 1 and Tab. 2. BP learning rate for shown experiments was chosen based on a parameter sweep across different values (Fig. S2a) to optimize the performance of each model type individually, while maintaining stability throughout training. b Validation loss, c validation accuracy, d percent BP recurrent weights, and e spectral radius or recurrent weights throughout training, otherwise same as a. Accuracy computation is described in the Supplementary Methods section. f Example spectrum of the recurrent weight matrix after training for both model types and tasks. Dashed lines indicate the unit circle. g Validation loss as a function of the spectral radius for all models for each task throughout training. Individual points represent a single random initialization at a specific training epoch. h Example trained recurrent weights distribution for the vanilla (left) and switching (right) model for each task. Long tails of distributions are truncated at 2 in units of network scale for visualization purposes. Switching thresholds (Fig 1f) are shown on the switching model weights distribution.

The performance gain was not caused by giving psRNNs more access to credit assignment. psRNNs used fewer BP-updated recurrent weights than vanilla RNNs throughout training (Fig. 2d), especially during the early learning period when the performance gap emerged. In the default model, BP also optimizes the Hebbian learning-rate matrix Λ, so this result should be interpreted as reduced access to credit assignment for recurrent weights rather than complete removal of gradient information from every Hebbian synapse. When BP optimized the Hebbian learning rates, psRNNs gradually moved more synapses into the BP subset. However, synapses switched in both directions rather than simply accumulating in the BP subset (Fig. S2d). The model therefore maintained an ongoing interaction between local Hebbian-like plasticity and non-local credit assignment.

psRNNs also changed recurrent dynamics during learning. The spectral radius of the recurrent weight matrix increased faster in psRNNs than in vanilla RNNs (Fig. 2e), potentially allowing longer timescales through larger real eigenvalues and faster input responses through larger complex eigenvalues. This difference is visible in the trained spectra (Fig. 2f), where psRNNs show more eigenvalues with large real and imaginary components. Across models and training epochs, larger spectral radius was associated with lower validation loss (Fig. 2g). Switching also produced a multimodal recurrent weight distribution after training (Fig. 2h), qualitatively matching the bimodal synaptic weight distributions observed in cortical connectomic data¹³ and the theoretical expectation that a single learning rule tends to produce unimodal synaptic weights⁴².

Several simpler psRNN variants also performed well, reaching performance comparable to or slightly better than vanilla RNNs (Fig. S2b and Supplementary Methods). This control is important because the default model uses a BP-trained Hebbian learning-rate matrix. The best-performing variant used heterogeneous BP-trained Hebbian learning rates (Λ is a BP-trained matrix), but simpler variants fixed the Hebbian learning rates to heterogeneous (Λ is a fixed matrix) or homogeneous (Λ is a fixed scalar) values. Thus, BP-based optimization of Λ improves the model but is not necessary for state-dependent switching to retain good performance. We also removed both Hebbian plasticity and switching to create a semi-reservoir baseline in which the smallest ~ 50% of recurrent synapses at initialization were frozen and the rest received BP updates. This baseline connects to classical reservoir computing^62-64 and performed close to vanilla RNNs, showing that fully training every recurrent weight is not necessary for these tasks. Nevertheless, the psRNN variants surpassed vanilla and semi-reservoir baselines in performance (Fig. S2b), credit-assignment efficiency (Fig. S2c), or both.

Variants with active Hebbian-like learning showed that the non-BP subset contributes task-relevant updates. Depending on how Λ was constrained, Hebbian-like plasticity either improved performance while moving more synapses into the BP subset (fixed or optimized Λ matrix) or preserved performance while shrinking the BP subset (fixed Λ scalar). When BP could not optimize Hebbian learning rates, the percentage of BP recurrent weights saturated at lower values (Fig. S2c, compare orange and green curves to blue curve), with strong performance even when only about 35% of recurrent weights belonged to the BP subset.

4.2. Several plasticity interactions are responsible for improved learning

The performance advantage of psRNNs arose from the interaction between local plasticity, credit assignment, and switching. First, Hebbian updates occurred multiple times within each BP batch, allowing BP to evaluate multiple nearby parameter configurations per gradient step (Fig. 3a). Second, Hebbian-like plasticity reshaped recurrent connectivity from the activity generated by each task, creating a dynamic and task-specific initialization for later BP updates (Fig. 3b). Third, switching prevented Hebbian-like updates from pushing synapses into unfavorable regions of parameter space and moved those synapses into the credit-assignment subset while they remained useful (Fig. 3c,d and S3).

Figure 3: Local plasticity improves learning only when coordinated with credit assignment and switching.

a Validation loss throughout training for switching models for each task as a function of the number of Hebbian steps during one BP batch (solid lines, color-coded). Dashed lines represent controls where the same number of weights update steps are taken during one BP batch, but the Hebbian updates are shuffled onto the Hebbian synapses. Shaded areas represent s.e.m. across 10 random initializations. b Validation loss throughout training for vanilla models (completely BP-trained) for each task, initialized with the same weights from pre-trained switching models, color-coded by the number of pre-trained epochs of the switching model used for initialization (solid colored lines). Gray lines represent the original vanilla (solid) and switching (dashed) models for reference. Plotted validation loss for the vanilla models starts at the corresponding number of pre-trained epochs. Shaded areas represent s.e.m. across 5 random initializations. Insets show the same data in log-log plot to better visualize the filled gap between the original models. c Similar to b, but initialization is done based on Hebb-only pre-trained models. Since no credit assignment is used during pre-training, plotted validation loss for the vanilla models starts at epoch 0 for all numbers of Hebb-only pre-trained epochs. d Best performance of the vanilla models in c as a function of the spectral radius of the pre-trained Hebb-only models used for initialization. Best performance computed as in Tab. 1. Each point represents a single random initialization.

To test the contribution of multiple parameter-space samples, we varied the number of Hebbian updates taken within each BP batch. Throughout the main results, psRNNs used 2 Hebbian updates per BP batch, the minimum needed for BP to account for the Hebbian updates without substantially increasing training cost. Increasing the number of Hebbian updates slightly improved performance up to a task-dependent point, after which excessive updates started to destabilize training (Fig. 3a). This parameter-space sampling benefit was more prominent in our control experiments: when we shuffled Hebbian updates across Hebbian synapses, removing the correct temporal correlations, performance still improved with the number of updates (Fig. 3a, dashed lines). The shuffled control nevertheless performed substantially worse than the original model, showing that the task-relevant structure of the Hebbian updates is crucial.

The Hebbian-like component also generated a dynamic, task-relevant initialization for the BP component. To isolate this effect, we restarted vanilla RNNs from psRNN checkpoints collected throughout training while matching the initial random seed (Fig. 3b). Vanilla RNNs initialized from psRNN checkpoints closed the performance gap between randomly initialized vanilla RNNs and psRNNs, and after enough pre-training they slightly surpassed psRNNs. The required amount of pre-training varied by task, but even 1 epoch of psRNN pre-training significantly improved vanilla RNN performance (Fig. 3b). This result suggests that Hebbian-like updates are most useful early in learning, when they move the recurrent network into a region of weight space that BP can then exploit more efficiently.

Hebbian-like learning alone did not explain the benefit. To test the role of the dynamic switching interaction on improved learning in our models, we trained “Hebb-only” psRNNs by setting the switching thresholds to ∞, preventing recurrent synapses from receiving BP updates. Vanilla RNNs restarted from these Hebb-only checkpoints generally performed worse than networks started from random initialization (Fig. 3c). DelayGo showed slight improvement with minimal Hebb-only pre-training in some cases, but this effect did not generalize across tasks and was highly sensitive to pre-training duration. These results show that the benefit of psRNNs does not come from local plasticity in isolation. It depends on switching, which allows local plasticity to shape useful weights while preventing unconstrained Hebbian growth from dominating the recurrent matrix.

The spectral radius provides one view of why switching is needed. Proper initialization scaling is known to be critical for training recurrent and feedforward networks ^44-48. In vanilla RNNs restarted from Hebb-only checkpoints, the starting spectral radius strongly predicted the final performance of the restarted network (Fig. 3d). Individual Hebb-only models often showed abrupt increases in spectral radius during training (Fig. S3a), and these increases coincided with poor learning after restart (Fig. S3b). Thus, Hebbian-like learning can create useful task-dependent structure, but switching is needed to keep that structure within a range where credit assignment remains effective.

4.3. Switching models predict distinctive recurrent circuit structure

If synaptic state routes plasticity, it should change not only learning speed but also the structure of the trained recurrent circuit. We therefore compared the recurrent weight matrices of psRNNs and vanilla RNNs after and during training. We present results for both tasks with N = 256 hidden units, and the findings held across all network sizes tested (Fig. S4). These structural differences provide a set of principled and falsifiable hypotheses that could be tested in connectomic graphs, based on our switching-plasticity premise.

Using a metric based on the symmetric and skew-symmetric components of the recurrent weight matrix⁶⁵ (see Supplementary Methods), we found that psRNNs developed more anti-symmetric recurrent weights than vanilla RNNs (Fig. 4a). The departure from 0 was small, but it was consistent across tasks, network sizes, and random seeds (Figs. 4a & S4a). This anti-symmetric shift is consistent with the faster growth of complex eigenvalues in psRNNs and with prior work linking anti-symmetric connectivity to learning and stable memories under some homeostatic mechanisms^66,67. The anti-symmetry emerged from the interaction between learning rules. Although the Hebbian update is nearly symmetric, both synapse subsets became more anti-symmetric during training, and the BP subset became more anti-symmetric than the Hebbian subset (Fig. 4b). This result suggests that the Hebbian subset changes how credit assignment shapes the synapses that receive BP updates.

Figure 4: Switching plasticity produces structural signatures.

a Symmetry metric of the recurrent weights matrix, b symmetry metric of the Hebb and BP subsets of the recurrent weights matrix for switching models, c normalized rank of the recurrent weights matrix, and d amount of feed forward structure in the recurrent weights matrix, all plotted as a function of training epochs for both model types and tasks. Symmetry metric, normalized rank, and feed forwardness computations are described in the Supplementary Methods section. Shaded areas represent s.e.m. across 5 random initializations.

psRNNs also developed lower effective rank. Low-rank structure is important in both neuroscience and machine learning because low-dimensional tasks are often solved by large neural populations^68-79. Prior work has typically studied low-rank networks by imposing explicit architectural constraints^{69,71-75,77,78}. We instead measured whether lower-rank structure emerges from plasticity switching. Using the participation ratio of the recurrent singular values (see Supplementary Methods), we found that psRNNs developed lower normalized rank than vanilla RNNs (Fig. 4c). The trained matrices were not low-rank in the strict sense, but switching moved the network toward lower effective dimensionality without imposing a low-rank constraint.

Finally, psRNNs developed stronger feedforward structure within the recurrent matrix. Recurrent circuits can rely on both feedback loops and feedforward pathways, and the balance between these components shapes circuit computation^80-90. We quantified feedforward structure using Henrici’s departure from normality ⁹¹, normalized by network size (see Supplementary Methods). More non-normal recurrent matrices contain stronger feedforward components in their Schur decomposition. psRNNs developed larger normalized departure from normality than vanilla RNNs (Fig. 4d), suggesting that these circuits may rely more on feedforward pathways within their recurrent structure.

These predictions can be tested in connectomic graphs by asking whether synapse classes associated with different structural states also differ in symmetry, effective rank, or feedforward organization. A direct test would separate recurrent excitatory connections by spine-apparatus annotation or by synaptic size proxy in lack of well-annotated synapses, compute structural metrics in the circuit as a whole and within each group, and compare the observed values against degree-, weight-, and spatially-matched null graphs. Our model predicts that neural circuits involving multiple state-defined synaptic groups should be different from control circuits in their main computational pathway type and effective dimensionality. Conversely, the structural hypothesis would be weakened if well-annotated connectomic datasets showed no state-dependent differences after matching for weight, degree, spatial proximity, and cell-type composition.

5. Discussion

We introduced psRNNs to test whether synaptic state can regulate how local plasticity and credit assignment are distributed across a recurrent circuit. In these models, weak synapses follow a Hebbian-like rule, whereas strong synapses receive an idealized non-local credit-assignment signal implemented with BP. This simple state-dependent rule improved learning across cognitive tasks even though fewer recurrent weights received BP-based updates. The benefit depended on the interaction between local plasticity, credit assignment, and switching: Hebbian-like updates created task-relevant recurrent structure, BP exploited that structure, and switching prevented unconstrained Hebbian growth from destabilizing the network.

This framework offers a computational interpretation of synaptic structural heterogeneity. Experimental studies often observe local plasticity rules, but these rules may represent only the first-order terms of richer synaptic update rules that include non-local components ^38,92,93. We considered a complementary possibility: different synapses may use different learning rules depending on their state. Spine-apparatus synapses are a natural motivation for this idea because they differ from smaller synapses in ultrastructure and calcium handling ^1-4. Our results suggest that this kind of state dependence could do more than reflect synaptic history. It could help a recurrent circuit decide where costly or specialized credit-assignment signals are most useful.

The model is intentionally idealized. BP provides a strong credit-assignment signal, but it is not a biologically realistic mechanism in its standard form. We therefore interpret BP here as an upper-bound proxy for non-local information that could be carried by more plausible mechanisms, such as three-factor learning rules, dendritic credit-assignment signals, or eligibility-trace-based approximations^29,33-35. The Hebbian-like updates also operate on longer timescales than many biological plasticity processes and omit short-term dynamics typically used in other computational models^39,40,65. The switching thresholds reduce complex molecular and structural synaptic states to a scalar weight rule and were chosen as computational hyperparameters, not inferred from biological synapse classes. Finally, the default psRNN optimizes the Hebbian learning rate at each synapse using BP, which is a computational convenience rather than a proposed biological process. Simpler variants with fixed Hebbian learning rates still performed well (Fig. S2b,c), suggesting that meta-learned plasticity improves learning speed and final performance but is not required for the switching principle to work. The current experiments therefore support the principle that state-dependent rule assignment can be useful, but do not identify the biological mechanism that implements credit assignment.

The strongest experimental implication of psRNNs is structural. Switching plasticity altered recurrent symmetry, effective rank, and feedforward structure, producing signatures that can be measured in connectomic graphs. Symmetry and anti-symmetry are related to attractor structure, memory, and stability^67,94,95; effective rank reflects how many dimensions of the population are used for computation^68,69,71; and feedforward structure captures directed pathways embedded within recurrent connectivity^{80,82,83,86-88}. These measurements can be applied to dense model matrices and sparse connectomic graphs. The effects in our models are modest, especially for the symmetry metric, so detecting them in biological data will require sufficiently large and well-annotated synapse classes. Nevertheless, the predictions are falsifiable: synapses in different structural states should differ not only in size or organelle content, but also in their contribution to recurrent circuit organization.

psRNNs also connect to machine-learning ideas, but the analogy is secondary to the circuit hypothesis. The switching mechanism resembles an online form of sparse training or network pruning, because it dynamically identifies which recurrent weights receive BP-based updates. It also relates to reservoir computing, where a fixed recurrent layer is paired with a trained readout^62-64. In psRNNs, however, the recurrent layer is neither fully fixed nor fully trained by BP. Local plasticity continuously changes part of the recurrent network, while switching determines which synapses join the credit-assignment subset. This hybrid organization may explain why psRNNs can outperform fully BP-trained RNNs during learning rather than merely match them with fewer trainable parameters.

Several extensions would test the biological scope of the switching principle. The most direct computational test is to replace BP with biologically plausible approximations, such as three-factor learning rules, and ask whether the benefit persists when credit assignment is noisy, delayed, or incomplete^29,35. That experiment is beyond the present study, so the current results establish the utility of selective access to a strong teaching signal rather than the sufficiency of any particular biological approximation. Adding Dale’s law and cell-type identity, for example with exponentiated gradients⁴³, would make the recurrent architecture closer to cortical circuits. Replacing fixed switching thresholds with state variables tied to calcium dynamics, synaptic history, or spine-apparatus state would make the switching mechanism more biological. Testing motor control, navigation, continual learning, or multi-task settings would also show whether switching helps beyond the cognitive tasks studied here.

Our results support a narrower claim: in an idealized recurrent model, local plasticity and a strong teaching signal need not be assigned uniformly across a network. When coordinated by synaptic-state switching, simple local plasticity can improve learning while reducing the number of recurrent weights that receive non-local updates. This proof-of-principle model, broadly focused on the computational role of synaptic heterogeneity rather than algorithms specifically designed for certain tasks, motivates the biological hypothesis that synaptic structural states may route learning signals and yields connectomic predictions for testing whether different synapse classes play different roles in recurrent circuit organization.