Discrete synaptic states and context-modulated readouts support continual learning

Abstract

Biological neural systems learn new behaviors while retaining earlier ones, whereas sequentially trained artificial networks often overwrite parameters and forget previously learned behaviors. We introduce the Context-modulated Synaptic-states Recurrent Neural Network (CoSyn-RNN), a new continual-learning model inspired by heterogeneous synaptic states and neuro-modulatory control. CoSyn-RNN combines sparse recurrent allocation with a discrete protection rule: after each task, recurrent neurons whose synaptic weights cross a fixed threshold form a task-specific neuron group, and their associated parameters enter a protected state during later learning. A neuron-specific gain and a task-cued mask over a fixed base readout control how recurrent activity contributes to behavior. Across various task sequences consisting of many cognitive tasks, training and validation losses and accuracies were maintained throughout learning. The protection rule produced modular, task-ordered recurrent structure compatible with forward reuse of earlier computations. Pairwise recruitment measurements further defined minimum- and maximum-cost curricula that produced markedly different final recurrent structures and different remaining capacity for future learning. Thus, CoSyn-RNN provides a biologically inspired model in which protected synaptic states, sparse allocation, and context-dependent modulation support continual learning.

1. Introduction

Animals acquire new skills throughout life while retaining and reusing prior knowledge. This capacity is central to adaptive behavior, but its implementation in biological circuits remains unresolved. Several mechanisms, at different scales, could contribute. Synaptic tagging can separate transient from longer-lasting changes in synaptic efficacy¹; neural activity associated with recent experience is re-expressed during subsequent sleep²; and contextual signals can restrict which pathways are active or plastic³. These processes are complementary rather than mutually exclusive: consolidation can protect selected changes, replay can revisit earlier activity, and gating can limit interference during new learning.

Synapses also differ in their subcellular organization and plasticity. Endoplasmic reticulum (ER) is distributed non-uniformly across dendritic spines and changes local calcium signaling and plasticity⁴. ER visits active spines and can limit runaway potentiation, while larger spines can contain a spine apparatus formed from smooth ER⁵. The distinct calcium-dependent plasticity of ER-containing spines and the recruitment of ER to highly active spines make spine-apparatus synapses plausible candidates for a relatively stable or protected synaptic state^4,5. This interpretation is a hypothesis rather than evidence that the spine apparatus implements discrete consolidation, but it motivates asking whether heterogeneous plastic and consolidated states could help a circuit balance stability and flexibility. Additionally, at the system level, neuromodulation changes neuronal response properties and population dynamics⁶. In a computational model, a context-dependent signal can therefore serve as a functional analogue of a broadcast control signal that selects how shared activity is expressed.

Artificial neural networks expose the same stability-plasticity tension in a controlled setting. Sequential gradient-based training can overwrite parameters needed for earlier tasks, producing catastrophic forgetting. Existing approaches stabilize important parameters or previously learned functions^7–10, allocate or prune task-specific capacity^11–13, or constrain new gradients to reduce interference with previously learned solutions¹⁴. Recent recurrent-network approaches instead emphasize reusable computations, including shared dynamical motifs¹⁵ and explicit separation of contextual inference from low-rank computation¹⁶. Context gating and neuromodulatory models further show how a fixed recurrent substrate can express different computations across tasks^3,17,18.

Here, we introduce the Context-modulated Synaptic-states Recurrent Neural Network (CoSyn-RNN) as a framework for investigating how recurrent parameters can be allocated and protected during sequential learning. Sparse regularization and a neuron-specific gain produce sparsity-regularized, threshold-assigned neuron groups. After each task, a fixed weight-threshold rule assigns the new synaptic group to a protected state. The update mask permits connections from earlier groups into later groups, and a cue-dependent mask modulates a fixed base readout. We apply the framework to cognitive tasks from the Yang et al. suite¹⁹, quantify directed allocation through additional neuron recruitment during pairwise learning, and compare training curricula derived from those measurements. CoSyn-RNN introduces a biologically inspired model for continual learning, where heterogeneous synaptic states maintain past learning while flexibly enabling future learning and context-dependent modulation controls how recurrent computations are used efficiently. The resulting task-ordered structure is compatible with reuse of computational motifs and learned dynamical components across tasks, although the present analysis does not directly establish causal reuse.

2. Methods

2.1. Model architecture

We used discrete-time recurrent neural networks to study sequential allocation within a shared recurrent population. The model combines leaky integration, sparse allocation, a neuron-specific gain, context-dependent readout modulation, and between-task synaptic state consolidation. Its dynamics are

$$\begin{array}{l}\tilde{h}(t)=(1-\alpha )h(t-1)+\alpha \mathrm{LeakyReLU}\left(W_{\text{in}}\cdot x_{\text{in}}(t)+W_{\text{rec}}\cdot h(t-1)+b_{\text{h}}+\xi (t)\right),\\ h(t)=\tilde{h}(t)\odot \mathrm{ReLU}(g),\\ M(t)=\mathrm{Sigmoid}\left(W_{\text{mask}}\cdot x_{\text{cue}}(t)\right),\\ y(t)=\left(W_{\text{out}}\odot M(t)\right)h(t)+b_{\text{y}},\end{array}$$ (1)

where $\tilde{h}(t)$, $h(t)$, $g\in {ℝ}^H$ are pre-gain hidden activity, hidden activity, and gain for all hidden neurons $H$, $\alpha =dt/\tau$ is the update rate and $\tau$ is the relaxation time constant, $x_{\text{in}}(t)\in {ℝ}^F$ is the task input for all $F$ input features, $x_{\text{cue}}(t)\in {ℝ}^C$ is the task cue that encodes one of the $C$ tasks as one-hot, $b_{\text{h}}\in {ℝ}^H$ is the hidden bias, $\xi (t)\in {ℝ}^H$ is 0-mean Gaussian noise with standard deviation ${\sigma }_{\text{rec}}=0.05$, $W_{\text{in}}\in {ℝ}^{H\times F}$ is the input weight matrix, $W_{\text{rec}}\in {ℝ}^{H\times H}$ is the recurrent weight matrix, $W_{\text{mask}}\in {ℝ}^{O\times H\times C}$ is the neuromodulation weight, $M(t)\in {ℝ}^{O\times H}$ is the neuromodulatory mask applied to the non-trainable fixed readout weight matrix $W_{\text{out}}\in {ℝ}^{O\times H}$, $b_{\text{y}}$ is the non-trainable fixed output bias, $y(t)$ is the model output, “ $\odot$ ” denotes element-wise multiplication, and “•” denotes dot-product. During training, independent 0-mean Gaussian noise with standard deviation ${\sigma }_{\text{x}}=0.01$ is added to all input channels, i.e. $x_{\text{in}}(t)$ and $x_{\text{cue}}(t)$. We used a LeakyReLU nonlinearity to promote sparse activity without permanently silencing units after negative fluctuations early in training, and we constrained the neuromodulation mask between 0 and 1 using the Sigmoid function.

Recurrent and context-modulated readout weights must remain coupled throughout training for precise assignment of neurons and synapses to specific tasks for continual learning. To maintain this coupling, we included the nonnegative $g\in {ℝ}^H$ state that scales the activity of each recurrent neuron. This acts as a learned neuron-specific gain without modifying the input or recurrent weight matrix. Biologically, neuromodulatory systems can alter neuronal gain and reorganize population dynamics, and low-dimensional neuromodulatory control in recurrent networks can support flexible context-dependent computation^6,18. We use this correspondence only as a functional abstraction. $g$ is not a mechanistic model of a particular transmitter, receptor, projection, or biological gain-control pathway. Although biological gain is constrained by intrinsic cellular properties, we did not need to impose an explicit upper bound on the gain parameter since the learned $g$ values remained stable during training.

2.2. Task setup

We selected 16 tasks from the cognitive task suite of Yang et al.¹⁹. The task set includes Go, Anti, Decision Making (DM), and Delayed Decision Making (DelayDM) tasks. Each trial contains a fixation input, two population-encoded stimulus inputs carrying angular information, and a noisy one-hot cue that specifies the task context. The network generates the appropriate behavioral response based on the task cue. Figure 1c shows an example trial, and Supplementary Fig. S1 shows the input-output structure of all tasks. These tasks are useful for studying recurrent reuse because similar tasks can share dynamical motifs while retaining task-specific demands^15,16,19.

Figure 1: Schematic overview of CoSyn-RNN.

a Conceptual schematic, adapted from⁷, illustrating two intersecting low-error parameter regions for tasks A and B. The black dot denotes the initial parameter state, and the yellow dot denotes a shared low-error solution. The green arrows illustrate how sequential optimization without protecting previously learned synaptic states can move the parameters out of task A’s low-error region, resulting in catastrophic forgetting of task A. In contrast, the orange arrow illustrates how our model constrains subsequent parameter updates to remain within task A’s low-error region, thereby preserving performance on task A. b Schematic of the model architecture. Training assigns synapses from recurrent neurons to threshold-defined neuron groups (NG). The update rule permits connections from groups assigned earlier into groups assigned later, but the present experiments do not test whether later behavior depends on those connections. Connections into protected earlier groups are constrained and regularized, but not necessarily zero. All tasks share a common non-trainable fixed base readout matrix $W_{\text{out}}$, while cue-specific modulation selects the effective output mapping for each task. c Example trial from the FdGo task, illustrating the task input and target structure. Each trial contains a fixation channel, two population-encoded stimulus inputs, and a noisy one-hot cue signal. Full examples across tasks are shown in the Supplementary Fig. S1.

2.3. Training and consolidation

We trained tasks sequentially using backpropagation through time and the Adam optimizer with learning rate 10^–4, following related prior work^15,16,19. The task objective was the mean-squared error (MSE) between the output and its target. To encourage each task to use a compact neuron group, we additionally regularized individual recurrent weights, neuron level incoming and outgoing recurrent connectivity, neuronal gain, and modulated readout. This allocation scheme is inspired by the group-sparsity regularization in dynamically expandable networks¹³. Additionally, since strong sparsity pressure may interfere with task acquisition before behavioral performance is established, the sparsity terms were scaled by the accuracy-dependent amplification factor

$$a=1.5\ast \max (\text{accuracy}-\text{chance},0)/(1-\text{chance})$$ (2)

where chance and accuracy are defined in Supplementary Methods. The amplification increases as task accuracy increases, shifting optimization toward a sparser recurrent solution as the successful behavioral solution starts to emerge. We empirically set 1.5 as the maximum amplification through parameter sweeping to provide a trade-off between learning and sparsity.

The complete objective was

$$\begin{gathered} ℒoss=\mathrm{MSE}(y,\widehat{y})+ \\ a\cdot \left({\mu }_1\frac{1}{H}\sum _h^H\left|g_h\right|+\right. \\ +{\mu }_2\frac{1}{H^2}\sum _i^H\sum _j^H\left|W_{\text{rec}i,j}\right|+ \\ +{\gamma }_1\sqrt{\frac{H}{3}}\frac{1}{TBH}\sum _{t,b,h}^{T,B,H}\sqrt{\sum _o^O{\left(W_{\text{out}h,o}{M}_{t,b,h,o}\right)}^2}+ \\ +{\gamma }_2\frac{1}{H}\sum _i^H\sqrt{\sum _j^H{\left(W_{\text{rec}i,j}\right)}^2+} \\ +{\gamma }_3\frac{1}{H}\sum _j^H\sqrt{\left.\sum _i^H{\left(W_{\text{rec}i,j}\right)}^2\right)} \end{gathered}$$ (3)

where $T\approx 50-150$, $B=256$, $H=256$, and $O=33$ are the time, batch, hidden, and output dimensions, respectively. The time dimension was determined by the task definition and corresponded to approximately 1–3 s of simulated time. The batch size was chosen to provide stable optimization within the available computational resources, and the hidden dimension was set to $H=256$, following prior work on the same or closely related tasks^15,16,19. These prior studies used random task sampling and focused on recurrent networks with 256 hidden neurons, providing a network-size baseline for sufficient capacity. Here, $H=256$ was also sufficient for sequentially learning all tasks while still leaving a subset of recurrent neurons available for future tasks. Thus, we did not vary the size of our models. The coefficients ${\mu }_i$ and ${\gamma }_i$ weigh the regularizer terms, and $\sqrt{H/3}$ compensates for the initialization scaling of the fixed readout weights. All reported experiments used ${\mu }_1=0.1$, ${\mu }_2=1$, ${\gamma }_1=0.1$, ${\gamma }_2={\gamma }_3=0.01$, and a threshold of $\theta ={10}^{-4}$. These values were selected through a parameter sweep based on validation performance.

After training each task, we consolidated the neuron group associated with that task. A trainable neuron was treated as inactive only if none of its incoming or outgoing recurrent weights and none of its context-modulated readout weights exceeded $\theta ={10}^{-4}$ in absolute value. All active neurons for the current task were assigned to the learned task group. The synapses within the active group neuron and all synapses incoming into the active neuron group were then protected from updates during later tasks. Earlier formed groups could provide forward recurrent input to newly recruited groups. Our model imposes a discrete between-task consolidation step and is inspired by the distinction between transient and stabilized plasticity¹, but it is not a mechanistic model of synaptic tagging, sleep, or replay. After consolidation and protection of just-learned-task synapses, all other recurrent synapses of the network became active for future tasks. During consolidation, we do not change or re-initialize any weights.

3. Results

3.1. Heterogeneous synaptic states and neuromodulatory readout control support sequential learning of cognitive tasks

We first tested whether CoSyn-RNN could learn a sequence of tasks while preserving solutions acquired earlier. We selected eight tasks that span the task families described before and ranked them by empirical single-task difficulty, from easy (FdGo) to hard (MultiDelayDM). We call the easy-to-hard sequence the forward order and its reversal the reverse order. Comparing the two sequences allowed us to test how task order affects continual learning.

CoSyn-RNN learned each task as it was introduced in both orders, as shown by the training task losses and accuracies (Fig. 2a-d). More importantly, the validation losses and accuracies for previously learned tasks remained stable as the model learned additional tasks (Fig. 2e-h). The learning trajectories differed between orders, but the final performance of most tasks changed little. Thus, the protected synaptic states and context-modulated readout maintained earlier behavior across sequential learning rather than merely fitting the current task.

Figure 2: Continual-learning performance across task orders.

The left column shows the forward order: FdGo, FdAnti, ReactGo, ReactAnti, DelayGo, DelayAnti, MultiDM, and MultiDelayDM. The right column shows the reverse order. Rows show training task loss (a,b), training accuracy (c,d), validation loss (e,f), validation accuracy (g,h), training sparsity loss (i,j), and accuracy-dependent amplification factor (k, l). Colors denote tasks, vertical lines indicate task transitions, solid lines show the mean across four random seeds, and shading shows the minimum-to-maximum range. Dashed lines indicate chance level in accuracy panels and unit line for reference in amplification panels. Training task loss is the MSE term in Eq. 3; training sparsity loss is the sum of the remaining regularization terms. Validation shows task MSE without sparsity terms. Accuracy is defined in Supplementary Methods.

Task order nevertheless affected how optimization balanced task performance against sparse allocation. The sparsity loss started high when a new task was introduced and decreased as the network adjusted to the new constraints (Fig. 2i,j). In the forward order, the balance between performance and sparsity favors sparsity for the later, harder tasks. While this balance can be manually adjusted, we aimed to choose a dynamic, automatic amplification factor (Eq. 2) which could potentially be biologically implemented and may need further fine-tuning. The same tasks reached better final performance when learned early in the reverse order (compare ‘forward’ Fig. 2c,g with ‘reverse’ Fig. 2d,h accuracy for MultiDelayDM). This pattern suggests that task order influences how optimization balances task performance against sparsity, particularly for tasks of different difficulty, rather than exposing an inherent limitation of the continual-learning mechanism. Adjusting this balance according to task difficulty may preserve sparse allocation while ensuring that harder tasks retain access to sufficient recurrent resources.

3.2. Emerging modular structure supports forward reuse of dynamics

Ordering neurons by their number of above-threshold connections and marking their assigned group boundaries revealed a block-structured recurrent matrix (Fig. 3a,b). Connections from later groups back to earlier protected groups were limited (above the block-diagonal), whereas the model permitted connections from earlier groups into later-recruited groups (below the block-diagonal). The sizes of assigned groups varied across tasks, and related Go and Anti family tasks often required smaller additions when one followed the other. While this does not directly prove forward reuse of dynamics, it suggests a structured allocation that could facilitate such reuse. Establishing reuse will require causal lesions or direct analysis of trajectories, subspaces, or fixed points¹⁵.

Figure 3: Recurrent structure and readout modulation after training.

a,b Recurrent matrices $W_{\text{rec}}$ for the forward- and reverse-order models. Neurons are sorted by the number of connections with $\left|W_{ij}^{\text{rec}}\right|\ge {10}^{-4}$, with a common order for both axes. Weights are clipped to $\left[-{10}^{-4},{10}^{-4}\right]$ for visualization purpose. Top and left colored strips and black lines indicate threshold-assigned groups and their boundaries. c,d Gains. e,f Fixed shared base readout matrices $W_{\text{out}}$. g-v Task-specific context-modulated readout matrices. Weights are clipped to $\left[-{10}^{-4},{10}^{-4}\right]$ for visualization purpose. Results for the other seeds are provided in the Supplementary Material.

The training curricula revealed a significant strength of the model’s sequential learning capabilities. CoSyn-RNNs have the ability to reuse previously learned dynamics for new tasks, reducing the need for additional task-specific recurrent neurons. For example, the FdAnti task recruited no additional task-specific recurrent neurons when it appeared last in the reverse curriculum (Fig. 3b), suggesting all required dynamics were already captured by the previously learned tasks. Instead, it relied on the existing recurrent population and task-specific readout modulation to perform the task. More broadly, the size of the neuron groups depended on the previously learned repertoire (Fig. 1a,b), generally increasing with task-group difficulty and decreasing in the reverse order. One interpretation is that earlier tasks established the available recurrent dynamics basis for later tasks, which are reused through the feedforward connections from previous blocks into these later groups. We explored this possibility further via task-order-dependent optimization in the next sections. Alternatively, the interaction between task loss and sparse regularization may also influence the observed allocation patterns, detailed below.

Our training method and orders also revealed a key trade-off between capacity and performance. A substantial fraction of neurons remained unassigned after all eight tasks and therefore available for additional recruitment. This enables the model to continue learning new tasks without immediately exhausting its recurrent computational capacity. The balance between task-loss and regularization tightly controls the capacity and performance trade-off. For example, in MultiDelayDM, increasing sparsity pressure during learning resulted in a small recruited recurrent group and imperfect, but still very strong, final performance (Fig. 2g). The current experiments do not isolate a causal effect of the regularizer, but the model is capable of learning the task successfully under different conditions: see the reverse-order curriculum that started with MultiDeiayDM. Alternative methods to balance the task loss and sparsity regularization could potentially mitigate this issue, but exhausting that search space is outside the scope of this work.

3.3. Pairwise recruitment varies with task order

We quantified pairwise task similarity for the tasks in the Yang et al. task set¹⁹ by counting the additional recurrent neurons recruited when Task 2 was learned after Task 1 (Fig. 4a). This measure is directed because the recurrent state established by Task 1 determines the resources available to Task 2. For reference, the diagonal reports the number of neurons recruited when a naive network learned each task independently. Although training several tasks subsequently can alter these first-order estimates, we restricted this analysis to first-order pairwise tasks because the number of longer task sequences grows combinatorially.

Figure 4: Pairwise recurrent allocation and task accuracy.

Columns denote the first learned task (Task 1), rows denote the second (Task 2), and statistics use four random seeds. a Mean of additional neurons recruited for Task 2. Diagonal entries are the mean of the number of neurons recruited by the task independently. b Mean of preserved Task 1 accuracy after Task 2 training. Diagonal entries are the mean of accuracy learning task independently. c Mean of Task 2 accuracy after its training. Diagonal entries are the mean of accuracy learning task independently.

The pairwise matrix revealed structure across task families. Transitions from the delay-DM family to the DM family and from the Go/Anti family to the delay-DM family required moderately larger neuron groups. By contrast, transitions from the Go/Anti family to the DM family required substantially greater expansion and formed the most prominent high-value region in the matrix. This pattern suggests that the dynamics established by Go/Anti tasks are less compatible with the computations required by DM tasks than with those required by delay-DM tasks. Within this block, transitions from FdGo to the context-dependent DM tasks recruited the largest additional groups, suggesting particularly limited dynamical reuse for these pairs. These allocation patterns are consistent with reuse of compatible recurrent dynamics, but recruitment alone does not provide causal evidence for shared trajectories or computations.

By design, learning Task 2 did not disrupt Task 1 performance. Task 1 accuracy stayed nearly constant after learning Task 2 within each column, apart from variation across random seeds (Fig. 4b). Task 2 accuracy was also approximately constant within various row-blocks, indicating that it was governed largely by task complexity rather than by which task came first (Fig. 4c). Performance was particularly reduced when a DM-family task preceded a DelayDM-family task, suggesting less compatible recurrent dynamics between these task families in that direction. In contrast, performance was generally strong when a DelayDM-family task preceded a DM-family task, indicating a directional asymmetry in their interaction during sequential learning. This asymmetry may also reflect differences in how the two task families respond to sparsity pressure, since, as in Fig. 2, the learned solutions for the DM and delay-DM families were sensitive to the balance between task loss and sparsity regularization. This tradeoff warrants further investigation to determine how regularization can preserve sparse allocation without limiting task performance.

3.4. Optimal recurrent recruitment training curricula

The pairwise allocation matrix provides a tractable first-order approximation of the directional cost of learning one task after another, where cost means neurons consumed by tasks. Although training several tasks subsequently can alter these first-order estimates, they provide a useful guideline for constructing task orders that minimize or maximize the number of neurons involved in learning. Let $N_k$ denote the mean number of recurrent neurons recruited when task $k$ is learned as the initial task, and let $A_{kl}$ denote the mean number of additional recurrent neurons recruited when task $l$ is learned immediately after task $k$. For an ordered task sequence $\left(t_1,t_2,\dots ,t_N\right)$ where each task appears exactly once, we defined the cumulative first-order allocation cost as

$$C=N_{t_1}+\sum _{i=1}^{N-1}{A}_{t_i,t_{i+1}}.$$ (4)

The first term accounts for the neurons required to learn the initial task, whereas the remaining terms approximate the additional neurons recruited during successive task transitions. We used a Held-Karp style dynamic programming algorithm²⁰ to identify the minimum- and maximum-cost Hamiltonian paths (Supplementary Methods).

We trained separate CoSyn-RNNs on the complete sequence of 16 tasks using the minimum- and maximum-cost orders (Supplementary Methods). Training and validation losses, as well as accuracies, showed that the models learned the full task set in both curricula (Fig. 5a–h), while their final recurrent structures clearly differed (Fig. 5m,n). As predicted by the pairwise approximation, the minimum-cost order tended to recruit fewer recurrent neurons overall, leaving a larger unassigned population available for subsequent tasks. The two curricula therefore produced different levels of structural flexibility even when both supported the learned repertoire.

Figure 5: Performance and structure under maximum- and minimum-transition-cost curricula.

The left and right columns show the minimum- and maximum-cost orders, respectively. a,b Training task loss. c,d Training accuracy. e,f Validation loss. g,h Validation accuracy. i,j Training Sparsity Loss. k,l Accuracy-Dependent Amplification Factor. Colors denote tasks, dotted vertical lines indicate task transitions, solid lines show the mean across four random seeds, and shading shows the minimum-to-maximum range. m,n Sorted recurrent matrices $W_{\text{rec}}$ of the final models for seed 1. Colored bars indicate task-associated groups, black lines mark group boundaries, red and blue denote positive and negative recurrent weights.

The minimum-cost order did not follow an intuitive progression based on task family, empirical difficulty, or complexity. Instead, it tended to place difficult tasks relatively earlier than most simpler tasks (see order of all 10 variants from DM-family in Fig. 5c), consistent with the previous eight-task comparison in which harder tasks were easier to learn before sparsity and protection constrained more of the network. This pattern is qualitatively consistent with previously proposed optimal ordering principle where tasks should be learned from the least representative to the most typical²¹. However, this ordering differs from the biological intuition behind behavioral shaping, where simpler behaviors often scaffold more complex ones, but it does not contradict that principle. Rather, it points to different learning mechanisms underlying artificial and biological learning systems. The model uses explicit task boundaries, gradient-based optimization, and a fixed balance between task loss and sparsity, all of which differ from biological learning.

The observed structures also depend on hyperparameters, particularly the balance between task loss and sparsity regularization. Exhaustively searching that space is not computationally feasible and is beyond the scope of this study, and different settings could alter both performance and recurrent recruitment. The selected curricula should therefore be interpreted as a first-order demonstration that task order can change how much recurrent capacity remains for future learning, not as a universal difficult-first learning rule.

4. Discussion

CoSyn-RNN combines plastic and protected synaptic states with context-modulated output control in a recurrent continual learning model. Across the tested curricula, the protection rule preserved task-associated parameters, the update mask permitted earlier-to-later recurrent connections, and a cue-dependent mask selected effective outputs from a fixed base readout. The resulting connectivity reflected the intended task-ordered constraints, and the current results suggest, but do not establish, causal reuse or successful retention with infinite capacity. Recruitment of neurons between pairs of tasks revealed task-specific allocation patterns that depended on the order of task presentation. Such allocation allowed us to infer a first-order approximation of how to best train our recurrent model while maintaining maximum flexibility to learn future tasks.

The framework is biologically inspired rather than biophysically implemented. Synaptic tagging, ER-dependent plasticity, and replay motivate distinct timescales and states of plasticity^1,2,4,5, but CoSyn-RNN assigns threshold-defined neuron groups to a protected set at explicit task boundaries. Likewise, cue-dependent readout masking captures a computational function that neuromodulation could support^6,18; it does not model a specific transmitter, receptor, projection, or multi-area circuit. The modular connectivity is a designed signature of the update mask, not a discriminating biological prediction. Discriminating tests would instead compare task-aware alternatives and ask whether perturbing earlier groups or their projections selectively changes later-task behavior.

Several limitations constrain the conclusions. The synaptic-state threshold treats participation as binary and may consolidate weak but dispensable weights or exclude weak but important ones. Sparse regularization schedule interacts with task difficulty and could compress a new solution prematurely. The selected task sequences and four initializations do not establish robustness across alternative task suites, network sizes, or hyperparameter ranges. Finally, neuron recruitment is an allocation measure, not direct evidence of dynamical motif reuse. Analyses of hidden-state geometry, trajectory alignment, fixed points, task-dependent dimensionality, and causal lesions are needed to determine what computations are shared.

Future work could replace the fixed synaptic-state threshold with an adaptive importance estimate, allow graded or reversible consolidation, and compress several consolidated groups without replaying the original training data. The last direction would require a principled way to sample or preserve the network’s internal activity rather than assuming access to past examples. Higher-order analyses could also quantify how an entire learned repertoire, rather than the immediately preceding task, changes the cost of acquiring the next task. Extending the model to interacting recurrent areas with explicit neuromodulatory pathways would directly test the cross-area interpretation suggested by the functional readout gate.

Together, our results show that continual learning in a recurrent network can be supported by coordinating three functions: protecting previously recruited synapses, allocating unused recurrent capacity to new tasks, and using context to control how shared dynamics drive behavior. CoSyn-RNN learned a diverse suite of cognitive tasks sequentially while preserving earlier performance, and its task-ordered connectivity and directed recruitment costs revealed how learning history shapes both network structure and the capacity remaining for future tasks. By assigning concrete computational roles to heterogeneous synaptic states and context-dependent modulation, CoSyn-RNN provides a tractable framework for connecting biological hypotheses about stability and plasticity with artificial continual-learning methods. More broadly, this work suggests that durable learning need not require either complete parameter stability or unconstrained plasticity, but can emerge by protecting established computations while selectively recruiting and expressing new ones.