Balanced cortical microcircuitry for maintaining information in working memory

Abstract

Persistent neural activity in the absence of a stimulus has been identified as a neural correlate of working memory, but how such activity is maintained by neocortical circuits remains unknown. Here we show that the inhibitory and excitatory microcircuitry of neocortical memory-storing regions is sufficient to implement a corrective feedback mechanism that enables persistent activity to be maintained stably for prolonged durations. When recurrent excitatory and inhibitory inputs to memory neurons are balanced in strength, but offset in time, drifts in activity trigger a corrective signal that counteracts memory decay. Circuits containing this mechanism temporally integrate their inputs, generate the irregular neural firing observed during persistent activity, and are robust against common perturbations that severely disrupt previous models of short-term memory storage. This work reveals a mechanism for the accumulation and storage of memories in neocortical circuits based upon principles of corrective negative feedback widely used in engineering applications.

Working memory on a time scale of seconds is used to hold information in mind during cognitive tasks such as reasoning, learning, and comprehension¹. Over forty years ago², a neural correlate of working memory was identified when the sustained activity of cells of the prefrontal cortex was shown to encode the identity of a remembered stimulus during a memory period. Since this time, such persistent activity has been observed in a wide range of contexts and brain regions³. However, the mechanisms by which it is maintained remain poorly understood.

Biophysically, neurons are inherently “forgetful” due to the rapid leakage of currents out of their membranes. Previous theoretical work^3–7 has suggested that this leakage of currents can be offset if memory cells lie within circuits containing positive feedback loops that precisely replace leaked currents as they are lost (Fig. 1a, top). Models based upon this principle can maintain arbitrarily finely graded levels of persistent activity that, in theory, can last indefinitely. However, if the strengths of the positive feedback loops are slightly too strong or too weak, activity quickly spirals upward or downward until it either saturates or comes to rest at a baseline level^6–7 (Fig. 1a, bottom). As a result, positive feedback models of graded persistent activity require a fine tuning of the level of feedback and are highly sensitive to common perturbations, such as global changes in neuronal or synaptic excitabilities, that disrupt this tuning.

Fig. 1.

Memory networks with negative-derivative feedback. a–c, Simple models of a neural population and their energy surfaces with positive feedback (a), derivative feedback (b), and hybrid positive and derivative feedback (c). Persistent activity can be maintained at different levels (horizontal axis of energy surface) either by a positive feedback mechanism that effectively flattens the energy surface (a, c, bottom) or by a negative-derivative feedback mechanism that acts like a viscous drag force opposing changes in memory activity (b, c, bottom). The wall at the left of the energy surface represents the constraint that activity cannot be negative. d, Illustration of how a negative-derivative feedback mechanism detects and corrects deviations from persistent activity. e–g, Effective time constant of activity from Eq. (2) as a function of the strengths of positive feedback W_pos (e,g) and derivative feedback W_der (f, g). As W_der increases, the network time constant τ_eff becomes less sensitive to changes in W_pos (g).

Anatomically, neocortical circuits exhibit a plethora of both positive and negative feedback pathways. While positive feedback has been studied in detail, negative feedback pathways have received relatively little attention in models of working memory. Inhibition typically has been arranged either in “double-negative” loops that mediate a disinhibitory form of positive feedback⁸ or has served as a global, normalizing background⁹. By contrast, here we suggest that inhibition plays a critical role in providing corrective negative feedback that stabilizes persistent activity.

Our model depends upon two primary observations. First, cortical neurons receive massive amounts of both excitation and inhibition that, in a wide range of conditions and brain areas are believed to be closely balanced¹⁰. Second, recent studies of frontal cortical circuits have reported differential kinetics in the excitatory pathways onto excitatory versus inhibitory neurons. Excitatory to excitatory connections, commonly associated with positive feedback, have relatively slow kinetics due to an abundance of slow NMDA conductances^11–14. Excitatory to inhibitory connections, necessary to drive negative feedback, are relatively fast. Together, we show that these two observations lead naturally to a corrective, negative-derivative form of feedback that counteracts drift in persistent activity.

Below, we first illustrate the basic mechanism by which negative-derivative feedback can contribute to persistent activity and temporal integration and construct network models based upon this mechanism. The resulting derivative-feedback models are more robust to many commonly studied perturbations than previous models based purely upon positive feedback and, due to their inherent balance of inhibition and excitation, produce the highly irregular firing typical of neocortical neuron responses^15–16. Finally, we provide experimental predictions that differentiate our model from common positive feedback models and discuss implications of our model for the NMDA-hypothesis of working memory generation and dysfunction in disorders such as schizophrenia.

Results

Error correction through negative-derivative feedback

In the following, we show how observed features of frontal cortical circuits^{11–14,17–18} lead to a mechanism of memory storage based upon basic principles of engineering feedback control. In systems utilizing feedback control, a corrective signal is generated to oppose errors whenever a deviation from desired behavior is sensed. For the maintenance of persistent activity in memory circuits, the deviation to be detected and corrected is a change in time of the memory-storing activity, i.e. a temporal derivative (Fig. 1b, d). If memory activity drifts upward, corresponding to a positive derivative of activity, net inhibition should be provided to reduce the magnitude of this drift. Likewise, if memory activity drifts downwards, net excitation should be increased to offset this drift. Thus, in both cases, the required form of corrective feedback is in a direction opposite to the derivative of the neural activity and describes negative-derivative feedback.

To gain a quantitative understanding of how the derivative-feedback mechanism compares to the traditional positive-feedback mechanism, we first consider a simple mathematical model of a memory cell with intrinsic time constant τ that receives a transient input I(t) to be stored in memory. To successfully remember this input after its offset, the memory cell should exhibit only very slow changes $\frac{dr}{dt}$ in its firing rate r(t). This requires that its intrinsic leakage of currents, represented by the term −r below, be offset by positive feedback of strength W_pos (Fig. 1a, c, black; second term below) and/or by negative-derivative feedback of strength W_der (Fig. 1b, c, red; third term below):

$$\begin{array}{l}\tau \frac{dr}{dt}=-r+W_{\mathit{pos}}r-W_{\mathit{der}}\frac{dr}{dt}+I(t)\\ \Rightarrow (\tau +W_{\mathit{der}.})\frac{dr}{dt}=-(1-W_{\mathit{pos}})r+I(t)\end{array}$$ (1)

Positive feedback models do not contain the W_der term. They maintain persistent firing by providing a feedback current that, when properly tuned by setting W_pos=1, offsets the intrinsic tendency of currents to leak out of the membrane. However, if the feedback is too weak (W_pos<1), memory activity decays to a baseline level in a manner analogous to an inertia-less particle drifting towards the bottom of a hill (Fig. 1a, bottom). Likewise, if feedback is too large (W_pos>1), activity grows exponentially on a time scale set by the intrinsic time constant τ. Thus, to perform correctly, positive feedback models require fine tuning of the strength of the positive feedback. Quantitatively, this fine tuning condition is defined by the relation τ_eff = τ/(1 − W_pos ), where τ_eff is the exponential decay time constant of network activity in the presence of positive feedback (Eq. (1), Fig. 1e).

Negative-derivative feedback networks instead slow memory decay by providing a force that opposes the drift of memory activity in a manner mathematically identical to viscous drag forces in fluid mechanics (Fig. 1b, bottom). This drag force effectively extends the time constant of memory decay in proportion to the strength of the derivative feedback pathway. For the case in which there is no positive feedback (W_pos = 0), this leads to an effective network decay time constant τ_eff = τ +W_der (Eq. (1), Fig. 1f).

More generally, negative-derivative feedback can complement positive feedback by opposing drifts due to imperfect tuning of positive feedback (Fig. 1c). In this case, the network time constant reflects the effects of both positive and negative-derivative feedback and, from equation (1), is given quantitatively by

$${\tau }_{\mathit{eff}}=(\tau +W_{\mathit{der}})/(1-W_{\mathit{pos}}).$$ (2)

This relation is illustrated in Fig. 1g, which shows that, as the negative-derivative feedback gets stronger (contours of increasing W_der), the system becomes increasingly robust to mistuning of the positive feedback W_pos. We refer to any network containing a strong negative-derivative feedback component, as in Figs. 1b and 1c, as a negative-derivative feedback network. The special subclass of negative-derivative feedback networks with no positive feedback (W_pos=0) are denoted as “purely negative-derivative feedback” networks, while those that additionally contain tuned positive feedback (W_pos=1) are denoted as “hybrid positive and negative-derivative feedback” networks.

Negative-derivative feedback in neocortical microcircuitry

How can negative-derivative feedback arise from interactions between excitatory and inhibitory neurons in neocortical circuits? Mathematically, temporal derivatives are created when a signal is subtracted from the same signal offset in time. Likewise, derivative-feedback can be created in memory networks by feeding back a memory-storing signal through positive-and negative-feedback pathways that are equal in strength but have different kinetics. When memory activity slips, fast negative feedback mediated by recurrent inhibition rapidly opposes this slip, and then slower positive feedback restores the original balance of excitation and inhibition in the circuit. The net effect of this fast inhibition and slow excitation is a feedback signal that opposes changes, i.e. generates a negative temporal derivative, of memory cell activity (Fig. 1b, bottom).

To show how negative-derivative feedback can arise in a neural network, we constructed a two-population memory circuit model consisting of excitatory (E) and inhibitory (I) populations. The populations were reciprocally connected by synapses of strength J_ij and time constant τ_ij, where j=E or I denotes the presynaptic population and i denotes the postsynaptic population (Fig. 2a, top). This architecture contains a positive feedback loop represented by the E-to-E connection of strength J_EE, and a negative feedback loop of strength J_EIJ_IE/(1+J_II) mediated by the E-to-I-to-E pathway and modulated in strength by the I-to-I connection (Fig. 2a, bottom).

Fig. 2.

Negative-derivative feedback networks of excitatory and inhibitory populations. a, Derivative feedback network structure (top) and component feedback pathways onto the excitatory population (bottom). b, In response to external input that steps the excitatory population between two fixed levels, the recurrent feedback pathways mediate a derivative-like signal resulting from recurrent excitation and inhibition that arrive with equal strength but different timing. c, d, Maintenance of graded persistent firing in response to transient inputs (c) and integration of step-like inputs into ramping outputs (d) with linear (top) and nonlinear (bottom) firing rate (f) vs. input current (I) relationships.

Mathematical analysis of this network to determine the conditions under which persistent activity could be stably maintained revealed two classes of solutions (Supplementary modeling). The first class corresponded to the positive feedback mechanism (W_pos=1 in Eq. (1)) and was characterized by having a stronger positive feedback pathway than negative feedback pathway so that the net feedback offset the intrinsic leakiness of the neurons. The second class corresponded to negative-derivative feedback, as expressed mathematically by the conditions (see Supplementary modeling for additional inequalities required to maintain network stability):

$$\frac{J_{EE}{J}_{II}}{J_{EI}{J}_{IE}}~1\text{for}\text{large}{J}^{\prime }s,$$ (3)

$${\tau }_{+}=({\tau }_{EE}+{\tau }_{II})>({\tau }_{EI}+{\tau }_{IE})={\tau }_{-}.$$ (4)

Equation (3) expresses the condition for balancing positive feedback and negative feedback in strength. Equation (4) ensures that the combination τ₊ of synaptic decay time constants associated with positive feedback is slower than the combination τ₋ associated with negative feedback – here, τ_II acts like a positive feedback contribution because it governs the reduction of negative feedback. Thus, together, equations (3)–(4) define the conditions for negative-derivative-like feedback. Strictly speaking, the derivative-like behavior is only at low frequencies, as high frequencies are low-pass filtered by the synapses (Supplementary modeling). This may be advantageous compared to a true derivative, which amplifies high-frequency noise.

To illustrate this derivative-like feedback, in Figure 2b we show a simulation in which the firing rate of the excitatory neuron was clamped by external current injection to go through a perfect step from one steady firing rate to another. During the periods of steady persistent firing before or long after the step in firing rate, excitation (Fig. 2b, top, blue) and inhibition (red) are balanced, so that the net recurrent synaptic input (Fig. 2b, bottom) is zero. However, if activity fluctuates, then the different kinetics of the positive and negative feedback pathways lead to a large, derivative-like recurrent input that opposes the change in network activity (Fig. 2b, black).

Both of the conditions for negative-derivative feedback are present in cortical memory networks. A balance between strong excitatory and inhibitory synaptic inputs has been observed under a wide range of conditions¹⁰, including during sustained activity in prefrontal cortex^17–18. Slow E-to-E synaptic kinetics have been found due to a prominence of slow NMDA-type receptors^11–14. When we incorporated these findings in the model, the network maintained long-lasting persistent activity that reflected the level of its transient input (Fig. 2c, Supplementary Fig. S1f). The network time constant of activity decay, τ_network, increased linearly with the J’s and with the difference between the time constants τ₊ and τ₋, allowing us to directly connect the network parameters to the strength of derivative feedback in the simpler model of Eq. (1) through the relation W_der ≈ τ_network ~ J(τ₊ − τ₋) (Fig. 3c, Supplementary modeling). More generally, the network acted as an integrator of its inputs with this same time constant, for example converting steps of input into linearly ramping activity (Fig. 2d, Supplementary Fig. S1i).

Fig. 3.

Negative-derivative feedback with mixture of NMDA/AMPA synapses in all excitatory pathways. a, Derivative feedback network structure. Blue, cyan, and red curves represent NMDA-mediated, AMPA-mediated currents, and GABA-mediated currents, respectively. q_EE and q_IE are the fractions of NMDA-mediated synaptic inputs in each excitatory pathway. b, Time constant of decay of network activity τ_network as a function of the average time constants of excitatory connections, aver(τ_EE ) and aver(τ_IE ). Each average time constant is varied either by varying the fractions or the time constants of NMDA-mediated synaptic inputs in each connection. The region in the red rectangle corresponds to a set of possible aver(τ_EE ) and aver(τ_IE ) obtained when varying q_EE and q_IE while holding the synaptic time constants fixed at values matching the experimental observations in [13]. c, Time constant of decay of network activity τ_network as a function of the connectivity strengths J_ij and the time constants of positive and negative feedback, τ₊ and τ₋. τ_network increases linearly with the balanced amount of positive and negative-derivative feedback J_der ~ J_EE ~ J_IEJ_EI/J_II, and with the difference between τ₊ and τ₋, as W_der~J_der(τ₊ −τ₋).

A potential concern is that the opposition to firing rate changes provided by the negative-derivative feedback mechanism might keep the network from responding to external inputs. However, external inputs comparable to the recurrent inputs in strength, as would be expected if the strengths of both recurrent and external inputs scale with population size, can overcome the derivative feedback and transiently imbalance excitation and inhibition, as observed experimentally during transitions between different levels of sustained activity^17–18 (Supplementary modeling). Furthermore, appropriate arrangement of the external inputs can reduce the derivative feedback by amplifying this transient imbalance (Supplementary modeling)¹⁹.

Reinterpretation of the NMDA-hypothesis for working memory

In traditional positive feedback models^{4–5,20–21}, NMDA-mediated synaptic currents computationally serve to provide a non-specific, slow kinetics process in all feedback pathways. Consistent with this role, NMDA-mediated currents in such models are typically present equally in all neurons, both excitatory and inhibitory. Our model suggests an additional role for NMDA-mediated currents in providing the slow positive feedback component of a derivative-feedback signal. This requires that the contribution of NMDA-mediated currents be stronger in positive-feedback than in negative-feedback pathways.

To investigate this revised NMDA-hypothesis for memory circuits, we extended our network models to include both NMDA-mediated and non-NMDA (AMPA-mediated) currents at all excitatory synapses (Fig. 3a). Experimentally, recent measurements of the AMPA and NMDA-driven components of excitatory transmission have identified two means by which NMDA may contribute more strongly to positive feedback than negative feedback pathways. First, NMDA-mediated currents can be a higher fraction of total excitatory synaptic currents in excitatory-to-excitatory than excitatory-to-inhibitory connections^11,13. Second, the NMDA-driven component can have slower kinetics^11–14 in excitatory neurons than inhibitory neurons. Below, we show quantitatively how this asymmetry in excitatory time constants contributes to negative-derivative feedback.

The model with multiple components of excitatory transmission is shown in Fig. 3a. All excitatory synapses contained both NMDA- and AMPA-type synapses so that both the positive and negative feedback loops contained slow and fast synaptic components. Nevertheless, we found that the conditions for derivative feedback-mediated persistent activity followed the same principles identified in the simple network model of Fig. 2, that is, a balance between the total positive and negative feedback in strength and slower positive feedback on average. More precisely, the conditions for negative-derivative feedback are still represented by equations of the form of equations (3) and (4) above. However, J_EE and J_IE in Eq. (3) now represent the sum of the strengths of NMDA- and AMPA-mediated synaptic currents onto excitatory and inhibitory neurons, respectively, and the time constants τ₊ and τ₋ of positive and negative feedback in Eq. (4) now represent the weighted average of the synaptic time constants contributing to positive and negative feedback, respectively (Methods, Supplementary modeling).

Thus, even in the presence of slow kinetics in the negative feedback (E-to-I) pathway or fast kinetics in the positive feedback (E-to-E) pathway, negative-derivative feedback arises when the positive feedback is slower than the negative feedback on average. As in the simpler networks, the time constant of decay of network activity increases with the difference between the average time constants of positive and negative feedback (Fig. 3b, c). This slower positive than negative feedback can be achieved either with a higher fraction of NMDA-mediated currents (q_EE> q_IE, Supplementary Fig. S2a) or with slower NMDA kinetics ( ${\tau }_{EE}^N>{\tau }_{IE}^N$, Supplementary Fig. S2b) in the E-to-E connection. Thus, this work suggests a revised NMDA hypothesis that highlights the experimentally observed^11–14 asymmetric contribution of NMDA receptors in positive and negative feedback pathways as a basis for negative-derivative feedback control.

Robustness of memory performance to common perturbations

A prominent issue in models of neural integration and graded persistent activity is their requirement for tuning of network connection strengths, and lack of robustness to perturbations that disrupt this tuning. Several biologically motivated solutions have been proposed to mitigate this problem; for example, a large body of work has shown that the tuning requirements can be greatly reduced if network feedback mechanisms are complemented by cellular^22–24 or synaptic^25–27 persistence mechanisms. However, a largely neglected question in these discussions is whether biological systems are designed to be robust against all types of perturbations and, if not, what types of circuit architectures are robust against the most commonly experienced perturbations.

In traditional positive feedback models of analog working memory and neural integration, both inhibition (through disinhibitory loops) and excitation mediate positive feedback (see Fig. 6a, b). As a result, many natural perturbations – loss of cells, change in cell excitabilities, or changes in the strengths of excitatory or inhibitory synaptic transmission –change the net level of positive feedback in the network and grossly disrupt persistent firing (Fig. 4a–f). By contrast, in models based upon derivative feedback (Fig. 4g–l), each of these natural perturbations leads to offsetting changes. For example, because excitatory cells drive both positive feedback (through E-to-E connections) and negative feedback (through E-to-I connections), loss of excitatory cells or decrease of excitatory synaptic transmission does not disrupt the balance of positive and negative feedback underlying derivative feedback (Fig. 4j). Similarly, changes in intrinsic neuronal gains do not imbalance the positive and negative feedback received by cells (Fig. 4i; Supplementary Fig. S1), and changes in inhibitory synapses or loss of inhibitory neurons produce offsetting changes in positive (I-to-I) and negative (I-to-E) feedback pathways (Fig. 4k). Mathematically, the origin of this robustness is that the tuning condition for the derivative-feedback networks (Eq. (3)) is ratiometric, with the excitation and inhibition received by and projected by a cell population appearing in both the numerator (positive feedback contributions) and denominator (negative feedback contributions).

Fig. 6.

Synaptic inputs in derivative feedback and common positive feedback models. a–c, Network structures of positive feedback models (a, b) and derivative feedback models (c) with two competing populations. d–f, Relation between firing rates of excitatory and inhibitory neurons. Firing rates of the E₂ (black points) and inhibitory (red points) populations are plotted as a function of E₁ firing rate. g–i, Relation between excitation and inhibition for different levels of maintained firing. X- and y-axes are normalized by the amount of excitation and inhibition received when the left and right excitatory populations fire at equal levels of 30 Hz. j–l, Persistent activity in the two competing excitatory populations (solid: E₁; dashed, E₂). Perturbing the networks by uniformly increasing the intrinsic gain in E₁ leads to gross disruptions of persistent firing in positive feedback models (green curves in j, k), but not negative-derivative feedback models (l). See Supplementary Fig. S5 for robustness to other perturbations.

Fig. 4.

Robustness to common perturbations in memory networks with derivative feedback. a–f, Non-robustness of persistent activity in positive feedback models. a, Positive feedback models with recurrent excitatory (left) or disinhibitory (right) feedback loops. b, Effective time constant of network activity, τ_network, as a function of connectivity strength. Green asterisks correspond to 5% deviations from perfect tuning. c–f, Time course of activity in perfectly tuned networks (black) and following small perturbations of intrinsic neuronal gains (c) or synaptic connection strengths (d–f). g–k, Robust persistent firing in derivative feedback models. To clearly distinguish the hybrid models with derivative and positive feedback, purely negative-derivative feedback models with no positive feedback are shown. All excitatory synapses are mediated by both NMDA and AMPA receptors as in Fig. 3, with parameters chosen to coincide with experimental observations [13]. h, τ_network increases linearly with the strength of recurrent feedback J. i–k, Robustness to 5% changes (green asterisks in h) in neuronal gains or synaptic connection strengths. l, Disruption of persistent activity in derivative feedback models following perturbations of NMDA-mediated synaptic currents. m, Hybrid model with positive and derivative feedback. n–q, As the strength of negative-derivative feedback is increased, τ_network decreases less rapidly with mistuning than in purely positive feedback models (n) and the network becomes robust against perturbations (o–q, shown for J_der/J_pos=150). r, Disruption of persistent activity in the hybrid model following perturbations of NMDA-mediated currents.

The negative-derivative feedback models are not robust against perturbations that break the balance of inhibition and excitation. For instance, perturbations that differentially affect excitatory-to-excitatory versus excitatory-to-inhibitory synaptic transmission, or inhibitory-to-inhibitory versus inhibitory-to-excitatory transmission will disrupt persistent firing. For example, because NMDA-mediated currents are relatively stronger onto excitatory neurons than onto inhibitory neurons, disruptions in such currents break the balance between positive and negative feedback (Fig. 4l), with the precise size of the disruption being dependent upon how asymmetrically NMDA receptors are distributed between the two pathways (Supplementary Fig. S3, Supplementary modeling). Such relative frailty to perturbations that break the E-I balance forms a prediction for the derivative feedback models (see Discussion).

We note that the negative-derivative feedback and positive feedback mechanisms are not mutually exclusive. Hybrid models receiving strong negative-derivative feedback and tuned positive feedback (Fig. 4m–r) can be obtained by increasing the strength of net excitatory feedback enough to offset the intrinsic decay of the neurons (Fig. 4m). Doing so leads to networks that are both perfectly stable when properly tuned and, due to the strong and approximately balanced negative-derivative feedback, decay only mildly when mistuned (Fig. 4n–q).

Irregular firing in spiking graded memory networks

A major challenge²⁸ to existing models of working memory has been generating the highly irregular spiking activity observed experimentally during memory periods (Fig. 5a). In traditional positive feedback models, the mean synaptic input is suprathreshold and therefore drives relatively regular firing. Previous theoretical²⁹ and experimental¹⁰ work instead suggests that the irregular activity seen in cortical networks results from strong inhibitory and excitatory inputs that mostly cancel on average but exhibit fluctuations that lead to a high coefficient of variation of the inter-spike intervals (CV_isi).

Fig. 5.

Irregular firing in spiking networks with graded persistent activity. a, Experimentally measured irregular firing (coefficients of variation of inter-spike intervals, CV_isi, higher than 1) during persistent activity in a delayed-saccade task. Adapted from [16]. b, Structure of network of spiking neurons with negative-derivative feedback. c–k, Network response to a brief (100 ms) stimulus applied at time 0. c–e, Raster plots illustrating irregular persistent firing of 50 example excitatory neurons. f–h, Instantaneous, population-averaged activity of excitatory neurons, computed within time bins of 1 ms (gray) or 10 ms (black). i–k, Balance between population-averaged excitation and inhibition following offset of external input. l–n, Histogram of CV_isi of active excitatory neurons during the persistent firing. Note that, for activity with strong input, a small subset of neurons fire regularly at high rate and exhibit low CV_isi (n). This reflects that the heterogeneity resulting from our simple assumption of completely randomly connected networks can result in excess positive feedback in some clusters of neurons.

To demonstrate irregular firing across a graded range of firing rates in the negative-derivative feedback model, we constructed a recurrently connected network of integrate-and-fire neurons consisting of excitatory and inhibitory populations with random, sparse connections between and within the populations³⁰. The averaged excitation and inhibition between the populations satisfied the same balance condition, J_EE~J_EIJ_IE/J_II, as in the firing rate models. Inhibitory currents were mediated by GABA_A receptors. Recurrent excitatory currents were mediated by a mixture of AMPA and NMDA receptors (Fig. 5b), with a greater proportion of and slower kinetics of NMDA receptors in the excitatory feedback pathways^11–14.

As in the simpler two-population model, the network exhibited graded persistent activity whose level reflected the strength of input (Fig. 5c–h) and integrated steps of input into ramping output (Supplementary Fig. S4). At each maintained level, the mean synaptic inputs to each population exhibited a close balance between inhibition and excitation, with spikes triggered primarily by fluctuations away from the mean input (Fig. 5i–k). This led to the observed highly irregular activity and, as observed experimentally, a CV_isi distribution whose mean value exceeded 1 (Fig. 5l–n). This irregular Poisson-like firing might serve a valuable computational purpose, as Bayesian network models have suggested that Poisson firing statistics may enable probability distributions from different inputs to be combined efficiently^31–32.

Circuits with a push-pull architecture: predictions

Above, we considered a single excitatory and inhibitory population. However, neuronal recordings during parametric working memory (e.g. [33]) or neural integration (e.g. [34–35]) typically show a functional “push-pull” organization in which competing populations of cells exhibit oppositely directed responses to a given stimulus. Here, we show that a push-pull organization is consistent with the derivative-feedback mechanism, has additional robustness to perturbations in external inputs, and generates predictions that differentiate the derivative-feedback and traditional positive-feedback models.

To construct a push-pull derivative-feedback network, we interconnected two of our two-population models (Fig. 6c, E₁ and I₁; E₂ and I₂) through mutual inhibitory connections (Fig. 6c, E₁ to I₂ and E₂ to I₁). When the circuit was tuned to have a balance of slow positive and faster negative feedback (Supplementary modeling), the circuit maintained a graded range of persistent firing, with the left population increasing its firing rate when the right population decreased and vice-versa (Fig. 6f, black points, different levels of sustained activity; Fig. 6l, example firing rate traces). Persistent activity was robust to common perturbations, as in the simpler 2-population models (Fig. 4), even when the perturbations were applied only to a single population (Fig. 6l, Supplementary Fig. S5). In addition, global shifts in background input, such as might be caused by system-wide changes in excitability, did not change the stability of persistent activity (Supplementary Fig. S5d) and noise caused temporally local jitter but was largely averaged out over the long time scales of integration (Supplementary Fig. S5h). The former result differs from simpler models based upon a single E and I population, which improperly exhibit ramping activity in response to global shifts in external input; this has been suggested as a fundamental reason for the observed push-pull architectures of integrator and graded short-term memory networks³⁶.

A prediction for how the derivative feedback model can be distinguished from traditional positive feedback models is provided by examination of the intracellular currents onto the excitatory cells in each network. In the derivative feedback models, these currents are balanced and therefore positively covary across different levels of sustained activity (Fig. 6i). By contrast, in traditional positive feedback models, inhibition is either driven by the opposing population of excitatory neurons (Fig. 6a) or receives equal strength connections from both populations (Fig. 6b). In the former case, synaptic inhibition reflects the firing rates of the opposing population (Fig. 6d, black) and is anti-correlated with the excitatory inputs arriving from the same population (Fig. 6g). In the latter case, inhibitory neuron firing represents an average of the activity in the competing excitatory populations – if the activities of the competing excitatory populations vary symmetrically about a common background level, inhibitory neuron firing will vary only weakly with different levels of activity (Fig. 6e, red), leading to minimal correlations between inhibitory and excitatory inputs (Fig. 6h). If instead the dominant (higher firing rate) population varies its activity more than the non-dominant population³⁴, then the summed inhibition will follow the activity of the dominant population, switching when the opposite population becomes dominant and leading to a non-monotonic pattern of synaptic input correlations when viewed across the entire firing rate range (data not shown).

Discussion

Here we demonstrated a new mechanism for short-term memory based on negative-derivative feedback control. Networks based upon this mechanism maintain activity for long durations following the offset of a stimulus and more generally act as temporal integrators of their inputs. The core requirement for negative-derivative feedback is that the pathways mediating positive and negative feedback be balanced in strength, but with slower kinetics in the positive feedback pathways. We showed that these two conditions lead to a balance between excitation and inhibition during steady persistent firing, and that this balance can be transiently disrupted by external inputs in order to allow a circuit to change its firing rates.

Compared to previously proposed memory networks based upon positive feedback, negative-derivative feedback networks have several advantages. First, negative-derivative feedback networks inherently incorporate the observation that frontal cortical circuits have both positive and negative feedback pathways, with an asymmetry in the time constants of synaptic excitation onto excitatory versus inhibitory neurons^11–14. Second, negative-derivative feedback networks are robust against many commonly studied perturbations to synaptic weights that grossly disrupt memory performance in positive feedback models. Third, negative-derivative feedback networks inherently generate irregular firing across a graded range of persistent activity levels. These advantages are still attained in hybrid networks containing both positive and negative-derivative feedback; thus, negative-derivative feedback is complementary to positive feedback and both mechanisms are likely to be used together in many circuits. A balance between excitation and inhibition has been suggested as a general principle underlying the dynamics of a wide variety of cortical circuits.

Physiologically, for cortical cells with large numbers of synaptic contacts and experimentally measured postsynaptic potential amplitudes, a close balance between excitation and inhibition may be essential to avoid saturation or total silencing of firing rates^37–38. In sensory systems, the balance between inhibition and excitation includes the contribution of the external excitation driving the circuit^30,39, and activity does not persist following the offset of the stimulus. By contrast, in the present work, the balance is obtained in the absence of external driving input and depends purely on recurrent synaptic inputs (or possibly a tonic background input). In bistable memory circuits, balanced excitation and inhibition^40–41 has been proposed to explain the irregular firing activity observed during elevated (UP) states of network activity¹⁶. However, these models used identical time constants for the positive feedback and negative feedback pathways so that there was no derivative feedback. As a result, they could not achieve both irregular firing activity and the graded range of persistent firing rates observed during parametric working memory and temporal integration.

A major challenge to models of graded persistent activity is maintaining the tuning of network connection strengths. In positive feedback networks, the quantity to be tuned is the net level of network positive feedback. In negative-derivative feedback networks, the tuned quantity is the balance between excitation and inhibition. Previous foundational work in positive feedback networks has shown that the severity of this requirement may be markedly decreased if circuit mechanisms are complemented by cellular persistence mechanisms such as slow synaptic facilitation^25–27 or dendritic plateau potentials generated by NMDA or other voltage-activated inward currents^3,22–24,42. Similar results hold for the derivative feedback models if the slow process is in the excitatory-to-excitatory connections, and both dendritic plateau potentials and slow synaptic facilitation have been observed experimentally at such connections^3,26,42. In addition, tuning of negative-derivative feedback can be accomplished locally if neurons can monitor their balance of excitatory and inhibitory inputs. Indeed, recent experimental^43–44 and theoretical⁴⁵ work suggest that both homeostatic and developmental processes regulate this excitatory-inhibitory balance, even at the level of localized dendritic compartments⁴³. The learning rules underlying the maintenance of this balance are currently unknown experimentally and are an important issue for future exploration. However, preliminary investigations suggest that a previously proposed differential Hebbian learning rule⁴⁶ may suffice to maintain the tuning of both the 2-population and 4-population derivative-feedback networks (Supplementary Fig. S6).

A separate question of robustness, focused upon here, is what types of perturbations biological networks typically experience and most need to be robust against. A principle of robust control theory is that systems cannot be robust against all possible perturbations, but should be robust against common perturbations⁴⁷. Implicitly invoking this principle, previous work has justified positive feedback models as robust in the sense that random perturbations of connectivity only minimally affect the mean level of positive feedback^36,48, and the same argument applies to the derivative-feedback models. However, many other common perturbations such as loss of cells or changes in neuronal gains severely affect positive feedback models. By contrast, derivative-feedback models can be dramatically more robust to these perturbations because they produce offsetting changes in the positive and negative feedback pathways (Fig. 4). Derivative-feedback models are susceptible to perturbations that disrupt the E-I balance of neurons, and this difference in robustness to different types of perturbations provides useful predictions. For example, we predict that completely silencing a subset of excitatory neurons would be less disruptive than silencing their synaptic inputs onto only their excitatory or only their inhibitory targets, consistent with a recent pharmacological perturbation study that showed severe disruption of persistent activity following selective targeting of NR2B-subunit containing NMDA receptors in prefrontal cortex that are primarily located at E-to-E synapses¹⁴. Similarly, we predict that globally perturbing GABAergic transmission from a subset of inhibitory neurons would be less disruptive than perturbing this input only onto its excitatory or only onto its inhibitory targets.

Slow excitation specifically in the positive feedback pathway of negative-derivative feedback networks suggests a revision of the NMDA-hypothesis for working memory storage^{4–5,20–21} and deficits in schizophrenia⁴⁹. Previously, the assumed role of NMDA receptors had been to provide a non-specific, slow cellular time constant in all excitatory pathways^{4–5,20–21}. By contrast, recent experimental studies^11–14 reported asymmetric contributions of NMDA receptors in different feedback pathways. Building upon these studies, we demonstrate an additional role of NMDA receptors in providing the delayed excitation required for negative-derivative feedback, and suggest that future efforts to develop drugs for working memory disorders consider the differential contributions of NMDA receptors onto excitatory versus inhibitory target neurons.

In summary, this work provides a new paradigm for the storage of short-term memory based on corrective negative feedback. Negative feedback is a common principle of engineering control systems, in which a fundamental tenet is that strong negative feedback leads to system output (for example, an integral) that reflects the inverse of the feedback signal (for example, a derivative). Our work suggests that a similar principle is used by neocortical microcircuits for the accumulation and storage of information in working memory.

Online Methods

In the main text, we proposed a neocortical circuit architecture for short-term memory storage based upon having a balance in strength of positive and negative feedback pathways, but with positive feedback pathways exhibiting slower kinetics. These networks implement an error-correcting signal of the form of negative-derivative feedback that enables the networks to maintain persistent activity. This principle was realized in both firing rate models composed of one or multiple excitatory and inhibitory populations (Figs. 2–4 and 6; Supplementary modeling 1, 2) and in spiking models consisting of recurrently connected integrate-and-fire neurons (Fig. 5, Supplementary modeling 3). Below, we describe the network structure and equations governing the dynamics of each of these network models.

Firing rate model of one excitatory and one inhibitory population

The firing rate models of Figure 2 were used to describe the dynamics of the average activities of, and synaptic interactions between, networks composed of one excitatory and one inhibitory population. We denote the mean firing rates of the excitatory and inhibitory populations by r_E and r_I, respectively, and the synaptic state variables for the connections from population j onto population i by s_ij. These firing rate and synaptic state variables are governed by the equations:

$$\begin{array}{ccc}{\tau }_E{\stackrel{.}{r}}_E& =& -r_E+f_E(J_{EE}{s}_{EE}-J_{EI}{s}_{EI}+J_{EO}i(t))\\ {\tau }_I{\stackrel{.}{r}}_I& =& -r_I+f_I(J_{IE}{s}_{IE}-J_{II}{s}_{II}+J_{IO}i(t))\\ {\tau }_{ij}{\stackrel{.}{s}}_{ij}& =& -s_{ij}+r_j\text{for}i,j=E,\text{or}I\end{array},$$ (5)

where the dot over a variable indicates differentiation with respect to time. Thus, the mean firing rate r_i approaches f_i(x_i) with intrinsic time constant τ_i, where f_i(x_i) represents the steady-state neuronal response to input current x_i. In the paper, we consider two types of neuronal response functions: linear f(x) = x (top panels of Fig. 2c, d and Figs. 3, 4, and 6) and a nonlinear neuronal response function (bottom panels in Figs. 2c, d and S1, S6) having the Naka-Rushton⁵⁰ form

$$f(x)=M\frac{{(x-x_{\theta })}^2}{{x_0}^2+{(x-x_{\theta })}^2}h(x-x_{\theta }),$$ (6)

where M represents the maximal neuronal response, x_θ represents the input threshold, x₀ defines the value of (x − x_θ) at which f(x) reaches its half-maximal value, and h(x) denotes the step function h(x) = 1 for x ≥ 0 and h(x) = 0 for x < 0.

Inputs x_i to each population include the synaptic current J_ijs_ij from population j to population i and the external current J_iOi(t), where the function i(t) (not to be confused with the subscript i) denotes the temporal component of the external current. J_ij represents the synaptic connectivity strength onto postsynaptic neuron i from presynaptic neuron j, and the synaptic variables s_ij approach the presynaptic firing rate r_j with time constant τ_ij. We assume that one external source provides the external input to the excitatory and inhibitory populations, with J_iO representing the strength of the input onto population i. To model in a simple manner how stimuli are smoothed before their arrival at the memory network, we assume that the externally presented pulses of duration t_window =100 ms (Fig. 2c) or step inputs (Fig. 2d) are exponentially filtered with time constant τ_ext =100 ms.

In Figure 2b, we performed a firing rate clamp experiment to illustrate how recurrent excitatory and inhibitory inputs provide negative-derivative-like feedback in response to a change in firing rate. In this experiment, in which r_E steps between two fixed levels, the external input to the excitatory population in Eq. (5) is adjusted so that the profile of r_E becomes a step function h(t). The remaining variables then are allowed to vary following the equations given in Eq. (5).

In Figures 3 and 4, we consider networks with a mixture of two different types of synapses, NMDA-type and AMPA-type, in both of the excitatory pathways (from E to E and E to I). Thus, the excitatory and inhibitory populations receive both types of excitatory synaptic inputs and the model is given by

$$\begin{array}{ccc}{\tau }_i{\stackrel{.}{r}}_i& =& -r_i+f_i\left(J_{iE}^N{s}_{iE}^N+J_{iE}^A{s}_{iE}^A-J_{iI}{s}_{iI}+J_{iO}i(t)\right)\\ {\tau }_{ij}^k{\stackrel{.}{s}}_{ij}^k& =& -s_{ij}^k+r_j\text{where}i,j=E\text{or}I,\text{and}k=N\text{or}A\end{array}.$$ (7)

Here, the superscripts N and A denote NMDA-type and AMPA-type synapses, respectively, and all other variables are the same as in Eq. (5). In Fig. 3a, the strengths of total excitatory synaptic currents and the fractions of NMDA-type synapses are represented by J_iE and q_iE; that is, $J_{iE}=J_{iE}^N+J_{iE}^A$ and $q_{iE}=J_{iE}^N/J_{iE}$ for i = E or I. In the purely negative-derivative feedback models of Fig. 4g–l, the network connectivity is tuned to have no net positive feedback by setting the strengths of positive and negative feedback to be precisely equal through the relation J_EE = J_EI J_IE/(1+J_II). On the other hand, in the hybrid models of Fig. 4m–r, excess positive feedback is tuned to precisely cancel the leakage by setting J_EE − J_EI J_IE/(1+J_II) = 1.

Throughout the paper, the intrinsic time constants of excitatory and inhibitory neurons, τ_E and τ_I, are set to 20ms and 10ms, respectively⁵¹. The time constants of GABA_A-type inhibitory synapses, τ_EI and τ_II, are set to 10ms^52–53. Based upon experimental measurements of excitatory synaptic currents in prefrontal cortex¹³, the time constants of excitatory synaptic currents and the fractions of NMDA-mediated synaptic currents are set as follows: in the networks with a mixture of NMDA- and AMPA-mediated excitatory currents (Figs. 3, 4), ${\tau }_{EE}^N=150\text{ms}$ and ${\tau }_{EE}^A=50\text{ms}$ in excitatory neurons, and ${\tau }_{IE}^N=45\text{ms}$ and ${\tau }_{IE}^A=20\text{ms}$ in inhibitory neurons. Note that these time constants reflect the kinetics of postsynaptic potentials observed to be triggered by activation of NMDA- or AMPA-type receptors, and likely include the effects of additional intrinsic ionic conductances since these experiments were performed without blocking intrinsic ionic currents¹³. The fractions of NMDA-mediated synaptic currents in excitatory neurons and inhibitory neurons, q_EE and q_IE, were set to 0.5 and 0.2, respectively. The time constants of excitatory synapses for networks with only a single type of synaptic current for each connection in Fig. 2 were set to τ_EE =100ms and τ_IE = 25ms in order to satisfy the average excitatory kinetics ${\tau }_{EE}=q_{EE}{\tau }_{EE}^N+(1-q_{EE}){\tau }_{EE}^A$ and ${\tau }_{IE}=q_{IE}{\tau }_{IE}^N+(1-q_{IE}){\tau }_{IE}^A$. Note that, since τ_EE > τ_IE, this provides slower positive than negative feedback (see Eq. (4)). The synaptic strengths J_ij were set to satisfy the balance conditions given by Eq. (3) and Supplementary modeling 4.

We note that the model presented here can similarly be extended to include both fast (GABA_A) and slow (GABA_B) components of synaptic transmission. In this case, the conditions for negative-derivative feedback have the same form as considered previously, but with replacement of τ_II and τ_EI by ${\tau }_{II}=q_{II}{\tau }_{II}^{GB}+(1-q_{II}){\tau }_{II}^{GA}$ and ${\tau }_{EI}=q_{EI}{\tau }_{EI}^{GB}+\left(1-q_{EI}\right){\tau }_{EI}^{GA}$, where the superscripts GA and GB denote the fast (GABA_A) and slow (GABA_B) components and q_EI and q_II denote the proportion of GABA_B currents. Supplementary Figure S3 shows an example simulation with inclusion of such a slow, inhibitory component of synaptic transmission.

Firing rate model of two competing populations

In Figure 6, we compare networks of competing populations utilizing positive feedback control versus negative-derivative feedback control. The connectivity between populations varies in different models but the dynamics of the firing rates and the synapses are the same as in Eq. (5),

$$\begin{array}{l}{\tau }_i{\stackrel{.}{r}}_i=-r_i+f_i\left(\sum _j{J}_{ij}{s}_{ij}+J_{iO}i(t)+J_{i,\mathit{tonic}}\right)\\ {\tau }_{ij}{\stackrel{.}{s}}_{ij}=-s_{ij}+r_j\text{where}i,j=E_1,I_1,E_2,\text{or}{I}_2.\end{array}$$ (8)

Here, E and I stand for excitatory and inhibitory populations, respectively, and the subscript 1 or 2 is the index of the population. The temporal component of i(t) is the same transient pulse-like input as in the firing rate model of Eq. (5) and J_i,tonic is the strength of the tonic input.

In the positive feedback network with direct mutual inhibition (Fig. 6a, d, g, j), population E_i receives recurrent excitatory input J_EiEis_EiEi and inhibitory input J_EiIis_EiIi from the same population, and external inputs J_EiOi(t) and J_{Ei, tonic}. The inhibitory sub-population I_i, for i = 1 or 2, receives only the excitatory inputs J_IiEjs_IiEjfrom the opposing population (j = 2 or 1, respectively).

The positive feedback network with a common inhibitory pool (Fig. 6b, e, h, k) is composed of three populations - two excitatory populations E₁ and E₂, and the common inhibitory population I. E_i receives recurrent excitatory input J_EiEis_EiEi from itself, inhibitory input J_EiIs_EiI, and external inputs J_EiOi(t) and J_{Ei, tonic}. The common inhibitory population I receives input J_IE1s_IE1 from E₁ and input J_IE2s_IE2from E₂.

In the negative-derivative feedback model with two competing populations (Figs. 6c, f, i, l and S6, S7e–h), each population has the same structure as in the single population in Eq. (5). Connections between the two competing populations are mediated by projections from the excitatory cells of each population that project weakly onto excitatory cells of the opposing population and more strongly onto inhibitory cells of the opposing population. Thus, the excitatory sub-population E_i receives inputs J_EiEis_EiEi and J_EiIis_EiIi from the same side, J_EiEjs_EiEj from the opposite side, and external inputs J_EiOi(t) and J_{Ei, tonic}. Similarly, the inhibitory sub-population I_i receives inputs J_IiEis_IiEi and J_IiIis_IiIi from the same side, and J_IiEjs_IiEjfrom the opposite side.

The intrinsic time constants of excitatory and inhibitory neurons and the synaptic time constants are the same as in the single population and the remaining parameters are given in Supplementary modeling 4. All the simulations of the firing rate models were run with a 4^th-order explicit Runge-Kutta method using the function ode45 in MATLAB.

Spiking network model with leaky integrate-and-fire neurons

In Figure 5 and Supplementary Fig. S4, we constructed a recurrent network of excitatory and inhibitory populations of spiking neurons with balanced excitation and inhibition. We showed that this spiking network maintains graded levels of persistent activity with temporally irregular firing. Here, we describe the dynamics of individual neuron activity and the synaptic currents connecting the neurons.

The spiking network consists of N_E excitatory and N_I inhibitory current-based leaky integrate-and-fire neurons that emit a spike when a threshold is reached and then return to a reset potential after a refractory period. These neurons are recurrently connected to each other and receive transient stimuli from an external population of N_O neurons (Fig. 5b, external population not shown). The connectivity between neurons is sparse and random with connection probability p so that, on average, each neuron receives N_Ep, N_Ip and N_Op synaptic inputs from the excitatory, inhibitory, and external populations, respectively.

The dynamics of the sub-threshold membrane potential V of the l^th neuron in population i, and the dynamics of the synaptic variable $s_{ij}^{lm,k}$ onto this neuron from the m^th neuron in population j are given as follows:

$${\tau }_i\frac{{dV}_i^l}{dt}=-\left(V_i^l-V_L\right)+\sum _m{\stackrel{\sim }{J}}_{iE}{p}_{iE}^{lm}\left(q_{iE}^N{s}_{iE}^{lm,N}(t)+q_{iE}^A{s}_{iE}^{lm,A}(t)\right)-\sum _m{\stackrel{\sim }{j}}_{iI}{p}_{iI}^{lm}{s}_{iI}^{lm}(t)+\sum _m{\stackrel{\sim }{J}}_{iO}{p}_{iO}^{lm}{s}_{iO}^{lm}(t)$$ (9)

$${\tau }_{ij}^k\frac{{ds}_{ij}^{lm,k}}{dt}=-s_{ij}^{lm,k}+\sum _{t_j^m}\delta (t-t_j^m),\text{for}j=E,I,\text{or}O\&k=N,\text{or}A.$$ (10)

The first term on the right-hand side in Eq. (9) corresponds to a neuronal intrinsic leak process such that, without the input, the voltage decays to the resting potential V_L with time constant τ_i. The second term is the sum of the recurrent NMDA- and AMPA-mediated excitatory synaptic currents as in Eq. (7). The dynamic variables $s_{iE}^{lm,N}$ and $s_{iE}^{lm,A}$ represent NMDA- and AMPA-mediated synaptic currents from cell m of the excitatory population. The sum of the strengths of NMDA- and AMPA-mediated synaptic currents, and the fractions of NMDA- and AMPA-mediated currents, are assumed to be uniform across the population and are denoted by J̃_iE, $q_{iE}^N$ and $q_{iE}^A=1-q_{iE}^N$, respectively. $p_{iE}^{lm}$ is a binary random variable with probability p representing the random connectivity between neurons. Similarly, the third and fourth terms represent the total synaptic inputs from the inhibitory population and the external population. As for the excitatory currents, the dynamic variables $s_{iI}^{lm}$ and $s_{iO}^{lm}$ denote the synaptic currents with strengths J̃_iI and J̃_iO, respectively, and $p_{iI}^{lm}$ and $p_{iO}^{lm}$ are binary random variables with probability p.

In the dynamics of $s_{ij}^{lm,k}$ in Eq. (10), a presynaptic spike at time $t_j^m$ from neuron m in population j causes a discrete jump in synaptic current followed by an exponential decay with time constant ${\tau }_{ij}^k$. Here, the spikes in the external population, representing inputs to be remembered, are generated by a Poisson process with rate r_O during a time window t_window (Fig. 5, with r_O =0 during the memory period) or with rate r_O after t=0 (Supplementary Fig. S4). Note that the strength of $s_{ij}^{lm,j}$, denoted by J̃_ij in Eq. (9), corresponds to the integrated area under a single postsynaptic potential, not the height of a single postsynaptic potential. Furthermore, the connectivity strengths J̃_ij were scaled as

$${\stackrel{\sim }{J}}_{ij}={\widehat{J}}_{ij}/\sqrt{N_{j}p}\text{for}\text{fixed}{\widehat{J}}_{ij}.$$ (11)

This scaling made the fluctuations in the input remain of the same order of magnitude as the mean input as the network size varied³⁰.

In Fig. 5l–n, the coefficients of variation of the inter-spike intervals were computed for 3 seconds from time 300 ms to 3300 ms using all excitatory neurons that exhibited more than 5 spikes during this period.

In the simulation, N_E = 16000, N_I = 4000, N_O = 20000, and p = 0.1. The time constants and the fractions of NMDA-mediated currents were the same as in the firing rate models: τ_E = 20ms, τ_I =10ms, τ_EI =τ_II =10ms, ${\tau }_{EE}^N=150\text{ms},{\tau }_{EE}^A=50\text{ms},{\tau }_{IE}^N=45\text{ms},{\tau }_{IE}^A=20\text{ms},q_{EE}^N=0.5$, and $q_{IE}^N=0.2$. The parameters for the synaptic strengths were tuned to achieve a balance between excitatory and inhibitory inputs during sustained activity, as shown in Supplementary modeling 3. The remaining parameters are given in Supplementary modeling 4.

The numerical integration of the network simulations was performed using the second-order Runge-Kutta algorithm. Spike times were approximated by linear interpolation, which maintains the second-order nature of the algorithm⁵⁴.