Abstract
Localized persistent cortical neural activity is a validated neural substrate of parametric working memory. Such activity “bumps” represent the continuous location of a cue over several seconds. Pyramidal (excitatory ()) and interneuronal (inhibitory ()) subpopulations exhibit tuned bumps of activity, linking neural dynamics to behavioral inaccuracies observed in memory recall. However, many bump attractor models collapse these subpopulations into a single joint /(lateral inhibitory) population and do not consider the role of interpopulation neural architecture and noise correlations. Both factors have a high potential to impinge upon the stochastic dynamics of these bumps, ultimately shaping behavioral response variance. In our study, we consider a neural field model with separate / populations and leverage asymptotic analysis to derive a nonlinear Langevin system describing / bump interactions. While the bump attracts the bump, the bump stabilizes but can also repel the bump, which can result in prolonged relaxation dynamics when both bumps are perturbed. Furthermore, the structure of noise correlations within and between subpopulations strongly shapes the variance in bump position. Surprisingly, higher interpopulation correlations reduce variance.
Keywords: neural fields, bump attractor, wave stability, stochastic differential equations, perturbation theory
1. Introduction.
Storing information “in mind” for short periods of time is essential for the performance of daily tasks [24]. Parametric working memory (as used in tasks requiring a delayed estimate of a continuous quantity) uses spatially localized persistent neural activity in the prefrontal cortex, parietal cortex, and frontal eye fields [16, 24] generated in neural circuits with strong local recurrent excitation and broad inhibition [9, 31, 44]. Neurophysiological recordings from nonhuman primate subjects performing visuospatial working memory tasks have shown that localized bumps of persistent activity encode remembered parametric quantities for a few seconds [9, 31, 21, 24]. For example, in the oculomotor delayed response task, a location cue is momentarily presented on a circle displayed on a monitor, then a delay period occurs during which the video is blank, and finally the subject is prompted to report their memory of the cued location. During the delay period, neural recordings identify cells tuned to specific cue locations around the circle and reveal that the strongest (peak) firing neurons represent the encoded location [24, 21, 22]. Peaked and localized activity wanders stochastically during the delay period, representing a time-dependent degradation of cue location memory consistent with subsequent response errors [8, 48].
Neural field and spiking models are capable of producing peaked, localized activity that wanders via spatially structured connectivity that weakens as the distance between neural cue location preference increases with excitation having a narrower spatial footprint [47, 27, 44, 31]. Bumps can be defined as standing pulse solutions usually with two key dynamical properties important for encoding memories of continuum variables: (1) they are self-sustained in the absence of stimulus (representing memory over a delay period) and (2) they exhibit marginal stability such that they can occur at any location in the network and integrate translational perturbations [27]. Such solutions have garnered interest due to the rich dynamical features that arise when varying the form of connectivity, introducing propagation delays, adding slow negative feedback, or considering stochasticity [4,27,31,12,19]. Response statistics and neural activity during oculomotor delayed response tasks are also well characterized by the output of bump attractor models with some form of stochasticity incorporated [48,9,3].
Previous psychophysical studies of delayed estimation of continuous quantities show subject errors scale linearly with the delay period [43, 46]. Such behavior can be accounted for by models whose low-dimensional dynamics evolve as Brownian motion, like a particle subject to diffusion. This is consistent with a bump attractor stochastically perturbed by noise, wandering along a marginally stable ring attractor [31, 8]. Extended models have also considered neural circuit mechanisms that help stabilize the movement of bumps to noise perturbations by breaking the marginal stability of the ring attractor. For example, spatially heterogeneous recurrent excitation leads to low-dimensional dynamics akin to a particle stochastically perturbed along a washboard potential, slowing the rate of diffusion [32]. Likewise, short-term facilitation locally potentiates synaptic excitation where the bump is istantiated, akin to a slowly moving local potential well that traps the particle near the true location of the original stimulus [44, 30]. In addition to stabilizing bumps within trials, short-term facilitation also transfers memory of the previous trial stimulus to the next, creating systematic errors referred to as serial dependence [35,7].
Despite advancements in understanding how these modifications to bump attractor models affect their stochastic dynamics, a detailed examination of the role of separate excitatory and inhibitory ( and ) population dynamics has been overlooked, and - interactions are often ignored entirely. Inhibition is often assumed to be flat and global in stochastic bump attractor models [9], but we know prefrontal cortical synaptic inhibition exhibits nontrivial preference-dependent interactions [24], which could have yet unidentified effects on the dynamics of bumps and the fidelity of memory systems that rely on them.
Here, we use a stochastic neural field model to investigate the wandering dynamics and interactions of separate and activity bumps. We focus on how a separate and spatially extended population shapes the variance of a bump attractor encoding a remembered location along a continuum. First, we introduce the neural field equations, notation conventions, and parameters whose effects on the variance of the bumps’ final position will be analyzed (section 2). We then review the existence and stability of both and bump solutions [47, 27, 4, 14, 37, 20], paving the way for examining how deterministic and stochastic perturbations deform and shift the attractor solution. We examine the impact of modifying both the amplitude and spatial extent of neural architecture, as well as the firing rate thresholds on the stability of solutions (section 3). Stable and unstable branches of bumps annihilate at a discontinuous saddle node bifurcation given sufficiently high firing rate thresholds.
Our stochastic analysis involves two different methods of identifying the impact of noise on the bump attractor: (1) a linear perturbative analysis (strongly coupled limit between the and bumps) in the limit of weak noise and (2) an interface-based analysis where we track the threshold crossing points of the / bumps (section 4). Thus, we develop two corresponding theoretical predictions of bump position variance, the second of which captures nonlinear interactions between the / bumps in a Langevin system that is a better match to full model simulations. This higher-order analysis starts by obtaining distinct and nonlinearly coupled equations for the bump interfaces. Bump position variance changes non-monotonically as a function of most model parameters, so intermediate tuning of network architecture maximizes memory degradation. Finally, we identify the impact of noise correlations between the and populations, finding they actually serve to reduce bump position variance.
2. The model.
To determine the effects of separately evolving and populations on bump attractor dynamics, we analyze a stochastic neural field model: a coupled system of nonlinear integro-differential equations describing interactions of spatially extended and neural subpopulations (Figure 1(a) and [47]). Recurrent connectivity targeting position from at time is described by convolving the synaptic kernel and the firing rate output of the neurons from which synapses originate. Along with additive noise terms, we obtain the stochastic neural field equations:
(2.1a) |
(2.1b) |
where and denote the and synaptic input profiles at location at time . We assume each unit of the time variable is equivalent to 10 ms on the order of typical membrane and synaptic time constants [25]. The sigmoidal firing rate function with threshold and gain determines the strength of the firing rate output based on the input . Analytical results can be obtained in the high gain limit such that
(2.2) |
is the Heaviside function. The and population firing rate thresholds, and , can differ. Given a Heaviside firing rate nonlinearity, it is possible to exactly characterize the dynamics of the model using interface equations that track the evolution of level sets and in space and time [14,18]. The strength of synaptic connectivity weakens with the spatial distance between positions and and is given by the distance-dependent synaptic weight profile functions . For explicit calculations, we take these to be symmetric exponentials
(2.3) |
where and (nonnegative constants). Commonly used values throughout for the weight profiles are presented in Table 1. Parameters for were chosen to non-dimensionalize the to connectivity (setting and ). Other parameters for the to and cross-population synaptic profiles were generally chosen to be broader than the synaptic footprint of to connections, and we generally assume the commonly identified 80% and 20% neuron ratio [1] in turn approximately determines the effective synaptic amplitude from those populations.
Table 1.
Parameter | ||||||
Definition | - strength | - strength | - strength | - spatial scale | Other spatial scales | time constant |
Value | 0.5 | 0.15 | 0 or 0.01 | 1 | 2 | 1 |
Parameter | ||||||
Definition | firing threshold | firing threshold | Noise amplitude | |||
Value | [0,0.5] | [0,0.5] | 0.001 or 0.002 |
The spatially structured multiplicative noise terms and are introduced with small amplitude , allowing asymptotic approximations of their stochastic effects. Multiplicative noise is predicted in neural fields by linearly approximating noise arising in a system-size expansion of a fully stochastic model [5]. Increments of spatially extended Wiener processes have zero mean and local spatial correlations . For simplicity, we start by assuming there are no correlations between the noise to the and populations
Based on prior studies of deterministic / population models [42, 4, 8] bump (standing pulse) solutions emerge in parameter regions we can identify using an existence/stability analysis. Recurrent excitation sustains both the and populations, and inhibition prevents the spread of the population activity. In the absence of noise, we obtain bump profiles that are even symmetric and translation symmetric. Using threshold crossing conditions, we develop implicit formulas for the half-width variable for the () bump, which ensure existence, defined as the distance from the bump’s center of mass to the interface (Figure 1(b)). The I bump’s half-width (Figure 1(c)) can vary nonmonotonically along certain parameter axes, such as increasing firing rate thresholds, and , simultaneously. When noise is applied to stable / activity bumps, they “wander” due to their neutral stability. In addition to small deformations of the bump profile itself, a bump’s center of mass wanders stochastically (Figure 1(d), and see Figure A.1 for an additional example). While past asymptotic analyses assumed this wandering was roughly Brownian motion [8,31], we will show that by relaxing this assumption we can better characterize nonlinear interactions between the bumps in the distinct and populations and how this shapes wandering dynamics.
Each bump has a region over which neural activity ( or ) is superthreshold (above or . We define the corresponding / population active regions to be and , respectively [27]. These active regions are bounded by the interfaces (threshold crossings) since and . Note, large deviations could lead to multiple disjoint active region segments in each layer, but we assume each bump’s active region is fully connected here (see [14,18,36] for elaborations on this problem). In the absence of noise, the interfaces relate directly to the half-widths in that and
Stochastic motion of the bumps will be tracked by estimating the center of mass of the active region of the bump
(2.4) |
Overall, we are interested in both how network parameters shape the form and stability of bumps and how this translates into the bump’s stochastic dynamics in the presence of noise. Our ensuing analysis will initially rely on techniques in local stability, as well as weak perturbations, but a major advancement will be the use of interface techniques to provide higher-order corrections to the effective nonlinear equations describing stochastic bump motion.
3. Deterministic analysis.
Our estimates for expected bump wandering (variance of center of mass) utilizes linearization about stable stationary solutions. This necessitates an analysis of stationary bump solutions’ (static bump solutions in the absence of perturbations) existence and stability in varying parameter regimes for the deterministic system. Using (2.1) without noise [4,27], we demonstrate the existence of stationary solutions through direct construction. For proofs of uniqueness of solutions with a Heaviside firing rate see [4,10].
3.1. Stationary solutions.
Assuming solutions are stationary and and using the Heaviside firing rate function (2.2), (2.1) becomes
(3.1a) |
(3.1b) |
We seek bumps with simply connected active regions and and . Exploiting the solutions’ translational invariance along (opting for a center of mass at ) and expected evenness (i.e., and ) we obtain the system
(3.2a) |
(3.2b) |
Integrating our chosen exponential synaptic weight functions , we find the explicit formulas for :
(3.3) |
Substituting (3.3) into (3.2) and utilizing the threshold crossing conditions and , we obtain an implicit set of equations for the half-widths, which depends on whether the or bump is wider:
(3.4a) |
(3.4b) |
Equation (3.4) is thus defined piecewise continuously (Figure 2(a), gold circle indicates where the change in cases occurs), and as we will show, cusps appear at these case switches in plots of eigenvalues and variance estimates.
Note, the above set of threshold conditions assume that both the and populations have nontrivial active regions. There is a second class of bump solutions where there is no active region in the population. As we will show, these bumps always tend to be linearly unstable, as there is no active inhibition to prevent the spread of excitation upon perturbation. Applying this assumption to (3.2), we obtain the simplified system,
(3.5a) |
(3.5b) |
with E bump half-width given by
or (for our particular weight functions) the formula
(3.6) |
For such a solution to be self-consistent, we also must ensure that everywhere. Assuming the peak of the subthreshold bump occurs at , this is ensured by the condition
There are, thus, two branches of bump solutions. One branch of “broad” bumps has superthreshold active regions in both the and populations and . For sufficiently high thresholds, we typically obtain marginally stable solutions, but these destabilize through a Hopf bifurcation for sufficiently low firing rate thresholds (see Figure 2(a) and subsequent stability analysis). In contrast, when the bump is subthreshold , we obtain unstable “narrow” solutions that create a separatrix between the broad solutions and the rest state (Figure 2(a)). Under each set of assumptions we obtain two distinct systems that approach the same solution defining a discontinuous saddle node bifurcation. The discontinuity arises from there being either two or one threshold condition and the peak of the narrow bump grazing the threshold . We derive these results in detail in the following subsection.
3.2. Eigenvalues and noiseless perturbations.
Our stability analysis utilizes linearization about stationary solutions and localizes the perturbation evolution problem to the bump edges, as in several previous analyses of bump dynamics in neural fields [4, 12, 19, 42, 27]. Since there are four threshold crossing points, analysis of the broad solution’s stability results in four corresponding eigenvalues and equations associated with the degrees of freedom in the stability problem (Figure 2(b)). To derive our linearized system whose spectrum defines the stability of bumps, we start by perturbing with small smooth functions, and . Substituting into (3.1), expanding about the stationary solution, and simplifying to first order we obtain the system
(3.7a) |
(3.7b) |
where the distributional derivatives are given:
Assuming separability of our perturbations, and , and substituting our exponential weight functions and and , we obtain the corresponding eigenvalue problem:
(3.8a) |
(3.8b) |
Note, in the above, we have assumed and to obtain an equation for the point spectrum, but taking these values to vanish would give us an equation for the essential spectrum which will not contribute to instabilities [13, 45]. We form a system of equations by evaluating the perturbation functions at the bump edges and in (3.8). Values for which the resulting 4×4 system is singular correspond to the four distinct eigenvalues in the point spectrum. Forming this system, taking the determinant, and factoring, we find the eigenvalues are the roots of the following pair of quadratics:
(3.9a) |
xs
(3.9b) |
where
Each quadratic could also be obtained by restricting the form of perturbation (scaling and shifting) at the interfaces (see e.g., Figure 2(d)). Scaling perturbations expand or contract the bump and , and we obtain (3.9a) resulting in the associated “scale” eigenvalues (Figure 2(b))
(3.10) |
which are both nonzero in general. The sign of their real part determines the linear stability of the bumps. For shifting perturbations and we obtain (3.9b). A straightforward calculation shows , resulting in the zero eigenvalue (green) corresponding to the system’s translational invariance (Figure 2(b)). Correlated shifts in the / bumps’ center of mass simply translate the solution along the real line. Such perturbations are integrated and neither grow nor decay. The nonzero shift eigenvalue is given by the simple formula
(3.11) |
which is real and negative. Hence the bumps are stable with respect to shifts of the and bump in the opposite direction.
Eigenvalues of the narrow solutions are obtained by following the same process, which is greatly simplified since the population is subthreshold and perturbations of this part do not contribute to instabilities or enter into the point spectrum equations. We therefore investigate the effect of small smooth perturbations to the population, with the nonzero part along the bump edges, . The resulting two eigenvalues are (corresponding to the marginal stability of shifts) and , which is clearly positive, confirming that narrow solutions are always unstable (Figure 2(c)).
We identify bifurcations in the bump solutions by checking the signs of the real parts of each solution branch’s eigenvalues. We find two types of bifurcations. The first is a Hopf bifurcation (green dot, HB in Figure 2(a,b)) occurring as the firing threshold is reduced, destabilizing bumps in a pattern-destroying oscillation (Figure 2(e)) [42]. This Hopf bifurcation boundary occurs when , which is given by the implicit formula
(3.12) |
The second bifurcation observed is a discontinuous saddle node bifurcation (SN in Figure 2(a)) [15,39] where the unstable narrow and marginally stable broad solutions meet. The peak of the narrow population bump rises to meet the firing threshold, and the broad bump active region shrinks to a single point at threshold. The meeting of two linearizations of different discrete dimension (4 for broad, 2 for narrow) results in a discontinuous change in the corresponding Jacobian matrices yielding a nonsmooth saddle node bifurcation where the solution branches annihilate one another (labeled “contraction instability” in Figure 2(f)).
3.3. Bounding stability regions in parameter space.
To identify how noise degrades stable representations of parametric working memory, we focus on the effect of stochastic perturbations on bumps along the broad branch. Given that varying certain parameters can stabilize the deterministic version of (2.1), we identify how the bifurcation boundaries change and stable solution regions expand/contract as parameters are varied (Figure 3). As discussed above, stable solutions only exist if broad bumps have nonpositive eigenvalues associated with their linear stability.
When varying the firing rate thresholds, we find larger thresholds, , expand the range of stable solutions (Figure 3(a)). We speculate that setting the population firing rate threshold too low makes it strongly responsive to network activity perturbations, as shown in oscillatory instability simulations (Figure 2(e)). Varying the strengths of synapses from to and to populations, we also observed bumps destabilize if to coupling is sufficiently strong (Figure 3(b)). Such changes increase the sensitivity of the drive to the population as bumps are expanded, leading to potential bump overcompensation as bumps widen, eventually silencing both bumps.
Setting firing rate thresholds equal and varying them along with changing the timescale, , result in a narrowing of the range of stable bumps as firing rate thresholds are decreased (Figure 3(c)). Specifically, as the timescale of the population is increased, the system reacts more slowly to being out of equilibrium, so the and bumps do not restabilize once perturbed. For instantaneous inhibition , such oscillatory (Hopf) instabilities never occur.
We also quantified the parameter ranges of stable bumps when varying the spatial scale of interpopulation synaptic footprints (Figure 3(d)). Stability is again largely dependent on the profile of to population synaptic connectivity; wider connectivity (high ) yields an unstable region. Similar to our previous findings, broadening the to profile leads to overcompensation of the population in response to dynamical increases in the population width.
The main panels of Figure 3 assume , which is common in studies of (2.1) due to the small amplitude of such connections. We also examined small, nonzero amplitude to connections , see insets in Figure 3) and found stable regions expand likely due to dampening of I population reactions.
4. Analysis of stochastic bump motion.
Considering stochasticity emerging from neural and synaptic variability [17], multiplicative noise in the neural field model (2.1) (taking ) causes stable bumps to wander like a particle subject to Brownian motion. The extent of this wandering has been linked to subject response errors on delayed estimation tasks [48, 8]. We are primarily interested in identifying how architectural features of the / neural circuit impact wandering [31, 30, 44, 32] as described by the time-dependent variance of the bumps’ center of mass. We can estimate this variance analytically using two different approaches. The first we refer to as the “strongly coupled limit” approximation, motivated by work from [31], which assumes the / bump has a single position (the and bumps do not stray too far from one another). To first order, this approximation treats the bump as wandering by pure diffusion, and so its stochastic motion is only characterized by a diffusion coefficient. However, the resulting formulas have limitations for certain parameter regimes on the timescale of interest since they do not consider that the bumps can drift apart. While we typically do expect the bump positions to separate from one another, we assume this separation is small. As we will show when deriving a complementary interface approximation, this separation is substantial enough to significantly affect the variance. One exception would be the limit in which the population responds infinitely rapidly , and so its bump’s position precisely tracks that of the bump. Our second approach, the “interface-based approximation,” tracks bump interfaces (following threshold crossing points) [36, 14, 27]. Pairing this together with a weak coupling assumption as in [28] allows us to estimate the time-dependent changes in the distinct and bumps centers of mass.
4.1. Strongly coupled limit approximation.
In the strongly coupled limit, we assume solutions to (2.1) take the form of stable bumps with positions weakly perturbed by the same amount and with . Thus, the and bumps are assumed to move together, and we have
(4.1a) |
(4.2b) |
where is the position perturbation (with and are, respectively, the leading order profile perturbations; and and are, respectively, the higher-order profile perturbations to the and bumps. Substituting (4.1) and truncating (2.1) to first order and taking averages, we find that the stationary solutions remain the same as before. Moving to , we find
(4.2) |
where we define the linear operator
Note that, due to translation symmetry of the noise-free system, the null space of the linear operator ) (any vector of functions such that ) is spanned by . To briefly show this, recall the stationary solutions take the form
Differentiating with respect to and applying integration by parts yield
exactly the form of the functions and spanning the
We ensure a bounded solution to (4.2) by requiring the inhomogeneous part to be orthogonal to the null space of the adjoint of the linear operator . There exists a single vector spanning , denoted . Enforcing our conditions for bounded solutions via requiring the inner product of the nullspace and inhomogeneity vanishes ; we isolate the bump position increment:
Since the above is simply a weighted integral over the spatiotemporal noises, to first order, we have a Brownian stochastic differential equation (SDE) for the bump position. Given noise is white in time, we find that the mean over realizations is , and the variance is
(4.3) |
where
and the spatial correlation functions are defined: , and . Note, there is a weighted contribution to the linear scaling variance from both the noise of the and populations.
To calculate the nullspace of the adjoint linear operator , we must first derive the adjoint using the definition based on the inner product , by which we find
We rearrange the equation , so
(4.4a) |
(4.4b) |
In the case of the Heaviside firing rate function we again can determine the distributional derivative of the Heaviside acting on the bump solution and obtain
(4.5) |
Plugging (4.5) into (4.4) and solving for the constants, we find , and . Thus we find that
(4.6) |
Finally, substituting (4.6) into (4.3), we have
(4.7) |
where
The contributions arising due to noise correlations within the and populations, and , are positive. In contrast, the diffusion contribution from the cross-population correlation term is negative assuming the correlation function is monotone decreasing. Thus, noise correlations between the and populations reduce wandering. Here, we assume Later, we discuss the effects of nonzero correlated noise components across the and populations and compare these theoretical results to numerical simulations.
4.2. Interface-based approximation theory.
Complementing our strongly coupled limit approximation of bump wandering, we also derive interface equations that approximate the stochastic dynamics of the bump in response to multiplicative noise in (2.1). While the strongly coupled limit approximation assumes the and bumps move as one, numerical simulations reveal that this is not the case (see Figure 4(a,c)). For instance, the bump tends to be more susceptible to profile deformations and may wander more, making the strongly coupled limit less valid. Higher-order corrections of the variance estimate can be obtained using interface theory to develop nonlinearly coupled Langevin equations for estimating the and bumps’ coupled centers of mass.
Interface theory for bumps in neural fields tracks the level sets of neural activity variables where firing rate thresholds are crossed. This approach was originally pioneered in the case of noiseless single-bump neural field models [27] and then subsequently for traveling fronts in inhomogeneous neural fields [11], as well as solutions in planar neural fields [14]. More recently, these approaches were adapted to obtain higher-order approximations for the timescale of front initiation [18], as well as the stochastic motion of multiple interacting bumps [36]. Here, we further adapt this work [36] to account for the motion of separate bumps in the and neural populations of (2.1).
As defined previously, the active regions of the and bumps are and , respectively. Without noise perturbations, we would expect these interfaces to remain constant for the equilibrium bump solution, but noise perturbs these values, so they wander over time. However, unlike in the strongly coupled limit, we do not expect this wandering to be pure Brownian motion but rather the result of a nonlinearly coupled system of SDEs. To obtain these SDEs, we start by writing the level set condition for the interfaces in each neural population ( and ):
(4.8) |
where and are the firing rate thresholds of the and populations. For a Heaviside firing rate, we substitute (4.8) into (2.1) and obtain
(4.9a) |
(4.9b) |
Differentiating (4.8), we obtain consistency equations for the interfaces :
We then approximate the above exact evolution equations by assuming the spatial gradients at the interfaces remain constant and odd symmetric throughout the evolution motivated by the form of the strongly coupled limit expansion:
Subsequently, we drop terms to obtain the relations
(4.10a) |
(4.10b) |
(4.10c) |
(4.10d) |
Plugging the formulas in (4.10) into (4.9), we obtain the following system of nonlinear Langevin equations describing the stochastic evolution of the interfaces:
(4.11a) |
(4.11b) |
(4.11c) |
(4.11d) |
where the coupling functions are given by the integrals over the active regions
To derive estimates for the evolution of the centers of mass of the and bumps, we apply the definitions and from (2.4) to (4.11) and combine equations to obtain
(4.12a) |
(4.12b) |
Assuming position perturbations remain small we approximate , and with . Plugging in these approximations, linearizing about the stationary solutions, and simplifying yield the multivariate OrnsteinUhlenbeck (OU) process:
(4.13a) |
(4.13b) |
Note, and are even and monotonic in distance, so we define the positive quantities
Then we have the matrix-vector representation of the multivariate OU process, , where is the coupling matrix, and
is the correlated Wiener process noise. Following methods for solving linear SDEs [23], we can determine the mean and variance of and to obtain estimates of the diffusion of each population. First we diagonalize :
Thus our eigenvalues are and the eigenvectors are and respectively. Note, this implies that there is a marginally stable direction along perturbations of the bump that move both the and bumps the same amount, and there is an attractive (stable) direction for perturbations that move the and bumps differently. Similar results have been found for coupled lateral layers for which each layer individually supports a self-sustaining bump in the absence of cross-population coupling [20, 28, 6]. The mean is given by and so
Next we seek the covariance , where the covariance matrix of the noise term is found to be
(4.14) |
with
Multiplying out and then integrating yield our predictions of and bump center of mass variance:
(4.15a) |
(4.15b) |
In the limit as we find that both variances are dominated by the term
which is essentially an estimate of the variance derived from assuming the and bumps are co-located as in the strongly coupled limit approximation. Relating these two routes to one another, we find
Using this relation and plugging in the expressions for , and we find that we obtain the diffusion coefficient expression from the strongly coupled limit prediction (4.7). At short times, the interface-based approximation (4.15), has contributions based on the interactions of the and bumps, which decay over time.
The main difference between the strongly coupled limit and interface-based methods is that the bumps are allowed to drift apart in the interface-based method, which allows us to separately estimate the bump’s variance. The most general form of the approximation can further describe stochastic widening and contraction of the bumps, described by a full fourdimensional and nonlinear approximation. Even collapsing to centers of mass approximations, using the interface-based interactions as a starting point is more accurate than the strongly coupled limit, as we will subsequently show. On the other hand, the strongly coupled limit is more straightforward to obtain, and there are fewer arbitrary assumptions we must make (e.g., the static gradient approximation). Yet, the interface-based method allows us to obtain greater precision by using the fully nonlinear approximation over all the interfaces, , making it a better metric for variance predictions.
4.3. Variance predictions and simulations comparison.
To validate our strongly coupled limit and interface-based predictions of bump variance, we ran stochastic simulations of (2.1) using spatiotemporal noises to the and populations that are not correlated between populations; . As firing rate thresholds are increased (Figure 4(a,b)), both our theoretical predictions and the averaged numerical simulations suggest that variance changes nonmonotonically. This stands in stark contrast to results from previous studies, which found that the effective diffusion of bumps generally tends to increase monotonically with the firing threshold for single-population lateral networks [31]. Interestingly, we find that the variance peaks at the precise point in parameter space where the noise-free and bumps have the same width (gold circles, Figure 4(a,b)).
Moreover, we see that the strongly coupled limit approximation adequately captures the effective motion of the bump but not the bump, which tends to stray further (Figure 4(a)). In contrast, the interface-based approximation is able to capture both the distinct and bumps’ variance by accounting for the stochastic dynamics of the bump being perturbed away from the bumps center of mass (Figure 4(b)).
The dynamics of the bump vary with firing rate thresholds and timescales. At higher firing rate thresholds, the bump wanders more than the bump, and the two bumps tend to be more weakly coupled. As the timescale increases, the bumps center of mass relaxes to wander slightly away from that of the bump, while the E bumps variance scales linearly in time like pure diffusion (Figure 4(c,d)). Since the bump is sustained by the population, the bump appears to weakly track the bump’s position (see Figure A.1 for single simulation of such activity), which can be estimated by OU processes [31,6]. Hence, the bump’s position variance can be higher than that of the population.
To further analyze how the variance in bump position depends on changes in parameters, we determined how it changes along several different parameter axes (Figure 5). For simplicity, we studied the variations in the bump’s position variance and compared it to the interfacebased approximation. We started by varying the amplitude of interpopulation connectivity, either or . Weakening either of these projection amplitudes tended to increase bump variance (Figure 5(a,b)). Weakening cross-population connectivity leads to less common movement of the / bumps. Thus, the bumps are more weakly stable to noise perturbations and reequilibrate more slowly, ultimately leading to more wandering.
Aside from nonmonotonicity arising as a function of the firing rate threshold, we also observe that narrowing the bump’s synaptic profile (varying the spatial extent of the I projections) yields a peak in variance (Figure 5(c)) which may be due to a greater susceptibility to noise perturbations when the width of to projections is decreased, resulting in a peak of variance. For smaller , we speculate that the equilibrium bump width is narrower and can be more easily stabilized by feedback. Note that the simulations do not match the peaks as well as in other panels likely due to approximations made in the interface-based approach, though the trends are still largely captured.
We also observe that lower firing rate thresholds lead to bumps that are more stabilized to noise perturbations (Figure 5(d)). However, the largest peaks in bump position variance occur where and bump half-widths are most similar, i.e., where () threshold crossing gradients are the lowest (highest) (Figure A.2).
These results assumed to connections are nonexistent . However, we found that even low amounts of connectivity, , decreased bump variance (Figure A.3).
4.4. Correlated versus uncorrelated noise.
The impact of noise correlations on neural circuit codes for delayed estimates is varied. Correlated noise within a neural subpopulation can improve working memory coding [38], but cross-population noise correlations can lead to an increase in bump attractor wandering that degrades memory [28]. We find in the subsequent investigation that cross-population noise correlations between and populations lead to less bump wandering than uncorrelated noise.
Our aim is to explore the system when there is cross-population noise (corresponding to the case where ). To obtain greater control over the extent of correlated noise we opt to express our spatiotemporal noise sources in the and populations as
(4.16) |
where the independent spatially correlated and temporally white noise process represents a correlated stochastic component in system (2.1). We define these three noise terms the same as before though now we have nonzero cross-population spatial correlation . The correlation parameter such that implies uncorrelated noise and fully correlated noise. Correlating noise across the and populations drastically alters our variance predictions (Figures 6(a) and A.4), with increased correlated noise decreasing the predicted and simulated bump variances (Figure 6(b)).
We analyze the response of the / bump structure to correlated as opposed to uncorrelated shifts. Correlated shifts translated both bumps centers due to translational invariance of the system (Figure 6(c)). Uncorrelated shifts move the and bumps in opposite directions but then show an attraction of the bump to the bump, while the bump is repulsed by the bump leading the bump to “catch up” to the shifted bump, and the bump moves further away from its original position (Figure 6(d)). Thus, uncorrelated shifts lead to additional drift of the bump and higher variance. This stands in stark contrast to the uncorrelated/correlated perturbation analysis carried out for two coupled identical lateral layers [28], in which opposing perturbations of two weakly connected bumps are effectively canceled by the attractive force between the bumps.
We also considered the effects of two small additive Gaussian inputs, more akin to the noisy kicks arising in stochastic simulations. When the position of these kicks was strongly correlated (Figure 6(e); i.e., the same application to each population), the and bumps stayed together and relaxed back to their original position. Uncorrelated kicks led to different perturbations in the position and profile of the and bumps (e.g., Figure 6(f)), with a relaxation where the bump was attracted the bump, but the bump was repelled by the bump.
Overall, we found that the separate / network model shows parametrically dependent bump wandering. First, bump position variance depends strongly on the relative widths of the and bumps, and relaxation dynamics from noise perturbations can cause the bump to stray from the bump. Second, nonmonotonic variance trends arise with respect to network connectivity amplitude and spatial scale parameters. Lastly, increases in interpopulation noise correlations reduce bump wandering by eliminating the relaxation effects that would otherwise extend the effects of stochastic perturbations on the and bumps.
5. Discussion.
Stochastic bump attractor models have become a useful tool for characterizing variability of the systems that code for memory-based estimates of continuous variables [8,48,3,34]. As we show here, it is important to appreciate the nuances of / architecture and noise correlations [24] when making predictions about continuous variable estimates. Our study has provided a suite of new predictions concerning how interpopulation connectivity amplitude and spatial profiles impact bump wandering. To obtain tractable expressions for bump position variance predictions, one form of our approximations relied upon the marginal stability of solutions to the noise-free system. Prior to performing our variance estimates, we observed that, along several parameter axes, we obtain nonmonotonic changes in half-widths and two types of instabilities: an oscillatory (Hopf) instability located in the red unstable regions and a contraction instability located at discontinuous saddle nodes (Figure 3). Partial versions of the results have been observed previously in / population models [4]. We also found that stability of solutions was significantly affected by nonzero synaptic strength (compare Figure 3 and insets). Thus, even “weak” connectivity can strongly affect the linearized dynamics of stationary bump solutions.
Our asymptotic analysis aimed at predicting the stochastic motion of bumps subject to noise moved beyond the standard single center-of-mass approximation often used to estimate the stochastic motion of bumps [37,8,31]. While a single center-of-mass approximation works reasonably well across some parts of parameter space, at high firing rate threshold, close to the discontinuous saddle node bifurcation, we find this breaks down and is better captured by a stochastic interface-based approximation previously developed in [36]. In particular, the bump diffuses more than the bump, which is well captured by the nonlinear Langevin approximation derived by approximating the motion of bump interfaces. The amplitude of bump wandering changes nonmonotonically in most network parameters, due to the change in bump half-width amplitudes, well captured by our interface-based approximation. Notably, interpopulation noise correlations reduced bump wandering. Uncoordinated and bump motion, arising in networks with uncorrelated interpopulation noise, leads to additional bump drift during relaxation periods while the bump “chased” the bump.
Note, our variance approximations are relatively accurate over the wide range of parameters we chose to analyze, but there are features of the full nonlinear system that can depart significantly from our basic assumptions, especially those involving linearizations. One rare but possible source of discrepancy between our asymptotic approximations and the full system could arise from the emergence of multiple distinct active regions in the and/or populations. Such a situation could emerge from (a) large and rare noise perturbations that activate a distinct region of the and/or population away from the bump and (b) large and rare noise perturbations that split bumps. While both are possible, they are extremely rare and so do not have a substantial impact on the numerically estimated variance. However, we could account for such splitting, nucleation, and annihilation events by extending our approximations to incorporate the appearance and merging of interfaces as in [18,36]. Moreover, we could account for such events in our numerical estimates with a more flexible definition of bumps which accounts for these transient events. Another approximation made in both the strong coupling limit and interface approximation is to assume the gradient of the activity variables at the interfaces is constant, though this is likely not true as shown in [18]. Nevertheless, we typically expect any deviation from this constant gradient value to be small and to not substantially impact the stochastic dynamics of the bump.
There are several other possible extensions of our analysis of bump stochastic motion in the / network. It is important to note that we made multiple approximations to collapse our stochastically evolving bump interface equations to a pair of SDEs describing the coupling between the and bump. Alternatively, we could have retained a higher-order approximation of the bump interface gradients in order to obtain a more accurate approximation [18]. Recall, the interface-based approximation begins as a fully nonlinear description and can be used to describe dynamics near the oscillatory (Hopf) bifurcation or the discontinuous saddle node. Alternatively, such near-bifurcation approximations could also be determined by choosing a scaling for a bifurcation parameter similar to the weak noise amplitude as in [33, 29]. Such approaches could also describe the stochastic dynamics of traveling pulses that emerge beyond bifurcations whereby bumps begin to drift at a constant speed due to the negative feedback brought about by the population.
Our analysis of bump position variance across multiple parametric axes helped identify a number of ways to reduce bump wandering via network architectural tuning. We largely chose parameters roughly assuming 80% and 20% neurons present in the prefrontal cortex [1], but we could certainly explore broader ranges of parameter space beyond this typical fraction. Another natural extension for this work would be to consider more complex mechanisms for synaptic tuning, such as short-term plasticity in the and populations, to reduce the propensity for bumps to wander. Recent studies in mean field reductions of spiking networks have demonstrated that short-term facilitation (depression) on the population with global inhibition tends to decrease (increase) bump drift and diffusion [44], ultimately improving parametric working memory. Extensions of our model and present analysis could be used to further investigate how the introduction of different forms of short-term plasticity into either the or population would impact bump wandering. Not only could short-term plasticity reduce the effect of stochastic perturbations on bumps but could also make them more robust to distraction inputs [40].
Finally, our investigation into the role of cross-population noise correlations raises questions regarding the coding advantages brought about by noise correlations. Noise correlations can increase, decrease, or not affect the amount of information encoded by a neural circuit [2]. Most of such results have been derived in network models devoid of spatial structure. However, recently work has demonstrated how disruptive broadly correlated spatiotemporal noise can be to information transmission in spatially organized neural circuits [26]. Our work adds to this ongoing line of inquiry by demonstrating variability-reducing mechanisms possible via increased correlation in noise between and populations. Our bump position variance predictions and model are sensitive to changes in the structure of noise. An improved understanding of the precise form and structure of noise in prefrontal cortex and other areas [41] could help further constrain neural circuit models of memory-encoding persistent activity. Mechanistic models that connect synaptic architecture, psychophysical performance, and stochastic and spatiotemporal dynamics, as well as the rich structure of internal and external noise, can help us further understand the dynamical principles underlying information coding in the brain.
Acknowledgment.
We thank Sage Shaw for assistance with Python code development.
Funding:
This work was funded by National Institutes of Health grants R01-MH115557-01 and R01-EB029847-01, as well as NSF grant DMS-1853630.
REFERENCES
- [1].Abeles M, Corticonics: Neural Circuits of the Cerebral Cortex, Cambridge University Press, London, 1991. [Google Scholar]
- [2].Averbeck BB, Latham PE, and Pouget A, Neural correlations, population coding and computation, Nat. Rev. Neurosci, 7 (2006), pp. 358–366. [DOI] [PubMed] [Google Scholar]
- [3].Barbosa J, Stein H, Martinez RL, Galan-Gadea A, Li S, Dalmau J, Adam K, Vallssolé J, Constantinidis C, and Compte A, Interplay between persistent activity and activity-silent dynamics in the prefrontal cortex underlies serial biases in working memory, Nat. Neurosci, 23 (2020), pp. 1016–1024. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [4].Blomquist P, Wyller J, and Einevoll GT, Localized activity patterns in two-population neuronal networks, Phys. D, 206 (2005), pp. 180–212, 10.1016/j.physd.2005.05.004. [DOI] [Google Scholar]
- [5].BRESSLOFF PC, Stochastic neural field theory and the system-size expansion, SIAM J. Appl. Math, 70 (2010), pp. 1488–1521 [Google Scholar]
- [6].Bressloff PC and Kilpatrick ZP, Nonlinear langevin equations for wandering patterns in stochastic neural fields, SIAM J. Appl. Dyn. Syst, 14 (2015), pp. 305–334. [Google Scholar]
- [7].Cicchini GM, Mikellidou K, and Burr DC, The functional role of serial dependence, Proc. Royal Soc. B, 285(2018),20181722. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [8].COMPTE A, BRUNEL N, GOLDMAN-RAKIC PS, and WAng X-J, Synaptic mechanisms and network dynamics underlying spatial working memory in a cortical network model, Cereb. Cortex, 10 (2000), pp. 910–923, 10.1093/cercor/10.9.910. [DOI] [PubMed] [Google Scholar]
- [9].Constantinidis C and Klingberg T, The neuroscience of working memory capacity and training, Nat. Rev, 206 (2016), pp. 1–12, 10.1038/nrn.2016.43. [DOI] [PubMed] [Google Scholar]
- [10].COOMBES S, BEIM GRABEN P, POTTHAST R, and WRIGHT J, Neural Fields: Theory and Applications, Springer, Berlin, 2014 [Google Scholar]
- [11].Coombes S and Laing C, Pulsating fronts in periodically modulated neural field models, Phys. Rev. E, 83(2011),011912. [DOI] [PubMed] [Google Scholar]
- [12].Coombes S, Lord GJ, and Owen MR, Waves and bumps in neuronal networks with axo-dendritic synaptic interactions, Phys. D, 178 (2003), pp. 219–241. [Google Scholar]
- [13].Coombes S and Owen MR, Evans functions for integral neural field equations with heaviside firing rate function, SIAM J. Appl. Dyn. Syst, 3 (2004), pp. 574–600. [Google Scholar]
- [14].Coombes S, Schmidt H, and Bojak I, Interface dynamics in planar neural field models, J. Math. Neurosci, 2 (2012), pp. 1–27, 10.1186/2190-8567-2-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [15].COOMBES S, Thul R, and WEDGWOOD K, Nonsmooth dynamics in spiking neuron models, Phys. D, 241 (2012), pp. 2042–2057, 10.1016/j.physd.2011.05.012. [DOI] [Google Scholar]
- [16].CuRTis CE, Prefrontal and parietal contributions to spatial working memory, Neuroscience, 139 (2006), pp. 173–180, 10.1016/j.neuroscience.2005.04.070. [DOI] [PubMed] [Google Scholar]
- [17].Faishl AA, Selen LP, and Wolpert DM, Noise in the nervous system, Nat. Rev. Neurosci, 9 (2008), pp. 292–303 [DOI] [PMC free article] [PubMed] [Google Scholar]
- [18].Faye G and Kilpatrick ZP, Threshold of front propagation in neural fields: An interface dynamics approach, SIAM J. Appl. Math, 78 (2018), pp. 2575–2596. [Google Scholar]
- [19].Folias SE and Bressloff PC, Breathing pulses in an excitatory neural network, SIAM J. Appl. Dyn. Syst, 3(2004), pp. 378–407. [Google Scholar]
- [20].Folias SE and Ermentrout GB, New patterns of activity in a pair of interacting excitatory-inhibitory neural fields, Phys. Rev. Lett, 107 (2011), 228103. [DOI] [PubMed] [Google Scholar]
- [21].Funahashi S, Bruce C, and Goldman-Rakic P, Mnemonic coding of visual space in the monkey’s dorsolateral prefrontal cortex, J. Neurophysiol, 61 (1989), pp. 331–349. [DOI] [PubMed] [Google Scholar]
- [22].Fuster JM, Unit activity in prefrontal cortex during delayed-response performance: Neuronal correlates of transient memory, J. Neurophysiol, 36 (1973), pp. 61–78. [DOI] [PubMed] [Google Scholar]
- [23].Gardiner CW, Handbook of Stochastic Methods: For Physics, Chemistry, and the Natural Sciences, Springer, Berlin, 2009 [Google Scholar]
- [24].Goldman-rakic PS, Cellular basis of working memory, Neuron, 14 (1995), pp. 477–485, 10.1016/0896-6273(95)90304-6. [DOI] [PubMed] [Google Scholar]
- [25].Häusser M and Roth A, Estimating the time course of the excitatory synaptic conductance in neocortical pyramidal cells using a novel voltage jump method, J. Neurosci, 17 (1997), pp. 7606–7625. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [26].Huang C, Pouget A, and DoIron B, Internally generated population activity in cortical networks hinders information transmission, Sci. Adv, 8 (2022), eabg5244. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [27].Ichi Amari S, Dynamics of pattern formation in lateral-inhibition type neural fields, Biol. Cybernet, 27 (1977), pp. 77–87, 10.1007/BF00337259. [DOI] [PubMed] [Google Scholar]
- [28].Kilpatrick ZP, Interareal coupling reduces encoding variability in multi-area models of spatial working memory, Front. Comput. Neurosci, 7 (2013), 82, 10.3389/fncom.2013.00082. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [29].Kilpatrick ZP, Ghosts of bump attractors in stochastic neural fields: Bottlenecks and extinction, Discrete Contin. Dyn. Syst. B, 21 (2016), pp. 2211–2231. [Google Scholar]
- [30].Kilpatrick ZP, Synaptic mechanisms of interference in working memory, Sci. Rep, 8 (2018), 7879, 10.1038/s41598-018-25958-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [31].Kilpatrick ZP and Ermentrout B, Wandering bumps in stochastic neural fields, SIAM J. Appl. Dyn. Syst, 12 (2013), pp. 61–94, 10.1137/120877106. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [32].Kilpatrick ZP, Ermentrout B, and Doiron B, Optimizing working memory with heterogeneity of recurrent cortical excitation, J. Neurosci, 33 (2013), pp. 18999–19011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [33].Kilpatrick ZP and Faye G, Pulse bifurcations in stochastic neural fields, SIAM J. Appl. Dyn. Syst, 13(2014), pp. 830–860. [Google Scholar]
- [34].Kim SS, Rouault H, Druckmann S, and Jayaraman V, Ring attractor dynamics in the drosophila central brain, Science, 356(2017), pp. 849–853. [DOI] [PubMed] [Google Scholar]
- [35].Kiyonaga A, Scimeca JM, Bliss DP, and Whitney D, Serial dependence across perception, attention, and memory, Trends Cogn. Sci, 21 (2017), pp. 493–497. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [36].Krishnan N, Poll DB, and Kilpatrick ZP, Synaptic efficacy shapes resource limitations in working memory, J. Comput. Neurosci, 44 (2018), pp. 1–23, 10.1007/s10827-018-0679-7. [DOI] [PubMed] [Google Scholar]
- [37].Laing CR and Chow CC, Stationary bumps in networks of spiking neurons, Neural. Comput, 13 (2001), pp. 1473–1494 [DOI] [PubMed] [Google Scholar]
- [38].Leavitt ML, Pieper F, Sachs AJ, and Martinez-TruJiLlo JC, Correlated variability modifies working memory fidelity in primate prefrontal neuronal ensembles, Proc. Natl. Acad. Sci. USA, 114 (2017), pp. E2494–E2503. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [39].Leine RI, Campen DHV, and Vrande BLVD, Bifurcations in nonlinear discontinuous systems, Nonlinear Dyn, 23 (2000), pp. 105–164. [Google Scholar]
- [40].Lorenc ES, Mallett R, and Lewis-Peacock JA, Distraction in visual working memory: Resistance is not futile, Trends Cogn. Sci, 25 (2021), pp. 228–239. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [41].Mcdonnell MD and Ward LM, The benefits of noise in neural systems: Bridging theory and experiment, Nat. Rev. Neurosci, 12 (2011), pp. 415–425. [DOI] [PubMed] [Google Scholar]
- [42].Pinto DJ and Ermentrout GB, Spatially structured activity in synaptically coupled neuronal networks: II. Lateral inhibition and standing pulses, SIAM J. Appl. Math, 62 (2001), pp. 226–243. [Google Scholar]
- [43].Ploner CJ, Gaymard B, Rivaud S, Agid Y, and Pierrot-Deseilligny C, Temporal limits of spatial working memory in humans, Eur. J. Neurosci, 10 (1998), pp. 794–797. [DOI] [PubMed] [Google Scholar]
- [44].Seeholzer A, Deger M, and Gerstner W, Stability of working memory in continuous attractor networks under the control of short-term plasticity, PLoS Comput. Biol, 15 (2019), e1006928, 10.1371/journal.pcbi.1006928S. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [45].Veltz R and Faugeras O, Local/global analysis of the stationary solutions of some neural field equations, SIAM J. Appl. Dyn. Syst, 9 (2010), pp. 954–998. [Google Scholar]
- [46].White JM, Sparks DL, and Stanford TR, Saccades to remembered target locations: An analysis of systematic and variable errors, Vision Res, 34 (1994), pp. 79–92. [DOI] [PubMed] [Google Scholar]
- [47].Wilson HR and Cowan JD, A mathematical theory of the functional dynamics of cortical and thalamic nervous tissue, Biol. Cybernet, 13 (1973), pp. 55–80, 10.1007/BF00288786. [DOI] [PubMed] [Google Scholar]
- [48].Wimmer K, Nykamp DQ, Constantinidis C, and Compte A, Bump attractor dynamics in prefrontal cortex explains behavioral precision in spatial working memory, Nat. Neurosci, 17 (2014), pp. 431–439, 10.1038/nn.3645. [DOI] [PubMed] [Google Scholar]