The stabilized supralinear network accounts for the contrast dependence of visual cortical gamma oscillations

Summary

When stimulated, neural populations in the visual cortex exhibit fast rhythmic activity with frequencies in the gamma band (30–80 Hz). The gamma rhythm manifests as a broad resonance peak in the power-spectrum of recorded local field potentials, which exhibits various stimulus dependencies. In particular, in macaque primary visual cortex (V1), the gamma peak frequency increases with increasing stimulus contrast. Moreover, this contrast dependence is local: when contrast varies smoothly over visual space, the gamma peak frequency in each cortical column is controlled by the local contrast in that column’s receptive field. No parsimonious mechanistic explanation for these contrast dependencies of V1 gamma oscillations has been proposed. The stabilized supralinear network (SSN) is a mechanistic model of cortical circuits that has accounted for a range of visual cortical response nonlinearities and contextual modulations, as well as their contrast dependence. Here, we begin by showing that a reduced SSN model without retinotopy robustly captures the contrast dependence of gamma peak frequency, and provides a mechanistic explanation for this effect based on the observed non-saturating and supralinear input-output function of V1 neurons. Given this result, the local dependence on contrast can trivially be captured in a retinotopic SSN which however lacks horizontal synaptic connections between its cortical columns. However, long-range horizontal connections in V1 are in fact strong, and underlie contextual modulation effects such as surround suppression. We thus explored whether a retinotopically organized SSN model of V1 with strong excitatory horizontal connections can exhibit both surround suppression and the local contrast dependence of gamma peak frequency. We found that retinotopic SSNs can account for both effects, but only when the horizontal excitatory projections are composed of two components with different patterns of spatial fall-off with distance: a short-range component that only targets the source column, combined with a long-range component that targets columns neighboring the source column. We thus make a specific qualitative prediction for the spatial structure of horizontal connections in macaque V1, consistent with the columnar structure of cortex.

When presented with a stimulus, populations of neurons within visual cortices exhibit elevated rhythmic activity with frequencies in the so-called gamma band (30–80 Hz) (Jia et al., 2013; Ray and Maunsell, 2010). These gamma oscillations can be observed in local field potential (LFP) or electroencephalogram (EEG) recordings and, when present, manifest as peaks in the LFP/EEG power-spectra. It has been proposed that gamma oscillations perform key functions in neural processing such as feature binding (Singer, 1999), dynamic communication or routing between cortical areas (Fries, 2005, 2015; Ni et al., 2016; Palmigiano et al., 2017), or as a timing or “clock” mechanism that can enable coding by spike timing (Buzsáki and Chrobak, 1995; Draguhn and Buzsáki, 2004; Fries et al., 2007; Hopfield, 2004; Jefferys et al., 1996). These proposals remain controversial (Burns et al., 2010; Ray and Maunsell, 2010).

While the computational role of gamma rhythms is not well understood, much is known about their phenomenology. For example, defining characteristics of gamma oscillations, such as the width and height of the spectral gamma peak, as well as its location on the frequency axis (peak frequency), exhibit systematic dependencies on various stimulus parameters (Gieselmann and Thiele, 2008; Henrie and Shapley, 2005; Jia et al., 2013; Ray and Maunsell, 2010). In particular, in the primary visual cortex (V1) of macaque monkeys, the power-spectrum gamma peak moves to higher frequencies as the contrast of a large and uniform grating stimulus is increased (Jia et al., 2013; Ray and Maunsell, 2010). This establishes a monotonic relationship between gamma peak frequency and the grating contrast. We will refer to this contrast-frequency relationship, obtained using a grating stimulus with uniform contrast, as the “contrast dependence” of gamma peak frequency.

Moreover, when animals are presented with a stimulus with non-uniform contrast that varies over the visual field (and hence over nearby cortical columns in V1), it is the local stimulus contrast that determines the peak frequency of gamma oscillations at a cortical location (Ray and Maunsell, 2010). Specifically, Ray and Maunsell 2010 used a Gabor stimulus (which has smoothly decaying contrast with increasing distance from the stimulus center), and found that the gamma peak frequency of different V1 recording sites match the predictions resulting from the frequency-contrast relationship obtained from the uniform grating experiment, but using the local Gabor contrast in that site’s receptive field. We refer to this second effect as the “local contrast dependence” of gamma peak frequency.

It is well-known that networks of excitatory and inhibitory neurons with biologically realistic neural and synaptic time-constants can exhibit oscillations with frequency in the gamma band (e.g., Brunel and Wang 2003^; Tsodyks et al. 1997; see Buzsáki and Wang 2012 for a review). However, no mechanistic circuit model of visual cortex has been proposed which can robustly and comprehensively account for the contrast dependence of gamma oscillations. Jia et al. 2013 did propose a rate model that accounts for the increase of gamma peak frequency with increasing global contrast. Their treatment only modeled the interactions between a single excitatory and a single inhibitory population, which is sufficient for spatially uniform stimuli, but cannot explain the local contrast dependence of the gamma peak frequency. Moreover, even in the case of a uniform-contrast stimulus, this model could only produce very weak contrast-dependence of peak frequency, and further required a contrast-dependent scaling of the intrinsic time-constant of excitatory neurons. Here, we develop a parsimonious and self-contained mechanistic model (with fixed neural and network parameters) which accounts for the global as well as local contrast dependence of the gamma peak.

It is not clear how the local contrast dependence of gamma oscillations can be reconciled with key features of cortical circuits. This locality would trivially emerge if cortical columns were non- or weakly interacting; in that case each column’s oscillation properties would be determined by its feedforward input (controlled by the local contrast). However, nearby cortical columns do interact strongly via the prominent horizontal connections connecting them (Gilbert and Wiesel, 1989). These interactions manifest, e.g., in contextual modulations of V1 responses, such as in surround suppression (Cavanaugh et al., 2002), which are thought to be partly mediated by horizontal connections (Schwabe et al., 2010).

Surround suppression is the phenomena wherein stimuli outside the classical receptive field (RF) of V1 neurons, which by themselves cannot drive the cell to respond, nevertheless modulate the cells’ response, typically by suppressing it. Surround suppression results in a non-monotonic “size tuning curve”, which is obtained by measuring a cell’s response to circular gratings of varying sizes centered on that cell’s RF: the response first increases with increasing stimulus size, but then decreases as the grating increasingly covers regions surrounding the RF. Here we test whether a model of V1, featuring biologically plausible horizontal connections, can capture both surround suppression and the local contrast dependence of gamma oscillations.

A parsimonious, biologically plausible model of cortical circuitry which has successfully accounted for a range of cortical contextual modulations and their contrast dependence is the stabilized supralinear network (SSN) (Ahmadian et al., 2013; Rubin et al., 2015). In particular, the SSN robustly captures the contrast dependencies of surround suppression, e.g., that size tuning curves peak at smaller stimulus sizes with increasing stimulus contrast (Rubin et al., 2015). Being a recurrently connected firing rate model with excitatory and inhibitory neurons, we expect the SSN to be able to exhibit oscillations similar to gamma rhythms. However, to capture fast dynamical phenomena, and in particular the gamma band resonance frequency, it is key to properly account for fast synaptic filtering as provided by the fast ionotropic receptors, AMPA and GABA_A (Barbieri et al., 2014; Brunel and Wang, 2003; Ledoux and Brunel, 2011). At the same time, it is useful to include the slower NMDA conductances, to help stabilize the network dynamics given strong overall recurrent excitation. We thus started by extending the SSN model to properly account for input currents through different synaptic receptor types, with different filtering timescales.

Gamma oscillations do not behave like sustained oscillations, as sustained oscillations display a sharp peak in the power spectrum, typically followed by trailing peaks at subsequent harmonics. Such oscillations would also be auto-coherent, i.e. have a consistent phase over multiple oscillation cycles. By contrast, gamma oscillations are not auto-coherent and their timing and duration vary stochastically (Burns et al., 2011, 2010), resulting in a single broad peak in the power-spectrum, with no visible higher harmonics (Jia et al., 2013; Ray and Maunsell, 2010), consistent with transient (damped) and noise-driven oscillations (Burns et al., 2011, 2010; Xing et al., 2012). We therefore studied the SSN in a regime where in the absence of time-dependent external inputs its firing rates reach a steady state, but when perturbed it can exhibit damped oscillations with a characteristic frequency (technically, this means the network is close to, but below, a Hopf bifurcation, i.e., a transition to a regime of sustained oscillations). When perturbed by structureless noise that is sufficiently fast (the biological network’s irregular spiking itself can provide such a noise source), these noise-driven damped oscillations manifest as a resonance peak in the power-spectrum of network activity (Kang et al., 2010; Xing et al., 2012).

We start the Results section by developing an extension of the SSN that models the dynamics of input currents through different synaptic receptor types, with different timescales. We then study a reduced noise-driven SSN composed of two units representing excitatory (E) and inhibitory (I) sub-populations. We show that, for a wide range of biological parameters, this reduced SSN model generates gamma oscillations with peak frequency that robustly increases with increasing external drive to the network. We show that this robust contrast dependence is a consequence of a key feature of the SSN: the supralinear input-output (I/O) function of its neurons (which is known to fit well the non-saturating and expansive relationship between the firing rate and membrane voltage of V1 neurons (Anderson et al., 2000; Priebe and Ferster, 2008)). We next investigate the gamma peak’s local contrast dependence using an expanded retinotopically organized SSN model o V1, with E and I units in different cortical columns. We show that this network is capable of reproducing the local contrast dependence of gamma peak frequency while exhibiting realistic surround suppression. However, as we show, this is only possible when the spatial fall-off of excitatory connection strengths has two distinct components: a sharp immediate fall across a cortical column’s width, and a slower fall off that can range over several columns. This “local plus long-range” spatial structure of horizontal connections, which we will more shortly refer to as “columnar structure”, balances the trade-off between capturing local contrast dependence (requiring short-range or weak horizontal connections) and surround suppression (requiring the opposite). We show that achieving this balance does not require fine-tuning of parameters and is robust to considerable parameter variations. We end by providing a mathematical explanation of the mechanism underlying local contrast dependence reconciled with strong surround suppression in this model, based on the structure of its normal oscillatory modes. Finally, in the Discussion, we conclude by discussing the implications of our findings for the structure of cortical horizontal connections and the shape of neural input/output nonlinearities.

Results

Noise-driven SSN with multiple synaptic currents

As motivated in the introduction, and with the aim of modeling gamma oscillations, we started by extending the SSN model to properly account for synaptic currents through different receptor types with different kinetics. In its original form, the SSN’s activity dynamics are governed by standard firing rate equations (Dayan and Abbott, 2001), in which each neuron is described by a single dynamical variable: either its output firing rate (Ahmadian et al., 2013; Rubin et al., 2015) or its total input current (Hennequin et al., 2018). In the extended model, by contrast, each neuron will have more than one dynamical variable, corresponding to its input currents through different synaptic receptor types. Concretely, we will include the three main ionotropic synaptic receptors in the model: AMPA and NMDA receptors which mediate excitatory inputs, and GABA_A (henceforth abbreviated to GABA) receptors which mediate the inhibitory input. For a network of N neurons, we will arrange these input currents to different neurons into three N-dimensional vectors, h^α, where α ∈ {AMPA, NMDA, GABA } denotes the receptor type. To model the kinetics of different receptors, we will ignore the very fast rise-times of all receptor types (as the corresponding timescales are much faster than the characteristic timescales of gamma oscillations), and only account for the receptor decay-times, which we denote by τ^α. With this assumption, the dynamics of ${\mathbf{h}}_t^{\alpha }$ are governed by (see Methods for a derivation)

$${\tau }_{\alpha }\frac{d{\mathbf{h}}_t^{\alpha }}{dt}+{\mathbf{h}}_t^{\alpha }=W^{\alpha }{\mathbf{r}}_t+{\mathbf{I}}_t^{\alpha }$$ (1)

where r_t is the vector of firing rates, W^α r_t and ${\mathbf{I}}_t^{\alpha }$ denote the recurrent and external inputs to the network mediated by receptor α, respectively, and W^α are N × N matrices denoting the contributions of different receptortypes to recurrent connectivity weightsl; the total recurrent connectivity weight matrix is thus given by W ≡ ∑_α W^α. As in the cortex, the external input to the network is excitatory, and for simplicity we further assume that it only enters through AMPA receptors (i.e. ${\mathbf{I}}_t^{\alpha }$ is nonzero only for α = AMPA, and we will thus drop this superscript and denote this input by I_t); including an NMDA components will not affect our results, as NMDA is slow relative to gamma band timescales. To close the system for the dynamical variables ${\mathbf{h}}_t^{\alpha }$, we have to relate the output rate of a neuron to its total input current. The fast synaptic filtering provided by AMPA and GABA allows for a static (or instantaneous) approximation to the input-output (I/O) transfer function of neurons (Brunel et al., 2001; Fourcaud and Brunel, 2002) (see Methods for further justification of this approximation):

$${\mathbf{r}}_t=F\left({\mathbf{h}}_t^{\text{total}}\right)=F\left(\sum _{\beta }{\mathbf{h}}_t^{\beta }\right),$$ (2)

where the I/O function F (⋅) acts element-wise on its vector argument. As in the original SSN, we take this 1/O transfer function to be a supralinear rectified power-law, which is the essential ingredient of the SSN (see Fig. 1 A inset): $F\left(v\right)\equiv k[v{]}_{+}^n$, where k is a positive constant, n > 1 (corresponding to supralinearity), and [x]₊ ≡ max(0, x) denotes rectification. While the I/O function of biological neurons saturates at high firing rates (e.g., due to refractoriness), throughout the natural dynamic range of cortical neurons firing rates stay relatively low. In fact, in V1 neurons the relationship between the firing rate and the mean membrane potential (an approximate surrogate for the neuron’s net input) shows no saturation throughout the entire range of firing rates driven by visual stimuli, and is well approximated by a supralinear rectified power-law (Anderson et al., 2000; Priebe and Ferster, 2008).

Figure 1.

Contrast dependence of the gamma peak frequency in the 2-population model. A: Schematic of the 2-population Stabilized Supralinear Network (SSN). Excitatory (E) connections end in a circle; inhibitory (I) connections end in a line. Each unit represents a sub-population of V1 neurons of the corresponding E/I type. Both receive inputs from the stimulus, as well as noise input. Inset: The rectified power-law Input/Output transfer function of SSN units (black). Red lines indicate the slope of the I/O function at particular locations. B: Local field potential (LFP) traces, modeled as total net input to the E unit, from the stochastic model simulations under four different stimulus contrasts (c): 0% (black) equivalent to no stimulus or spontaneous activity, 25% (blue), 50% (green), 100% (red). The same color scheme for stimulus contrast is used throughout the paper. (Note that we take stimulus strength (input firing rate) to be proportional to contrast, although in reality it is monotonic but sublinear in contrast, Kaplan et al., 1987). C: Mean firing rates of the excitatory (orange) and inhibitory (cyan) units as a function of contrast, from the stochastic simulations (dots) and the noise-free approximation of the fixed point, Eq. (3) (stars). Note that the dots and stars closely overlap. D: Reproduction of figure 1I from (Ray and Maunsell, 2010) showing the average of experimentally measured LFP power-spectra in Macaque V1. The inset shows the dependence of gamma peak frequency on the contrast of the grating stimulus covering the recording site’s receptive field. E: LFP power-spectra for c = 0%, 25%, 50%, 100% (black, blue, green, and red curves, respectively) calculated from the noise-driven stochastic SSN simulations (dots), or using the linearized approximation (solid lines). F: Gamma peak frequency as a function of contrast, obtained from power-spectra calculated using stochastic simulations (dots and dashed line) or the linearized approximation (stars and solid line).

Two-population model

We start by studying a reduced two-population model of V1 consisting of two units (or representative mean-field neurons): one excitatory and one inhibitory unit, respectively representing the excitatory and inhibitory neural sub-populations in the retinotopically relevant region of V1. This reduced model is appropriate for studying conditions in which the spatial profile of the activity is irrelevant, e.g., for a full-field grating stimulus where we can assume the relevant V1 network is uniformly activated by the stimulus. Both units receive external inputs, and make reciprocal synaptic connections with each other as well as themselves (Fig. 1 A).

As pointed out in the introduction, empirical evidence is most consistent with visual cortical gamma oscillations resulting from noise-driven fluctuations, and not from sustained coherent oscillations (Burns et al., 2011, 2010; Kang et al., 2010; Xing et al., 2012). To model such noise-driven oscillations using the SSN, as in (Hennequin et al., 2018), we assumed the external input consists of two terms I_t=I_DC + η_t, where I_DC represents the feedforward stimulus drive to the network (by a steady time-independent stimulus) and scales with the contrast of the visual stimulus, and η_t represents the stochastic noise input to the network. This input noise could be attributed to several sources, including sources that are (biologically) external or internal to V1. External noise can originate upstream in the lateral geniculate nucleus (LGN) of thalamus, or in feedback from higher areas. Internally generated noise results from the network’s own irregular spiking (not explicitly modeled) which survives mean-field averaging as a finite-size effect (given the finite size of the implicitly-modeled spiking neuron sub-populations underlying the SSN’s units). For parsimony, we assumed noise statistics are independent of stimuli, and thus of contrast.¹ More specifically, we assumed that noise inputs to different neurons are independent, and each component is temporally correlated pink noise with a correlation time on the order of a few milliseconds (our main results are robust to changes in this parameter, as well as to the introduction of input noise correlation across neurons).

For the first results shown in Fig. 1, we directly simulated the stochastic Eq. (1). Fig. 1C (dots) shows the average firing rates found in these simulations, and their contrast-dependence. The LFP signal is thought to result primarily from inputs to pyramidal cells, as they have relatively large dipole moments (Einevoll et al., 2013); we therefore took the net input to the E sub-population to represent the LFP signal. Fig. 1B shows examples of raw simulated LFP traces for different stimulus contrasts. For high enough contrast (including all nonzero contrasts shown), the LFP signal exhibits oscillatory behavior. These oscillations can be studied via their power-spectra (Fig. 1E, dots; see Methods). As Fig. 1F shows, the peak frequencies of the simulated LFP power-spectra shift to higher frequencies with increasing contrast. The two-population SSN model thus captures the empirically observed contrast dependence of gamma peak frequency (compare Fig. 1F with Fig. 1-I of Ray and Maunsell 2010 reproduced here as Fig. 1D).

To understand this behavior better, we employed a linearization scheme for calculating the LFP power-spectra. The linearization method allows for faster numerical computation of the LFP power-spectra, without the need to simulate the stochastic system Eq. (1). More importantly, the linearized framework allows for analytical approximations and insights, which as we show below, elucidate the mechanism underlying the contrast dependence of the gamma peak. We thus explain this approximation with some detail here (see Methods for further details). In any stimulus condition (corresponding to a given I_DC), we first find the network’s steady state in the absence of noise, by numerically solving the noise-free version of Eq. (1) (without linearization). The corresponding fixed-point equations can be simplified if we sum them over α, and define ${\mathbf{h}}_{*}\equiv {\sum }_{\alpha }{\mathbf{h}}_{*}^{\alpha }$ and ${\mathbf{I}}_{DC}\equiv {\sum }_{\alpha }{\mathbf{I}}_{DC}^{\alpha }\cdot$ We then arrive at the same fixed-point equation for h_* as in the original SSN (Ahmadian et al., 2013):

$${\mathbf{h}}_{*}=WF\left({\mathbf{h}}_{*}\right)+{\mathbf{I}}_{DC}.$$ (3)

After numerically finding h_*, we then expand Eq. (1) to first order in the noise and noise-drive deviations around the fixed point, $\delta {\mathbf{h}}_t^{\alpha }\equiv {\mathbf{h}}_t^{\alpha }-{\mathbf{h}}_{*}^{\alpha }$, to obtain

$${\tau }_{\alpha }\frac{d\delta {\mathbf{h}}_t^{\alpha }}{dt}=-\delta {\mathbf{h}}_t^{\alpha }+{\tilde{W}}^{\alpha }\sum _{\beta }\delta {\mathbf{h}}_t^{\beta }+{\mathit{\eta }}_t^{\alpha }$$ (4)

where we defined

$${\tilde{W}}^{\alpha }\equiv W^{\alpha }\text{diag}\left(F^{\text{'}}\left({\mathbf{h}}_{*}\right)\right),$$ (5)

where F′ (h_*) denotes the vector of gains (slopes) of the I/O functions of different neurons at the operating point h_* (see the red tangent lines in Fig. 1 A inset), and diag constructs a diagonal matrix from the vector. As we explain in the next subsection, the neural gains and their dependence on the operating point rates (themselves dependent on the stimulus ${\mathbf{I}}_{DC}^{\alpha }$, via Eq. (3)) play a crucial role in the contrast dependence of gamma peak frequency.

Technically, the linear approximation is valid for small noise strengths, but we found that for the noise levels that elicited fluctuations with realistic sizes, the approximation was very good. As shown in Fig. 1E–F, the LFP power-spectra and their peak frequencies obtained using the linear approximation agree very well with those estimated from the direct stochastic simulations of Eq. (1). The firing rates of E and I units at the fixed-point solution Eq. (3) also provide a very good approximation to their mean steady-state rates, at different contrasts, as obtained from direct stochastic simulations of Eq. (1) (Fig. 1C). Below, we will thus calculate all power-spectra using the computationally faster noise-free determination of the fixed point, Eq. (3), and the linear approximation, Eq. (4), instead of stochastic simulations of Eq. (1).

Robustness of the two-population model

To demonstrate that the SSN robustly produces the contrast dependence of gamma peak frequency, we simulated 1000 different instances of the 2-population network with parameters randomly drawn from wide but biologically plausible ranges. The sampled parameters were the weights of the connections between the two units (E → I, I → E) and their self-connections (E → E, I →I), the relative strength of input to the excitatory and inhibitory units, and the NMDA fraction of excitatory synaptic weights.

The parameters were sampled independently except for the enforcement of two inequality constraints which previous work has shown to be necessary for ensuring the network’s dynamical stability without strong inhibition domination leading to very weak excitatory activity (see Methods for details). The parameter set was also rejected if the resulting SSN did not reach a stable fixed point for all studied stimulus conditions (this corresponds to our modelling choice to have the SSN in a damped oscillation regime).

The majority of randomly sampled models produced steady-state excitatory and inhibitory firing rates that were within biologically plausible ranges, across all contrasts (Fig. 2A and D). Furthermore, many two-population networks produced peak frequencies that were in the gamma band (30–80 Hz) for all contrast conditions, though some produced peaks at higher frequencies for the highest contrasts (Fig. 2 B). The distributions also shift towards higher frequencies with increasing contrast, suggesting that the two-population SSN is indeed able to robustly reproduce the contrast dependence of gamma peak frequency. To demonstrate this more directly, we show the distributions of the changes in peak frequency normalized by the change in contrast in Fig. 2E. No sampled network produced a negative change in peak frequency with increasing stimulus contrast. As a further corroboration of our model, we also studied how the width of the gamma peak changed with increasing contrast. While (Jia et al., 2013; Ray and Maunsell, 2010) did not quantify changes in their gamma peak width with increasing contrast, their results suggest that no significant systematic change in width was observed (Fig. 1 D). Similarly, in our two-population network, changes in the half-width of the gamma peak are relatively small, and the direction of change can be positive or negative with similar probability (Fig. 2 F).

Figure 2.

Robustness of the contrast-dependence of gamma peak frequency to network parameter variations. One thousand 2-population SSN’s were simulated with randomly sampled parameters (but conditioned on producing stable noise-free steady-states), across wide biologically plausible ranges. All histograms show counts of sampled networks; the total numbers (n’s) vary across different histograms, as different subsets of network produced the corresponding feature or value in the corresponding condition (e.g., a gamma peak at 50% contrast). Panels A-E and F-J show results for the columnar and non-columnar models, respectively. A: Distributions of the excitatory unit’s firing rate in response to 25%, 50%, and 100% contrast stimuli (blue, green, red), plotted on a logarithmic scale. 100% of networks shown across all contrasts. B: Distributions of the gamma peak frequencies at different stimulus contrasts. The n’s (upper right) give the number of networks with a power spectrum peak above 20 Hz. C: Distributions of the gamma peak widths at different stimulus contrasts. D: Same as panel A, but for the inhibitory unit. E: Distributions of the change in gamma peak-frequency normalized by the change in stimulus contrast, either 25% and 50% (cyan) or 50% and 100% (yellow). F: Same as panel E, but for gamma peak-width.

Mechanism underlying the contrast dependence of gamma peak frequency

As we will now show, the SSN sheds light on the mechanism underlying the contrast dependence of the gamma peak, and specifically pins it to the increasing neuronal gain with increasing neuronal activity, due to the expansive, supralinear nature of the neuronal I/O transfer function. This also explains the robustness of the effect to changes in connectivity and external input parameters as demonstrated in the previous subsection.

In the linearized approximation, the LFP power-spectrum (see Eqs. (24)–(26) in Methods) can be expressed in terms of the so-called Jacobian matrix, i.e. the matrix of couplings of the dynamical variables, $\delta {\mathbf{h}}_t^{\alpha }$, in the linear system Eq. (4); thus, for a network of N neurons, the Jacobian is a 3N × 3N matrix, or 6 × 6 in the two-population model (see Eqn. 33 for the explicit form). The existence of damped oscillations and the value of their frequency (which is the resonance frequency manifesting as a peak in the power-spectrum) are in turn determined by the existence of complex eigenvalue of the Jacobian matrix, and the value of their imaginary parts. Previously, the eigenvalues of the Jacobian for a standard E-I firing rate network, without different synaptic current types, were analyzed by Tsodyks et al. 1997, and conditions for emergence of (damped or sustained) oscillations were found. In Appendix 1, we show that, given the slowness of NMDA receptors relative to gamma timescales, the effect of NMDA receptors on the relevant complex eigenvalues of the Jacobian can be safely ignored, and as a result, only two (out of 6) eigenvalues of the resulting Jacobian can become complex (and thus able to create a gamma peak). Moreover, this pair (which we denote by λ_±) correspond to the two eigenvalues of a standard E-I rate model (Tsodyks et al., 1997) whose E and I neural time-constants are given, respectively, by the AMPA and GABA decay times:

$$2{\lambda }_{\pm }={\gamma }_E\left({\tilde{W}}_{EE}-1\right)-{\gamma }_I\left({\tilde{W}}_{II}+1\right)\pm \sqrt{{\left[{\gamma }_E\left({\tilde{W}}_{EE}-1\right)+{\gamma }_I\left({\tilde{W}}_{II}+1\right)\right]}^2-4{\gamma }_E{\gamma }_I{\tilde{W}}_{EI}{\tilde{W}}_{IE}}$$ (6)

where ${\gamma }_E={\tau }_{\text{AMPA}}^{-1}$ and ${\gamma }_I={\tau }_{\text{GABA}}^{-1}$. Here we defined

$${\tilde{W}}_{ab}\equiv W_{ab}{F}^{\mathbf{\text{'}}}\left(h_a^{*}\right)\hspace{1em}(a,b\in \{E,I\left\}\right),$$ (7)

where $W_{ab}\equiv {\sum }_{\alpha }{W}_{ab}^{\alpha }$ is the total synaptic weight from unit b to unit a, and (as in Eq. (5)) $F^{\text{'}}\left(h_a^{*}\right)$ is the gain of unit a, i.e. the slope of its I/O function, at the operating point set by the stimulus. We refer to ${\tilde{W}}_{ab}$ as effective synaptic connection weights. Unlike raw synaptic weights, these effective weights are modulated by the neural gains, and thereby by the activity levels in the steady state operating point, which is in turn controlled by the stimulus.

As mentioned, (damped or sustained) oscillations emerge when the above eigenvalues are complex (in which case λ₊ and λ_- are complex conjugates). This happens when the expression under the radical in Eqn. 6 is negative, i.e.

$$4{\gamma }_E{\gamma }_I{\tilde{W}}_{EI}{\tilde{W}}_{IE}>{\left[{\gamma }_E\left({\tilde{W}}_{EE}-1\right)+{\gamma }_I\left({\tilde{W}}_{II}+1\right)\right]}^2\text{.}$$ (8)

Qualitatively, the left hand side of the above inequality is a measure of the strength of the effective negative feedback between the E and I sub-populations, while the right hand size is a measure of the positive feedback in the network (arising from the network’s recurrent excitation and disinhibition). Oscillations thus emerge when the negative feedback loop between E and I is sufficiently strong, in the precise sense of Eqn. 8.

During spontaneous activity (when the external input is zero or very weak), the rates of both E and I populations are very small. This means that the spontaneous activity operating point sits near the rectification of the neuronal I/O transfer functions where the neural gains are very low (Fig. 1A left). Thus in the spontaneous activity state, the dimensionless effective connections are relatively small. In the limit of ${\tilde{W}}_{ab}\to 0$, the left hand side of Eqn. 8 goes to zero, while its right side goes to (γ_I - γ_E)² which is generically positive; hence the inequality is not satisfied. This shows that the spontaneous A activity state generically does not exhibit oscillations, in agreement with lack of empirical observation of gamma oscillation during spontaneous activity.

On the other hand, when condition Eqn. 8 does hold, the frequency of the oscillations are given by the imaginary part of the eigenvalues, i.e.

$$\text{resonance}\text{frequency}=\frac{1}{2\pi }\sqrt{{\gamma }_E{\gamma }_I{\tilde{W}}_{EI}{\tilde{W}}_{IE}-{\left[{\gamma }_E\left({\tilde{W}}_{EE}-1\right)/2+{\gamma }_I\left({\tilde{W}}_{II}+1\right)/2\right]}^2}$$ (9)

(the division by 2π is because the eigenvalue imaginary parts give the angular frequency). As we will discuss further below, the resonance frequency (or approximately the gamma peak frequency [Fig. 3B]) thus depends on the effective connections weights and is thus modulated by the neural gains. (Note, however, that the scale or order of magnitude of this frequency is set by γ_E and γ_I, i.e., by the decay times of AMPA and GABA, as the effective connection weights are dimensionless and cannot determine the dimensionful scale of the gamma frequency; ignoring the “positive feedback” contribution in Eq. (9), we find $\text{resonance}\text{frequency}\propto \sqrt{{\gamma }_E{\gamma }_I}/\left(2\pi \right)$, which for τ_AMPA ~ τ_GABA ~ 4–6 ms, is on the order of 30–40 Hz.)

Figure 3.

The supralinear nature of the neural transfer function can explain the contrast dependence of gamma frequency. A: Schematic diagrams of the 2-population SSN (see Fig. 1A) receiving a low (left) or high (right) contrast stimulus. The thickness of connection lines represents the strength of the corresponding effective connection weight, which is the product of the anatomical weight and the input/ouput gain of the presynaptic neuron. The gain is the slope (red line) of the neural supralinear transfer function (black curve), shown inside the circles representing the E (orange) and I (cyan) units. A resonance frequency exists when the effective “negative feedback” (gray arrow enlcosing a minus sign) dominates the effective “positive feedback” (gray arrows enclosing positive signs), in the sense of the inequality Eq. (8). As the stimulus drive (c) increases (right panel), the neurons’ firing rates at the network’s operating point increase. As the transfer function is supralinear, this translates to higher neural gains and stronger effective connections. When a resonance frequency already exists at the lower contrast, this strengthening of effective recurrent connections leads to an increase in the gamma peak frequency, approximately given by the imaginary part of the linearized SSN’s complex eigenvalue, Eq. (6). B: The eigenvalue formula (9) provides an excellent approximation to the gamma peak frequency across sampled networks and contrasts simulated in Fig. 2; correlation coefficient =0.98 (p < 10⁻⁶), for all data points combined across 25% (blue), 50% (green), and 100% (red) contrasts. C: The negative feedback loop contribution to the resonance frequency (Eq. (9) with the second term under the square root neglected) overestimates gamma peak frequency but is positively and significantly correlated with it; correlation coefficient =0.67(p < 10⁻⁶).

Equation (9) provides the insight into the contrast dependence of the gamma peak frequency (see Fig. 3). As contrasts increase the fixed-point firing rates increase (Fig. 1C). Because the SSN I/O transfer function is non-saturating and supralinear, as the rates increase the gains (i.e., the slope of the I/O transfer functions) of the E and I cells are also guaranteed to increase (Fig. 3A right vs. left). The increase in the gains leads in turn to the strengthening of the effective connection weights, Eq. (7), and therefore of the network’s negative E-I feedback loop. When Eq. (8) is satisfied, a rough approximation (Fig. 3B vs. C) to the resonance frequency is obtained by ignoring the positive feedback contribution (the second term under the square root in Eq. (9)). With only the negative feedback contribution retained, it is clear that an increase in neural gains leads to an increase in the resonance frequency (the precise conditions for this to occur are given in Appendix 2 below). Thus as contrast increases, we expect the gamma peak in the LFP power-spectrum to move to higher frequencies, due to increasing neural gains and effective connectivity as dictated by the supralinear neural I/O transfer function.

Retinotopic SSN

We next investigated whether the SSN can account for the locality of contrast dependence of gamma peak frequency, when V1 receives a stimulus with a spatially varying contrast profile. To this end, we expanded our network from two units representing global E and I populations to many units that are retinotopically organized. We thus model the cortex as a two-dimensional grid that has an E and I sub-population (corresponding to SSN units) at each grid location, corresponding to a cortical column (Fig. 4A). In the retinotopic SSN, the stimulus input can vary across the network: each column can receive a different input proportional to the contrast within its receptive field. We presented this network with uniform-contrast grating stimuli of various sizes and contrasts (the stimulus in Fig. 4A), as well as a Gabor stimulus (Fig. 4E), similar to the one used in Ray and Maunsell 2010, with a contrast profile that decays smoothly with deviation from the stimulus center according to a Gaussian profile (see Eq. (54)). Gamma peak frequency shows only a weak dependence on stimulus orientation (Jia et al., 2013), possibly due to the averaging of LFP over an area larger than the size of orientation minicolumns. To keep our model parsimonious and computationally more tractable, we thus chose the size of our cortical columns to be roughly half the hypercolumn size in Macaque, and neglected the orientation map structure, and the dependence of external inputs and horizontal connections on preferred orientation.

Figure 4.

A retinotopically-structured SSN model of V1, with a boost to local, intra-columnar excitatory connectivity, exhibits a local contrast-dependence of gamma peak frequency, as well as robust surround suppression of firing rates. A: Schematic of the model’s retinotopic grid, horizontal connectivity, and stimulus inputs. Each cortical column has an excitatory and an inhibitory sub-population (orange and cyan balls) which receive feedforward inputs (green arrows) from the visual stimulus, here a grating, according to the column’s retinotopic location. Orange lines show horizontal connections projecting from two E units; note boost to local connectivity represented by larger central connection. Inhibitory connections (cyan lines) only targeted the same column, to a very good approximation. B: LFP power-spectra in the center column evoked by flat gratings of contrasts 25% (blue), 50% (green), and 100% (red). C: Gamma peak frequency as a function of flat grating contrast. Note that peaks were defined as local maxima of the relative power-spectrum, i.e., the point-wise ratio of the absolute power-spectrum (as shown in B), to the power-spectrum at zero contrast; see Methods. D: Firing rate responses of E (orange) and I (cyan) center sub-populations as a function of grating contrast. E: The Gabor stimulus with non-uniform contrast (falling off from center according to a Gaussian). The colored circles show the five different cortical locations (retinotopically mapped to the visual field) probed by the LFP “electrodes”. The orange probe was at the center and the distance between adjacent probe locations was 0.2° of visual angle (corresponding to 0.4 mm in V1, the width of the model columns). F: LFP power-spectra evoked by the Gabor stimulus at different probe locations (legend shows the probe distances from the Gabor center). G: Gamma peak frequency of the power-spectra at increasing distance from the Gabor center. The golden curve is the prediction for peak frequency in the displaced probe location based on the Gabor contrast in that location and the gamma peak frequency obtained in the center location for the flat grating of the same contrast. The predictor’s fit to actual Gabor frequencies is very tight (R²=0.98), exhibiting local gamma contrast dependence. H : Size tuning curves of the center E (orange) and I (cyan) subpopulations, at full contrast. E and I firing rates vary non-monotonically with grating size and exhibit surround suppression (suppression indices were 0.33 and 0.15, respectively.

We wish to study the trade-off in this model between capturing surround suppression of firing rates and capturing the local dependence of gamma peak frequency, and asked whether parameter choices exist for which the model can capture both of these effects. In particular, we studied the effect of the spatial profile of the horizontal recurrent connections between and within different cortical columns on this trade-off. In one extreme, we can consider a network in which long-range connections between different columns are very weak, and thus cortical columns are weakly interacting and can be approximated by independent two-population networks which were studied above (Figs. 1–1). In this case, the frequency of the gamma resonance in each column depends only on the gains and activity levels in that column, which are in turn set by the feedforward input to that column, controlled by the local stimulus contrast. Therefore a network with such a connectivity structure would trivially reproduce the local contrast-dependence of gamma peak frequency. However, due to lack of strong inter-columnar interactions, such a network would fail to produce significant surround suppression of firing rates. In the other extreme, inter-columnar strengths are strong and, importantly, have a smooth fall-off (e.g. an exponential fall) with growing distance between pre- and post-synaptic columns; this is the case in most cortical network models, including the SSN model of (Rubin et al., 2015) that captures surround suppression and its various contrast-dependencies. However, as we will show below, in such networks, when horizontal connections are strong enough to produce surround suppression, the gamma peak frequency is typically shared across all activated columns, regardless of the spatial contrast profile of the stimulus, and thus this connectivity structure cannot capture the local contrast-dependence of gamma peak frequency. Indeed, as shown below, within this class of networks (i.e., those with a smooth spatial connectivity profile), we did not find connectivity parameters (controlling the range and strength of horizontal connections) for which the network could produce significant surround suppression and yet capture the local contrast-dependence of gamma (see Fig. 5 and 6).

Figure 5.

Behaviour of retinotopic V1 models with and without boosted intra-columnar recurrent excitatory connectivity (columnar vs. non-columnar models, respectively) across their parameter space. We simulated 2000 different networks of each type with parameters (11 in total) randomly sampled across wide, biologically plausible ranges. All histograms show counts of sampled networks; the total number of samples vary across histograms, as only subsets of networks exhibited the corresponding feature with a value in the shown range. Panels A-E and F-J show results for the columnar and non-columnar models, respectively. A & F: Distributions of gamma peak frequency, recorded at stimulus center, for different contrasts of the uniform grating (histograms for 25%, 50%, and 100% contrasts in blue, green and red, respectively). D & I: Distributions of the change in gamma peak-frequency normalized by the change in grating contrast, changing from25% to 50% (cyan) or from 50% to 100% (yellow). B & G: Distributions of the suppression index for the center E (orange) and I (cyan) sub-populations. C & H : The distributions of the coefficient of variation R², as a measure of the locality of gamma peak contrast dependence. The R² quantifies the goodness-of-fit of predicted gamma peak frequency based on local Gabor contrast (see Fig. 4G showing such a fit in an example network). E & J: The joint distribution of R² and the suppression index of the center E sub-population. Only a very small minority of sampled non-columnar networks produced R² > −1 and R² > 0 to appear in H and; hence the small n’s, corresponding to 1.4% and 1% of samples, respectively

Figure 6.

Gamma peak contrast dependence and surround suppression in an example non-columnar retinotopic SSN without a boost in intra-columnar E-connections. Panel descriptions are the same as for the right three columns in Fig. 4. Among 2000 sampled non-columnar networks, this network was the best sample we found in terms of capturing the local contrast dependence of gamma peak frequency (subject to having a nonzero suppression index, and producing realistic non-multiple gamma peaks in the LFP power-spectra which are not unbiologically sharp). However, as seen in panel D, this network only yielded an R² of 0.61, as an index of locality for the gamma peak’s contrast dependence, and gamma peak frequency stayed roughly constant over most of the area covered by the Gabor stimulus (panel B).

We then asked whether connectivity structures which involve a sum of strong, spatially smooth long-range connections and an additional boost to local, intra-columnar connectivity could produce both of these effects. In such a structure, the connection strength between two units first undergoes a sharp drop when the distance between the units exceeds the width of a column, and then falls off smoothly over a longer distance, possibly ranging over several columns. This modification can be thought of as adding a local, intra-columnar-only component to a connectivity profile with smooth fall-off. Specifically, we let the excitatory horizontal connections in our model have such a form. Denoting the strength of connection from the unit of type b at location y to the unit of type a at location x by W_x,a∣y,b (with, a, b ∈ {E,I}), we thus chose:

$$W_{\mathbf{x},a\mid \mathbf{y},E}\propto {\lambda }_{a,E}{\delta }_{\mathbf{x},\mathbf{y}}+\left(1-{\lambda }_{a,E}\right)e^{\frac{-\parallel \mathbf{x}-\mathbf{y}\parallel }{{\sigma }_{a,E}}}\hspace{1em}(a\in \{E,I\left\}\right),$$ (10)

where δ_x,y (the local component) is the Kronecker delta: 1 when x = y and zero otherwise. The λ_E,E and λ_l,E parameters lie between 0 and 1, and interpolate between the two extremes of connectivity structure: for λ_a,E = 0 the horizontal connectivity profile has only one spatial scale and falls off smoothly with distance, while for λ_a,E = 1 connectivity is purely local and intra-columnar. The orange lines in Fig. 4A show examples of this connectivity profile. Below we will refer to horizontal excitatory connectivity structures with nonzero (and significant) λ_a,E as columnar, and to those with λ_a,E = 0 as non-columnar; we will also refer to SSN models with these connectivity types as “columnar” and “non-columnar” models or networks, respectively, for short. As in previous work (Rubin et al., 2015), we chose a smooth gaussian profile for inhibitory connections (see Eq. (51)), with a relatively short range (see the cyan lines in Fig. 4A); thus inhibitory projections essentially only targeted the source column.

In Fig. 4B–H, we show the behavior of firing rates and the gamma peak in an example columnar network (with λ_E,E = 0.72 and λ_l,E = 0.70) in response to different stimuli. We first presented this network with flat gratings of varying sizes and contrasts, and measured the LFP power-spectrum and the firing rate responses of the E and I sub-populations at the “center” column, i.e., at the retinotopic location on which the grating was centered. Firing rates of center E and I both increased with contrast (Fig. 4D), and for large enough gratings, we verified that the gamma peak frequency also increases with increasing contrast (Fig. 4B–C), matching the results of the reduced 2-population model and our previously built intuition. To study surround suppression, we formed the so-called size-tuning curve of the center E and I populations, based on their responses to full-contrast flat gratings of different sizes (Fig. 4H). Both E and I responses showed surround suppression: the response first grows but then drops with increasing grating size. The center E sub-population had a suppression index (SI, see Methods for the definition; SI= 0 is no suppression, = 1 is complete suppression) of 0.33 consistent with biologically reported values of suppression indices (Gieselmann and Thiele, 2008).

To study the locality of contrast dependence, we modeled the experiment of Ray and Maunsell 2010, and presented this network with a Gabor stimulus, which has spatially varying contrast with a gaussian profile. We then computed the LFP spectrum at five locations (“columns”) of increasing distance from the center of the Gabor stimulus (the colored squares in Fig. 4E) with the farthest one lying at 0.8 degrees of visual angle from the Gabor center (compare with the Gabor’s σ which was 0.5° as in Ray and Maunsell 2010). The LFP power-spectra at all locations are shown in Fig. 4F. As seen, the gamma peak moves to lower frequencies with increasing distance from the Gabor center, which is accompanied by a decrease in the local contrast (i.e. the contrast of the Gabor stimulus at the receptive field location of the recording site). To quantify the locality of this contrast dependence, we again followed Ray and Maunsell 2010, by comparing the actual peak frequency at location x with a prediction that solely depended on the local contrast, c(x), of the Gabor stimulus at x. The prediction was the peak frequency of gamma recorded at the center location, when the network is presented with a large flat grating of (uniform) contrast equal to c(x). We found that the prediction was in very close agreement with the actual peak frequencies at all distances (Fig. 4F). As a measure of the locality of gamma contrast-dependence, we used the corresponding coefficient of determination, R², which quantifies the agreement between the predicted and actual peak frequencies. (By definition, $R^2=1-\frac{\text{SSE}}{\text{Var}}$, where SSE denotes the sum of squared differences between the predicted and the actual gamma peak frequencies, and Var denotes the variance of the latter; R² is thus bounded above by 1, which is attained when the prediction perfectly matches the actual data.) In the example shown in Fig. 4F, we found R² = 0.98.

To investigate whether the above behavior did or did not require fine tuning of network parameters, we simulated 2000 networks with randomly picked parameters. There were 11 parameters in total, characterising the strengths and ranges of horizontal connections, including A λ_a,E, the strength of feedforward connections, and the ratio of NMDA to AMPA in recurrent excitatory synapses. These parameters were were picked randomly and independently (except for the enforcement of three inequality constraints, similar to the samplings of the two population model in Fig. 2) across wide ranges of values consistent with biological estimates; see Methods and Table 1 for details. Again, similar to the sampling of the two-population model, sampled networks which failed to reach a stable steady-state in any stimulus condition were discarded.

Table 1.

Parameters of models used in different figures.

Parameter	Fig. 1	Figs. 2–3	Fig. 4	Fig. 5	Fig. S1	Unit	Description
n	2					-	power-law I/O exponent
k	0.0194					mV−2 · ms	power-law I/O pre-factor
τ corr	5					ms	noise correlation time
τ AMPA	5					ms	AMPA decay time
τ GABA	7					ms	GABAA decay time
τ NMDA	100					ms	NMDA decay time
ρ N	0.39	0 – 0.5	0.39	0.3 – 0.5	0.45	-	NMDA share of excitation
J EE	124	100 – 300	124	100 – 300	165	mV	total E → E connection weight
J IE	116	100 – 300	116	100 – 300	123	mV	total E → I connection weight
J EI	103	50 – 150	103	50 – 150	114	mV	total I → E connection weight
J II	59.3	50 – 150	59.3	50 – 150	57.1	mV	total I → I connection weight
g E	21.9	10 – 30	21.9	10 – 30	21.7	mV/s	E feedforward current per 1% contrast
g I	10.3	5 – 15	10.3	5 – 15	10.6	mV/s	I feedforward current per 1% contrast
λEE	-	-	0.72	0.25 – 0.75†	0	-	locality of E → E connections
λIE	-	-	0.70	0.25 – 0.75†	0	-	locality of E → I connections
σ EE	-	-	0.296	0.15 – 0.60	0.265	mm	range of E → E connections
σ IE	-	-	0.554	0.15 – 0.60	0.294	mm	range of E → I connections
σ EI	-	-	0.09			mm	range of I → E connections
σ II	-	-	0.09			mm	range of I → I connections
N col	1	1	172			-	number of cortical columns
L	-	-	6.4			mm	retinotopic network width
Δx	-	-	0.4			mm	cortical column width
M	-	-	2			mm/degrees	cortical magnification factor
w RF	-	-	0.04°			degrees	grating input’s margin width
σ Gabor	-	-	0.5°			degrees	Gabor stimulus sigma

In figures 2,3 and 5, parameters were sampled independently and uniformly from the ranges given in the table, except for enforcing three inequality constraints (i.e., sampled parameter sets violating any of these inequalities were rejected). See the main text (Methods) for details.

^†:these were the ranges for sampled λ_EE and λ_IE of the columnar model; these parameters were zero in the non-columnar model.

The vast majority of sampled networks produced biologically plausible center excitatory and inhibitory firing rates which increased with increasing contrast (E and I firing rates did not exceed 100 Hz in, respectively, 90% and 81% of networks). The majority of networks produced surround suppression in excitatory (70% of samples) and inhibitory (53% of samples) populations, with many networks yielding strong suppression in both populations (Fig. 5B). In addition, in response to grating stimuli (with uniform contrast profile), many networks produced gamma-band peaks in the LFP power spectrum that moved upward in frequency with increasing contrast (Fig. 5A and D; the number of networks yielding gamma peak for each nonzero contrast is denoted in panel A). Finally, for each network, we again quantified the locality of contrast dependence of peak frequency, using the R² coefficient for the match between peak frequencies obtained at different recording locations on the Gabor stimulus, and their predictions based on the local Gabor contrast and the peak frequency obtained using the flat grating with that contrast. A sizable fraction of networks resulted in a high R² signifying local contrast-dependence of gamma peak frequency (Fig. 5C). Moreover, many of these networks exhibited strong surround suppression as well (Fig. 5E). Sixty six networks (4.2% of samples) yielded an R² > 0.8 and SI_E > 0.25.

In sum, the columnar model, which emphasizes the intra-columnar excitatory connectivity (Eq. (10)), can robustly exhibit strong surround suppression in conjunction with gamma peak frequencies controlled by the local contrast, as observed empirically, without requiring a fine tuning of parameters.

Retinotopic SSNs with non-columnar excitatory connectivity do not account for local contrast dependence of gamma frequency.

To further show the importance of a boost in intra-columnar excitatory connectivity for obtaining local contrast dependence despite strong surround suppression, we next sampled retinotopic SSN models without this structure in horizontal connections (“non-columnar” model). In these models horizontal excitatory connections fall off smoothly over distance between the source and target columns (corresponding to λ_a,E = 0 in the notation of Eq. (10)). We found that while many sampled non-columnar models exhibited strong surround suppression (on average stronger than in the sampled columnar models) none of the sampled models exhibited gamma peak frequencies with sufficiently local contrast-dependence.

The sampled non-columnar networks robustly exhibited surround suppression which was strong in a large fraction of these networks, and, especially in excitatory units, was on average stronger than in the sampled columnar models (Fig. 5G vs. B). Many networks produced LFP power-spectrum peaks in the gamma band with frequency increasing with contrast (Fig. 5F and I), but sampled networks with these properties were about half as common as in the case of the columnar model (Fig. 5A and D). Moreover, when presented with the Gabor stimulus, we found that the contrast dependence of gamma peak frequency in the vast majority of non-columnar networks was far from local, and the same peak frequency was shared across most of the retinotopic region stimulated by the Gabor stimulus (see the power spectra of an example sampled non-columnar network in Fig. 6A–B). Only 1% of sampled non-columnar networks exhibited a positive R² (our measure of local contrast dependence), compared to 15% of columnar networks (Fig. 5H and J). Only 4 non-columnar networks (out of 2000 sampled networks) exhibited an R² > 0.6 in conjunction with any degree of surround suppression (positive suppression index) of E firing rates (Fig. 5J). However, three of these networks which had the highest R²’s exhibited non-biological gamma-band power spectra, featuring either multiple gamma peaks or unrealistically sharp ones. The fourth network produced realistic gamma peaks and achieved an R² = 0.61; the LFP power spectra, gamma frequency contrast dependence, and size tuning curves for this example network are shown in Fig. 6.

We conclude that the non-columnar model class cannot robustly exhibit both surround suppression of firing rates and local contrast dependence of gamma peak frequency.

Mechanism underlying the local contrast dependence of gamma peak frequency in the columnar SSN

We can understand the mechanism underlying the local contrast dependence of gamma frequency in the columnar model, and its failure in the non-columnar model, by looking at the spatial profile of the normal oscillatory modes of these networks. Normal modes are the eigenvectors of the Jacobian matrix, the effective connectivity matrix of the linearized network introduced before Eq. (6), and normal oscillatory modes are the Jacobian eigenvectors with complex eigenvalues. (As described above, the oscillation frequency is given by the imaginary part of the eigenvalue, and thus we are particularly interested in the eigenvectors of eigenvalues with imaginary part in the gamma band.) The Jacobian is dependent on the operating point of the linearization, which is in turn set by the stimulus. The relevant stimulus condition for us is the Gabor stimulus, or more generally a stimulus with non-uniform contrast.

As we discussed above, gamma peak frequency would be trivially determined by local contrast in a network with only local connectivity (corresponding to λ_Ae = 1 in our model) and disconnected cortical columns. The disconnected columns act like the 2-population model of the first part, and can oscillate independently of other columns by a frequency set by the operating point of that column, which is in turn set by the stimulus input to that column and thus the local contrast. In such a network, all Jacobian eigenvectors are completely localized spatially at a single column. Since the mode is localized, its eigenvalue and hence its natural frequency, are entirely determined by the stimulus contrast over that column.

By contrast, in the model with long-range connections and λ_aE = 0 (the non-columnar model) the eigenvectors can spatially cover a large region of retinotopic space, and lead to coherent and synchronous oscillations at the same frequency (set by that mode’s eigenvalue) across many columns. To see this, consider the case of a stimulus with uniform contrast. Such a stimulus does not break the network’s translational symmetry (since recurrent connections do not care about absolute location, and only depend on relative distance of pre- and post-synaptic columns). Due to this symmetry all eigenmodes are completely delocalized and have (sinusoidal) plane wave spatial profiles extending over the entire retinotopic space. A non-uniform stimulus does break the translational symmetry and leads to relative localization of eigenvectors. However, the scale of this localization is set by the scale over which the stimulus contrast varies appreciably. If this variation is smooth, the eigenvectors can still cover a large region. In the case of the Gabor stimulus, the σ of the Gaussian profile of this stimulus sets this length scale. Thus eigenvectors tend to cover much of the space covered by the stimulus. This can be seen in Fig. 7D showing oscillatory eigenvectors of an example non-columnar network, for modes which maximally contribute to the gamma peaks recorded at different locations. When such an eigenvector has a complex oscillatory eigenvalue (which is sufficiently close to the imaginary axis so that its oscillations are not strongly damped), it will give rise to coherent gamma oscillations across the area covered by the stimulus, at the same frequency set by the mode’s eigenvalue. The corresponding peak will thus appear at the same frequency in the power-spectra of the LFP recorded across this space, despite smooth variations in local contrast (Fig. 7C top). This mechanism thus breaks the local contrast dependence of peak frequency.

Figure 7.

Mechanism of local contrast dependence in the retinotopic SSN with columnar structure and its failure in the non-columnar model. The left and right columns correspond to the columnar (retinotopic SSN with boosted local, intra-columnar excitatory connectivity) and non-columnar (retinotopic SSN without the boost in local connectivity) example networks in Fig. 4 and Fig. 6, respectively. A: Top: relative LFP power spectra recorded at different probe locations in the Gabor stimulus condition (see 4E; same colors are used here to denote the different LFP probes). Relative power spectrum is the point-wise ratio of the evoked power spectrum (evoked by the Gabor stimulus) to the spontaneous power spectrum in the absence of visual stimulus (the absolute power spectrum for the same conditions was given in Fig. 4F). Bottom: the eigenvalue spectrum in the complex plane, with real and imaginary axis exchanged so that the imaginary axis aligns with the frequency axis on top (eigenvalues are also scaled by 1/(2π) to correspond to non-angular frequency). The eigenvalues were weighted separately for each probe, according to Eq. 11, and the eigenvalue with the highest weight was circled with the probe’s color (see 4E). This eigenvalue contributes the strongest peak to the power spectrum at that probe’s location. B: Each sub-panel corresponds to one of the probe locations (as indicated by the frame color), and plots the absolute value of the highest-weight eigenvector (more precisely, the function |ℛ_a(x)| defined in Eq. (37)) over cortical space. Thus, this is the eigenvector corresponding to the circled eigenvalue in panel A, bottom. The λ-rank in each sub-panel is the order (counting from 0) of the eigenvalue according to decreasing imaginary part, which is the eigen-mode’s natural frequency. The green dot in each sub-panel shows the location of the LFP probe. C-D: Same as A and B, but for the retinotopic SSN model with no columnar structure.

Our columnar model, via its parameters λ_aE, interpolates between disconnected networks and the kind of network just discussed. With a sufficient boost of intra-columnar connectivity (i.e., with sufficiently large λ_aE) the eigenvectors of this model become approximately localized, not to single columns, but to a small number of columns receiving similar stimulus contrasts. This is shown in an example columnar network in Fig. 7B, showing different Jacobian eigenvectors (all for the Gabor stimulus condition) which are approximately localized at different locations. Even those eigenvectors that are relatively more spread, tend to extend over rings encircling the Gabor’s center, and thus cover an area receiving the same contrast. The eigenvalue and natural frequency of such modes is thus largely controlled by that contrast value: modes that do not extend to a given location do not contribute to the power-spectrum recorded at that location, while modes that are localized nearby only “see” the local contrast.

This observation further explains the typical shape of the eigenvalue spectrum observed in columnar networks. As seen in Fig. 7A (bottom) the eigenvalue spectrum consists of a near-continuum of eigenvalues extending along the imaginary axis. Eigenvalues with higher (lower) imaginary parts (i.e., the corresponding modes’ natural frequency) have eigenvectors localized at regions of higher (lower) contrast. In this way, eigenvalues with different imaginary parts roughly correspond to different locations that have different local contrasts, with imaginary part decreasing with local contrast. By contrast, oscillatory eigenvalues in non-columnar networks, especially eigenvalues near the imaginary axis within the gamma band, tend to be isolated (Fig. 7C bottom shows this in an example non-columnar network); the corresponding mode cannot be associated with a given location or contrast. Indeed since the eigenvectors of these modes extend over many columns receiving varying contrasts, the mode’s eigenvalue and oscillation frequency are not determined by stimulus contrast at any single column, but rather by the entire spatial profile of contrast, in a complex manner.

The above qualitative discussion can be made quantitative using the linearized approximation. In Methods (see Eqs. (39) and (41)), we derive an expression for the power-spectrum at a location as a sum of individual contributions by different eigenmodes.² As show in Eq. (41), mode a, with eigenvalue λ_a, contributes a peak to the power spectrum located at frequency given by the imaginary part of λ_a (the mode’s natural frequency). The half-width of the peak is given by minus the real part of the eigenvalue, which we denote by γ_a. Finally, the peak amplitude is proportional to

$$\frac{{\left|{ℛ}_a\left(\mathbf{x}\right)\right|}^2}{{\gamma }_a^2},$$ (11)

where ℛ_a(x) is the component of the mode’s right eigenvector at column x’s E population (after summing the components corresponding to different synaptic receptors; see Eq. (37)). Thus this peak only leaves an imprint on the LFP power-spectrum in locations where this eigenvector has appreciable components. The amplitude is also inversely related to the square of the peak’s half-width, γ_a, which by definition measures the distance between the eigenvalue and the imaginary axis. Thus eigenvalues closer to this axis produce stronger and sharper peaks, which appear in the LFP spectrum probed at location x, only if the corresponding right eigenvector has strong (excitatory) components there. In Fig. 7, separately for each of the five LFP probe locations on the Gabor stimulus, we picked the mode with the highest amplitude as defined in Eq. (11). The corresponding eigenvalue is circled in the eigenvalue spectrum plots (bottom plots in Fig. 7A and C for the columnar and non-columnar example models, respectively). In Fig. 7B and D we then plot the corresponding eigenvectors; more precisely, we have plotted |R_a(x)| which, according to Eq. (11), controls the strength of a mode’s contribution at different locations x. As observed, in the non-columnar model, the eigenvectors spread over the entire region covered by the Gabor stimulus. Thus the same mode makes the strongest contribution to the LFP spectra (shown in Fig. 7A and C, top plots) at all probe locations, except for the one that is farthest from the Gabor center. Since this best mode is shared across the first four probe locations, its fixed eigenvalue (Fig. 7C, bottom plot) determines the location of the power-spectrum peak (Fig. 7C, top plot) in all but the farthest probe location.

By contrast, in the columnar model, eigenvectors cover a considerably smaller area within which contrast varies little. As a result, each mode only affects the LFP power spectrum locally, and when the probe moves, the best mode changes quickly, as if the best eigenvector “moves” with the probe (Fig. 7B). In turn, the corresponding best eigenvalues also move to lower frequencies along the imaginary axis (Fig. 7A, top), as the probe moves farther from the Gabor center, according to the local contrast “seen” by their eigenvector.

Inter-columnar projections in the columnar model are nevertheless sufficiently strong to be able to give rise to strong surround suppression, as evidenced above. It is also worth noting that for large gratings that give rise to surround suppression, the contrast is uniform over a broad area, in which case even the eigenvectors of the columnar model tend to cover a broad area (mathematically, this is because when stimulated with a uniform stimulus, the columnar model also has approximate translational invariance, and therefore its eigenvectors tend towards delocalized approximate plane waves).

In summary, the columnar model can balance the requirements for locality of gamma contrast dependence and strong surround suppression, because of the intermediate spatial spread of its eigenvectors, which tend to cover relatively small areas with roughly uniform contrast.

Discussion

In this work we have shown that the expanded SSN is able to robustly display the contrast dependence of gamma peak frequency in both a two-population and a retinotopic network. The retinotopic model successfully balances the trade-off in horizontal connection strength such that both the local contrast dependence of the gamma peak frequency and the surround suppression of firing rates are observed robustly. In order to capture gamma oscillations using the SSN, we expanded the model beyond an E-I network to a varied synaptic network model. Crucially, the SSN account sheds light on the mechanism underlying the contrast dependence of gamma peak frequency and points to the key role of the non-saturating and expansive neural transfer function, observed empirically (Finn et al., 2007; Priebe and Ferster, 2005), in giving rise to this effect.

Finding the power-spectra using the linearization to Eqn. 1, helped us make analytic simplifications. From these simplifications, we gained insights on how the SSN captures the gamma contrast dependence. As contrast increases, firing rates increase, which, due to the supralinear neural transfer function of SSN, lead to increasing neural gains. This in turn strengthens effective connectivity, leading to faster oscillations. Moreover, by finding the power-spectra via linearization, we were able to rapidly compute power-spectra which allowed for extensive explorations of the model’s parameter space.

In this work, for simplicity, we assumed an instantaneous I/O function between net synaptic input (∑_β h_β) and the output rate. This is based on the approximations discussed in (Fourcaud and Brunel, 2002), which is valid when the fast synaptic filtering time-constants (τ_AMPA and τ_GABA) are much smaller than the neuronal membrane time-constants. However, our framework can easily be generalized beyond this approximation by using the full neuronal linear response filter obtained from the Fokker-Planck treatment of (Fourcaud and Brunel, 2002). The main change due to such a modification would be to render the neural gains frequency dependent. We expect this dependence to be weak because we are in the regime of fast synaptic filtering as compared to the neuronal membrane time constant, and so we expect the transfer function to be approximately instantaneous (Fourcaud and Brunel, 2002). Therefore we do not expect that including the full neuronal linear response filter would change our qualitative results.

As we have shown, retinotopic SSN networks account for the co-existence of the local contrast dependence of gamma peak frequency and strong surround suppression by balancing long-range horizontal connection strength that decreases exponentially with distance with an additional strengthening of very local excitatory connections. One possible scenario is that the additional local connection strength is needed to compensate for the possible effects of the model’s coarse retinotopic grid which might have distorted the functional effects of a connectivity profile with a smooth, single-scale fall-off, without a local boost. Alternatively, this may represent a measurable increase in connection strength above an exponential function of distance at short distances, below ~200–400μm. Indeed, it has been previously noted that anatomical findings on the spatial profile of horizontal connections in the macaque cortex point to such a mixture of short-range or local and long-range connections, with the local component not extending beyond 400μm (the size of our model’s columns) (Voges et al., 2010).

It is notable that, in modeling surround suppression in V1, Obeid and Miller (2021) also found that they needed to increase the central weight strengths, relative to an exponential fall-off of strength on a grid, for their SSN model to account for two other observations. These observations were the decrease in inhibition received by a cell when it is surround suppressed (Adesnik, 2017; Ozeki et al., 2009), and the fact that the strongest surround suppression occurs when surround orientation matches center orientation, even if the center orientation is not the cell’s preferred orientation (Shushruth et al., 2012; Trott and Born, 2015). Other phenomena they addressed could be explained by their SSN model with or without this extra local strength. We believe that other visual cortical phenomena previously addressed with the SSN model (Adesnik, 2017; Hennequin et al., 2018; Liu et al., 2018; Rubin et al., 2015) would not be affected by such boosting, as the mechanisms inferred behind them appear independent of these connectivity details.

Recently some evidence of enhanced connectivity at very short distances (~20μm) has been found in mouse V1 (Oldenburg et al., 2022). Optogenetic stimulation of ten cells found excitation of nearby cells only at such short distances from one of the stimulated cells, with suppression at longer distances. In a model, this required an extra component of connectivity that decreased with distance on a very short length scale, in addition to one with a longer length scale. As they point out, such extra local strength might account for the observation that preferred orientations in mouse visual cortex are correlated on a similar very short length scale (Kondo et al., 2016). It will be interesting to see if evidence of such a short-length scale component of connectivity is evident in monkeys, where the local contrast dependence of gamma was measured, and conversely to see if mice show similar local contrast dependence of gamma.

Methods

Stabilized supralinear network (SSN) with different synaptic receptor types

In its original form, the Stabilized Supralinear Network (SSN) is a firing rate network of excitatory and inhibitory neurons that have a supralinear rectified power-law input/output (I/O) transfer function:

$$F\left(h\right)=k[h{]}_{+}^n$$ (12)

where n > 1 and [h]₊ ≡ max(0,h) denotes rectification of h. The dynamics can either be formulated in terms of the inputs to the units (Hennequin et al., 2018) or in terms of their output firing rates (Ahmadian et al., 2013; Rubin et al., 2015). Here we adopt the former case for which the dynamical state of the network, in a network of N neurons, is given by the N-dimensional vector of inputs vv_t, which evolves according to the dynamical system

$$\text{T}\frac{d{\mathbf{h}}_t}{dt}+{\mathbf{h}}_t=WF\left({\mathbf{h}}_t\right)+{\mathbf{I}}_t.$$ (13)

Here, I is the external input vector, T = diag(τ₁, …, τ_N) is the diagonal matrix of synaptic time constants, and f acts element-wise. Finally, W is the N × N matrix of recurrent connection weights between the units in the network. This connectivty matrix observes Dale’s law, meaning the sign of the weight does not change over columns. If we order neurons such that excitatory neurons appear first and inhibitory neurons second, this matrix takes the form

$$W=\left(\begin{array}{ll}{W}_{EE}& -W_{EI}\\ W_{IE}& -W_{II}\end{array}\right)$$ (14)

where W_XY (X, Y ∈ [E, I]) have non-negative elements.

The above model does not take into account the distinct dynamics of currents through different synaptic receptor channels: AMPA, GABA_A (henceforth GABA), and NMDA. Only the fast receptors, AMPA and GABA, have timescales relevant to gamma band oscillations. These receptors have very fast rise times, which correspond to frequencies much higher than the gamma band. We therefore ignored the rise times of all receptors. With this assumption, upon arrival of an action potential in a pre-synaptic terminal at time t = 0, the post-synaptic current through receptor channel α (α ∈ {A = AMPA, G = GABA, N = NMDA}) with decay time τ_α is given by $w\frac{\theta \left(t\right)}{{\tau }_{\alpha }}{e}^{-t/{\tau }_{\alpha }}$ where θ(t) is the Heaviside step function and w_α is the contribution of receptor α to the synaptic weight. This is the impulse response solution to the differential equation ${\tau }_{\alpha }\frac{d{h}_{\alpha }}{dt}+h_{\alpha }=w\delta \left(t\right)$, where δ(t) is the Dirac delta representing the spike at time t = 0. In the mean-field firing rate treatment, the delta function is averaged and is replaced by a smooth rate function r(t). Extending this to cover post-synaptic currents from all synapses into all neurons we obtain the equation

$${\tau }_{\alpha }\frac{d{\mathbf{h}}_t^{\alpha }}{dt}+{\mathbf{h}}_t^{\alpha }=W^{\alpha }{\mathbf{r}}_t$$ (15)

where r_t and h^α are N-dimensional vectors of the neurons’ firings rates and input currents of type α, respectively, and W^α are N × N matrices containing the contribution of receptor type α to the recurrent synaptic weights. If we add an external input to the right side (before filtering by the synaptic receptors), we obtain Eq. (1). Since AMPA and NMDA only contribute to excitatory synapses, and GABA only to inhibitory ones, in general the W^α have the following block structure

$$\begin{array}{l}{W}^{\text{A}}=\left(\begin{array}{ll}{W}_{EE}^{\text{A}}& 0\\ W_{IE}^{\text{A}}& 0\end{array}\right),W^{\text{N}}=\left(\begin{array}{ll}{W}_{EE}^{\text{N}}& 0\\ W_{IE}^{\text{N}}& 0\end{array}\right),\\ W^{\text{G}}=\left(\begin{array}{ll}0& -W_{EI}^{\text{G}}\\ 0& -W_{II}^{\text{G}}\end{array}\right).& \end{array}$$ (16)

For simplicity, we further assumed that the fraction of NMDA and AMPA is the same in all excitatory synapses. In this case all W^α can be written in terms of the four blocks of the full connectivity matrix W ≡ ∑_αW^α, introduced in Eq. (14):

$$\begin{array}{l}{W}^{\text{N}}=\frac{{\rho }_{\text{N}}}{1-{\rho }_{\text{N}}}{W}^{\text{A}}={\rho }_{\text{N}}\left(\begin{array}{ll}{W}_{EE}& 0\\ W_{IE}& 0\end{array}\right),\\ W^{\text{G}}=\left(\begin{array}{ll}0& -W_{EI}\\ 0& -W_{II}\end{array}\right).\end{array}$$ (17)

where the scalar ρ_N is the fractional contribution of NMDA to excitatory synaptic weights.

As noted in Results, to close the system of equations for the dynamical variables ${\mathbf{h}}_t^{\alpha }$, we have to relate the output rate of a neuron to its total input current, ${\mathbf{h}}_t^{\text{total}}={\sum }_{\beta }{\mathbf{h}}_t^{\beta }$. In general, the relationship between the total input and the firing rate of a neuron, or the mean firing rate of a population of statistically equivalent neurons, is nonlinear and dynamical, meaning the rate at a given instant depends on the preceding history of input, and not just on the instantaneous input. However, as shown by (Brunel et al., 2001; Fourcaud and Brunel, 2002), the firing rate of spiking neurons receiving low-pass filtered noise with fast auto-correlation timescales is approximately a function of the instantaneous input. The fast filtered noise is exactly what irregular spiking of the spiking network generates after synaptic filtering (as in Eq. (15)) by the fast AMPA and GABA receptors. (While our rate model does not explicitly model (irregular) spiking, it can be thought of as a mean-field approximation to a spiking network where each SSN unit or “neuron” represents a sub-population of spiking neurons, with the rate of that unit representing the average firing rate of the underlying spiking population.) We thus use this static approximation to the I/O transfer function and assume the firing rates of our model units are given by Eq. (2): ${\mathbf{r}}_t=F\left({\mathbf{h}}_t^{\text{total}}\right)=F\left({\sum }_{\beta }{\mathbf{h}}_t^{\beta }\right)$, where F(.) is the rectified power-law function of Eq. (12).

We do note, however, that this static approximation can be lifted in a straightforward manner at the level of our linearized approximation (which underlies our qualitative understanding of the contrast-dependence of the gamma peak): upon linearization, a dynamic neural transfer function would result in the neural gain variables (see Eq. (22)) becoming frequency-dependent gain filters. However, as long as those gain filters are feature-less over the gamma band, their frequency dependence would not qualitatively affect the location of the gamma peak and its stimulus dependence. Thus we expect that the static I/O approximation will not alter our qualitative results.

Modelling of gamma oscillations and local field potential

As discussed in the introduction and Results, gamma oscillations are most consistent with noise-driven damped oscillations. We thus assumed the external input consisted of a time-independent term representing the feedforward drive due to a static stimulus, and dynamical noise:

$${\mathbf{I}}_t^{\alpha }={\mathbf{I}}_{DC}^{\alpha }+{\mathit{\eta }}_t^{\alpha }.$$ (18)

Given that external inputs to cortex are excitatory and only fast noise is relevant to gamma oscillations we assumed that ${\mathit{\eta }}_t^{\alpha }$ was only nonzero for α = AMPA. We took η_t to have independent and identically distributed components (with zero mean) across our sub-population units, and took it to be temporally pink noise, with fast correlation time, τ_corr:

$$⟨{\eta }_i\left(t_1\right){\eta }_j\left(t_2\right)⟩={\delta }_{ij}{\sigma }_{\eta }^2{e}^{-\frac{\left|t_1-t_2\right|}{{\tau }_{\text{corr}}}}.$$ (19)

We assume that local field potential (LFP) recordings predominantly measure the inputs to the surrounding pyramidal neurons (Einevoll et al., 2013), and thus in our model use the current input into our excitatory units as the surrogate for LFP. More precisely we take the LFP signal at location x to be the total current input, h^total, averaged over the E neurons within a given distance of x (the average could be weighted with weights that decrease with distance). This can be written as the inner product of h^total with an x-dependent weight vector:

$$LF{P}_t\left(\mathbf{x}\right)\propto {\mathbf{e}}_{\mathbf{x}}\cdot {\mathbf{h}}^{\text{total}}=\sum _{\alpha }{\mathbf{e}}_{\mathbf{x}}\cdot {\mathbf{h}}^{\alpha },$$ (20)

where the weight vector e_x only has nonzero components for E neurons that are within a given radius of location x. In particular, in the two-population model which lacks retinotopy e_x ≡ e = (1,0)^⊤. In the retinotopic model, we assumed that the spatial range of the LFP recording does not exceed the half-width of our the model’s cortical columns (0.2 mm), and therefore we took e_x to be a one-hot vector with the component for the E unit at location x equal to one, and the rest zero.

LFP power-spectra in the linearized approximation

In order to study the power-spectra, and gain intuition about them, we linearized the dynamics around the noise-free fixed point. (Recall that we are modelling gamma as noise-driven damped oscillations, i.e. the network is in a regime where without noise it reaches a stable fixed point.) As shown in Results, the fixed point satisfies Eq. (3). The linear approximation consists of a first-order Taylor expansion in powers of the noise, ${\mathit{\eta }}_t^{\alpha }$, and noise-driven fluctuations, $\delta {\mathbf{h}}_t^{\alpha }\equiv {\mathbf{h}}_t^{\alpha }-{\mathbf{h}}_{*}^{\alpha }$, around the stable fixed point. (Note that while the fixed point equation only involves the total current ${\mathbf{h}}_{*}\equiv {\sum }_{\alpha }{\mathbf{h}}_{*}^{\alpha }$, after numerically finding h_*, we can obtain the fixed-point value of the receptor-specific currents via ${\mathbf{h}}_{*}^{\alpha }=W^{\alpha }F\left({\mathbf{h}}_{*}\right)+I_{DC}^{\alpha }$.) This yields

$${\tau }^{\alpha }\frac{d\delta {\mathbf{h}}_t^{\alpha }}{dt}=-\delta {\mathbf{h}}^{\alpha }+W^{\alpha }\Phi \sum _{\beta }\delta {\mathbf{h}}_t^{\beta }+{\mathit{\eta }}_t,$$ (21)

where we defined the gain matrix Φ as a diagonal matrix whose diagonal entries are

$${\Phi }_{ii}\equiv F^{\mathbf{\text{'}}}\left(h_{i*}\right)=nk{\left[h_{i*}\right]}_{+}^{n-1}=n{k}^{\frac{1}{n}}{\mathbf{r}}_{i*}^{1-\frac{1}{n}}.$$ (22)

Taking the Fourier transform of (21), and solving for $\delta {\tilde{\mathbf{h}}}_f^{\alpha }$ (the Fourier transform of $\delta {\mathbf{h}}_t^{\alpha }$, where f denotes frequency) we obtain

$$\delta {\tilde{\mathbf{h}}}_f^{\alpha }=\sum _{\beta }{G}^{\alpha \beta }\left(f\right){\tilde{\mathit{\eta }}}_f^{\beta },$$ (23)

where the Green’s function, $G_h^{\alpha \beta }\left(f\right)$, is given by

$${\left[G(f{)}^{-1}\right]}^{\alpha \beta }\equiv \left(-i2\pi f{\tau }^{\alpha }+1\right){\delta }_{\alpha \beta }{\mathbb{I}}_{N\times N}-W^{\alpha }\Phi .$$ (24)

where ${\mathbb{I}}_{N\times N}$ is the identity matrix.

Since, by Eq. (20), the LFP is a linear function of h^α, the power-spectrum of LFP can be written in terms of the cross-spectrum matrix of $\delta {\tilde{\mathbf{h}}}_f^{\alpha }$, which we denote by $C_h^{\alpha \beta }\left(f\right)$. Specifically

$$P_{\mathbf{L}\mathbf{F}\mathbf{P}}(f;\mathbf{x})=\sum _{\alpha ,\beta }{\mathbf{e}}_{\mathbf{x}}^{\top }{C}_h^{\alpha \beta }\left(f\right){\mathbf{e}}_{\mathbf{x}}.$$ (25)

Using Eq. (23) and $C_h^{\alpha \beta }\left(f\right)\propto ⟨\delta {\tilde{\mathbf{h}}}_f^{\alpha }\delta {\tilde{\mathbf{h}}}_f^{\beta †}⟩$ we have

$$\begin{gathered} C_h^{\alpha \beta }\left(f\right)=\sum _{\gamma ,\delta }{G}^{\alpha \gamma }\left(f\right)C_{\eta }^{\gamma \delta }\left(f\right)G^{\beta \delta }(f{)}^{†}, \\ =G^{\alpha \text{A}}(f\left)C_{\eta }^{\text{AA}}\right(f\left)G^{\beta \text{A}}\right(f{)}^{†}. \end{gathered}$$ (26)

Here, $C_{\eta }^{\gamma ϵ}\left(f\right)$ is the cross-spectrum of the input noise, and in the second line we relied on our assumption that noise only enters the AMPA channel, and thus only γ = δ = A ≡ AMPA contribute to the sums. From Eq. (19), we have $C_{\eta }^{\text{A},\text{A}}\left(f\right)=P_{\text{noise}}\left(f\right){\mathbf{I}}_{N\times N}$, where

$$P_{\text{noise}}\left(f\right)=\frac{2{\tau }_{\text{corr}}{\sigma }_{\eta }^2}{{\left|1-2\pi i{\tau }_{\text{corr}}f\right|}^2}$$ (27)

is the power-spectrum of noise. We finally obtain

$$P_{\mathbf{L}\mathbf{F}\mathbf{P}}(f;\mathbf{x})=P_{\text{noise}}\left(f\right){∥{\mathbf{u}}_{\mathbf{x}}\left(f\right)∥}^2,$$ (28)

where we defined

$${\mathbf{u}}_{\mathbf{x}}\left(f\right)\equiv \sum _{\beta }{G}^{\beta \text{A}}(f{)}^{†}{\mathbf{e}}_{\mathbf{x}}.$$ (29)

Definition of suppression index, and gamma peak frequency and width

Suppression index was based on the size tuning curve, r(R), of the center E or I units, measured for gratings of 100% contrast. It was defined by

$$\text{SI}=1-\frac{r\left(R_{\text{max}}\right)}{{\text{max}}_{R}r\left(R\right)}$$ (30)

where R is the grating radius, r(R) is the size tuning curve, and R_max is the maximum grating radius used.

As in Ray and Maunsell 2010, we identified the gamma peak frequency with the frequency (within the extended gamma band 10–100 Hz) at which the difference of the evoked (c > 0) and spontaneous (c = 0) LFP log-spectra (or the ratio of those spectra) is maximized:

$$f_{\text{peak}}\left(c\right)=\text{arg}\underset{f}{\text{max}}\left[\text{log}{P}_{\text{LFP}}(f;c)-\text{log}{P}_{\text{LFP}}(f;c=0)\right].$$ (31)

As a measure of gamma peak width (or half-width) at contrast c, we used the half-width at half-height of the relative power spectrum $\frac{P_{\text{LFP}}(f;c)}{P_{\text{LFP}}(f;0)}$.

The eigen-decomposition of the LFP power-spectrum

The linearized dynamics of Eq. (4) can be written in terms of the Jacobian matrix, J as

$$\frac{d\delta {\mathbf{h}}_t^{\alpha }}{dt}=\sum _{\beta }{J}_{\alpha \beta }\delta {\mathbf{h}}_t^{\beta }$$ (32)

where

$$J_{\alpha \beta }\equiv {\tau }_{\alpha }^{-1}\left(-{\delta }_{\alpha \beta }{\mathbb{I}}_{N\times N}+W^{\alpha }\Phi \right).$$ (33)

(with α, β ∈ {AMPA, GABA, NMDA}) is the (α, β) block (an N × N matrix) of the full 3N × 3N Jacobian matrix. The normal modes of this linear system correspond to eigenvectors of the Jacobian, which evolve in time according to e^λt which for λ = γ + Iω₀ with real γ and ω₀ can be written e^−|γ|t(cos ωt + I sin ωt) (we used the fact that stability require that the eigenvalue real part, γ, is negative). We thus see that this mode oscillates at (angular) frequency ω₀ = Imλ, and decays at the rate given by |γ| = −Reλ. (For brevity, in this section we will use the angular frequency ω = 2πf instead of f.) Comparison of Eq. (4) and Eq. (24) shows that the inverse Green’s function can be written in terms of the Jacobian as ${\left[G(f{)}^{-1}\right]}_{\alpha \beta }={\tau }_{\alpha }(-i\omega \mathbb{I}-J{)}_{\alpha \beta }$.Defining a diagonal matrix T with the first, second, and last third of its diagonal elements given by τ_AMPA, τ_GABA and τ_NMDA, respectively. We can write

$$G\left(f\right)=(-i\omega \mathbb{I}-J{)}^{-1}{T}^{-1}$$ (34)

We start by rewriting the Green’s function in terms of the eigen-decomposition of the Jacobian J = V∧V⁽⁻¹⁾, where Λ is the diagonal matrix of eigenvalues, λ_a, and V is a matrix with columns given by corresponding (right) eigenvectors. Equivalently we can write, J = ∑_a λ_aR_aL_a, where the right eigenvectors R_a are the columns of V and the left eigenvectors, L_a, are the rows of V⁻¹. Using this decomposition we obtain

$$G\left(f\right)=\sum _a\frac{1}{-i\omega -{\lambda }_a}{\mathbf{R}}_a{\mathbf{L}}_a{T}^{-1}.$$ (35)

We can then rewrite Eq. (29) as

$$u_{\mathbf{x}}(f{)}^{†}={\tau }_{\text{AMPA}}^{-1}\sum _a\frac{{ℛ}_a\left(\mathbf{x}\right)}{-i\omega -{\lambda }_a}{\mathbf{L}}_a^A$$ (36)

where ${\mathbf{L}}_a^A$ is the row-vector formed by the AMPA components of L_a, and we defined

$${ℛ}_a\left(\mathbf{x}\right)=\sum _{\beta }{\left[{\mathbf{R}}_a\right]}_{\beta ,E,\mathbf{x}}$$ (37)

namely, the (possibly complex) scalar function ℛ_a(x) is the E, x component (component on E subpopulation at column x) of the right-eigenvector after summing over receptor indices. Here we have assumed that the LFP probe is completely local and reflects total current (hence the sum over β) into E (pyramidal) neurons of the recorded column x. The limitation to AMPA components of the left eigenvectors, on the other hand, reflects our assumption that external noise is entering only via AMPA receptors. Substituting in Eq. (28), we then obtain

$$\frac{P_{\text{LFP}}(f;\mathbf{x})}{P_{\text{noise}}\left(f\right)}={∥{\mathbf{u}}_{\mathbf{x}}\left(f\right)∥}^2$$ (38)

$$=\sum _{a,b}{A}_{ab}\left(x\right)\frac{1}{i\omega -{\lambda }_b^{*}}\frac{1}{-i\omega -{\lambda }_a}$$ (39)

where we defined

$$A_{ab}\left(x\right)={\tau }_{\text{AMPA}}^{-2}⟨L_b^A,L_a^A⟩{ℛ}_b^{*}\left(\mathbf{x}\right){ℛ}_a\left(\mathbf{x}\right)$$ (40)

and $⟨L_b^A,L_a^A⟩=L_a^A{\left(L_b^A\right)}^{†}$ is the Hermitian inner product of the two left eigenvectors, within the AMPA subspace. The key factor here is ${ℛ}_b^{*}\left(\mathbf{x}\right){ℛ}_a\left(\mathbf{x}\right)$ which determines the x dependence and can affect the local dependence of power spectrum on information at point x.

Equations 39–40 constitute our main result here. They express the ratio of LFP to noise power spectrum as a sum of contributions by pairs of eigen-modes, and contributions by individual eigen-modes (the latter corresponding to the diagonal summands with a = b).

In particular, the individual contribution of mode a is

$$P_a(f;\mathbf{x})\equiv \frac{A_{aa}\left(\mathbf{x}\right)}{{\left|-i\omega -{\lambda }_a\right|}^2}=\frac{A_{aa}\left(\mathbf{x}\right)}{{\gamma }_a^2+{\left(\omega -{\omega }_a\right)}^2},$$ (41)

where γ_a and ω_a are minus the real and imaginary components of λ_a, and $f=\frac{\omega }{2\pi }$. This contribution is a Lorentzian function that peak at the natural frequency frequency ω_a and has half-width γ_a. The amplitude of this peak is given by

$$\frac{A_{aa}\left(\mathbf{x}\right)}{{\gamma }_a^2}\propto \frac{{\left|{ℛ}_a\left(\mathbf{x}\right)\right|}^2}{{\gamma }_a^2},$$ (42)

where we used Eq. (40). This amplitude is thus proportional to |ℛ_a(x)|², i.e., the squared absolute value of the corresponding right eigenvector component at location x. It is also inversely proportional to the squared half-width which measures the distance between the eigenvalue and the imaginary axis. Thus eigenvalues closer to this axis produce stronger and sharper peaks, which appear in the LFP spectrum probed at location x if the corresponding right eigenvector has strong components at that location.

The sum in Eq. (39) also contains terms each of which can be interpreted as the contribution of a pair of (distinct) modes. When the left-eigenvectors of different modes are orthogonal (according to the inner product defined after Eq. (40), which corresponds to the orthogonality of the AMPA components of the vectors under the common inner product) these contributions vanish. More generally, the contribution of the pair (a, b) can be written as (making use of A_ba(x) = A_ab(x)*)

$$P_{ab}(f;\mathbf{x})\equiv 2\text{Re}{A}_{ab}\left(\mathbf{x}\right)\frac{1}{i\omega -{\lambda }_b^{*}}\frac{1}{-i\omega -{\lambda }_a}$$ (43)

$$=2\frac{N_{ab}^R(f;\mathbf{x})+N_{ab}^I(f;\mathbf{x})}{D_{ab}\left(f\right)},$$ (44)

where we defined

$$D_{ab}\left(f\right)=\left({\gamma }_a^2+{\left(\omega -{\omega }_a\right)}^2\right)\left({\gamma }_b^2+{\left(\omega -{\omega }_b\right)}^2\right),$$ (45)

and

$$N_{ab}^R(f;\mathbf{x})=\text{Re}\left[A_{ab}\left(\mathbf{x}\right)\right]\left({\gamma }_a{\gamma }_b+\left(\omega -{\omega }_a\right)\left(\omega -{\omega }_b\right)\right),$$ (46)

$$N_{ab}^I(f;\mathbf{x})=\text{Im}\left[A_{ab}\left(\mathbf{x}\right)\right]\left({\gamma }_a\left(\omega -{\omega }_b\right)-{\gamma }_b\left(\omega -{\omega }_a\right)\right).$$ (47)

Alternatively, the pair contribution P_ab(f; x) is given by the product of the individual contributions P_a(f; x) and P_b(f; x), with a correction factor given by $2A_{aa}^{-1}\left(\mathbf{x}\right)A_{bb}^{-1}\left(\mathbf{x}\right)\left(N_{ab}^R(f;\mathbf{x})+N_{ab}^{\text{'}}(f;\mathbf{x})\right)$.

Parametrization of the two-population and retinotopic models

The 2 × 2 (full) connectivity matrix of the 2-population model is parametrized by the four parameters J_ab (a, b ∈ {E, I}) as follows:

$$W=\left(\begin{array}{ll}{J}_{EE}& -J_{EI}\\ J_{IE}& -J_{II}\end{array}\right).$$ (48)

The DC stimulus input corresponds to feedforward excitatory inputs from LGN and targets both sub-populations only via the AMPA channel (since this input is time independent, its distribution across NMDA and AMPA channels is actually of no consequence). As in the original SSN, we assumed this input scales linearly with contrast, c, but with varying relative strengths to the E and I captured by the two parameters g_E and g_l:

$${\mathbf{I}}_{DC}=c\left(\begin{array}{c}{g}_E\\ g_I\end{array}\right).$$ (49)

In the retinotopic model we index the neurons by their E/I type and retinotopic location. We parametrized the recurrent connection weight from the pre-synaptic E and I units at location y to the type a(a ∈ [E, I]) post-synaptic unit at location x by

$$W_{\mathbf{x},a\mid \mathbf{y},E}\propto J_{a,E}\left[{\lambda }_{a,E}{\delta }_{\mathbf{x},\mathbf{y}}+\left(1-{\lambda }_{a,E}\right)e^{-\frac{\parallel \mathbf{x}-\mathbf{y}\parallel }{{\mathit{\sigma }}_{a,E}}}\right]$$ (50)

for excitatory projections, and

$$W_{\mathbf{x},a\mid \mathbf{y},I}\propto J_{a,I}{e}^{-\frac{(\mathbf{x}-\mathbf{y}{)}^2}{2{\sigma }_{a,l}^2}},$$ (51)

for inhibitory ones. We are using proportionality instead of equal signs in the above equations, because a normalization was done such that the total weight of each type received by a unit was given by the corresponding J_ab (independent of the σ_ab and λ_ab parameters). Recurrent connectivity was thus parametrized by the 2×2 matrices J_ab and σ_ab, the two λ_a,E, and the NMDA fraction, ρ_N,11 parameters in total. For σ_∥ and σ_El we used values (see Table 1) small compared to the distance between neighboring columns (0.4 mm) so that inhibition was effectively local (i.e., intra-columnar); we did not vary σ_II and σ_EI across our randomly sampled networks.

We modeled the external stimulus input to the type a unit at x by

$$I_{a,\mathbf{x}}^{DC}=c{g}_a{I}_{\mathbf{x}}\hspace{1em}(a\in \{E,I\left\}\right).$$ (52)

where g_E and g_I reflect the relative strengths of feedforward connections received by V1’s excitatory and inhibitory networks, and I_x captures the spatial profile of the visual stimulus. For a grating of radius r_grat we modeled the spatial contrast profile, I_x, as

$$I_{\mathbf{x}}=\frac{1}{1+e^{\frac{\parallel \mathbf{x}\parallel -r_{\text{grat}}}{w_{\text{RF}}}}}.$$ (53)

The parameter w_RF smooths the edges of the grating (due to feedforward filtering by receptive fields of width ~ w_RF). Note that for r_grat ≫ w_RF (which was true for most grating sizes employed), local contrast is nearly uniform under the support of the external input (except within a “boundary layer” of width w_RF near ∥x∥ = r_grat). For the Gabor stimulus we have

$$I_{\mathbf{x}}=e^{-\frac{\parallel \mathbf{x}{\parallel }^2}{2{\sigma }_{\text{Gabor}}^2}}.$$ (54)

We took σ_Gabor = 0.5°, as in Ray and Maunsell 2010, and the peak contrast (c in Eq. (52)) was always 100% for this stimulus.

Model parameters and parameter sampling

See Table 1 below for the values of all parameters or parameter ranges for models used in different figures. For the models used in Figs. 1 and 4, we found their parameters J_ab and g_a which are shared in both figures, and σ_a,E and λ_a,E for Fig. 4) using random sampling (as further described below) searching for networks that would exhibit the local contrast-dependence of the gamma peak together with strong surround suppression.

For studying the robustness of the contrast dependence of gamma frequency in the two-population model in Fig. 2, and in the case of the retinotopic SSN with a smooth fall-off of excitatory horizontal connectivity in Fig. 6, we sampled parameters from wide biologically plausible ranges. To determine these ranges for the recurrent and feedforward weights, we first made rough biological estimates for the recurrent E and I weights (i.e., J_aE and J_al, respectively, for a ∈ {E, I}), as well as the (excitatory) feedforward weights (g_E and g_I); we denote these estimates by $J_E^{*},J_I^{*}$ and g*, respectively. We then independently varied parameters controlling each type of weight between 0.5 to 1.5 times those estimates (see Table 1 for the actual values).

To come up with the mid-range estimates, $J_E^{*},J_I^{*}$ and g*, we relied on empirical estimates of the effect of recurrent and feedforward inputs on the membrane voltage of a post-synaptic neuron. Note that while J_ab (and thus $J_E^{*}$ and $J_1^{*}$), have dimensions of voltage (such that the recurrent input Wr has our units of current), g_a (and thus g*) have dimensions of current. In our model, we measured the currents in units of mV/s, by including an implicit factor of membrane capacitance in them. The membrane potential response to a unit current is normally given by the membrane resistance, which in our units becomes the membrane time constant, which we take to be τ_m = 0.01 s. So to obtain an estimate of g* from voltage measurements, we need to divide the estimate by τ_m=0.01 s.

We estimated the effect of feedforward inputs on membrane voltage using measurements in cats and mice (Chung and Ferster, 1998; Ferster et al., 1996; Finn et al., 2007; Li et al., 2013; Lien and Scanziani, 2013) (see Ahmadian and Miller 2021 for a review and discussion of these measurements). Based on these measurements, we estimate the maximum feedforward input, achieved for 100% contrast to be on the order of the rest to threshold distance, which is around 20mV (Constantinople and Bruno, 2013). This yields $g^{*}~{\tau }_m^{-1}20\text{mV}/\left(100\%\right)=20$ mV/s per 1% contrast. The J_ab parameters measure the total synaptic weight, which biologically is given by a unitary excitatory or inhibitory (depending on) post-synaptic potential (EPSP or IPSP) times the total number of pre-synaptic V1 neurons, K_b, of type b. Based on anatomical measurements for sensory cortex (reviewed in Ahmadian and Miller 2021) we estimate the effective K_E to be ~400 (with a wide margin of uncertainty). And based on electrophysiological measurements we assume the median EPSP amplitude to be ~0.5mV. This yields $J_E^{*}=0.5\times 400=200\text{mV}$. For unitary IPSP amplitude, we used the same value of 0.5mV, but assumed half as many inhibitory pre-synaptic inputs, due to the smaller number of inhibitory cells in the circuit. We thus took $J_l^{*}=J_E^{*}/2$. In Fig. 5, the four extra parameters controlling the spatial profile of excitatory horizontal connections in the retinotopic SSN were additionally sampled randomly as well. These parameters are λ_EE and λ_IE quantifying the intra-columnar excess connectivity, and σ_IE and σ_EE quantifying the length-scale (range) of the long-range components of excitatory recurrent connections. We sampled the two σ’s uniformly between 150μm and 600μm. The non-columnar model had λ_EE = λ_IE = 0, while for the columnar model we sampled these uniformly from the interval [0.25, 0.75].

Finally, we assumed that recurrent V1 excitatory synapses are dominated by AMPA, rather than NMDA, and therefore sampled ρ_N uniformly at random in the interval [0.3, 0.5].

All parameters were sampled uniformly and independently over their ranges, except for enforcement (by sample rejection) of three inequality constraints:

$$\begin{array}{c}{J}_{EI}{J}_{IE}>J_{EE}{J}_{IE},\\ J_{II}{g}_E>J_{EI}{g}_I,\\ {\sigma }_{IE}>{\sigma }_{EE}.\end{array}$$

Previous work has shown that the first inequality promotes stability (almost a necessary condition) (Ahmadian et al., 2013; van Vreeswijk and Sompolinsky, 1998), the second inequality ensures that the network is not too strongly inhibition-dominated such that excitatory rates become too small (Ahmadian et al., 2013; van Vreeswijk and Sompolinsky, 1998). The last inequality is necessary for obtaining considerable surround suppression (Rubin et al., 2015).

Appendix 1

Proof that the eigenvalue spectrum of a synaptic model with only AMPA and GABA is the same as the spectrum of an E/I rate model

Here we prove that the spectrum of a linearized synaptic model without NMDA is the same as the spectrum of a linearized E/I rate model, with the exchange τ_AMPA → τ_E and τ_GABA → τ_ϝ. This means that, in particular, the formulae of (Tsodyks et al., 1997) for eigenvalues in a 2neuron/population model still hold for this model with the above replacements.

We start by rewriting the inverse Green’s function, using the Green’s function defined in Eq. (24), which we now write in full matrix form. We will also start general, allowing for q different receptor types (we also write in terms of the angular frequency ω = 2πf).

$$G(\omega {)}^{-1}=A-\mathbf{W}\Phi P$$ (55)

where we define

$$A:=-i\omega \mathbf{T}+\mathbb{I}\hspace{1em}\in {\mathbb{R}}^{qN\times qN}$$ (56)

$$\mathbf{W}:=\left(\begin{array}{c}{W}^A\\ W^G\\ W^N\\ ⋮\end{array}\right)\hspace{1em}\in {\mathbb{R}}^{qN\times N}$$ (57)

$$P:={\mathbb{I}}_{N\times N}\otimes {\mathbf{1}}_q^{\top }=\left({\mathbb{I}}_{N\times N},{\mathbb{I}}_{N\times N},{\mathbb{I}}_{N\times N},\dots \right)\hspace{1em}\in {\mathbb{R}}^{N\times qN}$$ (58)

where $\mathbf{T}=\text{diag}\left({\tau }_s\right)\otimes {\mathbb{I}}_{N\times N}$ and ${\tau }_s\in \mathbb{R}$ (in our case τ_s = (τ_A, τ_G, τ_N) or (τ_A, τ_G)), and $1_q^{\top }=(1,\dots ,1)\in {\mathbb{R}}^q$.

The eigenvalue spectrum correspond to values of z = −iω which make the determinant of G(ω)⁽⁻¹⁾ vanish. Noting that the second term in Eq. (55) is rank-deficient (has at most rank N, instead of full-rank qN), we make use of the “matrix determinant lemma” to write:

$$\text{det}\left(G^{-1}\left(\omega \right)\right)=\text{det}\left(A\right)\text{det}\left({\mathbb{I}}_{N\times N}-P{A}^{-1}\mathbf{W}\Phi \right)$$ (59)

It is not hard to see that

$$P{A}^{-1}=\left(\frac{{\mathbb{I}}_{N\times N}}{-i\omega {\tau }_A+1},\frac{{\mathbb{I}}_{N\times N}}{-i\omega {\tau }_G+1},\frac{{\mathbb{I}}_{N\times N}}{-i\omega {\tau }_N+1},\dots \right)$$ (60)

and therefore

$$P{A}^{-1}\mathbf{W}=\sum _{\alpha =1}^q\frac{1}{-i\omega {\tau }_{\alpha }+1}{W}^{\alpha }$$ (61)

We now limit to q = 2 with only AMPA and GABA.

$$P{A}^{-1}\mathbf{W}=\frac{1}{-i\omega {\tau }_A+1}{W}^A+\frac{1}{-i\omega {\tau }_G+1}{W}^G=W{\tilde{A}}^{-1}$$ (62)

where we have made use of the specific forms of W^A and W^G (namely, that they have zero columns for inhibitory and excitatory neurons, respectively) from Eq. (17), and where we have defined

$$\begin{array}{rr}& W=\sum _{\alpha }{W}^{\alpha }\hspace{1em}\in {\mathbb{R}}^{N\times N}\\ & \end{array}$$ (63)

$$\tilde{A}:=z\tilde{T}+{\mathbb{I}}_{N\times N}$$ (64)

with $\tilde{T}=\text{diag}\left(\tilde{\tau }\right)\in {\mathbb{R}}^{N\times N}$ where $\tilde{\tau }=\left({\tau }_A,\dots ,{\tau }_G,\dots \right)\in {\mathbb{R}}^N$ is the N-dimensional vector with first N_E components equal to τ_A and the last N_l components equal to τ_G. After identifying τ_A/G with τ_E/I, we thus see that $\tilde{T}$ is the same as the T matrix of the r-model (which is N dimensional), as is W its connectivity matrix. Also noting that ${\left(z\tilde{T}+{\mathbb{I}}_{N\times N}\right)}^{-1}$ and Φ are both diagonal, we can commute them in Eq. (62) to obtain:

$$\text{det}\left(G^{-1}\left(\omega \right)\right)=\text{det}\left(A\right)\text{det}\left({\mathbb{I}}_{N\times N}-W\Phi {\tilde{A}}^{-1}\right)$$ (65)

$$=\frac{\text{det}\left(A\right)}{\text{det}\left(\tilde{A}\right)}\text{det}\left(z\tilde{T}+{\mathbb{I}}_{N\times N}-W\Phi \right)$$ (66)

$$=\frac{\text{det}\left(A\right)}{\text{det}\left(\tilde{A}\right)}\text{det}\left(z\tilde{T}+{\mathbb{I}}_{N\times N}-\Phi W\right)$$ (67)

(to get the last line, do a similarity transform with Φ, of the matrix in the last determinant).

Now it is explicit that the zeros of the last determinant factor are the eigenvalues of the N-dimensional r-system (after τ_A/G ↔ τ_E/I identification).

The first factor, on the other hand, can be written as:

$$\frac{\text{det}\left(A\right)}{\text{det}\left(\tilde{A}\right)}=\frac{{\left(z{\tau }_A+1\right)}^N{\left(z{\tau }_G+1\right)}^N}{{\left(z{\tau }_A+1\right)}^{N_E}{\left(z{\tau }_G+1\right)}^{N_I}}={\left(z{\tau }_A+1\right)}^{N_I}{\left(z{\tau }_G+1\right)}^{N_E}$$ (68)

So the spectrum also has N additional real eigenvalues (in addition to those of the r-model) with values $-{\tau }_A^{-1}$ and $-{\tau }_G^{-1}$, and multiplicities, N_I and N_E, respectively. (Thus in total we have 2N eigenvalues as we should.)

In particular, all oscillatory/complex eigenvalues are exactly those of the r-model in the no-NMDA case, which in the 2-neuron case are given by the formulae in Tsodyks et al. 1997.

Approximate statement about role of NMDA:

We consider two regimes for the effect of NMDA:

1. when |z| or ω are very small compared to the NMDA time-constant: $\omega \ll {\tau }_N^{-1}$.

2. when |z| or ω are very large compared to the NMDA time-constant: $\omega \gg {\tau }_N^{-1}$.

The first regime is relevant for DC response and DC properties (such as surround suppression of steady-state rates). The second regime is approximately valid for gamma oscillations, thanks to the relatively high frequency of those.

In regime 1, it is obvious that the breakdown of E weights into the two types doesn’t have any effects, simply because (setting ω to 0) time-scales don’t play any role here. So the parameter ρ_N makes no difference to fixed point response properties.

In regime 2, looking at Eq. (61), we note that the prefactor $\frac{1}{-i\omega {\tau }_{\alpha }+1}$ for NMDA is very small and can be ignored. This means that for high frequencies (e.g., approximately frequencies around gamma) we can simply kill all NMDA weights, and only consider the AMPA weight matrix, W^A. In particular, the model where W^A ∝ W^N, then the effect of NMDA on the gamma peak is approximately equivalent to reducing total excitatory weights (which all affect DC properties) by a scalar factor (which in our formalism is ├ 1-ρ_N) when it comes to gamma properties.

Appendix 2

Theorems for the two-population model

We consider now the case of a two-neuron model with one excitatory and one inhibitory neuron (the neurons in all models of this article should be thought of as “mean field” neurons, each representing a statistically homogenous population of neurons of excitatory or inhibitory type), what we have here called the two-population model. We will also assume no NMDA contribution (or equivalently work in the very slow NMDA regime, and replace all excitatory weights with their AMPA part, as explained at the end of Appendix 1).

In this case the gamma peak frequency is closely approximated by the imaginary part of the eigenvalues of the Jacobian matrix:

$$J=-{\mathbf{T}}^{-1}+{\mathbf{T}}^{-1}\mathbf{W}\Phi$$ (69)

$$=\left(\begin{array}{cc}{\gamma }_E\left(-1+W_{EE}{\Phi }_E\right)& -{\gamma }_E{W}_{EI}{\Phi }_I\\ {\gamma }_I{W}_{IE}{\Phi }_E& {\gamma }_I\left(-1-W_{II}{\Phi }_I\right)\end{array}\right)$$ (70)

where we defined ${\gamma }_E\equiv {\tau }_{\mathit{\text{AMPA}}}^{-1}$ and ${\gamma }_I\equiv {\tau }_{\mathit{\text{GABA}}}^{-1}$. Noting that the trace and determinant of J yield the sum and product of the eigenvalues, respectively, we obtain the expression (see (Tsodyks et al., 1997))

$$2{\lambda }_{\mathrm{1,2}}={\gamma }_E\left(W_{EE}{\Phi }_E-1\right)-{\gamma }_I\left(W_{II}{\Phi }_I+1\right)\pm \sqrt{{\left[{\gamma }_E\left(W_{EE}{\Phi }_E-1\right)+{\gamma }_I\left(W_{II}{\Phi }_I+1\right)\right]}^2-4{\gamma }_E{\gamma }_I{W}_{EI}{W}_{IE}{\Phi }_E{\Phi }_l}$$ (71)

A gamma peak exists only if the expression under the square root is negative, i.e.

$$4{\gamma }_E{\gamma }_I{W}_{EI}{W}_{IE}{\Phi }_E{\Phi }_I>{\left[{\gamma }_E\left(W_{EE}{\Phi }_E-1\right)+{\gamma }_I\left(W_{II}{\Phi }_I+1\right)\right]}^2,$$ (72)

in which case, for the gamma peak angular frequency ω₀, we (approximately) have

$$4{\omega }_0^2=4{\beta }_E{\beta }_I{W}_{EI}{W}_{IE}-{\left({\beta }_E{W}_{EE}+{\beta }_I{W}_{II}+{\gamma }_I-{\gamma }_E\right)}^2$$ (73)

where we defined β_X:=γ_XΦ_X for X ∈ {E, I}.

We will now obtain a simplified expression for the derivative of ${\omega }_0^2$ with respect to the contrast c, using the rectified supralinear nonlinearity of the SSN. Using ${\Phi }_{*}=n{k}^{\frac{1}{n}}{r}_{*}^{1-\frac{1}{n}}$ (where r_* is the firing rate at fixed point) we obtain

$$\frac{d{\beta }_{*}}{dc}=\frac{n-1}{n}{\beta }_{*}\frac{d\text{ln}{r}_{*}}{dc}$$ (74)

Then using Eq. (73), and defining

$$A:=\left({\beta }_E{W}_{EE}+{\beta }_I{W}_{II}+{\gamma }_I-{\gamma }_E\right)$$ (75)

and $(\cdots {)}^{\text{'}}:=\frac{d(\cdots )}{dc}$, we find:

$$\frac{n}{n-1}\frac{d{\omega }_0^2}{dc}={\beta }_E{\beta }_I{W}_{EI}{W}_{IE}{\left(\text{ln}{r}_E+\text{ln}{r}_l\right)}^{\text{'}}-\frac{2A}{4}\left({\beta }_E{W}_{EE}{\left(\text{ln}{r}_E\right)}^{\text{'}}+{\beta }_I{W}_{II}{\left(\text{ln}{r}_l\right)}^{\text{'}}\right)$$ (76)

$${=\omega }_0^2{\left(\text{ln}{r}_E+\text{ln}{r}_l\right)}^{\text{'}}+\frac{1}{2}{A}^2\left[\frac{{\left(\text{ln}{r}_E\right)}^{\text{'}}+{\left(\text{ln}{r}_l\right)}^{\text{'}}}{2}-\frac{\sum _a{w}_a{\left(\text{ln}{r}_a\right)}^{\text{'}}}{{\gamma }_I-{\gamma }_E+\sum _a{w}_a}\right]$$ (77)

where the sums are over a ∈ {E, I} and we defined

$$w_a:={\beta }_a{W}_{aa}\hspace{1em}a\in \{E,I\}$$ (78)

Analysis of the sign of $\frac{d{\omega }_0^2}{dc}$:

Assuming that we are in the gamma oscillatory regime (i.e., ω₀ is real) and that the fixed point rates increase with contrast, then from Eq. (77) we find that sufficient conditoin for $\frac{d{\omega }_0^2}{dc}>0$ is that the factor in the square brackets in Eq. (77) is positive. In the solutions of SSN most relevant to cortical biology, (ln r_l)′ tends to be larger than (ln r_E)′ (because excitatory rates tend to saturate or supersaturate earlier). We thus consider two extreme cases: (ln r_E)′ = (ln r_l)′ and (ln r_E)′ = 0.

In the first case, the bracket becomes ${\left(\text{ln}{r}_l\right)}^{\text{'}}\left[1-\frac{{\sum }_a{w}_a}{{\gamma }_I-{\gamma }_E+{\sum }_a{w}_a}\right]=$ ${\left(\text{ln}{r}_l\right)}^{\text{'}}\frac{{\gamma }_l-{\gamma }_E}{{\gamma }_I-{\gamma }_E+{\sum }_a{w}_a}$, which is positive as long as γ_i > γ_E (which is unfortunately not the case for GABA and AMPA).

In the second case, the bracket factor becomes ${\left(\text{ln}{r}_l\right)}^{\text{'}}\frac{{\gamma }_l-{\gamma }_E+{\sum }_a{w}_a-2w_l}{2\left({\gamma }_l-{\gamma }_E+{\sum }_a{w}_a\right)}={\left(\text{ln}{r}_l\right)}^{\text{'}}\frac{{\gamma }_l-{\gamma }_E+w_E-w_l}{2\left({\gamma }_l-{\gamma }_E+{\sum }_a{w}_a\right)}$. This is positive (as long as the denominator is positive, which is true as long as γ_I > γ_E) if

$$w_E-{\gamma }_E+{\gamma }_I>w_I$$ (79)

But the stability of the fixed point dictates that the expression on first line of Eq. (71) (the real part of the eigenvalues) has to be negative and thus

$$w_I>w_E-{\gamma }_E-{\gamma }_I.$$ (80)