Primate neocortex performs balanced sensory amplification

Summary

Sensory cortex amplifies relevant features of external stimuli. This sensitivity and selectivity arise through the transformation of inputs by cortical circuitry. We characterize the circuit mechanisms and dynamics of cortical amplification by making large-scale simultaneous measurements of single cells in awake primates and by testing computational models. By comparing network activity in both driven and spontaneous states with models, we identify the circuit as operating in a regime of non-normal balanced amplification. Incoming inputs are strongly but transiently amplified by strong recurrent feedback from disruption of excitatory-inhibitory balance in the network. Strong inhibition rapidly quenches responses, thereby permitting tracking of time-varying stimuli.

One-Sentence Summary:

Sensory cortex uses balanced excitatory and inhibitory circuitry to boost weak signals while maintaining fast sensory dynamics in a changing environment.

ETOC blurb:

Sensory circuits show high sensitivity to signals of interest even in noisy changing environments. Pattadkal et al place constraints on the circuitry for amplification using calcium imaging and electrophysiology in marmoset area MT. Different circuits for amplification are characterized by distinct dynamics. Area MT dynamics match a balanced amplification circuit.

Introduction:

Sensory cortical circuits are remarkable for their high sensitivity and precise selectivity for specific features in their inputs, even when these inputs are weak, noisy, and changing^1–5. These properties are proposed to emerge from neuronal connectivity patterns within the cortex that act to selectively amplify and sculpt their inputs^6–15. Several candidate circuit architectures have been proposed to model selective amplification, though each of these candidates is characterized by distinct dynamics in both evoked and spontaneous states^14,16,17. Here we combine a computational approach with empirical single cell activity measurements from large populations of neurons in awake primate area MT, a visual area where neurons are sensitive to visual motion ^18–21, to examine cortical dynamics in order to identify the operating regime of sensory cortex and supply fundamental constraints on its underlying circuitry.

Anatomical measurements of intracortical circuitry have revealed that thalamic synapses are sparse relative to intracortical synapses, supporting the idea that neocortical circuitry acts to amplify feedforward signals⁸. In concert with these anatomical observations, cortical neuron selectivity is maintained even as stimulus strength decreases^1,22,23. This selectivity even emerges in the absence of sensory stimulation, as patterns of spontaneous activity in visual cortex match those observed during visual drive²⁴. These observations motivated the development of diverse amplification models. The precise interplay between cortical excitation and inhibition in these models makes distinct predictions for network dynamics. One proposal involves strong recurrent excitation among neurons with similar selectivity, accompanied by inhibition between neurons with disparate selectivity. These models exhibit self-sustaining and amplifying states on a ring attractor when the recurrent weights are large (“continuous attractor [CA]”), and amplify even when the weights are weaker than needed for self-sustaining activity (“normal amplification [NA]”)¹⁶. Though excitatory connections are more numerous, measurements of inhibition indicate that they may be as strong and selective as excitation^25–31, leading to an alternative cortical model involving a balanced interplay between excitation and inhibition. In these models, excitation and inhibition share similar connectivity patterns, with inhibition cancelling recurrent excitation (“balanced amplification [BA]”) in steady-state ¹⁷; inputs disrupt the balanced state, resulting in amplification. Each of these three circuit models exhibits distinct network dynamics that can be distinguished in spontaneous and evoked states.

To examine how amplification emerges in the neocortex we studied area MT, a critical node on the dorsal visual pathway in primates where motion information is represented. The neuronal activity in area MT has been linked to both our perception and action in response to visual motion. Distinguishing circuit motifs requires simultaneous access to large populations of neurons, which two-photon calcium imaging affords in awake behaving marmosets. We first describe the functional organization of area MT, and demonstrate that weak sensory signals are amplified in area MT. We compare the dynamics of the MT population to the dynamics predicted by each of the proposed cortical amplification models. These experiments reveal that neurons rapidly respond to stimuli, populations do not exhibit states and shifts predicted by CA or NA models, display brief epochs of tuned response during spontaneous activity, and quickly change states when tuned input stimuli are varied, consistent with the proposal of BA.

Results:

To reveal the circuit origins of cortical amplification, we examined the dynamics of sensory neurons in visual area MT of awake primates. We recorded both spontaneous and evoked large-scale population activity to time-varying inputs in inhibitory interneurons using two-photon imaging of calcium signals³² (Fig. 1A; 69 imaging sessions, 5046 neurons). We examined inhibitory neuron activity because we have access to a strong, stable promoter and the cortical models we test do not exhibit distinct excitatory and inhibitory behavior. Neurons in area MT were selective for motion direction^6,33,34 and exhibited a functional map-like organization similar to primary visual cortex³⁵ (Fig 1B,C). Direction tuning shifted systematically across the surface of area MT ³⁴, such that nearby neurons had similar direction preferences (Fig. 1D). We characterized this organization by quantifying how the preferred directions of individual cells varied as a function of cell separation in cortical space (Fig 1E, length constant = 244 microns). Estimates of hypercolumn width (spatial period of preferred direction maps) across animals and chambers were similar (mean = 353 microns ± 129 s.d., Sup. Fig. 1A,B). This functional organization was stable when imaged over days or months (Sup. Fig 2).

Fig. 1. Population response characteristics in area MT.

(A) Schematic of the experimental setup. The marmoset is head-fixed and sits in a chair in front of the display monitor, under the tilted objective of a two-photon microscope above area MT. An infrared camera allows eye tracking. (B) An example imaging plane in area MT. The locations of 5 example cells are highlighted. (C) Responses of the 5 example cells to fullfield motion in different directions. Gray lines show individual trial responses and colored lines are average response. Scale bars indicate 0.5 ΔF/F. Plots to the right are direction tuning curves. The plotted points are mean response per direction and error bars show the standard error. The responses were fitted with a Von Mises direction tuning curve. Goodness of fit R² values for cell 1 = 0.7, cell 2 = 0.97, cell 3 = 0.97, cell 4 = 0.94, cell 5 = 0.82. Each cell’s direction selectivity index (DSI) is indicated on top of the tuning curve. (D) Organization of preferred direction within plane. Outlines of cells with a DSI ≥ 0.15 from the same imaging field as in B are colored with their computed preferred direction. (E) Nearby cells share direction preferences. Difference in preferred direction between cells is plotted as a function of the physical distance between cells. Each dot is a cell pair. All cell pairs with DSI ≥ 0.15 are considered. The distances are then binned in 25 μ and the black circles represent the median direction difference for each bin. Error bars represent the angular standard deviation. Blue line is an exponential fit to the data and red line is an exponential fit to the shuffled data. (F) Amplification of direction signals. Responses of an MT population to high (100%) and low (2.5%) coherence motion, in which neurons are binned by direction preference (left top panel) and fit by Von Mises direction tuning curves. Response of population of direction selective V1 neurons (DSI > 0.25, n = 21), to stimuli of 100%, 25% and 8% coherence shown in bottom left panel. The normalized selectivity of the population output, measured by the population DSI, is compared to the stimulus coherence (right panel). Lines connect responses from the same populations, different lines correspond to different populations. The green dot and black square represent the example population shown in left panel. Blue squares represent selectivity of V1 neurons measured using electrophysiology. Normalized selectivity is computed by dividing population DSI by the population DSI at 100% coherence. Even at low coherence, the MT population output is selective, and the value lies above the diagonal, indicating amplification of input (shaded region).

Because our goal is to understand signal amplification, we first characterized how the selectivity of area MT neuron population response is related to the signal strength. It had previously been shown that individual area MT neurons could respond selectivity to motion direction even when the coherence of dot motion is lowered⁴. Dot coherence characterizes the percentage of moving dots that carry a consistent motion signal. We examined the selectivity of MT populations by varying dot coherence from 100% (all dots move in the same direction) to much lower values. We find that MT neuronal populations remain selective even when motion signals are degraded by lowering coherence to 2.5% (Fig. 1F). To determine whether this amplification reflects a process earlier in the sensory pathway we next assayed the motion signals from primary visual cortex (V1) using extracellular electrophysiology with neuropixels. V1 neurons provide direct input to area MT, though area MT also receives input from other visual areas ³⁶. V1 neurons exhibit a broad diversity of direction selectivity and direction selective cells from layer 4B are known to provide direct input to area MT³⁷, so we restricted our analysis to direction selective V1 neurons. Using the same coherences we find that selectivity of populations of direction selective V1 neurons³⁷ declines dramatically with signal strength (Fig. 1F, Sup. Fig. 3).

To reveal the circuit basis of this feature-specific amplification we integrated three prominent recurrent amplification models^14,16,17 into a unified computational framework (Fig. 2; analytical derivations in Supplementary Information). Selective amplification in recurrent circuit models can occur through at least three distinct mechanisms: one in which amplification emerges from imbalances in excitation and inhibition (“balanced amplification”), one in which excitation is dominant and contributes enough positive feedback to stabilize non-zero activity in the absence of input (“continuous attractor”), and one in which the excitation is strong enough to amplify inputs but not enough to maintain its state when the inputs are withdrawn (“normal amplification”). We analytically and computationally implemented a network framework with segregated excitatory and inhibitory cell populations, in which two key model parameters alter network behavior: the strength of tuned recurrent excitation and the ratio of tuned recurrent excitation to tuned recurrent inhibition. This model yields selective amplification across large variations in the strengths of excitatory and inhibitory synaptic interactions between neurons, recapitulating the three regimes of selective amplification with distinct dynamical signatures (Fig. 2).

Fig. 2. Circuits for selective amplification.

(A) Top: Cartoon schematic of the cortical networks, including an input and separate excitatory and inhibitory populations. The input at direction θ is transformed by the model network to generate the population output shown at the top. Bottom: Cartoon schematic of connectivity parameters. Weights ${{\text{J}}_{\text{0}}}^{\text{E,I}}$ refer to untuned weights from excitatory or inhibitory cells; weights ${{\text{J}}_{\text{1}}}^{\text{E,I}}$ are the tuned interactions. All cells are connected to each other with untuned weights, and both E and I cells interact with other cells of similar direction preference through the tuned weights, which vary by differences in direction preference (see B) (B) The tuned excitatory (red) and inhibitory (blue) tuned weights as a function of difference between cell preferences for direction. The left panel shows the excitatory and inhibitory tuned synaptic weights for CA (dashed lines) and s-NA (solid lines) networks. The right panel shows tuned synaptic weights in the BA regime, excitatory and inhibitory tuned weights are the same in this regime (C) The two timescales of response dynamics. ${\mathrm{{\rm T}}}_{\text{on-off}}$ represents the time scale for changes off the feature amplification axis, when stimulus is turned on and off. ${\mathrm{{\rm T}}}_{\text{amplif}}$ represents time scale for changes along the amplification axis like when stimulus direction changes. Green dot represents the baseline stable state and green ring represents the ring of stimulus evoked stable states. (D) Dynamics of responses in the model regimes for changes off the feature axis (stimulus on-off) and changes along the feature axis (stimulus direction change) are shown. The top panel shows input amplitude for model cell with direction preference matching stimulus peak direction. Bottom panel shows the on-off timescale using responses to a brief step input. Responses are of model cells with direction preference matching stimulus peak direction. The different model regimes are shown by different lines: f-NA by dashed line, BA by continuous line, s-NA by dash dot line and CA by dotted line. Bottom panel shows amplification axis time scale. The input changes direction at time 0 from 90 degrees to 225 degrees. The corresponding change in location of the activity peak in the model networks is shown by the 4 lines, line types for each model are same as before, f-NA and BA are overlapping. (E-G) Linear model behavior depends critically on the ratio of the tuned excitatory $\left({{\text{J}}_{\text{1}}}^{\text{E}}\right)$ and inhibitory $\left({{\text{J}}_{\text{1}}}^{\text{I}}\right)$ strength and the amplitude of the tuned excitatory component. The degree of amplification of motion inputs (E) ($\left(J_1^E+J_1^I\right)/\left(1-J_1^E+J_1^I\right)$, see analytical derivations) ,the inverse time constant (F) and the ratio of the amplification and time constant (G) are shown across model parameters. Red points indicate specific network parameters that are explored. The dotted region centered around BA indicates the balanced amplification zone. The bigger dotted region to the right of it indicates the normal amplification zone. States beyond the continuous curved line are in the continuous attractor regime. The boundary is defined as ${{\text{J}}_{\text{1}}}^{\text{E}}=\left(1+{{\text{J}}_{\text{1}}}^{\text{I}}\right)$ (see analytical derivations).

When tuned excitatory weights are dominant (Fig. 2B), at lower weight strengths the network’s baseline state is a single low-activity fixed point and its stimulus-driven response is a fixed point given by the input shaped by the selective amplification dynamics of the circuit (normal amplification: NA). Total withdrawal of stimuli causes the NA to decay rapidly to the low-activity stable state¹⁶ (Fig. 2B–F). At higher weight magnitudes (Fig. 2B), the NA network’s tuned responses turn into a ring of fixed points (Fig. 2C), such that the network can maintain activity in one of these states even in the absence of stimulus (continuous attractor: CA). NA and CA amplification models are characterized by two time constants. They both exhibit slow dynamics when moving along the tuned activity states $\left({\mathrm{\tau }}_{\text{amplif}}\right)$ but faster dynamics $\left({\mathrm{\tau }}_{\text{on-off}}\right)$ when switching between the low-activity state and the tuned states (Fig. 2C–F), which in the NA case can be induced by withdrawal of all inputs, and in the CA case can be induced by changing the parameters that stabilize the CA states. A sliver of the NA regime has fast dynamics but weak amplification (the f-NA regime); most of the NA zone exhibits larger amplification and slower dynamics along the amplifying axis (the s-NA regime). The balanced amplification (BA) regime¹⁷ consists of a circuit with strong tuned excitation and inhibition in a nearly matched configuration³⁸ (Fig. 2B). Like the NA regime, it exhibits a single fixed point at baseline activation. But unlike the NA and CA regimes, it generates strong selective amplification through a process of transient dynamics (Fig. 2D), in which small inputs that contribute to an imbalance of excitation and inhibition along a feature dimension are amplified, but then subsequently rapidly quenched¹⁷. The BA regime occurs in networks with segregated excitatory and inhibitory populations (Sup. Fig. 4). Unlike the NA regime and as shown by Murphy & Miller¹⁷, amplification in the BA regime is not accompanied by a slowing of responses along the amplifying direction (Fig. 2D,F). Overall, increases in tuned excitation lead to increases in amplification across all regimes (Fig. 2E), but these regimes exhibit distinct tradeoffs between amplification and response speed (Fig. 2F–G). We next asked whether these models could amplify motion signals in a similar manner as we uncovered in our area MT recordings. Each of these models amplifies the motion signals with respect to the selectivity of their output, though the nonlinear transformation between input selectivity and output selectivity is distinct in each case (Sup Fig. 5). Though these regimes are all selectively amplifying, their differing dynamics allow us to examine empirically the operating regime of cortical circuits.

Given the distinct response dynamics predicted by these models, we first characterized MT dynamics at stimulus offset using a combination of intracellular and extracellular recordings (Fig. 3). Networks in the CA regime should exhibit persistent activity ³⁹ even after stimulus removal, whereas the other models should decay to baseline (Fig. 3A). MT cell activity rapidly declined to baseline after stimulus withdrawal, both at the level of membrane potential (Fig. 3B) and spike rate (Fig. 3C). We fit these offsets in the stimulus response using an exponent. The time constants of both membrane potential and spike rate at stimulus offset are similar, but rapid (Fig. 3D, Vm: mean tau = 34 ms +/− 3.7 s.e.m, Fig. 3E, Spike Rate: mean tau = 46.7 ms +/− 4.3 s.e.m.). These time constants are larger than the cellular time constants, but still rapid.

Fig. 3. On-off dynamics in model and physiology.

(A) Time scale of response of model networks to stimulus offset. Stimulus turns off at time 0. Bottom panel shows the input. Top panel shows the response for CA in blue, s-NA in orange, f-NA in green and BA in purple. F-NA and BA are overlapping. Responses and input are of and to model cell with direction preference matching stimulus peak direction. (B) In vivo whole cell measurements of stimulus decay times in marmoset MT. Top panel is an example cell membrane potential trace in response to stimulus motion recorded intracellularly from marmoset MT. Gray lines represent individual trials. Thick black line is the median filtered mean response. Box at the end of the trace shows the interval displayed in B. Next three panels show Whole cell records at the termination of preferred direction motion in marmoset area MT for 3 example cells (cell 1 same as top). Lines indicate an exponential fit of the Vm decline to baseline activation. Responses are normalized by the mean depolarization induced by the visual stimulus. Bottom panel shows average response across neurons (red) and the individual neurons in the sample population (gray lines). (C) In vivo extracellular measurements of stimulus decay times in macaque MT. Panel arrangement is same as in C. (D) The distribution of tau fit from the exponential decay to baseline for marmoset MT using whole-cell recordings. (E) The distribution of tau fit from the exponential decay to baseline for macaque MT using extracellular single unit recordings.

These rapid offset dynamics are inconsistent with dynamics purely in the CA regime but are feasible in the BA or f-NA regime. They are also consistent with the slow amplifying s-NA regime: near the NA-CA boundary, increases or decreases in overall excitatory drive (as likely happens when the stimulus is withdrawn) can shift the network between NA and CA, and the resulting response involves the relatively fast on-off time-constant ${\mathrm{\tau }}_{\text{on-off}}$ of the NA and CA regimes⁴⁰ (Fig. 2D). Therefore, the s-NA model, in addition to the BA and f-NA regimes, is consistent with the rapid stimulus offset responses observed in our recordings. A key prediction, however, is that the dynamics of s-NA models will be distinct for changes along the amplifying states, such as changes in stimulus direction (Fig. 2 D,E).

To distinguish between BA, f-NA and s-NA alternatives, we examined network dynamics when the stimulus is continually present, but changes midway. We first compared model and area MT network responses while a stimulus is present, but the stimulus direction is switched (Fig. 4A) ^41,42. In the BA and f-NA regimes, network activity is predicted to shift abruptly with the stimulus direction change: the population activity bump induced by the initial motion direction decays rapidly in place while a new bump emerges to represent the second direction (Fig. 4B). In contrast, s-NA network activity bumps are predicted to move smoothly along the ring of directionally tuned responses, passing through and activating neurons selective for intermediate directions along the way (Fig. 4B, green traces), if the second direction is not too far from the first. This smooth movement happens because tuned excitation slows the decay of the first bump, while the input at the second location amplifies the side of the first bump in the direction of the second. This results in an effective shift of the bump in the direction of the second input; the process continues until the bump has moved to the location of the second bump. The behavior of the different regimes is robust when the ratio of number of E cells to I cells is changed to 4:1, as observed in the neocortex (Sup. Fig. 6). Reducing the feedforward drive to inhibitory neurons relative to excitatory neurons can alter network behavior, causing all of these models to become excitation-dominant and exhibit movements along the ring of directionally tuned responses (Sup. Fig. 6).

Fig. 4. Shifts in motion direction elicit rapid changes in MT population responses.

(A) A cartoon of the stimulus in which stimulus direction is rapidly changed from 90 degrees to 225 degrees. Blue and orange dashed lines indicate first and second stimulus intervals respectively. (B) Response of model networks to a sudden 90–225 direction change. Left panels show spread of activity across cell groups of different preferred directions (heatmaps) and right panels show time courses for 3 relevant cell groups. Average response of a population of cells in a s-NA network elicits a smooth rotation in the bump of activity from those cells preferring the first direction to the second direction (left). The first stimulus starts at the left edge of all panels. Next stimulus starts at the first gray vertical line. The end of the second stimulus is at the right end of panels. The end of the second stimulus for calcium imaging heatmap is shown by the second vertical gray line. The response time courses of cells preferring the first direction (blue, cells with direction preference 90 ± 30), the second direction (orange, cells with direction preference 225 ± 30), or intermediate directions (green, cells with direction preference 127.5 ± 30) are shown in the right panels. Both the f-NA and BA network populations exhibit rapid shifts (left panels), and neurons tuned to intermediate directions do not respond. Calcium recordings from area MT neurons also make rapid shifts in population activity, but there is a lag in the calcium imaged response relative to stimulus onset and offset due to the decay time of the calcium fluorescence signal. The intermediate cells do respond, but no more than expected from the response to the single stimulus conditions. Area MT scale bar is in Z score. Asterisk shows where the example cell shown in D would lie in this plot. (C) Population tuning curves at different epochs do not exhibit a smooth shift in preferred direction, but instead are well fit by a tuning curve with two modes in the first and second directions. (D) The response of a single cell selective for 135 degrees (top). This cell does not respond to a motion shift between 90 degrees and 225 degrees but does to a direction change between 270 and 135 degrees.

When we shift the stimulus direction from 90 to 225 degrees, the MT population response average shows two distinct activity bumps at the two presented directions, but no smooth transition between the bumps in the form of elevated activation of cells preferring intermediate directions (Fig. 4B, bottom). Splitting the population responses into three groups based on direction preference (those selective for the first, second or intermediate directions) reveals that the cells responded to the first and second stimuli with distinct latencies (Fig 4B, bottom right). Population tuning curves based on neuronal preference reveal that following the change in stimulus direction, a new bump appeared at the second direction while the activity at the first direction decayed (Fig 4C). Individual MT neurons tuned to intermediate directions responded weakly, not exceeding the amount expected based on their direct evoked response to the two motion directions, inconsistent with a bump of activation that passes through the intermediate directions (Fig. 4D). The slow calcium signal may not fully register a rapidly moving bump that passes through the intermediate directions, but extracellular single cell electrophysiology measurements in MT revealed a similar absence of responses in neurons tuned to intermediate directions (Fig. 5). As predicted by our model, excitatory cells (from electrophysiology) and inhibitory cells (from imaging) exhibit similar population behavior.

Fig. 5. Changes in MT population responses, using electrophysiology, for rapid shifts in motion direction.

(A) Responses of an example cell for different motion directions. Its direction tuning curve, using Von-Mises direction tuning curve, is shown to the right (Goodness of fit; R² = 0.99). Cell prefers 135 degrees. (B) Response of the example cell for 90–225 stimulus change. Blue and orange indicate the first and second stimulus interval respectively (left). Response of the same cell for 270–135 stimulus change (right). (C) Comparison of observed responses of all cells to 90–225 stimulus change to predicted responses based on tuning properties. See methods for predicted response estimation. Observed responses are significantly lower than predicted (t-test p-value: 5.5x10⁻¹⁵). (D) Population trajectory in TDR space for 90–225 stimulus change. Thick lines indicate response directions for stimulus motion along color coded directions. Contours connect the same response levels across directions. Population trajectory does not move along the ring, but directly changes from 90 (purple) to 225 (yellow). (E) Average responses across conditions. Two discrete activity bumps are seen. (F) Time course of three cell classes for 135 degrees stimulus change. Blue is the average of cells preferring first presented direction, orange is average of cells preferring second motion direction, green is average of cells preferring intermediate direction. Dashed line is the response of the green cell class to stimulus 1 presentation alone, dotted line is response of the green cell class to stimulus 2 presentation alone. (G) Population tuning curves at different time points. Slices of population output are shown at 8 ms intervals from start of stimulus 1. Activity bump transitions discretely from stimulus 1 to 2.

To determine quantitatively whether MT population responses discretely switch their signals for direction (as in the f-NA/BA networks) or sweep smoothly through intervening directions (s-NA network), we compared the electrophysiological and calcium network responses to switch and sweep predictions based on the summed responses to single directions (Sup. Fig. 7). Both our calcium and electrophysiology records are highly correlated with a switch prediction and not the sweep prediction (Ca: R_switch mean = 0.52 +/− 0.2, R_sweep mean =0.13 +/− 0.09. Ephys: R_switch = 0.91, R_sweep = 0.13, Sup. Fig. 7B). One caveat to our interpretation is that the stimulus might be so strong that it overrides the slow internal dynamics of the s-NA response. We therefore repeated the experiment at the lower contrast of 8% (Sup. Fig. 8). Even at this contrast, we observed no evidence of a bump of activation moving through intermediate directions, indicating that area MT is not operating in the s-NA amplification regime.

Another distinguishing feature of amplifying models is how they respond to smooth decrements in signal strength (Sup. Fig. 9), while the stimulus remains present. In the s-NA network, amplification nearly saturates the network response and reducing the input signal strength smoothly has at most a weak and slow effect on network response (Sup. Fig. 9C, full width at half max = 23.9 degrees at all coherence levels). In contrast, the f-NA and BA networks are responsive to decreases in the input strength, showing rapid reductions in tuned output amplitude (Sup. Fig. 9C, bottom panels). To assay how decrements in signal strength alter MT response we presented dot motion stimuli that began at a high coherence but the coherence declined in 3 steps of 800 ms period. Like the f-NA and BA networks, the tuned responses of area MT populations declined as coherence decreased (Sup. Fig. 9D–G), despite an overall maintenance in the MT population activity. MT population responses to changes in both direction and coherence indicate that the network is highly sensitive to the amplitude and direction of the stimulus, inconsistent with operation in the s-NA regime but consistent with the BA and the f-NA networks.

These candidate amplification regimes also exhibit distinct spontaneous activity signatures¹⁶ which, we hypothesized, would help to distinguish between the BA and f-NA network regimes. Though spontaneous activity in MT is weaker on average than visually-evoked responses, it is characterized by sparse punctate bursts, with individual MT neuron amplitudes approaching those of evoked responses (Fig. 6AB). The sparse spontaneous activity events generated skewed activity distributions for both individual cell and network activity (Fig. 6C–F, skewness: 1.4, median skew =1.2 ± 0.5 s.d., across all sessions), a common phenomenon in neural circuits^43,44.Cells of similar direction preference were co-activated during these spontaneous bouts of activity (Fig. 6G), demonstrating that internally-generated feature-specific amplification occurs even in the absence of a motion stimulus^24,45–49.

Fig. 6. Area MT spontaneous and evoked activity comparison.

(A) Example of simultaneous spontaneous activity traces across area MT population. The population mean of z-scored responses across all cells $\left(\text{n}=260\right)$ is shown at the top. Vertical scale bars indicate a response of 1 $\mathrm{\Delta }\text{F/F}$. The yellow and blue regions mark two active epochs. The preferred direction of each cell is indicated by ${\mathrm{\theta }}_{\text{pref}}$ (B) Example of simultaneous stimulus-evoked activity traces across area MT neurons (same cells as in A). The population mean Z scored cell activity (top row) is displayed above a subset of the neuronal population (bottom rows). The vertical lines indicate start and stop of the stimulus, motion duration was 700 ms. Vertical scale bars to the right indicate a 1 $\mathrm{\Delta }\text{F/F}$. (C) Spontaneous activity distribution of the mean population activity shown in A. (D) Stimulus-evoked activity distribution of the mean population activity during peak interval per trial for the same population. (E) Spontaneous activity distribution of an example cell in the same population. The distribution has a skewness of 1.1. Inset shows the distribution of skews for spontaneous activity for all cells in the population. (F) Distribution of spontaneous skews for mean population activity across different spontaneous sessions across the 2 animals. (G) Example patterns of activity observed in the absence of visual stimulation. The patterns occurred at 6 different time points. Intensity indicates the degree of activation; color indicates direction preference which is same as the map shown in figure 1D. (H) Stimulus-evoked population response trajectories in principal components space. Mean population response trajectory for each of the 8 directions presented are projected along the principal component dimensions. The color coding indicates the stimulus direction. The black region in each trajectory shows the peak response interval. The individual neuron contributions to PC2 and PC3 were dependent on motion selectivity with cells having a higher degree of motion selectivity contributing more to these PCs (linear correlation coefficient between DSI and absolute PC2 weights is 0.56, with PC3 weights was 0.25, Sup. Fig. 10B, C, top). The PC2 and PC3 weights also had a sinusoidal relation with the preferred directions of cells (Sup. Fig. 10B, C, bottom). (I) PC 1 is related to mean population response. The individual neuron contribution to PC1 did not depend on its degree of motion selectivity or direction preference (linear correlation coefficient between PC1 weights and DSI was 0.17 and between PC1 weights and preferred direction was −0.05, Sup. Fig. 10A). (J) Population response projections in TDR space. Top panel shows projection of stimulus-evoked mean responses in TDR space, they show better separation along different directions than H. Bottom panel shows population trajectory during the two spontaneous epochs. Activity during the two epochs marked in blue and yellow is projected in TDR space shown in top panel. Saturation corresponds to the activity, more saturation indicating higher activity.

To compare quantitatively the patterns of spontaneous activity in the MT population with model predictions, we projected the high-dimensional spontaneous population activity into a low-dimensional direction space. We defined this direction space by the evoked responses to motion stimuli (Fig. 6B). Motion stimuli evoked large increases in the overall population response, but the motion direction modulated which population subsets responded (Fig 6B). To visualize the tuning of the population response, we first performed unsupervised dimensionality reduction using principal components analysis (PCA) to generate a low-dimensional projection of the network activity. PCA provided a compact representation of the 260 recorded neurons described in Figure 6 as merely 5 dimensions accounted for over 90% of the variance in the average stimulus-evoked population activity. The dimensions that emerged from PCA were easily interpretable: the first dimension (PC1) corresponded to mean population activity (PC1, mean population activity regression R²= 0.98), the next two dimensions (PC2, PC3) encoded the motion direction, with adjacent motion directions encoded by nearby polar angles in the PC2, PC3 plane (Fig. 6H,I Sup. Fig. 10). We consistently observed similar evoked response trajectories across multiple sessions in different animals (Sup. Fig. 11). PCA can provide a quantitatively distorted view of the population code for direction if the recorded sample contains unequal numbers of cells encoding different motion directions (Fig 6H, right panel). To correct for these distortions we applied targeted dimensionality reduction (TDR ⁵⁰), which explicitly identifies the dimensions that capture most of the direction-related variance in the population. TDR provided a similar representation to the PCA but separated the directional responses better (Fig. 6J, top panel). Projecting spontaneous MT population activity into the evoked PC direction space revealed that the large spontaneous events in MT neurons often resemble directionally tuned evoked responses (Fig. 6J, bottom panel for the two events shown in Fig. 6A), consistent with the concerted activation of neurons with common motion preferences^24,45–49.

Projecting spontaneous activity into the low dimensional space defined by motion might be problematic if the space of spontaneous fluctuations differs from that evoked by our simple motion stimuli. The dimensionality of the spontaneous population response in our data is higher than evoked, as 142 dimensions are required to explain 90% of the variance of the spontaneous activity. The first three PCs or the 1^st PC and the 2 TDR dimensions (Fig. 6H–J), however, accounted for a substantial fraction of spontaneous activity variance (23% and 22% respectively), similar to the fraction of the trial-by-trial evoked response variance explained by these components (31% for both). Therefore, while the dimensionality of neural response during spontaneous activity is higher than to motion stimuli, the motion selective patterns explain similar amounts of variance.

MT spontaneous activity projections into the low-dimensional direction space revealed a variable and fluctuating degree of directionality. While most of the population responses map to values near the center, the population makes brief excursions consistent with the coding of motion direction (Fig. 7A). These excursions come at periods of increased population activity (Fig. 6A, J). There was a consistent relationship between MT activity levels and the population directionality such that as activity increases the population exhibits larger direction signals, as reflected in the vector length of the population activity in the low-dimensional TDR space (Fig. 7B). This covariation between the directionality and magnitude of activity was not explained by a global rescaling of low amplitude responses (Fig 7B, open symbols), as shuffling the identity of activated neurons does not recapitulate this relationship. This relationship was observed across sessions in two animals (Sup. Fig 12), for both spontaneous and evoked responses.

Fig. 7. Comparison of Area MT spontaneous activity to predictions of different circuit models.

(A) The projection of MT spontaneous data into the TDR direction space indicates that there are significant deviations of the network response which signal direction spontaneously. Color-coded trajectories are the observed data and black is the shuffled response. Trajectories are max normalized using mean stimulus driven responses. (B) Relationship of spontaneous activity projection to activity magnitude for the example area MT population. Error bars are 95% bootstrapped confidence intervals over the median. The normalized distribution of mean activity across all cells in spontaneous period is shown in the inset. Spontaneous model activity and shuffled model activity are shown in filled and open circles, respectively. (C) Spontaneous activity predictions for the three circuit models. The simulated spontaneous activity is projected in direction space for the s-NA network (left panel), which has responses lying on the attractor ring, the f-NA network, which lacks directional responses (middle panel), and the BA network, which shows varying directionality of the responses (right panel). The colors indicate the output direction of the population response. The size and opacity of the color-coded dots indicate activity amplitude with bigger dots and higher opacity for higher activity. Shown in black is the projection of the same data after shuffling direction preferences of model neurons. Only steady state spontaneous activity is shown in each trial. Not all data points are presented, to avoid overcrowding. (D) Relationship between output directionality and activity in model networks. Filled and open circles are spontaneous model and shuffled model activity as in B. Error bars indicate the standard deviation. The inset histograms are normalized distributions of overall activity within the networks.

The different candidate amplification regimes also exhibited distinct spontaneous activity signatures (Sup. Fig. 13). Spontaneous activity was initially modeled as unstructured input into the network (see Methods). S-NA networks near the CA boundary exhibited saturated tuned responses, as even unstructured input drive pushes these networks to generate strongly tuned outputs (Fig. 7C, left “s-NA”). In contrast the f-NA network remained nearly quiescent with little tuned response. Networks in the BA regime, however, were characterized by large spontaneous activity fluctuations that were tuned, with covarying amplitude and directionality together and returns to low-activity states, much like that observed in the recorded MT populations (Fig. 7D). Therefore, spontaneous fluctuations in area MT match the activity patterns predicted by BA networks.

Spontaneous activity in cortical circuits is usually accompanied by fluctuations in overall thalamic drive ⁵¹. We therefore also modeled spontaneous activity using globally shared fluctuating input (Fig. 8A). As the common input fluctuates, s-NA exhibits near-saturated tuned states when the drive is strong and hops off to the baseline stable state when the drive is weak. S-NA with strong recurrent excitatory connectivity $\left({{\text{J}}_{\text{1}}}^{\text{E}} =1.8\right)$ requires very little input drive to exhibit saturating tuned responses (Fig. 8B,C left). Reducing the excitatory recurrence $\left({{\text{J}}_{\text{1}}}^{\text{E}} =0.5\right)$ but maintaining the ratio of excitatory and inhibitory tuned connectivity increases the hop frequency as the network does not generate a saturating tuned response (Fig. 8B,C middle), and further reducing recurrence into the f-NA regime results in states that remain near the baseline (Fig. 8B,C, right). Thus, global excitation changes mimicking thalamic drive fluctuations combined with the relatively small ${\mathrm{\tau }}_{\text{on-off}}$ (Fig. 2D) appear to rescue the ability of the NA models with intermediate recurrence to resemble MT data (Sup. Fig 14,15). But, even for these NA regimes with intermediate recurrence the network dwells in the tuned states with a higher probability than observed in area MT records (Sup. Fig. 14). Note that when projecting the response onto a single tuned axis, however, these models can appear similar to the BA model (Sup. Fig. 15). While f-NA responses to the change direction experiment matched our experimental measurements, this regime’s spontaneous responses are inconsistent with the data. There are NA models that exhibit fast switching to the change direction experiment and spontaneous fluctuations consistent with our MT records, but the parameters of this NA regime are extremely narrow and depend critically on input statistics (Sup. Fig. 14). In contrast, the BA model behavior is robust: under global excitatory fluctuations it remains similar to both the unstructured input scenario, and the MT measurements (Fig. 8D,E; cf Fig. 7B).

Fig. 8. Model predictions for spontaneous activity when the input fluctuates in time.

(A) Input used to simulate spontaneous activity in models. The shared input received by all model cells is now dynamic. The same input was provided to all model networks. (B) Spontaneous activity predictions for NA circuit models. Left, middle and right are NA models with decreasing ${{\text{J}}_{\text{1}}}^{\text{E}}$ of 1.8 (s-NA shown elsewhere in the paper), 0.5 and 0.15 (f-NA shown elsewhere in the paper). The s-NA model hops on and off the ring as the input fluctuates, model in between with less tuned connectivity also hops on and off but at higher frequency, f-NA is not able to hop on the ring. The colors indicate the output direction of the population response. The size and opacity of the color-coded dots indicate activity amplitude with bigger dots and higher opacity for higher activity. Shown in black is the projection of the same data after shuffling direction preferences of model neurons. (C) Top panels: Relationship between output directionality and activity in the three NA networks. Spontaneous model activity and shuffled model activity are shown in filled and open circles, respectively. Error bars indicate the standard deviation. Bottom panels: Vector length distributions for the three networks. (D) Spontaneous activity predictions for BA model. Only responses to initial 1/10^th of the input is plotted to avoid overcrowding (E) Same as C, for the BA network.

Discussion:

We have used recordings from large scale populations of neurons and computational models to constrain the circuit mechanisms underlying selective amplification in sensory cortex. Our findings demonstrate that evoked and spontaneous cortical dynamics are consistent with balanced amplification while excluding the other potential networks. Cortical dynamics in MT are highly sensitive to time-varying sensory signals, displaying selective amplification and fast responses to changes in the inputs, properties that are well-modeled by a circuit operating in a balanced amplification regime. Cortical dynamics exhibit large tuned spontaneous activity fluctuations that resemble evoked responses. Finally, the degree of selectivity in spontaneous activity grows proportionally with activity. This combination of features is present only in a balanced amplification cortical circuit ^17,52–56. Such a network achieves a balance between rapidly representing feedforward signals while boosting weak signals through recurrent connections (Fig. 2G). Note that our exploration of the cortical operating regime is for the awake primate and this regime may change with differences in behavioral state such as attention and motivation ^57–60. We believe our work is the first to directly arbitrate, through detailed comparison of multiple dynamics predictions with experimental recordings, between the different hypothesized models for how the neocortex performs selective amplification. We have provided multiple lines of specific evidence that the cortex amplifies tuned inputs through strong, tuned, and balanced excitatory and inhibitory feedback ¹⁷, rather than through more-classic amplification mechanisms based on strong recurrent excitation and disinhibition⁴¹.

We used dimensionality reduction based on evoked responses to examine the spontaneous state of network, but we also recognize that the spontaneous state is higher dimensional than is revealed using our motion paradigm. These differences in dimensionality may reflect the restricted dimensionality of the stimulus space. Many stimulus features are encoded across area MT neurons that we did not vary in our stimulus, including spatial frequency ⁶¹, speed ⁶² and disparity ⁶³. Employing a broader stimulus set could reveal a higher dimensional space in which spontaneous and evoked response spaces overlap more. Another aspect contributing to the overlap between the spontaneous and stimulus evoked response spaces may be the functional organization of responses. For example, the relationship between spontaneous and evoked activity is distinct in the rodent neocortex^46,64 which lacks a functional organization.

It is interesting that a directionally tuned balanced amplification cortical circuit may have qualitatively the same architecture as the head direction (HD) ring circuit with continuous attractor dynamics ^65–67, yet operate in a completely different dynamical regime. The differences in dynamical regime are likely rooted in the distinct nature of the computations that must be performed: rapid amplification and tracking of time-varying inputs in sensory cortex versus the storage of a persistent memory of head direction and integration of head velocity signals through continuous attractor dynamics in the HD system ⁶⁸. Though only relatively subtle adjustments in the relative strengths of excitatory and inhibitory connectivity are necessary to shift a network from a balanced amplification regime with a single attractor, as observed in area MT, to a ring attractor (Fig. 2) as observed in the HD circuit, the resulting changes in dynamics and function are large.

The balanced amplification regime only emerges from models in which there are distinct excitatory and inhibitory neurons (Sup. fig 4¹⁷). That sensory cortex operates in the balanced amplification regime provides a potential explanation for the specialization of neurons into distinct excitatory and inhibitory populations: the essential asymmetry that arises from coupling these populations is fundamental to the generation of large but responsive selective amplification of time-varying stimuli.

STAR Methods

RESOURCE AVAILABILITY

Lead contact

Further information and requests for resources should be directed to and will be fulfilled by the lead contact, Nicholas J. Priebe (nico@austin.utexas.edu).

Materials availability

This study did not generate new unique reagents.

Data and code availability

Source code has been deposited at Figshare and is publicly available. DOIs are listed in the Key Resources Table.

Key Resources Table:

REAGENT or RESOURCE	SOURCE	IDENTIFIER
Bacterial and virus strains
rAAV2/1:h56D-GCaMP6f	Mehta et al32	N/A
Deposited data
Figure data	This paper	Figshare: https://doi.org/10.6084/m9.figshare.24416362
Experimental models: Organisms/strains
Marmosets	TxBiomed	N/A
Software and algorithms
Matlab	Mathworks	https://www.mathworks.com/products/matlab.html
LabView	National Instruments	https://www.ni.com/en-us/shop/labview.html
Maestro	Maestro	https://sites.google.com/a/srscicomp.com/maestro/downloads
SpikeGLX	SpikeGLX	https://billkarsh.github.io/SpikeGLX/
Kilosort	Kilosort	https://github.com/MouseLand/Kilosort
Phy	Phy	https://phy.readthedocs.io/en/latest/
Custom source code	This paper	https://doi.org/10.6084/m9.figshare.24416362

Data have been deposited at Figshare and are publicly available. DOIs are listed in the Key Resources Table.

Any additional information required to reanalyze the data reported in this paper is available from the lead contact upon request.

Experimental model and study participant details

One male and one female adult marmoset was used in the current study. One of the animals had MT chambers on both left and right side, while the other had MT chamber only on the left side.

All marmoset experiments were conducted with the approval of The University of Texas at Austin Institutional Animal Care and Use Committee.

Methods Details

Surgical, virus injection and two-photon imaging procedures were similar to previous descriptions ^32,69.

Surgery:

Custom-made headpost ⁷⁰ and chambers were affixed to the skull in a sterile anaesthetized procedure. Throughout the procedure, the body temperature was maintained at 36–37°C and the heart rate, SPO2 and CO2 were monitored. Animals were placed in stereotaxic frames, circular craniotomies were performed on the intended chamber location, over area MT, identified using stereotaxic coordinates, chambers and the headpost were placed and the dura was removed. An implant from dental acrylic was built around the headpost and chambers, covering the remaining exposed skull. The skin around the implant was affixed to the implant using Vetbond. The animals were then returned to the cages after recovery from anaesthesia.

Chamber design:

The chamber consisted of 4 parts. The outermost part of the chamber was a ring of height 1.6 mm and of diameters ranging from 5–7mm. This ring had 1 mm long thin feet that were inserted inside the skull following craniotomy. The second piece was a thin chamber nut (thickness 1.5 mm) that was screwed on the outside of the chamber ring and rested on top of the skull. This assembly was further sealed using Metabond (Parkell, New York). A removable imaging well was screwed on the inside of the chamber ring. The well consisted of a metal insert to which a coverglass was attached at the bottom. A thin cap (1 mm) was screwed on top of the chamber ring to close it.

Virus injections:

Virus injections were performed in a separate anaesthetized procedure. Animals were head-fixed, the head was disinfected, and the procedure was performed under sterile conditions. The imaging well was removed from the chamber ring to physically access the cortex. The virus was injected using Nanoject II (Drummond Scientific) with pulled and beveled glass pipettes of tip diameters of 20–35 μm. rAAV constructs with GCaMP6f under the h56d promoter were used for these measurements ³². The glass pipette was filled with mineral oil and front loaded with the virus. The virus was injected at 23 nl/sec. Injections were made in multiple sites within the chamber at varying heights with each site receiving 500–1000 nl virus mix. The pipette was left in place for 2–5 min before changing position. Injection spread was estimated using trypan blue dye diluted with the virus mix.

Behavioral training and experimental control:

After recovery from surgery, marmosets were habituated to head fixation and trained to fixate visual targets ⁷⁰. Experimental control was provided by the Maestro software suite, which collected eye movement data, controlled visual stimulation, and provided juice reward (https://sites.google.com/a/srscicomp.com/maestro/).

Acute recordings:

The intracellular electrophysiology data presented in figure 3 was collected from recordings made in anaesthetized marmosets. The animals were anaesthetized using intravenous infusion of sufentanil (4–12 ug/kg/hr) and xylazine (0.3 mg/kg/hr) and paralyzed using vecuronium (0.2 mg/kg/hr). They were ventilated artificially. The vitals were continuously monitored, and body temperature regulated at 37 degrees C. Once anaesthetized, area MT was located using stereotactic coordinates and a small craniotomy was performed to expose the brain surface.

Stimulation:

A screen subtending 48 by 38 degrees was placed 45 cm from the animal. Monitor frame rate was 100 Hz. To obtain direction selectivity of cells, fullfield full coherence field of moving dots, speed 25 degrees/s, was presented to the animal at full contrast (dot density = 3.8 dots/deg², dot size 0.4 degrees, black and white dots on gray background). The trial was composed of a blank period following which the moving stimulus was presented to the animal for a duration ranging from 700–1000 ms. The animal was free to move its eyes and received a small marshmallow juice reward at the end of each trial, there was no task requirement, except during some sessions with fixation (see below).

For spontaneous activity, the screen was set to gray background and there was no task and no reward. The spontaneous trials were interleaved between behavioral measurements.

For change direction experiment, a similar dot field was presented at full contrast, unless otherwise specified, and full coherence, moving along one direction for a certain period. The motion direction was suddenly switched to the second direction. Both directions were presented for equal amounts of time, which was either 500 ms or 750 ms each for imaging and 64 ms for extracellular electrophysiology. The direction change was either 90 or 135 degrees, varying for different sessions. Some sessions also presented an initial fixation target (2 deg by 2 deg) at the screen center. The fixation window was 3 degrees by 3 degrees and the duration was 700 ms, with a grace period of 400 ms. Upon successful fixation, moving dot field was presented to the animal, there was no task requirement during the motion on period. Animals were juice rewarded at the end of successful trial (successful fixation followed by completion of motion stimulus). Direction tuning was estimated from trials with only one motion direction presented per trial. Such trials were either interleaved with the change direction stimuli or were run in a separate block.

For acute intracellular recordings, cells were presented with sinusoidal drifting gratings or plaids at their preferred directions and spatial frequencies.

Two-photon imaging: Neuronal activity was measured using a custom-made two-photon microscope equipped with resonant mirrors for video rate sampling (30Hz) ⁶⁹. Fluorescence was detected using standard PMTs (R6357, H7422PA-40 SEL, Hamamatsu, Japan) and amplified with a high-speed current amplifier (Femto DHPCA-100, Germany). Images were acquired using a 16x objective (Nikon N16XLWD-PF, Japan) from fields of view varying from 400 μm by 400 μm to 700 μm by 700 μm. Data were motion corrected using cross correlation ⁷¹. Any remaining frames where motion could not be corrected were manually marked for exclusion from analysis. Animals were imaged beginning 5–6 weeks post virus injection.

Widefield imaging: Widefield maps were obtained by imaging using a 5x or 2.5x objective. For each stimulus trial, a prestimulus response was generated by averaging frames in the pre-stimulus interval and a stimulus response was generated by averaging frames during the peak response interval (10 to 24 frames after stimulus onset). The trial response was then computed by subtracting the prestimulus average ${\text{F}}_0$ from stimulus average F. Trials at each stimulus direction were averaged to generate the mean response per direction. The obtained responses were smoothed using a 2-D Gaussian filter with a standard deviation of 1 to 3 pixels. For each pixel in the frame, the preferred direction was computed as the direction of the vector average of responses across presented directions.

Intracellular recordings: Recordings were made with glass patch electrodes (5–10 MΩ) filled with potassium-gluconate based solution. Cells were recorded in whole-cell configuration. The spikes were removed by median filtering the raw membrane potential traces.

Extracellular recordings: V1 data in figure 1F was collected using the Neuropixels 1.0 probe. We used an IMEC PXIe acquisition module mounted on a National Instruments (NI) PXIe chassis (PXIe-1071) with NI PXIe-8381 and NI PCIe-8381 for remote control. Voltage signals were recorded at 30 kHz from 384 channels using SpikeGLX. Waveforms were first automatically sorted using Kilosort ⁷² and then manually curated using the phy ⁷³ software.

Quantification and statistical analysis

Imaging analysis: Cells were marked using a custom code in MATLAB and fluorescence values were extracted from the regions of interests. Slow drift, if any, was subtracted by computing a moving average over a 10 s period. $\mathrm{\Delta }\text{F/F}$ was computed as follows:

$$\frac{\mathrm{\Delta }F}{F}=\frac{F-F_0}{F_0}$$

Where F is the raw fluorescence at each time point and F₀ is the average baseline fluorescence. Responses were averaged across trials at each direction to give the mean response per direction. For the cell to be included in the analysis, the response at the maximum responsive direction had to be significantly different from baseline as measured using t-test (p: 0.05). In addition, the response at this maximally responsive direction also had to exceed a threshold, which varied between sessions from 0.1 to 0.25 $\mathrm{\Delta }\text{F/F}$. For the session used in figure 1, 6 and 7, 82% cells were included in the analysis. Traces were also median filtered with a 5^th order filter and smoothed with a moving average over 15 frames.

The direction selectivity index (DSI) was computed as follows:

$$DSI=\frac{\sqrt{{\left({\sum }_{\theta }{R}_{\theta }\sin \theta \right)}^2+{\left({\sum }_{\theta }{R}_{\theta }\cos \theta \right)}^2}}{{\sum }_{\theta }{R}_{\theta }}$$

Where $R\left(\theta \right)$ is the response at direction $\mathrm{\theta }$.

Standard error over the mean response was computed as follows:

$$S.E=\frac{\sigma }{\sqrt{n}}$$

Where $\mathrm{\sigma }$ is the standard deviation of responses, and n is the number of trials.

The responses per trial were used to fit a von Mises direction tuning curve to each cell using least squares curve fitting. The form of the function is as below:

$$R\left(\theta \right)=\left(A_1{e}^{\left[\beta \hspace{0.17em}\hspace{0.17em}\cos \left(\theta -{\theta }_{pref}\right)\right]}\right)+\left(A_2{e}^{\left[\beta \hspace{0.17em}\hspace{0.17em}\cos \left(\pi -\theta -{\theta }_{pref}\right)\right]}\right)+c$$

Where $R\left(\theta \right)$ is the response at direction $\mathrm{\theta }$, $A_1$ is the maximum amplitude of the first peak, $A_2$ is the maximum amplitude of the second peak 180 degrees away, $\mathrm{\beta }$ is the tuning width factor and ${\theta }_{pref}$ is the preferred direction of the cell and $\text{c}$ is the amount of offset. The location of the peak of this fitted tuning curve is used as the preferred direction of each cell.

Distance between cells is calculated as the Euclidean distance between their measured $\text{x}$ and $\text{y}$ distance within the plane. For maps pooled across planes, the pairwise distance and difference in preferred directions are still computed only for within plane pairs.

The exponential fit to the data measuring dependence of preferred direction difference between cells with the distance between cells used the following equation:

$$y=C-A{e}^{-kx}$$

Where $\text{y}$ is the fitted direction difference for $\text{x}$ distance between cells, $\text{C}$ is the saturation value, $\text{A}$ is the start value and $\text{k}$ is the inverse decay space constant. The parameters were estimated using least squares curve fitting to individual data points.

The shuffled data for measuring this dependence was generated by keeping the same cell positions but shuffling their preferred directions.

Hypercolumn estimates were generated by peak of the fourier transform of the response maps obtained in wide field imaging.

Principal Components Analysis (PCA): Means and standard deviations were computed for each cell across all response traces across trials to be projected in the PC space. The data was then z-scored using these means and standard deviations. Covariance was computed over the z-scored mean responses for all directions. The eigenvectors of the PC space were then identified using singular value decomposition of the covariance matrix.

Targeted dimensionality reduction (TDR): To directly estimate the relevant population response space explaining motion related variance, we used TDR (adapted from ⁵⁰). The original data was converted to reduced dimensionality by keeping principal dimensions 2–10. The response at each time point in the trial could be described using the following equation:

$$r_{c,t,\theta }={\alpha }_{c,t}\cos \theta +{\beta }_{c,t}\sin \theta +\gamma$$

Where $r$ is the response of cell $\text{c}$ at time $\text{t}$ in the trial for stimulus $\mathrm{\theta }$. The regression coefficients $\mathrm{\alpha }$ and $\mathrm{\beta }$ define the direction subspace and $\mathrm{\gamma }$ is offset. The regression coefficients were estimated by taking the Fourier transform of the response vector across all directions at each time in the trial. The time-independent regression coefficients were identified as the coefficients at time point $t_{max}$ in the trial such that:

$$t_{max}=argma{x}_t‖\sqrt{{\alpha }_t^2+{\beta }_t^2}‖$$

Where ${\alpha }_t$ and ${\beta }_t$ are both of length N, N being the number of cells. These time independent regression coefficients were then orthogonalized using QR decomposition. The resulting vectors were used to project the data in the direction subspace.

TDR vector length was computed as follows:

$$TDR\hspace{0.17em}vector\hspace{0.17em}length=\sqrt{{\left(Projectio{n}_{TDR1}\right)}^2+{\left(Projectio{n}_{TDR2}\right)}^2}$$

Error bars in Fig 4J on binned vector lengths as a function of activity were 95% bootstrapped confidence intervals on the median vector length.

Extracellular electrophysiology analysis: Single unit records from macaque area MT were measured with tungsten electrodes as in ⁷⁴. The response of each single unit at its preferred direction was used to estimate the response decay timing. Cell responses were normalized by dividing each cell’s response by its sustained firing rate during stimulation at preferred direction. Response latency was identified as the time it took for the mean baseline subtracted response to exceed 5% of its peak value, following motion onset. Response at this latency following motion offset was used as starting point for decay analysis. An exponential decay was fit to the cell’s response. The form of the function is as follows:

$$R\left(t\right)=A{e}^{-t/\tau }+c$$

Where $\text{R}\left(\text{t}\right)$ is the response at interval $\text{t}$ following motion offset and response latency. $\text{A}$ is the amplitude of response at initial point, $\mathrm{\tau }$ is the decay time constant in ms and $\text{c}$ is the offset.

Pre and post-stimulus activity was averaged from a 64 ms interval. Prestimulus interval was 64 ms before the start of the stimulus. Post-stimulus interval started at 188 ms following end of stimulus.

Intracellular electrophysiology analysis: Responses were median filtered with filter 30 ms to remove spikes. The mean response was baseline subtracted. Sustained response was calculated by averaging response in the last 1s before stimulus offset. The mean response was then normalized by dividing by the sustained response. The response to stimulus offset was fitted with the same exponential decay as above.

Analysis of amplification of direction signals (figure 1F): Neurons were binned based on their preferred directions. Mean z-score is computed per bin and average response is computed across stimulus conditions and used to measure output direction selectivity of the population. Normalized output selectivity is computed by dividing selectivity at a certain stimulus coherence level by selectivity observed at 100% coherence level. Similar analysis was done for records from V1 population obtained using neuropixels. Only direction selective V1 neurons were used for this analysis. The impact of changing the direction selectivity criterion was small (Sup. Fig. 3).

Change direction analysis: For both imaging and electrophysiology measures, neural responses were z-scored using the mean and standard deviation across repeats across different conditions. For the change direction stimulus, the responses were binned into direction bins for each condition, averaging the responses of cells with preferred direction within one bin. The bins were then rotated per condition so that 1^st and 2^nd stimulus direction were aligned for all conditions. Then the responses were averaged across conditions.

For the partial correlation analysis, mean population response to change direction conditions was binned in direction bins (averaging cells with direction preference within each bin to generate the bin response) and time bins. The response in each time bin was fitted with two fit functions, either to generate the switch or the sweep prediction. The switch prediction was generated as fit to the response was with sum of two gaussians separated by the degrees of difference between the two stimulus directions (90 or 135). The form of the function is as below:

$$R\left(\theta \right)=\left(A_1{e}^{\frac{-{\left(\theta -{\theta }_{pref}\right)}^2}{2{\sigma }^2}}\right)+\left(A_2{e}^{\frac{-{\left(\theta -\left({\theta }_{pref}+\mathrm{\Delta }d\right)\right)}^2}{2{\sigma }^2}}\right)+c$$

Where $\text{R}$ is the response in direction bin $\mathrm{\theta }$, $\text{A1}$ and $\text{A2}$ are the amplitudes for the two peaks, $\mathrm{\sigma }$ is the width of the Gaussian, $\mathrm{\Delta }\text{d}$ is the difference in the two stimulus directions and $\text{c}$ is the offset. The width of the Gaussian was set to be a constant ±1, making $\mathrm{\sigma }$ to effectively be a set parameter instead of a free parameter. This is done so that the number of free parameters (three) is same for the switch fits $\left(\text{A}1,\text{A}2,\text{c}\right)$ and sweep fits $\left(\text{A}1,\text{sigma},\text{c}\right)$.

The sweep prediction was generated as the fit to response with a single Gaussian, with ${\mathrm{\theta }}_{\text{pref}}$ varying smoothly between the first and second motion directions. The form of the function is below, with parameters same as above.

$$R\left(\theta \right)=\left(A_1{e}^{\frac{-{\left(\theta -{\theta }_{pref}\right)}^2}{2{\sigma }^2}}\right)+c$$

The correlation values were computed as ${\text{c}}_{\text{switch}}$ for correlation between the data and switch prediction, ${\text{c}}_{\text{sweep}}$ for correlation between the data and the sweep prediction, ${\text{c}}_{\text{switch}-\text{sweep}}$ for correlation between the switch and sweep predictions. All direction bins were included for the period between the peak response to the first stimulus and the peak response to the second stimulus. The partial correlation values were computed as follows:

$$R_{switch}=\frac{\left(c_{switch}-c_{sweep}{c}_{switch-sweep}\right)}{\left(\sqrt{\left(1-{c_{sweep}}^2\right)\left(1-{c_{switch-sweep}}^2\right)}\right)}$$

$$R_{sweep}=\frac{\left(c_{sweep}-c_{switch}{c}_{switch-sweep}\right)}{\left(\sqrt{\left(1-{c_{switch}}^2\right)\left(1-{c_{switch-sweep}}^2\right)}\right)}$$

The population output for different time slices was fit with the switch model.

For electrophysiology analysis in figure 5, 33 single units were used. From each unit, 8 cell profiles were generated by shifting the cell tuning in 45 degrees step. This generated a population of 264 cells. All cells were used in this analysis. The predicted rate shown in figure 5C was estimated by taking the mean of the predicted trace within 192 ms following start of stimulus 1. The predicted trace is conservatively estimated as max of response to stimulus 1 and response to stimulus 2. The response to stimulus 1 is a trace of 192 ms obtained from cell’s response to stimulus 1 alone starting at the stimulus start. The response to stimulus 2 is also of same length, obtained from cell’s response to stimulus 2, but it is shifted forward in time by the stimulus duration to mimic the change direction timeline. The observed rate is also computed as mean within 192 ms of stimulus start. Even with the conservative estimate, the observed response was lower than predicted, defying expectation of the ring model.

Change coherence analysis: Responses of cells were z-scored using the mean and standard deviation across repeats across different conditions. For the change direction stimulus, the responses were binned into direction bins for each condition. The bins were then rotated per condition so that stimulus direction was aligned for all conditions. Then the responses were averaged across conditions.

Spontaneous activity analyses: Responses of cells were z-scored using the mean and standard deviation across both stimulus-evoked and spontaneous conditions. Motion direction space was identified using TDR (described earlier) on the stimulus-evoked responses. Spontaneous activity trace was then projected into this same space. Data in figure 7A represents all spontaneous trials recorded for that particular session. TDR vector length was then computed for each time point as defined before. The time points were assigned a bin based on the mean z-scored population activity for that time point. Vector lengths were then max normalized using the maximum vector length observed in stimulus driven mean responses to motion along different direction. These values were then averaged within each activity bin to generate figure 7B.

Numerical simulations:

The model network consisted of 512 excitatory cells and 512 inhibitory cells with preferred directions spaced evenly across E cells and I cells. Our network simulations were run in time steps of 0.1 ms. Spiking activity in each step was simulated using the rate equation, we used a decay constant of 20 ms;

$$s_t=\left(1-\frac{dt}{\tau }\right)\times s_{t-1}+\frac{dt}{\tau }\lambda$$

Where $\text{dt}=0.1$ ms and $\mathrm{\tau }=20$ ms. The $\mathrm{\lambda }$ above is the total input rectified ^40,41 and passed through a saturating nonlinearity (output capped at 10). The total input was generated by summing the local, external and noise inputs. These varied for different conditions.

Local input was modeled as $W_s$ where $\text{W}$ is the weight matrix of the population and $\text{s}$ is the response matrix. Connectivity weight from an excitatory cell $\text{j}$ to cell $\text{i}$ is as follows:

$$w_{ij}^E=J_0^E+J_1^E\left(1+\cos \left({\theta }_j-{\theta }_i\right)\right)$$

Where $\mathrm{\theta }$ is the direction preference of the cell.

Similarly, connectivity weight from inhibitory cell $\text{j}$ to cell $\text{i}$ is:

$$w_{ij}^I=J_0^I+J_1^I\left(1+\cos \left({\theta }_j-{\theta }_i\right)\right)$$

To simulate single population response, weights were defined as

$$w_{ij}=w_{ij}^E-w_{ij}^I$$

External input was modeled differently based on the required predictions. The general form of the external input was

$$M=\left(\frac{1}{\sigma \sqrt{2\pi }}\sum _{k=-1000}^{1000}{e}^{\frac{-{\left({\theta }_{pref}-\mu +2\pi k\right)}^2}{2{\sigma }^2}}\right)$$

$$M=M/\max \left(M\right)$$

$$Inpu{t}_j^{ext}=F_{stim\hspace{0.33em}coh}M{c}_{cell\hspace{0.33em}type}+offset$$

Where ${\text{c}}_{\text{celltype}}$ was set based on whether cell was $\text{E}$ or $\text{I}$. $\mathrm{\sigma }$ controlled the width of the stimulus and was set based on context. ${\Theta }_{\text{pref}}$ is the cell’s preferred direction, $\mathrm{\mu }$ is the stimulus direction. $F_{stim\hspace{0.33em}coh}$ was stimulus coherence expressed as a fraction; percent stimulus coherence was divided by 100. The offset was used to keep the area under the stimulus curve (to replicate equal motion for all coherence levels) same for all coherence levels, no offset was used for 100% coherence stimulus, additional offsets were added to coherence levels less than 100.

Noise input for different conditions are detailed below.

For demonstrating different model regime predictions under different scenarios, simulations were run for following connectivity parameters:

	${{\text{J}}_1}^{\text{E}}$	${{\text{J}}_1}^{\text{I}}$	${{\text{J}}_0}^{\text{E}}$	${{\text{J}}_0}^{\text{I}}$
f-NA	0.15	0.0937	0.0937	0.85
s-NA	1.8	1.1250	1.125	2.5
CA	4.5	2.8125	2.8125	5.2
BA	4.2	4.2	4.2	4.9
NA (fig 8B)	0.5	0.3125	0.3125	1.2

BA corresponds to the transient amplification regime in the analytical derivations, f-NA and s-NA are in the homogenous regime and CA is in the marginal regime.

To demonstrate the effects of brief input pulse on the network, shown in figure 2D, no noise was used. The length of the input pulse was 0.5 ms, starting at 0. The stimulus was at 50% coherence, c = 0.5 for E cells and 0 for I cells.

The numerical response maps in supplementary figure 4 were generated by stimulating across the range of connectivity parameters. Each trial was 3 s long, external input was on for first half of the trial with μ set to zero and σ set to 68.75 degrees. No noise was used in this context. The numerical amplification was defined as follows:

$$A=\left|\frac{{\sum }_{j=1}^N{R}_j{e}^{i{\theta }_j}}{{\sum }_{j=1}^N{S}_j{e}^{i{\theta }_j}}\right|$$

Where $R_j$ is response of each model cell, ${\theta }_j$ is its direction preference, $S_j$ is the feedforward input for the direction ${\theta }_j$. Input c = 0.5 for E cells and 0 for I cells.

Figure 3A was simulated with input pulse 1 s long. No noise was used. The stimulus was at 50% coherence, c = 0.05 for E cells and 0 for I cells.

To demonstrate amplification as a function of stimulus coherence levels, shown in supp. Fig. 5, external input was simulated at coherence levels ranging from 0 to 1. Parameter c was set to 2 for both E and I cells for panels in A. Differential input panels in B were simulated by reducing c to I cells to 1.6 For simulation with noise in panels in A and B, each cell received an independent noise drawn from a normal distribution with mean 1 and standard deviation 0.2.

To simulate the change direction predictions, network was provided input at first stimulus direction in the first half of the trial. In the second half of the trial, the stimulus direction was shifted to the second direction. The input was defined as above with c set to 2 for both cell types, stimulus direction μ as 90 for stimulus 1 and 225 for stimulus 2, σ was set to 68.75 degrees. No noise was used in this context. Each trial was 1s long. For supp. Fig. 6 panel C, c was 1.6 for I cells and 2 for E cells.

To simulate the change coherence predictions, the network was provided input at a single direction of 180. Each trial was divided into 4 segments of equal duration, the total length of the trial being 2s long. For the external stimulus, c was set to 2k for both cell types, k being a scaling factor changing with coherence levels, μ was 180, σ was set to 68.75 degrees. k was set to 1, 0.7, 0.225 and 0.025, to match the stimulus coherence levels of 100, 70, 22.5 and 2.5%. Offsets were added to coherence levels other than 100 (see above). No noise was used in this context.

To simulate spontaneous activity in figure 7, no external input was used. Every cell received a noise drawn from a normal distribution of mean 1 and standard deviation of 0.5, updated at each time step of the simulation.

To simulate spontaneous activity in figure 8, every cell received a noise drawn from a normal distribution of mean x and standard deviation of 0.5, where x fluctuated in time. X for each time point was drawn from a normal distribution with mean 1, standard deviation 100. This array of X was then smoothed in time with 1s moving average filter. The resultant input across cells had a mean of 1 and standard deviation of 0.8.

The $\text{x}$ and $\text{y}$ projections, $R_x$ and $R_y$ respectively, of this activity were computed as follows:

$$R_x=\frac{{\sum }_j{R}_j\cos \left({\theta }_{pref,j}\right)}{{\sum }_j{R}_j}$$

$$R_y=\frac{{\sum }_j{R}_j\sin \left({\theta }_{pref,j}\right)}{{\sum }_j{R}_j}$$

Where $\text{j}$ is each cell of the model network, $R_j$ denotes its response, and $\left({\theta }_{pref,j}\right)$ denotes its direction preference. The vector length of the output at each time step was calculated as follows:

$$Vector\hspace{0.17em}length=\sqrt{{\left(R_X\right)}^2+{\left(R_Y\right)}^2}$$

The corresponding activity level was computed as the mean activity across all model cells. This activity was binned and the vector lengths for time points within each bin were averaged to give the mean vector length value at each activity bin. The error bars were computed as the standard deviation over these vector lengths.

Same procedure was used to simulate spontaneous activity in supplementary figure 14. Green points were generated with input from a normal distribution with mean 1, gray points were generated with input from a normal distribution with mean 2. Fraction time in spontaneous selective state was calculated number of points with vector length > 0.5 divided by total number of time points. Change direction predictions were generated same as in figure 4.

Analytical derivations are described in detail later. Briefly, the plots in figure 2 D–F are based on analytical derivations. Amplification (see description of gain in analytical derivations is defined as: $\left({\text{J}}_{\text{1}}^{\text{E}}+{\text{J}}_{\text{1}}^{\text{I}}\right)/\left(1-{\text{J}}_{\text{1}}^{\text{E}}+{\text{J}}_{\text{1}}^{\text{I}}\right)$.

Scaled time constant is defined as: $1/\left(1-J_1^E+J_1^I\right)$

Highlights:

1. Population dynamics in area MT reveal the mechanisms of cortical amplification.

2. Cortical amplification models can be differentiated using their distinct dynamics.

3. The dynamics of area MT are consistent with a balanced amplification model.

4. Balanced amplification increases sensitivity while maintaining fast dynamics.