Locomotion optimizes sensory representations through a computational principle shared by rodents and primates

Jonathan M Gant; Wiktor F Młynarski

doi:10.1126/sciadv.aed4172

. 2026 Aug 28;12(35):eaed4172. doi: 10.1126/sciadv.aed4172

Locomotion optimizes sensory representations through a computational principle shared by rodents and primates

Jonathan M Gant ^1,², Wiktor F Młynarski ^1,^3,^*

PMCID: PMC13524049 PMID: 42664357

Abstract

Behavior modulates the activity of sensory systems in multiple ways: from gain changes in individual neurons to changing interactions in neural populations. These effects are not universal; while movement has a strong influence on sensory coding in rodents, its impact on primates is less prominent. The diversity of effects that locomotion exerts on sensory neurons, as well as disparities between species, raises questions about the existence of universal principles that may underlie sensation during behavior. We propose that sensory systems are internally modulated to match systematic changes in stimulus statistics caused by locomotion, to facilitate an accurate and efficient sensory code. We find that model neurons, adapted to stimuli recorded during movement in natural environments, predict and reproduce a broad spectrum of experimental observations in rodents and primates. This simple principle of maintaining coding efficiency across behavioral states reconciles the diversity of ways in which locomotion modulates visual coding in different animal species.

INTRODUCTION

Behavior exerts a substantial modulatory influence on brain regions traditionally regarded as purely sensory (1). Locomotion, in particular, has been shown to alter the gain of visual responses across multiple species, including fruit flies (2, 3) and mice (4, 5). In rodents, movement induces more complex changes in the visual system, modulating not only the response strength but also the temporal dynamics of neural activity (6), spatial integration of stimuli (7), and the correlation structure of neural populations (8). These findings challenge the classical notion of sensory processing as a behaviorally invariant computation, highlighting instead the dynamic interplay between sensory encoding and behavioral state.

Recent experimental findings reveal an even more complex relationship between sensory coding and movement: Neurons in the primary visual cortex (V1) of primates are modulated by running in a different way than those of mice (9). These disparities raise intriguing questions regarding whether strategies of visual coding during movement are universal or species specific, and whether they could be understood with the same principles.

In this work, we take a normative perspective and consider how locomotion should modulate sensory neurons such that stimuli are encoded accurately and efficiently in different behavioral states. Our approach is grounded in the efficient coding hypothesis, an established theoretical framework that proposes design objectives for sensory coding (10–12). The hypothesis postulates that sensory systems adapt to the statistical structure of stimuli to maximize the amount of information transmitted, minimize encoding errors, and satisfy metabolic constraints. According to this theoretical perspective, at the onset of movement, the brain should adjust sensory neurons to systematic changes in input statistics caused by behavior. This strategy would ensure that stimuli are encoded efficiently and accurately in different behavioral states.

To understand how sensory input changes with locomotion in the natural environment, we recorded videos of natural scenes with a stationary and moving camera. We processed these videos with filters designed to mimic the receptive fields in V1 of mice and primates. We then developed models of sensory coding that are optimally modulated to match different statistical features of natural stimuli recorded during movement and rest. Optimized model neurons capture a range of phenomena observed in the mouse visual system: gain (4) and tuning curve modulation (8), changes in encoding accuracy (13), as well as population-level interactions that modulate spatial integration (14). By analyzing the temporal structure of natural stimuli during movement, we formed predictions about the dynamics of neural activity during locomotion and confirmed them by reanalyzing the experimental data of (6). To confirm that the observed changes in stimulus statistics are a part of the animal’s experience during natural behavior, we reanalyzed visual input data from the perspective of freely moving mice, made available in (15). Going beyond the visual system of rodents, we found that neurons with high spatial frequency receptive fields, such as those in the foveal area of primate V1, do not benefit from modulation because their statistics do not change sufficiently strongly with the animal’s movement. Our findings indicate that the diverse and seemingly disparate effects of locomotion on sensory coding share a common general principle: maintain the accuracy and efficiency of information transmission across different behavioral states.

RESULTS

The proposed principle of sensory coding during locomotion is based on two observations (Fig. 1A). First, locomotion substantially affects the sensory experience of the organism by changing the statistics of sensory input compared to the stationary state (Fig. 1A, thick black arrow). Second, if this impact is systematic and predictable, then the brain can preemptively adjust (i.e., modulate) sensory systems at the onset of movement such that they are prepared to efficiently process stimuli whose statistics are affected by locomotion (Fig. 1A, thick orange arrow).

A key assumption is that natural selection shaped sensory systems over evolutionary timescales to exploit the statistics of natural scenes which change with locomotion in a predictable way. The resulting modulation of sensory systems should therefore persist even in experimental settings that are not entirely natural. This principle places seemingly diverse phenomena, such as gain modulation, changes in temporal filtering, or network interactions during locomotion, within a common framework (Fig. 1A).

To characterize how natural sensory input changes with locomotion, we recorded videos by moving a camera in different natural environments (Fig. 1B). The camera was held close to the ground, mimicking the visual experience of a rodent, while we performed recordings in two locomotion states: stationary, where the camera moved slightly while being held in place, and during movement (see Methods for details). To understand how locomotion affects the statistics of natural stimuli and, consequently, signals encoded by visual neurons, we processed videos with linear Gabor filters reminiscent of receptive fields in the mouse primary visual cortex (Fig. 1C) (16). We found that the transition from a stationary state to movement substantially changes the filter output, even if both recordings were performed in the same natural scene. The impact of locomotion extends over multiple statistical features: increased variance and dynamic range of filter outputs (Fig. 1D, left), changes in their frequency spectrum (Fig. 1D, middle), as well as decreased magnitude of correlations between filter outputs (Fig. 1D, right).

While our camera recordings capture the full complexity of natural scenes, they coarsely approximate animal movement. In addition to locomotion, freely moving mice move their head and eyes, which adds complexity to the sensory input. To confirm that recordings of natural scenes approximate the real animal experience sufficiently well, we reanalyzed videos recorded from the perspective of freely moving mice, made available by Parker and colleagues (15) (Fig. 1E). These videos were recorded using a head-mounted camera as mice freely explored an experimental setup where artificial stimuli were displayed. The resulting recordings were therefore an approximation of the input to the mouse visual system during unconstrained, natural behavior. The dependence of these stimuli on the locomotion state was closely reminiscent of that of the natural scene recordings (Fig. 1F). We highlight this correspondence in different contexts throughout the remainder of this work. More fine-grained behaviors such as eye movements also influence stimulus statistics. However, their effect is likely more subtle than changes caused by locomotion. To confirm this intuition, we performed additional analyses on natural scene recordings with simulated saccadic movements as well as videos corrected for the eye position of the mice. We found no qualitative differences in the impact of locomotion on the analyzed stimulus statistics (see figs. S1 and S2).

Another important factor to consider is that mice perceive both green and ultraviolet (UV) light. Since the statistics of spectral content vary systematically with elevation (17, 18), we investigated how color and elevation influence the effects of locomotion on stimulus statistics. We found that locomotion reliably increases the dynamic range of filter outputs across colors and elevations (see fig. S3).

To better understand why movement affects stimulus statistics and how universal this impact is, we constructed a minimal model of sensing during locomotion (Fig. 1G). In the model, a point agent equipped with two sensors moves in a two-dimensional environment in which the average stimulus value changes with spatial position. During a stationary state, the agent’s position fluctuates around a small area, thus sampling a small region of the spatially correlated environment (Fig. 1G, blue frame). In turn, during locomotion, the agent samples larger regions during the same time period. The transition between these behavioral states results in a change in stimuli registered by the agent’s sensors that qualitatively mimics changes in V1-like filter outputs to natural videos and experimental stimuli in freely moving mice (Fig. 1H). Simulations of the minimal model suggest that the observed dependence of stimulus statistics on movement is universal and originates from the coarse spatial features of the scene. It should therefore be experienced by any organism moving through a heterogeneous environment.

Gain modulation maintains accuracy of sensory coding during locomotion

Perhaps the most salient type of modulation of sensory neurons by locomotion is the increase in gain of sensory responses (4). We postulate that the gain increases to match an increase in the stimulus variance experienced during movement (Fig. 2A). To illustrate this effect, we consider a linear Gabor filter, reminiscent of a mouse V1 receptive field (Fig. 2B, inset). The variance of the filter response to videos recorded during camera movement is visibly higher than in the stationary state (Fig. 2B, left, orange and gray lines, respectively). This increase is reflected in the distributions of SDs of the filter outputs (Fig. 2B, middle). Sensory input statistics in freely moving mice, as well as the simulations of the minimal model, are qualitatively in agreement with the statistics of natural stimuli (Fig. 2B, middle, inset), suggesting that the increase in variance is a fundamental property of sensory streams experienced during movement in spatially inhomogeneous environments. The variance change observed during locomotion is regular and systematic, and it generalizes across different filters, natural videos, and timescales of the analysis (Fig. 2B, right). It is also generated by spatiotemporal filters that mimic temporal selectivity of cortical neurons (fig. S4).

Fig. 2. — (A) Locomotion affects filter output variances and modulates the dynamic range of neural nonlinearities. (B) Left: example filter outputs during locomotion (orange) and stationary periods (gray). Middle: distributions of SDs of the example filter across videos in movement (orange) and stationary (gray) conditions. Triangles denote averages. Inset: same distributions obtained from recordings in moving mice (15) and the minimal model. Right: ratio of SDs of filter outputs during movement to those during stationary periods over different time intervals. Line depicts average and shaded area SD. (C) Left: mutual information between inputs and neural response $I (s; r)$ as a function of slope k and maximum firing rate L during movement (top) and stationary state (bottom). Middle: average firing rate. Right: utility function computed for the constraint $λ_{r} = 5$ . (D) Example nonlinearities optimized for stationary (gray) and moving (orange) conditions. Insets depict receptive fields of optimized neurons. (E) Ratio of optimal maximal firing rate L (dashed line) and slope k (solid line) in the moving condition versus the stationary condition for different constraints $λ_{r}$ . Lines depict averages and shaded areas SDs across the neural population. (F) Comparisons of published experimental results (top) and theoretical predictions (bottom). From the left: responses of an example neuron and the population (inset) to visual contrast (5), modulation of orientation tuning curves (black and white symbols denote population average multiplicative and additive modulation respectively, bars denote SDs) (8), population firing rate as a function of speed (4) (bottom: circles, average responses to videos recorded at different speeds; black line, exponential fit; inset, results from the minimal model), decrease of decoding error (blue), and increase in firing rate (black) at the transition to movement (13). pdf, probability density function.

To efficiently and accurately encode stimuli whose variance increases rapidly, sensory neurons must expand their capacity to match the change in stimulus statistics (19–22). The systematic increase in stimulus variance with locomotion suggests a modulation strategy in which sensory neurons increase their dynamic range to accommodate the greater variability of the stimulus received during movement. However, expanding the dynamic range while keeping the maximal firing rate constant inevitably lowers the fidelity of the sensory code, as broader stimulus distributions are mapped onto the same limited range of responses. To rigorously test this intuition, we constructed simple linear-nonlinear models of individual neurons, where the output of each Gabor filter is transformed to simulated neural activity through a logistic nonlinearity (Fig. 2A). For each of the locomotion states, we optimized the slope term k and the maximal firing rate L to maximize the utility function $U (k, L)$ that trades off information transmission, measured as the mutual information $I (s; r)$ between filter outputs s and neural activity r, with the average firing rate $〈 r 〉$ , subject to a constraint $λ_{r}$ (Fig. 2C; see Methods for details). To focus only on changes in the dynamic range of stimuli, we assumed that neurons adapt to the mean change at the timescale of the considered excerpts. We found that to match the systematic increase in stimulus variance, neural nonlinearities optimized for the locomotion state simultaneously decrease their slope k to expand their dynamic range and increase their maximal firing rate L, compared to the stationary state, to maintain high-fidelity information transmission (Fig. 2D, orange and gray lines, respectively). This indicates that the benefit of encoding higher-variance stimuli with high fidelity during locomotion outweighs the metabolic cost of a higher average firing rate. These changes generalize over a range of constraint values $λ_{r}$ and across a broad range of model Gabor receptive fields (Fig. 2E). The predicted increase in gain during movement is also observed in neuron models that include input and output noise (fig. S5).

The relatively simple modulation of neural nonlinearities optimized to efficiently encode stimuli whose variance is affected by locomotion can already account for a range of experimental results published previously (Fig. 2F). The expansion of nonlinearities explains the increase in the gain of contrast tuning curves during locomotion reported in (5) (Fig. 2F, left) and the modulation of orientation tuning curves reported in (8) (Fig. 2F, second from the left). Beyond individual neurons, the theory provides a quantitative match at a population level, where locomotion-induced modulation of orientation tuning curves exerts a stronger multiplicative rather than additive effect (Fig. 2F, second from the left, black and white symbols, respectively). To understand how locomotion at different speeds should modulate neuronal gain, we optimized model neurons to statistics of natural videos recorded at different speeds of movement. Optimal gain modulation does not continue to increase with locomotion speed but saturates quickly after the onset of movement, which is reflected in the simulated population activity (Fig. 2F, third from the left, bottom), as well as the stimulus statistics in the minimal model (Fig. 2F, third from the left, bottom, inset). This result matches the experimental findings of (4) (Fig. 2F, third from the left, top), suggesting that there is no benefit in expanding the dynamic range of neurons beyond what is required to accurately encode stimuli observed at relatively low movement velocities. This result is robust to the statistics of individual environments and different spatial frequencies of the Gabor filters (fig. S6). Last, the increased dynamic range of sensory neurons during locomotion results in improved accuracy of sensory coding if the dynamic range of the stimulus is kept constant, as in the experiment. This improvement has been reported by Mineault et al. (13), who quantified it using the population decoding error (Fig. 2F, right, top). We performed an analogous analysis on simulated neurons and found that the population decoding error decreases at the transition to locomotion, which is accompanied by an increase in the average firing rate, thus resembling experimental results (Fig. 2F, right, bottom row, blue and black lines, respectively) (13).

Modulation of temporal filtering ensures efficiency of sensory coding during locomotion

Beyond gain increases, locomotion alters temporal processing and induces more sustained responses (6). We propose that this form of modulation adjusts sensory neurons to changes in the temporal statistics of stimuli induced by locomotion (Fig. 3A). These changes are apparent in the average temporal spectra of the response of the Gabor filters to natural videos, where locomotion causes an increase in the spectral power at all temporal frequencies relative to the stationary condition (Fig. 3B, left). During movement, a sensory neuron with a Gabor-like spatial receptive field is subject not only to a substantial change in the total power of its input (as reflected by variance; see the previous section) but also to a change in its frequency content. Statistics of videos from the perspective of freely moving mice, as well as the minimal model, reproduce the observed increase in spectral power seen in natural stimuli (Fig. 3B, left, inset). This correspondence supports the hypothesis that such temporal dynamics are intrinsic to sensory experiences during locomotion through spatially diverse settings.

Fig. 3. — (A) Locomotion affects the temporal spectra of filter outputs and modulates temporal filtering in individual neurons. (B) Left: the average power spectrum of filter outputs during locomotion (orange) and rest (gray). Inset: same spectra from recordings in moving mice (15) and the minimal model. Right: A temporal filter with a frequency threshold transmits a fraction of the signal power computed as the area under the curve. Frequency threshold during movement $f_{m}$ and stationary condition $f_{s}$ are denoted by orange and gray triangles, respectively. The total power during movement (orange area) is twice that of the stationary condition (gray area). (C) The total power transmitted by the temporal filter as a function of the frequency threshold for stationary (gray) and moving conditions (orange). Dashed lines: example power levels considered in (B). (D) Left: The difference in the frequency threshold between the stationary and moving conditions for a range of energy constraints ( $Δ$ power). Right: the autocorrelation functions of temporal filters for the stationary (gray) and moving conditions (orange). Shades of orange correspond to energy constraints denoted with orange circles in the left. (E) The responses of mouse V1 neurons to moving dot stimuli (gray background indicates stimulus period) during locomotion (orange) and stationary (gray) states. Experimental data from (6) is reanalyzed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/deed.en). (F) Comparisons of reanalyzed experimental results from (6) (top) and theoretical predictions (bottom). Top: the autocorrelation functions of trial-averaged responses of single neurons. Top right: population averages for the stationary and moving conditions. Shaded areas denote SDs. The decay time is significantly longer in the moving condition. Bottom: autocorrelations of individual model neurons whose temporal filters correspond to the 50% increase energy constraint. Bottom right: same as top-right but for population of model neurons. Top plotted from data provided by (6) under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/deed.en).

Sensory neurons operate under metabolic constraints and must therefore balance their activity with the need to transmit information. In the temporal domain, this entails prioritizing frequencies with higher power, because they contain a larger fraction of stimulus information (11, 23). Ideally, a neuron would encode the same frequency range in all behavioral states. However, the elevated power spectrum during locomotion poses a challenge: Even with increased metabolic capacity, a neuron may not be able to transmit all frequencies and instead needs to prioritize those with higher power.

To formalize this idea, we modeled the temporal processing of neurons as a linear filter in the frequency domain that transmits frequencies below a certain threshold (including a linear roll-off; see Methods), reflecting the power-law decay of temporal frequency power observed in natural stimuli (Fig. 3B). We considered a scenario where, rather than being fixed, the frequency threshold adapts to the changes in temporal statistics of stimuli by increasing the metabolic resources available during locomotion. This adaptive threshold modulated by the locomotion state can therefore take two different values $f_{m}$ and $f_{s}$ during movement and stationary states (Fig. 3B, right, orange and gray triangles, respectively). For example, when the frequency threshold for the stationary state enables transmission of all frequencies (i.e., $f_{s} =$ 15 Hz), the frequency threshold $f_{m}$ that corresponds to a 100% increase in total power with respect to the stationary condition is equal to 2.9 Hz (Fig. 3B, right, shaded areas), demonstrating the prioritization of lower temporal frequencies which have higher power. To systematically assess the effect of changing the frequency threshold, we computed the total power transmitted through the temporal filter as a function of frequency threshold in stationary and moving conditions (Fig. 3C). Using the power required to transmit the entire spectrum in the stationary condition as a reference for neuronal capacity, we systematically increased the allowable power during movement and determined the corresponding frequency thresholds. Across a wide range of metabolic constraints computed as a percentage increase in power relative to the stationary state, the filter thresholds in the moving condition were consistently lower than during rest (Fig. 3D, left). This effect is further evident in the autocorrelation functions of the temporal filter responses to broadband white noise signals (Fig. 3D, right). In the stationary state, all frequencies are passed through by a model neuronal filter, and the autocorrelation of the output is equal to that of the white noise input, i.e., a delta function (Fig. 3D, right, gray line). During locomotion, the model neuron acts as a low-pass filter, which is reflected in a slower decay of the corresponding autocorrelation functions, where the decay time is determined by the strength of the metabolic constraint (Fig. 3D, right, orange lines). These analyses highlight that even when metabolic constraints are relaxed, temporal processing during locomotion should remain more biased toward lower frequencies than during stationary periods. The theory thus generates a clear prediction: Neurons should exhibit more pronounced low-pass temporal filtering during locomotion than during rest.

To test this prediction, we reanalyzed data from Horrocks et al. (6), who recorded neuronal responses in mouse V1 to moving dot stimuli during periods of locomotion and rest (Fig. 3E). We computed autocorrelation functions of the average neural responses in stationary and moving conditions, as well as the average autocorrelation across the population (Fig. 3F, top row). Comparison of empirical autocorrelations with theoretical predictions for individual Gabor filters and the filterbank average revealed close agreement between data and theory (Fig. 3F, bottom row). As a quantitative metric, we calculated the decay time (the time lag at which the autocorrelation drops to 0.5) and found it was significantly longer during locomotion in both the experimental data and a population of simulated model neurons (Fig. 3F). We note that in the experimental setting considered here, the sensory input is held constant across behavioral states. However, in natural conditions, the input statistics change significantly during movement, and the power of higher frequencies increases strongly. Therefore, it is possible that despite low-pass filtering, neural activity in response to natural sensory input is less temporally correlated during movement than at rest. Together, these results demonstrate that the temporal processing strategies predicted by the proposed efficient coding principle are reflected in neural responses, supporting the idea that locomotion modulates temporal filtering in V1 to account for systematic changes in temporal stimulus statistics.

Modulation of network interactions maintains a nonredundant sensory code during locomotion

Locomotion modulates not only individual neurons but also activity at the population level by modifying interactions between cells (8). A prominent manifestation of this effect is changes in the spatial integration of visual stimuli during locomotion, which is likely mediated by network interactions modulated by the behavioral state (14).

As in the previous sections, our first aim was to characterize how inputs to multiple neurons change simultaneously during locomotion. To do so, we analyzed the correlations of responses of receptive field–like filters to natural videos (Fig. 4A). To ensure that filters were nonredundant and optimized as a population, we learned them using independent component analysis (ICA), which reproduces the properties of V1 receptive fields (Fig. 4C, right) (24, 25). ICA was trained on a large set of image patches consistent with the size of the receptive fields in mouse V1 (Fig. 4C, left). We observed that locomotion has a strong impact on the correlations of the filter outputs. In stationary periods, filter responses tend to be highly redundant due to the shared, slowly changing sensory drive corresponding to slow fluctuations of the visual field (Fig. 4B, left, gray lines). The onset of locomotion significantly decreases the magnitude of filter correlations (Fig. 4B, middle, orange lines). As previously, we confirmed that this pattern is systematic and that the magnitude of the correlation is significantly stronger during the stationary state across all filter pairs and natural videos, as well as videos recorded in behaving animals (Fig. 4B, right). The minimal model matches the statistics of the natural videos; neighboring sensors of an agent moving in two-dimensional environments reveal a similar dependence of correlation magnitude on locomotion speed, suggesting the universality of this pattern.

Strong correlations between filters are a form of redundancy. Redundant neurons simultaneously convey the same information, which decreases the efficiency of sensory coding in the low-noise regime (26). To minimize redundancy, neuronal networks should decorrelate their inputs (26, 27). In the visual cortex, center-surround interactions are believed to reflect lateral inhibitory interactions within a local network that support efficient neural representations through redundancy reduction (28, 29). These observations suggest a strategy for modulation of neuronal networks by locomotion state. During stationary periods, correlations between receptive fields are high, and neurons should decorrelate their outputs, which is achieved by strong lateral inhibition. During locomotion, correlations between sensory inputs decrease; hence, inhibitory interactions should weaken since there is no need to decorrelate stimuli. This network modulation mechanism ensures coding efficiency across behavioral states at the level of neuronal populations.

Following these lines of thought, we developed a model of lateral inhibition (Fig. 4D). Our approach is based on previous models of redundancy removal in natural scenes through local interactions in a sensory population (29). In the model, the output of each linear receptive field was transformed with a thresholding nonlinearity to a positive firing rate. The activity of each neuron was linearly suppressed by other neurons in the population to minimize the magnitude of correlations (i.e., redundancy; see Methods for details). For each neuron and each stimulus interval, this lateral inhibition was controlled by an independently optimized synaptic weight vector $\vec{w}$ . The lateral inhibition model successfully reduces the correlations between all pairs of model neurons and during all stimulus intervals (Fig. 4E, left). The decrease is most prominent during the stationary state, when the initial correlation magnitude is high (Fig. 4E, left, gray bars). However, even during locomotion when the correlation magnitude is much weaker, the relative effect of decorrelation is also noticeable (Fig. 4E, left, orange bars). We note that in reality, sensory systems must dynamically adapt their decorrelation mechanisms to changing stimulus statistics, a process which we approximate by optimizing weights independently for individual stimulus excerpts of 5 s. Although the neural implementation of dynamic decorrelation goes beyond the scope of this work, we believe that it could be instantiated using mechanisms similar to decorrelation (30) or adaptive whitening by inhibitory neurons (31).

The lateral inhibition model fit to natural videos predicts that, to maintain high efficiency of coding, the state of the network should adjust by decreasing the magnitudes of inhibitory interactions during locomotion (Fig. 4E, right). To verify these predictions, we simulated neural responses by exposing model neurons to oriented sinusoidal gratings of increasing diameter, a set of stimuli used to test spatial integration during locomotion (Fig. 4F) (14). To simulate different states of movement, we used two sets of inhibitory weights optimized for one excerpt of the natural video during the stationary state and one during locomotion. When the diameter of the stimulus extends beyond the receptive field of a neuron, it activates other members of the population, thus inducing an inhibitory effect and suppressing the response of the analyzed neuron. Because of differences in the optimized weights, suppression is much stronger in the simulated stationary condition than during movement (Fig. 4F, bottom row, first and second from the left, gray and orange lines, respectively). This effect matches the results reported in (14) at the level of individual cells (Fig. 4F, top row, first and second columns from the left), as well as the population average computed across all model neurons and natural video excerpts (Fig. 4F, third column from the left). The inhibitory effect can be further quantified by the suppression index (14), whose values are lower during locomotion and qualitatively match the experimental results (Fig. 4F, right column). A close resemblance of simulations and experimental data suggests that network interactions can also be modulated by locomotion to optimize processing of correlated stimuli in the visual cortex.

Stimulus statistics explain differential modulation of sensory coding by locomotion in rodents and primates

An intriguing recent line of research revealed that modulation of sensory areas by locomotion is not universally preserved in the animal kingdom. In contrast to rodents, movement has a very weak effect on sensory responses in V1 of marmoset monkeys (9). It is unclear whether this disparity implies different species-specific computational objectives or whether it could be understood within the same theoretical framework.

An important difference between receptive fields of neurons in mouse and marmoset V1 is their size and spatial frequency. While mouse neurons have large, low-frequency receptive fields on the order of 10° to 20° of visual angle (16), the foveal receptive fields in monkeys are much smaller and higher frequency (9, 32, 33). To understand the impact of locomotion on higher spatial frequencies, we analyzed responses of Gabor filters of differing sizes and spatial frequencies (Fig. 5A). As we have shown in previous sections (see Figs. 1 and 2), the outputs of a representative mouse-like receptive field (Fig. 5A, left, blue frames) fluctuate much more during movement than during stationary periods (Fig. 5A, middle, orange and gray lines, respectively). The output of a significantly smaller, higher-frequency Gabor filter reminiscent of a primate foveal receptive field (Fig. 5A, left, yellow frames) behaves differently. Already in the stationary state where the field of view (FOV) moves slowly, the filter output fluctuates very rapidly, and the relative change in the filter output variance during movement is negligible (Fig. 5A, right). To establish whether this effect is systematic and may be the basis for the previously observed differences between species, we generated two sets of Gabor filters whose spatial frequencies were matched to receptive fields recently measured in mouse and marmoset V1 reported in (33) (Fig. 5B). The SDs of mouse-like filter responses to videos recorded during movement are, on average, almost twice as high as those for stationary videos (Fig. 5C, left, blue bar). The relative change in SDs of marmoset-like filters between stationary and moving conditions is minimal, and their average ratio is 1.14 (Fig. 5C, left, yellow bar). We emphasize that while the SDs of marmoset-like filter outputs have lower magnitude than the SDs of mouse-like filter outputs due to the statistics of natural scenes, the predictions about gain modulation depend only on the relative change in filter output SDs between stationary and moving conditions. We also note that the variance of filter modulation is relatively high for low-spatial-frequency filters, and it practically vanishes for high spatial frequencies (Fig. 5C, left, error bars).

These disparities between mouse- and primate-like filter outputs can be explained by a minimal model of sensory signals during movement (Fig. 5C, right). In a spatially correlated environment, locomotion increases the variability of the sensor output (Fig. 5C, right, blue frame and bar, respectively). In natural scenes, these spatially correlated inputs are processed by low-spatial-frequency filters. In contrast, high-frequency filters process visual features that are much less spatially correlated. Movement in a simulated environment that matches this property and is weakly correlated does not affect the sensor variance (Fig. 5C, right, yellow frame and bar, respectively). Minimal model simulations suggest therefore a universal relationship between the spatial frequency of the filter and the impact of locomotion.

Our next step was to understand how the frequency-specific impact of locomotion on filter output should affect the gain of sensory neurons. Following the procedure described in a previous section (see Fig. 2), we optimized nonlinearities in two populations of model neurons with mouse- and marmoset-like receptive fields. To emulate experiments comparing these two species (9), we exposed these two populations of model neurons to identical random stimuli (see Methods). We simulated transitions from the stationary state to running by switching nonlinearities optimized to natural videos recorded in these two conditions. Following the analysis of (9), we quantified the locomotion-induced gain change by computing the ratio of the firing rates during locomotion and the stationary state. The distribution of the firing rate ratios of model neurons closely resembles V1 neurons in mice and marmosets (Fig. 5D). In mice and low-frequency model neurons, the average activity is significantly increased during locomotion (Fig. 5D, left column, blue). We observe a quantitative match between the average ratio of firing rates during running and rest {the average ratio reported in (9) is 1.402, 95% confidence interval (CI) [1.365, 1.440], while we find that the average ratio for the low-frequency model neurons is 1.65, 95% CI [1.63, 1.67]}. In contrast, the marmoset and high-frequency model neurons are not modulated on average (Fig. 5D, right column, yellow). In both experimental data and theoretical predictions, the average firing rate ratio is equal to 1 {the average ratio reported in (9) is 1.011, 95% CI [0.995, 1.027], while we find that the average ratio for the high-frequency model neurons is 1.00002, 95% CI [0.99996, 1.00008]}. The match between theory and data goes beyond averages of firing rate ratios—the variability of modulation by locomotion is higher in mice than in marmosets, as quantified by the CIs. The experimental study (9) has reported that the gain of foveal marmoset neurons can be minimally suppressed by locomotion, when encoding stimuli with preferred orientation, but remains unchanged when considering all stimuli. We believe that the weak suppression to preferred stimulus orientations may be caused by factors that go beyond the statistics of natural stimuli analyzed here. Our theory provides an explanation of the behavior of neurons averaged across a broad range of stimuli encountered in natural environments.

The transition to movement modulates the gain of many neurons simultaneously. This population-level effect can be quantified using a shared modulator model that infers a single latent variable that controls the gains of neural responses independently of the sensory input (Fig. 5E, left) (9). In mice, the onset of movement causes a strong increase in the response gain, which is reflected in the difference between the average modulator values during running and rest [Fig. 5E, middle, plot reproduced from (9)]. In marmosets, this difference becomes weakly negative and effectively vanishes [Fig. 5E, middle, plot reproduced from (9)]. We performed an analogous analysis on simulated populations by approximating the shared gain modulator using linear methods (see Methods). In mouse-like model neurons, the increase in firing rates induced by the onset of locomotion can be captured by a single latent modulator, which markedly changes its value between behavioral states (Fig. 5E, right). In marmoset-like neurons, the transition between locomotion states does not cause a population-level gain change; therefore, the difference between modulator values is effectively zero (Fig. 5E, right). We therefore conclude that modulation which preserves the efficiency of the sensory code offers an explanation of population-level effects associated with locomotion in these two species.

The results described above suggest that the differences between the modulation of V1 in mice and marmosets are caused by changes in input statistics associated with locomotion and do not result from species-specific computational principles. This observation generates an immediate prediction: Even in the same species, locomotion-induced gain modulation should be larger in neurons tuned to low spatial frequencies, whereas neurons with high-frequency receptive fields should remain unaffected. A recent study has shown that peripheral neurons in marmoset V1, whose receptive fields are comparable to mouse neurons, are significantly modulated by running [the average ratio of the firing rates reported in (34) is 1.33]. Following the procedure outlined above, we simulated a population of neurons with peripheral-like receptive fields [Fig. 5F, left and right; we relied on the data published in (32)]. The gain changes quantified by firing rate ratios in the simulated neurons are in agreement with the experimentally measured activity of marmoset neurons with peripheral receptive fields (Fig. 5F, right and middle, respectively; the average ratio is 1.57, 95% CI [1.55, 1.59]). Our findings thus suggest that differences between mice and marmosets are a manifestation of the same computational principle—modulation that adjusts sensory neurons to systematic changes in stimulus statistics evoked by behavior.

DISCUSSION

The simple yet general principle of adapting to the statistical structure of sensory input can explain many seemingly disparate effects that locomotion exerts on sensory systems. By extending the efficient coding hypothesis, we provide a unifying account of phenomena observed within individual species: modulation of gain, changes in temporal response dynamics, and spatial integration in mice, as well as differential modulation of foveal and peripheral neurons in primates. The theory was also able to reconcile the differences between species in gain modulation and the dimensionality of population responses during locomotion. We thus demonstrate that differences in locomotion-induced modulation of sensory systems do not need to be a species-specific computation but a manifestation of a general coding principle.

Toward a theory of sensory coding in behaving animals

Theories that underlie our understanding of sensory systems are largely concerned with passive processing and transmission of sensory information (12, 35). In its original statement, the efficient coding framework focuses on properties of natural stimuli (10, 25) and does not reference the interplay between the behavior of the organism and statistics of the sensory experience. In this work, we make a step toward developing a theory of sensory coding in freely behaving organisms. Among the many ways in which behavior may affect sensory input, we focus on a relatively coarse, yet prominent and relevant, behavioral state—locomotion. Our key proposal is that the strong and predictable change in stimulus statistics caused by the onset of movement is a form of regularity that can be exploited by the nervous system in the same way that receptive fields of neurons adapt to the correlation structure of natural scenes to increase efficiency of sensory codes (25, 36, 37). However, in contrast to tuning properties of sensory neurons, this form of adaptation to the sensory niche needs to take the form of an internal modulation mechanism. This modulation dynamically modifies gains, temporal filtering, and network interactions, which improves the coding accuracy and minimizes resource use.

The proposed principle is related to theories of sensory adaptation that are embedded within the efficient coding framework (12, 19, 20, 38). Adaptation is a change in a sensory code that occurs in response to a change in the environment to maintain the efficiency and accuracy of the representation. The key difference is that changes in the environment occur independently of the organism and have to be estimated from sensory input (22, 39–41). In contrast, fluctuations in stimulus statistics that are systematically and predictably caused by behavior do not have to be inferred or estimated. The brain can modulate the sensory system using only internal signals to maintain the efficiency of coding during movement and rest.

Extensions and future directions

A natural direction for future work is to explore progressively better approximations of natural sensory experience in freely behaving animals. One important factor is color vision. Mice perceive two colors: green and UV (17). The effects described here likely generalize to mouse color vision: The green channel in commercial camerascorresponds to the mouse spectral sensitivity, stimuli in UV and green channels are strongly correlated (17), and UV can be approximated by the blue channel, which is also captured by our recordings (42). At a more fundamental level, our minimal model demonstrates that the impact of locomotion on stimulus statistics is a general property of movement in heterogeneous environments. At the same time, color stimuli in natural environments have interesting and nuanced structure. For example, green and UV channels are more strongly correlated in the lower that in the higher visual field (17, 18) (although the effects described here persist across the entire visual field—see fig. S3). Sensory systems exploit these dependencies by instantiating opponent color representations (43, 44) and adapting to the differences between upper and lower visual fields (37, 45). It is therefore likely that behavior affects visual stimuli in different colors and parts of the visual field in subtle yet structured ways and that these changes are exploited by modulatory mechanisms similar to those described here. Future work will reveal how more nuanced, behaviorally induced changes in the ecologically valid range of the light spectrum can be exploited by the mouse visual system (46).

Across the animal kingdom, the fine-grained statistics of sensory experience are also affected by species-specific behaviors. In this work, we focus on two very general behavioral states shared by almost all species: locomotion or staying still. Our analyses, as well as the minimal model, show the generality of the impact that the onset of movement has on sensory stimuli. However, the differences in movement of rodents and primates certainly influence the higher-order stimulus features they experience. Understanding the impact of species-specific movements on sensory coding is one of the tantalizing directions for future research (47).

Even within the same species, forward locomotion is only one movement pattern that can systematically change sensory input. We expect that other types of movement associated with systematic changes in stimulus statistics can also modulate neurons to optimize the sensory code. In one example, looking up can increase the response strength of neurons in the mouse visual thalamus (48). Because contrast increases with elevation in the natural visual field of a mouse (17, 18, 37), increasing the gain of neural responses during head raises may be an adaptation to maintain the efficiency of the sensory code.

On average, firing rates of neurons in the mouse visual cortex increase at the onset of movement and saturate with increasing running speed (4). The theory proposed here predicts this effect based on the changes of stimulus variance at different speeds of movement in heterogeneous environments (Fig. 2F). A fraction of neurons in the mouse visual cortex deviate from this increasing and saturating gain modulation pattern. Running speed can, for example, decrease gain of sensory responses or modulate it nonmonotonically (7, 49, 50). Our theory offers a possible, speculative interpretation of this diversity. Neurons whose gain increases and saturates with running speed may be functioning as “sensors.” In this view, their role would be to encode sensory inputs with high accuracy and convey them downstream. Neurons with nonsaturating gain modulation could play different roles. Future work could explore and test this hypothesis further.

The computational principles described here may be implemented in many different ways by neural circuits. In one example, decorrelation of stimuli could be achieved either via lateral inhibition or by top-down feedback signaling (51). The locomotion state could also affect many more features of neuronal tuning. In particular, coordinated modulation of joint spatiotemporal receptive fields could, in principle, further improve the efficiency of sensory coding in different behavioral states.

An additional set of questions concerns the interaction between stimulus statistics in different environments and locomotion. Here, we consider systematic, average changes in input statistics elicited by movement in different natural scenes. These relationships could be more nuanced, and the activity of sensory neurons is likely modulated by a combination of internal signals, as well as adaptation to scene statistics. Movement in absolute darkness is a particularly salient example of such an interaction: Because there is no visual input to encode, behavioral modulation could be, in principle, suppressed as well. In mice, V1 neurons are modulated during running in total darkness (4, 7). These results are, however, consistent with our interpretation: Internal modulation prepares neurons for stimulus statistics experienced on average throughout the visual experience and not in a particular sensory scene (e.g., dark environment).

The influence of behavior on sensory coding takes many different forms (1) and extends beyond the effects described here. Although, on average, locomotion increases the strength of sensory responses in mice, it may also decrease it or modulate it in a nonmonotonic way (7, 49). These modulation patterns can also vary between different cell types (52, 53), suggesting a diversity of computational functions and objectives. In addition to locomotion, saccades are another behavioral factor that is particularly important for sensory coding. Eye movement patterns vary between species (54) and may play different roles in animals with and without a fovea (e.g., primates and mice, respectively). Recent results show that saccades interact with V1 activity in nuanced ways (33, 55), suggesting that the computational objectives of these interactions are different from the principles discussed here. In addition to saccades, cortical activity in mice is driven by fine-scale orofacial movements such as whisking (56) as well as “twitching,” i.e., minor nongoal-oriented behaviors (57). This kind of modulation is absent in macaques (47, 58). Because small body movements are unlikely to markedly influence the statistics of sensory input, they are unlikely to be a manifestation of the modulatory principles proposed here and may require a different explanation. To support flexible behavior and action planning, sensory computations must go beyond merely representing stimuli with high accuracy and must engage in more complex computations, for example, detection of sensorimotor mismatches (59), surprise detection (60), or perceptual inference (61). Although the current work does not focus on more complex computations, we foresee that the principles proposed here could extend to objectives other than efficient representation of stimuli.

Outlook

Our results complement previous theoretical studies that identify common themes in sensory systems of diverse organisms, such as the encoding of auditory spatial cues (62) or the sampling of natural scenes through saccadic eye movements (54). By highlighting the existence of simple principles that can explain different strategies of sensory coding used by various species, these studies offer the hope of developing a general theory of information processing by biological sensory systems.

METHODS

Video recordings

We recorded videos of a variety of natural environments while the camera was stationary and moving. Using a GoPro Hero10 Black mounted on an extension pole, we mimicked the perspective of a mouse by holding the camera close to the ground. We recorded 10 videos, each with a duration of ∼1 min, in five different environments for each locomotion condition for a total of 100 min of video data divided evenly between stationary and moving conditions. For the stationary recording, we held the camera in roughly the same position, and the FOV was weakly fluctuating. For the moving recordings, we walked with the camera in a straight line at an average speed of ∼50 cm/s. For the purpose of an additional analysis, we recorded videos at different speeds (see below). For all recordings, the camera was set to a linear digital lens with motion stabilization turned off. All videos were recorded in 1080p HD, 1920 by 1080 pixels (px), at 30 frames per second with a 16:9 aspect ratio. The FOV of the camera is approximately 92 (horizontal) by 61 (vertical) degrees (deg) according to online https://community.gopro.com/s/article/HERO10-Black-Digital-Lenses-FOV-Informations?language=en_USdocumentation. Hence, there are approximately 20 px per 1 deg of visual angle. Before recording in a new environment, the exposure of the camera was automatically computed and fixed for all videos in that environment.

Gabor filters

Before applying filters, each video was normalized by subtracting the mean and dividing by the SD across all pixels over time. To analyze statistics of natural videos, we used a large set of parameterized Gabor filters of the following form: $g (x, y, λ, θ, ϕ, σ, γ) = exp (- \frac{{x^{'}}^{2} + γ^{2} {y^{'}}^{2}}{2 σ^{2}}) cos (\frac{2 π x^{'}}{λ} + ϕ)$ , where $x^{'} = x cos θ + y sin θ$ and $y^{'} = - x sin θ + y cos θ$ . In this equation, $λ$ denotes the spatial wavelength (inverse of the spatial frequency), $θ$ defines the orientation, $ϕ$ is the phase, $σ$ is the SD of the Gaussian envelope, which controls the spatial extent of the filter, and $γ$ is the spatial aspect ratio. For all filters, we set γ = 1. We considered a range of orientations $θ \in {0,22.5,45,67.5,90,112.5,135,157.5}$ deg and phases $ϕ \in {0, 90, 180, 270}$ deg. The envelope size was determined by the spatial wavelength of the filter, i.e., $σ = λ / 3$ . Filters were replicated at nine different positions in the scene at 1/4, 1/2, and 3/4 of both the horizontal and vertical dimensions. Depending on the analysis, we considered different numbers of filters with different distributions of spatial frequencies, ranging from 0.02 to 6 cycles per degree (cpd). Details are described in the corresponding sections below.

Variance of filter outputs

We split the time series signal of each filter output for every video into overlapping excerpts of length $τ \in {1,2,5,10,20}$ s that were shifted by a single sample. In each of these windows, we computed the SD of the filter outputs. After grouping the data by locomotion condition and spatial frequency, we computed the mean SD over time windows and across various filter orientations, phases, and spatial positions.

Nonlinearity optimization

We simulated the response of neurons by passing the outputs of the Gabor filters, denoted by s, through a logistic nonlinearity: $r (s) = \frac{L}{1 + e^{- ks}}$ , where L is the maximal firing rate and k is the slope. Assuming that the neuron adjusts its offset to match the mean of the filter output in a short time window, we only consider the second-order moment and approximate the distribution of filter outputs using a Gaussian distribution with the same variance. We identified the optimal parameters $k^{*}$ and $L^{*}$ that maximize the utility $U (k, L) = I (s; r) - λ_{r} 〈 r 〉$ for a range of Gaussian stimuli, where the SD $σ$ was matched to the empirically observed range of SDs from the windowed filter outputs. Here, $I (s; r)$ is the mutual information between the stimulus and the response, $〈 r 〉$ is the mean firing rate, and $λ_{r}$ is a constraint parameter. Specifically, we evaluated 280 $σ$ values spaced logarithmically between 10⁻¹ and 10³ and, for each, performed a grid search over 200 logarithmically spaced k values from 10⁻³ to 10⁻¹ and 200 linearly spaced L values from 0.05 to 10. This process was repeated for 20 different constraint strengths $λ_{r}$ ranging linearly from 0.5 to 10. Since the mapping from stimulus to response is deterministic, $H (r ∣ s) = 0$ and we need only consider the response entropy for the mutual information computation, $I (s; r) = H (r) - H (r ∣ s) = H (r)$ . Discretizing the stimulus and response values with a fixed binning, we directly computed the probability for each stimulus bin using the Gaussian probability density function. Using the logistic function, the stimulus bins were mapped to response bins, and thus the probability for each response bin was computed as a sum of probabilities of the stimulus bins that were mapped to that response bin. We used 10⁵ stimulus bins ranging from −3 × 10³ to 3 × 10³ and used 10⁵ response bins ranging from 0 to 10. The discrete entropy of the response was calculated as $H (r) = - \sum_{i}^{n} p (r_{i}) {log}_{2} [p (r_{i})]$ , where $r_{i}$ denotes the midpoint value of the ith bin of the response values. The mean firing rate was calculated directly from the probability distribution of the response bins and the midpoint value of each bin: $〈 r 〉 = \sum_{i}^{n} r_{i} p (r_{i})$ . The optimal nonlinearities for each filter were selected by choosing the $k^{*}$ and $L^{*}$ optimized for the SD $σ$ of that filter’s output.

Temporal statistics of filter outputs

To understand how the temporal statistics change during locomotion, we computed the Fourier power spectrum of Gabor filter responses. We did so for every filter in each video in short time windows of 5 s. Before computing the spectrum in each window, we subtracted the mean of that window. The windows overlapped by 2.5 s. The spectra were then averaged across time windows and videos for each filter for the stationary and moving conditions separately.

Temporal filtering

We modeled the temporal filtering performed by a model neuron as a low-pass filter with a linear roll-off in the frequency domain. The filter was thus defined as a piecewise linear function

h (f; f_{threshold}) = \{\begin{matrix} 1, & if f < f_{threshold} \\ max [0, 1 - (f - f_{threshold})], & if f \geq f_{threshold} \end{matrix}

where $f_{threshold}$ denotes the frequency threshold beyond which higher frequencies are attenuated. We computed the average spectra of the outputs of individual Gabor filters (stimuli), as well as the spectra averaged across the entire Gabor filterbank, and normalized the stationary and moving spectra by the same constant such that the total power of the stationary spectra was equal to one. The spectra of stimuli were then multiplied by the filter $h (f; f_{threshold})$ to generate output spectra. We considered 1000 frequency thresholds ranging linearly from 0 to 15 Hz, which was equal to the Nyquist limit. The total power of the filtered spectra were then computed for each value of $f_{threshold}$ . Using the total power of the stimulus in the stationary condition as a reference, we allowed for the model neuron to increase the amount of spectral power transmitted during locomotion: ${Δpower}_{} = \frac{P_{moving} - P_{stationary}}{P_{stationary}}$ , where $P_{moving}$ and $P_{stationary}$ are the total stimulus power in moving and stationary conditions, respectively. We considered 300 different values of $Δ power$ ranging linearly from 0.01 to 3. For each value of $Δ power$ , we identified the corresponding frequency threshold $f_{threshold}$ . The frequency thresholds computed for stationary and moving conditions were denoted $f_{s}$ and $f_{m}$ , respectively. Using the Wiener-Khinchin theorem, we computed the autocorrelation function of the output of temporal filters in Fig. 3D to a broad-band white noise input.

Optimization of inhibitory connections

To determine a population of model receptive fields, we performed ICA using the FastICA algorithm (63) on 32 by 32 px image patches from videos down-sampled by a factor of 8. For each video, 5000 patches were chosen from random positions in random frames, resulting in 500,000 image patches. Each image patch was normalized to have zero mean and unit variance. We learned 101 independent components (ICs). Thus, for each reference neuron, the neighboring population had 100 model cells. The learned ICs have spatial frequencies ranging from 0.08 to 0.48 cpd (as determined by numerically fitting Gabor filters). The IC filters were then applied to each video in the same manner as the parametric Gabor filters described above. We then randomly selected 1000 windows of length = 5 s from across all of the videos for each filter. To simulate neuronal activity $r_{i}$ of each model neuron i, the output of each IC filter was demeaned and rectified to enforce positivity. Within each stimulus window, we computed the correlation coefficients $ρ_{ij}$ between all pairs of simulated neuron responses $r_{i}, r_{j}$ in the stationary and moving conditions. To remove correlations between filter pairs, we considered a simple model of a network interaction where the simulated neural responses $r_{i}$ of all neurons in the population are weighted and subtracted from a reference neuron $r_{j}$ to minimize correlation, i.e., $r_{j}^{decorr .} = r_{j} - \sum_{i \neq j}^{n} w_{i} r_{i}$ , where $r_{j}^{decorr .}$ is the response of the neuron $r_{j}$ after the decorrelation transform. We optimized the weights $\vec{w}$ independently for each window and reference neuron such that the average correlation strength was minimized while keeping the weights small: $L (\vec{w}; λ) = \frac{1}{n - 1} \sum_{i \neq j}^{n} |ρ_{ij}^{decorr .}| + λ_{w} {∥\vec{w}∥}_{2}$ . Here, $ρ_{ij}^{decorr .}$ is the Pearson correlation between the postdecorrelation neural response of the reference neuron $r_{j}^{decorr}$ and population neighbors $r_{i}$ . $λ_{w} = 0.1$ is a weak regularizer of the weight magnitude. We used a gradient-based optimization to find the weights which minimized the loss function.

Comparisons to published data

The theoretical predictions for the mouse visual cortex use filters which match the range of spatial frequencies of receptive fields in mouse V1 reported in (33). Unless stated otherwise, the spatial frequencies of the filters range linearly from 0.02 to 0.32 cpd with an increment of 0.01 cpd resulting in 8928 filters. For all results, the constraint strength $λ_{r}$ on the mean responses was set to 1, and the optimal nonlinearity parameters were chosen on the basis of SDs computed from windows with a size of 5 s.

Contrast tuning curves

We simulated the response of optimized model neurons to scaled Gabor patches, identical to the receptive field of the analyzed neuron (i.e., optimal stimuli). The scaling parameter emulated changes in the stimulus contrast. We varied the scaling parameter from 5 to 500 logarithmically and computed the contrast tuning curves using optimal nonlinearity parameters in stationary and moving conditions. For each filter, the contrast tuning curve was normalized by the maximal value between the stationary and moving conditions. The scaling parameter was also normalized by the maximal value and converted to a percentage.

Orientation tuning curves

To generate orientation tuning curves, each model neuron was exposed to Gabor stimuli identical to the receptive field of the analyzed neuron. Stimuli were presented at 16 different orientations θ linearly spanning 0 to 337.5 deg. Each stimulus was also scaled by a fixed contrast parameter to avoid saturating the nonlinearity while still evoking a response. The filter outputs to the different orientations were then passed through the optimal nonlinearity for each filter to generate tuning curves. The additive and multiplicative gain terms were fit by regressing the moving tuning curve from the stationary tuning curve as in (8). A contrast parameter value of 30 was used in the example tuning curve shown in Fig. 2F. For the additive and multiplicative gain terms, a contrast parameter value of 10 was used. We considered nonlinearities optimized for 5472 Gabor filters with spatial frequencies ranging linearly from 0.02 to 0.2 cpd.

Dependence of locomotion modulation on movement speed

Videos were recorded in three different environments at three different average speeds (20, 30, and 50 cm/s). We recorded two 60-s videos for each speed and environment resulting in the total of 360 s per speed. We computed the output of the Gabor filterbank, and the optimal nonlinearities of model neurons were found using the method described previously. The optimal nonlinearity parameters for the stationary videos were used for the zero speed condition. The response of optimal nonlinearities at each speed were computed to 10⁴ samples of Laplace-distributed stimuli with μ = 0 and a scale of b = 10. The response for each filter was then averaged across stimulus presentations. The mean response for each model neurons as a function of movement speed was then fit using an exponential function $r (v) = a - {be}^{cv}$ . To limit the computational cost, we only used filters with frequencies ranging from 0.01 to 0.12 resulting in 3456 filters.

Population coding fidelity

To simulate neural responses, we used nonlinearities optimized for a population of low spatial frequency Gabor filters of various orientations, phases, and positions approximately matching the distribution of spatial frequencies reported in (13) (spatial wavelength ranged from 0.01 to 0.12 resulting in 3456 filters). We computed the response of this population to 2 × 10⁵ samples of stimuli sampled from the Laplace distribution with a mean of μ = 0 and a scale of b = 50, using nonlinearities optimized for the moving and stationary states. We then split the data in half and trained a linear decoder on the first half of the data to predict the stimuli from the population response vector. Using the held out data, we split the stimuli into nonoverlapping blocks of 1000 samples and, in each block, computed the average response of the population and root mean squared error of the decoder. To simulate a switch from stationary to locomotion, the first half of the stimulus blocks simulated for the stationary condition were concatenated with the second half of the stimulus blocks for the moving condition. The resulting trajectories of average firing rate and decoding error were then smoothed with a Gaussian filter with σ = 1 windows.

Temporal dynamics

The data from (6) were reanalyzed by computing the autocorrelation of the normalized response of individual mouse V1 neurons to moving dot stimuli in stationary and locomotion conditions. The autocorrelations were computed on the average response of each neuron to each stimulus speed thus generating the number of response profiles equal to the number of neurons times the number of stimulus speeds. For the theoretical results, we allowed a 50% increase in power during locomotion compared the total power during the stationary condition and extracted the optimal $f_{threshold}$ for this constraint. Applying the Wiener-Khinchin theorem, we then computed the autocorrelation by taking the inverse Fourier transform of the power spectra of temporal filters responding to white noise stimuli, as described above. For each analyzed autocorrelation function, we computed the decay time, defined as the lag time where the autocorrelation decreases to or below 0.5. We performed a paired t test on the decay times of individual neurons in the stationary and locomotion conditions to determine whether there was a significant change in the temporal processing during movement. The same analysis was performed for simulated neurons.

Surround suppression

We first computed the response of each independent model neuron (i.e., an IC filter followed by rectification) to stimuli of increasing size. The stimuli were full-field gratings with the orientation, phase, and spatial frequency matched to the IC filters by fitting parametric Gabor filters. We presented grating stimuli of different sizes, determined by the radius of a circular mask. We tested 40 radii ranging from 0 to 16 deg. To simulate the surround suppression, we used the weights optimized to minimize correlation magnitude in the stationary or moving conditions, and for each model neuron, we subtracted the weighted response of the population for the stimuli of increasing size. To compare to individual neurons, we used weight vectors $\vec{w}$ optimized to example stimulus windows. To compare to the population average, we averaged across all stimulus windows. We computed a suppression index in the same manner as in (14). Lastly, we computed the population response as a function of stimulus size by first normalizing the response of each reference filter by the maximal value across both locomotion conditions and then averaging across filters.

Gain modulation in mice and marmosets

To perform an interspecies comparison, we relied on distributions of spatial frequencies of receptive fields in mouse V1 and foveal receptive fields in the marmoset V1 reported in (33). For each of these distributions, we generated a population of 2000 neurons whose corresponding Gabor filters had spatial frequencies sampled accordingly. The orientation, phase, and position of each filter were independently sampled from a uniform distribution with limits described above. For each of these Gabor filters, we used the precomputed optimal nonlinearity parameters for the stationary and moving conditions. We simulated neural responses by exposing model nonlinearities to 1000 Laplace-distributed stimuli with μ = 0 and b = 50. The same process was repeated for the comparison to peripheral V1 neurons in the marmoset where the distribution of spatial frequencies in the far periphery was based on the data from (32).

We quantified the change in the shared, multiplicative gain in populations of model neurons at the transition to locomotion in the following way. We first simulated responses to the identical set of random stimuli described above using nonlinearities optimized for locomotion and the stationary state. This resulted in two matrices of simulated neural activity $X^{m}$ and $X^{s}$ where an entry $X_{n, t}^{m}$ or $X_{n, t}^{s}$ corresponded to the response of the nth neuron to the tth stimulus during movement or in the stationary state, respectively. To compute the change in shared multiplicative gain in both states, we performed linear regression of the form $X^{m} = a^{m} X^{s}$ , where $a^{m}$ is a scalar multiplicative factor that affects the activity of all neurons when the behavioral state changes. By definition, the value of the multiplicative factor in the stationary state is equal to 1, i.e., $a^{s} = 1$ . We performed this analysis for populations of nonlinearities optimized to mouse-like, low-frequency, and primate fovea–like, high-frequency Gabor filters. In both cases, the linear fit was reasonable with the coefficient of the goodness of fit $R^{2}$ equal to 0.794 and 0.997, respectively, which implies that this simple model emulates well a shared gain change in the population of simulated neurons. We report differences in of the gain terms $a^{m}$ and a^s = 1 in Fig. 5E.

Analysis of data from freely moving mice

We reanalyzed the publicly accessible data from (15) to determine how locomotion affects the stimulus statistics experienced by freely moving mice. Using videos from the head-mounted “world-view” camera, which records the visual scene from the animal’s point of view, we applied a Gabor filter bank with parameters identical to those used in the analysis of the natural videos (see above). Since the video had a different resolution (40 by 30 px) and spanned a larger FOV of about 120°, a different scaling factor of $\frac{1}{3}$ px per degree of visual angle was used. The running speed data are occasionally missing values, so we only considered windows with complete data. The average running speed in each 5-s window was then computed, and the data were split on the basis of the first quartile of the average running speed into stationary (<0.42 cm/s) and moving (>0.42 cm/s) conditions. The SD of filter outputs in each window was then computed, and for each locomotion condition, we evaluated the average SD. The Fourier transform of the filter outputs was determined for each window, and we then took the average over all windows and filters. We then considered correlations between the responses of IC filters in 5-s windows. Because of the low resolution of the video data, we performed ICA again on 16 by 16 px image patches from the natural videos down-sampled by a factor of 20 using a similar procedure as above. Since the dimensionality was initially reduced with principal components analysis to retain 75% of the variance, only 48 ICs were learned. The learned IC filters have spatial frequencies ranging from approximately 0.01 to 0.33 cpd (as determined by numerically fitting Gabor filters). As before, the output of each IC filter was demeaned and rectified to emulate neural responses. The correlation between all pairs of IC responses in each window was then computed, and the average correlation magnitude was computed for each locomotion condition.

Minimal model of sensing during locomotion

We constructed a minimal model to explain the changes in signal statistics during locomotion. We modeled a point-agent that observes stimuli while moving in a spatially heterogeneous two-dimensional environment. The environments were created by convolving a two-dimensional Gaussian filter with independent identically distributed (i.i.d.) white noise in a 100 by 100 grid. The spatial extent of the Gaussian filter $σ_{spatial}$ determines how spatially correlated or smooth the resulting environment is. The output of the convolution was then rescaled to have a maximal value of 100 and a minimal value of −100. At each time step of the simulation, the agent observed a stimulus which was normally distributed with the mean specified by the grid position occupied by the agents sensor (see below) and with the SD σ = 0.1. The position of the agent ${\vec{x}}_{t}$ was governed by the following equation: ${\vec{x}}_{t + 1} = {\vec{x}}_{t} + α {\vec{v}}_{t} + (1 - α) v_{r} \vec{n}$ , where $α$ is a momentum term, $v_{t}$ is the current velocity of the agent, and $v_{r}$ is the translational speed that controls the variance of the stochastic term $\vec{n} \sim N (\vec{0}, 1)$ sampled from a two-dimensional multivariate normal distribution (64). We set α = 0.5. The starting position $x_{0}$ was randomly drawn from a uniform distribution over grid positions, and the initial velocity $v_{0}$ was assigned unit norm with a direction drawn uniformly over all angles. We simulated different sensors (corresponding to, e.g., overlapping receptive fields), by allowing the agent to sample from eight spatial positions equally distributed in a circle of radius r from its current position. For each combination of the speed and spatial scale, we simulated 10 different trajectories of length 1500 time steps for 10 different environments. For the simulated results in Figs. 1 to 4, $σ_{spatial} = 5$ was used, and for the stationary condition, v_r = 0.2 in comparison to the moving condition where v_r = 5. For Fig. 5, we considered spatial scales of $σ_{spatial} = 2$ and $σ_{spatial} = 0.5$ and adjusted the moving velocity to v_r = 1. For all simulated results, we used a radius of r = 10 for the sensors. For the inset in Fig. 2F, we used five speed values $v_{r} \in {0.10.2,1,2,5}$ .

Acknowledgments

We thank L. Busse, A. Hermundstad, S. Katzner, and R. Portugues for helpful discussions and suggestions.

Funding:

We acknowledge that we received no funding in support of this research.

Author contributions:

Conceptualization: J.M.G. and W.F.M. Methodology: J.M.G. and W.F.M. Software: J.M.G. Validation: J.M.G. and W.F.M. Formal analysis: J.M.G. and W.F.M. Investigation: J.M.G. and W.F.M. Resources: W.F.M. Data curation: J.M.G. and W.F.M. Writing—original draft: J.M.G. and W.F.M. Writing—review and editing: J.M.G. and W.F.M. Visualization: J.M.G. and W.F.M. Supervision: W.F.M. Project administration: W.F.M. Funding acquisition: W.F.M.

Competing interests:

The authors declare that they have no competing interests.

Data, code, and materials availability:

All data and code needed to evaluate and reproduce the results in the paper are present in the paper and/or the Supplementary Materials. The data (natural videos) used in the analysis are available at https://doi.org/10.12751/g-node.14m4gq. The code is publicly available at https://doi.org/10.5281/zenodo.20624372. No new materials were generated in this study.

Supplementary Materials

This PDF file includes:

Figs. S1 to S6

sciadv.aed4172_sm.pdf^{(402.1KB, pdf)}

REFERENCES

1.Parker P. R., Brown M. A., Smear M. C., Niell C. M., Movement-related signals in sensory areas: Roles in natural behavior. Trends Neurosci. 43, 581–595 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]
2.Maimon G., Straw A. D., Dickinson M. H., Active flight increases the gain of visual motion processing in Drosophila. Nat. Neurosci. 13, 393–399 (2010). [DOI] [PubMed] [Google Scholar]
3.Chiappe M. E., Seelig J. D., Reiser M. B., Jayaraman V., Walking modulates speed sensitivity in Drosophila motion vision. Curr. Biol. 20, 1470–1475 (2010). [DOI] [PMC free article] [PubMed] [Google Scholar]
4.Niell C. M., Stryker M. P., Modulation of visual responses by behavioral state in mouse visual cortex. Neuron 65, 472–479 (2010). [DOI] [PMC free article] [PubMed] [Google Scholar]
5.Erisken S., Vaiceliunaite A., Jurjut O., Fiorini M., Katzner S., Busse L., Effects of locomotion extend throughout the mouse early visual system. Curr. Biol. 24, 2899–2907 (2014). [DOI] [PubMed] [Google Scholar]
6.Horrocks E. A., Rodrigues F. R., Saleem A. B., Flexible neural population dynamics govern the speed and stability of sensory encoding in mouse visual cortex. Nat. Commun. 15, 6415 (2024). [DOI] [PMC free article] [PubMed] [Google Scholar]
7.Saleem A. B., Ayaz A., Jeffery K. J., Harris K. D., Carandini M., Integration of visual motion and locomotion in mouse visual cortex. Nat. Neurosci. 16, 1864–1869 (2013). [DOI] [PMC free article] [PubMed] [Google Scholar]
8.Dadarlat M. C., Stryker M. P., Locomotion enhances neural encoding of visual stimuli in mouse V1. J. Neurosci. 37, 3764–3775 (2017). [DOI] [PMC free article] [PubMed] [Google Scholar]
9.Liska J. P., Rowley D. P., Nguyen T. T. K., Muthmann J.-O., Butts D. A., Yates J., Huk A. C., Running modulates primate and rodent visual cortex differently. Elife 12, RP87736 (2024). [DOI] [PMC free article] [PubMed] [Google Scholar]
10.Barlow H. B., Possible principles underlying the transformation of sensory messages. Sens. Commun. 1, 217–233 (1961). [Google Scholar]
11.van Hateren J. H., A theory of maximizing sensory information. Biol. Cybern. 68, 23–29 (1992). [DOI] [PubMed] [Google Scholar]
12.Chalk M., Marre O., Tkacˇik G., Toward a unified theory of efficient, predictive, and sparse coding. Proc. Natl. Acad. Sci. U.S.A. 115, 186–191 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
13.Mineault P. J., Tring E., Trachtenberg J. T., Ringach D. L., Enhanced spatial resolution during locomotion and heightened attention in mouse primary visual cortex. J. Neurosci. 36, 6382–6392 (2016). [DOI] [PMC free article] [PubMed] [Google Scholar]
14.Ayaz A., Saleem A. B., Schölvinck M. L., Carandini M., Locomotion controls spatial integration in mouse visual cortex. Curr. Biol. 23, 890–894 (2013). [DOI] [PMC free article] [PubMed] [Google Scholar]
15.Parker P. R., Abe E. T., Leonard E. S., Martins D. M., Niell C. M., Joint coding of visual input and eye/head position in V1 of freely moving mice. Neuron 110, 3897–3906.e5 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]
16.Niell C. M., Stryker M. P., Highly selective receptive fields in mouse visual cortex. J. Neurosci. 28, 7520–7536 (2008). [DOI] [PMC free article] [PubMed] [Google Scholar]
17.Qiu Y., Zhao Z., Klindt D., Kautzky M., Szatko K. P., Schaeffel F., Rifai K., Franke K., Busse L., Euler T., Natural environment statistics in the upper and lower visual field are reflected in mouse retinal specializations. Curr. Biol. 31, 3233–3247.e6 (2021). [DOI] [PubMed] [Google Scholar]
18.Abballe L., Asari H., Natural image statistics for mouse vision. PLOS ONE 17, e0262763 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]
19.Fairhall A. L., Lewen G. D., Bialek W., de Ruyter van Steveninck R. R., Efficiency and ambiguity in an adaptive neural code. Nature 412, 787–792 (2001). [DOI] [PubMed] [Google Scholar]
20.Brenner N., Bialek W., Ruyter van Steveninck R. R., Adaptive rescaling maximizes information transmission. Neuron 26, 695–702 (2000). [DOI] [PubMed] [Google Scholar]
21.Dean I., Harper N. S., McAlpine D., Neural population coding of sound level adapts to stimulus statistics. Nat. Neurosci. 8, 1684–1689 (2005). [DOI] [PubMed] [Google Scholar]
22.Młynarski W. F., Hermundstad A. M., Efficient and adaptive sensory codes. Nat. Neurosci. 24, 998–1009 (2021). [DOI] [PubMed] [Google Scholar]
23.Borst A., Theunissen F. E., Information theory and neural coding. Nat. Neurosci. 2, 947–957 (1999). [DOI] [PubMed] [Google Scholar]
24.Bell A. J., Sejnowski T. J., The “independent components” of natural scenes are edge filters. Vis. Res. 37, 3327–3338 (1997). [DOI] [PMC free article] [PubMed] [Google Scholar]
25.Olshausen B. A., Field D. J., Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609 (1996). [DOI] [PubMed] [Google Scholar]
26.Tkacˇik G., Prentice J. S., Balasubramanian V., Schneidman E., Optimal population coding by noisy spiking neurons. Proc. Natl. Acad. Sci. U.S.A. 107, 14419–14424 (2010). [DOI] [PMC free article] [PubMed] [Google Scholar]
27.Karklin Y., Lewicki M. S., Emergence of complex cell properties by learning to generalize in natural scenes. Nature 457, 83–86 (2009). [DOI] [PubMed] [Google Scholar]
28.Vinje W. E., Gallant J. L., Sparse coding and decorrelation in primary visual cortex during natural vision. Science 287, 1273–1276 (2000). [DOI] [PubMed] [Google Scholar]
29.Schwartz O., Simoncelli E. P., Natural signal statistics and sensory gain control. Nat. Neurosci. 4, 819–825 (2001). [DOI] [PubMed] [Google Scholar]
30.King P. D., Zylberberg J., DeWeese M. R., Inhibitory interneurons decorrelate excitatory cells to drive sparse code formation in a spiking model of V1. J. Neurosci. 33, 5475–5485 (2013). [DOI] [PMC free article] [PubMed] [Google Scholar]
31.Duong L., Simoncelli E., Chklovskii D., Lipshutz D., Adaptive whitening with fast gain modulation and slow synaptic plasticity. Adv. Neural Inf. Process. Syst. 36, 12518–12535 (2023). [Google Scholar]
32.Yu H.-H., Verma R., Yang Y., Tibballs H. A., Lui L. L., Reser D. H., Rosa M. G., Spatial and temporal frequency tuning in striate cortex: Functional uniformity and specializations related to receptive field eccentricity. Eur. J. Neurosci. 31, 1043–1062 (2010). [DOI] [PubMed] [Google Scholar]
33.Parker P. R., Martins D. M., Leonard E. S., Casey N. M., Sharp S. L., Abe E. T., Smear M. C., Yates J. L., Mitchell J. F., Niell C. M., A dynamic sequence of visual processing initiated by gaze shifts. Nat. Neurosci. 26, 2192–2202 (2023). [DOI] [PMC free article] [PubMed] [Google Scholar]
34.D. P. Rowley, A. C. Huk, J. L. Yates, Activity in the peripheral representation within primate V1 is substantially modulated during running. bioRxiv 617723 [Preprint] (2024). 10.1101/2024.10.10.617723. [DOI]
35.Simoncelli E. P., Olshausen B. A., Natural image statistics and neural representation. Annu. Rev. Neurosci. 24, 1193–1216 (2001). [DOI] [PubMed] [Google Scholar]
36.Srinivasan M. V., Laughlin S. B., Dubs A., Predictive coding: A fresh view of inhibition in the retina. Proc. R. Soc. Lond. B. Biol. Sci. 216, 427–459 (1982). [DOI] [PubMed] [Google Scholar]
37.Gupta D., Młynarski W., Sumser A., Symonova O., Svatoň J., Joesch M., Panoramic visual statistics shape retina-wide organization of receptive fields. Nat. Neurosci. 26, 606–614 (2023). [DOI] [PMC free article] [PubMed] [Google Scholar]
38.Lesica N. A., Jin J., Weng C., Yeh C.-I., Butts D. A., Stanley G. B., Alonso J.-M., Adaptation to stimulus contrast and correlations during natural visual stimulation. Neuron 55, 479–491 (2007). [DOI] [PMC free article] [PubMed] [Google Scholar]
39.DeWeese M., Zador A., Asymmetric dynamics in optimal variance adaptation. Neural Comput. 10, 1179–1202 (1998). [Google Scholar]
40.Wark B., Fairhall A., Rieke F., Timescales of inference in visual adaptation. Neuron 61, 750–761 (2009). [DOI] [PMC free article] [PubMed] [Google Scholar]
41.Młynarski W. F., Hermundstad A. M., Adaptive coding for dynamic sensory inference. eLife 7, e32055 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
42.Franke K., Willeke K. F., Ponder K., Galdamez M., Zhou N., Muhammad T., Patel S., Froudarakis E., Reimer J., Sinz F. H., Tolias A. S., State-dependent pupil dilation rapidly shifts visual feature selectivity. Nature 610, 128–134 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]
43.Joesch M., Meister M., A neuronal circuit for colour vision based on rod–cone opponency. Nature 532, 236–239 (2016). [DOI] [PubMed] [Google Scholar]
44.Baden T., Schubert T., Chang L., Wei T., Zaichuk M., Wissinger B., Euler T., A tale of two retinal domains: Near-optimal sampling of achromatic contrasts in natural scenes through asymmetric photoreceptor distribution. Neuron 80, 1206–1217 (2013). [DOI] [PubMed] [Google Scholar]
45.Szatko K. P., Korympidou M. M., Ran Y., Berens P., Dalkara D., Schubert T., Euler T., Franke K., Neural circuits in the mouse retina support color vision in the upper visual field. Nat. Commun. 11, 3481 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]
46.Höfling L., Szatko K. P., Behrens C., Deng Y., Qiu Y., Klindt D. A., Jessen Z., Schwartz G. W., Bethge M., Berens P., Franke K., Ecker A. S., Euler T., A chromatic feature detector in the retina signals visual context changes. eLife 13, e86860 (2024). [DOI] [PMC free article] [PubMed] [Google Scholar]
47.Kang I., Talluri B. C., Yates J. L., Niell C. M., Nienborg H., Is the impact of spontaneous movements on early visual cortex species specific? Trends Neurosci. 48, 7–21 (2025). [DOI] [PMC free article] [PubMed] [Google Scholar]
48.Orlowska-Feuer P., Ebrahimi A. S., Zippo A. G., Petersen R. S., Lucas R. J., Storchi R., Look-up and look-down neurons in the mouse visual thalamus during freely moving exploration. Curr. Biol. 32, 3987–3999.e4 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]
49.Christensen A. J., Pillow J. W., Reduced neural activity but improved coding in rodent higher-order visual cortex during locomotion. Nat. Commun. 13, 1676 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]
50.Mayer J. M., Młynarski W. F., Subunit-specific behavioral modulation of sensory tuning in the visual cortex. PLOS Comput. Biol. 22, e1014123 (2026). [DOI] [PMC free article] [PubMed] [Google Scholar]
51.Busse L., The influence of locomotion on sensory processing and its underlying neuronal circuits. e-Neuroforum 24, A41–A51 (2018). [Google Scholar]
52.Aydın C., Couto J., Giugliano M., Farrow K., Bonin V., Locomotion modulates specific functional cell types in the mouse visual thalamus. Nat. Commun. 9, 4882 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
53.Dipoppa M., Ranson A., Krumin M., Pachitariu M., Carandini M., Harris K. D., Vision and locomotion shape the interactions between neuron types in mouse visual cortex. Neuron 98, 602–615.e8 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
54.Samonds J. M., Geisler W. S., Priebe N. J., Natural image and receptive field statistics predict saccade sizes. Nat. Neurosci. 21, 1591–1599 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
55.Miura S. K., Scanziani M., Distinguishing externally from saccade-induced motion in visual cortex. Nature 610, 135–142 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]
56.Stringer C., Pachitariu M., Steinmetz N., Reddy C. B., Carandini M., Harris K. D., Spontaneous behaviors drive multidimensional, brainwide activity. Science 364, 255 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]
57.Musall S., Kaufman M. T., Juavinett A. L., Gluf S., Churchland A. K., Single-trial neural dynamics are dominated by richly varied movements. Nat. Neurosci. 22, 1677–1686 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]
58.Talluri B. C., Kang I., Lazere A., Quinn K. R., Kaliss N., Yates J. L., Butts D. A., Nienborg H., Activity in primate visual cortex is minimally driven by spontaneous movements. Nat. Neurosci. 26, 1953–1959 (2023). [DOI] [PMC free article] [PubMed] [Google Scholar]
59.Keller G. B., Bonhoeffer T., Hübener M., Sensorimotor mismatch signals in primary visual cortex of the behaving mouse. Neuron 74, 809–815 (2012). [DOI] [PubMed] [Google Scholar]
60.Homann J., Koay S. A., Chen K. S., Tank D. W., Berry M. J., Novel stimuli evoke excess activity in the mouse primary visual cortex. Proc. Natl. Acad. Sci. U.S.A. 119, e2108882119 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]
61.Echeveste R., Aitchison L., Hennequin G., Lengyel M., Cortical-like dynamics in recurrent circuits optimized for sampling-based probabilistic inference. Nat. Neurosci. 23, 1138–1149 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]
62.Harper N. S., McAlpine D., Optimal neural population coding of an auditory spatial cue. Nature 430, 682–686 (2004). [DOI] [PubMed] [Google Scholar]
63.A. Hyvärinen, J. Hurri, P. O. Hoyer, Natural Image Statistics: A Probabilistic Approach to Early Computational Vision (Springer Science & Business Media, 2009), vol. 39.
64.Franzius M., Sprekeler H., Wiskott L., Slowness and sparseness lead to place, head-direction, and spatial-view cells. PLOS Comput. Biol. 3, e166 (2007). [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Figs. S1 to S6

sciadv.aed4172_sm.pdf^{(402.1KB, pdf)}

Data Availability Statement

[R1] 1.Parker P. R., Brown M. A., Smear M. C., Niell C. M., Movement-related signals in sensory areas: Roles in natural behavior. Trends Neurosci. 43, 581–595 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]

[R2] 2.Maimon G., Straw A. D., Dickinson M. H., Active flight increases the gain of visual motion processing in Drosophila. Nat. Neurosci. 13, 393–399 (2010). [DOI] [PubMed] [Google Scholar]

[R3] 3.Chiappe M. E., Seelig J. D., Reiser M. B., Jayaraman V., Walking modulates speed sensitivity in Drosophila motion vision. Curr. Biol. 20, 1470–1475 (2010). [DOI] [PMC free article] [PubMed] [Google Scholar]

[R4] 4.Niell C. M., Stryker M. P., Modulation of visual responses by behavioral state in mouse visual cortex. Neuron 65, 472–479 (2010). [DOI] [PMC free article] [PubMed] [Google Scholar]

[R5] 5.Erisken S., Vaiceliunaite A., Jurjut O., Fiorini M., Katzner S., Busse L., Effects of locomotion extend throughout the mouse early visual system. Curr. Biol. 24, 2899–2907 (2014). [DOI] [PubMed] [Google Scholar]

[R6] 6.Horrocks E. A., Rodrigues F. R., Saleem A. B., Flexible neural population dynamics govern the speed and stability of sensory encoding in mouse visual cortex. Nat. Commun. 15, 6415 (2024). [DOI] [PMC free article] [PubMed] [Google Scholar]

[R7] 7.Saleem A. B., Ayaz A., Jeffery K. J., Harris K. D., Carandini M., Integration of visual motion and locomotion in mouse visual cortex. Nat. Neurosci. 16, 1864–1869 (2013). [DOI] [PMC free article] [PubMed] [Google Scholar]

[R8] 8.Dadarlat M. C., Stryker M. P., Locomotion enhances neural encoding of visual stimuli in mouse V1. J. Neurosci. 37, 3764–3775 (2017). [DOI] [PMC free article] [PubMed] [Google Scholar]

[R9] 9.Liska J. P., Rowley D. P., Nguyen T. T. K., Muthmann J.-O., Butts D. A., Yates J., Huk A. C., Running modulates primate and rodent visual cortex differently. Elife 12, RP87736 (2024). [DOI] [PMC free article] [PubMed] [Google Scholar]

[R10] 10.Barlow H. B., Possible principles underlying the transformation of sensory messages. Sens. Commun. 1, 217–233 (1961). [Google Scholar]

[R11] 11.van Hateren J. H., A theory of maximizing sensory information. Biol. Cybern. 68, 23–29 (1992). [DOI] [PubMed] [Google Scholar]

[R12] 12.Chalk M., Marre O., Tkacˇik G., Toward a unified theory of efficient, predictive, and sparse coding. Proc. Natl. Acad. Sci. U.S.A. 115, 186–191 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]

[R13] 13.Mineault P. J., Tring E., Trachtenberg J. T., Ringach D. L., Enhanced spatial resolution during locomotion and heightened attention in mouse primary visual cortex. J. Neurosci. 36, 6382–6392 (2016). [DOI] [PMC free article] [PubMed] [Google Scholar]

[R14] 14.Ayaz A., Saleem A. B., Schölvinck M. L., Carandini M., Locomotion controls spatial integration in mouse visual cortex. Curr. Biol. 23, 890–894 (2013). [DOI] [PMC free article] [PubMed] [Google Scholar]

[R15] 15.Parker P. R., Abe E. T., Leonard E. S., Martins D. M., Niell C. M., Joint coding of visual input and eye/head position in V1 of freely moving mice. Neuron 110, 3897–3906.e5 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]

[R16] 16.Niell C. M., Stryker M. P., Highly selective receptive fields in mouse visual cortex. J. Neurosci. 28, 7520–7536 (2008). [DOI] [PMC free article] [PubMed] [Google Scholar]

[R17] 17.Qiu Y., Zhao Z., Klindt D., Kautzky M., Szatko K. P., Schaeffel F., Rifai K., Franke K., Busse L., Euler T., Natural environment statistics in the upper and lower visual field are reflected in mouse retinal specializations. Curr. Biol. 31, 3233–3247.e6 (2021). [DOI] [PubMed] [Google Scholar]

[R18] 18.Abballe L., Asari H., Natural image statistics for mouse vision. PLOS ONE 17, e0262763 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]

[R19] 19.Fairhall A. L., Lewen G. D., Bialek W., de Ruyter van Steveninck R. R., Efficiency and ambiguity in an adaptive neural code. Nature 412, 787–792 (2001). [DOI] [PubMed] [Google Scholar]

[R20] 20.Brenner N., Bialek W., Ruyter van Steveninck R. R., Adaptive rescaling maximizes information transmission. Neuron 26, 695–702 (2000). [DOI] [PubMed] [Google Scholar]

[R21] 21.Dean I., Harper N. S., McAlpine D., Neural population coding of sound level adapts to stimulus statistics. Nat. Neurosci. 8, 1684–1689 (2005). [DOI] [PubMed] [Google Scholar]

[R22] 22.Młynarski W. F., Hermundstad A. M., Efficient and adaptive sensory codes. Nat. Neurosci. 24, 998–1009 (2021). [DOI] [PubMed] [Google Scholar]

[R23] 23.Borst A., Theunissen F. E., Information theory and neural coding. Nat. Neurosci. 2, 947–957 (1999). [DOI] [PubMed] [Google Scholar]

[R24] 24.Bell A. J., Sejnowski T. J., The “independent components” of natural scenes are edge filters. Vis. Res. 37, 3327–3338 (1997). [DOI] [PMC free article] [PubMed] [Google Scholar]

[R25] 25.Olshausen B. A., Field D. J., Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609 (1996). [DOI] [PubMed] [Google Scholar]

[R26] 26.Tkacˇik G., Prentice J. S., Balasubramanian V., Schneidman E., Optimal population coding by noisy spiking neurons. Proc. Natl. Acad. Sci. U.S.A. 107, 14419–14424 (2010). [DOI] [PMC free article] [PubMed] [Google Scholar]

[R27] 27.Karklin Y., Lewicki M. S., Emergence of complex cell properties by learning to generalize in natural scenes. Nature 457, 83–86 (2009). [DOI] [PubMed] [Google Scholar]

[R28] 28.Vinje W. E., Gallant J. L., Sparse coding and decorrelation in primary visual cortex during natural vision. Science 287, 1273–1276 (2000). [DOI] [PubMed] [Google Scholar]

[R29] 29.Schwartz O., Simoncelli E. P., Natural signal statistics and sensory gain control. Nat. Neurosci. 4, 819–825 (2001). [DOI] [PubMed] [Google Scholar]

[R30] 30.King P. D., Zylberberg J., DeWeese M. R., Inhibitory interneurons decorrelate excitatory cells to drive sparse code formation in a spiking model of V1. J. Neurosci. 33, 5475–5485 (2013). [DOI] [PMC free article] [PubMed] [Google Scholar]

[R31] 31.Duong L., Simoncelli E., Chklovskii D., Lipshutz D., Adaptive whitening with fast gain modulation and slow synaptic plasticity. Adv. Neural Inf. Process. Syst. 36, 12518–12535 (2023). [Google Scholar]

[R32] 32.Yu H.-H., Verma R., Yang Y., Tibballs H. A., Lui L. L., Reser D. H., Rosa M. G., Spatial and temporal frequency tuning in striate cortex: Functional uniformity and specializations related to receptive field eccentricity. Eur. J. Neurosci. 31, 1043–1062 (2010). [DOI] [PubMed] [Google Scholar]

[R33] 33.Parker P. R., Martins D. M., Leonard E. S., Casey N. M., Sharp S. L., Abe E. T., Smear M. C., Yates J. L., Mitchell J. F., Niell C. M., A dynamic sequence of visual processing initiated by gaze shifts. Nat. Neurosci. 26, 2192–2202 (2023). [DOI] [PMC free article] [PubMed] [Google Scholar]

[R34] 34.D. P. Rowley, A. C. Huk, J. L. Yates, Activity in the peripheral representation within primate V1 is substantially modulated during running. bioRxiv 617723 [Preprint] (2024). 10.1101/2024.10.10.617723. [DOI]

[R35] 35.Simoncelli E. P., Olshausen B. A., Natural image statistics and neural representation. Annu. Rev. Neurosci. 24, 1193–1216 (2001). [DOI] [PubMed] [Google Scholar]

[R36] 36.Srinivasan M. V., Laughlin S. B., Dubs A., Predictive coding: A fresh view of inhibition in the retina. Proc. R. Soc. Lond. B. Biol. Sci. 216, 427–459 (1982). [DOI] [PubMed] [Google Scholar]

[R37] 37.Gupta D., Młynarski W., Sumser A., Symonova O., Svatoň J., Joesch M., Panoramic visual statistics shape retina-wide organization of receptive fields. Nat. Neurosci. 26, 606–614 (2023). [DOI] [PMC free article] [PubMed] [Google Scholar]

[R38] 38.Lesica N. A., Jin J., Weng C., Yeh C.-I., Butts D. A., Stanley G. B., Alonso J.-M., Adaptation to stimulus contrast and correlations during natural visual stimulation. Neuron 55, 479–491 (2007). [DOI] [PMC free article] [PubMed] [Google Scholar]

[R39] 39.DeWeese M., Zador A., Asymmetric dynamics in optimal variance adaptation. Neural Comput. 10, 1179–1202 (1998). [Google Scholar]

[R40] 40.Wark B., Fairhall A., Rieke F., Timescales of inference in visual adaptation. Neuron 61, 750–761 (2009). [DOI] [PMC free article] [PubMed] [Google Scholar]

[R41] 41.Młynarski W. F., Hermundstad A. M., Adaptive coding for dynamic sensory inference. eLife 7, e32055 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]

[R42] 42.Franke K., Willeke K. F., Ponder K., Galdamez M., Zhou N., Muhammad T., Patel S., Froudarakis E., Reimer J., Sinz F. H., Tolias A. S., State-dependent pupil dilation rapidly shifts visual feature selectivity. Nature 610, 128–134 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]

[R43] 43.Joesch M., Meister M., A neuronal circuit for colour vision based on rod–cone opponency. Nature 532, 236–239 (2016). [DOI] [PubMed] [Google Scholar]

[R44] 44.Baden T., Schubert T., Chang L., Wei T., Zaichuk M., Wissinger B., Euler T., A tale of two retinal domains: Near-optimal sampling of achromatic contrasts in natural scenes through asymmetric photoreceptor distribution. Neuron 80, 1206–1217 (2013). [DOI] [PubMed] [Google Scholar]

[R45] 45.Szatko K. P., Korympidou M. M., Ran Y., Berens P., Dalkara D., Schubert T., Euler T., Franke K., Neural circuits in the mouse retina support color vision in the upper visual field. Nat. Commun. 11, 3481 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]

[R46] 46.Höfling L., Szatko K. P., Behrens C., Deng Y., Qiu Y., Klindt D. A., Jessen Z., Schwartz G. W., Bethge M., Berens P., Franke K., Ecker A. S., Euler T., A chromatic feature detector in the retina signals visual context changes. eLife 13, e86860 (2024). [DOI] [PMC free article] [PubMed] [Google Scholar]

[R47] 47.Kang I., Talluri B. C., Yates J. L., Niell C. M., Nienborg H., Is the impact of spontaneous movements on early visual cortex species specific? Trends Neurosci. 48, 7–21 (2025). [DOI] [PMC free article] [PubMed] [Google Scholar]

[R48] 48.Orlowska-Feuer P., Ebrahimi A. S., Zippo A. G., Petersen R. S., Lucas R. J., Storchi R., Look-up and look-down neurons in the mouse visual thalamus during freely moving exploration. Curr. Biol. 32, 3987–3999.e4 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]

[R49] 49.Christensen A. J., Pillow J. W., Reduced neural activity but improved coding in rodent higher-order visual cortex during locomotion. Nat. Commun. 13, 1676 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]

[R50] 50.Mayer J. M., Młynarski W. F., Subunit-specific behavioral modulation of sensory tuning in the visual cortex. PLOS Comput. Biol. 22, e1014123 (2026). [DOI] [PMC free article] [PubMed] [Google Scholar]

[R51] 51.Busse L., The influence of locomotion on sensory processing and its underlying neuronal circuits. e-Neuroforum 24, A41–A51 (2018). [Google Scholar]

[R52] 52.Aydın C., Couto J., Giugliano M., Farrow K., Bonin V., Locomotion modulates specific functional cell types in the mouse visual thalamus. Nat. Commun. 9, 4882 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]

[R53] 53.Dipoppa M., Ranson A., Krumin M., Pachitariu M., Carandini M., Harris K. D., Vision and locomotion shape the interactions between neuron types in mouse visual cortex. Neuron 98, 602–615.e8 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]

[R54] 54.Samonds J. M., Geisler W. S., Priebe N. J., Natural image and receptive field statistics predict saccade sizes. Nat. Neurosci. 21, 1591–1599 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]

[R55] 55.Miura S. K., Scanziani M., Distinguishing externally from saccade-induced motion in visual cortex. Nature 610, 135–142 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]

[R56] 56.Stringer C., Pachitariu M., Steinmetz N., Reddy C. B., Carandini M., Harris K. D., Spontaneous behaviors drive multidimensional, brainwide activity. Science 364, 255 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]

[R57] 57.Musall S., Kaufman M. T., Juavinett A. L., Gluf S., Churchland A. K., Single-trial neural dynamics are dominated by richly varied movements. Nat. Neurosci. 22, 1677–1686 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]

[R58] 58.Talluri B. C., Kang I., Lazere A., Quinn K. R., Kaliss N., Yates J. L., Butts D. A., Nienborg H., Activity in primate visual cortex is minimally driven by spontaneous movements. Nat. Neurosci. 26, 1953–1959 (2023). [DOI] [PMC free article] [PubMed] [Google Scholar]

[R59] 59.Keller G. B., Bonhoeffer T., Hübener M., Sensorimotor mismatch signals in primary visual cortex of the behaving mouse. Neuron 74, 809–815 (2012). [DOI] [PubMed] [Google Scholar]

[R60] 60.Homann J., Koay S. A., Chen K. S., Tank D. W., Berry M. J., Novel stimuli evoke excess activity in the mouse primary visual cortex. Proc. Natl. Acad. Sci. U.S.A. 119, e2108882119 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]

[R61] 61.Echeveste R., Aitchison L., Hennequin G., Lengyel M., Cortical-like dynamics in recurrent circuits optimized for sampling-based probabilistic inference. Nat. Neurosci. 23, 1138–1149 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]

[R62] 62.Harper N. S., McAlpine D., Optimal neural population coding of an auditory spatial cue. Nature 430, 682–686 (2004). [DOI] [PubMed] [Google Scholar]

[R63] 63.A. Hyvärinen, J. Hurri, P. O. Hoyer, Natural Image Statistics: A Probabilistic Approach to Early Computational Vision (Springer Science & Business Media, 2009), vol. 39.

[R64] 64.Franzius M., Sprekeler H., Wiskott L., Slowness and sparseness lead to place, head-direction, and spatial-view cells. PLOS Comput. Biol. 3, e166 (2007). [DOI] [PMC free article] [PubMed] [Google Scholar]

PERMALINK

Locomotion optimizes sensory representations through a computational principle shared by rodents and primates

Jonathan M Gant

Wiktor F Młynarski

Roles

Abstract

INTRODUCTION

RESULTS

Fig. 1. Internal modulation adjusts sensory neurons to changes in stimulus statistics caused by locomotion.

Gain modulation maintains accuracy of sensory coding during locomotion

Fig. 2. Locomotion affects stimulus variance and modulates the gain of sensory neurons.

Modulation of temporal filtering ensures efficiency of sensory coding during locomotion

Fig. 3. Locomotion affects stimulus frequency spectrum and modulates temporal filtering in sensory neurons.

Modulation of network interactions maintains a nonredundant sensory code during locomotion

Fig. 4. Locomotion affects filter correlations and modulates lateral inhibition in sensory populations.

Stimulus statistics explain differential modulation of sensory coding by locomotion in rodents and primates

Fig. 5. Locomotion differentially affects high- and low-spatial-frequency filters and modulates the gain of sensory neurons in different species.

DISCUSSION

Toward a theory of sensory coding in behaving animals

Extensions and future directions

Outlook

METHODS

Video recordings

Gabor filters

Variance of filter outputs

Nonlinearity optimization

Temporal statistics of filter outputs

Temporal filtering

Optimization of inhibitory connections

Comparisons to published data

Contrast tuning curves

Orientation tuning curves

Dependence of locomotion modulation on movement speed

Population coding fidelity

Temporal dynamics

Surround suppression

Gain modulation in mice and marmosets

Analysis of data from freely moving mice

Minimal model of sensing during locomotion

Acknowledgments

Funding:

Author contributions:

Competing interests:

Data, code, and materials availability:

Supplementary Materials

This PDF file includes:

REFERENCES

Associated Data

Supplementary Materials

Data Availability Statement

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases