Adaptive Optimization of Visual Sensitivity

Abstract

Sensory systems adapt to environmental change. It has been argued that adaptation should have the effect of optimizing sensitivity to the new environment. Here we consider a framework in which this premise is made concrete using an economic normative theory of visual motion perception. In this framework, visual systems adapt to the environment by reallocating their limited neural resources. The allocation is optimal when uncertainties about different aspects of stimulation are balanced. This theory makes predictions about visual sensitivity as a function of environmental statistics. Adaptive optimization of the visual system should be manifested as a change in sensitivity for an observer and for the underlying motion-sensitive neurons. We review evidence supporting these predictions and examine effects of adaptation on the neuronal representation of visual motion.

1 Introduction

One of the fundamental tenets of sensory biology is that sensory systems adapt to environmental change. It has been argued that adaptation should have the effect of optimizing sensitivity to the new environment. Attempt to corroborate this view in visual neuroscience led to controversy. This is because adaptation to a visual environment has been expected to improve visual performance in that environment, relative to performance before adaptation. This expectation has been contradicted by the fact that adaptation has been observed to have different effects: it can decrease visual performance for the adapting stimulus, or it can leave performance intact, or it can change performance for stimuli very different from the adapting ones.

We proposed that the previous results can be explained by taking an economic perspective on neural function. According to this view, visual adaptation is mediated by reallocation of limited neural resources over a broad range of visual stimulation, i.e., by changing the tuning of multiple neurons in the visual cortex. The reallocation is expected to cause gains and losses of sensitivity by neurons tuned to different stimuli. Using a normative theory of neural resource allocation, we predicted that sensitivity changes should form a lawful pattern of gains and losses of sensitivity.

We have tested these hypotheses behaviorally using new psychophysical methods that allowed us to assay sensitivity rapidly across a wide range of spatiotemporal stimulation. Statistics of optical stimulus was varied, such that different speeds were prevalent on different days of the experiment. We found that the change of stimulus statistics caused a large-scale reorganization of sensitivity: an orderly pattern similar to that predicted by the theory of adaptive reallocation of neural resources.

The economic theory predicts that the expected change of sensitivity for individual neurons depends on where their tuning falls on the behavioral sensitivity function. We consider the possibility that such changes can be implemented in multiple neurons by means of mutually independent stochastic adjustments of synaptic weights. This mechanism will require no coordination between the changes in neurons tuned to very different stimuli, similar to the self-organizing process of “swarm intelligence” found in many biological systems.

2 The Puzzle of Visual Adaptation

2.1 Adaptation

Visual adaptation is one of the most striking and well-studied of visual phenomena²⁰. Yet, the mechanisms underlying adaptation remain elusive and the evidence for it controversial. Until recently, adaptation was viewed as a manifestation of neural fatigue. The contemporary view is more pragmatic; it holds that adaptation is a response of the organism to changes in statistics of stimulation (e.g.,²⁶). Adaptation is taken to manifest an optimization of the organism’s perceptual abilities under changing stimulation. However, evidence supporting this view is scarce and inconsistent. For example, some studies of adaptation to moving patterns have shown that speed discrimination performance improves for speeds similar to the adapting speed, but other studies have reported the opposite. Even more surprising are systematic changes in discrimination for stimuli that differ from the adapting. The previous theoretical efforts have failed to provide a comprehensive explanation for these findings.

Inconsistency is also a property of data obtained in neurophysiological studies of speed adaptation. Consider, for example, the study of neurons in cortical visual area MT following a brief motion adaptation by Krekelberg et al.¹⁴ Their main result is summarized in Fig. 1. Adaptation was found to reduce firing rates and to alter speed discriminability. These effects were puzzling in two ways. First, the effect of adaptation on response rate reduction was often greatest when the adapting speed was different from the preferred speed of the cell. Second, only some cells showed improved speed discriminability, while discriminability by many other cells was impaired.

Figure 1.

Speed adaptation in area MT.¹⁴ Neuronal speed discriminability before (abscissa) and after (ordinate) adaptation. Asterisks indicate neurons for which adaptation was significant. Points above and below the diagonal represent neurons whose discriminability was either improved or impaired by adaptation.

In the following, we describe an attempt to solve the puzzle of motion adaptation from the perspective of neural economy. We consider one of the most studied forms of motion adaptation, quantified in terms of visual contrast sensitivity.

2.2 Contrast Sensitivity

There are many ways to quantify visual sensitivity (e.g.,^{21, 24, 25, 30}). Perhaps, the most comprehensive and systematic of these methods is the one developed by Kelly¹² illustrated in Fig. 2a. Sensitivity in this case reflects contrast threshold for pattern detection as a function of spatial and temporal frequencies of stimuli (sf,tf). We will describe a theoretical approach to understanding the forces that shape the distribution of visual sensitivity across stimuli (Fig. 2a). From this perspective, it follows that the distribution of visual sensitivity as well as diverse and seemingly inconsistent transformations of sensitivity induced by adaptation are manifestations of efficient visual behavior. That is, adaptive changes reflect the optimization of visual performance with limited resources. We will then show how this perspective entails specific predictions for the distribution of visual sensitivity, i.e., how visual sensitivity should change in response to changes in stimulation. The changes should form a characteristic pattern across stimuli, including the previously observed results as special cases. In particular, visual sensitivity should either increase or decrease at the prevailing speed of stimulation, depending upon the conditions of measurement. In addition, the changes in sensitivity should propagate, in a lawful manner, across the entire stimulus space, including stimuli very different from the adapting ones, as we explain next.

Figure 2.

Spatiotemporal sensitivity functions. a Plot of luminance contrast thresholds as a function of (sf,tf)¹². The heavy line represents maximal sensitivity at every speed, and the contours represent equal increments of log sensitivity. b Simulation of (sf,tf) sensitivity for speed discrimination, from principles developed in our theoretical approach (section Innovation). Our preliminary measurements, using intensive psychophysical methods, have yielded a sensitivity function similar to that obtained by Kelly, but with a greatly reduced data collection.

3 Visual Adaptation from a Normative Perspective

3.1 Neural Economics

We present the economical approach using the plot introduced in Fig. 2a. Different points in the plot represent tuning parameters of motion-sensitive neurons¹ at the peak of their sensitivity. The parallel lines are constant-speed lines (“speed lines”). Each such line comprises points at which the ratio of temporal frequency to spatial frequency (i.e., the speed) of stimulus is constant. Low speeds appear at bottom right of the plot and high speeds at top left.

Gepshtein et al.¹⁰ developed a normative theory of motion estimation that allowed them to derive optimal conditions for estimating every speed. Under these conditions, the uncertainties associated with measurements of different stimuli are balanced.¹ Each optimal condition is represented by a point on the corresponding speed line. Such points for all speeds collectively form an “optimal set,” represented in Fig. 3 by the red curve. According to this normative theory, the optimal set has the invariant form of a rectangular hyperbola.

Figure 3.

Optimal sets for speed estimation. Oblique lines correspond to different speeds (“speed lines”); they are parallel to one another in the logarithmic coordinates. Speed (v = tf/sf) grows from bottom right to top left of plot. Points predicted to be most suitable for the estimation of every speed form a set represented by red curve: the optimal set of speed estimation. According to Gepshtein et al.¹⁰ the optimal set has an invariant shape, approximated by a rectangular hyperbola. Position of optimal set in the plot depends on prevailing speed of stimulation: the “expected speed” (Fig. 4).

From the theory, it also follows that the location of the optimal set in the (sf,tf) parameter space depends on the statistics of stimulus speed (Fig. 4). For example, suppose that in the natural environment, the prevailing speed is low,³¹ indicated in Fig. 4a by the oblique red line. Then, suppose that the environment changed and the prevailing speed has increased (oblique green line). The conditions at which the system has greatest sensitivity (the optimal set) are predicted to change, which can be summarized as a shift (a translation) of the optimal set in the (sf,tf) parameters space, as follows. In Fig. 4a, the directions of displacement of optimal points along two speed lines are shown by arrows. For the two indicated speeds, the optimal points are predicted to shift in opposite directions: toward lower spatiotemporal frequencies at the low speed and toward higher frequencies at the high speed. Thus, whether sensitivity grows or decays at a given location depends on whether the optimal point moves toward or away from that location. There also exist conditions where sensitivity is expected to remain unchanged, e.g., where the new and old optimal sets intersect and the optimal point does not move.

Figure 4.

Predicted effects of speed adaptation. a Changes in statistics of stimulation lead to displacement of optimal set for speed estimation: from red curve (low-speed context) to green curve (high-speed context). b In effect, different sensitivity changes are expected across the parameter space. Numbered disks mark locations of qualitatively different consequences of adaptation.

This illustration makes it clear that an adaptation experiment can lead to qualitatively different outcomes depending on the conditions at which motion sensitivity is measured, as shown in Fig. 4b. Sensitivity may increase (points 1 and 6), decrease (2 and 5), or not change at all (3 and 4). Paradoxically, it is predicted that sensitivity changes that make the system behave optimally as a whole can lead to sensitivity losses at the currently prevailing speed (point 2). Notice also that optimization can lead to large sensitivity changes (gains and losses) far away from the prevailing conditions (points 5 and 6).

It is expected that the shift of the optimal set is accompanied by sensitivity changes over the entire parameter space, as shown in Fig. 5. Figure 5a, b shows theoretical distributions of sensitivity in, respectively, high-speed and low-speed environments. Figure 5c shows a map of sensitivity change: the shades of red and blue indicate stimulus conditions where sensitivity is expected to increase and decrease.

Figure 5.

Predicted effect of adaptation across the entire parameter space. Predicted distributions of sensitivity in stimulus environments with high (a) and low (b) prevailing speeds. In both panels, warm colors indicate high sensitivity and cool colors indicate low sensitivity (normalized for this figure). c Predicted change map. Entries in c are 100*(B−A)/A, where A and B are entries in sensitivity maps of a and b, respectively. Here, shades of red indicate gains of sensitivity and shades of blue indicate losses of sensitivity.

It is convenient to think of the predicted gains and losses of sensitivity as foci of change formed around branches of the optimal set. Recall that this set resembles a rectangular hyperbola (similar to the curve “max” in Fig. 2a). Shifts of this hyperbola create systematic changes across the (sf,tf) plot summarized in Fig. 5. This hypothesis is supported by the experiments described in Sect. 4.

3.2 The Theoretical Context

Theories of perception based on the statistical decision theory, including Bayesian theories,^{13, 19, 27} also predict that stimulus statistics affect perception. Here, probabilities of sensory estimates (likelihood functions) and the probabilities of corresponding parameters in the stimulation (prior distributions) are combined by point-by-point multiplication, following the Bayes’ rule, making the prevalent stimuli more likely to be seen than the less common stimuli.

To illustrate differences of our approach from the above framework, consider a study of motion adaptation by Stocker and Simoncelli²⁸. As it is common in the Bayesian framework, the authors represented effects of adaptation by changes in the prior distribution. The model predicted that adaptation would cause increased similarity (“attraction”) of stimuli to the adaptor. Experimental results showed the opposite: a reduced similarly (“repulsion”) of stimuli to the adaptor (e.g.,² To remedy the discrepancy, the authors modeled adaptation by adjusting the likelihood function, rather than the prior distribution. They proposed that “adaptation acts to allocate more resources to the representation of the parameter values in the vicinity of the adaptor … resulting in a local increase in the signal-to-noise ratio” thus broadening the likelihood function, because the resources are withdrawn from conditions removed from the adaptor. Repulsion is caused by the broadening of the likelihood function. The authors had to step outside of the standard Bayesian framework and make an assumption for which they had no principled theory. We would like to contrast their approach with the normative approach presented here, which offers an explicit principled theory that predicts just how a visual system should reallocate its resources in response to changes in stimulus statistics.

4 Adaptive Transformation of Sensitivity

4.1 Rapid Assessment of Contrast Sensitivity

The formulation of spatiotemporal sensitivity by Kelly (introduced in Fig. 2) reflects contrast thresholds for pattern detection as a function of (sf,tf). Nakayama²¹ reviewed multiple other formulations and concluded that “seemingly very different phenomena: [motion aftereffect], pattern detection and direction discrimination, fit a similar set of functions.” In other words, the data represented as sensitivity over (sf,tf) all have the general “bent-loaf” appearance reported by Kelly. Based on our preliminary results, we chose to assess spatiotemporal contrast sensitivity using a direction discrimination task. This task helps to avoid the response biases characteristic of detection tasks.

For this study, we deployed a form of the “quick” method for assessing the Contrast Sensitivity Functions (quick CSF or qCSF) first developed by Lesmes et al.¹⁸. This method generalizes the previous procedures for rapid estimation of psychometric functions (PSI procedure by Kontesvich and Tyler 1999; also Cobo-Lewis 1997), external noise functions,¹⁷ and other threshold functions¹⁵. We validated the method by comparing estimates by qCSF and PSI procedures within one stimulus speed (one “slice” of the sensitivity function), and then generalized this method to measuring multiple slices concurrently in humans¹⁶ and monkeys²³.

4.2 Comprehensive Estimates of Sensitivity Change

To study how stimulus statistics affects the distribution of contrast sensitivity, we created two stimulus contexts, as shown in Fig. 6. We varied how often stimuli were sampled from the same stimulus grid, creating two contexts: low-speed and high-speed.

Figure 6.

Stimulus grid and stimulus statistics for adaptation experiments. Disks represent stimulus conditions arranged on seven constant-speed lines in domain of spatiotemporal sensitivity function (the “stimulus space”). Two complementary histograms at top right illustrate “stimulus contexts”. In low-speed context, low speeds are sampled more often. In high-speed context, high speeds are sampled more often. Mean speeds of contexts are 6 and 12 deg/s.

Examples of sensitivity functions obtained this way are displayed in Fig. 7a. Sensitivity changes are plotted (for one subject) in Fig. 7b: for two speeds at top and for the entire domain of the sensitivity function at bottom. As in theoretical change map (Fig. 5b), sensitivity changes were d_i = 100 (h_i−l_i)/h_i, where h_i and l_i are respective entries in the high-speed and low-speed sensitivity functions.²

Figure 7.

Large-scale assay of sensitivity change in human Ss. a Contrast sensitivity functions measured in two stimulus contexts for one S. Standard model of contrast sensitivity (Kelly function) was fitted to estimates of sensitivity in high-speed (top) and low-speed (bottom) contexts. Warm and cool colors represent high and low sensitivities. b Change map at bottom summarizes how sensitivity changed from the low-speed to high-speed stimulus contexts. Shades of red and blue represent gains and losses of sensitivity; white regions represent no change. At top samples of sensitivity changes for two speeds demonstrate that pattern of gains and losses of sensitivity is reversed across speeds, similar to prediction shown in Fig. 5.

The plots in the upper part of Fig. 7b demonstrate a reversal of sensitivity change across speeds, as predicted by our theory (Figs. 4, 5). At the low speed, sensitivity decreased for low-frequency conditions and increased for high-frequency conditions. At the high speed, the pattern was reversed. The seemingly erratic alterations of sensitivity within the narrow samples of stimulus conditions corroborate the notion that changes of sensitivity must be studied over large stimulus sets.

We evaluated patterns of sensitivity change across full range of (sf,tf) using templates of sensitivity change. The templates consisted of regions where gains and losses of sensitivity were predicted by the theory. We also compared the observed patterns of change to predictions of an alternative theory in which changes of sensitivity merely mirrored changes in stimulation. This analysis overwhelmingly supported our predictions, described in detail in Gepshtein et al.⁹

5 Neural Mechanisms of Adaptation

5.1 Approach

Our analysis of factors that control contrast sensitivity and the results of our psychophysical experiments⁹ demonstrate that local changes of sensitivity appear paradoxical when viewed in isolation but make sense in a larger context. These discoveries lead us to ask how multiple local changes are coordinated across the vast domain of visual sensitivity. Do visual systems monitor the distribution of stimuli? Is there a mechanism dedicated to coordinating the distribution of sensitivity?

Our general prediction is that responses of individual neurons (assessed as in²⁹) will reflect the changes of sensitivity discovered using psychophysical methods. Because the operating range of the (sf,tf) function for a given neuron will generally cover only a small portion of the behavioral (sf,tf) range, the predicted changes of the neuronal CSF are likely to be evident by one or both of: (1) the overall gain of the neuronal spatiotemporal frequency tuning function may either increase or decrease; (2) the peak of the neuronal (sf,tf) sensitivity function may shift. The study by Krekelberg et al.¹⁴ summarized in Fig. 1 provided preliminary evidence for both of these neuronal effects.

The general nature of predicted changes is illustrated in Fig. 8. Panel A portrays the global sensitivity function predicted for a low prevailing speed of 0.1°/s. Suppose that we record three MT neurons whose peak (sf,tf) preferences lie at the locations indicated by nodes ①, ②, and ③. Neurons ② and ③ lie near the peak of sensitivity, while neuron ① lies at a point of low sensitivity. Global effects of adaptation on sensitivity are illustrated in panel B: The sensitivity function has shifted slightly relative to its position in panel A. Panel E illustrates change in sensitivity from A to B. Neuron ② is expected to undergo a decline in sensitivity, while neurons ① and ③ are expected to become more sensitive.

Figure 8.

Predicted effects of speed adaptation on sensitivities of individual cortical neurons. Top row of panels illustrate global (sf,tf) “sensitivity maps”. a Pre-adaptation state depends upon distribution of speeds in the natural world (⁵). b Expected sensitivity following adaptation to 1.2°/sec (speed indicated by diagonal line). c Sensitivity following 8°/sec. As adaptation speed increases, global pattern of sensitivity retains “bent-loaf” shape but shifts markedly. Middle row of d–f illustrates fractional change in global sensitivity at indicated speeds (in D, change is nil, by definition). In a–f, we have indicated preferred (sf,tf) values for three hypothetical MT neurons. Predicted changes in neuronal sensitivity as a function of adapting speed are summarized for each of the three neurons in panels 1–3, at bottom. See text for further detail.

Similarly, panel C portrays the sensitivity function predicted for the prevailing speed of 8°/s, and panel F portrays the corresponding change in sensitivity. Sensitivities of the three neurons change: a marked increase for neuron ①, a substantial loss for neuron ②, and little change for neuron ③.

Predicted sensitivity changes for these neurons are summarized at bottom of Fig. 8. The horizontal axis in each figure indicates the prevailing speed; the vertical axis indicates sensitivity change. These simulations lead to the provocative prediction that sensitivity of individual neurons should change as a function of adaptation following a highly principled rule, in which the sensitivity change for a given neuron depends upon its position relative to the behavioral sensitivity change map. The second possibility (hinted at by results of¹⁴ is that psychophysical changes of sensitivity will be correlated with shifts of neuronal preference (not shown).

To understand the factors that control the distribution of neuronal sensitivities, we investigated how basic mechanisms of synaptic plasticity in single cells respond to changes of stimulation within their receptive fields (RFs). We pursued two specific goals: first, to characterize local changes of sensitivity and, second, to investigate the global distribution of these changes across the entire stimulus domain. Next, we describe numerical simulations of synaptic plasticity in spiking neural networks performed by in pursuit of both goals^{7, 11}.

5.2 Basic Neural Circuit

Figure 9a shows the generic circuit used in the simulations of synaptic plasticity. The circuit consists of one readout cell R and two input cells I₁ and I₂. Input cells have receptive fields of different sizes on a single dimension x.³ The receptive field size of I₂ is larger than that of I₁.

Figure 9.

Generic neural circuit. a Circuit consists of two input cells (I₁ and I₂) and one readout cell R. In this illustration, the receptive field size of I1 is smaller than that of I2 (represented by the number of lower level cells from which it receives information: three such cells for I1 and seven for I2). The receptive field size of readout cell (Sr) depends on weights w1 and w2. b Weights w₁ and w₂ determine the effects of I₁ and I₂ on R (see text). The weights are updated according to the stochastic co-activation of input and readout cells. Readout receptive field size Sr fluctuates on the interval between the sizes of input cells (here normalized to 1 and 2 for convenience). Histogram of Sr reveals the central tendency of the fluctuation. p (Sr) is the probability of Sr. c Small circles represent average changes of S_r—the amplitude of receptive field size fluctuation—for different magnitudes of S_r. The farther S_r is from its central tendency (green line copied from panel b), the larger the amplitude.

The receptive field size of the readout cell (S_r) depends on the synaptic weights (w₁ and w₂), which determine which of the input cells has a larger effect on R. For example, when w₂ > w₁, the size S_r is similar to the size of I₂ (i.e., is larger than in the cases when w₁ = w₂ or w₁ > w₂). Substantial changes in the weights w₁ and w₂ will entail changes in the receptive field size of R.

The weights w₁ and w₂ depend on the relative timing of presynaptic and postsynaptic spiking^{3, 4, 22}. When one of the input cells (presynaptic) and the readout cell (postsynaptic) co-fire, the corresponding weight increases. For example, when the cells I₂ and R co-fire more often than the cells I₁ and R, the weight w₂ increases more often than the weight w₁. In that case, the receptive field size of R becomes larger, i.e., more similar to the size of I₂ than I₁.

5.3 Receptive Field Dynamics

Because neuronal firing is a stochastic process, updating of synaptic weights is also a stochastic process that leads to temporal fluctuations of readout RF size. In Fig. 9b, c we illustrate two important features of these dynamics. First is the central tendency of readout RF size. The fluctuations of readout RF size are confined to the interval between the smallest and largest input RF sizes. Probabilities of readout RF sizes over the course of one simulation are captured in the histogram in Fig. 9b (see,¹¹ for details). The green line in Fig. 9b is the most likely readout RF size (S_r*): the central tendency of readout RF fluctuation.

The second important feature of readout dynamics is the amplitude of fluctuations (Fig. 9c). The amplitude is low when the readout RF size is similar to S_r* and it is larger when readout RF size is removed from S_r*. We illustrate the consequences of these dynamics in Figs. 10 and 11.

Figure 10.

Stochastic tuning of one RF. a Outcomes of 1000 simulations of circuit with 25 input cells and one readout cell. The two dimensions represent spatial and temporal frequencies of stimuli normalized, so that initial tuning of the readout cell corresponds to (0, 0) marked by the white cross. In each simulation, input-readout weights were updated 700 times. Final parameters of each readout RF are represented by a red dot (“endpoint”). Three simulations are illustrated by the gray-arrow “trajectories” in the (sf,tf) space (only 20 steps are shown). Contour plot in background represents average amplitude of fluctuation. It is a two-dimensional equivalent of the curve in Fig. 9c. a Histogram of endpoint density is plotted as a heat map: the warmer the color, the higher the endpoint density. The central tendency of fluctuation (the peak of endpoint density) is marked by intersection of while gridlines in the inset and by the asterisk in the main plot. b, c Stimulus bias. Mean speeds of stimuli were low in b and high in c. Histograms at left (depicting endpoint density as in the inset of a show that outcomes of RF fluctuation are biased by stimulation. The directed yellow markers at right point in direction of RF drift: from the initial conditions (central disk) to mean endpoints. These markers are used in Fig. 11 to summarize drift of readout RFs across (sf,tf) space.

Figure 11.

Pattern of RF drift across (sf,tf). The same simulations as in Fig. 10 were performed at multiple (sf,tf) condition. Local drift directions are represented by the markers introduced in Fig. 10b, c. These directions form a systematic global pattern. These results suggest that the globally optimal RF allocation predicted by the theory of Gepshtein et al.¹⁰ can be achieved by local means alone, requiring no mechanism that coordinates the local processes. The white curve is the sum of conditions on which the local processes converge yielding the highest RF density.

Notice that stimulus dimension x in Fig. 9a could stand for either location or frequency content of the stimulus. We introduced the circuit in terms of RF size (i.e., assuming x was location), to help intuition and to follow Jurica et al.¹¹. In what follows we present results of our simulations of RF plasticity in the frequency domain, because our present stimuli are defined by frequencies of luminance modulation and because we study the effects of adaptation on contrast sensitivity, which is traditionally rendered in the frequency domain.

Our numerical simulations of synaptic plasticity revealed that fluctuations of RFs are biased by stimulation. The dynamics are best described as drift of the readout RF in (sf,tf) space, where drift direction is determined by local stimulus statistics. In Fig. 10, we illustrate this drift in a circuit of 25 input cells and one readout cell. First consider panel A, where stimulation was uniform. The central white cross identifies the initial parameters of the readout RF for every simulation. Each red point corresponds to the readout RF parameters after one of 1000 simulations. The distribution of endpoints (inset) is an outcome of correlation between the amplitude of RF fluctuation and the proximity of RF parameters to the central tendency of fluctuation.

Stimulus statistics change the central tendency of fluctuation and thus bias the direction of RF drift. Results in Fig. 10b, c were obtained under different stimulus statistics, biased towards low (b) or high (c) speeds. Direction of RF drift is different in the two cases, represented by the markers on the right side of panels b, c.

In Fig. 11, we summarize RF dynamics for multiple locations in (sf,tf) space. The simulations were performed as in Fig. 10, except the parameters of input RFs and the stimuli were selected according to the location in the (sf,tf) space, as described in Jurica et al.¹¹. The initial condition for every simulation is indicated by a point and the direction of RF drift is indicated by a line. The flow pattern in Fig. 11 is a result of the different biases of RF fluctuations created by the different stimulus conditions in different parts of the (sf,tf) space.

This analysis predicts that the adaptive changes of individual neurons should follow a regular pattern determined by the local stimulus statistics. When stimulus statistics change, it is expected that the distribution of sensitivity will change too (not shown in Fig. 11). We have performed simulations of such changes of neuronal tuning using the same manipulation of stimulus statistics as in our psychophysical experiments (Fig. 6). We found that adaptive changes form a global pattern consistent with both the theory of RF allocation (Figs. 4, 5) and results of our psychophysical experiments (Fig. 7)⁸. Now, we pursue a program of studies aimed to test the hypothesis that changes in the population distribution of sensitivities of motion-sensitive neurons in cortical area MT reflect the changes in the distribution of behavioral spatiotemporal sensitivity.

6 Conclusions

The significance of this work is both theoretical and empirical. These experiments implement a new way of thinking about sensory adaptation as an economic optimization of the visual system through reallocation of limited neuronal resources. This approach has inspired us to do new kinds of experiments and look at the results of these and other experiments from a new principled perspective. We pointed out that adaptive changes of sensitivity appear paradoxical from a local perspective, i.e., by studying sensitivity in narrow bands of stimulus parameters. The paradox is resolved by taking a broader perspective, i.e., by studying the distribution of sensitivity across a broader range of spatiotemporal conditions.

The broader perspective also leads to new questions about the mechanisms of neuronal change. We have described a model of the synaptic plasticity that underlies tuning changes of individual cells. The model suggests that changes should follow a regular pattern: peak sensitivities of cells should follow different paths in different parts of the stimulus space, while the directions of these paths should form a gradient across stimulus conditions⁷. We expect that the sensitivity changes of individual cells will form a global pattern across stimuli consistent with our predictions. Forthcoming studies will show whether this prediction holds.

Biographies

Sergei Gepshtein trained in neurobiology, perceptual psychology, and vision science, investigates perception and perceptually behavior from the mechanistic point of view of neuroscience and from a point of view that respects experience as a research focus in its own right. As a scientist at the Center for Neurobiology of Vision at the Salk Institute for Biological Studies in La Jolla, California, Sergei studies boundaries of perception in the natural world and in sensory media using methods of psychophysics and computational neuroscience. In 2017, Sergei founded the Center for Spatial Perception & Concrete Experience at the University of Southern California, where basic and translational studies of perception are combined to create new forms of immersive “spatial media” that include virtual and augmented reality.

Thomas D. Albright is a Professor and Conrad T. Prebys Chair at the Salk Institute for Biological Studies in La Jolla, California. His laboratory seeks to understand the brain bases of visual perception, memory, and visually guided behavior. Albright received a Ph.D. in psychology and neuroscience from Princeton University. He is a member of the US National Academy of Sciences, a fellow of the American Academy of Arts and Sciences, and a fellow of the American Association for the Advancement of Science. Albright is a member of the Scientific Advisory Committee for the Indian National Brain Research Center and was the IISc-DST Centenary Chair Visiting Professor in the IISc Center for Neuroscience from 2011–2013. He is co-organizer of the biennial Bangalore Workshop on Cognitive Neuroscience.