Perceptual and neural consequences of rapid motion adaptation

Abstract

Nervous systems adapt to the prevailing sensory environment, and the consequences of this adaptation can be observed in the responses of single neurons and in perception. Given the variety of timescales underlying events in the natural world, determining the temporal characteristics of adaptation is important to understanding how perception adjusts to its sensory environment. Previous work has shown that neural adaptation can occur on a timescale of milliseconds, but perceptual adaptation has generally been studied over relatively long timescales, typically on the order of seconds. This disparity raises important questions. Can perceptual adaptation be observed at brief, functionally relevant timescales? And if so, how do its properties relate to the rapid adaptation seen in cortical neurons? We address these questions in the context of visual motion processing, a perceptual modality characterized by rapid temporal dynamics. We demonstrate objectively that 25 ms of motion adaptation is sufficient to generate a motion aftereffect, an illusory sensation of movement experienced when a moving stimulus is replaced by a stationary pattern. This rapid adaptation occurs regardless of whether the adapting motion is perceived. In neurophysiological recordings from the middle temporal area of primate visual cortex, we find that brief motion adaptation evokes direction-selective responses to subsequently presented stationary stimuli. A simple model shows that these neural responses can explain the consequences of rapid perceptual adaptation. Overall, we show that the motion aftereffect is not merely an intriguing perceptual illusion, but rather a reflection of rapid neural and perceptual processes that can occur essentially every time we experience motion.

The nervous system constantly adapts to the statistics of its sensory inputs, and such adaptation has important perceptual consequences (1, 2). Examples include the desensitization of the somatosensory system to constant stimuli (e.g., clothing) and the visual system's adjustments to prevailing light levels. Adaptation affects both neural activity during constant stimulation and neural responses to subsequent stimuli (1). Given the variety of timescales underlying events in the natural world, determining the temporal characteristics of adaptation is a key step in understanding sensory perception. Traditionally, perceptual adaptation is thought to occur over long time periods, with the adapting stimuli being presented for seconds or longer. Neurophysiological work, however, has shown that adaptation occurs at a variety of timescales and can be observed following stimulus exposures as brief as tens of milliseconds (1, 3–7). This rapid adaptation has strong implications for neural coding of sensory stimuli (5, 6, 8), which renders the relative lack of evidence about brief perceptual adaptation puzzling. Here we consider this question in the context of visual motion processing. Because of the highly dynamic nature of moving stimuli, rapid adaptation is potentially of great relevance for motion perception.

A compelling and extensively studied perceptual consequence of motion adaptation is the motion aftereffect (MAE). First documented by Aristotle, the MAE is defined as an illusory sensation of motion in a static stimulus resulting from prolonged adaptation to a moving stimulus (9, 10). However, the relevance of the adaptation processes that generate MAEs to natural vision is unclear: although the MAE seemingly requires seconds of motion adaptation (10), everyday sensory experience involves moving objects that change position rapidly. Moreover, neural motion adaptation can be elicited with stimuli as brief as 60 ms (4, 11–13). One possible explanation for this discrepancy is that the MAE is a perceptual consequence of slow adaptation processes that have been shown to occur on the order of seconds (14–16). If that is true, the MAE reflects adaptation processes that rarely occur during natural visual experience: aside from the special case of smooth pursuit, moving objects generally occupy a given retinal location for only a fraction of a second. Alternatively, the MAE might also occur following brief adaptation durations that are relevant to everyday visual experience. That would suggest that the rapid motion adaptation seen neurophysiologically (4, 11–13) might have perceptual and neural consequences that have not yet been discovered.

Here we addressed this question by objectively establishing the shortest adaptation duration that is sufficient to produce a perceivable MAE. An inherent shortcoming of MAE measurement is that the MAE is a subjective perceptual experience whose direction is predicted by the direction of the adapting motion. This is particularly problematic for measuring MAEs in response to brief adapting stimuli because the resulting MAEs are likely to be weak, increasing the possibility that observers’ responses will be biased by the knowledge of the expected MAE direction. To circumvent this problem, we devised an objective way of measuring MAEs by using adapting stimuli for which the motion direction is not perceived. Our experiments demonstrate that the MAE can be observed after adaptation durations as brief as 25 ms, indicating that perceptual motion adaptation reflects neural processes that can occur essentially every time motion is sensed.

These behavioral experiments were followed by neurophysiological recordings in the middle temporal (MT) cortical area of the macaque. Consistent with previous results (4, 11–13), we find that MT neurons exhibit adaptation to brief motion stimuli. More importantly, we find that such adaptation yields direction selective responses to subsequently presented stationary stimuli: a result that is arguably the simplest possible neural correlate of the perceptual MAE seen in static stimuli. A simple model shows that these MT responses can account for our perceptual results and their dependence on stimulus parameters. Together these results suggest that neural and perceptual adaptation have similar properties and reflect processes that are at work during natural visual experience.

Results

MAEs Generated by Brief Motion Adaptation.

In the first two experiments, our goal was to objectively measure the MAE generated by brief motion adaptation. To accomplish this aim, we investigated a special case in which direction discriminations of the brief adapting stimulus were at chance, which ensured that observers did not know the expected MAE direction. This required the use of specific stimulus parameters (as detailed later) and exclusion of observers who did not exhibit chance-level performance. Having objectively verified the existence of the rapidly generated MAE, we were able to relax these restrictions in subsequent experiments.

First, observers (N = 7) adapted to a foveal grating presented for 67 ms that moved to the left or right at 15°/s (99% contrast, radius of 8°; Materials and Methods includes details; Fig. 1A). These brief stimuli yielded at-chance motion direction discriminations (Fig. 1C, black circle), a result expected from our previous findings revealing strong spatial suppression of large, high-contrast motion signals (17). On trials in which the MAE was measured, the adapting stimulus was followed by a stationary test stimulus at one of four interstimulus intervals (ISIs). At the end of each trial, observers indicated the perceived direction of the test stimulus motion. A response was deemed correct if it was opposite to the adapting motion, as that would be expected if the MAE was perceived. Given that observers were unable to perceive the motion direction of the adapting stimulus, these responses constitute an objective MAE measure. The results (Fig. 1C) reveal that, for short ISIs, observers consistently perceived the test stimuli as moving in the opposite direction from the adapting motion, demonstrating that brief motion adaptation can be sufficient to generate the MAE. This pattern of results was found in all observers. As the ISI increased, test stimulus direction discriminations deteriorated to near chance levels, indicating that the strength of the MAE decreases rapidly with time after motion adaptation. (Subjectively, the 0-ms ISI condition yielded much stronger MAEs than the 150-ms ISI condition, but the present method lacks the dynamic range to resolve these differences because of the ceiling effect imposed by 100% correct performance. This issue was resolved in the nulling experiments described later.)

Fig. 1.

Consequences of rapid motion adaptation. (A) Space–time illustration of the stimuli and the task used in the main MAE experiment. On each trial, a high-contrast moving adapter was presented for 67 ms and followed by a low-contrast stationary test stimulus at variable ISIs. At the end of each trial, observers indicated the perceived direction of the test stimulus motion. Importantly, we used adapting stimuli whose motion direction could not be correctly discriminated, ensuring that observers’ reports of test stimulus motion constitute an objective measure of MAE presence. (B) Space–time illustration of the stimuli used to measure MAE by using adapter-induced afterimages. (C) Results of the main MAE experiment show correct perception of MAE direction at short ISIs (empty symbols). The filled symbol shows at-chance direction discriminations of the adapting motion. (D) Results demonstrating the shortest adaptation duration that is sufficient to produce a perceivable MAE. The results show direction discriminations of adapter-induced afterimages as a function of adapter duration. The filled symbol shows at-chance direction discriminations of a contrast-balanced 67 ms adapter. All error bars indicate SEM.

Having found that MAEs can be elicited by very brief motion stimuli, we next sought to determine the shortest adaptation duration for which MAEs could reliably be elicited. This required decreasing the number of movie frames of the adapting stimulus (Fig. 1B), which unbalanced the cumulative stimulus contrast (Materials and Methods). As a result, the adapting stimuli generated afterimages that were visible for a short time following adapter disappearance. We exploited this property of the adapting stimulus by using afterimages as MAE test stimuli. This use of afterimages as test stimuli is also a control for a possible eye movement confound; in the previous MAE experiment, any eye movements that occurred in response to the adapting stimulus would cause retinal motion of the test stimulus. However, such retinal motion would be absent in this experiment because afterimages move with the eyes. The results (Fig. 1D) show near-perfect discriminations of MAE direction for adapting durations between 33 and 58 ms and that perceivable MAEs can be generated with as little as 25 ms of adaptation to motion.

The results thus far demonstrate that adaptation to remarkably brief motion stimuli can yield perceivable MAEs, even when the adapting duration is shorter than the duration required to perceive adapter motion. In turn, this indicates that MAEs can be observed after an adaptation period that is sufficiently brief to affect perception during typical visual experience. Subjectively, the observed MAEs appeared qualitatively similar to those obtained with prolonged adaptation, albeit unmistakably weaker.

Rapid Motion Adaptation Reflects Early Visual Motion Processing.

The results described thus far (Fig. 1C) show that perception of adapting motion direction is not necessary for the generation of direction-specific MAEs. To directly test the role of perceptual visibility in rapid motion adaptation, we investigated how changes in the discriminability of adapting motion affected the resulting MAEs. Because this requires the use of stimulus conditions with perceivable adapting motion, we used speed nulling (18, 19) to measure the MAE in these experiments (Materials and Methods). Specifically, we measured the amount of real motion that was needed to counter the observed MAE (i.e., the stimulus speed required for the test stimulus to be perceived as stationary). This method can be used irrespective of whether the adapting motion direction is perceived.

Perceptual visibility of the adapting motion was manipulated by changing the size of the adapting stimulus. Because of surround inhibition, increasing the size of high-contrast moving stimuli results in a dramatic worsening of direction discriminations (17) (Fig. 2B). Thus, if the buildup of rapid adaptation depends on the perceptual visibility of adapting motion, MAE strength should decrease with increasing size. In contrast, we found just the opposite: MAE strength increased with increasing size (Fig. 2A). Notably, the increasing MAE strength with increasing stimulus size is well described by a power law with an exponent approximating unity, indicating a linear spatial summation process (20). Thus, although the observed adaptation effects likely occur early in visual processing (the following experiments and Discussion provide details), perception of the ensuing MAE is facilitated by the summation of motion signals over space, likely occurring within large receptive fields. We will return to this issue in the context of the neurophysiological experiments described later.

Fig. 2.

Dissociation between adaptation strength and perceptual visibility of adapting motion. (A) Effect of stimulus size on MAE strength, as measured by the nulling speed. All adapting stimuli were presented for 67 ms. Data were fit with a power law function (0.35 * x^0.97; r² = 0.986). Exponent near unity indicates linear spatial summation (20). (B) Effect of stimulus size on exposure duration required to correctly discriminate motion direction. To estimate 82% duration thresholds, six QUEST staircases were run for each condition. The gray area indicates threshold durations that are longer than the adapting duration used to measure MAE strength. Note that higher numbers in B indicate worsening motion visibility, whereas higher numbers in A show stronger MAE. All error bars indicate SEM.

The robust dissociation between perceptual visibility and adaptation indicates processes occurring at the earliest stages of motion processing. In the remainder of this section, we provide additional evidence for this hypothesis. First, we determined whether rapid motion adaptation exhibits any specificity for the adapted eye—a finding that would indicate a contribution of early, monocular visual mechanisms (21). Indeed, the MAE was stronger when the adapting and test stimuli were presented to the same eye [t(5) = 2.9, P = 0.03 vs. the case in which the adapting and test stimuli were presented to different eyes; Fig. 3A]. Second, we found that the rapidly generated MAE depends on the spatial frequency (SF) of the adapter [F(4,16) = 8.83, P < 0.001], being the strongest when the test frequency matched that of the adapting stimulus (Fig. 3B). This finding is consistent with results reported for the conventional MAE (10, 22), and is also suggestive of early, SF-tuned mechanisms.

Fig. 3.

Rapid motion adaptation reflects early visual motion processing. (A) Eye specificity of rapid motion adaptation. Adapting motion was always presented monocularly. Test stimuli were presented to the adapted eye or the opposite eye. (B) Effects of test SF on the rapidly generated MAE. Adapting SF was 1 cycle/°. (C and D) Effects of the orientation content of the adapting motion and test stimuli on the generation of rapid motion adaptation, as measured by the nulling speed. All error bars indicate SEM. Black circles in A and B show direction discriminations of the adapting motion.

Third, we used plaid stimuli to examine the dependence of the MAE on stimulus orientation. Such stimuli are comprised of two or more gratings with different orientations, and have been used to study motion integration at various levels of cortical processing (23, 24). Specifically, neurons in the primary visual cortex (V1) respond to the individual gratings that comprise the plaid, whereas many neurons in MT and MST encode the direction of the plaid pattern, irrespective of the motion of its components (23, 24). Thus, if adaptation occurs at the early stages of motion processing (e.g., V1), the strongest MAE should occur when the grating components of adapting and test stimuli match, regardless of pattern motion direction. Alternatively, if the adaptation occurs at the later stages in motion processing (e.g., pattern cells in MT), the strongest MAE should occur when the motion directions of adapting and test stimuli match (23).

The adapting stimulus was a horizontally moving vertical grating or a horizontally moving plaid whose components moved along the oblique axes (±45°). We used the nulling method to measure MAEs for three types of test stimuli: (i) a vertical grating, (ii) a plaid composed of oblique gratings, and (iii) a plaid composed of vertical and horizontal gratings (Fig. 3C). Note that the adapting grating stimulus shares component features with test stimuli i and ii, whereas the adapting plaid stimulus shares component features with only test stimulus ii. The results (Fig. 3C) showed that the strength of the resultant MAE depended on the relationship between the adapting and the test stimuli [F(2, 4) = 45, P = 0.002; no main effects, all P > 0.11] and was considerably stronger in those conditions in which the component orientation of the adapting and test stimuli matched. In an additional experiment, we tested whether obliquely moving plaids (composed of vertical and horizontal gratings) were equally effective at evoking an MAE in a vertical test grating as an adapting vertical grating. The results revealed no difference in the MAE strength between the two types of adapting stimuli [t(2) = 0.42, P = 0.72; Fig. 3D]. Overall, these experiments indicate that it is component orientation and not pattern motion direction that determines MAE strength—in turn suggesting that the observed rapid adaptation occurs at an early stage (or stages) of motion processing.

Characterizing Rapid Motion Adaptation.

To generate predictions for subsequent neurophysiological experiments, we characterized how changes in stimulus parameters affect rapidly generated MAEs. We first replicated our original experiment (Fig. 1C) with MAE nulling, both in the fovea and near periphery. The use of nulling measurements was required (i) because of its better dynamic range and (ii), more importantly, because it allowed us to measure MAEs for easily discriminated peripheral stimuli (17). For the peripheral adaptation, we set the stimulus eccentricity equal to the median receptive field location in our physiological recordings (5.1°; described later). The results (Fig. 4) were similar for the fovea and periphery, again showing that perceptual visibility of the adapting motion does not determine MAE strength. The nulling speeds started at approximately 3.5°/s and sharply decreased with increasing ISI (decay time constants of ∼84 ms). These findings are consistent with the subjective impressions of observers, who perceived MAEs at 0 ms ISI as being much stronger than MAEs at 150 ms ISI. In a separate experiment, we found that the MAE was substantially weaker when a high-contrast (99%) test stimulus was used (Fig. 4). This result is consistent with previous reports that motion adaptation affects the contrast gain but not the response gain of MT neurons, resulting in strong modulation of responses to low-contrast test stimuli and weak effects at high contrast (14). Both the exponential decay of MAE strength and the observed effect of the test contrast are consistent with MAEs obtained with prolonged motion adaptation (25), suggesting that adaptation duration changes the strength but not the nature of motion adaptation (1, 10).

Fig. 4.

Dependence of rapid motion adaptation on ISI, eccentricity, and test contrast. Results show sharply decreasing MAE nulling speed with increasing ISI for low-contrast test stimuli and near-zero nulling speeds for high-contrast test stimuli. Lines show exponential decay fits. Best fit decay time constants were 91 ms and 77 ms for 0 and 5.1° eccentricities, respectively. All error bars indicate SEM.

To summarize the human psychophysics findings, we show that the MAE can occur after remarkably brief periods of adaptation. Converging evidence from several experiments indicates that this rapid motion adaptation reflects early motion processing. We next investigated how the properties of rapid perceptual adaptation relate to the rapid adaptation seen in cortical neurons.

Neurophysiological Correlates of the Rapid MAE.

Our behavioral results (Fig. 3) suggest that the neural site of rapid motion adaptation is early in the visual pathway, perhaps as early as V1. However, the spatial characteristics of the effect are more consistent with an area in which neurons have larger receptive fields: for small adapting stimuli (i.e., those similar in size to V1 receptive fields), perceptual MAEs are very small (Fig. 2A), such that considerable spatial summation is needed to reveal strong effects (Fig. 2A). We therefore recorded from single neurons in the MT area, a cortical area known to be causally involved in sensory decisions about motion direction (26, 27). MT neurons have relatively large receptive fields (26), and various lines of evidence indicate that basic motion-processing properties change very little between V1 and MT (14, 28). Thus, recording from MT neurons should reveal adaptation processes occurring earlier in the visual pathway (14) and the spatial summation effects found in our psychophysical results.

Motion adaptation in area MT has been studied in some detail over a variety of timescales (4, 11–15, 29, 30), but despite decades of research, it is not known how MT neurons respond to stationary stimuli following exposure to adapting motion. Answering this question is needed to determine the role of area MT in the most compelling perceptual consequence of motion adaptation: MAEs seen in static test stimuli (i.e., the “waterfall illusion”). Moreover, there is no empirical evidence that allows direct comparison of rapid perceptual and single-neuron motion adaptation. Here, we recorded from 106 MT neurons in two alert monkeys during presentation of brief stimuli that were designed to correspond to those used in the psychophysical experiments (Materials and Methods). Our objective was to determine whether MT neurons exhibit direction selectivity to stationary stimuli following brief exposure to motion.

Fig. 5A shows the responses of an example MT neuron to brief adapting motion followed by a low-contrast stationary test stimulus. The initial response to the adapting motion is direction-selective, with a higher response for motion in the preferred direction than in the null (i.e., antipreferred) direction. However, the response to null motion is far slower to decay, leading to a reversal of direction selectivity beginning approximately 80 ms after the onset of the stationary test stimulus. Surprisingly, this sustained null-direction response persists for the duration of the test stimulus, suggesting that the observed effect is not simply a transient off-response. We therefore sought to determine whether this reversal of direction selectivity exhibited properties similar to those of the perceptual MAE. To relate the neuronal and perceptual responses, we implemented a simple model consisting of a leaky integrator (31, 32) followed by receiver operating characteristic (ROC) analysis (33) (Fig. 5 B and C). The integrator time constant was set to 50 ms. The use of the leaky integrator and its duration were motivated by previous results demonstrating that behavioral and neural responses to very brief motion signals (31, 32, 34) have an obligatorily low-pass character, possibly imposed by structures downstream from MT (31). Other models are possible, but this one had the virtues of being particularly simple and consistent with behavioral and neurophysiological findings.

Fig. 5.

Responses of area MT neurons to stationary stimuli following brief exposure to motion. (A) Responses of an example neuron to 67 ms of grating motion in the preferred (black) or null (red) directions, followed by a low-contrast stationary test stimulus. Neural activity is shown as raster plots and firing rates as a function of time (obtained by convolving raw spikes with a Gaussian filter of σ = 10 ms). Inset near the top of the y axis shows the space–time plot of the stimulus time course. (B) Exponential filter (τ = 50 ms) used to implement the leaky integrator model and the result of its convolution with the example activity shown in A. Insets: ROC curves at 150 and 300 ms after motion onset. A complete sequence of time-dependent ROC changes for this neuron can be seen in Movie S1. (C) Direction selectivity of the example neuron in response to a stationary test stimulus. Direction selectivity is expressed as the aROC computed by using a 1-ms sliding window from responses shown in B. Thin lines show ±1 SD computed from a resampling analysis (Materials and Methods). Graphic over x axis shows the stimulus time course. (D) Population averages showing changes in direction selectivity for different stationary test stimuli. Graphics just above the x axes illustrate time courses of three conditions differing in the onset timing (0 vs. 150 ms ISI) and contrast (low vs. high) of the test stimulus. All error bars indicate SEM. (E) Histogram shows total duration of significant null-direction selectivity for three conditions differing in the onset timing (0 vs. 150 ms ISI) and contrast (low vs. high) of the test stimulus. The means of two low-contrast conditions were significantly higher than expected by chance alone (P < 0.001 and P = 0.016; Results and Materials and Methods provide further details).

Following the filtering described earlier, ROC analysis was carried out in 1-ms sliding “windows.” The resulting ROC curves for an example neuron at 150 and 300 ms after stimulus onset are shown as insets in Fig. 5C, and a complete sequence of time-dependent ROC changes for this neuron can be seen in Movie S1. The area under these ROC curves (aROC) provides a scalar measure of direction selectivity (33), which is shown by the blue line in Fig. 5C. This analysis reveals strong and sustained direction selectivity in response to the stationary test stimulus, as indicated by the aROC values lower than 0.5. As this response pattern would signal motion opposite to the adapting motion, this late phase of the response is a possible neural correlate of MAEs observed in our psychophysical experiments, i.e., a neural MAE. To further examine this possibility, we recorded from MT neurons in three stimulus conditions designed to be similar to those in our psychophysical experiments. In the main condition (n = 38), a 3% contrast stationary test stimulus was presented immediately after the adapting motion offset. These stimulus parameters yield a strong MAE in our psychophysical experiments. In the second condition (n = 15), the test stimulus contrast was 99%, a manipulation that essentially abolished the perceptual MAE (Fig. 4). Finally, in the third condition (n = 27), the low-contrast test stimulus was presented after a 150-ms ISI, with no stimulus shown during the ISI. Our psychophysical results predict a weaker, but nonetheless detectable, effect of the test stimulus in this latter condition. In all conditions, cells were tested with large adapting stimuli, as our psychophysical data indicate that the perception of the resultant MAE requires spatial summation of motion signals over a large area (Fig. 2A).

The population analysis (Fig. 5D) demonstrates that the presence of the stationary test stimulus clearly modulates MT activity. The strongest null direction selectivity was observed for low-contrast test stimuli presented immediately after the adapting motion (Fig. 5D, Left, blue curve). Remarkably, this null direction selectivity persists for nearly 700 ms after the offset of the brief adapting motion, but is absent for test stimuli that are high in contrast (Fig. 5D, Middle, red curve). Finally, when the ISI was increased to 150 ms, null direction selectivity to low-contrast test stimuli was still present but was considerably weaker (Fig. 5D, Right, green curve). This lack of null-direction selectivity during the 150-ms ISI period shows that the observed results do not reflect the transient off-response to the adapting motion but rather constitute sustained neural selectivity evoked by the presence of the test stimulus. An additional analysis in which the aROC was computed directly from spike counts (i.e., without leaky integration) supports the conclusion that the sustained null-direction selectivity does not simply reflect transient off-responses (Fig. S1). Specifically, although the transient off-response does contribute to the overall null-direction selectivity, it is neither necessary nor sufficient for its existence.

To estimate cell-by-cell null-direction selectivity, we computed the total duration of each neuron's significant null-direction selectivity in the period after onset of the stationary test stimulus (Fig. 5E and Materials and Methods). Neurons tested with low-contrast test stimuli at 0-ms ISI had significant null-direction selectivity for an average of 59 ms (P < 0.001). When the ISI was increased to 150 ms, this duration decreased to 25 ms but was still significant (P = 0.016). Finally, for neurons tested with high-contrast test stimuli, the average duration of null-direction selectivity was not higher than expected by chance alone (13 ms; P = 0.145). Overall, these findings mirror our psychophysical results, which show strong MAEs for low-contrast test stimuli whose strength decrease with increasing ISI and increasing test contrast (Fig. 4).

The observed neural MAEs may derive from an increase in neural responses to stationary test stimuli following null-direction adaptation, a decrease in neural responses to static test stimuli following preferred direction adaptation, or both. To test these possibilities, we examined the responses of MT neurons (n = 22) to stationary, low-contrast stimuli that were not preceded by motion adaptation. The results (Fig. S2) suggest that the observed null-direction selectivity was mostly driven by decreased postadaptation preferred-direction responses. This finding is consistent with functional MRI work on motion adaptation (35) and a classic prolonged adaptation study by Barlow (36), and indicates that the MAE seen in static stimuli is likely driven by directional imbalances caused by a selective decrease of activity (rather than an increase). Moreover, this result also rules out the possibility that eye movements and/or attention caused null-direction selectivity by selectively increasing responses to stationary test stimuli. Indeed, a quantitative analysis (Fig. S3) confirms that eye movements did not contribute significantly to the adaptation effects reported here. Additional consideration of possible attentional confounds is provided in the Discussion.

Having found a potential neural correlate of the perceptual MAE in the sustained responses of MT neurons to stationary stimuli, we wondered whether these responses could also account for the size dependence of our perceptual findings (Fig. 2A). We therefore recorded from 26 MT neurons while presenting small adapting stimuli matched in size to the receptive field of each cell (mean radius of 3.7°). Other stimulus parameters were identical to the condition yielding the strongest null-direction selectivity for large stimuli (i.e., low-contrast test stimulus and 0-ms ISI; Fig. 5D). The population average (Fig. 6A) shows that, in contrast to the results with large stimuli (Fig. 6A, blue curve), MT neurons exhibited weaker direction selectivity to small test stimuli (Fig. 6A, green curve) presented immediately after adapting motion. This weakening of null-direction selectivity was surprising given that the smaller stimuli actually yielded considerably better direction selectivity to the adapting motion compared with large stimuli (Fig. 6A). The dissociation between direction selectivity to the adapting motion and direction selectivity to subsequent static test stimuli indicates that the latter does not necessarily depend on MT processing and may reflect processes occurring earlier in visual processing. Importantly, these findings match our psychophysical observations, which show that the rapidly generated MAE does not depend on the perceptual visibility of the adapting motion and is actually weaker for easy-to-discriminate small stimuli (Fig. 2).

Fig. 6.

Effects of stimulus size and surround suppression on responses of MT neurons to stationary stimuli following brief exposure to motion. (A) Population average showing changes in direction selectivity over time for small and large stimuli (data for large stimuli replotted from Fig. 5D). Small adapting stimuli were matched to the size of the classical receptive field of each neuron (average radius, 3.7°). Direction selectivity was computed as described in Materials and Methods and Fig. 5. Graphic over x axis shows stimulus time course. (B) Population average shows changes in direction selectivity over time for neurons exhibiting weak or strong surround suppression. A median split was used to divide neurons into two groups (median SI, 0.41). All error bars indicate SEM.

Although the results shown in Fig. 6A support our conclusion that the main site of the observed adaptation is likely early in the visual processing pathway, they also indicate that the expected neural MAEs for cells with small receptive fields (e.g., V1 neurons) are likely to be very weak. In other words, for brief adapting motions, considerable spatial pooling of local motion signals may be needed to reveal strong neural MAEs. This hypothesis is also suggested by our psychophysical results showing that perceptual MAEs are characterized by linear spatial summation of motion signals over large areas (Fig. 2A). Previous work has shown that approximately 50% of MT neurons have receptive fields with strong surrounds, leading to poor responses to large stimuli (37, 38). This population of neurons would seem to be an unlikely candidate to provide spatial pooling we see in psychophysical results (Fig. 2A). On the other hand, wide-field MT neurons by definition integrate motion information over larger retinal areas and, thus, are better suited to provide considerable spatial pooling. To examine this issue, we divided cells in the main condition (Fig. 5D, blue line) into those with strong surrounds and those without, based on a simple median split of the surround index (SI; Materials and Methods). The results (Fig. 6B) show that neurons with weak surrounds (Fig. 6B, black curve) show stronger and longer direction-selective responses to static test stimuli than those with strong surrounds (mean duration of significant null selectivity increased from 51 ms to 97 ms for cells with weak surrounds). Across the population, the SI was negatively correlated with both the peak magnitude (r = −0.57, P = 0.0002) and the duration (r = −0.56, P = 0.0003) of the postadaptation null-direction selectivity effect (Materials and Methods). No significant correlations were found in the other conditions shown in Fig. 5D and for cells tested with small stimuli. Taking the results in Fig. 6 A and B together, it is apparent that the neural MAEs are not determined by the MT response to adapting motion (which would be expected if the adaptation is occurring in MT), but rather by the extent to which MT neurons integrate their input signals over space. The link between the latter finding and our perceptual results (Fig. 2) indicates that although the observed adaptation likely occurs at earlier stages of processing, the neural signals within MT are a closer neural correlate of the observed perceptual MAEs.

In summary, consistent with previous results (4, 11–13), we find that MT neurons exhibit adaptation to brief motion stimuli. More importantly, we find that such adaptation yields direction selective responses to subsequently presented stationary stimuli—a response pattern constituting a likely neural correlate of the perceptual MAE. Indeed, our experimental manipulations of the contrast, size, and timing of the stationary test stimulus reveal similar effects on perceptually observed MAEs and MT responses. Namely, the strongest aftereffects were obtained with large, low-contrast test stimuli, and the strength of the observed effects decreased with increasing ISI and decreasing stimulus size. Crucially, the strength of the direction selectivity evoked by stationary test stimuli did not depend on neural selectivity to the adapting motion. This nicely dovetails with our psychophysical results, which showed that perceptual visibility of the adapting motion did not affect the strength of the ensuing MAE. These findings, along with results shown in Fig. 3, strongly indicate that the observed rapid adaptation effects are inherited from earlier in the motion pathway.

Discussion

Scientific investigation of the MAE has a rich tradition spanning more than a century of active research (10, 39). This work has focused almost exclusively on motion adaptation that is built up over many seconds of stimulation, as in the classic waterfall illusion. Here we showed that the MAE is not merely a perceptual illusion that follows prolonged exposure to a moving stimulus, but rather a consequence of a fast adaptation process that can occur essentially every time we experience motion. Our results show that, if an appropriate MAE test stimulus is used, 25 ms of motion adaptation is sufficient to yield an MAE.

In parallel MT recordings, we provide neurophysiological evidence of directional responses to static stimuli following exposure to adapting motion. We also show that the stimulus conditions that produce perceivable MAEs also modulate the responses of MT neurons to stationary stimuli. This response pattern likely constitutes the neural correlate of illusory motion perceived in stationary stimuli presented shortly after brief motion adaptation.

Relation to Other Psychophysical Findings.

We have demonstrated that as little as 25 ms of motion adaptation is sufficient to cause a stationary stimulus to appear as moving. To the best of our knowledge, that is two orders of magnitude shorter than the previously reported adapting durations used to elicit static MAEs (10). Subsecond motion adaptation has been investigated using dynamic, directionally ambiguous test stimuli (i.e., counterphasing gratings) (40–42). In these reports, however, brief adapting motion (∼100 ms) primes perceptual interpretation of the stimulus motion, i.e., a counterphasing test stimulus is seen as moving in the same direction as the adapting motion (40, 41). Longer adapting durations were required to observe adaptation (i.e., bias ambiguous test motion in the direction opposite to the adapting motion). It is likely that the susceptibility of directionally ambiguous stimuli to high-level priming precluded observations of rapid adaptation in these studies. Raymond and Isaak (43) investigated temporal interactions between pairs of kinematograms and found results whose direction is consistent with adaptation. However, the magnitude of the observed effects did not vary with the ISI, and exhibited only minor dependence on the “adapting” duration. These results are inconsistent with low-level adaptation (indeed, the authors suggest a higher-level perceptual mechanism). The existence of these high-level effects on dynamic test stimuli partially motivated our choice to use static test stimuli to measure MAE strength. In the broader field of visual pattern adaptation, reports of rapid adaptation are equally rare and are generally restricted to higher-level visual processing, such as nonretinotopic shape after-effects (44).

Neural Correlates of Rapid Perceptual Motion Adaptation.

We observed a close correspondence between neurophysiological and psychophysical results: both estimates of the rapid MAE were the strongest for large, low-contrast test stimuli, absent for high-contrast tests, and decreased with increasing ISI. This was despite the fact that different tasks were used with two groups of subjects. Our neurophysiological results were obtained from alert subjects whose only task was visual fixation, which likely reduced the amount of attention directed to the adapting stimuli. This decrease of attention likely somewhat decreased the strength of adaptation (45). Attention, however, has only a small modulatory effect, rather than an enabling effect, on the MAE (45). In fact, strong MT adaptation is observed even under anesthesia (4, 12, 14, 15). Another notable difference between neurophysiology and psychophysics is that human observers gradually improved in their ability to perceive rapidly generated MAE (Materials and Methods). Although it is plausible that practicing seeing the MAE would strengthen the observed direction selectivity of MT neurons to stationary stimuli, an alternative possibility is that MAE practice improves not the sensory representation of MAE but a subsequent readout of this information (46).

It is important also to consider a possible attentional confound that was a critical problem in early functional MRI studies of the MAE. In this previous work, increased MT+ responses to static stimuli following prolonged motion adaptation (47) were later shown to reflect increased attention to motion while subjects perceived the illusory MAE (35). This confound is unlikely to have affected our results, because we find that neural MAEs are mostly driven by decreased firing rates following preferred-direction adaptation (Fig. S2). Moreover, other known effects of attention on MT responses (e.g., its time course) (48, 49) are inconsistent with our results. The lack of evidence indicating attentional modulation of our MT results, however, is not surprising. Unlike conventional MAEs, MAEs caused by rapid adaptation are weak and require a deliberate effort and some practice to be reliably perceived (in human subjects).

Our results indicate that the observed effects of rapid adaptation, although present in the responses of MT neurons, are at least in part inherited from earlier stages of processing. First, the strength of an MT neuron's response to the adapting stimulus does not determine the magnitude of its directional response to the static test stimuli. Specifically, suppression of MT responses with increasing stimulus size actually resulted in stronger neural MAEs (Fig. 6A). Second, paralleling these neurophysiological results, we show that impairments in the perceptual visibility of adapting motion caused by spatial suppression (17) do not impede the buildup of MAE (Figs. 1 and 2). Given evidence that spatial suppression reflects MT mechanisms (38), this suggests that the observed adaptation takes place at earlier stages of motion processing. Third, we show that the MAE strength is determined by the orientation of low-level components and not the perceived pattern motion direction (Fig. 3B). Finally, the observed MAE exhibits partial specificity for the adapted eye (Fig. 3A), implicating a contribution of early, monocular mechanisms. This overall conclusion is consistent with physiological results demonstrating that long-term adaptation of MT neurons to moving gratings is likely inherited from earlier visual areas (14). An additional role for intrinsic MT circuitry in rapid adaptation has been suggested by studies that used moving random dots (4). Our results with brief adapting stimuli suggest that the stimulus differences and not adaptation duration likely explain the different MT results obtained from brief (4) and prolonged adaptation (14). Specifically, unlike moving gratings, the random dot stimuli used by Priebe et al. (4, 12) elicit rather poor direction selectivity in V1 (28) and seem to preferentially recruit MT mechanisms (50).

Theoretical work on motion perception has suggested the existence of an “opponent” processing stage, which subtracts the outputs of detectors tuned to opposite directions of motion (51). Subsequent physiological recordings found evidence in support of this idea, as many MT neurons respond poorly to motion in their preferred directions when it overlaps motion in their antipreferred directions (52). Functionally, such opponency is likely useful for reducing responses to noisy or static stimuli (53). As the test stimuli used in our experiments were static, it is interesting to examine the types of responses they might elicit in an opponent motion circuit. A static grating stimulus should normally provide equal stimulation to neurons that prefer opposite directions of motion, and to the extent that populations of such neurons inhibit each other, one would expect the responses to such a stimulus to be quite small. However, when one population of neurons is adapted, opponent inhibition is reduced (14), and current models suggest that the resulting disinhibition (2) might lead to the perceptual and physiological effects we have observed. However, it is far less obvious why such effects would last for as long as they do—in our MT recordings, direction-selective responses to a stationary grating could be observed several hundred milliseconds after 67 ms of motion adaptation. The observed effect is similar to that of brief electrical stimulation, which has been shown to cause transient excitation followed by a prolonged period of suppression (32), likely caused by slow inhibitory postsynaptic potentials (54).

Conclusion

We show that the adaptation mechanisms underlying the generation of MAEs are likely an integral part of sensory responses to natural motion stimuli, which typically occupy a given retinal location for only a fraction of a second. One feature of these results is that perceptual MAEs following brief motion exposure occur only for a limited set of test stimuli. Crucially, this does not indicate that rapid motion adaptation occurs only in those special cases, but only that revealing its consequences in the form of MAEs requires carefully chosen test stimuli. Whether the rapid motion adaptation processes we have revealed using MAEs have other functional consequences for our perception is currently unknown. The broader functional role of rapid adaptation is supported by neurophysiological work showing that the visual system quickly recalibrates its stimulus sensitivity in response to stimulus changes (1, 5–8, 55), but behavioral support for this hypothesis has been mixed and limited to adaptation durations longer than 1 s (1). One possibility is that the use of longer adapting periods in the past introduced changes in sensory sensitivity that masked beneficial effects of rapid adaptation. Thus, determining how shorter adapting periods affect subsequent perception is an important direction for future research that will elucidate the role of adaptation during natural vision and possibly strengthen links between neurophysiological and behavioral findings.

Materials and Methods

Psychophysical Methods.

Stimuli were created in Psychtoolbox (56) and shown on a linearized CRT monitor (1,024 × 640 resolution, 120 Hz) at a 78-cm viewing distance. Ambient and background illumination were 0.01 and 31 cd/m². In most cases, the stimuli were moving vertical sine-wave gratings presented in a stationary raised cosine spatial envelope and a square-wave temporal envelope. The exceptions were experiments shown in Fig. 3 C and D in which stimuli were plaids composed of orthogonal sine-wave gratings. On each trial, a moving adaptor was followed by a test stimulus. In most conditions, the observers’ task was to indicate the perceived motion direction (left or right) of the test stimulus. Feedback was not provided. All experiments complied with institutionally reviewed procedures for human subjects, and informed consent was obtained from all subjects. Before participating, each observer first observed MAEs generated by a series of prolonged adapting stimuli, whose duration gradually decreased until it was equal to the brief adapting duration used in this study. This ensured that each observer was familiar with the perceptual appearance of the MAE. [Most observers reported strengthening of observed MAEs with practice. This essentially constitutes a form of perceptual learning and is consistent with a classic MAE finding (39).]

Main MAE experiment.

The results are shown in Fig. 1C. The adapting stimulus grating generated foveal motion (15°/s, 99% contrast, radius, r = 8°, SF of 1 cycle/°, random starting phase). The adapting stimulus was presented for 67 ms and consisted of eight frames of grating motion whose phase was advanced by π/4 on each frame. Because the adapting stimulus consisted of one full period of grating motion (8*π/4), the temporal sum of its contrast equaled 0, minimizing afterimage generation. As a result of the size, contrast, and brief duration of the stimulus, the majority of observers were not able to discriminate the motion direction of the adapting stimulus (17). Observers who did not exhibit at-chance motion discriminations were excluded from this experiment (one such observer was a subject in the nulling experiment described later). On each trial, observers (n = 7) viewed adapting motion that was followed by a low-contrast stationary test stimulus (r = 8°, SF of 1 cycle/°) at four randomly interleaved ISIs: 0, 150, 400, and 1,000 ms. Test stimulus contrast was determined separately for each observer and equaled 3*detection threshold (200 ms presentation, two temporal alternative forced choice tasks, 82% threshold). Test stimulus phase was advanced one motion step (π/4) in the direction of the adapting motion; i.e., the transition from the adapter to the test stimulus contained motion opposite from that expected if the MAE was present. This was a conservative choice to ensure that the transition between adapter and test stimuli did not contain motion in the direction of the expected MAE. (In our pilot explorations, we used test stimuli with random starting phase and found identical results.) The test stimulus was presented until the observer responded by indicating the perceived direction of the test motion. For simplicity, we defined “correct” responses to be those opposite to the adapting stimulus motion. Each observer completed 640 trials in four blocks. A fixation cross was presented before each trial, but was removed during stimulus presentation. Direction discriminations of the adapting stimulus (black symbols in Figs. 1 and 3) were remeasured before and after each block of trials to ensure that observers continued to exhibit at-chance motion discriminations of the adapter (320 trials per observer).

Note that we chose to measure MAE strength with stationary, rather than dynamic, MAE (17, 19, 57) test stimuli for two key reasons. First, dynamic test patterns are susceptible to high-level bias effects such as priming, which, for brief adapting stimuli, can overshadow effects of motion adaptation (40, 41). Second, our aim was to reveal low-level adaptation mechanisms that are potentially at work regardless of whether motion is consciously perceived. Unlike MAEs measured with static test patterns, dynamic MAEs require conscious perception of adapting direction (58, 59) and are not suitable to reveal low-level adaptation processes.

MAE measured by using afterimages.

The results are shown in Fig. 1D. There were five conditions, with adapting stimuli ranging in duration between 17 and 58 ms (two to seven frames). Importantly, this manipulation unbalanced the total stimulus contrast (temporal sum of contrast ≠ 0) and resulted in afterimage generation. No physical test stimulus was presented. Observers (n = 7) were instructed to indicate the perceived motion direction of each afterimage. As in the main experiment, direction discriminations of the eight-frame motion stimulus were remeasured before and after each block of trials to ensure that observers continued to exhibit at-chance motion discriminations of the adapter (320 trials per observer).

MAE nulling experiments.

The results are shown in Fig. 2A and Fig. 4. The methods for MAE nulling experiments were identical to those for the main MAE experiment, except that the test stimuli moved in the same direction as the adapting motion and were presented for 150 ms. By using an interleaved staircase procedure, we measured the test stimulus speed required to null the resulting MAE. Nulling speed for each condition was obtained by averaging the results of five staircases. Each staircase converged after seven reversals, with the average of the last four reversals taken as the staircase result. In different conditions (Fig. 4), adapting motion was presented foveally or at 5.1° eccentricity (the median receptive field location in our MT recordings). Test stimuli were either low (3%) or high (99%) contrast. In a separate experiment (Fig. 2A), we also investigated the effect of stimulus size on MAE strength. Observers (n = 3) adapted to foveal, high-contrast (99%) stimuli of varying radius: 2°, 4°, 8°, and 12° (in different conditions). Test stimuli were low contrast (3%).

Eye specificity of adaptation.

The results are shown in Fig. 3A. The adapting motion (r = 6°) was presented monocularly by means of a stereoscope to the left or right eye (randomly selected). A low-contrast (2%) stationary test stimulus was presented to the adapted or the opposite eye. The ISI was set at 150 ms to avoid possible ceiling effects, ensuring appropriate dynamic range for our MAE measurements. High contrast “fusers” were used to aid binocular fusion. The use of slightly smaller stimuli was necessitated by the limited field of view the stereoscope. Other methods were the same as in the main experiment (n = 6).

SF tuning.

The methods were identical to the main MAE experiment, except that the ISI was fixed at 150 ms to avoid ceiling effects, and the test stimulus SF was varied over four octaves (0.25, 0.5, 1, 2, and 4 cycles/°; Fig. 3B).

Component versus pattern adaptation.

The results are shown in Fig. 3C and D. In different blocks, the high-contrast adapting stimulus was a vertical grating (Fig. 3C, black bars) or a plaid composed of two component gratings oriented at ±45° (Fig. 3C, white bars). In each case, the perceived motion was in the horizontal direction (i.e., left or right). All component gratings were presented at 49% contrast and moved at 15°/s (thus, the pattern speed for the plaid stimulus was 21.2°/s and its maximum contrast was 98%). The nulling method was used to estimate the resultant MAE for three randomly interleaved test stimuli (0 ms ISI). Two of the test stimuli were identical to the adapting stimuli (Fig. 3C, A and B). The third test stimulus was a plaid composed of a horizontal component and a vertical component, but only the vertical component moved for the purposes of nulling (Fig. 3C, C). Each test stimulus component was presented at 3% contrast, meaning the peak contrast for the plaid test stimuli (Fig. 3C, B and C) was 6%. Other methods were the same as in the nulling experiments described earlier.

In a follow-up experiment (Fig. 3D), two high-contrast adapting stimuli were randomly interleaved: a horizontally moving vertical grating and an obliquely moving plaid composed of a horizontal and a vertical component. Note that, whereas the plaid's global (i.e., pattern) motion was in one of four oblique directions, both adapting stimuli had a component moving in the horizontal direction. The contrast of each component was 48%, and their speed was 15°/s. In each case, the low-contrast (3%) test stimulus was a vertical grating, and the nulling method was used to estimate the resultant MAE.

Neurophysiological Methods.

Animal preparation.

Two rhesus macaque monkeys underwent a sterile surgical procedure to implant a headpost and recording cylinder. Following recovery, monkeys were seated comfortably in a primate chair (Crist) and trained to fixate a small red spot on a computer monitor in return for a liquid reward. Eye position was monitored at 200 Hz with an infrared camera (SR Research), and was required to be within 2° of the fixation point. All aspects of the experiments were approved by the animal care committee of the Montreal Neurological Institute, and were in compliance with regulations established by the Canadian Council on Animal Care.

We recorded from 106 well isolated single neurons in area MT. Single waveforms were sorted online and then resorted offline by using spike-sorting software (Plexon). Area MT was identified based on anatomical MRI scans, the prevalence of direction-selective neurons, and the correlation between receptive field size and eccentricity. Whe a neuron had been isolated, we determined its optimal SF and stimulus position manually. We then obtained a direction tuning measurement with a grating stimulus of optimal SF (duration of 500 ms). Measurements of preferred speed and size were then obtained by using stimuli moving in the cell's preferred direction. Preferred speeds were generally between 8 and 16°/s. For the main experiment, we tested each cell with grating stimuli moving at the preferred speed and in the preferred or null directions. Each direction was shown 25 times each in a pseudorandom order.

Visual stimuli.

Stimuli were displayed at 75 Hz at a 1,920 × 1,200 resolution. The viewing area subtended 70° × 42° at a distance of 42 cm. Stimuli consisted of grating patches displayed on a gray background (70.3 cd/m²) in the center of the receptive field. Receptive field eccentricities ranged from 2° to 15° (median, 5.1°). In most cases, the stimulus size was fixed at 25°. A subpopulation of cells was tested with smaller moving stimuli matched to the receptive field size (Fig. 6A). The motion stimulus consisted of five motion frames in the preferred or null direction (duration, 67 ms). Stimulus contrast was 99%. In the 0-ms ISI condition, the motion stimulus was followed by a stationary stimulus of 3% or 99% contrast that was presented for 333 ms. In the 150 ms ISI condition, the motion stimulus was separated from the stationary stimulus by a 147-ms gap.

Data analysis and model.

For each cell, spike times from each trial were filtered by a leaky integrator (Fig. 5 A and B). Specifically, the spike trains were represented as sums of δ functions and convolved with an exponential kernel with a time constant of 50 ms. The choice of this time constant was based on results demonstrating that leaky integration with time constants between 30 and 70 ms adequately explains both neural and behavioral responses to brief motion signals (32, 34). Additional analysis conducted without leaky integration is shown in Fig. S1.

The spike rate functions were then subjected to ROC analysis (33). The degree of direction selectivity of a cell's response was expressed as the aROC, with 1 indicating preferred direction selectivity and 0 indicating null direction selectivity. The variability of these measurements was estimated by a resampling analysis. From this analysis, we estimated a distribution of times at which each cell would be expected to dip two SDs below 0.5 by chance alone (i.e., have significant null direction selectivity with a one tailed P < 0.025) during a 500-ms time period starting 80 ms after the onset of the stationary test. This distribution was used to calculate P values for the analysis shown in Fig. 5E.

The strength of surround suppression was quantified from the size–tuning curve as SI = 1 – L/B, where L is the mean response to the largest stimulus (usually 25°) and B is the best mean response obtained from any stimulus size. The SI therefore ranges from 0 for cells with no spatial suppression to 1 for cells with complete spatial suppression. For the correlation analysis associated with Fig. 6B, the magnitude of direction selectivity to the static test stimulus was defined as the difference between 0.5 (i.e., chance-level direction discrimination) and the minimum aROC. The duration of the effect was defined as the total time under 0.5 starting at 80 ms after the onset of the stationary test stimulus.