The motion after-effect reloaded

Abstract

The motion after-effect is a robust illusion of visual motion resulting from exposure to a moving pattern. There is a widely accepted explanation of it in terms of changes in the response of cortical direction-selective neurons. Research has distinguished several variants of the effect. Converging recent evidence from different experimental techniques (psychophysics, single-unit recording, brain imaging, transcranial magnetic stimulation, and evoked potentials) reveals that adaptation is not confined to one or even two cortical areas, but involves up to five different sites, reflecting the multiple levels of processing involved in visual motion analysis. A tentative motion processing framework is described, based on motion after-effect research. Recent ideas on the function of adaptation see it as a form of gain control that maximises the efficiency of information transmission.

Introduction

After prolonged adaptation to a visual scene moving in a certain direction, observation of a stationary scene evokes an experience of motion in the opposite direction. This ancient perceptual effect, called the motion after-effect (MAE) [1, 2], is easy to generate and very robust. Research on the MAE has played a crucial role in the development of theories relating motion perception to neural activity in the brain. Sutherland [4] was the first to suggest a simple neural explanation of the MAE inspired by Hubel and Wiesel’s [3] discovery of direction-selective cortical cells in the cat:

“…the direction in which something is seen to move might depend on the ratios of firing in cells sensitive to movement in different directions, and after prolonged movement in one direction a stationary image would produce less firing in the cells which had just been stimulated than normally, hence movement in the opposite direction would be seen to occur” (p.227)

In 1963 Barlow and Hill [5] reported adaptation-induced changes in responsiveness in single cells in the rabbit retina, and Sutherland’s ratio account of the effect gained wide acceptance. Later discoveries of adaptation effects in cat and primate cortex encouraged the general view that the origin of the MAE was probably adaptation in motion-selective cells in primary visual cortex. The essential principle of population coding in the MAE is still universally accepted, but discoveries made possible with the introduction of new experimental techniques indicate that major changes to theoretical explanations of the MAE are required. These discoveries include work in human psychophysics [6-23], primate physiology [24-27], human neuroimaging [28-37], human electrophysiology (Visual Evoked Potentials - VEPs) [38-42] and transcranial stimulation [43-45]. Results indicate that the MAE is an amalgam of neural adaptation at several visual cortical sites. This short review offers a fresh appraisal of the MAE and its neural basis, based on this recent research.

Psychophysical evidence: How many after-effects?

The classical MAE seen in natural viewing conditions involves a static test pattern; after one observes movement for a while, such as a waterfall or the view from a moving vehicle, subsequently viewed stationary objects appear to move. We shall refer to this effect as the static MAE or SMAE. In the late twentieth century laboratory researchers began using dynamic test patterns such as dynamic visual noise or counter-phase flicker to study the after-effects of motion adaptation. A dynamic visual noise (DVN) pattern contains a dense field of randomly positioned dots which are replaced by a completely new set of random dots at pre-defined time intervals, typically up to 100 times every second. DVN has the appearance of a de-tuned television display. Counter-phase flicker is created by reversing the contrast of a luminance sine-wave grating repetitively - black bars become white and white bars become black - at a pre-defined frequency (exactly the same effect can be created by spatially superimposing two gratings drifting in opposite directions). The properties of MAEs obtained using these dynamic test patterns, which we shall call dynamic motion after-effects (DMAE), were markedly different from those obtained using stationary patterns, and led to the conclusion that the two after-effects were mediated by different populations of cells. The contrasting effects produced by first-order motion and second-order motion were particularly important. First-order motion involves patterns defined by variations in the luminance of single image points, such as drifting luminance gratings or dot patterns. Second-order patterns contain features defined by variations in the luminance of pairs of image points, such as variation in texture contrast, size, orientation, or binocular disparity. In moving second-order patterns the texture elements defining the pattern are usually replaced by new texture in each animation frame, so the pattern does not contain point-by-point correspondences over time. Adaptation to second-order motion does not produce a SMAE, but it does produce a DMAE [15, 16]. Furthermore, first-order and second order adapting patterns differ in terms of their inter-ocular transfer. It has long been known that when the adapting stimulus is presented to only one eye, and the test stimulus to the alternate eye, an after-effect is still reported. This inter-ocular transfer (IOT) relates to the binocularity of the underlying visual neurons. The SMAE shows only partial interocular transfer (IOT) [13], indicating that at least some of the cells involved are monocular, but the DMAE shows complete IOT [15], indicating that all the cells involved are binocular. These and other results led to the idea that the SMAE reflects adaptation in lower level, first-order motion sensors, while the DMAE reflects adaptation in higher level second-order sensors [15, 29].

Recent research reveals that the distinction between SMAEs and DMAEs is not as simple as was once believed. Early studies of DMAEs tended to use dynamic patterns that changed at a relatively slow rate or temporal frequency. For example, Nishida and colleagues [15, 16] used DVN at a frequency of 2 Hz (each pattern was replaced twice each second), and their DMAE from adaptation to second-order motion decreased progressively at higher temporal frequencies. More recent studies have measured DMAEs using first-order adapting stimuli and much higher frequency dynamic test patterns. Verstraten and colleagues [21, 22, 23] used test patterns that changed at rates of between 10 and 90 Hz. Some of their experiments [22, 23] involved adaptation to two transparently moving sets of dots, one at high velocity and the other at low velocity. In these stimuli two sets of dots drifting in different directions are spatially superimposed, and can be seen passing through each other. The direction of the resulting after-effect depended on the temporal properties of the test; stationary tests appeared to move in the direction opposite to that of the slow adapting stimulus, and dynamic tests flickering at 90 Hz appeared to move in the direction opposite to the faster adapting stimulus. Other recent work using dynamic test patterns [6, 48] has concluded that two low-level populations of motion sensitive cell are involved in motion after-effects, one maximally sensitive to flicker at 2 Hz and the other maximally sensitive at 8 Hz or higher (Figure 1).

Figure 1.

Motion after-effect duration as a function of the temporal frequency of the test pattern (abscissa) and the speed of the adapting stimulus (different plot symbols). Results are shown for four subjects. For the slowest adapting speed (2.3 deg/sec, squares), MAE duration is maximal for stationary tests and absent at the highest test temporal frequency; for the fastest adapting speed (36.8 deg/sec, circles), the MAE is absent for stationary tests and maximal at the highest test temporal frequency (taken from [6]).

At least three populations of cell are required to explain the diverse empirical properties of the after-effects reviewed so far. One low-level population mediates the classical SMAEs from first-order adaptation seen using static test patterns, and perhaps DMAEs seen in very low temporal frequency dynamic test patterns. A second low-level population mediates DMAEs from adaptation to rapid first-order motion seen using high temporal frequency test patterns. A third, ‘higher level’ population mediates DMAEs from second-order motion seen using low temporal frequency test patterns. A corollary of this conclusion is that DMAEs do not tap a single population of cells, but different populations depending on the properties of the adapting and test stimuli.

Recent psychophysical results have implicated two further sites of adaptation in motion after-effects. A number of studies have found evidence for adaptation at a relatively late stage in the motion pathway, where global movements such as rotation and expansion are computed. After-effects have been reported using adapting and test patterns of varying complexity [8, 10, 17]. Bex et al. [8], for example, found that adaptation to radial and rotational patterns produced stronger MAEs than adaptation to translating patterns. Several papers report so-called ‘phantom’ MAEs, which appear when the test stimulus is projected onto a region of the retina that was not exposed to the adapting stimulus, and that did not appear to contain motion during adaptation [eg. 49, 50, 51]. Meng et al. [50], for instance, found phantom MAEs only when the adapting pattern contained radial expansion rather than translation. The presumed cortical location of phantom after-effects is MT or MST, where receptive fields are very large and sensitive to large-scale rotary or radial motion.

Culham et al. [11] have argued that apparent motion mediated by attentional tracking can also generate an after-effect. During adaptation subjects viewed an ambiguous counter-phase grating, and were instructed to “…use attention to mentally track the bars of a radial grating in one of the two ambiguous directions…”. Tests on a static pattern showed no after-effect, but tests on a 2 Hz counter-phase grating did reveal an after-effect. Culham et al. [11] argued that their DMAE from attentive tracking arose in relatively late cortical areas, perhaps MST. Their adapting stimulus offered equal and opposite signals for motion sensors, so it is possible that attention served to modulate these signals rather than generate its own motion signal. The fact that their effects were confined to DMAEs may indicate the site at which the attentional modulation occurred.

So far the psychophysics indicates that up to five populations of cells all potentially contribute to motion after-effects. Are these populations functionally distinct? Do they occupy different cortical locations? Perhaps recent electrophysiological and brain imaging can clarify these fundamental questions.

Physiological evidence: How many sites?

Single-unit recordings

Important recent studies by Kohn and Movshon [24, 25] measured adaptation-induced changes in the response of direction-selective cells in macaque MT (previously reported in [26, 27]). One of their aims was to determine whether adaptation effects occur at the level of MT, or are inherited in responses fed forward from V1 cells. In the latter case, the spatial extent of adaptation in MT should be limited by the smaller size of receptive fields in V1. Kohn and Movshon [24] did indeed find spatially specific adaptation within MT receptive fields, consistent with adaptation ascending from V1 (Figure 2). Other results reported by Kohn and Movshon [24] suggest a particular role for MT responses in DMAEs. They found that adaptation to the null direction of an individual MT cell (opposite to its preferred direction) enhanced its response to a balanced counter-phase flickering grating, a neural correlate of the DMAE. Adaptation apparently weakened the opponent input to the MT cell, allowing an enhanced response to motion balanced stimuli. Kohn and Movshon [24] do not rule out the possibility that adaptation can also occur in MT neurons themselves.

Figure 2.

Contrast response functions of an MT neuron measured before adaptation (open symbols) and after adaptation (filled symbols). The inset in each graph shows the spatial arrangement of adapting and test stimuli in the cell’s receptive field (dotted lines). When adapting and test locations overlapped (A and D), the cell’s response was strongly reduced; when the adapting and test locations differed (B and C), response was largely unaffected by adaptation. (taken from [24]).

Human brain imaging

Results from recent fMRI studies of human motion processing support a functional distinction between at least two populations of motion sensors, responsive respectively to first- and second-order motion, but these populations do not appear to occupy anatomically segregated locations. Ashida et al. [28], for instance, employed a fMRI adaptation paradigm: When repeated presentation of similar stimuli reduced the blood oxygen level dependent (BOLD) response, their inference was that the change in response (fMRI adaptation) reflected changes in the responsiveness of cortical cells activated all stimuli; when there was little or no reduction in BOLD response they inferred that different cells were activated by the different stimuli. Using this technique they found evidence for separate populations of cells sensitive to first-order and second-order motion in several visual areas including V3A, MT and MST (technical issues may have prevented adequate examination of responses in V1). Nishida et al. [34] and Seiffert et al. [35] had previously found fMRI responses to both first-order and second-order motion in a number of visual areas including V1, V2, V3, VP, V3A, V4v, and MT.

These data are important because they are not consistent with early brain imaging studies of the MAE which implicated area MT so strongly [37]. Indeed other recent imaging studies have disputed the primacy of MT in MAEs [30, 32], and indicate that a number of brain areas are activated during the perception of MAEs. In Taylor et al.’s [36] study subjects were adapted for 21 seconds to drifting bars or to reversing bars (the control condition). Immediately after adaptation stationary bars were presented for 21 seconds, and subjects were instructed to press a button once the subjective experience of a SMAE ceased. During perception of the SMAE, significant activation was indeed found in V5/MT, but also in a network of posterior and anterior cortical sites (Figure 3). In particular there was consistent activation in the anterior cingulate gyrus, BA47 and BA40. Taylor and colleagues [36] argued that these brain regions could be candidates for mediating awareness of the MAE. Correlation analysis showed that two different neural networks are involved in the MAE; a posterior network mainly involved in motion analysis and an anterior network involved in the experience of the MAE. The posterior network includes V1, V2, V3 and V5/MT; the anterior network includes BA37, BA40, BA44, BA46, BA47 and cingulate gyrus (CG). As shown in Figure 3, the most posterior region of the anterior network is the BA37. There is evidence that this area belongs to the anterior network and is anatomically and functionally distinct from area V5/MT. So BA37 may be considered a bridge between the anterior and the posterior network. The joint activity of the two networks may constitute the neural basis of MAE perception.

Figure 3.

The posterior and anterior neural networks active during the perception of MAE; the connections between sites are derived from the correlation coefficients of the activation time courses. The lines join cortical sites which have cross correlations of at least 0.5. (Adapted from [36]).

The same message emerges from Hautzel et al. [31], who measured regional cerebral blood flow (rCBF) with positron emission tomography (PET) during MAEs. They found increased rCBF in areas V2, V3a and V5/MT. In addition, when subjects perceived the MAE an increase in rCBF was also seen in the lateral parietal cortex (BA40) predominantly on the right side, in the right dorsolateral prefrontal cortex (DLPFC), in the anterior cingulate and in the left cerebellum. These results are in agreement with those of Taylor et al. [36], and provide evidence of activation in multiple cortical areas during perception of the MAE. The increased rCBF in BA40 and DLPFC may represent activation of cognitive functions during perception of the MAE, such as alertness, attention and working memory.

The participation of attentional networks raises the question of whether activity detected in visual areas such as V5/MT may actually reflect attention to the MAE. Huk and colleagues [33] addressed this question. In previous imaging studies of the MAE there was no control of attentional state, so subjects were free to allocate and shift attention differentially between MAE and control conditions, and attention could have enhanced the V5/MT response. Huk et al. [33] initially replicated previous findings using a paradigm similar to that of He et al. [32]: there was a larger increase in V5/MT response in the MAE condition (after adapting to unidirectional motion) than in the control condition (adaptation to motion reversal at 2 Hz). To equate attention in both MAE and control conditions, subjects had to perform a sequence of 2-alternative forced-choice speed discriminations during 5-second test period. Results showed no difference in BOLD signal between MAE and control conditions.

Huk et al. [33] performed another experiment to test direction-selective adaptation. (adaptation is direction-selective if it larger for test stimuli moving in the same direction as the adapting stimulus than for test stimuli moving in the opposite direction.) Subjects adapted to a given motion direction and then viewed test stimuli moving either in the adapted direction or in the opposite direction. Direction-selective adaptation was observed in V1, V2, and V5/MT. Thus there are no grounds for claiming that V5/MT has any unique status in terms of MAE locus.

Human Transcranial Stimulation studies

Stewart and colleagues [43] were the first to succeed in reducing the duration of SMAE (but not of the colour after-effect) with magnetic stimulation over V5/MT, indicating a role for MT in the SMAE. Théoret et al. [45] applied rTMS over V5/MT during a storage period in between MAE adaptation and testing. Stimulation shortened the duration of the subsequent MAE, compared to the No-TMS condition. There was little effect of stimulation to V1 on storage. In a second experiment rTMS was applied to MT during perception of the MAE, and was found to reduce MAE duration, but stimulation to DLPFC and to posterior parietal cortex had no effect on after-effect duration. It should be noted, however, that Théoret et al. [45] used complex radial/rotational stimuli. Therefore, it is not surprising that they found a specific involvement of the MT complex. Antal et al. [44] used a relatively new technique, namely transcranial direct current stimulation (tDCS), to explore the role of MT in MAEs. They found that stimulation of V5/MT significantly decreased MAE duration. Stimulation of V1 did not affect MAE duration, though the authors admit that its retinotopic organization may have spared the relevant region of cortex from disruption. Overall, TMS and tDCS studies clearly implicate MT in the MAE, but the role of other areas is still unclear.

Visual Evoked Potentials (VEPs) and magnetoencephalography (MEG)

Which components of the Visual Evoked Potential (VEP) reflect activity related specifically to the MAE? Human electrophysiological studies have shown that the amplitude of a negativity peak at about 200 ms (N2) is affected by motion adaptation [39], but it is not clear whether this effect is direction selective. More recently, Kobayashi et al. [40] found a significant bilateral increase of a positive component at about 160 ms (P160) in the occipitotemporal region after motion adaptation. They also observed a laterally biased effect in the right posterior temporal region, perhaps related to the engagement of attentional circuits.

Neural gamma-band activity (GBA, high-frequency neural activity in the range 40-100 Hz) appears to be associated with synchronization among different brain regions, which is thought to be important for visual feature binding and motion perception (Rose and Büchel, 2005). Tikhonov et al., (2007) investigated GBA associated with the MAE using magnetoencephalography (MEG; a non-invasive technique used to measure magnetic fields generated by the electrical activity in neurons). Adaptation and test patterns were presented either in the right or left visual hemifield. Results showed GBA reflecting the MAE in channels over two locations (indicating the presence of two dipole sources), providing evidence that MAE depends on the synchronization of different brain regions. The first location was in parieto-occipital cortex (in the region of area MT). The second location was more posterior, but it was not possible precisely to locate the origin of this source. Two possibilities discussed were striate cortex and the cerebellum.

Conclusions

Figure 4 is a simple functional diagram which attempts to summarise the main stages of visual motion processing from the perspective of the motion after-effect research reviewed here. Motion sensors in the earliest cortical areas (V1, V2, V3) feed into a computation underlying the perception of ‘static’ and also into a local motion integration stage. First-order motion sensors tuned to slow velocities contribute to ”static’ computations, while first-order sensors tuned to higher velocities, and second order sensors both feed into motion integration. As explained in Box 2, adaptation in sensors contributing to the static computation leads to the SMAE; adaptation in sensors contributing to motion integration leads to a DMAE, probably in area MT. Adaptation of cells involved in computation of optic flow in area MST mediates phantom MAEs. Attention-mediated motion after-effects and subjective awareness of motion after-effects involve more anterior cortical areas such as parietal and cingulate cortex. The varieties of motion after-effect reviewed here tend to be regarded as a cognate group of effects but this framework highlights the fact that, in functional terms, they are distinct and separate. The SMAE, for example, involves an entirely different population of neurons and separate computations from those involved in phantom MAEs, in the same way that the tilt after-effect involves different processes from those underlying the size after-effect.

Figure 4.

Functional diagram relating the main stages of motion processing in the human brain to MAE adaptation sites. SMAEs are mediated by motion sensors that contribute to computation of ‘static’, while DMAEs are mediated by motion sensors that contribute to motion integration computations. Phantom MAEs involve sensors contributing to the computation of optic flow.

QUESTIONS: BOX 1.

Several important issues remain to be resolved, including:

• -How do the different adaptation components combine to determine the observed after-effect?
• -Why do static and dynamic tests tap different adaptation processes (see Box 2)?
• -Can attention generate its own after-effect, or does it only modulate lower level adaptation?
• -What is the functional significance of multiple adaptation sites (see Box 3)?

BOX 2.

What causes difference between SMAEs and DMAEs?

On the basis of empirical differences between the after-effects, a commonly held view is that static tests tap ‘low-level’ processes, and dynamic tests tap ‘high-level’ processes. Does recent research allow us to be more specific? One can infer from Kohn and Movshon’s [24] single-unit recording study that the SMAE reflects adaptation-induced changes in the response of first-stage cells in V1, while the DMAE reflects changes in the response of second-stage MT neurons that receive opponent inputs from V1 cells. The psychophysical study of Morgan et al. [12] concurs with this inference. They argue that SMAEs represent a shift in the population response of cells tuned to relatively slow velocities (we see ‘static’ when the outputs of low-velocity units are balanced), while DMAEs arise from adaptation-induced dis-inhibition in higher level cells tuned to higher velocities (see also Ref. 23). Why do some adapting stimuli, such as second-order patterns, only produce after-effects on dynamic tests? Perhaps the cells activated by such stimuli are not themselves involved in the computation of ‘static’, so an adaptation-induced imbalance does not lead to a SMAE, but their responses nevertheless feed forward to influence later motion computations and cause DMAEs.

BOX 3.

What is the mechanism and function of adaptation?

Adaptation can be viewed as a form of automatic gain control, in which a unit attenuates its own response to continuous intense stimulation. Van de Grind et al. [18, 19] developed a detailed computational model of MAE adaptation that uses divisive feed-forward inhibition as the gain control mechanism. During adaptation a leaky integrator in the gain control circuit of active cells charges up, and during testing the charge leaks out to cause an imbalance in output between adapted and unadapted units. The results of psychophysical experiments conducted by van de Grind and colleagues were consistent with their model. Other psychophysical [12] and electrophysiological [24] results are also consistent with this kind of gain control mechanism.

What function does gain control serve, and why should it be present at several different sites in the processing hierarchy? One idea is that gain control improves the efficiency of encoding by striving to maximize the information about the stimulus conveyed in neural spike trains [46]. Visual motion analysis involves the transmission of information between multiple processing stages. Gain control at each stage of transmission should serve to optimize the efficiency of coding in the system as a whole. According to this view there should be as many sites of adaptation for MAEs are there are processing stages in the motion analyzing system. MAE research indicates that adaptation at higher level sites is weaker than adaptation at lower level sites, because the after-effects tend to be short-lived. Why should this be so? The persistence of adaptation can be viewed as a prediction by the visual system that the current pattern of stimulation will continue into the future [47]. Perhaps the complex motions signaled at high-level sites are less predictable and persistent than the simpler motions signaled at lower levels, and the difference in adaptation strength is an expression of this difference in predictability.

This emerging framework is very much a work-in-progress, and a number of issues remain to be resolved – see the boxes.