Extra-retinal adaptation of cortical motion-processing areas during pursuit eye movements

Abstract

Repetitive eye movement produces a compelling motion aftereffect (MAE). One mechanism thought to contribute to the illusory movement is an extra-retinal motion signal generated after adaptation. However, extra-retinal signals are also generated during pursuit. They modulate activity within cortical motion-processing area MST, helping transform retinal motion into motion in the world during an eye movement. Given the evidence that MST plays a key role in generating MAE, it may also become indirectly adapted by prolonged pursuit. To differentiate between these two extra-retinal mechanisms we examined storage of the MAE across a period of darkness. In one condition observers were told to stare at a moving pattern, an instruction that induces a more reflexive type of eye movement. In another they were told to deliberately pursue it. We found equally long MAEs when testing immediately after adaptation but not when the test was delayed by 40 s. In the case of the reflexive eye movement the delay almost completely extinguished the MAE, whereas the illusory motion following pursuit remained intact. This suggests pursuit adapts cortical motion-processing areas whereas unintentional eye movement does not. A second experiment showed that cortical mechanisms cannot be the sole determinant of pursuit-induced MAE. Following oblique pursuit, we found MAE direction changes from oblique to vertical. Perceived MAE direction appears to be influenced by a subcortical mechanism as well, one based on the relative recovery rate of horizontal and vertical eye-movement processes recruited during oblique pursuit.

1. Introduction

The motion aftereffect (MAE) refers to the illusory motion produced by prolonged viewing of a moving stimulus. The MAE is typically thought to result from the direct adaptation of mechanisms sensitive to retinal image movement (Anstis et al. 1998). However, when the eye moves to track the adapting motion, the adaptor becomes stabilized on the retina and so any image motion corresponding to the moving target is either eliminated or drastically reduced. Despite this the MAE remain intact. In some cases the illusory motion produced by eye-movement adaptation is thought to result from adapting more peripheral motion sensors that are stimulated as the eye sweeps past objects visible elsewhere in the scene. Once adapted, they produce MAE in the central region by a process of motion induction (Morgan et al. 1976; Wade et al. 1996). However, when this type of peripheral adaptation is removed or carefully controlled for, the illusory motion persists (Chaudhuri 1990a,b; Chaudhuri 1991a,b; Freeman et al. 2003). This implies another route to the MAE, one that does not rely on direct retinal stimulation and one that remains little understood.

This second route could be related to the way adaptation affects the oculomotor system itself. One idea involves the suppression of residual motor activity known as afternystagmus. If observers are placed in complete darkness following the adaptation phase, one in which no stationary test stimulus is visible and so no MAE experienced, the repetitive eye movements persist for some time albeit with a smaller overall amplitude (Cohen et al. 1977; Chaudhuri 1991a). Conversely, if the stationary test is visible following adaptation, observers are able to suppress the afternystagmus and fixate the test accurately. The MAE experienced in the latter case may therefore be linked to the act of suppression, generating an erroneous motion signal that gives rise to illusory movement (Chaudhuri 1990a,b). Of course, moving the eyes involves considerable neural machinery, so there are potentially many other neural sites that could be affected by prolonged eye movement. The possibility we entertain here is that repeated eye movement also adapts cortical motion sensors responsible for integrating retinal movement with the motion of the eye.

The true motion of objects in the world is often concealed by an eye movement—a pursued target is stationary on the retina while the stationary background moves. Evidence suggests the visual system is able to compare the retinal motion it senses with the actual eye movement made (Wertheim 1994; Haarmeier et al. 1997; Freeman & Banks 1998; Freeman 2001; Haarmeier et al. 2001; Turano & Massof 2001). Eye-movement information is most likely carried by an extra-retinal signal based on the eye-muscle commands (von Holst 1954). Area MST, part of the primary motion pathway located in the parietal cortex, has been identified as a possible place for this integration, with subpopulations of cells responsive not just to retinal motion but also the motion of the eye and head (Newsome et al. 1988; Thier & Erickson 1992; Snowden 1994; Ilg et al. 2004). MST is therefore not only sensitive to some of the retinal consequences of self-motion but also the effectors that are responsible for it. Crucially, the motion complex in which MST resides is also known to play a key role in MAE (Tootell et al. 1995; Snowden & Milne 1997; Culham et al. 1999; Huk et al. 2001; Theoret et al. 2002; van Wezel & Britten 2002; Kohn & Movshon 2004). Given that signals from the eye-movement system arrive at MST, prolonged eye movement may therefore indirectly adapt the cortical machinery involved in interpreting the consequence of activity.

We tested this hypothesis by examining the ability of MAE following eye movement to survive a period of darkness. MAE induced by retinal motion exhibits storage in this manner and suggests that adaptation does more than simply fatigue motion detectors (Thompson & Wright 1994; Anstis et al. 1998; Clifford 2002). Recent evidence from functional magnetic resonance imaging and transcranial magnetic stimulation indicate that the motion complex in which MST resides mediates storage (Culham et al. 1999; Theoret et al. 2002). Hence MAE based on extra-retinal stimulation of this area should also exhibit storage. Crucially, MAE based on the suppression of post-adaptation motor signals should not do so because afternystagmus dissipates in the dark. In our experiments we found evidence to support both mechanisms, the key factor being the type of adapting eye movement made. When adapting to deliberate pursuit, the subsequent MAE stored. When adapting to reflexive and unintentional eye movement, it did not.

In a second experiment we found that adaptation of cortical motion-processing areas cannot be the only factor contributing to pursuit-induced MAE. Specifically, we show that MAE direction following diagonal pursuit changes from oblique to vertical. Observers who exhibited the effect also showed longer MAE duration following vertical pursuit compared to horizontal. These results suggest that perceived MAE direction is influenced by the relative rate of recovery of horizontal and vertical eye-movement mechanisms, arguably a process quite removed from the cortical processing of motion.

2. Material and methods

All stimuli were shown through a Sony Multiscan projector (VPH 1272QM) and controlled by a VSG 2/5 graphics card housed in a PC. They were rear-projected on to a large screen at a refresh rate of 72 Hz. Viewing was from a distance of 114 cm, using a head constraint comprising a chin-and-cheek rest. Data collection occurred in a completely darkened room. Observers were undergraduate Psychology students at Cardiff University. Each received course credit for participation.

(a) Storage experiment

Stimuli comprised two vertical strips of randomly positioned dim dots (density ∼0.5 dots/deg²) displayed on the black background of a large projection screen (see inset to figure 1b). The background luminance measured 0.001 cd/m² and light scatter from the sparse dot pattern was too low to illuminate any peripheral objects in the dark laboratory. Strips were 40.5° high and 13.5° deg wide and separated by 10°. Both moved upward at 20.25 °/s. Dots disappearing at the top of each strip reappeared at the bottom. At the end of 60 s the pattern was replaced by a stationary dot, cued by a beep and delayed by either 1 or 40 s of darkness. Observers indicated the time it took for the MAE to cease using a mouse press. They also indicated perceived direction as a check on how well they understood the instructions. Trials in which no MAE was seen were scored as 0 s duration.

Figure 1.

(a) Eye movements made when instructed to deliberately pursue the adapting stimulus (upper panel) or stare at it (lower panel). The histograms plot the frequency of the slow-phase durations (the time between individual saccades). (b) MAE duration following deliberate pursuit (closed symbols) or reflexive nystagmus (open symbols). Error bars are ±1 s.e.

Each session consisted of six adaptation trials, three with the long and three with the short delay. These were presented in a counterbalanced order. Observers undertook two sessions. In one they were told to stare at the central blank strip of the pattern and not to try and deliberately follow the motion. In the other they were told to deliberately pursue the stimulus, keeping their eyes within the central blank. Sessions were carried out in counterbalanced order. Prior to each, observers were light-adapted for 40 s using a bright homogenous field (∼23.5 cd/m²) followed by 20 s of darkness. Together with the black background and dark lab, this procedure rendered all other objects in the room invisible to the observer.

Eye movements were recorded using an ASL Series 4000 video-based eye tracker, sampling at 50 Hz. Velocity and acceleration profiles were computed by differentiating with respect to time after first low-pass filtering the position record. Saccades were then identified as zero crossing in the acceleration profile, so long as eye speed exceeded a fixed velocity threshold. Samples 110 ms either side of the zero-crossing peak were excluded, leaving a trace consisting of the slow-phase of eye movement only. Eye-movement accuracy was computed by averaging all samples over the slow phases. Slow-phase duration was computed as the length of time between saccades.

(b) Direction experiment

Pursuit eye movements were made to a small target moving for 60 one-second sweeps each over a distance of 12°. All the rest of the display was black. In the control condition the target moved in triangle-wave fashion, alternating smoothly back and forth at the fixed speed. In the main adaptation conditions, the target moved smoothly in a given direction and then abruptly returned to the beginning of the sweep. In the diagonal conditions the target moved along an oblique path oriented at 45 or 135 °. In the four other conditions, the target moved horizontally or vertically, the direction coinciding with the horizontal and vertical components present during diagonal adaptation (see inset to figure 2c).

Figure 2.

(a) Sample eye-movement recordings for one observer. The left column shows the horizontal component and the right the vertical. Black traces correspond to adaptation period and red the test phase. Vertical spikes correspond to eye blinks. (b) Mean MAE durations. Error bars are +1 s.e. (c) Final MAE direction in the diagonal-adaptation conditions, plotted as a function of the ratio of MAE duration in the vertical and horizontal adaptation conditions. (d) Final MAE direction in the diagonal conditions, plotted as a function of the ratio between vertical to horizontal component eye-movement speeds during diagonal pursuit.

At the end of the final sweep a stationary dot appeared 1 s after the moving target had disappeared, accompanied by a beep. Observers fixated this and adjusted an unseen rod attached to a table in front of them to indicate perceived direction throughout the duration of the MAE. Observers also indicated cessation of illusory motion using a mouse click. Each condition was repeated three times, making a total of 12 trials per observer displayed in a randomized blocks. Observers were assigned to different adaptation groups, completing the appropriate control and three adaptation conditions represented by either the filled or open symbols in figure 2. Direction and duration data were averaged over the appropriate three replications. Prior to data collection, rod settings were calibrated against the perceived direction of real motion using a small target that moved (sawtooth fashion) at 2 °/s for three one-second sweeps. Nine different directions were recorded and two replications of the calibration setting made at each. These were then fitted using a third-order polynomial. The result was used to transform all subsequent rod settings into perceived-direction scores. Eye movements were recorded and analysed in similar fashion to experiment 1.

3. Results

(a) Storage experiment

Figure 1a summarises the eye movements made when observers were instructed to either deliberately pursue the adaptation stimulus (top panel) or simply stare at it (bottom panel). In both cases the eye movement consists of a slow-phase ocular-following response alternated with a rapid, fast-phase saccade, as exemplified by the sample recordings shown in the insets. When told to pursue, the slow-phase consisted of more extensive, deliberate sweeps, interspersed with shorter, more reflexive eye movement. This is summarized by the two histograms, which plot the frequency of slow-phase durations for the two eye-movement styles. Both distributions peak at around 300 ms, with the deliberate eye movement producing the longer tail (Cheng & Outerbridge 1974). The absence of longer sweeps in the ‘stare’ condition is thought to result from an eye movement driven by more reflexive image stabilisation mechanisms, such as those involved in optokinetic nystamgus (Schor & Narayan 1981; van den Berg & Collewijn 1988; Ilg 1997).

Eye-movement style had a profound consequence for the type of storage exhibited by the subsequent MAE (figure 1b). Both eye-movement conditions produced the same duration of MAE following a brief 1 s delay between adaptation and test. However, a longer delay of 40 s in the dark virtually destroyed the MAE associated with the more reflexive eye movement, as we have shown before (Freeman et al. 2003). After a square-root transform to reduce skew in the data (Thompson & Wright 1994), the interaction between delay and eye-movement style approached significance (F(1,7)=5.161, p=.057), with the comparison at the longer delay significant (t(7)=3.04, p=.019). It is unlikely that this effect was mediated by the amount of incidental retinal motion created by different speeds of ocular following in the two conditions. To reiterate, peripheral objects in the lab were not visible. Moreover, the pursuit eye movement was more accurate than the reflexive one (mean eye-movement gains of 0.84 and 0.58, respectively), so on average the pursuit condition contained less retinal slip due to inaccurate ocular following. Despite this, the pursuit-induced MAE exhibited greater propensity to store.

(b) Direction experiment

Figure 2a provides sample recordings of the horizontal and vertical components of eye movement associated with the three main adaptation conditions (top rows) and the control (bottom row). Observers were able to pursue the target quite accurately. The mean pursuit gain was ∼0.78 for all adaptation conditions except downward, where pursuit gain is known to be lower (Rottach et al. 1996). There was no evidence of post-adaptation afternystagmus. Thus, observers were able to maintain accurate fixation of the stationary test dot, as shown by the red part of the recordings. Any MAE experienced could not have been due to actual movement of the eye during the test phase.

The control condition produced few reports of MAE. Observers occasionally experienced autokinesis, the perceptual jitter that occurs without adaptation. Duration measurements in this condition therefore allowed us to estimate the length of time it took for an observer to discriminate between the consequences of adaptation and the autokinetic effect. In the main adaptation conditions, observers initially reported MAE in a direction opposite to the adapting pursuit. The illusory motion lasted over twice as long as the durations reported in the baseline condition (figure 2b). More surprisingly, when the adapting eye movement was along either diagonal, some observers reported MAE that changed direction from oblique to vertical. These observers also reported different MAE durations for the horizontal and vertical conditions, with the MAE in the vertical condition typically longer. Conversely, observers who always saw oblique MAE did not exhibit this difference. The ratio of vertical and horizontal MAE durations was thus able to predict the final direction seen in the diagonal condition extremely well, as shown in figure 2c.

Biases in eye-movement accuracy cannot explain the findings. Higher speeds were mostly associated with the horizontal component of eye movement during diagonal adaptation, even in observers who reported the final MAE direction as vertical. Thus the ratios of vertical to horizontal eye velocity during diagonal pursuit were mostly less than 1 (figure 2d). The recorded eye movements suggest that there was greater retinal slip vertically during adaptation, but it is unlikely that the retinal slip could explain the perceived-direction data. The target was pursued in the dark, thus limiting retinal motion to the retinal slip created between eye and target alone. It is unlikely that this could produce MAE, especially as the retinal slip was inconsistently located and rarely stimulated the fovea, the region where the subsequent test stimulus was fixated. Nevertheless, we tried to induce MAE by viewing sawtooth target motion with stationary fixation but failed.

4. Discussion

Making an eye movement involves considerable neural machinery located at many sites within the central nervous system. The target's motion must be encoded, the current relationship between eye movement and target movement continuously monitored and information passed to the oculomotor control system in a form that can be used to drive the eye muscles. The eye-movement system uses computations based on retinal motion together with information based on both perceived and predicted motion to achieve these ends (Kowler 1989; Krauzlis & Stone 1999; Beutter & Stone 2000; Krauzlis & Adler 2001). The consequences of eye movement must also be accounted for, in particular the dislocation between retinal image motion and movement in the world. Eye-movement adaptation is therefore likely to modulate the output of mechanisms at many different stages.

In the first experiment we found evidence of storage but only when the adapting eye movement was deliberate. It is unlikely that this is because greater attention was being paid to the adaptor in this condition as this would act to considerably decrease the duration of the MAE (Chaudhuri 1990a,b). More probable is the idea that failure of reflexive eye movement to achieve strong storage reflects a strict demarcation of the type of eye-movement signals arriving at area MST. Reflexive eye movements depend on a phylogenetically older eye-movement system, one that aims to stabilize whole-field retinal motion as we move about the world (Post & Leibowitz 1985; Ilg 1997). The primary role of this type of eye movement is therefore to reduce the amount of motion in the retinal image, serving to physically offset the retinal effects of self-motion when it is made with respect to a stationary scene. In some respects, deliberate pursuit eye movements do the opposite. They evolved later, allowing us to pursue specific targets moving in the scene, as opposed to stabilizing the background over which they move. Pursuit therefore creates substantial image motion as the eye moves across the background. Hence knowing the velocity of pursuit is useful for two reasons. It not only allows the observer to estimate target motion but also provides a means for interpreting retinal motion adjacent to this. It is not surprising, therefore, that pursuit-related signals arrive at MST, an area thought to be involved in transforming retinal motion into other coordinate systems (Ilg et al. 2004). Conversely, information about reflexive eye movements can be discounted. This idea resembles key features in some models of motion perception during eye movement (Post & Leibowitz 1985; Wertheim 1994). It also tallies with claims that efferent signals relating to reflexive eye movements are fed directly into brainstem circuits (Ilg 1997). There may be a functional reason why MAE following prolonged reflexive eye movements does not store.

In the second experiment we found MAE direction following diagonal eye movement was not constant, quite unlike the perceived direction one would expect following adaptation to retinal motion. The final direction seen was predicted by the relative durations reported for separate vertical and horizontal adaptation conditions. The dependence on duration ratio suggests that MAE direction following diagonal pursuit is partly determined by the adaptation of processes located within the oculomotor system. To make an eye movement, stimulus motion must be mapped onto a coordinate system defined by the six ocular muscles, roughly aligned with horizontal, vertical and torsional axes of deviation. Recordings in the flocculus of the cerebellum show many cells tuned to horizontal and vertical directions (Krauzlis & Lisberger 1996). These cells therefore serve to decompose cortical motion information into the components required to drive the eye muscles. Both would be active during diagonal adaptation because diagonal pursuit recruits both horizontal and vertical eye-movement processes (Krauzlis & Lisberger 1996; Leung et al. 2000). The dependence of perceived direction on duration ratio (figure 2c) implies the horizontal process recovers more quickly following diagonal pursuit.

A key question is whether the perceived-direction findings indicate differential recovery from residual motor signals underpinning afternystagmus. According to the afternystagmus-suppression hypothesis, vertical afternystagmus should decay more slowly then horizontal to account for the perceived-direction data. Unfortunately, eye-movement recordings in cat (Kitima et al. 2001), monkey (Kroller & Behrens 1995) and human (Clement & Lathan 1991) suggest quite the opposite. Moreover, according to the results of experiment 1, pursuit-induced MAE stores. In a subsidiary experiment we confirmed this for the MAE following pursuit of the diagonally moving target (means of 11.94 and 11.91 s with and without a 30 s delay, N=9). Our results therefore suggest both cortical and subcortical mechanisms operate to produce MAE following pursuit.