The hierarchy of directional interactions in visual motion processing

Abstract

It is well known that context influences our perception of visual motion direction. For example, spatial and temporal context manipulations can be used to induce two well-known motion illusions: direction repulsion and the direction after-effect (DAE). Both result in inaccurate perception of direction when a moving pattern is either superimposed on (direction repulsion), or presented following adaptation to (DAE), another pattern moving in a different direction. Remarkable similarities in tuning characteristics suggest that common processes underlie the two illusions. What is not clear, however, is whether the processes driving the two illusions are expressions of the same or different neural substrates. Here we report two experiments demonstrating that direction repulsion and the DAE are, in fact, expressions of different neural substrates. Our strategy was to use each of the illusions to create a distorted perceptual representation upon which the mechanisms generating the other illusion could potentially operate. We found that the processes mediating direction repulsion did indeed access the distorted perceptual representation induced by the DAE. Conversely, the DAE was unaffected by direction repulsion. Thus parallels in perceptual phenomenology do not necessarily imply common neural substrates. Our results also demonstrate that the neural processes driving the DAE occur at an earlier stage of motion processing than those underlying direction repulsion.

1. Introduction

Coding of motion information by the visual system is a hierarchical process, with initial extraction of local motion measures being followed by ‘pooling’ of these measures at a later global-processing stage (Adelson & Movshon 1982; Albright 1984; Castelo-Branco et al. 2002; Huk & Heeger 2002). The considerable body of physiological and psychophysical data on the motion sub-system makes it an ideal substrate in which to study hierarchical processing. Consequently, there has been a recent focus on identifying where in the motion pathway various perceptual phenomena are mediated, such as motion transparency (Qian & Andersen 1995; Castelo-Branco et al. 2002; Rosenberg et al. 2008), structure from motion (Andersen & Bradley 1998) and biological motion (Grèzes et al. 2001), as well as the DAE (Kohn & Movshon 2004; Curran et al. 2006a; Wiese & Wenderoth 2007) and direction repulsion (Hiris & Blake 1996; Kim & Wilson 1997; Benton & Curran 2003; Grunewald 2004; Wiese & Wenderoth 2007).

The DAE (Levinson & Sekuler 1976) is induced through prolonged viewing of unidirectional motion (adaptor), followed by a brief presentation of a test stimulus for which direction differs from the adaptor by, for example, 25°. Observers typically overestimate the adaptor-test direction difference by as much as 40°–60°. Direction repulsion (Marshak & Sekuler 1979) occurs when the two moving patterns are superimposed to form transparently moving surfaces. Again, the direction difference is over-estimated. Their similar tuning for speed (Benton & Curran 2003; Curran et al. 2006a) and direction (Levinson & Sekuler 1976; Marshak & Sekuler 1979; Patterson & Becker 1996; Schrater & Simoncelli 1998; Braddick et al. 2002) reveals a common functional role of spatial and temporal contextual interactions in motion processing—a theme which is evident in other sensory coding (Schwartz et al. 2007). This functional commonality between the DAE and direction repulsion suggests a common process, inhibition, which drives both phenomena (Mather & Moulden 1980). This cannot, however, be taken as unequivocal evidence that the two phenomena are expressions of the same neuronal populations.

A number of studies have attempted to identify where in the motion-processing pathway the DAE and direction repulsion occur. In the case of direction repulsion, a number of authors have proposed that the mechanism driving it occurs at the early local motion-processing stages (Marshak & Sekuler 1979; Hiris & Blake 1996; Grunewald 2004; Wiese & Wenderoth 2007), while others have proposed it occurs at the later global motion processing stages (Wilson & Kim 1994; Kim & Wilson 1996, 1997; Benton & Curran 2003). These two stages of motion processing have been identified as occurring in area V1 and the human homologue of macaque MT/V5, respectively (Snowden 1994; Castelo-Branco et al. 2002; Huk & Heeger 2002). Again, in the case of the DAE, there is evidence supporting both a local (Kohn & Movshon 2004; Curran et al. 2006a) and global motion processing (Kohn & Movshon 2004; Wiese & Wenderoth 2007) account. Owing to these conflicting findings it is still unclear whether the DAE and direction repulsion are mediated by the same or different neuronal populations and, if they are mediated by different populations, which occurs first in the motion pathway.

We report on two experiments that address these questions. The strategy of our experiments was to use each of the illusions to create a distorted perceptual representation upon which the mechanisms generating the other illusion could potentially operate. Our first experiment used the binary direction after-effect (Curran et al. 2006b) to probe the neural mechanisms underlying these two phenomena. To induce the binary direction after-effect (see figure 1), observers adapt to a pattern containing superimposed fast (7° s⁻¹) and slow (2° s⁻¹) moving dots. The direction of the fast dots is offset 25° to one side (e.g. right) of vertical up, and the direction of the slow dots is offset 25° to the other side (left) of vertical. Following 30 s adaptation, the observers are presented with a test stimulus containing the same fast and slow dots, with all the dots moving vertically upwards. However, the fast and slow dots appear to move to the left and right of vertical, respectively. The difference between the perceived directions of the two test speeds is a measure of the binary direction after-effect. While previous investigations of this effect (Curran et al. 2006b) demonstrated that it comprises both DAE and direction repulsion components, the measurement paradigm employed did not distinguish whether these occur at the same stage or different stages of motion processing. We used an alternative paradigm with which to address this question in experiment 1. The results from this experiment were consistent with the DAE preceding direction repulsion in the motion-processing hierarchy.

Figure 1.

Direction adaptation affects direction repulsion. (a(i)–(iii)) Depiction of the binary direction after-effect. Observers adapt to a transparent stimulus containing fast and slow dots moving to the right and left of vertical, respectively. When followed by a test stimulus containing fast and slow dots moving vertically, the fast and slow dots appear to move left and right of vertical, respectively. (i) Adaptor stimulus directions; (ii) test stimulus directions; and (iii) perceived directions. (b) Grey bar plots magnitude of the binary direction after-effect. Black bar plots combined DAEs for single-speed test stimuli, indicating that the binary direction after-effect contains an additional direction repulsion component. White bar plots the additional direction repulsion. Error bars denote ±1 s.e.m.

Experiment 2 involved observers adapting to a ‘direction repulsion’ stimulus before making direction judgements of a briefly presented test stimulus. If (as suggested by the results of experiment 1) the DAE does precede direction repulsion, then perceived direction of the test stimulus should be distorted by the actual adaptor directions rather than its perceived directions. Again, our results were consistent with the DAE preceding direction repulsion. The combined results from these two experiments provide compelling evidence that the DAE occurs at an earlier stage of motion processing than direction repulsion and, consequently, that they involve different neural substrates.

2. Experiment 1: direction adaptation affects direction repulsion

(a) Methods

(i) Observers

Six observers—the three authors and three naive participants—took part in the experiment. All the observers had normal or corrected-to-normal visual acuity.

(ii) Stimuli

Experiment 1 was run in the Bristol and Belfast laboratories. Stimuli were random dot kinematograms (RDKs) presented within a circular aperture (6.2 deg²) on a Sony GDM-F500R monitor (Belfast) and a Sony CPD-500 monitor (Bristol). Each dot was randomly assigned a polarity (black or white), with its mean luminance equal to the background luminance (40.01 cd m⁻²). Dot density was 65 dots deg⁻². We chose viewing distances that would ensure that the stimuli subtended the same visual angle for each subject on the different experimental set-ups. Each monitor was driven by a Cambridge Research Systems VSG 2/5 graphics board at a frame rate of 80 Hz.

(iii) Procedure

During the initial motion adaptation phase (30 s duration), the observers were presented with a transparently moving random-dot mixed-speed stimulus in which 50 per cent of the dots moved at 7 deg s⁻¹ and the remaining dots moved at 2 deg s⁻¹. In addition to the difference in their speed, the dots also differed in their direction. Thus, the fast dots moved in a direction 25° to one side of vertical (upward), and the slow dots' direction was 25° to the other side of vertical. A central fixation spot was presented throughout the experiment. In the test phase immediately following adaptation, the observers were presented again with a mixed-speed stimulus with each dot moving at either 7 or 2 deg s⁻¹. However this time all the dots moved in the same direction—vertically up. The duration of the test stimulus was 0.4 s. A white line (length, 0.3° of visual angle) extended from the perimeter of the test stimulus. The observers were instructed to judge the direction of the dots (fast or slow) relative to the line segment. The line's orientation was chosen on each trial by an adaptive method-of-constants procedure (adaptive probit estimation), a method that dynamically updates the set of stimuli being presented depending on the observer's previous responses (Watt & Andrews 1981; Treutwein 1995). Line orientations were selected to optimize the estimation of the ‘point of subjective equality’, in this case the orientation of the line when the dots were perceived to be moving in the direction the line was pointing.

Each block of trials comprised 64 test stimuli; test phases alternated with adaptation ‘top-up’ phases of 5 s duration. Observers fixated a central fixation spot throughout. Each observer generated four psychometric functions per speed condition (7 and 2 deg s⁻¹ test dots), with each psychometric function being derived from 64 trials. Prior to each block of trials, the observers were informed of which speed set (slow or fast) they were to make direction judgements of.

A second experimental condition was run using a single-speed test stimulus, in which the test dot speed (2 or 7 deg s⁻¹) was randomly selected from trial to trial. Test dot density was the same as the equivalent speed set in the adaptor stimulus.

(b) Results

Figure 1 plots results of experiment 1. The binary direction after-effect (grey bar) was consistently and significantly greater (paired t-test, two-tailed, t(5)=3.01, p<0.05) than the sum of the DAEs obtained with the two single-speed test stimuli (black bar). It is important to note that the only difference between the conditions was the number of speeds in the test stimulus. The different after-effect magnitudes suggest an additional interaction, in the form of direction repulsion, occurring with the two-speed test stimulus. To test this, the observers judged the directions of a two-speed stimulus in which the slow and fast directions were offset to either side of vertical. These directions were determined by the DAEs from the earlier single-speed condition. Direction repulsion occurred for five of the six observers (white bar) and was significant across observers (one sample t-test, two-tailed, t(5)=2.81, p<0.05), consistent with the hypothesis that the binary direction after-effect is a combination of the DAE and direction repulsion. The magnitudes of the DAE and direction repulsion suggest that the binary direction after-effect results from a simple summing of the first two effects (although see Curran et al. (2006b) for a discussion of integrative processes underlying the binary direction after-effect).

These results support the view that the DAE precedes direction repulsion. This becomes clear when considering the type of test stimulus used in the binary direction after-effect condition. The test stimulus contained dots moving at one of two speeds, but all dots moved in the same direction. Note that direction repulsion effects only occur for patterns with two different motion directions. If presented without the prior adaptation, this mixed-speed test stimulus would not produce a direction repulsion effect. The adaptation resulted in speed-specific distorted representations of direction (DAE), such that the slow and fast test dots appeared to move in different directions. Our results suggest that the mechanisms underlying direction repulsion operated on these distorted representations. Of course this finding that the DAE precedes direction repulsion does not rule out the possibility that the two phenomena are the result of iterative processing occurring within the same neuronal population and, consequently, do not occur at different levels of the motion-processing hierarchy.

If the DAE truly precedes direction repulsion in the motion-processing hierarchy, then adapting to a pattern in which direction repulsion occurs should result in a DAE driven by the actual, rather than the perceived, directions. Our next experiment tested whether this is the case.

3. Experiment 2: direction repulsion does not affect direction adaptation

Experiment 2 was run in the Bristol and Sydney laboratories (the Sydney laboratory used a Sony G520 monitor and Cambridge Research Systems VSG 2/5 graphics board). In this experiment, we had observers adapt to a bidirectional dot pattern that created a strong direction repulsion effect. Following adaptation the observers judged the direction of a single-direction test stimulus. The key question here is which adaptor directions, perceived or actual, will induce a DAE in the test stimulus. If (as suggested by the results of experiment 1) the DAE precedes direction repulsion, then DAE measurements in this experiment will be determined by the adaptor's actual directions. Otherwise, they will be driven by its perceived directions.

(a) Methods

(i) Observers

Six observers—the three authors and three naive participants—took part in the experiment.

(ii) Stimuli

It was important that we used a stimulus which produced a large direction repulsion effect. Through piloting the experiment, we found that Laplacian of Gaussian (LOG) dot stimuli produced a larger effect than non-filtered dot stimuli. Adapting and test stimuli contained isotropic LOG dots:

$${\nabla }^{2}G(x,y,\sigma )=\frac{1}{{\sigma }^2}\left(1-\frac{x^2+y^2}{{\sigma }^2}\right)\text{exp}(-(x^2-y^2)/2{\sigma }^2),$$

with s.d.=0.1° (figure 2). Each micro-pattern had a peak spatial frequency of approximately 3.8 cycles deg⁻¹. At the start of each sequence, the polarity of each LOG function was randomly assigned. The contrast of the patterns was expressed as a proportional maximum deviation from the mean luminance and was 0.30. Mean luminance was 55 cd m⁻². The aperture edge was blurred (with integral of Gaussian; s.d.=0.1°). Stimuli were presented within a circular aperture (area=19.63 deg²), and micro-pattern density was 8.8 elements deg⁻².

Figure 2.

Direction repulsion does not affect direction adaptation. (a) The adaptor contained two groups of superimposed dots for which direction differed by 60° (blue arrows). Observers judged the direction of a single-direction test stimulus (red arrow) set halfway between one of the adaptor directions and its perceived direction. (b) Example frame from the LOG dot stimuli used in experiment 2. (c) For all but one observer, perceived direction of the test stimulus (black squares) is closely predicted by the mean DAE of the adaptor's physical directions (blue triangles), indicating that the DAE is driven by the adaptor's actual, rather than perceived, directions. Squares, DAE; green triangles, mean perceived DAE; blue triangles, mean actual DAE; and dashed line, test direction.

(iii) Procedure

As a precursor to running the experiment proper, we measured the direction repulsion of two superimposed sets of dots for which directions differed by 60°. Both the dot sets moved at the same speed (4 deg s⁻¹) and their directions were offset to the same side of vertical. Using a direction-judgement task, we identified the directions of both the dot sets when the dot set moving closest to vertical was perceived to be moving vertically up (figure 2). Observers were then tested with a stimulus containing these two directions and, using the line orientation task of experiment 1, we identified the perceived direction of the set of dots moving further from vertical. The direction repulsion of each dot set varied across observers—repulsion ranged from 8.34° to 11.56° for the dot set moving closest to vertical, and from 2.89° to 11.74° for the dot set moving further from vertical.

We now had the four direction parameters necessary for running the experiment—two actual directions and their perceived directions (figure 2). In the ‘bidirectional’ condition, the observers adapted to an RDK stimulus containing two motion directions differing by 60°; the directions were individually tailored for each observer using the direction parameters obtained from the previous condition. Initial adaptation lasted 30 s and subsequent top-up adaptation phases lasted 5 s. The test stimulus (speed 4 deg s⁻¹) contained dots moving in the direction half-way between vertical up and the adapting direction closest to vertical up. The directional offset of the test stimulus from vertical was determined by each observer's repulsion measurements from the previous condition. The line orientation task was used to measure perceived direction of the test stimulus. We also measured the DAEs induced by each of the actual and perceived adaptor directions individually, which were compared with the DAE from the bidirectional condition.

(b) Results

Figure 2 plots the DAE magnitudes obtained in the bidirectional condition (black squares) as well as the mean DAEs obtained using adaptors containing individual perceived directions (green triangles) and actual directions (blue triangles).

Across the observers, the mean DAE to the bidirectional adapting stimulus was 4.21±0.31°. This value is closely predicted by the average of the DAEs induced by the two actual adaptor directions (5.03±0.25°; t(5)=2.10; p=0.090)—model 1. By contrast, it differs markedly from the average of the DAEs induced by the perceived directions (0.44±0.82°; t(5)=4.33; p=0.008)—model 2. A quantitative comparison of the measured likelihoods of these two models yields a Bayes factor of 12.0, indicating that the data constitute strong evidence in favour of the hypothesis that the bidirectional DAE involves adaptation to the actual rather than the perceived directions of the component motions (Jeffreys 1961).

4. Discussion

Schwartz et al. (2007) highlight the tendency to treat temporal and spatial contextual effects separately, even when they reveal similar functionality and have a similar impact on vision. This observation applies to two well-known visual illusions brought about by temporal and spatial contextual manipulation—direction repulsion and the DAE, respectively. We sought to determine whether there is any justification in treating these effects separately or whether they do, in fact, reflect activity of the same neuronal populations.

In experiment 1 we were able to induce direction repulsion in a test stimulus that would not normally exhibit spatial contextual effects without prior adaptation. Using an appropriate adaptor, we were able to induce speed-specific DAEs in opposing directions; this perceptual distortion was, in turn, operated upon by the mechanisms underlying spatial contextual effects to produce additional direction repulsion. These results are strongly suggestive of separate mechanisms driving the DAE and direction repulsion, and imply that mechanisms driving the DAE precede those driving direction repulsion.

Our second series of experiments tested this hypothesis directly by determining which directions in a bidirectional adaptor, the actual or perceived, induce the DAE. If the DAE precedes direction repulsion, then the perceptual distortion of a single-direction test stimulus would be driven by the actual adaptor directions. Otherwise, the perceptual distortion should be driven by the perceived adaptor directions. The data from this experiment were consistent with the former scenario.

The combined results of these experiments provide compelling evidence that the DAE precedes direction repulsion in the motion-processing hierarchy; consequently, they are expressions of processing at different neural sites. Thus, although spatial and temporal contextual interactions in sensory coding may serve a common functional role (Schwartz et al. 2007), in the motion pathway at least they are mediated by different substrates of the processing hierarchy.

The finding that direction repulsion and the DAE are expressions of different neural substrates makes an important contribution to the current debate on the neural location of these phenomena. Kohn & Movshon (2004) report that changes in tuning functions of directionally sensitive neurons in macaque MT, but not V1, are consistent with perceptual distortions experienced with the DAE, suggesting that the DAE may occur at the global motion level. However, Kohn and Movshon note that their data can also be modelled by weakening feed-forward input from V1 into a recurrent model of MT circuitry, which would be consistent with a local motion-processing account of the DAE. Recent psychophysical data pointing to the DAE being a local motion phenomenon (Curran et al. 2006a) support the latter interpretation.

In the case of direction repulsion, two studies (Grunewald 2004; Wiese & Wenderoth 2007) found that the phenomenon fails to exhibit interocular transfer, suggesting it to be monocular in origin. Because monocular-driven cortical neurons do not exist beyond area V1, the findings support the notion of direction repulsion being driven by local motion detector activity. However, it should be noted that both studies used very sparse dot stimuli to avoid binocular rivalry (binocular rivalry describes how, when presented with different information to each eye, the different retinal inputs arriving at the cortex compete to dominate perception). Kim & Wilson (1997) avoided this rivalry problem by presenting a central test stimulus to one eye and a surrounding inducing stimulus to the other. They found robust interocular transfer of direction repulsion with this centre-surround configuration. Furthermore, the fact that the effect persisted for non-overlapping moving patterns suggests that direction repulsion may occur after the pooling of local motion measurements. Benton & Curran's (2003) finding that global-motion interactions play a major role in driving direction repulsion supports this position.

While data from the experiments reported here do not directly identify where in the motion pathway the DAE and direction repulsion occur, they compellingly illustrate that (i) the two phenomena are expressions of different neural substrates, and (ii) the DAE occurs in the motion pathway earlier than direction repulsion. Taken within the context of previous studies, our data are consistent with the DAE occurring at the local motion-processing stage and direction repulsion being driven by neural activity at the global motion-processing stage.