Motion signals bias localization judgments

Abstract

In the flash-lag illusion, a moving object aligned with a flash is perceived to be offset in the direction of motion following the flash. In the “flash-drag” illusion, a flash is mislocalized in the direction of nearby motion. In the “flash-jump” illusion, a transient change in the appearance of a moving object (e.g., color) is mislocalized in the direction of subsequent motion. Finally, in the Frohlich illusion, the starting position of a suddenly appearing moving object is mislocalized in the direction of the subsequent motion. We demonstrate, in a series of experiments, a unified explanation for all these illusions: Perceptual localization is influenced by motion signals collected over ∼80 ms after a query is triggered. These demonstrations rule out “latency difference” and asynchronous feature binding models, in which objects appear in their real positions but misaligned in time. Instead, the illusions explored here are best understood as biases in localization caused by motion signals. We suggest that motion biasing exists because it allows the visual system to account for neural processing delays by retrospectively “pushing” an object closer to its true physical location, and we propose directions for exploring the neural mechanisms underlying the dynamic updating of location by the activity of motion-sensitive neurons.

Introduction

Because neural processing takes time, perceptual systems must draw conclusions about the outside world based on data that are slightly outdated—in other words, perception lives slightly in the past (Changizi, 2001; Eagleman & Sejnowski, 2000a, 2003; Libet, 1993; Nijhawan, 1994). This fact becomes important when an observer is asked to specify the instantaneous position of a moving object (“where is the object now?”). By the time a conclusion is reached about location, the object has moved on to a new position in the real world. How and whether brains may compensate for this delay have been the subject of intensive discussion (Changizi, 2001; Eagleman & Sejnowski, 2002; Krekelberg & Lappe, 2001; Nijhawan, 1994; Whitney, 2002). For example, perceptual systems may extrapolate the position of moving objects, making a best guess where they will be at the present moment given delayed incoming data (Changizi, 2001; Nijhawan, 1994). An alternative hypothesis is that when the brain is triggered to make an instantaneous-position judgment about a moving object, it continues to collect information for a window of time after the triggering stimulus, causing the final perception to reflect events that happen after the stimulus (Eagleman & Sejnowski, 2000a, 2000b, 2000c).

In recent years, this debate has been explored in the context of the flash-lag effect (FLE; Eagleman & Sejnowski, 2000a; MacKay, 1958; Nijhawan, 1994). In this illusion, a moving object that is aligned with a flash appears to be located—at the moment of the flash—slightly displaced in the direction of motion that occurred after the flash. The illusion belies a systematic error in our judgment of the position of a moving object—but what explains this mislocalization? Recent experiments have narrowed the answer to two viable hypotheses, and understanding the difference between them is crucial to our picture of the neural representation of time and space. The first, known as the latency difference hypothesis, proposes that different kinds of neural signals are processed at different speeds. For example, it has been hypothesized that flashed objects may be processed more slowly than moving objects (Jancke, Erlhagen, Schoner, & Dinse, 2004; Krekelberg & Lappe, 2001; Patel, Ogmen, Bedell, & Sampath, 2000; Purushothaman, Patel, Bedell, & Ogmen, 1998; Whitney & Murakami, 1998; Whitney, Murakami, & Cavanagh, 2000). In this class of model, whichever signals reach a “perceptual end point” first are perceived first. Thus, the latency difference model (LD model) predicts that the moving object is seen in a real position but misaligned in time (Figure 1a). The second hypothesis, instead of postulating misalignment in time, postulates errors in localization of the moving object (Figure 1b; Eagleman & Sejnowski, 2000a, 2000b, 2000c, 2002; Krekelberg & Lappe, 2000). This second type of model is based on spatial rather than temporal mechanisms. We present here several new classes of experiments to distinguish these models. We will argue that when the brain is triggered to make an instantaneous-position judgment, motion signals that stream in over ∼80 ms after the triggering event (e.g., a flash) will bias the localization. For brevity, we will refer to this framework as the motion-biasing model (MB model).

Figure 1.

Temporal and spatial hypotheses for the FLE. The solid colored bar shows the real position of the bar at the time of the flash; the dashed bar shows the perceived position (size of illusion is exaggerated for illustration). (a) Temporal models, such as the LD model, propose that the moving object is seen in a real position but misaligned in time. In this example, the moving object and flash both occur at t₀. However, if moving objects are perceived more quickly than flashes, the flash at time at time t₀ is perceived as simultaneous as the bar at time t₄. (b) A spatial model proposes that the FLE results from instantaneous-position computations being influenced by motion signals collected for ∼80 ms after triggering by the flash. In other words, when the brain is asked “where was the bar at the moment of the flash?”, the localization is dragged in the direction of the motion that happens over the next ∼80 ms. Note that the influence of the events after the flash makes the perception postdictive (Eagleman & Sejnowski, 2000a) rather than predictive (Nijhawan, 1994). As this model involves a temporal window of integration, it might also be called “spatiotemporal”; for simplicity, we refer to it as “spatial” to put the emphasis on the mislocalization of position.

The influence of motion on perceived position is well known (see Whitney, 2002, for a review). Ramachandran and Anstis (1990) found that a drifting field of dots displaced the perceived edges of the field. De Valois and De Valois (1991) demonstrated that drifting Gabor patches appear mislocalized in the direction of their motion. Snowden (1998) demonstrated that a motion aftereffect produced a change in a pattern’s perceived position, and the same holds for rotational motion (when a spinning windmill comes to a stop, the perceived position of the vanes is slightly displaced in the direction of the motion aftereffect; Nishida & Johnston, 1999). Even second-order motion shifts perceived position (Bressler & Whitney, 2006). Finally, Whitney and Cavanagh (2000) found that motion in a scene can influence perceived position of separate, nonmoving flashes elsewhere in the scene (Durant & Johnston, 2004), a finding to which we return below.

In this article, we will demonstrate that the motion-biasing framework explains and unifies four classes of illusions: the FLE, the Frohlich effect, the flash-drag effect (FDE), and the feature flash-drag effect (FFDE). In the Discussion section, we suggest that the visual system may make instantaneous-position judgments only when triggered to do so—that is, on a need-to-know basis (Eagleman, manuscript in preparation).

Results

The FLE is explained by position biasing by motion signals after the flash

In the FLE, a moving object compared to a physically aligned flash is perceived to be mislocalized in the direction of postflash motion (Eagleman & Sejnowski, 2000a; Khurana & Nijhawan, 1995; Nijhawan, 1992; Whitney & Murakami, 1998). We here pit the LD and MB models (Figure 1) against one another with a simple apparent-motion flash-lag display, in which we degrade position information while retaining motion information.

Observers reported whether the starting position of a moving dot was to the right or to the left of a flashed bar (Figure 2a). In the “five-station” condition, the moving dot occupied five positions over a 67-ms period; in the “two-station” condition, the dot moved the same distance in the same time but can be thought of as invisible in the second, third, and fourth positions. Online demonstrations can be viewed at www.eaglemanlab.net/motionbias. Trial types and direction of movement were randomized, and the size of the effect was determined by 50% of the psychometric functions.

Figure 2.

The FLE is explained by motion signals biasing localization judgments. (a) Observers reported whether the starting position of a moving dot (1° above fixation) was to the right or to the left of a flashed bar (1° below fixation). The first dot and the flash appear simultaneously on a dark screen. In the five-station apparent-motion condition (left), the moving dot occupied five positions over 67 ms; in the two-station condition (right), the dot moved the same distance in the same time with no visibility in the second to fourth positions. Starting position, direction, and condition were randomized. Dot diameter = 0.5°, bar height = 0.5°, bar width = 0.1°. (b) Fifty percent of the psychometric function was determined as the measure of perceived offset for each observer. Observers perceived the moving dot to be ahead of the flash by 0.11° ± 0.04° (five-station condition) and 0.21° ± 0.15° (two-station condition). n = 6. (c) Again, observers watch a flash appear simultaneously with the first position of a moving dot. In this experiment, they click (with no time pressure) anywhere on the screen to indicate the perceived position of the first dot. The split condition is the same as the five-station condition (above), but here, the first dot splits into two trajectories ±45° to the horizontal. The five-station and two-station conditions were randomly interleaved with this condition, and direction of motion was randomized. Localization of the first dot is biased by the sum of motion vectors, as indicated by the distribution of mouse clicks normalized to a single direction of motion. The distributions in the three conditions are each significantly different from zero (p < .008, two-tailed t test) and not significantly different from one another in pairwise comparisons (all p values > .24). n = 6 observers, 40 trials each. (d) Frohlich effect. Stimulus was the same as the five-station condition in Panel a, except that the flashed bar was replaced by a continuously present bar (a “landmark”). Observers indicated whether the first visible position of the moving object was to the right or to the left of the landmark. Average offset = 0.1° ± 0.05°.

The MB model predicts that the motion signal should shift the localization of the first position of the moving object. In contrast, the simplest version of the LD model predicts that a moving object is seen misaligned in time but always in a veridical spatial position (i.e., in a position it actually occupied at some point). Therefore, in the case of two-station apparent motion, the basic LD model would predict that at the moment of the flash, the dot will be seen in either Station 1 or Station 2—but not somewhere in between.

Figure 2b demonstrates that the FLE is perceived in both conditions. In other words, the “moving” object is mislocalized in the direction of future movement when presented on the screen in physical alignment. This removes the possibility that the system is assigning the time of the flash to a later position actually occupied by the moving object because in the two-station condition, there is no physical occurrence of the moving object at points intermediate to its starting and ending positions—instead, observers perceive the moving object in a position where it never was. Moreover, there is no chance for 80-ms temporal misalignment here (as proposed by the LD model) because the last dot has appeared by 53 ms into the trial. Instead, it appears that the motion information inferred by the visual system from the apparent motion is sufficient to bias the first perceived position of the object.

The possibility still remains, however, that the moving dot is seen in an interpolated position of its trajectory (Barlow, 1979; Burr & Ross, 1979), which could allow a form of the LD model in which the flash catches up with a delayed interpolated position. Note that this possibility would require the perception to be constructed retrospectively because the interpolation of the trajectory has to wait for the second dot to appear before it can know which way the trajectory went. Given this possibility of a retrospective LD model, we wanted to directly test whether a dot could be misplaced by a sum of motion vectors, to be perceived in a position where it never actually appeared (i.e., off its interpolated path). To that end, we introduced a third condition that was similar to the other two, except that, now, the first dot is followed by two dots that move on a 45° trajectory above and below the horizontal (“split” condition). In a new set of experiments, we interleaved this condition with the previous two conditions and asked observers to click with their mouse to indicate the perceived location of the first dot in relation to the flash (which was physically aligned with the first dot). The results show that the distributions in all three conditions are biased in the direction of ensuing motion (Figure 2c, p < .004). In other words, when the flash and the first dot are aligned, one generally observes the first position of the dot to be slightly to the right of the flash. Most important, in the case of the split condition, the motion signals have biased the localization in the direction of the sum of the motion vectors. If observers’ mislocalizations were based on an interpolated trajectory (Barlow, 1979; Burr & Ross,1979), then, presumably, the first dot would be mislocalized along one of the apparent-motion paths or the other, or even possibly both. The distribution of clicks and posttest interviews indicate that these possibilities did not occur. This result rules against the LD model (Figure 1a), which hypothesizes that objects are seen in their actual or interpolated positions (albeit at the wrong time). In other words, the LD model would not predict that the perceived position of the split should be anything other than vertically aligned with the physically aligned flash.

Frohlich effect

In the Frohlich effect, the first position of a suddenly appearing moving object is mislocalized in the direction of movement (Frohlich, 1923). We suggest that the Frohlich effect is nothing but a special case of the FLE, wherein the trigger to localize an object is the object’s appearance rather than a separate temporal landmark (a flash; Eagleman & Sejnowski, 2000a). To test this, we made the flashed bar in the first experiment a “fixed” bar, which remained on the screen continuously instead of transiently. In this way, it served simply as a spatial landmark. Observers report roughly the same displacement as before (Figure 2d), implying that the Frohlich effect, like the FLE, is best understood as a mislocalization due to biasing by subsequent motion. Note that the first position of the splitting dot is clearly visible (see online demonstration), ruling out masking as an explanation for the Frohlich effect.

Flash lag is turned into flash drag when motion is attributed to the flash

Recently, Whitney and Cavanagh demonstrated that the perceived position of a brief flash can be shifted in the direction of motion occurring in the nearby visual space. We here refer to this phenomenon as the FDE. Note that the FDE runs in the opposite direction of, and dilutes, the FLE (Figure 3a). What is the difference between the conditions that yield the FLE versus FDE? This has so far remained unexplained. Whitney and Cavanagh (2000) suggested that motion “distorts” nearby visual space. Cai and Schlag (2002) suggested, on the other hand, that the FDE might result from the flash being interpreted as an instantaneous extension of the moving object itself. We make a different suggestion that addresses both the FLE and FDE by taking into account the degree to which motion signals bias different objects in the scene. In our framework, motion signals can bias the localization of a flashed object by different amounts, depending largely on proximity and perhaps also on higher level context (more on this in the Discussion section). We parameterize the degree of coupling between the motion signals and the localization judgment by λ ∈ [0,1], where λ = 1 indicates total coupling and λ = 0 indicates no coupling. When the flash is interpreted as a separate, accidental object, λ = 0. In cases where the flash is dragged, λ > 0 (we note that it is possible, although not explored here, that λ can take on negative values, such that localization is displaced in the direction opposite to the motion; consider, e.g., the onset-repulsion effect; Thornton, 2002).

Figure 3.

The FDE, which dilutes the FLE, occurs when motion signals shift the localization of nearby objects. (a) The measured FLE is the perceived offset between the motion-shifted moving object and the motion-shifted flash. When the flash is interpreted as part of the moving object, the flash drag is maximal and the flash lag is reduced to zero. The cartoon represents λ_flash = 0.2 (see text). (b) Reexamining the Baldo and Klein (1995) result. Stimulus: a spinning “bar” composed of three dots rotates around the central dot. At a random time, two end-segment flashes occur (red, 13.7 ms duration) at one of three random distances from the center point. In one block of trials, observers used a method of adjustment to change the position of the flashes until the bar and flashes appeared in alignment (this measured the FLE). In the other block, observers saw the same stimulus but were asked a different question: Now, they adjusted the flashes until the flashes appeared horizontally aligned with a continuously present landmark line on the screen (this measured the FDE). The order of the blocks was randomized across observers. Direction of spinning dots was reversed for half the trials. (c) As the FDE increased, the FLE decreased. Method of adjustment, three trials per condition, n = 4.

It is important to note that the FLE is always measured by quantifying the physical offset between the flash and the moving object, but this measure only captures the difference between the motion-shifted flash and the moving object (Figure 3a). In other words, if a flash is interpreted to be an instantaneous extension of the moving object, the flash drag is maximal and the measured flash lag is reduced to zero. Therefore, one must be careful in the interpretation of a reduced FLE: It could reflect a reduction in motion biasing of the moving object or an increase in the motion biasing of the flash.

With this in mind, we revisited an experiment by Baldo and Klein (1995), in which observers reported a larger FLE when flashes appeared at larger radial distances from a spinning bar. The result was interpreted by the authors as evidence of an attentional shift that requires more time to cover larger distances between the flash and the moving object. Our model offers a different interpretation: At smaller distances between flash and moving object, the FDE grows and, thus, dilutes the flash lag. To distinguish these interpretations, we had observers participate in two blocks of experiments. In one block, observers watched a spinning bar and adjusted the radial angle of flashed end segments until the bar and flashes appeared aligned (Figure 3b). Consistent with Baldo and Klein, we found that the measured FLE diminishes with decreasing distance between the flashes and the moving object (Figure 3c). In the second block, the stimuli were identical but the question was different: Now, observers adjusted the angle of the flashes until they appeared horizontally aligned with fixed horizontal landmark lines at the edges of the screen (the landmark lines were also present in the other block but immaterial to the experiment). This allowed us to measure the FDE, and here, we found that the mislocalization of the flashes diminishes with increasing distance from the moving bar (Figure 3c; results are consistent with those from Durant & Johnston, 2004).

Combining the two findings, we see that as the FDE decreases, the measured FLE increases. In other words, the increase of the FLE can be at least partially explained by the decrease of the FDE. As opposed to the MB model, neither an attention shifting model (Baldo & Klein, 1995) nor an LD model provides an explanation for these results.

Motion biasing, not asynchronous feature binding, explains the flash-jump illusion

When a feature of a moving object is flashed—say, a sudden color change from white to blue—the feature change is mislocalized to a later point in the trajectory (Cai & Schlag, 2001). We call this the FFDE because the localization of a brief feature flash is dragged in the direction of motion following the flash. Cai and Schlag (2002) have suggested that this illusion results from asynchronous feature binding (AFB), which means that the flashed color change is “delayed and assigned to a later occurring bar.” In other words, they suggest that the flashed property is incorrectly bound in time. Alternatively, the MB model suggests that the judged instantaneous position of the blue bar (about which the localization is being queried) is dragged in the direction of motion.

To distinguish these models, we asked observers to report on the relative positions of two white bars, moving in opposing directions, which simultaneously flash blue for 10 ms. In two randomly interleaved conditions, the apparent-motion stations of the bars were either “dense” (0.7° apart) or “sparse” (2.1° apart; Figure 4a). The bars move the same distance in the same amount of time in both conditions (two of every three stations can be thought of as invisible in the sparse case). As can be seen in Figure 4b, the amount of illusory displacement of the feature flash was not significantly different in the two conditions (p = .46, paired t test). This result is striking because in the sparse case, the blue bar is perceived in a location where it was never physically presented. This result rules against an illusory conjunction (or AFB) model, which predicts the assignment of blue to a later appearance of the bar. Instead, our evidence suggests that motion signals drag the position judgments of the feature-flashed bar (causing “flash jump”).

Figure 4.

Distinguishing two hypotheses for the FFDE. In the AFB model, assignment of the sudden feature change takes time and is, thus, assigned to a later appearance of the object. In the MB model, the object to be localized (here, a blue bar) is dragged by the motion signals that follow over the next ∼80 ms. When apparent-motion stations are dense, the models are difficult to distinguish, but if the bars are sparsified and observers report the blue bar in an intermediate position, that rules out assignment to a later appearance of the bar. (a) Moving bars were presented in either dense or sparse configurations. On each trial, observers watched two such bars moving in opposite directions, and at the moment of the synchronized color flash of both bars, they reported whether the top bar was to the left or to the right of the bottom bar. See www.eaglemanlab.net/motionbias for demonstrations. For each observer, 50% of the psychometric function was determined as the measure of perceived offset; one half of that offset is reported in the graph. The displacement of the blue bar was not significantly different in the two randomly interleaved conditions (1.03° dense, 1.18° sparse, p = .46). (b) Observers watched a single white bar moving in a random direction (sparse spacing condition). After a random number of appearances, the bar flashed blue for one station. Observers were asked to count how many white bars followed. If blue were assigned to a later appearance of the bar (asynchronous binding), observers should undercount. Instead, results were veridical.

Next, we asked subjects to report the number of white bars that followed the appearance of the blue bar. If blueness were superimposed on a later appearance of the bar, we would expect subjects to underestimate the number of subsequent white bars. However, observers counted the white bars veridically and with little effort, supporting the MB model.

Together, the results of Figures 4a and 4b indicate that the blueness is not being assigned to a later appearance of the bar. This rules out a simple misbinding model and, instead, suggests that the appearance of the blue bar is dragged in the direction of motion signals collected over a small window of time after the flash. The results do not, however, rule against the possibility that the position of the blue bar is interpolated and that the blue is assigned to an interpolated position. This possibility of feature binding to an interpolated position has been suggested by Cai and Cavanagh (2002), who found that the illusion can be seen even across blind spots. Thus, to distinguish the MB model from interpolation, we next presented two moving gray squares traveling along perpendicular trajectories. At the point of their intersection, the single visible square flashes blue (Figure 5a). The MB model predicts that the position of the blue square will be biased in the direction of motion signals, which, in this case, will sum to point to a position off the trajectories, whereas interpolation would predict that there would be one, or possibly two, blue square(s) perceived along the actual trajectory of motion (i.e., along one or both of the arms of the X).

Figure 5.

Localization of the feature-flashed moving object is biased by the sum of motion vectors. Stimulus: two gray boxes, 0.8° on a side, followed perpendicular trajectories. Each box occupied nine successive screen positions and remained in each position for 40 ms, for a total path of 7.2° over 360 ms. (a) Two X-shaped trajectories were compared with each other on opposite sides of fixation: one with both squares moving up and the other with both squares moving down. Direction of trajectories and offset between the center points were randomized. Offset was measured as half the shift indicated by the psychometric functions. All observers verbally reported seeing only one blue square; 120 trials, n = 4. (b) To indicate whether the blue square was being perceived along one trajectory or the other, observers indicated whether the blue square was to the left or to the right of a comparator line placed randomly in a horizontal range of ±1°; 80 trials, n = 4. (c) Observers clicked the computer mouse at the location on the screen where they perceived the center of the blue flash. The mouse pointer was invisible until the stimulus display ended. The X-shaped trajectory was randomly displaced ±5° horizontally and vertically from the screen center on each trial, and the direction of the paired trajectories (up or down) was randomized; 20 trials per subject, in a dark room with no fixation point. The slight upward-right and downward-left bias is unexplained but may relate to subject handedness or the tilt of the mouse arrow. Histograms on the right and top show the distribution of perceived positions. If the blue were assigned to a real position of the square or even to an interpolated position, the histogram on top would be double-humped (blue). Instead, the distribution best matched a Gaussian centered at zero (magenta).

Observers were presented with two versions of this stimulus on either side of the fixation point (one moving upward and the other downward) and were asked to report whether the blue square on the left appeared above or below the simultaneously flashed blue square on the right (Figure 5a, inset). Observers perceived a displacement of each blue square 1.85° ± 0.4° in its direction of motion (Figure 5a, a total difference of 3.7° between the squares; see online demonstration). All subjects verbally reported that they perceived only one blue square per X-shaped stimulus—in other words, the blueness did not split. This result suggests that the localization of the blue square is biased by the sum of the motion vectors.

However, this result does not rule out the possibility that the blueness was assigned to one leg of the trajectory or the other on any given trial. To address that, we presented observers with a single X-shaped trajectory and asked them to compare the horizontal position of the blue square with a comparison line below. By randomizing the position of the comparator, we constructed psychometric curves and found that the blue square was seen with high precision in the middle of the X, not displaced along one leg or the other (Figure 5b).

To further verify our interpretation that the location of the blue square is biased in the direction of the sum of motion vectors, we next asked observers to click with a mouse pointer to directly report the perceived position of the blue square. Figure 5c clearly confirms a localization bias in the direction of the sum of the two motion vectors, that is, above the X, not along one or both of its arms. Again, all observers verbally reported that they only perceived a single blue square and had never perceived two. Collectively, these support the MB model and disconfirm interpolation because the blue square is perceived in a location where it never physically was.

Finally, we turned to a striking feature of the feature-flash illusion, first demonstrated by Cai and Schlag (2001). If a moving bar changes height, then the flashed feature (e.g., blueness) not only appears mislocalized in the direction of motion but also appears to be the height of a bar further into the trajectory. For example, a moving and shrinking bar that is flashed blue will be reported to be smaller than it actually was; a growing bar flashed blue will be reported as taller. We asked observers to compare two opposing horizontal trajectories: one shrinking and one growing (Figure 6a). Using a method of adjustment, observers watched the stimulus repeatedly and adjusted the height of one of the bars until it appeared that both bars, when simultaneously flashed blue, were of equal height. In this way, we verified the original report by Cai and Schlag that the blue bar is seen at intermediate heights that never actually occurred, and we found no significant difference between dense and sparse conditions (Figure 6b, average adjustment = 0.47° to each edge; p = .53, paired t test). We suggest two nonexclusive hypotheses to explain this result, as illustrated in Figure 6c. First, it may be that motion signals bias the perception of the individual edges of an object: In this case, the top and bottom edges of a growing bar move orthogonally to the bar’s translation (whereas the left and right edges are both biased in the direction of the bar’s motion). Secondly, it may be that the visual system tends to interpret growing or shrinking bars as having movement in depth and that implied motion in depth shifts the localization further in that direction (Figure 6c, right). This speculation is consistent with the existence of the FLE in depth (Harris, Duke, & Kopinska, 2006; Ishii, Seekkuarachchi, Tamura, & Tang, 2004).

Figure 6.

Can motion signals differentially bias the perception of individual edges of a moving object? (a) Stimulus based on Cai and Schlag (2001). Aligned, equally sized blue bars flashed during opposite trajectories of gray bars (one translating horizontally and shrinking, the other translating the opposite direction and growing) will appear not only horizontally offset but also of different heights. Bar with dashed border represents perceived position. Nine bars in each stream, 2.1° apart. Direction of motion and initial height offset randomized. (b) Feature-flashed bars in opposing streams (one shrinking and one growing) were presented repeatedly. Observers used a method of adjustment to change the height of one of the streams until the heights of the simultaneous blue bars appeared equal. n = 3, 15 repeats per observer, randomized heights and directions. Average height adjustment was 0.94°, or 0.47° per edge (top and bottom). (c) Two hypotheses for the height deformation quantified in Panel b. Left, motion signals are specific to individual edges of the moving object. Note that the motion signals after the feature flash determine the perception. As shown by Cai and Schlag, if the shrinking bar suddenly begins to grow after the feature flash, the blue bar will appear larger, not smaller. Right, motion biasing of the height may also be conceived as adjusting height estimates of objects moving toward or away from the viewer. Arrows show a possible interpretation by the visual system.

Discussion

We have shown that four classes of illusions can be explained by the well-known phenomenon that motion signals bias position judgments (Figure 7). First, we demonstrated that the FLE is obtained even in simple apparent-motion displays—including a display that splits its trajectory—ruling out an LD model in which a slowly processed flash is seen as synchronous with a later position of a moving object. We note that an important finding for understanding the FLE is that the magnitude of the effect is the same whether the moving object has been moving when the flash occurs (continuous motion condition) or whether it appears on a blank screen simultaneous with the flash and then moves (flash-initiated condition; Eagleman & Sejnowski, 2000a; Khurana & Nijhawan, 1995). Our original interpretation was that the flash reset the motion interpolation. In our modified view, the instantaneous-position judgment about the moving object is biased by motion signals that follow.

Figure 7.

Summary: motion biasing of localization as an explanation for four illusions. (a) The FLE is caused by instantaneous-position computations being shifted by motion signals collected for ∼80 ms after triggering by the flash. The solid bar shows the real position of the bar at the time of the flash; the bar with dashed outline represents perceived position (size of illusion is exaggerated for illustration). (b) The Frohlich effect is explained by the same mechanism—here, however, the first position of the bar serves as the reference point for localization instead of the position of the bar at the time of the flash, as above. The outcome is the same in both cases. (c) The FDE obtained when motion signals from nearby objects bias the localization of flashed objects. (d) The feature-flash effect is obtained when observers are asked to isolate a particular location of a moving object (“where was the bar at the moment it flashed blue?”)—as above, motion signals that follow the localization event in question will shift the final position estimate.

Next, we pointed out that motion biasing also serves as an explanation for the Frohlich effect, in which the starting point of a suddenly appearing moving object is judged some distance into the trajectory. Whitney (2002), in addressing models of the Frohlich effect, reviews several models and notes that “the common theme among most models is that the timing of perception is important; the latency with which the initial position of the moving object is perceived determines where the object appears to be.” We suggest an alternative explanation that does not depend on latency and concentrates instead on spatial biasing by motion. Further, whereas other interpretations of the Frohlich effect appeal to latency (Whitney, 2002) or attention (Musseler & Aschersleben, 1998), the motion-biasing explanation offers a simple explanation for a lesser known experiment: When Frohlich (1923) covered the later part of the trajectory (after the point of first appearance), subjects now saw the bar in its veridical starting position, where they did not see it before. This indicates that the subsequent motion signals were required to bias the localization.

We also showed that the FDE occurs when neighboring motion signals influence the perceived position of a nonmoving object (in this case, the flash). Whitney and Cavanagh (2000) came to a similar conclusion when discussing the FDE: “The issue, then, is not the dissociation between the coding of stationary and moving stimuli, but how the configuration of motion in the visual field influences the localization of both moving and stationary stimuli.”The motion-biasing framework provides a new interpretation for an experiment by Baldo and Klein, which previously seemed to support an attentional role in the FLE—instead, we have found that their experiment illustrates a trade-off between flash lag and flash drag (Figure 3).

Finally, we addressed the feature-flash effect, for which it had been proposed by Cai and Schlag (2001) that a flashed feature of a moving object (say, a color change to blue) is incorrectly bound to a later appearance of the object. By degrading the position information, we demonstrated instead that a blue bar is shifted in the direction of motion to a position where it never actually existed. In the demonstration of Cai and Schlag, the bars were so close together that it was not possible to tell whether the blue bar was mislocalized or, instead, whether the blue was bound to a later appearance of a white bar. By sparsifying the appearances of the bars, while retaining the motion signals, we could rule out the asynchronous feature binding model. Further, subjects could veridically count the number of white bars presented after the blue bar. These findings indicated that there is no illusory conjunction of the blue color to a later white bar. Finally, we showed that when moving objects on intersecting trajectories shared a single flashed feature, localization was dragged in the direction of the sum of motion vectors.

Collectively, these results demonstrate the generality of motion biasing and offer a unified explanation for several types of illusion—suggesting that an error in localization, not relative timing, explains all the phenomena (Eagleman & Sejnowski, 2000a, 2000b, 2000c, 2002). The motion biasing of localization has been proposed for several other illusions, and, as noted elsewhere, it seems reasonable that motion biasing exists because it is normally useful (De Valois & De Valois, 1991; Whitney, 2002). That is, the visual system attempts to correct for the processing delays in signals from eye to perception and accounts for these delays by shifting its localizations closer to where they would be if there were no neural delay. Motion biasing will normally push objects closer to their true location in the world—not by extrapolation into the future (also a spatial model, proposed by Nijhawan, 1994) but, instead, by a clever method of updating signals that have become stale due to processing time. In addition, unlike extrapolation, motion biasing can even estimate position after the reversal of an object—for example, in the case of ricochets or bounces.

Instantaneous position computation on a need-to-know basis?

If motion biasing is the correct explanation for these illusions, it emphasizes a key principle: Localization computations might only be triggered on a need-to-know basis (Eagleman & Sejnowski, 2002). If true, this suggests that it may be computationally expensive to represent instantaneous-position information when the system does not need it and that the exact position of a moving object is only computed occasionally. When it is computed, the result is biased in the direction of motion signals that stream in over the next ∼80 ms. This further implies that other positions in the trajectory of the moving object are not necessarily explicitly computed. That is, the answer to the question “Is the entire trajectory seen in an upward shifted position, or is it seen in its veridical position?” may be “neither.” The rest of the trajectory may not be represented with precise localization until it is queried. This is consistent with the suggestion of Cai and Schlag (2001) that a continuously moving object may allow a compressed representation. Until the moment of a trigger, motion can be represented and analyzed without the instantaneous position being computed. Recall that a moving object causes a smear of activity across the cortex; therefore, computing an instantaneous position involves an unsmearing process, wherein the system extracts from the smear of retinotopic information a single, crisp view. While representing motion, it is not logically required for the brain to also represent position at every point in time because the mechanisms underlying motion and position are different and dissociable. Take the example of watching traffic flows—in these cases, we clearly see motion but there is no need to assume that the visual system computes the position of each moving car at each moment. Thus, it may be that only when we ask ourselves “where is the red Porsche now?” that we generate an answer.

The neural basis of motion biasing

Where does the localization of a moving object take place, when it does? While motion and localization are likely to interact at various levels in the visual system, a logical place to begin the search is by examining feedback from motion areas (such as MT) to primary visual cortex (V1) because V1 has the most precise representation of local spatial information. The influence of such feedback has been previously suggested by Nishida and Johnston (1999). Fu, Shen, Gao, and Dan (2004) suggested a different version of the feedback story, demonstrating that receptive fields in cat primary visual cortex are displaced by motion signals and suggesting that the effect could be due to asymmetric connections in direction-selective cells contained entirely within V1.

Recently, Sundberg, Fallah, and Reynolds (2006) recorded neuronal responses in monkey area V4 during the flash-jump illusion. They found that V4 retinotopic coding shifted along the motion trajectory in response to the colored flash, in apparent correspondence to the psychophysical effect. However, they found the same retinotopic shift even when the trajectory ended with the colored flash (“flash-terminal” condition), a condition that does not perceptually yield any localization illusion. This pair of findings suggests that the perceptual illusion involves the collaboration of many brain areas, perhaps communicating in a Bayesian manner for the final localization (Sundberg et al., 2006).

We suggest further steps for refining the details of the neural basis of motion biasing. First, Wu and Shimojo (2002) found that when subjects watch a moving object, a well-timed TMS pulse over primary visual cortex will cause the simultaneous perception of two bars: one in the correctly localized position and the other in the motion-shifted position. This supports the view presented here that the localized position is computed with the available data at some point in the system and is then influenced by feedback circuitry. In ways not currently understood, a TMS pulse seems to interfere with this feedback, which causes the unbiased position estimate to be perceived. Thus, although a feed-forward model of the FLE has been proposed (Baldo & Caticha, 2005), a final understanding may require the inclusion of feedback. Further studies interfering with feedback processing (using, e.g., cooling of the cortex in animals) and examining the effects on motion biasing should be revealing.

We have been exploring the simplest version of the model, wherein a motion vector linearly displaces location judgments. However, the magnitude of the motion-biased position shift may depend on the larger context of the scene. For example, in Ramachandran and Anstis’ (1990) demonstration of random dot movement displacing the “window” edges, the motion-position coupling is enhanced if the window region is seen as figure rather than ground. The magnitude of a position shift is also likely to be modulated by attention: For example, the Frohlich effect is reduced if a preceding peripheral cue indicates the starting position of the subsequent movement (Musseler & Aschersleben, 1998). More generally, what remains to be worked out, from a circuitry point of view, are the rules of grouping. For example, it is known that the FLE is modulated by perceptual organization of the moving items—the FLE is larger when a flash is compared against the leading edge of a moving square, and it is smaller if a flash is compared against the trailing edge (Watanabe, Nijhawan, Khurana, & Shimojo, 2001). Clues like these will guide the search for neural mechanisms.

Motion biasing may also involve higher level motion processing, as indicated by the observations that second-order motion shifts perceived position (Bressler & Whitney, 2006), the FLE occurs with illusory moving objects (Watanabe, Nijhawan, & Shimojo, 2002), and the FLE only occurs when the moving stimulus is perceived as a single object (when it becomes another object, the illusion disappears; Moore & Enns, 2004). Further, the FLE has been reported to occur in more generalized spaces, for example, by replacing physically moving object with changing colors (Sheth, Nijhawan, & Shimojo, 2000) or streaming letters (Bachmann & Poder, 2001). These experiments suggest that the same principles of “movement” may apply in different spaces, perhaps reflecting common neural circuits operating in different domains.

Finally, the motion illusions explored here provide an excellent arena in which to study two fundamental questions: (a) Does the brain compute answers only when needed, or does it continuously represent stimulus parameters? (b) Does the timing of neural signals map directly onto the timing of perception? Our results suggest that the answer to the first question is yes and that the answer to the second question is no.

Conclusion

At least four classes of illusion can be explained by the well-known phenomenon that motion signals bias position judgments. We have demonstrated, in a series of experiments, that perceptual localization is influenced by motion signals collected over ∼80 ms after a query is triggered. These experiments rule against temporal models in favor of a spatial model for explaining why moving objects and/or flashes are often mislocalized. As has been suggested elsewhere, motion biasing may exist because it allows the visual system to account for neural processing delays by retrospectively “pushing” an object closer to its true physical location.