The influence of visual motion on perceived position

Abstract

The ability of the visual system to localize objects is one of its most important functions and yet remains one of the least understood, especially when either the object or the surrounding scene is in motion. The specific process that assigns positions under these circumstances is unknown, but two major classes of mechanism have emerged: spatial mechanisms that directly influence the coded locations of objects, and temporal mechanisms that influence the speed of perception. Disentangling these mechanisms is one of the first steps towards understanding how the visual system assigns locations to objects when there are motion signals present in the scene.

One of the fundamental purposes of vision is the localization of objects. Because of continual eye [1] and image motion, the visual system must frequently assign locations to objects that are moving across the retina. To understand how the visual system localizes objects, we must therefore have some understanding of how the visual system processes motion information and whether the assignment of an object’s location is independent of its motion.

Historically, the motion and the position of an object were thought to be independent. Some of the earliest evidence favouring this view was provided by the motion aftereffect (MAE), where prolonged viewing of motion in one direction causes a stationary test pattern subsequently presented at the same location to appear to be moving in the opposite direction [2]. Because motion is perceived without a concomitant change in the position of the object, the two types of information were thought to be processed independently.

More recent research has shown, however, that there is a more complex relationship between an object’s motion and its perceived position. Several phenomena demonstrate that the apparent position of an object can be strongly influenced by the motion of the object and by the motion of other objects in the visual field. The visual system must therefore take into account an object’s motion when assigning its position. Why and how this is accomplished are still not known. Although numerous explanations have been proposed, there is little agreement. Nevertheless, several studies now suggest that motion processing is an integral component of visual localization.

Collectively, these studies confirm the long-held view that motion and position are indeed processed separately. They also show, however, that there are strong interactions between the two types of visual signal: the perceived position of an object is determined by the physical motion of the object, its perceived motion (which might differ from its actual motion), and the motions of other objects in the field.

A matter of time

In the early part of the twentieth century it was generally acknowledged that roughly separate streams process the position and motion of an object. However, this simple distinction was soon complicated by phenomena showing that the motion of an object can influence its apparent position. The question then – and one that still remains – is how the motion of the object influences its apparent position. Initially, explanations revolved around the novel idea that the perceived relative position of a moving object depends on the time at which the object is perceived (e.g. the ‘sensation time’ [3,4]). For example, perceiving one moving object earlier than another adjacent moving object will cause the first to appear ahead of the latter. These early accounts were the first arguments that the temporal coding of the position of a moving object (i.e. latency to perception) can influence its apparent position (for a modern counterargument, see Dennett and Kinsbourne [5]).

The Hess effect

One of the early examples revealing the influence of motion on perceived position is the Hess effect [6], which shows that when two physically aligned objects of differing brightness move in tandem, the brighter can appear to lead the dimmer [6–8] (Fig. 1a; both objects seem to move at the same speed).

Fig. 1.

Several visual illusions that show the influence of visual motion on perceived position. (a) The Hess effect: a high-contrast moving object appears to lead a lower contrast object, although they are physically aligned. (b) The Fröhlich effect: the initial position of an object that abruptly appears in motion (e.g. within a window) is perceived as being shifted in the direction of its motion. (c) The flash-lag effect: a moving object appears in a position ahead of a stationary flash, even though they are presented physically aligned. (d) The illusory misalignment produced by rotary motion: two rotating line segments appear shifted in the direction of their respective motions, creating an illusory misalignment (vernier). All four of these illusions could be caused by temporal mechanisms – the time at which a moving object is perceived determines where it appears relative to other objects in the scene.

A related illusion is the Pulfrich effect [9], in which a swinging pendulum seems to rotate in depth when a neutral density filter covers one eye. The neutral density filter reduces the contrast of the object’s image in that eye, which causes a perceived disparity between the images presented to the two eyes; like the Hess effect, the lower contrast object trails the higher contrast one, creating a perceptual disparity when the two objects are fused.

Most discussions of the Hess and Pulfrich effects have concluded that the illusions are due to the different processing times required to perceive objects of different luminance contrasts [10–13]. High-contrast targets are thought to be perceived more rapidly than lower contrast ones [12,14]. The Hess and Pulfrich effects are important because they show that, for moving objects, a difference in perceptual latency can result in a spatial dissociation. Therefore, it is not simply the location of a moving object on the retina that determines its apparent position; the relative locations of moving objects can be influenced by how rapidly the objects are perceived.

The Fröhlich effect

In the Hess and Pulfrich illusions, it is not the motion of the object that is responsible for the apparent shift in position but the luminance contrast that determines the perceived relative positions of the objects when they are moving. Luminance contrast is not the only characteristic of moving objects that can influence their perceived positions. Fröhlich showed that when a moving object appears abruptly from behind a static aperture, the object’s initial position seems to be shifted forwards in the direction of motion [3]; the initial segment of the object’s trajectory seems to be invisible (Fig. 1b). Because the object has a constant luminance contrast, there must be other properties of the moving object that contribute to its apparent location.

One of the earliest plausible explanations was that the Fröhlich effect is the result of differential perceptual delays for different parts of the moving object’s trajectory [4,15–18]. For example, if the initial position of the moving object had a longer perceptual latency than subsequent positions, we might expect the initial position of the moving object to appear at the same time or even later than subsequent positions of the object. Such a latency account could be realized by specific processes such as attentional shifts [16,18,19] or masking [16,20,21]. An obvious problem with the latency explanation is that it should cause some sort of blurring, where the object appears simultaneously at several positions. A de-blurring [22] or metacontrast masking mechanism [16,20], however, would reduce this artefact. Alternatively, observers might simply ignore the blurred motion, reporting instead its centre of mass or average position.

Despite the lack of consensus about the particular mechanism responsible for the Fröhlich effect, the common theme among most models is that the timing of perception is important; the latency with which the initial position of the moving object is perceived determines where the object appears to be.

The flash-lag effect

In the flash-lag effect, when a flashed stimulus is presented physically aligned with a continuously moving object, the moving object seems to lead the flash [4,23–28] (Fig. 1c). This illusion was, like the Hess and Fröhlich effects, originally explained as being due to differential perceptual latencies for different kinds of stimuli [4,16,24,29–32]. If the flash were perceived later than the physically aligned moving object, one would expect the moving object to appear as if it were ahead of the flash.

Alternative explanations for the flash-lag effect involve the temporal coding of moving objects but do not require explicit differential latencies for moving and flashed stimuli. For example, some authors have suggested that the effect is due to attentional shifts [33], temporal facilitation [34], priming and/or masking [35], temporal averaging of position [36–38], sampling error [39] or asynchronous feature binding [40]. (For more detailed reviews of the flash-lag effect and the ongoing debate, see Krekelberg and Lappe [41] and Murakami [29].)

Recently, some studies have suggested a connection between the flash-lag effect and the Fröhlich effect [16,36] (but see Refs [19,42–44]). Both illusions involve judging the instantaneous position of a moving object, which involves individuating the object in a series of successive presentations. One of the principal differences is the space–time marker used to cue the judgement – a flash in the flash-lag effect and the abrupt appearance of the moving object relative to a static frame in the Fröhlich effect. This might explain why the illusions differ when measured in different ways [43], suggesting that it is not simply the motion of the object that determines the two phenomena but the interaction between the moving object and the space–time marker.

Vernier misalignment of rotating line segments

The Fröhlich and flash-lag illusions depend on judging the instantaneous position of a moving object given a time marker that serves as a cue. Judging the instantaneous position of a moving object can be a surprisingly difficult task, however, because there is an inherent ambiguity when judgements are simultaneously spatial and temporal. Thus, it would be important to show the illusory displacement of a moving object without requiring subjects to select one particular momentary position of the object.

In an elegant study, Matin et al. did just this [45]. When two line segments rotate about an axis, each segment appears slightly shifted in the direction of motion, causing an illusory vernier (i.e. a misalignment between the two lines; Fig. 1d). The illusion is important because, unlike the phenomena mentioned above, the measurement of spatial displacement does not depend on some arbitrary time marker. The absence of the time marker makes the judgement much more reliable and the use of identical stimuli avoids the confound of comparing two different stimuli (i.e. in the previous sections two different stimuli were always compared – a moving object and a reference or two moving objects of differing brightness).

Although there are significant differences between the illusion of Matin et al. and those discussed above, there might be a common explanation for these effects. From the phenomena above, it is clear that processing delays could cause spatial illusions. Similarly, the vernier misalignment produced by rotating line segments might also be due to differential latencies [45]. According to this account, the rotating line segments appear misaligned because of a difference in the neural delays for different parts of the lines. The eccentric parts of the rotating lines travel fast, which means that the unit of energy or luminance over time is smaller. Matin et al. suggest that as the ratio of luminance to time is smaller, the latency is longer, which could make the outer part of the lines appear to trail further behind, compared with the inner part (Fig. 1d). The same result would also occur if the differential latency varied with eccentricity, as is thought to be the case [46].

A matter of space

The general theme among all of the illusions reviewed thus far is that the temporal coding of a moving object’s position can have a dramatic effect on the perceived position of that object relative to other objects. However, recent studies suggest that there is an equally plausible class of alternative explanations that emphasize where the object’s position is coded; the visual system might shift the apparent location of a moving object in the direction of its motion.

The flash-lag effect revisited

Although the flash-lag effect could be due to a temporal mechanism (or a combination of temporal mechanisms), as described earlier, an alternative explanation is that the illusion is due to a mechanism that operates strictly in spatial terms. If the coded position of the moving object were simply shifted forwards along its trajectory of motion, possibly but not necessarily to compensate for neural delays involved in processing the moving object [10,25,47–49], then we would expect the moving object to appear ahead of a flash presented in physical alignment. Note that the representation of the moving object does not need to be shifted forwards by any particular degree; any number of mechanisms could subserve such a shift, and might predict varying degrees of displacement or flash-lag effect.

Berry et al. demonstrated convincingly that there is such spatial shifting in the visual systems of the rabbit and salamander [50] by showing that the peak firing of retinal ganglion cells occurs at the leading edge of a moving stimulus – maximal firing occurs when the receptive field centre is at, or ahead of, the leading edge of the moving object. This anticipatory response varies as a function of contrast, such that higher contrasts result in more anticipation. Understood broadly, this type of spatial shifting mechanism could be invoked to explain the Hess and Matin et al. illusions described above. In the Hess effect, the lower contrast object appears to trail the brighter one. Berry et al. found just this – the higher the contrast of the moving object, the further forward the peak response is shifted in space [50]. Similarly, in Matin et al.’s vernier misalignment produced by rotary motion, the outer line segments move faster than the inner line segments [45]. According to Berry et al.’s results, higher speeds reduce the forward shift, which should produce a misalignment between the two rotating lines (Fig. 1d, right side), just as Matin et al. results show. Of course, care should be taken when comparing physiological data with visual illusions. The flash-lag effect, for example, occurs even if the ‘moving’ object jumps around unpredictably in each frame (i.e. does not move continuously [29]). The results of Berry et al. cannot explain this.

Displacement of equiluminous kinetic edges

Part of the difficulty in interpreting the cause of the illusions above, and part of the reason such diverse models have been proposed to explain them, is because of the spatio-temporal separability of the stimuli. That is, all of the phenomena described above could be due to strictly temporal mechanisms; they could all be caused by stimulus-driven variations in the latency to perception. Two significant papers, however, have shown without doubt that a spatial shifting mechanism does contribute to the perception of a moving object’s position.

Ramachandran and Anstis [49] and De Valois and De Valois [47] showed that the apparent position of a physically stationary aperture or window appears displaced in the direction of the enclosed moving texture (Fig. 2a). Like Matin et al.’s illusion [45], this phenomenon occurs continuously, without requiring a time marker or temporal reference. Moreover, the aperture, or kinetic edge, that appears displaced in the direction of motion is physically stationary, so there is no latency difference to be measured – no temporal mechanism could explain how the stimulus appears shifted in position. (The visual system certainly has ways of translating temporal delays into spatial offsets – Burr [51] and Morgan [52] have found interpolation of stroboscopic motion – but such interpolation could not occur with kinetic edge displacement because there is no actual change in the position of the edge; there is nowhere to interpolate.) The displacement of kinetic edges clearly demonstrates that motion signals attributed to an object not only cause an illusory motion (i.e. motion capture [49,53,54]) but also a positional bias.

Fig. 2.

Additional motion-induced position displacements. (a) Kinetic edge displacement: when a stationary window is filled with moving texture (black squares with arrows), the window appears displaced in the direction of motion. Note that in the actual display, no lines are drawn to denote the ‘windows’; they are defined only by the motion boundary. The regions inside and outside the ‘windows’ have equal average luminance. (b) After prolonged viewing of a moving stimulus in one direction (adapting, left-hand panel), a physically stationary test pattern (middle panel) in the same location appears to move in the opposite direction [motion aftereffect (MAE)]. The dashed arrows in the right-hand panel show the direction of the perceived MAE, which can be accompanied by a concurrent shift in the apparent position of the test pattern (the pattern on the right appears displaced in the direction of the illusory motion). (c) When two stationary flashes straddle two oppositely moving gratings, the flashes appear shifted in the direction of the nearest grating’s motion. These three illusions could not be due to the temporal coding of the displaced object’s position because the objects that appear shifted are in fact stationary. There must be a mechanism that shifts the locations of objects in the direction of the nearest or predominant motion signals.

Motion aftereffect

Snowden [55], Nishida and Johnston [56] and Whitaker et al. [57] showed convincingly that the MAE can be accompanied by a concurrent shift in the apparent position of the physically stationary test pattern (Fig. 2b). The illusory motion of the MAE might therefore contribute to the coding of the location of the test pattern (the motion and position of the object are, however, still thought to be coded by distinct mechanisms, as the time course of the perceived MAE and the position shift are slightly different [56]). The illusion is consistent with the displacement of kinetic edges described above, because neither illusion can be explained by a temporal mechanism; the visual system must employ a spatial shifting mechanism in which the motion attributed to the object directly influences its assigned location.

Remote influence of motion: what counts as a moving object?

In the kinetic edge displacement and MAE illusions described above, motion signals (whether due to physical motion or adaptation) in a local region of space were shown to influence the apparent position of an object in that same region. Does the visual system know that a particular motion signal arises from one particular object, and that only this object’s position should be shifted forward? In other words, does the motion of one object influence the apparent position of a different object?

Theoretically, the visual system could easily extract and link information about a particular object’s motion and its location, and this could determine whether the object is displaced. However, this is not the whole story. When a stationary object is flashed near another stimulus that is moving, the flashed object appears displaced in the direction of the nearby motion; this occurs even when the moving and flashed objects are separated by a substantial distance [58,59] (Fig. 2c). Note that in this case, as with kinetic edge displacement [47,49], the shift in the stationary object’s position could not be due to a temporal mechanism such as those described earlier.

The illusion described here is similar, in some respects, to well-known motion illusions, such as motion capture and induced motion. Motion capture and induction show that the perceived speed of one object depends on the motion of surrounding objects (for a review of induced motion, see Reinhardt-Rutland [60]). Similarly, the present illusion shows that the perceived position of an object depends on motion signals throughout the scene. Evidently, the perceived motion and position of any particular (even stationary) object is influenced by the predominant motion signals throughout large regions of visual space.

Conclusions

The illusions reviewed here demonstrate that there is an intricate relationship between an object’s coded location, its motion, and the motion of objects throughout the visual field. Although the visual system processes the motion and the position of an object separately, the perceived positions of objects throughout the scene depend critically on the collective motion signals that are present.

The mechanism or mechanisms by which positions are assigned to objects in the presence of motion have not been resolved, but two broad classes of explanation have emerged. The first suggests that the timing of perception determines the perceived relative positions of moving or changing objects. The second suggests that the coded location of an object varies systematically with the motion of the object and with the motion of surrounding objects. There is clear evidence for the latter in the form of a spatial shifting mechanism, although this type of mechanism alone is unable to explain all of the phenomena reviewed here (e.g. Refs [29,61]), which suggests that a temporal mechanism might contribute to the localization of moving objects as well. These two types of mechanism could be additive [40], although at present it is unclear whether they act as a unitary spatio-temporal process (e.g. Ref. [62]), as the spatial illusions described earlier could occur independent of any explicit temporal coding. Whether a single spatio-temporal mechanism can provide a grand unified explanation for the influence of motion on position will be a major issue in future research.

The two classes of mechanism are loosely defined, as each could incorporate any number of particular neural implementations. For example, the temporal mechanism could be instantiated by attention shifts [17,18,33], latency differences [30–32], masking [16,35], temporal integration [36,37], and so on. Despite this variety, the class as a whole does share a common feature in that it is the temporal coding of the stimulus that determines the perceptual error. Similarly, a spatial shifting mechanism could be implemented by any number of physiological processes that yield similar output [47,50,62].

There are clear differences between the two classes of mechanism. The spatial shifting mechanism operates on moving and stationary objects, and involves accessing motion signals over large regions of the visual field. The temporal mechanism, however, is thought to operate only locally; it only influences the time at which the (moving) object itself is perceived, not the time at which other objects in the field are perceived (future research will no doubt examine this more closely). It is therefore conceivable that the two mechanisms coexist in a complementary fashion.

The localization of objects is one of the most important functions of vision. The literature reviewed here suggests that motion information plays a crucial role in determining where objects (both stationary and moving) are perceived. There are serious questions about visual localization, however, that remain unresolved (see Questions for future research). One of the primary goals of future research in this area will be to pin down the two major classes of mechanism that have emerged over the last century, and to identify how these mechanisms contribute to numerous motion-induced mislocalizations. More challenging still will be the work of disentangling the mechanisms and revealing how and if they coexist, as a successful model of visual localization must necessarily explain how the visual system processes motion information.

Questions for future research.

• Are spatial and temporal mechanisms involved in coding the location of a moving object? Are they distinct mechanisms?

• What is the difference between the position of a moving object and the position of a stationary object? Are the positions of moving and stationary objects coded by the same mechanism? (This is a different question from whether the motion and position of an object are coded by the same mechanism.)

• Motion influences the perceived locations of objects, but does this have any relevance to behaviour? Are the effects of motion on perceived position also found for action (e.g. pointing movements, see Yamagishi et al. [63])? What is the relation between localization errors caused by retinal motion and those caused by head or body motion [40,64]?

• The illusions reviewed here show the influence of motion on visual localization. What is the relation between these illusions and mislocalizations that involve or depend on memory [65,66], illusory shape deformations of moving objects [8,67–70] and figural aftereffects [71]?

• Do localization errors occur in modalities other than vision? For example, does auditory or tactile motion information influence the apparent position of an auditory or tactile target? Are all of these examples of broader mechanisms that the brain uses to code location?