Effects of internal and external velocity on the perceived direction of the double-drift illusion

Abstract

In the double-drift illusion, the combination of the internal and external motion vectors produces large misperceptions of both position and direction of motion. Here, we investigate the role that speed plays in determining how these two sources of motion are combined to produce the double-drift illusion. To address this question, we measure the size of the illusion at seven internal speeds combined with six external speeds. We find that the illusion increases with increasing internal speed and decreases with increasing external speed. We model this by combining the external and internal vectors to produce the resulting, illusory direction (Tse & Hsieh, 2006). The relative effect of the two vectors is specified by a constant K in this model and the data reveal that K decreases linearly as external speed increases. This critical role of external speed in modulating the vector combination uncovers new details about how the visual system combines different sources of motion information to produce a global motion percept.

Introduction

Illusory motion-induced position shifts (MIPS) occur when the presence of one or more motion signals causes a divergence between the physical position of an object (i.e. where it is on the display) and its perceived position (i.e. where it appears to be). In some cases, the motion shifts an object's perceived position, as in the flash-lag, the flash-grab, and the flash-drag illusions (Cavanagh & Anstis, 2013; Eagleman & Sejinowski, 2007; Whitney, 2002); in other cases, it also changes the perceived motion speed or direction (Dunker, 1929; Kwon, Tadin, & Knill, 2015; Wallach, Bacon, & Schulman, 1978).

The double-drift illusion (Lisi & Cavanagh, 2015; also called “the curveball illusion,” Kwon et al., 2015; Shapiro, Lu, Huang, Knight, & Ennis, 2010; and “infinite regress illusion,” Tse & Hsieh, 2006) is a dramatic example of this second class of MIPS. Here, two motion signals from a single object are combined to produce a misperception of where the object is and where it appears to be going. The first motion signal, referred to here as the “external drift,” is the displacement of the object itself across the screen. The second motion signal, referred to here as the “internal drift,” is created by a translating internal texture confined within the object's boundary. The two motion vectors combine to produce large misperceptions of position and direction. The magnitude of this illusory effect is usually described in terms of its change in direction, the angle by which the external drift is rotated away from the object's physical direction (Cavanagh & Tse, 2019; Lisi & Cavanagh, 2015; Massendari, Lisi, Collins, & Cavanagh, 2018; see Figure 1A; Tse & Hsieh, 2006).

Figure 1.

(A) Schematic of the double-drift stimulus with external velocity vector at 0 degrees (thick blue arrow), internal velocity vector at 90 degrees (thin blue arrow), and perceived direction at 45 degrees (green dashed arrow). (B) Schematic of the response phase with the red cursor and a response direction (red dashed arrow) that corresponds to the perceived direction. The difference between the external direction and the response direction was used to obtain the angle offset θ, which measures the magnitude of the double-drift illusion.

These significant displacements from an object's true motion and position make the double-drift illusion a powerful tool for addressing questions about the emergence of perception from bottom-up signals. For example, it has been shown that the computation underlying the double-drift illusion must take place in quadrantic regions of visual cortex (i.e. later than V1; Liu, Tse, & Cavanagh, 2018) and after smooth pursuit eye movement signals have been discounted (Cavanagh & Tse, 2019). Although these and other studies (Hui, Wang, Zhang, Tse, & Cavanagh, 2020; Liu, Yu, Tse, & Cavanagh, 2019; Özkan, Tse, & Cavanagh, 2020) demonstrate progress regarding the question where in the brain the crucial computations occur, it remains to be determined how these computations are performed. The present study focuses on one such question: how do the internal and external motion signals combine to produce the illusion?

This question has been addressed before. In 2019, Cavanagh and Tse manipulated the external speeds of the Gabor and showed that the illusion strength decreases smoothly as external speed increases. In an earlier study of this effect, which they referred to as the “curveball” illusion, Shapiro et al. (2010) showed that the magnitude of the illusion increased as the internal speed increased. Earlier still, when Tse and Hsieh (2006) first studied this effect under the name of the “infinite regress” illusion, they showed that the size of the illusion decreased with increasing external speed and that it increased with increasing internal speed. However, the range of external and internal speeds they investigated did not overlap, preventing an evaluation of the contributions of the two motion signals over comparable ranges.

There have been several past efforts to model the integration of the internal and external motion vectors. The results obtained in both Tse and Hsieh (2006) and Cavanagh and Tse (2019) were best fit using a simple vector combination model. Shapiro et al. (2010) performed a motion energy analysis of the stimulus and observed that primarily first-order motion information was consistent with the internal direction and that second-order motion information was consistent with the external direction. They concluded that motion information from the first and second-order systems is segregated when the stimulus is viewed in the fovea, but that they are integrated when viewed in the periphery.

More recently, Kwon et al. (2015) developed an object tracking account of the illusion. Their model optimally integrates two sources of information about the state of the tracked object: (1) measured information from sensory signals, and (2) predicted information from an internal model of motion. When positional information is uncertain, then the measured information from the internal motion propagates into the predicted information about the external velocity and position. In this way, the internal and external velocities are mixed, causing the estimated position (i.e. the perceived position) to deviate from the actual position. The object tracking model accounts for several interesting properties of this illusion, including the roles of the object's boundary and eccentricity, and reveals a type of “motion silencing” for the internal motion. However, the model has not been tested over a range of internal and external speeds.

Here, we measure the size of the illusion at seven internal speeds that overlap with six external speeds. Our data confirm the trend observed by Tse and Hsieh (2006) that the illusion increases with increasing internal speed and decreases with increasing external speed. In addition, our results demonstrate an interaction between the two motion signals that has not previously been reported: specifically, the external speed mediates how the two motion vectors are combined. We then discuss a vector combination model that best accounts for our results.

Methods

Participants

Six adults (2 men and 4 women; mean age = 31 years, SD = 13.8) with normal or corrected to normal vision took part in this experiment. Three of the participants were experienced psychophysical observers, and are authors on this paper, whereas the other three participants had no previous experience with psychophysics experiments and were naïve to the purpose of this study.

Stimuli and apparatus

Due to the unusual circumstances during which these data were collected, caused by the novel coronavirus epidemic in the year 2020, viewing conditions were more variable than is usual for this type of experiment. The experiment was run with MatLab and PsychToolbox (Brainard, 1997) software on three different units of the same laptop type (13-inch MacBooks), with monitors all set at 60 Hz frame rate. Although lighting conditions varied for each participant, chin rests were used to stabilize head motion and viewing distance was measured to be 57 cm.

A black fixation point with a diameter of 0.6 dva was placed in the bottom left corner of the screen, 7 dva below and 9 dva to the left of the screen's center. The stimulus consisted of a Gabor patch with a sigma of 0.8 dva and a spatial frequency of 1 cycle/dva, and was presented against a grey background that matched the mean luminance of the Gabor. To produce the double-drift illusion, a Gabor started at the center of the screen, which was located 11.4 dva peripheral to fixation. The envelope of the Gabor moved in one of 10 directions (0 degrees = horizontally, 36 degrees, 72 degrees, 108 degrees, 144 degrees, 180 degrees, 216 degrees, 252 degrees, 288 degrees, and 324 degrees) at one of six external speeds (1.2 dva/s, 2.4 dva/s, 3.6 dva/s, 4.8 dva/s, 6.0 dva/s, and 7.2 dva/s) paired with one of six internal speeds (same values as the external speeds, for 36 combinations), which moved in the orthogonal direction counterclockwise from the external motion. For example, if the direction of the external motion was 0 degrees, then the internal motion was 90 degrees. To achieve this effect, the orientation of the Gabor was always aligned with the external direction. An additional control condition was included, in which the six external speeds were combined with an internal speed of 0 dva/s, and the angle of the luminance grating within the envelope of the Gabor was randomized. In each trial, the Gabor was presented for 500 ms.

Procedure and design

Participants completed four separate sessions of this experiment, in which a single session consisted of 360 double-drift trials (36 internal/external speed combinations × 10 external directions) combined with 60 control trials (6 external speeds × 10 external directions), for a total of 42 speed combinations. Thus, a single session consisted of 420 trials, resulting in 1280 trials per subject over the course of the experiment. All trials, and consequently speed, direction, and control conditions, were randomly intermixed within a single session.

At the start of a trial, a Gabor appeared in the center of the screen and completed its motion trajectory (see Figure 1A). Participants were instructed to maintain their gaze on the black fixation point during this time, as well as thereafter. After 500 ms, a red response ring (spanning 16 dva in diameter) and a red cursor dot (spanning 1 dva in diameter) appeared in the center of the screen (see Figure 1B). While fixating, participants were instructed to left-click the mouse and drag the cursor along the path that they perceived the Gabor to have moved. After left-clicking, the cursor turned from red to green indicating to the participant that their response had started. When the green cursor intersected with the red response ring, the response ring briefly turned green before disappearing, indicating to the participant that their response had been recorded and a new trial would begin shortly.

Results

For every trial, a value for the offset angle θ was recorded that could range from 0 degrees, to either −180 degrees or +180 degrees. A response with θ = 0 degrees corresponds to a report exactly consistent with the true external direction, whereas responses between 0 degrees and +180 degrees, correspond to reports that are shifted counterclockwise to the true external direction, and responses between 0 degrees and −180 degrees correspond to reports that are shifted clockwise to the true external direction. For each session and for each of the 42 internal/external speed combinations tested during that session, a separate VonMises distribution was fit to the values of θ corresponding to that condition for each participant individually. For example, during their first session, a participant responded to 10 trials (i.e. 10 possible external directions) in which the internal speed was 0 dva/s and the external speed was 1.2 dva/s. A VonMises distribution was fit to those 10 values, and after each of the four sessions was completed, the four VonMises distributions for that participant and condition were averaged together. These averaged distributions, for each subject and for each condition, are plotted in Figure 2.

Figure 2.

Each black curve corresponds to a VonMises distribution fit to an individual subject's responses for one of the 42 trial types, averaged across four sessions. Each row corresponds to a single internal speed and each column corresponds to a single external speed. Within each box, the x axis ranges from -100 degrees to +100 degrees, indicating in degrees how responses deviated from the physical path of the Gabor. In the first row, which corresponds to the control with no internal motion, all response distributions are centered near 0 degrees (red dotted line), indicating that they did not deviate from the physical path. Response distributions that are shifted in the positive direction correspond to conditions that induced a double-drift illusion.

The first row of plots in Figure 2 correspond to the six control conditions in which the internal speed was 0 dva/s. For this row of plots, each participant's response distribution is centered on 0 degrees, indicating that they perceived the Gabor to move along a path consistent with the true external direction. For every other row of plots (i.e. the 36 double-drift conditions), each participant's response distribution appears shifted (to varying degrees) in the positive direction, indicating that they perceived the Gabor to move along a path biased by the internal motion direction. The mean value of these 36 distributions (i.e. the mean angle offset) constitutes a measure of the magnitude of the double-drift illusion for each subject, for each of the double-drift combinations of internal and external motion speeds (see Figure 3)

Figure 3.

Perceived angle offset averaged across participants (n = 6) for each of the 42 internal and external speed combinations. The y-axis is the magnitude of the illusion measured in number of degrees (θ) that the perceived trajectory is offset from the physical trajectory. Each line corresponds to a single internal speed (V_i), as indicated by the number located at the right end of each line, plotted against external speed (V_e) on the x-axis. Error bars represent the standard error of the mean across participants.

Figure 3 is an average of mean angle offset (i.e. central value of the corresponding VonMises distribution) across participants for each condition. Each plotted line corresponds to one internal speed. As external speed increases, the illusion size (i.e. the magnitude of θ) decreases. Furthermore, for the data for each external speed individually, the illusion size increases as the internal speed increases. Additionally, there is a strong interaction between the effects of the internal and external speeds: at the highest internal speeds, the relation between the illusion strength and the external speed is approximately linear. However, at slower internal speeds, the relation is nonlinear.

To test the effects of motion speed on the size of the double-drift illusion, a two-way within-subjects ANOVA was conducted with internal speed and external speed as the two six-level factors (the control condition was not included). The main effects for internal speed (F[5,180] = 157.1, p < 0.001, ${\eta }_p^2=0.81$) and external speed (F[5,180] = 249.0, p < 0.001, ${\eta }_p^2=0.87$) were extremely significant. Furthermore, a significant interaction for internal speed and external speed was found (F[25,180] = 2.14, p = 0.002, ${\eta }_p^2=0.23$). Six simple main effects tests were conducted to test the effect of external speed at each level of internal speed. All six tests where significant (F[5, 30]>17, p < 0.001, ${\eta }_p^2>0.74$). Post hoc tests (Tukey's HSD) revealed significant differences between conditions that reflect the non-linear relationship for slower internal speeds, plotted in Figure 3, compared with the more linear relationship for the faster internal speeds.

Vector combination model

In the two previous papers that explored how the speed of the internal and external motion vectors contribute to the size of the DD illusion (Cavanagh & Tse, 2019; Tse & Hsieh, 2006), the authors considered the simplest plausible model for how the two vectors are combined: a vector combination model. This model assumes that the perceived external motion vector is a combination of the physical internal and external motion vectors according to a set of weights (see Figure 4). That is, for internal and external motion vectors V_i and V_e, and corresponding weights w_i and w_e, the perceived angle θ is given by the model:

$$\theta ={tan}^{-1}\frac{w_i{V}_i}{w_e{V}_e}$$ (1)

Figure 4.

The solid blue arrows represent the directional components of the internal velocity vector (V_i) and the external velocity vectors (V_e). The dashed blue line represents the directional components being scaled by weights w_i and w_e. The red dashed line represents the perceived trajectory, modeled here as resulting from a weighted vector combination of the internal and external velocities. Depending on the values of w_i and w_e, the internal and external velocities are combined to produce different perceived directions, denoted by θ.

Because we only measured the offset angle of the perceived motion vector from the true external motion vector, we cannot separately estimate w_i and w_e, but can only recover their relative value $K=\frac{w_i}{w_e}$. Thus, with the proportion K as the free variable, the model we fit to the present data has the form:

$$\theta ={tan}^{-1}K\frac{V_i}{V_e}$$ (2)

This model with K = 1 gives a good fit to the data (root mean square error [RMSE] = 8.1, Akaike information criterion [AIC] = 152.2) and an even better fit if we allow K to be a free parameter (K = 0.74; RMSE  = 3.6, AIC = 94.4). However, a closer inspection of these predictions reveals deviations that suggest that the relative influence of internal and external speeds might change with speed. Note that the predictions of Equation 2 for the illusion should all be the same where the ratio of  V_e and V_i are constant – we have several cases where this holds in the experiment. For example, there are six points where V_e is equal to V_i (see Figure 4A). If we now plot the data from the conditions with a constant ratio of V_e to V_i, we see that there is a systematic deviation from the expected constant value. The simplest explanation of this deviation is that K decreases with increasing external velocity – the effectiveness of the internal velocity gets relatively weaker as V_e increases. Perhaps this is a simple consequence of low-level visual factors for the registration of motion of a patch that is moving itself – we will cover this in more detail in the discussion.

We therefore fit the data with six independent values of K,   one for each external velocity, and to be fair, with six independent values of K,   one for each internal velocity. Allowing K to vary with the external velocity improved the fit significantly (RMSE = 1.9, AIC = 57.9), whereas independent values of K for the different internal velocities did not improve the fit (RMSE = 3.5, AIC= 103.1). Finally, a linear decrease was clear in the 6 values of K fit separately for each V_e (Figure 5B) so we fit the slope s in the expression K = 1 + s* V_e, obtaining a linear model for K (s  =   − 0.066). This linearly decreasing function of K maintains the good fit of the six independent K values but reduces the number of free parameters to 1 (RMSE = 2.0, AIC = 51.1; see Figure 6). See Table 1 for all model comparisons.

Figure 5.

(A) The size of the illusion, which is plotted for the six conditions when V_i = V_e (black stars; see supplementary videos for demos of these conditions), clearly decreases as speed increases. If K were a constant value such as 1, these values would form a flat line (blue line). (B) Two models were fit, one where a separate value for K was fit for each of the six V_e (red circles) and one where a separate value for K was fit for each of the six V_i (magenta circles). Regression lines (dashed lines) were fit to each set of six values for K. The solid blue line corresponds to K = 1. (C) Here, we plot the model predictions using the regression fits to V_e (red line) and V_i (magenta line) along with the data (black stars). Clearly, the values of K fit to V_e give the better approximation to the data.

Figure 6.

Here, the magnitude of the illusion (y-axis) for our collected responses (colored circles) are shown along with the prediction of the best fit model (colored lines), where variable values of K decrease linearly as V_e increases, with the form K = 1 + s* V_e, with s =   − 0.066.

Table 1.

Root mean squared error and ΔAIC for each model

	RMSE	ΔAIC
K = 1 + s* Ve	2.0
K fit to 6 individual Ve	1.9	7
K fit to 6 individual Vi	3.5	51.7
K = 0.74	3.6	43.0
K = 1.0	8.1	101.1

Discussion

In this study, we systematically varied the speed of internal and external motion vectors in a double-drift stimulus. Our results confirm previous work (Cavanagh & Tse, 2019; Tse & Hsieh, 2006), showing that the magnitude of the illusion, as measured by the deviation of the perceived trajectory toward the direction of internal motion, increases as the speed of the internal motion increases, and decreases as the speed of the external motion increases. However, our results also reveal an interaction between internal and external speeds that has not been previously reported. We show in our best fitting model that the combination of the internal and external motions depends on the speed of the external motion: the relative value of the internal and external weights (i.e. $K=\frac{w_i}{w_e}$) decreases linearly with the speed of the external motion.

Precisely how this dependency arises in the motion processing system is not clear, but there are three obvious possibilities. First, the contribution of the internal speed (i.e. w_i) may decrease as external speed increases. Specifically, at faster external speeds, the Gabor's envelope spends less time at each location, in effect reducing the time for local, motion-selective receptive fields to accumulate speed information. Motion discrimination does in fact decrease as stimulus duration decreases (Borghuis, Tadin, Lankheet, Lappin, & van de Grind, 2019). Second, the contribution of the external speed (i.e. w_e) may increase as external speed increases. Because the external motion of the Gabor is parallel to the orientation of its internal carrier, it is a second order motion that may be analyzed by a global tracking processes that integrates signals from units along the motion path (Cavanagh, 1992). At faster external speeds, the longer trajectory covers more receptive fields, and this larger sample pool may increase the fidelity of the external motion information. Finally, both internal and external weights may change. These are mere speculations at this point, and more experiments are needed to determine the specific mechanism that governs the special role that external speed plays in combining the two sources of motion.

A final potential source of the decrease in K is the possibility that, at higher speeds, the illusory paths may be curved (Kwon et al, 2015) or reset to the physical path (Nakayama & Holcombe, 2020) at some point. Kwon et al. (2015) found that the perceived path was strongly curved for the speeds they used (V_e = 4 dva/s, V_i = 10 dva/s). In their model, the drift in the direction of the internal motion saturated after about 200 ms so the path initially deviated from the physical path but then curved around to run parallel to it. M‘t Hart, Henriques, & Cavanagh (2019) reported both curvature and resets back to the physical path in longer duration trajectories. In contrast, our model and our measurement technique assume a linear path over the 500 ms during which the Gabor was present, and at the slower speeds we use, this may have been the case. However, if the perceived paths in our experiment had any curvature or resets, the apparent direction averaged over the 500 ms presentation time, would show less illusion strength – less deviation from the physical direction. If curvature and resets were more likely at higher external speeds, this might explain some of the reduction in illusion strength and the decrease of K with V_e. The possibility of curvature and resets can be evaluated informally by comparing the first and second part of Supplementary Video S1. In the first part of the video, when V_i = V_e = 1.2 dva/s, perceived paths appear relatively linear. Compare this with the second part of the video, when V_i = V_e = 7.2 dva/s. Here, the perceived path may appear to curve toward or reset to the physical path for some observers.

One of our aims in testing and modeling the double-drift illusion is to maximize the illusion size. Increasing the distance between physical and perceived locations increases our ability to differentiate the representations of the two in brain imaging studies. If a causal account can be given for how the two sources of motion accumulate and when they saturate, it may be possible to engineer greater illusory position displacements. The present findings support these efforts by revealing a piece of the puzzle: the external speed plays a unique role in mediating how the perceived path diverges from the physical path.