Optic flow drives human visuo-locomotor adaptation

Summary

Two strategies may guide walking to a stationary goal: (1) the optic flow strategy, in which one aligns the direction of locomotion or “heading” specified by optic flow with the visual goal [1, 2]; and (2) the egocentric direction strategy, in which one aligns the locomotor axis with the perceived egocentric direction of the goal [3, 4], and error results in optical target drift [5]. Optic flow appears to dominate steering control in richly structured visual environments [2, 6-8], while the egocentric direction strategy prevails in visually sparse environments [2, 3, 9]. Here we determine whether optic flow also drives visuo-locomotor adaptation in visually structured environments. Participants adapted to walking with the virtual heading direction displaced 10° to the right of the actual walking direction, and were then tested with a normally aligned heading. Two environments, visually structured and visually sparse, were crossed in adaptation and test phases. Adaptation of the walking path was more rapid and complete in the structured environment, with twice the negative aftereffect on path deviation, indicating that optic flow contributes over and above target drift alone. Optic flow thus plays a central role in both online control of walking and adaptation of the visuo-locomotor mapping.

RESULTS

During locomotion, one’s heading with respect to environmental objects is specified by the optic flow pattern, including the focus of expansion (FOE) and the pattern of motion parallax [1, 10]. Over 40 years ago, Held & Freedman [11] proposed that optic flow provides the necessary information for complete prism adaptation during walking. Even though wedge prisms shift the entire visual scene to one side, the one-to-one relation between the direction of travel through the visual scene and the heading specified by optic flow remains invariant. The discrepancy between the specified heading and the locomotor axis thus provides an error signal that could serve to recalibrate the visuo-locomotor mapping from visual direction to walking direction, to guide walking in the absence of flow [4, 12] (Figure 1A). Subsequently, Rushton & Harris [13] proposed that such a recalibration might be driven by optical target drift, a local aspect of the optic flow pattern that provides a related error signal but depends on target distance (Figure 1B).

Figure 1. Experimental displays, in which the virtual heading direction is displaced 10° to the right of the actual walking direction.

(A) The Texture environment, in which the focus of expansion in the optic flow specifies the virtual heading through the visual scene.

(B) The Line environment, in which a virtual heading to the right of the target produces optical drift of the target to the left.

(C) Optic flow strategy (plan view): To keep the virtual heading aligned with the target (as in A), the person “crab walks” by stepping forward and slightly left of the target. This produces a straight path in the Texture environment. Time steps are color coded in the diagram.

(D) Egocentric direction strategy (plan view): Walking in the egocentric direction of the target causes the target to drift (as in B), yielding a path that curves around to the left as the walker “chases” the target. The inset plots the increasing drift rate as the target is approached at a typical walking speed of 1 m/s. (Dotted line at 4 deg/s is the threshold at which target motion influences locomotion [23].

We used an ambulatory virtual environment to test these hypotheses in an adaptation and transfer paradigm. In the adaptation phase, participants walked to a target with the heading direction in the virtual environment displaced by 10° to the right of the actual walking direction. In the test phase they were transferred to normal conditions in which the virtual heading was aligned with the actual walking direction (0° displacement), to test for a negative aftereffect in the walking path. Others have previously reported aftereffects of prism adaptation during walking [14-18]; here we provide the first study of the course of adaptation in the walker’s path while varying visual structure.

To compare adaptation based on optic flow and target drift, we tested two virtual environments: (a) the Texture environment (Figure 1A), consisting of a texture-mapped room and posts with a target doorway, which provided rich optic flow (including motion parallax and target drift), and (b) the Line environment (Figure 1B), consisting of only a vertical target line, which provided target drift without surrounding flow. Half the participants received the same environment in the adaptation and test phases, while the other half switched environments, creating a 2 (adaptation environment) × 2 (test environment) design with four groups of 10 participants each. Following Held & Freedman [11] we hypothesized that optic flow would yield more rapid and complete adaptation than target drift alone. An analysis of the heading error reveals different contributions to adaptation of the initial direction of walking and the ongoing walking direction.

Adaptation Phase

During adaptation, the virtual heading was displaced 10° to the right of the locomotor axis. Consequently, walking in the egocentric direction of the target would produce a “virtual heading error” of +10°, causing the target to drift leftward; this would yield a path that curved around to the left as the participant “chased” the target (Figure 1D). Participants could learn to compensate by nulling target drift, thereby walking to the left of the target on a straighter path. Alternatively, participants in the Texture environment could use the surrounding optic flow to compensate by nulling the error between the visually specified heading and the target (e.g. placing the FOE on the target), yielding a straighter path (Figure 1C).

The effect of visual structure in online steering control was evident on the first adaptation trial. The path was highly curved in the Line environment, close to the egocentric direction prediction, but much less curved in the Texture environment (Figures 2A and 2B), with half the mean lateral deviation, F(1,38) = 35, p < .0001. On the second and third trials in the Texture environment, the path straightened after 2-3 m of walking once optic flow became available, whereas in the Line environment the path remained highly curved. This replicates a previous finding from our laboratory [2] that people primarily rely on optic flow for the online control of locomotion in richly structured visual environments.

Figure 2. Adaptation phase results.

(A, B) Plan view of walking paths in the Texture and Line environments, showing the mean paths for the first three adaptation trials (blue curves) and the mean of the last three trials (red curve). Dotted curve (black) is the prediction of the egocentric direction strategy, Y-axis corresponds to the prediction of the optic flow strategy.

(C, D) Mean virtual heading error as a function of distance in the Texture and Line environments. Dotted line is the egocentric direction prediction, solid line is the optic flow prediction; shading corresponds to the 95% confidence interval for the last three trials based on between-subject variability.

The structured environment also produced more adaptation than the drifting target alone (Figure 3A). Over trials, the mean lateral deviation was smaller in the Texture environment, F(1,38) = 48, p <.0001, though it decreased significantly in both environments, F(37,1406) = 7.1, p < .0001. The relative adaptation (the mean lateral deviation on the last three trials as a percentage of the first trial) was 52% in the Texture environment and only half that (28%) in the Line environment. Fitting the lateral deviation over trials with a first order exponential decay function revealed that the rate of adaptation in the Texture environment was more than six times that in the Line environment (parameter C in Table 1). Thus, by several measures optic flow in the Texture environment yields much greater and more rapid adaptation than does target drift in the Line environment.

Figure 3. Time course of adaptation and recovery, based on mean lateral deviation as a function of trial number.

(A) Adaptation phase: adaptation is more rapid and complete in the Texture environment (blue circles) than in the Line environment (red rectangles). Shading corresponds to the 95% confidence interval for the mean deviation; solid curves represent an exponential fit of the decay in lateral deviation (see Table 1).

(B) Test phase: post-adaptation recovery is more rapid and complete in the Texture environment than in the Line environment (crossed symbols indicate groups that switched environments between test and adaptation phases). Error bars indicate the 95% confidence interval for the negative aftereffect in Trial 1, based on between-subject variance. Curves represent an exponential fit of the decay in lateral deviation (see Table 1).

Table 1.

Exponential fits of decay in lateral deviation over adaptation and test trials

Condition	A	B	C	r²
Adaptation phase
Line	31.9	11.3	0.092	.71
Texture	12.7	20.2	0.64	.58
Test phase
Texture/Texture	-2.8	-13.0	-.50	.63
Texture/Line	-13.4	-20.2	-.26	.95
Line/Texture	.5	-22.9	-1.1	.69
Line/Line	-8.7	-7.4	-.31	.47

Equation: Y(t) = A + B * e^(-C*t) where Y is the lateral deviation per trial, t trial number, A asymptotic value of Y, B initial deviation, and C rate of decay; r² is the proportion of variance accounted for.

A detailed analysis of the virtual heading error (the angle between the virtual heading and the visual target) as a function of distance reveals adaptation in both the initial walking direction and the ongoing walking direction (Figures 2C and 2D). First, the initial heading error at trial onset reflects the mapping from target direction to initial walking direction. On the first two trials participants started walking in the target’s egocentric direction in both environments, such that the initial heading error was not statistically different from the egocentric prediction of 10°. But in the Texture environment the initial error decreased significantly on the third trial, t(19)=4.73, p<.0001, and by the last three trials it had dropped to 4.6°, significantly different from the first two trials, t(19)=2.3, p<0.05. This shift in the mean initial walking direction was 45% of the imposed flow displacement. On the other hand, in the Line environment the initial heading error never changed statistically from the egocentric prediction of 10°, indicating that adaptation in the initial walking direction is driven by optic flow, but not by target drift.

Second, the steering adjustment that occurs during the first 3 m of a trial reflects the control of ongoing walking. In the Texture environment, this adjustment reduces heading error significantly in trial 1 (p<.05), to about 2° in trial 2, and doesn’t change after that; we attribute this to online control from optic flow. But in the Line environment, the 3 m steering adjustment doesn’t appear until trial 8 and doesn’t stabilize until trial 20; this reflects a gradual adaptation of about 27% in the mapping from target direction to walking direction during ongoing locomotion. Over all trials, the heading error at 3 m was significantly smaller in the Texture than in the Line environment, t(38)=12.5, p<.0001, indicating greater steering accuracy in the presence of optic flow.

Test Phase

The adaptation effects were mirrored by negative aftereffects in the test phase. Held’s optic flow adaptation hypothesis predicts that the largest aftereffect should occur after adaptation in the richly structured environment. Indeed, the first test trial in the Adapt Texture/Test Line condition revealed a highly curved path in the opposite direction (Figure 4A), with a larger lateral deviation than any other test condition (adaptation environment × test environment interaction, F(1,36) = 4.5, p < .05). The relative aftereffect was twice as great in Texture adaptation as in Line adaptation conditions: 65% in Texture/Line and 51% in Texture/Texture, compared to 33% in Line/Line and 22% in Line/Texture (computed as the mean lateral deviation in the first two test trials as a percentage of that in the first two adaptation trials).

Figure 4. Test phase: Negative aftereffects.

(A-D) Plan view of walking paths for each combination of Adaptation and Test conditions, showing the mean paths for the first three adaptation trials (blue curves) and the mean of the last three trials (red curve).

(E-H) Mean virtual heading error as a function of distance in the corresponding conditions. Shading corresponds to the 95% confidence interval for the last three trials, based on between-subject variance.

These aftereffects were abolished over the first few test trials in the Texture environment (Figure 4C and 4D), but not in the Line environment (Figures 4A and 4B). The pattern of post-adaptation recovery is represented in Figure 3B, showing an overall decrease in mean lateral deviation in all conditions, F(9,324) = 9.0, p < .0001, but also significant differences between conditions, F(1,36) = 21.0, p < .0001. The decay rate during recovery was more than twice as fast in the Texture test conditions (parameter C in Table 1) and asymptoted near zero deviation (parameter A), whereas recovery was incomplete in the Line test conditions, with large aftereffects remaining at asymptote. This indicates that the visually structured environment was also more effective in overcoming adaptation, presumably due to the presence of optic flow.

Detailed analysis of virtual heading error as a function of distance reveals specific aftereffects of adaptation in both initial walking and ongoing walking (Figures 4E-H). First, there was a negative aftereffect in the initial heading error on the first three test trials in the Texture adaptation conditions (Figures 4E, and 4G), t(19)=4.4, p<.001, but not in the Line adaptation conditions (Figures 4F, and 4H). We interpret this to indicate that optic flow drives adaptation in the initial walking direction, whereas target drift does not. (Note that this aftereffect was also abolished in the Texture/Texture but not in the Texture/Line condition.)

Second, there were also negative aftereffects in the 3 m steering adjustment on the first three trials in the Line test conditions (Figures 4E, and 4F): 29% of the heading displacement in Texture/Line, t(9)=3.1, p<0.05, and 19% in Line/Line, t(9)=2.0, p=.07. We interpret this to indicate that both optic flow and, marginally, target drift produce adaptation in the mapping from target direction to ongoing walking direction. Over subsequent trials in the Texture/Line condition (Figure 4E), the 3 m aftereffect recovers gradually but completely, which we attribute to readaptation produced by target drift. Finally, the absence of the 3 m aftereffect in the first few Texture/Texture test trials (Figures 4G) is consistent with the use of optic flow for online steering control.

Discussion

The present results indicate that optic flow in visually structured environments plays a central role in visuo-locomotor adaptation as well as in online steering control. Both optic flow and target drift contribute to adaptation, but optic flow yields faster and more complete adaptation (and readaptation) than target drift alone, with much larger negative aftereffects. This is not surprising, for target drift is but a local aspect of the global flow pattern. Moreover, optic flow adapts both the initial and ongoing walking direction, whereas target drift adapts only ongoing walking.

First, consider the initial walking direction: optic flow produced rapid adaptation, yielding a total change of 45% in initial heading error. This reflects some combination of adaptation in the perceived egocentric direction of the target and in the mapping from perceived direction to initial walking direction. To check for adaptation of the perceived straight-ahead, we measured head orientation when participants faced a distant marker at the beginning of each Texture trial. The mean angle to the marker was 0.05° (SD = 1.0°) in the first third of adaptation trials, 0.04° (SD = 0.7°) in the second third, and 0.07° (SD = 0.7°) in the final third, revealing no significant shift in perceived egocentric direction. Thus, adaptation in initial walking can be entirely attributed to recalibration of the visual-motor mapping from perceived target direction to initial walking direction. In contrast, target drift produced no such effect, perhaps due to its low optical velocity at the starting distance (Figure 1D inset).

Second, consider ongoing locomotion: target drift produced gradual adaptation in the mapping from target direction to ongoing walking direction, with a total change of 27% in the 3 m steering adjustment and a marginally significant aftereffect. But optic flow produced even greater adaptation, as revealed by a larger aftereffect. Finally, the results confirm our previous finding [2] that optic flow plays a primary role in online steering control in richly structured visual environments. This is supported by the significant 3 m adjustment in the first few Adaptation trials and the abolition of the 3 m aftereffect in the first few Test trials.

We thus interpret the guidance of walking in visually structured environments as follows. On a given trial, the participant starts walking in a direction determined by the perceived egocentric direction of the target and the mapping to initial walking direction. This generates optic flow, which visually specifies a heading direction relative to environmental objects; it is quickly used to initiate an online steering adjustment, bringing the specified heading closer to the target. The optic flow also acts to recalibrate the visual-motor mappings from perceived target direction to initial and ongoing walking directions; these affect the initial walking direction on subsequent trials, and ongoing walking direction in the absence of optic flow.

In contrast, Rushton and colleagues [4, 19] have argued that steering control relies exclusively on the egocentric direction strategy, and that optic flow only plays an indirect role by influencing perceived egocentric direction. First, they suggest that displacing the FOE yields an immediate shift in the perceived straight-ahead, accounting for the 3 m steering adjustment. Indeed, Wu et al. [20] reported that the judged straight-ahead shifts by 28% of the flow displacement. However, this would account for only 2.8° of our observed 7° steering adjustment (Texture adaptation trials 2 and 3).

Second, Rushton and colleagues suggest that the optical drift created by displacing the FOE also yields an immediate shift in the perceived straight-ahead. Such a shift has indeed been observed in a rotating drum, but with a magnitude of about 1.5° at a drift rate of 5° s^-1 [21] and only after a latency of 10 s or more [22]. This cannot account for the 7° steering adjustment we observe in the first 2 s of a trial, at drift rates below 2° s^-1.

Third, they argue that exposure to displaced flow leads to rapid adaptation of the visual straight-ahead. However, we obtained no such effect in about 6 min of adaptation, and thus adaptation of the straight-ahead does not contribute to our observed steering adjustment. On the other hand, Redding & Wallace [16, 17] observed that the visual straight-ahead shifted by 9.3% of the prism displacement after 10 min of adaptation while walking in a hallway, and Held & Bossom [14] found a similar 10% shift in the straight-ahead even after two hours of walking on a tree-lined path with displacing prisms. Yet these findings would still account for only 1° of our steering adjustment. In sum, the magnitude of these effects could only explain a fraction of the 3 m steering adjustment, and hence our results confirm a central role for optic flow in on-line steering control.

We conclude that optic flow is simultaneously used to control online locomotion and to recalibrate the visuo-locomotor mapping in structured environments. The combination of optic flow and egocentric direction strategies thus provides a coherent account of both the control and adaptation of visually guided locomotion.

Experimental Procedures

Participants (n=40) walked freely in the 12 m × 12 m Virtual Environment Navigation Lab while wearing a head-mounted display (HMD), which projected stereo images generated at 60 Hz (a shield blocked peripheral vision). Head position was recorded at 30 Hz and used to update the display with a latency of about 70 ms. We created two virtual environments to vary the visual structure of the scene (Fig. 1). In the Line environment, the target was a vertical red line with a diameter of .008 m that ran off the top and bottom of the display, against a black background. In the Texture environment, the target was a doorway 0.5 m wide in a frontal wall of a virtual room with a ground plane and ceiling; these surfaces were mapped with a grayscale random noise texture. In addition, an array of vertical blue textured posts (.05 m diameter) was randomly positioned on the ground, with approximately 19 posts visible in the first frame. This visually structured environment provided rich optic flow, including motion parallax, to a walking participant.

The virtual heading was dissociated from the physical heading by displacing the simulated direction of travel through the virtual environment to the right of the actual direction of walking in the physical environment by 10°. Thus, an observer walking in the egocentric direction of the target would see an optic flow pattern that specifies a heading 10° to the right of the target, equivalent to a virtual heading error of 10°. Steering to place the specified heading on the target would reduce the virtual heading error to 0° (Fig. 1A,C).

We recorded 38 adaptation trials to measure the course of adaptation to the displaced virtual heading, immediately followed by 10 test trials to assess a negative aftereffect of adaptation and the rate of post-adaptation recovery. Before each trial participants positioned themselves at a starting post and turned to face a distant marker. They began walking when the marker disappeared and the environment appeared, with the target at an initial distance of 9 m. Participants were instructed to walk at a normal pace to the target and to take the most direct path. In the Texture environment they were explicitly told to walk through any blue posts that obstructed a direct path to the goal. Prior to the experimental trials, participants were given 5 practice trials with a normal alignment of the virtual heading to become familiar with this experimental procedure and to get acquainted with walking in a virtual environment.

The head position data were filtered with a 2^nd order Butterworth filter (0.6 Hz cutoff) to reduce the effects of gait oscillations. The first .25 m and last .50 m of each path were removed to avoid transients, and the intervening samples were normalized in space by divided them into 35 segments, each about .25 m in length; these were ensemble averaged to yield a mean path 8.25 m long. Statistics were computed on the segment means for analysis of the virtual heading error and lateral deviation of the paths (the latter was limited to the 1 m to 8 m section of the path). The research protocol was approved by Brown’s Institutional Review Board.