Proprioception gates visual object fixation in flying flies

SUMMARY

Visual object tracking in animals as diverse as felines, frogs, and fish supports behaviors including predation, predator avoidance, and landscape navigation. Decades of experimental results show that a rigidly body-fixed tethered fly in a “virtual reality” visual flight simulator steers to follow the motion of a vertical bar, thereby “fixating” it on visual midline. This behavior likely reflects a desire to seek natural features such as plant stalks, and has inspired algorithms for visual object tracking predicated on robust responses to bar velocity, particularly near visual midline. Using a modified flight simulator equipped with a magnetic pivot to allow frictionless turns about the yaw axis, we discovered that bar fixation, as well smooth steering responses to bar velocity, are attenuated or eliminated in yaw-free conditions. Body-fixed Drosophila melanogaster respond to bar oscillation on a stationary ground with frequency-matched wing kinematics, and fixate the bar on midline. Yaw-free flies respond to the same stimulus by ignoring the bar, and maintaining their original heading. These differences are driven by proprioceptive signals, rather than visual signals, as artificially “clamping” a bar in the periphery of a yaw-free fly has no effect. When presented with a bar and ground oscillating at different frequencies, a yaw-free fly follows the frequency of the ground only, whereas a body-fixed fly robustly steers at the frequencies of both the bar and ground. Our findings support a model in which proprioceptive feedback promote active damping of high gain optomotor responses to object motion.

eTOC blurb

Rimniceanu et al. show that, in Drosophila melanogaster, visual object tracking behavior operates differently in rigidly tethered flies and flies tethered to a magnetic pivot free to steer in yaw. This changes the way we think about object vision.

INTRODUCTION

The survival and success of animals is dependent on approaching or avoiding relevant features in their environment. Vertically elongated edges form conspicuous visual navigational landmarks indicating ethologically important objects such as plant stalks. A clever experimental manipulation developed in the 1960’s - in which the steering torque generated by a tethered fly controlled a servo motor that rotated a cylinder around the fly - revealed active centering or “fixation” of a vertical bar on the animal’s visual midline ^1,2. Frontal bar fixation persists with luminance-defined moving bars (dark or bright on uniform background), with motion-defined bars (sometimes called “fourier bar”, randomly textured to match the ground), and even when the bar is presented against a ground moving in the opposite direction, independently from the moving bar ^3-5. In fact, this object orientation response is so powerful that a single fly, with sufficient in-flight refueling, will actively center a vertical bar continuously for up to 36 hours ⁶.

The robustness and experimental tractability of bar fixation behavior inspired a classical theoretical model of object orientation driven by two variables: the static azimuthal position of the bar, and its motion, or image velocity ^2,7,8. Whereas tracking can be elicited with bars containing no net motion energy, or bars that present surface motion oriented opposite bar motion, responses to velocity-independent cues such as these are always smaller in magnitude and delayed by comparison to velocity-dependent responses⁹. Motion-elicited tracking responses are strongest near the fly’s visual midline ^3,10,11, and flies with genetically silenced motion detecting neurons show diminished fixation capability in flight, supporting the role of image velocity for active bar fixation ¹².

Under normal flight conditions, the central nervous system integrates motion cues from visual and proprioceptive sources as the fly maneuvers (Figure 1A). Bar fixation is typically studied with a fly rigidly fixed to a stiff wire ¹³. An optical readout of wing kinematics by the body-fixed fly is used to control an electronic visual display to revolve at a velocity proportional to the steering effort (Figure 1B), while the body remains stationary. A newer innovation for more naturalistic flight control has a fly tethered to a magnetic pin suspended within a magnetic field on a nearly frictionless pivot ¹⁴. In contrast to “body-fixed” flies, we refer to flies in this paradigm as “yaw-free” because the body can rotate freely in the horizontal plane, thereby providing real closed loop feedback conditions in yaw for both vision and proprioception feedback channels (Figure 1C) ^15,16. Two recent studies indicate that as a bar crosses visual midline in the yaw-free paradigm the fly’s heading does not seem to follow bar motion ^17,18. Rather interestingly, flies seem to ignore bar velocity within the very visual field that generates the strongest responses by body-fixed flies. To probe this paradox, we asked whether the fly’s proprioceptive state, body-fixed vs yaw-free, directly influences velocity dependent control of bar fixation. Given the importance of image velocity for models of object detection, such a finding would change our understanding of the mechanisms underlying visual object tracking behavior.

Figure 1. Sensory feedback conditions vary across rigid and magnetic tethering paradigms.

A. (left) Conceptual block diagram of signal flow in the visual and proprioceptive modalities. Flight behavior generates feedback within sensory pathways. These channels can be opened or closed in different experimental paradigms. (right) Visual sensory feedback is primarily relayed through the eyes while proprioceptive feedback is relayed through multiple sensory structures covering the entire body, such as wings, halteres and aristae of the antennae. B. In a rigid tether arena, the fly’s body is fixed to a stationary pin, the fly is illuminated with infrared light from above and wing steering kinematics are measured using a photodiode below. The difference in wing beat amplitude across the two wings (ΔWBA) is representative of steering torque, and can be coupled with variable gain to control the velocity of visual patterns such as a textured “motion-defined” bar on the LED display (outlined in dotted white line). In this condition the visual feedback channel is artificially closed but the proprioceptive feedback channel remains open since the fly’s body is stationary. C. In a magnetic tether arena (top), the fly’s body is glued to a magnetic pin suspended between magnetic north and south poles. The fly’s body is free to rotate in the yaw plane and orient toward visual cues. In this condition, the proprioceptive and visual feedback channels are both closed in the yaw plane of motion (bottom). The fly’s instantaneous angular heading in the 360° arena is recorded with a high speed camera from below the animal.

Recent work used a systems identification approach to show how the control of head movements, which stabilize gaze upon the visual ground, are modulated by the yaw-free state ¹⁹. Here, we expand this framework to examine how body state influences active object orientation. We make comparisons of well-known and robust large-field compensatory optomotor responses and small-field bar orientation behaviors across proprioceptive body state. We selectively open and close yaw feedback channels (Figure 1A) to show for the first time that yaw-free flies show weak or no velocity-driven smooth tracking responses to bar motion along the visual azimuth. We show that the influence of the fly’s body state on bar tracking is due to proprioceptive signals rather than visual feedback. Although the precise source of the relevant body movement signals remains speculative, our results support a model that has emerged several times in the literature in which intact proprioceptive signaling is necessary to actively damp optomotor control ^19-21. We build upon this model by empirically demonstrating that when yaw proprioceptive feedback is compromised, classical “frontal fixation” behavior emerges from saturating optomotor responses. However, under more naturalistic body movement conditions, bar tracking is achieved by means other than frontal fixation.

RESULTS

Yaw-free flies do not frontally fixate vertical bars

We developed an experiment to test the influence of body state on canonical active bar fixation. We first confirmed prior findings by evoking classical frontal bar fixation with the rigid tether paradigm in which the visual feedback loop was closed artificially (Figure 1B). We used a motion-defined bar stimulus in which randomly distributed ON-OFF pixels matched the ground thus making it distinguishable from the visual panorama only by relative motion, not brightness or any other parameter (see Methods). The bar was programmed to oscillate +/− 30° at constant velocity on a triangle waveform, and was presented to the fly at the start of each trial at one of three azimuthal positions: 0° (midline), +60°and −60°. We chose a relatively small oscillation amplitude, and short duration trials in order to examine smooth steering responses while evoking minimal saccades, which motion-defined bars readily evoke over long trajectories in yaw-free flies ^22,23.

In the body-fixed paradigm, the fly’s wing steering kinematics controlled the oscillating bar embedded within the ground. If the fly flew straight, the ground was stationary and the bar oscillated about a fixed azimuthal position. If the fly attempted to turn right, then the coupling electronics shifted the oscillating bar and ground pattern together to the fly’s left. Visually, the flies were in an artificial closed loop while the proprioceptive feedback channel was open. By contrast, in the yaw-free paradigm, if the fly turned right, its body rotated, thereby shifting the oscillating bar and ground together to the left (Figure 1C). In the yaw-free paradigm, both the visual and proprioceptive feedback channels were closed about the yaw axis. As our study aimed to investigate small-field bar-directed responses, we did not restrain the head because prior work demonstrated that head movements do not follow small-field bar stimuli ²⁴. Since the head was free to move, we report the position of the bar relative to the major body axis rather than its retinal position.

Given virtual control over the angular position of the oscillating bar, body-fixed flies produced steering effort towards it, as expected. A space-time plot and sample response trajectories of the visual stimulus relative to the fly’s body axis demonstrate typical “jittery” movement of the bar and the ground as the fly produces continuously variable steering responses to both (Figure 2A right, C,E,G left). Body-fixed flies generally steer towards the bar, thereby bringing it to visual midline even within the brief 4-second trail (Figure 2C,E,G middle). Population histograms show that regardless of where the bar initially appeared (orange histograms), the bar position was most frequently positioned at 0° by the end of the trial (blue histograms Figure 2C,E,G right). In stark contrast, a space-time plot and individual sample traces demonstrate that yaw-free flies do not track the bar oscillation (Figure 2B right, D,F,H right). Nor do yaw-free flies produce steering jitter, but rather fly straight between saccades. Remarkably, yaw-free flies do not orient towards the oscillating bar (Figure 2D,F,H middle). We quantified fixation behavior by summing the probability of bar position within 60-degree bins at the start and end of the trials, we found significant differences only for body-fixed flies, indicating that the bar was centered on midline by the end of the trial regardless of its starting location (Figure 2C,E,G dot plots). By contrast, we found no significant differences between the start and end of the trial for yaw-free flies (Figure 2D,F,H dot plots). Our results demonstrate that whereas body-fixed flies actively fixate a bar oscillating on a stationary ground, yaw-free flies do not.

Figure 2. Active bar fixation is body-state dependent.

Comparison of body-fixed artificial closed loop conditions in which steering effort of the stationary fly moves the bar (left panels) and yaw-free closed loop experiments in which steering effort moves the fly (right panels). A. (left) Cartoon of body-fixed experimental paradigm. (middle) Constant velocity trajectory of a motion-defined bar. (right) Example space-time plot of bar and ground motion as seen from the fly’s visual midline. B. Similar to A for a fly in a yaw-free paradigm. C. Body-fixed artificial closed loop bar fixation for a bar starting on visual midline. (left) Example traces of bar position for 3 trials each (same color) from 3 flies (3 colors). (middle) Heatmap plots (N = 19, flies, n = 97 trials) of bar position. Colormap indicates the probability of the bar occupying each bin (1ms x 5°). (right, top) Normalized probability histograms are computed from the first (yellow bracket) and last (blue bracket) 0.5 s of each trial. Dotted line indicates visual midline. (right, bottom) The summed probability of flies placing the bar within 30° of the frontal (0°) or lateral (−60° or 60°) angular position bin at the start (yellow) and end (blue) of the trial. Each dot is an individual fly sum. * = p<0.05; ** = p < 0.01; *** = p < 0.001. D. (right) Yaw-free example traces of bar position relative to visual midline in 3 trials (same color) from 3 individual flies (3 colors), presented with a bar oscillating on midline. Saccades are indicated by black arrowheads. (middle) Heatmap plots (N = 24, n = 106 trials) for bar starting on midline (bins 10 ms x 5°). (left, top) Normalized probability histograms at the start and end of trials. E. Same as C for a bar starting 60° to the left of midline (N = 19, flies, n = 71 trials). The probability of flies placing the bar in the 0° bin increases from the start to the end of the trial (Kruskal-Wallis test). F. Same as D with a bar starting 60° to the left of the fly (N = 24, n = 99 trials) G. Same as C for a bar starting 60° to the right of midline (N = 19, flies, n = 104 trials). H. Same as D with a bar starting 60° to the right of the fly (N = 24, n = 80 trials). Note that the probability of flies placing the bar in the 0° bin at the end of the trial is not significantly higher than at the start. See also Figure S1.

Yaw-free flies do not smoothly track bar dynamics

A central tenet of frontal fixation is asymmetric smooth steering responses to bar motion about visual midline, which therefore visually centers the object ¹². In order to test for body-state effects on bar steering dynamics, we reproduced prior results in which body-fixed flies produce robust directional wing steering responses to a motion-defined bar ^5,11. We presented open-loop constant-velocity bar oscillations across a stationary ground (triangle wave stimuli). Wing steering traces were filtered to remove any slow steering effort toward the bar’s position, as well as saccades, in order to quantify responses to 2 Hz bar oscillation. In addition to trials with the bar positioned at −60°, 0°, and 60°, a 360° large-field ground oscillation condition was added to normalize each fly’s bar response to their maximum optomotor response (Figure 3A). Note that the visual ground is a large-field stimulus because it subtends a large portion of the visual field, whereas the bar is a small-field stimulus. As expected, bar motion elicited robust smooth steering responses in body-fixed flies, with the largest amplitude at midline (Figure 3B)¹¹. We observed bilateral asymmetry in the average response trajectory for the bar at +/− 60°. We have observed similar asymmetries before, which are not an artifact of the electronics but rather invert under some visual conditions; as yet we do not understand their basis.

Figure 3. Smooth bar tracking dynamics are body-state dependent.

A. Depictions of motion-defined bars oscillating about different azimuthal positions (blue = 60° left of midline; yellow = midline; red = 60° right of midline) and randomly textured large-field ground stimulus (purple). B. Open-loop responses of body-fixed flies (N = 16) to bars at three azimuthal positions as indicated. Data were high-pass filtered in order to remove slow DC steering offsets. Note that bar-elicited steering responses are ~60% the amplitude of ground-elicited responses. Shaded envelopes around solid traces represent standard deviation of mean population responses. Gray bands highlight alternate stimulus cycles. The zero ΔWBA position is indicated by a dotted gray line. C. Closed-loop responses of yaw-free flies (N = 20) to the same stimuli as B. Saccades are eliminated from these traces to isolate inter-saccadic bouts in which the bar is in a near-constant position relative to the fly’s body axis. D. Body-fixed (FFTs) of steering responses in B, color coded for bar position. E. Same as D but for yaw-free conditions. F. Ratio of bar responses to ground responses, compared between body state conditions, color coded for bar position. Each circle represents an individual fly. Unpaired two-sample t-tests were performed with * = p<0.05; ** = p < 0.01; *** = p < 0.001. See also Fig S2.

As the steering responses were roughly sinusoidal to the constant velocity stimuli, we used a frequency domain analysis to quantify response strength. For each fly, we determined the magnitude of the Fast Fourier Transform (FFT) peak for bar movement at 120°/s (2 Hz). In body-fixed flies, the bar stimulus elicited steering responses approximately half as large as those elicited by the ground stimulus (Figure 3B, D, yellow). In contrast, we show that the same bar stimulus evoked essentially zero smooth body dynamics in yaw-free flies, despite an intact ground optomotor response (Figure 3C, E). Comparisons between bar responses normalized to the ground response revealed that body fixing results in a significant increase in bar smooth tracking for bar placement at any of the three azimuthal positions (Figure 3F, Wilcoxon Rank-sum test, p<0.01). This difference in smooth steering responses persisted for slower 30°/s (0.5 Hz) trials, suggesting that this phenomenon is independent of bar velocity (Figure S2). Body-fixed flies smoothly track bar motion; yaw-free flies do not.

In this experiment, body-fixed flies were in both visual and proprioceptive open loop conditions, which precludes isolation of the sensory modality that might govern body state effects on visual object tracking. We therefore designed a complementary experiment in which flies experienced the proprioceptive signals associated with yaw-free flight, but we used a tracking camera and high resolution digital projector to immobilize the visual scene under artificial open-loop. Prior work using a similar approach showed that artificially triggering large-field visual rotation during yaw-free body saccade has no influence over the trajectory of the saccade, thereby highlighting the role of proprioceptive rather than visual control in terminating a saccade ¹⁵. Our advance was to dynamically “clamp” the visual image to the all body axis movements (smooth and saccadic) at a 60 Hz update rate. This corresponds to a delay of 17 ms in coupling body movement to visual stimulus update, well below the 44 ms onset delay to a large-field velocity impulse¹¹ (Figure 4A, see STAR Methods).

Figure 4. Manipulating visual feedback dynamics does not influence smooth object tracking dynamics.

A. Setup for a modified magnetic tether arena in which computer controlled visual stimuli are projected onto a cube surrounding the fly on all sides. Live video tracking allows for “visual clamp” conditions that maintain an image at a fixed position relative to the fly’s major body axis. Thus, proprioceptive feedback is intact, but visual feedback is perturbed. B. (left) In true closed loop yaw-free conditions, a leftward steering effort produced by the fly (red arrow) will generate rightward image motion relative to the body axis (black arrow). (middle) Like Figure 3, constant velocity ground oscillation (blue trace) elicits smooth optomotor movements of the body. Each trial starts with a stationary phase (gray shaded box). (right) Paired dot plot of FFT magnitude for 2-second periods before and after stimulus motion onset. Each dot represents an individual fly’s mean response. Paired Student’s t-tests were performed. C. (left) Responses to movement of a 30° bar, in true visual and proprioceptive closed loop conditions. Middle and right panels are the same as in B. D. (left) Under visual clamp conditions, leftward steering effort produced by the fly results in a matched leftward displacement of the bar such that the position of the bar remains constant. Middle and right panels are the same as in B. Insets in B,C,D (center) zoom in on the population mean. E. Comparison of responses before and after motion onset across all three experimental conditions. Each circle represents an individual fly. See also Figure S3.

We validated this new paradigm by showing that under normal visual closed-loop conditions, robust ground optomotor body responses were elicited by constant velocity image oscillations (Figure 4B middle). We verified that upon the onset of open-loop ground motion, the amplitude of 2 Hz body yaw oscillation increased significantly (Figure 4B right). By contrast, yaw-free flies showed no obvious smooth steering responses to bar oscillation regardless of whether they were under normal closed-loop (Figure 4C middle) or artificial open loop visual conditions (Figure 4D middle). In both the presence and absence of visual feedback, and independent of the azimuthal position of the bar, flies did not steer back-and-forth with bar motion (Figure 4C,D right, Figure S3B). Comparisons across conditions revealed that the effect of ground motion was stronger than that of bar motion in either feedback condition; furthermore, the effects of bar motion were not different between artificial open and closed visual feedback conditions (Fig 4E). Similar to previous experiments, bar responses in visual open loop were still mostly composed of periods of straight flight interspersed with saccades (Figure S3A) ²². Together, these results show that the bar-elicited smooth steering behavior that results in frontal fixation in body-fixed flies is due not to compromised visual feedback, but rather to compromised proprioceptive feedback.

Body-fixing introduces variability to head and wing control

Comparisons across tethering paradigms are inherently problematic, in part due to the different control variables and measurement units for steering typically used in the two setups. More critically, it is not feasible to assess the behavior of the same animal across the two setups, precluding powerful within-subjects analysis on behaviorally relevant timescales. To facilitate body-state comparisons within subjects and directly assess the effect of compromising proprioceptive feedback, we designed a pneumatic gripper apparatus that allows switching between body states in a magnetically tethered fly (Figure 5A). Mounted above the magnetic tether arena, two arms clasp the rotating magnetic pin, locking the fly in place at a desired angular orientation relative to the visual display. A fly in the yaw-free state can therefore quickly become body-fixed and vice versa.

Figure 5. Body-fixing increases amplitude and variance of wing and head movements.

A. An actuated gripper modifies the magnetic tether to rapidly switch between body states. Video registers the fly’s heading (magenta vector). Head and wing steering kinematics are extracted (blue, green frames). B. (top row) Example traces of one yaw-free trial across 3 individuals. The stimulus is a 10 s sinusoidal ground oscillation indicated in light gray. Black arrowheads indicate body saccades, observed in wing and head traces. (bottom row) Upon closing the gripper on the same animals, the body angle is fixed in place. C. Mean head dynamics amplitude (left) and variance (right) for 374 yaw-free trials and 175 body-fixed trials in N = 34 flies. Each dot represents a single trial. Unpaired student’s t-tests were performed with * = p<0.05; ** = p < 0.01; *** = p < 0.001. D. Mean amplitude of Left minus Right wing beat amplitude (left) and variance (right) for 383 yaw-free trials and 171 body-fixed trials in N =35 flies. Response amplitude represents the averaged absolute value throughout the trial. Variance was similarly computed across the length of each trial. Each dot represents one trial. See also Figure S4.

In this setup, we first quantified the angular dynamics of the body, head, and wings, across body state within the same fly. We switched from the constant velocity stimuli of our previous experiments to constant frequency stimuli from here forward to facilitate frequency domain analysis. We found that a 2° difference between the left and right wing beat amplitude drives body movements roughly 3 times larger (Figure 5B data trace insets). In line with previous work, we confirmed that the body, wings and head of a yaw-free flies smoothly track large-field optic flow and minimize integrated error using syn-directional saccades generated by both head and wing motor systems (Figure 5B top row) ^17,25. We observed that the smooth dynamics and precise coordination of wing and head movements seen in the open gripper yaw-free state were severely impaired upon body-fixing by closing the gripper (Figure 5B bottom row). The mean amplitude and variance for both wing and head steering dynamics increased upon body-fixing (Student’s unpaired samples t-test p<0.01) (Figure 5C,D), a result that would appear as jitter in the visual panorama had these kinematics been artificially coupled to visual motion (Figure 2A).

Body-fixing increases gain of smooth tracking responses to vertical bars

The increased amplitude and variability of head and wing steering responses to image slip has been suggested to result from saturating gain of the large-field optomotor response in the body-fixed state ¹⁹. We hypothesized that impairing proprioceptive feedback by body fixing would similarly increase the gain of optomotor responses to small-field bars. To test this, we presented flies with a compound motion stimulus consisting of an oscillating ground and an oscillating vertical bar, each moving on separate single frequency sinusoid trajectories (Figure 6A). To remain within the dynamic range of all behaviors, we chose stimulus frequencies for which both body and head movements contribute to gaze stabilization and visual responses to large-field and small-field stimuli are comparable ^26,27. We placed the bar stimulus on midline and measured the amplitude of the ground and bar responses at the relevant stimulus frequencies. Conceptually, we presented two independent inputs to the fly and computed the output gain for each to assess how strongly each stimulus is tracked across body states (Figure 6B). We performed this experiment using motion defined bars, and solid dark bars, switched the frequencies of the bar and ground, and measured body angle, wing steering, and head smooth steering responses.

Figure 6. Body-fixing increases the wing steering bar response gain.

A. (top) Motion-defined bar or dark bar stimuli are oscillated within an oscillating ground. (bottom) Partial stimulus traces for ground oscillating at 2.3 Hz (black) and bar oscillating at 2.7 Hz (yellow), with 15°amplitude. Note that stimuli move in and out of phase. B. Systems identification framework where the two sinusoidal stimuli are inputs to the central nervous system (CNS) and the fly’s steering response is the sole output. C. (left) Body movement response for N = 18 flies to compound bar/ground stimuli. Magenta line represents mean body angular position, shaded envelope represents standard deviation. (middle) FFT plots of the body response (left y-axis, black) overlaid with FFT amplitude of ground and bar stimulus inputs (right y-axis, gray). (right) Gain of body responses to the ground (black) and bar (yellow). Gain in decibels is negative as the amplitude of the response output is smaller than the input. Each point represents an individual fly’s mean response. Nonparametric Wilcoxon Rank-sum tests were performed. D. Population (N=11) wing steering responses in the yaw-free, open gripper condition (left) and body-fixed, gripper closed condition (right). Time domain mean response +/− SD flank FFT plots throughout (green). E. Mean response gain, as in C, for ground (black) and bar (yellow) were compared across body states and stimulus types. F. Same as D but with motion-defined bar and ground oscillation frequencies swapped. G. Same as E but with motion-defined bar and ground oscillation frequencies swapped. H-K. Same as D-G but using a 30° solid dark bar stimulus. See also Figure S4.

Based on the responses to bar and ground motion individually, we reasoned that given a compound stimulus, yaw-free flies smoothly track ground motion over that of the bar. We computed the FFT of the body response and confirmed that the fly responded to the ground stimulus, with a significantly weaker response to bar motion (Wilcoxon Rank-sum test, p<0.001) (Figure 6C). Regarding wing and head responses to the compound stimulus, previous work has demonstrated that the head optomotor responses track ground motion and are unresponsive to a moving bar ²⁴. We thus focused on the steering responses of the wings to test whether object responses are affected by body state. We found that the yaw-free wing steering response matched the oscillation dynamics of the ground (Fig 6D, left). Correspondingly, the gain of the ground response was higher than that of the bar in the yaw-free state (Fig 6E). Upon opening the yaw proprioceptive feedback loop by closing the gripper, the wings showed increased response gain to the bar, while the ground response remained unchanged (Fig 6E). The increase in the smooth tracking gain of the bar upon body-fixing persisted when we swapped the frequencies to oscillate the ground faster than the bar (Fig 6F,G).

A motion-defined bar stimulus tests motion dependent tracking behavior because it does not provide positional cues such as static luminance; rather it is defined only by its movement relative to the background. In this sense, the stimulus is challenging for the visual system to detect. Physiologically, vertical bars with high spatial frequency content have been shown to reduce the efficacy of T4/T5 motion detecting neurons that supply the optomotor system ¹⁸. By contrast, a low spatial frequency solid dark bar more strongly stimulates T4/T5 motion detectors ¹⁸. Thus, we reasoned that if body-fixing increases the gain of the smooth steering response to a motion-defined bar, then this gain effect may be even more apparent for a dark bar that better drives the optomotor pathway. The dark bar did elicit more coherent steering and higher peaks at the bar FFTs for both head and wing responses (Figure 6H,J, Figure S4E,G), and response gain was similarly dependent on body state for the dark bar (Figure 6I,K). The effect of body-fixing was again independent of the relative frequencies of the bar and ground, persisting when oscillation frequencies were swapped (Figure 6J,K).

How head responses to large-field motion are modulated by body state has been described elsewhere ¹⁹. Our results confirm that body-fixing induces an overall gain increase in the amplitude of head responses, and thereby help to validate our gripper apparatus. We show that the head primarily tracks the dynamics of the ground rather than the bar, particularly for motion-defined bars, and notably show for the first time that large-field selectivity for the head optomotor response persists across body states (Figure S4 A,C,E,G). Finally, we also show that fixing the body in place causes an increase in amplitude of head movements regardless of the visual stimulus conditions (Figure S4 B,D,F,G).

DISCUSSION

We discovered that flies that are free to steer in the yaw plane do not fixate a motion-defined bar on visual midline the way they do in body-fixed experimental paradigms. While high gain smooth tracking responses to bar motion are characteristic of body-fixed flies and result in the bar being fixated on midline, yaw-free flies do not show smooth optomotor responses to the same bar stimuli. Switching from a yaw-free to a body-fixed state immediately induces exaggerated head and wing movements, as well as high gain wing steering responses to bar movement. Our results support a model proposing that proprioceptive signals engage active damping of optomotor steering control ^19-21. In this manner, object tracking and frontal fixation are enhanced by reduced proprioceptive feedback in body-fixed flight; the corollary is that intact proprioception reduces object optomotor responses and fixation. Our results extend this concept to the limit of no object fixation by yaw-free flies. Object tracking models that use a velocity dependent term resulting in midline bar fixation shall have to be updated in light of active modulation by mechanical body-state dependent mechanisms.

Body state modulates inner-loop optomotor responses for bar fixation

Discriminating relevant visual features of the environment requires maintaining stable gaze during locomotion. Stable gaze facilitates feature detection, after which a goal directed orientation maneuver may ensue. From an engineering perspective, these tasks can be described as low-level “inner-loop” stabilization responses and higher-level “outer-loop” goal-directed orientation ²⁸ that interact in order to generate flexible yet stable flight control ²⁵. Inner-loop flight control involves first detecting slip of the visual panorama on the retina arising from self motion or an external perturbation, and then generating a corrective optomotor maneuver to stay on course ⁸. In flies, retinal slip is detected by small-field T4 and T5 neurons, which in turn supply retinotopic, directionally-selective signals to spatially integrating large-field neurons housed in the lobula plate. Lobula plate large-field neurons project to pre-motor descending pathways ^29,30 to control wing optomotor control ³¹. Smooth optomotor responses may be evoked both by large-field (panoramic) stimuli, and small-field stimuli (single vertical stripe or bar). Large-field responses operate over a lower frequency range than small-field stimuli and are therefore thought to be mediated by separate lobula plate neurons ^26,32, but both pathways presumably draw from T4/T5 directional motion detectors. Here, we show that the transition from yaw-free to body-fixed state within individual flies evokes a concomitant transition to high gain, high variance kinematics of both the head and wings (Figure S5, 6, S4 and see ¹⁹). Based on these results, we conclude that rigidly fixing the body acts to increase inner-loop optomotor gain to modulate both large-field and small-field responses.

Several robust features of frontal bar fixation by body-fixed flies have implicated optomotor control, which is traditionally considered an inner-loop variable, in outer-loop visual orientation behavior. First, smooth steering responses to imposed bar motion are the strongest near visual midline, thereby centering the object on approach ¹⁰. Steering responses to long stimulus paths are stronger when a bar recedes from midline than when it approaches midline, an asymmetry that tends to “sink” the bar on midline ³³. Such findings have inspired object tracking models that rely heavily on bar velocity term ^7,10,12.

If velocity-dependent bar responses and frontal fixation are a manifestation of the inner-loop optomotor system then several predictions would follow. First, blocking the activity of T4/T5 motion detectors ought to compromise frontal bar fixation. Indeed, for body-fixed flight under virtual closed loop feedback, blocking synaptic transmitter release by T4/T5 neurons was found to significantly attenuate frontal bar fixation, particularly for fast movements ¹². However, body-fixed flies walking on a spherical treadmill under virtual closed loop conditions persisted in orienting toward a bar even with blocked motion detection neurons ³⁰. This discrepancy may result from known differences in neuromodulatory state between walking and flight or the fact that body-fixed walking flies do receive stride-coupled proprioceptive signals from leg mechanosensors ³⁴. A second prediction would be that bar stimuli composed of the spatial properties that best drive T4/T5 motion detectors would evoke correspondingly stronger smooth bar responses. Indeed we found that compared to a higher spatial frequency motion-defined bar, a solid black bar evokes stronger directional wing steering responses (Figure 6), and stronger T4/T5 calcium responses ¹⁸. Similarly, we found that in transitioning from a yaw-free to a body-fixed state, the resultant increase in bar evoked steering was more pronounced for the solid dark bar than for the motion-defined bar (Figure 6 and Figure S4). Furthermore, the body-state effect is accentuated at lower bar oscillation frequency (compare bar frequency in Figure 6D vs F and 6H vs J), consistent with the notion that bar responses are driven by the low-pass optomotor system ³⁵.

Proprioceptive signaling actively damps high gain visual responses

As we have discussed, but it is worth repeating, body movements attenuate inner-loop ground stabilization responses ¹⁹. However, optomotor gain could in principle be mediated by visual mechanisms alone, accentuated by the coupling algorithm that governs virtual closed loop visual conditions. Simply varying the coupling gain changes the sharpness of a bar fixation histogram during body-fixed flight ^36,37. However, our visual open-loop experiment showed that clamping the position of the oscillating bar relative to the fly’s body axis had no influence over bar steering behavior in a yaw-free fly (Figure 4).This suggests a non-visual mechanism of body state modulation of object tracking.

Our results are the first to demonstrate that body movement signals influence the gain of small-field (bar) responses and active visual fixation. By what mechanism does this occur? Proprioception is broadly defined as the sense of body position and movement, encoded by mechanosensory organs. A study that mechanically oscillated a body-fixed flying fly showed that even a small amount of body rotation decreases the gain of optomotor responses in Drosophila ³⁸. Our results show that naturalistic yaw proprioception essentially eliminates optomotor responses to bar motion across a range of conditions (Figure S2,3,5). In a control theoretic framework, robotics studies have posited that short latency proprioceptive feedback could be used to actively damp yaw rate, thereby imparting robustness and stability to a high gain visuomotor control system ²⁰. Physiological evidence for proprioceptive damping of optomotor responses has not yet emerged, but the hypothesis is broadly consistent with the behavioral observations we have made here.

In principle, any appendage equipped with a mechanosensory organ could serve a proprioceptive role. In flies, numerous mechanosensory organs encode body dynamics during flight, including the wings, antennae, and most notably, in Diptera, the multimodal haltere organs ^38-42. The haltere sensorimotor system acts both as a metronome to temporally pattern wing kinematics, and as a gyroscope to correct for rotational displacements of the body during flight.

As metronomes, halteres provide wingstroke-coupled feedback⁴³ on the timescale of 2-5 ms to wing motorneurons⁴⁴, which is combined with other afferents to modulate wing steering muscle activity ⁴⁵. Halteres are not passive sensors, but rather are actuated by their own set of muscles that are gated by descending visual signals ⁴⁶. Metronomic haltere control of wing kinematics operates continuously in body-fixed flies, and surgical ablation causes decreased open-loop optomotor gain and enhanced closed-loop bar fixation ^21,37. By contrast, we find that transitioning from the yaw-free to the body-fixed state, which presumably constrains proprioceptive feedback signals, causes increased small-field optomotor gain (Figure S3,4,5,6) and, in line with the studies above, stronger frontal bar fixation (Figure 2). One possible explanation for this mismatch is that the proprioceptive modulation of small-field object tracking is mediated not by the metronome, but rather by the gyroscope function of the haltere system. Feedback from this system is presumably either attenuated or drastically altered in the body-fixed state.

There are at least two plausible models of haltere gyroscope function that could underlie our results. When a fly experiences an involuntary perturbation like a gust of wind, gyroscopic haltere-elicited reflexes act to stabilize the body ³⁸. However, descending visual signals directly activate the haltere steering muscles, which in turn modulate wing steering responses ⁴⁷. Thus, in order to initiate a voluntary turn, an external visual command could activate the haltere muscles and trigger a fictive compensatory steering maneuver. In this model, the haltere sensory pathways of a body-fixed fly are overstimulated because the body cannot complete the turn that resets the gyroscope. Overstimulated haltere sensory signals would evoke exaggerated optomotor responses that act on small-field objects. An alternative model is that, in the yaw-free state, continuous small body movements activate gyroscopic signals that actively damp high gain optomotor signals to a more optimal dynamic range. In this scenario, body-fixing results in understimulated gyroscopic haltere signals and high gain optomotor control that operates on small-field objects to drive stripe fixation as well as exaggerated head and wing movements. Discerning between these two hypotheses will be experimentally challenging. But tackling the first-order question - whether the haltere system is involved at all - is feasible; any neurogenetic manipulation that re-engages frontal bar fixation in a yaw-free state and/or disengages frontal bar fixation in a body-fixed state would be promising.

For neurophysiology, a broken feedback loop is a feature not a bug

Our results show that yaw-free flies do not frontally fixate a motion-defined bar (Figure 2). And yet, both in free flight and free walking arenas, Drosophila melanogaster readily orient towards vertical stripes or posts ^48-51. Yaw-free and freely flying flies intersperse periods of straight flight with rapid body saccades for outer-loop object orientation either toward or away from visually discrete objects ^17,23,50, reserving smooth, proportional steering movements for inner-loop gaze stabilization via the wings and head ²⁷. Body-fixed flies simply show an additional control variable - the small-field optomotor response. Does this imply that body-fixed flight preparations are compromised by experimental artifacts? On the contrary, we know of no other sensory manipulation that imparts the same degree of robustness to a visual behavior. If one wishes to study the neurophysiology of inner-loop optomotor control topology, then the body-fixed bat tracking paradigm is the exemplary experimental model. Breaking feedback loops has provided us with the insight that object tracking does not require a continuous velocity component, and that frontal fixation does not represent the natural setpoint for object navigation in Drosophila melanogaster.

From a visual ecological perspective, Drosophila melanogaster is a generalist, occupying myriad habitats containing diverse visual background clutter that generates large-field self motion cues superposed with optically distinguishable foreground features. Other Drosophila species have specialized visually sparse habitats, and thereby have different control strategies for object pursuit. For example, D. melanogaster approaches high contrast elongated vertical objects, and avoids small objects that presumably reflect a threat ⁴⁹. By contrast, the desert dwelling Drosophila mojavensis approaches either of these object classes equally well, presumably reflecting its own visual ecological context ⁵². Even within body-fixed D. melanogaster, bar fixation is not “hard wired”, but rather can be modulated by food odor cues ⁵³. This suggests that ecologically adaptive visual-proprioceptive circuit interactions can be achieved with small changes to the low-level inner-loop optomotor responses that are ubiquitous across taxa.

To build a comprehensive understanding of how neural mechanisms govern behavior, we must be able to examine the isolated and combined effects of feedback across multiple sensory modalities. Well-tuned feedback is indispensable for control and stability of locomotion behaviors, and is compromised to some extent by tethering animals in place ⁵⁴. Yet, the approach of selectively opening and closing feedback loops is essential for probing how information flows through a dynamical sensory system ^55,56. In Drosophila melanogaster, body-fixed experimental paradigms have revealed much about visual information processing and continue to be necessary for functional imaging of circuits and individual neurons. Here, we provide an experimental framework to formalize control models and gain a deeper mechanistic understanding of the underlying circuit effects of manipulating the damping of inner-loop control. Our work highlights the importance of accounting for and experimentally exploiting proprioceptive feedback in body-fixed preparations.

STAR METHODS

RESOURCE AVAILABILITY

Lead Contact

Further information for resources should be directed to and will be fulfilled by the corresponding author, Mark A. Frye (frye@ucla.edu).

Materials Availability

This study did not generate any new unique reagents.

Data and Code Availability

The datasets and code generated during this study are available at Open Science Framework public data repository https://osf.io/7mq9j/

EXPERIMENTAL MODEL AND SUBJECT DETAILS

A wild-type Drosophila melanogaster strain was maintained at 25°C under a 12 hr: 12 hr light:dark cycle with access to food and water ad libitum. All experiments were performed with 3- to 6- day-old female flies within 4 hours of lights on and 4 hours of lights off.

METHOD DETAILS

Animal Preparation

We prepared the animals for each experiment according to a protocol that has been previously described ¹⁷. Briefly, we cold-anesthetized the flies by cooling them on a Peltier stage maintained at approximately 4°C. For magnetic tether paradigms, we glued stainless steel minutien pins (Fine Science Tools, SKU 26002-10) onto the thorax by applying UV-activated glue (Esslinger, SKU 12.201). The pin’s length was approximately 1 cm to minimize the moment arm about which the fly can generate cross-field torques in pitch and roll. The pins were less than 1 percent of the fly’s moment of inertia about the yaw axis. For rigid tether paradigms, we used thicker tungsten pins (diameter: 0.1mm). In both cases, the pin was placed on the thorax projecting forward at an angle of approximately 30°, in order to closely mimic the fly body’s angle of attack during free flight. Before running experiments, flies were allowed at least half an hour to recover upside-down in a custom-designed holder, inside a covered acrylic container where humidity and temperature could be controlled in order to avoid rapid dehydration (~ 24°C, 50% humidity). After recovering from anesthesia, flies were given small pieces of Kimwipe to cling to and prevent flight and energy expenditure.

Rigid tether experimental protocol

As previously described ⁴⁹, the rigid tether arena is a cylindrical display that consists of an array of 96 × 32 blue light emitting diodes (470 nm emission peak). The arena is modified by removing two columns of 8-pixel panels behind the animal for access into the center of the arena. Additionally, in order to maintain consistency with the magnetic tether paradigm, the top and bottom 8-pixel rows of LEDs were kept off. Thus, the visual panorama in this setup wrapped around the fly, subtending 300° horizontally and 60° vertically (Figure 1B). Each singular LED subtended 3.75° on the flies’ retina. Flies were illuminated from the top with an infrared diode (880 nm emission peak) which cast a shadow of the beating wings onto an optical sensor. An associated “wing-beat analyzer” (JFI Electronics Laboratory, University of Chicago) converted the optical signal into an instantaneous voltage measuring right and left wing beat amplitude (WBA) and frequency (WBF). The difference in the left and right WBA (ΔWBA), which is highly correlated with the fly’s steering effort in the yaw axis ⁵⁷, connected to the panel display controller to close a feedback loop with the rotational velocity of the visual display for the artificial closed loop feedback experiment in Figure 2. For open-loop experiments, signals from the wing-beat analyzer as well as from the panel display controller encoding the visual display position, were recorded on a DAQ (National Instruments USB 6259) at 1 kHz. The data acquisition was triggered through a voltage step sent by a second DAQ (USB-1208LS, Measurement Computing) interfaced with MATLAB that in turn controlled the pattern presentations. At the beginning of each experiment, we presented a dark bar on a uniform ground in closed-loop to calibrate the fly’s position within the arena. Flies which did not stabilize the bar on midline or displayed incorrect hutches, which is indicative of poor tethering, were discarded from the experiment. Only flies that flew continuously for at least 75% of the trials were included in the analysis. If flies stopped flying during a trial or wing beat frequency dropped below 150 Hz, the trial was discarded.

Magnetic tether and gripper experimental protocol

As previously described ^14,16, the magnetic-tether arena is comprised of a cylindrical display that consists of an array of 96 × 16 blue light emitting diodes (470 nm emission peak) that wrap around the fly, subtending 360° horizontally and 60° vertically (Figure 1B). Each singular LED subtended 3.75° on the flies’ retina. Flies were suspended between two magnets, allowing free rotation along the vertical (yaw) axis. We illuminated the fly from above with an array of six infrared LEDs (940 nm emission peak). Both the wings and body of the flies could clearly be visualized from below using an infrared-sensitive camera (BlackFly BFS-U3-04S2M-CS) fitted with a zoom lens (InfiniStix 1.0x/94mm, Edmund Optics) and an 850 nm longpass filter (FGL850M, ThorLabs) to block light from the LED panels. We recorded the angular position of the fly within the arena at 100 frames/s. At the beginning of each experiment, we characterized flies average optomotor behavior by presenting a large-field panorama rotating at 120°/s for 20s in the CW and the CCW directions. Flies who did not complete this trial or displayed excessive wobble were discarded from the experiment.

A pick-and-place pneumatic gripper (McMaster Carr Cat# 6220K51) was mounted above the arena, on a rail traversing its diameter. The gripper was fitted with custom designed 3D printed black PVA fingers that closed smoothly around the magnetic pin, locking it in place. The gripper was actuated manually by opening and closing a pneumatic switch connected to house air at 20 psi. When not actuated, the gripper was lifted out of the arena above the fly’s field of view. The order of body-fixed and yaw-free trial blocks were randomized across individual animals.

Visual clamp paradigm projector display

To open the visual feedback loop on a behaviorally relevant timescale, we built a custom virtual reality flight simulator combining the magnetic tether paradigm with the high resolution projector display and graphics library of ⁵⁸. Instead of the LED display described above, this setup used a digital projector and well-placed first-surface mirrors to wrap the projection around the 4 vertical sides of a 4”x4”x4” perspex cube lined with gray rear projection material along the cube's inner surface. The projector (TI DLP LightCrafter 4500 EVM) produced frames 1280 x 800 pixels in size resulting in ~.4°/pixel at 120 Hz. The magnetic tether was set up as described above, positioning the fly at the center of the cube. From this position, the display subtends the fly’s visual field by 360° horizontally and 70–90° vertically, decreasing from the center of each panel to the corners. This discrepancy is accounted for by programmatically restricting the vertical subtended angle to 70° all around and correcting for perspective. Using the same infrared-sensitive camera described above, this system detected the heading of the fly and optionally subtracted its generated motion by the next frame at 60 Hz. This results in a 17 ms delay between a change in the fly’s heading and a change in the visual display, which is less than half of the 44 ms onset delay in measured responses to ground perturbations along the yaw axis ¹¹. Additionally, this generates temporal frequencies up to 30 Hz, which is above the flies' behaviorally measured temporal resolution of 20 Hz ⁵⁹. In half of the trials, the fly was placed in virtual open-loop control, clamping the center of the bar’s oscillation to the initial position relative to their heading. In the other half, the fly remained in closed-loop, allowing them to turn towards or away from the bar. Though the stimuli projected are RGB, only the blue channel was used in order to mimic parameters of the LED display. Finally, all visual stimulus resolutions were matched to those of the 96 pixel LED arena.

Visual Stimuli Across Paradigms

Experiments for Figure s 2,3,4 were designed to test responses to constant velocity stimuli, thus containing power across frequencies. Trials lasted 4 s with 2-4 s rests between trials. Trials were explicitly kept short to assess immediate responses to bar motion and mitigate any learning effects of stimulus predictability ⁶⁰. Bar trials used either a 30° wide motion-defined bar on a randomized panorama or a 30° wide dark bar on a uniform grayscale panorama. Motion defined bars were used to elicit responses to object movement and eliminate luminance contrast cues that might provoke static positional responses. We repeated one experiment using a solid dark bar because this low spatial frequency stimulus better stimulates the columnar directional motion detecting neurons and their downstream pathways that control optomotor responses. As expected, the dark bar highlighted the key phenomenon that body-fixing increases bar tracking responses. In all cases, we presented a spatially randomized background scene to produce behaviorally relevant visual conditions.

In each bar trial, bars were presented in one of 12 pseudorandomized evenly distributed azimuthal positions relative to the fly’s heading. Bars were oscillated on a velocity triangle waves with 60° peak-to-peak amplitude moving at either 30°/s (0.5 Hz) or at 120°/s (2 Hz). Ground trials where the whole panorama oscillated were interleaved with bar trials as a positive control measurement of large-field optomotor performance. These trials were either a broadband randomized ground or a 30° wavelength grating. Flies that did not show significant optomotor responses were discarded from the dataset.

Experiments for Figure s 5 and 6 were designed to test responses to constant frequency stimuli, thus containing power across velocities. For experiments using the gripper apparatus, visual stimuli consisted of a bar and ground oscillating concurrently, on different sinusoidal motion trajectories at either 2.3 Hz or 2.7 Hz with 30° peak-to-peak amplitude. The frequencies of oscillation were selected to be 1) prime numbers with non-overlapping harmonics, 2) within a dynamical range where both heads and bodies are involved in ground stabilization ²⁷ and 3) within a dynamical range where flies have been shown to track both large and small-field stimuli robustly ²⁶. Ideally, the amplitudes of the oscillations ought to be scaled such that both stimuli would generate equal power in velocity - however, because the frequencies we chose are very close together, the amplitude difference is below the 3.75° resolution of our LED displays, and thus the amplitudes were kept equal at 15°. In these experiments, each trial was 10 s long. The angular position of the fly was extracted live at the start of each trial and the stimulus position was such that the bar, either dark or Fourier, was initially placed on the fly’s azimuth.

QUANTIFICATION AND STATISTICAL ANALYSIS

All steering responses, heading angle extraction and statistical analyses were performed using MATLAB (MathWorks, Natick, MA, USA). For rigidly tethered flight experiments, wing beat amplitudes (open-loop experiments) and bar angular positions (closed-loop experiments) were directly extracted from the data acquisition device. For magnetic tether experiments, the angular heading of the fly, the head angular position and the wing steering amplitudes were all extracted from video data recorded at 100 fps using a suite of custom machine vision tools provided by the Bio-Motion Systems lab at Penn State University (https://github.com/BenCellini/CrazyFly). Raw data was band-pass filtered and for all angular data extraction, clockwise (CW) was defined as the positive direction of motion. Head and body saccades were then identified from thresholded angular velocity²². Flies' angular heading relative to the moving bars was back-calculated from the position of the bar and that of the fly within the 360° arena and data was wrapped to 180°. If flies were yaw-free and free to reorient within the arena, we isolated inter-saccadic flight bouts of at least 1 s duration where the bar was on the fly’s visual midline and averaged all such bouts within an individual fly to obtain a mean fly response. If the flies were body-fixed, no such segmentation of the data was necessary and saccades were eliminated from the time-domain traces by virtue of the filters applied and trial averaging. Fly means were averaged across the population for each dataset. Fast Fourier Transforms (FFTs) were performed for each individual fly and the amplitude of bar responses for each fly was normalized to that fly’s optomotor ground response. The magnitude ratios at the relevant peaks were compared across yaw-free and body-fixed datasets using unpaired samples Student's t-test. For analyses comparing variance and amplitude of wing and head responses across body states, all individual trials were plotted and unpaired Student’s t-tests were performed. For all experiments where bars and ground stimuli were presented concurrently, input-to-output gains were calculated for the bar and ground sinusoidal stimuli and the gains were compared using a nonparametric Wilcoxon Rank-sum Test. Unless otherwise specified, each dot in a scatter plot represents an individual fly’s mean response.

KEY RESOURCES TABLE

REAGENT or RESOURCE	SOURCE	IDENTIFIER
Electronic equipment
LED panel visual display system	IO Rodeo	13
Neutral density filters	Rosco	Cat# 59
Wingbeat Analyser	JFI Electronics	N/A
BlackFly USB camera	FLIR	BFS-U3-04S2M-CS
Experimental models: Organisms/strains
Drosophila melanogaster	Wild	Population Cage Flies (PCF)
Mechanical equipment
Pneumatic gripper	McMaster-Carr	Cat# 6220K51
Software and algorithms
MATLAB	MathWorks	http://www.mathworks.com/
CrazyFly wing and head tracker	Jean Michel Mongeau	https://github.com/boc5244/CrazyFly

Highlights.

• Flies rigidly tethered in virtual reality flight steer to center a vertical bar

• Centering responses to bar motion motivate object tracking algorithms

• Flies on a tether that allows free yaw body maneuvers do not center a bar

• Proprioceptive signaling of body movement modulates object tracking