Cellular evidence for efference copy in Drosophila visuomotor processing

Abstract

Each time a locomoting fly turns, the visual image sweeps over the retina and generates a motion stimulus. Classic behavioral experiments suggested that flies use active neural-circuit mechanisms to suppress the perception of self-generated visual motion during intended turns. Direct electrophysiological evidence, however, has been lacking. Here we show that visual neurons in Drosophila receive motor-related inputs during rapid flight turns. These inputs arrive with a sign and latency appropriate for suppressing each targeted cell’s visual response to the turn. Precise measurements of behavioral and neuronal response latencies argue that motor-related inputs to optic flow processing cells represent internal predictions of the expected visual drive induced by voluntary turns. Motor-related inputs to small-object-selective visual neurons, on the other hand, could reflect either proprioceptive feedback from the turn or internally generated signals. Our results in Drosophila echo the suppression of visual perception during rapid eye movements in primates, demonstrating common functional principles of sensorimotor processing across phyla.

Humans scanning a visual scene show periods of stable gaze punctuated by rapid eye movements called saccades. During saccades the visual image translates briskly over the retina and our nervous system employs mechanisms to suppress the perception of such self-induced motion stimuli so as to help us perceive the outside environment as stationary^1–3. Primate saccades represent just one example in which sensorimotor processing must distinguish between self-generated sensory stimulation, also known as reafference, from externally generated stimulation, or exafference^4,5. In a few model systems, there is now even a developed understanding of how neuronal circuits distinguish exafference from reafference. For example, male crickets have an identified interneuron that activates during chirps and inhibits auditory neurons to prevent them from responding to the chirp⁶, whereas weakly electric fish use circuitry in a cerebellum-like structure to subtract the predicted sensory input due to the animal’s own electric organ discharge from the incoming electrical sensory stream^7,8.

One common scenario in which the suppression of reafference is essential, but remains more poorly understood, is in the inhibition of stability reflexes during voluntary locomotor turns. Consider a flying fly. Analogously with human eye movements, the fly punctuates periods of stable flight with rapid turns called body saccades⁹. Between saccades, the fly employs an optokinetic reflex, also known as the optomotor response, to help maintain stable flight. In this reflex, wide-field visual motion, say, to the right, is interpreted as being caused by an erroneous turn of the fly to the left—perhaps caused by a gust of wind or noise in the flight motor—and a corrective rightward turn is elicited (Fig. 1a). Whereas this reflex is important for stability, if it were always active, it would act against any intended change to the locomotor trajectory. This begs the question: how then do flies ever turn?

Figure 1. Patch-clamp recordings in tethered, flying Drosophila can reveal whether fly visual neurons receive motor-related inputs with voluntary turns.

(a) Schematic of the optomotor response. V: visual center. M: motor center. (b) Schematic of a volitional turn. An internal decision center (D) sends a command to initiate a volitional turn, an efference, to a motor center, which drives the fly to turn. (c) von Holst and Mittelstaedt’s concept of an efference copy. An internal decision center sends an efference to a motor center (blue) and also a copy of the efference, an efference copy, to visual centers (red). The copy signal has the form where it cancels the expected reafferent visual input caused by the turn. Use of block diagram notation from control theory is only for heuristic purposes; interactions between signals at each summator junction (circle with × through it) need not be strictly additive in the brain. (d) Experimental apparatus. (e) Camera images of a fly turning left and right. WBA: wingbeat angle. Angles were measured from the horizontal as indicated. Note that camera images are taken from below the fly whereas schematics of a fly turning, used later in the paper and shown here, are oriented, more naturally, as viewing the fly from above.

Guided by simple behavioral experiments, von Holst and Mittelstaedt famously postulated that with each motor command to initiate a voluntary locomotor turn, also known as an efference, (Fig. 1b, blue), flies send a copy of the command, an efference copy, to their visual system⁴ (Fig. 1c, red). This efference-copy signal was postulated to have the correct sign and magnitude for silencing the reafferent visual input caused by voluntary turns, thus preventing the optomotor response from kicking in. Subsequent behavioral experiments have continued to argue that locomoting insects send efference copies to their visual system^10,11 or make use of forward models to predict the sensory consequences of motor commands^12–14, however, electrophysiological evidence has been scarce.

Here we show that Drosophila visual neurons receive motor-related inputs during voluntary body saccades. These inputs are similar in magnitude but opposite in sign to the expected reafferent visual input caused by saccades. Responses of wide-field sensitive neurons in the visual lobe and a new class of small-object-selective cells in the central brain are both strongly suppressed during intended flight turns. Other visual cells, whose receptive-field properties are such that they should not respond to wide-field motion during saccades, are only mildly affected. These findings demonstrate cell type-tailored signals that are appropriate for silencing reafferent visual responses during voluntary locomotor turns in flies, as predicted by von Holst and Mittelstaedt sixty-five years ago.

Results

Patch-clamp electrophysiology during spontaneous flight saccades

To test whether fly visual neurons receive motor-related inputs during voluntary saccades, we performed whole-cell patch-clamp recordings in tethered, flying Drosophila¹⁵ (Fig. 1d) (Online Methods). We quantified the flies’ turning behavior based on video data (Fig. 1e). Visual stimuli were presented on a panoramic LED display. Turning is detected in this preparation by subtracting the right wingbeat amplitude from the left (L-R WBA), where positive and negative differences indicate right and left turns, respectively. Some spontaneous saccades made by Drosophila are initiated without any immediate external event that could be said to trigger them^16,17. We will consider these, operationally, as voluntary actions, during which the optomotor response should be suppressed. We use the terms voluntary and spontaneous synonymously to refer to rapid flight turns that occur at an unpredictable moment relative to any obvious external stimulus.

Lobula-plate neurons are silenced in a cell type-specific manner during flight saccades

Our initial recordings targeted the Horizontal System North (HSN) cells of the lobula plate (Fig. 2a). There are two HSN cells per fly, one per side. These prominent visual neurons are four synapses downstream of photoreceptors. They respond to global rotatory motion, or optic flow, about the yaw axis (Fig. 2b–d), likely contributing to eliciting the optomotor response to these stimuli^18,19. Specifically, HSN cells depolarize in response to ipsilateral front-to-back motion—that is, motion from the front of the ipsilateral eye (near the antennae) to the back of the eye (near the thorax)—and hyperpolarize to motion in the opposite direction (Fig. 2d and Supplementary Figure 1). HSN cells are believed to signal primarily through graded changes in their membrane potential (V_m), though they do have voltage-gated Na⁺ conductances that typically yield small regenerative potentials, called spikelets, which augment the graded response²⁰.

Figure 2. Identified optic flow processing neurons in Drosophila receive motor-related inputs during saccades.

(a) Schematic of the fly visual lobe and arborization pattern of the six vertical system cells (VS1–6) and three horizontal system cells (HSN: Horizontal System North cell. HSE: Horizontal System Equatorial cell. HSS: Horizontal System South cell) in the lobula plate. (b) Definitions of yaw, pitch and roll. (c) Maximal-intensity z-projections at different depths of the lobula plate to visualize HS- or VS-cell neurites (nc82 anti-Bruchpilot signal of neuropil is in magenta; immunoamplified green fluorescent protein driven by the DB331-GAL4 driver is in green). (d) Visual responses of horizontal system north (HSN) and vertical system (VS) 1–2 cells to horizontally moving gratings during flight. Between trials, we presented a uniformly lit screen at the grating’s mean luminance. Individual cell averages are shown in gray (3–12 trials / cell). Averages across cells are shown in black. All cells were recorded from in the right lobula plate. Brief VS-cell responses to the onset and offset of motion reflect flash responses to the grating’s appearance and disappearance. (e) Sample Vm traces from an HSN and VS1 cell in the absence of any motion stimulus. Below each Vm trace we show the spontaneous turning behavior of the fly. L-R WBA: left-minus-right wingbeat angle. Colored arrows indicate saccades. Low-pass-filtered L–R WBA (10 Hz cut off) is shown in black; unfiltered signal is in gray. (f) Cross-correlation between Vm and L-R WBA for HSN cells and VS1–2 cells in the absence of any motion stimulus (cf. Online Methods). Gray lines represent individual cell correlations and black lines represent averages across cells. Cross-correlation functions between Vm and time-inverted L-R WBA signals are depicted in salmon, and their population averages in red.

In Fig. 2e, we show the V_m of an HSN cell, recorded in the right lobula plate, in a flying fly viewing a uniformly lit screen. Though we presented no overt visual stimulus to the fly, she generated spontaneous saccades¹¹, which appear as brief deflections in L–R WBA, and HSN’s V_m fluctuated in synch with these saccades (Supplementary Video 1). Specifically, the HSN cell transiently depolarized by a few mV with rightward saccades (Fig. 2e, red arrows) and hyperpolarized by a few mV with leftward saccades (Fig. 2e, blue arrows). Note that these saccade related potentials (SRPs) have the correct sign to counteract reafferent visual responses in HSN cells. For example, when a fly turns right, the visual world rotates leftward on the retina, which will induce a hyperpolarization in the right HSN; the synaptic input that generates this visually induced hyperpolarization would be counteracted, and perhaps completely cancelled, by a motor-related input that yields a depolarizing SRP.

To assess the cell type specificity of SRPs, we also recorded from the vertical system (VS) cells in the lobula plate (Fig. 2a,c–d). VS cells are sister neurons to the HS cells in that the two cell classes have very similar electrophysiological properties; the main difference is that VS cells are tuned to respond to optic flow fields induced by rotations around different body axes than HS cells. Specifically, there are six VS cells per side (VS1–6) and VS1–2 should respond to rotations about the pitch axis as inferred from their sensitivity to vertical motion in the frontal visual field²¹ (Fig. 2d and Supplementary Figure 2). VS1–2 cells may not receive prominent motor-related inputs during saccades because saccades do not typically include strong pitch rotations (Fig. 2b) or changes in altitude²² that would visually drive these neurons. Indeed, a VS1 cell did not show obvious SRPs during spontaneous saccades (Fig. 2e, green arrows).

We quantified these effects by cross-correlating the V_m recorded from ten HSN and twenty-one VS1–2 cells with the behavioral data (L-R WBA) obtained in the same experiments (Online Methods). Even though the flies’ turning statistics were similar during all sessions (Supplementary Figure 3a–b), we observed a pronounced, statistically significant, peak in the cross-correlation function for HSN cells (Fig. 2f) ((t(9) = 10.34, P = 10⁻⁶, comparing the magnitude of the gray and salmon curves for each cell at the moment of the peak in the average black curve across all cells). In VS1–2 cells (Fig. 2f), we observed considerably smaller, bidirectional peaks, among which the negative peak immediately before zero was statistically different from the null expectation (t(20) = 3.09, P = 0.006, comparing the magnitude of the gray and salmon curves for each cell at the moment of the negative peak in the average black curve across all cells), but indicated an overall much weaker correlation between V_m and wing movements. The weak correlation in VS1–2 cells may reflect an attempt by the fly brain to cancel the visual consequences of very small pitch rotations during spontaneous saccades or this correlation could be the byproduct of electrical coupling between VS1–2 cells and other neurons that show more prominent SRPs. HS and VS cells may be indirectly coupled, for example, through electrical synapses with descending neurons²³.

Though it seemed unlikely, we wanted to make certain that SRPs in HS cells did not simply reflect visual responses to the movement of the wings, or other appendages, like the antennae, that move during saccades²⁴. We therefore reversibly blinded flies with a classic optical technique²⁵ (Online Methods) and found that the correlation between V_m and L–R WBA in HSN cells remained intact when flies were blind (Fig. 3). While we cannot exclude that retinal activation contributes in some manner to SRPs in normal flies, these results argue that motor-related inputs to HSN cells are by and large of an extraretinal origin.

Figure 3. Saccade-related inputs to optic flow processing neurons persist in blind flies.

We blinded flies by applying a strong pulse of blue light to activate a large fraction of R1–6 opsin in the compound eye, rendering this opsin non-functional for further signaling (cf. Online Methods). Opsin thus activated is recovered with application of longer wavelength light. (a) Visual responses to a moving grating in the preferred direction. (b) Sample L-R WBA and Vm traces from a single HSN cell showing saccade-related potentials (SRPs) before, during and after blinding. (c) Cross-correlation functions between Vm and L-R WBA before, during and after blinding (n = 5 HSN cells).

SRPs have the correct sign for serving a cancellation function (Fig. 2e), but are they sufficiently strong to effectively modify visual signaling in HSN cells? To address this question, we drove HSN cells with an optimal steady state stimulus, a wide-field grating moving left or right at 1 Hz temporal frequency¹⁸, while measuring spontaneous saccades and SRPs. When we presented an individual fly with a rightward moving grating, she responded with a tonic turn to the right—the classic optomotor response—and she additionally exhibited spontaneous saccades in the opposite direction¹¹ (Fig. 4a, blue arrows). With leftward motion, the fly turned tonically left and produced saccades to the right (Fig. 4a, red arrows). This behavior is consistent with optokinetic nystagmus in vertebrates in which the eye follows a rotating stimulus and occasionally saccades back in the other direction. We observed clear SRPs in this paradigm, similar to anecdotal evidence in a previous report²⁶, and these were often sufficiently large to return V_m to near rest (Fig. 4a and Supplementary Video 2).

Figure 4. Saccade-related inputs to optic flow processing neurons have the correct sign and magnitude to cancel reafferent visual input during saccades.

(a) Vm of an HSN cell and L-R WBA during preferred-direction motion (blue), null-direction motion (red), and no-motion (black) conditions. Raw Vm is shown in gray; low-pass filtered traces (20 Hz cutoff) are in black, red, or blue. (b) Saccade-triggered averages of L-R WBA and baseline-subtracted Vm, showing saccade-related potentials (SRPs). The baseline Vm was calculated in the 50 ms interval prior to saccade onset. Individual saccades were detected, aligned to their onsets, and averaged with other saccades obtained under the same stimulus condition. Individual cell averages are shown in gray. Averages across recorded cells are shown in black, blue and red (n = 10 HSN cells for the left two columns; n = 6 HSN cells for the right two columns in which 4 cells were dropped because the number of detected rightward saccades was < 5). (c) SRPs and visual responses have the opposite sign, but comparable magnitudes, for leftward saccades. The magnitude of SRPs and visual responses were calculated for each leftward saccade (Online Methods) and averaged for each cell in the absence (black dots) and presence (blue dots) of rightward visual motion. The magnitude of the visual grating-induced depolarization was also calculated prior to each SRP (blue dots on right, cf. Online Methods). Individual points represent individual cell averages with lines connecting the same cell across conditions. Note the inverted y-axis for visual responses, to facilitate comparing the magnitude of visual potentials and SRPs. Error bars indicate SD in this and all subsequent plots. (d) The magnitude of each leftward saccade was calculated (Online Methods) and averaged for each cell in the absence (black dots) and presence (blue dots) of visual motion. These distributions did not differ significantly ((t(9) = 0.003, P = 0.998). (e) Same as in panel c, but for rightward saccades. (f) Same as in panel d, but for rightward saccades. These distributions did not differ significantly (t(5) = 1.33, P = 0.242)

We developed an algorithm to cull saccades from the L-R WBA signal (Online Methods) (Supplementary Figure 3c–d) and plotted saccade-triggered averages of V_m (Fig. 4b) alongside metrics of culled saccades (Fig. 4c–f). With a uniformly lit screen, spontaneous saccades were associated with hyperpolarizing or depolarizing SRPs (Fig. 4b, black curves), which had an average value of −3.5 mV and +1.8 mV (Fig. 4c,e black points). During visual stimulation, the mean amplitude of the hyperpolarizing SRPs grew from −3.5 to −5.5 mV (t(9)=3.87, P = 0.004) (Fig. 4c, black and blue points connected with lines), a magnitude just shy of, and not significantly different from, the average ongoing visual depolarization (+6.4 mV) they counteracted (t(9) = 1.21, P = 0.258) (Fig. 4c, comparing the two sets of blue points). Similarly, the mean amplitude of the depolarizing SRPs (+1.8 mV) grew larger in the presence of visual motion (+2.9 mV), though this change did not reach statistical significance (t(5) = 2.01, P = 0.101) (Fig. 4e, black and red points connected with lines). Depolarizing SRPs during visual motion were not statistically different in magnitude from the ongoing visual hyperpolarization (−3.0 mV) that they counteracted (t(5) = 0.30, P = 0.774) (Fig. 4e, comparing the two sets of red points). Note that all measured potentials are small in absolute magnitude likely because they were measured at the soma, which is connected to the rest of the cell through a thin, electrically passive, neurite that is likely to attenuate the signal.

These data show that SRPs have not only the correct sign, but also sufficient magnitude, to counteract visual signaling in HSN cells. The fact that with a uniformly lit screen saccades in one direction lead to depolarizations from the resting V_m and saccades in the other direction lead to hyperpolarizations from this same potential (Fig. 4b, black curves) demonstrates that saccades do not activate a single, invariable, inhibitory conductance. This observation differentiates motor-related modulation in HSN cells from past examples of efference copy in insects that are consistent with simple inhibitory gating^6,27. Our data suggest that a more subtle calculation takes place in HSN cells, where each voluntary saccade is associated with a sign-inverted prediction of the visual motion input expected for a given motor action.

Optic-glomeruli interneurons are silenced during flight saccades

Are SRPs limited to cells in the lobula plate that are specialized for processing optic flow? Beyond the optic lobes, visual information is further processed in the optic glomeruli of the lateral protocerebrum^28,29. There have been very few physiological studies on the optic glomeruli³⁰ and the function of these neuropils in fly vision remains unclear. We recorded from previously uncharacterized optic-glomeruli interneurons (OGINs), which were labeled in GAL4 line 290 from the InSITE collection (Fig. 5a and Supplementary Figures. 4–5)³¹. The majority of these OGINs responded strongly to small moving objects but were not at all responsive to optic-flow (Fig. 5b,c), arguing that these neurons contribute to object-related behaviors. OGINs responded to a moving spot with a membrane depolarization independent of the spot’s motion direction, and they were visually sensitive at all tested positions in the ipsilateral visual hemisphere (Supplementary Figure 4). Thus, OGINs provided a functional contrast to HSN cells, both in their lack of direction selectivity and in their preference for small moving objects rather than optic flow fields. The receptive field properties of these OGINs appear superficially similar to small target movement detectors described previously in dragonflies and hoverflies³².

Figure 5. Saccade-related inputs to small object-selective optic-glomeruli interneurons have the correct sign and magnitude to cancel reafferent visual input during saccades.

(a) Schematic and immunohistochemistry of InSITE GAL4 line 290, at three z-levels, whose green fluorescent protein expression marks optic-glomeruli interneurons (OGINs) (max z-projection; nc82 anti-Bruchpilot signal of neuropil is in magenta). These representative images illustrate the anatomy of the neurons targeted for electrophysiology. (b) Sample Vm of an OGIN in response to gratings or small spots during non-flight. 4–6 single trials are shown in gray and averaged responses in black. All OGINs were recorded in the right half of the brain. (c) Averaged Vm of 21 OGINs in response to gratings or small spots during non-flight episodes. (d) Vm and L–R WBA in a single OGIN during flight. Periods when the spot was in the receptive field (Online Methods) are highlighted in blue and red. Saccades that took place while the spot was in the receptive field were analyzed to generate the blue and red curves/points in panels e-i. (e) Average SRPs for different stimulus conditions from 21 OGINs. Same format as Figure 4b, except that we also show SRPs associated with a spot stimulus. (f) SRPs and visual responses have opposite sign, but comparable magnitudes, for leftward saccades. Same format as Figure 4c, except that the visual response is to a rightward moving spot rather than a grating. The mean SRP was −0.7 mV in the absence of a moving stimulus (black dots) and −4.4 mV with a rightward moving spot (blue dots) (t(20) = 11.62, P = 10⁻¹⁰). The mean visual response to a rightward moving spot (blue dots) was 5.4 mV, whose absolute value was slightly larger than the mean associated SRPs at −4.4 mV, though this difference was not statistically significant (t(20) = 2.69, P = 0.014). (g) The magnitude of each saccade was calculated (Online Methods) and averaged for each cell in the absence (black dots) and presence (blue dots) of visual motion. Leftward saccades were a bit larger during visual motion (−28° vs. −24°, t(20) = 3.62, P = 0.002). (h) Same format as Fig. 5f, except that the visual response is to a leftward moving spot. The mean SRP was −0.9 mV in the absence of a moving stimulus (black dots) and −4.0 mV with a rightward moving spot (red dots) (t(20) = 9.07, P = 10⁻⁸). The mean visual response to a leftward moving spot (red dots) was 5.9 mV, whose absolute value was similar to, but slightly and significantly larger than, the associated SRPs at −4.0 mV (t(20) = 3.57, P = 0.002). (i) The magnitude of each saccade was calculated (Online Methods) and averaged for each cell in the absence (black dots) and presence (red dots) of visual motion. Rightward saccades were a bit larger during visual motion (28° vs. 25°, t(20) = 3.46, P = 0.003).

Analyzing 21 OGINs (Online Methods), we found clear evidence for saccade-related potentials (Fig. 5d). Unlike with HSN cells, however, saccades to the left (Fig. 5d, blue arrows and black arrows) or right (Fig. 5d, red arrows) both yielded hyperpolarizing SRPs, which is predicted for a cancellation function because the visual responses of these neurons are consistently depolarizing. As with HSN cells, SRPs were largest when cells were driven by strong visual input, which, in the case of OGINs, was a moving spot (Fig. 5e–i, see legend for statistical tests). SRPs during spot motion were not just opposite in sign, but also comparable in magnitude, to the visual response measured at the soma (Fig. 5f–i), arguing that OGIN SRPs robustly cancel, rather than just weakly modulate, visual signaling (Supplementary Video 3).

Motor-related inputs arrive with the correct latency to silence visual neurons during saccades

If the fly brain aims to effectively silence visual neurons during saccades, then SRPs must arrive with an appropriate latency for counteracting the expected visual input. To find out if SRPs are appropriately timed, we measured visual latencies of HSN cells and OGINs to rapidly sweeping visual stimuli that simulated the reafference experienced during a saccade (Online Methods) (Fig. 6a–c). For HSN cells, we presented a wide-field stimulus with naturalistic spatial-frequency statistics, and for OGINs we swept an 18° dark spot across the ipsilateral hemifield. All stimuli moved with the same saccadic velocity profile (Online Methods). OGINs responded with an average latency of 51 ms to a spot moving from front-to-back or back-to-front in the ipsilateral side (Fig. 6d, black curves). By comparison, the average latency between the first detectable change in the wingbeat signal during saccades and the arrival of the motor-related SRP to these neurons was 68–69 ms (Fig. 6e, black curves and Supplementary Table 1). The 17–18 ms discrepancy between visual latencies and SRP latencies in OGINs is consistent with the fact that it takes 3–4 wingstrokes (~15–20 ms) before changes in wingbeat kinematics induce a fly to start turning^22,33. Thus, the visual rotation of the world on the retina is expected to start only 15–20 ms after the wings first start to drive a saccade, and the OGINs will respond 51 ms after this rotation begins, yielding a total visual latency of 66–71 ms, which matches well with the measured SRP latency of 68–69 ms.

Figure 6. Visual and saccade-related inputs have similar latencies of arrival to fly visual neurons.

(a) We swept visual stimuli along a saccadic velocity profile, with a peak velocity of 1000°/s, as indicated (Online Methods). (b) A sample HSN cell depolarizes in response to a wide-field stimulus sweeping with a saccadic velocity profile in the preferred direction and hyperpolarizes in response to the same stimulus sweeping in the null direction. Each trace represents a single trial. (c) A sample OGIN depolarizes in response to a small spot sweeping with a saccadic velocity profile in either direction in the ipsilateral hemifield. (d) Population-averaged Vm responses from OGINs (black) and HSN cells (red) to a spot stimulus and a widefield stimulus, respectively, presented with saccadic velocity profiles shown in panel a. Onset latencies of each curve are indicated with arrows. (e) HSN cell SRPs are reproduced from the blue and red traces in Fig. 4b (green). OGIN SRPs are reproduced from the blue and red traces in Fig. 5e (black). Inset shows Vm surrounding the time of saccade onset at higher temporal resolution. All data in this figure were collected in flying flies.

HSN cells responded to saccadic visual motion twice as fast as OGINs, with an average latency of 24 and 30 ms for preferred and null directions, respectively (Fig. 6d, green curves). As expected, SRPs arrived earlier to HSN cells than to OGINs (Fig. 6e, green curves), consistent with SRPs functioning to cancel the expected visual input associated with saccades in both cell classes.

Motor-related inputs to lobula-plate neurons arrive before saccade-associated changes in wing kinematics or saccade-associated head movements.

Interestingly, the average SRPs in HSN cells began before the onset of the saccade (Fig. 6e insets and Supplementary Table 1; −39 ms for leftward saccades and −25 ms for rightward saccades), as estimated with video analysis or with a photodiode that measured the amplitude of each wingstroke at high temporal resolution (Supplementary Figure 6). Even though the flies’ heads were glued in place in our preparation, flies typically make head movements in addition to body movements during saccades³⁴. In principle, the early inputs to HSN cells could have reflected attempts to move the head prior to the wings with each saccade. To address this possibility, we measured the relative onset of head movements and wing movements during tethered flight saccades, in separate behavioral experiments where the head was free to move. We found that saccadic head movements typically followed saccadic wing movements by ~20 ms in tethered flight (Fig. 7). Thus, whether SRPs in HSN cells function to cancel self-generated visual motion caused by head or wing movements³⁵, the early input to these cells is unlikely to reflect sensory feedback resulting from these movements. Instead, the data support the hypothesis that motor-related inputs to HSN cells reflect an internally generated signal (Supplementary Figure 7). In contrast to the predictive inputs to HSN cells, motor-related inputs to OGINs arrive 68–69 ms after the wings move, meaning that SRPs to OGINs could reflect well-timed proprioceptive feedback—for example from mechanoreceptors sensitive to the animal’s own wing movements during flight³⁶—or internal processing.

Figure 7. Analysis of tethered, flying flies that are free to move their heads shows that head movements slightly follow saccade associated changes to wing kinematics.

(a) Drosophila tethered at the tip of their thorax to a fixed tungsten pin, with their heads not glued and free to move, generated spontaneous saccades while viewing a uniformly lit screen. We tracked head angles relative to the long axis of the body and wingbeat angles, as indicated, with analysis of videos captured at 100 frames / s. (b) Sample traces of L–R WBA and head angles show a strong correlation between wing and head movements. (c) We computed the cross-correlation function between head angle and L-R WBA signal, for each flight bout > 6 s separately, and averaged this signal for a given fly. Each line represents the average cross-correlation function from a single fly (n = 5 flies). (d) We culled saccades from the L–R WBA trace as described (Methods) and plotted the head angles associated with each saccade. Gray lines represent averages from individual flies. Black and red traces represent averages across flies. Insets show the same traces at higher temporal resolution.

Discussion

The data show that Drosophila visual neurons receive both visually driven and motor-related inputs with voluntary saccades. Motor-related inputs are tailored in sign and timing to effectively suppress neuronal responses to the reafferent visual motion resulting from saccades (Fig. 6 and Supplementary Figure 8). We observed motor-related modulations in cells that arborize in two late stages of visual processing, the lobula plate and the optic glomeruli, and these modulations are sufficiently strong to cancel, rather than just weakly modulate, visual signaling (Figs. 4 and 5).

Von Holst and Mittelstaedt⁴ were careful to distinguish silencing of expected visual motion—the computation they believed to take place in fly brains—from complete blindness during voluntary turns. At the cellular level, some visual neurons, like VS1–2 cells, do not receive prominent motor-related inputs (SRPs) during voluntary saccades (Fig. 2), and other cells, like HSN cells, are modified in a direction-selective manner that should allow them to still respond to unexpected (exafferent) visual motion (Fig. 4). Thus, while flies suppress visual input during rapid flight turns, they are not blind, supporting von Holst and Mittelstaedt’s model, whose validity in flies had been speculated on for decades^10,11,13,37. In many scenarios, it might make sense for motion-sensitive neurons to actually sense reafference³⁸, however, because HSN cells likely contribute to optomotor stability, eliminating reafference in their output signal during voluntary turns is sensible. It has been argued, based on behavioral experiments, that flying flies completely ignore certain visual motion during saccades³⁹ or selectively ignore visual motion depending on whether this motion is in the expected or unexpected direction¹¹. Our data provide a plausible cellular explanation for these behavioral results.

While the fly visual system, overall, shows cell type-specific and direction-selective silencing, it is not yet clear whether von Holst and Mittelstaedt’s computation is instantiated in its full form in the V_m of single cells. To do so, each visual neuron would receive a saccade-associated input that is not just of the correct sign, but also of the correct time-varying magnitude to exactly cancel the expected visual drive associated with each voluntary turn. Activating such a negative-image input^7,8 would require that fly brains instantiate a forward model (Supplementary Figure 8) to predict the visual drive that each neuron will experience from a given saccade. In realizing such a model, flies should scale their silencing signals by an internal estimate of the velocity time course of the upcoming saccade (using a so-called inverse model), as well as by how strongly the current visual environment (e.g., a forest or a fog) is expected to drive each visual neuron during each saccade. If flies make use of internal models of this sort, it will be important to determine how they are implemented in the nervous system and how widespread is their influence on sensory processing and behavior.

At face value, the fact that SRPs grow in magnitude when flies generate saccades in the context of a preferred steady-state stimulus compared with a uniformly lit screen (Figs. 4b and 5e) and the fact that this growth of SRP magnitude is quantitatively matched to the level of ongoing visual drive (Fig. 4c–f and Fig. 5f–i) supports the forward model idea. Note, however, that if saccade-related inputs activate a consistent membrane conductance, independent of visual context, for saccades of a certain direction and size, then saccade related potentials will naturally grow in size as the cell’s V_m moves further away from the reversal potential associated with that conductance. Since ongoing visual stimulation causes cells to depolarize or hyperpolarize from the resting V_m, SRPs may grow in magnitude due to this reason alone. Future work will be needed to differentiate this simple biophysical explanation for why SRPs grow in magnitude during ongoing visual drive—which may represent a rudimentary implementation of a forward model—from a more sophisticated process, in which the strength of motor-related conductances in visual neurons are actively scaled based on the structure of the visual environment. Interestingly, HS cells appear to show similar visual responses to moving natural scenes over a wide range of contrast levels and arrangements of local features⁴⁰. This fact may allow an efference copy system to get away with injecting silencing signals that have a consistent time-varying profile for saccades of a given direction and magnitude, independent of the structure of the visual environment. Flies may also continuously calibrate the strength of their efference copy signal based on the difference between predicted and experienced sensory feedback⁴¹.

Regardless of whether motor-related inputs are scaled by the structure of the visual environment, an important associated question is whether, within a fixed visual environment, the motor-related inputs are scaled by the magnitude and duration of each saccade? Because we do not directly measure torque in our current platform—even if L–R WBA acts as a decent proxy⁴²—and because tethered-flight saccades are known to have altered dynamics relative to free-flight saccades³⁹, it is difficult to provide a definitive answer to this question with current methods.

Classically, only neurons that contribute to optomotor stability, like HSN cells¹⁹, should be silenced during voluntary turns. We found that small object-selective OGINs are also silenced. Why might this be? Object-selective OGINs may contribute to behaviors such as small object avoidance during flight⁴³, or tracking of conspecifics during Drosophila courtship⁴⁴. Distinguishing reafference from exafference would seem critical for such object-orienting behaviors, to prevent behavioral responses to object motion on the retina caused by the fly’s own movements. When a locomoting fly turns in a cluttered visual environment, the image of the entire cluttered panorama, not just the small object, translates globally on the retina and such a global stimulus—simulated by the grating in our receptive field mapping experiments—will not excite these OGINs (Supplementary Figure 4). Thus, the native stimulus selectivity of these cells already helps to distinguish exafferent from reafferent object motion^1,32. However, if a single object were situated on a sparse background, like a spider against a homogenous blue sky, OGINs may very well respond to the reafferent motion of such an object during a locomotor turn and the motor-related silencing mechanism we describe would abrogate this deleterious sensory response.

Mechanistically, SRPs in OGINs are consistent with an inhibitory input that arrives on each saccade, either directly to these cells or to upstream neurons, to reduce feedforward excitatory drive. Because OGINs are spiking cells (though their spikes are often very small when measured at the soma), the role of SRPs in these neurons may be to simply eliminate spike output during saccades rather than to activate a precisely time-varying negative-image input. Indeed, SRPs in OGINs last ~400–500 ms, which is longer than the time course of the expected reafference during a typical saccade (compare black curves in Fig. 6d and Fig. 6e). If OGIN SRPs are the result of mechanosensory feedback associated with saccades (Supplementary Figure 8), this feedback signal is expected to be prolonged in tethered flight because tethered-flight saccades last longer (~300–500 ms) (Figs. 4–5) than free-flight saccades (~50–150 ms²²). By contrast, HSN cells are non-spiking neurons that signal both with hyperpolarizations and depolarizations of V_m. As such, HSN cells receive both depolarizing and hyperpolarizing SRPs (Fig. 4b), whose duration (~150–200 ms) is more closely matched to the expected reafference from a typical saccade (compare green curves in Fig. 6d and Fig. 6e). SRPs in HSN cells may serve a function closer to that of a negative image of the expected visual drive.

The average SRP in HSN cells begins ~30 ms before the wings initiate a saccade (Fig. 6e), arguing for an internal (rather than sensory feedback) origin for these signals. However, if the sole function of motor-related inputs to HSN cells were to silence visual reafference, one might expect the efference-copy signal to kick in only after the body starts turning, once reafferent visual input is arriving at the cell. One intriguing possibility is that the early component of the motor-related input to HSN cells might help to actually drive the voluntary turn, by injecting a small pre-charging signal into the optomotor reflex system, hijacking its natural coupling to the neck and flight motor systems¹⁰. The sign of this early component is consistent with this possibility, though a rigorous test will require a specific manipulation of the SRPs to the optomotor system, which likely includes many more cells than just HS cells. Flies stabilize flight not just with vision, but also with mechanosensory inputs from their modified hindwings, called halteres, and a similar pre-charging idea was postulated to take place in the haltere stability system during turns⁴⁵, a hypothesis that should now be revisited.

Dynamic modulations of visual signaling have been studied primarily during saccadic eye movements in primates^2,46. Our results open the door to studying cellular mechanisms for similar processes in Drosophila. For vision research, flies have already offered key insights into state-dependent sensory processing^15,47–49 and the circuit basis for direction selectivity⁵⁰. Drosophila may now help us understand how brains build forward models and how they use these models to modify sensory processing. The tiny fly brain may not perform these tasks in exactly the manner of the primate brain. However, the advanced genetic and physiological tools in Drosophila should allow for a detailed cellular- and circuit-level description of how the fly brain models and predicts the outside world. This description could yield insight on how similar predictive processes are implemented in all brains, including our own.

Online Methods

Flies

We studied female, Drosophila melanogaster, 0.75–to-3 days post-eclosion. Flies were reared in large bottles (Applied Scientific; 57 mm × 57 mm × 103 mm with a square bottom) with ~5–25 flies per bottle, with standard corn-meal agar, in 25°C incubators with a 12 hour light/dark cycle. Flies were not subjected to any other experiments prior to the study described here. To identify neurons, we crossed the following driver lines to UAS-EGFP reporter lines: DB331-GAL4 for HS/VS cells, GMR81G07-GAL4 for HS cells, and InSITE290-GAL4 for optic-glomeruli interneurons (OGINs). For experiments in which we blinded flies with blue light, GMR81G07-GAL4 flies were crossed with w;UAS-2xEGFP flies, yielding lightly pigmented eyes that facilitated induction of a blind state.

Visual Stimuli

We used a cylindrical visual display⁵¹ covering 216° and 75° in azimuth and elevation, respectively (IORodeo) with each pixel 2.25° in size (570 nm LEDs). We used four–or eight-level gray-scale interpolation to increase the apparent resolution of this display. We tilted the arena by ~30° downward from upright so as to roughly match the fly’s pitch-down head angle during electrophysiological experiments. Grating stimuli were square waves with an 18° wavelength, moving with a temporal frequency of 1 or 1.25 cycles/s. Small dark squares had an edge length of 9ånd moved at 22.5°/s. Dark bars were 9° wide by 75° high and moved at 22.5°/s. Spots, bars, and gratings had a nominal contrast of 100% (though the unavoidable fact that LEDs on one side of a panoramic display illuminate the dark regions on the other side of the display will tend to slightly reduce this value; contrast levels were not measured experimentally).

In Figure 6, we presented a wide-field stimulus with a naturalistic intensity profile along the horizontal dimension. The intensity profile was generated by linearly superimposing sinusoidal waves at a random phase after weighing their amplitudes by the reciprocal of their spatial frequency, thus approximating the known 1/f (f = spatial frequency) statistics of natural scenes. We presented this wide-field stimulus, or an 18° square spot, moving left or right in a manner that approximated the velocity profile of a spontaneous saccade. We approximated the velocity profile of a saccade performed by magnetically tethered flies who were free to rotate about their yaw axis (Figure 2 from ref. 52). The functional approximation was performed by superimposing two logistic functions, one for the rising and the other for the falling phase of the saccade’s velocity trajectory. The stimuli moved 68° over 130 ms, with a peak velocity of 1000°/s. The spot was always presented in the ipsilateral hemifield. With the saccade motion stimulus, the refresh rate of the visual display was increased to 500 Hz to ensure that no single frame update moved the stimulus more than 1 pixel (2.25°), which is approximately half of the typical ommatidial acceptance angle of ~5° in Drosophila⁵². Most frame updates moved the stimulus much less than 1 pixel, which was made possible with gray scale interpolation.

Electrophysiology

We performed whole-cell patch clamp recordings as described previously¹⁵. In most experiments, we fed flies with ~100 nL of 500 mM sucrose solution from a pipette tip, after tethering, to help promote long flight bouts. Experiments were conducted during the afternoon or evening. We perfused the preparation with oxygenated extracellular saline that contained (in mM): 103 NaCl, 3 KCl, 5 N-Tris(hydroxymethyl) methyl-2-aminoethanesulfonic acid (TES), 10 Trehalose, 10 Glucose, 2 Sucrose, 26 NaHCO₃, 1 NaH₂PO₄, 1.5 CaCl_2, 4 MgCl₂, pH 7.3 when equilibrated with 95% O₂ / 5% CO₂ (275 mOsm). Patch-clamp electrodes (4–8 MOhm) contained (in mM): 140 K-Aspartate, 1 KCl, 10 HEPES, 1 EGTA, 0.5 Na₃GTP, and 4 MgATP, 0.02 Alexa-568-hydrazide-Na and 13 Biocytin hydrazide, pH 7.3 (265 mOsm). The membrane voltage was amplified (A-M Systems Model 2400), digitized at 10 kHz (PCIe-6351, National Instruments; Digidata 1440a, Molecular Devices), and saved to a computer (WinEDR, University of Strathclyde; pClamp 10, Molecular Devices). Voltage measurements have been corrected for a 13 mV junction potential. We injected 0–10 pA of hyperpolarizing current into neurons to neutralize the depolarizing effects of the seal conductance⁵³. The membrane resistances of recorded cells were as follows: 147±35 MΩ for 10 HSN cells, 122±49 MΩ for 21 VS1–2 cells, and 771±375 MΩ for 21 OGINs. With HS and VS cells, we determined the cell’s identity immediately after the recording by a combination of the cell’s visual response profiles and examining the cell morphology based on the Alexa-568 fills.

Classification of object-selective OGINs and estimation of receptive fields to small-object motion.

We stably recorded 33 optic-glomeruli interneurons (OGINs). 23 of these were classified as spot-selective based on having a mean depolarization to moving spots in the ipsilateral visual hemisphere that exceeded the magnitude of depolarizations in response to gratings. In recording sessions associated from 21 of these 23 neurons, we measured a sufficient number of SRPs (> 5 for each stimulus condition) to effectively analyze the activity further. We immunoamplified biocytin¹⁵ to visualize OGIN morphology after recordings (Supplementary Figure 5). For Fig. 5, to determine when a moving spot was in the receptive field of an OGIN, we calculated the average visual response to a left or right moving spot in non-flight. We picked a threshold at 50% between the resting membrane potential (in a 3 s window before stimulus onset) and maximum V_m, both calculated from the trial-averaged trace. The first and last V_m samples that crossed this threshold were used to define the width of the receptive field.

Blinding experiment

In order to test whether saccadic potentials are visual in nature, we reversibly blinded the fly with a prolonged depolarizing afterpotential²⁵ (PDA). A PDA can be induced by application of intense blue light to the retina, which causes the majority of R1–6 opsin to photoconvert to a persistently active state, rendering the receptor cells insensitive to any further change in the visual stimulus. Visual sensitivity is recovered with application of longer wavelength light, which returns the opsin to the inactive state^25,54. To induce a PDA we applied a 470 nm light pulse (4.0 mW at the objective’s back aperture) (CoolLED, UK) to the head for 2 seconds. To recover vision, we applied a 565 nm light pulse (0.7 mW at the back aperture) for 2 seconds. To confirm elimination of visual activity in HS cells, we measured visual responses to horizontally moving gratings using a visual display with blue LEDs (470 nm), at the beginning, middle, and end of each PDA interval, which lasted for one minute. We also tried measuring SRPs in normal flies flying in the dark, however, we found that light-adapted flies stopped performing spontaneous saccades at an appreciable rate immediately after turning off the lights and with dark-adapted flies it is difficult to exclude the possibility that the animals might see a dim image of the moving antennae or wings.

Wingbeat measurements

We estimated wingbeat amplitudes of the left and right wings in real time, as described previously^15,55. Video data were collected at 100 Hz with an AVT-GE680 camera. While the frame-by-frame amplitudes for the left and right wings were clear even to the human eye, the real-time algorithm used to estimate these amplitudes, which needed to be simple and fast, would occasionally yield obvious errors due to a bright streak in the video signal at the middle of the wingstroke trajectory that the algorithm would consider as the peak amplitude of the wingstroke on some frames. Depending on lighting conditions for a given fly, we could observe anywhere from zero errors, to ~1% of samples showing a clear miscalculation of the wingbeat amplitudes. To correct these occasional errors, we analyzed videos offline to re-estimate the wingbeat amplitudes on each frame. We saved the time stamp of each frame-trigger pulse and used these to precisely align behavioral and electrophysiological data.

Cross-correlation analysis

To cross-correlate the membrane voltage (V_m) with flight turns (left minus right wingbeat amplitude, L–R WBA), we first high-pass-filtered the signals by a constrained least square FIR filter with a 0.5 Hz cut-off frequency to eliminate slow drift. We culled all the time intervals in the L-R WBA trace in which the fly flew for > 5 s while viewing a uniformly lit screen. For each time interval thus culled, we computed a cross-correlation function between and V_m and L-R WBA. We then averaged all the cross-correlation functions from different time intervals for a given cell to produce a single cross-correlation function for each cell (average number of sampled time intervals per cell = 49, min = 7, max = 101). We also computed cross-correlation functions between Vm and time-inverted L-R WBA signals (salmon and red traces in Fig. 2f). which were used for assessing the statistical significance of the peaks in the gray traces.

Statistical analysis

Two-tailed paired t-tests, with a threshold of P < 0.01, were used in all cases where statistical hypotheses were applied. Normality of all distributions entering a t-test were confirmed by the Kolmogorov-Smirnov test with a critical value of 95%. Sample sizes were similar to past experiments that yielded reliable results¹⁵.

Latency detection

To detect the onset latency of visual responses in the mean V_m curves of Fig. 6d, we first measured the minimum and maximum values during baseline (HSN cells: from 100 to 50 ms prior to the visual motion onset, OGINs: from 0 to 50 ms after the visual motion onset). These max and min values determined two threshold levels. Starting at the peak visual response we moved backward in time and called the point in time at which the V_m curve crossed one of these thresholds, whichever came first, the onset latency. We performed the same procedure for the mean V_m curves in Fig. 6e to obtain a mean latency for SRPs referenced to saccade onset. Using alternative criteria for the threshold that were based on the distribution of values in the baseline period (Supplementary Table 1), changed the estimates of visual response latency by ≤ 6 ms and the SRP latency by ≤ 21 ms. Regardless of the criterion used, the average SRPs in HSN cells always started at least 21 ms before the onset of wing motion.

Saccade detection

We implemented a derivative-based saccade detection algorithm as explained in Supplementary Figure 3. The parameters of this algorithm were selected to optimize the match between what the human eye and the computer deemed saccades. The final chosen parameters are not critical for any of our main results and conclusions, as evidenced the fact that we detected motor influences on visual neurons with a cross-correlation method that did not depend on this algorithm (Fig. 2).

To detect saccades, we low-pass-filtered the L–R WBA (6Hz cutoff) and took the derivative of this low-pass-filtered signal using the central difference formula (70 ms half width). Putative saccades were isolated from the derivative trace by first finding local maxima and minima that exceeded a threshold tailored to the wingstroke statistics of each fly (see below). For each putative saccade, we determined an onset time, peak time, and offset time. The peak time of the saccade was mapped to the point in time at which the derivative signal returned to zero after the local maximum or minimum. A putative onset-time for the saccade was mapped to the point in time at which the derivative signal returned to zero before the local maximum or minimum. Because the derivative signal is low-pass-filtered, without further correction this onset-time approximation was earlier relative to the actual onset. To correct for this error, we found the sample in the low-pass-filtered L–R WBA trace that corresponded to the point in time of the putative onset time, which we considered a threshold; we then looked forward in time in the unfiltered L–R WBA trace for the last sample before this threshold was crossed, which was used to define the actual onset time. To measure the amplitude of each saccade, we subtracted the mean L-R WBA signal in a 50-ms baseline interval immediately before saccade onset from the mean L-R WBA signal in a ±15 ms time window surrounding the peak.

We excluded saccades if two or more occurred in close succession because in this case the offset of one saccade overlapped with the onset of the next. Even with isolated saccades, the algorithm would typically detect the rising and falling phases of putative events as two potential saccades. To deal with both of the above issues, we measured the maximum deviation of L-R WBA in the 200 ms preceding saccade onset and eliminated putative saccades in which this value exceeded 55% of saccade amplitude. We also excluded tonic/persistent turns—which are not traditional saccades—from further consideration. A putative saccade was defined as a persistent turn if the L-R WBA signal stayed more than 45% above baseline for > 700 ms after the peak.

In each individual, we adjusted the L–R WBA deviation threshold to the wingstroke statistics of that fly. While our overall results remain the same even if we did not incorporate this step, we found that many obvious errors could be corrected with an adjustment of this sort. We examined the cumulative distribution function (cdf) of the absolute value of L–R WBA, after high-pass filtering the L–R WBA signal at 1 Hz (events of interest last < 1 s). We set an amplitude threshold for each fly at the L–R WBA value at which the cdf reached 90% of its final value (Supplementary Figure 3c) and, as the last step in the algorithm, we excluded saccades from analysis whose amplitude did not pass this threshold. The derivative threshold, used earlier in the algorithm to isolate putative saccades, was set to six times the amplitude threshold (e.g., a 15° amplitude threshold mapped to a 90°/s derivative threshold).

In Figs. 4c, 4e, 5f, and 5h we measured the amplitude of saccade-related potentials (SRPs) in V_m as follows. First, for a given saccade, we found a local maximum or minimum in the derivative of V_m (low-pass-filtered at 20 Hz to remove spikelets) in a window surrounding the onset time of the saccade (−50 to 100 ms for HSNs; 0 to 200 ms for OGINs). Using the same method described for L–R WBA events, we found the onset and peak times of the SRPs. To measure the amplitude of each SRP, we subtracted the mean V_m in a 50-ms baseline interval immediately before the SRP onset from the mean V_m in a ±2.5 ms window surrounding the peak. All values were calculated from the raw V_m trace. In some experiments, we presented a moving visual stimulus, a grating or a spot, that depolarized or hyperpolarized the cell to a relatively constant potential and we observed SRPs riding on this potential. In these experiments, we quantified SRPs in the same manner just described. We additionally measured a pre-saccadic visual response as the mean V_m in the 100 ms prior to the saccade, from which we subtracted the baseline V_m of the cell, calculated as the mean V_m in the 3 s prior to motion-stimulus onset.

Note that a supplementary methods checklist is available. All analysis code, written in Matlab, is available upon request.