Multisensory systems integration for high-performance motor control in flies

Summary

Engineered tracking systems ‘fuse’ data from disparate sensor platforms, such as radar and video, to synthesize information that is more reliable than any single input. The mammalian brain registers visual and auditory inputs to directionally localize an interesting environmental feature. For a fly, sensory perception is challenged by the extreme performance demands of high speed flight. Yet even a fruit fly can robustly track a fragmented odor plume through varying visual environments, outperforming any human engineered robot. Flies integrate disparate modalities, such as vision and olfaction, which are neither related by spatiotemporal spectra nor processed by registered neural tissue maps. Thus, the fly is motivating new conceptual frameworks for how low-level multisensory circuits and functional algorithms produce high-performance motor control.

Introduction

You are a machine. Neuronal circuits logically combine elemental electrochemical events to evoke each of your most elaborate perceptions, thoughts and emotions. Molecular algorithms are transformed by circuits into spectacularly complex behavioral phenomena. Indeed any sensory perception or motor action must have been computed by cellular networks. Brains across taxa integrate signals from multiple sensory modalities, and the resultant transformation into robust and flexible motor outputs then modify the inputs under reciprocal closed-loop feedback. The integration of disparate sensory signals is adaptive because environmental signals are noisy and often unreliable so combining data from different modalities enhances the reliability or dependability of sensory computations.

This principle has been co-opted by engineers. For example, a tracking system might utilize sound localization through a microphone array and image pixel information from video. Geometric equations can relate the angle at which the target arrives at both sensor platforms to improve the signal-to-noise ratio. In mammals and primates, the superior colliculus houses visual, auditory, and somatosensory tissue maps that are superposed in topographic spatial register to feed the receptive fields of multisensory neurons [1]. Thus, for a visually noisy sound source, spatially coincident auditory input can cross-modally facilitate a directional shift in gaze. More recently, evidence has been accumulating that multisensory processing occurs not only in association centers, but also within primary sensory cortex [2,3].

Multisensory processing is not relegated to large animal taxa with large brains, nor is it necessarily restricted to modalities that share spatiotemporal topology. From the moment it is born in its adult form, a fly's job in life is to seek out the source of an appropriately smelly object upon which it will find mates, food, and oviposition sites. The key challenges to this life history strategy are multisensory; first, a fly moves fast, which tends to compromise sensory information transmission. Second, olfactory resources are sparsely distributed such that there is often no odor plume to track – it must be found first.

Flies show sensory independent search behavior that transitions to sensory dependent tracking behavior when an attractive chemosensory signal is acquired [4,5]. Search trajectories in flight are characterized by segments of straight flight punctuated by ballistic changes in orientation called saccades for their functional analogy to our own gaze ballistic eye movements [6,7]. During search, saccade intervals tend to be short, with infrequent long relocation intervals [8]. Upon encountering a plume, the animal reduces both the frequency and amplitude of body saccades, and maintains a stable forward heading with reduced rotational optic flow (Figure 1A).

Figure 1.

Behavioral features and multisensory systems. (A) A cartoon highlighting the central sensory-ecological challenges to a fly: (i) in the absence of sensory cues, search with short inter-saccade intervals (ISI), (ii) upon acquiring a sensory signal, track the unpredictable odor plume, (iii) visually stabilize heading and avoid collisions. (B) Select sensory inputs.

Flies are equipped with an array of sensor inputs including simple lens ocelli that track the position of the horizon, compound eyes that provide input to process optic flow, antennae that measure mechanical disturbances and chemical signals, and mechanosensory halteres that act like a gyroscope (Figure 1B). These sensory inputs are integrated for the motor control of visual gaze through neck muscle motoneurons, antennae, wing, and haltere kinematics through cognate motor circuits.

In flies therefore, we find an exquisite research model characterized by highly sophisticated yet tractable motor behaviors that are under the control of many sensory inputs. Some of the integration algorithms are best understood on the quantitative behavioral level, whereas others have been resolved electrophysiologically within specific neuronal circuits. Combined, these results offer insight into how low-level multisensory algorithms produce high-level behavioral control.

Visual-olfactory integration

Classic experiments with moths flying freely in wind tunnels have revealed that upwind plume tracking is enhanced by the presence of rich visual feedback generated by a high contrast visual panorama [9]. Indeed visual feedback is useful for stabilizing an upwind heading since a flying animal has no other independent sensory reference to a ground vector [10,11]. However even in the absence of any background ambient wind, fruit flies fail to locate the source of an attractive odor in the absence of a richly textured visual panorama, which hints at a visual dependence that is not related to upwind tracking [12**,13].

A simple experiment revealed that attractive odor has a context dependent influence over the gain of optomotor responses during flight [14*]. Flies were rigidly tethered within a visual flight simulator and exposed to a plume of apple cider vinegar. The animals were tested for optomotor responses to two independent axes of optic flow, side-slip translation and rotation [15-17]. For side-slip, odor has the effect of reducing the amplitude of compensatory optomotor steering responses, whereas for the same group of flies rotation responses are amplified (Figure 2A). Increasing the strength of rotation responses would enable a fly to maintain a straight heading upon exposure to a plume (Fig. 1A), while reduced translation responses might facilitate approach to a landing surface.

Figure 2.

Visual-olfactory behavioral algorithms. (A) A fly tethered within an electronic visual flight simulator is presented with a plume of food odor. In response to oscillation of the visual panorama in an increasing frequency sweep, flies adjust their wing kinematics for a classical optomotor response. The difference in wing beat amplitude across the two wings (ΔWBA) is proportional to yaw torque. (B) A fly tethered to a pin and suspended in a magnetic field beats its wings and steers freely in the horizontal (yaw) plane. A plume of food odor is delivered at one side of the circular arena (0 degrees, orange triangle). At the start of the trial, the animal is positioned 90 degrees to the right of the plume (blue arrow). Solid lines indicate individual flight trajectories, grayscale coded for individuals. Silhouettes indicate approximate heading at three time points. The spatial odor gradient is not drawn to scale.

Flies are equipped with a pair of olfactory antennae. In larvae the pair have been shown to effectively boost signal detection [18**]. In walking adults, the pair have been shown to encode spatial variations in the static odor gradient [19]. Could flies use the minute intensity gradient across the antennae to steer up a plume during flight? An assay was constructed to measure active tracking of a spatially discreet plume [20]. A fly tethered to a pin and suspended within a magnetic field, enabling free rotation in the yaw plane, was placed within a cylinder of light emitting diodes that displayed visual stimuli and thereby allowed smooth pursuit and saccade behaviors qualitatively similar to those in free flight [21]. The arena was fitted with a nozzle that delivered a mass-flow regulated vapor stream on one side of the arena [20]. Flies did not track a control water plume, but actively tracked a plume of attractive vinegar vapor at identical flow rate.

In the magnet arena, the test for motion-dependent odor tracking required switching from a high contrast pattern of stripes that generates strong motion cues to an isoluminant grayscale which evokes no motion cues but maintained the light-adapted state of the photosensory systems. The switch effectively abolished the flies' ability to actively track the attractive vinegar plume [12**]. This was consistent with free-flight behavioral results indicating that flies are unable to locate a source of apple cider vinegar in still air if the walls of a 1-meter flight arena are lined with uniform grayscale rather than a high contrast checkerboard [13].

To specifically examine whether a fly can track a spatial gradient across the antennae while in flight, a vertical bar (a very attractive feature for a fly) was rotated around the arena to visually “drag” the animal 90-degrees to the side of the vinegar plume. Upon “release” each fly steered directly up the gradient, back toward the odor nozzle (Figure 2B left). Remarkably, the flies could not re-acquire the plume in the uniform grayscale visual arena (Figure 2B right) [22].

Strong visual-olfactory behavioral interactions have been documented in several fruit fly studies [12**-14*,23]. However, it is worth noting that whereas flies won't normally take to the wing in the dark since they can't see, tethered animals can be coaxed to fly within a dark arena [11,21]. A featureless grayscale panorama, in which the photosensory systems are driven without the contrasting features required for motion detection is not equivalent to flying in the dark, in which there is no photosensory input at all, masking the diurnal conditions under which flies naturally track plumes but potentially relaxing the visual dependence on odor tracking.

Visual integration for plume tracking in fruit flies operates only with panoramic visual cues. Small visual landmarks insufficient to elicit stable plume tracking [12**]. This is in contrast to what has been shown in hawkmoths, animals that forage for the nectar of visually conspicuous flowers, and that fuse information about the visual position of a flower and the spatial location of the plume [24], and may be processed by circuits within the mushroom body [25*], a neuropile that has been shown to gate decisions and modulate the apparent perceptual salience of environmental features [26].

However, unlike moths, Drosophila melanogaster are dietary generalists that have no particular need to visually identify the source of an odor (they alight on a wine glass as readily as on a piece of fermenting fruit). The evidence shows that odor selectively modulates optomotor equilibrium responses to facilitate straight flight in a plume. This low-level integration algorithm does not require object recognition or scene segmentation, but rather would operate in whatever visual landscape in which the animal may be tracking an appetitive odor plume.

Olfactory-mechanosensory integration

For Drosophila, the requirement of two intact antennae to track a spatial odor gradient in flight presents something of a paradox: the vast majority of olfactory sensory neurons project bilaterally to the first order antennal lobes [27]. There is evidence that unilateral stimulation results in higher metabolic activity in the ipsilateral axonal projections [28], but excitatory post synaptic currents have been shown to be similar across the two antennal lobes [29].

By contrast to olfactory sensory pathways, all mechanosensory neurons comprising the Johnston's Organ (JO) project ipsilaterally to a region that integrates antennal mechanosensory afferents and motor efferents [30], serving antennal proprioceptive and auditory functions [31]. During tethered flight, normal upwind orientation is perturbed by JO occlusion [11]. The antennae are sensitive to a rich array of stimuli, but are not passive sensors. Instead they are articulated with muscles and can be seen to twitch during flight, which may provide insight into their role in sensorimotor transformations.

Anatomical evidence suggests that third-order chemosensory interneurons and primary mechanosensory neurons converge in the brain at the antennal motor center [32], which could potentially provide multisensory coordination of antennal movement and feedforward activation of steering saccades [22]. Indeed, unilateral immobilization of the JO joint results in constitutive steering in the contralateral direction (Figure 3). A reasonable hypothesis motivated both by these results and by work in other insects [33,34], is that in flies antennal movements are both evoked and sensed during steering maneuvers, and that these feed-forward and feedback signals are somehow biased by asymmetric olfactory signals to mediate gradient tracking [22].

Figure 3.

Gradient tracking requires antennae to be mechanically functional. Experiments are similar to those in Figure 2, except that the mechanosensory Johnston's organ (JO) of the left antenna was immobilized with non-toxic epoxy (indicated in red).

Mechanosensory-visual integration

During flight, flies exhibit robust optokinetic stabilization responses to reduce image blur on the retina [35]. The retina is fixed, so eye movements are controlled by muscles of the neck that move the whole head. The fly stabilizes its gaze both during saccades [36-38] and also to counteract unplanned mechanical perturbations such as by a gust of wind [21,39]. Body rotations are encoded by gyroscopic equilibrium organs called halteres [40] that are themselves under neuromuscular control and can be steered much like the wings [41]. The sensory signals from the halteres are electrotonically coupled to wing muscle motoneurons (Figure 4), forming an extremely fast local reflex circuit to coordinate nearly instantaneous steering corrections to abrupt changes in body position [42].

Figure 4.

Multisensory reflex loops revealed by electrophysiology.

There are as yet no physiologically identified inputs directly from the visual system to the wings. Rather, visual motion evokes muscle potentials in the haltere steering muscles [41], which would presumably evoke corrective steering responses through the haltere sensory-motor arc to the wings. Visual and haltere signals also converge upon the neck motor system. Motoneurons are activated by the summation of visual and haltere afference, which together bring the motoneurons to firing threshold [43,44*]. The integration of visual and mechanosensory feedback at neck and haltere motor circuits ensures that amplitude-coded visual motion signals are transformed into phase-coded motor signals to coordinate the cycle-by-cycle variations in wing steering kinematics.

Visuo-visual integration

Insects in general and flies in particular have for decades been a rich model system to study the cellular basis of motion detection [45]. Most of what we understand of motion processing in the fly brain comes from in the 3^rd optic ganglion, the lobula plate, which houses some 60 large tangential cells (LPTCs). These interneurons have large dendrites and large receptive fields that are assembled from a retinotopic array of local motion detectors each with small receptive fields. Within some LPTCs, the spatial specification of local directional dendritic input is such that the full receptive field of the neuron matches the spatial patterns of optic flow on the retina generated by specific flight maneuvers such as roll or pitch [45]. These complex LPTC receptive fields are formed by a combination of dendritic integration and lateral synaptic interactions between axons and dendrites. Indeed fully integrated axonal receptive fields are not restricted to their cognate dendritic inputs, but rather they also incorporate inputs from neighboring LPTCs though gap junctions [46**].

Remarkably, the LPTCs also receive excitatory input from another visual modality entirely, the ocelli (Figure 4), which provide information about the overall light level and are thought to detect the visual horizon [47] and thus help keep the fly oriented upright against the bright sky [48*] where pursuit targets such as mates or territorial conspecifics would be strongly contrasted against the ultraviolet rich sky. The excitability of LPTC membranes are modulated during active flight both by heterosynaptic and neuromodulatory signals presumably to meet the increased performance demands of flying by comparison to walking [49,50]. As a result, the dynamic tuning properties of behavioral equilibrium responses themselves are shaped by the animal's current flight trajectory [51*].

Conclusions

In flies we find the fastest visual kinetics, an olfactory system that approaches the theoretical limit of chemical sensing, and a mechanosensory system that encodes complex forces on a wing beat time scale. These sensory systems converge upon one another and upon motor circuits to enhance the detection of sensory signals and also to synchronize sensorimotor coordination of high performance locomotion. It would appear that nearly every sensory modality in flies is wired to every other modality, providing robust reflex arcs that insure behavioral robustness in a noisy and unpredictable environment. Thus far, we see no example of a sensory modality that operates entirely independently of the others, making it difficult to fathom non-deterministic behavior in flies denied feedback from only one modality [52].

By contrast, under the presumption that any stimulus that is perceived must have been computed, it would appear that multisensory integration in flies operates in large part at surprisingly basal computational levels, but results in a highly complex and robust behavioral repertoire. Similar behavioral complexity can be observed in Braitenberg vehicles [53] in which simple operational algorithms evoke surprising behavioral complexity that would not be predicted by first principles.

The future is rich for multisensory research in flies. In cases where the neuronal microcircuits for multisensory integration have been well described, the behavioral ramifications have yet to be fully explored. Conversely, in instances where the behavioral algorithms have been well analyzed, the underlying circuits are mysterious. Future studies will benefit from integration across these levels of analysis.