Cycle-to-cycle variability as an optimal behavioral strategy

Abstract

Aplysia feeding behavior is highly variable from cycle to cycle. In some cycles, when the variability causes a mismatch between the animal's movements and the requirements of the feeding task, the variability makes the behavior unsuccessful. We propose that the behavior is variable nevertheless because the variability serves a higher-order functional purpose. When the animal is faced with a new and only imperfectly known feeding task in each cycle, the variability implements a trial-and-error search through the space of possible feeding movements. Over many cycles, this may be the animal's optimal strategy in an uncertain and changing feeding environment.

Animals live in an uncertain environment. They must act on the basis of sensory information that is ambiguous and incomplete; furthermore, they often must act in anticipation of the future, which is inherently unknown. Two strategies for dealing with this problem would seem to be possible. The better-known strategy is to refine the sensory capabilities so as to discern the state of the environment, and to evolve “intelligent” computing power to predict the future, as accurately as possible. However, there is another, indeed the opposite, strategy—to produce behavior that will always be good no matter what the state of the environment may be. We believe that we may have encountered a significant component of this strategy in the feeding behavior of the marine mollusc Aplysia.

Aplysia feeding behavior is highly variable

Aplysia consummatory feeding behavior—biting, swallowing, and rejection of unsuitable food—is a cyclical, rhythmic behavior produced by a complex neuromuscular system: the contractions of numerous muscles in the animal's feeding organ, the buccal mass, each controlled by the firing patterns of its individual motor neurons, all driven ultimately by the motor programs of a central pattern generator (CPG) in the buccal ganglia [5, 7]. Traditionally, the behavior has been thought of as stereotyped. Yet we have recently found that essentially all of the basic parameters of the cycling of the CPG, motor neuron firing, and contractions of the individual muscles [6], as well as higher-level parameters of the coordination between entire motor neuron-muscle subsystems within the feeding neuromusculature [11], and finally measures of the behavior itself [8], are extremely variable from one cycle of the behavior to the next. Several dimensions of this variability can be seen in Figs. 1-3. Figs. 1 and 2 present data from the ARC muscle, an extensively-studied, representative buccal-mass muscle [2, 4]. Fig. 1, top, shows the electrical activity in the muscle recorded with a chronically implanted electrode during an entire ∼2½-hour meal comprising 749 cycles of the feeding behavior. The successive expansions below show well the “fractal” nature of the activity, with the bursts corresponding to successive cycles grouped on multiple time scales, and the great variability in the timing and shape of each burst. A decomposition of the activity in the entire 2½-hour record to the firing frequencies of the muscle's two motor neurons, B15 and B16 [4], is shown at the top of Fig. 2. The rest of Fig. 2 summarizes the great variability in each of the principal parameters of the B15 and B16 firing pattern in successive cycles. Note, in particular, the histograms at the far right, showing the distributions of a local, cycle-to-cycle measure of variability that eliminates any global progressive trends, namely the pairwise differences in the value of the parameter between successive cycles, then normalized by the mean value of the parameter. With this normalization, −1 and +1 on the horizontal axis represent a decrease or increase, respectively, in the value of the parameter from one cycle to the next that is equal in magnitude to the mean. There are many cycle-to-cycle differences as large as this. Finally, Fig. 3 shows again the variability in the ARC electrical record as well as the variability of the motor programs produced by the CPG, reflected in the electrical activity recorded in a nerve that carries motor information from the buccal ganglia to the buccal-mass musculature, while the animal was swallowing, spaghetti-like, long, regular strips of seaweed. Many cycles were required to swallow each strip. In each cycle, the strip presented precisely the same sensory stimulus, yet successive cycles were very different. Remarkably, therefore, the large cycle-to-cycle variability is present even when the sensory stimulus, or, in more functional terms, the functional task or functional goal of the behavior, is the same in each cycle.

Fig. 1.

Variability of electrical activity in the ARC muscle recorded with a chronically implanted electrode during an entire 2½-hour meal in an intact, freely feeding animal. Modified from [1, 6].

Fig. 3.

Variability of successive feeding motor programs, and of functional feeding performance, during ingestion of long, regular seaweed strips. Top to bottom, electrical activity in the ARC muscle and in buccal nerve 2 recorded with chronically implanted electrodes and the movement of the seaweed strip recorded with a length transducer. From the experiments in [8].

Fig. 2.

Variable firing patterns of the ARC motor neurons B15 and B16 during the 2½-hour meal in Fig. 1. Top left, the instantaneous firing frequencies of B15 and B16 decomposed from the electrical activity in Fig. 1. Below, time series of four parameters of the firing pattern in each of the 749 cycles of the meal. To the right of each time series is a marginal histogram summarizing the distribution of the values, scaled so as to approximate the probability density function. At far right is the corresponding distribution of normalized cycle-to-cycle differences: pairwise differences between successive cycles, then normalized by the mean value of the parameter. The lines mark the 25th, 50th (median), and 75th percentiles, and the standard deviation of the distribution, σ, is given. The small arrows indicate that the first or last bar contains pooled smaller or larger values. Modified from [1].

To a first approximation, the variability is quasi-random, uncorrelated across different parameters in the same cycle or the same parameter in successive cycles [1, 6, 8, 11]. Furthermore, the variability at all levels of the neuromusculature appears to be driven by the variability of the motor programs produced by the CPG. Regularizing merely the timing of the motor programs regularizes many of the downstream neuromuscular parameters too [6, 12].

Hypothesis: the variability implements an optimal feeding strategy

In view of the functional penalties that the variability incurs in some cycles (see below), why does the CPG generate, almost actively it would seem, this variability? We have proposed [1, 6, 8, 11] that the variability is an integral part of the animal's strategy for dealing with an uncertain environment. At any point in its meal, the feeding Aplysia may be confronted with any of a wide range of seaweed types and qualities that are best ingested with somewhat different feeding movements. It is likely, furthermore, that the precise requirements of the best movement at any point are unknown. External tactile and chemical cues cannot fully distinguish, for example, the toughness of a piece of seaweed. How the seaweed can best be eaten, indeed whether it can be eaten at all, can only be determined, by internal sensory feedback from the buccal mass and esophagus, once the attempt has actually been made. In these circumstances, a trial-and-error strategy may be best. The CPG randomly generates a wide range of movements that efficiently sample the entire space of likely possibilities. Some of these movements will fail—and in a herbivore the penalty for failure in any particular cycle is relatively low—but at least some will succeed. The successful movements can then be reinforced and stabilized, by decreasing the range of the variability, so that in the next cycle the CPG is more likely to produce a successful movement and less likely to produce an unsuccessful movement. In this manner, we propose, the variability implements a feeding strategy that is optimal for the animal in an uncertain and changing environment.

Evidence for the hypothesis

A critical prediction of the hypothesis is that the internal variability generated by the CPG should pass through the neuromusculature to emerge in the feeding movements and indeed in the functional performance of the feeding behavior. The bottom record in Fig. 3 shows the movement of the seaweed strip—a readout of the swallowing movements and the same time a measure of functional performance, the amount of seaweed swallowed—recorded with a length transducer. The strip movements are indeed very variable from cycle to cycle, and there are even some completely unsuccessful cycles with no net strip movement at all or even net movement out of the mouth [6, 8]. According to our hypothesis, the penalty of these unsuccessful cycles is outweighed by the higher-order benefits that the variability brings overall.

In at least some experiments of the kind shown in Fig. 3, when the variability is followed through the sequence of cycles used to swallow each seaweed strip, there is indeed a progressive decrease in the variability, a stabilization of the feeding movements when the animal experiences the same stimulus—faces the same feeding task—in cycle after cycle [6].

Finally, is the variable trial-and-error strategy really “optimal”? To answer this question will require a cost-benefit analysis, performed by incorporating the variability into our overall model of the dynamics of the feeding CPG operating in various environments (see [9]), and ultimately experimentally. In the meantime, however, we have carried out such an analysis for the single ARC muscle, by running the variable firing patterns of the motor neurons B15 and B16 recorded during the 2½-hour meal (Figs. 1 and 2), and various regular patterns in comparison, through a model of the muscle's neuromuscular transform [2, 3] and the resulting variable contractions or movements then through a simple functional task, with one adjustable parameter that was either fixed or varied randomly to simulate a fixed or a varying feeding environment (for details see [1]). Performance in the task, averaged over the 749 cycles of the meal, is plotted in Fig. 4. Clearly, given a fixed but unknown task, the variable movements, which sometimes succeed, are better overall than regular movements that systematically miss the requirements of the task and never succeed. What if the fixed task could be known? Then regular movements that are specifically tailored to its particular requirements (Fig. 4, bar 3) perform better, but not very much better, than the variable movements (bar 1) that pay no attention to the requirements of the task at all. The regular movements have to be very specifically tailored: constructing a regular firing pattern simply by repeating 749 times the median values of the variable parameters in Fig. 2 (bar 2), or even 749 times the pattern of the single best-performing variable cycle, gave no performance at all. In any case, the proposal is that the task, too, varies. Then the performance of the regular movements degrades considerably (bar 6), whereas that of the variable movements is much less affected (bar 4). In other words, the regular movements, fine-tuned to the requirements of one specific task, break down when faced with a changing mix of tasks, whereas the variable movements continue to give essentially the same, robust performance no matter what may be the composition of the mix. There is, in that case, no need for the animal to have any precise information about the task requirements in any particular cycle.

Fig. 4.

With a variable task, variable movements are optimal. Performance (mean ± SE over 749 cycles) when the B15 and B16 firing patterns from Fig. 2, top, and comparable regular patterns (see text), were run through a model of the B15/B16-ARC neuromuscular transform and functional task [2, 3]. The “best regular” patterns were found by search of the entire parameter space of regular firing patterns. Modified from [1].

We conclude that the variability in the Aplysia feeding system may indeed implement an optimal behavioral strategy for the animal. If so, the variability is not, as it is usually taken to be, merely “noise,” a problem in the task of performing “reliable computation with unreliable components” [10], to be eliminated by averaging and redundancy to achieve a predictable final behavior, but rather, in this system, emerges in and plays a positive, active role in the behavior.