Dynamical basis of intentions and expectations in a simple neuronal network

Abstract

Selection of behavioral responses to external stimuli is strongly influenced by internal states, such as intentions and expectations. These internal states are often attributed to higher-order brain functions. Yet here we show that even in the simple feeding network of Aplysia, external stimuli do not directly specify which motor output is expressed; instead, the motor output is specified by the state of the network at the moment of stimulation. The history-dependence of this network state manifests itself in the same way as do intentions and expectations in the behavior of higher animals. Remarkably, we find that activity-dependent plasticity of a synapse within the network itself, rather than some higher-order network, mediates one important aspect of the change in the network state. Through this mechanism, changes in the network state become an automatic consequence of the generation of behavior. Altogether, our findings suggest that intentions and expectations may emerge within behavior-generating networks themselves from the plasticity of the very processes that generate the behavior.

Animal behavior is not merely a passive response to external stimuli; rather, it expresses also the internal state of the animal. This internal state is presumably somehow embodied in the state of the nervous system. Here we study the manifestations and the neurophysiological basis of the internal state in the experimentally advantageous feeding network of the mollusk Aplysia.

State-dependence of network function is not a new concept. In the neurophysiological literature, state-dependence is typically discussed as the ability of contextual cues to modify the response to a stimulus. This type of state-dependence has been demonstrated, for example, for locomotion by using stimulation of the mesencephalic locomotor region to elicit walking in the decerebrate cat. The speed of locomotion is determined by the speed of the treadmill on which the cat is placed (1). In another compelling example, stimulation of a command-like neuron in a leech immersed in water elicits swimming, whereas stimulation of the same neuron when the leech is placed on solid substrate elicits crawling (2). Similarly, stimulation of a command-like neuron in a cricket suspended in air elicits avoidance responses but it fails to do so when the cricket is placed on the ground (3). State-dependence of this type is also seen with neuromodulation. For instance, the ability of sensory stimulation to elicit stridulation in the grasshopper is critically dependent on the presence of a muscarinic agonist (4). The setting of network state by application of neuromodulators is a common phenomenon that has been well characterized in simple neuronal networks, such as the stomatogastric system of crustaceans (5–9).

This type of state-dependence fails, however, to account for a fundamental feature of the state-dependence that is observed in animal behavior and human psychology. In the type of state-dependence just discussed, the behavior is still unambiguously specified by the external influences, the sum total of the stimuli and contextual cues, at the moment of behavior. In contrast, what is particularly intriguing in many human and animal behaviors is that they can change even though all external influences remain exactly the same (10, 11). The change in behavior is not driven by any change in the environment but by the changing internal state. Fundamental to the operation of the internal state and to our recognition of its manifestations in behavior are thus likely to be its intrinsic dynamics.

The manifestations of such an internal state are illustrated in the following example adapted from refs. 12 and 13. A red dot and a blue dot are displayed on a diagonal, and the human subject is cued to answer, within a limited time, one of two questions: Is the blue dot above or below the red dot? or, Is the blue dot to the left or to the right of the red dot? These questions are asked in a pseudo-random sequence. It is typically found that if in successive trials the same question is asked repeatedly, performance improves. When the other question is then asked, the subject displays a tendency to disregard the change of question and to act as if the first question were still being asked, a phenomenon referred to as task-set inertia (13, 14). Task-set inertia indicates that changes in the internal state of the subject induced by the previous exposure to the task persist to influence subsequent responses. The improvement in performance upon repetition of the same question and the drop in performance after change of the question (switch cost) are two forms of history-dependence of behavior that have been interpreted to mean that the subject has developed an expectation that the same question will be repeated, or, equivalently, an intention to answer the same question.

Because of the cognitive connotations of task-set inertia and its association with traditionally conscious states, such as intentions and expectations, these phenomena have not been previously investigated in simple experimental systems. Here, we report that the behavior of the feeding network of Aplysia has history-dependent dynamics that give rise to task-set inertia, revealing the operation of the internal state of the network. We take advantage of the accessibility of the Aplysia preparation to analyze how these dynamical properties arise out of the neurophysiological processes within the feeding circuitry and discuss the implications of our findings for thinking about cognitive and conscious phenomena, such as intentions and expectations.

Materials and Methods

All experiments were performed on Aplysia californica obtained from Marinus (Long Beach, CA). Aplysia were maintained in circulating artificial sea water (ASW) made from Instant Ocean (Aquarium Systems, Mentor, OH) at 14–15°C. Animals weighing 150–250 g were anesthetized by injection of isotonic MgCl₂ (337 mM). Buccal and cerebral ganglia were dissected out of the animal and desheathed in a dissection chamber lined with Sylgard. The ganglia were then transferred to the recording chamber containing ≈1.5 ml of ASW (460 mM NaCl/10 mM KCl/55 mM MgCl₂/11 mM CaCl₂/10 mM Hepes buffer, pH 7.6). During all experiments, the ganglia were maintained at 14–17°C and continuously perfused with ASW at the rate of ≈0.3 ml/min.

Intracellular recordings were performed with either Axoclamp 2B (Axon Instruments, Union City, CA) or Getting 5A amplifiers. All neurons were identified as described in refs. 15–17. Extracellular recordings were performed by suctioning buccal nerves into electrodes constructed from polyethylene tubing. Signals were amplified with AC amplifier model 1700 (AM Systems, Carlsborg, WA).

All recordings were acquired at 5 KHz with the Digidata 1322A data acquisition system (Axon Instruments) and recorded on a personal computer (Dell, Austin, TX). Digitized recordings were plotted with sigmaplot 8 (SPSS, Chicago). Statistics were performed in sigmaplot 8 or excel (Microsoft). For details of the calculations of synaptic efficacy, see Statistical Analysis of B20 Contribution to B8 Firing in Feeding Motor Programs, which is published as supporting information on the PNAS web site.

Results

The feeding central pattern generator (CPG) of Aplysia generates two antagonistic behaviors, ingestion and egestion, that can be elicited by different stimuli. In vitro, the CPG produces feeding motor programs that are very similar to those recorded during ingestive and egestive feeding behaviors in intact animals (15, 18, 19). In both ingestive and egestive behaviors, the animal first protracts and then retracts the radula, its food-grasping organ. During ingestive behaviors the radula closes in the retraction phase to pull food in, whereas during egestive behaviors it closes in the protraction phase to push material out (20). In feeding motor programs there is a corresponding difference in the phasing of activity of the radula-closer motoneuron B8 such that ingestive and egestive programs form two distinct clusters in the plane spanned by the firing frequencies of B8 in the protraction and retraction phases (15–17) (Fig. 1A; representative examples of ingestive and egestive programs are shown by the colored traces in their respective clusters). The programs outside the cluster boundaries are intermediate. In behaving animals, these programs may mediate poorly articulated responses that do not produce movement of food in or out of the mouth (18, 19).

Fig. 1.

Stimulation of the same input elicits two types of mutually antagonistic motor programs. (A) Both ingestive and egestive programs consist of two phases of activity: radula protraction (Prot) coincident with activity in the I2 nerve (39), and radula retraction (Ret) coincident with large-unit activity in buccal nerve 2 (Bn2) (18). In Figs. 1, 2, and 4, activity associated with protraction is shown in red and activity associated with retraction is shown in blue. Gray ovals are cluster boundaries (15) of the ingestive and egestive programs in the plane spanned by the firing frequencies of radula-closer motoneuron B8 (19) in protraction (abscissa) and retraction (ordinate). Representative recordings of ingestive and egestive programs are shown in the clusters. Both programs were elicited by stimulating the same identified CBI-2 neuron at 9 Hz. (B) CBI-2 was stimulated at 9 Hz for the duration of the protraction phase (19.4 ± 0.6 s). A 30-s rest was then allowed before the beginning of the next CBI-2 stimulation. Eight consecutive programs (orange circle, 1–8) were elicited in this way, then a single program was elicited 2, 4, 7, and 12 min later (black circle) (n = 7). EN was stimulated with 3-ms current pulses at 2 Hz for 2 min, with stimulation amplitude adjusted so that about five programs (orange triangle, 1–5) were elicited. Then EN was stimulated 1, 3, and 6 min later to elicit single programs (black triangle) (n = 5). (C) Same data as in B plotted against time. Repeated CBI-2 stimulation (Upper, buildup) elicited progressively increasing B8 firing in retraction but not in protraction; repeated EN stimulation (Lower, buildup) elicited the converse changes in the pattern of B8 firing. Throughout the figures, all group data are shown as mean ± SE. Statistical significance was tested with the two-tailed t test: ***, P < 0.001; **, P < 0.01; *, P < 0.05; n.s., P > 0.05.

The quiescent feeding CPG can be activated to produce motor programs and functional behaviors by stimulation of either one of two behaviorally relevant input pathways: the cerebral-buccal interneuron (CBI)-2, which is activated when lips are stimulated with seaweed, and the esophageal nerve (EN), which is thought to convey sensory information from the esophagus (21, 22). Remarkably, stimulation of CBI-2 can elicit both the ingestive and the egestive programs. In fact, both programs in Fig. 1 A were elicited by stimulation of the same CBI-2 neuron in the same preparation. The CBI-2 stimulus was exactly the same, but the response was different. This difference in responses suggests that the type of motor program elicited by CBI-2 depends on the state of the network at the moment of stimulation.

To determine whether this state of the network is influenced by previous history, we characterized the motor programs elicited by repeatedly stimulating the same input and plotted them in the plane introduced in Fig. 1 A. Regardless of whether we stimulated CBI-2 or EN, the initial programs were neither ingestive nor egestive but intermediate (Fig. 1B: initial programs are labeled 1). With repeated stimulation, however, subsequent programs progressively migrated into either the ingestive or the egestive cluster, depending on whether CBI-2 or EN was stimulated [Fig. 1B, CBI-2 (orange circle) and EN (orange triangle), and Fig. 1C, buildup]. When the interstimulus interval was increased, the programs gradually returned to being intermediate over the course of several minutes [Fig. 1B, CBI-2 (black circle) and EN (black triangle) and Fig. 1C, recovery]. Thus, the type of motor program the CPG produces strongly depends on the previous history of the system. Notice the similarity of the progressive increase in ingestiveness or egestiveness to the improvement of performance with repetition of the same question in the two-dot task.

As in the two-dot task, we then characterized the behavior of the CPG upon change in the input. The results are plotted in Fig. 2B in the same plane as was used in Fig. 1; Fig. 2 A shows representative traces. By repeated stimulation of CBI-2, we first established fully ingestive programs to serve as a basis for comparison (CBI-2 control in Fig. 2 A, black circle in Fig. 2B). Then we stimulated EN until the programs became fully egestive (Last EN in Fig. 2 A, black triangle in Fig. 2B). When we then switched the stimulation back to CBI-2, the programs did not switch to ingestive, but remained egestive. Indeed, immediately after the EN stimulation, the first CBI-2-elicited program (CBI-2, 0 min; orange circle in Fig. 2B) was virtually indistinguishable from the last EN-elicited program in all measured parameters (Fig. 2 A and B and Fig. 5, which is published as supporting information on the PNAS web site). Only with repeated stimulation of CBI-2 did the programs return again to the ingestive cluster (Fig. 2B, CBI-2 at 1, 2, 3, 4, and 5 min (orange circle); the time course is shown in Inset). Thus, upon changing from EN to CBI-2 stimulation, the network did not respond to the change and continued to generate egestive motor programs as if, with the repeated EN stimulation, it had developed an expectation that it would continue to receive esophageal input, or, equivalently, an intention to generate an egestive program.

Fig. 2.

History-dependent network state determines the nature of the motor program. CBI-2 was stimulated for the duration of the protraction phase (19.5 ± 1.75 s) as in Fig. 1B to elicit eight consecutive programs, then EN was stimulated as in Fig. 1B for 5 min, then CBI-2 was stimulated again every minute. (A) Representative recordings. Shown are data for a control ingestive CBI-2-elicited program (CBI-2 control), last egestive EN-elicited program (last EN), egestive program elicited by CBI-2 immediately after EN stimulation (CBI-2, 0 min); ingestive program elicited by CBI-2 5 min after EN stimulation (CBI-2, 5 min). (B) Group data (n = 13). Control CBI-2-elicited programs are in the ingestive cluster (black circle); the last EN-elicited program is in the egestive cluster (black triangle). Immediately after the EN stimulation, the first CBI-2-elicited program remains in the egestive cluster (orange circle; CBI-2, 0 min). With repeated CBI-2 stimulation, the programs gradually return to the ingestive cluster (orange circle; CBI-2, 1, 2, 3, 4, and 5 min). (Inset) Time course of the changes in the firing of B8 in the protraction and retraction phases. After EN stimulation, B8 firing increases in protraction and decreases in retraction.

The experiments presented so far demonstrate that the nature of the motor program is dictated by the internal state of the network rather than the nature of the input. The input elicits a slow evolution of this network state toward a particular steady state. In the following set of experiments we sought to identify neurophysiological processes that may give rise to these dynamics.

Because changes in the network state are characterized by changes in the pattern of firing of B8, we focused on neurons that provide strong inputs to B8. One such interneuron, B20, is critical for the expression of egestive motor programs because it provides strong excitatory input to B8 during the protraction phase (17, 23). We found that the evolution of the network toward the egestive steady state elicited by EN stimulation was accompanied by enhancement of the amplitude of B20 to B8 postsynaptic potentials (PSPs) (Fig. 3A). During the EN stimulation itself, the time course of this enhancement was obscured by the EN-elicited programs, but the subsequent slow recovery of the PSPs paralleled the slow recovery from the egestive network state (Fig. 3A). Importantly, repeated CBI-2 stimulation did not enhance the PSPs (Fig. 3B). Thus, the enhancement of the B20 to B8 PSPs parallels the evolution of the network toward the egestive, but not the ingestive, steady state.

Fig. 3.

Activity-dependent synaptic plasticity selectively enhances B20 to B8 PSPs in EN-elicited programs. Each bar shows the amplitude of PSPs elicited by stimulating B20 with 25-ms current pulses to fire at 2 Hz, a frequency that itself did not enhance the PSPs pooled into 30-s bins. (A) EN was stimulated as in Fig. 2 to elicit programs for 5 min. PSPs were enhanced (n = 5). Representative recordings of PSPs are shown below. (B) CBI-2 was stimulated as in Fig. 1B to elicit eight programs. PSPs were not enhanced (n = 5). Representative recordings of PSPs are shown below. (C) B20 was stimulated to fire at 10 Hz for 30 s. PSPs were enhanced (n = 5). (D) EN was stimulated as in A, but B20 firing was suppressed by injecting a constant hyperpolarizing current. PSPs were not enhanced (n = 4).

The selective expression of synaptic plasticity in EN-elicited programs raises a question: How does the synapse “know” whether CBI-2 or EN was stimulated? An attractive hypothesis is that the information about the stimulus is conveyed through the classical homosynaptic mechanism of activity-dependent synaptic plasticity (24): The differences in synaptic plasticity may be induced by differences in the firing of B20 itself. Indeed, B20 fired at a significantly higher frequency in EN-elicited than in CBI-2-elicited programs (7.24 ± 0.39 Hz EN vs. 3.76 ± 0.37 Hz CBI-2, mean ± SE, n = 139; P < 0.001, t test). Even when we compared just the initial programs that were intermediate with both CBI-2 and EN stimulation, B20 fired significantly more in the latter case (P < 0.001). Furthermore, B20 firing alone, in the absence of programs, was sufficient to enhance the PSPs (Fig. 3C). Finally, the firing of B20 was necessary for the PSP enhancement during programs: When B20 firing was suppressed by injection of constant hyperpolarizing current, EN stimulation, although it still elicited programs, failed to enhance the PSPs (Fig. 3D).

Does the enhancement of the B20 to B8 synapse actually contribute to the changes in the network state rather than merely reflect them? We found that the suppression of B20 firing during EN stimulation that prevented the enhancement of B20 to B8 PSPs (Fig. 3D) also prevented the programs from entering the egestive cluster (Fig. 4C Top). Thus, B20 firing was necessary for the evolution of the network toward the egestive steady state.

Fig. 4.

Enhancement of the B20 to B8 synapse contributes to the history-dependent network state. (A) Representative record of the firing of B20 and B8 during the protraction phase of an EN-elicited program. White rectangles are 70-ms windows after each B20 spike; gray rectangles are intervening segments. Here, four of six B8 spikes are found in the 70-ms windows after the B20 spikes. (B) Crosscorrelation histogram between B20 and B8 spikes. Histogram was constructed from 27,843 B20 and 23,007 B8 spikes pooled from the protraction phases of 435 programs in 16 preparations, including all those in C and D. Gray area is the B20-independent firing of B8; black area is the contribution of B20 to B8 firing (see Statistical Analysis of B20 Contribution to B8 Firing in Feeding Motor Programs). (C) Repeated EN stimulation (as in Fig. 3A and including those experiments) resulted in progressive increase in total B8 firing in protraction (Top), contribution of B20 (Middle), and functional synaptic efficacy (Bottom). Suppression of B20 activity (as in Fig. 3D and including those experiments) prevented the change in B8 firing (Top). Altogether, 130 programs from 13 preparations were analyzed; n for each data point varied from 14 to 43. Programs 5–8 and 9–12 were pooled to maintain a sufficient n as the number of preparations in which this many programs were elicited during the 5 min of EN stimulation decreased. (D) Repeated CBI-2 stimulation (as in Figs. 2 and 3B and including those experiments) did not change the total B8 firing in protraction (Top Left), contribution of B20 (Middle Left), or functional synaptic efficacy (Bottom Left). After EN stimulation, all three parameters were increased (Right) and gradually recovered with resumed CBI-2 stimulation. Altogether, 184 programs from nine preparations were analyzed; n for each point varied from 6 to 12.

In a highly interconnected network, such as the feeding CPG, however, these results with the suppression of B20 firing do not necessarily implicate the B20 to B8 synapse directly. B20 may potentially excite B8 by means of a number of polysynaptic pathways, all of which would be interrupted by the suppression of B20 firing. To implicate the synapse directly, we adopted a statistical technique based on crosscorrelation analysis.

We observed that B8 spikes during the protraction phase are not randomly distributed with respect to B20 spikes but tend to follow them preferentially (Fig. 4A). This tendency is reflected in a pronounced peak in the crosscorrelation histogram (Fig. 4B, black area). If B20 spikes did not in any way influence B8 spikes, the crosscorrelation histogram would be flat. To compute the contribution of B20 to B8 firing, we estimated for each program a quantity equivalent to the area of the crosscorrelation peak (see Statistical Analysis of B20 Contribution to B8 Firing in Feeding Motor Programs for details of the method). We then divided the contribution of B20 by the number of spikes fired by B20, arriving at a parameter we refer to as functional synaptic efficacy. To a first approximation, the functional synaptic efficacy is the above-background probability that a spike in B20 is followed by a spike in B8. By using this method, we reanalyzed a dataset of relevant programs from the preceding experiments.

We confirmed that B20 was involved in driving the network toward the egestive steady state, because an increasing contribution of B20 largely explained the increase in total B8 firing in the protraction phase of successive EN-elicited programs (Fig. 4C Top and Middle). Furthermore, the increasing B20 contribution was itself largely explained by an increasing functional synaptic efficacy (Fig. 4C Bottom). In contrast, the B20 contribution and functional synaptic efficacy remained constant during evolution of the network toward the ingestive steady state with repeated CBI-2 stimulation (Fig. 4D Left).

The most dramatic demonstration of the history-dependence of the network state is the experiment shown in Fig. 2. Reanalyzing the programs from this experiment, we found that increased functional synaptic efficacy established during EN stimulation largely explained the fact that subsequent CBI-2-elicited programs remained in the egestive cluster (Fig. 4D Right; see Fig. 6, which is published as supporting information on the PNAS web site). This directly implicates the change in the strength of the B20 to B8 synapse as a mediator of the change in the network state.

Discussion

In this study we have demonstrated that the motor output of the feeding network of Aplysia is specified not by the input stimulus but by the internal state of the network at the moment of stimulation. The dynamical properties of this network state are such that the resulting patterns of behavior of the feeding network are formally similar to the behavior of subjects in the two-dot task. Namely, the slow dynamics of the network state give rise to both an improvement in performance upon repetition of the same stimulus and a task-set inertia after a switch in the stimulus. In the two-dot task and other similar paradigms, these phenomena have been interpreted to mean that as a result of repeated presentation of the same stimulus the subject develops an expectation that the same stimulus will be presented again or, alternatively, an intention to respond to the same stimulus (12, 13). It is precisely because we recognized the same formal properties in the dynamics of the state of the Aplysia feeding network that we have used the terms “intention” and “expectation” in this work.

Talking about the existence of intentions and expectations in a simple network may appear surprising, because intentions and expectations are usually discussed from a subjective (introspective) perspective in which these terms imply conscious awareness. Here we do not provide any evidence indicating whether the feeding network of Aplysia is conscious. Rather, we define intentions and expectations objectively (operationally). As pointed out by Hebb (10), this operational view demystifies psychological phenomena, in our case intentions and expectations, and allows them to be connected directly to the underlying neurophysiological mechanisms.

From the introspective point of view, intentions and expectations appear to be complex, “higher-order” phenomena. It is consequently often assumed that intentions and expectations must arise in some “higher-order” cortical networks and then are passed down to the behavior-generating networks (25–27). In this view, the processes that give rise to intentions and expectations are somehow different from those that generate behavior. However, recent indirect evidence suggests that intentions and expectations may arise in behavior-generating networks themselves even in primates (28–30). In that case, interestingly, the intentions and expectations inferred from behavioral observations are not always identical to the intentions and expectations that are consciously accessible. This phenomenon was illustrated in an elaboration of the two-dot task. After being asked each successive question, the subjects were allowed to prepare themselves consciously to answer that question. This manipulation, however, did not eliminate the history-dependence in the actual performance of the task, such as the switch cost (31). Thus, in addition to the conscious intentions and expectations reported by the subjects, the subjects' behaviors reflected another set of intentions and expectations that arose automatically (nonconsciously) based on previous experience. In this study we have demonstrated how such intentions and expectations arise automatically in the feeding network of Aplysia. In this simple system, we have been able to show how a ubiquitous neurophysiological mechanism, activity-dependent synaptic plasticity, can automatically implement both the dynamical evolution and the expression of the internal state of the network.

The slow dynamics of the feeding network give rise to the expectation that the stimulus that was presented before will be presented again. Although this may appear to be rather a simple-minded expectation, it is in fact an expectation expressed in many human behaviors: in a variety of cognitive tasks (12, 13, 31), the ability to recognize faces (32), and deciding where to look next (33). This expectation is expressed even when it is consciously known to be fallacious, as demonstrated in a recent study by Huettel et al. (34). Subjects were explicitly told that they would be presented with a random sequence of stimuli. However, over an extended period even random sequences form seemingly nonrandom patterns, such as runs of the same stimulus. When this happened, the subjects responded faster to each subsequent stimulus in the run and much slower to stimuli deviating from the apparent pattern. This indicates that, despite the subjects' conscious awareness of the contrary, their behavior expressed an expectation that a previously presented stimulus would be presented again. Similar expectations are expressed in the behavior of gamblers at the roulette table in the well known gambler's fallacy.

Why is this expectation expressed so strongly in so many behaviors? Perhaps the answer is that in the real world most events are not random. Rather, naturally occurring stimuli often cluster together. Thus, having recently encountered a certain stimulus, the animal can reasonably expect the same stimulus to recur. Presumably, nervous systems have evolved slow dynamics to respond to such patterns in the environment. This neuroethological perspective is formalized in a theoretical approach (35–38) that proposes that phenomena, such as intentions and expectations, traditionally viewed as cognitive, arise out of a continuous dynamic interaction between the animal and the environment and serve to stabilize appropriate coordinated patterns of behavior. In this dynamical formulation, the trajectory that the internal state of the nervous system traces as the animal continuously engages its environment gives rise to the history-dependence of behavior that we interpret as intentions and expectations. From this vantage point, intentions and expectations in human behavior can be seen to share essential similarities in their dynamical properties to those of simple organisms such as Aplysia.