Contingent-Dependent Enhancement of Rhythmic Motor Patterns: AnIn Vitro Analog of Operant Conditioning

Abstract

Operant conditioning is characterized by the contingent reinforcement of a designated behavior. Previously, feeding behavior inAplysia has been demonstrated to be modified by operant conditioning, and a neural pathway (esophageal nerve; E n.) that mediates some aspects of reinforcement has been identified. As a first step toward a cellular analysis of operant conditioning, we developed an in vitro buccal ganglia preparation that expressed the essential features of operant conditioning. Motor patterns that represented at least two different aspects of fictive feeding (i.e., ingestion-like and rejection-like motor patterns) were elicited by tonic stimulation of a peripheral buccal nerve (n.2,3). Three groups of preparations were examined. In a contingent-reinforcement group, stimulation of E n. was contingent on the expression of a specific type of motor pattern (i.e., either ingestion-like or rejection-like). In a yoke-control group, stimulation of E n. was not contingent on any specific pattern. In a control group, E n. was not stimulated. The frequency of the reinforced pattern increased significantly only in the contingent-reinforcement group. No changes were observed in nonreinforced patterns or in the motor patterns of the control and yoke-control groups. Contingent reinforcement of the ingestion-like pattern was associated with an enhancement of activity in motor neuron B8, and this enhancement was specific to the reinforced pattern. These results suggest that the isolated buccal ganglia expressed an essential feature of operant conditioning (i.e., contingent reinforcement modified a designated operant) and that this analog of operant conditioning is accessible to cellular analysis.

Operant conditioning, which was introduced by Thorndike (1911), is an example of associative learning in which an association is established between a specific behavior (the operant) and a stimulus (the reinforcement). A key feature of operant conditioning is the contingency of the reinforcement (i.e., the correlation between the expression of a designated operant behavior and the delivery of a reinforcement; Skinner, 1938; Konorski, 1948). As a result of this contingency the frequency of the reinforced behavior is modified. This phenomenon, known as the “law of effect” (Thorndike, 1933), provided evidence that the nervous system has mechanisms by which a particular motor output can be selected from among many different behaviors that may be expressed.

Rhythmic motor acts such as locomotion, feeding, respiration, and heart rate can be modified by operant conditioning (Skinner, 1938; Miller, 1969; Cook and Carew, 1986; Susswein et al., 1986; Jaeger et al., 1987;Lukowiak et al., 1996). It is believed generally that rhythmic motor acts are mediated by groups of neurons referred to as central pattern generators (CPGs; Delcomyn, 1980; Selverston and Moulins, 1985). CPGs are multifunctional networks that can mediate more than one behavior (Willows and Hoyle, 1969; Kupfermann, 1974a; McClellan, 1982; Simmers and Bush, 1983; Mortin et al., 1985; Heinzel, 1988; Oku et al., 1994;Green and Soffe, 1996) (see Getting, 1989; Harris-Warrick and Marder, 1991). Although significant progress has been made in analyzing the cellular mechanisms by which these networks switch between different motor outputs (Getting and Dekin, 1985; Hooper and Moulins, 1989;Dickinson et al., 1990; Meyrand et al., 1991, 1994) (see Dickinson and Moulins, 1992; Dickinson, 1995), the cellular mechanisms by which operant conditioning modifies such multifunctional circuits and thereby modifies a specified behavior remain unknown.

To address this issue, we used the isolated buccal ganglia ofAplysia and developed an in vitro analog of operant conditioning. These ganglia contain the CPG that mediates several different consummatory feeding behaviors (Kupfermann, 1974b;Morton and Chiel, 1993a; Baxter et al., 1995). These behaviors, in turn, can be modified by operant conditioning (Schwarz and Susswein, 1986; Susswein et al., 1986). Successful ingestion of food as well as failed attempts to consume food can function as reinforcement and can increase or decrease aspects of ingestion, respectively. In the present study tonic stimulation of the ventral branch of buccal nerve 2 (n.2,3) was used to elicit motor programs. At least two different motor programs were elicited, and these two motor programs were similar to neural activity previously observed in vivo during feeding behaviors (Morton and Chiel, 1993a). Thus, these two motor programs were used as analogs of operant behaviors. As suggested by the previous studies of Schwarz and Susswein (1986), stimulation of the anterior branch of the esophageal nerve (E n.2) was used as an analog of reinforcement. The results indicated that if stimulation of E n.2 was contingent on the expression of a designated pattern, then the expression of this reinforced pattern was selectively enhanced. No changes in nonreinforced patterned output were observed. This enhancement persisted for up to 1 hr after a 10 min training period. These results suggest that the isolated buccal ganglia expressed an essential feature of operant conditioning (i.e., contingent reinforcement modified a designated operant).

A preliminary report of these results has appeared in abstract form (Nargeot et al., 1996).

MATERIALS AND METHODS

Aplysia californica (150–250 gm) were obtained from Marinus (Westchester, CA), Marine Specimens Unlimited (Pacific Palisades, CA), and Alacrity Marine Biological (Redondo Beach, CA) and maintained in filtered artificial seawater (ASW) (Instant Ocean; Aquarium Systems, Mentor, OH) at 15°C.

Consummatory feeding behavior can be influenced by motivational states such as arousal or satiety (Kupfermann, 1974a). To help ensure that all animals were in a similar motivational state, we caged the animals and deprived them of food for 2 d before the experiment; each animal was fed with a piece of seaweed of ∼30 cm² for 45 min immediately before the experiment. After being fed, the animals were anesthetized by injecting ∼60 ml of isotonic MgCl₂into the hemolymph. The buccal mass was removed quickly and placed in a chamber containing ASW composed of (in mm): NaCl 450, KCl 10, MgCl₂(6 H₂O) 30, MgSO₄ 20, CaCl₂(2 H₂O) 10, and Trizma 10. The pH was adjusted to 7.4 with HCl. Buccal ganglia were isolated and pinned out in a SYLGARD-coated Petri dish containing ASW. The preparations were not perfused (i.e., the bathing solution was static). The ganglia were maintained at 15°C by means of a Peltier cooling device during the experiment.

Electrophysiology. Pulses for extracellular nerve stimulation were generated by a digital pulse generator (WPI 1800, Sarasota, FL) and applied, via a stimulus isolator, to bipolar wire electrodes that were placed against appropriate nerves and isolated from the bath with Vaseline. Stimuli composed of brief (0.5 msec) pulses were delivered to the anterior branch of the esophageal nerve (E n.2) (see Fig. 1) and the ventral branch of buccal nerve (n.2,3) (see Fig. 1). In nondesheathed preparations, stimulation of n.2,3 was delivered with a frequency of 2 Hz and an intensity of 7 V. In preparations in which intracellular recordings were performed, a single ganglion was desheathed on the caudal surface (see below). In these desheathed preparations, stimulation of n.2,3 was less effective in inducing neural activity. Therefore, n.2,3 was stimulated with a frequency of 4 Hz and an intensity of 8.5 V. The n.2,3 that was selected for stimulation was always contralateral to the nerves and cells from which the recordings were made. In those preparations that were not desheathed, the E n.2 that was stimulated was ipsilateral to the nerves from which recordings were made. In desheathed preparations, the E n.2 that was stimulated was contralateral to the desheathed ganglion (i.e., contralateral to the cells and nerves recorded). Stimulation of E n.2 was delivered at 10 Hz for 6 sec with an intensity of 7 V in nondesheathed preparations and 8 V in desheathed preparations. Once electrodes were in place, brief stimulation was used to test the efficacy of the stimuli to elicit neural activity (Nargeot et al., 1995). Experiments began after a 40 min rest period after the initial test stimuli. Spontaneous activity of the buccal ganglia was recorded during the last 10 min of this rest period.

Fig. 1.

The buccal ganglia and its peripheral nerves. A, Schematic representation of the buccal mass and the location of the buccal ganglia and its peripheral nerves. B, Schematic representation of in vitro buccal ganglia preparation showing the position of the recording electrodes (white triangles) on I2 n., n.2,1, and R n.1 and the stimulating electrodes (black triangles) on E n.2 and n.2,3. This schematic illustrates the placement of electrodes that was used in nondesheathed preparations (i.e., the E n.2 that was stimulated was ipsilateral to the nerves from which recordings were made). In desheathed preparations the E n.2 that was contralateral to the ganglion from which recordings were made was stimulated (data not shown). C-B conn., Cerebrobuccal connectives; E n., esophageal nerve; I2 n., nerve to intrinsic buccal muscle 2; n., buccal nerve; R n., radular nerve.

Extracellular recordings were made by using monopolar silver wire electrodes placed against appropriate nerves and isolated from the bath by Vaseline. Extracellular signals were filtered with 10 Hz high-pass and 1 kHz low-pass filters and were amplified by a differential AC amplifier (A-M Systems 1780, Everett, WA).

Intracellular recordings were made from the caudal side of the desheathed buccal ganglion with glass microelectrodes filled with 2m potassium acetate (resistance 5–10 MΩ) and connected to an Axoclamp-2A electrometer. Desheathing was performed in the presence of high divalent cation ASW containing three times (i.e., 30 mm) the normal concentration of CaCl₂ and three times (i.e., 90 mm) the normal concentration of MgCl₂. To maintain an appropriate osmolarity, we decreased the NaCl concentration to 330 mm. High divalent cation ASW was used to decrease neural activity during desheathing (Byrne et al., 1978). The buccal ganglia were washed with ASW immediately after desheathing. The frequency of spontaneous bursting motor activity expressed in the nondesheathed isolated buccal ganglia (0.003 ± 0.0005 Hz; n = 30) were comparable to the levels of spontaneous feeding activity observed in in vivo or in semi-intact preparations (Kupfermann, 1974a; Kabotyanski et al., 1995). In some desheathed preparations, however, this frequency was much higher. Preparations that expressed spontaneous bursting activity at a rate higher than 0.01 Hz were not used in the present study.

Classification of the different motor patterns. Consummatory feeding behaviors of Aplysia are composed of ingestion (i.e., biting and swallowing) and rejection behaviors. In general terms, these behaviors have two phases (Kupfermann, 1974a). The first phase is characterized by protraction of the odontophore and its two radula halves (toothed grasping surfaces). This protraction phase is followed by a second phase: the retraction of the odontophore and its two radula halves. Ingestion and rejection can be distinguished by examining the time at which the radula are open or closed relative to the protraction and retraction phases. During ingestion the two halves of the radula are open during the protraction phase and closed during the retraction phase and thereby draw food into the buccal cavity. Conversely, during rejection the radula are closed during the protraction phase and open during the retraction phase and thereby expel food from the buccal cavity.

With the use of in vivo recordings from buccal nerves, it has been possible to identify neural correlates of consummatory feeding behaviors (Morton and Chiel, 1993a). These authors identified three patterns, which they termed pattern I, pattern II, and intermediate pattern. Pattern I corresponded to a neural correlate of ingestion, and pattern II corresponded to a neural correlate of rejection. Intermediate patterns also were recorded during the consummatory feeding behavior, but their behavioral signification remains unclear. These three types of patterns were distinguished by the phase relationship of the neural activities in the buccal nerves, which, in turn, were associated with different movements of the radula (i.e., protraction, retraction, and closure). In pattern I (i.e., ingestion-like pattern) in vivo recordings revealed that neural activity associated with the closure of the radula primarily overlapped with neural activity associated with retraction of the radula. In particular, ingestion behaviors were observed when at least 50% of closure-related neural activity overlapped with retraction-related neural activity (Morton and Chiel, 1993a). In pattern II (i.e., rejection-like pattern) the neural activity associated with closure of the radula preceded neural activity associated with radula retraction (i.e., closure occurred during the protraction phase). In intermediate patterns the neural activity associated with closure of the radula partially overlapped with neural activity associated with radula retraction but primarily occurred during the protraction phase.

In the present study the protraction phase was monitored in the isolated buccal ganglia by activity in the nerve to the intrinsic buccal muscle 2 (I2 n.) (Hurwitz et al., 1996), the retraction phase was monitored by activity in n.2,1, and closure activity was monitored by large-amplitude activity in radular nerve 1 (R n.1) (Morton and Chiel, 1993a) (see Fig. 1). Action potentials expressed with a frequency lower than 0.25 Hz were not considered as part of a burst of action potentials. Transition between the protraction and the retraction phases was monitored by the termination of activity in I2 n. (Hurwitz and Susswein, 1996). The motor patterns recorded from isolated buccal ganglia were classified into the three categories, pattern I, pattern II, and intermediate patterns, in accordance with criteria similar to those developed from in vivo recordings (Morton and Chiel, 1993a) (see below). In the caudally desheathed preparations we occasionally observed bursts of activity that occurred in only one or two of the nerves. Because these incomplete patterns have not been described in vivo, they were not included in the present study, and preparations were discarded if >33% of the observed patterns were incomplete. The desheathed preparations that expressed >33% of incomplete patterns were distributed across the different experimental paradigm (i.e., contingent reinforcement, yoke-control, and control groups).

Cell identification. Motor neurons were identified by their axonal projections in peripheral nerves, by the phasic relationship of their activity to the patterned activity recorded in peripheral buccal nerves, and by their relative position in a buccal ganglion as described by Church and Lloyd (1991, 1994) and Church et al. (1991). Axonal projections were tested in I2 n., ipsilateral n.1, n.2, n.3, R n., and contralateral n.2, R n. by conventional electrophysiological methods (e.g., Fig. 3). In particular, we tested for a one-for-one relationship with a constant delay between intracellular and extracellular action potentials and the ability to elicit antidromic action potentials in the recorded cells, which were time-locked to the stimulation of a nerve.

Fig. 3.

Identification of neurons contributing to activity in I2 n., R n.1, and n.2,1. A, B, Simultaneous extra- and intracellular recordings during patterned activity elicited by stimulation of n.2,3. Cell B31/32 fires in phase with I2 n., cell B9 fires in phase with activity in n.2,1 (A), and cell B8 fires in phase with the large unit activity in R n.1 (B).C, A one-for-one relationship between intracellularly recorded action potentials from B31/32 and extracellular activity recorded from I2 n. (circle, C1) and antidromically activated action potentials in B31/32 elicited by stimulation of I2 n. C2 demonstrated thatB31/32 projects through I2 n. Four oscilloscope traces triggered by intrasomatic action potentials recorded fromB31/32 (C1) and six traces triggered by nerve stimulation (C2) were superimposed.D, A similar study as in C indicated that neuronB9 sends axons in both ipsilateral (i) and contralateral (c) n.2,1 (n.2,1i, n.2,1c;circles in D1 indicate the time-locked extracellular action potentials). Six oscilloscope traces triggered by intrasomatic action potentials recorded from B9(D1) and five traces triggered by nerve stimulation (D2) were superimposed. E, CellB8 projects bilaterally through the ipsilateral and contralateral R n.1 (R n.1i, R n.1c;circles in E1 indicate the time-locked extracellular action potentials). Five oscilloscope traces triggered by intrasomatic action potentials recorded from B8(E1) and four traces triggered by nerve stimulation (E2) were superimposed.

Data analysis. The primary variable studied was the frequency of motor patterns expressed by the isolated buccal ganglia during tonic stimulation of n.2,3. Statistical comparisons between two paired samples were made with the Wilcoxon signed rank test. Comparisons among three unpaired samples were made with the Kruskal–Wallis test. Critical values of Kruskal–Wallis test (H) were approximated by critical values of χ² distribution (Zar, 1984). Post hocpairwise multiple comparisons were made with the nonparametric Newman–Keuls multiple range test. Nonparametric tests were used because significant departures from normality of the data were found by using D’Agostino’s test, and/or significant heterogeneity of variances of the data were found by using Bartlett’s test. All tests were performed as described in Zar (1984) with a significance level of 5%. Because different experimental methods were used in nondesheathed and desheathed preparations, we first considered these preparations separately. Further analyses were made by pooling the results from both types of preparations. The data presented in Results were reanalyzed by an observer who was not aware of the purpose or the procedures of the experiments. Comparison of the analyses indicated that a few motor patterns (8%) were classified differently by the observers, but the results were statistically indistinguishable.

RESULTS

Rhythmic motor programs elicited by tonic stimulation of n.2,3

In freely moving animals, consummatory feeding behaviors involve rhythmic movements of the radula (i.e., protraction, retraction, and closure). The initiation and specific type of consummatory feeding behavior are all influenced by sensory information (Kupfermann, 1974a). We began by searching for an afferent pathway that could activate different buccal motor programs related to the consummatory feeding behavior. Although the search was not exhaustive, we found that the ventral branch of the buccal nerve 2 (n.2,3; Fig.1A) projected to the inner surface of the buccal mass rather than to one of the buccal muscles. We found that patterned motor activity could be elicited by physiological stimuli (e.g., seaweed) applied to a patch of the inner surface of the buccal mass attached to the isolated buccal ganglia by n.2,3.

In addition, we found that tonic (2–4 Hz) electrical stimulation of n.2,3 elicited patterned motor output from the isolated buccal ganglia. This neural activity, which was recorded in I2 n., n.2,1, and R n.1 (Fig. 1B), was composed of two successive phases. The first phase was represented by a burst of action potentials in I2 n. and the second phase by a burst of action potentials in n.2,1 (Fig.2A). During this biphasic pattern, activity in R n.1 was recorded, and this activity was composed of at least two classes of action potentials, large- and small-amplitude activity (Fig. 2A). The large-amplitude activity in R n.1 started at approximately the same time as activity in I2 n. but terminated at various times after the termination of activity in I2 n. (i.e., there were varying degrees of overlap of R n.1 and n.2,1 activity; Fig. 2A).

Fig. 2.

Both patterns I and II were elicited by tonic stimulation of n.2,3. A, In both patterns I and II, a burst of spikes in I2 n. preceded a burst of spikes in n.2,1. Pattern I was defined as one in which 50% or more of the large-amplitude activity in the R n.1 occurred after the end of the I2 n. burst (dashed line). In pattern II, large-amplitude activity in R n.1 (black arrow), which can be distinguished from small-amplitude activity (white arrow), does not extend beyond the burst in I2 n. (dashed line). These examples of pattern I and II were recorded from the same preparation. An artifact of the tonic stimulation of n.2,3 appears in I2 n. and n.2,1 traces. B, The average phase relationship of activity in I2 n. (black), n.2,1 (gray), and the large-amplitude R n.1 activity (white) in pattern I (n = 46) and pattern II (n = 8) recorded during the test period in the 10 nondesheathed control preparations (see Fig. 4). The key distinguishing feature of patterns I and II was the duration of large-amplitude activity in R n.1 that extended beyond the termination of the I2 n. phase. In this and subsequent figures, the bars indicate the mean values ± SEM.

Based on the neural correlates of the consummatory feeding behaviors (e.g., ingestion-like and rejection-like motor patterns; Morton and Chiel, 1993a), three types of motor patterns elicited in the isolated buccal ganglia by tonic stimulation of n.2,3 could be distinguished by the phase relationship of the large-amplitude activity recorded from R n.1 with that activity recorded from I2 n. and n.2,1 (see Materials and Methods). We defined pattern I as one in which at least 50% of the large-amplitude activity in R n.1 occurred after the termination of the activity in I2 n. (Fig. 2A). Thus, in pattern I the majority of large-amplitude activity in R n.1 occurred during the second phase of the biphasic pattern (i.e., during the n.2,1 phase; Fig. 2B). This pattern was comparable to the neural correlates of ingestion (Morton and Chiel, 1993a) (see Materials and Methods). In pattern II the large-amplitude activity in R n.1 coterminated with activity in I2 n. (Fig. 2A). Thus, in pattern II this large-amplitude activity was restricted to the first phase of the biphasic pattern (Fig. 2B). This pattern was similar to the neural correlates of rejection (Morton and Chiel, 1993a) (see Materials and Methods). Intermediate patterns were those in which the large-amplitude activity in R n.1 extended beyond the I2 n. phase, but <50% of this activity in R n.1 occurred after the bursting activity in I2 n.

The motor activity of the isolated buccal ganglia that was elicited by tonic stimulation of n.2,3 was a mix of pattern I, pattern II, and intermediate patterns. This mixture of motor patterns was expressed at a relatively high frequency (0.032 ± 0.002 Hz; mean ± SEM;n = 10) during the 10 first min of stimulation. This frequency decreased slowly during prolonged nerve stimulation. However, after >1 hr of stimulation, the frequency of evoked activity (0.008 ± 0.002 Hz) was still significantly higher than the frequency of spontaneous activity (0.0037 ± 0.0005 Hz;n = 10; T₊ = 48,T₋ = −7; p < 0.05). This ability of the tonic stimulation of n.2,3 to elicit a mix of the three different motor patterns at a high frequency indicated that the stimulation not only activated the CPG but also induced a state permissive for rapid switching among different functional configurations.

Insight into the possible behavioral relevance of the nerve activities elicited by tonic stimulation of n.2,3 was obtained by relating these activities to identified motor neurons that mediate different aspects of radula movement (i.e., protraction, retraction, and closure). For example, the action potentials in B31/32 cells contribute to the activity recorded from I2 n. (Fig.3A,C). These cells function both as pattern initiators and protractor motor neurons (Susswein and Byrne, 1988; Hurwitz et al., 1996). Closure motor neurons B8 contributed to the large-amplitude activity recorded from R n.1, and retractor motor neurons B3, B6, and B9 contributed to the activity recorded from n.2,1 (Fig. 3A,B,D,E). These results indicate that activity in I2 n., n.2,1, and R n.1 represented neural correlates of each phase of the radula movement. Thus, tonic stimulation of n.2,3 could be used to elicit rhythmic motor activity that represented neural equivalents of consummatory feeding behaviors (i.e., operants). We took advantage of these distinct motor programs to develop an experimental paradigm analogous to operant conditioning.

Contingent reinforcement enhanced the frequency of buccal motor programs

A key characteristic of operant conditioning is that the delivery of reinforcement is contingent on the expression of a given operant or behavior. In one study of operant conditioning of feeding behavior inAplysia, the reinforcement was contingent on successful ingestion of food (Susswein et al., 1986). Moreover, Schwarz and Susswein (1986) found that the reinforcing pathway for some aspects of the operant conditioning of consummatory feeding behavior was mediated by the esophageal nerve (E n.). Thus, in the isolated buccal ganglia we attempted to modify the buccal motor activity by contingent electrical stimulation of a branch of the esophageal nerve (E n.2). Specifically, stimulation of E n.2 was made contingent on the expression of pattern I (i.e., ingestion-like pattern).

Three groups, each composed of 20 preparations, were used (i.e., a total of 60 preparations). In one-half of these preparations, one of the buccal ganglia was desheathed, and intracellular recordings were made from identified neurons. For all groups, rhythmic buccal motor programs were elicited by tonic stimulation of n.2,3 throughout the experiments (Fig. 4A). In a contingent-stimulation group, a phasic (10 Hz, 6 sec) stimulation of E n.2 was delivered after each pattern I (i.e., immediately after bursting activity in n.2,1) during a 10 min training period (Fig.4A,B). The consequences of the contingency between stimulation of E n.2 and expression of pattern I were examined by comparing the contingent-reinforcement group with a yoke-control group. In the yoke-control group each preparation received stimulation of E n.2 (10 Hz, 6 sec) with the same timing as in a paired preparation from the contingent-reinforcement group (Fig. 4C). Thus, in a yoke-control experiment the delivery of the stimulation of E n.2 was determined by a previous contingent experiment rather than by ongoing activity in the yoked preparation. In a control group no stimulation other than the stimulation of n.2,3 was delivered (Fig.4D).

Fig. 4.

Experimental paradigms for neural analog of operant conditioning. A, In all paradigms, tonic stimulation of n.2,3 (n.2,3 stim.) was delivered throughout the experiment. The type of patterned activity induced by stimulation of n.2,3 was represented by black circles(pattern I) or by white circles (pattern II and intermediate patterns). Experiments were divided into three periods: a pretraining period (Pre-Training), a 10 min training period (Training), and a 10 min test period (Test), which immediately followed the training period. In a single block of three matched preparations, each preparation received one of the different stimulus paradigms (i.e.,Contingent Reinforcement, B; Yoke Control, C; Control, D). B, Contingent reinforcement. During the training period phasic (10 Hz, 6 sec) stimulation of E n.2 (black squares on E n.2 stim.) was delivered immediately after expression of each pattern I (black circles). In an experimental block the beginning of the training period was determined by the first occurrence of a pattern I and the contingent stimulation of E n.2.C, Yoke control. Stimulation of E n.2 (black squares in E n.2 stim.) was applied with the same parameters and the same timing as that in the contingent stimulation paradigm (compare E n.2 stim. with that inB). In this paradigm, however, E n.2 stimulation was not contingent with any specific pattern; rather, it was “yoked” to the previous contingent-stimulation preparation in the block.D, Control. In this paradigm, no stimulation of E n.2 was delivered.

The experiments were conducted in blocks of three preparations (i.e., a contingent-reinforcement, a yoke-control, and a control preparation) in which the beginning of the training period was determined by the occurrence of the first pattern I in the contingent reinforcement preparation. This training period lasted 10 min and was followed immediately by a 10 min test period (Fig. 4A). Thus, for all three preparations in a block, the test period began with the same delay after the onset of the tonic stimulation of n.2,3.

Because expression of pattern I was required for contingent reinforcement, we discarded preparations in which this pattern was not expressed during the experiment. In addition, only those preparations that received at least five reinforcements during the contingent training were used. This criterion was chosen for two independent reasons. First, we conducted a pilot study that indicated that the effect of contingent reinforcement depended on the number of reinforcements. Second, five training trials have been used in manyin vivo and in vitro studies of nonassociative learning in Aplysia (Kandel and Schwartz, 1982; Dale et al., 1987; Byrne et al., 1991; Kennedy et al., 1992; Kaang et al., 1993;Noel et al., 1993; Alberini et al., 1994). Thus, five reinforcements appear to be a reasonable approximation for the number of reinforcements that might induce contingent-dependent modulation. To assure that the neural activity expressed by the three groups of preparations was homogeneous initially, we compared the frequency of the spontaneous activity and the frequency of pattern I elicited by tonic stimulation of n.2,3 among the three groups. No significant difference in the frequency of the spontaneous activity was observed among groups (H = 1.097, df = 2 in nondesheathed preparations, contingent reinforcement: 0.003 ± 0.0007 Hz, yoke control: 0.002 ± 0.0007 Hz, control: 0.003 ± 0.0007 Hz;H = 2.785, df = 2 in desheathed preparations, contingent reinforcement: 0.003 ± 0.001 Hz, yoke control: 0.005 ± 0.001 Hz, control: 0.003 ± 0.0008 Hz;H = 0.167, df = 2 with both nondesheathed and desheathed preparations pooled, contingent reinforcement: 0.003 ± 0.0006 Hz, yoke control: 0.003 ± 0.0007 Hz, control: 0.003 ± 0.0005 Hz). Moreover, the number of occurrences of pattern I in a 5 min period beginning at the first occurrence of this pattern was not significantly different among the groups (H = 2.267, df = 2 in nondesheathed preparations, contingent reinforcement: 3.4 ± 0.2, yoke control: 3.0 ± 0.6, control: 3.7 ± 0.6; H = 4.394, df = 2 in desheathed preparations, contingent reinforcement: 4.6 ± 0.5, yoke control: 2.6 ± 0.8, control: 3.2 ± 0.6; H = 5.316, df = 2 in both nondesheathed and desheathed preparations, contingent reinforcement: 3.9 ± 0.3, yoke control: 2.9 ± 0.5, control: 3.5 ± 0.4). However, a change in the number of occurrences of pattern I was observed 10 min after the first occurrence of this pattern. This change was significant in nondesheathed preparations (H = 8.442, df = 2; p < 0.015). Specifically, the number of occurrences of pattern I increased in the contingent-reinforcement group as compared with the yoke-control group (contingent reinforcement: 6.7 ± 0.5, yoke control: 5.0 ± 1.5). There was no significant change between contingent reinforcement and control (6.4 ± 1.5) groups or between the yoke-control group and the control group. A similar change also was observed in desheathed preparations, but it was not significant (H = 4.836, df = 2, contingent reinforcement: 6.9 ± 0.5, yoke control: 4.0 ± 1.2, control: 4.8 ± 0.8). This observation presumably represents the effects of stimulation of E n.2 during training that increases the expression of the pattern I in the contingent-reinforcement group (see below).

To evaluate the effect of the contingent reinforcement on the buccal motor program, we counted the number of motor patterns expressed during the 10 min test period immediately after the training period (Fig.4A). Figure 5illustrates typical recordings of the rhythmic motor activity expressed during the test period in a preparation from the control group (Fig.5A), in a preparation from the contingent-reinforcement group (Fig. 5B), and in a preparation from the yoke-control group (Fig. 5C). The number of motor patterns expressed in the contingent-reinforcement preparations was higher than in preparations from either the control or yoke-control groups. In contrast, both control and yoke-control preparations expressed a comparable frequency of patterned activity. These observations were supported by statistical analyses (see below).

Fig. 5.

Representative recordings of rhythmic activity during the test period. The patterned activity (•, pattern I; ○, other patterns) recorded from I2 n., R n.1, and n.2,1 in nondesheathed preparations during a 10 min test period in a control (A), a contingent-reinforcement (B), and a yoke-control (C) preparation. The frequency of patterned activity was enhanced after contingent reinforcement, as compared with the control and yoke-control preparations.

In nondesheathed preparations, comparison of the three groups indicated a significant difference in the number of occurrences of motor patterns (H = 10.133, df = 2; p < 0.006). Similar results were observed in desheathed preparations (H = 8.258, df = 2; p < 0.02).Post hoc pairwise comparisons indicated that the differences resulted from an increase in the number of patterns in the contingent-reinforcement group compared with either the control (q₂ = 5.318, p < 0.001 in nondesheathed preparations; q₃ = 3.915,p < 0.025 in desheathed preparations) or the yoke-control groups (q₃ = 4.131,p < 0.01 in nondesheathed preparations;q₂ = 4.196, p < 0.005 in desheathed preparations). No significant differences were observed between the control and the yoke-control groups (q₂ = 0.829, control: 9.5 ± 0.8 and yoke control: 8.6 ± 2.1 in nondesheathed preparations;q₂ = 1.630, control: 5.7 ± 0.8, and yoke control: 7.2 ± 0.9 in desheathed preparations).

These observations also were supported by pooling data from nondesheathed and desheathed preparations (Fig.6). A significant difference in the number of occurrences of patterns was observed among groups (H = 17.200, df = 2; p < 0.001). Specifically, the contingent-reinforcement group expressed significantly more patterns than either the control (q₂ = 7.536, p < 0.001) or the yoke-control groups (q₃ = 5.077,p < 0.001). No significant difference in the number of occurrences of patterns was observed between the control and yoke-control groups (q₂ = 0.048, control: 7.6 ± 0.7, yoke control: 7.9 ± 1.1).

Fig. 6.

Contingent reinforcement increased the frequency of the rhythmic activity. Statistical comparison of the number of patterns expressed during the 10 min test period in the control (white bar), in the contingent-reinforcement (black bar), and in the yoke-control (gray bar) groups from both nondesheathed and desheathed preparations (n = 20 in each group). A significantly higher frequency of rhythmic activity was expressed in the contingent-reinforcement group, as compared with the control (p < 0.001) or yoke-control groups (p < 0.001). This effect resulted from the contingency of the reinforcement because no significant difference (N.S.) was observed between the yoke-control and the control groups.

These results indicated that contingent stimulation of E n.2 enhanced the frequency of rhythmic activity expressed by the isolated buccal ganglia. This enhancement did not result from a nonspecific effect of stimulating the esophageal nerve because no significant difference was observed between the yoke-control group, which received noncontingent stimulation of E n.2, and the control group, which received no stimulation of E n.2. If the enhancement of the buccal activity depended specifically on the contingency of the stimulation of E n.2, one would predict that this increase should result from a selective enhancement of pattern I (i.e., the contingently reinforced pattern) with no change in the number of the nonreinforced patterns.

Selective enhancement of a designated pattern

To determine whether the reinforced pattern (i.e., pattern I) was enhanced selectively, we counted the number of occurrences of each type of motor pattern (i.e., patterns I and II and intermediate patterns) during the test period in each preparation (Fig.7A). A comparison of the number of occurrences of pattern I indicated a significant difference among the three groups of nondesheathed (H = 8.459, df = 2; p < 0.025) or desheathed preparations (H = 7.769, df = 2; p < 0.025).Post hoc pairwise comparisons indicated that the expression of pattern I was significantly higher in the contingent group than in either the control (q₂ = 4.944,p < 0.001 in nondesheathed preparations andq₂ = 4.971, p < 0.005 in desheathed preparations) or the yoke-control groups (q₃ = 3.736, p < 0.025 in nondesheathed preparations and q₃ = 3.448,p < 0.05 in desheathed preparations). In contrast, no significant differences in the number of occurrences of pattern I were observed between the control and the yoke-control groups in nondesheathed (q₂ = 0.615, control: 4.6 ± 1.3, yoke control: 5.4 ± 2.3) or desheathed preparations (q₂ = 0.160, control: 4.1 ± 0.9, yoke control: 3.9 ± 1.0). Similar results were observed by pooling data from nondesheathed and desheathed preparations (Fig.7A1; H = 17.216, df = 2;p < 0.001). In the pooled data the contingent group expressed a significant enhancement of pattern I as compared with either the control (q₂ = 7.288,p < 0.001) or the yoke-control groups (q₃ = 5.224, p < 0.001). No significant difference was observed between the control and yoke-control groups (q₂ = 0.516, control: 4.3 ± 0.8, yoke control: 4.7 ± 1.2). These results indicated that, when stimulation of E n.2 was contingent on the expression of pattern I, the frequency of this pattern was increased. This phenomenon was not attributable to a nonspecific effect of the esophageal stimulation because a similar increase was not observed in the yoke-control group.

Fig. 7.

Only the reinforced pattern of activity was increased. A, Selective increase of pattern I. In both nondesheathed and desheathed preparations during a 10 min test period immediately after the training session in which stimulation of E n.2 was contingent on pattern I, the number of occurrences of pattern I was increased significantly in the contingent-reinforcement group (black bar), as compared with the control (white bar; p < 0.001) or the yoke-control groups (gray bar;p < 0.001), and no significant difference (N.S.) was observed between the control and yoke-control groups (A1). In contrast, in the same preparations and during the same test period the number of occurrences of other patterns (i.e., the nonreinforced patterns: pattern II and intermediate patterns) was not significantly different (N.S.) among the groups (A2). B, Selective increase of pattern II. In nondesheathed preparations contingent reinforcement of pattern II increased the number of occurrences of pattern II during the 10 min test period in the contingent-reinforcement group (black bar), as compared with the control (white bar; p < 0.05) or the yoke-control groups (gray bar; p< 0.005). No significant difference (N.S.) was observed between the control and yoke-control groups (B1), but in the same preparations and during the same test period the number of occurrences of the other patterns (i.e., the nonreinforced patterns: pattern I and intermediate patterns) was not significantly different among the groups (B2).

In the same preparations and during the same test period, we also compared the number of occurrences of the nonreinforced patterns (i.e., pattern II and intermediate patterns). As a first step in this analysis the numbers of pattern II and intermediate patterns were counted separately. In contrast to the enhancement of pattern I, no significant modification in the number of occurrences of pattern II (H = 2.181, df = 2, contingent reinforcement: 0.3 ± 0.2, yoke control: 1.3 ± 0.6, control: 0.8 ± 0.5 in nondesheathed preparations; H = 1.350, df = 2, contingent reinforcement: 0.7 ± 0.6, yoke control: 1.1 ± 0.7, control: 0.3 ± 0.2 in desheathed preparations) or the number of occurrences of intermediate patterns was observed among the different groups (H = 3.549, df = 2, contingent reinforcement: 3.7 ± 0.8, yoke control: 1.9 ± 0.8, control: 4.1 ± 0.9 in nondesheathed preparations; H = 2.690, df = 2, contingent reinforcement: 2.5 ± 0.8, yoke control: 2.2 ± 0.6, control: 1.3 ± 0.7 in desheathed preparations). Further analyses considered pattern II and intermediate patterns combined into a total number of “other patterns” (i.e., nonreinforced patterns). The data indicated that no significant difference in the number of occurrences of these other patterns was observed among groups (Fig. 7A2). This observation was obtained in nondesheathed (H = 2.074, df = 2, contingent reinforcement: 4.0 ± 0.8, yoke control: 3.2 ± 1.4, control: 4.9 ± 1.1) and desheathed preparations (H = 2.342, df = 2, contingent reinforcement: 3.2 ± 1.1, yoke control: 3.3 ± 1.0, control: 1.6 ± 0.7) as well as by pooling data from nondesheathed and desheathed preparations (H = 0.465, df = 2, contingent reinforcement: 3.6 ± 0.7, yoke control: 3.3 ± 0.8, control: 3.3 ± 0.7). Thus, contingent stimulation of E n.2 specifically increased the expression of pattern I but did not affect the expression of nonreinforced patterns elicited in the same test period.

To determine whether the modification of motor patterns by contingent stimulation of E n.2 is specific to pattern I or whether the same reinforcer (i.e., stimulation of E n.2) can modify different types of patterns depending on its contingency to the patterned activity, we conducted a separated series of experiments in which stimulation of E n.2 was made contingent on the expression of pattern II.

The experimental paradigm used is these experiments was similar to that described previously (see Fig. 4) except that the reinforced pattern in the contingent-reinforcement group was pattern II rather than pattern I. The group of preparations (contingent reinforcement,n = 12) that received stimulation of E n.2 contingent on expression of pattern II was compared with a yoke-control group (n = 12) and a control group (n = 12). In a 10 min test period immediately after the training period the number of occurrences of pattern II (i.e., the reinforced pattern) was significantly different among groups (Fig. 7B1;H = 7.597, df = 2; p < 0.025). The contingent-reinforcement group expressed significantly more occurrences of pattern II than the yoke-control group (q₂ = 4.307, p < 0.005) and the control group (q₃ = 3.439, p < 0.05). No significant difference in the number of occurrences of pattern II was observed between the yoke-control (0.8 ± 0.3) and the control groups (0.7 ± 0.3; q₂ = 0.816). In contrast, in the same test period the number of occurrences of the nonreinforced patterns (i.e., pattern I or intermediate patterns) was not modified among the same groups of preparations (Fig.7B2; H = 0.466, df = 2, contingent reinforcement: 3.3 ± 0.7, yoke control: 4.3 ± 0.7, control: 3.5 ± 0.7 for pattern I; H = 0.874, df = 2, contingent reinforcement: 2.8 ± 0.6, yoke control: 3.0 ± 0.6, control: 3.6 ± 0.7 for intermediate patterns;H = 0.592, df = 2, contingent reinforcement: 6.1 ± 1.0, yoke control: 7.3 ± 0.8, control: 7.1 ± 0.9 for pattern I and intermediate patterns pooled in a single category of nonreinforced patterns).

These results indicated that stimulation of E n.2 is a general reinforcer that can modify different types of motor patterns. However, the modifications of the motor activity are restricted to a given pattern when the reinforcer is applied contingently to this pattern. Thus, the modification of the buccal motor program induced in vitro depended on a key characteristic of operant conditioning, namely, the contingency of the reinforcement. To examine further the similarity between the associative neural plasticity exhibited by the buccal motor program and operant conditioning, we examined extinction and retention of the enhancement of pattern I.

Extinction and retention of the induced modification

Extinction of the increased expression of a motor pattern should be observed if the contingent reinforcement is withheld despite the continued expression of this pattern. Retention of the induced modifications should occur after a rest period after the training session (i.e., in absence of nerve stimulation). To examine extinction of the neural modifications induced by the contingent reinforcement of pattern I, the motor activity expressed in the three groups of nondesheathed preparations described previously (i.e., contingent reinforcement, n = 10; yoke control, n= 10; and control, n = 10) was examined 1 hr after the training session. In this set of experiments the tonic stimulation of n.2,3 was delivered continuously to elicit the patterned activity during the intervening hour after the training session. No reinforcing stimulation was applied after the training period in any of the groups. Although a higher number of occurrences of pattern I was expressed during the test period immediately after the training session in the contingent-reinforcement group, in the same preparations the number of occurrences of pattern I was not significantly different among the three groups 1 hr after the training period (H = 0.044, df = 2, contingent-reinforcement: 4.0 ± 1.9, yoke control: 2.5 ± 0.9, control: 2.2 ± 1.0). There were no significant differences 1 hr after training in the number of occurrences of pattern II and intermediate patterns considered separately (H = 3.171, df = 2, contingent-reinforcement: 0.0 ± 0.0, yoke control: 1.0 ± 0.9, control: 0.6 ± 0.3 for pattern II; andH = 0.179, df = 2, contingent-reinforcement: 1.9 ± 0.6, yoke control: 1.6 ± 0.6, control: 1.9 ± 0.9 for intermediate patterns) or when they were combined into a single group of nonreinforced patterns (H = 0.006, df = 2, contingent-reinforcement group: 1.9 ± 0.6, yoke-control group: 2.6 ± 1.3, control group: 2.5 ± 1.1). Thus, the neural modifications that previously were induced by the contingent reinforcement “extinguished” during the 1 hr of nonreinforced activity that followed the training session.

The extinction of the induced modifications did not result from an inability of the isolated buccal ganglia to retain the neural changes for 1 hr. In a separate set of experiments we investigated the retention of the contingent-dependent modifications in the isolated buccal ganglia by comparing the neural activity expressed in contingent-reinforcement (n = 5), control (n = 5) and yoke-control groups (n = 5) 1 hr after training. In contrast to the stimulation paradigm used to study extinction, in this set of experiments the tonic stimulation of the n.2,3 was discontinued for 1 hr at the end of the training period. The spontaneous frequency of pattern I expressed during this post-training period was very low in the three groups of preparations (contingent reinforcement, 0.0006 ± 0.0003 Hz; control, 0.0015 ± 0.0007 Hz; yoke, 0.0003 ± 0.0001 Hz). One hour after the training session, the tonic stimulation of n.2,3 was restarted for 20 min, and the induced patterned activity was compared in the three groups of preparations during a 10 min test period beginning 10 min after the onset of the stimulation.

A comparison of the number of the different patterns indicated a significant enhancement of pattern I (i.e., the reinforced pattern) 1 hr after training (Fig.8A; H = 8.874, df = 2; p < 0.03). The frequency of pattern I was significantly higher in the contingent-reinforcement group, as compared with the control (q₂ = 4.948,p < 0.001) and the yoke-control groups (q₃ = 3.550, p < 0.05). No significant difference was observed between the number of occurrences of pattern I expressed in the control (0.6 ± 0.4) and yoke-control groups (0.6 ± 0.6, q₂ = 0.295). In addition, no significant difference in the frequency of pattern II (H = 2.041, df = 2, contingent reinforcement: 0.0 ± 0.0, yoke control: 1.0 ± 1.0, control: 0.4 ± 0.2) and intermediate patterns was observed among the three groups (H = 1.734, df = 2, contingent reinforcement: 0.4 ± 0.4, yoke control: 0.6 ± 0.6, control: 1.6 ± 1.1). A similar result was obtained by combining the nonreinforced patterns (i.e., pattern II and intermediate patterns) in a single category of “other patterns” (Fig. 8B;H = 2.487, df = 2, contingent reinforcement: 0.4 ± 0.4, yoke control: 1.6 ± 1.0, control: 2.0 ± 1.0). These results indicated that the specific enhancement of the reinforced pattern by just 10 min of training can be retained in vitro for >1 hr after the training session. Thus, the effect of the contingent stimulation of E n.2 appears to be long-lasting.

Fig. 8.

Long-lasting effects of the contingent reinforcement. During a 10 min test period that began 1 hr after the training session, the contingent-reinforcement group (black bar) expressed a significantly greater number of occurrences of the reinforced pattern (pattern I, A), but not of the nonreinforced patterns (pattern II and intermediate, B), than in the control (white bar;p < 0.001) and the yoke-control groups (gray bar; p < 0.05). The effects of the contingent reinforcement persisted for >1 hr. Five nondesheathed preparations in each group were used.

Cellular modifications induced by contingent reinforcement

As a first step toward investigating the mechanisms underlying contingent-dependent modulation of the buccal CPG, we have begun to search for changes in the activity of identified cells that might be correlated with the effects of the contingent reinforcement of pattern I. Because the reinforcing stimulation specifically increased the occurrence of pattern I, we focused our attention on cellular activities that were specifically expressed during this pattern. The closure motor neurons B8 can be active primarily during either the protraction phase (see Fig. 3B) or the retraction phase of the pattern (Figs. 3A, 9A). Previous studies (Morton and Chiel, 1993b) indicated that an essential feature of pattern I was activity in B8 occurring primarily during the retraction phase of the pattern. However, we do not know whether the increasing occurrences of pattern I that were induced by contingent reinforcement resulted from a change in activity of B8 or in other neural activity. Thus, we investigated whether the effects of the contingent reinforcement were correlated with an enhancement of activity in B8 during the retraction phase.

Fig. 9.

Neural modification induced by contingent reinforcement. A, The phase relationship of the discharge in the closure motor neuron B8 relative to the burst of activity in I2 n. can vary from one pattern to another. In some B8 bursts at least 50% of the activity occurred after termination of burst in I2 n. In this case the majority of activity in B8 was out of phase with activity in I2 n. B, The number of occurrences of bursts in B8 characterized in A and that were associated with expression of pattern I significantly increased in the contingent-reinforcement group (black bar), as compared with the control (white bar;p < 0.001) and yoke-control groups (gray bar; p < 0.05). No significant difference (N.S.) was observed between the control and the yoke-control groups. The contingent reinforcement specifically enhanced the discharge of B8 during the retraction phase (i.e., after the burst in I2 n. terminated).

Using simultaneous intracellular and extracellular recordings, we characterized B8 activity during each occurrence of pattern I in the contingent-reinforcement, control, and yoke-control groups. The activity in B8 was characterized by using the same criterion that was used to distinguish the different motor patterns (i.e., the percentage of activity occurring after a burst in I2 n.; Fig.9A). The number of occurrences of B8 bursts in which at least 50% of the activity occurred after the burst in I2 n. (i.e., after the protraction phase) was significantly different among groups (Fig. 9B; n = 10 in each group, H = 7.300, df = 2; p< 0.03). The number of occurrences of B8 bursts that met the criterion was higher in the contingent-reinforcement group, as compared with the control (q₂ = 4.784, p < 0.001) and yoke-control groups (q₃ = 3.359,p < 0.05), and no significant difference was observed between the control (3.6 ± 0.9) and the yoke-control groups (q₂ = 0.214, 3.4 ± 1.0). These results indicated that contingent reinforcement of pattern I enhanced the discharge of B8 during the retraction phase of the pattern. We do not know whether the increase activity in B8 during the retraction phase represented a change in the cellular properties of B8 or the presynaptic inputs to B8.

DISCUSSION

The isolated buccal ganglia in Aplysia contain a CPG that produces different motor patterns (i.e., pattern I, pattern II, and intermediate patterns) similar to those previously describedin vivo during the consummatory feeding behaviors. The motor activity expressed by this multifunctional circuit can be modified by contingent reinforcement, and this contingent-dependent modification can persist for at least 1 hr after training. In particular, when stimulation of an esophageal nerve (E n.2) was made contingent on the expression of one of the motor patterns, the frequency of this pattern was enhanced. At least for the enhancement of pattern I, the modification was correlated with a change in the activity of the closure motor neuron B8.

Neural analog of operant conditioning

Modifications of behaviors by operant conditioning have been demonstrated in both vertebrates (Thorndike, 1911; Skinner, 1938;Berger, 1968; Beecher, 1971; Wolpaw, 1987) (see also Engel and Schneiderman, 1984; Byrne 1987) and invertebrates (Horridge, 1962;Hoyle, 1980; Booker and Quinn, 1981; Hawkins et al., 1985; Cook and Carew, 1986; Susswein et al., 1986; Lukowiak et al., 1996). Some cellular modifications induced by operant conditioning have been identified (Woollacott and Hoyle, 1977; Jaffard and Jeantet, 1981;Wyler, 1985; Skelton et al., 1987; Mahajan and Desiraju, 1988; Cook and Carew, 1989; Feng-Chen and Wolpaw, 1996), but the contribution of these cellular changes to the observed changes in the behavior is still poorly understood.

CPGs underlying rhythmic motor behaviors may be advantageous preparations to determine how a neural network can be changed by operant conditioning. Significant progress has been made in understanding the cellular, synaptic, and network processes that underlie several rhythmic motor behaviors (Selverston and Moulins, 1985; Getting, 1989; Syed et al., 1990; Stein et al., 1997). Moreover, such behaviors can be modified by operant conditioning (Cook and Carew, 1986; Susswein et al., 1986; Lukowiak et al., 1996).

Consummatory feeding behaviors in Aplysia can be conditioned operantly by positive or negative reinforcement, respectively, increasing or decreasing aspects of these rhythmic behaviors (Susswein et al., 1986). Key neural elements underlying these behaviors have been identified in the buccal ganglia. They include sensory neurons (Rosen et al., 1982), pattern-generating cells (Susswein and Byrne, 1988;Plummer and Kirk, 1990; Hurwitz and Susswein, 1996; Hurwitz et al., 1996), and motor neurons (Morton and Chiel, 1993b; Church and Lloyd, 1994). We developed a preparation to examine how a CPG might be modified by operant conditioning. In this preparation the esophageal nerve (E n.2), which we used as the reinforcer, previously was described to mediate the negative reinforcement (Schwarz and Susswein, 1986). It is not known, however, whether E n.2 also can mediate the positive reinforcement. Our data indicate that contingent stimulation of E n.2 can increase the expression of several motor outputs (i.e., pattern I or pattern II), suggesting that E n.2 could mediate positive reinforcement also. This preparation expressed several key features of operant conditioning. First, the contingent reinforcement modified the frequency of a motor output. Second, this modification was specific to the reinforced motor activity. Third, this contingent-dependent modification extinguished if the reinforcement was withheld. Fourth, the “memory” of the contingent-dependent change was long-lasting. Thus, contingent reinforcement of buccal motor patterns can be used as an in vitro analog of operant conditioning.

Selective enhancement of a specific pattern of neural activity

In operant conditioning a relevant operant designated by the delivery of contingent reinforcement is durably modified relative to irrelevant operants that do not provide the reinforcement. This phenomenon is referred to as the “law of effect” (Thorndike, 1933) and indicates that a specific operant can be selectively and durably modified by a reinforcement. Studies on the functioning of the CPGs mediating rhythmic motor behaviors indicate that several operants can be produced by changes in a single neural network (Heinzel, 1988;Mortin and Stein, 1989; Morton and Chiel, 1993b; Green and Soffe, 1996). To date, however, the neuronal mechanisms by which a specific network configuration generating a given motor output can be selectively and durably modified by a reinforcement have not been determined.

Investigating contingent-dependent modifications of a designated motor pattern in a multifunctional circuit may help to determine the neural processes underlying the selection of a specific motor output. The neural circuit of the buccal ganglia mediates rhythmic motor programs composed of at least three distinct patterns (i.e., pattern I, pattern II, and intermediate patterns), which in turn have been correlated previously with different aspects of feeding (e.g., ingestion and rejection; Morton and Chiel, 1993a). Our data indicate that phasic stimulation of a reinforcing afferent pathway (E n.2) made contingent on the expression of a given motor pattern induced a long-term and specific enhancement of the expression of this designated pattern. These results indicate that activity-dependent modulation of a neural circuit can depend not only on the activity produced by a neural network but also on the specific configuration of this activity. By studying the cellular mechanisms underlying such contingent-dependent modification of a specific pattern, we hope to gain insight into the modulation of a multifunctional network as well as in the neural basis of the law of effect.

Cellular mechanisms of contingent-dependent neural plasticity

The present study indicates that contingent-dependent plasticity induced by stimulation of E n.2 was expressed as an enhancement of the frequency of the buccal motor output and that this increased motor activity was associated with a specific enhancement of the number of occurrences of the reinforced pattern. These observations suggest that at least two features of the CPG are modulated: (1) the cellular mechanisms underlying pattern initiation, and (2) the cellular mechanisms underlying pattern selection. Presently, it is not known whether these two processes are mediated by common or distinct cellular loci. Potential loci might include elements of afferent pathways to the CPG, elements of the CPG, and elements of the efferent pathway.

Our study did not investigate the possibility that the n.2,3 pathway that elicited the rhythmic activity was modified by contingent stimulation of E n.2. A change in the efficacy of this pathway to the CPG by the stimulation of E n.2 could explain an increase in the patterned activity. Moreover, if different subtypes of afferents in n.2,3 elicited specific motor patterns, then modification of specific subtypes of afferents could explain the modification of a designated pattern. However, comparison of the control and the yoke-control groups indicated that patterned activity elicited by stimulation of n.2,3 was statistically similar when stimulation of E n.2 was delivered (i.e., yoke control) or not (i.e., control). Thus, stimulation of E n.2 appears to have no long-lasting effect on n.2,3 that elicited patterned activity. Moreover, because n.2,3 was stimulated tonically, there is no relationship between the reinforcing stimulation of E n.2 and stimulation of n.2,3 that could explain the contingent-dependent modification observed between the yoke-control and the contingent-reinforcement groups. Although we cannot totally exclude that a change in the afferent n.2,3 pathway occurs, it is unlikely that such a modification could, by itself, explain the contingent-dependent change in the motor output.

In contrast, the results indicated that contingent reinforcement was associated with a modification in the firing activity of a key element in the efferent pathway, the motor neuron B8. In particular, after contingent reinforcement, the activity of B8 during the retraction phase of the pattern I was increased. We cannot exclude the possibility that this modification resulted from a change in the intrinsic properties of the B8 neurons. However, B8 neurons are involved in each of three types of motor pattern (Morton and Chiel, 1993b), but expression of pattern II and intermediate patterns was not modified. Therefore, it is unlikely that a modification of the properties of B8 alone can explain how the neural modification induced by the contingent reinforcement was specific to pattern I. Moreover, there is no evidence to suggest that activity in B8 can influence pattern generation.

Our data are consistent with the hypothesis that an element of the CPG presynaptic to B8 is responsible for the contingent-dependent plasticity. Recent evidence suggests that neurons can be recruited dynamically into a CPG and thereby allow the expression of a particular motor pattern (Hooper and Moulins, 1989; Meyrand et al., 1991; Soffe, 1993; Norris et al., 1994) (see Dickinson and Moulins, 1992; Dickinson, 1995). In the buccal ganglia of Aplysia, several pattern-generating cells and motor neurons are active during each of three types of motor patterns (i.e., pattern I, pattern II, and intermediate patterns) or related behaviors (e.g., ingestion and rejection; Cropper et al., 1990; Morton and Chiel, 1993b; Church and Lloyd, 1994; Hurwitz et al., 1996), but additional elements are recruited for expression of a given behavior (Cropper et al., 1990). If neurons presynaptic to B8 are recruited specifically to express pattern I, then contingent reinforcement on activity in these cells may be responsible for a change in the neural activity that involves only the expression of pattern I. Thus, contingent reinforcement on recruitment of pattern-specific neurons in a multifunctional CPG could underly modifications induced by operant conditioning. As we investigate these issues further, new insights will be provided not only into the mechanisms of contingent-dependent neuromodulation of a CPG but also into the mechanisms of the initiation and selection of patterned motor output.