Operant Conditioning of Gill Withdrawal in Aplysia

Abstract

A basic question in neuroscience is how different forms of learning are related. To further address that question, we examined whether gill withdrawal in Aplysia, which has already been studied extensively for neuronal mechanisms contributing to habituation, sensitization, and classical conditioning, also undergoes operant conditioning. Animals were run in pairs. During the initial training period, the contingent (experimental) animal received a siphon shock each time its gill relaxed below a criterion level, and the yoked control animal received a shock whenever the experimental animal did, regardless of its own gill position. This was followed by an extinction period when there was no shock, a retraining period when both animals were contingent, and another extinction period. The experimental animals spent more time with their gills contracted above the criterion level than did the control animals during each period, demonstrating operant conditioning. The type of gill behavior modified by learning shifted over time: the experimental animals had a larger increase in the frequency and duration of spontaneous contractions than did the control animals during the first but not the last extinction period and a larger increase in the level of tonic contraction during the last but not the first extinction period. Because many of the neurons controlling spontaneous and tonic gill withdrawal have already been identified, it should now be possible to examine the cellular locus and mechanism of operant conditioning and compare them with those for other forms of learning of the same behavior.

Introduction

Learning can be divided into a number of categories including nonassociative forms such as habituation and sensitization, in which an animal learns about the properties of a single stimulus, and associative forms, in which an animal learns about the relationship between two stimuli (classical conditioning) or between a stimulus and the animal’s own behavior (operant conditioning). A basic question is how these different forms of learning are related. Do they involve the same underlying processes or fundamentally different processes? It should be possible to answer this question by comparing the neural mechanisms of the different forms of learning, ideally in the same preparation. The gill- and siphon-withdrawal reflex of Aplysia has been useful for such studies, because it exhibits several forms of learning including habituation, dishabituation, sensitization, and classical conditioning (Pinsker et al., 1970; Carew et al., 1971, 1981), and it is amenable to cellular analysis. Studies of the cellular mechanisms of learning in Aplysia have revealed both similarities and differences between the mechanisms contributing to dishabituation and sensitization (Carew et al., 1971; Wright et al., 1991; Cohen et al., 1997; Antonov et al., 1999) and also between those contributing to sensitization and classical conditioning (Hawkins et al., 1993; Antonov et al., 2001, 2003; Li et al., 2005).

We have now extended this type of analysis to operant conditioning. Operant conditioning has been demonstrated previously in Aplysia for rejection of inedible food (Susswein and Schwarz, 1983), head waving (Cook and Carew, 1986), and feeding (Nargeot et al., 1997; Brembs et al., 2002). To examine whether gill withdrawal also undergoes operant conditioning, we attempted to teach animals to keep their gills contracted to avoid electric shock, similar to operant conditioning of leg position in headless locusts (Horridge, 1962).

Materials and Methods

Aplysia californica weighing 100–200 g were obtained from Sea Life Supply (Sand City, CA) or Marinus (Long Beach, CA) and housed in a 150 gallon aquarium with circulating artificial seawater (Instant Ocean; Aquarium Systems, Mentor, OH) at 15°C on a 12 h light/dark cycle. Experiments were conducted in a similar aquarium, in which animals were restrained in individual cages by 10 SILASTIC (Dow Corning, Midland, MI) loops attached to the parapodia with hooks and a Plexiglas rod attached to the mantle shelf with surgical glue. The animals usually also held onto one of the forward loops with their anterior foot. A stimulating electrode (the bared end of a fine insulated silver wire) was implanted in the siphon near the anus, and gill withdrawal was recorded with an isotonic movement transducer (Harvard Apparatus, South Natick, MA) connected to the efferent vein of the gill with a silk suture. A criterion level of gill withdrawal was set at ∼20% of maximal withdrawal, and the siphon stimulation (200 ms AC shock) was tested one to three times and adjusted to produce a withdrawal above the criterion level of ∼30 s. Both animals received the same number of tests. The criterion levels of the experimental and control groups were not significantly different nor were their shock intensities (Table 1). The animals were then rested 1 h before the beginning of training.

Table 1.

Average parameters and results in the four series of experiments

Series	1 (n = 20)	2 (n = 14)	3 (n = 16)	4 (n = 19)	Total (n = 69)	Exp versus Con Wilcoxon T	Series K–W H
Shock (mA)
Exp	5.5	6.2	4.8	5.5	5.5	723.0NS	1.69NS
Con	5.5	5.0	5.0	5.2	5.0
Criterion (mm)
Exp	0.62	0.62	0.62	0.62	0.62	2.0NS	1.29NS
Con	0.62	0.62	0.62	0.62	0.62
First response (s)
Exp	28	28	40	41	33		10.61*
Time above criterion (%)
Pre
Exp	1	5	4	2	3	1145.5NS	2.56NS
Con	2	5	3	2	2
Phase I training
Exp	92	92	95	94	94	276.5**	2.38NS
Con	66	70	80	69	70
Phase I extinction
Exp	31	29	25	23	29	847.5*	3.47NS
Con	17	22	26	14	18
Phase II training
Exp	92	94	94	96	94	795.5*	3.02NS
Con	75	84	94	95	88
Phase II extinction
Exp	22	19	14	31	19	692.0**	1.65NS
Con	11	12	13	13	12
Spontaneous contractions (s)
Pre
Exp	34	108	101	29	65	1081.5NS	6.53NS
Con	60	99	78	39	63
First extinction
Exp	127	168	106	77	106	875.0*	7.69NS
Con	84	78	107	69	79
Last extinction
Exp	65	104	60	70	69	1015.5NS	4.99NS
Con	62	82	57	72	71
Baseline shift (mm)
Pre
Exp	0	0	0	0	0	812.0NS	4.07NS
Con	0	0	0	0	0
First extinction
Exp	1.41	0.78	0.78	0.94	0.62	842.5NS	1.96NS
Con	0.94	0.31	1.09	0.62	0.62
Last extinction
Exp	1.72	0.31	0.62	1.56	0.94	604.5*	1.06NS
Con	0.47	−0.16	0.16	0.62	0.31

Median values of the parameters and results in the four series of experiments (see Materials and Methods) and overall are shown. The experimental and control animals were compared by Wilcoxon’s signed-ranks tests (T), and the four series of experiments were compared by Kruskal–Wallis ANOVAs (K–W H). Exp, Experimental; Con, control; Pre, pretest period.

*p < 0.05;

**p < 0.01;

NS, not significant.

Animals were run in pairs (Fig. 1A1). There was a 10 min pretest period with no shocks to assess the animals’ baseline behavior, followed by two phases of training in one continuous session (Fig. 1A2). During phase I training, the contingent (experimental) animal received a siphon shock each time its gill relaxed below the criterion level for at least 2 s. If the gill stayed below the criterion level, the shock was repeated every 5 s. The yoked control animal received a shock whenever the experimental animal did, regardless of its own gill position. Thus, the pattern of shocks was identical for the two animals but was contingent on the behavior of only one of them. During phase II training, both animals received shock contingent on their own behavior. Each phase consisted of two 10 min training periods alternating with two 10 min extinction periods, during which no shock was delivered.

Figure 1.

A, Experimental arrangement (A1) and behavioral protocol (A2). See Materials and Methods for details. C, Contingent; Y, yoked; EXT, extinction. B, Example of the gill positions of an experimental and control animal during the pretest period (Pre), the beginning of phase I training, the end of phase I extinction, and the beginning of phase II training. The horizontal dashed lines indicate the criterion levels for each animal, the vertical dashed lines indicate the beginning of training, and the arrows indicate when each animal was shocked.

Animals were assigned to be contingent or yoked in such a way as to counterbalance the durations of their responses to the first shock (the duration of the response to the first shock was longer for the contingent animal than the yoked control animal in 49% of experiments). We measured the percentage of time that each animal kept its gill above the criterion level, and we also measured the number and duration of spontaneous gill contractions 1.5 mm above the momentary baseline and the lowest baseline point during the periods with no shock. We subtracted the corresponding values during the pretest period for each animal and compared the experimental and control animals in each experiment using Wilcoxon’s signed-ranks test. We used nonparametric statistics because our primary measure (percentage of time above criterion) had a highly non-normal distribution as a result of a ceiling effect (Fig. 2).

Figure 2.

Percentage of time above criterion. A, Median percentage of time that the gill was contracted above the criterion level for the experimental and control animals in each phase of the experiments. The error bars indicate the interquartile ranges; *p < 0.05, **p < 0.01 compared with the matched control animals. PRE, Pretest period. B, C, Cumulative probability plots showing the performance of each experimental (Exp) and control (Con) animal during phase I and II training (B) and extinction (C).

There were four series of experiments with slight variations on the basic experimental design (Table 1). The first series was as described above. In the second series, the response to the shock was tested and adjusted one additional time 30 min after the original adjustment. In the third series, the first shock response during training was required to be >20 s, and in the fourth series, there was only one training period in each phase. Because there was not a significant difference between the four series for any of the results by Kruskal–Wallis ANOVAs, we combined the data to give a better estimate of the average behavior (Table 1, Figs. 2, 3).

Figure 3.

Spontaneous contractions and baseline shift. A1, Total duration of spontaneous contractions for the experimental and control animals during the pretest, the first extinction period, and the last extinction period. A2, Cumulative probability plot showing the performance of each animal. B1, Shift in the baseline level of gill contraction during the first and last extinction periods, compared with the pretest. B2, Cumulative probability plot showing the performance of each animal. A1 and B1 show the medians and interquartile ranges. *p < 0.05 compared with the matched control animals. EXT or Ext, Extinction; PRE, pretest period; Con, control; Exp, experimental.

Results

Our primary measure was the percentage of time that the gill was above criterion level during each period. As expected, the control animals exhibited a nonassociative increase in gill withdrawal during phase I training, when the animals responded directly to the shock (Figs. 1B, 2). More importantly, the experimental animals exhibited a greater increase in gill withdrawal than did the control animals (T = 276.5 with no ties; p < 0.001). This difference may reflect associative learning, but another possible interpretation is that the experimental animals could perform nearly perfectly simply by responding reflexively to each shock, which was not true for the yoked control animals. We therefore assessed the animals’ learning with two additional types of tests, during which the experimental and control animals were treated identically. Thus, any difference in their behavior during these tests presumably reflects retention of learning about the contingencies during phase I training. First, we alternated training periods with extinction periods, during which no shock was delivered. Similar to training, during phase I extinction the control animals exhibited an increase in gill withdrawal (T = 166.5 with no ties; p < 0.001 compared with pretest), and the experimental animals exhibited a significantly greater increase (T = 847.5 with no ties; p < 0.05 compared with control). Second, we gave additional (phase II) training, during which both animals received shock contingent on their own behavior. Again, the control animals exhibited an increase in gill withdrawal during training and extinction (T = 354.5 with one tie; p < 0.001 during phase II extinction), and the experimental animals exhibited significantly greater increases during both periods (T = 795.5 with one tie, p < 0.05 during phase II training; T = 692 with one tie, p < 0.01 during phase II extinction). The differences between the experimental and control animals during these additional tests provide evidence for operant conditioning of gill withdrawal.

The training produced changes in two distinct gill behaviors, either one of which led to a reduction in shocks: there were increases in the frequency and duration of spontaneous contractions, which normally promote respiration (Kupfermann et al., 1974), and also in the tonic level of gill contraction (baseline shift) (Figs. 1B, 3A,B). Furthermore, there was a gradual shift between the first and last extinction periods in the type of behavior that was changed for the experimental and control animals. The control animals exhibited an increase in the total time of spontaneous contractions during the first extinction period (T = 704 with one tie; p < 0.01 compared with pretest) and then returned to near pretest levels by the last extinction period. The experimental animals had a significantly greater increase in the total time of spontaneous contractions during the first extinction period (T = 875 with no ties; p < 0.05 compared with control), as a result of greater increases in both the frequency (average, 118% of pretest for experimental animals and 109% for controls) and average duration (170% for experimental animals and 131% for controls) of the contractions. However, by the last extinction, period there was no difference between the experimental and control groups in spontaneous contractions. Thus, the increase in spontaneous contractions was maximal during the first extinction period for both the experimental and control animals, and the experimental animals showed a larger increase than did the control animals during that period.

Similar to their spontaneous contractions, the baseline level of contraction of the control animals also exhibited a significant increase during the first extinction period (T = 393.5 with eight ties; p < 0.001), which was smaller but still significant during the last extinction period (T = 625.5 with eight ties; p < 0.05). Unlike the spontaneous contractions, the baseline shift exhibited no significant difference between the experimental and control groups during the first extinction period, but it was significantly larger for experimental animals than for control animals during the last extinction period (T = 604.5 with eight ties; p < 0.05). Thus, the increase in baseline for the control animals was maximal during the first extinction period, but the increase in baseline for the experimental animals was maximal during the last extinction period, as was the difference between the two groups.

Discussion

A good deal is known about the cellular mechanisms of an associative form of learning, classical conditioning, as well as nonassociative forms of learning of several invertebrate behaviors (Hawkins et al., 1987; Benjamin et al., 2000; Burrell and Sahley, 2001; Menzel, 2001; Roman and Davis, 2001; Crow, 2004), including the gill- and siphon-withdrawal reflex in Aplysia (Hawkins et al., 1993; Antonov et al., 2001, 2003; Roberts and Glanzman, 2003). In recent years, there also has been progress in understanding mechanisms of another associative form of learning, operant conditioning, in invertebrates (Brembs, 2003), allowing comparisons with mechanisms of both classical conditioning and nonassociative forms of learning. The demonstration of operant conditioning of gill withdrawal in Aplysia should now make it possible to begin to examine the cellular locus and mechanisms of operant conditioning and compare them with those of the other forms of learning of the same behavior. Furthermore, comparisons with similar studies in other systems may also suggest which results are specific to the individual systems and which are more general.

Although there was evidence for operant conditioning in each phase of the experiments, the type of gill behavior that exhibited conditioning shifted over the course of the experiments. There was an associative (experimental vs control) increase in spontaneous contraction during the first but not the last extinction period and an associative increase in baseline contraction during the last but not the first extinction period (Fig. 3A,B). One possible explanation for this shift is that although the change in spontaneous contraction could be acquired more rapidly, it was more costly (perhaps because the spontaneous contractions are also involved in respiration) and was therefore not maintained when the baseline contraction became adequate to avoid shock. The nonassociative (control vs pretest) effect of shock exhibited a different pattern: for both gill behaviors, there were nonassociative increases during the first extinction period, which then declined by the last extinction period, although the control group received contingent training during phase II of the experiment. One possible explanation of that result is latent inhibition of phase II learning for the control animals caused by their noncontingent training in phase I.

The neural circuits mediating both types of gill withdrawal have been well characterized, which should facilitate a cellular analysis of operant conditioning of the behavior. Changes in the tonic level of contraction could be because of changes in the tonic firing of identified gill motor neurons (Kupfermann et al., 1974). Such changes are thought to contribute to operant conditioning of leg position in headless locusts with a learning paradigm similar to ours (Hoyle, 1982). Changes in spontaneous contractions of the gill probably involve changes in the interneuron II network, which controls those contractions (Kupfermann et al., 1974). Changes in the duration of spontaneous contractions may involve changes in the biophysical properties or synaptic connections of the L25 and R25 neurons, which are the major elements of the pattern generator for the contractions (Byrne and Koester, 1978; Byrne, 1983; Koester, 1989), and changes in the frequency of spontaneous contractions could involve changes in the firing or effects of the peptidergic modulatory neurons R15 and R20 (Alevizos et al., 1989, 1991), which produce slow excitation of the L25 and R25 neurons. Such changes in the properties of neurons in a central pattern generator are thought to contribute to operant conditioning of feeding in Aplysia (Nargeot et al., 1999a,b; Brembs et al., 2002) and may also contribute to operant conditioning of aerial respiration in Lymnea (Spencer et al., 1999, 2002; Scheibenstock et al., 2002). Because the gill often appears to contract just before it reaches the criterion level during training (Fig. 1B), conditioning might also involve proprioceptive feedback from the gill.

Both spontaneous (Levy and Susswein, 1990; Levy et al., 1994) and evoked (Carew et al., 1981; Hawkins et al., 1998) gill withdrawal in Aplysia can also undergo classical conditioning. There is as yet no information about mechanisms of classical conditioning of spontaneous contractions, but classical conditioning of evoked contractions involves plasticity at synapses of the siphon sensory neurons caused by activity-dependent presynaptic facilitation and Hebbian long-term potentiation (Hawkins et al., 1993; Antonov et al., 2001, 2003; Roberts and Glanzman, 2003). Activity-dependent presynaptic facilitation is thought to be an elaboration of a mechanism that contributes to a nonassociative form of learning, sensitization (Hawkins et al., 1993), and under some conditions, Hebbian long-term potentiation may be as well (Li et al., 2005). Similarly, the mechanism of operant conditioning of gill withdrawal (the difference between contingent and yoked training) might be an elaboration of mechanisms that contribute to nonassociative effects of the siphon shock (the difference between yoked training and pretest). Consistent with that idea, the time courses of the associative and nonassociative effects of training were similar for percentage of time above criterion (Fig. 2) and total time of spontaneous contractions (Fig. 3A). However, the associative and nonassociative effects had different time courses for baseline shift (Fig. 3B), suggesting that they may involve different mechanisms for that aspect of learning. That difference in time courses also argues against another possible explanation for the associative effect, which is that the shock might have been physically more effective during contingent training because it always occurred when the contingent animals’ (but not the yoked control animals’) mantle organs were relaxed. If so, the associative effect of training would have been expected to be simply an amplification of the nonassociative effect.

Operant and classical conditioning are not likely to involve the same forms of plasticity at synapses of siphon sensory neurons, because those neurons should be silent except during the shock during operant conditioning. However, plasticity in other neurons during operant conditioning could involve a variation of one of the molecular mechanisms that is thought to contribute to classical conditioning: activity-dependent enhancement of facilitation because of a reduction in K⁺ current (Hawkins et al., 1993; Antonov et al., 2001, 2003). If the facilitatory input occurred when a neuron happened to be spontaneously active, that mechanism would also produce an enhanced increase in the firing rate or bursting frequency of the neuron (Hawkins et al., 1983). Such a mechanism is thought to contribute to appetitive operant conditioning of feeding in Aplysia (Brembs et al., 2002). However, for that mechanism to contribute to aversive operant conditioning of gill withdrawal, its sign would have to be reversed, because the experimental animals receive shock when their gills relax and the neurons controlling gill withdrawal are presumably less active. Alternatively, classical and operant conditioning may involve fundamentally different mechanisms. Supporting that idea, genetic analyses in Drosophila suggest that although classical olfactory conditioning and operant place conditioning both involve adenylyl cyclase and phosphodiesterase (Wustmann et al., 1996), ribosomal S6 kinase plays different roles during the two types of learning (Putz et al., 2004). Ideally, the question of whether classical and operant conditioning involve similar mechanisms should be addressed by analyzing both types of learning of the same behavior. This is becoming possible for feeding in Aplysia (Lechner et al., 2000a,b; Mozzachiodi et al., 2003; Brembs et al., 2004; Lorenzetti et al., 2006) and should now be possible for gill withdrawal as well.