Associative Learning in Invertebrates

Abstract

This work reviews research on neural mechanisms of two types of associative learning in the marine mollusk Aplysia, classical conditioning of the gill- and siphon-withdrawal reflex and operant conditioning of feeding behavior. Basic classical conditioning is caused in part by activity-dependent facilitation at sensory neuron–motor neuron (SN–MN) synapses and involves a hybrid combination of activity-dependent presynaptic facilitation and Hebbian potentiation, which are coordinated by trans-synaptic signaling. Classical conditioning also shows several higher-order features, which might be explained by the known circuit connections in Aplysia. Operant conditioning is caused in part by a different type of mechanism, an intrinsic increase in excitability of an identified neuron in the central pattern generator (CPG) for feeding. However, for both classical and operant conditioning, adenylyl cyclase is a molecular site of convergence of the two signals that are associated. Learning in other invertebrate preparations also involves many of the same mechanisms, which may contribute to learning in vertebrates as well.

During associative learning, an animal mentally links two discrete stimuli or events. Much has been learned about the neural mechanisms of associative learning from invertebrates, particularly the marine mollusk Aplysia.

Learning can be divided into two general categories: nonassociative learning, in which an animal learns about the properties or occurrence of a single stimulus (see Byrne and Hawkins 2015), and associative learning, in which an animal learns about the relationship between two stimuli or events. Associative learning includes classical and operant conditioning. During classical or Pavlovian conditioning, one stimulus (the unconditioned stimulus [US], e.g., meat powder) is repeatedly given shortly after another stimulus (the conditioned stimulus [CS], e.g., a bell). Following conditioning, the animal makes a response to the CS (the conditioned response or CR, e.g., salivation) that resembles the response to the US, and it is generally thought to have learned that the CS predicts the occurrence of the US. During operant conditioning, the animal is repeatedly given a reinforcement (e.g., a food pellet) shortly after it shows a particular behavior (e.g., lever pressing). Following appetitive operant conditioning, the animal increases the frequency and vigor of the reinforced behavior, and it is generally thought to have learned that its behavior predicts the occurrence of the reinforcement. Thus, these two types of conditioning can be thought of as prototypes of learning about the regularity and predictability of events in the world.

The properties of classical and operant conditioning have been studied extensively in a wide range of species, and have been found to be basically similar throughout the animal kingdom. Thus, it is reasonable to suppose that at least some of the underlying mechanisms may be similar as well. That idea has encouraged the study of neural mechanisms of learning in relatively simple invertebrate species, which have a number of experimental advantages for such studies (Byrne and Hawkins 2015). In particular, many invertebrate nervous systems have comparatively few neurons, making it easier to elucidate the neural circuits for specific behaviors. Furthermore, some of the neurons are individually identifiable and many are very large, making it much easier to perform experimental manipulations, such as intracellular recording and injections, and to perform assays at the level if individual cells.

This review will focus on two types of associative learning in the marine mollusk Aplysia, classical conditioning of the gill- and siphon-withdrawal reflex, and operant conditioning of feeding behavior. The work will also briefly summarize progress on neural mechanisms of learning in several other invertebrate preparations that have made great contributions to the field as well.

NEURAL AND MOLECULAR MECHANISMS OF CLASSICAL CONDITIONING OF THE GILL- AND SIPHON-WITHDRAWAL REFLEX IN Aplysia

Behavioral Sensitization and Basic Classical Conditioning of the Withdrawal Reflex

The mechanisms of several simple forms of learning have been studied extensively in the marine mollusk Aplysia, which has a number of advantages for a reductionist approach (see Byrne and Hawkins 2015). Although conditioning has been shown for some more complex behaviors (e.g., Walters et al. 1981; Cook and Carew 1986), many of those studies have examined the gill- and siphon-withdrawal reflex, in which a light touch to the siphon (an exhalant funnel for the gill) produces contraction of the gill and siphon. As described in Byrne and Hawkins (2015), a noxious stimulus, such as a shock to the tail, produces an enhancement of subsequent responses to siphon stimulation or sensitization. In addition to this nonassociative form of learning, the reflex undergoes two associative forms of learning, operant conditioning (Hawkins et al. 2006) and classical conditioning, which has been studied much more extensively for this behavior. Classical conditioning resembles sensitization in that the response to stimulation of one pathway is enhanced by activity in another. Typically, in classical conditioning, an initially weak or ineffective CS becomes more effective in producing a behavioral response after it has been paired temporally with a strong US. What distinguishes classical conditioning from sensitization is the requirement for temporal pairing and contingency of the two stimuli during training (Rescorla 1967, 1968; Kamin 1969).

In most experiments on conditioning of the gill- and siphon-withdrawal reflex, the CS is a weak tactile stimulus to the siphon (which initially produces a weak withdrawal response), and the US is an electric shock to the tail (Fig. 1). If these two stimuli are paired for 20–30 trials, the siphon stimulation comes to elicit significantly larger gill and siphon withdrawals than if the two stimuli are presented in an unpaired or random fashion (Carew et al. 1981). This effect builds up during the training session and is retained for several days. The reflex also undergoes differential conditioning with stimulation of the siphon and mantle shelf (a region anterior to the gill), or two different sites on the siphon as the discriminative stimuli (Carew et al. 1983). Significant differential conditioning occurs either 15 min or 24 h after a single training trial, and there is stronger conditioning with five or 15 training trials. In addition, there is reliable conditioning when the CS precedes the US by 0.5 sec (the standard interstimulus interval) but no conditioning when the interval is 2 sec or longer or when the US precedes the CS (Hawkins et al. 1986). These results show stimulus and temporal specificities in conditioning of the reflex.

Figure 1.

Partial circuit diagram for the Aplysia withdrawal reflex. The circuit can account for basic classical conditioning and a number of higher-order features of conditioning. Sensory neurons (SN) that are activated by the siphon conditioned stimulus (CS) make monosynaptic connections onto motor neurons (MN) that produce the gill- and siphon-withdrawal conditioned response (CR) and also onto several interneurons (not shown). The tail shock unconditioned stimulus (US) excites modulatory interneurons including the L29s. In addition, the L29s are excited by the siphon CS and excite the siphon motor neurons, so that they are also excitatory interneurons. Firing of the L29 neurons produces facilitation and activity-dependent facilitation at the SN–MN synapses, and also at the synapses from the SNs onto the L29 neurons themselves. Furthermore, the L29 neurons undergo spike accommodation during prolonged stimulation, caused in part by inhibitory feedback (horizontal bar) from L30 interneurons.

Conditioning of the reflex also shows response specificity. That is, Aplysia learn not only to strengthen the magnitude of a previously existing reflex response, but they also learn to develop a new type of response to the CS that resembles the response to the US (Hawkins et al. 1989; Walters 1989). Thus, siphon stimulation initially produces straight contraction of the siphon, whereas tail shock produces backward bending of the siphon. When a siphon touch CS is paired with a tail shock US, the CS comes to produce backward bending as well. In all of these respects, conditioning of the siphon-withdrawal reflex is similar to many instances of vertebrate conditioning, such as conditioning of the rabbit eye-blink response (Gormezano 1972).

Cellular and Molecular Mechanisms of Sensitization and Basic Classical Conditioning of the Withdrawal Reflex

The neural circuit for the reflex consists in part of monosynaptic connections from siphon sensory neurons (SNs) to gill and siphon motor neurons (MNs), as well as polysynaptic connections involving excitatory and inhibitory interneurons (Fig. 1). The circuit also includes several identified modulatory neurons. A pair of neurons in the cerebral ganglia (the CB1 neurons), and a group of about five neurons in the abdominal ganglion (the L29 neurons) are excited by noxious stimulation, and produce facilitation of siphon SN–MN excitatory postsynaptic potentials (EPSPs) and broadening of action potentials in the SNs (Hawkins 1981; Hawkins et al. 1981b; Mackey et al. 1989). The CB1 neurons contain serotonin (5-HT), two to three of the five L29 neurons express the synthetic enzyme for nitric oxide (NO), and the other L29s may express an endogenous peptide (SCP), all of which contribute to facilitation and behavioral enhancement of the reflex (Abrams et al. 1984; Glanzman et al. 1989; Pieroni and Byrne 1992; Antonov et al. 2007).

It is possible to record the activity of these identified neurons and their synaptic connections during learning in a semi-intact preparation of the siphon withdrawal reflex, and thus to examine the contributions of plasticity at different sites in the circuit to behavioral learning. Such experiments have shown that heterosynaptic facilitation and activity-dependent facilitation at the SN–MN synapses contribute to sensitization and classical conditioning of the reflex, and that plasticity at other sites also contributes (Antonov et al. 1999, 2001). The mechanisms of facilitation at the SN–MN synapses have been examined more extensively in neural analogs of learning in isolated ganglia or in cell culture, in which tail shock is replaced by either nerve shock or application of an endogenous facilitatory transmitter, such as 5-HT, that is released following tail shock. As described in Byrne and Hawkins (2015), short-term facilitation by brief application of 5-HT to rested synapses (an analog of sensitization) involves presynaptic cyclic adenosine monophosphate (cAMP) and protein kinase A (PKA), which produce decreased K⁺ current and increased spike width, Ca²⁺ influx, and transmitter release from the SNs (Fig. 2).

Figure 2.

Cellular and molecular mechanisms of plasticity at sensory neuron (SN)–motor neuron (MN) synapses that contribute to basic classical conditioning. Classical conditioning (CC) involves a presynaptic associative mechanism, activity-dependent enhancement of presynaptic facilitation, which is due in part to Ca²⁺ priming of adenylyl cyclase leading to increased production of cyclic adenosine monophosphate (cAMP) and increased activation of protein kinase A (PKA). Conditioning also involves a postsynaptic associative mechanism, Hebbian potentiation, which is caused by Ca²⁺ influx through N-methyl-d-aspartate (NMDA) receptor channels. These two mechanisms interact through retrograde signaling. In addition, nitric oxide (NO) acts directly in both the sensory and motor neurons to affect different mechanisms of facilitation at the synapses between them. AC, Adenyl cyclase; ER, endoplasmic reticulum; PLC, phospholipase C.

Because of the similarity of sensitization and classical conditioning, it was attractive to believe that conditioning might also involve heterosynaptic facilitation as a mechanism for strengthening the CS pathway. Specifically, the CS and US might converge at the level of individual neurons in the CS pathway, with the US producing greater facilitation of those neurons if they fire action potentials just before the US is delivered (as occurs during conditioning). Consistent with that idea, tail shock produces significantly greater facilitation of the monosynaptic EPSP from an SN to an MN if the shock is preceded by intracellularly produced spike activity in the SN than if it is either unpaired with spike activity or is presented alone (Hawkins et al. 1983; Walters and Byrne 1983). Like behavioral conditioning of siphon withdrawal, this effect builds up during the training session and is retained for at least 24 h (Buonomano and Byrne 1990). Also like behavioral conditioning, 0.5-sec forward pairing of the spike activity and tail shock is more effective than pairing with a longer interval or backward pairing (Clark et al. 1994). Activity-dependent facilitation can thus account for aspects of both the stimulus and temporal specificities of conditioning.

Presynaptic Mechanisms

Activity-dependent facilitation could result from either a presynaptic or a postsynaptic mechanism. Because the facilitation underlying sensitization involves presynaptic mechanisms including broadening of action potentials in the SN, the initial studies of conditioning focused on that mechanism. In neural analogs of conditioning with SN spike activity as the CS and either tail shock or brief application of 5-HT as the US, paired training produced significantly greater broadening of action potentials in the SN than unpaired training (Hawkins et al. 1983; Eliot et al. 1994). Furthermore, forward pairing with 5-HT produced greater broadening than backward pairing (Clark et al. 1994). In experiments in isolated cell culture, paired training with 5-HT as the US also produced greater facilitation of the SN–MN EPSP than unpaired training, but it did not enhance either the frequency or amplitude of spontaneous miniature EPSPs (Eliot et al. 1994; Bao et al. 1998). These results support the idea that pairing selectively affects some aspect of evoked, synchronized release of transmitter, such as presynaptic spike broadening. They also show that activity-dependent facilitation can occur at the level of individual neurons, and does not require additional neuronal circuitry.

Further studies suggested that the influx of Ca²⁺ with each action potential is the critical aspect of spike activity, and that it “primes” the serotonin-sensitive adenylyl cyclase in the SNs so that the cyclase subsequently produces more cAMP in response to serotonin. Consistent with that idea, injection of either the slow Ca²⁺ chelator ethylene glycol tetraacetic acid (EGTA) or a specific inhibitor of PKA into the SN blocks activity-dependent facilitation in culture (Bao et al. 1998). Furthermore, serotonin produces a greater increase in cAMP levels in sensory cells if it is preceded by spike activity in those cells than if it is not (Kandel et al. 1983; Ocorr et al. 1985). Likewise, forward pairing of Ca²⁺ and serotonin produces a greater increase in cyclase activity than backward pairing in a cell-free membrane homogenate preparation (Abrams et al. 1991, 1998; Yovell et al. 1992). These results suggest that one site of convergence of the CS and US during conditioning is the cyclase molecule in the SNs.

Postsynaptic Mechanisms and Trans-Synaptic Signaling

Another mechanism that contributes to classical conditioning is Hebbian potentiation, which is induced by near coincident firing of a SN (excited by the CS) and a MN (excited by the US). The requirement for MN firing could account for the response specificity of conditioning. The conjunction of presynaptic and postsynaptic firing (or depolarization) is necessary for Ca²⁺ influx through postsynaptic N-methyl-d-aspartate (NMDA) receptor channels, which thus serve as another site of convergence of the CS and US during conditioning. The SN–MN EPSPs are glutamatergic and have AMPA- and NMDA-like components (Dale and Kandel 1993; Trudeau and Castellucci 1993; Antonov et al. 2003; Antzoulatos and Byrne 2004), and tetanic stimulation of the SN produces Hebbian potentiation that is blocked by the NMDA antagonist APV, postsynaptic hyperpolarization, or injection of the Ca²⁺ chelator BAPTA into the postsynaptic neuron (Lin and Glanzman 1994a,b). Activity-dependent facilitation in culture (Bao et al. 1998) or the ganglion (Murphy and Glanzman 1996,1997,1999) and pairing-specific facilitation during behavioral conditioning in the semi-intact preparation (Antonov et al. 2003) are also blocked by APV, postsynaptic hyperpolarization, or postsynaptic BAPTA, supporting a role for Hebbian potentiation in conditioning.

However, the facilitation in culture or the semi-intact preparation is also blocked by injection of a specific inhibitor of PKA into the presynaptic neuron (Bao et al. 1998; Antonov et al. 2003). Furthermore, postsynaptic hyperpolarization does not affect either the frequency or amplitude of spontaneous miniature EPSPs in culture, suggesting that it selectively blocks some aspect of evoked, synchronized release of transmitter from the presynaptic neuron, presumably through retrograde signaling (Bao et al. 1998). In support of that idea, postsynaptic BAPTA also blocks PKA-dependent, pairing-specific increases in evoked firing and membrane resistance of the presynaptic neuron during conditioning in the semi-intact preparation (Antonov et al. 2003). These results suggest that the facilitation during conditioning involves a hybrid combination of activity-dependent presynaptic facilitation and Hebbian potentiation, which are coordinated by trans-synaptic signaling (Fig. 2). Thus, facilitation of transmitter release from the SN depends on the combined actions and interactions of 5-HT, NO, SCP, activity, and a retrograde signal from the MN.

Higher-Order Features of Conditioning

In addition to these basic features, classical conditioning in Aplysia and other animals shows higher-order features that have a cognitive flavor and, therefore, may form a bridge to more advanced forms of learning. For example, studies of conditioning in vertebrates have shown that animals learn not only about the temporal pairing or contiguity of events but also about their correlation or contingency, that is, how well one event predicts another. Thus, presentation of extra, unpaired, or unpredicted USs during training, which decreases the degree to which the US is contingent on the CS, decreases conditioning (Rescorla 1968). A similar effect occurs in conditioning of the garden slug, Limax maximus (Sahley et al. 1981) and the siphon withdrawal reflex in Aplysia (Hawkins et al. 1986).

The gill-withdrawal reflex also shows second-order conditioning with two siphon CSs and a mantle shock US (Hawkins et al. 1998). Unlike first-order conditioning, which requires forward pairing of CS1 and the US, second-order conditioning occurs with either forward or simultaneous pairing of CS2 and CS1 in the second stage. Furthermore, following simultaneous second-order conditioning, extinction of CS1 produces a decrease in responding to CS2. That result is formally similar to a posttraining US exposure effect, and suggests that simultaneous second-order conditioning involves formation of a stimulus–stimulus (CS2–CS1) association.

These and other higher-order features of conditioning can be explained by a model based on known molecular mechanisms and circuit connections in Aplysia, in particular those of the L29 interneurons (Fig. 1) (Hawkins and Kandel 1984). The L29 neurons are excited by the CS as well as the US used in behavioral conditioning (Hawkins and Schacher 1989) and excite the siphon MNs, so that in addition to being facilitatory interneurons, the L29 neurons are also excitatory interneurons (SN–L29–MN) in the circuit for the siphon-withdrawal reflex (Fig. 1) (Hawkins et al. 1981a). A key feature of the model is that the L29 neurons produce facilitation and activity-dependent facilitation at all of the synapses of the SNs, including those onto the L29 neurons themselves (Hawkins 1981). In addition, the L29 neurons undergo spike accommodation during prolonged stimulation, caused in part by inhibitory feedback from L30 interneurons (Hawkins et al. 1981a). As a result, the L29 neurons should increase their firing to the CS and decrease their firing to the US during conditioning.

A computational model incorporating these circuit, cellular, and molecular mechanisms is able to simulate most of the known behavioral properties of habituation, dishabituation, sensitization, and basic classical conditioning, including both the stimulus and temporal specificity of conditioning (Hawkins 1989; see also Buonomano et al. 1990). The model provides a rudimentary cellular embodiment of the learning rule of Rescorla and Wagner (1972), and, thus, is also able to simulate the two higher-order features, contingency and second-order conditioning, which have been shown in Aplysia. In addition, the computational model can simulate several other higher-order features that have not yet been shown in Aplysia but have been shown in other invertebrates, including blocking, overshadowing, and CS and US preexposure effects.

Additional higher-order features of conditioning might be explained by two other properties of the circuit and synapses. First, different L29s respond to somewhat different USs (Hawkins and Schacher 1989). That result suggests that firing of an L29 can be considered the internal representation of a US, so that facilitation of SN-L29 synapses could be the basis for learning an association between a CS and the internal representation of the US. That idea is the basis for second-order conditioning in the model, and also suggests a possible mechanism for posttraining US exposure effects, in which exposure to the US after training alters the response to the CS (as happens following simultaneous second-order conditioning in Aplysia). Second, because Hebbian plasticity generally requires the near simultaneous pairing typical of stimulus–stimulus (S–S) learning, the Hebbian component of facilitation might contribute under conditions that are thought to involve S–S learning, such as simultaneous second-order conditioning.

Similar ideas might explain aspects of normal reward function and dysfunction in mammals as well. Dopamine (DA) neurons in the ventral tegmental area (VTA) are thought to mediate reward (Schultz et al. 1997; Tsai et al. 2009), and are thus analogous to the L29 neurons in Aplysia (except that they are generally activated by appetitive USs). The circuit properties of the VTA DA neurons are similar to those of the L29 neurons in Aplysia, and therefore might account for many of the same behavioral features of learning and reward (Hawkins 2013). In particular, they might explain why the VTA DA neurons increase their firing to the CS and decrease their firing to the US during conditioning, so that they come to fire in expectation of reward (Schultz et al. 1997).

NEURAL AND MOLECULAR MECHANISMS OF CLASSICAL CONDITIONING IN OTHER INVERTEBRATE MODEL SYSTEMS

Studies of other invertebrates have shown that classical conditioning involves cellular and molecular mechanisms similar to those in Aplysia as well as additional mechanisms.

Drosophila

The ease with which genetic studies are performed on Drosophila has made it an important system for studying associative learning. A commonly used protocol employs a differential odor–shock avoidance procedure in which animals learn to avoid odors paired (CS⁺) with shock but not odors explicitly unpaired (CS^–). This learning is typically retained for 4–6 h, but retention for 24 h to 1 wk can be produced by a spaced training procedure. Analysis of several Drosophila mutants deficient in learning has revealed that elements of the cAMP signaling pathway are key in learning and memory, similar to classical conditioning in Aplysia. The formation of long-term memory in Drosophila involves stimulation of D1 dopamine receptors, increased cAMP levels, and activation of PKA and cAMP response element-binding protein (CREB) in mushroom bodies and other brain structures that are crucial sites for olfactory learning. The adenylyl cyclase rutabaga acts as a site of convergence for associative learning, and the phosphodiesterase dunce limits the spatial spread of cAMP (Tomchik and Davis 2009; Gervasi et al. 2010; Shuai et al. 2011; Berry et al. 2012; Chen et al. 2012; for reviews of this model system, see Dudai and Tully 2003; Davis 2005; Keene and Waddel 2007; Tomchik and Davis 2013).

Honeybee (Apis mellifera)

Honeybees show classical conditioning of feeding behavior when a visual or olfactory CS is paired with application of sugar solution (US) to the antennae. Several regions of the brain necessary for this associative learning have been identified, including the antennal lobes and mushroom bodies. The different regions are thought to contribute to different aspects of a distributed engram. In addition, intracellular recordings have revealed that one identified cell that is thought to be octopaminergic, the ventral unpaired median (VUM) neuron, mediates reinforcement during olfactory conditioning and represents the neural correlate of the US. The learning has been dissected into several phases of memory, including short term, midterm, and long term. Numerous studies have revealed that, as in other species, the molecular mechanisms underlying memory formation in the honeybee involve up-regulation of the cAMP pathway and activation of PKA. Short-term memory involves brief activation of PKA, whereas long-term memory involves NO-cGMP signaling leading to longer activation of PKA and CREB-mediated transcription of downstream genes. In addition, midterm memory involves calpain-dependent cleavage of PKC to form PKM (for comprehensive reviews of this model system, see Menzel 2001, 2013; Menzel et al. 2006; Menzel and Benjamin 2013; Muller 2013).

Hermissenda

The gastropod mollusk Hermissenda shows associative learning of light-elicited locomotion and changes in foot length (CRs). The conditioning procedure consists of pairing visual stimuli (light, the CS) with vestibular stimuli (high-speed rotation, the US). After conditioning, the CS suppresses normal light-elicited locomotion and elicits foot shortening (Crow and Alkon 1978; Lederhendler et al. 1986). The associative memory can be retained from days to weeks depending on the number of conditioning trials administered during initial acquisition. The type A and B photoreceptors and interneurons in the CS pathway have been identified as critical sites of plasticity for associative learning. The initial studies focused on the B photoreceptors, which show an increase in excitability caused by a decrease in K⁺ current following training (Alkon et al. 1982). That change has been attributed to increased Ca²⁺ activating CMKII (Alkon 1984), and also 5-HT activating PKC (Farley and Auerbach 1986) and possibly PKA (Alkon et al. 1983) or MAPK (Crow et al. 1998). Subsequent studies have shown that long-term retention depends on both protein and RNA synthesis (Crow and Forrester 1990; Crow et al. 1997). In addition, training produces changes in the membrane properties of the interneurons and changes in the strength of the synaptic connections between the SNs and the interneurons (reviewed in Crow and Jin 2013).

Pond Snail (Lymnaea stagnalis)

The pulmonate Lymnaea stagnalis shows appetitive conditioning of feeding behavior when a neutral chemical or mechanical stimulus (CS) applied to the lips is paired with a strong stimulant of feeding, such as sucrose (US). Greater levels of rasping, a component of the feeding behavior, can be produced by a single trial, and this response can persist for at least 19 d. The early stages of learning involve PKA, MAPK, and CMKII, whereas long-term maintenance involves protein and RNA synthesis, CREB-dependent gene regulation, and up-regulation of NO. The circuit consists of command-like interneurons, a network of three types of central pattern generator (CPG) neurons, 10 types of MNs, and a variety of modulatory interneurons. An analog of the behavioral response occurs in the isolated central nervous system. The enhancement of the feeding motor program appears to be caused by facilitation of input to the interneurons, MNs, and presumably the CPG neurons, as well as persistent depolarization of identified modulatory neurons, resulting in increased activation of the CPG cells by mechanosensory inputs from the lips (reviewed in Kemenes 2013).

Limax

The pulmonate Limax shows food avoidance learning when a preferred food odor (CS) is paired with a bitter taste (US). In addition to this example of basic classical conditioning, food avoidance in Limax shows higher-order features of classical conditioning including blocking and second-order conditioning. An analog of the learning occurs in the isolated central nervous system, facilitating subsequent cellular analyses of learning in Limax. The procerebral lobe in the cerebral ganglion processes olfactory information, and is a likely site for plasticity. Both neurogenesis and oscillations of local field potentials in the procerebral lobe are thought to be involved in learning and memory. The oscillatory activity of the procerebral lobe is modulated by several endogenous substances including the gaseous transmitter NO and the neuropeptide FMRFamide. FMRFamide and another neuropeptide found in the procerebral lobe (SCPB) have also been shown to affect activity of the rhythmic motor network underlying feeding behavior, making those neuropeptides attractive candidates for plasticity within that network as well (see Gelperin 2013 for a comprehensive review of this model system).

NEURAL AND MOLECULAR MECHANISMS OF CLASSICAL AND OPERANT CONDITIONING OF FEEDING BEHAVIOR IN Aplysia

The study of feeding behavior of Aplysia has provided insights into the mechanisms underlying classical conditioning and also operant conditioning, which has been studied more extensively for this behavior. Feeding behavior in Aplysia shows several features that make it amenable to the study of learning. For example, the behavior occurs in an all-or-nothing manner and is therefore easily quantified, and the CPG underlying the generation of the behavior is well characterized to the extent that many of the key individual neurons responsible for the generation of feeding movements have been identified. These technical advantages have been exploited to identify loci of plasticity and changes in membrane properties in the key neurons of the CPG that occur during associative learning.

In the first example of conditioning of feeding behavior in Aplysia, Susswein and colleagues (Susswein and Schwartz 1983; Susswein et al. 1986) developed a training procedure in which animals were presented with food (i.e., seaweed) that was made inedible by wrapping it in a plastic net. Wrapped food still elicits bites and is initially brought into the mouth and buccal cavity. However, because netted food cannot be swallowed, it triggers repetitive failed swallowing responses and is eventually rejected. Additional behavioral features of this conditioning have been elucidated (Schwarz et al. 1991; Botzer et al. 1998; Katzoff et al. 2002, 2010), as well as some of the underlying biochemical and molecular mechanisms (Cohen-Armon et al. 2004; Levitan et al. 2008; Michel et al. 2011, 2012).

Feeding behavior can also be modified by appetitive associative paradigms (i.e., classical and operant conditioning), which induce an increase of its expression (Lechner et al. 2000a; Brembs et al. 2002). During classical conditioning, tactile stimulation of the lips with a soft paintbrush serves as the CS and seaweed presentation serves as the US. Paired training produces a significant increase in the number of CS-evoked bites, compared with unpaired training, both 60 min and 24 h after training (Colwill et al. 1997; Lechner et al. 2000a,b; Lorenzetti et al. 2006). Further analyses found that afferent information related to the US is mediated by an anterior branch of the esophageal nerve En2 (Brembs et al. 2002), which projects to the foregut (Lechner et al. 2000a). This nerve is rich in dopamine-containing processes. Neural correlates of classical conditioning were identified by removing the buccal ganglia, where the CPG underlying feeding behavior is located, from recently trained animals and examining the change in cellular properties of key cells that mediate the behavior. Classical conditioning produced an increase in the CS-evoked excitatory synaptic drive to pattern-initiating neuron B31/32. In addition, training led to a change in the burst threshold of decision-making neuron B51 (Nargeot et al. 1999).

Appetitive Operant Conditioning of Feeding

The finding that En2 was a reinforcement pathway facilitated the development of an appetitive operant conditioning behavioral protocol in which biting served as the operant, and electrical stimulation of En2 served as reinforcement (Brembs et al. 2002). During contingent training, bites were immediately followed by reinforcements (Fig. 3A), which were delivered via stimulating electrodes implanted on En2. During a 10-min training period, the En was stimulated each time the animal performed a spontaneous biting movement (contingent reinforcement) (Fig. 3B) (Brembs et al. 2002). Control animals received the same number of stimulations over the 10-min period; however, they were explicitly unpaired (i.e., yoked). The group of animals that had received paired training showed a significantly larger number of spontaneous bites during test periods both 1 and 24 h after training compared with control animals.

Figure 3.

Operant conditioning of feeding behavior. (A) Throughout the experiment, the animal was observed and all bites were recorded. In the contingent reinforcement group, a bite was immediately followed by a brief electric stimulation of the esophageal nerve (En stim.). A control group received the same sequence of stimulations as the contingent group, but the stimulation was uncorrelated with the animal’s behavior. (B) Experimental sessions consisted of a 5-min pretest, 10 min of training, and a final test period. In each period, the number of bites was recorded. The final test period was either 1 h or 24 h after training. The group of animals that had received contingent reinforcement showed a significantly larger number of spontaneous bites both 1 h and 24 h after training compared with control animals. (Modified from Brembs et al. 2002.)

An analysis of neuronal correlates of the conditioning revealed that the contingent-dependent increase in bites (i.e., ingestive buccal motor programs [iBMPs]) was associated with the regularization of the bursting activity of a cluster of pattern-initiating neurons, consisting of B30, B63, and B65 (Fig. 4A) (Nargeot et al. 2009). This synchronization of the pattern-initiating neurons appears to be produced by two distinct contingent-dependent mechanisms: (1) decreased burst threshold in B63, B30, and B65; and (2) enhanced electrical coupling between pairs of pattern-initiating neurons (Nargeot et al. 2009). In addition to the changes in the pattern-initiating neurons, operant conditioning is associated with modifications of the input resistance and burst threshold of B51 (Brembs et al. 2002), a neuron that is also modified by classical conditioning, but in opposite ways (Lorenzetti et al. 2006). The input resistance was significantly greater, and the burst threshold was significantly lower in neurons from trained animals compared with untrained animals. These types of changes would serve to increase the probability that B51 would become active and would, therefore, facilitate the generation of the neural activity underlying biting movements in the trained animals.

Figure 4.

Neuronal correlates of operant conditioning and mechanisms in neuron B51. (A) Simplified schematic of the feeding neural circuit. Sensory neurons (SNs) in the cerebral ganglia (interganglionic cerebral-buccal mechanoafferent [ICBM] and cerebral mechanoafferent [CM]) convey information to higher-order cells in the feeding circuit, such as the command-like cerebral-buccal interneurons (CBIs), and modulatory cells, such as the metacerebral cell (MCC), also located in the cerebral ganglia. Sensory information is also conveyed directly to the central pattern generator (CPG) in the buccal ganglia. In addition, the CPG receives inputs from the command-like cells and modulatory cells. Cells in the CPG can be classified, in part, by their activity during a buccal motor program (BMP). Some cells are active during the protraction phase (yellow shading), whereas others are active during the retraction phase (blue shading). Activity in radula closure motor neuron (MN) B8 occurs during the protraction phase in rejection BMPs (rBMPs) and during the retraction phase in ingestive BMPs (iBMPs). Identified loci of plasticity are indicated by red and white shading. Following operant conditioning, the electrical coupling among B30, B63, and B65 is strengthened, and the excitability of B51, B30, B63, and B65 is increased. (B) Model of the molecular mechanisms in B51 underlying operant conditioning. See text for details. (Modified from Lorenzetti et al. 2008.)

To examine whether the changes in the properties of B51 are intrinsic to that cell, Brembs et al. (2002) used an analogue of operant conditioning in which an individual B51 cell was removed from a naïve ganglion and maintained in culture. Reinforcement was mimicked by the application of a brief “puff” of DA onto the cell, which was made contingent on a plateau potential that was elicited in B51 by injection of a brief depolarizing current pulse. Controls consisted of cells that received the DA puff 40 sec after the plateau potential. Training produced a significant increase in input resistance and a significant decrease in burst threshold, similar to the changes observed in the neural correlates (Brembs et al. 2002) and in vitro analogues (Nargeot et al. 1999) of operant conditioning. These data suggest that B51 is an important locus of plasticity in operant conditioning of feeding behavior, and that intrinsic cell-wide plasticity may be one important mechanism underlying this type of learning.

A model of the molecular mechanisms underlying appetitive operant conditioning in neuron B51 is shown in Figure 4B. The CPG that mediates feeding behavior produces synaptic input to neuron B51. When this input is suprathreshold, it triggers an all-or-nothing sustained several-second burst of spikes (plateau potential) in B51, which is critical for the expression of ingestive behavior (i.e., an iBMP). A secondary consequence of the plateau potential is to produce an accumulation of Ca²⁺ in B51, which leads to the activation of PKC. The activated PKC then weakly activates and primes a type II adenylyl cyclase. Reinforcement (reward) activates the dopaminergic modulatory system. DA binds to a D1-like receptor, but the DA-induced activation of the cAMP cascade is weak and insufficient to modulate downstream effectors (e.g., membrane channels regulating input resistance and burst threshold). However, if the ingestive behavior just precedes the delivery of the reward, as occurs during operant conditioning, the adenylyl cyclase will have been primed by PKC, and because of the synergistic interaction of the two pathways, the level of cAMP will be significantly greater than that produced by either behavior (activity in B51) alone or reinforcement (DA) alone. After a sufficient number of contingent reinforcements, the increased level of cAMP activates PKA sufficiently to produce the increase in input resistance and excitability of B51. Consequently, subsequent CPG-driven synaptic input to B51 is more likely to fire the cell and lead to the increase in ingestive behavior associated with operant conditioning.

Interestingly, the mechanisms of activity-dependent neuromodulation for this appetitive form of operant conditioning appear to be very similar to those observed in SNs of Aplysia during aversive classical conditioning of withdrawal reflexes (Fig. 2). In the SNs, the coincidence detection involves, at least in part, a synergistic interaction between a Ca²⁺/calmodulin-sensitive adenylyl cyclase (type I) and a serotonin-activated cAMP cascade (Ocorr et al. 1985; Abrams et al. 1991, 1998; Yovell et al. 1992). Similarly, in Drosophila, a type I adenylyl cyclase is necessary for classical conditioning, but does not appear to be necessary for operant conditioning (Brembs and Plendl 2008). Although the specific isoform of adenylyl cyclase appears to differ (type I for classical conditioning and type II for operant conditioning), adenylyl cyclase appears to serve as a molecular site of convergence in both forms of learning. However, adenylyl cyclase is not the only coincidence detector for all examples of classical and operant conditioning. For example, following classical conditioning of feeding behavior, the burst threshold of B51 increases rather than decreases as it does following operant conditioning and the input resistance does not change (Lorenzetti et al. 2006), indicating that the mechanisms underlying the modulation of B51 by classical conditioning involve a different as-yet-unidentified coincidence detector from that for operant conditioning. In addition, as described above for conditioning of the withdrawal reflex in Aplysia and in Basu and Siegelbaum (2015), NMDA receptors also serve as coincidence detectors in many CNS circuits.

Operant conditioning has also been examined in other invertebrates including Drosophila and Lymnea. PKC appears to play a key role in operant conditioning of turning behavior in Drosophila (Brembs and Plendl 2008) as it does in conditioning of feeding behavior in Aplysia (Fig. 4). Operant conditioning of aerial respiratory behavior in Lymnea (Lukowiak et al. 1996) also involves PKC as well as NMDA receptors and MAPK (Rosenegger and Lukowiak 2010), and long-term memory involves protein and RNA synthesis as well as changes in the activity of an identified neuron in the CPG, as in Aplysia (Spencer et al. 2002; Lukowiak et al. 2003).

Growing evidence indicates that many of the same pathways that mediate operant conditioning of feeding in Aplysia are also involved in vertebrate reward learning in the striatum (see also Graybiel 2015). For example, in vivo operant conditioning involves D1 dopamine receptors and cAMP/PKA in the nucleus accumbens (Smith-Roe and Kelley 2000; Baldwin et al. 2002). D1 dopamine receptors are also necessary for potentiation of corticostriatal synapses in an analogue of reward learning (Reynolds et al. 2001). In addition to synaptic plasticity, striatal neurons display an increased level of intrinsic excitability known as the “up state” that depends on expression of the CREB transcription factor (Dong et al. 2006).

SUMMARY AND CONCLUSIONS

In this work, we have reviewed research on neural mechanisms of classical conditioning of the gill- and siphon-withdrawal reflex and operant conditioning of feeding behavior in Aplysia, and have briefly summarized progress in a few other important invertebrate preparations. It is instructive to compare and contrast mechanisms that have been described in these different preparations to try to determine to what extent they may be universal or unique:

• Classical conditioning of the withdrawal reflex and operant conditioning of feeding in Aplysia both involve changes in the biophysical properties of identified neurons in the circuits for the behaviors.

• In both cases, these changes include an increase in excitability that could be caused by a decrease in a K⁺ current. Classical conditioning in Hermissenda also involves an increase in excitability that is thought to be caused by a decrease in K⁺ current in identified cells.

• In addition, classical conditioning in Aplysia involves changes in the strength of excitatory synapses in the circuit. Synaptic plasticity is also thought to be involved in conditioning in Hermissenda and Lymnea.

• The changes in excitability and synaptic strength in Aplysia can both be reproduced in analogs of learning in single cells (or pairs of cells) in culture and, therefore, do not require additional circuitry.

• Modulatory transmitters carry the reinforcing signals—5-HT for classical conditioning and DA for operant conditioning in Aplysia. 5-HT, DA, and octopamine are also important for conditioning in Drosophila, Apis, and Hermissenda. In addition, NO is important for classical conditioning in Aplysia as well as conditioning in Apis, Lymnea, and Limax.

• For both classical and operant conditioning in Aplysia, adenylyl cyclase (AC) is a molecular site of convergence of the two signals that are associated. For classical conditioning, Ca²⁺ acts synergistically with 5-HT to activate type I AC, whereas for operant conditioning, Ca²⁺ stimulates PKC, which acts synergistically with DA to activate type II AC. Type I AC is also a molecular site of convergence for classical conditioning in Drosophila.

• For both types of learning in Aplysia, the AC-cAMP-PKA pathway plays an important role in the initial stages of memory. This is true for conditioning in Drosophila, Apis, and Lymnea as well. Other kinases including PKC, MAPK, and CMKII are also important for conditioning in Drosophila, Apis, Hermissenda, and Lymnea.

• Long-term maintenance of conditioning involves PKA, CREB, and protein and RNA synthesis in Drosophila, Apis, Hermissenda, and Lymnea. This has not yet been tested for the two forms of conditioning in Aplysia reviewed here, but it is true for nonassociative learning of the same behaviors in Aplysia (Byrne and Hawkins 2015).

• Classical conditioning in Aplysia also involves NMDA receptors as another molecular site of convergence of the CS and US pathways. Furthermore, the two sites of convergence (presynaptic AC and postsynaptic NMDA receptors) act cooperatively through retrograde signaling. Operant conditioning in Lymnea also involves NMDA receptors.

• In addition, classical conditioning in Aplysia has higher-order features that might be explained at the neural circuit level. Classical conditioning in Limax shows similar features.

Thus, the neural mechanisms of different types of associative learning in these different invertebrate preparations have many shared features, but they also have some unique features that may be adapted to the particular type of learning or species. As we have noted, initial studies suggest that associative learning in vertebrates can share some of the same behavioral, circuit, cellular, and molecular mechanisms as well. Therefore, important future goals will be to decipher the logic of the shared and unique features in invertebrates, and to use the results of these studies to suggest new directions for research on mechanisms of learning in vertebrates.