Abstract
Associative learning in goal-directed behaviors, in contrast to reflexive behaviors, can alter processes of decision-making in the selection of appropriate action and its initiation, thereby enabling animals, including humans, to gain a predictive understanding of their external environment. In the mollusc Aplysia, recent studies on appetitive operant conditioning in which the animal learns about the positive consequences of its behavior have provided insights into this form of associative learning which, although ubiquitous, remains mechanistically poorly understood. The findings support increasing evidence that central circuit- and cell-wide sites other than chemical synaptic connections, including electrical coupling and membrane conductances controlling intrinsic neuronal excitability and underlying voltage-dependent plateauing or oscillatory mechanisms, may serve as the neural substrates for behavioral plasticity resulting from operant conditioning. Aplysia therefore continues to provide a model system for understanding learning and memory formation that enables establishing the neurobiological links between behavioral, network, and cellular levels of analysis.
Keywords: Aplysia, Feeding, Operant conditioning, Learning, Central pattern generator, Endogenous bursting, Synaptic plasticity
Introduction
Numerous studies have addressed the neural origins of the generation, organization, and adaptability of behavioral actions. Two general mechanisms can be recognized to underlie the production of behavior: reflexive processes, whereby the expression of an act is the reactionary motor response to a triggering sensory stimulus arising from the periphery (Fig. 1a), and autonomous mechanisms, in which the basic determinants of a behavior arise from motor output that is produced by pattern-generating circuitry within the central nervous system (Fig. 1b). From these studies on a variety of animal model systems, particularly the simpler and more accessible behaviors and nervous systems of invertebrates, significant progress has been made in understanding the synaptic and cellular properties of the underlying neural pathways and processes of learning and memory by which reflexively driven acts can be dynamically adjusted in an history-dependent manner in accordance with an animal’s changing external environment. For example, much is known about the neural control of reflexes such as those involved in defensive gill, siphon, and tail withdrawal in the mollusc Aplysia, and the cellular and molecular basis of the ability of the underlying sensori-motor pathways to adapt through simple forms of non-associative (habituation and sensitization) and associative learning (classical and operant conditioning) [for reviews see 1–3].
Unlike the sensori-motor transformations underlying reflexive behavior in which the rate of occurrence is a function of that of the eliciting stimulus, the internally driven motor commands responsible for motivated or goal-directed acts, such as feeding or sexual behavior, which are emitted spontaneously at a rate set by the animal itself [4], depend on processes of decision-making in the selection and initiation of appropriate actions. Such autonomous behaviors can also be modified by sensory information through non-associative and associative learning, although the functional interaction between external stimuli and internal decision-making circuitry remains poorly understood. A relevant learning paradigm to address this issue is operant conditioning in which an animal learns about the consequences of its goal-directed behavior. In operant conditioning, a contingent association is made between a specific act (the operant) and a reinforcing (rewarding) or an aversive (punishing) stimulus [5, 6]. As a result of positive reinforcement, operant conditioning reduces variability in internally driven behavior not only by favoring appropriate (rewarded) action but also by regulating its spontaneous recurrence in time [7–11]. This can result in a habitual and ultimately compulsive expression of the rewarded action that becomes initiated in a stereotyped rhythmic manner at a relatively high frequency, even after the rewarding stimulus has been removed [11–16]. Conversely, in an operant-negative reinforcement relationship, an animal learns through unfavorable behavioral consequences to diminish the spontaneous expression of a particular act, thereby decreasing the likelihood of a future confrontation with the unfavorable stimulus [4]. Such behavioral changes in turn necessitate alterations in the central neuronal processes engaged in motor pattern selection and initiation [17]. However, the cellular mechanisms that underlie the operant reward-induced acquisition of compulsive behavior from an otherwise impulsively expressed act, or the opposing relationship between reward versus aversive associative learning, are still largely unknown.
Among a variety of invertebrates that express memory formation through operant conditioning [7, 8, 18–20, for review see 10], the feeding behavior of Aplysia has provided important insights into the underlying cellular and network mechanisms, due in large part to parallel studies in which the key neuronal components of the pattern-generating networks that drive feeding-related movements have been identified and characterized [21–28, for review see 29]. Thus, by using in vivo operant conditioning paradigms in combination with in vitro approaches, it has been possible in some cases to pinpoint the sites and actions of reinforcing sensory signaling on known target neurons and networks and in turn relate these specific influences to the outcome for the actual behavior that the circuits produce. The purpose of this review is to highlight recent investigations on Aplysia in which learning-induced plasticity in its feeding behavior, particularly that related to appetitive operant conditioning, has been able to be traced to the biophysical properties and synaptic interactions of identified components within the central pattern-generating circuitry that produces the behavior. The data obtained not only demonstrate the involvement of operant learning processes in exploratory and goal-directed behaviors in which decision-making about action selection and initiation is pivotal, but also that memory storage resulting from the convergence of operant behavior and reinforcement can occur at diverse cell- and circuit-wide loci.
Operant reward learning in Aplysia’s feeding behavior
The herbivorous Aplysia (Fig. 1a) performs a variety of actions during its search for food, including locomotion, head-waving and cycles of protraction and retraction of its rasp-like radula [30]. Repeated exploratory and consummatory radula biting movements are expressed in an all-or-none manner, rendering them easily observable and quantifiable. Moreover, this goal-directed behavior, which in the absence of ingestible food is expressed relatively infrequently with highly variable and unpredictable inter-bite intervals (Fig. 2ai), can be durably modified by action-reward contingency in appetitive operant conditioning [31]. After 40 min of contingent reinforcement training during which an animal received an ingestible food reward (either a piece of algae or a bolus of seaweed extract injected directly into the buccal cavity) in association with each spontaneous bite (Fig. 2aii), the variability and cycle period of bite occurrences both drastically reduced, leading to prolonged (for several hours following training) bouts of intense and seemingly automatic radula movement cycles with an elevated frequency and a stereotyped rhythmic organization (Fig. 2b, middle). A strict contingence of the appetitive reward with the intrinsically driven motor action was found to be both necessary and sufficient for the induction of this behavioral plasticity, since changes in radula movements did not develop with the presentation of an unpalatable food stimulus or when repetitive delivery of the seaweed reward was uncorrelated (i.e., non-contingent) with biting behavior (Fig. 2aiii, b, right). Thus, the long-lasting plasticity in the cycle-by-cycle initiation and temporal organization of a goal-directed component of Aplysia’s feeding behavior emerged from a specific associative and operant process through which the animal was able to “memorize” the positive consequences (access to food) of its own actions.
It is interesting, moreover, that this operant learning-dependent and persistent increase in the motivational state of the animal, in which an appetitive action switches from erratic, impulsive and irregular recurrences to an accelerated and stereotyped yet ultimately unsuccessful behavior (when the reward is withdrawn), bears a striking resemblance to the induction of non-pathological compulsive actions in more complex organisms, including humans. Here also, the internally driven impulse to produce a specific goal-directed act, such as feeding, sexual activity or drug taking, can be regulated by external rewarding stimuli that through contingent reinforcement in operant conditioning can lead to accelerated and quasi-automatic, compulsive-like recurrences of the action which persevere after reward stimulus withdrawal [11, 13–16]. Thus, the contingent reward-induced modifications observed in the simpler Aplysia’s feeding behavior could provide relevant insights into the neuronal basis of non-pathological compulsive actions resulting from rewarding feedback in other more complex organisms, as well as in drug addiction and compulsive eating disorders in humans.
Operant learning that food is inedible
In direct contrast to the positive reinforcement of spontaneous ingestive behavior through appetitive operant conditioning, an Aplysia can also learn to modify its responsiveness to food on the basis of whether it succeeds in actually swallowing it [8]. In an associative learning paradigm of aversive operant conditioning, animals can be trained to attempt to swallow a food enclosed in plastic netting. The food is tasted through holes in the net, inducing ingestive biting, food entry into the buccal cavity, and repetitive but failed attempts to swallow. The inedible food is eventually ejected through radula movements in a sequence that pushes the net-enclosed bolus out of the mouth, rather than pulling it in (see below). The food continues to stimulate the lips, eliciting ingestive radula bites that again lead to failed swallows. The ineffective attempts to swallow act as negative reinforcement so that as training proceeds, the netted food remains in the buccal cavity for shorter periods, inducing fewer swallows and more rejections until eventually the animal stops responding as it learns that the food is inedible [8]. Inedible food training for periods as short as 5–15 min causes a subsequent diminished behavioral responsiveness that persists for >24 h [32], and as for long-term memory formation in other systems [33, 34], this learning-induced response decrement was found to be correlated with an increased expression of the transcription factor CCAAT/enhancer-binding protein (C/EBP) that was specifically localized to the nervous centers controlling feeding [35].
Cellular and network mechanisms of operant learning
The central pattern generator (CPG) network that drives Aplysia’s feeding-related behavior, including radula biting movements, is distributed between the bilaterally paired buccal ganglia and many of the key network components, their bioelectrical properties and synaptic connectivity have been previously identified and characterized ([21–28, for review see 29]; Fig. 3a, b). Two operational features of the buccal CPG have been crucial to pinpointing the neuronal substrates of behavioral plasticity in this system. First the feeding neural circuitry, consisting of sensory input pathways, interneurons and motoneurons, continues to produce the motor output patterns for radula behavioral actions when the buccal ganglia are removed from the animal (Fig. 3a, upper panel). Secondly, this so-called “fictive” biting expressed by the motor pattern-generating circuitry of isolated buccal ganglia conserves essential features of the behavioral modifications induced in vivo by food-related classical or operant conditioning [8, 31, 36, 37] or via analogous paradigms of sensory nerve stimulation in vitro [38–45].
The motor output patterns that drive radula movements during feeding consist of all-or-none sequences of burst activity in motor nerves controlling radula protraction, retraction, opening and closure (Fig. 3a, lower panel). In the intact animal, depending on the quality of sampled food, the buccal CPG is able to alter the phase relationship between the two subsets of alternating protraction/retraction and opening/closing activity so that radula movements switch from normal ingestive biting that transports food through the buccal mass into the foregut to egestive actions that remove inedible objects from the foregut [8, 46]. During ingestion, the two radula halves are protracted out of the buccal cavity to close around food which is then withdrawn into the cavity during retraction. Alternatively, the radula can close on an inedible element in the buccal cavity and eject it by concomitant radula protraction. Thus, for an ingestive bite, radula closure occurs during the retraction phase, whereas in egestion, closure movement now occurs during radula protraction. On this basis, therefore, the feeding-related actions of Aplysia correspond to two fundamental components of behavioral choice and decision-making that are confronted by simpler [47, 48] and complex animals alike [49]: the decision as to whether or not to repeat a given act (e.g., an ingestive radula bite) and the selection between alternative behaviors (e.g., ingestion vs. egestion movements).
Cell-wide plasticity in operant conditioning of action selection
A key “decision-maker” for action selection in Aplysia’s feeding behavior is an identified element (neuron B51) of the buccal CPG (Fig. 3b) which, depending upon its inherent levels of excitability, can transfer the network between functional states of ingestive or egestive motor program production (Fig. 3b, lower; [39]). Specifically, the intrinsic membrane properties of B51 that include an ability to produce sustained, regenerative plateau potentials ([23]; see Fig. 5a) allows it to switch in an all-or-none manner between an inactive and active state during which the cell fires an intense burst of impulses. B51, which is not essential for the actual genesis of buccal motor patterns, exerts its effects through diverse synaptic connections, including mixed electrical and chemical excitatory synapses with radula closure motoneurons and with the B64 neuron, a major component of the CPG subcircuit that drives retraction phase motor output (see Fig. 3b). Thus, when B51 is in an active state (Fig. 3c, left), it reinforces and prolongs discharge in the B64 retraction generator, which additionally inhibits the protraction generator subcircuit (see Fig. 3b), and thereby ensures that closure motor activity occurs mainly during retraction. When B51 remains inactive (Fig. 3c, right), closure motoneurons are activated principally by central neurons belonging to the protraction generator (e.g., B65; see Fig. 3b), which in turn inhibits B64 and now leads to closure and protraction motor discharge occurring concomitantly. Therefore, although B51 is not involved in the cycle-by-cycle initiation of buccal motor programs, a dynamic switching between the cell’s two functional states is critical for determining the motor output phenotype that the buccal CPG produces: plateau-generated bursts in B51 promote the occurrence of ingestion-related motor patterns, whereas an absence of B51 activation represses ingestion patterns and promotes the expression of egestion-related patterns (Fig. 4ai) [40].
The excitability-dependent ability of the single B51 neuron to “decide” between the two feeding-related motor programs can be modified by operant conditioning and associative learning processes. In initial in vitro and subsequent in vivo conditioning experiments, direct electrical stimulation of an esophageal nerve branch (En2), which is presumed to convey sensory information about the presence of ingested food, was used to substitute contingent food reward reinforcement during actual feeding behavior [36, 39, 40]. Both operant training paradigms, in which En2 stimulation was made conditional upon each spontaneous radula motor pattern produced by the isolated buccal CPG [39, 40] or an actual radula bite in the freely behaving animal (see Fig. 2aii; [36]), produced a long-lasting (several hours) rate enhancement of ingestion-related biting activity (Fig. 4aii) that was associated with an increased input resistance and a decrease in burst plateau threshold of neuron B51 (Fig. 5aii). Stimulation of En2 in a non-contingent manner, either in vivo or in vitro, did not alter the membrane properties of B51 (Fig. 4aiii, 5aiii). The operant conditioning-induced cellular changes in B51, which together increased its excitability and hence the probability of the cell becoming active, thus increased the likelihood of the buccal CPG to produce ingestive rather than egestive motor patterns during its reinforced operation (Fig. 4aii). Moreover, the learning-induced excitability changes were intrinsic to B51 (rather than originating at a presynaptic locus) since they were also expressed in an isolated cell analogue of appetitive operant conditioning in which spontaneous plateau potentials in cultured B51 cells were contingently reinforced with iontophoretically applied pulses of dopamine [36, 45], the transmitter that is likely to mediate food signals from the esophageal input nerve in vivo [50]. Furthermore, injection of cAMP into naive isolated B51 neurons mimicked the excitability changes induced by single cell operant conditioning [44, 45], and consistent with this finding, cAMP injected into B51 in situ also increased bursting activity in the cell and in turn the number of ingestion-type motor patterns produced by the buccal CPG [45]. Such findings, together with evidence for an associative convergence (upstream of cAMP production and acting through adenylyl cyclase) of activity- and dopamine receptor-induced signaling cascades involving PKC and PKA, respectively [44], are therefore beginning to pinpoint the molecular processes by which memory storage for operant learning can be inscribed within the bioelectrical properties of key decision-making components of behavior-generating central circuitry [51].
Finally, it remains to be established whether B51, in light of its capacity to determine the type of motor output that the buccal CPG produces, also serves as a pivotal decision-maker in the opposing behavioral response to aversive operant learning that food is inedible [8; also see above]. In this case, the pairing of a punishing esophageal sensory nerve signal to failed attempts to consume food contained in the buccal cavity [52] could actively lead to a decreased excitability of B51, thereby promoting the production of egestive rather than ingestive motor programs as buccal network output declines until the animal eventually ceases responding to the inedible food.
Cell- and circuit-wide plasticity in operant conditioning of action initiation
Although considerable data are now available on the impact and consequences of operant conditioning in terms of the pattern-switching B51’s ability to determine how the buccal CPG should act once a feeding cycle has started, until recently, much less was known about learning-induced changes at the level of pattern-initiating neurons, the buccal circuit cells whose activity determines when a behavioral cycle is actually instigated. Since a natural food stimulus in operant conditioning increased the regularity and frequency of ingestive bite occurrences in vivo (see above), and for which the neuronal correlates continued to be expressed by feeding circuitry in isolated buccal ganglia [38], a first step towards a cellular analysis of this behavioral plasticity was to identify the buccal network elements that are primarily responsible for triggering the initial protraction phase of each radula bite cycle. Intracellular recordings made from a variety of previously identified protraction-initiating neurons [22, 25–28] revealed that spontaneous bursts in three cell types—the bilaterally paired and electrically coupled B63, B30 and B65 interneurons (see Fig. 3b)—always started before each spontaneous radula motor pattern occurrence (see Fig. 4b), and were therefore considered to be the primary instigators in the bite-generating process. Consistent with this action-initiating role, burst activity elicited experimentally in any one of these three cells was found to be capable of initiating complete cycles of fictive biting in otherwise quiescent buccal ganglia. However, although depolarizing current evoked burst in a B63 cell could elicit a complete bite without the involvement of the electrically coupled B30 and B65 neurons (Fig. 3d, left), this initiating ability was dependent upon the co-activation of both bilateral B63 cells, since holding one B63 neuron silent (with negative current injection) while activating the contralateral partner blocked radula motor pattern production (Fig. 3d, right). Similarly, experimentally induced bursts in either B30 or B65 could trigger fictive bites, but again these only occurred when associated with burst activation in B63 as a result of their electrical coupling. Therefore, although burst onsets in these three cell types was sufficient to instigate radula motor patterns that thereby can have different cellular origins according to the arbitrary order in which these neurons become active in a given bite cycle (Fig. 3e), although in all cases, the actual pattern-initiating process is critically dependent on the intervention of the bilateral pair of B63 neurons. Such randomly distributed burst activity in this kernel of decision-making neurons thus underlies the erratic and irregular expression of motor output commands that subserve the hungry Aplysia’s “trial and error” sampling of its environment in search for food [53].
Importantly, the decision-making B63/B30/B65 subset in vitro continues to express the memory trace of the previous associative experience of the intact animal. Accordingly, in ganglia taken from naive control or non-contingently trained animals (i.e., had received food reward in a sequence that was uncorrelated with spontaneous bite occurrences, Fig. 2aiii) which generated biting behavior at irregular intervals and relatively low cycle rates (Fig. 2b, left and right panels), the three cell types produced impulse bursts that were variable, erratic and weakly coordinated, with burst onset in any one cell type being capable of initiating a given bite cycle (left and right panels in Fig. 4b, c). In contrast, in buccal ganglia from contingently rewarded animals in which the feeding CPG network produced accelerated and regularized biting motor patterns, all three cell types expressed faster and stereotyped rhythmic bursts that were now tightly coordinated, with B63 bursts almost always leading (middle panels in Fig. 4b, c). Thus, the cellular correlates of the rate increase and regularization of radula biting behavior induced by in vivo operant conditioning were expressed within a module of electrically connected buccal circuit neurons that are necessary (B63) and/or sufficient (B63, B30, B65) for initiating each bite cycle. Moreover, pharmacological [38, 54] and electrophysiological evidence [55] has indicated that, as for pattern-switching B51, the reinforcing food reward signal to this motor pattern-initiating subset is mediated by dopaminergic inputs from the En esophageal nerve. Together these findings have wider implications for understanding decision-making processes and identifying the convergence points of operant behavior and dopamine-mediated reward [15], since they show that learning-induced neural plasticity underlying the selection of an appropriate motor action (e.g., ingestive biting by B51) and its actual initiation (by B63/B30/B65) maybe encoded at different cellular loci.
It is also instructive to compare reward and decision-making in Aplysia’s simple feeding behavior to more complex goal-directed motor systems such as the corticostriatal brain circuitry of rodents and primate mammals. Here, decision-making processes involving behavioral actions that are flexible, goal-directed and sensitive to rewarding feedback (i.e., analogous to the Aplysia’s search for food) are mediated by striatal pathways that are distinct from those mediating habitual, and relatively automatic decision-making actions resulting from associative learning [15, 17, 56]. This is in direct contrast to Aplysia where the action-outcome dependent transition from sporadic food reward-driven actions to a stereotyped, compulsive-like expression of the behavior is accomplished by functional plasticity within the same single subcircuit of central neurons. In this case, moreover, experience-related variability in action initiation is inscribed within a discrete neuronal subset of the feeding CPG network itself, rather than arising from distributed excitability changes among the wider CPG circuitry [57] or among specialized command-like neurons that are upstream to the CPG [48].
Operant learning-induced changes in neuronal burst-generating properties
Further electrophysiological exploration of buccal ganglia from animals with different behavioral training histories has enabled identifying the cellular sites at which the induction and maintenance of operant memory in the pattern-initiating B63/B30/B65 subset occurs. In principle, the learning-induced changes in the rate and regularity of burst activity in these neurons could be the result of intrinsic biophysical changes or of modifications in the properties of other presynaptic buccal network neurons that in turn modulate the activity of B63/B30/B65. To distinguish between these two possibilities, the burst-generating capability of individual B63/B30/B65 neurons were examined by depolarizing current injection under conditions of in situ isolation achieved by continuously hyperpolarizing one of the critical B63 cell pair to block activation of the other pattern-initiating cells and thereby of remaining feeding CPG circuitry (Fig. 5b, see also Fig. 3d); [58]. This functional isolation procedure revealed not only that each of the pattern-initiating cell types possesses an endogenous oscillatory mechanism that underlies its bursting but also that the expression of this intrinsic property varied in close accordance with learning-induced changes exhibited by the functionally intact feeding network. First, individually isolated B63, B30 and B65 neurons in buccal ganglia from operantly conditioned animals were found to be significantly more excitable (indicated by an increase in input resistance and a decrease in threshold for depolarization-induced burst activation) than in control and non-contingent ganglia. This enhanced excitability, which led to significantly higher rates of endogenous bursting in the three neuron types in response to similar levels of suprathreshold depolarizing current injection (Fig. 5b), therefore suggested a subcellular mechanism by which the learning-induced rate increase in radula motor pattern production occurred.
Second, in situ isolated pattern-initiating neurons also expressed cell-wide correlates of the learning-induced increase in regularity of radula biting behavior and underlying buccal network operation. Depolarization-activated bursting occurred erratically with highly variable durations in all three cell types in control and non-contingent buccal ganglia whereas in contingent ganglia, bursts elicited by similar current magnitudes were now rhythmically recurring with stereotyped durations and inter-burst intervals (Fig. 5b, lower). Moreover, in direct contrast to burst frequency, neither the rhythmic nor erratic forms of single cell activity, associated, respectively, with the occurrence or absence of associative learning, were affected by experimental depolarization, therefore indicating that the regularization of bursting by operant conditioning was not a consequence of the concomitant increase in neuronal excitability. This finding in turn suggested that the acquisition of stereotyped rhythmic bursting in these neurons arose from learning-induced plasticity in a voltage-independent process that was separate from the voltage-sensitive component of their intrinsic oscillatory mechanism. A plausible substrate for this second modulatory process derived from earlier experimental and modeling studies suggesting that the non-linear dynamics of calcium exchange between the endoplasmic reticulum and cytoplasm, which occurs independently of membrane voltage [59], maybe at the origin of chaotic bursting activity in endogenous oscillatory neurons [60, 61]. Moreover, theoretical evidence suggested that an extrinsic stimulus-driven damping of lumenal calcium exchange could lead to a regularization of the otherwise erratic membrane voltage oscillations [61, 62], although whether such a subcellular process underlies the transition between irregular and regular forms of endogenous bursting in B63/B30/B65 as a function of dopamine-mediated operant experience remains to be determined.
Operant learning-induced changes in network properties
In addition to independently enhancing the excitability and regularity of intrinsic bursting within individual pattern-initiating neurons, operant conditioning also led to a widespread increase in the functional coupling between cells and thereby the coherence with which bursts were produced. Experimental evidence for this further learning-induced plasticity and its likely restriction to the pattern-initiating cell subset derived from examining the effects of contingent training on the degree of coordination between the bursting in cell pairs in functional isolation from the rest of the buccal CPG network. For this, a B63 neuron was again continuously hyperpolarized to repress CPG activation, while burst-generating oscillations at approximately similar frequencies were elicited simultaneously in the second B63 and one of its B30 or B65 partners (Fig. 6a). In such independently activated cell pairs from control and noncontingent preparations, and as seen in the functionally intact CPG network (see Fig. 4b), burst onsets in B30 or B65 relative to B63 occurred at highly variable phase intervals that could be advanced or delayed relative to B63 in successive burst cycles (Fig. 6a, left and right panels). However, in the regularized activity of contingent preparations, burst onsets in B30 and B65 were closely coordinated with those of neuron B63 (Fig. 6a, middle panel).
This transition from weakly coordinated to coincident bursting as a consequence of operant conditioning was associated with corresponding changes in the strength of electrical coupling that was previously known to be widespread amongst ipsi- and bi-lateral pattern-initiating neurons [29, 30, 32]. Measurements of coupling strength between cell pairs with presynaptic hyperpolarizing current pulse injections into pre-junctional B30 or B65 revealed significantly stronger voltage responses in post-junctional B63 neurons of contingent buccal ganglia (Fig. 6b, middle panel) than in B63 of either control or noncontingent ganglia (Fig. 6b, left and right panels). An equivalent increase in electrical coupling strength, which was likely to underpin the coordinated bursting as a result of in vivo operant training, was found between ipsilateral components of the pattern-initiating subset as well as their contralateral partners. Whether this enhanced electrical connectivity results from direct alterations in the actual junctional resistance between pattern-initiating cell pairs is presently unknown, although the finding that their membrane input resistances increase with operant learning [58] suggested that the changes in coupling were likely, at least in part, to be attributable to non-junctional resistance changes.
Therefore, as a consequence of this network-wide plasticity acting in concert with a cell-wide transition to stereotyped endogenous bursting, individual radula motor patterns that are otherwise instigated by sporadic burst onsets in any one of the three cell types are transformed by operant conditioning to more frequent and regular occurrences in which the essential B63 neurons become the cycle-to-cycle leaders. The increase in electrical coupling between these cells can be seen to further strengthen the regularity and intensity of bursting in the bilateral B63 cell pair and thereby ensure motor pattern initiation in every burst cycle. Although electrical connectivity plays a major role in rhythmogenic circuit function [e.g., 63, 64] and can be modified by sensory and modulatory transmitter actions [65–67], its involvement in network plasticity associated with learning processes and the expression of memory has not, until very recently [68], been previously reported.
Ubiquity of dopamine in mediating associative learning
The transmitter dopamine is known to play a critical role in reward reinforcement and associative learning processes, including operant conditioning in both invertebrates and mammals alike. Reinforcing stimuli activate dopaminergic neurons in the mammalian brain [for review see 69–75], and electrical or pharmacological stimulation of DA systems can elicit behavioral responses that mimic the effects of primary reward reinforcement [69, 76, 77]. Conversely, the pharmacological blockade of DA receptors or the destruction of dopaminergic pathways leads to deficits in the acquisition of conditioned behaviors produced by associative learning procedures [69, 78–81].
Equivalent lines of evidence indicate that DA plays a similarly important role in reward-induced learning in Aplysia. For example, the esophageal sensory nerve En, the reinforcement pathway that likely conveys food reward signals in vivo [36, 42, 52] and which mediates electrical stimulation-induced operant learning in vitro [38–40] is rich with DA-containing fibers [26, 82] that provide synaptic input to buccal network elements, including the B51 [50] and B63 neurons [55]. Moreover, exogenous DA reproduces the effects of contingent reinforcement when applied to isolated B51 neurons in culture in a single cell analogue of operant conditioning [36, 45]. Conversely, blockade of endogenous dopamine’s actions with methylergonovine, a DA receptor antagonist [54, 83–86], blocks the contingent-dependent alterations in buccal CPG activity produced by an in vitro analogue of operant conditioning [50] and correlatively, in the bursting and synaptic properties of in situ isolated pattern-initiating neurons [55]. Such findings therefore not only emphasize the extent to which dopamine-mediated reward is conserved in the animal kingdom, but also point to the utility of Aplysia’s feeding behavior as a model system for studying the underlying cellular and molecular mechanisms [44] that underlie the transmitter’s actions in associative learning.
Conclusions
Much of our current understanding of the operation of the neuronal circuits that generate behaviors has derived from the study of stimulus-driven motor reflexes, or stereotyped rhythmic behaviors (such as locomotion and respiration) in which the underlying motor programs recur spontaneously in a relatively predictable manner. Unlike such stereotyped behaviors, however, much less is known about the neural basis of other intrinsically driven motor acts that are irregularly expressed, such as feeding or sexual activity, and in which the incentive (or impulse) to act, and how to act, are to a large extent governed by a dynamical interplay between the motivational state of the organism and learning processes. Moreover, in contrast to reflexive behaviors in which learning-induced adaptability essentially modifies the through-pathway transmission of input to output signals at sensori-motor connections (Fig. 7a), in internally driven goal-directed actions, behavioral plasticity emerges from an association between reinforcing sensory information and the functional dynamics of separate autoactive circuitry intercalated between the input–output stages of the system (Fig. 7b). This in turn adds a further level of complexity to learning and memory processes since here, activity-dependent changes in the intrinsic properties of this central circuitry and its constituent neurons (non-linear membrane properties and synaptic interconnectivity) provide the essential substrates through which learning-induced behavioral changes occur.
Moreover, unlike the CPG networks responsible for most stereotyped rhythmic behaviors where cycle-to-cycle automatism is conferred by the regular pacemaking properties of component neurons and their synaptic connectivity, the central circuitry that drives Aplysia’s erratic and irregularly repeating biting actions in its search for food, incorporates decision-making elements that are inherently capable of choosing whether or not to instigate individual cycles of motor activity or of selecting the output pattern that the CPG network produces.
The lack of a predesignated leader amongst these weakly coupled decision-makers further contributes to the variability (or “impulsiveness”) with which radula biting behavior is expressed. However, once the animal learns the positive consequences of its biting actions upon detecting food, the compulsive-like stereotypy that is now expressed in its predominantly ingestive behavior (selected by an increased excitability of B51) arises from the acquisition of accelerated, regularized and coincident bursting within an action-triggering neuronal subset (B63,B65,B30) which effectively stabilizes buccal network output into a phenotype resembling that of a more typical CPG.
Such observations on operant reward learning-induced changes in feeding behavior and its underlying circuitry in the simpler Aplysia model have thus begun to provide mechanistic insights into how animals, including humans, might learn to anticipate future appetitive or aversive events on the basis of past experience gained from the positive or negative consequences of their own behavior. Moreover, a greater understanding of the cellular and subcellular mechanisms underlying the switch in behavioral plasticity resulting from opposing positive and negative reinforcing learning processes may help guide future treatments for addictive behavioral disorders.
References
- 1.Byrne JH, Baxter DA, Buonomano DV, Cleary LJ, Eskin A, Goldsmith JR, McClendon E, Nazif FA, Noel F, Scholz KP. Neural and molecular bases of nonassociative and associative learning in Aplysia . Ann N Y Acad Sci. 1991;627:124–149. doi: 10.1111/j.1749-6632.1991.tb25918.x. [DOI] [PubMed] [Google Scholar]
- 2.Glanzman DL. The cellular mechanisms of learning in Aplysia: of blind men and elephants. Biol Bull. 2006;210:271–279. doi: 10.2307/4134563. [DOI] [PubMed] [Google Scholar]
- 3.Hawkins RD, Kandel ER, Bailey CH. Molecular mechanisms of memory storage in Aplysia . Biol Bull. 2006;210:174–191. doi: 10.2307/4134556. [DOI] [PubMed] [Google Scholar]
- 4.Reynolds GS. A primer of operant conditioning, 2nd edn. Glenview: Scott Foresman; 1975. [Google Scholar]
- 5.Thorndike EL. Animal intelligence: experimental studies. New York: Macmillan; 1911. [Google Scholar]
- 6.Skinner BF. The behavior of organisms: an experimental analysis. New York: Appleton-Century-Crofts; 1938. [Google Scholar]
- 7.Cook DG, Carew TJ. Operant conditioning of head-waving in Aplysia . Proc Natl Acad Sci USA. 1986;83:1120–1124. doi: 10.1073/pnas.83.4.1120. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Susswein AJ, Schwarz M, Feldman E. Learned changes of feeding behavior in Aplysia in response to edible and inedible foods. J Neurosci. 1986;6:1513–1527. doi: 10.1523/JNEUROSCI.06-05-01513.1986. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Skinner BF. Selection by consequences. Science. 1981;213:501–504. doi: 10.1126/science.7244649. [DOI] [PubMed] [Google Scholar]
- 10.Brembs B. Operant conditioning in invertebrates. Curr Opin Neurobiol. 2003;13:710–717. doi: 10.1016/j.conb.2003.10.002. [DOI] [PubMed] [Google Scholar]
- 11.Everitt BJ, Robbins TW. Neural systems of reinforcement for drug addiction: from actions to habits to compulsion. Nat Neurosci. 2005;8:1481–1489. doi: 10.1038/nn1579. [DOI] [PubMed] [Google Scholar]
- 12.Pitman RK. Animal models of compulsive behavior. Biol Psychiatry. 1989;26:189–198. doi: 10.1016/0006-3223(89)90022-X. [DOI] [PubMed] [Google Scholar]
- 13.Cardinal RN, Everitt BJ. Neural and psychological mechanisms underlying appetitive learning: links to drug addiction. Curr Opin Neurobiol. 2004;14:156–162. doi: 10.1016/j.conb.2004.03.004. [DOI] [PubMed] [Google Scholar]
- 14.Kelley AE. Ventral striatal control of appetitive motivation: role in ingestive behavior and reward-related learning. Neurosci Biobehav Rev. 2004;27:765–776. doi: 10.1016/j.neubiorev.2003.11.015. [DOI] [PubMed] [Google Scholar]
- 15.Balleine BW. Neural bases of food-seeking: affect, arousal and reward in corticostriatolimbic circuits. Physiol Behav. 2005;86:717–730. doi: 10.1016/j.physbeh.2005.08.061. [DOI] [PubMed] [Google Scholar]
- 16.Belin D, Jonkman S, Dickinson A, Robbins TW, Everitt BJ. Parallel and interactive learning processes within the basal ganglia: relevance for the understanding of addiction. Behav Brain Res. 2009;199:89–102. doi: 10.1016/j.bbr.2008.09.027. [DOI] [PubMed] [Google Scholar]
- 17.Balleine BW, Delgado MR, Hikosaka O. The role of the dorsal striatum in reward and decision-making. J Neurosci. 2007;27:8161–8165. doi: 10.1523/JNEUROSCI.1554-07.2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Hoyle G. Learning, using natural reinforcement, in insect preparations that permit cellular neuronal analysis. J Neurobiol. 1980;11:323–354. doi: 10.1002/neu.480110402. [DOI] [PubMed] [Google Scholar]
- 19.Wustmann G, Rein K, Wolf R, Heisenberg M. A new paradigm for operant conditioning of Drosophila melanogaster . J Comp Physiol A. 1996;179:429–436. doi: 10.1007/BF00194996. [DOI] [PubMed] [Google Scholar]
- 20.Spencer GE, Syed NI, Lukowiak K. Neural changes after operant conditioning of the aerial respiratory behavior in Lymnaea stagnalis . J Neurosci. 1999;19:1836–1843. doi: 10.1523/JNEUROSCI.19-05-01836.1999. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Gardner D. Interconnections of identified multiaction interneurons in buccal ganglia of Aplysia . J Neurophysiol. 1977;40:349–361. doi: 10.1152/jn.1977.40.2.349. [DOI] [PubMed] [Google Scholar]
- 22.Susswein AJ, Byrne JH. Identification and characterization of neurons initiating patterned neural activity in the buccal ganglia of Aplysia . J Neurosci. 1988;8:2049–2061. doi: 10.1523/JNEUROSCI.08-06-02049.1988. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Plummer MR, Kirk MD. Premotor neurons B51 and B52 in the buccal ganglia of Aplysia californica: synaptic connections, effect on ongoing motor rhythms, and peptide modulation. J Neurophysiol. 1990;63:539–558. doi: 10.1152/jn.1990.63.3.539. [DOI] [PubMed] [Google Scholar]
- 24.Hurwitz I, Susswein AJ. B64, a newly identified central pattern generator element producing a phase switch from protraction to retraction in buccal motor programs of Aplysia californica . J Neurophysiol. 1996;75:1327–1344. doi: 10.1152/jn.1996.75.4.1327. [DOI] [PubMed] [Google Scholar]
- 25.Hurwitz I, Kupfermann I, Susswein AJ. Different roles of neurons B63 and B34 that are active during the protraction phase of the buccal motor programs in Aplysia californica . J Neurophysiol. 1997;78:1305–1319. doi: 10.1152/jn.1997.78.3.1305. [DOI] [PubMed] [Google Scholar]
- 26.Kabotyanski EA, Baxter DA, Byrne JH. Identification and characterization of catecholaminergic neuron B65, which initiates and modifies patterned activity in the buccal ganglia of Aplysia . J Neurophysiol. 1998;79:605–621. doi: 10.1152/jn.1998.79.2.605. [DOI] [PubMed] [Google Scholar]
- 27.Jing J, Weiss KR. Interneuronal basis of the generation of related but distinct motor programs in Aplysia: implications for current neuronal models of vertebrate interlimb coordination. J Neurosci. 2002;22:6228–6238. doi: 10.1523/JNEUROSCI.22-14-06228.2002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Jing J, Cropper EC, Hurwitz I, Weiss KR. The construction of movement with behavior-specific and behavior-independent modules. J Neurosci. 2004;24:6315–6325. doi: 10.1523/JNEUROSCI.0965-04.2004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Elliott CJH, Susswein AJ. Comparative neuroethology of feeding control in molluscs. J Exp Biol. 2002;205:877–896. doi: 10.1242/jeb.205.7.877. [DOI] [PubMed] [Google Scholar]
- 30.Kupfermann I. Feeding behavior in Aplysia: a simple system for the study of motivation. Behav Biol. 1974;10:1–26. doi: 10.1016/S0091-6773(74)91644-7. [DOI] [PubMed] [Google Scholar]
- 31.Nargeot R, Petrissans C, Simmers J. Behavioral and in vitro correlates of compulsive-like food-seeking induced by operant conditioning in Aplysia . J Neurosci. 2007;27:8059–8070. doi: 10.1523/JNEUROSCI.1950-07.2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Botzer D, Markovich S, Susswein AJ. Multiple memory processes following training that a food in inedible in Aplysia . Learn Mem. 1998;5:204–219. [PMC free article] [PubMed] [Google Scholar]
- 33.Alberini CM, Ghirardi M, Metz R, Kandel ER. C/EBP is an immediate-early gene required for the consolidation of long-term facilitation in Aplysia . Cell. 1994;76:1099–1114. doi: 10.1016/0092-8674(94)90386-7. [DOI] [PubMed] [Google Scholar]
- 34.Taubenfeld SM, Milekic MH, Monti B, Alberini CM. The consolidation of new but not reactivated memory requires hippocampal C/EBPβ. Nat Neurosci. 2001;4:813–818. doi: 10.1038/90520. [DOI] [PubMed] [Google Scholar]
- 35.Levitan D, Lyons LC, Perelman A, Green CL, Motro B, Eskin A, Susswein AJ. Training with inedible food in Aplysia causes expression of C/EBP in the buccal but not cerebral ganglion. Learn Mem. 2008;15:412–416. doi: 10.1101/lm.970408. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Brembs B, Lorenzetti FD, Reyes FD, Baxter DA, Byrne JH. Operant reward learning in Aplysia: neuronal correlates and mechanisms. Science. 2002;296:1706–1709. doi: 10.1126/science.1069434. [DOI] [PubMed] [Google Scholar]
- 37.Lechner HA, Baxter DA, Byrne JH. Classical conditioning of feeding in Aplysia: I. Behavioral Analysis. J Neurosci. 2000;20:3369–3376. doi: 10.1523/JNEUROSCI.20-09-03369.2000. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Nargeot R, Baxter DA, Byrne JH. Contingent-dependent enhancement of rhythmic motor patterns: an in vitro analog of operant conditioning. J Neurosci. 1997;17:8093–8105. doi: 10.1523/JNEUROSCI.17-21-08093.1997. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Nargeot R, Baxter DA, Byrne JH. In vitro analog of operant conditioning in Aplysia. I. Contingent reinforcement modifies the functional dynamics of an identified neuron. J Neurosci. 1999;19:2247–2260. doi: 10.1523/JNEUROSCI.19-06-02247.1999. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Nargeot R, Baxter DA, Byrne JH. In vitro analogue of operant conditioning in Aplysia. II. Modifications of the functional dynamics of an identified neuron contribute to motor pattern selection. J Neurosci. 1999;19:2261–2272. doi: 10.1523/JNEUROSCI.19-06-02261.1999. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Brembs B, Baxter DA, Byrne JH. Extending in vitro conditioning in Aplysia to analyze operant and classical processes in the same preparation. Learn Mem. 2004;11:412–420. doi: 10.1101/lm.74404. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Lechner HA, Baxter DA, Byrne JH. Classical conditioning of feeding in Aplysia: II. Neurophysiological correlates. J Neurosci. 2000;20:3377–3386. doi: 10.1523/JNEUROSCI.20-09-03377.2000. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Lorenzetti FD, Mozzachiodi R, Baxter DA, Byrne JH. Classical and operant conditioning differentially modify the intrinsic properties of an identified neuron. Nat Neurosci. 2006;9:17–19. doi: 10.1038/nn1593. [DOI] [PubMed] [Google Scholar]
- 44.Lorenzetti FD, Baxter DA, Byrne JH. Molecular mechanisms underlying a cellular analog of operant reward learning. Neuron. 2008;59:815–828. doi: 10.1016/j.neuron.2008.07.019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Mozzachiodi R, Lorenzetti FD, Baxter DA, Byrne JH. Changes in neuronal excitability serve as a mechanism of long-term memory for operant conditioning. Nat Neurosci. 2008;11:1146–1148. doi: 10.1038/nn.2184. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Morton DW, Chiel HJ. In vivo buccal nerve activity that distinguishes ingestion from rejection can be used to predict behavioral transitions in Aplysia . J Comp Physiol A. 1993;172:17–32. doi: 10.1007/BF00214712. [DOI] [PubMed] [Google Scholar]
- 47.Briggman KL, Abarbanel HDI, Kristan WB. Optical imaging of neuronal populations during decision-making. Science. 2005;307:896–901. doi: 10.1126/science.1103736. [DOI] [PubMed] [Google Scholar]
- 48.Kristan WB. Neuronal decision-making circuits. Curr Biol. 2008;18:R928–R932. doi: 10.1016/j.cub.2008.07.081. [DOI] [PubMed] [Google Scholar]
- 49.Schall JD. Decision making. Curr Biol. 2005;15:R9–R11. doi: 10.1016/j.cub.2004.12.009. [DOI] [PubMed] [Google Scholar]
- 50.Nargeot R, Baxter DA, Patterson GW, Byrne JH. Dopaminergic synapses mediate neuronal changes in an analogue of operant conditioning. J Neurophysiol. 1999;81:1983–1987. doi: 10.1152/jn.1999.81.4.1983. [DOI] [PubMed] [Google Scholar]
- 51.Mozzachiodi R, Byrne JH. More than synaptic plasticity: role of nonsynaptic plasticity in learning and memory. Trends Neurosci. 2010;33:17–26. doi: 10.1016/j.tins.2009.10.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Schwarz M, Susswein AJ. Identification of the neural pathway for reinforcement of feeding when Aplysia learn that food is inedible. J Neurosci. 1986;6:1528–1536. doi: 10.1523/JNEUROSCI.06-05-01528.1986. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53.Horn CC, Zhurov Y, Orekhova IV, Proekt A, Kupfermann I, Weiss KR, Brezina V. Cycle-to-cycle variability of neuromuscular activity in Aplysia feeding behavior. J Neurophysiol. 2004;92:157–180. doi: 10.1152/jn.01190.2003. [DOI] [PubMed] [Google Scholar]
- 54.Reyes FD, Mozzachiodi R, Baxter DA, Byrne JH. Reinforcement in an in vitro analog of appetitive classical conditioning of feeding behavior in Aplysia: blockade by a dopamine antagonist. Learn Mem. 2005;12:216–220. doi: 10.1101/lm.92905. [DOI] [PubMed] [Google Scholar]
- 55.Nargeot R, Cornet C, Simmers J. Dopaminergic modulation of oscillatory membrane properties of buccal motor-pattern initiating neurons and their electrical coupling in Aplysia . FENS Abstr. 2008;4:021.13. [Google Scholar]
- 56.Balleine BW, Liljeholm M, Ostlund SB. The integrative function of the basal ganglia in instrumental conditioning. Behav Brain Res. 2009;199:43–52. doi: 10.1016/j.bbr.2008.10.034. [DOI] [PubMed] [Google Scholar]
- 57.Kemenes G, Staras K, Benjamin PR. Multiple types of control by identified interneurons in a sensory-activated rhythmic motor pattern. J Neurosci. 2001;21:2903–2911. doi: 10.1523/JNEUROSCI.21-08-02903.2001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 58.Nargeot R, Le Bon-Jego M, Simmers J. Cellular and network mechanisms of operant learning–induced compulsive behaviour in Aplysia . Curr Biol. 2009;19:975–984. doi: 10.1016/j.cub.2009.05.030. [DOI] [PubMed] [Google Scholar]
- 59.Chay TR. Electrical bursting and luminal calcium oscillation in excitable cell models. Biol Cybern. 1996;75:419–431. doi: 10.1007/s004220050307. [DOI] [PubMed] [Google Scholar]
- 60.Selverston AI, Rabinovich MI, Abarbanel HDI, Elson R, Szücs A, Pinto RD, Huerta R, Varona P. Reliable circuits from irregular neurons: a dynamical approach to understanding central pattern generators. J Physiol (Paris) 2000;94:357–374. doi: 10.1016/S0928-4257(00)01101-3. [DOI] [PubMed] [Google Scholar]
- 61.Varona P, Torres JJ, Huerta R, Abarbanel HDI, Rabinovich MI. Regularization mechanisms of spiking-bursting neurons. Neural Netw. 2001;14:865–875. doi: 10.1016/S0893-6080(01)00046-6. [DOI] [PubMed] [Google Scholar]
- 62.Falcke M, Huerta R, Rabinovich MI, Abarbanel HDI, Elson RC, Selverston AI. Modeling observed chaotic oscillations in bursting neurons: the role of calcium dynamics and IP3 . Biol Cybern. 2000;82:517–527. doi: 10.1007/s004220050604. [DOI] [PubMed] [Google Scholar]
- 63.Lewis TJ, Rinzel J. Self-organized synchronous oscillations in a network of excitable cells coupled by gap junctions. Network. 2000;11:299–320. doi: 10.1088/0954-898X/11/4/304. [DOI] [PubMed] [Google Scholar]
- 64.Placantonakis DG, Bukovsky AA, Aicher SA, Kiem H-P, Welsh JP. Continuous electrical oscillations emerge from a coupled network: a study of the inferior olive using lentiviral knockdown of connexin36. J Neurosci. 2006;26:5008–5016. doi: 10.1523/JNEUROSCI.0146-06.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 65.Lasater EM. Retinal horizontal cell gap junctional conductance is modulated by dopamine through a cyclic AMP-dependent protein kinase. Proc Natl Acad Sci USA. 1987;84:7319–7323. doi: 10.1073/pnas.84.20.7319. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 66.Yang XD, Korn H, Faber DS. Long-term potentiation of electrotonic coupling at mixed synapses. Nature. 1990;348:542–545. doi: 10.1038/348542a0. [DOI] [PubMed] [Google Scholar]
- 67.Johnson BR, Peck JH, Harris-Warrick RM. Amine modulation of electrical coupling in the pyloric network of the lobster stomatogastric ganglion. J Comp Physiol A. 1993;172:715–732. doi: 10.1007/BF00195397. [DOI] [PubMed] [Google Scholar]
- 68.Kokarovtseva L, Jaciw-Zurakiwsky T, Mendizabal Arbocco R, Frantseva MV, Perez Velazquez JL. Excitability and gap junction-mediated mechanisms in nucleus accumbens regulate self-stimulation reward in rats. Neuroscience. 2009;159:1257–1263. doi: 10.1016/j.neuroscience.2009.01.065. [DOI] [PubMed] [Google Scholar]
- 69.Schultz W. Dopamine neurons and their role in reward mechanisms. Curr Opin Neurobiol. 1997;7:191–197. doi: 10.1016/S0959-4388(97)80007-4. [DOI] [PubMed] [Google Scholar]
- 70.Schultz W. Dopamine signals for reward value and risk: basic and recent data. Behav Brain Funct. 2010;23:6–24. doi: 10.1186/1744-9081-6-24. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 71.Wise RA, Rompre PP. Brain dopamine and reward. Annu Rev Psychol. 1989;40:191–225. doi: 10.1146/annurev.ps.40.020189.001203. [DOI] [PubMed] [Google Scholar]
- 72.Wise RA. Dopamine and food reward: back to the elements. Am J Physiol Regul Integr Comp Physiol. 2004;286:R13. doi: 10.1152/ajpregu.00590.2003. [DOI] [PubMed] [Google Scholar]
- 73.Di Chiara G, Bassareo V. Reward system and addiction: what dopamine does and doesn’t do. Curr Opin Pharmacol. 2007;7:69–76. doi: 10.1016/j.coph.2006.11.003. [DOI] [PubMed] [Google Scholar]
- 74.Dalley JW, Everitt BJ. Dopamine receptors in the learning, memory and drug reward circuitry. Semin Cell Dev Biol. 2009;20:403–410. doi: 10.1016/j.semcdb.2009.01.002. [DOI] [PubMed] [Google Scholar]
- 75.Fulton S. Appetite and reward. Front Neuroendocrinol. 2010;31:85–103. doi: 10.1016/j.yfrne.2009.10.003. [DOI] [PubMed] [Google Scholar]
- 76.Olds J, Milner P. Positive reinforcement produced by electrical stimulation of septal area and other regions of rat brain. J Comp Physiol Psychol. 1954;47:419–427. doi: 10.1037/h0058775. [DOI] [PubMed] [Google Scholar]
- 77.Le Moal M, Simon H. Mesocorticolimbic dopaminergic network: Functional and regulatory roles. Physiol Rev. 1991;71:155–234. doi: 10.1152/physrev.1991.71.1.155. [DOI] [PubMed] [Google Scholar]
- 78.Ettenberger A. Dopamine, neuroleptics and reinforced behavior. Neurosci Biobehav Rev. 1989;13:105–111. doi: 10.1016/S0149-7634(89)80018-1. [DOI] [PubMed] [Google Scholar]
- 79.Smith-Roe SL, Kelley AE. Coincident activation of NMDA and dopamine D1 receptors within the nucleus accumbens core is required for appetitive instrumental learning. J Neurosci. 2000;20:7737–7742. doi: 10.1523/JNEUROSCI.20-20-07737.2000. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 80.Liu Z, Richmond BJ, Murray EA, Saunders RC, Steenrod S, Stubblefield BK, Montague DM, Ginns EI. DNA targeting of rhinal cortex D2 receptor protein reversibly blocks learning of cues that predict reward. Proc Natl Acad Sci USA. 2004;101:12336–12341. doi: 10.1073/pnas.0403639101. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 81.Fadok JP, Dickerson TMK, Palmiter RD. Dopamine is necessary for cue-dependent fear conditioning. J Neurosci. 2009;29:11089–11097. doi: 10.1523/JNEUROSCI.1616-09.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 82.Martínez-Rubio C, Serrano GE, Miller MW. Localization of biogenic amines in the foregut of Aplysia californica: catecholaminergic and serotonergic innervation. J Comp Neurol. 2009;514:329–342. doi: 10.1002/cne.21991. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 83.Ascher P. Inhibitory and excitatory effects of dopamine on Aplysia neurons. J Physiol (London) 1972;225:173–209. doi: 10.1113/jphysiol.1972.sp009933. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 84.Swann JW, Sinback CN, Carpenter DO. Evidence for identified dopamine motor neurons to the gill of Aplysia . Neurosci Lett. 1978;10:275–280. doi: 10.1016/0304-3940(78)90239-2. [DOI] [PubMed] [Google Scholar]
- 85.Wright NJ, Walker RJ. The possible site of action of 5-hydroxytryptamine, 6-hydroxytryptamine, tryptamine and dopamine on identified neurons in the central nervous system of the snail, Helix aspersa . Comp Biochem Physiol C. 1984;78:217–225. doi: 10.1016/0742-8413(84)90073-2. [DOI] [PubMed] [Google Scholar]
- 86.Teyke T, Rosen SC, Weiss KR, Kupfermann I. Dopaminergic neuron B20 generates rhythmic neuronal activity in the feeding motor circuitry of Aplysia . Brain Res. 1993;630:26–237. doi: 10.1016/0006-8993(93)90661-6. [DOI] [PubMed] [Google Scholar]