Reframing appetitive reinforcement learning and reward valuation as effects mediated by hippocampal-dependent behavioral inhibition

Abstract

Traditional theories of neuroeconomics focus on reinforcement learning and reward value. We propose here a novel reframing of reinforcement learning and motivation that includes a hippocampal-dependent regulatory mechanism which balances cue-induced behavioral excitation with behavioral inhibition. This mechanism enables interoceptive cues produced by respective food or drug satiety to antagonize the ability of excitatory food- and drug-related environmental cues to retrieve the memories of food and drug reinforcers, thereby suppressing the power of those cues to evoke appetitive behavior. When the operation of this mechanism is impaired, ability of satiety signals to inhibit appetitive behavior is weakened because the relative balance between inhibition and simple excitation is shifted toward increased retrieval of food and drug memories by environmental cues. In the present paper, we (1) describe the associative processes that constitute this mechanism of hippocampal-dependent behavior inhibition; (2) describe how a prevailing obesity-promoting diet and drugs of abuse produce hippocampal pathophysiologies that can selectively impair this inhibitory function; and (3) propose how glucagon-like peptide 1 (GLP-1), an incretin hormone that is recognized as an important satiety signal, may work to protect the hippocampal-dependent inhibition. Our perspective may add to neuroscientific and neuroeconomic analyses of both overeating and drug abuse by outlining the role of hippocampal-dependent memory processes in the control of both food and drug seeking behaviors. In addition, this view suggests that consideration should be given to diet- and drug induced hippocampal pathophysiologies, as potential novel targets for the treatment of dysregulated energy and drug intake.

1. Introduction

Combining methods and concepts from the behavioral and neurosciences, neuroeconomics is an interdisciplinary field that seeks to explain the ability of humans and other organisms to choose and follow a course of action [1]. A starting point for many neuroeconomic perspectives is that reinforcement learning and reward valuation are critical processes in this type of decision making (e.g., [2]). Reinforcement learning is a process by which humans and other animals learn to predict the outcomes of their actions. The goal of reinforcement learning is to maximize future rewards. This component of neuroeconomics also takes into account that maximizing reward involves not only the ability to predict when rewards are available, but also the value of the rewards that are predicted [1,2]. In addition to changes in reward value produced by the actual rewards or penalties (punishment, nonreward) received after each action, reward value can be updated based on the animal’s motivational state without the animal having to directly experience the rewarding outcome [3]. That is, after a valued outcome is experienced under a specific motivational state, subsequent exposure to the state can augment appetitive behavior, in anticipation of acquiring that outcome [4]. The addition of this type of cognitive processing provides a way for an animal to adjust reward value immediately and conditionally based on updated information about its internal state. For example, food satiation or satiation for a particular type of food reward, would be expected to diminish reward value in anticipation of contact with food [5].

Research on overeating leading to obesity and drug abuse leading to addiction has often focused on the development of maladaptations in brain reward circuitry (e.g., anterior cingulate cortex (ACC), ventral tegmental area (VTA), ventromedial prefrontal cortex (vmPFC), orbitofrontal cortex (OFC), and the ventral striatum) with different views proposing that deficiencies in reward sensitivity (motivating a person to obtain more or larger rewards to experience pleasure [6,7] or atypical reward valuation (e.g., overvaluation of the anticipated positive hedonic consequences of obtaining rewards (e.g., [8,9]) are responsible for these examples of behavioral excess. However, both views would seem to agree that physiological mechanisms that normally function to control intake are overwhelmed by powerful motivations to obtain food and drug rewards [5,10]. Conceptualized this way, the regulation of food and drug intake involves a balance between the ability of environmental food and drug cues to excite appetitive behavior and the ability of physiological satiety signals to inhibit the response-inducing power of those cues (e.g., [11])

Associations of events with reward and prediction errors resulting from discrepancies between rewards anticipated and rewards received are undoubtedly important determinants of behavior. Furthermore, it seems clear that reward-based choices involve an interplay between processes that excite and those that inhibit appetitive behavior [12,13]. However, the purpose of the present paper is to review evidence that a signaling mechanism and brain substrate that underlie the regulation of eating and drug use have been largely overlooked by current neuroeconomic approaches. We propose that the influence of reinforcement learning and brain reward mechanisms and circuitry on behavior is mediated by the ability to use physiological satiety signals to predict when environmental events will not be followed by rewarding postingestive outcomes and we argue that utilization of this information depends on the functional integrity of the hippocampus. We will describe an associative mechanism that (a) integrates reward with this hippocampal-dependent process; (b) explains how obesity-promoting diets and drugs of abuse impair this process and; (c) identifies hippocampal pathophysiologies that can underlie this impairment. A final focus of this paper will be on glucagon-like peptide 1 (GLP-1), an incretin hormone that is recognized as an important satiety signal. This hormone is of interest because of recent findings that longer-lasting GLP-1 analogs, such as liraglutide, have beneficial effects in the treatment of both obesity and cognitive dysfunctions. We will review evidence that links the therapeutic benefits of liraglutide on both of these disorders to the enhancement of hippocampal-dependent learning and memory processes. The implications GLP-1 analogs for the treatment of drug abuse and addiction will also be considered.

2. A hippocampal-dependent model of the learned control of energy intake and body weight

A central proposition of our analysis is that the regulation of food intake and body weight depends on maintaining a delicate balance between the power of environmental events to evoke appetitive behavior and the ability of physiological signals to counter or curb that power. This balance is delicate in the sense that even modest increases in behavioral evocation or decreases in behavioral inhibition would be expected to promote increased energy intake and body weight gain. Consistent with reinforcement learning and neuroeconomic views, we agree that the ability of environmental stimuli and responses to evoke appetitive responses depends on the degree to which those events are predictive of rewarding outcomes. However, in addition to this type of excitatory learning, we propose that human and nonhuman animals learn to use their interoceptive satiety cues to anticipate when rewarding outcomes will not be forthcoming. Many current views focus on the enticements of obesogenic environments, rich in highly palatable, energy-dense food and beverages, and replete with reminders of the rewards associated with eating, as main contributors to excess energy intake and continuing increases in the rates of obesity [14–16]. But it seems no less plausible that weakening of the signaling mechanisms that normally function to inhibit such urges could also contribute significantly to excessive intake.

We have suggested previously that the mechanism that integrates excitatory learning about the relationship between environment cues and food rewards with inhibitory learning about relationship between physiological signals and the absence of those rewards can be illustrated by what is known as the serial feature negative (sFN) discrimination problem [17–19]. In sFN discriminations, a “target” stimulus “A” (e.g., a brief tone) is followed by reinforcement (e.g., a sucrose pellet) (+), except on trials where it is preceded by the presentation of negative “feature” stimulus “B” (e.g., a light), yielding a conditional discrimination problem of the form A+, B →A−. To solve this problem, animals can use the negative feature cue to signal that the target cue will be followed by nonreward. The sFN discrimination is of interest because it encompasses a set of stimulus-event relations that appear to underlie the learned control of food intake [20,21]. For example, environmental food and food-related stimuli associated with the rewarding postingestive consequences of eating (A+) become strong elicitors of appetitive behaviors, except when they are encountered in the context provided by interoceptive food “satiety” cues (B→A−). In other words, similar to the sFN discrimination, the regulation of energy intake requires animals to use the information provided by their satiety signals to anticipate the likely consequences of eating. Viewed this way, to decide to eat or refrain from eating, animals must resolve a type of approach-avoidance conflict where food stimuli are associated with both rewarding and nonrewarding (including potentially aversive) outcomes. To resolve the conflict, animals use the information provided by their satiety cues to signal that consequences of eating will not be rewarding, thereby providing the basis for withholding appetitive responding.

3. The Role of the Hippocampus in Behavioral Inhibition

The results from several studies indicate that resolution of the types of conflicts animals face in sFN and related problems depends on the functional integrity of the hippocampus (e.g., [22–26]). For example, Figure 1 (right panel) shows the results of a seminal experiment by Holland et al., [22] which found that rats with highly selective neurotoxic lesions of the hippocampus (LES) were impaired relative to control rats (CON) at solving a sFN problem (A+, B→A−) in which punctate visual and auditory cues served as respective feature and target stimuli. However, in the converse serial feature positive (sFP) discrimination in which the feature cue signaled the reinforcement of the target (A−, B→A+), the performance of rats with hippocampal lesions was much less affected. Moreover, the pattern of results shown in Figure 1 indicates that the hippocampus is needed for rats to use a negative feature cue to inhibit appetitive behavior by signaling nonreinforcement of the target (on B →A− trials), whereas the ability of a positive feature cue to excite appetitive behavior by signaling that the target will be reinforced (on B→A+ trials) is not dependent on the hippocampus. Similarly, when exposed to a compound stimulus comprised of one component that had been previously associated with sucrose pellets and another that had been associated with shock, Schumacher et al., [26] found that the rats with ventral hippocampus lesions exhibited a significantly greater tendency to make approach responses compared to intact controls. Yet, compared to animals with an intact hippocampus, ventral hippocampal lesions did not increase responding to the cue associated with sucrose when that cue was presented separately. This outcome suggests that increased approach tendencies by lesioned rats during approach-avoidance testing were based on weakening of the inhibition of approach responses by the cue associated with shock. The tendency for responding to be elevated to previously nonreinforced cues in the sFN problem and in the approach-avoidance task seems unlikely to be the result of a generalized or nonspecific behavioral hyperactivity as terminal responding on nonrewarded trials in the sFP problem were largely unaffected by hippocampal damage and there was no evidence of overresponding on reinforced trials in the absence of inhibitory cues on either the sFN or sFP problems. Additionally, work by Noble et al., [27] showed that changes to melanin-concentrating hormone signaling in the ventral hippocampus increases impulsive responding without impacting overall activity or motivation for palatable food. These results, and those found by Holland et al., [22] are in agreement with other findings indicating that the hippocampus is part of a brain circuit that is selectively involved with the inhibition of learned appetitive responses [28] of reward memories [29], and with the ability of physiological satiety signals to inhibit food intake [30].

i. Figure 1:

Rats with neurotoxic lesions of the hippocampus (LES) were impaired relative to control rats (CON) at solving a serial feature negative (sFN) problem (A+, B→A−), but their performance was much less affected in the serial feature positive (sFP) problem (A−, B→A+), showing that the hippocampus is needed to inhibit appetitive responding by using the negative feature cue as a signal of nonreinforcement of the target. (adapted from. [22]).

4. The hippocampus: a site for integration of the physiological and learned controls of intake

As noted above, we suggest that the decision to eat or refrain from eating is based on the solution of a sFN problem that requires animals to use their satiety cues to signal when environmental food cues will not be followed by reinforcing postingestive stimulation. Within this framework, environmental food cues are like target (A+ trials) because they are associated with postingestive reinforcement when satiety cues (negative feature (B)) cues are absence, but not when satiety cues are present (B→A− trials). When an environmental food cue is encountered it will excite appetitive behavior based on its association with reinforcement only to the extent that the competing inhibitory association between those two events is not activated by satiety signals.

If the control exerted by physiological satiety cues over appetitive behavior involves a mechanism like that underlying behavioral control exhibited by negative feature cues in sFN problems, then interference with hippocampal function would be expected to interfere with satiety signaling. A number of findings are consistent with this prediction. For example, in both rats and humans, hippocampal damage impairs the ability to discriminate between interoceptive cues produced by periods of ad libitum and restricted feeding [31–34]. Rats with hippocampal lesions also exhibit increased food intake and elevated body weight gain relative to intact controls [35,36]. Furthermore, rats with lesions of both the dorsal and ventral hippocampus increase meal frequency independent of body weight changes [37] and rats with inactivation of the dorsal [38] or ventral [39] hippocampus decrease intermeal intervals and increase meal size and number of meals consumed. These observed alterations are consistent with an inability to use interoceptive satiety stimuli to anticipate when cues associated with food and eating will be nonreinforced.

Not only do impairments in hippocampal functioning increase food intake, but activation of the hippocampus decreases food intake. In a paper by Sweeney & Yang [40], optogenetic activation of glutamatergic neurons that project from the ventral hippocampus to the lateral septum, a region of the brain thought to be involved in gastric emptying and innervation of the lateral hypothalamus, suppresses food intake. In addition, there are bi-directional pathways for communication between the hippocampus and many brain regions that are important sites for energy and body weight regulation (e.g., hypothalamus, nucleus accumbens, medial prefrontal cortex (mPFC) [41,42]). The hippocampus is also densely populated with receptors for many hormones (e.g., insulin, leptin, GLP-1) that are known to be involved with the suppression (e.g., insulin, leptin, GLP-1) and evocation (e.g., ghrelin) of food intake. Activation of these receptors has been shown to impact learning and memory (see [43]).

5. Western diet and impairment in hippocampal functioning.

If impairments in hippocampal function can promote overeating and weight gain, the possibility that factors that promote overeating and weight gain produce hippocampal dysfunction is of interest. Energy rich diets containing high amounts of saturated fats and sugar (a.k.a., Western diets (WD)) have been implicated as important contributors to the high or increasing incidence of obesity in western and westernized societies [44–46]. Much evidence shows that intake of WD and/or the increased adiposity these diets produce can impair hippocampal-dependent cognitive functioning in both rats and humans [20,47–49].

In an experiment modeled after the previously described study by Holland et al., [22], Kanoski et al., [50] assessed the effects of 90-day ad libitum exposure to WD and standard low fat, low sugar, chow on the ability of rats to solve sFN and sFP discrimination problems. Figure 2 (right panel) shows that the pattern of impairment exhibited by rats fed WD in this study were remarkably similar to the pattern shown by Holland et al’s hippocampal-lesioned rats (see Figure 1). Specifically, both lesions and WD impaired performance on the sFN, but not on the sFP discrimination problems. In addition, both WD and hippocampal damage, reduced the ability of the negative feature to inhibit responding evoked by the target on B→A− trials. Furthermore, the lack of effect on the sFP problem strongly argues against the hypothesis that the detrimental effect of WD on sFN performance was based on nonspecific factors (e.g., motivational, hedonic, sensory, arousal, behavioral competence) because the rats were trained with the same stimuli, response requirements, reinforcer, number of trials, and intertrial intervals in both problems. Rather, the findings with both WD and lesions indicate that both manipulations reduced the inhibitory control of target cue appetitive responding by the negative feature cue in the sFN task. This type of disruptive effect of WD on sFN performance was also observed in several subsequent studies [51,52]. Furthermore, disruptive effects on sFN and other hippocampal-dependent problems have been reported following as few as 12 days of WD exposure (e.g., [19,53]). Performance impairments have also been observed on a variety of hippocampal-dependent water-maze and dry-maze problems in rodents maintained on a WD [54–56]. In humans, impairments in hippocampal-dependent memory, in the ability to discriminate interoceptive cues, and in the ability to inhibit “wanting” (based primarily on remembering the past consequences of eating), but not “liking” food (based primarily on direct evaluation of food’s sensory properties) have been reported following brief periods (4–8 days) of exposure to high-fat and high-sugar foods (see [57] for review).

ii. Figure 2:

Similar to hippocampal-lesioned rats, WD-fed rats were impaired at solving the sFN problem compared to the Chow-fed controls, but their performance on the sFP problem was not affected. Specifically, WD consumption, like hippocampal damage, affected the ability of the negative feature cue to inhibit appetitive responding evoked by the target cue on the B→A− trials (adapted from [50]).

5.1. WD and hippocampal pathophysiology.

Many of the studies mentioned above also found that WD intake was associated with pathophysiologies that appeared selective for the hippocampus. For example, the blood-brain barrier (BBB) regulates the entry of nutrients into brain tissue via specialized transporter mechanisms and prevents access by potentially harmful substances circulating in the blood supply [58]. Kanoski et al. [50] found, that in addition to being impaired relative to chow-fed controls on the sFN problem, WD-fed rats exhibited significantly increased hippocampal BBB permeability and reduced expression of tight-junction proteins which help to maintain the structural integrity of the BBB. In the same rats, no effect of WD on BBB permeability was observed in the PFC or striatum - two brain regions that have also been implicated in reinforcement learning and reward valuation. The hippocampus is also highly vulnerable to neuroinflammation that can occur in response to a variety of insults [59,60]. Several studies have shown that WD intake for as few as 15 days results in increased levels of hippocampal pro-inflammatory cytokines (e.g., [61–63]). Other findings suggest that inflammation and other hippocampal pathophysiologies may be a consequence of a WD-induced reduction in glucose transport across the BBB. In mice, decreased expression of GLUT1 protein at the BBB and reduced glucose uptake in the brain was observed following only 3–7 days of high-fat diet exposure [64]. The same study reported that expression of the SLC2A1 gene that provides instructions for the production of GLUT1 is significantly lower in overweight compared to normal weight humans. We [65] also found that expression of hippocampal GLUT1 is reduced in rats following as few as 10 days of ad libitum WD, relative to chow-fed controls. This reduction was accompanied by a deficit in memory functions that are thought to be hippocampal-dependent (i.e., vicarious trial and error). These findings are of special interest because changes in GLUT1 levels in rodents occur much sooner following WD exposure (< 10 days) compared to the earliest reported WD-related changes in BBB permeability (> 30 days). This suggests that reductions in GLUT1 may contribute to the manifestation of deficits in hippocampal-dependent cognitive function that occur soon after WD exposure and may serve as an early marker for chronic, longer-term dysregulation of eating and body weight.

6. GLP-1: A satiety and a memory enhancing hormone

Glucagon-like peptide-1 (GLP-1) is an incretin hormone that is predominantly produced in intestinal L-cells [66,67] and in a small subset of neurons in the nucleus tractus solitarius (NTS; for review see [68]). Elevation of GLP-1 following glucose consumption contributes to glucose homeostasis by stimulating insulin secretion in pancreatic β-cells [69] and decreasing glucagon levels [70,71]. GLP-1 neurons innervate brain areas associated with the rewarding or motivational aspects of feeding, such as the hypothalamus [72–74], VTA and nucleus accumbens [75–77]. Activation of GLP-1 receptors in these brain regions decreases and inactivation of these receptors increases eating and appetitive behavior (e.g., [30,76,78,79]). Reductions in GLP-1 response to intake have also been associated with obesity and higher body mass index (BMI; [80–82]). Based on these types of findings, GLP-1 is widely regarded as promoting both glucoregulation and “satiety” [72,83] (in contrast see [84] for a discussion of GLP-1’s limitations as a satiety hormone). GLP-1 expressing neurons are also abundant in the PFC and hippocampus (e.g., [43]). Early findings indicate that an overexpression of GLP-1 receptors in the hippocampus is associated with improvements on learning and memory tasks (e.g., passive avoidance, Morris water maze), and that mice with GLP-1 receptor deficits show impairments on hippocampal-dependent forms of spatial (e.g., Morris water maze) and contextual memory (e.g. contextual fear conditioning) [85].

These findings suggest that GLP-1 acts on both reward/motivation and cognitive control mechanisms in the brain, and therefore could have potential as a treatment for both obesity and cognitive dysfunction. However, because GLP-1 is rapidly degraded following exogenous administration [86], most research has focused on evaluating the efficacy of more potent, longer-lasting GLP-1 analogues, such as liraglutide and exendin-4, for use as therapeutic tools (e.g., [87]). There is now a large body of research documenting the positive effects of GLP-1 agonists on body weight and glucoregulation and data substantiating the beneficial effects of these drugs on cognitive functioning is rapidly accumulating.

7. GLP-1 analogues in the treatment of obesity and cognitive dysfunction

7.1. The effects GLP-1 analogues on intake and body weight.

Food consumption is reduced following both peripheral and central administration of GLP-1 analogues. In the periphery, GLP-1 analogues activate receptors on vagal afferent neurons which are known to influence brain function not only in response to endogenous GLP-1 [88], but to gastrointestinal signals at large [89,90]. However, peripheral administration of GLP-1 analogues can reduce intake and body weigh in rats following subdiaphragmatic vagal deafferentation [91], indicating that GLP-1 acts directly at sites in the central nervous system (CNS) independent of vagal input [92]. Evidence for effects of GLP-1 on brain reward circuitry comes from studies which show that following direct administration of GLP-1 agonists in the brain, intake is negatively correlated with activation of GLP-1 receptors in the VTA [93,94], nucleus accumbens [93], and hypothalamus [95]—brain regions that also appear to be activated in peripheral GLP-1 agonist administration [96,97].

7.2. The effects of GLP-1 analogues on memory and cognitive functions.

One potential extension of GLP-1 analogues is in the treatment of cognitive disorders. It is possible that GLP-1-induced improvement in cognitive disorders, such as Alzheimer’s disease (AD) may be due, in part, to changes in peripheral metabolic health. Improved performance on learning and memory tasks following the administration of GLP-1 agonists have been associated with alterations in hippocampal synaptic plasticity and long-term potentiation [98,99] which appear to be independent of improvements in metabolic functioning. Furthermore, treatment with GLP-1 agonists in animal models of neurodegenerative diseases has been associated with robust improvements on cognitive tasks and neuroprotective effects specific to the hippocampus, such as prevention of cell death [100], increased neurogenesis [101], and reduction in tau hyperphosphorylation [102], amyloid plaques, neurofibrillary tangles, and neuroinflammation [103]. Liraglutide is also able to not only prevent but reverse the negative effects of AD [104]. The potential use for GLP-1 agonists as treatments or preventions for cognitive disorders, such as AD, still needs to be explored further.

8. A hippocampal-dependent mechanistic hypothesis for GLP-1’s control of behavioral output

Based on the preceding brief review, one could reasonably conclude that GLP-1 has two independent functions. Consistent with neuroeconomic perspectives, GLP-1 can be seen as influencing the power of food rewards to evoke appetitive behavior. At the same time, GLP-1 appears to promote learning and memory functions that depend on the hippocampus. It may be possible (and parsimonious) to integrate both of these functions in one interpretive framework. In agreement with neuroeconomic and reinforcement learning concepts, we consider the encoding of predictive relationships between environment events and reinforcing outcomes as fundamental to the control of appetitive and consummatory behavior. Furthermore, a role for mesocorticolimbic circuitry in representing reinforcing outcomes and embedding them in predictive relationships fits within our view. However, instead of suppressing appetitive behavior by reducing reward valuation, our model proposes that GLP-1 acts on the hippocampus to antagonize or inhibit the ability of food-related stimuli to excite or retrieve the memory of food rewards. In other words, rather than decreasing reward value, GLP-1 decreases the activation of the associative/memorial representation of reward, thereby curbing the power of reward-related environmental cues to elicit appetitive responses.

The results of a recent study provides support for this hypothesis. Using the GLP-1 agonist liraglutide, Jones et al., [105] trained male and female rats to solve a sFN problem. Rats were then maintained on ad libitum WD or chow for 12 days during which time half of the rats of each diet group (WD or chow) received either a daily injection of liraglutide (10 μg/kg, intraperitoneal (i.p.)) or isotonic saline (sex was counterbalanced across conditions). After injections on Day 12, sFN performance was tested. During testing, liraglutide failed to reduce responding on rewarded trials where the target cue was presented alone (A+) (Figure 3, left). These data provide no evidence that liraglutide reduced the value of the reward associated with the target cue. In contrast, when the feature negative cue preceded presentation of the target on the nonrewarded (B→A−) trials, liraglutide significantly augmented the suppressive effect of the negative feature cue (Figure 3, right). This effect was not dependent on sex or diet. The failure of liraglutide to suppress responding in the absence of the inhibitory cue argues against the notion that liraglutide reduced the value of food reward and favors the idea that it enhanced a hippocampal-dependent form of memory inhibition.

iii. Figure 3.

12 i.p. injections of the GLP-1 analogue liraglutide enhances sFN discrimination by selectively improving hippocampal-dependent behavioral inhibition while leaving simple excitation relatively unaffected the effect did not differ by sex or diet. Asterisks (*) denote a significant difference at p < .05 and error bars represent standard error of the mean. Adapted from [105].

9. Extension to drug abuse and addiction

Although developed as an account of the regulation and diet-induced dysregulation of energy intake, we have recently begun to investigate the generality of our hippocampal-dependent model for understanding dysregulated use of cocaine. In one study [106], rats received concurrent training to asymptote on sFN and simple discrimination problems. Following asymptotic behavioral performance on the sFN and simple discriminations, rats were injected with either saline or cocaine (20 mg/kg/day, i.p.) daily. After the third day of injections, rats were tested on their ability to solve the discriminations. Probe tests were conducted on days 4, 7, 8, 10, 12, 14, 15, 16, 17 and 18. Figure 4 shows that while the cocaine-treated rats solved both the sFN problem and simple discrimination, they exhibited significantly more responding during the inhibitory negative feature cue compared to saline controls. Furthermore, in a separate study using the same parameters, higher BBB permeability in the hippocampus of the cocaine-treated rats was observed relative to rats injected with saline, whereas no differences in permeability were observed between the effect of cocaine and saline in the striatum. Although disruption in BBB function emerged faster with cocaine, than in previous studies with WD, the finding of impaired inhibition of responding by the negative feature cue was obtained with both the drug and diet manipulations.

iv. Figure 4.

18 days of i.p. cocaine injections impaired the ability to suppress hippocampal-dependent responding to an inhibitory cue in a sFN task. Cocaine did not alter responding in a hippocampal-indpendent simple discrimination. Asterisks (*) denote a significant difference at p < .05 and error bars represent standard error of the mean. Adapted from [106].

If obesity, caused by overeating, and cocaine abuse are both based, in part, on the impairment of a common hippocampal-dependent inhibitory memory mechanism produced by intake of WD and cocaine, respectively, then one expectation would be that the effects of ad libitum access to WD might synergize with those of cocaine to increase of cocaine self-administration. To test this prediction, Clasen (see [107]) gave male rats that had been maintained on WD or chow for approximately 55 days prior to and after the beginning of lever press training with an “active” lever that delivered intravenous self-administration (IVSA) cocaine (0.75 mg/kg/infusion) on fixed ratio (FR) 1, FR 5, FR 10, FR 20, and progressive ratio (PR) schedules. Drug availability was signaled by a cue light. Responding on an “inactive” lever was never reinforced. Under all of these conditions, animals maintained on the WD exhibited higher rates of cocaine IVSA than chow-fed controls. Figures 5a, b and c show that the number of cocaine infusions earned by rats fed WD exceeded that for chow-fed controls by substantial margins. Furthermore, compared to rats maintained on chow, WD-fed rats failed to inhibit responding on the active lever even when cocaine IVSA was not available. The findings indicate that WD exposure elevated cocaine intake and impaired the ability to inhibit behavior associated with cocaine reinforcement and are consistent with the hypothesis that WD-induced interference with hippocampal-dependent behavioral inhibition contributed to these results.

vi. Figure 5.

The number of cocaine IVSA infusions earned by rats in an ‘active’ lever press (A-fixed ratio; B-progressive ratio) was higher in WD fed rats relative to controls. There were also significantly more lever presses in a fixed ratio schedule during non-drug periods in WD-fed rats, indicating decreased behavioral inhibition in these rats. Asterisks (*) denote a significant difference at p < .05 and error bars represent standard error of the mean. Adapted from [107].

Several recent preclinical studies have provided evidence that liraglutide and other GLP-1 agonists have therapeutic potential as treatments for cocaine addiction and other substance abuse disorders (see [108,109] for reviews). For example, Hernandez et al., [110] reported that during abstinence following cocaine self-administration training, intraperitoneal injections of the GLP-1 receptor agonist exendin 4 (0.1 and 0.2 μg/kg) significantly attenuated the reinstatement of cocaine-seeking behavior elicited by a priming injection or by conditioned cues previously paired with cocaine delivery. Findings that administration of GLP-1 agonists can reduce intake of both food and drugs has typically been interpreted, consistent with current neuroeconomic perspectives, as evidence that GLP-1 receptor activation modulates the power of food and drug rewards to motivate or reinforce behavior via its influence on brain reward circuitry (e.g., [111]). However, the possibility that the suppressive effects of GLP-1 on both drug-seeking and food seeking are attributable, at least in part, to the augmentation of a hippocampal-dependent form of behavioral inhibition remains to be investigated.

10. Conclusions and implications

The findings reviewed in this paper support the conclusion that while hippocampal-independent reward mechanisms underlie the ability of environmental cues to excite appetitive behavior, hippocampal-dependent inhibitory memory mechanisms act to counter the excitement of those responses. The ability of satiety signals to exert this inhibitory power depends on the information they provide about likely nonreinforcing consequences of intake and the hippocampus is needed to utilize this information. Obesity-promoting western diets increase the response-evoking power of reward-related stimuli by interfering with this hippocampal function. In other words, western diets promote excess intake by shifting the control of appetitive behavior away from hippocampal-dependent inhibitory control toward relatively stronger control by reward-based learning. Preliminary research suggests that a similar mechanism may underlie cocaine abuse. Our studies with liraglutide provide evidence that satiety signaling by GLP-1 enhances the power of hippocampal-dependent memory mechanisms to inhibit appetitive responding and to prevent WD-induced impairment in those mechanisms, under conditions in which little evidence for a reduction in reward evaluation was obtained. An important implication of the present analysis is that a comprehensive account of the learned and physiological controls of appetitive behavior will describe how brain reward mechanisms underlying the capacity of reinforcement learning to excite responding are integrated with hippocampal-dependent memory processes that function to inhibit those responses [112]. This perspective may add to both behavioral neuroscientific and neuroeconomic analyses of both obesity and drug addiction.

Although we have emphasized a role for the hippocampus, we acknowledge that other structures are also important components in a larger brain circuit that underlies the regulation of appetitive behavior (see [20]). In our view, the hippocampus both utilizes information that comes from other brain loci and sends information to other loci to be utilized. For example, the hypothalamus [113] and insula [114] are potential sites for the detection of interoceptive cues that are used by the hippocampus to modulate evocation of appetitive responses. The evocation of those responses depends on the formation and utilization of excitatory and inhibitory associations between food cues and reinforcing outcomes. The striatum and the amygdala are likely substrates for these processes [115]. Similarly, decision making and inhibitory control function of the prefrontal cortex [116] presumably depend on information provided memories of past experiences with environmental cues, satiety and postingestive events and their interrelationships. It seems plausible that at least some of this information is transmitted by hippocampal signaling pathways [30,40]. Moreover, direct reciprocal connections of the hippocampus with the amygdala, insula and prefrontal cortex make the hippocampus neuroanatomically well-situated to serve as an interface for each of these regions in humans [117].

Our analysis also suggests that research on therapeutic interventions might be aimed at finding ways to protect the hippocampus from the harmful effects of obesity-promoting diets and drug of abuse. For effective interventions to be developed more knowledge of the mechanisms that underlie diet-induced and drug-induced hippocampal pathophysiologies (e.g., BBB disruption, interference with glucose transport, inflammation; see [20]) is required. In this regard, studies of GLP-1 analogs, which appear to act as endogenous satiety signals and may also enhance cognitive inhibitory processes [105] have promise.

Abbreviations

ACC

Anterior cingulate cortex

VTA

ventral tegmental area

vmPFC

ventromedial prefrontal cortex

OFC

orbitofrontal cortex

GLP-1

glucagon-like peptide 1

sFN

serial feature negative

sFP

serial feature positive

LES

lesion

CON

control

mPFC

medial prefrontal cortex

WD

Western diet

BBB

blood-brain barrier

PFC

prefrontal cortex

CNS

central nervous system

AD

Alzheimer’s disease

i.p.

intraperitoneal

IVSA

intravenous self-administration

FR

fixed ratio

PR

progressive ratio