The Brain’s Reward System in Health and Disease

Abstract

Rhythmic gene expression is found throughout the central nervous system. This harmonized regulation can be dependent on- and independent of- the master regulator of biological clocks, the suprachiasmatic nucleus (SCN). Substantial oscillatory activity in the brain’s reward system is regulated by dopamine. While light serves as a primary time-giver (zeitgeber) of physiological clocks and synchronizes biological rhythms in 24-h cycles, nonphotic stimuli have a profound influence over circadian biology. Indeed, reward-related activities (e.g., feeding, exercise, sex, substance use, and social interactions), which lead to an elevated level of dopamine, alters rhythms in the SCN and the brain’s reward system. In this chapter, we will discuss the influence of the dopaminergic reward pathways on circadian system and the implication of this interplay on human health.

4.1. The Dopaminergic Mesolimbic System and Reward

The mesolimbic system, also known as the reward system, is composed of brain structures that are responsible for mediating the physiological and cognitive processing of reward. Reward is a natural process during which the brain associates diverse stimuli (substances, situations, events, or activities) with a positive or desirable outcome. This results in adjustments of an individual’s behavior, ultimately leading them to search for that particular positive stimulus. Reward requires the coordinated release of heterogenous neurotransmitters. However, of the brain substrates implicated in reward, dopamine has a central position. Dopamine plays a critical role in mediating the reward value of food, drink, sex, social interaction, and substance abuse (Hernandez and Hoebel 1988; Everitt 1990; Robbins and Everitt 1996; Bardo 1998; Beninger and Miller 1998).

The dopaminergic pathway mostly involved in reward is the so-called mesolimbic system, which is formed by projections of midbrain dopamine neurons of the ventral tegmental area (VTA) to the striatum, prefrontal cortex, amygdala, hippocampus, and many other structures of the limbic system. When rewarding stimuli are experienced, the dopaminergic mesolimbic system is activated which causes the release of dopamine to the targeted nuclei (Small et al. 2003; Cameron et al. 2014). The ventral striatum, including the nucleus accumbens (NAcc), is a major substrate involved in reward (Marche et al. 2017). The dorsal striatum is critically involved in action selection and initiation components of decision making and also seems to mediate feedback properties such as valiance and magnitude in addition to controlling habitual behavior (Balleine et al. 2007; Burton et al. 2015; Lipton et al. 2019). Therefore, both dorsal and ventral regions have collaborative roles in mediating reward. Nevertheless, the NAcc is most appreciated for its involvement in reward processing and its role in evaluation and incentive-based learning (Schultz et al. 1992; Daniel and Pollmann 2014).

The most prominent striatal neurons are the γ-aminobutyric acid (GABA) producing medium spiny neurons (MSNs). These cells make up to 90–95% of the neuronal population and serve as the sole output from the striatum (Kemp and Powell 1971; Graveland and DiFiglia 1985). MSNs outputs generate two pathways: the direct pathway formed by dopamine D1 receptor (D1R) expressing medium spiny neurons (dMSNs) and the indirect pathway by dopamine D2 receptor (D2R) expressing medium spiny neurons (iMSNs). Coordinated dopamine signaling to dMSNs and iMSNs within the striatum is critical for integrating and responding to rewarding stimuli.

The other 5–10% of striatal neurons are interneurons, which serve as intrastriatal regulators of MSNs activity (Oorschot 2013). The majority of interneurons are inhibitory GABAergic interneurons which modulate reward through their signaling to MSNs and expression of a variety of modulatory peptides (Gittis et al. 2010). About 1–2% are formed by the tonically active cholinergic interneurons which, despite their low abundance, critically regulate MSNs (Kharkwal et al. 2016a; Lewis et al. 2020). Indeed, activation of cholinergic interneurons has been linked to the salience of events (Gittis and Kreitzer 2012). Thus, inter- and intra-striatal connections modulate striatal circuits and play a critical role in reward processing.

Natural rewards, such as eating, drinking, and mating are necessary for survival and maintenance of a species. At its core, the reward system determines the valence of a stimulus and signals whether it is to be avoided or approached, as well as assigning the priority of one stimulus over another. Substances of abuse, whether illicit (e.g. cocaine, heroin, etc.) or licit (e.g. alcohol, nicotine, etc.), hijack the mesolimbic system by offering a reward without an obvious biological function. However, the pleasure and reward linked to initial substance use are then lost by their abuse, which leads to a vicious circle of addiction (Volkow et al. 2016).

Recent studies have shown that reward is subjective and is highly influenced by the chemistry of the individual, homeostatic state (Paulus 2007; Keramati and Gutkin 2014) and genetics (Comings and Blum 2000; Jia et al. 2016), as well as by the environment and epigenetics (Xu et al. 2007; Solinas et al. 2009; De Decker et al. 2017). Indeed, how, when, and where rewarding stimuli are experienced can have a profound influence on reward-related behaviors, as a result of activation of several circuits located in the striatum as well as in other brain regions responsible for encoding and storing memory of events. Importantly, the mesolimbic system is connected to the suprachiasmatic nucleus (SCN)—the master regulator of circadian rhythms (Grippo et al. 2017). The SCN is known to influence reward-related behavior and reciprocally, rewarding stimuli can serve as time-givers (zeitgebers) to entrain the SCN as well as peripheral clocks through the release of dopamine (Honma and Honma 1995; Davidson et al. 2005; Baba et al. 2017).

4.2. Rhythmic Variation in Dopamine-Related Activity

Rhythmic control of an organism’s behavior is a critical part of adapting and anticipating environmental changes in light, temperature, and resources. Though time-keeping mechanisms are more complex and developed in mammals, diurnal control is conserved throughout nature (Edgar et al. 2012). In mammals, the SCN organizes behavior and its correlated cellular activity through hormone and neurotransmitter release, in a 24-hour cycle based on daily light and dark phases (Dunlap 1999).

Support of a circadian regulation of reward was initially highlighted by admittance of patients experiencing substance overdose into the emergency room predominantly in the evening (Morris 1987; Raymond et al. 1992). Thus, the night spikes in overdose are likely related to differences in the metabolism of drugs of abuse during different times of day (Baird and Gauvin 2000; Abarca et al. 2002). Importantly, a variety of medications have been shown to have better clinical efficacy at precise times during day (Musiek and FitzGerald 2013; Nobis et al. 2019; Samir et al. 2020). Timing effects of rewarding stimuli also extend to natural rewards where time of day influences physiological responses as well as anticipatory rhythms (Castro 2004; Landry et al. 2012; Johnston 2014).

Dopamine levels in SN and VTA follow circadian oscillations, rising in the active phase and falling in the resting phase of the day (Smith et al. 1992; Hood et al. 2010; Ferris et al. 2014), as does its precursor and metabolites (Paulson and Robinson 1996; Castañeda et al. 2004) (Fig. 4.1). Rhythmic expressions of clock genes including Clock, Rev-ERBα, Per, Npas2, and Bmal1 are involved in dopamine metabolism (McClung et al. 2005; Chung et al. 2014). Indeed, Clock and Rev-ERBα negatively regulate the expression of tyrosine hydroxylase (TH), the rate limiting enzyme in dopamine synthesis (Musacchio 1975). Levels of TH increase during the active phase, which is opposite to that of Clock and Rev-ERBα; loss of either circadian gene results in disrupted rhythmic TH expression (McClung et al. 2005; Chung et al. 2014). Transcription of monoamine oxidase A (MAOA), the enzyme responsible for dopamine breakdown, is regulated by the expression of NPAS2, BMAL1, and PER2 (Hampp et al. 2008). Deletion of Per2 causes a lack of MAOA expression during the resting phase, which leads to elevated basal levels of dopamine in the NAcc (Hampp et al. 2008).

Fig. 4.1.

Overview of dopamine-related activity in the reward system. Dopamine levels peak in the active phase when tyrosine hydroxylase (TH) levels are high while monoamine oxidase A (MAOA) levels are low and dopamine transporter (DAT) activity is decreased. This corresponds to core clock gene expression in the striatum, which regulates the expression of these dopamine metabolism-related gene expressions

Psychostimulants increase extracellular dopamine levels and alter the expression of clock genes in the striatum (Nikaido et al. 2001; Uz et al. 2003; Lynch et al. 2008). Though drugs like cocaine and methamphetamine simultaneously alter levels of other neurotransmitters such as serotonin (Haughey et al. 2000; Andrews and Lucki 2001), their impact on clock gene expression is largely dependent on dopamine signaling. Indeed, administration of the D1R agonist, SKF-38393, increases mRNA levels of Per1, Clock, Bmal1, and Npas2 while the D2R agonist, quinpirole, decreases Clock and Per1 expression (Imbesi et al. 2009). D1R signaling plays a critical role in Per2 expression, as D1R-null mice have reduced Per2 in the striatum (Gallardo et al. 2014). Interestingly, depletion of dopamine by 6-hydroxydopamine lesions of dopaminergic neurons results into suppression of PER2 oscillations which can be rescued by D2R agonists (Hood et al. 2010). These results imply that the simultaneous activation of both D1R and D2R is necessary for the normal Per2 oscillations in the striatum. The effect of D2R on Per2 expression might not be direct, but mediated by the inhibitory regulation of iMSNs on dMSNs through collaterals (Lemos et al. 2016; Kharkwal et al. 2016b).

Dopamine’s influence on the SCN was inferred by expression of both D1R and D5R on its neurons (Weiner et al. 1990; Rivkees and Lachowicz 1997; Doyle et al. 2002). In neonatal hamsters, light pulses mirror the effects of D1R agonists suggesting that the maternal levels of DA correspond to the active phase in the fetal SCN (Viswanathan and Davis 1997). Dopamine has been reported to play a critical role in entraining fetal development through the SCN and that after this period the SCN’s responsiveness to dopamine declines (Weaver and Reppert 1995; Mendoza and Challet 2014). Nevertheless, D1R activation in the SCN shifts the phase of circadian rhythms and a direct connection between the VTA and the SCN has been described (Grippo and Güler 2019). Furthermore, D2R seems to be absolutely required for the light-induced suppression of locomotor activity (masking), whereas other visual or nonvisual photic responses seem to be D2R independent (Doi et al. 2006). These results showed a yet unappreciated function of D2R-mediated signaling in regulating the proper organization of daily locomotor activity in light-dark cycles.

Thus, the daily fluctuation in VTA dopamine neuron activity has a substantial role in SCN entrainment and other circadian activities.

4.3. Food and its Relationship to the Circadian Control of the Mesolimbic System

It is a complex process that both the type of food we consume and how much is consumed integrates the homeostatic and reward systems. Controlled food intake relies on balanced responses between orexigenic and anorexigenic neurons of the hypothalamus, which respond to circulating hormones and nutrients (Kalra et al. 1999; Meister 2007). The hypothalamus regulates the production of neuropeptides like ghrelin, leptin, and neuropeptide Y (NPY) in a diurnal manner, which contributes to appetite regulation (Kalra et al. 1999). Genetically engineered mice with deletions of genes encoding either ghrelin, leptin, or NPY have aberrant feeding behaviors or metabolic fuel preference (Bannon et al. 2000; Wortley et al. 2004; Cristino et al. 2013; Schéle et al. 2016). In a simplistic model, low levels of nutrients such as glucose, fats, and amino acids increase levels of ghrelin and decrease leptin (Weigle et al. 1997; Tschöp et al. 2000; Klok et al. 2007). Ghrelin acts on NPY-producing neurons in the hypothalamus which cause the release of NPY (Kohno et al. 2003). Food intake restores deficits in nutrients, decreasing ghrelin and causes the release of leptin from adipose tissue (Izadi et al. 2014). Leptin acts on NPY-producing neurons in the hypothalamus, reducing the amount of NPY released (Baver et al. 2014). An intact control of homeostatic regulation through integration of these signals and the subsequent response is necessary for the maintenance of a stable body weight. Dysregulation of this system leads to obesity and its associated comorbidities including heart disease and diabetes (Turek et al. 2005; Depner et al. 2014; Reutrakul and Knutson 2015).

Taste, smell, texture, and temperature all contribute to the subjective pleasantness of food and rely on the mesolimbic dopamine system. The taste of saccharin sweetened water, for example, is chosen over intravenous cocaine administration in mice (Lenoir et al. 2007). Food that is more palatable, and as a result more rewarding, is expected to cause increased release of dopamine in the NAcc (Volkow et al. 2010, 2012). Indeed, palatable foods containing high levels of sugars (Rada et al. 2005) and fats (Rada et al. 2012; Cone et al. 2013) are known to stimulate the release of dopamine into the NAcc. Dopamine has an essential role in mediating appetite which goes above the homeostatic system. Dopamine-deficient mice with inactive TH in VTA neurons (Szczypka et al. 2001) as well as mice with constitutive deletions of both D1R and D2R (Kobayashi et al. 2004) develop early fatal hypophagia. Dopaminergic pathways have been found to be altered in obese subjects. Striatal D2R expression is reduced by a palatable food diet in mice (Johnson and Kenny 2010) and in humans striatal D2R availability is significantly lower in obese patients compared to control individuals (Wang et al. 2001).

Repeated exposure to food with high fat and sugar content results in compulsive food consumption, poor control of food intake, and food stimulus conditioning (Jauch-Chara and Oltmanns 2014). These results suggest that palatable food can disrupt endogenous homeostatic regulation of food intake through activation of the reward system. Interestingly, leptin receptors have been found in the VTA and SNpc, and a putative role in regulating dopamine release has been proposed (Figlewicz et al. 2003). Moreover, ghrelin is known to stimulate VTA dopamine neurons to release dopamine into the NAcc (Abizaid et al. 2006). Thus, endogenous and exogenous signals control appetite through important interactions between the physiological need for food and the reward system.

Food consumption follows circadian rhythms. Through regulation of complex networks involving the homeostatic and reward systems, food intake sets time. One hypothesis posits that orexigenic pathways, which increase feeding behavior, become gradually activated during fasting while sleeping. However, this hypothesis contrasts evidence in humans showing that hunger has an endogenous circadian rhythm with lowest levels in the morning (8am) and greatest in the evening (8pm) regardless of the type of food intake (Scheer et al. 2013). Moreover, in the absence of external time cues individuals seek 2–3 meals during their active phase; however, the timing when these meals occur shows massive subject variability and is influenced by differences in circadian period and wakefulness (Aschoff et al. 1986).

A number of clocks in the brain can be reset by peripheral metabolic signals, which may contribute to food anticipation. Palatable foods can trigger anticipatory bouts of locomotor activity and arousal indicating an activation of the mesolimbic dopamine system (Mistlberger 1994). Despite this insight, the anatomical locations and molecular mechanisms for the food clock remain elusive. Mice with genetic deletions of Bmal1, Per1, and Per2 have normal food anticipatory behavior as do SCN-lesioned mice (Storch and Weitz 2009). This information indicates that the central clock is not required for food anticipation. However, mutations in Per1 have been shown to shift food intake to the resting phase, which leads to obesity in mice (Liu et al. 2014). Additionally, mice carrying deletions of Bmal1 and Per2 become obese from eating food equally during day and night as do ClockΔ19 mutant mice (Turek et al. 2005; Guo et al. 2012). The most likely candidates for the food clock lie in other regions of hypothalamus as well as in the striatum (Gallardo et al. 2014).

Nutrition, metabolism, and circadian rhythms are intricately linked with each other (Fig. 4.2). Timing of food intake can alter the circadian system positively or negatively. Indeed, meal timing can affect sleep/wake cycles, body temperature, performance, and alertness (Hotz et al. 1987; Hawley and Burke 1997; GRANT et al. 2017; Hou et al. 2019). These effects are enhanced by calorie restriction, high-fat and high-sugar, among others. Rhythmic dopamine levels from the VTA to the NAcc underlie motivation, food craving, and anticipation (Parekh et al. 2015). The SCN indirectly projects to the VTA through the medial preoptic nucleus of the hypothalamus (Luo and Aston-Jones 2009); this connection might conceivably allow for food-seeking directed movement through modulation of dopamine signaling in the striatum. This connection may also affect reinforcement and conditioned learning associated with food intake. Thus, the control of food intake is dependent on a balanced interaction between metabolic and hedonic circadian brain circuits.

Fig. 4.2.

A schematic representation of the brain's reward system in relation to circadian rhythms. Diurnal rhythms of the mesolimbic dopamine system and suprachiasmatic nucleus directly influence the activity of these brain regions. Rhythmic dopamine levels influence the activity of the mesolimbic dopamine system and suprachiasmatic nucleus. The activation of dopamine receptors in these brain regions alters core clock gene expression. The expression of core clock gene affects rewarding behaviors including food consumption, drug use, sexual activity, and social interactions, which activate the mesolimbic dopamine system

4.4. Rhythmicity of Mating Behavior and Sex-Driven Reward

To facilitate necessity for species survival, mating activity highly engages the reward system and principally involves dopamine (Balfour et al. 2004). Dopamine release critically affects mating at the motor, arousal, motivation, and reward levels. In rats, systemic pharmacological treatments, which increase or decrease dopamine signaling, improve or worsen parameters of copulatory activity, respectively (Melis and Argiolas 1995). Dopamine signaling in the striatum has been postulated to mediate the reinforcing properties of sexual reproductive activity (Becker et al. 2001; Sanna et al. 2020). Regardless, dopamine signaling appears to play a critical role in sex-driven reward.

Like almost all physiological parameters in animals and humans, mating also shows some rhythmicity. In humans, most sexual encounters occur around midnight (Refinetti 2005). Environmental factors, namely partner availability, is the predominant factor important for human sexual activity. Peripheral tissues in the reproductive axis have been shown to have rhythmic clock gene expression, which might influence or synchronize with sexual behavior (Sen and Hoffmann 2020).

In animals, mating rhythmicity is important for avoiding predation and is also important for finding the right mating partner. Strong seasonal rhythms, which are linked to the amount of light, are apparent in males of many species including rodents and sheep, which are better suited models in this area of chronobiology (Reiter et al. 1980). As an example, rams are sensitive to daily changes in light across the year which induce hormonal variations and modulate gonadal function as well as libido without changes in hormone secretion (Lincoln et al. 2003). This provides evidence that, unlike other natural rewards like food, reproductive behavior is not under homeostatic regulation.

Anticipation rhythms have been observed in rodents in response to schedules; thus, suggesting that anticipatory rhythms may be located within the reward system or could be entrained by stimuli, which also engage the reward system. Indeed, circadian clock genes in the dopaminergic pathways can be shifted by natural rewards as well as dopaminergic compounds. These findings imply that copulation could also induce robust circadian anticipatory rhythms. Male rodents can anticipate daily opportunities to mate (Landry et al. 2012). Interestingly, rats can anticipate scheduled mating toward the end of their daily active phase and in the middle of their resting phase. Reproductive behavior also shows diurnal variation as does sex-related reward, which peaks in the daily active phase and corresponds with dopamine levels in the striatum (Webb et al. 2009). These results suggest that sexual anticipation and reward are linked with diurnal rhythms in the dopaminergic mesolimbic system (Melis and Argiolas 1995) though the molecular mechanisms remain elusive.

4.5. Drugs of Abuse

Although our understanding of the specific actions of drugs on the reward system has been growing, the complexity of the fundamental mechanisms underlying drug abuse and dependence increases. Drugs of abuse share one common mechanism: they raise dopamine levels in the brain, which elicits reward, driving vulnerable (Swendsen and Moal 2011) substance users to seek for more drugs leading to addiction. At the cellular level, the drug-induced dopamine increase alters neuronal plasticity at the molecular level leading to alterations of gene expression and the consequent modification of neuronal circuits.

A growing body of evidence connects perturbations of circadian rhythms and clock genes to the development and progression of addictive disorders (Webb 2017). People with addiction have highly disrupted rhythms which could be a result of genetic and/or epigenetic factors like sleep deprivation (Logan et al. 2018). Indeed, those with an evening chronotype (night owls) have been linked to disorders of the mesolimbic dopamine system including depression, insomnia, and substance abuse (Merikanto et al. 2013; Kivelä et al. 2018). Many behaviors that depend on the mesolimbic system, such as psychomotor sensitization and drug-seeking, show rhythmic patterns and are under the control of circadian genes (Abarca et al. 2002). Surprisingly, substance abuse leads to lasting changes in circadian rhythms, which can persist even after cessation of the drug intake (Jones et al. 2003).

Like for natural rewards, there are diurnal variations in the behavioral response to substances of abuse. Addictive drugs are known to influence behavioral rhythms, through modifications of the expression of clock genes such as Clock, Per1, and Per2. Clock is expressed in the VTA and NAcc and has been implicated in modulating reward processing. Mice with Clock null mutations show enhanced sensitivity to cocaine which has been demonstrated by conditioned place preference (CPP) (McClung et al. 2005) and self-administration (Ozburn et al. 2012) models of substance abuse. Similarly, Per1 and Per2 seem to have roles in cocaine sensitization (Uz et al. 2003), which is thought to be a critical component of drug craving that leads to dependence (Robinson and Berridge 2008). Per1 and Per2 expressions appear modulated by D1R and D2R. Interestingly, Per1 and Per2 mutants show increased alcohol CPP compared to WT controls (Gamsby et al. 2013). Per1 null mice show decreased morphine CPP (Perreau-Lenz et al. 2017) and absence of cocaine CPP (Abarca et al. 2002). In contrasts Per2 mutants show no difference from WT littermates when tested for cocaine CPP (Abarca et al. 2002).

4.6. Social Reward, Electronics, and the Clock

The developed world has a long-held fascination for technologies with entertainment purposes, which continues to grow. Adults in the United States spend 2–4 h per day using electronic devices, making technology a deeply engrained part of our lives (Dyck et al. 2011). The aberrant and persistent usage of these devices has called into question whether one could become addicted to them. Indeed, research focusing on television (Horvath 2004), internet (Caplan 2010), and smartphone (van Deursen et al. 2015) use has sought to understand these behaviors in terms of addiction. As previously discussed, natural rewards release dopamine through activation of the mesolimbic system to promote survival and maintenance of the species. Like for substance use disorders, could technology equally hijack the reward system? Social media platforms leverage the reward system in ways similar to what gambling does to promote usage as much as possible through activation of the dopaminergic pathways (Izuma et al. 2008). Evidence has recently been presented which connects successful social interactions and the dopaminergic mesolimbic system (Torquet et al. 2018).

Growing evidence suggests that electronic devices can negatively impact circadian rhythms. Studies recently emerged have linked smartphone usage to increased anxiety and depression as well as poor sleep quality (Demirci et al. 2015). Indeed, lights from backlit screens can delay and advance circadian timing causing asynchronization (Blume et al. 2019). Associations between loss of sleep and electronic media exposure have been extensively reported in adolescents and adults (Suganuma et al. 2007; Fossum et al. 2014; Lemola et al. 2015). The alerting effects of night time use of electronics could be due to the suppression of melatonin by blue light exposure from the device in the retina (West et al. 2010). This emphasizes the utility of using “night shifting” modes, which switch displays to decreased blue light. Beyond the effects light has on sleep disruption, nighttime use of electronics and media engagement likely has profound effects on circadian rhythms. This effect can in part be ascribed to the release of dopamine in the striatum from the rewarding nature of social interactions. It is clear that more studies are required to trace the links between dopamine-mediated reward and circadian rhythms. While the connection is warranted, the molecular mechanisms that define dopamine-circadian interactions and their consequences on our health are still in their infancy.