Neural Circuits Linking Sleep and Addiction: Animal Models to Understand Why Select Individuals Are More Vulnerable to Substance Use Disorders After Sleep Deprivation

Abstract

Individuals differ widely in their drug-craving behaviors. One reason for these differences involves sleep. Sleep disturbances lead to an increased risk of substance use disorders and relapse in only some individuals. While animal studies have examined the impact of sleep on reward circuitry, few have addressed the role of individual differences in the effects of altered sleep. There does, however, exist a rodent model of individual differences in reward-seeking behavior: the sign/goal-tracker model of Pavlovian conditioned approach. In this model, only some rats show the key behavioral traits associated with addiction, including impulsivity and poor attentional control, making this an ideal model system to examine individually distinct sleep-reward interactions. Here, we describe how the limbic neural circuits responsible for individual differences in incentive motivation overlap with those involved in sleep-wake regulation, and how this model can elucidate the common underlying mechanisms. Consideration of individual differences in preclinical models would improve our understanding of how sleep interacts with motivational systems, and why sleep deprivation contributes to addiction in only select individuals.

Sleep disturbances can lead to, or exacerbate, a multitude of psychological disorders involving impulse control, behavioral inhibition, and addiction. Even partial sleep deprivation alters multiple aspects of executive function (Tkachenko and Dinges 2018), and chronic insomnia is linked to an increased risk of alcohol and substance use disorders (Marmorstein 2017; Stein and Friedmann 2006) and obesity (Katsunuma et al. 2017). The causal, mechanistic relationship between sleep and addictive disorders is difficult to study in human clinical populations. This is because a history of drug or alcohol consumption results in long-term alterations in sleep during active use, during withdrawal, and even after years of abstinence (Knapp et al. 2007; Knapp et al. 2014). Therefore, it is difficult to determine whether underlying sleep-related traits contribute to the initial development of substance use disorders, or whether sleep disturbances are the result of past exposure to drugs or alcohol. There are two important questions that are essential for understanding how sleep loss can lead to altered reward processing. First, are there underlying pre-existing differences in sleep characteristics that can predispose some individuals to either the initial development of addictive tendencies or to relapse? Second, are there individual differences in how the consumption of addictive substances, or a state of physical dependence, interacts with neural architecture to cause distinct post-addiction sleep patterns across individuals? The development of an animal model that can address these questions would be a major step toward understanding the impact of sleep quality on reward processing and addiction-related disorders.

1. The importance of studying individual variation

The study of sleep and cognitive functioning is strongly influenced by individual differences in human subjects. Several studies have looked at different aspects of cognitive processing after experimentally induced sleep deprivation, often focusing on different aspects of sustained attention, subjective ratings of sleepiness and mood, and performance in cognitive tasks. Results are strongly influenced by individual variability, which in many cases is stable over repeated testing with different SD parameters, suggesting that stable underlying behavioral traits determine why some people are more vulnerable to the negative effects of SD than others (reviewed in (Tkachenko and Dinges 2018; Van Dongen et al. 2005).

Several limbic brain regions play a key role in both reward and sleep. The dopamine-mediated mesolimbic circuitry responsible for reward and reinforcement is also heavily involved in the regulation of sleep/wake states and is strongly affected by sleep loss. In humans, a single night of sleep deprivation can decrease D2/D3 dopamine receptor availability in the ventral striatum (Volkow et al. 2008; Volkow et al. 2012; Wiers et al. 2016), which is associated with a greater propensity for risk-taking behavior (Linnet et al. 2011) and an increased risk for compulsive drug consumption (Dalley et al. 2007). Furthermore, sleep disturbances have been shown to mediate the reduced D2/D3 receptor availability that has been observed in chronic cocaine abusers (Wiers et al. 2016). However, in human populations, there is tremendous individual variation in the degree to which sleep deprivation impairs cognitive performance and enhances reward sensitivity. For example, genetic variation in the human dopamine transporter (DAT) gene has been shown to influence neural responses to sleep loss; with individuals with the DAT allele that is linked to higher phasic dopamine activity demonstrating greater striatal responses to monetary reward after sleep deprivation (Greer et al. 2016).

A recent study in humans by Satterfield et al. showed that the degree of activation of the nucleus accumbens (NAcc) by a task (but not pre-task baseline NAcc activity) predicted the number of calories consumed during a sleep deprivation session that took place days later. This suggests that stronger mesolimbic dopamine reactivity, when combined with the alterations in prefrontal functioning that result from sleep loss, can make some individuals more likely to overeat than others (Satterfield et al. 2018). This dynamic may be critically important for the regulation of motivated behavior. If sleep loss causes deficits in the “top-down” prefrontal regulation of behavior, it is likely to have greater consequences for those individuals that experience stronger “bottom-up” motivational drive to begin with.

Preclinical studies in rodents are frequently used to investigate the relationship between sleep and reward processing. However, the impact of individual variation has rarely been addressed in these models, and it would be beneficial to use the individual differences that are naturally found in animal populations to better understand the potential sources of variability in human populations. In particular, we believe the sign-tracker/goal-tracker model of incentive salience has translational relevance that cannot be captured by other models, and can provide a unique perspective on how sleep disruption can enhance reactivity to reward-paired cues. The balance between top-down and bottom-up circuitry has specifically been implicated as a mechanism underlying sign- and goal-tracking propensities (Haight et al. 2017); therefore, if humans show individual variability in how strongly the dynamic between these two processes is altered by sleep deprivation, then the ST/GT model may be a powerful tool for isolating the neural mechanisms responsible for this interaction. Here, we review evidence for a link between the brain regions involved in sleep and those responsible for the motivational impact of reward cues. We argue that by examining populations of rats that show natural phenotypic variation in the degree to which food cues engage mesolimbic circuitry, we can learn more about the role of sleep in the emotional and motivational states that are triggered by cues.

2. Studies of sleep deprivation (SD) and addictive behavior

Sleep deprivation has a major impact on reinforcement learning, often resulting in hypersensitivity to reward and cue-induced motivation. SD has been shown to enhance the sensitizing effects of amphetamine (Kameda et al. 2014), increase the drug-primed reinstatement of conditioned place preference for methamphetamine (Karimi-Haghighi and Haghparast 2018), and enhances the acquisition of cocaine self-administration and the rate of responding on a progressive ratio schedule (Puhl et al. 2013). In addition, the fragmentation of REM sleep that is caused by cocaine withdrawal expedites the development of the incubation of cocaine craving (Chen et al. 2015). SD also enhances motivation for food reward, and selectively increases the consumption of sucrose and highly palatable food, but not regular lab chow (Liu et al. 2016; McEown et al. 2016).

Sleep loss is known to cause major impairments in hippocampal-dependent memory, with several studies showing that SD prior to learning prevents the formation of new memories, and SD after learning impairs the consolidation of newly formed memories (Kreutzmann et al. 2015). Impairment in the hippocampus and prefrontal cortex may also contribute to reward sensitivity, since SD impairs the type of complex cognitive processes that are mediated by these areas, such as spatial and contextual learning (Hagewoud et al. 2010a; Kreutzmann et al. 2015; McDermott et al. 2003). For example, the ability to withhold an operant response in order to receive a reward is reduced after SD, indicating reduced prefrontal inhibition and an increase in impulsive responding (Kamphuis et al. 2017). In some cases, even when sleep deprivation does not directly impair performance, it can influence the type of learning strategy recruited to perform a task, with the enhanced activity in striatal circuits acting as a compensatory mechanism to preserve performance on reward-related tasks that would normally engage hippocampal learning processes (Hagewoud et al. 2010b; Watts et al. 2012).

There is a bidirectional relationship between circadian rhythms and reward-related behavior, and given that circadian rhythms play such an integral role in regulating sleep, it is likely that individual traits related to circadian mechanisms play a role in the interaction between sleep and substance abuse (DePoy et al. 2017). Polymorphisms in circadian genes have been shown to contribute to addiction-related behaviors. For example, in mice, mutations of the clock gene can lead to a hyperdopaminergic state with elevated dopamine transmission in the VTA and NAcc, leading to enhance cocaine self-administration and conditioned place preference (McClung et al. 2005; Ozburn et al. 2012), and mutations in Per1 and Per2 can increase the consumption of alcohol (Dong et al. 2011; Spanagel et al. 2005b). In addition, rodent lines that are selectively bred for high or low alcohol preference (P/NP lines and HAD-2/LAD-2 lines) also show phenotypic variation in circadian rhythms, indicating that common genetic determinants underly the two processes (Rosenwasser et al. 2005). In humans, polymorphisms in clock genes are more prevalent in populations with alcohol use disorders (Kovanen et al. 2010). Polymorphisms in the Per2 gene have been associated with the regulation of alcohol consumption (Spanagel et al. 2005a), and reduced dopamine D2 receptor expression in the striatum (Shumay et al. 2012). Furthermore, circadian misalignment that occurs due to external factors, such as jet lag and shift work, is associated with a higher rate of substance abuse and is known to increases the risk of overeating and the consumption of high calorie food (Mendoza 2019; Partonen 2015; Webb 2017).

The link between circadian rhythms and addictive behavior is complicated by the fact that exposure to alcohol, drugs of abuse, and food reward, can directly cause disruption or entrainment of circadian timing (Gillman et al. 2019; Hasler et al. 2012; Webb 2017). Therefore, as with other aspects of sleep, it is not yet known which features of circadian rhythms represent underlying predisposing traits, and which results from drug exposure or environmental factors. Furthermore, sleep and circadian rhythms are closely intertwined. The timing of sleep is largely under the control of circadian mechanisms, while sleep deprivation inevitably causes disruption of normal circadian rhythmicity (Rosenwasser 2009). Therefore, when evaluating the effect of sleep manipulations on addictive behavior, it is difficult to parse the impact of SD alone (i.e. total amount of sleep time lost) from the affect this would have on circadian rhythms.

3. Individual differences in the attribution of incentive salience to cues

It has long been known that a food-paired cue (conditioned stimulus; CS) that has been repeatedly paired with food reward will reliably elicit a conditioned response in rats. However, it is also true that the form of the conditioned response can vary due to individual differences, with some rats approaching and interacting with the cue itself (“sign trackers”, STs) and some approaching the site of impending food delivery (“goal trackers”, GTs). In the standard conditioning procedure used to determine sign- and goal-tracking tendencies (Flagel et al. 2009; Meyer et al. 2012; Tomie et al. 2012), a retractable lever is used as the CS. In each trial (i.e. CS-reward pairing) the lever-CS is inserted into the cage for 8s, then is retracted and a banana pellet is immediately dispensed. The reward is delivered non-contingently and is independent of any action by the rat. Rats are categorized as STs or GTs based on whether they preferentially contact the lever or the food cup during the performance of their conditioned response (Meyer et al. 2012).

For both STs and GTs the CS acquires predictive value; however, only for STs does the CS also acquire incentive value, which causes STs, but not GTs, to become attracted to the lever and interact with it when it is present. By predictive value, we mean the learning of associations and the cognitive expectation of reward; in other words, an animal understands that the CS predicts the reward and reacts with a conditioned response. By incentive value, we mean that not only does a cue elicit the cognitive expectation of reward; it also elicits a dopamine-mediated motivational state akin to craving, which in rats can be expressed as desire for the cue itself (Flagel et al. 2009; Robinson et al. 2014; Saunders and Robinson 2013; Singer et al. 2016a; Singer et al. 2016b). This tendency to attribute incentive salience to a CS makes STs more susceptible to the motivational attraction of cues than GTs (Flagel et al. 2009; Saunders and Robinson 2013). As a result, STs work harder than GTs to gain access to the CS in a conditioned reinforcement paradigm (Beckmann and Chow 2015; Lomanowska et al. 2011; Robinson and Flagel 2009), STs are more resistant to Pavlovian extinction than GTs (Ahrens et al. 2016b), and discrete cues elicit greater reinstatement of food- and drug seeking behaviors in STs than GTs (Saunders and Robinson 2010, 2013; Yager and Robinson 2010, 2013).

Genetic differences underlie many of the traits that predispose STs to be more attracted to cues than GTs. Selective breeding for addiction-related traits also co-selects for the associated ST versus GT tendencies (Flagel et al. 2010), and certain commercial vendor colonies are more likely to produce STs than others (Fitzpatrick et al. 2013). The differences between STs and GTs are associated with other psychological tendencies that are not directly related to cue responses but may contribute to individual vulnerability to addiction. Compared to GTs, STs are more impulsive (Lovic et al. 2011), have diminished attentional control and reduced cholinergic activity in the prefrontal cortex (Koshy Cherian et al. 2017; Paolone et al. 2013), show greater locomotor reactivity to a novel environment (Flagel et al. 2010), show altered dopamine regulation even in the absence of rewarding stimuli (Flagel et al. 2010; Singer et al. 2016b), show greater expression of conditioned fear (Morrow et al. 2011; Morrow et al. 2015), and are more susceptible to incentive motivation during adolescence compared to adulthood (DeAngeli et al. 2017).

Sleep disturbances can strongly influence the expression of many of the behaviors that differ between STs and GTs, such as attentional control and impulsivity (Pilcher et al. 2015), and can alter dopamine activity (Volkow et al. 2008; Volkow et al. 2012; Wiers et al. 2016) and responses to drug-paired cues (Chen et al. 2015; Puhl et al. 2013; Volkow et al. 2012). However, the direct relationship between sleep and ST/GT behavior has never been studied. Given the existing literature, we predict that SD will impact these two groups differently, particularly with regard to cue- or drug-induced motivation, as well as other measures of reward-seeking behavior. Given the dramatic differences in their cholinergic and dopaminergic circuitry (Flagel and Robinson 2017; Pitchers et al. 2017), it is also likely that STs and GTs will show differences in their baseline sleep duration and sleep architecture, as well as differences in the precise nature of sleep disturbances caused by drug exposure or environmental stressors.

One important consideration is whether the rodent stocks or strains that show individual variability in ST/GT behavior also show variability in circadian rhythms or sleep patterns. Studies of Pavlovian conditioned approach have frequently been performed in relatively homogenous rodent populations (Fitzpatrick et al., 2013); however, there is the option to use more genetically diverse populations which could provide a wide range of individual variability in phenotypic traits. For example, recent studies have used heterogenous stock (HS) rats to examine behavioral correlates of ST/GT traits. HS rats are a genetically diverse colony created from eight inbred founder strains (Solberg Woods and Palmer 2019), which can provide more powerful insight into the relationships between genetically determined traits than inbred strains. Some of the traits that have previously been linked to sign-tracking in inbred populations (like premature responding in choice reaction time tasks) are still correlated within HS rat populations (King et al. 2016), while other traits (such as “novelty seeking” and “sensation seeking”) are not (Hughson et al. 2019); demonstrating the need for further investigation into genetic correlates.

Another option is to use genetically diverse populations of mice, such Collaborative Cross (CC) and Diversity Outbred (DO) strains. It was thought that PCA was not feasible in mice after it was shown that C57BL/6J mice did not develop the clear ST/GT phenotypes seen in rats (Tomie et al. 2012). However, sign-tracking in mice varies depending on strain and sex, with C57BL/6J mice showing much lower rates of sign-tracking than some of the CC and DO founder strains (Dickson et al. 2015). Therefore, it is possible to use the ST/GT model in genetically diverse populations of rats and mice to allow precise correlations between the genetic processes underlying cue responses and those involved in various sleep-related traits.

We next discuss the precise neural circuits (Fig. 1; Table 1) that are simultaneously involved in regulating sleep/wake states as well as encoding the responses to drug- and reward-related cues. As we point out, many of these structures have already been found to be differentially activated in sign-trackers vs goal-trackers, further highlighting their potentially crucial role in explaining individual differences linking sleep and addiction.

Figure 1.

Overlapping circuits for sleep/wake regulation and drug/reward-related encoding. There is substantial overlap in the neural pathways involved in the attribution of incentive salience to cues (red) and those that mediate the effects of sleep disturbances on motivated behavior and reward seeking (blue). Solid lines represent pathways that have been directly studied in these functions, and dashed lines represent connections that are hypothesized to play a role. There are individual differences in the degree to which rats are susceptible to the incentive motivational effects of reward cues, with some rats (STs) demonstrating stonger attraction to cues than others (GTs). Sign- and goal-tracking behavior are associated with different patterns of activity in mesolimbic circuitry, most notably expressed as greater activity in dopaminergic VTA projections (thick lines) and reduced activity in PVT and mPFC projections (thin lines) in STs relative to GTs. Since much of the same circuitry also plays a critical role in the ability of sleep to influence emotional and motivational states, this ST/GT model should reveal important information about how sleep affects reward processing, and how the loss of sleep can enhance the ability of reward cues to gain control over behavior. The regions shown represent pathways involved specifically in the ability of sleep to alter motivation for reward; brainstem mechanisms of sleep-wake regulation are not shown. Abbreviations: mPFC – medial prefrontal cortex; PVT – paraventricular nucleus of the thalamus; HPC – hippocampus; NAcc – nucleus accumbens; VP – ventral pallidum; VTA – ventral tegmental area; SCN – suprachiasmatic nucleus; AMG – amygdala.

Table 1.

Pathways with overlapping roles in motivational responses to cues and the effects of sleep deprivation on reward-related behavior.

Pathways	Role in motivation	Effect of sleep deprivation	References
VTA – NAcc – VP	Attribution of incentive salience to cues (STs > GTs)	Hyperactivity and enhanced reward seeking	Flagel et al., 2017; Satterfield et al., 2019
PFC – PVT – NAcc	Predictive value of reward-paired cues (GTs > STs)	Weakened inhibitory control over reward seeking	Haight et al., 2017; Liu et al., 2016
AMG – NAcc	Amplification of conditioned responding without changing the target of approach	Increase in the subjective valuation of food cues	DeFeliceantonio and Berridge 2012; Rihm et al., 2018

4. Role of the mesolimbic system in incentive motivation and sleep

4.1. Ventral tegmental area (VTA) and nucleus accumbens (NAcc)

Dopamine activity in the pathway from the VTA to the NAcc is well known to be a crucial mediator of reward and reinforcement. However, there is debate about the exact role that dopamine plays in reward; whether it encodes hedonic pleasure and euphoria, reward-prediction, or motivational salience (Berridge 2007). There is a compelling argument that dopamine specifically encodes the state of incentive motivation, much of which comes from evidence that dopamine signaling is important for sign-tracking but not goal-tracking behavior (Berridge 2007; Flagel et al. 2011; Flagel and Robinson 2017). For example, reward-paired cues elicit greater dopamine release in the NAcc in STs than GTs (Flagel et al. 2011) and STs have greater surface expression of the DA transporter in the NAcc than GTs (Singer et al. 2016b). Although disruption of activity in the NAcc-shell does not impair sign-tracking acquisition (Chang and Holland 2013; Chang et al. 2018), disruption of activity in the NAcc-core will reduce approach to the lever in STs, but not approach to the food cup in GTs (Chang et al. 2012b; Flagel et al. 2011; Fraser and Janak 2017; Saunders and Robinson 2012). Thus, it appears that cues must be attributed with incentive salience to engage dopamine-mediated reward circuitry, and that the predictive value of cues is not sufficient (Flagel et al. 2011; Yager et al. 2015).

Key structures in the mesolimbic system are also activated during sleep and play a vital role in the reprocessing and encoding of memories with high emotional or motivational content (Oishi and Lazarus 2017; Perogamvros and Schwartz 2012; Perogamvros et al. 2013). There has been some debate about the role of the VTA in sleep (Oishi and Lazarus 2017) since early electrophysiological studies found that VTA dopamine neurons did not change firing rates across sleep-wake states (Miller et al. 1983). However, in more recent studies, single unit recordings have shown that burst firing patterns in the VTA, but not mean firing rates, differ between REM sleep, NREM sleep, and wakefulness, with burst firing observed during REM sleep that is similar to activity patterns seen during the consumption of food reward (Dahan et al. 2007). Consequently, dopamine concentrations in the NAcc are higher during waking and REM sleep compared to NREM sleep (Eban-Rothschild et al. 2016; Lena et al. 2005). In addition, it is well known that stimulant drugs that increase dopamine concentrations are powerful promoters of wakefulness (Boutrel and Koob 2004). There is also recent evidence that VTA dopaminergic neurons are directly involved in promoting wakefulness in response to environmental stimuli. Optogenetic or chemogenetic activation of VTA neurons initiates and maintains wakefulness despite high homeostatic sleep pressure, and this effect is primarily mediated by projections to the NAcc (Eban-Rothschild et al. 2016; Oishi et al. 2017). Finally, there is evidence that mesolimbic dopamine activity also plays an important role in the generation of dreams (Feld et al. 2014), and it has been suggested that the dopaminergic forebrain pathway plays a larger role in dreaming than the cholinergic brain stem mechanisms that trigger REM sleep (Solms 2000). Since the same pathway from the VTA to the NAcc is critically involved in both the attribution of incentive salience to cues and the regulation of sleep-wake states, it is likely that individual differences between STs and GTs will also be expressed as differences in dopaminergic control of sleep and wakefulness.

4.2. Hippocampus (HPC)

The hippocampus is known to play a critical role in learning, as well as the consolidation of memories during sleep (Dudai 2004). Network connectivity between the NAcc and the hippocampus may also play an important role in mediating the effects of SD on reward-related behavior and motivation. The encoding of memory by the hippocampus depends on the reactivation of specific experience-related neural firing sequences during NREM sleep, and if this neuronal replay is interrupted by sleep loss it impairs the subsequent recall of spatial and contextual memories (Chen and Wilson 2017). A similar process of spontaneous replay has been observed in the ventral striatum during NREM sleep following performance of a reward-related task (Ahmed et al. 2008; Lansink et al. 2008; Pennartz et al. 2004), and this reactivation is critical for the encoding of reward-related memories, such as the spatial location of food reward (Lansink et al. 2012). Replay sequences in the NAcc can be triggered by sharp wave ripples in the hippocampus (Singer and Frank 2009) and replay in the NAcc is dominated by pairs of neurons in which hippocampal “place” cells fire immediately before the reward-related NAcc neuron (Lansink et al. 2009). Therefore, joint reactivation of hippocampal and NAcc firing patterns represents an important mechanism for consolidation of place-reward associations, and may be particularly vulnerable to disruption by SD.

Reward encoding can also take place within the hippocampus itself. Place cell firing fields accumulate near goal locations (Hollup et al. 2001), and there are dedicated populations of neurons in the HPC that specifically encode proximity to reward (Gauthier and Tank 2018). The ventral region of the HPC plays a particularly important role in reward processing. The ventral HPC sends prominent glutamatergic projections to the NAcc which are responsible for carrying spatial information to the NAcc and are critical for linking reward learning with contextual information (Britt et al. 2012; Lansink et al. 2008; Lansink et al. 2009). For example, the learning of context-drug associations selectively strengthens the connection between ventral HPC place cells and medium spiny neurons in the NAcc (Sjulson et al. 2018), and disruption of this pathway by inactivation of the ventral (but not dorsal) HPC impairs the retrieval of contextual reward memory (Riaz et al. 2017).

The ventral HPC also plays an important role in sign-tracking. One study found that lesions of the ventral HPC, but not dorsal HPC, impaired the initial learning of a sign-tracking response (Fitzpatrick et al. 2016a). In another study, STs were found to have elevated myo-inositol (a marker of glial activity and proliferation) in the ventral (but not dorsal) HPC relative to GTs (Fitzpatrick et al. 2016b). Therefore, individual differences in the HPC inputs to the NAcc could be a contributing factor in the development of ST versus GT behavioral responses. It is possible that differences in hippocampal ripple-triggered activity in the NAcc during sleep may play a critical role in how some individuals develop stronger incentive motivational associations with reward cues than others. Examination of this connection in the ST/GT model would be a critical first step in understanding the importance of the HPC in the attribution of incentive salience to cues.

4.3. Ventral pallidum (VP)

The VP has received less attention than the VTA and the NAcc, but also plays an important role in reward and reinforcement. The VP is the primary output structure for mesolimbic reward circuitry. It is heavily innervated by the GABAergic medium spiny neurons in the NAcc (Creed et al. 2016; Ho and Berridge 2013; Kupchik et al. 2015; Root et al. 2015), and projects back to the VTA and to several areas involved in the regulation of movement (Root et al. 2015; Zahm 2000). Due to these connectivity patterns, the VP is thought to be a primary hub where motivational output from the NAcc is translated into appetitive behavior (Smith et al. 2009); however, there is also evidence for bi-directional communication between the NAcc and VP, as cue responses in the VP sometimes precede and drive those in the NAcc (Chang et al. 2018; Richard et al. 2016).

The VP is a heterogeneous structure, with rostral-caudal differences in cell morphology and connectivity patterns (Kupchik and Kalivas 2013; Root et al. 2015; Zahm 2000). For example, there are topographic differences in projection patterns, with the anterior VP receiving projections from the NAcc shell and the posterior VP receiving projections from the NAcc core (Kupchik et al. 2015; Root et al. 2015). The functional differences between anterior and posterior regions are not well understood; however, some studies have found that they play different roles in modulating reward-related behavior (Root et al. 2010; Root et al. 2013), and have even been shown to have opposite effects on hedonic responses to food reward (Smith and Berridge 2007; Smith et al. 2009).

Several studies have shown that neurons in the caudal VP respond to food cues, with the magnitude of the response reflecting the strength of the cue’s motivational impact (Avila and Lin 2014a, b; Smith et al. 2011; Tachibana and Hikosaka 2012; Tindell et al. 2005; Tindell et al. 2006). The VP has also been shown to specifically encode the incentive value of a cue in a way that can be experimentally dissociated from reward prediction (Smith et al. 2011; Tindell et al. 2005; Zhang et al. 2009). For example, chemogenetic inactivation of the VP can impair the acquisition (but not expression) of sign-tracking behavior, while leaving goal-tracking unaffected (Chang et al. 2015). Importantly, the VP is the only structure where differences in single-unit neural activity have been documented between STs and GTs. In two previous studies, STs have shown sustained changes in neural activity during exposure to the lever cue that are greater, in terms of proportion of responsive cells and the magnitude of responses, than that of GTs (Ahrens et al. 2016a; Ahrens et al. 2018). The heightened VP activity in STs was specifically evoked by the lever cue. When the same animals were trained with a tone cue that predicted identical reward, but did not support the attribution of incentive salience, the tone did not elicit the robust changes in neural activity that were seen with the lever. Therefore, not only does the VP reflects individual differences in motivational tendencies, it tracks dynamic changes in incentive value of cues as they change from trial to trial within a single animal (Ahrens et al. 2018). Few studies have specifically focused on the role of the VP during sleep; however, the VP has been examined as part of the larger basal forebrain region, which has been shown to play a very important role in mediating both sleep and waking states (Jones 2017; Yang et al. 2017). The basal forebrain describes a large area that encompasses the VP in addition to other subcortical structures, such as the medial septum, bed nucleus of the stria terminalis, substantia innominata, magnocellular preoptic nucleus, and extended amygdala (Yang et al. 2017). The basal forebrain contains a mix of cholinergic, glutamatergic, and GABAergic cells that co-express different calciumbinding proteins. Among these cell types four different functional activity patterns have been identified. The most common type (~50%) are cortically-projecting cells that show maximal firing during waking and REM sleep, but not NREM sleep (Jones 2017), and when optogenetically stimulated produces a rapid desynchronization of EEG and an increase in wakefulness (Irmak and de Lecea 2014; Xu et al. 2015). The cholinergic neurons almost exclusively fall in this wake-promoting category (Lee et al. 2005), as do most glutamatergic neurons and some parvalbumin-positive GABAergic neurons (Hassani et al. 2009). A second type (~20%) are sleep-active, meaning they respond more during NREM sleep than during active brain states. Most these neurons are somatostatin-positive GABAergic neurons, with some glutamatergic neurons, and they project primarily to the prefrontal cortex (Hassani et al. 2009; Xu et al. 2015). The third type is relatively infrequent (~10%) and are glutamatergic neurons that respond maximally during waking. The fourth type (~20%) is maximally responsive during REM sleep, but not waking. These are a mix of GABAergic and glutamatergic cells that project primarily to the posterior hypothalamus (Jones 2017). Although basal forebrain neurons have been well characterized in the context of sleep and wakefulness, it is not known whether there are individual differences in the composition or function of these different cell types. It is also not known whether the VP itself shares all of the same characteristics as this larger basal forebrain region. Finally, further research is needed to determine what role the VP plays (if any) on the ability of sleep to alter reward seeking behavior.

4.4. Medial prefrontal cortex (mPFC)

The mPFC is a key component of the mesolimbic reward system. It is involved in the evaluation of the salience and motivational significance of reward-paired cues, and in selection and initiation of motivated actions (Moorman and Aston-Jones 2015; Moorman et al. 2015). Two important areas of the mPFC, the prelimbic and infralimbic regions, have very different projection patterns and can play different roles in reward-driven behavior. Although both are connected with several areas that are involved in motivation and emotion, such as the VTA, hippocampus, and amygdala, and paraventricular nucleus of the thalamus (PVT) (Li and Kirouac 2012; Vertes 2004), there are especially prominent projections to the NAcc. The prelimbic cortex projects primarily to the NAcc core, and the infralimbic cortex projects primarily to the NAcc shell (Vertes 2004). These pathways have been shown to have opposite effects on a range of motivated behaviors, such as cocaine self-administration (LaLumiere et al. 2010; Peters et al. 2008; Peters et al. 2009), sucrose reinforcement (Peters and De Vries 2013), and conditioned fear (Maren and Quirk 2004; Peters et al. 2009). The prelimbic cortex is important for the acquisition of excitatory conditioning (Meyer and Bucci 2014), and often acts as a “go” signal that instigates reward seeking; whereas the infralimbic cortex is important for the expression of well-learned inhibitory behavior, acting as a “stop” signal that suppresses previously learned conditioned responses (LaLumiere et al. 2010; Peters et al. 2009; Peters and De Vries 2013). However, other studies suggest that the function of the mPFC is more complex than this simple dichotomy (Moorman et al. 2015); for example, the prelimbic cortex has also been shown to inhibit dominant responses in favor of more adaptive, goal-driven behavior (Meyer and Bucci 2014).

In addition to its role in the expression of reward-seeking, the mPFC is important for the consolidation of reward-related memories during sleep. For example, the ability of sleep deprivation to enhance sucrose seeking and consumption is associated with a selective weakening of the glutamatergic pathway from the mPFC to the NAcc (Liu et al. 2016). The mPFC also has strong functional connections with the hippocampus. It receives direct projections from the ventral hippocampus CA1 (Adhikari et al. 2010; Hoover and Vertes 2007), and studies that have recorded from both the mPFC and hippocampus have found correlations between spike times in the two regions, as well as coherent theta rhythms (Adhikari et al. 2010; Benchenane et al. 2010; Colgin 2011). Furthermore, the mPFC and hippocampus reactivate together during slow-wave sleep, with synchronous bursts of activity occurring in both structures during sharp-wave ripples in the hippocampus (Colgin 2011; Wierzynski et al. 2009).

The mPFC also mediates aspects of executive function that may be particularly relevant to STs and GTs, such as impulsivity and attentional control. It has been shown that STs have low levels of cholinergic activity in the mPFC relative to GTs, and that this causes poor attentional control in STs compared to GTs (Paolone et al. 2013). At the same time, STs have greater dopamine responses to cues in the mPFC than GTs, which couple with low cholinergic activity, is thought to contribute to the reduced “top-down” control of behavior seen in STs relative to GTs (Pitchers et al. 2017). Furthermore, as mentioned above, projections from the prelimbic cortex to the PVT are thought to mediate behavioral control in GTs (Haight et al. 2017). Therefore, given that the mPFC plays such a prominent role in multiple reward-seeking paradigms, consolidation of reward memories during sleep, and individual differences in incentive salience attribution, we believe this this is an area where fundamental phenotypic differences in ST/GT neurocircuitry are very likely to be observed.

4.5. Amygdala (AMG)

The AMG is an essential part of the mesolimbic circuitry; however, there is limited information on the role the AMG plays in ST/GT behavior. It does not appear to be essential for the initial attribution of incentive salience to cues, but it may act to amplify incentive value once it has been acquired, possibly through dense glutamatergic projections from the amygdala to the NAcc (Britt et al. 2012; Stuber et al. 2011). In one study, opioid stimulation of the central nucleus of the amygdala enhanced the intensity of conditioned responses without changing the target of approach, causing STs to show stronger sign-tracking and GTs to show stronger goal-tracking (DiFeliceantonio and Berridge 2012). In another study, lesions of the basolateral amygdala reduced the rate of lever pressing in STs after extended training, and disconnection of the basolateral amygdala and the NAcc produced deficits in both the acquisition of sign-tracking and in the rate of responding in trials when sign-tracking occurred (Chang et al. 2012b). In contrast, lesions of the central nucleus of the amygdala had no effect on the acquisition or expression of sign-tracking behavior (Chang et al. 2012a).

The amygdala is also involved in the ability of sleep deprivation to enhance the motivational effects of food cues. In a recent human study, a single night of sleep deprivation increased the subjective valuation of food cues and caused a parallel increase in activity in the amygdala and hypothalamus (Rihm et al. 2018). In another study, subjects that experienced sleep debt in daily life demonstrated elevated amygdala reactivity to food cues, which was reduced after optimal sleep (Katsunuma et al. 2017). Therefore, the increased activity in the amygdala that results from suboptimal sleep may act to amplify the incentive motivation that is triggered by reward-paired cues, and contribute to the heightened reward-seeking behavior that often follows sleep deprivation.

4.6. Paraventricular nucleus of the thalamus (PVT)

The PVT is a thalamic midline structure with numerous connections to cortical, limbic, and motor structures (Kelley et al. 2005; Li and Kirouac 2012), including dense projections to the NAcc, prelimbic and infralimbic cortices, and amygdala (Vertes and Hoover 2008). Recent studies have found that the PVT may be one of the key structures mediating incentive versus predictive cue responses seen in STs and GTs. Under normal conditions the PVT appears to suppress the attribution of incentive salience to cues and plays a role in preventing GTs from expressing attraction to cues. For example, disruption of PVT activity has been shown to increase sign-tracking behavior and decrease goal-tracking behavior, causing rats previously identified as GTs to switch to sign-tracking (Haight et al. 2015). Furthermore, disruption of the PVT increases cue-induced reinstatement of drug seeking in GTs to the level normally seen in STs (Kuhn et al. 2017). A recent dual-labeling study (c-fos and flourogold) found that in both STs and GTs a food-paired cue activated the projection from the prelimbic cortex to the PVT, suggesting that this pathway mediates the predictive value of the cue, which both STs and GTs experience equally. However, in STs, the cues also activated subcortical pathways from the hypothalamus and amygdala to the PVT, as well as projections from the PVT to the ventral striatum, suggesting that these connections are involved in processing the incentive value of a cue. Therefore, the prelimbic-to-PVT pathway is hypothesized to be part of an inhibitory mechanism by which GTs exert greater cortical “top-down” control over motivated behavior and react primarily to the predictive value of a cue. The lack of this inhibition causes STs to act more on “bottom-up” emotional impulses driven by subcortical circuitry (Haight et al. 2017).

In addition to its prominent role in incentive motivation, the PVT is also ideally situated to play an important role in modulating sleep-wake states. The PVT receives synaptic inputs from, and projects back to, the suprachiasmatic nucleus (SCN), which is the master clock that regulates circadian rhythms in mammals (Alamilla et al. 2015; Colavito et al. 2015; Moga et al. 1995; Peng and Bentivoglio 2004; Vertes and Hoover 2008). Through its dense projections to limbic areas, the PVT can relay information about circadian rhythms from the SCN to the NAcc, amygdala, infralimbic and prelimbic cortices (Vertes and Hoover 2008). In addition, axon terminals of SCN fibers terminate on PVT neurons projecting to the amygdala (Peng and Bentivoglio 2004). Many of these connections are reciprocal, and since the PVT projects back to the SCN it can mediate the ability of behavioral arousal and attentive states to alter circadian rhythms. For example, inputs from the PVT can shift membrane potential in SCN neurons and make them more responsive to external light cues transmitted through the retinohypothalamic tract (Alamilla et al. 2015). Therefore, the PVT is in an ideal position to relay information about circadian timing from the SCN to brain regions involved in motivation aspects of behavior, and to also provide regulatory feedback to the SCN.

4.7. Other relevant brain regions

There are other brain areas that are potentially involved in the regulation of both sleep and reward in addition to those discussed above. For example, the habenula is likely to play a role, since it has been suggested to regulate specific aspects of sleep, namely motor suppression and the generation of REM sleep (Hikosaka 2010). It is also involved in reinforcement learning, and is thought to mediate the suppression of behavior when faced with failure to receive an expected reward or the avoidance of punishment (Hikosaka 2010). The lateral habenula also plays a role in symptoms of depression and in the processing of negatively-valenced information. It is hyperactive in individuals suffering major depression and shows greater synaptic activity during learned helplessness in rats, which can be reversed with antidepressant treatment (Yang et al. 2018).

Another potential area of investigation is the locus ceruleus, which ennervates the majority of the brain and regulates norepinephrine levels. The locus ceruleus is relevant for sleep as norepinephrine is one of the major wake-promoting neurotransmitters in the brain, and is important for attention and vigilance (Hofmeister and Sterpenich 2015). It is also thought to be involved in the negative, stress-inducing effects of drug withdrawal (Belujon and Grace 2011). Therefore, although the roles of the habenula and locus ceruleus are less well characterized in the context of addicition than other mesolimbic areas, it is likely that variability in the functioning of these areas could contribute to the individual differences seen in both sleep characteristics and reward-seeking behavior.

5. Critical need for the study of individual differences in sleep-reward circuitry

There is significant overlap between the neural circuitry controlling the motivation for reward and the circuitry controlling sleep-wake states (Fig. 1). Given the many points of interaction between these systems, it is easy to see why these behavioral states are so closely intertwined, and how acute or chronic sleep disturbances can cause such profound changes in the consumption of food, drugs, and alcohol. Individual differences are a major concern for the study of both addiction and sleep disorders. There is tremendous individual variation in how these conditions develop, the effects they have on health and social functioning, and how they respond to treatment. Rodents show natural phenotypic differences in the degree to which food-paired cues engage mesolimbic dopaminergic activity and elicit incentive motivational states. These individual differences have a strong genetic component, and are associated with other behavioral traits that are frequently comorbid with addictive tendencies. By taking advantage of these individual differences, it may be possible to determine whether certain sleep patterns represent an underlying predisposing factor for addictive behavior. For example, a predisposition for poor quality or fragmented sleep, prior to any drug or reward exposure, may be one of several traits that are part of an addictive phenotype. Disordered sleep may cause certain individuals to experience greater attraction to incentive cues when they are first encountered (leading to sign-tracking), which may then be exacerbated by further cue exposure or by the sleep deficits that can result from the comsumption of drugs or alcohol. Another possibility is that baseline sleep characteristics do not differ between STs and GTs, but the hyper-responsive mesolimbic circuitry that underlies sign-tracking may render STs especially vulnerable to the negative effects of sleep deprivation on reward-seeking behavior. In either case, the ST/GT model could provide a better understanding of how the neural pathways mediating sleep and motivation interact with each other, and ultimately lead to treatment strategies for substance use disorders that are more closely tailored to the unique needs of each invidual.

Highlights.

• There is individual variation in the effects of sleep loss on reward circuitry

• This individual variation has been difficult to study in preclinical animal models

• There exists a Sign/Goal Tracker model of individual differences in cued motivation

• There is overlapping circuitry responsible for both sleep and ST/GT behavior

• ST/GT model can be used to study individual variation in sleep-reward interactions