Pre-Clinical Models of Reward Deficiency Syndrome: A Behavioral Octopus

Abstract

Individuals with addictive, compulsive, impulsive and some personality disorders can share in common a dysfunction in how the brain perceives reward, where processing of natural endorphins or response to exogenous dopamine stimulants is impaired. Reward Deficiency Syndrome (RDS) is a polygenic trait with implications that suggest cross-talk between different neurological systems that include the known reward pathway, neuroendocrine systems, and motivational systems. In this review we evaluate well-characterized animal models for their construct validity and as potential models for RDS. Animal models used to study substance use disorder, depression, early life stress, immune dysregulation, ADHD, PTSD, compulsive gambling and compulsive eating disorders are discussed. These disorders recruit underlying reward deficiency mechanisms in multiple brain centers. Because of the widespread and remarkable array of associated/overlapping behavioral infractions with a common root of hypodopaminergia, the basic endophenotype recognized as RDS is indeed likened to a behavioral octopus. We conclude this review with a look ahead on how these models can be used to investigate potential therapeutics that target the underlying common deficiency.

Introduction

The quest for pleasure and satisfaction balanced against basic survival needs and chronic indulgence is timeless and has been described and debated for millenia. The Greek philosopher, Epicurus of Samos (341–270 BC), necessitated a hedonistic calculus to achieve contentment balance that includes an accounting for physical and mental experiences of pleasure and pain (Konstan, 2018). We now know that feelings of well-being, satisfaction, and achievement after accomplishing a task are mediated by natural neurotransmitters released in the brain’s reward centers which form a functional network primarily encompassing the midbrain, limbic system, and cerebral cortex, termed mesocorticolimbic (Berridge and Kringelbach, 2015). Here, Berridge and Kringelbach explore concepts of pleasure involving ‘liking’, ‘wanting’, euphoria, anhedonia and disgust that are also hypothesized to have a common currency shared by brain systems that compute reward (Berridge and Kringelbach, 2015). Indeed many mental health conditions represent states where ‘satisfaction’ or ‘elation’ is elusive because of an imbalance of neurotransmitters such as (5-hydroxytryptamine (serotonin), dopamine, norepinephrine, GABA, glutamate) and neuropeptides (endorphins) which heightens the chemical requirement to compute pleasure and reward in the mesolimbic system (Neumann and Landgraf, 2012, Michels et al., 2014). This same “brain reward” imbalance has been identified for addictions to food, drugs, sex, gambling, etc., and converge on inadequate dopamine release or ineffectual mechanisms to process dopamine, leading to a hypodopaminergic state (Lopresti and Drummond, 2013, Pennington et al., 2014, Mukherjee et al., 2008). Epicurus was likely referring to obtaining a balance of the biologically defined ‘brain reward’ from both a philosophical and physiological standpoint long before these concepts were supported with neuroscience data. Psychiatric disorders identified as having impaired dopamine homeostasis are many, and the complex mechanistic bases for these disorders are intensively investigated.

Reward Deficiency Syndrome

The conceptual framework that includes conditions with hypodopaminergia, is referred to as Reward Deficiency Syndrome (RDS), an umbrella term coined to emphasize the common foundational basis for each of these diseases converging on dopamine (Blum et al., 2000, Blum et al., 1996). In fact, the current literature supports a broader consensus of endorphinergic and neurotransmitter mechanisms that converge on dopamine tone as influencing reward, and impacting motivation, anti-stress, incentive salience (wanting) and well-being. These are influenced by genetic and epigenetic factors that influence individual behavior, anatomical circuits, and indeed dopamine homeostasis (Blum et al., 2020). It is recognized that many disorders encompass a large number of biologically distinct entities which are not necessarily reflected in the Diagnostic and Statistical Manual of Mental Disorders (DSM-V) and yet represent real components of a given diagnosis (Casey et al., 2013, Lilienfeld and Treadway, 2016). The National Institute of Mental Health, in 2009, developed a Research Domain Criteria (RDoC) conceptual framework to guide in the study of mental health disease. These constructs to identify common neurobiological deficits, are meant to expand and cross-cut traditional classification boundaries and reduce heterogeneity in ‘symptom-based’ diagnoses by promoting the integration of biological and behavioral measures in clinical and pre-clinical analyses (https://www.nimh.nih.gov/research/research-funded-by-nimh/rdoc/index.shtml). Using those principles, researchers have examined more broadly common underlying features of various disorders. In a review in 2012, Dichter et al. explored reward networks by examining impaired reward circuits with neuroimaging, biochemistry, and epigenetics across neurodevelopmental and psychiatric disorders, as well as genetic syndromes (Dichter et al., 2012). The approach of RDoC if properly employed, may become an alternative manner to classify mental illnesses (Lilienfeld and Treadway, 2016). Here we explore hypodopaminergia as a cross-cutting concept to study different psychopathologies using animal models.

In a five generational genomic analysis of a number of candidate dopaminergic genes and associated polymorphisms (i.e. DRD2, DAT1, DRD1) a case has been made for RDS to be considered as the endophenotype that best describes a hypodopaminergic condition that results of the brain reward circuitry dysfunction. An endophenotype is defined as quantifiable biomarkers reliable in defining disease liability – the clinical phenotype- based on genetics (Gottesman and Gould, 2003, Kendler and Neale, 2010, Bearden and Freimer, 2006) and as such, RDS is best thought of as an etiological root cause rather than a strict DSM 5 categorization (Blum and Gold, 2011, Casey et al., 2013), and may evolve to be considered a hypodopaminergia spectrum disorder.

In human subjects with addictions, Volkow and colleagues show with PET (positron emission tomography) imaging an under-activation of dopamine circuits and reduced dopamine D2 receptors associated with reduced activity in the basal forebrain of addicted subjects; presumably associated with heightened drug seeking behavior (Volkow et al., 2006, Volkow et al., 2002, Volkow et al., 2008). Although substance use disorder is a strong model of RDS, the RDS concept extends beyond illicit drug use to incorporate a wide variety of addictions, compulsive behaviors, as well as affective disorders where reduced function of dopamine circuits are implicated (Bowirrat and Oscar-Berman, 2005, Koob and Le Moal, 2005, Gardner, 2011, Borsook et al., 2016, Blum et al., 2017, McLaughlin et al., 2017, Blum et al., 2019). To try to unpack the complexity of reward deficiency, Leyton hypothesized that hypodopaminergia may manifest in various ways, where for some individuals, reward may be muted, necessitating potent events to induce dopamine release such as in late stage SUD (substance use disorders), binge alcohol drinking or chronic alcoholism, and even genetically pre-disposed thrill-seeking individuals. Some thrill seeking, however, can be due impaired interoceptive sensitivity and be sex-specific (Kruschwitz et al., 2014). For others, DA responses may be augmented in the presence of substance cues and inhibited by cues not associated with the reward (Leyton, 2014). Indeed, gene transfer in DRD2-expressing and DRD2-deficient mice support the idea that a threshold level of the D2 receptor is necessary for excessive alcohol consumption, and deviation from that threshold level could impact substance use as in the case of cocaine (Thanos et al., 2005).

A large body of literature exists, that defines addiction as loss of control over drug use, discounting of negative consequences to acquire the reward, and intense craving, associated with genetic vulnerabilities together with environmental cues. Koob, Volkow and colleagues in several independent and joint articles leading to a landmark paper in 2016, discuss the severe dysregulation of motivational circuits in addiction (Koob and Le Moal, 2005, Koob and Volkow, 2010, Koob and Volkow, 2016, Volkow et al., 2014). This dysregulation broadly involves exaggerated incentive salience, reward deficits and stress surfeits (increases in negative emotional states), and compromised executive function and impulsivity (Koob and Volkow, 2016). Animal studies that model these domains may model the rewarding aspect of binge/intoxication, of compulsive drug seeking behavior, stress surfeits or separate components using specific assays with predictive validity, including self-administration, two-bottle choice, conditioned place preference or intracranial self-stimulation. These pre-clinical models in rodents and non-human primates have been invaluable to understanding the biology of reward circuits, to test medications that treat binge behaviors, compulsive behaviors or the underlying mechanisms (Bell et al., 2016) with validity and consistency to human data (Banks and Negus, 2017). Garcia-Pardo et al. constructed an excellent review on animal models of addiction and the various paradigms used to elicit information about brain reward (García Pardo et al., 2017) and Belin-Rauscent and others discuss how the recent advancements in these pre-clinical models closely mimic the DSM diagnostic criteria for addiction, from controlled drug use to escalation, and relapse (Belin-Rauscent et al., 2016, Spanagel, 2017).

Although excessive drug intake and addiction fit under the umbrella, RDS is a more broad concept. Borsook et al. suggest that pain pathophysiology is mediated via neuroadaptations in reward and stress related brain circuits (Borsook et al., 2016). They propose a hand-in-hand combined model of reward deficiency and anti-reward where biopsychosocial variables that modulate brain reward motivation and stress interact to create conditions of hypodopaminergia that manifest as anhedonia and diminished motivation for natural reinforcers (Borsook et al., 2016). Similar to other disease spectra, the RDS concept has gained momentum and is currently included in listings of clinical psychological conditions (Blum, 2017). It is proposed that individuals predisposed to RDS may share genetic underpinnings that converge onto the reward pathway. This commonality involving genetic dysregulation may manifest in behavioral circuit-related, and molecular abnormalities characterized as behaviors of reward deficiency (Comings and Blum, 2000, Green et al., 1999).

Therefore, to fully understand the biochemical basis of RDS, animal models are invaluable to investigate behavior and neurochemistry of mental health and substance use disorders, as well as their co-morbidities. Whereas many extensive reviews of individual pre-clinical models of alcoholism exist (Bell et al., 2017), those for schizophrenia (Khokhar and Todd, 2018), stress (Campos et al., 2013) major depressive disorder (MDD) (Czéh et al., 2016), opioid addiction (Fattore and Diana, 2016), obesity (Fernández-Quintela et al., 2016), ADHD (Majdak et al., 2016, Phillips et al., 2018), PTSD (Flandreau and Toth, 2018) and more are available and address multiple co-morbidly occurring conditions. For example, Gould et al. argue that whereas it is difficult to replicate or confirm suicidal ideation in rodents, investigating HPA axis dysfunction, impulsivity, neuroimmune changes and neurotransmitter imbalance in appropriate models could be beneficial in addressing endophenotypes of psychiatric diseases including MDD (Gould et al., 2017, Gould and Gottesman, 2006). There currently exists no comprehensive assessments of animal models as reward deficiency models. Early on, Gardner, Kosten and others recognized that functional deficiency of dopamine in the nucleus accumbens of the Fischer 344 rat which self-administered alcohol, psychostimulants and opiates more than the non-deficient rat, may be considered a model of RDS (Gardner, 2001) as these rats not only had less dopamine, they exhibited fewer DRD2 receptors and dopamine transporter DAT (Haile and Kosten, 2001). Furthermore there is a strong link between stress responsiveness of the hypothalamic-pituitary-adrenal (HPA) axis and behavioral responsiveness to psychoactive drugs in these and other animals (Kosten et al., 1998, Kosten and Ambrosio, 2002, Gondré-Lewis et al., 2016b).

Here, we propose that animals commonly used to investigate mental health disorders could be used as RDS models as they possess many characteristics of RDS. We present an example of a naturally occurring model, the Wistar Kyoto (WKY) rat strain with many RDS phenotypes, an induced model of early life stress (ELS) with epigenetic ramifications, and a carefully bred, genetically alcohol-preferring (P) rat with multiple co-morbid neuropsychiatric behavioral patterns. We also review animal models which were developed to investigate PTSD, gambling behavior and compulsive eating disorder. By no means should these be considered exhaustive as there are rodent models of obesity, impulsivity, social defeat, and many other conditions that are not developed in depth in this review. Furthermore, animals with individual genetic mutations or with inflammatory molecule imbalance that could confer RDS should be evaluated in detail but are only briefly discussed in order to tighten the scope of the review (Alguacil and González-Martín, 2015, Kuss et al., 2018, Blum et al., 2014b, Lochner et al., 2005, Montagud-Romero et al., 2020)

Consideration of these common animal models as exhibiting RDS traits will expand our understanding of the foundational basis for the genetic correlates of addiction. Additionally, these rodents may serve as experimental models for which the scientific community may further explore RDS symptomatology with the intended outcome of developing treatments that may more closely address broader phenotypic and genotypic dysregulation. In the study of RDS, these models could serve as essential tools to investigate the neurochemistry, behavior, genetics, epigenetics, and neuroanatomy of RDS (Gold et al., 2018, Scheggi et al., 2018, Bruijnzeel et al., 2004). We contend that these disorders arise as many arms of the foundational deficiency of dopamine signaling, although distinct in their clinical phenotype share similarities at their root (Figure 1). As a note of caution, animal models are very useful to determine underlining mechanisms as well as important addictive like behaviors, but of course cannot replace research on the real human condition.

Figure 1: Reward Deficiency Syndrome: The Behavioral Octopus.

Schematic shows many arms of individual disorders with unique characteristics that share a common foundation of low dopamine signaling tone (hypodopaminergia); a foundational cause/consequence of reward deficiency.

Animal Models

The Wistar Kyoto Rat

Reward deficiency syndrome (RDS) involves an individual’s dissatisfaction of natural rewards due to functional underlying genetic deviations of one or more of the reward pathway components. These individuals seek enhanced stimulation of reward pathways through “external” means which may include but are not limited to; drug and alcohol abuse, risky sports, pathological gambling and compulsive sexual activity (Comings and Blum, 2000). Researchers have developed and characterized mouse and rat models of major depressive disorder accomplished by selective breeding for genetic abnormalities which are expressed as phenotypic behavior indicative of depressive-like behavior (Will et al., 2003, Wegener et al., 2012). Among many rat models, the Wistar-Kyoto (WKY) rat stands out as a reliable model for stress-related depression and anhedonia and exhibits behavioral and biochemical similarities with humans who are characterized as reward deficient (Belujon and Grace, 2014). They exhibit high levels of depression-like behaviors (Rittenhouse et al., 2002), enhanced anxiety-like behaviors (Shepard and Myers, 2008), and diminished activity in novel environment (Ferguson and Gray, 2005). They also exhibit increased plasma corticosterone levels (Solberg et al., 2001), impaired gastric accommodation and visceral hypersensitivity (Nielsen et al., 2006), which may be due to the fact that WKY rats are hyper-reactive to stress and show dysregulation of the hypothalamic–pituitary–adrenal (HPA) and hypothalamic–pituitary–thyroid (HPT) axes (Will et al., 2003). The following text will discuss Wistar-Kyoto rat as well as mouse models of depression characteristics as they pertain to RDS.

The Wistar Kyoto (WKY) rat strain was first developed as a normotensive control strain for the spontaneously hypertensive Wistar rat (Okamoto and Aoki, 1963, Louis and Howes, 1990). Although WKY serves as the control strain for studies on hypertension, its behavior is not ‘normal’ (Will et al., 2003). It is now an established depression model characterized by elevated anxiety- and depressive-like behavior as compared to the Wistar and Sprague Dawley outbred strains usually used as controls (Will et al., 2003, Paré and Redei, 1993)(Tejani-Butt et al., 1994), (Redei et al., 1994) (López-Rubalcava and Lucki, 2000), (Allard et al., 2004), (Kalejaiye et al., 2013, Pardon et al., 2002).

WKY rats exhibit altered dopaminergic signaling in various brain regions. They exhibit higher DA turnover in the NAc shell (De La Garza and Mahoney, 2004, Scholl et al., 2010) and higher D2 receptor binding levels in the NAc shell and VTA, but lower D2 receptor binding in the caudate putamen, NAc core and hypothalamus (Yaroslavsky et al., 2006). It is known that D1 and D2 receptors represent sites where DA modifies behavior related to anxiety and reward; and the altered expression of the dopamine receptor may reflect the susceptibility to anxiety observed in this rat strain (Langen and Dost, 2011).

Wistar Kyoto RDS behavior and associated brain circuits

In this context, depression itself may be considered an indicator of underlying RDS as it relates to the inability to find pleasure or reward from naturally occurring rewards (Blum, 2017, Gold et al., 2018). WKY rats exhibit anhedonic symptomatology as defined by reduced sucrose intake (when % sucrose preference is significantly less than or equal to 50%) in the sucrose preference test (Malkesman et al., 2005), (Hurley et al., 2014). They display exaggerated immobility in the forced swim test (FST), an established measure of depressive-like behavior (Paré and Redei, 1993), (Tejani-Butt et al., 1994), (Redei et al., 1994), (Armario et al., 1995), (Lahmame et al., 1997), (López-Rubalcava and Lucki, 2000), (Allard et al., 2004), (Kalejaiye et al., 2013) which is attenuated by the administration of tricyclic antidepressants (TCA) (López-Rubalcava and Lucki, 2000) as well as other antidepressants. Albeit, some reports suggest the efficacy of treatment outcomes are variable according to the individual animal genetic profile of this strain (Paul et al., 1990, Millan et al., 1997, Skrebuhhova et al., 1999) and may actually be considered treatment resistant (López-Rubalcava and Lucki, 2000). WKY rats also show decreased locomotor activity (Paré, 1994, Berton et al., 1997), diminished activity in a novel environment (Armario et al., 1995, Ferguson and Gray, 2005, Pardon et al., 2002) and a tendency to freeze in stressful situations and in situations where activity would ordinarily be observed (Paré, 1994, Paré, 1993). WKY rats display visceral hypersensitivity characteristic of irritable bowel syndrome (IBS), likely due to their exaggerated response to stress (Gunter et al., 2000, Nielsen et al., 2006). IBS has been shown to be co-morbid with depression and anxiety (O’Malley et al., 2010), conditions which are associated with RDS. In this animal model, stress hyper-reactivity manifests physiologically by an enhanced susceptibility to stress-induced ulcers and increased plasma adrenocorticotropic hormone (ACTH) levels after experiencing stressful situations such as restraint stress (Redei et al., 1994). Furthermore, WKY rats are also known to exhibit enhanced alcohol consumption during self-administration studies (Jiao et al., 2006), (Yaroslavsky and Tejani-Butt, 2010). Excessive alcohol intake here, could represent a failure to cope and mirror the fact that some individuals with anhedonia abuse alcohol to overcome reduced reward.

Thus, it is not surprising that WKY rats show dysregulated HPA axis, with consequent exaggerated stress response. Compared to Wistar rats, basal plasma adrenocorticotropic hormone (ACTH) and corticosterone levels of WKY rats remain significantly higher for several hours after the diurnal peak (Solberg et al., 2001). Since WKY rats demonstrate enhanced secretion of stress hormones in response to stressful stimuli, this may result in a dysfunctional negative feedback mechanism, and therefore contribute to the behavioral or endocrine deficits observed in the WKY rats.

The hippocampal formation, the amygdala and the prefrontal cortex which are involved in learning and memory, strong emotions, and executive function, respectively, are all targets of stress hormones which exacerbate mood disorders (McEwen, 2005). People with long-term depressive illness exhibit atrophy of the PFC, hippocampus, and the amygdala, a structure which also is hyperactive in anxiety and mood disorders. In fact, structural abnormalities were found in the frontal lobe, basal ganglia, temporal lobe and hippocampus of patients with different mood disorders (Brambilla et al., 2002, McEwen, 2005), including reduced hippocampus in PTSD (Brambilla et al., 2002). There is a dearth of research into the neuroanatomical variations between WKY and other rat strains, however one study found the hippocampal volume of WKY rats to be 20% less than that of Wistar rats (Lemos et al., 2011), consistent with human data suggesting a structural remodeling of brain circuits in MDD (McEwen, 2005, McEwen, 2007). WKY behavior characterized as reward deficiency symptomatology are shown in Figure 2.

Figure 2: Behaviors categorized as symptomatology of reward deficiency syndrome in the WKY and HL rats.

Rodent behaviors reported in various studies which this paper proposes are RDS characteristics.

The helpless mouse (HL)

Mouse models exist which share behavioral and neurochemical commonalities with the WKY rat. Selective genetic breeding for specific traits is often the method of procuring the phenotype of depressive-like behavior (Schulz et al., 2013) One such study by Yacoubi and colleagues bred mice who exhibited exaggerated immobility in the FST and tail suspension test (TST) for fourteen generations (El Yacoubi et al., 2003). These helpless (HL) mice were then investigated for their behavioral, neurochemical and electrophysiological deviations from their non-helpless (NHL) counterparts. With respect to behavior, and similar to WKY rats, HL mice are significantly more immobile in the FST and TST. This behavior is particularly more pronounced in the 14^th generation of breeding and is effectively and significantly attenuated by TCA administration (El Yacoubi et al., 2003). Again, similar to WKY rats, locomotor activity was diminished in the open field locomotor activity test. Sleep and wakefulness cycles are also disrupted in HL mice implying depressive-like symptomatology mimicking other rat models of depression and humans with depression (Steiger and Kimura, 2010, Steiger and Pawlowski, 2019). Finally, and perhaps most notably, HL mice reportedly consume considerably less of a 2% sucrose solution measured in grams per kg of body weight, during a 96-h period when compared to their NHL counterparts. This significant difference was more pronounced in female HL mice than in their male counterparts. Similar to the WKY rat, this behavior is indicative of anhedonic symptomatology. Basal corticosterone levels of HL mice reveal a higher baseline level of serum corticosterone concentration (μg/100 ml) compared to NHL mice. Altogether, this data suggests that mice which are readily abundant, considerably cheap, and whose genetic profiles are easily manipulated, may serve as excellent representative models of reward deficiency (Figure 2).

The WKY rat and the HL mouse as RDS models

Many of the aforementioned behaviors displayed by WKY rats and HL mice correlate quite closely to RDS symptomatology in humans. The umbrella term of RDS encompasses diminished responsiveness to natural rewards, exhibited by both rodent models (Der-Avakian and Pizzagalli, 2018). This occurrence is directly linked to increased reward-seeking behavior in an attempt to self-medicate or bolster dysregulated or deficient reward pathways. Thus, both WKY rats and HL mice have baseline anhedonic behavior as characterized by the FST and TST. It is known that depressed individuals are more prone to alcohol abuse (Pavkovic et al., 2018), a behavior recapitulated in the WKY rat (Jiao et al., 2006). The basis of this phenomenon can be credited to dysfunctional neurotransmission or neuromodulation involving cannabinoids, dopamine (DA), norepinephrine (NE), serotonin (5-HT), gamma-aminobutyric acid (GABA), glutamate, acetylcholine (ACH), neuropeptides and hormones (Markou et al., 1998, Ollat et al., 1988). While RDS sufferers may seek more complex rewards, WKY rats also have notable increases in alcohol consumption during self-administration studies (Paré et al., 1999, Jiao et al., 2006).

The Wistar-Kyoto rat and the HL mouse both demonstrate circuitry abnormalities which account for behavioral manifestations that can allow for their classification under reward deficiency syndrome -- see Figure 2 for a summary of research to substantiate these RDS behaviors. The mesolimbic pathway, which plays a major role in the reinforcement of motivational behavior, originates in the ventral tegmental area (VTA) and projects to the nucleus accumbens (NAc). The dopamine-regulated nigrostriatal pathway which plays a crucial role in voluntary motor control, originates in the substantia nigra and projects to the dorsal striatum/basal ganglia (Carli et al., 1985), (Carr and White, 1986, Di Chiara and Imperato, 1988). Using tract tracing, Bourdy et al. show that the tegmental and nigral systems of the midbrain are not independent, that the VTA tail (RMTg) serves as a GABA brake for the nigrostriatal system, meaning that if the RMTg is ablated, motor enhancement akin to amphetamine-mediated performance enhancement can be observed (Bourdy et al., 2014). Both ventral and dorsal striatal (Belujon and Grace, 2014) pathways are activated during appetitive activities such as copulation, drinking, and eating (Wilson et al., 1995) (Mirenowicz and Schultz, 1996). The mesolimbic and substantia nigra pathway contain dopaminergic neurons which respond to chronic stress by diminishing extracellular DA levels (Cabib and Puglisi-Allegra, 1996), (Gambarana et al., 1999). This may in turn lead to impulsive drug or pleasure-seeking behavior in order to increase DA release in these areas vital to reward (Comings and Blum, 2000), (Kapur and Mann, 1992, Twining et al., 2015). One recent study suggests that the dysregulation in WKY rat neurocircuit components is specifically due to a diminished number of spontaneously active DA neurons in the VTA (Belujon and Grace, 2014). The administration of the anti-depressant ketamine, a fast-acting N-methyl-D-aspartate (NMDA) antagonist used to treat treatment-resistant depression can restore activity of dopamine neurons and synaptic function in DA circuits of WKY rats rendered helpless via an inescapable footshock paradigm (Belujon and Grace, 2014). This study also indicates that long term depression, i.e. LTD, in the shell and not core of the NAc appears to be a hallmark trait of WKY rats. Thus, reward function is intimately linked to MDD mechanisms as well as treatments that target MDD. Some individuals suffering from RDS may share similar neurocircuitry underpinnings with this depression rat model of RDS which precipitate the aberrant reward behavior that can be reversed with target NMDA receptors (Zorumski et al., 2015, Belujon and Grace, 2014).

Nevertheless, an argument can also be made for interoception as an important component of constructs related to addiction, as well as sensation seeking as discussed later. Interoception involves the receiving, processing, and integrating of body-relevant signals to influence arousal, attention, stress, and reward (Paulus and Stewart, 2014). The insula-mediated process can use the internal state of the individual to modulate approach or avoidance behavior (Naqvi and Bechara, 2010, Paulus and Stewart, 2014). Indeed, Naqvi et al have shown that damage to the insula profoundly disrupts addiction to cigarette smoking (Naqvi et al., 2007). Thus, the insula, by serving as a presenter of information regarding conscious awareness and memory to frontal control networks can be impacted to modulate decision-making and pleasure components of addiction (Naqvi and Bechara, 2010). It is not yet clear how interoception might or might not fit into the RDS endophenotype and related hypotheses.

Neurotransmitters modulating DA in WKY and the HL models

Animal models of depression exhibit downstream molecular perturbations resulting in disruptions in dopaminergic systems (Söderlund and Lindskog, 2018) that often co-segregate with anhedonia. Anhedonia refers to the loss of pleasure in response to natural reinforcers possibly due to abnormalities in the midbrain, striatum, amygdala and prefrontal cortex (Gold et al., 2018). The convergence and interaction of neurotransmitters and second messengers that control the release of dopamine has been referred to as the Brain Reward Cascade (BRC). DA neurons of the VTA and substantia nigra pars compacta (SNc) project to the dorsal raphe nucleus (DRN) as well as the medial raphe nucleus (MRN) in the brainstem (Mansour et al., 1990), (Peyron et al., 1995); (Kitahama et al., 2000). The DRN is a vital 5-HT-containing brain region known for its role in reward-seeking behavior and commonly affected in the WKY rat strain and HL mice. Thus, because the release, uptake and modulation of DA and 5-HT are altered in HL and WKY rodents, investigations of the VTA-SNc/DRN circuits may reveal a facilitatory relationship in which 5-HT neurons in the DRN are affected by the degree of dopamine receptor activation in these rodents (Ferré and Artigas, 1993), (Mendlin et al., 1999), (Haj-Dahmane, 2001). With respect to the mouse model of depression, electrophysiological recordings of the basal firing rate of spontaneously active DRN neurons was performed in the HL mice. While the results did not reveal a diminished basal firing rate of serotonergic neuron in this brain region, when a 5-HT_1A autoreceptor agonist was administered subcutaneously, it exerted a more potent inhibition of the firing rate of these neurons in HL mice, known to have a higher density of 5-HT_1A than control NHL (El Yacoubi et al., 2003). This exaggerated sensitivity of the 5-HT_1A autoreceptors is a trait that has been reported in humans and in other animal models of depression (Eley and Plomin, 1997) (Maudhuit et al., 1997) (Overstreet, 2002) and is a target of therapeutics aimed at reducing 5-HT_1A autoreceptor mediated negative feedback (El Yacoubi et al., 2003). When levels in the prefrontal cortex and hippocampus were analyzed, HL mice expressed an up-regulation of 5-HT associated with a reduction in the 5-HT metabolism index. With respect to neurotransmission, when compared with their Wistar controls or Sprague-Dawley rats, WKY rats display reduction in DA, serotonin and NE which results in a reduction in extracellular synaptic catecholamine tone (Scholl et al., 2010). In particular, the study by Scholl and colleagues reveals that WKY rats demonstrate significantly reduced basal 5-HT levels in a number of limbic brain regions including the basolateral amygdala, DRN, hypothalamus and NAc.

Combined, these data suggest that the examination of RDS-like symptomatology, although complex, can be more accessible if we reconsider previously accepted models of anhedonia and depression as newly emerging models of RDS. Thus, the underlying disruptions in neurocircuitry, molecular neurotransmission and behavior which have been studied previously in WKY rats for instance, may serve as foundational approaches to now study RDS.

Other models of Anxiety and Depression:

CD1 mice are bidirectionally and selectively bred for their high-anxiety behavior (HAB) displayed on the elevated plus maze (Landgraf et al., 2007). HAB rats are characterized by exaggerated anxiety- and depressive-like behaviors, freezing response during social defeat and a more passive coping style compared to their Low-anxiety behavior (LAB) counterparts (Keck et al., 2003, Frank et al., 2006). HAB unlike LAB responds strongly even to repeated stress exposures, suggestive of a hyperactive HPA axis (Landgraf et al., 2007). Prast et al. (Prast et al., 2014a) reported that following a conditioned place preference (CPP) paradigm, HAB found cocaine more rewarding than in NAB mice, which suggests that cocaine relieves anxiety in HAB mice. Furthermore, the cocaine CPP-induced expression of the immediate early genes EGR1 was increased in the medial but not the lateral regions of the Nac of HAB mice (Prast et al., 2014b). This suggests that the Nac are not only important for drug or reward-seeking as illustrated in the CPP paradigm, but also for anxiety-related behavior (Prast et al., 2014a, Prast et al., 2014b). Although HAB mirrors some characteristics described in RDS, there is dearth in literature linking their behaviors to hypodopaminergicity.

EARLY LIFE STRESS MODELS OF RDS

Stress can be defined as any condition that constitutes a perceived threat to altering an organism’s homeostasis. This may come by way of traumatic experiences and environments, which activates the HPA axis and releases glucocorticoids via the adrenal cortex, thereby eliciting a physiological response in the central and peripheral nervous systems (Chrousos, 2009). Chronic exposure to stress, during critical periods of development, is a strong determinant of subsequent vulnerabilities to later emotional and cognitive pathologies, and has been widely implicated in a plethora of neuropsychiatric disorders, including attention deficit/hyperactivity disorder, conduct disorders, anxiety, depression, drug abuse, and posttraumatic stress disorder (Buchmann et al., 2010, Taylor, 2010, Talge et al., 2007). Early life stress (ELS) can originate from prenatal stress, which include prenatal drug exposure and perinatal trauma. ELS can also be from postnatal stress, including the effect of maternal postpartum depression on the newborn, and maternal deprivation, isolation, bedding deprivation, all of which are environmental perturbations to ‘normal’ development (Walker et al., 2017, Brunton, 2013, Rice et al., 2008, O’Mahony et al., 2006, Maniam et al., 2014). Others also consider ELS to occur during adolescence, as a result of social isolation or other environmental and drug stressors to the developing brain. One of the established pre-clinical models for analyzing the effects of ELS is maternal deprivation (MD) in rats (Huang et al., 2002, Lai et al., 2006, Lai and Huang, 2011, Francis et al., 2002, Pryce et al., 2005, Caldji et al., 2000, Ladd et al., 2000, Gondré-Lewis et al., 2016b, Gondré-Lewis et al., 2016a); as well as prenatal exposure to drugs such as nicotine, cocaine and amphetamine (Kalejaiye and Gondre-Lewis, 2017, Wang et al., 2011, Wang and Gondre-Lewis, 2013, Malanga et al., 2008, Akbari et al., 1992, Fukushiro et al., 2015), or the combination of the two (Bassey and Gondré-Lewis, 2018, Wang and Gondre-Lewis, 2013). In the MD model, rat pups undergo restricted separation from their mother for lengths of time varying from a single 24h exposure at PND 9 to 3–6 hours daily during the first two to three postnatal weeks (Gondré-Lewis et al., 2016b, Fabricius et al., 2008, Hulshof et al., 2011, Lehmann et al., 1999, Michaels and Holtzman, 2006). Here, we will use prenatal nicotine exposure (PNE) to discuss the effects of early life exposure to drugs of abuse as a stressor that changes behavior and brain chemistry later in life. In the PNE paradigm, the drugs are administered via an implanted canula to the pregnant dams from early gestation until parturition.

ELS and RDS behavior

Perinatal drug exposure and maternal deprivation are critical forms of ELS that result in marked brain morphological and behavioral changes as well as mental health outcomes (Weinstock, 2001, Weinstock, 2005, Gondre-Lewis et al., 2016a), comparable to the aberrant behaviors observed with RDS. Exposure to ELS causes lasting effects that alter the epigenetic landscape in the organism, including DNA methylation, post-translational modification and non-coding RNAs (Burns et al., 2018, Kinnally et al., 2011). Behavioral assessments have revealed hyperactivity in adolescent or adult rats following prenatal nicotine exposure (PNE), MD, or the combination of the two (Gondré-Lewis et al., 2016b, Gondré-Lewis et al., 2016a, Bock et al., 2017, Newman et al., 1999, Schneider et al., 2012, Bassey and Gondré-Lewis, 2018). PNE has also been linked to increased anxiety-like behaviors (Eppolito et al., 2010, Cheeta et al., 2001, File et al., 1998, File et al., 2000, Ouagazzal et al., 1999) and depressive-like behaviors in 3-week old (Parameshwaran et al., 2013) and adult (Parameshwaran et al., 2013, Vaglenova et al., 2004) animals. Other findings show that separation of neonatal rats from their mothers during ELS equally increase anxiety-like behavior during adulthood: Adult male rats exposed to a 3-hours daily maternal separation protocol over a 3-week period (postnatal days 2–21) displayed significantly increased anxiety as evidenced by a reduction in the time spent in the open arms of the EPM compared with control animals (Aisa et al., 2007). Similar anxiety-like behavior was also recorded in adult rats after shorter (2-week) periods of maternal separation (postnatal days 2–14) (Huot et al., 2001, Kalinichev et al., 2002, Lee et al., 2007). Other studies, however showed that MD groups had a reduced sign of anxiety, since they ventured more often onto the open arms than the equivalent control group (Fabricius et al., 2008, Lehmann et al., 1999, Bassey and Gondré-Lewis, 2018), and it has been proposed that this may even be indicative of risk-taking behavior (Bassey and Gondré-Lewis, 2018). Nonetheless, most studies agree that MD stress is correlated with depression and anhedonia as evidenced by greater immobility time in Porsolt’s forced swim test and reduced sucrose drinking in MD groups.

With regarding to priming the brain to respond more robustly to a drug reward, MD stress resulted in higher vulnerability to initiate self-administration of ethanol (Huot et al., 2001, Cruz et al., 2008). Ample evidence in the literature shows significant and sustained increase in the amount of responding for alcohol in adult MD rats following separation of newborns from the dams and their littermates during lactation. These findings suggest that ELS can have protracted effects on both binge drinking and impulsivity in the adult long after the experience of (LeDoux, 2007) maltreatment; see Figure 3 (Gondré-Lewis et al., 2016b, Gondre-Lewis et al., 2016a). In fact, whether by operant responding, two-bottle choice or other preference tasks this increased ethanol consumption has been reported in several studies (Cruz et al., 2008, Moffett et al., 2007).

Figure 3: Shared RDS behaviors for animal models of ELS and PTSD.

A-C are modified from Gondré-Lewis et al., Stress 2016, March; 19 (2):235–247 with permission. (A) Responding for alcohol is increased in maternally deprived (MD) (B) Blood alcohol content of MD rats were elevated to >80mg%/dL following 2 hours of 10% EtOH drinking. (C) Adjusted amount is decreased (impulsivity is increased) in delay discounting task. (D) shows the different RDS behaviors that have been test in PTSD and MD. * represents traits also shown for PTSD animal models. The absence of a shared behavior may indicate that it is not yet reported or tested.

ELS and brain circuits correlated with RDS

ELS impacts immature neural circuitry in the limbic system, resulting in hyperactivity of the hypothalamus-pituitary-adrenal (HPA) axis, thereby leading to elevated gluococorticoid levels (Koe et al., 2014). Corticotropin-releasing hormone/factor (CRH/CRF) producing neurons coordinate the autonomic, endocrine, immune, and behavioral responses to stress (Arborelius et al., 1999). As discussed previously, there are persistent structural and functional changes to CNS structures and circuits in relation to ELS. The brain regions involved in emotional processing and regulation of the limbic system are the hypothalamus, amygdala, hippocampus, septal nuclei and the anterior cingulate gyrus, nucleus accumbens (NAc) and ventral tegmental area (VTA). The VTA, which can be regulated by corticotropin releasing factor (CRF), is the primary mesolimbic locus mediating addictive behavior (Brake et al., 2004), and consist of dopaminergic neurons that project to the NAc, PFC, and other reward processing regions (Aransay et al., 2015). Studies by Gondré-Lewis et al. (Gondre-Lewis et al., 2016a) show that maternal separation impairs the regulation of VTA-mediated rewarding effects of drugs thereby promoting appetitive behavior. Hauskenet et al. similarly reported a reduction in spontaneous activity of VTA DA neurons following prenatal stress exposure, another form of ELS (Hausknecht et al., 2013). The amygdala is prominently associated with threat perception and anxiety processing, and regulates expression of emotional behavior (Davis, 1992). The amygdala is as an integrative center that provides emotional salience to internal and external stimuli, and is recruited as a brain stress center that provide the negative motivational states that drives addiction and negative affect (Koob, 2009, Agoglia and Herman, 2018). An active research area is to identify specific amygdala circuits that serve as targets for stress and alcohol-induced plasticity (Agoglia and Herman, 2018) and in RDS. Increased number of projection neurons was reported in the amygdala following MD (Gondre-Lewis et al., 2016a), implicating that structural connectivity of the amygdala may be altered as a result of ELS. An increase of amygdala volume was correlated with elevated CRF concentrations, along with reduced hippocampal neurogenesis and increased anxiety. The dentate gyrus works in close association with the amygdala in memory processing and is highly susceptible to MD-induced stress. Granule neurons in the principle layer of the DG are reduced with MD exposure (Oomen et al., 2011, Wang and Gondre-Lewis, 2013) showing an MD-induced decrease in neurogenesis or an increase in apoptotic processes (Lee et al., 2001, Fabricius et al., 2008, Hulshof et al., 2011). In fact, one stereology study reported that alterations in CA1, CA3 and DG due to MD were restricted to the ventral hippocampus which normally sends strong afferents to the amygdala. This implies major aberrations in connectivity of limbic fibers. These possible aberrations are substantiated with evidence of reduced dendritic length and dendritic spine number as a result of MD stress in the frontal cortex, hippocampus and nucleus accumbens (Huot et al., 2001, Monroy et al., 2010, Romano-Lopez et al., 2012). The medial prefrontal cortex (mPFC) is implicated in executive function and affective processing (Bush et al., 2000, Davidson, 2002). Both dorsal and ventral parts of mPFC is suggested to determine the level of brain structure activity and behavioral response to stress (Bissiere et al., 2006, Amat et al., 2005, Radley et al., 2006) and thus are likely impacted by MD. The finding of dendritic atrophy in the PFC (Murmu et al., 2006), and hippocampus (Jia et al., 2010) is not limited to postnatal MD, but has also been documented in prenatally stressed animals.

Neurotransmitters that influence RDS, altered by ELS

ELS reportedly alters the development of neurotransmitter circuits associated with dopaminergic, serotonergic, GABA-ergic, glutamatergic and the endogenous cannabinoid system (Llorente et al., 2010, Ellenbroek et al., 2005, Suárez et al., 2009, Suárez et al., 2010, Llorente-Berzal et al., 2012, López-Gallardo et al., 2012), proposed in mechanisms of addiction and other disorders. In this review, the dopaminergic system is of special interest as its deficient function is being directly correlated with reward deficiency syndrome (RDS) (Febo et al., 2017). The ELS behavioral model exemplifies the significant crossover between emotion and reward. The mesolimbic dopamine system is a key neurotransmission system in emotional responses to stress (Zhu et al., 2011, Hirano et al., 2007). ELS was shown to affect the development of dopaminergic neurons and the expression of dopamine receptor genes leading to its consequent dysfunction during adulthood (Zhu et al., 2011). The dopamine D2 receptor (D2R) is highly expressed in the central nervous system, regulates neural functions (Baik et al., 1995, Picetti et al., 1997, An et al., 2004, Kim et al., 2006) and as previously discussed, is implicated in various psychiatric disorders in addition to disorders related to addiction, stress, impulsivity and other reward-related behaviors (Morgan et al., 2002, Dalley et al., 2007, Johnson and Kenny, 2010, Baik et al., 1995).

The normal expression of D2 receptors is influenced by in utero experiences and is vulnerable to environmental insults; and so prenatal stress can impair DA neurotransmission (Diaz et al., 1995, Diaz et al., 1997). ELS may impair elimination of excessive receptor pruning that usually occur at puberty, thereby resulting in higher levels of D1 and D2 receptors in adulthood (Diaz et al., 1997). Berger et al. (Berger et al., 2002) reported increase of DA receptors in frontal cortex, nucleus accumbens, caudate putamen and hippocampus following prenatal stress. MD results in increased DA release as well as a decrease in the number of D2-dopamine receptors in the ventral tegmental area (VTA) and lowered DA transporter levels in the NAc (Meaney et al., 2002). The NAc is known to play a critical role in motivated behaviors and reward seeking via modulation of DA levels (Sulzer, 2011). It is also implicated in the modulation of stress and anxiety-like negative affective behaviors (Radke and Gewirtz, 2012, Sulzer, 2011). Several studies have found a strong correlation between variants of DRD2 receptor gene and alcoholism and polysubstance abuse (Blum et al., 1990, Suarez et al., 1994, Blum et al., 1993, Parsian et al., 1991, Smith et al., 1992, Noble et al., 1993). There is evidence that certain variants of DRD2 are associated with impulsive-compulsive- and addiction-related neuropsychiatric disorders (Blum et al., 1994). Crabbe et al. (Crabbe et al., 1994) in working with animal models of alcohol and drug seeking behavior mapped the D2 gene in mice to chromosome 9. These findings indicate that D2 dopamine receptor gene may be an important common genetic determinant of RDS and aberrant behavioral phenotypes associated with early life stress. Additional studies will implicate other components of the brain reward cascade.

It is important to note that in addition to the deficits discussed above, the MD model has been suggested as a potential model of schizophrenia, because of disruptions in prepulse inhibition and sensory gating, and startle habituation, which resemble characteristics exhibited in schizophrenia (Ellenbroek and Riva, 2003). The severity of the deficit, in turn, can be impacted by baseline dopamine sensitivity of different animal strains (Ellenbroek and Cools, 2000). Recent work suggests sex differences play a role in how exposure to ELS impacts reward and RDS conditions: Exposure to MD caused increased cocaine-seeking and consumption of palatable foods in males, for example, whereas females had a higher acquisition percentage for cocaine but were resistant to MD-induced anxiety (Ströher et al., 2020, de Lima et al., 2020). Moreover, the MD rat is postulated to share commonalities with post-traumatic stress disorder (PTSD) models that are recently developed (Diehl et al., 2012). The behavioral traits of ELS and PTSD are represented in Figure 3.

Alcohol Use Disorder

Drinking to intoxication is a critical component of risky behaviors in humans and includes binge drinking (BD) as an element of human alcohol use disorders (AUDs). BD refers to a form of alcohol abuse where individuals consume a specific amount of alcohol during a short time span (Courtney and Polich, 2009). According to the National Institute on Alcohol Abuse and Alcoholism (NIAAA), in BD, sufficient alcohol is consumed within a 2 hour period to raise blood alcohol concentration (BAC) to 0.08 g/dL or higher (DHHS-NIH, 2004). Drinking like this corresponds to men or women who take in five or four drinks in about two hours, respectively.

Both clinical and pre-clinical studies that use pharmacological manipulations (e.g. naltrexone) (Swift and Aston, 2015) and environmental manipulations (e.g. rearing environment, exposure to stress) have enabled characterization alcohol abuse and alcoholism as a disorder. Mechanistically, BD and excessive alcohol consumption are characterized by low levels of DA or dopamine tone in the reward pathway (Volkow et al., 2006). As discussed for stress and depression, the mesolimbic DAergic pathway involving the VTA, NAc, and PFC play an especially important role in mediating the reinforcing effects of alcohol. It has been hypothesized that BD and abuse of other drugs may be co-morbid with more generalized Reward Deficiency Syndrome (RDS) with overlapping associated genetic and epigenetic phenomena resulting in abnormal craving behavior (Blum et al., 1996).

Among the multifactorial variables associated with alcohol abuse and RDS, the contribution of heredity or genetic predisposition has gained special attention. This is mainly based on the observation that people differ considerably in their drinking despite having very similar environmental backgrounds (Collins, 2016). Scientific inquiry into genetic predispositions for alcohol abuse have been addressed using several preclinical models. The animal models of genetic susceptibility to alcohol drinking and associated deficits have served as invaluable tools in advancing our understanding of the molecular underpinnings of this complex disorder. Here, we focus on ethological pre-clinical models employed widely to study aspects of alcohol use disorder. Since excessive alcohol consumption is often co-morbid with neuropsychological symptoms, we aim to provide detailed information about animal models’ known mechanism, neurocircuitries and associated behavioral deficits that could make them a good candidate for studying RDS.

Rodent models/ The Alcohol-Preferring Rat as a model of RDS

Simply put, animal models cannot completely mimic the complex nature of all disorders under the RDS umbrella (Doremus-Fitzwater and Spear, 2016, García Pardo et al., 2017). Animal models of excessive alcohol intake in a controlled setting using techniques and paradigms that would otherwise be impossible using human participants because of ethical constraints have, however, provided useful information about the comorbid relationship with other depressive, compulsive, impulsive disorders. Several lines of mice and rats that are considered useful as animal models of BD/RDS are as follows: selectively bred lines and inbred lines. Selectively bred lines are divergent groups of rodents with either a strong preference for and high consumption level of alcohol (preferring line) or do not prefer and consume very little alcohol (non-preferring line). The inbred lines are populations of homozygous animals that share the same alcohol preference/consumption due to their identical genetic makeup.

A set of seven criteria have been established to evaluate alcohol preferring rodents as an animal model of BD (Cicero, 1979, Lester and Freed, 1973, McBride and Li, 1998). To qualify as an animal model for BD, rats should 1) self-administer alcohol orally (e.g., drink from a sipper), 2) consume enough alcohol to attain a pharmacologically high BAC, 3) consume alcohol for its pharmacological effects, irrespective of its taste, smell, or caloric value, 4) be willing to work for alcohol (e.g. operant responding), 5) express both metabolic and functional tolerance after chronic alcohol access, 6) show an alcohol dependence as characterized by withdrawal symptoms (e.g. seizure threshold and anxiety) when no longer provided access to alcohol, and 7) exhibit a “loss of control” (an increase in consumption levels over baseline) when alcohol is reinstated after a period of imposed abstinence to BD (Cicero, 1979, Lester and Freed, 1973, McBride and Li, 1998). Several alcohol preferring rat lines exist and have been evaluated (to some extent) using the 7 criteria listed above.

Alcohol-Preferring (P) Rat Lines

Several selectively bred alcohol preferring and non-preferring rat lines have been developed worldwide. As reported by Mardones, two of the earliest lines developed were the University of Chile B (UChB; alcohol preferring) and A (UChA; alcohol non-preferring) rat lines which date back to the early 1950’s (Mardones and Segovia-Riquelme, 1983). Approximately 15 years later, the researchers in Helsinki, Finland began breeding the Alko-Alkaline (AA; alcohol preferring) and Alko-non-alkaline (ANA; alcohol non-preferring) rat lines (Eriksson, 1968). The alcohol preferring (P) and non-preferring (NP) rat lines followed in the next decade, bred originally at the Walter Reed Army Institute of Research in Washington, D.C. and then continued at the Indiana University School of Medicine (Li, 1977). In 1981, researchers at the University of Cagliari (Italy) began breeding the Sardinian alcohol-preferring (sP) and non-preferring (sNP) rats (Mardones and Segovia-Riquelme, 1983). Finally, in the mid 1980’s the high-alcohol drinking (HAD) and low-alcohol drinking (LAD) selectively bred replicate rat and mouse lines were developed at the Indiana University School of Medicine (Li et al., 1993). These alcohol-preferring selectively bred rat lines have been evaluated to some extent using the criteria for an animal model of BD. Among the selectively alcohol-preferring rat lines, the P rat line meets all seven criteria for an animal model of BD. It is important to note that there exists high alcohol drinking (HAD) rats and mice (Crabbe et al., 2010) that are commonly used in research which may also have RDS behaviors, however, the current review will focus on the Indiana University alcohol-preferring (P) rat lines.

Behavioral Phenotypes of the P rat

The P rat line has proven useful in depicting phenotypic behaviors, heritable factors, and neural systems associated with excessive alcohol drinking (McBride et al., 2014). While table I summarizes relevant publications on behavioral phenotyping of the P rat line, table II lists neurochemical findings. When subjected to the two-bottle choice between 10% v/v alcohol and water, P rats voluntarily consumed larger amounts of alcohol than the other strains including Fawn-Hooded (FH), alcohol-accepting (AA), alcohol-non preferring (NP), and alcohol avoiding (ANA) (Boris A. Badishtov, 1994). Under similar conditions, P rats drink greater than 4 g of ethanol/kg body weight/day, whereas NP rats drink less than 1g/kg/day (Li et al., 1987). Similarly, sardinian alcohol-preferring (sP) rats exhibit increased preference for ethanol over water in the two-bottle choice test (Colombo et al., 1995). Not only do P rats self-administer greater amounts of ethanol (Murphy et al., 1989) than NP, but, with continuous access to 2–30% w/v ethanol and water on an FR5 schedule of reinforcement (i.e., five lever responses required per reinforcement), P rats exhibited greater preference for all ethanol concentrations versus water whereas NP exhibited the opposite effect. Furthermore, P, but not NP, rats have been reported to intracranially self-administer nanoliter quantities of ethanol (50–200 mg%) into ventral tegmental area (VTA), possibly attributed to differential sensitivity to the reinforcing effects of ethanol in the VTA of P and NP rats. More recently, within the P rat line, Marchant and colleagues report a bimodal distribution in response to punishment in the form of three constant shock intensities, potentially identifying variations in P rats that segregate out compulsive drug seeking behavior (Marchant et al., 2018). A key feature of addiction is compulsive drug use despite the negative consequences. Thus, like other models of drug addiction (Blackwood et al., 2019, Cadet et al., 2019), the P rat, with its well-studied neurobiological correlates, can be used to segregate various submodalities of drug abuse.

Table I.

Behavioral phenotypes of the P rat line

Behavioral Phenotype	Finding/Observation	Reference/s
Alcohol preference (2-bottle choice drinking)	Voluntarily consumed the most alcohol P > Long Evans	(Boris A. Badishtov, 1994) (Beckwith and Czachowski, 2016)
Operant self-administration	FR-4: 10% EtOH. P > NP in number of drinking bouts/day All FRs; P > Long Evans	(Files et al., 1998, Samson et al., 1998) (Beckwith and Czachowski, 2016)
5- Choice Serial Reaction Time Task (Impulsivity)	P = NP in number of premature, anticipatory responses P > NP in time spent in food magazine area P > NP in goal-tracking responses P < NP in sign-tracking behavior	(Pena-Oliver et al., 2015)
Delay-discounting procedure (Impulsivity)	P > Wistar P > Long Evans, HAD2	(Linsenbardt et al., 2017) (Beckwith and Czachowski, 2016)
Open field test (locomotor activity)	P > active than NP	(Boris A. Badishtov, 1994)
Open field defecation	NP > defecation than P	(Boris A. Badishtov, 1994)
Anxiety-Like Behavior Elevated plus Maze (Time in open arms)	No difference between P and NP rats (P = NP)	(Irina V. Viglinskaya, 1994)
	Exhibited anxiety-like behavior as compared to NP (P < NP)	(Stewart et al., 1993) (Hwang et al., 2004)
Depression-like Behavior Forced Swim Test (Immobility)	Significantly less immobile than NP rats (P < NP)	(Irina V. Viglinskaya, 1994) (Kiianmaa et al., 1991, Stewart et al., 1993, Tuominen et al., 1990)
Forced Swim Test (Escape attempts)	Significantly more escape attempts than NP rats (P > NP)	(Irina V. Viglinskaya, 1994) (Godfrey et al., 1997)
Pain sensitivity (hot plate)	P < NP	(Kampov-Polevoy et al., 1996)
Acoustic Startle Response	P = NP or P > NP	(Jones et al., 2000, McKinzie et al., 2000)
Novelty seeking	P > NP in response to novel odors P = NP in nose-poking response to novel odors	(Nowak et al., 2000)

Table II:

Neurochemical correlates of the P rat lines

Neurotransmitter system/receptor/metabolite/s	Finding/Observation	Reference
5-HT and 5-hydroxy-indole-acetic acid (5-HIAA)	P < NP (in cortical and limbic regions)	(Murphy et al., 1982, Murphy et al., 1987)
5-HT immunostained fibers	P < NP (in anterior frontal cortex, Acb and ventral hippocampus)	(Zhou et al., 1991a, Zhou et al., 1991b)
DA	P < NP (in the Acb and anterior striatum)	(Murphy et al., 1982, Murphy et al., 1987)
DA neuronal projections	P < NP (from VTA to Acb)	(Zhou et al., 1995)
DA metabolites (DOPAC and HVA)	P < NP (in the Acb and anterior striatum)	(Murphy et al., 1982, Murphy et al., 1987)
D2 receptors	P < NP (in the VTA and Acb)	(McBride et al., 1993)
GABAergic terminals	P > NP (in the Acb)	(Hwang et al., 1990)
μ-opioid receptors	P > NP (in limbic areas)	(McBride et al., 1998)
Neuropeptide Y (NPY)	P < NP (amygdala, hippocampus, and frontal cortex)	(Ehlers et al., 1998)
CRF	P < NP (Amygdala, hypothalamus, prefrontal cortex, cingulate cortex)	(Ehlers et al., 1992) (Hwang et al., 2004)
N-acetylaspartate (NAA), Choline-containing compounds (Cho), Total creatine (tCr)	P < NP	(Zahr et al., 2014)
Neurotensin	P < NP	(Ehlers et al., 1999)
Substance P, Neurokinin	P < NP	(Slawecki et al., 2001)

Mechanistically, BD/AUD/RDS are all characterized by low levels of DA and/or impaired DA homeostasis in the reward pathway, a dysregulation necessary to transition to addiction. P rats can be said to emulate RDS characteristics as follows: P rats have 1) decreased dopaminergic neuronal projections from VTA to Acb, 2) decreased expression of dopamine D2 receptors in the VTA and Acb of P rats, and 3) decreased dopamine and its metabolites in the Acb and anterior striatum. Overall, these neurobiological findings suggest that impaired dopaminergic system within the reward circuitry of the P rats is not only responsible for its increased preference for ethanol, but also for depressive, anxious, and impulsive characteristics reported (Sakharkar et al., 2014). A therapeutic aimed to replenish precursors for neurotransmitter systems and other influencers of dopamine tone was effective in reducing appetitive and consummatory behavior in P rats regardless of the route of administration (Solanki et al., 2020). Abnormalities in the serotonergic (5-HT) system of P rats (Stewart and Li, 1997) and the increased number of GABA terminals (Davis and Wu, 2001) are thought to contribute to the increased alcohol tolerance and withdrawal symptoms observed in the P rat line. Greater densities of the μ-opioid receptors are thought to increase DA transmission within the reward circuit, working in an indirect manner to increase the reinforcing properties of alcohol (Herz, 1997). Pertinent to this, P rats have been shown to have increased expression of μ-opioid receptors (MORs) in the mesolimbic brain regions: activation of MORs in the NAc shell with DAMGO enhanced operant self-administration of alcohol and cue-induced reinstatement (Richard and Fields, 2016), and direct blockade of μ-opioid receptors in the VTA blocks ethanol-induced conditioned place preference, important for context associations (Campos-Jurado et al., 2020). Altogether, impaired DAergic, serotonin and opioid systems of the P rats underscores the validity of this pre-clinical model for alcoholism or RDS. Moreover, neuropeptides CRF and NPY (neuropeptide Y) have are lower in the amygdala, hypothalamus, and pre-frontal cortex of P compared to NP rats (Ehlers et al., 1992, Ehlers et al., 1998). A study by Hwang (Hwang, 2001) identified significantly lower CRF levels in the amygdala of P relative to NP rats. These findings are of interest as restraint stress and ethanol withdrawal-induced increase in CRF levels within the amygdala has been reported (Merlo Pich et al., 1995), suggesting the potential role of CRF in mediating aversive effects of ethanol. Furthermore, reduced levels of NPY in the amygdala is linked with high ethanol consumption in P rats (Murphy et al., 2002). Interestingly, intra-cerebroventricular administration of NPY in the P rats decreased ethanol intake and subsequently increased ethanol-induced sedation in P rats (Badia-Elder et al., 2001). These findings support the notion that NPY may play a central role in a genetic predisposition for increased alcohol seeking and drinking behavior, a characteristic feature of RDS. Based on the knowledge available in literature, some hypothetical structural characteristics within the mesocorticolimbic system of the P rat line compared to a control animal with intact reward processing is depicted in figure 4. Neurotransmitter and neuromodulator data in P rats are summarized in Table II.

FIgure 4: Possible Neurocircuitry in Alcohol-Preferring (P) compared to Non-Preferring (NP) rat.

Dopaminergic signaling is likely blunted in P rats, possibly due to low levels of dopamine or dopamine receptors. P rats also exhibit reduced serotonin, CRF, and NPY compared to the NP rat line. 5-HT, 5-hydroxytryptamine; AMG, amygdala; CRF, Corticotropin Releasing Factor; DA, Dopamine; PFC, prefrontal cortex; NAc, nucleus accumbens; NPY, Neuropeptide Y; VTA, ventral tegmental area; NAc, Nucleus Accumbens; PFC, prefrontal cortex; VTA, ventral tegmental area; VP, ventral pallidum.

Although we focus on the P rat as a model of addiction, there are many other well-developed models for nicotine, amphetamine, cocaine, and heroin overuse and abuse. Towers et al. show that extended access to heroin IV self-administration leads to increased heroin intake and dependency in mice(Towers et al., 2019). Even Sprague Dawley rats can be trained to self-administer oxycodone and then stratified for OUD based on their continued escalation of oxycodone use despite punishment with electrical shock (Blackwood et al., 2019). These shock-resistant rats uniquely expressed increased immediate early gene egr3 mRNA in the PFC, and this finding could lend insight into persistent opioid use in the presence of adverse consequences (Blackwood et al., 2019).

Impulsivity in the P rat and other models

Impulsivity falls within the RDS behavioral spectrum, and a strong body of evidence supports the notion that impulsive decision making is a heritable risk factor that co-segregates with alcohol use disorder (Linsenbardt et al., 2017, Dalley and Robbins, 2017), even as a handful of studies dispute the link between the two (Peña-Oliver et al., 2015). There do exist numerous findings of impulsive choice in P rats and other rodent models of alcohol preference when assayed with a delay discounting task, and this impulsivity does not seem to be influenced by prior exposure to alcohol (Balan et al., 2018, Dalley et al., 2007, Giorgi et al., 2019, Linsenbardt et al., 2017, Winstanley et al., 2010). Dalley and Robbins penned a comprehensive review segregating neural circuits and tasks that test specific aspects of impulsivity, i.e., impulsive and/or risky choice associated with temporal discounting versus impulsive action associated with premature (motor) responding; they discuss functionally opposing roles of the NAc shell and core in mediating these actions (Dalley and Robbins, 2017). Mainly, impulsivity is associated with low DA in the NAc core and elevated DA release in the NAc shell. By contrast, reduced D2/D3 availability in both the NAc shell and core have been reported in high-impulsive versus low impulsive subjects, albeit only the core is implicated in impulsive choice associated with temporal delay discounting (Barlow et al., 2018, Dalley et al., 2007, Dalley and Robbins, 2017, Diergaarde et al., 2008, Oberlin and Grahame, 2009, Winstanley et al., 2010). These functional differences in the NAc are important to consider as we evaluate molecular, biochemical, and neuroanatomical findings surrounding impulsivity. In addition to the NAc, Drug dependence and impulsive behavior are tightly regulated by corticostriatal circuits involving the PFC, in addition to the NAc and are modulated by dopamine (DA) as well as serotonin (5-HT) in the dorsal raphe. Two separate studies by Bolla and colleagues report dysfunction within the pre- and orbito-frontal cortex in recently abstinent cocaine abusers (Bolla et al., 2004, Bolla et al., 2003) whilst it was also shown in another study that excessive cocaine intake is associated with decreased dopamine signaling in male Wistar rats (Willuhn et al., 2014).

Hypodopaminergia is indeed thought to contribute to the cognitive deficits associated with drug abuse and impulsivity. Impulsivity is associated with not just substance use disorders, but is a natural part of average developing adolescents where genetics and environment may cause a significant increase in risk-taking behavior, in a gender-specific manner: Conner et al, in a study of adolescents who averaged 14.5 years of age, developed predictive models supporting hypodopaminergia as a predictor of drug use in males, but for females, a deleterious environment was the salient predictor (Conner et al., 2010). A study of early childhood impulsivity concluded that early impulsivity alone was a risk factor for substance use by age 22, but when combined with their rejecting of parenting, was highly correlated with problems of aggression in adolescence (Hentges et al., 2018). The increase in risk-taking behavior was also reported in individuals with attention deficit/hyperactivity disorder (ADHD), gambling, and other neuro-psychiatric disorders (Congdon and Canli, 2008, Evenden, 1999). Several components of impulsivity, including delay discounting and poor attention were reported in alcohol and substance abusing individuals (de Wit, 2009) and in ADHD versus healthy volunteers, tonic and phasic release of dopamine were attenuated as determined by PET imaging (Badgaiyan et al., 2015), consistent with animal models discussed previously.

Preclinical studies use delay discounting to assess impulsive choice of small, immediate rewards over greater, delayed rewards. This is thought to involve executive working memory components, mediated by reciprocal connections between frontal cortex, hippocampus and amygdala (Winstanley, 2007). A recent study reports that P rats exhibited increased delay discounting (decision-making impulsivity) along with lack of behavioral inhibition (motor impulsivity) (Beckwith and Czachowski, 2016). These data corroborate the notion that P rats are a highly impulsive as well as a “high-seeking” model to be used to test those etiological components of RDS.

In terms of negative emotional states, CRF is the neuropeptide central to the stress response and that mediates anxiety and negative emotional states associated with drug dependence and impulsivity (Logrip et al., 2011). Pertinent to this, Ehlers et al. reported increased CRF-induced neural activation in P rats compared with NP rats and though they found decreased basal levels of CRF in P rats, it was postulated that it is the differences in CRF neural regulation in the P and NP rats that may be responsible for the variation in their levels of alcohol consumption (Ehlers et al., 1992). Drugs of abuse, including alcohol, acutely activate the HPA axis leading to increased hypothalamic release of CRF in the extended amygdala (Buckingham and Hodges, 1979, Calogero et al., 1989, Rivier et al., 1984, Sarnyai et al., 1992). Such acute HPA axis activation reportedly mediates drug-induced locomotor sensitization (Cole et al., 1990). Indeed, P rats exhibited increased locomotor activity in the open field test as compared to NP. Several studies have reported anxiety- and depression-like behavior in P rats along with poor cognition, which could be attributed to three key brain regions – hippocampus, amygdala and PFC (Arnsten, 2009, Patki et al., 2013). However, it is important to note there are some conflicting reports about locomotor and anxiety-like behavior in P rats. In a study by Roman and colleagues, exploratory, shelter seeking and risk-assessment behavior in P and NP rats were assessed using multivariate concentric square field (MCSF) test. They found that P rats exhibited lower activity and explorations but higher risk-taking behavior than NP as evidenced by increased percentage visits to the ‘risk’ areas (Roman et al., 2012). They also did not find higher anxiety-like behavior in P rats on the elevated plus maze test (Roman et al., 2012). These findings contrast with previous studies suggesting higher anxiety-like behavior in P rats compared with NP rats (Hwang et al., 2004, Stewart et al., 1993). The factors that underlie the discrepancy between these studies are not clear. However, undefined procedural differences could have affected the outcomes. In fact, environmental factors such as light/dark circadian cycles are known to significantly impact ethanol intake, whereby darkness increases and light decreases consumption. Earlier work developed and promoted the concept of darkness induced drinking in rodents, but provided a melatonin –related mechanism stemming from the pineal gland (Blum et al., 2014a, Reiter et al., 1973). Thus, the P rat, with its complex but well characterized phenotype may have construct validity for investigating many of the behavioral spectra associated with RDS, including compulsive behaviors and the emotionality associated with discounting punishment (Marchant et al., 2018).

Impulsivity and Gambling Disorder as a model of RDS

As discussed in previous sections, both the MD model and the P rat shown higher impulsivity, co-morbid with binge alcohol drinking and depressive-like behavior using a Delay Discounting paradigm (Gondre-Lewis et al., 2016b, Liu et al., 2011, Balan et al., 2018, Beckwith and Czachowski, 2016, Everitt et al., 2008). This same paradigm has been fine-tuned to not only measure impulsivity, but also decision-making and gambling addiction (Nower et al., 2004, Potenza, 2008). At its base, gambling involves the reward pathway in much the same way as other types of addiction, where the maladaptive behavior persists, regardless of adverse consequences. For humans, this can include loss of income, livelihood and family. It is suggested that differences in decision-making may relate to genetic factors or early life experience prior to exposure to drugs of abuse, and the reverse may also be true where exposure to drugs of abuse during adolescence can impact neural mechanisms that lead to impulsive behaviors in later life (Potenza et al., 2009).

The presence of Attention-Deficit/Hyperactivity Disorder (ADHD) may have an influence on the genesis and perpetuation of gambling disorder (Blaszczynski and Nower, 2002). Studies support a bidirectional relationship with respect to comorbidity such that neuropsychiatric disorders such as severe anxiety and depression can also serve as risk factors in the development of, or can arise as consequences of gambling disorders (Chou and Afifi, 2011, Dussault et al., 2011), thereby manifesting as a maladaptive coping mechanism (Blaszczynski and Nower, 2002). Again, the mesocorticolimbic pathway from the VTA to the NAc is implicated in behavioral addictions. Interestingly, gambling disorders are associated with dysfunctional process of the reward system described in RDS, with evidence of a functional dissociation between reward anticipation and outcome rather than a single reward circuit (Knutson et al., 2001). It would be important to establish how early pathological gambling in adolescents and young adults could load onto anxiety, depression, obsessive-compulsive disorder, hostility or other mood and personality disorders (Estevez et al., 2015). Rodent models of early-life gambling do not yet exist.

Cocker and Winstanley (Cocker and Winstanley, 2015), as well as Nautiyal and colleagues (Nautiyal et al., 2017) recently reviewed animal models of gambling-related behavior where a series of rat and mouse gambling tasks have been developed to mimic the human Iowa Gambling task where a choice is given between risky (high risk, high reward) and safe choices (Buelow and Suhr, 2009). There is also a model primarily focused on rodent gambling in the slot machine, i.e., the near miss effect on subsequent decision making, and various aspects of impulsivity, tasks of nonoptimal decision-making, and obsessive-compulsive behavior (Joel, 2006). It was found that a D2/D3 agonist, ropinirole, enhanced gambling performance, whereas propranolol, the beta adrenergic receptor blocker reduced compulsive-like gambling behavior in the rodent slot machine task (Cocker et al., 2019). One caveat of the slot machine task is the clear sensory effect that may help the animal discriminate between the outcomes. Other less frequently used gambling paradigms involve the use of intracranial self-stimulation as a reward instead of food, which has some advantages by eliminating potential feeding-related confounds (Rokosik and Napier, 2011, Tedford et al., 2014).

In gambling disorder, a hallmark is risk-taking similar to some RDS conditions. Young et al (Young et al., 2011) observed that the knockdown of the gene encoding the DA transporter in mice caused increased extracellular dopamine levels, resulting in riskier choices in the mIGT (mice Iowa Gambling task). In another study, D₁ antagonists, as well as dopaminergic lesions to the dorsolateral striatum or the nucleus accumbens core was observed to increase impulsive choice in delay-discounting tasks in rodents, resulting in choice of the smaller non-delayed reward more often (Koffarnus et al., 2011). Some animal models more relevant to the compulsive behavior found in gambling disorders have used the persistence of motivation to obtain a reward despite negative consequences as a measure of compulsivity (Radke et al., 2017). Reduced expression of D₂ receptors in the striatum results in increased susceptibility and earlier onset of this compulsive behavior. Furthermore, there is strong evidence supporting increased sensation seeking in patients with GD compared with healthy volunteers (Nower et al., 2004). A study by Reuter et al. (Reuter et al., 2005) reported reduced activation of the fronto-striatal circuit in response to monetary rewards, suggesting that gambling disorder is typified by a blunted response to reward stimuli as found in RDS. Another study also reported that gambling disorder showed diminished reward and punishment sensitivity, which is indicated by hypoactivation of the ventrolateral prefrontal cortex when money was gained or lost (de Ruiter et al., 2009). Additionally, firing of dopamine neurons in the brain reward circuitry of gamblers is evoked not just by a reward, but also the prediction of a reward (Fiorillo et al., 2003), which may be one of the factors driving the compulsive need to gamble. The impulsive–compulsive spectrum shift that occurs in drug use disorders also takes place in gambling disorders (Jentsch and Pennington, 2014), with evidence indicating that dopamine D2 receptors underlie the experience of reward secondary to both disorders (Zack and Poulos, 2007, Volkow et al., 1999). DA transmission is strongly implicated in the neural basis of sensation seeking. Several studies have suggested altered punishment and reward sensitivity in gambling disorders, as seen in the Sensitivity to Punishment and Sensitivity to Reward Questionnaire (SPSRQ) (Goudriaan et al., 2004), and so in a translational report using both human and mouse subjects, increasing overall DA signaling through DA reuptake blockade resulted in increased sensitivity to high-reward outcomes (van Enkhuizen et al., 2014).

Furthermore, dopamine agonists often utilized as pharmacotherapy for individuals with Parkinson’s disease and restless leg syndrome can lead to pleasure-seeking behaviors such as hypersexuality and gambling, ostensibly through dysregulation of the dopamine reward pathway (Driver-Dunckley et al., 2007). On the contrary, pharmacotherapy with dopamine antagonists has been effective in treating alcohol dependence (Hutchison et al., 2006), although there is no evidence to support the efficacy of this approach in gambling disorders (Fong et al., 2008). Moreover, there are alterations in DNA methylation of the dopamine D2 receotor gene and association with lifetime history of pathological ganbling (Hillemacher et al., 2015).

Post-Traumatic Stress Disorder as a Model of RDS

Post-traumatic stress disorder (PTSD) is also a complex and multifaceted neuropsychiatric disorder that can develop from a combination of genetic factors and prior exposure to intensively stressful and traumatic event(s) which produce psychological distress and a cocktail of behavioral disruptions comparable to RDS. PTSD is co-morbid with other psychiatric disorders such as major depressive disorder (MDD) (Rytwinski et al., 2013), anxiety spectrum disorders (Ginzburg et al., 2010), and substance abuse disorders (McCauley et al., 2012). Based on the fact that these traumatic memories are usually retrieved by exposure to conditioned cues, it is therefore proposed that PTSD combines some aspects of exaggerated stress responsiveness and enhanced fear conditioning (Ressler, 2010). There are several studies reporting the association of PTSD with a blunted hypothalamic–pituitary–adrenal (HPA) activity following a traumatic occurrence, shown by hypersensitive glucocorticoid receptor and greater glucocorticoid suppression following dexamethasone administration (Belda et al., 2008, Ströhle et al., 2008, Daskalakis et al., 2013).

While it is impossible to model all the complex behavioral phenotypes of PTSD in animals, animal models must at least satisfy a combination of criteria that lend face, construct and predictive validity (Siegmund and Wotjak, 2006). Importantly, there is no single acceptable model for PTSD, though there are several stress paradigms that have successfully modelled some of the neurobiological mechanisms and behavioral deficits associated with the disorder (Yehuda and Antelman, 1993). These stress paradigms used to induce PTSD-like symptoms can be grouped into: physical, social and psychological stressors.

Animal models of PTSD that use physical stressors include foot shock, restraint stress, and single prolonged stress (Whitaker et al., 2014). These can be brief electric shocks administered via a metal rod floor with ranging intensities (Van Dijken et al., 1992, Pynoos et al., 1996), longer duration tail shock and inescapable shock paradigms (Servatius et al., 1995). These types of exposure induce decreased locomotor activity in novel environments, robust cue-conditioned fear responses (Johansen et al., 2011), increased anxiety-like behavior following testing on the EPM (Belda et al., 2008) and learned helplessness that mimics depression (Bali and Jaggi, 2015). Acute and chronic restraint stress produces increased anxiety-like behaviors on the EPM (Vyas et al., 2002), and is sometimes used in conjunction with the forced swim test or other kinds of stressors as a type of unpredictable stress. Social stressors such as social defeat, social isolation and housing instability has been shown to produce long lasting anxiety-like behaviors (Zoladz et al., 2012), submissive-type isolations and depression (Krishnan et al., 2008), and enhanced acoustic startle responses (Pulliam et al., 2010). Furthermore, the predator and predator odor exposure, a psychological stressor, increases freezing and avoidance behaviors and is often combined with other stress models such as social instability (Zoladz et al., 2008). These combined stress paradigms also produce significantly increased anxiety, and enhanced startle responses. A study by Perez-Garcia et al (Perez-Garcia et al., 2018) used an animal model of blast exposure mimicking a low-level blast exposure to produce PTSD-like traits, including stress and increased anxiety. Thus, although PTSD may have a genetic predisposition component, it can be induced in children and adults via a variety of traumatic stressors in everyday life such as food and housing insecurity, gun and domestic violence, bullying and traumatic shocks, in addition to combat or war violence. Toxic stress due to the potential to be a crime victim and fear of law enforcement has also been reported as PTSD for some inner-city dwellers. It is beyond the scope of this review to address each of these, but the view of PTSD must be comprehensive. As far as animal models, we compare here characteristics discussed in some PTSD models, many of which overlap with MD ELS models discussed earlier (Figure 3). In some PTSD models, many behavioral/physiological phenotypes have not yet been investigated. There is also a genetic mouse model of PTSD, the 129S1/SvlmJ which is characterized by risk-taking behaviors (Hefner et al., 2008).

Although PTSD is associated with a plethora of behavioral deficits including increased anxiety, exaggerated stress reactivity and anhedonia, with reports of decreased interest in pleasurable activities, lack of motivation and inability to feel positive emotions even with rewards (Franklin and Zimmerman, 2001, Nawijn et al., 2015), they are supported by differential findings of dopamine receptors. Comings et al (1996) provided the first evidence that the DRD2 gene is associated with susceptibility to PTSD (Comings et al., 1996). Increased striatal levels of the DA transporter (DAT) using Single-photon emission computed tomography imaging has been shown in PTSD patients (Hoexter et al., 2012). A negative correlation between increased dissatisfaction and kappa opioid receptor (KOR) bioavailability in ventral striatal circuits of PTSD-type patients has also been shown (Pietrzak et al., 2014). Due to the role of DAT and KORs in the regulation of DA levels, this may contribute to the hypo-dopaminergic state observed in RDS. Dysregulation of dopaminergic pathway in RDS, as discussed previously, contributes to drug-seeking behavior, and thus may be linked to the vulnerability of PTSD patients to addiction. This is further supported by studies linking traumatic stress reactivity with a wide variety of impulsive behaviors, including substance use disorders (Edwards et al., 2013, Gilpin and Weiner, 2017, Brady et al., 2004, Enman et al., 2015), anti-social behaviors (Booth-Kewley et al., 2010), self-harm (Sacks et al., 2008) and risky sexual behaviors (Strom et al., 2012).

Compulsive Eating Behaviors

The compulsive need for drugs despite its adverse consequences are trademarks of substance use disorders and addictive behaviors. This compulsion is not limited to the abuse of illicit or traditional drugs. It is suggested that compulsive drug intake shares several neural mechanisms with compulsive eating disorders (CED) (Berridge et al., 2010, Volkow et al., 2013). CED is characterized by binge eating, bulimia nervosa and food addiction (Moore et al., 2017), with binge eating disorders (BED) defined as uncontrolled consumption of palatable foods (high fat or sugar-rich foods) within a short period of time (within any 2-hour period) (Moore et al., 2018). This uncontrolled eating pattern has led to the assumption that compulsive eating disorders might embody increased impulsivity within the obesity spectrum phenotypes (Schag et al., 2013). Bulimia nervosa, in specific, is very complex as it involves the consumption of a large amount of food in a short period of time, i.e., binge eating, followed by purging or fasting due to feelings of guilt or shame. These BEDs may straddle the line between an emotional disorder and an addictive disorder (Kinzl and Biebl, 2010).

Binge-eating occurs regardless of presence or absence of hunger; and is implicated in emotional distress. Since subjective measures of human distress cannot be achieved in animal models, measures of depression, anxiety, stress, and fear may provide a means to evaluate distress possibly associated with binge-eating (Coscina and Garfinkel, 1991). Several studies have used a palatable diet to induce various forms of binge-like eating disorders in animal models (Hone-Blanchet and Fecteau, 2014, Corwin et al., 2011, Ifland et al., 2009, Hagan et al., 2002, Morgan and Sizemore, 2011, Avena et al., 2012). Avena et al. proposed the sugar addiction model (Avena et al., 2008) where the rats are deprived of food daily for 12-hours, followed subsequently by access to 10% sucrose/ 25% glucose and rodent chow for another 12 hours. After a few days of treatment, the food-deprived rats were shown to binge on the sugar solution with daily increase in intake compared to control (Hoebel et al., 2009, Di Segni et al., 2014). Corwin et al. proposed the limited access model (Corwin et al., 2011) where the rats are given intermittent, time-limited (generally 1–2 h) access to palatable food in addition to the rodent chow, which induced binge eating even without hunger.

In addition, several studies have evaluated compulsive eating disorders by measuring the animal’s impulse to work for and ingest palatable foods despite negative consequences, modeled by pairing an unconditioned stimulus like foot shock with a cue-conditioned stimulus like light (Johnson and Kenny, 2010, Corwin et al., 2011, Heyne et al., 2009). This combination of stimuli is reportedly more effective in inducing binge-eating behaviors when the animals are exposed to at least three cycles of fasting and refeeding before foot shock is applied (Artiga et al., 2007). Environmental forms of stress like maternal separation which had been discussed earlier as a form of RDS have also been used in place of physical stress, resulting in depressive-like and anxiety-like behaviors. The combination of MS and repeated fasting and refeeding cycles during adolescence resulted in binge-eating behaviors (Jahng, 2011). Severe chronic early life stress has also been shown to alter eating behavior in adult animals, with the results more prominent in females (Iwasaki et al., 2000, McIntosh et al., 1999).

Major contributions of hereditary factors to the development of binge-eating disorders have been reported (Trace et al., 2013, Yilmaz et al., 2015, Thaler and Steiger, 2017). Hence, the study by Patrono et al. comparing two mice inbred strains, C57BL/6J and DBA/2J substantiates a genetic vulnerability of DBA mice to compulsive eating behavior evidenced by low availability of D2 receptors (Patrono et al., 2015). Another study implicated C57BL/6NJ in the development of binge-eating disorders in mice models, and identified Cytoplasmic FMR1-interacting protein 2 as a major genetic factor in development of BED (Kirkpatrick et al., 2017). Furthermore, variations in the proneness of rat strains have been studied, with results showing that the Sprague-Dawley female rat strain, but not the male strain is particularly vulnerable to binge-eating behaviors, while the Wistar female rat strain is resistant (Hildebrandt et al., 2014).

CEDs reflect a maladaptive stimulus-driven behavior similar to that which is postulated for drug-seeking; persistent repeated stimulations of the dopaminergic pathway in the nucleus accumbens (NAc) with palatable foods and accompanying signalling to the dorso-striatal dopaminergic pathways which results in food addiction (Everitt and Robbins, 2016, Everitt and Robbins, 2005). Self-restraint from these palatable foods may result in withdrawals signaled by dysphoria, anxiety, and depression (Iemolo et al., 2012). Furthermore, CED is the result of diminished reward sensitivity, a key determinant of RDS, which is the functional desensitization of the mesocorticolimbic dopaminergic pathway resulting in overeating as a means to alleviate negative emotional affect and a need to reactivate a hypofunctional reward circuit (Moore et al., 2017, Parylak et al., 2011, Geiger et al., 2009, Wang et al., 2001). This is an underlying characteristic of the disorders demonstrated in figure 1 as part of the behavioral octopus. Stice and colleagues used fMRI while presenting ice-cream/milkshake or water to individuals who gained or lost weight within a six-month period, and found that weight gain was associated with a reduction in striatal activation in response to palatable foods (Stice et al., 2010). Consistently, other research on obesity find a reduced availability of striatal D2 receptor in both humans (Volkow et al., 2008) and animal models of obesity(Halpern et al., 2013, Johnson and Kenny, 2010), compared to lean controls.

In the previous sections, the down-regulation of dopamine D2 function has been used to describe RDS, and is well documented in obesity and binge eating. Reduced D2R expression in the striatum can therefore infer a neuroadaptive response in order to compensate the excessive ingestion of palatable foods, with reduced sensitivity resulting in CED (Johnson and Kenny, 2010). While D2R antagonism in NAc reportedly increases binge eating behavior, activation of serotonin 2C receptors in DA inhibits binge eating behaviors in mice (Halpern et al., 2013, Doucette et al., 2015, Xu et al., 2017). Other pharmacological approaches to the treatment of binge-eating disorder includes administration of the D2 receptor antagonist raclopride which reportedly reduced sucrose intake in BED models (Wong et al., 2009). Methylphenidate which is known to inhibit the monoamine uptake transporters for DA (Bello and Hajnal, 2006), and GS 455534 which reduces DA synthesis was are also purported to reduce binge-like behaviors (Bocarsly et al., 2014). Moreover, a putative pro-dopamine regulator customized to one’s DNA polymorphisms, especially hypodopaminergia, successfully targeted obesity (Blum et al., 2007, Blum et al., 2008, Blum et al., 2015).

Other Models: Knockouts, immune deficiency

Of course, some of the under investigated rodent models might include the various mutant and knockout animals that have been developed as these have potentially severe RDS phenotypes. Many of the receptors, enzymes, transporters and catabolic partners of neurotransmitters and neuromodulators within the brain reward cascade could be candidates. In addition to the well characterized DRD2 and DRD1 knockout mice, recent studies with type 2 metabotropic glutamate receptor transgenic knockout (mGluR2-KO) rat compared to wild-type indicates that low mGluR2 expression may be a risk factor for opioid use disorder (Gao et al., 2018) and likely other RDS phenotypes. They exhibit higher heroin self-administration and dopamine in the nucleus accumbens in response to heroin. Likewise, Roman High Avoidance (RHA) rats discussed in the next section fits with the hypothesis of enhanced drug intake in the absence of mGluR2 in that they are a naturally-occurring KO mutant of the mGluR2 receptor and exhibit high cocaine self-administration (Fattore et al., 2009), high alcohol intake, impulsivity, and risk-taking behavior (Wood et al., 2017). Nonetheless careful consideration of converging region-specific mechanisms are necessary in the study of reward in these KO models as linked neurotransmitter systems may be more potently impacted in one region and not another. For example, mGluR2 deficient RHA rats have heightened serotonin 2a expression in the PFC, but not in the striatum (Fomsgaard et al., 2018) and this could lead to unique co-morbid conditions of this particular knockout.

In addition, receptors for CRF1 (Contarino, Kitchener et al., 2017), BDNF, 5-HT1, nAChR have been implicated in mediating reward, and the list is exhaustive (Li and Wolf, 2015, Faulkner and Deakin, 2014, Mohammadi et al., 2017). Genetic polymorphisms that confer RDS are strong candidates for study in animals, however these can have an ancestry-specific impact on reward deficiency and opioid use disorder in specific in humans (Abijo et al., 2019). It is not known how the differential genetics of groups of individuals can be modeled in the animal with accuracy in terms of eliciting a susceptible phenotype, a resilient phenotype, or a non-effect. In the age of personalized medicine, this is an important avenue that needs developing in animal models. Lastly, models of immune compromise or toll-like receptor (TLR) dysregulation can also serve as risk factors for RDS (Kashima and Grueter, 2017, June et al., 2015, Liu et al., 2011). Abnormal function of TLR4, in specific, has been shown to lead to long-term epigenetic changes that impact reward, anxiety, depression and synaptic physiology (Balan et al., 2018, Kashima and Grueter, 2017, Liu et al., 2011, Montesinos et al., 2016). In addition, there are some receptors implicated in neuropsychiatric disorders such as neuregulin 2 and its receptor ErRB (Yan et al., 2018) that have not yet been studied for other RDS phenotypes. Even extracellular matrix components CD44 and hyaluronic acid have been implicated in stress responses. CD44 KO mice exhibit stress-induced behaviors associated with anxiety, depression, anhedonia and despair, with reduced turnover of striatal dopamine and cortical serotonin (Barzilay et al., 2016).

Other models: Possible Limitations of the Hypodopaminergic hypothesis

The animal models of reward deficiency syndrome discussed thus far focus on low dopaminergic tone as a driver of the aberrant behaviors and associated clinical symptoms. Hypodopaminergia is the most prominent manifestations of RDS but there may be limitations in this hypothesis. There are some animal models in which a central hypodopaminergic trait does not seem directly correlated with the expected behavior, and others still where dopamine systems in reward are not fully investigated. Clearly, not all RDS behaviors are co-morbid with one another and thus there must be additional underlying features to unpack which understanding features of different models will allow.

There are additional animal models that have been bidirectionally selected and bred over several generations based on depression-like, anxiety-like, and impulsivity or novelty-seeking profiles. Of note are the Roman rats where behavioral correlates of mental health conditions are co-segregated with either extremely rapid or poor acquisition of avoidance behavior in a shuttlebox paradigm. The Roman low-avoidance (RLA) line is associated with exaggerated HPA-axis reactivity. RLA rats exhibits increased anxiety, decreased novel exploratory activity, and increased freeze response in fear-conditioning paradigms (Boersma et al., 2009), in addition to less robust mesolimbic dopamine tone and diminished vulnerability to drug-seeking behaviors (Giorgi et al., 2005, Giorgi et al., 2019). In fact, Fattore and colleagues showed that compared to RHA rats, RLA rats did not acquire cocaine self-administration or lever press as robustly for cocaine (Fattore et al., 2009). Conversely, the RLA line phenotypic counterpart, the Roman High-avoidance (RHA) rat strain is characterized by its impulsivity -with impairment at solving a spatial reversal learning tasks. They exhibit risk-taking behavior, reduced anxiety, high novelty seeking and intense natural and drug-seeking behaviors (Giorgi et al., 2019), including higher acquisition, maintenance, and slower extinction of cocaine self-administration(Fattore et al., 2009). In addition, RHA rats exhibit more robust hormonal responses to various forms of stress. Interestingly, decreased Dopamine D2 but increased Dopamine D1 receptors in RHAs is proposed to be linked to its novelty seeking and vulnerability to addictive behaviors (Guitart-Masip et al., 2006, Tournier et al., 2013), but more data regarding dopamine functionality in the reward system is necessary. Thus, these Roman rat models could help segregate behavioral health phenotypes, but importantly, consideration of both RLA and RHA behaviors relative to common outbred control strains is essential and will allow us to determine if these rats represent ‘two sides of the same coin’, or if they are truly divergent.

As discussed in the previous section, the natural absence of mGlu2 receptors in RHA rats is most likely as a result of a stop codon at cysteine 407 resulting in the functional absence of mGluR2, resulting in a naturally occurring KO mutant of the mGluR2 receptor. This may confer unique epigenetic and gene regulation profiles leading to the behavioral and neurochemical phenotypes characteristic of this strain (Wood et al., 2017, Klein et al., 2014) and further studies of both RHA and RLA in comparison to wild-type will help demonstrate how depressive behaviors segregate from addiction and impulsivity.

In the high anxiety-like (HAB) and low anxiety-like behavior (LAB) rats, voluntary alcohol consumption did not segregate with the high anxiety trait as in some other models (Henniger et al., 2002): LAB rats consume more alcohol at the onset than LAB, but alcohol did have more of an anxiolytic effect on HAB rats. Anxiety- and depressive-like behavior are comorbid in these rats, with an activated HPA axis compared to LAB neuroencodrine profiles(Landgraf et al., 2007). Importantly, heightened vasopressin release in the paraventricular nucleus of the hypothalamus led to the proposition of the vasopressin gene as a candidate involved anxiety(Landgraf et al., 2007, Neumann and Landgraf, 2012).

The Flinders Sensitive rat line (FSL), like WKY, is an accepted animal model of depression characterized by decreased locomotor activity, increased rapid eye movement sleep and exaggerated immobility with the forced swim test (Overstreet and Wegener, 2013, Overstreet, 1993). Unlike WKY, anxiety-like behavior is decoupled to depressive-like behavior in pre-adolescent FSL rats, i.e., they do not co-express anxiety behavior compared to control Wistar or SD rats, nor do they seem to exhibit adaptive responses to stress (Overstreet, 2002, Braw et al., 2006). Baseline D₁ and not D2 receptor mRNA in FSL rats are higher compared to SD controls, and after exposure to social isolation stress, D2 receptors were severely reduced (Bjørnebekk et al., 2007), potentially arguing for a stress-induced hypodopaminergia rather than an innate condition in these animals. Findings were inconclusive when examining changes to threshold in brain stimulation reward (BSR) experiments of the medial forebrain bundle in FSL, FRL, and outbred controls with a caveat that this study diverged from characteristic features previously published for the FSL/FRL rats (Matthews et al., 1996).

Finally, the apo-morphine susceptible (APO-SUS) line from Wistar rats, proposed as an animal model of schizophrenia-prone patients, was developed based on its behavioral response to a subcutaneous injection of apomorphine, a dopamine D1/D2 receptor agonist (Ellenbroek et al., 1995). They are marked by high behavioral response to apomorphine, hyperlocomotor response to novelty and exaggerated, long-lasting HPA-axis response to stress (van Vugt et al., 2014). The APO-SUS rats also exhibit a fleeing, unlike the freezing behaviors exhibited by its phenotypic counterpart, the apomorphine unsusceptible (APO-UNSUS) rats. APO-SUS rats are also characterized by a high density of D2 receptors and a higher stress-induced dopaminergic activation of the striatum, leading to increased stress-induced cocaine self-administration (van der Kam et al., 2005). van Schijndel and colleagues found that HPA-axis activity of the APO-SUS did not differ from the APO-UNSUS rats, and thus postulate that HPA-axis hyperactivity is not necessarily causally linked to dopamine responsiveness (van Schijndel et al., 2011). Although dopamine neurotransmitter levels were not measured in these studies, it is possible that the APO-SUS rat, as a model of ‘personality disorder’, may fail to meet conditions of hypodopaminergia. More research is necessary to segregate primary, secondary or tertiary causes of addiction, depression, anxiety, and impulsivity as we refine the characteristics of the endophenotype known as RDS.

Discussion and Conclusion

In this review, we evaluate animal models of Reward Deficiency Syndrome, the umbrella term that encapsulates impulsive compulsive, addictive behaviors as well as affective disorders. RDS is metaphorically represented as a behavioral octopus with low dopamine tone as a foundational characteristic leading to many different manifestations of hypodopaminergia inherent in numerous psychiatric diagnoses (Figure 1). We propose that many animal models already exist in the scientific community that could be framed in the context of hypodopaminergia to study RDS as a whole rather than monolithic disease paths. These models are well documented to have numerous overlapping symptomatologies associated with various RDS phenotypes. Yet, investigations are frequently uni-dimensional and not framed in the context of RDS. It is likely that the phenotypes being studied are not broadly thought of as belonging to the RDS family. However, the Wistar Kyoto rat, the HL mouse, models of maternal deprivation, abandonment, perinatal drug exposure and other ELS, the P rat, the HAD rat, rodents modeling gambling and other addiction, as well as the PTSD-induced or genetic rat -- there exist a number of phenotypes such as depression, learned helplessness, anxiety, impulsivity, drug addiction, hyperactivity (Figure 5). These provide a spectrum of behaviors to facilitate the study of RDS. From a neurochemical and neuroanatomical standpoint, there is striking overlap of biochemical correlates of these behaviors.

Figure 5. Animal Models of RDS and Core Behavioral Traits.

Behavioral traits associated with low dopamine (blue pie slices), and the animal models reviewed here (Clear rectangles) in which they have been detected (black). The animal models in red shows those for which no data about the behavioral trait is available as of the writing of this review.

Humans are unique. While there are a plethora of sophisticated animal models to help understand mechanisms related to RDS behaviors including alcoholism, heroin dependence, psychostimulant dependence, nicotine dependence, aggression, gambling, sex addiction, overeating, stress, and even ADHD, these animal models cannot truly capture the complexities of, for example, human depression that may lead to suicidal ideation. Nor can they truly capture the allure of video gaming, excessive shopping, or the compulsion for hoarding, or other very deep feelings that are part of daily life. However, these animal models can serve as mediational models that could help reduce the liability index for a given condition as described by Kendler and Neale (Kendler and Neale, 2010).

Understanding commonalities in these already established pre-clinical models, now currently viewed as models of RDS, could spark specific intervention on cross-cutting endophenotypes. Gene therapy or behavioral or pharmacological treatment approaches that reduce risk for clinical-like symptoms is a strength afforded by animal studies. As has been suggested these same targeted interventions would not be immediately possible in humans without longer-term studies (Gould et al., 2017, Gould and Gottesman, 2006). Of course, a more careful examination of sex differences is necessary as sometimes opposing or exaggerated responses occur in males versus females. Moreover, the epigenetic basis for the different behaviors might shed additional detail into the underlying characteristics to define a complex RDS biological signature. While we can’t know what the future holds for RDS and its treatment per se, through the understanding of genetics, epigenetics and drug and nutraceutical trials, much of which will benefit from animal studies, our future looks promising and will be enriched by continued exploration in both humans and animals.

Highlights:

• Reward deficiency is at the root of many mental/behavioral health disorders

• Commonly used animal models have construct and face validity for reward deficiency syndrome

• Rodents that model learned helplessness, early life stress, ADHD, depression, PTSD, excessive drug intake, gambling, and eating disorders share the hypodopaminergic trait and therefore may qualify as models of RDS

• Impulsive, compulsive, and addictive disorders exhibit low dopamine tone in brain reward centers.