Astrocyte-Neuron Interaction in the Dorsal Striatum-Pallidal Circuits and Alcohol-Seeking Behaviors

Abstract

In the striatum, two main types of GABAergic medium spiny neurons (MSNs), denoted striatonigral (or direct-pathway MSNs, dMSNs) and striatopallidal neurons (indirect-pathway MSNs, iMSNs), form circuits with distinct pallidal nuclei, which sends “GO” or “NO-GO” signals through the thalamus. These striatopallidal circuits evaluate and execute reward-seeking and taking behaviors. Especially, the dorsal striatum can be further divided into the dorsomedial striatum (DMS, equivalent to caudate in primates and humans) and dorsolateral striatum (DLS, equivalent to putamen), which orchestrates goal-directed and habitual reward-seeking and taking behaviors, respectively. Using optogenetics, chemogenetics and in vivo calcium imaging technologies combined with electrophysiology and digitalized behavior phenotyping, recent studies have revealed cell-, circuit- and context-specific functions of these microcircuits in addictive behaviors. Also, region-specific astrocytes regulate the homeostatic activities of the dMSNs and iMSNs as well as the downstream circuits, which determine the net balance of cortico-striato-pallidal activities to the thalamic neurons. This review will summarize the recent progress of striatopallidal circuits focusing on astrocyte-neuron interaction and, reward- and alcohol-seeking behaviors. Our review will also discuss the translational and clinical implications of these microcircuit studies.

1. Introduction

Uncontrolled compulsive reward-seeking despite negative consequences is a hallmark of addiction (Luscher et al., 2020). What role do conscious thoughts or actions play in abstaining from substance (like alcohol, cocaine, nicotine, or heroin) or non-substance (like gambling or gaming) rewards? The weakened executive control over compulsive reward-seeking or consumption is often attributed to irrational or uncontrollable behaviors, including addiction. Addicted humans and animals may exhibit the addictive behavioral modes without evaluating the proper or healthy cost/benefit analysis, which results in uncontrolled desire to procure the reward despite the negative consequences such as damaged relationships, financial and legal challenges, and addiction-related illnesses.

Among several plausible explanations of addiction theories, in this review, we will focus on how controllable goal-directed reward-seeking behavior shifts to uncontrollable (or less controllable) habitual seeking or taking behavior. Upon the loss of control in drug-seeking or -taking, repeated withdrawal and craving cycles will eventually lead to compulsive drug-seeking behavior (Koob, 2020; Koob and Volkow, 2010). Of note, it is a matter of imbalance between goal-directed and habitual reward-seeking behaviors, rather than complete switching between two behavioral patterns (Graybiel, 2008; Yin and Knowlton, 2006). Likewise, imbalance of the brain regions and circuits regulating these two behaviors may result in addiction including alcohol use disorder (AUD) (O’Tousa and Grahame, 2014).

This review will concentrate on brain circuits and microcircuits between the dorsal striatum and globus pallidus, although the ventral striatum is also a critical neural substrate for the value-based behavior, including goal-directed behavior (Keiflin and Janak, 2015; Mannella et al., 2013). Two distinct neurons denoted as direct-pathway medium spiny neurons (dMSNs) and indirect-pathway medium spiny neurons (iMSNs) interact with several different types of pallidal neurons directly or through the subthalamic nucleus (STN). Thus, these striatopallidal circuits will modify the cortical inputs by integrating many other brain regions, including ventral tegmental (VTA), substantial nigra pars compacta (SNc), amygdala, and hippocampus.

Interestingly, astrocytes are known to regulate striatal MSNs’ activities in the dorsal striatum and alter animal behaviors (Yu et al., 2018). Moreover, astrocyte activation changes behaviors through astrocyte-neuron interactions (Bull et al., 2014; Chen et al., 2016; Reissner and Pletnikov, 2020). Thus, we will provide updated information on how astrocyte-neuron interaction in the striatopallidal circuits contributes to goal-directed and habitual reward-seeking behaviors.

2. Striatal MSNs-GP circuit and its implication in goal-directed and habitual behaviors

Although no anatomical marker exists in rodents, the dorsal striatum (DS) has two functionally recognizable subregions. The dorsomedial striatum (DMS, equivalent to caudate in humans) is mainly known to regulate goal-directed reward-seeking behaviors, while the dorsolateral striatum (DLS, equivalent to putamen in humans) is primarily involved in movement and habitual behaviors (Graybiel, 2008; Voorn et al., 2004; Yin and Knowlton, 2006). These two dorsal striatum areas, DMS and DLS, are part of cortico-striato-pallidal circuits (Fig. 1a and b). In rodents, pyramidal glutamatergic neurons in layer 5 of the prefrontal cortex project to the striatum, where ionotropic and metabotropic glutamate receptors mediate the excitatory function of glutamate signaling in the striatal neurons. Since most striatal neurons (> 90%) are GABAergic, upon excitation, the axon terminals of striatal neurons release GABA to the output neurons, including pallidal areas. Upon activation of striatal neurons, the inhibited pallidal neurons disinhibit the thalamic neurons to drive the excitatory glutamatergic projections to the cortical areas, which completes a cortico-striato-pallido-thalamo-cortical loop (Fig. 1a).

Fig. 1.

Schematic illustration of cortico-basal ganglia (striatopallidal)-thalamic circuit in mice. (A) Sagittal view of the circuits. (B) Coronal view of the circuits. (C) Illustration of the cortico-striato-pallidal circuits for goal-directed and habitual reward- or substance-seeking behaviors. OFC, orbitofrontal cortex; mPFC, medial prefrontal cortex; S-MS Ctx, sensorimotor cortex; Chol, cholinergic neurons; dMSN, direct medium spiny neurons; iMSN, indirect medium spiny neurons; GPe, external part of globus pallidus; GPi, internal part of GP; STN, subthalamic nucleus; SNc, substantia nigra pars compacta, SNr, substantia nigra pars reticulata. Red arrow, glutamatergic neurons; Blue arrow, GABAergic neurons; Green arrow, dopaminergic neurons.

What determines goal-directed and habitual behaviors? The sensitivity to outcome devaluation and A-O contingency (a.k.a., A-O association) are the most commonly used behavioral measurement for goal-directed and habitual behaviors (Yin and Knowlton, 2006). Early studies suggested a functional dichotomous dissociation between DMS and DLS. While the DMS is involved in flexible goal-directed learning, the DLS is responsible for inflexible habitual response learning (Devan et al., 1999; Devan and White, 1999). Through DMS lesioning experiments (particularly the posterior DMS), the DMS’s inactivation abolished sensitivity to devaluation, which contributes to the A-O contingency degradation (Yin et al., 2005). Other studies validated the role of the DMS in the acquisition and expression of goal-directed behaviors (Ragozzino, 2003; Ragozzino et al., 2002; Yin and Knowlton, 2004). In humans, the DMS homolog region, the caudate, has a similar role in A-O contingency and goal-directed behaviors (Delgado et al., 2004; Tricomi et al., 2004; Zink et al., 2004). In contrast, DLS (or putamen) lesions impede the transition from goal-directed behavior to habit, and thereby goal-directed behavior remains through high A-O contingency (Yin et al., 2004, 2006). Additional preclinical and clinical studies further confirmed that neural connectivity from ventral to dorsal, and from caudate to putamen (DMS to DLS) transitions are correlated to severity or risk of addictive behaviors (Burton et al., 2015; Ersche et al., 2020; Everitt and Robbins, 2005).

Both the DMS and DLS containing two morphologically indistinguishable striatal MSNs, dMSNs, and iMSNs are known to project mainly to the internal part of GP (GPi) and the external part of GP (GPe), respectively. As illustrated in Fig. 1c, a majority of GPe neurons are GABAergic projections to the STN, then glutamatergic STN neurons activate the GPi, which balances the circuits between iMSNs and dMSNs in both the DMS and DLS to ultimately regulate movement and motivational behaviors. Since the DMS is receiving most cortical inputs from the medial prefrontal cortex (mPFC), including the anterior cingulate cortex (ACC), prelimbic (PL) and infralimbic (IL) regions, and orbitofrontal cortex (OFC) (de Kloet et al., 2021; Hunnicutt et al., 2016; Joel and Weiner, 2000; Lu et al., 2021), DMS → GP circuits regulate goal-directed behavior or A-O contingency, which requires the instrumental acquisition and learning to maximize the chance to acquire a reward. In contrast, the DLS forms a circuit with the sensorimotor cortex (SMC). Thus, it is known that DLS → GP circuits govern habitual behavior with stimulus-response (S-R) contingency. From the reward-seeking perspective, it is an effective brain system to shift from an energy-consuming A-O to an energy-saving S-R system in similar environments. Simply, repetition and over-training in the same context is a critical factor for habit formation (Packard, 1999; Packard and Knowlton, 2002).

Principally, animals have common patterns in procuring rewards depending on the types of conditioning they experience (a.k.a., experienced contingency). Two notable schedules are employed to shape goal-directed and habitual behaviors (Fig. 2a and b). In ratio schedule (often called random ratio, or RR), a proper and effortful response (such as a lever-press or a nose-poke; action) is associated with reward outcome with a certain probability (various probability in the case of RR). If the outcome value is high (or more pleasurable), its reinforcing effects promote a higher action rate. Thus, once the animal has sufficient motivation, the animal will establish optimal behavioral policy along with updating the probability of reward acquisition based on the action demand (the number of lever-press or nose-poke) for each reward outcome. Upon the outcome devaluation via free and unlimited access to the reward before operant tasks (mostly 1 h in animal’s home cage), the animal reduces its action rate compared to the animal in the valued state (even though the animal is still motivated to obtain the reward), showing flexible action selection. In an interval schedule (often called random interval, or RI), a response is only rewarded in a given time with a range of variable lap time regardless of response rates. The interval of uncertainty for acquiring reward upon a response is known to generate S-R habits (Derusso et al., 2010), meaning that action is not an essential factor for determining reward delivery. In contrast to RR schedules, the animal trained through RI schedules exhibits a lack of behavioral flexibility and a similar response between valued and devalued states (Fig. 2c and d).

Fig. 2.

Operant conditioning schedules for goal-directed and habitual reward-seeking behaviors. (A) Mice are trained in an operant chamber to learn an association between nose-poke (NP) and reward-delivery in the magazine. Tone and sound are reinforcing this association during the fixed ratio (FR) schedule. Through the repeated FR1 (one nose-poke = one reward), mice learn that nose-poking (action) results in reward (outcome). (B) After the FR1 training, when mice are given a random ratio schedule (RR), mice learn a policy-related probability of getting the reward with a trial and error process. In the case of RR, mice have to nose-poke random numbers to get a reward in a variable requirement. Since mice need to keep updating the policy through different RR schedules (e.g., RR5, RR10, etc), this schedule is similar to model-based (MB) reinforcement learning (RL). By contrast, after the FR1 training, when mice are given random interval (RI) schedule, mice learn that the reward is delivered in a random time interval independent of nose-poke. Mice learn a dissociation between nose-poke (action) and reward (outcome). Through repeated exposure to a different RI schedule (e.g., RI 30, RI60), mice learn that a regular nose-poke is the best policy to maximally obtain the reward, equivalent to model-free (MF) RL. (C) To test whether mice exhibit goal-directed or habitual behavior after experiencing the RR and RI schedules, mice are transferred to a home cage and given excessive amounts of reward or water (or food chow, nothing). After experiencing effortless reward, mice devalued the reward and lost motivation to nose-poke (action) while non-rewarded mice in the home cage are still eager to nose-poke in an operant chamber. (D) Thus, right after reward devaluation in the home cage, when mice are exposed to extinction in an operant chamber, mice trained in RR inhibit nose-poking (action). However, mice are trained in RI to keep nose-poking due to a strong S-R association (contingency) or weak A-O association (contingency).

3. Goal-directed and habitual reward-seeking to compulsive addictive behaviors

Addiction has been viewed as maladaptive and habitual reward-seeking behavior (Everitt and Robbins, 2016). Repeated drug use shifts from positive reinforcement-based drug-taking behaviors to negative reinforcement-driven drug-taking, which generates a vicious cycle of relapse (Koob, 2020; Koob and Volkow, 2010). The negative reinforcement stage values alleviation of negative affects through drug use rather than drug itself for previous pleasure seeking, and thereby it drives compulsive and habitual-seeking behaviors (Gillan et al., 2014; Voon et al., 2015). This stage seems to overlap with the extreme S-R contingency regarding the deficit of behavioral inhibition or uncontrolled drug-seeking/taking (Luscher et al., 2020). As compulsive drug-taking is defined as “uncontrolled use despite negative consequences” (Luscher et al., 2020), habitual drug-seeking (less controllable and higher S-R contingency) is an intermediate state between goal-directed (controllable and higher A-O contingency) and compulsive drug-seeking/taking behaviors (Fig. 1 and 2). What can be utilized as negative consequences in animal studies? For alcohol, resilient drinking of quinine adulteration is commonly used (Lesscher and Vanderschuren, 2012; Vengeliene et al., 2009). Collectively, the persistent drug-seeking/taking behaviors against harmful punishment merge into the generalized two other criteria of addiction-like behavior in animal studies, consisting of deficit of behavioral inhibition, high motivation for the drug which can be measured by progressive ratio schedule (Deroche-Gamonet et al., 2004; Luscher et al., 2020). Interestingly, enhanced orbitofrontal cortex (OFC)-dorsal striatum activities (especially DMS) are likely responsible for compulsive drug-taking for stimulants (Hu et al., 2019). Since the OFC is known to regulate goal-directed behavior and value-guided decisions (Hirokawa et al., 2019), compulsive drug-taking can be viewed as excessive goal-directed drug choice under the negative effects (Hogarth, 2020). Context-dependent circuit studies will precisely illustrate the nature of addiction and AUD (Vandaele and Ahmed, 2021).

4. Distinguishable expression and roles of dopamine and adenosine receptors in the Striatal MSNs and AUD

Using bacterial artificial chromosome (BAC) transgenic mice expressing enhanced green fluorescent protein (EGFP), the gene expression profile of the two distinct striatal neurons (dMSNs and iMSNs) are now well-characterized (Gong et al., 2003; Surmeier et al., 2007). Many serial studies revealed that dMSNs express dopamine D1 receptors (D1R) or adenosine A1 receptors (A1R) while iMSNs contain dopamine D2 receptors (D2R) or adenosine A2A receptors (A2AR), which comprise approximately 90~95% of striatal neurons (Humphries and Prescott, 2010; Lobo et al., 2006) (Fig. 3). Interestingly, D1R are coupled to mainly stimulatory G-protein (Gs) and subsequently increase cAMP through adenylyl cyclase when activated, whereas A1R is coupled to inhibitory G-protein (Gi) in the dMSNs and has an opposite effect on the downstream signaling cascade. Furthermore, D1R and A1R are co-expressed in dMSNs and can form D1R-A1R heterodimers (Ferre et al., 2010; Fuxe et al., 2007; Nam et al., 2013a). Because of the opposite receptor-mediated signaling between D1R and A1R, pharmacological activation of A1R dampens the D1R function (Fuxe et al., 2007). In contrast, A1R antagonists increase D1R-mediated signaling (Fuxe et al., 2007). Similarly, D2R and A2AR which are coupled to Gi and Gs proteins respectively, form a D2R-A2AR heterodimer and have an antagonistic effect on iMSNs (Ferre et al., 2010; Ferre et al., 2008; Ferre et al., 1993). Specifically, A2AR agonists inhibit D2R activation, minimizing Gi signaling and promoting the activation of striatal neurons (Ferre et al., 1993; Stromberg et al., 2000). Conversely, D2R signaling via Gi directly counteracts A2AR-mediated increases in cAMP driven by Gs activation in the striatal cells (Hillion et al., 2002). Interestingly, a recent study demonstrated that low D2R expression leads to compulsive-like ethanol drinking (Bocarsly et al., 2019). Selective loss of D2Rs on iMSNs not only promotes ethanol-induced stimulation via D1 activation but also creates resilience to ethanol sedation. Those mice with low striatal D2R also showed more escalation of ethanol intake in an operant paradigm (Bocarsly et al., 2019). Thus, the precise phase of adenosinergic and dopaminergic signaling may be necessary for the precise response to either transmitter, leading to the changes in downstream signaling events and consequent behaviors (Morita and Kawaguchi, 2018).

Fig. 3.

Illustration of cortico-striatal and substantia nigra-striatal circuits in the direct medium spiny neurons (dMSNs) and indirect medium spiny neurons (iMSNs). Astrocyte-neuron interactions play a critical role in fine-tuning the dMSN and iMSN and sorting out the signals. dMSNs express dopamine D1R and adenosine A1R, while iMSNs express mainly dopamine D2 and adenosine A2AR. In the two distinct MSNs, D1R and A1R, or D2R and A2AR interact through direct receptor-receptor interaction and/or downstream signaling pathways.

In a simplified perspective, activation of D1R or A1R expressing GABAergic dMSNs encodes “GO” signaling through dMSNs → GPi circuits while D2R or A2AR expressing GABAergic iMSNs has an inhibitory control (NO GO or STOP) through iMSNs → GPe → STN → GPi (Graybiel, 2008). Having considered the anatomical distinction between DMS and DLS, and the functional difference between dMSNs and iMSNs, the dysregulation or disruption of harmonized signaling of the striatopallidal circuits is critical for drug-seeking/taking behaviors.

In AUD, the DMS and DLS’s distinct roles have been elegantly examined (Corbit et al., 2012). The chemogenetic activation of dMSNs in the DMS mimics the strengthening of glutamatergic neurotransmission and promotes ethanol drinking (Cheng et al., 2017; Wang et al., 2015). Considering the opposite role of dMSNs and iMSNs, inhibition of iMSNs may increase ethanol consumption. Consistently, our previous studies demonstrated that pharmacological inhibition of DMS A2AR increases goal-directed ethanol-seeking behavior and ethanol consumption (Nam et al., 2013b). Recently, we revealed that pharmacological activation of A2AR or optogenetic activation of iMSNs in the DMS decreases ethanol-seeking behaviors (Hong et al., 2019), further validating the opposite role of dMSNs and IMSNs in the DMS. However, the correlation of behavioral transition between goal-directed and habitual ethanol-seeking and circuit changes between DMS and DLS is not fully understood in AUD and other addictions.

5. Astrocyte-neuron interaction in the striatum and AUD

Synaptic transmission is regulated by a “tripartite synapse” consisting of a presynaptic neuron, postsynaptic neuron, and perisynaptic astrocyte (Araque et al., 2014; Araque et al., 1999; Haydon and Carmignoto, 2006) (Fig. 3). Astrocytes play an essential role in energy metabolism, synaptic signaling, neurotransmitter clearance, and ionic homeostasis (Khakh, 2019). Recently, several studies have demonstrated that astrocyte dysregulation is associated with neuropsychiatric disorders including substance use disorder (SUD) (Corkrum et al., 2020; Scofield and Kalivas, 2014) and AUD (Ayers-Ringler et al., 2016; Erickson et al., 2021; Erickson et al., 2018; Lindberg et al., 2018). Interestingly, extrasynaptic neurotransmitters can trigger astrocyte Ca²⁺ signaling (Charles et al., 1991; Dani et al., 1992), and reciprocally astrocyte Ca²⁺ signals regulate the function of neural circuits (Smith, 1994), especially through gliotransmitters (Araque et al., 2014; Jiang et al., 2016; Wojtowicz et al., 2013), which illustrates the dynamic astrocyte-neuron interaction. Indeed, astrocytic Ca²⁺-dependent signaling in the dorsal striatum has been reported to alter the MSNs’ activities and animal behaviors (Yu et al., 2018). Several studies show that the activation of Gq-GPCRs induces calcium influx in astrocytes (Bazargani and Attwell, 2016; Chai et al., 2017; Durkee et al., 2019), thereby facilitating astrocyte-neuron interaction and resulting in significant behavioral changes (Bull et al., 2014; Chen et al., 2016; Reissner and Pletnikov, 2020). Recently, we demonstrated that chemogenetic activation of astrocytes in DMS differentially regulated MSNs’ activities and shifted from habitual to goal-directed reward-seeking behavior, which reveals the potential role of astrocyte-regulated adenosine signaling in reward-(Groman, 2020; Kang and Choi, 2021; Kang et al., 2020) and alcohol-seeking behaviors (Hong et al., 2020). Also, chemogenetic activation of prefrontal cortical neurons increases alcohol-drinking in alcohol-naïve mice, but not in mice pre-exposed to alcohol in an adenosine-dependent manner (Erickson et al., 2021; Moffat and Ron, 2021), validating the importance of astrocyte-neuron interaction and adenosinergic signaling. Despite these findings, it has not been fully answered how astrocytic activation may distinctly regulate neuronal activities in the DMS and DLS, encoding goal-directed and habitual reward/ethanol-seeking behaviors. Given the unique characters of striatal astrocytes such as the paradoxical increase of intracellular Ca²⁺ through even Gi signaling-driven activation (Nagai et al., 2019), a further detailed investigation is required on how DMS and DLS astrocytes regulate MSN activities.

6. Globus pallidus and astrocytes

As shown in Fig. 1, the main output of MSNs is the internal or external GP (GPi and GPe), mainly composed of GABAergic neurons (Shink and Smith, 1995). Of note, iMSNs are principally projecting to the GPe, while dMSNs form circuits with both the GPi and SNr. In movement control, the GPe is an essential nuclei of inhibitory indirect pathway (Hegeman et al., 2016). However, GPe is also critical for reward prediction (Arkadir et al., 2004), learning (Schroll et al., 2015), and sleep (Qiu et al., 2016).

Approximately two-thirds of striatal inputs of the GPe are from iMSNs and one-third from dMSNs (Kawaguchi et al., 1990). About 80% of inputs to the GPe are inhibitory GABAergic MSNs and 20% of synapses are excitatory inputs from the STN, cerebral cortex, and thalamus (Kita, 2007). Thus, the GPe does not simply project to STN and subsequently innervating the GPi, but also receives glutamatergic inputs from the STN (Kita, 2007). However, the STN → GPe circuit’s role remains unknown although the numbers of synapses of MSNs → GPe are well characterized in the rat brain (Oorschot, 1996). Furthermore, the role of cortico → STN circuit (a.k.a., hyper-direct circuit) has not been fully studied in addictive behaviors (Lovinger and Gremel, 2021; Mathai and Smith, 2011) (Fig. 4).

Fig. 4.

Dorsal striatum to pallidal circuits. Two main dorsal striatal regions, dorsal medial striatum (DMS) and dorsolateral striatum (DLS) project to the external part of globus pallidus (GPe), especially indirect MSNs are mainly projecting to the GPe. GPe containing three main GABAergic neurons. Foxp2 and Npas positive arkypallidal neurons project back to the dorsal striatum. Lhx6 positive prototype neurons evenly project to substantia nigra (SNc) and subthalamic nucleus (STN). PV positive neurons project to the STN and substantia nigra pars reticulata (Pf). STN glutamatergic neurons are projecting to GPe as well. Red arrow, glutamatergic neurons; Blue arrow, GABAergic neurons.

The GPe contains several subtypes of GABAergic neurons categorized by its localization (medial and lateral), unique gene expression (PV, Lhx6, Npas1, or Foxp2), and firing patterns (slow or fast-spiking, or low or high-frequency) (Abrahao and Lovinger, 2018; Hernandez et al., 2015; Mastro et al., 2014). Although multiple inhibitory neuronal subtypes in the GPe are innervated from the dorsal striatum (Hernandez et al., 2015), it is unknown how the individual GPe cell types respond to iMSNs’ activities of the DMS and DLS and how GPe cellular activities change during operant alcohol-seeking behaviors (Lovinger and Gremel, 2021).

Two largely different prototypical GPe cell types are classified as Lhx6-positive slow-firing and PV-positive fast-spiking neurons (Abrahao and Lovinger, 2018). Although overlapping, each subpopulation of GPe GABAergic neurons forms a distinct circuit with different brain regions (Fig. 4). For example, while the major output pathways of the GPe neurons are STN and GPi (Mastro et al., 2014), the Lhx6- and PV-positive GPe neurons distinctly project to SNc and the parafascicular nucleus of the thalamus (Pf), respectively (Mastro et al., 2014). Also, slow-firing arkypallidal neurons in the GPe are known to express Foxp2 (Abrahao and Lovinger, 2018) or Npas (Glajch et al., 2016; Hernandez et al., 2015). Noteworthy, slow-firing arkypallidal neurons in the GPe project back to the dorsal striatum (DS) (Glajch et al., 2016; Hernandez et al., 2015). These feedback circuits (GPe → DMS or DLS) potentiate the “NO-GO” signal to the striatum (Mallet et al., 2016). Interestingly, a clinical neuroimaging study suggests that alcohol dampens GPe neuronal activity (Nikolaou et al., 2013). Recently, in mice, alcohol decreases the firing of low-frequency GPe neurons, specifically Lhx6 and Npas expressing neurons, without altering the firing of the high-frequency neurons (Abrahao et al., 2017). Thus, alcohol-induced dampening of the Npas neurons may result in decreasing “NO-GO” or increase “GO” signal at least transiently with low-moderate doses of alcohol (Abrahao et al., 2017). Interestingly, astrocytes are highly enriched in the GPe (Cui et al., 2016; Dervan et al., 2004; Lange et al., 1976; Salvesen et al., 2015). Considering the feedforward and feedback circuits of GPe neurons, the role of GPe astrocytes may be critical for fine regulation of behavioral flexibility. However, further studies are warranted to investigate how two distinct striatal regions (DMS and DLS)-specific circuits to the GPe, synergistically with the GPe astrocytes, affects the GPe neuronal activities, leading to the changes in alcohol-seeking behaviors.

7. Computation modeling of goal-directed and habitual behaviors and striatopallidal circuits in addictive behaviors

Considering the complexity of neural networks and information processing, computational modeling for reward-seeking and addictive behaviors becomes a critical tool (Ferrante et al., 2019; Friston et al., 2017; Redish, 2004; Redish et al., 2016). In both preclinical and clinical studies, DMS-based goal-directed and DLS-habitual circuits are emerging topics in the field of computational neuroscience (Baladron and Hamker, 2020; Keramati et al., 2016). To employ a computational approach, we have to surmise that subjects (humans and animals) are comprised of hierarchic circuits for information processing (Kamali Sarvestani et al., 2011). Especially, it is noteworthy that the “learning” process is a critical component to maximize the chance to procure the reward (desired outcome) in the context of behavioral economics (internal cost-benefit analysis). Thus, in a “reinforcement learning (RL)” model (Dasgupta et al., 2014), the striatopallidal circuits are the subject for computational analysis, especially focusing on the role of dopamine signaling (Dabney et al., 2020; Keiflin and Janak, 2015). However, computational neuroscience approaches should consider beyond dopamine-centered neurobiology since other neurotransmitters contribute to reinforcement learning (Morita and Kawaguchi, 2018).

In an actor-critic model (Fig. 5a and b), goal-directed processes adjust actions based on updated reward prediction to obtain the goal (outcome), which is applicable with the “model-based (MB)” strategy while the habitual processes (learning) use generalized responses with a previously rewarded stimulus without flexible modification of the action, or “model-free (MF)” strategy (Daw et al., 2005; Groman et al., 2019; van Steenbergen et al., 2017) (Fig. 5a). Several recent studies indicate that dorsal striatum-GP circuits are functioning as the actor while ventral striatum-VP circuits estimate the value as the critic in an actor-critic model (Dunovan and Verstynen, 2016; Takahashi et al., 2008) (Fig. 5b). This dichotomic approach has been elegantly used for fMRI-based goal-directed (MB) and habitual (MF) decisions, especially in human studies (Daw et al., 2011; Glascher et al., 2010; Huang et al., 2020; Lee et al., 2014). In addition to the existing MB model, two MF models, motor MF and stimulus MF models have been suggested to distinguish between compulsive and impulsive-related disorders (Deserno and Hauser, 2020). In particular, cell-specific molecular dynamics with high-resolution of temporal neural recording for a state (or an activity)-dependent manner will be useful to understand neural-based goal-directed and habitual behaviors rather than region-specific simplified denotation (e.g., iMSNs = inhibitory GABAergic neurons receiving cortical glutamatergic inputs, or GPe = inhibitory GABAergic system receiving iMSNs GABAergic inputs). Recently, in the OpAL (opponent actor learning) model, the actor has two components D1R/A1R-expressing dMSNs and D2R/A2AR expressing iMSNs (Morita and Kawaguchi, 2018). Several studies showed the differential regulation of dMSNs and iMSNs in reward-seeking (Shin et al., 2018) and action selection (Markowitz et al., 2018). As we described in section 4, in the OpAL model, the antagonistic and synergistic relationship between dopamine and adenosine receptors between dMSNs and iMSNs or between DMS and DLS will provide comprehensive features if proper mathematical and computational modeling can be employed. Furthermore, we revealed that DMS astrocyte activation differentially regulates postsynaptic currents of iMSNs in an adenosine signaling-dependent manner (Kang et al., 2020). Considering the fast responsiveness of astrocytes to various stimulation or environmental changes (Khakh, 2019; Khakh and Deneen, 2019), astrocyte activity will be a critical modifier in the OpAL model (Kang and Choi, 2021). With the advent of innovative neuro-modulation and recording methods, including optogenetics, chemogenetics, fiber-photometry and endo-microscopy, and real-time and digitalized behavioral monitoring systems (Baker et al., 2020; Geuther et al., 2019; Markowitz et al., 2018), additional studies will shed light on the full-scope of reward-seeking/taking behaviors (Mujica-Parodi and Strey, 2020).

Fig. 5.

Computational modeling of striatopallidal circuits for addictive behaviors. (A) Actor-critic model of reinforcement learning in computation. The agent selects action based on policy and value information, which will be revised and optimized depending on the environment and state. (B) Dorsal striatum (DMS/DLS)-pallidal circuits with dopaminergic inputs from SNc function as an actor while nucleus accumbal (NAc)-ventral pallidum (VP) circuits with dopaminergic inputs from VTA act like a critic.

8. Discussion and Conclusion

We reviewed the essential role of striatopallidal circuits focusing on two different MSNs in the DMS and DLS on goal-directed and habitual reward (alcohol)-seeking/taking behaviors. Furthermore, astrocytes in this circuit play a critical role in regulating neural activities or possibly a specific circuit. Emerging computation modeling with an activity-dependent neural or astrocyte recording approach will elucidate precise mapping and circuits associated with behaviors (Markowitz and Datta, 2020; Markowitz et al., 2018). With the aid of machine-learning and prediction modeling, additional studies will improve our understanding of context- and cell-specific AUD-related behaviors.

From a neuropsychiatric perspective, AUD is in part attributed to circuitry dysfunction. Restoring the dysfunctional circuits associated with compulsive and severe alcohol-seeking behaviors may be a daunting task (Madeo and Bonci, 2018). However, harnessed with advanced brain activity monitoring and high-resolution brain imaging technologies such as fMRI and PET, several brain stimulation or modulation approaches are now being considered as electroceuticals for reverse-engineering brain dysfunction, including AUD (Diana et al., 2017; Ekhtiari et al., 2019; Famm et al., 2013; Javitt et al., 2020). Besides, human-based reverse translational studies (Venniro et al., 2020) will guide us toward clinically useful research.

Highlights.

Striatopallidal circuits are critical for goal-directed and habitual reward-seeking behaviors

Direct- and indirect-pathway medium spiny neurons regulate reward-seeking behaviors.

Astrocyte-neuron interaction in the dorsal striatum is essential for alcohol use disorder.