Astrocyte activities in the external globus pallidus regulate action-selection strategies in reward-seeking behaviors

Abstract

An imbalance in goal-directed and habitual behavioral control is a hallmark of decision-making–related disorders, including addiction. Although external globus pallidus (GPe) is critical for action selection, which harbors enriched astrocytes, the role of GPe astrocytes involved in action-selection strategies remained unknown. Using in vivo calcium signaling with fiber photometry, we found substantially attenuated GPe astrocytic activity during habitual learning compared to goal-directed learning. The support vector machine analysis predicted the behavioral outcomes. Chemogenetic activation of the astrocytes or inhibition of GPe pan-neuronal activities facilitates the transition from habit to goal-directed reward-seeking behavior. Next, we found increased astrocyte-specific GABA (γ-aminobutyric acid) transporter type 3 (GAT3) messenger RNA expression during habit learning. Notably, the pharmacological inhibition of GAT3 occluded astrocyte activation–induced transition from habitual to goal-directed behavior. On the other hand, attentional stimuli shifted the habit to goal-directed behaviors. Our findings suggest that the GPe astrocytes regulate the action selection strategy and behavioral flexibility.

Astrocytic dynamics in the external globus pallidus are engaged with goal-directed and habitual-seeking behaviors.

INTRODUCTION

Goal-directed and habitual behaviors depend on whether an individual changes or adheres to an action strategy according to environmental information, which mainly changes the reward value. Upon repetition of goal-directed behavior, brain circuits routinely process information as a habit, which is influenced less by executive functions and is more cost-effective in procuring the desired outcome serving an energy-saving autopilot system (1). However, maladaptive habits with recurring and detrimental behavioral consequences and imbalanced controls of action strategy are a hallmark of decision-making disorders, such as obsessive-compulsive disorder and addiction (2, 3).

In the striatopallidal circuits, the external globus pallidus (GPe) has been considered an integrative hub for the behavioral flexibility in reward-related behaviors because it coordinates the top-down neurotransmission from the two distinctive dorsal striatum regions, the dorsomedial and dorsolateral striatum (DMS and DLS). These regions are the major neural substrates for goal-directed and habitual reward-seeking behaviors, respectively (4–7). The GPe redistributes the dorsal striatal inhibitory Gamma-Aminobutyric Acid (GABA)ergic tone and signals through the fast-spiking prototypical and slow-spiking arkypallidal neurons in the GPe to the downstream brain regions (8–10). Recent studies demonstrate that the GPe contains abundant astrocytes, distinguishable from the neighboring brain regions. As expected, GPe astrocytes directly modulate the neuronal activities in the GPe, leading to consequent behavioral changes (11, 12). However, the role of GPe astrocytes in goal-directed and habitual action strategies is mainly unknown. First, we used the two operant tasks upon effort- or time-based reward (10 μl of 20% sucrose solution) delivery to establish the dominance of goal-directed behavior or habit. Then, we used in vivo calcium signaling to probe the temporal dynamics of GPe astrocytes during goal-directed and habit learning. Subsequently, using a support vector machine (SVM), we tested whether the type of operant tasks is predictable with GPe astrocytic calcium dynamics. Using chemogenetics, we investigated the inverse relationship between astrocytes and neuronal activities in the GPe. We also examined the astrocyte-specific γ-aminobutyric acid (GABA) transporter type 3 (GAT3) expression during habit formation and the effect of the GAT3 inhibition on astrocyte activity and behavior. Last, we determined whether enhancing the calcium signaling in GPe astrocytes and attentional stimulus increase behavioral flexibility.

RESULTS

Disruption of A-O contingency shapes habitual seeking behaviors

Habitual behavior is insensitive to changes in action-outcome (A-O) reward contingencies, unlike goal-directed behavior (4). However, rather than adhering to this dichotomous classification, naturalistic behaviors exist on a spectrum. Thus, we evaluated the relationship between A-O contingency and the degree of habit. First, we trained mice to acquire action (nose-poking)–outcome (reward in the magazine) contingency in an operant chamber through four sessions of FR1 (fixed ratio 1). We used 20% sucrose solution (10 μl) as a reward because it has strong rewarding effects in mice, especially in operant conditioning (13–15). The two randomly assigned groups showed similar operant behaviors [fig. S1A, nose-poke rate; fig. S1B, discrimination ratio; fig. S1C, session duration; and fig. S1D, latency to the magazine; n = 8 for the random ratio (RR) group and n = 7 for the random interval (RI) group], and the repeated training with the FR1 tasks facilitated the collection of rewards after nose poking (fig. S1D, first versus fourth session; n = 8 for the RR group and n = 7 for the RI group).

After completion of conditioning under the FR1 task, mice conducted the RR and RI tasks. These two tasks differ in the rules of action necessary for reward acquisition. The accumulated actions yield a reward in the RR task. In contrast, the flow of time unrelated to the number of actions produces a reward (Fig. 1A). The session length of the two task schedules was the same. We determined what kind of action strategy mice have through the extinction test following reward devaluation (Fig. 1B). After the FR1 task, the RR and RI schedules showed a distinct pattern of nose-poking behavior (Fig. 1C). At the last session of each schedule, lower A-O contingency appeared during the RI120 task compared to the RR20 task (Fig. 1D). As expected, the RR and RI schedules led to the development goal-directed and habitual behavioral phenotypes, respectively (Fig. 1E; n = 8 for RR and n = 7 for RI). Furthermore, A-O contingencies at the last session of the RR20 and RI120 tasks were inversely correlated with the habit index measured in the extinction test (Fig. 1F; n = 15). Thus, these results indicate that disruption of A-O contingency may drive a more fixed and habitual action strategy to procure the reward.

Fig. 1. Reduced A-O contingency after habit learning.

(A) Task schemes for operant conditioning. (B) Experimental schedules for operant conditioning and evaluation test. (C) Nose-poking behaviors from the FR1 to the RR20 or RI120 task. (D) A-O contingency in the last sessions of RR20 and RI120 tasks. (E) Nose-poke changes in the outcome evaluation tests after the RR or RI task. (F) Correlation between A-O contingency and habit index. Data represented mean ± SEM; (F) shadow indicated 95% confidence bands of the best-fit line; (C) to (F) n = 8 for the RR group and n = 7 for the RI group. *P < 0.05 comparing each group. (D) Two-tailed Mann-Whitney test. (E) Wilcoxon matched-pairs signed-rank test. (F) Spearman correlation analysis. A-O, action-outcome; DV, devalued; FR1, fixed ratio 1; MT, magazine training; RI, random interval; RR, random ratio; V, valued. See table S3 for full statistical information.

GPe astrocytes show distinct temporal dynamics during learning for goal-directed behavior and habit

Next, we used in vivo fiber photometry–based calcium signaling to identify how the GPe astrocytes encode the action strategy in the task with distinct A-O contingencies. In particular, we targeted astrocytes because the GPe harbors abundant astrocytes (11, 12), unlike other neighboring brain areas (fig. S2A). Thus, we recorded astrocytic calcium dynamics during the last sessions of the RR and RI tasks, where the types of action strategy are correlated with GPe astrocyte activities (fig. S2B). As expected, mice behaved along with the rules of both tasks (fig. S2, C to F; n = 282 for the RR group and n = 66 for the RI group).

We analyzed operant behaviors with six action events: a nose poke that provided a reward (NP R+), a nose poke that did not provide a reward (NP R−), magazine entry where there was a reward (M_entry R+), magazine entry where there was no reward (M_entry R−), magazine exit after reward consumption (M_exit R+), and exit from the magazine where there was no reward (M_exit R−) (Fig. 2A). Then, we analyzed mouse behaviors and astrocytic calcium signaling from 8 s preceding and 8 s following (total of 16 s, 480 frames) the action event. All datasets from one mouse were averaged and expressed as n = 1. In addition, a bin was created with 2 s (60 frames) as one block to analyze nose-poking behavior across time (Fig. 2A).

Fig. 2. Astrocyte dynamics in the GPe during goal-directed and habitual reinforcement learning.

(A) Data collection method. We collected behavior and fiber-photometry data from 8 s before each behavior event to 8 s after each behavior event. At the action events [nose poke (B to D), magazine entry (E to G), and magazine exit (H to J)], (B), (E), and (H) temporal fluorescence changes, (C), (F), and (I) averaged fluorescence intensity, and (D), (G), and (J) difference of averaged fluorescence during the RR and RI tasks. (B), (E), and (H) The gray zones were used to calculate averaged fluorescence intensity. (C), (F), and (I) Comparison of the average fluorescence intensity in the previous 2 s (before) and the next 2 s (after) the action events. (D), (G), and (J) Difference of averaged fluorescence resulting from subtracting the “before” from the “after” averaged fluorescence intensity. Data represented mean ± SEM; n = 6 per group. *P < 0.05 comparing each group. (B), (E), and (H) Two-way repeated-measures ANOVA, followed by Bonferroni’s multiple comparisons tests; (C), (F), and (I) two-tailed Wilcoxon matched-pairs signed-rank test; (D), (G), and (J) two-way ANOVA, followed by Tukey’s multiple comparisons tests. M_entry, magazine entry; M_exit, magazine exit; NP, nose poke; RI, random interval; RR, random ratio; R+, with reward; R−, without reward. See table S3 for full statistical information.

First, we confirmed that mice trained with the RR20 or RI120 task performed distinct reward-seeking (nose-poking) and reward-taking (collecting a reward at the magazine) behaviors at the last RR and RI sessions (fig. S3, A to F, top, for reward-seeking behavior; fig. S3, A to F, bottom, for reward-taking behavior; n = 6 per group). Moreover, the RI task showed a reduced seeking-taking association compared to the RR task in all the behavioral events except for M_entry (R−) (fig. S3G). In addition, this phenomenon was seen throughout the session (fig. S3H). Moreover, the seeking-taking association was positively correlated with A-O contingency (fig. S3I; n = 12). These results indicate that reward-seeking behavior may not be for collecting the actual reward when mice have disrupted A-O contingency.

Along with these distinct behavioral performances, mice showed lower astrocytic calcium dynamics during the last session of the RI120 task than the RR20 task [Fig. 2B, left, for NP (R+); Fig. 2B, right, for NP (R−); Fig. 2E, left, for M_entry (R+); Fig. 2E, right, for M_entry (R−); Fig. 2H, left, for M_exit (R+); and Fig. 2H, right, for M_exit (R−); n = 6 per group]. Then, we narrowed down the time range (2 s before and after the action event; gray zone of Fig. 2, B, E, and H) to see event-dependent dynamic changes. We also examined the substantial difference when we altered the time ranges (table S1). Astrocytic dynamics did not change when mice sought the reward (Fig. 2C; n = 6 per group), and there was no dynamic difference between tasks (Fig. 2D; n = 6 per group). However, astrocytic dynamics were reduced and increased when mice went into the magazine (Fig. 2F; n = 6 per group) and went out (Fig. 2I; n = 6 per group), respectively. In the RI task, the differences in dynamic change were smaller than in the RR task (Fig. 2, G and J; n = 6 per group). These results suggest that GPe astrocytic dynamics are closely related to reward-taking behaviors, and GPe astrocytic dynamics become silent while learning habits.

Predictable action strategies by GPe astrocytic dynamics

Next, we sought to examine whether we could predict the behavior tasks mice performed using astrocytic dynamic data. We used an SVM, an optimal neural network with minimal risk of overfitting (16, 17). The time window was set to 1 s before and after the action events to focus more on the event-dependent GPe astrocytic dynamics. Overall, the SVM successfully predicted the types of behavioral tasks (Fig. 3A). Even with different time ranges, the SVM still predicted behavioral tasks well (table S2). Moreover, the SVM predicted behavioral tasks better when using the astrocytic dynamics data during M_exit than during other action events (Fig. 3B; n = 1600). These results demonstrate that GPe astrocytic dynamics may be a reliable factor for predicting action strategies in operant behaviors. In addition, averaged receiver operator characteristic (ROC) curves predicted reward experience around behavioral events (nose poke, magazine entry, and magazine exit) during the RR20 and RI120 tasks (fig. S4).

Fig. 3. Predictable behavioral tasks by using GPe astrocyte dynamics of behavioral events.

(A) Averaged ROC curves for predicting behavioral tasks around behavioral events (nose poke, magazine entry, and magazine exit). (B) Comparison of accuracy, sensitivity, and specificity among the behavioral events. Data represented mean ± SEM; n = 1600 per group. *P < 0.05 comparing nose-poke and magazine exit groups. Kruskal-Wallis test followed by Dunn’s multiple comparisons test. See table S3 for full statistical information. AUC, area under the curve.

Chemogenetic activation of GPe astrocytes decreases habit behaviors and increases goal-directed reward-seeking in operant conditioning

GPe astrocytic calcium dynamics, a reliable predictor of operant behaviors, were notably attenuated during habit learning. Therefore, we asked whether changes in astrocytic dynamics can modify acquired action strategies. Because astrocytes are known to have no active ionotropic receptors (ligand-gated ion channels) (18, 19), we used the chemogenetic approach instead of optogenetic stimulation. First, we injected a GFAP promoter–driven human M3 muscarinic receptor (hM3Dq)–expressing virus into the GPe and injected the GFAP promoter–driven mCherry-expressing virus as the control group (Fig. 4A). Thus, we prepared three control groups (vehicle in mCherry mice, C21 in mCherry mice, and the vehicle in hM3Dq mice). Four to five weeks after the virus delivery, we confirmed viral expression (Fig. 4A and fig. S5) and validated whether C21 treatment increases intracellular calcium dynamics in GPe astrocytes using in vivo calcium signaling. Thirty minutes after C21 [3 mg/kg, intraperitoneal (ip)] treatment, we found that C21 increased the averaged fluorescence intensity in freely moving mice with hM3Dq expression but not in mCherry mice (Fig. 4, B and C; n = 5 per group). On the other hand, we confirmed that C21 (3 mg/kg, ip) did not alter locomotor activity in the open field (Fig. 4D; n = 7 for the hM3Dq group and n = 6 for the mCherry group), although GPe governs motor-related behaviors (20, 21).

Fig. 4. Chemogenetic activation of the GPe astrocyte activity on goal-directed (RR) and habit (RI) behaviors.

(A) Histological validation of viral delivery of hM3Dq in GPe astrocytes. (B and C) Functional validation of GPe astrocytic hM3Dq. (D) No change in locomotor activity. (E) Experimental scheme for operant conditioning. (F) Habit index after RI or RR task. (G) Habit index of the vehicle (Veh) or compound 21 (3 mg/kg, ip; C21) treatment group after RR or RI task. Nose poke between valued and devalued states after the (H) RR or (I) RI task. Data represented mean ± SEM; (C) n = 5 per group, (D) n = 5 per group, (F) to (I) n = 14 for the RR group and n = 21 for the RI group. *P < 0.05 comparing each group. (C) and (F) Two-tailed Mann-Whitney test; (G) and (I), two-tailed Wilcoxon matched-pairs signed-rank test. C21, compound 21; DV, devalued; FR1, fixed ratio 1; RR, random ratio; V, valued. See table S3 for full statistical information.

Next, we investigated whether chemogenetic astrocytic activation alters operant behaviors. We treated vehicle or C21 (3 mg/kg, ip) 30 min before the extinction test (Fig. 4E) in mice that were trained with the RR and RI tasks. As expected, the RI task yielded the dominance of habitual seeking (100% of the index indicates intact habit) (Fig. 4F; n = 14 for the RR group and n = 30 for the RI group). While C21 treatment did not alter the habit index after the RR task, it reduced the habit index after the RI task in GPe hM3Dq-expressing mice compared to vehicle treatment (Fig. 4G; n = 14 for the RR group and n = 21 for the RI group). Vehicle and C21 treatments in hM3Dq mice exhibited similar patterns of nose-poking behaviors between the valued and devalued states after the RR schedule (Fig. 4H; n = 14 per group). On the other hand, C21 reduced nose-poking behaviors in reward devaluation, which was not observed in vehicle-treated hM3Dq mice after the RI schedule (Fig. 4I; n = 21 per group). However, C21 did not change nose-poking behaviors in mCherry mice even after the RI task (figs. S6, A and B; n = 8 per group). We also found similar locomotor activities in an open-field task when mice were treated with 1, 2, or 3 mg/kg of C21 (fig. S7, A to D). Therefore, these results indicate that chemogenetic activation of the Gq pathway in GPe astrocytes dampens habitual seeking behavior.

Chemogenetic GPe astrocyte activation dampens GPe neuronal firing

Activation of the astrocytic Gq pathway induces calcium influx, the facilitation of the astrocyte-neuron interaction, and temporal activity changes in the surrounding neurons and their circuits (22–24). Previous research reported that the neuronal responses to astrocytic activation depend on their neuronal characteristics (25–27). Thus, we examined how the astrocyte activation via modified Gq-pathway G protein–coupled receptors (GPCRs) (chemogenetic hM3Dq) affects neighbor GPe neurons. To preserve the cellular structure of the recorded neurons and surrounding astrocytes expressing chemogenetic GPCRs, we performed in vivo electrophysiological recordings with the anesthetized mice expressing GFAP promoter-driven hM3Dq in the GPe (Fig. 5A). The expression of hM3Dq did not affect the spontaneous firing rates of GPe neurons (fig. S8, A and B; n = 49 for the mCherry group and n = 46 for the hM3Dq group).

Fig. 5. Effects of chemogenetic activation on GPe neuronal activities.

(A) Experimental scheme and protein expressions of GFAP promoter–driven mCherry (red) and NeuN (green). (B) Different firing patterns between before and after in the hM3Dq group. (C) Relative firing changes along with C21 treatment (3 mg/kg, ip) in mice that expressed non-hM3Dq (as the mCherry group) and hM3Dq in the GPe. (D) Different distributions of relative firing changes after C21 treatment between the mCherry and hM3Dq groups. Correlations between GPe neuronal firing before C21 treatment and relative firing changes after C21 treatment in (E) the mCherry and (F) hM3Dq groups. Data represented mean ± SEM; n = 49 from 5 mice in the mCherry group, n = 46 from 5 mice in the hM3Dq group. *P < 0.05 comparing each group. (B) Two-way repeated-measures ANOVA, followed by Bonferroni’s multiple-comparisons test; (D) Kolmogorov-Smirnov test; (E) and (F) Spearman correlation analysis. C21, compound 21. See table S3 for full statistical information.

Notably, systemic application of C21 (3 mg/kg, ip) reduced the spontaneous firing rates of GPe neurons of mice expressing astrocytic hM3Dq, compared to those without hM3Dq expression (Fig. 5, B, C, and D; n = 49 for the mCherry group and n = 46 for the hM3Dq group) at the time point we assessed animal behaviors. Furthermore, this reduction of firing was generally observed independent of the basal firing frequencies of the neurons (Fig. 5, E and F; n = 49 for the mCherry group and n = 46 for the hM3Dq group), unlike the positive correlations between firing rates before and after C21 application in both groups (fig. S9, A and B; n = 49 for the mCherry group and n = 46 for the hM3Dq group). These results suggest that the Gq pathway activation in astrocytes reduces the overall GPe neuronal activities, regardless of their neuronal characteristics.

Chemogenetic inhibition of pan-GPe neurons increases goal-directed behavior

Next, we sought whether direct inhibition of GPe neurons exhibits similar behavior effects as the activation of GPe astrocytes in transitioning from habit to goal-directed behaviors. To inhibit the GPe neuronal activities, we microinjected the Adeno-associated viruses serotype 5 (AAV5) virus containing the pan-neuronal [calcium/calmodulin-dependent protein kinase type II subunit alpha (CaMKIIα)] promoter-driven Gi-linked chemogenetics in the GPe (AAV-hM4Di-mCherry). We confirmed the expression of hM4Di in GPe neurons using coimmunolabeling of mCherry expression with anti-NeuN (Fig. 6A). Systemic application of C21 (3 mg/kg, ip) reduced the spontaneous firing rate of GPe neurons in mice with hM4Di expression (Fig. 6, B to D; n = 21 for the Veh group and n = 20 for the C21 with hM4Di group). To examine the effect of neuronal activation on habitual learning in RI, we treated C21 30 min before extinction testing, where mice were tested in the same operant chamber for 10 min without reward (Fig. 6E). C21-treated mice exhibited goal-directed reward-seeking behaviors, while vehicle-treated animals showed habit behaviors (Fig. 6, F and G). This result confirms the inverse relationship between astrocytes and neuronal activities in the GPe; both require transitioning from habit to goal-directed behaviors or enhancing behavioral flexibility.

Fig. 6. Chemogenetic inhibition of GPe neuronal activity on habitual behavior.

(A) Histological validation of viral delivery of hM4Di in GPe astrocytes. (B) Different firing patterns in the hM4Di group before and after C21 treatment. (C) Relative firing changes along with vehicle or C21 treatment (3 mg/kg, ip) in mice that expressed hM4Di in the GPe. (D) Firing rate after Vehicle or C21 treatment in the GPe. (E) Experimental scheme for operant conditioning. Nose poke between valued and devalued states after vehicle (Veh) or compound 21 (3 mg/kg, ip; C21) treatment after the RI task. (F) Habit index after the RI task. Data represented mean ± SEM; (G) n = 10 for the RI group. *P < 0.05 comparing each group. Two-tailed Mann-Whitney test. C21, compound 21; DV, devalued; FR1, fixed ratio 1; RR, random ratio; V, valued. See table S3 for full statistical information.

Increased GAT3 expression during habit and GAT3 inhibition prevents the activated GPe astrocyte–induced increase of goal-directed seeking behaviors

Accumulative evidence demonstrates that astrocyte GAT3 regulates GPe astrocyte activities (28). Furthermore, dampened astrocyte activities using repetitive behaviors are associated with increased GAT3 expression and function (29). Thus, first, we sought to examine the GAT3 expression in the GPe during habit learning in RI schedule using quantitative real-time polymerase chain reaction (PCR) (Fig. 7, A and B). We found that GAT3 expression was increased in RI (habit) approximately five times compared to FR1 or RR (goal) schedules (Fig. 7, C and D), which is consistent with the previous finding (29). In two other brain regions involved in habit and goal-directed brain regions, DLS and DMS (27, 30), GAT3 mRNA expressions were similar in FR1, RR, and RI (fig. S10). Next, we investigated whether GAT3 inhibition occludes the effect of the GPe astrocyte–induced transition from habit to goal-directed reward-seeking behavior. SNAP5114 (SNAP), a GAT3-specific inhibitor (29, 31, 32), reversed the astrocyte activation (C21)–dependent reduction of GPe neuronal firing (Fig. 8, A to C; n = 16 for the hM3Dq + C21 group and n = 17 for the hM3Dq + C21 + SNAP group). Then, we microinjected SNAP into the bilateral GPe region where the hM3Dq were expressed in the astrocytes (Fig. 8D). We treated C21 (3 mg/kg, ip) 30 min before extinction testing in mice trained with the RI task. After 10 min, vehicle or SNAP (20 mM) was treated (Fig. 8E). As expected, C21 treatment of hM3Dq mice after the RI task reduced nose-poking behavior in reward devaluation, and habitual exploration was changed to goal-oriented, confirming our finding in Fig. 4I. However, when hM3Dq mice were treated with SNAP after C21 treatment, mice exhibited a similar nose-poking behavior between the valued and devalued conditions (Fig. 8F; n = 10 for the C21 + Vehicle group and n = 10 for the C21 + SNAP group), indicating that inhibition of GAT3 function nullified the astrocyte activation-induced transition from habit to goal-directed reward seeking. Without astrocyte activation (C21−), vehicle and SNAP treatment of hM3Dq mice showed a similar habitual behavior (Fig. 8G; n = 5 for the Vehicle group and n = 5 for the SNAP group). On the other hand, we confirmed that SNAP did not alter locomotor activity in the open-field task (fig. S11). Therefore, these results suggest that increased GAT3 expression is required for habitual seeking behaviors, and pharmacological inhibition of GAT3 function is sufficient to reverse the activated GPe astrocyte–driven behavioral changes.

Fig. 7. Expression of GAT3 mRNA in goal-directed and habit behaviors using quantitative real-time PCR in the GPe.

(A) Behavioral schedules to prepare the GPe tissues. We isolated the GPe immediately after the last session of FR1, RR20, and RI120. (B) Nose-poking behaviors from the FR1 to RR20 or RI120 task. (C) Schematic illustration of RNA extraction, cDNA synthesis, and real-time PCR using an Applied Biosystems 2720 thermal cycler. (D) Amplification plot of GAT3 and β-actin expression. (E) Quantification of GAT3 expression expressed as normalized fold change (FC). Data represented mean ± SEM. n = 5 for FR1, n = 5 for RR, n = 8 for RI. One-way ANOVA and Tukey post hoc tests, *P < 0.05. See table S3 for full statistical information. qRT-PCR, quantitative real-time polymerase chain reaction.

Fig. 8. Pharmacological inhibition of GAT3 on GPe neuronal activities and astrocyte activation-induced transition from habit to goal-directed behaviors.

(A to C) The representative traces (A) and summarized data (B) and (C) show the effects of GAT3 blockade in the GPe neuronal firing after the astrocyte activation. (B) Relative firing changes after C21 treatment with vehicle or SNAP5114 treatment in hM3Dq-expressing mice at the GPe. (C) Firing rate after C21 treatment with vehicle or SNAP5114. (D) Schematic diagram showing hM3Dq virus injection and cannula implantation into mouse GPe. (E) Experimental scheme for operant conditioning. (F) Nose poke between valued and devalued states after C21 treatment and Vehicle or SNAP5114 treatment in extinction test after the RI task. (G) Nose poke between valued and devalued states after vehicle or SNAP5114 treatment without C21 treatment in the extinction test after the RI task. Data represented mean ± SEM; (A) to (C) n = 16 for Veh, n = 17 for SNAP. Scale bar, 500 ms, 100 μV. (F) n = 10, (G) n = 5; *P < 0.05 comparing each group. (A) to (C) Two-way repeated-measures ANOVA, followed by Bonferroni’s multiple-comparisons test; (F) and (G) two-tailed Wilcoxon matched-pairs signed-rank test. C21, compound 21; V, valued; DV, devalued; FR1, fixed ratio 1; RI, random interval; Veh, vehicle; SNAP, SNAP5114. See table S3 for full statistical information.

Attentional stimuli reduce habitual seeking behaviors

Individuals performing habitual actions are insensitive to reward devaluation and can mindlessly repeat even detrimental actions regardless of the consequences (33). Thus, we hypothesized that the animal would no longer act habitually if salient attentional stimuli disrupted the repetitive behavior pattern. Furthermore, because vigilance is a state of readiness to respond to state changes (34), if we provide mice with highly salient attentional stimuli to keep them alert, habitual behavior may be disrupted, resulting in goal-directed behavior. Therefore, we gently released a nonhazardous ball along a parabolic path to lightly touch the mouse’s face in the home cage as an attentional stimulus and simultaneously recorded in vivo calcium signaling in the GPe of GFAP-Cre/DIO-GCaMP6s mice to investigate whether the attentional stimulus yields GPe astrocyte–dependent brain signals (Fig. 9A). Astrocytic calcium dynamics in all 32 trials were sharply increased after the attentional stimuli (Fig. 9, B and C; n = 32 per group from four mice), suggesting that the attentional stimuli can mimic chemogenetic activation of the Gq pathway in GPe astrocytes. Therefore, we applied the attentional stimulus in mice that had developed habitual-seeking behavior. We introduced the attentional stimulus eight times over 4 min in the home cage before the extinction test in the operant chamber (Fig. 9D). Consistent with our hypothesis, the attentional stimulus reduced habit index (Fig. 9E; n = 9 per group) and altered mouse nose-poking behaviors between the valued and devalued states (Fig. 9F; n = 9 per group), while the control group (no stimulation) showed similar nose-poking behaviors (Fig. 9F; n = 9 per group). Moreover, these attentional stimuli did not alter locomotor activity (Fig. 9, G to I; n = 7 for the Ctrl group and n = 6 for the Stim group) and anxiety-like behavior (Fig. 9J; n = 7 for the Ctrl group and n = 6 for the Stim group). These results demonstrated that attentional stimuli reduce goal-directed behavior by activating the GPe astrocytes.

Fig. 9. Effects of attentional stimuli on GPe astrocytic dynamics and behaviors.

(A) Scheme for attentional stimuli. (B) Temporal dynamics of GPe astrocytes by attentional stimuli. (C) Difference of averaged astrocytic dynamics between 2 s before and after the attentional stimuli. The data of pre–2 s and post–2 s of attentional stimuli were collected from the gray zone of Fig. 7B. (D) Experimental scheme for attentional stimulation in operant behavior. (E) Reduced habit index by attentional stimulation. (F) Change in nose poke between valued and devalued states by attentional stimulation. (G) Experimental scheme for an open-field test with attentional stimulation. (H and I) No change in locomotor activity for 30 min. (J) No change in anxiety-like behavior. Data represented mean ± SEM. (B) and (C) n = 32 from 4 mice. (D) to (F) n = 10 per group. (G) to (J) n = 7 for the control group, n = 6 for the stimulation group (C), (E), and (F). *P < 0.05 comparing each group. (C), (E), and (F) Two-tailed Wilcoxon matched-pairs signed-rank test. (H) Two-way repeated-measures ANOVA, followed by Bonferroni’s multiple-comparisons test. (I) and (J) Two-tailed Mann-Whitney test. Ctrl, control; FR1, fixed ratio 1; DV, devalued state; RI, random interval; Stim, stimulation; V, valued state. See table S3 for full statistical information.

DISCUSSION

In this study, we demonstrated that GPe astrocytes encode action strategies. Using astrocyte-specific calcium signaling, we found that temporal dynamics of GPe astrocytes were attenuated during habit learning. In addition, we found that an SVM reliably predicted whether mice harbor goal-directed or habitual action strategies using the data of GPe astrocytic dynamics. Moreover, chemogenetic activation of the Gq pathway in GPe astrocytes and chemogenetic activation of the Gi pathway in GPe neurons attenuated the dominance of habitual seeking. An increase in GAT3 expression is also required for habit formation. Evidently, pharmacological inhibition of GAT3 function precludes the GPe astrocyte activation–induced transitioning from goal-directed to habitual seeking behavior. This implies that increased GAT3 inactivates the GPe astrocytes, suppressing the change from habitual search to goal-directed behavior (Fig. 10).

Fig. 10. Schematics summarizing the results of the present study.

We found a reduced astrocyte activity in the GPe and increased GAT3 expression during habit training (RI120). Chemogenetic activation facilitates the transition from habit to goal-directed behavior and reduces GPe neuronal activities. We then inhibited the GPe neuronal activities, which showed a similar result to the astrocyte activation. Last, we demonstrated that the GAT3 blocker prevents the effect of astrocyte activations on GPe neurons and transitioning from habit to goal-directed behavior.

Habits are a state of insensitivity to reward devaluation or degradation of A-O contingency (35), as we measured the differences of nose-poking behaviors during the extinction period between valued and devalued states. In this assessment, maintaining nose-poking behavior despite reward devaluation indicates maladaptive habits. Although sucrose does not fully represent addictive drugs (33), our research could be relevant to addiction because sucrose has similar rewarding and reinforcing properties to other substances of abuse, especially in rodents (15, 34). A hallmark of addiction is an impaired ability to stop seeking and taking substances and a dominance of habitual over goal-directed behaviors (36, 37). Uncontrollable habits and a deficit of behavioral flexibility are the aspects of compulsive behaviors that are insensitive to negative consequences (2, 6, 38–40). However, several experimental and theoretical studies disagree with the progressive transition from goal-directed to habitual-seeking behaviors, emphasizing excessive goal-directed behaviors as addiction (41, 42). Despite this debate, intervening in uncontrollable and excessive seeking behaviors is critical to addiction treatment.

In this study, GPe astrocyte activation disrupted the dominance of habitual behaviors. This is likely because Gq pathway activation in GPe astrocytes dampened pan-GPe neuronal activities. Consistent with our findings, enhanced astrocyte activities in the DMS, which abolish habitual seeking behavior, also reduce GPe neuronal activities (27). Thus, this firing reduction of GPe GABAergic neurons may disinhibit the glutamatergic subthalamic nucleus (STN) neurons to the internal globus pallidus (GPi) and GPi neurons to the thalamus, inhibiting thalamic glutamatergic neurons (4, 43, 44), thereby inhibiting habitual behavior. Thus, GPe astrocytes may indirectly control habitual behaviors by intervening in the cortico-striatal-pallidal-thalamic-cortical loop. Although our study focuses on the GPe, which receives GABAergic inputs from the dorsal striatum, the nucleus accumbens core and shell are also critical for operant behaviors (45, 46). Therefore, additional research will delineate the circuit-specific regulation of operant behaviors.

Notably, we found a substantial increase in astrocytic dynamics after magazine exit, and the SVM analysis highly predicted behavioral task types using astrocytic dynamics during magazine exit. The sharp increase in GPe astrocytic dynamics did not appear upon magazine exit without reward consumption when mice behaved with loose A-O contingency and seeking-taking association. The magazine exit without any reward collection may motivate nose-poking behavior when mice expect the reward and behave with the high seeking-taking association. However, nose-poking behavior to acquire the reward may be unnecessary in mice with weakened A-O contingency and seeking-taking association, as these mice may perform nose-poking behavior regardless of the reward outcome. Because reward-seeking and -taking behaviors have conceptual distinctions (39), our cellular and behavioral findings support the involvement of GPe astrocytes in action strategy.

Consistent with our findings, several previous studies confirmed that astrocytes are highly enriched in the GPe (11, 47–49), implying a crucial role of astrocytes in the GPe. Moreover, in addition to astrocytes, GPe harbors anatomically and molecularly divergent neuronal pathways: arkypallidal feedback neurons to the dorsal striatum and prototypic feedforward neurons; and Forkhead box protein P2 (Foxp2)–, LIM homeobox 6 (Lhx6)–, and parvalbumin (PV)–expressing neurons (50–52). Moreover, the GPe receives not only GABAergic input from the dorsal striatum, particularly from dopamine D2 receptor and/or adenosine A2A receptor–expressing medium spiny neurons, but also glutamatergic input from the STN. Furthermore, GPe PV neurons mainly project to the STN and parafascicular nucleus, and GPe Lhx6 neurons project to STN and substantia nigra pars compacta. Considering these complex input and output circuits of the GPe, the abundant GPe astrocytes may act as fine moderators balancing activity among these diverse output pathways. In this study, we confirmed the inverse relationship between astrocytes and neuronal activities in the GPe. However, it remains unknown whether astrocytes differentially shape the activity of the diverse subtypes of GPe neurons. Similarly, although our pan-neuronal inhibition elicited a similar behavioral transition from habit to goal-directed reward seeking, additional studies are warranted to reveal specific cell-type function in behavioral flexibility.

The GABA plasma membrane transporter (GAT) regulates GABA levels in the brain. Among three subtypes of GAT (GAT 1 to GAT3) (53), GAT3 is expressed exclusively in astrocytes. Thus, astrocyte GAT3 expression and function may be critical for regulating GABA levels in the GPe (54). The substantial increase of GAT3 expression during habit learning in the RI 120 schedule agrees with previous studies showing the increased GAT3 expression in repetitive behaviors (29). Because GABA uptake in GPe astrocytes elicited more frequent neuronal action potentials coupled with reduced astrocyte activity (55, 56), GAT3 may alter the local synaptic and extrasynaptic GABA concentrations in the GPe. However, there are various types of GPe projection neurons (e.g., prototypical and arkypallidal neurons). Thus, further cell type–specific behavioral and electrophysiological experiments are critical to elucidate the GPe neuronal type–dependent changes upon GAT3 expression and function. In addition, further molecular studies will reveal the precise role of astrocytes in moderating the GPe circuit-dependent activities, such as through glutamate, dopamine, and adenosine signaling (27, 57–59).

Goal-directed behavior requires cognitive attention to modify actions based on the outcome, while habit demands less attention (60–62). According to Posner’s attention model, the three attention types (alerting attention, orienting attention, and executive control) contribute to cognitive function before taking action (63). Among these attention types, our finding supports that enhancing alerting attention (also known as vigilance) breaks habitual action strategy (64–66). Our alternative approach demonstrates that contact with an unexpected object is sufficient to alert mice and subsequently break habitual action performance in the operant chamber. These attentional stimuli increased intracellular calcium dynamics of GPe astrocytes, similar to the chemogenetic activation of the Gq pathway in GPe astrocytes. This implies that (i) the GPe may be in the hierarchical signal stream of alerting attention as the leading actor or (ii) the GPe may receive signals from alerting attention-related brain areas and process or record this signal to modify the action strategy. Although the brain circuit and area studies related to attention have been mainly conducted in the cortical regions, such as frontoparietal circuits and prefrontal cortex, subcortical structures, including the anterior cingulate cortex, thalamus, and basal ganglia, are also the critical elements of the attention network (67–71). Moreover, the right side of GPe that we recorded using in vivo calcium signaling in mice contributes to altered causal awareness in humans (72), providing the possibility that the GPe is critical to the attention function according to changes in state. However, there is not enough knowledge on how neurotransmitters, including norepinephrine, which strongly influences arousal modulation and cognitive preparation in response to urgent stimuli (68), is related to the GPe dynamics. In addition, we only use male mice in this study. Because female rats exhibit more habitual behaviors in operant training than males (73), it is worth comparing the sex differences of astrocyte and neuronal activities in the GPe. Nevertheless, striatal astrocytes, through GABA signaling, intervene in attention disruption in a rodent study (70).

In summary, our study demonstrated that GPe astrocytic dynamics are reduced during habit learning. Chemogenetic and attentional stimulation can prevent the dominance of habitual reward-seeking, at least partly through the activation of the Gq pathway in GPe astrocytes. Our finding may help to treat maladaptive habit-related disorders such as addiction and obsessive-compulsive disorders.

MATERIALS AND METHODS

Animals

All experimental procedures were approved by the Mayo Clinic Institutional Animal Care and Use Committee and performed following National Institute of Health guidelines. C57BL/6J mice and bitransgenic GFAP-Cre (stock no. 024098)/DIO-GCaMP6s (stock no. 028866) mice were purchased from the Jackson Laboratory (Bar Harbor, ME). Mice were housed in standard Plexiglas cages. The colony room was maintained at a constant temperature (24° ± 1°C) and humidity (60 ± 2%) with a 12-hour light/12-hour dark cycle (lights on at 7:00 a.m.). We used 8- to 10-week-old male mice for all experiments. Mice were allowed ad libitum access to food and water. For the operant conditioning tests, mice were food-restricted to 85% of their baseline weight, at which time they were maintained for the duration of experimental procedures.

Stereotaxic surgery and virus injection

Mice were anesthetized with isoflurane (1.5% in oxygen gas) with the VetFlo vaporizer with a single-channel anesthesia stand (Kent Scientific Corporation, Torrington, CT) and placed on the digital stereotaxic alignment system (model 1900, David Kopf Instruments, Tujunga, CA). Hair was trimmed, and the skull was exposed using an 8-gauge electrosurgical skin cutter (KLS Martin, Jacksonville, FL). The skull was leveled using a dual tilt measurement tool. Holes were drilled in the skull at the appropriate stereotaxic coordinates. For chemogenetics, viruses were infused unilaterally to the GPe [anteroposterior (AP) −0.46 mm, mediolateral (ML) +2.0 mm, dorsoventral (DV) −3.2 mm from dura] at 100 nl/min for 3 min through a 33-gauge injection needle (catalog no. NF33BV, World Precision Instruments) using a microsyringe pump (model UMP3, World Precision Instruments). The injection needle remained in place for an additional 6 min following the end of the injection. We injected buprenorphine sustained-release LAB (1 mg/kg, subcutaneously; ZooPharm, Laramie, WY) to alleviate postsurgery pain. Mice were used for the experiments 4 to 5 weeks after the virus injection. We injected viruses at the following titers: AAV5-GFAP-mCherry (for control), 1.6 × 10¹³ GC/ml (Vector Biolabs, Malvern, PA); AAV5-GFAP-hM3Dq-mCherry, 1.2 × 10¹³ GC/ml. AAV-GFAP-hM3Dq-mCherry was a gift from B. Roth [Addgene viral prep no. 50478-AAV5; http://n2t.net/addgene:50478; Research Resource Identifiers (RRID): Addgene_50478]. AAV5-CamKIIα-hM4Di-mCherry, 2 × 10¹² GC/ml (Addgene viral prep no. 50477).

Chemogenetics (designer receptors exclusively activated by designer drugs)

We administered compound 21 (3 mg/kg, ip) or saline (vehicle) 30 min before the experiments in mice. We purchased compound 21 (C21) from Hello Bio (Princeton, NJ).

Cannula implantation and microinfusion

Mice were infused with 20 mM 1-[2-[tris(4-methoxyphenyl)methoxy]ethyl]-(S)-3-piperidinecarboxylic acid (SNAP5114, dissolved in dimethyl sulfoxide; Tocris) bilaterally in the GPe (AP −0.46 mm, ML ±2.0 mm, DV −3.2 mm from dura) using a cannula injection system (C313DC, C313G, and C313I; P1 Technologies) 20 min before the extinction test (28).

Quantitative real-time PCR

Mice were euthanized by CO₂ asphyxiation and rapidly decapitated. The GPe was then isolated under a surgical microscope. RNA isolation was done using an RNeasy Plus mini kit (Qiagen, catalog no. 74136). RNA was reverse-transcribed by a SuperScript VILO complementary DNA (cDNA) synthesis kit (Invitrogen, catalog no. 11754250). For thermal cycling, an Applied Biosystems 2720 thermal cycler was used. The thermal cycling protocol for reverse transcription was 10 min at 25°C, 60 min at 42°C, followed by 5 min at 85°C. Real-time PCR was performed on the QuantStudio 7 Pro Real-Time PCR System using TaqMan gene-specific assays. The catalog numbers of the TaqMan assays used for β-actin and GAT-3 were Mm02619580_g1 and Mm00556476_m1, respectively. The targeted gene mRNA expression was normalized to β-actin. Percentage changes were calculated by subtracting β-actin cycle threshold (Ct) values from Ct values for the gene of interest using the 2^−ΔΔCT method (74).

Behavioral experiments

Operant conditioning

Mice were trained with 20% sucrose solution on two different schedules after FR1: RR and RI to develop goal-directed and habitual seeking behaviors, respectively (30). Following the nose-poking behavior, mice can acquire the reward at the magazine. Briefly, we conducted operant conditioning using the same operant chambers (Med Associates Inc., Fairfax, VT) with our previous publications (27, 75) consisting of an active hole, an inactive hole, a magazine, a house light, a speaker, and a cue light at each nose port. Reward (20% sucrose solution) was presented to the liquid receptacle in the magazine once per reward signal by a syringe pump. The correct nose-poke action was either rewarded or unrewarded depending on the reward contingency. In a rewarded condition, the chamber presented a tone, light from the nose port, and one reward from the magazine. The tone and light stimulations occurred for 2 s immediately after nose poking. In the case of nonrewarded nose poke, the chamber did not present any stimuli or reward (Fig. 1A). The operant conditioning schedule is as follows. On the first day, mice learned during magazine training (30 min) that they could collect rewards in the magazine. Then, the FR1 was performed for four sessions and 60 min, and if mice obtained 60 rewards, the session was terminated regardless of the remaining time. For rapid learning of operant behavior, 10 μl of 20% sucrose was placed in the active hole as bait before starting the FR1 session. If mice showed an average latency time from nose poke to the magazine of fewer than 2 s for three consecutive sessions of the FR1 task, we did not place bait from the next session.

Afterward, an RR schedule was performed to develop goal-directed behaviors, and an RI schedule was performed to form habits. The RR schedule consisted of RR2 (one session), RR5 (two sessions), RR10 (three sessions), and RR20 (two sessions). The RI schedule consisted of RI30 (one session), RI60 (three sessions), and RI120 (four sessions). The RR and RI tasks were performed for 30 min, and if mice obtained 60 rewards, the session was terminated regardless of the remaining time.

The rules for FR1, RR, and RI tasks were as follows. In the FR1 task, all nose pokes were rewarded. In the RR task, obtaining one reward requires multiple nose pokes. Only the last nose poke was rewarded when a certain number of nose pokes was reached, and all others were nonrewarded. The RI task requires a certain amount of time to flow between two rewarded nose pokes. Therefore, the nose poke over time becomes a nonrewarded nose poke. One tone, cue light, and reward followed a rewarded nose poke. If mice poke nonrewarded nose poke, nothing happens. For example, mice can obtain a reward only after performing an average of 20 nose pokes after a rewarded nose poke in the RR20 task. Mice can obtain a reward only after nose poking after an average of 120 s in the RI120 task (Fig. 1A). Med-PC-IV software (Med Associates Inc., Fairfax, VT) recorded the time points of nose poke and magazine entry, session duration, latency time from nose poke to magazine approach, time spent in the magazine, and the number of nose poke and magazine entries, and the time resolution was 10 ms. The discrimination ratio (%) was calculated as the number of nose poke for active holes divided into total nose-poke numbers.

Reward devaluation and extinction test

According to the sensory-specific satiety theory, we conducted reward devaluation for 1 hour and the extinction test for 10 min to determine which is more dominant between goal-directed behavior and habit. We provided unlimited food chow to make the reward worthwhile for the valued state, whereas we provided an unlimited 20% sucrose solution to devaluate the outcome value. We conducted the extinction tests after the last sessions of FR1, RR20, and RI120. Habit index (%) = 100 −|difference in the number of nose poke between the valued and devalued states|/total nose pokes in the valued and devalued states × 100. The 100% habit index means no signs of goal-directed behaviors.

Open-field test

We conducted the open-field test in chambers (Med Associates, Fairfax, VT). We recorded locomotor activities in mice for 30 min, and the first 10 min was for checking anxiety-like behavior. In addition, we recorded locomotor activities 30 min after drug treatment (vehicle, C21) and immediately after providing the attentional stimuli. Distance traveled was measured using beam breaks. Time spent in an open zone (%) = time spent in an open zone/total time × 100.

Attentional stimuli

We released a ball along a parabolic path to lightly touch the mouse’s face to give brief but salient attentional stimulation (Fig. 9A). First, the thread was attached to a hard rubber ball (60 g), and we grabbed an end of the thread with one hand. Then, we gently released the ball along a parabolic path to touch the mouse’s face in the home cage. We applied this stimulation once every 30 s for a total of eight times for 4 min before the open-field and extinction tests. For in vivo Ca²⁺ signaling, we gave mice this attentional stimulation once every 30 s for a total of six to eight times.

In vivo calcium signaling with fiber photometry

As described previously, we recorded the cellular calcium transients in real time in vivo using fiber photometry (27). Briefly, we implanted an optic cannula (200/240 μm diameter, 200 μm end fiber) and fixed it into the GPe (AP −0.46 mm, ML +2.0 mm, DV −3.0 mm from dura) of GFAP-Cre/DIO-GCaMP6s mice. The implanted fiber was linked to a patch cord, and the light intensity at the fiber tip was 60 μW consistently. These output signals were projected onto a photodetector by the same optical fiber and passed through a green fluorescent protein filter. Data analysis was performed using the CineLyzer software (version 4.4, Plexon), and light intensities were measured as relative fluorescence change (ΔF/F₀). ΔF/F₀ (%) was calculated relative to the signal (F₀) averaged in the 3 s preceding the time alignment [ΔF/F₀ = [F(t) − F₀]/F₀]: F(t) is the fluorescent value at a given time, and F₀ is the resting averaged fluorescence value in the 3 s preceding the time alignment. The camera for observing the mice’s performance and the GCaMP6s-based astrocyte recordings were synchronized and recorded at 30 frames per second. In operant conditioning, we recorded Ca²⁺ signaling in the GPe of mice during the last sessions of RR20 and RI120. For attentional stimulation, we measured cellular calcium transient for 3 to 4 min. To compare calcium transients between before and after the action events, the ΔF/F₀ data were averaged over the 2 s before or after the behavioral event. Furthermore, to compare the difference between calcium transients before and after the behavioral event, we subtracted averaged ΔF/F₀ data before the behavioral event from averaged ΔF/F₀ data after the behavioral event.

In vivo electrophysiology

The experiments were performed as described previously (27). Briefly, mice, 3 to 4 weeks following virus injections, were anesthetized by intraperitoneal injection of urethane (1.5 g/kg; Sigma-Aldrich, St. Louis, MO) (21) and placed horizontally on a stereotaxic frame (RWD Life Science, San Diego, CA). We constantly monitored respiratory rate and pedal withdrawal reflex during anesthesia, and the physiological body temperature was maintained using a small-animal feedback-controlled warming pad (Kent Scientific Corporation, Torrington, CT). After the scalp incision, small burr holes were drilled to insert high-impedance microelectrodes (Cambridge NeuroTech, Cambridge, UK). We placed the reference wire (Ag/AgCl, 0.03 inches in diameter, A-M systems) in the contralateral parietal cortex. Electrophysiological signals were digitized at 20 kHz and band pass–filtered from 300 to 3000 Hz (RHS 2000, Intan Technologies, Los Angeles, CA). The data were analyzed with Clampfit (version 11.2, Molecular Devices, San Jose, CA) and a custom-written code in MATLAB (R2019a, The MathWorks, Natick, MA). Spike sorting was performed offline by superparamagnetic clustering of wavelet coefficients (Wave Clus toolbox) (76). An amplitude threshold of four medians of the absolute value of the signal was used for spike detection in each channel. The sorting result was then complemented by visual inspection of waveforms, variability, and interspike-interval distributions. Last, spiking frequency was calculated in 1-s bins.

Immunofluorescence

Brains were fixed with 4% paraformaldehyde (Sigma-Aldrich) and transferred to 30% sucrose (Sigma-Aldrich) in phosphate-buffered saline at 4°C for 72 hours. Brains were then frozen in dry ice and sectioned at 40 μm using a microtome (Leica Corp., Bannockburn, IL). Brain slices were stored at −20°C in a cryoprotectant solution containing 30% sucrose (Sigma-Aldrich) and 30% ethylene glycol (Sigma-Aldrich) in phosphate-buffered saline. Sections were incubated in 0.2% Triton X-100 (Sigma-Aldrich) and 5% bovine serum albumin in phosphate-buffered saline for 1 hour, followed by incubation with the primary antibody in 5% bovine serum albumin overnight at 4°C. Primary antibodies that were used in the present study included mouse anti-GFAP Alexa Fluor 488 (1:100; monoclonal immunoglobulin G1, #53-9892-82, Thermo Fisher Scientific, Waltham, MA) antibody. After washing with phosphate-buffered saline three times, the sections were mounted onto a glass slide coated with gelatin and cover-slipped with a VECTASHIELD antifade mounting medium with 4′,6-diamidino-2-phenylindole (Vector Laboratories, Burlingame, CA). Images were obtained using an LSM 700 laser scanning confocal microscope (Carl Zeiss, Heidelberg, Germany) using a 10× lens (Fig. 4A and fig. S2B) and a 63× water-immersion lens (Figs. 4A and 5A).

Classification analysis

To investigate whether astrocyte activity can predict behavioral tasks (RR or RI), we used an SVM and fourfold cross-validation. Average astrocyte activity during 1-s periods immediately before and after the action events (NP, M_entry, and M_exit) was used as input. Data from different mice were normalized by dividing by F₀ and then pooled together (n = 6). The number of labels was 2 (rewarded or nonrewarded). To balance the dataset [3035 times of NP (R+) and 174 times of NP (R−) in the RR task, 64 times of NP (R+) and 1045 times of NP (R−) in the RI task; 120 times of either M_entry (R+) or M_exit (R+) and 170 times of either M_entry (R−) or M_exit (R−) in the RR task, 58 times of either M_entry (R+) or M_exit (R+) and 549 times of either M_entry (R−) or M_exit (R−) in the RI task], we performed random undersampling. Random undersampling was repeated 400 times for each analysis, and an averaged performance of the SVM was calculated. We used MATLAB (version R2020a) for SVM analyses.

Data analysis

All data are represented as mean ± SEM and were analyzed by two-tailed Mann-Whitney U test, two-tailed Wilcoxon matched-pairs signed-rank test, Kolmogorov-Smirnov test, Kruskal-Wallis test followed by Dunn’s multiple comparisons test, two-way analysis of variance (ANOVA) followed by Tukey’s multiple comparisons tests, two-way repeated-measures ANOVA followed by Bonferroni’s multiple comparisons tests, Spearman correlation, simple linear regression, and normality test with Shapiro-Wilk test using Prism 9.0 (GraphPad Software, San Diego, CA). The statistical significance was set at P < 0.05. All P values less than 0.05 were marked as “*” in the graphs. Detailed statistical data with exact P values are listed in table S3.

To display operant behavior patterns before and after the behavioral events, we collected reward-seeking and -taking behaviors during the last sessions of the RR20 and RI120 schedules. First, we calculated the accumulated number of nose poke every 2 s (a bin) for reward-seeking behavior. Moreover, for reward-taking behavior, we gave scores (0, if mice stayed at the magazine; 1, if mice stayed outside of the magazine) as the probability of reward-taking behavior in each frame (30 frames per second). For the A-O contingency, we created two behavioral data arrays from the last sessions of the RR20 and RI120 tasks: the action array for accumulated numbers of rewarded and nonrewarded nose pokes and the outcome array for accumulated numbers of reward outcomes in every 60-s period (77). Then, we calculated the Spearman r correlation coefficient between two arrays. The A-O contingency (%) = |Spearman r correlation coefficient| × 100. All the r values (from eight mice) in the RR group were positive numbers. Three of seven mice in the RI group showed negative numbers of the r values.

For the seeking-taking association, we calculated two types of association at the last session of the RR20 and RI120 tasks: behavioral event-dependent seeking-taking association (fig. S3G) and the seeking-taking association throughout the session (fig. S3H). We created two arrays: The reward-seeking behavior array, the accumulated numbers of total nose poke, and the reward-taking behavior array, which is the averaged probability of taking behavior in every 2-s (60 frames) period or 60-s (1800 frames) period. Every 2 s of the bin was for the behavioral event-dependent association, and every 60 s of the bin was for seeking-taking association throughout the session. Then, we calculated the Spearman r correlation coefficient. The seeking-taking association (%) = |Spearman r correlation coefficient| × 100.

Supplementary Materials

This PDF file includes:

Figs. S1 to S11

Tables S1 to S3

View/request a protocol for this paper from Bio-protocol.