Astrocyte adenosine signaling and neural mechanisms of goal-directed and habitual reward-seeking behaviors

While goal-directed behaviors pursue maximized reward through action–outcome contingency, repeated behaviors often produce invariances between action and outcome, which constitute habits [1]. The inflexible maladaptive and harmful habits are attributed to addiction and many decision-making disorders. Recently, computational neuroscience and psychiatry approaches are burgeoning to decipher the neural networks underlying reward-seeking patterns and reveal the predictable biological determinants [1, 2]. In line with this trend, computational network models have been proposed for model-based and model-free reinforcements learning, which are representatives of goal-directed and habitual controls, and dynamic “neuronal” interactions between dorsomedial and dorsolateral striatum (DMS, DLS) through cortico-striatal-pallidal-thalamic circuits have been spotlighted as a core module in the balance between goal-directed and habitual actions [1, 2].

In addition to well-known neurosupportive roles, astrocytes are emerging as a key determinant of neuronal synaptic function and consequent behavioral changes through the release of various gliotransmitters including adenosine [3]. Indeed, astrocyte dysregulation is associated with neuropsychiatric disorders including substance use disorder [4]. Yet, how astrocytes temporally affect the neuronal synaptic and circuit function underlying the transition between goal-directed and habitual reward-seeking was not known. In our recent study [5], we showed astrocytic activation-induced changes in neuronal synaptic transmissions in the DMS and reward-seeking behavioral patterns. We found that spatiotemporal chemogenetic activation of the astrocytes in the DMS shift from learned habits to goal-directed reward-seeking behaviors. In addition, we provided evidence that DMS astrocytes could alter the neuronal activities in the downstream circuit, indicating the importance of local microcircuits for the designated behavioral consequences. However, this shift was blunted by pharmacological and genetic depletion of astrocyte adenosine signaling. These complementary mechanisms of adenosine-driven astrocyte-neuron interaction reveal a complex neural network which is required for the fine regulation of reward-seeking behavior [6].

Although we are on an initial stage to employ mathematical and computational algorithm for the neural circuits’ modulation that underlie the reward-seeking patterns, our recent finding is potentially contributing to refining the actor-critic model of reinforcement learning, especially for the OpAL (opponent actor learning) model. In this model, actor is divided into two components in the striatum as dopamine D1 receptor expressing direct medium spiny neurons and dopamine D2 or adenosine A2A receptors expressing indirect MSNs (iMSNs) [2]. Our recent study showed that astrocyte activation in DMS differentially dampening inhibitory postsynaptic currents of iMSNs in adenosine signaling dependent manner [5], indicating that peri-synaptic astrocytes may be an additional determinant for the actor activities. As we presented, calcium oscillation of astrocytes is a hallmark of astrocyte activity and adenosine signaling [5]. Thus, fiber-photometry or microendoscopy-based calcium dynamics will be quantifiable data for mathematical models of modified OpAL model, which encoding the neural changes and predictable components for the behavior outcomes.

Overall, a better understanding of how astrocyte-neuron interaction has a net-change outcome as actors or critics in the networks will be critical in helping computational neuroscientists to construct better artificial neural networks for improved deep machine learning algorithm and artificial intelligence to create prediction models for addiction and other neuropsychiatric disorders.

Funding and disclosure

This work was supported by the Samuel C. Johnson for Genomics of Addiction Program at Mayo Clinic, the Ulm Foundation, and National Institute on Alcohol Abuse and Alcoholism (K01 AA027773 to SK, R01 AA018779 to DSC). DSC is a scientific advisory board member to Peptron Inc. and the Peptron had no role in preparation, review, or approval of the manuscript; nor the decision to submit the manuscript for publication. SK has no biomedical financial interests or potential conflicts of interest.

Author contributions

SK and DSC prepared and wrote the manuscript.