Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2017 Feb 3.
Published in final edited form as: Neuron. 2016 Jan 21;89(3):472–479. doi: 10.1016/j.neuron.2015.12.032

Pathway-Specific Striatal Substrates for Habitual Behavior

Justin K O’Hare 1,2,*, Kristen K Ade 1,2,*, Tatyana Sukharnikova 3, Stephen D Van Hooser 4, Mark L Palmeri 5, Henry H Yin 3, Nicole Calakos 1,2
PMCID: PMC4887103  NIHMSID: NIHMS753257  PMID: 26804995

Abstract

The dorsolateral striatum (DLS) is implicated in habit formation. However, the DLS circuit mechanisms underlying habit remain unclear. A key role for DLS is to transform sensorimotor cortical input into firing of output neurons that project to the mutually antagonistic direct and indirect basal ganglia pathways. Here we examine whether habit alters this input-output function. By imaging cortically-evoked firing in large populations of pathway-defined striatal projection neurons (SPNs), we identify features that strongly correlate with habitual behavior on a subject-by-subject basis. Habitual behavior correlated with strengthened DLS output to both pathways as well as a tendency for action-promoting direct pathway SPNs to fire before indirect pathway SPNs. In contrast, habit suppression correlated solely with a weakened direct pathway output. Surprisingly, all effects were broadly distributed in space. Together, these findings indicate that the striatum imposes broad, pathway-specific modulations of incoming activity to render learned motor behaviors habitual.

INTRODUCTION

Habit formation is a fundamental adaptive mechanism drawn upon daily to provide efficiency for common behaviors (Graybiel, 2008; Yin and Knowlton, 2006). Mechanisms underlying habit formation are also thought to be co-opted by drugs of abuse and disrupted in numerous neuropsychiatric diseases involving compulsivity (Gerdeman and Lovinger, 2003). Habits arise from goal-directed learning as a particular behavior becomes automated with repetition and its performance becomes less dependent on the outcome. As demonstrated by behavioral assays designed to assess the mode of behavioral control, the transition from goal-directed to habitual behavior, as well as the maintenance of habit, requires DLS (Furlong et al., 2014; Yin et al., 2006; Yin and Balleine, 2004). Habit formation is also accompanied by altered in vivo activity (Desrochers et al., 2015; Rueda-Orozco and Robbe, 2015; Tang et al., 2007) in this region.

While the DLS is strongly implicated in habitual behavior, habit-related activity changes have also been described in afferent cortical regions (Coutureau and Killcross, 2003; Gremel and Costa, 2013). This observation highlights a key ambiguity in our understanding of the neural encoding of habit: Are behaviorally salient features locally encoded in the striatum? Or, is the habit-related striatal activity observed in vivo driven by afferent cortical activity? To address these questions, it is necessary to isolate the DLS contributions to habitual behavior independently of changes in cortical activity. In support of the idea that DLS circuitry might locally store habit-related information, studies in brain slices have found long-lasting changes in synaptic and intrinsic membrane properties of projection neurons in mice extensively trained on a nose-poke task with chronic cannabinoid exposure (Nazzaro et al., 2012), a lever press task Shan et al., 2015), and on a rotarod task (Yin et al., 2009). However, no clear model for how DLS plasticity alters striatal output has emerged from these observations.

One approach to address these issues is to directly monitor striatal output using calcium imaging to report firing activity (Smetters et al., 1999). By performing such experiments in brain slices under conditions that include contributions of glutamatergic synapses, GABA-ergic synapses, and intrinsic membrane properties along with experimenter-controlled activation of cortical inputs, one can elucidate how local DLS circuit plasticity modifies striatal output in habit. We therefore sought to develop such an approach using vector-mode two-photon laser scanning microscopy (2PLSM) calcium imaging to measure corticostriatal input-output relationships in DLS as a function of habitual behavior in individual mice.

At the circuit level, the DLS is known to integrate incoming action-related information from sensorimotor cortices into firing of striatal projection neurons (SPNs) which constitute its two output pathways. Historically, these pathways have been regarded as mutually antagonistic, with the direct (striatonigral) pathway facilitating and the indirect (striatopallidal) pathway inhibiting movement (Albin et al., 1989; DeLong, 1990). Optogenetic manipulations of direct pathway SPNs (dSPNs) and indirect pathway SPNs (iSPNs) support this model (Kravitz et al., 2010; Kravitz et al., 2012). However, recent observations that both pathways are concurrently activated with action initiation (Cui et al., 2013), and that balanced activity between pathways is necessary for normal movement (Tecuapetla et al., 2014), have led to an updated model of basal ganglia function in which dSPNs facilitate voluntary movement while iSPNs concomitantly suppress competing, unintended motor programs (reviewed in Calabresi et al., 2014 and Nelson and Kreitzer, 2014). To examine the circuit contributions of DLS to behavior, we further developed our approach to enable simultaneous measurement of activity of both SPN types. We identified pathway-specific DLS SPN firing properties that robustly predicted how habitually individual subjects behaved in a lever press task. These findings indicate that local DLS plasticity mechanisms are sufficient to drive habit-related changes in DLS output and further illustrate how the direct and indirect pathways contribute to behavior.

RESULTS

Pathway-specific analysis of striatal output in trained mice

To test the relationship between habitual behavior and corticostriatal output in the DLS, we first trained mice to varying degrees of habit formation in an operant lever press task (Figure 1A). Adult mice were trained to press a lever for sucrose reward, first under continuous reinforcement (CRF) and then underwent random reinforcement schedules designed either to retain goal-directed behavior (random ratio, RR) or to induce habit formation (random interval, RI) (Dickinson et al., 1983; Hilário et al., 2007) (Figures 1A and 1E). A defining feature of habitual behavior is its persistence when the value of the outcome is reduced (Dickinson, 1985). To evaluate habitual behavior, mice were pre-fed ad libitum with sucrose pellets to devalue the outcome specifically associated with the lever press just before a 3-minute extinction “probe” test in which no reward was delivered. To control for general satiety, mice were pre-fed grain pellets prior to the same test on a different day (Figure 1A). To minimize confounds from the probe test experience, its duration was set similar to periods during which mice occasionally received no reward during random reinforcement training (3-minute no-reward bouts per session: RR = 1.44 ± 0.26, n = 17; RI = 1.36 ± 0.43, n = 14). To minimize any effects from the gap in daily training during probe tests, mice were re-exposed to an additional lever press training session after devaluation testing and prior to ex vivo analysis of DLS circuit properties. Habitual behavior was quantified in individual animals by comparing devalued LPr (pre-fed sucrose) to non-devalued LPr (pre-fed grain). This metric, referred to here as normalized devalued LPr, represents the tendency for an animal to persist in reward-contingent behavior despite devaluation of reward and is expressed as log2(devalued LPrnon-devalued LPr).

Figure 1. Simultaneous optical evaluation of DLS output pathways in operantly trained mice.

Figure 1

Open in a new tab

(A) Schematic of behavioral design for producing habitual (RI) and goal-directed (RR) lever pressing behavior. (B) Illustration of the parasaggital brain section used for imaging experiments with stimulating electrode placed at the corpus callosum. Red dashed box indicates typical region from which data were acquired. (C) Typical 410 × 410 µm field of view containing overlay of fura-2-loaded cells (green) and dSPNs expressing tdTomato (red). Inset: magnified view containing 4 cells (dSPNs indicated by white arrows, two remaining are putative iSPNs). (D) Pathway-specific analysis of SPN event amplitude in real X–Y space. (E) Learning curves produced by RI (n = 14) and RR (n = 17) training protocols. (F) Normalized devalued LPr measured during probe tests (*p < 0.05). Data are represented as mean ± SEM.

As expected, RI reinforcement induced habit formation more effectively than RR reinforcement (t(28.81 = 2.21, p < 0.05) (Figure 1F). Although RR subjects pressed at a higher rate than RI subjects on the final training session prior to devaluation testing (t(23.90) = 4.5454, p < 0.001), subjects’ learned rates (measured on R6, Figure 1E) were unrelated to their degrees of habitual behavior (r(29) = −0.12, p > 0.05, n = 31). This result shows that the measure of normalized devalued LPr for habit is not influenced by general lever pressing rates.

Although RI subjects were more habitual than RR subjects on average, there was considerable overlap between groups for individual subject behavior (Figure 1F). We took advantage of this overlap by merging RI and RR groups to form a continuous spectrum of habitual responding. We reasoned that if local DLS circuit plasticity significantly contributed to habitual behavior, then changes in the DLS response to cortical activity should correlate with quantitative behavioral measures of habit. To investigate this hypothesis, we evaluated the relationship between habitual lever pressing and ex vivo properties of cortically-evoked SPN firing in individual subjects.

Action potential activity was imaged in dozens of SPNs per animal with defined cell type using 2PLSM, calcium indicator dye fura-2, AM, and BAC transgenic Drd1a-tdTomato reporter mice (RRID: IMSR_JAX:016204) (Ade et al., 2011) (Figure 1B–1D). By imaging with a calcium indicator that fluoresces at resting calcium levels to identify all loaded cells and by taking advantage of the high fidelity of the Drd1a-tdTomato reporter to assign SPN types (Ade et al., 2011), we were able to evaluate both pathways simultaneously in response to the same stimulus. Calibration experiments indicated single spike sensitivity (Figures S1A–S1C), allowing us to evaluate both spike probability (% of trials with event detected) and spike number (event amplitude) (Figures S1F and S1G).

Strengthened DLS output and a shift in latency to fire between pathways predicts habitual behavior

We observed strong, positive correlations between the amplitude of stimulus-evoked SPN calcium transients and the degree to which mice were habitual. This was true in both dSPNs and iSPNs (Figures 2A and 2B). In contrast, SPN spike probability did not correlate with habitual behavior (Figures 2D–2F). Because event amplitude corresponds to the number of action potentials underlying a calcium transient (Figure S1F and S1G), and because spike probability did not predict habitual behavior, our data suggest that habit is encoded in the DLS as an integrated readout of burst firing rather than a binary code of whether or not SPNs fire at all. Finally, comparison of data from slices from the least and most habitual mice revealed the surprising finding that habit-related differences in event amplitude were broadly distributed spatially throughout the imaged populations, rather than in discrete clusters (Figures 2J and 2K).

Figure 2. Pathway-specific striatal substrates for habit formation in a lever press task.

Figure 2

Open in a new tab

(A–C) Linear regressions of mean event amplitude against normalized devalued LPr for dSPNs (A), iSPNs (B), and log2-transformed dSPN/iSPN ratio (C). (D–F) Relationship between SPN spike probability and normalized devalued LPr as in A–C. (G–I) Relationship between SPN event latency and normalized LPr as in A–C. (J–K) Heat maps of dSPN (X) and iSPN (●) event amplitudes in X–Y space of representative brain slice from least (J) and most (K) habitual mice. n = 15 animals; 24–88 dSPNs and 13–63 iSPNs per animal (*p < 0.05, **p < 0.01). Data are represented as mean ± SEM for all cells imaged from an animal. See also Supplemental Figures 2 and 3.

While increased event amplitude predicted habit expression in SPNs of both pathways, the balance of activity between dSPNs and iSPNs as measured by event amplitude was unrelated (Figure 2C). In contrast, we found that habitual behavior strongly correlated with a shift in the relative timing between dSPNs and iSPNs. Specifically, dSPNs tended to fire before iSPNs in habitual mice and iSPNs fired before dSPNs in goal-directed mice (Figure 2I). Absolute latency to fire did not correlate with habitual behavior for either SPN type; only relative timing (Figures 2G and 2H). Thus, we found evidence that pathway imbalance contributes to habitual behavior but in the unexpected domain of timing.

We next examined whether factors other than habit could similarly explain our findings. As mentioned, normalized devalued LPr isolates the behavioral insensitivity to outcome devaluation from absolute rates of lever pressing. Therefore, we predicted that lever press rates alone should not drive the observed correlations. Indeed, a correlational analysis using LPr instead of normalized devalued LPr failed to find relationships with any measures of DLS corticostriatal output (Figure S2A–S2I). Consistently, lever press rates did not correlate with degree of habitual behavior either (r(29) = −0.14, p > 0.05, n = 31). To further test whether lever press rates alone were sufficient to drive the correlations that we identified, we trained a separate cohort of mice according to the same RR and RI schedules but without exposure to devaluation testing or subsequent reinstatement training (Figure S3A and S3B). No correlations were found between learned LPr and DLS corticostriatal output (Figure S3C–S3K).

Lastly, although within-subject variability of SPN firing properties was not factored into correlational analyses, it was notable that subjects with the greatest cell-to-cell variability also had the poorest fits to the regression lines in Figures 2A, 2B, and 2I (Figure S2J–S2L). This suggests that experimental variability in some of our calcium imaging experiments, rather than other biological determinants of habit, may have reduced the strength of our correlations and that SPN firing properties may predict habitual behavior even more strongly than our data indicate.

Weakening of direct pathway output predicts suppression of habitual behavior

We next sought to identify the DLS circuit properties associated with learning to suppress a habit. To do this, we first induced habitual lever pressing in all subjects by increasing the total number of RI and RR training sessions (extended training groups RIext and RRext, Figure 3A). With enough repetition, an RR schedule can also induce habitual responding (Dickinson, 1985). To control for features related to learning to suppress lever pressing independent of habit, a third group was trained under conditions optimized for goal-directed responding (shortened training group RRshort, Figure 3A). LPr was then reduced by omission training in which each lever press resulted in temporary withholding of an otherwise periodically-delivered reward (Figure 3A). This paradigm trains the animal to suppress lever pressing in order to receive a reward. After omission training, lever pressing behavior was reinstated to isolate learned suppression from any acute effects related to decreased lever press activity (Figure 3A). Suppression of lever pressing was quantified as the percent decrease in LPr relative to the last random reinforcement session before omission training began (% LPr omitted). RRext (n = 5) and RIext (n = 6) groups had no significant differences in omission behavior (F(1,9) = 1.29, p > 0.05) and, consistent with habitual responding (Rossi and Yin, 2012), were significantly more resistant to omission training compared to the RRshort group (n = 8) (F(1, 17) = 24.38, p < 0.001) (Figure 3B).

Figure 3. Pathway-specific striatal substrates for suppressing habitual lever pressing.

Figure 3

Open in a new tab

(A) Schematic of behavioral design for producing habitual (RIext, RRext) and goal-directed (RRshort) lever press behavior and subsequently omitting the same behavior. (B) Percent LPr omitted on each day of omission testing. (C–F) Linear regressions of dSPN event amplitude (C), iSPN event amplitude (D), dSPN spike probability (E), and log2-transformed dSPN/iSPN ratio for event latency (F) against LPr omitted in final omission test. (G–H) Heat maps of dSPN (X) and iSPN (●) event amplitudes in X–Y space of representative brain slices from habitual mice with the poorest (G) and best (H) lever press suppression in final omission test. n = 9 animals; 22–116 dSPNs and 15–83 iSPNs per animal (*p < 0.05). Data are represented as mean ± SEM for all cells imaged from an animal. See also Supplemental Figure 4.

We found a strong negative correlation between suppression of lever pressing in the final omission session and dSPN event amplitude and spike probability. This was true of the combined RIext/RRext cohort (Figures 3C, 3E, 3G, and 3H) but not the RRshort group (dSPN amplitude r(5) = 0.19, p = 0.69, n = 7; dSPN spike probability r(5)= 0.13, p = 0.78, n = 7). Notably, other habit-predictive features did not correlate with suppression in either the habitual cohort (RIext/RRext) (Figures 3D and 3F) or the goal-directed cohort (RRshort) (iSPN amplitude r(5) = −0.12, p = 0.80, n = 7; dSPN/iSPN latency ration r(5) = −0.36, p = 0.43, n = 7). No correlations with pathway balance were observed for any of the measured SPN firing properties (Figures 3F, S4D, and S4E). Lastly, because the correlations were not shared by the goal-directed group, decreased direct pathway output in the DLS is specific for suppression of habitual behavior.

DISCUSSION

Here we have found that changes in local DLS processing of incoming cortical activity to generate projection neuron firing correlate with habitual behavior. While other brain regions are likely to contribute behaviorally relevant information, the fact that DLS corticostriatal input-output features alone were sufficient to strongly predict habitual behavior shows that this brain region plays a crucial role in the expression of habitual behavior. Additionally, our results suggest for the first time that habits may be expressed and suppressed through separate, pathway-specific processes. It is notable that, despite some differences between the substrates for habit expression and suppression, a general circuit logic favoring action unifies our findings.

Taken together, our findings support a model in which habit expression is characterized by an increased gain in both DLS output pathways and a shift in their relative timing such that firing of dSPNs precedes that of iSPNs. On the other hand, habit suppression correlates solely with reduced dSPN firing (Figure 4A). Notably, dSPN event amplitude predicted habit suppression in a near-perfect inverse of its correlation with habit expression (mexpression = 1.1 & msuppression = −0.96; Figure 4B), whereas iSPN event amplitude predicted habit expression but was surprisingly unrelated to suppression (Figure 4C). These data suggest that habits are not broken by a simple undoing of the formative process but through different mechanisms. Such a view is consistent with the finding that motor memory reversal can occur through new learning processes that are separate from those underlying formation of the same memory (Boyden and Raymond, 2003).

Figure 4. Distinct DLS plasticity mechanisms for expression and suppression of a habit.

Figure 4

Open in a new tab

(A) Diagram showing parallel increase in gain of dSPNs and iSPNs and timing reconfiguration with habit followed by a dampening of dSPN firing when the same habit is broken. (B–C) Juxtaposition of roles for dSPNs (B) and iSPNs (C) in habit expression and suppression plotted against successful target behavior. Normalized devalued LPr and % Omission datasets were adjusted to zero minima and then normalized to their respective maxima to yield the normalized success metric: data+abs(min(data))max(data+abs(min(data))). Data are represented as mean ± SEM for all cells imaged from an animal.

Our finding that an increase in gain of both types of SPNs correlates with habitual lever pressing is in accord with recent in vivo observations that a transient increase in activity of both pathways is associated with action initiation (Cui et al., 2013). Similarly supportive is the observation that synaptic strength of both dSPNs and iSPNs is increased in rotarod-trained mice (Yin et al., 2009). Interestingly, an increase of corticostriatal gain has been hypothesized to hasten the onset of action at the cost of outcome errors, as predicted by a model for increased corticostriatal efficacy in a visuomotor discrimination task (Lo and Wang, 2006). Such a mechanism would be consistent with our findings and could potentially explain the relative insensitivity of habitual behavior to outcome value.

Whereas a symmetrically heightened event amplitude was observed in SPNs of both pathways, we found evidence for a timing imbalance between the direct and indirect pathways. Specifically, dSPNs tended to fire before iSPNs in the most habitual mice whereas, in the most goal-directed mice, iSPNs tended to fire before dSPNs (Figure 2I). This somewhat unexpected finding is in accord with recent evidence for a timing competition between the direct and indirect basal ganglia pathways as a mechanism for action cancellation (Schmidt et al., 2013). In that study, Schmidt and colleagues showed that, to successfully cancel an action, the subthalamic nucleus (STN) must excite the substantia nigra pars reticulata (SNr) before striatal dSPNs can inhibit the SNr response to STN input. Here we find evidence that long-lasting, local DLS plasticity might contribute to such an overall basal ganglia circuitry timing competition. Specifically, a short dSPN latency to fire relative to iSPN latency could allow dSPNs to preempt stop-related SNr activity that is driven by iSPNs through a globus pallidus externus-mediated disinhibition of the STN, thereby reducing the probability of action cancellation.

Additionally, a systematic difference in the timing of firing between SPN types is not ostensibly explained by a model in which dSPNs promote one action and iSPNs inhibit other actions; such a model would seem to necessitate parallel processing of competing action plans rather than a competition between pathways. Rather, the timing advantage of the direct pathway SPNs in habit is more readily explained by a model in which expression of a given learned behavior would be promoted in habit by biasing the relative timing between output pathways to favor the direct pathway. Interestingly, we found evidence to support a correlation between habit and relative, but not absolute, latencies to fire (Figure 2G–I). We suspect that this distinction relates to the additional experimental variability that is imposed by electrode placement in brain slices and highlights an advantage of simultaneously monitoring direct and indirect pathway SPNs. Elucidating the underlying cellular and microcircuit mechanisms for such timing differences is an exciting future direction that could suggest novel pharmacological targets to manipulate habit.

Similar to our findings for timing in habit expression, we found that the DLS substrate for suppressing a habit was pathway-specific. Although at first estimation one might hypothesize a role for an increase in indirect pathway activity with action suppression, our finding that weaker direct pathway activity positively correlated with habit suppression is also consistent with decreased drive for voluntary action (Cui et al., 2013; Sippy et al., 2015; reviewed in Calabresi et al., 2014 and Nelson and Kreitzer, 2014) or for action in general (Albin et al., 1989; DeLong, 1990). It was somewhat unexpected that DLS substrates for suppression of lever pressing manifested in habitual but not goal-directed subjects. This result indicates that mechanisms for suppressing a habitual and goal-directed behavior are distinct. However, because goal-directed behaviors are typically associated with the dorsomedial striatum (DMS) (Yin et al., 2009; Yin et al., 2005), we suspect that similar mechanisms in principle might occur here. In support of this possibility, synaptic modifications have been found in the DMS of goal-directed mice (Shan et al., 2014; Yin et al., 2009).

Likewise, while the design of our study allowed us to identify predictive DLS circuit substrates for habit expression and suppression, the particular formative processes that gave rise to these substrates represent an important area for future studies. For example, since our findings describe a behavioral continuum from goal-directed to habitual responding, it would be interesting to know whether acquisition of plasticity solely occurred in subjects that became habitual or whether there was also DLS circuit plasticity that shaped goal-directed responding. It has been proposed that action-outcome (goal-directed) and stimulus-response (habit) associations form in parallel through separate circuits where stimulus-response associations eventually gain control of behavior over time (Everitt and Robbins, 2005). If this were the case, then DLS output may begin to change well before a habit is noticeably expressed and plasticity underlying goal-directed behavior may persist separately as the behavior becomes more habitual.

Finally, perhaps most surprising is the observation that habit-related SPN firing properties were broadly distributed in space. Given that previous in vivo studies have identified discrete subsets of DLS neurons as task-related (Desrochers et al., 2015; Tang et al., 2007; Yin et al., 2009), one might expect habit-related plasticity to occur only in subsets of neurons, i.e. those SPNs related to the lever press task. Instead, our findings indicate that an interaction between the broad DLS circuit modifications we describe and specific action-related inputs may ultimately be necessary to generate the specificity of habit for some behaviors and not others. According to such a model, a broadly-distributed “habit engram” in the DLS could serve to bias an overall competition between the direct and indirect pathways toward action execution. If it is the case that patterns of incoming cortical activity represent specific actions and that the striatum broadly tunes how SPNs respond to such input, then one might predict that forming one habit would influence the likelihood that other actions become habitual. Indeed, the observations that alcoholics displaying compulsive drug-seeking behavior demonstrate an overreliance on habit learning for other behaviors (Sjoerds et al., 2013) and that repeated ethanol exposure leads to a generalized increase in DLS control over Pavlovian learning (DePoy et al., 2013) support such a possibility. Thus we conclude that, as opposed to a model in which the DLS alone encodes specific habitual actions, our data favor a model in which broadly distributed changes in the DLS circuitry alter the propagation of cortical activity related to specific actions through the basal ganglia to render behaviors habitual.

EXPERIMENTAL PROCEDURES

Detailed methods are in Supplemental Experimental Procedures. 2- to 4-month old hemizygous Drd1a-tdTomato line 6 BAC transgenic mice (Ade et al., 2011) in C57Bl/6 background were trained in Med Associates operant chambers and behavioral data were recorded by Med-PC-IV software. After training, acute brain slices were prepared using cutting and holding solutions that promote survival of brain tissue from adult and aging animals (Ting et al., 2014). Slices were bulk loaded with cell-permeable fura-2, AM. SPN action potential firing in response to electrical stimulation of cortical afferents was monitored using vector-mode 2PLSM (Nikolenko et al., 2007). Data were analyzed using automated custom MATLAB software. Experiments and data analysis were performed with experimenter blind to training schedule.

Supplementary Material

1

Acknowledgments

The authors thank S. Lisberger, M. Platt, R. Mooney, R. O’Brien and L. Glickfeld for their productive discussions and comments on the manuscript. The authors thank S. Raghavachari, L. Lim, A. Jain, D. Wei for assistance with design of custom software for image analysis, J. MacLean, R. Yuste, S. Pal, V. Nikolenko and J. Waters for imaging advice, and E. Gaidis for technical help performing lever-press training behavioral experiments. We gratefully acknowledge the following sources of funding: NS064577 (N.C.), ARRA supplement to NS064577 (N.C.), AA021075 (H.Y.), McKnight Foundation (N.C., H.Y.), GM008441-23 (J.O.), NS051156 (K.A.), The Brain and Behavior Foundation (K.A.), and the Ruth K. Broad Foundation (J.O.).

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

The authors declare no competing financial interests.

AUTHOR CONTRIBUTIONS

All authors contributed to the design of the experiments. J.O. conducted the experiments correlating striatal physiology to behavioral states, designed data analysis software and performed data analysis; K.A. developed the experimental methodology for 2PLSM imaging of SPN activity and performed the experiments to validate and calibrate optical reporting of action potentials; K.A., J.O., S.V. and M.P. developed calcium imaging analysis approach and software; T.S. conducted behavioral training; H.Y. consulted on all aspects of design of behavioral experiments; J.O. and N.C. wrote the manuscript.

REFERENCES

  1. Ade KK, Wan Y, Chen M, Gloss B, Calakos N. An Improved BAC Transgenic Fluorescent Reporter Line for Sensitive and Specific Identification of Striatonigral Medium Spiny Neurons. Front. Syst. Neurosci. 2011;5:32. doi: 10.3389/fnsys.2011.00032. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Albin RL, Young AB, Penney JB. The functional anatomy of basal gnaglia disorders. TINS. 1989;12:366–375. doi: 10.1016/0166-2236(89)90074-x. [DOI] [PubMed] [Google Scholar]
  3. Boyden ES, Raymond JL. Active Reversal of Motor Memories Reveals Rules Governing Memory Encoding. Neuron. 2003;39:1031–1042. doi: 10.1016/s0896-6273(03)00562-2. [DOI] [PubMed] [Google Scholar]
  4. Calabresi P, Picconi B, Tozzi A, Ghiglieri V, Di Filippo M. Direct and indirect pathways of basal ganglia: a critical reappraisal. Nat. Neurosci. 2014;17:1022–1030. doi: 10.1038/nn.3743. [DOI] [PubMed] [Google Scholar]
  5. Coutureau E, Killcross S. Inactivation of the infralimbic prefrontal cortex reinstates goal-directed responding in overtrained rats. Behav. Brain Res. 2003;146:167–174. doi: 10.1016/j.bbr.2003.09.025. [DOI] [PubMed] [Google Scholar]
  6. Cui G, Jun SB, Jin X, Pham MD, Vogel SS, Lovinger DM, Costa RM. Concurrent activation of striatal direct and indirect pathways during action initiation. Nature. 2013;494:238–242. doi: 10.1038/nature11846. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. DeLong MR. Primate models of movement disorders of basal ganglia origin. TINS. 1990;13:281–285. doi: 10.1016/0166-2236(90)90110-v. [DOI] [PubMed] [Google Scholar]
  8. DePoy L, Daut R, Brigman JL, MacPherson K, Crowley N, Gunduz-Cinar O, Pickens CL, Cinar R, Saksida LM, Kunos G, et al. Chronic alcohol produces neuroadaptations to prime dorsal striatal learning. PNAS. 2013;110:14783–14788. doi: 10.1073/pnas.1308198110. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Desrochers TM, Amemori K-i, Graybiel AM. Habit learning by naive macaques is marked by response sharpening of striatal neurons representing the cost and outcome of acquired action sequences. Neuron. 2015;87:853–868. doi: 10.1016/j.neuron.2015.07.019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Dickinson A. Actions and Habits: The Development of Behavioural Autonomy. Phil. Trans. R. Soc. B: Bio. Sci. 1985;308:67–78. [Google Scholar]
  11. Dickinson A, Nicholas DJ, Adams CD. The effect of the instrumental training contingency on suceptibility to reinforcer devaluation. Q. J. Exp. Psychol. 1983;35:35–51. [Google Scholar]
  12. Everitt BJ, Robbins TW. Neural systems of reinforcement for drug addiction: from actions to habits to compulsion. Nat. Neurosci. 2005;8:1481–1489. doi: 10.1038/nn1579. [DOI] [PubMed] [Google Scholar]
  13. Furlong TM, Jayaweera HK, Balleine BW, Corbit LH. Binge-like consumption of a palatable food accelerates habitual control of behavior and is dependent on activation of the dorsolateral striatum. J. Neurosci. 2014;34:5012–5022. doi: 10.1523/JNEUROSCI.3707-13.2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Gerdeman GL, Lovinger DM. It could be habit forming: drugs of abuse and striatal synaptic plasticity. TINS. 2003;26:184–192. doi: 10.1016/S0166-2236(03)00065-1. [DOI] [PubMed] [Google Scholar]
  15. Graybiel AM. Habits, Rituals, and the Evaluative Brain. Annu. Rev. Neurosci. 2008;31:359–387. doi: 10.1146/annurev.neuro.29.051605.112851. [DOI] [PubMed] [Google Scholar]
  16. Gremel CM, Costa RM. Orbitofrontal and striatal circuits dynamically encode the shift between goal-directed and habitual actions. Nat. Commun. 2013;4:1. doi: 10.1038/ncomms3264. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Hilário MRF, Clouse E, Yin HH, Costa RM. Endocannabinoid signaling is critical for habit formation. Front. Integr. Neurosci. 2007;1:6. doi: 10.3389/neuro.07.006.2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Kravitz AV, Freeze BS, Parker PRL, Kay K, Thwin MT, Deisseroth K, Kreitzer AC. Regulation of parkinsonian motor behaviours by optogenetic control of basal ganglia circuitry. Nature. 2010;466:622–626. doi: 10.1038/nature09159. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Kravitz AV, Tye LD, Kreitzer AC. Distinct roles for direct and indirect pathway striatal neurons in reinforcement. Nat. Neurosci. 2012;15:816–818. doi: 10.1038/nn.3100. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Lo C-C, Wang X-J. Cortico–basal ganglia circuit mechanism for a decision threshold in reaction time tasks. Nat. Neurosci. 2006;9:956–963. doi: 10.1038/nn1722. [DOI] [PubMed] [Google Scholar]
  21. Nazzaro C, Greco B, Cerovic M, Baxter P, Rubino T, Trusel M, Parolaro D, Tkatch T, Benfenati F, Pedarzani P, et al. SK channel modulation rescues striatal plasticity and control over habit in cannabinoid tolerance. Nat. Neurosci. 2012;15:284–293. doi: 10.1038/nn.3022. [DOI] [PubMed] [Google Scholar]
  22. Nelson AB, Kreitzer AC. Reassessing models of Basal Ganglia function and dysfunction. Annu. Rev. Neurosci. 2014;37:117–135. doi: 10.1146/annurev-neuro-071013-013916. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Nikolenko V, Poskanzer KE, Yuste R. Two-photon photostimulation and imaging of neural circuits. Nat. Methods. 2007;4:943–950. doi: 10.1038/nmeth1105. [DOI] [PubMed] [Google Scholar]
  24. Rossi MA, Yin HH. Methods for studying habitual behavior in mice. Curr. Protoc. Neurosci / editorial board, Jacqueline N. Crawley … [et al.] 2012;CHAPTER(Unit8.29–Unit28.29) doi: 10.1002/0471142301.ns0829s60. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Rueda-Orozco PE, Robbe D. The striatum multiplexes contextual and kinematic information to constrain motor habits execution. Nat. Neurosci. 2015;18:453–460. doi: 10.1038/nn.3924. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Schmidt R, Leventhal DK, Mallet N, Chen F, Berke JD. Canceling actions involves a race between basal ganglia pathways. Nat. Neurosci. 2013;16:1118–1124. doi: 10.1038/nn.3456. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Shan Q, Christie MJ, Balleine BW. Plasticity in striatopallidal projection neurons mediates the acquisition of habitual actions. Eur. J. Neurosci. 2015;42:2097–2004. doi: 10.1111/ejn.12971. [DOI] [PubMed] [Google Scholar]
  28. Shan Q, Ge M, Christie MJ, Balleine BW. The acquisition of goal-directed actions generates opposing plasticity in direct and indirect pathways in dorsomedial striatum. J. Neurosci. 2014;34:9196–9101. doi: 10.1523/JNEUROSCI.0313-14.2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Sippy T, Lapray D, Crochet S, Petersen CC. Cell-Type-Specific Sensorimotor Processing in Striatal Projection Neurons during Goal-Directed Behavior. Neuron. 2015;88:298–205. doi: 10.1016/j.neuron.2015.08.039. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Sjoerds Z, de Wit S, van den Brink W, Robbins TW, Beekman ATF, Penninx BWJH, Veltman DJ. Behavioral and neuroimaging evidence for overreliance on habit learning in alcohol-dependent patients. Transl. Psychiatry. 2013;3:e337. doi: 10.1038/tp.2013.107. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Smetters D, Majewska A, Yuste R. Detecting action potentials in neuronal populations with calcium imaging. Methods. 1999;18:215–221. doi: 10.1006/meth.1999.0774. [DOI] [PubMed] [Google Scholar]
  32. Tang C, Pawlak AP, Prokopenko V, West MO. Changes in activity of the striatum during formation of a motor habit. Eur. J. Neurosci. 2007;25:1212–1227. doi: 10.1111/j.1460-9568.2007.05353.x. [DOI] [PubMed] [Google Scholar]
  33. Tecuapetla F, Matias S, Dugue GP, Mainen ZF, Costa RM. Balanced activity in basal ganglia projection pathways is critical for contraversive movements. Nat. Commun. 2014;5 doi: 10.1038/ncomms5315. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Ting JT, Daigle TL, Chen Q, Feng G. Acute brain slice methods for adult and aging animals: application of targeted patch clampanalysis and optogenetics. Methods Mol. Biol. 2014;1183:221–242. doi: 10.1007/978-1-4939-1096-0_14. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Yin H, Knowlton B, Balleine B. Inactivation of dorsolateral striatum enhances sensitivity to changes in the action–outcome contingency in instrumental conditioning. Behav. Brain Res. 2006;166:189–196. doi: 10.1016/j.bbr.2005.07.012. [DOI] [PubMed] [Google Scholar]
  36. Yin HH, Balleine B. Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning. Eur. J. Neurosci. 2004;19:181–189. doi: 10.1111/j.1460-9568.2004.03095.x. [DOI] [PubMed] [Google Scholar]
  37. Yin HH, Knowlton BJ. The role of the basal ganglia in habit formation. Nat. Rev. Neurosci. 2006;7:464–476. doi: 10.1038/nrn1919. [DOI] [PubMed] [Google Scholar]
  38. Yin HH, Mulcare SP, Hilário MRF, Clouse E, Holloway T, Davis MI, Hansson AC, Lovinger DM, Costa RM. Dynamic reorganization of striatal circuits during the acquisition and consolidation of a skill. Nat. Neurosci. 2009;12:333–341. doi: 10.1038/nn.2261. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Yin HH, Ostlund SB, Knowlton BJ, Balleine BW. The role of the dorsomedial striatum in instrumental conditioning. Eur. J. Neurosci. 2005;22:513–523. doi: 10.1111/j.1460-9568.2005.04218.x. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1
NIHMS753257-supplement-1.pdf (876KB, pdf)

RESOURCES