Emergence of stable striatal D1R and D2R neuronal ensembles with distinct firing sequence during motor learning

Significance

The striatum is a brain region critical for motor and procedure learning. The distinct roles of the two neural pathways through the striatum, known as the direct and indirect pathways, remain to be clarified. Using two-photon calcium imaging of neuronal activity of the same population of dorsolateral striatum (DLS) neurons in behaving mice throughout the learning of a cue-triggered motor task, we discovered that DLS neuronal populations in these two pathways developed stable sequential firing patterns of a distinct time course after motor learning, with direct pathway neurons involved in the initiation of the learned motor action and indirect pathways responsible for suppressing erroneous motor actions. Thus, both pathways are required for motor learning and proper execution of learned motor behaviors.

Abstract

The dorsolateral striatum (DLS) is essential for motor and procedure learning, but the role of DLS spiny projection neurons (SPNs) of direct and indirect pathways, as marked, respectively, by D1 and D2 receptor (D1R and D2R) expression, remains to be clarified. Long-term two-photon calcium imaging of the same neuronal population during mouse learning of a cued lever-pushing task revealed a gradual emergence of distinct D1R and D2R neuronal ensembles that reproducibly fired in a sequential manner, with more D1R and D2R neurons fired during the lever-pushing period and intertrial intervals (ITIs), respectively. This sequential firing pattern was specifically associated with the learned motor behavior, because it changed markedly when the trained mice performed other cued motor tasks. Selective chemogenetic silencing of D1R and D2R neurons impaired the initiation of learned motor action and suppression of erroneous lever pushing during ITIs, respectively. Thus, motor learning involves reorganization of DLS neuronal activity, forming stable D1R and D2R neuronal ensembles that fired sequentially to regulate different aspects of the learned behavior.

The dorsolateral striatum (DLS) plays an essential role in motor and procedure learning (1–4). The learning process could be divided into cognitive, associative, and autonomous stages (5). In such a scheme, learning begins with a rapid improvement in performance, followed by gradual refinement, until the motor skills become consolidated into long-lasting autonomous actions (6–8). The DLS integrates information from the motor cortex, thalamus, and substantia nigra pars compacta during motor action, motor learning, and habit formation. Studies using progressive depletion of dopamine neurons in mice showed that neurons in the dorsal striatum represent movement vigor (9). Electrophysiogical recording from the striatum while the rat was performing a well-trained treadmill-running task also indicates that DLS neurons may code for the running speed, position, and timing of motor actions (10). During different phases of the mouse’s learning of a rotarod task, region-specific changes in dorsal striatal activity have been observed (11). In rats that had learned to perform a T-maze task, there were widely distributed changes in activity patterns of DLS neurons, suggesting the role of the DLS in building a neural representation of habit (12). The striatum consists of mainly two types of spiny projection neurons (SPNs) that differ in their axon projection patterns. The SPNs of the direct pathway express the D1 dopamine receptor (D1R) and send projections to the internal segment of the globus pallidus (GPi) and substantia nigra pars reticulata (SNr), whereas those of the indirect pathway express the D2 dopamine receptor (D2R) and project to the external segment of the globus pallidus (GPe) (13, 14). The traditional rate model suggests that activation of the direct pathway promotes movement, while that of the indirect pathway inhibits movement (15–18). However, more recent studies indicate that coordinated activity of both pathways is involved in the initiation and execution of movements (19–23).

Many issues concerning the functions of D1R and D2R neurons in the DLS remain to be addressed. For example, it is unclear how motor learning affects the activity of SPNs in the DLS and whether D1R and D2R neurons differ in their activity and function during learning. In vivo two-photon calcium imaging has been used to monitor the activity of large populations of cortical neurons in awake-behaving mice (24) and to examine changes of neuronal activity during motor learning (25). In the primary motor cortex, reproducible sequential firing activity emerged during mouse motor learning (26). In the present study, we conducted deep-brain two-photon calcium imaging of the same populations of striatal neurons during mouse learning of a cued lever-pushing task. Using D1- and D2-Cre mice injected with a viral vector expressing Cre-dependent GCaMP6s (27), we monitored the activity of D1R and D2R neuronal populations at a cellular resolution throughout the entire duration of the motor learning task (1–3 wk). We found that the pattern of activity of both D1R and D2R neurons in the DLS underwent marked changes during motor learning, with a gradual emergence of spatially dispersed neuronal ensembles that reproducibly fired in a sequential manner. Moreover, using a chemogenetic method to selectively suppress D1R and D2R neuronal activity after motor learning, we found that silencing D1R neurons impaired the initiation of lever pushing, whereas silencing D2R neurons resulted in increased erroneous (uncued) lever pushing during intertrial intervals (ITIs). Importantly, silencing either D1R or D2R neurons affected the learned trajectory of lever pushing, suggesting that normal activity of both types of neurons was required for the mice to perform the stereotyped motor action. These results support the notion that motor learning involves the establishment of stable patterns of sequential D1R and D2R neuronal activity that encode different aspects of the learned motor behavior.

Results

Mouse Learning of a Cued Lever-Pushing Task.

After 3 d of habituation, head-fixed mice were trained to perform a lever-pushing task (26, 28) (Fig. 1A, SI Appendix, Materials and Methods, and Movies S1 and S2) that involves rightward pushing of a lever with the left forelimb within a fixed duration following the onset of a sound cue. Mice that pushed the lever over a set threshold immediately received a drop of water as a reward. Failure in pushing the lever over the threshold within the trial duration was punished with white noise. Uncued lever pushing during ITIs caused additional time out before the next trial. We found that the success rate (percentage of rewarded trials) gradually increased during early days of training and reached an average above 95% after 1–2 wk of training (Fig. 1B). The average reaction time (from cue onset to rewarded lever pushing) decreased with days of training (Fig. 1C). The average frequency of uncued pushing events during ITIs showed an initial increase, probably reflecting a mouse’s motivation to seek reward, followed by a decrease as the task rule was learned (Fig. 1D). Furthermore, the lever movement trajectory was irregular from trial to trial at the beginning of training, and became stereotyped after training (Fig. 1E). Changes in the movement regularity were quantified by calculating pairwise correlation coefficients (CCs) of lever-pushing trajectories for all trial pairs within the same day and between two different days. As shown by the data from one example mouse (Fig. 1F) and from all mice (Fig. 1 G and H), the average pairwise CCs gradually increased with days of learning. One example of video-recorded mouse behavior, together with corresponding measurements of left forelimb movement and the lever displacement, is shown in Movie S1. Given their close resemblance, we used the trajectory of lever displacement as a quantitative assay of the forelimb movement in the present study.

Fig. 1.

Mouse learning of a cued lever-pushing task. (A) Schematic diagram of the motor task. A head-fixed mouse was trained to push a lever in response to a sound cue within a fixed time window. Successful lever pushing was rewarded with a drop of water. (B) Success rate (percentage of rewarded trials) in cued lever pushing steadily increased to a plateau over days of training (r = 0.62, P < 0.001). (C) Reaction time (duration from the cue onset to the rewarded movement onset) decreased with days of training (r = −0.69, P < 0.001). (D) Average number of lever-pushing events in each trial during the ITI with days of training. The frequency of uncued pushing during ITIs showed an initial increase (days 1–4; r = 0.59, P < 0.001), followed by a decrease (days 5–12; r = −0.53, P < 0.001). (E) Example traces of the lever-pushing movement in one mouse on days 1 and 12 of training. (F) Pairwise correlation matrix of lever-pushing trajectories for all trial pairs within the same day and between two different days (data from one example mouse). The pairwise correlation of lever-pushing trajectories for all trial pairs within each day (G; r = 0.69, P < 0.001) and between adjacent days (H; r = 0.68, P < 0.001) is shown. In B–D, G, and H, gray lines depict the average of data of all trials from individual mice and the black line represents the average of data of all mice (B–D; n = 19 mice for days 1–7; n = 17, n = 11, n = 7, n = 7, and n = 6 mice for days 8–12, respectively; data from the same group of mice were used in G and H). Error bars represent SEM.

We next examined whether the activity of DLS neurons is involved in the lever-pushing task. By injecting the GABA receptor agonist muscimol into the DLS contralateral to the lever-pushing forelimb (SI Appendix, Fig. S1A), we found that the success rate was greatly reduced and reaction times were lengthened significantly (SI Appendix, Fig. S1 B and C). Thus, activity of DLS neurons is important for properly performing the learned motor action.

Chronic Recording of the Same Population of D1R and D2R Neurons.

To distinguish the activity of SPNs that express either D1Rs or D2Rs, we first verified the SPN projection patterns in D1-Cre and D2-Cre mice by crossing them with the Ai9 tdTomato Cre reporter line (29). As expected, SPNs of the direct pathway (D1R⁺/tdTomato⁺) selectively sent projections to the GPi and SNr (Fig. 2A), whereas those of the indirect pathway (D2R⁺/tdTomato⁺) specifically projected to the GPe (Fig. 2B). Thus, these D1-Cre and D2-Cre mice faithfully expressed Cre proteins in direct and indirect pathways, respectively. To perform two-photon in vivo imaging of DLS neurons, we implanted a cannula above the DLS of the right hemisphere through an oblique track to avoid damaging the motor cortex (Fig. 2C and SI Appendix, Materials and Methods). Using D1- and D2-Cre mice expressing Cre-dependent GCaMP6s in DLS neurons, this chronic window allowed us to record simultaneously the activity of the same population of a few hundred neurons repetitively over a period of weeks, during which behavioral events were also monitored. As shown in Fig. 2D for an example mouse, fluorescent images of the same population of D2R neurons on the first and last (seventh) days of training were identified. Changes of fluorescence intensities in 10 example neurons, reflecting neuronal spiking activity (27), and associated task-related events on day 7 are shown in Fig. 2E. We observed robust Ca²⁺ transients in some neurons as the mouse was performing the cued lever-pushing task.

Fig. 2.

Chronic recording of activity of the same populations of D1R and D2R neurons during learning of a cued motor task. (A) Sagittal brain section obtained from a D1-Cre mouse that was crossed with the Ai9 tdTomato Cre reporter line, showing that tdTomato-labeled D1R SPNs in the direct pathway selectively send projections to the GPi and SNr. (Scale bar, 1 mm.) (B) Sagittal brain section from a D2-Cre mouse that was crossed with the Ai9 tdTomato Cre reporter line, showing that D2R SPNs of the indirect pathway specifically project to the GPe. (Scale bar, 1 mm.) (C) Schematic diagram showing the location of the objective lens and cannula for chronic imaging of DLS neurons. (D) Example fluorescence images of DLS neurons expressing GCaMP6s in D2R neurons on day 1 and day 7 (last day) of training. (Scale bars, 100 μm.) (Insets) Magnification of area in red box. (Scale bars, 30 μm.) (E) Traces of relative Ca²⁺ transients (ΔF/F) for 10 sample neurons and simultaneous recordings of behavioral events on day 7 in an example D2-Cre mouse expressing GCaMP6s. Pink lines represent the lever movement zone. Black, lever movement (Move); red, Cue; blue, reward (Rewd). (Horizontal bar, 10 s; vertical bar, 5 ΔF/F).

We then examined the relationship between neuronal activity and various task events in consecutive single trials over a period of ∼20 s in each mouse during the first and last days of learning. As shown in the example mice in SI Appendix, Fig. S2 A and B, Upper, most responsive D1R and D2R neurons (definition provided in SI Appendix, Materials and Methods) recorded on day 1 showed activity roughly correlated in time with the lever pushing, but the set of neurons that were active exhibited trial-to-trial variation. However, on the last day of training (day 9 and day 7 for D1R and D2R, respectively), the firing patterns of these neurons became reproducible from trial to trial (SI Appendix, Fig. S2 A and B, Lower; a single trial at higher resolution is shown in SI Appendix, Fig. S2 C–J). This qualitative observation suggests that both D1R and D2R neurons underwent a task-associated reorganization of their activity into reproducible patterns during learning.

Emergence of Stable Sequentially Firing D1R and D2R Neuronal Ensembles.

To characterize the firing patterns of recorded D1R and D2R neuronal populations, we aligned the activity of all responsive neurons from all trials over the period from 1 s before (as baseline) to 3 s after the onset of the rewarded movement, averaged them for each neuron, and minimum/maximum-normalized and color-coded the average values. In the examples shown in Fig. 3 A and C, the same population of neurons was monitored throughout the learning period, together with corresponding changes in the success rate, average reaction time, and average number of lever-pushing events during ITIs (Fig. 3 B and D). Changes in the population activity profile were visualized by arranging the order of neurons according to the time of averaged peak activity of each cell on the last day of training. The resultant order of neurons was used to sort the same population of cells in all earlier days. Comparison of the activity patterns over the entire training period revealed that the sequential firing pattern gradually emerged and became reproducible during the late stage of training, for both D1R and D2R neurons. Interestingly, more D1R neurons appeared to fire during the lever-pushing period, whereas more D2R neurons fired during ITIs after the motor action. Furthermore, the neuronal ensembles that fired closely in time were not spatially clustered (SI Appendix, Fig. S3 A, B, D, and E). Quantitative analysis of the pairwise difference in the time of peak activity also showed no significant dependence on the distance between each cell pair, in both D1- and D2-Cre mice (SI Appendix, Fig. S3 C and F).

Fig. 3.

Emergence of reproducible sequential firing of D1R and D2R neurons. (A) Averaged activity of all responsive neurons (aligned by the movement onset time, black line) from day 1 to day 9, as shown by normalized (Norm.) ΔF/F of GCaMP6s fluorescence, and sorted according the order of the time of averaged peak activity on day 9 (D1R, 99 cells; 45, 60, 58, 91, 69, 103, 85, 102, and 75 trials for days 1–9, respectively). The activity covered the period from 1 s before to 3 s after the movement onset. (B) Corresponding behavioral performance of the example mouse in A, showing the success rate (SR; Top), reaction time (RT; Bottom, blue line), and number of lever-pushing events during ITIs (#P in ITI; Bottom, orange line) throughout the learning period. (C) Averaged activity of all responsive cells (D2R, 115 cells) from day 1 to day 7 (52, 24, 19, 39, 52, 73, and 90 trials for days 1–7, respectively), sorted by the order of the time of average peak activity on day 7. (D) Corresponding behavioral performance of the example mouse in C. (E) Similarity of activity patterns quantified by the CC of activity of the same population of neurons between adjacent days (early stage: between E1 and E2, E2, and E3; late stage: between L1 and L2, L2, and L3; D1-Cre, n = 20 mice; ***P < 0.001, *P < 0.05 by Wilcoxon rank sum test). Error bars represent SEM. (F) Similar analysis as in E, except that CCs were calculated by comparing the activity of each day with that of L3 (D1-Cre, n = 20mice; ***P < 0.001 by Wilcoxon rank sum test). (G and H) Similar analysis as in E and F for 20 D2-Cre mice. (***P < 0.001, *P < 0.05 by Wilcoxon rank sum test). (I) Ridge/Bgd analysis of the sequential firing. The population activity on E1 and L3 (sorted by the order of the time of average peak activity on L3) and shuffled data of L3 are shown, with the ridge area marked by the black lines. The ridge was defined as five frames (0.33 s, sampling rate: 15 frames per second) before and 15 frames (1 s) after the movement onset, and the remaining frames were taken as the background. The shuffled data were obtained using the data on day L3. (J and K) Ridge/Bgd ΔF/F ratio of each cell during the early and late stages was calculated as the mean ΔF/F value of the ridge frames divided by that of the background frames. The black horizontal line represents the average value for all cells in each mouse (D1-Cre, n = 20 mice; D2-Cre, n = 20 mice; ***P < 0.001 by Wilcoxon rank sum test). Error bars represent SEM. Sfd, shuffled data obtained using L3 data.

To further demonstrate that the stable sequential firing observed at the late stage of learning was not an artifact of sorting, we performed sorting of the same population of neurons in the same mice for all days according to the order determined by the alignment of firing activity on each of the first 3 d and a later day (day 6 for D1R and day 5 for D2R). We found that sequence firing based on alignment in the first 3 d disappeared on subsequent days, whereas that based on alignment in the later day persisted, in both D1- and D2-Cre mice (SI Appendix, Figs. S4 and S5). Notably, our results were not caused by differences in the total number of trials used on different days [as listed in SI Appendix, Table S1 (and the table legend)], since the same conclusion could be reached when the same total trial number was used (SI Appendix, Fig. S6).

In total, we have repeatedly imaged the activity of the same population of D1R and D2R neurons throughout the training of the lever-pushing task in 20 D1-Cre and 20 D2-Cre mice. There was substantial variability in the training duration, which was determined by having success rates >90%, average reaction times <1 s, and an average of fewer than two lever-pushing events during ITIs. To pool data from all mice, we defined the first three training days [early stage 1 (E1), E2, and E3] and the last three training days [late stage 1 (L1), L2, and L3] as the early and late stages of learning, respectively. To quantitatively analyze changes in the firing pattern of the same neuronal population, we measured the CCs of activity patterns (26) (SI Appendix, Materials and Methods) between two neighboring days and between each day and L3, and found that CCs at the late stage were significantly higher than those of the early stage for both D1R and D2R neurons (Fig. 3 E–H).

To further analyze the reproducibility of sequential activity patterns at the late stage of learning, we used a previously reported method (30) to delineate the outline of the sequence activity (“ridge”), and compared the activity within the ridge with that outside the ridge (“background”; Fig. 3I). The ridge-to-background (“Ridge/Bgd”) ΔF/F (relative change of the fluorescence intensity) ratio was first calculated for each cell and then averaged over all cells on each day. We found that these ratios for the last 3 d were significantly higher than those for the first 3 d, for both D1R and D2R neurons, in all mice examined (Fig. 3 J and K). When the activity of each cell was shuffled in time by a random amount and the cells were resorted by the time of the average peak activity, the shuffling led to a ridge along the diagonal (Fig. 3I), with Ridge/Bgd ΔF/F ratios close to 1 (Fig. 3 J and K). Thus, there was an increase in activity within the ridge on L3 and reduced background activity after learning. Furthermore, we also calculated Ridge/Bgd ΔF/F ratios based on sequence alignment of neuronal activity for each day to define its own ridge area rather than using that of L3 (SI Appendix, Fig. S7 A and C), and found that the ratio on L3 was still significantly higher than that on E1 (SI Appendix, Fig. S7 B and D). There was also a trend of gradual increase of the ratio during E1–E3, indicating a reorganization of neuronal firing pattern with increasing reduction of background activity (SI Appendix, Fig. S7 B and D).

Coemergence of Stable Sequential Firing and Stereotyped Movement.

To examine whether learning-induced stable sequential firing is associated with the selection of a stereotyped movement pattern, we used a previously reported method (26) to define the “learned movement pattern” and “learned activity pattern” by averaging the movement trajectories and neuronal activity patterns, respectively, for 50% of randomly chosen trials from L2 and L3. We then calculated the CCs between the learned movement pattern and the movement trajectory of each individual trial (referred to hereafter as “movement CC”) on E1 to E3 and L1 to L3 (excluding the 50% of trials on L2 and L3 chosen for defining learned movement pattern). Similarly, we also calculated the “activity CC” between the learned activity pattern and the activity pattern of all neurons for each individual trial on E1 to E3 and L1 to L3.

When the CC data for all individual trials were plotted (Fig. 4 A and C), we found that the percentage of movements on L1 to L3 with high movement CCs (0.8–1.0) was higher than that for E1 to E3, and those movements with high movement CCs were mostly associated with activity patterns with high activity CCs. On the other hand, the percentage of movements with low movement CCs (−0.2 to 0.6) during E1 to E3 was higher than that for L1 to L3, and neuronal activity patterns corresponding to these trials also showed low activity CCs. The changes in the relationship between activity patterns and movement trajectories were better visualized by grouping all trials with different levels of movement CCs (Fig. 4 B and D and SI Appendix, Fig. S8). We found that for movements with the same movement CCs at early and late stages, the corresponding activity CCs were significantly higher for the late-stage movements (Fig. 4 B and D). Notably, for movements with different movement CCs during the late stage (orange line in Fig. 4 B and D), the corresponding activity CCs were significantly higher for those with high movement CCs, in both D1R and D2R neurons. Taken together, these results support the notion that the mice were exploring many movement patterns during the early stage of learning, and neuronal firing patterns were not stable. The emergence of stable sequential activity patterns of the same neuronal population at the late stage is related to the selection of the stereotyped movement, leading to the high percentage of movements with high movement CCs.

Fig. 4.

Correlation analysis of changes in activity patterns and in movement trajectories. CCs between the single-trial activity pattern on each day and the learned activity pattern plotted against the CCs between the single-trial movement trajectory on each day and the learned movement pattern are shown for all D1-Cre mice (A, n = 20) and D2-Cre mice (C, n = 20). Curves illustrating the percentage distribution of movement CCs [% (Move)] (Top) and percentage distribution of activity CCs [% (Activity)] (Right) at early (E1–E3) and late (L1–L3) stages. Thin and thick lines represent the mean (averaged over 0.2 bins) for individual mice and for all mice, respectively. Note that the increased percentages of higher activity CCs coincided with increased percentages of higher movement CCs at the late stage of learning (***P < 0.001, **P < 0.01 by Wilcoxon rank sum test). Averaged activity CCs with different movement CCs (at 0.2 CC bins) plotted against the movement CCs at the early (blue) and late (orange) stages of experiments are shown for D1-Cre (B) and D2-Cre (D) mice. Significant differences were found among data points at the late stage (P < 0.001 by one-way ANOVA; ***P < 0.001, **P < 0.01 by Wilcoxon rank sum test; numbers of mice used are shown in parentheses) and between the early and late stages at the same bin of movement CCs (P < 0.001 in bins 1–6 for D1-Cre mice, P < 0.05 and P < 0.001 in bins 1 and 2–6 for D2-Cre mice by Wilcoxon rank sum test). Error bars represent SEM.

Relationship Between Neuronal Activity and Behavioral Events.

To further determine the relationship between SPN activity and various behavioral events, including cue perception, reward reception, lever pushing, and licking, we designed a dual-task experiment in which the mouse was subjected on the same day to two different tasks in tandem, with the cue and reward delivery either without or with the presence of the lever (Fig. 5A), while the same population of neurons was imaged for both tasks, on E1 to E3 and on L1 to L3. In the cue-reward task without lever pushing, the water reward was delivered 1–2.5 s after the onset of the cue; thus, we were able to identify neurons that responded after the cue onset (designated as “cue cells”) and those that responded after water reward delivery (designated as “reward cells”). Fig. 5 B and C depicts trial-by-trial activity of a cue cell and a reward cell among the D1R neurons on L3 from the same mouse. We found the percentages of cue cells showed a significant increase after learning among responsive D1R neurons but did not change among D2R neurons (Fig. 5 D and F). The percentages of reward cells among both D1R (Fig. 5E) and D2R (Fig. 5G) neurons did not show any significant change. Thus, among sequentially firing neuronal ensembles, only small percentages of cells were related to the cue and reward, and motor learning resulted in a significant increase in the percentage of cue cells among the D1R ensembles, consistent with involvement of D1R neurons in early sensory detection (31).

Fig. 5.

Relationship between D1R and D2R neuronal activity and various behavioral events. (A) Schematic diagram of the dual-task experiment. In the cue-reward without lever-pushing task, the lever was replaced with a platform for the mouse forelimb to rest on and the mouse received a water reward after a random delay period (within 1–2.5 s) following the auditory cue. On each day, the mouse first performed the cue-reward without lever-pushing task (for 30–40 trials) and then performed the standard cued lever-pushing task. The same populations of neurons were recorded during both tasks. (B) Example D1R neuron that responded to the cue, identified in the cue-reward without lever-pushing task. (Top) Heatmap of fluorescence transients of individual trials, aligned to cue onset. The red dashed line indicates cue onset, and white lines indicate reward delivery time. (Bottom) ΔF/F for individual trials (gray lines) and the average (black line) of all trials for the example neuron above in one session. (C) Example D1R neuron that responded to reward delivery. The red dashed line indicates cue onset, and white lines indicate reward delivery time. The heatmap and ΔF/F traces were plotted in the same manner as in B. (D and F) Percentages of cue cells among all responsive neurons observed in cued lever-pushing task during the early and late stages for D1-Cre and D2-Cre mice (D1, n = 5; D2, n = 9; **P < 0.01 by Wilcoxon rank sum test). (E and G) Percentages of reward cells among all responsive neurons observed in the cued lever-pushing task during the early and late stages (D1, n = 5 mice; D2, n = 9 mice; Wilcoxon rank sum test). Example D1R (H and I) and D2R (J and K) neurons that responded during the lever-pushing movement (M cell) and after lever pushing (PM cell) are shown. The red dashed line indicates movement onset. The heatmap and ΔF/F traces were plotted in the same manner as in B. Percentages of neurons responding during the lever-pushing period (M cell) and postmovement period (PM cell) on E1–E3 and L1–L3 for D1-Cre mice (L) and D2-Cre mice (M) are shown. Significant differences were found between M and PM cells at the late stage for D1-Cre mice and at the early stage for D2-Cre mice (D1, n = 5 mice; D2, n = 9 mice; *P < 0.05, **P < 0.01, ***P < 0.001 by Wilcoxon rank sum test). Averages of the percentages of M and PM cells for all 3 d at the early stage (N, E1–E3) and late stage (O, L1–L3) are shown. The Wilcoxon rank sum test was used as the statistical test. Error bars represent SEM.

By subtracting the cue cells and reward cells from all responsive neurons observed in the cued lever-pushing task, we identified neurons that were associated with the lever-pushing action. The latter were further classified into two groups: those fired during the lever-pushing period (i.e., movement-related “M cells”) and those fired after the pushing action (i.e., postmovement “PM cells”) (SI Appendix, Materials and Methods). Fig. 5 H–K depicts trial-by-trial activity of an M cell and a PM cell among the D1R and D2R ensembles. Analysis of the percentages of M cells and PM cells during early-stage and late-stage learning (E1–E3 and L1–L3) revealed that among D1R neurons, M cells increased and PM cells decreased after learning. Furthermore, the percentages of M cells and PM cells were similar on E1–E3, but percentages of M cells became markedly higher than those of PM cells on L1–L3 (Fig. 5L). For D2R neurons, there was a gradual decrease of PM cells and increase of M cells during E1–E3, and the two groups became similar in percentages on L1–L3 (Fig. 5M). These results indicate that motor learning is accompanied by a reorganization of DLS neuronal activity, with the increased representation of the lever-pushing movement and the decreased representation of postmovement events in both D1R and D2R neurons. However, there were significantly higher percentages of D1R M cells and higher percentages of D2R PM cells after learning. These learning-induced changes were better visualized by the bar graphs that depict the average percentages of M and PM cells in the early and late stages (Fig. 5 N and O).

To further explore the function of PM cell activity after the lever-pushing action, we examined the possibility that these cells are related to licking, which often occurred for a prolonged period. We plotted the lick rate of individual trials vs. peak activity amplitude and the total activity (integrated ΔF/F) for all PM cells among D1R and D2R neurons during ITIs and found no significant correlation between PM cell activity and lick rates (SI Appendix, Fig. S9). Thus, the neuronal activity we observed in the DLS is unlikely to be related to licking events.

Stable Sequential Firing Is Specifically Associated with the Learned Motor Task.

To further examine whether the sequential firing of DLS neurons is specifically associated with the learned motor task, we subjected the mice that had learned the cued rightward lever pushing to three different motor tasks on subsequent days. The first two tasks were leftward lever pushing and downward lever pressing in response to the same sound cue (SI Appendix, Materials and Methods). In these two new tasks, the lever was designed to move only leftward and downward, respectively (Fig. 6A), and the mouse needs to use the left forelimb to move the lever beyond a threshold distance to receive the water reward (Movies S3 and S4). Most mice that had learned the rightward lever-pushing task could perform either new task with high success rates, but the reaction times were longer and motor events during ITIs were more frequent than those associated with the learned rightward lever-pushing task (SI Appendix, Fig. S10). Neuronal ensemble activity of all rewarded trials during the last 2 d of rightward lever pushing (L2, L3, sorted on L3) and two subsequent days of performing the new tasks [new task 1 (N1), leftward lever pushing; N2, downward lever pressing; sorting based on the order on L3] is shown for one example mouse each for D1R and D2R neurons (Fig. 6 B and C). We found that the same neuronal ensembles that showed stable sequential activity during the rightward lever pushing (on L2 and L3) exhibited markedly different activity patterns when the mouse pushed the lever leftward or downward on subsequent days (on N1 and N2; Fig. 6 B and C).

Fig. 6.

Reproducible sequential firing was motor task-specific. (A) Schematic diagram for the change in motor tasks. L2 and L3, the last 2 d of learning the rightward lever-pushing task; N1 and N2, 2 d of performing the leftward lever-pushing (N1) and downward lever-pressing (N2) tasks following L3. (B and C) Averaged activity of the same populations of D1R and D2R neurons on L2, L3, N1, and N2 recorded in one example mouse each, with cells sorted according to the sequence firing order determined on L3. Black lines indicate the onset time of lever movement. Norm., normalized. CCs of average activity between L3 and all three other days, calculated using the temporal population activity vector similar to that in Fig. 3, are shown (D, D1-Cre, n = 8 mice; F, D2-Cre, n = 10 mice; **P < 0.01, *P < 0.05 by Wilcoxon rank sum test). Ridge/Bgd ΔF/F ratios on L2, L3, N1, and N2 for the same dataset are shown (E, D1-Cre, n = 8 mice, P = 0.65; G, D2-Cre, n = 10 mice, P = 0.16; **P < 0.01, *P < 0.05, Wilcoxon rank sum test). NS, not significant. Error bars represent SEM.

The activity patterns of the neuronal ensembles during different motor tasks were further analyzed quantitatively in the same way as described for Fig. 3. We found that the CCs of activity patterns between L3 and subsequent days in performing leftward and downward movement tasks were significantly lower than between L2 and L3, for both D1R and D2R neurons (Fig. 6 D and F). The Ridge/Bgd ΔF/F ratios for sequential firing activity (ridge based on sequence activity on L3) also showed significant reduction for activity during performance of the new tasks (Fig. 6 E and G).

In separate experiments, we have also subjected the mice that had learned rightward lever pushing to a passive motor task that involves resting the forelimbs on a rolling cylinder, without cue and reward (Movie S5 and SI Appendix, Materials and Methods and Fig. S11A). As shown in SI Appendix, Fig. S11, the same neuronal ensembles that exhibited stable sequential activity after learning the cued rightward lever-pushing task showed totally different activity patterns during the rolling task on the subsequent day. Furthermore, sequential firing activity patterns were recovered when the mice were returned to the rightward lever-pushing task again on the next day. Thus, stable sequential firing of both D1R and D2R neurons at the late stage of learning is specifically associated with the learned task.

Distinct Functions of D1R or D2R Neurons Revealed by Selective Chemogenetic Silencing.

To dissect specific functions of D1R and D2R neurons in performing the learned motor task, we manipulated the activity of these neurons using a chemogenetic approach. A viral construct expressing the Cre-dependent engineered G protein-coupled receptor hM4Di was injected into the DLS of either D1- or D2-Cre mice (Fig. 7A) about 3 wk before the training of the cued lever-pushing task. After the mice had fully learned the task, we injected i.p. clozapine N-oxide (CNO), a synthetic ligand that activates hM4Di, to suppress neuronal firing through the G_i signaling pathway (32). Compared with the performance on the day before CNO injection, D1-Cre mice injected with CNO showed a reduced success rate (Fig. 7B) and increased reaction time (Fig. 7C), with reduced frequency of lever pushing during ITIs (Fig. 7D). By contrast, D2-Cre mice injected with CNO showed significantly elevated frequency of lever pushing during ITIs (Fig. 7D), without changing the success rate or the reaction time of the task performance (Fig. 7 B and C). Thus, the activity of D1R neurons is critical for initiating the motor action, whereas the activity of D2R neurons is required for suppressing uncued lever pushing during ITIs.

Fig. 7.

Distinct effects of selective silencing of D1R or D2R neurons. (A) Fluorescence image of a brain section from a D1-Cre mouse with a Cre-dependent hM4Di-expressing viral vector in the DLS. Success rates (B), reaction times (C), and number of lever-pushing events during the ITI (D) are shown in D1- and D2-Cre mice expressing Cre-dependent hM4Di on the last day (L3) of learning and on the next day with CNO administration. Gray lines connect data from the same mouse (D1-Cre, n = 16 mice, except those indicated in parentheses; D2-Cre, n = 13 mice; *P < 0.05, ***P < 0.001 by Wilcoxon rank sum test). Traces depicting lever movement trajectories from one example hM4Di-expressing D1-Cre mouse (E) and one example hM4Di-expressing D2-Cre mouse (F) on L2, L3, and the CNO injection day are shown, demonstrating that highly stereotyped movement trajectory on L2 and L3 were impaired after CNO injection. (G and H) Pairwise CCs of movement trajectories between single-trial trajectories within the same day and between two different days (number of mice used for analysis are indicated in parentheses). Significant effects of CNO treatment on movement trajectories were found for both D1-Cre and D2-Cre mice (D1-Cre, n = 16 mice, except those indicated in parentheses; D2-Cre, n = 13 mice; *P < 0.05, **P < 0.01 by Wilcoxon rank sum test).

We also analyzed the trial-to-trial reproducibility of movement trajectories (Fig. 7 E and F), as reflected by the pairwise CCs of movement trajectories within and across days during the last 2 d of training before CNO injection (L2 and L3) and the day following injection. We found that CNO injection in either D1-Cre or D2-Cre mice resulted in a significant reduction in pairwise CCs of movement trajectories within the same day or between adjacent days (Fig. 7 G and H), suggesting that activity of both D1R and D2R neurons is important for reliable execution of the stereotyped movement. Control mice injected with vehicle (saline plus 4% DMSO) instead of CNO showed no significant effects (SI Appendix, Fig. S12). Thus, the activity of both D1R and D2R neurons is required for proper execution of the learned motor action, with each mainly responsible for different aspects of the entire learned behavior.

Discussion

In this study, we have trained mice to perform a lever-pushing task in response to a sound cue, and monitored the activity of the same population of striatal neurons throughout the period of learning. We observed marked changes in neuronal firing patterns during the training process, with a gradual emergence of stable neuronal ensembles of both D1R and D2R neurons that fired in a sequential manner, with more D1R neurons fired during the lever-pushing period and more D2R neurons fired after lever pushing. Chemogenetic experiments showed that silencing D1R neurons impaired initiation of the lever-pushing action, whereas silencing D2R neurons resulted in increased erroneous lever pushing during ITIs. Furthermore, inhibition of either D1R or D2R neurons impaired the execution of stereotyped movement. These observations support the notion that SPNs of both direct and indirect pathways are actively involved and serve different functions in the initiation and execution of the learned motor task. Here, we have defined the learning to perform a cued motor task as motor learning in a broad sense, including not only lever pushing with a stereotyped trajectory but also the procedural learning involving cue perception, licking, body coordination, reward association, and inhibition of lever pushing during ITIs.

Emergence of Stable Sequential Firing of D1R and D2R Neurons.

Calcium imaging was used in this study to measure neuronal activity, with the peak GCaMP6s fluorescence changes (ΔF/F) representing integrated neuronal spiking activity (27). Compared with GCaMP6f, GCaMP6s is more sensitive to the Ca²⁺ elevation, thus yielding a higher percentage of responsive cells (27). GCaMP6s is widely used in many Ca²⁺ imaging studies (33–35), including striatal neurons (22, 36, 37). Despite the slow decay kinetics of calcium signals, the onset time and the peak value of ΔF/F are good indicators of the timing and overall spiking activity. The average time for an ΔF/F value to reach its peak for movement-related D1R and D2R neurons was shorter than the duration of lever pushing (SI Appendix, Fig. S13). Thus, we have sorted the neurons for their firing sequence based on the time of peak ΔF/F, similar to previous studies (38, 39).

The sequential firing was revealed by ordering the neurons according to the time of their average peak ΔF/F on the last day of training. Several lines of evidence indicate that the sequence was not due to cell sorting, but was related directly to motor learning. First, although the cells could be ordered to show sequential firing on any day, a stable sequential firing pattern was found only during the last few days of training (Fig. 3 A and C and SI Appendix, Figs. S4 and S5). Second, when the activity of the same population of neurons during each trial was randomly shuffled in time, we could not obtain the observed sequence by sorting based on the firing order (Fig. 3I). Third, on a single-trial basis, the increased correlation between the activity pattern with the final sequential firing pattern (activity CCs) was accompanied by an increased correlation of the lever-pushing trajectory with the learned stereotyped trajectory (movement CCs) (Fig. 4), consistent with the notion that the stable sequential activity pattern is related to the learned motor behavior. Finally, the stable sequential firing pattern was specifically associated with the learned motor action, since the pattern was markedly changed when the mouse subsequently performed different cued motor tasks or a passive motor task. Thus, the sequential firing pattern established in the late phase of training is related to the learning of the specific motor task.

Neuronal Populations Related to Different Aspects of Motor Behavior.

In sequentially firing ensembles, D1R and D2R neurons that fire at different times may be involved in different aspects of the learned motor behavior. We have separated neurons associated with the cue perception and reward reception from those associated with lever pushing by using dual-task experiments (Fig. 5). However, the movement-related M cells and postmovement-related PM cells obtained after subtracting cue- and reward-related cells from all responsive neurons may contain neurons involved in events unrelated to lever pushing, including not only lever pushing with the left forelimb but also coordinated body movements, licking, and responses to reward/punishment. Many sensorimotor and neuromodulatory processes and feedback from other brain areas may also be linked to neuronal activity in the striatum. Thus, extensive further studies are required to dissect the functions of different subpopulations within the sequentially firing ensembles. Nevertheless, we did show that the ensemble activity was unlikely to be related to licking, since the activity of PM cells was not correlated with the lick rate on a trial-by-trial basis, consistent with the previous finding that licking involved the ventral lateral striatum rather than the DLS (40, 41). Our chemogenetic results showed that D1R neurons are more relevant to cue perception and initiation of specific motor action, whereas D2R neurons are more involved in postmovement events, including inhibition of uncued motor actions during ITIs (Fig. 7). Further confirmation of the causal relationship between the activity of specific neuronal populations and the related behavior events would require selective activation of specific subpopulations of the D1R and D2R neurons that are active at different times of sequential firing during the task performance.

Distinct Roles of D1R and D2R Neurons in Motor Learning.

The role of direct and indirect pathways in motor actions has been controversial. The textbook notion of antagonistic functions of direct vs. indirect pathways and the associated interpretation for the pathogenesis of movement disorders (15–17) have been supported by recent optogenetic studies (18, 42). However, this notion was challenged by the findings that coordinated, rather than antagonistic, activity of these two pathways regulates motor functions (19, 20, 22, 23), with the activity of D1R neurons initiating the intended movements and those of D2R neurons suppressing other competing motor actions (21). There is also evidence for a temporal difference in the activity of D1R and D2R neurons (31).

Our results suggest that both D1R and D2R neurons were required to perform the learned motor task, which has a high success rate, short reaction time, and low-frequency motor action during ITIs. Chemogenetic silencing of D1R neurons resulted in marked reduction in the frequency of both cued and uncued lever pushing and increased the reaction time for successful trials, indicating that D1R neuronal activity is essential for initiation of motor actions. By contrast, inhibition of D2R neurons did not affect the frequency of cued lever pushing and the reaction time, but increased the frequency of uncued lever pushing during ITIs. Notably, both types of neurons were involved in the execution of the stereotyped movement, because silencing either D1R or D2R neurons led to a significant reduction in pairwise CCs (i.e., reproducibility) of movement trajectories. Learning-induced changes in the pattern of neuronal activity were consistent with behavioral results from chemogenetic experiments. First, learning resulted in a significant increase in cue cells only in D1R neurons (Fig. 5D), consistent with the increased reaction time following chemogenetic suppression of D1R neurons (Fig. 7C). Second, more D1R neurons were activated during the lever-pushing period and more D2R neurons were activated during ITIs (Fig. 5 L–O), consistent with the reduction of cued lever pushing caused by suppressing D1R neurons and increased uncued lever pushing during ITIs following suppression of D2R neurons (Fig. 7 B and D). Finally, the percentages of both D1R and D2R neurons associated with the lever pushing (M cells) increased throughout learning (Fig. 5 L and M), consistent with the finding of a significant reduction in the pairwise CCs of movement trajectories following suppression of either D1R or D2R neuronal activity (Fig. 7 E–H). Together, these results support the complementary model in which both D1R and D2R neurons are necessary for proper motor action, with the direct pathway involved in the initiation of a particular motor action and the indirect pathway responsible for suppressing unwanted motor programs (21).

In chemogenetic experiments using D2-Cre mice, ∼1% of hM4D-expressing neurons may be cholinergic interneurons (CINs), as suggested by a previous study using D2-Cre mice to express channelrhodopsin-2 (18). These CINs form axo-axonal connections with dopamine fibers and could trigger dopamine release in the striatum (43, 44). Acetylcholine could also inhibit excitatory inputs from the cortex and thalamus onto SPNs via muscarinic receptors or directly modulate the excitability of SPNs (45–49). Further experiments using choline acetyltransferase-Cre mice are required to examine the potential role of CINs during motor learning and execution of learned motor behavior. In chemogenetic experiments using D1-Cre mice, however, CINs are unlikely to affect the interpretation of our results because CINs do not express D1Rs.

Spatial Distribution of Learning-Induced Neuronal Ensembles.

An issue of interest is whether neuronal ensembles with correlated firing formed during motor learning are spatially clustered in the DLS. Previous optical imaging of SPN activity in the dorsal striatum has identified spatially compact neural clusters that encode locomotion-related information (22). However, D1R and D2R neuronal activity in this region during self-paced natural behaviors did not appear to be distributed in compact clusters (23), and no clustering was found in a learned three-port sequence task in striatal neurons (50). In addition, anatomically intermixed distribution of neurons was reported in a choice-specific sequential firing activity in the parietal cortex during a virtual-navigation decision task (30). In primary motor cortex, movement-related excitatory neurons were also shown to lack spatial clustering when mice performed the lever-pressing task (26). Thus, dispersed distribution appears to be a common mode for establishing neuronal ensembles related to specific behavioral tasks. In the present study, we did not find evidence for clustered distribution of neuronal ensembles with correlated firing after training, but whether further overtraining could eventually induce ensembles that are clustered remains to be examined. Another issue to be addressed is whether the same neurons could participate in multiple stable ensembles that are specific for different tasks, a scenario that would be better realized by a dispersed, rather than clustered, distribution of neurons of each ensemble associated with a specific motor task.

In summary, our study provided a comprehensive characterization of the activity of striatal D1R and D2R neurons during the entire course of motor learning and uncovered a reorganization of D1R and D2R neuronal activity into stable sequentially firing ensembles during the learning process. The temporal difference in the activity pattern of D1R and D2R ensembles supports the notion that direct and indirect pathways are actively involved in different aspects of motor behaviors and differentially reorganized during learning. Further studies of the neural circuit mechanisms in the formation and action of these neuronal ensembles will offer new insight into the organization and function of direct and indirect pathways.

Materials and Methods

Experimental procedures for animal surgery, behavior training, two-photon calcium imaging, and data analysis are described in detail in SI Appendix, Materials and Methods. Animal use procedures were approved by the Animal Use Committee of the Institute of Neuroscience, Shanghai Institutes for Biological Sciences, Chinese Academy of Sciences.