Spatial organization of functional clusters representing reward and movement information in the striatal direct and indirect pathways

Significance

The striatum is critically involved in voluntary motor control and reward-based learning. To formulate a nuanced model of the striatum, it would be important to understand how functional ensembles of striatal neurons are organized in the spatiotemporal domain. In this perspective, we monitored ensemble activity of a large number of striatal neurons in mice performing a classical conditioning task and a self-paced open-field test. We found partially overlapping and learning-dependent spatial organizations of striatal neuronal clusters for movement- as well as reward-related variables in both the direct and indirect pathways. These findings provide insights into striatal neural circuit operations underlying reward-based learning and movement control.

Abstract

To obtain insights into striatal neural processes underlying reward-based learning and movement control, we examined spatial organizations of striatal neurons related to movement and reward-based learning. For this, we recorded the activity of direct- and indirect-pathway neurons (D1 and A2a receptor-expressing neurons, respectively) in mice engaged in probabilistic classical conditioning and open-field free exploration. We found broadly organized functional clusters of striatal neurons in the direct as well as indirect pathways for both movement- and reward-related variables. Functional clusters for different variables were partially overlapping in both pathways, but the overlap between outcome- and value-related functional clusters was greater in the indirect than direct pathway. Also, value-related spatial clusters were progressively refined during classical conditioning. Our study shows the broad and learning-dependent spatial organization of functional clusters of dorsal striatal neurons in the direct and indirect pathways. These findings further argue against the classic model of the basal ganglia and support the importance of spatiotemporal patterns of striatal neuronal ensemble activity in the control of behavior.

The striatum is critically involved in voluntary motor control and reward-based learning (1–3). The striatum sends its output to its downstream structures via two distinct streams of projections, namely the direct and indirect pathways. The direct pathway projects directly to the internal globus pallidus (GPi; in primates) and the substantia nigra pars reticulata (SNr), while the indirect pathway projects to the external globus pallidus (GPe) and then to the SNr/GPi. The direct and indirect pathways of the basal ganglia have long been thought to play opposing roles, with the former promoting and the latter inhibiting behavior (4, 5). This classic model of the basal ganglia has played a central role in conceptualizing normal circuit operations as well as the pathophysiology of the basal ganglia (6). This view of the direct and indirect pathways is supported by numerous neuromodulation studies in which activation of the direct and indirect pathways promoted and suppressed behavior, respectively (7–9), or, conversely, inactivation of the direct and indirect pathways impaired the initiation of learned motor actions and suppression of erroneous motor actions, respectively (10, 11). However, recent studies have provided evidence against the classic model of the basal ganglia. For example, not only direct-pathway but also indirect-pathway striatal neurons are activated during action initiation (12). This finding can be explained by the refined classic rate model, which posits that the direct pathway promotes targeted movement while the indirect pathway suppresses competing motor programs (13, 14). This “suppression-selection model” has been challenged by more recent studies, however; unilateral inactivation of either pathway induced similar ipsiversive movement (15), and bilateral inactivation of either pathway similarly suppressed a sequential lever press behavior (11). Moreover, the two pathways showed comparable spatial patterns of activity during movement, a finding difficult to explain with the suppression-selection model (16).

Contradicting bodies of evidence for and against opposing roles of the direct and indirect pathways in controlling behavior call for a more nuanced model considering functional organizations of the two pathways beyond a model assuming a simple seesaw-like balance between the two pathways. In this regard, the striatum has been proposed to control behavior based on spatially organized, mutually competing functional clusters of direct- and indirect-pathway neurons (17). In this scenario, direct- and indirect-pathway neurons of a given functional cluster promote and inhibit a specific behavior, respectively. However, precise representation of selected behavior would be determined by precise activity patterns of direct- and indirect-pathway neurons across multiple functional clusters, which are shaped by patterns of input activity and lateral inhibition among different functional clusters. The functional cluster model is consistent with the findings that subpopulations of striatal neurons show synchronized activation in association with movements and rewarding events (12, 18, 19).

To formulate and refine such nuanced models of the basal ganglia, it would be important to understand how functional ensembles of striatal neurons are organized in the spatiotemporal domain. Currently, there is no consensus even on the simple issue of whether and how striatal neurons form spatially organized functional clusters. The initial study reported that movement-related functional ensembles form compact local clusters in the striatum (20). However, later studies reported broad and loosely organized spatial patterns of functional neuronal ensembles (16, 21). Furthermore, other studies concluded the absence of spatially organized functional neuronal ensembles in the striatum (10, 22).

In the present study, unlike in the previous studies that focused on movement-related variables, we examined spatial organizations of functional clusters related to both voluntary movement and reward-based learning. We also investigated whether spatially organized functional clusters, if they exist, can be refined with learning. For this, we performed endoscopic calcium imaging of dorsal striatal neurons in mice performing a classical conditioning task and a self-paced open-field test. Our results indicate partially overlapping and learning-dependent spatial organizations of striatal neuronal clusters for movement- and reward-related variables in both the direct and indirect pathways.

Results

Virus Injection and Lens Implantation.

We injected the adeno-associated virus (AAV) vector carrying the gene for GCaMP6f with enhanced yellow fluorescent protein (eYFP) into the dorsal striatum of mice. Specifically, we injected AAV-hsyn-GCaMP6f-eYFP into wild-type (WT) mice (n = 3), and injected AAV-flex-GCaMP6f-eYFP into D1R-Cre (n = 5) and A2a-Cre (n = 5) mice for selective expression of the calcium sensor in direct- and indirect-pathway medium spiny neurons, respectively (23, 24) (Fig. 1A). Two weeks after the virus injection, we implanted a graded-index (GRIN) lens (diameter, 1 mm) in the dorsal striatum to record calcium fluorescence signals from both dorsolateral and dorsomedial striatal neurons (SI Appendix, Fig. S1). Histological examinations at the completion of recording revealed selective expression of GCaMP6f in direct- and indirect-pathway striatal neurons in D1R-Cre and A2a-Cre mice, respectively (Fig. 1 B and C).

Fig. 1.

Virus expression and classical conditioning task. (A) A schematic sagittal brain diagram showing AAV virus injection into the dorsal striatum. (B and C) Sagittal sections of D1R-Cre (B) and A2a-Cre (C) mouse brains showing GCaMP6f expression (green) in the dorsal striatum and SNr (D1R-Cre mouse) or GPe (A2a-Cre mouse). (D and E) Schematics for a probabilistic classical conditioning task (D) and its experimental setting (E). (F) Cue-dependent licking responses averaged across all sessions and all animals. Shown are lick density functions (σ = 100 ms; shading indicates SEM across animals). Trials were grouped according to cue. (G–I) Cue-dependent licking responses during the final session for each animal. Shown are lick rates during the delay period (G), outcome period (1 s) in rewarded trials (H), and outcome period (1 s) in unrewarded trials (I). Outcome onset was defined as the time of the first lick after reward delivery in rewarded trials and as the time of buzzer onset in unrewarded trials. Circles indicate individual animal data (n = 13 mice consisting of 3 WT, 5 D1R-Cre, and 5 A2a-Cre mice). Bar graphs and error bars indicate their mean and SEM across animals. ***P < 0.001 (ANOVA followed by post hoc Tukey test). (J) Mean (±SEM across animals) delay-period lick rates shown by training day. Green arrowheads denote significantly differential lick rates to all three cues (P < 0.01, ANOVA followed by post hoc Tukey test).

Behavior.

Calcium signals were monitored while the mice were engaged in a probabilistic classical conditioning task under head fixation (Fig. 1 D and E) as well as self-paced free exploration in a square box (Fig. 2A). In the classical conditioning task, three different odor cues (1 s) were paired with three different probabilities (20, 50, and 80%) of water delivery (5 μL) following a delay of 1 s. A buzzer sound cue (2,300 Hz, 100 ms) was presented in unrewarded trials (Fig. 1D). The mice showed anticipatory licking responses before reward delivery (cue and delay periods) and the rate of anticipatory licking during the delay increased as a function of reward probability [one-way ANOVA, F_(2,36) = 33.53, P = 6.0 × 10⁻⁹; Fig. 1 F and G]. Lick rate during the outcome period (1 s following outcome onset) was similar across the three cues in rewarded trials [1 s after reward delivery; F_(2,36) = 0.083, P = 0.922; Fig. 1H] but different across the three cues in unrewarded trials [F_{(2, 36)} = 11.30, P = 1.5 × 10⁻⁴; Fig. 1I]. The mice started to show significantly differential anticipatory licking to either 20 or 80% reward cue versus the other two cues on the first day of training, and showed differential licking to all three cues within 2 to 5 d of training (see Materials and Methods for the performance criterion) (Fig. 1J). The mice were trained for 2 additional days after reaching the criterion. Mean (±SD) training durations were 5.6 ± 0.6, 4.8 ± 0.6, and 5.0 ± 0.7 daily sessions for WT, D1R-Cre, and A2a-Cre mice, respectively (SI Appendix, Fig. S2).

Fig. 2.

Open-field test. (A) A schematic for the open-field arena used in the experiments. (B) A sample video frame showing tracking of the head (red), center (green), and tail (blue) of a mouse with DeepLabCut. The angle symbol denotes the head direction angle. (C) Cumulative positions of the head (red), center (green), and tail (blue) during a sample session. (D) The animal’s behavioral state was classified into five different categories (Middle; orange, complete stop; green, forward motion; red, contralateral turn; blue, ipsilateral turn; gray, other states) based on velocity traces of head, center, and tail positions along with head angular velocity (Top). (D, Bottom) Calcium transient signals of 226 dorsal striatal neurons recorded during the sample session. (E–G) Mean velocity traces and corresponding mean normalized calcium traces in D1R-Cre (n = 5) and A2a-Cre (n = 5) mice aligned to the onset of forward motion (E), stop (F), or ipsilateral/contralateral turn (G). Shading indicates SEM across animals.

Each classical conditioning session was followed by a session of self-paced open-field exploration (10 to 15 min) in a square box (30 × 30 × 30 cm) with a 5-min intermission. The animal’s movement was monitored with a high-definition video camera (Fig. 2A), and head, body center, and proximal part of the mouse tail were tracked with a deep learning-based position-tracing algorithm (DeepLabCut; Materials and Methods and Fig. 2 B and C). The animal’s movement was classified as five types (forward motion, contralateral turn, ipsilateral turn, stop, and others; Fig. 2 D–G; see SI Appendix, Fig. S3 for their characteristics) based on movement velocities of the head, body center, and proximal part of the tail along with head angular velocity (Materials and Methods).

Activity Correlation as a Function of Distance.

The recorded calcium signals were processed with nVista image-processing software and calcium transients were extracted using the constrained nonnegative matrix factorization for endoscopy (CNMF-E) algorithm (25). To ensure proper handling of fluctuating background fluorescence signals, we extracted calcium signals using substituted background signals from other sessions. Calcium transient signals obtained with this procedure were similar to the original ones (SI Appendix, Fig. S4), indicating that fluctuating background signals were well-adjusted by the CNMF-E algorithm. As previously reported in our neurophysiological study (26), both direct- and indirect-pathway striatal neurons showed diverse patterns of responses to reward- and movement-related variables. In particular, they showed a strong tendency to increase activity in association with movement during open-field exploration; calcium transient signals were much greater after compared with before movement onset (Fig. 2 D–G), which is consistent with previous reports (12, 20, 21).

To examine the functional spatial clustering of striatal neurons, we first examined how pairwise correlation in calcium transient signals varies as a function of the distance between neurons (Fig. 3). We found that neighboring neurons tend to show positively correlated changes in calcium signal with the strength of correlation decaying gradually. The distance (mean ± SD) where the correlation curve reached the baseline (“correlation decay distance”; Materials and Methods and SI Appendix, Fig. S5) was 208 ± 44, 61 ± 3, and 53 ± 5 µm in WT, D1R-Cre, and A2a Cre mice, respectively, during the classical conditioning task, and 138 ± 1, 68 ± 1, and 50 ± 6 µm, respectively, during the open-field test. The difference was significant across the animal groups and the tasks [two-way mixed ANOVA, animal group, F_(2,20) = 151.5, P = 8.3 × 10⁻¹³; task, F_(1,20) = 15.6, P = 7.9 × 10⁻⁴; animal group × task interaction, F_(2,20) = 14.4, P = 1.3 × 10⁻⁴; post hoc Tukey test for difference between animal groups, classical conditioning task, WT versus D1R-Cre, P = 2.1 × 10⁻⁸, WT versus A2a-Cre, P = 2.1 × 10⁻⁸, D1R-Cre versus A2a-Cre, P = 0.947; open-field test, WT versus D1R-Cre, P = 2.4 × 10⁻⁵, WT versus A2a-Cre, P = 5.0 × 10⁻⁷, D1R-Cre versus A2a-Cre, P = 0.245; SI Appendix, Fig. S5]. These results indicate similar activity patterns among closely spaced striatal neurons, which is consistent with the spatial clustering of functionally related striatal neurons. The longer correlation decay distance in WT than the other animal groups is presumably because spatial clusters of the two pathways are overlaid in WT mice.

Fig. 3.

Cross-correlation of calcium signals as a function of distance. (A) A sample spatial map of striatal neurons recorded from a D1R-Cre mouse. Colors indicate three sample neurons whose calcium signals are shown in B. The spatial orientation is denoted as A, P, M, and L for anterior, posterior, medial, and lateral, respectively. (B) Normalized calcium signal traces of the three sample neurons that were more active in the classical conditioning task (red), more active in the open-field test (blue), or equally active in the two tasks (black). (Scale bars, time [2 min] and normalized intensity [10].) (C) Scatterplots showing cross-correlations of calcium signals for all pairs of simultaneously recorded striatal neurons during the classical conditioning task (Left), open-field test (Middle), and entire task periods (Right) in WT (Top), D1R-Cre (Middle), and A2a-Cre (Bottom) mice. Black, light blue, and orange lines denote mean cross-correlations for WT, D1R-Cre, and A2a-Cre mice, respectively, and shading indicates 2 SDs.

Clustering of Functionally Related Striatal Neurons.

We then examined the spatial distribution of striatal neurons coding the same variable. Previous studies have shown that striatal neurons convey value and outcome signals in the form of persistent neural activity for multiple trials (27, 28). For the classical conditioning task, therefore, we examined calcium signals significantly correlated with value (i.e., reward probability), outcome (reward delivery versus reward omission), previous outcome (outcome of the previous trial), and previous value (i.e., reward probability associated with the cue in the previous trial) using a multiple-regression analysis that included lick rate as an additional regressor to control for lick-dependent neural activity (Eq. 1). The analysis time windows were as follows: outcome, 1 s after outcome delivery; value, 1 s after cue delivery; and previous value and previous outcome, 3 s after cue delivery. For the free-exploration session, we examined calcium signals significantly correlated with forward motion, stop, contralateral turn, and ipsilateral turn (Fig. 2 E–G) using a simple regression analysis (Eq. 2). The analysis windows for movement-related variables were 1-s periods following each movement onset. We used the data collected on the last day of classical conditioning for these analyses. To illustrate how we determined spatial clustering, we describe in detail how we assessed spatial clusters related to reward value as follows. Spatial clustering for other variables was determined in the same way.

For the analysis of spatial clustering of value-coding striatal neurons, we first determined the regression coefficient for value for each neuron (Eq. 1). We found both positive and negative correlations between striatal neuronal activity and reward value (Fig. 4 A and B). The fraction of neurons that increase their activity as a function of value was significantly greater in D1R-Cre than A2a-Cre mice (delay period, χ² test, χ² = 34.96, P = 3.4 × 10⁻⁹) and, conversely, the fraction of neurons that decrease their activity as a function of value was significantly greater in A2a-Cre than D1R-Cre mice (χ² = 10.22, P = 0.001; Fig. 4 C–E), which confirms our previous report (26). Fig. 4 F–H shows spatial distributions of the regression coefficient for value in sample WT, D1R-Cre, and A2a-Cre mice. Red and blue indicate positive and negative signs of the regression coefficient, respectively, and color saturation denotes the magnitude of the regression coefficient. As shown, in all mice, value-related neuronal responses were broadly and nonuniformly distributed. We adopted a method used previously for identifying orientation columns in the visual cortex (29) to determine the significance and size of a functional cluster. Briefly, we calculated the local cluster index, which is the mean absolute difference in the regression coefficient between a reference neuron and all other neurons within a circular boundary. We estimated the local cluster index while changing the radius of the circular boundary from 30 to 600 µm, and the size and significance of a cluster were determined by comparing the original local cluster index curve with those obtained after a random shuffling of neuronal positions (Materials and Methods and SI Appendix, Fig. S6). We found significant spatial clusters related to reward value in all animals tested (Fig. 4 I and J). The mean (±SD) cluster size was 280 ± 194, 258 ± 278, and 129 ± 36 μm in WT, D1R-Cre, and A2a-Cre mice, respectively, which did not vary significantly [one-way ANOVA, F_(2,10) = 0.76, P = 0.481; Fig. 4K]. We performed a spatial autocorrelation analysis to further validate the spatial clustering of value-coding striatal neurons. We found that spatial autocorrelation maps deviated significantly from a random distribution which was consistent across different filter sizes (permutation test; SI Appendix, Fig. S7). These results indicate that 1) value-coding striatal neurons are spatially organized, 2) such value-related functional clusters are broadly rather than compactly organized, and 3) value-related spatial clustering is similar between direct- and indirect-pathway neurons.

Fig. 4.

Spatial clustering of value-coding striatal neurons. (A) A sample spatial map of striatal neurons recorded from a D1R-Cre mouse. Two sample neurons that increased and decreased activity as a function of value are indicated by arrowheads with red and blue colors, respectively. (B) Calcium signals of the two sample neurons (shading indicates SEM across trials) during the classical conditioning task. Trials were grouped according to cue. (C–E) Temporal profiles of value signals (0.5-s moving window, 0.1-s steps). (C–E, Left) Fractions of neurons significantly responsive to value. (C–E, Middle and Right) Value-coding neurons were divided into those increasing (positive value coding; Middle) and decreasing (negative value coding; Right) activity as a function of value. Gray, WT (n = 3 mice, 596 neurons); light blue, D1R-Cre (n = 5 mice, 1,004 neurons); orange, A2a-Cre (n = 5 mice, 1,255 neurons). Green triangles indicate significant difference between D1R-Cre and A2a-Cre mice (P < 0.05, χ² test). (F–H, Top) Sample spatial maps showing spatial distributions of value-related neural activity obtained from a WT (Left), D1R-Cre (Middle), and A2a-Cre mouse. Value coefficients obtained from multiple linear regression are color-coded for each neuron. (F–H, Bottom) The local difference in value coefficient as a function of ROI size for the corresponding sample spatial map. Blue and red denote local-difference curves obtained from the original and position-shuffled data, respectively. Red shading indicates 1 SD; light red shading indicates 2 SDs. Vertical dashed lines indicate the cluster size. (I) Shown are the local cluster index histogram of the position-shuffled data (1,000 iterations; red), its cumulative probability density function (orange curve), and local cluster index of the original data (blue line) for the WT sample spatial map shown in F. P value was calculated as the integral of the probability density function between the position of the real data and infinity. (J) Local cluster indices and P values for all animals (WT, gray; D1R-Cre, light blue; A2a-Cre, orange). The black vertical line denotes the threshold for significant clustering (P = 0.05). (K) Cluster sizes (means ± SEM across animals) in WT, D1R-Cre, and A2a-Cre mice. Circles indicate individual animal data.

A previous one-photon imaging study measured “pairwise distance” and “similarity score” (the degree of simultaneous activation) between simultaneously recorded neurons and found no evidence for functional spatial clustering (22). We applied the same analysis procedures to our data to investigate why different conclusions were drawn across studies. When we calculated pairwise distance from our data, we also failed to find evidence for significant spatial clustering (SI Appendix, Fig. S8 A and B); however, this measure successfully detected simulated compact functional clusters (SI Appendix, Fig. S8 C and D), suggesting that pairwise distance might not be an appropriate measure for detecting broadly and loosely organized spatial clusters. When we calculated the similarity score from our data, 17.5% of all neuron pairs were estimated to have significant similarity scores (i.e., positively correlated activity; SI Appendix, Fig. S8 E and F) which is substantially larger than the previously reported value (1.7%) (22). A possible explanation for this discrepancy is that the number of simultaneously recorded neurons within a field of view (1-mm-diameter GRIN lens) was relatively small in the study that failed to find evidence for functional spatial clustering (555 neurons from 8 mice; 69.3 neurons per session on average) (22) compared with our study (2,855 neurons from 13 mice; 219.6 neurons per session) and an earlier one-photon imaging study that found significant spatial clustering (7,434 neurons from 25 mice; 297.4 neurons per session) (16). Indeed, when we downsampled our neural data (randomly selected 70 neurons per session), only 2.7% of all neuron pairs were estimated to have significant similarity scores (SI Appendix, Fig. S8 G and H).

We obtained overall similar results with variables other than reward value (SI Appendix, Fig. S9). We found significant spatial clustering of all variables in all mice during the classical conditioning task except for “previous outcome” in one A2a-Cre mouse, and “previous value” in one WT and one D1R-Cre mouse. Likewise, we found significant spatial clustering of all variables in all mice during the free-exploration session except for “forward motion” in one D1R-Cre mouse, “stop” for one WT and one A2a-Cre mouse, “contralateral turn” in one D1R-Cre mouse, and “ipsilateral turn” in one A2a-Cre and one D1R-Cre mouse (SI Appendix, Fig. S9 and Table S1). No significant variation in cluster size was found across the three animal groups in any of these variables (one-way ANOVA, P values > 0.05). Cluster sizes averaged across different variables in WT, D1R-Cre, and A2a-Cre mice (in μm; ±SD) were 205 ± 72, 222 ± 110, and 91 ± 29, respectively, in the classical conditioning task and 226 ± 99, 170 ± 94, and 98 ± 70, respectively, in the open-field test. The difference was significant across the animal groups but not across the tasks [two-way mixed ANOVA, animal group, F_(2,18) = 4.9, P = 0.019; task, F_(1,18) = 0.06, P = 0.807; animal group × task interaction, F_(2,18) = 0.5, P = 0.644; post hoc Tukey test for difference between animal groups, WT versus D1R-Cre, P = 0.887, WT versus A2a-Cre, P = 0.024, D1R-Cre versus A2a-Cre, P = 0.061]. These results indicate broad functional spatial clusters for reward- as well as movement-related variables in both the direct and indirect pathways.

Relations among Different Functional Clusters.

Next, we examined how functional spatial clusters for different variables are related, and whether these relationships vary between direct- and indirect-pathway striatal neurons. Fig. 5A shows spatial distributions of regression coefficients for the reward- and movement-related variables in a sample D1R-Cre mouse. Note that the coefficient spatial maps are distinct but partially overlapping between different variables. For quantification, we calculated cross-correlations between coefficient spatial maps for all possible combinations of variables. We found both positive and negative correlations between different coefficient spatial maps (Fig. 5B shows cross-correlations averaged across animals). For example, coefficient spatial maps for forward motion, contralateral turn, and ipsilateral turn were positively correlated with each other whereas they were negatively correlated with the coefficient spatial map for stop. This suggests that a common set of striatal neurons might be activated during movement initiation.

Fig. 5.

Relationships among clusters coding different variables. (A) Sample spatial maps for reward-related variables (Top, classical conditioning task) and movement-related variables (Bottom, open-field test) recorded from a D1-Cre mouse. The same format is used as in Fig. 4F. (B) Spatial correlations (color-coded) between maps for different variables (CT, contralateral turn; F, forward motion; IT, ipsilateral turn; O, outcome; PO, previous outcome; PV, previous value; S, stop; V, value) in WT, D1R-Cre, and D2R-Cre mice (mean across animals). CC task, classical conditioning task. (C) Mean absolute spatial correlations between reward-related variables (classical conditioning task), between movement-related variables (open-field test), and across reward- and movement-related variables. ***P < 0.001 (one-way ANOVA followed by Tukey tests). (D) Dendrograms showing hierarchical clustering of spatial clusters for different variables. The spatial correlation maps in B were reordered according to the clustering.

The absolute spatial-map correlation was larger between variables estimated in the same behavioral task than across the two tasks [one-way ANOVA, F_(2,36) = 16.64, P = 7.6 × 10⁻⁶; post hoc Tukey test, within classical conditioning vs. across tasks, P = 0.001; within free exploration vs. across tasks, P = 6.2 × 10⁻⁶; Fig. 5C]. Thus, stronger relationships (both positive and negative) were found between spatial clusters of variables estimated in the same behavioral context than those estimated across different behavioral contexts (classical conditioning under head fixation versus free exploration). It remains to be determined whether this is because movement- and reward-related variables are orthogonally represented and/or because the striatal neural activity is strongly modulated by a set of sensory signals (contextual information). Unexpectedly, we found a negative relationship between clusters for previous value and previous outcome in all animal groups (Fig. 5B), which suggests that striatal neurons might carry reward prediction error (the difference between predicted and actual outcomes) signals until the next trial.

That functional spatial clusters are partially overlapping suggests individual neurons are responsive to multiple variables. Indeed, as reported previously (26), individual striatal neurons were responsive to multiple variables in both D1R-Cre and A2a-Cre mice [“mixed selectivity” (30); SI Appendix, Fig. S10]. This explains partially overlapping patterns of functional spatial clusters for different variables. We found that <30% of striatal neurons were significantly responsive to only one variable and the rest were responsive to two or more variables with some responsive up to seven (SI Appendix, Fig. S10A). We also found that ∼30% of striatal neurons show significant responses to both reward- and movement-related variables (SI Appendix, Fig. S10B). Striatal neuronal populations showed positively correlated, negatively correlated, as well as uncorrelated responses to two different variables (examples are shown in SI Appendix, Fig. S10C). When we calculated correlations between two different variables for all possible combinations, the resulting pattern was similar to that for cross-correlations between coefficient spatial maps shown in Fig. 5B (SI Appendix, Fig. S10D).

The pattern of spatial-map correlations was similar between D1R-Cre and A2a-Cre mice. Thus, both direct- and indirect-pathway striatal neurons formed functional spatial clusters for different variables in a partially overlapping manner. However, cluster relationships were not identical between D1R-Cre and A2a-Cre mice. When we performed hierarchical clustering to further characterize relationships among different functional spatial clusters, we found some differences between D1R-Cre and A2a-Cre mice (Fig. 5D). Spatial clusters for movement initiation-related variables, namely forward motion, contralateral turn, and ipsilateral turn, were grouped in D1R-Cre as well as A2a-Cre mice. However, spatial clusters for reward-related variables were differently grouped between D1R-Cre and A2a-Cre mice. Specifically, spatial clusters for value, outcome, and previous outcome were grouped in A2a-Cre mice (and also in WT mice) but not in D1R-Cre mice (Fig. 5D). To test whether this difference is significant, we compared cluster relationships among these three variables (outcome, previous outcome, and value) between D1R-Cre and A2a-Cre mice using a permutation test (Materials and Methods). The results indicated that the spatial cluster relationship between value and outcome is significantly different between D1R-Cre and A2a-Cre mice (P = 0.016; value versus previous outcome, P = 0.105; outcome versus previous outcome, P = 0.408; alpha adjusted to 0.017 for Bonferroni correction). These results indicate that relationships between different functional spatial clusters are overall similar but those related to reward-based learning (outcome and value) are more related in A2a-Cre than D1R-Cre mice.

Cluster Changes during Learning.

The results described so far are based on the analysis of the data obtained on the last day of classical conditioning. To examine how functional spatial clustering may change during learning, we compared spatial clusters between the first and last days of classical conditioning. We used the data obtained from the D1R-Cre and A2a-Cre mice for this analysis. Fig. 6 A and C shows sample spatial coefficient maps for value on days 1 and 5 of a D1R-Cre mouse. As shown, striatal neurons formed a significant value-related spatial cluster not on day 1 (P = 0.210) but on day 5 (P = 0.002) in this sample mouse (Fig. 6 B and D). In the group analysis (n = 10; 5 D1R-Cre and 5 A2a mice), local cluster index was significantly greater on day 5 than day 1 for value [paired t test, t₍₉₎ = −3.77, P = 0.004] and previous value [t₍₉₎ = −3.47, P = 0.005] but not for the other variables [outcome, t₍₉₎ = 0.091, P = 0.929; previous outcome, t₍₉₎ = −2.00, P = 0.077; forward motion, t₍₉₎ = −0.48, P = 0.604; stop, t₍₉₎ = 0.40, P = 0.780; contralateral turn, t₍₉₎ = −0.48, P = 0.720; ipsilateral turn, t₍₉₎ = 0.48, P = 0.400; alpha adjusted to 0.006 for Bonferroni correction; Fig. 6E]. Note that the only learning-dependent variables in our tasks were value-related ones (value and previous value). These results show that spatial clusters of striatal neurons coding learning-dependent variables can be shaped by learning.

Fig. 6.

Formation of value-related spatial clusters during training. (A–D) Sample spatial maps (A and C) and corresponding local cluster functions (B and D) for value on the first (A and B) and last (day 5; C and D) days of training in a D1R-Cre mouse. The same format is used as in Fig. 4F. (E) The bar graphs show local cluster indices for different variables for the first day (gray) and the last day (blue) of training (mean ± SEM). Circles and thin lines denote individual animal data (five D1R-Cre and five A2a-Cre mice). Asterisks indicate significant difference between the first and last days (*P < 0.006, paired t test).

Discussion

To examine the functional spatial clustering of direct- and indirect-pathway striatal neurons, we recorded calcium signals from >100 dorsal striatal neurons simultaneously in WT, D1R-Cre, and A2a-Cre mice performing a probabilistic classical conditioning task and an open-field test. We found that both direct- and indirect-pathway neurons form spatially clustered activity patterns for movement- as well as reward-related variables. The relationship between clusters for different variables was generally similar, but the relationship between outcome- and value-related clusters was stronger in the indirect than direct pathway. We also found that value-related spatial clusters are progressively refined with learning.

Evidence for Functional Spatial Clusters in the Striatum.

We employed two different analysis procedures, the local cluster index and spatial autocorrelation, to examine whether functionally related striatal neurons are spatially clustered. Both methods indicated that dorsal striatal neurons form significant functional spatial clusters. One-photon calcium imaging may contain local background fluctuations of fluorescence signals from out-of-focus neurons and neuropil contaminations (25) which may lead to false-positive spatial clusters unless background fluctuations are adequately corrected. To control for such misidentification, we used an algorithm that extracts local background fluorescence signals (25). We found that the calcium traces extracted using background signals from other sessions were highly correlated with the originally extracted calcium traces (SI Appendix, Fig. S4). Furthermore, calcium signal correlations were low between neurons as previously reported (21). These results cannot be explained if the calcium signals were contaminated with local background signals.

Previous studies exploring the spatial organization of striatal neurons (10, 16, 20–22, 31, 32) have yielded inconsistent results regarding functional spatial clustering. The conclusions range from compact spatial clusters (20) to broad and loosely organized spatial clusters (16, 21) and no obvious spatial clusters (10, 22). Our results support broad and loosely organized spatial clustering of functional striatal units. It remains to be clarified why different conclusions were obtained across studies. Nevertheless, some of the previous studies provide clues to why different outcomes may have been obtained depending on experimental and analytical procedures. Klaus et al. (21) have shown that inadequate correction of background signals may lead to erroneous identification of compact functional clusters, which can explain why compact clusters were found in an earlier study (20). Parker et al. (16) found that functional spatial clustering of striatal units is difficult to detect with two-photon imaging unless followed by a volumetric analysis, which can explain why functional spatial clustering was not detected in a previous two-photon imaging study (10). Our study suggests that the use of a measure inappropriate for detecting broadly organized functional clusters (pairwise distance) and relatively small numbers of simultaneously recorded neurons might be reasons why no significant functional spatial clusters were found in a previous one-photon imaging study (10, 22).

Our results show that the size of a spatial cluster is highly variable across different reward- and movement-related variables. The three-dimensional (3D) structure of a functional spatial cluster may be irregular. Moreover, it may undergo dynamic refining during learning as shown in our study. Considering that the cluster we observed with one-photon imaging represents a 2D sampling of a 3D structure, the shape and size of a cluster may vary widely depending on the position of the imaging lens relative to a cluster. A reliable estimation of the functional spatial cluster may require a larger sample size, a volumetric analysis with sampling at multiple focal planes, and an analysis method for handling irregular shapes.

Spatial Clusters for Reward- and Movement-Related Variables.

Previous studies examined the functional spatial clustering of striatal neurons related to voluntary movement (16, 20, 21). Here, we examined functional spatial clustering related to movement as well as reward. We found that spatial clusters for both movement- and reward-related variables are broadly organized. Broadly organized spatial clustering might be a general organizational principle for representing a variable in the striatum.

Even though characteristics of functional spatial clusters were similar between direct- and indirect-pathway neurons, they were not identical. Both direct- and indirect-pathway neurons increased activity during movement onset (forward motion, contralateral turn, and ipsilateral run) in the open-field test so that movement-related neural activity is positively correlated with each other and their functional clusters are grouped by hierarchical clustering. However, reward-related neural activity was more variable between direct- and indirect-pathway neurons. In particular, functional clusters for two closely related variables, namely outcome and value, were more similar in A2a-Cre than D1R-Cre mice so that hierarchical clustering classified the functional clusters for these two variables in the same group in A2a-Cre mice but in different groups in D1R-Cre mice. A permutation test also indicated a significant difference in the similarity of the functional clusters for these two variables between D1R-Cre and A2a-Cre mice. These results suggest a difference in the spatial organization of reward-related functional clusters between the direct and indirect pathways, which may be related to a more important role of the indirect than direct pathway in reward-based learning (26, 33). Furthermore, the size of the functional cluster estimated with the local cluster index varied across the animal groups, suggesting that functional clusters of the two pathways are not in perfect register. Note that the results obtained from the WT versus Cre-line mice should be interpreted with caution because the WT sample size is small (n = 3) and the calcium indicator (GCaMP6f) was likely to be expressed more profusely in striatal interneurons in WT compared with the Cre-line mice (24, 34). Also note a critical limitation of our finding, namely that direct- and indirect-pathway neuronal activity was recorded from different animals. Therefore, our observation needs to be confirmed in future studies by recording direct- and indirect-pathway neuronal activity simultaneously from the same animal (16).

Learning-Induced Functional Spatial Clusters.

One of the major objectives of this study was to test whether learning influences functional spatial clusters in the dorsal striatum. Our results show clearly that neurons coding learning-related variables (value and previous value) progressively form spatial clusters as cue–outcome contingency learning progresses. Note that clusters related to trial outcome did not change during the classical conditioning. Clusters related to movement-related variables during the free-exploration test did not change, either. That the effect of learning was specific to learning-related variables argues against the possibility our results are an outcome of nonspecific changes induced by behavioral training. Our findings are consistent with the previous physiological studies that found training-induced refining of striatal neuronal ensemble activity (35, 36). A recent calcium imaging study also found that striatal neuronal ensemble activity is refined with motor learning (10). The striatum plays an important role in reward-based learning (1, 3, 37, 38). Our results indicate that striatal neurons respond to initially neutral sensory cues in a way to form functional spatial clusters as neutral cues gain predictive values about upcoming rewards. It remains to be determined whether value-related functional spatial clusters observed in the present study are an outcome of neural circuit changes in the striatum, cortex, or both. Elucidating whether and how changes in striatal neural circuits give rise to value-related functional spatial clusters might provide an important clue for understanding striatal neural processes underlying reward-based learning.

Beyond the Classic Model of the Basal Ganglia.

We found that functional spatial clusters formed by D1R-expressing (direct pathway) and D2R-expressing (indirect pathway) striatal neurons are similar for both reward- and movement-related variables. When functional spatial clusters of direct- and indirect-pathway neurons were examined in the same animal, no clear difference was found between them (16). We also found both positive and negative relationships in striatal neuronal responses to two different variables, which was generally similar between the direct and indirect pathways. These features are unexpected and cannot be readily explained by the classic dichotomous model (5, 6, 39) or the selection-suppression model (13, 14) of the basal ganglia. Our and previous imaging studies (16, 21) suggest that, for both the direct and indirect pathways, multiple, overlapping groups of striatal neurons represent various movement- and reward-related variables. They also suggest that functional clusters of direct- and indirect-pathway neurons are intermingled rather than being spatially segregated. Previous studies have shown similar encoding of movement- and reward-related variables by direct- and indirect-pathway striatal neurons (11, 12, 26, 40). Thus, unlike as proposed by the conventional dichotomous models, it is likely that striatal control of behavior is shaped by overall activity patterns of direct- and indirect-pathway neurons across multiple functional clusters. Given that direct- and indirect-pathway neurons have opposing functional output connectivity (41), striatal control of behavior is likely to be determined by precise spatiotemporal dynamics of direct- and indirect-pathway neurons. This might increase the degree of freedom for flexible control of behavior compared with the conventional dichotomous models.

Materials and Methods

Experimental Models and Subject Details.

The experimental protocol was approved by the Animal Care and Use Committee of the Korea Advanced Institute of Science and Technology. C57BL/6J bacterial artificial chromosome transgenic mice expressing Cre recombinase under the control of the dopamine D1 receptor (D1R-Cre, EY217; Gene Expression Nervous System Atlas; n = 5) or adenosine A2a receptor (A2a-Cre, Tg [Adora2a-Cre] KG139Gsat/Mmucd; Mutant Mouse Resource & Research Centers; n = 5) were used along with C57BL/6J (The Jackson Laboratory) WT mice (n = 3). All mice were individually housed and the experiments were conducted in the dark phase of a 12-h light/dark cycle. The mice were restricted from access to water for 2 d prior to training in the classical conditioning task. Their body weights were monitored and maintained at >80% of ad libitum body weights. The mice were 10 to 18 wk old at the time of surgery.

Virus Injection and Lens Implantation.

The mouse’s head was fixed in the stereotaxic frame (Neurostar) under isoflurane anesthesia (1.5 to 2.0% [vol/vol] in 100% oxygen flow). Three small burr holes were made on the cranial surface (unilateral, counterbalanced across animals). The coordinates of the three cranial holes were (in millimeters) 0.7 anterior (A) and 2.2 lateral (L), 0.7 A and 1.8 L, and 0.2 A and 2.0 L to bregma. A drop of diluted adeno-associated virus carrying the gene for a genetically encoded calcium indicator (0.3 μL; AAV1-hsyn-GCaMP6f, 2.3 × 10¹² genome copies per milliliter for WT; AAV1-flex-GCaMP6f, 2.1 × 10¹² genome copies per milliliter for D1-Cre and A2a-Cre mice) was injected into each cranial hole at 2.3 mm below the brain surface using a glass pipette. The rate of virus injection was <0.1 μL/min and the injection pipette was held in place for 5 min after completing the injection to ensure sufficient diffusion of the virus. The cranial holes were covered with bone wax and the head skin was sealed with surgical suture (5-0 Vicryl and Vetbond adhesive; 3M). A GRIN lens was implanted under isoflurane anesthesia 2 wk after the virus injection surgery. A cranial hole (diameter, 1.2 to 1.3 mm; 0.45 mm A and 2.0 mm L to bregma) was made using a trephine drill bit (diameter, 1 mm), dura and bone debris were gently removed, and ∼2 mm of cortical tissue was aspirated using a blunt 26G needle and a custom-built microsuction device. Then, a GRIN lens (diameter, 1 mm) that was placed in a custom-built lens holder was lowered into the target site at the speed of 100 μm/min. Care was taken to prevent bleeding during lens implantation to avoid the formation of blood clots that may block the view of the lens. The lens and head-fixation plate were firmly fixed on the cranial surface using adhesive resin (Sun Medical), and the upper surface of the implanted lens was covered with a small plastic cap to prevent potential physical damage. The expression of GCaMP6f in the target site was monitored regularly after GRIN lens implantation with an nVista microscope (Inscopix). When the mouse showed sufficient expression of GCaMP6f in the target site, the baseplate for the nVista microscope was attached to the mouse’s head using ultraviolet glue (Edmund Optics) and covered with black orthodontic resin (Lang).

Behavioral Task.

The mice performed a probabilistic classical conditioning task under head fixation in a custom-built apparatus (Fig. 1) and then free exploration in a square arena (30 × 30 × 30 cm; Fig. 2) with calcium signals being monitored during the tasks. In the classical conditioning task, 0.5 s following the delivery of a trial-onset signal (a clicking sound from a solenoid valve), a head-fixed mouse was presented with one of three different odor cues (citral, isoamyl acetate, and L-carvone diluted 1/1,000 [vol/vol] in mineral oil) for 1 s. Pulses of odor were delivered through a custom-designed olfactometer in a pseudorandom order. Each odor cue was followed by a different probability (80, 50, or 20%) of water (5 μL) delivery following a delay of 1 s. A buzzer sound (2.3 kHz, 100 ms) was delivered in unrewarded trials. The animal’s licking response was detected with an infrared-beam sensor. Trials in which the mouse did not consume the delivered water until the beginning of the next trial were considered incomplete. These trials (no water consumption) and their next trials (unconsumed water remained available) were excluded from the analysis. All behavioral events were recorded with the Cheetah Data Acquisition System (Neuralynx). After completion of the classical conditioning task (total 320 trials per day), the mouse was gently moved to the open-field arena (square box, 30 × 30 × 30 cm) and allowed to freely explore the arena. Calcium imaging was performed for 10 min in the arena after 5 min of an intermission. The mice were initially habituated to the experimental settings (head fixation in the classical conditioning apparatus and free exploration in the open-field arena) for 3 d (1 h/d) before behavioral training began. The mice were then trained until they showed differential delay-period licking rates to three different odor cues (one-way ANOVA followed by post hoc Tukey test, P < 0.01 for all comparisons). Once they reached this criterion, they performed the tasks for 2 additional days. Calcium signals were monitored from the first day of training.

Calcium Imaging.

Calcium fluorescent images were acquired through the implanted GRIN lens and nVista microscope (Inscopix) at 20 frames per second with 0.1 to 0.2 mW/mm² of light-emitting diode power. External pulse signals (5-V transistor–transistor logic) generated by a custom-designed pulse generator were delivered to the nVista and Cheetah acquisition systems to synchronize calcium images and task events. The synchronizing pulse signal was delivered at the initiation of each trial in the classical conditioning task and every second in the open-field test.

Image Processing.

All image processing was performed with Inscopix Data Processing Software (version 1.1). The raw image of each frame was translated into a 16-bit TIFF image with an image decompressor (Inscopix) and dropped frames and defective pixels were corrected with preprocessing. Spatial downsampling by a factor of 4 was applied to the images for efficient data processing. No temporal downsampling was used. Motion correction was performed using a prominent vascular structure as a reference frame (Movie S1). The video was then exported in TIFF format and further analyzed with the CNMF-E algorithm (25) which is used widely for background fluctuation correction of one-photon calcium imaging data (16, 21).

Regression Analysis.

Neural activity related to value, outcome, previous value, and previous outcome was examined using the following regression model:

$$F\left(t\right)=a_0+a_1\cdot O\left(t\right)+a_2\cdot V\left(t\right)+a_3\cdot L\left(t\right)+a_4\cdot O\left(t-1\right)+a_5\cdot V\left(t-1\right),$$ [1]

where F(t) represents a calcium trace in a given analysis window, O(t), V(t), and L(t) are outcome (1 if rewarded and 0 if unrewarded), value (reward probability; 0.2, 0.5, or 0.8), and lick rates, respectively, in each analysis time bin in trial t, and a₀ to a₅ are regression coefficients. We analyzed neural spikes in a moving window of 0.5 s that was advanced in 0.1-s steps to examine temporal profiles of neural activity related to these variables. There were moderate correlations between value and anticipatory lick rate during the delay period (0.53 ± 0.17) and between outcome and lick rate during the outcome period (first 1 s; 0.27 ± 0.10, mean ± SD across sessions). Their variance inflation factors, which are measures for multicollinearity (42), were 1.47 ± 0.40 and 1.1 ± 0.1, respectively.

In the regression analysis of the open-field locomotion test, only those epochs with a given behavioral category longer than 1 s (Fig. 2D) were used. The mean of randomly selected (1,000 times) calcium traces during 1-s time epochs during the open-field test was used as the baseline trace. Neural activity related to each type of behavior (forward motion, stop, ipsilateral turn, and contralateral turn) was examined separately as the following:

$$B\left(t\right)=a_0+a_1\cdot C\left(t\right),$$ [2]

where B(t) represents the mean calcium trace during a 1-s epoch, C(t) is a behavior index (0 for baseline trace and 1 for a given behavioral category), and a₀ and a₁ are regression coefficients.

Video Tracking.

The animal’s behavior during the open-field test was recorded with a high-definition video camera (30.0 Hz; Hauppauge) located 30 cm above the arena. The head, center, and proximal part of the mouse tail were tracked using an open-source, deep learning-based tracking algorithm (DeepLabCut) (43). The head was defined as the lower part of the nVista microscope attached to the skull of the mouse (Fig. 2B). For the training of the algorithm, 18 or 19 frames were selected from ∼10 min (8,500 to 9,500 frames) of video data, and head, center, and the proximal part of the tail were labeled manually. After an episode of supervised training, the estimated marker positions were validated, corrected, and trained further to achieve satisfactory tracking. The training was performed at least 200,000 iterations until the performance of position estimation reached a plateau. Then, the head, center, and the proximal part of the tail positions were acquired using the trained algorithm, head directions were determined by calculating the center-to-head position angles (Fig. 2B), and linear and angular velocities were calculated and smoothed with a Gaussian filter (σ = 10 ms). Forward motion was defined as the center velocity exceeding 2 SDs above the mean center velocity. Contralateral and ipsilateral turns were defined as the head angular velocity exceeding 2 SDs above the mean head angular velocities toward the contralateral and ipsilateral sides, respectively, to the lens-implanted hemisphere. If a forward-motion event overlapped with a contra/ipsilateral turn event, the event was defined as the contra/ipsilateral turn. Initiation and termination of each behavioral event were defined as when the velocity (center or head angle) crossed 0.5 SD above and below the mean velocity, respectively. A complete stop event was defined as when each center, head, and tail velocity fell 0.5 SD below the mean of the corresponding velocity. All of the other epochs not included in the above four categories were defined as the “other” state.

Cross-Correlation of Calcium Signals.

We calculated Pearson’s correlation coefficients of calcium transient signals for all pairs of simultaneous recorded striatal neurons in each behavioral task (classical conditioning and open-field test) (21). We then calculated the mean coefficient as a function of the distance between two neurons using a sliding window of 30 μm that was advanced in 1-μm steps. The baseline correlation was defined as the mean correlation coefficient between 300 and 500 μm from the center. The distance from the center to the first point where the correlation curve crosses the baseline (correlation decay distance) was determined for each task for each animal.

Spatial Clustering Algorithm.

We calculated the mean absolute difference in the regression coefficient between a reference neuron and all other neurons within a circular region of interest (ROI) centered at the reference neuron. We repeated this procedure while designating each and every neuron in the ROI as the reference neuron, and averaged the resulting local-difference values to obtain a “local difference” for a given ROI (SI Appendix, Fig. S6A). We calculated the local difference as we increased the radius of the ROI from 30 to 600 μm in 15-μm steps. We also calculated the local difference after randomly permuting spatial positions of the recorded neurons, which was repeated 1,000 times. We then calculated the area between the local-difference curve of the original data and the averaged local-difference curve of the 1,000 spatial permutation data (“local cluster index”; SI Appendix, Fig. S6B). We determined the P value of the original local cluster index based on a cumulative probability function of the local cluster index obtained from the 1,000 permuted data (SI Appendix, Fig. S6C). Cluster size was defined as the size of the ROI where the local-difference curve of the original data crosses 2 SDs of that of the shuffled data (SI Appendix, Fig. S6B).

Relationship between Different Spatial Clusters.

The relationship between two different functional spatial clusters was quantified by calculating spatial correlation (neuron-by-neuron correlation) between two spatial maps of regression coefficients. To test whether spatial cluster relationships between different variables vary significantly between D1R-Cre and A2a-Cre mice, we randomly assigned 10 mice (5 D1R-Cre and 5 A2a-Cre) to two groups and calculated the spatial correlation between two different variables. This was repeated 1,000 times, and P values were determined based on the frequency of the original spatial correlation difference between D1R-Cre and A2a-Cre exceeding the spatial correlation difference of the permuted data.

Statistical Analysis.

One-way ANOVA and Tukey post hoc tests were used to compare licking responses to three different odor cues to compare cluster size among different animal groups and to compare absolute spatial-map correlation between variables within and across behavioral tasks. Two-way ANOVA and Tukey post hoc tests were used to compare correlation decay distance and averaged cluster size across behavioral contexts and animal groups. χ² tests were used to compare fractions of direct- and indirect-pathway striatal neurons coding a given variable. Permutation tests were used to test the statistical significance of spatial clustering and to compare the relationship between spatial clusters for different variables across D1R-Cre and A2a-Cre mice (see above for detailed procedures). Student’s t tests were used for statistical comparisons of all other measures. All statistical tests were two-tailed and P values < 0.05 were considered significant unless otherwise noted. All data are expressed as means ± SEM unless otherwise noted.

Data Availability.

Calcium traces from striatal neurons and custom-written code for data analysis reported in this paper have been deposited in GitHub (https://github.com/LuckyFace/Str_functional_cluster).