Pre-existing visual responses in a projection-defined dopamine population explain individual learning trajectories

Summary

A key challenge of learning a new task is that the environment is high dimensional—there are many different sensory features and possible actions, with typically only a small reward-relevant subset. Although animals can learn to perform complex tasks that involve arbitrary associations between stimuli, actions, and rewards,^1–6 a consistent and striking result across varied experimental paradigms is that in initially acquiring such tasks, large differences between individuals are apparent in the learning process.^7–12 What neural mechanisms contribute to initial task acquisition, and why do some individuals learn a new task much more quickly than others? To address these questions, we recorded longitudinally from dopaminergic (DA) axon terminals in mice learning a visual decision-making task.⁷ Across striatum, DA responses tracked idiosyncratic and side-specific learning trajectories, consistent with widespread reward prediction error coding across DA terminals. However, even before any rewards were delivered, contralateral-side-specific visual responses were present in DA terminals, primarily in the dorsomedial striatum (DMS). These pre-existing responses predicted the extent of learning for contralateral stimuli. Moreover, activation of these terminals improved contralateral performance. Thus, the initial conditions of a projection-specific and feature-specific DA signal help explain individual learning trajectories. More broadly, this work suggests that functional heterogeneity across DA projections may serve to bias target regions toward learning about different subsets of task features, providing a potential mechanism to address the dimensionality of the initial task learning problem.

Graphical abstract.

Results

Idiosyncratic and side-specific learning of a standardized visual decision-making task

To study individual differences in learning, we leveraged a standardized visual decision-making task.^7,13–15 In this task, at the beginning of each trial, a visual grating was presented on a screen on either the right or left side. Mice received a reward by turning a steering wheel with their front paws in the direction that centered the grating on the screen (Figures 1A and S1A−S1C). Correct wheel turns were rewarded with 3 μL sucrose water, whereas incorrect ones led to a short timeout (2 s) and white noise (0.5 s). Across trials, the visual gratings varied in contrast (100%, 25%, 12%, and 6.25%), with contrasts randomly and uniformly selected. To maximize interpretability of the neural recordings over the course of learning, we did not use a shaping procedure,⁷ a debiasing protocol,^7,16 or other experimenter interventions.

Figure 1. Idiosyncratic and side-specific learning trajectories.

(A) Schematic of the task. On each trial, a Gabor patch of a different contrast (6.12%, 12.5%, 25%, or 100%) is presented on the right or left side of a screen. Centering the patch with a steering wheel leads to a small water reward, whereas moving it out of the screen results in a short timeout (2 s) and white noise (0.5 s).

(B) Accuracy (fraction of trials rewarded) across training sessions.

(C) Probability of right choices across training sessions. In (B) and (C), each line represents one mouse, colored by their mean accuracy in sessions 16–20.

(D) Schematic of the behavioral model. Choice (left or right) on each trial is predicted with a logistic function based on weighting the contrast of the right and left visual stimulus (β_right and β_left), a bias term (β_bias) coded such that positive values indicate rightward choice, and a choice history kernel. Weights evolved across sessions (see STAR Methods for details).

(E) Psychometric curves (“data”) and model fits (“model”) from 3 example mice on the first, middle, and last session of training. Lines and shading represent mean ± SEM.

(F) Model weights across training for the same mice from (E). Lines and shading represent mean and 95% confidence intervals.

(G) Early β_bias (average of sessions 1–5) for all the mice, showing the subdivisions used in subsequent panels between mice with weak, left, or right initial bias.

(H–J) Average trajectories of bias, right and left stimulus weights across training for mice subdivided by their initial bias as shown in (G). Lines and shading represent mean ± SEM across mice.

(K and L) Relationship between early β_bias (average of sessions 1–5) and the late difference in stimulus sensitivity weights (β_right - β_left for sessions 16–20). r = 0.417, p = 0.0007.

(I) Relationship between early β_bias (sessions 1–5) and late β_bias (sessions 16–20). R = 0.174, p = 0.522.

In (K) and (L), each dot is a mouse. Correlation and p values from robust regression. **p < 0.01; ns, not significant.

Across all panels, n = 22 mice.

See also Figure S1.

Mice showed a large degree of variability in their learning trajectories, both in terms of the number of sessions to reach high accuracy (Figure 1B) as well as in their probability of choosing left vs. right wheel turns (Figure 1C). Whereas some mice selected both options (right and left) to an equal extent from the beginning of training, others appeared to prefer one side or the other.

To quantify choices across learning, we constructed a behavioral model (Figure 1D) that described each session’s contrast-dependent choices (i.e., psychometric curve) based on weights that evolved across sessions: a weight on the visual stimulus contrast on the left (β_left) or right (β_right) that captured the stimulus-dependent tendency to make left or right choices, a choice bias (β_bias) that captured the tendency to choose one side or the other, irrespective of the stimulus, and a choice history regressor (β_hist) that captured the tendency to repeat previous choices (see STAR Methods for model details). This model successfully reproduced the diverse psychometric curves observed across learning (Figure 1E; 3 example sessions from 3 example mice) and allowed us to isolate the contribution of each variable to each mouse’s behavior across learning (Figure 1F).

The model revealed that whereas some mice learned to increasingly weight visual stimuli similarly on both sides as training progressed (e.g., show an increase across training in both β_right and β_left in Figure 1F, top), many instead displayed idiosyncratic learning trajectories where they preferentially weighted one stimulus vs. the other (e.g., large β_right vs. small β_left in Figure 1F, middle; large β_left vs. small β_right in Figure 1F, bottom). This side-specific learning is consistent with the choice asymmetries evident in the raw data (Figure 1C), though note that the model enables distinguishing between stimulus-contrast sensitivity (the stimulus weight) and a stimulus-independent choice bias. The side specificity of stimulus learning at the end of the training (β_right - β_left) was predicted by the bias parameter β_bias at the beginning of training, as evidenced both by plotting the stimulus weights based on groups defined by the level of initial bias (Figures 1G–1J) and by correlating initial bias with the difference between the final stimulus sensitivity weights (Figure 1K). By contrast, early β_bias did not predict final β_bias (Figure 1L).

Based on these observations, we concluded that mice display idiosyncratic side-specific learning trajectories that could be predicted by initial bias. We next explored how individual differences in striatal dopaminergic (DA) signals might explain these individual learning patterns.

Across the striatum, contrast-dependent DA visual responses tracked side-specific individual learning trajectories

We recorded striatal DA signals longitudinally over the course of learning using fiber photometry. To ensure consistent expression of the activity indicator across time and individuals, we used double-transgenic mice that expressed GCaMP6f in DA neurons (DAT-Cre × Ai148). Before training, optical fibers were implanted into the following striatal subregions of each mouse: dorsomedial striatum (DMS), dorsolateral striatum (DLS), and nucleus accumbens core (NAc; Figures 2A and S2A−S2C). Each subregion was recorded unilaterally, with a mixture of left and right hemispheres across subregions within each mouse and across mice for each subregion. During each session for each mouse, we recorded simultaneously from all 3 subregions.

Figure 2. In DA terminals across the striatum, contrast-dependent visual responses track individual side-specific learning trajectories.

(A) Experimental strategy used for collecting the fiber photometry data from DA terminals. Left: schematic of the recorded projections using the GCamPG6f × DAT::Cre mouse line. Right: example histology. Scale bar, 1 mm.

(B) Contralateral stimulus response kernels from an example mouse on an example session.

(C) Z scored dF/F (solid line) and predictions from the encoding model (dashed line) on 5 different trials for an example mouse on an example session. R² is the variance explained across the session within all trial epochs (from stimulus onset to 1 s after feedback).

(D) Stimulus response magnitudes (L2-norm) in each region and session, averaged across mice, for contralateral (top) and ipsilateral (bottom) stimuli. Lines and shading represent mean ± SEM.

(E) Trajectories of the contrast-dependence of neural stimulus response magnitudes (“neural”; difference in L2-norm for 100% and 6.25% contrast) and the behavioral stimulus choice weights (“behavioral”) for contralateral (top) and ipsilateral (bottom) stimuli (from an example mouse in which DMS is recorded on one hemisphere and DLS/NAc on the other).

(F) Correlations of the neural and behavioral trajectories as shown in (E). p values calculated with t tests. *p < 0.05, **p < 0.01, ***p < 0.001; ns, not significant. See Table S1 for statistical details for (F). n = 22 mice in (D) and (F).

See also Figures S2 and S3.

Given that DA neurons encode reward-predictive cues,¹⁷ behavioral and DA sensitivity to the stimuli should be expected to increase during task acquisition, as animals learn that the stimuli are predictive of reward.^8,9,18–21 To test this, we quantified DA sensitivity to the visual stimuli by fitting the normalized fluorescence data with a linear encoding model for each session and subregion (model schematic in Figure S3A). This allowed us to estimate stimulus response kernels (Figure 2B), which reflect the contribution of the visual stimulus to the neural signals while (linearly) accounting for other task events (actions and outcomes). This model could accurately capture moment-by-moment fluctuations in fluorescence on each trial (Figures 2C and S3B) as well as the trial-averaged PSTHs (Figures S3C and S3D).

Averaged across mice, across all regions, the magnitude of these stimulus responses increased across sessions, as mice learned that the stimuli were predictive of reward (Figure 2D; L2-norm of stimulus response kernels). Moreover, stronger stimulus responses emerged to the stronger contrasts, consistent with stronger contrasts becoming more predictive of reward as the animals acquired the task (i.e., increasing stimulus-sensitivity behavioral weight with training in Figures 1F–1J). Whereas stimulus responses in DLS and NAc dopamine were similar for contralateral and ipsilateral stimuli, the DMS had much stronger stimulus responses for contralateral than ipsilateral stimuli (Figure 2D).^22,23

How do these stimulus responses relate to each mouse’s idiosyncratic and side-specific learning profiles (Figure 1)? For each mouse, across sessions, we correlated the contrast dependence of the behavioral trajectories for stimuli on one side (stimulus contrast weight from the behavioral model; Figure 1) with the contrast dependence of the neural trajectories for stimuli on the same side (difference in L2-norm for highest vs. lowest contrast stimulus kernel). There was a strong correlation between the behavioral and neural measures across all regions (example trajectories, Figure 2E; neural and behavioral correlations for all mice, Figure 2F). Whereas DLS and NAc dopamine showed this correlation for both ipsilateral and contralateral stimuli, in the DMS, this correlation was only apparent for contralateral stimuli (Figure 2F), presumably reflecting the contralateral specificity of the stimulus responses themselves (Figure 2D).

Thus, across the striatum, we observed the co-evolution during task acquisition of side-specific dopamine and behavioral sensitivity to the visual stimuli, consistent with reward prediction error signaling in DA neurons.

Pre-existing visual responses in DMS DA terminals predict learning on the contralateral side

Although the striatum-wide correlations between DA trajectories and learning trajectories were striking (Figure 2), they do not clarify whether there are DA signals that precede behavioral changes that could potentially explain individual differences in task acquisition. Notably, the DMS (unlike the other regions) exhibited contrast-dependent responses to the visual stimuli from the very first session (Figure 2D). To determine whether these signals existed before training or whether they emerged during the first session, we examined DA responses during an earlier pre-exposure session (“session 0,” before the first session) in which the visual gratings were presented but mice did not receive rewards nor could they turn the wheel (Figure 3A).

Figure 3. Pre-existing visual responses in DMS DA terminals predict side-specific learning trajectories.

(A) Schematic of the stimulus pre-exposure session before training (“session 0”).

(B) Stimulus response kernels in the NAc, DMS, and DLS for contralateral and ipsilateral stimuli of each contrast, averaged across mice, during session 0 (pre-exposure). Lines and shading represent mean ± SEM.

(C) Heatmap of stimulus responses on session 0 to 100% contrast stimuli in the DMS for the first 25 trials, averaged across mice.

(D) Histogram across mice of contrast-dependent contralateral stimulus responses on session 0, quantified as the difference in the L2-norm of the highest and lowest contrast contralateral stimulus, colored by a median split.

(E) Contralateral stimulus sensitivity weights from the behavioral model for mice with strong vs. weak contralateral contrast-dependent stimulus responses during session 0 (subdivision of mice shown in D). Lines and shading represent mean ± SEM. ***p < 0.001 for the interaction between DMS stimulus response on session 0 and session in a two-way ANOVA (see Data S1.1–S3.2 for model details and full results).

(F) Same as (E), except for the ipsilateral stimulus weight from the behavioral model. No significant interaction (ns) between DMS stimulus response on session 0 and session (see Data S1.3–S3.4 for model details and full results).

(G) Same as (E) and (F), but for the bias weights from the behavioral model (transformed such that positive means contralateral bias). No significant interaction (ns) between DMS stimulus response on session 0 and session (see Data S1.5–S3.6 for model details and full results). In all panels, n = 18 mice.

(H) Same as (G), but for the choice history weights from the behavioral model. No significant interaction (ns) between DMS stimulus response on session 0 and session (see Data S1.7 and 3.8 for model details and full results).

See also Figures S2, S4, and S5 and Data S1.

In the DMS, but not the NAc or DLS, DA terminals had contrast-dependent visual responses during stimulus pre-exposure before training (session 0; DMS, Figures 3B and 3C; NAc and DLS, Figure S4). These DMS DA responses were contralateral-side specific, similar to the pattern observed throughout training (Figure 2D). The stronger response to higher contrast stimuli may be interpreted as a salience-related signal.^18,24–35 Although dopamine has been associated with novelty coding,^{18,32,34–37} these pre-existing visual responses did not attenuate during the 25 presentations of each stimulus (Figure 3C).

A previous theoretical account³⁸ suggests that novelty- or salience-related DA signals could provide animals with a “bonus” (or head start) in forming stimulus-reward associations. Therefore, we wondered whether variability in these DMS-specific pre-existing DA responses to the visual stimuli might predict side-specific differences across individuals in learning trajectories.

Indeed, we observed a striking relationship between these pre-exposure stimulus responses in DMS DA and individual differences in learning about stimuli on the side contralateral to the recording. For visualization purposes, we median-split mice into 2 groups based on their pre-exposure visual responses (Figure 3D; “strong” vs. “weak” contrast-dependent contralateral DMS DA responses on session 0) and plotted the trajectory of the behavioral model weights during subsequent task training in each group (Figures 3E−3H). Over the course of training, the animals with larger pre-exposure visual stimulus responses developed larger contralateral behavioral stimulus sensitivity weights (Figure 3E). By contrast, the pre-exposure DMS DA visual responses were not predictive of the bias, choice history, or ipsilateral stimulus sensitivity behavioral weights (Figures 3F−3H). Moreover, in the NAc and DLS, pre-exposure visual responses did not significantly correlate with behavioral weight trajectories (Figures S4 and S5), consistent with the very weak or non-existent pre-exposure visual responses in those regions (Figure 3B).

Thus, pre-existing visual responses in DMS dopamine (but not NAc or DLS dopamine) predict individual differences in learning about the contralateral visual stimuli, even before any training.

Activation of DMS DA terminals during contralateral stimulus presentation improves side-specific performance

The strength of pre-existing visual responses in DMS DA terminals predicted side-specific learning (Figures 3D, 3E, and S5). Could these visual signals causally impact side-specific performance during learning? Although it is clear that DA signals at the time of outcome reinforce previous actions,^39–43 whether DA signals at the time of a preceding cue impact learning is less clear.^{33,39,40,44–46}

To test for a causal role of visual-stimulus-related DA signals in the DMS, we performed brief unilateral optogenetic stimulation of DMS dopamine terminals at the presentation of the contralateral visual stimulus throughout training (Figures 4A, 4B, and S2D). The stimulation, which terminated before the outcome period (Figures S1D and S1E), led to a significant improvement in performance between contralateral vs. ipsilateral trials for the opsin vs. control groups, which grew over early training (Figure 4C). We therefore concluded that DA signals in the DMS could improve performance for contralateral stimuli.

Figure 4. Stimulating DMS DA terminals at the onset of contralateral stimulus presentation improves side-specific performance.

(A) Schematic of the optogenetic stimulation of DMS DA terminals. Mice either expressed ChRmine or a control construct in DA neurons. DA terminals in the DMS were optogenetically stimulated unilaterally (532 nm, 0.2 s burst duration, 5 ms pulse width, 20 Hz pulses, ~0.25 mW) at the onset of the contralateral stimulus presentation throughout training.

(B) Example histology image of optical fiber location and terminal expression of ChRmine-mScarlet. Scale bar, 900 μm.

(C) Comparison of performance for contralateral and ipsilateral stimulus trials in control (n = 7, left) and ChRmine (n = 6, right) mice. Lines and shading represent mean ± SEM. *p < 0.05 for cohort (ChRmine/YFP) and side (contra/ipsi) interaction in three-way ANOVA with cohort (ChRmine/YFP), day, and side (contralateral/ipsilateral) as factors (see Table S2 for model details and full results).

See also Figures S1 and S2 and Table S2.

Discussion

As mice learn to perform a visual decision-making task, variation across individuals in learning trajectories closely tracks visual responses in DA terminals across striatum, consistent with widespread reward prediction error coding (Figures 1 and 2). In contrast to these striatum-wide patterns, prior to any reward or training, pre-existing contralateral-side-preferring visual responses are present in DMS DA terminals, and these signals predict and help explain contralateral-side-specific learning trajectories (Figures 3 and 4). This work is significant in suggesting that (1) the initial conditions of the DA system are important in explaining individual differences in learning and (2) feature- and projection-specific DA signals could be a mechanism to simplify the problem of initial task acquisition.

Pre-existing visual responses in DMS-projecting DA neurons could serve to simplify initial task acquisition

A major reason that initial task acquisition is challenging is the issue of credit assignment: in virtually any task, even nominally simple ones, there are multiple possible dimensions of the environment that an animal could try to learn about, with most being irrelevant to reward. In the case of the visual decision-making task used here, mice need to learn that the side-specific relationship between visual stimuli and actions is what leads to reward, whereas other stimuli, actions, or stimulus-action relationships do not (including many high-dimensional, uncontrolled incidental features of the environment⁴⁷).

Our data suggest that pre-existing side-specific and projection-specific visual DA responses could serve to decrease the dimensionality of this learning problem. Different striatal subcircuits could be initialized to more easily learn reward associations for different subsets of stimuli or actions. Although our data reveal that the pre-existing response in DA terminals in the DMS to contralateral visual stimuli explains learning for stimuli that are contralateral, other striatal circuits may have pre-existing DA sensory responses to other modalities that could, in turn, predispose learning in favor of those associations. For example, DA terminals in the tail of the striatum, an area critical for auditory-motor learning,^48–51 have been shown to have Please cite this article in press as: pre-existing auditory responses.⁵² Under our proposed framework, these pre-existing auditory DA responses may contribute to auditory-motor learning, much like the visual responses we report here appear to contribute to visual-motor learning. However, this possibility remains to be directly tested.

Indeed, such hypothetical specialization simplifies the curse of dimensionality insofar as any given subcircuit would be initially biased to quickly discovering simple associations, appropriate for tasks in which a small set of stimuli are associated with reward. Of course, there is no free lunch. Such a bias for sparse solutions would not help and might indeed hinder detecting contingencies that depend on more complex (e.g., multimodal) combinations of features. This reasoning also leads to testable predictions about which types of contingencies are most easily learnable: those that well match the feature selectivity of a projection-defined DA population. This may explain why, in the present data, animals often learn more quickly to respond to stimuli on one side or the other, independently from those on the other side, rather than the fuller response rule combining both sides (Figure 1).

This framework is consistent with a classic idea³⁸ about the role of salience signals in DA neurons^18,24–35 in providing an optimistic bonus to support learning. However, it adds to this idea by suggesting that there are a number of separate and specialized circuits rather than a single, global prediction error. Specifically, the finding that a visual contralateral-side-specific signal in the DMS supports contralateral-side-specific learning is what is most novel about the current work.

Initial conditions of the DMS DA system can explain individual differences in task acquisition

Our data also help to explain why different individuals learn the same task much more quickly than others (Figure 1). The variation across individuals in the pre-existing visual response in DMS DA signals before task training and the presentation of any reward predicts individual differences in acquisition of the visual-motor task on the contralateral side (Figure 3). This supports the idea that the pre-existing DMS DA visual responses facilitate reward learning and highlights the importance of the initial conditions of the DA system in understanding the emergence of individual differences in behavior.

Relationship to prior experimental work on the DMS DA system

Our results complement prior work characterizing the DMS DA system, demonstrating contralateral response preferences,^22,23,53 visual responses,^22,52 and stimulus-value-related responses.⁵⁴ Our work adds to the literature primarily by revealing a pre-existing sensory-feature-specific and projection-specific DA signal that explains later learning (Figures 3 and 4).

On the other hand, the relationship between our work and recent work implicating DMS dopamine in individual differences in the development of a habitual⁵⁵ or punishment-resistant⁵⁶ behavioral strategy is not yet clear. Although it seems possible that a pre-existing sensory-feature-specific DA signal could contribute to the emergence of those strategies, more work is required to clarify a potential connection.

Whereas here we focus on visual responses in DMS DA, previous work from our group and others has identified putative action correlates in this population.^23,53,57,58 Lateralized action-related DMS DA responses may relate to head-orienting behavior, as DMS is a target of frontal-orienting fields,^59,60 has been implicated in tasks with orienting behavioral outputs,^61–63 and the DMS DA signal itself reverses with changes in orientation.⁵³ For this reason, in the present task, action responses may be at least partially obscured by head-fixation, which prevents orienting behavior. Regardless, we are confident that the contrast-dependent responses examined here are visual, as we isolated them from wheel movement with an encoding model and, furthermore, confirmed their presence during stimulus pre-exposure in the absence of task-related movements (Figures 3A and 3B).

Relationship to the feature-specific RPE model of DA heterogeneity

The presence of a visual-feature-specific DMS DA signal is consistent with a recent model that proposes that response variation across DA neurons can be explained, at least in part, by differences in the feature representations in the inputs that are used to calculate reward prediction errors (“feature-specific reward prediction error model”⁴⁷). In this framework, different dopamine neurons calculate different reward prediction errors based on different subsets of the full feature space, based on the corticostriatal inputs they preferentially receive. This model thus predicts similar feature selectivity of the DMS-projecting DA neurons relative to the DMS neurons themselves, assuming an anatomical arrangement where the DMS projects (indirectly or directly) primarily to DMS-projecting DA neurons.⁶⁴ Consistent with this prediction, the DMS receives direct visual cortical inputs⁶⁵ and has visual responses before task training.⁶⁶ Alternatively, the visual responses we observed in DMS DA neurons could originate from the superior colliculus rather than the basal ganglia.^67–69

Resource Availability

Lead contact

Further information and requests should be directed to and will be fulfilled by the lead contact, Ilana Witten (iwitten@princeton.edu).

Materials availability

This study did not generate new unique reagents.

Star⋆Methods

Detailed methods are provided in the online version of this paper and include the following:

• KEY RESOURCES TABLE

• EXPERIMENTAL MODEL AND STUDY PARTICIPANT DETAILS

  • ∘Mice

• METHOD DETAILS

  • ∘Surgery
  • ∘Behavioral task
  • ∘Fiber photometry
  • ∘Histology
  • ∘Optogenetic stimulation
  • ∘Behavioral model
  • ∘Neural model

• QUANTIFICATION AND STATISTICAL ANALYSIS

  • ∘Linear Mixed Models variable coding

Star⋆Methods

Key Resources Table

REAGENT or RESOURCE	SOURCE	IDENTIFIER
Bacterial and virus strains
AAV2/5-EF1a-DIO-ChRmine- mScarlet-WPRE-hGHpA	Princeton Neuroscience Institute Viral Core	N/A
Deposited data
Photometry and behavioral data	Generated by this study	https://github.com/YSanchezAraujo/PRE_VIS_DA_CURR_BIO/blob/main/alex_figure_1_4_supp_1/download_data.py. (This script downloads all raw and processed data from International Brain Lab servers) Documentation for loading the dataset: https://int-brain-lab.github.io/iblenv/notebooks_external/loading_photometry_data.html
Experimental models: Organisms/strains
B6.SJL-S/c6a3tm1.1(cre)Bkmn/J (DAT∷IRES-Cre)	The Jackson Laboratory	Stock # 006660; RRID: IMSR_JAX:006660
B6.Cg-lgs7tm148.1(tetO-GCaMPSf,CAG-tTA2)Hze/J (Ai148)	The Jackson Laboratory	Stock # 030328; RRID: IMSR_JAX:030328
Software and algorithms
Neural Encoding model	Generated by this study	https://github.com/YSanchezAraujo/PRE_VIS_DA_CURR_BIO/blob/main/pipeline
Behavioral model	Generated by this study	https://github.com/YSanchezAraujo/PRE_VIS_DA_CURR_BIO/blob/main/pipeline
Figure visualization	Generated by this study	https://github.com/YSanchezAraujo/PRE_VIS_DA_CURR_BIO/tree/main/pipeline/figures
Statistical analyses	Generated by this study	https://github.com/YSanchezAraujo/PRE_VIS_DA_CURR_BIO/blob/main/pipeline
Other
Fibers for optogenetics	Thor Labs	BFL37-300
Ferrules for optogenetics	Precision Fiber Products	MM-FER-2006SS-330
Fibers for fiber photometry	Neurophotometrics	FOC_BF_200um/1.25mm
IBL Behavior Rig	International Brain Laboratory	https://figshare.com/articles/preprint/A_standardized_and_reproducible_method_to_measure_decision-making_in_mice_Appendix_3_IBL_protocol_for_setting_up_the_behavioral_training_rig/11634732
Multi-fiber photometry rig	Neurophotometrics	FP3002
Branching Bundle Patchcord - Low Autofluorescence	Doric Lenses Inc.	BBP(3)_200/220/900-0.37_2m_SMA-3xMF1.25_LAF

Experimental Model and Study Participant Details

Mice

For the fiber photometry experiments (Figures 1, 2, and 3), a total of 22 mice (n=14 male and n=8 female) were used from a cross of DAT::IRES-Cre mice (JAX 006660) and the GCaMP6f reporter line Ai148 (JAX 030328). For the optogenetic experiments (Figure 4), we used a total of 13 DAT::IRES-Cre mice (n=4 male and n=9 female). Mice were maintained on a reversed 12 h light cycle and experiments were formed on their dark cycle. All mice used were 3-4 months old at the start of training. All experimental procedures were conducted in accordance with guidelines from the National Institutes of Health and were reviewed by the Institutional and Animal Care Use Committee at Princeton University.

Method Details

Surgery

Prior to the start of the surgery, mice received a preoperative antibiotic (5 mg/kg Baytril) and analgesic (10 mg/kg Ketofen). Postoperative analgesic (10 mg/kg Ketofen) was administered daily for 3 days from the day of the surgery.

Headbar implantation

For all stereotaxic surgeries, mice underwent sterile stereotaxic surgery under anesthesia (5% isoflurane for induction, 1.5-2% for maintenance). Briefly, the scalp and underlying periosteum was removed. Bregma and lambda were leveled, and a small steel headbar was centered at -6.9 mm Anterior-Posterior relative to bregma⁷⁰ and cemented to the skull with Metabond (Parkell). Headbar implantation was followed by virus infusion and/or optical fiber implantation (see sections below).

Optical fiber implantation

For fiber photometry experiments (data shown in Figures 1, 2, and 3), low-autofluorescence optical fibers encased in a ferrule (0.37 NA, ø200 μm core, 1.25mm ferrule, Neurophotometrics) were implanted at each of the following locations (fiber tip location relative to bregma):

• DLS: 2.6 mm (Medio-lateral, M-L), 0 mm (Anterior-Posterior, A-P), -2.8 mm (Dorso-ventral, D-V).

• DMS: 1.5 mm M-L, 0.74 mm A-P, -2.4 mm D-V.

• NAc: 1 mm M-L, 1.45 mm A-P, -4.5 mm D-V.

In each mouse, fibers targeting DLS and NAc were always inserted in the same hemisphere, the DMS fiber was positioned in the opposite hemisphere. The hemisphere allocation was counterbalanced across mice. For DLS, the location above was targeted with a fiber rotated at 10° in the M-L/D-V plane. For the optogenetics experiments (Figure 4), fiber optic fibers (ø300 μm core/0.39 NA, 2.5 mm ferrule, ThorLabs) were implanted bilaterally to target the DMS at the following coordinates: +/- 1.5 mm M-L, 0.74 mm A-P, -2.4 mm D-V. These locations were reached with a 10° M-L/D-V rotation.

Virus injections

For the optogenetics experiments (Figure 4), AAV2/5-EF1a-DIO-ChRmine-mScarlet-WPRE-hGHpA (opsin virus, titer: 9e¹² genome copies/ml, Princeton Neuroscience Institute viral core) or AAV2/5-EF1a-DIO-EYFP-WPRE-hGHpA (control virus, titer: 1.5e¹⁴ genome copies/ml, Princeton Neuroscience Institute viral core) was infused bilaterally in the VTA-SNc (+/- 1 mm M-L, -3.1 A-P, -4.66 D-V) of ~4-6 weeks old mice. 500 nl were infused in each hemisphere at a speed of 75 nl/min. In order to achieve sufficient terminal expression by the start of training (3/4 months), all viral injections were performed a minimum of 8 weeks in advance of training and prior to the headplate implantation surgery.

Behavioral task

Behavioral apparatus

We used the standardized behavioral apparatus from the International Brain Laboratory. For detailed instructions on the components and operations of behavioral apparatus used please see International Brain Laboratory.⁷¹ Briefly, the rig consisted of an LCD screen (LP097Q × 1, LG) and a custom 3D-printed mouse holder and head fixation system that held the mouse in front of the screen such that its forepaws rested on a rubber steering wheel (86652 and 32019, LEGO). A spout was positioned in front of the holder, which the mouse could reach it with its tongue but it did not occlude the field of vision. The spout was connected to a water reservoir and water flow was controlled with a solenoid pinch valve (225P011-21, NResearch). The rig was constructed with Thorlabs parts inside a small soundproof cabinet (9U acoustic wall cabinet 600 × 600, Orion). A speaker (HPD-40N16PET00-32, Peerless by Tymphany) positioned on top of the screen was used to play task-related sounds, and an ultrasonic microphone (Ultramic UM200K, Dodotronic) was used to record ambient noise from the rig. Wheel position was recorded with a rotary encoder (05.2400.1122.1024, Kubler) controlled by the Bpod Rotary Encoder Module (Sanworks). Video of the mouse was recorded with a USB camera (CM3-U3-13Y3M-CS, Point Grey). All task-related devices were controlled by a Bpod State Machine (Sanworks) and synched with a data acquisition board (USB201, Measurement Computing). The task logic was programmed in Python and the visual stimulus presentation and video capture was handled by Bonsai⁷² and the Bonsai package BonVision.⁷³

Behavioral task and training

We used a standardized visual decision-making task.^7,13 In this task, mice are head fixed in front of a LCD screen. A visual grating (Gabor patches, 0.1 spatial frequency) whose contrast varied across trials (100%, 50%, 25%, 12.5%, 6.25%) appeared on either the right or left side of the screen (+/- 35° azimuth), accompanied by a 0.1 s tone (5 kHz sine wave, 10ms ramp). A steering wheel that could be used to move the visual grating along the horizontal axis was placed under the mouse’s paws (4° of visual grating movement /mm of wheel movement). The mouse could obtain a small reward of 10% sucrose water (3 μl) by moving the visual grating to the center of the screen. On the contrary, if the mouse steered the grating away from the center (35° from initial position) or failed to center the grating in 60 s, the trial was considered an error. Errors were signaled by the lack of reward delivery and a brief noise (0.5 s, 65 dB, white noise). After a choice was completed (correct or incorrect), wheel movements could no longer move the visual grating for 1 or 2 seconds on correct versus incorrect trials, respectively. After this timeout, all trials were followed by a 0.5 s inter trial interval where no gratings were presented. In order for a new trial to start, the steering wheel had to be still for a “quiescent period”, whose duration was randomly sampled on a trial by trial basis from an exponential distribution of mean 0.55 s and truncated from 0.4 to 0.7 s.

To simplify the interpretation of neural activity correlates of behavior across learning, mice experienced the full extent of the task (no shaping or debiasing protocol) from the first session. All mice were trained for a minimum of 18 sessions, 5-7 days a week, for a 1h session each day. In order to motivate the mice to do the task, mice had restricted water access from 1 week before starting training until the end of the experiment. We monitored that their weight never dropped more than 20% from their pre-water restriction weight, and ensured that they consumed a daily minimum of 1 ml of water per 25 g of weight. Most mice were able to obtain their daily allocation of water through the task alone after a few sessions. When this minimum was not achieved, mice were supplemented at the end of the day. Mice did not have a limit on how much water they could obtain in the task (See Figure S1A for average trials completed across training). Mice were video recorded every session.

Stimulus pre-exposure

For the experiment in Figure 3, before training on the task (1-10 days prior to the start of training), 18 mice underwent two 1h pre-exposure sessions where we measured neural responses to task features in the absence of reward. In the first pre-exposure session, mice were presented with the same visual gratings used in the task (with the same range of contrasts) on either side of the screen for 250-272 trials with a 10 seconds inter-trial interval. The stimulus contrast and side in each trial was randomly sampled between the 8 possible combinations. As in the standard task, presentation of the visual gratings was accompanied by a brief 0.1 s tone. However, during this pre-training session the wheel was locked and the visual gratings remained static on either side of the screen. In the second session (data not shown), mice were allowed to move the wheel and move the visual grating but no rewards were given.

Fiber photometry

Data acquisition

We simultaneously recorded GCaMP6f signals from DA terminals in DMS, DLS and NAc with a multi-fiber photometry system (FP3002, Neurophotometrics) controlled with the Bonsai Neurophotometrics module.⁷⁴ Briefly, the system consists of a CMOS camera acquiring fluorescence emissions and an LED exciting with 470 nm light of 10 ms width pulses at 50Hz (464/476 sessions) or 20 Hz (12/476 sessions). At the tip of the patch cable, the excitation light was ~0.04 mW. The camera acquisition epochs were timed with the emission lights. We used a Low Autofluorescence Patch Cord with 3 branches (BBP(3)_200/220/900-0.37_2m_SMA-3xMF1.25_LAF, Doric) to be able to image DMS, DLS and NAc simultaneously. Prior to each recording day, we passed 0.5 mW 470 nm light through the patch cord for 1 hour in order to photobleach autofluorescence within the patch cord, and improve recording quality.

Signal processing

Fluorescence signals recorded during each session from each location were transformed to dF/F using the following formula:

$$\frac{dF}{F}=\frac{F-F_0}{F_0}$$

F₀ was the +/- 30 s rolling average of the raw fluorescence signal. Finally, dF/F signals were z-scored per-session, using a mean and standard deviation calculated based on all the data from each session. To be included for analysis, every recording (i.e. one session from one fiber location) had to have at least one >=1% dF/F & > 3 standard deviation transient for every 10 min of recording (55/1440 recordings excluded). All data were sampled or resampled at 50 Hz for analysis. Example recording traces can be found in Figure S3.

Histology

To confirm the locations of the opticals fibers and viral expression (Figure S2), mice were anesthetized with pentobarbital sodium (2 mg/kg, Euthasol) and transcardially perfused first with 10 ml of ice-cold phosphate buffered saline (PBS) followed by 25 ml of 4% paraformaldehyde (PFA) in PBS. Brains were then dissected and post-fixed in 4% PFA overnight at 4°C. After fixation, brains were sliced with a vibrating blade microtome (Vibrotome VT100S, Leica) and mounted with DAPI Fluoromount-G (Southern Biotech). All slices were imaged with an automated slide scanner (NanoZoomer S60, Hamamatsu).

Optogenetic stimulation

For the optogenetic experiment in Figure 4, fibers were implanted bilaterally in the DMS to avoid potential behavioral biases related to an asymmetrical surgery. Stimulation was delivered unilaterally to DMS terminals expressing the red-shifted opsin ChRmine. The stimulated hemisphere was chosen randomly and kept constant throughout training for any given animal. The group identity of the mice (opsin vs. control) were blinded to the experimenter throughout the duration of training. The stimulation procedure consisted of a 200 ms laser train timed to the onset of any visual stimulus presentation contralateral to the stimulated hemisphere. Each 200 ms train of stimulation consisted of 20 Hz and 5 ms width light pulses at a wavelength of 532 nm (Shanghai Laser and Optics & Co). The light power was adjusted daily to 0.25 mW at the fiber tip (in the brain). Light power was chosen to ensure activation of the terminals immediately below the fiber tip but minimize off-target activation outside DMS. ChRmine can reliably be activated with an irradiance of >= 0.1 mW/mm².⁷⁵ Therefore, we chose a stimulation power that ensured irradiance above this threshold within but not outside of DMS. According to Deisseroth⁷⁶ the chosen power and fiber (~0.25 mW 532nm light through ø300 μm core/0.39 NA fiber) yields an irradiance of 0.88 mW/mm² (above threshold) just below the fiber tip (DMS) and 0.01 mW/mm² (below threshold) at the DMS/NAc border (1.7 mm ventral from the fiber tip).

Behavioral model

Our approach in modeling behavior aims to descriptively characterize the relatively long time-scale dynamics of learning that would be required to correctly associate stimuli, actions, and outcomes, particularly in the absence of shaping, de-biasing, or other experimental protocols. This relates to previous modeling efforts of similar datasets; however, instead of focusing on trial-to-trial fluctuations in psychophysical weights⁷⁷ or the emergence of multi-state behavior,⁷⁸ we focus on session-level changes in psychophysical weights. We leveraged advances in MCMC^79–81 to infer a set of parameters and weights for Bernoulli generalized linear models (GLM) that were expressive enough to capture the full set of behaviors that mice in our task explored.

To model the behavioral data, we built a hierarchical Bernoulli GLM to describe the relationship between the animal’s choices and a variety of task covariates. The dependent variable was per-trial choice (a Bernoulli variable). The covariates included the stimulus presented to the animal on each trial on the left or right (capturing the classic psychometric curve) together with two additional effects: the animal’s exponentially filtered choice history, and a bias term. We parameterized the stimulus using two regressors, x_L and x_R, corresponding to the contrast of the left-side and right-side stimulus on each trial; because the stimulus only appeared on a single side in each trial, one of these regressors was zero on each trial. We transformed each contrast regressor using a tanh function: $x_{side}=\frac{tanh(\alpha S)}{tanh(\alpha )}$, where S is either x_L or x_R and α is a positive constant governing the shape of the nonlinear transformation. Dividing by tanh(α) ensures that at 100% contrast trials x_side = 1. This parametrization allowed the model to saturate at contrast levels below 100%, sidestepping the need to use lapse parameters to account for the flattening of the psychometric function at high contrast levels.^82–84 We generated the choice history regressor c_t by exponentially filtering previous choices with time constant π: c_t = c_{t − 1} + π y_{t − 1} − c_{t − 1}), where choice y_t takes values of -1 and +1 for left and right choices, respectively. We inferred the time constant p using MCMC sampling, along with the other model parameters (see all parameters below). Across animals, α was fit to 2.5+/- 0.5 (mean+/-sem), and π to 0.52+/-0.004.

Bernoulli GLM weights varied across sessions within each mouse. All other parameters were shared across sessions, within each mouse. For each mouse we built a hierarchical model over sessions, instantiating a separate set of Bernoulli GLM weights for each session. We placed broad prior distributions on the means of the weights on the first session. For all subsequent sessions, we assigned the weights a prior centered around the previous session’s inferred values.

In particular, weight priors for session d took the form of a Student’s t-distribution: β_d ~ StudentT (ν, β_{d − 1}, Σ), where v is the degrees-of-freedom parameter for the Student T prior. This allowed the model to partially pool data across sessions (smoothing the estimates), but to do so adaptively per-animal, taking on larger values of v for animals that steadily and slowly learned the task, and smaller values for v for animals that had sudden and large changes in their learning. We also inferred the covariance of the StudentT priors, allowing parameters to change across sessions in coupled fashion. Unlike the Bernoulli GLM weights, we used a single shared covariance across all sessions. This covariance was parameterized as the quadratic-form product of a diagonal matrix: D = σ⊙I and a correlation matrix Ω: Σ = DΩD. The diagonal had a truncated Gaussian prior. The correlation matrix Ω was constructed from a lower triangular matrix L, which is a cholesky factor of the correlation matrix. These factors had a prior distribution LKJCholesky. The LKJCholesky prior itself has a parameter that tunes the strength of the correlations of the cholesky factor, which we also inferred. Functions and distributions specified here were from the STAN probabilistic programming language, and all model fits were performed in STAN.⁸⁵

We summarize the behavioral model below, (we note that x_side below refers to only the contrast regressors, while x refers to the vector of all regressors i.e., bias, left and right contrast weight, choice history):

Model Variable	Description
ν ~ Gamma(2,0.2)	Student T degrees of freedom
η ~ Normal(0,10)+	LKJ correlation parameter
L ~ LKJCholesky(η)	Lower triangular factor
σ ~ Normal(0, 1)+	Diagonal scale of covariance
λ ~ Normal(0, 1)	Pre-transformed choice kernel time constant
π = Phi(λ)	Choice kernel time constant
Ω = LL′	Correlation Matrix
D = σ ⊙I	Scale Matrix
∑ = D ΩD	Covariance Matrix
μ ~ StudentT(ν, 0,5)	Initial prior mean
β1 ~ StudentT (ν, μ, ∑)	First session choice weight vector
βd ~ StudentT (ν, βd −1, ∑)	d’th session choice weight vector
l ~ Normal(− 2,0.5)	Pre-transformed alpha
α = softplus(l)	Scale factor on stimulus
$x_{side}=\frac{tanh\left(\alpha S\right)}{tanh\left(\alpha \right)}$	Transformed stimulus
pt,d = logistic(xt,d · βd)	Probability of right choice
yt,d ~ Bernoulli (pt,d)	Distribution over choices

Neural model

To model the dopaminergic signals across learning, we built a linear-Gaussian regression encoding model to describe the relationship between task events such as the visual stimuli, actions, and reward delivery with the measured dopamine (DA). Since these events can be correlated in time and their effects on dopamine are partly overlapping, estimating such an encoding model helps to tease apart their individual contributions.

The regression model was defined by a set of temporal kernels that describe the DA impulse response to different task-related events, namely “stimulus”, “action”, and “reward”. For stimulus onset events, we used contrast-specific right and left temporal kernels, giving us 4 temporal kernels per side. All kernels were strictly causal, lasting for a period of 1 second (50 Hz).

Similarly, we used contrast-specific action kernels triggered at the onset of the first significant wheel movement for left and right choices (first movement larger than 0.1 radians after the end of the quiescent period). In addition to separating these kernels by contrast and side (right / left choices) we separated them by correct and incorrect trials, resulting in 8 temporal kernels per side. Separating the action kernels in this manner provided an estimate of the DA response to the interacting effects of initial stimulus location and the movement of the stimulus towards or away the center of the screen. Finally, we defined reward kernels corresponding to the moment when the animal received a water reward or a short time-out period in the same fashion as the action kernels, giving us another 8 temporal kernels per side. Thus, in total we had 40 temporal kernels in the encoding model.

We parameterized the temporal kernels in this model using a basis of linearly scaled “raised cosine” functions spanning a 1-second window after each event.⁸⁶ The cosine basis significantly reduces the dimensionality of the design matrix X (compared to a full series of individual lagged event dummies). The effect on estimation of using a cosine basis is regularization. Use of a temporally smooth basis is also justified by the observation that temporally adjacent responses are strongly correlated.

We used ridge regression to estimate the model parameters, with ridge parameter g and observation noise σ_f ² estimated via evidence optimization.⁸⁷ We optimized for the vector of weights β, and the two scalars σ_f², γ, which are related to the vector of neural response f as follows: Σ = α_f ^{- 2} X^′X+γl)^{- 1}, β = σ_f ^{- 2}ΣX^′f

All weights that made up the entire set of temporal kernels were denoted by β, and could be indexed by their corresponding event type. For example the vector of weights β_1:50 contained the weights for the temporal kernel for stimulus appearing on the right at 6.25% contrast (after one transforms them into the standard basis). The vector of weights β_51:100 contained the weights for the temporal kernel for stimulus appearing on the right at 12.5% contrast, and so on for all remaining contrast levels, and event types. We further computed summary statistics of these temporal kernels, specifically the L2-norm for the stimulus responses. These summary statistics gave us a scalar measure of neural response for each training session that we then related to the estimates from the Bernoulli GLM.

The encoding model for Figure 3 was fit as described above, however we only modeled the stimulus responses, and so the full set of coefficients that made up the kernels was restricted to an intercept, the 4 temporal kernels for stimulus appearing on the right, and 4 temporal kernels for stimulus appearing on the left side of the screen.

Quantification and Statistical Analysis

All statistics reporting a correlation coefficient and a p-value on that correlation coefficient were computed using robust regression, in order to reduce the sensitivity of our statistical conclusions to outliers. Robust regression was performed using the rlm function from the RobustModels package in the Julia programming language. We used M-Estimators with a HuberLoss Function,⁸⁸ the cholesky solver for the method argument, and an initial scale value of 10. For the robust regressions, we computed correlation coefficient-like statistics analogous to Pearson’s R for classic regression. In particular, we computed a pseudo-R² statistic, and its signed square root r, using the RobustModels package deviance and nulldeviance functions: $pseudo{R}^2=1-\frac{deviance(model)}{nulldeviance(model)},r=sign\left({\beta }_1\right)\times \sqrt{pseudo{R}^2}$. Deviance is a generalization of the residual sum of squares for linear models, and null deviance is a generalization of the total sum of squares.

Statistics in Figure 2F were computed with the OneSampleTTest function in the HypothesisTests package from the Julia programming language. Significance was determined at p < 0.05, and all p-values reported are two-sided unless otherwise noted. See Table S1 for detailed results of these tests.

Statistical tests for group differences in behavioral trajectories in Figures 3E−3H, 4C, S4C−S4F, and S5I−S5L, were carried out with the MixedModels and AnovaMixedModels packages in the Julia programming language. Linear Mixed Models from the MixedModels package were used to test simple effects such as the relationship between session 0 DMS stimulus response on behavioral weight values within each training period (early, middle, late). We further used type-3 F-test ANOVAs from the AnovaMixedModels package to test the overall effects in the model, such as, across training periods, is there an influence of session 0 DMS strength on behavioral weight trajectories. For all Linear-Mixed Models and ANOVAs, a*b*c expands into a + b + c + a*b + a*c + b*c + a*b*c.

Linear Mixed Models variable coding

Across Table S2; Data S1, the variable “session” is a transformation of sessions 1 to 20. Sessions are split into 3 categories: early, middle, and late. The early category contains sessions 1 to 7, the middle category contains sessions 8 to 14, and the late category contains sessions 15 to 20. This categorical coding of sessions is motivated by the non-linear trajectory of accuracy in Figure 4C. LinearMixedModels package in Julia uses the first session category as the reference category. Thus in these tables “dms” can be interpreted as “session early & dms”.

In Data S1 the variable denoting the striatal region (DMS, DLS, NAc) is the mean-subtracted session 0 striatal region’s contrast dependent stimulus response magnitude. (L2-norm of the difference of the 100% contralateral stimulus contrast response to the 6%.) session & <region> denotes the interaction of the variables session and <region>. The dependent variables: β_contra, β_ipsi, β_bias β_{choice history} correspond to the behavioral model choice weights.

In Table S2 the variable cohort denotes the group identity of each mouse, either Chrmine or YFP. The variable contra denotes whether the trial corresponded to a stimulus contralateral from the recording site. The dependent variable correct is the side-specific (contra or ipsi) accuracy. Interactions and reference levels are as described above, thus the term: cohort: chrmine & contra:ipsi is the 3-way interaction of the reference level for session (sessions 1–7, e.g. “session early”), cohort, and contra.

Highlights.

• Dopaminergic responses track side-specific learning trajectories in a visuomotor task

• Dopamine in the DMS has pre-existing responses to contralateral visual stimuli

• Pre-existing DMS dopaminergic responses predict learning trajectories for contralateral stimuli

• Activation of dopamine terminals in the DMS improves contralateral performance

In brief.

Pan-Vazquez, Sanchez Araujo, et al. show how pre-existing contralateral visual responses in dopamine terminals in the dorsomedial striatum help explain side-specific learning trajectories of a visuomotor task. These results suggest that feature- and projection-specific dopamine signals could simplify the problem of learning a new task.

Data and code availability

• All the code for the analyses and visualizations performed in this project has been made publicly available. The URLs are listed in the key resources table.

• All behavioral data have been made publicly available. The URLs are listed in the key resources table.

• All photometric data have been made publicly available. The URLs are listed in the key resources table.

• Any additional information required to reanalyze the data reported in this paper is available from the lead contact upon request.