Ventral tegmental dopamine neurons control the impulse vector during motivated behavior

Summary

The Ventral Tegmental Area (VTA) is a major source of dopamine, especially to the limbic brain regions. Despite decades of research, the function of VTA dopamine neurons remains controversial. Here, using a novel head-fixed behavioral system with five orthogonal force sensors, we show for the first time that the activity of dopamine neurons precisely represents the impulse vector (force exerted over time) generated by the animal. Distinct populations of VTA dopamine neurons contribute to components of the impulse vector in different directions. Optogenetic excitation of these neurons shows a linear relationship between signal injected and impulse generated. Optogenetic inhibition paused force generation or produced impulse in the backward direction. At the same time, these neurons also regulate the initiation and execution of anticipatory licking. Our results indicate that VTA dopamine controls the magnitude, direction, and duration of force used to move towards or away from any motivationally relevant stimuli.

One Sentence Summary

VTA dopamine bidirectionally controls impulse vector and anticipatory behavior

Graphical Abstract

Introduction

The ventral tegmental area (VTA) is a midbrain region with dopamine (DA) neurons that project widely to the frontal cortex, limbic basal ganglia, and other brain regions [1]. Despite decades of research, the function of these neurons remains controversial. According to one prominent hypothesis, DA neurons encode a reward prediction error (RPE), which serves as a teaching signal that updates the value of learned associations [2–4]. Others argue that DA encodes incentive salience to motivationally relevant stimuli, or the amount of effort (or vigor) the animal exerts towards these stimuli [5–10]. Still others argue that DA neurons can multiplex different cognitive, reward, and kinematic variables [11, 12].

One potential source of ambiguity in DA function is the lack of quantitative behavioral measurements. In most studies investigating VTA DA, animals are often head-fixed, and the behavioral measures are usually limited to time stamps of licking, eye or arm movements [13–15]. Although animals in head-fixed setups often produce many movements of the head and body, these are usually ignored. In studies with freely moving animals, behavioral measures also lack spatial resolution and cannot accurately measure subtle and direction-specific movements [16]. Consequently, although DA activity has been extensively linked to the delivery of rewards and reward-predicting cues, whether these neurons are involved in the generation of movements remains unclear due to the lack of adequate measures. To improve behavioral measures in head-fixed animals, we developed a novel apparatus for measuring the head and body force generated during behavior [17]. Using in vivo electrophysiology and optogenetics, we show that VTA DA neurons provide a precise representation of the impulse vector (force exerted over time), and regulate the direction, amplitude, and duration of force generated by the animal during motivated behavior.

Results

In order to record the single-unit activity of VTA DA neurons, we used both WT mice (n = 3) and DAT-Cre mice crossed with Ai32 mice (n =4), and chronically implanted them with electrode arrays (Figure 1A)[18]. DAT-Cre mice express Cre-recombinase only in neurons that express the dopamine active transporter (DAT), and Ai-32 mice express Cre-dependent excitatory channelrhodopsin (ChR2). We can thus confirm that the neurons we recorded from are dopaminergic (DAT+) neurons by selectively activating VTA^DAT+ neurons while recording their single unit activity (tagged neurons: n = 44, from 4 different mice: n = 34, 5, 4 and 1; total neurons: n = 298; Figures 1B–E)[19]. Despite heterogeneity in the waveform widths and firing rates [15], most optically tagged VTA^DAT+ neurons exhibited low firing rates (FR) and wide waveform widths (mean FR = 3.99 ± 0.35 Hz; Valley Full Width at Half Maximum (FWHM) = 624.10 ± 38.72 μs; Figures 1C & 1E). Based on our optogenetic tagging results, we classified the remaining population as DA if their Valley FWHM was more than 500 μs and their mean firing rate was less than 10 Hz (n = 127 from 7 different mice: n = 56, 48, 11, 5, 4, 2, 1; Figure 1E). Using an unbiased Gaussian Mixture Model (GMM), we then functionally classified the DA neurons into three separate populations (Figures 1F–G, see below for detailed description).

Figure 1. Summary of Optic Tagging and Classification of VTA DA Neurons.

A Top: Coronal section showing optrode placement along with colocalization of tyrosine hydroxylase (TH) and channelrhodopsin (ChR2) in DAT + Ai32 mice. Bottom: Schematic of locations of electrode implants (n = 7) B Left: Representative example of a tagged VTA dopamine (DA) neuron with 10 Hz stimulation. Right: Zoomed in view of the same tagged neuron’s spiking showing latencies of 4–5 ms (top) and the latency of each individual spike (bottom). C Fidelity analysis of tagged VTA^DAT+ neurons showing spontaneous waveforms (black) and light evoked waveforms (blue). D All tagged VTA DA neurons had a latency of ≤= 6 ms (n = 32). E Classification for remaining DA population. The majority of tagged VTA DA neurons (n = 44) had a firing rate (FR) of less than 10 Hz and a valley full width at half max (FWHM) greater than 500 μs. F A Gaussian Mixture Model (GMM) was applied to all VTA DA neurons based on their functional properties during the behavioral task. The graph depicts 3 clusters or populations. G More than 85% of the variance of each population could be accounted for with 3 principal components.

To precisely quantify the forces that animals produce while restrained, we developed a head-fixation apparatus that incorporates orthogonal force-detecting load-cells coupled to the head-bar clamps (Figure 2A). We measured VTA DA activity during the performance of a fixed-time (FT) reward task, in which a drop of sucrose solution is delivered every 10 s in the absence of an explicit conditioned stimulus (Figure 2A). After 7 to 14 days of training, the mice exhibited clear anticipatory licking behavior prior to the onset of reward (Figure 2B). Close inspection of the signals produced by the force sensors revealed the wide diversity of movements generated by the head fixed mice. In addition to producing anticipatory licking and active consummatory behavior characteristic of the FT schedule [20], animals generated forces that varied continuously in their duration and amplitude in specific directions (Figures 2B & S1). The movements of the mice closely coincided with the duration of anticipatory lick bouts and exhibited stereotyped patterns during licking (Figure 2B). At the onset of the anticipatory lick bout, the mice typically exerted force in the forward direction, as well as pushed their head down and to one side to position themselves optimally in front of the reward spout (Figures 2B & S1). At the time of reward delivery, the mice at first either froze or moved backward very briefly, and then exerted a large push forward resulting in a sudden change in the force sensor signals. As registered by the force sensors, there was a general forward force exerted by the animals toward the reward spout (Figures 2B & S1). At the end of the consummatory lick bout, we usually detected a steep drop in force, reflecting a slight backward movement (Figure 2B). Not surprisingly, the sensor with the greatest force amplitude and most consistent pattern corresponded to the forwards and backwards direction (F/B sensor; Figure S1). We therefore focused on F/B movements in our analysis.

Figure 2. The VTA contains three types of dopamine neurons that are distinct in their responses to reward and direction of movement.

A Top: Schematic representation of novel head-fixation apparatus with five orthogonal force sensors. Inset shows the movement axis measured by the forward/backward (F/B) force sensor. Bottom: Mice were trained on a fixed-time reinforcement schedule, where they received a 10% sucrose reward every 10 seconds. B Mice exhibited anticipatory licking before reward delivery. Top: Representative raster plot of licks aligned to reward. Bottom: F/B force sensor reading during the same behavior shown above. Trials in both panels are sorted according to the duration of the lick bout. C Schematic and coronal section showing optrode placement into the VTA. Inset shows ChR2-infected neurons also contain tyrosine hydroxylase (TH). D Representative optically tagged fast-backward (FB) VTA DA neuron is inhibited during forward movement, increases its firing rate during backward movement, and displays a short-phasic burst (< 200 ms) at reward delivery. E Representative optically tagged VTA DA fast-forward (FF) neuron increases its firing rate when aligned to forward movement outside of reward, is inhibited during backward movement, and displays a long-phasic burst (> 200 ms) following reward that is sustained throughout the forward movement. F Representative optically tagged VTA DA slow-forward (SF) neuron increases its firing rate during forward movement, but is inhibited during backward movement, and displays little or no burst at reward. Trials in all panels are sorted according to the duration of F/B force. Also see Figure S1.

We found three functional populations of VTA DA (DAT+) neurons that were distinct in their relationships to movement and reward. The first population typically displayed a short phasic burst (50 – 200 ms) at the time of reward (Figure 2D). These neurons decreased firing for forward movement but increased firing for backward movement, as indicated by the force sensors (Figure 2D; Fast-Backward (FB) neurons, total neurons: n = 44 from 6 mice: n = 23, 14, 4, 2, 1, 0; tagged neurons: n = 23). A second population of neurons typically displayed a longer phasic burst (200 ms - 1 s in duration) at the time of reward (Figure 2E). In contrast to the FB DA neurons, these neurons increased their firing rate during forward movement, and decreased their firing rate during backward movement (Figure 2E; Fast-Forward (FF) neurons, total neurons: n = 32 from 6 mice: 13, 11, 4, 2, 1,1; tagged neurons: n = 17). Finally, a third population of DA neurons exhibited firing rates that increased during forward movement and decreased during backward movement, like FF neurons. However, they were distinct from the FF group in that they showed little firing rate modulation at the time of reward (Figure 2F). Overall, their firing rate modulation reflected slower changes of force over time (Slow-Forward or SF neurons, total neurons: n = 35 from 7 mice: 14, 13, 3, 3, 1, 1; tagged neurons: n = 4). We used the combination of responses to forward movements, backward movements, and reward to classify each DA neuron that we recorded. To further verify and validate our classification, we concatenated and scaled the response profiles of all VTA DA neurons around each of these events, then performed a principal component analysis using a GMM, which found 3 distinct clusters corresponding to our 3 functional groups (Figures 1F–G, Figure 3).

Figure 3. Three populations of VTA dopamine neurons can be distinguished based on direction, magnitude, and time scale of force generation.

A Each population of VTA DA neuron is responsive to force on different time scales and direction of movement when aligned to reward. The FB population scales with small backward forces. The FF population scales with fast forward force. The SF population scales with slower force modulation. Left: all populations plotted on the same time scale. Right: three different time scales are used on the x-axis to reveal clearly the key relationship between neural activity of each population of DA neurons and the associated force change. B Average FB, FF, and SF population activity aligned to onset of forward movement outside of immediate reward delivery. The FB population decreases its firing rate during forward movement, whereas the FF population increases its firing rate. Similar to the FF population, the SF population increases its firing rate during forward movement. C Heatmaps of F/B force and DA populations aligned to forward movement outside of reward times. FB neurons are inhibited during forward force generation. FF and SF neurons increase their firing rates during the duration of forward force. Blue tick marks on the left denote optically tagged neurons. D Average FB, FF, and SF population activity aligned to onset of backward movements outside of immediate reward delivery. The FB population increases its firing rate during backward movement, whereas the FF population decreases its firing rate. Similar to the FF population, the SF population decreases its firing rate during backward movement. E Heatmaps of F/B force and DA populations aligned to backward movement outside of reward times. FB neurons increase their firing rate for the duration of backward force. FF and SF neurons are inhibited for the duration of backward force. Blue tick marks on the left denote optically tagged neurons. F Left: Schematic of air puff trials during the task. Right: Average FB, FF, and SF population activity aligned to air puff. All three populations increase their firing rate at the time of air puff. G Heatmaps of F/B force, and all DA populations aligned to air puff. Blue tick marks on the left denote optically tagged neurons. Error denotes SEM. Also see Figures S2 and S3.

DA neural activity usually preceded movement onset after reward (Figure S2). However, the three cell populations showed distinct relationships with force measures. After reward delivery, FB neurons showed phasic activity just before the brief backward movement (latency after reward = 64.57 ± 1.76 ms; Figures 3A & S2). In contrast, the activity of FF neurons preceded the large forward force generation following reward (Figures 3A, S2 & S3; latency after reward = 107.10 ± 8.45 ms). Neurons in the SF population, on the other hand, do not significantly modulate their firing rates at the time of reward consumption. These neurons display a much slower ramping pattern that is only apparent when a much larger time window is used (Figures 3A & S3). This suggests that the SF activity could contribute to the tonic changes seen in prior work during anticipatory approach, while the FF and FB neurons reflect the well-established phasic firing pattern of VTA DA neurons [21]. When the neural activity was aligned to forward movements outside of reward context, each population had consistent modulations that were seen around reward times. The firing rates of the FB population decreased, while the FF and SF population increased (Figures 3B–C). In contrast, the opposite patterns were observed when the neural activity was aligned to backward movement. The FB population’s firing rate increased, while the FF and SF population’s decreased (Figures 3D–E). These firing patterns during reward, forward movements, and backward movements were also seen early in training, indicating they are not responses that develop as a function of learning (Figure S3).

Although we have shown a clear relationship between force generated and DA activity, it is unclear whether these force signals are found only during reward-guided behavior. It is well established that many VTA DA neurons also respond to aversive stimuli [14, 22–24]. To examine how each DA population responded to an aversive stimulus, an air puff was delivered in some sessions to the face of the animal at random times during the task. All mice also showed very characteristic force changes: they first moved backwards, then returned to their normal position and remained motionless after the initial startle response (Figures 3F–G). Unlike during appetitive behaviors, the air puff elicited burst firing in all three populations of DA neurons (Figures 3F–G). SF and FB populations are activated first, followed by the FF population much later (Figure S2). This activation pattern reflects opponent populations producing forces in opposite directions. In other words, the aversive stimulus appears to generate conflicting force vectors, which resulted first in backward movement (due to the FB population being activated first), then forward movement followed by no movement or freezing. This interpretation is supported by our observation that the air puff produced a quick startle response that is followed by freezing behavior and no movement in any direction.

Our results show for the first time that the phasic components are responsible for fast force changes, whereas the slow component is responsible for slow force changes. Strikingly, we found that the firing rates of all three populations of DA neurons can be explained by a single variable, namely force generated over time (impulse [Newton-seconds(J)]); Figure 4A). We calculated the impulse resulting from each individual movement and counted the number of spikes detected in each DA neuron during that movement. This resulted in a precise linear relationship between the number of spikes of each population of VTA DA neurons and the impulse generated in a specific direction (Figure 4B). For the FB population, the number of spikes correlated highly with negative impulse, which corresponds to movement in the backward direction (Figure 4B). In contrast, the number of spikes produced by the FF and SF populations had high correlations with positive impulse, which corresponds to forward movement. These correlations were weaker when considering peak force alone (Figure S2). The FB neurons produce much less impulse over time than the FF or SF populations (Figures 4C). This is not surprising, as most natural movements are in the forward direction. In addition, the FF population appears to govern larger and faster force changes compared to the SF population, which is more responsible for generating low amplitude force changes that are typical of the force changes that occur across a longer time-window. Finally, all three types were approximately found in the same proportion (Figure 4D). Altogether, the functional contributions of each population to the generation of impulse are distinct with respect to the direction, magnitude, and the time scale of force exertion.

Figure 4. The impulse vector explains most of the variance in the activity of VTA DA neurons.

A Schematic of how the impulse vector is calculated. It is equal to the time integral of force (N). Negative values reflect impulse in the backward direction. B High correlation with negative impulse (backwards movement) was found for the FB population, while high correlations with positive impulse (forward force) were found for the FF and SF populations. Cumulative probability shows absolute amount of impulse associated with 7 spikes (the maximum number of spikes common to all neurons during a movement). C) Mean impulse generated from 7 spikes from each population of neuron. There is a significant difference between all three groups (One-way ANOVA, F(_2,21) = 143.70, p < 0.0001. Tukey’s post-hoc test revealed significant differences between all three groups p < 0.01). D Pie chart shows relative proportion of different VTA DA populations (FB: n = 44; FB: n = 33; SF: n = 35). E Firing rates of all three populations of DA neurons can be used to predict force over time. Schematic of neural decoder used to predict forward force. F F/B force was accurately decoded from all simultaneously recorded VTA DA neurons regardless of functional classification. Traces are representative 20 s examples from two different animals. Error bar indicates SEM. Also see Figure S4.

Together, the activity of VTA DA neurons could be used to explain the continuous and time-varying force generated by each mouse. We used a machine learning algorithm (support vector regression) to predict the force exerted by the animal across the entire session using all simultaneously recorded DA neurons from that session regardless of their functional classification[25]. We were able to accurately decode the F/B force using only the DA population activity (Figures 4E–F & S4). Similar to the correlation analyses, the decoder was able to use the raw DA neuron population activity to determine the force generated by the animal on a moment-to-moment basis, rather than using data averaged across many trials (Figures 4E–F & S4).

Because force generation and licking bouts often co-varied, we also performed correlation analyses between VTA DA activity and licking across the entire session. All three cell populations showed significantly greater correlations with force than with lick rate (p < 0.0001, Figure 5A). Furthermore, when we examined all forward movements that did not coincide with a lick bout, we observed the same modulations in the activity of DA neurons (Figures 5B–C). Interestingly, when trials were sorted according to the onset of the first consummatory lick after reward, the FB neurons display several phasic bursts after reward that occurred in between licks (Figure 5D). This could be due to a strong backward force during each licking cycle, visible in the continuous force measures. To examine this possibility, we aligned the neural activity and backward force to the first and second lick that followed reward. As expected, the FB neurons increased their firing rate in a manner that closely tracked backward forces generated during consummatory licking (Figure 5E). This was also true for licking in general, not just licking at the time of reward delivery (Figure 5F).

Figure 5. DA neurons represent force, not licking behavior.

All three populations of neurons showed similar relationships with force during periods without any reward delivery or lick bouts. A) Correlations between neural activity and force or lick rate across the entire behavioral session. All three populations showed significantly greater correlations with force than with lick rate (p < 0.0001). Bottom right: To determine if VTA DA neural activity is still correlated with force in the absence of lick bouts, we filtered the data to examine all non-rewarded forward movements with no lick bouts. Data were filtered by removing all forward movement with lick bouts. Forward movements with lick bouts had significantly greater licks per second than forward movements without lick bouts (p < 0.0001). B Average FF, FB, SF population activity and FB force aligned to onset of forward movement outside of reward and lick bouts. C Peri-event heat map of FB force, and all neuronal populations aligned to forward movement outside of reward and lick bouts. FB neurons are inhibited for the duration of forward force. FF and SF neurons increase their firing rates during the generation of forward force. D Representative optically tagged FB neuron aligned to reward and sorted according to the onset of the lick bout. When licks are overlaid with neural activity, an antiphase relationship is seen between licks and neural activity. E This relationship is due to the backward force generated in between licks, which can be seen when aligned to both the first and second consummatory licks. F This relationship is also seen when no reward is present during anticipatory licking. G Schematic of timing precision. The onset of the anticipatory lick bout can indicate how well the reward is predicted. Lick bouts initiated just before reward delivery reflect more precise timing and are more highly predicted. H There was no correlation between the firing rate of phasic bursts and timing precision for the FF and FB populations (FF: mean r = −0.01; FB: mean r = −0.02). I The FB and FF populations have significantly different peak firing rates (Two-way ANOVA, p < 0.0001). However, the firing rates in each population are not modulated by reward prediction (Linear regression, FF: r² = 0.0005, p = 0.87; FB: r² = 0.00001, p = 0.10). Error bars reflect SEM.

In our task, rewards are always fully predicted [20]. Just like Pavlovian conditioning, reward delivery on a fixed-time schedule can be predicted, as indicated by anticipatory licking. It is possible that phasic activity of the FB and FF populations following reward could be modulated by variability in timing, reflecting uncertainty about reward delivery time (Figure 5G). We would expect phasic firing rate to be higher when the reward is less well predicted corresponding to trials where timing is less precise. To test this possibility, we examined the firing rate of each individual reward trial and the corresponding timing of the lick bout onset before reward. We did not find a significant correlation between timing precision and post-reward DA firing rate (Figures 5H–I). There was, however, a significant difference between the peak firing rate of the FF and FB populations regardless of their timing precision, indicating they have distinct functional roles independent of a unitary prediction error signal (Figure 5I).

Because DA activity generally preceded the change in force exerted (Figure S2), it could have a causal role in force generation. To test this hypothesis, we injected Cre-dependent channelrhodopsin (DIO-ChR2) into the VTA of DAT-Ires-Cre (DAT::ChR2^VTA, n = 5) or of Wild-type (WT::ChR2^VTA n = 5) mice for controls (Figures 6A & S5). We optogenetically excited (40 pulses at 20 Hz) DA neurons starting at 6 seconds before reward delivery, when animals were least likely to show anticipatory behavior. Stimulation resulted in forward force generation (Latency: 122.50 ± 7.53 ms; Figures 6B–C & S5, Video S1). Because optogenetic stimulation cannot be limited to specific functional populations of neurons, this manipulation is expected to reflect the net effect of activating all three populations. As we found a greater proportion of neurons that increased their firing rate during forward movement (FF & SF populations; Figure 4B), a net increase in forward force generation was expected.

Figure 6. Optogenetic excitation of VTA DA neurons is sufficient to generate forward force and optogenetic inhibition of VTA DA neurons is sufficient to generate backward force, followed by rebound forward force.

A Left: Schematic illustration showing the implantation of optical fibers above VTA. Right: Histological confirmation of selective ChR2-expressing neurons in VTA DA neurons. Inset shows ChR2-infected neurons are TH+. B Top: Diagram of DAT::ChR2^VTA excitation at 6 seconds before reward delivery. Bottom: Representative heatmap showing reliable forward force generation resulting from optical stimulation of VTA DA neurons at 20 Hz (5 ms pulse width) for 2 seconds. C Average F/B force signals resulting from stimulation of VTA DA neurons at 20 Hz for DAT:: ChR2^VTA (n = 5) and WT::ChR2^VTA (n = 5) mice. D Representative heatmaps showing reliable forward force generation from 25 Hz excitation with 1,3,6 and 12 pulses (5 ms pulse duration) of light at 6 seconds before reward delivery. E Each pulse of light produced longer duration movements and higher peak force. Plot shows forward force signals across all subjects as a function of the number of light pulses delivered at 25 Hz. F There is a strong linear relationship between mean impulse (area under the force-time curve) and number of pulses delivered at 25 Hz in DAT::ChR2^VTA (n = 5, p = 0.05), but not in WT control mice (n = 5, p = 0.14). G) Left: Schematic illustration showing injection of SIO-stGtACR2 into the VTA along with the implantation of optical fibers above VTA. Right: Histological confirmation of selective expression of stGtACR2 in VTA DA neurons. Inset shows colocalization of stGtACR2-infected neurons with TH-positive neurons. H In vitro whole cell patch clamp recordings in current clamp mode demonstrate robust inhibition of DA neurons with stGtACR2. Left: Example neuron showing inhibition of VTA DA neurons using constant stimulation with 473 nm light. Right: Constant stimulation significantly reduced firing frequency (n = 5, p < 0.01). I Top: Diagram showing DAT::stGtACR2^VTA inhibition (constant for 2 seconds) occurring at 1 second prior to the delivery of rewards. Middle: Representative heatmap showing backwards movement due to DA inhibition. Note the presence of large rebound excitation after the termination of stimulation. Bottom: Representative heatmap showing forward force generation prior to reward delivery on control trials without laser delivery. J Top: Diagram showing WT::stGtACR2^VTA inhibition (constant for 2 seconds) occurring at 1 second prior to the delivery of rewards. Middle: Representative heatmap for stimulation trials for WT controls. Bottom: Example heatmap of control trials without laser. K Average force during stimulation trials across all DAT::stGtACR2^VTA and WT::stGtACR2^VTA animals. L Top: There was a significant negative impulse generated by stimulation in DAT::stGtACR2^VTA (n = 4) compared to WT (n = 5) controls (unpaired t-test, p = 0.0118) during the first second of inhibition. Bottom: There was a significant increase in positive impulse for DAT::stGtACR2^VTA (n = 4) compared to WT (n = 5) controls (unpaired t-test, p = 0.0024) due to rebound excitation after inhibition ended (** p < 0.01; *** p < 0.005). Error bar indicates SEM. Also see Figures S5 and S6, and Videos S1 and S2.

It has been proposed that VTA DA neurons operate in two modes: a low firing tonic mode and a high-firing phasic mode, and that these signals can represent distinct variables at different time scales [4, 6, 12, 21, 26]. Stimulation at 20 Hz for 2 seconds could be increasing tonic DA levels as well as mimicking phasic activity. To assess the role of brief phasic firing in force generation, we also stimulated dopamine neurons with short pulse trains (< 500ms) to mimic phasic firing activity (Figures 6D–F). Stimulation with brief trains of light resulted in consistent forward force generation (Figures 6D–F & S5). Even 1 5ms pulse of light would reliably generate movement (Figures 6D–F & S5, Video S2). Remarkably, we also observed a strong linear relationship between the number of pulses in the stimulation and the impulse generated by the mice (Figure 6F).

Because optogenetically mimicking RPE signals in VTA DA neurons failed to produce movement in previous work [27], we tested whether forward movement generation would occur in naïve mice not previously trained on the task. Optogenetic excitation (20 Hz for 2 seconds) did not produce movement in naïve mice (Figure S5). This suggests that the motivational context is necessary for VTA DA to produce movement generation.

We next tested the effects of optogenetic inhibition on VTA DA neurons. We injected Cre-dependent soma-targeted Guillardia theta anion-conducting channelrhodopsin (stGtACR2), which has been shown to effectively shut down neural activity with high temporal precision[28–30], into the VTA of DAT-Ires-Cre (DAT::StGtACR2^VTA, n = 4) or of Wild-type (WT::StGtACR2 ^VTA, n = 5) mice for controls (Figure 6G). We also confirmed potent optogenetic silencing of DAT::StGtACR2^VTA neurons using in vitro whole cell patch-clamp recording in acutely prepared brain slices (Figures 6H & S6), demonstrating for the first time robust inhibition of DA neurons with this newly developed opsin.

We then silenced DA neurons immediately before reward delivery, when the mice normally showed high forward force production (Figure 6I). DA inhibition produced backward movement (Figures 6I & 6K–L). After stimulation ended, the mice also exhibited a rebound forward movement, most likely due to rebound excitation of VTA DA neurons from sustained inhibition, which was also seen in our in vitro experiments (Figures 6H, 6K–L & S6). The effects from both optogenetic stimulation and inhibition also carried over to the trial that followed stimulation: less force was exerted during anticipatory, but not consummatory, behavior on the trial that immediately followed excitation, while we observed the opposite effect following inhibition (Figure S7).

Interestingly, we found that optogenetic excitation of VTA neurons 6 seconds before reward delivery (when baseline licking was at its lowest) also reliably elicited anticipatory licking in addition to exertions of forward force (Figures 7A–B, Video S1). In contrast, optogenetic inhibition caused only a slight reduction in licking, which likely reflected a ‘floor effect’ (Figures 7E–F). Both optogenetic excitation and inhibition significantly disrupted anticipatory licking if stimulation occurred just before reward delivery, when the rate of licking was high (Figures 7C–D & 7G–H). However, while neither optogenetic excitation nor inhibition had any effect on consummatory licking, we observed altered anticipatory licking on the trial that followed stimulation (Figure S7). Together, these optogenetic stimulation results show for the first time that VTA DA neurons play a causal role in the generation of anticipatory—but not consummatory—licking, in accord with previous work [5, 20, 31].

Figure 7. Optogenetic manipulation of VTA DA neurons initiates and disrupts anticipatory licking depending on the timing of stimulation.

A Top: Diagram of DAT::ChR2^VTA excitation at 6 seconds before reward delivery. Bottom: Optogenetic excitation produces anticipatory licking. Representative peri-event raster plot showing licking during non-stimulation trials (grey) and stimulation trials (blue). Traces reflect the mean lick rate calculated from the raster above. B The lick rate significantly increased during stimulation trials for DAT::ChR2^VTA (n = 5) animals compared to non-stimulation trials and control groups (n = 6; two-way mixed ANOVA, Interaction: F(_1,9) = 15.45, p < 0.01). C Top: Illustration of DAT::ChR2^VTA excitation 1 second before reward. Bottom: Optogenetic excitation disrupted anticipatory licking. Representative peri-event raster plot during non-stimulation trials (black) and stimulation trials (blue) for representative animal, as well as average traces of lick rate. D The rate of licking prior to reward delivery was significantly decreased during stimulation trials for DAT::ChR2^VTA (n = 5) animals compared to non-stimulation trials and control groups (n = 6; two-way mixed ANOVA, Interaction: F(_1,9) = 15.14, p < 0.01). E Top: Illustration of DAT::stGtACR2^VTA inhibition 6 seconds before reward. Bottom: Optogenetic inhibition slightly disrupted anticipatory licking. Representative peri-event raster plots of licking during non-stimulation trials (black) and stimulation trials (maroon), with mean lick rate plotted below. F The rate of licking did not significantly decrease during stimulation trials compared to non-stimulation trials for DAT::stGtACR2^VTA (n = 4) animals and control groups (n = 6; two-way mixed ANOVA, Interaction:, F(₁,₈) = 2.36, p > 0.05), though this was due to a floor effect. G Top: Illustration of DAT::stGtACR2^VTA inhibition 1 second before reward. Bottom: Optogenetic inhibition 1 second prior to reward delivery significantly disrupted anticipatory licking. Peri-event raster plots of licking and average traces of lick rate during non-stimulation trials (black) and stimulation trials (maroon) for representative animal. H Lick rate significantly decreased during stimulation trials for DAT::stGtACR2^VTA (n = 4) animals compared to non-stimulation trials and control groups (n = 6; two-way mixed ANOVA, Interaction:, F(_1,8) = 20.13, p < 0. 01). p values are corrected for multiple comparisons using Dunnett’s multiple comparison test. * p < 0.05; ** p < 0.01. Error bar indicates SEM. Also see Figure S7.

Discussion

Collectively, our results demonstrate that VTA DA controls the exertion of force in motivated behavior. Strikingly, we discovered a precise quantitative relationship between the activity of VTA DA neurons and the impulse vector. Distinct populations of DA neurons contribute to different components of the impulse vector. The FF and FB populations are responsible for sudden force changes especially in response to salient stimuli. In contrast, the SF population is responsible for slower and more gradual force generation during anticipatory approach behavior. In normal behavior, all three populations are involved, but their relative contributions depend on the force requirements of the movements.

In freely moving animals, it has been previously shown that DA neurons in the nearby substantia nigra pars compacta (SNc) are modulated by postural adjustments[32, 33], and highly correlated with the vector components of velocity and/or acceleration [34], just like their target neurons in the striatum [35]. Optogenetic experiments also demonstrated that selective stimulation of SNc DA neurons can directly initiate movements [34], and optogenetic inhibition of SNc DA neurons can retard the initiation of movements [16]. In agreement with these observations, we showed that VTA DA neurons are also critical for the initiation of movements.

Our optogenetic experiments showed that every pulse of light exciting VTA DA neurons produced a proportional increase in impulse, demonstrating a causal role of these neurons in force generation and a linear relationship between DA activity and impulse (Figure 4). Impulse, or change in momentum, is necessary for the initiation of any movement.

According to the impulse-momentum theorem:

$$J=\Delta p$$ (1)

where J is impulse and p is momentum. Since momentum is defined as:

$$p=mv$$ (2)

where m is mass and v is velocity, the relationship between impulse and velocity is:

$$J=\mathrm{m}\mathrm{\Delta }\mathrm{v}$$ (3)

From equation (3), and assuming a constant mass, impulse simply changes velocity. Thus, it is easy to see how DA activity could be correlated with velocity and acceleration in freely moving animals [34, 36, 37], while representing impulse or change in momentum in our head-fixed preparation.

Furthermore, in naïve mice stimulation of VTA DA neurons did not generate movements, an observation suggesting that the role of these neurons in movement initiation is not unconditional, as would be the case for motor neurons. Instead, the effect of VTA DA neurons depends on the motivational context. This could explain why others have not been able to elicit movement by mimicking phasic DA activity [28]. This is in accord with the idea that DA, as a neuromodulator, primarily adjusts the gain of striatal medium spiny projection neurons[37]. Striatal neurons receiving DA projections require glutamatergic drive as well as DA to be activated. The VTA DA signal alone may not be sufficient to generate movements, but instead regulates the gain of context-specific commands for motivated behaviors [38, 39]. VTA DA may adjust the gain for commands that originate from the prefrontal cortex and other limbic regions[38]. It has been proposed that the basal ganglia act as a transition controller in a control hierarchy[39, 40]. Distinct cortico-basal ganglia loops control different types of transitions in different perceptual variables. The dorsal striatum primarily receives input from sensorimotor and associative cortices that convey proprioceptive and exteroceptive information. Thus the nigrostriatal pathway from the SNc may adjust the gain for steering, postural changes, and goal-directed actions. In contrast, the ventral striatum receives interoceptive input from limbic regions. Since the VTA primarily innervates the ventral striatum via the mesolimbic DA pathway, it could be responsible for regulating the gain of transitions for consummatory behaviors, as well as simple approach and avoidance behaviors.

VTA DA regulates anticipatory licking

Another novel finding from our study is that stimulation of VTA DA neurons not only generated force but also licking behavior, even though their firing rates represented impulse rather than licking (Figure 5). When stimulated in the absence of any anticipatory licking, stimulation simply induces licking. This finding suggests that the underlying mechanisms for force generation also play a key role in generating licking behavior. Previous work has shown the critical importance of descending basal ganglia outputs in the top-down regulation of licking in mice [20, 41]. Thus, VTA DA may promote anticipatory lick generation in the appropriate motivational context, in accord with the gain control mechanism. In contrast, inhibition of DA neurons generally suppressed anticipatory licking. Surprisingly, neither excitation nor inhibition significantly affected consummatory licking, so that the effect of DA manipulation appeared to be limited to anticipatory reward seeking behavior only.

Because we also showed that DA activity is not correlated with licking (Figure 5), these results also support the hypothesis that VTA DA may regulate the gain of context-dependent behavioral sequences. This type of regulation is manifested in the force generated. This is consistent with extensive past research implicating VTA dopamine in effort, wanting or vigor related to motivationally relevant stimuli[7, 42].

It has recently been reported that VTA GABA neurons, which are directly connected with the DA neurons, can represent and command head angles about orthogonal axes of rotation[28]. Given the interaction between VTA GABA and DA, DA could contribute to head angle commands by providing the requisite amount of torque necessary to achieve a change in head position (i.e. angular velocity) in order to move toward or away from motivationally relevant stimuli. It should be noted, however, the force measured in our study cannot be used to predict actual kinematics in freely moving animals, because the force sensors do not accurately measure torque due to the geometry of the body in the head-fixed animal and the positioning of the limbs and associated mechanical advantage. In addition, depending on the mass of the body, some minimum amount of force must be generated for overt movement to occur.

Reward Prediction Error

Although our findings do not rule out a role of DA in learning, they are inconsistent with the popular RPE hypothesis of DA function[4] for the following reasons: 1) All three populations of VTA DA neurons increased their firing rate regardless of the motivational valence of the stimuli involved (reward or air puff, Figure 3). 2) In our paradigm, rewards are fully predicted, yet post-reward DA bursts are always observed. This could be due to uncertainty as reflected in timing precision, but we found that variability in the timing of anticipatory licking, a measure of how well the reward is predicted, is unrelated to the phasic burst response of DA neurons after reward delivery (Figure 5). 3) Each population of DA neurons shows a unique pattern after reward delivery, corresponding to distinct components of the impulse vector and exhibited clear and opposite changes in their firing rates depending on the direction of force (Figures 3–4). 4) Optogenetically mimicking phasic RPE bursts generated net forward force, and optogenetic inhibition could result in backward force generation (Figure 6).

More generally, the RPE hypothesis only predicts increases or decreases of DA activity based on the magnitude of the prediction error, which is not a vector quantity and contains no spatial or directional component. A well-known problem with models based on prediction error is that learning and performance are confounded, as change in performance is usually equated with learning, in spite of clear evidence against such an assumption. Previous attempts to reconcile learning and performance in RPE models failed to consider the spatial and temporal dimensions of behavior [6, 7, 43]. Previous experiments in support of the RPE hypothesis also neglected confounding variables associated with movements. This is especially true of head-fixed experiments, where overt movements are subtle and difficult to detect. Just because animals are prevented from expressing clear movements does not mean that they do not attempt to move. Only with careful behavioral measures, such as the sensitive force sensors used here, can such subtle behaviors be quantified.

Recent work by Coddington and Dudman also found that VTA DA neurons can both be inhibited or excited during the initiation of movement [27]. Although they attempted to quantify movement in a head-fixed preparation, their behavioral measures lacked spatial resolution. The accelerometer baskets they used could not distinguish between forward and backward movements. Our more sensitive behavioral measures revealed that there are discrete VTA DA populations based on the direction and amplitude of head movement. Furthermore, there is often a pronounced, but very brief, backward movement during reward delivery which coincides with the phasic burst. While their proposed model also attempts to reconcile movement initiation effects and RPE, it also lacks vector quantities, and leaves action vaguely defined, a major limitation in all reinforcement learning models.

Our findings indicate that phasic dopamine bursts signal a sudden change in force associated with rapid but subtle movement in response to salient stimuli. Variable patterns and directions of movement produced in different experimental set ups could explain why conflicting results have been found for the ‘encoding’ of aversive and rewarding events in projection- specific populations of VTA DA neurons [24, 44]. It would be critical for future studies examining the functional heterogeneity in DA neurons to carefully quantify movements in different directions, which can vary with standard manipulations intended to vary reward magnitude, probability, and valence [15, 45].

More recent work has claimed that VTA DA neurons can multiplex many different variables, including reward and movement [11, 12]. While this interpretation may appear on the surface to explain the greatest number of previous findings, it is also challenged by our results. First, the statistical methods used in such studies allow them to describe nearly any type of neural activity as multiplexing any number of behavioral variables. Without an explicit model of how the signals can be de-multiplexed, this interpretation is not even falsifiable. In reality, there is no clear relationship between the neural activity and any of the variables measured, as indicated by very low correlations between behavioral variables and DA activity, probably due to the technical limitations of conventional measures as explained above. More importantly, these studies did not measure the key variable, namely the impulse vector, that actually accounts for most of the variance in the activity of DA neurons (Figure 4). In our study, we were also able to establish a direct causal connection with our optogenetic experiments. Thus, task-relevant variables used in models that purport to predict DA activity may in fact be correlated with forces exerted by the animal or vector components of kinematic variables [34, 35]. Previous studies may have omitted a universal property of the animal’s behavior that co-vary with all the variables selected by the experimenter for analysis.

Impulse and Motivation

Our results are more consistent with the idea that VTA DA plays a key role in motivation by energizing behavior. The magnitude and direction of the movement depends on the intensity and motivational relevance of the stimuli encountered. It has been proposed that VTA DA represents incentive salience of motivationally relevant stimuli, and how much vigor or effort is being expended towards these stimuli [6, 7, 46, 47]. In addition, there is an extensive literature linking VTA output and locomotion[48, 49]. One limitation of such proposals is that, like the RPE hypothesis, they also lack vector quantities and tell us nothing about the actual behavioral trajectories. On the other hand, we show that the impulse vector explains the activity of most VTA DA neurons, and suggests a simple, quantitative, and falsifiable hypothesis of VTA DA function.

In summary, our results indicate that VTA DA neurons regulate anticipatory behavior by controlling the force required to move in specific directions and initiate the appropriate behavioral sequence. We show that the signals carried by the VTA DA neurons represent vector quantities that can be used to predict both the direction and amplitude of forces exerted over time, which is a significant advance towards predicting continuously generated behavior.

While our results significantly advance our understanding of VTA DA’s role in behavior, several caveats must be noted. First, while the impulse vector explained most of the variance in the firing of VTA DA neurons in the fixed-time task used here, it remains to be seen whether this is true in other experimental paradigms such as Pavlovian conditioning tasks. Second, while we were able to optically tag a significant number of VTA DA neurons, they came from a small number of animals. In addition, we were only able to optically tag a few of the SF neurons. It is possible they could be DA neurons that co-express other neurotransmitters or are topographically separate from the other neurons. Further work needs to be done to characterize the neuronal populations described here more fully, to investigate their responses in other behavioral contexts, as well as to determine their precise spatial distribution and molecular markers.

STAR Methods

Resource Availability

Lead Contact

Further information and requests for resources and reagents should be directed to the Lead Contact, Henry Yin (hy43@duke.edu).

Materials Availability

This study did not generate new unique reagants.

Data and Code Availability

Raw data from the current study were not deposited into a public repository due to the large size of the data sets, but are available from the corresponding author upon request.

Experimental Model and Subject Details

All experimental procedures were approved by the Animal Care and Use Committee at Duke University. 13 DAT-ires-cre mice, 4 DAT-Cre + Ai32 mice and 6 wild-type male and female mice were used for experiments with virally delivered opsins (Jackson Labs, Bar Harbor, ME). For experiments involving endogenously expressed opsins, Ai32 mice were crossed with DAT-Cre animals (both heterozygous and homozygous DAT-ires-Cre mice). All mice were aged between 2–8 months old. Mice were housed on a 12:12 light cycle, with tests occurring in the light phase. All mice were housed in groups of 3–4 animals per cage. For experiments, mice were put on water restriction and maintained at 85–90% of their initial body weights. Animals received free access to water for approximately 30 minutes following daily experimental sessions.

Viral Constructs

rAAV5.EF1α.DIO.hChR2(H134R) was obtained from the Duke University Vector Core. pAAV-hSyn1-SIO-stGtACR2-FusionRed was purchased from Addgene. pAAV_hSyn1-SIO-stGtACR2-FusionRed was from Ofer Yizhar (Addgene viral prep # 105677-AAV1; http://n2t.net/addgene:105677; RRID:Addgene_105677).

Surgery

Mice were initially anesthetized with 2.0 to 3.0% isoflurane before being placed into a stereotactic frame (David Kopf Instruments, Tujunga, CA) and were maintained at 1.0 to 1.5 % for surgical procedures. For experiments involving viral injections, 200–300 nL of DIO-ChR2, SIO-StGtaCR2, or DIO-eYFP were bilaterally injected at a rate of 1 nL/s into the VTA (AP: 3.2 – 3.4 mm relative to bregma, ML: 0.4 – 0.6 mm relative to bregma, DV: 4.0 – 4.4 mm relative to brain surface) using a microinjector (Nanoject 3000, Drummond Scientific). The injection pipette was left to sit for 3–5 minutes immediately after the injection to allow absorption of the virus and prevent leakage. Custom-made optic fibers (5 – 6 mm length below ferrule, >80% transmittance, 105 μm core diameter) were implanted at a fifteen-degree angle above the middle of the VTA = (AP: 3.2 – 3.4 mm, ML: 1.6 mm, DV: 3.8 mm). For electrophysiological recordings, drivable electrodes were placed just above the VTA (AP: 3.2 – 3.4, ML: 0.5 mm, DV: −3.8 mm) and 16-channel recording electrodes were lowered into the VTA (AP: 3.2 – 3.4 mm, ML: 0.5 mm, DV: −4.0 – 4.4 mm). For optotagging experiments, an optic fiber was attached to the electrodes. Fibers and electrodes were secured to the skull using screws and dental acrylic and all mice were fitted with a steel head implant for head fixation. All animals were allowed to recover for two weeks before beginning training on the fixed-time task.

Head-fixed behavioral setup

We created a customized head-fixation device for measuring forces exerted by the animals during behavioral testing and stimulation. Animals’ head implants were clamped to a frame that was suspended on a set of 3 load-cells (RB-Phl-203, RobotShop.com) that were arranged orthogonally to detect force changes in three dimensions (forwards-backwards, up-down, and left-right). Animals stood on an enclosed platform that incorporated two additional load-cells that detected downward forces exerted by the left and right feet. Load cells translate small mechanical distortions caused by an applied load into a voltage signal. We amplified the signal using an INA125P (Texas Instruments) in a custom circuit configuration[17]. A metal spout attached to a reservoir containing a 10% sucrose solution was placed close to the animals’ mouth. Sucrose delivery was gravity fed and controlled by the opening of a solenoid valve (161T010, NResearch, NJ). A capacitance-touch sensor (MPR121, AdaFruit.com) clamped to the metal spout was used to detect individual licking events. All continuous load cell voltages were recorded at 1 kHz along with timestamps corresponding to reward, laser train delivery times, individual licks, and electrophysiological data (see below) using a Blackrock Cerebrus recording system (Blackrock Microsystems) for offline analysis.

Behavioral task

Water-deprived animals were first habituated to the head fixation apparatus and trained to receive water rewards delivered manually by the experimenters. Once reliable licking was observed, animals were switched to the fixed-time task, in which mice receive a 5 uL drop of 10% sucrose solution every 10 seconds. Reward delivery was not contingent upon animals’ behavior. On a subset of recording sessions (n = 6), air puffs were delivered during the interval task. The air puffs could occur randomly any time during the interval between 1 and 9 seconds with a probability of 15% for a given trial. The task was controlled using Matlab (version 2018b, Mathworks, MA) commanding digital outputs through a data acquisition system (USB6001, National Instruments, NJ).

Wireless in Vivo Electrophysiology

Both fixed and drivable 16-channel electrode arrays or were used (Innovative Neurophysiology, Inc.). Drivable electrodes were single-drive movable microbundles of tungsten electrodes (1 × 16; 23 μm diameter) placed within a guide cannula. Fixed arrays were composed of tungsten electrodes in a 4 × 4 array (35 μm diameter, 150 μm spacing, 5 mm length). Electrophysiological data were recorded via a miniaturized wireless head stage (Triangle Biosystems) that communicated with a data acquisition system (Blackrock Microsystems). Both analog and digital bandpass filters were applied to the electrophysiological data (analog highpass 1^st order butterworth filter at 0.3 Hz, analog lowpass 3^rdorder butterworth filter at 7.5 kHz, digital highpass 4^th order butterworth filter at 250 Hz). Filtered data were then sorted offline using OfflineSorter (Plexon). All raster plots of spiking activity and force signals were generated using NeuroExplorer (Nex Technologies) using 50 ms timebins and smoothed using a Gaussian filter with a standard deviation of 3 bins. A 3:1 signal-to-noise ratio, and an 800 μs or greater refractory period were required for the neural data to be used for analysis. For driveable electrodes, the electrodes were lowered after each session by 50 μm. In order to be considered a different unit within the same channel, the waveform had to be significantly different from the previously recorded unit as determined by OfflineSorter. For optotagging experiments, optic fibers were epoxied to driveable electrodes.

Histology

Mice were transcardially perfused with 0.1M phosphate buffered saline (PBS) followed by 4% paraformaldehyde (PFA) in order to confirm viral expression as well as optic fiber and electrode placement. To confirm placement, brains were stored in 4% PFA with 30% sucrose for 72 hrs. Tissue was then post-fixed for 24 hours in 30% sucrose before cryostat sectioning (Leica CM1850) at 60 μm. Fiber and electrode implantation sites were then verified. To confirm eYFP and FusionRed expression in DAT+ cells in the VTA of DAT-ires-Cre and DAT + Ai32 transgenic mice, sections were rinsed in 0.1M PBS for 20 min before being placed in a PBS-based blocking solution. The solution contained 5% goat serum and 0.1% Triton X-100 and was allowed to sit at room temperature for 1 hr. Sections were then incubated with a primary antibody (polyclonal rabbit anti-TH 1:500 dilution, ThermoFisher, catalog no. P21962; polyclonal chicken anti-EGFP, 1:500 dilution, Abcam, catalog no. ab13970) in blocking solution overnight at 4 °C. Sections were then rinsed in PBS for 20 min before being placed in a blocking solution with secondary antibody used to visualize DAT neurons in the VTA (goat anti-rabbit Alexa Fluor 594, 1:1000 dilution, Abcam, catalog no. ab150080; goat anti-chicken Alexa Fluor 488, 1:1000 dilution, Life Technologies, catalog no. A11039) for 1 hr at room temperature. Sections were mounted and immediately coverslipped with Fluoromount G with DAPI medium (Electron Microscopy Sciences; catalog no. 17984–24). Placement was validated using an Axio Imager.V16 upright microscope (Zeiss) and fluorescent images were acquired and stitched using a Z780 inverted microscope (Zeiss).

Whole-cell patch clamp recording

For whole-cell patch-clamp recordings, 3 DAT-Cre animals were used. StGtACR2 was injected into the VTA and the mice were sacrificed 8–12 weeks after the injection. The brain was removed quickly and left in ice-cold solution bubbled with 95% O2–5% CO2 containing the following (in mM): 194 sucrose, 30 NaCl, 2.5 KCl, 1 MgCl2, 26 NaHCO3, 1.2 NaH2PO4, and 10 D-glucose. After 5 minutes, 250 μm coronal slices were cut and then placed in 35.5°C oxygenated artificial cerebrospinal fluid (aCSF) solution containing the following (in mM): 124 NaCl, 2.5 KCl, 2 CaCl2, 1 MgCl2, 26 NaHCO3, 1.2 NaH2PO4, and 10 D-glucose. After 30 minutes, the slices were left in aCSF at ~22 −23°C for at least 30 min before recording. Following recovery, whole-cell patch clamp recordings were performed in current clamp mode with continuous perfusion of aCSF at 29–30°C. The internal solution contained (in mM) 150 potassium gluconate, 2 MgCl2, 1.1 EGTA, 10 HEPES, 3 sodium ATP, and 0.2 sodium GTP.

To measure the inhibition by light stimulation, slices were injected with current that was adjusted to evoke action potentials (30 – 150 pA). Current was delivered for 2 seconds; and 500 ms after the start of current injection slices were stimulated with 470-nm light from an LED (Thor Labs). Both constant (1 s) and pulsed stimulation (5 ms pulses at 10, 25 and 50 Hz; MASTER-8) was delivered to the entire ×40 field with an LED current driver (Thor Labs). Power density was ~2 mW/mm². Action potentials were recorded for 500 ms after light stimulation. All recordings were performed with a MultiClamp 700B amplifier (Molecular Device). Signals were filtered at 10 kHz and digitized at 20 kHz with a Digidata 1440A digitizer (Molecular Devices). The inhibition ratio was measured by comparing firing rates from before and after light stimulation.

Optogenetic experiments

Animals were well trained before beginning optogenetic manipulations. Optogenetic stimulation sessions were identical to electrophysiological sessions described above, except for the delivery of light on pseudo-randomly determined trials (see diagram, Figure S6A). After 2 consecutive reward deliveries, the program would enter into a choice point. The probability of the interval following the 3^rd reward containing a laser presentation was 50%. Whether laser was presented or not, the consecutive reward counter was reset to 0 after the delivery of the 3^rd reward. The overall probability of receiving laser stimulation for a given training session was approximately 15%. For experiments with ChR2, optical power was measured at 8 mW at the end of the fiber that connected to the optical implants. For experiments with stGtaChR2, power was capped at 5 mW.

Quantification and Statistical Analyses

All analyses were performed using Matlab, Python, NeuroExplorer, and Graphpad Prism. All statistical analyses were performed in MATLAB and GraphPad Prism. A power analysis was not conducted to determine sample size a priori.

Force conversions

To calculate the forces exerted by animals on the load-cells, we calibrated each load-cell circuit by determining a linear conversion factor (expressed in Newtons per Volt) between the load cell circuit output and known masses placed on the sensor. Force was determined by multiplying the recorded voltage signal by the conversion factor to obtain the force in Newtons exerted by the animals on each load cell.

Detection of movement initiation

We applied thresholds to the forward/backward force signal to detect forward and backward movement events. Forward movements were defined as continuous events exceeding 150 mN lasting longer than 250 ms and separated by at least 100ms. Backward movements were defined as events lower than −300 mN lasting longer than 100 ms and separated by at least 100 ms.

Optical tagging and unit classification

Tagging of dopamine neurons was performed during the performance of the task with laser stimulation occurring at −6 seconds prior to reward. Neural spiking data was recorded simultaneously with timestamps for optical stimulation pulses. Neural data and optic stimulation timestamps were imported into NeuroExplorer and aligned to the start of stimulation. Neurons were classified as tagged DA neurons if the first evoked action potential had a latency of less than 6 ms and resulted in a waveform identical to the spontaneously-occurring waveform for that unit. In order to classify non-tagged neurons as DA, optically tagged and non-tagged waveforms were exported from Plexon into Matlab, where the valley full-width half max (FWHM) was calculated using a custom Matlab script. The average FWHM and firing rate were then computed. The majority of the optically tagged neurons had a valley FWHM of at least 500 μs and a firing rate below 10 Hz. The remaining non-tagged DA neurons were classified as VTA DA neurons if their firing rate was below 10 Hz and their valley FWHM was over 500 μs.

Functional classification of DA neurons

Neurons were initially manually observed as either a fast-forward (FF), fast-backward (FB), or slow-forward (SF) DA neurons based on their activity during the time of reward, forward and backward movement. To further confirm and validate distinct functional VTA DA populations, an unbiased clustering algorithm (Gaussian mixture model [GMM]) was used. Neural data from units classified as DA were aligned to reward, forward movement and backward movement in Neuroexplorer, and then exported to Microsoft Excel. Data were then concatenated into one response profile and analyzed using Python 3 and the sci-kit learn package. Data was smoothed with a Guassian filter (sigma=2), and then scaled (MinMaxScaler). The scaled response profiles were whitened, and a PCA decomposition (3 components) was performed on the data. The resulting principal components were clustered using a GMM (3 clusters). Over 95% of neurons initially classified as an FF, FB, or SF neuron fell within the same cluster using the GMM.

Analysis of behavioral and neural data

To examine the relationship between neural activity and force generated over time (impulse), we first identified all forward and backward movement events occurring in a recording session (see threshold criteria above). We compared spiking activity of FF and SF neurons to impulse of forward movements, and spiking activity of FB neurons to impulse of backward movements. Impulse was estimated using the trapz function in Matlab as the area under the curve of F/B force over the duration of each movement. We also recorded the peak force for each movement and counted the number of spikes recorded for each DA neuron in the same time interval. Thus, for each neuron and each movement event, we obtained a single number for impulse, a single number for peak of the force signal, and a single number of spikes. For each neuron, we correlated all spike count values with the corresponding impulse values, and obtained an average impulse per spike count for each cell. These values were averaged across neurons from each functional class. In order to compare across different neurons, we plotted impulse as a function of spike number up to 7 spikes (7 was the maximum spike number that all neurons have in common during a movement). To account for the fact that neural activity often preceded force generation, for each neuron we adjusted the time window used to count spikes associated with each movement by the lag estimated from the cross-correlation with force over the entire session (see method for cross-correlation below).

Analysis of force and licking in optogenetic experiments

Force signals and lick timestamps were aligned to the onset of laser delivery in NeuroExplorer and binned at 50 ms. The mean peri-event force signal across all laser presentation trials in the forward/backward direction was exported to Matlab where we estimated the maximum value and also found the area under the curve using the trapz function. This resulted in one value for each animal for each laser stimulation condition. Trials were excluded from this average if laser delivery did not evoke any movement. These values were averaged together before regression analysis with pulse number. For estimation of force generated in control animals, we obtained the average duration of the movements evoked by a given stimulation parameter. We then determined the maximum force and measured impulse of the mean force signals that were recorded over the same durations following laser activation in control experiments. To quantify negative impulses that resulted from inhibition of VTA DA neurons, we superimposed the mean force signals generated during trials with and without laser, and measured the area between the two curves. This impulse measure was determined for the first second of laser presentation which coincided with the anticipatory phase of the interval. Impulse resulting from inhibition rebound was determined in the same manner as with ChR2 experiments described above. Lick rate was quantified by determining the mean lick rate within a 1 second window after the start of stimulation in NeuroExplorer. Trials following stimulation were selected by taking the first and second rewards following stimulation. Force and licking rate were averaged across 5 seconds prior to reward for ‘pre-reward’ condition and for 2 seconds following reward for ‘post-reward’ condition.

Analysis of Forward Force Movements with No Lick Bouts

In order to determine if VTA DA neural activity is still correlated with force in the absence of lick bouts, identified non-rewarded forward and backward movement events that occurred outside of periods of licking. Lick bouts consisted of licking events with a maximum inter-lick-interval of 0.11 s and had to contain more than one lick. Neural data was then aligned to these filtered forward movements. All three populations showed similar relationships with F/B force during periods without any reward delivery or lick bouts being initiated (Figure S6).

Support Vector Regression Decoder

Support vector regression was implemented via the scikit-learn Python package to fit the continuous forces recorded from each load cell (forward/back, up/down, side/side) using the firing rate (bin size: 50 ms) of DA neurons. As the number of neurons and the size of the training data affects decoding performance, we limited our analysis to sessions that contained at least 6 DA neurons (63% of datasets; maximum number of DA neurons was 20). Prior to fitting the model, the neural data was z-scored, and the force was zero-centered. We then convolved the force data with a Gaussian filter (width: 5 bins). For each dataset, the model was trained on the first 60% of the data, and performance was evaluated on a continuous set of held-out data (15%).

Correlation Analyses Across Entire Behavioral Session

For each animal, neural data and continuously monitored F/B force or lick rate for the entire recording session were constructed using 10 ms time bins in NeuroExplorer and exported to MATLAB. A custom matlab script was then used to sort neural firing rate variables and the corresponding force signal or licking rate according to firing rate magnitude. Neural activity was binned into twenty bins, and the mean force signal or licking rate corresponding to the firing rate in each bin was calculated. The Pearson correlation was calculated between binned neural activity and mean force signal or licking rate.

Cross-Correlation Analyses

A custom MATLAB script was written in order to perform cross-correlation analyses between the force measurements and the corresponding VTA DA neuronal subtype. Cross-correlations were performed in MATLAB by utilizing the in-built function xcorr. The behavioral variable was used as the reference time-series and the neural data as the shifted time series. Latencies were obtained by determining the lag of the maximum value of the cross-correlation for positively correlated neurons and the lag to the minimum value for anti-correlated neurons. Cross-correlations were then Z-score normalized and all data sets from all animals were averaged together to create population average plots.

Supplementary Material

3

Video S1. Stimulation of VTA DA neurons results in the generation of force in the forward direction along with licking in a reward-timing task. Related to Figure 6.

Video shows a representative trial in which DAT::ChR2^VTA stimulation results in forward force generation and anticipatory licking. Laser stimulation was 40 pulses, delivered 6 seconds prior to reward delivery at 20 Hz, and is indicated with the blue square.

4

Video S2. Single light pulse stimulation of VTA DA neurons is sufficient to produce forward force. Related to Figure 6.

Video shows forward force generation and anticipatory licking following single light pulse stimulation of VTA DA neurons in DAT::ChR2^VTA mice. Stimulation occurred during the fixed-time task at 6 seconds prior to reward. The blue rectangle presented in the slow-motion replay indicates time of single-light pulse delivery.

Key Resources Table.

REAGENT or RESOURCE	SOURCE	IDENTIFIER
Antibodies
Goat polyclonal anti-Rabbit Alexa Fluor 488	Abcam	Cat# ab150077; RRID: AB_2630356
Goat polyclonal anti-Rabbit Alexa Fluor 594	Abcam	Cat# ab150080; RRID: AB_2650602
Rabbit polyclonal anti-Tyrosine Hydroxylase	Millipore	Cat# 657012; RRID: AB_566341
Bacterial and Virus Strains
rAAV5-EF1α-DIO-hChR2(H134R)-eYFP	Duke Vector Core	N/A
AAV1-hSyn-SIO-stGtACR2-FusionRed	Addgene	Cat# 105677-AAV1
Experimental Models: Organisms/Strains
Mouse: DAT-ires-Cre:1tm2(cre)Lowl/J	Jackson Laboratory	Mouse Strain: 006660
Mouse: C57BL/6J	Jackson Laboratory	Mouse Strain: 000664
Mouse: Ai32: B6;129S-Gt(ROSA)26Sortm32(CAG-COP4*H134R/EYFP)Hze/J	Jackson Laboratory	Mouse Strain: 012569
Software and Algorithms
MATLAB 2016b	MathWorks	https://www.mathworks.com/products/new_products/release2015b.html
Python 2.7	Anaconda	https://www.anaconda.com/download/?lang=en-us
Offline Sorter 4.0	Plexon	https://plexon.com/products/offline-sorter/
NeuroExplorer 4.0	Nex Technologies	http://www.neuroexplorer.com/downloadspage/
Cortex 5.0	MotionAnalysis	http://ftp.motionanalysis.com/html/industrial/cortex.html
GraphPad Prism 8	GraphPad	https://www.graphpad.com/scientific-software/prism/
Other
LED Driver	Thorlabs	LEDD1B
DAPI Fluoromount-G	Southern Biotech	Cat# 0100-20