Pattern of dopamine signaling during aversive events predicts active avoidance learning

Significance

Exposure to aversive stimuli can elicit a range of responses across a population, with some readily learning to avoid the aversive outcome whereas others do not. The neurotransmitter dopamine is thought to regulate behavioral responses to aversive stimuli. However, it is unclear whether intrinsic differences in dopamine signaling can account for the diversity of actions exhibited in response to an aversive event. We performed voltammetry recordings of dopamine transmission in the ventral striatum of rodents exposed to inescapable and escapable aversive stimuli. These experiments 1) identified the characteristics of the dopamine response during an aversive event that are predictive of learning to avoid aversive outcomes and 2) demonstrated that dopamine encodes a safety prediction error signal.

Abstract

Learning to avoid aversive outcomes is an adaptive strategy to limit one’s future exposure to stressful events. However, there is considerable variance in active avoidance learning across a population. The mesolimbic dopamine system contributes to behaviors elicited by aversive stimuli, although it is unclear if the heterogeneity in active avoidance learning is explained by differences in dopamine transmission. Furthermore, it is not known how dopamine signals evolve throughout active avoidance learning. To address these questions, we performed voltammetry recordings of dopamine release in the ventral medial striatum throughout training on inescapable footshock and signaled active avoidance tasks. This approach revealed differences in the pattern of dopamine signaling during the aversive cue and the safety period that corresponded to subsequent task performance. Dopamine transmission throughout the footshock bout did not predict performance but rather was modulated by the prior stress exposure. Additionally, we demonstrate that dopamine encodes a safety prediction error signal, which illustrates that ventral medial striatal dopamine release conveys a learning signal during both appetitive and aversive conditions.

Learning to avoid aversive outcomes can subsequently mitigate the influence of aversive cues on behavior (1, 2). However, the response toward an aversive stimulus varies considerably among subjects (3–5). For example, some rodents in a population will readily learn to avoid an aversive outcome whereas others do not (3, 4, 6). These behavioral differences could arise from intrinsic differences in the neural response during aversive events in circuits mediating reinforcement learning and motivated behavior. One candidate is the mesolimbic dopamine system, which regulates motivation and facilitates reinforcement learning in reward-based tasks (7–10). In support, stimulating dopamine neuron activity improves active avoidance performance (11). However, it is unclear whether intrinsic differences in dopamine signaling account for the heterogeneity in active avoidance learning within a population.

The dynamics of dopamine transmission throughout reinforcement learning have been studied primarily in the context of reward-based tasks. During stimulus-reward learning, the value of the outcome is initially conveyed by the dopamine response to the reward (12). As the cue-reward relationship is acquired, dopamine release to the reward decays, and the value of the outcome is instead signaled by dopamine release to the cue (12–14). Dopamine also conveys a reward prediction error signal that reflects the difference between the expected reward and actual reward that is received (15). If dopamine functions in an analogous manner during aversive tasks, the dopamine response to the aversive stimulus should dissipate over training and transfer to the aversive cue. Additionally, dopamine should also convey a safety prediction error signal (16). However, to date the dynamics of dopamine signaling throughout active avoidance learning have not been established.

Here, we performed voltammetry recordings of dopamine transmission in the ventral medial striatum (VMS) in rats trained on inescapable footshock or signaled active avoidance tasks. By recording dopamine levels across training, we could determine whether patterns of dopamine signaling during aversive events are predictive of subsequent active avoidance learning. Furthermore, we could ascertain if dopamine functions similarly during aversive and appetitive learning.

Results

Dopamine Motifs for Distinct Behavioral Responses During Aversive Events.

Voltammetry recordings in the VMS were performed in male rats undergoing training on an inescapable footshock task or a signaled active avoidance task (SI Appendix, Fig. S1). The inescapable task allowed us to relate differences in active responding during aversive events to VMS dopamine levels without the confounding influence of successfully escaping/avoiding the aversive stimulus. In this manner, we could identify behaviorally relevant patterns of dopamine signaling from the inescapable footshock task to serve as a template to predict performance in the active avoidance task. In both the inescapable footshock and active avoidance tasks, trials consist of presenting a footshock-predicting cue (lever extension, cue light illumination, and whitenoise; 30 s), a bout of footshocks (0.3 mA for 0.5 s, delivered every 3 s during a 60-s bout), followed by the safety period in which the footshocks terminate and the cue is removed (90 s). Rats trained on the signaled active avoidance task could either escape or avoid the footshock by pressing the lever during the footshock bout or cue period, respectively. In contrast, lever presses had no consequences in the inescapable variant of the task (Fig. 1A).

Fig. 1.

Ventral striatal dopamine levels relate to lever pressing during exposure to inescapable aversive stimuli. (A) Trial structure for the inescapable footshock task. (B) Representative color plots of voltammetry recordings in the VMS illustrating the dopamine response to the aversive cue (Left), the aversive stimulus (Center), and the safety period (Right). (C) Average dopamine response across electrodes. (D) Lever presses across training sessions, ***P < 0.001 main effect of pressing. (E) Occurrence of lever presses within trials, **P < 0.01 post hoc Sidak’s test. (F) Cue-elicited freezing, ***P < 0.01 main effect of pressing. (G) Average dopamine traces over all training sessions. (H–J) Timecourse of the dopamine response during the cue (H), the footshock bout (I), and the safety period (J), **P < 0.01, ***P < 0.001 main effect of pressing.

Rats trained on the inescapable footshock task exhibit a sustained reduction in VMS dopamine levels during the aversive cue, a transient reduction in dopamine levels during the footshock bout, and an increase in dopamine release during the safety period (n = 15 electrodes; Fig. 1 B and C). We categorized rats based upon the amount of nonreinforced lever presses performed over inescapable footshock training sessions. Rats completing at least 2 consecutive sessions with >1 press per trial were classified as “High Pressing,” whereas all other rats were classified as “Low Pressing” (n = 4 High Pressing; n = 6 Low Pressing; Fig. 1D). High Pressing rats exhibited an elevated level of responding across training sessions (2-way ANOVA: group effect F_1,80 = 45.7, P < 0.001; session effect F_9,80 = 0.4, P = 0.92; interaction effect F_9,80 = 0.8, P = 0.66; Fig. 1D), although there was no difference in responding on the first session between the groups (unpaired t test: t₈ = 0.0, P = 0.98). Lever presses were performed primarily during the footshock bout, with few responses occurring during the cue (2-way ANOVA: group effect F_1,16 = 8.5, P = 0.01; presses within a trial effect F_1,16 = 11.7, P = 0.0035; interaction effect F_1,16 = 7.6, P = 0.014; post hoc Sidak’s test: t₁₆ = 4.0, P = 0.002; Fig. 1E). Cue-elicited freezing was unchanged over training in both groups, with a greater level of freezing observed in High Pressing rats (2-way ANOVA: group effect F_1,79 = 31.0, P < 0.001; session effect F_9,79 = 0.2, P = 0.99; interaction effect F_9,79 = 0.2, P = 0.99; Fig. 1F). These data illustrate that both active responses (lever presses) and passive responses (freezing) were elevated in High Pressing rats during exposure to inescapable aversive events.

High and Low Pressing rats additionally differed in the pattern of VMS dopamine signals throughout the inescapable footshock task (n = 5/10, High/Low Pressing electrodes; Fig. 1G). Specifically, the reduction in dopamine levels during the aversive cue was attenuated in High Pressing rats (2-way ANOVA: group effect F_1,78 = 8.1, P = 0.0056; time effect F_5,78 = 2.7, P = 0.027; interaction effect F_5,78 = 0.2, P = 0.96; Fig. 1H). In contrast, there was no difference in the dopamine response throughout the footshock bout (2-way ANOVA: group effect F_1,221 = 0.1, P = 0.79; time effect F_16,221 = 0.5, P = 0.96; interaction effect F_16,221 = 0.0, P = 0.99; Fig. 1I). Finally, the increase in dopamine release during the safety period was absent in High Pressing rats (2-way ANOVA: group effect F_1,78 = 36.1, P < 0.001; time effect F_5,78 = 2.7, P = 0.029; interaction effect F_5,78 = 0.5, P = 0.75; Fig. 1J). These data indicate the dopamine response during the aversive cue and the safety period could provide a neurochemical signature of the behavioral strategies employed in response to aversive situations. As employing active strategies can facilitate active avoidance learning, we theorized that the dopamine profile of High and Low Pressing rats would map onto the characteristics of rats that respectively learn or fail to learn in an active avoidance task.

A separate group of rats were trained on a signaled active avoidance task, which was identical to the inescapable footshock task except that lever presses resulted in escaping or avoiding the footshock bout (Fig. 2A). Animals were categorized as either a “Learner” or a “Non-Learner” based upon whether a rat successfully escaped or avoided more than half of the trials for at least 2 consecutive sessions (n = 12 Learners, n = 7 Non-Learners). Learner rats exhibited a progressive increase in the number of trials per session that were escaped or avoided (one-way ANOVA, F_9,107 = 3.0; P = 0.003), which was absent in Non-Learner rats (1-way ANOVA, F_9,59 = 0.4, P = 0.94; Fig. 2B). Learner rats also exhibited a faster latency to respond across training sessions (1-way ANOVA: Learner, F_9,107 = 2.8; P = 0.006; Non-Learner, F_9,59 = 0.4, P = 0.94; Fig. 2C). Cue-elicited freezing was not related to the number of successful trials (r² = 0.02, P = 0.63; Fig. 2D), which illustrates that freezing is not indicative of subsequent active avoidance learning.

Fig. 2.

Pattern of dopamine signaling on failed trials is related to subsequent active avoidance learning. (A) Trial structure for the active avoidance task. (B) Escaped/Avoided trials across training sessions. (C) Latency to respond. (D) Cue-elicited freezing as a function of task performance. (E) Average dopamine traces on failed trials without a lever press. (F–H) Timecourse of the dopamine response during the cue (F), the footshock bout (G), and the safety period (H), ***P < 0.001 main effect of performance. (I) Dopamine response profile exclusive for High Pressing rats (blue shading) and Low Pressing rats (red shading). (J) Overlay of the dopamine response profiles for High and Low Pressing rats onto the dopamine response profile for Learner and Non-Learner rats.

We next examined the dopamine response on failed trials in which rats did not make a lever press, which allowed us to perform a like-for-like comparison between Learner and Non-Learner rats (Fig. 2E). Learner rats exhibit a smaller reduction in dopamine levels during the aversive cue (2-way ANOVA: group effect F_1,114 = 15.4, P < 0.001; time effect F_5,114 = 7.0, P < 0.001; interaction effect F_5,114 = 0.4, P = 0.85; Fig. 2F). Dopamine transmission throughout the footshock bout was no different between Learner and Non-Learner rats (2-way ANOVA: group effect F_1,270 = 2.0, P = 0.15; time effect F_16,270 = 0.3, P = 0.99; interaction effect F_16,270 = 0.1, P = 0.99; Fig. 2G). However, the increase in dopamine release during the safety period was attenuated in Learner rats (2-way ANOVA: group effect F_1,114 = 17.2, P < 0.001; time effect F_5,114 = 1.7, P = 0.15; interaction effect F_5,114 = 0.2, P = 0.94; Fig. 2H). The differences in the dopamine response during the cue and safety period between Learner/Non-Learner rats parallel the respective differences between High/Low Pressing rats (Fig. 1).

These data suggest the pattern of dopamine signaling during the inescapable footshock task could be utilized to predict Learner/Non-Learner rats based upon the dopamine signatures observed during the failed trials of the active avoidance task. To address this, we plotted cue-elicited dopamine levels as a function of safety dopamine levels from the inescapable task, graphically illustrating dopamine response profiles that encompassed only High or Low Pressing rats (Fig. 2I). Mapping these boundaries onto the dopamine response from failed trials on the active avoidance task predicted subsequent avoidance learning with 87.5% accuracy (21 of 24 sessions in the defined quadrants; Fig. 2J). Likewise, applying the dopamine response profile from Learners and Non-Learners faithfully predicted the behavioral phenotype in the rats trained on the inescapable footshock task (29 of 29 sessions in the defined quadrants; See SI Appendix, Fig. S2). Additionally, a linear discriminant analysis model trained with the dopamine response profiles of High/Low Pressing rats classified Learner/Non-Learner rats in the active avoidance task above chance levels of accuracy (SI Appendix, Fig. S2). Together, these data illustrate that the behavioral actions in response to aversive stimuli are related to the pattern of VMS dopamine signaling throughout the aversive event.

Dopamine Signals Affected by Exposure to Aversive Stimuli.

Prior experience with aversive stimuli can alter the firing properties of midbrain dopamine neurons (17, 18), which suggests increasing exposure to footshocks could induce changes in dopamine transmission throughout the aversive event. However, there was no relationship between dopamine levels during the cue and the cumulative amount of footshocks experienced in rats trained on the inescapable footshock task (High Pressing: r² = 0.00, P = 0.57; Low Pressing rats: r² = 0.00, P = 0.80; Fig. 3A). Similarly, prior footshock experience did not affect the cue-elicited dopamine response on trials without an avoid response during the active avoidance task (Learners: r² = 0.0, P = 0.77; Non-Learners: r² = 0.00, P = 0.82; Fig. 3B).

Fig. 3.

Increasing exposure to footshock modulates dopamine signals during the aversive stimulus and not during the aversive cue. (A and B) Dopamine levels during the aversive cue as a function of the cumulative prior exposure to footshock trials in rats trained on the inescapable (A) and active avoidance task (B). (C–E) Dopamine levels throughout the aversive stimulus bout as a function of prior exposure to footshock in High Pressing rats (C), Low Pressing rats (D), and Non-Learner rats (E), ***P < 0.001 main effect of prior shock.

We next examined how dopamine signaling during the footshock bout was affected by exposure to the aversive stimulus. A median split was performed on the data based upon the maximum number of prior footshock trials rats could have received over training sessions. The decrease in dopamine levels during the footshock bout was abolished with increasing exposure to the aversive stimulus in both High Pressing rats (2-way ANOVA: prior shock effect F_1,136 = 32.6, P < 0.001; time effect F_16,136 = 1.4, P = 0.18; interaction effect F_16,136 = 0.1, P = 0.99; Fig. 3C) and Low Pressing rats trained on the inescapable footshock task (2-way ANOVA: prior shock effect F_1,255 = 17.7, P < 0.001; time effect F_16,255 = 0.8, P = 0.68; interaction effect F_16,255 = 0.1, P = 0.99; Fig. 3D). This effect was also evident in Non-Learner rats when examining the footshock dopamine response on the trials without an escape response (2-way ANOVA: prior shock effect F_1,221 = 52.7, P < 0.001; time effect F_16,221 = 0.4, P = 0.98; interaction effect F_16,221 = 0.2, P = 0.99; Fig. 3E). These data illustrate that increasing exposure to aversive stimuli selectively alters dopamine levels during the aversive stimulus, without affecting dopamine levels during the aversive cue.

Dopamine Encodes a Safety Prediction Error Signal.

In appetitive tasks, the dopamine neuron response to the reward delivery decreases as the outcome becomes fully expected and increases when the outcome is better than expected (15, 19). While this reward prediction error signal is encoded by dopamine release in the VMS (20), it is not known whether dopamine serves a similar role during aversive behaviors. If dopamine conveys a safety prediction error signal, dopamine levels during the safety period should 1) decrease as the action–outcome relationship is learned in the active avoidance task and 2) update when the expected occurrence of the safety period is changed. To address this, we first examined the dopamine response during the safety period in Learner rats following escape and avoid responses. Plotting the cumulative responses over training sessions demonstrates that Learner rats predominantly perform escape responses before executing avoid responses (paired t test, t₁₁ = 3.9, P = 0.003; Fig. 4 A and B and SI Appendix, Fig. S3). This pattern of responding indicates rats initially learn to lever presses during the footshock bout (escape) before learning to lever press during the cue (avoid). Consistent with dopamine encoding a safety prediction error signal, escape trials elicited a long-lasting increase in dopamine levels following the lever press, which was absent following avoid responses (2-way ANOVA: trial type effect F_1,132 = 22.0, P < 0.001; time effect F_5,132 = 0.9, P = 0.50; interaction effect F_5,132 = 0.3, P = 0.91; paired t test: t₁₂ = 2.6, P = 0.02; Fig. 4 C and D). In contrast to the performance-related dopamine signals during the safety period, there was no difference in the sustained dopamine response during the aversive cue between avoid trials and trials without a lever press during the cue (SI Appendix, Fig. S4). We next examined if lever presses were preceded by a transient elevation in dopamine levels, as premotor increases in dopamine transmission are commonly observed in reward-based operant tasks (21–23). However, analysis of the premotor dopamine response on avoid trials revealed a decrease in dopamine levels before lever presses in early training sessions, but not in late training sessions (paired t test, t₁₀ = 2.8, P = 0.02; Fig. 4 E and F). Therefore, the presence and directionality of a premotor dopamine response is influenced by the type of reinforcement that can be earned as well as by the training history.

Fig. 4.

The dopamine response during the safety period tracks active avoidance performance. (A) Example Learner rat illustrating the cumulative number of escape and avoid responses as a function of training sessions. Large symbols denote sessions where >50% cumulative responses occurred. (B) Sessions where >50% cumulative response occurred for Escape and Avoid responses across Learner rats, **P < 0.01 paired t test. (C) Time course of the dopamine response during the safety period on Escape and Avoid trials in Learner rats, ***P < 0.001 main effect of trial type. (D) Average safety dopamine response across electrodes, *P < 0.05 paired t test. (E) Premotor dopamine response on avoid trials. (F) Premotor dopamine response across training sessions, *P < 0.05 paired t test. (G) Time course of dopamine response during the safety period on Escape and No Response trials in Non-Learner rats.

If the safety dopamine response reflects the process of learning the action–outcome relationship, it follows that the safety dopamine response in Non-Learner rats should be unrelated to task performance. While Non-Learner rats infrequently performed escape responses (average of 2.4 per session), there was no improvement over training sessions (Fig. 2 B and C), which indicates these rats did not acquire the action–outcome relationship. Dopamine levels during the safety period were no different between escape trials and trials without a response in Non-Learner rats (2-way ANOVA: trial type effect F_1,84 = 0.0, P = 0.90; time effect F_5,84 = 0.9, P = 0.45; interaction effect F_5,84 = 0.0, P = 0.99, Fig. 4G). Together, the data from Learner and Non-Learner rats illustrates that task performance is reflected in the dopamine response during the safety period.

For dopamine to convey a safety prediction error signal, dopamine release during the safety period should decrease with training as the shock cessation becomes predictable and increase if safety is encountered unexpectedly. To determine if these criteria were supported by the dopamine responses from inescapable trained rats, we first examined how safety dopamine levels were affected by training history. Increasing exposure to footshocks as a consequence of training attenuated dopamine release during the safety period in Low Pressing rats (2-way ANOVA: prior shock effect F_1,90 = 0.0, P = 0.015; time effect F_5,90 = 1.9, P = 0.11; interaction effect F_5,90 = 0.1, P = 0.99; Fig. 5A) but did not affect the safety dopamine response in High Pressing rats (2-way ANOVA: prior shock effect F_1,48 = 0.2, P = 0.65; time effect F_5,48 = 4.8, P = 0.0013; interaction effect F_5,48 = 0.1, P = 0.98; Fig. 5B). We posited that the absence of dopamine release during the safety period could represent the learning of the temporal dynamics of the task. In other words, the offset of the aversive stimulus after 60 s of footshock would be fully anticipated and, as such, there would be no safety prediction error signal. To test this hypothesis, a subset of rats underwent a single training session comprised of trials in which the footshock bout was shortened to 15 s in duration (Catch Session; n = 7 electrodes from 4 rats, equally split between Low and High Pressing rats; Fig. 5C). With the safety occurring earlier than expected, the safety dopamine response was elevated during the Catch session relative to the preceding standard training session (2-way ANOVA: session effect F_1,72 = 82.4, P < 0.001; time effect F_5,72 = 2.2, P = 0.06; interaction effect F_5,72 = 3.6, P = 0.0054; Fig. 5D). Furthermore, the Catch session only influenced dopamine levels during the safety period (paired t test: t₆ = 4.1, P = 0.0065; Fig. 5G) and was without effect on dopamine during the cue (paired t test: t₆ = 1.0, P = 0.10; Fig. 5E) or the footshock bout (paired t test: t₆ = 0.7, P = 0.48; Fig. 5F). Collectively, these experiments demonstrate that dopamine in the VMS encodes a safety prediction error signal.

Fig. 5.

Dopamine encodes a safety prediction error signal. (A) Dopamine response during the safety period in Low Pressing rats, *P < 0.05 main effect of prior shock. (B) Dopamine response during the safety period in High Pressing rats. (C) Schematic of trials during the Catch session. (D) Dopamine response during the safety period on the Catch session and the preceding standard training session, ***P < 0.001 main effect of session. (E–G) Average dopamine response during the aversive cue (E), the footshock bout (F), and the safety period (G), **P < 0.01 paired t test.

Discussion

Differences in the behaviors exhibited during appetitive learning have linked distinct patterns of dopamine release to rewards and reward-predictive cues (12). Here, we demonstrate that behavioral differences during aversive tasks are also accompanied by distinct patterns of VMS dopamine signaling. Engaging in active behavioral strategies, such as lever presses during an inescapable aversive stimulus, was associated with a difference in dopamine levels during the aversive cue and the safety period, but not during the aversive stimulus itself. Furthermore, subsequent active avoidance learning can be inferred based upon the profile of the dopamine response to the cue and safety period. Prior work has demonstrated that the distinct behavioral phenotypes exhibited following exposure to aversive situations is accompanied by differences in dopamine neuron firing patterns (17, 18). However, it is unclear whether the behavioral phenotypes result from differential stress-induced neuronal adaptations or, alternatively, from preexisting differences in dopamine transmission. Our voltammetry data recorded over training sessions supports the latter, as we have identified a dopamine signature during aversive events that is indicative of future active avoidance learning.

In appetitive tasks, the dopamine response to a reward conveys a prediction error signal that reflects the difference between the expected reward and the actual reward that is delivered (15). In aversive tasks, dopamine levels increase during the safety period, although it was unclear whether this dopamine signal conveys the expectancy that the aversive stimulus will end (16, 24, 25). We demonstrate that dopamine release during the safety period decreases as the action–outcome relationship is learned in the active avoidance task. Additionally, safety dopamine levels increase when the safety period occurs earlier than expected. These data collectively illustrate that dopamine in the VMS encodes a safety prediction error signal. Despite the similarities in dopamine’s ability to convey both reward and safety prediction error signals, the evolution of the dopamine response in our aversive task is inconsistent with reinforcement learning models that are frequently applied in reward-based paradigms. For example, temporal difference reinforcement learning models account for the observed transfer of the dopamine signal from the reward to the reward-predictive cue across learning in appetitive tasks (13–15). However, dopamine levels during the aversive cue did not change, although the dopamine response during the aversive stimulus diminished over training sessions. Therefore, traditional reinforcement learning models used to account for dopamine neuron function during appetitive tasks may not be broadly applicable to aversive paradigms.

While dopamine release in the VMS encodes both reward and safety prediction error signals, it is important to note that the time course of the dopamine response to the safety period is substantially longer than the dopamine response to food reward delivery (20, 26, 27). This observation suggests that distinct patterns of afferent input to midbrain dopamine neurons and/or regulation at VMS dopamine terminals contribute to the differential duration of the dopamine response during positive and negative reinforcement (28–30). During reward-seeking behavioral tasks, dopamine levels increase before engaging in an operant action (22, 23). In contrast, we find that dopamine levels decrease before lever presses during negative reinforcement, which illustrates the directionality of the premotor dopamine response depends upon the type of reinforcement that can be earned. Despite the parallels between reward and safety prediction error signals conveyed by dopamine, the notable differences in dopamine transmission during positive and negative reinforcement indicates dopamine likely serves distinct functions in appetitive and aversive tasks.

Microdialysis experiments have reported both increases and decreases in dopamine levels during aversive tasks (31–34). We identified both increases and decreases in VMS dopamine levels that occurred during distinct periods of the aversive event, which suggests the discrepancies between earlier microdialysis studies likely arise from differences in the experimental parameters of the aversive task. We previously found that the engagement of the dopamine system during reward-based Pavlovian conditioning can be influenced by changes in the training context (35). Therefore, the training procedures, aversive cue duration, intensity and frequency of the footshock, presence of safety cues, and location of the recordings in the ventral striatum are all factors that likely contribute to the dopamine signals observed during aversive tasks (11, 25, 31–34, 36–38). Irrespective of these methodological considerations, our data illustrates VMS dopamine levels can be both increased and decreased during different elements of an aversive event.

Prior studies employing pharmacological and genetic approaches have demonstrated that dopamine transmission is required for active avoidance learning and performance (32, 39). By examining individual variability across animals, we found that Low Pressing/Non-Learner rats exhibited lower dopamine levels only during the aversive cue relative to their High Pressing/Learner counterparts. Given that dopaminergic tone influences motivated behavioral output (40–42), a greater cue-elicited reduction in dopamine levels may prevent active responding. Therefore, pharmacologically antagonizing dopamine signaling functionally recapitulates the dopamine phenotype during the aversive cue that is observed in Low Presser/Non-Learner rats. Recent work supports this prediction, as stimulating dopamine neurons during the aversive cue improved active avoidance performance (11). Whereas dopamine during the aversive cue relates to behavioral performance and is unaffected by increasing footshock exposure, dopamine during the aversive stimulus is unrelated to behavioral performance and is modulated by increasing footshock exposure. Together these results illustrate that temporally defined dopamine signals in the VMS convey information that is predictive of the behavioral response to aversive situations and indicative of the cumulative prior exposure to aversive stimuli.

When encountering an aversive stimulus, one must determine what action to pursue as well as how intensely to engage in the chosen action. Rodents can exhibit both passive and active responses toward aversive stimuli, although these behaviors are often assayed in separate tasks (fear/threat conditioning and active avoidance, respectively). The inescapable footshock task we employed allowed us to assay both passive responses (freezing) and active responses (unreinforced lever presses) in the same setting. Rats exhibiting high levels of passive responding also exhibited high levels of active responding. Therefore, individual heterogeneity in the intensity of the response to an aversive stimulus is expressed during both active and passive behaviors. Animals that are either highly responsive or minimally responsive to aversive situations exhibit distinct intrinsic patterns of dopamine signaling in the ventral striatum. It is important to note that aversive behaviors are not mediated by a single brain region, but rather depends upon circuits involving the thalamus, cortex, amygdala, and ventral striatum (43). As such, individual heterogeneity in the intensity of the behavioral response during aversive situations is likely controlled by intrinsic differences in the pattern of neural activity throughout these circuits.

Methods

Subjects and Surgery.

All procedures were approved by the Institutional Animal Care and Use Committee at the University of Texas at San Antonio. Male Sprague–Dawley rats (Charles River) were pair-housed upon arrival and given ad libitum access to water and chow and maintained on a 12-h light/dark cycle. Rats were single-housed following surgery for the duration of the experiment. Voltammetry electrode implantation surgeries were performed under isoflurane anesthesia on rats weighing 300–350 g. During these surgeries, carbon fiber electrodes were implanted bilaterally in the VMS (relative to bregma: 1.3 mm anterior; ±1.3 mm lateral; 7.0 mm ventral) along with a Ag/AgCl reference electrode placed under the skull.

Behavioral Training.

Lever pressing to avoid a footshock often requires shaping, that is, manually bringing the rat to the lever to facilitate operant responding (11, 24, 44). However, the use of shaping procedures is incompatible with examining the dopamine response during self-paced active avoidance learning. To overcome this limitation, rats were first placed on mild food deprivation to 90% free feeding weight and were trained to lever press for food rewards for up to 3 training sessions. In these sessions, the house light was constantly illuminated, a single lever was extended (without the use of a corresponding cue light), and up to 100 food pellets (45 mg) could be earned in a session under a fixed ratio 1 reinforcement schedule. Following lever press training, rats were given ad libitum access to food for the rest of the experiment. Numerous contextual elements differentiated the inescapable footshock/active avoidance training environment from lever press training environment, as the lever on the opposite side of the box was presented, the cue light over the lever was illuminated, the houselight was turned off, and the food magazine was blocked. Together, this ensured lever presses during inescapable footshock/active avoidance training were goal-directed and not due to perseverative responding on the food-associated lever, which is supported by the improvement in behavioral performance across sessions in High Pressing and Learner rats (Figs. 1 and 2). Additionally, the prior lever press training sessions had no direct influence on the dopamine signals during the first session of active avoidance training (SI Appendix, Fig. S5).

Inescapable footshock/active avoidance training sessions consisted of 20 trials that commenced with the presentation of the cue (white noise, lever extension, illumination of the cue light over the lever; 30 s), followed by the footshocks (0.3 mA for 0.5 s every 3 s for up to 60 s), after which the footshocks ceased, the white noise and cue light turned off, and the lever retracted for 90 s. Lever presses had no consequences during inescapable footshock training. Rats were post hoc classified as High Pressing rats if they completed at least 2 consecutive sessions with >1 press per trial, with all other rats classified as Low Pressing rats. During active avoidance training sessions, a lever press during the cue will avoid the footshock bout, whereas a lever press during the footshocks will escape the bout. Rats were post hoc segregated into Learners and Non-Learners based upon if they successfully escaped/avoided more than 50% of trials on at least 2 consecutive training sessions over 10 total training sessions. The latency to respond for each trial was capped at 90 s, which was the combined time of the cue and footshock periods. Cue-elicited freezing was assessed by blinded video scoring.

A subset of inescapable footshock rats underwent a single Catch session in which the duration of the footshock bout was shortened to 15 s. The Catch session was performed after 10 (High Pressing) or 20 (Low Pressing) inescapable training sessions. The extended training in Low Pressing rats was implemented to further diminish safety dopamine levels (Fig. 5A) so that the safety dopamine response was similar between High and Low Pressing rats.

Voltammetry Recordings.

Chronically implanted carbon-fiber microelectrodes were connected to a head-mounted voltammetric amplifier for dopamine detection in behaving rats using fast-scan cyclic voltammetry as described previously (35, 45). The potential applied to the carbon fiber was ramped in a triangular waveform from −0.4 V (vs. Ag/AgCl) to +1.3 V and back at a rate of 400 V/s during a voltammetric scan and held at −0.4 V between scans at a frequency of 10 Hz. Chemical verification of dopamine was achieved by obtaining high correlation of the cyclic voltammogram during a reward-related event to that of a dopamine standard (correlation coefficient r² ≥ 0.75 by linear regression). The voltammetry data and corresponding behavioral data for a session were not analyzed if the detected voltammetry signal did not satisfy the chemical verification criteria, identical to the exclusion criteria used in prior studies (26, 27, 35).

Data Analysis.

Dopamine-associated faradaic current was isolated from the voltammetry signal using chemometric analysis (46), using a standard training set accounting for dopamine and pH (26, 27, 35). Trials were excluded from analysis if the chemometric analysis failed to identify dopamine on >25% of the data points within the first 30 s following the onset of the cue, footshock, or safety period. The dopamine concentration was estimated based on the average postimplantation sensitivity of electrodes (34 nA/μM) (45). Electrical artifacts arising from the footshock were removed before data analysis. The voltammetry background reference was set to 0.5 s before the event of interest for the analysis of dopamine signals during the aversive cue and aversive stimulus. To analyze dopamine levels during the safety period in a systematic manner and to avoid the contributing influence of footshock artifacts, the voltammetry background reference was set to 0.5 s into the safety period. The timecourse of the dopamine response was calculated in 5-s bins for the cue and safety periods, and in 3-s intershock bins for the footshock period. The average dopamine response was quantified as the average response during the 30-s cue, during the 60-s footshock bout, and during the first 30 s of the safety period. Voltammetry data for a given session were sorted into low and high prior shock experienced based upon a median split of the maximal amount of footshock trials that could have been experienced by the last training session (90 prior shock trials). The effect of cumulative footshock exposure on the dopamine response during the aversive cue was based upon the total number of full and partial footshock trials, whereas the analysis for the dopamine response during the aversive stimulus and the safety period were based upon the total number of full footshock trials that had been experienced. The premotor dopamine response was calculated as the average dopamine levels during the 0.5 s preceding the lever press.

To identify the dopamine response profiles for the behavioral phenotypes (Low Pressing, High Pressing, Learner, or Non-Learner), dopamine levels during the aversive cue were plotted as a function of dopamine levels during the safety period for each session. The dopamine response profiles were identified by isolating the maximal quadrant containing data exclusively from a single behavioral phenotype. Linear discriminant analysis-based classification was performed in R using the lda and predict functions of the MASS package (47). Model training included cue and safety period dopamine levels from each session and the corresponding animal’s behavioral phenotype (High/Low Pressing; Learner/Non-Learner). The model trained with the data from the inescapable footshock task was then used to classify animals from the active avoidance task and vice versa. Significant effects were determined by ANOVAs, t tests, and linear regression analyses, with adjustments for differences in sphericity where appropriate. Data were analyzed using MATLAB and Prism.

Histology.

Electrical lesions were made in anesthetized rats to detect the electrode placement. Then rats were intracardially perfused with 4% paraformaldehyde and brains were removed and postfixed in the paraformaldehyde solution for at least 24 h. Brains were subsequently placed in 15% and 30% sucrose solutions in PBS until the brains had completely sunk into the sucrose solution. Next, brains were flash frozen in dry ice, coronally sectioned, and stained with cresyl violet (SI Appendix, Fig. S1).