Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2017 Jun 1.
Published in final edited form as: Biol Psychiatry. 2015 May 7;79(11):878–886. doi: 10.1016/j.biopsych.2015.04.026

Reward anticipation is encoded differently by adolescent ventral tegmental area neurons

Yunbok Kim 1, Nicholas W Simon 1, Jesse Wood 1, Bita Moghaddam 1
PMCID: PMC4636980  NIHMSID: NIHMS698791  PMID: 26067679

Abstract

Background

Elucidating the neurobiology of the adolescent brain is fundamental to our understanding of the etiology of psychiatric disorders such as schizophrenia and addiction, the symptoms of which often manifest during this developmental period. Dopamine neurons in the ventral tegmental area (VTA) are strongly implicated in adolescent behavioral and psychiatric vulnerabilities, but little is known about how adolescent VTA neurons encode information during motivated behavior.

Methods

We recorded daily from VTA neurons in adolescent and adult rats during learning and maintenance of a cued, reward-motivated instrumental task, and extinction from this task.

Results

During performance of the same motivated behavior, identical events were encoded differently by adult and adolescent VTA neurons. Adolescent VTA neurons with dopamine-like characteristics lacked a reward anticipation signal and showed a smaller response to reward delivery compared to adults. After extinction, however, these neurons maintained a strong phasic response to cues formerly predictive of reward opportunity.

Conclusions

Anticipatory neuronal activity in the VTA supports preparatory attention and is implicated in error prediction signaling. Absence of this activity, combined with persistent representations of previously rewarded experiences, may provide a mechanism for rash decision making in adolescents.

Keywords: Adolescent, Reward, Dopamine, Ventral tegmental area, Extinction, Instrumental learning

Introduction

Adolescence is associated with risky behavior and the symptomatic onset of major psychiatric disorders [1–3]. Immaturities in processing rewarding experiences have been suspected in adolescents and are thought to contribute to poor decision making and increased susceptibility to develop addictive and psychiatric disorders [1, 4, 5]. The dopamine system has been implicated strongly in adolescent behavioral and illness vulnerabilities [6–8]. In the adult brain, dopamine neurons in the ventral tegmental area (VTA) are critically involved in reward processing [9], and have been implicated in the pathophysiology of addiction, mood disorders and schizophrenia [10, 11]. In the adolescent brain, there is a general assumption of exaggerated VTA or dopamine responses to reward [12–14]. But while a few studies suggest differences in dopamine related measures between adolescents and adults [6, 15, 16], little is known about how adolescent VTA neurons process reward-related events.

We recorded daily from populations of VTA neurons in adolescent (postnatal days 38–45) and adult rats during learning, brief maintenance and extinction of a cued reward-driven instrumental task. Animals first learned that a rewarding outcome would be available if they performed an instrumental action (nose poke) after cue presentation, and then, during extinction, learned that the reward was no longer available after cue presentation. This task was chosen because it (i) is sensitive to dopamine neurotransmission, (ii) is feasible to complete during the short adolescent period, (iii) provides measures of reward mediated learning and extinction, which are critical processes for adapting to changes in environmental association [17], and (iv) is learned and performed similarly by adolescents and adults, allowing the confound of potential behavioral differences contributing to neuronal activity differences to be ruled out. Data were analyzed considering the response of all VTA neurons, as well as comparison between neurons with non-dopamine or dopamine-like characteristics. These analyses revealed critical age-related differences in VTA neuronal activity, which may provide novel insight for understanding dopamine-related behavioral vulnerabilities in adolescents.

Methods and Materials

Subjects and electrode implantation

Experimental procedures were in accordance with the University of Pittsburgh Institutional Animal Care and Use Committee. Male Sprague-Dawley (Harlan, Frederick, MD) adolescent (n = 7) and adult (n = 11) rats were used in this experiment. Surgical methods are as described previously [18]. Adolescents (P21) and adults (P70+) were received at least one week before surgery (approximately P28 in the adolescent group) to habituate to the vivarium (12 hour light/dark cycle with lights on at 7 p.m.). We implanted laboratory-made 8 channel electrode arrays (50μm diameter tungsten wire insulated with polyimide, California Fine Wire Company, Grover Beach, CA) into the VTA of adults (AP, −5.3; ML, 0.8; DV, 8.1) and adolescents (AP, −4.9; ML, 0.7; DV, 7.5). Animals recovered 7–8 days from surgery. Daily recordings were performed in adolescents beginning on P38 and ending P45.

Behavior

Animals were first habituated for two days to an operant chamber (Coulbourn Instruments, Allentown, PA) equipped with a food trough and reward magazine opposite a nose-poke port with a cue light and infrared photo-detector unit, and a tone-generating speaker (Fig. 1A). Similar to previous work [18], rats then learned to nose poke into the lit port for reward delivery in six sessions. Each trial began with the cue onset (the cue light and 500 Hz tone 0.2 sec in duration). Following the nose poke (action), the cue light was extinguished and a reward was delivered 2 sec later. A 15 sec intertrial interval (ITI) initiated upon reward collection (45 mg sugar pellet, Bio-Serv, Frenchtown, NJ), after which began the next trial. For each trial, the cue light remained illuminated until the rat responded. Each session lasted 30 min. In session 7, actions were reinforced for 30 trials (session 7R). Following this, reward was no longer available during a 30 minute block of trials (within-session extinction, session 7E). In session 8 (30 min), the cue light still was extinguished after an action, but no reward was delivered and the trial proceeded to the ITI (full session of extinction).

Figure 1.

Figure 1

Open in a new tab

Adult and adolescent rats learn instrumental associations and extinction at similar rates. (A) Task Schematic: When rats nose poked into the lit port, a reward was delivered following a 2-second delay. After animals retrieved the sugar reward, a 15 second ITI began. Sessions 1–6 proceeded in this fashion for 30 minutes each (Learning and Maintenance). Session 7 began with 30 rewarded trials, and then proceeded to 30 minutes of extinction in which responses were not rewarded (Extinction). Session 8 was 30 minutes of extinction trials only. (B) There were no differences between adolescents (n = 7) and adults (n = 11) in total trials completed per session. During extinction in sessions 7 and 8, there were no group differences in trials completed. Total trials completed in each session are depicted as the session mean ± SE, across all animals in an age group. Data are depicted for Sessions 1–6 (prior to extinction) and Sessions 7E and 8. Underlined sessions (3–6) and extinction sessions were used for electrophysiological analysis. The shaded portion of the figure depicts extinction sessions. Data from session 7R are not plotted, as all animals were required to complete exactly 30 trials.

Electrophysiology

Rats were connected to a light-weight headstage (Plexon, Dallas, TX) and a motorized commutator (Plexon, Dallas, TX) that allowed free movement during experiments. Data were high pass filtered at 100Hz and digitized at 40 kHz (OmniPlex system, Plexon, Dallas, TX). Data were sorted with standard techniques (Offline Sorter, Plexon, Dallas, TX); minimum acceptable signal to noise ratio approximately 3:1. Neurons were not pre-screened for physiological characteristics or response properties prior to recording. Behavioral events were synchronized with neuronal activity. NeuroExplorer (NEX Technologies, Madison, AL) was used for preliminary analysis.

Data analysis

The number of trials completed served as a behavioral index of learning and performance. A 2-way repeated measures ANOVA, with session as a repeated measure, was used to compare behavioral performance between age groups and sessions. Independent samples t-tests were used to quantify age-related behavioral differences in single sessions. Greenhouse-Geisser corrections were applied in all cases in which unequal variances between groups was detected. Isolated single unit data were analyzed with custom written Matlab functions (MathWorks, Nattick, MA). We classified neurons recorded in consecutive recording sessions as different units, despite the possibility that units were recorded serially. This approach is the most conservative assessment of unit identity. Though neuronal classification is necessarily arbitrary, for comparison to previous work, neurons were classified as dopamine-like or non-dopamine-like based on baseline firing rate (dopamine-like neurons < 10 Hz) and width of waveform (dopamine-like neurons > 1.2 ms). Note that, because the adolescence period in rodents is short (~10 days), use of optogenetics to identify dopamine or GABA neurons is not feasible because there is not enough time after weaning to allow for sufficient virus expression after surgery.

Basal activity levels were measured the final 10 sec of the ITI. Baseline firing rates in both age groups were non-normally distributed and compared with a Wilcoxon rank sum test. Unit firing rates during task events were binned (50 ms), smoothed with a 5 point moving rectangular kernel, and Z-score normalized against a stable baseline period (−3.5 – −0.5 sec from cue onset). A 0.25 sec window, beginning at cue onset, was utilized for cue-evoked response. The neuronal responses around instrumental action execution were analyzed in both small (0.5 sec) and large (0.75 sec) windows that were centered on the time of the action. This activity may be evoked directly by the action, or by stimuli in the environment. The terminology “action-evoked” neuronal responses refers to these factors collectively, without assuming that the action is solely responsible for evoking this response. Pre-reward delivery activity was assessed in a 0.5 sec window ending 0.05 sec prior to delivery. Finally, reward delivery-evoked activity was assessed in a 0.75 sec window, relative to delivery. Differences in peri-event activity between age groups and across sessions were assessed with 2-way ANOVA. Because of the low number of trials completed in the first two sessions, these sessions were excluded from neuronal activity analyses. Neuronal responses to extinction in session 7 were measured in blocks of trials (repeated measure). In session 8, 2-way repeated measures ANOVA was used to detect post-extinction cue-evoked responses in each group.

Histology

Rats were anesthetized with chloral hydrate and intracardially perfused with 0.9% saline followed by 10% buffered formalin. Brains were fixed, sectioned (coronal slices, 60–100um) and stained with cresyl violet. Electrode placements were confirmed with a light microscope according to the atlas of Paxinos and Watson [19].

Results

Behavior

Rats were trained to perform actions after cue onset for reward according to a fixed ratio 1 reinforcement schedule (Figure 1A). In sessions 1–6, trials completed increased with training (Figure 1B, session: F(5,80) = 42.608, p < .001). There was no difference in the progression of task learning and performance between age groups (Figure 1B, age: F(1,16) = 1.898, p = .187; interaction: F(5,80) = 1.245, p = .304), indicating that both groups learned at a comparable rate and reached similar levels of peak responding. In session 7, the first 30 trials were rewarded (session 7R) reinforced, and followed by 30 minutes of extinction trials. Adolescent and adult rats executed similar numbers of trials after extinction (Figure 1B; session 7E; t(16) = −0.153, p = .880). Session 8 was a full extinction session, during which both groups performed an equivalent number of trials (Figure 1B; session 8; t(16) = −0.244, p = .811). The number of trials performed in the final pre-extinction session (6) versus the final extinction session (8), decreased in both age groups (Figure 1B; session: F(1,16) = 74.033, p < .001), without a significant difference between groups (Figure 1B; age: F(1,16) = 0.046, p = .834; interaction: F(1,16) = 0.402, p = .535). These data indicate that the behavior of both age groups was equally sensitive to extinction of the instrumental association. The lack of overt behavioral differences suggests that gross behavioral differences do not account for divergent neuronal activity.

Age differences in event-evoked activity in VTA population response

We recorded VTA neurons in adolescent (n = 7) and adult (n = 11) male rats across 8 consecutive recording sessions (Supplementary Figure S1). There were no consistent age-related differences in baseline population firing rates (Table 1). For event-evoked data analysis, we began by analyzing the activity of the entire population of VTA neurons grouped together (both non-dopamine and dopamine-like neurons). Adolescent VTA neurons had smaller reward delivery-evoked responses than adults (F(1,584) = 10.962, p = .001; Figure 2B). In both age groups, the magnitude of this response was consistent throughout training (session: F(3,584) = .620, p = .603; interaction: F(3, 584) = .166, p = .920). Adolescent VTA neurons showed a greater action-evoked response than adults during sessions 3–6 (F(1,584) = 7.761, p = .006; Figure 2). Cue-evoked neuronal activity increased across training sessions 3–6 (F(3,584) = 5.097, p =.002; Figure 2), but was equivalent in magnitude between age groups (F(1,584) = .277, p = .599).

Table 1.

Baseline firing rate of VTA neurons. Baseline firing rate of VTA neurons in adult and adolescent rats.

Session Adults (Hz) Adolescents (Hz) z p
1 4.726 3.163 −2.730 0.005
2 5.998 2.879 −4.029 0.000
3 5.198 3.667 −3.008 0.002
4 4.951 3.245 −2.851 0.004
5 5.502 3.581 −2.985 0.002
6 4.943 4.723 −0.809 0.418
7 4.262 3.801 −0.962 0.336
8 3.750 3.941 0.030 0.975
Open in a new tab

Figure 2.

Figure 2

Open in a new tab

Age related differences in VTA population responses during learning, maintenance and extinction. Each panel depicts the normalized and smoothed neuronal responses of all VTA neurons in both age groups, averaged across trials and neurons. Adolescent VTA neurons had smaller reward delivery evoked neuronal responses than adults during sessions 3–6. Adolescents demonstrated a greater action evoked response than adults. Finally, during extinction (session 8), adolescents showed increased cue evoked activity compared to adults. Data are aligned to the onset time of the cue (left column), time of the instrumental action (middle column), and the time of reward delivery. Data are depicted as mean ± SE. The cumulative number of dopamine-like units recorded in each session is depicted in the upper left corner of the left column of each row. The topmost number indicates adolescent units. Tan overlay indicates maintenance sessions and blue overlay indicates extinction sessions.

Session 7 was divided into rewarded (session 7R) and extinction (session 7E) blocks. There was a reduction in reward-evoked activity across all subjects (F(1,159) = 27.057, p < .001), but not cue (F(1,159) = 1.931, p = .167) or action-evoked activity (F(1,159) = 3.467, p = .064). In session 8, which was an entirely extinction session, adolescent cue responses were significantly greater than adults (F(1,127) = 6.430, p = .012; Figure 2B). Thus, during extinction, adolescent VTA neurons maintained a strong phasic response to a cue previously predictive of reward availability. There were no age-related differences in action- or reward-evoked activity during extinction (action: F(1,127) = 2.121, p= .121; reward: F(1,127) = .317, p = .574), indicating that other task-related events are processed similarly by both age groups during extinction.

Age-related differences in event-evoked activity in neurons classified as non-dopamine or dopamine neurons

The VTA contains dopamine and non-dopamine (primarily GABA) neurons [20]. Importantly, dopamine and non-dopamine neurons may encode either similar or distinct information about salient events [21–24]. To address potential functional heterogeneity, we analyzed neuronal activity surrounding cue onset, instrumental actions and reward delivery separately for dopamine-like and non-dopamine-like VTA neurons.

Because average firing rate of dopamine- and non-dopamine-like neurons was not normally distributed, we analyzed these data using a non-parametric Wilcoxon rank sum test, which compares median rather than mean values. This metric revealed no consistent differences in the baseline firing rates of dopamine-like neurons between age groups (Figure 3). An example of the distribution of adolescent and adult baseline firing rates is depicted for both dopamine-like and non-dopamine-like neurons in session 6 (Figure 3B, Table 1).

Figure 3.

Figure 3

Open in a new tab

Adolescent and adult VTA neurons have similar activity levels. (A) The baseline firing rates of dopamine-like and non-dopamine-like neurons in adolescent and adult rats are displayed for sessions 4–6. Baseline firing rates were non-normally distributed, and accordingly, firing rates are depicted as median ± 95% CI. (B) The full distributions of baseline firing rates of dopamine-like neurons and non-dopamine-like neurons in session 6 are plotted on a log axis. VTA units were classified as dopamine-like neurons if they had low baseline firing rates (< 10 Hz) and wide spike waveforms (> 1.2 ms). Each dot represents one unit. One hundred example spike waveforms from one unit in each group are shown in the lower right (scale bar x axis = 0.5 ms, y axis = 0.1mv).

During learning and maintenance (sessions 3–6), adolescent dopamine-like neurons were phasically activated by cue onset. These responses increased magnitude as learning progressed (Figure 4; session: F(3,419) = 5.59 p = .001), which is generally consistent with previous work in adult animals [10]. This neuronal response was statistically indistinguishable between age groups (age: F(1,419) = 0.550, p = .459; interaction: F(3,419) = 0.18, p = .910). Action-evoked responses did not differ by age group, nor across sessions (Figure 4; group: F(1,419) = 1.181, p = .179; session: F(3,419) = 1.47, p = .222).

Figure 4.

Figure 4

Open in a new tab

Dopamine-like responses evoked by cue develop similarly with learning, while adults have larger reward delivery evoked responses than adolescents. Cue-evoked population responses increased with learning, and the progression of this response did not differ by age group. However, adolescents demonstrated a reduced reward-evoked response compared to adults. In addition, adults, but not adolescents, showed a sustained elevation in activity beginning 1 second prior to reward delivery. Peri-reward neuronal activity grew stronger in magnitude across sessions. Each panel depicts the normalized and smoothed neuronal responses of dopamine-like neurons in both age groups, averaged across trials and neurons. Data are aligned to the onset time of the cue (left column), time of the instrumental action (middle column), and the time of reward delivery. Data are depicted for the first 6 sessions (one session per row) prior to extinction as mean ± SE. The cumulative number of dopamine-like units recorded in each session is depicted in the upper left corner of the left column of each row. The topmost number indicates adolescent units. Tan overlay indicates maintenance sessions.

After instrumental actions, but before reward delivery (reward expectation period), adult dopamine-like neurons showed a sustained increase in firing rate that was absent in adolescents (Figure 4; age: F(1,419) = 20.370, p < .001). This pattern was consistent across sessions 3–6 (session: (F(3,419) = .786, p = .190). Adolescent dopamine neurons also had smaller reward delivery-evoked responses than adults (Figure 4; group: F(1,419) = 10.627, p = .001) across sessions (session: F(3,419) = 1.476, p =.223; interaction: F(3,419) = 0.018, p = .997). In summary, cue and action-evoked dopamine-like neuronal activity did not differ by age group throughout learning (Figure 5A). In contrast, adult dopamine-like neuronal activity was greater than adolescent activity during reward anticipation and delivery (Figure 5B and C). When activity was compared across individual sessions, group differences were observed in reward anticipation on days 4, 5, and 6 (Figure 5B; day 3: F(1,95) = 2.580, p =.112; day 4: F(1,106) = 3.92, p = .05; day 5: F(1,120) = 10.252, p <.01; day 6: F(1,98) = 5.439, p <.05). Reward delivery activity was only different between groups on day 5 (Figure 5C; day 3: F(1,95) = 1.605, p = .439; day 4: F(1,106) = 2.912, p = .09; day 5: F(1,120) = 7.344, p < .01; day 6: F(1,98) = 2.144, p = .146).

Figure 5.

Figure 5

Open in a new tab

Summary of learning and maintenance data in dopamine-like neurons. (A) Cue evoked dopamine-like neuronal responses develop similarly in adult and adolescent animals, (B) pre-reward delivery delay period dopamine-like neuronal responses are larger in adults, and (C) reward evoked dopamine-like neuronal responses are larger in adults. All data are displayed as the mean ± SE normalized and smoothed firing rates from both age groups in all learning and maintenance sessions. Cue-evoked response time window was immediately following the cue (0 – 0.25 sec, A), pre-reward delivery window was prior to the reward (−0.55 – −0.05 sec, B), and reward evoked response window was immediately following reward delivery (0 – 0.75 sec, C). All statistical tests were performed on sessions 3–6 (underlined).

For sessions 7 and 8, we focused on cue and reward related neuronal activity, as these task periods were associated with strong modulations of activity during task maintenance. When dopamine-like neuronal activity was examined across blocks in session 7, cue-evoked neuronal responses were not statistically different between adolescents and adults (Figure 6A; age: F(1,121) = 2.876, p = .092; block: F(1,121) = 2.973, p = .087; interaction: F(1,121) = 0.001, p = .092). However, in session 8 (Figure 6A), adolescent dopamine-like cue-evoked responses were greater than those of adults (F(1,101) = 7.932, p = .006). Examination of the magnitude of cue-evoked activation revealed a significant interaction between pre-cue and post-cue epochs and age (F(1,101) = 6.227, p = .014; Figure 6B), indicating increased cue-evoked phasic response relative to baseline in adolescents compared to adults. In addition, adult cue-responsive neurons were more likely to reduce neural activity across trials during extinction compared to adolescents (Figure 7).

Figure 6.

Figure 6

Open in a new tab

Adolescent dopamine-like neurons are associated with weaker adaptation to extinction. (A) Adult dopamine-like cue-evoked responses decrease magnitude more completely than those of adolescent neurons. Each panel depicts the normalized neuronal responses of dopamine-like neurons in both age groups. Data are displayed for sessions 7 and 8. Session 7 was divided into a block of reinforced actions (session 7R, 30 trials, top row) and a block of trials in which actions were no longer reinforced (session 7E, 30 minute block, middle row). Session 8 contained only trials in which actions were never reinforced (30 minute session, bottom row). Data are depicted similarly to Figures 2 and 4. The blue overlay depicts data from extinction trials. In extinction trials, data in the right column are aligned to the time that rewards were formerly delivered, but now withheld. (B) Adult evoked neuronal responses are of smaller magnitude than those of adolescents during extinction. Normalized neural activity of dopamine-like neurons is displayed for a window of time just prior to cue onset (−1.5 – −1.2 sec) and just after cue onset (0 – .3 sec). Data are displayed as the mean + SE across all trials and neurons.

Figure 7.

Figure 7

Open in a new tab

Individual dopamine-like neuronal responses during extinction. (A) Peri-event histogram showing cue-evoked response of a dopamine-like neuron in an adult rat in session 7. Phasic response to the cue (0 sec) was attenuated after extinction (lower panel). (B) Cue-evoked response of a dopamine-like neuron in an adolescent rat. Phasic response to the cue persisted even after extinction (lower panel). (C) Histogram showing correlations of response magnitude with post-extinction trial number for all dopamine-like neurons that activated to the cue in session 8. Filled and white bars indicate significant and non-significant correlation coefficients, respectively. Circles and error bars indicate mean ± SE correlation coefficients.

The reward expectation period in the first block of session 7 (7R, rewarded trials) was associated with increased activity in adult dopamine-like neurons (Figure 6A, column 3). During extinction, adult dopamine-like neuronal responses decreased during the reward expectation period (Figure 6A; session 7E, interaction: F(1,121) = 10.185, p = .002). Since both extinction and reinforced trials proceed similarly until reward delivery, the rapid attenuation of pre-reward response is likely related to reward expectancy.

Non-dopamine-like neurons were not significantly modulated by cue onset (Figure 8; session: F(3,157) = 1.476, p = .223; age: F(1,157) = 0.140, p = .708; interaction: F(3,157) = 0.018, p = .997). Adolescent non-dopamine-like neurons had greater action-evoked neuronal responses than adults (Figure 8; age: F(1,157) = 8.367, p = .004). We further examined action-evoked responses in a larger time window (0.75 sec) and found similar results (age: F(1,157) = 7.372, p = .007). The magnitude of this response was consistent across sessions (session: F(3,157) = 1.274, p =.286). Normalized reward delivery-evoked responses were not significantly different between age groups at any point during learning or maintenance (Figure 8; age: F(1,157) = 1.746, p = .180). During extinction, there was no change in action-induced neuronal activity in these neurons in either age group in sessions 7 or 8 (Figure 8; block: F(1,36) = 1.276, p = .266; session 8 vs session 6: F(1,67) = .372, p = .515), and age-related differences were still evident on session 8 (Figure 8; age: F(1,67) = 4.972, p = .029). The lack of change in neuronal activity was not affected by window size, as similar results were obtained with larger windows in session 7R versus 7E, and session 7E versus session 8. Thus, the adolescent non-dopamine-like neuron response in peri-action windows was sustained during extinction where association of instrumental action and reward was not present.

Figure 8.

Figure 8

Open in a new tab

Adolescent, but not adult, non-dopamine-like neurons develop action-evoked neuronal responses. Action evoked neuronal responses grew increasingly strong across learning and maintenance sessions. Data depict the task evoked, normalized and smoothed neuronal activity of non-dopamine-like neurons in both age groups. Tan overlay indicates maintenance sessions. Blue overlay indicates extinction sessions. Data are depicted identically to Figure 2 and similarly to Figures 4 and 6.

Discussion

In adolescents, reckless decision-making and increased sensation seeking, as well as their enhanced vulnerability to substance abuse and psychiatric disorders, have been attributed to immaturities in dopamine systems [7, 13, 25, 26] and exaggerated reward responsiveness of VTA dopamine neurons [12, 14]. We found that adolescent dopamine neurons lacked sustained reward anticipation responses, and had reduced, as opposed to exaggerated, responses to the reward. These neurons, however, maintained representations of previous reward opportunities by responding phasically to cues that had once predicted reward availability.

In addition to classic phasic responses to reward delivery and predictive cues [10], dopamine neurons also respond to anticipation of an impending salient event [27–29]. These latter responses are linked to perceptual and cognitive aspects of dopamine-mediated functions including preparatory attention and reward prediction error signaling [9, 27–30]. Here, we observed failure of adolescent VTA dopamine neurons to activate during reward anticipation, which, combined with enhanced representation of previous reward opportunities, may bias adolescent action selection and decision making in response to environmental cues that represent a potential reward opportunity.

Reduced reward response by VTA dopamine neurons in adolescents

Adult VTA neuronal activity, including that of dopamine-like neurons, was consistent with previous work showing that phasic responses evoked by reward predictive cues increase as learning progresses [10]. Neuronal responses to rewards persisted during learning and maintenance. Of note, while in some paradigms, fully predicted outcomes no longer activate dopamine neurons after training (a phenomenon that is an integral component of the temporal difference error signal theory), dopamine neurons can maintain reward-evoked responses well into learning [21, 27, 31–33].

Adolescent neurons, similar to adults, were increasingly activated by the reward-predicting cue as learning progressed. Reward-evoked responses, however, differed as a function of age in two ways. First, adolescent dopamine neurons lacked pre-reward responses. In adults, these neurons increased their firing rate approximately 1 second before reward delivery. This response was linked to reward expectancy, as it was not evident early in training (before learning the action-outcome association), and was attenuated when the reward was removed during extinction. Similar responses by VTA neurons have been observed during the expectation period before an impending stimulus [29], and may contribute to attentional tuning during expectation periods [34] by increasing dopamine output in terminal regions such as nucleus accumbens and prefrontal cortex. Our observation that this anticipatory response was absent in adolescents may relate to the inability to “wait” before responding (impulsive action), which is increased in adolescents [35–37]. Furthermore, reduced dopamine activation may attenuate maintenance of reward representations across temporal delays, leading to a strong preference for immediate over delayed rewards (impulsive choice) in adolescents [38, 39]. These interpretations are consistent with a plethora of basic and clinical data indicating that dopaminergic stimulants, such as Ritalin, may enhance attentional capacity and reduce impulsivity in adolescents by increasing dopamine signaling [40].

Second, reward-evoked dopamine responses were smaller in adolescents than in adults. Dopamine’s role in incentive motivation and reward processing has emerged recently as a critical component of theories related to adolescent behavioral vulnerabilities [2, 6, 12, 14, 41, 42]. These theories have advanced the hypothesis that adolescent dopamine neurons are hyperactive at baseline and in response to rewards compared to adults, and that this hyperactivity contributes to adolescent behavioral vulnerabilities [14, 15]. Contrasting these theoretical predictions, our data indicate that adolescent dopamine neurons do not fire faster overall, and are less activated before and during reward delivery than adult dopamine neurons. Cue-evoked neural responding only differs between age groups after extinction.

Significant age-related differences in cue processing were observed during extinction. Adult VTA dopamine neurons, consistent with previous work [17], had diminished phasic responses to the cue predicting reward availability. But in adolescents, the phasic response of dopamine neurons to the cue remained robust after extinction. This difference in neural processing occurred even though adolescents displayed behavioral extinction to the same degree as adults, demonstrating that they had learned the absence of reward availability. This finding indicates that dopamine-like neurons in adolescent VTA maintain representations of a previous reward opportunity for longer than adults after behavioral extinction, and suggests that behavioral extinction and adaptation of dopamine neuronal responses in adults and adolescents may occur via distinct processes.

In adolescents, enduring representations of previously attained rewards may facilitate renewal and reinstatement of earlier associations that lead to negative outcomes, such as cue-evoked drug seeking. Accordingly, adolescents demonstrate greater reinstatement of conditioned place preference for drugs of abuse [43, 44]. VTA dopamine neurons, through their projections to ventral striatum and prefrontal cortex, play a critical role in value learning, attention and working memory [45, 46]. Previous studies suggest that cue-evoked dopamine release encodes incentive value, allowing the cue itself to take on motivational properties originally associated with the reward [47]. Cues that had previously predicted reward availability may evoke an ongoing representation of previous action-reward associations in adolescent dopamine neurons, allowing this information to inform behavioral selection and executive function. In agreement with these data, adolescent rats are more motivated to work for cues formerly associated with reward (conditioned reinforcement) than adults [41]. Long-lasting representations of previously rewarded associations may also increase flexibility of cue-guided behavior by adolescents in certain behavioral contexts [48].

Another interesting implication of these data is that adolescent behavior may be less functionally coupled to neuronal activity, as adolescents reduced behavioral responding in extinction even as dopamine neurons continued to activate to the cue. It is possible that cue-evoked dopamine activation in VTA, a signal classically associated with reward prediction [10], does not drive behavior as strongly in adolescent rats compared to adults. Previous work from our lab has found age-related differences in orbitofrontal and dorsal striatal in neuronal activity during instrumental behaviors [49, 50]. These regions are implicated in representing outcome value and driving behavioral selection, respectively [51–53], and activity processing in these regions could uniquely drive behavioral responding during extinction in adolescence.

VTA non-dopamine neurons encode actions in adolescents

VTA neurons that possessed shorter spike duration, and/or faster firing rates, than dopamine-like neurons displayed some differences in adolescents compared to adults. These neurons may be comprised predominantly of GABAergic neurons [20]. The adult non-dopamine neurons were not consistently responsive to any aspects of the task. In adolescents, however, these neurons were activated by the instrumental action, suggesting a novel role for non-dopamine VTA neurons in adolescents during reward-associated actions. GABA neurons in VTA have been implicated previously in reward expectation [21] and the cessation of consummatory behavior in adults [23]. In addition, non-dopamine neurons in the VTA may encode similar and redundant information as dopamine neurons [22]. The current data is consistent with GABA-related differences in the neural processing of behaviorally relevant events in adolescents and adults [54].

Conclusions

While there is little disagreement that adolescence is a vulnerable period for several psychiatric disorders, and that the dopamine system may contribute to these vulnerabilities, our theoretical and preventive approaches to these disorders have been based on knowledge obtained from adult electrophysiological models [55]. These theories, which have influenced articles and books for lay audience with titles like “Dopamine and Teenage Logic,” presumptively attribute a host of behaviors and clinical symptoms of adolescents to excessive reward-driven dopamine activity [56]. Contrary to these assumptions, we find that adolescent VTA dopamine neurons have a reduced response to reward expectation and delivery. This is significant in the context of dopamine-related behavioral vulnerabilities because attenuated dopamine signaling during expectation period may impact attentional capabilities to impending events and rewards. Our other key finding was that adolescent VTA dopamine neurons continue to signal previously reinforced associations after behavioral extinction, suggesting that they maintain a “latent” representation of cues that once predicted reward opportunity. Thus, even after extinction of an explicit action-reward association has occurred, similar cues in the environment could elicit robust dopamine neuron activation in adolescents and influence goal-directed behavior. These two properties of adolescent VTA neurons may deliver a one-two punch that leads to imprudent decision making when faced with a potential reward opportunity.

Supplementary Material

supplement

Acknowledgments

This work was supported by the National Institutes of Health grants R01MH48404 (BM) and T32DA03111 (NS, JW).

Footnotes

The authors report no biomedical financial interests or potential conflicts of interest.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  • 1.Blakemore S, Robbins TW. Decision-making in the adolescent brain. Nature Neuroscience. 2012;15:1184–1191. doi: 10.1038/nn.3177. [DOI] [PubMed] [Google Scholar]
  • 2.Spear LP. The adolescent brain and age-related behavioral manifestations. Neuroscience & Biobehavioral Reviews. 2000;24(4):417–463. doi: 10.1016/s0149-7634(00)00014-2. [DOI] [PubMed] [Google Scholar]
  • 3.Adriani W, Laviola G. Windows of vulnerability to psychopathology and therapeutic strategy in the adolescent rodent model. Behav Pharmacol. 2004;15(5–6):341–52. doi: 10.1097/00008877-200409000-00005. [DOI] [PubMed] [Google Scholar]
  • 4.Tomasi D, Volkow ND. Functional Connectivity of Substantia Nigra and Ventral Tegmental Area: Maturation During Adolescence and Effects of ADHD. Cereb Cortex. 2012 doi: 10.1093/cercor/bhs382. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Sturman DA, Moghaddam B. The neurobiology of adolescence: Changes in brain architecture, functional dynamics, and behavioral tendencies. Neuroscience & Biobehavioral Reviews. 2011;35(8):1704–1712. doi: 10.1016/j.neubiorev.2011.04.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Matthews M, Bondi C, Torres GE, Moghaddam B. Reduced presynaptic dopamine activity in adolescent dorsal striatum. Neuropsychopharmacol. 2013;38:1344–51. doi: 10.1038/npp.2013.32. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Niwa M, Jaaro-Peled H, Tankou S, Seshadri S, Hikida T, Matsumoto Y, et al. Adolescent stress–induced epigenetic control of dopaminergic neurons via glucocorticoids. Science. 2013;339(6117):335–339. doi: 10.1126/science.1226931. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Jia JM, Zhao J, Hu Z, Lindberg D, Li Z. Age-dependent regulation of synaptic connections by dopamine D2 receptors. Nat Neurosci. 2013;16(11):1627–36. doi: 10.1038/nn.3542. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Steinberg EE, Keiflin R, Boivin JR, Witten IB, Deisseroth K, Janak PH. A causal link between prediction errors, dopamine neurons and learning. Nat Neurosci. 2013;16(7):966–973. doi: 10.1038/nn.3413. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Schultz W. Predictive reward signal of dopamine neurons. Journal of Neurophysiology. 1998;80(1):1–27. doi: 10.1152/jn.1998.80.1.1. [DOI] [PubMed] [Google Scholar]
  • 11.Wise RA. Dopamine, learning and motivation. Nat Rev Neurosci. 2004;5(6):483–94. doi: 10.1038/nrn1406. [DOI] [PubMed] [Google Scholar]
  • 12.Wong WC, Ford KA, Pagels NE, McCutcheon JE, Marinelli M. Adolescents are more vulnerable to cocaine addiction: behavioral and electrophysiological evidence. J Neurosci. 2013;33(11):4913–22. doi: 10.1523/JNEUROSCI.1371-12.2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Chambers RA, Taylor JR, Potenza MN. Developmental neurocircuitry of motivation in adolescence: a critical period of addiction vulnerability. Am J Psychiatry. 2003;160(6):1041–52. doi: 10.1176/appi.ajp.160.6.1041. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Luciana M, Collins PF. Incentive motivation, cognitive control, and the adolescent brain: Is it time for a paradigm shift? Child Dev Perspect. 2012;6:392–99. doi: 10.1111/j.1750-8606.2012.00252.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.McCutcheon JE, Conrad KL, Carr SB, Ford KA, McGehee DS, Marinelli M. Dopamine neurons in the ventral tegmental area fire faster in adolescent rats than in adults. Journal of Neurophysiology. 2012;108(6):1620–1630. doi: 10.1152/jn.00077.2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Robinson DL, Zitzman DL, Smith KJ, Spear LP. Fast dopamine release events in the nucleus accumbens of early adolescent rats. Neuroscience. 2011;176(0):296–307. doi: 10.1016/j.neuroscience.2010.12.016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Pan W-X, Schmidt R, Wickens JR, Hyland BI. Tripartite mechanism of extinction suggested by dopamine neuron activity and temporal difference model. The Journal of Neuroscience. 2008;28(39):9619–9631. doi: 10.1523/JNEUROSCI.0255-08.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Sturman DA, Moghaddam B. Striatum processes reward differently in adolescents versus adults. Proc Natl Acad Sci U S A. 2012;109(5):1719–24. doi: 10.1073/pnas.1114137109. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Paxinos G, Watson C. The rat brain in stereotaxic coordinates. 3. Academic Press; 1998. [DOI] [PubMed] [Google Scholar]
  • 20.Nair-Roberts RG, Chatelain-Badie SD, Benson E, White-Cooper H, Bolam JP, Ungless MA. Stereological estimates of dopaminergic, gabaergic and glutamatergic neurons in the ventral tegmental area, substantia nigra and retrorubral field in the rat. Neuroscience. 2008;152(4):1024–1031. doi: 10.1016/j.neuroscience.2008.01.046. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Cohen JY, Haesler S, Vong L, Lowell BB, Uchida N. Neuron-type-specific signals for reward and punishment in the ventral tegmental area. Nature. 2012;482:85–8. doi: 10.1038/nature10754. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Kim YB, Matthews M, Moghaddam B. Putative gamma-aminobutyric acid neurons in the ventral tegmental area have a similar pattern of plasticity as dopamine neurons during appetitive and aversive learning. Eur J Neurosci. 2010;32(9):1564–72. doi: 10.1111/j.1460-9568.2010.07371.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.van Zessen R, Phillips JL, Budygin EA, Stuber GD. Activation of VTA GABA neurons disrupts reward consumption. Neuron. 2012;73(6):1184–94. doi: 10.1016/j.neuron.2012.02.016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Kim Y, Wood J, Moghaddam B. Coordinated activity of ventral tegmental neurons adapts to appetitive and aversive learning. PLoS One. 2012;7(1):e29766. doi: 10.1371/journal.pone.0029766. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Ernst M, Romeo RD, Andersen SL. Neurobiology of the development of motivated behaviors in adolescence: A window into a neural systems model. Pharmacology Biochemistry and Behavior. 2009;93(3):199–211. doi: 10.1016/j.pbb.2008.12.013. [DOI] [PubMed] [Google Scholar]
  • 26.O’Donnell P. Adolescent onset of cortical disinhibition in schizophrenia: insights from animal models. Schizophr Bull. 2011;37(3):484–92. doi: 10.1093/schbul/sbr028. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Fiorillo CD, Tobler PN, Schultz W. Discrete coding of reward probability and uncertainty by dopamine neurons. Science. 2003;299(5614):1898–1902. doi: 10.1126/science.1077349. [DOI] [PubMed] [Google Scholar]
  • 28.Bromberg-Martin ES, Matsumoto M, Hikosaka O. Distinct tonic and phasic anticipatory activity in lateral habenula and dopamine neurons. Neuron. 2010;67(1):144–55. doi: 10.1016/j.neuron.2010.06.016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Totah NK, Kim Y, Moghaddam B. Distinct prestimulus and poststimulus activation of VTA neurons correlates with stimulus detection. Journal of Neurophysiology. 2013;110(1):75–85. doi: 10.1152/jn.00784.2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Glimcher PW. Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis. PNAS. 2011;108(Suppl 3):15647–54. doi: 10.1073/pnas.1014269108. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Pan W-X, Schmidt R, Wickens JR, Hyland BI. Dopamine cells respond to predicted events during classical conditioning: Evidence for eligibility traces in the reward-learning network. The Journal of Neuroscience. 2005;25(26):6235–6242. doi: 10.1523/JNEUROSCI.1478-05.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Morris G, Nevet A, Arkadir D, Vaadia E, Bergman H. Midbrain dopamine neurons encode decisions for future action. Nat Neurosci. 2006;9(8):1057–63. doi: 10.1038/nn1743. [DOI] [PubMed] [Google Scholar]
  • 33.Joshua M, Adler A, Prut Y, Vaadia E, Wickens JR, Bergman H. Synchronization of midbrain dopaminergic neurons is enhanced by rewarding events. Neuron. 2009;62(5):695–704. doi: 10.1016/j.neuron.2009.04.026. [DOI] [PubMed] [Google Scholar]
  • 34.Rohenkohl G, Cravo AM, Wyart V, Nobre AC. Temporal expectation improves the quality of sensory information. The Journal of neuroscience. 2012;32(24):8424–8. doi: 10.1523/JNEUROSCI.0804-12.2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Burton CL, Fletcher PJ. Age and sex differences in impulsive action in rats: The role of dopamine and glutamate. Behavioural Brain Research. 2012;230(1):21–33. doi: 10.1016/j.bbr.2012.01.046. [DOI] [PubMed] [Google Scholar]
  • 36.Romer D. Adolescent risk taking, impulsivity, and brain development: Implications for prevention. Developmental Psychobiology. 2012;52(3):263–276. doi: 10.1002/dev.20442. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Andrzejewski ME, Schochet TL, Feit EC, Harris R, Mc Kee BL, Kelley AE. A comparison of adult and adolescent rat behavior in operant learning, extinction, and behavioral inhibition paradigms. Behav Neurosci. 2011;125:93–105. doi: 10.1037/a0022038. [DOI] [PubMed] [Google Scholar]
  • 38.Doremus-Fitzwater TL, Barretto M, Spear LP. Age-related differences in impulsivity among adolescent and adult Sprague-Dawley rats. Behav Neurosci. 2012;126:735–41. doi: 10.1037/a0029697. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Green L, Fry AF, Myerson J. Discounting of delayed rewards: a life-span comparison. Psychological Science. 1994;5:33–36. [Google Scholar]
  • 40.DeVito EE, Blackwell AD, Kent L, Ersche KD, Clark L, Salmond CH, et al. The Effects of Methylphenidate on Decision Making in Attention-Deficit/Hyperactivity Disorder. Biological Psychiatry. 2008;64(7):636–639. doi: 10.1016/j.biopsych.2008.04.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Burton CL, Noble K, Fletcher PJ. Enhanced incentive motivation for sucrose-paired cues in adolescent rats: Possible roles for dopamine and opioid systems. Neuropsychopharmacol. 2011;36(8):1631–1643. doi: 10.1038/npp.2011.44. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Naneix F, Marchand AR, Di Scala G, Pape J-R, Coutureau E. Parallel maturation of goal-directed behavior and dopaminergic systems during adolescence. The Journal of Neuroscience. 2012;32(46):16223–16232. doi: 10.1523/JNEUROSCI.3080-12.2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Kota D, Sanjakdar S, Marks MJ, Khabour O, Alzoubi K, Damaj MI. Exploring behavioral and molecular mechanisms of nicotine reward in adolescent mice. Biochemical Pharmacology. 2011;82(8):1008–1014. doi: 10.1016/j.bcp.2011.06.019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Brenhouse HC, Andersen SL. Delayed extinction and stronger reinstatement of cocaine conditioned place preference in adoelscent rats, compared to adults. Behav Neurosci. 2008;122:460–5. doi: 10.1037/0735-7044.122.2.460. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Seamans JK, Yang CR. The principal features and mechanisms of dopamine modulation in the prefrontal cortex. Prog Neurobiol. 2004;74(1):1–58. doi: 10.1016/j.pneurobio.2004.05.006. [DOI] [PubMed] [Google Scholar]
  • 46.Goldman-Rakic PS. Regional and cellular fractionation of working memory. Proc Natl Acad Sci U S A. 1996;93(24):13473–80. doi: 10.1073/pnas.93.24.13473. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Flagel SB, Clark JJ, Robinson TE, Mayo L, Czuj A, Willuhn I, et al. A selective role for dopamine in stimulus-reward learning. Nature. 2011;469(7328):53–57. doi: 10.1038/nature09588. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Simon NW, Gregory TA, Wood J, Moghaddam B. Differences in response inhibition and behavioral flexibility between adolescent and adult rats. Behav Neurosci. 2013;127:23–32. doi: 10.1037/a0031328. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.Sturman DA, Moghaddam B. Striatum processes reward differently in adolescents versus adults. Proceedings of the National Academy of Sciences. 2012;109(5):1719–1724. doi: 10.1073/pnas.1114137109. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Sturman DA, Moghaddam B. Reduced Neuronal Inhibition and Coordination of Adolescent Prefrontal Cortex during Motivated Behavior. The Journal of Neuroscience. 2011;31(4):1471–1478. doi: 10.1523/JNEUROSCI.4210-10.2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.Rolls ET. The orbitofrontal cortex and reward. Cerebral Cortex. 2000;10:284–294. doi: 10.1093/cercor/10.3.284. [DOI] [PubMed] [Google Scholar]
  • 52.Schoenbaum G, Takahashi Y, Liu T-L, McDannald MA. Does the orbitofrontal cortex signal value? Annals of the New York Academy of Sciences. 2011;1239(1):87–99. doi: 10.1111/j.1749-6632.2011.06210.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Balleine BW, Delgado MR, Hikosaka O. The Role of the Dorsal Striatum in Reward and Decision-Making. J Neurosci. 2007;27(31):8161–8165. doi: 10.1523/JNEUROSCI.1554-07.2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54.Caballero A, Thomases DR, Flores-Barrera E, Cass DK, Tseng KY. Emergence of GABAergic-dependent regulation of input-specific plasticity in the adult rat prefrontal cortex during adolescence. Psychopharmacology (Berl) 2014;231(8):1789–96. doi: 10.1007/s00213-013-3216-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 55.Addington J, Heinssen R. Prediction and prevention of psychosis in youth at clinical high risk. Annu Rev Clin Psychol. 2012;8:269–89. doi: 10.1146/annurev-clinpsy-032511-143146. [DOI] [PubMed] [Google Scholar]
  • 56.Siegel D. Dopamine and Teenage Logic. The Atlantic 2014 [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

supplement
NIHMS698791-supplement.pdf (130.7KB, pdf)

RESOURCES