A critical time window for dopamine actions on the structural plasticity of dendritic spines

Abstract

Animal behaviors are reinforced by subsequent rewards following within a narrow time window. Such reward signals are primarily coded by dopamine, which modulates the synaptic connections of medium spiny neurons in the striatum. The mechanisms of the narrow timing detection, however, remain unknown. Here, we optically stimulated dopaminergic and glutamatergic inputs separately and found that dopamine promoted spine enlargement only during a narrow time window (0.3 to 2 seconds) after the glutamatergic inputs. The temporal contingency was detected by rapid regulation of adenosine 3′,5′-cyclic monophosphate in thin distal dendrites, in which protein-kinase A was activated only within the time window because of a high phosphodiesterase activity. Thus, we describe a molecular basis of reinforcement plasticity at the level of single dendritic spines.

Animal behaviors are reinforced only when rewarded shortly after a motor or sensory event (1, 2). The neocortex, hippocampus, and amygdala process the sensorimotor signals and send glutamatergic synaptic output to the striatum (3), where connections can be modified by Hebbian learning mechanisms, such as spike-timing-dependent plasticity (STDP) (4). Animals learn to associate the sensorimotor signals with subsequent rewards through reinforcement of the neuronal circuits involving dopamine (5-7). Despite its importance, this narrow timing detection has never been demonstrated at the cellular level and might be ascribed to neural network properties (6, 8).

Dendritic spine morphology is correlated with spine function (9), and dendritic spines enlarge during long-term potentiation in the cortices (10-12). We examined the effects of dopamine on the structural plasticity in striatal medium spiny neurons (MSNs). Results show that dopamine affected spine structural plasticity in a narrow time window consistent with behavioral conditioning (5). Functional imaging revealed the molecular interrelationships between the reinforcement and Hebbian plasticity.

We investigated dopamine actions on glutamatergic synapses on MSNs using optogenetics and two-photon uncaging. For optogenetic stimulation of dopaminergic fibers, a Cre-dependent adeno-associated virus (AAV) vector expressing channelrhodopsin-2 (ChR2) was injected into the ventral tegmental area (VTA) of DAT-Cre mice expressing Cre specific to dopaminergic neurons (Fig. 1A and fig. S1). The direct pathway–constituting MSNs, which mainly express dopamine 1 receptors (D1Rs) (13), were labeled by an AAV vector with a specific promoter for D1R-MSNs (Fig. 1A and fig. S1). In acute coronal slices, including the nucleus accumbens (NAc) core, whole-cell recordings were obtained from the identified D1R-MSNs. Dendritic spines were visualized by means of two-photon microscopy (980 nm) detecting fluorescence of Alexa488 loaded through a recording pipette (Fig. 1B and fig. S1C). Stimulation of a single spine by means of two-photon uncaging (720 nm) of CDNI-Glu (Fig. 1B) induced two-photon excitatory postsynaptic potentials (2pEPSPs) with amplitudes similar to miniature EPSPs (mEPSPs) (Fig. 1C and fig. S1D). The amplitudes of 2pEPSCs positively correlated with spine volumes, as they did with pyramidal neurons (fig. S1E) (9). The STDP protocol of repetitive uncaging of glutamate paired with action potentials (APs) (Glu + AP) (Fig. 1D) (14) induced robust spine enlargement that was selective for the stimulated spine (Fig. 1, E and F) when dopamine (100 μM) was puff applied, whereas only a weak enlargement occurred in the absence of dopamine (Fig. 1G), which is consistent with previous findings (14, 15).

Fig. 1. A temporal profile of dopamine actions on spine enlargement.

(A) Injection of AAV vectors for ChR2 and the D1R-MSN marker (PPTA-mCherry) in 3-week-old DAT-Cre mice. (B) Selective stimulation of dopaminergic and glutamatergic inputs by means of blue laser field irradiation to ChR2 and two-photon uncaging of caged-glutamate at a single spine, respectively, in acute slices of NAc obtained from 5- to 7- week-old mice. (C) An amperometric measurement of dopamine (top) by carbon-fiber electrode and whole-cell recording of glutamate-induced current (bottom, 2pEPSP) in identified D1R-MSNs. (D) An STDP protocol with dopamine puff application. (E) Images of the dendritic spine (red arrowhead) that received STDP stimulation in the presence of dopamine (100 μM). (F and G) Time courses of spine enlargement in the presence [(F), 13 spines, 4 dendrites] and absence of dopamine [(G), 58 spines, 14 dendrites]. (H) Amplitudes of spine enlargements with or without dopamine. **P = 0.0041 by Mann-Whitney U test. (I) STDP with repetitive activation of dopaminergic fibers containing ChR2 (blue lines) at 30 Hz, 10 times (DA_opto). (J) Images of the dendritic spine (arrowhead) that received STDP + DA_opto with a delay of 1 s. (K to M) Time courses of spine enlargement induced by STDP + DA_opto at 1 s [(K), 48 spines, 14 dendrites], −1 s [(L), 20 spines, 5 dendrites] and 5 s [(M), 28 spines, 7 dendrites] after STDP onset. (N) Timings of DA_opto application. (O) Increases in spine volumes by STDP + DA_opto plotted versus DA_opto delay (fig. S2, A to C). Data are presented as mean ± SEM. P = 4.2 × 10⁻⁶ with Kruskal-Wallis and **P = 0.0018 (0.6 s) and 0.0027 (1 s) by Steel test in comparison with STDP in the absence of DA_opto. Scale bars, 1 μm.

A single pulse of blue laser stimulation of ChR2-expressing dopaminergic fibers induced a transient increase in the tissue dopamine concentration (Fig. 1C). We tested the actions of a physiologically relevant phasic release of dopamine (30 Hz, 10 times) induced by the optogenetic stimulation of dopaminergic fibers (DA_opto) and found that DA_opto just after STDP stimulation (DA_opto delay = 1 s) (Fig. 1I) induced robust spine enlargement (Fig. 1, J and K). Stimulation of dopaminergic fibers 1 s before (DA_opto delay = −1 s) or 5 s after (DA_opto delay = 5 s) STDP (Fig. 1, L and M) resulted in only a slight enhancement. The actions of DA_opto were examined with various timings (Fig. 1N), revealing that DA_opto timing was critical to enhance plasticity, with maximal effects for a delay of 0.6 s (Fig. 1O and fig. S2, A to C), and decayed in a few seconds, which is consistent with behavioral study results (5, 16). The spine enlargement was accompanied by an increase in the 2pEPSC (fig. S3). A similar DA_opto timing was observed when we induced STDP through electrical stimulation of presynaptic fibers (fig. S4).

Pharmacological studies revealed that D1R-MSN structural plasticity was dependent on N-methyl-d-aspartate receptors (NMDARs), Ca²⁺/calmodulin–dependent protein kinase II (CaMKII), and protein synthesis (Fig. 2A and fig. S5A), suggesting that the molecular mechanisms for D1R-MSN plasticity are similar to those underlying structural plasticity in hippocampal pyramidal neurons (10-12). Plasticity also depended on D1R and protein kinase A (PKA), but not on D2R (Fig. 2B and fig. S5A). Spine enlargement was prevented by an inhibitory peptide blocking the interaction of dopamine- and adenosine 3′,5′-cyclic monophosphate (cAMP)–regulated phosphoprotein 32 kD (DARPP-32) with protein phosphatase 1 (PP-1) (17), but not its control peptide (Fig. 2, C and D). Moreover, spine enlargement was induced even in the absence of DA_opto when PP-1 was inhibited by calyculin A (fig. S5, B and C). These results suggest that similar to hippocampal preparations (18), the phosphorylation of DARPP-32 by PKA would inhibit PP-1 and disinhibit CaMKII.

Fig. 2. Pharmacology of spine enlargement induced by STDP plus DA_opto with a 0.6-s delay.

(A) Time courses of spine enlargement induced by STDP + DA_opto with a 0.6-s delay in the absence (control, 24 spines, 7 dendrites) and presence of NMDAR antagonist (50 μM D-AP5, 22 spines, 6 dendrites), CaMKII inhibitor (3 μM KN62, 23 spines, 6 dendrites), or protein synthesis inhibitor (5 μM anisomycin, 25 spines, 6 dendrites). (B) Time courses of spine enlargement in the presence of D1R antagonist (3 μM SCH23390, 23 spines, 6 dendrites), D2R antagonist (10 μM sulpiride, 22 spines, 6 dendrites), or PKA inhibitor (10 μM PKI, in the pipette, 24 spines, 6 dendrites). (C) Time courses of spine enlargement in the presence of inhibitory (100 μM, in the pipette, 24 spines, 6 dendrites) or control peptide for DARPP-32 (100 μM, in the pipette, 24 spines, 6 dendrites). (D) Averaged volume changes in the absence and presence of the compounds. Data are presented as mean ± SEM. P = 3.4 × 10⁻⁶ with Kruskal-Wallis and *P = 0.023 (AP5), 0.023 (KN62), 0.037 (AIP) (fig. S5A), 0.023 (anisomycin), 0.035 (SCH23390), 0.023 (PKI), 0.037 (KT5720) (fig. S5A), and 0.023 (DARPP-32 inhibitory peptide) with Steel test.

To test whether changes in Ca²⁺ signaling account for DA_opto timing (19), we imaged increases in cytosolic Ca²⁺ concentrations ([Ca²⁺]_i) in spines using a low-affinity calcium indicator Fluo-4FF (K_Ca = 10 μM) to avoid saturation. The [Ca²⁺]_i increases gradually built up and quickly waned after cessation of the STDP protocol (Fig. 3A). We found that DA_opto did not affect Ca²⁺ transients (Fig. 3, B to D, and fig. S6, A to C), indicating that Ca²⁺ signaling modulation did not play a major role.

Fig. 3. DA_opto effects on STDP stimulation-induced increases in [Ca²⁺]_i and CaMKII activities.

(A and B) Increases in Fluo4-FF fluorescence, representing [Ca²⁺]_i increases, within single spines in response to a train of STDP stimulation in the absence [(A), 20 spines, 15 dendrites] and presence of DA_opto with a 0.6-s delay [(B), 15 spines, 8 dendrites]. Blue laser irradiation during DA_opto is blanked by the blue bar. (C) STDP and DA_opto protocols for Ca²⁺ and CaMKII imaging. Unlike plasticity induction (Fig. 1N), only one train was applied. (D) No effect of DA_opto on the peak values of [Ca²⁺]_i (fig. S6, A to C). P = 0.91 with Kruskal-Wallis test. (E and F) Ratiometric imaging with Camuiα-CR during STDP stimulation in the absence [(E), 33 spines, 14 dendrites] or presence of DA_opto with a 0.6-s delay [(F), 42 spines, 14 dendrites]. Relative increases in the ratio are shown as pseudocolor coding in (E). (Bottom) Time courses of FRET ratios in the spines stimulated with glutamate uncaging or the neighboring spines. Scale bars, 1 μm. (G) Dependence of the peak Camuiα-CR FRET ratios on the DA_opto delay (fig. S6, D to F). P = 1.3 × 10−5 with Kruskal-Wallis test and ***P = 8.4 × 10−5 with Steel test versus those without DA_opto. (H) Normalized increases in Camuiα-CR ratios by STDP + DA_opto with a 0.6-s delay in the stimulated (42 spines, 14 dendrites) and neighboring spines (42 spines, 14 dendrites), and in the presence of DARPP-32 inhibitory peptide in the stimulated spines (43 spines, 10 dendrites) (fig. S6G). Data are presented as mean ± SEM. P = 8.8 × 10⁻⁹ with Kruskal-Wallis test and *** P = 2.6 × 10⁻⁹ and 1.1 × 10⁻⁶ with Steel test.

We examined whether CaMKII activation was related to DA_opto timing by using the Förster resonance energy transfer (FRET) indicator of CaMKII, Camuiα-CR (20-22), which was virally transfected into D1R-MSNs. CaMKII activation was weak in the absence of DA_opto (Fig. 3E) but was greatly potentiated by DA_opto (Fig. 3F), with timing similar to DA_opto timing for spine enlargement (Fig. 3G and fig. S6, D to F). This enhancement was specific to the stimulated spine (Fig. 3H), abolished by the inhibitory peptide for DARPP-32 (Fig. 3H and fig. S6G), and mimicked by PP-1 inhibitors, calyculin A and tautomycetin (fig. S6, H and I). These results support the hypothesis that PKA/DARPP-32 disinhibits CaMKII via PP-1 (fig. S11).

We addressed whether the DA_opto timing was formed at the level of PKA activation by using a FRET probe of PKA, AKAR2-CR (22), which was virally delivered to the D1R-MSNs. Unlike structural plasticity or Camuiα-CR activation, PKA activation in response to stimulation of a dendritic spine by STDP and DA_opto was not restricted to the stimulated spine; neighboring spines also exhibited significant PKA activation (fig. S7, A to D). Even without glutamate uncaging, PKA activation was observed in the spine and dendritic shaft (Fig. 4A) in an AP-dependent manner (Fig. 4B), suggesting that PKA activation is a cell-wide phenomenon. When DA_opto was applied at various times relative to APs, we obtained a timing (Fig. 4C and fig. S7, E to I) similar to DA_opto timing on spine enlargement and CaMKII activation (Fig. 1O). The extent of spine enlargement positively correlated with that of PKA activation (Fig. 4D). APs themselves were not sufficient to activate PKA (fig. S8, C to E). The contingency between APs and DA_opto might be detected by Ca²⁺/calmodulin–dependent adenylyl cyclase 1 (AC1), which is synergistically activated by Ca²⁺/calmodulin and Gs (23). Consistent with this, AC1 blocker (NB001) eliminated AKAR2 activation, as well as structural plasticity (fig. S9, A to D).

Fig. 4. AP effects on DA_opto-induced PKA activation in proximal and distal dendrites.

(A and B) Images and time courses of FRET ratios of AKAR2-CR in the spine and distal dendrites stimulated by APs + DA_opto with a 0.6-s delay [(A), 153 spines, 25 dendrites] or DA_opto only [(B), 158 spines, 26 dendrites]. The relative increases in the ratio were pseudocolor coded as shown in (A). Scale bar, 2 μm. (C) AKAR2-CR responses to APs + DA_opto with various delays (fig. S7, E to I). P = 5.3 × 10⁻¹⁰ with Kruskal-Wallis test and *P = 0.012, ***P = 1.3 × 10⁻⁶ (0.6 s), and 4.0 × 10⁻⁴ (1 s) with Steel test versus DA_opto only. (D) AKAR2-CR response (C) plotted against spine volume changes (Fig. 1O) for various DA_opto timings. The Spearman’s correlation coefficient was 0.94, and P = 0.0048. The blue and gray bars indicate the dynamic ranges of volume changes predicted by the dynamic ranges of AKAR2 responses at dendritic spine (blue) and soma (gray). (E) AKAR2-CR responses at the soma and first, second, and distal dendrites in response to DA_opto only and DA_opto + APs with a 0.6-s delay (fig. S8, A and B).

In the soma and proximal (first branch) dendrites, however, DA_opto alone was sufficient to activate PKA (Fig. 4E and fig S8, A and B) (24), and APs could only slightly enhance PKA. We predict that APs might have modulated spine enlargement to a small degree in these regions, if there had been spines (Fig. 4D). Thus, DA_opto-induced PKA activation must be suppressed in distal dendrites in order to attain the large dynamic range for the timing detection. In fact, when phosphodiesterase 10A (PDE10A), the major phosphodiesterase in MSNs, was blocked by its inhibitor papaverine (25), PKA activations were similarly induced in distal dendrites as in the soma (fig. S9, E to G). Papaverine also disrupted the time window for structural plasticity (fig. S9, H and I). Why were PDE10A actions particularly potent in the distal dendrites? Subcellular differences in PDE10A expression might not account for this, considering that PDE10A is expressed at the plasma membrane and is uniformly distributed along the dendrites (26). Instead, we found a negative correlation between dendrite diameter and DA_opto-induced PKA activity (fig. S10A, blue), which was lost when the phosphodiesterase was inhibited (fig. S10A, orange). Therefore, PDE10A might counteract the increases in cAMP more potently in the thin distal dendrites because of its high surface-to-volume ratios (fig. S10, B and C). Spines are only found in the distal thin dendrites of MSNs (fig. S10, D to E) (27), suggesting that spines are distributed to be efficiently modulated by dopamine timing in MSNs.

It has been enigmatic why dopamine reinforces preceding, but not subsequent, sensorimotor events. If dopamine always activates PKA, its effects should last long enough to reinforce the subsequent events over tens of seconds (Fig. 4A). However, dopamine did not activate PKA unless [Ca²⁺]_i primed AC1 to outcompete the high phosphodiesterase activity in thin dendrites (fig. S11). Our data show that [Ca²⁺]_i priming should occur strictly before dopamine delivery (0 s) (Figs. 1O, 3G, and 4C), which is reminiscent of serotonin’s action in the classical conditioning of the siphon withdrawal reflex in Aplysia (28), in which serotonin, carrying an aversive signal, was only effective when it was preceded by increases in [Ca²⁺]_i for activation of calcium-dependent adenylyl cyclase (AC) and PKA in presynaptic terminals (29). The delay in [Ca²⁺]_i priming of AC may compensate the time lag of monoaminergic signals after reward or punishment. Our data suggest that reinforcement plasticity occurs at the single spine level, even though PKA activation is a cell-wide phenomenon, in such a way that dopamine regulates the gain of NMDAR-dependent Hebbian plasticity via CaMKII activity (fig. S11). This interdependence between Hebbian and reinforcement plasticity has been implicitly assumed in the reinforcement learning theory, in which the Hebbian term is used for the credit assignment (6, 30), and the dopamine timing in our study corresponds to the eligibility trace that determines the time window for reward action (30). Thus, we have clarified a molecular and cellular basis of reinforcement plasticity at the level of single dendritic spines.