Abstract
In songbirds, as in mammals, basal ganglia-forebrain circuits are necessary for the learning and production of complex motor behaviors; however, the precise role of these circuits remains unknown. It has recently been shown that a basal ganglia-forebrain circuit in the songbird, which projects directly to vocal–motor circuitry, has a premotor function driving exploration necessary for vocal learning. It has also been hypothesized that this circuit, known as the anterior forebrain pathway (AFP), may generate an instructive signal that improves performance in the motor pathway. Here, we show that the output of the AFP directly implements a motor correction that reduces vocal errors. We use disruptive auditory feedback, contingent on song pitch, to induce learned changes in song structure over the course of hours and find that reversible inactivation of the output of the AFP produces an immediate regression of these learned changes. Thus, the AFP is involved in generating an error-reducing bias, which could increase the efficiency of vocal exploration and instruct synaptic changes in the motor pathway. We also find that learned changes in the song generated by the AFP are incorporated into the motor pathway within 1 day. Our observations support a view that basal ganglia-related circuits directly implement behavioral adaptations that minimize errors and subsequently stabilize these adaptations by training premotor cortical areas.
Keywords: consolidation, LMAN, motor learning, reinforcement learning, anterior forebrain pathway
Birdsong is a complex motor behavior that, like many human motor skills, improves with practice. Songbirds learn to sing by imitation, using auditory feedback to compare their own vocalizations with the memorized song of a tutor (1). Learning birds initially produce a highly variable juvenile song that, after thousands of repetitions, converges to a stable adult song, often a remarkably precise imitation of the tutor song (2, 3). Like many learning tasks in mammals (4–6), this goal-directed behavior requires a basal ganglia-thalamocortical circuit (Fig. 1A) known as the anterior forebrain pathway (AFP) (7–11). The mechanisms by which basal ganglia circuitry support motor learning are largely unknown, but evidence suggests that the basal ganglia are necessary to express recently learned behavior (12), and that changes in neural activity in response to learning appear first in basal ganglia circuits (13, 14).
It has recently been shown that the AFP plays a premotor role in driving vocal variability. Inactivation or lesions of LMAN (lateral magnocellular nucleus of the nidopallium), the output pathway of the AFP, largely eliminate song variability in both juvenile (8, 15) and adult birds (16). Furthermore, single-unit recordings in young birds show that LMAN neurons projecting to the motor pathway exhibit highly variable bursts of activity immediately preceding modulations in vocal output (17). Finally, electrical stimulation of LMAN can produce transient changes in song amplitude or pitch (16). Thus, LMAN neurons projecting to the motor pathway exert a direct premotor influence on vocal output, driving vocal variability that can be used for learning (18). These results support the hypothesis that vocal learning proceeds by trial-and-error (or reinforcement) learning (15, 16, 18–21).
It has been proposed that the AFP, in addition to generating variability, evaluates vocal errors and transmits a signal to guide plasticity in the motor pathway (7, 22). This hypothesis is supported by the fact that lesions of the AFP prevent changes in juvenile (8) and adult (10) song. Given the premotor influence of the AFP, an interesting possibility is that this hypothesized signal takes the form of a premotor drive that biases the song away from vocal errors. It has been suggested that such a premotor bias could instruct long-term plasticity in the motor pathway (21). Here, we combine 2 techniques—experimentally controlled vocal learning and transient localized brain inactivation—to demonstrate that the AFP is involved in generating a corrective premotor bias.
Results
Natural song learning proceeds slowly and unpredictably, and it is difficult to quantitatively associate song changes with a reduction in perceived vocal error. To make vocal learning more experimentally accessible, we developed a training protocol in which we controlled vocal learning by manipulating auditory feedback, similar to a recently described approach (18). Disruptive auditory feedback was played to the bird during singing (23) using an implanted speaker, and was made contingent on the pitch of a region within a targeted song syllable (Fig. 1 A and B; see SI Methods and Figs. S1 and S2). Playing disruptive auditory feedback during moments when the pitch of the targeted syllable was above a threshold value caused the bird to sing the targeted syllable at gradually lower pitches (Fig. 1 C and D). Upward movement of pitch was produced by reversing the contingency (Fig. 1D). To induce many sequential days of learning, we recentered the threshold of the disruptive auditory feedback each morning before the bird awoke. The average pitch of the targeted syllable changed most rapidly during the first 4 h after waking, at a rate of 3.5 ± 0.4 Hz/h (Fig. 1E, n = 5 birds, 80 experimental days, SEM unless otherwise indicated) in the instructed direction. The induced changes in pitch resulted in a reduction in the amount of disruptive auditory feedback (Fig. 1F), quantified as the average power of the feedback played during the targeted syllable. Although the majority of pitch changes in the targeted syllable occurred during the day, the pitch in the morning had a tendency to begin beyond what was acquired by the end of the previous day (Fig. 1G, 5.86 ± 1.4 Hz in the instructed direction, n = 43, P < 0.01, 2-tailed t test). For comparison, the average total shift during the day was 15.5 ± 1.5 Hz in the instructed direction (Fig. 1 H and P < 10−14). This change in pitch was larger, on average, than the standard deviation of the pitch of the targeted syllable (by 144 ± 13%); thus, to summarize, our protocol produced learned changes in pitch over the course of several hours that dramatically reduced the amount of auditory feedback “error.”
Does the AFP make a premotor contribution to these adaptive changes in pitch? To address this question, we used tetrodotoxin (TTX) to transiently inactivate LMAN during learning (Fig. 2A and Figs. S3 and S4). If premotor signals from the AFP directly contribute to learned pitch changes, we would expect inactivation to cause an immediate regression of the learned change. In contrast, if the learned pitch changes are immediately implemented by synaptic plasticity in the motor pathway, we would expect the average pitch to be the same before and after LMAN inactivation. Each day, after the first 4 h of conditional feedback, we infused either TTX or vehicle (on alternating days) into LMAN. Inactivation of LMAN with TTX produced an immediate change in pitch in a direction opposite that of the ongoing learning (Fig. 2 B and E, 16.3 ± 2.2 Hz, n = 34 inactivations, P < 10−7, 2-tailed t test, also see Figs. S5 and S6). Inactivation most often resulted in an increase in the amount of disruptive auditory feedback played to the bird (Fig. 2D, 30/34 inactivations, feedback power increased by a factor of 15 ± 6, P < 0.02, 1-tailed t test). In contrast, infusion of vehicle had no significant effect on average pitch (Fig. 2 C and F, n = 34 infusions, P > 0.4). These observations suggest that the AFP contributes to vocal output by biasing syllable pitch in a direction that reduces experimentally imposed error.
In the remaining text, we refer to the pitch of the targeted syllable generated with LMAN intact and LMAN inactivated as “LMAN(+) pitch” and “LMAN(−) pitch” respectively. The motor pathway encodes a stereotyped motor program for singing that operates independently of the AFP (7, 8, 15, 16). Thus, LMAN(−) pitch reflects this motor-pathway-encoded version of the targeted syllable. Furthermore, the contribution of the AFP to the pitch of the targeted syllable can be quantified as LMAN(+) pitch minus LMAN(−) pitch, which we refer to as “AFP-dependent bias,” or more briefly as “AFP bias.”
Across multiple days of conditional feedback, we observed large accumulating changes in the pitch of the targeted syllable (Fig. 1D). Are all of these changes due to accumulating AFP bias, or is there also a contribution from AFP-independent mechanisms, such as plasticity in the motor pathway? To answer this question, we examined whether, over the course of the experiment, LMAN(−) pitch deviated from the baseline pitch sung before the first day of conditional feedback. We found that LMAN(−) pitch exhibited substantial deviations from baseline (Fig. 3A and B, range = 121 ± 5 Hz SD). In fact, deviations in LMAN(−) pitch closely tracked the deviations of LMAN(+) pitch (Fig. 3 C and D, n = 34, r2 = 0.85, slope = 1.08 ± 0.08), indicating that during the course of our experiments, AFP-independent mechanisms contributed substantially to the expression of learned changes in pitch. In contrast, the magnitude of the AFP bias was not correlated with how far the pitch deviated from baseline (Fig. 3E, P > 0.3, slope not different from 0), indicating that continued learning in the same direction over multiple days was not a result of increasing AFP-dependent bias, but rather an increasing AFP-independent contribution.
Consistent with this result, we found that LMAN inactivation caused the pitch of targeted syllable to regress to near the pitch in the first morning songs. That is, the size of the AFP bias was correlated with the size of the pitch shift that was learned during the morning before inactivation (Fig. 4A and B, n = 34, r2 = 0.73, slope = 0.98 ± 0.11, P < 10−9, slope > 0). Taken together, our findings thus far suggest that the AFP makes a direct contribution to learned pitch changes, but that the size of the AFP contribution is limited and is proportional to the amount of learning that occurred within the last day. In addition, during multiple days of learning, the majority of the accumulated deviations in pitch from baseline are encoded in the motor pathway.
The syllable pitch encoded in the motor pathway is highly plastic, as evidenced by the large changes in LMAN(−) pitch between successive inactivations (Fig. 3B). We tested the hypothesis that these plastic changes are predicted by the amount of AFP bias observed during some preceding interval. Plastic changes in the motor pathway were assessed as the difference between successive LMAN(−) syllable pitch measurements (Fig. 5A). Because LMAN inactivations were carried out every other day, this measure reflects the change over a 2-day interval. The total AFP bias was also calculated in 2-day intervals as a sum over consecutive days. AFP bias was directly measured on TTX days, and was estimated on vehicle days as the total amount of learning that occurred on that day—motivated by our finding that the AFP bias is approximately equal to the amount of learning that occurred before TTX infusion (Fig. 4).
We started by examining the relation between motor pathway plasticity and AFP bias 1 day earlier. Specifically, we find that plastic changes in the motor pathway over a 2-day interval (e.g., between days n − 2 and n) are strongly correlated with the sum of the AFP bias measured on day n − 2 and estimated on day n − 1 (Fig. 5 A–C, r2 = 0.93, slope = 0.99 ± 0.06). This correlation is further supported by examining variations in the amount of learning. On up days, a large AFP bias was typically followed by a large change in the motor pathway, and a small AFP bias was followed by a small change. This same relation held for down days, but the signs of the quantities were negative. To combine all of the data, we inverted the sign of the data for down days. We find that variations in the AFP bias were correlated with variations in motor plasticity observed 1 day later (Fig. 5D, lag = −1 day: r2 = 0.70). These findings suggest a strong link between AFP bias and plasticity in the motor pathway within the next day.
We next examined the temporal specificity of the relation between AFP bias and subsequent plasticity in the motor pathway. Do variations in AFP bias only correlate with changes in LMAN(−) pitch on the next day, or do they also correlate with motor plasticity on the same day, or 2 days later? We find that variations in AFP bias were not as strongly correlated with variations in motor plasticity observed on the same day (lag = 0, r2 = 0.18), or 2 days later (Fig. 5 D and E, lag = −2: r2 = 0.08, also see Fig. S7). Of course, a careful examination of the temporal relation between AFP bias and plasticity in the motor pathway will require experiments in which LMAN inactivations are carried out at a finer time resolution. Nevertheless, these results lend further support to the hypothesis that AFP bias may be subsequently consolidated in the motor pathway by the end of the following day.
AFP bias was not only correlated with plastic changes in the motor pathway over the next day, but these quantities have the same magnitude. The change in LMAN(−) pitch between successive measurements (average 16.7 ± 1.2 Hz/day) was not significantly different from the size of the estimated AFP bias at a lag of minus 1 day (average 15.1 ± 1.7 Hz, P > 0.11, paired t test, see Methods), suggesting a remarkable correspondence between the mechanisms that underlie AFP bias and subsequent motor plasticity.
Discussion
Our experiments take advantage of the fact that, in response to an experimentally controlled association between syllable pitch and auditory error, a singing bird makes rapid corrective pitch changes (18). Applied over many days this protocol produced large accumulating changes in pitch. By inactivating the AFP during learning, we identified 2 contributions to the observed vocal plasticity. First, we found an AFP-dependent contribution that biases vocal output to reduce the probability and intensity of imposed error. Second, we observed an AFP-independent contribution, likely due to plasticity in the motor pathway, that accounts for the majority of the accumulated pitch changes. We also found a remarkable correspondence between AFP-dependent bias and plastic changes in the motor pathway observed within the next day. Thus, our results establish the time scale on which adaptive changes in vocal output become encoded in the motor pathway and expressed in an AFP-independent manner.
Our findings suggest a possible role for AFP-dependent bias during natural vocal imitation. It will be important to determine whether during natural song learning, the AFP biases motor output to improve the match between auditory feedback and tutor song. This could be examined by carrying out LMAN inactivations during the rapid, early phases of vocal learning in juvenile songbirds. We anticipate that inactivation of LMAN in the evening would result in a regression of learned changes in the song that occurred during the hours before inactivation and furthermore would result in a decrease in similarity with tutor song.
There are several mechanisms by which the bias we observe could be dependent on AFP activity. The AFP could serve a permissive role. For example, it is possible that HVC drives bias, but that its expression is dependent on AFP input to RA. Alternatively, the AFP could serve a direct premotor role in generating bias. Given the premotor contribution of the AFP to generating vocal variability, we favor the interpretation that AFP-driven variability itself becomes biased, generating larger or more frequent variations in the direction of reduced vocal error.
The mechanistic role of the 3 AFP nuclei in generating bias remains to be determined. Striatal neurons have been shown to be involved in the rapid evaluation of rewarded associations (13) and in the coding of action-specific reward values (24, 25). One possibility is that “random” activity patterns in LMAN produce variability in vocal output and are evaluated by basal ganglia homologue Area X. The results of this evaluation could then be sent to LMAN (through pallidal-recipient thalamic nucleus DLM) to reinforce LMAN activity patterns that produced desirable vocal output, thereby resulting in bias. Area X receives an efference copy of activity in LMAN (26, 27) and HVC (28, 29), thus placing Area X in a position to evaluate exploratory activity in LMAN in the context of the ongoing song. It is not known, however, whether Area X receives evaluative feedback about song performance either from auditory areas (30) or from midbrain dopaminergic areas (31).
Trial and error, or reinforcement, learning requires both exploratory behavior and the evaluation of performance (32). Biased variability could subserve both of these functions. First, biased variability could make motor exploration more efficient by increasing the speed at which effective motor control parameters are discovered. Second, AFP bias could instruct plasticity in the motor pathway (21). Although we cannot rule out the possibility that AFP bias and motor plasticity are implemented by separate evaluative mechanisms, our finding that the amount of AFP bias on any 1 day is correlated with, and has the same magnitude, as the amount of plasticity in the motor pathway within the next day, favors a causal role for the AFP in driving this plasticity. Furthermore, it suggests a view in which plasticity in the motor pathway temporally integrates, or accumulates, a more rapidly learned motor signal expressed by the AFP.
The precise time course and mechanism of consolidation into the motor pathway remains an open question. For example, it is possible that AFP bias actively drives plasticity in the motor pathway during singing by an on-line mechanism, such as Hebbian learning (33). However, several studies have shown that sleep may play an important role in the consolidation of learned skills (34–37). In particular, observations of sleep-replay activity in the motor pathway have led to the proposal that plasticity in this circuit occurs off-line (34), perhaps during sleep. Repeated brief inactivation of LMAN at several time points during the day would help distinguish these hypotheses.
Broadly speaking, our observations shed light on the function of basal ganglia-forebrain circuits in vertebrates, particularly the mechanisms by which practice of complex motor skills results in improved performance, and even more generally, the mechanisms by which goal-directed behaviors become entrained as highly stereotyped sequences or habits (5).
Materials and Methods
Subjects.
Juvenile and young adult (age 77–182 dph) male zebra finches (Taeniopygia guttata) were used. Animals were selected for sufficient singing rates (>200 song bouts per day), and for songs containing a harmonic stack with a pitch unique within the song and between 500 and 2,000 Hz. All procedures were approved by the Massachusetts Institute of Technology Committee on Animal Care. For surgical procedures, see SI Methods.
Song Recording.
Birds were housed individually in custom sound isolation chambers. Singing was measured using a miniature microphone (WBHC-23910, Knowles Electronics, Inc, 0.08 g, Fig. S1A) attached to the bird's head during surgery, providing measurement of song largely independent of position in the cage. Monitoring and recording of song was performed using custom MATLAB (Mathworks) software.
Conditional Auditory Feedback.
The auditory signal perceived by the bird during singing was disrupted by playing broadband noise. This disruptive auditory feedback was generated using a hearing aid speaker (EM-23046-CX, Knowles Electronics, 0.21 g) and transmitted into the cranial airsac surrounding the cerebellum via an implanted speaker tube (Fig. S1). Sound levels were calibrated individually for each bird (see SI Methods). In this configuration, feedback did not distort the signal recorded by the head-mounted microphone, allowing uninterrupted monitoring of singing.
In our conditional feedback protocol, feedback was contingent on a measure of syllable pitch (18). This measure was computed using a set of finite-impulse response (FIR) filters implemented in custom software running on a digital signal processor (RX8, Tucker-Davis Technologies) (Fig. S2 and SI Methods). The measure of syllable pitch was used to calculate the loudness of the feedback noise relative to the loudness of the ongoing song vocalization. In this way, the bird could not escape feedback simply by singing louder (see SI Methods). The pitch threshold for feedback was constant during each day, and was set each morning to the average pitch of the targeted syllable measured at the end of the previous day. All feedback powers reported are calibrated acoustic power played into the cranial airsac.
Transient LMAN Inactivation.
Custom reverse microdialysis probes were built using dialysis tubing (200-μm diameter) attached to a drug reservoir (Fig. S3 A and B), as described in ref. 17. Probes were implanted bilaterally into LMAN using stereotaxic coordinates (Fig. S3C). Inactivation was carried out by filling the reservoir and dialysis tubing with 25 μM TTX in PBS. This concentration was found to saturate the reduction of pitch variability resulting from inactivation of LMAN (from dose–response curve, Fig. S4C). After nominally 4 h, drug was removed by flushing the tubing and filling the reservoir with PBS. On vehicle days, the same procedures were carried out with PBS instead of TTX. Electrophysiological recordings in anesthetized birds (n = 2) confirmed that 25 μM TTX infusion completely suppressed electrical activity within 0.75 mm of the probe, thus fully encompassing LMAN. In contrast, activity persisted in medial MAN (1.05 mm from the probe) for the duration of the recording session, at least 1–1.5 h after infusion.
Experimental Design.
The birds were housed individually in custom sound-attenuated chambers equipped with 10-channel commutators (SL-88, Dragonfly), and were maintained on a 12 h:12 h light–dark cycle. Conditional feedback was started once the birds consistently sang after infusion. TTX and vehicle were infused on alternate days, nominally 4 h (3.9 ± 1.0 SD) after the first morning singing. Before lights on, the FIR filters were updated so that the edge of the band of targeted pitches (i.e., the threshold) was placed approximately at the center of the pitch distribution at the end of the previous day (last 50 syllables, Fig. S2C). On the morning after a TTX day, the threshold was centered to the average of the last 50 syllables before TTX infusion. See SI Methods for further details and statistics. Conditional feedback remained on throughout the entire day, including during and after drug infusion.
Data Analysis.
Song was segmented into syllables based on song amplitude. The fidelity of the head-mounted microphone signal made segmentation highly reliable. Acoustic feature vectors were calculated for all segmented syllables (duration, amplitude, entropy, pitch goodness, and variance of the last 2 measures). Syllables were classified using these feature vectors and custom hand-clustering software. For each rendition of the targeted syllable, the time course of the pitch (Fig. S5) and feedback power were computed. By averaging this time course, the mean pitch and mean feedback power of each rendition was calculated.
For each experimental day, pitch and feedback power were computed at 4 key time points during the day by averaging the first 50 renditions of the morning, the last 50 renditions before infusion, the first 50 renditions after infusion, and the final 50 renditions of the day. These values were used to quantify the changes in pitch: (i) during learning before infusion, (ii) as a result of infusion, and (iii) overnight. Overnight changes in pitch were quantified as the difference between the morning pitch and that of the final renditions of the preceding day; nights after TTX infusion were excluded from this analysis. Baseline pitch was computed as the average pitch of the targeted syllable on the day before starting feedback.
The time series of motor pathway plasticity (Δm) and the estimated AFP bias (β + β*) were plotted (Fig. 5C) as follows: The motor pathway plasticity plotted on day n was calculated as LMAN(−) pitch on day n minus LMAN(−) pitch on day n − 2. These points appear every other day because LMAN was only inactivated on alternate days. The running 2-day sum of AFP bias plotted on day n was calculated as the sum of AFP bias on day n − 2 and the AFP bias on day n − 1. The 95% confidence intervals for the correlation coefficients in Fig. 5E were computed using nonparametric bootstrapping.
Histology.
Animals were deeply anesthetized with pentobarbital and perfused with 4% paraformaldehyde (Sigma). Brains were removed from the skull and postfixed in 4% paraformaldehyde. Brains were sectioned parasagitally with a vibrating Microtome (100 μm thick, Vibratome 1000, TPI), and the location of the dialysis membrane was determined (Fig. S3C).
Supplementary Material
Acknowledgments.
We thank Dmitriy Aronov, Martha Bergmark, Tom Davidson, Jesse Goldberg, Ann Graybiel, Wrenn Levenberg, Michael Long, Ben Scott, Sebastian Seung, and Matt Wilson for their helpful comments on earlier versions of this manuscript. This work was supported by National Institutes of Health Grant R01DC009183 (to M.S.F.) and a Friends of McGovern Institute Fellowship (to A.S.A.).
Footnotes
The authors declare no conflict of interest.
This article is a PNAS Direct Submission.
This article contains supporting information online at www.pnas.org/cgi/content/full/0903214106/DCSupplemental.
References
- 1.Konishi M. The role of auditory feedback in the control of vocalization in the white-crowned sparrow. Z Tierpsychol. 1965;22:770–783. [PubMed] [Google Scholar]
- 2.Immelmann K. In: Bird Vocalizations. Hinde RA, editor. London: Cambridge Univ Press; 1969. pp. 61–74. [Google Scholar]
- 3.Tchernichovski O, Lints T, Mitra PP, Nottebohm F. Vocal imitation in zebra finches is inversely related to model abundance. Proc Natl Acad Sci USA. 1999;96:12901–12904. doi: 10.1073/pnas.96.22.12901. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Knowlton BJ, Mangels JA, Squire LR. A neostriatal habit learning system in humans. Science. 1996;273:1399–1402. doi: 10.1126/science.273.5280.1399. [DOI] [PubMed] [Google Scholar]
- 5.Graybiel AM. Habits, rituals, and the evaluative brain. Annu Rev Neurosci. 2008;31:359–387. doi: 10.1146/annurev.neuro.29.051605.112851. [DOI] [PubMed] [Google Scholar]
- 6.Wise SP, Murray EA, Gerfen CR. The frontal cortex-basal ganglia system in primates. Crit Rev Neurobiol. 1996;10(3–4):317–356. doi: 10.1615/critrevneurobiol.v10.i3-4.30. [DOI] [PubMed] [Google Scholar]
- 7.Bottjer SW, Miesner EA, Arnold AP. Forebrain lesions disrupt development but not maintenance of song in passerine birds. Science. 1984;224:901–903. doi: 10.1126/science.6719123. [DOI] [PubMed] [Google Scholar]
- 8.Scharff C, Nottebohm F. A comparative study of the behavioral deficits following lesions of various parts of the zebra finch song system: Implications for vocal learning. J Neurosci. 1991;11:2896–2913. doi: 10.1523/JNEUROSCI.11-09-02896.1991. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Sohrabji F, Nordeen EJ, Nordeen KW. Selective impairment of song learning following lesions of a forebrain nucleus in the juvenile zebra finch. Behav Neural Biol. 1990;53:51–63. doi: 10.1016/0163-1047(90)90797-a. [DOI] [PubMed] [Google Scholar]
- 10.Brainard MS, Doupe AJ. Interruption of a basal ganglia-forebrain circuit prevents plasticity of learned vocalizations. Nature. 2000;404:762–766. doi: 10.1038/35008083. [DOI] [PubMed] [Google Scholar]
- 11.Farries MA, Perkel DJ. A telencephalic nucleus essential for song learning contains neurons with physiological characteristics of both striatum and globus pallidus. J Neurosci. 2002;22:3776–3787. doi: 10.1523/JNEUROSCI.22-09-03776.2002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Atallah HE, Lopez-Paniagua D, Rudy JW, O'Reilly RC. Separate neural substrates for skill learning and performance in the ventral and dorsal striatum. Nat Neurosci. 2007;10:126–131. doi: 10.1038/nn1817. [DOI] [PubMed] [Google Scholar]
- 13.Pasupathy A, Miller EK. Different time courses of learning-related activity in the prefrontal cortex and striatum. Nature. 2005;433:873–876. doi: 10.1038/nature03287. [DOI] [PubMed] [Google Scholar]
- 14.Yin HH, et al. Dynamic reorganization of striatal circuits during the acquisition and consolidation of a skill. Nat Neurosci. 2009;12:333–341. doi: 10.1038/nn.2261. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Ölveczky BP, Andalman AS, Fee MS. Vocal experimentation in the juvenile songbird requires a basal ganglia circuit. PLoS Biol. 2005;3:e153. doi: 10.1371/journal.pbio.0030153. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Kao MH, Doupe AJ, Brainard MS. Contributions of an avian basal ganglia-forebrain circuit to real-time modulation of song. Nature. 2005;433:638–643. doi: 10.1038/nature03127. [DOI] [PubMed] [Google Scholar]
- 17.Aronov D, Andalman AS, Fee MS. A specialized forebrain circuit for vocal babbling in the juvenile songbird. Science. 2008;320:630–634. doi: 10.1126/science.1155140. [DOI] [PubMed] [Google Scholar]
- 18.Tumer EC, Brainard MS. Performance variability enables adaptive plasticity of “crystallized” adult birdsong. Nature. 2007;450:1240–1244. doi: 10.1038/nature06390. [DOI] [PubMed] [Google Scholar]
- 19.Doya K, Sejnowski TJ. In: Central Auditory Processing and Neural Modeling. Poon PWF, Brugge JF, editors. New York: Plenum; 1998. pp. 77–88. [Google Scholar]
- 20.Fiete IR, Fee MS, Seung HS. Model of birdsong learning based on gradient estimation by dynamic perturbation of neural conductances. J Neurophysiol. 2007;98:2038–2057. doi: 10.1152/jn.01311.2006. [DOI] [PubMed] [Google Scholar]
- 21.Troyer TW, Doupe AJ. An associational model of birdsong sensorimotor learning I. Efference copy and the learning of song syllables. J Neurophysiol. 2000;84:1204–1223. doi: 10.1152/jn.2000.84.3.1204. [DOI] [PubMed] [Google Scholar]
- 22.Troyer TW, Bottjer SW. Birdsong: Models and mechanisms. Curr Opin Neurobiol. 2001;11:721–726. doi: 10.1016/s0959-4388(01)00275-6. [DOI] [PubMed] [Google Scholar]
- 23.Leonardo A, Konishi M. Decrystallization of adult birdsong by perturbation of auditory feedback. Nature. 1999;399:466–470. doi: 10.1038/20933. [DOI] [PubMed] [Google Scholar]
- 24.Samejima K, Ueda Y, Doya K, Kimura M. Representation of action-specific reward values in the striatum. Science. 2005;310:1337–1340. doi: 10.1126/science.1115270. [DOI] [PubMed] [Google Scholar]
- 25.Kawagoe R, Takikawa Y, Hikosaka O. Expectation of reward modulates cognitive signals in the basal ganglia. Nat Neurosci. 1998;1:411–416. doi: 10.1038/1625. [DOI] [PubMed] [Google Scholar]
- 26.Vates GE, Nottebohm F. Feedback circuitry within a song-learning pathway. Proc Natl Acad Sci USA. 1995;92:5139–5143. doi: 10.1073/pnas.92.11.5139. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Vates GE, Vicario DS, Nottebohm F. Reafferent thalamo-“cortical” loops in the song system of oscine songbirds. J Comp Neurol. 1997;380:275–290. [PubMed] [Google Scholar]
- 28.Kozhevnikov AA, Fee MS. Singing-related activity of identified HVC neurons in the zebra finch. J Neurophysiol. 2007;97:4271–4283. doi: 10.1152/jn.00952.2006. [DOI] [PubMed] [Google Scholar]
- 29.Nottebohm F, Stokes TM, Leonard CM. Central control of song in the canary, Serinus canarius. J Comp Neurol. 1976;165:457–486. doi: 10.1002/cne.901650405. [DOI] [PubMed] [Google Scholar]
- 30.Keller GB, Hahnloser RH. Neural processing of auditory feedback during vocal practice in a songbird. Nature. 2009;457:187–190. doi: 10.1038/nature07467. [DOI] [PubMed] [Google Scholar]
- 31.Gale SD, Perkel DJ. Physiological properties of zebra finch ventral tegmental area and substantia nigra pars compacta neurons. J Neurophysiol. 2006;96:2295–2306. doi: 10.1152/jn.01040.2005. [DOI] [PubMed] [Google Scholar]
- 32.Sutton RS, Barto AG. Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press; 1998. p. 322. [Google Scholar]
- 33.Houk JC, Wise SP. Distributed modular architectures linking basal ganglia, cerebellum, and cerebral cortex: Their role in planning and controlling action. Cereb Cortex. 1995;5:95–110. doi: 10.1093/cercor/5.2.95. [DOI] [PubMed] [Google Scholar]
- 34.Dave AS, Margoliash D. Song replay during sleep and computational rules for sensorimotor vocal learning. Science. 2000;290:812–816. doi: 10.1126/science.290.5492.812. [DOI] [PubMed] [Google Scholar]
- 35.Deregnaucourt S, Mitra PP, Feher O, Pytte C, Tchernichovski O. How sleep affects the developmental learning of bird song. Nature. 2005;433:710–716. doi: 10.1038/nature03275. [DOI] [PubMed] [Google Scholar]
- 36.Walker MP, Brakefield T, Hobson JA, Stickgold R. Dissociable stages of human memory consolidation and reconsolidation. Nature. 2003;425:616–620. doi: 10.1038/nature01930. [DOI] [PubMed] [Google Scholar]
- 37.Wilson MA, McNaughton BL. Reactivation of hippocampal ensemble memories during sleep. Science. 1994;265:676–679. doi: 10.1126/science.8036517. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.