Abstract
Neural representations of the external world are constructed and updated in a manner that depends on behavioral context. For neocortical networks, this contextual information is relayed by a diverse range of neuromodulatory systems, which govern attention and signal the value of internal state variables such as arousal, motivation, and stress. Neuromodulators enable cortical circuits to differentially process specific stimuli and modify synaptic strengths in order to maintain short- or long-term memory traces of significant perceptual events and behavioral episodes. One of the most important subcortical neuromodulatory systems for attention and arousal is the noradrenergic locus coeruleus. Here we report that the noradrenergic system can enhance behavior in rats performing a self-initiated auditory recognition task, and optogenetic stimulation of noradrenergic locus coeruleus neurons accelerated the rate at which trained rats began correctly responding to a change in reward contingency. Animals successively progressed through distinct behavioral epochs, including periods of perseverance and exploration that occurred much more rapidly when animals received locus coeruleus stimulation. In parallel, we made recordings from primary auditory cortex and found that pairing tones with locus coeruleus stimulation led to a similar set of changes to cortical tuning profiles. Thus both behavioral and neural responses go through phases of adjustment for exploring and exploiting environmental reward contingencies. Furthermore, behavioral engagement does not necessarily recruit optimal locus coeruleus activity.
Keywords: auditory cortex, behavior, locus coeruleus, norepinephrine, perceptual learning, plasticity
1. Introduction
The brain dynamically represents sensory information, allowing animals to adequately explore and exploit complex, changing, and potentially hazardous environments. Sensory input interacts with ongoing neural activity and various internal state variables to produce appropriate outputs at the levels of single neurons, networks, and behavior. Neural circuits and behavioral outputs are plastic, and can be modified by changes in the pattern of sensory inputs. Sensory stimuli that are novel, salient, potentially hazardous, or otherwise behaviorally relevant can trigger the central release of endogenous neuromodulators that alter excitability and synaptic transmission in target neuronal networks. While these 'modulatory' effects can sometimes be relatively subtle, in many cases the effects of neuromodulation on cognition and neural function are substantial and profound, such as enabling or gating the induction of long-term synaptic plasticity (Bear and Singer, 1986; Froemke, 2015), triggering brain state transitions (Carter et al., 2012; Constantinople and Bruno, 2011; Steriade, 1997), or controlling selective attention to ensure that some incoming stimuli are detected and recognized while others are ignored (Disney et al., 2007; Hasselmo and Sarter, 2011; Roberts and Thiele, 2008).
The locus coeruleus was first discovered in the human brain by J.C. Reil in 1809 as a streak of dark blue substance in the brainstem, near the lateral wall of the fourth ventricle (Reil, 1809). This structure was later named by Wenzel and Wenzel (1812), after the Latin words describing the appearance (a 'blue place'), and stereotaxically identified by Russel and subsequent anatomists (Amaral and Sinnamon, 1977; German et al., 1988; Russell, 1955). In the rat brain, locus coeruleus is a small structure, around 300 μm wide and up to 600 μm along the dorsal-ventral axis. Rat locus coeruleus contains around 1500–2000 cells, 200 of which are in a more ventral location called the subcoeruleus area (Swanson, 1976). Human locus coeruleus contains roughly 10–20 times as many neurons (German et al., 1988). Neurons in locus coeruleus are electronically coupled (Christie et al., 1989; Christie and Jelinek, 1993; Ishimatsu and Williams, 1996) and can be divided into subpopulations according to their morphology, into fusiform, large multipolar and small round cells.
One of the most striking features of the locus coeruleus is the widespread efferent network, constituting the sole source of central nervous system noradrenaline, with axonal projections being found in all regions and layers of cortex (Levitt and Moore, 1978). This is related to the involvement of locus coeruleus in many important neural and physiological functions including respiration, cardiac function, micturition, motivation, attention, arousal, regulation of sleep-awake cycles, stress, and learning and memory (Amaral and Sinnamon, 1977; Aston-Jones and Bloom, 1981; Aston-Jones et al., 1997; Aston-Jones and Cohen, 2005; Berridge et al., 1993; Berridge and Waterhouse, 2003; Bouret and Sara, 2004; Bouret and Sara, 2005; Carter et al., 2010; Constantinople and Bruno, 2011; Devauges and Sara, 1990; Foote et al., 1975; Froemke and Schreiner, 2015; Gu, 2002; Martins and Froemke, 2015; Roussel et al., 1967; Sara and Devauges, 1988; Vazey and Aston-Jones, 2014; Yu and Dayan, 2005). Notably, locus coeruleus activity can improve perception across numerous sensory percepts (Escanilla et al., 2010; Manella et al., 2017; Martins and Froemke, 2015; Navarra et al., 2017). Recordings from locus coeruleus have shown that these neurons have both tonic and phasic firing patterns, believed to have differential effects on behavior performance, arousal, and attention (Aston-Jones and Cohen, 2005; Berridge and Waterhouse, 2003). Tonic firing is important for maintaining long-term changes in sensory networks, associated with different states of arousal (Aston-Jones and Bloom, 1981; Constantinople and Bruno, 2011; Martins and Froemke, 2015). In contrast, phasic firing is thought to modulate target areas more acutely, changing signal-to-noise ratios and modifying sensory representations such as receptive fields to accommodate newly salient and/or surprising, sensory information (Castro-Alamancos, 2002; Devilbiss and Waterhouse, 2000; Hirata et al., 2006; Martins and Froemke, 2015; Nieuwenhuis et al., 2005). Direct recordings from locus coeruleus in monkeys performing an attention task showed that changes in firing related with overall behavioral performance and could precede behavioral shifts (Aston-Jones et al., 1994).
Previous studies in the auditory cortex found that noradrenergic modulation could affect tuning curves and improve auditory perception in some cases. Manunta and Edeline (2004) found that iontophoretic application of norepinephrine paired with pure tones could persistently change tonal tuning profiles largely through activation of noradrenergic α-receptors. Many of these changes were suppressive, but Edeline et al. (2011) showed that pairing tones with endogenous noradrenergic release via locus coeruleus stimulation (‘locus coeruleus pairing’) could be more effective at enhancing responses relative to iontophoretic pairing. Locus coeruleus pairing could affect thalamic responses as well, but changes endured much longer in the cortex than in the auditory thalamus (Edeline et al., 2011). Similarly, we found that pairing tones with either electrical or optogenetic locus coeruleus stimulation could modify tuning curves in adult rat auditory cortex (Martins and Froemke, 2015). These changes in auditory responses could improve auditory perception and enhance learning rates when a rewarded tone and an unrewarded tone switched behavioral meaning (i.e., the reward schedule for different stimuli was suddenly reversed from one behavioral testing session to the next). However, in those previous behavioral experiments, auditory stimuli were presented in an uncued manner during training and testing, requiring that animals maintain a high level of alertness throughout the entire behavioral session or performance would lapse. Thus locus coeruleus pairing might have just enhanced overall arousal and behavioral engagement, rather than have specifically promoted behaviorally-relevant plasticity. Here we now examine this issue more directly by examining the consequences of locus coeruleus pairing on a self-initiated auditory recognition task, in which the level of task engagement should be more standardized across trials.
2. Results
Here we examined how animals behaviorally respond to a switch in reward on an auditory task, before asking how locus coeruleus stimulation affects behavior or task-relevant neural activity.
2.1 Rats have stereotyped behavioral responses to changes in reward
To examine how animals responded to a change in reward contingency, we trained 20 rats on an auditory recognition go/no-go task. Animals were operantly conditioned to self-initiate trials, nosepoking for a food reward to target tones of a given frequency (initially 4 kHz) and withholding responses to non-target foil tones (Fig. 1A). Stimuli were 0.5–32 kHz pure tones at one octave intervals, presented at 70 dB sound pressure level (SPL) and 100 msec in duration. Animals were trained and tested 1–2 hours/day daily or near-daily for about two months. After 2–3 weeks of training, animals had high hit rates (80–90%) and low false alarm rates, resulting in d’ values of > 1.5. We have previously used this task to assess how self-initiation modulates auditory cortex during behavioral engagement and found that auditory cortical responses are required for task performance (Carcea et al., 2017). Furthermore, previously we have examined how cortical neuromodulation and plasticity can affect performance on an uncued version of this task in rats (Froemke et al., 2013) and mice (Kuchibhotla et al., 2017), including via activation of locus coeruleus (Martins and Froemke, 2015).
Once animals reached criteria for reliable performance, the rewarded tone was switched from 4 kHz to a different previously-unrewarded frequency. Behavioral performance was monitored for weeks thereafter to document when and how rats began to recognize the switch or ‘reversal’ in rewarded sound. One example animal is shown in Figure 1B, where day 1 is the first day that 1 kHz became the new rewarded target tone and 4 kHz became an unrewarded non-target tone. This animal perseverated at the original target tone for weeks, reliably nosepoking to the unrewarded 4 kHz tone until day 30. Additionally, this animal began exploring the behavioral consequences to other tones, nosepoking at a high rate to nearly all stimuli starting on day 6 and continuing through day 11, at which point this behavioral generalization persisted only for lower-frequency tones between 0.5–4 kHz until day 23.
These three features of auditory learning were consistent across animals: 1) rats initially perseverated on the original target (Fig. 2A), 2) after a few days rats began exploring responses to other tones (Fig. 2B), and 3) performance (as measured by d’) returned to originally high levels after several weeks (Fig. 2C). The duration of each of these behavioral epochs could be variable across animals, but for a given animal, the onset or offset of behavioral responses to a tone could be abrupt in terms of daily performance. Averaged across animals though, d’ values dropped to approximately zero on the first day that the target was switched, and appeared to gradually return to originally-high levels over a period of weeks. These features of reversal learning are similar to previous studies that have documented perseveration and exploration behaviors (Butter, 1969; Chudasama and Robbins, 2003; Judge et al., 2011).
In some animals, the new target differed from the original target by one octave (switched up in frequency from 4 kHz to 8 kHz or down in frequency to 2 kHz); Fig. 2 green symbols), in other animals, the new target was two octaves from the original (switched either up in frequency to 16 kHz or down in frequency to 1 kHz; Fig. 2 red symbols). Two-octave switches seemed to be more challenging for animals to re-learn compared to the one-octave switch (Fig. 2C), regardless of whether the target was higher (16 kHz) or lower (1 kHz). This was not due to the amount of perseverance at the original target, which was similar between one-octave and two-octave groups (Fig. 2A), but instead was a consequence of longer exploratory phases for the two-octave animals (Fig. 2B).
2.2 Locus coeruleus pairing accelerates auditory learning
Previous studies in rodents and primates indicate that locus coeruleus is activated during behavioral conditioning and particularly sensitive to switches of reward (Aston-Jones et al., 1997; Bouret and Sara, 2004). To examine whether locus coeruleus activity could promote auditory learning, we optogenetically paired locus coeruleus stimulation with the new rewarded target tone after reversal, using a combination of transgenic and viral approaches for expressing channelrhodopsin-2 in locus coeruleus noradrenergic/tyrosine hydroxylase (TH) cells. One animal was transgenic, with Cre recombinase expressed in TH+ cells, injected with pAAV5-EF1α-DIO-ChETA-EYFP (Witten et al., 2011). A second animal was a wild-type with CAV2-PRS-ChR2-mCherry (Hickey et al., 2014; Li et al., 2016) injected into locus coeruleus, utilizing the PRS promoter (Hwang et al., 2001) to selectively express in the locus coeruleus noradrenergic neurons. A third animal was a wild-type with pAAV5-CaMKII-ChETA-EYFP injected into locus coeruleus. We verified channelrhodopsin expression in TH+ cells in locus coeruleus with immunohistochemistry of tissue sections from animals after the end of the experiments (Fig. 3A). After initial training with a 4 kHz target, followed by surgery, animals were then re-trained to criterion on the original 4 kHz target before switching the target to 16 kHz on day 1 of testing. Starting on day 1 and every day thereafter, the new target tone was paired at 3 Hz with optogenetic locus coeruleus stimulation at 10 Hz for 5–10 minutes (Fig. 3B). These pairing sessions occurred outside of the context of the behavior, prior to the daily training sessions.
An example animal receiving locus coeruleus pairing is shown in Figure 3C. This animal rapidly learned the switch in rewarded tone, with behavioral performance returning to original levels within two weeks. This is in contrast to the slower learning rates in control uninjected wild-type animals (Figs. 1,2), including only the animals reversed to the same target tone as the locus coeruleus stimulated animals (Fig. 3D). This cohort of control animals includes two TH-Cre Long-Evans rat expressing only YFP in the locus coeruleus receiving sham optical stimulation, whose reversal learning was comparable to control animals.
In general, locus coeruleus pairing decreased the duration of perseveration and decreased the length of the exploratory phase, collectively leading to faster recovery of d’ after the target tone was switched (Fig. 4). In the first six days, control animals and locus coeruleus paired animals had similar levels of perseveration, but starting at day 7, locus coeruleus paired animals had significantly decreased perseveration (Fig. 4A). This difference persisted throughout reversal learning, peaking at day 23. Both control animals and locus coeruleus paired animals explored early, but locus coeruleus paired animals refined their exploration more quickly. As with perseveration, there was no difference in rates of exploration through day 6, but starting on days 7, locus coeruleus stimulated animals already had significantly reduced exploration compared to control animals, indicating that they were refining responses to the new target tone. Maximal differences in exploration rates occurred after three weeks of reversal training. This gap in exploration rates continued through nearly the end of reversal learning, when rates began to converge on days 36–40. The combination of decreased perseveration and a shortened exploratory phase led to faster rates of reversal learning. By day 7 of reversal learning, locus coeruleus paired animals were performing significantly better than controls on the auditory perceptual task as measured by d’. By day 8, locus coeruleus paired had returned to baseline performance levels while control animals did not consistently perform at baseline levels until day 32. When comparing the correlation of exploration and perseveration during the first and second weeks of reversal learning, there was no difference between locus coeruleus paired animals and control animals in the first week (Fig. 5A). During the second week, locus coeruleus animals had markedly lower rates of both exploration and perseveration than control animals (Fig. 5B). The observation that both rates decreased in a similar time frame suggests that these two aspects of reversal learning may co-vary. Notably, sham optically stimulated animals performed very similarly to control animals in both weeks one and two (Fig. 5). Furthermore, when compared to animals reversed on the one-octave variant of the task (Figs. 2,4), locus coeruleus paired two-octave animals had similar exploratory phases (p>0.5, performance on week two, Student’s unpaired two-tailed t-test with Bonferroni correction), but less perseveration (p<0.002), leading to overall faster reversal learning (p<0.005).
2.3. Locus coeruleus pairing has complex effects on cortical tuning curves
Previously we examined the effects of locus coeruleus pairing on cortical tuning curves with electrophysiological recordings in vivo (Martins and Froemke, 2015). Here we made new recordings in wild-type and TH-Cre rats expressing channelrhodopsin-2 in locus coeruleus neurons, and pairing a specific pure tone with optogenetic locus coeruleus stimulation (Fig. 6A). Optically-evoked responses were confirmed in a wild-type Long-Evans animal expressing ChETA in locus coeruleus under the CaMKII promoter. Locus coeruleus localization was first confirmed through multi-unit recordings of responses to noxious stimuli and optogenetic stimulation (Fig. 6B). We asked two questions: first, if the effects of pairing could improve representations in the auditory cortex and make those representations more discriminable to aid decoding; second, we asked if additional pairing episodes could further consolidate or sharpen tuning curves.
We made multiunit recordings from primary auditory cortex of anesthetized wild type Long-Evans and TH-Cre rats, performing nine pairing episodes in a total of three different animals. Two recordings showing the effects of single episodes of pairing immediately post-pairing are shown in Figure 6C. One of these recordings was made in the low frequency region of primary auditory cortex, and initially the best frequency was 2 kHz (Fig. 6C, top). The tone to be paired with locus coeruleus stimulation was 16 kHz, which did not initially evoke a response in this recording (Fig. 6C, upper left). The neural d’ for 16 kHz as effective ‘target’ was -1.9, meaning that the lack of response relative to the responses to other ‘foil’ tones would make 16 kHz tones difficult to detect and recognize. After pairing, the tuning profile broadened, increasing the relative response to the paired 16 kHz tone and normalizing responses to other unpaired tones (Fig. 6C, upper right). This broadening of responses across frequencies at the neural level is similar to the ‘exploratory’ phase of increased responses to non-target frequencies observed behaviorally. In the second example recording, the neural responses indicated the paired frequency was detectable compared to unpaired frequencies and the neural d’ was relatively high (Fig. 6C, bottom). Regardless, pairing could further refine responses (Fig. 6C, lower right). Across all nine pairing episodes, d’ values measured for the paired frequency increased from 0.4±0.4 to 1.0±0.4 (p<0.05, Student’s paired two-tailed t-test) immediately after pairing (Fig. 6D).
Several of these pairing episodes were not the first pairing, but occurred 60–120 minutes later (Fig. 7A) after a previous pairing. A series of three locus coeruleus pairings during a six hour recording is shown in Figure 7B,C. The first pairing increased the d’ at this recording site from 0.9 to 1.2, while the second pairing had no additional effect on tone-evoked responses or d’. However, the third pairing refined the tuning profile to accentuate the differences between the paired 16 kHz frequency and spectrally-similar tones, enhancing d’ from 1.2 to 1.9. These changes took tens of minutes to develop after the pairing episodes (Fig. 7B,C). Thus neural responses in auditory cortex have complex dynamics reflecting the behavioral changes that occur over the course of reversal learning, and sensitive to one or more episodes of locus coeruleus pairing in behaving rats.
3. Discussion
The locus coeruleus is the primary source of norepinephrine for the central nervous system. Activity in locus coeruleus can enable long-lasting changes in sensory input due to changes throughout the central nervous system including within the auditory thalamus, auditory cortex, and locus coeruleus itself (Devilbiss et al., 2006; Edeline et al., 2011; Martins and Froemke, 2015). Here we focused on relating the dynamics and discriminability of activity in auditory cortex to behavioral performance. Consistent with previous results (Martins and Froemke, 2015), we found that pairing locus coeruleus stimulation with a previously unrewarded tone on an auditory perceptual go/no-go task accelerated the rate at which animals learned to accurately respond to the newly rewarded, paired tone. Recordings from auditory cortex also showed that discrimination of a tone paired with locus coeruleus stimulation was increased post-pairing and was further potentiated with multiple pairings. While other neuromodulators such as acetylcholine and dopamine can also promote neuroplasticity (Bao et al., 2001; Froemke et al., 2013; Froemke, 2015), the effects of norepinephrine and locus coeruleus stimulation tend to be more potent. A single episode of locus coeruleus pairing can improve sensory detection for days to weeks (Edeline et al., 2011; Martins and Froemke, 2015).
It remains a major challenge in neuroscience to connect long-term synaptic plasticity to learned changes in behavior. In this study, we utilized auditory psychophysical methods to monitor consequences of plasticity due to locus coeruleus pairing. ‘Reversal learning’ (here referring to a change in reward contingency from one tone to another) is ideal for documenting the dynamics of perceptual learning and differences in these processes between groups of animals. This is because during the initial behavioral shaping and baseline training, animals must express several types of learning, including motor skills and habituating to the environment. Conversely, during reversal learning, stimulus-response associations can be more easily isolated for study. Although the averaged behavioral changes appeared incremental after switching reward contingency, changes could happen within single behavioral sessions in individual animals. Animals receiving locus coeruleus pairing made these transitions earlier than control animals, moving more quickly through the exploratory phase and refining responses to the new target tone, similar to a shift from exploration to exploitation (Doya, 2002; Usher et al., 1999; Yu and Dayan, 2005). These behavioral shifts occasionally occurred after 1–2 day breaks, which may suggest an interesting enhancement in performance following a longer consolidation period. However, there was no significant difference between locus coeruleus paired and control animals in the timing and number of these breaks, which did not occur systematically across animals or experimental groups. The possible significance of the effect of the breaks requires additional investigation.
While locus coeruleus pairing animals had lower rates of perseveration and exploration than controls, we also observed differences in performance between animals reversed to tones with a one-octave vs two-octave spectral difference, specifically in terms of exploration. This decrease in exploration is similar to that seen in two-forced alternative choice tasks, where little exploration is necessary when a reversal occurs (Costa et al., 2015). Even when well-trained animals have higher response rates to the tones closest to the target-tone, brief exploration would be sufficient for discovery of the new target stimulus. It is possible that in the one-octave version of the task, the “explore-exploit” phase is minimized already due to the lower need for exploration compared to the two-octave version of the task. However, in both versions of the task, increased locus coeruleus activity could reduce responses to the original target tone or best frequency (Martins and Froemke, 2015).
It had previously been shown that locus coeruleus activity was sensitive to changes in reward contingency and other surprising behavioral events (Aston-Jones and Cohen, 2005; Sara, 2009). This includes responses to conditioned stimuli such as sensory cues. The circuit organization and plasticity that produces such responses remains open for further investigation, as do the differential mechanisms of noradrenergic plasticity that affect auditory thalamus, cortex, and other regions of the central nervous system.
4. Experimental Procedure
Surgical preparation
All procedures were approved under an NYU IACUC Institutional Animal Care and Use Committee protocol, in animals kept in a vivarium on a 12/12 hour light/dark cycle and housed individually or in pairs. Female Long-Evans, TH-Cre, and Sprague-Dawley rats 3–5 months old were anesthetized with ketamine (1.2 ml/kg) and dexmedetomidine (1.0 ml/kg). Viral injections were performed using stereotaxic coordinates (from lambda, in mm: 3.7 posterior, 1.2 lateral, 5.6–6 ventral) with the head at a 15° downward angle. A craniotomy was placed over the left locus coeruleus and location was verified during procedures by measuring responses multiunit responses to noxious stimuli (tail pinch) and other electrophysiological criteria (spontaneous rates), and afterwards using histological methods. Injections were performed with a 5 μL Hamilton syringe and a 33 gauge needle. For optogenetic stimulation of locus coeruleus, we used three different methods. One animal was transgenic, with Cre recombinase expressed in TH+ cells, allowing for locus coeruleus restricted expression of Cre-inducible pAAV5-EF1α-DIO-ChETA-EYFP (Witten et al., 2011). Another animal was a wild-type Sprague-Dawley, with CAV2-PRS-ChR2-mCherry (Hickey et al., 2014; Li et al., 2016) injected into the locus coeruleus, utilizing the PRS promoter (Hwang et al., 2001) to selectively express in the locus coeruleus noradrenergic neurons. Finally, a third animal was a wild-type Sprague-Dawley with pAAV5-CaMKII-ChETA-EYFP injected into locus coeruleus. For sham optogenetic stimulation, two TH-Cre Long-Evans rats were used. Either Cre-inducible pAAV5-EF1α-DIO-ChETA-EYFP, pAAV5-EF1α-DIO-EYFP, CAV2-PRS-ChR2-mCherry, or pAAV5-CaMKII-ChETA-EYFP virus was injected into locus coeruleus at 0.1 nl/s for a final injection volume of 1.2–1.5 μl. For behavioral experiments, a calibrated optical fiber ferrule was then implanted in locus coeruleus, and the craniotomy and implant was sealed with silicone sealant and dental cement. For electrophysiology, the craniotomy was seal with silicone sealant for access after viral expression. For behavioral and electrophysiology experiments, virus was allowed two weeks for expression.
At the end of behavioral or electrophysiology experiments, animals were perfused with 4% paraformaldehyde, brains recovered, and embedded in Optimal Cutting Temperature compound prior to freezing at −80 °C. Afterwards, 15 μm thick slices were cut from the brainstem and stained using standard immunohistochemistry histological methods. Staining for tyrosine hydroxylase (primary antibody 1:1000, Aves Labs catalog number TYH; secondary antibody, DYL488 anti-chicken, 1:500, Life Technologies Labs) was co-localized with YFP (Abcam #ab290).
Behavior
The behavioral task used here was similar to that we used previously (Carcea et al., 2017; Froemke et al., 2013; King et al., 2016; Martins and Froemke, 2015). Animals were trained on a go/no-go task to nosepoke in response to a target tone frequency for a food reward in 9" x 10" x 12” operant conditioning chambers (Med Associates, Inc.). Each chamber contained a speaker (on the right wall) calibrated across frequencies at 70 dB SPL, a food dispenser on the left wall and three nosepoke ports (two on either side of the food dispenser and one on the wall opposite). Each chamber was placed in a larger wood enclosure and insulated with foam. The measured background noise in each chamber was <30–40 dB SPL.
18 adult female Long-Evans and 2 adult female Sprague-Dawley rats were used in these behavioral studies. Animals were food restricted to maintain the weights at 80–85% of their initial pre-training weights. First, animals were shaped with two days of training to nosepoke for one food pellet. Next, rats were trained to nosepoke within 2.5 seconds after a target tone was played. When the rats had hit rates of >80%, three non-target tones were introduced (2–16 kHz at one octave intervals excepting the target frequency), and animals were trained to hit rates >90%, along with false positive rates <40%. Finally, the non-target tones were expanded to six total (0.5–32 kHz at one octave intervals excepting the target frequency), and animals were trained to the same criteria. Target and non-target pure tones were 100 ms in duration presented in a pseudo-random order at 70 dB SPL. For correct trials, each trial ended at either the time of food pellet delivery (hit trials for targets) or 2.5 s after the tone (correct reject trials for non-targets). On error trials, failure to respond (miss trials for targets) as well as incorrect responses (false alarm trials for non-targets) were punished with a time-out of 7 s before the next trial began. Random nose pokes were punished with time-out as well. Rats self-initiated the trials by nosepoking in a different port than the ‘response’ port. After 0.5–1.5 s, either a target or non-target tone was played.
Animals that achieved criterion behavioral performance on the baseline task with the target tone of 4 kHz underwent surgery as described above, had optical fibers chronically implanted in left locus coeruleus, and were allowed to recover for about a week. At this point, animals were retrained on the baseline task (target tone 4 kHz) until original performance on the task was achieved. Starting on the first day of reversal learning, the new target tone (16 kHz) was paired with activation of the locus coeruleus with blue light. For optogenetic stimulation, locus coeruleus-tone pairing was conducted at a rate of 3 Hz, for 5–10 minute daily prior to behavioral testing. Specifically, tones were played at 3 Hz, and optogenetic stimulation of locus coeruleus began at tone onset. Tone duration was 100 ms and locus coeruleus optogenetic stimulation was 10 Hz, 10 ms pulses, 500 ms duration. The tone duration reflects that used in the behavioral context. The pairing protocol was continued until behavioral performance returned to baseline levels or at least seven days.
Behavioral performance was estimated with hit rate measurements (percent of times the rats respond to the target frequency) and the discriminability index d’ (the difference in the z-scores for the distribution of responses to targets and for the distribution of responses to non-targets). d' values were computed as the difference in z-scores between hits and false positives: d' = z(hit rate) – z(false positive rate). Uninjected animals and the sham stimulated animal reversed from 4 kHz to 16 kHz (two octaves up in frequency) were used in analysis of the one-octave versus two-octave reversals as well as in the analysis of the locus coeruleus pairing versus control animals. Unless otherwise noted, all statistics and error bars are reported as means±s.e.m. although normality was not formally tested for all data sets, and p-values determined from Student's paired or unpaired two-tailed t-tests.
Electrophysiology
Experiments were carried out in a sound-attenuating chamber. Two wild-type Long-Evans animals injected in locus coeruleus with pAAV5-CaMKII-ChETA-EYFP and one TH-Cre Long-Evans animals injected in locus coeruleus with pAAV5-EF1α-DIO-ChETA-EYFP were used. After at least two weeks of viral expression, the silicone sealant was removed, the craniotomy was re-opened in the same location over locus coeruleus, and position was re-verified by recording responses to tail pinch. An optrode was then placed. The optrode was constructed from a 10 mm long, 200 μm diameter optic fiber and a 0.5 MΩ tungsten electrode. The tungsten electrode was oriented such that the tip was 0.4–0.5 mm below the tip of the optic fiber. Once locus coeruleus was localized through multiunit recordings as described in the methods, the tip of the tungsten electrode portion of the optrode was advanced to the identified coordinates, and optically evoked responses were confirmed (Fig. 6B). A craniotomy was then performed over the left temporal lobe and the left auditory cortex was exposed. Pure tones (70 dB SPL, 0.5–32 kHz, 50 msec, 3 msec cosine on/offramps) were delivered in pseudo-random sequence at 1 Hz. AI location was determined by mapping multiunit responses 500–700 μm below the surface using tungsten electrodes.
In vivo multi-unit recordings from AI were made with a Multiclamp 700B amplifier (Molecular Devices). Recordings were obtained from 500–1000 μm below the pial surface. For locus coeruleus pairing, after recording baseline multi-unit activity responses to the pseudo-random tone sequence, a non-preferred tone of a given intensity level and frequency was repetitively presented for 10min, concurrent with locus coeruleus optogenetic stimulation (500 ms, 10 Hz, 1–3 mW, 10 ms pulse) starting at tone onset. Locus coeruleus stimulation was then ceased and pseudo-random tone sequences were resumed. After 60–120 minutes, an additional pairing with the original paired tone was repeated, followed again by pseudo-random tone sequences. This paradigm was continued as long as cortical responses were viable. For analysis of tuning curve shifts, neural d’ was calculated. These was computed as the difference in the z-score of the d’ of the paired frequency and the average of the z-scores of the non-paired frequencies.
Statistics
Unless otherwise noted, all statistics and error bars are reported as means±SEM s.e.m. although normality was not formally tested for all data sets, and all p-values determined from Student's paired or unpaired two-tailed t-tests. For behavioral comparisons, t-tests were conducted on days 1–40 unless otherwise stated.
Highlights.
Rats go through stereotyped behavioral epochs when reward contingency is switched.
Locus coeruleus pairing accelerates learning new reward associations.
Cortical receptive fields shift with similar dynamics as behavioral changes.
Acknowledgments
We thank Tony Pickering for providing the CAV2-PRS-ChR2-mCherry virus, Susan Sara for technical advice on targeting locus coeruleus for stimulation and recording, Elena Vazey for advice on use of viruses, and K. Kuchibhotla, K.A. Martin, and M. Semerkant for comments, discussions, and technical assistance. This work was supported by a Vilcek Scholar Award [to E.G.]; the Portuguese Foundation for Science and Technology [to A.R.O.M]; a Katowitz/Radin NARSAD Young Investigator award [to I.C.]; a Hirschl/Weill-Caulier Career Award [to R.C.F.]; a Sloan Research Fellowship [to R.C.F.]; and the National Institutes of Health [grant number K99/R00-MH106744 to I.C., R01-DC003937 to M.A.S., and R01-DC012557 to R.C.F.]. Partial support was also received from a research contract from Cochlear Americas to J. Thomas Roland, Jr., M.D.
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- Amaral DG, Sinnamon HM. The locus coeruleus: neurobiology of a central noradrenergic nucleus. Prog Neurobiol. 1977;9:147–96. doi: 10.1016/0301-0082(77)90016-8. [DOI] [PubMed] [Google Scholar]
- Aston-Jones G, Bloom FE. Activity of norepinephrine-containing locus coeruleus neurons in behaving rats anticipates fluctuations in the sleep-waking cycle. J Neurosci. 1981;1:876–86. doi: 10.1523/JNEUROSCI.01-08-00876.1981. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Aston-Jones G, Rajkowski J, Kubiak P, Alexinsky T. Locus coeruleus neurons in the monkey are selectively activated by attended stimuli in a vigilance task. J Neurosci. 1994;14:4467–4480. doi: 10.1523/JNEUROSCI.14-07-04467.1994. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Aston-Jones G, Rajkowski J, Kubiak P. Conditioned responses of monkey locus coeruleus neurons anticipate acquisition of discriminative behavior in a vigilance task. Neuroscience. 1997;80:697–715. doi: 10.1016/s0306-4522(97)00060-2. [DOI] [PubMed] [Google Scholar]
- Aston-Jones G, Cohen JD. An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance. Annu Rev Neurosci. 2005;28:403–50. doi: 10.1146/annurev.neuro.28.061604.135709. [DOI] [PubMed] [Google Scholar]
- Bao S, Chan VT, Merzenich MM. Cortical remodelling induced by activity of ventral tegmental dopamine neurons. Nature. 2001;412:79–83. doi: 10.1038/35083586. [DOI] [PubMed] [Google Scholar]
- Bear MF, Singer W. Modulation of visual cortical plasticity by acetylcholine and noradrenaline. Nature. 1986;320:172–6. doi: 10.1038/320172a0. [DOI] [PubMed] [Google Scholar]
- Berridge CW, Arnsten AF, Foote SL. Noradrenergic modulation of cognitive function: clinical implications of anatomical, electrophysiological and behavioural studies in animal models. Psychol Med. 1993;23:557–64. doi: 10.1017/s0033291700025332. [DOI] [PubMed] [Google Scholar]
- Berridge CW, Waterhouse BD. The locus coeruleus-noradrenergic system: modulation of behavioral state and state-dependent cognitive processes. Brain Res Brain Res Rev. 2003;42:33–84. doi: 10.1016/s0165-0173(03)00143-7. [DOI] [PubMed] [Google Scholar]
- Bouret S, Sara SJ. Reward expectation, orientation of attention and locus coeruleus-medial frontal cortex interplay during learning. Eur J Neurosci. 2004;20:791–802. doi: 10.1111/j.1460-9568.2004.03526.x. [DOI] [PubMed] [Google Scholar]
- Bouret S, Sara SJ. Network reset: a simplified overarching theory of locus coeruleus noradrenaline function. Trends Neurosci. 2005;28:574–82. doi: 10.1016/j.tins.2005.09.002. [DOI] [PubMed] [Google Scholar]
- Butter CM. Perseveration in Extinction and in Discrimination Reversal Tasks Following Selective Frontal Ablations in Macaca Mulatta. Physiology & Behavior. 1969;4:163. [Google Scholar]
- Carcea I, Insanally MN, Froemke RC. Dynamics of auditory cortical activity during behavioural engagement and auditory perception. Nat Commun. 2017;8:14412. doi: 10.1038/ncomms14412. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Carter ME, Yizhar O, Chikahisa S, Nguyen H, Adamantidis A, Nishino S, Deisseroth K, de Lecea L. Tuning arousal with optogenetic modulation of locus coeruleus neurons. Nat Neurosci. 2010;13:1526–33. doi: 10.1038/nn.2682. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Carter ME, Brill J, Bonnavion P, Huguenard JR, Huerta R, de Lecea L. Mechanism for Hypocretin-mediated sleep-to-wake transitions. Proc Natl Acad Sci U S A. 2012;109:E2635–44. doi: 10.1073/pnas.1202526109. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Castro-Alamancos MA. Role of thalamocortical sensory suppression during arousal: focusing sensory inputs in neocortex. J Neurosci. 2002;22:9651–5. doi: 10.1523/JNEUROSCI.22-22-09651.2002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Christie MJ, Williams JT, North RA. Electrical coupling synchronizes subthreshold activity in locus coeruleus neurons in vitro from neonatal rats. J Neurosci. 1989;9:3584–9. doi: 10.1523/JNEUROSCI.09-10-03584.1989. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Christie MJ, Jelinek HF. Dye-coupling among neurons of the rat locus coeruleus during postnatal development. Neuroscience. 1993;56:129–137. doi: 10.1016/0306-4522(93)90568-z. [DOI] [PubMed] [Google Scholar]
- Chudasama Y, Robbins TW. Dissociable contributions of the orbitofrontal and infralimbic cortex to Pavlovian autoshaping and discrimination reversal learning: Further evidence for the functional heterogeneity of the rodent frontal cortex. Journal of Neuroscience. 2003;23:8771–8780. doi: 10.1523/JNEUROSCI.23-25-08771.2003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Constantinople CM, Bruno RM. Effects and mechanisms of wakefulness on local cortical networks. Neuron. 2011;69:1061–8. doi: 10.1016/j.neuron.2011.02.040. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Costa VD, Tran VL, Turchi J, Averbeck BB. Reversal learning and dopamine: a bayesian perspective. J Neurosci. 2015;35:2407–16. doi: 10.1523/JNEUROSCI.1989-14.2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Devauges V, Sara SJ. Activation of the noradrenergic system facilitates an attentional shift in the rat. Behav Brain Res. 1990;39:19–28. doi: 10.1016/0166-4328(90)90118-x. [DOI] [PubMed] [Google Scholar]
- Devilbiss DM, Waterhouse BD. Norepinephrine exhibits two distinct profiles of action on sensory cortical neuron responses to excitatory synaptic stimuli. Synapse. 2000;37:273–82. doi: 10.1002/1098-2396(20000915)37:4<273::AID-SYN4>3.0.CO;2-#. [DOI] [PubMed] [Google Scholar]
- Devilbiss DM, Page ME, Waterhouse BD. Locus ceruleus regulates sensory encoding by neurons and networks in waking animals. J Neurosci. 2006;26:9860–72. doi: 10.1523/JNEUROSCI.1776-06.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Disney AA, Aoki C, Hawken MJ. Gain modulation by nicotine in macaque v1. Neuron. 2007;56:701–13. doi: 10.1016/j.neuron.2007.09.034. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Doya K. Metalearning and neuromodulation. Neural Netw. 2002;15:495–506. doi: 10.1016/s0893-6080(02)00044-8. [DOI] [PubMed] [Google Scholar]
- Edeline JM, Manunta Y, Hennevin E. Induction of selective plasticity in the frequency tuning of auditory cortex and auditory thalamus neurons by locus coeruleus stimulation. Hear Res. 2011;274:75–84. doi: 10.1016/j.heares.2010.08.005. [DOI] [PubMed] [Google Scholar]
- Escanilla O, Arrellanos A, Karnow A, Ennis M, Linster C. Noradrenergic modulation of behavioral odor detection and discrimination thresholds in the olfactory bulb. Eur J Neurosci. 2010;32:458–68. doi: 10.1111/j.1460-9568.2010.07297.x. [DOI] [PubMed] [Google Scholar]
- Foote SL, Freedman R, Oliver AP. Effects of putative neurotransmitters on neuronal activity in monkey auditory cortex. Brain Res. 1975;86:229–42. doi: 10.1016/0006-8993(75)90699-x. [DOI] [PubMed] [Google Scholar]
- Froemke RC, Carcea I, Barker AJ, Yuan K, Seybold BA, Martins AR, Zaika N, Bernstein H, Wachs M, Levis PA, Polley DB, Merzenich MM, Schreiner CE. Long-term modification of cortical synapses improves sensory perception. Nat Neurosci. 2013;16:79–88. doi: 10.1038/nn.3274. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Froemke RC. Plasticity of cortical excitatory-inhibitory balance. Annu Rev Neurosci. 2015;38:195–219. doi: 10.1146/annurev-neuro-071714-034002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Froemke RC, Schreiner CE. Synaptic plasticity as a cortical coding scheme. Current Opinion in Neurobiology. 2015;35:185–199. doi: 10.1016/j.conb.2015.10.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- German DC, Walker BS, Manaye K, Smith WK, Woodward DJ, North AJ. The human locus coeruleus: computer reconstruction of cellular distribution. J Neurosci. 1988;8:1776–88. doi: 10.1523/JNEUROSCI.08-05-01776.1988. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gu Q. Neuromodulatory transmitter systems in the cortex and their role in cortical plasticity. Neuroscience. 2002;111:815–835. doi: 10.1016/s0306-4522(02)00026-x. [DOI] [PubMed] [Google Scholar]
- Hasselmo ME, Sarter M. Modes and models of forebrain cholinergic neuromodulation of cognition. Neuropsychopharmacology. 2011;36:52–73. doi: 10.1038/npp.2010.104. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hickey L, Li Y, Fyson SJ, Watson TC, Perrins R, Hewinson J, Teschemacher AG, Furue H, Lumb BM, Pickering AE. Optoactivation of locus ceruleus neurons evokes bidirectional changes in thermal nociception in rats. J Neurosci. 2014;34:4148–60. doi: 10.1523/JNEUROSCI.4835-13.2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hirata A, Aguilar J, Castro-Alamancos MA. Noradrenergic activation amplifies bottom-up and top-down signal-to-noise ratios in sensory thalamus. J Neurosci. 2006;26:4426–36. doi: 10.1523/JNEUROSCI.5298-05.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hwang DY, Carlezon WA, Isacson O, Kim KS. A high-efficiency synthetic expression selectively promoter that drives transgene in noradrenergic neurons. Human Gene Therapy. 2001;12:1731–1740. doi: 10.1089/104303401750476230. [DOI] [PubMed] [Google Scholar]
- Ishimatsu M, Williams JT. Synchronous activity in locus coeruleus results from dendritic interactions in pericoerulear regions. J Neurosci. 1996;16:5196–204. doi: 10.1523/JNEUROSCI.16-16-05196.1996. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Judge PG, Evans DW, Schroepfer KK, Gross AC. Perseveration on a reversal-learning task correlates with rates of self-directed behavior in nonhuman primates. Behav Brain Res. 2011;222:57–65. doi: 10.1016/j.bbr.2011.03.016. [DOI] [PubMed] [Google Scholar]
- King J, Shehu I, Roland JT, Jr, Svirsky MA, Froemke RC. A physiological and behavioral system for hearing restoration with cochlear implants. J Neurophysiol. 2016;116:844–58. doi: 10.1152/jn.00048.2016. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kuchibhotla KV, Gill JV, Lindsay GW, Papadoyannis ES, Field RE, Sten TA, Miller KD, Froemke RC. Parallel processing by cortical inhibition enables context-dependent behavior. Nat Neurosci. 2017;20:62–71. doi: 10.1038/nn.4436. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Levitt P, Moore RY. Noradrenaline neuron innervation of the neocortex in the rat. Brain Res. 1978;139:219–31. doi: 10.1016/0006-8993(78)90925-3. [DOI] [PubMed] [Google Scholar]
- Li Y, Hickey L, Perrins R, Werlen E, Patel AA, Hirschberg S, Jones MW, Salinas S, Kremer EJ, Pickering AE. Retrograde optogenetic characterization of the pontospinal module of the locus coeruleus with a canine adenoviral vector. Brain Res. 2016;1641:274–90. doi: 10.1016/j.brainres.2016.02.023. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Manella LC, Petersen N, Linster C. Stimulation of the Locus Ceruleus Modulates Signal-to-Noise Ratio in the Olfactory Bulb. J Neurosci. 2017;37:11605–11615. doi: 10.1523/JNEUROSCI.2026-17.2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Manunta Y, Edeline JM. Noradrenergic induction of selective plasticity in the frequency tuning of auditory cortex neurons. J Neurophysiol. 2004;92:1445–63. doi: 10.1152/jn.00079.2004. [DOI] [PubMed] [Google Scholar]
- Martins AR, Froemke RC. Coordinated forms of noradrenergic plasticity in the locus coeruleus and primary auditory cortex. Nat Neurosci. 2015;18:1483–92. doi: 10.1038/nn.4090. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Navarra RL, Clark BD, Gargiulo AT, Waterhouse BD. Methylphenidate Enhances Early-Stage Sensory Processing and Rodent Performance of a Visual Signal Detection Task. Neuropsychopharmacology. 2017;42:1326–1337. doi: 10.1038/npp.2016.267. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nieuwenhuis S, Aston-Jones G, Cohen JD. Decision making, the P3, and the locus coeruleus-norepinephrine system. Psychol Bull. 2005;131:510–32. doi: 10.1037/0033-2909.131.4.510. [DOI] [PubMed] [Google Scholar]
- Reil JC. Untersuchungen über den Bau des grossen Gehirns im Menschen. Arch Physiol (Halle) 1809;9:136–524. [Google Scholar]
- Roberts MJ, Thiele A. Spatial integration and its moderation by attention and acetylcholine. Front Biosci. 2008;13:3742–59. doi: 10.2741/2963. [DOI] [PubMed] [Google Scholar]
- Roussel B, Buguet A, Bobillier P, Jouvet M. Locus coeruleus, sommeil paradoxal, et noradrenaline cérebrale. C R Seances Soc Biol Fil. 1967;161:2537–41. [PubMed] [Google Scholar]
- Russell GV. The nucleus locus coeruleus (dorsolateralis tegmenti) Tex Rep Biol Med. 1955;13:939–88. [PubMed] [Google Scholar]
- Sara SJ, Devauges V. Priming stimulation of locus coeruleus facilitates memory retrieval in the rat. Brain Res. 1988;438:299–303. doi: 10.1016/0006-8993(88)91351-0. [DOI] [PubMed] [Google Scholar]
- Sara SJ. The locus coeruleus and noradrenergic modulation of cognition. Nat Rev Neurosci. 2009;10:211–223. doi: 10.1038/nrn2573. [DOI] [PubMed] [Google Scholar]
- Steriade M. Synchronized activities of coupled oscillators in the cerebral cortex and thalamus at different levels of vigilance. Cereb Cortex. 1997;7:583–604. doi: 10.1093/cercor/7.6.583. [DOI] [PubMed] [Google Scholar]
- Swanson LW. The locus coeruleus: a cytoarchitectonic, Golgi and immunohistochemical study in the albino rat. Brain Res. 1976;110:39–56. doi: 10.1016/0006-8993(76)90207-9. [DOI] [PubMed] [Google Scholar]
- Usher M, Cohen JD, Servan-Schreiber D, Rajkowski J, Aston-Jones G. The role of locus coeruleus in the regulation of cognitive performance. Science. 1999;283:549–54. doi: 10.1126/science.283.5401.549. [DOI] [PubMed] [Google Scholar]
- Vazey EM, Aston-Jones G. Designer receptor manipulations reveal a role of the locus coeruleus noradrenergic system in isoflurane general anesthesia. Proc Natl Acad Sci U S A. 2014;111:3859–64. doi: 10.1073/pnas.1310025111. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wenzel J, Wenzel C. De Penitiori Structura Cerebri Hominis et Brutorum. Tübingen: 1812. [Google Scholar]
- Witten IB, Steinberg EE, Lee SY, Davidson TJ, Zalocusky KA, Brodsky M, Yizhar O, Cho SL, Gong S, Ramakrishnan C, Stuber GD, Tye KM, Janak PH, Deisseroth K. Recombinase-driver rat lines: tools, techniques, and optogenetic application to dopamine-mediated reinforcement. Neuron. 2011;72:721–33. doi: 10.1016/j.neuron.2011.10.028. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Yu AJ, Dayan P. Uncertainty, neuromodulation, and attention. Neuron. 2005;46:681–92. doi: 10.1016/j.neuron.2005.04.026. [DOI] [PubMed] [Google Scholar]