LEARNING STRATEGY REFINEMENT REVERSES EARLY SENSORY CORTICAL MAP EXPANSION BUT NOT BEHAVIOR: SUPPORT FOR A THEORY OF DIRECTED CORTICAL SUBSTRATES OF LEARNING AND MEMORY

Abstract

Primary sensory cortical fields develop highly specific associative representational plasticity, notably enlarged area of representation of reinforced signal stimuli within their topographic maps. However, overtraining subjects after they have solved an instrumental task can reduce or eliminate the expansion while the successful behavior remains. As the development of this plasticity depends on the learning strategy used to solve a task, we asked whether the loss of expansion is due to the strategy used during overtraining. Adult male rats were trained in a three-tone auditory discrimination task to bar-press to the CS+ for water reward and refrain from doing so during the CS− tones and silent intertrial intervals; errors were punished by a flashing light and time-out penalty. Groups acquired this task to a criterion within seven training sessions by relying on a strategy that was “bar-press from tone-onset-to-error signal” (“TOTE”). Three groups then received different levels of overtraining: Group ST, none; Group RT, one week; Group OT, three weeks. Post-training mapping of their primary auditory fields (A1) showed that Groups ST and RT had developed significantly expanded representational areas, specifically restricted to the frequency band of the CS+ tone. In contrast, the A1 of Group OT was no different from naïve controls. Analysis of learning strategy revealed this group had shifted strategy to a refinement of TOTE in which they self-terminated bar-presses before making an error (“iTOTE”). Across all animals, the greater the use of iTOTE, the smaller was the representation of the CS+ in A1. Thus, the loss of cortical expansion is attributable to a shift or refinement in strategy. This reversal of expansion was considered in light of a novel theoretical framework (CONCERTO) highlighting four basic principles of brain function that resolve anomalous findings and explaining why even a minor change in strategy would involve concomitant shifts of involved brain sites, including reversal of cortical expansion.

1. Introduction

That primary (“early”) sensory cortical fields are deeply involved in learning and memory is now well established. In contrast to traditional assumptions that primary sensory cortical fields function only as stimulus analyzers, associative learning is now known to specifically modify the representations of stimuli in animals and humans in the primary auditory (A1) (Scheich et al., 2011; Weinberger, 2011), somatosensory (S1) (Galvez, Weiss, Weible, & Disterhoft, 2006; Pleger, Blankenburg, Ruff, Driver, & Dolan, 2008), visual (V1) (Hager & Dringenberg, 2010; Miller et al., 2008), olfactory (Li, Howard, Parrish, & Gottfried, 2008) and gustatory (Ifuku, Hirata, Nakamura, & Ogawa, 2003) cortices. Most extensively studied in A1, learning can shift acoustic frequency tuning to strengthen the encoding of sounds that predict reinforcement (Bakin & Weinberger, 1990; Edeline & Weinberger, 1993; Kisley & Gerstein, 2001), which can also produce increased cortical representational area for a tone signal within the tonotopic “map” of A1 (Recanzone, Schreiner, & Merzenich, 1993; Rutkowski & Weinberger, 2005).

Highly specific learning-induced tuning shifts and increased area are instances of a previously unknown type of learning-dependent plasticity, “highly-specific associative representational plasticity”. (For brevity, hereafter we generally use simply “representational plasticity”.) Whereas “plasticity” is very widely applied to almost any instance of non-transient neural change, representational plasticity consists of systematic modification of the processing of a parameter of sound, e.g., acoustic frequency. A defining feature of representational plasticity is that it cannot be known as the animal is actively learning during training trials. Rather, it is detected by sensory neurophysiological testing in trained subjects with a wide range of stimulus values, outside of the training context. This procedure can reveal whether cortical plasticity is limited to a particular stimulus or is a manifestation of a global modification in the cortical processing of a stimulus parameter. The implications of representational plasticity transcend local plasticity because representational plasticity alters not merely responses to a current stimulus, but rather the processing of future stimuli along a sensory dimension. Furthermore, the amount of expansion of representational plasticity can encode both the acquired importance of sensory stimuli (Rutkowski & Weinberger, 2005) and the strength of specific memory (Bieszczad & Weinberger, 2010c).

Representational plasticity in A1 is ubiquitous as it develops across species (including humans), types of learning, varieties of tasks, motivational valences and other sound parameters (Scheich, Brechmann, Brosch, Budinger, & Ohl, 2007; reviewed in Weinberger, 1995, 2004, 2007). Further evidence implicating primary sensory cortex in mnemonic processes is that representational plasticity in A1 has the same attributes as important features of memory, e.g., associativity, specificity, rapid formation, consolidation and long-term retention (reviewed in Weinberger, 2007). Moreover, directly enhancing A1 responses to a tone, by pairing it with stimulation of the cholinergic nucleus basalis (NB) (Bakin & Weinberger, 1996; Kilgard & Merzenich, 1998), implants specific behavioral memory (McLin, Miasnikov, & Weinberger, 2002) that also shares the major attributes of natural memory (Miasnikov, Chen, & Weinberger, 2006, 2011; Miasnikov & Weinberger, 2012; Weinberger, Miasnikov, & Chen, 2006) and does so by increasing its area of representation (Bieszczad, Miasnikov, & Weinberger, 2013).

Although representational plasticity is a reliable process that appears to favor behaviorally important sensory events, the factors that are responsible for its development during instrumental learning are not well understood. Learning strategy has been identified as an unexpectedly important influence (Berlau & Weinberger, 2008; Biesczcad Weinberger, 2010a, 2010b, 2010c). Thus, most tasks can be solved in more than one way and whether or not representational plasticity develops seems to depend not on whether or how well a task is learned, but rather on how it is solved. For example, there is no unique solution to the problem of obtaining water rewards contingent on making bar-presses in the presence of a tone, while withholding them during silent intertrial intervals to avoid an error signal (flashing light) that initiates a time-out “penalty” period. Although apparently a very simple task, different strategies can be employed because a tone has different components: an onset, a plateau (steady state) and an offset. The problem could be solved simply by starting to respond at tone onset and stopping at tone offset (“tone duration” strategy, ON–OFF). However, subjects could also obtain rewards by responding from tone onset, past tone offset until receiving an error signal (“tone-onset-to-error” strategy, TOTE).

Rats trained in this instrumental reward task learn to solve the problem regardless of whether they use the ON–OFF or TOTE strategies. However, representational plasticity, particularly an expanded representation of the CS+, develops in A1 only if animals use the TOTE strategy (Berlau & Weinberger, 2008; Biesczcad & Weinberger, 2010b). Indeed, use of the TOTE strategy is more critical for the formation of representational plasticity than is motivational level (Biesczcad & Weinberger, 2010a). Furthermore, the magnitude of the tone signal’s representation is a function of the extent to which animals use the TOTE strategy: the greater the use of TOTE, the greater the representational area (Bieszczad & Weinberger, 2010b).

Although learning strategy has been identified as an important factor for the development of representational plasticity, its role in the maintenance of representational plasticity is unknown. This is particularly important because learning-related representational expansions in A1 can diminish or completely disappear when training is continued, usually for weeks, after a task has been solved; this process that has been referred to as “renormalization” (Reed et al., 2011). This type of loss of learning-induced plasticity is a general and enigmatic process transcending the auditory system, e.g., visual cortex (Yotsumoto, Watanabe, & Sasaki, 2008), somatosensory cortex (Ma et al., 2010), motor cortex (Tennant et al., 2012). The goal of this experiment was to determine if the maintenance or loss of cortical representational plasticity is linked to the behavioral strategy employed in learning. Beyond the specific question at hand, the findings have extensive implications for a general theory of neural systems underlying learning and memory.

2. Methods

2.1. Subjects

The subjects were male Sprague–Dawley rats (300–325 g, n = 21) from Charles River Laboratories (Wilmington, MA). They were individually housed in a vivarium (temperature maintained at 22° C, 12/12 h light/dark cycle, lights on 7 am), with ad libitum access to food and water before the onset of training. During training with water restriction (see Section 2.3), continuous access to water was restored on the weekends and supplements were provided after training sessions to maintain weight, as necessary. All procedures were conducted with care to minimize pain or discomfort and were in accordance with the University of California, Irvine, Animal Research Committee and the NIH Animal Welfare guidelines.

2.2. Experimental groups and treatments

The main goal of this experiment was to determine if the maintenance or loss of cortical representational plasticity is linked to the behavioral strategy employed during learning. As the loss of expanded representation has been reported in cases of prolonged training after subjects had initially learned to solve an instrumental task (e.g., Reed et al., 2011), we studied the effects of three different amounts of such overtraining.

First, animals were divided into three groups and were trained on the same three-tone discrimination task (3TD) to the same criterion. Specifically, they were trained to bar-press for water reward contingent on the presence of a CS+ tone, and not to press during presentation of either of two CS− tones (Low CS− and High CS−) (see Section 2.3.2.2). Training continued until each subject reached criterion, defined as three consecutive sessions during which its coefficient of variation (CV) for performance (P, see also Section 2.4.1) was ≤ 0.10 (CV = standard deviation/mean of daily performance level). Second, they received different amounts of continued training after reaching criterion. Group ST (n = 11) received no additional training. Group RT (n = 6) continued to be trained for one week (5 sessions), and then underwent a two-week retention period prior to further treatment. Group OT (n = 10) was given more extensive overtraining of three weeks. Third, to determine the frequency specificity of learning, all groups underwent a stimulus generalization test after they completed training (or retention in the case of Group RT) (see Section 2.3.2.3). (The only exception to this sequence is that Group RT underwent a single training session after its retention interval to determine how well it remembered the task.) Fourth, all groups underwent mapping of the frequency representation of their primary auditory cortices (see Section 2.4). The RT group experienced the same temporal separation between attainment of criterion and physiological mapping as the OT group (t₍₁₄₎ = 1.00, p = 0.33). Treatment phases and their sequences are summarized in Fig. 1.

Fig. 1.

Experimental phases and sequence (not to temporal scale for readability). All animals received identical treatment prior to and during tone training up to the day they attained criterion on the 3TD task. They were shaped to bar-press for water reward (4 days) prior to the start of training. Next, each animal was trained to a criterion of three consecutive sessions during which its coefficient of variation (CV) for performance (P) (see Sections 2.2 and 2.4.1) was ≤ 0.10 (CV = standard deviation/mean of daily performance level). Subjects in the ST group received no overtraining after reaching criterion and then underwent stimulus generalization testing followed by physiological mapping 2–3 days later. The RT group received one additional week (5 sessions) of training after which they were returned to their home cages for two weeks. At the end of this period, they received a single 3TD session to determine if they had retrained the task solution. The next day they underwent stimulus generalization testing then physiological mapping 2–3 days later. The OT group was overtrained for three weeks and then underwent stimulus generalization testing and then physiological mapping 2–3 days later.

2.3. Behavioral training

2.3.1. Training apparatus and stimuli

The training apparatus and equipment used to generate auditory stimuli were the same as those used previously (Bieszczad & Weinberger, 2012). Training was conducted in two, identical, sound-attenuated instrumental conditioning chambers (H10-11R; Coulbourn Instruments, Whitehall, PA). Daily sessions were counterbalanced between the two chambers to ensure equal exposure to both contexts. They were fitted on one wall with a bar manipulandum and a water cup attached to a lever (H14-05R) that could deliver 0.02 ml of water to a small port 9 cm to the left of the bar (H21-03R), a speaker (H12-01R) positioned 13 cm above the reward port, and an overhead house light (H11-01R). Each chamber was enclosed in a larger sound-attenuating chamber (H10-24A). During all training stages, depression of the bar manipulandum, i.e., bar-press, triggered the water cup to be present in the port. The cup remained in place for 5 s to allow complete reward consumption, during which time additional BPs were ineffective. Thus, only one reward could be obtained every 5 s.

All tones were generated using Tucker–Davis Technologies (TDT, Alachua, FL) System 3 (RP2.1 Enhanced Real-Time Processor) and RPvdsEx software. Tones were always 10 s in duration (70 dB sound pressure level [SPL]) and cosine-squared gated with a rise/fall time (10–90%) of 10 ms. Tone levels for all frequencies used in training were calibrated for three locations in the training chambers at animal head height and set to average 70 ± 2 dB SPL.

2.3.2. Training and testing protocols

2.3.2.1. Handling and bar-press shaping

Upon arrival, all subjects were acclimated to the vivarium for 1–2 days after which they underwent 3 days of handling to familiarize them with the experimenter and movement to and from the laboratory. Subjects were then water-restricted to ~85% body weight of unrestricted litter controls and were taught to bar-press (BP) for water reward (1:1 ratio) over four consecutive, daily, 60 min sessions. Auditory training began 24 h after the final bar-press training session.

2.3.2.2. Three-tone discrimination task (3TD)

The three-tone discrimination task required that animals bar-press during the CS+ tone (5.0 kHz) and not bar-press during presentation of either of two CS− frequencies, located 1.25 octaves lower (2.1 kHz, Low CS−) and higher (11.8 kHz, High CS−), respectively. Two CS− tones were used to avoid the possibility that animals would solve the discrimination aspect of the task by categorically responding to the higher or lower of two tones.

A trial began with the presentation of a 10 s tone. Each session consisted of ~70% CS+ trials and ~30% CS− trials, the latter divided approximately equally between the two CS− frequencies. Intertrial intervals (ITI) were silent and averaged 20 s (range = 12–28 s). Bar-presses during a CS+ produced a water reward in which the dipper was available for 5 s. Thus, animals could receive a maximum of two rewards per CS+ tone. Bar-presses made during CS− trials and/or the silent intertrial interval period triggered a time-out period (extension of the intertrial interval) signaled by flashing the house light. The duration of the error signal matched the duration of the time-out period for bar-presses made during silence while the error signal resulting from bar-presses made during a CS− triggered the flashing light error signal for the remaining duration of the tone.

As explained above (see Introduction), expanded representation in the auditory cortex develops when subjects use the TOTE strategy, i.e., begin bar-pressing at the onset of the CS+ tone and continue responding after its offset, until receiving an error signal (flashing light and time-out penalty); hence, called “Tone-Onset-To-Error” (TOTE) strategy. To promote use of the TOTE strategy, we modified the basic training protocol to provide an additional reward for bar-presses made during the first 7 s of silence starting immediately after offset of the CS+. We refer to this interval as the “free period”. Animals were rewarded for a bar-press during this free period if they had made a rewarded bar-press during the CS+ of that trial. Thus, a maximum of 3 rewards were available during CS+ trials: up to two rewards during the tone presentation and one more during the post-tone free period. This protocol reliably produces an expanded representation of the CS frequency in the auditory cortex (Berlau & Weinberger, 2008; Bieszczad & Weinberger, 2010a, 2010b). Fig. 2 shows details of this protocol and four exemplar cases that include illustrations of the free period reward contingency. For example, Case 1 met these criteria and therefore the subject received three rewards. However, it also made four errors: a second bar-press during the free period and three presses during the silent intertrial interval. Case 2 differs from Case 1 despite the fact that the animal also received three rewards, but the free period reward was obtained by a bar-press near the end of the free period. The difference between the cases is that Case 2 made only one error instead of four errors (Case 1); it made a single response during the intertrial interval. Training sessions of 60 min were conducted five days per week until animals attained asymptotic performance (see Section 2.4.1).

Fig. 2.

Four examples of different behaviors on CS+ trials, illustrating reward and “punishment” (flashing light error signal and 7 s time-out) contingencies. Bar-presses made during the CS+ triggered a 5 s water reward, allowing a maximum of two rewards during the 10 s CS+. The first BP made during the free period was rewarded, contingent on receiving at least one prior reward during the CS+ on that trial. The free period lasted for a maximum of 7 s after tone offset; a second BP constituted an error. All BPs made during the silent intertrial intervals (mean duration of 20 s, range 12–28 s, pseudo-randomly scheduled) were errors. Case #1: The rat received the maximum number of three rewards but also made errors. It made a second BP before the end of the 7 s free period and three BPs during the ITI. Case #2: The animal also received three rewards, the last of which was obtained by a BP before the end of the free period. It also made an error by a BP immediately after consuming the water reward during the ITI. Case #3: This subject received two rewards, one during the CS+ and one during the free period, which was available because it had received the prior CS+ reward on that trial. Case #4: This animal made a single BP during the free period. Although the latency of its BP was the same as for the subject in Case #1, it was not rewarded but treated as an error because the subject had not received a prior reward on this trial.

2.3.2.3. Stimulus generalization

All animals underwent a stimulus generalization test session to determine the frequency-specificity of learning. Seven different frequencies were tested, which included the three frequencies used during training as well as four other frequencies, spectrally similar to the CS+ and CS− frequencies, that were never presented during training (1.4, 2.1 [CS−], 3.2, 5.0 [CS+], 7.8, 11.9 [CS−] and 18.3 kHz; all at 70 dB). This session began with twenty standard training trials (14 CS+ and 6 CS− trials, randomly intermixed) to insure performance was still stable immediately prior to test. Test frequencies were presented in 140 trials in a pseudo-random order totaling 20 trials for each frequency. The generalization test was conducted under extinction conditions, i.e., no rewards were given, to avoid confounding effects of reward presentation during the test.

The number of BPs for each of the seven test frequencies throughout the 140 extinction trials (20 trials per frequency) was tallied and expressed as a percentage of the total number of responses to all frequencies to control for different levels of baseline responding.

2.4. Behavioral analyses

Behavioral analyses were conducted in three domains: (a) daily performance, including discrimination learning (“macroanalysis”); (b) trial-by-trial within each training session, to determine the strategy used on each trial (“microanalysis”); (c) specificity of frequency learning, based on post-training stimulus generalization gradients.

2.4.1. Task performance

Overall performance (P) was based on bar-presses made during all three tones (CS+, Low CS−, High CS−), and during the silent intertrial intervals. (BPs during the free period were excluded as they indexed neither tone-induced behavior nor errors, but were merely inducements to continue responding after tone offset.) Daily session performance level was based on the rate of bar-presses during the CS+ tone and the CS− tones, calculated as:

$$P=\frac{{\text{BPR}}^{\text{CS}+}}{{\text{BPR}}^{\text{CS}+}+{\text{BPR}}^{\text{ER}}}-(\frac{{\text{BPR}}^{\text{ER}}}{{\text{BPR}}^{\text{CS}+}+{\text{BPR}}^{\text{ER}}}\times \frac{{\text{BPR}}^{\text{CS}+}}{{\text{BPR}}^{\text{CS}+}+{\text{BPR}}^{\text{ER}}})\times 100$$

where BPR^CS+ was defined as the bar-press rate during CS+ presentations (i.e., BPR^CS+ = number of BPs during the CS+ tone presentation, divided by the 10 second duration of CS+ tone), and BPR^ER, was the error rate, defined as the sum of the BP rate during CS− presentations and BP rate during the silent ITI period. Thus, BP rates were calculated for each session as follows: BPR^CS+ = (# BPs during CS+ tones)/(total amount of time during CS+ periods); BPR^ER = (# BPs during CS− and silent ITI periods)/(total amount of time during CS− and silent ITI periods). With this measure, performance consisting of all BPs made only to the CS+ would yield a value of 100 and BPs only during CS− trials or only during ITIs (or both) would yield a value of zero. Thus, P provides an overall measure of learning to respond to the CS+ and learning not to response to the Low CS− and the High CS−.

Although the P measure includes responses to both the CS+ and the two CS− tones, it does not provide detailed information on the level of discrimination between each of the CS− and the CS+. To obtain this information, we used the standard discriminability index, d-prime (d′) based on signal detection theory (Green & Swets, 1966; see Talwar & Gerstein, 1999 for application to rat discrimination). The d′ is the difference between the z-normalized Hit and False Alarm rates. A Hit (H) is defined as a trial on which at least one bar-press was made during the CS+ tone; a False Alarm (FA) is defined as a trial on which at least one bar-press was made during either the Low CS− or High CS− tones. The Hit rate was calculated as the percentage of CS+ H trials (total H trials divided by total number of CS+ trials in a session; the False Alarm rate was calculated at the percentage of CS− trials (total FA trials divided by the total number of CS− trials in a session). Separate d′ values were computed for the two CS− tones (Low CS− and High CS−) to determine if subjects discriminated them the same or differently from the CS+.

2.4.2. Tracking the use and refinement of strategy

A central issue in this study is the examination of how animals solve the task, not simply if they do. We characterized each animal’s pattern of behavior on a trial-by-trial basis to quantify behavioral strategy during each session, and examine strategy use across days throughout training. This analysis was confined to successful trials, i.e., trials on which at least one reward was obtained; this restriction follows from our goal of determining how, rather than if, animals solved the task. Fig. 3 provides examples of the three main patterns of behavior that occurred during training.

Fig. 3.

Example trials categorized as Non-Stop (NS), Tone-Onset-to-Error (TOTE) or internalized TOTE (iTOTE) strategies. Note that the critical difference between TOTE and iTOTE is that the latter lacks BP errors made during the intertrial interval (see Section 2.4.2).

The first pattern, which was prevalent early in training, was termed “Non-Stop” (NS). It was defined as making one or more BPs during all three trial periods, i.e., during the tone, during the free period and during the intertrial interval. A defining characteristic of NS is the production of repeated errors that trigger a chain of many error signals and time-outs. This pattern occurred mainly during the first few sessions and probably reflects the preceding bar-press “shaping” period during which no tones were present.

The second pattern, “Tone-Onset-To-Error” (TOTE) was identified previously (Berlau & Weinberger, 2008; Bieszczad & Weinberger, 2010b, 2010c). Similar to NS, TOTE is also characterized by one or more BPs during the tone, free and ITI periods. However, in stark contrast, BPs were temporally organized. They started near tone onset, often with a second rewarded BP ~5 s later. To be classified as TOTE, ITI BPs (triggering the error signal and timeout) had to be made within the first 5 s of the ITI after a rewarded BP during the free period. TOTE behavior developed early in training and increased during the criterion period as NS behavior diminished.

The final pattern, “internalized Tone-Onset-To-Error” (iTOTE) is distinct from TOTE in that BPs (errors) were not made after the rewarded period (tone plus free period). This pattern was observed primarily after TOTE had been established and intensified during overtraining. It is termed “internalized” TOTE on the hypothesis that the animals came to internalize the contingencies related to rewards and errors, and no longer used the error signal/time-out to indicate lack of reward. Rather, they used the first post-tone BP that was not rewarded to stop bar-pressing, thus avoiding the error signal and time-out penalty. iTOTE can be viewed as a refinement of TOTE, producing optimization of behavior, i.e., more rewards in less time and with less effort.

Each trial was classified as an NS, TOTE or iTOTE trial. Some trials could not be classified according to these categories, e.g., a failure to BP during a CS+, while responding during the ITI, so were categorized as “Other”. Such trials occurred mainly early in training following bar-press shaping. Strategy use was calculated as the percentage of trials in a session that fell into each of these four categories (NS, TOTE, iTOTE and Other).

2.5. Neurophysiology

2.5.1. Neurophysiological recording

Complete mapping of A1 was performed 2–3 days following completion of the extinction/generalization session. Subjects were water deprived the night previous to surgery to help reduce salivary secretions. An additional group of untrained naïve animals (n = 6) was mapped as a comparison group to the trained groups to determine the effects of training on frequency representation in A1.

Methods were the same as previously reported (Bieszczad & Weinberger, 2012). Briefly, subjects were anesthetized (sodium pentobarbital, 0.1 ml/kg i.p., 55 mg/ml) with supplemental doses administered as necessary to maintain a state of areflexia. Atropine sulfate (0.2 ml i.m., 0.54 mg/ml) was administered to minimize bronchial secretions. Body temperature was maintained at 37 °C with the use of a homeothermic heating pad (Harvard Apparatus, Cambridge, MA), and ophthalmic ointment was applied to keep the eyes moist. Subjects were mounted in a stereotaxic frame (David Kopf Instruments, Tujunga, CA) inside a double-walled sound attenuated room (Industrial Acoustics, Bronx, NY). The scalp was resected after subcutaneous administration of lidocaine hydrochloride (AstraZeneca, Wilmington, DE). After clearing the calvaria, stainless steel screws were threaded into burr holes and cemented to a dental acrylic pedestal fixed to the frame. Once affixed to the frame the ear bars were removed leaving the ear canals unobstructed. A craniotomy was performed over the right auditory cortex and the cisterna magna drained of cerebrospinal fluid to reduce brain pulsation. The dura was resected and warm saline was applied frequently to prevent desiccation. Photographs of the cortical surface were taken using a digital camera prior to each recording. These images were aligned using vascular landmarks to construct a relative map of each recording site.

Stimuli were delivered to the contralateral ear using an open field speaker positioned 2–3 cm away from the ear canal. Stimuli consisted of pure tone bursts (50 ms, cosine-squared gate with 8 ms rise/fall time, 0.5–53.8 kHz in quarter-octave steps, 0–70 dB SPL in 10 dB increments, 8 presentations of each stimulus). Stimuli were presented, pseudo-randomly, once every 700 ms using a TDT RX6 Multifunction Processor controlled by custom MATLAB software.

Extracellular recordings were made with a linear array of four parylene-coated microelectrodes (1–2 MΩ, FHC, Bowdoin, ME) lowered to the middle cortical layers (III–IV; 400–600 μm deep) via a microdrive (Inchworm 8200, EXFO Burleigh Instruments, Victor, NY). Neural activity was amplified (TDT RA16 Amplifier, TDT RZ5 Bioamp Processor) and stored for offline analysis. Offline spike detection was performed using custom MATLAB software. Recordings were filtered (0.3–3.0 kHz) and multiunit discharges were characterized using temporal and amplitude criteria. Acceptable spikes were defined as waveforms with peaks separated by no more than 0.6 ms and with a threshold amplitude greater than 2.0 (for the positive peak) and less than 2.5 (for the negative peak) × RMS of 500 random traces from the same recording.

2.5.2. Neurophysiological analyses of A1 frequency maps

Analysis of tone-evoked responses in A1 was the same as reported previously (Biesczcad & Weinberger, 2012). Briefly, tone-evoked activity was determined by subtracting the spontaneous firing rate (recorded during the 50 ms period prior to tone onset) from the firing rate during stimulus presentation. The mean evoked activity for each stimulus was used to construct frequency response areas (FRAs) for each recording site. Each FRA threshold was based upon its recorded spontaneous activity; only evoked responses greater than the mean ± 2 se of spontaneous activity were considered true evoked responses. The FRAs were used in determination of the characteristic frequency (CF) for each site, defined as the stimulus frequency having the lowest threshold for an evoked response. If multiple frequencies were found having the same lowest threshold, the CF was defined as the geometric mean between these frequencies. Voronoi tessellations were constructed for all recording sites sampled during mapping. The primary auditory cortex (A1) was then physiologically defined and those tessellations were selected to determine CF area. A1 was identified as having a caudal–rostral, low–high frequency tonotopic organization. Borders with the anterior (AAF) and ventral auditory fields (VAF) were identified through reversals in frequency tuning. The caudal border was identified by discontinuities in the tonotopic gradient while the dorsal border was identified by discontinuities in the tonotopic gradient as well as the presence of sites with multi-peaked FRAs and broad (> 3 octaves) tuning (Doron, LeDoux, & Semple, 2002; Polley, Read, Storace, & Merzenich, 2007; Rutkowski, Miasnikov, & Weinberger, 2003; Sally & Kelly, 1988). Frequency area was quantified by determining the percentage of total A1 area occupied for each frequency band.

2.6. Statistics

All behavioral and neural comparisons between groups were made using ANOVA (α = 0.05) and paired t-tests where applicable, as described in the text. Post hoc analyses were performed using t-tests with Bonferroni α correction for multiple comparisons (e.g., to determine the frequency-specificity of differences between groups). Brain–behavior correlations were assessed using Pearson’s r statistic.

3. Results

3.1. Task Performance (P)

All three groups attained criterion, and did so after the same amount of training: ST, 6.27 ± 0.45 sessions; RT, 6.5 ± 0.47; OT, 6.5 ± 0.59 (mean ± sem) (1-way ANOVA, Group, F_(2,24) = 0.07, p = 0.93). Although groups attained criterion at the same time, they might have differed in their criterion levels of performance. However, there was no significant difference in P on the criterion day (Group, F_(2,24) = 0.98, p = 0.39) (Fig. 4). Moreover, while P increased across sessions (2-way ANOVA, Session, F_(2,168) = 72.68, p < 0.001), group performances did not differ from the start of training to the criterion day (Group, F_(2,168) = 0.49, p = 0.61) or interact with sessions across that period (Session × Group, F_(12,168) = 0.41, p = 0.96).

Fig. 4.

Average daily task performance (P metric) for all groups. Note that all three groups show similar performance leading up to criterion and that the RT and OT groups show no differences during their common week of overtraining after attaining criterion. As individual animals showed different rates of attaining criterion, daily performance is depicted here only for days where at least 75% of each group’s total N is present. Solid lines depict group mean while shaded areas depict ± sem.

Performance continued to increase during overtraining for both RT and OT groups. We compared P for the criterion session vs. their last training session, which was three weeks later. For the RT group, the last training session was the retention session that was run after two weeks without training (see Section 2.2) while the OT group had continuous training during this period. P significantly increased for both groups (RT, t₍₅₎ = 5.69, p < 0.01; OT, t₍₉₎ = 8.72, p < 0.001). The gain in performance was greater for the OT group (t₍₁₄₎ = 2.26, p < 0.05), perhaps due to their continuous training during the two-week period when Group RT was not trained. However, Group RT did show good retention, as there was no difference between performance on their retention test vs. their last training session two weeks previously (t₍₅₎ = 1.02 p = 0.36) (Fig. 4).

3.2. Discrimination performance (d’)

Discrimination performance for each group during learning to criterion is shown in Fig. 5. A 3-way ANOVA (Group × Session × Frequency) showed that d′ increased leading up to criterion (Session, F_(6,336) = 31.15, p < 0.001), and that groups improved at comparable rates, there being no differences in the Group (F_(2,336) = 1.01, p = 0.37) or Group × Session factors (F_(12,336) = 1.56, p = 0.10) (Fig. 5A). Furthermore, there was no difference in the groups’ discrimination between the CS+ and the two CS− tones as indicated by a lack of effect for Frequency (F_(1,336) = 0.15, p = 0.70), or any of its interactions: Group × Frequency (F_(2,336) = 0.87, p = 0.84), Frequency × Session (F_(6,336) = 0.46, p = 0.84), Group × Session × Frequency (F_(12,336) = 0.78, p = 0.67). An analysis of the criterion session for each animal also failed to reveal any group differences: 2-way ANOVA, Group (F_(2,48) = 0.01, p = 0.99), Frequency (F_(1,48) = 2.74, p = 0.10), Group × Frequency (F_(2,48) = 0.32, p = 0.73) (Fig. 5B and 5C, “Criterion”).

Fig. 5.

Discrimination performance over days measured with d′. Animals in all groups learned to discriminate the CS+ from the two CS− frequencies equally well during initial training up to criterion (A). At criterion there were no differences among the groups in their ability to discriminate the CS+ from the Low CS− (B) or the High CS− (C). The two extra weeks of training experienced by the OT group caused them to have significantly better discrimination performance compared to RT. Solid lines in A depict mean while shaded areas depict ± sem. Bars in B and C depict mean ± sem. **p < 0.01; ***p < 0.001.

Overtraining of one (RT) and three (OT) weeks produced further gains in d′, but no differences between Low CS− and High CS−; Group RT: Session (F_(2,30) = 11.96, p < 0.001), CS− Frequency (F_(1,30) = 0.02, p = 0.89), Session × CS− Frequency (F_(2,30) = 0.66, p = 0.52); Group OT: Session (F_(1,36) = 91.19, p < 0.001), CS− Frequency (F_(1,36) = 0.11, p = 0.30), Session × CS− Frequency (F_(1,36) = 2.62, p = 0.11). As was seen for task performance, the magnitude of change was greater for the OT group that received more overtraining than the RT group (Low CS−, t₍₁₄₎ = 4.42, p < 0.001; High CS−, t₍₁₄₎ = 3.21, p < 0.01) (Fig. 5B and 5C, “Final”).

3.3. Dynamics of the use of strategies

The use of behavioral strategies is a major focus of this study. Therefore, we tracked the three behavioral patterns during learning to criterion (sessions 1–7): NS, TOTE and iTOTE (see Section 2.4.2). Recall that “non-stop bar-pressing” (NS) involves bar-presses made throughout a trial, without particular regard for tone onset or the error signal. In contrast, the “tone-onset-to-error” (TOTE) strategy is characterized by the initiation of bar-presses by onset of a tone, usually producing two rewards during a CS+ and a third reward during the “free period” immediately after tone onset; BPs continue into the intertrial interval, where they trigger a flashing light error signal and a time-out, i.e., delay to onset of the next trial. Finally, “internalized TOTE” (iTOTE) consists of cessation of bar-pressing after the rewarded period (tone plus free period). This avoids triggering the error signal and time-out penalty so that subjects received the next trial more quickly and thus had more opportunities to obtain reward in the given timed session (Fig.3).

Analyses of each of the three behavioral patterns revealed changes in the prevalence of strategy use (Session factor) use but no differences among groups. For TOTE, Session was significant (F_(6,168) = 43.62, p < 0.001), but both Group and its interaction with Session were not significant: Group (F_(2,168) = 2.88, p = 0.06); Session × Group (F_(12,168) = 0.19, p = 0.99) (Fig. 6B). NS showed a similar outcome: Session was significant (F_(6,168) = 8.25, p < 0.001) but neither the Group (F_(2,168) = 0.07, p = 0.94) nor the Session × Group interactions were significant (F_(12,168) = 0.98, p = 0.47) (Fig. 6A). Finally, iTOTE also changed in the same manner: Session (F_(6,168) = 3.74, p < 0.005), Group (F_(2,168) = 0.88, p = 0.42), Session × Group (F_(12,168) = 0.62, p = 0.83) (Fig. 6C). Thus TOTE, NS and iTOTE behavioral strategies changed in their prevalence during training to criterion but Groups ST, RT and OT did not differ among themselves.

Fig.6.

Strategy use over days leading up to criterion. Strategy use changed in a consistent way during training and all groups showed the same dynamics. (A) NS use was used more frequently than either TOTE or iTOTE during early training sessions (1–3) after which point NS use began to decline and TOTE use increased markedly and was more frequent than NS or iTOTE in sessions leading up to criterion (6–7) (B). All groups used NS (D), TOTE (E) and iTOTE (F) in equal amounts at criterion. Solid lines in A–C depict mean while shaded area depicts ± sem. Bars in D–F depict mean ± sem.

Notably, the magnitudes of changes were markedly different. TOTE use increased to about 60–70% of trials at criterion (Fig. 6E). NS increased and then decreased so that at criterion, it accounted for ~20% of trials (Fig. 6D). In stark contrast, iTOTE increased only to about 10% of the trials (Fig. 6F). At criterion, animals used TOTE significantly more than either NS (t₍₅₂₎ = 9.14, p < 0.001) or iTOTE (t₍₅₂₎ = 15.51, p < 0.001). While NS use had decreased by criterion, it was still used more frequently than iTOTE (t₍₅₂₎ = 5.58, p < 0.001). Overall, by criterion, animals had learned to solve the 3TD task by mainly adopting the TOTE strategy.

Most importantly, strategy use changed during the post-criterion period. In particular, Group OT, which underwent three weeks of overtraining, developed a shift from the use of TOTE to the use of iTOTE. Fig. 7 shows exemplar behavior from one subject from the first to the last training session. Note that while TOTE use clearly increased earlier in training, iTOTE quickly came to dominate behavior so that it constituted the vast majority of trials well before the completion of overtraining (Fig. 7).

Fig. 7.

Example behavior from one animal in the OT group across all training sessions. On the left the animal’s BP behavior during all CS+ trials is presented as a raster plot; a cumulative bar graph depicting proportional strategy use for each session is presented on the right. All trials in the raster plot are aligned with CS+ tone onset at 0 s. Horizontal dotted lines demarcate the boundaries between daily training sessions while the dashed vertical lines at 0, 10 and 17 s denote tone-onset, tone-offset and the maximum duration of the free period, respectively. Note that there are no BPs between −5 and 0 s. This is because the animal was required to cease BPing during the intertrial interval (ITI) in order for the next trial to begin. Note that the first five sessions are noticeably thinner (briefer) than the other sessions; during the early training sessions it was common for animals to make numerous BPs during the ITI extending it up to 5 min on some occasions. As each daily session lasted 60 min, these extended ITI periods resulted in the animal experiencing fewer trials during these days compared to later in training when it learned to restrict its ITI BPs. Note the shift from the use of TOTE to iTOTE strategies. The precise timing of three BPs during the CS and free periods reflects cueing by the onset of the tone (first BP) and the withdrawal of the water cup after 5 s (second and third BPs).

Group OT’s strategy use across all training sessions is presented in Fig. 8A. Notice the sequential prevalence of the three strategies. First, NS quickly increased to about 50% of trials by sessions 3 and 4 and then began to decline. It was then supplanted by TOTE, which peaked at approximately 70% of trials at or near criterion (session 7). Finally, iTOTE started to increase just prior to criterion and continued thereafter. It became more frequent than TOTE on session 14 and attained a level of almost 80% of trials at the conclusion of training.

Fig.8.

Strategy use after criterion. (A) Average daily strategy use for the OT group. Note how NS and TOTE show similar profiles of increased use up to a peak and then gradual decreased use, albeit on different time scales, while iTOTE only increases in use over time. The OT group was trained for three weeks after attaining criterion, while the RT group received one week of training past criterion and then received no further training for the following two weeks. At the end of this retention period the RT group underwent a single 3TD training session to test their memory for the task. The RT group displayed a significantly different profile of strategy use at this point compared to the OT group. While both groups used NS in equal, minimal, amounts (B) the RT group used TOTE to a greater extent than the OT group (C), while the OT group relied on iTOTE significantly more than the RT group did (D). During the final training session the RT group relied on TOTE and iTOTE equally. In contrast, the OT group relied on iTOTE significantly more than TOTE (see Section 3.3). To determine whether iTOTE use was dependent or independent of prior TOTE, use we examined iTOTE use relative to the session of maximum TOTE use for each animal in the OT group. This alignment revealed that iTOTE use increased only after the session with maximum TOTE use indicating that iTOTE use was dependent on prior use of TOTE and reflects a refinement of the TOTE strategy. Solid lines in A and E depict mean while shaded area depicts ± sem. Bars in B–D depict mean ± sem. *p < 0.05; ***p < 0.001.

Group analyses validated the shift from TOTE to iTOTE during overtraining. Comparison of the criterion day vs. the final day of training three weeks later showed a significant increase in the use of iTOTE, from 4.70% (± 2.16 sem) to 77.25% (± 2.78), paired t-test (t₍₉₎ = 21.61, p < 0.001). Similar analysis showed a significant decrease in the percent of trials on which TOTE was employed: 65.20% (± 7.16) to 15.41% (± 2.56), paired t-test (t₍₉₎ = 7.82, p < 0.001). NS also declined: 27.66% (± 6.82) to 6.05% (± 2.13), paired t-test (t₍₉₎ = 3.43, p < 0.01).

An estimate of how rapidly these opposite changes in strategy can achieve significance can be obtained from the data for Group RT, which had only one week (five sessions) of training beyond criterion. Comparisons of its criterion day vs. its last training day (5 sessions later) showed a significant increase in the percent of trials on which iTOTE was employed: 6.24% (± 2.81) to 29.23% (± 8.53), paired t-test (t₍₅₎ = 2.84, p < 0.05). However, decline of the TOTE strategy had not yet reached statistical significance: 58.51% (± 8.08) to 35.12% (± 11.0), paired t-test (t₍₅₎ = 1.44, p = 0.21). The decrease in NS behavior for Group RT did attain significance by the end of the overtraining week: 25.43% (± 3.48) to 11.58% (± 4.10), paired t-test (t₍₅₎ = 3.69, p < 0.05).

We also compared strategy use for Groups OT and RT on their final training sessions, immediately prior to stimulus generalization and neurophysiological mapping (OT, last day of training; RT, memory test after two weeks of no training). While the two groups both increased their use of iTOTE, they still demonstrated significantly different profiles during their final training sessions: Group (F_(1,42) = 4.02, p = 0.05), Strategy (F_(2,42) = 68.62, p < 0.002), Group × Strategy (F_(2,42) = 29.62, p < 0.001). At the end of training both groups used NS very little but in equal measure (t₍₁₄₎ = 1.30, p = 0.22) (Fig. 8B). The RT group relied more on TOTE use (35.12% ± 11.0) than did the OT group (15.41% ± 2.56) (t₍₁₄₎ = 2.25, p < 0.05) (Fig. 8C). In contrast, OT used iTOTE strategy (77.25% ± 2.78) more than did RT (29.23% ± 8.51) (t₍₁₄₎ = 7.09, p < 0.001) (Fig. 8D). Thus, although the two groups had the same profile of strategy use at criterion (Fig. 6), at the time of mapping they differed in preferred ways to solve the 3TD task.

3.4. Dynamics of strategy refinement: the transition from the TOTE to the iTOTE strategy

To better understand the dynamics of strategy use during training, we sought to determine how iTOTE became the main strategy with overtraining. One possibility is that iTOTE develops independent of and possibly parallel with TOTE, e.g., at a slower rate. Alternatively, iTOTE may develop as a refinement of TOTE, where its emergence as the dominant strategy with overtraining is dependent upon an animal’s prior use of the TOTE strategy. The observation that iTOTE use remained negligeable in the sessions leading up to criterion argues for iTOTE emerging as a refinement after criterion (Fig. 5C). However, that the RT group significantly increased its iTOTE use without significantly decreasing its TOTE use during its one week of overtraining suggests that the two strategies might develop independently.

While both of these possibilities can explain high levels of iTOTE use after overtraining, they have contradictory predictions about its development during the period at which TOTE use is predominant. If iTOTE develops independently of TOTE, then its use could well increase prior to the point where TOTE use happens to be at its apex. However, if iTOTE develops as a refinement of TOTE, then it should increase only after TOTE use starts to decline, i.e., begins to replace TOTE use. To distinguish between these possibilities, we analyzed the prevalence of iTOTE use relative to the session at which TOTE use was highest. This analysis was restricted to Group OT because it alone used iTOTE as the dominant strategy and therefore was the only group suitable for determining how iTOTE became the dominant strategy.

We determined for each OT animal the session on which it attained its highest level of TOTE use. We then plotted its levels of TOTE and iTOTE use aligned relative to when this “apex” session occurred, and averaged across the group. The results are presented in Fig. 8E. Note that iTOTE did not begin to increase until the amount of TOTE use began to decline. There was no significant change in the use of iTOTE during the sessions leading up to the peak in TOTE use (Session, F_(9,27) = 1.29, p = 0.29). Instead, an increase in the use of iTOTE only occurred in the sessions after the peak of TOTE use (F_(9,27) = 4.52, p < 0.01). These results show that increasing use of iTOTE depends on the prior use and beginning of decline of TOTE. They support the conclusion that iTOTE is a refined version of TOTE rather than an independent strategy.

3.5. Specificity of learning: frequency generalization gradients

All groups underwent a stimulus generalization test after training to determine the specificity of frequency learning. The generalization gradients for all three groups are depicted in Fig. 9. They are highly specific, and similar. A two-way ANOVA showed that all groups learned about Frequency (F_(6,168) = 161.14, p < 0.001), but there were no differences between Groups (F_(2.168) = 0.018, p = 1.0) and their interaction was not significant (Frequency × Group, F_(5,105) = 0.85, p = 0.53). Therefore, while there was specificity of frequency discrimination learning, it did not differ among Groups ST, RT and OT.

Fig. 9.

The specificity of each animal’s memory for frequency was tested via stimulus generalization during extinction following their final training session. No significant differences were observed between any of the groups. The generalization test used seven frequencies, including the CS+, Low CS− and High CS−. Gradients were sharp with the vast majority of responses made to the CS+ or to its adjacent higher frequency (7.7 kHz, 0.63 octaves away). Few responses were made to the CS− or other tones. There were no significant differences between groups.

3.6. Representational area in the primary auditory cortex after learning and overtraining

Physiological mapping of A1 was performed to determine whether learning strategy was related to representational plasticity in early sensory cortex. Fig. 10A shows exemplar maps from each experimental group as well as a representative map from a naïve animal (see Section 2.5.2). The mean relative areas for the CS+ frequency band (5.0 kHz ± 0.3 octaves) and the CS− frequency bands (Low CS− = 2.1 kHz ± 0.3 octaves and High CS− = 11.8 kHz ± 0.3 octaves) are also presented for each group.

Fig. 10.

(A) Example tonotopic maps from subjects in the ST, RT and OT groups as well as from a Naïve group. There were no differences in representational area for either the Low CS− (2.1 kHz) (B) or High CS− (11.8 kHz) (D) frequency bands. In contrast, representational area for the CS+ was significantly larger for the ST and RT groups compared to both the OT and Naïve groups, which did not differ from each other (C).

We found a significant group effect for the CS+ (one-way ANOVA, F_(3,32) = 4.72, p < 0.01) but not for either of the CS− tones (Low CS−: F_(3,32) = 0.79, p = 0.51; High CS−: F_(3,32) = 1.67, p = 0.19). Post hoc analysis revealed that CS+ area was significantly greater than naïve in both the ST (t₍₁₈₎ = 2.94, p < 0.01) and RT groups (t₍₁₃₎ = 2.18, p < 0.05) but not in the OT group (t₍₁₇₎ = 0.41, p = 0.69). Furthermore, both the ST and RT groups developed significant expansions of the CS+ area compared with the OT group (ST: t₍₁₉₎ = 2.98, p < 0.001; RT: t₍₁₄₎ = 2.26, p < 0.05) but did not differ from each other (t₍₁₅₎ = 0.29, p = 0.78) (Fig. 10B–D). That the OT group was not significantly different from naïve animals indicates that the loss of representational plasticity occurred either during overtraining or due to the passage of time (three weeks of training) between criterion and mapping. However, the RT group, that had a CS+ expansion, had the same passage of time as the OT group. Therefore, the absence of increased area in the latter was not caused simply by time passing. Rather, it was experience dependent.

3.7. Learning strategies predict map reduction or absence of representational plasticity in A1

Group OT exhibited both a refinement of behavioral strategy from TOTE to iTOTE during its three-week period of overtraining and the absence of representational plasticity in primary auditory cortex. To determine if these findings were related, we compared individual animals’ use of TOTE and iTOTE during their final training session in the ST, RT and OT groups with their magnitudes of representational expansion of the CS+.

We found that TOTE use was positively correlated with the amount of CS+ area (r = 0.41, p = 0.03) (Fig. 11A). This relationship had been reported previously in a simpler, nondiscrimination task (Biesczcad & Weinberger, 2010b). More important for the present study, there was also a significant correlation for iTOTE use with CS+ representational area. However, it was negative: the greater the use of iTOTE, the smaller the CS+ area (r = −0.58, p < 0.01) Fig. 11B).

Fig. 11.

The relationship between strategy use during the final 3TD training session and the relative CS+ area for all subjects. (A) TOTE was significantly positively correlated with CS+ area, replicating and extended prior findings (Bieszczad & Weinberger, 2010c). (B) Most importantly, the use of iTOTE was significantly negatively correlated with CS+ area: the greater the use of iTOTE, the smaller the CS+ area. Thus, the loss of CS+ expansion by overtraining is attributable to a change in learning strategy. As strategy use was determined prior to mapping this indicates that it is predictive of CS+ area.

4. Discussion

4.1. A note on nomenclature

As noted in the Introduction, the loss of plasticity in A1 with continued training has been called “renormalization” (Reed et al., 2011). We have chosen not to use this term because it may carry the assumption that an involved area had returned to its naïve state when its plasticity is no longer observed. A reasonable alternative is that other cortical areas that become newly involved in strategy changes with overtraining may exert tonic inhibition on sites previously involved in the superseded strategy. In any event, we suggest, and will hereafter use, a simple descriptive term that incorporates the present findings: “reversal of plasticity by overtraining”.

Also, while we emphasize “cortical plasticity” given the current findings and for expository ease, we do not intend that sub-cortical sensory and brain systems be excluded from participation in any of the processes considered below, as they have also been implicated in learning-related plasticity (Kraus, Skoe, Parbery-Clark, & Ashley, 2009; Kraus, Strait, & Parbery-Clark, 2012).

4.2. Overview of the findings

4.2.1. Summary of the findings

This study concerns the loss of learning induced highly-specific representational plasticity in primary sensory (auditory) cortex when training is continued after a task has been solved. We sought to clarify the role of such overtraining in the loss of plasticity. Given prior findings, we focused on the potential causative role of a change or refinement of learning strategy during overtraining.

Adult male rats were trained in a three-tone auditory discrimination task (CS+ = 5.0 kHz, Low CS− = 2.1 kHz, High CS− = 11.8 kHz) to bar-press to the CS+ for water reward and refrain from responding to the CS− tones and bar-pressing during silent intertrial intervals (Fig. 2). They solved this task within in an average of seven training sessions by relying on a strategy that was “bar-press from tone-onset-to-error signal” (“TOTE”). Three groups then received different levels of overtraining: Group ST, none; Group RT, one week; Group OT, three weeks. Post-training mapping of their primary auditory fields showed that Groups ST and RT had developed significantly expanded representational areas restricted to the frequency band of the CS+ tone. In contrast, Group OT showed no such representational plasticity and did not differ from naïve control animals (Fig. 10). The present finding of “reversal of plasticity by overtraining” replicates prior reports (Ma et al., 2010; Reed et al., 2011; Tennant et al., 2012; Yotsumoto et al., 2008) and extends the phenomenon to a three-tone discrimination task.

Importantly, trial-by-trial analysis of behavior (hereafter, “behavioral microanalysis”) showed that the OT group had shifted its mode of solving the task from TOTE to iTOTE. Moreover, the use of strategies was predictive of the area representing the CS+ tone. Regarding the development of representational plasticity, we found a significant positive correlation between the use of TOTE during training and the area of representation (Fig. 11A). This finding replicates prior results (Biesczcad & Weinberger, 2010c) and extends this relationship for the development and maintenance of specific representational plasticity to discrimination learning.

More critical for the present study is the relationship between strategy use and reversal of plasticity. We found that the use of iTOTE is negatively related to CS+ area: the greater the use of iTOTE, the smaller the CS+ area (Fig. 11B). Therefore, the current findings provide an explanation for the loss of specific plasticity in early sensory cortex after overtraining, i.e., a shift or refinement of learning strategy.

Although a change in how to solve the 3TD task has the marked effect of eliminating the expression of learning-induced expanded representation in primary auditory cortex, it was accomplished by a seemingly minor change in how the animals solved the task. The animals simply increased efficiency by self-terminating bar-pressing in the silent intertrial intervals, before they made a response that would have triggered the error signal (flashing light) and timeout penalty. Therefore, based on the apparently small change in behavior, the shift to iTOTE is more properly considered to be a refinement of TOTE rather than a completely new strategy. Why such an apparently minor shift in behavior could have such a substantial effect on cortical plasticity will be considered later (see Sections 4.3 and 4.4).

4.2.2. Alternative explanations to strategy refinement for reversal of plasticity in Group OT

Alternative explanations must be considered before concluding that the refinement of learning strategy during overtraining is responsible for the reversal of expanded representational plasticity. First, perhaps Group OT never had developed representational plasticity, so there was never any plasticity to be lost. This seems highly unlikely because all three groups had the same behavior and learning during training to criterion. They did not differ on how rapidly they achieved criterion, the magnitude of correct performance (P) on the criterion day, or the trajectory of performance leading up to criterion (Fig. 4). Moreover, they did not differ in the rate at which they learned the discrimination or their discriminability of both the Low CS− and the High CS− from the CS+. Nor was there any difference in d’ on the criterion day (Fig. 5). Second, perhaps the use of learning strategies was different for the OT group vs. the ST and RT groups leading up to criterion. However, analysis of the strategies showed no differences among groups (Fig. 6). Third, Group OT’s learning during its three weeks of overtraining may have been inferior to ST and RT. However, OT subjects demonstrated significantly higher performance than Group RT at the end of training. Fourth, the loss of representational expansion in Group OT might be attributable to the mere passage of time as it was trained for a longer period than Groups ST and RT. However, the interval between reaching criterion and the completion of training was equal for RT and OT, three weeks. Finally, Group OT might have learned something different about acoustic frequency than Groups ST and RT. This alternative can be rejected because there was no difference among groups in their stimulus generalization gradients.

In summary, as other explanations do not account for the findings, we conclude that the loss of representational expansion is attributable to the change in how this group solved the task during overtraining.

4.2.3. Concerning the amount of change in strategy necessary to produce reversal of plasticity

As this is the first study on the reversal of plasticity and learning strategy/refinement, it raises many questions such as the minimal effective amount of strategy change to produce a reversal of plasticity. Although there are not yet systematic data on this issue, the current findings from Group RT may help set some boundaries. These animals retained representational plasticity after one week of overtraining and two weeks of no training; in fact, they did not differ in A1 representation from Group ST that had no overtraining. This observation suggests that more than one week of overtraining is needed to produce a reversal of plasticity. However, this conclusion must be qualified because the one week of training was followed by a two-week period of no training, during which time representational plasticity might have become stronger through time-dependent consolidation. The strategy use of this group may be more helpful. Group RT developed a significant increase in its use of iTOTE during that one week of overtraining, from about 6% to 29% (paired t-test, t₍₅₎ = 2.64, p < 0.05) but its use of TOTE had not yet declined significantly (~59% to 35%, t₍₅₎ = 2.01, p = 0.10). Therefore, the reversal of expanded cortical representation may require sufficient overtraining to produce both a significant decrease in use of the strategy used to solve the task (TOTE) and a significant increase in another strategy (iTOTE) that replaces the original method of solving the task.

4.3. Implications for a theory of directed cortical plasticity in learning and memory

We introduce here the concept of “directed cortical plasticity”. It is intended to focus attention and research, beyond documenting that plasticity has developed, toward a fundamental explanation of why learning-based plasticity develops in some particular cortical fields and not others in a given situation. That is, the neural domain of interest should transcend a particular locus of observed plasticity during learning, which instead can be seen as an entry point into a relevant network. The current finding that a reversal of cortical plasticity occurs due to a shift of learning strategy strongly implies that the loss of observed expansion must involve other cortical fields or systems. One subtle alternative is that overtraining leads to the use of another neural code than an increase in firing rate to represent a significant stimulus. This would enable primary sensory cortices to be responsible for learning strategies as well as their sensory functions. The current approach is fundamentally different from formulations that assume adequate explanations can be found within the cortical field under observation (e.g., Reed et al., 2011; Yotsumoto et al., 2008) but is compatible with findings of learning-induced auditory plasticity within the interconnections of cortical and subcortical targets (Letzkus et al., 2011; Xiong, Znamenskiy, & Zador, 2015; Znamenskiy & Zador, 2013).

In the following sections, we provide a framework for understanding the seemingly strange current findings of reversal of plasticity during continued reinforcement and apparently the same instrumental behavior. More importantly, we attempt to initiate a broader conceptualization that ultimately will incorporate much of the research on the neural bases of learning and memory. We limit our focus here to instrumental learning situations, i.e., those in which reinforcement requires that subjects perform an instrumental act. We believe that the postulates discussed below also apply to Pavlovian situations of classical conditioning. However, the discovered importance of learning strategy requires a focus on instrumental learning. Non-instrumental forms of associative learning have little need of or opportunity for the formulation and use of learning strategies because the stimuli and reinforcers are presented without requiring that subjects make an instrumental response or select a particular means of solving the task.

4.3.1. The importance of knowing how subjects solve problems: directed cortical plasticity

The current findings on the loss of plasticity with continued training are complementary to the prior discovery that the acquisition or development of expanded CS representation in A1 also depends on the learning strategy employed. Specifically, use of TOTE produces CS+ expansion, the greater the use, the larger the expansion (Bieszczad & Weinberger, 2010c). Together, these findings validate the importance of learning strategy to conceptions and investigations of the neural substrates of learning and memory.

Heretofore, approaches have focused on only a few issues: (a) whether or not learning develops, (b) its temporal dynamics, and (c) the brain areas in which various correlative forms of neural plasticity are found. Two other important issues have received less attention than they deserve: (d) what was learned and (e) memory strength. To this list must now be added a previously ignored factor, (f) behavioral strategy, i.e., “how” the task or problem was solved.

An adequate systems level theory of brain and instrumental learning and memory needs to account for all the involved cortical areas and non-cortical structures. However, a deeply embedded assumption has been that when an animal learns and continues to perform an instrumental behavior, the same neural mechanisms or structures are involved. Consequently, the mere occurrence of the instrumental response (e.g., bar-press) is assumed to be sufficient to identify the involved cortical areas. But unless one asks “how” the subject is solving the task, the issue of a change in solution strategy does not arise. Consequently, the possibility of potential changes in the involved neural sites never arises. The current and prior reports of the importance of behavioral strategy render this assumption moot, if not untenable.

It is well to remember that a subject confronted by a new learning opportunity in any situation is likely to use whatever neural resources it has available. Even if the situation is dominated by one sensory modality, all experiences are multimodal and therefore several sensory systems will be engaged. Moreover, they have neural resources that far transcend sensory systems, i.e., the rest of the neuraxis including numerous cortical regions and fields, subcortical systems and structures, etc. Therefore, all brain systems that are engaged in the new situation may become involved in solving or resolving the situation for the subject. The same subject may alter the relative deployment of its neural resources one or more times during a given learning situation, i.e., change its learning strategy. Similarly, different subjects are free to “assemble” different neural components to deal with the current task. They are also free to construe the task differently. For example, given a two-tone discrimination problem, some rats will treat it as an absolute frequency situation, attending to and learning the actual spectral components of the CS+ and CS−; others will treat it as a relative discrimination, attend to and learn only which stimulus was higher and which was lower. That was our rationale for using a three-tone discrimination (see Section 2.3.2.2).

If memory storage occurs where the experiential information is processed (see Section 4.4), then the brain regions that develop representational or other experience based plasticity are likely to be those that are engaged in processing the experience in question. Therefore, one should not expect plasticity to be confined to the dominant sensory system and certainly should not be surprised to find that different subjects have a non-identical distribution of sites for learning-related plasticity. Greater commonality of areas involved would be expected when different subjects face the identical task and use the same learning strategy to solve the problem.

Collectively, answers to all of these “(a)–(f)” questions are needed to develop a general neural systems level theory of how information is acquired, represented, stored and ultimately used to guide behavior. There is now sufficient information, some of it counterintuitive, to begin this task. We present such an account in the next section. It is necessarily brief, given constraints of space.

4.4. Support for a model of directed cortical substrates of learning and memory

Here we show how a new model of directed cortical plasticity can uniquely account for the current findings that learning-induced cortical plasticity can decrease or disappear while the instrumental behavior with which it was correlated during acquisition remains intact.

4.4.1. “CONCERTO”: A scaffold for a comprehensive theory of brain, learning and memory

We introduce here aspects of CONCERTO, as the framework of a theory for brain, learning and memory. The complete CONCERTO model posits both system level and circuit-cell-molecule levels of explanation, aspects of which have been published elsewhere: e.g., neuromodulators/muscarinic receptors (Bieszczad et al., 2013; McLin et al., 2002; Miasnikov, Chen, & Weinberger, 2008; Weinberger et al., 2006; see also Edeline, 2012; Edeline, Manunta, & Hennevin, 2011), and neuronal synchrony and gamma band activity (Headley & Weinberger, 2011, 2013; Weinberger, Miasnikov, Bieszczad, & Chen, 2013). We are able to present here only major elements of its system level formulation. These have been selected to incorporate findings on the importance of learning strategy to understand the neural bases of learning and memory. We specifically focus on how four of its postulates can account for the five anomalies presented above, without necessarily contravening major views on learning, e.g., changes in synaptic strength, convergence of associated environmental representations, importance of the medial temporal lobe for major forms of memory, etc.

4.4.1.1. Postulates

We present four principles as the systems level cornerstones of CONCERTO. The following are stated informally but organized as postulates (P1–P4) for ease of exposition.

P1

Memories in the conventional sense are multi-sensory and can include information about bodily and emotional state as well as stimuli from major sensory systems: auditory, gustatory, olfactory, somatosensory/nociceptive and visual.

P2

The components of memory that become stored, even briefly, are stored wherever they are processed, particularly in the cerebral cortex. Thus, there are no privileged sites of memory storage, i.e., “higher” cortical areas are not more predisposed to memory storage than are “primary” sensory cortical fields. (Sensory system components that exhibit no moderate to long-term storage properties, e.g., the retina or cochlea, are not included as memory sites.)

P3

Anatomically and physiologically distinct regions of the brain, especially the cerebral cortex, are somewhat specialized to process certain patterns of their input. Such patterns are not restricted to modality-matching, e.g., visual patterns for the visual cortex. Rather, any region of the cerebral cortex is influenced by numerous sources even if one or two dominate. Of the totality of input at any given time, that portion which can be and is processed by the extant circuitry (responded to) will be subject to plasticity mechanisms during learning.

P4

The substrates of memories are networks of neural elements that form at or shortly after an experience, initially tying together many, if not most, of the elements of that experience. Memory networks may be of various sizes and complexities, and importantly, have varying degrees of overlap with networks representing other memories.

4.4.1.2. CONCERTO: Application of the postulates—Learning-induced cortical plasticity can decrease or disappear while the instrumental behavior with which it was correlated during acquisition remains intact

This finding is counter-intuitive from the standpoint of conventional assumptions about brain and learning, which themselves seem quite reasonable. Thus, a neural correlate of the learning of a behavior or solution of a task is generally thought to actually constitute a neural substrate, i.e., cause, of some aspect of that behavior or learning process, rather than be in parallel. This view can be supported by demonstrating a “predictive correlate”, plasticity that develops before the behavior under observation. The usual resolution suggested when the behavior continues in the absence of discernable cortical plasticity is that extensive training has rendered the underlying circuitry more efficient, so that fewer cells are needed to accomplish the original behavior. Such a mechanism has been proposed by Reed et al. (2011) for primary auditory cortex and Yotsumoto et al. (2008) for primary visual cortex. In short, the plasticity is still there, but it cannot be seen. The present finding that this process is due to a change in strategy renders that explanation incomplete at best.

This anomaly can be resolved by referring to P3, i.e., “distinct regions of the cortex are specialized to process certain patterns of their input”. As noted above, by “pattern” we refer to the totality of afferentation reaching a cortical zone, in this case A1. This includes input not merely from the auditory periphery via the subcortical auditory system but from all neural sources that engage A1 when a subject is in a particular situation, using a particular strategy, behaving in a particular way, and learning particular things.

When coupled with the present observations that a refinement of a learning strategy led to the demise of representational plasticity in A1, we would hypothesize that A1 no longer received the pattern that had been present when the TOTE strategy had been dominant. Actually, the reversal of plasticity is graded: the greater the use of iTOTE, the greater the loss of CS+ representation (Fig. 11B). As iTOTE increases, the use of TOTE decreases. The reduction of the TOTE pattern input to A1 would then produce the loss of the plasticity on which it was based. The initially formed plasticity might still be present, but under tonic inhibition, perhaps from cortical regions that were processing whatever constituted the iTOTE pattern of input. Such possibilities can be evaluated by obtaining simultaneous recordings from a large number of recording sites in behaving animals.

An evaluation of CONCERTO requires testable predictions as well as plausible candidates for inputs to A1 with TOTE that are lost as subjects shift to iTOTE. One such candidate is the error signal feedback that occurs with erroneous bar-presses made during the silent intertrial intervals. Indeed, A1 does exhibit just such feedback activity (Brosch, Selezneva, & Scheich, 2011). The current formulation also predicts that some other regions, whether cortical, subcortical or both, became engaged in iTOTE behavior, and developed concordant plasticity to support this behavioral strategy. One might assume that the sensorimotor striatum and infralimbic neocortex are likely candidates because they become particularly engaged when subjects refine sensory–motor tasks (Smith & Graybiel, 2013, 2014). However, the shift in strategy from TOTE to iTOTE cannot adequately be considered as a phase of optimization of sensory-motor learning. This shift may appear to be a minor change to investigators because the bar-pressing behavior remains the same. However, it is a major difference to the subjects because the control of the cessation of bar-pressing changed from an external cue (flashing light error signal) to an internal prediction that the next BP will produce the error signal and its consequent time-out. This is reasonably characterized as a “cognitive act”. Until we gain a better understanding of neural substrates of strategy formulation and control, it will be difficult to predict the location of critically involved cortical fields.

4.5. Extension of CONCERTO to other anomalous findings

There are many other anomalous findings in research on learning and memory. Space limitations constrain us to account for only four most relevant here.

4.5.1. Primary (“early”) sensory cortical fields develop plasticity that has the major characteristics of associative learning and memory

This observation does not fit with standard accounts of brain and learning because of the traditional and still dominant assumption that early sensory fields are purely sensory analyzers whose functions subserve only perception. This view is dominating also in light of traditional assumptions of systems-level consolidation, which proposes hippocampal plasticity is critical for new learning and only later do cortical regions become essential for long-term storage (Alvarez & Squire, 1994; Nadel, Winocur, Ryan, & Moscovitch, 2007; Smith & Squire, 2009). However, associative learning rapidly shifts receptive fields (RF) in primary auditory cortex to favor processing of specific signal tones within trials (Bakin & Weinberger, 1990), revealing an essential and immediate cortical role in learning and memory. Shifts of frequency (tonal) RFs are the basis of increased area of representation in A1 and share the major characteristics of behavioral learning: associativity, specificity, rapid development, consolidation and long term retention (reviewed in Weinberger, 2004).

This anomaly is resolved by P1 and P4. The former holds that information is stored wherever it is processed and that there are no privileged sites of memory storage. P4 posits that memories are stored in distributed, multimodal networks. Therefore, information stored in a primary sensory field would not be isolated, e.g., as alone encoding a sensory parameter that is experienced. Rather, even if that sensory storage were so limited and specific, the information would be part of a much larger network.

4.5.2. A task can be learned by some subjects in the absence of any detectable plasticity in a cortical area that develops correlative plasticity in other subjects

Rats trained to bar-press for water all solved the task but only some did so with the development of specific plasticity in A1; those that used the TOTE strategy (Berlau & Weinberger, 2008; Bieszczad & Weinberger, 2010a, 2010b). This anomaly can be explained by P3, which holds that areas of the cortex (or brain in general) are specialized for processing certain patterns of their input. As discussed below, primary sensory fields are multimodal. The particular pattern(s) generated when animals are using the TOTE strategy may be quite different from those generated when they use another strategy or even a refinement, i.e., iTOTE. Thus, primary auditory cortex may receive a “TOTE pattern” but not an “iTOTE pattern”. Reference to P2 is also highly relevant, as it asserts that information is stored where it is processed. Thus, animals using the TOTE strategy would activate primary auditory cortex in some complex, multimodal pattern, the results of which would be acted on and stored therein. Presumably, patterns emanating in the use of iTOTE would be processed and stored elsewhere, in site(s) from which recordings were not obtained in the present study.

4.5.3. Different findings in different species of animals are attributed to “species differences”

The attribution to species differences is commonplace when findings between different species are not the same, e.g., two-tone discrimination: monkey, plasticity is present (Recanzone et al., 1993) vs. cat, lack of plasticity (Brown, Irvine, & Park, 2004). Once again, P3 provides a potential explanation. It holds that as different cortical areas have various specializations to process their input patterns, the same area may or may not develop plasticity depending on whether or not it receives inputs that are involved in learning to solve the task. As P2 claims that mnemonic plasticity will develop where neurons are activated by relevant elements of an experience, it follows that the species differences may reflect the fact that monkeys and cats use different strategies.

4.5.4. Early sensory cortical fields are multi-sensory, i.e., can display highly specific responses to extra-modal stimuli, particularly after learning

Traditional accounts of cortical organization view auditory, somatosensory and visual cortices as unimodal structures, with inter-sensory processing localized to “higher” or “association” fields located between early sensory fields. However, recent research indicates that primary sensory fields are also multimodal (Ghazanfar & Schroeder, 2006; Pascual-Leone & Hamilton, 2001). Primary sensory fields may even achieve multisensory status as a function of associative learning. For example, primary auditory cortex in humans is responsive to specific silent visual scenes of speech (Ruytjens, Albers, van Dijk, Wit, & Willemsen, 2006) and there is multisensory integration of faces and voices in A1 of rhesus monkeys (Ghazanfar, Maier, Hoffman, & Logothetis, 2005). Multisensory integration in A1 can be formed during learning (Brosch, Selezneva, & Scheich, 2005) and in subsequent memory effects of multimodal sensory pre-conditioning (Headley & Weinberger, 2015).

Multi-modality status can also be explained by the postulates that memories are multi-sensory (P1) and that they are stored in distributed networks that incorporate all or much of the sensory content of an experience (P4).

4.6. Future directions

A change in how animals solved the 3TD problem accounts for overtraining induced reversal of cortical representational plasticity. This both solves one problem while substantially enlarging the domain of research on brain mechanisms of learning and memory. Instrumental behaviors or “behavioral acts” comprise an extensive and essential part of experience-dependent behavior because they encompass all instances in which animals and people act to modify their current and future situations. The centrality of this ability to the psychological and neurobiological sciences can hardly be over emphasized.

The discovery of the critical importance of learning strategy for cortical plasticity highlights a neglected factor in neural approaches to learning and memory. Indeed, it seems likely that without taking into account “how a task is solved”, it will be impossible to reach an adequate understanding of the neural substrates of learning, or at least instrumental learning. This complicates matters for the experimenter, but the problems need not be serious. The necessary data are, or should be available, i.e., all instrumental responses in an experiment are ordinarily recorded, even if not analyzed with respect to the solution of problems set for the subjects. Thus, retrospective analysis of past experiments can be done; hypotheses can be formulated and tested, as an easy means of incorporating learning strategy into an ongoing research program. Furthermore, in new studies, behavioral strategies can easily be controlled by the experimenter because either different protocols can be used to promote different approaches to problem solution (Berlau & Weinberger, 2008) or subjects can be overtrained so they will initiate a modification of their initial strategy, as in the present case.

Highlights.

• Learning expands the representation of a sensory signal in primary sensory cortex

• Extensive overtraining after task solution can eliminate the expanded representation

• Rats solved a three-tone discrimination using a strategy expanding CS+ representation

• Animals that changed strategy during overtraining lost their expanded representation

• Understanding cortical substrates of learning requires determining strategies used