Cortical neural responses to previous trial outcome during learning of a directional choice task

Abstract

The outcomes that result from previous behavior affect future choices in several ways, but the neural mechanisms underlying these effects remain to be determined. Previous studies have shown that the lateral (AGl) and medial (AGm) agranular areas of the rat frontal cortex are involved in the learning and selection of action. Here we describe the activity of single neurons in AGl and AGm as rats learned to perform a directional choice task. Our analysis shows that single-cell activity in AGl and AGm was modulated by the outcome of the previous trial. A larger proportion of neurons encoded the previous trial's outcome shortly after cue onset than during other time periods of a trial. Most of these neurons had greater activity after correct trials than after error trials, a difference that increased as behavioral performance improved. The number of neurons encoding the previous trial's outcome correlated positively with performance accuracy. In summary, we found that neurons in both AGl and AGm encode the outcome of the immediately preceding trial, information that might play a role in the successful selection of action based on past experience.

learning depends in large measure on adapting future actions based on previous outcome of either a success or a failure, which corresponds with either reward or no reward at the end of the previous trial. However, only recently have researchers started to address neural correlates of learning from this perspective. Earlier results show that several areas in the frontal lobe of primates appear to encode upcoming trial outcome. They include the prefrontal cortex (PFC), anterior cingulate cortex, ventral premotor cortex, and supplementary eye field (Carter et al. 1998; Pardo-Vázquez et al. 2008, 2009; Stuphorn et al. 2000; Schall et al. 2002; Roesch and Olson 2003), as well as other medial frontal and motor areas (Holroyd and Coles 2002; van Schie et al. 2004).

Neurophysiological evidence suggests that both the medial and lateral agranular frontal areas of rats, AGm and AGl, respectively, are involved in predicting trial outcome (Laubach et al. 2000) and in learning (Rioult-Pedotti et al. 1998; Laubach et al. 2000; Cohen and Nicolelis 2004; Huber et al. 2012). These findings are consistent with those showing that the rat's lateral agranular cortex (AGl), which is homologous to the primary motor cortex in primates, shows considerable neuronal plasticity (Kartje-Tillotson et al. 1985; Kleim et al. 1998; Sanes and Donoghue 2000; Nudo et al. 1990; Viaro et al. 2011) and encodes information about task context (Carpenter et al. 1999; Matsuzaka et al. 2007). The medial agranular cortex (AGm) is a well differentiated field from AGl. Even though it is suggested by some as a part of the prefrontal area (Uylings et al. 2003; Dalley et al. 2004), its dense and direct projection to spinal cord makes it look similar to one of the nonprimary motor areas but not prefrontal (Neafsey and Sievert 1982; Donoghue and Wise 1982; Passingham et al. 1988; Remple et al. 2007; Erlich et al. 2011). In addition to its neurons involved in predicting trial outcomes, lesions of AGm cause an increase in reaction time, which suggests an involvement in movement preparation (Smith et al. 2010). Additionally, AGm neurons encode the action selected before movement onset in a value-based directional choice task (Sul et al. 2011).

Recent studies (Narayanan and Laubach 2008; Histed et al. 2009; Genovesio et al. 2014) reported persistent firing rate modulation by previous trial outcome even into a new trial. While their observations were in prefrontal areas, published results reviewed above also point to AGm and AGl as candidates for encoding the outcome of previous trials and suggest that this information plays a role in improving performance during learning. Accordingly, to study neural correlates of experience-based learning, we monitored the activity of single neurons in AGm and AGl of rats, as they learned a directional choice task. We studied the progression of neuronal activity through the course of learning. We found that single-unit activity was associated with two task factors: directional choice (left vs. right) and previous trial outcome (success vs. failure).

MATERIALS AND METHODS

General.

Male Long-Evans rats (n = 12) were cared for in accordance with the guidelines set forth by the Institutional Animal Care and Use Committee at Arizona State University, which reviewed and approved the study. The animals weighed 50 g upon arrival into the laboratory. They were handled daily and started the pretraining stage of this experiment when they reached 300 g.

During the pretraining stage, the rats were placed in a Skinner box (Med Associate, St Albans, VT) where two light-emitting diodes (LEDs), one on each side, were placed right above two extensible paddles. It is equally likely to have one of the two LEDs turned on as the start of a new trial. The response paddle on the same side under the LED would extend 2.0 s later. Pressing the paddle within 30.0 s would turn off the LED and also lead to a 1-kHz reward tone and a sugar pellet reward. No action within the time allowance would automatically terminate the trial and lead to a punish tone of 12 kHz, no sugar pellet reward, and a 10.0-s timeout. The intertrial interval was 5.0 s.

The rats used in this study were pretrained for a mean of 72 (±36SD) sessions and reached 90% correct or higher accuracy. At the end of this stage of the experiment, the rats were not only proficient with the association between an LED light cue and the paddle, but they also had become skilled at paddle pressing.

After the pretraining stage and when the animals reached 400 g, they were implanted with microwire arrays. Upon recovery, the animals were placed in another Skinner box (Med Associate), and began learning the directional choice task described below, while single-unit neural signals and behavioral performance data were recorded simultaneously. One experimental session of about an hour was conducted per rat each day. Throughout the experiments, the rat was free to move inside the Skinner box. Of the 12 rats used in this study, six rats (rats C, D, E, F, G, and H) provided behavioral and single unit data; four rats (rats I, J, K, and L) yielded behavioral and video recordings; and two rats (rats A and B) provided behavioral, neural, and video recordings.

Surgery.

The surgical procedure was performed using aseptic techniques. Each rat was anesthetized by injection of the KXA mixture (10.0 mg/ml ketamine, 2.0 mg/ml xylazine, and 0.1 mg/ml acepromazine; 0.1 cc/100 g, administered intramuscularly). Heart rate and O₂ saturation were continuously monitored. Once into an anesthetized state, each rat was placed in a stereotaxic frame with ear bars inserted and front teeth latched. The surgical area was then draped. Skin and fascia were removed to expose bregma plus sufficient area for performing the craniotomy and placing anchoring screws. A craniotomy of ∼2 × 4 mm centered at 2.0 mm lateral and 3.0 mm rostral from bregma was performed. The 16-channel (2×8) microwire array [Tucker Davis Technology (TDT), Alachua, FL] was then lowered into the left hemisphere of each rat's brain. The tips of the microwire array were cut at 60°. The electrode row spacing was 500 μm (rats A, B, and H) or 375 μm (rats C, D, E, F, and G). The electrode column spacing was 500 μm. An acrylic head cap was fixed on the skull with three anchoring bone screws. Sutures were used, if needed. Two supplemental doses of KXA (0.05 cc/100 g) were provided with an ∼1-h separation during the procedure. A KX (20 mg/ml of ketamine and 3 mg/ml of xylazine in 0.9% NaCl solution) update (0.05 cc/100 g) was applied if needed after the two KXA supplements. Systemic antibiotics and analgesics were administered for 3 days after the surgery.

Directional choice task.

Five LEDs (1 in the center, 2 on each side) were located on the front panel of the box as shown in Fig. 1C. Hereafter, when we refer to a “light” we identify one LED that was illuminated while the others were not. There were three paddles (center, left, and right) that the rat could press.

Fig. 1.

Implant site and illustration of the behavioral task. A: implant site and hit map of the recorded units. A 16-channel (2×8) microwire array with electrode numbers labeled was implanted in the rat's left hemisphere centered at 3 mm AP, 2 mm ML from the bregma. The delineation of lateral (AGl) and medial (AGm) agranular was made according to the Rat Brain Atlas (Paxinos and Watson 2005). The hit map is a summary of all units from the 8 implanted rats. B: example spike waveforms and interstimulus interval histograms in 2 different sessions. C: behavioral interface panel facing the rat and task trial timelines. The rat would press the center paddle to start a trial and 1 of the 5 cue lights would be on immediately (as shown in the example, the 1st cue light of the left side was on for that trial). Both left and right response paddles would extend 2 s after cue onset. The rat would press response paddles to shift the position of the lighted cue light to the center for a sugar pellet reward, which was dispensed into the little tray shown by the arc along the center line of the face panel. The task was for the rat to press on the paddle of the same side as the cue light. As shown in the example, the correct response was a press on the left-side paddle, which moved the cue light to the right by one position, or the center position.

To obtain a sugar-pellet reward, the rat needed to press either the left or right paddle to control the location of the illuminated light, which varied among the five possible positions. The rat began each trial by pressing on the central paddle. Later, a left-paddle press would shift the light to the right by one position and a right-paddle press would shift the light to the left by one position. Thus the correct response was to press the paddle on the side of a light to shift it toward the central position. No response was required to the center light.

Each trial proceeded as follows (Fig. 1C): as soon as the center paddle was depressed by the rat, one of the five cue lights, chosen at random, was illuminated immediately. We called this event “cue onset.” Both the left and right response paddles extended into the testing box 2.0 s later. The rat was allowed 1.0 s to complete a first paddle press and another 4.0 s for a second paddle press if the cue light required two presses to illuminate the center light. If the light remained at the center position for at least 1.0 s (for the case of center light cue, no response paddle press was needed), the trial ended as a success. All successful trials were associated with an immediate low-frequency reward tone of 1 kHz along with a sugar pellet reward 0.5 s later. Failed trials were associated with a high-frequency tone of 12 kHz without any reward. The rat could start a new trial after an intertrial interval of 8 s for successful trials and 15 s for failed trials. An example of a task sequence is given in Fig. 1C.

According to Fig. 1C, three task periods were defined. The precue task period lasted 1 s. A 2.0-s cue-on task period followed immediately. The cue-on data window was 0.2 to 0.9 s after the cue onset. The response task period started from the instance of the extension of side response paddles and lasted for 1.0 s.

Video analysis.

We captured image sequences with a spatial resolution of 0.47 mm/pixel at 25 frames/s when the rats were performing the directional choice task. The rat's head and body positions were clearly visible on most frames. For each video frame, we extracted three pairs of variables: the implant head cap position (h_x, h_y), the left ear position (l_x, l_y), and the right ear position (r_x, r_y). The rat head position for this frame is then calculated as the means of the three pairs of variables: (H_x, H_y) = $\left(\frac{h_x+l_x+r_x}{3},\hspace{0.17em}\frac{h_y+l_y+r_y}{3}\right)$. Each trial usually consisted of 59 ± 5 frames, which amounted to 2.32 ± 0.20 s from the trial cue onset through the time when the rat made the response press. A sequence of rat's head positions were obtained accordingly. The extraction of these variables from the image sequences was performed in a semiautomatic fashion with part of the procedures performed by custom Matlab code (Mathworks, Natick, MA).

Analysis of rat movement trajectories, start, and end time.

After the head positions (H_x, H_y) were obtained for each trial, we calculated the rat's movement speed at the ith frame as s_i = $\sqrt{{(H_x^{i+1}-H_x^i)}^2+{(H_y^{i+1}-H_y^i)}^2}/40$ (pixel/ms). Let the movement threshold parameter t = 0.5 × $\sqrt{\frac{1}{n-1}(s_1^2+s_2^2+\mathrm{\dots }+s_{n-1}^2)}$, where n is the number of frames in the trial. The movement start time S was considered as the time when the rat's speed rose above the threshold and remained so for five consecutive frames while the movement end time E was considered as when the rat's speed fell below the threshold and remained so for five consecutive frames. By the end time E, the response paddle was usually within reach by rats. The movement duration was thus obtained as D = E − S. Corresponding to the start time, the movement start position was the head position where the rat started to move. Similarly, the movement end position was the rat's head position at the end time.

Recording session and spike sorting.

Multichannel chronic single unit recordings from the rat's AGm and AGl areas were obtained using either the RX5 Pentusa Base Station or the RX7 Microstimulator Base Station (TDT). Original neural waveforms were digitized at 24.414 kHz and saved. Spike sorting and other processing of the raw waveforms took place offline. The neural signals usually maintained strong spike presence for ∼20 to ∼30 recording sessions postsurgery. All behavioral event markers were recorded simultaneously with the neural signals.

The stored waveforms were extracted from TDT data tank and bandpass filtered between 300 and 3 kHz. Offline spike detection and sorting was performed using the M-Sorter from our own laboratory (Yuan et al. 2012). The sorter utilizes the multiscale correlation of wavelet coefficients (MCWC) for spike detection (Yang et al. 2011). Then, the k-means clustering and template matching algorithms were used to classify single units. The M-Sorter has been tested extensively using artificial data sets and real neural recordings. Testing results by the M-Sorter were comparable with or better than the automatic mode (T-Distrubution E-M) of the Offline Sorter (Plexon, Dallas, TX) and the Wave_clus algorithm (Quiroga et al. 2004). The sorted spikes were further inspected by the authors to ensure accuracy. The isolated putative single units were thus obtained. The spikes so obtained from each unit were consistent in waveform shape and interspike interval (ISI) histogram. They also formed separable clusters in PCA space. The spike width was usually >0.5 ms. Isolated units and the time stamps of spiking events along with behavioral event markers were then stored for the analyses performed in this study. Figure 1B shows example neural spike waveforms and their respective ISI histograms.

Same neuron identification.

A composite score was calculated to measure if a unit record from the same recording channel across different recording sessions would be regarded as the same neuron. The score was the sum of six measurements, accounting for spike waveform, peak-to-peak amplitude of spikes, firing rate, interspike interval distribution, firing rate difference between left and right side choices, and whether rate difference between the two choices is statistically significant. The closer the score to zero, the more likely the unit recorded in one session was the same neuron as in previous sessions. We calculated this score for each and every unit record for all rats used in this study. A unit with a score of at least one standard deviation higher than the mean was considered a different neuron from the one recorded in the previous session. On the other hand, units from consecutive sessions with score values within one standard deviation from the mean were considered same neuron. When we present results of the population of single neurons, unit records from the putative same neuron were combined to contribute one data point. Therefore, in neural population results presented below, single data samples corresponded with different neurons.

Stimulation.

To verify the recording sites, intracortical microstimulation was applied using the RX7 Microstimulator Base Station and MS16 Stimulus Isolator by TDT. Passive headstage was used in the procedure. Two rats, J and K, were stimulated after their behavioral and neural data recording using the same implanted electrodes as those for recording neural activities. The animals were lightly anesthetized using KX during stimulation when a train of 13 cathodal pulses was presented at 312.5 Hz. Each pulse was 0.2 ms in duration. Stimulation always started using a small current of 20 μA and increased up to 60 μA in most cases. In one case (rat J) the stimulation current was increased to 100 μA. For rat J, electrodes 5–8 (Fig. 1A) elicited contralateral whisker movements at a 60-μA current. When the current was increased to 100 μA, electrode 6 elicited right forelimb movements and whisker movement. For rat K, electrodes 7 and 8 elicited contralateral whisker movements and neck movement at a 60-μA current.

Performance accuracy measures.

We calculated each rat's performance accuracy, R, in each session. R was the ratio between the number of successful trials N_s and the total number of trials N during the session, expressed as percent correct, i.e., R = $\frac{N_{\text{s}}}{N}$ × 100%. Also, we let N_e denote the number of failed trials. Posterror accuracy R_es was the ratio between the number of successful trials immediately after a failed trial N_es and N_e, also expressed as percent correct, i.e., R_es = $\frac{N_{\text{es}}}{N_{\text{e}}}$ × 100%. Likewise, the postsuccess accuracy R_ss was based on the ratio between the number of successful trials immediately succeeding a successful trial N_ss and N_s, i.e., R_ss = $\frac{N_{\text{ss}}}{N_{\text{s}}}$ × 100%. These two measures together with performance accuracy R were indicators of a rat's ability to learn the task; the larger the R_es and R_ss values, the higher the performance accuracy.

Task-related unit records.

The following three task periods were identified based on the behavioral task protocol (Fig. 1C): precue (1.0 s before cue onset to cue onset), cue-on (from cue onset to 2.0 s after), and response (from response paddle extension to 1s after). We considered a unit record task related if during at least one of the above defined three task periods, its trial averaged firing rate was significantly different from any of the other two task periods (Mann-Whitney U-test, P < 0.01), where the firing rates were calculated as spike counts in those task periods averaged over all trials during a session. The neural results presented in this report were based on all trials that the rat responded with a paddle press (left or right) with either successful or failed outcomes.

Neural modulation.

To study neural modulations by previous trial outcome (success or failure) within the cue-on task period, we compared trial-averaged firing rates using 20-ms bins for each neuron. First, for each trial, a neuron's firing rate in the cue-on task period was represented by a 1 × 100 vector (2.0 s of data). Let f_E(1) denote a neuron's firing rate vector averaged over all trials succeeding a previously failed trial. Also, let f_S(1) be a neuron's average firing rate vector of all trials succeeding a previously successful trial. To study the firing rate difference between posterror trials and postsuccess trials, we define the previous trial modulation measure I₁ as:

$$I_1=\frac{f_{\text{E}}\left(1\right)-f_{\text{S}}\left(1\right)}{\max \left[f_{\text{E}}\left(1\right),f_{\text{S}}\left(1\right)\right]},$$

where max[f_E(1), f_S(1)] denotes the largest element among f_E(1) and f_S(1), and I₁ = (I₁¹, I₁²,…, I₁ⁱ,…, I₁¹⁰⁰) ∈ R¹⁰⁰ with i denoting time bins of the cue-on task period (2.0 s) within a trial. In addition, i = 11, 12,…, 45 correspond with the cue-on data window (700 ms), which was 0.2 to 0.9 s after the cue onset. The measure I₁ is an indicator of a neuron's firing rate difference in the cue-on task period in relation to previous trial outcome. Let |I₁| = (|I₁¹|, |I₁²|,…, |I₁¹⁰⁰|). For each neuron, let τ₁ = arg max_i{|I₁ⁱ|, i = 1, 2,…, 100}, i.e., at time τ₁, the difference in firing rates between posterror and postsuccess trials was the largest among all time bins in a trial. Also, we define a scalar Ī₁ = ∑_{i = 11}⁴⁵ I₁ⁱ representing the averaged values of I₁ⁱ over the cue-on data window.

Neural modulation measured by receiver operating characteristics and area under the curve.

The receiver operating characteristics (ROC) analysis provides a measure for the degree of overlap between two (posterror vs. postsuccess, or left press vs. right press) neural response distributions, and it is independent of the firing rate of a neuron (Green and Swets 1966; Britten et al. 1992; Wallis and Miller 2003). Increased separation of firing rate distributions between postsuccess and posterror trails leads to an increased deflection of the ROC pushing toward the corner or away from the diagonal. The area under the ROC curve (AUC) provides a measure of the level of separation between the two compared distributions, the value of which ranges from 0.5 to 1. To interpret the separation level, a baseline ROC, and consequently AUC, is obtained from using randomly shuffled data. In this study, a bootstrap analysis was performed to estimate the chance-level AUC. For this purpose, neural data were randomly placed into one of the two categories for obtaining an AUC. The same procedure was repeated for 1,000 times. The chance-level AUC was then obtained as the average of the 1,000 sampled results. The farther an AUC value is from the chance-level AUC, or the closer an AUC value is to 1, the larger the distinction between the two (posterror vs. postsuccess, or left press vs. right press) neural response distributions.

The AUC_l and the AUC_r were used to measure differences in the distributions of neural responses between posterror and postsuccess trials among all left and right side responses, respectively. Similarly, the AUC_s and the AUC_e were used to measure differences in distributions of neural responses between left and right directional choices among all postsuccess and posterror trials, respectively.

ANOVA analysis.

Two-way ANOVAs were performed to study how single unit firing rates changed dynamically over all three task periods (precue, cue-on, and response) as a function of the two task factors (previous trial outcome and directional choice). For each neuron, a window of 500 ms traversed the time axis in 200-ms steps starting from the precue task period (i.e., 1,000 ms before cue onset) to the end of response period. Each unit would thus correspond with one data matrix: the number of rows corresponded with the trials in the session while the number of columns time bins. Two-way ANOVA could then be applied to the corresponding column in the unit's data matrix to determine which of the two factors (previous trial outcome or directional choice) was statistically significant.

In this study, all analyses were performed using custom Matlab (Mathworks, Natick, MA) code.

RESULTS

Task performance and single-unit data from eight rats (A, B, C, D, E, F, G, and H) were used for the analysis of neural modulation in relation to the two task-related factors: previous trial outcome and directional choice of the current trial. In this study, we considered the first paddle press within the response task period (Fig. 1C) when analyzing the directional choice factor.

The recording sessions with insufficient trials for statistical analysis were excluded. Units with firing rates <1 Hz were excluded from the analysis as well. For the units used in this study, average firing rates were 34.16 ± 23.53 (spikes/s). A total of 41,840 trials from 232 neural and behavioral recording sessions were obtained from the 8 rats and were examined in the analysis. Spikes were sorted offline after each recording session. When isolated units from each session were treated independently, there were 1,058 unit records from 8 rats, including 514 from AGm and 544 from AGl (Fig. 1A). We recognize, however, that some cells remained the same from session to session, so the number of unit records does not correspond to the number of isolated neurons. Based on the same neuron analysis, we identified 192 different neurons from all unit records, including 103 neurons from AGm and 89 from AGl. The segregation of AGm from AGl cells was verified by intracortical microstimulation. In addition, six rats (A, B, I, J, K, and L) were used in analyzing the movement characteristics such as start and end times and movement speed.

Performance accuracy and response latency.

The rat's performance accuracy, R, was obtained at the end of each recording session. Among the eight rats used in this study (A, B, C, D, E, F, G, and H), five of them (A, D, E, G, and H) reached 80% or higher task performance accuracy. Specifically, four rats (A, D, E, and H) reached 80% in ∼21 sessions and rat G reached 80% in 30 sessions. The other three rats (B, C, and F) reached performance accuracy of ∼60% by the 20th recording session but never achieved 80% correct performance. The learning curve for rat A is displayed in Fig. 2A as an example.

Fig. 2.

Examples of performance. A: daily performance accuracy of rat A. Three learning stages were defined based on daily accuracy: prelearning, learning, and postlearning, which are delineated by the vertical lines. B and C: response latency for failed and successful trials (B) and posterror and postsuccess trials (C) of rat A in each recording session.

Three learning stages were identified according to performance accuracy using trend analysis. Two parameters were taken into account: the slope of a linear regression line fitting data from respective learning stage, and t-statistics with P < 0.05 indicating that the trend was statistically distinguishable from random behavior. When the rat's performance accuracy reached 80% or higher for three consecutive sessions, we considered those sessions corresponding with the postlearning stage. The three slow rats (B, C, and F) did not reach this stage. Actually, the corresponding regression slope was around 0 (t-statistics, P > 0.05). Note that some rats' performance may occasionally fluctuate during the postlearning stage. Next, the learning stage was defined as during which the rats showed a steadily increasing trend in accuracy (regressions slope >> 0; t-statistics, P < 0.05). Sessions 8 through 21 as illustrated in Fig. 2A are such examples. For all eight rats, the performance accuracy increased by 32 ± 10% (means ± SD) during the learning stage. The prelearning stage was defined as the first several sessions in which the rats' performance accuracies fluctuated significantly without displaying a clear upward trend (regression slope near 0; t-statistics, P > 0.05). The lengths of individual learning stages usually varied for different rats.

The response latency was defined as the time between extension of the response paddles and the first paddle press by a rat. The response latency of rat A is shown as an example in Fig. 2, B and C. In summary, only occasional difference was found in response latency between successful and failed trials, and between postsuccess and posterror trials.

Kinematic analysis.

Video sequences were captured for six rats (A, B, I, J, K, and L) while they performed the learning task. Rats A and B provided both neural and video data, while the other four rats (I, J, K, and L) provided video data only. We analyzed the video sequences of the rats from the start of a trial (directional cue onset) to the time of the first response paddle press. For each of the six rats, the movement trajectories of the rat performing left and right press trials within a single session were first extracted. Figure 3, A–C, illustrates movement trajectories of rat J during one entire session where the x- and y-coordinates represent the positions of the rat's head over time from a front view. Using the definition of movement start and stop time defined earlier (see materials and methods) from Fig. 3, B and C, we can see that the mean movement latency relative to cue onset was 249 ms (±20SD) for left trials and 298 ms (±23) for right trials for rat J, respectively; the average end time was around 747 (±35) ms and 818 (±39) ms, respectively. The respective start and end positions of the rats could then be obtained based on the start and end time.

Fig. 3.

Movement characteristics. A: example movement trajectories from cue onset to paddle press of rat J. The background is the front view of the interface panel facing the rat. The light and dark gray curves represent the trajectories of left and right trials, respectively. The circles represent movement onset and the squares represent end of movement, which correspond to the rat's head positions. B and C: x- and y-positions corresponding to movement trajectory in rat A along time. Error bars represent 1SD. D: start time, end time, and movement duration as a function of learning sessions of rat I. Movement start and end time was longer in the 1st 3 sessions of learning and stays more stable afterwards. E: histograms of start time, end time, and movement duration for all 6 rats (A, B, H, I, J, and K). Left: movement start time; middle, movement end time; right: movement duration.

Figure 3D is an illustration of the movement start and end time as task learning progressed by session. Apparently, they decreased significantly during the initial three sessions (one-way ANOVA, P < 0.05) and reached a stable state on the 4th day, whereas movement duration decreased significantly (one-way ANOVA, P < 0.05) and reached a stable state by the 3rd day. The data suggest that only the initial trials during the first three or four sessions may have been modulated by movement kinematics. Similar analysis was performed for the remaining five rats (A, B, I, L, and K), and the same conclusion held true. The trend in those movement characteristics is in accordance with the trend of response latency profile in Fig. 2B. Together, these data suggest that the rat's movement parameters became stable after only a few sessions. This finding probably reflects the fact that the rats were already pretrained for paddle press in association with a cue light but without directional choice before their electrode implant. In addition to the sample results from some rats as shown in Fig. 3, A–D, movement start time across trials of all six rats was 342 ± 87 ms (means ± 1SD), the end time was 829 ± 102 ms, and the movement duration was 487 ± 76 ms (Fig. 3E). Note that the start time and end time could vary from rat to rat. The data displayed in Fig. 3D were typical for each of the six rats. Namely, the rats' movement parameters became stable after the first few sessions.

To further test if previous trial outcome may affect movement parameters, start and end time, movement duration, and movement start and stop positions were evaluated again as learning progressed. Video sequences of all trials from the six rats were used (A, B, I, J, K, and L). Movement start and end times and movement durations were compared between posterror and postsuccess trials using the U-test. These parameters were found not affected by previous trial outcome (P > 0.05). Figure 4 is an example showing movement trajectories organized according to previous trial outcomes (error or success) in the x-coordinate. At each time point, the x-positions for posterror and postsuccess trials were compared using one-way ANOVA for the left press trials and right press trials, respectively. The comparison was repeated for each time point from cue onset to ∼2.0 s after. No significant difference was found between posterror and postsuccess trials (P > 0.05). Similar observations held true for the y-coordinate. This result, together with the response latency measurements (Fig. 2C), supported the idea that the movement parameters in posterror and postsuccess trials did not differ significantly.

Fig. 4.

Rat's movement was not dependent on the previous trial outcome. The x-coordinate is plotted as a function of time. Black lines and gray lines represent postsuccess and posterror trials, respectively. A: movement trajectories in an early session (#5) of learning of rat L, i.e., in the prelearning stage. B: movement trajectories in a later session (#24) of rat K in the learning stage.

Thus neither response latency nor movement kinematics varied according to the current or previous trial outcome. However, our data clearly show that the first two to three sessions (Figs. 2 and 3) included a confounding factor involving motor skill acquisition (Laubach et al. 2000; Cohen and Nicolelis 2004), in addition to directional choice learning that is under study in this article. Since our data revealed that the rat's movement parameters decreased mainly during the initial sessions, while the percentage of correct trials remained low, we removed the first three sessions of all rats from the data analysis performed next to eliminate the potential impact of motor skill learning.

Effect of previous trial outcome on performance accuracy.

Behavioral task performance data were then evaluated as a function of previous trial outcome using all eight rats (A, B, C, D, E, F, G, and H). Trial outcomes immediately after a successful trial and a failed trial were considered by means of postsuccess accuracy, R_ss, and posterror accuracy, R_es, respectively. For each rat, the durations (in terms of sessions) of each of the three learning stages were normalized by the eight-rat averages. They are 5.0 sessions for the prelearning stage (excluding the 1st 3 sessions), 14.9 sessions for the learning stage, and 8.6 sessions for the postlearning stage, respectively. The two measures developed in materials and methods, R_ss (postsuccess accuracy) and R_es (posterror accuracy), during each of the three normalized learning stages are summarized in Fig. 5. Two respective fourth order polynomial regression lines are also shown for each learning stage to illustrate data trends. As shown in Fig. 5, only during the learning stage that the posterror accuracy R_es (65.7 ± 14.9%) was slightly higher than the postsuccess accuracy R_ss (63.5 ± 14.2%) with statistical significance (paired t-test, P < 0.05).

Fig. 5.

Performance in 3 learning stages for 8 rats. The x axis represents the normalized session number, and the y axis represents posterror accuracy (R_es; unfilled circles) and postsuccess accuracy (R_ss; *). The 4th order polynomial regression lines for R_ss and R_es are provided separately for each of the learning stages. Only during the learning stage was the posterror accuracy R_es slightly higher than the postsuccess accuracy R_ss (paired t-test, P < 0.05).

Neural activity modulated by task factors.

For this analysis, we used neural data from eight rats (A, B, C, D, E, F, G, and H). Among the entire set of 1,058 isolated unit records of the eight recorded rats, excluding the initial three recording sessions, a total of 918 isolated unit records were included in the following neural analysis (AGm: n = 464; AGl: n = 454). By using our previously described same neuron identification procedure, 192 single neurons were determined and used in this study. To analyze rats' neural correlates during the directional choice learning, two task-related factors were investigated: 1) directional choice (left press or right press) of the current trial, and 2) previous trial outcome (success or failure).

Two examples of neural modulation represented in perievent time histograms are shown in Fig. 6 for an AGl unit and an AGm unit, respectively. Note that for presentation clarity, one-sixth of the total trials in a session were uniformly sampled and displayed here.

Fig. 6.

Example AGl unit and its spike raster of rat D isolated from channel 7 on the 12th recording session (left) and example AGm neural spike raster for rat E isolated from channel 14 on the 16th recording session (right). The x-axis is the time course with event markers. Top: spike raster plots from individually recorded trials and the average firing rates. Middle: area under the curves (AUCs) to compare differences in neural responses between postsuccess and posterror trials for a left (green) or right (black) directional choice, respectively. Bottom: AUCs to compare differences in neural responses between left and right directional choices after a successful trial (blue) or an error trial (red), respectively. The respective chance level AUC values are shown in dashed lines in middle and bottom.

Figure 6 shows example records in which the average firing rate was modulated by both the directional choice and previous trial outcome in the cue-on and response periods. It also shows that neural modulations were stronger in some task periods than in others (Fig. 6, top). This property suggests that the firing rates of these single units encode two factors, directional choice and previous trial outcome, and do so dynamically.

To gain insight into these two factors, we evaluated them separately using ROC analysis. The AUCs over time were calculated using a time bin of 100 ms that was slid in 20-ms steps. The time coverage spans from 1.0 s before cue onset to 3.0 s after cue onset, which corresponds to the entirety of the three task periods (see Fig. 1C). The AUC_l and the AUC_r provided measures of differences in the distributions of neural responses between posterror and postsuccess trials among all left and right side responses, respectively. The AUC_s and the AUC_e provided measures of differences in distributions of neural responses between left and right directional choices among all postsuccess and posterror trials, respectively.

The first set of AUCs, AUC_l and the AUC_r, showed that both left trials and right trials were affected by previous trial outcome. Also, for the two example units (Fig. 6, middle), the firing rates of posterror trials were lower than those of postsuccess trials. The second set of AUCs, AUC_e and the AUC_s, showed that posterror or postsuccess trials still carried directional choice information (Fig. 6, bottom).

Population analysis.

Two-way ANOVAs were performed to study how single unit firing rates changed over all three task periods (precue, cue-on, and response) as a function of the two task factors: previous trial outcome and the directional choice of the current trial. We examined the statistical significance of the two task factors by the following three P values: the two single P values, P₁ for directional choice and P₂ for previous trial outcome, for the main effects, and the additional P value P₃ for two-factor interactions between directional choice and previous trial outcome. From this section and onward, the reported results were based on 192 putative single neurons.

The two-factor interaction terms were obtained for each unit at each of the 18 time points by two-way ANOVA. To inspect the interacting effect of the two task factors, the number of units corresponding to a statistically significant interaction (P₃ < 0.01) was accounted for at each of the 18 time points. It turned out that during the three task periods, only a small number of AGm and AGl units exhibited significant two-factor interactions. On average (for time points 1–18), 3.6% (n = 3.1) and 6.9% (n = 7.1) of the AGm and AGl units, respectively, were modulated by the interaction between directional choice and previous trial outcome. Since the interaction between the two task factors was not as pronounced, we focus on studying the previous outcome and the directional choice factors, respectively.

Next, we show that neural modulations encoding the two task factors varied over time. From Fig. 7A, a large number of AGm units were modulated by previous trial outcome in the precue (51.7%, n = 53.2), cue-on (53.0%, n = 54.6), and response (38.3%, n = 39.5) task periods. The largest number of units encoding previous trial outcome was found at the eighth time point indicated by letter “c” (400–900 ms after cue onset, 66.0% of units, n = 68). Also, an increasing number of units were significantly modulated by directional choice after cue onset and peaked at 71.8% (n = 74) in the 17th time point during the response task period. Similarly, a large number of AGl units were modulated by previous trial outcome in the precue (57.0%, n = 50.7), cue-on (52.1%, n = 46.4), and response (37.4%, n = 33.3) task periods (Fig. 7B). The largest number of units encoding previous trial outcome was found at the eighth time point indicated by letter “d” (68.5% of units, n = 61). Also, an increasing number of units were significantly modulated by directional choice after cue onset and peaked at 70.8% (n = 63) at the 17th time point during the response task period. Therefore, the neural activity in the cue-on task period, especially the beginning portion, appeared to be modulated by previous trial outcome. As shown by the dark lines in Fig. 7, A and B, a large portion of both AGl and AGm units encoded the previous trial outcome within the cue-on data window. Examining the cue-on data period more closely, it is interesting to note that starting from the 11th (for AGm) or the 13th (for AGl) time point and onward, a larger number of AGm or AGl units responded to directional choice than to the previous trial outcome.

Fig. 7.

Neural population (n = 192) and their dynamic modulations according to the two task factors: directional choice of current trial (left press or right press) or previous trial outcome (success or error). A and B: dynamic representations of the percentages of neurons that showed significant modulation to the two task factors using two-way ANOVA. The peak modulation value for previous trial outcome is marked as “c” and “d” for the AGm and AGl area, respectively. C and D: scatter plots of P value pairs of each neuron. C: point “c” in A corresponds with the 8th time point (400–900 ms after cue onset) during the cue-on task period. 30.1% (n = 31) of the AGm neurons were encoding previous trial outcome, 22.3% (n = 23) were encoding directional choice, and 35.9% (n = 37) were encoding both. D: point “d” in B corresponds with the 8th time point (400–900 ms after cue onset) during the cue-on task period. 41.6% (n = 37) of the AGl neurons were encoding previous trial outcome, 24.7% (n = 22) were encoding directional choice, and 27.0% (n = 24) were encoding both.

We further investigated the distributions of P value pairs of all AGm and AGl units based on the two task factors at the eighth time point corresponding to points “c” and “d” in Figs. 7, A and B, respectively. The corresponding results are shown in Fig. 7, C and D. The same calculation was then conducted for each of the time points from 5 to 18 as for point 8. Only a small portion of the AGm and AGl units were modulated by both directional choice and previous trial outcome as main effects during the cue-on and response task periods (7.4%, n = 7.6 for AGm and 6.6%, n = 5.8 for AGl), respectively. This indicates that a single unit was predominately modulated by a single task factor at a time.

Previous-outcome coding in the cue-on period.

Since both AGm and AGl neural modulation was most affected by previous trial outcome during the initial part of the cue-on window (dark lines in Fig. 7, A and B), which also corresponds to the period when the rat's movement was stereotypical with little variation (Fig. 3), we focused our analysis on this time period. To do so, we used the previous trial modulation measure, I₁, which is a 1 × 100 vector covering the cue-on task period and providing a measure of discrimination in neural firing rates between postsuccess and posterror trials. Since this is a collective account of all units and because the properties of AGm and AGl resembled each other closely, we pooled AGm and AGl units for this analysis.

The differences in firing rates between postsuccess and posterror trials were strongly affected by previous trial outcome at the time of the cue-on data window (Fig. 8A). The time bin τ₁ at which |I₁| achieved the highest value was computed for each units. From Fig. 8B, the largest number of units (20.0% of total task related numbers, n = 38) was found at τ₁ = 600 ms after the cue onset. Also, 74.0% of (n = 142) unit records had τ₁ within the cue-on data window (Fig. 8B). This finding provides further support for the idea that the greatest degree of modulation encoding the previous trial outcome occurred during the cue-on data window.

Fig. 8.

Summary of previous trial modulation measure I₁ vector over the cue-on task period (0–2,000 ms) for all units (n = 192). A: component-wise absolute values of I₁ of each unit over time during the cue-on task period. Unit records were sorted in ascending order according to the absolute values of Ī₁. B: distribution of τ₁ using 100-ms time bin resolution. C: histogram of all units according to Ī₁.

For the 192 AGm and AGl units, the average I₁ⁱ (i = 11,…, 45, corresponding to the cue-on data window) value over the cue-on data window was denoted as Ī₁. The histogram of the Ī₁ values for all units is displayed in Fig. 8C. Recall that Ī₁ = $\frac{f_{\text{E}}\left(1\right)-f_{\text{S}}\left(1\right)}{\max \left[f_{\text{E}}\left(1\right),\hspace{0.17em}{f}_{\text{S}}\left(1\right)\right]}$. The finding that the Ī₁ was <0 for the largest fraction of these units (88.0%, n = 169) indicates that most units had greater activity in postsuccess trials than in posterror trials, thus encoding positive outcomes. Figure 8, A and B, shows that differences in neural firing rates between posterror and postsuccess trials changed dynamically during the time course of a trial, with the differences being most pronounced during the cue-on data window. Furthermore, the firing rates of most units decreased after the rat made an error.

Previous-outcome coding and performance accuracy.

Next, we examined how performance accuracy correlated with the observed neural activities. Given that previous outcomes and chosen response direction did not commonly have interactive effects, it was adequate to analyze the factors of previous outcome and chosen direction independently. The firing rates of AGm and AGl units in the cue-on data window (from 200 to 900 ms after cue onset, Fig. 1C) were thus inspected using one-way ANOVA. The units showing significant neural modulation (P < 0.01) in the cue-on data window in response to previous trial outcome were classified as “previous outcome selective,” and units with this property were analyzed separately for AGm and AGl. A total of 129 units including 66.0% (68 out of 103) of AGm and 68.5% (61 out of 89) of AGl encoded the previous outcome.

Of the eight rats included in this study, their behavioral task performance accuracy over recorded sessions ranged from 33.4 to 92.1%. This distribution was equally divided into 12 intervals ranging from 35 to 90% in 5% increments. The isolated units were placed into one of these 12 intervals. If the unit records from the same putative unit in consecutive sessions fell into multiple intervals, then records of the unit in the same interval collectively contributed one data sample to that interval. The intervals with <10 units were excluded from this analysis, resulting in performance accuracies ranging from 40 to 85% distributed over 10 intervals for final examination. As a result, 21.6 ± 8.4 (means ± SD) units in each of the 10 intervals for AGm and 27.0 ± 12.3 units for AGl. The results from AGm and AGl are plotted separately in Fig. 9. In the cue-on data window, the fraction of previous trial outcome selective units were found strongly correlated with performance accuracy for both AGm and AGl units (correlation coefficient, r = 0.72 for AGm, and r = 0.86 for AGl, both of which were highly significant, P < 0.05).

Fig. 9.

The percentage of previous outcome selective AGm and AGl units as a function of daily performance accuracy R. Solid line represents the combined regression line for AGm and AGl. This analysis used 10 performance accuracy intervals, ranging from 40 to 85%. The percentage of previous outcome selective neurons in each interval was expressed as the number of previous trial outcome selective neurons divided by the total number of neurons in the interval.

To answer the question of whether there is any significant difference between AGm and AGl units in percentages as they were modulated by performance accuracy shown in Fig. 9, we performed a paired t-test. No significant difference was found in the percentages of AGm and AGl units (P = 0.66).

Previous-outcome selectivity and performance accuracy.

To further examine previous-outcome coding as learning progressed, we computed the AUCs between posterror trials and postsuccess trials. For each previous outcome selective neuron, the firing rate from each trial in the cue-on data window (200–900 ms after cue onset) was obtained and placed into either the posterror or the postsuccess category, respectively. All trials of the same neuron were used to obtain the AUCs, which is based on the neural response distributions of posterror or postsuccess trials. This process was repeated for all previous outcome selective neurons.

The chance-level AUC value was 0.55. The AUC values for each behavioral accuracy interval were significantly higher than chance (t-test, P < 0.01; Fig. 10). Note that one AGl data point at behavioral accuracy of 45% was removed since it only contained one neural data entry. Furthermore, linear regressions of AUC values with respect to behavioral accuracy were performed. Positive regression slopes were observed for both AGm (0.0026, P < 0.05) and AGl neurons (0.0044, P < 0.05). This suggests a positive correlation between the firing rate modulation by previous trial outcome and the behavioral performance adaptation as learning progresses.

Fig. 10.

AUC measurement of difference in neural responses between posterror trials and postsuccess trials as behavioral accuracy improved.

DISCUSSION

A rat model was used in this report to examine neural correlates to behavioral adaptation during learning of a directional choice task. Two task-related factors (previous trial outcome of either success or failure; current trial directional choice of either left or right) were analyzed in association with the rat's neural activity and behavioral learning performances. Our study revealed that the recorded AGm and AGl units encoded both current trial directional choice and previous trial outcome during the time course of a single trial (Figs. 6 and 7). Our major findings were as follows: 1) in the time course of a trial, a large fraction of recorded individual AGm and AGl units were modulated by previous trial outcome especially in the cue-on data window (Figs. 6–8); 2) more units had greater activity after successful trials than after error trials in the cue-on data window (Fig. 8C); 3) the number of previous trial outcome selective units was highly correlated with the rat's performance accuracy (Fig. 9); and 4) for those previous trial outcome selective units, the differences in the neural responses between posterror and postsuccess trials increased as performance accuracy improved (Fig. 10). These results suggest that the AGm and AGl of the rat frontal cortex are not static but adaptive at fine time scales of seconds or subseconds. Our data may further suggest that modulations in those units contributed to retrospective information processing for rats during the learning process.

Task factors and single units.

In our experiment, other factors that might also affect recorded neural activity are summarized as follows. All the rats used in the study learned the contingency between a light cue and a paddle press during pretraining, before learning the directional choice task. Furthermore, the variations in the rat's kinematics measured by response latency (Fig. 2, B and C) and movement characteristics (Fig. 3) quickly reduced to an insignificant level during the prelearning stage. Hence, the data used in this analysis excluded the first three recording sessions (refer to discussions on Fig. 3) but only included the sessions when rat's kinematics became stable. The choice that the rat made in the previous trial would potentially modulate neural activity during the current trial as well. However, we found that only a small number of recorded units (6.3%, 12 out of all 192 neurons; 7.8%, 8 out of 103 AGl neurons; 4.5%, 4 out of 89 AGm neurons) were modulated by the previous choice factor. In the Genovesio et al. (2014) study, the percentage of primate PFC neurons showing a significant effect of the previous choice decreased to a chance level soon after trial start. However, they observed much higher percentage of neurons with effect of the previous outcome factor. Given the above considerations of those unlikely confounding factors and the insignificant previous choice factor that may elicit additional neural responses in the areas we recorded from, previous trial outcome and directional choice are considered the two primary task factors.

Additionally for the issue of chronic recording from single units, one analysis of macaque monkeys suggested that about one-third of the chronically recorded primary and premotor cortical neurons retained their isolation across sessions (Dickey et al. 2009). Jackson and Fetz (2007) suggested that 50% of the original units were stable through 1 wk and 10% were stable through 2 wk based on their motor cortical chronic recordings from two monkeys. In this study, we employed a quantitative analysis to identify same neurons recorded from consecutive sessions, upon which our study is performed. Nevertheless, the results were not about neural adaptation of a single unit before and after learning but rather they were to describe how a population of multiple single units adapted as behavioral learning progressed.

Distributed neural modulation of previous outcome.

Significant evidence from primate and rodent studies indicated a widespread neural modulation in response to previous outcome during learning. Trial outcome as a transient event was shown clearly encoded in single unit activity in PFC (Narayanan and Laubach 2008; Histed et al. 2009; Genovesio et al. 2014), anterior cingulate cortex (Seo and Lee 2007; Quilodran et al. 2008), striatum (Histed et al. 2009), and hippocampus (Wirth et al. 2009). The signal was sustained in the intertrial interval and carried over to the next trial, which may have contributed to linking past outcomes with future actions (Donahue et al. 2013). Additional evidence was also available in human studies (Danielmeier et al. 2011; Danielmeier and Ullsperger 2011). Among those brain areas, the PFC has been suggested as an important node playing the role of top-down control of error processing, which may require well-coordinated participation from a large network of cortical and even subcortical areas. For instance, the monkey's premotor cortex was shown to be involved in abstract rule learning (Wallis and Miller 2003), and the ventral premotor was proposed for processing and evaluating the behavior of previous decisions (Acuña et al. 2010). Our results for the first time pointed to the rat's AGm and AGl for their involvement in processing previous outcome in relation to outcome-dependent behavioral adaptation.

By reviewing evidence based on cytoarchitectonics, topology, and corticostriatal projection of rats with primates, Preuss (1995) did not consider rat having a prefrontal cortex. While others argued differently (Uylings and Van Eden 1990; Brown and Bowman 2002; Uylings et al. 2003), some researchers referred to the anterior cingulate, prelimbic, and infralimbic areas of rats as the prefrontal cortex and reported sustained trial outcome related signals in those areas (Narayanan and Laubach 2008). Even though the part of the AGm that we recorded from does not overlap with the PFC in the sense just described, there has been clear evidence that the AGm has reciprocal connections with the dorsomedial prefrontal cortex (dmPFC). Specifically, it takes afferent projections from two subregions of the dmPFC: the dorsal prelimbic cortex and pregenual anterior cingulate cortex (Reep et al. 1990; Condé et al. 1995; Hoover and Vertes 2007). Also, AGm is one of the few nonlimbic cortical regions that project back (Sesack et al. 1989). Additionally, the rat's AGm was proposed to be possibly homologous to the premotor cortex, supplementary motor area, and frontal eye field in primates (Reep et al. 1987, 1990; Passingham et al. 1988; Erlich et al. 2011) and to be a multimodal association area (Reep et al. 1987, 1990). However, agreement has not been reached for a clear homology between the rat's AGm and the primate's counter parts. Since the rat's AGm projects to the spinal cord, it is adequately identified as one of the nonprimary motor areas in primates (Donoghue and Wise 1982; Remple et al. 2007) while leaving the specific identifications of the rat's AGm conjectural. On the other hand, the rat's AGl has been considered homologous to the primary motor cortex (Donoghue and Wise 1982; Donoghue and Parham 1983) and is connected reciprocally with AGm (Reep et al. 1987, 1990; Ueta et al. 2013). It is therefore not surprising that previous trial outcome information could be conveyed to AGm and AGl for their involvement in mediating the rat's future choice behavior.

Neural correlates to behavioral error correction.

We recorded rat's neural activity in the AGm and AGl areas along with rat's behavioral performance simultaneously during the entire process of rat's learning the directional choice task. This has given us the unique opportunity to correlate neural modulation in rat's AGm and AGl areas with their behavioral adaptation. Specifically, we found that the number of previous outcome selective neurons was positively correlated with the rat's learning performance and the difference in neural responses between postsuccess and posterror trials increased as learning accuracy improved.

Behaviorally, Narayanan and Laubach (2008) reported posterror slowing when rats learning a reaction time task. Here we did not observe significant differences in response latency (Fig. 2) between postsuccess and posterror trials, probably because our rats had sufficient time to make their choice during the cue-on period. Histed et al. (2009) reported much higher behavioral accuracy in postsuccess than posterror trials when monkeys repeated learning reversed cue-response associations. This observation could be the effect of the task itself in which a correct trial reinforces cue-response associations, while an error trial could be due to either a wrong choice or the reversal of associations without explicit cue. In our task, an error could only result from rats making incorrect choices. An error trial may get them thinking differently from the correct trial responses. This might be the reason that we observed a slight but statistically significant performance improvement after error trials in the learning stage (Fig. 5).

Narayanan and Laubach (2008) recorded rats' dorsomedial PFC and primary motor cortical neurons. Histed et al. (2009) recorded PFC neurons in monkeys. Both studies found that the firing rates of a fraction of the recorded single neurons were modulated by previous trial outcome. Furthermore, their study found about half of the recorded PFC neurons increased their firing rates in postsuccess trials than posterror trials. For most of the recorded primary motor cortical neurons in (Narayanan and Laubach 2008), the respective firing rates increased in postsuccess trials than the posterror trials.

Narayanan and Laubach (2008) focused on neural modulation by previous trial outcome in relation to posterror slowing. Histed et al. (2009) studied previous outcome encoding as monkeys learning repeatedly reversed associations. Even though we focus on different aspect in our respective studies from Narayanan and Laubach (2008) by using different behavioral tasks, and our task is also different from the arbitrary stimulus response association task in Histed et al. (2009), we also observed strong neural modulation by previous outcome in a large number of AGm and AGl neurons as associative learning took place. However, we have also realized differences of our results from theirs. Specifically, we found ∼88% AGm and AGl neurons had higher firing rates in postsuccess trials than in posterror trials, while 12% had higher rates in posterror trials than in postsuccess trials. In addition, Histed et al. (2009) reported a significant effect of previous outcome on PFC neuron's direction selectivity, while we found a small portion of AGm and AGl neurons modulated by both previous trial outcome and current trial choice. These specific differences in the number of modulated neurons and the specific modulation trend may be due to variations in experimental conditions and specific recording sites.

Nonetheless, these results together suggest that several areas in the frontal cortex such as the AGm, AGl, and the PFC are important in processing and passing on error related information. Donahue et al. (2013) pointed out that the widespread presence of signals in different brain areas encoding previous outcome may not imply that they serve the same functions. A better understanding of how information is processed at each node requires further investigation in terms of how past experience is used to guide future actions.

Finally, our results together with previous studies demonstrate that neural adaptation to previous trial outcome may be a part of a general mechanism for learning and such neural adaptation may involve a distributed system of several well-coordinated brain areas.

GRANTS

This work was supported in part by the National Science Foundation Grants ECS-0702057 and ECS-1002391.

DISCLOSURES

No conflicts of interest, financial or otherwise, are declared by the author(s).

AUTHOR CONTRIBUTIONS

Author contributions: Y.Y. and H.M. performed experiments; Y.Y. analyzed data; Y.Y. and J.S. interpreted results of experiments; Y.Y. and H.M. prepared Figs.; Y.Y., H.M., and J.S. drafted manuscript; Y.Y., H.M., and J.S. edited and revised manuscript; Y.Y., H.M., and J.S. approved final version of manuscript; J.S. conception and design of research.