Reward-Modulated Motor Information in Identified Striatum Neurons

Abstract

It is widely accepted that dorsal striatum neurons participate in either the direct pathway (expressing dopamine D1 receptors) or the indirect pathway (expressing D2 receptors), controlling voluntary movements in an antagonistically balancing manner. The D1- and D2-expressing neurons are activated and inactivated, respectively, by dopamine released from substantia nigra neurons encoding reward expectation. However, little is known about the functional representation of motor information and its reward modulation in individual striatal neurons constituting the two pathways. In this study, we juxtacellularly recorded the spike activity of single neurons in the dorsolateral striatum of rats performing voluntary forelimb movement in a reward-predictable condition. Some of these neurons were identified morphologically by a combination of juxtacellular visualization and in situ hybridization for D1 mRNA. We found that the striatal neurons exhibited distinct functional activations before and during the forelimb movement, regardless of the expression of D1 mRNA. They were often positively, but rarely negatively, modulated by expecting a reward for the correct motor response. The positive reward modulation was independent of behavioral differences in motor performance. In contrast, regular-spiking and fast-spiking neurons in any layers of the motor cortex displayed only minor and unbiased reward modulation of their functional activation in relation to the execution of forelimb movement. Our results suggest that the direct and indirect pathway neurons cooperatively rather than antagonistically contribute to spatiotemporal control of voluntary movements, and that motor information is subcortically integrated with reward information through dopaminergic and other signals in the skeletomotor loop of the basal ganglia.

Introduction

The basal ganglia are composed of the dorsal and ventral striatum, globus pallidus (external segment, GPe; internal segment, GPi), subthalamic nucleus (STN), and substantia nigra (pars compacta, SNc; pars reticulata, SNr), and they connect with the cerebral cortex and thalamus to organize parallel cortico-basal ganglia-thalamo-cortical loop circuits (Graybiel, 2000). In primates, these parallel loops are both structurally and functionally classified as the limbic, prefrontal, oculomotor, and skeletomotor loops (Alexander and Crutcher, 1990), and rodents also have similar topographic organization (Gerfen, 2004; Thorn et al., 2010). For example, the dorsal striatum receiving input from the motor cortex participates in the skeletomotor loop that crucially controls voluntary movements, while the ventral striatum receiving input from the orbitofrontal and cingulate cortices participates in the limbic loop that processes motivational information related to reward expectation or acquisition.

Most (>90%) of striatal neurons are medium spiny neurons, the only cell type of striatal projection neurons, that receive glutamatergic inputs from the cortex and send GABAergic projections to other parts of the basal ganglia. Half of the striatal projection neurons directly innervates GABAergic neurons in the GPi and SNr (direct pathway), whereas another half reaches the GPi/SNr neurons indirectly via GABAergic neurons in the GPe and then glutamatergic neurons in the STN (indirect pathway) (Kawaguchi et al., 1990; Gerfen, 2004). The excitation of striatal direct pathway neurons will increase the activity of cortical neurons, whereas the excitation of indirect pathway neurons will decrease it. The striatal projection neurons for the direct and indirect pathways exclusively express dopamine D1 and D2 receptors, respectively (Gerfen et al., 1990; Gerfen, 2004; Gerfen and Surmeier, 2011). Dopamine, released from SNc neurons, activates the direct pathway neurons through D1 receptor activation, and inactivates the indirect pathway neurons through D2 receptor activation (Tritsch and Sabatini, 2012). Loss of SNc dopamine neurons eventually leads to excessive suppression of functional activity in the motor cortex in Parkinson's disease that is characterized by akinesia, muscular rigidity, and tremor (DeLong, 1990).

It is, therefore, plausible that the direct pathway neurons (D1 positive) in the skeletomotor loop may carry motor information to generate a selective voluntary movement, while the indirect pathway neurons (D2 positive) may carry opposed motor information to silence it (for review, see Cui et al., 2013). It is also possible that reward expectation may modulate the motor information inversely in the D1-positive versus D2-positive neurons within the skeletomotor loop, which is achieved by dopamine released from the SNc neurons coding reward prediction errors (Schultz et al., 1997; Kawagoe et al., 1998). However, it still remains unknown how differently the striatal projection neurons constituting these antagonistic pathways individually represent motor information and, also, how they are modulated by reward information in the skeletomotor loop. Here, we examined functional motor-related activations and reward-related modulations in D1-positive and D1-negative neurons that were identified by a juxtacellular visualization, in the dorsolateral striatum (included within the skeletomotor loop) of the rats performing voluntary forelimb movements in a reward-predictable condition.

Materials and Methods

Animal preparation.

All experiments were performed in accordance with the Animal Experiment Plan approved by the Animal Experiment Committee of RIKEN. Adult Long–Evans rats (N = 67; male 150–250 g; SLC) were handled to adapt to the experimenter and experimental environment. A sliding head-attachment (custom-made by Narishige) was surgically attached to the skull, and reference and ground electrodes were implanted above the cerebellum under 2% isoflurane anesthesia (Isomura et al., 2009; Kimura et al., 2012). After recovery from the primary surgery, the rats were deprived of drinking water in their home cages where food was available ad libitum, and they were able to acquire sufficient water as a reward for task performance in the laboratory. The rats were, if necessary, provided an agar block (containing 15 ml of water) to maintain >80% of their body weight.

Behavioral task training.

The behavioral task, alternate-reward forelimb movement task, was the same as used in the previous study (Isomura et al., 2009) except for an alternation of reward and no reward delivery (see Fig. 1A). Briefly, we made the rats learn the behavioral task in a head-fixed condition efficiently by using a multi-rat task-training system consisting of six training boxes (O'Hara & Co). In the training box, each rat was automatically trained to perform the alternate-reward forelimb movement task under head fixation for several hours a day (total eight training days in 2 weeks). At the endpoint of this task learning, the rat had to start each trial by pushing a constant-torque lever and hold it with the right forelimb for at least one second. When the rat pulled the lever spontaneously (without any cue signal) for >60% of a full lever shift (12 mm), a high-tone sound (11 kHz, 0.6 s) was presented with a 0.2–0.8 s delay (0.1 s steps at random) to indicate that the response was correct in the trial. A drop of 0.1% saccharin water (0.01 ml; dispensed by a syringe pump accurately) as a reward was or was not delivered alternately during the presentation of high-tone sound; thereby, he was able to anticipate the delivery of reward or no reward by learning. The period of sound presentation was followed by a short intertrial interval (ITI, ∼1 s). If the rat failed to make a correct response, a low-tone sound (3 kHz, 0.6 s) was emitted 0.2 s after the failure, and the rat must retry the same trial type (rewarded or nonrewarded) after the ITI until it was successfully cleared. We inserted the ITI period between trials to isolate neuronal activity for the forelimb movements from any responses to the high/low-tone sounds and reward delivery. The rats were allowed to start pushing the lever during or after the short ITI period without any external signal; the hold time could be a bit over 1 s. Thus, our behavioral task required the animals to initiate the pull action just in a self-paced manner (unlike a time-measurement task). The lever position was always monitored by the task-training system through an 8-bit encoder, and forelimb motion was monitored with an infrared video camera. Lever-shift and hold-time requirements for correct response progressively increased during the first training week. For a just-in-time recording experiment, the rat experienced a waiting period (1 h) with the lever locked before every task learning session in the second week.

Figure 1.

Behavioral performance of alternate-reward forelimb movement task. A, Schematic diagram of electrophysiological recordings from a head-fixed rat (left) performing the alternate-reward forelimb movement task (right). See Materials and Methods for details. B, Increase in the number of successes in rewarded (blue) and nonrewarded (red) trials through a 2 week training period (8 d), transfer day (T) and recording day (R) (N = 63 rats; means and SD). C, Typical task performance on the recording day. Left, Trial distribution of lever hold time in rewarded (blue) and nonrewarded (red) trials. Lever pulls after holding for >1 s were correct task responses. Right, Averaged lever trajectory (mean and SD) that was aligned with the onset of pull movement (0 ms). D, Group analysis on task performance. Time for lever hold (left), pull movement (middle; i.e., pull velocity), and trial completion (right; previous success (high-tone) sound to current success sound) in rewarded (blue) and nonrewarded (red) trials. Inset for pull represents mean difference in pull time between rewarded and nonrewarded trials.

Once the animals completed the operant task learning, they were transferred to the recording room to practice performing the same behavioral task in a novel environment for the final behavioral and electrophysiological experiment. The following day, the rats were subjected to a second surgery under isoflurane anesthesia, and a tiny hole in the skull and dura mater were made above the left forelimb (FL) area (0.5 mm anterior, 2.5–3.0 mm lateral of bregma). The coordinate of FL area was determined by intracortical microstimulation (approximately –50 μA, 20 pulses at 500 Hz) to evoke movements on contralateral forelimb (Isomura et al., 2009). The hole was covered with silicon sealant until the recording experiment commenced.

Electrophysiological recordings.

One day after the second surgery, we obtained juxtacellular recordings (Isomura et al., 2009) from single neurons in the dorsolateral part of left striatum, which is directly innervated from the FL area of the motor cortex (see Fig. 5A), of individual rats performing the alternate-reward forelimb movement task. A glass electrode (BF150–75-10; Sutter Instrument) was prepared by a laser puller (P-2000; Sutter Instrument) and a blunt tip was created under a microscope. The electrode was filled with 2% Neurobiotin (Vector Laboratories) in 0.5 m KCl (7–18 MΩ). It was inserted through the hole (above the FL area) into the dorsolateral striatum laterally at a 12° angle (3.5–5.6 mm in depth) with a microdrive (LSS-8000 Inchworm; Burleigh Instrument) that was installed on a fine micromanipulator (1760–61; David Kopf Instruments) on a stereotaxic frame (SR-8N; Narishige). Juxtacellular signal was amplified with an intracellular amplifier (IR-283; Cygnus Technologies) and sampled at 20 kHz (final gain, 1000; bandpass filter, 300–10 kHz) with a hard-disk recorder (DataMax II; R.C. Electronics). First, we just electrically recorded spike activity of several striatal neurons one by one along one track in each animal, and then tried to electroporate Neurobiotin into the last recorded neuron with positive current pulses (2–20 nA, 50% duty cycle at 1 Hz, typically for 15 min) for morphological visualization. Consequently, most of the juxtacellularly recorded neurons were not identified morphologically.

Figure 5.

Reward modulation in D1-positive and D1-negative striatal neurons. A, Corticostriatal projections from the FL area of motor cortex to the dorsolateral part of striatum (CPu; dotted area). The corticostriatal axons were labeled by an injection of anterograde tracer BDA into the FL area. B, The axonal distribution magnified from the square in A. C, Striatal patch area expressing μ-opioid receptors (MOR). A juxtacellularly recorded neuron was located in striatal matrix area (arrowhead). D, Distribution of D1-positive and D1-negative neurons with different activity. The visualized neurons were indicated by colors (red, D1-positive; green, D1-negative) and symbols (square, Hold-type; triangle, Movement-type; circle, others). E, Left, Spike width and D1 mRNA expression in individual neurons (left). The plot for spike width and baseline spike rate is the same as shown in Figure 2C. Colored symbols represent the visualized neurons, while gray symbols represent nonvisualized neurons. Right, RMI distribution for the Hold-type and Movement-type neurons with or without D1 mRNA expression.

In combination with the juxtacellular recordings for single striatal neurons, we recorded multineuronal activity in the FL area of the left motor cortex (Isomura et al., 2009). A one-shank 16-channel silicon probe, which had two sets of tetrode-like electrodes (at the tip and 800 μm above it, respectively) for unit activity and eight electrodes for local field potentials (LFP8 + TetrodeSD; NeuroNexus Technologies), was inserted up to 1200 μm or 1600 μm into the FL area vertically. In this way, we obtained multineuronal activity at 400 and 1200 μm (N = 12 rats) or at 800 and 1600 μm (12 rats) in depth from the cortical surface (sampling rate, 20 kHz; final gain, 2000; original bandpass filter, 0.5 Hz to 10 kHz).

Spike activity analysis.

Multineuronal (and juxtacellular) recording data were processed to isolate spike events by the automatic spike-sorting software, EToS, using wavelet transform and robust variational Bayes (Takekawa et al., 2010, 2012). The spike clusters were combined/divided/discarded manually to refine single-neuron clusters by the manual clustering software Klusters and NeuroScope (Hazan et al., 2006). The relationship of spike activity with behavioral performance was analyzed by MATLAB (MathWorks). The behavior-related spike activity in the striatum and motor cortex was classified simply into three functional groups (see Fig. 2A) as follows: Hold-type activity was a constant or slowly changing activation during lever-hold period; Movement-type activity was a phasic activation starting <0.5 s before the onset of pull movement and falling down during or after the movement; Reward-type activity was a phasic activation in relation to the reward delivery (anticipating or responding) rather than the forelimb movement itself. The Hold-type activity was defined as a significant increase in spike activity during the lever-hold period (–0.8 to –0.2 s from the pull onset) compared with the lever-pull period (0 to +0.2 s from it; p < 0.05 in Mann–Whitney U test). The Movement-type activity was defined as a phasic spike increase around the pull onset (–0.5 s to +0.2 s from the pull onset; z-score > 2 in at least four of five consecutive bins; 20 ms each) compared with a baseline period (–1.0 to –0.5 s from it). We focused mainly on the first two groups (Hold-type and Movement-type) for analysis in the present study.

Figure 2.

Reward modulation of task-related activity in striatal neurons. A, Three types of functional activation in striatal neurons that were recorded juxtacellularly. Hold-type (left) and Movement-type (middle) activities were aligned with the onset of pull movement (0 s) in rewarded and nonrewarded trials. Reward-type activity was aligned with the onset of high-tone success sound (and simultaneous reward delivery in rewarded trials). Calibration for spike waveform: 0.5 ms, 0.2 mV. B, Population ratios for the activity types in the striatum. The number of neurons is in parenthesis. C, Spike rate averaged for the whole recording time (top) and for lever hold time (bottom) plotted against spike width at 1/2 amplitude for individual neurons. The activity types were indicated by the same color index as shown in B. D, Populational Hold-type (upper) and Movement-type (lower) activities that are aligned with the end of push (left) and the onset of pull (right). Thick line and light color area indicate mean and SEM, respectively. Inset, Push-aligned activity of the same Hold-type neuron shown in A, left. E, Distribution of RMI for the Hold-type neurons (left) and Movement-type neurons (right). Filled columns represent significantly reward-modulated neurons. F, Time course of Movement-type activities around the pull onset (0 ms) (right), which are sorted by the RMI (left). Each symbol with a leftward line represents the onset to peak time of Movement-type activity. Filled symbols represent the significantly modulated neurons. Red and green represent D1-positive and D1-negative neurons, respectively (see Fig. 5 for details). Top horizontal bars indicate mean onset-to-peak time for the modulated (lower) and non-modulated (upper) neurons.

The ongoing and baseline spike rates, the spike duration and spike width, and the onset and peak of phasic activation were all defined in the same manner as described in our previous study (Isomura et al., 2009). Multineuronal activity in the motor cortex was further classified into regular-spiking (RS) and fast-spiking (FS) neuron groups according to our criteria (Isomura et al., 2009). The reward modulation index (RMI) was defined as (R – NR)/(R + NR); R is averaged spike rate of the Hold-, Movement-, or Reward-type activity in the rewarded trials and NR is that in the nonrewarded trials. If the RMI is >0, the activity is considered positively modulated by reward anticipation/acquisition. Similarly, the completion modulation index (CMI) was defined as (NR – E)/(NR + E); NR is the averaged spike rate of the Reward-type activity for correct responses in the nonrewarded trials, and E is that for premature (error) responses with the hold time < 0.8 s. The significance of reward modulation or completion modulation was judged in individual neurons using a Mann–Whitney U test. To compare the Movement-type activity in nonrewarded trials with that in rewarded trials in a similar behavioral condition (one of three behavioral parameters: hold time, pull time, or trial interval), we adjusted (reorganized) the database for nonrewarded trials to those for rewarded trials by collecting the nonrewarded trial in which the parameter value was closest to that in each rewarded trial. For example, to correct the database with hold time, if it was 1.241 s in the first rewarded trial, we picked up one nonrewarded trial that had the hold time closest to 1.241 (e.g., 1.237 s) out of all the nonrewarded trials. This process was repeated for all the rewarded trials one by one, with the picked-up one left available for remaining rewarded trials. The database corrections with the three parameters were performed in each of striatal neurons showing reward-modulated Movement-type activity.

Histological observations.

After the recording experiments, the rats were perfused intracardially with cold saline followed by 4% paraformaldehyde in 0.1 m phosphate buffer (PB) under deep anesthesia with urethane (2–3 g/kg, i.p.). Postfixed brains were sliced coronally into 80-μm-thick serial sections by a microslicer (VT-1200S; Leica Microsystems). The sections were subjected to an in situ hybridization to detect the mRNA expression for the dopamine receptor D1 subtype (Nishizawa et al., 2012), and then, to a fluorescent visualization for Neurobiotin, accompanied by an immunostaining for the striatal patch-marker μ-opioid receptor (Matsuda et al., 2009). Briefly, after a pretreatment with 100% methanol, they were incubated with denatured, digoxigenin (DIG)-labeled ribo-probe (rat D1r; Nishizawa et al., 2012) in HB4 solution (50% formamide, 5×SSC, 0.1% Tween 20) at 55°C overnight. The DIG was fluorescently visualized with 1 mg/ml Fast Red chromogen (Fast Red Tablets, 11496549001; Roche Diagnostics) after an overnight incubation with a sheep anti-DIG antibody conjugated with alkaline phosphatase (11093274910; Roche Diagnostics, 1:2000 dilution) in 1% blocking maleic acid buffer (Blocking Reagent, 11096176001; Roche Diagnostics). Then, the sections were incubated with a rabbit antibody against rat μ-opioid receptor (AB5511; Millipore, 1:500 dilution) at 4°C overnight, followed by an overnight incubation with a goat anti-rabbit antibody conjugated with Alexa Fluor 350 (A11046; Invitrogen, 1:125 dilution) along with streptavidin-Alexa Fluor 488 (S11223; Invitrogen, 1:250 dilution). The triple-stained sections were temporarily mounted on slides to check the mRNA expression for D1 receptor (red) in a Neurobiotin-loaded neuron (green) inside/outside the striatal patch areas (blue) under a fluorescence microscopy. Furthermore, some of them were returned to wells to re-stain the Neurobiotin-loaded neuron using the avidin-biotin-horseradish peroxidase complex (ABC method; Vectastain Elite ABC; Vector Laboratories, 1:200 dilution) with diaminobenzidine and nickel (Isomura et al., 2006, 2009). We mounted them on slides again, and observed the visualized neuron under the microscope after Nissl counterstaining with Neutral Red.

To confirm the striatum area receiving the corticostriatal projection from the FL area of the motor cortex, we injected the anterograde tracer, biotinylated dextran amines (BDA; 10,000 MW, Invitrogen) into the FL area iontophoretically (BAB-501, Kation Scientific; +5 μA, 7 s on/off for 30 min), and visualized the BDA-labeled axons by the same ABC method (after 5–7 survival days).

Data in the text and figures are expressed as the mean ± SD, if applicable, with statistical significance.

Results

Behavioral performance

In our experiments, a total of 67 rats were trained to perform the alternate-reward forelimb movement task in a head-fixed condition for eight training days (Fig. 1A), and we obtained enough experimental data from 63 rats for behavioral and electrophysiological analyses (Fig. 1B; total 689 ± 254 correct trials for 2 h in the recording experiment). A subject that had finished the task-training learned to pull the lever without a cue signal ∼1.0 s after the beginning of a lever-hold period, although they often made a false (too early) start (Fig. 1C, left). The onset of pull movements was significantly earlier in the rewarded trials than in the nonrewarded trials in most of the rats tested (54 of 63 rats; p < 0.05 in Mann–Whitney U test). In a group analysis, also, the median of lever hold time in each rat was much shorter in the rewarded trials than in the nonrewarded trials (Fig. 1D, left; rewarded 1.57 ± 0.30 s, nonrewarded 2.90 ± 1.30 s, N = 63, paired t test, p < 0.001). Obviously, these rats were able to anticipate the alternate delivery of reward and no reward for their correct task response. Thus, reward anticipation appeared to affect their reaction time as established well psychologically (Bowman et al., 1996; Kawagoe et al., 1998). The time for pull movement (i.e., pull velocity) was not largely different in the rewarded and nonrewarded trials in individual rats (Fig. 1C, right); nevertheless, the group analysis showed slightly but significantly faster pull movements in the rewarded trials (Fig. 1D, middle and inset; rewarded 93.8 ± 13.6 ms, nonrewarded 98.8 ± 14.2 ms; difference 5.0 ± 7.9 ms, N = 63, paired t test, p < 0.001). There was no difference between the mean time to complete a rewarded trial (i.e., nonrewarded success to subsequent rewarded success) and that in a nonrewarded trial (rewarded to nonrewarded) in the group data (Fig. 1D, right; rewarded 7.2 ± 2.5 s, nonrewarded 6.6 ± 3.5 s, p > 0.2), since the animals impatiently made false starts more frequently in the rewarded trials. These behavioral observations suggest that the rats successfully understood the alternate reward rule in the forelimb movement task.

Reward-modulated activation in striatal neurons

We obtained juxtacellular recordings from a total of 84 neurons in the dorsolateral striatum (a component of the skeletomotor loop of the basal ganglia), while the rats were performing the alternate-reward forelimb movement task. With respect to task-related discharge activity, the striatal neurons were classified into three functional groups: Hold-type, Movement-type, and Reward-type neurons (Fig. 2A,B; Hold-type, n = 21; Movement-type, n = 46; Reward-type, n = 11; others, n = 6). It is known that the width of spikes is statistically different among several neuronal subtypes in the striatum: shorter, middle, and longer spike widths roughly correspond to FS interneurons, medium spiny neurons (phasically active neurons), and cholinergic interneurons (tonically active neurons), respectively (Mallet et al., 2005; Inokawa et al., 2010). In our juxtacellular data, the three functional neuron groups (Hold-, Movement-, and Reward-types) commonly possessed a wide range of spike width (Fig. 2C, top), and in particular, the Movement-type neurons with modest spike rates (<1 Hz) during the hold period were clustered ∼0.1–0.2 ms in spike width (Fig. 2C, bottom). We confirmed that almost all of the Hold-type neurons (19 of 21) showed lower spike rates during the push movement than during the hold period (Fig. 2D, top; as an average). On the other hand, many of the Movement-type neurons (30 of 46) increased spiking rates during both the push and the pull movements (Fig. 2D, bottom). We did not further analyze the push movement because it was not controlled explicitly in our behavioral task condition.

Hereafter, we focused mainly on the Hold- and Movement-type neurons with a sufficient number of correct rewarded and nonrewarded trials for analysis (n = 21 and 42 neurons, respectively), because the Reward-type activity may be related with various behavioral events such as touching and licking the spout. As illustrated in Figure 2A, the Hold- and Movement-type activities were often augmented by the reward-anticipatory condition. The distributions of RMI in the Hold- and Movement-type neuron groups were biased toward positive reward modulation (Fig. 2E; Hold-type, RMI 0.16 ± 0.31, n = 21, one sample t test, p < 0.03: Movement-type, 0.17 ± 0.27, n = 42, p < 0.0002), and some of these neurons individually showed significant positive reward modulation (Fig. 2E; 6 Hold-type and 16 Movement-type neurons; p < 0.05 in Mann–Whitney U test). The duration (onset to peak) of reward-modulated Movement-type activity was longer than that of non-modulated activity (Fig. 2F; modulated 254 ± 163 ms, non-modulated 144 ± 109 ms, t test, p < 0.025), while there was no difference in the onset itself between them (modulated –134 ± 161 ms from the pull onset, non-modulated –96.8 ± 123 ms, p > 0.3). We found no significant correlation between the reward modulation and the spike width (data not shown). It should be noted that the reward modulation in the striatum (Fig. 2) is underestimated statistically compared with that in the motor cortex (see Figs. 7, 8), because the sample number of success trials during juxtacellular recording (striatum; 123 ± 50 trials) was much smaller than that during multineuronal recording (motor cortex; 874 ± 499 trials) in our experiments.

Figure 7.

Minor reward modulation in motor cortex neurons. A, Representative Hold-type (left) and Movement-type (right) neurons with no reward modulation in the FL area of motor cortex. Single units were isolated by spike-sorting software in multineuronal recording (top, spike waveforms and autocorrelogram). Calibration: 0.5 ms, 0.1 mV. B, Population ratios for the activity types in the motor cortex. C, Ongoing spike rate plotted against spike duration. We classified the isolated units to RS (triangles) and FS (circles) neurons by the spike duration (0.5 ms). D, RMI distribution for the Hold-type neurons (upper) and Movement-type neurons (lower). E, Time course of Movement-type activities around the pull onset (right), which are sorted by the RMI (left), for the RS (upper) and FS (lower) neurons.

Figure 8.

Minor reward modulation along different layers of motor cortex. A, No reward modulation in Movement-type neurons recorded in the superficial (left), middle (middle), and deep (right) layers of motor cortex. B, RMI distribution in Movement-type neurons at these cortical layers. C, Similar spatial profiles of averaged local field potentials (LFP) around the pull onset (Pull) between rewarded (left) and nonrewarded (right) trials (N = 12 rats).

It is possible that the positive reward modulation of Movement-type activity may merely result from the behavioral changes in reward-oriented (i.e., faster) pull movement. To test the possibility, we corrected the activity data for nonrewarded trials by adjusting one of three behavioral parameters (hold time, pull time, or trial interval; compare Fig. 1D; see also Materials and Methods for details), and compared them with the original activity data for rewarded trials. Figure 3A illustrates the Movement-type activities corrected in regard to the hold time, pull time, and trial interval. In this case, the Movement-type activity for rewarded trials was much larger than not only the original but also the three corrected Movement-type activities for nonrewarded trials. Likewise, all of the 16 Movement-type neurons preserved similar positive RMI values after these activity corrections (Fig. 3B; the original vs hold time r = 0.73, pull time r = 0.83, trial interval r = 0.42). It is, therefore, unlikely that the reward modulation was simply due to the behavioral difference in forelimb movement. This is totally consistent with the results obtained from the motor cortex (Figs. 7, 8).

Figure 3.

Reward modulation observed in similar behavioral performance. A, Correction of Movement-type activity in nonrewarded trials by one of three behavioral parameters (hold time, pull time, and trial interval; see Fig. 1D). The activity in rewarded trials (top left) was much larger than any corrected activity in nonrewarded trials (bottom). The same neuron as shown in Figure 2A. B, RMI distribution of all the reward-modulated neurons (black, 16 positively modulated neurons; gray, 1 negatively modulated neuron) before (abscissa) and after (ordinate) the activity correction in nonrewarded trials by hold time (left), pull time (middle), and trial interval (right). C, Left, A reward-modulated Movement-type activity (one of the above 16 neurons) showing its reduction for premature (error) pull responses in reward-expectable trials (hold time 0.2–0.8 s). Right, Relative activity change for premature responses plotted against RMI in the Movement-type neurons (n = 14 available for analysis). Red and green represent D1-positive and D1-negative neurons (Fig. 5). D, RMI distributions of all the Movement-type neurons during the pull movement (left, the same as shown in Fig. 2F), before the presentation of high-tone sound with or without reward delivery (middle, –0.5 to 0 s from the sound onset), and during the sound presentation (right, 0 to +0.5 s from it). Filled symbols represent significantly reward-modulated neurons.

We then examined the Movement-type activity for premature pull responses (i.e., false starts). A typical Movement-type neuron with positive reward modulation exhibited a greatly reduced activity for premature responses even in the reward-expectable condition (Fig. 3C, left). In the Movement-type neurons, the larger they showed reward modulation, the more they reduced the activity for error responses, except for several outlier-like neurons (Fig. 3C, right). Also, the positively biased reward modulation of Movement-type activity was abolished quickly after the presentation of high-tone sound, by which the animals knew a success in the trial [Figure 3D; before sound presentation, RMI 0.09 ± 0.26, n = 42, one sample t test, p < 0.04 (i.e., positively biased); during sound presentation, –0.13 ± 0.40, p < 0.05 (negatively biased); the bias shift was also statistically significant (p < 0.002)]. It strongly suggests that the reward modulation of Movement-type activity is associated with reward expectation rather than feedback information on reward acquisition or trial success/failure.

D1-independent reward modulation in striatal neuron subtypes

We visualized part of the juxtacellularly recorded striatal neurons, and examined whether they expressed the mRNA for the dopamine receptor D1 subtype by using a fluorescent in situ hybridization. Figure 4A shows an example of Hold-type neuron containing Neurobiotin (green) and expressing the D1 mRNA in the perikaryon (red). Later, its morphology was revisualized by the ABC method (dark brown). In this case, the Hold-type activity was not modulated by the reward alternation (Fig. 4B). The second case is a Movement-type neuron expressing the D1 mRNA (Fig. 4C), and the Movement-type activity was positively modulated by the reward alternation (Fig. 4D). These neurons may participate in direct striatopallidal/nigral inhibition, since most of the D1-positive neurons in the striatum are medium spiny neurons projecting their GABAergic axons to the GPi and SNr as the direct pathway (Gerfen, 2004; Tritsch and Sabatini, 2012). The last case is a Movement-type neuron with no D1 mRNA expression (Fig. 4E). Usually, the D1 mRNA is not expressed in either medium spiny neurons for the indirect pathway or aspiny interneurons (Tritsch and Sabatini, 2012). This D1-negative neuron had a lot of dendritic spines, suggesting it was a medium spiny neuron for the indirect pathway, probably expressing the D2 mRNA. The Movement-type activity was also positively modulated by the reward alternation (Fig. 4F).

Figure 4.

Morphological visualization of juxtacellularly recorded striatal neurons. A, Fluorescent labeling of a recorded striatal neuron (arrowheads) for Neurobiotin (left) and D1 mRNA (middle). The morphology was revisualized by subsequent ABC method (right). B, Hold-type activity of the same D1-positive neuron in rewarded (left) and nonrewarded (right) trials. Inset indicates its spike waveform. Calibration: 0.5 ms, 0.2 mV. C, D, Another D1-positive neuron, which exhibited reward-modulated Movement-type activity. E, F, A D1-negative medium spiny neuron with reward-modulated Movement-type activity.

We attempted to visualize 57 neurons morphologically of the 84 neurons that were recorded juxtacellularly (electrically), and obtained 27 successfully visualized neurons in the dorsolateral striatum that is densely innervated by the FL area of the motor cortex (Fig. 5A,B). Twenty-six of the 27 visualized neurons were located in the striatal matrix where the μ-opioid receptor was not expressed (Fig. 5C). We found 10 D1-positive neurons (Hold-type n = 5, Movement-type n = 5) and 17 D1-negative neurons (Hold-type n = 3, Movement-type n = 7, Reward-type n = 4, and others) (Fig. 5D). Thus, the Hold- and Movement-type neurons were not specific to the D1-negative or D1-positive subpopulation, respectively. There was a slight, but not statistically significant, tendency for the D1-negative Movement-type neurons to lengthen the onset-to-peak time of Movement-type activity, compared with the D1-positive neurons (Fig. 2F, colored symbols; D1-positive 176 ± 107 ms, D1-negative 396 ± 190 ms, t test, p = 0.0541). Importantly, there were no D1-positive neurons with large spike width (>0.17 ms) in our visualized neurons (Fig. 5E, left), which is consistent with the fact that the cholinergic interneurons (D1 negative) exhibit wider spikes than the other striatal neurons (Mallet et al., 2005; Inokawa et al., 2010), suggesting a high reliability of our juxtacellular visualization method. To our surprise, the reward modulation of Hold-type activity seemed independent of the expression of D1 mRNA (Fig. 5E, right; RMI 0.06 ± 0.22 and 0.17 ± 0.23 for D1-positive and D1-negative, respectively; p > 0.5). Similar result of the independence of D1 expression was obtained for the reward modulation of Movement-type activity (RMI 0.27 ± 0.40 and 0.39 ± 0.27 for D1-positive and D1-negative, respectively; p > 0.6).

As described above (Fig. 2A–C), we found the Reward-type activity that was an increase in the spike rate around an auditory signal for correct responses in the rewarded and nonrewarded trials. Figure 6A illustrates a Reward-type activity with positive reward modulation in response to the auditory signal for correct responses (high-tone sound). In addition, the neuron decreased the spike rate after another signal for error responses (low-tone sound). Thus, its activity was modulated positively by reward acquisition for correct responses (rewarded vs nonrewarded) as well as by task completion (correct vs error). Overall, however, the Reward-type group seemed very heterogeneous in terms of spike coding for reward expectation/acquisition and task completion (Fig. 6B).

Figure 6.

Reward-type activity in striatal neurons. A, A reward-type activity that was modulated by reward acquisition as well as by trial completion. This neuron differentially responded to auditory signal for correct responses (upper left and right; 11 kHz, with and without reward delivery, respectively) and for error responses (lower; 3 kHz, without reward). B, Diverse modulation of Reward-type activity (n = 11 available for analysis) by reward acquisition and trial completion. RMI was calculated using spike activity (0–500 ms after the onset of auditory signal) in the presence and absence of reward delivery for correct responses. CMI was also calculated using the spike activity for correct responses with no reward and for error responses. Filled and open symbols represent RMI with and without statistical significance (p < 0.05), respectively. Triangles and circles represent CMI with and without statistical significance, respectively.

Minor reward modulation in motor cortex neurons

Given that the reward modulation in the striatal neurons is biased to positive in a dopamine-independent manner (Fig. 5), reward information may be conveyed from the motor cortex rather than the substantia nigra. Therefore, we checked whether the functional activity of motor cortex neurons was changed by the reward alternation. By using the multineuronal recording, we obtained a total of 216 neurons in the FL area of the motor cortex in 24 rats performing the behavioral task. We again classified them into the three functional groups: Hold-type, Movement-type, and Reward-type neurons (Fig. 7A). Unlike the striatum, we rarely found the Reward-type neurons in the motor cortex (Fig. 7B; Hold-type, n = 46; Movement-type, n = 106; Reward-type, n = 6; others, n = 58; cf., Fig. 2B). We also divided them into 164 RS and 52 FS neurons by spike duration (Fig. 7C; spike duration < 0.5 ms for FS). Many of the FS neurons displayed the Movement-type activity, but not the Hold-type activity (RS: Hold-type 45, Movement-type 69; FS: Hold-type 1, Movement-type 37), which is consistent with our previous observation (Isomura et al., 2009).

As illustrated in Figure 7A, most of the motor cortex neurons were not largely modulated by the reward alternation. Although the Hold-type motor cortex neurons were slightly but significantly shifted to positive in the distribution of reward modulation (Fig. 7D; RMI 0.057 ± 0.137, one sample t test, p < 0.007), the Movement-type neurons as a whole showed no such bias in the reward modulation (–0.023 ± 0.173, p > 0.19). This may reflect the behavioral difference observed for hold and pull movements shown in Figure 1C and D. Nevertheless, some of the RS neurons with the Movement-type activity were modulated by the reward either positively (n = 8) or negatively (n = 12; p < 0.05 in Mann–Whitney U test). These RS neurons exhibited a significantly earlier onset and a longer onset-to-peak duration of the Movement-type activity compared with non-modulated RS neurons [Figure 7E; onset: modulated (n = 20) –109 ± 110 ms from the pull onset, non-modulated (n = 46) 2 ± 90 ms, t test, p < 0.0001; duration: modulated 155 ± 82 ms, non-modulated 77 ± 59 ms, p < 0.0001]. On the other hand, the time course of Movement-type activity in the FS neurons was not affected by the reward modulation [onset: modulated (n = 6) –40 ± 115 ms, non-modulated (n = 30) 5 ± 85 ms, p > 0.9; duration: modulated 127 ± 109 ms, non-modulated 110 ± 80 ms, p > 0.6].

It is known that the corticostriatal pyramidal cells are located in deeper layers (e.g., layer 5A and B) of the motor cortex in rodents (Anderson et al., 2010; Morishima et al., 2011). Our Movement-type neurons were obtained from the superficial (400 μm or 800 μm in depth, putative layers 2/3, n = 28), middle (1200 μm, putative layer 5, n = 45), and deep (1600 μm, putative layer 6, n = 29) layers in the FL area of the motor cortex. We found no largely biased distribution in the reward modulation of Movement-type activity in these subpopulations, although a small number of neurons still had positive or negative reward modulation (Fig. 8A,B). We also observed similar spatial profiles of averaged local field potentials around the pull onset in the rewarded and nonrewarded trials (Fig. 8C; N = 12 rats). Thus, the vast majority of motor cortex neurons seemed to be specialized to process motor information with no reward information, unlike their partner striatal neurons.

Discussion

In the present study, we investigated how motor information and reward information are represented and integrated by individual neurons in the dorsolateral striatum and the motor cortex of the rat skeletomotor loop. So far, there have been no experimental methods available to identify the functional subtype of each neuron of which spiking activity was recorded during voluntary movements. A combination of our task-learning system and juxtacellular recording with in situ hybridization made it possible to visualize individual recorded neurons in behaving animals very efficiently and reliably. The experimental techniques revealed that the striatal neurons, regardless of the expression of dopamine D1 receptors (positive to direct pathway neurons), exhibited functional motor-related activations during a halt or an execution of voluntary forelimb movement (i.e., Hold-type and Movement-type). In many of these striatal neurons, such functional activations were positively modulated by expecting a reward given after the motor response. In contrast, the motor cortex neurons at any cortical layers exhibited only minor and unbiased reward modulation of their activation in relation to the execution of forelimb movement. Our results suggest that striatal neurons for the direct (putatively enhancing a movement) and the indirect (putatively suppressing it) pathways are activated concurrently, contrary to a simple prediction in the classical basal ganglia model, and that reward expectation augments subcortical motor information processed by the striatal neurons for both pathways within the skeletomotor loop.

According to the DeLong's model, the excitation of striatal neurons for the direct and indirect pathways, respectively, enhances and suppresses the activity of cortical neurons antagonistically (DeLong, 1990). Recent behavioral works combined with gene modification techniques in rodents conclusively support this classical model in terms of reinforcement learning (Hikida et al., 2010; Kravitz et al., 2012) as well as motor balance (Sano et al., 2003; Bateup et al., 2010; Hikida et al., 2010; Kravitz et al., 2010). Then, one may naively presume that the direct pathway neurons of the skeletomotor loop increase the spike activity during voluntary movements, whereas the indirect pathway neurons do so in the absence of movements. However, we found, to all appearances, no large difference in the motor-related activation between D1-positive and D1-negative neurons in the dorsolateral striatum. Very recently, Cui et al. (2013) have measured mass calcium transients specific to the direct and indirect pathways in the dorsolateral striatum of behaving mice and have shown that these two pathways are concurrently and similarly activated during voluntary movements. Our results obtained at a single-cell resolution level favor this “hot ” observation, which challenges the classical view of basal ganglia function (for review, see Nambu, 2008).

These findings suggest that the antagonistic pathway system may be achieved by a more intricate involvement of the two sets of striatal neurons in our behavioral task condition. For the Hold-type activity, for example, the D1-positive neurons might prepare for the intended movement or just maintain the overall muscle tone, while the D1-negative neurons might serve to avoid temporally incorrect expression of the movement. For the Movement-type activity, the D1-negative neurons might suppress concurrent expression of antagonistic muscular (or spatially unnecessary) movements, while the D1-positive neurons would surely contribute to the execution of intended movement itself. In other words, even the simple movement actually consists of a complex of spatiotemporal motor components, and individual neurons for the direct and indirect pathways may antagonistically code subprograms for the motor components to complete the whole movement cooperatively. Thus, this notion can be accounted for by postulating several cooperative motor functions in the antagonistic pathway system.

It is well known that reward-related or motivational information is processed not only by the ventral striatum (including the nucleus accumbens), but also by the dorsal striatum (the caudate nucleus and putamen) in primates (for the ventral striatum, Bowman et al., 1996; Cromwell and Schultz, 2003; for the dorsal striatum, Cromwell and Schultz, 2003; Samejima et al., 2005) and rodents (Roesch et al., 2009; for the ventral striatum, Ito and Doya, 2009; Stalnaker et al., 2010, 2012; for the dorsal striatum, Oyama et al., 2010; Goldstein et al., 2012; Tai et al., 2012). The SNc dopamine neurons send reward-related information, specifically reward prediction error signals (Schultz et al., 1997; Kawagoe et al., 1998; Satoh et al., 2003; Roesch et al., 2007) to the striatal projection neurons through densely arborized axonal branches (Matsuda et al., 2009). Dopaminergic transmission is obviously essential for motor information processing in the dorsal striatum, because the depletion of dopamine from the dorsal, but not the ventral, striatum impairs motor performance in a lever press-release task in rats (Amalric and Koob, 1987). Dopamine promotes the activation of D1-expressing striatal projection neurons constituting the direct pathway and diminishes that of D2-expressing neurons constituting the indirect pathway (Gerfen and Surmeier, 2011; Tritsch and Sabatini, 2012). It is, therefore, very likely that the dopamine signals carrying reward information affect motor information represented by projection neurons in the dorsal striatum, although the reward information may not be essential for voluntary movements (Yin et al., 2004). Indeed, D1- or D2-selective blockade of dopaminergic transmission in the dorsal striatum occluded reward-dependent alterations of saccade performance in monkeys (Nakamura and Hikosaka, 2006).

But, does dopamine clarify every event that we observed here? Primates usually display both positive and negative correlations with reward expectation by the activity of dorsal striatum neurons (Cromwell and Schultz, 2003; Samejima et al., 2005). On the other hand, rodents often exhibit a strong populational shift to the positive direction in the reward correlation of dorsal striatum neurons (Oyama et al., 2010; Stalnaker et al., 2012), as also shown in our present experiments. If dopamine alone regulated the activity of the two groups of striatal projection neurons, it might conflict with the positive shift of their reward modulation in rodents, for they express the D1 and D2 receptor subtypes exclusively at the exact 1:1 ratio (Gerfen, 2004). Moreover, the reward-susceptible activation of the dorsal striatum neurons was temporally more diverse (i.e., reward modulations observed for Hold-, Movement-, and Reward-types) than the reward prediction error signals generated by dopamine neurons in the SNc. Accordingly, we should consider the possibility that the dorsal striatum neurons receive synaptic inputs carrying reward information from cortical or subcortical (nondopaminergic) neurons, as well as the SNc dopamine neurons, within the skeletomotor loop. Yet, we found only small and unbiased reward modulation (especially for the Movement-type activity) in the motor cortex where pyramidal neurons in layer 5 send direct axonal projections to the dorsal striatum. The additional reward information might be conveyed to the dorsal striatum through subcortical structures such as the thalamus (Doig et al., 2010). Alternatively, the biased reward modulation might result merely from altered synaptic plasticity in the striatal projection neurons under the influence of a combination of dopamine and other neuromodulators (Shen et al., 2008).

Together, although the direct and indirect pathways are certainly balanced in basal ganglia (DeLong, 1990), the two-pathway system appears to work, not just antagonistically, but rather cooperatively by various functional activities of individual striatal neurons that contribute to an integration of motor and reward information. In fact, selective elimination of the direct or indirect pathway resulted in an extension of the response time or a decrease in the response accuracy, respectively, in reinforced learning (Fukabori et al., 2012; Nishizawa et al., 2012), suggesting that these pathways are not just functionally opposite. This is supported by anatomical evidence that the direct pathway neurons send axonal projections not only to GPi/SNr neurons, but also to GPe neurons (Kawaguchi et al., 1990; Gerfen, 2004) with different axonal arborization patterns from the indirect pathway neurons (Fujiyama et al., 2011). Furthermore, main information processed through the direct and indirect pathways may be affected by the cortico-subthalamo-pallidal (hyperdirect) pathway, another important pathway in the basal ganglia (Nambu et al., 2002; Nambu, 2008; Inoue et al., 2012). In conclusion, our findings in favor of the reward-modulated motor information represented in identified striatum neurons will shed light on functional cooperation of individual neurons in basal ganglia-related networks to accomplish goal-oriented behaviors.