Multiple neuronal circuits for variable object–action choices based on short- and long-term memories

Abstract

At each time in our life, we choose one or few behaviors, while suppressing many other behaviors. This is the basic mechanism in the basal ganglia, which is done by tonic inhibition and selective disinhibition. Dysfunctions of the basal ganglia then cause 2 types of disorders (difficulty in initiating necessary actions and difficulty in suppressing unnecessary actions) that occur in Parkinson’s disease. The basal ganglia generate such opposite outcomes through parallel circuits: The direct pathway for initiation and indirect pathway for suppression. Importantly, the direct pathway processes good information and the indirect pathway processes bad information, which enables the choice of good behavior and the rejection of bad behavior. This is mainly enabled by dopaminergic inputs to these circuits. However, the value judgment is complex because the world is complex. Sometimes, the value must be based on recent events, thus is based on short-term memories. Or, the value must be based on historical events, thus is based on long-term memories. Such memory-based value judgment is generated by another parallel circuit originating from the caudate head and caudate tail. These circuit-information mechanisms allow other brain areas (e.g., prefrontal cortex) to contribute to decisions by sending information to these basal ganglia circuits. Moreover, the basal ganglia mechanisms (i.e., what to choose) are associated with cerebellum mechanisms (i.e., when to choose). Overall, multiple levels of parallel circuits in and around the basal ganglia are essential for coordinated behaviors. Understanding these circuits is useful for creating clinical treatments of disorders resulting from the failure of these circuits.

How we feel, think, and act is completely controlled by the brain, which contains many cells, including neurons and glial cells (1, 2). These amazing abilities are largely based on the connections between neurons. Many neurons are connected to many other neurons through many synapses. These connections evolve or disappear during development, learning, and aging. Moreover, different neurons transfer different information to other neurons and do so depending on history, context, or prediction. Overall, the brain is created and retained by many different types of physical–chemical mechanism.

These facts suggest another perspective: If any of the mechanisms works improperly, the brain may become dysfunctional in various manners, causing various kinds of behavioral, mental, and emotional disorders. In order to develop treatments for these disorders, we need to know which mechanisms (among many) are dysfunctional in patients with brain disorders. This requires many types of testing, including recording and manipulation of single neuronal activity and synapses across many brain areas. Under these complex conditions, monkeys would play a critical role because they are evolutionarily close to humans. Indeed, their brains and behaviors are similar to human. In our experiments, monkeys and humans often learned and performed the same behavioral tasks in a similar manner and with a similar excitement, while their neuronal circuit mechanisms were studied at the same and different dimensions, as shown below.

Basic Mechanism of Behavioral Control by Basal Ganglia

Many kinds of behavioral and mental disorders are caused by dysfunctions of the basal ganglia. A basic mechanism of behavioral and emotional control by the basal ganglia was discovered by examining their effects on saccadic eye movement (3), which is controlled mainly by the superior colliculus (SC) (4, 5). As shown in Fig. 1 A and B, neurons in the intermediate layer of the SC fire a burst of spikes and send the signal to the brainstem saccade generator (SG) (6), which generates a saccadic eye movement to a particular position in the contralateral visual hemifield (e.g., activation of SC neurons on the right side generates a leftward saccade) (7). The SC receives excitatory inputs from many cortical regions (8), but GABAergic inhibitory inputs mainly from the substantia nigra pars reticulata (SNr) (Fig. 1 A and B) (9–11). SNr neurons fire tonically with high frequency and thus keep inhibiting SC neurons (Fig. 1B).

Fig. 1.

Basic mechanism of basal ganglia. (A) Medial view of macaque monkey brain, showing neuronal circuits for saccadic eye movement in the basal ganglia and the brainstem. (B) Neuronal activity in the CD-SNr-SC circuit that facilitates saccade. (C–E) Neuronal activity and saccade latency during memory-guided saccade in 2 versions: 1DR and ADR. (C) Population activity of DA neurons in the SNc in response to target cue that predicted reward [red: R(+)] and no reward [blue: R(-)] during the 1DR task and during the ADR (gray). Reprinted from ref. 24. (D) Population activity of CD neurons. Reprinted from ref. 24. Same format. (E) Distribution of saccade latencies: Reward-predicted (red), no-reward-predicted (blue). Reprinted from ref. 22.

Notably, such tonic inhibition is common among the output of the basal ganglia, not only the SNr but also the globus pallidus internus (GPi) (12). In order to examine the significance of the tonic inhibition, we made a temporary lesion method (local injection of GABA agonist, muscimol) and inactivated SNr neurons (13). This produced irrepressible saccades to the contralateral side in monkeys (13) as well as rats (14). These results suggest that the fundamental function of the basal ganglia is to keep suppressing unnecessary movements. In fact, people with various basal ganglia disorders often make body movements involuntarily and continuously (e.g., locomotion, reaching, eye movement) (15, 16). Such unnecessary body movements may occur temporarily in a particular context (17), which we will describe later (see, for example, Fig. 4).

Fig. 4.

Local parallel circuits within the CDt-cdlSNr-SC circuit for object choice. (A) Direct and indirect pathways from the CDt, both targeting the SC. The direct pathway is already shown in Figs. 2 and 3. The indirect pathway is mediated by the cvGPe. Information on good and bad objects is transferred mainly by the direct and indirect pathways, respectively. Reprinted from ref. 48. Copyright (2017), with permission from Elsevier. cdlSNr neurons are thus inhibited by good objects and excited (disinhibited) by bad objects. (B) Procedure of memory-guided saccade task. Left vertical line: onset of the target cue. Center vertical line: offset of the center spot. Right vertical line: onset of the target. Reprinted from ref. 52. Copyright (2011), with permission from Elsevier. (C) Eye movements in human subjects (3 groups) during memory-guided saccades in 2 phases: Response to target cue (Left) and memory-guided saccade (Right). Data (top part is eye position; bottom part is eye velocity) are shown for a representative subject in each of 3 groups: Control (normal), early Parkinson’s disease (early PD), advanced Parkinson’s disease (advance PD). Reprinted from ref. 52. Copyright (2011), with permission from Elsevier. In each trial the target cue was presented at one of 8 positions: 5, 10, 20, 30° right (top row) or left (bottom row).

In addition, we found that SNr neurons stop firing occasionally and briefly, during which a saccade may occur (Fig. 1B) (18). The saccade is promoted by the phasic reduction of SNr-SC inhibition (i.e., disinhibition). We found that the disinhibition occurs based on another GABAergic inhibitory connection from the caudate nucleus (CD) (19).

Predicted Control of Behavior Based on Short-Term Memory

Taken together, these data revealed a basic mechanism of the basal ganglia: Tonic inhibition and phasic disinhibition. What kind of information is used by this mechanism to promote action? We found that predicted reward value is one important signal. This answer was provided by a new behavioral task we call the 1 direction reward task (1DR task) (20). While the subject was fixating on the central dot, the target position was indicated briefly for an upcoming memory-guided saccade (target cue) (see, for example, Fig. 4B). In each block of 30 trials, reward was given when the target appeared at 1 of 4 target positions; the reward position changed across the blocks.

Using the 1DR task, we recorded neuronal activity in a particular position of the CD where visual-saccadic neurons are mostly located (20). Fig. 1D shows the population activity of CD neurons in response to the target cue. A majority of visual CD neurons responded to the target cue differently depending on the predicted reward outcome; typically, response was higher if reward was predicted (Fig. 1D, red) and lower if no reward was predicted (Fig. 1D, blue) than when reward was equally predicted across the positions (all-direction reward task: ADR task) (Fig. 1D, gray).

Notably, changes in reward prediction were correlated with changes in action (i.e., saccade) (21, 22): Shorter latency if reward was predicted (Fig. 1E, red). This may be explained by changes in the state of the CD-SNr-SC circuit. If CD neurons respond to the target cue more strongly, SC neurons would be activated more strongly (due to the stronger disinhibition) and therefore a saccade starts earlier.

How do CD neurons change visual responses based on the predicted reward value? This is, at least partially, caused by dopamine (DA) neurons. We recorded activity of DA neurons in the middle part of the substantia nigra pars compacta (SNc), some of which project to the CD (23). They were excited by the target cue if it predicted reward, while inhibited if no reward was predicted (Fig. 1C) (24). These responses were based on recent changes in the predicted reward, which is typically called “reward prediction error” (25, 26). We also found that the DA input to the CD had causal effects on saccade behavior, since the local injection of DA-D1 or DA-D2 antagonist in the CD elongated saccade latency differently (27, 28).

Basic mechanism and function of the basal ganglia (described above) are illustrated as the left circuit in Fig. 2B. The CD shown in Fig. 1 is actually the caudate head (CDh), which is functionally different from caudate tail (CDt). We found that the CDh sends signals to the SC through the rostral-ventral-medial part of the SNr (rvmSNr) (29). To summarize, the CDh-rvmSNr-SC circuit controls saccades by changing its activity quickly based on predictive reward values. This relies on short-term memories (or working memories), which encode recent events (e.g., change in object value) (29). Long-term memories encoding old events may not be useful. This is probably a controlled (not automatic) process and the capacity of the underlying memories is limited (30, 31).

Fig. 2.

Parallel circuits in the basal ganglia using short-term and long-term object–value memories. (A) Shape of CD nucleus in macaque monkey. (B) Parallel circuits originated from the CDh and CDt, which have serial GABAergic inhibitory connections to the SC. They are mediated by separate groups of SNr neurons in the rvmSNr and cdlSNr. These circuits are controlled by separate groups of DA neurons in the rvmSNc and cdlSNc.

Such a flexible process is enabled by inputs from a particular group of DA neurons that encode reward prediction error (i.e., recent increase or decrease of reward outcome) (32, 33). They are shown in Fig. 2B as “update-value” DA neurons, which are located in the rostral-ventral-medial SNc (rvmSNc) and are close to GABAergic neurons in the rvmSNr.

Historical Control of Behavior Based on Long-Term Memory

As shown in Fig. 2A, the CD nucleus extends from the CDh caudally and ventrally to a thin but long structure called the CDt. Such a shape is relatively unique to humans and monkeys (34), suggesting that they share the same brain mechanisms and behaviors.

Importantly, the CDt also controls saccadic eye movement using a separate circuit to the SC through the caudal-dorsal-lateral SNr (cdlSNr) (Fig. 2B), which is often called substantia nigra pars lateralis (35). The CDt receives inputs mainly from visual cortical areas, including the inferotemporal cortex (36, 37). Indeed, most neurons in the CDt respond to visual stimuli (38, 39). We used computer-generated fractal objects as visual stimuli (Fig. 3A) (39) and found 2 critical features. First, CDt neurons respond to fractal objects very selectively (e.g., 1 of 8 objects), even when the subject saw them for the first time. Second, CDt neurons have spatial selectivity, responding to visual objects presented on the contralateral hemifield.

Fig. 3.

Long-term high-capacity memories of object values encoded by cdlSNr neurons. (A) Fractal objects learned by monkey G repeatedly (>5 d), each of which was associated with a larger reward (good) and a small reward (bad). Reprinted from ref. 40. (B) Responses of an SC-projecting cdlSNr neuron to 120 objects (60 good, 60 bad) that had been learned more than 3 d before. Reprinted from ref. 40. These objects were presented in the neuron’s receptive field when the monkey was fixating gaze at center. In this passive viewing task, reward was delivered randomly, with no association with each object. (C) Averaged responses of 151 cdlSNr neurons in the passive viewing task. Reprinted from ref. 40. (D) The CDt-cdlSNr-SC circuit that receives inputs mainly from visual cortical areas and mainly controls saccade.

Why are there 2 parallel circuits within the basal ganglia in order to control saccadic eye movements? We first found that visual neurons in the CDt-circuit (CDt and cdlSNr) showed no change in their responses to visual objects even when the reward outcome changed, although the monkey changed their choice quickly (i.e., whichever is recently associated with a big reward) (40, 41). This feature was completely different from neurons in the CDh-circuit (Fig. 1).

In real life we (and animals) experience many objects whose values are recognized initially and do not change later. We thus made a new task for such stable values (stable value learning). Many fractal objects were presented one at a time, after which either a big or small reward was presented. Half of the objects were always followed by a big reward (good objects); the other half was followed by a small reward (bad objects) (Fig. 3A). During the object–value learning across days, the monkey became attracted by good objects. This is shown consistently in a free viewing task (40, 41): When some of these objects were presented at the same time, the monkey tended to look at good objects, avoiding bad objects, even though no reward was given. Thus, the gaze biased to good objects is an automatic behavior.

We then found that the automatic gaze bias is controlled by the CDt-cdlSNr-SC circuit (Fig. 2 B, Right) (40, 41). This was shown clearly by cdlSNr neurons during the passive viewing task (Fig. 3 B and C) (40). In this task, while the monkey was fixating a gaze at the center, some of the learned objects were presented one at a time in the neuron’s receptive field (i.e., contralateral hemifield). Virtually all neurons in the cdlSNr were inhibited by good objects and excited by bad objects even when there was no consistent reward outcome that had been associated with each object. These robust activities may originate from the CDt (41). The inhibition of cdlSNr neurons would and actually did disinhibit the SC neurons (Figs. 1 and 4), facilitating automatic saccades to good objects (37). In contrast, the excitation of cdlSNr neurons inhibited SC neurons more strongly, suppressing saccades to bad objects. Based on the outcome-irrelevant information, the CDt-cdlSNr-SC circuit can generate saccades to good objects automatically (described above), which is a clear contrast to the CDh (Fig. 1) (42) and rvmSNr (29).

After the stable value learning, we sometimes stopped showing the learned objects for a long time (e.g., >100 d), during which the monkey continued to see many other fractals (e.g., >100 objects). Surprisingly, cdlSNr neurons still showed clear value-coding (i.e., inhibited by good objects and excited by bad objects) (40). The CDt-SNr-SC pathway continuously discriminated learned objects to make biased saccade-gaze to good objects, even after more than 1 y (37). These data suggest that the CDt-cdlSNr-SC circuit encodes long-term memories of object values, which may persist across the whole life course and may be called “historical values” (Fig. 2).

We then let the monkeys experience many fractal objects with different values, sometimes more than 1,000 objects. Fig. 3A shows well-learned objects for 1 monkey. Many days after the learning, cdlSNr neurons discriminated virtually all of them (Fig. 3B) and the monkey looked at any of the good objects during the free viewing task (40). These results indicate that the long-term memory in the CDt-cdlSNr-SC circuit has very high capacity (Fig. 2B). These features (i.e., long-term and high-capacity) are important because many objects we encounter have often not been experienced recently. If the CDt-cdlSNr-SC circuit does not work normally, we cannot choose good objects based on their historical values, which we actually showed experimentally (42).

Importantly, the CDt-cdlSNr-SC circuit, which acts automatically, plays an essential role in goal-directed behavior. When some of the learned objects (e.g., 1 good and 8 bad objects) are presented at the same time, the monkey typically made a single saccade to the good object, looked at it, and got a big reward (43). The targeting saccade is often very quick, with short latency (sometimes <150 ms). These results are rather unexpected because the goal-directed saccade occurred based on peripheral vision. Indeed, neurons in the CDt and cdlSNr can discriminate visual objects located in periphery. In real life, we (and animals) need to find a valuable object that is located among many useless objects. Without the CDt-cdlSNr-SC circuit, we would need to make saccades to all of these objects until finally finding a good object, which is again a waste of energy.

The difference between the CDh-rvmSNr-SC circuit and CDt-cdlSNr-SC circuit may be supported partially by dopaminergic inputs. Separate groups of DA neurons project to the CDh and CDt (Fig. 2B) (44), sending different signals (i.e., update vs. sustain) that are relevant to short- vs. long-term memory (33). Update-DA neurons are selectively sensitive to predicted reward values, while sustain-DA neurons are more sensitive to historical reward values (33). They are located separately in the SNc. These areas have another difference: Threat-predicting stimulus inhibits DA neurons in the rvmSNc, but excites DA neurons in cdlSNc (i.e., value vs. salience) (32).

Parallel Circuits: Direct and Indirect Pathways for Good–Bad Discrimination

According to the scheme in Fig. 2B, the CD (CDh or CDt) inhibits the SNr (rvmSNr or cdlSNr), disinhibits the SC, and facilitates a saccade to a target. It has been known that the output of the striatum (CD or putamen) is transferred to the basal ganglia output (i.e., SNr or GPi), either directly (direct pathway) or indirectly through the globus pallidus externus (GPe; indirect pathway) (45). These pathways are composed of GABAergic inhibitory connections, suggesting that they have opposite effects on action: Facilitation by direct pathway (because of 2 serial inhibitions), suppression by indirect pathway (because of 3 serial inhibitions).

Indeed, locomotion of mice changed oppositely by these pathways (by optogenetic activation): Facilitation by the direct pathway and suppression by the indirect pathway (46). On the other hand, these pathways become active simultaneously in action initiation and termination (47). These seemingly inconsistent data may be explained by different contexts of the parallel circuits (direct and indirect pathways), as shown below.

The CDt projects to the caudal-ventral GPe (cvGPe), which then projects to the cdlSNr (48, 49), in addition to its direct connection to the cdlSNr (Fig. 4A). We found that many cvGPe neurons were strongly inhibited by bad objects (48), which should disinhibit SNr neurons and suppress saccades to bad objects. This is opposite to the effect of the direct pathway (CDt-cdlSNr), which facilitates saccades to good objects.

In other words, the direct pathway is active if a good object appears, while the indirect pathway is active if a bad object appears. Such independent mechanisms are consistent with the data by Kreitzer and colleagues (46). In a different context, a good object and a bad object appear simultaneously. Then, it is necessary to reject the bad object and accept the good object at the same time. Therefore, the direct and indirect pathways, together, must be active simultaneously to make a correct choice. This is probably equivalent to the data by Costa and colleagues (47), in which different body parts may be activated and inactivated simultaneously to perform a particular action.

There is another context in which the indirect and direct pathways act sequentially, rather than independently or simultaneously. When we experience new objects, we first need to check the outcomes (e.g., rewarding or not) of all objects. For correct choice, we then need to start rejecting bad objects. This is actually what all monkeys (we tested) did. When the inhibitory input to the cvGPe from the CDt was suppressed by local injection of bicuculline (GABA antagonist) into the cvGPe, the monkeys became unable to reject bad objects (50). Activation of the indirect pathway disrupted the subsequent action sequence (51). Therefore, the indirect pathway is particularly important for learning of goal-directed behavior.

These data may be related to basal ganglia dysfunctions. One example is shown by people with Parkinson’s disease (Fig. 4 B and C) (52). Here, we asked them to make memory-guided saccades. After pressing a lever, a small spot of light appeared at the center. While the subject kept fixating gaze on the center spot, another spot (target cue) appeared briefly on the left or right (left vertical line), which indicates the goal position of the upcoming saccade. The saccade is supposed to be made after the center spot disappears (center vertical line), which is guided by short-term memory of target cue.

Most adult people were able to follow this rule (Fig. 4 C, Top). They kept fixating (i.e., not making saccades to target cue) and made a saccade to the remembered position. People with Parkinson’s disease often made a saccade to the target cue (even though asked not to do so), yet had difficulties in making memory-guided saccades (Fig. 4 C, Bottom). These unnecessary behaviors occurred more often in people with advanced levels of Parkinson’s disease. Similar problems were observed in other groups of people with basal ganglia deficiencies: DA deficiency (53) and Tourette’s syndrome (54).

Our experimental neuronal data suggest the underlying mechanism for memory-guided saccade (Fig. 4C, related to Fig. 4A). First, the indirect pathway should be active before making a saccade, even when a visual stimulus (target cue) appears. Second, the direct pathway should be active when we are ready to make the saccade (55). Since many contexts emerge in real life, selective activation and suppression of parallel circuits would be critical. Dysfunctions of such parallel mechanisms may cause serious behavioral disorders, which appear in Parkinson’s disease and others. In this sense, we now need to test more complex behavioral procedures for many patients and control subjects, including monkeys.

These data together suggest that parallel circuits controlling the same action—CDh and CDt circuits (Fig. 2B), direct and indirect pathways (Fig. 4A)—can generate multiple functions that are appropriate in multiple contexts. We predict that there are similar parallel circuits for different actions.

Parallel Circuits: Cerebrum, Basal Ganglia, and Cerebellum for Action Skill

So far, we have shown multiple levels of parallel circuits in the basal ganglia that control a selective behavior (e.g., saccade) (Figs. 2B and 4A). On the other hand, natural behavior is controlled by many actions (e.g., eye, hand, body movements), thoughts, and emotions. These ideas raise another question: How can so many circuits work cooperatively or competitively?

As an attempt to address these questions, we made an eye–hand sequence task for both monkeys (2 × 5 task) (56) and humans (2 × 10 task) (57). The subject has to press 5 or 10 pairs of illuminated buttons, in the correct order for each pair (Fig. 5A), to get a rewarding outcome. The sequence of the pairs was fixed in a block of trials (e.g., 20 correct trials). Many sequences can be created, since 1 pair of illuminated buttons was chosen randomly from 16 (4 × 4) buttons. Many subjects, especially monkeys, learned many sequences (e.g., 8 for the left hand, 8 for the right hand), until their performances became very skillful (fast with no error) (Movie S1) after long learning (>30 d). This procedure is useful because monkeys and humans learn and perform the task very similarly, including hand movements. Moreover, virtually all subjects loved this task, like playing a game.

Fig. 5.

Parallel circuits for eye–hand sequential action. (A) The 2 × 5 task for monkey subject; 2 × 10 task for human subjects (not shown). (B) Monkey’s skillful hand movement from set 2 (second button) to set 3 (first button) in A. Reprinted from ref. 61. The hand started moving from the second button of set 2 (Button Release) to the first button of set 3 (A). These movements started before set 3 buttons were illuminated (Target On). Eye movement (saccade) preceded the hand movement and the eye gaze arrived at the first button of set 3 before the illumination (not shown). Reprinted from ref. 61. (C) Parallel circuits for eye–hand sequential action, including the basal ganglia, cerebral cortex, and cerebellum. Information differences between these circuits are shown in detail.

Along the repeated learning across days, the performance changed drastically (as shown in Movie S1). In the early learning stage, the rate of correct button press increased quickly within a block of trials, while the speed of action (hand movement) increased only slightly (56). The correct choice, however, was not retained completely in the next test (e.g., 1 d later) (56), suggesting that short-term memories were used (Fig. 5 C, Left) (58). In the late stage, both the correct choice and the action speed were very high. Such skillful behavior (especially, speed) was retained for a long time (e.g., 18 mo) after the last learning in both monkeys and humans (59, 60), suggesting that long-term memories were used (Fig. 5 C, Right).

We found that skillful behavior has several unique features, many of which have been known. Anticipatory: During skillful performance of a sequence, both eyes and hand move toward the target position before it is illuminated (Fig. 5B) (61). Unilateral: After becoming skillful using only 1 hand, the skill is abolished if the other hand was used (59). Subconscious: During skillful performance of an old sequence (>1 y later), the subject does not recognize the sequence consciously (60). Rhythm: During learning, the performance of a long sequence starts with a rhythm with several chunks (e.g., 3-2-2-3) (62).

We then found that different brain areas are involved in different stages of learning. In the basal ganglia, the anterior striatum (including the CDh) is responsible for early learning, while the posterior striatum (including posterior putamen) is responsible for late learning (Fig. 5C) (63, 64). In the anterior striatum, neurons were more active during early than late learning and temporary inactivation by muscimol disrupted early rather than late learning. In the posterior striatum, neurons were more active during late learning and inactivation disrupted late learning. These results are analogous to the parallel object–value circuits (Fig. 2).

Importantly, the neuronal mechanism of eye–hand sequential behavior also includes the cerebral cortex and the cerebellum, both of which can be divided to the 2 parallel circuits (Fig. 5C) (58). According to fMRI experiments on human subjects, association (nonprimary) cortical areas (dorsolateral prefrontal, presupplementary motor area [pre-SMA], intraparietal sulcus, precuneus) were active during the first learning block, although the activity shifted from prefrontal to parietal cortical areas (65). Consistently, neurons in the pre-SMA in the monkey became very active during the first learning, but were nearly silent after sufficient learning (66). Inactivation of the pre-SMA by muscimol disrupted new learning, but not skillful performance (67). These data suggest that the pre-SMA and CDh work together for new learning. Sensorimotor cortical areas are connected with the posterior part of the basal ganglia (e.g., CDt for eye movement, posterior putamen for hand movement) (68, 69) and are likely to be responsible for generating skillful actions (58) rather than early learning.

The cerebellum is also critical for skillful behavior and learning. First, local inactivation (with muscimol) of the cerebellar nuclei disrupted skillful performance of eye–hand sequential behavior only when the ipsilateral hand was used (70). This happened selectively in the dorsal part of the dentate nucleus, which receives inhibitory inputs from Purkinje cells in the anterior part of the cerebellar cortex (70) and sends outputs mainly to the motor cortex (71). In human fMRI experiments with various behavioral tasks, we found that the cerebellum is highly sensitive to timing of actions (72) and events (73), supporting previous studies (74). The anterior cerebellum is sensitive and controls temporally stable behaviors, which are likely to be established by long-term memories. In contrast, the posterior cerebellum is sensitive and controls frequently changing behaviors, which are likely to be established by short-term memories.

These results suggest that the basal ganglia and the cerebellum contribute differently to the learning of sequential actions. According to this model, the basal ganglia decide which position, object, and action should be chosen at each stage, while the cerebellum decides when position, object, and action should be activated at each stage. This is relatively consistent with our fMRI data (72), if the pre-SMA cooperates with the basal ganglia rather than the cerebellum. The “when” signal from the cerebellum may play a crucial role in coordinating many neuronal circuits, because otherwise many body parts would move at different timings. However, this idea is speculative and requires proper behavioral experiments.

Prefrontal Cortex for New Learning and Switching

To summarize, the basic mechanism of sequential actions (Fig. 5C) is analogous to the basic mechanism of object–value choice (Fig. 2). Because of the high-capacity long-term memories, the automatic circuit (Figs. 2 and 5 C, Right) can deal selectively with many objects (e.g., fractals) and actions (e.g., left vs. right hand). Such consistency generates stable prediction, which initiates a set of actions (e.g., eye–hand movements) before any physical event (61, 75).

In contrast, the controlled circuit (Figs. 2 and 5 C, Left) has no access to long-term memory and instead is highly sensitive to recent events whose outcomes have changed: Value in the basal ganglia (20, 42) and timing in the cerebellum (72). Such recency information is essential for new learning, which is shown in detail in Fig. 6C. As described above, the pre-SMA is very active at the beginning of new learning in humans (2 × 10 task) (65) and monkeys (2 × 5 task) (66). Anatomically, the pre-SMA is part of the dorsomedial prefrontal cortex, which is common between humans and monkeys (Fig. 6B) (76). Interestingly, neurons in the monkey pre-SMA increased activity before pressing the first button in each set during new learning (Fig. 6 C, Upper). Their activity decreased as the monkey repeated the same sequence for learning. Almost the same types of neuronal activity appeared in the CDh (64). These data suggest that the pre-SMA-CDh circuit contributes to new learning, especially by suppressing automatic behaviors by using the indirect pathway (Fig. 6A).

Fig. 6.

The prefrontal cortex contributes to decision making through basal ganglia circuits. (A) Excitatory connections of the pre-SMA, part of the dorsomedial prefrontal cortex, to basal ganglia circuits. (B) Comparison of the medial frontal-motor cortical areas between monkeys (Left) and humans (Right). Reprinted from ref. 76. Copyright (2001), with permission from Elsevier. (C) Activity of a pre-SMA neuron during new learning (Upper) and learned skillful performance (Lower). Reprinted from ref. 66. For each dataset, spike activity is shown across trials (by rasters) from the first trial (Upper). Spike activity is aligned on the first button press in each set (see Fig. 5A, as another sequence), which is preceded by the illumination of the 2 buttons (stimulus onset) and followed by the second button press.

The activation of the pre-SMA and CDh neurons may also be caused by uncertainty. In fact, a group of neurons in the CDh (77) are activated selectively when the reward outcome is uncertain. These results suggest that uncertainty is a critical source of new learning. In real life, outcomes are often uncertain or volatile. This is the situation that encourages us to experience the same objects repeatedly. In fact, the monkey subject is typically attracted by objects with uncertain outcomes (78).

As expected, the learning-sensitive neurons in the pre-SMA (Fig. 6 C, Upper) and CDh were almost silent when the monkey subject was performing a well-learned sequence (Fig. 6 C, Lower) (66). Interestingly, many of them (especially pre-SMA neurons) were very active exclusively in the first set in the first trial (Fig. 6 C, Lower, first raster in set 1). The monkey performed 1 sequence (new or learned) continuously in a block of 10 to 20 trials, and another sequence started after a while. This activity thus represents volatility.

We found that the pre-SMA is also sensitive to another important behavior: Switching based on volatility (79). In a switching task, yellow and red objects appeared in each trial, and the monkey had to choose 1 of them by making a saccade to it. The choice was determined by a cue (yellow or red) at the center, which occurred shortly after the appearance of the 2 objects. Importantly, the color of the cue remained the same across 1 to 10 trials. Therefore, the monkey was ready to make a saccade quickly to whichever color had been cued in preceding trials. When the cue color changed (occasionally), the monkey had to switch the goal quickly, otherwise making a wrong choice. This is a volatile condition.

While pre-SMA neurons were relatively quiet, they became active quickly when the cue color changed (volatile), especially when the monkey switched the choice successfully. Electrical stimulation of the pre-SMA during the cue presentation improved the switching performance, suggesting that the phasic activation of pre-SMA neurons causes successful switching. Indeed, a human patient with a focal lesion in the pre-SMA had a selective deficit in rapidly switching between response plans (80).

Our data further suggested that the switching occurred by the connection from the pre-SMA to the basal ganglia through the subthalamic nucleus (STN) (Fig. 6A) (81). This hyperdirect pathway to the output regions in the basal ganglia (GPe and SNr) would cause effects opposite to the direct and indirect pathways in the basal ganglia: excitatory (82, 83) [vs. inhibitory (84, 85)], fast (83) [vs. slow (19)]. These differences may enable the fast switching, because the ongoing habitual process through direct/indirect pathways (i.e., choice based on recent repeated experiences) can be overridden rapidly by the hyperdirect pathway for switching (i.e., choice based on a sudden change). The causal role of the hyperdirect pathway in switching performance is now directly testable in monkey models using an immunotoxin-mediated tract targeting technique (86).

Discussion

We have shown that there are many parallel circuits at different levels (i.e., local to global) in the monkey basal ganglia. A significant effect of these parallel circuits is that a particular behavior (e.g., saccade) is controlled by multiple inputs with different information (e.g., short-term and long-term memories). These multidimensional parallel circuits would allow the brain to choose objects and actions in various ways. Based on these data, we characterize the basal ganglia as a “choice generator,” which is relevant to “pattern generator” in the brainstem and spinal cord (87). This allows other brain areas, which have no significant output circuits, to make decisions by sending information to the basal ganglia circuits. This mechanism seems to be used by the cerebral cortex (e.g., pre-SMA), STN, and DA neurons, cerebellum (Figs. 2, 5, and 6), and amygdala (88). Then, a choice generator (e.g., basal ganglia) can generate a correct and accurate action by sending information to pattern generators (e.g., saccade generator) (87).

However, in order to use these data for treating people with brain disorders, we need to check if humans and monkeys share the same neuronal circuits. It has been shown that basic neuronal circuits in the basal ganglia are shared by virtually all vertebrates, probably because they work effectively to control their behaviors (89). During evolution, however, new as well as old circuits may have happened to create new behavioral functions, depending on environment, which may be relevant to exaptation (90). In monkeys, indeed, different neuronal circuits in the basal ganglia controls different behaviors, many of which are related to human neurological/psychiatric symptoms (91).

We thus need to check if the same neuronal circuit contributes to the same (or different) behaviors in monkeys and humans. Importantly, behaviors are strongly influenced by emotion and motivation, which is critical for clinical treatments. An important way to address this issue is to make new behavioral tasks that are emotionally applicable to both monkeys and humans. According to our recent studies (88), monkeys are often excited to learn quickly and perform perfectly such behavioral tasks, even (or especially) if the monkeys are in complex environments, as if playing exciting games as humans do. We propose that such emotionally motivated experiences for monkeys are necessary to create new treatments for humans with deficits in behavior, thought, or emotion.

All animal experimental procedures were performed in accordance with the NIH Guide for the Care and Use of Laboratory Animals (92) and approved by the Institutional Animal Care and Use Committee at Juntendo University and NIH. For human studies, all participants gave their informed consent in accordance with the guidelines of ethics committees of Juntendo University School of Medicine in Tokyo.