The basal ganglia in action

Abstract

The basal ganglia (BG) are the major subcortical nuclei in the cerebral hemispheres. Disorders implicating the BG are characterized by diverse symptoms, which cannot be explained by traditional models of BG function. Here I will review recent electrophysiological and behavior studies that shed light on the computations performed by the BG circuits, and provide a new conceptual framework for understanding the role of the BG in behavior.

The basal ganglia (BG) have been associated with movements ever since they were first described. Unilateral stimulation of the striatum, the input nucleus of the BG, can produce contraversive movements (Ferrier, 1876), and large lesions abolish voluntary movements altogether (Sorenson and Ellison, 1970; Bjursten et al., 1976). With more restricted lesions, however, the behavioral consequences are more variable. Sometimes no conspicuous symptoms are observed. Sometimes movements are enhanced, but of an undesirable nature. Indeed both the lack of movement (hypokinesia) and uncontrollable movements (hyperkinesia) are associated with many neurological disorders implicating the BG, including Parkinson’s disease (PD), Huntington’s disease (HD), and Tourette syndrome (Martin, 1967; Mink, 2003). Why are there so many symptoms? What do they have in common? These questions remain unanswered, even as many additional labels, such as motivation, attention, and learning, were added to the list of BG functions.

The present review attempts to elucidate BG function by revisiting the old question of how they contribute to movements. I shall first describe current models of BG function and recent findings that question their basic assumptions. I shall then describe a transition control model, which offers a unified conceptual framework for understanding how the BG generate behavior.

Action selection and a methodological problem

The basic organization of the BG is summarized in Figure 1. Because the output neurons of the BG are GABAergic, with high tonic firing rates, they are believed to suppress behavior normally. According to the dominant model, the BG select specific actions by disinhibiting downstream structures. The two major pathways, direct (striatonigral) and indirect (striatopallidal) pathways, select desired actions and suppress competing ones (Mink, 1996).

Figure 1. The place of the BG in the central nervous system.

a. The BG consist of two major groups of nuclei (striatum and pallidum). Unlike the cerebral cortex, which is characterized by glutamatergic projection neurons, the BG nuclei contain GABAergic projection neurons. The striatal regions (including caudate-putamen, nucleus accumbens) are characterized by medium spiny projection neurons which receive massive glutamatergic inputs from the entire cerebral cortex as well as intralaminar thalamus. The pallidal nuclei (including substantia nigra pars reticulata and internal globus pallidus/entopeduncular nucleus). The BG output neurons are GABAergic and usually exert an inhibitory effect on their target nuclei. It is important to emphasize that the major components of the BG as well as connectivity with the rest of the nervous system are highly conserved in evolution (Grillner and Robertson, 2015).

b. The traditional event-based approach to studying behavior assumes that behavior consists of discrete events, marked by time stamps, and ignores what happens between actions or during actions.

c. According to conventional action selection models, inhibitory BG output exerts tonic inhibition on downstream structures and suppresses behavior. A pause in the BG output neurons “opens the gate” and allows a specific action to be selected. The photograph is taken from www.freeimages.com.

While this standard model is intuitively appealing, it does not define action clearly, or propose any mechanism for action selection. The major assumption is that an action is a discrete event—it either occurs or it does not. This assumption is revealed in the experimental methods used. Many studies have attempted to understand how BG activity “encodes” different aspects of behavior using in vivo electrophysiology in behaving animals, and nearly any conceivable behavioral variable was found to be correlated with BG activity, which led to considerable confusion. Yet these early studies suffer from a methodological problem, due to the assumption that action is all or none. Behavior is usually recorded as a series of discrete time stamps, which are then used in peri-event histograms, the most common representation of electrophysiological data from awake behaving animals (Figure 1b). The variables used for correlation analysis typically do not vary with time.

The “event” character of perception, however, is imposed by the observer. Although events often characterize conscious perception, they are not necessarily the appropriate measures of behavior. By assuming that all behavior consists of discrete events, neuroscientists have ignored much of the richness of behavior as a continuous process.

To understand how BG activity generates behavior, recently we have begun to use a “process-based” approach, which minimizes restraint to the animal while monitoring their single unit activity and behavior, treating both as continuous processes that change over time. Using this approach, we found a striking relationship between BG activity and movement kinematics.

Striatal activity and movement velocity

The sensorimotor (dorsolateral) striatum is known to be somatotopically organized, reflecting the projections from the overlying sensorimotor cortices. Classic work showed that movements of individual body parts can be evoked by electrical stimulation of this region (Alexander & DeLong 1985b) and that the activation of some striatal neurons was correlated with movement speed (Alexander et al., 1986; Carelli and West, 1991; Turner et al., 1998).

We combined continuous video tracking of head position with wireless recording of striatal activity in mice making simple movements to collect a sucrose reward (Kim et al., 2014). The medium spiny projection neurons (MSNs) are normally quiet, firing in brief bursts only with coordinated glutamatergic drive from the cerebral cortex or thalamus. We found that over half of recorded MSNs are monotonically tuned to vector components of movement velocity (Figure 2). Their firing rates are correlated with either horizontal velocity or vertical velocity but not both. They exhibit direction specificity, e.g. for motion along the x-axis, a “leftward selective” neuron increases firing during leftward movement, but suppress firing during rightward movement. Such neurons are also selective for contraversive movements, consistent with the observation that unilateral striatal stimulation produced contraversive movements (Ferrier, 1876; Kravitz et al., 2010).

Figure 2. Nigrostriatal pathway and movement velocity.

a. Activity of medium spiny neurons (MSN) can show strong correlation with velocity

b. DA neurons show similar correlation with vector components of velocity.

Moreover, bilateral optogenetic stimulation of striatonigral neurons, which presumably activate different classes of neurons, can also generate movements (Bartholomew et al., 2016). Each pulse produces a change in position (Figure 2a). The frequency of pulses thus determines movement speed.

Because striatal activity is commonly associated with reward guided behavior, we also tested whether the valence of the behavioral outcome can influence the correlation between kinematics and neural activity. When an aversive air puff was delivered from the same position instead of the normal sucrose reward, we observed the same correlation between vector components of velocity and neural activity, despite very different movement trajectories (avoidance rather than approach). The relationship between kinematics and firing rate is therefore independent of behavioral outcome.

The sensorimotor striatum receives direct dopaminergic (DA) projections from the pars compacta of the substantia nigra (SNc). This nigrostriatal pathway, which is degenerated in PD, can alter the excitability of striatal neurons (Gerfen and Surmeier, 2011). Using the same behavior task, we also examined the activity of nigral DA neurons in relation to kinematics. Similar to striatal neurons in their correlation with kinematics, most DA neurons showed activity correlated with vector components of either velocity or acceleration (Barter et al., 2015a). Just like striatal neurons, they are also monotonically tuned to movement velocity (and sometimes acceleration) in a given direction, independent of whether the outcome is aversive or rewarding (Figure 2b). Mimicking phasic DA activity with selective optogenetic stimulation also elicited movements, though not as readily as direct stimulation of striatal neurons (Rossi et al., 2015; Bartholomew et al., 2016).

These results suggest that the nigrostriatal pathway is critical for the control of movement velocity. Both DA neurons and their target striatal MSNs show clear correlations with velocity components. DA depletion is therefore expected to reduce movement speed. Indeed, degeneration of the nigrostriatal pathway in PD results in slowed movements or bradykinesia.

BG output and position

The major target of striatal projections is the substantia nigra pars reticulata (SNr), which contains mostly GABAergic projection neurons that in turn project to the tectum, thalamus, and brainstem. When we recorded from the SNr, we found that the GABAergic neurons show properties which are remarkably different from the neighboring SNc DA neurons. When both types of neurons were recorded simultaneously during the same movements, the DA activity represented acceleration and velocity, whereas the GABA activity represented position. During behavior, SNr output neurons are continuously correlated with distinct components of the position vector (Barter et al., 2015b). As summarized in Figure 3, two classes of neurons were found for each axis of motion, one increasing and the other decreasing their firing rate depending on movement direction (Figure 3). These results, in addition to the results from the nigrostriatal pathway, suggest that the actual movement kinematics can be explained by a computational process akin to vector addition. But the difference between velocity and position representations also reveal, for the first time, another fundamental computation performed by the BG, namely integration. The magnitude (firing rate) of the striatal output represents velocity, and it is proportional to the rate of change in the BG output.

Figure 3. BG output reflect x and y coordinates of head position.

The GABAergic output neurons of the SNr map instantaneous position coordinates. Top raster plots show the relationship between a SNr neuron and the y position coordinates. Below are schematic illustrations of four major classes of neurons, based on the relationship between their firing rates and position coordinates. For example, for horizontal motion, two types of neurons were found: 1) one type (red) increases firing with leftward movement and decreases firing with rightward movement; 2) a second type (blue) increases firing with rightward movement and decreases firing with leftward movement. The same is true of the vertical component of the movement along the y-axis. A change in firing rate therefore reflects a position change in a specific direction.

Problems with the standard model

These findings contradict some common assumptions.

1. High firing rates of BG output neurons should not be interpreted as increased behavioral inhibition, as traditionally assumed. SNr neurons increase firing for movement in one direction and decrease firing for movement in the opposite direction. Increases and decreases in firing rate reflect direction of motion, not absolute degree of inhibition.

2. Multiple classes of SNr neurons are found. For any movement, some will increase whereas others decrease firing. To generate a specific action, it is not sufficient for one type of neuron to increase or decrease firing. Multiple types of neurons must coordinate their activity at the same time, some increasing and others decreasing their firing rates depending on the direction of motion.

3. Of course, if the firing rates of different types of BG output neurons are kept constant over time, there would not be normal movement (Rossi et al., 2016). The absence of overt movement is associated with fixed firing rates. The actual rate reflects body position.

4. We found continuous and high correlation between neural activity and kinematic variables. The relationship is monotonic and independent of reward valence. Velocity neurons typically have very low baseline firing rates, whereas position-related neurons have high tonic firing rates. The high tonic firing rates reflect the neutral position, from which both increases and decreases represent signals sent to target neurons.

5. Unlike previous work on direction tuning (Georgopoulos et al., 1986), which has dominated thinking on the motor system, neurons do not fire more or less according to the direction of movement (e.g. cosine tuning). Rather they are monotonically tuned to vector components.

6. There is no suppression of postural response during movement. Rather the same neurons are activated during passive postural disturbances and during voluntary movements (Fan et al., 2012; Barter et al., 2014). Our results shed light on the role of the BG in both postural control and movement, suggesting that similar mechanisms are involved in both.

Posture/movement problem

Our results raise some questions. Why are velocity and position variables represented by different parts of the BG? Are SNr signals used to drive effectors or perceptual signals conveying proprioceptive feedback? To address these questions, I recently proposed a transition control model of the BG (Yin, 2014b). This model not only explains the results described above but also offers a natural solution for the posture/movement, perhaps the most fundamental problem in the study of behavior (Box 1). According to this model, there are no separate mechanisms for postural control and for action selection. Rather BG outputs dictate position coordinates by altering the reference signals for lower position control systems.

Box 1. Posture and movement.

Unlike a table or a tank, the body, which is balancing on ball and socket joints, lacks static stability. If the environmental disturbances were to have their way, the body would simply collapse. Continuous postural adjustments are needed just to stand still, though these variations in muscle tension largely remain unconscious to oneself and invisible to the external observer.

If a posture is maintained by providing a compensatory response to a postural disturbance, then the same response is expected to be produced when the animal is changing posture voluntarily, which would have prevented the voluntary movement altogether (von Holst and Mittelstaedt, 1950). Why is self initiated action not treated as a disturbance to be corrected? The standard answer is that postural reflexes are inhibited during movement. This is also the assumption underlying the focused selection model (Mink, 1996). But this assumption turns out to be wrong. A reflex is by definition open loop mechanism, explaining behavioral output by the path linking stimulus to response. Historically many attempts were made identify the input/output transformation, but always in vain. The failure is usually attributed to variation in the path, or context, or top down modulation of the stimulus-response path. But the main problem is with the underlying assumption of linear causation. Outputs vary so tremendously given similar inputs, or diverse inputs give rise to similar outputs. What is the ignored is the presence of the feedback function that closes the loop in any apparently reflexive behavior (Box 2 and Figure 4).

Neither posture nor movement can be identified with a set of neural outputs due to the additional and ever-present hidden influence of the environment (Yin, 2013). If the exact pattern of the signals sent to the motor neurons when you are standing now is replayed five minutes later, they will not be sufficient to keep you standing. To paraphrase Heracleitus, you cannot even stand in the same environment (body) twice.

Control hierarchy

The transition control model is based on the premise that the nervous system comprises a hierarchy of closed loop negative feedback control systems (Figure 4). They are the only systems capable of resisting unpredictable disturbances, such as those found in the real environment (Powers et al., 1960). Unfortunately, although the concept of feedback is often invoked, it is also widely understood (Box 2).

Figure 4. Closed loop negative feedback control.

a. The major misunderstanding is based on the assignment of the input and output of the control system, according to engineering convention, as shown in the basic diagram. In this diagram, the output is thought to be controlled, and the input is the command from the user. This way of illustrating the relationship between the controller and the environment creates the appearance that the controller is some device that transforms error into output. It ignores the autonomy of the organism, defined by intrinsic reference signals representing desired states.

b. Correct illustration of the organism-environment relationship. The reference represents the “should-be” or “desired” value of a perceptual signal. Because the system produces output that, via the feedback function, reduces the discrepancy or error, it is capable of reaching the desired or referenced perception. The feedback is negative, when it reduces the error or system output. For comparison to be possible, it is necessary that the input and reference have opposite signs. Disturbances are those effects that push the value of the controlled variable away from the value specified by the reference signal, generating the error signal. As they are defined by the internal reference, they cannot be equated with physical effects in the environment. The comparator function implements a subtraction. There is no mysterious agent defying physical law. But variability in the output of a control system is to be expected, as it mirror the deviations from reference at all times. Linear causation is violated in any closed loop system, as a result of the simultaneous effects of the output on input and input on output, and the asymmetry in loop gain (usually found inside the controller).

c. Illustration of a control hierarchy based on the principle of input control. On the left are the hypothesized controlled variables and their proposed neural substrates.

Box 2. What is control?

Despite the widespread use of the concepts of “feedback” and “control,” these terms are widely misunderstood when applied to the study of behavior. The key mistake is to assume that a control system controls its outputs, the convention taught in engineering. Thus interpreted, feedback control is just a reliable way to generate some desired output. Consequently, reference signals are thought to be environmental inputs to the organism, and the comparison takes place outside the organism, between system output and reference input (Figure 4a). However, the control system does not control the actual output, but a function of that output. That function is fed back to the system and compared with the desired value inside the organism. Thus the controlled variable is not the output but the input. Biological organisms are autonomous, which means that they possess intrinsic reference signals, whether innate or learned. The comparator, which generates error signal, is inside the system (Powers et al., 1960). The “control of output” assumption reverses the organism/environment relationship. Consequently, attempts to apply control theory to neuroscience have largely failed, even though the correct equations were used. It is as if one were to use the wrong end of a key to open a lock, only to conclude that it was the wrong key.

In current theories of motor control, the basic assumption is that inverse and forward models of the plant are necessary to generate movements, e.g. computing the inverse dynamics so the right temporal patterns of signals can be sent to the final common path for producing a desired final effect (Shadmehr and Wise, 2005). This is only possible given a static and highly artificial environment, using actuators that are vastly more accurate and reliable than those found in biology. For decades, the field of robotics has been crippled by this assumption, producing robots that require immensely complex calculations for simple movements, only to fail in a natural environment with disturbances.

A feedforward model requiring inverse kinematics and dynamics fail when operating in an environment subject to unpredictable disturbances—i.e. the natural environment in which all animals find themselves. No amount of knowledge put into the model is ever enough. On the other hand, with a negative feedback control model, this inverse kinematics problem does not exist; nowhere in the model is the computation needed, yet in interacting with the environment the behavior itself actually embodies the correct “calculations.”

A key property of negative feedback controllers is the control of input. This means that the signals in selected input channels will approximate the relevant internal reference signals. The descending signals in the motor system are not commands that specify behavioral outputs, but orders to request specific inputs. These inputs are generated by variations in outputs. Kinematic signals in the BG can simply reflect descending reference signals, which are brought to the desired values by variations in output. But such variability merely mirrors the environmental disturbances (both internal and external) due to feedback.

In the control hierarchy, the error signal of a given level is used to alter the reference signal of a lower level, thus requesting specific inputs to be reached. Each level varies outputs in order to acquire inputs that match reference signals. Moreover, the same effectors or lower levels must serve multiple higher levels. Measuring the muscle output does not tell us how the behavior is produced. A wink and a blink use the same muscle output but differ in the variable being controlled (i.e. purpose). There is a hierarchy of ‘purposes,’ some conscious, others unconscious, some innate, others acquired, but all working to reach and to maintain desired signals through sensory channels using the same negative feedback organization.

The cerebral cortex and the BG occupy the highest levels of this hierarchy. BG function as a transition control system, controlling the rate of change in various perceptual representations. Movement velocity, sensed as rate of change in a body configuration, is one example of such control. The BG output sends reference signals to orthogonal position controllers, to request specific changes in position at a specific rate. The BG outputs send orders to the position controllers in the midbrain and brainstem, requesting specific position inputs. When the descending reference signals alter the reference signals of these position controllers, movements are produced.

The reference signal for the velocity controller is determined by the error signals in higher levels with distinct controlled variables. Repeating an action, e.g. pressing a lever, requires repeating the same reference at the transition level, but not the same muscle outputs.

Levels below the BG

The targets of the BG implement two major classes of position control: kinesthetic position control, and exteroceptive position control. These are classified according to the type of inputs being controlled. First, body configuration control requires the control of multiple kinesthetic position inputs from the body. The reticulospinal is a key pathway influenced by descending BG outputs (Grillner and Robertson, 2015). On the other hand, exteroceptive position control moves exterosensors (e.g. eyes, ears, and whiskers) to acquire desired signals. This function is orientation control, which requires the tectum. The error signals from tectal comparators can in turn alter the reference signals for body configuration.

Although BG outputs are inhibitory, it does not follow that they inhibit behavior. Inhibition means that some value (firing rate) is subtracted from the signal arriving at the comparator. With inhibitory reference signals, the reference can be translated as “ do not let the input exceed this value.” Thus, if the perceptual inputs to the comparator is lower than the inhibitory reference signal, then no error is generated. This arrangement sets up a threshold-like effect, with the reference value serving as a threshold. Below this threshold, the system is turned off.

Velocity control and position control

Position control is optimal for maintaining a fixed position. The position can be changed by changing the reference signal, but there is no control of how quickly to move from point A to point B. A key feature of voluntary behavior, on the other hand, is that movement speed can be regulated. This is achieved by a set of velocity controllers in the BG.

Velocity control is just one example of transition control, the control of rates of change in perceptual representations. In velocity control, the rate of change in position is controlled, but position itself is not. According to the present model, the velocity controller is hierarchically higher than the position controller. The velocity error signal becomes the reference signal for the position controller (Figure 5). Since the key feature of control systems is that inputs come to resemble reference signals, much as the voltage follows the voltage command in a voltage clamp circuit, SNr output can approximate the achieved position coordinates.

Figure 5. Velocity control and position control.

a. For position controllers, a given internal reference signal specifies a position coordinate. Each degree of freedom requires a pair of controllers. For example for x axis motion, left and right. There are multiple independent controllers needed for motion in different axes (Masino, 1992). These controllers control for different orthogonal variables, e.g. horizontal and vertical motion. Orthogonal means that the effect of one does not cancel the effect of the other. Still lower levels control muscle length and muscle tension (Yin, 2014b). Their outputs would adjust the muscle length controllers and force control. For example, to maintain arm position, it is necessary to send multiple length reference signals to different length controllers.

b. There are at least three independent controllers for three degrees of freedom: up-down, left-right, and forward-backward. A single value of the reference signal corresponds to a single position along an axis of motion. Thus the magnitude of the velocity error is proportional to the rate of change in the position reference. An integrator can produce steady output in the absence of any error: when the error has reached zero, the output simply stops changing.

c. Due to the integrator in the output function of the velocity controller, the velocity error is turned into the rate of change in position reference.

Velocity in this sense is not identical to the speed of locomotion, though the two are related. Running on a treadmill without visual input, we are controlling movement velocity. When the dog chases a tennis ball, a higher level controller for proximity is needed to command the velocity based on visual feedback. The higher proximity controller is concerned with mostly visual feedback and can be activated when playing a video game (Yin, 2016). Certainly types of transition in body configuration produce locomotion and other rhythmic patterns, e.g. alternation in swimming, protrusion and retraction in rodent drinking (Rossi and Yin, 2015). BG output do not generate these fixed patterns, but can initiate and terminate them (Roseberry et al., 2016).

Integration

The position reference signals from the BG output must be derived from the error signal from velocity controller. Striatal outputs can resemble velocity error, especially when initial velocity is low. A velocity error signal is related to velocity, so to convert that to a reference signal for position, it is necessary to turn that into the rate of change.

The present model postulates the existence of a neural integrator in the output function of the movement velocity controller, which converts velocity-related signals from the striatum to yield position related signals from the SNr (Figure 5). This is known as integral gain, which can stabilize the control loop despite lags, dead times, and varying loads.

According to the present model, the striatonigral output reflects the velocity error that enters an integrator. Accumulation is enabled by disinhibition (i.e. two inhibitory synapses from striatum to the targets of the BG). The velocity error is proportional to the rate of change in the integrator output.

Damping and leak

The usual analogy for an integrator is a bucket holding water. As water accumulates, the water level rises. Without inflow (error), the water level (position) will remain steady. As the last value attained is maintained, there is no movement.

Now we can introduce another feature to this system, by adding a leak to the bucket, which is analogous to damping in a controller with integral gain. Damping acts like a force in the opposite direction. The net change in position is therefore equal to the inflow minus outflow or leak, as given by the following equation:

$${\text{dQ}}_{\mathrm{o}}={\mathrm{G}}_{\mathrm{o}}*\mathrm{e}-{\mathrm{K}}_{\mathrm{d}}*{\mathrm{Q}}_{\mathrm{o}}$$

Here dQ_o denotes the change in the system output (Q_o). This is the amount that is added to the integrator. G_o is the multiplicative gain, and e is the velocity error. The leak is a product of the damping constant (K_d) and the current output Q_o. Here, as in a bucket, the leak is made to be proportional to current output (position or water level). Whether this is true of the transition controller remains to be seen, but it leads to some interesting properties. In particular, for movement in each direction, the leak is maximal when the position is farthest from neutral and minimal at the neutral position. In the absence of additional reference signals, the position will return to neutral after some time.

The addition of a leak mechanism can explain complex dynamics of movements. The neutral position can be defined as the position with minimal leak. For each time step, the water level can increase or decrease depending on the inflow and the leak. If inflow equals outflow, then there is no position change. So long as the inflow is greater than outflow, the change will be positive. As soon as the outflow exceeds the inflow, the change will be negative, which can be interpreted as movement in the opposite direction. When all the added water has leaked out, the original position is achieved.

In a simple movement, the velocity increases and peaks after some time, only to decrease to zero. To show such a velocity profile, the firing rates of the BG output nuclei (SNr and GPi) must change as position changes, and stop changing once the target is reached. The reduction in velocity can be explained by increasing leak relative to inflow. An adjustable leak/damping mechanism provides parsimonious explanations of common movement velocity profiles, return to neutral position, and canceling of movements. It can also determine the precision of movements, providing an explanation for any speed/accuracy tradeoff.

Direct and indirect pathways

According to the present model, the direct pathway serves as the inflow to the integrator, whereas the indirect pathway serves as the leak (Figure 6). In other words, the indirect pathway discharges the integrator by introducing an error signal with the opposite sign. Any mechanism with the opposite effect on BG output as direct pathway activation is assumed to be a damping mechanism.

Figure 6. Direct and indirect pathways.

a. Using the bucket analogy, the change in water level is proportional to inflow minus outflow, for any given time step. The direct pathway reflects the inflow, whereas the indirect pathway acts as the leak or damping in transition control. These two pathways exert opponent influences on the output nuclei. For example, striatonigral projections can decrease SNr firing, whereas the striatopallidal projections can increase SNr firing (Freeze et al., 2013).

b. Illustration of the net effects of the two pathways on downstream structures. For the sake of simplicity, the intrinsic BG circuit connections are omitted here. Note that, because from the striatum to downstream targets there are two inhibitory synapses (striatonigral and nigrotectal), the direct pathway has a net excitatory effect on the target region of SNr outputs (e.g. tectum). Thus the bucket analogy is still useful despite the sign of the signals. Bridging collaterals from striatonigral axons to GPe, the classic indirect pathway (cortex-striatum-GPe-SNr/GPi), and the hyperdirect pathway (Cortex-STN-SNr/GPi) can all enhance damping in the transition control system. The highly plastic bridging collaterals allow adjustment of the damping constant.

Direct and indirect pathways share extensive cortical inputs. The circuit acts as a phase splitter, turning excitatory inputs into antiphase outputs. The traditional model of the BG fail to explain why both striatonigral and striatopallidal are activated during actions (Cui et al., 2013; Isomura et al., 2013), why unilateral indirect pathway stimulation generates movement in the opposite direction as that produced by direct pathway stimulation (Kravitz et al., 2010), or why under certain conditions there is suppression of movement by direct pathway stimulation (Cazorla et al., 2014). The assumption that actions are all or none naturally leads to the view that the direct and indirect pathways provide go/no go signals, respectively. As discussed above, this view is not tenable.

I hypothesize that the difference between direct and indirect pathway determines the net velocity error. The difference between their activation will predict the actual velocity of turning in a given direction. This is the rate of accumulation in the integrator or displacement in a time step. Since normal voluntary movements are damped, both neuronal populations are expected to be concurrently activated. Whenever the outflow exceeds the inflow, there will be movement in the opposite direction. Reversal of movement direction can be achieved by a rapid increase in the leak mechanism.

Thus the same mechanism is used for damping, stopping a movement, and for returning to neutral starting position. In the classic indirect pathway organization, via shared excitatory inputs, the signal that reaches the striatonigral neuron is inverted by the GPe and sent to the SNr. Recent work has also shown extensive bridging collaterals from striatonigral axons that directly project to the GPe (Cazorla et al., 2014). In these collaterals, the signal generated by the striatonigral neuron is inverted by the GPe and sent to the SNr, presumably with a slight delay.

Related to the indirect pathway is the so-called “hyperdirect pathway,” which include direct motor cortical projections to the subthalamic nucleus (STN), which in turn excites BG output nuclei like the SNr (Nambu et al., 2002). Because the net effect of the STN on the SNr is similar to that of the indirect pathway or bridging collaterals, the hyperdirect pathway also affects damping mechanism. By increasing the leak, this pathway is in a position to stop actions or reverse direction of movement (Aron and Poldrack, 2006).

This account explains a number of observations. For example, abrupt termination of some movement midcourse will often generate movement in the opposite direction. Unilateral optogenetic stimulation of the indirect pathway can produce ipsiversive movements instead of contraversive movements produced by unilateral direct pathway stimulation (Kravitz et al., 2010).

According to the present model, the bridging collaterals serve to damp actions (Figure 6b). When these collaterals are artificially enhanced, the activation of these striatonigral neurons will have the paradoxical effect of reducing movement, because the net effect approaches zero velocity error in a given direction (Cazorla et al., 2014). This is similar to driving with a foot on both the gas pedal and the brake, except that damping occurs at an earlier stage, so physical braking is unnecessary. It also explains the suppression of the indirect activity during rapid movements (Jin et al., 2014), as ballistic movements require less damping.

DA and gain control

In the transition control model, DA contributes to the multiplicative gain, which is a constant multiplied by the velocity error. But it exerts opposite effects on the direct and indirect pathways. DA normally has the net effect of increasing flow to the integrator while simultaneously reducing the leak or damping. In the direct pathway, DA activation of D1 receptors increases excitability of striatonigral neurons and further potentiates GABA release at the terminals. The net effect of DA is to promote action selection, in accord with traditional assumptions. In the indirect pathway, DA activation of D2 receptors suppresses excitability of striatopallidal neurons, which is assumed to reduce damping.

The net effect of DA on the velocity controller, then, is to maximize velocity in one direction while simultaneously reducing velocity in the opposing direction. This is also true of dedicated body configuration (gait) controllers for stereotyped behaviors like locomotion. Striatal activity and DA modulation are critical for regulating the speed of locomotion (Wang and Tsien, 2011; Wang et al., 2013; Kim et al., 2015; Rueda-Orozco and Robbe, 2015; Roseberry et al., 2016).

As predicted by the model, DA depletion should impair the velocity representation, because the responsiveness of striatal neurons to velocity reference commands will be impaired when the gain is reduced (Yin, 2014a; Panigrahi et al., 2015). Consequently, the descending signals that alter the velocity reference cannot be turned into an adequate velocity error to change the BG output.

D2 activation is expected to reduce damping, whereas D2 blockade should increase damping. Activation of D2 receptors, by suppressing striatopallidal activity, will reduce the leak in the integrator. To compensate for the effects of repeated and excessive D2 receptor activation, the number of bridging collaterals is increased (Cazorla et al., 2014), thereby increasing the leak. By contrast, D2 blockade should increase indirect activity. The D2 antagonist haloperidol is expected to increase damping, and the resulting overdamped system exhibits bradykinesia (Parr-Brownlie and Hyland, 2005). On the other hand, chronic D2 receptor blockade or excessive indirect activity is expected to trigger compensatory mechanisms to reduce bridging collaterals to compensate for excessive leak. This has also been confirmed empirically (Cazorla et al., 2014).

What commands the BG?

If the BG can send orders to lower level position controllers, what is above the BG in the control hierarchy? Here it is assumed that the massive corticostriatal projections send reference signals to transition controllers. It is beyond the scope of this review to discuss the higher levels in any detail (Yin, 2016). Briefly, it is assumed that the corticostriatal (and thalamostriatal) projections convey most of perceptual and reference signals that reach the striatum. But they can represent a wide variety of reference signals. Intuitively, this is where goals are represented. And where the “will” is translated into “action.” The velocity reference represents the will, but the velocity controllers are general-purpose, in that their reference signals can be altered by any higher control system. The meaning of these reference signals is independent of the velocity controller. The same action, such as raising a hand, can be used for many different purposes. The velocity controller can thus be viewed as the final common path of the cerebral hemispheres.

One example of a higher level variable is proximity. Whether for reaching or chasing a prey, this system provides the key error that can directly alter the velocity reference. The “reach” error is proportional to the proximity between self and target, which in turn is driven by the reward error (one more reward), similar to marginal utility or economic value. Such errors are generated by a combination of innate homeostatic systems and representations acquired through incentive learning. Thus the velocity reference is not usually a desired velocity per se, but the urgency by which one controls some higher variable like proximity. Its magnitude will depend on the error and gain of the higher system. Whether the errors are computed in the cortex, and exactly how the error is transformed into the reference signal remains to be determined.

The transition control model explains action selection as a process by which competing cortical representations alter the reference signals of transition controllers. This account explains what are commonly called value signals or prediction errors. Such signals appear to be predictive to the external observer, but their actual function is to control. In other words, they do not modify open loop state-behavior mappings, as assumed in reinforcement learning, but represent reference signals that request specific inputs in a closed loop. In short, each action requires coordinated operation of multiple velocity controllers. Acquired goals or purposes are remembered reference signals, which are assumed to be stored in the cerebral cortex. For these higher level reference signals to achieve specific goals, they must recruit the combination of velocity controllers, which represent at once the highest level of the motor hierarchy and the final common path in a labile motivational hierarchy using transition control. The same movement velocity controllers must serve multiple higher controllers for different transitions. One implication of this model is that most actions are learned via changing the synaptic strength of corticostriatal and thalamostriatal synapses. Such plasticity is critical for various types of learning, by recruiting and dismissing specific transition controllers (Costa et al., 2004; Yin et al., 2009; Xiong et al., 2015; O’Hare et al., 2016).

Problems with the “vigor theory”

Early primate studies found that the BG activity was related to amplitude and speed of movements, though these studies typically used average or peak values for both neural activity and kinematic measures, instead of treating them as continuous and time-varying variables (Horak and Anderson, 1984; Alexander et al., 1986). Based on these results, it was proposed that, instead of initiating movements, the BG are responsible for behavioral “vigor,” which is scaled by DA (Mazzoni et al., 2007; Turner and Desmurget, 2010). Although this hypothesis is based on the correct observation that PD patients lack “vigor,” it is largely a re-description of the observations. When attempts are made to define vigor more clearly, as in the concept of implicit motivation, the proposed explanation is inadequate (Mazzoni et al., 2007). According to this argument, movement time is selected by the brain based on energy costs associated with moving. DA depletion produces a shift in the cost/benefit function of moving fast. Consequently, PD patients are reluctant to move fast, because DA depletion increases their estimation of energy cost and reduces implicit motivation. But energy cost is not measured at all in these studies. Not moving or moving slowly is not necessarily associated with lower cost. In fact, if a patient shows bradykinesia and rigidity, the measurable energy cost, in terms of muscle output is actually higher. Rigidity can involve more, rather than less, muscle output. Like performing isometric exercises by maintaining position against disturbances associated with gravity, holding a posture can in fact be exhausting. Of course the patient wants to move faster, but what is impaired is velocity control.

In short, the vigor theory does not explain why kinematic variables like velocity or position are represented by neurons or any account of how DA can “invigorate” movements. To say that DA is important for vigor because vigor is reduced after DA depletion is similar to Moliere’s virtus dormitiva. In the absence of negative feedback control, a descending signal representing velocity will not generate movement with that velocity.

Understanding neurological symptoms

The model proposed has many implications for understanding symptoms associated with BG damage. Diverse symptoms can result from simple changes in the parameters of the transition control system. What are usually classified as hyperkinetic and hypokinetic symptoms reflect common failures in the transition control system.

Hypokinetic symptoms

A number of symptoms, such as bradykinesia and rigidity, are commonly classified as “hypokinetic.” In the most extreme case, complete DA depletion can abolish voluntary behavior altogether (Palmiter, 2008). According to the present model, DA depletion reduces the responsiveness of striatal neurons to velocity command and reduces peak velocity (Yin, 2014a). One prediction is that velocity representation in striatal projection neurons will be degraded, as supported by recent work (Panigrahi et al., 2015). By reducing the velocity error entering the integrator, DA depletion produces a fixed position reference signal (Table 1). Following DA depletion, the affected striatal neurons fail to generate sufficient velocity errors, and the SNr projection neurons fail to change their firing rates quickly or at all. Moreover, depletion has opposite effects on direct and indirect neurons (Mallet et al., 2006). It reduces the direct pathway velocity error entering the integrator. It also enhances damping because, without D2 receptor activation, the striatopallidal pathway is more active.

According to the rate model, bradykinesia is due to excessive BG output, which inhibits downstream structures and prevents behavior (DeLong, 1990). But higher BG output does not necessarily lead to more inhibition of behavior. What is reduced in bradykinesia is not the BG output per se but the rate of change in BG output. Even though the lower position controllers can still produce output to resist disturbances, what is lacking is descending commands to change the position reference signals. Resistance to position disturbance across the full range of motion is experienced as rigidity, and simply reflects the normal operation of the position controllers. The body feels unyielding, because the position controller is defending its fixed reference by generating continuous outputs. The resistance generated is proportional to disturbance. Because disturbance varies with posture, rigidity will also vary accordingly. For example, if postural disturbance from gravity is reduced by supporting the body well, rigidity will also be reduced.

On the other hand, despite rigidity, PD patients struggle with sudden postural disturbances, such as a push in the back. A local position deviation, such as a change in the joint angle, can be corrected by adjusting tension in relevant muscles. But a large and sudden disturbance to the body cannot be resisted successfully using this ‘local” strategy. Dynamic postural adjustments are needed for large disturbances. A higher level perceptual variable, perhaps corresponding to the center of gravity, must be controlled instead. For such global adjustments, the transition controller is needed. A combination of proprioceptive, visual, and vestibular inputs to the striatum may be needed for dynamic postural control (Stiles and Smith, 2015).

Abnormal postures, as a result of asymmetrical DA depletion or striatal lesions, can be explained by asymmetric changes in position reference signals. Such abnormal postures are still maintained as usual by position control systems, so that attempts to change them will be met with resistance, so there is still rigidity regardless of the actual deviant posture maintained.

Following DA depletion, there are numerous compensatory adaptation for the insufficient accumulation of velocity error, as a result of the lack of striatonigral activation. To compensate, the net inflow to the integrator can be increased by potentiating presynaptic striatonigral GABA release (Ding et al., 2015). Alternatively the leak can be reduced, e.g. by reducing glutamatergic corticostriatal synapses onto the striatopallidal neurons (Day et al., 2006). The lack of striatonigral velocity error is exacerbated by the lack of D2 activation on the indirect pathway This results in excessive striatopallidal activity and damping, and can lead to tonically increased firing rates of the BG output neurons. Under these conditions, reducing STN output reduces damping, responsible for the therapeutic effects of STN lesions and deep brain stimulation (DBS). On the other hand, reducing STN output in a normal transition controller would result in insufficient damping, which leads to abnormal movements (see below).

Hyperkineic symptoms

Hyperkinetic symptoms include dyskinesia, chorea, and ballismus—undesirable movements that do not follow the dictates of the will (Mink, 2003). They can be highly variable depending on the region and body part affected, but in each case there appears to be reduced damping. In Huntington’s disease, the loss of striatopallidal neurons early in disease progression means that there is less signal transmitted through the indirect pathway, thus less leak in the integrator. This leads to loss of damping and uncontrollable movements (chorea). With disease progression, there is additional loss of direct pathway neurons, which will eventually result in akinesia, even though there is no significant degeneration of DA neurons as in PD.

Unilateral STN lesions can produce hemiballismus uncontrollable movements on the contralateral side. Since the excitatory STN output enhances damping in the present model, reducing its output is expected to reduce damping. Consequently, the system will show oscillations and unable to cancel or slow down movements. However, the same lesion in PD patients can alleviate bradykinesia. Returning to our bucket analogy, such treatments compensate for the lack of the inflow to the integrator by reducing the leak. The net effect depends on the inflow or signal transmitted by the direct pathway. This also explains why DA replacement treatment often produces dyskinesia. As a result of compensatory changes in the depleted striatum (e.g. receptor supersensitivity) the effect of DA is much enhanced. There is excessive reduction of the damping mechanism as well as excessive velocity error from the striatonigral activation. Consequently, the controller becomes unstable and shows oscillatory behavior.

The net difference between direct and indirect pathway determines the signal entering the integrator in the present model. Although reduced damping is responsible for most hyperkinetic symptoms, these symptoms can result whenever the inflow far exceeds the outflow. Strong stimulation of the direct pathway, for example, can produce dyskinesias (Rossi et al., 2015; Bartholomew et al., 2016).

Conclusions

I have focused on the role of the BG in movement, in order to develop a new conceptual framework that explains BG function as control of perceptual transitions. The sensorimotor cortico-BG network and its DA innervation are critical for the control of movement velocity, or proprioceptive transitions, and changes in the properties of the control system, especially in the dynamics of the leaky integrator, can explain common symptoms in movement disorders.

Velocity control is only one type of transition control. These can range from movement kinematics to more abstract representations such as the tempo of music, optic flow. Other cortico-BG networks (associative and limbic) are hypothesized to be critical for other types of transitions (Yin, 2016). The key concept in transition control is that of the rate of change, which applies to all perceptual variables. At the higher levels, movement velocity is the means by which any purpose is achieved, much as the driver can only reach any destination by means of operating a speed controller. But in all these cases, motor output varies in order to control the rate of change in specific perceptual representations. Despite the enormous complexity of the control hierarchy and the diversity of controlled variables, the fundamental principle, the control of input, remains the same.