Goal-directed and habitual control in the basal ganglia: implications for Parkinson’s disease

Abstract

Progressive loss of the ascending dopaminergic projection in the basal ganglia is a fundamental pathological feature of Parkinson’s disease. Studies in animals and humans have identified spatially segregated functional territories in the basal ganglia for the control of goal-directed and habitual actions. In patients with Parkinson’s disease the loss of dopamine is predominantly in the posterior putamen, a region of the basal ganglia associated with the control of habitual behaviour. These patients may therefore be forced into a progressive reliance on the goal-directed mode of action control that is mediated by comparatively preserved processing in the rostromedial striatum. Thus, many of their behavioural difficulties may reflect a loss of normal automatic control owing to distorting output signals from habitual control circuits, which impede the expression of goal-directed action.

The basal ganglia are a group of subcortical nuclei that have been linked to movement control since the end of the nineteenth century when David Ferrier concluded that the corpus striatum contained “the centres of automatic or sub-voluntary integration”(REF. ¹). This view was expanded in the early twentieth century by observations that basal ganglia lesions were associated with movement disorders (BOX 1). The first functional model of basal ganglia architecture was developed in the late 1980s (FIG. 1a). In this model, cortical inputs enter the basal ganglia through the striatum (in primates this consists of the caudate nucleus and the putamen), and the internal globus pallidus (GPi) and the substantia nigra pars reticulata (SNr) serve as the principal output nuclei. The activity of striatal medium spiny projection neurons is conveyed to the output nuclei (GPi and SNr) through a monosynaptic GABA (γ-aminobutyric acid)-ergic projection (the ‘direct’ pathway) and a polysynaptic (‘indirect’) pathway that involves relays in the external globus pallidus (GPe) and the subthalamic nucleus (STN)^2,3. Output from GABAergic GPi and SNr neurons keep targeted structures in the thalamus and brainstem under tonic inhibitory control: this tonic inhibition is blocked (that is, paused) by phasic inhibitory signals from the ‘direct’ striato–nigral-pallidal projection⁴, which releases thalamocortical and brainstem structures from inhibition, thereby allowing movement to proceed. Dopaminergic input from the substantia nigra pars compacta (SNc) modulates corticostriatal transmission by exerting a dual effect on striatal projection neurons (FIG. 1). Neurons that co-express dopamine D1 receptors, substance P and dynorphin and give rise to the ‘direct pathway’ are excited by dopamine, whereas neurons that co-express D2 receptors and encephalin, and that give rise to the ‘indirect pathway’, are inhibited⁵ (FIG. 1a). Consequently, according to this model, in the normal state, activation of the ‘indirect circuits’ at the level of the striatum would promote movement inhibition or arrest, whereas activation of the ‘direct circuit’ would facilitate movement^3,6.

Box 1. Historical hallmarks of basal ganglia function and pathophysiological models (1912–1990).

• Wilson’s disease and striatal dysfunction (1912–1925). The main clinical manifestation of a bilateral striatal lesion secondary to copper deposition was movement impairment¹⁶⁰. Later, the striatum was associated with the control of muscle tone¹⁶¹. These early findings established the link between the basal ganglia and movement disorders.

• Lesions of the subthalamic nucleus (STN) associated with hemiballismus (1927 and 1934). Patients who suffered a sudden onset of violent and ceaseless movements of the contralateral hemibody were shown to have a haemorrhage that destroyed the subthalamic nucleus^162,163. Consequently, the STN was originally thought to inhibit movement¹⁶⁴. However, it is now recognized that the subthalamus provides excitatory input to the tonically active inhibitory output architecture of the basal ganglia.

• Lesions of the globus pallidus pars interna (GPi) or the ventrolateral thalamus block hemiballismus (1948 and 1959). Pallidotomy was shown to abolish the dyskinesia in monkeys with hemiballismus induced by unilateral lesion of the STN^165,166. In a patient with hemiballismus that did not abate spontaneously, a thalamotomy subsequently abolished the dyskinesias¹⁶⁶. These findings represent early examples of how lesions of basal ganglia circuitry can both induce and ameliorate movement disorders.

• Recognition of the importance of globus pallidus output (1962). Experiments in monkeys and clinical studies indicated that the GPi is the ‘head ganglion’ of the motor system¹⁶⁴. Together with the substantia nigra pars reticulata and the GPi, the basal ganglia output nuclei exert powerful inhibitory control over multiple levels of the neuraxis.

• Striatal dopamine depletion associated with Parkinson’s disease and the effect of levodopa (1960). A profound reduction (>80% below control values) in dopamine striatal concentration was detected in patients with Parkinson’s disease and in patients with postencephalitic parkinsonism¹⁶⁷. This was followed by investigations into the effect of levodopa in patients with Parkinson’s disease¹⁶⁸. After some initial doubts, levodopa was confirmed as being an effective treatment when used in the correct dose¹⁶⁹ (see REF. 170 for a review). Tonic activity in ascending dopamine systems was finally acknowledged as being essential for normal function within the basal ganglia.

• Proposal of the ‘selection’ function of the basal ganglia (1976 and 1978). The basal ganglia have all the aspects of a ‘clearing house’ that receives samples of ongoing cortical activity and facilitates (selects) one while suppressing all others¹⁷¹. Hassler wrote “The basal ganglia at the same time focus the attention, emotional participation and excitability on one single event by simultaneously suppressing and fading all other happenings and motivational objects” (REF. ¹⁷²). From this point onwards a recurring theme in basal ganglia literature has been ‘selection’ that is implemented by selective disinhibition⁴ within the looped architectures that connect the basal ganglia with the cerebral cortex and brainstem structures⁴⁹.

• Importance of the basal ganglia for the expression of automatic movements (1982). Sequential or simultaneous motor tasks are characteristically impaired by the loss of dopamine in early stages of Parkinson’s disease. The basal ganglia are responsible for the “automatic execution of learned motor plans” (REF. ¹¹¹). This idea has developed into the recognition that parts of the basal ganglia play a crucial part in the expression of habits^64,66,157.

Figure 1. Organization of intrinsic connections within the basal ganglia.

a ∣ Model based on the influential proposal by Albin and colleagues³, according to which the output of the basal ganglia is determined by the balance between the direct pathway — which involves direct striatonigral inhibitory connections that promote behaviour — and the indirect pathway — which involves relays in the external globus pallidus (GPe) and subthalamic nucleus (STN), and suppresses behaviour. The balance between these two projections was thought to be regulated by afferent dopaminergic signals from the substantia nigra pars compacta (SNc) and the ventral tegmental area (VTA) acting on differentially distributed D1 and D2 dopamine receptors. b ∣ Recent anatomical investigations have revealed a rather more complex organization in which the transformations that are applied to the inputs to generate outputs are less easy to predict. GPe, globus pallidus pars externa; GPi, globus pallidus pars interna; SNr, substantia nigra pars reticulata. Figure is modified from REF. 10 © P. Redgrave (2007).

In the parkinsonian state, reduced dopamine input from the SNc to the striatum would have a dual effect, facilitating neurons that give rise to the ‘indirect’ projection and reducing the activation of ‘direct pathway’ neurons⁷. Increased inhibition from ‘indirect’ striatal neurons to the GPe disinhibits the STN, which then overdrives inhibitory output neurons in the GPi and SNr^6,8. In parallel, decreased activation of ‘direct pathway’ striatal output neurons would effectively reduce the phasic inhibitory influences on the GPi and SNr, thereby contributing to excessive activation of inhibitory basal ganglia output. The dyskinetic state is thought to result from an opposite set of changes (inhibition instead of excitation)⁸. Here, reduced STN activity and corresponding reductions in basal ganglia inhibitory output would facilitate involuntary movements^6,8,9.

Although this original model has provided a useful heuristic and spawned many valuable experimental studies and clinical developments, it is increasingly perceived as having important limitations. Over the past quarter century, many new anatomical, functional and clinical data have become available and they provide a rather different appreciation of basal ganglia architecture and function. In this review, we present a contemporary summary of the structural organization of the basal ganglia and a developing systems-level analysis of their general functional organization. Specifically, we review a body of recent data showing that regionally segregated functional territories of the basal ganglia contribute to networks that regulate goal-directed and habitual behaviours. Based on this information, we propose that the well-established preferential loss of dopamine in the sensorimotor territories of many patients with Parkinson’s disease may cause a major deficit in habitual control, and that pathological output from dysfunctional sensorimotor circuitry will distort the expression of residual goal-directed responses.

Structural organization of the basal ganglia

Since the ‘direct versus indirect pathway’ model of basal ganglia function was introduced, there have been important anatomical discoveries that were not anticipated by the model’s original proposers^2,3,6,8. Here we present a brief and necessarily selective review of relevant anatomical findings that describe intrinsic connections between the basal ganglia nuclei and the pathways that connect the basal ganglia to other brain regions. These structural details place important constraints on the computational processing that the basal ganglia can be expected to perform.

Intrinsic connections

Although the ‘direct’ and ‘indirect’ pathways proposed in the original model remain valid (FIG. 1a), it is now recognized that they represent a subset of the connections between the basal ganglia nuclei, albeit an important one (FIG. 1b¹⁰). First, an additional feature of the ‘direct’ pathway from the striatum to the output nuclei is the branching collateral fibres that terminate sparsely in GPe^11,12 (FIG. 1b). Second, the exclusive feedforward nature of the ‘indirect projection’ from the striatum is no longer tenable. The STN is now recognized as a major input station of the basal ganglia with external afferents from both cortical and subcortical structures^13–18 (FIG. 1b). The GPe projects not only to the STN but also directly to the GPi and SNr, and to nigrostriatal dopamine neurons, often via a network of branched collaterals¹⁹. Although the feedforward nature of information processing in the basal ganglia is still prominent, clear examples of reciprocal connectivity between basal ganglia nuclei are now evident between the STN and GPe^19–21, and between the GPe and the striatum^20,22–24 (FIG. 1b).

A widely recognized feature of connections between the intrinsic nuclei of the basal ganglia is their high degree of spatial topography^25–27. Thus, in primates, projections from the striatum to the globus pallidus and substantia nigra maintain a general arrangement throughout the different intrinsic nuclei, whereby the posterior putamen is engaged in sensorimotor functions, the caudate and anterior putamen in associative functions, and the ventral (limbic) striatum in motivational and emotional functions²⁸ (FIG. 2). Accordingly, the direct projections between the striatum and the basal ganglia output nuclei represent the internal component of the general, partially segregated, looped architecture that connects the basal ganglia with extrinsic cortical²⁹ and subcortical structures³⁰. A further feature with functional importance is that, although they are individually different, the internal microcircuits in each of the basal ganglia nuclei are, with certain quantitative differences, qualitatively similar (in terms of cell types and local connections) across the different functional zones of each structure (limbic, associative and sensorimotor)^31–36. Thus, insofar as function is an emergent property of connectivity, it is likely that throughout the different basal ganglia nuclei, similar input–output computations are being applied to widely differing functional signals (affective, cognitive and sensorimotor signals).

Figure 2. Corticobasal ganglia–cortical loops in animals and humans.

a ∣ The connections between the cerebral cortex and the basal ganglia can be viewed as a series of parallel-projecting, largely segregated loops or channels²⁹ conveying limbic (shown in red), associative (shown in yellow–green) and sensorimotor (shown in blue–white) information. Functional territories represented at the level of cerebral cortex are maintained throughout the basal ganglia nuclei and thalamic relays. Note, however, that for each loop the relay points in the cortex, basal ganglia and thalamus, offer opportunities for activity inside the loop to be modified or modulated by signals from outside the loop. Black arrows indicate glutamatergic projections, grey arrows indicate GABA (γ-aminobutyric acid)-ergic projections. b ∣ The spatially segregated ‘rostral caudal gradient’ of human frontal cortical connectivity in caudate, putamen and pallidum. The colour-coded ring denotes limbic (shown in red), associative (shown in yellow–green) and sensorimotor regions of the cerebral cortex in the sagittal plane (colour coding after Haber¹⁷³). Using probabilistic tractography on magnetic resonance-diffusion weighted imaging data, Draganski et al. identified the regions of the striatum that receive the strongest input from the identified cortical regions (sagittal oriented images contained within the ring)³⁷. For better visualization, segmented basal ganglia nuclei are superimposed on T1-weighted structural images in the far right of the diagram (the rostral part is shown on the left, the caudal part is shown on the right). PFC, prefrontal cortex. Part a is modified, with permission, from REF. 174 © (1995) MIT Press. Part b is reproduced, with permission, from REF. 37 © (2008) Society for Neuroscience.

Extrinsic connections

We now consider the main connections between the basal ganglia and external structures, namely the parallel-projecting, partially segregated reentrant loops^27,29,30. Most of the functionally segregated regions of the cerebral cortex provide topographically organized input to the principal input nuclei of the basal ganglia^14,25,29,35. Cortical regions associated with limbic (that is, emotional), associative (that is, cognitive) and sensorimotor functions provide topographically ordered input extending from ventromedial to dorsolateral zones of the striatum — limbic input projects to ventromedial zones, sensorimotor input to dorsolateral zones and associative input to the region in between ventromedial and dorsolateral zones (FIG. 2). These projections are the first link in the parallel loop component of corticobasal ganglia architecture^27,29. This looped architecture has been shown in anatomical studies of a range of species^25–27, including humans^37–39.

The re-entrant loops described above were not the first evolutionary example of closed-loop architecture involving the basal ganglia and external structures³⁰. Indeed, there is a phylogenetically older set of closed-loop subcortical connections between the basal ganglia and brainstem sensorimotor structures, which include the superior and inferior colliculi^40,41, periaqueductal grey⁴², pedunculopontine nucleus⁴³, cuneiform area and parabrachial complex⁴⁰, and various pontine and medullary reticular nuclei⁴⁰.

Finally, the ascending dopaminergic system provides important modulatory influences on basal ganglia processing. The projection from the SNc and the ventral tegmental area targets all intrinsic nuclei of the basal ganglia but with preferential concentrations of terminals in the dorsal and ventral striatum⁴⁴ (FIG. 1b). The main nigrostriatal projection maintains a general medial to lateral topography⁴⁵. There is also a sparser dopamin ergic innervation of both the globus pallidus⁴⁶ and STN⁴⁷. The ascending dopaminergic systems have both tonic and phasic patterns of activity⁴⁸. Tonic firing is essential for normal ‘selective functions’ of the basal ganglia^49–51, whereas sensory-evoked phasic activity provides ‘teaching signals’ for instrumental learning^52–54.

To summarize, layers of complexity at all levels have been added beyond those envisaged in the original ‘direct versus indirect pathway’ model (FIG. 1b). It is now recognized that the basal ganglia are subdivided anatomically into functional domains that have topographically organized connections with external structures that contribute to emotional, associative and sensorimotor functions.

Functional divisions of the basal ganglia

Micro-stimulation and micro-recording investigations in behaving monkeys^55–60, and functional imaging studies in humans^26,61,62 have shown that signals from subregions of the basal ganglia nuclei accord well with the functional status of their inputs. Thus, the consequent tripartite division of the basal ganglia into limbic (motivational and emotional), associative (cognitive) and sensorimotor networks has provided a functional framework onto which different forms of associative learning⁶³ have been successfully mapped^64–66. However, to provide an appreciation of the neural systems responsible for instrumental learning in the form of goal-directed action–outcome control and the acquisition of habits, we will restrict ourselves to a consideration of the respective roles of the associative and sensorimotor territories in the basal ganglia.

Dual systems for instrumental control

Early analyses of instrumental behaviour stressed the importance of reinforced associations between stimuli and responses^67,68. However, during the past 20 years it has become clear that animals can also encode causal relationships between actions and outcomes^63,64,69. Consequently, when behaviour is goal-directed, action selection is determined primarily by the relative utility of predicted outcomes, that is, the value of the possible outcomes of different actions. To discover whether a particular action is goal-directed or under stimulus control (that is, habitual or automatic), a series of formal behavioural tests have been developed.

The first involves changing outcome desirability⁷⁰. Here, an animal learns that an action (for example, pressing a lever) triggers the delivery of a particular type of food. However, if the animal is allowed to satiate itself with that food before the test, the utility of the food would be devalued. Thus, if during subsequent testing the animal makes fewer actions to obtain the (now) devalued outcome, its behaviour is considered to be goal-directed. Alternatively, if the animal maintains its behaviour (that is, presses the lever) despite the devalued outcome, the behaviour is considered to be under stimulus control (that is, habitual)^63–66,69. A second test involves degrading the contingency relationship between action and outcome⁷¹. For example, the experimenter can ensure that food delivery sometimes occurs without the animal performing the required action (for example, pressing a lever). Under these conditions decreased responding would reflect goal-directed control of behaviour, whereas maintained responding would reflect habitual control of behaviour⁷².

These tests have been used to show that different functional territories within the basal ganglia — and the structures to which they are connected through the looped architecture — are responsible for goal-directed and habitual control of behaviour, respectively, in both animals and humans^65,73. However, we will first consider how dual control systems can operate in tandem.

Goal-directed or habit?

What happens when an Englishman drives a car in Spain or a Spaniard drives a car in England? Goal-directed systems will work hard to keep the car on the right and left side of the road, respectively. However, it only takes a moment’s inattention or distraction, or an emergency, for stimulus–response habit systems to gain control, with distressing consequences. Given that a behaviour (in this case, driving) can be regulated either by goal-directed or habitual control systems, what processes or mechanisms determine whether the dual controllers act cooperatively or in competition?

Experimentally, several factors have been shown to determine whether the agent (animal or human) operates in goal-directed or habitual mode. The first is over-training: here, initial control is largely goal-directed, but with consistent and repeated training there is a gradual shift to stimulus–response, habitual control⁷⁰. Once habits are established, habitual responding tends to dominate, especially in stressful situations in which quick reactions are required⁷⁴. The second related factor is task predictability: in the example of driving⁶⁴, talking on a mobile phone is fine so long as everything proceeds predictably. However, if something unexpected occurs, such as someone stepping out into the road, there is an immediate switch from habitual to goal-directed control⁷⁵. Making this switch takes time and this is one of the reasons why several countries have banned the use of mobile phones while driving⁷⁶. The third factor is the type of reinforcement schedule: here, fixed-ratio schedules promote goal-directed control as the outcome is contingent on responding (for example, a food pellet is delivered after every n responses). By contrast, interval schedules (for example, schedules in which the first response following a specified period is rewarded) facilitate habitual responding because contingencies between action and outcome are variable⁷⁷. Finally, stress, often in the form of urgency, has a powerful influence over which mode of control is used. The fast, low computational requirements of stimulus–response processing ensure that habitual control predominates when circumstances demand rapid reactions (for example, pulling the wrong way in an emergency when driving on the opposite side of the road). Chronic stress also favours stimulus–response, habitual control^78,79. For example, rats exposed to chronic stress become, in terms of their behavioural responses, insensitive to changes in outcome value and resistant to changes in action–outcome contingency⁷⁹.

Goal-directed and habitual control in concert

Although these factors can be seen as promoting one form of instrumental control over the other, real-world tasks often have multiple components that must be performed simultaneously or in rapid sequences. Taking again the example of driving⁶⁴, a driver is required to continue steering while changing gear or braking. During the first few driving lessons, when steering is not yet under automatic stimulus–response control, things can go horribly awry when the new driver attempts to change gears. By contrast, an experienced (that is, ‘over-trained’) driver can steer, brake and change gear automatically, while holding a conversation, with only fleeting contributions from the goal-directed control system. This suggests that many skills can be deconstructed into sequenced combinations of both goal-directed and habitual control working in concert^75,80. Formal demonstrations of this have been reported by Balleine and Ostlund⁸¹.

Nevertheless, a fundamental problem remains: at any point in time, which mode should be allowed to control which component of a task? Daw et al.⁸² have used a computational approach to address this problem. Their analysis was based on the recognition that goal-directed responding is flexible but slow and carries comparatively high computational costs as opposed to the fast but inflexible habitual mode⁸³. They proposed a model in which the relative uncertainty of predictions made by each control system is tracked. In any situation, the control system with the most accurate predictions comes to direct behavioural output. It is likely that the looped architecture that connects relevant functional territories of the cerebral cortex with the basal ganglia plays a crucial role in switching between habitual and goal-directed modes of action control⁷⁵.

Goal-directed and habitual control in the basal ganglia

In a seminal series of studies Yin et al.^66,84–87 showed that inactivation of the dorsomedial (associative) region of the rodent striatum, either with neurotoxins or local injections of the GABA agonist muscimol, blocked goal-directed control, as revealed by a reduced sensitivity (in terms of behaviour) to outcome devaluation and a degradation of action–outcome contingency^85,86 (FIG. 3a). In experiments in which habitual responding was induced by over-training, control animals were insensitive to outcome devaluation⁸⁴ and contingency degradation⁸⁷ (that is, they showed habitual responding), whereas animals with dorsolateral (sensorimotor) striatal lesions returned to goal-directed control and adjusted their behaviour accordingly. These results show that the dorsomedial and the dorsolateral striatum regulate goal-directed and stimulus–response habitual control, respectively.

Figure 3. Striatal determinants of goal-directed and habitual action in rodents and humans.

a ∣ Photomicrographs of NMDA (N-methyl-d-aspartate)-induced cell-body lesions of the rat dorsomedial and dorsolateral striatum (shown in the right hemisphere) with approximate regions of lesion-induced damage illustrated with red and purple circles, respectively (shown in the left hemisphere). Lesions of the dorsomedial striatum abolished acquisition and retention of goal-directed learning⁸⁵, whereas lesions of dorsolateral striatum abolished the acquisition of habit learning⁸⁴. b ∣ The regions of the human anterior dorsomedial striatum that exhibits sensitivity to instrumental contingency. c ∣ The region of the human posterior lateral striatum (posterior putamen) that exhibits a response profile that is consistent with the development of habits in humans. Part a is reproduced, with permission, from REF. 65 © (2010) Macmillan Publishers Ltd. All rights reserved. Part b is reproduced, with permission, from REF. 95 © (2008) Society for Neuroscience. Part c is reproduced, with permission, from REF. 96 © (2009) Blackwell Publishing Ltd.

Miyachi and colleagues provided evidence for a homologous functional organization within the nonhuman primate basal ganglia — in which the caudate nucleus and rostral putamen correspond to the rodent dorsomedial striatum, and the posterior putamen to the rodent dorsolateral territories — in two important experiments. In the first⁸⁸, local inactivation of associative territories of the monkey striatum impaired the acquisition of novel motor sequences but left the performance of well-learned sequences comparatively unaffected. Conversely, inactivation of the middle-to-caudal, sensorimotor part of the putamen disrupted the execution of well-learned sequences, but animals were still able to acquire new sequences. In the second experiment⁸⁹, neurons in the associative part of the striatum were preferentially activated when monkeys were learning new motor sequences, whereas neurons in the sensorimotor territories were active when the animals performed well-learned, automatic sequences of movements. The concept of homologies across species was extended further in a recent review by Balleine and O’Doherty⁶⁵, in which parallels were drawn between the functional organization of the striatum in rodents and that in humans. In humans, blood oxygen level-dependent (BOLD) activation of the caudate nucleus and the regions of the prefrontal cortex associated with it occurs during goal-directed behaviours, such as set shifting⁹⁰, action planning⁹¹ and cognitive manipulations⁹². During prolonged training on motor tasks (when performance would be expected to shift from being under goal-directed to being under habitual control), the activation pattern gradually shifts from rostromedial (associative) to caudolateral (sensorimotor) regions of the striatum^93,94. Studies using formal tests of goal-directed and habitual control have provided confirmation that behavioural control systems are localized in different regions of the human striatum. For example, Tanaka et al.⁹⁵ reported increased activation of the anterior caudate nucleus in subjects performing on a high (goal-directed) action–outcome contingency schedule compared with subjects performing a low action–outcome contingency schedule that promotes habitual responding (FIG. 3b). By contrast, the caudal putamen was preferentially activated during an outcome devaluation test when subjects’ behavioural responses were habitual⁹⁶ (FIG. 3c). Thus, in rodent, monkey and human striatum, goal-directed and habitual control is mediated independently by circuits that connect with the associative and sensorimotor functional territories of the striatum, respectively⁶⁵. In view of the relative differences in the location of these territories in the rodent and primate striatum, from this point we will use the general terms ‘rostromedial’ (associative) and ‘caudolateral’ (sensorimotor) when referring to striatal functional territories in either species.

The need for selection

The structural evidence reviewed above established that the parallel looped architecture is common to all functional territories of the basal ganglia. In addition, components of intrinsic micro-circuits are qualitatively, if not quantitatively, similar across the functional zones of basal ganglia nuclei. It therefore seems likely that the basal ganglia perform the same (or similar) general computational function in goal-directed and stimulus–response habitual control systems. A prominent view, which comes in several forms^{49–51,53,66}, holds that basal ganglia architecture seems to be ideally configured to perform a generic ‘selection’ function⁴⁹. Indeed, the parallel loop architecture^29,30 — combined with selective disinhibition at the level of the output nuclei⁴ — results in effective action selection, in computational models^97–99 and in behavioural control of a mobile robot¹⁰⁰.

Presumably, it is the capacity of the basal ganglia to select between competing options that defines their role in goal-directed and habitual control of behaviour. Thus, if in any situation brain regions with access to associative territories of the basal ganglia are likely to generate diverse, often mutually exclusive behavioural options¹⁰¹ (that is, a range of potentially incompatible outcomes and their associated causal actions), a mechanism would be required to select a ‘winner that takes all’. Similarly, afferent structures that provide input to sensorimotor regions of the basal ganglia may also simultaneously represent multiple habit-evoking stimuli. In this case, the question arises as to which stimulus–response pairing should be permitted to dominate. Therefore, the external structures that generate multiple cognitive (goal-directed) and stimulus–response (habit) options may need to interact with the basal ganglia to determine which option should be permitted to guide behavioural output^10,49,53.

The final common motor path

It was noted above, first, that the same act (for example, pressing a lever) can be performed under either goal-directed or habitual control⁸⁰. Second, findings from inactivation and lesion studies have shown that goal-directed and stimulus–response habitual control are mediated independently by neural circuits that course through different regions of the striatum^{66,84–88,95,96}. Thus, if the same act, set of movements or action sequence can be performed independently by spatially separated functional systems (at the level of the basal ganglia), the implication is that the output of goal-directed and habitual control systems must at some point converge on lower level motor structures that generate behavioural output. Where the proposed convergence takes place is unknown, and there are many possible areas where the ‘final common motor path’ might become ‘common’. An early location where associative and sensorimotor outputs from the basal ganglia could converge is on the motor output neurons in the cortical²⁹ or brainstem sensorimotor regions³⁰ that connect to the basal ganglia via the sensorimotor loops. These motor neurons may be driven by direct sensory inputs (for stimulus–response habitual responding)¹⁰² and also by inputs from higher-level structures that are involved in goal-directed control^103–105. The last place where the proposed convergence could take place is through multi-synaptic relays terminating on common motor neurons in the brainstem and spinal cord¹⁰⁶. In any case, it is necessary to assume that spatially segregated goal-directed and habitual control systems (at the level of the basal ganglia) at some point converge on motor output mechanisms, and that this inferred convergence will have important implications for the interpretation of the functional deficits in patients with Parkinson’s disease.

A model of basal ganglia dysfunction

When the classic motor features of Parkinson’s disease appear, dopaminergic neurons in ventrolateral substantia nigra and their terminals in the caudolateral sensorimotor putamen are differentially susceptible to degeneration¹⁰⁷. Indeed, positron emission tomography (PET) imaging typically reveals reduced levels of ¹⁸fluorodopa uptake mainly in caudal regions of the striatum¹⁰⁸ (FIG. 4). This is consistent with post-mortem data showing the greatest loss of dopaminergic innervation to be from the posterior putamen¹⁰⁹ and a corresponding loss of dopaminergic cells in the ventrolateral SNc¹¹⁰. Thus, dysfunction of basal ganglia sensorimotor circuits marks the initial stages of Parkinson’s disease^111,112. Given that these are the regions through which stimulus–response habitual control is exercised, a principal feature of the disease should be an impaired ability to acquire and express (in other words, to select) habitual actions (FIG. 5). Indeed, it has long been known that patients with Parkinson’s disease have difficulties expressing automatic components of behaviour^113,114 and are impaired in their implicit learning of habits¹¹⁵. If the territories of the basal ganglia that underlie habitual control malfunction in these patients (FIG. 5), then these patients would have to perform most, if not all, aspects of tasks — from the mundane to the complex — in a volitional and goal-directed manner¹¹⁶. Direct evidence supporting this point is, however, scarce at present and will depend on the continuing translation of the formal behavioural tests from animal learning theory^65,70,71 into the human arena, in a form that is suitable for testing in patients with Parkinson’s disease.

Figure 4. Striatal dopamine innervation assessed by ¹⁸fluorodopa positron emission tomography (PET).

a ∣ PET scan from a control subject showing high uptake of ¹⁸fluorodopa in the striatum (highest uptake values are shown in white). b ∣ An example of a PET scan from a patient with Parkinson’s disease with motor signs that are mainly confined to the right limbs. ¹⁸Fluorodopa uptake is markedly reduced in the left posterior putamen (the uptake in the area indicated by the arrow is 70% below normal) and reduced to a minor extent in the anterior putamen and caudate of the left hemisphere.

Figure 5. Functional and dysfunctional loops through the basal ganglia in the parkinsonian state.

Both goal-directed and habitual control systems receive sensory inputs. Goal-directed control is dependent on the associative networks through the basal ganglia, whereas stimulus–response habitual control relies on the sensorimotor territories. Both networks independently have the capacity to direct behavioural output via ‘final common motor pathways’ (REFS ⁸⁴,⁸⁵,⁸⁷). There may be many sites at which output from the two control systems might converge, and two have been illustrated: one at the level of cortical motor output, the other in the brainstem. In Parkinson’s disease, differential loss of dopamine innervation from sensorimotor regions in the basal ganglia (shown by the red cross) causes dysfunctional output signals from these territories and their associated networks (shown by lightning symbols). Presumably, this loss of function causes an increased reliance on goal-directed control. In addition, the distorting inhibitory output from sensorimotor territories is likely to impede the execution of goal-directed behavioural control that is otherwise comparatively preserved. BG, basal ganglia.

Trapped in goal-directed control

A brief survey of the observed differences between the goal-directed and automatic (stimulus–response habit) modes of behavioural control will provide insights into what a preferential loss of the automatic stimulus–response system in patients with Parkinson’s disease might entail. First, extended and consistent training is required to develop automatic processing⁸³. Consequently, patients with Parkinson’s disease would be expected to have great difficulty developing habitual responding despite repeated practice^115,117,118. This has also been demonstrated in a rodent 6-hydroxydopamine lesion model of Parkinson’s disease. Here, selective loss of dopamine from the lateral SNc and lateral striatum caused behaviour to remain goal-directed, despite over-training, whereas control animals showed normal habit development¹¹⁹. Second, automatic processing is fast and parallel, whereas controlled processing is slow and serial⁸³. With a progressive loss of rapid, low-cost habitual processing, we would predict that patients with Parkinson’s disease would have to rely increasingly on the slower, serial goal-directed control system. Third, automatic control requires little cognitive effort and can operate in multitasking situations involving high workload⁸³. Thus, a gradual replacement of automatic control by effortful, goal-directed processing in patients with Parkinson’s disease should render their task in hand increasingly subject to interference from other goal-directed tasks^113,120,121. Indeed, these patients require a conscious decision to start walking and they stop abruptly if they are distracted by a different external stimulus, a new idea or another behaviour (for example, talking)^116,122. Fourth, once action sequences have become automatic it is often difficult to revert to conscious volitional control⁸³. Therefore, if our proposal that patients with Parkinson’s disease increasingly have to operate in goal-directed control mode is correct, this could explain why they have particular problems with the automatic components of action sequences^123–125 and are generally slow during the execution of common daily life activities.

In addition to the increasing difficulties with the automatic, habitual components of behaviour, patients with Parkinson’s disease often experience executive dysfunction, which can also appear early during the disease¹¹². In most cases, however, it is unclear whether these deficits reflect a partial, disease-related loss of dopaminergic input to associative territories of the striatum¹²⁶ or whether the loss of normal habits imposes a debilitating load on the serial operation of goal-directed control^83,116,121.

Impaired automatic (habitual) movements

Because many actions have both goal-directed and habitual components⁸⁰, malfunctioning of sensorimotor loops should preferentially block the habitual components from reaching expression. Accordingly, in patients with early stage Parkinson’s disease, the ability to carry on performing normal automatic, habitual components of actions (for example, blinking, arm-swinging, facial expressions, pacing of gait, speech modulation and swallowing) is impaired¹¹¹, and this ability is further aggravated when patients are required to engage in goal-directed activities^114,121.

A possible objection to our proposal is that although automatic and well-learned movements and tasks are reduced in frequency, amplitude and speed in patients with Parkinson’s disease, automatic functions are not completely eradicated¹²⁵. One reason for this preservation of function could be that the degeneration of dopaminergic input to sensorimotor striatal areas is incomplete. At the time of diagnosis, dopamine depletion in the posterolateral putamen on the clinically affected side is usually about 60%¹²⁶. A second objection could be that patients with Parkinson’s disease show an improvement in motor performance when they are guided by sensory stimuli^116,127,128, which suggests that their sensorimotor control circuits may still be operational. However, sensory input is equally important for goal-directed control¹²⁹ (FIG. 5). Thus, for patients with Parkinson’s disease, the presence of a salient external stimulus to guide their movements could offer substantial advantage over movements initiated by internally generated ‘cognitive’ signals^116,121. Third, it is also noted that some automatic movements (for example, conditioning of the blink reflex) are preserved in patients with Parkinson’s disease¹³⁰. However, such movements commonly reflect the expression of classically conditioned reflexes associated more with cerebellar and limbic circuitry^131,132 than with instrumental action–outcome or stimulus–response learning mediated by the basal ganglia.

Difficulty in initiating and executing movements

The dual control model in FIG. 5 shows goal-directed and habitual control circuits converging on a final common motor path. Although, as observed above, the precise location (or locations) of this convergence remains to be determined, two possible sites are illustrated; one at the level of cortical sensorimotor output and the other in brainstem motor systems. Many experimental investigations have shown that basal ganglia output is abnormal in patients with Parkinson’s disease and in animal models of the disease^133,134. Typically, neurons in the STN and GPi show increased firing^135,136 and abnormal patterns of synchronised oscillations^137–139. Therefore, in the Parkinsonian state, the downstream motor structures onto which both control systems presumably converge would be expected to receive one relatively normal set of signals (from the goal-directed system) and one set of pathological signals (from the stimulus–response habit system) (FIG. 5). Given that basal ganglia output is tonically active and inhibitory⁴, the distorting signals from dopamine-deprived stimulus–response habit circuits are likely to result in increased inhibition^133,134 and abnormal oscillations¹⁴⁰. Thus, for the goal-directed system to operate effectively (that is, to generate appropriate behavioural output), it would have to overcome the inhibitory distortion from the malfunctioning stimulus–response habitual control circuits in the basal ganglia. It is therefore noteworthy that a recent imaging investigation¹¹⁷ reported increased activity but less effective connectivity in areas of the brain that are associated with the production of automatic movements in patients with Parkinson’s disease. The patients also failed to show the normal reduction in dorsolateral prefrontal cortical connectivity as learning progressed from novel (that is, goal-directed) to automatic stages. These results are consistent with the notion that goal-directed control has to overcome ‘abnormal inhibition’ in order for what used to be ‘automatic’ movements to be expressed by patients with Parkinson’s disease. This idea would also explain many features of bradykinesia, like reduced amplitude of agonist muscle recruitment of a ballistic movement and rapid decline of repetitive movements^141,142. Furthermore, if distorting oscillatory inhibitory output from the sensorimotor loops is imposed on downstream motor systems in the absence of substantial goal-directed input, perhaps this could account for the tremor observed in patients with Parkinson’s disease at rest. This would be consistent with the observation that an attempt to mimic parkinsonian tremor by stimulating GPi in monkeys did not change motor cortex excitability or induce tremor, suggesting that parkinsonian tremor may develop in downstream motor circuits¹⁴³. When patients attempt to exercise goal-directed control, initially, they may be able to direct sufficient excitatory input to motor systems to overcome distorting oscillatory activity from the habit circuits. However, this level of goal-directed input seems to be difficult to sustain and so performance fatigues rapidly. This could explain why tremor stops at the beginning of a voluntary movement, only to resume seconds later¹⁴¹.

We are aware that the present functionally-based framework does not account for the origin of the typical parkinsonian resting tremor or for the resistance to passive displacement of a joint. That said, both cardinal features could represent other distorting influences that are imposed on basic downstream motor control mechanisms by abnormal output from the sensorimotor basal ganglia¹¹².

Why surgical lesions can be beneficial

In the last 6 decades, many patients with Parkinson’s disease and other movement disorders (perhaps some 100,000) have been treated by stereotactically defined lesions directed to different parts of the basal ganglia. The general observation is that regardless of how beneficial the procedure is, the patients’ underlying motor problems are not aggravated and new motor difficulties are not induced¹⁴² when lesions are unilateral. The apparent impunity with which the basal ganglia can be lesioned to treat movement disorders without catastrophic effects on behaviour is a well recognized paradox¹⁴⁴. The current framework permits an earlier idea¹⁴² that “it’s better to have no basal ganglia output than a ‘noisy’ one” to be refined. We can now say that it may be better to have no output from stimulus–response habitual control circuits than a ‘noisy’ one. The proposal of dual mechanisms for control of instrumental behaviour predicts that any treatment that stops or reduces distorting inhibitory signals coming from malfunctioning stimulus–response habitual control circuits that originate in the sensorimotor basal ganglia, should make goal-directed control easier. Presumably, therefore, surgical treatments work by blocking distorting efferent signals that make it difficult for the goal-directed system to operate efficiently (FIG. 5). In this context, it is worth pointing out that the most beneficial therapeutic results are achieved when inactivating treatments are directed to the sensorimotor — as opposed to the limbic or associative — territories of relevant basal ganglia nuclei¹⁴⁵.

A double dissociation

Our review of the neural substrates of goal-directed and stimulus–response habitual responding in animals highlights a double dissociation in which inactivating rostromedial striatal loops impair goal-directed — but not stimulus–response habitual — control^85,86, whereas inactivation of caudolateral striatal loops blocks habitual — but not goal-directed — responding^84,87. A similar double dissociation seems to emerge from comparing patients with Parkinson’s disease with those suffering from cognitive abulia^146,147. Patients with abulia exhibit apathy, loss of motivation, a reduction in spontaneous behaviour and extreme poverty of thought, yet can perform a wide range of tasks if they are instructed to do so. For behaviour to continue it must be maintained by external sensory stimulation. Abulia seems to be caused by focal lesions to medial sectors of the basal ganglia, which may include both associative and mesolimbic territories. A favoured interpretation is that abulia represents a catastrophic loss of internally motivated goal-directed behaviour with relative preservation of direct stimulus–response control^146,147. If this is the case, patients with abulia show a double dissociation with patients in the early stages of Parkinson’s disease, who seem to retain goal-directed abilities but have a relative loss of stimulus–response habitual control. It is, however, important to recognize that with increasing and progressive involvement of more medial sectors of the basal ganglia some of the motivational and cognitive symptoms of abulia — in particular, apathy — also become evident in Parkinson’s disease¹⁴⁸.

Affective assistance

Although we have concentrated on goal-directed and habitual control by the basal ganglia, to ignore influences of the mesolimbic territories on the behaviour of patients with Parkinson’s disease would be a mistake (see REF. 149 for a recent review). Insofar as the ventromedial dopaminergic systems seem to be comparatively spared^107–112 in Parkinson’s disease, we would predict a commensurate sparing of motivational and emotional function in patients with Parkinson’s disease. Thus, the relatively preserved motivation- and emotion-based selections made within intrinsic microcircuits of the limbic territories of the basal ganglia should still be able to guide and energize¹⁵⁰ similarly spared goal-directed responding. The notion that powerful affective stimuli assist in goal-directed behavioural control can be seen in the oft-quoted example of patients with Parkinson’s disease responding appropriately to a fire alarm or other emergencies. Similarly, classical Pavlovian associations and their ability to influence instrumental responding would be expected to be comparatively preserved in these patients, especially at the beginning of early-onset Parkinson’s disease — this is because these associations are crucially dependent on neural processing in limbic structures^131,132 and their connections with striatal territories that are innervated by the mesolimbic dopamine system, which is comparatively preserved during the initial stages of Parkinson’s disease.

Conclusions and perspectives

The conceptual level of the anatomical framework proposed in this article, and its implications for the mechanisms underlying the symptoms of Parkinson’s disease, are different to those proposed by the original ‘direct versus indirect pathway’ model^2,3. In the original model the opposition was between anatomical pathways (the ‘direct’ and ‘indirect’ circuits facilitating and suppressing movement, respectively) (FIG. 1a). In the current framework, the opposition is based on dual, goal-directed and stimulus–response habitual control systems operating through regionally segregated functional territories of the basal ganglia (FIGS 3,5). Because the current proposal is at a different functional level of description, it makes no prescriptions concerning the relative contributions of the ‘direct’, the several ‘indirect’¹⁹ and ‘hyperdirect’ projections¹⁴, and different cell types¹⁵¹ to the dysfunctional output signals in Parkinson’s disease. Similarly, it is at a different level of discussion to the current debate over whether the distortion of basal ganglia output in Parkinson’s disease and animal models is best characterized by disturbances of rate-coding or pattern-coding^140,144,152. The current analysis simply acknowledges that — although patterns of intrinsic connectivity are viewed differently now — it is within the ‘direct’, ‘indirect’ and ‘hyperdirect’ circuits of mainly caudolateral sensorimotor territories of the basal ganglia nuclei that a chronic lack of dopamine causes both the principal distorting pathological output from the basal ganglia and deficits in the selection of habitual responses in patients with Parkinson’s disease. Clearly, the ongoing research effort to discover the intrinsic mechanisms underlying sensorimotor dysfunction in the basal ganglia is vitally important, particularly with regard to the discovery of novel therapeutic advances.

The importance of the current proposal is that it provides a higher, systems-level account of the functional architecture of the basal ganglia by taking into account that different regions of a common structural architecture seem to be crucial for goal-directed and habitual control. As the loss of dopamine is greatest in territories associated with habitual control, the main symptoms of Parkinson’s disease should reflect a relative reduction in the expression of habits. The current analysis also connects with an extensive literature^{115,116,121,125} that has characterized the comparative loss of automatic and implicit control mechanisms in patients with Parkinson’s disease. Moreover, because goal-directed and habit systems operate through a ‘final common motor path’, the distorting output from stimulus–response habit circuits must be overridden for goal-directed control to be expressed. Consequently, the reports that surgical lesions ameliorate behavioural disturbances associated with the disease¹⁴² become less paradoxical.

However, important issues remain. First, throughout our discussion of behavioural control we use formal terminology from the learning theory literature (‘goal-directed’ and ‘habit’), and wherever possible restrict our review of the animal and human literature to studies that include the formal tests of reinforcer devaluation and/or contingency degradation^63,69,72,81. However, in the clinical, psychological, animal behaviour and neuroscience literatures, different terminologies are used to refer to what might turn out to be the same — or similar — concepts. Thus, the terms ‘conscious’, ‘voluntary’, ‘explicit’, ‘internally generated’ and ‘controlled’, may often have substantial overlap with the concept of goal-directed control⁸³. Alternatively, ideas encapsulated by the terms ‘unconscious’, ‘involuntary’, ‘implicit’, ‘externally generated’ and ‘automatic’, may all denote behaviour that is mainly under habitual control⁸³. Future investigations will determine the degree of overlap between these concepts and how far they need to be refined. In part, this will benefit from the continuing development of formal tests of goal-directed and habitual control in both human and animal studies⁶⁵. However, the results of these investigations will have serious implications for how the behavioural deficits of patients with basal ganglia diseases, including Parkinson’s disease, will be interpreted, and how such interpretations are associated with neural processing in the different functional territories of the basal ganglia. Furthermore, our analysis makes different predictions for the different therapeutic interventions used for Parkinson’s disease. On the one hand, dopamine replacement therapies might be expected to reinstate some degree of habitual sensorimotor control^153,154, whereas this should be impossible following the bilateral destruction or tonic suppression of sensorimotor loops in the basal ganglia by surgical interventions¹⁴⁴. As has been mentioned elsewhere¹¹⁸, experiments that test the effects of a range of therapeutic interventions on the performance of formally determined goal-directed and habitual tasks by patients with Parkinson’s disease are warranted. Initial studies that were limited to a few patients are encouraging in that lesions of basal ganglia output systems in patients with Parkinson’s disease improved basic motor features, whereas new learning and complex motor performance remained impaired¹⁵⁵. The importance of formal testing is also highlighted by a recent report that habit formation in an instrumental conflict task was preserved in patients with Parkinson’s disease¹⁵⁶. Unexpectedly, however, in the condition in which patients were assumed to be operating in habit mode their reaction times were longer than when their performance was goal-directed. It is therefore possible that conflict-induced habit formation may be different from that induced by extensive training (which results in quicker habitual responding)^83,157. A further important issue for future research will be to determine whether executive difficulties associated with initiating and executing goal-directed performance are due to the primary disease state (that is, the loss of dopamine from goal-directed circuits¹²⁶) or because goal-directed control is overwhelmed by the absence of automatic control routines that are normally provided by the stimulus–response habit systems^116,121. In the case of future animal models, it will be important to mimic the loss of dopamine from sensorimotor territories of the basal ganglia seen in patients with Parkinson’s disease¹¹⁹, rather than creating near-complete loss of dopamine, as is common in many of the current models. The same would be true for other conditions in which differential impairment across the functional subregions of the striatum may have taken place — for example, in patients with different phenotypes of Huntington’s disease and in patients with Tourette’s syndrome with or without obsessive–compulsive disorder^158,159. It is equally important to investigate the consequences of differential loss of dopamine between brain hemispheres and the role of the less affected hemisphere in therapeutic reinstatement of behavioural control. Indeed, the manner in which the basal ganglia operate bilaterally in normal and pathological states has been sadly neglected and should be an important focus of future research.

Glossary

Basal ganglia

One of the fundamental processing units of the vertebrate brain. In mammals they comprise the striatum, globus pallidus, substantia nigra and subthalamic nucleus.

Striatum

The principal input nucleus of the basal ganglia consisting of the caudate nucleus and putamen.

Parkinsonian state

A state characterized by akinesia, bradykinesia and tremor associated in the initial stages with a differential loss of dopamine from the sensorimotor territories of the striatum.

Dyskinetic state

A state characterized by involuntary, repetitive body movements.

Hemiballismus

A movement disorder that is characterized by large-amplitude involuntary flinging motions of the extremities.

Instrumental learning

Learning in which a reinforcing outcome is contingent on the performance (or withholding) of a particular behaviour. Thus, the subject’s response is ‘instrumental’ in producing an outcome (typically a food reward).

Associative learning

Any learning process in which a new response becomes associated with a particular stimulus. In animal behaviour it generally denotes learning that occurs through classical and instrumental conditioning.

Fixed ratio schedules

(Often abbreviated to FR schedules.) Situations in which there is a fixed relationship between responses and outcomes. For example, in an FR3 schedule, every third response is reinforced.

Interval schedules

Reinforcement is delivered with the first response following a fixed or variable interval, making the association between response and outcome much weaker.

Set shifting

The ability to modify ongoing behaviour in response to changing goals or environmental experiences.

6-hydroxydopamine lesion model of Parkinson’s disease

A neurotoxin that, when administered to the substantia nigra of rodents, causes degeneration of the ascending nigrostriatal dopaminergic pathway, thereby mimicking the loss of dopaminergic innervation in Parkinson’s disease.

Oscillations

(Referring here to oscillations in Parkinson’s disease.) Resting tremor that is thought to be related to the increased levels of synchronous neuronal activity observed within the basal ganglia of patients with Parkinson’s disease and in animal models of the disease.

Abulia

A condition characterized by a reduction in goal-directed verbal, motor, cognitive and emotional behaviours and that is associated with focal lesions of the basal ganglia.

Poverty of thought

A form of ‘psychic akinesia’ in which patients are generally apathetic and exhibit a profound lack of ‘will’.