Cortico-basal ganglia mechanisms for overcoming innate, habitual and motivational behaviors

Abstract

Most of the human behaviors are executed automatically under familiar circumstances. These behaviors are prepotent in that they take precedence over any other potential alternatives. Yet, humans are also capable of engaging cognitive resources to inhibit such a prepotent behavior and replace it with an alternative controlled behavior in response to an unforeseen situation. This remarkable capability to switch behaviors in a short period of time is the hallmark of executive functions. In this article, we first argue that the prepotent automaticity could emerge at least in three different domains – innate, habitual and motivational. We then review neurophysiological findings on how the brain might realize its switching functions in each domain, primarily by focusing on the monkey oculomotor system as the experimental model. Emerging evidence now suggests that multiple neuronal populations in the shared cortico-basal ganglia network contribute to overriding prepotent eye movement, be its origin innate, habitual or motivational. This consideration suggests the general versatility of the cortico-basal ganglia network as the neural mechanism whereby humans and other animals keep themselves from becoming subservient to reflex, habit and motivational impulses.

Introduction

Many of the human behaviors can take place nearly automatically with little conscious, volitional or attentional control. These automatic behaviors can be considered prepotent in that they predominate over other potential alternatives under certain circumstances. For example, we automatically take the same route when driving to work. On the way to work, we unintentionally read advertising copies on signboards beside the road while steering or changing gear without thinking much about it. These prepotent and automatic responses may continue as long as the environment remains as usual. However, when confronted with a novel or potentially dangerous situation (e.g. roadworks), we need to stop the ongoing, prepotent responses and decide quickly what to do next in accordance with the novel situational demand (Norman & Shallice, 1986). Such a response override allows us not only to discontinue now-invalid prepotent responses, but also to replace them with an alternative, more contextually appropriate response (Hikosaka & Isoda, 2010). The capacity to switch or redirect action in this way is crucial. Without it, our behavior would lose flexibility and become increasingly bound to the external stimulus (Fernandez-Duque et al., 2000).

Switching of actions in the face of new contexts constitutes the core aspect of an executive function (Luria, 1973). Executive function is an umbrella term to encompass a complex cognitive construct, such as inhibitory control, switching attention, working memory, and planning and monitoring of actions. These functions share the need to disengage from the immediate environment to guide actions in more flexible and adaptive manners. That is, executive functions allow us to keep ourselves from being automatically controlled by the prepotent response tendency.

The purpose of the present article is to review recent findings on functional roles of the cerebral cortex and subcortical basal ganglia in overcoming prepotent responses in favor of an alternative behavioral goal. For this purpose, we focus mainly on the saccadic eye movement system in non-human primates. Saccades are rapid eye movements by which we shift our line of sight to objects of interest. Saccades contribute to a variety of ocular motor behaviors, ranging from reflexive movements towards novel stimuli to remembered sequences of gaze shifts made as part of learned tasks. The reason for using saccades as the experimental tool is threefold (Leigh & Kennard, 2004). One reason is that saccades are easily accessible to measurement in the laboratory or clinical observation. The second reason is that their dynamic properties are well delineated. The third is that their neurobiological substrate has been well characterized. Neural network models obtained from the saccadic system may be applicable to other domains of motor behavior.

In this review, we first summarize the concept of response automaticity and response override. Secondly, we outline three domains of prepotent automaticity – innate, habitual and motivational. Thirdly, we summarize the neural circuitry by which the cerebral cortex and the basal ganglia control saccadic eye movement. Fourthly, we review recent findings supporting that the cortico-basal ganglia network contributes to the switching of saccades. Finally, we discuss the coexistence of two opposing yet functionally interrelated systems in the cortico-basal ganglia network, one for accelerating the prepotency and the other for breaking the prepotency.

Concept of prepotent automaticity – efficient yet inflexible

As we become more practiced at something, the action becomes more automatic and demands less control. In his book, The Principles of Psychology (1890), William James noted, ‘the more of the details of our daily life we can hand over to the effortless custody of automatism, the more our higher powers of mind will be set free for their own proper work’. This remark exactly points to the importance of automaticity in thought, emotion and action. In everyday life, we find ourselves automatically prompted to think, feel or do what we have before, and are accustomed to think, feel or do under similar circumstances. It seems as if we have a diverse range of ‘default’ response repertoires, each of which is prone to being elicited by a particular contextual stimulus. Such a default mode of responding is prepotent, as it is dominant over any other potential responses that one might make in a given circumstance. The past several years have seen dramatic breakthroughs in our understanding of how the prepotent automaticity is formed by the network linking the cerebral cortex and subcortical neural structures (Salmon & Butters, 1995; Hikosaka et al., 1999; Graybiel, 2008; Ashby et al., 2010).

The prepotent automaticity indeed makes our life efficient and easy in many respects – it can free limited mental resources from the numerous routine requirements of life. However, automatic processing is not always a blessing – it is inflexible and therefore difficult to control (Schneider & Chein, 2003). What is worse, the automatic prepotency sometimes interferes with a contrary intention. The Stroop effect is a good example in the laboratory setting (Stroop, 1935; MacLeod, 1991). When subjects are presented with a color word written in a conflicting color (e.g. ‘RED’ printed in blue) and are asked to name the color of the word (i.e. blue), they show difficulty suppressing the dominant tendency to read the word. In this example, an irrelevant stimulus dimension, i.e. the word’s meaning, influences subjects’ responses despite their efforts to ignore the irrelevance. Note that the degree of prepotency critically depends on the level of familiarity with a language used in the task. A subject who has never been exposed to English should not be subjected to the Stroop interference effect when color words are written in English.

The automatic prepotency seems enhanced in some disorders of the brain. For example, people suffering from a neuropsychiatric disorder, such as obsessive-compulsive disorder and drug addiction, show considerable difficulty suppressing their maladaptive, impulsive behaviors – such as recurrent, unwanted thoughts and repetitive drug-seeking behavior (Gerdeman et al., 2003; Hyman et al., 2006; Graybiel, 2008). In the case of utilization behavior, subjects are unable to inhibit actions afforded by everyday instruments, such as matches, scissors and combs, even when those actions are contextually maladaptive (Lhermitte, 1983; Shallice et al., 1989). The enhancement of automaticity in these disorders may also be caused by the attenuation of voluntary control.

Origins of prepotency

There are at least three types of mechanism that are capable of inducing the prepotent response tendency – innate, habitual and motivational. The innate mechanism directs our attention naturally or instinctively to a salient stimulus in the environment, which in turn triggers a certain type of movement. Examples include an orienting response to a flashing visual stimulus. The innate mechanism can occur without particular prior training. The habitual mechanism, in contrast, shifts our attention toward those areas of the environment in which one has considerable experience and familiarity. By the habitual mechanism, a given environmental cue can cause a person to behave in a certain way that has been chronically associated with the cue. For example, a driver automatically presses the brake pedal upon seeing a red traffic light. The innate and habitual mechanisms might be equivalent to those put forth by James in the domain of attention (James, 1890) – ‘immediate’ and ‘derived’ sources of involuntary attention, respectively. The distinction between innate and habitual prepotencies is also supported by another framework for attentional selection (Trick & Enns, 2009). In addition to the two mechanisms above, there appears another type of response prepotency originating from a motivational mechanism. That is, a prospect of a reward, or a stimulus with a high motivational value, captures one’s attention and elicits approach behavior toward it.

What is a unique feature commonly observed in these prepotencies? Notably, subjects exhibit a robust tendency to generate a certain type of movement despite not being instructed to do so. That is, subjects have difficulty keeping themselves from automatically responding. In this respect, these prepotencies are different from another form of response dominancy produced by a task instruction (such as a verbal instruction to look to the right). The precise neuronal substrate for the three types of prepotency has yet to be established fully, and may overlap with one another at least to some extent. Nonetheless, the above classification provides a useful operational division according to which we can develop behavioral paradigms that allow for the clarification of underlying physiological mechanisms.

Response override in favor of alternative behavioral goals

If our behavior is bound to external stimuli, it would be under the immediate control of the environment and lose flexibility. In fact, humans have a remarkable capability to switch from a prepotent response to a non-prepotent response in favor of a novel behavioral goal (Norman & Shallice, 1986). This capacity – often referred to as response switching (Isoda & Hikosaka, 2007) – is the hallmark of executive functions, and is thought to be accomplished by inhibitory processes that overcome a dominant prepotent response while pursuing a non-prepotent, contextually more appropriate response (Rothbart & Posner, 1985; Logan, 1994; Knight et al., 1995).

The conceptual idea behind response switching is schematically illustrated in Fig. 1. According to most prudent models, a motor response is initiated when a neural signal reflecting decision processes grows over time and reaches a certain criterion (response threshold; Ratcliff, 1978; Carpenter, 1981; Logan & Cowan, 1984). In these models, the level of the response threshold is assumed to be fixed; hence, the variability in response time originates from the variability in the rate of growth of the decision process. When two conflicting motor processes are in progress, the one that reaches the threshold first will be emitted.

Fig. 1.

Schematic illustration of fixed-threshold accumulator model in the context of overcoming habitual prepotencies. A motor response is generated when neuronal activation grows and reaches the response threshold. A, habitual response; B, non-habitual response. A putative switching mechanism inhibits or delays neural processes for the habitual response (A′) and boosts those for the non-habitual response (B′). Note that the slope is steeper for A than for B.

Now let us consider the case where a prepotent habitual response must be overcome to generate a non-habitual controlled response (Fig. 1). Thus, for the non-habitual response to occur successfully, its underlying neural process must exceed the threshold before the prepotent habitual process. However, it is generally thought that the habitual process is fast, whereas the cognitively controlled, non-habitual process is slow (Schneider & Chein, 2003; Fig. 1; expressed as a difference in slope between response processes A and B). Furthermore, the habitual process could start in an anticipatory manner under the familiar circumstance before an actual cue is given in the environment (Fig. 1; expressed as the start of rise for response A occurs before cue onset). Thus, without active – and perhaps powerful – inhibitory control, the prepotent habitual response would always win the race. Only when the inhibitory control is recruited during a critical period of time, the alternative non-habitual response has a chance to win the competition. Note that the inhibitory process does not need to eliminate the prepotent response process immediately. What is critical for the response-switching mechanism is to delay the development of the prepotent motor process (indicated by A’ in Fig. 1), during which the alternative controlled process can reach the threshold. In parallel with this inhibition, the switching mechanism may boost the rate of growth of the controlled motor process (indicated by B’ in Fig. 1). How might the brain accomplish such a switching mechanism then?

Organization of ocular motor networks

Before going into physiological mechanisms, it is useful to give a brief sketch of the anatomical organization of the oculomotor system (Fig. 2). The most critical node in the oculomotor networks is the superior colliculus (SC) on the roof of the midbrain. The integrity of the SC is crucial for the generation of saccades (Wurtz & Albano, 1980; Sparks & Hartwich-Young, 1989), including extremely reflexive ‘express’ saccades (Fischer & Boch, 1983; Schiller et al., 1987; Isa, 2002). It has been postulated that the SC contains two types of neuron (Munoz & Wurtz, 1993, 1995): one in the rostral part related to visual fixation (fixation neurons); and the other in the caudal part concerned with the movement of the eyes (saccade neurons). Other studies suggest that the rostral SC encodes small saccades (Gandhi & Keller, 1999; Hafed et al., 2009). The SC sends a command signal to the saccade generator in the midbrain and pons (Sparks & Hartwich-Young, 1989; Leigh & Zee, 1999). The SC is driven by direct inputs from cortical eye fields, including the frontal eye field (FEF; Bruce & Goldberg, 1985), supplementary eye field (SEF; Schlag & Schlag-Rey, 1987) and parietal eye field, known as the lateral intraparietal area (LIP; Barash et al., 1991). A combined ablation of the FEF and SC severely impairs the generation of saccades (Schiller et al., 1980), suggesting that the FEF–SC constitutes the main axis for saccade generation. The FEF is important for both the selection of a saccade target (Thompson et al., 1996; Sato et al., 2001) and the production of purposive saccades (Bruce & Goldberg, 1985). The SEF in the medial frontal cortex is involved in the control of saccades in more context-dependent manners, such as conditional visuomotor associations (Chen & Wise, 1995) and temporal sequencing (Isoda & Tanji, 2002; Lu et al., 2002; Histed & Miller, 2006). The LIP plays a role in the visual guidance of saccades – neurons in the LIP are strongly modulated by attention and behavioral relevance of visual stimuli (Colby et al., 1996; Gottlieb et al., 1998). Other studies suggest that LIP neurons encode intention of saccades (Snyder et al., 1997).

Fig. 2.

Schematic illustration of cortico-basal ganglia oculomotor networks. Open circles are excitatory projections (glutamatergic) and filled circles are inhibitory projections (GABAergic). CD, caudate nucleus; GPe, external segment of the globus pallidus; SC, superior colliculus; SNr, substantia nigra pars reticulata; STN, subthalamic nucleus; Thal, thalamus. The gray background indicates key nuclei in the basal ganglia. There are three major streams in the networks: the direct pathway (cortex–CD–SNr); indirect pathway (cortex–CD–GPe–STN–SNr); and hyperdirect pathway (cortex–STN–SNr).

In addition to their direct routes, the cortical eye fields can regulate the activity in the SC indirectly via the basal ganglia. The basal ganglia are a group of subcortical nuclei that have been linked to motor control. The substantia nigra pars reticulata (SNr), an output node of the basal ganglia, tonically inhibits the saccade-related activity in the SC, thereby preventing unnecessary saccades (Hikosaka & Wurtz, 1985). These γ-aminobutyric acid (GABA)ergic SNr neurons decrease or cease their firing when a saccade is initiated, leading to disinhibition of the SC (Hikosaka & Wurtz, 1983). It has been postulated that the SNr acts as a gate for saccades. The internal segment of the globus pallidus (GPi), another output node of the basal ganglia, may also participate in the control of saccades (Yoshida & Tanaka, 2009; Shin & Sommer, 2010). The saccade-related decrease in activity in the SNr and GPi is directly caused by saccade-related increases in activity in the caudate nucleus (CD). This direct innervation, which is called ‘the direct pathway’ of the basal ganglia, has been shown to be GABAergic and inhibitory (Yoshida & Precht, 1971; Hikosaka et al., 1993; Kita, 1993). In parallel with the direct pathway, the CD is capable of influencing the activity of the SNr/GPi via a serial connection with the external segment of the globus pallidus (GPe) and the subthalamic nucleus (STN). This ‘indirect pathway’ could increase the activity of SNr/GPi neurons, as GPe neurons are inhibitory (Smith & Bolam, 1990) and STN neurons are excitatory (Hammond et al., 1978; Nakanishi et al., 1987). Finally, the frontal cortex innervates both the CD and STN (Monakow et al., 1978; Inase et al., 1999; Takada et al., 2001), and the direct route from the cerebral cortex to the STN is called the ‘hyperdirect pathway’ (Nambu et al., 2002). To summarize, the cortical eye fields provide the SC with a driving force to generate a saccade. The direct pathway in the basal ganglia opens the gate for generating a saccade by removing tonic inhibition of the SC. By contrast, the indirect and hyperdirect pathways increase inhibitory outputs to the SC, thereby preventing the occurrence of unnecessary saccades.

In the following, we will review recent progress in the understanding of the role of the cortico-basal ganglia oculomotor networks in response switching, separately for each domain of response prepotency. For this purpose, we will place particular emphasis on the findings obtained using three behavioral paradigms – the antisaccade task for the innate prepotency; the saccade-overriding task for the habitual prepotency; and the biased-reward saccade task (BST) for the motivational prepotency.

Overriding innate prepotency – antisaccade task

When organisms see a bright flash of light, it automatically draws observers’ attention, leading to a saccade toward it even before they identify it. This orienting response is called visual grasp reflex (Hess et al., 1946) and is present from birth, thus requiring no particular training for its emergence. The prepotency of this kind has long been demonstrated in laboratory settings. In the visuospatial cuing task developed by Posner (1980), cues indicate the location of an upcoming target in ‘valid’ trials and the location away from the target in ‘invalid’ trials. Under certain conditions, valid trials lead to reaction time benefits and invalid trials lead to reaction time costs. In another study, exogenous cues automatically trigger shifts in visuospatial attention despite never appearing in the same location as the target (Remington et al., 1992). Furthermore, such exogenous cues produce cuing effects despite instructions that they be ignored (Jonides, 1981). Although the mechanism underlying the cueing effect is likely to include different processes, such as attention capture and inhibition of return (Klein, 2000; Dorris et al., 2002), the above observations suggest that the innate prepotency is robust and difficult to control. However, our gaze is not solely guided by salient visual stimuli. We are capable of using internal motives to bias oculomotor behavior against prepotent orienting reflexes. Among a battery of oculomotor tasks, the antisaccade paradigm (Hallett, 1978; Munoz & Everling, 2004) offers an ideal experimental tool for investigating the neuronal mechanism of response switching by overriding innate, prepotent response tendency.

Figure 3A shows a typical experimental setup for the antisaccade task. In this task, a visual stimulus is presented in the peripheral visual field, and subjects are instructed to look either toward the stimulus (prosaccade) or away from it (antisaccade). The instruction to make a prosaccade or an antisaccade is usually given by the color (or shape) of a fixation point, and the two types of trial can be randomly interleaved within an experimental session (as shown in Fig. 3A) or given in blocks. Thus, for the successful antisaccade performance, subjects must override reflexive glancing toward the stimulus (prosaccade) and instead make an antisaccade to the internally specified location. The competition between a prepotent prosaccade and a volitionally controlled antisaccade gives rise to the occasional occurrences of two saccades in quick succession, which are often called ‘turn-around’ or ‘back-to-back’ saccades (Schlag-Rey et al., 1997; Amador et al., 1998). The intersaccadic interval of these saccades is extraordinarily short (< 125 ms), indicating that their movement processes occur concurrently (McPeek & Keller, 2002).

Fig. 3.

Schematic illustration of three behavioral paradigms involving response switching. White dotted lines indicate the correct eye gaze location. (A) Antisaccade task used to examine switching from innate automaticity. Subjects are instructed to look toward (prosaccades) or away from (antisaccades) the peripheral stimulus by the fixation point color. Shown are examples in which the green fixation point instructs prosaccades and the red fixation point instructs antisaccades. The two types of trial are randomly interleaved. (B) Saccade-overriding task used to examine switching from habitual automaticity. The correct saccade target is indicated by the color of the central cue (yellow or pink). The central cue color remains unchanged during a block of trials and then is switched in the next block. Subjects cannot predict when the switching would occur. Shown are examples in which the ‘yellow block’ switches to the ‘pink block’ in the trial indicated by the red arrow. (C) BST used to examine switching from motivational automaticity. In the ‘right big’ block, the saccade to the right target is followed by a large reward, whereas the saccade to the left target is followed by a small reward. The position-reward contingency is reversed in the ‘left big’ block. Note that the position-reward contingency is fixed in each block, but the actual position of the target is unpredictable from trial to trial. Subjects have to perform a small-reward trial to proceed to the next trial.

Several lines of clinical studies converge on the notion that the preparation and execution of antisaccades is accomplished by a large-scale neuronal network involving cortical and subcortical structures (Everling & Fischer, 1998). Subjects with lesions in the dorsolateral prefrontal cortex (DLPFC; Guitton et al., 1985; Pierrot-Deseilligny et al., 1991), ventrolateral PFC (Walker et al., 1998) or medial frontal cortex, possibly including the supplementary motor area (SMA; Guitton et al., 1985), are severely impaired in antisaccade performance. Other studies show that people with basal ganglia disorders such as Huntington’s disease (Lasker et al., 1987) and Parkinson’s disease (Briand et al., 1999; Rivaud-Pechoux et al., 2007) exhibit antisaccade deficits. Consistent with these findings, neuroimaging studies of intact subjects performing the antisaccade task have demonstrated a significant signal increase in a number of neural structures, including the prefrontal cortex (PFC), FEF, pre-SMA, SEF, anterior cingulate cortex (ACC), posterior parietal cortex, basal ganglia and thalamus (O’Driscoll et al., 1995; Kimmig et al., 2001; Connolly et al., 2002; Curtis & D’Esposito, 2003; DeSouza et al., 2003; Matsuda et al., 2004; Ford et al., 2005; Miller et al., 2005; Polli et al., 2005; Brown et al., 2006, 2007; Cameron et al., 2009). These neuroimaging studies suggest that the cortico-basal ganglia networks are involved in overriding prepotent prosaccades in favor of volitional antisaccades.

How, then, does each of these areas specifically contribute to the performance of antisaccades? To answer this question, it is of critical importance to examine neuronal activity in fine temporal resolution while characterizing the motor effect individual neurons might exert on antisaccade behavior. In this sense, single-neuron recordings in behaving monkeys could provide valuable complementary data.

In the SC, saccade-related neurons exhibit reduced prestimulus, poststimulus and movement-related activities in antisaccade trials compared with prosaccade trials (Everling et al., 1999). Saccade-related neurons in the FEF show similar activity properties in antisaccade trials (Everling & Munoz, 2000). A portion of these FEF neurons was found to directly project to the SC (Everling & Munoz, 2000). The reduced activity in antisaccade trials before and after the stimulus onset could prevent the occurrence of unwanted prosaccades to a visual stimulus. However, the significantly weaker activity before antisaccades appears insufficient for overcoming a reflexive prosaccade, calling for an additional signal favoring the generation of antisaccades. Such a signal may originate from the SEF. In contrast to the SC and FEF, saccade-related activity in the SEF is increased in antisaccade trials (Schlag-Rey et al., 1997; Amador et al., 2004). The increased signal in the SEF may bypass the SC and directly reach the brain stem oculomotor center (Shook et al., 1988, 1990; Huerta & Kaas, 1990). The increased activity in the SEF might also be associated with the use of a more difficult rule in guiding saccades or conflict between two incompatible motor processes in antisaccade trials (Olson & Gettner, 2002).

Other cortical areas that play a role in antisaccade performance are the DLPFC, LIP and ACC. SC-projecting DLPFC neurons display enhanced saccade-related activity before antisaccades, particularly when the stimulus appears on the contralateral visual field (Johnston & Everling, 2006). This suggests that the DLPFC sends task-selective signals directly to the SC, thereby suppressing a reflexive glance to the contralateral stimulus and/or facilitating a non-prepotent antisaccade to the ipsilateral side. Indeed, in a human subject with likely interruption of the prefrontotectal pathway, the frequency of uninhibited reflexive saccades markedly increases toward a contralateral stimulus, and the saccadic reaction time becomes significantly longer for antisaccades directed to the ipsilateral side (Gaymard et al., 2003). In the LIP, the response to the stimulus is significantly more enhanced for correct antisaccades than for erroneous prosaccades (Gottlieb & Goldberg, 1999). This enhancement does not occur in response to the same stimulus in a memory-guided prosaccade task, suggesting that this effect cannot be attributed simply to the requirement to inhibit an immediate, reflexive saccade to the stimulus. One possibility is that the enhancement of visual responses in the LIP might be involved in the process of vector inversion leading to correct antisaccades (Zhang & Barash, 2000). Electrical microstimulation in the ACC facilitates contraversive antisaccades but delays contraversive prosaccades, suggesting a causal role of the ACC in antisaccade performance (Phillips et al., 2011).

As discussed earlier, a critical operation for successful switching is the suppression of the prepotent automatic process. Given that the outputs of the basal ganglia are inhibitory and are directed to a variety of motor structures (Alexander et al., 1986; Mink, 1996; Hikosaka et al., 2000), it can be hypothesized that the basal ganglia play an important role in the performance of antisaccades. Recent studies support this hypothesis. First, neurons in the CD display activity changes consistent with their role in the inhibition of contraversive reflexive saccades through the indirect pathway (Ford & Everling, 2009; Watanabe & Munoz, 2009), or in the generation of contraversive volitional antisaccades through the direct pathway (Watanabe & Munoz, 2009). These antisaccade-related signals may be transmitted to the globus pallidus (GP). The great majority of GP neurons exhibit an enhancement of saccade-related activity modulation in antisaccade trials (Yoshida & Tanaka, 2009). Although individual GP neurons are less sensitive to movement direction, a reversible inactivation discloses direction-selective antisaccade errors (Yoshida & Tanaka, 2009), suggesting an executive role of the GP in antisaccade performance. It is generally thought that the signal in the basal ganglia originating from the cerebral cortex is sent back to the cortex via the thalamus (Alexander et al., 1986). Therefore, it seems reasonable to assume that the thalamus receives antisaccade-selective signals from the basal ganglia. Indeed, neurons in the ‘motor thalamus’ are significantly more activated before the onset of antisaccades (Kunimatsu & Tanaka, 2010). Interestingly, however, the time courses of activity modulation are different between the GP and the thalamus – the peak of activation is earlier and activity modulation is more transient in the thalamus (Yoshida & Tanaka, 2009; Kunimatsu & Tanaka, 2010). Moreover, inactivation of the motor thalamus produces a change in saccade parameters, such as reaction times and response accuracy, whereas inactivation of the GP does not. These findings led the concerned authors to an interesting discussion that the basal ganglia may regulate the gain of the thalamocortical processing by modulating the strength of their inhibitory outputs to the thalamus, rather than regulating the antisaccade motor command per se (Kunimatsu & Tanaka, 2010).

In addition to visual and motor-related responses, the preparatory activity before the onset of the imperative stimulus could contribute to the successful performance of antisaccades. In the antisaccade paradigm, the visual target is typically identical in both antisaccade and prosaccade trials (Fig. 3A). Yet, its behavioral significance has already been cued by the previously given instruction, such as the color or shape of a fixation point. This prior information could preset the subject against responding prematurely. Indeed, the differential prestimulus activity in the DLPFC (Everling & DeSouza, 2005), FEF (Everling & Munoz, 2000), pre-SMA (Curtis & D’Esposito, 2003), CD (Watanabe & Munoz, 2010) and GP (Yoshida & Tanaka, 2009) could predict whether the reflexive glances are successfully inhibited in that trial. Such a preparatory set before target onset may be a requisite for blocking the occurrence of a prepotent prosaccade made in error, thereby guiding a volitional antisaccade.

Overriding habitual prepotency – saccade-overriding task

People find themselves automatically prompted to think or do what they have before accustomed to think or do under similar circumstances. Such habitual tendencies are prepotent and take precedence over other potential responses in a certain environment. For example, when we think of numbers, we often associate them with spatial positions as if they were placed on a horizontally extending ‘mental number line’. Dehaene et al. (1993) found in a parity judgment task that French participants were faster in pressing a left button in response to smaller digits (e.g. 0 or 1) than in response to larger digits (e.g. 8 or 9), whereas larger digits evoked faster response latencies with a right button press. In Iranian participants, by contrast, who write from right to left in their native language, large digits were associated with the left side of space and small digits with the right side of space (Dehaene et al., 1993), suggesting that directional scanning habits might play a critical role. This effect generalizes from verbal and manual responses to saccadic eye movement (Fischer et al., 2004). Another example is responses to symbolic cues (e.g. arrows). Tipples (2002) reported that reaction times were reliably faster when arrow cues pointed toward, rather than away from, the location of the upcoming target letter. This automatically triggered attentional orientation occurred despite the fact that the participants were informed that the arrows did not predict where the target would appear.

As mentioned earlier, a common task to study the effect of habitual priming on behavior has been the Stroop task (Stroop, 1935; MacLeod, 1991). However, the Stroop paradigm has rarely been applied in the domain of oculomotor behavior. The Stroop interference effect on eye movement responses has only been recently shown (Hodgson et al., 2009), which clearly demonstrates that oculomotor programming is also influenced by word meanings. Importantly, many direction errors are followed by corrective saccades within 100 ms of the end of the first saccade (Hodgson et al., 2009). Again, such very short latency corrections suggest that saccades are programmed in parallel to two goals (McPeek & Keller, 2002) – one defined by the word meaning and the other by the word color. The Stroop effect observed in human subjects critically depends on the dominant and habitual tendency to read words, which is acquired through a long period of practice. However, it is also possible to induce the Stroop-like interference effect in non-human primates (Washburn, 1994; Lauwereyns et al., 2000).

To systematically investigate the neuronal basis of switching from a habitual saccade to a controlled saccade in monkeys, Isoda & Hikosaka (2007) developed a different behavioral model called the saccade-overriding task (Fig. 3B). In this task, two stimuli (pink and yellow) are presented on each side of a central fixation point. The positions of the stimuli are randomized from trial to trial out of two possible locations. After a brief delay (200 ms), the fixation point becomes either pink or yellow, which serves as a cue instructing a monkey to saccade to the stimulus whose color is the same as the central cue. Importantly, the central cue color remains unchanged in a block of successive trials and is then switched in the next block. The task can be viewed as a change-signal task (Husain et al., 2003; Nachev et al., 2005), in which immediately before the subject is about to perform a prepotent response according to a preceding cue, a different cue could be presented. Unlike the change-signal task, however, the prepotency in the saccade-overriding task is generated internally by repeating the same response. This is the hallmark of the habitual prepotency.

The monkey performing the saccade-overriding task develops a prepotent response tendency when the habitual saccade is repeated, as indicated by very short saccade latencies yet with a high percentage of correctness (Isoda & Hikosaka, 2007). This, in turn, leads to a substantial cost (Monsell, 2003) when the animals had to switch to the alternate saccade in switch trials (red arrow in Fig. 3B). Several important findings emerged from the study of Isoda & Hikosaka (2007). First, many neurons in the pre-SMA are selectively activated in the switch trials (Fig. 4). Second, a response switching is successful if the switch-selective neuronal activity precedes the initiation of the habitual saccade (Fig. 4, red). When switching fails, these pre-SMA neurons do fire but only after the initiation of the invalid, habitual saccade (Fig. 4, blue), suggesting that the timely activation of pre-SMA neurons is required for successful switching. Third, the activation of pre-SMA neurons using electrical stimulation significantly increases the likelihood of correct switches. Fourth, the use of a go/no-go task as a control task revealed that the properties of the switch-selective activity are consistent with the role of pre-SMA neurons in the inhibition of habitual saccades, in the facilitation of alternative saccades, or both of these. Finally, and notably, the pre-SMA enables switching by first suppressing the habitual saccade and then boosting the controlled saccade. This order of neuronal actions is consistent with the requirement for the switch mechanism to occur efficiently (see Fig. 1). The suppressive role of the pre-SMA in saccade execution is supported by other studies using an electrical stimulation technique in a delayed saccade task (Isoda, 2005) and a single-neuron recording technique using a stop-signal paradigm (Scangos & Stuphorn, 2010). The importance of the pre-SMA in overcoming the habitual saccade has also been shown by a human functional magnetic resonance imaging study using a stimulus–response incompatibility task (Merriam et al., 2001).

Fig. 4.

The ensemble averaged activity for the population of pre-SMA neurons that were selectively activated when switching from the habitual prepotency in the saccade-overriding task. These neurons were active for both correct (red) and incorrect (blue) switching; however, activation was delayed when switching was unsuccessful. The activity is aligned with saccade onset.

How does the pre-SMA achieve switching from the habitual action to the alternative controlled action in the oculomotor system? Theoretically, the pre-SMA could exert facilitatory effects on saccade performance via the direct pathway, and inhibitory effects via the hyperdirect and indirect pathways in the basal ganglia (Mink, 1996; Hikosaka et al., 2000; Nambu et al., 2002), given that the pre-SMA projects to the CD and the STN (Inase et al., 1999; Takada et al., 2001). As mentioned, however, the priority of the switch mechanism is to inhibit outputs of the automatic process that proceed more quickly than the controlled process (Fig. 1). In this regard, the hyperdirect pathway involving the STN may play a leading role in successful switching, as the conduction time in the hyperdirect pathway is much shorter than that in the indirect pathway (Hikosaka et al., 1993; Nambu et al., 2000). This hypothesis is, indeed, supported by a follow-up study using the saccade-overriding task. Specifically in that study, a group of neurons in the STN showed a switch-selective activity change similar to pre-SMA neurons (Isoda & Hikosaka, 2008b). The activity in the STN, however, occurred slightly later, consistent with the hypothesis that STN neurons receive the switch-related signal from the pre-SMA. Moreover, the action of those STN neurons, assessed using the saccade go/no-go task, was usually inhibitory, suggesting that the STN mainly suppresses the no-longer valid habitual saccade. The suppressive role of the STN is consistent with studies of intact subjects performing a stop-signal task (Aron & Poldrack, 2006; Aron et al., 2007). By devising a manual version of the saccade-overriding task for human subjects, Rushworth and co-workers have recently shown that the pre-SMA and right inferior frontal gyrus play a critical role in inhibiting and reprogramming of actions via both the cortico-cortical and subcortical pathways (Mars et al., 2009; Neubert et al., 2010).

The direction-selective oculomotor effects can be accomplished by the crossed or uncrossed innervations from the output node of the basal ganglia to the SC (Beckstead et al., 1981; Jiang et al., 2003) and/or thalamus (Hazrati & Parent, 1991). For example, the pre-SMA could inhibit the ipsiversive habitual saccade by projecting to the SNr via the STN on the same side, which in turn goes to the SC on the opposite side (crossed nigrocollicular projections). Alternatively, the pre-SMA could suppress the contraversive habitual saccade using the uncrossed nigrocollicular projections. It is of interest to determine in what contexts these two populations of neurons in the output node of the basal ganglia are preferentially used. This question can be answered by identifying crossed and uncrossed nigrocollicular neurons using antidromic stimulation and characterizing their activity properties during appropriate response-switching tasks.

Overriding motivational prepotency – BST

Expectation of reward motivates our decisions and motor behaviors. When two actions are associated with different outcome values, a behavioral bias is generated internally toward the action that leads to a larger gain. Thus, subjects respond more quickly and accurately to an option that is linked to a larger or more preferable reward. This behavioral effect, which appears to reflect a positive motivational state, has been shown in laboratory settings in monkeys. For example, in making a simple manual movement, reaction times and movement times are significantly shorter when the upcoming reward is larger or more preferred (Schultz et al., 1992; Hollerman et al., 1998; Watanabe et al., 2001; Cromwell & Schultz, 2003; Minamimoto et al., 2005). Moreover, the value of a reward is higher if it will be available ‘now’ as opposed to later, a phenomenon known as temporal discounting (Mazur, 1984; Green & Myerson, 2004; McClure et al., 2004; Tanaka et al., 2004). When monkeys perform schedules containing several trials with a visual cue indicating reward proximity, their reaction times and error rates significantly decrease as the number of remaining trials decrease (Bowman et al., 1996; Shidara et al., 1998). These findings indicate that the prospect of an immediate reward exerts a powerful impact on motor behavior, thereby giving rise to a prepotent response tendency (motivational prepotency).

However, the execution of actions that are motivationally prepotent is not necessarily adaptive in everyday life. For example, people have to perform a less motivational task in the pursuit of a long-term goal or in accordance with an order from one’s superior. Moreover, humans often need to place public good before private gain in the face of social dilemma (Dawes, 1980). In these circumstances, people must override a motivationally dominant response in favor of another behavioral goal.

To investigate the neuronal mechanisms with which subjects counteract motivational eye movement, Lauwereyns et al. (2002) devised the BST for monkeys. In this task (Fig. 3C), a visual target is presented randomly at one of two possible locations and the monkey is required to make a visually guided saccade to the target. Importantly, during a block of successive trials (usually 20 trials), the saccade to one location is followed by a big reward, whereas the saccade to the opposite location is followed by a small or no reward. The mapping between the location and the reward outcome is reversed in the next block. In addition to this asymmetric reward mapping, the BST has another critical feature. The animal has to make a saccade to the target even if a smaller reward is expected after its completion; otherwise the same trial will be repeated. The use of eye movement for inducing motivational prepotencies seems advantageous, as an important component of reward-modulated behavior is orienting movement (Swanson, 2000) and an important component of orienting movement is saccade (Hayhoe & Ballard, 2005). The animals often orient their eyes to the location where a reward is available before foraging it (Ewert, 1980).

Animals develop a robust motivational prepotency toward the reward-predicting target. The saccadic response is reliably faster when a big reward is expected than when a small reward is expected (Lauwereyns et al., 2002; Watanabe et al., 2003a; Ding & Hikosaka, 2007). Moreover, the animals occasionally make misdirected saccades in small-reward trials (Isoda & Hikosaka, 2008a). In such direction errors, the first saccade is invariably directed to the position associated with a big reward despite the absence of the actual target in that location. The misdirected saccade is then quickly followed by a second saccade toward the correct, visible target. These behavioral findings suggest that the animals are motivated to make a saccade to the position associated with a big reward. Interestingly, such a response bias toward a big-reward position grows over time during a fixation period before target onset (Ding & Hikosaka, 2007).

One illustration of how the oculomotor system effectively generates a motivational bias is provided by neurons in the CD, which exhibited a gradual increase in activity before target onset (Lauwereyns et al., 2002). This activation occurs selectively or preferentially when one particular spatial position (usually contralateral to a neuron under study) is associated with a big reward (Takikawa et al., 2002). The anticipatory activity would then be transmitted to the SC (Ikeda & Hikosaka, 2003) via the SNr (Sato & Hikosaka, 2002), leading to a strong imbalance in favor of saccades that are associated with a high incentive (Hikosaka et al., 2006).

In the BST, monkeys have to make a saccade to the target associated with no or less incentive value. Hence, a motivational conflict arises in small-reward trials between two incompatible saccade plans – one triggered by the motivational prepotency (a saccade directing toward a big-reward position) and one that is instructed by the visual target (a saccade toward a small-reward position). Indeed, a neuronal correlate of such a motivational conflict has been found in the SC, a subcortical structure close to the motor output (Isoda & Hikosaka, 2008a). It has been proposed that the cerebral cortex has a mechanism that can effectively prevent competing motor commands from arising simultaneously in the downstream motor areas (Schlag et al., 1998). The occurrence of the motivational conflict in the SC implies, however, that such a cortical mechanism is not sufficient to fully resolve a response conflict.

The fact that the response conflict exists in the SC calls for additional mechanisms that inhibit the motivationally prepotent saccade and facilitate the less-dominant but now-valid saccade. How might this response override be achieved? Whereas neurophysiological investigations have largely concentrated on reward-seeking behaviors and the generation of a motivational bias (Schultz, 2000; Wickens et al., 2003; Sugrue et al., 2005; Hikosaka et al., 2006), the neuronal basis for counteracting the motivational prepotency is less clearly understood. However, the following observations may provide important clues. During the performance of the BST, about 40% of saccade-related neurons in the CD fire selectively or preferentially in unrewarded trials (Watanabe et al., 2003b). Notably, the activity of these neurons inversely correlates with saccade latency. This suggests that their increased activity in unrewarded trials is used to counteract the motivational bias rather than to enhance it. This interpretation fits well with the finding that the blockade of dopamine-D2 receptors in the CD enhances response bias by further delaying saccade initiation in small-reward trials (Nakamura & Hikosaka, 2006). Another possible mechanism to counteract the motivational bias is via mutual inhibitory projections between two sides of the SC (Munoz & Istvan, 1998; Takahashi et al., 2005, 2007). According to this view, subthreshold activation of two opposing motor programs continues until a motor command for the required saccade prevails against the other, which inevitably causes a response delay. Yet, such a competitive intercollicular mechanism would eventually contribute to the resolution of conflict. A role of conflict-related cortical areas (Stuphorn et al., 2000; Botvinick et al., 2001; Nachev et al., 2005) might be to tip the balance of activity between two sides of the SC using their direct connections or indirect connections through the basal ganglia. Future work should address whether the neural substrate for intercollicular inhibition (Munoz & Istvan, 1998; Takahashi et al., 2005, 2007) is fully functional during the actual performance of purposive oculomotor behavior.

Although tested using a manual motor task, experimental findings by Minamimoto et al. (2005) offer an important insight into a role of the thalamus in counteracting response bias triggered by the motivational prepotency. In their task, the monkeys performed a go/no-go task with asymmetric reward schedules. They found that virtually all of the reward-sensitive neurons in the centromedian nucleus (CM) of the thalamus fired significantly more strongly in small-reward trials. Furthermore, when the activity of these neurons was higher, the animals were more likely to complete small-reward trials. The CM might counteract response bias by directly acting on the basal ganglia or through the cerebral frontal cortex (Nakano et al., 1990; Steriade et al., 1997; Hatanaka et al., 2003). It is interesting to determine whether this region of the thalamus also participates in counteracting motivational bias in the oculomotor domain.

Concluding remarks and future perspectives

The oculomotor prepotency can occur in at least three different domains, each of which has been successfully incorporated into a unique experimental paradigm. In the antisaccade task, subjects’ attention is automatically drawn to the sudden onset of a visual stimulus, which is the only sensory stimulus in the visual field. In the saccade-overriding task, there are two visual stimuli in the environment, yet the response prepotency is internally generated toward one of them by repeating the same response habitually. In the biased saccade task, a prepotent response tendency is generated with the expectation of a reward toward a spatial location where no visible stimulus is present. This motivational bias continues even after a visual stimulus is flashed as a saccade target in the opposite location. Although the origins of these prepotencies may differ from each other, several lines of research demonstrate converging evidence that the cortico-basal ganglia networks play an important role in overcoming response prepotencies in favor of an alternative, less-prepotent response. This is achieved by inhibiting an automatic, inappropriate movement and by promoting a now-valid controlled movement. Such switching of actions underlies adaptive behavior in the changing environment and in the face of a response conflict.

Note that the cortico-basal ganglia networks have been implicated in the formation and expression of response automaticity. That is, many habitual and stereotyped behaviors develop as the result of experience-dependent plasticity in the basal ganglia and associated neural circuits (Graybiel, 2008; Ashby et al., 2010). For example, neurons in the sensorimotor striatum respond more strongly after overlearning of sequential procedures (Hikosaka et al., 1999), and temporary inactivation of this region disrupts the execution of previously acquired motor repertoires (Miyachi et al., 1997). The formation of motivational bias to a reward or a reward-predicting stimulus depends on the activity of dopaminergic neurons and their projections to the striatum (Schultz, 1998; Kawagoe et al., 2004; Hikosaka et al., 2006). Furthermore, it has been shown that the cortico-basal ganglia subcircuits may be reactivated or misactivated in disorders producing repetitive thoughts and overt behaviors (Graybiel, 2008). From these findings, it is conceivable that the networks linking the cerebral cortex and the subcortical basal ganglia are used not only for overriding response prepotency but also for the generation of the prepotent automaticity. To what extent are subcircuits underlying the two opposing functions anatomically segregated within the cortico-basal ganglia networks? The following findings may help answer this question. The acquisition of novel sequential procedures requires a more anterior ‘associative’ part of the networks, whereas the execution of old procedures is mediated by a more posterior ‘sensorimotor’ part (Hikosaka et al., 1999). This anatomical segregation is consistent with findings obtained in lesioned rats, demonstrating that the dorsomedial (associative) and dorsolateral (sensorimotor) striata regulate goal-directed and habitual control, respectively (Yin & Knowlton, 2006). In contrast to these observations favoring the existence of anatomical segregation, however, it has been shown that neurons associated with automatic prosaccades and volitional antisaccades are intermingled within the CD during the performance of the antisaccade task (Watanabe & Munoz, 2009). This discrepancy might be attributable to the difference in the movement effector (arm or eye) or to specific aspects of behavioral paradigms.

The above argument can be extended to include functional roles of the thalamus that constitute the cortico-basal ganglia-thalamo-cortical loops. As mentioned earlier, the motor thalamus is important for overcoming innate prepotency in antisaccade performance (Kunimatsu & Tanaka, 2010), and CM is concerned with counteracting the motivational prepotency (Minamimoto et al., 2005). The CM appears unique in that virtually all neurons are more activated under small-reward conditions, where response override is necessary in the pursuit of long-term goals. On the other hand, the CM–parafascicular complex (CM/PF) is involved in the generation of prepotent automaticity. Lesioning or chemical inactivation of the CM/PF complex impairs attentional orientation (Mancia & Marini, 1995; Minamimoto & Kimura, 2002), and inactivation of the CM/PF complex almost completely abolishes the responses of striatal neurons to reward-associated stimuli (Matsumoto et al., 2001). One possibility is that PF neurons having the ‘short-latency facilitatory responses’ (Matsumoto et al., 2001) are more involved in automatic processes, whereas CM neurons having the ‘long-latency facilitatory responses’ (Matsumoto et al., 2001; Minamimoto et al., 2005) are more concerned with controlled processes. This view is consistent with the finding that habituation of sensory responses is particularly common for the latter neuronal population (Matsumoto et al., 2001). The use of the two separate populations in the CM/PF may critically depend on the behavioral context (Kimura et al., 2004).

A conceptual distinction between automatic and controlled processing has a long history, which has led to a theory proposing that controlled behavior requires much attentional resources and flexible cognitive processes, and is therefore more dependent on the cerebral cortex. By contrast, automatic behavior requires neither of these and is therefore primarily subserved by subcortical structures. However, as we have reviewed in this article, both types of behavior are mediated by the intact cortico-basal ganglia network. Most of our intelligent behaviors necessitate the dynamic interplay between automatic and controlled modes of neural processing. These two modes operate in a mutually complementary manner: automatic processes permit efficient everyday functioning, whereas controlled processes enable coping with novel or difficult situations. Future research should address the issues of whether there exists a direct cross-talk between automatic and controlled systems in the cortico-basal ganglia network and whether these systems use the same computational operation across cognitive, emotional and sensorimotor domains.

Abbreviations

ACC

anterior cingulate cortex

BST

biased-reward saccade task

CD

caudate nucleus

CM

centromedian nucleus

CM/PF

centromedian–parafascicular complex

DLPFC

dorsolateral prefrontal cortex

FEF

frontal eye field

GABA

c-aminobutyric acid

GPe

globus pallidus externus

GPi

globus pallidus internus

LIP

lateral intraparietal area

SC

superior colliculus

SEF

supplementary eye field

SMA

supplementary motor area

SNr

substantia nigra pars reticulata

STN

subthalamic nucleus