How do the basal ganglia contribute to categorization? Their role in generalization, response selection, and learning via feedback

Abstract

This article examines how independent corticostriatal loops linking basal ganglia with cerebral cortex contribute to visual categorization. The first aspect of categorization discussed is the role of the visual corticostriatal loop, which connects the visual cortex and the body/tail of the caudate, in mapping visual stimuli to categories, including evaluating the degree to which this loop may generalize across individual category members. The second aspect of categorization discussed is the selection of appropriate actions or behaviors on the basis of category membership, and the role of the visual corticostriatal loop output and the motor corticostriatal loop, which connects motor planning areas with the putamen, in action selection. The third aspect of categorization discussed is how categories are learned with the aid of feedback linked dopaminergic projections to the basal ganglia. These projections underlie corticostriatal synaptic plasticity across the basal ganglia, and also serve as input to the executive and motivational corticostriatal loops that play a role in strategic use of feedback.

Categorization of people (friend or foe), objects (food or nonfood), and environments (dangerous or safe) is vital for survival in the world. The process of categorization involves both knowledge of category structure and linkage of category membership to behavior. Categorical knowledge must be sufficient to allow the organism to correctly classify each member. Categorization requires an appropriate level of generalization: generalization needs to be sufficient to correctly identify category members that are encountered for the first time, but limited so that nonmembers are not included. Categorization processes must also link category members to appropriate behaviors. For example, take the problem of identifying whether a fruit is good to eat. The organism must have a category of edible fruit acquired from past experience (e.g., the blackberries consumed last summer), and will ideally generalize so that related items are also considered edible (e.g., berries that somewhat differ in size or color), but not over generalize to fruits that are sufficiently novel that they might not be edible (e.g., holly berries). Then the categories must be linked to appropriate behaviors (e.g., ingestion of the berries categorized as safe, and avoidance of the others).

To learn new categories, there must be plasticity that allows learning of both representations (acquiring new categories, extending or tuning already acquired categories), and new links between categories and behaviors (both learning new behaviors, and extending previously learned behaviors to new categories). Traditional cognitive psychology approaches to categorization have emphasized the how category structure is represented and learned. This approach has led to a rich literature examining learning of many different forms of category structure, including prototype learning (stimuli are distortions of a prototypical stimulus), family resemblance (stimuli share varying subsets of features), decision bound (categories are defined on the basis of a decision bound in feature space separating members of each category), among others. The linkage between categorization and the selection of appropriate behavior has historically received less attention, though recently behavioral research in this area has begun (Ashby et al., 2003; Maddox et al., 2004).

One important neural system involved in categorization and category learning is the corticostriatal system connecting cortex and basal ganglia. Almost all regions of cortex send projections to the input structures of the basal ganglia, which include the striatum (caudate, putamen, and nucleus accumbens) and the subthalamic nucleus (STN). The striatum and STN send projections to several nuclei collectively termed basal ganglia output nuclei, including the globus pallidus, internal segment (GPi) and the substantia nigra pars reticulata (SNr). From the basal ganglia output nuclei, there are projections to thalamus and then back to the cortex, forming “loops” (Alexander, DeLong, & Strick, 1986). Figure 1 shows the pathways connecting cortex and basal ganglia.

Figure 1.

Main pathways through the basal ganglia. GPe: Globus pallidus, external portion. GPi: Globus pallidus, internal portion. SNr: Substantia nigra pars reticulata. SNc: Substantia nigra pars compacta. STN: Subthalamic nucleus. VTA: Ventral tegmental area.

Different cortical areas have predominant projections to different striatal regions (Alexander et al., 1986; Lawrence et al., 1998; Nambu et al., 2002). From striatum, cortical input from different regions is kept separate as it projects to basal ganglia output structures, then back to cortex (Parent & Hazrati, 1995). Based on these differences in connection, it is possible to identify independent corticostriatal loops. This paper will focus on the roles in categorization played by the four loops identified by Lawrence et al. (1998), referred to here as the executive (called spatial by Lawrence et al., 1998), visual, motor, and motivational (called affective by Lawrence et al. 1998) loops.¹ The primary cortical and striatal areas involved in each loop are shown in Figure 2. All four of these loops play some role in categorization, but the specific role of each depends to a large part on the particular cortical regions participating in the loop.

Figure 2.

Four primary corticostriatal loops. GPi: Globus pallidus, internal portion. SNr: Substantia nigra pars reticulata.

This paper examines how the basal ganglia contribute to all three aspects of categorization: category representation, behavioral selection, and learning. It will focus on how the four corticostriatal loops (visual, motor, executive, and motivational) each contribute to the cognitive processes that are important for categorization. The first aspect of categorization is the representation of category structure. Particularly important for visual categorization are the projections from the visual cortex acting through the visual corticostriatal loop. One important question is what role, if any, the visual corticostriatal loop plays in generalization across category members. The second aspect of categorization is the adoption of appropriate behaviors. The basal ganglia play a general role in selecting actions, which in the domain of categorization translates into enabling appropriate behavior on the basis of category membership. In this section, the important corticostriatal loops are the output of the visual loop (which projects to Pre-Supplementary Motor areas), and the motor loop (which directly selects movements from the motor programs present in motor planning regions). The third aspect of categorization is learning from experience, in particular via feedback from the environment. Dopaminergic projections to the striatum deliver information concerning feedback or reward that follows the executed behavior, which can then be used to enhance synaptic plasticity across all corticostriatal projections. In addition, feedback signals are processed in the motivational and executive loop to allow for secondary responses to feedback such as strategy modification.

1. Categorization in the visual corticostriatal loop

The first role of the basal ganglia in categorization is interacting with visual cortex to process visual information. In this section I first discuss the anatomy of the visual corticostriatal loop, and evidence that this loop is important in visual categorization and related tasks. I then discuss the question of generalization within categories, and how the basal ganglia may or may not participate in generalization.

1.1 The visual corticostriatal loop

The visual corticostriatal loop links ventral visual pathway regions in the inferior temporal cortex and extrastriate occipital cortex with the body and tail of the caudate (Middleton & Strick, 1996; Updyke, 1993; Webster et al., 1993). Outputs from the visual loop return to visual cortex, and additionally project to premotor regions; the function of these latter projections in action selection is discussed in more detail below in section 2. 3. 1. In monkeys, there is a trend for earlier visual processing areas (such as extrastriate occipital area BA19) to project to the body of the caudate and later visual processing areas (such as the anterior temporal lobe) to project to inferior portions of the tail of the caudate (Webster et al., 1993, Saint-Cyr et al., 1990, Yeterian & Pandya, 1995). Electrophysiological studies show that caudate cells are sensitive to visual information; they increase their activity when patterns are changed (Rolls, 1994) and when stimuli are novel (Cann et al., 1984).

Human fMRI research finds activity of the body and tail of the caudate during visual categorization (Cincotta & Seger, 2006; Nomura et al., 2007; Seger & Cincotta, 2005, 2006). Activity in the body and tail of the caudate is independent of the receipt of feedback, and is specifically linked to correct categorization of category members (Seger & Cincotta, 2005). Convergent research indicates that patients with basal ganglia disorders are impaired on visual categorization tasks (space considerations preclude complete discussion of these impairments here; for reviews, see Ashby & Maddox, 2005, Packard & Knowlton, 2002, and Shohamy et al., this issue).

Research in nonhuman animals indicates that the visual loop is necessary for several visual learning tasks. The tail of the caudate (Fernandez-Ruiz et al., 2001; Teng et al., 2000) and temporal lobe visual processing area TE (Buffalo et al., 1998) are both required for monkeys to learn concurrent visual discrimination tasks. Conversely, visual discrimination learning is preserved when all other connections from visual areas of the inferior temporal lobe other than the connection to the striatum are severed (Gaffan & Eacott, 1995). Cells in the tail of the caudate are active in monkeys during visual discrimination learning (Brown et al., 1995). In addition, c-2-deoxyglucose imaging shows that the body of caudate is active in object alternation working memory and pattern discrimination learning (Levy et al., 1997).

1.2 The question of generalization in corticostriatal categorization

One of the important aspects of category learning is representing the category members in such a way as to allow for an appropriate level of generalization. Do the basal ganglia contribute to generalization? Previous research, reviewed in the next section, is inconclusive as to whether the basal ganglia generalize, and, if so, what the mechanism underlying generalization is. An examination of the neural circuitry of the basal ganglia reveals two potential ways generalization may be achieved. One is as an effect of the convergence of cortical information onto the striatum. The other is by receiving input from the cortex that reflects generalizations across stimuli represented within higher order visual areas.

1.2.1 Empirical results relevant to category generalization in the basal ganglia

Similar recruitment of the basal ganglia is observed both in tasks that do and do not require generalization to novel stimuli. Tasks involving generalization to novel stimuli include learning dot pattern classification via feedback (Vogels et al., 2002), and information integration categorization (Cincotta & Seger, 2007; Seger & Cincotta, 2002; Nomura et al., 2007). These tasks provide some evidence that generalization to perceptually similar stimuli can occur in parallel with basal ganglia activation. However, these studies did not examine whether activity in the corticostriatal loop was modulated solely on the basis of category membership. Many tasks that recruit the striatum do not require generalization. In probabilistic classification, subjects categorize repeated stimuli from a limited set (Poldrack et al., 1999, 2001; Seger & Cincotta, 2005). In arbitrary visuomotor association learning subjects learn to respond to particular stimuli with particular motor responses, with no generalization involved; animal lesion studies (Wise & Murray, 2000), single unit recording (Brasted & Wise, 2004), and human functional imaging studies (Toni et al., 2002) all report striatal involvement these tasks. Similar patterns of striatal recruitment also occur in studies designed to examine decision making, in which subjects have to choose the rewarded stimulus from a set of visual stimuli (e.g., slot machine images; Daw et al. 2006) or need to perform the rewarded response associated with invariant visual stimuli (Delgado et al., 2005); again, these tasks also don’t require generalization across stimuli. Theoretical and behavioral work by Ashby and colleagues using the information integration task, in which initial learning is believed to be subserved by the visual corticostriatal loop, indicates that this loop does not make any parametric assumptions about category structure (such as assuming a prototype or decision bound structure) during learning (Ashby & Waldron, 1999). They argue that the striatum links small regions of perceptual space to categories, and thus can represent almost any category structure as a conjunction of small, similarity based regions of perceptual space that surround the learned exemplars. In summary, activation of the basal ganglia appears to be similar whether or not generalization is required, but no studies have directly compared tasks that differ only in the requirement to generalize.

1.2.2 Generalization via corticostriatal convergence

One way in which the striatum may contribute to generalization is via the impressive convergence of cortical input in the projections from cortex to striatum to basal ganglia output nuclei to thalamus. There is roughly a 10:1 reduction from cortex to striatum, and 300:1 from striatum to the basal ganglia output nuclei (GPi/SNr; Bar-Gad et al., 2003). Therefore, the entire pattern of cortical activity cannot be represented in the striatum (Morris et al., 2003), and some cortical patterns must either not be represented in striatal output, or treated as equivalent. Many bits of information that are represented separately at the cortical level may be combined to affect behavior. Bar-Gad and colleagues (2003) model this convergence as a process of dimensionality reduction. Their model compresses the information present in cortex with the aim of extracting important common features and eliminating redundancy while losing a minimum of useful information. Dimensionality reduction will necessarily result in generalization as information that is irrelevant for categorization is eliminated. Bar-Gad et al. (2003) argue that an efficient representation of cortical activity is vital for the computation of an appropriate subsequent action.

Although information is compressed by the convergence of cortex on striatum and of striatum on GPi/SNr, there are constraints on the generalization potential of this convergence that are due to the anatomy of corticostriatal and striatopallidal projections. The axon of each cortical projection neuron extends longitudinally down the striatum, innervating a subset of striatal neurons within a long rod-shaped projection territory or module (Selemon & Goldman-Rakic 1985). Each axon makes relatively few synaptic contacts (generally less than 1000 synapses per projection axon), and thus the connectivity between cortical projection neurons and striatal projection neurons is sparse (Zheng & Wilson, 2002). Within the visual corticostriatal loop, Cheng et al. (1997) tracked multiple individual axonal projections from temporal lobe area TE (which codes complex visual features) to caudate and found that each striatal module was innervated by a large number of axons projecting from the same TE column. Therefore, each striatal module likely combines information from relatively limited extents of cortex, and generalization by each module is limited to the information it receives from each column (typically cells within a cortical column code for slightly different variants of the same complex feature). Adjacent striatal cells have little common input, resulting in no correlation between their activities (Zheng & Wilson, 2002). Thus, each striatal projection neuron is an independent integrator of the information it receives. These patterns implies that generalization in the striatum via cortical projection convergence will be limited to generalization across the original cortical input, which is typically limited to a small region of cortex.

Less is known about the degree of generalization possible via the convergence of striatal projection neurons from different striatal modules onto GPi/SNr. Supporting limited generalization is evidence that projections from striatum to GPi/SNr also have sparse connectivity (Bar-Gad et al., 2003). However, neurons in the GPi are often activated by multiple cortical regions (Yoshida et al., 1993), and striatal compartments receiving projections from separate cortical regions can send converging projections onto GPi (Flaherty & Graybiel, 1994), which indicates that some generalization across cortical regions may take place via convergence onto GPi/SNr.

1.2.3 Generalization via visual cortical representations

The visual corticostriatal loop may also contribute to generalization via learning that occurs within the visual cortical regions projecting to the striatum. The visual corticostriatal loop receives projections from all of visual cortex beyond primary visual cortex. Visual cortex is hierarchically organized. Intermediate regions such as lateral occipital (in humans) and IT (in monkey) code for features such as object shape. Higher order regions in humans specialize for particular visual forms, such as faces in the fusiform face area, and landmarks in the parahippocampal place area (See Grill-Spector & Malach, 2004, for review). These specialized areas imply that much of the visual system is in a broad sense categorical, representing common visual form categories that are learned across the life span. However, it is unclear how acquisition of novel categories in adulthood under conditions of relatively limited training involves visual cortex.

The degree to which the category membership of visual stimuli in novel visual categories is represented in higher visual cortex is controversial and an important current area of research. Jiang et al. (2007) argue that category membership is not represented in visual form processing areas, but rather in separate higher brain regions. Their two stage categorization learning model claims that visual cortex implements a necessary first stage in categorization, in which neurons develop sharper tuning and selectivity for trained stimuli. These stimulus specific representations provide input to higher brain regions that develop categorical representations in their proposed second stage of categorization.

FMRI studies in humans (Jiang et al., 2007) and electrophysiological studies in monkey have used stimuli developed with morphing software that can continuously vary stimuli along a single dimension or set of dimensions. These continuously varying stimuli are placed into categories on the basis of a decision bound falling at a particular value of the dimension(s). For example, two category prototypes are selected and stimuli are generated with different percentages of similarity to each prototype (e.g., 90% similar to A and 10% to B, or 30% to A and 70% to B). Stimuli are divided into two categories, with those that are 50–100% similar to A are in one category and remaining stimuli in the other category. Cells in human lateral occipital complex (Jiang et al., 2007) and monkey area IT (Freedman et al., 2003) are modulated primarily by visual similarity. These cells do not show sensitivity to category membership as defined as differential activity to within category and between category items that are otherwise equivalent in terms of similarity. For example, a pair of stimuli consisting of one 20% A and one 40% A will differ perceptually from each other to the same degree as pair of stimuli consisting of one 40% A and one 60% A, although the stimuli in the first pair belong to the same category and those in the second pair belong to different categories. Visual system activity reflects the perceptual similarity, not category membership. Category membership, however, does affect activity in higher cortical regions in the parietal and frontal lobes (Freedman & Assad, 2006; Freedman et al., 2003; Jiang et al., 2007)

Other studies have found enhanced selectivity for category diagnostic features relative to nondiagnostic features in inferotemporal cortex after categorization training in monkey (Sigala & Logothetis, 2002). Studies of the dot pattern prototype learning task indicate that activity in early visual areas is sensitive to whether stimuli belong to a single category, but in this situation degree of category membership and degree of perceptual similarity to the prototype are confounded (Reber et al., 2002). It is unclear whether these early visual regions differentially represent multiple categories, or represent category membership when dissociated from perceptual similarity. In summary, there is evidence that visual cortex can represent individual items, and can generalize to items that are perceptually similar to studied individual items, but it is unclear whether it can represent other aspects of category membership. However, research to date has largely been limited to early and intermediate visual regions, and it is unknown whether higher order regions such as the fusiform face area may show sensitivity to category membership.

2 Basal ganglia pathways enabling action selection

The second important aspect of categorization is using categorical information to enable appropriate behavior. One of the primary roles of the basal ganglia in motor control is in the selection of responses via opposing excitatory and inhibitory effects on the thalamus. When extended to the realm of categorization, the basal ganglia perform a similar role: selection of appropriate responses linked to category membership of stimuli. The important corticostriatal loops involved in categorical responding are the motor loop, and cross loop interactions between the visual loop and motor cortex.

2.1 Mechanisms of selection in the basal ganglia

The corticostriatal system modulates cognition and behavior by helping to select, or gate, a subset of activated representations or movements. This selection role of the basal ganglia is clearly seen within the domain of motor processing. Many potential goals and behaviors are represented in prefrontal cortex and premotor areas. The primary role of the basal ganglia is selecting which of these many possible behaviors to execute (Gurney et al., 2004; Humphries et al., 2006; Lo & Wang, 2006). Inhibitory projections from the GPi/SNr to the thalamus, as shown in Figure 1, exert a tonic inhibition that keeps all potential behaviors suppressed. When an appropriate behavior is identified, this tonic inhibition is reduced for the selected action, which then is executed. Analogously, the basal ganglia may perform similar selection or gating of representations in other cortical areas that are not associated with overt behavior, such as cognitive strategies mediated by the frontal lobes (Houk & Wise, 1996). The role of the basal ganglia in selection of movements is illustrated by some of the symptoms of the major basal ganglia disorders. In Parkinson’s disease tonic inhibition of movements is increased, and it is difficult for patients to initiate desired movements (Dauer & Przedborski, 2003). Conversely, in Huntington’s disease many unwanted movements (such as chorea) fail to be inhibited.

The basal ganglia perform action selection via three pathways (see Figure 2) connecting cortex to thalamus: the direct, indirect (Alexander & Crutcher, 1990), and hyperdirect (Nambu et al., 2000) pathways. The direct pathway contributes to action selection by reducing the inhibition from the GPi/SNr on the thalamus, as outlined above; it is sometimes termed the “Go” pathway because it results in the release of a movement. The indirect and hyperdirect pathways are involved in suppressing actions by increasing the inhibition from the GPi/SNr on the thalamus. The indirect pathway, like the direct pathway, begins with projections from cortex to the striatum; as a result, the two pathways generalize to a similar extent, and both exert specific effects on particular responses. The indirect pathway is sometimes called the “NoGo” path (Frank, 2005). The hyperdirect pathway passes through the subthalamic nucleus (STN) rather than the striatum. The projections from cortex to STN are more diffuse than the projections from cortex to striatum, and as a result the hyperdirect pathway effects on thalamus are less specific to particular stimuli and responses than are the direct and indirect pathway effects. The hyperdirect pathway is particularly important for stopping behaviors that have already begun execution (Aron & Poldrack, 2006) and preventing premature responding (Frank, 2006).

2.2 Behavioral selection within the domain of categorization

Within categorization, the basal ganglia can serve to help select appropriate behaviors. In laboratory categorization tasks, these may be finger movements used to push keys on a response box; in the real world they may be choosing behaviors such as grasping for objects falling in a desired category. Behavioral research using tasks that have been shown to recruit the basal ganglia finds that category learning is impaired when the motor selection aspects of the task are changed, for example, when subjects being to use new responses to indicate category membership (Ashby et al., 2003), or when there is no consistent category-response relationship (Maddox et al., 2004). The release of an action appropriate to a category is postulated to occur through the direct pathway through the basal ganglia (Ashby et al., in press). However, all three pathways, direct, indirect, and hyperdirect, may be used in selecting appropriate behaviors on the basis of category membership (Frank, 2005, 2006).

2.3 Corticostriatal loops in categorical selection

2. 3. 1 The visual corticostriatal loop

The first section of this paper described the role of the visual corticostriatal loop in receiving input from visual cortical regions. In addition, the visual corticostriatal loop sends output projections to premotor regions, in particular to the pre supplementary motor area (pre-SMA), also known as Brodmann’s area 8 (Passingham, 1993). These projections may enable the selection of category appropriate motor programs represented in these premotor regions. Ashby and colleagues’ SPEED (in press) and COVIS (1998) models of categorization learning propose that initial learning the connection between visual categorical information and motor response occurs through this output pathway from the visual loop to premotor regions.

2. 3. 2 The motor loop

The motor loop connects the motor and somatosensory cortexes of the frontal and parietal lobes with the putamen (Lawrence et al., 1998). These include areas across a hierarchy of motor control, from those that interface between executive functions and motor planning such as the pre-SMA, to those that perform motor planning such as premotor cortex and the supplementary motor area (SMA), to the primary motor and primary somatosensory cortexes. The motor loop may be broken down further into sub loops according to the motor planning hierarchy. Lehericy et al. (2006) identified a gradient within the putamen such that primary motor cortex projects to more inferior and posterior areas, SMA to relatively more anterior and superior areas, and the pre-SMA to even more anterior and superior regions bordering on striatal regions that participate in the executive loop (described in more detail in section 3, below).

The motor loop may contribute to categorization learning in two ways. First, the selection of appropriate motor responses may require not only the output of the visual loop onto early motor planning areas, as described above, but also additional recruitment of the motor loop to select the more specific motor programs represented in later motor planning areas such as premotor cortex and primary motor cortex. In support of this possibility, several studies have found that the putamen is active in parallel with the body and tail of the caudate during categorization learning (Cincotta & Seger, 2007; Seger & Cincotta, 2005). A second role is in supporting categorization expertise and automaticity. As described below, there is evidence that the visual corticostriatal loop is important in early learning, but that as subjects develop automaticity in performance, the putamen becomes more important. This role of the motor loop in categorization is consistent with theories of motor loop participation in motor sequence learning that propose that the motor loop helps in chunking or consolidating a complex motor program or sequence so that it is no longer executed as individual movements but instead is executed as a single program. As organisms chunk an action sequence, the activity pattern of striatal cells changes so that most activity occurs at the beginning of the sequence, which is consistent with the striatum selecting the program as a whole (Graybiel, 1998; Barnes et al., 2005).

2. 3. 3 Interaction between corticostriatal loops

There is evidence that during procedural learning in general, there is a shift from visual or executive corticostriatal loops to the motor loop across the time course of learning. This transfer is consistent with patterns of interaction between corticostriatal loops²., which tend to follow a pronounced gradient from the most ventral, anterior, and medial portions (typically nucleus accumbens / ventral striatum) out to the most superior, posterior, and lateral portions (the putamen; Haber et al., 2003, Voorn et al., 2004). Specifically, the visual loop is in the middle of the gradient, receiving feedforward connections from the motivational loop, and projecting in turn to the motor loop.

The shift to the motor loop is often seen in sequence learning, both in monkey (Miyake et al., 2002), and human (Poldrack et al., 2005). Nakahara et al. (2001) developed a model of the interactions between the executive loop and the motor loop during sequence learning, with the former involved in acquisition and the latter in skilled execution. There has been less examination of categorization across extended training, but there is some evidence that putamen activity increases across training while activation in the body and tail of the caudate decreases (Seger & Cincotta, 2005), and evidence that categorization acquired under dual task conditions (which may interfer with processing in the executive loop) is more likely to recruit the putamen than when acquired under single task conditions (Foerde et al., 2006).

3. Feedback processing in learning and the executive and motivational corticostriatal loops

For categories to continue to accurately reflect the world, category information must change as a result of new experiences. Categorization tasks that recruit the basal ganglia typically involve learning via trial and error, with feedback on each trial indicating whether performance is correct or incorrect (see Seger & Cincotta, 2005, for a review, and Shohamy et al., this volume). Feedback is important for learning in potentially two ways, both of which involve the basal ganglia. First, feedback results in dopamine signals that project to the striatum and affect synaptic plasticity at the corticostriatal synapse (Reynolds & Wickens, 2002). Second, feedback is a signal used by the executive and motivational corticostriatal loops to modulate activity in their associated cortical regions (Kimura & Graybiel, 1995).

3.1 Modulation of basal ganglia function by dopamine

One way the basal ganglia are modified to reflect experience is through the effects of dopaminergic projections from the midbrain that give feedback about the effects of ones actions. Dopamine neurons are sensitive to both intrinsic (food, drugs) and secondary (money, praise) rewards. Particularly important within the realm of categorization is that dopamine neurons are sensitive to task related feedback, which serves as the reinforcement learning signal in the many laboratory categorization tasks that involve learning via trial and error. Dopamine neuron firing patterns specifically reflect violations of predicted reward, termed prediction error (Schultz & Dickinson, 2000).³ In a situation in which the organism can choose from multiple options, the observed dopamine neuron activity is consistent with the prediction error for the item that the organism actually chooses (Morris et al., 2006), which implies that dopamine neuron activity reflects a decision that has already been made elsewhere in the brain. Morris and colleagues argue that the dopamine activity is important not for determining which object to choose on the current trial, but instead for tuning representations that will be used in future decisions.

Dopamine is involved in modulating the plasticity of corticostriatal synapses. Synapses between axons of cortical projection neurons and striatal spiny cell dendritic spines receive as a third input a dopaminergic projection from the midbrain. Long term depression and long term potentiation, neural mechanisms that underlie synapse strengthening and weakening, may only occur when dopamine is present (Reynolds & Wickens, 2002, but see also Fino et al., 2005). Ashby and colleagues (Ashby et al., 1998; Ashby et al., in press) propose that dopamine mediated plasticity in corticostriatal synapses underlies category learning within the visual corticostriatal loop; this theory is supported by research finding that disruption of dopaminergic input to the striatum impairs stimulus-response learning in rats (Faure et al., 2005). Behaviorally, manipulations that interrupt feedback processing have been shown to impair category learning (Maddox et al., 2003), and learning in categorization tasks that recruit the basal ganglia is usually worse in observational than feedback learning conditions (Ashby et al., 2002).

3.2 Feedback processing in the executive and motivational loops

In addition to strengthening and weakening of corticostriatal synapses, feedback itself may serve as information that helps determine the effect of basal ganglia modulation on cortical targets. Specifically, the receipt of feedback, in particular negative feedback, can be used as a signal by the executive loop to switch strategies, and/or update working memory representations. These functions are likely subserved by the executive and motivational corticostriatal loops. The head of the caudate participates in the executive corticostriatal loop, with primary connections with dorsolateral prefrontal cortex and posterior parietal cortex (Lawrence et al., 1998). The motivational loop connects the ventral striatum with ventromedial cortical areas. The ventral striatum consists of the nucleus accumbens and the most inferior and anterior portions of the caudate and putamen. In addition to ventral and medial frontal regions (including orbitofrontal cortex and the anterior cingulate), the motivational loop also receives projections from the hippocampus and amygdala (Lawrence et al., 1998).

The executive loop through the head of the caudate has been shown to be sensitive to feedback in many cognitive tasks, including gambling tasks (Delgado et al. 2000, 2004) and instrumental learning tasks (Haruno et al. 2004; O’Doherty et al. 2004) in addition to a variety of categorization tasks (Cincotta & Seger, 2007; Filoteo et al., 2005; Seger & Cincotta, 2005, 2006; Tricomi et al., 2006). Parkinson’s disease, which particularly affects the head of the caudate (Dauer & Przedborski, 2003), impairs learning via feedback but not learning via observation (Shohamy et al., 2004, Smith & McDowall, 2006). Sensitivity to feedback valence differs across tasks and appears to be context dependent: greater activity for negative than positive feedback was found in the Wisconsin Card Sorting task (Monchi et al. (2001), and in a probabilistic classification task (Aron et al., 2004), both of which were successfully learned by subjects. Greater activity for positive than negative feedback is found in gambling tasks (Delgado et al. 2000; Tricomi et al. 2004) and in categorization tasks for stimuli for which feedback is randomly determined (Seger & Cincotta, 2005), in which learning is impossible.

Feedback processing may interact with other executive corticostriatal functions such as set shifting and working memory updating. Set shifting involves disengaging from a particular representation (either of an external object such as a particular stimulus feature, or from an internal representation such as a rule) before engaging with a new representation. Set shifting is an important component of the Wisconsin Card Sorting task, which is impaired in many forms of basal ganglia damage. Cools et al. (2004) found activity near the border of the ventral striatum and head of the caudate associated with switching between objects in a rule application task. A more general function of the head of the caudate is to detect changes in the behavioral context, and gate activity in the prefrontal cortex to allow updating of working memory (Frank et al., 2001; Houk & Wise, 1995). Haber et al. (2006) found converging projections in the head of the caudate from reward related cortical areas (including orbitofrontal cortex and the anterior cingulate), which indicates that the executive loop may be important in integrating reward related processing with cognitive functioning.

The ventral striatum receives strong projections from midbrain dopaminergic (ventral tegmental area) reward processing areas. Activity in the ventral striatum is sensitive to prediction error and to reward uncertainty (Dreher et al., 2006; Preuschoff et al., 2006). Within categorization learning tasks, the ventral striatum is sensitive to uncertainty about correct category membership due to difficulty in perceiving the critical feature (Grinband et al., 2006).

The ventral striatum and head of the caudate are both sensitive to feedback, and previous research (Seger & Cincotta, 2005) found that they were similarly active during categorization learning. Do these loops perform the same function in categorization learning, or do they perform different functions that might be dissociated in future research? Research from other cognitive domains suggest that the ventral striatum and head of the caudate may be dissociable on the basis of what aspects of feedback and reward each reflects. Actor-critic models of reward processing include separate functions for the dorsal and ventral striatum. The dorsal striatum (head of the caudate) is the actor: it is involved with deciding what action to take. The ventral striatum is the critic: it indicates whether the expected reward is received or not (Joel et al., 2002). The general pattern of ventral and dorsal (typically head of the caudate) striatal activity follows the actor-critic theory. The ventral striatum is sensitive to prediction error for cues indicating delivery of a reward (O’Doherty et al., 2004). In category learning, ventral striatal activity is sensitive to the degree of prediction error for probabilistic negative feedback (Haruno & Kawato, 2006; Rodriguez et al., 2006). In contrast, the head of the caudate codes the likelihood that an action will lead to reward, linking reward with behavior (Haruno & Kawato, 2006; Knutson & Cooper, 2005). The head of the caudate is only active when subjects relate the reward to their behavior (Tricomi et al., 2004; O’Doherty et al., 2004). In addition to differing in sensitivity to action contingency, ventral and dorsal striatum also differ in their sensitivity to reward timing. The ventral striatum is most sensitive to immediate rewards. Sensitivity increases to delayed rewards along a gradient progressing dorsally and posteriorly into head of the caudate nucleus (Tanaka et al., 2004).

4. Independent roles of corticostriatal loops in categorization

Much of the research in our laboratory has as its goal to identify the roles that different corticostriatal loops play in different aspects of categorization tasks. The approaches taken broadly fall into two groups. One approach is to dissociate the loops via manipulation of task variables expected to differentially affect the loops. Another approach is to examine differences in recruitment of and interaction between cortical and striatal regions during categorization learning.

4. 1 Dissociation approach

4. 1. 1 Feedback related activity (executive and motivational loops) versus learning related activity (visual and motor loops)

The first aim was to dissociate activity associated with correctly learning to categorize from activity associated with feedback processing. Previous research, typically using block designs, was unable to distinguish between activation due to feedback processing and activation due to learning. Based on the anatomy of the corticostriatal loops, we proposed that the head of the caudate, as part of the executive loop, should be modulated by feedback, whereas the body and tail of the caudate, as part of the visual loop, should be involved in learning to successfully categorize stimuli. The first study (Seger & Cincotta, 2005) required subjects to learn to classify abstract line drawings (see Figure 3, left column) arbitrarily grouped into two categories, “rain” and “sun”. Some stimuli were deterministically associated with a particular outcome (100% rain or 100% sun), some probabilistically (90 or 80% rain and 10 or 20% sun, or vice versa), and some randomly (50% rain, 50% sun). Areas involved in categorization were determined by examining trials on which subjects correctly classified deterministic or probabilistic stimuli and received feedback indicating they were correct, in comparison with baseline trials. As predicted, areas of the body and tail of the caudate were more active in correct categorization. In addition, across subjects, there was a correlation between recruitment of the body and tail of the caudate and categorization accuracy. In addition to the body and tail of the caudate, activity in the putamen also was associated with correct categorization.

Figure 3.

Tasks and results from three studies of categorization. Top row: Sample stimuli from each task. Middle row: Areas of activity in the body and tail of the caudate during correct classification of stimuli. Bottom row: Areas affected by feedback processing.

Feedback related activation was examined by comparing activity associated with receiving positive and negative feedback when categorizing random stimuli. Since subjects could not learn a consistent category for the random stimuli, this comparison allowed us to examine activity associated with feedback processing in the absence of activity associated with category learning. The head of the caudate and the ventral striatum were more active for positive feedback than negative feedback.

The second study (Cincotta & Seger, 2007) used an information integration task developed by Ashby et al. (2002). Subjects categorized stimuli that vary along two dimensions, length and angle (see Figure 3, middle column); ideal performance on this task requires combining information across both dimensions. This task differs from the first study (Seger & Cincotta, 2005) in many respects, most notably in that it required subjects to generalize across stimuli and apply the generalization to new stimuli. It was also an implicit task, in that subjects typically did not have verbalizable knowledge of rules that could distinguish between categories. The main comparison was of categorization blocks with baseline task blocks: both the putamen and the body and tail of the caudate were more active during categorization than baseline. Feedback related processing was measured by comparing feedback learning, in which subjects learned via trial and error, with observational learning, in which subjects were told the category membership of each item. The presence of feedback modulated activity in the head of the caudate but not in the body and tail of the caudate or in the putamen.

In the third study (Seger & Cincotta, 2006) we examined learning using a hypothesis testing task (see Figure 3, right column). In this task, subjects viewed two stimuli on each trial and chose one of them. The stimuli differed along four dimensions: letter identity (e.g., F or T), color (e. g., yellow or blue), size (large or small) and font (solid or outline). Subjects had to learn via trial and error a simple, unidimensional rule that would allow them to choose the correct stimulus. This task involves hypothesis testing, and differed from the other two tasks considerably in frontal lobe involvement. However, the recruitment of the basal ganglia was similar across tasks. The body and tail of the caudate was associated with learning to choose the correct feature: this area was more active while subjects were figuring out the rule, and was more active in good learners than poor learners. The head of the caudate and ventral striatum were more active when subjects received positive feedback than when they received negative feedback.

4. 1. 2 Visually related activity versus motor related activity

The putamen and the body and tail of the caudate are both often active during categorization. In the study using the probabilistic classification task (Figure 3, left column; Seger & Cincotta, 2005) putamen activity was similar to that of the body and tail of the caudate: more active during correct categorization than baseline, with the difference from baseline increasing across the course of learning. However, putamen recruitment did not correlate with learning success across subjects. The second study (Figure 3, middle column; Cincotta & Seger, 2007) found that both the putamen and body/tail of the caudate were active in comparison with baseline when learning an information integration task.

How might the body and tail of the caudate and putamen each contribute to categorization? One possibility is that categorizing stimuli requires interaction between the visual and motor loops. The visual loop through the body and tail of the caudate receives input from visual cortex; its output is a disinhibition of the motor programs represented in premotor areas that are appropriate for the stimulus. The motor loop connects premotor cortex with the putamen; its input is the premotor area activation, and its output a disinhibition of primary motor cortex that allows motion to take place. Under this theory, learning should affect activity in both the body and tail of the caudate and putamen. However, the loops should be sensitive to different manipulations. It is possible that the body and tail of the caudate will be affected more by perceptual manipulations, whereas the putamen will be affected more by motor manipulations. Additionally, the balance of importance may shift across learning, with the body and tail more important during early learning of stimulus-response contingencies, and the putamen more important during later time periods in which performance has become automatic. The SPEED model of Ashby and colleagues (in press) proposes that initial categorization learning may rely on the visual corticostriatal loop, but that as expertise is achieved categorization may increasingly rely on direct corticocortical interactions between visual and motor cortexes, skipping the visual loop. However, the motor loop may still be required for selection of motor programs represented in motor cortex.

These possible interactions between the visual and motor loops are supported by a recent single unit recording study of associative learning in monkeys. Williams and Eskander (2006) found caudate activity early in the time course of learning associated with the steepest part of the learning curve, when most of the learning about perceptual motor associations was taking place. In contrast, the activity of the putamen followed performance: It was high whenever accuracy was high, even when learning had reached asymptote. In humans, Foerde et al. (2006) compared categorization learning in single and dual task conditions and found recruitment of the body of the caudate and putamen in both conditions. However, the body of the caudate was more active when applying a category rule learned under single task conditions than dual task conditions. In contrast, activity in the putamen correlated with learning across subjects, but only in the dual task condition. These results indicate that learning with full attention may have loaded on the visual loop through the caudate nucleus, whereas learning with divided attention loaded more on the motor loop through the putamen. A similar pattern of early recruitment during learning for the caudate nucleus and later recruitment during overlearned performance of the putamen has been found for sequence learning tasks using fMRI (Poldrack et al., 2005) and single unit recording in the monkey (Miyake et al., 2002). Research in instrumental condition finds that the rat dorsolateral striatum (which corresponds to the primate putamen) is important for implementing overlearned or habitual behavior whereas the dorsomedial striatum (corresponding to the primate caudate nucleus) is important for learning stimulus-outcome contingencies(Atallah et al., 2007; Yin & Knowlton, 2006).

4. 2 Corticostriatal interaction approach

Another approach to examining corticostriatal loops is to examine the patterns of cortical and striatal activation to see how these areas might interact. In categorization, researchers have examined frontal and striatal interaction particularly in the executive loop and the visual loop. This is an exciting area of research that is likely to expand in the near future with recent methodological advances in functional imaging that allow examination of interregional connectivity, and in electrophysiology that allow for simultaneous recording from multiple brain regions.

Within the executive loop, prefrontal regions interact with the head of the caudate. Computational models postulate that this network is particularly important for selecting or gating activity in the frontal cortex (Frank et al., 2001; Houk & Wise, 1995). This gating may result in updating of working memory, or selection of one of a number of strategies represented in the frontal lobe. Pasupathy and Miller (2005) examined single unit activity in the head of the caudate and prefrontal cortex while monkeys learned to reverse simple rules associating visual stimuli with actions. They found that the activity in the head of the caudate changed more rapidly and reached its asymptotic value faster than activity in the prefrontal cortex. Prefrontal cortex activity followed more closely the monkey’s actual behavior in the task. This pattern is consistent with the head of the caudate providing a signal to prefrontal cortex to change strategy.

We examined interaction between the head of the caudate and prefrontal cortex using fMRI during a hypothesis testing task (shown in Figure 3, right column; Seger & Cincotta, 2006). The time course of activity in prefrontal areas differed from that of the head of the caudate. Whereas activity in the head of the caudate peaked early and then dropped quickly, the prefrontal cortex peaked later, in parallel with the behavioral measures of learning. Again, this pattern is consistent with the caudate gating representations in frontal cortex.

Little research has so far examined corticostriatal interaction during category learning in the visual or motor loops. Brasted and Wise (2004) examined activity in the premotor cortex and putamen while monkeys learned arbitrary visuomotor associations. They found that the activity in both areas was similar, and in both the activity followed the learning curve, increasing as the animals became more proficient at performing the responses.

5. Conclusion

The basal ganglia participate in several aspects of categorization in interaction with cortical regions, including category representation, categorical motor responding, and category learning. The corticostriatal systems interact to support categorization in dynamic ways across the time course of each individual categorization trial, and across the time course of learning overall. In each trial in a typical categorization learning task, the subject sees a stimulus, makes a response, and receives feedback. Each aspect of this task may recruit corticostriatal loops. Viewing the stimulus activates visual cortical regions; these visual regions project to the body and tail of the caudate nucleus in the visual loop. The visual loop sends output to motor planning regions of the frontal lobe that helps select the appropriate categorization response, in conjunction with the motor corticostriatal loop. Based on the feedback received, synapse strength is modulated across the corticostriatal loops. Feedback also serves as a signal, processed by the head of the caudate and ventral striatum in the executive and motivational loops, which is used to update memory representations about what category the stimulus was in to be used in future trials.

Across the time course of learning, subjects progress from an initial state in which they must guess, to successful categorization ability, and finally to automatic performance. Early performance is associated with strong reliance on feedback, processed through the motivational and executive corticostriatal loops, as a general strategy for categorization is established and initial representations are set up. As subjects become skilled at categorizing, the visual corticostriatal loop is recruited during correct categorization. Finally, as automaticity is achieved, the visual loop may become less important, with direct cortical links between visual and motor representations helping activate the appropriate behaviors within the premotor regions, in conjunction with the motor corticostriatal loop through the putamen.