Human Category Learning 2.0

Abstract

During the 1990’s and early 2000’s, cognitive neuroscience investigations of human category learning focused on the primary goal of showing that humans have multiple category learning systems and on the secondary goals of identifying key qualitative properties of each system and of roughly mapping out the neural networks that mediate each system. Many researchers now accept the strength of the evidence supporting multiple systems, and as a result, during the past few years, work has begun on the second generation of research questions – that is, on questions that begin with the assumption that humans have multiple category learning systems. This article reviews much of this second generation of research. Topics covered include: 1) How do the various systems interact? 2) Are there different neural systems for categorization and category representation? 3) How does automaticity develop in each system?, and 4) Exactly how does each system learn?

Cognitive neuroscience investigations of human category learning began in the second half of the 1990’s.^1,2 During the ensuing decade, much of this work focused on the primary goal of showing that humans have multiple category learning systems and on the secondary goals of identifying key qualitative properties of each system and of roughly mapping out the neural networks that mediate each system. Although this work continues, many researchers now accept the strength of the evidence supporting multiple systems. As a result, during the past few years, work has begun on a second generation of research questions. By second generation, we mean questions that begin with the assumption that humans have multiple category learning systems. Included in this list are questions such as: 1) How do the various systems interact? 2) Do different neural networks mediate categorization and category representation? 3) How does automaticity develop in each system?, and 4) Exactly how does each system learn? Although a number of recent publications review work on the first generation of research questions,^3,4 no existing reviews focus on these second generation questions. This article reviews this more recent work.

Brief Review of First Generation Research

With the realization that humans might have different category-learning systems that are each adept at learning about different types of category structures came more careful attempts to characterize existing category-learning tasks. One popular nomenclature includes rule-based (RB), information-integration (II), prototype-distortion, and unstructured category-learning tasks.⁵ Briefly, RB tasks are those where the optimal strategy is easy to verbalize (e.g., long objects are in one category and short objects are in another) and the categories can be learned via a logical reasoning process (e.g., by hypothesis testing). Optimal performance in II tasks requires integrating perceptual information from different stimulus components at a pre-decisional level. In II tasks the optimal strategy is not easily verbalized. In prototype distortion tasks, the category exemplars are created by randomly distorting a category prototype. Finally, in unstructured (or ad hoc) tasks the category exemplars are defined arbitrarily, rather than according to similarity (as in prototype distortion) or on the basis of some abstract logical (as in RB) or mathematical (as in II) relationship. There are proposals that these four tasks primarily load on different memory systems,⁵ although this hypothesis is far from resolved.

Of these four tasks, the most widely studied is the RB task, followed in order by the II, the prototype distortion, and the unstructured tasks. As described below, much evidence suggests that learning in RB tasks depends on an explicit, hypothesis-testing system that is mediated by a broad neural network that includes the prefrontal cortex, anterior cingulate, head of the caudate nucleus, and the hippocampus and other medial temporal lobe structures. In contrast, the evidence suggests that II tasks recruit a procedural learning system that depends heavily on the striatum and on dopamine-mediated reinforcement learning at cortical-striatal synapses. Less is known about prototype distortion and unstructured tasks, but several studies suggest that the most popular prototype distortion task recruits the perceptual representation memory system.^6,7

The evidence for multiple systems came primarily from two types of studies – behavioral experiments that reported empirical dissociations between different category-learning tasks and cognitive neuroscience studies that used either neuroimaging or else studied various neuropsychological patient groups. The dissociation studies manipulated variables that affected the structure of the categories, the placement, timing, and richness of the feedback, the response characteristics associated with the task, the number of categories and decision bounds, and the working memory load, to name a few. For example, delaying feedback by a few seconds,^8,9 switching the location of the response keys,^10,11 or informing participants of the category label before the stimulus rather than after the response¹² all interfere with performance in II tasks much more than in RB tasks. In contrast, adding a secondary (dual) task^13,14 or reducing the time available to process the feedback¹⁵ interferes with performance in RB tasks much more than in II tasks. No single-system account of all these dissociations has been proposed.

Neuroimaging and neuropsychological patient studies helped map out the neural circuits active in these different tasks. Neuroimaging studies of all four tasks have been reported, and a variety of patient groups have been tested, including patients with Parkinson’s disease, Huntington’s disease, frontal-lobe damage, medial temporal lobe amnesia, and schizophrenia.¹⁶ Perhaps the most consistent result from all of this work is the finding that almost all forms of category learning depend, to some extent, on the striatum (although different tasks may depend on different striatal subregions).

One theoretical reason why the striatum is seen as a plausible site for initial category learning is that the conditions under which cortical-striatal synapses are strengthened and weakened closely match the conditions for reinforcement learning^17–19 with dopamine serving as the reinforcement training signal. In accordance with this observation, Parkinson’s patients, who have reduced brain dopamine levels, are impaired in a wide variety of category learning tasks.^2,20,21 In addition, II category learning, which is thought to rely primarily on reinforcement learning at cortical-striatal synapses,¹ is disrupted by feedback delays just as predicted by reinforcement learning.^8,9 For a thorough review of the evidence implicating the striatum in category learning see Ashby and Ennis²² or Seger²³.

Although many cognitive theories of categorization have been proposed,³ only a few theories have been developed in enough neurobiological detail to make predictions in cognitive neuroscience experiments. The COVIS theory of category learning assumes separate explicit and procedural-learning categorization systems that compete for access to response production.^1,24 The explicit system selects and tests simple verbalizable hypotheses about category membership. The procedural system gradually associates categorization responses with regions of perceptual space via reinforcement learning. COVIS assumes that RB categorization is mediated by a broad neural network that includes the prefrontal cortex, anterior cingulate, head of the caudate nucleus, and the hippocampus. The network that includes the prefrontal cortex, anterior cingulate, and head of the caudate nucleus is assumed to select, test, and switch among alternative hypotheses using working memory and executive attention,²⁵ whereas the primary role of the hippocampus in COVIS is to mediate the long-term retention of RB learning.²⁶ To perform well in RB tasks, participants must remember which rules they have already tested and rejected, in order to avoid revisiting these failed rules again. Thus, COVIS predicts normal learning by medial temporal lobe amnesiacs in simple RB tasks where the correct rule can be discovered before the list of rejected hypotheses is lost from working memory. In more difficult RB tasks, the search for the correct rule will exceed working memory capacity, so COVIS predicts that in these cases medial temporal lobe amnesiacs will be impaired. Much evidence supports the former prediction.^27,28 The latter prediction has not been as rigorously tested, although several studies have reported normal performance by amnesiacs on the first 50 trials of a difficult task, but impaired performance later on.^2,29

The key structures in the COVIS procedural-learning system are the putamen and the premotor cortex (i.e., the supplementary motor area). Early versions of COVIS assumed that the striatal regions most critical to the procedural system were the body and tail of the caudate nucleus.¹ More recent evidence however, suggests that the procedural system has a strong motor association,^10,11 which has caused the focus to switch to the putamen. The key site of learning in this model is at cortical-striatal synapses, and this synaptic plasticity is presumed to be facilitated by a dopamine-mediated reinforcement training signal from the substantia nigra pars compacta.

Interactions among the Systems

Once the multiple systems hypothesis is accepted, a next obvious question is how the various systems interact. The available evidence suggests competition. Less clear is whether this competition is at the learning or output stages.

Behavioral Studies

An ideal task for studying interactions between two systems is one in which each system is active on different trials and the experimenter is reasonably confident of which responses are controlled by each system. Two studies have attempted to achieve these goals.^30,31 Both studies used a hybrid category-learning task in which perfect accuracy required applying an RB strategy on some trials and an II strategy on the others. In the Ashby and Crossley³⁰ study sine-wave gratings with an orientation of greater than 50 degrees required an RB strategy, whereas gratings with a shallower orientation required an II strategy. In the Erickson experiments,³¹ three cues signaled participants which strategy to apply. First, the stimuli requiring an II strategy were perceptually distinct from the stimuli requiring an RB strategy. Second, stimuli requiring an II strategy were presented in one color, whereas stimuli requiring an RB strategy were presented in a different color. Third, the II categories required different responses from the RB categories (i.e., A and B versus C and D).

Participants in the Erickson³¹ study successfully switched between RB and II strategies on a trial-by-trial basis. In contrast, participants in the Ashby and Crossley³⁰ experiments did not switch. Instead, almost all of these participants used either a single RB or II strategy on all trials. By far, the most common strategy was RB.

If the two systems can operate independently then people should have performed well in both tasks. Taken together, these data suggest that switching between systems does not occur automatically. The prevalence of RB strategies in the Ashby and Crossley³⁰ experiments suggests that use of an RB strategy might limit access to the procedural system, either because the procedural system is prevented from learning or because it is denied access to motor output systems. The Erickson³¹ results however, suggest that this inhibition can be overcome if enough cues are provided to signal the participant which system should be used on each trial.

Cognitive Neuroscience Studies

A number of neuroimaging studies have reported negative correlations between medial temporal lobe and striatal activation that are consistent with a competitive relationship between declarative and procedural memory systems during category learning.^32–37 Similar results have been reported within the more general memory-systems literature.^38–40 In addition, a number of animal lesion studies have reported that medial temporal lobe lesions can improve performance in striatal-dependent habit-learning tasks, and conversely that striatal lesions can improve performance in medial temporal lobe-dependent spatial learning tasks.^41–43

All data reviewed so far suggest that the use of declarative memory either inhibits procedural learning, or denies access of the procedural-learning system to control of the response. Although this issue is far from resolved, some results support the latter of these two hypotheses – that is, that the inhibition is at the output stage, and that the use of one system does not necessarily inhibit learning in the other. First, Packard and McGaugh reported that animals displaying hippocampal-dependent (place learning) behavior immediately exhibited behavior that showed prior striatal-dependent (response) learning following inactivation of the hippocampus.⁴⁴ Similarly, animals displaying striatal-dependent behavior immediately exhibited hippocampal-dependent behavior following inactivation of the striatum. Second, using fMRI, Foerde, Knowlton, and Poldrack reported that the introduction of a secondary task shifted the brain region that correlated with learning from the hippocampus to the striatum, but that the overall level of striatal activation was equal in the two conditions.⁴⁵ Furthermore, Foerde, Poldrack, and Knowlton reported evidence that this striatal activation had a behavioral effect.⁴⁶ In particular, they showed that a dual task that impaired learning in declarative memory systems in probabilistic classification did not prevent implicit learning of the correct cue–response associations. These results are consistent with the hypothesis that the use of strategies that depend on declarative memory systems does not prevent simultaneous striatal-mediated procedural learning, but it does restrict access of that procedural learning to motor output systems.

Although these cognitive neuroscience results are consistent with the hypothesis that there is inhibition between declarative and procedural memory systems during category learning, it is important to note that they do not necessarily imply that this inhibition is between the COVIS explicit and procedural learning systems. For example, another possibility is that the declarative memory system tapped in these (human) studies may have been episodic memory-based explicit memorization, rather than working memory-based hypothesis-testing.

A Tentative Theory

What brain mechanisms could be mediating this type of output inhibition? Ashby and Crossley tentatively proposed that frontal cortex and the subthalamic nucleus might control system interactions via the hyperdirect pathway through the basal ganglia.³⁰ The hyperdirect pathway begins with direct excitatory projections from frontal cortex to the subthalamic nucleus.^47,48 Some evidence suggests that the key cortical input may come from the pre-supplementary motor area.⁴⁹ The subthalamic nucleus then sends excitatory projections directly to the internal segment of the globus pallidus (GPi).^50,51 This extra excitatory input to the GPi tends to offset inhibitory input from the striatum, making it more difficult for striatal activity to affect cortex. Hence, the hyperdirect pathway could permit (by reducing subthalamic activity), or prevent (by increasing subthalamic activity) signals coming from the striatum from influencing cortex.

Note that this hypothesis accounts for asymmetries in the Ashby and Crossley³⁰ data. Specifically, Ashby and Crossley found that the use of an explicit strategy prevented access to procedural knowledge, but they failed to find evidence of the opposite influence. The hyperdirect pathway provides a mechanism via which the prefrontal cortex can inhibit a response selected by the striatum, but it does not allow the striatum to inhibit a response selected by the prefrontal cortex. Furthermore, because frontal cortex controls the excitatory input to the hyperdirect pathway, this hypothesis could also account for the success of Erickson’s³¹ participants. The extra cues present in the Erickson study could be sufficient to inform participants when to turn this signal on and off. In addition, note that this model is consistent with the neuroscience data suggesting that the inhibition between systems is at the output stage. The hyperdirect pathway has no direct effect on processing within the striatum, which has frequently been identified as a key site of procedural learning. Thus, this hyperdirect pathway hypothesis predicts that when declarative memory systems control behavior, procedural learning and memory operates normally but is blocked from (cortical) motor output systems.

Evidence supporting this model comes from studies using the stop-signal task. On a typical stop-signal trial, participants initiate a motor response as quickly as possible when a cue is presented. On some trials, however, a second cue is presented soon after the first and in these cases participants are required to inhibit their response. A variety of evidence implicates the subthalamic nucleus in this task.^52–54 A popular model is that the second cue generates a “stop signal” in cortex that is rapidly transmitted to the GPi via the hyperdirect pathway, where it cancels out the “go signal” being sent through the striatum. When declarative memory is controlling behavior, Ashby and Crossley proposed that a similar stop signal may be used to inhibit a potentially competing response signal generated by the procedural memory system.³⁰

This hypothesis might also account for results from a recent study that examined category learning in older adults, and another recent study of category learning in frontal lesion patients.^55,56 Maddox et al. found an age-related deficit in both RB and II category learning.⁵⁵ When the analyses focused only on participants who used the task appropriate strategy, however, the age-related RB deficit disappeared whereas the II deficit remained. For this group of individuals, the II deficit was due to less consistent application of the task appropriate strategy by older adults, and over the course of learning these older adults shifted from an explicit hypothesis-testing strategy to the task-appropriate strategy later in learning. These data support a two-component model of II category learning that includes a striatal component that mediates procedural learning, and a frontal-cortical component (possibly via the hyperdirect pathway) that mediates the transition from hypothesis-testing to procedural-learning strategies.

Schnyer et al. also found RB and II deficits for frontal lesion patients.⁵⁶ A more careful lesion analysis pointed to damage in a fairly circumscribed region of ventral medial prefrontal cortex as common to the impaired group of patients, whereas those patients without ventral medial PFC damage mostly performed normally. These results provide further evidence that the ventral medial PFC is critically important for the ability to monitor and integrate feedback in order to select, maintain, and switch strategies in the interest of obtaining optimal performance. Taken together, these data implicate frontal brain regions in both RB and II category learning.

One intriguing possibility is that the primary role of the ventral medial PFC in II tasks may be to control the hyperdirect pathway. According to this account, damage to this PFC region would disrupt the normal transition from RB to II strategies. Schnyer et al. reported that ventromedial PFC patients were impaired in II tasks because they were more likely to use explicit rules. This result suggests that the default state of the hyperdirect projection from frontal cortex to the subthalamic nucleus may be “on” and that one role of the ventral medial PFC may be to switch this excitatory projection off. If so, then as Schnyer et al. reported, damage to the ventral medial PFC would make it more difficult for participants to switch away from explicit RB strategies. Obviously, much more work is needed to test this hypothesis.

Categorization versus Category Representation

One question where significant progress has been made in recent years is whether there are different neural networks for categorization and category representation. Interest in this question arose with reports of a variety of category-specific agnosias that can result following lesions to certain areas of visual cortex.^57,58 The most widely known of such deficits, which occur with human faces (i.e., prosopagnosia), are associated with lesions to the fusiform gyrus in inferotemporal cortex. Subsequently, it was discovered that long periods of practice with a category cause well-documented changes in how visual cortex responds to exemplars of that category.^59,60 For example, Gauthier, Tarr, Anderson, Skudlarski, and Gore reported that 7 hours of training with categories of novel 3-dimensional objects called greebles recruited a greeble-sensitive area in fusiform gyrus.⁶¹ Similar results were later reported for other novel categories and other regions of visual cortex.^62,63

These results all seem to suggest that much categorization might be mediated within visual cortex. Despite these results, however, there is more recent evidence that the primary role of visual cortex might not be to categorize, but rather to create the visual representation that other brain areas categorize. Categorization is a behavior that requires the subject to take some action, and therefore categorization requires linking a percept to an action. There is good reason to believe that this linkage is not encoded in visual cortex.⁶⁴ First, several studies have reported that following categorization training, cells in inferotemporal cortex showed enhanced sensitivity to diagnostic features compared with features that were irrelevant to the categorization judgment.^65,66 Such changes are consistent with the widely held view that category learning is often associated with changes in the allocation of perceptual attention.⁶⁷ Second and most critical are the studies showing that categorization training did not make inferotemporal cortex neurons more likely to respond to other stimuli in the same category, or less likely to respond to stimuli belonging to the contrasting category.^65,68–72 Third, Rolls, Judge, and Sanghera showed that the firing properties of cells in inferotemporal cortex did not change when the motor responses associated with category membership were switched (i.e., from “approach” to “avoid” and vice versa).⁷³ For these reasons, the best evidence seems to suggest that inferotemporal cortex does not mediate the learning of new categories. Even so, this visual association area is crucial to the categorization process because it appears to encode a high-level representation of the visual stimulus.

Although the standard view has long been that object recognition is mediated primarily by the ventral visual stream, more recent evidence suggests that the dorsal stream is also proficient at object recognition.^74,75 Inferotemporal cortex projects heavily to the body and tail of the caudate nucleus, whereas much of parietal cortex projects to the putamen. Thus, most of the striatum has access to high-level object representations.

Automaticity

A typical adult makes hundreds of categorization judgments every day. Almost all of these are automatic. When we sit in a chair or pick up a book we are making an automatic categorization judgment. Although adults sometimes make categorization decisions that are not automatic, categorization decisions based on newly acquired knowledge are probably far less common than categorization decisions that are made automatically. Despite this imbalance, initial category learning has been investigated much more extensively than categorization automaticity. There are both practical and theoretical reasons that the published literature has focused on initial learning.

One practical reason is that studies of automaticity require more time, patience, and resources than studies of initial learning (where meaningful data are available from the first trial). For example, to study the neural basis of automaticity Muhammad, Wallis, and Miller had monkeys practice the same RB classification task almost daily for more than a year.⁷⁶

A theoretical challenge when studying automaticity is to identify a point in training at which the categorization behavior has become automatic. This is a difficult problem because many different criteria for identifying automaticity have been proposed and none of these are widely accepted as definitive. Perhaps the most influential criteria were proposed by Schneider and Shiffrin.^77–79 For example, they proposed that a behavior should be considered automatic if it can be executed successfully while the participant is simultaneously engaged in some other secondary task. Another criterion they popularized is that a behavior should be considered automatic if it becomes behaviorally inflexible. For example, if switching the location of the response keys interferes with the expression of the behavior, then it should be considered automatic. These criteria are especially problematic if applied to categorization. For example, as mentioned above, several studies have reported that a dual task requiring working memory massively interferes with initial RB category learning but not with II category learning.^13,14,80 Similarly, several studies have reported that switching the locations of the response keys interferes with initial II performance but not with initial RB performance.^10,11,81,82 Therefore, by the Shiffrin and Schneider criteria II categorization is automatic after the first training session. Such a conclusion is incompatible with intuitive notions of automaticity, because accuracy in II tasks requires several thousand trials to asymptote. ⁸³

Behavioral Results

As humans gain practice in virtually any skill, they naturally become faster and more accurate. In many laboratory studies of automaticity, asymptotic accuracy is perfect, and as a result the primary focus is on response time (RT). The most widely replicated and best known empirical result in this area is that mean RT decreases as a power function of the amount of practice. Among many other examples, a power-function speedup has been reported for skills as diverse as cigar rolling and proving geometry theorems.^84,85 Not surprisingly, Nosofsky and Palmeri reported that RT improvements in an II task that used categories of color patches also followed the power law.⁸⁶

A more subtle question is whether there are multiple systems for automatic categorization in the same way that there are multiple systems for initial category learning. In the only known study to address this question Helie, Waldschmidt, and Ashby compared the performance of participants who each had more than 11,000 trials of training either with one of two RB tasks (one simple, one complex) or with an II task.⁸³ Qualitative differences were evident in the behavioral data up through the third of 20+ experimental sessions, but after session 4 or so, there were no more behavioral differences among any of the conditions. As mentioned above, during initial learning, a dual task interferes with RB learning but not II learning, whereas switching the response keys induces the opposite pattern of results. Helie et al. however, reported that after 20+ sessions of practice both of these differences disappear – that is, RB and II tasks both show no dual-task interference and they both show a response-key switch interference.⁷⁸ Thus, although much more work is needed on this question, at present there is no behavioral evidence for separate RB and II automatic categorization systems. Instead, the behavioral data suggest two possibilities. One is that after extensive training, RB and II categorization are both mediated by the same procedural system that mediates initial II category learning. A second possibility is that automatic performance in the two tasks is mediated by some new, but common neural network.

Cognitive Neuroscience Results

Interest in the neural basis of automaticity has a long history, dating back at least to Sherrington (1906), who argued that long periods of practice gradually make skills reflexive.⁸⁷ These ideas led to the theory that dominated the 20th century: Novel behaviors require attention and flexible thinking and therefore are dependent on cortex, whereas automatic behaviors require neither of these and so are not mediated primarily by cortex. Instead, it has long been assumed that automatic behaviors are primarily mediated by subcortical structures. For example, in his classic and influential article entitled “In search of the engram,” Lashley (1950) wrote that “it has been widely held that although memory traces are at first formed in the cerebral cortex, they are finally reduced or transferred by long practice to subcortical levels” (p. 466).⁸⁸

Striatum

As mentioned above, perhaps the most ubiquitous cognitive neuroscience finding about category learning is the important role that the striatum seems to play.^22,23 Thus, a natural question to ask is whether the striatum also participates in automatic categorization. Based on their different inputs and outputs, the striatum is often subdivided into associative and sensorimotor regions. Roughly speaking, the associative striatum, which includes all of the caudate nucleus and the anterior putamen, receives input from sensory association areas in the temporal lobes and from prefrontal cortex and sends projections primarily to prefrontal cortex (i.e., via the medial dorsal and ventral anterior thalamic nuclei). By contrast, the sensorimotor striatum, which includes all of the putamen except its most anterior portion, receives input from the parietal lobes and from motor and premotor cortex and sends projections primarily to premotor and motor cortex (i.e., via the ventral lateral thalamic nucleus).

More recent evidence suggests that associative and sensorimotor regions of the striatum may play different roles in learning and automaticity.⁸⁹ Several studies have reported that the associative striatum is active during initial skill learning and that its activity decreases with extended training.^90–93 In contrast, the sensorimotor striatum often shows the opposite pattern – that is, evidence suggests that the sensorimotor striatum might be needed for the performance of automatic behaviors and/or for the transition from initial learning to automaticity.^91,94,95 Largely because of these results, there have been several proposals that the development of automaticity involves a gradual transfer of control from the associative to the sensorimotor striatum,^96–98 although the exact details of how such a transfer is mediated have not been described.

It is also important to note that some results seem inconsistent with the hypothesis that the sensorimotor striatum mediates automatic responding. First, using fMRI, Helie, Roeder, and Ashby examined changes in RB categorization at four separate time points as participants practiced on the same category structures for more than 10,000 trials.⁹⁹ Striatal activation increased with practice, but the correlation between striatal activation and behavioral performance decreased to zero by the end of training. Thus, this study found no evidence that the striatum contributed to automatic RB categorization. Note that this result also suggests that automatic RB categorization is not mediated by the same procedural system that is critical for early II learning, which as mentioned earlier, is thought to depend heavily on the striatum.

Second, temporary inactivations of sensorimotor regions of the internal segment of the globus pallidus (via injections of the GABA agonist muscimol) should prevent the sensorimotor striatum from influencing cortex because this area is a relay between the striatum and thalamus. Thus, if the striatum mediates the expression of automatic behaviors, then such inactivations should disrupt highly practiced actions. In contrast to this prediction, however, Desmurget and Turner reported that such inactivations did not prevent monkeys from fluidly executing highly practiced motor sequences.¹⁰⁰ As a result, they concluded that circuits through the sensorimotor striatum do not contribute “to motor sequencing or the storage of overlearned serial skills” (p. 7685). Third, several studies have shown that disconnecting the bird homologue of the basal ganglia completely blocks new song learning, but has little effect on the expression of well-learned songs.¹⁰¹

The Role of Dopamine

As mentioned earlier, the evidence is good that dopamine plays an important role in early category learning. Even so, other evidence suggests it plays a diminishing role in the expression of automatic behaviors.¹⁰² For example, some human subjects with Parkinson’s disease are able to emit an automatic motor response when presented with a familiar visual cue (e.g., kicking a ball), despite difficulties in initiating novel voluntary movements.¹⁰³ As another example, blockade of dopamine D1 receptors in rats strongly disrupts a simple Pavlovian approach response to a sensory cue during early stages of training, but has little or no disruptive effect if extended training is given before the dopamine antagonist is administered.^104,105

Another important question is whether the role of dopamine is the same in the striatum and cortex. Both regions receive prominent dopamine projections from midbrain dopamine cells. As mentioned above, striatal dopamine is widely thought to serve as the training signal for reinforcement learning at cortical-striatal synapses. This raises the obvious question of the role that cortex might play in reinforcement-mediated category learning. A necessary feature of any reinforcement training signal is high temporal resolution. If the first response is correct then dopamine must be released into the relevant synapses quickly, before the critical traces disappear. But after the correct synapses have been strengthened, it is also essential that excess dopamine be quickly cleared from the synapse. If it is not, and the next response is an error, then the residual dopamine will strengthen inappropriate synapses – namely, those responsible for producing the incorrect response. This would undo the beneficial learning that occurred following correct responses, and thereby prevent skill learning.

Dopamine re-uptake is exceptionally fast within the striatum.¹⁰⁶ In contrast, in frontal cortex, because of low concentrations of the dopamine reuptake molecule DAT, it takes much longer to clear dopamine from synapses.^107–109 For example, the delivery of a single food pellet to a hungry rat elevates dopamine levels in prefrontal cortex for approximately 30 minutes.¹¹⁰ Ashby, Ennis, and Spiering argued that this poor temporal resolution effectively rules out dopamine as a trial-by-trial reinforcement training signal in cortex.¹¹¹ Instead, although dopamine may facilitate cortical LTP, there is much evidence that synaptic plasticity at cortical-cortical synapses follows classical two-factor Hebbian learning rules.¹¹²

Transfer to Cortex

Many sensory association areas of cortex project directly into premotor cortex. Ashby et al. proposed that these cortical networks, by themselves, are incapable of skill learning because of the absence of reinforcement learning at cortical-cortical synapses.¹¹¹ Instead, they proposed that via reinforcement learning, a subcortical path through the striatum learns to activate the correct post-synaptic target in premotor cortex, which allows the appropriate cortical-cortical synapses in the premotor cortex to then be strengthened via Hebbian learning (because the product of pre- and post-synaptic activations will be greatest at the correct synapse). In this way, control is gradually passed from the subcortical path through the basal ganglia to the faster cortical-cortical path. Thus, according to this model, the development of automaticity is a gradual process via which control is passed from subcortical pathways through the basal ganglia to purely cortical networks that connect sensory association areas of cortex with premotor cortex. In other words, rather than to serve as a long-term store of procedural knowledge, a primary function of the basal ganglia may be to train cortical-cortical representations that mediate automaticity. Note that this theory accounts for results showing that automatic behaviors are striatal- and dopamine-independent.

Learning Characteristics of Each System

As mentioned earlier, the overriding research goal in the first generation was to establish that multiple systems exist, rather than to learn as much as possible about any one system. More recently, some studies have focused more exclusively on a single system, with the goal of developing more detailed models of each system. This section reviews some of this work.

Separate Stages of Explicit and Procedural Category Learning

The category shift literature suggests that RB category learning includes two separate stages: a stimulus-to-label stage that associates stimuli and category labels, and a label-to-response stage that associates category labels and responses.^113–117 In a recent study, Maddox et al. reported evidence that II classification is also mediated by two learning stages.⁸¹ After training participants in an II task, Maddox et al. either changed the mappings between stimuli and category labels or between the category labels and the response locations. Both manipulations change the mappings between stimuli and responses, so if there is a single stage of learning then both manipulations should cause similar deficits. In contrast to this prediction, breaking the association between stimulus and category label caused more interference and led to greater recovery than breaking the association between category label and response location.

COVIS¹ postulates no learning after the cortical-striatal synapses, so it predicts no qualitative performance difference between the stimulus-to-label and label-to-response conditions. Both manipulations should require cortical-striatal relearning. For this reason, COVIS, in its current form, is not consistent with the Maddox et al.⁸¹ results. COVIS might be extended, however, to include a second stage of learning that associates a category label with a specific response location. Logically, such learning must be downstream from the site of category label learning, which suggests that plausible sites of response learning could be at synapses in the internal segment of the globus pallidus, the ventral anterior or ventral lateral nuclei of the thalamus, or within premotor cortex. Each of these brain regions has been implicated in procedural learning and thus represents plausible loci of such learning.⁹²

Switching in the Explicit System

RB category learning depends heavily on executive function. For example, one classic neuropsychological assessment of executive function is the Wisconsin Card Sorting Test, which requires participants to learn a series of RB categorization tasks.¹¹⁸ The recent literature on the cognitive neuroscience of executive function is immense and a review is well beyond the scope of this article. Interested readers should consult Banich¹¹⁹ or Jurado and Rosselli¹²⁰. This section touches on some recent findings that are relevant to rule switching in the explicit system.

Theoretical accounts of RB category learning postulate a number of separate processes. For example, COVIS assumes separate rule selection, maintenance, and switching operations.¹ Neuroimaging and neuropsychological results have provided evidence for multiple processes in RB category learning,^21,121–123 and recent work has begun focusing on these sub-processes in more detail. For example, Chiu and Yantis reported fMRI evidence that the mechanism that mediates rule switching is the same as the mechanism used in other types of attentional switching.¹²⁴

Neuropsychological and pharmacological studies have long implicated dopamine in rule switching,^125–128 but recent work also suggests a role for norepinephrine.^129,130 For example, pharmacological manipulations of prefrontal norepinephrine alter rule switching performance in rats.^131,132 Although the mechanism by which norepinephrine supports rule switching is unknown, one possibility is that a lack of positive reinforcement following the rule switch induces exploratory behaviors (e.g., switching away from the present rule) that depend on the ceruleo-cortical norepinephrine system.¹³³

A growing body of research suggests that motivational incentives can also influence executive control processes and thus rule switching in the explicit system. In a recent behavioral study, Maddox, Filoteo, Glass and Markman showed that the global and local motivational incentives in a modified Wisconsin Card Sorting Task systematically accentuated rule switching when they matched (i.e., when both the global and local incentives emphasized either maximizing gains or minimizing losses) and systematically attenuated rule switching when there was a mismatch (i.e., when the global incentive was to maximize gains and the local incentive was to minimize losses or vice versa).¹³⁴ In a recent fMRI study, Savine and Braver examined the effects of reward incentives on performance and neural activity in a cued task-switching paradigm.¹³⁵ They found smaller task-switching costs on incentive trials relative to no incentive trials. In addition, they found that activation in cognitive control networks tracked fluctuations in incentive value.

Positive and Negative Feedback

Feedback is known to play a different role in RB and II learning. A number of recent studies have explored these differences in detail.

A long history of research has shown that in RB category learning, providing feedback about errors allows faster learning than providing feedback about correct responses.^136,137 Two studies have investigated similar questions in II tasks. First, Ashby and O’Brien reported that II category learning requires both types of feedback.¹³⁸ Participants in this study who received feedback either only about (some) correct responses or only about (some) errors generally used explicit, RB strategies. Second, using a semi-supervised training paradigm in which participants received feedback about both correct responses and errors, but only on some trials, Vandist, De Schryver, and Rosseel reported that they found no evidence of any II learning on no-feedback trials.¹³⁹

Plasticity at cortical-striatal synapses is known to be bi-directional,^140,141 and thus current theory would predict that the relevant cortical-striatal synapses should be modified following feedback after both correct and incorrect responses, but should not change much on trials when no feedback is given (i.e., because striatal dopamine levels should not fluctuate much on no-feedback trials). This latter prediction is consistent with the Vandist et al.¹³⁹ results.

The COVIS procedural-learning system included only the direct pathway through the basal ganglia¹. In this pathway synapses are strengthened only if the following three conditions are present: (1) strong presynaptic activation, (2) strong postsynaptic activation (i.e., strong enough to activate NMDA receptors), and (3) dopamine levels above baseline (which occurs following feedback that indicates a correct response). Thus, on positive feedback (correct response) trials, COVIS assumes that synapses for which all three conditions are met will be strengthened, synapses for which the NMDA activation threshold is not met will be weakened, and synapses for which there is no activation will not change. On negative feedback (error response) trials, on the other hand, COVIS assumes that synapses either will be weakened or will remain unchanged. Thus, there is an asymmetry in the COVIS procedural-learning system which predicts that positive feedback should be more effective than negative feedback (because there is a greater dynamic range for increases in dopamine levels in the striatum than for decreases).

Another theoretical proposal, which more evenly balances the effects of positive and negative feedback, is that the direct pathway mediates GO learning, whereas the indirect pathway mediates NO-GO learning.¹⁴² According to this account, the direct pathway primarily uses positive feedback to learn which response to make when a stimulus is presented, while the indirect pathway primarily uses negative feedback to learn which response not to make. Although the indirect pathway has not been formally incorporated into COVIS, it does appear that an augmentation of this sort would allow COVIS to predict learning in positive-only or negative-only conditions.

COVIS assumes that learning should occur in the positive-only condition, and a version that incorporates the indirect pathway would predict learning in the positive- and negative-only conditions. Contrary to both versions of the model, Ashby and O’Brien found that participants in the positive-only and negative-only feedback conditions generally used explicit, RB strategies. One limitation of these studies, and an area for future research, is that the primary effect of changing the feedback in any way may be to change the system that people learn to rely on in the task. In fact, this may be more important than the change in the amount of learning that occurs within each system. For example, in the Ashby and O’Brien study,¹³⁸ participants using a one-dimensional rule could achieve an accuracy of 78%, which is only 8% below the accuracy rate of the optimal II strategy. This 8% difference was large enough to induce full-feedback control participants to adopt an II strategy, but it is difficult to rule out the possibility that removing all positive feedback or all negative feedback led participants to rely more heavily on explicit rules, effectively masking learning effects on the procedural system. Future work should examine category structures for which the difference in accuracy between the optimal II and most accurate one-dimensional rule is larger than 8%.

Future Directions

The era of work on second generation problems is just beginning and many important topics await systematic investigation. First, future work needs to address the nature of retention and generalization processes in each system. It will be important to understand how categorical knowledge is retained over long periods of time, how it is generalized, and how using that knowledge in one way might interfere with or enhance other uses of the same information. Second, desirable difficulties occur when one training protocol leads to poor learning but better long-term retention than other protocols.¹⁴³ This phenomenon has been observed in some motor and verbal learning paradigms and should be studied in categorization. Third, more developmental work is needed to determine how the different categorization systems develop and interact in childhood and in normal aging.⁵⁵ Fourth, more work is needed to understand the psychopathological implications of multiple systems. One exciting avenue for future research will be to apply a neuro-rehabilitation approach to category learning in which a normally functioning system is used to “bootstrap” learning or performance in a poorly functioning system. These are just a few of the many exciting avenues of research that will be pursued in the coming years.

Conclusions

Many categorization researchers now accept the large and growing body of behavioral, neuropsychological, and neuroimaging evidence that humans have multiple category learning systems that are functionally distinct at both the neural and cognitive levels. Thus, over the past few years the emphasis has begun to shift to the study of second generation problems – that is, questions that begin with the assumption that humans have multiple systems. This article reviewed much of the existing second generation research.