Exploring Explicit Learning Strategies: A Dissociative Framework for Research

Abstract

To explain learning, comparative researchers invoke an associative construct by which immediate reinforcement strengthens animal’s adaptive responses. In contrast, cognitive researchers freely acknowledge humans’ explicit-learning capability to test and confirm hypotheses even lacking direct reinforcement. We describe a new dissociative framework that may stretch animals’ learning toward the explicit pole of cognition. We discuss the neuroscience of reinforcement-based learning and suggest the possibility of disabling a dominant form of reinforcement-based discrimination learning. In that vacuum, researchers may have an opportunity to observe animals’ explicit learning strategies (i.e., hypotheses, rules, task self-construals). We review initial research using this framework showing explicit learning by humans and perhaps by monkeys. Finally, we consider why complementary explicit and reinforcement-based learning systems might promote evolutionary and ecological fitness. Illuminating the evolution of parallel learning systems may also tell part of the story of the emergence of humans’ extraordinary capacity for explicit-declarative cognition.

1. Introduction

Researchers of human cognition acknowledge multiple learning systems. Humans learn associatively—through the well-known mechanisms of conditioning and instrumental learning—but also declaratively using explicit cognitive processes that are supported by executive attention and working memory. For example, humans test hypotheses, discover rules, experience insights, even if denied direct reinforcement. Research on humans’ explicit-declarative cognition has informed cognitive science for decades (e.g., Ashby & Maddox, 2011; Atkinson & Shiffrin, 1968; Baddeley & Hitch, 1974; Cohen & Squire, 1980; Evans, 2003; Jacoby, 1991; Kahneman, 2011; Schacter, 1990; Smith & Church, 2018; Tulving, 1985).

Comparative psychology has explored less fully the possibility of animals’ explicit-declarative learning. This exploration has been slowed by the difficulty of implementing with animals the declarative paradigms that reveal explicit cognition in humans, and by the difficulty of presenting evidence that disconfirms the range of associative explanations. But we believe that studying animals’ explicit learning processes might inform comparative psychology as it has cognitive psychology. Therefore, we describe here a new dissociative framework and new experimental paradigms that may push animals’ learning processes toward the explicit-declarative pole of cognitive functioning and demonstrate their capacities in that domain.

Because we know that a dissociative framework may strike sparks, we first acknowledge that associative learning plays a central role in animals’ minds and lives, so that associative learning will always be a crucial aspect of comparative theory. Nothing here seeks to replace the associative perspective. Rather, we consider adding a complementary empirical dimension, through principled paradigms that may dissociate explicit and associative learning.

These paradigms target and disable the reinforcement-learning system known to be dominant in many tasks of categorization and discrimination learning. Then the animal’s learning is freer from the shaping forces of reinforcement-based associative learning. These paradigms still provide subjects instructive feedback, but feedback is now packaged so that it cannot support important kinds of associative learning. Facing this situation, the animal may need to adopt instead some other kind of learning process (if it can!).

2. Dissociable Systems of Learning

2.1. Implicit-procedural learning.

Neuroscientists commonly assume that humans have dissociable learning systems. Implicit-procedural learning is thought to be supported by the primary reinforcement-learning system. This system is proposed to underlie skill and habit learning (Mishkin et al., 1984) and performance in instrumental-conditioning, perceptual-categorization, and some discrimination tasks (Ashby & Ennis, 2006; Barnes et al., 2005; Knowlton et al., 1996; O’Doherty et al., 2004; Seger & Cincotta, 2002; Yin et al., 2005). This system instantiates well (but does not exhaust) the associative-learning construct (Smith & Church, 2018). Implicit-procedural learning occurs associatively through processes related to conditioning. Learning is gradual, as immediate reinforcement helps appropriate responses selectively occur with higher probability. In this kind of learning, behaviors are entrained, not explicit awareness or conscious knowledge. Given such learning, participants—even humans—may behave appropriately without easily verbalizing their strategy (Ashby et al., 1998).

This implicit system is probably linked to the basal ganglia (e.g., McDonald & White, 1994). It seems natural that systems for categorization and discrimination—ancient vertebrate adaptations—might lie in evolutionarily older brain regions. In primates, extrastriate visual cortex projects to the tail of the caudate nucleus—with a convergence of visual cells onto caudate cells that then project to premotor cortex (Alexander et al., 1986). By connecting perceptual and behavioral circuits, the caudate nucleus is well placed to associate perception to action (Rolls, 1994; Wickens, 1993). The role of reinforcement is to strengthen the connection between particular perceptual representations and their appropriate behavioral responses (e.g., their correct discrimination response). Lesions of the tail of the caudate nucleus apparently disable this implicit system neuropsychologically, impairing the learning of discriminations that require different responses to different stimuli (McDonald & White, 1994). Our dissociative paradigms intend to disable this implicit system experimentally, by controlling the packaging and delivery of reinforcement.

2.2. Explicit-declarative learning.

With this implicit learning system disabled, humans and animals may turn instead to explicit-declarative learning. Explicit-declarative learning uses executive attention (Posner & Petersen, 1990) and working memory (Goldman-Rakic, 1987), cognitive utilities that support hypothesis testing and rule formation (Robinson et al., 1980). This system learns by testing hypotheses (e.g., “Maybe the Category A things are the RED ones”). It learns rules that humans can describe verbally (Ashby et al., 1998). In essence, participants self-construe the task and develop their own rule to guide performance.

Explicit learning is probably linked to the prefrontal cortex, the anterior cingulate gyrus, the head of the caudate nucleus, and the hippocampus. Explicit rules may reverberate in memory loops involving prefrontal cortex (Alexander et al., 1996) so that they are kept relevant and salient during information processing. The anterior cingulate may select hypotheses to place in working memory, so that the participant pays attention to them and tests them actively. The head of the caudate nucleus may support hypothesis switching. Patients with frontal dysfunction are impaired in rule tasks (Robinson et al., 1980). Elliott and Dolan (1998) implicated the anterior cingulate gyrus in rule generation.

These hypothesis-testing and rule-formation processes are our working definition of explicit learning here. Human cognition is certainly sometimes conscious and declarative in these ways. However, as comparative psychologists, we naturally wonder whether animals’ learning performances can be stretched toward this explicit pole of cognition. We understand that the conscious character, the verbal grounding, and the phenomenal feel of these explicit processes could be different between humans and animals. In fact, new paradigms might give researchers the means to discover important differences in explicit cognition across species. Our dissociative paradigms intend to facilitate and release these explicit-declarative learning processes—because associative processes have been disabled.

3. Reinforcement in Implicit-Procedural Learning

The reinforcement dynamic in implicit-procedural learning is the key to understanding how one might disable this system. Rewards cause dopamine release into the tail of the caudate (Hollerman & Schultz, 1998; Wickens, 1993). The dopamine signal strengthens recently active synapses that were plausibly involved in reward (Arbuthnott et al., 2000). In conditions allowing successful learning, the appropriate behavior will become more strongly associated to its occasioning perceptual representation.

However, this mechanism is limited because the neural system soon returns to baseline with no trace of contributing synapses and no way to strengthen them. At that point, the strengthening that is procedural learning can no longer occur. Yagishita et al. (2014) illustrated this constraint by exerting temporal control over sensori-motor inputs and dopaminergic inputs. Thus, they could vary the temporal offset between the two. Dopamine did not strengthen synapses given offsets beyond 2.0 s. This temporal constraint is also seen experimentally (e.g., Maddox, et al., 2003; Maddox & Ing, 2005). Learning theorists have noted this time constraint attending reinforcement learning for almost a century (e.g., Pavlov, 1927).

Note that the implicit-procedural learning system cannot access working memory, verbal rules, or anything else in assigning neural credit for rewards. In caudate-mediated discrimination learning, the idea of stimulus-response (SR) bonds is literal, because the caudate links cortical stimulus representations (the caudate’s direct inputs) to adaptive responses (its indirect outputs). Thus, this kind of implicit learning depends on a time-critical sequence: stimulus-response-reward. It should therefore be disrupted by disrupting the sequence. If reinforcement is displaced enough that the neural system returns to baseline, dopamine release will not strengthen appropriate synapses and procedural learning will fail.

However, the explicit-declarative system should still function. These explicit processes can use working memory to maintain recent stimuli and hypotheses, pending feedback (Ashby & Maddox, 2011; Maddox & Ashby, 2004). For example, feedback displaced until the end of a block of trials would not derail that system, because working memory could bridge the displacement: (“In that block the rule Red-Category A—Blue-Category B worked: keep that rule!”). Of course, an animal’s explicit cognition would not be verbalizable, but rules could be represented nonverbally. One could imagine an assemblage of Category A (red) things (highlighting their common feature), or represent Category A and Category B things as in different spatial locations (a category method of loci), or label categories synaesthetically (e.g., big items heavy and sinking, small items light and rising). Indeed, an exciting aspect of our dissociative framework is that it might let researchers study the psychological nature of rules and hypotheses with the abstraction of language stripped away. Wordless rules may have been the original state of humans’ explicit cognition, and so these rules might be part of the story of humans’ cognitive emergence, too.

4. Two Dissociative Paradigms

4.1. One-back reinforcement.

In Smith et al. (2018), humans learned two categories in a task drawn from neuroscience (Ashby et al., 1998). Figure 1 shows a two-dimensional perceptual space—the X and Y axes are separable perceptual continua (e.g., size, pixel density). Each dot represents a two-dimensional stimulus (e.g., a size-density combination) that can be presented and rewarded as Category A (gray) or B (black). Converging evidence—from patient data, neuroimaging, cognitive research—confirms that this diagonal task depends on the implicit-procedural learning described earlier (Maddox & Ashby, 2004; Waldron & Ashby, 2001).

Figure 1.

Note. An information-integration categorization task with two dimensions cooperatively relevant to categorization. Tasks are depicted within an abstract 101 × 101 stimulus space. From “One-back reinforcement dissociates implicit and explicit category learning” by J. D. Smith, S. Jamani, J. Boomer, and B. A. Church, 2018, Memory & Cognition, 46, p. 262. Copyright 2018 by Springer. Reprinted with permission.

In the 0-back reinforcement condition in Smith et al. (2018), trials unfolded traditionally (stimulus, response, immediate feedback). In the 1-back condition, participants received feedback as follows: After seeing the N+1 stimulus and making the N+1 response, they received feedback about Trial N. Thus, reinforcement was displaced away from the SR pairings.

Figure 2 (A) shows the 0-back result. Using Bayesian strategy modeling (details in Smith et al., 2018), each participant’s performance was summarized by the decision bound that best partitioned their Category A and B responses. These bounds are drawn through Figure 1‘s two-dimensional space. The many diagonal bounds (running from bottom-left to top-right) separate well Figure 1‘s two categories. Thus, many participants learned to discriminate the categories appropriately. They learned the task’s actual (diagonal) category structure and responded to the task’s underlying reinforcement contingencies.

Figure 2.

Note. The decision bounds that provided the best fits to the last 100 responses of participants in the 0-Back (A.) and 1-Back (B.) reinforcement conditions. From “One-back reinforcement dissociates implicit and explicit category learning” by J. D. Smith, S. Jamani, J. Boomer, and B. A. Church, 2018, Memory & Cognition, 46, p. 268. Copyright 2018 by Springer. Reprinted with permission.

In contrast, Figure 2 (B) shows that only 1 of 30 1-back participants appreciated the task’s true category structure. These participants performed poorly (averaging 65% correct). They mainly showed vertical/horizontal bounds. These bounds confirm that participants imposed on the task an incorrect rule. Their decisions were not shaped by the task’s reinforcement contingencies. That mode of learning had been disabled.

4.2. Blocked reinforcement.

In Smith et al. (2014), humans completed trial blocks without feedback. At block’s end, they received the reinforcements from all correct trials clustered together—that is, these positive reinforcements were taken out of trial-by-trial order and presented consecutively. Then they received the penalty timeouts from all errors clustered in the same way. Reinforcement was displaced away from the SR pairings in time and scrambled out of trial-by-trial order. SR learning was doubly disrupted.

Participants completed Figure 1‘s category task. Each participant’s performance was summarized by a decision bound as already described (Figure 3).

Figure 3.

Note. The decision bounds that provided the best fits to the last 100 responses of participants in the Immediate (A.) and Deferred (B.) reinforcement conditions in Smith et al. (2014). From “Deferred Feedback Sharply Dissociates Implicit and Explicit Category Learning,” by J. D. Smith, J. Boomer, A. C. Zakrzewski, J. Roeder, B. A. Church, & F. G. Ashby, 2014, Psychological Science, 25, 453. Copyright 2013 by Sage Publishing. Reprinted with permission.

Under immediate reinforcement, many participants responded according to the appropriate major-diagonal decision bound. Under blocked reinforcement, none did. Implicit-procedural learning was disabled, because the SR-binding force of reinforcers was undercut by reinforcement’s displacement. Under displaced reinforcement, participants adopted the explicit strategy of applying an incorrect rule (producing vertical/horizontal decision bounds). That is, despite the poor effect in producing correct responding, some participants assumed a small size-big size category-differentiation rule (the vertical decision bounds in Fig. 3B). Others assumed a sparse pixels-dense pixels rule (the horizontal decision bounds in Fig. 3B). Our modeling techniques let us clearly discern the systematic use of such adventitious strategies. Through decades of research, the misapplication of dimensional rules has proved diagnostic of explicit cognition (Ashby & Maddox, 2011; Smith & Church, 2018). Verbalizations (Ashby et al., 1998) and the shape of learning curves (Smith et al., 2014; Smith & Ell, 2015) help confirm this diagnosis. When the formation of SR associative connections is defeated, humans’ explicit mind steps in to compensate. This is an elemental statement of a cognitive-science principle that is testable regarding animal minds, too.

One may interpret these dissociations narrowly as showing the disabling of a striatal, SR mapping function powered by immediate reinforcement. Even this interpretation is important. If one can switch off different learning systems in humans and animals with experimental control, possibilities for theoretical illumination follow immediately.

However, our methods may disable associative learning more broadly. If participants under displaced reinforcement could map the task associatively in any way at all—mapping stimuli to outcomes, responses to outcomes, stimuli to stimuli, and so forth—they would learn the task’s true reinforcement contingencies and show diagonal decision bounds. They do not. Instead, they exhibit every sign of learning explicitly (using rules). Thus, the dissociation methodology described here apparently has the potential to disable associative-learning processes quite broadly, forceably bringing the explicit-declarative cognitive mode into sharper relief.

5. A Dissociative Framework for Comparative Psychology

In turn, these methods could give comparative psychology powerful tools to disable animals’ associative-learning processes, letting researchers ask whether animals can step in with their own explicit learning processes. Indeed, the most important application of these techniques could be in studying animals.

This is why. Humans are almost too obviously explicit. They introspect. They tell you so. It seems the explicit mode must be their default. If you give humans a rule task, and they learn it, you will easily accept they learned the rule explicitly.

However, assigning a default mode can be misguided. Explicit cognition is likely not the default for young children (e.g., Parkin & Streete, 1988), before the prefrontal system fully matures and the language system links to it. Explicit cognition is likely not the default of aging adults whose frontal systems are failing (e.g. Midford & Kirsner, 2005). It may not be the default of animals, with their smaller frontal cortices and reduced prefrontal development. Instead, we liken the interplay of learning systems in cognition to a mixer board. The implicit and explicit channels can be turned up or down. The two channels will vary in their cognitive salience, in their ease of cognitive access, in their cognitive dominance. For humans, no doubt, the explicit treble is turned up loud.

But perhaps not for monkeys, who likely have less sophisticated systems for executive attention and working memory. Their associative bass may be turned up loud. We believe that many associative-learning theorists would agree. And if so, then the controlling and dominant associative system will become more difficult to move out of the way. This has two implications for research. First, if monkeys learn a “rule” task, no strong conclusion is warranted, because the dominant, controlling associative system may have found some cue to successful performance. Second, therefore, the present dissociative methodology becomes most interpretatively crucial. For it offers the means to unplug the associative-learning system, leaving the organism with no means to task solution except the explicit. Then, if successful learning occurs, one is in a stronger interpretative position. The last question raised by this article is whether these dissociative techniques can be applied to test nonverbal animals.

6. A Preliminary Study of Explicit Learning Processes in Monkeys

Exploring this possibility, Smith et al. (2020) tested macaques (Macaca mulatta) in a series of two-choice discriminations (Figure 4). Each task required a choice between clearly different perceptual stimuli (e.g., Category A: yellow-framed blue circle; Category B: unframed blue circle). Reinforcement was lagged by one trial as described above. The lag should disrupt the SR learning system of monkeys as it does for humans. Then, if monkeys have no alternative learning system, discrimination learning would fail.

Figure 4.

Note. Illustrations of several of the two-choice discrimination tasks used in Smith et al., 2020. From “Monkeys (Macaca mulatta) Learn Two-Choice Discriminations under Displaced Reinforcement” by J. D. Smith, B. N. Jackson, & B. A. Church, in press, Journal of Comparative Psychology. Copyright 2020 by APA, reprinted with permission.

Several monkeys reached criterion on multiple tasks. They sometimes learned very rapidly starting their criterion run on Trial 1 of a new task. They learned to high levels, even when feedback brought only a re-view of the one-back stimulus, together with their reinforcement whoop and food pellet (with no previous-response information provided at all). In humans, these phenomena support an attribution to explicit cognition. Thus, the results suggest that monkeys may have alternative learning algorithms usable when reinforcement is displaced. Based on contemporary neuroscience, a strong contender for this alternative algorithm would be task rules held in working memory. A video of an animal performing this type of task is available at this link.

We offer this study as an illustration of a method that could have broad utility within comparative psychology by possibly fostering animals’ transitions to explicit cognition. The underlying tasks are elemental—any species testable in a discrimination task can be studied. Indeed, implicit/explicit category tasks have been used in studies with pigeons, rats, and nonhuman primates that reveal important phylogenetic contrasts (Broschard et al., 2019; Qadri et al., 2019; Smith et al., 2012). The additional question raised here is about introducing displaced reinforcement, allowing researchers to ask whether other species have emerging forms of explicit cognition available when associable reinforcement is eliminated.

7. Adaptive Complementarity between Implicit and Explicit Learning Systems

Thinking about our dissociative framework from an ecological perspective, one sees why organisms might have parallel implicit and explicit learning systems. The implicit system has strengths: it produces stable behavior, it maximizes the probability of reinforcement, it commits to behavioral solutions slowly and conservatively, it relinquishes those solutions grudgingly (during extinction), and it does not depend on conscious awareness (granting it phylogenetic breadth). But it also has constraints. It depends on immediate reinforcement, time-critical event sequences, and persistent event repetition. Learning cannot occur off-line or with displacement in time or space from the task’s trials. Old approaches cannot be replaced instantly at need. Explicit learning is a perfect complement to implicit learning. It is not time rigid. It does not need immediate reinforcement or event repetition. Learning can occur off-line and with displacement. Learning and unlearning can occur instantly at need. Thus, complementary explicit and reinforcement-based learning systems may confer fitness.

8. Conclusion

Comparative researchers have long faced a difficult empirical/theoretical problem. Immediate reinforcement is motivating—it powers behavior and promotes learning. Animals probably prefer it, but it may be limiting and circular because immediate reinforcement fosters animals’ preferred associative learning and enables theorists’ preferred associative interpretations. How can we ever know from immediate-reinforcement paradigms whether animals can transcend associative learning?

Our dissociative framework can break this circle. Here we showed that a close analysis of discrimination learning and its reinforcement dynamic suggests new empirical approaches. We showed that one can essentially shut down an associative-learning system that serves animals and humans in two-choice discrimination tasks. By doing so, one removes an important kind of associative learning from the empirical situation and from the theoretical discussion. In fact, humans’ insensitivity to our tasks’ underlying reinforcement contingencies suggested that we had disabled their associative-learning processes broadly.

If humans and monkeys still learn under displaced reinforcement, as they do, then one can suggest that there exists in both species other learning processes that survive displaced reinforcement. Cognitive neuroscience suggests that this alternative system could be a form of explicit cognition.

The general principle expressed by our dissociative framework—that displaced reinforcement suppresses associative learning producing a transition to explicit learning—has applications to other domains of animal learning. For example, consider research on eye-blink conditioning (review in Clark et al., 2002). Humans can learn delayed-reinforcement Pavlovian procedures (i.e., trace procedures). But, to do so, they engage declarative memory, hippocampal brain areas, frontal brain structures, and even consciousness. In short, here, too, delayed reinforcement disrupts the primary associative level so that learning transitions to the explicit-declarative level as demonstrated in category learning and discussed in this article.

Because we understand that a multiple-systems view produces theoretical tension, we close by explaining why our view might be productive within comparative psychology. Over a century, the associative construct expanded to become more comprehensive. This stretched construct sometimes even seems to encompass explicit hippocampal episodes, pre-frontal rules, and conscious metacognitive judgments. However, this stretch has consequences. To unite the blink reflex and a metacognitive judgment together as “associative” risks conflating crucial distinctions about mind that cognitive scientists, comparative psychologists and behavioral neuroscientists would like to pursue. Thus, we believe that a limited, disciplined construct of associative learning will remain strong and lasting in comparative psychology. But, beyond that disciplined boundary, researchers should be encouraged to study explicit learning and memory as well. Then, comparative psychology can draw on the best theoretical perspectives from the whole range of cognitive science.