The Cognitive Architecture of Uncertainty

Abstract

The authors consider theory in the animal-metacognition literature. Theoretical interpretation was long dominated by associative descriptions, as illustrated in the 2009 special issue. We suggest that this approach risks a self-limiting understanding of animal mind, and an imprecise understanding of the cognitive requirements inherent in metacognition tasks. In fact, some tasks self-entail the need for higher-level decision-making processes, processes that—in humans—we would call explicit, declarative, and conscious. These points are illustrated using the inaugural study on dolphin metacognition. We urge researchers to turn more toward illuminating the cognitive architecture of capacities like metacognition, including illuminating the depth, and structure, the learning/memory systems, the cognitive levels, and the declarative awareness possibly present in animals’ minds. The empirical development of this literature demonstrates that researchers are now prepared to do so. This study can produce strong synergies across the allied fields of biopsychology, comparative and cognitive psychology, and neuroscience.

1. Introduction

Humans’ cognitive self-awareness or metacognition is crucial to every aspect of intellectual adaptation. It is a fundamental cognitive capacity that deserves full scientific understanding, including understanding its evolutionary origin. This understanding might reveal the roots of reflective mind in humans, or the emergence of reflective mind during cognitive development. It could provide animal models for studying cognitive awareness and its neuroscientific organization. It might even give researchers leverage on the problem of animal consciousness. In this study also lies the potential for cross talk among comparative psychologists, cognitive psychologists, and neuroscientists—cross talk still far too infrequent given the powerful synergies possible. For these reasons, in 2019 as in 2009 for the CCBR special issue, metacognition is still a worthy focus of cross-species research.

2. The Dolphin’s Tale

Smith et al.’s (1995) first application of the uncertainty paradigm—25 years ago— tested a bottle-nosed dolphin (Tursiops truncatus) in a psychophysical threshold task. Natua (Figure 1) discriminated High tones (half the trials, always 2,100 Hz, deserving the “High” response) from Low tones (half the trials, 1,200–2,099 Hz, deserving the “Low” Response). The dolphin could also make a third response that has become known as the uncertainty response (UR). That is, he could choose to decline to answer any trials he chose. In Smith et al.’s original paradigm, the UR received no primary reinforcer. Instead, it advanced the task into a following trial that was chosen to be easy for the dolphin.

Figure 1.

Natua.

Figure 2A shows that the dolphin responded “Low” and “High,” respectively, for many Low and High tones. His extraordinary auditory performance produced a Low-High psychophysical threshold at about 2,080 Hz. But the crucial result was that Natua judged correctly when he was in perceptual difficulty in the discrimination and likely to err. On those threshold trials, he made URs selectively. The fact that the UR was used most at the dolphin’s true psychophysical limit must shape one’s theoretical interpretation of that response. We will come to the matter of that theoretical interpretation.

Figure 2.

(a) Natua’s performance in a threshold-uncertainty paradigm. He touched a High response when a 2,100 Hz tone sounded. He touched a Low response when any lower tone sounded. The uncertainty response let him decline the trial and move instead into an easy next trial. The pitch of the low tones was varied to find his auditory psychophysical threshold and to study his uncertainty responses in this threshold region of difficult trials. The horizontal axis indicates the frequency of the tone. The green and red lines show the percentage of Low and High responses, respectively, at each pitch level. The blue line shows the percentage of trials receiving the uncertainty response at each pitch level. (b) The dolphin’s weighted overall Factor 1 behavior (hesitancy, slowing, wavering) for different pitch levels. From ‘The Uncertain Response in the Bottlenosed Dolphin (Tursiops truncatus),’ by J.D. Smith, J. Schull, J. Strote, K. McGee, R. Egnor, and L. Erb, 1995, Journal of Experimental Psychology: General, 124, p. 399, p. 402. Copyright 1995 by the American Psychological Association. Reprinted with permission.

Natua produced an additional crucial result. On some trials, he swam slowly and hesitantly, or waggled his head, or worked his jaw. We conducted a factor-analytic study of these behaviors, asking observers—who were blind to the trials’ pitch levels—to provide ratings of these incidental behaviors. A strong general factor of hesitation and wavering emerged. Figure 2B shows that scores on this factor varied systematically with pitch level, reproducing the curve of the animal’s URs.

Thus, Natua not only used the experimenter-provided UR to manage difficulty and uncertainty. He also expressed symptoms of something like his own uncertainty, through dolphin-specific behaviors. These facts must also help shape one’s interpretation of Smith et al.’s (1995) experiment.

3. The Associative Debate

As (human) cognitive researchers, these results raised many questions for us. We wanted to understand the cognitive architecture of uncertainty, and whether nonhumans shared that architecture. We wanted to know at what cognitive level humans and animals feel doubt, and whether it is the same level. We wondered about the role of working memory and working consciousness in monitoring uncertainty, and whether animals, as humans, do so with some conscious awareness. We thought that metacognition experiments might give scientists leverage on the problem of animal consciousness, an exciting possibility.

But the theoretical trajectory of the literature did not go in those directions. Instead, for years thereafter, the dolphin experiment, along with companion experiments with rhesus monkeys (Macaca mulatta), received heavy incoming constructive criticism. Many colleagues chose to emphasize the associative-learning processes they thought could produce results like the dolphin’s over the alternative explanation using processes like metacognition or cognitive monitoring. Other colleagues applied formal-mathematical models to show that mathematical gradients could reproduce observed findings. The general idea was that URs might be cued by “difficult” stimuli and that animals might be conditioned to make URs by the poor outcomes that often accompany these stimuli.

In fact, examining the CCBR issue from 2009, one sees that it focused sharply on this possibility. Hampton (2009) concluded that “many of the demonstrations of metacognition in nonhumans can be explained in terms of associative learning.” Jozefowiez, Staddon, & Cerutti (2009) attributed animals’ so-called metacognitive performances to “familiar learning principles, and thus lacking any metacognition construct.” Crystal and Foote (2009) concluded that “low-level mechanisms are sufficient to explain data from existing methods.”

Shortly thereafter, Le Pelley (2014) wrote his self-titled Primate Polemic—criticizing the attribution of metacognition to animals and attributing their “metacognitive” performances to associative learning. Readers will understand that it was not actually the primates’ polemic. Le Pelley’s (2012) article title made the same point: Metacognitive monkeys or associative animals? Simple reinforcement learning explains uncertainty in nonhuman animals.

In fact, one can summarize simply the second epoch of the animal-metacognition literature. It was dominated by an associative debate that overshadowed the field’s theoretical development (e.g., Carruthers, 2008; Hampton, 2009; Jozefowiez et al., 2009; Le Pelley, 2012, 2014; Smith, 2009; Smith, Couchman, & Beran, 2014; Smith, Zakrzewski, & Church, 2016; Staddon, Jozefowiez, & Cerutti, 2007; review in Smith, Church, Beran, & Washburn, 2018).

Therefore, researchers turned to the problem of eliminating associative concerns. There wasn’t a choice in this. The associative debate was front and center. It had to be adjudicated in each study and article. During this time, human cognitive researchers asked us to please stop harping on associationism. (We couldn’t.) They said it was obvious what was going on—animal metacognition was going on. (No comment.) The gulf between human-cognitive interpretations and animal-associative interpretations still must be taken seriously today, to understand what it means for collaboration, cross talk, and synergies across the disciplines of bio-behavioral research.

Yet it has been a decade of positive development since 2009. There have been elegant studies on animal metamemory (e.g. Morgan, Kornell, Kornblum, & Terrace, 2014; Templar & Hampton, 2012; Templar, Lee, & Preston, 2017). Various associative explanations of metacognitive performances have been carefully and systematically ruled out (Basile, Schroeder, Brown, Templer, & Hampton, 2015). URs have been shown to be recruited effectively even when immediate reinforcement is denied the animal, so that the UR cannot be conditioned to “difficult” stimuli (Smith, Beran, Redford, & Washburn, 2006). URs have been shown to be based on executive cognitive processes that are especially working-memory intensive (Smith, Coutinho, Church, & Beran, 2013; see below). Beautiful comparisons across animal species have been arranged that are beginning to draw the map of the evolutionary emergence of the metacognitive capacity (Beran, Smith, Coutinho, Couchman, & Boomer, 2009; Fujita, 2009; Paukner, Anderson, & Fujita, 2006; Roberts, et al., 2009; Sutton & Shettleworth, 2008).

The associative debate has largely evaporated, replaced by a strong theoretical consensus that some animals share some aspects of humans’ metacognitive capacity, though not necessarily all aspects of that capacity. The field has arrived to a strong theoretical belay stance, based on a decade of productive research.

4. Two Possible Lessons from the Associative Decade

Nonetheless, the associative decade in animal metacognition may offer lessons for comparative psychology as it enters another productive decade. First, associative descriptions of animals’ performance generally draw a flat, dimensionless theoretical picture of animal mind. There isn’t a focus on the mind’s cognitive architecture, but on the SR bonds. There aren’t cognitive levels, just the associative level. There aren’t different systems guiding learning and behavior, just the associative system. There aren’t processes especially executive, or especially working-memory intensive—these ideas clash with the associative construct. There aren’t declarative or conscious cognitive processes in the associative framework, for the same reason. Consequently, empirical investigations into these kinds of issues are not directly encouraged by theory, and they may be resisted by some.

Possibly, though, animal minds may be trying to tell us that they have depth, and levels, and awarenesses, some consciousness, structure, and different systems guiding learning and behavior. If so, and if these kinds of empirical explorations could be more proactively fostered by the field, then the field’s theoretical development would benefit.

Second, we believe that the associative attribution is sometimes applied too easily, too glibly. The result is that we may take our theoretical eyes off the specifics of the cognitive problem the animal faces, the exact mental representations that need to be generated and sustained during performance, and the necessary structure of a cognitive system that can achieve the given performance.

But our tasks may be trying to tell us that they require a mind that has depth, and levels, and systems, and so forth. If so, then there is great benefit in moving beyond the surface associative description to focus sharply on the task’s cognitive demands. The SR bond is really only the hitch joining the cart to the tractor. We need also to ask what is in the cart, where is the tractor going, and why.

5. The Inescapable Logic of Doubt

The dolphin experiment illustrates these problems. In fact, we believe the focus on associative explanations of the dolphin study was a theoretical mistake that arose from imprecise and misapplied constructs. It was not understood a) that threshold stimuli are profoundly special and aren’t even stimuli; b) that uncertainty cannot control behavior as supposed by associative-learning theory; c) that doubt is definitionally not an associative construct; and d) that uncertainty self-entails the higher-level cognitive psychology that must manage it. Here is how these problems can be seen reflected in Smith et al. (1995).

Figure 2a showed that the dolphin’s URs peaked near 2, 085 Hz. In essence, with the 2, 100 Hz High tone as the musical pitch C, the dolphin felt maximally uncertain to musical pitch B########: that is, 1/9^th of a half step away. One knows that there was no discrete, intermediate pitch class there that could have prompted some conditioned response—no auditory system could achieve this feat of resolution. The threshold stimulus is never a stimulus in this discrete, intermediate sense.

In fact, the threshold stimulus isn’t a stimulus at all. Imagine yourself in the shower, when you suddenly wonder if the phone is ringing. No-one in this moment has ever said: There is one of those it-might-be-a-phone-ringing-or-it-might-not-be-type stimuli. No, they say: I don’t know if the phone is ringing. It is interpretatively important that there is even an I, a me, there—a selfness. The psychology is self-evidently no longer about the world of stimuli and responses.

Possibly the dolphin at threshold was expressing that he didn’t know whether the High “phone” was ringing.

Next consider that at threshold the relevant, competing response strengths are at equipoise. So, the threshold state cannot condition, or exert stimulus control over, either primary response (High, Low). Nor can equipoise power the uncertainty response. In many experiments, the UR earns no reward, no differential consequence, no special next trial, nothing. It isn’t rewarding in the required sense to exert behavioral control at all. And yet some species use it to manage difficulty. Why? The threshold state cannot be about stimulus or behavioral control as associative-learning theory understands these constructs. Indeed, even committed associative theorists have acknowledged that the threshold state changes profoundly the nature of stimulus control and associative behavior (Boneau & Cole, 1967; Commons, Nevin, & Davison, 1991; Davison, McCarthy, & Jensen, 1985; Miller, Saunders, & Bourland, 1980; Terman & Terman, 1972).

To illustrate this point, we will lay down a challenge here that we know we’ll regret. It’s this: No-one will ever be able to condition any classical reflex (not salivation, not eye blinks, not fear suppression) using a threshold feeling-state of uncertainty as the CS.

On several occasions, we tried to remind associative theorists that threshold uncertainty tasks were inconsistently mapped in the manner discussed by Shiffrin and Schneider (1977; also Schneider & Shiffrin, 1977). At threshold, the animal genuinely cannot discern what pitch (High?, Low?) it is hearing. The perceptual system is variable and prone to some error. Either stimulus could be present and either response could be warranted. Therefore, making either reflexive response would be unwise. In that case, the cognitive system must resort to some kind of cognitive referee to break the associative tie, to set response criteria, to remember what has worked out well in this situation in the past. Shiffrin and Schneider called these higher-level processes controlled cognitive processes.

Probably Natua brought an analog of these controlled processes to bear on the threshold task that always demands them. If it doesn’t demand them, you’re not at threshold yet.

Another consideration goes to the dolphin’s own uncertainty behaviors—the slowing, the head waggling, and so forth. Why was his behavior so distinctively different on the threshold trials? It wouldn’t be if the UR were just the 3^rd reaction to the 3^rd stimulus class. Yet these behaviors found important historical resonance in the work of Tolman. He observed similar behaviors in rats that he called “lookings and runnings back and forth” and that he thought might become “A Behaviorist’s Definition of Consciousness” (Tolman, 1927, 1938) because they made visible mental turmoil in the animal and changing cognitive determinations in mind. Here, Tolman raised the striking possibility that the higher-level, cognitive-resolution processes at threshold might even be conscious.

If the dolphin’s conflict behaviors were expressing his conscious decisional turmoil on difficult trials, it is an extraordinary demonstration. We do not assert that. However, the possibility is a matter of fact, a matter of theoretical consideration, and a matter of empirical investigation that must be clearly acknowledged.

In a sense, therefore, threshold uncertainty is the associative theorist’s nightmare. The usual associative constructs like stimulus control, SR bonds, and so forth explode in the tight quarters of a psychophysical threshold task. But some minds rally at threshold, bringing alternative criterial and decision-making and conflict-resolving processes to bear. The essential requirement of a threshold task is for a higher-level, controlled process that can adjudicate uncertainty. Animals time and again have shown that their reflective minds have the depth and structure to manage effectively this adjudication. For these reasons, threshold uncertainty is a goldmine for the cognitive or metacognitive psychologist, or the behavioral neuroscientist.

6. Confirming the Distinctive Cognitive Architecture of Uncertainty

Twenty-five years on from Smith et al. (1995), Natua has gone to the endless school of herring in the sky. As a practical matter, it has emerged that nonhuman primates (enjoying tang-flavored Cap’n Crunch food rewards) can perform 10 times the daily trials that a dolphin (enjoying whole-herring rewards) can. Nonetheless, the lessons discussed here from the earliest experiment—that the threshold state requires a qualitatively distinctive psychological explanation, and that the UR cannot be understood associatively—still continue to be confirmed. In closing, we will illustrate this with two recent studies, with monkeys and with the human primate.

First, Smith et al. (2013) gave macaques a concurrent memory load while they performed the Sparse-Uncertain-Dense discrimination task shown in Figure 3. Macaques saw an unframed box filled to one of 60 densities with lit pixels. Each level was 1.8% more pixel dense. They touched the S or D icons with their computer cursor to respond that a box was Sparse or Dense (Stimulus Levels 1–30 and 31–60, resp.). Trial difficulty centered and focused on the trial levels near the Sparse-Dense breakpoint (Levels 30–31). They touched the ? icon to decline the trial, transitioning out of any difficult trials they did not wish to complete.

Figure 3.

Illustrating a trial in a Sparse-Uncertain-Dense task.

Smith et al.’s first hypothesis was that Sparse and Dense responses would be grounded in associative learning as traditionally understood, making few working-memory demands so that they would be unaffected by the concurrent memory load. Their second hypothesis was that URs would be an expression of higher-level, controlled decision making. Thus, URs might be strongly affected by the load—uncertainty responding might evaporate under load conditions. Figure 4 shows that URs were selectively disrupted by the concurrent memory task. Sparse and Dense responses were not. The observation that the metacognitive responses of macaques can be working-memory intensive fits perfectly with the idea that uncertainty self-entails the intervention of more executive processes.

Figure 4.

A, B: Percentage of uncertainty responses (solid line), sparse responses (dashed line), and dense responses (dotted line) made by macaques Murph and Lou in their baseline performance and in their first phase of concurrent-load testing. From “Executive-Attentional Uncertainty Responses by Rhesus Macaques (Macaca mulatta),” by J. D. Smith, M. V. C. Coutinho, B. A. Church, & M. J. Beran, Journal of Experimental Psychology: General, 142, 472. Copyright 2013 by the American Psychological Association. Reprinted with permission.

Second, Paul, et al. (2015) placed humans in a similar Sparse-Uncertain-Dense task. Then, using rapid event-related fMRI, they showed that the neural circuit recruited during humans’ URs is distinctively different from that recruited during humans’ Sparse and Dense responses. Figure 4 (top) shows the simple activation pattern when participants made correct Sparse or Dense responses in lieu of URs. There was selective activation in the occipital lobe bilaterally and in the caudate nucleus and nucleus accumbens. This supports the interpretation that Sparse and Dense responses are first-order associative responses in which discrete stimuli elicit well-conditioned responses. Figure 4 (bottom) shows the complex activation pattern when participants made URs in lieu of Sparse/Dense responses. URs reflected the activation of a broad-based network including prefrontal cortex, anterior and posterior cingulate cortex, anterior insula, and posterior parietal areas. Thus, as discussed above, uncertainty monitoring apparently benefits from, and may inherently demand, a larger-scale cognitive-control network.

To illustrate this point, we will lay down another challenge. It’s this: When well-acclimated macaques eventually participate in the equivalent fMRI assessment to that in Paul et al. (2015), they will confirm that the uncertainty-monitoring circuit is broad-based and complex, even in a species that thinks without any language or declarative strategies. We think this must be so, because the threshold state of doubt inevitably self-entails a higher-level cognitive capacity in mind—for humans, for macaques, and for that awesome dolphin Natua.

We hope that a special issue organized in 2029 will reflect that comparative psychology turned more toward exploring the cognitive architecture of important capacities like uncertainty monitoring and metacognition, including considering how animal minds may show depth, and structure, and systems, and cognitive levels, and awareness, and declarative capability. Recent research in the field of animal metacognition makes it clear that researchers are now prepared to do so. We note that this study will simultaneously be the study of humans’ cognitive emergence during evolution, and so it will knit together more closely and constructively the allied fields of biopsychology, comparative and cognitive psychology, and neuroscience.

Figure 5.

Top. Whole-brain results from the contrast Correct Sparse-Dense Response > Uncertainty Response projected on an inflated lateral and medial cortical surface. Table 2 in Paul et al. (2015) listed the coordinates and number of voxels for every significant cluster. Images were cluster thresholded (correcting for multiple comparisons) at z > 3.54, p < .01. Bottom. Whole-brain results from the contrast Uncertainty Response > Correct Sparse-Dense Response projected on an inflated lateral and medial cortical surface. Table 3 in Paul et al. (2015) listed the coordinates and number of voxels for every significant cluster. Images were cluster thresholded (correcting for multiple comparisons) at z > 3.54, p < .01. From “Neural Networks of the Psychophysical Uncertainty Response,” by E. J. Paul, J. D. Smith, V. Valentin, B. O. Turner, A. K. Barbey, & F. G. Ashby, Cortex, 71, 316 (top), 317 (bottom). Copyright 2015 Elsevier Ltd. Reprinted with permission.