A metacognitive illusion in monkeys

Abstract

Like humans, monkeys can make accurate judgements about their own memory by reporting their confidence during cognitive tasks. Some have suggested that animals use associative learning to make accurate confidence judgements, while others have suggested animals directly access and estimate the strength of their memories. Here we test a third, non-exclusive possibility: perhaps monkeys, like humans, base metacognitive inferences on heuristic cues. Humans are known to use cues like perceptual fluency (e.g. how easy something is to see) when making metacognitive judgements. We tested monkeys using a match-to-sample task in which the perceptual fluency of the stimuli was manipulated. The monkeys made confidence wagers on their accuracy before or after each trial. We found that monkeys' wagers were affected by perceptual fluency even when their accuracy was not. This is novel evidence that animals are susceptible to metacognitive illusions similar to those experienced by humans.

1. Introduction

Humans are explicitly metacognitive—they are able to declare what they know and do not know. For example, when asked a question, you can decline to answer if you know that you do not know, give a qualified answer or answer with certainty. Non-human animals cannot declare metacognitive judgements verbally but they nonetheless might have reflective processes that are similar to humans’ [1–3]. Research on human and non-human metacognition is a window on the emergence of the reflective mind.

Monkeys, non-human apes and dolphins can make non-verbal metacognitive choices accurately using a variety of naturalistic and computerized tasks [4–10]. Early studies showed that animals could make appropriate use of an ‘uncertain’ response button in psychophysical experiments [4,5,8]. For example, in one study monkeys were presented with some number of dots and were required to make a judgement about whether there were greater or less than 10 using a computerized categorization task [8]. Monkeys were then given an ‘uncertain’ option, which allowed them to decline to answer and move on to a new trial after a short delay. Monkeys used the ‘uncertain’ option on difficult trials when the number of items was close to the category boundary, suggesting that monkeys could make accurate uncertainty decisions. Similar findings have emerged from other tasks. Monkeys seek information before making choices [8,11,12], opt out of trials in which they are unable to answer [7] and make accurate bets on their own performance [9]. All of these studies concluded that monkeys can track their own uncertainty.

(a). Theoretical explanations of non-human animal metacognition

Some researchers have suggested that non-human metacognitive decisions could be based on the process of associative learning of responses and rewards [13–15]. Proponents of the associative learning models point out that it is a basic principle of operant conditioning that rewarding a behaviour increases its frequency, and that in metacognition experiments animals are rewarded for behaving in a way that appears to be metacognitive. Thus, they argue, perhaps what appears to be metacognition is actually operant conditioning. One study used an exclusively associative model of learning to explain the results of multiple animal metacognition experiments [15]. Researchers have tried to find procedures that are not subject to this criticism [16], or argued that the logic underlying this argument is flawed [17,18], but the debate remains. Compelling evidence against an associative learning model would be a study in which behaving metacognitively did not increase reward rate [13]. One question, in the present research, is whether animals would behave metacognitively in such a study.

(b). Theoretical explanations of human metacognition

Human metacognitive decisions are susceptible to metacognitive illusions [19,20]. For example, a recent study showed that more perceptually fluent stimuli (words written in large font) are judged as being more memorable than less fluent stimuli (words written in small font), yet fluency did not affect actual recall in a later memory test [19]. The effects of perceptual fluency have been seen in a variety of tasks and modalities in humans [20–24]. These effects of stimulus fluency support the cue-utilization view, which suggests that metacognitive judgements are based upon available cues at the time of the judgement rather than direct access to one's memory traces [25–27].

Research on human metacognition generally focuses on metacognitive errors, whereas non-human animal research has focused on metacognitive success [17]. Errors are revealing because they are separate memory from judgement: when a cue affects judgements but not memory, it is possible to eliminate memory strength as an explanation and identify another cue as controlling the judgement (as, for example, font size was identified as an important cue in the studies described above). If monkeys were shown to be sensitive to a cue, such as perceptual fluency, that was divorced from memory strength, it would be the first direct evidence about the processes that underlie metacognitive judgements in non-human animals. It would also suggest that the metacognitive processes in the human mind have an evolutionary antecedent in monkeys.

One might ask whether an inaccurate metacognitive judgement should be considered metacognitive. In past research with animals, only accurate metacognitive judgements have been used as evidence of metacognition. We argue, however, that inaccurate judgements can also be metacognitive. Research with humans provides a strong theoretically grounded prediction that perceptual fluency will affect monkeys' confidence in their memories. If it does, we would argue that is strong evidence that the animals are being metacognitive.

(c). The present study

The research we present here is modelled after a recent study that tested both prospective and retrospective metacognitive abilities in monkeys using a gambling task [28]. Subjects were presented with a set of stimuli, took a memory test and made low-risk or high-risk bets on their accuracy. The betting occurred either before (prospective) or after (retrospective) the memory task. If subjects chose the low-risk option, they received a small reward regardless of accuracy on the memory task; if they selected the high-risk option, they would either receive a large reward or equally large punishment based on their performance on the memory task. High-risk responses were seen more often on correct trials, and this was true whether the bet was made before or after the memory task.

In this study, we experimentally manipulated perceptual fluency in a memory task and examined monkeys’ confidence choices using a computerized gambling task. Monkeys made bets on their own performance either before answering a memory task (prospective) or after answering a memory task (retrospective).

If monkeys use the same cues as humans, then perceptual fluency should affect their confidence more than it affects their learning. As a consequence of this learning-confidence disconnect, judgements based on perceptual fluency do not increase metacognitive accuracy, and they can even make it decrease. This experiment will test whether monkeys are susceptible to the same metacognitive illusions that affect humans [19,20]. Metacognitive illusions are not easily explained by the associative models commonly used to explain metacognitive behaviour in animals. Thus, this study will provide novel insight on theories of animal metacognition.

2. Material and methods

(a). Subjects

Two adult male rhesus monkeys (Macaca mulatta) were tested. Both animals had previously been trained on metacognitive tasks including both retrospective and prospective confidence judgements [1,9,28,29]. Animals were individually housed and tested in their home cages in the colony room. Animals were kept on an ad libitum food and water diet approved by the University of Rochester Committee on Animal Resources and veterinary staff. All animal care procedures were in accordance with an Institutional Animal Care and Use Committee (IACUC) protocol.

The goal of the study was to test whether a particular cognitive capacity exists in non-human primates. To show the existence of a capacity, one only needs proof of a single example (e.g. to show that prime numbers go higher than 5, one only needs one example, 7). We chose a sample size of 2 because it allowed us to test the existence of a cognitive capacity with rich data (many data points within each individual and condition). A limitation of a small sample size is that one can only make generalizations to a population if the ability is homogeneous. Many experiments have shown that metacognitive abilities are phylogenetically widespread across Old World primates and apes [6–12], indicating that individuals within these species probably are homogeneous for metacognitive capacity. By extension, any effect of fluency may generalize across species. However, that question of homogeneity is independent of our existence proof that fluency can effect monkeys' metacognitive choices.

(b). Materials and stimuli

Stimuli were presented on an Elo (ET1529 L) touchscreen monitor using software written in Xojo (REALbasic). Four hundred line drawings from the Boston Naming Test and Snodgrass picture libraries were used [30,31]. The training stimuli were created by decreasing the contrast on the line drawings to 70% (medium fluency—dark grey and white). The testing stimuli were made using the 100% contrast (high fluency—black and white) and 40% contrast (low fluency—light grey and white).

(c). Procedure

The primary task consisted of a match-to-sample memory task. Subjects started a trial by pressing a white start box in the centre of the screen. Subjects then saw a sample stimulus and were required to press it to move onto the next screen. In the retrospective condition, subjects then were presented with the sample stimulus and two to three distractor images (figure 1a). Subjects were required to press one stimulus to move onto the betting screen.

Figure 1.

Example trial protocol. Subjects saw a neutral start screen followed by a sample stimulus. In the retrospective condition subjects then completed the match-to-sample task, followed by the betting screen (a). In the prospective condition, subjects completed the betting screen before answering the match-to-sample task (b). (Online version in colour.)

On the betting screen subjects could either make a small bet (low confidence) or a large bet (high confidence). If subjects chose the small bet they received one token in their token bank regardless of their accuracy on the memory task. If subjects chose the large bet, they either gained or lost three tokens based on their performance on the memory task. When this token bank was filled (12 tokens), a small food reward was released and the number of tokens reset to nine tokens. Subjects had previous exposure to this token bank reward system.

In the prospective condition, subjects moved directly to the betting screen after touching the sample image (figure 1b). After making a bet, subjects were shown the match and distractor images and were supposed to press the match. After responding to the memory task, subjects received the guaranteed one token for small bets, or gained or lost three tokens based on performance on the memory task for high bets.

A bias reduction procedure was used to reduce biases in betting behaviour. The bet icon that had been used less recently was shown immediately once the betting screen was initiated. The more frequently used bet icon was shown after a short delay (mean delay: 3.7 s). This delay was modified on each trial based on previous biases within the session. As bias increased so did the wait time for that bet icon. The delay was calculated using the following formula: delay = [(previous bias value)(0.97) ± 1]/2.

(d). Training

Subjects were trained on computerized gambling task before the fluency manipulation was added. Subjects received training on retrospective and prospective confidence judgement conditions with stimuli that had an intermediate value of image contrast (dark grey stimuli) for 20 sessions (150 trials per session) before testing sessions began. Trials in which subjects took longer than 5 s to respond were removed from the analyses (4% of trials).

(e). Testing

Once familiarized with the task, a fluency manipulation was added. For each trial, the contrast of the sample and match/distractors was either presented in light grey on a white background (low fluency) or black on a white background (high fluency). The fluency was randomized such that no more than five trials in a row would have the same fluency. Subjects received 10 sessions of 160 trials in each condition. Trials in which subjects took longer than 6 s to respond were removed from the analyses (1% of trials).

3. Results

During training, both animals performed above chance on the match-to-sample task (mean accuracy in the last 10 sessions = 82%, binomial test p < 0.001). Additionally, we found a significant correlation between accuracy and risk, such that both animals made more high-risk bets than low-risk bets when they correctly identified the match (overall φ = 0.30, p < 0.001). Thus animals made accurate metacognitive judgements during training, before the fluency manipulation was added.

As in the training trials, once the fluency manipulation was added subjects performed above chance on the match-to-sample task (binomial test; mean accuracy = 76%, p < 0.001). Subjects were also more likely to make high-risk bets on trials in which they correctly identified the match (overall: φ = 0.33, p < 0.001; Lashley: φ = 0.20, p < 0.001; Ebbinghaus: φ = 0.46, p < 0.001). We tested whether there was a linear relationship between risk and response time. We found a correlation between response time and risk choice in one monkey but not the other (Lashley: r = 0.05, p = 0.79; Ebbinghaus: r = 0.26, p < 0.001). To test whether the relation between response time and risk could account for subjects’ betting behaviours, we conducted a partial correlation between accuracy and risk controlling for response time. We found that accuracy and risk choice were positively correlated in both monkeys, even when response time was controlled (overall partial correlation: r = 0.34, p < 0.001; Lashley: r = 0.05, p < 0.001; Ebbinghaus: r = 0.47, p < 0.001). This suggests that response time was not the only cue used to make confidence judgements.

We measured the effects of stimulus fluency on confidence judgements. We found that, overall, subjects were significantly more likely to make a high bet than a low bet when the sample, match and distractors were high fluency compared with low fluency (overall φ correlation = 0.04, p = 0.01; Lashley: φ = 0.04; Ebbinghaus: φ = 0.04; figure 2a). In comparison, the correlation between fluency and accuracy on this task did not reach significance (overall φ correlation = 0.03, p = 0.09; Lashley: φ = 0.02; Ebbinghaus: φ = 0.03; figure 2b). To test whether the effect of fluency on confidence is greater than its effect on accuracy, we ran a partial correlation to measure the effect of fluency on confidence while controlling for accuracy and response time (overall partial correlation: r = 0.04, p = 0.01; Lashley: r = 0.04; Ebbinghaus: r = 0.03). Stimulus fluency had a significant effect on confidence judgements that was independent of both response time and accuracy.

Figure 2.

Monkeys made more high bets for trials presented in high fluency compared with low fluency (a). There was no effect of fluency on the memory task performance (b).

To further investigate the effects of stimulus fluency on confidence judgements versus discrimination accuracy, we analysed the data using a signal detection approach [32]. This allows us to calculate metacognitive sensitivity, the relation between confidence choices and accuracy (d′), using the proportion of correct hits (high bets when correctly responded to the memory task) and correct rejections (low bets when incorrectly responded to the memory task) [32]. It also allows us to test for a response bias (c′) while accounting for metacognitive sensitivity. We found that there was a significant difference in metacognitive bias between fluency conditions such that subjects were more conservative (higher c′ values) on low-fluency trials compared with the high-fluency trials while taking account for sensitivity (low fluency c′ = 0.38, high fluency c′ = 0.17; t₁₉ = 2.05, p = 0.05). By contrast, metacognitive sensitivity was positive in both conditions which shows that they made accurate metacognitive judgements, yet there was no difference in the sensitivity between fluency conditions (low fluency d′ = 0.79, high fluency d′ = 0.75; t₁₉ = 0.64, p = 0.53). Thus while metacognitive sensitivity remained the same across conditions, there was a bias to choose the high bet more often in the high-fluency condition compared with the low-fluency condition. This shows that fluency specifically affected subjects' confidence in their answers.

To test if the fluency effect was learned during testing, we conducted a logistic regression predicting risk choice using fluency, and the interaction between fluency and session number. This analysis allows us to use continuous and binary predictors within the same analysis. If the fluency effect was learned we should see a positive interaction between session number and fluency on risk choice. We found a significant effect of fluency on risk, but no interaction between fluency and session (B_fluency = 0.16, p < 0.01, B_{fluency*session} = 0.01, p = 0.71). Thus the effect of fluency on monkeys’ behaviour was consistent throughout testing and not learned from experience with different fluencies.

An associative learning account could explain these data only if the monkeys received more rewards by responding based on fluency. In fact, basing bets on fluency decreased the number of pellets monkeys received during testing. The average number of rewards received by monkeys was significantly less during testing sessions (when fluency was used to make risk choices) compared with non-testing sessions (χ² (1, N = 10 153) = 18.9, p < 0.001). Monkeys used fluency to make risk choices even though it led to a decrease in their overall reward amount. This finding shows that associative learning cannot account for the monkeys' fluency-based betting decisions.

Few studies have directly compared the effects of stimulus fluency on retrospective and prospective confidence judgements. In order to investigate differences between prospective and retrospective judgements, we compared them in separate analyses. We tested the correlation between confidence and accuracy. In both conditions, accuracy was positively correlated with confidence judgements (retrospective: φ = 0.38, p < 0.001; prospective: φ = 0.30, p < 0.001; figure 3a,d). Thus, subjects were making accurate retrospective and prospective confidence judgements. Response time also had a significant correlation with risk (retrospective: r = 0.06, p = 0.05; prospective: r = 0.25, p < 0.001; figure 3b,e), although this correlation is difficult to interpret in the retrospective condition because it appears nonlinear (figure 3b).

Figure 3.

Differences between the retrospective and prospective conditions. Accuracy and response time were positively correlated with confidence choices in both conditions (a,b,d,e). There was a significant effect of fluency in the retrospective condition (c), but not the prospective condition (f). (Online version in colour.)

We tested the effects of fluency in each condition. In the retrospective condition, subjects made more high-confidence judgements when presented with high-fluency stimuli (retrospective: φ = 0.07, p < 0.01; figure 3c). The effect of fluency did not reach significance in the prospective condition but was in the expected direction (prospective: φ = 0.02, p = 0.22; figure 3f). To test if the effect of fluency on confidence could be explained by accuracy or response time, we conducted a partial correlation between fluency and risk, controlling for accuracy and response time in each condition (partial correlations: retrospective: r = 0.07, p = 0.01; prospective: r = 0.02, p = 0.22). In sum, fluency affected confidence in the retrospective condition and this effect cannot be attributed to accuracy or response time. Fluency did not have a significant effect on confidence in the prospective condition.

4. Discussion

This study has three main implications. Firstly, associative models are unlikely to explain monkeys’ susceptibility to the perceptual fluency illusion because relying on perceptual fluency to make metacognitive judgements did not increase the monkeys' reward rate.

Secondly, our results provide a new explanation for how animals make confidence judgements. Previous research on animal metacognitive abilities has sometimes concluded that animals can internally monitor their own memory [3,29], although this idea has been questioned [17,33]. Our results offer a distinct (but not exclusive) alternative to internal memory monitoring hypothesis. We suggest that in order to make metacognitive judgements, our monkeys made inferences based on perceptual cues. That is, although they were making judgements about their memory accuracy, non-memorial cues, such as the level of contrast of the images, helped guide their judgements. The level of contrast could not have been the only cue that guided the monkeys’ judgements because the data showed a positive correlation between desire for risk and response accuracy that was independent of contrast. Thus, other cues apparently influenced judgements. Our data do not speak to what these cues were, because we only manipulated contrast, but a variety of non-associative explanations of accurate metacognitive judgements have been proposed [2,3,13,16–18,33].

Finally, monkeys and humans are subject to some of the same metacognitive fallacies. Both respond as though stimuli that are easier to see will be remembered better, and this can lead to inaccurate confidence judgements [19,20,34,35]. This similarity suggests that metacognitive processes in monkeys and humans may have shared evolutionary origins.

An important open question is what is the cognitive process by which fluency affects metacognitive confidence in humans. Although it is well established that more fluent stimuli elicit higher metacognitive judgements in humans, there is debate about the possible role of beliefs in this relationship. Some have suggested that, for example, participants judge stimuli shown in a large font as memorable not because they process these stimuli fluently, but because they believe words shown in large fonts are especially memorable [36,37]. Other studies have suggested that the effects of fluency are not mediated by explicit beliefs [35,38–40]. Under this hypothesis, when people perceive a stimulus fluently, a heuristic process leads them to make high memory ratings without the need for explicit beliefs about how fluency affects memory. The data presented here are relevant to this debate because monkeys' metacognitive judgements were affected by fluency. This means we either need to attribute explicit beliefs about how fluency affects memory to monkeys, or favour the hypothesis that fluency affects metacognitive decisions directly via heuristics in both monkeys and humans.

Ethics

All experiments and procedures were done in accordance with an IACUC protocol and were approved by the local ethics committee.

Data accessibility

The dataset has been made available at Dryad Digital Repository (http://dx.doi.org/10.5061/dryad.m9s45) [41].

Authors' contributions

All authors developed the study concept and contributed to the study design. S.F. collected the data. S.F. and J.F.C. performed the data analysis. All authors wrote the manuscript, discussed the results and commented on the manuscript.

Competing interests

We have no competing interests.

Funding

This work is supported by the National Science Foundation (DRL1459625), the James S. McDonnell Foundation, the Alfred P. Sloan Foundation and the University of Rochester.