Cashing Out: The Decisional Flexibility of Uncertainty Responses in Rhesus Macaques (Macaca mulatta) and Humans (Homo sapiens)

Abstract

Researchers are exploring whether animals share with humans something like a metacognitive capacity. Though some results point to human-animal continuities in this domain, they face the dominant criticism that animals’ performances might be associative. A persistent problem is that animal-metacognition paradigms present static environments of risk and reward that may foster inflexible and conditioned responding. Those environments do not challenge animals to show the flexibility in their decision strategies that could indicate an antecedent capacity to metacognition. Accordingly, we tested macaques and humans on an uncertainty-monitoring paradigm in which risk changed dynamically. Participants classified stimuli of different difficulties while also choosing when to use a cashout response to collect the accumulated rewards that would be forfeit on a discrimination error. Macaques and humans flexibly adjusted their decision criteria to achieve appropriate protection against the cost of error that could differ depending on trial difficulty and the number of rewards at risk. In particular, monkeys widened their cashout-response region as their accumulated rewards increased, providing more protection against a more costly error. These findings demonstrate a new continuity between humans’ and animals’ uncertainty systems. They reveal a calibration by macaques of present risk to trial difficulty tolerated. They show that animals’ uncertainty-monitoring and risk-management systems have substantial trial-by-trial flexibility.

Humans have metacognition—an explicit, declarative awareness of their states of learning, and knowing (Dunlosky & Bjork, 2008; Flavell, 1979; Koriat, 1993; Metcalfe & Shimamura, 1994; Nelson, 1992; Schwartz, 1994). It is a late cognitive-developmental achievement (Balcomb & Gerken, 2008). It may be linked to humans’ conscious awareness (Koriat, 2007; Nelson, 1996). It may be linked to their self-awareness (Gallup, 1982). For all these reasons, this sophisticated cognitive capacity might be humanly unique (Metcalfe & Kober, 2005).

Therefore, it is an important question whether nonhuman animals (hereafter, animals) have a functional analog to human metacognition. There is a growing consensus that some species do (e.g., Basile, Hampton, Suomi, & Murray, 2009; Beran, Smith, Coutinho, Couchman, & Boomer, 2009; Call, 2010; Foote & Crystal, 2007; Fujita, 2009; Hampton, 2001; Kornell, Son, & Terrace, 2007; Middlebrooks & Sommer, 2011; Roberts, Feeney, McMillan, MacPherson, Musolino, & Petter, 2009; Smith, Coutinho, Church, & Beran, 2013; Suda-King, 2008; Sutton & Shettleworth, 2008; Washburn, Gulledge, Beran, & Smith, 2010).

However, interpretative caution is justified. Even if an animal’s performance in an uncertainty task seems metacognitive, one must consider whether that performance might reflect low-level mechanisms based in stimulus-response associations (Morgan, 1906). This possibility has always been this literature’s dominant theoretical concern (Hampton, 2009; Jozefowiez, Staddon, & Cerutti, 2009a; Kornell, 2009; Le Pelley, 2012; Smith, 2009; Smith, Beran, Couchman, & Coutinho, 2008; Smith, Couchman, & Beran, 2012a,b; Smith, Couchman, & Beran, 2014a,b; Smith, Shields, & Washburn, 2003; Staddon, Jozefowiez, & Cerutti, 2007). In fact, the animal-metacognition literature mainly comprises diverse approaches toward ruling out associative explanations. We offer a new empirical approach toward constraining associative explanations that complements other approaches (Call & Carpenter, 2001; Hampton, 2001; Kornell et al., 2007; Smith, Beran, Redford, & Washburn, 2006; Suda-King, 2008). We also offer the theoretical perspective that this debate (associative-metacognitive) is not dichotomous. Even if one constrains traditional associative interpretations, the alternative explanation is not necessarily full-fledged human metacognition.

An influential paradigm in animal metacognition illustrates the theoretical concern about interpretation. This paradigm lets human and animal subjects use an uncertainty response proactively to decline difficult trials that would likely cause them errors and cost them timeouts. In testing humans, the uncertainty response is clearly an elemental behavioral index of a higher-level cognitive capacity like metacognition. Humans universally report that their use of this uncertainty response is based on their conscious, metacognitive uncertainty (e.g., Shields, Smith, & Washburn, 1997). Psychophysicists have long endorsed that the uncertainty response is psychologically distinctive and metacognitive (e.g., Boring, 1920; George, 1917). Paul, Smith, Valentin, and Ashby (submitted) showed that the fMRI signature of humans’ uncertainty response is very different from the signature of perceptual responses in the same task.

In testing animals, though, uncertainty tasks have the limitation that they present a steady-state landscape of difficulties, outcomes, and contingencies. There is a constant distribution of stimuli from which trials are selected randomly, a constant frequency of difficult trials, constant rewards and timeouts for correct and incorrect responses, and thus stable and predictable reinforcement contingencies. Over thousands of trials, the same risks, rewards, and optimal response strategies apply. These static contingencies are ideal for entraining inflexible, conditioned response patterns learned from extensive experience within an unchanging task. Indeed, all of this field’s formal models depend on this static-risk environment to build the reinforcement histories and response-strength registers that might let one explain animals’ performances associatively (Smith et al., 2008; Jozefowiez, Staddon, & Cerutti 2009b,c; Le Pelley 2012; Staddon et al., 2007). However, such conditioned response patterns would of course not amount to metacognition. Moreover, these static environments in no way challenge animals to show trial-to-trial flexibility in their risk assessments, confidence judgments, or decisional strategies that might reveal that they have higher-level, executive systems for managing risk and uncertainty. Human confidence and uncertainty is situationally flexible. This is why humans can use verbal instructions to shift on demand their confidence criteria in a psychophysical task. Many psychophysical methods (e.g., the construction of receiver operating characteristic [ROC] curves) depend on flexibility of this kind. It has remained a prominent view that criterion-setting mechanisms are cognitive, confidence-based, and possibly meta- to the ongoing discrimination (Swets, Tanner, & Birdsall, 1961; Treisman & Faulkner, 1984). But static contingencies cannot show whether animals’ uncertainty performances have flexibility of this kind, and this is an important failure of existing animal-metacognition paradigms.

Therefore, we assessed humans’ and macaques’ uncertainty performance in an environment of dynamically changing risk. Participants needed to adjust their decisional strategies and response criteria flexibly based on changing levels of risk. They had to broaden or narrow the range of trial difficulties they were willing to engage. Under some conditions, it was fine to engage difficult trials because there was little risk to guessing. Under other conditions, it was not.

In our task, we instituted a token-bank system and a cashout procedure in which macaques and humans could exchange tokens for food rewards. Macaques have proven competent in both situations. For example, Evans (2007) gave macaques a computerized task in which they chose stimuli in a learned serial order. Each selection led to the accumulation of more rewards until a monkey eventually chose to “cash out” those rewards. In another paradigm that influenced the present research, Kornell et al. (2007) introduced a token bank into their metacognition test with macaques using a gambling-betting paradigm in which monkeys accumulated tokens onscreen through making correct responses to strings of trials. The token-bank system we used here also allowed participants to see onscreen how much reward was at risk and to choose when they would take their accumulated rewards. More important, it let us take tokens away from participants, a possibility not allowed under normal trial-by-trial reinforcement. This produced a dynamic risk environment. Sometimes, participants had seven food rewards at risk as they approached a trial; sometimes none. If they adjust their tolerance for risk in an effective way, then they should consider both the number of possible rewards and their uncertainty about the present trial. By integrating risk with trial difficulty, they might show us a flexible decision-making process, with response criteria changeable from trial to trial.

To make this assessment, we gave humans and macaques a cashout response. This response replaced the uncertainty response that is common in animal-metacognition research. The cashout response let participants choose when to recover accumulated rewards and when to continue accumulating them by directly completing trials. The cashout response did not free participants from having to respond to potentially aversive stimuli or from having to complete difficult trials because a response was required on every trial. That is, one could not avoid difficult trials that might end in penalty by using the cashout response. Instead, the cashout response allowed humans and macaques to manage risk by recovering rewards prior to completing a difficult trial, so that it could then be completed without the risk of the losing accumulated rewards.

We asked whether participants would selectively make cashout responses before attempting difficult trials that might cause errors and therefore losses (any error would empty their token bank). And, crucially, we asked whether participants’ standards for which trials deserved cashout responses would change dynamically depending on the current risk conditions. We hypothesized that participants might have stricter criteria for which trials should be directly attempted when more accumulated tokens were on the line. That is, we hypothesized that the cashout-response region would spread out more broadly, progressively encompassing less difficult trial levels, as tokens accumulated and the costs of a potential error grew more severe.

At the limit, this behavioral pattern would verge on participants’ deliberately adjusting their confidence criteria from moment to moment based on prevailing risk conditions (i.e., how many tokens were on the line, and how hard the present trial would be to solve). This would demonstrate an important new continuity between animals’ and humans’ uncertainty-monitoring performances. It would show a flexible use of uncertainty, complementing the controlled and executive use of uncertainty demonstrated by macaques in Smith et al. (2013). It would encourage the consideration of cognitive and executive interpretations of macaques’ performances in uncertainty-monitoring paradigms.

Method

Participants

Macaques

Two male rhesus monkeys (Macaca mulatta), Murph and Lou, were tested in their home cages with 24-hr access to the test apparatus, working or resting as they chose. They were neither food deprived nor weight reduced for the purposes of testing, and they had continuous access to water. The primate component of the research was conducted with IACUC approval. These monkeys had significant experimental histories in cognitive research using computerized tasks. They had participated in studies pertaining to numerical cognition (e.g., Beran, Johnson-Pynn, & Ready, 2008), analogical reasoning (e.g., Flemming, Beran, & Washburn, 2007), episodic memory (Hoffman, Beran, & Washburn, 2009), and prospective memory (e.g., Evans & Beran, 2012). They had participated in uncertainty-monitoring paradigms (e.g., Beran, Smith, Redford, & Washburn, 2006), including during cognitive multitasking (Smith, Redford, Beran, & Washburn, 2010) and under concurrent cognitive loads (Smith et al., 2013). They had also participated in information-seeking paradigms (Beran & Smith, 2011). These monkeys instantiated well the individual differences we have observed in uncertainty responding among macaques. Murph and Lou, respectively, had shown themselves to be generous and reluctant uncertainty responders. Finally, these monkeys also had experience performing in a token economy with the option to decide when to cash out accumulated rewards (Evans, 2007).

Humans

Forty-one University at Buffalo undergraduate introductory psychology students—with normal or corrected-to-normal vision—participated in a 1 hr session in fulfillment of a course requirement. Their average age was 19.0 years. There were 43% females and 57% males. Each participant completed the number of trials they could within the experimental session. Each week, the participant who earned the most points received a $10 reward. This incentive was intended to improve humans’ task motivation. The human component of the research was IRB approved, and participants gave their informed consent.

Apparatus

Macaques

Monkeys were tested using the Language Research Center’s Computerized Test System (Richardson, Washburn, Hopkins, Savage-Rumbaugh, & Rumbaugh, 1990). Monkeys reached though the cage mesh to manipulate a joystick that controlled a cursor’s position on a computer monitor. The computer automatically delivered 94-mg fruit-flavored pellets (Bio-Serv, Frenchtown, NJ) when they were due through a dispenser interfaced to the computer using a relay box and output board (Keithley Instruments, Cleveland, Ohio).

Humans

Humans were tested on Windows PCs, making responses on a keyboard by pressing one of three adjacent keyboard keys that were labeled and arranged in spatial correspondence to the response icons on the screen. Stimuli for both species were created and presented, and responses were collected and analyzed, using Turbo Pascal 7 software.

Procedure

Both species were presented with the same computer-controlled task. Each trial consisted of a psychophysical sparse-dense judgment task with a 201 × 101 borderless stimulus rectangle presented in the top center of the computer screen. The rectangle contained a variable number of randomly placed pixels. The 42 steps of the continuum ran from 1,085 pixels (Level 1) to 2,255 pixels (Level 42), with each level containing 1.8% more pixels than the last. The task was to discriminate sparse boxes (Levels 1–21, 1,085–1,550 pixels) from dense boxes (Levels 22–42, 1,578–2,255 pixels). On each trial, the stimulus level chosen was a random selection from the range 1–42. Trial levels near the ends of the continuum were easily classified by humans and macaques. Trials near the breakpoint of the discrimination (i.e., near Levels 21–22) were difficult to classify and could cause classification errors.

Figure 1 shows the screen from a trial. Participants had two primary perceptual responses—Sparse and Dense—as indicated by the S and D icons. These were correct for trial levels 1–21 and 22–42, respectively. The cashout response was indicated by gold coins in a box at the screen’s center below the pixel rectangle. This indicated the task’s current risk state, but the cursor could also be moved to this box to let the participants take their accumulated rewards.

Figure 1.

The screen from a trial in the cashout task in gray scale.

For every correct primary response, one additional gold coin appeared in the coin box (up to a maximum of 8). When an incorrect response was made, all the accumulated gold coins disappeared one at a time until the coin box was completely empty and the participant began the accumulation process again. Thus, the situation was riskier when the token box was fuller because there were more tokens at stake for the chance to earn just one more token. After cashing out, macaques and humans received the same number of food pellets or task points as coins shown. Macaques and humans, respectively, received a 20s and 8s pause after each cashout before receiving their pellets or points, to prevent their cashing out after every correct response. These differential delays were based on our long experience with building comparable uncertainty-monitoring paradigms for humans and macaques. During the delay humans saw the instruction “Wait to Cash out”, and monkeys heard a whoop to signal that they had successfully cashed out and their rewards would be forthcoming. After the cashout delay ended, monkeys and humans completed the same trial they had just cashed out. If an incorrect response was made following a cashed-out or incorrect trial, no coins were added to the already empty coin box.

For monkeys, 10% of trials were randomly selected to be forced-cashout trials and 10% were randomly selected to be forced-discrimination trials in which the monkey could not cash out. Respectively, these trials helped monkeys learn the utility of the cashout response to manage risk and learn the utility of saving coins up before cashing out (see Supplemental Materials for analyses of the forced non-cashout trials). There were no forced trials for humans because they were given verbal instructions (see Supplemental Materials for verbatim instructions). These instructions were an integral part of this study because they let us study humans’ explicit, deliberate criterion setting in this paradigm. Thus, humans became the comparative standard to which the performance of macaques could be compared, and an assessment made of how closely macaques come to reaching that standard. Human and animal participants could increase their overall rewards if they managed risk successfully by monitoring difficulty, cashed out adaptively in advance of attempting difficult trials, and cashed out more liberally (including less difficult trials) as momentary risk (i.e., the number of coins at stake) increased.

The monkeys Murph and Lou were trained as follows. At the beginning of a training session, for 50 trials, a correct response produced a single gold coin in the coin box and the pixel rectangle S and D disappeared. The monkey was required to cashout to receive the reward at that point. Then, for 100 trials, the monkey was required to accumulate two gold coins and then cash out. This process continued, with 150 trials accumulating 3 coins before cashing out, 200 trials accumulating 4 coins before cashing out, and so forth. Monkeys moved on to the test trials when they were clearly discriminating sparse and dense trials and showed gradients of poorer performance moving in toward the breakpoint of the discrimination.

After starting testing, we made one adjustment to Murph’s task, by changing the color of the pixels in his pixel box from white to light magenta. This was done to more strongly differentiate for him the pixel box (formerly in white) from the gold-coin box (in yellow). Only the data from the light-magenta version of Murph’s program are reported here.

Analysis and Predictions

We instantiated a formal model of the cashout task—based on Signal Detection Theory (Green & Swets, 1966; Macmillan & Creelman, 2005)—to illustrate for readers the experiments’ predictions and to support optimality analyses of humans’ and macaques’ performances. Our model assumed that performance was organized along an ordered series (a continuum) of psychological representations of increasing intensity (i.e., with subjective impressions of density running from clearly sparse to clearly dense). Given this continuum, the model assumed that the same objective stimulus creates subjective impressions from trial to trial that vary in a Gaussian distribution around the objective stimulus level presented. This perceptual error (the standard deviation of the Gaussian distribution) is what produces sparse-dense discrimination errors and fosters difficulty and possibly uncertainty for trials near the discrimination breakpoint. Finally, the model assumed a decisional process by which criterion lines are placed along the continuum to define response regions. Here, by the overlay on the stimulus continuum of a sparse–cashout criterion (SC)—separating the leftmost and central response regions—and a cashout–dense criterion (CD)—separating the central and rightmost response regions—the continuum would be divided up into sparse, cashout, and dense response regions from left to right.

Our models took the form of a simulated cashout task as macaques or humans experienced it. We placed simulated observers in those task environments for 500,000 trials. We varied the placement of the SC and CD criterion points the simulated participant defended at each level of risk (i.e., at each coin level). As an illustration, we assumed that when the participant had 1 to 7 coins at risk, the SC and CD criteria chosen were 20–23 (a very narrow region of the continuum producing few cashout responses), 19–24, 18–25, 17–26, 16–27, 15–28, and 14–29 (a broad region—nearly half of the stimulus continuum—producing many cashout responses). In this way, we modeled the result if an organism progressively widened its cashout-response region to include less difficult trials as the cost of error increased.

On each trial, given some stimulus (Level 1–42), simulated observers misperceived the stimulus obedient to their perceptual error. This misperceived level became the subjective impression on which the simulated observer based a response choice. As the subjective impression was below SC, above CD, or between the two, the simulated participants chose a sparse, dense, or cashout response. We assumed that for more coins at risk, participants would spread their criteria wider so that a broader range of trials would prompt cashout responses to insure more generously against a discrimination error that would forfeit more accumulated tokens.

Figure 2 shows the effect of the spreading criteria (i.e., the broadening cashout-response region) through the performance of a simulated participant. In this and in Figures 3–5, Bin Levels 1, 2, 3, respectively, summarize performance on Trial Levels 1–14 (Sparse trials varying from very easy to moderately difficult), 15–28 (Sparse and Dense trials varying from moderately difficult to very difficult), and 29–42 (Dense trials varying from very easy to moderately difficult). The peaked curves show the proportion of all trials at each Bin Level receiving the cashout response. The troughed curves show the proportion of correct responses (for all trials completed directly without cashing out). In both cases, progressively darker lines depict increasingly risky situations in which there were more coins/rewards at stake. A companion figure in the Supplemental Materials (Figure SM1) shows the same analysis graphed across 42 stimulus levels separately. The rising and steepening cashout-response curves with greater risk—against a backdrop of discrimination performance that falls for the difficult discrimination trials of Bin 2—were the primary phenomenon under investigation.

Figure 2.

Results from a simulation that illustrates the cashout experiment’s predictions. Bin 1 includes trial levels 1–14 (sparser trials); Bin 2 includes trial levels 15–28 (sparse and dense trials surrounding the discrimination boundary); Bin 3 includes trial levels 29–42 (denser trials). The peaked curves show the proportion of all trials at each Bin Level receiving the cashout response. The troughed curves show the proportion of correct responses (for all trials completed directly without cashing out). For both cashout responses and proportion correct, progressively darker lines depict increasingly risky situations in which there were more coins/rewards at stake.

Figure 3.

Performance in the cashout task by macaques Lou (A) and Murph (B), depicted as in Figure 2.

Figure 5.

Performance in the cashout task by humans, depicted as in Figure 2.

We emphasize that our model lets us describe abstractly—not psychologically--the character of performance in the cashout task. The model is solely mathematical. It does not imply any kind of information processing. If humans or animals showed a pattern like that in Figure 2, one would still not know exactly the processes by which their cashout-response regions widened and their risk criteria spread.

However, this pattern would show a flexible decision-making process in which difficulty was assessed trial by trial, and measured against momentary risk, so that response criteria could be placed along the continuum on each trial appropriately to the current risk conditions (e.g., on one trial, SC and CD criteria of 14–29; and on the very next trial SC and CD criteria of 20–23). In effect, the subject would maintain multiple pairs of criterion lines simultaneously, and use them flexibly and adaptively trial by trial. It would be an interesting decision-making performance raising theoretical questions to which we return.

Results

Monkey Lou

Lou completed 31,360 testing trials in the experiment over 12 sessions. Our animals are free to work or not work at a task during a session, or to work at any pace, and as a result the trials per session and the total testing trials differ somewhat between the two animals. Figure 3A shows Lou’s results, binned as described above. Table S3 in the Supplemental Materials supplies measures of dispersion for the data points in this figure. Lou showed Figure 2’s data pattern by responding appropriately to dynamic risk. With more food tokens at stake, his cashout-response curves rose and grew more peaked. Figure 2 showed that this pattern is predicted if Lou progressively widened his cashout-response region as risk increased, increasing his overall cashout responses but also spreading them more broadly across the stimulus continuum. The formal modeling considered shortly also documents this widening.

To statistically analyze Lou’s cashout performance, we calculated his proportion of cashout responses at each of the 42 stimulus levels with each of the 7 possible coin levels. These proportions were then entered into a 3 × 7 mixed items general linear model (GLM) with bin level (1–3) as the between-items factor and coin number (1–7) as the within-item factor. The 42 stimulus levels (14 within each bin) created the error variance in this analysis. The significant main effects of bin level, F (2, 273) = 42.81, h_p² = .24, p < .001, and number of coins, F (6, 273) = 173.20, h_p² = .79, p < .001 were qualified by a significant interaction between those variables, F (12, 273) = 6.40, h_p² = .22, p < .001. A simple main effects analysis revealed no significant main effect of bin level for 1 to 3 coins. But there were significant effects of bin level for 4 to 7 coins. Table S1 in the Supplemental Materials contains the details of these analyses. For 5 to 7 coins, pairwise comparisons revealed that the proportion of cashout responses in Bin Level 2 (the middle difficult region) was significantly higher than Bin Level 1 (the sparse region) or Bin Level 3 (the dense region). For 4 coins, Bin Level 2 was significantly higher than Bin Level 1 but not Bin Level 3.

Formal modeling shed an additional light on Lou’s performance. By sampling ranges of SC and CD in the SDT model, we used the model already described to fit Lou’s data when he had different numbers of coins at risk. For each pair of criteria, we found the predicted response proportions and compared them to Lou’s by computing the sum of the squared deviations (SSD) between the predicted and observed performance. We minimized this SSD fit measure to find the best-fitting criteria. The model estimated that for 1 to 7 coins at risk, Lou’s SC and CD criteria were placed at 20–24, 21–25, 21–25, 21–27, 17–31, 12–33, and 9–35. The model reinforced the statistical analyses in suggesting that Lou widened his cashout-response region as the stakes grew higher and more accumulated rewards were forfeit through loss. Our modeling also showed that Lou’s decision strategy with 1 coin at risk could have achieved 83% accuracy, because he accepted even quite difficult trials that caused him a lot of errors; whereas his strategy with 7 coins at risk could have achieved 97% accuracy, because he did not respond to the trial directly unless it was really quite easy. This illustrates in another way the adaptive tradeoff Lou made toward performance accuracy when circumstances warranted.

Monkey Murph

Murph completed 37,176 testing trials in the experiment over 27 sessions. Figure 3B shows his results (corresponding measures of dispersion are reported in Table S3, Supplemental Materials). Given more accumulated rewards, Murph’s cashout-response curves also rose and steepened, consistent with a broadening cashout-response region. We statistically analyzed Murph’s cashout performance by calculating his proportion of cashout responses at each of the 42 stimulus levels with each of the 7 possible coin levels. These proportions were then entered into a 3 × 7 mixed items GLM with bin level (1–3) as the between-items factor and coin number (1–7) as the within-item factor. Again, the 42 density levels (14 within each bin) created the error variance in this analysis. The main effects of bin level, F (2, 273) = 49.87, h_p² = .27, p < .001, and number of coins, F (6, 273) = 569.08, h_p² = .93, p < .001, were again qualified by a significant interaction between those variables, F (12, 273) = 4.62, h_p² = .17, p < .001. A simple main effects analysis revealed that there were no significant main effects of bin level for 1 to 3 coins. There were significant effects of bin level for 4 to 7 coins (Table S1). For 4 to 7 accumulated coins, the proportion of cashout responses in Bin 2 (the middle region) was significantly higher than Bin 1 (the sparse region). For 5 coins, the proportion of cashout responses in Bin 2 was also significantly higher than Bin 3 (the dense region).

Confirming Murph’s widening cashout-response region, the SDT model estimated that for 1 to 7 coins at risk, Murph’s SC and CD criteria were placed at 20–22, 20–23, 20–24, 19–28, 11–38, 1–42, and 1–42. Our modeling also showed that Murph’s decision strategy with 1 coin at risk could have achieved 81% accuracy, once again because he accepted many difficult trials when the risk conditions were low; whereas his strategy with 7 coins at risk could have achieved 99% accuracy, because in that case he accepted only the easiest discrimination trials that he would very likely answer correctly. This illustrates that Murph made the same transition that Lou did toward performance accuracy when accuracy counted.¹

It is an interesting aspect of the results that Murph’s and Lou’s uncertainty regions were somewhat asymmetrical about the true breakpoint of the discrimination at Level 21.5. One sees this in the criteria established by the modeling of both macaques’ data that are asymmetrical about the true discrimination breakpoint at trial level 21.5. One can also see this in the reduced cashout responding for Bin 1 compared to Bin 3. The task’s true contingencies are fully symmetrical about the task’s objective discrimination breakpoint. The animals’ reinforcement histories are as well. Their response curves should be, too. In fact, all the formal associative models in this area predict symmetry and fail to explain asymmetries when they occur (Smith et al., 2008; 2014b). Thus, asymmetry suggests that the animals’ processing is not strictly associative, and rather that they are setting their response criteria using some subjective decisional process that is not fully yoked to the reinforcement contingencies and reward histories in the task. For this reason, asymmetries have been an important animal-metacognition phenomenon (discussion in Couchman, Coutinho, Beran & Smith, 2010; Le Pelley, 2012; Smith et al., 2008, 2014b).

The SDT model also allowed optimality analyses of the macaques’ performances, because we could incorporate into the model the task’s temporal properties as macaques experienced them and estimate the rewards per minute that different response strategies could produce. The model estimated that for 1 to 7 coins at risk, the optimal SC and CD decision criteria were placed at 21–22 (0 cashout responses), 21–22, 21–22, 17–26, 12–31, 10–33, and 7–36. That is, one should keep accumulating tokens by not cashing out when there are only 1–3 tokens at risk, but from that point the width of the cashout-response region should be widened with each additional coin at risk to let one protect against a widening range of less difficult trial levels that could potentially cause a costly error. Figure 4 shows this optimal strategy. Both macaques (compare Figure 3A,B) used evaluations of trial difficulty and criterion shifts to interleave many different decisional strategies for different risk conditions into an overall performance that was efficient and nearly optimal.

Figure 4.

Optimal performance in the cashout task, including optimal levels of cashout responding, depicted as in Figure 2.

Humans

The 41 humans completed 32,263 trials in total, an average of about 787 trials per participant. Humans’ trial counts were slightly variable because they were determined based on their participation rate during a 1-hr session. Figure 5 shows their results (corresponding measures of dispersion are reported in Table S3, Supplemental Materials). Humans’ cashout-response curves also rose and steepened as a function of trial difficulty and the prevailing risk conditions. Because we had multiple humans (41) who each experienced fewer trials than Murph or Lou, we analyzed the proportion of cashout responses in two different ways. First (Items Analysis), to parallel the items analysis used with the monkeys we determined the average cashout responses across participants at each of the 42 density levels with each of the 7 possible numbers of coins. These proportions were then entered into a 3 × 7 mixed items GLM with bin level (1–3) as the between-items factor and coin number (1–7) as the within item factor. As for the monkeys, the 42 levels (14 within each bin) created the error variance in this analysis. This analysis collapsed data across participants to stabilize the data for an items analysis (i.e., by reducing missing and dichotomous values produced when no trials or just one trial occurred at a particular bin-level—density-level combination). Second (Subjects Analysis), to provide a statistical analysis with participant differences creating the error variance, we calculated the overall average proportion of cashout responses in each bin level (1–3) at each number of coins (1–7) for each person, and conducted a 3 × 7 within-participants GLM with participants creating the error variance. In both analyses, the significant main effects of bin level, F_{Items Analysis} (2, 273) = 212.39, h_p² = .61, p < .001, F_{Subjects Analysis} (2, 840) = 136.06, h_p² = .24, p < .001 and number of coins, F_{Items Analysis} (6, 273) = 82.78, h_p² = .65, p < .001, F_{Subjects Analysis} (6, 840) = 63.71, h_p² = .31, p < .001, were qualified by significant interactions between those variables, F_{Items Analysis} (12, 273) = 23.50, h_p² = .51, p < .001, F_{Subjects Analysis} (12, 840) = 15.59, h_p² = .18, p < .001. Simple main effects analyses revealed that there were no significant main effects of bin level for 1 or 2 coins by item or participant (All F’s < 2). However, there were significant effects of bin level for 3–7 coins by item (Table S1) and participant (Table S2, Supplemental Materials). Bin Level 2 was significantly higher than Bin Level 1 and Bin Level 3 at all coin levels with significant main effects. Confirming that the data showed once again the widening of the cashout-response region, the SDT model estimated that for 1 to 7 coins at risk, humans placed their SC and CD criteria at 21–22, 20–22, 20–23, 20–23, 18–24, 18–24, and 17–26.

Figure 6 shows for each human participant (gray bars) the percentage of trials on which a cashout response occurred. There were substantial individual differences among humans. However, this variability is less than found for use of the typical uncertainty response (Smith et al., 2006). This may be because the failure to cash out can have a tangible, large cost through the loss of the entire reward cache. That prospect may create a more consistent use of the cashout response across participants. For comparison’s sake, the percentages of cashout responding for Lou and Murph are shown as well (black bars). The macaques lay squarely in the decisional space of the humans, another indication of continuities across species in the performance of the cashout task. Lou and Murph also used the cashout response more similarly to one another than they have sometimes used the typical uncertainty response (e.g., Smith et al., 2006).

Figure 6.

The overall percentage of cashout responses by 41 human participants and macaques Lou (L) and Murph (M).

However, comparing Figures 3 and 5 (peaked response curves), one sees a notable human-monkey difference. At the highest coin levels, humans used the cashout option qualitatively correctly but quantitatively sparingly. Perhaps this illustrates the common overconfidence effect in studies of human metacognition (Galotti, 2008). Or perhaps it illustrates another phenomenon long reported in studies of uncertainty during psychophysical judgments (Smith et al., 2012b). That is, humans sometimes link uncertainty responses to mental weakness. This was an important reason why the early psychophysicists took uncertainty responses to be psychologically distinctive and elementally metacognitive. Consequently, at the highest coin levels, this linkage may have caused some humans to be less optimal than macaques.

Analyses of Response Accuracy

Figures 3 and 5 (troughed response curves) show that macaques’ and humans’ correct-response proportions were not strongly affected by the number of accumulated food rewards at stake. To statistically test these conclusions, we conducted the same analyses described in earlier sections with proportion of accurate responses rather than proportion of cashout responses as the dependent variable. There was always a significant main effect of bin level indicating reduced performance for the difficult trials in Bin Level 2, F (2, 273) = 103.37, h_p² = .43, p < .001 for Lou, F (2, 256) = 100.33, h_p² = .44, p < .001 for Murph, and F_{Items Analysis} (2, 273) = 335.17, h_p² = .71, p < .001, F_{Subjects Analysis} (2, 833) = 335.21, h_p² = .45, p < .001 for humans. There was a small but significant effect of coin for the humans in the latter analysis, F_{Subjects Analysis} (6, 833) = 3.66, h_p² = .03, p = .001 (1 coin = .86, 7 coins = .91), but no effect of coin on correct responding in the human item analysis or for the monkeys (F’s < 2). There were also no significant interactions between coin level and bin level (All F’s < 2).

There are different reasons why discrimination accuracy might change with risk level. A perceptual explanation would be that subjects bear down on high-risk trials, shrinking their perceptual error and becoming more sensitive perceivers and categorizers. A decisional explanation would be that fewer trials are included in the accuracy proportions for high-risk conditions, because more trials, and more of the difficult trials, received cashout responses instead of categorization responses. Figures 2 and 4 showed the results of simulations in which the perceptual error was held perfectly constant. Perceptual sharpening was eliminated, so that only the decisional mechanism would produce accuracy improvements with risk. In Figures 3 and 5, human and animal participants generally showed about this much accuracy improvement, in some cases less. We provisionally conclude that participants did not—perhaps they cannot—really become more perceptually acute at higher risk. Dynamic risk in our perceptual situation may not be managed by bearing down perceptually. Instead, dynamic risk may be managed in a decisional way, through changes to one’s decisional strategy. We do not claim that this pattern is general. In other tasks, like foraging or maze memory, animals may be able to improve their behavioral sensitivity by bearing down and exerting cognitive effort (e.g. Roberts & Ilersich, 1989).

Discussion

We gave humans and macaques an uncertainty-monitoring task that featured a dynamic risk environment in which participants faced different risk environments from trial to trial. We asked whether they could flexibly adjust their response strategies—trial by trial—to achieve adequate protection against the momentary cost of discrimination errors. In particular, we asked whether they could widen appropriately the stimulus region that prompted cashout responses, thus more generously protecting themselves as needed against losses. Both species did so.

The cashout response let participants choose when to recover accumulated rewards and when to continue accumulating them. It did not free participants from having to respond to potentially aversive perceptual stimuli, and thus it overcame the influential aversion-avoidance associative description of uncertainty performances (i.e., that the “uncertainty” response is a default avoidance response to perceptual stimuli that are unrewarding or punishing). The cashout response did not remove the trial. A response was always due on every trial. So, participants could not avoid trials that might lead to mistakes by using the cashout response. Instead, the cashout response allowed humans and monkeys to manage risk by taking collected rewards prior to completing the trial, so that the difficult trial could be completed after the coin box had been safely emptied. In doing so, participants had to accept the time cost imposed by the cashout response. These aspects of the cashout paradigm are theoretically important because the aversion-avoidance criticism has sometimes been used to argue for low-level interpretations of uncertainty responses (Smith, 2009). That criticism is not applicable here—the cashout response neither removed the stimulus nor obviated a response. However, participants could increase their overall rewards if they managed risk successfully by monitoring difficulty and cashing out in advance of attempting difficult trials, and cashing out more liberally as momentary risk (i.e., the number of coins at stake) increased. Because the cashout response helps address avoidance explanations of uncertainty responses, it may be a constructive tool within the animal-metacognition literature.

This study also allays concerns about the steady-state nature of uncertainty-monitoring paradigms, which through their constant risk and contingency landscapes might be conditioning non-metacognitive strategies. Here, monkeys advanced the empirical animal-metacognition literature by showing appreciable trial-by-trial flexibility in their decision criteria. By this flexibility, they showed continuity with the decision flexibility of metacognitive humans performing the same task. In related research, Haun, Nawroth, and Call (2011) showed similar flexibility in the response of ape species to a changing risk landscape. Call (2010) also showed that apes adjusted their uncertainty behaviors according to a whole complex of informational sources.

In fact, the dynamic-risk environment presented by the cashout paradigm distances our findings from traditional senses of associative learning and stimulus control. For example, our results are not subject to the interpretation that monkeys were simply drawn more to the cashout response as more gold coins were displayed on the screen. In that case, they would just have cashed out more—at all trial levels—when more rewards were available.

Likewise, purely stimulus-based accounts are insufficient. In the present task, the very same stimulus on different occasions, and even on the very next trial, and even the very same subjective stimulus impression, needed to be, and was, responded to differently by participants depending on the reward cache that was on the line. A stimulus 18, for example, was not a static thing to which a stable behavioral association could attach. Instead, the animal had to consider also the prospects for classifying that stimulus correctly, and the momentary cost of error. Stimuli had to be interpreted in the context of judgments about risk and anticipated rewards.

Another possible associative description would be that Lou and Murph were able to commit to memory an associative register that would contain 882 separate bins of reinforcement history (42 stimulus levels X 7 risk levels X 3 possible responses) that would generate all of their best moves in the task. In fact, associative registers and reinforcement-history bins are the structural essence of all the formal models that have tried to explain animals’ performances in uncertainty tasks associatively (Jozefowiez et al., 2009b,c; Smith et al., 2008; Staddon et al., 2007). However, we believe that this reinforcement-history mechanism is implausible and unparsimonious, and readers may agree. In 150 years of research in human perception and psychophysics, there has never been any indication that humans track and consult huge numbers of independent reinforcement-history bins in choosing among alternative responses. It is unclear why modelers (including ourselves!) would have assumed that monkeys perform in a profoundly different way than humans, and in a way that places far heavier burdens on their cognition and memory.

Or, associative theorists might interpret our results as showing that animals respond to the “associative cue” of difficulty. This attribution is deeply problematic. Difficulty is neither a stimulus nor an associative cue. Just imagine you are looking at a standardized test question and concluding it was difficult. That process would be conscious, explicit, and metacognitive. Difficulty is a derived and second-order cue, based in a form of cognitive monitoring, though not necessarily based in full-fledged metacognition. Calling it associative illustrates an important problem within the current animal-metacognition literature.

The problem is that determined associationism has begun to impede theoretical development within the animal-metacognition literature. Carruthers (2014) made this point strongly. Smith et al.’s (2014a) target article explained why associationism in the animal-metacognition literature has become an empty construct, because it fails to reckon with the cognitive and representational basis of animals’ uncertainty performances. Basile and Hampton (2014) echoed the failures of associative accounts (see also Smith, Couchman, & Beran, 2014b). The current associative interpretations also have a cost because they facilely conclude that performance is associative just because the animal is maximizing rewards, or maximizing expected utility, or minimizing the time between rewards. These descriptions do not attend to the process or representational psychology of the situation, and they do not point in any way to the associative character of a performance.

In fact, there is a plausible and parsimonious way in which the monkeys could have performed the cashout task. One only needs to assume that their behavior was organized trial by trial based on the subjective difficulty of the trial level then present on the screen. They adjusted their behavior—not by cashing out more overall—but by progressively including easier trials in their cashout region when more rewards were at stake. These trials carried less risk but still some risk of error, and that risk became more significant and worth avoiding as the stakes grew. This account is consistent with 150 years of psychophysics research, because it is consistent with everything known about human criterion setting and human confidence judgments (Green & Swets, 1966; Macmillan & Creelman, 2005). This account avoids burdens on memory and reinforcement bookkeeping. This account simplifies task performance in a sense. Animals only need to monitor their uncertainty about the present trial, and determine whether their probability of answering correctly justifies answering to build the reward cache higher without cashing out.

Indeed, it might be theoretically productive to think of humans and monkeys in this task as flexibly adjusting the confidence criteria they use to either accept trials straightaway or to defer doubtful trials until after cashing out. Facing low stakes, one would plausibly complete even difficult trials because the potential loss is reduced. But with 6 or 7 reward tokens accumulated, one would appropriately only accept trials that one was very confident of answering correctly. By using confidence in this way, the cashout region in the task would naturally widen out as the stakes increased, just as observed. In a similar way, Smith, Shields, Allendoerfer, & Washburn (1998) showed that monkeys were able to maintain their error rate perfectly at 90% in a memory-monitoring task, by declining to complete more trials when the memory probes grew more difficult. Smith et al. also concluded that macaques were using an animal version of confidence criteria.

Of course one could model these spreading confidence criteria with increasing risk—in fact, the simulations used in the present article instantiated that model. This model points to another important problem in this area. That is, there is a temptation to conclude that an animal’s performance must be low-level and associative if it can be modeled with parameters and mathematical transformations. In fact, though, the models used in the animal-metacognition literature are purely mathematical. They do not reflect the processes or representations an animal might use. They do not specify the level of attention or awareness at which uncertainty performances unfold. That a performance can be modeled is perfectly neutral as to the appropriate psychological description of that performance. Only the empirical facts of the animal’s performance can potentially illuminate the nature of the underlying processes and representations.

One must also avoid the temptation to reify “risk” as a possible associative cue. Risk in our task is not a stimulus. It is not perceivable. It is not governed by the density level of the pixel box. It is not governed by the number of coins at stake. Risk is a derived cue based on a situational evaluation. It is non-trivial to flexibly and appropriately adjust two decisional criteria based on the difficulty of the present trial, and it likely requires an information–processing description.

The use of confidence criteria in the present article is an important observation to make about animal minds, and it provides a parsimonious information-processing description. Our results suggest the intriguing possibility that one might extend the present paradigm in ways that would let macaques essentially make confidence ratings within psychophysical tasks. We believe that the present paradigm would also be productively included within the set of paradigms preferred by neuroscientists for exploring the brain organization of states of uncertainty and confidence (e.g., Barraclough, Conroy, & Lee, 2004; Daw, O’Doherty, Dayan, Seymour, & Dolan, 2006; Dorris & Glimcher, 2004; Kepecs, Uchida, Zariwala, & Mainen, 2008; Kiani & Shadlen, 2009). Thus one might complement the present discussion of behavioral dynamics with a grounding discussion of neural dynamics.

However, it is not clear that confidence criteria are necessarily to be given the label metacognitive, or whether that qualitative label is really meaningful, though it is conceptually loaded. Metacognition may well not be an all-or-none capacity. Macaques might succeed in our task—showing continuities with human metacognition and showing their version of confidence criteria —without sharing all of metacognition’s subjective experiences. They might not be fully conscious of their confidence. They might not feel like a confident or doubtful self as humans sometimes do. Thus, to be fair, calling a performance metacognitive has its own problems that in some ways resemble the problem of determinedly calling a performance associative. The answer to both questions lies in the details of the information-processing, and the answers may thus be nuanced.

Nonetheless, the flexible use of confidence criteria is one diagnostic feature of human metacognition, and by this demonstration the monkeys continue to show functional parallels to it. The results suggest that a productive empirical avenue is to think systematically about the operating characteristics of human metacognition and ask whether animals’ uncertainty systems have those characteristics, too. Our results do not confirm that macaques have a metacognitive capacity fully identical to that in humans. However, they do further the present goal in animal-metacognition research to show that animals’ uncertainty systems have substantial cognitive sophistication, operating at decisional and even executive cognitive levels. Smith et al. (2013) took a converging approach toward the same conclusion. They found that monkeys’ uncertainty responses, far more than the primary Sparse and Dense responses, were compromised by a working-memory load. This supports the idea that uncertainty-based responses by monkeys deserve executive and attentional information-processing descriptions.

Thus, we see the present results as a constructive step in advancing the theoretical dialog in the animal-metacognition literature by carefully analyzing the empirical continuities between humans’ and animals’ uncertainty systems. These results, and others to come, may bring us closer to the conclusion that some species share with humans a basic capacity for uncertainty monitoring that may have some continuities with human metacognition. They bring us closer to having true animal models of uncertainty monitoring or metacognition that could have important applications in the areas of remediation, education, neuroscience, and medical research. And, they may bring us closer to understanding the evolutionary origins of reflective mind in the primates, and to seeing the beginnings of our own cognitive self-awareness.