When uncertain, does human self-motion decision-making fully utilize complete information?

Abstract

When forced to choose humans often feel uncertain. Investigations of human perceptual decision-making often employ signal detection theory, which assumes that even when uncertain all available information is fully utilized. However, other studies have suggested or assumed that, when uncertain, human subjects guess totally at random, ignoring available information. When uncertain, do humans simply guess totally at random? Or do humans fully utilize complete information? Or does behavior fall between these two extremes yielding “above chance” performance without fully utilizing complete information? While it is often assumed complete information is fully utilized, even when uncertain, to our knowledge this has never been experimentally confirmed. To answer this question, we combined numerical simulations, theoretical analyses, and human studies performed using a self-motion direction-recognition perceptual decision-making task (did I rotate left or right?). Subjects were instructed to make forced-choice binary (left/right) and trinary (left/right/uncertain) decisions when cued following each stimulus. Our results show that humans 1) do not guess at random when uncertain and 2) make binary and trinary decisions equally well. These findings show that humans fully utilize complete information when uncertain for our perceptual decision-making task. This helps unify signal detection theory and other models of forced-choice decision-making which allow for uncertain responses.

NEW & NOTEWORTHY Humans make many perceptual decisions every day. But what if we are uncertain? While many studies assume that humans fully utilize complete information, other studies have suggested and/or assumed that when we're uncertain and forced to decide, information is not fully utilized. While humans tend to perform above chance when uncertain, no earlier study has tested whether available information is fully utilized. Our results show that humans make fully informed decisions even when uncertain.

INTRODUCTION

Perceptual decision-making is often studied using binary forced-choice tasks in which the subject is required to select one of two alternatives (e.g., left vs. right) at the end of a trial. But subjects often report being uncertain and feel they are guessing totally at random when forced to decide. This may be unsurprising since sensory neurons (Goris et al. 2014), including vestibular neurons (Yu et al. 2015) that are critical to the self-motion perception task studied herein, are noisy and respond in a probabilistic manner. While the origins of this neural response variability are complex and are not completely understood (Moreno-Bote et al. 2014), it is reasonable to expect this may lead to uncertainty in the perceptual estimates encoded more centrally.

The standard signal detection model (Green and Swets 1966) has a single decision boundary (e.g., Fig. 1A) and does not allow for uncertain responses. It further assumes that even if subjects were to report being uncertain, they would make a binary decision fully utilizing the complete information available. However, a number of publications (García-Pérez and Alcalá-Quintana 2011, 2013; Morgan et al. 2012) have explicitly included an uncertain response option, and thus require two decision boundaries (Fig. 1B). This type of model suggests the central nervous system does not access and/or fully utilize complete information when uncertain. For example, a recent paper stated that when forced to make a binary decision, “observers are sometimes undecided and must guess at random” (García-Pérez and Alcalá-Quintana 2013). While response time tasks are not a focus of this paper, we briefly note that the original and influential drift-diffusion model (i.e., sequential analysis) includes two decision boundaries (Ratcliff 1978, 2001; Smith 2000). Until the decision variable reaches a boundary no response is made, which is analogous to an uncertainty region (Merfeld et al. 2016).

Fig. 1.

Theoretical frameworks for binary standard signal-detection model, trinary uncertainty model, as well as three hypotheses (H1–H3) for how decisions are made with one or two decision boundaries. A–E show decision variable probability density functions for an example stimulus that is rightward with magnitude 0.5. We assume Gaussian noise, which in our example has no bias and a standard deviation of 1, N(μ = 0, σ = 1), is added to the physical stimulus. This yields a decision variable distribution (A–E), which in our example is N(0.5,1). To mimic García-Pérez and Alcalá-Quintana (2013) the x-axis of the decision variable is “subjective location,” while that for the psychometric curve is “physical location.” For standard binary signal-detection theory (A) there is a single decision boundary (solid vertical line) and the area under the curve to the right of the decision boundary represents the likelihood of a rightward response (in this example, P_R = 0.69, red), while the area to the left is the likelihood of a leftward response (P_L = 1 − P_R = 0.31, blue). These probabilities are shown in F at the physical stimulus location of 0.5. Assuming the sensory noise is independent of stimulus, the psychometric curves as functions of physical location are cumulative Gaussians (F). In the uncertainty (trinary) decision model (B and G) there are two decision boundaries, separated by 2δ (δ = 1.2 in this example). Now there are three outcomes: P_L = 0.04, P_R = 0.24, and the probability of the subject responding “uncertain” P_U = 0.72 (green) for our example. We pose three hypotheses (see text for details) for how subjects utilize decision variables and boundaries when forced to make binary and trinary decisions (C–E and H–J). For hypothesis 1 (H1) (C), when uncertain (P_U = 0.72) the probability of a left binary response (P_L&U = 0.36, cyan) is equal to a right response (P_R&U = 0.36, magenta), independent of stimulus. This yields overlaid left and uncertain (L&U) and right and uncertain (R&U) psychometric curves (H). For H2, when uncertain a second binary decision variable is sampled [inset of D, we assume it is similarly distributed, N(0.5,1), but only has a net area of P_U = 0.72], yielding P_L&U = 0.22 and P_R&U = 0.50, in our example. This results in L&U and R&U psychometric curves that are offset such that uncertain trinary responses for leftward (rightward) physical stimuli are more likely to yield a left (right) binary response (I). For H3 (E), the probability of L&U is the area between the left trinary decision boundary and binary decision boundary (P_L&U = 0.26), while P_R&U = P_U − P_L&U = 0.46, in our example. This similarly creates an offset in the L&U and R&U psychometric curves (J), but less so than in H2.

We aim to investigate how binary forced-choice responses are made when a subject reports being uncertain. In doing so, we hope to assess the compatibility of the trinary uncertainty model with the standard signal detection (i.e., binary) model. Historically, it has been assumed that even when uncertain, subjects fully utilize complete information in making a binary forced-choice decision (e.g., Green and Swets 1966; Macmillan and Creelman 2005). However, to our knowledge, this has not been experimentally verified. Several studies have demonstrated humans tend to perform at better-than-chance levels even when uncertain (Adams 1957; Peirce and Jastrow 1885), but they did not test whether these mechanisms fully utilize complete information. In fact, some recent thinking (Beck et al. 2012) suggests decision-making includes suboptimal inference (i.e., a lack of fully utilizing complete information), which may apply to uncertain decisions.

We develop and use a procedure that defines a model fully using complete information when uncertain, as well as two other models that define responses when uncertain—a “guessing” model and a resampling model—and determine which of these models fits our data best. We compare empiric data obtained using a passive, self-motion direction-recognition task with model predictions for responses when uncertain. While further detailed below under Theoretic hypotheses, each of these models makes specific, quantitative predictions regarding the subject’s threshold—a standard measure of the smallest stimuli that can be perceived reliably—when using tasks that allow uncertainty vs. forced-choice tasks that do not.

We investigated three hypotheses (H1, H2, and H3) that posit how a “forced-choice” binary decision is made when uncertain and accompany each with a formal theoretical framework to make quantitative predictions. Hypothesis 1 (H1) posits that decision-making fundamentally includes an “uncertainty” region, in which subjects will be forced to simply guess at random (e.g., a 50/50 guess) (Fig. 1C) (García-Pérez and Alcalá-Quintana 2011, 2013).

Hypothesis 2 (H2) posits that when uncertain, humans have information that leads to better than chance performance. One source for such information might be that when uncertain, a second independent decision variable is sampled to produce the binary decision when forced to choose (Fig. 1D). While the mechanism of this hypothesis remains less clear, it provides a second quantitative performance comparison, which, as we will show (Figs. 5A and 6), opposes H1 in its predicted impact on thresholds. Both H1 and H2 represent mechanisms for making uncertain decisions that fail to fully utilize complete information.

Fig. 5.

Comparison of binary and trinary fits. In each panel, shapes (circle, square, diamond, triangle) correspond to each of the four subjects (S1–S4, respectively). The 95% confidence intervals for the parameter estimates for each subject were estimated using a delete-one jackknife procedure (Quenouille 1956; Tukey 1958). A shows the ratio of binary (${\widehat{\sigma }}_B$) over trinary (${\widehat{\sigma }}_T$) threshold estimates. The curves show the mean simulated response pattern for each of the three hypotheses (see methods and materials for details on simulations). Each subject’s data are consistent with H3. Inset table in A shows sum of square error (SSE) between each data set and the mean simulation data for each hypothesis. In B and C, shading corresponds to each response task (black = 2-stage binary/uncertain task, gray = 2-stage trinary/binary task, and white = 1-stage binary/uncertain task). B shows the threshold estimates plotting ${\widehat{\sigma }}_B$ vs. ${\widehat{\sigma }}_T$ for each subject on each task, along with the unity line (theoretical prediction of H3). C displays the “subjective bias” estimates plotting ${\widehat{\mu }}_B$ vs. ${\widehat{\mu }}_T$ for each subject on each task, along with the unity line. Subjects with moderate subjective biases maintain these biases across tasks, suggesting the binary decision boundary is centered between the two trinary decision bounds.

Fig. 6.

Ratio of binary threshold to trinary threshold in experiment 2. A shows the mean simulations now matching the 100 trials and sampling method of experiment 2. Squares show 98 example simulations for H3. B shows the 98 experimental subjects overlaid on the mean simulations. The four subjects from experiment 1 that also participated in experiment 2 (S1–S4) are highlighted (circle, square, diamond, triangle, respectively). The empirical data are most consistent with H3, in terms of sum of square error (SSE) between ${\widehat{\sigma }}_B\hspace{0.17em}/{\widehat{\sigma }}_T$for each data set and the mean simulated response curves for each of the hypotheses (see inset tables in B). The $\widehat{\delta }\hspace{0.17em}/{\widehat{\sigma }}_T$ ratios for four subjects are greater than 1.5 and are excluded from the plot but included in the SSE analysis.

Hypothesis 3 (H3) posits humans have complete access to the pertinent decision variable when making decisions, even when uncertain (Fig. 1E). As will be shown (Figs. 5A and 6), this leads to equally low thresholds when fitting with binary vs. trinary/uncertain models to data from the associated tasks. This intuitively can be explained because even when uncertain and forced to make a binary, forced-choice decision, H3 assumes complete information is fully utilized. If information is not fully utilized when uncertain, the fitted binary threshold will be either greater or less than the fitted trinary threshold depending on how that information is utilized (H1 and H2, respectively. See Figs. 5, A and B, and 6. Finally, each hypothesis yields a different distribution of Left&Uncertain (L&U) vs. Right&Uncertain (R&U) responses in the psychometric functions (Fig. 1, H–J).

While a limitless number of alternative hypothesis and variants of hypotheses could be proposed, H1–H3 provide behavioral performance bounds and specific hypotheses for neural decision variable behavior. Furthermore, our theoretical framework provides quantitative relationships between a neural decision variable and behavioral responses that do or do not fully utilize complete information. These mechanistic hypotheses lay the groundwork for future investigations to identify the specific neural populations underlying inference when uncertain.

METHODS AND MATERIALS

Overview.

To investigate the extent to which subjects fully utilized complete information even when uncertain, we performed theoretical analyses (e.g., Fig. 1), human studies (e.g., Figs. 3–6), and numerical simulations (e.g., Figs. 2, 5A, and 6). We performed a standard vestibular one-interval direction-recognition task (Chaudhuri et al. 2013) in which humans were rotated in yaw to their left or to their right (Grabherr et al. 2008; Roditi and Crane 2012; Soyka et al. 2012; Valko et al. 2012). After the motion, subjects indicated their perceived motion direction and their uncertainty. For each trial, our task required subjects provide a binary (left or right) response as well as a trinary response (Fechner 1966; Kaernbach 2001) that added an “uncertain” option.

Fig. 3.

Binary and trinary psychometric curves and response probability data in four subjects (experiment 1) using the two-stage binary-uncertain (primary) task. Each column shows one subject (S1–S4, respectively). The top row (A–D) shows the binary (signal detection) model data and fits while the bottom row (E–H) shows the trinary (uncertain) model data and fits. Responses are grouped by stimulus (~200 responses per stimulus grouping) and shown as data points. The proportion rightward is shown in red, the proportion leftward is in blue, and the proportion uncertain (in the trinary fits) is in green. The best fit parameters, each in degrees per second, are provided in each panel. For each subject, ${\widehat{\sigma }}_B\approx {\widehat{\sigma }}_T$, consistent with hypothesis 3, and ${\widehat{\mu }}_B\approx {\widehat{\mu }}_T$ such that the two trinary decision boundaries are symmetric about the single binary decision boundary. The uncertain region width (δ) is set somewhat arbitrarily by each subject and thus may vary between subjects (δ for S4 $\ll$ δ for S1–S3).

Fig. 2.

Example parametric bootstrap cross-fitting method (PBCM) analysis comparing statistical deviance of four-alternative model fits. The example shows the analysis for the pairwise comparison of H1 and H3 for subject S1. As detailed elsewhere (Wagenmakers et al. 2004), distributions of differences in the statistical deviance (Dev; see methods and materials) assuming H1 (Dev*_H1−Dev*_H3|H1, light gray) and assuming H3 (Dev*_H1−Dev*_H3|H3, dark gray) are formed using a parametric bootstrap with M = 1,000. Histograms are overlaid with a Gaussian kernel smoothing estimate. The statistical boundary between H1 and H3 is shown as the dotted vertical line, which may not necessarily be at 0 if one of the models is better at explaining data bootstrapped assuming the other model. The solid vertical line shows the difference in deviance observed in fits to empirical data. The solid line being to the right of the dotted line corresponds to H3 being a preferable model fit as compared with H1. For this example, the likelihood ratio (calculated as the ratio of the H3 distribution to that of the H1 distribution at the empirical difference in deviance) is 6×10⁴³, indicating H3 is by far the more likely model fit. The solid line being well within the bootstrapped distribution assuming H3 indicates a very good fit (P = 0.78 for a standard two-tailed test).

To analyze these data, we utilized a standard signal detection analysis (Chaudhuri and Merfeld 2013; Green and Swets 1966; Merfeld 2011) that includes one decision boundary as well as a recently developed uncertainty analysis (García-Pérez and Alcalá-Quintana 2011, 2013) that includes two decision boundaries. The uncertainty analyses (or “indecision” model) has recently been applied to animal studies (Sridharan et al. 2014) and to explain otherwise discrepant human results by showing that common timing processes contribute to simultaneity and temporal-order judgments (Garcia-Perez and Alcala-Quintana 2012, 2015; García-Pérez and Peli 2014).

As one metric, we quantify perceptual decision-making “performance” using psychometric curve fits; specifically, the standard deviation of a cumulative Gaussian fit, which in our task corresponds to the “threshold” at which the magnitude the stimuli is indistinguishable from perceptual noise. When information is fully utilized to make decisions when uncertain (H3), an individual’s threshold is predicted to be equally low for tasks with and without an uncertainty option. Figure 1, A and B shows analytic decision variable distributions and boundaries for signal detection and uncertainty models, respectively; Fig. 1, F and G shows the respective psychometric functions (i.e., the relationship between the physical stimuli and the subject responses).

Theoretic hypotheses.

As introduced earlier, we propose three hypotheses (H1–H3) for how the forced binary decision is made when a subject is uncertain (Fig. 1). H1 posits that humans simply guess (García-Pérez and Alcala-Quintana 2011, 2013), H2 suggests that a second decision variable is utilized, and H3 posits that available information is fully utilized. We note that each hypothesis shown in Fig. 1 represents a family of hypotheses. For H1, a subject could guess left more often than right (or vice versa); in Fig. 1H, this would vertically scale the cyan curve up and the magenta curve down, but not shift them left or right. For H2–H3, the binary decision boundary could be located closer to one of the two trinary decision boundaries than the other—again causing only vertical shifts in the cyan and magenta curves. We will show later (Figs. 4 and 5C) that our data are inconsistent with these alternatives.

Fig. 4.

Psychometric curves and response data for each of the four subjects (experiment 1). Each panel (A–D) shows one subject (S1–S4, respectively). Responses are grouped by stimulus (~200 responses per stimulus grouping) and shown as data points. The proportion of left (blue, P_L), right (red, P_R), or uncertain (green, P_U) responses are shown as in García-Pérez and Alcalá-Quintana (2013). Note that P_L + P_U + P_R = 1. In addition the proportion of left and uncertain (cyan, P_L&U) and right and uncertain (magenta, P_R&U) are presented. Note that P_L&U + P_R&U = P_U. For S1–S3, the L&U and R&U probability distributions are offset such that uncertain trinary responses for leftward (rightward) physical stimuli are more likely to yield a left (right) binary response. This is inconsistent with H1 and indicates that human subjects are not just guessing on the forced-choice binary response when uncertain for the trinary response (contrast with theory for H1 in Fig. 1H). Building on Garcia-Perez’s approach (2013), we fit the four-alternative response (L, L&U, R&U, R) data trials for each subject with models corresponding to each of the three hypotheses. For each subject, the best fit, in terms of lower statistical deviance (Dev; see inset tables; see Four-alternative models for details), is the model for H3. (S4’s data, which showed few uncertain trials, were fit equally well by H1 and H3.) The psychometric curve fits for the H3 model are shown as the overlaid lines (blue, green, red, cyan, and magenta for L, U, R, L&U, and R&U, respectively).

Procedural details.

Subjects were informed that they should experience each stimulus trajectory in its entirety before making a decision. In fact, subjects were alerted when the stimulus began and ended, and no response could be entered before the stimulus ended. Each motion lasted 1 s (i.e., 1 Hz) and was verified to have peak velocities within ±2% of the commanded peak velocity (Nesti et al. 2014). As in our previous studies (Valko et al. 2012), the motion profiles were single cycle sinusoids of yaw angular acceleration. The smooth, single cycle sinusoids of acceleration motions, each only 1 s in duration, encouraged subjects to make their decision after the stimulus ended. Since it is known that bounded integration can lead to decisions even when stimulus duration is controlled by the environment/operator (Kiani et al. 2008), we specifically targeted stimulus levels that we knew would yield indecision on many trials. Specifically, the stimuli magnitude was modulated using a three-down-one-up adaptive staircase procedure (Karmali et al. 2016) with each staircase beginning at 4 degrees per second for the peak angular velocity. The staircase specifically targeted stimuli magnitudes near threshold at which the subject may be uncertain. The direction (i.e., left vs. right) was determined randomly.

Furthermore, we chose to avoid reaction time tasks, which require a decision be made as soon as one of two decision boundaries is crossed, because 1) we wanted to minimize any potential contributions of reaction-time to confidence (Fetsch et al. 2014; Kiani et al. 2014) and 2) we wanted a sample of trials when the subject remained uncertain. That three of the four subjects in experiment 1 (see Subjects below) indicated that they were uncertain on ~20% of trials demonstrates that our paradigm successfully achieved this latter goal.

We used three response tasks to assay human perception that were new to these experiments, so we provide details here. Each of these three tasks allowed us to collect a “binary” response (i.e., left vs. right) as well as a “trinary” response (i.e., left vs. right vs. uncertain) for each motion stimuli. This was done using a custom iPad interface with backlighting turned off, to maintain total darkness. The iPad was fixed to the motion device to avoid its inertia providing an additional cue. White noise was used to mask auditory cues and indicate the beginning and end of each motion stimuli.

In our primary task, labeled the “2-stage binary/uncertainty” task, subjects were given the following instructions: “Each trial has just one motion, either a yaw rotation to your right or a yaw rotation to your left. There will always be a motion, even if it is very small. You will use the iPad to record your responses, in two phases. First, if you feel that you have rotated to your right, please press the right side of the screen with your right thumb. If you feel that you have rotated to your left, press the left side of the screen with your left thumb. Even if you are not sure of the direction, you must choose and press only one button; just make your best guess.” The iPad confirmed the response using a computer-generated voice (“left” or “right”). This was followed by an indication of whether they were uncertain or not uncertain. Specifically the subjects were instructed, “After you have made this first selection, you will indicate your confidence. If you are uncertain, press both the right and left sides of the screen at the same time. Otherwise, tap the same side of the screen again to confirm your selection.” The iPad confirmed this second entry using a computer-generated voice (“guess” or “not a guess”). Subjects were instructed to inform the operator whenever they made an error entering their response, which allowed the operator to make a manual correction. Such entry errors were rare (<0.1% of trials).

As control tasks, we also assayed perception using 1) a “2-stage trinary/binary” task, and 2) a “1-stage binary/uncertain” task. In the “2-stage trinary/binary” control task, subjects first provided a trinary response (left, right, or uncertain) by pressing the left side of the screen, right side of the screen, or both sides of the screen at the same time (uncertain). Computer-generated voice feedback was provided (“left” or “right” or “guess”). This was followed by a forced binary response if they first reported being uncertain (left or right reported by tapping that side of the screen). Again voice feedback was provided (“left” or “right”).

Finally, in our “1-stage binary/uncertainty” control task, each subject provided their indication by tapping one of four virtual “buttons” in the dark. To indicate perceived motion to the right (left) and that they were uncertain, the subject was instructed to push the lower right (left) button. The computer-generated voice confirmed the entry by saying “right, guess” (or “left, guess”). To indicate perceived motion to the right (left) and that they were not uncertain, the subject was instructed to push the upper right (left) button. The computer-generated voice confirmed this entry by saying “right, not a guess” (or “left, not a guess”). According to Okamoto (2012) the general approach of using four-alternatives with certain/uncertain grading was proposed by Klein (2001).

Subjects.

In experiment 1, ~2,850 trials of the primary “2-stage binary/uncertainty” task were collected in each of four subjects (2 male and 2 female) across 38 test sessions consisting of 75 trials that were performed on 30 separate test days. Data from these test sessions were pooled since no learning or session effect was found, which is consistent with earlier findings (Hartmann et al. 2013). An additional 675 trials (again pooled across multiple sessions of 75 trials) were collected for each of the same four subjects that participated in experiment 1 for each of two control tasks (“2-stage trinary/binary” task and “1-stage binary/uncertain” task). Four of the authors (T. K. Clark, Y. Yi, R. C. Galvan-Garza, and M. C. Bermúdez Rey) formed the subject pool for experiment 1. We emphasize that individual stimuli direction was randomized by the computer so neither subject nor operator had “inside information” regarding motion stimuli. This is emphasized by the fact that the testing of 98 naive subjects (49 male and 49 female) in experiment 2 yielded results consistent with those obtained in experiment 1. Subjects in experiment 2 performed a single session of 100 trials using the primary “2-stage binary/uncertainty” task. The data for experiment 2 have previously been published (Bermúdez Rey et al. 2016) using a different set of analyses focused on threshold variations with age and on correlations between the measured thresholds and balance.

Protocols were approved by the local Internal Review Board and all subjects provided written, informed consent. Study data were collected and managed using REDCap electronic data capture tools hosted at the Massachusetts Eye and Ear Infirmary (Harris et al. 2009). REDCap (Research Electronic Data Capture) is a secure, web-based application designed to support data capture for research studies.

Data analysis.

In effect, for each task, a four-category response (L, L&U, R&U, R) was recorded. These data were fit using both binary and trinary (“uncertainty”) models. The standard signal detection model having a single decision boundary (Chaudhuri and Merfeld 2013; Green and Swets 1966; Merfeld 2011) was fit to the left/right binary responses provided. For the uncertainty analysis, when the subject indicated they were uncertain (e.g., in the second stage of the 2-stage binary/uncertain task), we replaced the binary response provided in the first response stage with the uncertain response provided in the second stage. These trinary data were fitted using the published uncertainty model (García-Pérez and Alcalá-Quintana 2011, 2013) with two decision boundaries. In each case, we report the “one-sigma threshold” in which threshold is equivalent to the standard deviation of the physiological noise (σ) (Merfeld 2011). The parameters are estimated by model fits, denoted by “hat” symbols, such that threshold is equivalent to $\widehat{\sigma }$.

Four-alternative models.

Building on the uncertainty (three-alternative) model, we developed and fit four-alternative models to the L, L&U, R&U, R data, one for each of the three hypotheses (H1–H3) to see which yielded the best fit (i.e., lowest statistical deviance). In each of these four-alternative models, as in García-Pérez and Alcalá-Quintana (2013), the probability of a leftward response (P_L) as a function of stimulus x is

$${\text{P}}_{\text{L}}(x)=1-\phi (x,\mu -\delta ,\sigma )$$ (1)

where φ is a cumulative Gaussian distribution with mean = μ − δ and standard deviation = σ, evaluated at stimulus x, where negative x values correspond to leftward stimuli. Here and for the uncertainty (trinary) model, μ is the center of the two uncertainty decision boundaries that are separated by 2δ (Fig. 1). Furthermore, the probability of the rightward response (P_R) is given by

$${\text{P}}_{\text{R}}(x)=\phi (x,\mu +\delta ,\sigma )$$ (2)

In each of the four-alternative models (H1–H3), the probability of a Left&Uncertain response (P_L&U) and that of a Right&Uncertain response (P_R&U) are modeled explicitly. (For each H1–H3, the probability of being uncertain (P_U), as in the trinary model, is simply the sum of Left&Uncertain and Right&Uncertain probabilities (P_U = P_L&U + P_R&U)].

For H1, in which the subject simply guesses left vs. right when uncertain (e.g., 50/50 guess), these functions are

$${\text{P}}_{\text{L\&U,H1}}(x)={\text{P}}_{\text{U}}(x)/2=[1-{\text{P}}_{\text{L}}(x)-{\text{P}}_{\text{R}}(x)]/2=[\phi (x,\mu -\delta ,\sigma )-(x,\mu +\delta ,\sigma )]/2$$ (3)

$${\text{P}}_{\text{R\&U,H1}}(x)={\text{P}}_{\text{L\&U,H1}}(x)=[\phi (x,\mu -\delta ,\sigma )-\phi (x,\mu +\delta ,\sigma )]/2$$ (4)

When uncertain for H2, another separate decision variable is sampled (assuming the same bias and standard deviation as the original decision variable) using a single decision boundary centered between the two uncertainty decision boundaries. This is mathematically equivalent to using the same decision variable when uncertain, but now applying the binary decision boundary to the entire distribution. This yields the following response curves:

$${\text{P}}_{\text{L\&U,H2}}(x)={\text{P}}_{\text{U}}(x)*[1-\phi (x,\mu ,\sigma )]=[\phi (x,\mu -\delta ,\sigma )-\phi (x,\mu +\delta ,\sigma )]*[1-\phi (x,\mu ,\sigma )]$$ (5)

$${\text{P}}_{\text{R\&U,H2}}(x)={\text{P}}_{\text{U}}(x)*\phi (x,\mu ,\sigma )=[\phi (x,\mu -\delta ,\sigma )-\phi (x,\mu +\delta ,\sigma )]*[\phi (x,\mu ,\sigma )]$$ (6)

Finally, for H3, information from the initial decision variable is utilized, even within the uncertainty region, yielding the following distributions:

$${\text{P}}_{\text{L\&U,H3}}(x)=1-{\text{P}}_{\text{L}}(x)-\phi (x,\mu ,\sigma )=\phi (x,\mu -\delta ,\sigma )-\phi (x,\mu ,\sigma )$$ (7)

$${\text{P}}_{\text{R\&U,H3}}(x)=1-{\text{P}}_{\text{L}}(x)-{\text{P}}_{\text{R}}(x)-{\text{P}}_{\text{L\&U,H3}}(x)=\phi (x,\mu ,\sigma )-\phi (x,\mu +\delta ,\sigma )$$ (8)

Each of the models for the three hypotheses was fit using a maximum likelihood approach with the four-alternative response data (L, L&U, R&U, R). For each fit we calculate the statistical deviance (Dev) as follows:

$$\text{Dev=}-2*\ln (\mathcal{L})$$ (9)

Note that in comparing the models, each model has the same number of free parameters (μ, σ, δ), such that the deviance measure of goodness of fit fully accounts for model complexity (i.e., Bayesian information criteria or another metric that accounts for the number of free parameters would yield identical conclusions as simply using deviance). $\mathcal{L}$ represents the likelihood function, the logarithm of which is defined as follows:

$$\text{ln}\left(\mathcal{L}\right)={\displaystyle \sum }_{i=1}^n\left\{{\text{L}}_i\ln \left[{\text{P}}_{\text{L}}\left(x_i\right)\right]+{\text{L\&U}}_i\ln \left[{\text{P}}_{\text{L\&U}}\left(x_i\right)\right]+{\text{R\&U}}_i\ln \left[P_{R\&U}\left(x_i\right)\right]+{\text{R}}_i\ln \left[{\text{P}}_{\text{R}}\left(x_i\right)\right]\right\}$$ (10)

For each stimulus x_i (i = 1…n), L_i, L&U_i, R&U_i, and R_i are dummy variables, such that the value is 1 for the response which has occurred and 0 for each of the others. For example, if the response on trial 4 was Right&Uncertain, then R&U₄ = 1 and L₄ = L&U₄ = R₄ = 0. The probabilities of each response (e.g., P_R) are defined as above, where P_L&U and P_R&U are different for each of the three hypotheses. The free parameters (μ, σ, δ) were determined that maximize the likelihood ($\mathcal{L}$), or equivalently, that minimize the negative log likelihood (−ln($\mathcal{L}$)].

This formulation provides theoretical frameworks for alternative mechanisms linking a neural decision variable under uncertainty and the expected behavioral performance (i.e., psychometric curves).

Simulations.

We utilized simulations to provide predictions for each hypothesis, which we could then compare with our experimental data. Briefly, to simulate each “trial,” we randomly sample a “decision variable” (Gaussian with µ = stimulus, σ = 1) and compare it to the trinary decision boundaries (defined by the uncertainty region width, δ). If “uncertain,” the binary response is simulated according to each hypothesis. Specifically, for H1, random 50/50 guess; for H2, resample a decision variable (Gaussian with µ = stimulus, σ = 1) and compare with binary decision boundary; and for H3, compare the original decision variable to the binary decision boundary. Each set of simulated trials is fit using standard binary (Chaudhuri and Merfeld 2013) and trinary (García-Pérez and Alcalá-Quintana 2013) methods to yield a binary threshold (${\widehat{\sigma }}_B$), a trinary threshold (${\widehat{\sigma }}_T$), and an uncertainty boundary ($\widehat{\delta }$), which yield two normalized variables ${\widehat{\sigma }}_B\hspace{0.17em}/{\widehat{\sigma }}_T$ and $\widehat{\delta }\hspace{0.17em}/{\widehat{\sigma }}_T$. We simulate a range of underlying δ values (0–2.5), corresponding to each subject potentially setting different uncertainty criteria. The sampling method of the simulations was matched to that used in each experiment. Simulation results (${\widehat{\sigma }}_B\hspace{0.17em}/{\widehat{\sigma }}_T$ and $\widehat{\delta }\hspace{0.17em}/{\widehat{\sigma }}_T$) are binned by increments of 0.05 $\widehat{\delta }\hspace{0.17em}/{\widehat{\sigma }}_T$ and the mean ${\widehat{\sigma }}_B\hspace{0.17em}/{\widehat{\sigma }}_T$ of each bin is calculated (Fig. 5A and Fig. 6). To yield smooth curves 510,000 simulations (10,000 at each of 51 discrete δ values between 0 and 2.5 with steps of 0.05) were performed for Fig. 5A, and 260,000 simulations (10,000 at 26 δ values between 0 and 2.5 with steps of 0.1) were done for Fig. 6.

Parametric bootstrap cross-fitting method.

As one approach to compare our three competing hypotheses, we calculated statistical deviance (Dev) for each model fit associated with each hypothesis (see Four-alternative models). While lower deviance corresponds to a better fit, a model with only slightly lower deviance might not be significantly better than an alternative model. Furthermore, a model whose form has more flexibility to explain a wide range of responses, even with the same number of free parameters, may yield lower deviance when fitting data without actually being a better explanation of the data. This issue can be addressed by assessing “model mimicry,” in which a model has the ability to account for data generated by a competing model.

To assess model mimicry, we apply a parametric bootstrap cross-fitting method (PBCM) (Wagenmakers et al. 2004) using a standard M = 1,000 bootstrapped samples. PBCM compares two competing model hypotheses. Since we have three competing hypotheses, we report pairwise comparisons for H1 vs. H3 and H2 vs. H3. We do not report the remaining pairwise comparison, H1 vs. H2, since H3 will be shown (e.g., Fig. 4) to be far superior to H1 and to H2.

An example of PBCM for one subject (S1) and one comparison (H1 vs. H3) is shown in Fig. 2. The solid vertical line shows the empirically observed difference in deviance between a fit of an H3 model and that of an H1 model (see Four-alternative models). This empirical deviance difference falls to the right of the dotted vertical line, which represents the statistical boundary between distributions of deviance differences for the two hypotheses (i.e., where the two distributions are equal). This shows that the empirical data from subject S1 are more consistent with bootstrapped samples assuming H3 than those assuming H1. To quantify how decisively preferable H3 is, a likelihood ratio (Λ) can be computed as the ratio of the smoothed distribution of H3 to that for H1 at the observed difference in deviance (in this example, Λ = 6 × 10⁴³; also see Table 1). Likelihood ratios greater than 100 or less than 0.01 can be used as cutoffs for a decisively better or worse fits, respectively (Jeffreys 1998).

Table 1.

Likelihood ratios for deviance differences using PBCM model comparisons

Likelihood Ratios	S1	S2	S3	S4
$\frac{\mathcal{L}(D_{H1}-D_{H3}|H_3)}{\mathcal{L}(D_{H1}-D_{H3}|H_1)}$	6 × 1043	2 × 10165	3 × 1064	1.03
$\frac{\mathcal{L}(D_{H2}-D_{H3}|H_3)}{\mathcal{L}(D_{H2}-D_{H3}|H_2)}$	1 × 1012	1 × 1022	6 × 1021	0.17

Ratios greater than 1 indicate a preferable model fit for hypothesis 3 (H3) than either hypothesis 1 (H1; row 1), or hypothesis 2 (H2; row 2); ratios greater than 100 indicate a “decisively” better fit while less than 0.01 corresponds to a significantly worse fit (Jeffreys 1998). The fits for each subject (S1–S4) from experiment 1 are shown. S1–S3 are fit decisively better by the H3 model while there are not enough uncertain responses for subject S4 to determine the best fit. PBCM, parametric bootstrap cross-fitting method.

Furthermore, the solid vertical line is well within the expected distribution under hypothesis H3. We calculated a P value of the observed difference in deviance within a smoothed estimate of the probability distribution assuming H3 (in this example, for a standard 2-sided test P = 0.78). This suggests H3 not only is the better fit of these two alternatives (H1 vs. H3) for this subject but is not significantly different than would be expected under hypothesis H3.

RESULTS

Binary vs. trinary psychometric curves.

The subject responses and psychometric curve fits for experiment 1 using our primary “2-stage trinary/binary” task are shown in Fig. 3 for the four subjects tested repeatedly (2,850 trials per subject). Fig. 3, A–D show the binary data (i.e., Left/Right) and fits for each subject; Fig. 3, E–H show trinary data (i.e., Left/Right/Uncertain) and fits. Using standard conventions appropriate for our direction-recognition task, psychometric function width is parameterized as a threshold (σ) that represents all noise sources (e.g., Merfeld 2011), leftward or rightward psychometric function shift is parameterized as a bias (μ) (e.g., Merfeld 2011), and the trinary uncertainty region is parameterized using δ (e.g., García-Pérez and Alcalá-Quintana 2011, 2013).

For each subject, the trinary data and psychometric curves (Fig. 3, E–H) were qualitatively similar to those observed in previous studies (García-Pérez and Alcalá-Quintana 2011, 2013). As more rigorously analyzed later (e.g., Fig. 5), note that the width of the binary psychometric function (${\widehat{\sigma }}_B$) was about the same as the width of the trinary psychometric function (${\widehat{\sigma }}_T$) (i.e., the thresholds were equally low). The uncertain region width (2δ) is set somewhat arbitrarily by each subject and thus may vary between subjects. Note that the width of the uncertainty region (δ) for S4 was much less than the other subjects. In fact, since S4 seldom indicated being uncertain, there were insufficient trials to visualize the binary response when uncertain (Fig. 4D).

For the other three subjects (S1–S3), there were a sufficient number of uncertain response trials to visualize the corresponding forced-choice binary responses (L&U and R&U psychometric curves, shown in Fig. 4). For S1–S3, the L&U and R&U probability distributions are horizontally offset such that uncertain trinary responses for leftward (rightward) physical stimuli are more likely to yield a left (right) binary response. This is inconsistent with H1 and indicates that human subjects are not just guessing on the forced-choice binary response when uncertain for the trinary response (contrast data in Fig. 4, A–C with theory for H1 in Fig. 1H).

Four-alternative psychometric curve fits.

Building on García-Pérez’s approach (2013), we fit the four-alternative response (L, L&U, R&U, R) data trials for each subject with models corresponding to each of the three hypotheses. For each subject, the best fit, in terms of lower statistical deviance (Dev in inset tables in Fig. 4; see methods and materials for details), is the H3 model. S4’s data, which had few uncertain trials, were fit nearly as well by H1 as by H3.

We further assessed the model fits using a PBCM analysis (Wagenmakers et al. 2004). For the three subjects (S1–S3) who reported that they were uncertain on ~20% of trials, the difference in the deviance observed between the two models was most consistent with the differences in the deviance that would be expected (from bootstrapped samples) assuming H3. Specifically, Table 1 shows the likelihood ratios for each paired comparison. For S1–S3 the likelihood ratio is far greater than 100, indicating H3 is a decisively preferable fit (Jeffreys 1998). Consistent with our earlier finding for S4 (e.g., Fig. 4D), Table 1 shows it is difficult to distinguish between the three hypotheses for S4 due to the small number of uncertain responses provided by this subject.

Comparison of subject response data to numerical simulations.

To further quantify predicted differences under the three hypotheses, we numerically simulated the stochastic decision-making process using methods similar to those we have previously published (Chaudhuri and Merfeld 2013; Karmali et al. 2016; Lim and Merfeld 2012). As shown in Fig. 5A, each of the three hypotheses yield a different prediction for how the ratio of the binary over trinary threshold estimates (${\widehat{\sigma }}_B\hspace{0.17em}/{\widehat{\sigma }}_T$) vary as the size of the uncertainty region relative to the trinary threshold estimate increases ($\widehat{\delta }\hspace{0.17em}/{\widehat{\sigma }}_T$). This provided a further opportunity to delineate these three hypotheses. As described in detail before (García-Pérez and Alcalá-Quintana 2013), guessing when uncertain (H1) yields ${\widehat{\sigma }}_B>{\widehat{\sigma }}_T$ or equivalently ${\widehat{\sigma }}_B\hspace{0.17em}/{\widehat{\sigma }}_T>1$. This is intuitively due to these uncertain guess responses not being accounted for in the binary model and thus artificially increasing the binary threshold estimate (${\widehat{\sigma }}_B$). For H2, resampling the binary decision variable when uncertain causes ${\widehat{\sigma }}_B<{\widehat{\sigma }}_T$ or ${\widehat{\sigma }}_B\hspace{0.17em}/{\widehat{\sigma }}_T<1$. Finally, for H3, full utilization of the decision variable, even when between the two decision boundaries, yields ${\widehat{\sigma }}_B={\widehat{\sigma }}_T$ or ${\widehat{\sigma }}_B\hspace{0.17em}/{\widehat{\sigma }}_T=1$.

All four subjects from experiment 1 exhibited behavior consistent with H3 and ${\widehat{\sigma }}_B\hspace{0.17em}/{\widehat{\sigma }}_T=1$ (one-sample t-tests: P = 0.07, 0.73, 0.44, and 0.88 for subjects S1–S4, respectively. N = 2,750 trials for each subject/test, degrees of freedom = 2,745). Data points for each subject are shown in Fig. 5A with 95% confidence intervals estimated using a delete-one jackknife procedure (Quenouille 1956; Tukey 1958). Three of the four subjects were inconsistent with H1 and H2 (one-sample t-tests: P < 0.001 for S1–S3. N = 2,750 trials for each subject/test, degrees of freedom = 2,745); the three hypotheses were indistinguishable for the fourth subject who used uncertainty criteria such that $\widehat{\delta }\hspace{0.17em}/{\widehat{\sigma }}_T$ was very small, yielding just 2% uncertain responses. As a group, the subjects were far more consistent with H3 than for H1 or H2. Specifically, H3 yielded a mean square error between simulated and empirical threshold ratios (${\widehat{\sigma }}_B\hspace{0.17em}/{\widehat{\sigma }}_T$) that was more than a factor of 17 smaller than H1 or H2 (Fig. 5A inset table).

Control studies verifying the non-impact of response order.

Next, we asked whether the response ordering (binary then uncertainty, trinary then binary, or simultaneous) impacted this apparent relationship. We tested the same four subjects from experiment 1 using the “2-stage trinary/binary” and “1-stage binary/uncertain” control tasks (675 trials per subject on each control task). For each session, only one task was used and the subject was familiarized with that task before testing. Recall that for H3, theoretically ${\widehat{\sigma }}_B={\widehat{\sigma }}_T$. In each of the response tasks the data (Fig. 5B) align along the unity line if plotted as ${\widehat{\sigma }}_B$ vs. ${\widehat{\sigma }}_T$ (R² = 0.76). This further confirms that the ordering of the response task is not critical; humans are able to make decisions equally well with one or two decision boundaries depending on task requirements.

Subjective “bias” in binary and trinary tasks.

Until this point, we have focused on thresholds (σ), which for our direction-recognition task quantifies decision variable noise variance (i.e., width of the psychometric function, shown in Fig. 3). Now we turn to the “subjective bias” (µ), which is the physical stimulus level at which the probability of left and right responses are equal for forced-choice binary tasks and also the peak probability of an uncertain response for the trinary task. Subjective bias is poorly understood because it could originate from 1) a bias in the perceptual information, 2) a bias in the placement of the decision boundary (or boundaries), or 3) a bias in the noise distribution. Here we simply ask whether the subjective bias is similar for binary vs. trinary decisions. Specifically, are the two trinary decision boundaries set symmetrically about the single binary decision boundary? Figure 5C shows that for each of the four subjects and three response tasks from experiment 1, the binary and trinary subjective biases were consistent with one another (${\widehat{\mu }}_B={\widehat{\mu }}_T$) (R² = 0.90 relative to the unity line). This corresponds to the binary decision boundary being centered between the two trinary decision bounds.

Furthermore, the biases were quite consistent across tasks for each subject. Subjects who demonstrated a moderate bias were consistent in its direction across tasks. For example, S1 (circles in Fig. 5) had a rightward (negative) bias for each task, while S3 (diamonds) had a leftward (positive) bias for each.

Generalization of findings to a broad population.

To see whether the findings of experiment 1 applied to a broader population, we performed a second experiment in which we tested 98 subjects using our “2-stage trinary/binary” task and more typical testing procedures (i.e., 100 trials instead of nearly 3,000 for each subject). As before (Fig. 5A), each subject’s data were fit to yield ${\widehat{\sigma }}_B\hspace{0.17em}/{\widehat{\sigma }}_T$ and $\widehat{\delta }\hspace{0.17em}/{\widehat{\sigma }}_T$ (Fig. 6B). Mean simulations for each hypothesis, modified to match the number of trials and sampling method of experiment 2, are overlaid in Fig. 6, A and B.

Here, since only 100 trials were fit for each subject, individual subjects often have quite variable ratios of ${\widehat{\sigma }}_B\hspace{0.17em}/{\widehat{\sigma }}_T$ (Fig. 6B). For comparison, 98 “subjects” were simulated with only 100 trials, assuming H3, and are shown in Fig. 6A. The simulated data points assuming H3 show similar response patterns as those empirically observed (Fig. 6B, shapes).

The inherent variability in threshold estimates with only 100 trials is further substantiated by considering the four subjects that participated in both experiment 1 (2,850 trials) and experiment 2 (100 trials). These subjects are highlighted in Fig. 6B using the same shapes as in Fig. 5A. With 2,850 trials the ratios of ${\widehat{\sigma }}_B\hspace{0.17em}/{\widehat{\sigma }}_T$ tightly converge toward 1 (Fig. 5A), while the ratios are much more variable with only 100 trials (Fig. 6B). We also note that the subjects displayed a fair degree of consistency in the setting of their uncertainty boundaries. Specifically, subject S4 (triangles in Figs. 5A and 6B) maintained a very small uncertainty region ($\widehat{\delta }/{\widehat{\sigma }}_T$), S2 (squares) had a larger uncertainty region, and S1 and S3 (circles and diamonds, respectively) maintained moderate uncertainty regions.

DISCUSSION

We proposed three hypotheses for how forced binary decisions are made when uncertain and developed three corresponding four-alternative models. Our data show that humans can make decisions with one or two decision boundaries equally well (i.e., ${\widehat{\sigma }}_B={\widehat{\sigma }}_T$ as in H3). Specifically when reporting “uncertain” in a trinary decision, subjects are not simply guessing when forced to make a binary decision. Instead, the same decision variable (defined by both µ and σ) appears to be fully utilized when a forced-choice binary decision is required while uncertain. This was true for two different human studies—one that tested 98 subjects once each and another that extensively tested 4 subjects. Furthermore, we found that the ordering of these two-stage decisions was not critical.

We emphasize that data from both experiments matched H3 the best. The data acquired in experiment 2 with 98 subjects (Fig. 6B) did suggest a slight trend toward some subjects displaying ${\widehat{\sigma }}_B/{\widehat{\sigma }}_T>1$ (Fig. 6B), which is more consistent with H1 [sum of square error (SSE) = 21.9] than H2 (SSE = 23.4). Whatever the cause of this subtle preference, we emphasize that it seemed to disappear for data obtained with nearly 3,000 trials in experiment 1 (Fig. 5A). Even for tests with 100 trials (Fig. 6B), which is within the test duration range commonly used to estimate thresholds, the squared error for H3 (SSE = 11.6) was about half that for H1 and H2 (Fig. 6B). Furthermore, the variability in behavior experimentally observed (Fig. 6B) was similar to that from simulations of the same conditions assuming H3 (Fig. 6A). Taken together, these results demonstrate that humans can utilize complete information to make decisions with either one or two decision boundaries equally well (i.e., yield equivalent threshold estimates, ${\widehat{\sigma }}_B\approx {\widehat{\sigma }}_T$).

Comparison of findings with previous studies on uncertainty.

Findings using a similar four-category response task for visual discrimination appear consistent with our primary findings supporting H3 (i.e., ${\widehat{\sigma }}_B={\widehat{\sigma }}_T$) (Okamoto 2012). However, far fewer trials and the use of a different six-parameter model prohibited that earlier study from ruling out H1 or H2. Consistent with our finding of the two trinary decision boundaries being symmetric about the binary decision boundary (${\widehat{\mu }}_B={\widehat{\mu }}_T$), Okamoto found for five of six subjects that the separation of the three decision boundaries was roughly equal. In fact, if the underlying psychometric curve is an idealistic cumulative Gaussian, but uncertain decisions do not fully utilize complete information (H1 or H2), this could potentially be observed by distortions in the measured psychometric curve [i.e., a flattening of the psychometric curve as previously noted (García-Pérez and Alcalá-Quintana 2013)]. However, to observe such distortions would require a very large number of trials per subject and/or many subjects with a moderate number of trials, as we tested here. To systematically validate H3 required the development of a theoretical model and Monte Carlo simulations to compare with empirical data.

A number of earlier studies suggest that humans tend to demonstrate better-than-chance performance, even when they indicate that they are uncertain. This has been reported for: discrimination of lifted weights (Fullerton and Cattell 1892; Peirce and Jastrow 1885), visual letter/digit recognition task (Coover 1917; Sidis 1898; Stroh et al. 1908), visual shape recognition (Miller 1939; Williams 1938), visual orientation recognition (Baker 1938), auditory recognition (Baker 1938; Coover 1917), and word memory recognition (Sheridan and Reingold 2011; see Adams 1957; Merikle et al. 2001; Tversky and Kahneman 1974 for reviews). Before proceeding, it is crucial to note that better-than-chance performance when uncertain does not demonstrate that subjects fully utilize complete information (H3). For example, our H2 is a model that yields better-than-chance performance but does not appropriately utilize all available information.

On the other hand, some recent studies suggest that subjects guess when uncertain (García-Pérez and Alcalá-Quintana 2011, 2013). The contradiction between this conclusion and earlier findings of better-than-chance performance when uncertain is not fully understood. One possible explanation is that different modalities or different tasks yield such differences. Since it is impossible to evaluate every possible task, we chose to focus on a simple passive, self-motion, direction-recognition task, and we intentionally included both standard test procedures (e.g., 100 trials) and extensive testing (i.e., 39 test sessions totaling almost 3,000 trials) of four subjects. These data (e.g., Figs. 5A and 6B) and the associated quantitative analyses (Table 1) showed task performance consistent with the hypothesis that, when uncertain, humans fully utilize complete information (H3).

While our results are conclusive for our vestibular direction-recognition task that uses the method of single stimuli, this task is relatively simple. Beck and Ma et al. (2012) note that suboptimal inference (i.e., mechanisms that do not fully utilize complete information) may be especially applicable for more complex tasks. For example, decision-making paradigms that require subjects to compare two or more stimuli (e.g., 2-interval forced choice or temporal order judgments) may be more complex and thus decisions when uncertain may not fully utilize available information.

To determine whether complete information is fully utilized when uncertain in determining motion direction, we 1) intentionally used a task in which subjects could not respond before the end of the trial, 2) applied measurement techniques that directly sampled both the binary decision and “uncertainty” for each trial, and 3) sampled around levels where subjects were uncertain about ~20% of the time. Our chosen task contrasts with standard response time tasks in which subjects are encouraged to respond as soon as they have made a decision. Such data are typically analyzed using an important and influential drift-diffusion modeling approach (e.g., Ratcliff 1978). Critically, this approach typically utilizes two decision boundaries that implicitly define an “uncertainty region” (Fig. 1 of Merfeld et al. 2016), similar to that of the trinary uncertainty model (García-Pérez and Alcalá-Quintana 2011, 2013), until the subject has accumulated enough information to yield a decision. By showing that humans can perform a forced-choice binary task and trinary task with an uncertain option equally well, our findings suggest that approaches that study decision-making using tasks with a single decision boundary (e.g., binary forced-choice tasks) may assess the same neural mechanisms as tasks with two decision boundaries (e.g., trinary tasks with an uncertainty option or response-time tasks).

Four-alternative models of decision-making.

Building on Klein’s (2001) concept of a four-alternative response task, we developed theoretical models that allow for two-alternative responses (e.g., left vs. right) with a certain/uncertain categorization. Our theoretical models included formal definitions of the corresponding psychometric curves as well as the maximum likelihood formulation for fitting the models to response data. By formulating a theoretical model for each of our three mechanistic hypotheses and fitting each model to the data, we determined that the best fit corresponded to full information utilization, even when uncertain (H3). This theoretic model for H3 could be used in the future to fit four-alternative response data of a similar type. However we note that our findings suggest applying the traditional binary signal detection model (Green and Swets 1966) or the trinary uncertainty model (García-Pérez and Alcalá-Quintana 2013) would yield similar threshold estimates. The experimental evidence here helps validate and unify these otherwise contradictory models and tasks for perceptual decision making. Critically, we suggest that the test for future neurophysiology experiments aimed at identifying a neural decision variable under uncertainty is relatively straightforward. Specifically, our theoretical frameworks could be used to test whether a neural decision variable is fully utilized.

One decision variable used for both two-alternative and three-alternative tasks.

Our analyses also considered an option that, when uncertain, subjects have some information that leads to better than chance performance. While many such partial information variants are feasible, we specifically hypothesized that it may be achieved by sampling another, separate decision variable to make the binary decision when uncertain (H2). Our analysis found the data to be inconsistent with this hypothesis, which parallels recent work suggesting a common decision variable for both choice and confidence judgment (of which certain/uncertain is a simplified version) in humans (Kiani et al. 2014; Rahnev et al. 2012) and animals (Kepecs and Mainen 2012; Kepecs et al. 2008; Kiani and Shadlen 2009; Komura et al. 2013). This finding is consistent with recent studies that showed that the same sensory signals support a binary decision and represent confidence (Fetsch et al. 2014; Grimaldi et al. 2015).

It makes sense that confidence in a decision, including confidence classification (i.e., uncertain vs. not uncertain), as we investigated here, would utilize the same decision variable as the decision. The alternative is that confidence would only indirectly relate to the decision. We intentionally did not assess confidence on a continuous scale (i.e., ranging from 50%, i.e., “just guessing,” to 100%), as we have done recently for similar direction-recognition tasks (Lim et al. 2017; Yi and Merfeld 2016), but instead focused on a binary evaluation of uncertain vs. not uncertain to simplify the subjects’ psychophysical task and enable our statistical analyses. However, uncertainty is in many ways the inverse of confidence and the uncertain vs. not uncertain classification likely involved subjects setting some boundary separating low and high confidence regions, respectively. Yet we also note that some recent thinking treats confidence and certainty as distinct quantities (Pouget et al. 2016) or they may just appear distinct due to processing delays (van den Berg et al. 2016).

Conclusion.

We conclude that subjects fully utilize the same decision variable yielding equal estimates of the noise/threshold for both binary and trinary decisions (${\widehat{\sigma }}_B={\widehat{\sigma }}_T$). These findings show that binary forced-choice and trinary uncertainty tasks provide different, yet harmonious, views of the same decision-making process. Our study provides a model to test whether when uncertain, subjects fully utilized complete information for any specific task/modality.

GRANTS

This work was supported by the National Space Biomedical Research Institute (NSBRI) through NASA Grant NCC9-58 (T. K. Clark), via National Institute of Deafness and Other Communications Disorders Grants R01DC04158 and R01DC014924 (Y. Yi and D. M. Merfeld), and a NASA Space Technology Research Fellowship, Grant no. NNX13AM68H (R. C. Galvan-Garza).

DISCLOSURES

No conflicts of interest, financial or otherwise, are declared by the authors.

AUTHOR CONTRIBUTIONS

T.K.C., M.C.B.R., and D.M.M. conceived and designed research; T.K.C., Y.Y., R.G.-G., and M.C.B.R. performed experiments; T.K.C., Y.Y., and M.C.B.R. analyzed data; T.K.C., Y.Y., R.G.-G., M.C.B.R., and D.M.M. interpreted results of experiments; T.K.C., Y.Y., and D.M.M. prepared figures; T.K.C. drafted manuscript; T.K.C., Y.Y., R.G.-G., M.C.B.R., and D.M.M. edited and revised manuscript; T.K.C., Y.Y., R.G.-G., M.C.B.R., and D.M.M. approved final version of manuscript.