The confirmation and prevalence biases in visual search reflect separate underlying processes

Abstract

Research by Rajsic, Wilson, and Pratt (2015; 2017) suggests that people are biased to use a target-confirming strategy when performing simple visual search. In three experiments, we sought to determine whether another stubborn phenomenon in visual search, the low-prevalence effect (Wolfe, Horowitz, & Kenner, 2005), would modulate this confirmatory bias. We varied the reliability of the initial cue: For some people, targets usually occurred in the cued color (high prevalence). For others, targets rarely matched the cues (low prevalence). High cue-target prevalence exacerbated the confirmation bias, indexed via search RTs and eye-tracking measures. Surprisingly, given low cue-target prevalence, people remained biased to examine cue-colored letters, even though cue-colored targets were exceedingly rare. At the same time, people were more fluent at detecting the more common, cue-mismatching targets. The findings suggest that attention is guided to “confirm” the more available cued target template, but prevalence learning over time determines how fluently objects are perceptually appreciated.

Imagine that you are getting ready for work, searching your closet either for your favorite blue shirt or your favorite red shirt. Both are part of your normal work wardrobe, and you will happily grab whichever one you discover first. Many garments in the closet are blue and relatively few are red, so finding the red shirt would likely be easier. Nevertheless, suppose that the blue shirt was foremost in your mind, simply because wearing it had occurred to you first. In this case, the blue shirt would be prominent in your search template, potentially encouraging you to search among all blue items, despite the relative inefficiency of that approach. Recent work by Rajsic, Wilson, and Pratt (2015; 2017) demonstrates this aspect of visual search behavior: When people have a specific search template in mind, they generally search for that available item, even if another search strategy would be more efficient. Although the optimal strategy is to guide attention to the salient minority color in the display (Duncan & Humphreys, 1989; Treisman & Gelade, 1980), people show a stubborn tendency to search for template matches. Rajsic et al. described this perseverative strategy as a form of confirmation bias (Koehler, 1991; Nickerson, 1998; Wason, 1960; 1966; 1968; Klayman & Ha, 1987). This bias persisted even when participants are directly instructed to adopt potentially more efficient search strategies.

Given that the confirmation bias resists cognitive control, we tested how it would interact with another automatic tendency in visual search. The low-prevalence effect is a robust finding wherein people fail to detect rare search targets, a maladaptive criterion shift that is gradually learned over time (Hout, Walenchok, Goldinger, & Wolfe, 2015; Papesh & Goldinger, 2014; Wolfe et al., 2007; Wolfe, Horowitz, & Kenner, 2005; Wolfe & Van Wert, 2010). In the confirmation bias (Rajsic et al., 2015), people preferentially search for targets that match an initial search template, such as searching for red letters given a red cue. We hypothesized that the target prevalence statistics might modulate this tendency: If targets are rarely red, might people guide their attention more frequently to green objects? Conversely, if targets are almost always red, might this exaggerate the confirmation bias? We tested these ideas in three experiments, examining the effects of target prevalence on confirmatory strategies in visual search.

Rajsic et al. (2015) initially discovered the confirmation bias using a simple task. They showed people circular displays of letters, with a specified target letter that was always present. Prior to search, participants were shown a specific color; they were instructed to press a key if the target occurred in this color, and press a different key otherwise (e.g., “press Z if the target is red, otherwise press M”). Targets appeared in the cued and uncued colors equally often. Color proportions were varied in each display, however, such that half (.5), a minority (.25), or a majority (.75) of the letters occurred in the cued, template color (see Figure 1).

Figure 1.

General example of conditions from Rajsic et al. (2015) and the current research

As noted, the (apparently) optimal strategy is to restrict search to the minority color in each display (see also Sobel & Cave, 2002), as it affords search through relatively few items. If the target is not located in the minority subset, people could infer the answer (i.e., the target must be the other color). If people adopted this optimal strategy, search RTs would follow a quadratic function with faster search in the .25 and .75 template-match conditions, and slower in the .50 match condition, wherein no strategy is optimal; see Figure 2, left panel. Additionally, this optimal strategy might be characterized by a crossover interaction: If people use the optimal strategy, they may adopt an “ad hoc” mental template to guide search, according to the characteristics of the current display. If the minority letters in the display are green, and people attend to only those letters, they should be faster when making a corresponding “green” response, compared to when the target is red (when they must use process-of-elimination and switch responses). Conversely, given a suboptimal strategy of searching only through template-matching letters, RTs would follow a linear trajectory with fastest search in the .25 template- match condition and slowest search in the .75 template-match condition (Figure 2, right panel). This linear pattern was observed by Rajsic et al., which persisted even when participants were explicitly informed of the optimal strategy (Experiment 4). People only adopted a more flexible strategy after seeing display previews prior to search (Experiment 5) allowing them to plan searches within the smaller subset of colors. Nevertheless, all other experiments suggested that the default strategy is confirmatory search.

Figure 2.

Predicted results from alternative search strategies

In decision making, confirmation bias refers to the tendency to accept hypothesis- confirming information, while ignoring disconfirming information. For example, Forer (1949) gave students personality assessments and their resultant personality descriptions. The students rated their descriptions as accurate and personally tailored, and therefore rated the assessment as highly valid. Unbeknownst to the students, identical descriptions were given to everyone, demonstrating their willingness to uncritically accept overly general descriptions. Wason (1960) observed confirmation bias using a neutral problem-solving task. Participants were given sets of three numbers following a specific rule (ascending numbers; e.g., 1, 3, 5), and were instructed to discover the rule by testing hypotheses. Participants tested hypotheses by writing down a series of numbers; the experimenter indicated whether the series conformed to the rule. Eventually, participants announced rules, and, if incorrect, were given the opportunity to test additional hypotheses. Critically, participants who announced many incorrect rules tended to follow a hypothesis-testing strategy in which only further confirmatory evidence was possible. For example, if the proposed series “4, 6, 8” was declared correct by the experimenter, the participant might assume an “ascending by two” rule, subsequently testing similar series such as “8, 10, 12” and “7, 9, 11” rather than a potentially hypothesis-disconfirming (but correct) series such as “3, 6, 10.”

Such illogical selection is difficult to counter. According to Koehler (1991), once the initial (or focal) hypothesis is established, subsequent evidence is considered within the context of a conditional reference frame, wherein the hypothesis is considered to be correct. Koehler recognized that bias toward hypothesis-confirming evidence extends to the visual domain (p. 513): In a classic study by Carmichael, Hogan, and Walter (1932), two groups of participants were shown ambiguous figures paired with different object names. For example, two circles connected by a line might labeled “eyeglasses” for one group and “dumbbells” for the other. When asked to reproduce these figures in a subsequent memory test, participants’ drawings often reflected the names used during encoding. These results suggest that the object names became established as focal hypotheses, which were unable to be divorced from the original, neutral images: Participants “confirmed” the object names in their reproductions.

Rajsic et al. (2015) suggested that confirmation bias in visual search may stem from cognitive capacity limits (Evans, 2006; Mynatt, Doherty, & Dragan, 1993; Mynatt, Doherty, & Sullivan, 1991) and the burden of searching for simultaneous targets. For example, although people can hold multiple objects in visual working memory (Luck & Vogel, 1997), search is impaired when searching for multiple targets (Hout & Goldinger, 2010; 2012; 2015; Menneer et al., 2012) and people may guide attention using only a single template at a time (Olivers, Peters, Houtkamp, & Roelfsema, 2011; although see Beck, Hollingworth, & Luck, 2012; Irons, Folk, & Remington, 2012; Stroud, Menneer, Cave, & Donnelly, 2012). The temporal cost of switching target templates may outweigh the cost of randomly searching through more items (Wolfe, Horowitz, Kenner, Hyle, & Vasan, 2004). Indeed, random movement of attention during search is more efficient than volitional movement (Wolfe, Alvarez, & Horowitz, 2000) and attention is generally amnesic, often returning to previous locations (Horowitz & Wolfe, 1998; 2001; although see Klein & McInnes, 1999; Peterson et al., 2001). The confirmation bias may therefore indicate that people search in the cognitively “cheapest” way possible.

In the low-prevalence effect, statistical learning over many searches encourages template prioritization for more frequent targets. Over time, people become more likely to detect frequent targets while missing rare ones (Wolfe et al., 2005; Wolfe et al., 2007; Wolfe & Van Wert, 2010; Hout et al., 2015; Scarince & Hout, 2018). Hout et al. (2015) found this pattern in multiple-target search: Participants were instructed to equally search for a teddy bear or a butterfly, only one of which appeared in any given trial. Target prevalence was varied such that one target occurred less frequently than the other (e.g., 45% bears, 5% butterflies, 50% target-absent). Rare targets were found more slowly than common targets, and were often missed entirely. Eye-tracking revealed that these errors were largely perceptual: Even after directly fixating rare targets, participants often failed to recognize them. In Rajsic et al.’s paradigm, people also searched for two potential targets (e.g., a green or red letter p). Although relative prevalence was balanced, the observed confirmation search bias may have resulted from people adopting the more “available” (i.e., cued) search template by default (Kunda, 1990; Tversky & Kahneman, 1973; 1974). More recently, Rajsic et al. (2017) confirmed this account using eye-tracking: People demonstrated an attentional bias, overwhelmingly inspecting template-colored items in the display. This bias was reduced only when object inspections were made costly, such as by making object inspections gaze-contingent (i.e., occluding each letter with a colored circle and only revealing it during fixation).

In the present study, we investigated the logical next question: What happens when cued templates are made more or less reliable? We conducted three experiments that closely followed the Rajsic et al. (2015) paradigm, while adding three between-subjects conditions that varied the relative prevalence of trials containing targets that matched or mismatched the initial cue: (1) high prevalence (HP): 85% template-matching targets, 15% mismatching, (2) balanced prevalence: 50% matching and mismatching (the proportions used by Rajsic et al., 2015), and (3) low prevalence (LP): 15% matching and 85% mismatching. In Experiment 1, following procedures from Hout et al. (2015), participants terminated search by pressing the space bar, then specified target identity afterwards. This procedure was adopted to minimize motor errors that result when people respond disproportionately often to frequent targets (Fleck & Mitroff, 2007). To foreshadow, this procedural change reduced the confirmation bias, although we observed indirect evidence that it remained active. We next ran two additional experiments exactly following the Rajsic et al. (2015) procedure, wherein participants simultaneously terminated search and classified targets by pressing one key for “green” and another for “red.” Experiment 2 focused on behavioral RT and accuracy data only; Experiment 3 included eye-tracking.

In the balanced conditions, we expected to replicate Rajsic et al. (2015). Our predictions for the HP conditions were straightforward: Because the prevalence and confirmation biases should reinforce each other, we expected strong linear trends in search RT, as shown in Figure 2. We also expected people to easily detect the frequent template-matching targets, but to often miss the rare, template-mismatching targets. For the LP conditions, wherein people rarely encountered template-matching targets, we anticipated three possible outcomes: (1) Dominant confirmation bias: In this scenario, the confirmation bias would be stronger than prevalence learning, resulting in a linear pattern of search RTs (as in Figure 2) with little difference between rare and common targets. (2) Dominant prevalence bias: In this scenario, people would prioritize the more common, uncued targets. They would be faster at finding cue-mismatching targets and search RTs would be a mirror reversal of the right panel in Figure 2. RTs would decrease as a function of the template-matching subset size (i.e., faster search with fewer template-mismatching stimuli). (3) Flexible search: Here, people would avoid the confirmation bias, producing quadratic search RTs that resemble the left panel in Figure 2. They would, however, be faster and more accurate to detect more prevalent targets. After establishing the RT and accuracy patterns in Experiments 1 and 2, we used eye tracking in Experiment 3 to elucidate how confirmation bias and prevalence affect attentional guidance and perceptual decision- making.

Experiment 1

Experiment 1 tested the influence of color prevalence on confirmatory strategy use in visual search. Rajsic et al. (2015) found that people perseverated with a confirmatory strategy of seeking template-matching colors, even when the strategy was inefficient. If cued template colors rarely predict target identity (LP), people might learn to search for the uncued color.Conversely, if the target cue is almost always reliable (HP), this should exaggerate the confirmation bias. Given balanced prevalence, search should be confirmatory, as reported by Rajsic et al. (2015).

Method

Participants.

A total of 176 Arizona State University students participated for course credit. All procedures were approved by the university’s Institutional Review Board and all participants gave informed consent. In order to determine whether we had adequate statistical power, we conducted power analyses using G*Power 3 (Faul, Erdfelder, Lang, & Buchner, 2007) based on the effect sizes provided by Rajsic et al. (2015) and Hout et al. (2015). The power analyses are provided in Appendix A.

Apparatus.

Data were collected on up to 10 computers simultaneously, each sharing identical hardware and software profiles: Dell Optiplex 380 PCs at 3.06 GHz and 3.21 GB RAM, in 1366 × 768 resolution on Dell E1912H 18.5” monitors at a 60 Hz refresh rate, with the display controlled by an Intel G41 Express chipset, each running on Windows XP. All stimuli were presented and data were collected using E-Prime 2.0 software (Schneider, Eschman, & Zuccolotto, 2013).

Stimuli.

Following Rajsic et al. (2015), visual search displays were circular arrays of the letters p, q, b, and d in Arial font, each approximately 2° in height and 1° in width, with each letter drawn approximately 8° from fixation. Eight letters were presented against a gray background (RGB: 128, 128, 128) in two possible colors, randomly selected from these seven RGB values: 200, 0, 255; 200, 200, 0; 0, 255, 0; 255, 128, 0; 255, 128, 255; 50, 50, 255; 255, 50, 50. Colors selected for any participant were held constant throughout the experiment.

Design.

Within-subjects variables included (1) Target Color (matching vs. mismatching the initial template cue) and (2) Template Color Proportion (stimuli in the display that matched the initial cue; .25, .50, or .75; for brevity, we abbreviate this as TCP in text, but spell it out in figure axes). Participants were separated by Prevalence Group (Balanced: Targets matched the initial template color in 50% of trials, High: Targets matched the template in 85% of trials, and Low: Targets matched the template in 15% of trials).

Procedure.

Participants were familiarized with the target letter, template color, and response mappings at the beginning of the experiment. Six initial practice trials were administered. Prior to the main task, participants completed 60 baseline trials with equal target prevalence. This baseline block was included to assess any preexisting group differences. Following the baseline trials, participants completed three blocks of 240 trials. Participants were reminded of the target letter, template color, and response mappings prior to each block. All possible stimulus combinations were presented equally within each block, and a one-minute rest period was presented after the first and second blocks. Within a trial, following a 500-ms fixation cross, distractor letters were randomly presented in both the template and non-template colors in varying proportions. These colors remained constant throughout the experiment, following Rajsic et al. (2015, Experiment 3), because keeping target and distractor colors constant was necessary for our prevalence manipulation. The response requirements, however, differed from Rajsic et al. (2015): Participants were instructed to terminate search with the space bar upon locating the target. In a subsequent display, they indicated whether the target matched or mismatched the cued template color with the “f” and the “j” keys. This response mapping was counterbalanced by participant. Also differing from Rajsic et al. (2015), we omitted accuracy feedback in each trial, to minimize awareness of the prevalence manipulation (see Hout et al., 2015). See Figure 3 for sample trial progressions from all three experiments in this article.

Figure 3.

Trial progression in all experiments.

Results

The present investigation includes three large-scale experiments with many potential effects and interactions. When eye-tracking was examined (Experiment 3), there were also numerous potential dependent measures. For brevity and clarity, we report many ANOVA tables, figures, and some non-focal analyses in the supplemental materials. In the main article, we report effects (whether positive or null) of key interest and refer to tables and figures in the supplement using the prefix S (e.g., Table S2).

From the original sample of 176 participants, 11 were removed prior to analysis: Two were excluded due to technical errors, one for falling asleep, and one for cellular phone use. One participant was excluded due to missing data (no correct trials in one cell of the ANOVA design). Finally, six participants were excluded for low accuracy, >2.5 standard deviations below their group means, one of whom also produced excessively fast RTs. The final sample sizes for the balanced, HP, and LP groups were 54, 54, and 57 participants, respectively. The respective mean accuracy for these groups were 96%, 98%, and 98%.

All analyses were conducted using repeated-measures ANOVAs, first examining the full 3×3×2 mixed-model, with the between-subjects factor Prevalence Group and within-subjects factors Target Color and TCP. To more directly compare the low and high prevalence groups, we conducted similar ANOVAs excluding the balanced prevalence group. Additionally, we conducted repeated-measures ANOVAs examining TCP and Target Color within each prevalence group, whenever Prevalence Group interacted with either of these variables in the overall ANOVAs. Separate analyses were conducted for mean accuracy, and median RT in correct trials (following Rajsic et al., 2015). The results addressed in the text primarily include main effects, the key Target Color × TCP interaction, and polynomial contrasts within TCP. Where applicable, we report multivariate statistics (Pillai’s Trace) to account for potential violations of the sphericity assumption (Keppel & Wickens, 2004). All pairwise comparisons were conducted using Bonferroni-corrected, two-tailed t-tests.

Baseline trials.

To assess any preexisting group differences, we analyzed the baseline trials, which maintained equal target-color prevalence (as in Rajsic et al., 2015). The results are shown in the supplemental materials. Regarding accuracy (see Figure S1, Table S1), there was a Target Color × Prevalence Group interaction, but it followed no discernible pattern and accuracy was generally at ceiling. Regarding median RTs (see Figure S2, Table S2), we observed main effects of Target Color and TCP, and a Target Color × TCP interaction. There was no effect of Prevalence Group, nor any remaining interactions. Thus, we observed no a priori group differences.

Overall performance for the main trials of Experiment 1 are shown in Figure 4, with mean accuracy in the upper row, and median RTs in the lower row.

Figure 4.

Mean accuracy and median RTs for each prevalence group in Experiment 1, as a function of Target Color and TCP. Reliable pairwise comparisons are indicated with asterisks (where applicable following reliable interactions) and error bars represent SEM.

Accuracy.

Supplemental Table S3 summarizes all the ANOVA results for accuracy in Experiment 1, with an omnibus analysis for the entire design, followed by more focused analyses on specific conditions. As suggested by the upper row of Figure 4, the main finding of interest was a robust interaction of Target Color x Prevalence Group (in the omnibus ANOVA, F(2, 162) = 78.1, p < .001, ${\eta }_p^2$ = .49). Accuracy was equivalent when prevalence was balanced, but systematically increased for whichever color was more frequently correct.

Response Times.

Supplemental Table S4 summarizes the ANOVA results for RTs, and we highlight key effects here. In the omnibus ANOVA, all main effects and interactions were reliable except for the main effect of Prevalence Group. The same was true in the analysis including only the HP and LP groups. As in the accuracy data, the most striking aspect of the RT results is the powerful prevalence effect, shown by the reversal of the green and red functions in Figure 4 (representing template matching and mismatching trials). At a finer grain of analysis, the results illustrate that the confirmation and prevalence biases can either cooperate or compete with each other, but neither appears to negate the other. We briefly focus on each sub-condition, examining the results in accordance with the predictions from Figure 2, and then consider the bigger picture. For each, we followed the analytic strategy from Rajsic et al. (2015), testing for interactions of Target Color × TCP, followed by tests for linear and quadratic trends. The trend analyses were relatively simple, testing for reliable fits and noting their proportions of explained variance. With only three points per function, there will typically be a reliable linear trend with any rise or fall; the more important question is whether any quadratic trend better characterizes the RT function. As we address in the General Discussion, these comparisons were occasionally complex and more global aspects of the RT data proved more informative. Nevertheless, the trend analyses provide valuable insight into confirmatory versus flexible search.

Balanced prevalence group.

Looking at Figure 4, it is immediately clear that the RTs resemble the “optimal” schematic pattern from Figure 2, rather than the “confirmatory” pattern. We observed a main effect of TCP and a Target Color × TCP interaction. We therefore tested the predicted trends separately: In template-matching trials, the effect of TCP was reliable [F(2,52) = 42.63, p < .001, ${\eta }_p^2$ = .62], as were the linear [F(1, 53) = 52.25, p < .001, ${\eta }_p^2$ = .50] and quadratic [F(1, 53) = 34.21, p < .001, ${\eta }_p^2$ = .39] trends. In template-mismatching trials, the same main effect was reliable [F(2, 52) = 18.42, p < .001, ${\eta }_p^2$ = .42], as were the linear [F(1, 53) =11.55, p = .001, ${\eta }_p^2$ = .18] and quadratic [F(1, 53) = 34.21, p < .001, ${\eta }_p^2$ = .39] trends. As the lower left panel of Figure 4 suggests, the trend in template-matching trials was more linear than quadratic, while the converse was true for template-mismatching trials. Reliable pairwise comparisons between template-matching and mismatching are depicted with asterisks (both t > 2.8). The crossover interaction suggests that people adopted flexible search templates according to each trial’s minority subset color (as in the left panel of Figure 2): If the minority color matched the template, they were faster when the target actually occurred in the template color. Conversely, when the minority of items were presented in the uncued color, they were faster to find template-mismatching targets. These results stand in contrast to the confirmation bias observed by Rajsic et al. (2015).

High prevalence group

There were main effects of Target Color and TCP, but no interaction. We therefore examined overall contrasts for TCP: Both linear F(1, 53) = 22.87, p <.001, ${\eta }_p^2$ = .30] and quadratic [F(1, 53) = 15.79, p < .001, ${\eta }_p^2$ = .23] trends were reliable. As the lower-middle panel of Figure 4 illustrates, the better-fitting trend was linear, suggesting that people primarily used a template-confirming strategy in the HP condition. Importantly, there was a large effect of prevalence, with much faster search for more likely targets, with an average difference of 390 ms.

Low prevalence group.

Both main effects and their interaction were reliable, and all paired comparisons between template-matching and mismatching trials were reliable (all t > 5.4; see Figure 4, lower-right panel). In the template-matching trials, the effect of TCP was reliable [F(2, 55) = 6.95, p = .002, ${\eta }_p^2$ = .20], with a quadratic trend [F(1, 56) = 13.99, p < .001, ${\eta }_p^2$ = .20] but no linear trend [F(1, 56) = 0.20, p = .661]. In the template-mismatching trials, the effect of TCP was again reliable [F(2, 55) = 29.13, p < .001, ${\eta }_p^2$ = .51]. Both the quadratic [F(1, 56) =39.24, p < .001, ${\eta }_p^2$ = .41] and linear trends [F(1, 56) =15.43, p < .001, ${\eta }_p^2$ = .22] were reliable,although the better-fitting trend was quadratic. Once again, there was a large effect of prevalence, with faster search for more common targets.¹ Now, however, the average difference was only 256 ms, a marked reduction from the HP condition.

Discussion

In Experiment 1, we examined the influence of target prevalence on the “confirmation bias” in visual search. Rajsic et al. (2015) found that people fall into a biased, confirmatory search strategy, seeking items that match the cued template. This strategy persisted even though the optimal strategy was to restrict search to whichever subset of items was smaller, using inference if necessary. However, we predicted that, if targets either rarely or frequently matched the cued template, strategies might change accordingly. Under HP conditions, people should become even more reliant on confirmatory search, strongly preferring to search among template- matching items, which we observed. In LP conditions, people should begin to favor searching among the opposite subset, or at least become more flexible. While we did observe an overall quadratic trend (Figure 4, lower-right panel), people were clearly faster at detecting template- mismatching targets, indicating that prevalence effects were stronger than any tendency toward confirmatory search. This interpretation is complicated, however, by our failure to observe the confirmation bias in the balanced condition, a near-direct replication of Rajsic et al. (2015), but with one procedural change that we address in Experiment 2.

Despite the unexpected results in the balanced condition, Experiment 1 did provide evidence for the confirmation bias. This can be appreciated in the overall results, especially when examining the HP and LP conditions in tandem. Recall that, in the HP condition, the biases were expected to be additive: Both the confirmation and prevalence biases would encourage people to focus cue-colored objects. Operating together, these biases would produce rising functions across color proportions (the confirmation bias pattern schematized in Figure 2) and large RT differences between rare and common targets. This exact pattern was observed in the HP condition, but is challenging to interpret in isolation. The LP condition offers helpful context. In the LP condition, the confirmation and prevalence biases were expected to conflict with each other, although we could not anticipate which bias would be “stronger.” In this case, the confirmation bias would encourage attention to cue-colored targets, whereas the prevalence bias would encourage attention to opposite-colored targets. Thus, we anticipated results showing a “compromise” between biases, assuming that neither one could totally overpower the other. The results appear consistent with our expectations: Comparing the lower-middle and lower-right panels of Figure 4, the difference between Target Color conditions was 35% smaller in the LP group (256 ms), relative to the HP group (390 ms; t(109) = 14.63, p < .001, d = 2.77). Thus, although the quadratic trends in the LP group suggest flexible search, something shrank the prevalence effect by more than 130 ms, suggesting a lingering tendency to allocate attention toward cue-consistent colors. Despite almost always encountering template-mismatching targets, people at least occasionally defaulted to template-confirming search. We further isolate the sources of this complex pattern by examining eye movements in Experiment 3.

Experiment 2

Notably, in Experiment 1, we found evidence for flexible search (i.e., no confirmation bias) in the balanced condition, failing to conceptually replicate Rajsic et al. (2015) despite very similar paradigms. There were, however, several differences across the experiments. We tested more participants and trials, which seems unlikely to affect the qualitative pattern. We also required separate keyboard responses for terminating search and classifying targets. This procedure was adopted (following Hout et al., 2015) to avoid prevalence-consistent response biases (Fleck & Mitroff, 2007) that might arise when participants press the same key in 85% of trials. Conceptually, this procedure separates searching from responding, which may have inadvertently altered search strategies. We elaborate in the General Discussion, but the core idea is simple: Moving spatial attention is cognitively cheap (Wolfe et al., 2004) whereas making response decisions is costly (Donders, 1868; Sternberg, 1966). Given a procedure that confers a “head-start” to one potential key-press (via the cue color), people may unconsciously adopt the confirmatory search process, as moving attention is easier than altering prepotent responses. In Experiment 2, we altered our procedure to better follow Rajsic et al. (2015). We reduced the trial count and participants terminated search and classified targets simultaneously, by pressing either of two keys. As in Experiment 1, we included three prevalence conditions. The balanced condition was an exact replication of Experiment 3 from Rajsic et al. (2015).

Method

The apparatus, stimuli, and design were identical to those used in Experiment 1.

Participants

One-hundred-sixty-six students participated for course credit, following the protocols from Experiment 1. None had participated previously.

Procedure

Following the procedure of Rajsic et al. (2015, Experiment 3), participants were informed of the target letter, template color, and response mappings only once prior to beginning the experiment, and were instructed to memorize this search rule. No practice or baseline trials were administered, and participants completed one block of 300 trials. Responses were again counterbalanced, with the response keys “z” and “m.” In each trial, the search display followed a 500-ms fixation cross, participants terminated search by pressing the appropriate key, and then received “correct” or “incorrect” accuracy feedback for 1500 ms (see Figure 3).

Results

From the original sample of 166 participants, 11 were removed prior to analysis: Two participants were excluded due to cellular phone use, one for falling asleep, and one due to technical error. In the balanced prevalence group, three participants were excluded (two for low accuracy, one for slow RTs). Mean accuracy for the remaining 52 participants was 95%. In the HP group, four participants were excluded (three for low accuracy, one for slow RTs). Mean accuracy for the remaining 49 participants was 97%. In the LP group, another four participants were excluded (three for low accuracy, one for slow RTs). Mean accuracy for the remaining 50 participants was 96%. The results are shown in Figure 5, with mean accuracy in the upper row, and median RTs in the lower row.

Figure 5.

Mean accuracy and median RTs for each prevalence group in Experiment 2, as a function of Target Color and TCP. Error bars represent SEM.

Accuracy.

Supplemental Table S5 summarizes all the ANOVA results for accuracy in Experiment 2, with an omnibus analysis for the entire design, followed by focused analyses on specific conditions. As suggested by the upper row of Figure 5, the main finding of interest was a robust interaction of Target Color x Prevalence Group (in the omnibus ANOVA, F(2, 148) = 44.1, p < .001, ${\eta }_p^2$ = .36). As shown, accuracy was equivalent when prevalence was balanced, but systematically increased for whichever color was more frequently correct in the LP and HP conditions. There were no other effects of interest.

Response Times.

Supplemental Table S6 summarizes the ANOVA results for RTs. Examining Figure 5, there were clear differences from Experiment 1, although the same key effects were observed. In the balanced prevalence condition, we observed a pattern that better replicates Rajsic et al.’s (2015) findings. In the LP and HP conditions, we again observed strong prevalence effects, but with clear evidence of a persistent confirmation bias.

Balanced prevalence group.

The main effects of Target Color and TCP were reliable, but the interaction was not. As shown in Figure 5, lower-left panel, people were faster in template- matching trials (1256 ms) than in template-mismatching trials (1443 ms). Regarding the overall TCP effect, search time increased as a function of the proportion of template-matching letters in the display. This trend was mostly linear [F(1, 51) = 35.58, p < .001, ${\eta }_p^2$ = .41], although the quadratic trend was also reliable [F(1, 51) = 16.85, p < .001, ${\eta }_p^2$ = .25]. These results generally replicate the pattern found by Rajsic et al. (2015): The lack of a Target Color × TCP interaction and the positive linear trend of TCP suggest a confirmatory strategy wherein participants preferentially inspected template-matching letters in the display.

High prevalence group.

We observed main effects of Target Color and TCP, but again no interaction. As shown in Figure 5 (lower-middle panel), the prevalence effect was robust, with a mean difference of 477 ms. RTs increased with increasing proportions of template- matching letters. Again, this trend was mostly linear [F(1, 48) = 42.29, p < .001, ${\eta }_p^2$ = .47], although the quadratic trend was reliable [F(1, 48) = 13.72, p = .001, ${\eta }_p^2$ = .22].

Low prevalence group.

Again, the main effects of Target Color and TCP were reliable, but the interaction was not. Compared to the HP group, the RT pattern reversed for template- matching and mismatching targets, with faster search (by 239 ms) when targets mismatched the template. RTs followed a quadratic trend across levels of TCP, and only the quadratic trend was reliable [F(1, 49) = 8.19, p = .006, ${\eta }_p^2$ = .14; linear: F(1, 49) = 0.16, p = .692].

Discussion

Similar to the pattern in Experiment 1, the confirmation and prevalence biases were both observed, and either cooperated or competed to affect search RTs. In the HP group, there was a large prevalence effect (477 ms) and a linear rise in RTs as proportions of cue-colored objects increased, consistent with the confirmation bias. In the LP group, the prevalence effect was 50% smaller (239 ms; t(97) = 10.98, p < .001, d = 2.21), although quadratic trends again indicated flexible search. Thus, as in Experiment 1, although the “confirmatory pattern” (i.e., the linear rise in RTs across TCP) was not observed in the LP group, its design dramatically reduced the prevalence effect. As discussed earlier, the prevalence and confirmation biases were mutually reinforcing in the HP condition, but mutually conflicting in the LP condition. Although there was little direct evidence for the confirmation bias in the LP condition, the overall data pattern suggests that it affected search behavior.

Regarding the balanced condition, the procedures in Experiment 2 more faithfully followed those from Rajsic et al. (2015), such that people terminated search and classified targets with the same key response. The balanced condition now replicated their findings, with RTs suggesting that people preferentially searched among cue-matching objects. It seems unlikely that the different outcomes in Experiments 1 and 2 (and Rajsic et al., 2015) reflect statistical power. Rajsic and colleagues have replicated and extended their results, minimizing concerns about false-positive findings. In Experiment 1, we continuously reminded people of the cued target template color, in the verification screen following each trial. Presumably, such reminders would exaggerate confirmatory bias by increasing the cued template’s cognitive availability (Tversky & Kahneman, 1973). Indeed, Rajsic et al. (2015, Experiment 2) observed the confirmation bias when people were reminded of the target in every trial. Instead, we suggest that the difference arises from response selection: Terminating search by generating a choice RT is cognitively “expensive,” relative to generating a simple RT (Luce, 1986). Given a goal to quickly press one of two possible keys, the cue template might “prime” its manual response, facilitating consistent trials. We consider this further in the General Discussion.

In both Experiments 1 and 2, we observed patterns suggesting that the prevalence and confirmation biases co-exist during search, either reinforcing or conflicting with each other. In the HP conditions, wherein both biases “work together,” there were large prevalence effects (≈ 430 ms) and strong linear trends indicating confirmatory search. In the LP conditions, wherein the biases conflict, prevalence effects were smaller (≈ 250 ms) and quadratic trends suggested more flexible search. Overall, these results suggest that (1) the nearly intractable perseverative strategy observed by Rajsic et al. (2015) can be modulated by target prevalence, but (2) people still apply the confirmation strategy, even when cue-colored targets are extremely rare. We note, however, that our evidence for the confirmation bias in the LP condition was somewhat indirect, as its modulation of the prevalence effect was our main indicator. In Experiment 3, we examined eye-movements to better understand these patterns, investigating both attentional guidance and perceptual decision-making.

Experiment 3

In Experiment 3, we sought to better understand the cognitive bases of the confirmation and prevalence effects. Although the RT data in Experiments 1 and 2 showed both effects, their underlying sources could be attentional guidance, perceptual decision-making, or a combination. Recently, Rajsic et al. (2017) replicated their confirmation bias experiments using eye-tracking and localized the bias to attentional guidance: People preferentially inspected letters that matched the initial cue color (although this bias decreased slightly when more distractors matched the cue). We therefore anticipated that, when initial cues are extremely reliable under high prevalence, people would almost exclusively inspect template-matching letters. However, given the apparent combination of the confirmation and prevalence biases in the LP condition, we expected eye movements to reveal dissociated sources: Confirmation bias should arise in attentional guidance (object selection; Rajsic et al., 2017) whereas prevalence should primarily affect perceptual decision-making (Hout et al., 2015). In the LP condition, people should still be biased to inspect cue-consistent items (although perhaps to a lesser degree than the balanced or HP conditions), despite their rarity as targets, but should more fluently identify the frequent, template-mismatching targets.

Method

Participants

Ninety-six Arizona State University students passed the eye-tracking calibration and completed the full task. All procedures were approved by the Institutional Review Board, and all participants provided informed consent.

Apparatus

All eye-tracking data were collected on a Dell Optiplex 755 dual-core PC (2.66 and 1.97 GHz) with 3.25 GB of RAM, on Windows XP. The monitor used to display stimuli was a NEC MultiSync 2111 CRT with a 20-inch viewable display, running at 1280 × 1024 resolution at 75 Hz and rendered using an ATI Radeon HD 2400 XT video card. We tracked eye movements using the SR Research Eyelink 1000 desktop system (SR Research, Ltd., Mississauga, Ontario, Canada), recording the left eye at 500 Hz (viewing was binocular). Head movements were minimized using a chin rest. E-Prime 2.0 software was used to present the stimuli, and data were exported using SR Research’s Data Viewer software.

Stimuli and Design

All stimuli and manipulations were identical to those in Experiment 2.

Procedure

The procedure was identical to Experiment 2, but with additional eye-tracking routines: Prior to the experiment, all participants completed a 9-point calibration to ensure accurate tracking. To verify accurate tracking in each trial, the fixation cross displayed prior to each search was gaze-contingent: Participants fixated this cross for 500 ms to begin each trial. Additional calibrations were conducted as necessary, if the gaze-contingent fixation cross revealed calibration drift.

Results

Because eye-tracking data is typically powerful, we typically oversample during data collection, then apply stringent criteria regarding data retention (as problematic gaze recordings are often undetected during data collection). Prior to analysis, visual filtering was conducted using Data Viewer software, which displays a representation of each trial and all eye fixations. Fourteen participants with systematic fixation deviations (e.g., a consistent downward shift) were excluded. An additional seven participants were excluded because they made very few saccades (most likely using covert attention while fixating the center), creating numerous empty cells for eye-tracking measures. Five participants were excluded for various behavioral issues (e.g., moving off the chin rest) and three were excluded for outlier performance. In the final sample, the balanced-prevalence group had 22 participants, with 96% group accuracy. The HP group had 22 participants, with 97% group accuracy. The LP group had 23 participants, with 96% group accuracy. The results are shown in Figure 6, with mean accuracy in the upper row, and median RTs in the lower row. Only valid eye-tracking trials were included for the RT analyses (e.g., trials with no object fixation were excluded); 95% of correct trials were retained.

Figure 6.

Mean accuracy and median RTs for each prevalence group in Experiment 3, as a function of Target Color and TCP. Reliable pairwise comparisons are indicated with asterisks (where applicable) and error bars represent SEM.

Accuracy.

Supplemental Table S7 summarizes the accuracy ANOVA results, with an omnibus analysis for the entire design, followed by focused analyses on specific conditions. As suggested by the upper row of Figure 6, the main finding of interest was the robust interaction of Target Color x Prevalence Group (omnibus ANOVA, F(2, 64) = 40.2, p < .001, ${\eta }_p^2$ = .56). Accuracy was equivalent when prevalence was balanced, but systematically decreased for whichever color was less often correct. There were no other effects of interest.

Response Times.

Supplemental Table S8 summarizes the ANOVA results for RTs. Examining Figure 6, we replicated the general patterns from Experiment 2, although the confirmation bias was somewhat less compelling (in terms of linear versus quadratic patterns). This was especially true for the balanced condition, a point we revisit in the discussion. In the LP and HP conditions, we observed strong prevalence effects with indirect evidence of the confirmation bias, as evident from the prevalence effect waxing and waning.

Balanced prevalence group.

The main effects of Target Color and TCP and their interaction were all reliable. Examining the lower-left panel of Figure 6, the search RT pattern closely resembles the balanced group from Experiment 1. In template-matching targets, the TCP effect was reliable [F(2, 20) = 26.39, p < .001, ${\eta }_p^2$ = .73], as was the linear trend [F(1, 21) = 54.78, p < .001, ${\eta }_p^2$ = .72], but the quadratic trend was null [F(1, 21) = 3.51, p = .075]. In template-mismatching trials, the TCP effect was again reliable [F(2, 20) = 10.27, p = .001, ${\eta }_p^2$ =.51] and the trend was quadratic [F(1, 21) = 17.92, p < .001, ${\eta }_p^2$ = .46], but not linear [F(1, 21) = 1.91, p = .181]. Template-matching and mismatching trials reliably differed at the .25 and .50 levels of TCP (both t > 2.7), but not in the .75 condition. Thus, despite using the choice-RT method in Experiment 3, results from the balanced group did not clearly indicate confirmatory search. Nevertheless, the upcoming oculomotor results provide evidence that people retained a confirmation bias, even in the balanced condition.

High prevalence group.

Both the main effects of Target Color and TCP were reliable but their interaction was not. The lower-middle panel of Figure 6 shows a large prevalence effect (458 ms). Both the linear [F(1, 21) = 13.07, p = .002, ${\eta }_p^2$ = .38] and quadratic [F(1, 21) = 19.75, p < .001, ${\eta }_p^2$ = .49] trends were reliable for TCP. Surprisingly, the quadratic trend was better- fitting, due mainly to the template-mismatching trials.

Low prevalence group.

The main effect of TCP was reliable, but Target Color was marginal and their interaction was null. As the lower-right panel of Figure 6 indicates, the prevalence effect essentially disappeared (or was offset by the confirmation bias), with similar RTs for rare and common targets. Nevertheless, people were numerically faster to detect more common targets (by 76 ms). There was an overall quadratic trend for TCP [F(1, 22) = 9.79, p =.005, ${\eta }_p^2$ = .31], with no linear trend [F(1, 22) = 0.83, p = .373].

The RT results from Experiment 3 generally replicated the previous experiments, although the hypothesized conflict between the prevalence and confirmation biases appeared more balanced in the LP condition. The difference between Target Color conditions in the LP group was 65 ms, compared to a 458 ms difference in the HP group [t(34.4) = 7.61, p < .001, d = 2.28; equal variances not assumed]. In our prior experiments, prevalence effects were robust in the LP conditions, although clearly diminished relative to the HP conditions. In Experiment 3, we observed the same asymmetry, to a larger degree. Notably, the prevalence effect in the LP condition was still reliable in accuracy, and was marginal (p = .077) in RTs. Across all three experiments, there has been mixed RT evidence for the confirmation bias, with occasional linear patterns (which are considered diagnostic) and interactions with prevalence. The main goal for Experiment 3 was to move beyond RTs, adding eye-tracking data for greater leverage.

Eye-tracking Results

By its nature, eye-tracking affords many potential dependent measures, regarding the location and timing of fixations. Our present goals were to investigate attentional guidance and perceptual decision-making, as both processes have been previously linked to the confirmation and prevalence biases. Following Rajsic et al. (2017), we measured attentional guidance by (1) overall counts of object inspections, (2) selectivity, a measure of attentional bias toward cue-colored stimuli, and (3) inference, essentially the inverse of target inspections. This estimates flexible strategy use, as decisions are made without direct target fixation. To assess perceptual decision-making, we tested (1) inspection durations on the first fixated object (see Rajsic et al., 2017), which indicate how efficiently people reject non-targets. In the present case, inspection durations produced no clear findings of interest. The results are shown in the supplemental materials, but are excluded from this discussion. We also tested (2) decision times, the time between target fixation and key-press, and (3) perceptual failures, misses that occur despite direct target fixation. All analyses included only valid eye-tracking trials, and all analyses (except perceptual failures) included correct trials only.

Object Inspection Counts.

Following Rajsic et al. (2017), we defined a single object inspection as a fixation (or a collection of nearby fixations) falling within 2.5° of the center of a stimulus, prior to exiting its invisible radius. The results shown Figure 7 are counts of how many objects were examined per trial (out of 8 possible), as a function of target color and TCP. Full ANOVA results are reported in Supplement Table S9, and are briefly described below. The key findings are easily summarized by reference to Figure 7. Examining the functions for cue-matching targets (i.e., the green lines in all three panels), it is apparent that inspection counts were nearly identical across all three groups. Additionally, when targets were cue-colored, people examined fewer objects overall (comparing the red and green functions). This suggests a general preference to examine cue-matching objects, as Rajsic et al. (2017) reported. In the present case, this pattern is particularly surprising for the LP group. Despite cue-colored targets only appearing 15% of the time, LP participants inspected cue-colored objects at the same rate as HP participants.

Figure 7.

Mean overall inspection count for each prevalence group in Experiment 3, as a function of Target Color and TCP. Reliable pairwise comparisons are indicated with asterisks, where applicable, and error bars represent SEM.

In the omnibus ANOVA, we found main effects of Target Color and TCP, and reliable Target Color × Prevalence Group and Target Color × TCP interactions. In the analysis including only HP and LP groups, these same effects were again reliable. Given the interactions with Prevalence Group, we conduct further within-group analyses:

Balanced group.

Both main effects (Target Color and TCP) and their interaction were reliable. We therefore tested polynomial contrasts for TCP: For template-matching targets, the TCP effect was reliable [F(2, 20) = 36.24, p < .001, ${\eta }_p^2$ = .78], with reliable linear [F(1, 21) = 72.76, p < .001, ${\eta }_p^2$ = .78] and quadratic [F(1, 21) = 6.57, p = .018, ${\eta }_p^2$ = .24] trends, although the linear trend was better-fitting. For template-mismatching targets, the overall effect was reliable [F(2, 20) = 6.81, p = .006, ${\eta }_p^2$ = .41], but only the quadratic trend was reliable [F(1, 21) = 13.06, p = .002, ${\eta }_p^2$ = .38]. Template-matching and mismatching trials reliably differed at the .25 and.50 levels of TCP (both t > 2.5) but not at the .75 level.

High prevalence group.

Both main effects and their interaction were reliable. For trials with template-matching targets both the linear [F(1, 21) = 70.59, p < .001, ${\eta }_p^2$ = .77] and quadratic [F(1, 21) = 8.40, p = .009, ${\eta }_p^2$ = .29] trends were reliable, although the better-fitting trend was linear. In template-mismatching trials, only the quadratic trend was reliable [F(1, 21) = 9.92, p = .005, ${\eta }_p^2$ = .32]. Template-matching and template-mismatching trials reliably differed at all three levels of TCP.

Low prevalence group.

The main effect of TCP and the Target Color × TCP interaction were reliable. In trials with template-matching targets, the main effect of TCP was reliable [F(2,21) = 6.93, p = .005, ${\eta }_p^2$ = .40]. Both the linear [F(1, 22) = 8.34, p = .009, ${\eta }_p^2$ = .28] and quadratic [F(1, 22) = 5.28, p = .031, ${\eta }_p^2$ = .19] trends were reliable, although the better-fitting trend was linear. In template-mismatching trials, the overall ANOVA was reliable [F(2, 21) = 5.78, p = .010, ${\eta }_p^2$ = .36], and only the quadratic trend was reliable [F(1, 22) = 10.66, p = .004,${\eta }_p^2$ = .33]. Template-matching and mismatching trials only differed at the .25 level of TCP (t = 3.9). Overall, the object inspection findings resembled the RT findings, showing simultaneous evidence for both the confirmation and prevalence biases. The remaining analyses help clarify their underlying mechanisms.

Selectivity.

Following Rajsic et al. (2017; also Cohen, 1960), we computed a bias measure to template-matching stimuli that accounts for chance,

$$p(bias)=\frac{p(observed)-p(chance)}{1-p(chance)}$$ (1)

where p(observed) is the proportion of inspections to template-matching stimuli, and p(chance) is the proportion of template-matching letters in the display (i.e., TCP). A value of 0 indicates no bias to inspect template-matching stimuli; a value of 1 indicates complete bias to inspect template-matching stimuli. Negative values indicate a bias away from template-matching stimuli. When the numerator is negative, the denominator is changed to,

$$p(bias)=\frac{p(observed)-p(chance)}{1-(1-p(chance))}$$ (2)

such that p(bias) represents a bias to inspect template-mismatching stimuli (Rajsic et al., 2017; J. Rajsic, personal communication, March 2017). Also following Rajsic et al., we calculated p(bias) separately for the first inspection of each trial, and then for all subsequent inspections, calling this new variable Epoch. Simplified results are shown in Figure 8, showing that participants were generally biased to inspect template-colored objects. More elaborate figures (showing separate template color proportions) are provided in the supplemental materials.

Figure 8.

Mean selectivity results for the balanced prevalence group of Experiment 3, as a function of Target Color, TCP, and Epoch. Error bars represent SEM.

Full ANOVA results for selectivity are in Supplemental Table S10. Because the results are straightforward, we highlight only the main findings. As shown in Figure 8, participants in all groups were biased to inspect template-colored objects (all main effects of Target Color were robust). As anticipated, this bias was stronger in the HP group, where both biases work together, and was reduced in the LP group. Nevertheless, the bias was still robust in the LP group [F(1,22) = 73.1, p < .001, ${\eta }_p^2$ = .77]. There were also clear signs that LP participants attempted to balance the countervailing confirmation and prevalence biases. First, their selectivity bias was relatively smaller than the HP group [F(1, 43) = 4.16, p < .05, ${\eta }_p^2$ = .09]. Second, their bias showed a different temporal pattern, with HP participants showing stronger selectivity in Epoch 1, and LP participants in Epoch 2, shown by a Prevalence Group x Epoch interaction [F(1, 43) = 7.56, p < .01, ${\eta }_p^2$ = .15].

Inference.

In the present paradigm, inference is possible: If the target is not encountered while searching for one color, it must be the other color. Figure 9 displays the proportions of trials (per group) in which targets were actually inspected. Examining the figure, several trends are apparent: Participants in all three groups used inference to similar degrees and preferentially fixated template-colored objects. In the omnibus ANOVA (Supplemental Table S11), there were reliable main effects of Target Color and TCP, and their interaction, but no main effects or interactions involving Prevalence Group.

Figure 9.

Inference (proportion of trials with a target inspection) results for Experiment 3, as a function of Target Color and TCP. Error bars represent SEM.

Figure 9 also shows Target Color × TCP interactions in all three groups. Collapsing across groups, in trials with template-matching targets, the overall ANOVA for TCP was reliable [F(2, 65) = 15.92, p < .001, ${\eta }_p^2$ = .33], as was the linear trend [F(1, 66) = 27.46, p < .001, ${\eta }_p^2$ =.29] but not the quadratic trend [F(1, 66) = 1.40, p = .241]. In trials with template-mismatching targets, the overall ANOVA was reliable [F(2, 65) = 18.94, p < .001, ${\eta }_p^2$ = .37], as were the linear [F(1, 66) = 35.98, p < .001, ${\eta }_p^2$ = .35] and quadratic trends [F(1, 66) = 9.61, p = .003, ${\eta }_p^2$ =.13], although the linear trend was better-fitting. As observed by Rajsic et al. (2017), people inspected template-colored stimuli by default, using inference as a backup strategy, even in the LP group. Again, despite rarely encountering template-matching targets, people strongly preferred to inspect such targets.

Decision Times.

Decision times (shown in Figure 10) reflect how fluently people identify targets, defined as the duration between target fixation and key-press (Hout & Goldinger, 2012). One participant from the balanced group and three from the HP group were excluded due to missing data. The analyses (see Supplemental Table S12) included 21, 19, and 23 participants in the prevalence groups, respectively. Because a target must be fixated in order to calculate this measure, along with a correct response, 79% of trials were retained for the decision time analyses after filtering. We observed a reliable main effect of TCP and a Target Color × Prevalence Group interaction. Examining Figure 10, the finding of interest is the reversal in decision times across the LP and HP groups [F(1, 40) = 7.36, p = .01, ${\eta }_p^2$ = .16]. This result shows that, although participants were consistently biased to examine template-colored objects, they were faster to perceptually appreciate targets that appeared more frequently. This is a key indication that the confirmation and prevalence biases may operate at “different stages” in visual search.

Figure 10.

Decision times for each prevalence group in Experiment 3, as a function of Target Color and TCP. Error bars represent SEM.

Perceptual Failures.

Having observed prevalence effects that were increased (high prevalence) or decreased (low prevalence) by the confirmation bias, we sought to determine how often people missed targets despite directly fixating them. This final analysis investigated accuracy only for trials in which targets were fixated, the same sample from the decision time analysis. The results are shown in Figure 11; ANOVA results are provided in Supplemental Table S13. As shown, target prevalence affected miss rates, without any clear influence of the confirmation bias. In the analysis comparing the HP and LP groups, only the Target Color × Prevalence Group interaction was reliable. Overall, people often failed to detect rare targets that were directly inspected, regardless of the initial template. Although this effect may partly reflect motor errors resulting from habitually pressing the more frequent response (Fleck & Mitroff, 2007), a perceptual failure account accords with prior analyses of perceptual decision making and previous studies that controlled for motor errors (e.g., Hout et al., 2015).

Figure 11.

Perceptual failures (accuracy in trials with a target inspection) for each prevalence group in Experiment 3, as a function of Target Color and TCP. Error bars represent SEM

Discussion

Experiment 3 was designed to investigate the cognitive underpinnings of our previous behavioral results from Experiments 1 and 2. Again, the confirmation bias was exaggerated (in both RT and accuracy) when reinforced by high target prevalence. Participants’ eye movements revealed a strong attentional bias to inspect template-colored items and faster identification of template-colored targets, relative to template-mismatching targets. And, when we reduced the frequency of template-colored targets, we again found a conflict between the prevalence and confirmation biases. In Experiments 1 and 2, we observed a consistent pattern: Search accuracy and RTs were strongly affected by target prevalence, but the prevalence effect was modulated by the confirmation bias, being smaller in magnitude when template-mismatching targets were more common. In Experiment 3, we observed an exaggerated version of this same pattern, especially in search RTs. The oculomotor measures clarify how these separate biases exert their effects.Even when template-matching items were rare, people still disproportionately inspected template-colored items, suggesting that attention is automatically drawn toward cue-matching objects. This was evident in the selectivity analyses and inspection counts. These findings support Rajsic et al.’s (2017) suggestion that the confirmation bias arises in attentional guidance. Regarding attentional mechanisms, there is theoretical leverage to the surprising finding that, even when targets only appear in cued colors 15% of the time, people still preferentially examine cue-colored objects in each display. This appears inconsistent with selection history accounts of visual attention (e.g., Awh, Belopolsky, & Theewues, 2012; Anderson, 2017; Failing & Theeuwes, 2018), which would predict that reward history in the LP condition (i.e., the cue color is almost never correct) would train people to allocate attention toward the non-cued color. It appears that priming from the visual cue is too salient for selection history to negate, at least in the current paradigm. In contrast to guidance, prevalence rates mainly affected perceptual decision-making. People struggled to appreciate rare targets once they were fixated, with higher miss rates and slower responses during hits.

Considering the RT results, we must acknowledge that the balanced condition of Experiment 3 (lower-left panel of Figure 6) did not produce the clear confirmation-bias pattern that was anticipated (both from Rajsic et al., 2015, and our own Experiment 2). Instead, the data appear mixed, showing some confirmatory and some flexible search. The precise reason for this pattern is unclear, especially because further evidence (gaze behavior) showed clear evidence for the confirmation bias in the balanced condition. In the General Discussion, we address this disparity further, regarding the value of oculomotor measures.

General Discussion

In three experiments, we examined the relationship between the low prevalence effect and confirmation bias in visual search. Previous research by Rajsic et al. (2015) suggests that, when people are cued with an initial target template color, they rigidly adopt that template as a mental guide when seeking targets. Although the seemingly optimal strategy would be to restrict attention to items comprising the minority color per display (enabling search through fewer items), people adhered to a template-seeking search pattern. This held true even when people were informed of the optimal strategy, suggesting that it is automatic. We sought to determine how another stubborn, automatic phenomenon in visual search, the prevalence effect, would interact with the confirmation bias. In the introduction, we introduced three potential result patterns. These were (1) dominant confirmation bias, wherein the confirmation bias would remain, even when rarely helpful; (2) dominant prevalence bias, wherein selection history would shape search behavior, overriding the confirmation bias in the LP conditions, and (3) flexible search, wherein people would search optimally in the balanced condition, use the confirmation bias in the HP condition (wherein it was optimal) and avoid it in the LP condition. Of these potential outcomes, we can reject hypothesis 2 (dominant prevalence), as attention guidance was reliably driven by cue colors. Hypotheses 3 does not fare particularly well, although there was clearly a stronger confirmation bias in the HP conditions. The best account is hypothesis 1, in which confirmation bias remains, throughout. Prevalence effects were strong, but appeared to affect decision-making, rather than guidance.

All the present experiments suggest that the confirmation and prevalence effects operate on different aspects of visual search. When template-consistent targets occurred 85% of the time, the confirmation bias grew stronger. This finding was not a surprise. However, when template-consistent targets only occurred 15% of the time, people maintained a strong bias to inspect items template-matching items, suggesting an automatic attentional bias (Rajsic et al., 2017). Target prevalence, on the other hand, mainly affected the perceptual fluency of target identification (as in Hout et al., 2015). Because these biases affect separate aspects of visual search, they can either cooperate or compete with respect to overall search RTs.

In visual search, the confirmation bias is the tendency to seek template-matching, even when a disconfirmation strategy would be more efficient. In Experiment 3 (and Rajsic et al., 2017), eye-tracking revealed that people usually attempted to “confirm” the initial template color, even when such targets were quite rare. Analogous tendencies are well-established in more deliberative cognition, such as Wason’s card selection task (1966; 1968). In this task, people often display a general matching bias, wherein they match responses to the specific instances initially posed in the problem (Evans, 1972; 1998). Indeed, Rajsic et al. (2017) suggested that confirmation bias in visual search may reflect a general matching bias. This hypothesis is consistent with the broader literature, as the contents of visual working memory often drive attention allocation (Duncan & Humphreys, 1989; Olivers et al., 2011; Schmidt & Zelinsky, 2009; Wolfe, 2007; Wolfe, Cave, & Franzel, 1989). In the present case, the template matching bias was exaggerated in high-prevalence conditions, wherein initial templates robustly predicted actual target colors. In low-prevalence conditions, the cued template was unreliable, yet people persisted in seeking template-colored items, as indicated by eye movements. Participants also showed statistical learning, such as changes in perceptual classification. The results show that attention is guided to the cued objects, but people are more “prepared” to perceive more frequent targets after fixation (Hout et al., 2015).

The capricious confirmation bias?

Although the confirmation bias in visual search was initially described by Rajsic et al. (2015) as a stubborn phenomenon, further research (including the present study) has elucidated its boundary conditions. Rajsic et al. (2015) found that the bias persisted even when people were instructed to adopt a seemingly more efficient strategy, searching among the minority subset in each display. Given the theoretical assumption that automatized processes (e.g., visual search) are near-optimal, confirmatory search is likely more efficient, despite intuitions. Although scanning the minority subset per display appears mathematically efficient, cognitive efficiency is determined by the relative costs of different potential operations. To enact the minority subset strategy, when each display appears, the observer would have to (1) determine which color is the minority subset, (2) plan to search for that color, then (3) initiate search. Because step one already requires visual inspection of the entire display, this three-step process requires a person to search twice (or some approximation). Given the predictable displays (eight letters in a circular array), it likely requires less cognitive demand to simply search. As noted earlier, moving spatial attention is “cheap” whereas planning search is “costly” (Wolfe et al., 2004). With the onset of each display, people could assess the relative proportions of the display colors, or they could simply look for the more cognitively available target. Although the former strategy appears mathematically optimal within-trials, it requires unstable information across trials and is therefore less efficient. Finally, it is established that visual working memory preferentially guides attention to template-consistent objects (e.g., Beck et al., 2012; Duncan & Humphreys, 1989; Olivers et al., 2011; Schmidt & Zelinsky, 2009; Wolfe, 2007), which would encourage the template-confirmation pattern.

By this analysis (which echoes Rajsic et al., 2015; 2017), the confirmatory search bias arises because – given certain task parameters – it is the most optimal way to quickly locate targets. Supporting this view, when the procedures are changed, the bias can be reduced or eliminated. The confirmation bias is reduced when participants preview each search display (Rajsic et al., 2015), or when attentional deployments are made costly via gaze-contingent or mouse-contingent displays (Rajsic et al., 2017). In these cases, the previously optimal “just look” strategy becomes ineffective, driving people to a more nuanced approach. In a recent study, Walenchok (2018; Walenchok & Goldinger, 2018) used randomized spatial arrays (rather than constant circles) with varying set sizes. This procedure again reduced the confirmation bias in a balanced prevalence condition.

In the present study, we unintentionally found another way to moderate the confirmation bias. Specifically, in Experiment 1, we enacted a seemingly modest change in the keyboard responses, relative to Rajsic et al. (2015). Rather than have participants simultaneously terminate search and classify targets via two-button choice responding, we had them terminate search using the space bar, then classify targets afterwards. By making this change (which was chosen to accommodate the prevalence manipulation), we accidentally reduced the confirmation bias, specifically in the balanced conditions that were meant to replicate Rajsic et al. (2015). In search RTs, the anticipated confirmatory pattern (i.e., linearly rising RTs with increasing proportions of cue-colored objects) was not observed. Once we reinstated the choice-RT procedure, the expected pattern emerged.

We have recently extended these findings. Walenchok (2018) conducted an experiment that was near-identical to Experiment 3 in the current experiment, but using the space-bar response. As in the present report, there was little evidence for a confirmation bias in the RT data, but eye-movements revealed a clear preference to examine cue-colored objects. The eye- tracking results resembled the current Experiment 3 in essentially all regards. In another experiment, Walenchok (2018; Walenchok & Goldinger, 2018) tested the confirmation bias using randomized displays of real-world objects. One condition used the space-bar response for search termination; the other used choice-RT responding. The space bar condition showed little evidence for the confirmation bias, but it was reinstated in the two-button condition. We are reasonably confident about the empirical pattern, which motivates two points of discussion, one methodological and one theoretical. They are not mutually exclusive.

First, our data suggest that search RTs may be insufficiently sensitive for detecting the confirmation bias. Across experiments, it was consistently challenging to categorize our RT patterns as being more linear or more quadratic, requiring a degree of qualitative judgment in many cases. For any single function, the proportions of explained variance can adjudicate between linear and quadratic. In many conditions, however, the RTs for template-matching trials appeared more linear, while the RTs for template-mismatching trials appeared more quadratic. The logic from Rajsic et al. (2015) appears perfectly sound, but our data rarely conformed to clean schematic patterns. More important, even when the balanced prevalence conditions suggested flexible search (i.e., the absence of confirmation bias), the broader results told a contradictory story. In all experiments, we found powerful interactions between the prevalence and confirmation biases. The prevalence effect was either large or small, when the biases were configured to cooperate or compete, respectively. In our view, this positive evidence for the confirmation bias should outweigh the null evidence from RT functions. Conversely, the oculomotor measures (e.g., the selectivity index) appear more sensitive, showing exactly where peoples’ eyes “want to go” in each trial. In Experiment 3, we found oculomotor evidence for the confirmation bias in the balanced condition, even when the RT evidence was less compelling. For future studies, we recommend eye-tracking methods to help avoid potential false-negative RT results.

Second, of greater theoretical interest, our results suggest that response priming is partly responsible for the confirmation bias reported by Rajsic et al. (2015; 2017). In both the original and modified versions of the task, the initial search cue presents a colored patch (see example in Figure 1). Assuming this cue color is encoded into visual working memory, it should guide attention to matching objects in the search display (e.g., Wolfe, 2007), leading to a confirmation (or matching) bias. The present findings suggest there is more to the story. We found that the confirmation bias is stronger when people perform the choice-RT version of the task, relative to terminating search with a nondiscriminating space-bar press. Why would visual search change when the response requirements change? Our hypothesis is that, in the two-button task, the colored search creates a VWM template and it also primes the color-consistent button response. If one button corresponds to green and the search cue is green, basic principles of stimulus- response compatibility (e.g., Fitts & Seeger, 1953) predict that “green” will become primed, as the prepotent response. Once the cue-consistent response is primed, visual search is more strongly biased toward response-consistent objects in the display. This is another example of optimization: As noted earlier, moving spatial attention is fast and easy (Wolfe et al., 2004), whereas programming motor responses is slower (Donders, 1868; Sternberg, 1966). In the space-bar version, no such response priming is possible, reducing the attentional draw of cue- colored objects. Although this is a novel observation, it is consistent with previous experiments on the confirmation bias, as search tendencies appear tightly attuned to overall cognitive costs. It is also consistent with findings that people optimize biological energy in visual search: For example, Solman and Kingstone (2014) found that people would use memory to guide search if head movements were required, but would make unplanned fixations if only eye movements were required. Small changes in energetic requirements can qualitatively alter human visual search behavior.

To conclude, when people search for objects, the default strategy is template matching or confirmation: People typically seek what they are shown. When the template information is highly reliable, people become more persistent in this confirmatory search. However, even when the template is highly unreliable, people continue to seek visual instances that confirm what is mentally salient. Target prevalence can dramatically change search speed and accuracy, but does little to counteract the nature of attentional guidance.

Public Significance:

The human ability to visually scan for important targets has vital importance. An obvious example is baggage screening for security, as agents must efficiently scan numerous bags for potential weapons that rarely occur. Although it seems intuitive that rare objects (e.g., guns) would “jump off the screen” and capture attention, people in fact struggle to notice rare targets. This robust finding carries frightening implications, as dangerous objects can “hide in plain sight.” In the present research, we coupled this target prevalence phenomenon with another habit of human visual search – the confirmation bias, which shows that people preferentially seek targets that match recently seen examples (e.g., green things), even when this requires consideration of more potential targets. We found that these biases operate at separate stages in visual search, either cooperating or competing to affect behavior.

Appendix A: A Priori Power Analyses

Table A1:

Sample size required to detect each analogous behavioral effect from Rajsic et al. (2015) and Hout et al. (2015) at power = .80

Analysis and Original Effect	Original N	F	p	${\eta }_p^2$	N needed
Rajsic et al. (2015) accuracy results, Experiment 3
*Target Color (TC)	12	<1.02	>.380	<.09	42
*Template Color Proportion (TCP)	12	<1.02	>.380	<.09	36
*†TC × TCP	12	<1.02	>.380	<.09	101
Rajsic et al. (2015) search RT results, Experiment 3
Target Color (TC)	12	7.36	.020	.40	9
Template Color Proportion (TCP)	12	8.20	.002	.43	9
*†TC × TCP	12	0.38	.690	.03	315
Hout et al. (2015) accuracy results, Experiment 1b
Target Color (TC)	43	18.82	<.001	.32	12
*Prevalence Group (PG)	43	1.48	.240	.07	132
TC × PG	43	3.30	.047	.14	33
Hout et al. (2015) search RT results, Experiment 1b
Target Color (TC)	43	41.36	<.001	.51	9
*Prevalence Group (PG)	43	0.08	.919	<.01	957
TC × PG	43	7.05	.002	.26	18

Note: Sphericity was assumed, where applicable. All original effect sizes listed were used for each power analysis.

^*Interpret with caution, as original effect was not significant.

^†Due to G*Power limitations, we used the most conservative option, treating the within-subjects interaction as between-subjects in the power analysis.