Normal blindness: when we Look but Fail To See

Abstract

Humans routinely miss important information that is “right in front of our eyes”, from overlooking typos in a paper to failing to see a cyclist in an intersection. Recent studies on these “Looked But Failed to See” (LBFTS) errors point to a common mechanism underlying these failures, whether the missed item was an unexpected gorilla, the clearly defined target of a visual search, or that simple typo. We argue that normal blindness is the by-product of the limited-capacity prediction engine that is our visual system. The processes that evolved to allow us to move through the world with ease are virtually guaranteed to cause us to miss some significant stimuli, especially in important tasks like driving and medical image perception.

When we miss what’s right in front of our eyes

Most readers of Trends in Cognitive Sciences will be familiar with Simons and Chabris’ classic gorilla experiments [1]. Observers watch two groups of actors passing a ball around. The observers are told to count the number of times that the white-shirted team passes the ball. In the middle of this game, another actor in a gorilla suit walks into the middle of the game, pounds on her chest, and leaves. When asked, after the video clip is done, observers can offer reasonable answers about the number of passes. However, even when asked in several different ways, about 50% of observers fail to report the gorilla. Those who missed it are surprised when shown the gorilla. How could they have been so ‘blind’? It was clearly visible but, somehow, they didn’t see it. Now consider a different task. You dutifully proofread your latest paper, but it comes back later with several obvious typos marked. The misspellings were clearly visible, but you missed them. Finally, imagine doing a standard visual search task; perhaps searching for a T among some number of Ls [2]. The stimuli are visible until the observer responds. Nevertheless, observers will typically reply that no target is present on 5–10% of trials containing a clearly visible “T”. These all seem like different errors – what do they have in common? The gorilla task is a canonical example of inattentional blindness [3] [4] [5]. Problems with proofreading might be considered “satisfaction of search” errors [6, 7] [8] [9] or, perhaps, vigilance failures [10] [11, 12]. Miss errors in visual search are often thought of in the context of the problem of search termination (i.e., when to end an unsuccessful search [13], [14]). Though generally thought of and studied as different phenomena, each of these is an example of a “Look But Fail To See” (LBFTS) error [15]: a broad class of errors, defined by the failure to respond successfully to stimuli that are unambiguously visible and “right in front of your eyes”.

LBFTS errors might occur in an ‘incidental’ setting [16], when observers are looking for or attending to something else, as in the gorilla example. Alternatively, these errors can occur in an ‘intentional’ setting, as when observers miss a perfectly visible T during an intentional search for a T. Some tasks lie in between. For instance, radiologists are asked to report ‘incidental findings’ that might be clinically significant [17]. For example, a radiologist, looking for lung cancer, commits a potentially dangerous LBFTS error if they miss visible evidence of pneumonia or vice versa [18, 19]. In much more artificial situations, radiologists and others have been shown to miss gorillas in the context of performing their expert searches [20, 21] [22]. In the case of incidental findings, radiologists are specifically looking for signs of one pathology but know that there is a broader list of potential targets that might be present. In contrast, when a radiologist misses a gorilla in the lung [20], the situation is different because gorillas are not on the broader list.

Even though these and other LBFTS situations seem like distinct phenomena, we argue, based on recent work, that they can all be seen as products of the same normal mechanisms of attention and object recognition. Specifically: (1) Observers only select a subset of what they could process on each fixation (though they are not blind to the rest of the visual input; (2) Even the items that are selected by attention will be missed if too little time is given to their processing; (3) The processes that give rise to routine visual awareness routinely persuade us that we have seen more than we have actually seen; and (4) Attentional guidance (attentional set) can guide observers away from targets as well as toward them. Taken together, these factors produce a state of ‘normal blindness’ that has significant implications for the way people function in the world.

Why do we care?

While missing a gorilla in an odd video or a typo in a manuscript are not serious errors, the family of LBFTS errors is broad and consequential. The term, LBFTS, comes from research on road collisions [15] (Box 1). In “On-the-spot” interviews, after a collision drivers often insist that, yes, they did look at the location where the pedestrian or motorcyclist was located but, somehow, they just did not “see” the victim [e.g. 23, 24, 25]. In medicine, radiologists routinely end up in court [26], sued for malpractice when they fail to report some clinically significant finding that is “retrospectively visible” [27], meaning that the problem is visible when attention is drawn to it later. When people talk about ‘blindness’ in a clinical sense, they are usually referring to a devastating loss of visual function that has profound effects on the individual. By contrast, the “Normal blindness” that we are discussing here is not devastating at the individual level, because, by definition, it is part of normal experience. However, because it is normal the societal cost adds up. Developing a unified account of LBFTS errors and determining how to mitigate these errors promises to improve clinical outcomes, road safety, and, perhaps, even our ability to find typos.

Box 1: Looked but failed to see errors in driving.

The term “looked but failed to see” is borrowed from road safety research, where it was coined to describe drivers’ post-collision self-reports in which they reported that they had just not seen what they collided with [87]. These errors are not due to reduced visual acuity, impaired visual functioning, or reduced visual field extent [88]. Research on driver behavior was describing these failures as attentional in nature, long before the term LBFTS was coined [89], suggesting that purely visual accounts are insufficient to explain why drivers make these errors.

Research on LBFTS errors on the road emphasizes the role of expectation in causing these errors; a driver who does not expect to have to share the road with a motorcycle [90] or a cyclist [91] is less likely to see such fellow road users. Conversely, in environments where motorcycles or cyclists are common, drivers may be more likely to see them [92, 93]. Interestingly, the driver’s attentional set and level of expertise in driving do not seem to protect against LBFTS errors [24]. Drivers often multitask and distract themselves with other tasks while driving (from adjusting the radio to texting on a smartphone) [94] and the attentional and cognitive demands of these secondary tasks could magnify the risk of LBFTS errors even if hands are on the wheel and eyes are on the road [95]. These errors are closely linked with where drivers look and for how long [96], though applied research has focused on what drivers look at - to the exclusion of what is in the functional visual field (FVF), what is beyond it [97], or how drivers plan their eye movements [98]. Recently, the classic LBFTS account has been questioned, with researchers asking whether this phenomenon can be attributed to memory failures (i.e., a “saw but forgot” error [99]).

Of course, LBFTS errors are not exclusive to driving. If researchers in multiple fields from medical image perception to basic cognitive science come to realize that LBFTS errors in one field are related to those in others, then the insights from different lines of research may converge on new ways to mitigate Normal Blindness.

A unifying model of LBFTS errors

Figure 1 offers a framework in which multiple types of LBFTS errors arise from the same underlying processes. To illustrate this model of LBFTS errors, it is useful to use a relatively complex task, as cartooned in Figure 1, as an example. Observers could be asked to find white 3s, a highly specific target, as well as any letters from the second half of the alphabet, a more categorical class of targets. The stimuli are presented in a noisy scene that, in this case, happens to be a mammogram. This is a hybrid search task, where participants are looking for more than one type of target [28]. Such a task is intended to mimic important properties of real-world tasks. For example, a driver might be attending to navigational cues, like signs and signals on the road and, at the same time, pointing out landmarks to the children. A radiologist might scan chest x-rays for signs of pneumonia while, at the same time, keeping watch for other “incidental findings” that might be clinically significant, like cancerous nodules [29]. Drivers and radiologists are expected to be able to do this, and people blame them when they fail, and it would be useful to have a model of why people make these errors that could suggest ways to prevent them.

Figure 1: A model of normal visual processing that will produce a range of LBFTS errors.

(A) Stimulus for a ‘mixed hybrid search’, where observers search for both specific targets (here, white 3s) and categorical targets (any letter from the second half of the alphabet). Plusses show hypothetical fixation points. Circles show hypothetical deployments of attention. (B) Making a decision about a selected item may involve many information-accumulating processes (represented by each panel in the top row). One process seeks evidence for “white 3s”. Another process, for letters in the second half of the alphabet, will take longer (higher decision boundary). Evidence for other “surprises” would require still more evidence. Once enough evidence accumulates in one accumulator and the decision boundary is reached, the observer can report the target. If an item was identified as a distractor or if an item-by-item timing criterion was reached (bottom row), processing of that item would be terminated. At that point, the selection is cleared and a new item is selected for further processing. (C ) At the level of the entire task, a limited number of items are selected into an asynchronous diffusion process, represented by the individual lines in the upper panel. The task ends if the targets are found or when a task-level timer reaches a quitting threshold (bottom panel in C).

We can’t fully process everything

Based on prior work, a story can be told about how this task might proceed. The eyes will be pointed at some location; e.g. on the red plus in Figure 1a. When the eyes are directed at one location, there is a region surrounding that fixation within which a target can be identified. This is called the Useful Field of View [UFOV - 30] or Functional Visual Field [FVF - 31] – the terms are roughly equivalent. UFOV is more common in the applied literature; FVF, in the basic, cognitive literature. The size of the FVF will vary with the target. Acuity limits and crowding [32] will mean that, to identify an object, the point of fixation needs to be much closer to a small target in a noisy scene than to a large item in an empty field [33].

A standard, if often unstated, assumption is that people fully process whatever is inside the current FVF [e.g. 34]. This is incorrect [35]. To see why, consider the simpler search for Ts among Ls in Figure 2a. If you fixate on the red star, you will be able to identify the “T” to the left. It is, by definition, inside the current FVF. Without moving your eyes, you will not be able to identify the T to the lower right. It is outside your FVF. If the left-hand T marks the outer bound of the current FVF, there would be something like 17 items within the FVF. A standard fixation lasts about 200–250 msec. Many lines of evidence tell us that you simply cannot fully process all of those 17 items in a quarter of a second. Perhaps the most straightforward evidence comes from standard visual search experiments in which observers search for targets like the T among Ls in Figure 2A. In a typical experiment, the set size (the number of items) varies from trial to trial. If we measure the response time (RT) to say “yes, there is a target present” or “no, there is not”, we can calculate RT x set size functions. Those slopes estimate a rate of processing. The proper way to estimate the rate is the subject of debate [36–38]. Nevertheless, 40–50 items per second is about the highest rate that anyone would propose [39]. A rate of 20 items per second is more plausible for a T among L search. In a 200–250 msec fixation, that would suggest that at most 8–12 of the 17 items in the FVF shown in Fig. 2a could be processed to the point of identification. For present purposes, it does not matter how one envisions the details of processing within the FVF. It could involve serial selection of one item after another [40] or parallel but imperfect processing of all items [41] or some combination of serial and parallel processes [38, 42]. The critical point is it is not possible to fully process and identify everything within the FVF on a single fixation.

Figure 2: The role of the functional visual field (FVF) in look but fail to see errors.

(A) If observers fixated on the star, they would be able to identify the “T” to the left but, in a quarter of a second fixation, they could only sample a subset of the items in the FVF (green circle) and might fail to select that T even if it is visible. (B) A section of a lung CT image with an inserted gorilla. A radiologist, looking for small, white, round ‘lung nodules’ might be guided to select small white items while fixated on the star and, thus, be actively guided away from selecting and identifying the gorilla (“misguidance”). The missed T and missed gorilla LBFTS errors could both arise from the inability to fully process the contents of the FVF.

After ~200–250 msec, the eyes will move to a new location (e.g. the pink plus in Fig 1a) and the selection process will be repeated. The task, as a whole, has a structure as cartooned in Figure 1c. The search engine can be seen as an asynchronous diffuser [40, 42] with information accumulating about several selected items at any one time. In addition, a task-level quitting mechanism [43] [14] [40] terminates searches that are not ended by the discovery of a target.

Inadequate processing of selected items

Returning to Figure 1a, during that first fixation, attention might select the five items circled in red. When an item is selected, the system must decide what it is and if it is a target. Such decisions are frequently modeled as diffusion processes where information accumulates in a noisy manner toward some decision boundary [44]. In talking about search, it is not often appreciated that recognition in all but the simplest tasks must involve multiple diffusers operating in parallel. For instance, look at the object in the lower left of Figure 3. Presumably, you identify “bread rolls” when enough information accumulates to support that conclusion. At the same time, some weak evidence may accumulate for the conclusion that these are rocks, while little or no evidence supports the conclusion that this is, for instance, a rabbit. Moving to the right in the figure, it seems intuitively clear that you would be faster to accumulate the information needed to determine that the red thing is a car than to determine that it is a luxury item. Now, returning to the task in Figure 1, Figure 1b shows one diffuser collecting evidence that the item is a “white 3” target, another, that it is a target from the second half of the alphabet, and a third, making a decision about unlikely but incidentally interesting findings. Like the earlier examples of object identification (e.g., “car” vs “luxury goods”), these will have different thresholds (and/or rates of information accumulation). It would be easier/faster to accumulate evidence that the selection is a specific item like a ‘3’. It will be harder/slower to confirm a letter in the second half of the alphabet, and still harder to reach the threshold for an unspecified but interesting ‘other’ item. Since only a small number of items can be selected, it is important to quickly reject useless selections. Figure 1B shows two ways this might happen. If the item is identified as a distractor (e.g. an “A”), it can be rejected and space cleared in a limited-capacity workspace for a new selection. Hawkins and Heathcote’s recent Timed Racing Diffusion Model (TRDM) [45] argues for an important, second way to reject an item. Sometimes, no diffuser reaches a boundary in a reasonable amount of time. In that case, it must be possible to stop processing that item and move on to another one. The TRDM solution is to propose a time-based accumulator that ends processing for that item if its time boundary is reached [see also 46]. Imagine that you are trying to identify a bird. If you are not succeeding, it needs to be possible to disengage your attention and move on to other tasks. We would argue that it has not generally been recognized that, if only a few items can be selected for identification at one time, there must be a way to clear unprofitable selections and move on to other objects. Thus, there are two quitting mechanisms: one to end processing a single item, and another to end the task as a whole.

Figure 3: Mixed hybrid search.

In the mixed hybrid search paradigm, observers search for a mix of specific targets (upper-left; e.g., this specific drink) and categorical targets (upper-right; e.g., any animal) within an array of distractor items (bottom-left). In these types of searches, observers frequently miss categorical targets (dark purple bar), especially if categorical targets are rare. The error rate is lower if all targets are categorical (light purple), and lower still for specific targets. The green bars show the error rates for specific targets when they appear together with categorical targets (‘specific mixed’) or on their own (‘specific’).

Missing the designated target

The need to move on to the next item can explain some of the disturbing LBFTS errors where observers fail to see items that they were actually looking for. Consider the mixed hybrid search paradigm illustrated in Figure 3 [47]. In a hybrid search, observers search for any of several targets. In the mixed hybrid task depicted in Figure 3, targets were divided into two sets. There were three specific items (e.g. this drink, this bell, and this shoe) and three categorical targets (e.g. any animal, any fruit, any game). The visual displays consisted of arrays of easily identifiable object photographs. As shown in the graph in Figure 3, the key finding is that observers missed many more of the categorical targets than the specific targets, especially on mixed blocks where they looked for a combination of specific and categorical targets. Importantly, in terms of the Figure 1 model, there is no reason to assume that categorical items would be fixated and/or selected less often than specific targets. Random selection would not favor the specific targets [48]. Something else must be contributing to the failure to identify a clearly visible lemon as a categorical fruit target. A hint is found in the response times (RTs). It takes longer to successfully respond to categorical targets than to specific targets. It takes longer to decide that any item is an example of the category fruit than to recognize that it is the correct, specific drink [49]. In Figure 1b terms, the categorical diffusers will have higher bounds and/or slower information accumulation.

The threshold for the item-by-item timer that Figure 1 borrowed from the Timed Racing Diffusion Model (TRDM) [45] must be set for each task, presumably by some adaptive process that quits sooner if it can and quits later if too many mistakes are being made [13]. If the threshold is set too low, observers will flush items before identifying some more slowly identified, categorical items, even though they had been selected. This is a form of a speed-accuracy tradeoff [50, 51]. If all the targets are categorical (light purple bar in the Figure 3 data graph), the item quitting threshold will be set at a level producing a moderate number of errors. If specific items are mixed in, their faster identification will shift the quitting threshold to a shorter value, causing proportionally more categorical items to be missed. If specific targets are made markedly more common than categorical, observers will learn to move on even more quickly. Under such conditions, the data show that observers missed slower categorical targets at seven times the rate of specific targets, even though there is no reason to think that they were selected less often [47]. Thus, under the right circumstances, the same timing mechanism that normally speeds information processing can produce large numbers of LBFTS errors for targets that observers are actively searching for. This path to LBFTS errors may be an important contributor to failures to report ‘incidental findings’ in medical images or to react to visible pedestrians in the roadway.

The role of target prevalence in LBFTS errors

In a mixed hybrid search, LBFTS errors increased as the probability (or “prevalence”) of the categorical targets declined. Target prevalence has a substantial effect on visual search tasks [52]. False-negative (miss) errors increase at low prevalence [53] making prevalence a plausible source of LBFTS errors in socially important, low prevalence tasks like airport [54] and cancer screening [55]. These errors, too, can fall out of the model in Figure 1. In search experiments, low prevalence makes decision criteria more conservative (fewer target-present responses) [56]. In a diffusion model like Figure 1, this is accomplished by moving the starting point of the accumulator closer to the distractor decision boundary and further from the target boundary for rare targets. Absent responses also become faster. This can be modeled as a change in the task level quitting mechanism in Figure 1C [56]. Thus, at the airport or the radiology clinic, an expert will require more evidence for the important low-prevalence targets and be more willing to give up on a search that has not yielded a target; again, a recipe for LBFTS errors (Box 2).

Box 2: Prevalence effects: LBFTS errors for rare targets.

Classic studies of vigilance, dating back to the 1940s, show that when observers monitor for rare, but important signals (e.g., a target on a radar screen), these events are frequently missed [100]. Although these targets are visible and easily detected under normal circumstances, observers miss them after a prolonged period of sustained attention. Much of this work has attributed these misses to a loss in sensitivity (i.e., a reduced ability to distinguish signal events from noise) [101]. However, more recent findings challenge this account, attributing the vigilance decrement to a shift in decision boundaries, or criteria [102]. In other words, as observers monitor for rare events over a long period, they may be less willing to say that a target is present.

This result parallels those seen in visual search tasks. When observers look for rare items, like an abnormality in a CT scan, or threats in luggage [103], these targets are often missed. As in the vigilance literature, this increase in misses can be attributed to a shift in decision boundaries or criteria [104]. In other words, our perceptual ability to detect the target is the same, but as they become rare, we become less willing to say that something is there. Many efforts have been made to ‘cure’ this prevalence effect, with mixed success [105–108]. To what extent are classic vigilance failures and low-prevalence search errors the same type of LBFTS error? Both seem to involve changes in decision criteria, though typical vigilance errors involve transient stimuli while LBFTS errors in search usually involve stimuli that are present until the observer dismisses them. Moreover, while vigilance tasks generally involve monitoring for a single event, prevalence effects in visual search involve a series of decisions (i.e., is this item a T or an L? What about the next one?), along with an overall decision about when to stop searching (i.e., a task-level quitting threshold). Models have been proposed for prevalence effects in search accounting for both types of decisions [14, 109, 110] and vigilance [11]. An integrated model would be useful.

As a more mundane example of these mechanisms at work, think about the process of proofreading. Typos are rare as a fraction of all words. Most words serve as distractors in typo search, and the reader wants to quickly finish proofreading. Under these circumstances, the low prevalence pressures will come into play and clearly identifiable misspellings will be missed.

Perceiving the most likely world

What do observers consciously experience as this process unfolds? Even if we can only select and attend to a subset of the items in the visual field, non-selective processes will deliver the impression of rapidly seeing something throughout the visual field [57, 58]. While people are demonstrably not aware of the accurate details of the scene in the absence of selection and attention [59], the non-selective processes deliver a statistical summary of the current contents of a scene. This summary is often enough to create a reasonable and useful hypothesis about what is present [60–63], even if it is not enough to tell us if a 3 (or a breast nodule) is present or where it is. In addition, observers do not have introspective access to what occurs at the time scale of attentional deployments, lumping perhaps several hundred msec into a rough sense of “now” [64] [65]. People are also surprisingly bad at knowing where they have looked [66]. Indeed, patients with visual deficits can be surprisingly bad at knowing where in the visual field they are impaired. For example, patients with significant scotomata may assert that the visual field appears not to have any holes in it [67] and, of course, people do not see their normal physiological blindspots [68].

But what do we ‘see’? We don’t see a mass of image statistics. It seems that observers experience their best guess about the state of the world based on what Helmholtz called “unconscious inference” [69]. A more modern, Bayesian “predictive coding” account would say that “the hypothesis with the highest posterior probability (i.e., most probable given the input) wins and gets to determine the perceptual content of the system.” [70] [for a review see 71]. For present purposes, the important point is that after fixating on a location for a quarter of a second, observers are likely to be quite convinced that they saw and understood whatever was there to be seen, even if their actions and the data can clearly show that this was not the case. Moreover, an important aspect of maintaining a plausible theory about the state of the world is what Friston [72] describes as “suppressing surprise”. In terms of the model in Figure 1, this would correspond to raising decision bounds on unlikely accumulators so that the system does not jump to unlikely conclusions about a selected item. For example, observers should not conclude that an “A” is the Matterhorn peak, even though some signal to that effect might accumulate. Of course, the system should be tuned so that it can change its current hypothesis about the world in the face of enough surprising data, but a high decision boundary for unlikely items decreases the probability that a radiologist will report an incidental finding or that a driver will notice a pedestrian in an unlikely location.

We have described the role of three of the four factors, listed at the outset: (1) During a fixation, a subset of all possible objects of attention will be selected for limited-capacity processing. (2) If a selected item doesn’t yield its identity reasonably quickly, it will be flushed to make space for a new, perhaps more promising, source of information [73]. (3) The observer will experience the currently most probable state of the world, avoiding most false alarms. That observer will be surprised when they miss a clearly visible item because they will not realize that they never selected it or that they selected it but moved on before fully analyzing it. One more factor helps us to understand the phenomena like the missed gorilla in the opening example.

Missing an unexpected target: Guidance and Misguidance

In the case of a T among L search as in Figure 2a, observers probably process a fairly random subset of items, with, perhaps, a bias toward selecting items closer to fixation [74]. In other tasks, including the task in Figure 1, the choice of what to process is unlikely to be random. In the Figure 1A task, attention would be guided to white items in the search for a white 3 or, in a version of the Figure 2A task, imagine observers are looking for red Ts among black and red Ls. They will not attend to many black Ls. Attention will be guided by basic feature information [75], in this case, to the red items. [76].

Normally, guidance is very useful. In some instances, like the classic gorilla study, guidance becomes misguidance. In that study, observers monitored the white-shirted team. In another version, they were less likely to miss the gorilla if they monitored the black-shirted team [see also 78]. If they were guided to black shirts, they were more likely to notice a black gorilla. Figure 2B illustrates this point using a piece of a lung CT image from an inattentional blindness experiment with radiologists as the participants [20]. The radiologists were searching for small, white, round lung nodules and missed a large, black, irregular but easily detectable gorilla inserted into the image. Eye-tracking shows that the radiologists almost always fixated on or near the gorilla. As shown in 2B, a radiologist hypothetically fixated on the red star might be guided, as a result of their expertise, toward 4–5 plausible lung nodules and thus be guided away from the gorilla. Even if the gorilla was selected, its processing might have been terminated before the gorilla was identified because, in keeping with the efforts of a predictive coding process to avoid false alarm surprises, the threshold for identifying an item as unlikely as a gorilla would be set high. After all, gorillas just don’t happen in the lung or, for that matter, in the midst of ball passing games [79]. Thus, misguidance and the need to end the processing of selections that do not produce rapid identifications can give rise to inattentional blindness. By the way, if you failed to note the gorilla in Figure 1A, you can search for it now. It should be visible. Misguidance is central to many effective magic tricks as well [80] [81].

Concluding Remarks

Thus, from gorillas in the lung, to a cyclist on the road, to the lowly typo, a constellation of LBFTS errors can be accounted for as the by-product of quite reasonable settings of the parameters of a mechanism like that shown in Figure 1. Some of the LBFTS components, reviewed here, have been suggested previously. Here, we unite them into a single framework. In previous discussions, there has been less emphasis on the idea that an item could be attended to and still missed. The mixed hybrid result (Figure 3) makes it clear that this must happen. The novel, item-by-item quitting mechanism in Hawkins and Heathcote’s TRDM [45] offers a potential mechanism by which items can be missed, even if attended.

In our view, LBFTS errors are not the result of doing something wrong. They are the product of situations where the normal functioning of the visual system produces surprising or undesirable outcomes. We refer to this as ‘normal blindness’ to emphasize the idea that LBFTS errors are a form of visual impairment. Unlike clinical blindness, everyone experiences normal blindness and people are mostly blind to its costs (normal blindness blindness? [82]). Normal blindness is probabilistic. An item that is missed by one observer at one time will be seen at a different time or by a different observer. This guarantees, for instance, that a continuity error, missed by a filmmaker, will be noted and reported by some of the many observers who later see the film [16]. What can be done? (see Outstanding questions). Given that errors can arise from statistical bad luck, a second pair of eyes can help (that is who found those typos in your paper, after all.) That second reader might be a computer [e.g. 83, 84], (Box 3). In settings like driving, we can try to build an environment that helps us use our limited resources [85, 86]. Since people do not want to continue paying the costs of normal blindness in lives lost in medicine or on the road (or even in typos missed), having a model of normal blindness gives us some of the tools we need to understand it and work to reduce it. If we all suffer from normal blindness, we should work to minimize LBFTS errors because the result will benefit all of us.

Outstanding Questions:

• Clearly, the central question is “How can we reduce Normal Blindness?” As we argue here, it may be based in mechanisms of visual attention and not easily resolved through training or awareness (e.g., by informing observers of these errors). Can LBFTS errors be reduced through manipulations that implicitly or explicitly shift the observer’s decision boundary, perhaps through changing the perceived costs and payoffs?

• To what extent can technology help? (Artificial intelligence in medicine, self-driving cars, etc.)

• Do motivated observers make fewer errors? Surely, the answer is “yes” but how far can motivation go in eliminating errors? More broadly, how do LBFTS errors vary with the mental state of the observer (e.g., are these errors more likely under fatigue?)

• Similarly, are some individuals more susceptible to these errors? For example, how do LBFTS errors compare between younger and older adults? Are there particularly dangerous interactions of normal blindness with clinical visual or cognitive impairments?

• Observers often experience LBFTS errors as ‘blindness’ – I did not see it. How much information about the unseen target is, in fact, processed outside of awareness, and to what extent can any of the sub-consciously processed information affect observers’ actions? Would you walk around a gorilla that you could not report seeing?

• The model discussed here focuses on visual / attentional factors. How much of the LBFTS problem is a memory problem? The observer fails to recall the whole list of possible targets or, alternatively, they remember all the targets but do not successfully load them into working memory or activated long term memory.

Box 3: Can the computer save us from LBFTS errors?

“Curing” normal blindness by focusing on the observer may be hard [104]. Training radiologists or pilots to use a certain gaze pattern to scan a mammogram [20] or the view out the windscreen [111] may help, but is only part of the solution [112]. Perhaps computer algorithms that do not forget to do what they are told could be the answer to LBFTS errors because they can tell us what we might miss. Your car can tell you when a child runs into the road [113], or if you’re a radiologist, it can tell you that this region of a mammogram looks suspicious [114] Large investments have led to significant advances in computer-aided detection and more progress will be coming. That said, the ubiquitous spellchecker alerts us to possible problems:

1) While it can be amusing when the spellchecker offers up the wrong word, if the “positive predictive value” (proportion of correct suggestions out of all suggestions) is not high, users will tend to ignore an AI system [115]. Similarly, a driver may be less willing to trust an autonomous driving system if it is inconsistent or unreliable (see [116] for a review).

2) At the other extreme, if the system is very reliable, the user may lose the skill required to recover in those cases where the AI fails. For example, how are your map reading skills now that you are regularly taking direction from your phone? [117]. This kind of overreliance or ‘de-skilling’ has long been a concern for pilots and is often a question in aviation accidents [118].

Highlights.

• Looked But Failed to See (LBFTS) errors occur when observers fail to notice a clearly visible item, and occur across a wide range of tasks and settings, from driving and medical image perception to laboratory visual search tasks.

• We outline a new, unified account of LBFTS errors, arguing that processes that serve us well under most circumstances are guaranteed to produce a steady stream of LBFTS errors under some circumstances.

• LBFTS can be thought of as a form of “Normal Blindness”. It is obviously far less severe than clinical blindness but it is so universal that its costs are substantial at a societal level.

Glossary:

Asynchronous diffuser

Information can accumulate in multiple ‘diffusers’ (see below) at the same time. If these processes start and stop at different times, the set of diffusers can be called an asynchronous diffuser.

Attentional guidance

The use of information from across the visual field to determine where the observer should attend next. For example, attention can be restricted to a subset of visual input based on basic features, location, or higher-level information (e.g., knowledge of scene structure). Also referred to as ‘guidance’.

Diffuser

A process that accumulates information and outputs a response when the accumulating information reaches a threshold level.

Fixation

The location in the visual field that forms its image on the fovea (i.e., where the eyes are ‘pointed’).

Functional Visual Field (FVF)

Also called the Useful Field of View, the area around the point of fixation within which items can be (but might not be) recognized.

Hybrid Search

A type of visual search task in which the observer needs to look for multiple different types of items (e.g., a radiologist looking for a cancerous nodule or pneumonia) within a single image.

Non-selective processes

Visual processes that operate across the entire visual field allowing us to see something everywhere, even if selection and attention are required to actually identify what we are seeing.

Inattentional blindness

A failure to notice a clearly visible, but typically unexpected item in the absence of any visual impairment. Inattentional blindness can be understood as a Looked But Failed to See error.

Visual search

The process of looking for an item or multiple items (targets) among other objects (distractors).