Downgraded phenomenology: how conscious overflow lost its richness

Abstract

Our in-the-moment experience of the world can feel vivid and rich, even when we cannot describe our experience due to limitations of attention, memory or other cognitive processes. But the nature of visual awareness is quite sparse, as suggested by the phenomena of failures of awareness, such as change blindness and inattentional blindness. I will argue that once failures of memory or failures of comparison are ruled out as explanations for these phenomena, they present strong evidence against rich awareness. To accommodate and explain these massive failures of awareness, any theory of phenomenal consciousness must downgrade phenomenology to a degree where it is functionless or, ironically, does not reflect what we experience.

This article is part of the theme issue ‘Perceptual consciousness and cognitive access’.

1. Introduction

We seem to experience a rich visual world. As we go through our day, we encounter all types of colours, objects and events. This sense of experiencing things right in front of our eyes has inspired—and continues to inspire—many aspects of perception research. Vision scientists have a special affinity for phenomenologically convincing demonstrations of visual phenomena. When a new phenomenon ‘works as a demo’, it effectively and intuitively reveals an aspect of how the mind works [1]. This functionality of phenomenology carries a lot of weight when building mechanistic and theoretical accounts of perceptual processing. In short, vision scientists take phenomenology seriously.

Even though we spend much of our life in a series of in-the-moment experiences (when we are not remembering, planning or sleeping), it is surprisingly difficult to assess the contents of what Block [2] has called phenomenal consciousness: ‘what it's like to be in [a] state’. (p. 227). Do we experience a rich world that we simply cannot describe due to limitations of attention, memory or other cognitive processes, as proposed by Block [2,3] and others (e.g. [4–6])? Or is the nature of awareness quite sparse, as suggested by demonstrations of failures of awareness, such as change blindness [7,8] and inattentional blindness [9,10]? These questions are typically explored in the domain of visual awareness, but they are relevant questions for other domains of conscious experience as well. In the domain of olfaction, for example, there does not seem to be a distinction between what we experience and what we can access [11]. But in visual perception, whether awareness is rich and ‘overflows’ our ability to report about it or whether it is constrained by cognitive limitations has been debated at length [12–15].

I will argue that to accommodate and explain inattentional blindness and change blindness, theories of rich awareness downgrade phenomenology to a degree where it is functionless or, ironically, does not reflect what we experience. I will specifically argue that:

• (1) distinguishing inattentional blindness and change blindness is important because the two phenomena provide different evidence against rich awareness;

• (2) failures to encode specific details into memory occur in many different paradigms, and such memory failures are a reasonable alternative explanation for inattentional blindness;

• (3) one can rule out memory failures by instructing people to immediately report what they see when they look at a visual display;

• (4) immediate report instruction in repeated inattentional blindness experiments demonstrates that inattentional blindness is a perceptual deficit;

• (5) to accommodate and explain repeated inattentional blindness and massive change blindness, theories of rich awareness downgrade phenomenology.

2. Distinguishing change blindness and inattentional blindness

The claim that in-the-moment experiences have rich phenomenology is challenged by two types of evidence: change blindness and inattentional blindness. Although often discussed together, these two types of ‘blindness’ need to be distinguished.

Change blindness is the failure to notice changes to a visual scene, even if those changes happen right before one's eyes (e.g. [8]; see figure 1 for an example from [16]). Change blindness has been demonstrated in dozens of different ways. Many examples of change blindness include a visual interruption: from the simplest demonstrations, in which an image will flash on and off with some detail changing between the two images (e.g. [17]), to more complex demonstrations, such as short movies that use careful camera work or editing to obscure a mid-scene change [18], or real-life demonstrations where an experimenter swaps places with another experimenter when a large object (e.g. a plywood board or door) temporarily blocks the subject's view [19]. The failure of participants to notice these changes is all the more surprising because built into many of the demonstrations (especially those with images that flash on and off) is the task instruction to pay attention to and detect changes to the scene (e.g. [17]). Therefore, in many cases of change blindness, the inability to report the change is not limited by a general lack of attention.

Figure 1.

Overview of experimental paradigms. The leftmost column represents the visual displays that are presented to participants. The middle column represents what a participant would see and respond if they had rich visual awareness of the display. The rightmost column represents what a participant would see and respond if they had sparse awareness of the display.

Do these failures to notice changes mean that awareness is not in fact as rich and detailed as people seem to experience? Not necessarily. First, many of the changes people fail to report are small and irrelevant to the meaning of the scene. Failing to notice some tree branches disappearing or the colour of a person's shirt changing generally has no consequences for further cognitive processing. Thus, missing small and irrelevant details is not convincing evidence against rich awareness. Second, a failure to report a change could be caused not by a failure to represent the scene richly enough, but a failure to compare the representations before and after the change¹ [21]. Third, a failure to report a change could be caused by a failure to encode the perceptual states into memory. Even if viewers could make the before/after comparison, they would not be able to report the change if the perceptual representations did not make it into a durable form of memory in the first place. For these reasons, failing to report a change in a change blindness demonstration does not necessarily indicate a lack of rich phenomenal experience.

Inattentional blindness paradigms overcome some of the limitations of change blindness paradigms. Inattentional blindness occurs when people fail to notice an otherwise salient event when their attention is occupied [9,22,23] (see figure 1 for an example from [24]). One of the most famous demonstrations of sustained inattentional blindness [10], in which a man in a gorilla suit goes unnoticed by observers performing an attentionally demanding task, is one of the most widely recognized demonstrations in psychology, presumably because it violates people's intuitions of what they should be able to notice given the apparent richness of phenomenal consciousness.

Compared to change blindness paradigms, the unexpected events in sustained inattentional blindness paradigms are usually very salient, such as a novel item appearing on screen that is a new colour or shape [23]. The events are readily visible when a participants' attention is directed toward these events, but—allegedly—become invisible when attention is directed toward another task, such as counting how many times a distractor shape bounced off the edge of the display. Because the unexpected events stay in view for several seconds, there is not an obvious need to compare the event to a pre-event representation (such as in the case of change blindness paradigms), so it seems unlikely that inattentional blindness would be due to a failure of comparison. However, because participants are only asked about their experience of the unexpected event after the fact, it has been a long-standing possibility that inattentional blindness could be due to a limitation of memory rather than a failure of visual awareness. Until recently, this seemed like an insurmountable problem with using inattentional blindness to challenge rich phenomenology.

3. The problem of failures of memory

If inattentional blindness can be explained as a failure of memory, then demonstrations of it, like demonstrations of change blindness, do not pose a challenge to the view that we have rich visual awareness. Therefore, it is important to be clear about what it means for something to be a failure of memory and about what approaches can be taken to rule out this possibility. A failure of memory in this case is not the same as forgetting where you parked your car or put your keys; it is subtler than that. Memory failures of this type are best illustrated by two well-known paradigms in cognitive science: the partial report paradigm used in studies of iconic memory [25] and the rapid serial visual presentation (RSVP) paradigm [26,27].

Partial report was originally used to demonstrate iconic memory [25]. In these studies, participants viewed grids of 9 or 12 letters that were presented for 50 ms (see figure 1 for a schematic). First, participants were instructed to report all the letters that had been presented. In this whole report condition, participants were only able to report about four letters. Next, participants were instructed to report only the letters that had been presented in one of the rows. In this partial report condition, they were able to report about 75% of the letters from a row of three or four letters. This indicated that the letters available to them from the whole grid was about nine (75% × 12 letters), since they could report 75% of any row. Critically, participants maintained this level of performance even when the row was cued after the entire display had disappeared. So although participants did not know beforehand which row they would be asked to report, they nonetheless were able to report any subset of the letters that were post-cued. This result was taken to show the existence of a high-capacity, but fragile iconic memory store in which all the letters of the display are encoded, but that fades rapidly and cannot be fully accessed or reported.

These findings provided strong empirical inspiration for distinguishing between phenomenal versus access consciousness. Although participants were unable to report much of what they saw when their reports were unconstrained (i.e. whole report), the data appear to show that they had a richer, more detailed representation of the display—if only briefly. But this rich, detailed representation was not encoded durably into memory. These experiments [25] also provided subjective inspiration for distinguishing between phenomenal versus access consciousness: in the original paper by Sperling [25], it is reported that participants felt that ‘they have seen more than they can remember [or] report afterwards’. (p. 1). This statement is important because it initially established the phenomenology of iconic memory (one could imagine a case where the same results were obtained but where participants were not so sure about what they saw). However, there is perhaps too much emphasis on this one statement, because beyond it, Sperling [25] did not directly assess participants' phenomenology. In addition, it is not clear whether the statement reflects participants' initial impression of the display, or their impression after their substantial experience with the display (five participants took part in seven experiments spread across 12 sessions that were scheduled three times weekly). More recent research using a modified iconic memory paradigm has shown that feeling that you saw more than you can report does not guarantee that the in-the-moment phenomenology was of high fidelity: for example, people mistakenly perceive letter-like symbols as real letters when presented alongside normal letters [28].

Nonetheless, Sperling [25] demonstrates that an inability to report one's experience due to fragile memory encoding does not mean the experience was sparse. This can be demonstrated in another way by viewing an RSVP stream. In an RSVP paradigm, visual items are presented rapidly (usually approx. 100 ms) to the observer, one right after another (see figure 1 for a schematic). As a result, if you were to view an RSVP stream of letters, it is unlikely you would be able to report all the letters you saw in order, and perhaps you would not even be able to report any specific letter you saw in the stream. But, in the moment, your impression of the letters would be that they seemed clearly visible—though fleeting—and that you were unable to report the letters only because you were asked after the stream had been presented.

In both the iconic memory and RSVP paradigms, participants' experience is queried after the display has disappeared and they cannot accurately report what they saw. This pattern of results is also what is obtained with the inattentional blindness paradigm: in most demonstrations of inattentional blindness, participants are asked about their experience after the unexpected event had come and gone. This leaves open the possibility that participants saw the unexpected event, but failed to encode it durably into memory, in much the same way as in iconic memory and RSVP paradigms.

4. Ruling out failures of memory

Determining whether participants saw and forgot a visual display or failed to see the display in the first place can be difficult. In the case of iconic memory and RSVP, special report instructions are given to the participants to allow them to access (at least part of) their experience. The most straightforward way to distinguish between a failure to encode into memory and a failure of perception is to have participants immediately report what they see, when they see it.

For example, if a participant in an RSVP experiment is told to ‘press the space bar as soon as you see the letter M’, they will press the space bar if they see the M and fail to press the space bar if they do not see the M. This immediate report instruction thus provides an accurate report about in-the-moment experience. Participants can accurately detect targets even at exceedingly brief presentations (possibly as fast as 13 ms, e.g. [29] but certainly as fast as 53 ms, e.g. [30]). The task does not rely on memory because a response is given based on what the participant does or does not perceive when the target is present. Therefore, immediate report instructions can identify failures of perception separate from failures of memory.

However, using an immediate report task in an inattentional blindness experiment presents a problem: if participants are instructed to immediately report when they see something unexpected, they then have an expectation for the unexpected event! So while they can give an accurate report of what they saw or failed to see, their attention to the unexpected event will attenuate or eliminate inattentional blindness.

Because of this dilemma, determining whether inattentional blindness was truly a perceptual deficit or simply just a failure to encode into memory had been thought to be unsolvable in principle, e.g. that ‘there are serious problems with any experimental effort to directly ask subjects if something is consciously perceived without attention’, and that this ‘proves to be impossible because the demand to report on [an unexpected event] directs attention to [it]’ [31, p. 73]. Although there has been scepticism of the inattentional amnesia account of inattentional blindness [32], it nonetheless remained a possibility, and thus did not constitute convincing evidence against rich visual awareness for the reasons described previously.

5. Repeated inattentional blindness

Using a new technique, my colleagues and I have found a way to escape this dilemma and have shown that inattentional blindness truly is a deficit of perception [24]. The usual account of inattentional blindness is that it is due to a lack of any expectations about the unexpected events. But what if instead of having no expectations, participants formed a specific expectation about what type of unexpected event was to occur? If this were the case, participants could be given the instruction to immediately report seeing anything unexpected, but if the unexpected event did not match their specific expectation, they would still experience inattentional blindness paradigm (if it were in fact a perceptual deficit).

Using a sustained inattentional blindness (e.g. [23,33]), we showed participants a display containing the letters L and T which could be either black or white and which moved randomly across the screen. Participants counted how many times the white Ls crossed the midline of the display. There were four trials of this sort, but on the fifth trial, an unexpected object—a dark red cross—slowly traversed the midline (bottom row, figure 1). Immediately after this trial, participants were asked if they noticed anything about the last trial, and then asked whether they noticed that a dark red cross had appeared on screen. We found that a substantial portion of the participants did not report seeing the unexpected event, demonstrating the basic inattentional blindness effect.

Instead of ending the experiment there, we then gave participants one more instruction: to keep an eye out for anything else unexpected and to press the spacebar as soon as they see something unexpected. As described previously, this immediate report instruction would permit the participants to accurately report their in-the-moment experience without relying on their memory for the experience at all. The participants then completed several more trials: several trials in which nothing unexpected happened, but also several trials in which the same unexpected event (red cross) appeared. By repeating the unexpected event in this manner, we built up participants' expectation about what type of event could appear. On the final critical trial, a novel unexpected event (blue letter E moving in the opposite direction) appeared for half of the participants, while yet another occurrence of the same unexpected event as before (i.e. red cross) appeared for the other half of participants.

We found that more participants missed the novel unexpected event compared to the repeated unexpected event. This demonstrated repeated inattentional blindness in the same participants in the same session. But critically, even when participants were willing and able to provide immediate report of the earlier unexpected events, they still missed the novel unexpected event. Their failure to give immediate report in this experiment thus indicates that they truly did not consciously perceive the event, rather than failing to encode it into memory. With these results, we concluded that inattentional blindness genuinely reflects a deficit in perception rather than memory, presenting a strong challenge to the thesis of rich visual awareness.

6. Downgrading phenomenology

To maintain that participants in inattentional blindness experiments have any in-the-moment experience of the unexpected event, proponents of rich awareness must concede that this representation cannot be used. For example, participants cannot use it to provide immediate report of any of the specifics of the unexpected event (such as colour or shape); cannot use it to pick out the encountered item from a line-up (indicating that the unexpected event does not even serve as a perceptual prime); and they cannot provide immediate report about anything at all, even as the experience occurs in front of their eyes for several seconds.

Given these results, what then is their conscious experience of the unexpected event? Even if participants were able to indicate that ‘something was different’ about trials in which the novel unexpected event occurred (there is no evidence that they feel this way), there is nothing functional and nothing rich about that phenomenology. To accommodate our results showing inattentional blindness is a perceptual deficit, the ‘richness’ of phenomenology must be severely downgraded. If phenomenal consciousness is ‘what it's like to be in [a] state’ [2] (p. 227), it does not seem like it is like anything to participants when they encounter an unexpected event when their attention is otherwise engaged.

Another example of how phenomenology must be downgraded to accommodate new empirical evidence can be demonstrated by showing that people miss a large magnitude of details or changes to scenes. As discussed previously, there are several limitations to change blindness that make it problematic as evidence against rich awareness. However, it should not be dismissed entirely, especially in cases where the number or magnitude of changes is substantial. In another recent study, my colleagues and I combined change blindness and iconic memory to test whether individuals required conscious perception of all parts of a complex visual display in order to report a summary statistic about the display [16]. Our study was based on one by Bronfman and colleagues [4], which probed the content of in-the-moment experience by testing whether people could report the colour diversity of an array of coloured letters, even if their experience of the individual letters fades too quickly to access. Other examples of summary statistics in perception, such as statistics like size [34,35] and location [36], can be reported without any awareness of the individual elements that make up the statistic [37,38]. Likewise, we hypothesized that people would be able to report the colour diversity of the display, but their ability to do so would not require awareness of any of the individual letters' colours.

Using a modified iconic memory paradigm, participants in this experiment were presented with a brief array of coloured letters (see figure 1 for a schematic). The colours could either be drawn from a narrow part of a colour wheel (low diversity: e.g. purples and pinks) or from the whole colour wheel (high diversity). Participants were cued to a specific row of letters and had to report the identity of one of the letters after the display had disappeared. Thus, participants attended to a specific row, but did not know beforehand which letter they would have to report (similar to [25]). Participants also had a secondary task of reporting the colour diversity of either the attended row or nonattended rows. Replicating Bronfman et al. [4], we found that participants could report the identity of the target letter, and could report the colour diversity of both the attended row and the nonattended rows.

Are people able to report colour diversity because they are consciously encoding each of the letters' colours, even when the letters were unattended? Or are people able to do this because they perceive just the diversity summary statistic and not the individual elements?

Other studies have demonstrated that participants mistakenly perceive letter-like symbols as real letters when presented alongside normal letters [28], suggesting that people are not consciously encoding each element with high detail. In our study, we theorized that if participants were aware of every element in the display, they should notice when at least one of the elements changes during the course of the experiment. To test this, we incorporated a change blindness component to the task [4]: during half of the trials, all of the letters' colours in the unattended rows were shuffled mid-trial (18 colour changes). Across two experiments, none of the 12 participants in each noticed any colour change during 192 change trials. This totalled 3456 missed changes. Had any participant noticed any one of these changes, our experiment could have been used to support rich awareness. But with these results, we concluded that people can be aware of and report ensemble properties, like colour diversity, without being aware of individual elements.

To be clear, this experiment cannot rule out rich phenomenology in an absolute sense. It could be the case that participants saw all the colours in rich detail, but failed to encode the items into memory to compare them to their initial colour before the change. But the failure to report any of the 3456 changes highlights how functionless and impoverished in-the-moment phenomenology must be. In this experiment, proponents of rich awareness must concede that if people do experience the colour change, the representation of this experience is not being used in any way: the changes do not influence colour diversity or letter recall performance; participants cannot report any specific colour change; and they cannot report that any change happened at all, despite participating in the experiment for nearly an hour.

7. Conclusion

Overall, to accommodate and explain repeated inattentional blindness and massive change blindness, any theory of phenomenal consciousness must downgrade phenomenology to a degree where it is functionless or, ironically, does not reflect what we experience.

If it is necessary to explain these failures by appealing to failures of comparison or memory, then proponents of rich awareness may find themselves in an uncomfortable position when these failures occur in real life. Based on the results discussed above, a driver may hit a child who has run into the street because the driver's attention was otherwise occupied and he failed to perceive the child. The alternative is that the driver saw the child in rich detail, but failed to compare his representation of the child to a previous version, or failed to encode the child properly into memory, and hit her with his car anyway. If this alternative is true, then phenomenology does not seem like one worth advocating for.

Phenomenology should be functionally useful, even if what we are consciously aware of is sparse. Fortunately, there are new aspects of perception that we are learning more about as a consequence of this debate. In particular, we are learning that our perceptual system is capable of very sophisticated statistical perception, especially in the absence of awareness [16,39]. Statistical perception may help reconcile cognitive and physiological limitations with our subjective impression of a rich detailed world [39]. For example, we are only seeing rich detail and colour in central vision to begin with, and our perception of scenes arises through stitched-together fixations [40]. By better understanding the representations that result from statistical perception, we may better understand why we have a holistic experience of our visual environment. There is also much we do not understand about how cues affect visual awareness, especially how cueing a stimulus after it has disappeared may bring it into awareness for the first time [41]. Exploring visual awareness through these avenues may help us understand why we feel like we experience a rich visual world when we in fact do not.

Data accessibility

This article has no additional data.

Competing interests

I declare I have no competing interests.

Funding

I received no funding for this study.