Expectations Change the Signatures and Timing of Electrophysiological Correlates of Perceptual Awareness

Abstract

Previous experience allows the brain to predict what comes next. How these expectations affect conscious experience is poorly understood. In particular, it is unknown whether and when expectations interact with sensory evidence in granting access to conscious perception, and how this is reflected electrophysiologically. Here, we parametrically manipulate sensory evidence and expectations while measuring event-related potentials in human subjects to assess the time course of evoked responses that correlate with subjective visibility, the properties of the stimuli, and/or perceptual expectations. We found that expectations lower the threshold of conscious perception and reduce the latency of neuronal signatures differentiating seen and unseen stimuli. Without expectations, this differentiation occurs ∼300 ms and with expectations ∼200 ms after stimulus in occipitoparietal sensors. The amplitude of this differentiating response component (P2) decreases as visibility increases, regardless of whether this increase is attributable to enhanced sensory evidence and/or the gradual buildup of perceptual expectations. Importantly, at matched performance levels, responses to seen and unseen stimuli differed regardless of the physical stimulus properties. These findings indicate that the latency of the neuronal correlates of access to consciousness depend on whether access is driven by stimulus saliency or by a combination of expectations and sensory evidence.

Introduction

Conscious perception is not solely determined by stimulus saliency. Strong stimuli can remain unnoticed if attention is deployed elsewhere as shown in the attentional blink or change blindness paradigms, and weak sensory stimuli can be readily perceived if they are attended to (Carrasco et al., 2004). Consequently, perceptual awareness (PA) is proposed to depend on two factors (Dehaene et al., 2006): the intensity of sensory stimulation and top-down attention, which enhances sensory processing. However, evidence suggests that attention may not be the only top-down factor that determines perception. Everyday experience indicates that recognition is greatly facilitated if one knows what to expect. In laboratory settings, when subjects are confronted with fragmented black-and-white images of an object, they may fail in perceiving the object. However, once the object has been identified, it pops out and will henceforth be recognized immediately (the Eureka effect) (Dolan et al., 1997; Ahissar and Hochstein, 2004). Similarly, providing subjects with the name or category of the object in anticipation of the stimulus lowers the threshold for its visibility (Eger et al., 2007; Esterman and Yantis, 2010). In patients with visual extinction caused by parietal lesions, information held in working memory can reduce the effects of extinction on awareness (Soto and Humphreys, 2006). These observations suggest that perceptual thresholds depend not only on attention but also on a priori knowledge. Such top-down effects agree with theories that consider perception as a Bayesian process in which sensory information is matched with priors (i.e., with prior knowledge and expectations) (Gregory, 1997; Kveraga et al., 2007; Friston, 2010). An increasingly popular implementation of this Bayesian integration is predictive coding, in which the brain constructively predicts upcoming sensory input rather than passively registering it (Rao and Ballard, 1999; Friston, 2005): Top-down predictions “explain away” lower level representations through recurrent interactions such that the mismatch between expected and observed evidence (prediction error) is minimized. Accordingly, weaker neural responses to anticipated than to unexpected stimuli have been reported (Garrido et al., 2009; Alink et al., 2010).

Here, we test whether the threshold of perceptual awareness is modulated by previous experience. In particular, we examined how sensory evidence and top-down expectations, respectively, influence the threshold of awareness and whether the two factors modulate brain activity differently. To this end, we measured electroencephalographic (EEG) activity in a visual paradigm in which we generated perceptual hysteresis by parametrically increasing and then decreasing the sensory evidence for an initially hidden stimulus. Under these conditions, an expectation is built up once the subject has perceived the stimulus, and this in turn increases the visibility of subsequent lower contrast stimuli. This allows for contrasting brain states with and without expectations and with perceived and nonperceived stimuli for identical stimulation conditions. The results confirm that expectations lower the threshold for perceptual awareness and provide new evidence on the electrophysiological signatures associated with conscious perception. Importantly, the amount of prior knowledge and ensuing expectations determine whether the electrophysiological signatures of awareness occur early or late after stimulus presentation.

Materials and Methods

Participants

Sixteen volunteers participated in the main EEG study (mean age, 26 ± 2 years; 10 females; 14 right-handed). Of those, nine participants also took part in a control EEG experiment (five females; eight right-handed). Ten additional participants took part in a behavioral control experiment (mean age, 26 ± 5 years; five females; nine right-handed). All participants had normal or corrected-to-normal vision and had no history of neurological or psychiatric disorders. Participants gave written informed consent before the experiment. The study protocol was conducted in accordance with the Declaration of Helsinki and approved by the local ethics committee.

Material

Stimuli were presented on a thin-film transistor (TFT) monitor (resolution, 800 × 600) at a refresh rate of 60 Hz and located ∼1 m from the subjects. The visible screen size subtended 16.197 × 12.615° in the horizontal and vertical plane. Presentation software (version 10.3) was used for stimulus presentation and response collection.

Stimuli were either letters, numbers, or symbols embedded into a background field of random noise (Fig. 1A). We parametrically manipulated stimulus visibility by modifying the dot density of the target shape, while keeping the dot density of the background constant. This results in a parametric modulation of the signal-to-noise ratio, based on which we created six different visibility levels. For the lowest visibility level, target dot density was similar to that of the surrounding background, creating the perception of just a field of random dots. Visibility was increased by linearly decreasing the dot density of the target. Differences in dot density between target and background provide a grouping cue for segmentation allowing to perceive the illusory contours of the target. To avoid sensory adaptation the location of the dots and of the target were randomized on each trial (i.e., the presentation of each stimulus). All stimuli were created using Matlab (The MathWorks).

Figure 1.

Experimental design A, The task was to rate the visibility of target stimuli embedded on a field of random noise. After a fixation cross of variable length was presented, the target stimuli appeared for 500 ms, followed by a question mark indicating subject to give their visibility ratings. To assess visibility, we used the Perceptual Awareness Scale (Overgaard et al., 2006). B, Bottom-up information was manipulated by degrading the stimulus visibility. Six degradation levels were used. Degradation was achieved by decreasing the dot density of the target stimuli while keeping dot density of the background constant. The successive increases and decreases in stimulus degradation are referred to as “sequence.” For the main experiment, the same stimulus was used during a given sequence, whereas in the control EEG experiment, the stimulus was varied every trial.

Stimuli were displayed at the center of the screen surrounded by a gray background and comprised 13.134 × 10.626°. Forty different stimuli were used (25 letters, 9 numbers, and 6 symbols). In addition, target stimuli were presented in three different sizes, with an absolute vertical extent of the target of 75, 100, and 125 pixels, corresponding to a height of ∼1.637, 2.182, 2.726° visual angle, respectively.

Procedure

In all experiments, the subjects' task was to rate the visibility of the target stimuli on the four-point Perceptual Awareness Scale (PAS) (Overgaard et al., 2006): “1” corresponds to “no experience of the stimulus,” “2” corresponds to “brief glimpse of the stimulus but could not recognize what it was,” “3” corresponds to an “almost clear impression of the stimulus,” and “4” corresponds to a “clear impression of the stimulus.” These questions were aimed at the subjects' fine-grained phenomenal impression of the stimulus. Thus, our scale differs from previously used scales that assessed only confidence (Wilimzig et al., 2008). We provided subjects with verbal as well as written instructions. To assure similar criteria between participants, we extensively discussed the operationalization of PA with the subjects before the experiment started. In particular, we emphasized the distinction between point 2 and point 3 on the scale, the criterion separating no recognition from explicit recognition of the target. Furthermore, we pointed out that they should rate only their perception of the target and that there were neither right nor wrong answers. Responses were given by button presses on a keyboard, with a stress on accuracy rather than speed. To verify that subjects used the PA scale properly, some of the stimuli were clearly visible, which should lead to high visibility ratings, whereas highly degraded stimuli that could not be perceived should be reported as invisible. This allowed us to assess the reliability of the subjects' judgments.

A trial started with a fixation cross presented on a gray background for a random duration of 1000–1500 ms. Then, the random dot field containing the target (letter, number, or symbol) was presented for a duration of 500 ms. After that, a response screen containing a question mark indicated to the subjects that they should rate the visibility of the target (Fig. 1A).

To study the differential effects of sensory evidence and expectancy, in a first experiment, we presented the targets in a sequence of 11 trials (for simplicity, we will refer to this as “sequence”). The same target was used throughout the sequence. The visibility of the target progressively increased, from trial 1 to 6 and then decreased again until trial 11 (Fig. 1B). Thus, once the target became visible on the ascending part of the sequence, subjects could generate target-specific expectancies, the effect of which could be tested in the descending part (hysteresis effect) (Kleinschmidt et al., 2002). The generation of target-specific expectancies can also be regarded as a form of priming (Grill-Spector et al., 2006). We presented 120 sequences that were divided into six blocks (20 sequences each). After each block, a break of variable length was introduced.

To distinguish the specific contribution of the expectancy from the sequential change in sensory evidence, we ran a control experiment in which we used the same sequences but changed the identity of the target in each trial, preventing the generation of target-specific expectancies. If changes in visibility are related to expectancy and prediction about stimulus identity, and not to a mere sequence effect, no signs of hysteresis should be observed in this case. Stimuli were identical with those used in the previous study and were repeated for an equal number of times throughout the experiment. In total, 120 sequences were presented, divided into six blocks. For both studies, we collected EEG data while subjects performed the task.

To further evaluate whether target-specific expectations lead to changes in the threshold of perceptual awareness or changes in response bias, we ran a behavioral experiment in which we presented stimuli, only in descending order of visibility, in three conditions: (1) random sequence, (2) predictable sequence, and (3) interrupted sequence. In the random sequence, target identity was randomly changed for each successive degradation level, precluding any form of target-specific expectations. In the predictable sequence, the same target was presented at all degradation levels. In the interrupted sequence, one unpredicted target per sequence was introduced with a probability of 20% at one of the degradation levels. This creates a scenario of high certainty regarding the future target (80% probability), which is unfulfilled in some trials. In other words, predicted and interrupted sequences only differ in that one unexpected target was introduced per sequence. The six degradation levels were presented in descending visibility order only (from degradation level L6 to L1) promoting the development of target-specific expectations. Random and predictable sequence differed from the second position in the sequence onward, whereas predictable and interrupted sequences could differ from the third position in the sequence. This was done to assure that subjects develop target expectations and thereby allowing us to evaluate the associated visibility to an unexpected target. In total, we presented 180 sequences: 20 random, 80 expected, and 80 interrupted sequences (20 unexpected trials per degradation level), divided into four blocks. The three types of sequences were randomized throughout the experiment. The same stimuli were used for all sequences, and they were the same as those used in the two experiments described above.

Our rationale for these control experiments was that if subjects have a tendency to respond with higher visibility ratings (response bias) simply because they think they know what will be seen next, we would expect higher ratings for unexpected stimuli in the interrupted sequence. Conversely, if expectations truly augment visibility, the ratings of these unexpected stimuli should be the same as those presented under the random sequence.

Data acquisition

All experiments was conducted in an electrically shielded, sound-attenuated, and dimly lit cabin. A TFT monitor was placed outside the cabin behind an electrically shielded window. All devices inside the cabin were battery operated to avoid interference of the line frequency. For study 1 and 2, EEG was recorded with a QuickAmp amplifier (Brain Products) using a custom 128-channel elastic cap (Easy Cap), with average reference and a forehead ground electrode. The electrodes were equally spaced over the whole head and symmetrically placed between both hemispheres. The inferior row of electrodes was placed down to a line from the low cheeks back to several centimeters below the inion, thus providing sufficient coverage of the lower parts of the head. Two additional electrodes were placed on the infraorbital ridges of the left and right eyes, respectively, and two additional electrodes were placed on the neck below the edge of the electrode cap. Data were sampled at 1000 Hz and analog filtered between 0.01 and 100 Hz during recording. Electrode impedances were kept <10 kΩ. Data were digitally saved on a computer outside the cabin for later off-line analysis.

Data analysis

Behavioral data and curve fitting.

Although we applied a four-point visibility scale, for analysis the four-point scale was recoded into a single response category (visibility), collapsing responses 1 + 2 and responses 3 + 4 as “invisible” and “visible,” respectively. This dichotomy was based on explicit recognition of the target in the stimulus. Then, the percentage of the rating “visible” was calculated for each stimulus in the sequence. For the main and control EEG experiment, we analyzed our data using an unconstrained repeated-measure ANOVA and a constrained parametric model. In the unconstrained ANOVA, we treated degradation level and order as two separate factors with six and two levels (ascending and descending sequence), respectively. In the constrained model, we created a new dependent variable (inflection point) by fitting, per subject, a sigmoid function of degradation level to the behavioral data to compute an inflection point. We then tried to explain differences in the inflection point data in terms of a repeated-measure ANOVA with one factor (ascending vs descending sequence). The rating scores for visibility were fitted with a generalized sigmoid function defined as follows: f(x) = 1/(1 + e^{−a(x − ϕ)}) + b, where a determines the slope (steepness) of the sigmoid; b, the offset; and ϕ, the horizontal shift of the function (the threshold). The sigmoid function was fitted to best represent the data points by using a gradient descent method to minimize approximation error. Before the fitting, values were normalized to the interval [0.1]. For our purposes, the most important parameter of the sigmoid was ϕ, because it determines the threshold (i.e., the point on the horizontal axis at which the function reaches one-half of its maximum). If predictions influence visibility this point should shift along the horizontal axis for visibility functions computed from ascending and descending sequences, respectively.

For the behavioral control experiment, we ran an unconstrained repeated-measure ANOVA with factors sequence type (random, predictable, interrupted) and degradation level (four levels, from L4 to L1). Only four degradation levels were included since only in those interruptions of expectations could occur (as required to promote target-specific expectations).

In all ANOVAs with >1 df, we used the Greenhouse–Geisser correction. We report adjusted degrees of freedom and adjusted p values.

To investigate whether the increase in visibility observed in the main EEG experiment is the consequence of a top-down perceptual expectation (the generation of target-specific expectancies) as opposed to mere passive repetition of the stimuli, we run a control analysis taking advantage of the fact that the number of repetitions for each stimulus was the same in the main EEG and control EEG experiment. For the control EEG experiment, we tested whether visibility increased systematically with the number of repetitions. Each stimulus was recoded depending on whether it had been previously presented (old) or not (novel). Old stimuli were further subdivided depending on the number of previous repetitions (from 1 up to 10). Thus, we could explicitly test whether the number of repetitions had an effect on recognition in a repeated-measures ANOVA with factors repetition, order, and degradation level. We further compared zero repetitions (new) with the maximal amount of repetitions (10; old) and when repetitions were regrouped into few (1–3), intermediate (4–6), and many repetitions (7–9).

Analysis of event-related potentials.

For the analysis of event-related potentials (ERPs) of the main and control experiment, the continuous EEG signal was bandpass filtered (0.3–20 Hz) with a phase shift-free Butterworth filter [time constant (in s) = 0.530526; slope (in db/oct) = 24]. For each degradation level, the filtered signal was then segmented in series of 1200-ms-long epochs. Each epoch started 200 ms before the onset of the stimuli. Trials containing voltages exceeding ±100 μV, or gradients exceeding ±50 μV, were rejected. The automatic artifact rejection was supplemented by visual inspection of every trial to reject cases with electrode drifts, eye movements, or electromyographic activity. Artifact-free trials were averaged per degradation level in synchrony with the onset of the stimuli, and baseline corrected over a 200 ms window. Since the different degradation levels led to different ratios of seen versus unseen trials, we did not consider behavioral performance in this analysis. We solely compared the electrophysiological responses between different degradation levels, assuming that differences reflect both the degradation level and differences in visibility. Linear response modulations are expected for components reflecting physical properties of the stimuli, whereas nonlinear responses should reflect the behaviorally assessed visibility (as in Fig. 2A). Additionally, we directly compared seen and unseen trials (see below). Mean amplitudes were analyzed for P1 (110–130 ms), N1 (130–150 ms), P2 (180–270 ms), and P3 (300–500 ms) components. We focused on those ERP components because previous studies have related them either to visual awareness (for review, see Koivisto and Revonsuo, 2010) or to predictions (Garrido et al., 2009). To increase statistical power, mean amplitude was computed over groups of electrodes representative of the topography of each scalp component. Regions of interest (ROIs) were selected based on the compound waveform of all conditions to minimize selection bias favoring a particular condition. For P1, we selected a group of occipitoparietal electrodes (P5, P7, PO7, PO5, P6, P8, PO6, PO8); for N1, the occipito-central electrodes (O1, OZ, O2, O9, Iz, O10); for P2, the occipitoparietal electrodes (P5, P7, PO7, PO5, PO9, O1, O9, Iz, Oz, O2, O10, P6, P8, PO6, PO8, PO10); and for P3, the parieto-central electrodes (CP3, CP1, P1, P3, Pz, CPz, PO1, PO2, P2, P4, CP2, CP4). All EEG processing was done using Brain Vision Analyzer 1 (Brain Products). The Matlab toolbox EEGLAB was used for visualization and topographic plots (Delorme and Makeig, 2004).

Figure 2.

A, B, Behavioral results. Percentage of seen stimuli per degradation level for ascending sequences and descending sequences for the main experiment with predictable sequences (A) and the control EEG experiment with random sequences (B). The black lines refer to ascending sequences, and the gray lines, to descending sequences. C, Subjective threshold data (inflection point) for the main (predictable target) and control (random target) EEG experiment. D, Percentage of seen stimuli per degradation level for random, interrupted, and predicted sequences. In the random sequences, target stimuli varied at each degradation level leading to uncertainty regarding stimulus identity. In the predictable sequences, the same target was used throughout the sequence, thus generating high certainty about stimulus identity. In the interrupted sequences, subjects had high certainty regarding the identity of the upcoming target (80% probability) that was in 20% of the cases unexpectedly unfulfilled. Visibility ratings in the interrupted sequences correspond to ratings given to the unexpected stimuli. Error bars indicate SEM.

We also directly studied amplitude modulations in relation to seen and unseen stimuli for the P1, N1, P2, and P3 components. For this analysis, only data from the main experiment were included. We further restricted our analysis to level 4 stimuli in the ascending and to level 3 stimuli in the descending part of the stimulation sequence because the ratio between seen and unseen stimuli was similar for these levels (∼50%). Preprocessing steps were the same as those described above for the whole sequence. Trials in which subjects identified the stimuli (visibility rating 3 and 4) were categorized as seen, and trials in which subjects could not identify the stimuli (visibility rating 1 and 2) were classified as unseen. Pairwise comparisons confirmed that, after artifact rejection, the ratio of seen versus unseen trials were similar for level 4 (T₍₁₅₎ = 0.816; p = 0.427) and level 3 (T₍₁₅₎ = 1.020; p = 0.324).

As for the behavioral data, we analyzed the electrophysiological data—mean amplitude per ERP component—with an unconstrained ANOVA treating degradation level and order as two separate factors. Thus, both for the behavioral and electrophysiological data, we looked for the main effect of degradation, order, and their interaction. For some components (P1, P2, and P3), region of interest was included as a factor. In all ANOVAs with >1 df, we again used the Greenhouse–Geisser correction (Picton et al., 2000). We report adjusted degrees of freedom and adjusted p values.

Acknowledging that the preselection of electrodes and components might have precluded observing additional components related to awareness (e.g., with a different time course than the ones already investigated) and to further evaluate the time points at which changes in electric field strength distinguish between seen and unseen trials at threshold, we calculated global field power (GFP) (Lehmann and Skrandies, 1980) for level 4 ascending sequence (AS) and level 3 descending sequence (DS). GFP allows for investigation of amplitude differences between conditions without any preselection of electrodes, and thus to contrast conditions across the whole scalp. GFP is equivalent to the spatial SD of the scalp electric field and is calculated as the square root of the mean of the squared value recorded at each electrode (vs the average reference). To statistically test for differences in GFP between experimental conditions, we performed a series of paired nonparametric statistical analyses based on randomization tests (Manly, 1991). The empirical probability distribution of the differences in GFP between seen and unseen conditions was created separately for L4(AS) and L3(DS) by randomly shuffling the conditions within subjects in the original data and recomputing the GFP for these shuffled conditions. This procedure was repeated 65,536 times (2ⁿ, n being the number of subjects; in our case, 16) so as to be able to estimate the probability (p < 0.01) that the difference between the two distributions is observed by chance (Murray et al., 2008). We applied an additional criterion of temporal stability for 10 consecutive time points, corresponding to 10 ms at 1000 Hz sampling rate (for a similar approach, see Murray et al., 2008; Pourtois et al., 2008).

Results

Behavioral results

Figure 2A shows the percentage of responses signaling a stimulus as seen as a function of the degradation level of the stimuli and their place in the presentation sequence, when the target is predictable. A significant effect of degradation level was observed confirming that our manipulation of visibility was successful (degradation level: F_{(1.702,25.535)} = 407.375, p < 0.0001): the percentage of “seen stimuli” increases significantly as degradation level decreases. For all subjects, the curve relating visibility to degradation levels was well fitted by a sigmoid and exhibits a sharp transition between degradation level 3 and 5 from judgments “mainly invisible” to “mainly visible.” Confirming the nonlinearity in the visibility function, we found that the rate of increase in subjective visibility was higher between degradation level 3 (L3) and degradation level 5 (L5)—around the threshold of visibility in the ascending sequence—than for the increase between L2–L3 and L5–L6 (t₍₁₅₎ = −10.227; p < 0.0001) (for a similar approach, see Del Cul et al., 2007). A clear hysteresis effect was observed between the ascending and the descending part of the stimulus series (F_(1,15) = 178.819; p < 0.0001): subjects reported to detect a stimulus more often in the descending (59%) than in the ascending sequence (42%). The increase in visibility attributable to the order effect (hysteresis) was not the same for all degradation levels (degradation level by order: F_{(2.064,30.956)} = 41.900, p < 0.0001), and this effect is explained by a significant quadratic function (F_(1,15) = 126,667; p < 0.0001). Pairwise comparisons contrasting the same physical stimulus (degradation level) during the ascending and descending sequence revealed that all stimuli but L1—which was clearly not distinguishable—showed hysteresis. To quantify this effect, we fitted sigmoid curves, per subjects, to the ascending and descending series and set the subjective threshold at the inflection point of the sigmoid. The threshold for the ascending sequence was at degradation level 3.96 and for the descending sequence at 2.76. Thus, the gain in visibility caused by prior knowledge of the stimulus corresponds to 1.2 degradation steps. Accordingly, the threshold of visibility was reached at degradation level 4 in the ascending (53.02% seen stimuli) and at degradation level 3 in the descending part (54.01% seen stimuli).

To evaluate whether the hysteresis effect results from a memory-based prediction of stimulus identity or from the sequential presentation of the stimuli, we used our control experiment in which the target stimuli differed in each trial, both for the ascending and for the descending sequence. We ran an ANOVA with factors predictable, degradation level, and order. As expected, in the ascending part of the curve, main and control experiment showed comparable percentages of seen stimuli (Fig. 2B). However, during the descending sequence, the percentage of seen responses was significantly higher in the main (59.36%) than in the control experiment (52.42%) (interaction predictable and order: F_(1,8) = 28.849, p = 0.001; interaction predictable by order by degradation level: F_{(2.069,16.551)} = 10.273, p = 0.001). Similar results were obtained when comparing the inflection point data for the control and main experiment (Fig. 2C). Visibility thresholds were similar in the ascending part but differed significantly in the descending part (T₍₈₎ = 3.363; p = 0.01). In summary, the psychophysical functions exhibit clear signs of perceptual hysteresis, suggesting that the buildup of an internal representation of the upcoming stimulus enhances visibility.

Given that the degree of uncertainty regarding the stimulus identity differs between predictable and random sequences, it can be argued that the introduction of target-specific expectations do not lead to real changes in the threshold of perceptual awareness but simply promote subjects to change their response criterion across the predictable sequences. To directly evaluate this possibility, we turned to our behavioral experiment in which stimuli were presented, only in descending visibility order, in three conditions: (1) random sequence, (2) predictable sequence, and (3) interrupted sequence. The critical manipulation was the degree of certainty regarding the identity of the upcoming stimuli. Target stimuli either randomly varied per degradation level (random sequences), creating high uncertainty about the stimulus identity or were kept the same throughout the sequence (predictable sequence), creating high certainty about the stimulus identity. Crucially, for the interrupted sequence subjects had high certainty regarding the identity of the upcoming target (80% probability) that was unexpectedly unfulfilled. As can be seen in Figure 2D, visibility ratings for the unexpected stimuli in the interrupted sequences were the same as for stimuli in the random sequences at each degradation level; both showed lower visibility ratings than for the predicted stimuli in the predictable sequence (sequence type: F_{(1.635,14.712)} = 19.358, p < 0.001; degradation level: F_{(2.122,19.100)} = 163.902, p < 0.001; sequence type by degradation level: F_{(3.321,29.891)} = 1.98, p = 0.133). This shows that the increases in visibility ratings reflect a true change in the threshold of visibility because of target-specific expectations and not a simple response bias, since such a bias would carry over to unexpected targets during the interrupted sequences.

An additional analysis revealed that the mere number of repetitions of a stimulus cannot explain the improved identification levels observed in the main experiment (see supplemental Fig. 1, available at www.jneurosci.org as supplemental material).

Event-related potentials

We focused on the following two questions: (1) which ERP components correlate with the sharp transition in visibility and (2) how ERP components are affected by the change in visibility caused by the buildup of prior expectations. To determine brain activity (ERP components) that directly relates to subjective visibility, we used the following criteria: (1) ERP components should invariably display differential amplitudes for seen and unseen stimuli regardless of whether visibility is solely caused by stimulus saliency or by a combination of saliency and expectancy, and (2) exhibit a similar response profile in terms of their amplitude as the behaviorally assessed visibility. In particular, they should exhibit nonlinear amplitude modulations with a sharp transition at the threshold of visibility. In contrast, linear response modulations were expected for components reflecting physical properties of the stimuli.

Inspection of the ERPs revealed four major components: P1, N1, P2, and P3. For each of these components, we examined whether they exhibited amplitude modulations related to the physical properties of the stimuli (degradation level) or subjective visibility, the order of the stimuli (hysteresis), or the interaction between stimulus visibility and order.

The P1 component showed no modulation neither for degradation nor order and also no interaction between these factors (all p > 0.2) (Fig. 3A). There was only a trend toward reduced amplitude for the stimuli with lowest visibility. This confirms previous results showing that P1 is not related to perceptual awareness (Sergent et al., 2005; Del Cul et al., 2007). The N1 component, when averaged across the ascending and descending sequence, respectively, had a reduced amplitude for the descending compared with the ascending part (effect of order F_(1,15) = 6.122, p = 0.026) (Fig. 3B; supplemental Fig. 1, available at www.jneurosci.org as supplemental material). N1 changes further showed an interaction between degradation level and order (degradation level by order: F_{(3.335,50.030)} = 2.822, p = 0.043), suggesting that the modulation of this component differs for the ascending and descending part. A clear effect of degradation existed for the ascending part (F_{(3.647,54.702)} = 3.386; p = 0.018). The N1 amplitude decreased linearly with increasing visibility (F_(1,15) = 9.639; p = 0.007) as would be predicted by a decrease in prediction error under predictive coding (Garrido et al., 2009). In contrast, no effect of degradation was observed for the descending part (p > 0.9). Thus, in the ascending part, the N1 modulation mainly reflects differences in sensory stimulation, whereas in the descending part it seems to reflect expectations. To test whether expectations differentially modulate responses to the same stimuli, we performed pairwise comparisons between responses to the same physical stimuli (degradation level) in the ascending and descending parts. This analysis revealed amplitude modulations only for those stimuli that were barely visible but not for the highly visible stimuli [(L1 (p = 0.058), L2 (p = 0.010), L3 (p = 0.037), L4 (p = 0.316), L5 (p = 0.116)]. This suggests that top-down effects selectively affect the processing of stimuli with poor visibility (bottom-up information). In fact, the generation of a memory trace seems to change responses to sensory stimulation in such a way that, once expectancy is established, stimuli evoke similar N1 amplitudes regardless of degradation level. In summary, the amplitude of the N1 component is modulated by expectations indicating that prior knowledge affects responses already 130 ms after stimulus presentation. Importantly, however, the modulation of the N1 component does not reflect subjective visibility. The latter follows a sigmoidal function, whereas the N1 component follows the linear change of stimulus degradation, at least in the ascending sequence.

Figure 3.

A–D, Electrophysiological results (main experiment). Amplitude modulations as a function of degradation level and order (ascending sequence/descending sequence) for P1 (A), N1 (B), P2 (C), and P3 (D). Scalp topographies are shown for ascending sequence at degradation level 1 (invisible), degradation level 4 (threshold of visibility), and degradation level 6 (highest visibility).

The other components, P2 and P3, showed clear but different amplitude modulations related to degradation level and subjective visibility. For the P2 component, a within-subject ANOVA with factor degradation level, order, and electrode location (ROI) (three levels: left, right, and central sensors) revealed that P2 amplitude decreases significantly with decreasing degradation (degradation level: F_{(1.530,22.954)} = 52.667, p < 0.001), and this relationship can be well described by a cubic function (F_(1,15) = 15.019; p = 0.001) (Figs. 3C, 4), thus resembling the nonlinear behavior of the psychometric curve. This suggests that the amplitude of the P2 component is inversely related to the visibility of the stimuli. P2 amplitude was also significantly smaller for the descending than for the ascending sequence (order: F_(1,15) = 12.641, p = 0.003), suggesting that perceptual expectations also attenuate this ERP component. However, the attenuation of the P2 component by prior knowledge depends on the degradation level (degradation level by order: F_{(3.449,51.738)} = 6.228, p = 0.001). Pairwise comparisons between responses to the same physical stimuli (degradation level) in the ascending and descending sequence revealed that only responses to barely visible stimuli showed lower amplitudes, whereas responses to clearly visible stimuli showed similar amplitudes [L1 (p = 0.169), L2 (p = 0.003), L3 (p = 0.001), L4 (p = 0.116), L5 (p = 0.479)] (Fig. 3C). This indicates that top-down effects on P2 amplitudes are stronger when bottom-up saliency of the stimuli is low. This suggests that P2 attenuation is mainly related to visibility rather than degradation or expectancy per se. There was also a lateralization effect. P2 amplitudes were higher for the right than for central and left sensors (F_{(1.708,25.615)} = 10.251; p = 0.001) (Fig. 4), which is in agreement with previous findings (Freunberger et al., 2007). Scalp topographies of P2 modulation are shown in Figure 5. Both the lateralization effect and the decreases in amplitude with increasing visibility are clearly visible. It is important to note that, contrary to the modulations observed in N1 amplitude in which an effect of degradation level was only observed for the ascending sequence, P2 showed amplitude modulations as a function of visibility in both the ascending and descending sequences. Thus, P2 is the first ERP component whose amplitude modulation corresponds well to the behaviorally assessed visibility levels, following the nonlinear function of visibility in both the ascending and the descending sequences.

Figure 4.

Waveform P2 component (main experiment). Left (PO7) and right (PO6) parieto-occipital electrodes in which P2 modulations can be clearly distinguished. A shows amplitude modulations per degradation level during the ascending sequence. B shows amplitude modulations during the descending sequence.

Figure 5.

P2 scalp topographies (main experiment). Scalp topographies of P2 amplitude per degradation level and ascending and descending sequence. Clear amplitudes decrements can be observed as a function of visibility.

The P3 component differed from the P2 component in that it showed no main effect of order. Its amplitude increased significantly with decreasing degradation (F_{(2.387,35.798)} = 35.593; p < 0.001). In addition, a significant interaction between degradation level and order was found (F_{(2.588,38.826)} = 7.179; p = 0.001). In the ascending sequence, stimuli that were judged as visible (4–6) elicited a significantly higher (T₍₁₅₎ = −7.072; p < 0.001) amplitude than invisible stimuli (1–3), with a sharp transition at the visibility threshold between degradation 3 and 4 (Fig. 3D; supplemental Fig. 2, available at www.jneurosci.org as supplemental material). In the ascending sequences, no differences in amplitude were found across stimuli judged as visible (all p > 0.08) or invisible (all p > 0.4), respectively. This nonlinearity replicates previous findings relating P3 amplitude to subjective visibility (Del Cul et al., 2007). In contrast, when the analysis was performed on the descending sequence, the separation between visible and invisible stimuli disappeared. Instead, all stimuli but stimulus 1 elicited a P3 response of similar amplitude, despite marked differences in visibility and physical properties (for L1 against all others, all p > 0.03; for comparisons of L2 to L5, all p > 0.2) (Fig. 3D). As for the P2 component, there was a lateralization effect. Responses in the right sensors were more pronounced, especially for stimuli with lower visibility.

To further confirm these results, we analyzed the different ERP components in the condition in which the target stimuli differed for every step in the ascending and descending sequences such that perceptual expectations could not be built up.

As displayed in Figure 6, none of the components showed variations in amplitude when comparing responses to stimuli at equal degradation levels in the ascending and descending sequence (all p > 0.2). In contrast, a clear effect of degradation level was observed for all components (P1: F_{(2.678,21.423)} = 4.209, p = 0.020; N1: F_{(2.925,23.402)} = 4.227, p = 0.016; P2:F_{(1.611,12.886)} = 34.809, p < 0.001; P3: F_{(2.618,20.940)} = 17.836, p < 0.001). With decreasing degradation, P1 amplitude increased (F_(1,8) = 11.459; p = 0.010), whereas the amplitude of the N1 and P2 decreased (F_(1,8) = 11.510, p = 0.009; F_(1,8) = 8.804, p = 0.018, respectively). For the P1 and N1 component, the functions relating degradation with amplitude were linear; for the P2 component, this function was quadratic. The quadratic relationship is mainly explained by a saturation effect at degradation 1 (Fig. 6C). If this degradation level is removed, the modulation of the P2 component becomes also linear both for the ascending and descending sequence. The modulation of the P3 component followed a cubic function (F_(1,8) = 6.386; p = 0.035), with a sharp transition between seen and unseen stimuli, thus correlating well with the psychometric function of visibility. Furthermore, two clear clusters of amplitudes emerged dissociating seen from unseen stimuli: P3 amplitudes were similar for the groups of stimuli with low (1–3) (all p > 0.05) and high visibility (4–6) (all p > 0.05), respectively, whereas clear amplitude differences existed between the groups. This was true for both the increasing and for the decreasing sequence (increasing: T₍₈₎ = −5.481, p = 0.001; decreasing: T₍₈₎ = −5.193, p = 0.001). Waveform traces for the control experiment can be seen in the supplemental Figure 3 (available at www.jneurosci.org as supplemental material).

Figure 6.

Electrophysiological results (control experiment). A–D, Amplitude modulations as a function of degradation level and order (ascending sequence/descending sequence) for P1 (A), N1 (B), P2 (C), and P3 (D).

Effects of visibility at threshold

Finally, we directly tested the factor visibility (seen vs unseen responses) for the data obtained from the main experiment to determine which ERP component varied with subjective visibility when the visual stimulus remains constant. To this end, we used only stimuli at the threshold of subjective visibility: stimuli with degradation level 4 for the ascending sequence, and degradation level 3 for the descending sequence. These stimuli produced similar numbers of seen and unseen trials even though the physical properties of the stimuli differed. We reasoned that ERP components that differentiate between seen and unseen trials at both degradation levels are likely to be related to visibility and not to expectations or differences in sensory stimulation. The only ERP component that differentiates between seen and unseen trials at both degradation levels was P2 (Fig. 7). Significant amplitude reductions for visible stimuli compared with invisible stimuli were found during the ascending sequence for degradation level 4 (seen, 3.450 μV; unseen, 4.810 μV; T₍₁₅₎ = −4.711, p = 0.00028) and during the descending sequence for level 3 (seen, 4.331 μV; unseen, 5.030 μV; T₍₁₅₎ = −3.091, p = 0.007). None of the other components (P1, N1, or P3) distinguished visible from invisible trials at both degradation levels (interaction visibility by degradation level: P1: F_(1,15) = 7.748, p = 0.014; P3: F_(1,15) = 20.429, p = 0.0004). The N1 component showed no modulations in relation to visibility during the ascending and descending sequence (N1: all p > 0.4). The P1 and P3 component showed higher amplitudes for seen compared with unseen stimuli only for the ascending sequence (P1: T₍₁₅₎ = 3.721, p = 0.002; P3: T₍₁₅₎ = 6.601, p > 0.001). Additional time-resolved analyses support the conclusion that the difference between seen and unseen trials is captured by the amplitude of the investigated ERP components (see supplemental Fig. 4, available at www.jneurosci.org as supplemental material).

Figure 7.

Effect of visibility (seen vs unseen trials) at threshold for the P2 component. A, Lower amplitudes were recorded for seen compared with unseen trials for both stimuli at threshold. That is, for degradation level 4 during the AS and degradation level 3 during the DS. B, C, Scalp topographies for seen and unseen trials (B) and their corresponding waveforms (C). Period of significant difference is marked with a dashed line.

Discussion

Our results indicate that expectations alter the threshold of visibility. Subjects identified more stimuli in the descending than in the ascending sequences. Since stimuli in both sequences were identical, changes in visibility must be attributed to predictions about the upcoming stimulus. This was reflected in a shift of the psychometric function. Control experiments ruled out that the effects were attributable to response bias. Predictions have been shown to aid perception (Biederman, 1972; Snodgrass and Feenan, 1990). For example, prior knowledge about the direction and velocity of moving targets enhances their detectability (Sekuler and Ball, 1977; Schwiedrzik et al., 2007). Similarly, previous exposure aids the recognition of incomplete or ambiguous figures (Dolan et al., 1997; Kleinschmidt et al., 2002). Predictions can also have detrimental effects on perception when they are wrong (Bruner and Potter, 1964) or not updated [as in change blindness (Rensink, 2000)].

Signatures of visibility and expectations

By parametrically manipulating stimulus evidence and expectations, we distinguished brain activity related to visibility from activity reflecting sensory properties and/or perceptual expectations of the upcoming stimuli. Our results indicate that even the earliest ERPs change in the presence of expectations. However, not all reflect visibility. Such components should (1) differentiate between seen and unseen stimuli regardless of whether visibility was solely attributable to stimulus saliency or a combination of saliency and expectancy, and (2) show a nonlinear dependence of their amplitude on degradation as the psychophysical function. In contrast, linear response modulations reflect physical stimulus properties.

The control EEG experiment in which no predictions about stimulus identity could be established revealed that P1 and N1 were the only ERPs that followed a linear profile: P1 amplitude increased with decreasing degradation, whereas N1 amplitude decreased. In the presence of expectations, P1 amplitude was not modulated by physical stimulus properties. N1, however, showed linear amplitude decrements as a function of degradation, but only during the ascending sequence. During the descending sequence, N1 amplitude was low and not further modulated. This suggests that N1 is reduced once a hypothesis is built and subsequently remains low, regardless of visibility. This is consistent with predictive coding that anticipates decrements in prediction error as a function of predictability (i.e., top-down predictions that explain away bottom-up inputs via recurrent interactions). Simulations of predictive coding in the auditory domain show that unpredicted stimuli evoke large N1 components, reflecting prediction error when expecting the predicted standard (Garrido et al., 2009). The neuronal processes associated with the visual N1 represent, then, the first stage at which top-down predictions are compared with bottom-up information (Hopf et al., 2002). Studies investigating the effect of predicting words by previous text (Dambacher et al., 2009) and target location by previous experience in visual search (Chaumon et al., 2008) similarly found effects of predictions ∼100 ms after stimulus. Perceptual expectations seem to increase visibility if they are verified by bottom-up information, shortly after input is available.

These effects cannot be explained by attention: spatial and temporal attention were kept constant in our paradigm. It could be argued that attentional load declines as contrast increases, because the number of distracting dots decreases as the target stimulus evolves. N1 amplitude has been shown to be higher for high compared with low load (Rorden et al., 2008). In the main experiment, load decreases during the ascending and increases during the descending sequence. However, N1 amplitude remains low throughout the descending sequence. Thus, attentional load cannot fully explain the observed N1 modulations. Furthermore, a hallmark of attention is the increase in neural activity elicited by attended relative to unattended stimuli (Luck and Hillyard, 1995; Mangun, 1995; Kastner and Ungerleider, 2000). However, the main effect of expectations on N1 was a reduction in amplitude, similar to what is observed for the auditory mismatch negativity (Garrido et al., 2009). Amplitude attenuation for predicted stimuli has also been reported in functional magnetic resonance imaging (fMRI) experiments (Summerfield et al., 2008; Alink et al., 2010). Thus, attention and expectations boost saliency, but their signatures are reversed (Summerfield and Egner, 2009).

Components after N1 also showed an effect of expectations. However, these effects were related to visibility. At threshold, P2 exhibited lower amplitudes for seen compared with unseen stimuli independently of physical stimulus properties. Also, P2 amplitude decreased as visibility increased, with a sharp transition at threshold. Furthermore, during the descending sequence, when expectations are firmly established, P2 amplitude was lower than during the ascending sequence, when responses depended mainly on stimulus properties. Thus, the neuronal processes associated with P2 are related to perceptual awareness and, as the latter, modulated by prior knowledge.

The inverse relationship between visibility and P2 amplitude is proposed to reflect reentrant mechanisms by which perceptual expectations are compared with bottom-up information (Freunberger et al., 2007; Kotsoni et al., 2007). Higher areas send predictions to lower areas, which detect divergence between prediction and evidence, generating an error signal that increases with mismatch (Mumford, 1992; Friston, 2010). If a prediction fails and a stimulus is not seen, the error signal is highest. Di Lollo et al. (2000) and Lamme (2006) propose that feedback–feedforward interactions are the core of conscious perception. Preconscious, top-down hypotheses about incoming stimuli are sent to lower areas where they are iteratively compared with the input (Di Lollo et al., 2000). Consequently, conscious perception is the result of a hypothesis test that iterates until information is consistent across higher and lower areas.

The proposal that P2 indexes reentrant activity is substantiated by source analysis indicating that P2 reflects reactivation of early visual areas including V1 (Di Russo et al., 2008), latency data from intracranial ERPs (Olson et al., 2001), and increased flow of mutual information from extrastriate to striate areas at ∼220 ms (Hinrichs et al., 2008). For object substitution masking thought to target reentrant processing, Kotsoni et al. (2007) found that seen trials elicited lower P2 amplitudes than unseen trials, a pattern also reflected by BOLD activity in V1 (Weidner et al., 2006).

P3 amplitude also correlated with visibility. P3 amplitudes were higher for seen than for unseen trials. However, this was only so during the ascending sequence when perception relied mostly on sensory stimulation. In the descending phase when expectations had been established, P3 amplitude did not differentiate seen from unseen trials. Moreover, the nonlinear increase of P3 amplitude as a function of visibility was only present during the ascending sequence. Thus, P3 fulfills all criteria for a signature of visibility, but only in the absence of expectations. Previous studies have found correlations between P3 amplitude and visibility (Sergent et al., 2005; Del Cul et al., 2007). In a masking paradigm, Del Cul et al. (2007) found that P3 was the only component whose amplitude distinguished visible and invisible stimuli and followed visibility sigmoidally. Had we not manipulated expectations, we would have drawn similar conclusions. However, controlling expectancy, we found that already P2 was related to visibility. Considering P3 latency, it is likely that it reflects postperceptual processes related to visibility (e.g., update of memory) and not visibility as such, at least in our experiment. However, in our control experiment, when no predictions were generated, only P3 showed a sharp transition at threshold, clearly distinguishing between seen and unseen stimuli. Thus, in the absence of expectations, P3 reflects visibility more closely than P2.

Signatures of visibility: early or late?

Previous studies have suggested either early (∼100 ms) (Pins and Ffytche, 2003; Roeber et al., 2008) or late (∼300 ms) (Sergent et al., 2005; Del Cul et al., 2007) activity correlated with visibility. Our results, however, suggest that the signatures of visibility are not bound to processes with a strict latency but depend on the presence of expectations (Banquet and Grossberg, 1987). Latencies are short (indexed by P2, ∼200 ms) when expectations are present, and long (indexed by P3, ∼300 ms) when expectations are absent. Similar changes in latency have been reported in fMRI experiments as a function of previous exposure (James et al., 2000; Kleinschmidt et al., 2002).

An open question is why expectations shorten the latency of signatures of visibility. One possibility is that previous experience speeds up the accumulation of evidence necessary for visibility (James and Gauthier, 2006). Predictions could also take the form of a decisional bias [i.e., a change in the weight of a particular stimulus (higher prior evidence) over others]. Thus, the starting point for the accumulation of evidence lies closer to the decision bound, accelerating the collection of information. Biasing signals correlating with detectability have been observed in lateral occipital complex before stimulus onset (Stokes et al., 2009).

In light of our results, it appears necessary to reinvestigate the neuronal correlates of consciousness, taking into account how cognitive functions (attention, expectations, memory, etc.) influence the timing (and potentially other features) of processes required for access to consciousness. A thorough understanding of the neuronal correlates of consciousness might require a departure from the strategy of merely comparing seen with unseen conditions, and instead necessitates a proper characterization of the interactions among all cognitive processes that ultimately lead to conscious experience (Melloni and Singer, 2010).