Predictive Context Influences Perceptual Selection during Binocular Rivalry

Abstract

Prediction may be a fundamental principle of sensory processing: it has been proposed that the brain continuously generates predictions about forthcoming sensory information. However, little is known about how prediction contributes to the selection of a conscious percept from among competing alternatives. Here, we used binocular rivalry to investigate the effects of prediction on perceptual selection. In binocular rivalry, incompatible images presented to the two eyes result in a perceptual alternation between the images, even though the visual stimuli remain constant. If predictive signals influence the competition between neural representations of rivalrous images, this influence should generate a bias in perceptual selection that depends on predictive context. To manipulate predictive context, we developed a novel binocular rivalry paradigm in which rivalrous test images were immediately preceded by a sequence of context images presented identically to the two eyes. One of the test images was consistent with the preceding image sequence (it was the expected next image in the series), and the other was inconsistent (non-predicted). We found that human observers were more likely to perceive the consistent image at the onset of rivalry, suggesting that predictive context biased selection in favor of the predicted percept. This prediction effect was distinct from the effects of adaptation to stimuli presented before the binocular rivalry test. In addition, perceptual reports were speeded for predicted percepts relative to non-predicted percepts. These results suggest that predictive signals related to visual stimulus history exist at neural sites that can bias conscious perception during binocular rivalry. Our paradigm provides a new way to study how prior information and incoming sensory information combine to generate visual percepts.

Introduction

The visual system often receives ambiguous patterns of stimulation that are compatible with multiple interpretations of the visual environment. It therefore must use additional information to construct a single perceptual interpretation of the world. What is the nature of this additional information, and how does the visual system combine this information with incoming sensory signals to determine the contents of perceptual experience at any given moment? One possibility, based on Bayesian accounts of perception, is that prior knowledge about the likely contents of a visual scene influences the interpretation of sensory signals (von Helmholtz, 1866; Gregory, 1997; Weiss et al., 2002; Kersten et al., 2004; Knill and Pouget, 2004; Kveraga et al., 2007; Hohwy et al., 2008). Indeed, expectations arising from repeated presentations of visual stimuli or explicit instructions have been shown to facilitate processing of expected stimuli, resulting in improved visibility (Sekuler and Ball, 1977; Eger et al., 2007; Esterman and Yantis, 2010; Melloni et al., 2011) and both speeded (James et al., 2000; Eger et al., 2007; Esterman and Yantis, 2010) and enhanced (Dolan et al., 1997) recognition of visual stimuli.

Since natural environments are structured in time, one potentially rich source of prior information is patterns of visual stimulation in the recent past. Predictive coding frameworks describe how such a prior might be represented by neural activity, proposing that the brain continuously generates predictions of forthcoming sensory signals (Rao and Ballard, 1999; Friston, 2005). Comparisons of brain activity during expected and unexpected sensory stimulation have provided physiological evidence consistent with these frameworks (Summerfield and Koechlin, 2008; Alink et al., 2010). However, the effects of predictive neural signals on conscious perception have not been well explored. In particular, little is known about how prediction may influence the selection of a specific percept from competing alternatives.

A few studies have used ambiguous stimuli to provide initial insights into this question. For example, it has been shown that pairing secondary cues with rotating stimuli whose direction of rotation is defined by binocular disparity allows these cues to influence perception of rotation direction when disparity information is removed, making the physical rotation direction ambiguous (Haijiang et al., 2006; Sterzer et al., 2008). Specifically, the secondary cues increase the probability that the ambiguous stimuli will be perceived to rotate in the same direction as in the preceding conditioning period, showing that cue-induced expectations can influence perceptual selection. In addition, priming one perceptual interpretation of a binocular rivalry stimulus using either unambiguous low-contrast stimuli (Brascamp et al., 2007; Pearson et al., 2008) or mental imagery (Pearson et al., 2008) has been shown to bias perception during subsequent rivalry in favor of the primed percept.

Closer to the question of prediction, Maloney et al. (2005) found that recent visual experience influences the perception of an ambiguous apparent motion quartet. In this study, subjects viewed sequences of quartets with unambiguous rotation directions followed by an ambiguous quartet that could be perceived as rotating either clockwise or counterclockwise. Subjects’ perceptual reports were influenced by the pattern of the preceding sequence, with an increased probability of interpreting ambiguous motion in a manner that was consistent with the expectation generated by the sequence.

Binocular rivalry provides a powerful and well-studied paradigm for investigating the effects of predictive context on visual perceptual selection. Binocular rivalry occurs when incompatible images are presented to the two eyes, leading to a perceptual alternation between the images, even though the visual stimuli remain constant. Unlike many other types of multistable percepts (Liebert and Burk, 1985; Peterson, 1986; Toppino, 2003; Shimono et al., 2011), binocular rivalry is often only weakly susceptible to cognitive control (Meng and Tong, 2004). In addition, there is evidence that binocular rivalry can be resolved at stages of visual processing as early as monocular regions of V1 (Polonsky et al., 2000; Tong and Engel, 2001) and the LGN (Haynes et al., 2005; Wunderlich et al., 2005), although this point continues to be debated (Logothetis et al., 1996; Lee and Blake, 1999; Blake and Logothetis, 2002; Tong et al., 2006). Therefore, studying the effects of predictive context on perceptual selection in binocular rivalry may improve understanding of the role of expectation in early visual processing. In one theoretical proposal, perceptual alternations during binocular rivalry are a product of predictive coding mechanisms (Hohwy et al., 2008), but specific hypotheses arising from this framework have not yet been experimentally tested.

In order to investigate the effects of predictive visual information on perceptual selection, we developed a novel binocular rivalry paradigm. On each trial, we first presented a sequence of identical images to the two eyes that generated an expectation about the next image in the series. We followed this predictive sequence with a rivalry display in which the predicted image was presented to one eye and a non-predicted image was presented to the other eye. We found that subjects were initially more likely to select the predicted image than the non-predicted image. In three additional experiments, we showed that only patterns of visual stimulation in the recent time period before the onset of rivalry contributed to the prediction effect and that prediction of the upcoming stimulus and adaptation to preceding stimuli had separate influences on perceptual selection. We also observed speeded perceptual selection of predicted relative to non-predicted stimuli. Our results suggest that predictive signals exist at neural sites that contribute to perceptual selection during binocular rivalry, and they emphasize the importance of prior information in determining the contents of conscious visual experience.

General Methods

Subjects

Forty-five subjects participated in one or more of the experiments. Five data sets were excluded from analysis (see Subject Exclusion), resulting in a total of 41 subjects (aged 18–41 years, 27 female), 15 of whom participated in Experiment 1, 8 in Experiment 2, 16 in Experiment 3, and 13 in Experiment 4. Two of the authors participated in two experiments, and one author participated in all four experiments. All subjects provided informed consent, and all experimental protocols were approved by the Committee for the Protection of Human Subjects at the University of California, Berkeley.

Visual stimuli

Stimuli were generated on a Macintosh PowerPC computer using Matlab and Psychophysics Toolbox (Brainard, 1997; Pelli, 1997) and were displayed on two halves of a gamma-corrected NEC MultiSync FE992 CRT monitor with a refresh rate of 60 Hz at a viewing distance of 100 cm. Subjects viewed all stimuli through a mirror stereoscope with their heads stabilized by a chin rest. Visual stimuli were circular patches, 1.8° in diameter, and were surrounded by a black annulus with a diameter of 2.6° and a thickness of 0.2°. Binocular presentation of this annulus allowed it to serve as a vergence cue to stabilize eye position and to ensure that the rivaling stimuli were presented to corresponding retinal locations in the two eyes. All stimuli were presented at 10% contrast on a neutral gray background (luminance of 59 cd/m²), except in Experiment 4, in which the contrast of the stimuli was varied. All stimuli had the same mean luminance as the background.

On each trial, subjects viewed a stream of items presented identically to both eyes (the “pre-rivalry stream”), followed by a pair of rivalrous stimuli. A brief auditory cue signaled the start of each trial. Each stream item was presented for 300 ms and was followed by a 100 ms blank period. The rivalrous test stimuli were always two monochromatic, sinusoidal grating patches with a spatial frequency of 3 cpd and orthogonal (±45°) orientations. Rivalrous stimuli were presented for 4, 5, or 10 s in Experiment 1 (fixed stimulus duration for a given subject, with N = 4, 5, and 6, respectively), 5 s in Experiments 2 and 4, and 10 s in Experiment 3.

One of the rivalrous test gratings always had an orientation that was consistent with the preceding predictive context; that is, it was the expected next image following the pre-rivalry stream. We call this the “matching” stimulus, since it matches the predictive context. The orientation of the other rivalrous test grating (the “non-matching” stimulus) was orthogonal to that of the matching stimulus and was inconsistent with the predictive context.

Rivalry task

After passively viewing each pre-rivalry stream, subjects continuously reported their percept during presentation of the rivalrous test stimuli by holding down one of two keys: (1) grating tilted to the left, and (2) grating tilted to the right. Subjects were instructed to begin responding whenever the stimuli stopped moving or changing orientation in a regular fashion, to press a key continuously for as long as the corresponding percept persisted, and not to press any key for ambiguous percepts. Trials in which there was no response during the rivalry period were excluded from the analysis.

Measures of perceptual selection

We expected that predictive context effects would be strongest at the beginning of the rivalry period, so analysis was focused on the initial response to the rivalry stimuli. In particular, we measured the proportion of trials in which the initial percept was the matching vs. the non-matching grating. We also measured the latency and duration of the initial response for both matching and non-matching percepts. The experiments were designed to investigate the effects of predictive context on initial rivalry percepts, and the relatively short rivalry presentation durations did not allow these effects to be assessed for subsequent percepts.

Catch trials

To ensure that subjects were following task instructions, approximately 10% of the trials in each experiment were catch trials, in which both eyes viewed identical left- or right-tilted gratings in the “rivalry test” portion of the trial. Catch trials were counterbalanced for grating orientation predicted by the stream (left or right tilt) and direction of tilt of the test stimuli (left or right). Catch trial latencies were used to assess the possibility of response bias, since response bias would be expected to lead to shorter response latencies for catch trial stimuli matching perceptual expectations than for non-matching catch trial stimuli.

Eye dominance screening

Before participating in the study, each subject’s eye dominance was measured in a brief pre-test. On each of 24 trials, subjects viewed static orthogonal rivalrous gratings with ±45° orientations for 10 s and continuously reported their percept as described above. Pre-rivalry streams were not presented in these screening trials. Eye dominance was defined as the relative number of trials in which the initial perceptual report corresponded to the grating presented to the left eye vs. the right eye.

Subject exclusion

Subjects with strong eye dominance were excluded because a substantial bias in favor of the left or right eye during binocular rivalry limits assessment of the effects of experimental manipulations in this study. Subjects whose initial eye dominance in either eye was greater than 85% during the eye dominance screening session were excluded and did not participate in any portion of the study. We also measured eye dominance throughout each experiment by analyzing initial rivalry responses and excluded subjects who had >85% eye dominance for at least half of the experimental session.

In addition to subjects who did not pass the initial eye dominance screen, five data sets from individual subjects were excluded from specific experiments (one from Experiment 1, three from Experiment 3, and one from Experiment 4). In each of Experiments 1 and 3, one subject was excluded for exhibiting excessive eye dominance during the experiment. In Experiment 3, one subject was excluded as an outlier (proportion first response matching was more than 2.5 SD away from the sample mean for one condition comparison). Finally, one subject was excluded in each of Experiments 3 and 4 for using incorrect response keys.

Experiment 1

Subjects viewed a predictive stimulus stream consisting of a series of oriented gratings presented identically to the two eyes. This stream generated a percept of rotating apparent motion, thereby establishing an expectation regarding the orientation of the next image in the series (Figure 1A). We measured the effect of this predictive context on subsequent perceptual selection during rivalry between orthogonal gratings. We hypothesized that predictive and sensory information would be integrated, increasing the likelihood of selecting the predicted percept. In this framework, predictive context functions as a prior that influences perceptual interpretation of the ambiguous visual stimuli.

Figure 1.

Experiment 1: Predictive context in a pre-rivalry rotation stream influences initial perceptual selection in binocular rivalry. (A) Schematic of the stimuli (pre-rivalry stream and rivalry test). This example stream has clockwise rotation and five stream items shown. Stream items were always presented identically to both eyes, and rivalry items were always a pair of gratings with orthogonal ±45° orientations, with one of the two gratings matching the rotation direction (i.e., it was the expected next item in the rotation stream). (B) Subjects initially perceived the rivalrous grating that matched the rotation direction more often than they initially perceived the non-matching grating whenever the number of stream items was sufficient to define a rotation direction (two or more stream items). The size of this effect was similar for all stream durations from 2 to 15 items. Error bars are SEM.

Methods

The pre-rivalry stream consisted of a sequence of monochromatic sinusoidal grating patches with a spatial frequency of 3 cpd. Orientations of successive stream items either increased or decreased by 45°, generating an apparent motion percept of rotation in either the clockwise or counterclockwise direction (Figure 1A). In the subsequent rivalry test, one of the two static gratings (the “matching” grating) had the orientation that came next in the apparent-motion series, and the other grating (the “non-matching” grating) had an orientation orthogonal to that predicted by the stream. The orientation of the first pre-rivalry stream stimulus was selected so that the rivalrous gratings would always have oblique (±45°) orientations.

There were four pre-rivalry stream conditions: number of stream items (between 0 and 15), the grating orientation predicted by the stream (left or right tilt), the eye to which the “matching” rivalrous grating was presented (left or right eye), and stream rotation direction (clockwise or counterclockwise). The four stream conditions were fully counterbalanced across trials, resulting in a 16 × 2 × 2 × 2 design. Subjects completed either 24 or 32 trials for each stream length, and all conditions were randomly intermixed.

Results

Perceptual selection is biased in favor of the predicted percept

A rotation stream of variable length (0–15 stream items; Figure 1A) was followed by presentation of a rivalrous pair of test gratings. The rotation stream generated a consistent percept of rotating apparent motion in either a clockwise or counterclockwise direction, and one of the rivalrous stimuli was consistent with this apparent motion percept (the “matching” stimulus), while the other (the “non-matching” stimulus) was not.

The results provide clear support for our hypothesis that prediction would influence perceptual selection: for rotation streams with two or more items (the minimum number needed to establish an apparent motion percept), perceptual selection in binocular rivalry was consistently biased in favor of the matching grating. Specifically, the matching grating was initially selected on about 60% of trials, regardless of the number of items in the stream (Figure 1B).

Experiment 2

In Experiment 1, the predictive context provided by rotation streams with 2–15 items enhanced selection of the matching grating, but the size of this effect did not depend on the length of the stream. We therefore asked in Experiment 2 whether very recent stimulus history (only the two items immediately preceding the rivalry test) was sufficient to bias perceptual selection, even for longer stream conditions.

Methods

Half of the streams were composed of gratings that rotated either clockwise or counterclockwise (as in Experiment 1), and the other half were scrambled such that each orientation in the rotation stream was presented in a random position in the scrambled stream sequence (Figure 2A). However, for both rotation and scrambled trials, the final two stream items preceding the rivalrous test stimulus were always consistent with a particular rotation direction. This rotation direction defined the “matching” and “non-matching” rivalrous test grating. For scrambled streams, we ensured that there were no complete rotations in the stream by requiring at least two items in the first part of the stream to be different from the original rotation sequence. There were five stream conditions which were fully counterbalanced across trials: stream type (scramble or rotation), number of stream items (between 4 and 7), and the same final three conditions as in Experiment 1 (grating orientation predicted by the stream, the eye to which that matching grating was presented, and stream rotation direction). Subjects completed 24 trials for every combination of stream type and number of stream items, and all conditions were randomly intermixed.

Figure 2.

Experiment 2: Effects of predictive context depend only on recent stimulus history. (A) Schematic of the stimuli. For the scrambled condition, the order of the gratings in the first part of the rotation stream (preceding the final two items) was randomized so that the first part of the stream did not contain a consistent rotation. However, the two stream items immediately before the rivalry test were always consistent with a particular rotation direction (in this example, clockwise) for both rotation and scrambled trials. (B) Scrambling the sequence of orientations in the stream prior to the final two stream items did not diminish the rotation matching effect. Error bars are SEM.

Results

Recent stimulus history drives predictive effects on perceptual selection

We compared initial perceptual selection for rivalrous test stimuli presented after streams with either a scrambled sequence of orientations in the initial part of the stream or with fully coherent rotation throughout (Figure 2A). If consistent predictive stimulus history over an extended viewing period is required for the rotation matching effect, then disruption of predictive context in the early part of the stream in the scrambled condition should reduce the size of this effect, compared to the full rotation condition. However, if only recent stimulus history is responsible for the rotation matching effect, then the size of the effect should be identical in the rotation and scrambled conditions, and this is the result that was obtained.

For rotation streams, we found increased selection of the predicted percept (combining all stream lengths), replicating the results of Experiment 1 [t(7) = 21.28, p < 0.001; Figure 2B]. Scrambled streams also generated a significant rotation matching effect [t(7) = 29.77, p < 0.001; Figure 2B], suggesting that consistent rotation throughout the stream was not required for the effect. Moreover, there was no significant difference between the rotation and scrambled conditions in the size of the mean matching effect across all stream lengths [paired t-test, t(7) = 0.19, n.s.; Figure 2B]. Together, these results show that the predictive context provided by only the two items immediately preceding the rivalry test was sufficient to maximally bias perceptual selection in this paradigm.

Experiment 3

Orientation-specific adaptation to stream gratings might have contributed to the rotation matching effects observed in Experiments 1 and 2. The stimuli in these experiments controlled for adaptation to the final stream item, because the angular difference between the final stream grating and each of the rivalrous gratings (both matching and non-matching) was always 45°. However, it was still possible that the rotation matching effect was influenced by orientation-specific adaptation to the second-to-last stream item (Figure 3A). This grating always had the same orientation as the non-matching rivalry grating, so it was possible that adaptation to this grating biased perceptual selection against the non-matching grating orientation (e.g., Blake and Overton, 1979, but also see Brascamp et al., 2007), perhaps contributing to the rotation matching effect. We conducted Experiment 3 to compare the effects of prediction and adaptation on perceptual selection in this paradigm.

Figure 3.

Experiment 3: Separate contributions of prediction and adaptation to the matching effect. (A) Schematic of the stimuli. Left: An example rotation stream showing how adaptation to the second-to-last stream item could bias selection toward the matching stimulus. Right: To control for adaptation, the final stream item was replaced with one of three other stimuli (plaid, bullseye, or blank; the blank condition is shown here). These alternative final stream items reduced the perception of rotation while maintaining any orientation-specific adaptation to the second-to-last stream item. (B) The size of the rotation matching effect decreased with increasing disruption of rotation perception. The bullseye and blank conditions quantify the effects of adaptation alone, as no perception of rotation was possible for these conditions. The matching effect for the rotation condition was significantly larger than that for the bullseye or blank condition, indicating an effect of predictive context that cannot be accounted for by adaptation. Error bars are SEM. **p < 0.01; ***p < 0.005.

Methods

Subjects viewed two pre-rivalry stream items on all trials, based on our finding from Experiments 1 and 2 that two stream items were sufficient to produce the rotation matching effect. The first stream item was always a sinusoidal grating with an orientation of either +45° or −45°, presented to both eyes. The second stream item, also presented binocularly, was one of the following: a vertical or horizontal grating (generating, together with the first stream item, apparent clockwise or counterclockwise rotation, as in Experiment 1), a blank (mean luminance), a bullseye pattern of 3-cpd sinusoidal concentric circles, or a plaid pattern formed by superimposing vertical and horizontal 3-cpd gratings.

The blank, bullseye, and plaid stimuli were designed to disrupt rotational apparent motion perception for the pre-rivalry stream, compared to the rotation stimulus. In all trials, the orientation of the first stream item determined the orientation of the test grating that would be consistent with perceived rotational motion and therefore defined which rivalrous test grating was “matching” and which was “non-matching.” Each trial had four fully counterbalanced conditions: stream type (the four types described above) and the same final three conditions as in the earlier experiments (grating orientation predicted by the stream, the eye to which that matching grating was presented, and stream rotation direction). Each subject completed 48 trials for each stream type, and all conditions were randomly intermixed.

Results

Separate effects of prediction and adaptation on perceptual selection

To determine the contribution of adaptation to the matching effect, we created streams that preserved the second-to-last stream item, thereby maintaining orientation-specific adaptation, but that altered the final item in the stream, thereby reducing or eliminating the perception of stream rotation (Figure 3A). Perceptual selection for these reduced predictive context streams was compared to that for a full rotation condition.

This experiment included a total of four stream conditions (Figure 3B). Matching effects in the rotation condition could be due to prediction, adaptation, or some combination of these factors. In the blank and bullseye conditions, there was no apparent-motion percept (and therefore no predictive information available), so any bias in perceptual selection for this condition could only be due to adaptation. Finally, the plaid condition was an intermediate condition in which the presence of both vertical and horizontal grating components in the plaid may have interfered with the perception of apparent motion in the stream without abolishing it altogether. This is because both vertical and horizontal components were consistent with the same “matching” rivalry grating. For example, a left-tilted grating followed by a plaid could be seen as clockwise apparent motion if the vertical plaid component were emphasized or as counterclockwise apparent motion if the horizontal plaid component were emphasized, but both of these apparent rotation percepts predict a right-tilted matching grating during the rivalry test. The plaid condition therefore contains some predictive context but presumably generates a weaker apparent motion percept than the rotation condition.

The results of Experiment 3 revealed separate effects of adaptation and prediction on perceptual selection (Figure 3B). There were reliable matching effects in the adaptation-only conditions (blank and bullseye), indicating that orientation-specific adaptation to the second-to-last stream item biased selection against the non-matching (adapted) grating. However, the matching effect for the rotation condition (resulting from both adaptation and prediction effects) was significantly larger than the adaptation-only effects [rotation vs. blank, t(15) = 3.36, p < 0.005; rotation vs. bullseye, t(15) = 2.96, p < 0.01], indicating that prediction enhances the rotation matching effect beyond what is found for adaptation alone. The size of the matching effect for the plaid was in between that of the rotation condition and the adaptation-only conditions, as expected if this stimulus produced intermediate levels of apparent motion perception. Thus, the effects of pre-rivalry stream rotation on perceptual selection of binocular rivalry stimuli reflect a combination of adaptation and prediction effects that can be experimentally dissociated.

Experiment 4

The strength of orientation-selective adaptation depends on stimulus contrast (Blakemore and Nachmias, 1971), while predictive context is provided for any contrast for which the stream items are visible. We therefore conducted Experiment 4 to measure the contrast dependence of the adaptation and prediction effects described above.

Methods

The rotation and blank stream conditions from Experiment 3 were used, and the items in the streams had 5, 25, or 100% contrast. The contrast of the rivalrous gratings was also 5, 25, or 100%, independent of the stream contrast. Thus, each trial had six fully counterbalanced conditions: stream type (rotation or blank), stream item contrast, rivalrous stimuli contrast, and the final three conditions as in the earlier experiments. “Matching” and “non-matching” rivalrous test gratings were defined as in Experiment 3. Each subject completed 32 trials for every combination of stream type, stream item contrast, and rivalrous stimuli contrast, and all conditions were randomly intermixed.

Results

Effects of stimulus contrast dissociate prediction and adaptation

We independently varied the contrast of the stream items and of the rivalrous stimuli for both the rotation and blank conditions from Experiment 3 and observed a main effect of stream contrast [ANOVA, F(2,48) = 11.95, p < 0.001; Figure 4], with increasing stream contrast causing a dramatic increase in the magnitude of the matching effect in the blank (adaptation-only) condition. However, increasing stream contrast led to a smaller increase in the matching effect in the rotation condition [stream contrast × stream type interaction, F(2,48) = 10.74, p < 0.001], mainly due to the significantly larger matching effect for the rotation compared to the blank condition at the lowest stream contrast [ANOVA for 5% stream contrast condition, main effect of stream type, F(1,24) = 7.31, p < 0.05]. At this low contrast, adaptation is weak, and prediction effects dominate.

Figure 4.

Experiment 4: Effects of stream contrast on prediction and adaptation. Increasing the contrast of the stream items increased the size of the matching effect for the blank condition more than for the rotation condition. The prediction effect corresponds to the difference between the rotation and blank values and was greatest at the lowest stream contrast. The blank condition quantifies the effects of adaptation alone, and these effects were larger for higher stream contrasts. Error bars are SEM. *p < 0.05.

We also observed a main effect of contrast of the rivalrous test gratings [F(2,48) = 8.65, p < 0.005], with the size of the matching effect decreasing as rivalry stimulus contrast increased, for both rotation and blank conditions [no interaction between rivalry stimulus contrast and stream type: F(2,48) = 0.01, n.s.]. This effect of rivalry stimulus contrast could be because competition between higher contrast rivalry stimuli is less affected by either prediction or adaptation. Because the blank condition represents only adaptation effects, while the rotation condition includes effects of both adaptation and prediction, the lack of a significant interaction indicates that adaptation was the more important factor in the effect of rivalry stimulus contrast. Figure 4 displays data for each stream contrast condition, collapsed across rivalry stimulus contrast.

The different effects of stream contrast in the rotation and blank conditions provide an additional dissociation of prediction and adaptation effects. This experiment showed a robust prediction effect even for a very low (but still above the visibility threshold) stream contrast of 5%, while adaptation effects were minimized at this contrast. These results suggest that low-contrast stimuli can be used to reduce adaptation effects, providing a strategy for emphasizing prediction effects that could be employed in future studies of predictive context. Experiments 1–3 used 10% contrast for both pre-rivalry stream and rivalry stimulus items, and this relatively low contrast may have helped to reveal prediction effects in these experiments.

Additional Results

Prediction speeds perceptual selection as measured by latency to report initial percept

So far we have demonstrated that predictive context affects which percept is initially selected during binocular rivalry. We also analyzed the effects of predictive context on the latency and duration of the initial response to the rivalrous test stimuli. We present latency data for Experiment 3 because it contains the most effective experimental control of adaptation effects. For the conditions that contain predictive context (rotation and plaid), response latencies were shorter for matching than for non-matching initial percepts, while no differences between matching and non-matching initial percepts were observed for the bullseye and blank conditions, which lack predictive context [Figure 5; rotation, t(15) = 4.26, p < 0.001; plaid, t(15) = 3.06, p < 0.01; bullseye, t(15) = 1.73, n.s.; blank, t(15) = 1.63, n.s.]. Therefore, adaptation effects alone do not reliably speed perceptual report, while prediction does. Similar results were also obtained in Experiments 1, 2, and 4: we found shorter response latencies for matching than for non-matching initial percepts, indicating that predictive context speeded perceptual report for predicted stimuli.

Figure 5.

Effects of prediction and adaptation on latency of initial response. Results from Experiment 3 are shown. Response latencies were shorter for initially reported matching compared to non-matching rivalry stimuli. This effect was observed only in the rotation and plaid conditions, suggesting that prediction but not adaptation speeded perceptual report. Error bars are between-subject standard errors of the difference between matching and non-matching percept response latencies. **p < 0.01; ***p < 0.001.

We observed less consistent effects of prediction on the duration of the initial response across the experiments. Our ability to accurately estimate the initial response duration may have been affected by the duration of the rivalry test period, which was 10 s or shorter, depending on the experiment. Because of this limited response window, some initial responses were maintained until the end of the trial and terminated at that point. When we excluded those truncated responses, we found a longer mean duration of initial responses for matching than for non-matching stimuli in the rotation condition for all four experiments. In Experiment 3, the difference in mean first response duration between matching and non-matching trials with non-truncated initial responses was significant only in the rotation condition [rotation, t(15) = 2.82, p < 0.05; plaid, t(15) = 0.19, n.s.; bullseye, t(15) = 0.19, n.s.; blank, t(15) = 1.11, n.s.]. This suggests that prediction prolongs the predicted initial percept compared to the non-predicted percept.

Possible contribution of response bias to predictive context effects

In principle, the effects of predictive context we report could be due to perceptual selection of matching stimuli and/or a response bias in favor of these stimuli. However, data from catch trials argue against a simple response bias as the source of our predictive context effects. Every experiment contained catch trials, in which the pre-rivalry stream sequences were the same as in the experimental trials, but instead of a rivalrous pair of gratings, the same grating was presented to both eyes. For these unambiguous test stimuli, there were no significant effects of predictive context in any experiment, either on the initially selected percept (proportion of responses matching the rotation direction) or on the initial response latency (matching vs. non-matching responses). These findings suggest that predictive context did not result in errors in perceptual report that led subjects to report the matching percept when they did not actually see it and did not result in subjects responding to a matching stimulus more quickly, given identical perceptual latencies for matching and non-matching stimuli.

Discussion

Our experiments provide the first evidence that predictive information influences perceptual selection during binocular rivalry: stimuli that were consistent with the established predictive context tended to dominate over inconsistent stimuli in initial perceptual selection. We further characterized three key aspects of the effects of predictive information on perceptual selection in our paradigm. First, we showed that only recent visual stimulus history contributed to the prediction effect. As few as two stream items (the minimal number required to establish a rotation direction) produced the maximal rotation matching effect (Experiment 1), and introducing random sequences of grating orientation prior to these two stream items did not change the size of the matching effect (Experiment 2). Second, prediction and orientation-specific adaptation separately contributed to the matching effect (Experiment 3), and the results of Experiment 4 suggest a strategy for minimizing the influence of adaptation, namely using a low stream contrast. Adaptation effects were reduced for low stream contrasts, while prediction effects were robust for all tested contrasts. Third, subjects were faster to report initial percepts that matched the predictive context compared to those that did not. In Experiment 3, this effect was specific to the prediction conditions and was not found in the adaptation-only conditions.

Predictive context and response latency

We found that the latency of the first response was shorter for percepts that matched the expectations established by predictive context. Physiological studies also suggest that prediction may reduce the latency of neural responses to expected stimuli. Melloni et al. (2011) found that EEG activity differentiating seen and unseen stimuli occurred about 100 ms earlier when the visual stimulus was expected compared to when it was unexpected. In addition, James et al. (2000) showed that BOLD responses peaked earlier for primed than for unprimed visual stimuli in a manner that correlated with behavioral report. Finally, latencies of single cell responses to images embedded in natural sequences are shorter than response latencies for the same images presented in isolation (Perrett et al., 2009).

Effects of priming on perceptual selection

Perceptual history has previously been shown to contribute to perceptual selection during binocular rivalry in various priming paradigms. Intermittent presentations of rivalry stimuli tend to stabilize the perceptual interpretation, such that the perceptual alternations characteristic of continuous rivalrous viewing are markedly slowed (Leopold et al., 2002; Pearson and Brascamp, 2008). In this case, priming arises from a neural signal associated with the previous perceptual decision and not with the stimulus per se, since the stimulus is always ambiguous. Unambiguous primes can also increase the likelihood that the primed stimulus will be selected during subsequent rivalry. This effect depends strongly on contrast, with lower contrast primes facilitating subsequent selection of the prime, and higher contrast primes suppressing it as a result of adaptation (Brascamp et al., 2007; Pearson et al., 2008). Mental imagery can also bias subsequent selection during rivalry toward the previously imagined percept (Pearson et al., 2008).

In the present study, the rotating pre-rivalry stream could be considered a prime for the predicted rivalry stimulus. However, our experiments are importantly different from previous binocular rivalry priming studies in that the predicted (“primed”) orientation never appeared immediately before the rivalry period and indeed was often not presented at any time during the pre-rivalry stream. Therefore, the effects of predictive context in our study could not be due to a residual memory trace from a previously presented stimulus but instead were due to a predictive signal specific to the expected grating orientation. Likewise, our predictive effects were likely not due to subjects imagining the expected next stimulus, since imagery effects are negligible for brief imagery durations (Pearson et al., 2008), and the rivalry stimuli were always presented immediately after the pre-rivalry stream in our experiments. Nonetheless, it is possible that selection biases due to stimulus priming, imagery, and prediction share common neural and/or psychological substrates, a question which will be of interest in future research.

Attention and predictive context

It is possible that increased allocation of attention to the features of the expected stimulus may have played a role in the prediction effects we observed. Exogenously cueing attention to one of two superimposed stimuli has been shown to increase the likelihood that the cued stimulus will initially dominate when the two stimuli are made rivalrous (Ooi and He, 1999; Mitchell et al., 2004; Chong and Blake, 2006; Hancock and Andrews, 2007). Similar effects on initial dominance have been found when endogenous attention is directed toward one of the stimuli during a difficult task prior to binocular rivalry (Chong and Blake, 2006).

It should be noted, however, that in these studies, a cue draws attention to a currently visible stimulus, thereby increasing the likelihood that the cued stimulus perceptually dominates in a subsequent rivalry period. This is different from our study, in which the grating presented immediately before the rivalry display (the final stream item) has equal angular distance from the two rivalrous gratings. If attention were simply cued to the features of the final grating in the pre-rivalry stream, it would not favor either of the rivalrous gratings.

In creating predictive context that generates an expectation about the appearance of a future stimulus, our study should also be distinguished from previous studies of expectation that have used instructions to generate an attentional set for a particular kind of stimulus (Summerfield et al., 2006; Summerfield and Koechlin, 2008; Summerfield and Egner, 2009). In these studies, many types of stimuli appear with equal likelihood, but only one type (the “expected” stimulus) is relevant for performing the task. We might call expectations of this type “attentional expectations.” In our study, on the other hand, subjects presumably expect that a stimulus rotating in a particular direction will continue to rotate in that direction, but the predicted and non-predicted stimuli (matching and non-matching orientations) are equally task relevant. Such expectations about the likely future state of the stimulus are “perceptual expectations.” An important task for future research will be to understand how these two types of expectations are represented in the brain and how they influence the processing of sensory signals (Summerfield and Egner, 2009).

Finally, attention and predictive coding mechanisms are thought to have different effects on sensory responses in the brain, with attention facilitating (Carrasco, 2011) and predictive coding mechanisms suppressing responses at early stages of visual processing (Summerfield and Koechlin, 2008; Garrido et al., 2009; Alink et al., 2010; Melloni et al., 2011; but see Spratling, 2008, 2010 for an attempt to reconcile effects of attention and predictive coding in a single model). An attention-based account of our predictive context effects would postulate enhanced responses in neurons representing the predicted stimulus at lower hierarchical levels of the visual system, while reduced responses in these areas would be consistent with predictive coding models.

Relationships with other effects of predictive visual motion context on perception

The prediction effects we describe may share mechanisms with recently reported effects of predictive motion extrapolation on a visual detection task (Roach et al., 2011). In this study, detection performance for patterned targets at the leading edge of a moving grating was measured, and the results suggest that the visual system generates a predictive signal resembling a low-contrast extrapolation of the grating in the direction of motion. The effects depended on the spatial phase of the gratings and extended over only about 1° of visual angle, leading the authors to speculate that they could be mediated by cortical area V1. A similar weak but pattern-specific representation generated by extrapolation of rotational motion could also account for the predictive rivalry effects we observed. Our results suggest that this type of motion signal extrapolation could influence not only visual sensitivity but also perceptual selection during ambiguous visual stimulation.

Our findings may also be related to the phenomenon of representational momentum – the observation that memory for the final position of a moving target is mislocalized in the direction of motion (Freyd and Finke, 1984; Hubbard, 2005). Representational momentum can be observed following presentation of a series of discrete views of a rotating target, in which motion was implied (as in our study; Freyd and Finke, 1984; Freyd and Johnson, 1987). The spatiotemporal continuity of motion may be a particularly strong prior that could play a role in a range of perceptual and neural effects (Watamaniuk and Mckee, 1995; Doherty et al., 2005).

Priors and perception

An important question for future research is the extent to which the predictive effects we report generalize to other types of priors. Although they are not always discussed in a Bayesian framework, other rivalry studies have also documented what may be the effects of priors on perceptual selection. For example, images with natural image statistics tend to dominate over more artificial images (Baker and Graf, 2009), upright faces tend to dominate over inverted faces (Engel, 1956; Zhou et al., 2010), and images of floors tend to dominate over images of ceilings (Ozkan and Braunstein, 2009). These findings, including our own, can be interpreted as empirical evidence for a long-standing notion, that perception is an inference process (von Helmholtz, 1866; Gregory, 1997; Kersten et al., 2004; Kveraga et al., 2007). Bayesian modeling of perception of ambiguous visual displays has been a particular focus of theoretical work in this vein (Dayan, 1998; Schrater and Sundareswara, 2007; Hohwy et al., 2008; Sundareswara and Schrater, 2008; Gershman et al., 2009), and recent empirical work shows that Bayesian cue combination can explain perception of a bistable depth stimulus (Moreno-Bote et al., 2011).

Predictive coding and neural mechanisms of binocular rivalry

In predictive coding models of the visual system (Mumford, 1992; Rao and Ballard, 1999; Friston, 2005; Friston and Kiebel, 2009), higher levels of the visual hierarchy predict upcoming responses in lower levels, and these predictions are compared with actual responses in the lower levels via an inhibitory mechanism. Residual signal in the lower levels therefore serves as an error signal that is then transmitted to higher levels in order to improve future predictions. According to this model, posterior information about the percept is represented at higher hierarchical levels, and the dominant percept corresponds to the perceptual hypothesis with the highest posterior probability. Top-down predictions therefore explain away predicted bottom-up signals, and so the representation of a stimulus at the lower levels should have reduced error-related activity while that stimulus is perceived (Murray et al., 2002; Friston, 2005; Hohwy et al., 2008; Summerfield and Koechlin, 2008; Alink et al., 2010).

Neurophysiological studies during binocular rivalry have yielded mixed results regarding correlations between perception and activity in different visual areas. Few (if any) neurons in early visual areas such as the LGN (Lehky and Maunsell, 1996; Wilke et al., 2009) and V1 (Leopold and Logothetis, 1996) have spiking responses that vary as a function of perception during binocular rivalry. In contrast, later visual cortical areas such as V4, MT, and IT have more neurons with perceptually correlated responses (Logothetis and Schall, 1989; Leopold and Logothetis, 1996; Sheinberg and Logothetis, 1997). This increase in the proportion of neurons whose activity reflects the perceptual interpretation of a rivalry stimulus at increasingly higher levels of the visual processing hierarchy is consistent with predictive coding frameworks, in that the highest levels of predictive coding hierarchies should most closely reflect the final perceptual interpretation. That being said, these neurophysiological studies all employed stimuli that were matched to the response preferences of the recorded neurons in each visual area, raising the possibility that perception-related neural modulation depends on stimulus complexity, as neurons in higher-order areas respond preferentially to more complex stimuli than those in lower-order areas. However, even for similar rivalrous grating stimuli, the proportion of neurons with perceptually modulated responses is higher in V4 than in V1/V2, making it unlikely that stimulus complexity is the only factor accounting for differences between visual areas in percept-related modulations (Leopold and Logothetis, 1996).

In predictive coding frameworks, lower hierarchical levels should carry an error signal for suppressed percepts, and the existence of neurons in V4 (Leopold and Logothetis, 1996) and MT (Logothetis and Schall, 1989) that show enhanced responses during rivalry suppression of their preferred stimulus may be consistent with this. On the other hand, Leopold and Logothetis (1996) did not find V1/V2 neurons that showed enhanced responses when their preferred stimulus was perceptually suppressed, which is at odds with predictive coding models and may be an interesting avenue for further investigation.

In contrast to single cell activity, fMRI and low frequency (<30 Hz) LFP signals have been shown to correlate strongly with perception during binocular rivalry in visual areas as early as V1 (fMRI: Polonsky et al., 2000; Tong and Engel, 2001; Lee and Blake, 2002; Lee et al., 2005; LFP: Wilke et al., 2006) and the LGN (fMRI: Haynes et al., 2005; Wunderlich et al., 2005; LFP: Wilke et al., 2009). In the context of predictive coding, these responses could be interpreted as reflecting top-down predictive feedback from higher cortical regions (Hohwy et al., 2008), particularly if BOLD and LFP signals primarily reflect synaptic inputs to a given population of neurons (Logothetis et al., 2001; Logothetis, 2008).

Conclusion

The extent to which the resolution of binocular rivalry is driven by competition between representations at lower levels, higher levels, or multiple hierarchical levels in the visual system has been the subject of much debate (Logothetis et al., 1996; Lee and Blake, 1999; Blake and Logothetis, 2002; Tong et al., 2006). Our approach of experimentally manipulating top-down priors on perceptual selection could help to clarify this question by providing experimentally distinguishable hypotheses about how prior information and sensory information combine within neural circuits. Such studies could be especially informative when psychophysical manipulations of prior information are combined with physiological measures of neural activity at different hierarchical levels in the visual system.

Here, we have demonstrated predictive effects on perceptual selection during binocular rivalry. Therefore, predictive context influences what is often thought to be a low-level competitive process in a manner consistent with theories of predictive coding. Our findings suggest that the visual system uses recently encountered visual information to help construct a single perceptual interpretation of a scene. Thus, predictive information may play an important role in natural vision by helping to constrain perceptual interpretations of the visual world to those that are most consistent with the recent past.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.