When the brain changes its mind: Interocular grouping during binocular rivalry

Abstract

The prevalent view of binocular rivalry holds that it is a competition between the two eyes mediated by reciprocal inhibition among monocular neurons. This view is largely due to the nature of conventional rivalry-inducing stimuli, which are pairs of dissimilar images with coherent patterns within each eye’s image. Is it the eye of origin or the coherency of patterns that determines perceptual alternations between coherent percepts in binocular rivalry? We break the coherency of conventional stimuli and replace them by complementary patchworks of intermingled rivalrous images. Can the brain unscramble the pieces of the patchwork arriving from different eyes to obtain coherent percepts? We find that pattern coherency in itself can drive perceptual alternations, and the patchworks are reassembled into coherent forms by most observers. This result is in agreement with recent neurophysiological and psychophysical evidence demonstrating that there is more to binocular rivalry than mere eye competition.

Binocular rivalry is produced by providing dissimilar views for the two eyes that cannot be fused into a single percept and give rise to spontaneously alternating percepts. A tug-of-war is created in the brain when the two images of Fig. 1A fall on corresponding retinal locations, and the percept flips between the monkey face and the jungle scene every few seconds. Visual signals arriving from two corresponding retinal locations are interpreted by the brain as coming from the same location in three-dimensional space. Rivalry is based on a natural constraint: two things cannot occupy the same space at the same time in the observed world. At what level and how is this constraint implemented in the brain?

Figure 1.

The dichoptic pairs shown in A–D induce binocular rivalry when brought into correspondence by means of converging (or diverging) the eyes (the black fixation marks should be fused). These stereo pairs offer a critical test for investigating the role of eye competition and pattern coherence in binocular rivalry. (A) Conventional rivalry inducing pair: eye of origin and pattern coherence are correlated. Alternation of the monkey face and the jungle scene is observed. (B) Patchwork rivalry stimulus: eye of origin and pattern coherence are uncorrelated. Alternation of the monkey face and the jungle scene can still be observed, which is unexplained by eye competition theories of rivalry. (C) Conventional color rivalry condition: eye of origin and pattern coherence are correlated. (D) Patchwork color rivalry condition: eye of origin and pattern coherence are uncorrelated. Eye competition would only predict mixed percepts here; however, all-red and all-green percepts are also observed. Similar stimuli were used in the reported experiments. Each eye’s frame was tessellated into fictitious abutting rectangular tiles (four columns by seven columns), and an element was placed within each tile with a random positional jitter. Elements were equiluminant to the uniform yellow background, the luminance of which was 7.5 candelas per m², and their color was low-pass filtered spatially with a circular Gaussian envelope that had a σ of 11.2 min of arc. Thus, the element’s color was pure red (or green) at its center, and it gradually blended with the background at its edges. The Gaussian bell was truncated to zero at a radius of 16.4 min of arc. The equiluminant settings were individually obtained for each observer using a heterochromatic flicker photometry technique.

According to conventional accounts of binocular rivalry, the alternation is due to interocular suppression, mediated by reciprocal inhibitory interactions among monocular neurons (1–5). The common theme of all these models is the notion that the neuronal mechanisms of the eye that receives the temporarily dominant percept suppress the corresponding mechanisms of the other eye at homologous retinal locations. This suppression is assumed to take place before binocular combination, already at the level of monocular input to the lateral geniculate nucleus or primary visual cortex.

Since both members of a conventional rivalrous pair, such as that in Fig. 1A, are coherent patterns, it is not clear whether the dominance of one percept over the other is due to the temporary dominance of the eye that the pattern originated from or due to the temporary dominance of the coherent pattern itself. In other words, does the monkey face of Fig. 1A dominate because all its details are arriving from a dominant eye, or because it is a nice and coherent monkey face? To make this distinction, we replace conventional stimuli by complementary patchworks of intermingled rivalrous images. Eye of origin and pattern coherency are decorrelated in our paradigm of “patchwork rivalry.” Can the brain unscramble the pieces of the patchwork arriving from different eyes to obtain a coherent percept in Fig. 1B?

Obviously, theories of eye competition would predict alternation between the monkey face and the jungle scene in Fig. 1A and alternation between the two patchwork images in Fig. 1B. However, one can observe alternation between the monkey face and the jungle scene even in Fig. 1B. Only pattern coherency can explain these interocularly grouped percepts. The principle of eye competition is necessarily violated during those instants when the face comes together as a whole, or the text can be read throughout the picture. Local competition rules of binocular rivalry [related to, for example, stimulus strength (6, 7)] are also overridden, and global stimulus structure seems to determine the fate of each local patch with respect to perceptual suppression/dominance.

Fig. 1B demonstrates that complex images that are defined by several stimulus dimensions can induce complex, interocularly grouped percepts. Pattern coherency in these complex images might include not only texture similarity or contour continuity at the lower ends of the cortical processing hierarchy, but also semantic interpretation or familiarity at the higher ends. To clearly define the role of pattern coherency, let us simplify the image, and take a single attribute from the many dimensions: chromaticity. Coherency will indicate color similarity-based grouping for that single attribute. Fig. 1C is a conventional rivalry inducing stimulus in these terms, with coherent color pattern within each eye’s image, while Fig. 1D is a patchwork rivalry stimulus, with mixed colors within each eye (notice that we have removed all cues other than color and eye of origin). Eye of origin and color similarity are pitted against each other in the patchwork stimulus of Fig. 1D. We find a tendency for color similarity-based grouping in both Fig. 1 C and D. We note that stable fixation and some experience with rivalrous stimuli are important for obtaining the unified percept in patchwork rivalrous stimuli. When fixation is not stable and the patches are not on corresponding retinal locations, the visual system may find other solutions (e.g., fusion of similar patches). Strong dominance of one eye may also result in difficulties obtaining the coherent percepts.

To find out how powerful pattern coherency is in driving color rivalry as opposed to eye of origin, we measured the probability of grouped (all-red or all-green) and ungrouped (mixed color) percepts in conventional and patchwork conditions. We took advantage of the fact that artificially induced binocular rivalry is one of the rare instances when transitions from one conscious perceptual state to another are so clearly observed that the length of each transition phase can be measured. The probability of each percept can be then approximated by the cumulative duration of the percepts during relatively long observation periods.

Rivalrous stereo images were displayed on a computer-driven monitor, and observers viewed the images through a prism-and-mirror haploscope. Each eye’s image consisted of a set of nonoverlapping red and green elements placed randomly in a 6° × 6° rectangular frame that was centered around the fixation mark. The right-eye image contained elements in exact positional correspondence with the left eye, but of the opposite color, designed to produce binocular rivalry (as illustrated in Fig. 1 C and D). Elements were equiluminant to the uniform yellow background. During an initial practice session, observers were instructed to keep the binocular fixation cross in registration, and they were asked to report verbally all the possible percepts they could obtain. Four possible stable percepts were identified across the observers: (i) “all-red” and (ii) “all-green” percepts, in which all the elements appeared to be of only one color; (iii) “mixed-color” percept, in which some elements appeared as red and the rest appeared as green; and (iv) “all-yellow” percept, in which only the yellow background and fixation marks were perceived, and the elements disappeared. In the main experiments, the observers viewed the stimuli for at least 24 trials per condition, each lasting 2 min, during which they were instructed to report changes in the percept as soon as they occurred, by pressing one of four buttons, corresponding to the new percept that the change resulted in. These actions were recorded by the computer, together with the time of their occurrence, and were analyzed off-line to yield data on the time course of the perceptual changes. For each trial, we measured the length of each perceptual dominance period.

Table 1 shows that the observed frequency of the mixed percept is 0.37 and that of a uniform (all-red or all-green) percept is 0.60 in the conventional color rivalry condition of Fig. 1C. What determines these frequencies? The simplest assumption is that perceptual alternations occur randomly and independently for each local patch within our stimuli. In that case, the probability of a uniform percept (p_u = 2 × 1/2ⁿ, where n is the number of bistable patches) would be incredibly small. Even if one assumes that independent alternations occur in larger patches, the predicted probabilities will be smaller than what we observed: for n = 8, p_u = 0.007; for n = 4, p_u = 0.125; and for n = 2, p_u = 0.5. It is also obvious that the observed p_u = 0.60 is less than it would be expected in the case of a completely global alternation (n = 1, p_u = 1.0), revealing that mixed percepts also arise [in agreement with earlier observations on “patchy” rivalry (8) of large stimulus fields]. This analysis indicates that there must be some organizing forces that give the impetus to override—although not at a full extent—random local fluctuations of monocular activity.

Table 1.

Summary of the temporal dynamics of color-rivalry in conventional (A) and patchwork (B) arrangements

Parameters	Percepts
	All-Red		All-Green		Mixed		Yellow
	A	B	A	B	A	B	A	B
N	1045	737	1181	978	1399	1154	228	207
Average duration, sec	2.16	1.88	2.68	2.18	2.35	3.22	1.18	1.23
Relative cumulative duration, %	25.1	18.5	35.3	28.5	36.6	49.6	3.0	3.4
λ	4.51	4.85	4.35	5.1	3.6	3.76	—*	—*
r	4.89	5.11	4.76	5.53	3.51	3.72	—*	—*

Results are averaged across three observers (A.F., M.Y., and T.Y.P.). Average duration of each percept is expressed in seconds, the relative cumulative duration is expressed in percentages. λ and r are parameters of the theoretical Gamma distributions shown in Fig. 2. 

*Gamma functions could not be fitted for the yellow percept. 

In the simple stimulus conditions we used, there are only two cues that can serve organization: eye of origin and chromaticity of the patches. These two cues are perfectly correlated in the conventional color—rivalry condition, therefore it is not discernible which one of them has the organizing power. The patchwork color rivalry condition can help to clarify this issue, because eye of origin and chromaticity are decorrelated in this condition. If one assumes that a uniform (all-red or all-green) percept can only be driven by eye of origin, then the probability of a uniform percept in the patchwork condition would be practically zero. On the other hand, if one assumes that chromaticity can bring about uniform percepts independently of eye of origin, we would expect p_u = 0.60 (same as in the conventional condition). However, we find p_u to be 0.47. This implies that chromaticity is indeed a strong organizing force in itself, but may not be completely independent of eye of origin [in agreement with observations on interactions among parallel processing streams in the visual system (9)].

Fig. 2 shows frequency histograms of the relative dominance durations—i.e., normalized by dividing by the mean. Histograms for the all-red and all-green percepts are shown for conventional (Fig. 2A) and patchwork (Fig. 2B) images. We find that the data are well fit with a Gamma distribution, which has been well documented in other binocular rivalry studies (10, 11). Parameters of the Gamma distribution are presented in Table 1. The shape of the Gamma distribution can be though of as expressing the combination of two tendencies of multistable percepts: first, the tendency to change states at random with an exponential distribution (exp(−λx), decaying part of the curve) and, second, the tendency to stay at the current state due to some “inertia” (x^r−1, ascending part of the curve). The distributions of Fig. 2 indicate that the readiness to change states during a uniform percept is very similar in conventional and patchwork conditions. This would not be expected if only eye of origin played a role in generating uniform percepts. Thus, we conclude again that chromaticity cues can generate uniform percepts, lending further credence to the hypothesis that eye suppression per se cannot explain binocular rivalry.

Figure 2.

Histograms of normalized phase durations for the all-red (Left), all-green (Right) percepts in conventional (A) and patchwork (B) color rivalry conditions. Results are averaged across three observers (A.F., M.Y., and T.V.P.). Data are fitted using the Gamma function (smooth black lines)—i.e., f(x) = [λ^r/Γ(r)] x^r−1exp(−λx), with Γ(r) = (r−1)!. The parameters of the Gamma functions (see Table 1) were not significantly different in the two conditions.

Local examples of pattern coherency in the condition of binocular rivalry have been found earlier (12–15). Treisman (14) and Kulikowski (15) reported that binocularly viewed patches colored in antiphase (e.g., a red-green vs. a green-red bar) appear to be of the same color (either red or green). In accord with Kulikowski’s observation, we find the most enhanced chromatic rivalry in isoluminant color conditions. We extend chromatic rivalry for spatially distributed stimuli, where a uniform percept requires grouping of distinct red (or green) patches over a large region. Local chromatic rivalry (14, 15) might still be explained by the interaction between adjacent ocular dominance columns; however, interocular grouping, where spatially separated elements are grouped together based on their color, not on the eye-of-origin, cannot be explained by that. The success of our paradigm to demonstrate interocular grouping is based on the removal of fine spatial details from the borders of the local monocular patches. Blurred edges allow the visual system to temporarily overcome local mechanisms, such as the stringent binocular fusion of patches with similar image properties.

There is additional psychophysical evidence for the competition of sensory events independently of the eye of origin in a different paradigm, modulating the stimulus structure in time rather than space. During the relatively long (2–3 sec) dominance phases of binocular rivalry, Logothetis et al. (16) swap the stimuli between the eyes every 330 msec. The distribution of the dominance periods is not disturbed by this swapping, indicating that the brain is trying to make sense of both pictures together, not simply suppressing the input from one eye. Although these studies support the stimulus competition hypothesis against the eye competition hypothesis, we must notice the possibility that the dominance periods are simply initiated by interocular suppression, and inertia keeps them from quick ceasing as the stimuli are being swapped. The inertia parameter is in the range of a few hundred milliseconds for binocular rivalry (17) and about 300 msec for binaural listening (18). The inertia parameter of the brain might be set to perceptually optimal values, reflecting, for example, how often the environment changes. Fortunately, the inertia problem is easily overcome by applying the patchwork images described in the present paper, because the patchworks involve grouping of elements from the two eyes simultaneously.

Binocular rivalry is a very unique phenomenon regarding the search for the neural substrates of perceptual awareness (19). This is partly due to the fact that binocular vision is mediated by well-defined pathways that can be followed easily from the two retinal images, through the monocular layers of the lateral geniculate nucleus and the cortex, to the binocular layers of several cortical areas. What is the neural locus of binocular rivalry and interocular grouping? Suppression of one eye or suppression of ocular dominance columns can only explain conventional rivalry conditions, and would never lead to one-color percepts in chromatic rivalry of patchwork images. For all-green or all-red percepts to occur in the patchwork condition, monocular information has to be selectively channeled through neural units that are smaller than the ocular dominance columns. Color information from certain regions within an ocular dominance column might be channeled forward, while other sites might be suppressed (or simply not channeled) at the same time according to the global stimulus structure. This might, in fact, result from the superposition of ocular dominance columns and cortical color maps, still at a monocular level, but would probably require specific feedback pathways providing information about the global structure. The only known feedback pathway that could veto input from one eye in case of conflicting information is the corticofugal pathway from the primary visual cortex to the lateral geniculate nucleus. However, most of the cells of the corticofugal pathway are nonspecific and monocular or strongly biased toward one eye or the other (20, 21), leaving little room for binocular interactions to happen at this level. It has been demonstrated recently that “rivalry neurons,” which follow the perceptual alternations of rivalrous stimuli in their firing, are not monocular, but binocular neurons and can be found at several levels of the cortex [area 17 of cats (22, 23); V1/V2, V4, and the midtemporal cortex of awake monkeys (23, 24)]. Based on this information, the most plausible model rejects input attenuation (i.e., assumes no suppression at the monocular level) and leaves the burden of selection to later stages of processing.

In summary, interocular grouping is a novel way of binocular stimulus combination. It clearly indicates that binocular rivalry can be driven by pattern coherency, not only by eye of origin. The reported phenomena show that the brain has many different ways to assemble new “realities” from competing pieces of concurrent external and internal events. With respect to the neural locus of the assemblage, it might occur beyond the input layers of the visual cortex. That implies that even rivalrous input is fully represented in the primary visual cortex—although it is probably represented only for a short time and is not fully available for visual awareness (see also refs. 26 and 27).