Normalization regulates competition for visual awareness

Summary

Signals in our brain are in a constant state of competition, including those that vie for motor control, sensory dominance and awareness. To shed light on the mechanisms underlying neural competition, we exploit binocular rivalry, a phenomenon that allows us to probe the competitive process that ordinarily transpires outside of our awareness. By measuring psychometric functions under different states of rivalry, we discovered a pattern of gain changes that are consistent with a model of competition in which attention interacts with normalization processes, thereby driving the ebb and flow between states of awareness. Moreover, we reveal that attention plays a crucial role in modulating competition; without attention, rivalry suppression for high-contrast stimuli is negligible. We propose a framework whereby our visual awareness of competing sensory representations is governed by a common neural computation: normalization.

Introduction

Our visual environment is brimming with information, but the high bioenergetic costs of cortical computations limit how much of that information can be effectively processed at any given moment (Lennie, 2003). Because of this limitation, the brain is chronically dealing with competition amongst neural representations of objects and events. One prominent mechanism for regulating competing neural signals is attention, which allows us to selectively process relevant information (Reynolds and Chelazzi, 2004). A recent model proposes that attention shapes perception by means of a normalization framework, whereby attentional modulation hinges on three critical factors: the locus of attentional modulation, the size of the attended stimulus, and the size of the attentional window (Reynolds and Heeger, 2009; Herrmann et al, 2010). Changes in any of these factors can tip the balance between neuronal excitatory and inhibitory processes, thereby impacting how attention affects perception (Reynolds and Heeger, 2009).

Is this normalization framework a general property of visual competition? Although attention can deftly regulate neural representations, the process of conflict resolution is not always so seamless. In some instances the visual system struggles to reconcile competing sensory information, a compelling example being when dissimilar images are presented to the two eyes. In this case, visual awareness alternates between the two images, creating the phenomenon known as binocular rivalry (Blake and Logothetis, 2002; Leopold and Logothetis, 1999; Tong et al., 2006; Wheatstone, 1838). Binocular rivalry offers a unique opportunity to probe competitive processes within the brain, by allowing us to see with our own eyes a process that ordinarily transpires outside of our awareness, namely dynamic competition between neural representations (Blake and Logothetis, 2002; Leopold and Logothetis, 1999). Here, we explore whether a common neural computation may mediate competitive processes embodied in rivalry and in attention.

The notion that attention and rivalry are intertwined has been debated for over a century (Helmholtz, 1909; James, 1890; Lack, 1978; Zhang et al, 2011; Watanabe et al. 2011), and some have gone so far as to directly attribute the alternations in visual awareness to switches in attention (Helmholtz, 1909, Lack, 1978). Moreover, a growing body of research suggests that modulation of visual awareness through rivalry in early visual cortices depends on attentional state (Lee et al, 2007; Zhang et al, 2011; Watanabe et al. 2011). However, the mechanisms subserving these interactions between attention and rivalry remain unknown. Here, we develop and test the idea that attention and rivalry reconcile competing visual information via a common framework, one in which modulation of awareness through rivalry interacts with attention. We propose a computation model for visual competition, whereby modulation of competing neural signals relies on interactions between normalization and attention: gain modulation depends on the size of the competitor stimulus and the attentional state. Finally, we empirically test a core prediction of this computational model, revealing that the degree of suppression between competing neural representations is regulated by attentional state.

Experiment 1: Contrast response and visual awareness

The normalization model of attention makes a very clear prediction: changing the size of the ‘attentional field’ relative to the stimulus will differentially modulate the signal’s contrast response, causing either contrast gain or response gain modulation depending on the configuration (Reynolds and Heeger, 2009; Herrmann et al, 2010). Under binocular rivalry, the stimulus presented in one eye typically abolishes the visibility of a rival stimulus in the other eye. For the moment, we propose that the ‘attentional field’ be more generally conceptualized as a ‘modulatory field’ that can either boost the response to a stimulus, as is the case with attention, or attenuate the response to a stimulus, as is the case with rivalry suppression.

With rivalry, the size of this modulatory field can be directly controlled by changing the size of a stimulus in one eye relative to the other. With standard models of binocular normalization, introducing a stimulus in a competing eye should contribute to the pooled inhibitory component of normalization (Ding and Sperling, 2009; Moradi and Heeger, 2009), which predicts shifts in contrast gain (strongest effects at mid-contrasts), but not in response gain (strongest effects at high contrasts), regardless of size. However, if rivalry also includes a process that behaves like attention, the shape of contrast response functions for attenuated signals should differ depending on the size of the dominant stimulus in the other eye –a manipulation that would alter the size of the modulatory field. Specifically, when the dominant stimulus is substantially larger than the stimulus in the other eye, thereby evoking a large modulatory field, the normalization framework of attention predicts a reduction in contrast gain for the probe stimulus (Fig 1a). However, when the dominant stimulus evokes a small modulatory field, the contrast response functions should transition towards a reduction in the response gain (Fig 1b).

Fig. 1.

Examples of competing stimuli used in the experiment, and their predicted impact on contrast response functions. (a) In some trials, the probe stimulus was dichoptically suppressed by a large stimulus. Under this configuration, the normalization model predicts a contrast gain shift, with the largest effects occurring at mid-contrasts, and little-to-no effect at low and high contrasts. (b) In other trials, the probe stimulus was the same size as the stimulus in the competing eye. Under this configuration, the normalization framework predicts both a shift in the contrast gain, as well as an attenuation of the response gain, with the largest effects occurring at high contrasts.

To explore whether normalization modulates visual competition, we examined how psychometric functions change for an attenuated stimulus under rivalry, and whether those changes depend on the size of the putative modulatory field. We measured observers’ ability to discriminate fine changes in the orientation of a probe stimulus (4° clockwise or counterclockwise) that was either presented monocularly, or was suppressed under binocular rivalry (Fig 2). To control the size of the modulatory field in the rivalry conditions, we manipulated the size of the dominant competing stimulus such that in some trials, it was either the same size as the probe (small: 1.5°), somewhat larger than the probe (medium: 2.5°), or substantially larger (large: 8°). The R.M.S. contrast of the probe stimuli ranged from 0.8–23%, allowing us to measure the entire psychometric function, a behavioral measure that scales proportionally to the signal-to-noise ratio of the underlying contrast response function (Herrmann et al., 2010; Pestilli et al, 2009). Specifically, changes in the neural contrast response function under this framework directly impacts an observer’s ability to discriminate orientation changes in the probe, which would in turn be reflected in corresponding changes to the behavioral psychometric functions.

Fig 2.

Example trial sequence in Experiment 1. Two competing bandpass-filtered noise stimuli were viewed dichoptically. To control rivalry state, we used the flash suppression technique. Following flash-induced suppression, the probe in the suppressed eye changed orientation slightly (4° clockwise or counter-clockwise), and observers reported the direction of tilt of that probe (two-alternative, forced-choice task).

Results: Experiment 1

Rivalry had a substantial impact on psychometric functions (Fig 3a). Specifically, the size of the dominant stimulus evoked notable qualitative differences in implied contrast response functions: whereas a large, dominant stimulus solely shifted the contrast gain, a intermediary dominant stimulus reduced both the response gain and the contrast gain, and a small dominant stimulus further reduced the response gain. To quantify these effects, we fit the data for each observer with Naka-Rushton functions (Herrmann et al, 2010; Pestilli et al, 2009; Naka and Rushton, 1966; Ling et al, 2010), for which two key parameters are predicted to change under the normalization framework: C50 and d′_max. These parameters have been used in previous psychophysics studies as metrics for changes in contrast gain and response gain. The C50 parameter corresponds to the semi-saturation constant, and changes in this parameter with rivalry suppression indicate a contrast gain shift. The d′_max parameter corresponds to the asymptotic response at high contrasts, and changes in this parameter indicate a response gain reduction. Parameter estimates revealed a pattern consistent with predictions of the normalization model of attention: C50 shifted towards higher contrasts for dominant stimuli regardless of their size, whereas d′_max was attenuated the most when the dominant stimulus was the same size as the probe stimulus. Consistent with these results, response gain-like modulation has previously been found with rivalry when similar-sized stimuli are pit against each other, both in single-unit (Sengpiel and Blakemore, 1994) and behavioral studies (Ling et al, 2010, Watanabe et al, 2004).

Fig 3.

The effect of rivalry on contrast psychometric functions. (a) Example psychometric functions from one observer. When the competing stimulus was the same size as the probe, we found both a reduction in the asymptote, or response gain, as well as a shift in the probe’s psychometric function, or contrast gain. However, as the size of the competitor increased, the response gain modulation decreased, and when the competing stimulus was much larger than the probe, the psychometric functions only shifted in their contrast gain. (b) C50 estimates for each observer, indexing changes in contrast gain. We found decreased contrast gain (higher C50) under suppression relative to an unsuppressed stimulus, regardless of the size of the competing stimulus. (c) d′_max estimates for each observer, indexing changes in response gain. We found the strongest decrease in the response gain (lower d′_max) when the competing stimulus was the same size as the probe stimulus, and this reduction in response gain diminished as the competitor size increased. Error bars are bootstrapped 95% confidence intervals.

Fitting the data separately for each individual yielded a similar pattern of results (Fig 3b,c; Fig S1). When the dominant stimulus was large, there was solely a change in C50 for all observers (Fig 3b), with no change in d′_max (Fig 3c). However, as the size of the competitor approached that of the probe, changes in both C50 and d′_max emerged. While standard normalization models would only predict a contrast gain shift (Moradi and Heeger, 2009), our results indicate that an additional mechanism is needed to account for our results; indeed, the conjoint reduction in both contrast gain (C50) and response gain (d′_max) when the dominant stimulus is small is a prediction borne from the normalization model of attention for scenarios where the probe is small and the modulatory field is roughly the same size (Reynolds and Heeger, 2009).

One alternative explanation for the large competitor’s inability to suppress high contrast probes is center-surround interactions that plausibly could weaken the strength of the center region of the competing stimulus. Although center-surround inhibition has been shown to be least effective in the fovea (Petrov et al, 2005), the retinal region targeted by our stimuli, we sought to rule out this alternative explanation explicitly by performing an additional control experiment, where we measured the degree to which the surround region of the large stimulus attenuated its center portion (Fig S2).

In a given trial of the control experiment, observers were shown two consecutive displays: one with a surround, and one without a surround (Test stimulus). The size of the surround matched the size of the large suppressor stimulus in our main study (8°). Observers performed a 2-interval forced choice task, indicating which one of the two intervals contained a center stimulus with higher perceived contrast. We used an adaptive staircase procedure to estimate the contrast the Test needed to match the perceived contrast of the stimulus in the presence of a surround. Using this task, we found that the surround reduced perceived contrast of the target by only ~0.08 log-units, implying the involvement of very weak surround suppression at best; with the surround, a 23% contrast stimulus appeared to observers as if it were between 18–19% R.M.S. contrast (S1 = 18.2%, S2 = 18.9%, S3 = 19.2%, S4 = 19%).

To verify that this minor reduction in apparent contrast near the center region could not account for our rivalry results with the large competitor, we measured contrast psychometric functions for the same observers with the small rivalry suppressor, after dropping the physical contrast of this competitor down to 15% R.M.S. contrast. With this small, lower-contrast competitor, we still found a substantial reduction in the response gain for psychometric functions (Fig 4; Fig S3), thereby ruling out center-surround suppression as a possible explanation for our results.

Fig 4.

(a) Psychometric functions for probes pitted against small competitors, after contrast has been adjusted for the reduction in perceived contrast with a surround. Although the physical contrast of the competitor was lower, we still found a contrast gain shift (b), as well as a drop in the response gain of the psychometric functions (c) –thus ruling out an alternative, surround suppression-based explanation for our large-suppressor results. Error bars are bootstrapped 95% confidence intervals.

Could the large competitor’s inability to suppress high contrast probes result from impaired fusion between the eyes, arising from the size disparity (Ooi and He, 2006) between the large competitor and the smaller probe? To explicitly rule out this explanation, we conducted an additional control experiment in which the large competitor was once again pitted against the smaller probe, but with additional circular fusion markers presented to both eyes, surrounding the probe region (1.75°). If the effects we observed were due to differences in fusion between size conditions, the contrast psychometric functions should now resemble that of the smaller competitor: a response gain reduction. But that was not the case, because the additional fusion markers failed to alter the contrast gain-like pattern of suppression evoked by the large competitor (Fig 5; Fig S4).

Fig 5.

(a) Psychometric functions for probes pitted against large competitors, when the probe region was surrounded with an aperture in both eyes, facilitating binocular fusion. (b) Despite the addition of these fusion markers, the largest competitor still only evoked a contrast gain shift in the contrast psychometric function, and no change in the response gain (c). Error bars are bootstrapped 95% confidence intervals.

Normalization Model of Visual Competition

Taken together, the results presented above can be construed to mean that the regulation of visual competition, whether through attention or through rivalry, relies on normalization. A central idea behind the normalization model for attention is that the modulatory field augments the strength of a stimulus prior to divisive normalization (Reynolds and Heeger, 2009); were that not the case, then the modulatory signature would always be one of contrast gain. Moreover, standard models of normalization (Ding and Sperling, 2009; Moradi and Heeger, 2009), when applied to binocular representation, only predict a pure contrast gain shift. Our results, however, clearly show that the response function can be modulated under rivalry by both contrast gain and response gain, implying that there must be an additional modulatory field at play during visual competition –a mechanism with the same signature of effects as those associated with the normalization model of attention.

What is the source of this additional modulatory field component in rivalry? One plausible candidate is, in fact, attention as embodied in a recently proposed normalization model (Reynolds and Heeger, 2009): when a stimulus is suppressed from awareness during rivalry, attention may be directed toward the competing, dominant stimulus, rather than the suppressed probe. This dominant stimulus may thus act much like a modulatory attentional field, withdrawing attentional resources from the suppressed probe across a spatial extent that spans the size of the dominant stimulus. The impact of this withdrawal of attention would depend on the size of the modulatory field. A small modulatory field would solely decrease the response in the center region of a suppressed probe stimulus, tipping the balance between excitation and inhibition (Sundberg et al, 2009) in favor of the inhibitory component and thus causing both a reduction in both contrast gain and response gain. A large modulatory field, however, would decrease the response to the probe across a much larger spatial extent, thus maintaining the balance between excitation and inhibition and causing only a contrast gain shift.

This relationship between attention and awareness, and their combined impact on a probe stimulus, can be formalized in the normalization framework (Fig 6). The normalization model proposes that the response to a stimulus is comprised of an excitatory component that is divided by an inhibitory component (Heeger, 1992). The neural response to a probe stimulus, R_p, can thus be expressed as,

Fig 6.

Normalization model of visual competition. (a) The neural response to a probe stimulus is modulated by attention, and the spatial extent of this attention modulation hinges on the size of the competitor stimulus. When the competitor is small, attention is withdrawn from a small, center region of the probe. When the competitor is larger, attention is withdrawn from a larger swath of space, which includes both the probe and its surroundings. This attention-modulated probe response then undergoes divisive normalization, yielding a population response with signature gain responses that depend on the size of the probe, the size of the competitor, and the amount of attentional modulation. (b) Model predictions for attended stimuli. When the competitor stimuli are attended, a large competitor withdraws attention from the probe equally across both the excitatory and inhibitory components: yielding contrast gain. A small competitor, however, only withdraws attention from the center, excitatory region of the probe, leaving its surround unmolested: yielding both a response gain and contrast gain change, whereby the largest suppressive effects occur at high contrasts. (c) Model predictions for unattended stimuli. When attention is diverted from the competing stimuli, the balance between excitation and inhibition is maintained, regardless of competitor size, and thus the model only predicts a shift in the contrast gain, whereby the only suppressive effects occur at mid-contrasts.

$$R_p=\alpha \frac{{\gamma }_P{C}_P}{{\gamma }_P{C}_P+\omega {\beta }_P{C}_P+{\gamma }_S{\beta }_S{C}_S+\sigma },$$ (1)

where C_P is the contrast of the probe stimulus in one eye (between 0–1), C_S represents the contrast of the competing stimulus in the other eye, σ determines the contrast gain (contrast at which neural response reaches half its maximum), α is the maximum attainable response, γ_P and γ_S represent the peak attentional gain for the suppressed probe stimulus (γ_P) and the competitor (γ_S), and ω determines the relative impact of the modulatory field on the surround region of the probe. Note that an additional exponent parameter, ⁿ, would need to be added to account for nonlinearities in signal transduction (i.e., Cⁿ). However, for simplicity we have left that out of the models; the model predictions would be qualitatively similar with or without this nonlinearity.

To model stimuli of varying sizes, the stimulus in each eye is broken down into two components: the center and the surround. First, consider the probe stimulus. We represent the strength of the center region as C_P, and the surround as β_PC_P, where β_P is a scaling factor on the suppression driven by the surround region. A large probe would have a large surround (β_P ≈1), which would increase the suppressive drive, attenuating its maximum attainable contrast response (lowered asymptote).

Next, consider the competing stimulus in the other eye. This stimulus is treated identically to the probe, with C_S corresponding to the center region of the competitor, and β_SC_S corresponding to the surround, where β_S is a scaling factor on the suppression driven by the surround region of the competitor. Much like the probe, a large competitor stimulus will have a large surround (β_S ≈1 ), which will also increase the suppressive drive, lowering the asymptotic response. We can model the condition where the probe was viewed in the absence of any competitor in the other eye by simply setting the contrast of the other eye’s stimulus to zero (C_S = 1).

We modeled the response to the probe stimulus assuming that attention plays a critical role in visual awareness. Specifically, we assume that the dominant stimulus during rivalry receives more attentional resources than the suppressed stimulus: there is high attentional gain directed towards the features of the probe when it is dominant, which we denote with γ_P > 1, but when the competitor in the other eye is dominant, that dominant competitor receives the lion’s share of attentional resources instead, leaving only a small portion of attentional resources directed towards the representation of the suppressed probe stimulus, which we denote with γ_P > γ_S. While this modulatory field could be the result of feature-based attention and spatial attention, note that recent evidence suggests that attention may also be directed towards eye-specific information (Zhang et al, 2012).

The degree to which the withdrawal of attention affects the probe’s surround relies on ω. When the modulatory field is small, attentional resources are assumed only to be withdrawn from the center portion of the probe, leaving the surround component less affected; we denote a scenario with a small modulatory field as ω = γ_S. However, when the dominant competitor is large, we assume that attention was withdrawn from the suppressed probe across a large spatial extent, which could decrease response in both the excitatory center and inhibitory surround components of the probe equally; we denote a scenario with a large modulatory field as ω = γ_P.

Taken together, this model fully accounts for our observed results (Fig 6b). In our simulations, we assume that γ_S is substantially larger than γ_P, as would be the case if attention were withdrawn from the suppressed stimulus. Consider a scenario where the modulatory field size is large, as would be the case when the rival stimulus is large and has withdrawn attention from the probe across a large spatial extent. Here, the modulatory effects of attention would encompass both the center and surround regions equally (ω = γ_P). Because the balance between these excitatory and inhibitory processes is thus maintained, this pattern results solely in a shift in the contrast gain of the contrast response function, as depicted in Fig 6b (green dotted curve). This matches our observed behavioral results, where the large competitor only caused a contrast gain shift for all our observers.

Now consider a similar scenario, only now the modulatory field size is smaller. In this case, the effects of attention are solely from the center region of the probe, with little impact on the surround region (ω = γ_S). Because this tips the balance between excitatory and inhibitory processes, this scenario results in both a shift in the contrast gain, as well as a decrease in the response gain of the contrast response function, as depicted in Fig 6b (red dashed curve). This matches our behavioral results, where a competitor of the same size as the stimulus caused both contrast gain and response gain changes. Note that, in our model, we assume that the size of modulatory field scales proportionally to the size of the dominant stimulus, but that it is not necessarily the exact same size as that stimulus. Specifically, we assume that the modulatory field is smaller than the dominant stimulus, and thus the surround region of the probe is less affected by the withdrawal of attention. This could come about simply because the attentional field size is Gaussian-like in shape, and therefore has a stronger effect in the center region than it does on the outer region. Indeed, spatial attention can be directed to a specific region of an object (Vecera et al, 2000), even when there is no visual boundary present to ‘halt’ the spread of attention across the object (Hollingworth et al, 2012), as was the case in our experiment. Moreover, attention is known to be capable of selecting ‘annular’ stimuli (Somers et al, 1998).

Experiment 2: How attention impacts visual competition

The model advanced here proposes that attention plays a key role in visual competition: a dominant, small competitor withdraws attention from the center region of the probe stimulus and, as the consequence of normalization, causes a reduction in that probe’s response gain. Interestingly, this component of the model makes an explicit prediction: diverting attention away from both competing stimuli would leave the balance between excitation and inhibition unaltered, thereby abolishing the response gain-like effects of the smaller competitor, which would be signified by a lack of suppression with high contrast stimuli. To test this prediction, we conducted an additional experiment where observers directed their attention either toward or away from a pair of competing rivalry stimuli. During both conditions, we measured the strength of suppression produced by either a large or small competitor. The model predicts that when attention is withdrawn from the competing stimuli (γ_P = 1 and γ_S = 1; Fig 6c), the competitor will only elicit contrast gain modulation, regardless of the competitor’s size.

To test attention’s role in mediating visual competition, we created conditions where negative afterimages were induced under conditions of binocular rivalry. An afterimage is the illusory ‘photo negative’ experienced immediately following exposure to a real stimulus. Afterimages used to be attributed exclusively to retinal adaptation, but a growing body of work suggests that adaptation within cortical visual areas also contributes to afterimage formation (Brascamp et al, 2009; Ito, 2012; Shimojo et al, 2001; Tsuchiya and Koch, 2005). Of relevance for our purposes, a stimulus suppressed under rivalry causes weaker subsequent afterimages –a phenomenon believed to arise from attenuated responses within phase-sensitive neural representations (Brascamp et al, 2009). We reasoned that if attention plays a critical role in modulating the shape of the contrast response under suppression, two key effects should emerge in the induction of afterimages under rivalry, depending on whether attention is directed toward or away from the rival stimuli. Directing attention toward a small, high-contrast competitor viewed by one eye should elicit a response gain shift when the other eye’s competitor is small, but a contrast gain shift when that other eye’s competitor is large. For a high-contrast, suppressed stimulus, this should induce a weaker afterimage when the competitor stimulus viewed by the dominant eye is small compared to when that competitor is large. The model predicts that without attention the balance between excitation and inhibition will be preserved regardless of competitor size. Thus, diverting attention away from high-contrast, competing stimuli should transform the response gain modulation associated with small stimuli into a contrast gain modulation. For a high-contrast stimulus, a contrast gain shift would be signified by an attenuation of the suppressive effects that rivalry has on afterimages, for all competitor sizes.

To test afterimage strength, we implemented a psychophysical paradigm that quantitatively indexes the strength of negative afterimages (Brascamp et al, 2009; Kelly and Martinez-Uriegas, 1993; Georgeson and Turner, 1985; Leguire and Blake, 1982). To induce afterimages, observers were given brief, 2-sec exposures to a sinusoidal grating (the inducer) presented to one eye while, at the same time, the other eye received one of three possible stimulus arrangements (Fig. 7): 1) an uncontoured field that produced no suppression of the inducer, 2) a large (8°) competitor or, 3) small (1.5°) competitor, both of which suppress visibility of the inducer. Immediately following each brief induction period, the competitor grating, if present, was removed and the contrast of the inducer viewed by the other eye was ramped off and was replaced by a ‘nuller’ stimulus, itself a sinusoidal grating presented to the same eye that received the inducer. The nuller stimulus always had the same spatial phase as the inducer and, therefore, the opposite spatial phase of any negative afterimage caused by the inducer. After each period of nuller presentation (750 ms), observers indicated whether or not a grating was seen. Depending on the contrast of the nulling stimulus and the strength of the afterimage, observers might see the negative afterimage, they might see the nuller, or they might see no grating at all because the particular combination of afterimage strength and nuller contrast cancelled one another creating perception of a uniform field. By measuring the proportion of “grating seen” trials as a function of the nuller stimulus’ contrast, we obtained ‘afterimage functions’ that approximated inverted Gaussian curves. The nuller contrast at which afterimage functions reach their troughs (the mean of an inverted Gaussian function) corresponds to the physical contrast required to nullify the negative afterimage, thus providing a quantitative measure of the afterimage strength. To direct attention towards the competing stimuli, we had observers detect orientation changes that occurred stochastically while the competitor stimulus was dominant. To divert attention away from the competing stimuli, we required observers to perform a letter identification task (RSVP task), detecting target letters within a stream of distractor letters appearing in the periphery.

Fig 7.

Example trial sequence in Experiment 2. A phase-reversing noise stimulus in one eye (Competitor; 10 Hz) was pitted against a sinusoidal grating in the other eye (Inducer). The ‘Dominant eye’ received one of three possible stimulus arrangements: 1) an uncontoured field that produced no suppression of the inducer, 2) a large (8°) competitor or, 3) small (1.5°) competitor, both of which suppress visibility of the inducer. After viewing this display for 2 seconds, these stimuli disappeared, and a Nuller stimulus appeared in the same eye that the inducer had been in. This was followed by a mask stimulus, during which observers indicated whether or not a grating was seen. To direct attention towards the competing stimuli, observers detected orientation changes that occurred stochastically while the competitor stimulus was dominant. To divert attention away from the competing stimuli, observers performed a letter identification task (RSVP task), detecting target letters within a rapid stream of distractor letters appearing in the periphery.

Results: Experiment 2

Turning first to the condition in which attention was directed toward the visual competition, we observed the typical U-shaped afterimage function, regardless whether there was a competitor or not, and regardless of the competitor’s size. However, the troughs of these functions differed, implying that afterimage strength depended on stimulus size. We observed no difference in afterimage strength between the large competitor and no-competitor conditions (Fig 8a; Fig S5a). This is consistent with the contrast gain shift observed in the first experiment, whereby the modulatory effects of suppression are weak-to-nonexistent at high stimulus contrasts. However, we discovered significantly weakened afterimages when the small competitor was pitted against the inducer (Fig 8a; Fig S5a). This pattern of results is consistent with the response gain reduction we observed in the first experiment, whereby the modulatory effects of suppression are greatest at high stimulus contrasts. This is also consistent with previous reports showing that rivalry between similarly-sized small competitors can attenuate afterimage formation (Brascamp et al, 2009). To quantify the impact of suppression on afterimage strength, we fit the data for each observer with inverted modified Gaussian functions, where the estimated mean provides the index of afterimage strength. The afterimage strength indices reveal the same pattern of effects for all observers: while afterimage strength was unaltered by a large competitor, afterimage strength was diminished by a small competitor (Fig 8b; Fig S5a).

Fig 8.

The effect of attention and rivalry on afterimage formation. Top row (a,b) corresponds to the Attended condition, and the bottom row (c,d) corresponds to the Unattended condition. (a) When attention was directed toward the visual competition, the troughs of the afterimage functions (dotted arrows), which index afterimage strength, changed depending on stimulus size. There was no difference in afterimage strength between the large competitor and no-competitor conditions, yet when the inducer was pitted against the small competitor, the inducer afterimages were weakened. (b) Estimated afterimage strength indices reveal the same pattern of effects for all observers: while afterimage strength was unaltered by a large competitor, afterimage strength was diminished by a small competitor. (c) Without attention, the competitor had no effect on afterimage strength. (d) Afterimage strength estimates revealed a similar pattern of effects across all observers: for none did afterimage strength differ across conditions. Error bars are bootstrapped 95% confidence intervals.

We measured a distinctly different pattern of effects when attention was diverted away from the visual competition. While unattended stimuli still evoked negative afterimages, we found that without attention the competitor had no effect on afterimage strength, and this was true for both the large competitor and the small competitor (Fig 8c; Fig S5b). Fits with the afterimage functions revealed a similar pattern of effects across all observers: for none did afterimage strength differ across conditions (Fig 8d; Fig S5b). This is consistent with the model predictions: the response gain reduction brought about by the small competitor is the byproduct of attentional modulation of normalization, and without attention, the gain change consists of only a contrast gain shift –just like what we observed with the large suppressor. These results suggest that the type of modulation of awareness through rivalry hinges critically on attention. Without attention, the suppression of competing stimuli is substantially weakened at high contrasts.

Discussion

We propose a computation model, under the normalization framework, whereby attention plays a pivotal role in modulating competition for visual awareness. Previous studies have reported that, without attention, rivalry is weakened or altogether abolished in visual area V1 (Zhang et al, 2011; Watanabe et al. 2011) and in other, extrastriate cortical areas (Lee et al, 2007). The model proposed by us can accommodate the results from these studies, because in this model attentional modulation is a driving force behind the suppression of awareness typically observed under rivalry. The present results, however, do not compel us to conclude that rivalry suppression simply does not occur at all without attention. Rather, the model proposes that the interaction between attention and awareness is more nuanced, with the magnitude of suppression relying on a variety of factors that include stimulus size, attentional state, and contrast of the competing stimuli. It is possible, for instance, that previous failures to find evidence for suppression without attention were working in a high-contrast regime where suppression may not reveal itself when under the influence of contrast gain modulation.

While the effects of binocular rivalry suppression have been observed throughout the visual hierarchy (Tong et al, 2006), the results from our experiments hint at a very early cortical locus for the effects suppression, due to the small size (1.5°) of the probe stimulus used in our study. Under the normalization framework, reductions in the response gain of a stimulus would occur only if probe stimuli were large enough to encompass not only the excitatory field, but the inhibitory field as well. Otherwise, we would observe no difference between competitor sizes. Although normalization as a neural computation likely occurs throughout the brain (Carandini and Heeger, 2011), our relatively small probe suggests that the particular competitive interactions we observe in this study are occurring very early in the visual processing hierarchy, such as in V1 or the lateral geniculate nucleus (LGN), where receptive fields are quite small (Derrington and Fuchs, 1979). Although the reported effects of attention and rivalry have been variable when measure physiologically in V1 (e.g. Tong et al, 2006; Reynolds and Chelazzi, 2004), this could be due to a variety of factors including variability in the properties of the stimuli used, such as stimulus contrast and size, which under the normalization framework, predict variable levels of modulation. Ultimately, however, psychophysical methods can only go so far in pinpointing the neural locus of such effects, and further work in neuroimaging and electrophysiology may shed further light on where in the visual processing hierarchy attention modulates the neural events underlying visual competition.

In summary, our results support a normalization model for visual competition, in which attention plays a crucial role in regulating the neural contrast response. Attention has long been known to affect rivalry, with some studies reporting that attention modulates the temporal dynamics of binocular rivalry (Paffen and Alais, 2011, Mitchell et al, 2004), and others reporting that rivalry does not occur in the absence of attention in certain early visuocortical areas (Lee et al, 2007; Zhang et al, 2011; Watanabe et al. 2011). While these studies suggest that attention can modulate rivalry, our results and model reveal that these two processes are even more intricately intertwined: visual awareness during dominance phases of rivalry dictates what receives attention and what does not, which in turn interacts with normalization to determine the gain of the neural response.

Experimental Procedures

Four observers participated in the study. All had normal or corrected-to-normal vision. Stimuli were generated on a Macintosh running Matlab and the Psychophysics Toolbox (Brainard, 1997; Pelli, 1997). Observers viewed the display in a darkened room on a gamma-corrected CRT (21” Sony MultiScan; refresh rate: 100 Hz). Observers’ heads were stabilized with a chin and forehead rest, 96 cm from the display. The display was viewed through a mirror stereoscope that presented the left half of the display exclusively to the left eye and the right half of the display exclusively to the right eye.

Experiment 1

Throughout the experiment, each eye viewed a fixation point (0.14° × 0.14°), along with circular fusion frames (9° × 9°) to help stabilized binocular eye alignment (Fig 1). In each trial, stimuli were presented dichoptically, with both eyes viewing orthogonally oriented filtered noise patches. Both noise patches were bandpass spatial frequency filtered (bandpass frequencies: 1–6 cpd), as well as filtered in the orientation domain (10° bandwidth), with the center frequency of the noise band fixed at 45° from vertical in one eye (probe stimulus), and −45° from vertical in the other eye (competing stimulus). The diameter of the probe stimulus always subtended 1.5°, whereas the suppressor could be either the same size as the probe (small competitor), or subtended 8 (large competitor). The contrast of the competing stimulus was fixed at 23% R.M.S. contrast. The probe stimulus ranged from 0.8 – 23% R.M.S. contrast, allowing us to measure the entire psychometric function.

In half of the trials, contrast psychometric functions were assessed for the probe stimulus presented monocularly, which served as a baseline condition. In each of these trials, the stimulus briefly changed its orientation content either clockwise or counterclockwise (4°), and observers reported which direction that stimulus had rotated. In the other half of the trials, observers viewed stimuli dichoptically, with each eye viewing a different orientation bandpass-filtered noise display. The orientation content of the display in one eye was always orthogonal to that of the other eye –a stimulus mismatch that provokes visual competition. To manipulate the suppression of these stimuli, we used the flash suppression technique (Wolfe, 1984): on each trial, the to-be-suppressed probe stimulus was presented monocularly for 3,000 ms, after which time the competing stimulus (flash suppression competitor) abruptly appeared in the other eye, thereby suppressing perception of the initially presented stimulus in favor of the newly presented image. The timing and relatively small size of the stimulus were specifically chosen to maximize flash suppression duration, and to minimize instances of piecemeal rivalry within the probe duration. Each observer participated in a practice block of 50 trials and 30 experimental blocks of 50 trials each, for a total of 50 data points per condition.

Experiment 2

Throughout the experiment, each eye viewed a fixation point (0.14° × 0.14°), along with circular fusion frames (9° × 9°). To induce afterimages in each trial, observers were shown brief, 2-sec exposures of a sinusoidal grating (the inducer; 1.5° × 1.5°; 80% contrast; 1 cpd) in one eye while, at the same time, the other eye viewed one of three possible stimulus arrangements (Fig. 7): 1) an uncontoured field that produced no suppression of the inducer, 2) a large (8°) competitor or, 3) small (1.5°) competitor. The large and small competitors were identical to the competitors used in the Experiment 1, with the exception that the stimuli counter-phase flickered at 10 Hz, which suppressed the sinusoid during that 2-sec exposure duration (Tsuchiya and Koch, 2005). Immediately following each brief induction period, the competitor grating, if present, was removed and the contrast of the inducer viewed by the other eye was ramped off and was replaced by a ‘nuller’ stimulus (750 ms), itself a sinusoidal grating presented to the same eye that received the inducer. An auditory tone was played coincident with the nuller onset, which helped distinguish the switch from the inducer to the nuller. The nuller stimulus contrast ranged from 2–80% contrast, and always had the same spatial phase as the inducer. After each period of nuller presentation (750 ms), a spatial mask was presented, which was a bandpass spatial frequency filtered noise patch (3°; bandpass frequencies: 1–6 cpd; 23% R.M.S. contrast). Once this mask appeared, observers indicated whether or not a grating had been seen.

To direct attention towards the competing stimuli (Attended condition), we had observers detect orientation changes (10°) that occurred stochastically (0.3 probability of occurrence) to the dominant competitor stimulus (175 ms). To divert attention away from the competing stimuli (Unattended condition), we required observers to perform a letter identification task (RSVP task), detecting target letters (‘J’ or ‘K’; 1.5° × 1.5°; 0.3 probability of occurrence) within a stream of distractor letters (‘X’ ‘L’ ‘V’ ‘H’ ‘B’ ‘A’ ‘C’ ‘F’ ‘Z’ ‘Y’ ‘O’ ‘U’ ‘N’ ‘W’ ‘E’), appearing in the periphery of the inducer eye (3.35° eccentricity) every 175 ms. Prior to each block of trials, observers were told which task to perform throughout the block. In both the Attended and Unattended conditions, the RSVP stream ended, and the fixation point changed color 750 ms prior to the onset of the nuller, providing ample time for observers to prepare for the task in which they would report whether a grating was seen or not.