Strength of early visual adaptation depends on visual awareness

Abstract

We measured visual-adaptation strength under variations in visual awareness by manipulating phenomenal invisibility of adapting stimuli using binocular rivalry and visual crowding. Results showed that the threshold-elevation aftereffect and the translational motion aftereffect were reduced substantially during binocular rivalry and crowding. Importantly, aftereffect reduction was correlated with the proportion of time that the adapting stimulus was removed from visual awareness. These findings indicate that the neural events that underlie both rivalry and crowding are inaugurated at an early stage of visual processing, because both the threshold-elevation aftereffect and translational motion aftereffect arise, at least in part, from adaptation at the earliest stages of cortical processing. Also, our findings make it necessary to reinterpret previous studies whose results were construed as psychophysical evidence against the direct role of neurons in the primary visual cortex in visual awareness.

Visual adaptation has been dubbed the psychologist’s microelectrode (1) because the resulting visual aftereffects presumably reveal response properties of neural mechanisms that are activated by adapting stimuli. Also, measuring visual adaptation under visual conditions that render the adapting stimulus invisible allows one to draw inferences about the neural concomitants of the conditions that produce stimulus invisibility. Specifically, a result showing a full-strength aftereffect that is generated by an invisible stimulus implies normal, unperturbed neural activation at the site of adaptation. This outcome implies that the neural correlates of the visual phenomenon that are used to render the adapting stimulus invisible lie beyond the neural mechanisms that are responsible for the aftereffect. This line of reasoning has been applied to the study of binocular rivalry and visual crowding, which are two extensively studied visual phenomena that are used to “erase” visual stimuli from awareness (2). The results have shown that full-strength pattern and motion aftereffects (MAEs) can be induced even when the high-contrast inducing stimuli were absent from awareness for a substantial portion of the adaptation period during binocular rivalry (3–7) and crowding (8, 9). Because adaptation producing these aftereffects includes neural events that presumably occur within cortical areas ranging from the primary visual cortex (V1) (10–12) to the middle-temporal visual area (12, 13), these psychophysical findings have reasonably been interpreted as evidence for the high-level origin of both rivalry (14, 15) and crowding (8, 14). Also, these same results were regarded by some workers as key psychophysical evidence against the direct involvement of V1 neurons in conscious visual awareness (16–19). Measurement of full-strength aftereffects under conditions of rivalry and crowding shows a clear dissociation between the abolished perceptual awareness of the adapting stimulus and unperturbed pattern and motion adaptation. Because area V1 is the first neural site of motion and pattern adaptation (10–12), these findings were interpreted as indicating that activity in V1 does not correlate with visual awareness.

However, these visual-adaptation studies and the accompanying conclusions are at odds with neurophysiological investigations that typically report some degree of correlation between the activity in early visual areas and fluctuations in visual awareness. Single-cell recordings in alert, behaving monkeys show that some, but certainly not all, neurons in V1 modulate their activity coincident with the reported perceptual state of the evoking stimuli (20), with the proportion of neurons that “track” perceptual fluctuations in rivalry increasing within higher visual areas (20, 21). Results from human subjects show robust awareness-dependent modulations in early visual cortex, with the initial supporting evidence being obtained from electroencephalogram recordings (22, 23). Brain-imaging results consistently show that neural events in V1 are attenuated in response to visual stimuli that are suppressed from awareness during rivalry (24–29). The magnitude of this attenuation is particularly strong within monocular regions of V1 (25). Also, recent reports of rivalry-evoked fluctuations in human LGN (28, 29) are difficult to reconcile with the apparent inability of binocular rivalry to attenuate the buildup of adaptation to a suppressed stimulus.

We sought to determine how fluctuations in visual awareness can reduce neural activity at early stages of visual processing and yet fail to attenuate visual adaptation that is thought to occur at those early stages. In this article, we report a resolution to this puzzling inconsistency by showing that rivalry suppression and visual crowding can interfere with the buildup of pattern aftereffects and MAEs. Our reexamination of susceptibility of adaptation to binocular rivalry and crowding was motivated by the dependence of these visual aftereffects on adapting contrast; both the aftereffects (30, 31) and the neural activity in V1 that is responsible for them (32, 33) exhibit a compressive nonlinearity, with aftereffect strength saturating at moderate to high contrast levels. This compressive nonlinearity led us to question whether the presentation of adapting stimuli at high contrast might have created adaptation conditions that conceal the effects of rivalry suppression and visual crowding. Suppose that at the site of neural adaptation suppression of vision from rivalry or from crowding involves a reduction, not an abolishment, of neural activity (34). In terms of aftereffect strength, the consequence of this reduction would remain latent when the actual adapting contrast is high (Fig. 1). Only when the adapting contrast is situated on the rising portion of the contrast/response function would suppression have a measurable effect on aftereffect strength. Studies showing no effect of suppression on aftereffect strength have used a single level of adapting contrast that, indexed to threshold, was relatively high (3–9). We questioned whether those results (and the accompanying conclusions) were unwittingly confounded by the use of adapting contrasts producing saturated aftereffects.

Fig. 1.

Effects of binocular rivalry and crowding could remain latent at high adapting contrasts. Dashed arrows indicate the effect that a modest reduction in effective contrast would have on aftereffect strength at a high and an intermediate adapting contrast.

To test this possibility, we first measured the dependence of aftereffect strength on adapting contrast (see Figs. 2B and 5B). From the resulting contrast/response curves, we then selected different contrasts and used them for adaptation during binocular rivalry (35) and visual crowding (36). We found that rivalry and crowding do weaken the effectiveness of adapting stimuli, producing the orientation-dependent threshold-elevation aftereffect (TEAE) and translational MAE. It is important to note that the weakening of TEAE and MAE was correlated with the extent to which the adapting stimulus was removed from visual awareness during crowding and during rivalry. This covariation of the strength of low-level visual aftereffects and the efficacy of both rivalry and crowding indicates that neural events underlying rivalry and crowding are inaugurated early in visual processing, which make it necessary to reinterpret previous adaptation studies (3–5, 8, 9).

Fig. 2.

Results from binocular-rivalry experiments. Error bars indicate SEM. (A) Rival stimuli used in TEAE and MAE experiments. (B) Static MAE duration as a function of the adapting contrast. The data are fitted with Naka–Rushton function (with 300 ms subtracted from each data value to compensate for motor reaction time). (C) Static MAE duration in different viewing conditions at two adapting contrasts. (D) TEAE in different viewing conditions at two adapting contrasts.

Fig. 5.

Results from crowding experiments. Error bars indicate SEM. (A) Stimuli that were used to measure TEAE in three different viewing conditions. In all conditions, the observer fixated a small, circular spot located 25° directly below the middle of the adapting grating. (B) Threshold elevation as a function of the adapting contrast. The data are fitted with the Naka–Rushton function. (C) Threshold elevation in different viewing conditions at two adapting contrasts. (D) The relationship between the TEAE index (TEAE strength when crowded/TEAE strength in isolation) and the crowding index (orientation threshold when crowded/orientation threshold in isolation). Numbers by the data points indicate the adapting contrast. Filled and open circles show the data for the crowded and crowded HC conditions, respectively. (E) Stimuli used for dynamic MAE measurements. (F) Dynamic MAE strength in different viewing conditions at two adapting contrasts.

Results

Binocular Rivalry.

In the first series of experiments, we measured the effect of visual suppression during binocular rivalry on two low-level visual aftereffects: the translational MAE and the TEAE. In these experiments, the adapting stimulus was presented to one eye while the other eye viewed a dynamic radial grating that by itself produced neither the MAE nor the orientation-specific TEAE (Fig. 2A). These stimulus conditions produced clear-cut perceptual alternations between the stimuli of the two eyes, with the adapting stimulus completely suppressed from visual awareness for a substantial portion of the adapting period. When the contrast of the adapting stimulus fell on the plateau of the contrast/response curve, suppression had no effect on the magnitude of either the MAE or the TEAE (Fig. 2 C and D). This finding replicates the earlier results putatively implying that suppression occurs after the site of adaptation (3–5). However, when the adapting contrast assumed a lower, nonasymptotic value, suppression significantly weakened both the MAE and the TEAE. This finding undermines the conclusions of refs. 3–5 and their interpretation about the site of rivalry suppression. While suppressed, an adapting stimulus cannot be the focus of attention, and thus, it could be argued that the reduction in aftereffect magnitude results from intermittent withdrawal of attention, not from suppression per se. However, this argument can be rejected because the TEAE is not selectively modulated by attention focused on the adaptation stimulus (37), and the MAE is modulated by attention even at very high, asymptotic contrast values (38) for which we find no effect of suppression.

Having established an effect of rivalry suppression on aftereffect strength, we next investigated whether trial-by-trial variability in predominance of the adapting stimulus was correlated with the resulting aftereffect strength. Failure to find trial-by-trial correlation would indicate that the neural events underlying the aftereffect reduction during rivalry are not related directly to the fluctuations in visual awareness and, by extension, neural activity supporting consciousness (cf. ref. 39). To generate variability in the predominance of the stimulus producing the MAE, we fixed the contrast of the adapting stimulus at an intermediate, nonasymptotic value and over blocks of trials varied the contrast of the rival radial grating over a 1.7 log-unit range. In this way, we could manipulate the percentage of total time that the radial grating was dominant during each 60-s adaptation period, and, hence, the percentage of time the adapting stimulus was suppressed. The magnitude of the resulting MAE was inversely related to the total duration of suppression of the adapting stimulus (Fig. 3A), indicating that rivalry suppression affects MAE. When we repeated this experiment by using an asymptotic, high-contrast adapting stimulus, MAE strength was unrelated to the total duration of suppression (for all, r < 0.31; P > 0.12), which varied from 0% to 80%. Again, the effect of suppression remained latent when using high-contrast stimulation.

Fig. 3.

The relationship between the adapting stimulus visibility and the resulting aftereffect. (A) Static MAE duration for two observers as a function of the percentage of time the adapting grating was suppressed. Straight lines are linear fits to each observer’s data. (B) Proportion of trials in which leftward MAE was experienced as a function of the percentage of time that the right-moving grating was suppressed. The data for each of five observers were sorted into four 25-trial bins, with the x-axis position indicating the mean of each bin. The straight line is the average linear fit to the data.

In the previous experiment, we manipulated the contrast of the rival radial grating to generate variability in the predominance of the adapting grating. Therefore, one could argue that the results shown in Fig. 3A arise from the trial-by-trial changes in the stimulus conditions and not directly from fluctuations in visual awareness (39). To test this possibility, we conducted an experiment in which observers viewed a left- and a right-moving grating, each of which was presented to a different eye. The adapting gratings were presented for 10 s, and their contrast was fixed at a 16× threshold. During this brief adaptation period, observers pressed buttons to indicate exclusive dominance of one direction of motion or the other (these dichoptic targets were small, and therefore, rivalry was complete and unambiguous). After the adaptation, observers binocularly viewed a stationary, vertical grating and indicated the direction of illusory drift of that grating (the MAE) by pressing one of two keys; observers pressed a third key if no aftereffect was experienced (which occurred on 27% of trials). For each trial, we calculated the percentage of time that the right-moving grating was suppressed (i.e., the percentage of time the left-moving grating was dominant exclusively). Results (Fig. 3B) unequivocally show that the longer the rightward moving grating was erased from awareness during rivalry, the less frequently observers experienced a leftward MAE (i.e., the aftereffect produced by the rightward moving stimulus). Thus, this result confirms that trial-by-trial variability in visual awareness is correlated with the strength of associated MAE. This tight linkage between duration of awareness and strength of adaptation was not found in a study of the effect of rivalry suppression on afterimage strength (39), perhaps for reasons concerning the site of adaptation underlying afterimage generation.

To gauge the effectiveness of rivalry suppression, we measured aftereffect strength after adaptation periods during which the adapting stimulus was physically turned on and off in a pattern that precisely matched the alternations of rivalry that were recorded during the rivalry trials. This nonrivalry, “mimic” condition weakened aftereffect strength substantially (Fig. 2 C and D, gray bars), which makes sense because the TEAE and the MAE both depend on adaptation duration. For our purposes, it is important to note that the effect of suppression is not equivalent to physical removal of the adapting stimulus but, rather, to a modest but appreciable attenuation of its effective contrast. To estimate the reduction in effective contrast produced by suppression, we modified the mimicking condition so that adaptation contrast fluctuated between two levels, one the actual contrast value presented during rivalry and the other a somewhat lower value. By systematically testing different, lower values coinciding with suppression phases, we determined that the effect of suppression is approximately matched by a 0.3-log reduction in actual contrast.

The modest depth of suppression revealed by our results probably grows in strength within higher stages of the visual hierarchy (35, 40), culminating in the abolishment of neural activity and, thus, visual awareness (6). This hypothesis is supported by an experiment in which MAE was measured with a dynamic, counterphase-flickering test pattern. Dynamic measures of MAE are believed to reflect higher levels of motion processing than MAE measured with a static test pattern (31, 41). In accordance with this belief, our results show a larger effect of rivalry suppression on the dynamic MAE, even at an asymptotic contrast level (Fig. 4). The effect of suppression on the dynamic MAE is approximately equivalent to intermittent, physical removal of the adapting stimulus (gray bars in Fig. 4).

Fig. 4.

Dynamic MAE strength in different viewing conditions at two adapting contrasts.

Visual Crowding.

In the second series of experiments, we assessed the effect of visual crowding (36) on adaptation producing the TEAE and MAE. In one experiment, observers adapted to a grating imaged at 25° eccentricity (Fig. 5A) and judged which of two test intervals contained a low-contrast grating with an orientation that matched that of the adapting grating. As expected, the resulting TEAE varied in magnitude with adapting contrast (Fig. 5B). Next, in different conditions, the adapting stimulus was presented either alone (and, thus, was continuously visible) or closely flanked by “crowding” patterns that impaired visibility of the adapting pattern. When the adapting pattern was high contrast, addition of the crowding patterns had no influence on the strength of the resulting TEAE, consistent with earlier findings (8). However, at lower contrasts, the crowded adapting stimulus produced a substantially weaker TEAE (Fig. 5C). This pattern of results parallels the outcome we found by using binocular rivalry, and it implies that visual crowding affects build-up of aftereffects earlier in the visual processing stream than had been believed. Also, we measured TEAE with an orthogonal test grating, and we found no effect of crowding over the entire range of adapting contrasts. In other words, the non-orientation-specific component of TEAE is unaffected by visual crowding.

To gauge the relation between crowding strength and effectiveness of adaptation, we performed two additional experiments. First, we estimated crowding strength by measuring orientation-discrimination thresholds for a test grating presented on its own and in the presence of either matched-contrast or high-contrast crowding patterns (e.g., Fig. 5A). Orientation thresholds in the crowded conditions were three to nine times higher than when the test grating was presented in isolation. Even when able to correctly identify the orientation of a crowded grating, observers reported not “seeing” its orientation but, instead, relying on global impressions of texture throughout the crowded display to deduce the orientation of the test pattern (42). The strength of crowding increased with contrast (filled circles in Fig. 5D) and was stronger when the crowders were highly visible, consistent with previous observations (43). Next, we measured TEAE for the same range of contrasts and crowding conditions. It is significant that the weakening influence of crowding on the build-up of the TEAE was correlated with the crowding strength indexed by performance on the orientation discrimination task (Fig. 5D).

We also measured the effect of visual crowding on the MAE. We restricted these measurements to the dynamic MAE, because spontaneous fading in the visual periphery (Troxler’s effect) disrupted visibility of the stationary test used to assess the static MAE. By using the adapting patterns shown in Fig. 5E, we found that crowding, like binocular rivalry, reduced dynamic MAE strength at both high and intermediate adapting contrasts (Fig. 5F). Refs. 9 and 44 found that the static MAE could be experienced after adaptation with crowding, but those studies measured only the direction of MAE and not the effect of crowding on its strength.

Discussion

We show that rivalry suppression and visual crowding interfere with the buildup of orientation-specific TEAE, static MAE, and dynamic MAE. These results force reinterpretation of previous findings (3–5, 8, 9), which reported no effect of rivalry suppression and crowding on the strength of early visual aftereffects evoked by high-contrast adapting stimuli. Our results imply that periods of phenomenal invisibility of an adapting stimulus are accompanied by a reduction in the effective strength of that stimulus, a reduction that can go undetected when applied to high-contrast adapting stimuli.

It is generally believed that the TEAE arises from neural events within V1 (10) and that at least a component of the MAE arises from V1 activation (11, 12). Thus, our results imply that at least some of the neural events underlying rivalry suppression and crowding transpire at early stages in visual processing, before or at the site(s) of adaptation producing the TEAE and the MAE. This finding reconciles the effect of rivalry suppression on visual adaptation with neuroimaging results pointing to the involvement of LGN and V1 in binocular rivalry (24–29). Similarly, our crowding results undermine the conclusion that visual crowding is inaugurated exclusively in higher visual areas after orientation processing (8, 14). Considering our work together with previous findings (20, 21, 35), we hypothesize that the absence of visual awareness occasioned by rivalry and by crowding results from a cascade of neural events inaugurated early in visual processing and culminating in the abolishment of neural activity that is ordinarily associated with visual awareness. This conclusion is supported by our results demonstrating the relatively strong effect of suppression on the dynamic MAE as well as by other work showing that suppression abolishes a high-level face-adaptation aftereffect (6). However, note that suppression and crowding do not necessarily arise from the same neural events; the two phenomena exhibit different stimulus properties (3). However, rivalry and crowding have in common the ability to block an ordinarily visible stimulus from visual awareness.

Last, it is important to note that our results show that full-blown neural adaptation happens only when observers are aware of the adapting stimulus, including forms of adaptation that are thought to transpire within V1. Thus, it is no longer justified to cite earlier psychophysical findings on adaptation during rivalry and during crowding as evidence against the direct involvement of V1 in conscious visual awareness (16–19). However, the dependence of early visual aftereffects on awareness does not indicate that we are aware of all activity within V1. V1 responds to a range of stimulus attributes that fall outside of visual awareness (45–47), with a forthright example being our inability to identify which of two eyes is monocularly stimulated (48) despite overwhelming selectivity for the eye of origin in V1. Nevertheless, our findings suggest that fluctuations in visual awareness are related to patterns of neural activity in V1, which is a conclusion that is in accord with neuroimaging results (24–27). Thus, our results undermine the strong assertion “that the firing of none of the neurons in V1 directly correlates with what we consciously see” (16). Instead, our results are consistent with interactive models in which V1 constitutes an essential component in a recurrent circuit whose activation represents the neural signature of visual awareness (49–51).

Methods

Visual displays were generated by using matlab and the psychophysics toolbox (52, 53), and they were presented on a Multisync (NEC Electronics, Santa Clara, CA) 21-inch monitor (frame rate, 85 Hz; resolution, 1,024 × 768 pixels) for binocular rivalry measurements and on a CPD-E540 (Sony, Tokyo) 21-inch monitor (frame rate, 85 Hz; resolution, 1,600 × 1,200 pixels) for crowding measurements. A color look-up table linearized the output luminance of the monitor and made 1,792 luminance gradations available by exploiting a bit-stealing technique (54). Five experienced observers (always including at least two who were unaware of the purpose of the experiment) participated in each experiment, except in the experiment shown in Fig. 3A (two observers were tested) and the experiment shown in Fig. 5 (four observers were tested). Experiments complied with institutionally reviewed procedures for human subjects.

Before each experiment, we measured each observer’s contrast threshold for detecting what would constitute the adapting stimulus. In TEAE experiments, two interleaved quest staircases were used to estimate the contrast that was necessary to identify (82% correct) in which of two successive test intervals a brief (180 ms, delimited with brief beeps) grating pattern was presented, which was a two-interval, forced-choice task. For MAE experiments, we measured contrast thresholds for the direction discrimination of a briefly presented (100 ms) drifting grating. These measurements were used in ensuing experiments to scale adapting contrasts relative to the each observer’s contrast threshold. In crowding experiments, the contrast of high-contrast crowders in “Crwd. HC” condition (Fig. 5 A and E) was always 64 times the contrast threshold.

Binocular-Rivalry Experiments.

Observers viewed a pair of dichoptic displays through a mirror stereoscope (viewing distance, 83 cm). For both the TEAE and MAE experiments, the square rival stimuli (Fig. 2A) were 1° on a side and surrounded by a high-contrast checkerboard “frame” that promoted stable binocular alignment. For TEAE measurements, the adapting pattern (two cycles per degree) flickered sinusoidally in counterphase at 1 Hz to preclude afterimages. In the main MAE experiments, the horizontal adapting pattern (two cycles per degree) drifted upward at 2 Hz. In those experiments, observers initially viewed the adaptation stimulus for 60 s, followed by a test period, the details of which depended on the aftereffect under study. For the MAE experiment shown in Fig. 3B, observers adapted for 10 s to dichoptically viewed, vertical gratings drifting in opposite directions (2 Hz; two cycles per degree) and then viewed a stationary, vertical test grating. Each observer was tested on 100 such trials, with a minimum of 10 s between successive trials. For TEAE and dynamic-MAE measurements, stable adaptation was maintained by inserting between each brief test period a 5-s “top-up” adaptation period.

To measure TEAE during the test period, we used the same procedure that was used to measure the contrast threshold (see above). For the static MAE measurements, a stationary grating was presented immediately after a 60-s presentation of the drifting adapting grating (i.e., the adapting grating stopped moving). Observers were instructed to tap a key when illusory motion of the stationary grating ceased altogether; the duration of this static MAE provided an estimate of its strength (31).

For the dynamic MAE, each test trial consisted of the brief presentation (1,000 ms) of a compound grating composed of two superimposed, horizontal gratings drifting in opposite directions (cf. ref. 55). After each test presentation, observers indicated by button press in which one of the two directions the grating appeared to drift. Over successive trials the contrast values of the two opposite direction components of the compound grating were adjusted under control of two interleaved “one-up/one-down” staircases to estimate the nullification value at which the grating showed no net directional bias. Each staircase converged after six reversals, with the average of the last four reversals taken as the nulling value. The resulting contrast ratio was taken as a measure of MAE strength.

Visual-Crowding Experiments.

Observers binocularly viewed the display from a distance of 35 cm, with the head stabilized by a chin and headrest. For TEAE measurements, the adapting patterns (Fig. 5A; radius, 2.67°; center-to-center distance, 6.67°; one cycle per degree) flickered in counterphase at 1 Hz. Each adaptation period lasted 5 s and was followed by a test period during which a test grating (radius, 2.4°) was presented in one of two intervals (as described above). The initial 10 adaptation periods were taken as practice, allowing for the build-up of adaptation. The orientation of the test grating was either the same as the adapting grating or, in different conditions, orthogonal to it. In pilot work, we also measured TEAE using a linear array of five nonflickering grating patterns, identical to the display configuration used in ref. 8. The results were similar to those in Fig. 5C, but we opted for the present layout because counterphase flicker precluded unwanted afterimages and Troxler’s fading.

Adapting patterns for the dynamic MAE measurements (Fig. 5E; radius, 2.67°; center-to-center distance, 6.67°; one cycle per degree) drifted at 4 Hz. MAE strength was measured with a pair of superimposed gratings (radius, 2.4°) drifting in opposite directions, with the contrast of the two adjusted to nullify any sense of net drift in one direction (see Binocular-Rivalry Experiments for details).

Abbreviations

V1

primary visual cortex

MAE

motion aftereffect

TEAE

threshold-elevation aftereffect.