Visual crowding is correlated with awareness

Summary

Crowding by nearby features causes identification failures in the peripheral visual field [1, 2]. However, prominent visual features can sometimes fail to reach awareness [3, 4], causing scenes to be incorrectly interpreted. Here we examine whether awareness of the flanking features is necessary for crowding to occur. Flankers that were physically present were rendered perceptually absent with adaptation-induced blindness [5]. In a letter identification task, targets were presented unflanked or with up to four flanker letters. On each trial, observers reported both the number of letters they perceived and the identity of a target letter. This paradigm allowed trial-by-trial assessment of awareness and crowding, and ensured that both targets and flankers were attended. Target letter identification performance was correlated with the number of flanking letters that were perceived on a given trial, regardless of the number that were physically present. Our data demonstrate that crowding can be released when flanking elements at attended locations are suppressed from visual awareness.

Results

When an object or letter is presented in the periphery of vision surrounded by other features it can be difficult to identify. This is more than an acuity limitation, since the target can be easily identified when the flanking features are removed (Figure 1A) [1, 2]; and it specifically concerns identification, since detection is negligibly impaired [6]. The mechanism of crowding has been characterised as a consequence of a compulsory averaging of features within an integration region [7–10], which would serve to regularise peripheral input to provide an illusion of detail [11, 12]. Another account posits that crowding is a consequence of the limited spatial resolution of attention [13]. In either case, crowding makes it impossible to individuate and thus recognise an individual object in a cluttered display [14, 13, see 15, 16 for recent reviews].

Figure 1. Crowding and identification task methods.

(A) A depiction of crowding. Fixate the central spot. The identity of the central letter on the left is difficult to distinguish, but the same letter can be easily identified when presented unflanked (right). (B) Experimental methods for the identification experiment. In adapt blocks, the flanker locations are adapted by letters updated at 10.7 Hz in random sequence. On gradual test presentations, the crowding array was smoothly ramped on and off over time (Gaussian contrast window with σ = 200ms). On abrupt tests, the crowding array was presented with abrupt onsets and offsets (square wave contrast window with duration 307 ms). In this example, five letters are presented; in the experiment, one (target only) to five (target and four flankers) letters were presented. After each trial, observers reported how many letters they saw and the identity of the target letter. See also Figure S1 for additional methods.

Despite much recent work on crowding, the role of visual awareness remains unclear. If the observer does not perceive flankers that are physically present, is the target still crowded? The answer to this question provides insight into both the mechanism of and locus of crowding. Several previous studies are relevant to the issue. Degrading the flankers by reducing their contrast releases crowding [14, 6, 17, 18]. Similarly, masking the identity [19] of flankers or manipulating their global grouping [20, 21] can cause partial release from crowding. While it is therefore clear that the effectiveness of visible flankers can be altered, it remains unknown whether crowding depends on phenomenal awareness of the flanking elements. Since we are sometimes unaware of prominent components of a scene [3, 4], it is important to know whether crowding occurs in these situations.

Here we examine crowding using a concurrent report of flanker awareness. To remove flankers from awareness, we use the recently reported phenomenon of adaptation-induced blindness [AIB; 5]. Adaptation to stimuli that alternate at high temporal frequencies (8 – 10 Hz) reduces the visual system’s sensitivity to stimulus transients at the adapted location. Subsequently, a stimulus presented gradually within a smooth contrast envelope (thus minimising onset and offset transients) often fails to enter awareness at all, whereas stimuli presented abruptly and causing robust transients are perceived more often. The authors [5] argue that visual awareness is gated by the detection of temporal transients [see also 22]. AIB is spatially localised to the site of adaptation and does not require presentation of additional visual features during the trial to mask the flankers, which could themselves directly or indirectly modulate crowding. We therefore used AIB to suppress flanker letters in a paradigm in which letter awareness and letter identification were measured on each trial.

We assessed observers’ ability to identify a target letter flanked by up to four additional letters (Figure 1A). In some blocks, the locations of flanker letters were adapted with high temporal frequency letter streams to induce AIB masking (Figure 1B). Letters could be presented within a gradual (Gaussian σ = 200ms) or an abrupt (square-wave of 307 ms duration) temporal window, and targets and flankers always had matched temporal characteristics. The adapting stimulus was a stream of random letters presented at all four flanker locations updated at 10.7 Hz.

Since adaptation can reduce apparent contrast [23–25], and crowding is known to be less effective for flankers of lower contrast [14, 12], target letters were presented at a contrast that was approximately equated to the apparent contrast of the flankers after adaptation. Based on the results of a contrast matching experiment (see Figure S1), targets were presented at a Michelson contrast of 0.2. The flanker letters were presented at 0.8 contrast. In adapted blocks, targets and flankers therefore appear to have approximately the same contrast, even though the (adapted) flankers have physically higher contrast. In addition, we included a condition in which unadapted flankers were presented at 0.2 contrast. There were therefore six experimental conditions: no adaptation with 0.8 contrast flankers, no adaptation with 0.2 contrast flankers, and adaptation with 0.8 contrast flankers, each for both abrupt and gradual temporal windows. Targets always had 0.2 contrast and the stream of adapting letters were always 0.9 contrast.

To assess the frequency of disappearances across all experimental conditions, we recorded subjective reports of the number of letters perceived on each trial, from zero to five. This procedure also ensured that all spatial locations (both target and possible flanker locations) were attended, since observers were required to make a judgment about the entire stimulus area. In addition, since trials with different numbers of letters were interleaved within blocks of trials, observers had no knowledge of the true number of letters presented. Figure 2 plots the physical number of letters presented versus the mean number reported across the four observers. In the unadapted conditions (green and blue data), observers reported the veridical number of letters. After adaptation (red data), the number of letters was underreported, by 25% in the abrupt condition (slope = 0.753) and by 62% in the gradual condition (slope = 0.379). Adaptation therefore caused subsequently presented stimuli to be less likely to enter awareness, and this effect was strongest when these stimuli had weak transients [5].

Figure 2. Physical versus perceived number of letters in identification experiment.

The physical versus perceived number of letters in the display, for stimuli presented in gradual (circles) and abrupt (squares) temporal windows in adapted (red), unadapted with 0.8 contrast flankers (blue) and unadapted with 0.2 contrast flankers (green) conditions. Data points show the mean number reported across four observers, error bars show +/− 1 SEM. The points in each condition have been fitted with a linear regression (solid lines); the 95% confidence limits on slope estimates are shown by the dashed lines. After adaptation, participants perceived fewer letters in the display, particularly in the gradual condition.

Target letter identification performance for gradual and abrupt presentations is shown in Figure 3A and 3B respectively. These scores were calculated by taking the mean proportion correct across observers at each level of physical number of letters. Crowding occurred in all conditions and performance deteriorated approximately linearly as the number of flanking letters increased to four, consistent with some previous results [26, 17, 27, 12, 10, 28], though one study [6] found that the effect of increasing flanker number saturated at 2 flankers. However, the adapted condition (red points, Figure 3A–B) showed a weaker crowding effect (i.e. improved letter identification) than the unadapted condition with matched physical contrast (blue points, Figure 3A–B). Therefore, presenting flankers in adapted locations reduces their crowding effect.

Figure 3. Identification performance.

Number of letters versus target identification performance (proportion correct), for four observers. (A) Physical number of letters presented, from one (target alone) to five (target with four flankers) versus target identification performance for gradual condition for adapted (red) and unadapted with 0.8 contrast flankers (blue), and unadapted with 0.2 contrast flankers (green), averaged across the four observers. Solid lines show best-fitting logistic curves to these data (dashed lines indicate 95% confidence limits on curve fits). Error bars on data points indicate +/− 1 SEM across observers. (B) Same as (A) but for abrupt condition. (C) The average of all trials (pooled across observers) when 5 letters were presented gradually (pink shaded points in A) replotted according to the number of letters reported on each trial. Error bars depict +/−1 binomial standard deviation. Symbol sizes are scaled according to the number of responses making up that data point, where larger symbols indicate more reports (see inset legend). The faint red line is a logistic curve fit to the adapted data. Performance is well predicted by the number of letters perceived. (D) Same as (C) for abrupt condition. Statistical analyses of these data are presented in Figure S2.

However, the apparent contrast of adapted flankers is also reduced (see Figure S1). If the release from crowding caused by adaptation were mediated solely by a reduction of the apparent contrast of the flankers, similar levels of crowding should be observed with unadapted flankers whose physical contrast matched the apparent contrast of adapted flankers. This was not supported by our data: unadapted 0.2 contrast flankers (green points, Figure 3A–B) released crowding compared to unadapted 0.8 contrast flankers (blue points, Figure 3A–B), but not as effectively as adaptation of 0.8 contrast flankers (red points, Figure 3A–B). Thus, adaptation reduces flanker apparent contrast, and this accounts for some but not all of the measured release from crowding.

What can account for the crowding release we observe? To address this question we examined the data according to the number of letters that observers reported perceiving on each trial. These data are shown in Figure 3C–D for those trials where 5 letters (the target and four flankers) were presented (similar results were found when less than 5 letters were presented). Some reports of perceived number were more frequent than others. We depict this variation by adjusting the size of the marker according to the total number of responses each point reflects. It can be seen in Figure 3C–D that observers reported the veridical 5 letters on the vast majority of trials in the unadapted conditions. However, perceived number judgments after adaptation were more varied, particularly in the gradual condition. These effects can be anticipated from Figure 2.

There is a systematic effect of perceived number for the adapted conditions (particularly the gradual case): On trials where observers report only one or two letters in the display, identification performance was very good (>80%; Figure 3C), indicating that it was the unadapted target letter that was detected in these conditions. The greater the number of letters observers report on any trial, the more poorly they identify the target letter (logistic curve fit, Figure 3C). This trend was reflected even when target and flanker transients were abrupt (Figure 3D). Statistical analysis supported these observations. We compared the fit of a logistic regression with the perceived number of letters as predictor to a regression with physical number of letters as predictor. In the adapted gradual condition (but not in any other condition), the number of letters perceived better predicted identification performance than the number physically present (see Figure S2).

Discussion

Our data demonstrate a clear effect of visual awareness on crowding: the strength of crowding is correlated with the number of flanking letters perceived, rather than the number physically present. To induce disappearances, flanker locations were adapted to streams of alternating letters [5]. This has the effect of reducing the apparent contrast of subsequently presented letters (Figure S1) and frequently causing them to fail to reach awareness (Figure 2). A reduction of the apparent contrast of the flanking letters could account for some, but not all of the improvement in identification following adaptation (Figure 3A–B). Our paradigm allowed the relationship between perception and performance to be explored on a trial-by-trial basis, and shows that the perception of fewer flanking letters is correlated with improved performance, irrespective of how many letters were physically present (Figure 3C–D). Thus visual awareness is correlated with crowding, and this is not simply an effect of reduced apparent contrast.

The present finding augments previous reports concerning the role of flanker signal strength in crowding. Previous research [6, 12] demonstrates that the inhibitory effect of the flankers on target identification increases sharply with flanker contrast above detection threshold. This finding can be seen as an “all-or-none” effect where if the flanker is detected it causes crowding. We demonstrate that not only does this all-or-none rule apply to failures to detect low contrast flankers, but also to illusory disappearances of high contrast flankers.

While both crowding and “ordinary masking” [6] will impair detection performance in our experiment, our result cannot be explained by ordinary masking alone. First, adaptation produced greater release than the unadapted 0.2 contrast condition, indicating that the effect of number perceived is separable from contrast reduction. Second, if target detection were impaired by ordinary masking on some trials, we would expect more frequent “zero” responses, where observers fail to see the target letter, than observed in Figure 3C–D. Indeed, only 7 “zero” responses were made across some 6000 trials. This indicates that target disappearances, either caused by masking of the target by the flankers or by spatially nonspecific adaptation effects, do not greatly influence our findings.

An additional effect that can be observed in our data concerns the higher overall performance in the abrupt condition after adaptation. While performance in this condition shows a weak effect of perceived number (see Figure 3D), disappearances are not frequent enough for perceived number to predict performance better than physical number (Figure S2). Instead, this condition shows higher overall performance than the gradual condition after adaptation, even when five letters are perceived. One possible reason for this difference is that the adaptation at the flanker locations may exert a weak suppression across the entire stimulus area that affects both flankers and targets. Owing to asymmetries in temporal adaptation [29, 30], this suppression would attenuate the effective contrast of gradual more than abrupt targets. These asymmetries may also cause abrupt targets to appear dissimilar to abrupt flankers following adaptation, whereas gradual targets continue to resemble gradual flankers. Since flankers that appear dissimilar to targets can cause less crowding [31, 18], this could also account for the higher performance in this condition. In either case, the clear effect of perceived number for the adapted gradual condition remains.

Our result linking crowding to phenomenal awareness of the flankers complements a recent study that used masking to impair flanker identification [19]. In that study, flanker identity was masked using backward masking [32], metacontrast masking [33] or object substitution masking [34]. All masks impaired observers’ ability to identify the flankers (the orientation of a Landolt C) to the same degree. Crowding was released when flankers were masked using backward and metacontrast masking, but not object substitution masking. The site of the target-flanker interaction mediating crowding is therefore after the site of backward and metacontrast masking (putatively early visual cortex) but before object substitution masking (putatively later in the visual hierarchy). Our results suggest a similar interpretation, namely that the site of crowding is after the site of AIB.

The neural substrate of AIB is currently unexplored. However, in the original report of the phenomenon, annular gratings that were suppressed from awareness caused unadapted nearby gratings to appear tilted and lower in contrast [5]. This result implies that the representation of the suppressed gratings remains intact at the level of V1, which is the likely neural locus of these orientation [35] and contrast [36] interactions. With this finding in mind, our results could be consistent with previous reports [13, 19] that crowding occurs after V1. This could involve awareness-dependent feedback from extrastriate areas that modulate V1 [37]. An alternative (though not mutually exclusive) mechanism involves a gain reduction in precortical channels, resulting in a V1 representation strong enough to elicit orientation and contrast interactions but weak enough to be suppressed from awareness further along the feedforward processing hierarchy.

Our findings suggest that AIB [5] is comprised of two effects: a reduction in apparent contrast in the adapted location and an increased likelihood of complete disappearances beyond that predicted by a reduction in apparent contrast alone. The idea that such disappearances could be mediated by both a reduction in signal strength and an additional stochastic component (in which the reduced signal sometimes drops below detection threshold) is similar to a recent proposal regarding motion-induced blindness [38], which is also thought to depend on the magnitude of transient responses [39].

Crowding is the primary constraint on the functionality of peripheral vision. Our data do not distinguish between pooling and attention models of crowding, but only suggest that the inputs to crowding have first entered awareness. That crowding can be released when flanking structure fails to enter awareness could have implications for everyday vision: structure in the periphery may be uncrowded until it enters visual awareness. Alternatively, natural scenes may more readily enter awareness than artificial stimuli such as letters [40, 41], implying that for all practical purposes, natural vision is always crowded.

Experimental Procedures

Observers

The two authors and two observers naïve to the experimental hypotheses participated in the experiment. All were experienced psychophysical observers with normal or corrected-to-normal visual acuity. The procedures conformed to the tenets of the declaration of Helsinki and were approved by the Institutional Review Board.

Stimuli

Stimuli were generated in the Matlab (Mathworks) environment using the PsychToolbox libraries [42, 43] running on an Apple Macintosh computer. Stimuli were displayed on a LaCie Electron Blue 22 CRT monitor at a resolution of 1152 × 870 pixels and a refresh rate of 75 Hz. The monitor was calibrated using a Minolta LS110 photometer and gamma-corrected using software look-up tables, with a mean luminance of 52 and a maximum luminance of 104 cd/m². A bit-stealing algorithm was implemented to provide 10.8 bits of luminance resolution. Observers viewed stimuli binocularly from a distance of 57 cm, in a darkened room, and responded using a USB number pad.

Stimuli consisted of dark Sloan letters (C D H K N O R S V and Z) on a light background [44]. Letters subtending 1° square were windowed within a Gaussian contrast envelope (σ = 0.33°) to produce a smooth falloff in contrast to the mean grey background (see Figure 1A). To reduce luminance artifacts caused by more light than dark pixels in these stimuli, the mean luminance of the Gaussian-enveloped letters was subtracted from the raw letters before multiplying by the Gaussian window. These were presented on a background of mean grey.

Procedure

A target letter was presented 12° from fixation on the horizontal meridian, and between zero and four flanker letters could appear above, below, left or right of the target (see Figure 1). On trials with fewer than four flankers, flanker position was randomly determined. Target and flanker identities were randomly determined each trial, with no restriction for repeated letters. The center-to-center separation between target and flankers was 2°, well within the critical region for crowding at this eccentricity [1]. Four adapting letters were presented on both the left and right of fixation, but the location (left or right) of target and flanker letter array was randomized across trials in order to reduce the benefit of any predictive eye movements made to the stimulus.

On each trial, the observer first indicated how many letters they saw (0–5) by pressing the corresponding number, then the identity of the target letter by pressing one of the 10 number keys corresponding to the perceived Sloan letter. A small depiction of all 10 letters was presented at fixation during this time to facilitate responding without breaking fixation. Feedback was provided for the letter identity judgment (not for the number judgment) in the form of a fixation brightness change. To aid localization and reduce positional uncertainty, dark lines 2° in length were presented 2° further out than the possible flanker positions, forming a crosshair configuration around the target letter.

The adapting stimulus was a random permutation of the 10 Sloan letters that was cycled between letters every 7 video frames (producing temporal transients centered at ~10.7 Hz). At the start of each adapt block, letter streams were shown for 60 seconds with a minimum of 5 seconds of top-up adaptation after each response. In addition, to minimize recovery from adaptation, the adapting stimuli were presented while the observer made their responses.

When flankers had 0.8 Michelson contrast, each level of flanker number (0 to 4) was presented 10 times with both gradual and abrupt onsets, creating 100 trials per block. Three observers (TW, PB and N1) completed at least five blocks each with adaptation and without; one naïve observer (N2) completed three blocks of each. The unadapted 0.2 flanker contrast condition was run in separate blocks; observers completed at least 2 blocks of 25 trials at each level of flanker number for both gradual and abrupt. Including aborted runs, observers TW, PB, N1 and N2 completed a total of 1747, 1600, 1530 and 1350 trials respectively across the entire identification experiment.

Analysis

In analysing the counting data (Figure 2), the mean of each observer’s mean in each condition was fitted with a regression line using Matlab’s (Mathworks) robustfit function, and 95% confidence limits on the slope estimates were derived by multiplying the standard error of the parameter estimate by 1.96. Logistic model fits in Figure 3 were derived using Matlab’s glmfit function, and 95% confidence curves were determined using the glmval function.