The allocation of attention and working memory in visual crowding

Abstract

When the distance between a visual target and nearby flankers falls below a critical distance, target discrimination declines precipitously. This is called crowding. Many researchers have proposed that selective attention plays a role in crowding. However, although some research has examined the effects of directing attention toward versus away from the targets, no previous research has assessed how attentional allocation varies as a function of target-flanker distance in crowding. Here, we used event-related potentials to assess the operation of attention during crowding, focusing on the attention-related N2pc component. We used a typical crowding task in which participants were asked to report the category (vowel/consonant) of a lateralized target letter flanked by distractor letters at different distances. We tested the hypothesis that attention fails when the target-flanker distance becomes too small for attention to operate effectively. Consistent with this hypothesis, we found that N2pc amplitude was maximal at intermediate target-flanker distances and decreased substantially when crowding became severe. In addition, we examined the sustained posterior contralateral negativity (SPCN), which reflects the amount of information being maintained in working memory. Unlike the N2pc component, the SPCN increased in amplitude at small target-flanker distances, suggesting that observers stored information about the target and flankers in working memory when attention failed to select the target. Together, the N2pc and SPCN results suggest that attention and working memory play distinctive roles in crowding: attention operates to minimize interference from the flankers at intermediate target-flanker distances, whereas working memory may be recruited when attention fails to select the target at small target-flanker distances.

Introduction

The perception of a peripheral visual target is dramatically impaired if the target is surrounded by nearby flankers. This phenomenon is called crowding (Pelli & Tillman, 2008), and it presents a major limit on real-world perception. The mechanisms underlying crowding are hotly debated, and recent reviews have highlighted the potential role of failures of spatial attention (Levi, 2008; Whitney & Levi, 2011). At intermediate target-distractor distances at the edge of the crowding range, attention may minimize crowding by effectively shrinking neural receptive fields around the target, thereby filtering the flankers (Moran & Desimone, 1985). Consistent with this proposal, Yeshurun and Rashal (2010) showed that precuing the target location decreased the target-flanker distance at which crowding occurred. This shows the effect of attention on crowding, but it does not provide any information about how the allocation of attention varies as a function of target-flanker distance (Yeshurun & Rashal, 2010).

The present study addressed this by recording the N2pc event-related potential (ERP) component, which reflects the focusing of attention onto a lateralized target (Hickey, Di Lollo, & McDonald, 2009; Luck & Hillyard, 1994). N2pc is generated at multiple stages along the ventral processing pathway depending on the scale of target-distractor competition (Hopf et al., 2006), and it is widely used to assess the allocation of attention (Eimer & Kiss, 2008; Hilimire, Mounts, Parks, & Corballis, 2009, 2010; Luck, 2012; Luck, Girelli, McDermott, & Ford, 1997; Xu & Franconeri, 2012).

We used a typical crowding task in which participants were asked to report the category (vowel/consonant) of a lateralized target letter flanked by distractor letters at different distances. Distant distractors were also present in the opposite visual field to create a well-controlled paradigm for measuring the N2pc component. We tested the specific hypothesis that attention can effectively suppress the flankers as long as they are not too close to the target, leading to excellent target discrimination performance. However, when the target-distractor distance becomes too small, the resolution of attention may be insufficient to select the target from the flankers (He, Cavanagh, & Intriligator, 1996; Intriligator & Cavanagh, 2001), leading to impaired behavioral performance (crowding). If spatial attention is unable to operate effectively at small target-distractor distances, then we should observe both reduced N2pc amplitude and impaired behavioral accuracy at these distances.

In addition, if spatial attention is unable to select the target from the distractors at small target-flanker distances, then it may be necessary to store information from all of the items in working memory so that the observer can spend the time needed to make an optimal target decision on the basis of the jumble of target and distractor information. This leads to the prediction that more information will be represented in working memory at small target-flanker distances in the period between stimulus offset and the target decision. This should lead to greater amplitude of the sustained posterior contralateral negativity (SPCN), which has been shown to increase as more items are stored in working memory (Jolicoeur, Brisson, & Robitaille, 2008; Robitaille et al., 2010; Vogel & Machizawa, 2004). The SPCN has also been reported in visuo-spatial attention tasks that do not explicitly require storage of information in working memory but might still benefit from the maintenance of information after the stimulus has disappeared (Brisson & Jolicoeur, 2007a, 2007b; Corriveau et al., 2012). Thus, an increase in SPCN amplitude under conditions of crowding would suggest that more items are being maintained in working memory when the distractors cannot be effectively filtered.

Experiment 1

Methods

Participants

Fourteen observers (8 female; mean age = 20.6 years, age range = 18 – 29) with normal or corrected-to-normal vision and no history of neurological disorders participated. All volunteers gave informed consent and were paid for their participation. The study was approved by the University of California-Davis Institutional Review Board. Two subjects were excluded from the analysis, one due to poor performance (correct response rate at chance level in all conditions), reflecting noncompliance with the task instructions, and the other one because of electroencephalographic artifacts on >25% of trials (we always exclude subjects for whom >25% of trials are rejected).

Stimuli and procedure

The stimuli were presented at a distance of 70 cm on a cathode ray tube monitor (60 Hz refresh rate) with a black background and a continuously visible white fixation cross. Each stimulus consisted of two vertical arrays of three letters (uppercase Geneva font, height = 0.75°), with one array on each side of the fixation cross (see Fig. 1). One array was red and the other was green, varying randomly across trials. The letters on each trial were selected at random without replacement from a set of five vowels (A, E, I, O, U) and five consonants (N, F, L, G, J).

Figure 1.

Example stimulus display. Subjects fixated on the central cross while attending to either the red or green items. The side containing the attended color varied unpredictably across trials. The task was to report whether the letter located in the middle of the attended-color array (the target) was a vowel or a consonant. Experiment 1 included three target-flanker distances: 16.8°, 3.13°, and 1.48°. Experiment 2 included no-distractor trials and five target-flanker distances: 16.8°, 4.12°, 3.13°, 2.14°, and 1.48°. Conditions included in both experiments are indicated with a * symbol.

Each array was centered 8.8° horizontally from the vertical meridian. The central letter in each array was on the horizontal meridian, with flanker letters directly above and below. The flankers were centered 1.48°, 3.13°, or 16.8° from the horizontal meridian. This provided one distance with substantial crowding (1.48°), one distance at the edge of the crowding range (3.13°), and one distance well beyond the crowding range that served as a no-crowding control condition (16.8°). Each distance was equiprobable. Stimulus duration was 200 ms, and the stimulus onset asynchrony varied randomly between 1600 and 1800 ms.

At the beginning of each trial block, the observer was instructed to attend either to red or to green and to indicate whether the middle letter in the array of that color (the target) was a vowel or a consonant. Observers responded by pressing a left or right button depending on whether the target was a vowel or a consonant. Speed and accuracy were equally stressed. Note that this is a conventional design for measuring crowding, except that observers must shift attention to the side containing the attended color to perform the task. The bilateral displays were necessary for measuring N2pc and SPCN without contamination from lateralized sensory activity.

Each participant performed 12 blocks of 200 trials. There were 6 attend-red and 6 attend-green blocks, presented in alternating order. Half of the trials were from an additional condition in which the possible flanker locations were arranged in a circular manner rather than in the linear manner shown in Figure 1. However, there was a programming error for this condition, so the data are not reported for these trials.

Recording and analysis

The electroencephalogram (EEG) was recorded using active Ag/AgCl electrodes (BioSemi ActiveTwo) from the left and right mastoids and 32 scalp sites (Fp1, Fp2, F7, F3, Fz, F4, F8, T7, C3, Cz, C4, T8, P9, P7, P5, P3, P1, Pz, P2, P4, P6, P8, P10, PO7, PO3, POz, PO4, PO8, O1, Oz, O2, and Iz), according to the modified International 10–20 System (American Electroencephalographic Society, 1994; Picton et al., 2000). To detect eye movements and blinks, the electrooculogram (EOG) was recorded from electrodes placed adjacent to the outer canthi of each eye and above and below the right eye. All signals were recorded in single-ended mode and referenced offline. Electrode offsets were kept below 40 mV. The EEG and EOG were low-pass filtered with a fifth-order sinc filter (half-power cutoff at 208 Hz) and digitized at 1024 Hz. All analyses after data acquisition were conducted using EEGLAB Toolbox (Delorme & Makeig, 2004) and ERPLAB Toolbox (Lopez-Calderon & Luck, 2014), which are open-source Matlab packages for EEG/ERP analysis.

The EEG signals were referenced offline to the average of the left and right mastoids, and the four EOG signals were referenced offline into bipolar vertical and horizontal EOG derivations. These signals were resampled to 256 Hz (with an antialiasing filter) and filtered offline using a non-causal Butterworth high-pass filter (half-amplitude cutoff = 0.1 Hz, slope = 12 dB/octave). A low-pass filter (half-amplitude cutoff = 30 Hz, slope = 12 dB/octave) was used after averaging for plotting purposes. Averaged ERP waveforms were computed with an 800 ms epoch, beginning 200 ms before stimulus onset.

Trials were automatically excluded if the behavioral response was incorrect, if the peak-to-peak voltage exceeded 200 μV in any 200 ms window in any channel, or if a sudden voltage step was detected in the horizontal or vertical EOG (Luck, 2005). To assess residual eye movements, we computed averaged horizontal EOG waveforms on left- and right-target trials for each participant, as in previous research (Woodman & Luck, 2003). The average residual eye movements in these waveforms were less than 3.2 μV for all conditions in both experiments, corresponding to an eye rotation of less than ± 0.1°, with an estimated voltage propagation of less than 0.1 uV at the posterior scalp sites (Lins, Picton, Berg, & Scherg, 1993). Fewer than 25% of trials were rejected owing to artifacts in any of the included subjects (mean = 11.5%, range = 2.2% – 22.2%).

The N2pc and SPCN components were measured from contralateral-minus-ipsilateral difference waves (relative to the side of the to-be-identified target) within an a priori region of interest (ROI) consisting of four pairs of electrodes: P5/P6, P7/P8, PO3/PO4, and PO7/PO8. We measured the mean amplitude from these difference waves using a priori time windows of 200–300 ms for N2pc and 400–600 ms for SPCN, relative to the 200 ms pre-stimulus baseline period. The waveforms were collapsed across stimulus colors and locations before amplitudes were measured to eliminate sensory confounds related to these factors.

Results and Discussion

Behavior

As shown in Figure 2A and Table 1, accuracy was high when the target-flanker distance was large but dropped steeply for the most crowded condition. A one-way ANOVA showed a significant effect of distance (F _{(2, 22)} = 211.81, p < 0.001), and follow-up Tukey HSD tests showed significant differences between all pairs of distances (p < 0.05). Reaction times (RTs) exhibited a complementary pattern, increasing as the target-flanker distance decreased (Figure 2B and Table 1). A one-way ANOVA yielded significant effects of distance (F _{(2, 22)} = 9.73, p < 0.001), and Tukey HSD paired tests yielded significant pairwise differences between 1.48° and each of the other two distances (3.13° and 16.8°). These behavioral results are typical of crowding experiments, indicating that our stimuli and task produced normal crowding.

Figure 2.

Summary of the effect of target-flanker distance on the behavioral and electrophysiological measures in Experiment 1. A, correct response rate. B, reaction time. C, N2pc mean amplitude. D, SPCN mean amplitude. Error bars show the within-subjects standard error (Morey, 2008).

Table 1.

Mean accuracy, reaction time, N2pc amplitude, SPCN amplitude, and number of error trials for each condition in Experiment 1 (SEM in parentheses).

Condition	% Correct	Reaction time (ms)	N2pc (μV)	SPCN (μV)	Number of error trials
16.8°	89.6 (0.36)	654 (6.78)	−0.32 (0.07)	−0.58 (0.06)	30.4 (5)
3.13°	83.8 (0.55)	682.2 (5.84)	−0.58 (0.06)	−0.96 (0.07)	52 (8)
1.48°	60 (0.48)	721.7 (7.84)	−0.08 (0.06)	−1.47 (0.08)	129.3 (9.1)

Event-Related Potentials

N2pc

Figure 3 shows the ERP waveforms at contralateral and ipsilateral electrode sites, relative to the location of the target, at each target-flanker distance, and Figure 4 shows the contralateral-minus-ipsilateral difference waves. In addition, mean amplitude in the N2pc measurement window is shown in Figure 2C and Table 1. The N2pc can be seen most clearly in the difference waves, where it appears as a negative deflection peaking at approximately 225 ms. To avoid biasing the statistical analyses by choosing a measurement window on the basis of the observed waveforms, N2pc amplitude was quantified using an a priori time window of 200-300 ms, which is the canonical time period of this component. However, the N2pc was somewhat earlier than that in the present experiment. Thus, to demonstrate that the results of our analyses were not an artifact of the a priori time window of 200-300 ms, we conducted follow-up analyses using windows of 150-350 ms, 150-320 ms, 150-300 ms, and 200-350 ms. All of these analyses yielded the same pattern of statistical significance as the a priori window (which was used for all analyses reported below).

Figure 3.

Grand average contralateral and ipsilateral waveforms relative to the side containing the attended color for Experiment 1. Waveforms from the different target-distractor distances are shown in A (16.8°), B (3.13°), and C (1.48°). The N2pc (200 – 300 ms) and SPCN (400 – 600 ms) measurement windows are highlighted. The waveforms in this and all subsequent ERP figures were low-pass filtered for visual clarity (half-amplitude cutoff = 30 Hz, slope = 12 dB/octave).

Figure 4.

Grand average difference waveforms (contralateral minus ipsilateral relative to the side containing the attended color) for the three target-distractor distances in Experiment 1. The N2pc (200 – 300 ms) and SPCN (400 – 600 ms) measurement windows are highlighted.

As shown in Figures 2-4, N2pc amplitude was non-monotonically related to target-flanker distance, with the greatest (most negative) amplitude at a distance of 3.13°. Note that this was the same distance at which behavioral accuracy began to deviate from ceiling. A one-way ANOVA yielded a significant main effect of target-flanker distance (F _{(2, 22)} = 4.72, p = 0.019). The highly nonlinear pattern led to a significant quadratic trend (F _{(1, 11)} = 4.98, p < 0.05). This pattern was also confirmed by a Tukey HSD test, which showed that the N2pc was significantly larger (p < 0.05) for the 3.13° distance than for the 1.48° distance. These results indicate that maximal attention was allocated at the edge of the crowding range (3.13°), with significantly less attention for the most crowded displays (1.48°). Thus, severe crowding leads to both impaired attentional allocation and impaired behavioral performance. A more detailed discussion of the implications of the reduced N2pc amplitude will be provided in the General Discussion.

SPCN

The SPCN can be seen in Figures 3 and 4 as a negativity that begins at approximately 350 ms and continues through the end of the epoch. SPCN amplitude was quantified in an a priori time window of 400-600 ms, but the same pattern of statistical significance was obtained with a window of 350-600 ms. Mean SPCN amplitude is also shown in Figure 2D and Table 1.

As shown in Figures 2-4, SPCN amplitude increased monotonically as the target-flanker distance decreased. A one-way ANOVA yielded a significant main effect of target-flanker distance (F _{(2, 22)} = 11.56, p < 0.001), with a significant linear trend (F _{(1, 11)} = 35.73, p < 0.001). Post-hoc Tukey HSD tests showed significantly larger SPCN for 1.48° compared to both 3.13° and 16.8°. This result suggests that working memory, reflected by the SPCN, is increasingly recruited as crowding increases. As will be described in more detail in the General Discussion, it is possible that the SPCN reflects the sustained operation of attention rather than directly reflecting working memory storage; however, given the brief stimulus duration, this kind of sustained attention would require that the information being attended had been stored in working memory.

Analysis of Error Trials

The prior analyses were restricted to trials with correct behavioral responses. However, we also examined the error trials to determine, for example, whether errors reflected reduced allocation of attention. The contralateral-minus-ipsilateral difference waveforms for correct trials and error trials are overlaid in Figure 5. The waveforms were quite noisy on error trials, especially for the 3.13° and 16.8° distances, for which there were few errors (see Table 1). There was some evidence of a reduced N2pc on error trials, especially at the 3.13° distance, but there were no obvious differences in the SPCN between correct and error trials. To analyze the data statistically, we conducted 2-way ANOVAs on N2pc and SPCN amplitude with factors of target-distractor distance and response type (correct vs. error). Neither the main effect of response type nor the interaction was significant for either N2pc (F _{(1, 11)} = 1.75, p = 0.21; F _{(2, 22)} = 2.06, p = 0.15, respectively) or SPCN (F _(1,11) = 0.49, p = 49; F _{(2, 22)} = 0.05, p = 0.95, respectively).

Figure 5.

Grand average difference waveforms (contralateral minus ipsilateral relative to the side containing the attended color) for correct and error trials in Experiment 1. The N2pc (200 – 300 ms) and SPCN (400 – 600 ms) measurement windows are highlighted. Waveforms from the different target-distractor distances are shown in A (16.8°), B (3.13°), and C (1.48°).

We also conducted a comparison of correct and error trials just at the 1.48° distance, where the number of error trials was reasonably high. The N2pc was very small on correct trials at this distance, so not surprisingly there was no significant difference in N2pc amplitude between correct trials and error trials (t₁₁ = −0.67, p = 0.51). There was also no significant difference in SPCN amplitude between correct trials and error trials (t₁₁ = −0.53, p = 0.61).

Thus, although there was some evidence that the N2pc was reduced on error trials, there was no hint of a reduction in SPCN amplitude on error trials. The SPCN result suggests that errors were not a result of reduced effort but instead reflect an effect of crowding on the information available to the neurocognitive process reflected by the SPCN.

Experiment 2

The goal of Experiment 2 was to demonstrate the replicability of the results of Experiment 1 and to provide a finer manipulation of target-flanker distance. A control condition with no flankers was also included.

Methods

The methods were identical to those of Experiment 1 except as follows. Twenty volunteers (13 female; mean age = 22.6 years, age range = 19 – 32) with normal or corrected-to-normal vision and no history of neurological disorders participated in this experiment. Two subjects were excluded from the analysis because of artifacts on >25% of trials, and two were excluded because of aberrant behavioral performance (unusually low accuracy and fast reaction times, indicating a lack of compliance with the task instructions). Fewer than 25% of trials were rejected due artifacts in the remaining subjects (mean = 11.9%, range = 2.8% – 24.7%).

The stimuli and task were identical to those in Experiment 1 except that flankers were either absent (no-distractors) or centered 1.48°, 2.14°, 3.13°, 4.12°, 16.8° from the horizontal meridian. All stimuli were presented in vertical columns.

Results and Discussion

Behavior

As shown in Figure 6A and Table 2, accuracy was high when the target-flanker distance was >4° and then fell monotonically at smaller distances. A one-way ANOVA again showed a significant effect of distance (F _{(5, 75)} = 176.06, p < 0.001), and follow-up Tukey HSD paired tests showed significant differences between the two most crowded conditions (1.48° vs 2.14°), and also between 1.48° and 2.14° compared to each of the other distances (p < 0.05). RTs showed the complementary pattern, increasing as the target-flanker distance decreased (Figure 6B). A one-way ANOVA yielded a significant effect of distance (F _{(5, 75)} = 70.91, p < 0.001). Post-hoc Tukey HSD paired tests yielded significant differences between the two most crowded conditions (1.48° vs 2.14°), and also between 1.48° and 2.14° compared to each of the other distances (p < 0.05). These results replicate the behavioral pattern obtained in Experiment 1 and demonstrate that this paradigm yields typical crowding results.

Figure 6.

Summary of the effect of target-flanker distance on the behavioral and electrophysiological measures in Experiment 2. A, correct response rate. B, reaction time. C, N2pc mean amplitude. D, SPCN mean amplitude. Error bars show the within-subjects standard error (Morey, 2008).

Table 2.

Mean accuracy, reaction time, N2pc amplitude, SPCN amplitude and number of error trials for each condition in Experiment 2 (SEM in parentheses).

Condition	% Correct	Reaction time (ms)	N2pc (μV)	SPCN (μV)	Number of error trials
No Distractors	93.5 (0.19)	589.9 (3.14)	−0.37 (0.04)	−1.29 (0.06)	22.8 (2.7)
16.8°	93.2 (0.20)	596.6 (3.27)	−0.65 (0.03)	−1.20 (0.06)	23.8 (2.7)
4.12°	92 (0.21)	613.6 (3.35)	−0.73 (0.04)	−1.17 (0.05)	27.6 (2.9)
3.13°	89 (0.27)	628.2 (3.67)	−0.89 (0.05)	−1.55 (0.07)	35.3 (3.6)
2.14°	80.3 (0.48)	673.3 (3.46)	−0.35 (0.06)	−1.70 (0.05)	64.7 (6.1)
1.48°	62.7 (0.43)	731 (4.93)	−0.26 (0.05)	−1.91 (0.05)	122.8 (6.2)

Event-Related Potentials

N2pc

Figure 7 shows the ERP waveforms at contralateral and ipsilateral electrode sites, relative to the location of the target, at each target-flanker distance, and Figure 8 shows the contralateral-minus-ipsilateral difference waves. For comparison, the difference wave for each target-flanker distance is overlaid with the difference wave for the no-distractor trials. The N2pc appears in the difference waves as a negative deflection peaking between 225 and 275 ms. As in Experiment 1, N2pc amplitude was quantified using an a priori time window of 200-300 ms (see mean amplitudes in Figure 6C and Table 2), but the same pattern of significance was obtained with windows of 150-350 ms, 150-320 ms, 150-300 ms, and 200-350 ms.

Figure 7.

Grand average contralateral and ipsilateral waveforms relative to the side containing the attended color for Experiment 2. The N2pc (200 – 300 ms) and SPCN (400 – 600 ms) measurement windows are highlighted. Waveforms from the different target-distractor distances are shown in A (No Distractors), B (16.8°), C (4.12°), D (3.13°), E (2.14°), and F (1.48°).

Figure 8.

Grand average difference waveforms (contralateral minus ipsilateral relative to the side containing the attended color) for Experiment 2. The N2pc (200 – 300 ms) and SPCN (400 – 600 ms) measurement windows are highlighted. Waveforms from the different target-distractor distances are shown in A (16.8°), B (4.12°), C (3.13°), D (2.14°), and E (1.48°). For comparison, the waveforms from the no-flanker trials are overlaid with the waveforms for each target-flanker distance.

As shown in Figures 6-8 and Table 2, the greatest (most negative) N2pc amplitude was found at a target-flanker distance of 3.13°, the same distance as in Experiment 1 and the distance at which behavioral performance began to be impaired. A one-way ANOVA yielded a significant main effect of target-flanker distance (F _{(5, 75)} = 5.1, p < 0.001). The highly nonlinear pattern led to a significant quadratic trend (F _{(1, 15)} = 24.55, p < 0.001). This quadratic trend was also significant when we excluded the no-distractors condition from the analysis (F _{(1, 15)} = 9.22, p < 0.01). This pattern was also confirmed by Tukey HSD paired tests, which showed that the N2pc was significantly larger for 3.13° compared to both the most and least crowded conditions (1.48° and no distractors; p < 0.05). These results replicate the pattern observed in Experiment 1, in which N2pc amplitude was greatest just at the point where behavioral crowding began to occur (3.13°) and declined significantly when the target-distractor distance decreased. This experiment therefore confirms that both attention and behavioral performance are impaired when the target-flanker distance is sufficiently small.

SPCN

The SPCN can be seen in Figures 6 and 7 as a negativity that begins at approximately 350 ms and continues through the end of the epoch. As in Experiment 1, SPCN amplitude was quantified in an a priori time window of 400-600 ms, but the same pattern of statistical significance was obtained with a window of 350-600 ms. Mean SPCN amplitude is also shown in Figure 6D and Table 2.

As in Experiment 1, we found that SPCN amplitude increased monotonically as the target-flanker distance decreased, yielding a significant main effect of target-flanker distance (F _{(5, 75)} = 5.81, p < 0.001) with a significant linear trend (F _{(1, 15)} = 10.89, p < 0.01). This linear trend was also significant when we excluded the no-distractors condition from the analysis (F _{(1, 15)} = 12.54, p < 0.01). Post-hoc Tukey HSD paired tests showed significantly larger SPCN for the most crowded condition (1.48°) compared to the least crowded ones (4.12°, 16.8° and no-distractors). These results replicate the findings of Experiment 1 and suggest that working memory is recruited when crowding prevents attention from selecting the target and filtering out the distractors.

Analysis of Error Trials

The contralateral-minus-ipsilateral difference waveforms for correct trials and error trials in Experiment 2 are overlaid in Figure 9. The waveforms were noisier on error trials than on correct trials, especially for the larger target-distractor distances, for which there were few errors (Table 2). As in Experiment 1, N2pc amplitude was reduced on error trials, especially at distances of 3.13° and greater, but there was no consistent difference in the SPCN between correct and error trials. To analyze the data statistically, we conducted 2-way ANOVAs on N2pc amplitude and SPCN amplitude with factors of target-distractor distance and response type (correct vs. error). For the N2pc, the main effect of response type was significant (F _{(1, 15)} = 11.71, p = 0.004), reflecting the reduced N2pc on error trials; the interaction between response type and target-distractor distance was also significant (F _{(5, 75)} = 2.92, p = 0.018), reflecting a larger difference between correct and error trials at the larger target-distractor distances. For the SPCN, the main effect of response type was not significant (F _{(1, 15)} = 2.47, p = 0.137), but the interaction was significant (F _{(5, 75)} = 2.5, p = 0.038), reflecting differences between correct and error trials at the intermediate target-distractor distances.

Figure 9.

Grand average difference waveforms (contralateral minus ipsilateral relative to the side containing the attended color) for correct and error trials in Experiment 2. The N2pc (200 – 300 ms) and SPCN (400 – 600 ms) measurement windows are highlighted. Waveforms from the different target-distractor distances are shown in A (No Distractors), B (16.8°), C (4.12°), D (3.13°), E (2.14°), and F (1.48°).

We also conducted a comparison of correct and error trials just at the 1.48° distance, where the number of error trials was reasonably high. There was no significant difference between correct trials and error trials for either N2pc amplitude (t₁₅ = −1.92, p = 0.07) or SPCN amplitude (t₁₅ = −1.8, p = 0.09) at this distance.

Although these error-related findings must be considered somewhat speculative given the small number of error trials, the results suggest that errors at large target-distractor distances are accompanied by a failure of the attention mechanism indexed by the N2pc component. This could, in turn, reflect occasional mind wandering or other kinds of off-task performance. It is difficult to draw conclusions about the presence or absence of differences in N2pc between correct and error trials at the small target-distractor distances, however, because the N2pc was small on both correct and error trials at these distances. However, these were the distances at which the SPCN was largest, and there was no evidence of a reduced SPCN on error trials at these distances. Thus, when crowding leads to behavioral errors, this does not reflect a reduction in the neurocognitive mechanism reflected by the SPCN, but may instead reflect a jumbling of the perceptual information that is passed to this mechanism by earlier stages of processing.

General Discussion

Both experiments confirmed the prediction that visuospatial attention (as indexed by the N2pc component) would be maximally recruited at intermediate target-flanker distances, near the edge of the crowding range. This effect was reflected by the finding of maximal N2pc amplitude at 3.13° in both experiments. However, when the target-flanker distance decreased from 3.13° to 1.48°, dramatic changes were observed¹. The correct response rate decreased from nearly 90% to 60%, and N2pc amplitude decreased by over 50%. These results are consistent with the proposal that attention has a limited spatial resolution (He et al., 1996; Intriligator & Cavanagh, 2001); after the limit is reached, attention can no longer select the target from the distractors (as reflected in reduced N2pc amplitude), and behavioral performance falls precipitously. This does not rule out the possibility that other factors also contribute to crowding, but it clearly demonstrates that crowding is accompanied by a failure of attention.

Other studies have examined how attention is modulated by target-distractor distance. For example, Hilimire et al. (2009, 2010) found a linear reduction in N2pc amplitude according to the distance between the target and a task-relevant distractor (decoy), both of which were distinctive colors presented within a ring of homogeneously colored distractors. These results are broadly consistent with our findings. However, the Hilimire et al. (2009, 2010) studies did not look at crowding per se, because they varied the distance between two pop-out colors within a ring of distractors, leaving the overall spacing of the stimuli constant.

Yeshurun and Rashal (2010) previously showed that directing attention toward the target increases performance and reduces the distance at which crowding impacts behavioral performance (the critical distance). This finding dovetails perfectly with the present results, which provide direct evidence that the attentional mechanism indexed by N2pc amplitude is maximal in the range of target-flanker distances in which behavioral performance is impacted by attention-directing cues. However, the present results go beyond those of Yeshurun and Rashal (2010) by showing how the allocation of attention varies as a function of target-flanker distance. That is, Yeshurun and Rashal’s (2010) results showed the effects of attention on perceptual performance, with no way of determining how attentional allocation varied across target-flanker distances. For example, their results are perfectly consistent with an equivalent allocation of attention to the cued side at all target-flanker distances. In contrast, the present study demonstrated substantial N2pc amplitude at intermediate target-flanker distances relative to no-flanker trials and showed that the N2pc was significantly reduced when the target-flanker distance fell below a critical distance.

It is important to consider exactly what can be concluded from the reduction in N2pc amplitude observed under conditions of crowding in the present experiment, especially given that it is always challenging to reason backward from a physiological effect to the underlying neurocognitive processes (Kappenman & Luck, 2012; Poldrack, 2006). There are two main ways in which the N2pc could plausibly be reduced during crowding. First, the reduced N2pc may indicate that participants attempted to focus attention on the target but that the process reflected by N2pc could not work effectively. For example, most of the available evidence indicates that the N2pc component reflects an attentional process that is applied to a target but is needed mainly when competition is present from distractors (Hickey et al., 2009; Luck, 2012). Under conditions of crowding, the resolution of attention may be too coarse to allow the target to be selected from among the distractors, and this may result in reduced N2pc amplitude. An alternative, however, is that the visual system detects the crowding preattentively, and participants do not even attempt to focus attention under conditions of severe crowding (which could be either a conscious strategy or an unconscious, learned response). In either of these scenarios, however, attention fails to be allocated to the target under conditions of close crowding. Thus, whether participants attempt to focus attention but fail, or whether they do not even attempt to focus attention; the present results indicate that the mechanism of attention indexed by the N2pc component is reduced under conditions of crowding.

Whereas the potential role of attention has been widely discussed in the crowding literature (Levi, 2008; Whitney & Levi, 2011), we know of no previous reports suggesting a role for working memory in visual crowding. However, the present results suggest that working memory was actively involved, with increasing SPCN amplitude as the amount of crowding increased. Sustained contralateral negativities are typically associated with the maintenance of information in working memory, with an increasing amplitude as more items are stored in working memory (Jolicoeur et al., 2008; Robitaille et al., 2010; Vogel & Machizawa, 2004). The SPCN has also been shown in tasks that do not explicitly ask participants to maintain information online. For example, Brisson and Jolicoeur (2007a, 2007b) and Corriveau et al. (2012) reported SPCN activity using a psychological refractory period paradigm, in which subjects were asked to select a spatial target without any memory manipulation, and this was interpreted as reflecting the storage of this information in working memory.

However, sustained contralateral negativities may also be obtained as a result of the sustained focusing of visuospatial attention on stimuli that remain visible for long periods of time, as in the multiple object tracking paradigm (Drew, Horowitz, Wolfe, & Vogel, 2012). Thus, the SPCN observed in the present study might reflect the sustained operation of attention rather than reflecting working memory storage, per se. However, because the stimulus duration in the present study was only 200 ms, and the SPCN lasted for at least 600 ms, information about the stimuli must have been maintained in memory for attention to have operated on them for this long period of time. Thus, the SPCN effects observed here must be at least an indirect reflection of the storage of information in memory.

The present results therefore suggest that working memory, as indexed by the SPCN, is recruited when attention has reached its spatial resolution limit. In other words, when attention cannot focus on the target and filter the distractors, information from multiple items is stored in working memory. The participant can then spend time trying to make an optimal target decision on the basis of this mixture of target and distractor information. This is consistent with the fact that RTs increased substantially at the small target-distractor distances. The above-chance target discrimination accuracy observed in the most crowded condition may reflect this working memory-based decision process. However, additional research is needed to determine the precise role that working memory plays in crowding.

The analysis of error trials revealed a substantial SPCN component for the most crowded conditions in both experiments, without any substantial differences between trials with correct and incorrect responses. These results suggest that participants had successfully stored information in working memory on error trials and that the errors occurred because this information was noisy (e.g., a jumbled combination of features of the target and flankers).

In sum, our results suggest that attention and working memory play distinct roles in accomplishing the goal of identifying a target that is surrounded by nearby distractors. When the target is presented without nearby distractors, the need for selective attention and working memory is reduced (although not eliminated). At intermediate target-flanker distances, attention is maximally recruited to suppress the flankers and select the target, avoiding a reduction in accuracy that would occur at these distances in the absence of focused attention (Yeshurun & Rashal, 2010). When the target-flanker distance falls below the critical distance, attention can no longer select the target and suppress the flankers. At this point, information from both the target and flankers may be stored in working memory so that the best possible decision can be made from the jumbled perceptual information.