Dissecting the Neural Focus of Attention Reveals Distinct Processes for Spatial Attention and Object-Based Storage in Visual Working Memory

Abstract

Complex cognition relies on both on-line representations in working memory (WM), said to reside in the focus of attention, and passive off-line representations of related information. Here, we dissected the focus of attention by showing that distinct neural signals index the on-line storage of objects and sustained spatial attention. We recorded electroencephalogram (EEG) activity during two tasks that employed identical stimulus displays but varied the relative demands for object storage and spatial attention. We found distinct delay-period signatures for an attention task (which required only spatial attention) and a WM task (which invoked both spatial attention and object storage). Although both tasks required active maintenance of spatial information, only the WM task elicited robust contralateral delay activity that was sensitive to mnemonic load. Thus, we argue that the focus of attention is maintained via a collaboration between distinct processes for covert spatial orienting and object-based storage.

Working memory (WM) facilitates the temporary maintenance of small amounts of information so that information can be manipulated or acted on. Contemporary theories of WM coalesce around variations of embedded-process models (Cowan, 1999), in which performance in WM tasks depends on memory mechanisms that represent information in two distinct states: an on-line, active state (focus of attention) and off-line states of memory that include representations in long-term memory (LTM) as well as a subset of those representations that are still rapidly accessible as a result of recency or contextual priming (activated LTM). While on-line representations in the focus of attention have typically been associated with persistent neural firing (Curtis & D’Esposito, 2003), recent work suggests that representations in activated LTM could be supported by rapid changes in synaptic connectivity that allow recently attended items to be quickly reinstated (Lewis-Peacock, Drysdale, Oberauer, & Postle, 2012; Rose et al., 2016; Stokes, 2015).

Here, we sought to characterize the neural mechanisms supporting the focus of attention. Broad neuroscientific support for focus-of-attention-related activity has been observed in sustained neural firing in monkey electrophysiological studies (Buschman, Siegel, Roy, & Miller, 2011; Funahashi, Chafee, & Goldman-Rakic, 1993), in uni- and multivariate measurements of blood-oxygen-level-dependent (BOLD) signals in human functional MRI studies (Cowan et al., 2011; Todd & Marois, 2004; Xu & Chun, 2006), and in sustained electrical and magnetic fluctuations in human electroencephalogram (EEG) and magnetoencephalography (MEG) studies (van Dijk, van der Werf, Mazaheri, Medendorp, & Jensen, 2010; Vogel & Machizawa, 2004).

Within EEG and MEG studies, two candidate measures are consistent with the focus-of-attention construct. The first is alpha power (8–12 Hz), which shows sustained modulations during the retention period and contains precise spatial information about the remembered and attended stimulus (Foster, Bsales, Jaffe, & Awh, 2017; Foster, Sutterer, Serences, & Awh, 2016). Another candidate is contralateral delay activity (CDA), which is a sustained negativity over the hemisphere contralateral to the positions of to-be-remembered items. CDA amplitude is modulated by the number of items held in WM, reaches an asymptote at WM capacity, dynamically tracks dropping information, and predicts individual differences in WM capacity (Vogel & Machizawa, 2004; Vogel, McCollough, & Machizawa, 2005; Williams & Woodman, 2013). A prevailing view of CDA is that it tracks the number of task-relevant objects that are stored in WM (Balaban & Luria, 2017; Luria, Balaban, Awh, & Vogel, 2016).

While the literature on CDA and alpha power have largely developed independently, recent proposals claim that they reflect isomorphic measures of the focus of attention. Specifically, van Dijk et al. (2010) argued that CDA is an averaging artifact of trial-level alpha modulation and, therefore, reflects attention to spatial positions of the memoranda, rather than representations of items in WM. A similar proposal was made by Berggren and Eimer (2016), who found that when two arrays were presented sequentially in different hemifields, CDA amplitude tracked the positions of the most recently seen items (but see also Feldmann-Wüstefeld, Vogel, and Awh, 2018). Such spatial-attention accounts make two broad, but untested, assertions regarding neural measures of the focus of attention. The first is that sustained EEG activity reflecting the focus of attention exclusively represents the current regions of attended space, rather than the on-line maintenance of items in those regions of space (Berggren & Eimer, 2016). The second assertion is that such neural measures amount to a monolithic focus of attention, rather than a collection of distinct but overlapping mechanisms that together comprise the focus of attention.

Here, we provide evidence that the focus of attention in WM is not a monolithic construct but rather involves at least two neurally separable processes: (a) attention to regions in space and (b) representations of objects that occupy the attended regions (i.e., object files). Alpha activity, but not CDA, tracked attention to relevant spatial positions. Conversely, when participants stored object representations, lateralized alpha activity that tracked the attended positions was accompanied by robust, load-sensitive CDA. These results suggest that the neural focus of attention can be dissected into at least two complementary, but distinct, facets of activity: a map of prioritized space and on-line representations of objects.

Method

Experimental design

Our broad strategy was to compare delay-period activity across two tasks that employed physically identical displays but distinct cognitive requirements. We designed discrete attention and WM tasks to disentangle the neural correlates of hypothesized subcomponents of the focus of attention. Both tasks are known to recruit sustained spatial attention (i.e., representation of a spatial priority map), but only the WM task invokes on-line storage of items (i.e., representation of the objects that occupy the attended locations). In all experiments, participants completed both a WM task and an attention task, and the sequence of physical stimuli was identical for both tasks; the tasks differed only in the instructions given to participants and in the response mapping to keys. In Experiment 1, the attention task required participants to direct spatial attention toward the locations of the items in the sample array (item color was irrelevant), whereas the WM task required that participants remember the color of the items in the sample array. Consequently, the key difference between the two very similar tasks was that participants were required to remember nonspatial features only in the WM task. To test whether the requirement to remember nonspatial features was responsible for our findings in Experiment 1, we employed tasks in Experiment 2 that were even more similar: The WM task required that participants store the spatial positions of items in the sample array, and the attention task required that participants covertly attend to objects’ spatial positions in anticipation of rare targets during the delay.

Participants

Experimental procedures were approved by The University of Chicago Institutional Review Board. All participants gave informed consent and were compensated for their participation with cash payment ($15 per hour); participants reported normal color vision and normal or corrected-to-normal visual acuity. Participants were recruited from The University of Chicago and the surrounding community. For each subexperiment (e.g., Experiment 1a), we set a minimum sample size of 20 participants (after attrition and artifact rejection). This minimum sample size was chosen to ensure that we would be able to robustly detect set-size-dependent delay activity. Prior work employing sample sizes of 10 to 20 participants per experiment robustly detected set-size-dependent CDA (Vogel & Machizawa, 2004; Vogel et al., 2005) and differences in CDA amplitude between novel experimental conditions (Balaban & Luria, 2017). We chose a minimum sample size toward the upper end of this conventional range.

A total of 63 and 54 participants were run in Experiments 1 and 2, respectively. Because of a technical error, EEG activity was not recorded for three participants in Experiment 1. In addition, data from some participants was excluded because of excessive EEG artifacts (< 120 trials remaining in any of the four experimental conditions) or poor behavioral performance. This left 48 participants in Experiment 1 (28 in Experiment 1a, 20 in Experiment 1b) and 49 participants in Experiment 2 (20 in Experiment 2a, 29 in Experiment 2b).

EEG acquisition

Participants were seated inside an electrically shielded chamber, with their heads resting on a padded chin rest 74 cm from the monitor. We recorded EEG activity from 30 active Ag/AgCl electrodes (actiCHamp, Brain Products, Munich, Germany) mounted in an elastic cap positioned according to the international 10-20 system (Fp1, Fp2, F7, F8, F3, F4, Fz, FC5, FC6, FC1, FC2, C3, C4, Cz, CP5, CP6, CP1, CP2, P7, P8, P3, P4, Pz, PO7, PO8, PO3, PO4, O1, O2, Oz). Two additional electrodes were affixed with stickers to the left and right mastoids, and a ground electrode was placed in the elastic cap at position Fpz. Data were referenced on-line to the right mastoid and rereferenced off-line to the algebraic average of the left and right mastoids. Incoming data were filtered (low cutoff = .01 Hz, high cutoff = 80 Hz; slope from low to high cutoff = 12 dB/octave) and recorded with a 500-Hz sampling rate. Impedance values were kept below 10 kΩ.

Eye movements and blinks were monitored using electrooculogram (EOG) activity and eye tracking. We collected EOG data with five passive Ag/AgCl electrodes (two vertical EOG electrodes placed above and below the right eye, two horizontal EOG electrodes placed ~1 cm from the outer canthi, and one ground electrode placed on the left cheek). We collected eye-tracking data using a desk-mounted EyeLink 1000 Plus eye-tracking camera (SR Research, Ontario, Canada) sampling at 1,000 Hz. Usable eye-tracking data were acquired for 25 out of 28 participants in Experiment 1a, 19 out of 20 participants in Experiment 1b, 17 out of 20 participants in Experiment 2a, and 29 out of 29 participants in Experiment 2b.

Artifact rejection

Eye movements, blinks, blocking, drift, and muscle artifacts were first detected by applying automatic criteria. After automatic detection, trials were manually inspected to confirm that detection thresholds were working as expected. Participants were excluded if they had fewer than 120 total trials remaining in any of the four conditions. In Experiment 1a, we rejected an average of 25% of trials across all four conditions. This left us with the following average number of trials in each condition: 282 for WM Set Size 2, 275 for WM Set Size 4, 302 for attention Set Size 2, and 302 for attention Set Size 4. In Experiment 1b, we rejected an average of 32% of trials across all four conditions. This left us with the following average trials: 291 for WM Set Size 2, 285 for WM Set Size 4, 320 for attention Set Size 2, and 320 for attention Set Size 4. In Experiment 2a, we rejected an average of 22% of trials across all four conditions. This left us with the following average trials: 302 for WM Set Size 2, 301 for WM Set Size 4, 322 for attention Set Size 2, and 323 for attention Set Size 4. In Experiment 2b, we rejected an average of 27% of trials across all four conditions. This left us with the following average trials: 283 for WM Set Size 2, 283 for WM Set Size 4, 298 for attention Set Size 2, and 295 for attention Set Size 4.

Eye movements

We used a sliding-window step function to check for eye movements in the horizontal EOG (HEOG) and the eye-tracking gaze coordinates. For HEOG rejection, we used a split-half sliding-window approach (window size = 100 ms, step size = 10 ms, threshold = 20 µV). We used the HEOG rejection only if the eye-tracking data were bad for that trial epoch. We slid a 100-ms time window in steps of 10 ms from the beginning to the end of the trial. If the change in voltage from the first half to the second half of the window was greater than 20 µV, it was marked as an eye movement and rejected. For eye-tracking rejection, we applied a sliding-window analysis to the x-gaze coordinates and y-gaze coordinates (window size = 100 ms, step size = 10 ms, threshold = 0.5° of visual angle).

Blinks

We used a sliding-window step function to check for blinks in the vertical EOG (window size = 80 ms, step size = 10 ms, threshold = 30 µV). We checked the eye-tracking data for trial segments with missing data points (no position data are recorded when the eye is closed).

Drift, muscle artifacts, and blocking

We checked for drift (e.g., skin potentials) by comparing the absolute change in voltage from the first quarter of the trial to the last quarter of the trial. If the change in voltage exceeded 100 µV, the trial was rejected for drift. In addition to slow drift, we checked for sudden steplike changes in voltage with a sliding window (window size = 100 ms, step size = 10 ms, threshold = 100 µV). We excluded trials for muscle artifacts if any electrode had peak-to-peak amplitude greater than 200 µV within a 15-ms time window. We excluded trials for blocking if any electrode had at least 30 time points in any given 200-ms time window that were within 1µV of each other.

Analysis of horizontal gaze position

We rejected all trials that included eye movements greater than 0.5° of visual angle. Nevertheless, participants could still move their eyes within the 0.5° threshold (i.e., microsaccades). To contrast eye movements in the two tasks, we compared the horizontal gaze position recorded by the eye tracker. We were most concerned with horizontal eye movements, as these could contaminate our lateralized EEG measures. We drift-corrected gaze-position data by subtracting the mean gaze position measured 200 ms before the precue to achieve optimal sensitivity to changes in eye position (Cornelissen, Peters, & Palmer, 2002). We then took the mean change in gaze position (in degrees of visual angle) for left and right trials during the same time window that we used in the CDA analysis—400 to 1,450 ms after stimulus onset. Eye-gaze values from left trials were sign-reversed so that left and right trials could be combined. Consequently, positive values indicate eye movements toward the cued side, and negative values indicate eye movements away from the cued side. The eye-tracking data from some participants was poor in quality and could not be included in this analysis. Therefore, only 25 participants from Experiment 1a, 19 participants from Experiment 1b, 17 participants from Experiment 2a, and 27 participants from Experiment 2b were included.

Analysis of pupil dilation

As an additional metric of task difficulty, we compared task-evoked pupil dilation between the WM and attention tasks. Many studies have demonstrated that task-evoked pupil dilation correlates with cognitive load; the pupil dilates more when there are higher attentional and working memory demands (Beatty, 1982; Steinhauer & Hakerem, 1992). Since we were most interested in assessing the relative difficulty of the two tasks, we collapsed the data across set size within each task. For our analysis, the baseline for pupil-dilation data was 400 to 0 ms before the onset of the colored squares. Taking this baseline into account, we calculated differences in pupil dilation between the WM and attention tasks (collapsed across set sizes) by comparing pupil size during the same time window as that used in the CDA analysis (400–1,450 ms after stimulus onset). Just as in the analysis of horizontal gaze position, the eye-tracking data from some participants were not good enough to be included in the analysis. As a result, the same participants included in the previous analyses of horizontal gaze position were also included in the analyses of pupil dilation.

Analysis of CDA

EEG activity was calculated using a baseline from 400 ms to 0 ms before the onset of the stimulus array. Trials containing targets for the attention task were excluded. Event-related potentials were calculated by averaging baseline activity at each electrode across all accurate trials within each condition (WM Set Size 2, WM Set Size 4, attention Set Size 2, and attention Set Size 4). We calculated amplitude of contralateral and ipsilateral activity for five posterior and parietal pairs of electrodes chosen a priori on the basis of prior literature: O1/O2, PO3/PO4, PO7/PO8, P3/P4, and P7/P8. Statistical analyses were performed on data that were not filtered beyond the .01- to 80-Hz on-line data-acquisition filter; we low-pass-filtered data (30 Hz) for illustrative purposes in the figures.

Analysis of lateralized alpha power

EEG signal processing was performed in MATLAB (Version 2015a; The MathWorks, Natick, MA). We bandpass-filtered trial epochs in the alpha band (8–12 Hz) using a filter from the FieldTrip toolbox (ft_preproc_bandpass.m; Oostenveld, Fries, Maris, & Schoffelen, 2011) and then extracted instantaneous power by applying a Hilbert transform (hilbert.m) to the filtered data. Trials containing targets for the attention task were excluded. We calculated alpha power for the same five posterior and parietal pairs of electrodes as in the CDA analysis: O1/O2, PO3/PO4, PO7/PO8, P3/P4, and P7/P8.

Stimuli and procedures

Stimuli in all experiments were presented on a 24-in. LCD computer screen (BenQ XL2430T; 120-Hz refresh rate) on a Dell Optiplex 9020 computer. Participants were seated with their heads on a chin rest 74 cm from the screen. Stimuli were presented on a gray background, and participants fixated a small black dot with a diameter of approximately 0.2° of visual angle.

Experiment 1a

We ran two very similar versions of Experiment 1. In Experiment 1a, each trial began with a blank intertrial interval (750 ms), followed by a diamond cue (300 ms) indicating the relevant side of the screen (right or left). This diamond cue (maximum width = 0.65°, maximum height = 0.65°) was centered 0.65° above the fixation dot and was half green (RGB value: = 74, 183, 72) and half pink (RGB value: = 183, 73, 177). Half of the participants were instructed to attend to the green side, and the other half were instructed to attend to the pink side. After the cue, two or four colored squares (1.1° × 1.1°) briefly appeared in each hemifield (150 ms) with a minimum of 2.10° (1.5 objects) between each square. The squares then disappeared for a 1,300-ms delay period. Squares could appear within a subset of the display subtending 3.1° to the left or right of fixation and 3.5° above and below fixation. Colors for the squares were selected randomly from a set of nine possible colors (RGB values: red = 255, 0, 0; green = 0, 255, 0; blue = 0, 0, 255; yellow = 255, 255, 0; magenta = 255, 0, 255; cyan = 0, 255, 255; orange = 255, 128, 0; white = 255, 255, 255; black = 1, 1, 1). Colors were chosen without replacement within each hemifield, and colors could be repeated across, but not within, hemifields. On 10% of trials, two small, black lines (0.02° wide, 0.4° long) appeared (66.7 ms), one at the location of a colored square in the cued hemifield and one at the location of a colored square in the uncued hemifield. The lines could appear at any point during the delay period from 100 to 1,200 ms after the offset of the stimuli. Each line could be tilted 31.3° to the left or 31.3° to the right. At test, a probe display, consisting of 1 colored square in each hemifield, appeared until response.

Experiment 1b

All stimuli and procedures were the same as in Experiment 1a, with the following exceptions. On 10% of trials, two small black lines appeared in the cued hemifield. One line appeared at the location of a colored square. The other line appeared in an unoccupied location in the cued hemifield a minimum of 2.10° (1.5 objects) from the locations of the memory-array items.

Experiment 2a

Stimuli in Experiment 2a were similar to those in Experiment 1b, with the following exceptions. Participants were presented with two or four black circles (0.61° in diameter; 1.1° × 1.1°; RGB value = 1, 1, 1) in each hemifield with a minimum of 1.53° (1.5 objects) between each item. These circles and distractor lines could appear within a subset of the display subtending 2.44° to the left or right of fixation and 3.06° above and below fixation. On target-present trials, the two small lines that were presented briefly during the retention interval were 0.04° wide and 0.76° long. Both were presented in the attended hemifield. One was at the location of a colored square, and the other was in an unoccupied location that was a minimum of 1.5 objects away from any other memory location.

Experiment 2b

Stimuli were similar to those used in Experiment 2a, with the following exceptions. Participants were presented with two or four colored circles (0.84° in diameter) in each hemifield with a minimum of 2.10° (1.5 objects) between each item. Within each hemifield, colors for the circles were randomly selected without replacement from a set of 10 possible colors (RGB values: red = 255, 0, 0; green = 0, 255, 0; blue = 0, 0, 255; yellow = 255, 255, 0; magenta = 255, 0, 255; cyan = 0, 255, 255; orange = 255, 128, 0; brown = 102, 51, 0; white = 255, 255, 255; black = 1, 1, 1). On target-present trials, the two small lines that were presented briefly during the retention interval were 0.04° wide and 0.99° long. Both were presented in the attended hemifield. One was at the location of a colored square, and the other was in an unoccupied location that was a minimum of 1.5 objects away from any other memory location.

WM and attention tasks

Participants in all experiments completed a WM task and an attention task (Fig. 1). Within each experiment, the sequence of physical stimuli was identical for both tasks. Differences in procedures between the experiments are described below. The attention and WM tasks differed only in the instructions given to participants and in the keys used to respond. Task order (attention first or WM first) and relevant cue color (pink or green) were counterbalanced across participants. Participants completed 20 blocks of 80 trials each (1,600 trials total, or 400 per condition).

Fig. 1.

Working memory (WM) and attention tasks for all experiments. At the start of each trial, participants were cued by a diamond shape to attend to one side of the screen. Then an array of two or four colored squares (Experiments 1a and 1b) or circles (Experiments 2a and 2b) briefly appeared. On 10% of trials, during the blank retention interval (1,300 ms), two small lines appeared for 66 ms between 100 and 1,200 ms after memory-array offset. In Experiment 1a, one line appeared in each hemifield. In all other experiments, both lines appeared in the same hemifield, one in an attended location and one in an unattended location. After the retention interval, a response screen appeared with one square (Experiments 1a and 1b) or circle (Experiments 2a and 2b) in each hemifield. In the WM task, participants reported whether the square or circle that reappeared in the attended hemifield was the same color (Experiments 1a and 1b) or in the same location (Experiments 2a and 2b). Participants pressed “z” if it was the same and “/” if it was different. In the attention task, if a line was not present during the delay period, participants pressed the space bar. If a line was present during the delay, participants had to report the orientation of the line that appeared in one of the cued locations. Participants pressed “z” if it was tilted left and “/” if it was tilted right. The response screen remained visible until a response was made.

In the WM task for Experiments 1a and 1b, participants were instructed to remember the colors of the presented squares in the cued hemifield and to ignore the lines that might flash during the middle of the delay period. At test, participants were asked to identify whether the color presented at the relevant probed location was the same as the color held in mind (same trial) or different (change trial). The color changed on 50% of the trials. Participants pressed the “z” key to indicate “same” and the “/” key to indicate “different.” For Experiments 2a and 2b, the procedures for the WM task were very similar to the procedures from Experiment 1a and 1b, except that participants were asked to identify whether the location of the presented circle in the attended hemifield was in the same location as any of the original circles or in a different location.

Procedures and instructions for the attention task were identical in all experiments. Only the visual stimuli differed, so as to match the visual stimuli presented in the WM task. Participants were instructed to maintain their attention at the locations of the presented squares in the cued hemifield in order to identify the orientation of a small line that appeared at one of the attended locations on 10% of trials. Participants were instructed to press the “z” key if the line appeared and was tilted left and the “/” key if the line appeared and was tilted right. On 90% of trials, no line was presented, and participants were instructed to press the space bar to indicate that there was no target present. The physical stimulus displays were identical to those in the memory task; thus, one colored square appeared in each hemifield at the end of the attention trials. Participants were told that the appearance of the test display indicated that it was time to respond and that the location and the color of the squares were irrelevant to the task.

Stimuli and procedures in Experiment 1a differed from those in Experiments 1b, 2a, and 2b only for the target-present trials (10% of trials). Specifically, in these trials, we presented both a relevant and an irrelevant line within the cued hemifield. One line always appeared at the same location as one of the colored squares; the second line appeared at a foil location where no colored square had been presented (a minimum distance of 1.5 items’ width from any of the colored squares’ locations). Thus, participants were required to maintain their attention at precise locations within the relevant hemifield so they knew which line to report. We reasoned that the inclusion of an irrelevant item in the cued hemifield in Experiment 1b would encourage participants to orient attention more precisely. However, subsequent analyses revealed no main effect or interactions associated with the changes in procedure between Experiments 1a and 1b. Therefore, data were collapsed across these two versions of the task.

Additionally, in both the WM and attention tasks, the circles and squares in the sample array were always at least 1.5 objects apart from each other. In the attention task, the rare target probes appeared at the location of one of the original squares, while the distractor probe appeared in an uncued location that was at least 1.5 objects away from any of the attended locations. Thus, the attention task required participants to make the same spatial discrimination that participants had to make in the memory task in order to relate the test probe to the proper item from the memory array. In other words, the positions of the sample items had to be maintained with equal precision in the memory and attention tasks. This is most clear for Experiment 2, in which space was the sole relevant attribute for the memory task.

Results

We aggregated data across Experiments 1 and 2 (N = 97) to provide the most power for understanding the distinctions between the WM and attention tasks. While the aggregate results mirrored those of the individual experiments (see the Supplemental Material available online for a study-by-study analysis), the data taken together provide a clear demonstration of the essential empirical patterns. In this aggregate analysis, we focus on CDA, alpha power, and pupil size. For analyses of behavior across all experiments, see Figure 2. For further analyses of behavior, as well as analyses of eye position, for each experiment, see the Supplemental Material.

Fig. 2.

Behavioral performance. The average K score in the working memory task (a) and average accuracy in the attention task (b) is shown separately for each set-size condition, collapsed across all experiments. The distributions of data for all participants are represented by the violin plots. In each plot, the thick horizontal bar represents the mean of the sample, and the error bar shows the standard error of the mean. The density of the plot represents the distribution of data. Each of the small circles represents the mean for an individual participant, and the thin lines connect that participant’s means across conditions.

Preliminary analysis of the effect of experiment

In a preliminary analysis, we examined whether the small variations in task design between Experiments 1 and 2 had an effect on the observed results. For this purpose, we ran repeated measures analyses of variance (ANOVAs) for each analysis—CDA, alpha power, and pupil size—with the within-subjects factors task (WM, attention) and set size (two items, four items) and the between-subjects factor experiment (1, 2). For all analyses, there was no main effect of experiment, p ≥ .16. Therefore, it was justified to collapse data across the all experiments.

For the horizontal gaze position and the lateralized alpha analyses, none of the factors significantly interacted with experiment, p ≥ .19. However, for the pupil-dilation analysis, there was a significant interaction of task and experiment, F(1, 86) = 10.76, p = .002, η_p² = .11. This significant interaction is explained by greater pupil dilation in the attention task than in the WM task in Experiment 2 but not in Experiment 1.

For the CDA analysis, there was a significant three-way interaction of laterality, set size, and experiment, F(1, 95) = 6.73, p = .01, η_p² = .07. To further delineate this three-way interaction, we ran follow-up ANOVAs with the factors laterality (contralaterality, ipsilaterality) and set size (two items, four items), separately for Experiment 1 and Experiment 2. These follow-up analyses revealed that there was a significant interaction of laterality and set size for Experiment 2, F(1, 48) = 16.88, p < .001, η_p² = .26, but not for Experiment 1, F(1, 47) = 0.08, p = .78, η_p² = .002.

CDA

Using all data from Experiments 1 and 2 together, we ran a repeated measures ANOVA with the factors task (WM, attention) and set size (two items, four items). This analysis revealed significant main effects of laterality, F(1, 96) = 74.41, p < .001, η_p² = .44, and set size, F(1, 96) = 33.27, p < .001, η_p² = .26. Because these main effects were collapsed across task, they do not resolve our central question of how storage-related neural signals differ across tasks. Thus, the first important finding was a significant two-way interaction between laterality and task, F(1, 96) = 81.27, p < .001, η_p² = .46, which reflected a greater laterality effect in the WM task than in the attention task. To confirm this impression, we ran a follow-up two-way paired-samples t test that compared contralateral and ipsilateral activity separately for the WM and attention tasks and each set size (WM Set Size 2, WM Set Size 4, attention Set Size 2, attention Set Size 4). This analysis revealed that CDA was significantly more lateralized in the WM task (Set Size 2: M = −0.38 µV, SD = 0.44; Set Size 4: M = −0.54 µV, SD = 0.54) than in the attention task (Set Size 2: M = −0.09 µV, SD = 0.34; Set Size 4: M = −0.10 µV, SD = 0.37) for both set sizes—Set Size 2: t(98) = −6.71, p < .001; Set Size 4: t(98) = −8.57, p < .001. We note, however, that there was reliable lateralized activity for both tasks, p ≤ .007.

Another key finding was that the number of items in the sample array had a selective impact on CDA in the WM task, while CDA in the attention task showed no reliable effect. This impression was verified by a reliable triple interaction among task, laterality, and set size, F(1, 96) = 8.75, p = .004, η_p² = .08. As Figure 3 shows, CDA was set-size dependent in the WM task but not in the attention task. To characterize the triple interaction, we ran separate follow-up repeated measures ANOVAs for each task (WM, attention) with the factors laterality (contralaterality, ipsilaterality) and set size (two items, four items). This analysis revealed that there was a significant interaction of laterality and set size for the WM task, F(1, 96) = 14.39, p < .001, η_p² = .13, but not the attention task, F(1, 96) = 0.07, p = .79, η_p² = .001. Thus, while data from the WM task showed that CDA amplitude was larger for Set Size 4 (M = −0.55 µV, SD = 0.54) than for Set Size 2 (M = −0.39 µV, SD = 0.44), data from the attention task showed no evidence of a difference in CDA amplitude between Set Size 2 (M = −0.10 µV, SD = 0.34) and Set Size 4 (M = −0.11 µV, SD = 0.37).

Fig. 3.

Contralateral delay activity (CDA). CDA amplitude over time (a) is shown separately for each condition in the working memory and attention tasks, collapsed across all experiments. Error bands show within-subjects standard errors of the mean. Time 0 marks the onset of the memory array, and Time 1,450 marks the onset of the response array. Average CDA amplitude for each condition (collapsed across all experiments) during the time window of interest (400–1,450 ms) is shown in (b). The distributions of CDA amplitudes for all participants are represented by the violin plots. In each plot, the thick horizontal bar represents the mean of the sample, and the error bar shows the standard error of the mean. The density of the plot represents the distribution of data. Each of the small circles represents the mean for an individual participant, and the thin lines connect that participant’s means across conditions.

To summarize, the aggregate analysis showed that CDA was substantially stronger in the WM task than in the attention task. Moreover, CDA tracked the increase in mnemonic load from two to four items, while the CDA signal in the attention task—in addition to being over four times smaller than in the WM task—showed no effect of mnemonic load at all, a defining feature of CDA. This core result motivates our conclusion that CDA is directly linked with the on-line maintenance of object representations in WM and not with the deployment of attention to the positions of the sample items.

Lateralized alpha power

As Figure 4 shows, we observed the typical suppression of alpha power contralateral to the relevant hemifield in both tasks, though it was larger in the WM task. We confirmed these impressions with a repeated measures ANOVA on average alpha power with the factors laterality (contralaterality, ipsilaterality), task (attention, WM) and set size (two items, four items). This analysis revealed a significant main effect of laterality, F(1, 96) = 45.57, p < .001, η_p² = .32, and a significant interaction between laterality and set size, F(1, 96) = 9.75, p = .002, η_p² = .09. Paired-samples t tests confirmed that this interaction reflected a stronger lateralization of alpha power in the Set Size 2 condition (M = −12.24 µV², SD = 17.17) than in the Set Size 4 condition (M = −10.04 µV², SD = 16.06), t(96) = −3.123, p = .002. Thus, the strength of lateralized alpha activity varied with the number of stored or attended positions in both the WM and attention tasks. Critically, however, the effect of set size on lateralized alpha power was in the opposite direction from the effect we observed with CDA. CDA was stronger for Set Size 4 than for Set Size 2, whereas alpha lateralization was stronger for Set Size 2 than for Set Size 4. These findings support the hypothesis that CDA and alpha activity reflect distinct aspects of on-line storage in visual WM.

Fig. 4.

Lateralized alpha power. Lateralized alpha power over time (a) is shown separately for each condition in the working memory and attention tasks, collapsed across all experiments. Error bands show standard errors of the mean. Time 0 marks the onset of the memory array, and Time 1,450 marks the onset of the response array. Average lateralized alpha power (b) is shown for each condition (collapsed across all experiments) during the time window of interest (400–1,450 ms). The distributions of lateralized alpha power for all participants are represented by the violin plots. In each plot, the thick horizontal bar represents the mean of the sample, and the error bar shows the standard error of the mean. The density of the plot represents the distribution of data. Each of the small circles represents the mean for an individual participant, and the thin lines connect that participant’s means across conditions.

Our analysis also revealed a significant interaction between laterality and task, F(1, 96) = 27.22, p < .001, η_p² = .22, which reflected the greater lateralization of alpha power in the WM task than in the attention task. This impression was confirmed with a two-way paired-samples t test that revealed a significant difference in alpha power lateralization between the WM task (M = −7.79 µV², SD = 11.61) and the attention task (M = −3.35 µV², SD = 5.69), t(96) = −5.22, p < .001. Critically, both tasks showed clear evidence of lateralized alpha power in both set sizes (p < .001 for all conditions), confirming that covert attention was deployed to the position of sample items in a sustained fashion in both tasks. The greater lateralization of alpha power in the WM than in the attention task was a robust empirical pattern present in both experiments and in the aggregate analysis. Though we did not expect this pattern a priori, this reliable difference in alpha lateralization between the two tasks may reflect a direct influence of on-line object representations on the deployment of spatial attention.

Pupil dilation

We argue that the WM task encouraged on-line storage of object representations but the attention task did not. Thus, the restriction of load-dependent CDA to the WM task could reflect a direct link between CDA and item storage in WM. A clear alternative hypothesis, however, is that the WM task may differ from the attention task in the intensity or effort applied to the task rather than in the specific cognitive operations that were invoked. While accuracy was similar (and not at ceiling) in the two tasks, this does not provide strong evidence of equivalent effort. Fortunately, pupil-dilation measurements (Fig. 5) have been shown to provide a sensitive index of cognitive effort and arousal when bottom-up stimulus factors are controlled. Thus, we ran a two-way paired-samples t test to examine whether pupil size differed during the time window in which CDA was measured. This analysis revealed a greater level of pupil dilation in the attention task (M = 0.63 arbitrary units, SD = 1.89) than in the WM task (M = 0.29 arbitrary units, SD = 2.20), t(87) = −3.13, p = .002, suggesting that the attention task recruited greater levels of cognitive effort. Thus, our finding that CDA was far larger in the WM task cannot be explained by increased effort in the WM task. Indeed, pupil analysis of the aggregated data suggests that the WM task was the easier of the two.

Fig. 5.

Task-evoked pupil response. Pupil dilation (in arbitrary units, or a.u.) over time (a) is shown for the two working memory tasks (collapsed; blue) and two attention tasks (collapsed; red), across all experiments combined. Error bands show standard errors of the mean. Time 0 marks the onset of the memory array, and Time 1,450 marks the onset of the response array. Average pupil dilation (b) is shown for each condition (collapsed across all experiments) during the time window of interest (400–1,450 ms). The distributions of pupil dilation for all participants are represented by the violin plots. In each plot, the thick horizontal bar represents the mean of the sample, and the error bar shows the standard error of the mean. The density of the plot represents the distribution of data. Each of the small circles represents the mean for an individual participant, and the thin lines connect that participant’s means across conditions.

These findings suggest a difference in the nature of the cognitive operations evoked by the WM and attention tasks rather than in the degree to which similar operations were carried out. In line with this, we also note that while behavioral data from the attention task showed that monitoring four locations was more difficult than monitoring two locations, CDA in the attention task was unaffected by set size. Therefore, our findings provide a strong argument against the hypothesis that stronger delay period signals in the WM task were a consequence of greater cognitive effort.

Summary

With 97 participants, our aggregate analysis provided strong statistical power for documenting how neural activity differed between the WM and attention tasks. CDA was more than four times larger in the WM than the attention task. Moreover, CDA in the WM task clearly tracked changes in mnemonic load, whereas CDA in the attention task showed no evidence of load sensitivity. Thus, given that these tasks employed identical stimulus displays, we conclude that CDA may be directly tied to the unique object-representation requirements in the WM task and not to covert attentional orienting to the sample-array positions.

The WM and attention tasks differ in terms of object storage, but past work suggests that both these tasks may require a common spatial-attention process that elicits orderly changes in the scalp topography of alpha power. In addition to past research showing the broad involvement of alpha activity across a wide range of attention and memory paradigms (Canolty & Knight, 2010; Fries, 2005; Klimesch, 2012), more recent work has also established the topography of alpha activity on the scalp can be used to precisely track the locus of covert attention (Rihs, Michel, & Thut, 2007) and locations stored in WM (Foster, Bsales, et al., 2017; Foster et al., 2016). In line with this work, our experiments showed clear evidence from both the WM and attention tasks that alpha power in posterior electrodes was reduced contralaterally to the sample array. Importantly, the aggregate analysis also had enough power to reveal a reliable effect of set size on alpha lateralization; specifically, greater lateralization was observed in the Set Size 2 than Set Size 4 condition. This effect should be interpreted with caution, however, as some previous studies have not found an effect of set size on lateralization (Fukuda, Mance, & Vogel, 2015), while others have found greater lateralization for larger set sizes (Sauseng et al., 2009). Nevertheless, the fact that CDA showed the opposite pattern, with higher CDA for the larger set size, highlights the possibility that these two neural signals (measured from the same set of electrodes) index distinct aspects of maintenance within the focus of attention.

The WM task required participants to make an attentionally demanding response after every trial. However, in the attention task, participants made an attentionally demanding orientation determination on only 10% of trials. On the remaining 90% of trials, participants pressed the space bar, indicating that no lines were present. To our knowledge, the influence of probe probability on CDA amplitude has not been investigated. Therefore, the difference in CDA amplitude in the two tasks could be affected by probe probability. This alternate explanation is unlikely, as a large body of research has shown that CDA tracks information stored in WM across a wide range of response modalities, including two-alternative change detection (Vogel & Machizawa, 2004), whole report of discrete colors (Adam, Robison, & Vogel, 2018), and tracking of dynamic displays (Balaban & Luria, 2017; Drew & Vogel, 2008).

Discussion

The focus of attention refers to the set of mental representations that are held in an on-line, or readily accessible, state. There has been a longstanding effort to elucidate the neural signals that track the contents of this internally attended information because of the focus of attention’s central role in intelligent behaviors. This literature has tended to treat the focus of attention as a monolithic entity by suggesting that internal attentional processes influence selection and maintenance of cognitive representations in the absence of sensory input (Chun, 2011; Chun, Golomb, & Turk-Browne, 2011). This idea has been supported by neural evidence showing that sustained WM representations in the brain occur in the same regions as perceptual representations, which are modulated by attentional mechanisms (Postle, 2006). However, in this study, we extended the growing evidence that the focus of attention may be implemented via multiple component processes playing distinct functional roles: one that represents currently prioritized space (alpha) and another that reflects item storage (CDA). This proposal converges with findings that suggest a dissociation between spatial attention and WM storage (Sheremata, Somers, & Shomstein, 2018; Tas, Luck, & Hollingworth, 2016).

CDA and lateralized alpha power: distinct components of the focus of attention

Other researchers have proposed that asymmetric modulations of alpha power at the trial level can generate a CDA-like negative slow wave in an event-related average (van Dijk et al., 2010). However, there is growing evidence that these two measures can be dissociated. For example, Fukuda, Kang, and Woodman (2016) used a lateralized change-detection task in which they cued participants to one side of the screen but delayed the stimulus-onset asynchrony between cue and memory array longer than normal (1,000 ms). During this blank cue period, participants knew which hemifield would contain memory items, but no items had yet appeared. During this time, there was robust lateralized alpha power but no CDA. However, after the memory array appeared, CDA and lateralized alpha power appeared in concert during the memory maintenance period. These results suggest that lateralized alpha power, and thus attention, can be shifted to empty space, but that CDA necessitates object storage (see also Fukuda et al., 2015). A similar dissociation between attentional deployment to objects and to spatial location has been found with the N2PC, a related component (Woodman, Arita, & Luck, 2009).

CDA as an index of item-based storage in WM

What was the critical difference between the WM and attention tasks? Both tasks elicited covert orienting to the positions of the items in the sample array, as shown by sustained lateralized-alpha-power modulations. Moreover, despite their distinctive monikers, both tasks required the sustained maintenance of spatial information across a blank delay. This storage requirement is obvious for the WM change-detection task. But even in the attention task, participants must have maintained the cued positions to distinguish targets from lures. Indeed, in all experiments, change detection in the WM task required precisely the same spatial discriminations as did target identification in the attention task. Thus, we propose that the critical difference between the WM task and the attention task was that the WM task encouraged the representation of the items in the sample array, while in the attention task, participants directed spatial attention to those positions without maintaining the items themselves.

CDA as a neural index of object file maintenance

Our interpretation of CDA as an index of continued representations of object files critically hinges on a distinction between the maintenance of items and the maintenance of spatial information without an accompanying item representation. Although some readers may view this as provocative, recent work has shown dissociable patterns of activity in the parietal lobe between WM and spatial-attention demands (Sheremata et al., 2018). Additionally, we note that there is a longstanding precedent for a distinction between representations of objects and representations of features or identifying labels associated with that object. Kahneman, Treisman, and Gibbs (1992) elucidated this idea with the object-file construct, which proposes two separable stages of processing. The first involves the parsing of the scene into a set of individuated items that are indexed on the basis of spatial and temporal coordinates. Subsequently, the specific feature values (e.g., color and orientation) are processed and incorporated into the associated object file. Thus, object files anchor the episodic representation in a specific time and place and are distinct from the specific feature values that are bound together by an object file.

In the present context, an intriguing possibility is that CDA indexes maintenance of object files in WM. This proposal is consistent with recent work showing that CDA is sensitive to object-hood cues (Balaban & Luria, 2016) and tracks the number of encoded objects, not the number of features (Luria & Vogel, 2011). Thus, even though the attention task required the sustained maintenance of location information, CDA was minimal or absent (and insensitive to mnemonic load) because the task did not encourage the maintenance of the object files that were created during the encoding of the sample array. Finally, although some researchers have argued that object files may mediate the binding of features within an object, we note that this does not preclude the operation of object files for single-feature objects (Kahneman et al., 1992), such as those required by the spatial WM task (Experiment 2). Thus, we propose that CDA may provide a neural index of object-file maintenance.

Open question on the impact of object files on the allocation of spatial attention

In this series of experiments, lateralized alpha power was a useful tool to illustrate that participants sustained their attention to the cued side even when CDA was completely absent (Experiment 1). However, we also observed a main effect of our task manipulation on lateralized alpha power. When task demands required participants to encode object representations, alpha power was significantly more lateralized than when participants had to sustain their attention only to empty space. Though we did not predict this pattern a priori, it was reliable in both experiments. This suggests that, like CDA, lateralized alpha power respects the dissociation between object representations and spatial priority maps. One possible interpretation of this effect is that object representations serve as “anchors” for the allocation of spatial attention, amplifying attention effects and increasing lateralized alpha power. Such an anchoring effect may provide a productive perspective on prior demonstrations of object-based attention (Egly, Driver, & Rafal, 1994). While future work is needed to investigate the complex interrelationship between lateralized alpha power and on-line object representations, the present work clearly suggests that lateralized alpha power does not directly generate, and is dissociable from, CDA.

Conclusions

A growing body of evidence has shown that CDA and alpha power are tightly linked to the maintenance of information in the focus of attention. Here, we present new evidence that these two neural signals represent distinct facets of this on-line system. A topographic distribution of alpha power indexes the current locus of spatial attention, a process that is integral to both visual selection and voluntary storage of items in WM. By contrast, CDA tracks active maintenance of object files, item-based representations that allow observers to integrate the ensemble of features and labels that are associated with visual objects. The dissociable activity of CDA and alpha power suggests that the focus of attention is composed of at least two distinct but complementary neural processes, a conclusion with strong implications for both cognitive and neural models of this on-line storage system.