The Relationship between Visual Attention and Visual Working Memory Encoding: A Dissociation between Covert and Overt Orienting

Abstract

There is substantial debate over whether visual working memory (VWM) and visual attention constitute a single system for the selection of task-relevant perceptual information or whether they are distinct systems that can be dissociated when their representational demands diverge. In the present study, we focused on the relationship between visual attention and the encoding of objects into visual working memory (VWM). Participants performed a color change-detection task. During the retention interval, a secondary object, irrelevant to the memory task, was presented. Participants were instructed either to execute an overt shift of gaze to this object (Experiments 1–3) or to attend it covertly (Experiments 4 and 5). Our goal was to determine whether these overt and covert shifts of attention disrupted the information held in VWM. We hypothesized that saccades, which typically introduce a memorial demand to bridge perceptual disruption, would lead to automatic encoding of the secondary object. However, purely covert shifts of attention, which introduce no such demand, would not result in automatic memory encoding. The results supported these predictions. Saccades to the secondary object produced substantial interference with VWM performance, but covert shifts of attention to this object produced no interference with VWM performance. These results challenge prevailing theories that consider attention and VWM to reflect a common mechanism. In addition, they indicate that the relationship between attention and VWM is dependent on the memorial demands of the orienting behavior.

Visual working memory (VWM) is a limited-capacity, short-term storage system used to maintain visual representations relevant to the current task (for a review, see Campbell & Thompson, 2012). Only a tiny proportion of the visual information that could be encoded into VWM—either visible in the environment or available for retrieval from long-term memory—is task relevant at a given moment, and VWM capacity is severely limited (Irwin, 1992a; Luck & Vogel, 1997; Pashler, 1988). In addition, VWM must have the capability for rapid, flexible updating as goals evolve (e.g., adapting one’s search template to specify the features of the bread, then the butter, then the knife) yet retain the ability to preserve information of persisting relevance (e.g., maintaining a template of one’s keys during an extended search). Given these demands for tight strategic control over a limited-capacity system, one of the central topics of VWM research is the set of operations that selectively encode and maintain information in VWM.

A common view in the literature holds that these control mechanisms are equivalent with visual attention. This position has been driven by evidence that cuing manipulations causing attention to be directed to a particular location influence which objects are encoded into VWM (e.g., Schmidt, Vogel, Woodman, & Luck, 2002), and which objects are retained after encoding (e.g., Griffin & Nobre, 2003a; Landman, Spekreijse, & Lamme, 2003). The conceptualization of VWM control as equivalent with visual attention forms part of a larger claim that VWM and attention are simply two terms to describe the same selective mechanism (Cowan, 2001; Jiang & Song, 2005; Kiyonaga & Egner, 2013; Rensink, 2002; Theeuwes, Belopolsky, & Olivers, 2009; Wang & Spelke, 2002; Wheeler & Treisman, 2002). In this view, when this mechanism is directed to the representations of currently visible stimuli, we term it visual attention; when it is directed to the representations of previously visible stimuli, we term it VWM. These claims were first developed in Cowan’s influential model of working memory (Cowan, 2001), which proposes an “equivalence of the focus of attention and the capacity-limited portion of STM [short-term memory]” (p. 91). They were applied to feature binding in visual perception and VWM by Rensink (2002) and Treisman (Wheeler & Treisman, 2002). Finally, equivalence between visual attention and VWM has been endorsed and expanded in several recent reviews of the literature on VWM control. For example, Chun and colleagues (Tipper, Weaver, Jerreat, & Burak, 1994; Wang & Spelke, 2002) have argued that VWM should be conceived as the allocation of visual attention to internal representations of previously visible stimuli: “Working memory is the interface through which attentional mechanisms select relevant perceptual information from the external world and actively maintain the information as internal representations within the mind” (Wang & Spelke, 2002, p. 1409). Similarly, Kiyonaga and Egner (2013) have claimed that working memory and attention “should no longer be considered as separate systems or concepts, but as competing and influencing one another because they rely on the same limited resource” (p. 228).

In contrast, we have argued that although visual attention and VWM often play complementary roles in visually guided behavior, they are distinct mechanisms and can be dissociated when those roles diverge (Baizer, Ungerleider, & Desimone, 1991; Hollingworth & Henderson, 2002; Johnson, Hollingworth, & Luck, 2008; Woodman, Vogel, & Luck, 2001; Zwaan, Stanfield, & Yaxley, 2002). We have shown that a concurrent VWM task has minimal effects on the efficiency of attentional selection during visual search (Baizer et al., 1991; Woodman et al., 2001), that the binding of object features in VWM does not depend on sustained visual attention to the object (Johnson et al., 2008), and, of particular relevance to VWM control mechanisms, that the selective maintenance of visual object representations in VWM can be dissociated from the locus of visual attention (Block, 1980; Zwaan et al., 2002).

Reconciling these competing views requires careful consideration of how attention and working memory are defined. It is potentially problematic to equate working memory with attention given that there are clearly many varieties of attention (Alhazen, 1083; Luck & Vecera, 2002) and multiple dissociable working memory subsystems (Godijn & Theeuwes, 2012). Working memory for objects and surface features (VWM) is at least partially dissociable from spatial working memory (SpWM) (Szinte, Carrasco, Cavanagh, & Rolfs, 2015; Tresch, Sinnamon, & Seamon, 1993). Correspondingly, visual selection on the basis of surface features (feature-based attention) is at least partially dissociated from visual selection on the basis of location (spatial attention) (Bichot, Rossi, & Desimone, 2005; Martinez-Trujillo & Treue, 2004; Zhou & Desimone, 2011). Moreover, the operation of attention within perceptual systems can be distinguished in several ways from central attentional mechanisms that play a role in response selection, memory search, and so on (Lleras & Enns, 2004). In the present study, we will focus on VWM, and we will focus on the role of spatial attention in the selective encoding of perceptual information into VWM. The relationship between attention and VWM maintenance will be addressed in the General Discussion as part of a broader discussion of the overlap between working memory and attention systems.

Under the view that visual attention and VWM constitute a common mechanism, there should be no functional distinction between attending to an object and representing that object in VWM. Wherever one directs attention, the attended object should necessarily be encoded in VWM. Previous studies suggest that attention facilitates the encoding of items into VWM, but they fall short of demonstrating equivalence between attention and VWM. For instance, Schmidt et al. (2002) showed that shifting attention to a particular spatial location increases the probability that the item at that location will be encoded and retained (see also, Averbach & Coriell, 1961; Scholl, 2000; Sperling, 1960). In addition, objects that capture attention are preferentially consolidated into VWM (Belopolsky, Kramer, & Godijn, 2008; Carmel & Lamy, 2014). Finally, objects near the target location of an impending saccade are preferentially encoded into VWM (Currie, McConkie, Carlson-Radvansky, & Irwin, 2000b; Henderson & Hollingworth, 2003; Irwin, 1992b), and it is well established that spatial attention is directed to the saccade target location before the saccade (e.g., Hoffman & Subramaniam, 1995b; Kowler, Anderson, Dosher, & Blaser, 1995). Although these findings are consistent with a close relationship between spatial attention and VWM encoding, none of the studies created a situation in which perceptual selection diverged from the demands of selective entry into VWM: There was no disincentive to remember items at the attended location. Thus, these studies cannot address whether there is an obligatory relationship between attention and VWM encoding, as implied by the claim that attention and VWM reflect a common mechanism.

Despite broad interest in the relationship between attention and VWM, to our knowledge, only one study has examined whether attended objects are automatically encoded into VWM. Olson, Moore, and Drowos (2008) had participants view a set of unfamiliar shapes presented sequentially at central fixation, with the assumption that every object would be attended. Randomly intermixed target objects (to be remembered) and distractor objects (to be ignored) were differentiated by the presence or absence of a surrounding rectangle. At test, a single object appeared, and participants responded to indicate whether it had or had not been a member of the target set. Critically, “no-match” trials were divided between trials on which the non-matching object was novel and trials on which the non-matching object was drawn from the distractor set. A higher false alarm rate for these latter “lure” trials was taken as evidence that attended distractors were encoded into VWM despite instructions to ignore them. Although it is clear that distractors were sometimes encoded into memory in Olson et al. (2008), the false alarm rate was only moderately higher for lures than for novel objects and thus could have been driven by a relatively small proportion of trials on which control over access to VWM lapsed. In addition, distractors were not strictly task irrelevant. If a distractor was remembered as such and then appeared as the test item, participants could confidently report “no match,” providing incentive to remember distractors despite the instructions. False alarms on lure trials may then have reflected cases when the stimuli were miscategorized as target/distractor during acquisition or on which the binding of object to target/distractor category was perturbed during retention. Thus, the question of whether attended items are automatically encoded into VWM remains open.

Present Study

In the present study, we tested the relationship between spatial attention and VWM encoding. Participants remembered a set of colors that exceeded VWM capacity. During the retention interval, they were or were not required to shift attention to an intervening, secondary object. The secondary object was irrelevant to the memory task, and encoding it into VWM could only impair memory performance. The shift of attention to the secondary object was either overt or covert. If attending an object is equivalent to representing it in VWM, then the secondary object should be encoded into VWM, interfering with the maintenance of the previously encoded colors and reducing color-memory performance relative to a baseline condition with no secondary object. In contrast, we predicted that a shift of attention to an object should result in VWM encoding only if the shift itself introduces a demand to encode information into VWM. Specifically, we predicted that overt shifts of attention, which introduce a perceptual gap and a memorial demand to bridge that gap, will lead to automatic encoding of an attended object into VWM, but purely covert shifts of attention, which do not generate any memorial demand, will not necessarily lead to encoding of the attended object. We elaborate on each of these predictions below.

Because saccade execution creates a disruption in visual input, establishing representational contuity across the saccade requires memory. Classic studies suggest that transsaccadic memory is equivalent with VWM (Irwin, 1991; Irwin & Andrews, 1996), and that its contents are dominated by items at or near the saccade target location (Irwin, 1992a; Irwin & Gordon, 1998). Specifcally, before the saccade, visual attention is directed to the saccade target object (Armstrong, Fitzgerald, & Moore, 2006; Hoffman & Subramaniam, 1995a; Kowler et al., 1995), leading to preferential encoding of that object into VWM. After the saccade, the VWM representation of the saccade target allows it to be identified among objects near the fovea (Hollingworth & Luck, 2009; Hollingworth, Richard, & Luck, 2008a). The use of VWM to establish target correspondence is a key mechanism supporting the perception of visual stability across saccades (Currie, McConkie, Carlson-Radvansky, & Irwin, 2000a; Irwin, McConkie, Carlson-Radvansky, & Currie, 1994; Tas, Moore, & Hollingworth, 2012). Given that transsaccadic VWM supports operations that will apply to thousands of saccades each day, the relationship between saccade preparation and VWM encoding is likely to be highly automatized. Thus, in the present study, we expected an obligatory relationship between overt shifts of attention to the secondary object and VWM encoding (Hollingworth, Richard, & Luck, 2008b; Shao et al., 2010). A possible exception, examined in Experiment 2, is when a saccade is directed to empty space rather than to an object.

Although saccade preparation requires a shift of attention to the saccade target location, behavioral (Bae, Olkkonen, Allred, Wilson, & Flombaum, 2014; R. M. Klein, 1980; R. M. Klein & Pontefract, 1994) and neurophysiological evidence (Golomb, Marino, Chun, & Mazer, 2011; Juan, Shorter-Jacobi, & Schall, 2004; Thompson, Biscoe, & Sato, 2005) suggests that it is possible to attend covertly without preparing a saccade. Such purely covert shifts of attention may certainly be initiated in the service of VWM encoding, but the shift itself creates no strong demand to encode visual information into VWM. Unlike saccades, purely covert shifts of attention do not produce a disruption in perceptual input, and therefore introduce no demand to bridge perceptual disruption. Thus, we predicted that in the present study, with no incentive to encode the secondary object into VWM and a mode of orienting that does not itself introduce a demand for memory encoding, covert shifts of attention would not lead to object encoding, and there would be minimal interference with the primary memory task.

In sum, we stress a functional distinction between the memorial demands of different forms of orienting and argue that attention can be dissociated from VWM encoding, except under overt orienting conditions that themselves are likely to require memory. Experiments 1–3 examined the relationship between overt orienting and VWM encoding and also provided initial tests of the relationship between covert orienting and VWM encoding. Experiment 4 and 5 examined the latter relationship in more depth using detection and discrimination tasks to provide independent evidence that attention was indeed covertly directed to the secondary object.

Experiment 1

The principal aim was to test the hypothesis that saccade target objects are encoded into VWM, but objects attended under orienting conditions that do not entail memory encoding are not automatically encoded into VWM. The paradigm is illustrated in Figure 1. The primary task was color change detection: Participants saw a memory array of five colored disks, followed by a test array in which one disk might have changed color. During the retention interval of the memory task, we manipulated the circumstances under which attention was directed to a secondary object. In the Saccade block, a black, square secondary object (SO) was presented parafoveally during the retention interval (SO-present), and participants executed a saccade to this object. Saccade trials were intermixed with SO-absent trials on which no secondary object appeared, and participants simply maintained central fixation.

Figure 1.

Sequence of events in a trial of Experiment 1. The top row illustrates a trial in which the secondary object was absent. The middle row illustrates a SO-present trial in the Saccade block. The bottom row illustrates a SO-present trial in the Fixation block. The eye image shows participant’s gaze positon during the events.

In addition, we included a Fixation block. In this block, when a secondary object was present, participants continued to maintain central fixation. The object first appeared parafoveally and was then shifted to central fixation. As in the Saccade block, SO-present trials were intermixed with SO-absent trials. The Fixation block served two purposes. First, it presented the object under conditions in which it should have been attended covertly, but without the demand to generate a saccade. We can be confident that the secondary object was attended covertly, because it generated an abrupt onset (Franconeri, Hollingworth, & Simons, 2005; Yantis & Jonides, 1984), and it shifted dynamically from the parafoveal position to central fixation (Franconeri & Simons, 2003). Second, the Fixation block served as a control to ensure that effects of object presence in the Saccade block were due to saccade execution and not due to the retinal events generated by the secondary object. The presentation of the secondary object in the Fixation block simulated the retinal events generated in the Saccade block (parafoveal input followed by foveal input) but within a fixation.

The design was 2 (orienting block: saccade, fixation) X 2 (secondary object presence). We predicted that, in the Saccade block, secondary object presence would lead to a substantial decrement in memory performance, but this effect would be reduced or eliminated for a covertly attended object in the Fixation block. In contrast, the hypothesis that VWM is equivalent with attention holds that because attending an object is equivalent with representing this object in VWM, interference should be observed in both blocks, because the secondary object was attended under both types of orienting instructions.

Method

Participants

Twenty-two undergraduate students from the University of Iowa participated for course credit. All reported normal or corrected-to-normal vision. Two were eliminated for failure to perform above chance on the change detection task.

Stimuli

Stimuli were presented against a gray background with a central black fixation cross subtending 0.5°. The memory and test arrays consisted of five colored disks (4° diameter) placed evenly around central fixation at an eccentricity of 6.2°. A set size of five was used because this is at or above the storage capacity of the vast majority of undergraduate students (Vogel & Awh, 2008); consequently, there would be no spare capacity to encode the secondary object into VWM without displacing some of the information about the colored disks. Indeed, no participant’s memory performance approached ceiling in any condition of the experiment. The color of each memory disk was selected randomly without replacement from a set of nine colors: red, blue, green, yellow, fuchsia, brown, pink, orange, and cyan. On color-change trials, one randomly selected color from the memory array was replaced with a color selected randomly from the remaining four. The secondary object was a filled black square subtending 1.0°×1.0°. In the Saccade block, it was presented at an eccentricity of 2.8°, with the nearest contour 2.3° from central fixation. Note that the secondary object did not overlap spatially with the locations of the memory array stimuli. In the Fixation block, the object was initially presented at an eccentricity of 2.8° and was then moved to central fixation, obscuring the fixation cross. In both blocks, when the object appeared parafoveally, its left/right position was selected randomly on each trial. The secondary object was chosen to be highly distinct from the memory stimuli, so that it would not be miscategorized as part of the memory set: It was smaller, had a different shape, and was presented in black, which was not one of the colors used for the memory stimuli.

Apparatus

Stimuli were displayed on a 17-in. CRT monitor (120 Hz refresh rate). Eye position was monitored by SR Research Eyelink 1000 eyetracker sampling at 1000 Hz. A chin and forehead rest ensured a 70-cm viewing distance and minimized head movements. Responses were collected by a serial button box. The experiment was controlled by E-prime software (Schneider, Eschmann, & Zuccolotto, 2002).

Procedure

The primary task was color change detection. The experimenter initiated each trial as the participant fixated centrally. After 500-ms delay, the memory array was presented (300 ms), followed by a retention interval (2400 ms) and the test array. On half the trials, one color changed. Participant pressed one of two buttons to indicate “same” or “changed”.

The experimental conditions differed in the events occurring during the 2400-ms retention interval. For SO-absent control trials in both the Saccade and Fixation block, the retention interval consisted only of the central fixation cross. Participants were instructed to maintain central fixation throughout the retention interval.

For SO-present trials, the events during the retention interval differed between the Saccade and Fixation blocks. In the Saccade block, there was a 400 ms delay after the offset of the memory array. Then, the secondary object appeared parafoveally for 1500 ms. Participants were instructed to execute a saccade immediately to the object and to maintain fixation at that location until the appearance of the test array. The object was removed 500 ms before the appearance of the test array. Thus, the 2400 ms retention interval consisted of the following events: 400 ms delay; 1500 ms parafoveal secondary object presentation; 500 ms delay.

In the Fixation block, the 1500 ms presentation of the secondary object was divided into two events. The object appeared parafoveally for 300 ms and then centrally for 1200 ms. Participants maintained central fixation throughout the retention interval. The 300-ms duration for the parafoveal presentation was chosen on the basis of the average saccade latency in a pilot study with similar conditions. Thus, the timing of the retinal events in the Saccade and Fixation blocks was designed to be roughly the same.

Upon arriving for the experiment session, participants provided informed consent and received detailed instructions. The eyetracker was calibrated and was re-calibrated if the estimate of gaze position deviated by more than approximately 0.75° from the central fixation cross. Participants completed a Saccade block and a Fixation block, with block order counterbalanced across participants. Each block began with 12 practice trials, followed by 150 experiment trials: 100 trials on which the secondary object was present and 50 on which it was absent, randomly intermixed. Participants’ gaze was monitored during the retention interval to ensure that they followed the eye movement instructions. For the Saccade block, if the participant failed to execute a saccade to the object and maintain fixation at that location, a red error screen was displayed, and the trial was aborted and repeated. Similarly, for trials on which participants were instructed to maintain central fixation during the retention interval, if the eyes left a 1° diameter circular region surrounding the fixation cross, the trial was also aborted and repeated. On average, 35% of the trials were repeated. This is a fairly large percentage but is to be expected when using naïve participants who have little experience controlling gaze position.

Results

Figure 2 shows the proportion of correct responses on the change-detection task as a function of trial type. A 2 (Saccade block, Fixation block) X 2 (secondary object presence) repeated-measures ANOVA was conducted. There was a reliable main effect of object presence, F(1,19) = 8.4, p = .009, and a marginal effect of block type, F(1, 19) = 3.2, p = .09. Critically, these factors produced a significant interaction, F(1, 19) = 6.5, p = .02. For the Saccade block, there was a reliable decrement in memory performance in the SO-present condition (when participants executed a saccade to the object) compared with the SO-absent condition, F(1, 19) = 9.1, p = .007. However, in the Fixation block, there was no reliable difference in memory performance between the SO-present condition (in which the object was attended covertly) and the SO-absent condition, F < 1. A Bayes factor analysis (Akaike, 1974) indicated that the null hypothesis was 4.2 times more likely to account for the observed data than the hypothesis that change-detection performance differed between SO-present and SO-absent trials in the Fixation block.

Figure 2.

Mean proportion correct on the color change-detection task plotted as a function of block type and secondary object presence in Experiment 1. Error bars represent 95% confidence intervals based on the object presence effect in each block.

We also examined eye movement behavior during the retention interval. A saccade was defined as an eye movement velocity > 30°/s or acceleration > 8000°/s². When the secondary object was present in the Saccade block, mean saccade latency was 282 ms (SD = 65 ms). Thus, we were successful in approximately matching the amount of time that the object was visible parafoveally before it was fixated in the Saccade and Fixation blocks.

Finally, we conducted two control experiments to eliminate two alternative explanations of the Experiment 1 results. The first experiment confirmed that the memory decrement associated with saccade execution was not caused by a change in the retinal locations of the memory and test arrays. The second experiment confirmed that the effect was not due to prioritization of memory for colors near the saccade target location (i.e., a “retro-cue” effect). These results are described in the online supplemental materials.

Discussion

We predicted that overt shifts of attention would lead to automatic encoding of the saccade target object into VWM, given the central role for VWM in establishing target correspondence across saccades (Hollingworth et al., 2008b). However, purely covert shifts of attention to an object would not necessarily result in memory encoding, as they do not produce a disruption in perceptual input. The results supported these predictions. The presence of a secondary object led to substantial interference with a concurrent memory task when participants executed a saccade to it, but no interference was observed if participants merely attended it covertly. These results, indicating a dissociation between covert spatial attention and VWM encoding, challenge claims that VWM is equivalent with attention.

Experiment 2

In Experiment 2, we replicated the principal features of Experiment 1 and added a condition to eliminate an additional alternative explanation of the Experiment 1 results. We have assumed that the interference in the Saccade block of Experiment 1 was caused by the encoding of the saccade target into VWM. However, that interference could have been caused, instead, by the simple act of executing a saccade rather than by saccade target encoding per se. In Experiment 2, we added a block in which participants did or did not execute a saccade to empty space during the retention interval of the memory task (see Figure 3). This Saccade (no target) block was divided between saccade trials and no-saccade trials. On saccade trials, an arrow appeared at central fixation during the retention interval, cuing the participant to execute a saccade either to the left or right. On no-saccade trials, a double-headed arrow was presented, indicating that the participant should maintain central fixation. A lack of interference from saccade execution in this block would rule out mere saccade execution as an alternative explanation for the interference observed in the Saccade block of Experiment 1.

Figure 3.

Schematic illustration of the events in a Saccade and No Saccade trial of Saccade (no target) condition of Experiment 2.

We also made two smaller modifications to the paradigm. First, because two subjects in Experiment 1 performed near chance on the memory test, memory set size was reduced from five items to four. In addition, in the Fixation block, the secondary object remained at a parafoveal location throughout its 1500-ms presentation rather than moving to the center. Because the object appeared abruptly, it still should have attracted attention covertly during the retention interval.

Method

Participants

Eighteen undergraduate students from the University of Iowa participated for course credit. All reported normal or corrected-to-normal vision.

Stimuli and Apparatus

The stimuli and apparatus were the same as in Experiment 1, with the exception of the Saccade (no target) block. In that block, the arrow or a double-headed arrow was displayed in black and subtended 1.0° horizontally and 0.33° vertically. In addition, memory set size was reduced from five colors to four on all trials of the experiment.

Procedure

Experiment 2 had three blocks: Saccade, Fixation, and Saccade (no target). Three participants completed each of the six possible block orders. The Saccade block was the same as in Experiment 1. The Fixation block was also the same as in Experiment 1, except that the secondary object was presented at the parafoveal location for its entire 1500 ms duration. In the Saccade (no target) block, the fixation cross was replaced with a central arrow cue, 400 ms after the offset of the memory array. For 2/3 of trials (Saccade trials), participants executed a saccade to either the left or right side of the screen (evenly divided), indicated by the direction of the arrow. Participants were instructed to make a saccade to the location where the secondary object would have appeared had it been present and to maintain fixation on this location until the test array. All participants were shown examples of the secondary object locations so that even those participants who completed the Saccade (no target) block first knew the location to which they should make a saccade. For the remaining 1/3 of the trials (No-Saccade trials), a double-headed arrow cued participants to maintain central fixation throughout the trial.

For each block, participants first completed 12 practice trials followed by 90 experimental trials. As in Experiment 1, trials on which the participant did not follow the eye movement instructions were aborted and repeated (14% of trials).

Results and Discussion

Color change-detection data are reported in Figure 4. We first replicated the analysis from Experiment 1, comparing Saccade and Fixation blocks in a 2 (Saccade block, Fixation block) X 2 (secondary object presence) repeated-measures of ANOVA. There was a reliable main effect of block type, F (1, 17) = 9.7, p = .006, but no reliable effect of object presence, F(1, 17) = 2.2, p = .15. As in Experiment 1, there was an interaction between these variables, F(1, 17) = 4.3, p =. 05. For the Saccade block, there was a reliable decrement in memory performance in the SO-present condition compared with the SO-absent condition, F(1, 17) = 9.1, p = .008. However, in the Fixation block, memory performance did not reliably differ between the SO-present condition and SO-absent conditions, F < 1. A Bayes factor analysis indicated that the null hypothesis was 3.2 times more likely to account for the data than the hypothesis that change-detection performance differed between SO-present and SO-absent trials in the Fixation block.

Figure 4.

Mean proportion correct on the color change-detection task in Experiment 2. Error bars represent 95% confidence intervals based on the effects of secondary object presence in the Saccade and Fixation blocks, and on the effect of saccade execution in the Saccade (no target) block.

Importantly, saccade execution in the Saccade (no target) block did not influence change-detection performance. There was no reliable difference between memory accuracy on the saccade and no-saccade trials, F <1. Bayes factor analyses indicated that the null hypothesis was 4.3 times more likely to account for the data than the alternative hypothesis in the Saccade (no target) block. Note that memory accuracy was lower for trials in the Saccade (no target) block than for other conditions of the experiment in which no secondary object was present. This was likely caused by the need to process the perceptual features of the arrow and translate this information into a particular eye movement behavior. Thus, the data indicate that although processing the arrow cue may have interfered with VWM, the execution of the saccade itself generated no observable interference with memory.

We also examined eye movement behavior during the retention interval. When the secondary object was present in the Saccade block, mean saccade latency was 297 ms (SD = 69 ms). In the Saccade (no target) block, mean saccade latency based on the central cue was 276 ms (SD = 45 ms). Mean saccade amplitude was 2.27° in the Saccade block and 2.22° in the Saccade (no target) block. Thus, although there was a general tendency to undershoot the center of the secondary object (2.8° eccentricity), participants executed saccades of similar amplitude when the secondary object was absent as when it was present. Saccade amplitudes were significantly more variable in the Saccade (no target) block (SD = 1.7°) than in the Saccade block (SD = 0.9°), t(17) = 3.6, p=.002, consistent with the absence of a target object in the former condition. Moreover, on Saccade (no target) trials, participants made 1.3 additional saccades, on average, after their initial saccade directed to the empty screen location. Only a minute percentage of these were directed back to central fixation (0.3%). These additional saccades were likely to reflect corrections caused by variability in the initial saccade amplitude, but they clearly had no observable effect on VWM performance.

In sum, Experiment 2 replicated the results of Experiment 1. Overtly shifting gaze to a secondary object led to VWM interference, whereas covertly attending the object did not. In addition, we demonstrated that the former effect depends critically on the presence of a saccade target object: Saccades to empty space produced no observable interference with memory. The results are consistent with the hypothesis that the memorial demands of executing a saccade lead to automatic encoding of the target object into memory. In contrast, covert attention and VWM encoding can be dissociated.

Experiment 3

We have assumed that the secondary object is encoded in VWM prior to the execution of the eye movement to support the operation of establishing transsaccadic object correspondence. If so, then the interference observed in Experiments 1 and 2 should have resulted, primarily, from processes that occurred before the saccade. To provide a strong test of this hypothesis, in Experiment 3 we included trials on which participants executed a saccade to the secondary object, but it was removed during the saccade (SO-removed). If the object is encoded into VWM before the saccade is executed, then we should still observe interference with memory, even though the object was never fixated. The Fixation block was eliminated from the experiment. Participants completed one block of trials in which they always executed a saccade to the object when it appeared. There were three trial types: SO-present, SO-removed, and SO-absent.

Method

Participants

Twenty-two undergraduate students from the University of Iowa participated for course credit. All reported normal or corrected-to-normal vision.

Stimuli and Apparatus

The stimuli and apparatus were the same as in Experiment 2.

Procedure

The experiment replicated the two conditions in the Saccade block of Experiment 1 (SO-present and SO-absent) and added an SO-removed condition, in which the object was removed when the eyetracker detected that the eye crossed a virtual boundary, 1° from the central fixation. The screen change was initiated immediately and was completed in a maximum of 8.3 ms, well before the beginning of the next fixation. Participants completed 12 practice trials, followed by one block of 360 experimental trials (120 of each trial type, randomly intermixed). As in the previous experiments, trials on which participants did not follow the eye movement instructions were aborted and repeated. On average, 29% of the trials were repeated.

Results and Discussion

A one-way, repeated-measures ANOVA was conducted over the change-detection data (see Figure 5). There was a reliable effect of trial type, F(2,42) = 11.1, p < .001. Planned pairwise comparisons indicated that change-detection accuracy was higher on SO-absent trials than on either SO-present trials, F (1, 21) = 19.1, p< .001, or SO-removed trials, F(1, 21) = 14.1, p < .001. Critically, there was no observable difference in memory accuracy between the SO-present and SO-removed trials, F < 1. Bayes factor analyses indicated that the null hypothesis was 4.7 times more likely to account for the data than the alternative hypothesis that the SO-present and SO-removed conditions differed.

Figure 5.

Mean proportion correct on the color change-detection task in Experiment 3 as a function of block type. Error bars are 95% confidence intervals based on the effect of block type.

The results of Experiment 3 indicate that, when executing an eye movement to an object, the encoding of object properties into VWM is not strongly dependent on the events that occur after the saccade. Instead, the act of preparing and initiating a saccade is sufficient for memory encoding of the saccade target. This is consistent with the results of a recent study by Shao et al. (2010). The findings support our account of the functional relationship between attention and VWM encoding: The impending perceptual disruption creates a demand to encode the saccade target into VWM before the saccade is executed.

Experiment 4

Having obtained evidence consistent with the idea that saccade targets are automatically encoded into VWM, the purpose of Experiments 4 and 5 was to provide a stronger test of the claim that purely covert shifts of attention, not associated with saccade preparation, can be dissociated from VWM encoding. It is reasonable to assume that in Experiment 1, an object was attended if it appeared abruptly in the periphery and moved to central fixation. Thus, the absence of memory interference in the Fixation block of Experiment 1 provides substantial evidence that covert attention can be dissociated from memory encoding. However, the case would be stronger if we could verify that the secondary object was attended. Thus, in Experiments 4 and 5, we used detection and discrimination tasks to ensure that attention was directed to the object during the retention interval of the memory task.

The design of Experiment 4 is illustrated in Figure 6. In all conditions, participants maintained central fixation throughout the retention interval. The secondary object was a black outline square instead of a filled square. In one block of trials (Ignore block), participants were instructed to ignore the black square, as it was task irrelevant. In a second block of trials (Attend block), participants were instructed to attend the black square, as it cued the probable location of a briefly presented dot that appeared during the retention interval on some trials. We could ensure that the black square was attended if there was a substantially higher probability of dot detection when the dot appeared at the cued square location (73.3% probability) versus at an uncued location (26.7% probability). The critical memory data then came from the Attend block trials on which no dot appeared. We could be confident that the secondary object was attended on these trials, but memory performance was not confounded by the need to detect and respond to an additional stimulus (the dot).

Figure 6.

Sequence of events in a trial of Experiment 4. The top row illustrates an SO-absent trial. The middle row illustrates a trial in which the secondary object was present but no target dot appeared. The bottom row illustrates a SO-present trial on which a target dot appeared. When a target dot appeared, it was presented at the object location on 73.3% of trials (valid) and on the opposite side of the screen on 26.7% of trials (invalid).

The design was 2 (Attend block, Ignore block) X 2 (secondary object presence), with a subset of SO-present trials in the Attend block also containing the dot target (eliminated from the analysis of memory performance). We predicted that there would be no reliable effect of object presence on memory performance in either the Ignore block or the Attend block.

Method

Participants

Eighteen undergraduate students from the University of Iowa participated for course credit. All reported normal or corrected-to-normal vision. One was eliminated for failure to report the dot onset on any trial.

Stimuli

The memory set size was the same as in Experiment 1 (five). The color disks each subtended 3.4° at an eccentricity of 5.3°. The secondary object was an outline square (0.06° contour width) subtending 1.1°×1.1° at an eccentricity of 2.4°. The onset dot (0.1°) was presented in a gray that was of slightly higher luminance than the background. It appeared either within the square (valid) or at the corresponding positon in the opposite hemifield (invalid).

The apparatus was the same as in Experiment 1, with the following exceptions. In Experiments 4 and 5, the eyes were not monitored using an eye tracker. The experimenter monitored a video image of the eye on every trial, manually recorded trials on which an eye movement occurred, and reminded the participant to maintain central fixation in this event.

Procedure

Participants completed two blocks, Attend and Ignore, each with a total of 220 trials. Block order was counterbalanced across participants. In the Ignore block, trials were divided evenly between SO-present and SO-absent. In the Attend block, 160 trials were divided evenly between SO-present and SO-absent. On the remaining 60 trials, the secondary object was present, and in addition, a target dot appeared either within the square (44 trials, 73.3%) or on the opposite side of the screen equidistant from central fixation (16 trials, 26.7%).

For the Attend block, participants were instructed to fixate the central cross throughout the trial and respond, immediately via button press, if a dot appeared. They were informed that the dot was more likely to appear within the square than at the opposite location and thus that the best strategy would be to attend to the square. For the Ignore block, participants were instructed to fixate the central cross and ignore the appearance of the square.

On SO-absent trails, the entire 2400-ms retention interval consisted of a central fixation cross. On SO-present trials, the square appeared 400 ms after the offset of the memory array and remained visible for 1500 ms. On dot-present trials, the dot appeared 300 ms after the onset of the square and remained visible for 250 ms.

Participants first completed 12 practice trials implementing only the color change detection task. Then, they completed the two experimental blocks. Each was preceded by eight practice trials implementing the complete set of events on a trial. Trials on which participants moved their eyes away from central fixation were eliminated from the analyses (1%).

Results and Discussion

In the Attend block, participants were significantly better at detecting the low-contrast dot when it appeared in the square (M = .94) than when it appeared on the opposite side of the screen (M = .86), t(16) = 2.7, p = .018. The false alarm rate for the dot-absent trials was .02. Mean correct RT was 307 ms on valid trials and 323 ms on invalid trials, which was not a significant difference, t(16) = 0.88, p = .39. The substantial cuing effect provides independent evidence that participants did indeed direct attention covertly to the secondary object during the retention interval.

For the main analyses, we excluded trials in the Attend block on which a dot appeared and on which participants falsely reported a dot when none appeared.¹ Change detection performance is reported in Figure 7. A repeated-measures ANOVA indicated that neither the main effect of block nor the main effect of secondary object presence was significant, Fs < 1, nor was the interaction, F < 1. Bayes factor analyses indicated that the null hypothesis was 1.5 and 3.7 times more likely to account for the data than the alternative hypothesis for the Attend and Ignore blocks, respectively. Note that in the Attend block, memory performance was actually slightly (but not significantly) higher in the SO-present condition than in the SO-absent condition, which is the opposite of the expected pattern if the shift of attention interfered with memory.

Figure 7.

Mean proportion correct on the color change-detection task plotted as a function of block type and secondary object presence in Experiment 4. Error bars represent 95% confidence intervals based on the effect of object presence in each block.

Experiment 5

As a converging method to assess the locus of attention during the retention interval, in Experiment 5 we replaced the dot detection task with a masked discrimination task, a well validated paradigm for measuring the locus of spatial attention (e.g., Yeshurun & Carrasco, 1999). On a subset of SO-present trials, a small upright or inverted “T” appeared briefly within the object or on the opposite side of the screen, followed by a mask. Discrimination accuracy provided independent confirmation that the secondary object was attended.

The method in Experiment 5 differed from Experiment 4 in two additional ways. First, the secondary object was enlarged and was rendered so that it appeared to be extending toward the observer in depth (see Figure 8). This was done to maximize the perception of a discrete object. Second, when the “T” appeared in the Attend block, the discrimination response ended the trial; there was no color memory test on these trials. This ensured that participants placed sufficient priority on accurately reporting the “T” orientation.

Figure 8.

Illustration of the retention interval events for trials on which a masked discrimination target (“T” or inverted “T”) was present in Experiment 5. The top row shows a valid trial (73.3% of target-present trials). The bottom row shows and invalid trial (26.7% of target-present trials)

Participants

Eighteen undergraduate students from the University of Iowa participated for course credit. All reported normal or corrected-to-normal vision.

Stimuli

The stimuli were the same as in Experiment 4, with the following exceptions. The secondary object was enlarged to 1.3°×1.3°. It was rendered in a 3D modeling program as a four-sided pyramid with a flat top. The top area consisted of an outline square with a uniform gray center. The target object was a white “T” or inverted “T”, subtending 0.6°×0.7°. The mask was composed of jumbled white contours against a black background. It fit within the black outline square at the center of the secondary object, subtending 1°×1°.

Procedure

The distribution of trial types was the same as in Experiment 4. On the trials when the “T” target was presented, its orientation was randomly determined. It was presented for 50 ms, followed by the mask until participants made an unspeeded button response to indicate “upright” or “inverted”. The response terminated the trial (there was no memory test on these trials).

Participants first completed 20 trials of practice on the masked discrimination task alone. The practice session was repeated until the participant achieved 80% accuracy for valid trials. No participant required more than two practice blocks. In the experimental session, participants completed two blocks (Attend and Ignore) of 220 trials each. Each block was preceded by eight practice trials implementing the complete set of events on a trial.

Results and Discussion

Discrimination accuracy for the “T” orientation task was reliably higher on valid trials when the target appeared at the secondary object location (M = .79) than on invalid trials when it appeared on the opposite side of the screen (M = .72), t(17) = 2.2, p = .04. Thus, we can be confident that participants did indeed selectively attend to the object during the retention interval. Note that this difference was observed despite the potential for crowding at the secondary object location, with the contours of the central square region of the secondary object appearing only 0.3° from the nearest contour of the target “T”.

Trials on which participants made an eye movement away from central fixation were eliminated (2%). Obviously, the SO-present trials with a “T” target in the Attend block did not contribute to the analysis of memory performance, as no memory test was administered. The memory accuracy data are displayed in Figure 9. A 2 (Attend block, Ignore block) X 2 (secondary object presence) repeated-measures ANOVA was conducted. We found a main effect of block, F(1, 17) = 14.9, p = .001. Overall, change-detection performance was lower in the Attend block (M = .69) compared with the Ignore block (M = .73). However, there was no main effect of object presence, F < 1, nor an interaction, F(1, 17) = 2.6, p = .13. Bayes factor analyses indicated that the null hypothesis was 3.5 and 3.9 times more likely to account for the data than the alternative hypothesis in the Attend and Ignore blocks, respectively. The lower performance in the Attend block suggests that trials associated with the difficult discrimination task were subject to some degree of dual-task interference. Critically, however, this interference was not specific to trials in the Attend block on which the secondary object was present; equivalent memory performance was observed for SO-present and SO-absent trials. Thus, whatever the source of interference, it cannot be attributed to VWM encoding of the secondary object.

Figure 9.

Mean proportion correct on the color change-detection task plotted as a function of block type and secondary object presence in Experiment 5. Error bars represent 95% confidence intervals based on the effect of object presence in each block.

The results from the discrimination task indicate that participants directed spatial attention to the secondary object location during the retention interval of the Attend block. The finding that this attention shift had no negative effect on memory accuracy relative to the SO-absent condition provides strong support for the claim that covert attention can be dissociated from VWM encoding. One issue raised in the comparison of covert and overt attention effects on VWM encoding is whether purely covert shifts involve a similar deployment of attentional “resources” as the shift of attention preceding a saccade. This was examined in a study by Hoffman and Subramaniam (1995a). Using the same target discrimination task, they compared the effect of covert attention in a cuing paradigm with the effect of saccade preparation. The covert cuing benefit (valid minus invalid) was 12.4 percentage points. The discrimination benefit at the saccade-target versus non-saccade-target location was 14.2 percentage points. The authors did not conduct an analysis comparing the magnitude of these effects. Thus, although we cannot say with complete confidence that the strength of the attention shift is equivalent in these two cases, covert shifts are clearly sufficient to generate large differences in discrimination accuracy, and the magnitude of the effect is similar to that generated by the shift of attention before a saccade.

Omnibus Analyses

The reliable effects of distraction in the saccade conditions of Experiments 1–3 demonstrate that the presentation of the secondary object was sufficient to generate robust interference with memory under some orienting conditions. However, our experiments probing purely covert shifts of attention depend on drawing an inference from null effects. To improve the strength of that inference, we conducted omnibus analyses combining the data from multiple experiments. First, we combined the data from conditions in the four experiments in which the object was likely to have been attended covertly (as an abrupt, singleton onset) but with no explicit demand to monitor it (Fixation blocks of Experiments 1 and 2 and Ignore blocks of Experiments 4 and 5). SO-present trials were compared with SO-absent trials, and Experiment was a between-subjects factor. There was no reliable effect of object presence, F < 1, with mean color memory accuracy of .74 for SO-present and .73 for SO-absent. Bayes factor analyses indicated that the null hypothesis was 7.7 times more likely to account for the data than the alternative hypothesis. Second, we combined the data from the two experiments on which participants covertly monitored the secondary object for the appearance of a target (Attend blocks of Experiments 4 and 5). There was no reliable effect of object presence, F < 1, with mean color memory accuracy of .70 for SO-present and .69 for SO-absent. Bayes factor analyses indicated that the null hypothesis was 4.3 times more likely to account for the data than the alternative hypothesis. Thus, we can claim with substantial confidence that the secondary object produced no memory interference under conditions of purely covert orienting.

General Discussion

The present study investigated the relationship between spatial attention and VWM encoding. In all experiments, participants performed a color change detection task. The key manipulations concerned the presence of a secondary object during the retention interval and the conditions under which it was attended. When that object was a saccade target, it reliably interfered with the performance of the memory task (Experiments 1–3), suggesting that it was encoded into VWM. This effect was not caused merely by saccade execution (Experiment 2) and was driven by events that occurred before the saccade was completed (Experiment 3). The results are consistent with the idea that saccade targets are encoded automatically into VWM, so as to establish object correspondence across the saccade (Hollingworth et al., 2008b). In contrast, when the secondary object was merely attended covertly, and no saccade was executed, it did not interfere with memory (Experiments 1 and 2), even when independent detection and discrimination tasks confirmed that it was focally attended (Experiments 4 and 5). Thus, it is possible to attend to an object without encoding it into VWM. Spatial attention and VWM encoding can be dissociated when the selective demands on the two systems diverge.

Implications for Understanding the General Relationship between Attention and VWM

The idea that attention and VWM are equivalent first gained prominence with Cowan’s paper on the “magical number 4” (Cowan, 2001), in which he argued that similar capacity limits in working memory tasks (e.g., the 3–4 object limit in visual change detection, Luck & Vogel, 1997) and attention tasks (e.g., the 4–5 item limit in multiple object tracking, Pylyshyn & Storm, 1988) indicate that they depend on a common system that implements selective processing over perceptual representations and over memory representations. The idea was further developed within the context of change blindness research in Rensink’s “coherence theory” (Rensink, 2002). Change detection was proposed to be limited to the locus of attention within the scene, because VWM is limited to currently attended objects, which disintegrate into constituent features upon the withdrawal of attention. Key support for this idea came from Wheeler and Treisman (2002), who claimed that, just as spatial attention is necessary for feature binding in perception; spatial attention is required to maintain feature bindings in VWM. Finally, the finding that control of VWM encoding and maintenance is sensitive to spatial cuing manipulations (Griffin & Nobre, 2003a; Schmidt et al., 2002) provided additional evidence that the two constructs are equivalent, with VWM encoding simply reflecting attentional selection, and VWM maintenance reflecting the allocation of attention to representations of previously visible stimuli (Jiang & Song, 2005; Kiyonaga & Egner, 2013; Wang & Spelke, 2002).

However, throughout this accretion of evidence there has been strong counterevidence indicating that the relationship between attention and VWM is not one of simple unity, that the relationship is strongly dependent on the demands placed on perceptual and memorial selection, and that the two systems exhibit substantial independence when those demands diverge. Broad overlap between attentional and VWM systems has been tested in paradigms combining VWM tasks with visual search tasks. If attention and VWM constitute a single system, then substantial interference should be observed when the need to remember one set of objects conflicts with the need to attend to a different set of objects and locations during search. Yet, VWM maintenance generates minimal interference with the efficiency of attentional selection during visual search (Baizer et al., 1991; Woodman et al., 2001). When interference has been observed, the source has been the memorial demands of the search task, such as the need to maintain a different target template on each trial and the need to remember previously attended locations (Castel, Pratt, & Craik, 2003; Woodman & Luck, 2004; Woodman, Luck, & Schall, 2007). Thus, these broad tests are consistent with dissociable mechanisms rather than a unitary system.

With respect to the idea that coherent VWM representations depend on sustained attention (Rensink, 2002), it is now clear that visual memory representations maintain coherence after the withdrawal of attention (Block, 1980; Hollingworth, 2004; Hollingworth & Henderson, 2002). On the issue of the role of spatial attention in feature binding, the initial evidence reported by Wheeler and Treisman (2002) was somewhat indirect and has not been confirmed. Subsequent studies have found that secondary tasks engaging spatial attention produce no specific decrement in binding memory, with robust memory for binding in the absence of sustained visual attention (Delvenne, Cleeremans, & Laloyaux, 2010; Gajewski & Brockmole, 2006; Johnson et al., 2008; Shen, Huang, & Gao, 2015; Wijdenes, Marshall, & Bays, 2015). Complementary evidence has been observed in studies focusing on executive attention (Allen, Baddeley, & Hitch, 2006; Morey & Bieler, 2013; Von Grünau & Dubé, 1992; Wolf & Schütz, 2015). Recently, Snowden and Milne (1997) showed that several object-based attention tasks selectively impair binding in VWM, but this is to be expected given that their tasks, such as mental rotation and delayed feature report, directly depend on VWM themselves (e.g., Hyun & Luck, 2007).

There are two exceptions to the general finding that spatial attention manipulations do not selectively impair memory for feature binding. Squire (1986) found selective interference with binding memory, but their attention task, multiple object tracking, is known to place strong demands on VWM (Teng & Squire, 1999), making it difficult to isolate the source of interference. In a task closely related to that of Johnson et al. (2008), Zokaei, Heider, and Husain (2014) found that the probability of binding errors in memory for conjunction stimuli increased with the presence and difficulty of an intervening visual search task. However, a difficult search task also tended to reduce the precision of the memory representations, increasing the probability of an accidental match between the reported value and the value of a different item in the display. That is, report of the wrong feature value may have reflected imprecise memory rather than a binding error. Given these concerns, and given consistent results from the studies reviewed in the previous paragraph, the balance of evidence indicates that feature binding in VWM does not depend centrally on sustained visual attention.

Recent research has focused on control mechanisms of VWM, equating VWM encoding and selective maintenance with the allocation of visual attention (for a review, see Jiang & Song, 2005). In the literature on retro-cuing and selective maintenance, participants engage in a standard VWM change-detection task. During the retention interval, a spatial cue indicates the location (now unoccupied) of the item that will be tested. Memory performance is more accurate in this valid cue condition compared with a baseline, neutral cue condition. In the standard account of the effect (Griffin & Nobre, 2003a), the spatial cue allows attention to be sustained on the representation of the object appearing at that location, and sustained attention either enhances the persisting perceptual code (in the same manner that attention enhances initial perceptual processing at attended locations) or protects the item from decay and interference (Makovski, Sussman, & Jiang, 2008; Matsukura, Luck, & Vecera, 2007). However, there is an unconfirmed assumption at the heart of this inference. The cue could certainly be used to direct visual attention to a particular item or location. But it also provides information (the probable test item) that could be used by the participant to selectively retain that item in some other manner. Missing is direct evidence that the cue led to a shift of attention and that it was visual attention to the cued item that was functional in modulating memory.

To provide this test, Hollingworth and Maxcey-Richard (2013) implemented the basic retro-cuing paradigm but manipulated whether the cue was followed by a demanding visual search task before memory was tested. The visual search task precluded sustained attention on the location of the cued item. Yet, its addition introduced no decrement in the magnitude of the cuing benefit, indicating that sustained attention was not required for selective maintenance in VWM. This result was replicated in a similar design by Prime, Niemeier, and Crawford (2006). Rerko et al. also added a key test, in which the manipulation of attention after the retro-cue was a shift of attention within VWM rather than over a representation of currently visible stimuli. This type of attention task also had no effect on the magnitude of the cuing benefit.² In these studies, it is possible that attention acted early, before the secondary task, to modulate memory, and that this state of memory persisted despite later withdrawal of attention. But the key point is that preferential retention in VWM is not equivalent with the locus of attention, as selective maintenance survived events that prevented sustained attention on the cued item. Selection in VWM is not simply visual attention directed to memory representations. This is consistent with electrophysiological and neuroimaging data showing differences as well as similarities between the neural correlates of the orienting of attention to sensory inputs versus VWM representations (Ariely, 2008; Griffin & Nobre, 2003b).

In the present study, we demonstrated that directing visual attention to perceptual representations is not equivalent to encoding those representations in VWM. Although attention certainly facilitates VWM encoding of task-relevant objects (Schmidt et al., 2002), it is possible to attend to a perceptual object without encoding that object into VWM. If the attended object is not task-relevant, and if the act of orienting does not itself introduce a demand to encode the item into VWM, the item does not appear to gain access to VWM. Critically, our tests were conducted in conjunction with independent measures of the allocation of visual attention (Experiments 4 and 5). Despite perceptual facilitation at the secondary object location, the object produced no memory interference.

These results contrast with those of Olson et al. (2008), who found that fixated objects cued as irrelevant were nonetheless remembered and were sometimes confused for targets on the memory test. As discussed in Introduction, however, the to-be-ignored items were not actually task irrelevant in their experiment, because remembering a distractor as a distractor would have allowed accurate report of “no match” if that item appeared during the test. In contrast, the secondary object in the present experiments was entirely irrelevant to the memory task; encoding it could only impair performance. Moreover, the object was designed to be dissimilar from the memoranda. It was smaller, it had a different shape, and its color was not part of the memory set. The purpose of these differences was to ensure that the secondary object would not be miscategorized as part of the memory set, either during initial presentation or later, during retention, if it happened to be encoded. The secondary object reliably interfered with memory when it was a saccade target, so we can be confident that the object stimuli were sufficient to generate memory interference had they been encoded under conditions of purely covert orienting. In sum, given these extensive controls and given independent confirmation that we did indeed engage attention at the secondary object location, the present data provide strong evidence that VWM encoding can be dissociated from the locus of attention.

Our discussion has thus far focused on VWM and spatial attention. But we have yet to consider other forms of attention (feature-based attention) and other forms of working memory (SpWM). Current evidence indicates much greater overlap between VWM and feature-based attention than VWM and spatial attention, due to common representational content. Indeed, some overlap between feature memory and feature-based attention is required. Feature-based attention involves biasing selection in favor items that match a particular known feature value. That feature value must be retrieved from memory or remembered from a visible sample, making memory central to the selective operation. Critically, there is strong evidence that VWM content automatically implements feature-based selection. In the memory-based capture effect (Olivers, Meijer, & Theeuwes, 2006; Soto, Heinke, Humphreys, & Blanco, 2005), participants remember a particular feature value for a later memory test. During the retention interval, they search for a target defined on a different dimension. The presence of a memory matching distractor reliably interferes with visual search, indicating that it captured attention, despite the fact that participants knew a matching item would never be the target. Although the demands of memory maintenance and search guidance diverge in this paradigm, attention is nonetheless directed to memory matching items, suggesting dependence on a common set of processes.

It is important to note, however, that the overlap between VWM and feature-based attention is not complete. Several studies have now indicated that only a subset of items in VWM interacts with perceptual selection (Baizer et al., 1991; Downing & Dodds, 2004; Houtkamp & Roelfsema, 2006). Baizer et al. (1991) found that a color de-prioritized for retention in VWM failed to interact with perceptual selection, even on trials when the color was nevertheless remembered accurately (see also van Moorselaar, Theeuwes, & Olivers, 2014). Irons, Folk, and Remington (2011) discuss this as a distinction between an active template item, which interacts with perceptual selection, and accessory items, which do not. There is debate over whether the subset of items that interacts with perceptual selection is limited to one (Irons et al., 2011; van Zoest, Donk, & Van der Stigchel, 2012) or spans multiple items (Vickery, Sussman, & Jiang, 2010), but it is clear that not all objects maintained in VWM guide attention. It is also clear that feature-based selection can be implemented without the direct involvement of VWM. Long-term learning has the capability to implement feature-based attention, as observed in the literature on reward-based capture (e.g., Alais, Morrone, & Burr, 2006; Kravitz & Behrmann, 2011) and in studies that repeat a particular target feature over many consecutive trials (Zelinsky & Schmidt, 2009). Thus, although there is overlap between VWM and feature-based attention, VWM maintenance of a particular feature is neither sufficient nor necessary to implement feature-based perceptual selection.

Accounts of the relationship between attention and working memory have often focused on the role of spatial attention in the maintenance of locations in SpWM (for a review, see Awh, Armstrong, & Moore, 2006). Again, there is substantial overlap in representational content between these systems, and mechanisms that selectively index spatial positions (such as spatial attention, overt gaze, or even, presumably, manual pointing) will clearly play an important role in establishing a memory representation of locations. But is the act of directing spatial attention to a location equivalent with maintaining that location in SpWM, as implied by the claim that they depend on a common selective mechanism (Cowan, 2001)? The critical data comes from studies that have placed the demands of SpWM and visual attention in conflict. However, the results have been mixed. In the original test by Awh, Jonides, and Reuter-Lorenz (1998, Experiment 3), spatial memory performance was probed in an interaction between 1) the presence of a secondary task and 2) the need to shift attention in order to perform that task. When the secondary task required a shift of spatial attention, this placed the spatial selective demands of the secondary task in conflict with those of SpWM maintenance. However, the interaction only approached statistical significance in a one-tailed test. This design has been replicated only once, to our knowledge, and in that replication (Mathôt & Theeuwes, 2013, Experiment 4), there was no reduction in memory accuracy when spatial attention was engaged by the secondary task during retention. Thus, it is currently unclear whether sustained spatial attention is necessary for the maintenance of locations in SpWM. Resolving this issue will require empirical clarification of the results obtained from the Awh et al. (1998) paradigm.

In sum, there is no doubt that selective mechanisms in vision and VWM often serve a common function, that they are coordinated, and that in some cases (particularly feature-based VWM and feature-based attention) they may be substantially overlapping. However, it is also the case that a strong account of this relationship—that attention and working memory are two different names for the same mechanism—is not tenable. When the demands on the two systems diverge, they can be dissociated.

Implications for Understanding the Relationship between VWM and Eye Movements

In the present experiments, we found that, in contrast with purely covert orienting, an eye movement to a secondary object led to automatic encoding of the object into VWM. A mandatory relationship between saccade execution and VWM encoding is consistent with object-based theories of transsaccadic perception and visual stability (Currie et al., 2000b; Deubel, Schneider, & Bridgeman, 1996; Irwin et al., 1994), which have largely supplanted earlier, image-based theories. Image-based theories held that stability across saccades is achieved by the global integration of sensory information, with an efference copy of the saccade motor command used to shift the pre-saccadic image representation and align it spatially with post-saccadic sensory input (Jonides, Irwin, & Yantis, 1982; McConkie & Rayner, 1976). Subsequent empirical work demonstrated that global, image-based integration does not occur across saccades (Bridgeman & Mayer, 1983; Irwin, 1991; Irwin, Yantis, & Jonides, 1983; O’Regan & Lévy-Schoen, 1983), that the perceptual information retained across a saccade is highly limited in capacity (Irwin, 1991), that it exhibits properties consistent with VWM retention (Hollingworth et al., 2008b; Irwin, 1992a; Irwin & Andrews, 1996), and that it is strongly biased toward objects at or near the impending saccade target location (Currie et al., 2000b; Irwin, 1992a; McConkie & Currie, 1996). Thus, instead of global, image-based integration, visual stability appears to depend on a more local solution. Before the saccade, spatial attention shifts to the saccade target object (Deubel & Schneider, 1996; Hoffman & Subramaniam, 1995b; Kowler et al., 1995), enhancing the perceptual processing of the target (Moore & Fallah, 2004) and facilitating the encoding of saccade target properties into VWM. When the eyes land, this target representation is used to confirm that eyes have acquired the appropriate object (Hollingworth et al., 2008b; Richard, Luck, & Hollingworth, 2008) and thereby establish continuity (Fodor, 1987; Zehetleitner, Krummenacher, & Muller, 2009).

Note that the neurophysiological literature on perisaccadic “remapping” of visual receptive fields has followed a similar development. Early evidence suggested that a global remapping of receptive fields could account, in an image-based manner, for the retinal displacement caused by the saccade (e.g., Duhamel, Colby, & Goldberg, 1992). However, recent work has indicated that receptive field shifts do not necessarily follow a global remapping pattern. Rather, receptive fields converge in all directions toward the saccade target location (Zimmermann, Morrone, & Burr, 2014), increasing the density of perceptual sampling at that location before the saccade. This provides a plausible neural instantiation of the pre-saccadic shift of attention to the saccade target and points toward a local, object-based solution to transsaccadic stability rather than a global, image-based solution.

If, with each saccade, saccade target properties are encoded into VWM to support object continuity and visual stability, then it follows that this memory encoding will be highly automatized, as in the present experiments (for an extended discusssion, see Hollingworth et al., 2008a). That is, the execution of the saccade to the secondary object in our experiments placed the same demand on memory as every one of the other 15,000+ saccades executed each day. It appears that participants cannot decide that, for a particular saccade, they will refrain from encoding saccade target properties.

Conclusion

We identified an important distinction between covert shifts of attention, not associated with saccade preparation, and the selective events that immediately precede the execution of a saccade. Covert attention does not necessarily produce automatic encoding of the attended object into VWM, favoring a view in which attention and VWM constitute separable systems rather than a common mechanism. However, saccade targets were automatically encoded into VWM, reflecting the demand to bridge transsaccadic perceptual disruption and establish object correspondence. In general, the results suggest that the relationship between attention and VWM is strongly dependent on the memorial demands of the orienting behavior.