Spatial context learning survives interference from working memory load

Abstract

The human visual system is constantly confronted with an overwhelming amount of information, only a subset of which can be processed in complete detail. Attention and implicit learning are two important mechanisms that optimize vision. This study addresses the relationship between these two mechanisms. Specifically we ask: Is implicit learning of spatial context affected by the amount of working memory load devoted to an irrelevant task? We tested observers in visual search tasks where search displays occasionally repeated. Observers became faster searching repeated displays than unrepeated ones, showing contextual cueing. We found that the size of contextual cueing was unaffected by whether observers learned repeated displays under unitary attention or when their attention was divided using working memory manipulations. These results held when working memory was loaded by colors, dot patterns, individual dot locations, or multiple potential targets. We conclude that spatial context learning is robust to interference from manipulations that limit the availability of attention and working memory.

Introduction

Images of natural scenes are staggeringly complex, and in many respects our ability to extract and remember visual detail is extremely limited (Levin & Simons, 1997; Rensink, O'Regan, & Clark, 2000; Simons & Chabris, 1999). Nevertheless, humans are capable of coping with this computational difficulty in the real world and navigate their surroundings without major problems. Two mechanisms that are essential to overcoming the complexity of visual input are implicit learning and visual attention. Implicit learning allows us to extract regularities in the visual environment. It reduces the chaos in the input and tunes the sensitivity of the visual system to previously learned structure (Chun & Jiang, 1998; Gibson, 1966). Unlike explicit learning, no attention to the presence of regularities, and no awareness thereof, is requisite for implicit learning to occur. Visual attention prioritizes processing to a subset of visual input (Pashler, 1998). The importance of these two mechanisms naturally leads us to ask: what is the relationship between the two? Is implicit learning impaired when attention is drawn to an unrelated task?

Most researchers agree that visual attention modulates conscious perception and explicit learning and memory (Mack & Rock, 1998). When attention is diverted by a secondary task, such as counting basketball passes, conscious perception and memory of a salient object (such as a gorilla passing through the scene) are impaired (Simons & Chabris, 1999). Whether attention also modulates implicit memory, however, remains controversial. This problem is exacerbated by the multifaceted nature of implicit processes and multiple senses of the term “attention.” Implicit learning, for example, has been tested in vastly different paradigms, including artificial grammar learning (Reber, 1993), the serial reaction task (Nissen & Bullemer, 1987), visual statistical learning (Fiser & Aslin, 2001; Turk-Browne, Junge, & Scholl, 2005), and spatial context learning (Chun & Jiang, 1998). The computational problems involved in different paradigms are different, as are the mechanisms used to solve them. For example, artificial grammar learning, the serial reaction task, and visual statistical learning usually involve learning a sequence of single items, whereas spatial context learning entails learning a complex visual array. The difference in spatial complexity and in task requirements makes it likely that attention is involved in different ways across different paradigms (Jimenez, 2003; Stadler & Frensch, 1998). Findings obtained from one implicit learning paradigm may not readily transfer to another.

The current study addresses the relationship between attention and contextual cueing, a form of implicit spatial context learning. We focus on contextual cueing because it is tightly related to visual search, an attention demanding and ecologically important activity (Wolfe, 1998). In the typical procedure for contextual cueing, participants search for a T-shaped item among L-shaped items. Unknown to the participants, some search displays occasionally repeat in the experiment. Participants become faster at locating the target in repeated displays than unrepeated ones (Chun & Jiang, 1998, 2003), implying that the layout of repeated displays is learned, which facilitates faster search. Both the learning itself and the memory supporting contextual cueing is implicit: when queried, participants report no realization that repetitions of layout are occurring, and when forced to choose the more familiar of two layouts (repeated vs. novel), they fail to choose repeated layouts at above-chance rates. .

Like perceptual learning (Goldstone, 1998) and priming, contextual cueing is an increased sensitivity or heightened response to a familiar stimulus. Unlike either perceptual learning or priming, however, contextual cueing is associative in nature, relating the organization of a visual scene to a target location – relocating a target in a cued scene destroys the contextual cueing benefit (Chun & Jiang, 1998). Further, it shows greater generalization to changes in constituent elements than would be expected of perceptual learning or priming, transferring to scenes in which the stimuli are changed in identity, as long as the arrangements of targets and distracters remains the same (Chun & Jiang, 1998). Unlike priming, contextual cueing is long-lasting, persisting for at least 1 week (Chun & Jiang, 2003).

Although contextual cueing is implicit, it is modulated by selective attention (Jiang & Chun, 2001). For example, when participants are presented with a search display of black and white items and must search for a black target, their attention tends to be restricted to the black elements (Kaptein, Theeuwes, & Van der Heijdt, 1995). If these items repeat their locations, search is facilitated to the same degree as when all items repeat. But if the white distracters (the ones rejected preattentively) repeat their locations, search is not facilitated, suggesting that contextual cueing is driven primarily by the attended context (Jiang & Chun, 2001; Jiang & Leung, 2005).

The aim of the current research is to test the hypothesis that contextual cueing is sensitive not only to selective attention, but also to divided attention. The term “attention” is used in many different ways (Parasuraman & Davies, 1984; Pashler, 1998). Two common usages refer to its selectivity and its limited resources. Selective attention prioritizes a subset of input over the rest, while divided attention allocates processing time and resources to distinct tasks.

The different usages of the term “attention” can lead to discrepancies in addressing the relationship between visual implicit learning and attention. In the serial reaction task, for example, selective attention and divided attention have different effects. Increasing the demand for selective attention by embedding the target among distracters does not reduce learning (Rowland & Shanks, 2006), suggesting that learning is robust to increased selection demands. Dividing attention between the serial reaction task and a secondary load task, however, often reduces learning (Jimenez, 2003). These findings suggest that even within a single implicit learning paradigm, one may find discrepant results for effects of attention depending on which aspect of attention is manipulated (see also Turk-Browne et al., 2005, for an example in visual statistical learning). Whether the strength of implicit learning depends on the strength of attention is therefore an empirical question that must be addressed for different learning paradigms and for different usages of attention.

In the present study, we specifically ask whether contextual cueing is reduced or eliminated by secondary tasks. Our choice of secondary tasks is guided by a long and controversial literature on divided attention. Since Kahneman's proposal that attention is limited by general capacity (Kahneman, 1973), researchers have debated about whether attention consists of a single pool of resources (Navon & Gopher, 1979), multiple pools of resources (Wickens, 1981), or whether the concept of resources is at all useful (Navon, 1984). Despite this controversy, divided attention paradigms have been highly successful at delineating components of the mind, such as different components of working memory (Baddeley, 1986). They have also been used in recent years to characterize attentional dependency of various visual tasks (Li, VanRullen, Koch, & Perona, 2002). The divided attention literature also makes several general observations that allow researchers to predict whether certain tasks may interfere with each other (Pashler, 1998) Generally, tasks tend to interfere with each other if they share common processes, resulting in cross-talk or competition for common resources. Even tasks that are very different can compete with each other for access to the central bottleneck.

We selected secondary tasks that, a priori, may interfere with contextual cueing. These tasks all place significant demands on visual working memory (VWM), including VWM for an array of colors, simultaneously presented dot locations, sequentially presented dot locations, and target set. These tasks may interfere with contextual cueing for several reasons. First, as in any dual-task procedure, adding a secondary task increases the demand for scheduling and configuring multiple tasks. In turn, central executive capacity is reduced for the primary task (Monsell & Driver, 1999). If contextual cueing is sensitive to the availability of central executive processes, then it should be interfered with by any secondary task.

Second, the various working memory tasks occupy spare short-term memory stores. If contextual cueing depends on holding associations in VWM, then it should be interfered with when VWM is filled. Indeed, recent studies on contextual cueing show that cueing rests primarily on a few items near the target (Brady & Chun, 2007; Olson & Chun, 2001). One possible reason for the local reliance is that items near the target are currently held in VWM, and this strengthens the association between them and the target. Distant items are already discarded from VWM, reducing associative strength. Occupying VWM to capacity with colors, shapes, dot patterns, or individual locations may reduce the availability of VWM for contextual cueing.

Third, the different working memory tasks exert different demands on spatial processing. VWM for array locations or a sequence of locations, for example, requires spatial processing of elements. Such spatial processing interferes with the efficiency of visual search (Woodman & Luck, 2004). In turn, it may reduce participants’ ability to form spatial associations in contextual cueing.

Experiments 1-3 manipulated attentional load in a traditional manner, by dividing attention on half of the search trials with a secondary VWM task. We manipulated the type of stimulus maintained in VWM for the three experiments. Experiment 4 manipulated VWM load by varying the number of potential targets for visual search. Thus a VWM load for potential targets was an integral part of the search task. These manipulations all reduce the availability of central executive processes to contextual cueing, but individually they probe the parameter space of tasks that may share processing components with spatial context learning. Examining the interference pattern from the different VWM tasks can inform us not only whether contextual cueing is sensitive to divided attention, but also which components of a secondary task is most interfering.

Experiment 1A: Color working memory load with intermixed training

We tested the dependency of contextual cueing on VWM by manipulating memory load for an array of colors while observers searched for a target. During training, all search trials repeated occasionally, but on half of the trials observers searched while maintaining 4 colors in VWM, and on the other half of the trials they ignored these colors. Given that VWM capacity for colors is about 4 (Luck & Vogel, 1997), the dual-task condition was designed to occupy VWM during search. In the testing phase, the VWM task was removed from all trials, and observers simply searched for a target among previously repeated displays and novel displays. If adding color VWM load interferes with contextual cueing, then repeated search displays exposed in the dual-task condition should be learned less well than those exposed in the single-task condition. Alternatively, if contextual cueing is insensitive to color VWM load, then cueing should be equivalent for displays learned under dual-task and single-task conditions.

Experiment 1 consists of two separate studies. In Experiment 1A, the working memory load conditions were intermixed throughout training blocks, and tested only two working memory loads (low, consisting of 0 memory items, and high, consisting of 4 memory items). In Experiment 1B, each training block consisted of 3 sub-blocks, which each consisted of trials in one of the working memory conditions (no load, 2-item load, and 4-item load).

Method

Participants

Participants tested in all experiments of this study were volunteers from Harvard University or University of Minnesota and participated for payment or course credit. They were 18-35 years old, and all had normal or corrected-to-normal visual acuity and normal color vision. No participant completed more than one experiment in this paper. Fifteen participants completed Experiment 1A.

Stimuli and apparatus

Participants were tested individually in a normally lit room. They sat unrestricted at about 50 cm away from a 19” monitor. Search stimuli consisted of one T-shaped target item (1.15°×1.15°) rotated either to the left or right and 15 L-shaped distracters (1.15°×1.15°) rotated to one of the four cardinal orientations. The L-shaped distracters were formed from two segments, one of which was joined to the other near the corner, with an offset of 0.1°. Items were presented in white against a gray background. They occupied randomly selected locations from an imaginary 10×10 grid (23.6°×23.6°). Items for the VWM task were colored squares (sampled without replacement from six salient colors: yellow, green, blue, cyan, magenta, and red) and subtended 4°×4°. The colored items were arranged near fovea with one color situated on top of three other colors. They were presented against a filled square that was either black or white (16.2°×16.2°). Figure 1 shows a schematic illustration of the stimuli and trial sequence.

Figure 1.

Illustration of task sequence for the two conditions used in the training phase of Experiment 1A. Textures represent unique colors. Search phase always consisted of 16 items (12 shown in this and all future figures for schematic purposes).

Procedure

Each participant completed 24 trials of practice followed by a training phase and a testing phase.

Training Phase

The training phase consisted of 20 blocks of 24 trials apiece. These 24 trials contained unique search displays involving one T target and 15 L distracters. Each particular target and distracter spatial configuration repeated once per block, for a total of 20 times. The orientation of the target was randomly selected, so the repeated search context was predictive of the target's location but not its orientation on any given trial. Of the 24 trials, half were presented with a VWM load (dual-task) and the other half were presented without the load (single-task). Which trials were paired with VWM load was consistent across blocks, but the order of trial presentation was randomized in each block. The single and dual task conditions involved similar presentation sequence but different VWM task demands. Participants pressed the spacebar to initiate each trial. They first saw a memory display of 4 colored squares presented in a black or a white background box for 500 ms. They were informed that if the background box was white they should remember the four colors (dual-task), and if the background box was black they should ignore the four colors (single-task). The memory display was followed by a 500 ms blank screen, which was followed by the visual search display. The search display stayed on until participants responded with a key press to the orientation of the target. A test display immediately followed, which was identical to the memory display except that one of the four colors was replaced with a color not presented in the original display. In the single-task condition (again signaled by the background box being black), participants were told to ignore this display, which disappeared after 1sec. In the dual-task condition, the test display disappeared once participants indicated (using the arrow keys) which of the four colors had changed. Participants received visual feedback (in the form of a happy or a sad face icon) for their visual search task and for the VWM task (on dual-task trials). The single-task and dual-task trials were randomly intermixed in presentation order.

Testing phase

Each block of the testing phase contained 48 trials: half of these were displays seen during the training phase and the other half were untrained displays. The untrained displays consisted of one target whose position matched a trained display's target location, along with distracters presented at new locations. Because 12 trials per condition was not enough to obtain a stable RT measure, we repeated all 48 trials in additional blocks for a total of 4 testing blocks.

Data analysis

We included correct trials in the RT analysis for visual search. In addition, extreme outliers were filtered out by removing RTs below 100 ms, and RTs above 3 standard deviations from the mean of each condition in each participant. All analyses on the testing phase were collapsed across the four testing blocks.

Results

1. Training Phase

(1) Accuracy

Accuracy in the visual search task was high for both the single-task (98.4%) and the dual-task (98.9%) trials; the difference between the two conditions was significant, t(14) = 2.48, p < .03, d = 0.64, perhaps reflecting a small speed-accuracy trade-off (see RT data).

Accuracy in the VWM task (for dual-task trials) was 85%, well above chance (25%, one-sample t-test, p < .01). This accuracy implies that participants retained 2 to 3 colors in VWM, as remembering 3 would yield 100% accuracy and remembering 2 would yield 75% accuracy (e.g., if 3 colors are remembered, but the changed color is not one of the 3 remembered colors, the answer can still be determined by process of elimination – the 3 remembered colors changed, so it had to be the forgotten color). The estimated VWM capacity here was slightly lower than the typical estimation of 3-4 (Luck & Vogel, 1997; Vogel & Machizawa, 2004).

(2) RT

We analyzed search RT for trials with correct visual search response. An ANOVA on VWM task (single vs. dual-task) and training blocks (1-20) revealed a significant main effect of block, with RTs faster at later blocks than earlier ones, F(19, 266) = 18.98, p < .001, partial eta² = 0.67. This improvement may reflect general procedural learning and specific learning of repeated search context. The main effect of VWM task was also significant, F(1, 14) = 28.96, p < .001, partial eta² = 0.58, with search RT much slower in the dual-task than single-task condition. The interaction effect was not significant, F(19, 266) = 1.41, p > .10, partial eta² = 0.10, showing similar degree of RT improve across the 20 training blocks for single- and dual-task trials. Figure 2A shows the training RT results.

Figure 2.

a.) Mean RT results from training phase of Experiment 1A (error bars represent SEM in training graphs). b.) Mean RT results from the testing phase of experiment 1A (error bars represent standard error of the difference between learned and unlearned in testing graphs). c.) Mean RT results from the training phase of Experiment 1B. d.) Mean RT results from the testing phase of Experiment 1B.

2. Testing Phase

Accuracy in the testing phase was 97.3% on average and was not significantly affected by training load (F(1,14)=3.10, p=.1), spatial context learning (F(1,14)=1.26, p=.28) or their interaction (F < 1). Mean RT (Figure 2B) shows a significant main effect of spatial context, with faster RT to repeated trials than unrepeated trials, F(1, 14) = 37.37, p < .001, partial eta² = 0.73. Importantly, the main effect of training VWM load was not significant, F < 1, neither did training VWM load interact with spatial context, F < 1. Thus, there is no evidence that contextual cueing was reduced under secondary VWM load of colors. Planned contrast showed that contextual cueing was significant for displays learned under the dual-task condition, t(14) = -4.88, p <.001, d = 1.26, and for those learned under the single-task condition, t(14) = -4.12, p < .001, d = 1.06.

Experiment 1B: Color working memory load with blocked load training

Experiment 1A employed training blocks that included both low and high memory load conditions randomly intermixed within the block. The reason for this choice was to eliminate state-based or task-based effects (i.e., results due solely to the increased executive load required by a memory task, and switching between memory and search tasks). Thus, it's possible that any observable effect is actually masked by irrelevant executive processes, which might impede learning in both low-load and high-load trials. Further, intermixing these trial types with an endogenous cue to memory load may lead to confusion, such that on some low-load trials the memory items are encoded despite the instructions. Experiment 1B tested 3 different color VWM loads, with a training session that blocked load type. Each training block consisted of 3 sub-blocks of varying load.

Method

Participants

Participants were 24 individuals from the Harvard University and University of Minnesota subject pools.

Training phase

The training phase consisted of 20 blocks of 30 trials a piece. Each training block was subdivided into three sub-blocks of 10 trials. Each sub-block required a different number of colors to be encoded across each search component. The working memory load conditions were low (0 memory items), medium (2 memory items), and high (4 memory items). Participants initiated each block, while sub-blocks were cued by text indicating the memory requirements of the next 10 trials (e.g., “0 memory items”), which stayed on-screen for 1 s, followed by a 1 s blank interval before the first trial of the sub-block. The order of the sub-blocks was randomized for each block.

Each trial began with a 750 ms memory display. Under all conditions, 4 frame rectangles (2.7°×2.7°) appeared at the four cardinal positions 5° from the center of the screen. In the 0-item memory condition, no colors appeared in these placeholders. In the 2-item condition, the left and right frames were each filled with a 2.3°×2.3° patch of color (chosen without replacement from a set of 11 possible colors). In the 4-item condition, all 4 locations were filled with a unique color from the same set. Following a 250 ms blank, the search screen appeared. The search task and stimuli were identical to Experiment 1A: participants found the one T-shaped item and reported its orientation. A beep notified the participant of an error. Like Experiment 1, all configurations were repeated in each block of training, and consistently paired with one of the three memory load conditions. Following the response and a 500 ms blank, the frames and memory stimuli reappeared. In 0-item memory trials, the empty frames were removed after 1 s. In 2-item and 4-item memory trials, there was a 50% chance that one of the items had changed to a different color. The memory test remained on-screen until the participant responded “same” or “different” by pressing the “s” or “d” key, respectively. A short beep alerted participants to memory errors.

Testing phase

Each block of the testing phase contained 60 trials of search only: half of these were displays seen during the training phase and the other half were untrained displays, as before. The untrained displays consisted of one target whose position matched a trained display's target location, along with distracters presented at new locations. We repeated all 60 trials in additional blocks for a total of 4 testing blocks. No memory display was employed during testing, and a 1 s blank interval intervened between search trials.

Results

1. Training Phase

(1.) Accuracy

Search accuracy was very high overall during training (98.7%) and did not differ between the three memory load conditions (F < 1).

Average accuracy in the color VWM task was well above chance (50%) for both the medium-load (91.6%) and high-load (78%) conditions (both p < .001, one-sample t-test).

(2.) Search RT

Search was faster in later blocks of the training phase, F(19,437) = 24.1, p < .001, partial eta² = 0.51, reflecting a general procedural learning effect as well as learning of specific repeated contexts. Memory load (low, medium, and high) did significantly modulate RT during training, with slower search RT with higher memory loads, F(2,46) = 3.36, p < .05, partial eta² = 0.13. However, the interaction of the block effect and load effect was not significant, F(38,874) = 1.27, p > .12, partial eta² = 0.05, suggesting that the learning effect was equivalent across memory load conditions. Search RTs during training are shown in Figure 2C.

2. Testing Phase

Search accuracy during the testing phase was high (98.1%) and was not modulated by learning (F < 1), associated memory load (F < 1), or their interaction (F(2,46) = 1.39, p > .26).

Search RT during testing (Figure 2D) showed a significant main effect of repeated spatial context, with faster RT to repeated than unrepeated configurations, F(1,23) = 15.35, p < . 001, partial eta² = 0.40. The main effect of VWM load during training was not significant, F < 1, and neither was the interaction between training VWM load and repeated spatial context, F < 1. Thus, this experiment again provides no evidence that VWM load reduced spatial context learning. Planned comparisons of trained and untrained displays for each level of trained VWM load show significant cueing effects for displays learned under low-load, t(23) = 2.65, p < .02, d = 0.54, and high-load, t(23) = 2.08, p < .05, d = 0.42, but not medium-load, t(23) = 1.57, p < 0.13, d = 0.32. A lack of significant cueing in the medium-load condition may reflect noise in the data. The recovery of contextual cueing at high-load, and the lack of an interaction between trained load and learning, implies that VWM load did not modulate contextual cueing overall.

Discussion

This experiment tested the possibility that spatial context learning depends on the availability of VWM. In some search displays, participants maintained 2 or 4 colors in VWM while conducting search. In other search displays, participants focused on the search task with no additional VWM load. Although VWM was filled close to capacity on dual-task trials, contextual cueing was not reduced for displays learned under the most taxing dual-task conditions.

These results weaken the possibility that contextual cueing is sensitive to any kind of secondary task that stresses central executive processes. Even though color VWM tasks place significant demands on central executive processes (Makovski, Shim, & Jiang, 2006; Vogel, McCollough, & Machizawa, 2005) and on VWM's color memory store (Luck & Vogel, 1997), contextual cueing was not significantly affected by it. This suggests that increasing the demands on the central executive and on color VWM stores does not limit the capacity to implicitly learn spatial contexts.

However, results from Experiment 1 do not rule out the possibility that contextual cueing may be sensitive to specific cognitive processes engaged by a secondary task, such as spatial VWM. Indeed, VWM for colors does not interfere with visual search (Woodman, Vogel, & Luck, 2001), but VWM for spatial locations does interfere with search (Woodman & Luck, 2004). It remains possible that if the secondary VWM task involves memory for spatial locations, visual search would be interfered with and learning of repeated search context would deteriorate. The next two experiments test this hypothesis by employing VWM tasks that stress memory for spatial locations.

Experiment 2: VWM load from an array of dots

This experiment was similar to Experiment 1 except that we replaced the color VWM task with a spatial VWM task for dot locations (Figure 3). In the dual-task condition, an array of 10 dots was presented for participants to hold in spatial VWM. During the retention interval visual search was conducted. Once search was complete, another array of dots was presented for participants to compare with the first dot array. We tested whether learning for repeated search context was sensitive to adding this spatial VWM task.

Figure 3.

Depiction of the single and dual-task trials during the training phase of Experiment 2 (not drawn to scale).

Method

Participants

Nineteen participants completed this experiment. Data from 4 of these were excluded from analyses due to chance-level performance in the secondary VWM task.

Stimuli and Procedure

Stimuli and procedures for the experiment were similar to those used in Experiment 1, with the following exceptions. The VWM task was replaced with a task that required participants to remember spatial locations. On each trial, a memory display of 10 dots (1.0°×1.0°) was presented at randomly selected locations from an imaginary 10×10 grid (the same size as that used for search) for 500 ms. Participants were told to remember these dot locations if the dots were white and ignore the dots if they were black. After a blank retention interval of 500 ms the search display of T and Ls was presented. Upon participants’ response to the target's orientation, a memory test display of 10 dots was presented (in the same color as originally presented). For low-load trials, participants were told to ignore the test display, but on high-load trials they responded whether the test display was the same or different from the memory display. On half of the trials the test display was the same as the memory display; on the other half of the trials one of the dots moved to a previously unoccupied location (randomly chosen from all unoccupied locations in the grid). Participants pressed one of two keys to report their memory judgment. Similar to Experiment 1, the training phase was followed by a testing phase where the spatial VWM task was removed from all trials.

Results

1. Training Phase

(1) Accuracy

Visual search accuracy was 98.7% overall and was unaffected by experimental conditions (p > .20).

Accuracy on the VWM task was 64%, well above chance (one-sample t-test vs. 50%, p < .001). Using Cowan's K measure (Cowan, 2001), we estimated that the number of locations held in VWM was 3.2, a value that was slightly lower than previous estimates of spatial VWM (Jiang, Olson, & Chun, 2000).

(2) Search RT

Search RT became faster as the experiment drew on, F(19, 266) = 14.19, p < .001, partial eta² = 0.50, reflecting a combination of general procedural learning and specific learning of repeated search context. RT was not significantly affected by spatial VWM load, F(1,14) = 1.36, p >.26, and the improvement of RT across training blocks was comparable for trials learned under either single-task conditions or dual-task conditions, interaction F(19,266) = 1.19, p > .26. Figure 4A shows search RT data.

Figure 4.

Experiment 2 results. A.) Mean RT during training phase of Experiment 2, by block. B.) Mean RT from testing phase of Experiment 2.

2. Testing Phase

Accuracy in the testing phase was 96.7% on average and was unaffected by spatial context repetition (F(1, 14) = 1.98, p > .15), training VWM load (F < 1), or their interaction, (F < 1).

In RT data (Figure 4B), we obtained a significant main effect of spatial context, with faster RT to repeated trials than unrepeated ones, F(1, 14) = 12.88, p < .003, partial eta² = 0.48. However, the main effect of training VWM load was not significant, F < 1, neither was the interaction between training VWM load and spatial context significant, F(1, 14) = 1.27, p > .28, partial eta² = 0.08. Thus, there is no evidence that contextual cueing was reduced when training was carried out under spatial VWM load. Planned contrast showed that the contextual cueing effect was significant for displays learned under dual-task condition, t(14) = 2.61, p < .025, d = 0.67, but it failed to reach significance for displays learned under single-task condition, t(14) = 0.88, p > .30, d = 0.23, perhaps due to noise in the data.

Discussion

Even when VWM was loaded up with an array of dot locations, contextual cueing from repeated display was not reduced under VWM load. If anything, learning appeared more robust under dual-task than single-task conditions. These results converge onto Experiment 1's findings, suggesting that neither the central executive processes nor VWM storage for an array of dot locations has a significant influence on contextual cueing. The lack of interference from VWM for dot locations is surprising, given that the VWM task requires spatial monitoring and processing of dot configurations (Jiang et al., 2000), processes that may be similar to those involved in visual search. A possible reason for the lack of interference from the spatial VWM task is that observers may have extracted the virtual shape formed by the dots. Instead of using spatial VWM to remember individual dot locations, they may have relied on nonspatial VWM, or used the virtual shape as a compressed representation of space, thus sparing capacity in spatial VWM. This possibility is supported by recent findings showing that VWM for dot patterns is more similar to VWM for objects than to VWM for individual dot locations (Klauer & Zhao, 2004; Logie, 1995). Thus, the next experiment used a secondary VWM task requiring spatial VWM for individual spatial locations.

Experiment 3A: Spatial working memory load with interleaved training

Experiment 3 employs a spatial VWM task that has previously been shown to interfere with a concurrent visual search task (Woodman & Luck, 2004). In this task, two dots were presented sequentially near the center of fixation. Observers were asked to remember the exact location of each dot and judge whether two test dots matched the memory locations (Figure 5). Dots were presented sequentially rather than simultaneously, so as to impair extraction of a static pattern representation. Previous studies have shown that tasks like this engage a form of spatial VWM that is not engaged when memory for an array of dot locations is tested (Logie, 1995). In addition, spatial attention hovers around the memorized locations during the retention interval (Awh, Jonides, & Reuter-Lorenz, 1998). Both the spatial nature of the VWM and the tendency for attention to dwell near the fovea may interfere with spatial processing needed for contextual cueing.

Figure 5.

Depiction of the single and dual-task trials during the training phase of Experiment 3A (not drawn to scale).

Method

Participants

Twenty-two participants from the Harvard University study pool completed this experiment. Data from one of them were excluded due to chance-level performance in the spatial VWM task.

Stimuli and Procedure

All aspects of this experiment were identical to those of Experiments 1 and 2 except for the VWM task. The memory displays appeared in two sequential 500 ms exposures separated by a 500 ms interval. Each display contained a single circle (0.5° in diameter) presented within 1.6° of fixation. The two dots were at least 0.5° away from one another and from the fixation spot (center-to-center distance). Participants were asked to remember the dot locations when the dots were in white and ignore the dot locations when the dots were in black. After a further delay of 500 ms the search display of T and Ls was presented until a response was made. Then a memory test display was presented with two dots shown simultaneously. Participants judged whether the test dots occupied the same locations as the memory dots in the dual-task condition (the dots were in white) and ignored the test display in the single-task condition (the dots were in black). When a change occurred, the new location of the changed dot was at least 0.5° from both the fixation point and the prior positions of either dot.

Results

1. Training phase

(1) Accuracy

Accuracy in the search task was 99% and was unaffected by experimental manipulations, ps > .50. Accuracy in the spatial VWM task was 84%, again significantly above chance, p < .01 (one-sample t-test compared to 50%). This level of performance was comparable to that observed previously (Woodman & Luck, 2004), and corresponded to retaining about 1.36 locations in spatial VWM. The estimated Cowan's K value here was much lower than that seen in Experiment 2, partly because we only presented 2 dots on the display (so K could not exceed 2), and partly because this task required much greater precision of memory than that required by Experiment 2.

(2) Search RT

RT improved as the experiment went on, F(19, 380) = 20.28, p < .001, partial eta² = 0.50, and this improvement was comparable across dual-task and single-task conditions, F(19, 380) = 1.61, p > .05, partial eta² = 0.07. The main effect of VWM load was also not significant, F < 1. Figure 6A shows search RT during training.

Figure 6.

Results from Experiment 3. A.) Mean RT from training phase of Experiment 3A. B.) Mean RT from testing phase of Experiment 3A. C.) Mean RT from the training phase of Experiment 3B. D.) Mean RT from the testing phase of Experiment 3B.

2. Testing phase

Accuracy in the testing phase was 98.1% and was unaffected by training load, training block, or their interaction, all Fs < 1.

In the testing phase we again obtained a significant contextual cueing effect, F(1, 20) = 6.96, p < .02, partial eta² = 0.26, with faster RT on trained displays than novel ones. However, the main effect of training VWM load was not significant, F < 1, neither was the interaction between training VWM load and spatial context, F < 1. Planned contrast showed that contextual cueing was significant for displays learned under dual-task condition, t(20) = 2.99, p < .01, d = 0.65, but it failed to reach significance for displays learned under single-task condition, t(20) = 1.52, p > .10, d = 0.33. Thus, there is no evidence that loading spatial VWM up reduced spatial context learning.

Experiment 3B: Spatial working memory load with blocked load training

Similar to our approach in Experiment 1B, in Experiment 3B we tested the effect of spatial VWM load on contextual cueing when the load was blocked during training. We varied spatial working memory load with an additional level of difficulty: low-load (no location memory), medium-load (one location to remember), and high-load (two locations to remember).

Methods

Participants

Eighteen participants completed Experiment 3B.

Stimuli and Procedure

The procedure was very similar to that used in Experiment 1B. There were 20 training blocks subdivided into 3 blocks with low-load (0 memory items), medium-load (1-location spatial VWM load), or high-load (2-location spatial VWM load). Participants initiated each block, while sub-blocks were cued by text indicating the memory requirements of the next 10 trials (e.g., “0 memory items”), which stayed on-screen for 1 s, followed by a 1 s blank interval before the first trial of the sub-block. The order of the sub-blocks was randomized for each block.

Low-load and high-load trials were identical in all respects to low- and high-load trials in Experiment 3A, except that on low-load trials no dots were presented at all, and the search task began 500 ms after the last trial ended. Medium-load trials were similar to high-load trials, but only the first dot appeared, followed by a longer duration in which only the fixation mark was visible (1500 ms), followed by the onset of the search array. On medium-load trials, there was a 50% chance that the VWM probe dot that appeared following the search would appear in a different location.

Results

1. Training phase

(1) Accuracy

Performance on the search task during the training phase was very high under low-load (98.7%), medium-load (98.3%), and high-load (98.3%), and did not differ significantly among the three conditions (F < 1).

Performance in the spatial VWM task was well above chance (50%) for both one and two spatial locations, with 80.1% accurate responses to one location, and 73.9% to two locations (both p < .001, one-sample t-test).

(2) Search RT

An analysis of search RTs during training (Figure 6C) showed a significant effect of block, F(19,323) = 16.9, p < .001, partial eta² = 0.50, reflecting the general improvement in search over time, which is probably attributable to task-specific learning as well as learning of spatial context. There was no main effect of spatial VWM load on search RT, F < 1, but there was a significant interaction between load and training block, F(38,646) = 1.55, p < .05, partial eta² = 0.08. This interaction may reflect the fact that searches in the high-load condition were initially slower than those in the low-load condition (p < .05 for block 1), but by the end of training RTs were equivalent across all loads (n.s. for the last block).

2. Testing phase

Accuracy was high for all conditions in the testing phase (97.5%), and was not modulated by training load, F < 1, spatial context, F < 1, or their interaction, F(2,34) = 2.42, p > .1.

Searches RTs in the testing phase (Figure 6D) were significantly modulated by spatial context learning, F(1,17) = 13.1, p < .005, partial eta² = 0.44, reflecting faster search times for trained than untrained displays across all spatial VWM training loads. There was no main effect of training VWM load, F < 1, and no interaction between training load and learning, F < 1. Planned comparisons of learned vs. unlearned configurations within each trained VWM load show significant contextual cueing effects for all three training loads: low-load, t(17) = 2.46, p < .05, d = 0.58, medium-load, t(17) = 2.66, p < .05, d = 0.63, and high-load, t(17) = 2.42, p < .05, d = 0.57. The results of this experiment give no suggestion that spatial VWM load modulates contextual cueing.

Discussion

Although the spatial VWM task used in Experiment 3 was known to divert spatial attention to memorized locations (Awh et al., 1998) and to disrupt visual search (Woodman & Luck, 2004), it did not interfere with contextual cueing. If anything, learning was more robust under secondary VWM load, as the results of both Experiments 2 and 3A show a weaker contextual cueing effect under single-task than dual-task conditions (although ANOVA didn't reveal a significant interaction, and in Experiment 3B the contextual cueing benefit is comparable between low load and higher loads). This implies that the lack of learning impairment under dual-task conditions was not simply due to a lack of statistical power.

Together, data from the first three experiments showed that contextual cueing was highly robust to interference from secondary VWM tasks. Several sources of interference, including those from the central executive, short-term visual stores, and spatial processing, fail to reduce contextual cueing. These data provide strong evidence that divided attention does not eliminate or significantly reduce contextual cueing.

Experiment 4: VWM for target defining features

In the first three experiments, the VWM load came from a secondary task that was entirely separate from the search task. Because the VWM contents were irrelevant to search, their influence may have been weakened. The main purpose of Experiment 4 was to load up VWM with information relevant to the search task. We achieved this goal by varying target set for search. Unlike earlier experiments where observers always searched for a T among Ls in all trials, in Experiment 4 the search targets changed from trial to trial. The target for a given trial was specified by a cue presented before the search display. In the low-load condition, the cue contained a single shape that would appear on the search display as the target. In the high-load condition, the cue contained two shapes, only one of which would appear on the search display (Figure 7). Because participants did not know which one of the two cued items would appear, they had to retain in VWM both shapes until the target was found. Working memory load was thus integral to the visual search task. All prominent models of visual search ascribe significant functions to the top-down processes of keeping potential target templates in working memory (Desimone & Duncan, 1995; Treisman, 1988; Wolfe, 2005). The properties maintained in working memory are the source for top-down attentional guidance. Increasing working memory load affects search by increasing target uncertainty (Duncan, 1980; Vickery, King, & Jiang, 2005). It also reduces the likelihood that search will become automatic with practice (Schneider & Shiffrin, 1977).

Figure 7.

Depiction of high and low load trials from Experiment 4. Search arrays contained 16 items on a 10×10 grid (8 of 16 depicted here on an 8×8 grid). On finding the target, participants responded with a keypress, then chose a number that appeared in the target item's location.

Increasing the number of potential targets on each trial can affect several processes. First, search is slower and less accurate (Menneer, Phillips, Donnelly, Barrett, & Cave, 2004; Menneer, Cave, & Donnelly, 2009), because observers must match each search element with multiple potential targets. Increasing the amount of time dwelling on each item may reduce processing of the entire search configuration, which may negatively affect associative learning of the configuration and the target location. Second, keeping multiple items in working memory reduces the availability of working memory for other components of search and learning. The random shapes used in Experiment 4 load up visual short-term storage. If contextual cueing is sensitive to the availability of working memory, then it should be reduced or eliminated under high load conditions. Because the capacity of VWM for novel shapes is as low as 2 (Alvarez & Cavanagh, 2004), we used 2 potential targets in the high-load condition and 1 target in the low-load condition.

Method

Participants

Twenty-nine participants completed this experiment.

Stimuli and Procedure

Search stimuli were novel black shapes (0.75°×0.75°), developed in the lab and intended to be difficult to name. Each search trial began with a cue display, which consisted of either 1 or 2 shapes shown at the center of the screen. After committing the cue to memory, participants pressed the spacebar. After a 500 ms blank interval, 16 shapes were presented at randomly selected locations from a 10x10 imaginary grid, (jittered randomly up to 0.4° horizontally and vertically). One of these items matched the previously cued shape (single-target) or one of the previously cued shapes (two-target search). Participants pressed the space bar when the target was located, after which half of the items (randomly selected) were replaced with a “1,” while the other half were replaced by a “2.” Participants indicated the position of the target by reporting the digit at the target's former location. Search RTs were measured as the interval between the onset of the search screen and the space bar response on successful trials.

Participants completed 20 blocks of training, each involving 20 trials. Half of these trials were paired with the low load condition (a single target) and the other half were paired with the high-load condition (two potential targets). The pairing between a particular layout and load was consistent across blocks, but the exact target shape(s) cued for a given display was random, as were the exact shapes of the distracters. Just as in standard contextual cueing tasks, distractor locations reliably predicted target locations, but the identities of the target and distracter shapes were randomized from trial-to-trial, even for a particular trained configuration.

After the training phase, participants completed 4 blocks of testing trials. Each block contained 80 trials, half of which were trained displays and half were untrained displays that were matched in terms of target location. Of the 40 trials in each type of search context (repeated or unrepeated), there were 4 conditions created by factorial combination of training load (low or high) and testing load (low or high). In the low-low condition, displays trained under the low load condition were again tested under low-load during the testing phase. In the high-high condition, displays trained under the high load condition were again tested under high-load. These two conditions thus maintained the task requirement from training to testing (always search on a given display with a given level of load). In contrast, the low-high (trained under low load and tested under high load) and high-low (the opposite arrangement) conditions involved a change in task requirement from training to testing. These conditions allowed us to test not only the main effect of training load but also the consistency of training and testing load.

Results

1. Training phase

(1) Search accuracy

Search accuracy was significantly affected by load, t(28) = 6.54, p < .001, d = 1.21, as participants were less accurate under high load (92%) than low load (97%), but was still well above chance under high load. The reduction in accuracy might result from a loss of memory of the cued target shapes.

(2) Search RT

Search RT was also significantly slower under high load than low load condition, showing dramatic slowing of search target uncertainty, F(1, 28) = 226.87, p < .001, partial eta² = 0.89. Search RT improved as the experiment went on, F(19, 532) = 14.37, p < .001, partial eta² = 0.34, and the improvement was greater for high-load trials than low-load trials, F(19, 532) = 3.27, p < .001, partial eta² = 0.10.

Cues were viewed longer under high load than under low load (an average of 812 ms under low load, and an average of 1262 ms under high load, t(28) = 6.953, p < .001).

2. Testing phase

Accuracy in the testing phase was examined in a 2×2×2 ANOVA with training load (low or high), testing load (low or high), and spatial context (repeated or unrepeated) as factors. There was a significant main effect of testing load, F(1, 28) = 35.08, p < .001, partial eta² = 0.56; participants were less accurate under high testing load (90.4%) than low testing load (96.4%). No other effects were significant, ps > .13.

In RT analysis, we first examined cases in which the training and testing loads were consistent (low-low and high-high). An ANOVA on load and spatial context revealed a significant main effect of spatial context, in that observers were faster searching through trained than untrained displays, F(1, 28) = 17.84, p < .001, partial eta² = 0.39. The main effect of load was also significant, in that observers were slower on high-high trials than low-low trials, F(1, 28) = 147.42, p < .001, partial eta² = 0.84. Critically, there was no interaction between the two conditions, F(1, 28) = 2.07, p > .15. The non-significant trend points to greater contextual cueing for high load than low load, as the overall contextual cueing effect was 188 ms for high-high (t(28)=-3.376, p<.002) and 106 ms for low-low conditions (t(28)=-3.397, p<.002). Expressed as percentage change in RT, learning reduced RT compared with untrained displays by 8.7% in the high load condition and 8.2% in the low-load condition, demonstrating that learning was roughly proportional across high-load and low-load conditions.

We then analyzed RT in the two conditions where the training and testing loads were inconsistent (high-low and low-high conditions). Here, there was a significant main effect of testing load, with longer RT for low-high than high-low conditions, F(1, 28) = 195.32, p < .001, partial eta² = 0.88. However, the main effect of spatial context was not significant, F < 1, neither did spatial context interact with load, F < 1. The cueing effect for displays trained under high load and tested under low load was not significant (15 ms, t(28)=-.406, p=.69). The cueing effect was 19 ms (not significant, t(28)=-.231, p=.819) for displays trained under low and tested under high load. Thus, when the training load and the testing load mismatched, contextual cueing was not observed, even though the trained displays were clearly learned (as shown in the low-low and high-high conditions). A full omnibus ANOVA on training load, testing load, and spatial context revealed a significant three-way interaction, F(1, 28) = 5.54, p < .03, partial eta² = 0.17, confirming that contextual cueing was observed only when the testing load matched the training load.

Discussion

This experiment reinforced the notion that contextual cueing is insensitive to additional working memory load. Learning of repeated search context was comparable when observers searched for a unique target and when they searched for one out of two possible targets. Adding a potential target increased the need to hold multiple targets in working memory during search. It dramatically slowed down RT and impaired search accuracy, yet it yielded as strong a learning effect as when working memory was less loaded. These results are consistent with the overall thesis developed across all experiments: contextual cueing is insensitive to manipulations that reduce availability of working memory.

We also observed an interesting specificity of transfer in Experiment 4, where contextual cueing transferred to a testing session whose search load matched that of the training load, but it did not transfer to a testing session with a mismatched load. This kind of specificity to training conditions has been observed previously (Jiang & Song, 2005). It highlights the need to separate learning from expression of learning (Jiang & Leung, 2005). The repeated search context was clearly learned whether training was done with low load or high load, but learning was expressed in behavior only when the testing load matched the training load. This kind of load-specificity, however, was not observed in Experiments 1-3. There are at least two possibilities why transfer was specific to the training load in Experiment 4 but not in the other experiments. First, the load was an integral component of search in Experiment 4 (but not in other experiments) and thus may be critical for expression of learning. Second, observers were tested in low and high load in Experiment 4 but only in single-task conditions in Experiments 1-3. This difference may sensitize observers to the mismatch in load in Experiment 4. Future studies are needed to explore the specificity of learning to properties that seem irrelevant to repeated search context.

General Discussion

How does divided attention affect spatial context learning? The answer from the four experiments carried out here is clear: dividing attention with working memory load does not significantly weaken spatial context learning. In Experiments 1-3, observers conducted visual search either under single-task conditions, or while holding in visual working memory colors, a dot array, or two sequentially presented dot locations. Contextual cueing was observed for repeated search displays independent of whether search was carried out as a single-task or under dual-task conditions. These findings show that contextual cueing was relatively insensitive to availability of visual working memory. These results are unexpected because VWM tasks used here, such as the spatial VWM task, are known to disrupt visual search (Woodman & Luck, 2004). One might propose that our participants ignored the dual-task instruction and focused on visual search in all trials. This proposal seems unlikely given that performance in the secondary VWM task was close to or comparable to that seen in other studies (Vogel & Machizawa, 2004; Woodman & Luck, 2004). Nonetheless, to ensure that participants followed the working memory instruction, in Experiment 4 we changed the task by making the VWM load an integral component of visual search. Here, participants must maintain in working memory 1 or 2 potential search targets. Increasing working memory load in this task dramatically increased search RT. It also reduced search accuracy. Still, this manipulation did not reduce or eliminate contextual cueing.

Why was contextual cueing sensitive to selective attention but not to divided attention? Given that selective attention is simply a more extreme form of divided attention where attention is exclusively devoted to some stimuli but not to others, it is paradoxical to see how robust contextual cueing is to divided-attention manipulations. The discrepancy is easy to understand, however, when effects of selective attention are separated for learning and for the expression of learning. Repeated ignored context does not enhance search RT, but when the previously ignored repeated context is attended in a testing session, it can facilitate RT (Jiang & Leung, 2005). In fact, when a previously learned context becomes unattended later, it does not enhance RT. Thus, selective attention modulates the expression of learning in contextual cueing but does not gate latent learning itself. These findings are analogous to results in the serial reaction task (Frensch, Wenke, & Runger, 1999), where diverting attention to a secondary task reduces the expression of learning without reducing latent learning. Taken together, studies on selective attention and divided attention both show that the learning of repeated visual context is relatively insensitive to attention. These results support the general theoretical framework that divides explicit from implicit processes, with the latter being more robust to many stressors, including the availability of attention and working memory resources (Reber, 1993).

Our results are also important for theories of the underlying mechanisms of contextual cueing. So far at least two theories have been proposed to account for all or portions of the contextual cueing effect: an attentional guidance account (Chun & Jiang, 1998) and a response bias account (Kunar, Flusberg, Horowitz, & Wolfe, 2007). The attention guidance account suggests that once learned, the repeated context can serve as an attention-guiding cue, allowing search to be completed on the basis of memory rather than on the basis of serial deployment of attention. Our study constrains this account, such that both the storage and the retrieval of this memory proceed with little or no attention or working memory. Since we can never truly eliminate working memory resources without obliterating the ability to perform any task successfully, we cannot rule out that attention and working memory have some role in linking context to target position. However, the requisite demands of spatial context learning must be relatively low. The storage and retrieval of repeated context seem to proceed relatively automatically, requiring little additional resources beyond those needed for successful search.

The response bias account (Kunar et. al, 2007) proposes that once the target is located, observers are more confident of their decision and can make a faster response to it on repeated displays. Kunar and colleagues provide evidence for this account, showing that contextual cueing reduces the intercept but not the slope of a search function, and finding that interference at the response stage can eliminate contextual cueing. Thus, a response bias account may account at least partially for the contextual cueing effect. Again, our results constrain theories such as this, such that minimal additional memory or attention is required to match the current display to past displays, and recall the associated response selection. Our results suggest that as long as the context is attended enough to perform a search, this primed selection is stored during training and automatically employed in the decision-making process associated with search.

Regardless of whether one endorses an attentional guidance or a response bias account, the question of whether learning depends on attention is an important issue. Both theories must explain how a configuration is learned, and under what conditions it can be used to facilitate RT. The finding that contextual cueing does not rely on excess VWM capacity, however, allows us to reject models that give VWM a prominent role in registering repeated context and in use of the context for future search. The idea that limits in VWM may specifically underlie the local nature of contextual association is unsubstantiated by our data. Thus the highly local requirements of the learned context (Brady & Chun, in press; Olson & Chun, 2001) cannot be explained by the availability of working memory resources. That is, it seems unlikely that VWM is the cause for preferential association between the target and a few adjacent distracter locations.

In conclusion, our study has provided strong empirical evidence for the robustness of contextual cueing to divided attention. The formation of contextual memory does not rely heavily on visual working memory for repeated context. This finding provides new constraints to theories of contextual cueing: it may require no excess attentional capacity, or only so much attention as is required to accomplish the search task.

Figure 8.

Results from Experiment 4. a.) Mean RT for each block of training under high and low load. b.) Mean RT for consistent components of testing phase: Search arrays that were trained and tested under the same amount of load. c.) Mean RT for inconsistent components of testing phase: Search arrays that were trained and tested under different load conditions.