Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2010 Feb 18.
Published in final edited form as: Atten Percept Psychophys. 2009 Oct;71(7):1525–1533. doi: 10.3758/APP.71.7.1525

Flexible working memory representation of the relationship between an object and its location as revealed by interactions with attention

Joseph B Sala 1, Susan M Courtney 2,3,4
PMCID: PMC2824549  NIHMSID: NIHMS177248  PMID: 19801613

Abstract

Working memory (WM) selectively maintains a limited amount of currently relevant information in an active state to influence future perceptual processing, thought, and behavior. The representation of the information held in WM is unknown, particularly the degree of separation between the representation of an object’s identity and its location. The current experiments examined the flexibility of object and location WM representations by measuring reaction times on a visual discrimination task during the delay period of a WM recognition task for object identities, locations, or both. Results demonstrate that during WM delay periods attention is biased toward information that matches the current contents of WM. Attention is not biased toward information that was present in the encoded memory sample but not relevant for the memory recognition test. This specificity of the interaction between WM and attention applies to both the identity and the location of the remembered stimulus and to the relationship between an object and its location. The results suggest that when this relationship is necessary for task performance, WM represents an object and its identity in an integrated manner. However, if this relationship is not task relevant, the object and location information are represented in WM separately.

Introduction

In describing working memory, Baddeley and Hitch (1974) suggested a multiple component model, hypothesizing divisions between the maintenance of different information types. There is considerable evidence suggesting separate systems for maintenance of verbal and visuo-spatial information (for review see: Baddeley, 1998; Smith & Jonides, 1998). Dissociations between visuo-spatial information types, such as objects and locations, have also been identified (e.g. Logie 1995). Research has also suggested that visuo-spatial WM tasks are composed of distinct “processing” and “storage” components, and furthermore, that “storage” may be dissociated into distinct visual form and spatial components (Duff & Logie, 1999).

Other evidence suggests that different types of information are not so independently maintained. Some evidence suggests individual nonspatial features of objects (eg. color and shape), are integrated within WM to form a unified object identity representation (Luck & Vogel, 1997; Vogel, Woodman, & Luck, 2001). In contrast, other results suggest that the “binding” among features of objects is an additional, non-obligatory and attention demanding process requiring extra capacity (Wheeler & Treisman, 2002). In addition, some studies have found spatial configuration and other contextual information can influence working memory performance even when this information is not task-relevant (Jiang, Olson, & Chun, 2000; Postle 2003). Inherent to this debate is the underlying question concerning the nature of the representation of information actively maintained in visual WM. While some forms of irrelevant information may influence working memory performance through influences on perceptual processing of the test stimuli, it is unclear whether this information is part of the representation actively maintained in working memory. The goal of the current study was to ascertain whether the identities and locations of objects held in working memory necessarily have an integrated active representation, or whether the relationship between an object and its identity can be actively represented in WM or not according to whether that information is task relevant. To probe this question we used the interaction between working memory and the effect of selective attention on visual processing.

The terms ‘Attention’ and ‘Working Memory’ (WM), as used here, refer to systems that, respectively, provide for 1) selective prioritization for processing a subset of perceptual information and 2) short-term maintenance and manipulation of information necessary for performance of complex tasks. Attention is thought to act as a gate to WM; what is attended is allowed entry to WM. In addition, it has been suggested that selective attention can serve as a rehearsal mechanism subserving WM maintenance (Awh, Jonides, & Reuter-Lorenz, 1998). The purpose of WM, however, is not only to remember what has occurred, but also to use this information to affect future behavior and direct attentional resources for future processing of expected stimuli. Neurophysiological studies support the “biased competition” model of visual attention in which information maintained in WM may serve as “top-down” signals that influence the outcome of mutually inhibitory perceptual representations (e.g. Chelazzi, Duncan, Miller, & Desimone, 1998). Thus, interactions between attention and working memory appear to work in both directions. One consequence of these interactions is that selective attention effects on perceptual processing during the delay period of an unrelated working memory task are often congruent with the contents of working memory (e.g. Downing, 2000).

This interaction between attention and WM may not be obligatory or automatic. Investigations using WM and visual search dual tasks show that attending to an item does not automatically enter it into WM, and WM for a specific item does not necessarily hinder attention to other items (Downing & Dodds, 2004; Woodman, Vogel, & Luck, 2001; Woodman & Luck, 2004 ). Others have argued, however, that working memory does automatically influence selective attention (Olivers, Meijer, & Theeuwes, 2006; Soto, Humphreys & Blanco, 2005). Taken together, these previous results suggest that WM does automatically bias attention, although this bias may sometimes be overridden by stronger biases from other task demands. In any case, under conditions in which WM and attention do interact, perceptual processing biases can be used to probe the nature of the representation of information held in WM.

In the current set of five experiments, differences in RT during a perceptual discrimination task embedded within the delay of a WM task were used as indicators of the representation maintained in WM. These studies bolster and extend previous results (e.g. Awh et al., 1998; Downing, 2000; Olivers et al., 2006), demonstrating that both spatial and nonspatial WM can, in certain circumstances, interact with perceptual selective attention. Furthermore, we demonstrate that, to the extent that the working memory representation interacts with attention, active maintenance in WM of the relationship between an object and its location is depends on whether this relationship is task relevant.

Experiment 1

Experiment 1 addressed whether the need to selectively maintain the location versus the identity of an object in working memory differentially affects biasing of attention in an unrelated task during the WM delays.

Methods

Overall Experimental Design

Subjects performed a delayed recognition task requiring the WM maintenance of either the identity or location of fractal-like pattern images. During the delay, subjects performed a visual discrimination task measuring reaction time to identify a target that appeared unpredictably in one of two possible locations. Two cues marked the potential locations for the presentation of the target. Very briefly after the cues appeared, the target and a distractor appeared, each superimposed on top one of those cues, The subject’s task was to identify the target, which was either a 2 or a 5. Subjects were told that the cue images were irrelevant to the task and did not predict where the target would appear. See Figure 1. One cue matched information held in working memory. The other cue matched information present in the sample but not relevant for the memory task. Specifically, one cue matched the location but not the identity of one of the sample items, and the other matched the identity but not the location of one of the sample items. If the representations in WM for object identity and location are distinct, both types of information are maintained only when task relevant, and both types of representations interact with selective attention, then targets appearing atop the cue that matches the working memory task-relevant aspect of the information should receive a bias in processing. This predicted bias in processing should lead to faster target discrimination.

Figure 1.

Figure 1

Open in a new tab

(A) Trial event sequence and timing, which were common to all experiments. An example trial from Experiment 2 is shown, with the perceptual discrimination task using the CONJUNCTION and NON-CONJUNCTION target types within the delay period of an AND working memory recognition task. In this example, the participant would have made a speeded button press response to indicate that a “2” was present and then a “non-match” button press response with the other hand because while the test image is one of the patterns in the memory sample set and it is in one of the sample locations, it is not the correct conjunction of the two. During the discrimination task, the “2” appears as the NON-CONJUNCTION target and thus the discrimination task RT is expected to be slower than if it had appeared atop the other cue image, which matches the remembered conjunction information. The dashed line circles shown in the test screen illustrate the other two sample locations but did not appear in the actual experiment. The test item was presented together with 2 phase-scrambled images, which were irrelevant to the task, and were used only to balance visual input for the sample and test phases for a planned, subsequent fMRI experiment with these tasks. The test item and the irrelevant phase-scrambled stimuli were clearly distinguishable to the subjects. (B) Some potential cues and target types for an example memory set.

Participants

20 healthy Johns Hopkins undergraduates participated in Experiment 1, and in each of the following five experiments, for course credit, after giving informed consent. Each experiment recruited a new group of participants. All protocols were approved by the Homewood Institutional Review Board of Johns Hopkins University.

Stimuli

Twelve colorful abstract patterns (Figure 1) were created using Artmatic 2.1 (U&I Software). All patterns were initially unfamiliar and each appeared equally often in all conditions. These stimuli, each measuring approximately 3.5° of visual angle in diameter, were presented in 18 different locations centered about a fixation cross at a maximum distance of 6° of visual angle from fixation. These stimuli were used in all experiments.

Procedure

Figure 1A depicts the trial sequence, which was the same for all experiments. Each trial began with a one second instruction, indicating which aspect of the sample stimulus was to be remembered (identity (“WHAT”), location (“WHERE”), or both for Experiments 2, 4, and 5 (“AND”). Next, three sample images were presented simultaneously for one second. A delay of one, two, or three seconds followed, after which the cue images for the target discrimination task were presented. A semitransparent block figure eight was superimposed on each cue image. These cue images with the block eights superimposed did not provide any information regarding the probability that the target would appear atop one versus the other. Then, two bars from each of the figure eights were removed, transforming one into a “2” or “5” (target), and the other into an “E” or “3” (distractor). Subjects responded with the press of one of two buttons as quickly and accurately as possible regarding whether the target was a “2” or a “5”. Presenting the cues with figure eights first and changing both figure eights into one of two possible targets and one of two possible distractors ensured that the attention bias between the competing locations would not be influenced by sudden changes in the display or by a search template for either a particular target or a particular distractor. The cues and superimposed block eights were presented for 180 ms before the targets and distractors appeared, based on previous neurophysiological data in monkeys indicating that this would be the minimal time required for attention to resolve competition between the two cue locations during visual search (Chelazzi et al., 1998).

The WM delay then continued for a total time of seven seconds from the end of the sample so that the total delay length was the same for all working memory trials although the timing for the appearance of the discrimination task within that delay was unpredictable. The trial concluded with the presentation of the test image for which subjects made a match/nonmatch judgment. During the WHAT WM task, a “match” was when the test item was the same pattern image as any of the three sample items for that trial. During the WHERE WM task, a match was when the test item appeared in the same location as any of the three sample items. During the WHAT task, the test item always appeared in a location not used at sample or during the discrimination task. During the WHERE task, the test item was always a pattern image not used at sample or during the discrimination task. Button press responses were made with one hand for the discrimination task and the other for the WM recognition task, counterbalanced across subjects. WM tasks were blocked with 32 trials of the same task condition (WHAT or WHERE) performed consecutively. Task blocks occurred in an ABBA fashion, counterbalanced across subjects. Before data collection began, subjects were trained on 16 trials of each WM task, including the discrimination task, with feedback regarding recognition performance.

Analysis

The critical dependent variable was the reaction time for making the 2/5 discrimination judgment. In each trial of both WM tasks (WHAT or WHERE), if the target (2 or 5) appeared atop the cue that matched the identity of a sample, this was deemed a “PATTERN Target” in the analysis. If the target appeared atop the cue matching a location used at sample in that trial, this was deemed a “LOCATION Target” (Figure 1B) in the analysis. A repeated measures ANOVA on the 2/5 discrimination RT was performed with WM task type (WHAT or WHERE) and target type (PATTERN or LOCATION) as factors. Only trials in which participants responded correctly to both the 2/5 discrimination and the WM delayed recognition test were included in the analysis.

Results and Discussion

No difference in percent correct was found between WM tasks (WHAT: 77, WHERE: 80; t(19) = −1.4, p > .1). The results for reaction time on the 2/5 discrimination task are shown in Figure 2. There were no main effects of WM Task (WHAT: 931ms, WHERE: 919ms; F(1,19) = .215, p > 0.1) or Target type (PATTERN: 928ms, LOCATION: 923ms; F(1,19) = .099, p > 0.1). There was, however, a significant interaction between WM Task and Target type (F (1,19) = 20.903, p < 0.01). Post hoc tests showed discrimination RT was different for LOCATION versus PATTERN target types during both the WHAT and WHERE tasks (HSD: p < .01), and the difference between target types was different according to the type of information being held in working memory. Subjects were faster to process LOCATION targets when they were doing the WHERE working memory task, and were faster to process PATTERN targets when they were doing the WHAT working memory task.

Figure 2.

Figure 2

Open in a new tab

RTs for the 2/5 perceptual discrimination task in Experiment 1, according to working memory task (WHAT or WHERE), and whether the target appeared on the cue that matched the sample location (“Location Target”, green) or on the cue that matched the sample pattern image (“Pattern Target”, red).

These results suggest that the representation for WM maintenance that interacts with attention is task-dependent, and that representations of spatial and nonspatial information in working memory are dissociable. Although the samples presented are identical for the WHAT and WHERE WM tasks, the information maintained, and the corresponding bias in perceptual attentional selection during the memory delay, differs.

Experiment 2

Experiment 1 suggests that location and identity information in working memory are dissociable when only one or the other type of information is task relevant. However, it does not specifically address the degree to which the representations of these information types may be dissociated or integrated with one another when both are task relevant.

Methods

The overall experimental design and task timing were the same as in Experiment 1, but the particular information to be maintained and the nature of the cues for the target discrimination were different. In Experiment 2, subjects performed two different WM tasks requiring the maintenance of both identity and location information. During the OR task, the test stimulus was considered a match if either the pattern or the location matched that of one of the three sample stimuli. During the AND task, a test item was considered a match only if the matching sample pattern was in the same location that it had occupied at sample. Thus, the AND task, but not the OR task, required subjects to hold in working memory the relationship (“conjunction”) between the sample patterns and their locations.

For the 2/5 discrimination task during the delay period of the AND and OR working memory tasks, both cues matched identity and location information present in the memory sample set. As in Experiment 1, this information was irrelevant for the 2/5 discrimination task: subjects were instructed merely to decide as fast as they could whether a 2 or 5 was present on either cue. One cue was one of the sample patterns appearing in the location where it originally appeared (labeled a “CONJUNCTION target” for the analysis). The other cue was one of the other two sample patterns appearing in the location originally occupied by the third sample pattern (labeled a “NONCONJUNCTION target” for the analysis). This condition necessitates using 3 items in the working memory sample set, and we kept this memory load consistent across all five experiments.

As in Experiment 1, the 2/5 discrimination task RT was analyzed using ANOVA with WM task (AND, OR) and target type (CONJUNCTION, NON-CONJUNCTION) as factors. If the WM representation is composed of separate and independent identity and location information stores, then cues matching identity and location information from the sample should be processed equally regardless of whether it is conjoined in the cue as it was at sample. Conversely, if the WM representation is an integrated whole, then CONJUNCTION Targets would receive prioritization during the AND task. If these integrated object-in-location representations are the default in WM, even when this information is not task relevant (and may interfere with OR task performance) then CONJUNCTION Targets would be prioritized also during the OR task.

Results and Discussion

No difference in percent correct was found between WM tasks (AND: 79, OR: 76; t(19) = 1.6, p > .1). Results for RT on the discrimination task are shown in Figure 3. There were no main effects of WM Task (AND: 990ms, OR: 978ms; F(1,19) = .215, p > 0.1) or Target type (CONJUNCTION: 971ms, NON-CONJUNCTION: 996ms; F(1,19) = 1.353, 3.685, p > .07 respectively). There was a significant interaction between WM Task and Target type (F(1,19) = 22.604, p < .01). Post hoc tests showed RTs to the two target types were different during the AND task (HSD: p < .01) but not the OR task. Subjects were faster to decide whether the target was a 2 or a 5 when the target appeared on the cue consisting of the matching pattern in its sample location (the CONJUNCTION target) during the AND but not the OR WM task.

Figure 3.

Figure 3

Open in a new tab

RTs for the 2/5 perceptual discrimination task in Experiments 2 and 3, according to working memory task (AND or OR), and whether the target appeared on the cue that matched the sample pattern in the location where it was presented at sample (“Conjunction Target”, blue) or on the cue that matched a sample pattern image but was in the location of the other sample item (“Nonconjunction Target”, orange).

These results indicate that the actively maintained representations during the OR task differ from those during the AND task. The conjunction of identity and location information is represented in WM, as reflected in the bias of selective attention for matching object-location conjunctions, during delays in the AND task only. During the OR task, the relationships between the to-be-remembered patterns and their locations are irrelevant to successful completion of the working memory task so both the CONJUNCTION and NONCONJUNCTION Targets are equal candidates for the allocation of attention during the 2/5 discrimination task.

This result suggests that the relationship between an object and its location is not necessarily actively maintained in WM. This result is consistent with work showing that color and location information may be maintained in parallel without either interfering with the other (Wheeler & Treisman, 2002; Magnussen, 2000). Given the nature of the OR task, however, it is not possible to assess the degree to which the integration may occur when only identity or location information is relevant to WM task performance. It is possible that the conjunction representation is indeed the default, but the independent judgments for identity and location matches required in the test of working memory in the OR task encouraged an active “unbinding” of the patterns and their locations that would not happen in the WHAT or WHERE tasks.

Experiment 3

Methods

Experiment 3 used the WHAT and WHERE WM tasks from Experiment 1 and the CONJUNCTION and NONCONJUNCTION Targets from Experiment 2. If when required to remember only one piece of information the relationship between an object’s identity and its location is represented by default, then the CONJUNCTION Targets should receive a bias in selection over NONCONJUNCTION Targets during the WHAT and WHERE WM task delays, as they did during the AND task of Experiment 2.

Results and Discussion

No difference in percent correct was found between WM tasks (WHAT: 81, WHERE: 82; t(19) = −.32, p > .1) A repeated measures ANOVA on discrimination task RT data (shown in Figure 3) revealed no main effect of WM Task or Target type, and no interaction between the two (F(1,19) = .215, .099, .064 respectively, p > .1). When the CONJUNCTION and NONCONJUNCTION Targets were presented during WHAT or WHERE delays, no differences were seen in the 2/5 discrimination RTs (Figure 3). This result is different from that in Experiment 2 where the CONJUNCTION and NONCONJUNTION targets were presented during the AND and OR task delays. During the AND task, the conjunction between spatial and nonspatial information was specifically relevant to task performance. During the OR task, subjects may have specifically inhibited this information. During the WHAT and WHERE tasks, however, this conjunction information is irrelevant. The results indicate that in this case, the conjunction information does not appear to be represented in working memory in that it does not interact with the selection of visual information during the memory delay. It is possible that the conjunction may be represented in WM during the WHAT and WHERE tasks, but in such a way that it is not reflected in the bias of selective attention. However, when the conjunction is task relevant (as in the AND task of Experiment 2), this relationship is clearly part of the representation maintained in WM and interacts with selective attention. It appears, therefore, that the conjunction representation is not present in the WHAT and WHERE tasks in the same manner as in the AND task.

The pattern of results emerging implies that representations within WM are specifically dependent on the task demands. It appears that active representations reflect only task relevant information, including the relationship between an object and its location, and that only this relevant information results in a bias of attention for processing visual information during the delay. Experiments 4 and 5 further test the possibility that irrelevant information may be represented in working memory and interact with perceptual selective attention.

Experiment 4

Experiments 2 and 3 used CONJUNCTION and NONCONJUNCTION Targets. These target types both contain matching identity and location information, differing only in whether these pieces of information are correctly conjoined. While these experiments suggested that this conjunction is not automatically represented, it is still possible that irrelevant identity or location information is represented, independent of the relationship between a particular pattern and its location. Such a representation could result from incidental encoding of the irrelevant feature upon encoding of the relevant feature. If so, then during WHAT or WHERE tasks a target matching both task relevant and irrelevant information should be attended preferentially over one matching only task relevant information. Experiment 4 investigated whether WM representations during the WHAT task would reflect irrelevant location information, in addition to relevant, pattern identity information.

Methods

Experiment 4 utilized the AND and WHAT WM tasks and the CONJUNCTION and PATTERN Target types. It was expected that during WM maintenance in the “AND” task attention would be biased toward a target containing the conjunction of identity and location information, as during Experiment 2. It was unclear whether, when both target cue types match the relevant information, the addition of matching irrelevant location information to one target cue (CONJUNCTION target), would bias selection during the WHAT task over the target that matched only the relevant information (PATTERN target).

Results and Discussion

No difference in percent correct on the WM tasks was found (AND: 85, WHAT: 87; t(19) = 1.6, p > 0.1). Results for the 2/5 discrimination task RT are shown in Figure 4. A repeated measures ANOVA revealed a main effect of WM Task and Target type as well as an interaction between the two (F(1,19) = 9.614, 4.221, 9.108, p < .01, .01, .05 respectively). Post hoc tests showed that RTs to the CONJUNCTION and PATTERN target types were different during AND (HSD: p < .01) but not WHAT WM task performance. Discrimination RTs are faster for CONJUNCTION targets, which consist of a matching pattern in its sample location, relative to PATTERN Targets only during the AND WM task when this relationship between the pattern and its location is task relevant. Again, attentional biasing favors the behaviorally relevant information of the WM task. Additional task irrelevant information (matching location in the CONJUNCTION targets during the WHAT delays) does not affect selection between the discrimination task cues when maintenance requires the representation of only identity information.

Figure 4.

Figure 4

Open in a new tab

RTs for the 2/5 perceptual discrimination task in Experiments 4 and 5, according to working memory task and target type. Colors for each target type are the same as in Figures 2 and 3.

Experiment 5

The motivation for Experiment 5 was similar to that of Experiment 4: to test whether irrelevant information is represented in WM in such a way that a target cue that matches both relevant and irrelevant information from the sample stimulus will be preferentially attended over a target cue that only contains matching relevant information. While Experiment 4 tested whether irrelevant matching location information biased attention during the WHAT task, Experiment 5 tested whether irrelevant matching pattern information biased attention during the WHERE task.

Methods

Experiment 5 utilized the AND and WHERE WM tasks and the CONJUNCTION and LOCATION Target types.

Results and Discussion

No difference in percent correct on the WM tasks was found (AND: 78, WHERE: 81; t(19) = 1.5, p > 0.1). Results for the 2/5 discrimination task RTs are shown in Figure 4. A repeated measures ANOVA revealed no significant main effect of Target type (F(1,19) = 3.212, p > 0.08). There was a significant main effect of WM Task, and an interaction between WM Task and Target type (F(1,19) = 7.635, 6.882, p < 0.05 for both). Post hoc tests showed that RTs to the CONJUNCTION AND LOCATION target types were different during AND (HSD: p < .01) but not during WHERE task performance, indicating again that only WM task relevant information biases visual attention during the delay. Both pattern and identity information, and the relationship between the two were relevant during the AND task, and correspondingly the CONJUNCTION TARGET (matching all relevant information) was preferentially processed over the LOCATION Target (which matched only part of the relevant information). During the WHERE WM task, the additional matching pattern information in the CONJUNCTION target was not task relevant, and did not bias visual attention for that target over the LOCATION Target, which matched only the relevant information.

General Discussion

The present study focused on the representation of information during active maintenance in WM as indicated by its interaction with selective attention. In agreement with recent previous studies, attention was biased specifically in favor of information matching the current WM representation (e.g. Awh et al., 1998; Downing 2000; Olivers et al., 2006; Soto & Humphreys, 2006; Huang & Pashler, 2007). This result generalized across five experiments allowing us also to draw the following conclusions: 1) representations in WM that interact with selective attention reflect task relevant, and not task irrelevant, information, and 2) the relationship between an object and its location is actively maintained in WM only when this relationship is task-relevant.

The current results lend support for a biased competition framework of selective attention. During delay periods of the current study, one of two competing cues could receive prioritization. To the degree that the cue underlying the target in the discrimination task matched the information represented in WM, its perceptual representation was biased, allowing its location to “win out” over its competitor and enhance the perceptual processing of the target at that location. Previous studies have indicated that this biasing appears to be automatic (Olivers et al., 2006; Soto et al., 2005). There can be multiple biasing signals that influence attention, however. The interaction between WM and attention can be either facilitative or inhibitory, and might be observed only in situations where other task demands on selective attention do not dominate competition (Woodman & Luck, 2007). It is also possible that the effects on RT for processing of the target were due to faster processing of the underlying cue stimulus rather than reflecting a shift in selective attention to the location of the matching cue. Nevertheless, regardless of the nature of this influence and what other factors might interfere, when such influences of the contents of WM on visual processing are observable, as in the current study, they can provide valuable information about the nature of the representation of information in WM.

Another potential source of bias in visual processing is the tendency for novel images to attract or capture attention (e.g. Johnston, Hawley, Plewe, Elliott, & DeWitt, 1990). The competing stimuli in the current experiments were equally familiar, with both stimuli matching either relevant or irrelevant aspects of the sample stimuli. All stimuli were repeatedly used throughout the experiment, chosen for each trial from a single stimulus set. Thus novelty was likely not a factor in the biasing of attention to one or the other discrimination target.

The present results highlight the importance of task relevance and demonstrate that the contents of WM interact with selective attention in a very task-specific manner. During delay periods, when matching task relevant and irrelevant information were in competition with one another, performance differences on the perceptual discrimination task favored a representation of the WM task-relevant information. The results are consistent with previous studies by Olivers and colleagues (2006). In that study when the shape of a sample object was held in working memory, a shape-matching distractor affected subsequent search, but a color-matching distractor did not. Similarly, when the color of the sample object was remembered, the matching color interfered, but the matching shape did not. The current study demonstrated an additional level of specificity for this interaction. When both the discrimination target and distractor contained matching task relevant information, the addition of matching task irrelevant information had no effect on biasing (e.g. CONJUNCTION Targets during the OR, WHAT, and WHERE trials of Experiments 2, 3, 4, and 5). Furthermore, selective attention was biased to the CONJUNCTION targets relative to other target types only during the AND WM task. During the AND working memory task trials, subjects were specifically queried at test regarding which pattern had been presented in which location. While this integration of spatial and nonspatial information is possible during the WHAT, WHERE, and OR tasks, maintenance of this information is not necessary for performance. In Experiment 2, both the CONJUNCTION and NONCONJUNCTION Targets contained matching spatial and nonspatial information. During the AND task, attentional selection favored the CONJUNCTION Target. During the OR task, however, no differences were observed. This result suggests that the integration of spatial and nonspatial information for active working memory maintenance occurs only when necessary for completion of the WM task goal. When this conjunction is irrelevant to the task, WM maintenance apparently does not represent the information, insofar as attention is not biased to this matching information in a perceptual stimulus during the memory delay. These findings inform us as to the nature of information representation in WM and the selectivity with which it interacts with attention.

These results may seem at odds with previous studies of irrelevant information in short-term and working memory tasks. Previous studies have demonstrated that spatial configuration plays an integral role in visual short-term memory with delays of 500 to 1000ms even when such information is not explicitly relevant (Jiang et al., 2000; Hommel, 2002; Hollingworth, 2007). Similar effects of irrelevant temporal grouping or color context have been observed in a working memory updating task (Postle, 2003). In all of these studies, the effects of irrelevant contextual information are seen during the memory task test performance. Such effects could be due to the irrelevant sample information having an effect during perceptual processing of the memory test stimuli. Our tasks were designed to probe the actively maintained representations by examining the interaction between working memory and attention during the memory delay. Furthermore, our discrimination procedure was specifically designed to create competition between relevant and irrelevant features of individual items maintained in WM rather than relationships among items (Vidal, Gauchou, Tallon-Baudry & O’Regan, 2005). Both the relevant and the irrelevant information were present during the encoding of the memory sample set and task-relevancy effects were symmetrical regardless of whether the irrelevant information was the identity, the location, or the relationship between the two. In the previous studies, the effect was tested only unidirectionally: irrelevant context affected memory performance.

Another potentially important difference between the current study and these previous studies that did demonstrate effects of irrelevant spatial and contextual on working memory is the delay length. The delay length in most of these studies was much shorter (1 second) than in the current study. In the current study, the cues appeared 1000–3000ms after the disappearance of the sample stimuli. These previous studies indicate that contextual information may be represented for some period of time within some component of short-term or WM. Irrelevant features of individual objects have also been shown to influence attentional capture with very brief memory delays (Soto & Humphreys, 2009). The current results, however, show that the actively maintained representations of individual items that influence biases in attention after more than one second of memory delay specifically reflect relevant features over irrelevant ones. Resolution of competition between relevant and irrelevant information may be the result of a weaker or a slower influence of the irrelevant information rather than a lack of influence (Johnson, Hutchison & Neill, 2001). However, in Experiments 3, 4, and 5 any influence of the irrelevant information in addition to the relevant information was not strong enough to be detectable regarding interactions with selective attention.

Task instructions and seemingly small details of experimental design have the potential to change the nature of WM representations. One potentially important factor in whether irrelevant information is maintained in WM may be the frequency of task switches. If the WM tasks were randomly intermixed trial by trial, rather than blocked as in the current experiment, perhaps subjects would have settled on a WM representation that could by used to perform either of the tasks rather than opting to maintain distinct types of representations for each task and dynamically switching between the appropriate maintenance task sets on a trial-by-trial basis. This possibility could be tested in future studies. Regardless of whether the type of WM representation is modified on each task trial, or only when faced with repeated trials in a task block, however, the current results indicate that WM representations are indeed malleable, and can be created to best suit the needs and goals of the organism. The current results indicate that irrelevant information is not maintained in WM when it is not immediately relevant and when it is not advantageous to maintain it in order to overcome potential switch costs in anticipated future trials.

It is unlikely that our results can be attributed to verbal recoding in working memory. The abstract patterns used in the current study were designed to be difficult to verbally label. Even with images that have familiar associated verbal labels, participants tend to use dual visual and verbal codes for working memory rather than converting to a purely verbal code (Rämä, Sala, Gillen, Pekar, & Courtney, 2001). Although some highly imageable, familiar verbal cues have been shown to affect selective attention (Soto & Humphreys, 2007), nonverbalizable visual working memory representations have a more effective influence on selective attention than do verbalizable visual representations (Olivers et al., 2006). Thus, it is possible, indeed likely, that some of our subjects may have supplemented their visual working memory representations with rehearsed verbal descriptions. However, this possibility does not change the interpretation that the current results reflect the representation of information in visual working memory.

In accord with previous behavioral (e.g. Baddeley, 1998; Della Sala, Gray, Baddeley, & Allamano, 1999), neuroimaging (e.g. Sala & Courtney, 2007), and neurophysiology results (review in Levy & Goldman-Rakic, 2000), the findings suggest that visual WM can be fractionated into separable stores for spatial and nonspatial information. On the other hand, the current results also suggest that the storage of this information is neither separately maintained nor combined into an integrated whole de facto (Rao, Rainer & Miller, 1997). Rather, spatial and nonspatial information is combined only when integration is itself task-relevant. There appears to be an optional interaction between the separate storage systems for spatial and nonspatial information within WM, resulting in a malleable representation scheme and the ability to create an integrated representation if the task demands it (Sala & Courtney, 2007). The results also have implications for cognitive models proposing distinct processing and storage modules in visuo-spatial WM (Darling, Della Sala & Logie, 2007), suggesting that while processing demands such as selective attention can display some independence from memory storage, these two systems are highly interactive (Courtney, Roth & Sala, 2007).

Acknowledgments

This research was supported by the National Institute of Mental Health (R01 MH61625). The authors thank Drs. Steve Yantis and Howard Egeth for helpful discussions during the design of these experiments and editing of the manuscript.

References

  1. Awh E, Jonides J, Reuter-Lorenz PA. Rehearsal in spatial working memory. Journal of Experimental Psychoology: Human Perception and Performance. 1998;24(3):780–790. doi: 10.1037//0096-1523.24.3.780. [DOI] [PubMed] [Google Scholar]
  2. Baddeley A. Recent developments in working memory. Current Opinion Neurobiology. 1998;8(2):234–238. doi: 10.1016/s0959-4388(98)80145-1. [DOI] [PubMed] [Google Scholar]
  3. Baddeley A, Hitch GJ. Working Memory. In: Bower GA, editor. Recent advances in learning and memory. Vol. 8. New York: Academic Press; 1974. pp. 47–90. [Google Scholar]
  4. Carlesimo GA, Perri R, Turriziani P, Tomaiuolo F, Caltagirone C. Remembering what but not where: independence of spatial and visual working memory in the human brain. Cortex. 2001;37(4):519–534. doi: 10.1016/s0010-9452(08)70591-4. [DOI] [PubMed] [Google Scholar]
  5. Chelazzi L, Duncan J, Miller EK, Desimone R. Responses of neurons in inferior temporal cortex during memory-guided visual search. Journal of Neurophysiology. 1998;80(6):2918–2940. doi: 10.1152/jn.1998.80.6.2918. [DOI] [PubMed] [Google Scholar]
  6. Courtney SM, Petit L, Maisog JM, Ungerleider LG, Haxby JV. An area specialized for spatial working memory in human frontal cortex. Science. 1998;279(5355):1347–1351. doi: 10.1126/science.279.5355.1347. [DOI] [PubMed] [Google Scholar]
  7. Courtney SM, Roth JK, Sala JB. A Hierarchical Biased-Competition Model of Domain-Dependent Working Memory Maintenance and Executive Control. In: Osaka Naoyuki, Logie Robert, D’Esposito Marl., editors. The Cognitive Neuroscience of Working Memory. Oxford University Press; 2007. [Google Scholar]
  8. Darling Stephen, Della Sala Sergio, Logie Robert H. Behavioural evidence for separating components within visuo-spatial working memory. Cognitive Processing. 2007;8(3):175–181. doi: 10.1007/s10339-007-0171-1. [DOI] [PubMed] [Google Scholar]
  9. Della Sala S, Gray C, Baddeley A, Allamano N, Wilson L. Pattern span: a tool for unwelding visuo-spatial memory. Neuropsychologia. 1999;37(10):1189–1199. doi: 10.1016/s0028-3932(98)00159-6. [DOI] [PubMed] [Google Scholar]
  10. Downing PE. Interactions between visual working memory and selective attention. Psychological Science. 2000;11(6):467–473. doi: 10.1111/1467-9280.00290. [DOI] [PubMed] [Google Scholar]
  11. Downing PE, Dodds CM. Competition in visual working memory for control of search. Visual Cognition. 2004;11(6):689–703. [Google Scholar]
  12. Duff SC, Logie RH. Storage and processing in visuo-spatial working memory. Scandinavian Journal of Psychology. 1999;40(4):251–259. doi: 10.1111/1467-9450.404124. [DOI] [PubMed] [Google Scholar]
  13. Folk CL, Remington R. Can new objects override attentional control settings? Perception and Psychophysics. 1999;61(4):727–739. doi: 10.3758/bf03205541. [DOI] [PubMed] [Google Scholar]
  14. Hollingworth A. Object-position binding in visual memory for natural scenes and object arrays. Journal of Experimental Psychology: Human Perception and Performance. 2007;33(1):31–47. doi: 10.1037/0096-1523.33.1.31. [DOI] [PubMed] [Google Scholar]
  15. Hommel B. Responding to object files: automatic integration of spatial information revealed by stimulus-response compatibility effects. Quarterly Journal of Experimental Psychology A. 2002;55(2):567–580. doi: 10.1080/02724980143000361. [DOI] [PubMed] [Google Scholar]
  16. Huang L, Pashler H. Working memory and the guidance of visual attention: Consonance-driven orienting. Psychonomic Bulletin & Review. 2007;14(1):148–153. doi: 10.3758/bf03194042. [DOI] [PubMed] [Google Scholar]
  17. Jiang Yuhong, Olson Ingrid R, Chun Marvin M. Organization of visual short-term memory. Journal of Experimental Psychology: Learning, Memory, and Cognition. 2000;26(3):683–702. doi: 10.1037//0278-7393.26.3.683. [DOI] [PubMed] [Google Scholar]
  18. Johnson JD, Hutchison KA, Neill WT. Attentional capture by irrelevant color singletons. Journal of Experimental Psychology: Human Perception and Performance. 2001 Aug;27(4):841–847. doi: 10.1037//0096-1523.27.4.841. [DOI] [PubMed] [Google Scholar]
  19. Johnston WA, Hawley KJ, Plewe SH, Elliott JM, DeWitt MJ. Attention capture by novel stimuli. Journal of Experimental Psychology General. 1990;119(4):397–411. doi: 10.1037//0096-3445.119.4.397. [DOI] [PubMed] [Google Scholar]
  20. Levy R, Goldman-Rakic PS. Segregation of working memory functions within the dorsolateral prefrontal cortex. Experimental Brain Research. 2000;133:23–32. doi: 10.1007/s002210000397. [DOI] [PubMed] [Google Scholar]
  21. Logie R. Visuo-spatial working memory. Hove: Lawrence Erlbaum Associates, Inc; 1995. [Google Scholar]
  22. Luck SJ, Vogel EK. The capacity of visual working memory for features and conjunctions. Nature. 1997;390(6657):279–281. doi: 10.1038/36846. [DOI] [PubMed] [Google Scholar]
  23. Magnussen S. Low-level memory processes in vision. Trends in Neuroscience. 2000;23(6):247–51. doi: 10.1016/s0166-2236(00)01569-1. [DOI] [PubMed] [Google Scholar]
  24. Olivers CNL, Meijer F, Theeuwes J. Feature-Based Memory-Driven Attentional Capture: Visual Working Memory Content Affects Visual Attention. Journal of Experimental Psychology: Human Perception and Performance. 2006;32(5):1243–1265. doi: 10.1037/0096-1523.32.5.1243. [DOI] [PubMed] [Google Scholar]
  25. Postle BR. Context in verbal short-term memory. Memory and Cognition. 2003;31(8):1198–207. doi: 10.3758/bf03195803. [DOI] [PubMed] [Google Scholar]
  26. Rämä P, Sala JB, Gillen JS, Pekar JJ, Courtney SM. Dissociation of the Neural Systems for Working Memory Maintenance of Verbal and Nonspatial Visual Information. Cognitive, Affective, and Behavioral Neuroscience. 2001;1(2):161–171. doi: 10.3758/cabn.1.2.161. [DOI] [PubMed] [Google Scholar]
  27. Rao SC, Rainer G, Miller EK. Integration of what and where in the primate prefrontal cortex. Science. 1997;276(5313):821–824. doi: 10.1126/science.276.5313.821. [DOI] [PubMed] [Google Scholar]
  28. Sala JB, Courtney SM. Binding of what and where during working memory maintenance. Cortex. 2007;43(1):5–21. doi: 10.1016/S0010-9452(08)70442-8. [DOI] [PubMed] [Google Scholar]
  29. Sala JB, Rämä P, Courtney SM. Functional topography of a distributed neural system for spatial and nonspatial information maintenance in working memory. Neuropsychologia. 2003;41(3):341–356. doi: 10.1016/S0028-3932(02)00166-5. [DOI] [PubMed] [Google Scholar]
  30. Smith EE, Jonides J. Neuroimaging analyses of human working memory. Proceedings of the National Academy of Sciences U S A. 1998;95(20):12061–12068. doi: 10.1073/pnas.95.20.12061. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Soto D, Humphreys GW, Blanco MJ. Early, Involuntary Top-Down Guidance of Attention From Working Memory. Journal of Experimental Psychology: Human Perception and Performance. 2005;31(2):248–261. doi: 10.1037/0096-1523.31.2.248. [DOI] [PubMed] [Google Scholar]
  32. Soto D, Humphreys GW. Seeing the content of the mind: Enhanced awareness through working memory in patients with visual extinction. Proceedings of the National Academy of Sciences of the United States of America. 2006;103(12):4789–4792. doi: 10.1073/pnas.0510718103. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Soto D, Humphreys GW. Automatic Guidance of visual attention from verbal working memory. Journal of Experimental Psychology: Human Perception and Performance. 2007;33(3):730–757. doi: 10.1037/0096-1523.33.3.730. [DOI] [PubMed] [Google Scholar]
  34. Soto D, Humphreys GW. Automatic selection of irrelevant object features through working memory: Evidence for top-down attentional capture. Experimental Psychology, 1618–3169. 2009;56(3):165–172. doi: 10.1027/1618-3169.56.3.165. [DOI] [PubMed] [Google Scholar]
  35. Vidal JR, Gauchou HL, Tallon-Baudry C, O’Regan JK. Relational information in visual short-term memory: the structural gist. Journal of VIsion. 2005;25;5(3):244–56. doi: 10.1167/5.3.8. [DOI] [PubMed] [Google Scholar]
  36. Vogel EK, Woodman GF, Luck SJ. Storage of features, conjunctions and objects in visual working memory. Journal of Experimental Psycholology Human Perception and Performance. 2001;27(1):92–114. doi: 10.1037//0096-1523.27.1.92. [DOI] [PubMed] [Google Scholar]
  37. Wallis JD, Anderson KC, Miller EK. Single neurons in prefrontal cortex encode abstract rules. Nature. 2001;411(6840):953–956. doi: 10.1038/35082081. [DOI] [PubMed] [Google Scholar]
  38. Wheeler ME, Treisman AM. Binding in short-term visual memory. Journal of Experimental Psychology General. 2002;131(1):48–64. doi: 10.1037//0096-3445.131.1.48. [DOI] [PubMed] [Google Scholar]
  39. White IM, Wise SP. Rule-dependent neuronal activity in the prefrontal cortex. Experimental Brain Research. 1999;126(3):315–335. doi: 10.1007/s002210050740. [DOI] [PubMed] [Google Scholar]
  40. Woodman GF, Luck SJ. Visual search is slowed when visuospatial working memory is occupied. Psychonomic Bulletin and Review. 2004;11(2):269–274. doi: 10.3758/bf03196569. [DOI] [PubMed] [Google Scholar]
  41. Woodman GF, Luck SJ. Do the Contents of Visual Working Memory Automatically Influence Attentional Selection During Visual Search? Journal of Experimental Psychology: Human Perception and Performance. 2007;33(2):363–377. doi: 10.1037/0096-1523.33.2.363. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Woodman G, Vogel E, Luck S. Visual Search remains efficient when visual working memory is full. Psychological Science. 2001;12(3):219–224. doi: 10.1111/1467-9280.00339. [DOI] [PubMed] [Google Scholar]

RESOURCES