Abstract
Theories of attention and visual search explain how attention is guided toward objects with known target features. But can attention be directed away from objects with a feature known to be associated only with distractors? Most studies have found that the demand to maintain the to-be-avoided feature in visual working memory biases attention toward matching objects rather than away from them. In contrast, Arita, Carlisle, and Woodman (2012) claimed that attention can be configured to selectively avoid objects that match a cued distractor color, and they reported evidence that this type of negative cue generates search benefits. However, the colors of the search array items in Arita et al. were segregated by hemifield (e.g., blue items on the left, red on the right), which allowed for a strategy of translating the feature-cue information into a simple spatial template (e.g., avoid right, or attend left). In the present study, we replicated the negative cue benefit using the Arita et al. method (albeit within a subset of participants who reliably used the color cues to guide attention). Then, we eliminated the benefit by using search arrays that could not be grouped by hemifield. Our results suggest that feature-guided avoidance is implemented only indirectly, in this case by translating feature-cue information into a spatial template.
Keywords: visual attention, visual working memory, visual search, attentional control, exclusionary template
Most theories of attention propose that goal-directed orienting is implemented by means of an attentional template specifying the features of task-relevant items (Bundesen, 1990; Duncan & Humphreys, 1989; Wolfe, 1994). Consistent with this assumption, numerous studies have demonstrated that participants are able to use feature information (e.g., color) to limit attention to task-relevant objects in a search array (Beck, Hollingworth, & Luck, 2012; Vickery, King, & Jiang, 2005; Wolfe, Horowitz, Kenner, Hyle, & Vasan, 2004). In addition, most theories propose that an attentional template is implemented by an active visual working memory representation (VWM) of the relevant features (Bundesen, 1990; Desimone & Duncan, 1995). The VWM representation biases perceptual competition among objects in favor of matching items (Desimone & Duncan, 1995; Hollingworth, Matsukura, & Luck, 2013).
Although VWM-based attentional biases are well documented, there is debate over whether they are automatic. Memory-matching objects often capture attention even when the matching feature is always associated with distractors (e.g., Olivers, Meijer, & Theeuwes, 2006; Soto, Heinke, Humphreys, & Blanco, 2005). Because VWM maintenance involves sustained activation of feature-specific neural populations in sensory cortex (Harrison & Tong, 2009; for a review, see Postle, 2006; Serences, Ester, Vogel, & Awh, 2009), it may not be possible to maintain an active VWM representation that does not facilitate perceptual processing of matching items. However, Woodman and Luck (2007) argued that VWM can be configured flexibly either to prioritize or deprioritize memory-matching objects. They found faster response times (RTs) when a majority of the distractors matched a color in memory, suggesting use of an exclusionary feature template. More recently, Arita, Carlisle, and Woodman (2012) consistently found evidence in favor of an exclusionary template. They presented a circular search array with items in the two hemifields drawn in different colors (Figure 1). A color cue at the beginning of the trial was either negative (distractor color), positive (target color), or neutral (color not present in the array), and cue condition was blocked. Search RT was faster in both the positive and negative cue conditions compared with the neutral condition. Participants were able to use the negative cue information to restrict search to relevant items, though not as efficiently as in the positive cue condition.
The results from Arita et al. (2012) suggest that attention is not oriented automatically to memory-matching items and that, moreover, participants can avoid memory-matching items when they have sufficient incentive. However, there is a clear alternative explanation of the Arita et al. (2012) results that would not require configuring a feature-based, exclusionary template. In their study, search array colors were always segregated by hemifield. Participants may have used this relationship to convert the negative feature cue into a simple spatial template specifying the relevant hemifield. This process could be similar to the “search and destroy” mechanism of inhibition described by Moher and Egeth (2012), in which attention is first directed to an object with the to-be-ignored feature before other objects with that feature can be avoided. However, Moher and Egeth did not specify whether avoidance is implemented by a spatial or feature-based template. We propose that, in the present context, avoidance is implemented by spatial recoding of feature information, and we are agnostic as to whether this depends on directing attention first to a single cue-matching object. Resolving whether avoidance is implemented directly or indirectly is central to understanding how VWM modulates attentional selection. Direct, feature-based avoidance would suggest that an active VWM representation could inhibit the processing of or bias perceptual competition against matching items. Indirect, spatially-mediated avoidance would be consistent with a fundamentally facilitatory relationship between VWM and perceptual selection. Without an efficient means to spatially recode the search array, it may not be possible to implement an exclusionary search template.
To test this, we first replicated the results obtained by Arita et al. (2012) using their method. Then, we modified the paradigm to manipulate the ease with which participants could spatially recode the cue information. We were able to replicate the basic avoidance effect of Arita et al. (2012), but this effect was eliminated when spatial recoding was rendered inefficient.
Experiment 1
Participants
Twenty-nine participants (18-30 years old; 12 female) from the University of Iowa community participated and received either course credit or pay.
Stimuli and Procedure
Stimuli were presented on an LCD monitor (100 Hz) at a distance of 88 cm. Each circular search array (5.3° radius) contained 12 Landolt-square objects (0.8° × 0.8°; line thickness 0.1°), with six objects in each hemifield. The two colors (one for each hemifield) were drawn randomly from a set of three (red, green, blue). The target had a top or bottom gap (0.2°). Distractors had a left or right gap.
Each trial began with a central fixation cross (500 ms). A cue square (0.8° × 0.8°; presented 1.0° above fixation) appeared for 100 ms, followed by a 900-ms blank screen and the search array. The cue-square either indicated the distractor color (negative), the target color (positive), or a color absent from the array (neutral). Participants were instructed to locate and respond to the target item as quickly as possible by pressing one of two buttons to indicate gap location. The search array remained visible until response or until 5 seconds had elapsed. Cue condition was blocked, and the order of conditions was counterbalanced across participants. For each condition block, there were 15 practice trials, then three sub-blocks of 75 trials. At the end of practice and at the end of each sub-block, participants were given average accuracy and RT feedback.
Results and Discussion
Three participants were excluded for accuracy less than 75% in one or more conditions. Mean accuracy for the remaining participants (N=26) was 97%, 98%, and 98%, for negative, neutral, and positive, respectively. RTs 2.5 SD beyond a participant's condition mean were excluded as outliers (2.7% of trials). These exclusions did not alter the pattern of results.
Replicating Arita et al. (2012), mean correct RT was lower in the positive-cue condition than in the neutral-cue condition, t(25)=4.65, p<.001 (Figure 2). Unlike in Arita et al. (2012), there was no RT advantage for the negative-cue condition compared with the neutral-cue condition, t(25)=0.77, p=.448. However, there were substantial individual differences in the pattern of cuing effects: participants who demonstrated a strong positive cue benefit also exhibited a negative cue benefit. To analyze this pattern, we calculated the positive and negative cuing effect for each participant relative to their neutral baseline (with half of the neutral trials used to calculate the positive cue effect and half used to calculate the negative cue effect, so that the two measures were independent). The two effects were strongly correlated (Figure 3), r=.67, p<.001. We split the participants into two groups based on the magnitude of the positive cue benefit. Little or no positive cue benefit indicates poor use of cue information, making the negative cue effect difficult to interpret. It is possible that some participants perceived little advantage to using the color cue, because the target item could be identified solely by gap location throughout the experiment, and the color cue information had to be applied differently in each block. Participants in the low group (N=13) did not show either a positive or a negative cuing benefit (p>.1 for both comparisons; Figure 4). However, the high group (N=13) demonstrated a significant negative cue benefit (t(12)=2.73, p=.018), in addition to a significant positive cue benefit (t(12)=9.97, p<.001). Thus, within the group of participants that consistently used cue information to guide search, Experiment 1 replicated the Arita et al. (2012) results.
Experiment 2
To examine whether the negative cue benefit here and in Arita et al. (2012) was caused by translating a feature cue into a spatial template, we manipulated whether the colored items in the search array were spatially segregated (as in Experiment 1) or intermixed (Figure 5). If participants can implement direct, feature-guided avoidance, then a negative cuing benefit should be observed in both types of spatial configuration. If, however, guidance by a negative cue can be implemented only by translation to a spatial template, this process should be hindered when the items are spatially intermixed, reducing or eliminating the negative cue benefit.
Participants
Eighteen participants (18-30 years old; 11 female) from the University of Iowa community participated and received either course credit or pay.
Stimuli and Procedure
Stimuli and procedure were the same as Experiment 1, except that there were seven total colors (red, green, blue, cyan, magenta, yellow, and grey), though only two colors appeared in any one search display, and the search displays were arranged differently. Items were no longer arranged on an invisible circle but were assigned random locations in two rectangular regions (4.4° × 13.9°, nearest edge 1.8° to the left/right of center), one in each hemifield (Figure 5). Placing items in rectangular regions, rather than on an invisible circle, allowed greater variation in generating intermixed displays while also keeping the two hemifields distinct.1 The two colors were either segregated by hemifield or intermixed (three items of each color in each hemifield). To compare feature-based and location-based guidance, half of the trials used a spatial cue instead of a feature cue. The spatial cue was negative (arrow pointed to the hemifield that did not contain the target), neutral (double-headed arrow), or positive (arrow pointed to hemifield containing the target). Cue condition (negative, neutral, positive), cue type (color, location), and array type (segregated, intermixed) were blocked. Cue condition order was counterbalanced across participants, but the order of cue type and array type blocks was fixed within each cue condition. Participants began the session with twelve practice trials (one trial of each possible trial type), then completed twelve blocks of 55 trials each.
Results and Discussion
Manual response accuracy was uniformly high across all twelve conditions (M=98.7%) and did not differ between any comparisons of interest. RT trimming (2.5 SD) eliminated 2.8% of trials but did not alter the pattern of results.
Location Cue
Positive and negative spatial cues both generated search RT benefits relative to the neutral condition, t(17)=3.65, p=.002; t(17)=2.27, p=.037, respectively (Figure 6). In neither case was the benefit modulated by the spatial distribution of colors in the display, (ps>.24). These results demonstrate that participants were able to efficiently implement a negative spatial template (consistent with Munneke, Van Der Stigchel, & Theeuwes, 2008) based on the type of location information that could have been inferred from the distribution of colored items in the segregated arrays of Experiment 1 and Arita et al. (2012).
Color Cue
There was a reliable positive cue benefit (t(17)=3.07, p=.006) but no negative cue benefit (t(17)=0.51, p=.617), replicating the pattern of results in Experiment 1 (Figure 6). Moreover, the cuing effects were not modulated by the spatial distribution of colored items, (ps>.27). However, there was again a strong correlation between participants’ positive and negative cue effects, r=.62, p=.006, indicating individual differences in the utilization of feature-cue information (Figure 7). When the RT data were split by the magnitude of the positive cue effect (using only trials from the positive color cue condition), the high group (N=9) reliably demonstrated a negative cue benefit in the color cue condition when the array was segregated (t(8)=2.64, p=.030; Figure 8A), replicating the results from Experiment 1, but not when the array was intermixed (t(8)=0.15, p=.886; Figure 8B). The low group (N=9) did not demonstrate a negative or positive cue benefit for either array type (p>.86; Figure 8). In fact, mean RTs in the low group were higher in the negative cue condition than in the neutral condition (t(8)=2.39, p=.044). In sum, participants were able to benefit from the negative cue information, but only if they demonstrated a benefit from a positive cue and only if color cue information could be easily recoded into a spatial template (segregated array type).
General Discussion
In several theories of attention, it is possible to configure a VWM-template to either prioritize or deprioritize objects that match a particular feature value. For example, in Bundesen's (1990) Theory of Visual Attention, the pertinence value of one color could be set to a level either above or below that of other colors, implementing a bias to select for or against that color. In practice, however, implementing a negative feature template may be difficult, if not impossible. To avoid attending to a particular feature, a participant must remember the feature value, typically done by maintaining that value in VWM. Maintenance of a feature in VWM involves sustained activation of feature-specific neural populations in sensory cortex (Harrison & Tong, 2009; Postle, 2006; Serences et al., 2009), which is likely to interact with subsequent sensory processing, biasing competition for selection in favor of the remembered feature (Bundesen, 1990; Desimone & Duncan, 1995; Duncan & Humphreys, 1989; Wolfe, 1994). Thus, attention will be drawn to objects containing the to-be-avoided feature, a consequence of the demand to remember which feature to avoid. This type of capture effect has been observed in numerous studies: Visual attention is oriented to memory-matching objects, even when these objects should be avoided (Downing & Dodds, 2004; Folk, Remington, & Johnston, 1992; Han & Kim, 2009; Hollingworth & Luck, 2009; Hollingworth et al., 2013; Olivers et al., 2006; Olivers, 2009; Soto et al., 2005; Soto, Humphreys, & Heinke, 2006).
In contrast, a handful of studies have suggested that this capture effect can be overridden, or even reversed, suggesting the content of VWM can be used flexibly to select against memory-matching items (Arita et al., 2012; Han & Kim, 2009; Moher & Egeth, 2012; Woodman & Luck, 2007). We have argued that the most compelling evidence in support of a negative feature template can be explained instead by spatial recoding. Arita et al. (2012) observed a substantial benefit for negative color cue trials in a visual search task. However, the displays had colored items segregated by hemifield, allowing simple spatial recoding of the side of the display to be avoided (or, equivalently, the side to be attended). If the negative cue benefit resulted from direct feature-guided avoidance, it should not depend strongly on the spatial arrangement of items. However, when we spatially intermixed colored objects within the search array (Experiment 2), making location-grouping more difficult, participants no longer demonstrated a negative cue benefit. Without a means to efficiently recode the negative cue information into a simple spatial template, they were unable to use the negative feature cue to direct attention away from irrelevant objects.
Furthermore, spatial recoding provides a plausible explanation for the findings of two other experiments in which negative feature cues generated benefits in visual search. In Han and Kim (2009) and Moher and Egeth (2012), participants saw a negative color cue, followed by a set of placeholders corresponding to the upcoming search locations. Importantly, the placeholders had the same colors as the subsequent search items. During the placeholder display, participants had ample time to identify the location of the color-matching item and to create a simple spatial template marking that location as to-be-avoided (or marking other locations as to be-attended). When participants were cued to avoid a particular location in a similar paradigm, Munneke et al. (2008) found a cuing benefit, indicating participants were able to successfully avoid searching the cued location. Although we cannot be certain that spatial recoding was the source of the negative-cue benefit in Han and Kim (2009) and in Moher and Egeth (2012), this clear alternative would need to be eliminated before those data could provide strong evidence in favor of feature-based avoidance.
In sum, our data are consistent with a fundamentally facilitatory relationship between VWM and perceptual selection. We usually know the features of the object we are looking for. Maintaining those features in VWM facilitates selection of matching items to guide attention toward plausible objects. It is rarely the case that an object is best described by a feature that it lacks. It may be possible to implement the latter type of guidance under some circumstances, but the present data suggest that this guidance is at best indirect, implemented by translating feature information into a spatial template.
Acknowledgments
This study was supported by a grant from NEI (R01EY017356) and by an NSF Graduate Research Fellowship to Valerie M. Beck.
Footnotes
Because of the design changes, we first replicated Experiment 1 using the Experiment 2 design parameters (7 possible colors, items placed in rectangular regions instead of on a circle), but with the same trial types as in Experiment 1. We replicated the Experiment 1 results. When all subjects were included, there was a reliable positive cue benefit, t(15)=2.72, p=.008, but no negative cue benefit, t(15)=0.04, p=.484. After performing a median split on the basis of the positive cue benefit, the high group demonstrated significantly faster RTs on both positive cue trials, t(7)=4.22, p=.002, and negative cue trials, t(7)=2.35, p=.025, compared with neutral trials. However, the low group showed neither a positive cue benefit (t(7)=0.36, p=.365) nor a negative cue benefit, with a non-reliable trend toward a negative cue cost (t(7)=1.76, p=.061).
References
- Arita JT, Carlisle NB, Woodman GF. Templates for rejection: Configuring attention to ignore task-irrelevant features. Journal of Experimental Psychology. Human Perception and Performance. 2012;38(3):580–4. doi: 10.1037/a0027885. doi:10.1037/a0027885. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Beck VM, Hollingworth A, Luck SJ. Simultaneous Control of Attention by Multiple Working Memory Representations. Psychological Science. 2012 Jul; doi: 10.1177/0956797612439068. doi:10.1177/0956797612439068. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bundesen C. A theory of visual attention. Psychol Rev. 1990;97(4):523–547. doi: 10.1037/0033-295x.97.4.523. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/2247540. [DOI] [PubMed] [Google Scholar]
- Desimone R, Duncan J. Neural mechanisms of selective visual attention. Annu Rev Neurosci. 1995;18:193–222. doi: 10.1146/annurev.ne.18.030195.001205. doi:10.1146/annurev.ne.18.030195.001205. [DOI] [PubMed] [Google Scholar]
- Downing P, Dodds C. Competition in visual working memory for control of search. Visual Cognition. 2004;11(6):689–703. doi:10.1080/13506280344000446. [Google Scholar]
- Duncan J, Humphreys GW. Visual search and stimulus similarity. Psychol Rev. 1989;96(3):433–458. doi: 10.1037/0033-295x.96.3.433. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/2756067. [DOI] [PubMed] [Google Scholar]
- Folk CL, Remington RW, Johnston JC. Involuntary covert orienting is contingent on attentional control settings. Journal of Experimental Psychology. Human Perception and Performance. 1992;18(4):1030–44. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/1431742. [PubMed] [Google Scholar]
- Han SW, Kim M-S. Do the contents of working memory capture attention? Yes, but cognitive control matters. Journal of Experimental Psychology. Human Perception and Performance. 2009;35(5):1292–302. doi: 10.1037/a0016452. doi:10.1037/a0016452. [DOI] [PubMed] [Google Scholar]
- Harrison S. a, Tong F. Decoding reveals the contents of visual working memory in early visual areas. Nature. 2009;458(7238):632–5. doi: 10.1038/nature07832. doi:10.1038/nature07832. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hollingworth A, Luck SJ. The role of visual working memory (VWM) in the control of gaze during visual search. Attention, Perception & Psychophysics. 2009;71(4):936–949. doi: 10.3758/APP.71.4.936. doi:10.3758/APP.71.4.936. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hollingworth A, Matsukura M, Luck SJ. Visual working memory modulates rapid eye movements to simple onset targets. Psychological Science. 2013;24(5):790–6. doi: 10.1177/0956797612459767. doi:10.1177/0956797612459767. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Moher J, Egeth HE. The ignoring paradox: Cueing distractor features leads first to selection, then to inhibition of to-be-ignored items. Attention, Perception & Psychophysics. 2012 doi: 10.3758/s13414-012-0358-0. doi:10.3758/s13414-012-0358-0. [DOI] [PubMed] [Google Scholar]
- Morey RD. Confidence Intervals from Normalized Data : A correction to Cousineau ( 2005 ). Tutorial in Quantitative Methods for Psychology. 2008;4(2):61–64. [Google Scholar]
- Munneke J, Van Der Stigchel S, Theeuwes J. Cueing the location of a distractor: an inhibitory mechanism of spatial attention? Acta Psychol (Amst) 2008;129(1):101–107. doi: 10.1016/j.actpsy.2008.05.004. doi:10.1016/j.actpsy.2008.05.004. [DOI] [PubMed] [Google Scholar]
- Olivers CNL. What drives memory-driven attentional capture? The effects of memory type, display type, and search type. Journal of Experimental Psychology: Human Perception and Performance. 2009;35(5):1275–91. doi: 10.1037/a0013896. doi:10.1037/a0013896. [DOI] [PubMed] [Google Scholar]
- Olivers CNL, Meijer F, Theeuwes J. Feature-based memory-driven attentional capture: visual working memory content affects visual attention. Journal of Experimental Psychology: Human Perception and Performance. 2006;32(5):1243–1265. doi: 10.1037/0096-1523.32.5.1243. doi:10.1037/0096-1523.32.5.1243. [DOI] [PubMed] [Google Scholar]
- Postle BR. Working memory as an emergent property of the mind and brain. Neuroscience. 2006;139(1):23–38. doi: 10.1016/j.neuroscience.2005.06.005. doi:10.1016/j.neuroscience.2005.06.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Serences JT, Ester EF, Vogel EK, Awh E. Stimulus-specific delay activity in human primary visual cortex. Psychological Science. 2009;20(2):207–14. doi: 10.1111/j.1467-9280.2009.02276.x. doi:10.1111/j.1467-9280.2009.02276.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Soto D, Heinke D, Humphreys GW, Blanco MJ. Early, involuntary top-down guidance of attention from working memory. Journal of Experimental Psychology: Human Perception and Performance. 2005;31(2):248–261. doi: 10.1037/0096-1523.31.2.248. doi:10.1037/0096-1523.31.2.248. [DOI] [PubMed] [Google Scholar]
- Soto D, Humphreys GW, Heinke D. Working memory can guide pop-out search. Vision Research. 2006;46(6-7):1010–8. doi: 10.1016/j.visres.2005.09.008. doi:10.1016/j.visres.2005.09.008. [DOI] [PubMed] [Google Scholar]
- Vickery TJ, King LW, Jiang YV. Setting up the target template in visual search. Journal of Vision. 2005;5(1):81–92. doi: 10.1167/5.1.8. doi:10:1167/5.1.8. [DOI] [PubMed] [Google Scholar]
- Wolfe JM. Guided Search 2.0 A revised model of visual search. Psychonomic Bulletin & Review. 1994;1(2):202–38. doi: 10.3758/BF03200774. doi:10.3758/BF03200774. [DOI] [PubMed] [Google Scholar]
- Wolfe JM, Horowitz TS, Kenner N, Hyle M, Vasan N. How fast can you change your mind? The speed of top-down guidance in visual search. Vision Res. 2004;44(12):1411–1426. doi: 10.1016/j.visres.2003.11.024. doi:10.1016/j.visres.2003.11.024. [DOI] [PubMed] [Google Scholar]
- Woodman GF, Luck SJ. Do the contents of visual working memory automatically influence attentional selection during visual search? Journal of Experimental Psychology. Human Perception and Performance. 2007;33(2):363–377. doi: 10.1037/0096-1523.33.2.363. doi:10.1037/0096-1523.33.2.363. [DOI] [PMC free article] [PubMed] [Google Scholar]