Negative cues minimize visual search specificity effects

Abstract

Prior target knowledge (i.e., positive cues) improves visual search performance. However, there is considerable debate about whether distractor knowledge (i.e., negative cues) can guide search. Some studies suggest the active suppression of negatively cued search items, while others suggest the initial capture of attention by negatively cued items. Prior work has used pictorial or specific text cues but has not explicitly compared them. We build on that work by comparing positive and negative cues presented pictorially and as categorical text labels using photorealistic objects and eye movement measures. Search displays contained a target (cued on positive trials), a lure from the target category (cued on negative trials), and four categorically-unrelated distractors. Search performance with positive cues resulted in stronger attentional guidance and faster object recognition for pictorial relative to categorical cues (i.e., a pictorial advantage, suggesting specific visual details afforded by pictorial cues improved search). However, in most search performance metrics, negative cues mitigate the pictorial advantage. Given that the negatively cued items captured attention, generated target guidance but mitigated the pictorial advantage, these results are partly consistent with both existing theories. Specific visual details provided in positive cues produce a large pictorial advantage in all measures, whereas specific visual details in negative cues only produce a small pictorial advantage for object recognition but not for attentional guidance. This asymmetry in the pictorial advantage suggests that the down-weighting of specific negatively cued visual features is less efficient than the up-weighting of specific positively cued visual features.

Visual search is embedded in many everyday tasks. To preferentially direct spatial attention to relevant objects or scene regions (i.e., target guidance), people are thought to call to mind a representation of the target item to aid the search process (i.e., Wolfe, 1994). In a lab setting, participants are often asked to remember a positive cue, which provides valid information about the upcoming search target. Models of visual search suggest the information provided in the cue is encoded into an attentional template and maintained in visual working memory (VWM; Bundesen, 1990; Bundensen et al., 2005; Desimone & Duncan, 1995). The target feature information (i.e., the attentional template) selectively biases, or up-weights, relevant target features within the search display to guide attention to similar items (e.g., Desimone & Duncan, 1995; Wolfe, 1994). Cues that designate targets can contain varying amounts of target information and range from pictorial representations (e.g., a picture of a shoe) to text descriptions (e.g., the word “shoe” referring to a specific shoe or any member of the shoe category). The accuracy and specificity of the information in the cue affects later search performance. For example, when the pictorial cue and the target are more visually similar, search performance is stronger, suggesting more effective attentional biasing (e.g., Alexander et al., 2018; Alexander et al., 2019; Hout & Goldinger, 2015; Vickery et al.,2005). Similarly, more specific text cues (e.g., “boot” vs. “footwear”) result in stronger search performance (Maxfield et al., 2014; Schmidt & Zelinsky, 2009). Although pictorial cues are more commonly studied, categorical text cues are more ecologically valid; when searching for something, we rarely see a picture of the target object that precisely matches its appearance. For example, TSA baggage screeners search for the broad category of “weapon” rather than a specific gun or knife, and radiologists do not know in advance precisely how a cancerous nodule will look until it is found (Alexander et al., 2020; Waite et al., 2020). Likewise, when looking for any pen on a cluttered desk, many of the precise visual details (orientation, color, etc.) are unknown. Accordingly, categorical search is especially critical to the study of visual attention in real-world contexts (i.e., Muhl-Richardson et al., 2021).

It is well-established that participants can configure an attentional template in VWM to bias attention towards template-matching features (Carlisle et al., 2011; Luria & Vogel, 2011; Zelinsky & Bisley, 2015). However, it is unclear if participants can use an attentional template to bias attention away from features known to correspond to a distractor. In these cases, improving search by avoiding the distractor would require suppression or “down-weighting” of features held in the template, a negative attentional template (Arita et al., 2012; Beck & Hollingworth, 2015; Beck et al., 2018; Becker et al., 2015). When investigating negative attentional templates, participants are cued with a feature (or features) corresponding to a distractor rather than a target and should therefore be avoided during search. Electrophysiological data suggest that positively and negatively cued visual features are maintained similarly in VWM (de Vries et al., 2017; Rajsic et al., 2020). Conversely, fMRI data showed reduced neural activation in visual areas for negative cues, providing some evidence for the active suppression of negatively cued information (Reeder et al., 2017). Accordingly, the ability to guide search to the target using a negative attentional template depends on how automatically attention is biased toward cued features stored in VWM.

With respect to negative attentional templates’ ability to guide search, there are currently two main viewpoints in the literature. The flexible attentional control hypothesis (Carlisle & Nikta, 2019) suggests negatively cued features can be actively suppressed. The alternative view suggests that negatively cued features cannot be proactively suppressed, we term this view – the initial capture hypothesis.

The flexible attentional control hypothesis

The flexible attentional control hypothesis suggests participants can apply top-down strategic control to suppress negatively cued features and avoid them in the search display (Carlisle & Nitka, 2019). According to the flexible attentional control hypothesis, participants can apply strategic top-down control to increase (up-weight) or decrease (down-weight) the attentional biasing of specific features of the attentional template in VWM (Carlisle & Nikta, 2019). Top-down control of negatively cued features would result in down-weighting features, causing attention to be directed toward the cued distractor less and towards the target more. In related literature, this is termed proactive suppression (for review see, Geng, 2014). Given this, the utility of a negative cue should be most evident when there is a strong strategic benefit to use the cue. For example, some studies show negative cues result in faster search relative to neutral cues (cues that do not provide target or distractor information), suggesting that knowing what not to look for can be better than not having any information at all (Arita et al., 2012; Rajsic et al., 2020). However, the benefit of negative cues over neutral cues is not always observed and may not be apparent at small set sizes in which search is relatively easy (Arita et al., 2012). Response times (RT) also decrease as the number of negative cue-matching distractors in the search array increases, leaving fewer items to inspect (Woodman & Luck, 2007).

However, Beck and Hollingworth (2015) argued that the negative cue benefit observed by Arita et al. (2012) resulted from the task itself. In Arita et al. (2012), the search display was comprised of Landolt Cs of two colors, separated by hemifield. Accordingly, when participants were given a negative color cue, they were able to reject one half of the display, as the target would always appear on the other side of the display in the non-cued color. Instead of active suppression, Beck & Hollingworth (2015) hypothesized that participants could recode the negative feature-based template into a positive spatial template (i.e., attend or avoid left/right), thus making it more useful than a neutral cue that provides no information. Using the method in Arita et al. (2012), they did not find an overall benefit of negative cues relative to neutral cues. However, they did find that participants with the largest positive cue benefit also showed a negative cue benefit, suggesting that some participants used the cues effectively, while others did not. In the second experiment, they added location cues (left/right) and search displays with the colors mixed across hemifields (all blocked). They found a negative cue benefit with the location cues. However, they did not find an overall effect with color cues (see also Becker et al., 2015; Conci et al., 2019, Experiment 1), replicating the results of experiment 1. Instead, they replicated the individual differences, suggesting that some people use the cues and others do not. This was also observed in Carlisle and Nitka (2019).

Carlisle and Nitka (2019, Experiment 2) noted that the participants in Beck & Hollingsworth (2015) might have been unduly prompted to use a spatial recoding strategy by explicitly instructing participants to use this strategy in the location-cue blocks (attend left or attend right). To test this, they removed the location-based cues and included segregated (two-thirds of trials) and intermixed displays (one-third of trials) interleaved throughout the task. These changes restored the negative relative to neutral cue RT benefit. It is unclear if the change in trials (removal of location cues and proportion of segregated trials) or the change in instructions were responsible for the discrepant findings (Beck & Hollingsworth, 2015; Carlisle & Nitka, 2019). However, previous work has shown that explicit instructions have little effect on attentional capture (Olivers, 2009, Experiment 7). Moreover, several other studies have also shown negative cue benefits with spatially intermixed displays, further obfuscating this debate (Reeder et al., 2017; Conci et al., 2019, Experiment 2). Although these results seem conflicting, they demonstrate that small changes to the design can alter the way participants use negative cues and thus the subsequent effect on search performance.

Although RT benefits offer some evidence for the effective use of negative attentional templates, RT differences may arise from the guidance of attention or other task elements (such as the time required to reject distractors or recognize the target). In line with the flexible attentional control hypothesis, neurophysiological evidence suggests the benefit arises from an improved ability to direct spatial attention. To assess if participants could suppress the negative cue and shift attention directly to the target, an N2pc (an event-related potential indicative of spatial attention shifts; Eimer, 1996) paradigm was used with positive, negative, or neutral color cues (Carlisle & Nitka, 2019, Experiment 1). Using the Arita et al. (2012) method, they predicted suppression of the negatively cued color and attention directed towards the target, resulting in an N2pc to the target side. Conversely, if negatively cued information cannot be suppressed (i.e., Beck et al., 2018; de Vries et al., 2017; Moher & Egeth, 2012) attention would first be directed to the negatively cued color, resulting in an N2pc to the distractor side, followed by a “corrective” N2pc to the target side (Woodman & Luck 2003; Grubert & Eimer, 2016). Consistent with the flexible attentional control hypothesis, both negative and positive cues elicited an N2pc toward the target side, suggesting participants could actively suppress the distractor side of the display and direct attention to the target side. However, the N2pc following a negative cue was significantly delayed by 41ms relative to the N2pc following a positive cue. The implications of this are somewhat unclear, but participants may have filtered out the distracting information before covertly shifting attention and selecting the target, resulting in a delayed N2pc to the target side of the display.

The initial capture hypothesis

The second main viewpoint, which we will call the initial capture hypothesis, suggests that negatively cued features are held in VWM and automatically bias spatial attention towards those features, resulting in the initial capture of attention (Beck et al., 2018; de Vries et al., 2017; Moher & Egeth, 2012) but later avoidance of cue matching items (however, later avoidance may not be contingent upon initial capture, see Beck et al., 2018). Moher & Egeth (2012) found an increase in RTs following negative, pictorial, and text cues relative to neutral cues. Results from the embedded probe-dot task suggested that the increase in RT resulted from early selection but later inhibition of the negatively cued information. Moher & Egeth termed this pattern of results “search and destroy,” in which attention was initially captured by the distracting information but is effectively avoided later in the trial. This has also been termed “reactive suppression” in related literature (for review, see Geng, 2014). Note, this suggests that negative cues will improve performance if the initial cost of capture is less than the benefit of later suppression. In contrast, if the initial cost of capture is greater than the benefit of later suppression, negative cues will hinder performance relative to neutral cues.

Eye-tracking metrics have provided insight into how attention is directed over time. Beck et al. (2018) observed that negative cues result in a high (above chance) likelihood of fixating the cued distractor early in the trial but a low (below chance) level later in the trial. Conversely, positive cues result in a high likelihood of fixating the target at the beginning of the trial, increasing over time (Beck et al., 2018). However, they also found that later avoidance of the negatively cued distractors was not dependent on the earlier attentional capture. They suggested this might be further evidence for spatial recoding rather than a search and destroy mechanism (i.e., the item does not need to be attended to later be suppressed). This demonstrates a dissociation between where eye movements are directed early versus late during search. Early attention seems to be highly influenced by VWM contents, whereas later attention may be more heavily influenced by goals and the other items in the search display. Also assessing the early direction of attention, Foester & Schneider (2018) cued participants with colored realistic silhouettes; however, color was irrelevant to search. They observed a decreased likelihood of the first search saccade going to the target and an increase in initial saccade latency when the cue matched the color of the distractor in the two-item search display (Foester & Schneider, 2018). Although not explicitly testing negative cues, this suggests an inability to suppress the cued features in VWM even when they are irrelevant to search.

The current study

Although there is a considerable body of evidence supporting each theory, neither the flexible attentional control hypothesis nor the initial capture hypothesis have been tested in the context of categorical search. The present study tests the effects of negative cues in both categorically cued and pictorially cued search. Manipulating cue specificity (pictorial, most specific vs. categorical, less specific) with real-world stimuli offers a unique test of the competing theories. Search displays in this study included a search target, a lure (the cued distractor on negative cue trials), and other unrelated distractors. Search targets and lures came from the same basic-level category and differed in their dominant color and other basic-level features. Categorical cues require participants to rely on long-term memory knowledge of potential target features, as there is a lack of usable perceptual information provided directly from the cue. Although neither theory explicitly predicts a different pattern of results based on cue specificity, predictions become clear when paired with the broader literature. Whereas numerous studies have demonstrated similar negative cue effects with pictorial cues and specific text cues (i.e., “Ignore Red”; Beck et al., 2018; Kawashima & Matsumoto, 2018; Moher & Egeth, 2012), to the authors’ knowledge, no prior work has directly compared the two or assessed negative cues in the context of categorical search.

According to the flexible attentional control hypothesis, negatively cued features should be suppressed, resulting in attention being directed more often to the target (stronger target guidance) than to the lure (weaker attentional capture). Given that the flexible attentional control hypothesis suggests a suppression mechanism, anything that improves suppression should improve performance. Conversely, the initial capture hypothesis predicts that the negative cue will automatically bias attention to the lure. This would result in negative cues producing stronger attentional capture by the lure rather than guidance to the target. Given that the initial capture hypothesis relies on an attentional capture mechanism, anything that increases capture should produce a performance cost.

When considering cue specificity, it is important to note that positive pictorial cues produce stronger target guidance than categorical cues, likely because of the knowledge of the exact target features and the automatic priming of the target’s visual features (Maljkovic & Nakayama, 1994; Maxfield & Zelinsky, 2012; Schmidt, & Zelinsky, 2009; Wolfe et al., 2004). Given that the initial capture hypothesis predicts capture by the negatively cued lure and stronger visual priming is known to increase capture (Olivers et al., 2006), the negative pictorial cue should result in greater lure capture relative to categorical text cues. However, the flexible attentional control hypothesis may predict less lure capture with negative pictorial cues than categorical cues. Assuming a similar underlying mechanism as positive attentional templates, the precise visual details in a pictorial cue should result in more efficient suppression than broad categorical features. Whereas both pictorial and categorical cues allow participants to strategically attend to the non-cued categorical features, pictorial cues provide the exact lure features, leading to more precise suppression and thus stronger target guidance with negative pictorial cues relative to categorical cues. As such, the flexible attentional control hypothesis would also be broadly consistent with a smaller specificity benefit with negative cues relative to positive cues. Despite more precise suppression with negative pictorial cues, the non-cued categorical features that afford positive information are similar with pictorial and categorical negative cues. Alternatively, the initial capture hypothesis might predict an inverse specificity effect across positive and negative cues. Positive pictorial cues may improve performance over categorical cues but may create a performance cost due to increased attentional capture via priming when negatively cued.

Method

Participants

Participants were recruited through the University of Central Florida’s Psychology Research Participation System and were given course credit in exchange for their participation. Participants provided verbal consent prior to participation. Data were collected from 40 participants; 55% were female, and 45% were male, with an average age of 18.5 years. 52% were White, 33% Hispanic, 8% Asian, 8% Black, and 1% other. All participants completed a vision screening with the Logarithmic Visual Accuity Chart and the Ishihara’s Test for Colour Deficiency; all participants had normal or corrected-to-normal acuity and did not have any color vision deficiencies. The University’s Institutional Review Board approved the study.

Stimuli & Apparatus

The experiment was presented at a screen resolution of 1920 × 1080 pixels and a refresh rate of 144Hz on a 21-inch Asus monitor (Model VG248). Eye movements were sampled at 1000 Hz using a desktop mounted EyeLink 1000 Plus in head-stabilized mode using default saccade detection thresholds. Participants sat with their heads in a headrest, approximately 98 cm from the screen. Before the experiment, the eye tracker was calibrated using a 13-point calibration routine, which was only accepted once the average error was less than .49° and the maximum error was less than .99°. The calibration procedure was repeated after breaks (one after the practice trials and three between condition blocks) and as necessary (i.e., when the eye position was lost) throughout the study.

All stimuli were collected from the Hemera Photo Objects database, the Bank of Standardized Stimuli (BOSS; Brodeur et al., 2010), MIT Object Categories (Konkle et al., 2010), and various other web sources (via Google image search). Stimuli were resized to an area of 8000 pixels while allowing the aspect ratio to vary, which yielded a visual angle of 1.5° × 1.5° for square images. Categorical cues were presented in Times New Roman black 20-point font. Depending on the word length, the categorical cues subtended a horizontal visual angle of 0.9° to 2.1°, with an average of 1.5°. The text consistently subtended a vertical visual angle of 1.1°. Targets and lures were selected from 50 different basic-level categories. The lures were paired with images from the same basic level category but differed in the primary color of the object, as determined independently by two researchers. The target and lure categories were repeated (10 categories appeared in 8 trials each, and 40 categories appeared in 7 trials each), but all individual stimuli were presented only once per participant (different images appeared on each trial and did not repeat throughout the experiment). The circular search array was comprised of one target, one lure, and four random distractors. Stimuli were randomly positioned about 8.8° from the center of the screen in one of six possible locations. The non-lure distractors were from separate basic-level categories relative to the cued item and each other. Across participants, targets and lures were balanced such that each object appeared equally often as a target or a lure.

Design

This experiment employed a two-by-two design, with two within-subjects factors of cue specificity (pictorial or categorical) and cue validity (positive or negative). In the positive pictorial condition, the cue was identical to the target, and the lure was from the same basic level category but differed in the primary color and other features known to vary with the basic-level category (Hout et al. 2017; Yu et al., 2016; Zelinsky et al., 2013a; Zelinsky et al 2013b). In the negative pictorial condition, the cue was identical to the lure, and the target was from the same basic-level category, but differed in the primary color and other basic-level category features. In the positive categorical condition, the text cue consisted of two words: the first word corresponded to a color present in the target, and the second word indicated the target’s basic-level category (i.e., “blue headphones”). The lure was again from the same basic-level category but did not contain the color given in the cue. Conversely, the negative categorical condition provided a text cue that indicated the color and basic-level category of the lure rather than the target; the target was from the same basic level category but did not contain the cued color. In both negative conditions, the participant was told that the categorical information was accurate, but the color provided in the cue would pertain to the lure rather than the target.

These conditions were blocked and counterbalanced over participants. At the beginning of each block, participants were given the corresponding instructions. Importantly, across participants, the identical search displays appeared equally often in the positive pictorial, positive categorical, negative pictorial, and negative categorical conditions. This counterbalancing resulted in the same manipulation of the target and lure across cue specificity and ensured that any observed effects arose from differences in the cues.

Procedure

The sequence for the experiment can be seen in Figure 1. At the beginning of each trial, participants were instructed to maintain their gaze on a fixation cross in the center of the screen. Once the eye-tracker detected a fixation within 2.5° of the fixation cross for a minimum of 500 ms, the trial automatically started. The cue then appeared in the center of the screen. It remained visible for 300 ms, followed by a 1000 ms interstimulus interval in which only a central fixation cross was visible, followed by a six-item search array. A target and a lure were present on every trial. Participants were instructed to fixate the target in the search array and press a button on a gamepad to indicate they had localized the target. If participants were fixating within 2.5° from the center of the target, the response was considered correct, and participants proceeded to the next trial. If participants were fixating within 2.5° from the center of the lure or a distractor, the response was considered incorrect, the word “INCORRECT” was displayed in the center of the array, and a circle appeared around the correct target stimulus for 2000 ms. If participants were not fixated on any object when making a response, a noise occurred, and the participant was told to make sure they were fixating on a specific object when responding. Participants had no more than 4000 ms to respond before the trial timed out. If they did not respond, the search array was removed, and the words “TIME’S UP” appeared in the middle of the screen.

Figure 1.

Procedure for the search task. For visualization purposes, the black arrows indicate the target in each example search display. Note that the target always matched the cued category but either matched the color of the cue (positive) or did not match the color of the cue (negative). Stimuli in this figure are not drawn to scale.

Participants completed 40 practice trials followed by 360 experimental trials. Cue validity and cue specificity were blocked (four blocks, 90 trials each) and counterbalanced across participants so that all possible combinations of the condition order appeared equally often.

Statistical Methods

The main effects of cue validity (positive/negative), and cue specificity (pictorial/categorical), and interactions of the two factors were assessed using 2 × 2 repeated measures omnibus ANOVAs (with significance levels set to α = 0.05 throughout). To determine the source of any significant interactions, we performed four two-tailed paired-samples t-tests (with a Bonferroni-corrected α of .0125 to correct for multiple comparisons). These t-tests allow us to detect any positive cue advantage (i.e., improved search with positive cues relative to negative cues) for pictorial or categorical cues and any pictorial cue advantage (i.e., improved search with pictorial relative to categorical cues) for positive or negative cues.

Eye Movement Analysis

We define guidance as the preferential direction of the first search saccade to the target relative to the average likelihood of directing the first saccade to any of the four random distractors in the search array (see Alexander & Zelinsky, 2011, for a similar approach). Likewise, we define attentional capture of the lure as the preferential direction of the initial search saccade to the lure rather than to a random distractor. Because the percentage of initial saccades directed at the target, lure, and distractor will necessarily sum to one-hundred percent, we conducted paired-samples t-tests comparing the percentage of initial saccades directed to the target or lure relative to the average of the random distractors to determine if guidance and capture are present.

To assess the later suppression of negatively cued information, we examined the amount of time it took to reject the lure¹. Although neither hypothesis explicitly makes predictions concerning the amount of time needed to verify a target or reject a distractor (lure or non-lure distractor), we felt it was important to assess how negative cues may affect object processing/recognition. Most importantly, we questioned if prior knowledge of the lure would speed up or slow down the rejection process relative to a random distractor. To that end, we measured dwell times (the cumulative fixation duration) on the target (to assess target verification) and also dwell times on the lure or distractor (to assess distractor rejection times).

Results

Manual Measures

Means and standard errors for accuracy and RT can be found in Table 1. Consistent with prior work, accuracy was higher with positive cues relative to negative cues, F (1, 78) = 30.77, p < .001, η_p² = .44, and higher with pictorial cues relative to categorical cues, F (1, 39) = 21.60, p < .001, η_p² = .36. Cue validity and cue specificity also significantly interacted, F (1, 39) = 17.81, p < .001, η_p² = .31. Specifically, there was a positive cue advantage in the pictorial conditions, t (39) = 8.14, p < .001, d = 1.14, and a pictorial advantage in the positive conditions, t (39) = 5.01, p < .001, d = .83. Conversely, there was no positive cue advantage in the categorical conditions, t (39) = 1.61, p = .12, d = .20, nor a pictorial advantage in the negative cue conditions, t (39) = 1.93, p = .06, d = .18. Thus, there was a pictorial advantage with positive—but not negative—cues, and a positive cue advantage with pictorial—but not categorical—cues. All further analyses were conducted on correct trials only.

Table 1.

Accuracy and Response Time for the Search Task

Metric	Positive Cue	Negative Cue
Accuracy (%)
Pictorial Cues	96.91 (.01)	90.35 (.01)
Categorical Cues	90.77 (.01)	89.07 (.01)
Response Time (ms)
Pictorial Cues	745.76 (20.77)	1335.44 (35.33)
Categorical Cues	1150.62 (35.77)	1542.54 (40.35)

Note: Values in parentheses indicate the standard error of the mean (SEM). RTs are for correct trials only.

RTs showed similar patterns of improved performance with positive cues, F (1, 39) = 301.96, p < .001, η_p² = .86, and with pictorial cues, F (1, 39) = 188.64, p < .001, η_p² = .83. As with accuracy, the interaction was also significant, F (1, 39) = 33.55, p < .001, η_p² = .465. Unlike accuracy, all four follow-up t-tests were significant: there was a positive cue advantage in both, pictorial, t (39) = −19.96, p < .001, d = 3.22, and categorical conditions, t (39) = −10.85, p < .001, d = 1.63, as well as a pictorial advantage in both positive, t (39) = −12.99, p < .001, d = 2.19 and negative cue conditions, t (39) = −8.43, p < .001 .06, d = .86. However, the means indicate that the interaction is likely due to a larger pictorial advantage with positive cues and a larger positive cue advantage with pictorial cues—see Table 1.

These results suggest the experimental manipulations had their intended effect. The pictorial advantage tends to be larger with positive cues, and the positive cue advantage also tends to be larger with pictorial cues (i.e., the effect of cue specificity is smaller or absent with negative cues and the benefit of a positive cue is smaller with categorical cues). The smaller pictorial advantage with negative cues is generally consistent with the flexible attentional control hypothesis.

Guidance to the Target

Participants were able to preferentially guide their eye movements towards the target in all conditions: initial saccades were directed more often to the target relative to a random distractor (all p < .0125)—see Figure 2.² In line with previous work, guidance to the target was stronger with pictorial cues than categorical cues, F (1, 39) = 65.95, p < .001, η_p² = .63. Guidance to the target was also stronger with positive cues than with negative cues, F (1, 39) = 240.77, p < .001, η_p² = .86. There was also a significant interaction between the two factors, F (1, 39) = 48.13, p < .001, η_p² = .55. Post hoc tests indicated positive cue advantages in both the pictorial, t (39) = 15.17, p < .001, d = 3.12, and categorical conditions, t (39) = 12.77, p < .001, d = 2.34. There was also a pictorial advantage in the positive conditions, t (39) = 9.32, p < .001, d = 1.05, but not in the negative conditions, t (39) = 1.70, p = .10, d = .31. Thus, the effect of cue specificity is absent with negative cues, and the benefit of a positive cue is smaller with categorical cues. Negative cues resulted in preferential guidance to the target but eliminated the pictorial advantage. The presence of target guidance from negative cues is broadly consistent with the flexible attentional control hypothesis.

Figure 2.

The percentage of trials in which the first search saccade was directed to the target, the lure, or random distractors (on average). Error bars indicate the standard error of the mean.

Attentional Capture by the Lure

To assess attentional capture, we next compared capture by the lure relative to capture by random distractors. Significant attentional capture by the lure was present in the negative cue conditions (both, p < .001) but not in either the positive categorical condition, t (39) = −.31, p = .76, d = .08 or in the positive pictorial condition, t (39) = −2.02, p = .05, d = .52 (not significant after Bonferroni correction). Negative cues resulted in stronger attentional capture relative to positive cues, F (1, 39) = 199.15, p < .001, η_p² = .84. Whereas there was no main effect of cue specificity, F (1, 39) = 1.25, p = .27, η_p² = .03, there was an interaction between validity and cue specificity F (1, 39) = 13.55, p = .001, η_p² = .26. Post hoc tests indicated positive cue advantages in both the pictorial, t (39) = −14.69, p < .001, d = 3.47, and categorical conditions, t (39) = −10.29, p < .001, d = 2.64. That is, there was less capture by the lure in the positive cue conditions. There was also a pictorial advantage (less capture) in the positive conditions, t (39) = −5.48, p < .001, d = 1.07, but not in the negative conditions, t (39) = 1.42, p = .17, d = .23. Consistent with the initial capture hypothesis, this suggests participants could not overcome attentional capture by the lure with negative pictorial or categorical cues. Interestingly, attentional capture by the lure was observed even in the absence of perceptual information provided in the categorical cue condition, suggesting little effect of priming or specific visual details with negative cues. Despite the lack of a priming effect with pictorial cues, the presence of capture provides some support for the initial capture hypothesis. However, similar levels of capture by the negatively cued items are not consistent with either theory. The significant interaction and follow-up tests demonstrate that negative cues eliminate the benefit of cue specificity, and the benefit of a positive cue is smaller with categorical cues.

Guidance or Capture

To determine whether guidance to the target was stronger than attentional capture to the lure, we directly compared how frequently the first saccade was directed towards the target or towards the lure within each condition. There was more guidance to the target relative to capture by the lure in both positive cue conditions (both p < .001). The positive cue results are consistent with previous reports in which positive cues can produce strong guidance even in the presence of a similar item (i.e., the lure; Alexander et al., 2014; Alexander & Zelinsky, 2012). Conversely, negative cues produced significantly more attentional capture by the lure relative to target guidance (both, p < .001). Because there is more capture than guidance with negative cues, this is consistent with the initial capture hypothesis. However, two findings are also generally consistent with the flexible attentional control hypothesis, 1) the presence of target guidance, and 2) similar target guidance across negative cue specificity levels. We argued that the positive categorical features used to guide search might be similar for negative pictorial and categorical cues despite the assumption of stronger suppression with a pictorial cue.

Object Processing/ Recognition

Target Verification

We measured dwell time on the target when it was the first item fixated to assess target verification³. There was a main effect of cue validity, F (1, 39) = 53.06, p < .001, η_p² = .58, in which positive cues resulted in shorter dwell times relative to negative cues. There was also a main effect of cue specificity, F (1, 39) = 92.05, p < .001, η_p² = .70, in which pictorial cues resulted in shorter dwell times compared to categorical cues. Lastly, there was an interaction of the two, F (1, 39) = 20.09, p < .001, η_p² = .34. Post hoc tests indicated a positive cue advantage in the pictorial conditions, t (39) = −8.34, p < .001, d = 1.32, and in the categorical conditions, t (39) = −3.87, p < .001, d = .61. There was also a pictorial advantage in the positive, t (39) = −9.68, p < .001, d = 1.53 and negative cue conditions, t (39) = −4.87, p < .001, d = .77. Given that all follow-up tests were significant, we examined the means to determine the locus of the interaction. Consistent with the prior analyses, the interaction appears to be driven by a larger pictorial advantage with positive cues relative to negative cues and a larger positive cue advantage with pictorial cues relative to categorical cues. Means and standard errors for the target dwell times can be found in Figure 3.

Figure 3.

Target dwell time when the target was the first object fixated. Error bars indicate the standard error of the mean.

Distractor Rejection

To assess distractor rejection, we compared dwell times on the lure and random distractors when fixated first⁴, resulting in a 2 (validity) × 2 (cue type) × 2 (stimulus type) ANOVA (see Figure 4). There was a main effect of validity, F (1, 39) = 126.93, p < .001, η_p² = .77, in which positive cues resulted in shorter dwell times relative to negative cues. Because participants had access to the exact identity of the target in the positive cue condition, participants were able to quickly reject non-target items. We interpret this as evidence that negatively cued search is more difficult than positive cued search, which is consistent with previous work. There was also a main effect of cue type, F (1, 39) = 282.84, p < .001, η_p² = .88, in which pictorial cues resulted in shorter distractor dwell times compared to categorical cues. Lastly, there was a main effect of stimulus type, F (1, 39) = 79.82, p < .001, η_p² = .67, in which distractors were rejected faster than the lure. This is likely because the lures were more visually similar to the targets, making it more difficult to reject. All interactions, with the exception of stimulus type and validity, F (1, 39) = 2.10, p = .16, η_p² = .05 were significant (stimulus type and cue type, F (1, 39) =28.41, p < .001, η_p² = .42, validity and cue type, F (1, 39) = 10.12, p < .01, η_p² = .21), and the three-way interaction, F (1, 39) = 5.06, p < .05, η_p² = .12).

Figure 4.

Dwell time on the lure or distractor when each was the first object fixated. Error bars indicate the standard error of the mean.

To interpret the interactions, we first ran two 2 (validity) × 2 (cue type) repeated measures ANOVAs separately for each stimulus type (lure and distractor). The lure produced a main effect of validity, F (1, 39) = 47.45, p < .001, η_p² = .55, and cue type, F (1, 39) = 183.28, p < .001, η_p² = .83, but no interaction, F (1, 39) = .41, p = .53, η_p² = .01, This suggests that both cue specificity levels result in faster lure rejection with positive cues and that pictorial cues were faster than categorical cues in both cue validity conditions. Negative cues slow the rejection of the lure in both cue specificity levels, suggesting a cost to maintaining lure features in a negative attentional template. Interestingly, despite the visual priming of the lure in negative pictorial cues, rejection was faster compared to negative categorical cues, suggesting minimal visual priming effects and a benefit of knowing the precise lure features to reject.

The random distractors produced a main effect of validity, F (1, 39) = 205.51, p < .001, η_p² = .84, and cue type, F (1, 39) = 128.25, p < .001, η_p² = .77, and an interaction, F (1, 39) = 41.83, p < .001, η_p² = .52. Post hoc tests indicated a positive cue advantage in the pictorial, t (39) = −17.94, p < .001, d = 2.84, and in the categorical conditions, t (39) = −5.00, p < .001, d = .79, and also a pictorial advantage in the positive, t (39) = −12.04, p < .001, d = 1.90 and negative conditions, t (39) = −4.62, p < .001, d = .73. Consistent with the prior analyses, examination of the means suggest a larger positive cue advantage with pictorial cues and a larger pictorial advantage with positive cues.

To determine the source of the three-way interaction, we calculated the difference scores between lure dwell time and the random distractor dwell time (see Figure 5) and conducted a cue type by cue validity repeated measures ANOVA. Results indicated a main effect of cue type, F (1, 39) = 28.41, p < .001, η_p² = .42, in which pictorial cues produce a smaller difference in lure-distractor dwell times compared to categorical cues. Additionally, we observed an interaction, F (1, 39) = 5.06, p < .05, η_p² = .12, in which there was a positive cue benefit in categorical but not in pictorial cues. There was no effect of cue validity, F (1, 39) = 2.10, p = .16, η_p² = .05. This demonstrates that pictorial cues produce a small but consistent rejection time cost for lures relative to random distractors. Interestingly, this is unaffected by cue validity: relative to random distractors, lures are rejected similarly if cued with the target or the lure. However, categorical cues result in a lure rejection time costs relative to pictorial cues. Additionally, the lure produces a disproportionately large cost after a negative categorical cue, explaining the source of the three-way interaction. Yet, interpreting this effect requires careful consideration of the baseline condition (the non-lure distractor). The baseline condition produced an interaction of cue specificity and cue validity, and thus subtraction of the random distractor dwell times from the lure dwell times also creates an interaction. This is generally consistent with a specificity effect in object recognition, irrespective of cue validity, i.e. the pictorial benefit is present in both positive and negative cue conditions.

Figure 5.

The difference scores between lure and distractor (lure minus distractor) per condition. Error bars indicate the standard error of means.

Discussion

This study investigated the effects of cue specificity on the utility of negative attentional templates. To test this, participants viewed positive and negative pictorial and categorical cues. According to the flexible attentional control hypothesis, we expected to see more target guidance than lure capture and less lure capture with pictorial cues relative to categorical cues, consistent with greater lure suppression. Alternatively, the initial capture hypothesis predicts an inability to overcome the lure capture caused by the negative cue, resulting in more lure capture than target guidance and greater lure capture with pictorial cues relative to categorical cues.

The results show mixed support for each hypothesis. Consistent with the flexible attentional control hypothesis, we found that negative pictorial and categorical cues result in guidance to the target as participants were more likely to make the first search saccade towards the target than one of the four random distractors. Importantly, target guidance in our task could only arise from suppressing the negatively cued lure features and enhancing categorical target features. Our target guidance data cannot distinguish between these two possibilities as they both must occur in our task for target guidance to be present. Future work may seek to design a study specifically to dissociate these two possibilities. Target guidance did not significantly differ between the negative pictorial and categorical cues, suggesting participants strategically used the categorical information in both cases. In other words, to achieve target guidance in our task, participants had to suppress the color information in the negative cue and search for the target categorically. If participants only enhanced the target categorical features, we would have observed equivalent guidance to the target and capture by the lure, as they are from the same category. However, that is not what we see with negative cues. Instead, we found more attentional capture by the lure, suggesting some interference of the negatively cued information. This provides some evidence for the initial capture hypothesis. This indicates that the negatively cued color information cannot be fully suppressed; irrespective of whether it arrived via text or a pictorial cue, it captures attention. Like negatively cued target guidance, capture was essentially categorical in nature as it did not vary across cue specificity levels, suggesting little effect of visual priming or specific visual features when negatively cued. The automatic priming that results from a negative pictorial cue did not increase attentional capture (i.e., the initial capture hypothesis), and the specific visual features afforded by pictorial cues did not decrease capture relative to negative categorical cues (i.e., the flexible attentional control hypothesis). In sum, the presence of target guidance and the general elimination of cue specificity effects with negative cues are generally consistent with the flexible attentional control hypothesis. In contrast, the greater lure capture relative to target guidance is consistent with the initial capture hypothesis. Collectively, this suggests that categorical features can be strategically attended, but lure features cannot be down-weighted completely.

To assess how negative cues affect object processing, we examined target, lure, and distractor dwell times for the first search item fixated. Pictorial cues resulted in faster target verification compared to categorical cues irrespective of cue validity. This suggests that knowing the exact features of the target or lure speeds target verification relative to a categorical representation. Furthermore, cue validity modulates target verification; positive cues result in faster target recognition, suggesting a benefit of knowing the target features to accept. Additionally, the pictorial advantage was larger with positive cues, and there was a larger positive cue advantage with pictorial cues. Similarly, positive cues resulted in faster lure rejection relative to negative cues, and pictorial cues resulted in faster lure rejection relative to categorical cues. These effects even extend to the random distractors, suggesting that negative cues hinder all object recognition effects, not just the cued lure. This pattern of results demonstrates that knowledge of the exact target or lure features improves distractor rejection over categorical features, and target knowledge improves distractor rejection relative to lure knowledge. This demonstrates a clear benefit of target specificity and suggests negative cue specificity effects arise from object recognition processes.

Furthermore, when comparing lure rejection to random distractor rejection, we found that distractors were rejected more quickly than the lures in all conditions, suggesting that disengaging from the lure was more difficult than disengaging from a random distractor. Importantly, the difference in these rejection times was consistent across positive and negative pictorial cues. However, categorical cues produced a larger difference than pictorial cues. Categorical cues also showed a cue validity effect in which there is a larger cost with negative categorical cues than with positive categorical cues. However, it is important to note that the random distractor baseline produced a similar interaction, whereas the lure did not generate an interaction. That suggests that the source of the interaction in the calculated difference scores may arise from the random distractor baseline. Collectively, this is consistent with negatively cued specificity effects arising from object recognition.

When considering the guidance/capture, and recognition results in tandem, a dissociation emerges between the information used to direct attention and recognize an object. For attentional guidance, increased specificity results in a pictorial advantage with positive cues but not with negative cues. However, with recognition, increased specificity results in a pictorial advantage with both positive and negative cues, albeit a smaller benefit for negative cues. Taken together, this suggests that visual priming and knowledge of the exact target features can significantly improve overall search performance when positively cued. Conversely, when negatively cued, visual priming and knowledge of the exact lure features only improves object recognition, but not the guidance of spatial attention. This asymmetry demonstrates that the ability to up-weight or positively bias features is much stronger than the ability to down-weight or negatively bias features.

Limitations and Future Work

It is important to discuss the similarities and differences between this study’s negative cues and those used in previous work. In prior work discussed above, targets were designated at the beginning of the experiment or block (i.e., the target is always a Landolt-C with a gap facing up or down). This provides participants with target-defining information, but it is not cued on a trial-by-trial basis as was the case with our work. This suggests that attending to the targets categorical features in our negative cues should be analogous to up-weighting target Landolt-C orientations, as was the case with prior work. Additionally, in prior work, the negative cues could theoretically be completely ignored as they provide no target-defining information. In fact, several studies demonstrated that some participants use cues far less than other participants (Beck & Hollingsworth, 2015; Carlisle & Nitka, 2019). Conversely, in the current study, the targets were categorically defined on a trial-by-trial basis, thus requiring the participant to attend to the negative cues to ascertain the target category and complete the task. In fact, the significant attentional capture and target guidance, as well as the high accuracy in this task suggest that the cue-related information was thoroughly encoded and not simply ignored. This suggests that the only way to prevent attentional capture was to suppress or down-weight the negatively cued lure features (which occurred on a significant proportion of trials, as is evidenced by the target guidance data). We argue that this design difference is a strength of the current study; the first step in examining the effect of negative cues is to ensure that participants attend to and process the cue.

Another difference from prior work is that the targets in our negative cue condition could be any member of the cued basic level category. As a result, these targets are less precisely defined relative to previous work. Prior work had very few possible targets: previous studies had as few as four (e.g., Foester & Schneider 2018, Experiment two) and at most sixteen possible targets (e.g., Beck et al., 2018; Becker et al., 2015, Experiment three; Foester & Schneider 2018, Experiment one). Consequently, negative cues in these studies eliminate some targets, providing some positive target-defining information (as the target must be one of the other items from the set of possible targets). Future work should explore if the number of target possibilities and the number of cue matching items impact the effectiveness of negative cues. Manipulating the number of cue matching items would also allow for analyses similar to Beck, et al., (2018; i.e., if later suppression is contingent upon initial capture); which was not possible in our current design.

It is also important to note that the current study did not allow for a neutral cue baseline condition. Given that this study examined how the specificity effects changed with cue validity, a neutral baseline condition wasn’t necessary to determine if a negative pictorial cue produces a benefit over negative categorical cues. Nonetheless, future work should assess the specificity effect with a true neutral cue baseline by assigning a small number of target categories at the beginning of the experiment or block, allowing for the cues to be used or not (Beck & Hollingsworth, 2015; Carlisle & Nitka, 2019). A neutral baseline would allow us to focus on the absolute benefit or cost of a negative cue.

Future work should also compare specific text cues (corresponding to a specific item) and pictorial cues without categorical information. This would examine if our results were partly due to the strategic use of categorical information. An additional line of research might assess the effects of negative cues for non-color features. Most of the negative cue work has been done in the context of color cues (but see, Beck & Hollingworth, 2015; Munneke et al., 2008 for negative location cues). Because search guidance is typically dominated by color (e.g., Alexander et al., 2019; Hwang et al., 2009; Williams, 1966), the current findings may not extend to other feature dimensions. Additionally, our usage of basic-level categories ensured a high degree of target-distractor similarity (e.g., Rosch et al., 1976). This is an important point as previous work has demonstrated high target-distractor similarity increases the utility of negative cues (Conci et al., 2019). Therefore, future work should more strictly control and manipulate target-distractor similarity to assess if target-distractor similarity in real-world stimuli affects positive and negative cues differently.

Lastly, our results suggest that participants can set attentional weights that guide attention to the search target on some negatively cued trials, consistent with the flexible attentional control hypothesis. However, the lure captured attention on a greater percentage of negatively cued trials, consistent with the initial capture hypothesis. Future work should seek to assess negative cue encoding on trials that result in successful target guidance relative to instances in which the lure captured attention. Perhaps differences in the cue encoding and preparation processes will further clarify under what circumstances spatial attention is later directed to the target or lure, explaining the partial support for each theory. Essentially, target encoding and preparation differences may predict which theory will better explain the data on any given trial. If pre-search preparation/encoding differences predict target guidance/lure apture, cognitive processes that improve target guidance could be trained, resulting in the more effective use of negative cues.

Conclusion

Consistent with prior work, the current findings confirm that the attentional template is imprecise, features are broadly biased, and that a range of feature values can all attract attention (e.g., Alexander et al., 2019; Hout, & Goldinger, 2015; Wolfe, 1994). However, the specificity effects observed with positive cues and the near absence of negatively cued specificity effects demonstrate that the down-weighting of specific visual features is far less effective than the up-weighting of specific visual features. Rather, negative cues seem to result in attentional capture at a categorical level, producing no observable specificity effect in guidance or capture. Critically, small negative cue specificity effects were present in object recognition measures. This disassociation between attention directing mechanisms (guidance/capture) and object recognition (target recognition/lure rejection) supports the idea that attention directing mechanisms rely on less precise information than object recognition (e.g., Alexander & Zelinsky, 2018; Ercolino, et al., 2020). This may suggest that the RT benefits of negative cues arise from object recognition rather than early attention directing processes.