The Attentional Template is Shifted and Asymmetrically Sharpened by Distractor Context

Abstract

Theories of attention hypothesize the existence of an “attentional template” that contains target features in working or long-term memory. It is often assumed that the template contents are veridical, but recent studies have found that this is not true when the distractor set is linearly separable from the target (e.g., all distractors are “yellower” than an orange colored target). In such cases, the target representation in memory shifts away from distractor features (Navalpakkam & Itti, 2007) and develop a sharper boundary with distractors (Geng, DiQuattro & Helm, 2017). These changes in the target template are presumed to increase the target-to-distractor psychological distinctiveness and lead to better attentional selection, but it remains unclear what characteristics of the distractor context produce shifting vs. sharpening. Here, we test the hypothesis that the template representation shifts whenever the distractor set (i.e., all of the distractors) is linearly separable from the target, but that asymmetrical sharpening only occurs when linearly separable distractors are highly target-similar. Our results were consistent, suggesting that template shifting and asymmetrical sharpening are two mechanisms that increase the representational distinctiveness of targets from expected distractors and improve visual search performance.

Introduction

It is impossible for humans to simultaneously process all available information in complex visual scenes. Thus, when searching for a target object (e.g., a friend at a party), we must use known features (e.g., hair color) to guide our attention and gaze. Theories of attention posit that this occurs by using information held within a memory representation (i.e., the attentional or target template) to bias sensory processing towards target features and serve as a decisional boundary for target selection (Bundesen, 1990; Desimone & Duncan, 1995; Duncan & Humphreys, 1989; Reynolds & Heeger, 2009; Treue & Martínez-Trujillo, 1999; Liu, Larsson, & Carrasco, 2007; Wolfe, 2007). Importantly, the target template not only modulates the sensory gain of neurons, but also determines if the visual input matches the target (Duncan & Humphreys, 1989; Hout & Goldinger, 2015; Malcom & Henderson, 2010). Thus, while it is clear that the contents of the attentional template are critical for defining “task-relevance” at multiple stages of attentive processing, there is limited knowledge of what factors shape the “tuning” of the template. It has been largely assumed that the template is optimal when it perfectly matches the veridical target so that it can tune the most veridical sensory neurons and make the most accurate decision for a target-match. However, recent evidence suggests that the attentional template may actually be more optimal when “off-veridical” if it increases the psychological distance between targets and distractors (Navalpakkam & Itti, 2007; Becker, 2010; Scolari & Serences, 2009; Wolfe, 2000; Bauer, Joliceour, & Cowan, 1996; D’Zmura, 1991; Hodsoll, & Humphreys, 2001). In these studies, we extend those demonstrations to investigate the exact properties of the visual context that alter tuning properties of the target template.

One of the earliest studies to report “off-veridical” template representations found that shifts in the target representation in response to distractor context produced better target selection than a veridical template representation (Navalpakkam & Itti, 2007). Navalpakkam and Itti (2007) asked observers to search for a target line oriented 55° amongst 50° distractor lines during visual search “training” trials and then measured the target representation held in memory on separate “probe” trials. On probe trials, participants selected the target from five briefly presented oriented lines (80°, 60°, 55°, 50° and 30°). Notably, while the visual search “training” trials contained distractors that were linearly separable from the target (e.g., all were at the same orientation rotated counterclockwise from the target), the “probe” stimuli were sampled from both sides of the target (e.g., both counterclockwise and clockwise rotations of the target). Navalpakkahm and Itti (2007) found that the 60° stimulus was chosen more frequently than the 55° (true target) stimulus as the target on probe trials. This demonstrated that the target representation was shifted away from visual search distractors. In a second experiment, the same effect of target “shifting” was found using color stimuli, suggesting that shifted target representations occur across stimulus dimensions (see Figure 1). The authors argued that the shift reflected a bias in sensory gain towards neurons tuned to orientations more distant from the distractors (Figure 2A), in order to optimize the perceptual distinctiveness of the target from distractors (see also Hodsoll & Humphreys, 2001; Scolari, Byers & Serences, 2012).

Figure 1.

Illustration of linear separability between target and distractor stimuli. The target color is denoted with letter “T” and distractors with “D”. (A) A case where the orange target is linearly separable from “yellower” distractors. The linear operator is represented by dashed line. (B) A case where the target is linearly nonseparable from distractors. (Adapted from Bauer, Jolicoeur & Cowan, 1998).

Figure 2.

Examples of template adjustment to increase the distinctiveness between an orange target and yellow distractors. (A) Shifting alone. (B) A combination of shifting and asymmetrical sharpening. See texts for details.

In addition to use of target “probe” trials, Scolari and Serences (2009; Experiment 2, 3) used an independent contrast sensitivity task to test for attentional biases in target features. They reasoned that contrast detection thresholds should be lowest for orientation-selective neurons with greater attentional gain. The results when distractors were similar to the target (i.e., 5° away) showed lower contrast detection thresholds for off-target features. Interestingly, both distractor orientations as well as target-exaggerated orientations had significantly lower thresholds, but there was an asymmetry early in training such that the exaggerated target features (i.e., orientations most distant from distractors) had the lowest thresholds (Scolari & Serences, 2009, Figure 4C). Furthermore, the off-target enhancement disappeared when the visual search distractor orientations were 90° from the target, suggesting that attentional shifting is unnecesary when distractors are uniform and very distinct from the target.

Figure 4.

Group averages of “target yes” responses in Experiment 1. Black solid lines are split normal distribution fits. The center gray bar indicates proportion of “hits” in response to the true target color and other bars indicate “false alarms” to non-target colors. All error bars are SEM. (A) The bidirectional group. Both negative and positive colors were distractors during visual search trials. (B) The unidirectional group. Only positive colors (green bars) were seen as distractors during visual search trials.

Similar findings of shifted target representations have also been reported by other researchers using very different methods. For example, Becker and colleagues (2014) asked participants to search for a colored target (e.g., orange) amongst linearly separable distractors (e.g., yellow). Each search display was preceded by a cue display with task-irrelevant colored “cues” surrounding each possible target location. They found stronger attentional capture by red colored cues compared to orange ones, suggesting that attention was biased toward “redder” colors than the true target. These results led Becker and colleagues to develop the relational account of attentional guidance, which hypothesizes that target features are represented relative to distractors (e.g., the target is the “redder” or “bigger” object) rather than by specific features (Becker, 2010; Becker et al., 2014; Becker, Folk, & Remington, 2010). Although the proposed mechanism differs from Navalpakkam and Itti (2007) and Scolari and Serences (2009), they also conclude that the contents of the attentional template are adjusted to maximize the observer’s ability to distinguish targets from expected distractors.

In addition to shifting the target representation, there is some evidence that the target template may also be asymmetrically sharpened when distractor competition is strong (Geng, DiQuattro & Helm, 2017). Sharpening has been observed in sensory neurons in response to attentional selection and has long been hypothesized to decrease the selectivity of task-irrelevant stimulus features (Series, Latham & Pouget, 2004; Lee et al., 1999; Ling, Jehee & Pestilli, 2016; Serences et al., 2009; Sompolinsky & Shapley, 1997; see also Scolari & Serences, 2009). In our previous study, the visual search display was composed of a single target and a distractor. The target color was fixed across the experiment; the distractor color varied continuously in similarity from the target color, but was always selected from one side of color space (i.e., target and distractors were linearly separable). Two groups of subjects saw the same distractors (ranging from 5°-60° along a color wheel), but the “high-similarity group” experienced a greater proportion of the most target-similar distractors and the “low-similarity group” saw the reverse distribution. In a separate template “probe” task, only the high similarity group had a target representation characterized by narrower tuning on the distractor side (i.e., asymmetrical sharpening) (Figure 2B); both groups had similar sized shifts in the central tendency of the target representation. This suggested that only the high-similarity group counteracted frequent pressure from highly competitive distractors by increasing the sharpness of the target template asymmetrically on the distractor side. The template “probe” task asked subjects for an explicit choice regarding the remembered target color and was therefore very different from those used in other studies in which the probe task targeted sensory processes (e.g., Navalpakkam & Itti, 2007; Soclari & Serences, 2009). This study provided evidence that expectations regarding the distractor context might shift and asymmetrically sharpen the attentional template, which may in turn affect visual search processing through sensory gain or decisional processes.

The aim of the current experiments is to determine if different distractor characteristics produce shifting vs. sharpening of the target template. The goal is to measure the “tuning” of the template as a memory representation based on expectations built from visual search trials, but is agnostic as to whether the representation affects sensory or decisional processes, or both. Specifically, we investigated two distractor features: linear separability and the strength of distractor competition. We hypothesize that when distractor colors are predictable and linearly separable from the target, the central tendency of the target representation will shift away from distractor values, but this will occur irrespective of exactly how similar the distractors are to the target. In contrast, we hypothesize that asymmetrical sharpening will occur in response to increasing competition from target-similar distractors. To test these hypotheses, we use a visual search “training” task to establish expectations for the distractor colors and a separate template “probe” task to measure the contents of the target template. Trials from the two tasks were interleaved. The separation of the visual search training trials and the template probe trials is essential for obtaining a measurement of the attentional template in memory that is uncontaminated by processes involved in active target selection from distractor competition.

Experiment 1

The goal of Experiment 1 was to test for the presence of shifting and asymmetrical sharpening in the target template due to the predictable distractor context. The distractor context was manipulated in visual search “training” task across two groups. In the “unidirectional” group, the distractors were all from one direction on the color wheel (e.g., bluer than the target color) and could be predicted from trial-to-trial. In the “bidirectional” group, the distractor set on each trial could be from either direction from the target color. It was therefore impossible to predict the directionality of distractor colors on a trial-by-trial basis. All distractors were highly similar to the target (5°−15° from the target) and therefore maxmized competittion for attention. The content of the target template was measured on separate template “probe” trials that calculated the likelihood of observers mistaking a range of color hues as the target color. Notably, because the probe trials were distinct from the visual search trials, our measurements of the template reflect information held in memory about the target feature.

Method

Participants.

Forty students (12 males, 4 left handed, ages 18 – 26) from University of California, Davis participated in Experiment 1 in partial fulfillment of a course requirement. They were randomly assigned into the unidirectional or the bidirectional color group. We chose twenty participants for each group based on power calculations (.8 power, .05 two-tailed significance) using results from Experiment 2 in Geng, DiQuattro, and Helm (2017). Each participant was provided written informed consent in accordance with the local ethics clearance as approved by the National Institutes of Health. Each participant’s color vision was assessed by self-report and an online color blindness test (https://colormax.org/color-blind-test). All participants had normal or corrected-to-normal vision and all had normal color vision.

Apparatus.

Participants were seated in a sound attenuated room 65cm away from a 27-in BenQ LCD monitor with a spatial resolution of 2560 × 1440 pixels and a refresh rate of 144hz. The operating system was Windows 7, and Matlab Psychtoolbox (Brainard, 1997; Pelli, 1997) was used to create all stimuli.

Design.

The target and distractor colors were selected from a color wheel defined in LAB color space (a,b coordinates = 0, 0; luminance = 70; radius = 39). Two target colors (190°, 274°) were counterbalanced across subjects. Each subject was assigned a single target color throughout the experiment.

In the unidirectional group, the three distractors in each visual search training trial were different from each other and always 5°, 10°, and 15° away from the target color. The rotational direction (negative or positive) of those three distractors from the target color was counterbalanced across subjects: half of the subjects always saw negatively rotated distractors (−5°, −10°, and −15°) and half saw positively rotated distractors (5°, 10°, and 15°). These distractors were chosen to exceed the average just noticeable difference, yet be confusable with the target when presented in a competitive visual search context (Geng, DiQuattro & Helm, 2017). Because the two target colors and distractor color directions did not affect performance, ps > .2, data from these conditions were collapsed to maximize power. For descriptive simplicity, the distractors in the visual search training trials in both groups (i.e., “trained” distractor colors) will always be referred to as positive rotations from the target (i.e., 5°, 10°, 15°) and the non-target colors opposite to the distractor colors that appeared only in the template probe trials (i.e. “untrained” colors) will be labeled as negative rotations from the target (i.e., −5°, −10°, −15°).

In the bidirectional group, the visual search displays were identical to those used in the unidirectional group, but now the direction of the distractor sets (negative, positive) were randomly interleaved within a single subject: Half of the trials contained the three positive color distractors (i.e., 5°, 10°, 15°) and the other half contained the negative color distractors (i.e., −5°, −10°, −15°). An initial analysis was conducted to assess whether the specific target color affected performance, but there were no significant differences, ps > .3. Thus, all analyses collapse across the two target colors.

Subjects in both the unidirectional and bidirectional groups, saw the same colors during the template probe task. The colors included the target color (color 0°), and the three colors from each side of the target (i.e., −5°, −10°, −15°, 5°, 10° and 15°). Because the template probe task was identical across the two groups, we were able to assess the consequences of the specific visual search context on the target representation.

Procedure.

Prior to the start of the experiment, an example of the target color was presented. The visual search training task (Figure 3A) began with the presentation of four circles (3° of visual angle in diameter) for 1000ms on a gray background (37.0 cd/m²). The target color was always present and was located randomly at one of the 4 vertexes along an imaginary square (6° of horizontal and vertical visual angle from center to edge) while the distractors appeared at the other 3 vertices. In the unidirectional group, the three distractors were either all negative rotations (−5°, −10° and −15°) or positive rotations (5°, 10° and 15°) from the target color. The distractor set (negative or positive) was counterbalanced across subjects, so that each subject only saw one set of distractors. In the bidirectional group, the same distractors were used, but now the distractor set (negative or positive) was interleaved between trials, within each subject. Both groups saw the same stimuli, but individuals in the unidirectional group only saw one set of distractors while individuals in the bidirectional group saw both distractor sets. A number from 1–4 (1° of visual angle; white) was centrally located within each circle. Upon presentation of the display, participants searched for the predefined target-color circle and reported the number inside by pressing button ‘U’ for 1, ‘I’ for 2, ‘O’ for 3, or ‘P’ for 4 with their right hand. If no response was recorded within 2000ms, the trial automatically terminated. Auditory feedback was provided immediately following response or after 2000ms had elapsed (600hz tone for correct; 200hz tone for incorrect; no feedback for missing). A fixation cross (subtending .5° of visual angle; white) was centrally presented for 1000–1500ms before the next trial.

Figure 3.

Example of visual search and template probe tasks for Experiment 1. Two trials for each task are illustrated. (A) Visual search task: participants were instructed to locate the target color circle and report the number within. A high tone (600 hz) was given for correct responses, and a low tone (200 hz) for incorrect responses. The dashed squares illustrate the target but were not visible to the participants. (B) Template probe task: participants were instructed to report if the centrally presented color circle was target color or not. A medium tone (400 hz) occurred on all trials regardless of responses to equate the presence of auditory feedback with the visual search task. Non-target color values are exaggerated for visual clarity in both figures (see methods for true values).

In the color template probe task (Figure 3B), each trial consisted of a centrally presented circle (3° of visual angle in diameter) for 500ms, after which a circular checkerboard mask (3° of visual angle) was displayed for 66ms. Participants reported whether the presented color was the target color (“yes” response, button ‘U’) or not (“no” response, button ‘I’) with their right hand. The ratio between target “yes” and “no” trials was 3:4. An uneven ratio was used to maximize the number of non-target color presentations. A “no” response bias, even if present, would not be selective for the analyses of interest based on differences in “yes” and “no” responses to each color presented in the template probe task. Participants were informed that a pseudo-feedback (400hz) tone would be given no matter what their response was in order to equate the presence of auditory events between the visual search and template task. A fixation cross was centrally presented for 1434ms-1934ms before the next trial began. The trial was terminated if no response was made within 2000ms.

Prior to the beginning of the experiment, participants completed 32 practice trials composed of both visual search training and template probe tasks. Participants were instructed to fixate on the center cross throughout the whole experiment. The main experiment was composed of 320 visual search trials and 336 color probe trials. Trials were presented in four blocks, each containing alternating blocks of visual search training trials and template probe trials. Within each block, the first 4 alterations involved 10 visual search training trials followed by 7 template probe trials and the remaining 8 alterations had only 5 visual search training trials followed by 7 template probe trials. This design was created to maximize measurements of the target template on template probe trials between periodic visual search training. The use of independent probe trials is essential for measuring the contents of the attentional template as an ongoing memory representation of critical target features that is independent of concurrent visual search, which involves many processes beyond target representation, such as those necessary for resolving distractor competition.

Statistical Analyses.

The visual search training task was used to establish expectations for the target color and the probe trials measured the contents of the attentional template independent of simultaneous distractor competition. Probe trials assessed the likelihood of each of seven colors being identified as the target color. “Target yes” responses on probe trials were false alarms when the color was a non-target, but a hit when it was the target color. While the response rate for each color is independent from other colors, we hypothesize that the underlying source of the response profile across colors comes from an underlying distribution – i.e., the “tuning” of the target template. Therefore, in order to estimate the underlying tuning function of the target representation, we used the density function of the split normal distribution to model the probability of “target yes” responses to each probe color (Figure 4A and 4B). The split normal distribution was selected because it allows for estimation of asymmetrical standard deviations around the central tendency but reduces to a normal distribution when the estimated standard deviations are equivalent (see below).

The split normal distribution is formed by merging two opposite halves of two probability density functions of a normal distribution at their common mode. Equation 1 gives the probability density function for estimating a split normal distribution (Johnson, Kotz & Balakrishnan, 1994). The distribution takes the left half of normal distribution with parameters (μ, σ₁) and the right half of a normal distribution with parameters (μ, σ₂), and scales them to a common value $\text{f}(\text{μ})=\frac{\sqrt{2\pi }}{{\sigma }_1+{\sigma }_2}$ at the mode, μ. In a special case when σ₁ = σ₂, the split normal distribution reduces to a normal distribution.

$$h(x|\mu ;{\sigma }_1;{\sigma }_2)=\{\begin{array}{ll}\frac{\sqrt{2\pi }}{{\sigma }_1+{\sigma }_2}{\text{e}}^{-\frac{{(x-\mu )}^2}{{\sigma }_1{}^2}},\hfill & ifx<\mu ;\hfill \\ \frac{\sqrt{2\pi }}{{\sigma }_1+{\sigma }_2}{\text{e}}^{-\frac{{(x-\mu )}^2}{{\sigma }_2{}^2}},\hfill & otherwise\text{.}\hfill \end{array}$$ (1)

Because the response for each color in our probe task is independent, the probabilities of responding “yes” to all the colors do not sum to 1. We therefore introduced a subject-specific scaling parameter “a” that scales the distribution from each subject leading to equation 2. An individual who has a small value of “a” is more conservative in responding “yes”; conversely, an individual who has a large value of “a” is more liberal in responding “yes”.

$$f(x\left|a;\mu ;{\sigma }_1;{\sigma }_2)=ah(x\right|\mu ;{\sigma }_1;{\sigma }_2)$$ (2)

Instead of using a more conventional method for parameter estimation (e.g. maximum likelihood estimation), all parameters were estimated using hierarchal Bayesian analysis (HBA). The hierarchical approach is particularly useful for this study given the small number of data points per subject because it captures commonalities across individuals and at the same time estimates each individual’s parameter values (Gelman, et al., 2013). To perform HBA, we used the R package, Rstan (Stan Development Team, 2016). Normal and half Cauchy distributions were used to set the hyper priors of the normal mean (μ), standard deviations (σ), and free parameter (a) (Equation 3). We used weakly informative prior distributions (Gelman et al., 2013), to avoid biasing the posterior distributions.

$$\begin{array}{l}\text{μ}~\text{ Normal }(0,15)\hfill \\ {\text{σ}}_1,{\text{σ}}_2~\text{ Cauchy }(0,30)\hfill \\ \text{a}~\text{ Cauchy }(0,50)\hfill \end{array}$$ (3)

A total of 40000 samples were drawn after 20000 warming-up samples from 8 chains. We estimated individual and group parameters separately for each group (“uni-direction” and “bidrection” group). Goodness of fit was visually inspected with the posterior predictive check method (Gelman et al., 2013).

The mode value (μ) reflects the central tendency of the target template. Non-zero values indicate that the point in color space over which the target template is centered. For the unidirectional group, a positive μ indicates a shift towards the visual search distractor colors and a negative value indicates a shift away from the visual search distractor colors. But for the bidirectional group, a non-zero μ value would not reflect distractor properties because the distractor colors came from the both sides of the target color. Thus, μ serves as the statistical analogue for the magnitude of template shifting. The sigma values (σ) reflect the dispersion of the target template. Therefore, σ₁ (subsequently referred as σ_neg in reference to the color stimulus space, see above) characterizes the width of the template over “negative” color values; and σ₂ (σ_pos) represents the width of the template over “positive” colors. Recall that negative colors were never seen as distractors during visual search in the unidirectional group, but appeared as distractors in the bidirectional group.

In addition to modeling the “target yes” responses with the split-normal distribution, we also directly compared the false alarm rates between the negative and positive non-target colors. This analysis is complementary to the analysis of in σ_neg and σ_pos from the split-normal distribution. However, in contrast to σ_neg and σ_pos, which are estimated in conjunction with μ, the raw false alarm rates are not related to estimations of central tendency and therefore are a more direct approximation of the likelihood of mistaking a non-target color as the target color. The “target yes” data are analyzed using ANOVA and posthoc analyses are always corrected for multiple comparisons using the Bonferonni method.

Finally, in addition to null hypothesis testing, we also computed Bayes factors (BF) (Rouder et al., 2009) for all student-t statistical analyses using BayesFactor package in r (Morey, Rouder & Jamil, 2015). The BF is a statistical index of the evidence the data provides for either the null or the alternative hypothesis. BF values in favor of the null hypothesis are denoted as BF₀₁ and for the alternative as BF₁₀. It is important to note that these methods for estimating the underlying template likely reflect aggregate sensory and decisional mechanisms involved at various stages of processing (e.g., Smith & Ratcliff, 2009).

Results

The template probe trials asked subjects to indicate whether a particular colored stimulus was the target, or not. Although the proportion of “target yes” responses were independent for each color probed, we hypothesized that the responses reflect the “tuning” of the underlying target template. In order to recover the tuning profile of the template, the target probe data were first modeled with the split-normal distribution (see Methods above, Figure 4A and 4B), which estimates template tuning use estimated μ, σ_neg and σ_pos values. However, because σ_neg and σ_pos are estimated relative to μ, we conduct a second complementary analysis of the raw false alarm rates, which are an unbiased estimate of the likelihood of mistaking colors as the target.

Modeling the target template with the split normal distribution.

To determine if there was a difference in the shift in central tendency of the template between groups, the μ values were compared using an independent sample t test. The difference was significant, t(38) = −5.29, p <.001, d = −1.67, BF₁₀ = 2,814.47, indicating that the μ value for the unidirectional group was more negative than the bidirectional group. Additionally, the μ values from each group were compared against 0° (the veridical target feature). The μ values of both groups were significantly different from 0°, but only the unidirectional group’s μ was negatively shifted (M = −3.64°, sd = 3.38°): t(19) = −4.82, p < .001, d =−1.08, BF₁₀ = 240.52. The bidirectional group value was positively shifted (M = .94°, sd = 1.90°): t(19) = 2.22, p = .04, d = .50, BF₁₀ = 1.69), although the evidence based on Cohen’s d and the BF was relatively weak (cf. BF₁₀ = 240.52 for the unidirectional group). The results indicate that the unidirectional group shifted their target representation away from the distractor.

Next, the σ values were entered into a 2 color direction (negative, positive) x 2 group (unidirectional, bidirectional) ANOVA. There was a significant main effect of color direction, F(1, 38) = 7.29, p = .01, ${\eta }_p{}^2$ = .16, a marginally significant main effect of group, F(1, 38) = 3.23, p = .08, ${\eta }_p{}^2$ = .08, and a marginally significant interaction, F(1, 38) = 3.17, p = .08, ${\eta }_p{}^2$ = .07. The main effect of color direction was due to larger σ_neg than σ_pos values overall. Although the interaction between groups was only marginally significant, our priori hypothesis was that there would be asymmetrical sharpening in the unidirectional group, but not the bidirectional group. We therefore additionally conducted simple effects t tests along with BF to compare σ values for color direction differences in each group. The paired sample t tests showed that the unidirectional group had a significantly larger σ_neg than σ_pos values, t(19) = 3.01, p = .007, d = .67, BF₁₀ = 6.71. In contrast, there was no statistical difference between σ_neg and σ_pos in the bidirectional group, t(19) = .69, p = .50, d = .15, BF₀₁ = 3.48. Thus, while the BF in the unidirectional group indicates that the data provides 6.71 times more evidence for the alternative hypothesis than the null hypothesis, the BF in the bidirectional group indicates that the data are 3.48 times more likely if the null hypothesis is true than the alternative. Together, these results are consistent with asymmetrical sharpening in the the unidirectional group, but not the bidirectionalal group; however, the results must be interpreted with caution given the nonsignificant interaction in the ANOVA.

Analysis of false alarm rates.

Unlike σ values which are influenced by estimates of μ, we turn next to the raw false alarm data as a more direct measurement of which non-target colors were mistaken as the target color. The false alarm rates of non-target colors during the identification task (Figure 5A) were entered into a 2X2 ANOVA to test for differences between color direction (negative, positive) and group (unidirectional, bidirectional). There was a significant main effect of color direction, F(1, 38) = 23.55, p < .001, ${\eta }_p{}^2$ = .38, a nonsignificant effect of group, F(1, 38) = .07, p = .79, ${\eta }_p{}^2$ = .002, as well as a significant interaction, F(1, 38) = 38.07, p < .001, ${\eta }_p{}^2$ = .51. Post-hoc t-tests, corrected for multiple comparisons, indicated the unidirectional group had significantly more false alarms to negative colors compared to positive colors, t(19) = 8.09, p < .001, d = 1.81, BF₁₀ = 108,165.60, whereas the bidirectional group had similar false alarm rates for colors in both directions, t(19) = −.96, p = 1, d = −.21, BF₀₁ = 2.86. Additionally, the unidirectional group had higher false alarm rates for negative colors than the bidirectional group, t(38) = 3.99, p = .001, d = 1.26, BF₁₀ = 86.77, but the bidirectional group had higher false rates for positive colors, t(38) = −5.53, p < .001, d = −1.75, BF₁₀ = 5,559.68). These results strongly support the hypothesis that asymmetrical sharpening occurs when highly similar visual search distractors that are linearly separable from the target and can be expected from trial-to-trial, but not when distractor sets from both sides of the target color alternate between trials.

Figure 5.

(A) False alarm rates collapsed across color degree for each group and color direction in Experiment 1. (B) Visual search accuracy for each group. (C) Reaction times for each group in visual search task. All error bars are SEM.

Visual search performance.

Having established that the target template in the unidirectional group was shifted and asymmetrically sharpened, we next assessed visual search performance. Recall that stimuli were identical in the unidirectional and bidirectional groups, except that only one of the two distractor color sets were used for a given participant in the unidirectional group whereas distractor sets were randomly interleaved trial-by-trial in the bidirectional group. Accuracy (Figure 5B) and reaction time (RT, Figure 5C) from the visual search trials were entered into between-groups t tests. Only RT data from correct trials were included in these analyses. The results show that the unidirectional group (M = 91%, sd = 6%) had significantly higher accuracy than the bidirectional group (M = 63%, sd = 12%), t(38) = 9.06, p < .001, d = 2.86, BF₁₀ = 119,350,885. The unidirectional group (M = 841ms, sd = 154ms) also had significantly shorter RTs than the bidirectional group (M = 1021ms, sd = 137ms), t(38) = −3.90, p < .001, d = 1.23, BF₁₀ = 69.17. The unidirectional group accuracy was on average more than 25% greater than the bidirectional group and RTs more than 150ms shorter. The big difference in performance indicates that being able to shift and sharpen the target representation away from expected distractors effectively increased the psychological distinctiveness of the target from distractors and enhanced performance substantially.

In addition, to compare performance between groups, we investigated the effect of repetition in distractor color set in the bidirectional group. If learned expectations contribute to changes in the attentional template and better visual search performance, we might expect chance repetitions to also produce smaller advantages. In other words, we expected that subjects would have better performance on “repeat” trials when distractors are the same as the N-1 trial than on “switch” trials, when the distractor set is opposite from the N-1 trial. A paired sample t test showed that the accuracy of distractor repeat trials (M = 67%, sd = 13%) was significantly higher than the accuracy of distractor switch trials (M = 61%, sd = 12%), t(19) = 4.03, p < .001, d = .90, BF₁₀ = 48.78. Similarly, RT of distractor repeat trials (M = 943ms, sd = 133ms) was shorter than distractor switch trials (M = 1116ms, sd = 155ms). t(19) = −6.86, p < .001, d = −1.53, BF₁₀ = 12,272.99. This suggests that repeated exposures to the same distractor set helps facilitate visual search and that learned expectations about distractor context over longer periods of time might build upon mechanisms of repetition priming.

Discussion

The goal of Experiment 1 was to test the hypotheses that the target representation held in the attentional template is shifted and asymmetrically sharpened away from distractors that are expected to be linearly separable and highly target-similar. Observers were either exposed to only a linearly separable distractor set during visual search “training” trials, or distractor sets from both sides of the target color that varied from tiral-to-trial. On separate “probe” trials that were identical between groups, we found evidence that the unilateral group shifted the central tendency of the target representation away from distractors and sharpened the boundary between the target and distractors. Additionally, performance on the visual search task was substantially better in the unidirectional group, suggesting that shifting and sharpening are attentional mechanisms that increase the psychological distance between targets and highly similar distractors and aid search efficiency. This establishes a clear link between the contents of the target template and visual search performance.

Experiment 2

Experiment 1 supported that hypothesis that the unidirectionality of target-similar distractors adjusted the shape of the target template. However, the experiment did not address whether shifting and sharpening are separable mechanisms that respond to different aspects of distractor expectations. To address this in Experiment 2, we manipulated the strength of distractor competition, defined by the target-to-distractor similarity across blocks while holding the linear separability of all distractors constant. We hypothesized that shifting would occur in response to the overall separability of the distractor set and therefore stay constant across blocks, but that sharpening would change with the strength in competitive pressure from distractors and increase over blocks.

Method

Participants.

Twenty new UC Davis undergraduates (7 males, 3 left-handed, ages 19–27) participated in Experiment 2. Each provided written informed consent in accordance with the local ethics clearance as approved by the Nation Institutes of Health. Color vision was assessed through self-report and an online color blindness test (https://colormax.org/color-blind-test). All had normal or corrected-to-normal visual acuity and color vision.

Design.

The experimental design was the same as the unidirectional group in Experiment 1, with the following exceptions: There were five distractor color sets manipulated over five sequential blocks (Figure 6). The first distractor color set in block 1 was composed of three identical gray distractors (average across LAB color space, 67.4 cd/m²), which served as a baseline condition. Distractors in the remaining four blocks increased in similarity to the target: (60°, 55°, 50°) in block 2, (45°, 40°, 35°) in block 3, (30°, 25°, 20°) in block 4, and (15°, 10°, 5°) in block 5. As before, each of the three different distractor colors in each set were present in each visual search display. Distractor color (negative or positive rotations from the target color) was counterbalanced across subjects. As in Experiment 1, there were no spurious differences based on counterbalancing of target and distractor colors (ps > .39), so the data were collapsed in all subsequent analyses. Consistent with our labeling convention in Experiment 1, the distractors from the visual search trials (i.e., “trained” distractor colors) are always referred to as being “positive” rotations from the target (i.e., 5°, 10°, 15°) and the colors that appeared only in the template probe trials (i.e. “untrained” colors) are labeled as “negative” values from the target (i.e., −5°, −10°, −15°). The template probe task was identical to the one in the Experiment 1 and remained the same across five blocks. Recall that the colors in the probe task included the target color (color 0°), and the −5°, −10° −15, 5°, 10° and 15° color rotations from the target.

Figure 6.

Visual search task design in Experiment 2. One visual search trial for each block is shown here to illustrate the increase in target-to-distractor similarity over blocks. The template probe task was identical to that in Experiment 1 (Figure 3B). See Methods for color values.

Procedure.

The procedure of Experiment 2 was identical to Experiment 1, except for the additional factor of target-distractor similarity, which increased in 5 blocks over the experiment (Figure 6). The order of the 5 blocks was held constant moving from blocks with dissimilar distractors to those with greatest distractor similarity so as to avoid possible carry-over effects in learning about the strength of competition. The session began with 32 practice trials composed of both visual search and template probe trials. Each block contained 112 visual search training trials and 112 probe trials. Each block began with 4 alternations of 10 visual search training trials and 7 probe trials followed by 12 alternations of 6 visual search training and 7 probe trials. As in Experiment 1, this design was created to maximize the number of template probe trials with periodic visual search training.

Results

Modeling the target template with the split normal distribution.

Following the analysis strategy from Experiment 1, we fitted the likelihood of “target yes” responses with a split normal distribution for each person and block in order to estimate the central tendency and variance of the “tuning” of the target template (Figure 7). The μ values (Figure 8A) were then entered into a one-way ANOVA with the factor block (1–5). There was a significant main effect of block, F(4, 76) = 6.65, p < .001, ${\eta }_p{}^2$ = .26. We then measured the difference in μ values between two consecutive blocks using post-hoc t tests (Bonferroni corrected for multiple comparisons). Consistent with visual inspection of the data (Figure 8A), the only significant difference was between block 1 (M = 0.42°, sd = 2.97°) and block 2 (M = −2.13°, sd = 3.10°), t(19) = 3.32, p = .01, d = .74, BF₁₀ = 12.04; all other ts < 1.54, ps > .56, ds < .34, BF₀₁ > 1.56. Specifically, there was a negative shift in μ in block 2 when colored distractors were introduced and this shift was sustained in all remaining blocks, suggesting that the target template shifted as soon as linearly separable distractors were introduced and that the magnitude of the shift did not change with increases in distractor competition (i.e., distractor similarity) over blocks.

Figure 7.

Group averages of “target yes” responses for each block in Experiment 2. Black solids lines are split normal distribution fits. The center gray bar indicates the true target color. All error bars are SEM.

Figure 8.

Model fits and false alarm rates from the template probe task in Experiment 2. (A) Mode values estimated from the split normal distribution. (B) Standard deviation of each color direction estimated from the split normal distribution. (C) False alarm rates to negative and positive non-target color probes. All error bars are SEM.

Next, we examined changes in σ_neg and σ_pos as a function of block (Figure 8B). An ANOVA with block (1–5) and color direction (negative, positive) as factors yielded a significant main effect of block, F(4, 76) = 5.16, p < .001, ${\eta }_p{}^2$ = .21, a significant main effect of color direction, F(1, 19) = 9.43, p = .006, ${\eta }_p{}^2$ = .33, as well as a significant interaction, F(4, 76) = 8.19, p < .001, ${\eta }_p{}^2$ = .30. The interaction was due to a linear decrease in σ_pos over blocks, but no change in σ_neg over blocks (Figure 8B). To confirm this, slope values were calculated as a summary value of how σ changed over blocks. We found a significant negative slope for σ_pos, t(19) = −7.95, p < .001, d = −1.78, BF₁₀ = 84,968.52, but a non-significant slope for σ_neg, t(19) = .36, p = .73, d = .08, BF₀₁ = 4.06. These results demonstrate that the positive half of the target template was selectively sharpened over blocks to counteract competition from highly similar distractors, while the width of tuning over the negative half did not change. This indicates that asymmetrical template sharpening occurs selectively in response to distractor competition and not just distractor color directionality, but lateral shift in the target template occurs in response to color directionality and is insensitive to similarity.

Analysis of the false alarm rate.

The raw false alarm rates for negative and positive non-target colors were entered into a 2 color direction (negative, positive) x 5 block (1–5) repeated measures ANOVA (Figure 8C). Recall that the raw false alarm rates are not related to estimations of central tendency and are a direct likelihood of mistaking a non-target color as the target. There was a significant main effect of color direction, F(1, 19) = 20.29, p < .001, ${\eta }_p{}^2$ = .52, a significant main effect of block, F(4, 76) = 9.66, p < .001, ${\eta }_p{}^2$ = .34, and a significant interaction, F(4, 76) = 9.57, p < .001, ${\eta }_p{}^2$ = .34. Similar to the analyses of σ_neg and σ_pos, the interaction was due to a linear decrease in positive false alarm rates over blocks, but no change in negative false alarm rates over blocks (Figure 8C). To confirm this, slope values were calculated as a summary value of how false alarm rates changed over blocks. We found a significant negative slope for σ_pos, t(19) = −7.63, p < .001, d = −1.71, BF₁₀ = 48,801.63, but a non-significant slope for σ_neg, t(19) = .67, p = .51, d = .15, BF₀₁ = 3.52. These results were consistent with standard deviation results, suggesting asymmetrical template sharpening in response to distractor competition.

Visual search performance across blocks.

The previous results demonstrate that the representation of the target color changed across blocks in response to the visual search context. Next, to understand how the increasingly asymmetrical template relates to search performance, we analyzed search accuracy (Figure 9A) and RT (Figure 9B) using two one-way ANOVAs with block as a within-subject factor. Both accuracy, F(4, 76) = 23.95, p < .001, ${\eta }_p{}^2$ = .56, and RT, F(4, 76) = 50.87, p < .001, ${\eta }_p{}^2$ = .73, were significant. The results were due to poorer performance in block 5 compared to any other block (Figure 9), post hoc t-tests comparing block 5 with all other blocks (accuracy: all ts < −4.88, ps < .005, ds < −1.00, (BF₁₀)s > 270; RT: all ts > 8.14, ps < .001, ds > 1.82, (BF₁₀)s > 118,707; RT was also longer in block 2 than block 3, t(19) = 4.76, p = .001, d = 1.06, BF₁₀ = 210.18). Visual search performance was consistently at ceiling until block 5, when distractor competition was strongest. This suggests that changes in the target template were sufficient to maintain performance at ceiling over increases in distractor similarity for the first four blocks, but competition was sufficiently strong in block 5 that an increase in template sharpening was no longer able to fully exclude highly similar distractors.

Figure 9.

Experiment 2 visual search accuracy (A) and reaction time (B). All error bars are SEM.

Discussion

The goal of Experiment 2 was to test whether the shifted central tendency and asymmetrical sharpening of target template identified in Experiment 1 were due to different distractor properties during visual search. The results demonstrated that the shift in central tendency occurred in response to the distractor set being linearly separable from the target, but was not sensitive to changes in target-to-distractor similarity. In contrast, asymmetrical sharpening, seen in the exclusion of positive (but not negative) non-target colors within the target template, was continuously updated as distractor similarity increased. This suggests that distractor competition dynamically sharpened the representational boundary between the target and the distractors in order to better exclude highly similar distractors. Together, the results suggest that shifting and sharpening of the target template occur in response to different distractor properties to maximize the ability to distinguish target from distractors during visual search.

Experiment 3

Experiment 2 provided evidence that shifting and sharpening are separate mechanisms to optimize the target template for visual search within predictable distractor contexts. However, it is possible that the continuous asymmetrical sharpening seen in Experiment 2 was not due to increases in distractor competition (as we concluded), but simply due to practice over time. In order to rule out this alternative hypothesis, Experiment 3 was identical to Experiment 2 (5 blocks of equal duration), but only the most dissimilar distractor set was used (identical to block 2 in Experiment 2).

Method

Participants.

Twenty new UC Davis undergraduates (4 males, 1 left-handed, ages 18–27) participated in Experiment 2. Each provided written informed consent in accordance with the local ethics clearance as approved by the National Institutes of Health. Color vision was self-reported and assessed with an online color blindness test (https://colormax.org/color-blind-test). All had normal or corrected-to-normal visual acuity and color vision.

Design & Procedure.

All stimuli and procedures were identical to Experiment 2, with one exception: The distractor set was identical in blocks 2–5 (Figure 10), (i.e., 60°, 55°, and 50° from the target color; identical to block 2 of Experiment 2). The two directions of distractors and two target colors were again counterbalanced across subjects and because there were no spurious differences (ps > .17), the data were collapsed in all subsequent analyses, with “negative” colors referring to colors that were not seen during visual search and “positive” colors referring to values seen as distractors during visual search. The template probe trials were identical to those in Experiment 2.

Figure 10.

Visual search task design in Experiment 3. One visual search trial for each block is shown here to illustrate that the target-distractor similarity remained constant in blocks 2–5. The template probe task was identical to Experiment 1 (Figure 3B). See Methods for color values.

Results

Modeling the target template with the split normal distribution.

Consistent with the previous experiments, the likelihood of “target yes” responses were fit by a split normal distribution for each person and each block (Figure 11). We found a significant main effect of block on μ values, F(4, 76) = 3.08, p = .02, ${\eta }_p{}^2$ = .14 (Figure 12A). We then measured the difference in μ values between two consecutive blocks using post-hoc t tests (Bonferroni corrected for multiple comparisons). The main effect was driven by a significant difference between block 1 (M = −.32°, sd = 2.84°) and block 2 (M = −2.10°, sd = 2.91°), t(19) = 2.91, p = .04, d = .65, BF₁₀ = 5.63, which replicates Experiment 2. The central tendency of the target representation shifted away from visual search distractor colors once they appeared in block 2 following the all gray distractor control condition. Moreover, the μ values remained constant over block 2–5, suggesting that the magnitude of shift was not affected by practice over time.

Figure 11.

Group averages of “target yes” responses for each block in Experiment 3. Black solids lines are split normal distribution fits. The center gray bar indicates the true target color. All error bars are SEM.

Figure 12.

Model fits and false alarm rates from the template probe task in Experiment 3. (A) Mode values estimated from the split normal distribution. (B) Standard deviation of each color direction estimated from the split normal distribution. (C) False alarm rates to negative and positive non-target color probes. All error bars are SEM.

Additionally, we conducted a two-way ANOVA with factors block (1–5) and experiment (Experiment 2, Experiment 3) to assess if there is between experiment differences for μ values. The results yielded a significant main effect of block, F(4, 152) = 9.38, p < .001, ${\eta }_p{}^2$ = .20, a non-significant main effect of experiment, F(1, 38) = .005, p = .95, ${\eta }_p{}^2$ = 0 and a non-significant interaction, F(4, 152) = .50, p = .74, ${\eta }_p{}^2$ = .01. The lack of between experiment differences suggest that the magnitude of shift was insensitive to specific distractor feature values, and only sensitive to the directionality of the entire distractor feature space.

Next, we analyzed σ_neg and σ_pos values as a function of block (Figure 12B). An ANOVA with color direction (negative, positive) and block (1–5) as factors yielded a non-significant effect of color direction, F(1, 19) = .18, p = .68, ${\eta }_p{}^2$ = .009, a significant main effect of block, F(4, 76) = 10.13, p < .001, ${\eta }_p{}^2$ = .35, and a non-significant interaction, F(4, 76) = .80, p = .53, ${\eta }_p{}^2$ = .04. The lack of a difference between σ_neg and σ_pos as a function of color direction suggests that asymmetries in template sharpness did not occur simply with practice. This result is evidence against the possibility that changes in σ asymmetry in Experiment 2 were due simply to practice effects.

To quantify the apparent difference in results for σ_neg and σ_pos between experiments, we calculated slope of change in σ values over block for Experiments 2 and 3 (Figure 13A). Slope values were calculated as a summary index of how σ changed over blocks. The results yielded a significant main effect of color direction, F(1, 38) = 4.93, p = .03, ${\eta }_p{}^2$ = .11, no main effect of experiment, F(1, 38) = 1.15, p = .29, ${\eta }_p{}^2$ = .03, but a significant interaction, F(1, 38) = 12.95, p < .001, ${\eta }_p{}^2$ = .25. Post hoc t-tests found that for the slope of σ_pos values was significantly more negative in Experiment 2 than Experiment 3, t(38) = −2.59, p = .01, d = −.82, BF₁₀ = 3.95. However, the opposite pattern was found for the slope of σ_neg: the slope was more negative in Experiment 3 than 2, t(38) = −2.34, p = .02, d = −.74, BF₁₀ = 2.52. This cross-over interaction indicates that the asymmetry in the width of template tuning found in Experiment 2 was due to the change in the strength of competition during visual search and not practice.

Figure 13.

Comparison of slope values for model standard deviations and false alarm rates over block for Experiment 2 and 3. (A) the slope values for standard deviations in each color direction and each experiment. (B) the slope values for false alarm rates in each color direction and each experiment. All error bars are SEM.

Analysis of the false alarm rate.

Similar to Experiment 2, the false alarm rates collapsed across different negative and positive color degrees (Figure 12C) were entered into a 2X5 repeated measures ANOVA with factors color direction (negative, positive) and block (1 to 5). There was a significant main effect of block, F(4, 76) = 4.16, p = .004, ${\eta }_p{}^2$ = .18, a marginally significant main effect of color direction, F(1, 19) = 3.39, p = .08, ${\eta }_p{}^2$ = .15, and a significant interaction, F(4, 76) = 4.25, p = .003, ${\eta }_p{}^2$ = .18. The interaction was due to a significant difference between negative and positive colors in block 2 (t(19) = 2.94, p = .042, d = .66, BF₁₀ = 5.93), but none others (all ts < 2.5, ps > .10, ds < .56, (BF₁₀)s < 2.78). These results converge with those from modeling of the split-normal distribution and suggest that the asymmetry of the template tuning was not systematic and more importantly did not increase steadily over time, as it did in Experiment 2.

Next, to examine differences in asymmetrical sharpening between Experiments 2 and 3, the slope of false alarm rates over block were (Figure 13B) entered into a two-way ANOVA with factors color direction (negative, positive) and experiment (Experiment 2, Experiment 3). The results yielded a significant main effect of color direction, F(1, 38) = 20.17, p < .001, ${\eta }_p{}^2$ = .35, a non-significant main effect of experiment, F(1, 38) = 2.10, p = .16, ${\eta }_p{}^2$ = .05, and a significant interaction, F(1, 38) = 6.31, p = .02, ${\eta }_p{}^2$ = .14. Post hoc t-tests found that only the slope of the positive false alarm rates was different between Experiment 2 and Experiment 3, t(38) = −3.17, p = .003, d = −1.00, BF₁₀ = 12.70, but not the slope of negative false alarm rates, t(38) = .95, p = .35, d = .30, BF₀₁ = 2.27 (Figure 13B). Specifically, the slope of false alarm rates was more negative over positive distractors in Experiment 2, suggesting that the template tuning continued to sharpen with distractor similarity in Experiment 2, but not with just practice in Experiment 3. These results again support the conclusion that the asymmetrical sharpening effect found in Experiment 2 was due to the change in the strength of competition during visual search and not practice.

Visual search performance across blocks.

Visual search accuracy (Figure 14A) and RT (Figure 14B) were entered into two one-way ANOVAs using block (1–5) as the within subject variable. A significant effect was observed for RT, F(4, 76) = 2.50, p = .05, ${\eta }_p{}^2$ = .11, but not accuracy, F(4, 76) = .44, p = .78, ${\eta }_p{}^2$ = .02. RT was longer in block 2 (M = 682ms) than block 1(M = 661ms), t(19) = 2.58, p_uncorrected = .02, p_corrected = .20, d = .58, BF₁₀ = 3.08. RT was also shorter in block 5 (664ms) than block 2, t(19) = −2.53, p_uncorrected = .02, p_corrected = .20, d = −.57, BF₁₀ = 2.84, and block 4 (671ms), t(19) = −3.01, p_uncorrected = .007, p_corrected = .07, d = −.67, BF₁₀ = 6.74. This indicated that visual search performance improved over blocks, in confirmation of expected practice effects that were independent of stimuli, and in contrast to the decrease in performance in block 5 in Experiment 2.

Figure 14.

Experiment 3 visual search accuracy (A) and reaction time (B). All error bars are SEM.

Discussion

The goal of Experiment 3 was to test if the asymmetries in template sharpening seen in Experiment 2 could be due to practice effects over time. However, there was no systematic increase in asymmetrical sharpening over blocks, suggesting that the effects seen in Experiment 2 were due to changes in distractor competition and not time. Moreover, direct analyses of data from the two experiments showed greater asymmetrical sharpening in Experiment 2, consistent with our conclusions that distractor competition, and not practice, is responsible for the degree of asymmetrical sharpening in template representations. Finally, also consistent with findings from Experiment 2, there was a shift in the template central tendency away from distractors that occurred early (block 2) and was sustained throughout the experiment. This suggests that shifts in template representations occur rapidly in response to the directionality of the distractor set, but not specific distractor values.

General Discussion

Theories of visual attention posit that individuals hold target relevant information in an “attentional template” during visual search (Bundesen, 1990; Bundesen, Habekost & Kyllingsbaek, 2005). Recent research has shown that the “tuning” of the attentional template is not always veridical, as previously assumed, and can be shifted away from distractors to enhance “off-target” features to increase the distinctiveness of the target representation from distractors (Becker, Folk & Remington, 2010; Becker et al., 2014; Navalpakkahm & Itti, 2007; Scolari, Byers & Serences, 2012; Scolari & Serences, 2009; Soto et al., 2008). It has also been suggested that the template may be asymmetrically sharpened to increase the precision of the boundary between target and distractors (Geng, DiQuattro & Helm, 2017). However, these previous studies have not explored whether different properties of the visual search distractors might independently contribute to template shifting or sharpening. The goal of the current experiments was to address this question. We hypothesized that shifting and asymmetrical sharpening would occur in response to different qualities of distractor pressure.

We tested this hypothesis in three experiments by separately manipulating distractor directionality (i.e., the overall linear separability of visual search distractors from the target) and the strength of distractor competition (i.e., the similarity of visual search distractors to the target). Importantly, in all experiments, visual search trials were used to “train” the template. The template was then measured by a separate “template probe” task in which participants classified colors as the target or a non-target. Using a separate probe task is necessary to measure the contents of the template, which are presumed to be held in memory (Giesbrecht, Sy, & Guerin, 2012; Woodman, Carlisle, & Reinhart, 2013; Myers et al., 2015), uncontaminated by attentional processes involved in selecting the target from concurrent distractors. This method was first used by Navalpakkam and Itti (2007) (see also Scolari & Serences, 2009) in order to test hypotheses of how expectations for a visual search display adjusts sensory gain.

However, because the attentional template is a memory representation of the target, it does not operate only on sensory processing. There is a long history of work showing that template contents impact processing by modulating sensory processing and by serving as the comparator against which target-match or mismatch decisions are made (Duncan & Humphreys, 1989; Hout & Goldinger, 2015; Wolfe, 2007; Malcom & Henderson, 2010; Smith & Ratcliff, 2009; Geng and Witkowski, under review). Therefore, the contents of the template should be expected to affect both sensory processing as well as decisional processes during visual search and our probe task. It is not possible, in our task, to dissociate the effect of the template contents on sensory and decisional processes, but this was not our goal. Instead, our goal was to get a direct measurement of the template without contamination from additional attentional mechanisms involved in target selection and concurrent distractor suppression. Importantly, the template probe task allowed us to do this by interrogating the template contents (in memory) independently from concurrent visual search, which involves additional mechanisms for target selection and distractor suppression.

There were three main findings. First, in all three experiments we found the central tendency of the target representation (i.e., the μ value estimated from modeling the split-normal distribution) was shifted away from the distractor colors. Importantly, the magnitude of this shift was insensitive to the levels of increased distractor competition in our experiments and practice over time. Thus, template shifting appears to occur in response to the entire distractor feature space, but not to specific feature values. This result may superficially appear to be at odds with findings from Scolari and Serences (2009) in which they did not find changes in the sensory template when distractors were very distant. However, their distractors were 90° rotations in orientation from the target and all three distractors were identical, possibly producing some target pop-out. Our distractors were variable, with the most extreme one being on 60° away in color space. It may be that our distractors were never sufficiently different from the target to render shifts in the template completely unnecessary. It remains an open question what specific conditions affect the magnitude of the shift in target representation.

Second, there was an asymmetrical sharpening in the width of the target template in response to the strength of distractor competition. This asymmetry was characterized by differences in the σ_neg and σ_pos parameters from split-normal distribution modeling as well as direct calculation of the false alarm rates to negative and positive non-target colors. In Experiment 2, we found that asymmetrical sharpening increased with distractor competition, suggesting that sharpening occurs to better exclude highly competitive distractors from erroneous target selection (see also, Geng, DiQuattro & Helm, 2017). Participants appeared to build a more asymmetrically precise template only when it was necessary to counteract pressure from competitive distractors. This gradual increase in sharpening suggests that there could be a potential carry over effect if we randomized distractor similarity in Experiment 2. Future work will be necessary to test how rapidly templates are updated in response to changes in distractor competition. Together, these results suggest that asymmetrical sharpening may be more effortful and is only used when necessary to increase the distance between the target and expected distractors; in contrast, shifting appears to occur even when distractor colors are easily distinguishable from the target. While our current results suggest that shifting and sharpening are sensitive to different distractor properties (set and similarity, respectively), it is not at all clear that these two profiles are due to one or more mechanisms. While we have suggested that they might be separate, others have shown that dynamical systems models can account for both (Simmering et al., 2006; Johnson et al., 2008). Further work is necessary to better understand the mechanisms underlaying shifting and sharpening of the template representation.

Third, the results suggest that shifting and sharpening the target template may increase the efficiency of visual search performance, although it is not possible to test causality between the template contents and visual search performance in these studies. This suggestion was particularly salient in Experiment 1 where exposure to highly similar distractors from only one side of target space during visual search produced relatively good search performance, but exposure to distractors from both sides of color space (at exactly the same degrees of similarity) produced substantially poorer performance. Interesting, this occurred despite the fact that on any single trial, the distractors were linearly separable from the target in both groups. The only difference was that the directionality of the distractor set was a between subject manipulation in the unidirectional group, but a within subject factor in the bidirectional group. The stimuli and procedures were otherwise identical, suggesting that the greater “difficulty” in the bidirectional group was due to an inability to shift the target representation (within the template) away from distractors, as the unidirectional group could. In Experiment 2, performance was sustained at a high level until the final block, when competition was the most severe. Perhaps continued sharpening protected performance against increasing distractor competition, but only to a degree. Together, the results suggest that changes in the target template are based on learned expectations of what the visual search context will look like in the next moment of time, not just what is currently available to the visual system. This pattern also highlights the difference between the contents of the attentional template, which is held over time, and the use of that template on any given trial to select the target from distractors. The template represents the psychological distance between the target and distractors, and therefore anticipates the visual search context.

Together these results suggest that attentional templates are not static recreations of target features, but are flexibly shaped to anticipate the quality of distractor competition. Specifically, this flexibility manifested in two ways: a shift in the central tendency of the target template away from linearly separable distractor features, and an asymmetrical sharpening to increase the precision of the target-to-distractor boundary (see Figure 2). One open question remains regarding the mechanisms that support these changes in representation. The optimal model of attentional gain suggests that template shifting increases signal-to-noise ratio by selectively increasing the gain of sensory neurons tuned to elements of the target that are most distant from distractors (Navalpakkam & Itti, 2007). Sharpening, instead, may result from decreasing the gain of sensory neurons tuned to distractor features (Reynolds & Heeger, 2009). Alternatively, the pattern we have seen may reflect a “higher level” memory representation that impacts visual search processes by modulating sensory gain as well as serving as the “template” against which decisional processes determine if a stimulus is a target match (Geng and Witkowski, under review; Smith & Ratcliff, 2009; Duncan & Humphreys, 1989; Hout & Goldinger, 2015; Malcom & Henderson, 2010). Although it is impossible to fully address the question of where in processing the template shift and sharpening measured in our task is encoded with the current behavioral data, the results suggest that more than one mechanism is used to adjust the target template in order to increase the representational distinctiveness of the target from expected distractors.

In conclusion, our experiments reveal that the target template is shaped by expectations regarding multiple distractor features. Expectations regarding the linear separability of the distractor set from the target produces a systematic shift in the target template away from distractors, but this adjustment is insensitive to the strength of distractor competition. In contrast, asymmetrical sharpening of the target template on the side of distractors is sensitive to strength of distractor competition. Both modulations of target template increase the psychological distinctiveness of targets from distractors and therefore facilitate better visual search performance.

Public significance statement:

Theories of attention hypothesize the existence of an “attentional template” in memory that contains goal-relevant information. For example, when searching for an apple, you might hold its color (red) and shape (round) in memory while looking for it. It is frequently assumed that visual search is most efficient when we hold veridical features that perfectly match the target. However, this is not always true when the distractor context is predictable. Our work demonstrates that expectations about the distractor features and similarity to the target can cause the remembered features of the target to be exaggerated away. The results suggest that visual search is most efficient when we search for target characteristics that are more distinct from visual distractors.