Modulation of neuronal responses during covert search for visual feature conjunctions

Abstract

While searching for an object in a visual scene, an observer's attentional focus and eye movements are often guided by information about object features and spatial locations. Both spatial and feature-specific attention are known to modulate neuronal responses in visual cortex, but little is known of the dynamics and interplay of these mechanisms as visual search progresses. To address this issue, we recorded from directionally selective cells in visual area MT of monkeys trained to covertly search for targets defined by a unique conjunction of color and motion features and to signal target detection with an eye movement to the putative target. Two patterns of response modulation were observed. One pattern consisted of enhanced responses to targets presented in the receptive field (RF). These modulations occurred at the end-stage of search and were more potent during correct target identification than during erroneous saccades to a distractor in RF, thus suggesting that this modulation is not a mere presaccadic enhancement. A second pattern of modulation was observed when RF stimuli were nontargets that shared a feature with the target. The latter effect was observed during early stages of search and is consistent with a global feature-specific mechanism. This effect often terminated before target identification, thus suggesting that it interacts with spatial attention. This modulation was exhibited not only for motion but also for color cue, although MT neurons are known to be insensitive to color. Such cue-invariant attentional effects may contribute to a feature binding mechanism acting across visual dimensions.

A dominant experimental approach to understanding the directed allocation of visual attention has been the search paradigm, in which subjects must identify targets defined uniquely by the conjunction of features they possess. Psychophysical studies of this sort identified two search strategies: Rapid search for salient targets that “pop out” in the presence of distractor stimuli and slower search for nonsalient targets requiring serial allocation of focal attention (1, 2). The second of these search strategies often results in search times proportional to the number of distractors and can be elicited when the target is defined by a unique combination of visual features, such as color and direction of motion (2, 3). Treisman and colleagues (2) hypothesized that the seriality of the feature conjunction search is a result of a feature binding process that requires allocation of focal attention. Behavioral studies indicate that the allocation pattern during conjunction search may have temporal structure that reflects a combination of the rapid parallel feature-based and slower serial spatial allocation mechanisms. For example, subjects can use target features to guide their search, such that objects possessing those features are processed preferentially (4–6). In particular, it has been hypothesized that guided search is accomplished via attentional allocation to specific features (e.g., refs. 5 and 6), which may be manifested as parallel modulations of neuronal responses to all objects that share the guiding feature. Recent human brain MR imaging studies supply some evidence for such parallel feature-based mechanism: Attention to a feature value (such as upward direction of motion or a specific color) boosts responses in early visual cortical areas to unattended stimuli sharing a feature with the target (7). Such feature-specific effects are detectable globally in the visual field even in the absence of visual stimulation (8). It remains to be seen, however, how the feature-specific guidance mechanisms interact dynamically with the serial attentional modulation during visual search.

Neurophysiological experiments employing a covert visual search paradigm have identified neuronal correlates of attention allocation in the ventral visual cortical stream. These effects consist of modulations of neuronal response to a search target in the receptive field (RF), which immediately precedes target identification (9–11) or “noticing” (12). Additionally, a recent study found neuronal correlates of feature-specific and serial focal attention in macaque visual area V4 during overt search for targets defined by either color or shape or both (13).

To characterize the dynamics of feature and spatial attention during visual search, we recorded responses of directionally selective neurons in cortical visual area MT of rhesus monkeys (Macaca mulatta). These animals were trained to perform a covert search task requiring identification of a target defined by a conjunction of color and motion cues, and to make a saccadic eye movement to the target location upon detection. Both humans and macaques exhibit search patterns consistent with serial search when performing this task (3, 14). MT neurons represent one of the features—visual motion—conjoined in our search stimuli (e.g., refs. 15–17), and they are influenced by spatial (18) and motion direction-specific attention (18). The choice of feature dimensions—color and motion—permitted us to address effects of both the intra-modal (direction-of-motion) and extra-modal (color) feature-specific attention in this cortical area. Because area MT is sensitive to direction of motion, we expected feature-specific modulations in response to a distractor sharing direction of motion with the target (motion “semitarget”). However, recent fMRI experiments predict the existence of an additional, cross-dimensional, effect (7), such that the response to a stimulus sharing color with the target (color semitarget) would also be modulated, despite the lack of color selectivity in area MT. The presence of cross-dimensional effects would suggest an object-based mechanism operating across multiple visual dimensions and grouping distractors according to shared features (19).

We observed activity of many MT neurons to be modulated in two ways. First, when the RF stimulus was a target, responses became elevated immediately before target identification (end-stage modulation). The modulation was more prominent during correct target detections as compared to erroneous saccades to distractors in the RF (18). Second, we observed transient feature-specific effects on neuronal responses. These latter effects were present when the RF stimulus was not a target but had one feature in common with the target, and they occurred as response elevations that often abated before target identification. Such modulations were present not only in responses to distractors sharing direction of motion but also color with the target. Thus the dynamic allocation of attention during search was guided by target features defined across multiple feature dimensions (i.e., both color and direction of motion). Both forms of modulation are consistent with a dynamic pattern, whereby at the initial stage of search attentional gain is enhanced globally over multiple objects sharing features with the target, but during the end stage of search the attentional focus converges on the target.

Results

Behavioral.

Two monkeys were trained to perform a search for targets defined by conjunctions of color and direction of motion (Fig. 1). Behavioral data collected during training experiments have been reported previously, along with data from human subjects that performed the same task (14). Briefly, for both humans and monkeys, the conjunction search task elicited search reaction times and error rates that were proportional to the number of distractors in the display (see also ref. 3). By contrast, reaction times and error rates for targets defined by single features were independent of the number of distractors (see Fig. S1). This behavioral pattern suggests that identification of color-motion conjunctions requires attentive scrutiny and may engage perceptual feature binding mechanisms (2, 5, 20).

Fig. 1.

Color/motion conjunction search task. The horizontal arrow along the bottom signifies passage of time within a trial. The following sequence occurred during each trial until either a fixation error occurred (trial aborted) or a saccade was made to a search item. A target-in-RF trial is shown. (A) Trials began with appearance of a centrally located white spot (0.2°), to which subjects were trained to fixate. The broken line represents a hypothetical RF; the arrow represents preferred direction. (B) After fixation was achieved, the cue appeared at the center of gaze for 1,000 ms. The cue consisted of a stationary circular aperture (1° of visual angle in diameter) within which moved a colored texture. (C) The cue was extinguished and the display contained only the fixation spot for additional 300 ms. (D) The search array appeared and remained on for the duration of the trial. The array contained eight stimuli, constructed similarly to the cue and distributed in a ring centered on the fixation spot. Ring eccentricity and stimulus diameter were optimized for each neuron. The search array was designed such that one item fell in the RF; that item moved in preferred or antipreferred direction. To obtain reward, the subject was required to search covertly (i.e., while maintaining central fixation) for the cued target. (E) Upon target identification, the subject was required to make an eye movement to its location and remain fixated at that location for 150 ms (white lines represent trace of eye position). (F) Correct choices yielded juice reward and darkening of search display. Incorrect trials concluded with brief dimming of nontargets before darkening of the display.

Experiment I: End-Stage Modulation of Response to Search Target in RF.

The primary objective of this experiment was to determine whether activity of neurons representing individual features of a target changes during the target detection phase of the search. This objective was approached by comparing response modulation to targets vs. nontargets (Table 1). We recorded from 71 neurons in two monkeys.

Table 1.

Experimental conditions for experiments I and II

	Condition	Behavioral significance of RF stimulus
		Target	Semitarget	Antitarget	Target absent
Experiment I
Sensory significance of RF stimulus	Preferred direction	Condition I.1		Condition I.3	Condition I.5
	Antipreferred direction	Condition I.2		Condition I.4	Condition I.6
Experiment II
Feature cueing of RF stimulus	Cueing by color	Condition II.1	Condition II.2	Condition II.3	Condition II.4
	Cueing by motion	Condition II.1	Condition II.2	Condition II.3	Condition II.4

MT neurons responded vigorously to search items placed within the classical RF and moved in the preferred direction. Conversely, responses to RF search items moving in the anti-preferred direction consisted of weak transient excitation or sustained inhibition. To ascertain whether these sensory responses could be modulated by the status of the RF stimulus as a target, we compared responses to conditions I.1–4. Conditions I.1 and I.3 enabled us to evaluate the consequences of a target vs. a nontarget within the RF, with RF motion in the preferred direction. Similarly, conditions I.2 and I.4 enabled us to evaluate the consequences of a target vs. a nontarget within the RF, with RF motion in the antipreferred direction. Nontargets in this experiment shared neither color nor direction of motion with the cue (henceforth termed “antitargets”).

The responses of an MT neuron that was differentially affected by these conditions are illustrated in Fig. 2 A–D. Each image contains: (i) the neuronal spike raster plot for each correct behavioral trial, aligned to the onset of the behavioral response (i.e., the saccade) and (ii) the saccade-aligned spike density function. Independent of their behavioral significance, RF stimuli that moved in the preferred direction elicited larger responses than stimuli that moved in the antipreferred direction. When the RF stimulus was a search target that moved in the preferred direction, however, we observed a substantial increase in neuronal activity, relative to baseline. This modulation began 100–200 ms before the behavioral response. No such modulation was observed when the RF stimulus was a nontarget. Furthermore, responses to stimuli moving in the antipreferred direction were not modulated, regardless of whether the RF stimulus was a target or a nontarget.

Fig. 2.

Modulation of neuronal response during covert visual search for conjunctions of color and motion. (A–D) Responses of a typical MT neuron to a search-array stimulus in the RF under target-present condition (Experiment I condition I.1–4). Each image contains (i) spike density function (SDF, continuous line, 10-ms bins) for the indicated condition aligned at saccade onset, plotted along with corresponding baseline (broken line, data from target-absent trials), (ii) stimulus onset-aligned SDF (Inset), and (iii) response rasters aligned to saccade onset (Each dot in raster plot is an action potential. Short vertical lines represent onset times for search display. Trials sorted by saccade latency.) The broken vertical line indicates the shortest latency time so that the saccade-aligned SDF right to the line (left to the line for the stimulus-aligned SDF in the Inset) represents the mean activation vertically. The number of trials contributing to the left side of the saccade-aligned SDF depends on time and is included as the validation for the saccade-aligned baseline calculation method used herein (see Materials and Methods). (A) When the RF stimulus was a target moving in the preferred direction (condition I.1), the neuronal response nearly doubled relative to baseline beginning ≈100 ms before the saccade. No such modulation was seen for condition I.2, when RF stimulus was a nontarget moving in preferred direction (B) or conditions I.3 and I.4, when RF stimulus moved in antipreferred direction (C and D). (E and F) Population-averaged spike-density functions (solid lines) and standard errors (broken lines) to preferred and anti-preferred directions when each was a target (conditions I.1–2) (E) and nontarget (conditions I.3–4) (F). The difference between responses elicited by preferred and antipreferred directions was significantly greater when RF stimulus was a target. Cumulative saccade latency histograms (below-average SDFs) reveal that the distributions were nearly identical for preferred and antipreferred directions of motion in the RF.

A total of 18 neurons were fully tested by using conditions I.1–4. Data from this subpopulation are summarized in Fig. 2 E and F in the form of averaged saccade-aligned spike density functions, separately for each condition. Although the sensory stimuli used were identical for target-in-RF and nontarget conditions, the response during the 100-ms time window preceding the saccade was increased by 24% relative to the baseline for targets in the RF when the stimulus in the RF moved in the neuron's preferred direction (median modulation index 16%, two-sided Mann–Whitney U test: P = 0.02). When the target was outside the RF, but a distractor in RF moved in neuron's preferred direction the end-stage response did not differ from the baseline (median MI = 1.5%, U test: P = 0.67). We did not observe significant modulation for conditions when the antipreferred stimulus was placed in the RF (target in RF: median MI = 7%, U test P = 0.13; target outside RF: median MI = −3%, U test P = 0.73).

We analyzed a larger dataset (n = 71) from neurons that were tested with the preferred direction only (target-in-RF conditions I.1 and II.1; antitarget-in-RF conditions I.3 and II.3; see Table 1 and Materials and Methods for condition definitions and SI Text for details on pooling data across these conditions). For each neuron, we computed response modulation relative to baseline during the last 100 ms before saccade onset (Fig. 3A). Modulation indices for color-cueing and motion-cueing conditions were pooled together because we did not find statistically significant difference in their distributions, and modulation indices for the two conditions were not correlated (P > 0.05). When the RF stimulus was a target, both the mean (39%) and median (17%) MI differed significantly from 0 (Mann–Whitney U test, P < 10⁻¹⁰). By contrast, when the RF stimulus was an antitarget, neither the mean (7%) nor median (0.3%) values differed from 0 (Mann–Whitney U test: P = 0.73). Furthermore, the distribution of MIs for the target-in-RF condition differed significantly from that of the antitarget-in-RF condition (Mann–Whitney U test, P < 10⁻⁵).

Fig. 3.

Attentional modulation indices for target-in-RF, antitarget-in-RF, and error-to-RF cases. (A) Distribution of modulation indices for sampled neurons. Modulation index was calculated as (response/baseline-1) × 100%, and based upon activity measured during the last 100 ms before saccade onset. (Upper) Modulation indices for target-in-RF condition pooled across Experiments I and II (conditions I.1 and II.1). The distribution is positively skewed (U test, P < 10⁻¹⁰; n = 110 tests), indicating that allocation of attention consistently elevates response rates for a subset of MT neurons. The arrow indicates target-in-RF modulation strength for neuron illustrated in Fig. 2 A–D. (Lower) Modulation indices for target outside RF conditions (I.3 and II.3). The distribution is centered on zero (U test, P = 0.73; n = 93 tests), and is significantly different from the target-in-RF distribution (U test, P < 10⁻⁵). The arrow indicates target-out-RF modulation strength for neuron illustrated in Fig. 2 A–D. All data included in these analyses were acquired during trials that concluded with a correct response; the stimulus in the RF of a neuron under study always moved in preferred direction. (B) The comparison of modulation indices obtained from correct target-in-RF trials and incorrect saccades to the RF in target-outside-RF trials (conditions I.3 and II.2–3). Only the cells that exhibited significant modulations during target-in-RF trials and for which sufficient number of error trials were generated were used (n = 17). Triangles denote cells that were significantly modulated during error trials. Diamonds mark cells that failed to reach significant modulations during error trials.

We also addressed the significance of end-stage modulations for individual neurons. A total of 24 of 110 tests (10 for motion cueing and 14 for color cueing cases) found significant modulations (t test, P < 0.05). Collectively, these data indicate that the neuronal modulation exhibits a spatially focal character and is limited to cells responding to the target (but see Experiment II, below).

Next, we analyzed error trial responses of neurons that exhibited significant end-stage modulation during correct trials. We considered only error trials resulting in saccades to nontarget (antitarget or semitarget) RF stimuli that moved in the neuronal preferred direction (conditions I.3, II.2, II.3). A sufficient number (>4) of such error trials was acquired for 17 cells. Fig. 3B shows modulation indices for these cells for correct vs. error trials. The end-stage modulation during error trials is reduced by a factor of ≈2 on average compared with modulations during correct trials (paired t test: P < 0.005). This reduction might be due to the fact that during error trials, the RF stimulus possesses at least one nontarget feature (see Transient Feature-Specific Effects below). Alternatively, this reduction might reflect suboptimal attention allocation associated with a blind-guess trial. For additional information see Fig. S2 and On end-point modulation onset times in the SI Text.

Experiment II: Transient Feature-Specific Effects During Conjunction Search.

Guided search is characterized by search patterns that favor display items possessing a target feature (see ref. 5 and refs. 6 and 21), and may be implemented by heightening sensitivity of a population of neuronal detectors tuned to the guiding feature (e.g., ref. 18) (see Fig. S3 for additional information). The predicted effect is that neurons will exhibit response modulations to distractors possessing the feature. To test this prediction, we compared the evolution of modulatory effects on neuronal responses obtained under the conditions identified in Table 1. Two sets of conditions were used. One set enabled us to evaluate the effect of motion direction. The other set enabled us to evaluate the effect of color. Below, we focus on population responses and their modulation by target features.

Fig. 4A illustrates the average behavior of a sample of MT neurons (n = 42) tested with our three motion-cueing conditions (II.1–3: target-, semitarget-, antitarget-in-RF). Each plot shows the population-averaged differential population histogram (saccade-aligned histograms minus the matched baselines) across all neurons as a function of time before saccade onset. Fig. 4 Insets represent population responses aligned to stimulus onset. The graph under the x axis plots the cumulative distribution of saccade latencies across all trials of all cells. The broken vertical bar indicates the time point when all trials of all neurons start contributing to the population histogram (right to the line for the saccade-aligned histograms and left to the line for the insets). The broken lines around the differential histograms represent the standard error of the mean. The significance of the modulation was tested by using the two-tailed Wilcoxon–Mann–Whitney rank-sum test on bins of size 100 ms. For the target-in-RF condition, the average response modulation reached significance (P < 0.05) ≈200 ms before saccade onset and remained so until the saccade was made. Peak average modulation was ≈20%. For the semitarget-in-RF condition, modulation above baseline was less pronounced but met our criterion for significance within a window extending from 140 to 90 ms before saccade onset. The median modulation in this window (estimated using 100-ms bin) was 6% (mean = 9%). Moreover, in contrast to modulation seen on target-in-RF trials, motion-specific modulation elicited by semitargets decayed to zero before saccade onset. No significant modulation was seen for the antitarget-in-RF condition.

Fig. 4.

Time course of attentional modulations as a function of search condition for Experiment II. (A) Results for motion-cueing conditions. (B) Results for the color-cueing conditions. Continuous line in the main panel represents the population-averaged differential histogram (saccade-aligned histogram minus the matched baseline) calculated for 100-ms sliding window with 1-ms step. Broken error-lines represent standard error of the mean. The monotonic nondecreasing graph below the zero line is the cumulative histogram of saccade latencies, normalized to range from 0 to 1. The vertical dotted lines denote the time point after which all trials contribute to the histogram. The Insets display stimulus onset-aligned histograms (continuous line) with population-averaged matched baselines (broken lines). The vertical lines denote the time point before which all trials contribute to the averaged histogram. Target-in-RF conditions (top row) exhibited characteristic presaccadic modulation of population response (sliding-window Wilcoxon–Mann–Whitney rank-sum test, P < 0.05). A smaller and transient, but significant, modulation of population response was present for semitarget-in-RF condition (middle row). No modulation was seen for antitarget-in-RF condition (bottom row).

Fig. 4B illustrates responses to color-cueing condition sets. At the population level, we observed a presaccadic modulation of ≈20% for the target-in-RF condition, which reached significance ≈150 ms before saccade onset. For the semitarget-in-RF condition, we detected significant modulation within a window extending from 240 to 160 ms before saccade onset. Interestingly, this semitarget modulation was stronger (median modulation = 15%, mean = 41%) and extended for a longer period than that observed for the motion-cueing case. This is consistent with our behavioral results, which indicate that color semitargets were more salient than motion semitargets (14). No significant modulation was seen for the antitarget-in-RF condition. As for motion-specific modulations, color-specific modulation decayed to a non-significant level immediately before saccade onset.

The transient nature of motion and color-selective modulations suggests that, as visual search progresses, the spatial focus of the feature-specific parallel search mechanism narrows from a globally distributed pattern to conclude with a pattern constrained to the target location.

Discussion

The experiments reported herein were designed to identify neuronal events associated with visual search for conjunctions of features. We identified two types of neuronal response modulation, which had different dynamics and were associated with different components of the search process. In the remainder of this Discussion, we address the relationship between our findings and those of other studies that have explored the neuronal bases of visual search and focal attention. We also consider the significance of our findings for theories of attention dynamics during visual search and neuronal mechanisms of feature binding.

Relationship to Other Studies of Attentional Modulation.

Attentional selection.

At the behavioral level, attentional selection refers to the phenomenon in which a target is selected from among other stimuli that are competing for attention. The phenomenon is well-characterized by the process of visual search. Chelazzi et al. (9) recorded from inferior temporal cortex in monkeys that had been trained on a visual search task. On each trial, a target and one distractor were positioned in the RF of the recorded neuron, and target identification was indicated by a saccade to the target location. Neuronal response rate remained high when the behavioral target was a “good” stimulus (i.e., a stimulus eliciting a strong response when presented alone), but declined ≈120 ms before the saccade when the target was a “poor” stimulus. Similar effects were seen in area V4 (22).

Although there are intriguing similarities between the Chelazzi et al. (9) results and our own findings, the paradigms differ in several key respects. For example, the behavioral tasks differ in terms of (i) the number of features that conjointly defined the target (two in our study vs. many in Chelazzi et al.), and (ii) the number of distractors in the search display (seven in our study vs. one and four in Chelazzi et al.). Most importantly, perhaps, were differences in the positioning of stimuli relative to the RF of each recorded neuron. Chelazzi et al. placed a target and a distractor in the RF, whereas we placed only one stimulus in the RF. The presaccadic response differential of Chelazzi et al. may reflect a process through which multiple RF stimuli “compete” for representation by the cell, and competition is resolved in favor of any one of them (23, 24). By contrast, the analogous response differential in our experiments is between targets and nontargets in the RF. The observed response enhancement to RF targets may be a product of an interneuronal competition that serves attentional selection, but the overt manifestation of this selection is simply facilitation.

Feature-specific attention and guided search.

Thus far we have discussed our findings in the context of spatial allocation of attention. Our results also complement electrophysiological studies of feature-specific attention, which was first reported in area V4 by Motter (25). Similarly, Treue and Martinez-Trujillo (18) reported that MT neurons exhibited direction-specific attentional response modulation. Functional imaging studies have revealed similar effects in human visual areas (7). Our finding of a direction-specific modulation in area MT is consistent with these feature-specific effects. Moreover, our results are also consistent with the additive interaction of spatial and featural attention as reported in (18): the potent attentional modulation during the end-stage of search may be a manifestation of additive interaction between spatial and feature-specific attentional effects. It remains to be seen, however, whether the feature-specific modulatory mechanisms continue to be active outside the focus of spatial attention during the final search stage, since feature-specific modulation of responses to semitargets subsides during that stage.

The guided-search model was inspired by unexpectedly rapid search times and feature-specific search errors, which indicated that search was being carried out preferentially amongst a subset of display elements possessing a feature in common with the target (e.g., 4, 26). Behavioral performance data from monkeys engaged in our color-motion conjunction search task possess these hallmark characteristics of guided visual search (14). The study of neuronal correlates of visual search in area V4 by Bichot et al. (13) addressed the guided search model by employing a paradigm that resembles ours. These investigators trained monkeys to perform visual search for both feature (color or shape), and conjunction of color and shape. The key difference, however, was that their search task was overt, so that subjects were allowed to explore the search array by making multiple saccades and signal target detection by an extended fixation on the target.

Cue-Invariant (Cross-Dimensional) Modulation.

Although MT neurons are not color selective (15–17, 27), we have observed marked attentional modulation that is color specific, which is consistent with an earlier fMRI study (7). A possible explanation for these effects may be found in the hypothesized operations of a stimulus-independent “saliency map,” which summates activities from separate feature maps (21) and mediates feature binding across the maps. According to this view, the retinotopic location in the saliency map that is maximally active represents the most salient collection of features, which may be selected for further processing. Neurons of area LIP possess some properties of a saliency map (28), as do those of the frontal eye fields (29). If activity at the selected location in the saliency map were fed back to feature maps as a retinotopically-specific signal, the resulting sensory response modulation would possess the property of cross-dimensionality. The color-specific modulatory effect that we have seen in MT is consistent with this type of mechanism.

Materials and Methods

Subjects and Surgical Preparation.

Subjects were two female rhesus monkeys (M. mulatta) weighting 6.5 and 8.0 kg. Subjects had normal color vision and no refractive error. Protocols were approved by the Salk Institute Animal Care and Use Committee, and they conform to U.S. Department of Agriculture regulations and National Institutes of Health guidelines for humane care and use of laboratory animals. Monkeys were surgically prepared by using conventional techniques (see ref. 27). Briefly, a stainless-steel post for head restraint and recording chamber were affixed to the skull with dental acrylic and anchoring screws. A search coil for measuring eye position was implanted in one eye (30). All surgical procedures were performed under aseptic conditions by using isoflurane anesthesia.

Electrophysiological Procedures.

Insulated tungsten microelectrodes (0.5–1.5 MΩ impedance) were positioned vertically and lowered through a craniotomy centered over the parietal cortex. Extracellular action potentials were amplified, filtered, and recorded by standard means. Area MT was identified on the basis of (i) the pattern of cell groups encountered as the electrode was lowered (with reference to previously obtained MR images) and (ii) the RF size/eccentricity relationship, visual topography, and stimulus selectivity. Once each neuron was isolated, RF location and extent were mapped and speed and direction tuning assessed while the animal was fixating. After general response properties were fully characterized, the visual search paradigm was begun. Animals were required to complete at least eight correct trails per condition for the data to be included for further data analysis. This resulted in missing data for some conditions.

Visual Stimuli.

Each trial consisted of a briefly presented “sample stimulus,” followed by a “search array.” The behavioral paradigm (described below) required the subject to identify the search-array stimulus that matched the sample stimulus, i.e., the “target.” The sample was presented at the center of gaze and consisted of a stationary circular aperture (1° diameter) through which a moving textured pattern was viewed (Fig. 1B). The luminance of each texture element was 10 cd/ft² and background luminance was 1 cd/ft², yielding a local contrast (Michelson) of 82%. The sample stimulus (“cue”) could vary along two dimensions: color and direction of motion. Color was either red or green, and direction was either in the preferred or antipreferred direction for the neuron under study (the sample stimulus was not presented within the RF). Thus the color–motion conjunction stimuli were of four distinct types (red/preferred, red/antipreferred, green/preferred, green/antipreferred), which were presented as samples on a pseudorandom schedule. Red and green stimuli were photometrically isoluminant. Speed of motion was 3, 4, 6, or 9°/sec and selected to match neuronal speed preference. The search array consisted of eight circular stimuli each constructed in a manner similar to the sample. These stimuli appeared in a radially symmetric pattern around the sample location (Fig. 1D). The radius of the pattern and the size of the stimuli were varied with RF eccentricity, such that one stimulus was always centered on the RF of the cell under study and scaled to match RF diameter.

In principle, each stimulus in the search array could possess any of the four possible conjunctions of color and direction. For our purposes, the search stimulus of greatest relevance was the one within the RF. The key independent variables in our experiments were the relationships between the attributes of this stimulus and (i) the selectivity of the RF and (ii) the properties of the sample stimulus. The first independent variable reflected the “sensory significance” of the RF stimulus to the sampled neuron. The neurons recorded were selective for direction but not color (15, 17). The first independent variable thus took one of two directional values: (i) RF stimulus moved in preferred direction, (ii) RF stimulus moved in antipreferred direction.

The second independent variable reflected the behavioral significance of the RF stimulus and took one of three values: (i) RF stimulus was the same direction and color as the sample, i.e., it was the target of visual search. All other stimuli were distractors. (ii) RF stimulus was the same direction or color as the sample, but not both. We termed these RF stimuli semitargets. Semitargets functioned as distractors; the target was at a non-RF location. (iii) RF stimulus was neither the same direction nor color as the sample. We termed these RF stimuli antitargets. Antitargets functioned as distractors; the target was at a non-RF location.

Experimental Design.

Both experiments used stimuli as in Fig. 1 (for stimulus details see SI Text). In experiment I, we sought to determine whether neuronal activity depended on whether the RF stimulus was a target. To do so, we considered the effects of six different conditions, identified in Table 1. Conditions I.1–4 constitute the conjunction of direction of motion (preferred vs. antipreferred in RF) with target location (target-in-RF vs. nontarget-in-RF), which enabled a comparison of directional selectivity as a function of whether the RF stimulus was a target. (For experiment I, antitargets were placed in the RF on all nontarget-in-RF trials.) Two additional conditions (I.5–6) were of the “target-absent” type, meaning that a stimulus matching the sample was not present in the search array. On such trials monkeys were required to maintain central fixation for 1 sec to receive reward. Neuronal activity on target-absent trials was used to estimate sensitivity to RF stimulation in the absence of a target-directed response, which provided a baseline (see below) for evaluating the effects of the other experimental conditions. Additional details are in SI Text.

In experiment II, we sought to determine whether neuronal activity elicited by nontarget RF stimuli depended on whether the stimulus possessed a feature in common with the target. To do so, we considered the effects of two analogous sets of four conditions each (Table). Both sets allowed comparison of the effects of (i) target, (ii) semitarget, and (iii) antitarget stimuli in the RF. The two sets were configured such that one allowed assessment of RF sensitivity to the cued motion and the other allowed assessment of RF sensitivity to the cued color (see SI Text and Fig. S3). As for experiment I, target-absent conditions were included in each condition set (condition II.4), and sample stimuli for these conditions were noninformative (both red and green CRT guns were on and random dots moved in both preferred and antipreferred directions). For all conditions in both sets, RF stimuli moved in the preferred direction.

Within each set of conditions, search stimuli were identical within the RF and in neighboring locations (per proximity rules described in SI Text). Nonneighboring search stimuli were assigned pseudorandomly across trials. All conditions within each set were randomly interleaved within a recording session.

Data Analysis.

Time-dependent measures of neuronal activity (spike density functions) recorded on target-present trials were obtained by convolving spike trains from individual trials with a Gaussian kernel (20-ms width at half-height). Our goal was to compare attentional visual search-related modulations during different conditions rather than neuronal responses per se. To that end, we designed a matched-baseline method that enabled us to calculate neuronal modulations in a manner that discounted the variability of the saccadic latency distributions across neurons and experimental condition. As a reference for each target-present condition, the matched baseline neuronal activity measure was derived from data obtained on target-absent trials (i.e., conditions I.5, I.6, and II.4). This measure reflects neuronal activity elicited by the search array in the absence of a saccade. In practice, the matched baseline was computed by averaging spikes on target-absent trials by using the same distribution of trial durations as that produced by saccade latencies during target-present trials (see SI Text). The modulation indices were calculated as the percent deviation from the matched baseline of the presaccadic response during the last 100 ms before saccade (see SI Text for details).