All-or-none context dependence delineates limits of FEF visual target selection

SUMMARY

Choices of where to look are informed by perceptual judgments, which locate objects of current value or interest within the visual scene. This perceptual-motor transform is partly implemented in the frontal eye field (FEF), where visually-responsive neurons appear to select behaviorally relevant visual targets and, subsequently, saccade-related neurons select the movements required to look at them. Here we use urgent decision-making tasks to show (1) that FEF motor activity can direct accurate, visually-informed choices in the complete absence of prior target-distracter discrimination by FEF visual responses, and (2) that such discrimination by FEF visual cells shows an all-or-none reliance on the presence of stimulus attributes strongly associated with saliency-driven attentional allocation. The present findings suggest that FEF visual target selection is specific to visual judgments made on the basis of saliency and may not play a significant role in guiding saccadic choices informed solely by feature content.

ETOC Blurb

Visual neurons in the primate frontal eye field help guide eye movements. Now Scerra et al. demonstrate that their contribution is limited: these cells distinguish targets from distracters prior to some but not all visually informed saccadic choices; they can select goals on the basis of saliency but not on the basis of arbitrary visual features.

INTRODUCTION

Goal-directed behavior is a consequence of dynamic interactions between sensation, cognition, and motor control. Accordingly, attempts to understand its neural basis have focused principally on how motor alternatives are prioritized based on sensory evidence and internally-specified goals. This has been extensively studied within the frontal eye field (FEF), a prefrontal region that has been implicated in visual target selection, spatial and feature-based attention, and saccade execution [1–9]. Using search tasks that require discriminating a target from distracters on the basis of some visual feature dimension (e.g., color), such studies have consistently demonstrated the primacy of visually-responsive cells in first responding to the stimulus in space, and next, in signaling, through a modulation in firing rate, its identity as either target or distracter [9–12]. This visual target selection is followed by selection of the motor response, i.e., by activation of the FEF motor (or movement) cells encoding the saccade vector needed to acquire the target [9–13].

In this paradigmatic and empirically supported view, visually-responsive cells select a target to inform the correct motor choice — a serial order of events that serves as the basis for models detailing the perceptual-to-motor transformation that leads to successful goal acquisition in such tasks [14–19]. The idea that visually-contingent activity in FEF is integral to the process of informing saccadic choices on the basis of distinguishing visual features is intuitive. However, in a recent study [20], we documented an exception to such a requirement for performance of a simple visual feature discrimination (red versus green) task. A unique element of this task was urgency: the compelled-saccade (CS) task [21] requires modification of ongoing motor plans by newly arriving information about target identity, which precludes a strict serial order to the unfolding of perceptual and motor events, and results in saccadic choices that are informed to varying degrees depending on their relative timing. We found that the extent to which urgent saccadic choices were perceptually informed, while strongly evident in both behavior and modulations of FEF motor activity, was not manifest in the FEF visual activity. In the CS task, purely visual neurons showed no evidence of target/distracter discrimination in advance of a correct choice, suggesting that such target selection is not a general prerequisite to saccadic choices preceded by a visually-based decision.

The present study seeks to further delineate the factors that govern the apparently flexible relationship between FEF target selection and visually informed motor choice and, in doing so, to more precisely characterize the nature of the signal conveyed by FEF visual “target selection” neurons. Resolving this is essential not only to circumscribing the functional roles of FEF neuron types, but also to understanding how visual perception and action selection interact more generally. In this regard, there are at least two potentially critical distinctions between the CS task, for which visual cells failed to demonstrate target selection, and other search tasks in which they have consistently done so. First, decision/choice urgency may expose a heretofore unappreciated temporal limitation on the activity of FEF visual cells, with their participation in the target selection process contingent on the dilated timescale of paradigms that serialize target selection and motor planning [20]. Alternatively, it is possible that robust target selection by this class of FEF neuron is observed only for visual targets that can be identified on the basis of saliency, as is true for many search tasks (e.g., [10, 22]), and not when the relevant visual feature value is defined by a pre-specified rule (e.g., a color template match), as in the CS task.

We recorded from visually-responsive FEF cells as monkeys performed tasks that varied in relative urgency and in how a red or green target was defined relative to complementary colored distracting stimuli. The same individual neurons were studied in a standard oddball search task, an oddball task that incorporates urgency, and the two-alternative CS task. The new dataset replicated our previous finding that FEF visual cells do not select the targets of accurate, perceptually-informed saccadic choices in the CS task. But notably, the same visual cells, though sensitive to task urgency requirements, demonstrated robust target selection in both standard and urgent variants of the oddball search task. The task-specific dichotomy suggests that FEF visual cells distinguish visual feature values on the basis of their saliency, a defining characteristic of targets in oddball tasks, and not on the basis of behavioral relevance more generally.

RESULTS

Behavioral tasks and psychophysical performance for urgent choices

Monkeys performed four tasks (Figure 1) during recording of each of the 166 single neurons in this data set (see Methods for detailed description). A standard delayed-saccade task (Figure 1A) required a visually-guided saccade to a single peripheral stimulus. Delayed-saccade trials were used for online estimation of a cell’s response field (RF) and for off-line qualitative assessment of a cell’s classification as visual (V), visuomotor (VM), or motor (M). The easy-oddball (EO) task (Figure 1B) required a saccade to an oddball stimulus, either a red target among three green distracters or the converse. The “easy” designation denotes that the Go signal (offset of the central fixation point) was delivered after the Cue (i.e., the color change revealing target and distracter identities/locations), so that the perceptual judgment could be completed before commitment to a saccadic choice. In contrast, the CS and compelled-oddball (CO) tasks (Figure 1C, D) incorporated urgency by delivering the Go signal before the informative Cue. Thus, by design (see Methods; [21]), the urgent tasks compelled subjects to plan a motor response in advance of viewing the Cue, and to use the color information, when available, to guide any ongoing motor plan.

Figure 1.

Behavioral tasks used while recording from single cells. A, The delayed-saccade task required a saccade to single peripheral target. B, The EO task required a saccade to the color oddball that was revealed before the Go signal. C, the CS task required a saccade to the stimulus that matched the color of the central fixation spot. D, The CO task required a saccade to the oddball stimulus. The CS and CO tasks incorporated urgency by revealing target/distracter identities (Cue) a variable amount of time (Gap) after the Go signal. A saccade was required to occur within 400 ms of the Go signal. RT was measured as the time from the Go signal to saccade onset. rPT is the maximum amount of time during which the Cue could inform the target selection (rPT = RT – Gap). Shaded area around one stimulus location denotes RF location; it is not displayed during trials. See text for additional details.

The key distinction between the CS and CO tasks is that the CS task defines “target” as the stimulus that matches the color of the fixation point (i.e., match to prior template) whereas the CO task defines “target” as the stimulus that is distinct from the others in the array (i.e., the oddball or feature singleton). The CS task and corresponding behavioral performance have been described previously [20, 21], and the same logic applies to the newly introduced CO task. Accordingly, the CO task incorporates the temporal constraints of the CS task in order to create an urgent variant of a typical oddball search task. For either task, performance on a given trial can be understood as follows. Triggered by the Go signal, the initial saccade plan is based on a guess as to where the target will appear (two possible locations in the CS task, four in the CO). Then, after a variable Gap period (0–275 ms), the Cue is presented, allowing the subject to modify the ongoing motor plan (if necessary). By design, on average, the longer the Gap, the shorter the amount of time available to see the Cue and process the color information; however, for any given trial, the actual cue viewing time — the raw processing time (rPT) — is determined by both the subject’s reaction time (RT) and the Gap on that trial (rPT = RT – Gap). Thus, the probability of success on any given trial varies most closely with rPT, and the function that relates these two quantities, which we refer to as the tachometric curve, is the key behavioral metric that describes perceptual capacity in the compelled tasks (Figure 2A, B). According to it, performance should increase from chance (50% in CS task; 25% in CO task) to an asymptotic level as a function of rPT. Based on the amount of rPT necessary to reach the 75% correct benchmark, comparison of the tachometric curves for the two tasks suggests that the perceptual information was able to guide the monkey’s choices earlier in the CO task (Monkey R: 114 ms, 95% CI [111, 116] from bootstrap; Monkey I: 113 ms, 95% CI [110, 115]) than in the CS task (Monkey R: 147 ms, 95% CI [144, 150]; Monkey I: 124 ms, 95% CI [123, 128]).

Figure 2.

Behavioral data from two monkeys in the compelled tasks. A, C, E, CS task performance from Monkey R (magenta, 9814 trials) and Monkey I (purple, 10079 trials). B, D, F, CO task performance from Monkey R (magenta, 8671 trials) and Monkey I (purple, 8944 trials). A, B, Percentage of correct responses as a function of rPT (tachometric curves). Shading denotes standard error, dotted lines indicate chance performance, and arrows indicate time points for separation of long- and short-rPT trials. Note differences in y-axes reflecting different chance levels in 2 (CS) and 4 (CO) stimulus tasks. C, D, Percent correct as a function of Gap duration. E, F, RT as a function of Gap duration. Error bars denote standard deviations. Data points include correct and incorrect responses.

As noted, the CS task required target identification based on a match-to-template rule whereas the CO task required selection of an oddball stimulus, and differences in performance may reflect these distinct perceptual demands. In addition, some idiosyncratic differences between the two monkeys were evident. One monkey tended to respond more quickly than the other (Figure 2E, F) and thus, on average, had a lower success rate (Figure 2C, D). In contrast to performance on the urgent tasks, that on the EO task (not shown) was consistently high for both monkeys (on average, 98.8% correct for Monkey R, 3870 trials; 98.3% correct for Monkey I, 4952 trials) across Cue durations (250–1000 ms) and recording sessions (114 sessions for Monkey R, 108 sessions for Monkey I), indicating that, given enough time to view the stimulus array, both monkeys could perform the standard search task almost perfectly.

V cell target selection depends on context

In serial processing models, V cells are credited with selecting the target in a visual search array before M cells produce the associated motor plan [10, 11, 15–17]. As reported by Costello et al. [20], in the CS task, where perceptual and motor processes unfold in parallel, V cells failed to select the target before saccade execution, even at long processing times, when performance was asymptotic and choices fully informed. Importantly, the main implication of this finding — that target selection by V cells is not an obligatory step toward the making of visually-informed saccadic choices in certain contexts — is predicated on the assumption that such non-discriminating V cells do not comprise a distinct class from those that typically show robust selection in visual search paradigms (see Discussion; [9–12]). The present findings, which are based on recordings of individual V cells during performance of both easy and urgent variants of the oddball search task, in addition to the CS task, confirm and extend the interpretation that V cell target selection is, indeed, context-dependent.

The dependence of V cell target selection on task type is shown for two example V cells (Figure 3) and for the population of cells so classified (Figure 4, n = 40). Profiles of activity associated with performance of the single-target delayed-saccade task (Figures 3A, E; 4A) typify the V classification in showing a clear transient increase in activity when aligned on target onset (Target on; Figures 3A, E, left; 4A, left), and a gradual peri- to post-saccadic decline in sustained activity when aligned on the saccade (Figures 3A, E, right; 4A, right). Comparison of target/distracter differentiation across the three choice tasks (Figures 3B–D, F–H; 4B–D) suggests that the way in which the target was defined by the task determined whether or not V cells selected it prior to the saccadic choice. In the EO task (Figures 3B, F; 4B) Cue onset was associated with a well-defined increase in target-related activation accompanied by a decrease in distracter-related activation beginning approximately 100 ms post-Cue. Such differentiation was qualitatively similar in both its timing and magnitude to that reported in prior studies of FEF target selection [9–13] and serves as an important benchmark for considering the differentiation that may precede informed choices in the urgent tasks. Aligned on Cue, qualitatively similar differentiation to that observed in the EO task was evident for the CO task (Figures 3D, H, left; 4D, left) but not the CS task (Figures 3C, G, left; 4C, left). Aligned on saccade, the most relevant comparison for the two tasks is that for informed trials in which performance was guided by the Cue (uninformed and informed trial categories were defined based on task-corresponding tachometric curves, as described in Methods). For the CS task, there was no indication that activity discriminated between a target and a distracter at any time prior to the saccadic choice (Figures 3C, G, right; 4C, right). In contrast, for the CO task, V cells showed clear differentiation, with target and distracter-related activity diverging early enough to yield presaccadic activity that was strongly differential (Figures 3D, H, right; 4D, right). For uninformed trials, the lack of presaccadic differentiation was clear for the individual cells in the CS task (Figure 3C, D, middle), and although a paucity of short rPT trials in the CO task precluded such demonstration for these particular example cells, target-distracter ambivalence for short rPTs is evident for both tasks across the population of V cells (Figure 4C, D, middle).

Figure 3.

Activity of two example V cells recorded in all four tasks. Traces show firing rate (in spikes/s) as a function of time, with spikes aligned on task events, as indicated by the corresponding icons. Red and green traces are for correct saccades to targets inside and outside the cell’s RF, respectively. Light colors indicate ± 1 SE across trials. Numbers of trials in each condition are indicated. A, E, Activity in the delayed saccade task aligned on target onset (left) and saccade onset (right). B, F, Activity in the EO task aligned on cue onset (left) and saccade onset (right). C, G, Activity in the CS task aligned on cue onset (left) and saccade for guesses (rPT < 110 ms; center), and informed choices (rPT > 155 ms; right). D, H, Activity in the CO task aligned on cue onset (left) and saccade for guesses (rPT < 95 ms; center), and informed choices (rPT > 130 ms; right). Note clear differentiation between inward and outward responses before saccade onset. Gray, shaded regions in C, D, G, and H indicate the range of cue-onset times for each condition.

Figure 4.

Population activity from 40 V neurons recorded in all four tasks. Same format as in Figure 3, except that light red and green shades indicate ± 1 SE across cells. Y-axes correspond to mean, normalized firing rate, with equal scale for all plots. A, Delayed-saccade task. B, EO task. For compelled tasks (C-D) data are aligned on cue onset (left) and saccade onset for short (center) and long (right) processing times. Note lack of target/distracter discrimination in the CO task at short rPTs (D, center), but clear discrimination at long rPTs (D, right), when perceptual processing informs the choice. Gray inset histograms (C, D; right) illustrate distributions of saccade onset times corresponding to rPT < 110 and rPT > 155 ms (C; CS task) and rPT < 95 ms and rPT > 130 ms (D; CO task).

Interaction of urgency and saliency

Costello et al. [20] showed that V cells in the FEF did not discriminate targets from distracters in an urgent task. The current findings suggest that V cells can select the target of an urgent choice if it is more salient than the alternative stimuli. Importantly though, a comparison of the activity of the same cells for the CO and EO tasks suggests that the degree to which such selection occurs is yet constrained by the temporal demands of the task.

To demonstrate this for the population of V cells, normalized activity for correct target out of RF trials was subtracted from that for correct target in RF trials to yield time-dependent target/distracter difference profiles, which were aligned on Cue (Figure 5A) and saccade (Figure 5B) for the EO (green), CO (teal), and CS (purple) tasks. Consistent with data shown in previous figures (Figures 3C, G; 4C), there was no target-distracter discrimination for the CS task following the Cue, nor at the time of the saccade. In contrast, target selection in both the EO and CO tasks emerged approximately 100 ms after Cue presentation and briefly followed a similar upward trajectory, but thereafter the trace for the CO task reversed course to produce a lower maximum value than that for the EO (0.30 versus 0.60; p = 0.04, permutation test). When the same data are aligned on saccade (Figure 5B) it becomes more obvious that the decline in the CO task profile was linked to saccade onset, suggesting that, in the urgent task, the evolution of target/distracter differentiation was curtailed by execution of the saccadic choice. The respective impact of urgency and putative saliency on target selection is summarized in the form of scatter plots and bar graphs (insets) comparing CO to EO differentiation (Figure 5C) and CO to CS differentiation (Figure 5D) for individual cells based on the Cue-aligned epoch indicated (Figure 5A, shade). A tendency for points to cluster above the line of unity (Figure 5C) indicates a moderate trend (p = 0.049, permutation test) for greater selectivity in the non-urgent EO task as compared to the urgent CO task. A stronger bias (p < 0.001, permutation test) for selectivity favoring the CO (oddball) over the CS (color match to fixation sample) task is indicated by a strong tendency for the corresponding points to fall below the line of unity (Figure 5D).

Figure 5.

V cell target discrimination as a function of time and task type. Continuous traces (A, B) show difference in normalized activity of V cells (n = 40), R_IN – R_OUT, as a function of time, where R_IN corresponds to target-in/saccade-in choices and R_OUT corresponds to target-out/saccade-out choices, all informed (CS task, rPT > 155 ms; CO task, rPT > 130 ms). Data are aligned on cue (A) and saccade (B) for the EO, CO, and CS tasks. Color-coordinated inset histograms (A, top) indicate time of saccade onset for compelled tasks. Scatterplots (C-D) show average firing rate for each cell (scatter dots) taken at the time window indicated by the gray shading (A) for EO and CO tasks (C), and CS and CO tasks (D). Inset bars and error lines indicate mean ± SE across all V cells.

V cell modulation consistent with saccadic choice in the compelled-oddball task

The dichotomy between CS and CO target selection and the subtler differences between CO and EO target selection indicate that the relationship between V cell activity and motor choice is highly context dependent, with saliency and urgency as factors jointly defining that context. As shown above, at long rPTs, correct trials for the CO task were associated with robust presaccadic, cue-driven differentiation (Figures 3D, H; 4D; 5B). Diverging target- and distracter-related activity is recapitulated in Figure 6A (left) and accompanied by a plot of continuous-time ROC score (right; see Methods) to quantify the strength of the selection signal as it evolves during the time leading up to the saccadic choice. The ROC score rises above 0.5 (activity favoring target) beginning approximately 100 ms before the saccade and approaches a plateau near the time of saccade onset. Although it resembles a presaccadic motor burst when aligned on saccade (see also Figures 3D, H; 4D, right; 5B, right), CO task differentiation by V cells was in fact a cue-driven response that appears motor-related only because of the compressed time scale of the urgent task. Aligned on Cue, this divergence begins approximately 100–120 after Cue presentation (Figure S1E) and is entirely absent for uninformed trials, in which saccades were initiated less than 95 ms after the Cue (Figure S1A).

Figure 6.

Contrasting activity generated by 40 V cells across multiple conditions of the CO task. All plots are based on long-rPT CO trials aligned on saccade onset. For this comparison of correct and error trials, all plots were based on an extended rPT range that included the transition from chance to asymptotic performance (rPT > 95 ms). Plots on the left show continuous firing rate traces for two conditions distinguished by color, as indicated by corresponding icons. Shaded interval indicates range of cue-onset times. Plots on the right show running ROC scores quantifying the difference between the curves on the left. Gray shade indicates ± 1 SE across neurons. A, Comparison between correct choices into (red) and out of (green) the RF. B, Comparison between errors (blue) and correct choices (green) out of the RF. C, Comparison between error (orange) and correct choices (green) when the target is outside the RF. D, Comparison between correct (red) and error choices (orange) into the RF. See also Figure S1 for same data aligned on Cue for both short and long rPT.

Importantly, during the transition from chance to asymptotic performance, performance at any given rPT is presumed to reflect a mixture of guesses and informed choices. Error trials for rPTs throughout and beyond this transition stage are interesting because they occur when, in principle, the theoretical minimum rPT for generating an informed choice was exceeded (rPT > 95 ms for CO task). Although such errors are uncommon (< 8% of all trials), they represent an opportunity to test the premise that V cell selection (when it does occur) is predictive of the impending saccadic choice. A test of this premise is based on analysis of two types of error trial considered from the perspective of the V cell receptive field: RF misses are errors in which the target was in the RF and the saccade was made to a distracter out, and RF false alarms are error trials in which the target was out of the RF and the saccade was made to the distracter in. We found that in both cases the V cell activity was consistent with the saccadic choice, not the location of the oddball stimulus. Accordingly, presaccadic activity for misses (Figure 6B, blue trace) was weak, much like that for correct trials in which a distracter was in the RF (Figure 6B, green trace). The corresponding ROC plot (Figure 6B, right) does show target/distracter differentiation, but it was predominantly postsaccadic, beginning approximately 75 ms later than that for trials in which an RF target was correctly identified (Figure 6A). Complementarily, false alarms were associated with distracter-related activity (Figure 6C, orange trace) that far exceeded that for trials in which the RF distracter was correctly rejected (Figure 6C, green trace), and was indeed similar in timing and magnitude to that observed when an oddball was in the RF and correctly identified (Figure 6A, D, red trace). As suggested by a prior study [23] and elaborated below (Discussion), such errors may reflect a mischaracterization of the stimulus by visually responsive cells.

V and M responses are differentially sensitive to endogenously- and exogenously-defined goals.

Having demonstrated that choice urgency was not the key limiting factor in determining whether V cells selected the targets of informed, urgent choices, we sought to examine how V, VM, and M cells compared in their sensitivity to target saliency as defined by the CS and COtasks. Previously, Costello et al. [20] demonstrated that, for the CS task, the magnitude of presaccadic target/distracter differentiation scaled with the relative dominance of visual to motor-related activation; that is, V cells failed to differentiate, M cells did so robustly, and VM neurons showed selectivity intermediate between these extremes. Using the same quantitative classification scheme (see Methods; [20]) to distinguish pure V (Figure 7A) and M (Figure 7C) groups from those that expressed both visual- and motor-contingent activation (VM; Figure 7B), we observed the same visual-to-motor hierarchy for the CS task in the current data set when target/distracter difference (target in RF – target out of RF) profiles were plotted separately for the V, VM, and M groups (Figure 7D, E, F; purple traces). In addition, the degree of context dependence, manifest as the change in the amount of differentiation for the CS versus CO tasks, showed a similar dependence on cell type. As shown earlier (Figure 5), the same V cells that failed to differentiate on the CS task did so reliably on the CO task, thus showing strong dependence on task context (Figure 7D). In contrast, the strong differentiation of M cells (Figure 7F) was virtually identical for targets defined via an oddball (CO) or a color match-to-template (CS). The VM population was once again intermediate between the V and M groups, differentiating for both CS and CO tasks, but much more so for the latter (Figure 7E). These trends in the average activity profiles are clearly evident in the accompanying scatterplots that relate the magnitude of CS and CO differentiation for the individual neurons comprising the V (Figure 7G, CO > CS, permutation test, p < .001), VM (Figure 7H, CO > CS, permutation test, p < .001), and M groups (Figure 7I; CO ≈ CS, permutation test, p = .620). The results suggest that the moderate differentiation of the VM cells observed in the CS task likely reflected the motor component of their response exclusively, whereas their larger differentiation in the CO task likely reflected both their visual and saccade-related contributions.

Figure 7.

Urgent target discrimination as a function of time, task, and cell type. Continuous normalized firing rate traces (A-C) show activity in the delayed-saccade task aligned on both Cue and Saccade for V (A), VM (B), and M (C) cells. Colors indicate activity elicited by the target inside (red) or outside (green) of the RF. Blue and purple traces (D-F) show difference in normalized activity (R_IN – R_OUT) for V cells (D), VM cells (E), and M cells (F) as a function of time, where R_IN corresponds to target-in/saccade-in choices and R_OUT corresponds to target-out/saccade-out choices, all informed (CS task, rPT > 155 ms; CO task, rPT > 130 ms). Data are aligned on saccade. Scatterplots (G-I) show average firing rate difference for each cell taken at the time window indicated by the gray shading in D-F, for the CS task (abscissa) and the CO task (ordinate). Green circles indicate the population mean for each task.

DISCUSSION

We investigated the interaction between urgency and saliency during target selection with a primary focus on FEF cells classified as exclusively visual. As has been reported more generally for visually-responsive (V and/or VM) FEF neurons [4, 9–12, 24], we found that individual V cells strongly signal the location of a red (green) color singleton in an array of green (red) distracters during performance of a standard oddball search task, and that these V cells also do so for accurate, perceptually-informed choices under the urgent search conditions imposed by the CO task. Critically however, the very same cells failed to do so for similarly accurate and informed red-versus-green choices in the two-alternative CS task. The demonstration of an all-or-none context dependence (CO vs CS task) for a presumptive signal of target selection has implications for understanding the nature of the information carried by FEF visual activity and the role of FEF in guiding saccadic choice based on visual decisions more generally. As discussed below, our findings suggest that target selection by FEF V cells, and likely by FEF visually-contingent activity more broadly (V and VM), is specific to the selection of stimuli on the basis of salience and does not inform behaviorally-relevant visual decisions made on the basis of feature value alone. The findings agree with prior studies linking FEF target selection to saliency-based attentional deployment [4, 16, 23, 25, 26] but are novel in defining limits to its role in guiding visually-informed saccadic choice.

The impetus for the current study arose from an earlier observation that traditionally classified FEF V cells do not distinguish target from distracter in the two-alternative CS task [20]. The lack of V cell target selectivity was potentially attributable to (1) the urgency of the decision, (2) the specific way in which target (vs distracter) is defined within the context of the CS task, or (3) selective sampling of a non-differentiating class of V cell. Although the possibility of sampling bias seemed remote, a class of FEF V cell with more limited selection-related capabilities has been described previously [27]. The present data definitively rules out this potentially trivial explanation by demonstrating a strict and absolute CO vs CS task dependence for target selection within individual V cells and for the aggregate population overall. We could further surmise from the results of Costello et al. [20] that choice/decision urgency is not the sole reason that V neurons fail to select targets in the CS task because, in that study, V cells also neglected to select targets in a non-urgent variant of the task employing the identical two-alternative color choice. However, that earlier investigation did not impose an urgency requirement on a task, like oddball search, that is definitively associated with V cell target selection; thus, the possibility that urgency constrains or otherwise dictates FEF visual target selection could not be ruled out.

The present results show that task urgency does in fact constrain the magnitude of presaccadic target selection by FEF V neurons; cue-driven differentiation of the target at saccade onset tends to be less robust in the urgent (CO) than in the standard (EO) oddball task (Figure 5). Importantly though, it seems that urgency simply curtails the time available for such differentiation to develop, as if its expression is merely truncated by saccade execution. This alone is interesting because it suggests that such differentiation develops in parallel with and not as a necessary antecedent to the execution of a visually-informed motor plan. However, a key point here is that the attenuating impact of urgency on FEF target selection is quantitative, not qualitative, and our findings instead point to differences in the way target and distracter are defined within the CS and oddball tasks as the primary reason for the observed task dependence.

Visual target selection or visual saliency detection?

The term “visual target selection” refers to the capacity of FEF neurons to discriminate between visual stimuli that differ along one or more feature dimensions (e.g., color, shape), most often in the context of a visual search task variant [9–11, 28–31]. As in prior studies, we quantified the capacity of a given neuron to “select” the target of an impending saccade by comparing activity for trials in which a target versus a distracter was in its RF. Our finding that FEF V cells appear to strongly signal target identity ahead of informed choices in the CO task but are ambivalent for equally informed choices in the CS task demonstrates that target selection by these neurons is not necessary to produce veridical motor output in a feature-based visual decision task. Some prior studies have interpreted differentiation by visually-responsive cells (including V cells) as the deployment of spatial attention [4, 23, 25, 27], noting that it occurs even if targeting saccades are prohibited [25]. Interpreted in this way, V cell activity, along with the visually-contingent activity of VM cells, might be viewed as a substrate for attentional deployment that is independent of saccadic intent. The present findings reveal an equally important, entirely complementary dissociation in which visually informed saccadic intent can occur in the absence of V cell signaling in favor of the saccadic goal. That the dissociation is evident for the CS, but not the oddball tasks, provides additional clarity on the nature of a putative attention-related signal carried by FEF V cell activity.

A primary distinction between the CS and CO tasks is the degree to which they are presumed to engage exogenous, or saliency-dependent, attention (defining the saliency of a stimulus requires some nuance that goes beyond consideration of physical attributes, but the distinction is clear across our experimental conditions; see [32, 33]). Regardless of urgency, oddball tasks strongly promote congruence between target location, spatial attention, and ultimately, motor planning. Feature singletons are inherently salient and capture attention automatically even when appearing at a location that is inconsistent with current task goals [34–36]. Moreover, the saliency of a feature singleton is positively modulated by perceptual priming effects that occur when the oddball stimulus is the same color on multiple trials in succession [37–39]. Such non-volitional priming, along with the learned association between singleton and rewarded saccadic choice, are factors that would tend to reinforce the inherent saliency of the target and presumably its neural correlate in V cell activity [26, 33, 40]. In contrast, the CS task minimizes the import of saliency-based mechanisms to guiding choice by limiting physical saliency as a means of preferentially drawing attention to the target on any given trial (for discussion, see [20]). Notably, the behavior of both monkeys in our urgent paradigms is consistent with the presumed saliency advantage that oddball targets have over their CS task counterparts. Specifically, tachometric curves for the CO task were shifted toward shorter processing times compared to those for the CS task (Figure 2A, B), indicating that oddball targets were more readily discriminated. Moreover, performance on the CS task varied considerably for the two monkeys, but was nearly indistinguishable for the CO task, consistent with the premise that bottom-up, saliency-driven behavior is more stereotyped.

The observed task-specific dichotomy suggests that whether V cell activation aids, hinders, or contributes not at all to selecting a target location based on its visual feature content is likely to depend on the degree to which a given task renders the feature value and its corresponding location uniquely salient. In standard oddball tasks, the congruence of high physical saliency and target location creates a circumstance in which V activity is most likely to bias saccadic choice in favor of the behavioral goal. In the CS task, for reasons outlined above and discussed previously [20], the lack of such a spatial correspondence probably accounts for V cell ambivalence to stimulus relevance. As we show here (Figure 7) and in prior studies [20, 21], in the CS case it is the FEF motor-related activity that evolves in a processing time-dependent way to signal the perceptually-informed choice, and apparently does so without the aid of a V cell-mediated spatial pointer.

Prior empirical findings already support strong ties between FEF target selection and saliency (for review, see [4]). In search tasks, the magnitude of neural differentiation varies as a function of target-distracter dissimilarity, i.e., the degree to which the target “pops out” perceptually. But notably, such differentiation also occurs for salient stimuli that are behaviorally irrelevant or even counter to task goals. Thus, whether a potentially rewarded target or an irrelevant distracter, the degree to which a stimulus is “selected” by FEF may portend the likelihood that a saccade will be directed toward it, and there is recent evidence to suggest that such activity may exert its influence by modulating extrastriate representations of the stimulus itself [41]. In this regard, it is important to emphasize that the relationship between subjective saliency as might be represented in FEF and the computation of objective saliency based solely on physical stimulus attributes is not one-to-one. Thus, mischaracterization of a distracter stimulus as “target” by FEF visual activity is associated with error trials such as the “misses” and “false alarms” reported here for the CO task (Figure 6) and for saccades to distracters in the standard oddball task as previously reported [23]. Although it seems clear that, when present, such activity can influence the saccadic choice, our finding that it may do so on the CO but not the CS task points to the circumstantial nature of its relationship to planning a motor response to a behaviorally relevant visual goal.

Underscoring this point are data from three search task variants involving a search step, an anti-saccade, and a varying speed-accuracy trade-off. During performance of the search-step task, in which a target singleton abruptly steps to a new location on some trials, neural selection of the stepped target is dissociated from performance: on correct trials its timing is not consistent with having informed the ensuing movement, and on error trials it is not spatially congruent with the choice [13, 42–44]. In the anti-saccade search task, in which the correct response is a saccade away from the singleton, cells with visual activity tend to select the singleton anyway, even though a saccade to its location must be suppressed [45]. Lastly, visually-responsive cells in FEF have been shown to be less selective for oddball targets when subjects are required to emphasize accuracy over speed in the standard oddball search task [18]. This is a particularly telling case, in that it highlights an essential distinction between a signal that identifies the target versus one that reflects salience. If such activity represents the perceptual evidence upon which the FEF targeting response is based, then a higher accuracy requirement should be associated with more, not less robust discrimination. On the other hand, if the activity signals salience at the RF location regardless of target/distracter status, then reduced activity is parsimonious with the action of top-down mechanisms that aim to prevent hasty commitment to a potentially erroneous choice.

Concurrent processing of perception and action

By design, the CS and CO tasks mandate that target selection and motor planning mechanisms operate concurrently such that perceptual judgments influence ongoing motor plans. As reported previously [20, 21], the impact of the perceptual evaluation can be observed as the rPT-dependent resolution of conflict in the activities of FEF M cells representing potential target locations. Even though performance criteria in non-urgent tasks incentivize serial staging of these events, there is ample evidence to indicate that parallel processing of potential targets does occur and influences incipient motor plans as well as saccade execution. Behavioral manifestations of concurrent processing in standard search tasks include erroneous saccades to distracters that are followed by exceedingly short-latency corrective saccades [39, 46, 47] and correct saccades having trajectories that deviate toward an adjacent distracter [39, 48, 49]. Neural correlates of such target/distracter conflict have been observed in presaccadic activity recorded in the FEF [49, 50] and the superior colliculus [47, 48]. Although such correlates have not been reported for pure V cells in FEF, the idea that such activity would reflect the relative salience of competing stimuli is consistent with the behavioral findings, data from VM cells [49, 50], and a framework in which V cell activity reflects saliency-based attentional deployment to potentially relevant locations.

Implications of V, VM, and M cell classification

The present study focuses on FEF cells classified as exclusively visual, a.k.a., V cells, even though prior studies of target selection have associated this capacity with cells (V and VM) having both visual and visuomotor signatures [6, 11, 29, 30]. As noted in past studies, the parsing of FEF cells into discrete V, VM, and M categories obscures the fact that there may be a continuum of responsivities between the V and M extremes. Nevertheless, our effort to identify visually-responsive neurons lacking any quantifiable sign of motor-contingent activity is crucial to the interpretive framework, and the findings highlight potential confounds for interpreting any results based on mixed populations of V and VM neurons.

That FEF visual- and motor-related activations represent distinct functional capacities is well established, particularly in the context of visual search where they have been linked to target and response selection, respectively [10, 11, 43, 51]. Presumably because of their shared visual response characteristics, both V and VM cells have been shown to differentiate targets from distracters, and likewise, owing to their shared motor-related capacity, both M and VM cells select the saccadic responses necessary to acquire targets. However, for VM cells, which constitute the majority in many samples, distinguishing target selection from response selection is not trivial. The distinction relies on quantitative assessment of whether neural differentiation is better time-locked to the stimulus or the saccade [43, 44], but even in strictly serial tasks, and given ample variance in RT, parsing the respective contributions of target selection and response selection to a time-varying activity profile is challenging. The problem is exacerbated, if not rendered impossible, for tasks, like the CS, CO, and other search task variants (e.g., search step, anti-saccade), in which sensory and motor-related responses overlap in time. The current dataset illustrates the potential for such a confound: in contrast to the all-or-none CS versus CO task dependence evident for V cells, differentiation for VM cells is a matter of degree, not kind (Figure 7E). For these cells, differentiation on the CS task likely reflects the synchronous contributions of a non-selective V component and a highly selective M component, an inference that can only be drawn from consideration of how the pure V and M activity profiles would merge as the urgent decision/choice unfolds. In point of fact, without knowledge of those V and M components it is impossible to know what the VM activity actually represents when a perceptual judgement and motor plan interact in time.

The interpretative ambiguity of VM activity has important implications for understanding if FEF V activity is part of the substrate for deploying endogenous, as well as exogenous attention. Previous studies suggest that FEF neurons “select” target locations in tasks for which saliency is not considered to be the defining characteristic of the target. Such neural discrimination has been observed in the context of feature conjunction search [37, 38] and in tasks that dissociate behavioral relevance from any discernible differences in saliency among stimuli [27]. In these cases, there is a strong argument to suppose that saliency-based mechanisms do in fact make a substantial contribution to performance owing to feature priming effects that blur the line between bottom-up and top-down mechanisms [32, 40]. But beyond that caveat, whether visual selection is mediated by pure V cells (and thus V activity) is difficult to discern in most such studies. Given the potential for confound, efforts to delineate the respective contributions of V and M-contingent activation to the VM population and to task performance will be essential for defining the roles of FEF activity phenotypes to visuomotor control.

STAR METHODS

Contact for reagent and resource sharing

Request for additional resources may be directed to the Lead Contact (vscerra@salk.edu).

Experimental model and subject details

All experimental procedures were conducted in accordance with the NIH Guide for the Care and Use of Laboratory Animals, USDA regulations, and the polices set forth by the Institutional Animal Care and Use Committee (IACUC) of Wake Forest School of Medicine, an AAALAC accredited institution. Two healthy, adult male rhesus monkeys (Macaca mulatta) served as experimental subjects. Animals were purpose-bred for research purposes (Alpha Genesis, Yemassee, SC; Monkey R - 4.68 kg.; Monkey I - 3.52 kg. at time of procurement) met all WFSM veterinary criteria for health and immune status at the time of procurement. One of the subjects had participated in a closely related study of prefrontal cortex and its participation in the current set of experiments was designed to take advantage of preceding training history and existing surgical preparation.

Monkey subjects were trained via standard operant methods to obtain water or other preferred liquid reinforcement for performing behavioral tasks to criterion. During training or recording sessions, subjects performed visual tracking tasks (i.e., look at visual stimuli on a video monitor) until sated as indicated by waning motivation to continue performing trials. If necessary, animals were supplemented after the session to meet a daily requirement as specified by WFSM policy. Daily health status was monitored and recorded. A healthy body weight was ensured by comparing measured weight to a pre-determined, non-experimental baseline.

In preparation for training and neural recording, a MRI-compatible head post and a MRI-compatible recording cylinder were secured to the skull. The head post served to orient and stabilize the head for video-based eye tracking and neural recording. The recording cylinder allowed access to the arcuate region of prefrontal cortex for single-unit recording in the FEF. All sterile surgical procedures were performed under general anesthesia. Post-surgical pain and inflammation were managed with an opioid analgesic (Buprenorphine hydrochloride; 0.01–0.02 mg/kg IM, once post-op & q12hrs thereafter for 24–48 hrs. or until lacking painful signs) and a NSAID (Ketoprofen; 2.5–5.0 mg/kg IM, post-op, once daily until painful signs dissipate).

The monkeys were either pair or singly housed in Allentown quad format cages that met all regulatory requirements. Animals were singly housed if documented as incompatible in accordance with WFSM environmental enrichment policy. Animals received supplemental food and physical enrichment as provided by laboratory and/or WFSM Animal Resources Program staff. Food was provided ad libitum unless otherwise specified for veterinary reasons.

Method details

Behavioral and electrophysiological recording techniques

Eye position was monitored via infrared tracking (EyeLink 1000, SR Research, Ottawa, Ontario, Canada) and analog signals proportional to horizontal and vertical eye position recorded. Stimulus presentation, reward delivery, and data acquisition were controlled using a purpose-designed software/hardware package (Gramalken; Ryklin Software, New York, NY). Gray, red, and green stimuli were presented on a HD LCD monitor (Syncmaster 2233, Samsung) at a viewing distance of 57 cm. Saccade onset was identified as the time at which eye velocity exceeded 50°/s.

Neural activity was recorded using single tungsten microelectrodes (FHC, 2–4 MΩ) driven by hydraulic Microdrive (FHC, Bowdoin, ME). Neural signals were amplified and filtered, and individual neurons were isolated based on amplitude criteria and/or waveform characteristics (FHC; Plexon, Inc, Dallas, TX). Putative FEF neurons were recorded within the boundaries of areas from which saccade-like movements could be evoked by low current microstimulation (76 ms stimulus trains at 350 Hz, with amplitude ≤ 50 μA) [1, 21, 52]. Evoked vectors ranged from 5° to 15° across recording sites and were generally consistent with RF estimates based on recording of single neurons in the vicinity.

Behavioral tasks

The delayed saccade task (Figure 1A) required a saccade to a lone peripheral target to obtain a liquid reward (drop of juice). Each trial began with fixation of a central red spot. Once achieved, a single red target appeared (Target on) at a peripheral location. After a variable delay (500–1000 ms), the fixation spot was extinguished (Go signal) and a saccade to the peripheral target required to obtain reward. Delayed saccade trials were used to estimate the RF of the recorded neuron prior to study with the three (EO, CS, CO) choice tasks. Once a neuron was isolated, RF estimates were determined from 16 directions and 3 eccentricities (6°, 10°, 12°) and were generally consistent with the vectors of saccades evoked by microstimulation (see above) of nearby tissue.

The EO task (Figure 1B) required a saccade to the color oddball, selection of which is traditionally thought to engage bottom-up attentional mechanisms [34, 35]. During fixation, an array of four gray stimuli (Targets on) was presented for a variable period (150–250 ms). The uninformative gray stimuli were followed by presentation of the Cue, at which time, one stimulus changed to red or green and the remaining three changed to the complementary color. The target was randomly assigned to any one of the four possible locations on a trial. After a variable period of 500–1000 ms, the fixation point was turned off (Go signal) and a saccade to the oddball stimulus required to obtain a reward. The stimuli were equally-spaced and separated by 90°. Array orientation and eccentricity were chosen to locate one of the four within the RF of the recorded cell. Typically, two 30 trial blocks, one of red oddball trials and one of green oddball trials, were collected during recording of a single cell.

The CS task (Figure 1C) required a saccade to the stimulus that matched the color of the initial point of fixation. The color of the fixation stimulus, either red or green, served as the template for the target and it was randomly assigned from trial to trial. After a 500 ms delay, two uninformative gray stimuli (potential targets) were presented at diametrically opposed locations (Targets on), with direction and eccentricity chosen to place one stimulus inside and one opposite to the cell’s RF. After a short delay, the fixation point disappeared (Go signal) instructing the monkey to begin preparing an uninformed choice. During the RT interval, the Cue was presented after a variable delay termed “Gap” (0 – 250 ms), at which time one stimulus changed to red and the other to green. The locations of the red and green stimuli were randomly assigned for each trial. A saccade to the stimulus that matched the color of the fixation spot on that trial was required within 400 ms to achieve reward. Typically, two 50 trial blocks of color-interleaved trials were collected.

Like the EO task, the CO task (Figure 1D) required a saccade to the color oddball within the array of four stimuli. In contrast to the EO task, the CO task incorporated urgency by mirroring the time course of the CS task. Accordingly, rather than being presented before the Go signal, the Cue that revealed target and distracter identities/locations was presented a variable Gap interval after the Go signal. The location of the oddball was randomly assigned on each trial to one of the four possible locations. A saccade to the oddball within 400 ms was required to obtain reward. During recording, short blocks of 20–40 trials with a red target and green distracters were interleaved with comparably sized blocks with a green target and red distracters. This presentation formula, which parallels that used in seminal studies of FEF target selection, (e.g., [2, 10, 11]), is thought to promote short-term perceptual priming but not long-term perceptual learning of color information [53].

As described previously [20, 21] for the CS task, the monkey subjects’ imperative for the urgent tasks (CS and CO) was to use information provided by the color cue to modify an ongoing motor plan in an effort to acquire the target. On any given trial, the likelihood of success depended on the amount of time available to view the Cue (rPT) before committing to a saccadic choice. For any given trial, the rPT was a function of both RT and Gap according the formula rPT = RT – Gap.

Quantification and Statistical Analysis

Behavioral metrics

All data analyses were performed in Matlab (The MathWorks). Tachometric curves, which indicate the probability of a correct choice as a function of rPT, were constructed as previously described [20, 21, 54, 55]. Bootstrapping was used to determine 95% confidence intervals for those curves. Unless otherwise stated, trials were categorically divided into “uniformed” and “informed” choices for subsequent analyses of neural data. The uninformed sample consisted of trials of rPT < 110 ms for the CS task and rPT < 95 ms for the CO task, and corresponded to chance performance (i.e., before the respective tachometric curves began to rise). The informed sample consisted of trials of rPT > 155 ms for the CS task and rPT > 130 ms for the CO task, and corresponded to asymptotic performance (i.e., tachometric curves saturated). Cutoff values of rPT were estimated using a piecewise linear sigmoid fit to the task-specific tachometric curves and bootstrapping to establish 95% confidence intervals. For each cutoff value, the average of these estimates for the two monkeys was used to allow for data pooling.

Neuron classification

Neurons were designated to be V, VM, or M in a manner consistent with established criteria [1]. Neurons were classified by assessing differences in activity between stimulus onset (20–250 ms after Targets on) and a pre-stimulus baseline (250–0 ms before Targets on), as well as between saccade (75–10 ms before movement onset) and a pre-movement baseline (150 before to 50 ms after the Go) of the CS task. Significance (p < 0.01) was assessed via permutation test [56] (20,000 permutations). Of 222 recorded neurons, (114 for Monkey R and 108 for Monkey I), 168 were activated during either the visual, presaccadic, or both visual and presaccadic epochs described above. Of these, 166 neurons were classified as M (n = 50), VM (n = 76), or V (n = 40) based on stated statistical criteria. All of the 166 classified neurons were also tested with a smaller number of single-target delayed-saccade task for qualitative verification of response properties.

Normalization of neural activity

Continuous firing rate traces were computed by convolving evoked spike trains with a Gaussian function (σ = 15 ms) and averaging across trials. From such traces, peak firing rates during a sensory epoch (0–200 ms after stimulus onset) and a motor epoch (300 ms before to 300 ms after saccade onset) were determined, and responses were normalized to the larger of the twovalues. For each neuron, this same factor was applied to the responses obtained in the four behavioral tasks. All population averages are based on normalized firing rate.

Evaluation of presaccadic spatial selectivity

For neurons determined to have significant presaccadic activation, the firing rate for movements into the RF was compared to that for movements to the diametrically opposed location in the CS task. A permutation test ([56]; 20,000 permutations) was used to establish significance (criterion set at p < 0.01). These trial inclusion criteria were designed specifically to identify neurons with spatially-selective presaccadic activity, and thus putatively related to response selection. Beyond the main criteria, neurons (n = 2) were excluded if the RF location appeared to be inconsistent across the different task types.

ROC score

A ROC ideal observer analysis [57] was used to quantify neural discrimination capacity. The calculation of a ROC score allows investigation of differences in activity between arbitrary pairs of trial classes, say A and B, to quantitatively gauge the strength of the neural signal differentiating conditions A and B. For each neuron, the spike train in each trial, aligned on a relevant event (e.g., saccade onset), was converted into a continuous firing rate function by convolving it with a Gaussian of σ = 15 ms. Continuous ROC curves were generated by calculating the ideal observer’s probability of successfully discriminating the responses in A versus B trials at each millisecond time point. Mean ROC curves were obtained by averaging across neurons. With this method, values of 1 or 0 indicate perfect ideal observer performance, whereas 0.5 indicates chance performance. This procedure was used to quantify spatial discrimination (A and B trials corresponding to movements into vs. away from the RF; Figure 6A, C) as well as target/distracter discrimination (A and B trials corresponding to distracter in vs. distracter out of the RF; Figure 6A, B, D).

Key Resources Table

REAGENT or RESOURCE	SOURCE	IDENTIFIER
Experimental Models: Organisms/Strains
Macaca mulatta	Alpha Genesis, Yemassee, SC	N/A
Software and Algorithms
Matlab with Statistics Toolbox R2013b – R2018b	Mathworks	https://www.mathworks.com/
Gramalken data acquisition system	Ryklin Software	http://www.Ryklinsoftware.com
Other
Plexon neural data acquisition system	Plexon Inc.	https://plexon.com/
EyeLink 1000 Eye tracker	SR Research	https://www.sr-research.com/

Highlights.

Urgent choice tasks reveal temporal relationships between neural firing and behavior

FEF visual neurons select targets only for some visually-informed saccadic choices

FEF visual target selection reflects feature saliency rather than feature relevance

Isolating the pure visual signal is critical for interpreting FEF activity in general