Timing of response onset and offset in macaque V4: stimulus and task dependence

Abstract

In the primate visual cortex, both the magnitude of the neuronal response and its timing can carry important information about the visual world, but studies typically focus only on response magnitude. Here, we examine the onset and offset latency of the responses of neurons in area V4 of awake, behaving macaques across several experiments in the context of a variety of stimuli and task paradigms. Our results highlight distinct contributions of stimuli and tasks to V4 response latency. We found that response onset latencies are shorter than typically cited (median = 75.5 ms), supporting a role for V4 neurons in rapid object and scene recognition functions. Moreover, onset latencies are longer for smaller stimuli and stimulus outlines, consistent with the hypothesis that longer latencies are associated with higher spatial frequency content. Strikingly, we found that onset latencies showed no significant dependence on stimulus occlusion, unlike in inferotemporal cortex, nor on task demands. Across the V4 population, onset latencies had a broad distribution, reflecting the diversity of feedforward, recurrent, and feedback connections that inform the responses of individual neurons. Response offset latencies, on the other hand, displayed the opposite tendency in their relationship to stimulus and task attributes: they are less influenced by stimulus appearance but are shorter in guided saccade tasks compared with fixation tasks. The observation that response latency is influenced by stimulus- and task-associated factors emphasizes a need to examine response timing alongside firing rate in determining the functional role of area V4.

NEW & NOTEWORTHY Onset and offset timing of neuronal responses can provide information about visual environment and neuron’s role in visual processing and its anatomical connectivity. In the first comprehensive examination of onset and offset latencies in the intermediate visual cortical area V4, we find neurons respond faster than previously reported, making them ideally suited to contribute to rapid object and scene recognition. While response onset reflects stimulus characteristics, timing of response offset is influenced more by behavioral task.

INTRODUCTION

To understand the neuronal basis of visual encoding and perception, neurophysiological studies typically relate the magnitude of the neuronal response to the visual stimulus or the animal’s behavior. Another important yet less studied attribute of the neuronal response, its onset latency, can provide valuable insight as well. It has been argued that response timing carries stimulus information that is complementary to that contained in the spike rate (Reich et al. 2001; VanRullen et al. 2005). The onset latency may reflect the neuron’s role in the network, i.e., its position within the information processing hierarchy in the cortex (Chen et al. 2007; Felleman and Van Essen 1991; Nowak and Bullier 1997; Schmolesky et al. 1998; Ungerleider et al. 2008; for comprehensive reviews on relative timing and synchrony, see Singer 1999 and Oram 2010) and whether the primary driving inputs to the neuron come from feedforward or feedback connections. Furthermore, the timing of the neuronal response relative to the percept or behavior can reveal whether the response in question could contribute to the percept. It has also been argued that the latency of response offset, rather than onset, provides a more accurate measure of information transmission in lateral geniculate nucleus (LGN), area V1, and middle temporal area (MT), since the response onset, unlike offset, depends critically on the stimulus and the adaptation state of the neuron (Bair et al. 2002). Thus characterizing the timing of neuronal responses, in terms of their onset and offset, may be critically important for gaining a complete understanding of perception.

Here we focus on the response latency of neurons in area V4, an intermediate stage in the primate ventral visual pathway that is involved in form and color processing and that is important for object recognition (for a review, see Roe et al. 2012). The most widely cited reference for V4 response latencies, Schmolesky et al. (1998), reports a range of latencies from 72 ms to 159 ms. This is significant for several reasons. First, it would suggest that responses in area V4 are considerably delayed with respect to those in visual areas at a similar level in the processing hierarchy, like area MT and the frontal eye fields (FEF; Felleman and Van Essen 1991). Second, because recognition of complex scenes is thought to be possible within 80–150 ms after stimulus onset (Crouzet et al. 2010; Kirchner and Thorpe 2006; Thorpe et al. 1996), reported response latencies of 72–159 ms imply that many V4 neurons do not contribute to this capacity. Lastly, the mean response onset latency reported by Schmolesky et al. (1998) differs from what we have observed during our recordings from primate V4 (Bushnell et al. 2011; Oleskiw et al. 2018). With regard to latency of response offset, even less information is available; to our knowledge there is no published report of offset latencies in macaque V4.

To revisit the question about V4 onset latencies, and to quantify V4 offset latencies, we collated data from four studies conducted in three awake monkeys engaged in passive fixation or a shape discrimination task and investigated how stimulus characteristics and animal behavioral state influence response latency. We found that the median onset latency in V4 is 75.5 ms, and this measure is greatly influenced by stimulus characteristics. At the same time, the median response offset latency is 84 ms and this measure tended to depend less on the content of the stimulus and more on the behavioral state of the animal. These results support the idea that responses of a majority of V4 neurons emerge early enough to contribute to rapid recognition of objects. Our observation that offset latencies were longer than onset latencies provides a striking contrast to previous results from LGN, V1, and area MT. These findings reveal a potential role for recurrent circuits in the maintenance of V4 responses long after feedforward inputs are turned off.

METHODS

Experimental Data

The data analyzed here come from four studies conducted in three awake macaque monkeys by Pasupathy and colleagues at the University of Washington (El-Shamayleh and Pasupathy 2016; Kosai et al. 2014; Popovkina and Pasupathy 2019; Popovkina et al. 2019; studies 1–4, respectively). These four studies yielded six different data sets collected under a variety of stimulus and behavioral conditions (see details below). Here we analyze the onset and offset latencies of the 305 neurons included in these data sets (see Table 1). As detailed in Table 1, for each stimulus and behavioral condition, we include only the subset of neurons that exhibited robust evoked responses (see Neuron Inclusion Criteria). The results reported here have not been previously published. Below we describe the visual stimuli and behavioral tasks associated with each data set and identify the animals that contributed to each.

Table 1.

Number of neurons used in latency analysis

Study/Data Set No.	No. of Neurons Tested	Weak Responses	Unreliable Offset Latency	Included for Onset Analysis	Included for Offset Analysis	Median; Min No. of Trials
1. Partial occlusion
#1 (fixation)	91	4	0	87	87	72; 8
#2 (discrimination)
100% unoccluded	99	5		94		161.5; 60
95–99% unoccluded	98	13		85		68; 13
82–92% unoccluded	99	24		75		72; 17
59–77% unoccluded	84	33		51		46; 8
27–52% unoccluded	22	9		13		18; 7
2. Size
#3 (fixation)
Size 0.5x	36	5	0	31	31	150; 31
Size x	77	3	1	74	73	148; 14
Size 1.5x	80	3	2	77	75	149; 10
Size 2x	80	2	2	78	76	144.5; 6
Size 2.5x	80	1	2	79	77	121; 10
3. Behavioral engagement
#4 (fixation)	83	9	1	74	73	360; 60
#5 (discrimination)	83	8	7	75	68	205; 43
4. Stimulus interior fill
#6 (fixation)
Filled shapes	43	6	1	37	36	375; 48
Shape outlines	43	17	1	26	25	312.5; 32

Analysis	No. of Neurons	Excluded	Included for Analysis
Onset latency with fixation data data set #1, #3 (size 2x), #4, and #6 (filled)	91 + 80 + 83 + 43 = 297	4 + 2 + 9 + 6 = 21	276
Offset latency with fixation data (same as above)	297 (as above)	4 + 4 + 10 + 7 = 25	272
Onset latency: fixation vs. behavior data set #1 vs. #2; #4 vs. #5	91 + 83 = 174	9 + 13 = 22	152
Offset latency: fixation vs. behavior data set #4 vs. #5	83	21	62

“Study/Data Set No.”: number of neurons grouped by experimental condition in each data set. Rightmost column: the median and minimum number of trials across all cells used in each experiment to construct PSTHs and calculate onset and offset latencies. “Analysis”: summary of included subsets contributing to onset and offset latency analyses performed here.

Study 1.

Data sets #1 and #2 come from an experiment conducted in monkeys M and O investigating how occlusions modulate V4 responses. During this experiment, we first conducted a preliminary shape screen (data set #1) while animals were engaged in a passive fixation task (see Behavioral Tasks). We then asked how occlusion modulated shape responses as animals were engaged in a nonreaction time shape discrimination task (data set #2; see Behavioral Tasks). Data set #1 includes 91 neurons. Data set #2 includes an additional eight neurons (i.e., n = 99) for which a preliminary screen was not available.

For the preliminary shape screen (data set #1) we studied the responses of each neuron to 43 shapes (for example shapes, see Fig. 1A), each presented at up to eight rotations in 45° increments. For further details about stimuli, see Kosai et al. (2014). During the behavioral task for data set #2, we used two shapes, one preferred and one nonpreferred, that were identified using shape tuning characterization (data set #1). To determine how neuronal responses and behavior were modulated by occlusion levels, shape stimuli were partially occluded by a field of randomly positioned dots (Fig. 1B). Occlusion level was titrated by varying dot diameter and was quantified by the percentage of stimulus area that was unoccluded (see Kosai et al. 2014 for further details). We recorded the responses of each neuron under multiple occlusion levels ranging from 27% (high occlusion) to 100% (unoccluded stimuli); unoccluded stimuli were always included. To ask how occlusion affects neuronal response latency, we binned occlusion levels into five groups (see Table 1) because we did not study every occlusion level for every neuron.

Fig. 1.

Sample stimuli used in the experiments. A: subset of shape stimuli used in studies 1, 3, and 4. See Kosai et al. (2014) and Popovkina et al. (2019) for the entire stimulus set. B: example stimuli showing different levels of occlusion used in study 1. Partial occlusion was provided by 36 randomly positioned dots of varying diameter. Numbers denote the %stimulus area that was not occluded by the occluding dots; thus occlusion level increases with decreasing %unoccluded area. See Kosai et al. (2014) for details on how occluding dots were positioned and the diameter was varied. C: example stimuli used to study how neuronal responses are modulated by stimulus size (study 2). See El-Shamayleh and Pasupathy (2016) for details. D: example stimuli used to investigate how interior fill modulates neuronal responses (study 4). See Popovkina et al. (2019) for details on stimulus design.

Study 2.

Data set #3 comes from an experiment investigating how stimulus size influences neuronal responses. These data include responses of 80 V4 neurons recorded in monkeys M and O during a fixation task. A set of 9 or 13 shapes were presented at 3, 4, or 5 sizes (x) (0.5x, 1.5x, x, 2x, and 2.5x, Fig. 1C), each at 8 rotations in 45° increments and within the receptive field (RF) of the neuron under study. See El-Shamayleh and Pasupathy (2016) for further details about visual stimuli.

Study 3.

Data sets #4 and #5 come from an experiment investigating how behavioral engagement modulates neuronal responses. These data sets include 83 V4 neurons recorded from monkeys O and P. In this study, we used a set of nine stimuli to study responses of each neuron under passive fixation (data set #4) and behavior (data set #5). The stimuli for each experimental session were constructed from a crossed design of three shapes and three colors; the shapes and colors were customized for each neuron based on preliminary characterization of shape and color preferences and were chosen to span the dynamic range of responses for individual neurons. Data set #5 was recorded while animals were engaged in a reaction time shape discrimination task (see Behavioral Tasks) using the same nine stimuli used for the fixation experiment in data set #4. See Popovkina and Pasupathy (2019) for further details.

Study 4.

Data set #6 comes from an experiment investigating how neuronal responses are influenced by the interior fill of shape stimuli. It includes the responses of 43 V4 neurons recorded from monkeys O and P during a fixation task. Stimuli included 51 shapes each presented at up to 8 rotations in 45° increments. Stimuli were either simple outlines (see Fig. 1D for examples) or also included a filled interior.

Animal Preparation

Three adult male macaque monkeys (Macaca mulatta; monkeys M, O, and P) participated in these studies. All animal preparation details are available elsewhere (El-Shamayleh and Pasupathy 2016; Kosai et al. 2014). Briefly, animals were implanted with custom headposts and recording chambers positioned over dorsal area V4. All animal procedures conformed to the National Institutes of Health guidelines and were approved by the Institutional Animal Care and Use Committee at the University of Washington.

Behavioral Tasks

Animals were trained to perform a fixation and a shape discrimination task. In the fixation task, animals were required to maintain fixation on a central spot (0.1° white spot) within a circular window of 0.5° to 0.75° radius, while stimuli were presented within the receptive field (RF) of the neuron under study. For the passive fixation data sets #1, #3, and #6, each stimulus was presented for 300 ms. For data set #4, in the majority of sessions stimuli were presented for 250 ms (71/83 neurons), and in the rest of the sessions, stimuli were presented for 300 ms (12/83 neurons). Three to five stimuli were presented per trial, with a 200-ms blank period preceding each stimulus. Stimuli were randomly interleaved and presented at least three times (median: 3 repetitions for data set #1; 9 repetitions for data set #3; 60 repetitions for data set #4; 5 repetitions for data set #6).

In the discrimination task, animals were trained to report whether two sequentially presented shapes were the same or different. As animals maintained fixation, two shapes were presented in sequence. The first shape (“reference”) was presented at central fixation; the second shape (“test”) was presented at the center of the RF of the neuron. Animals had to indicate whether the test and reference shapes were the same or different by making a saccade to one of the target dots on the screen: right for same and left for different. Animals were trained on two variants of this task: a fixed duration version (monkeys M and O) and a reaction time (RT) version (monkeys O and P). In the fixed duration task, the test stimulus was presented for 600 ms within the RF. Then, the test stimulus and the fixation spot were extinguished, and the two target dots appeared 6° to the left and right of central fixation (see Kosai et al. 2014). In the RT task, the target dots and the test stimulus appeared simultaneously on the screen and the animal was free to make a saccade at any time within 1,500 ms of the test stimulus onset. In this case, the test stimulus was extinguished as soon as the eye exited the central fixation window (see Popovkina and Pasupathy 2019). Average reaction time was 214 ± 56 (SD) ms across both animals. Data set #2 was collected as animals performed the fixed duration (non-RT) task; data set #5 was collected during the performance of the RT task. In both cases we analyzed the latency of responses to test stimulus presentation. Median number of repetitions was 34 for data set #2 and 40 for data set #5.

Visual Stimulus Presentation

In all studies, stimuli were presented against a uniform gray background (luminance: 5.4 cd/m²) on a spectrally calibrated (PR650, PhotoResearch) CRT monitor positioned 45 to 57 cm away from the animal. Eye position was monitored using an infrared eye tracking system (EyeLink 1000; SR Research) and coordinated with stimulus presentation using custom software based on Pype (Mazer 2013). The animal was rewarded with drops of water or juice for successful fixation for the duration of a trial (typically 2,000–2,500 ms) in the fixation task and for correct performance (saccade to correct target location) in the discrimination task. To synchronize the stimulus with the recorded spikes, we digitized the output of a photodiode on the screen using the same I/O device that recorded the spikes. To adjust our latency measurements for the difference in time between stimulus onset within the RF and the photodiode onset at the lower left of the monitor, for each neuron we linearly estimated the time difference between the presentation of stimulus within the RF and the photodiode onset.

Data Collection

Each day, we lowered a single tungsten microelectrode (FHC) into area V4 using a stepper motor drive (Gray Matter Research). We amplified and filtered the signals from the electrode and sorted waveforms (Plexon Systems) to identify single units. All details about mapping of single neuron RFs, preliminary characterization of color and shape preferences and sizing of stimuli are presented in detail elsewhere (El-Shamayleh and Pasupathy 2016; Kosai et al. 2014; Popovkina et al. 2019).

Data Analysis

We determined how response onset and offset latencies of individual neurons depended on a variety of task and stimulus conditions. For each neuron studied during a fixation task (data sets #1, #3, #4, and #6), we computed response onset latency based on responses to preferred stimuli defined as those stimuli that evoked greater than half of the maximum average response during the stimulus presentation duration which was typically 250 ms or 300 ms. We constructed peristimulus time histograms (PSTHs) based on tens of repetitions of preferred stimuli (see Table 1, rightmost column, for median number of repeats across neurons) and smoothed them with a Gaussian kernel (σ = 4 ms). PSTHs were aligned on the time of stimulus onset/offset for calculation of response onset/offset latency as described below. For neurons studied during behavioral tasks (data sets #2 and #5), we similarly computed onset and offset latency based on responses to preferred stimuli. For data set #2, where only two stimuli were used, the preferred stimulus was defined as the one that evoked stronger responses. Such restriction of trials to preferred stimuli did not affect response latencies (see results). All analyses were conducted using MATLAB (MathWorks).

Onset and Offset Latency

We calculated latencies according to the commonly used half-height method (Gawne et al. 1996; Lee et al. 2007; Levakova et al. 2015). Specifically, onset latency was taken as the first time point after stimulus onset at which the PSTH exceeded the mean of the peak and baseline rates. The baseline was estimated as the mean activity during a 75-ms window before stimulus onset, and the peak was the maximum rate in the PSTH between 25 ms and 250 ms after stimulus onset. The upper limit of 250 ms allowed us to identify the first peak in neurons with multiple peaks in their response profile.

Response offset latency was calculated similarly (also based on the half-height method) as the first time point after stimulus offset at which the PSTH was lower than the mean of the “sustained” and baseline rates. The sustained rate was the mean activity in the 100-ms window before stimulus offset, and the baseline rate was the same as defined above.

To ask whether the difference in response latencies for two stimulus/task conditions for the same neuron were statistically significant, we conducted a randomization test. We randomly shuffled and assigned trials to one condition or the other and recomputed latencies; we repeated this procedure 1,000 times. The original latency difference between the two stimulus/task conditions was deemed statistically significant for a neuron if it fell outside of the 95% limits of the latency distribution based on the shuffled trials.

Neuron Inclusion Criteria

For each stimulus condition, we excluded those neurons from our analysis that did not exhibit robust responses, i.e., neurons with a maximum firing rate that did not exceed the baseline rate by 10 SD and neurons with a maximum firing rate that was less than 5 spikes/s (see Table 1), We also excluded neurons for which latency estimates were unreliable. We assessed reliability using bootstrap analysis. On each of 1,000 iterations, we randomly chose 75% of the trials (with replacement) and constructed a distribution of estimated response latencies. Following Sundberg et al. (2012), we deemed latency estimates unreliable if the 95% confidence interval was ≥30 ms. For the onset latency, none of the neurons were excluded since all confidence intervals were less than this threshold. For offset latency calculations, several neurons did not meet this threshold and were therefore excluded (Table 1).

RESULTS

We quantified onset and offset latencies of 305 V4 neurons in three macaque monkeys performing either a passive fixation or active shape discrimination task. We investigated how response latency depended on the stimulus attributes and on the behavioral task performed by the animal.

Distribution of Response Onset Latency across V4 Neurons

Figure 2 shows the distribution of onset latency for responses of 276 V4 neurons to the presentation of a variety of shape stimuli (Fig. 1, shapes are scaled to RFs, see methods) while animals were engaged in a passive fixation task (see methods). The neurons included in this analysis are from data sets #1, #3, #4, and #6 (see Table 1 and Fig. 2 legend for details). For each neuron, we first constructed an average PSTH based on responses to preferred stimuli, i.e., those that evoked greater than half of the maximum neuronal response. Then, we computed the response onset latency by identifying the time from stimulus onset to half of the maximum response in the PSTH. Across our data set, onset latency varied over a broad range: from 25 to 204 ms (median 75.5 ms). Thus roughly 42% of the neurons in these data sets displayed shorter onset latency than the shortest V4 latency previously reported (72 ms; Schmolesky et al. 1998). Notably, a minority of neurons in our data set (38/276) that exhibited a transient increase in response upon stimulus offset also exhibited significantly shorter onset latencies: median onset latency was 63.1 ms vs. 82.3 ms for neurons with and without offset transients, respectively (Fig. 2B). Thus the group with the offset transients may be a distinct fast-responding class of cells that is V1-like both in terms of their shorter onset latency and their offset transients.

Fig. 2.

Distribution of response onset latencies of macaque V4 neurons. A: onset latencies of 276 V4 neurons from 3 monkeys (shades of gray indicating animals). Latencies were calculated based on responses to a variety of shape stimuli using the half-height method (see methods for details). Across V4 neurons, onset latencies ranged from 25 ms to 204 ms. Arrow indicates mean latency ± SD (79.7 ± 27.3 ms). Median latency = 75.5 ms. Variability in response onset latency ranged from 0 to 9.7 ms (median = 0.44) and there was a small but statistically significant negative correlation (r = 0.28, P < 0.01) between the variability in neuronal response and onset latency. B: onset latencies of cells with (n = 38) and without (n = 238) off response. Neurons with an offset transient exhibited significantly shorter onset latencies: means ± SD = 63.1 ms ± 16.6 ms (for neurons with off response) versus means ± SD = 82.3 ms ± 27.7 ms (for neurons without off response); P = 4.53e-05.

Influence of Stimulus Features on Response Onset Latency

Past studies have demonstrated that V4 response magnitude depends on stimulus identity (Desimone and Schein 1987; Gallant et al. 1993, 1996; Kobatake and Tanaka 1994; Nandy et al. 2013, 2016; Pasupathy and Connor 1999, 2001), size of visual stimuli (Cheng et al. 1994; Desimone and Schein 1987; El-Shamayleh and Pasupathy 2016) and the extent to which stimuli are occluded (Bushnell et al. 2011; Kosai et al. 2014). In addition, we recently demonstrated that V4 response strength is modulated by whether the 2-dimensional stimulus is an outline or a filled shape (Popovkina et al. 2019). Here we investigate whether these attributes, stimulus preference, size, interior fill, and partial occlusion, also influence response latency in V4.

Stimulus preference.

By definition, preferred stimuli are those that evoke stronger neuronal responses. To determine whether preferred stimuli are associated with shorter response latency, we analyzed the responses of 276 neurons in 3 monkeys to a set of 43 shapes presented at eight different orientations during a passive fixation task (data sets #1, #3, #4, and #6). We divided stimuli into preferred and nonpreferred (see methods). Ninety-two of 276 neurons were excluded because the data set included no stimuli that evoked a statistically significant response that was less than half of the maximum. In other words, we could not calculate an onset latency based on nonpreferred responses. We constructed PSTHs based on responses to preferred and nonpreferred stimuli for each of the remaining 184 neurons and compared the corresponding onset latencies (Fig. 3A). We did not find a statistically significant difference in the onset latencies of responses (paired t test, P = 0.23) even though there was a substantial difference in the response magnitudes for preferred and nonpreferred stimuli across the population (Fig. 3B). To ensure that the latency measure was not influenced by a smaller number of nonpreferred than preferred stimuli that were used, we considered another stimulus partition in which stimuli were divided into two equal-sized groups based on response magnitude, i.e., larger or smaller than median. The results were similar. Across neurons, there was no correlation between response magnitude difference and latency difference for preferred and nonpreferred stimuli (Pearson’s correlation, r = −0.042, P = 0.57). These results are consistent with previous findings in V1 (Celebrini et al. 1993; Gawne et al. 1996; Gawne 2000) and V4 (Mysore et al. 2006).

Fig. 3.

Effect of stimulus shape on the onset latency and magnitude of responses. A and B: onset latencies (A) and response magnitudes (B) for neurons in four fixation tasks (n = 184), calculated for preferred (abscissa) and nonpreferred (ordinate) stimuli. A: across the V4 population, mean onset latency ± SD for preferred stimuli was 73.7 ± 23.6 ms (median = 70 ms) and for nonpreferred stimuli was 74.8 ± 25.7 ms (median = 70 ms) and there was no statistically significant difference between the 2 (mean difference = −1.054 ms; paired t test, P = 0.23). Filled symbols: neurons with statistically significant difference between response onset latencies to preferred and nonpreferred stimuli (P < 0.05). B: for preferred stimuli, mean response rate ± SD = 25.6 ± 17.3 spikes/s (median = 20.9 spikes/s); for nonpreferred, mean response rate ± SD = 10.9 ± 10.7 spikes/s (median = 7.21 spikes/s). Mean difference ∆ between responses to preferred and nonpreferred stimuli across all neurons in our study was 14.7 spikes/s (paired t test, P < < 0.001).

Interior fill.

We have previously reported that the responses of a majority of V4 neurons are modulated by the interior fill of visual stimuli: some neurons respond more strongly to stimuli defined both by a boundary and an interior fill, and other neurons to stimuli defined by an outline alone (Popovkina et al. 2019; for sample stimuli, see Fig. 1D). We used these data (data set #6) to assess whether interior fill also modulated response onset latency. In this analysis, we included 23 neurons for which both filled and outline stimuli evoked a robust response (see methods, Neuron Inclusion Criteria). The difference between response latencies for filled and outline stimuli was large (mean Δ ± SD = 8.57 ± 21.6 ms; Fig. 4A) and statistically significant (Wilcoxon signed rank test, P < 0.02). Latencies were shorter for responses to filled stimuli regardless of whether the neuron responded more robustly to filled or outline stimuli (Fig. 4B, top and bottom, respectively). Thus, while response magnitude may be higher or lower for filled stimuli, response latency for filled stimuli was shorter for 20 of 23 V4 neurons. Previous studies have demonstrated that stimuli defined by lower spatial frequencies (SFs) evoke responses with shorter latencies than stimuli defined by higher SFs (Bair and Movshon 2004; Frazor et al. 2004; Mazer et al. 2002). Since filled stimuli include lower SF power, our observation of shorter latencies for filled stimuli is consistent with these previous results.

Fig. 4.

Effect of shape interior fill on response onset latency. A: onset latencies of neuronal responses to filled stimuli (abscissa; mean latency ± SD = 82.5 ± 19.8 ms; median = 75 ms) and to stimulus outlines (ordinate; mean latency ± SD = 91.2 ± 25.5 ms; median = 82 ms). Neurons with larger average responses to filled stimuli are represented as circles while those with stronger responses to outline stimuli are represented as diamonds. Open symbols identify neurons with no significant difference between filled and outline stimuli in terms of response latencies. Across the population, mean difference ∆ between filled and outline stimulus latencies = 8.7 ms (Wilcoxon signed-rank test, P = 0.015, n = 23). B: peristimulus time histograms (PSTHs) of 2 example neurons, one that responded more strongly to filled stimuli (top), and another that responded more strongly to outline stimuli (bottom). In both cases, responses emerged significantly earlier for filled stimuli (see red and cyan markers in the scatter in B). Dotted line: stimulus onset. Shaded area: SE.

Stimulus size.

Stimulus size has been shown to modulate V4 responses: while some neurons respond better to larger stimuli, others respond better to smaller stimuli (El-Shamayleh and Pasupathy 2016). To determine whether stimulus size also modulates onset latency, we used the responses of 80 neurons to shape stimuli (as in Fig. 1C) varying in size (data set #3). We compared the response latency of individual V4 neurons for stimuli of a given size (x) with those for stimuli scaled to sizes 0.5x, 1.5x, 2x, and 2.5x (Fig. 5A, top to bottom; see Fig. 5C for example stimuli). In all pairwise size comparisons, response latency was shorter for larger sizes. We also found a systematic decline in latency across the 31 neurons that were tested with and exhibited robust responses (>5 spikes/s) to stimuli of all five sizes (Fig. 5B). Response latency tended to decrease with increasing stimulus size even among neurons that responded more strongly to smaller stimuli (e.g., Fig. 5C, bottom). For each of the 80 neurons, we conducted a linear regression to relate stimulus size to the latency of neuronal response onset. For 59 of the 80 neurons, we found a negative slope, indicating that larger stimuli were associated with shorter latency. This was also true for 13 of the 19 neurons that responded better to smaller stimuli, thus implying a dissociation between response magnitude and latency. This finding is consistent with previous findings that stimuli with greater low SF content evoke faster responses, since SF content varies inversely with stimulus size.

Fig. 5.

Effect of stimulus size on response onset latency. A: onset latencies of neuronal responses to stimuli of base size (abscissa, size x) compared with those at other sizes: 0.5x, 1.5x, 2x, and 2.5x (ordinate, from top to bottom) [data set #3, n = 80 neurons]. The number of neurons contributing to each panel differed since not all neurons were probed with all five stimulus sizes, and some neurons responded poorly to some of the sizes (see Table 1 for details). Circles: neurons with larger average responses to the size shown on the ordinate; diamonds: larger responses to size x on the abscissa. Open symbols identify neurons with no significant difference between response latencies to stimuli of 2 given sizes. Mean difference in onset latency between sizes (abscissa – ordinate) across all neurons is shown. B: distribution of onset response latencies for each of the five sizes (shown for a subset of neurons from A, where responses were collected to stimuli of all 5 sizes; n = 31). Arrows: mean latencies. C: peristimulus time histograms (PSTHs) of 2 example neurons to stimuli of different sizes (color coded correspondingly). In both cases onset latencies were shorter for the largest stimuli regardless of whether the responses increased (top) or decreased (bottom) with size.

Partial occlusion.

We have previously demonstrated that V4 response magnitude declines with increasing levels of occlusion of the preferred stimulus (Kosai et al. 2014). To investigate whether response latency also varies with occlusion, we studied 99 neurons at 4–9 levels of stimulus occlusion (see Fig. 1B for examples), ranging from 27% unoccluded area (high occlusion) to 100% unoccluded area (unoccluded stimulus; all from data set #2, see Table 1 for details). We found that onset latency for unoccluded stimuli (100% unoccluded area) was not significantly different than that for the other four occlusion levels (Fig. 6A; paired t test). We also found a similar result when we focused our analysis on a subset of 42 neurons that exhibited robust responses for unoccluded stimuli and the three lower levels of occlusion (Fig. 6B). Overall, the vast majority of neurons responded more strongly to unoccluded stimuli (87/99; example in Fig. 6C, top) while a few responded better to occluded stimuli (Fig. 6C, bottom). However, we found no systematic change in response latency as a function of occlusion in either group.

Fig. 6.

Effect of partial occlusion on response onset latency. A: onset latencies of neuronal responses to stimuli under different partial occlusion levels (ordinate) compared with responses to unoccluded stimuli. Circles: neurons with larger average responses to unoccluded stimuli than stimuli at a given occlusion level; diamonds: larger responses to given occlusion level than unoccluded stimuli. Open symbols: neurons with no significant difference between response latencies to stimuli under the occlusion levels shown. Mean difference ∆ between response latencies to unoccluded stimuli and the different levels of occlusion is as indicated. B: distribution of neuronal response latencies to stimuli under different occlusion levels (subset of neurons from A, where responses were collected to all stimuli within 59–100% unoccluded range; n = 42). Arrows: mean latencies. C: responses of 2 sample neurons to stimuli under different levels of occlusion (color-coded correspondingly). There were no systematic changes in response latency regardless of whether response strength declined (top) or increased (bottom) with increasing occlusion.

Influence of Task Type on Response Onset Latency

To determine how behavioral task engagement influences response onset latency in V4, we compared V4 latencies for two tasks: fixation and shape discrimination. In the fixation task, animals were required to passively fixate a dot in the center of the screen while stimuli were presented in the receptive field of a neuron. In the shape discrimination task, animals were required to report whether two sequentially presented stimuli were the same by making a saccade to an appropriate target dot (see methods). The discrimination task was either a reaction time task (RT; data set #5), where the animal initiated the saccade when ready, or a non-RT task (data set #2), where the animal was required to make a saccade when the target dots appeared on the screen. We assessed the response onset latency of 152 neurons whose responses were statistically significant during both the fixation and behavior tasks (with RT and non-RT data combined) but found no significant difference between these tasks (Fig. 7A; paired t test, P = 0.76; mean latency difference = −0.57 ms). This result agrees with previous findings (Boch and Fischer 1983). Response onset latencies were not significantly different when the RT and non-RT tasks were compared separately against corresponding fixation tasks (see Fig. 7 legend). Across the population, we found no correlation between the difference in response magnitude and difference in latency between the two tasks. In other words, neurons with stronger and those with weaker responses during the fixation task (e.g., see Fig. 7B, top or bottom, respectively) could exhibit shorter (or longer) onset latencies during the discrimination task.

Fig. 7.

Effect of task type on response onset latency. A: onset latencies of neuronal responses during a fixation task (abscissa; mean latency ± SD = 77.6 ± 25.2 ms; median = 76 ms) and a discrimination task (ordinate; mean latency ± SD = 78.2 ± 27.8 ms; median = 72 ms) tasks. Circles and diamonds: neurons with larger average responses in fixation and discrimination tasks, respectively. Open symbols: neurons with no significant difference between response latencies in the 2 tasks. Across the population there was no statistically significant difference in latencies between the 2 tasks. Data from the both reaction time (RT) and non-RT discrimination tasks are included here because the trends in the 2 data sets were similar: mean difference between latencies in fixation and non-RT discrimination task = −1.71 ms (t test, P = 0.57, n = 82); fixation and RT discrimination tasks = 0.76 ms (t test, P = 0.7, n = 70). B: peristimulus time histograms (PSTHs) of 2 example neurons during fixation and discrimination tasks are shown. Dotted line: stimulus onset. Shaded area: SE.

Response Offset Latency in V4 Neurons

Responses under passive fixation can linger long after stimulus offset in many V4 neurons. An example is illustrated in Fig. 8A. For this neuron the sustained response rate, measured in the 100 ms window before stimulus offset (denoted by 0 ms; trials are aligned on stimulus offset), was ~50 spikes/s. After stimulus offset, there was no significant change in the neuronal response for ~75 ms and the responses declined to baseline level at ~100 ms. To establish the trend across the population, we assessed the sustained responses before stimulus offset and the response offset latencies in 272 V4 neurons (data sets #1, #3, #4, and #6). We found that responses tended to remain long after stimulus offset, and based on fixation data, offset latency varied from 23 ms to 216 ms (median 81 ms, Fig. 8B). For almost all neurons (270/272), the sustained responses before stimulus offset were significantly (P < 0.01) greater than baseline (Fig. 8D; y-axis shows baseline-subtracted responses). Thus long offset latencies were not due to the lack of a measurable sustained response. We also found that offset latency was significantly shorter for preferred stimuli than nonpreferred (mean Δ ± SD = −9.85 ± 36 ms, paired t test, P < < 0.01; Fig. 8C). There were no significant differences in the offset latency for stimuli of any size compared with standard or for filled versus outline stimuli (P > 0.01).

Fig. 8.

Response offset latencies. A: peristimulus time histogram (PSTH) of an example neuron aligned to stimulus offset (0 ms, dotted line). Stimulus duration was 250 ms, so stimulus onset was roughly at −250 ms; the offset latency for this neuron was 85 ms. Shaded area: SE. B: distribution of offset latencies of 272 V4 neurons from 3 monkeys (shades of grey indicating animals). Offset latencies were calculated based on responses to a variety of shape stimuli using the half-height method (see methods for details). Across V4 neurons, offset latencies ranged from 23 ms to 216 ms. Arrow indicates mean latency ± SD (91 ± 34.4 ms). Median latency = 84 ms. Variability in response offset latency ranged from 0.05 ms to 28 ms (median = 1.8 ms). There was a small but statistically significant negative correlation between the variability of neuronal response and offset latency variability (r = −0.29; P < 0.01). C: response latencies based on preferred and nonpreferred stimuli. Nonpreferred stimuli were associated with significantly longer offset latencies as indicated. D: magnitude of sustained response (baseline-subtracted) in the 100 ms before stimulus offset compared with offset latency. For most neurons (270/272; filled symbols), the sustained response was significantly (P < 0.01) above baseline response that was computed before stimulus onset.

Influence of Task Type on Response Offset Latency

Whether the animal was engaged in passive fixation or active behavior had a strong influence on offset latency. For this comparison, we considered offset latency for neurons studied during the RT discrimination task (data set #5) and for the same neurons during the fixation task. We excluded data collected when animals were engaged in the non-RT discrimination task (data set #2) because the stimulus presentation time in that task was twice as long as in the corresponding fixation data set (600 ms vs. 300 ms). We found that responses declined much more rapidly after stimulus offset during the RT discrimination task than during the fixation task (Fig. 9A). Across 62 neurons with statistically significant responses during discrimination and fixation trials, offset latency was significantly shorter on discrimination trials compared with fixation trials (mean Δ ± SD = 47.2 ± 38.6 ms, paired t test, P < < 0.01; Fig. 9A). This is strikingly different from response onset latency, which was found to be independent of task type (see Fig. 7). Interestingly, on average response onset latency was significantly shorter than offset latency during fixation trials (Fig. 9B; mean Δ ± SD = 17.1 ± 37.4 ms, P << 0.01, paired t test), but significantly longer for discrimination trials (Fig. 9C; mean Δ ± SD = 29.9 ± 41.2, P ≪ 0.01). In other words, even though responses have similar onset trajectories for both fixation and RT discrimination tasks, the offset is faster than onset for RT discrimination and slower for fixation. Furthermore, faster offset latencies appear to be unrelated to the behavioral reaction time of the animal: we did not find a statistically significant correlation between animal’s behavioral reaction time and offset latency (r = 0.0013; P = 0.89). The typical disparity between onset and offset latencies for fixation and discrimination tasks is shown for an example neuron in Fig. 9D, top and bottom, respectively).

Fig. 9.

Effect of task type on response offset latency. A: response offset latencies during the discrimination (ordinate) and fixation (abscissa) tasks. Circles and diamonds: neurons with larger average responses on the fixation and discrimination tasks, respectively. Open symbols: neurons with no significant difference between offset latencies in fixation and discrimination tasks. Across the population, there was a sizable and statistically significant mean difference between offset latencies during the fixation and discrimination tasks. Mean Δ ± SD = 47.2 ± 38.6 ms (t test, P < < 0.001, n = 62). B and C: comparison of onset (abscissa) and offset latencies (ordinate) for responses during fixation tasks (B) and discrimination tasks (C). During the fixation task, onset latencies were substantially shorter, while the opposite was true for the discrimination task. We did not find a statistically significant correlation between onset and offset latencies for either fixation or reaction time (RT) behavioral task: Pearson’s correlation coefficients between onset and offset latencies were 0.17 (P = 0.18) for fixation data and 0.014 (P = 0.91) for RT behavior data. D: responses of an example neuron during fixation and discrimination tasks (black and gray, respectively) aligned on stimulus onset (top) and stimulus offset (bottom). As with the population, this example neuron shows no difference in onset latency with task but a sizable difference in offset latency. Shaded area: SE. E: normalized population PSTH for fixation and RT discrimination tasks aligned on stimulus offset (dotted line). Shaded area: SE.

Other Factors

Saccade direction and value.

We asked whether the response onset and offset latency of V4 neurons during discrimination tasks (data set #2; data set #5) depended on whether the animal made a rightward or leftward saccade, and whether the animal made a correct or incorrect saccade. We found no statistically significant difference in onset latency either for saccade direction (rightward vs. left, mean Δ = 0.246 ms shorter, paired t test, P = 0.92, n = 138) or for behavioral performance (correct vs. error, mean Δ = 2.77 ms shorter, P = 0.21, n = 130). This was also true for offset latency (mean Δ = 7.43 ms shorter for rightward saccades, P = 0.07, n = 54; mean Δ = 0.927 longer ms for errors, P = 0.84, n = 55). As the recording sites of both animals involved in this study were located in the left hemisphere, rightward saccades were toward the receptive field of recorded neurons, perhaps explaining the shorter offset latencies.

Fixation trial order.

During the fixation experiments, multiple stimuli were shown in sequence separated by interstimulus intervals of 200 ms. We examined whether the response onset and offset latencies of neurons (data set #4) were different for stimuli that were first, second, or last in the sequence. Stimuli presented later in the trial had slightly longer onset latencies (Fig. 10A; black, on average 1.46 ms longer for second than first stimuli, paired t test, P = 0.09, n = 59; white, on average 2.8 ms longer for last than first stimuli, P = 0.004). On the other hand, offset latency decreased with the increase in the ordinal position of a stimulus within a trial, even though the difference was not significant (Fig. 10B; black, 1.12 ms shorter for second than first stimuli, P = 0.72, n = 57; white, 2.9 ms shorter for last than first stimuli, P = 0.28).

Fig. 10.

Effect of stimulus presentation order on onset and offset latency. Difference in onset (A) and offset (B) latencies of neuronal responses between first versus second (black bars) and first versus last (white bars) stimuli in the trial sequence of the fixation experiment. Gray bars: overlap between black and white bars. Triangles: mean differences. Across the population, there was a statistically significant difference between first and last stimuli in terms of onset but not offset latencies.

Receptive field eccentricity.

We assessed whether there was an effect of the receptive field eccentricity on response latency. The eccentricities in four fixation tasks (data sets #1, #3, #4, and #6) varied from 0.9° to 11.9° (median = 4.11°). There was no correlation between response onset latency and eccentricity of the neuron’s receptive fields for either onset (r = −0.07, P = 0.24, n = 276) or offset latencies (r = −0.08, P = 0.18, n = 272).

Interanimal difference.

The mean latency estimates (±SD) varied between monkeys (monkey O, onset, 81.9 ± 27.98 ms, offset, 85.96 ± 29.2 ms; monkey M, onset, 64.24 ± 17.7 ms, offset, 86.75 ± 31.05 ms; monkey P, onset, 90 ± 27.4 ms, offset, 104.4 ± 42.4 ms). However, the data trends reported above were consistent across animals (see Figs. 2 and 8).

DISCUSSION

We quantified the onset and offset latencies of single V4 neurons under a variety of stimulus and behavioral conditions. Our results demonstrate that onset latency spans a broad range in V4, but for many neurons latency can be quite short: roughly 42% of neurons had latencies shorter than 72 ms. Onset latency distributions have lower limits much shorter than previously reported: many V4 neurons respond with latencies comparable to neurons in MT and FEF. We also found that onset latency was strongly modulated by stimulus size and interior fill but not stimulus shape or partial occlusion, supporting the hypothesis that onset latency is dictated by SF content of the stimulus. These results imply that V4 responses emerge sufficiently early to underlie the rapid recognition of objects thought to be possible within 80–150 ms from stimulus onset (Crouzet et al. 2010; Kirchner and Thorpe 2006; Thorpe et al. 1996). On the other hand, in striking contrast to results from LGN, V1, and MT, responses of many V4 neurons lingered up to 100 ms or longer after stimulus offset but this was significantly shortened when stimulus offset was triggered by a planned saccadic eye movement. Such extended maintenance of V4 responses after the withdrawal of feedforward inputs may be critical for higher order processes that facilitate object recognition and memory.

Faster Response Onsets Than Previously Reported in V4

Schmolesky et al. (1998) previously reported that response latency ranged from ~70–159 ms in V4. While their reported upper limit is consistent with our own measurements their reported lower limit is not. Specifically, 271/276 neurons had latencies <159 ms but roughly 42% displayed response latencies shorter than 72 ms. The median of our distribution, consistent with the measurements of Boch and Fischer (1983) and Mysore et al. (2006) for luminance-defined stimuli, was essentially the lower limit of Schmolesky et al. (1998). This substantial discrepancy between our own results and theirs may be due to several factors: size of the data set, anesthesia, stimuli and latency computation. First, our results are based on a larger data set: 276 neurons in 3 monkeys versus their 29 neurons in 1 monkey. Previous results (Maunsell and Gibson 1992; Schroeder et al. 1998) and our own analyses (see above) indicate that onset latency can vary across animals. Thus our larger data set, both in terms of number of animals and neurons, may provide a better estimate of the true latency distribution of V4 neurons. Second, Schmolesky et al. (1998) recorded V4 responses in an anesthetized animal, while all our responses were recorded from awake behaving animals. Anesthesia may suppress responses in extrastriate areas (Girard et al. 1992; Lamme et al. 1998) and may change response dynamics (Mody et al. 1991; Uhl et al. 1980). Third, we studied V4 responses with a variety of complex shape stimuli known to drive V4 responses robustly and customized the stimulus color and luminance for each neuron. Stimuli were also scaled to occupy ~80% of the RF area. In contrast, Schmolesky et al. (1998) studied responses with white and black bars or spots. While stimulus shape does not influence V4 onset latency as long as the evoked responses are significantly above baseline, stimuli with higher SF content, small area, or low contrast are associated with longer latencies (Chen et al. 2007). Finally, the metric used to quantify latency may also underlie the observed differences in our results. A variety of methods have been used in the literature to estimate response latency (for a review, see Levakova et al. 2015). Instead of the Poisson spike train analysis of Hanes et al. (1995) employed by Schmolesky et al. (1998), we chose to use the half-height technique (Gawne et al. 1996; Lee et al. 2007; Sundberg et al. 2012). The advantages of this choice are in its invariance to response magnitude, its statistical reliability, and robustness in estimating latency even when responses are sparse. Moreover, computer simulations confirm that Poisson spike train analysis sometimes does not produce any latency estimate, unlike other techniques (Friedman and Priebe 1998; Lee et al. 2010).

Consistent Relationship between Onset Latency, SF Content, and Stimulus Contrast

It has been previously argued that the onset latency provides information about the visual stimulus independent of what is encoded in the response magnitude (Reich et al. 2001; VanRullen et al. 2005). For example, in V1, neurons exhibit a consistent relationship between onset latency and stimulus luminance contrast, i.e., shorter latencies for higher contrast, regardless of the orientation preference of the neuron, which may be encoded by the response magnitude (Gawne et al. 1996; Gawne 2000; Oram 2010; Richmond et al. 1997). Similarly, for equiluminant stimuli, V4 neurons exhibit shorter onset latencies for stimuli of greater chromatic contrasts regardless of the chromatic preferences encoded by the response magnitude (Bushnell et al. 2011). V1 neurons also exhibit shorter latency for low SF compared with high SF stimuli (Bair and Movshon 2004). Our findings lend support to this argument: we found that onset latencies of V4 responses provide a consistent measure of stimulus size and stimulus fill but are independent of stimulus shape. V4 response magnitudes, on the other hand, are dictated by stimulus form. Essentially, response magnitude may be stronger for smaller or larger stimuli, and for filled or outline stimuli, but response latency has a consistent relationship with these stimulus attributes: always longer for smaller sizes and outline, i.e., unfilled, stimuli. We also found no significant onset latency difference for shapes with and without occluders. This latter result is unlike what has been reported previously in inferior temporal (IT) cortex, where neurons exhibit significantly longer latencies in response to occluded stimuli (Kovács et al. 1995).

Longer Offsets Than Onsets in V4

Previous studies have demonstrated that offset latency of neuronal responses in monkey LGN, V1, and MT is shorter than onset latency by tens of milliseconds (Bair et al. 2002; Smith et al. 2006). This is also true in cat LGN (Coenen et al. 1972; Humphrey and Weller 1988; Mastronarde 1987; Saul and Humphrey 1990) and cat cortex (Von Baumgarten and Jung 1952) and based on VEP signals in human observers (Kreegipuu and Allik 2007). Offset latencies are also less variable than onsets and independent of the visual stimulus, and discrimination thresholds for event offsets are lower than for onsets in human subjects (Tadin et al. 2010). As a result, offset latencies are considered to provide the most reliable measure of the first arrival of excitatory signals to a neuron (Bair et al. 2002) and are thought to represent a change in the visual environment rather than stimuli themselves (Tadin et al. 2010). This, however, is fundamentally different from what we observed in V4. During passive fixation, V4 offset latencies were on average 10–20 ms longer than onset latencies. Thus, when a stimulus is replaced by the background gray screen, V4 responses linger for a period of time, even after feedforward signals are withdrawn. There was no statistically significant relationship between the level of the sustained response before stimulus offset and the ensuing latency. In our data set, ~14% of neurons exhibited an excitatory transient response at stimulus offset; the mean latency across the population was no different even when these neurons were excluded (88.6 ms). These results suggest a role for recurrent circuits in the maintenance of V4 responses in a state of dynamic equilibrium. If a stimulus was replaced by another stimulus, rather than background gray, it is conceivable that V4 responses would be shut down faster, but this was not tested. The long offset latencies observed here appear consistent with those in IT cortex based on the PSTHs presented in several prior publications (e.g., Komatsu et al. 1992; Matsumora et al. 2008). It is possible that the longer offsets we observed in V4 compared with V1 and MT (Bair et al. 2002) maybe due to top-down influences that modulate V4 but not V1 or MT at least in the anesthetized animal. PSTHs presented in prior publications (e.g., Hegdé and Van Essen 2004; Knierim and Van Essen 1992) qualitatively support this possibility, but rigorous latency estimates with precise control of stimulus timing would be needed to draw conclusions. Additional experiments that estimate V1 and V2 offset latencies in the awake animal are needed to determine whether longer offsets in mid- and high-level processing stages are due to distinct mechanisms for response maintenance that may be important for short-term memory and object recognition.

Shorter Offsets with Saccade Initiation

Consistent with previous studies, we found that active engagement in a behavioral task (as compared with passive fixation) did not influence onset latencies in V4 (Gee et al. 2010). However, offset latencies were dramatically shortened: V4 offset latencies were ~50 ms shorter when stimulus offset was triggered by saccade initiation rather than passive offset. This resulted in considerably shorter offsets than onset latencies (~30 ms shorter) during behavioral tasks. It is unlikely that faster offset latencies during behavior are due to faster sensory integration, since onset latencies are unaffected. Rather, faster offset latencies observed in the RT task most likely reflect a change in V1 inputs, since offset latencies are shorter than onset latencies just like in V1 (Bair et al. 2002). Faster offsets could also relate to predictive receptive field remapping as the animal’s eyes move (Bichot et al. 2005; Ghose and Maunsell 2002; Hartmann et al. 2017; Neupane et al. 2016; Rolfs et al. 2011; Steinmetz and Moore 2010; Tolias et al. 2001), saccadic suppression thought to be a perceptual consequence of efficient sensorimotor estimation (Crevecoeur and Kording 2017), or a resetting or clearing of visual responses with a change in environment due to a saccade, perhaps related to top-down influence. This may be fundamentally different than passive disappearance of a visual stimulus, which may occur when a stimulus travels out of the receptive field or becomes invisible due to occlusion. Consistent with these hypotheses, we found that the offset latency on the non-RT task where the stimulus disappeared as the animals continued to maintain fixation was more comparable to the fixation task: the mean latency ± SD in the non-RT task was 103.1 ms ± 33.6 ms (median 106 ms; n = 77). However, because of the longer stimulus duration in the non-RT task (600 ms vs. ~250 ms in the RT and fixation experiments), additional experiments are needed to rule out the possibility that the longer latencies were caused by longer stimulus duration.

Implications for Processing and Circuitry

The response latency of any neuron is a function of its anatomical connectivity, and the broad range of onset latencies observed in our data set suggests a great diversity in the input patterns across V4 neurons. Neurons in the left tail of the onset latency distribution respond soon after stimulus onset, and this observation is consistent with demonstrations that individual V4 neurons receive both magnocellular and parvocellular input (Ferrera et al. 1992; Ninomiya et al. 2011). However, our data set includes many neurons with very long latencies (>100 ms). For these neurons, the primary driver of responses may be recurrent connections within V4 and/or feedback projections from other areas (Chen et al. 2007; Felleman and Van Essen 1991; Markov et al. 2014; Ninomiya et al. 2011, 2012; Ungerleider et al. 2008) rather than feedforward input directly from V1/V2.

A broad range of onset latencies also hints at diversity in function: neurons occupying the left and right tails of the latency distribution may play different roles in visual processing. Specifically, neurons with short latencies may be important for ultrafast object recognition. Past studies have demonstrated that human subjects can detect animals in natural scenes very rapidly, possibly based on only ~95–100 ms of processing (Kirchner and Thorpe 2006). While few neurons in IT cortex have initiated their response by this time (Brincat and Connor 2006; Kiani et al. 2005), our results suggest that in 82% of V4 neurons responses are well underway and these signals likely contribute to this rapid decision making (Thorpe et al. 1996). On the other hand, neurons occupying the right tail of the distribution may be more important for gating sensory signals relevant for behavior. Responses of these neurons emerge well after stimulus onset, and typically there is minimal distinction between the transient and sustained portions in terms of magnitude (data not shown). Further experiments are needed to understand how feedforward and feedback signals inform these responses and how these responses influence behavior.

GRANTS

This work was funded by National Eye Institute (NEI) Grants R01-EY-018839 an R01-EY-029997 (to A. P.); NEI Center Core Grant for Vision Research P30-EY-01730 to the University of Washington; National Institutes of Health (NIH)/Office of Research Infrastructure Programs (ORIP) Grant P51-OD-010425 to the Washington National Primate Research Center; University of Washington (UW) Vision Training Grant (NEI Grant T32-EY-007031) and National Science Foundation (NSF) Graduate Research Fellowship Program (GRFP) Grant DGE-1256082 (to D.V.P.); and UW Computational Neuroscience Training Grants (NIH Grants T90-DA-032436 and 5R90-DA-033461-05; to D.V.P. and P.Z).

DISCLOSURES

No conflicts of interest, financial or otherwise, are declared by the authors.

AUTHOR CONTRIBUTIONS

P.Z. and A.P. conceived and designed research; P.Z. analyzed data; P.Z. and D.V.P. prepared figures; P.Z., D.V.P., and A.P. drafted manuscript; P.Z., D.V.P., and A.P. edited and revised manuscript; P.Z., D.V.P., and A.P. approved final version of manuscript.