Neural Correlates of Fine Depth Discrimination in Monkey Inferior Temporal Cortex

Abstract

Binocular disparity is an important visual cue that gives rise to the perception of depth. Disparity signals are widely spread across the visual cortex, but their relative role is poorly understood. Here, we addressed the correlation between the responses of disparity-selective neurons in the occipitotemporal (ventral) visual pathway and the behavioral discrimination of stereoscopic depth. We recorded activity of disparity-selective neurons in the inferior temporal cortex (IT) while monkeys were engaged in a fine stereoscopic depth discrimination (stereoacuity) task. We found that trial-to-trial fluctuations in neuronal responses correlated with the monkey's perceptual choice. We suggest that disparity signals in the IT, located in the ventral visual pathway, are functionally linked to the discrimination of fine-grain depth.

Introduction

Neural signals regarding specific stimulus parameters are generally widely spread across multiple areas in the monkey brain. In binocular vision, neurons selective for binocular disparities are found in many areas including the primary visual cortex (V1) and areas within the occipitoparietal (dorsal) visual pathway and the occipitotemporal (ventral) visual pathway (Hubel and Wiesel, 1970; Poggio and Fischer, 1977; Maunsell and Van Essen, 1983; Burkhalter and Van Essen, 1986; Felleman and Van Essen, 1987; Poggio et al., 1988; Roy et al., 1992; Eifuku and Wurtz, 1999; Janssen et al., 1999; Taira et al., 2000; Uka et al., 2000; Hinkle and Connor, 2001, 2005; Prince et al., 2002; Watanabe et al., 2002; DeAngelis and Uka, 2003; Tanabe et al., 2005). The specific roles that these multiple areas play in stereopsis are poorly understood.

In this study, we determined whether there is a correlation between the responses of disparity-selective neurons in the ventral visual pathway (Janssen et al., 1999; Uka et al., 2000; Hinkle and Connor, 2001, 2005; Watanabe et al., 2002; Tanabe et al., 2005) and the discrimination of fine-grain depth by monkeys. A recent study showed that disparity-selective neurons in the inferior temporal cortex (IT) are sensitive to small changes in disparity, especially near the plane of fixation (Janssen et al., 2000). This implies that disparity-selective IT neurons might actually be involved in discriminating fine-grain depth, although this has not been directly tested yet. We therefore recorded extracellular activity from disparity-selective neurons in the IT while monkeys performed a fine stereoscopic depth discrimination task. We measured whether trial-to-trial fluctuations in neuronal responses correlated with trial-to-trial variations in the monkey's perceptual choice. Our results show that this is indeed the case, suggesting that disparity signals in the IT, located in the ventral visual pathway, are functionally linked to the discrimination of fine-grain depth.

Materials and Methods

Two Japanese monkeys (Macaca fuscata) were used in the experiments. Experimental procedures for single-unit recordings were described in detail previously (Uka et al., 2000). All animal care and experimental procedures were in accordance with National Institutes of Health guidelines and were approved by the animal experiment committee at Osaka University.

Task and visual stimulus. Monkeys were trained on a fine-depth discrimination (stereoacuity) task. They fixated a gray spot (0.2 × 0.2°) over a black background on a 15 or 21 inch color monitor placed 57 cm in front of the eyes. The fixation point was located at the center of the monitor where one of nine shapes (see Fig. 1a) was presented in red. The fixation point was always visible during stimulus presentation. The monkeys were trained to maintain their fixation within 2.0 × 2.0° while the shape was presented for 1 s and to make a saccade toward one of two targets appearing right and left after a 500 ms delay. Rightward saccades were required for crossed (“near”) disparity cues, whereas leftward saccades were required for uncrossed (“far”) disparity cues. The monkeys performed the task at the following horizontal disparity levels: monkey 1, ±0.1°, ±0.05°, ±0.025°, ±0.01°, ±0.005°, ±0.0025°, ±0.00125°, and 0°; monkey 2, ±0.14°, ±0.07°, ±0.035°, ±0.014°, ±0.007°, ±0.0035°, and 0°. Stereo presentation was achieved using a liquid crystal stereoscopic modulator (SGS610; Tektronix, Beaverton, OR), and sub-pixel resolution was achieved using anti-aliasing methods. Each disparity was presented randomly interleaved. Correct responses were rewarded with a drop of water. Zero-disparity trials were rewarded randomly. The task was aborted if a monkey broke his fixation during stimulus presentation.

Figure 1.

Visual stimulus and the stereoacuity task. a, In each experiment, one of nine shapes that elicited the best response at zero-disparity was used in the task. Scale bar, 3°. b, The task schedule. The dotted square shows the fixation window, which was not actually visible. Monkeys were trained to make a leftward saccade for uncrossed (far) disparities and a rightward saccade for crossed (near) disparities. c, Behavioral performance from both monkeys. The proportion of far choices is plotted as a function of horizontal disparity. In addition to binocular (•) presentations, monocular (▵, left eye; ▿, right eye) presentations were tested. deg, Degree. d, Histological reconstruction of the recording site in monkey 1. Monkey 2 was still used in a different experiment, but the recording chamber was placed in a similar position. st, Superior temporal sulcus; amt, anterior middle temporal sulcus.

Because the solid shapes presented to each eye were shifted horizontally to create binocular disparities, the shift in monocular images could have been used by the monkeys as the cue to perform the stereoacuity task. We therefore performed a monocular control session in both monkeys, in which they were presented with each monocular image separately. For monkey 1, rewards were given the same way as in binocular trials (i.e., rightward saccades were required for rightward shifts in the left eye-only condition and leftward shifts in the right eye-only condition; leftward saccades were required for leftward shifts in the left eye-only condition and rightward shifts in the right eye-only condition). For monkey 2, rewards were given randomly for each monocular presentation.

Cross talk was 10% for the left-eye image to the right eye, and there was no measurable cross talk for the right-eye image to the left eye (this was true for both monocular and binocular presentation). Cross talk may have affected the precise disparity applied. In particular, the position of the left-eye image may have been shifted slightly toward the right-eye image. In this case, the real disparity would be slightly smaller than what we intended it to be. Thus, we may have underestimated the monkeys' psychophysical performance. Cross talk should not have affected choice probabilities (described below) because they do not depend on the exact amount of disparity.

Experimental protocol. Recordings were made in areas TE and TEO of the IT, from three cerebral hemispheres in the two monkeys (see Fig. 1d). We did not use any physiological criteria to distinguish between areas TE and TEO, but the histology shown in Fig. 1d indicates that most of the recordings were anterior to the posterior mid-temporal sulcus. Thus, most of the recordings were from area TE, although some may have been from area TEO. Recording electrodes were advanced from the side of the skull. Extracellular activities from single neurons were recorded using a tungsten microelectrode (impedance, 2-3 MΩ at 1 kHz). For each recorded neuron, the most effective shape was first determined from nine different shapes (see Fig. 1a), while the monkey was fixating. Only neurons that responded to one or more shapes were further tested. Binocular disparity was then added to the most effective shape at nine horizontal disparities (-0.8 to +0.8° at 0.2° step for monkey 1; -1.12 to +1.12° at 0.28° step for monkey 2), each presented 10 times in random sequence, to obtain a disparity-tuning curve. We next recorded while the monkey performed the stereoacuity task. Data were acquired for 10-105 trials for each disparity (25th percentile, 21 trials; median, 42 trials; 75th percentile, 69 trials) with usually twice as many numbers of trials for zero disparity. The stereoacuity task was performed regardless of whether the neuron was tuned for disparity.

Eye movements. The position of both eyes was monitored using the search coil technique (Judge et al., 1980) at a sampling rate of 100 Hz for monkey 1 and 1000 Hz for monkey 2. There was no correlation between the disparity of the shape figure and the monkey's vergence angle for most cases (38 of 39 in monkey 1; 42 of 55 in monkey 2; p > 0.05) while the monkeys were performing the stereoacuity task. Thus, the monkeys' vergence was anchored to the fixation point and did not track the disparity of the target shape. Results did not change when data were restricted to the 80 neurons with no systematic errors in vergence eye positioning (i.e., the neurons with no correlation between disparity and vergence angle). The accuracy of the vergence measurements was as follows: the within-trial SD was 0.07°, and the across-trial SD was 0.33°.

Data analysis. We recorded from a total of 94 neurons (39 in monkey 1 and 55 in monkey 2). Responses to visual stimuli were assessed by counting spikes starting 80 ms after stimulus onset and ending 80 ms after stimulus offset. We analyzed neurons that showed a preference for near or far disparity in the disparity-tuning run after responses were combined across different disparities with the same sign (t test, two-tailed; p < 0.05). Fifty-seven neurons (25 in monkey 1 and 32 in monkey 2) were selected for additional analysis based on this criterion. We did encounter some tuned excitatory (11 of 94) and tuned inhibitory (3 of 94) cells. These cells were excluded from the analysis because they were not likely to convey any information about near or far.

To calculate how well the responses of a neuron predicted the monkey's perceptual choice (choice probability), we used receiver-operating characteristic (ROC) analysis (Green and Swets, 1966; Britten et al., 1996). First, responses at each disparity were z-scored (i.e., normalized so that mean = 0 and SD = 1) separately to isolate trial-to-trial variability. For each neuron, z-scored responses at all disparities, except for disparity levels at which the monkey did not make at least one-quarter of choices to each target, were combined; these disparity levels were excluded from the analysis, because they would generate inaccurate measurements of choice probabilities. An ROC curve was generated by plotting the proportion of preferred choice trials on which the response exceeded a criterion against the proportion of null choice trials on which the response exceeded that criterion. The area under the ROC curve determined the probability that the neuron predicted the monkey's perceptual response. Significant deviation from chance level (choice probability of 0.5) was determined using a permutation test. Choice probabilities were calculated after the relationship between responses and choices was permuted randomly. The p value was given as the proportion of choice probabilities at which the deviation from 0.5 exceeded that of the original choice probability from 0.5.

Results

To address whether the responses of IT neurons are linked to the behavioral discrimination of stereoscopic depth, we recorded extracellular neuronal activity from single IT neurons in two alert monkeys performing a stereoacuity task. The monkeys were trained to report perceived depth of a solid shape figure relative to the fixation point by making a saccade to one of two targets (Fig. 1a,b). A range of small horizontal disparities was applied to the figure to span the monkey's psychophysical threshold. Because the solid shapes presented to each eye were shifted horizontally to create binocular disparities, the shift in monocular images could have been used by the monkeys as the cue to perform the task. We therefore determined whether monkeys used these monocular cues by letting the monkeys discriminate while presenting each monocular image separately. Monkeys did not discriminate the monocular cues available in the visual stimulus because their performance did not depend on stimulus position when presented with the left- or right-eye image alone (Fig. 1c). The proportion of far choices between far-disparity trials and near-disparity trials was significantly different in the binocular condition (t test; p < 0.001 for both monkeys) but not in the monocular condition (p = 0.09 and p = 0.50 for the left-eye and right-eye conditions, respectively, in monkey 1; p = 0.32 and p = 0.30 for the left-eye and right-eye conditions, respectively, in monkey 2).

Covariation of neuronal responses with behavioral choice

We recorded from 94 neurons in the lateral surface of the IT (Fig. 1d) and analyzed 57 neurons that had a preference for either near or far disparity. To assess whether monkeys made use of disparity signals in the IT to form their judgment about depth, we sought the relationship between responses of disparity-selective IT neurons and the monkey's perceptual choice. Figure 2 shows an example of the analysis that we performed. We first obtained a disparity-tuning curve during a fixation task, and the neuron responded more strongly to near disparities than to far disparities (Fig. 2a). Next, we recorded neuronal activity while the monkey performed the depth discrimination task. Figure 2b shows trial-to-trial responses of the example neuron during visual stimulation along with the monkey's near or far behavioral choices (rightward or leftward saccades, respectively) for zero-disparity trials. On these trials, the monkey was forced to “guess” the depth of the stimulus, because no disparity cues were applied. Surprisingly, the monkey made more near choices when the responses of the neuron were higher than the mean and more far choices when the responses of the neuron were lower than the mean. Thus, on zero-disparity trials, the monkey behaved as if he had seen depth toward the preferred (near) disparity of the neuron when the neuron responded with a relatively high firing rate and behaved as if he had seen depth toward the null (far) disparity when the neuron responded with a relatively low firing rate. The responses of the neuron during presentation of the visual stimulus predicted the monkey's perceptual choice made after the stimulus presentation period and a subsequent delay. This was true at all disparity levels (Fig. 2c).

Figure 2.

Example disparity tuning and the relationship between neuronal responses and the monkey's choice. a, Discharge rate (mean ± SEM) of 10 presentations is plotted as a function of horizontal disparity. Data points are connected using a spline fit. deg, Degree. b, Trial-to-trial responses and the monkey's subsequent choices (•, near choice; ○, far choice) are plotted for zero-disparity trials. c, Response magnitude histograms for near (▪) and far (□) choice trials are superimposed for seven disparities (shown on the right). The choice probabilities calculated for each disparity (shown on the left) were 0.55-0.70.

To quantify this effect, we used ROC analysis (Green and Swets, 1966) to calculate the probability of predicting the monkey's future behavioral choice given the firing rate of a neuron during stimulus presentation [referred to as choice probability (Britten et al., 1996)]. A choice probability of 1.0 indicates that a neuron has 100% power for predicting the monkey's future choice, whereas a choice probability of 0.5 indicates that a neuron does not predict the monkey's choice (chance level). Choice probability for the zero-disparity trials (Fig. 2b) was 0.61, indicating that an ideal observer can predict the monkey's choice 61% of the time just by listening to the responses of the neuron. Figure 2c shows that choice probability was >0.5 at non-zero disparities as well.

To show that choice probabilities are independent of disparities at the population level, Figure 3a illustrates the average choice probability (±1 SD) from all 57 neurons as a function of binocular disparity, in which positive and negative values correspond to disparities presented closer to the preferred and null disparities, respectively. Consistent with Figure 2c, choice probabilities do not depend on binocular disparities (ANOVA; monkey 1, p = 0.39; monkey 2, p = 0.91), indicating that IT responses are larger when the monkey's choice corresponds to the preferred disparity of the neuron, regardless of stimulus strength.

Figure 3.

Choice probabilities do not depend on visual stimulus parameters. a, The average choice probability across 57 neurons is plotted as a function of signed binocular disparity (positive and negative values correspond to stimuli at the preferred and null disparities, respectively). Disparity levels at which the monkey made choices to one target >75% of the time were excluded from the analysis. Error bars indicate SDs. Points with no error bars indicate those calculated from one data. deg, Degree. b, Choice probabilities calculated from responses to the preferred disparity are plotted against those for the null disparity. Responses of each neuron to each disparity were normalized using z scores (subtracting the mean response and dividing by the SD), and the normalized responses were then combined across preferred and null disparities separately to compute choice probabilities. Disparity levels at which the monkey made choices to one target >75% of the time, as well as zero-disparity trials, were excluded from the analysis. There is one datum for each of 55 neurons. Two cells were discarded because no disparity levels passed the criteria described above for either the preferred or null disparity.

The independence of choice probabilities on stimulus disparity also holds true on a neuron-by-neuron basis. Figure 3b shows a scatter plot of choice probabilities calculated for disparities presented closer to the preferred versus null disparity of each individual neuron. There is a strong correlation between the two (r = 0.77; p << 0.001; n = 55), with no significant tendency for the data to be biased away from the diagonal (paired t test; p = 0.40).

The finding that choice probabilities do not depend on stimulus parameters allows us to combine responses across stimulus conditions to obtain a single choice probability for each neuron. To derive a single choice probability, responses at each disparity (excluding some with few error trials; see Materials and Methods) were z-scored (i.e., normalized so that mean = 0 and SD = 1) and combined to calculate a “grand” choice probability (Britten et al., 1996). The grand choice probability for the neuron in Figure 2 was 0.62, which is significantly greater than 0.5 (permutation test; p < 0.001). Thirty-five of the 57 neurons had choice probabilities significantly different from 0.5 (permutation test; p < 0.05), with the majority (28 of 35) being >0.5 (see top frequency distribution in Fig. 4). On average, the grand choice probability of the 57 neurons was 0.56 (monkey 1, 0.58; monkey 2, 0.54), a value statistically significantly greater than 0.5 (sign test; p < 0.001). This indicates that one can predict the monkey's behavioral choice in this task by listening to a population of disparity-selective IT neurons.

Figure 4.

The dependence of vergence angle on choice probabilities. a, Vergence-corrected choice probabilities are plotted against original grand choice probabilities (n = 57). Frequency histograms for each dimension are plotted on the top and right. Filled bars represent cells with choice probabilities significantly different from 0.5 (permutation test; p < 0.05). b, Grand choice probabilities calculated from vergence angles are plotted against those calculated from neuronal responses. CP, Choice probability.

To reassure that the large choice probabilities we found were not artifactual, we also analyzed the grand choice probability for 20 neurons that were not selective for disparity (ANOVA; p > 0.05). Seventeen neurons that passed an ANOVA but did not have a preference for near or far based on our t test criteria were excluded from this analysis. Because there is no basis for assigning the preference of these neurons, we folded the distribution around 0.5 and reported values as >0.5. Although the largest value was 0.75, only 6 of 20 choice probabilities were significantly >0.5 (permutation test; p < 0.05). The ratio of significant choice probabilities (30%, 6 of 20) is smaller than what we found for neurons with near or far preference (61.4%, 35 of 57; p = 0.015; test of the difference between two proportions).

Control of vergence posture

Systematic errors in vergence eye positioning can potentially produce large choice probabilities. If neuronal responses correlated with vergence angle and vergence angle systematically changed with the monkey's choice, then we might expect a secondary correlation between neuronal responses and behavioral choice, potentially leading to a large choice probability. We corrected for potential vergence artifacts as follows. We fitted a regression line between neuronal responses and mean vergence angle (calculated starting 80 ms after stimulus onset and ending 80 ms after stimulus offset for each trial) at each disparity and extracted the residual neuronal responses that do not depend on vergence angle. Vergence-corrected choice probabilities were calculated from these residual responses. This analysis removes any linear relationship between vergence angle and neuronal responses and thus isolates the relationship between neuronal responses and behavioral choice. In Figure 4a, vergence-corrected choice probabilities are plotted against original choice probabilities for the 57 neurons. No difference was observed between the two values (paired t test; p = 0.30), indicating that vergence artifacts were not the source of the large choice probabilities.

We also tested the relationship between vergence angle and behavioral choice by calculating choice probabilities using vergence angles instead of neuronal responses. If neurons were responding to the absolute disparity of the stimulus, divergence would enhance and convergence would suppress the responses of near cells. Thus, large choice probabilities may arise if monkeys diverged on near (preferred) choices and converged on far (null) choices. For far cells, convergence would enhance and divergence would suppress the responses of the neurons. Thus, large choice probabilities may arise if monkeys converged on far (preferred) choices and diverged on near (null) choices. In both cases, large choice probabilities may artificially arise if monkeys converged on far choices and diverged on near choices. We therefore calculated choice probabilities using vergence angles to determine whether this was the case. Average vergence angles were calculated for each trial and sorted by choice (near or far) at each disparity. Convergence was assigned a positive value, and divergence was assigned a negative value. The values were then used to calculate choice probabilities after z-scoring at each disparity. Here, we calculated the probability that far choices have larger values; thus, choice probabilities would be >0.5 if the monkeys happened to make far choices when converging and near choices when diverging. The distribution of choice probabilities using vergence angles is shown in Figure 4b. The average (0.50) was not significantly different from 0.5 (sign test; p = 0.99), although 14 of 57 choice probabilities were statistically significant (permutation test; p < 0.05). Furthermore, there was no correlation between the original choice probabilities and those using vergence angles (r = -0.008; p = 0.95). Thus, there was no systematic relationship between vergence angle and the monkeys' behavioral choice, indicating that the original choice probabilities were not attributable to systematic errors in vergence eye positioning.

Time course of choice probabilities

If the monkey's behavioral choice in this task is based on the responses of disparity-selective IT neurons, we might expect responses for different choices to diverge early in time. We therefore analyzed the time course of predictive activity. Here, we concentrated on the 28 neurons with choice probabilities >0.5 (permutation test; p < 0.05). Figure 5a shows the time course of responses to disparities regardless of choice. For large disparities (0.1 or 0.14°), responses to preferred disparities diverge from responses to null disparities immediately after the initial transient response, although the response difference between preferred and null disparities becomes more prominent in the latter half of the response. For intermediate disparities (0.025 or 0.035°), response difference between preferred and null disparities become prominent only after 320 ms. Figure 5b shows the time course of responses to preferred and null choices for zero-disparity trials in addition to the time course of responses to large disparities. Interestingly, the time course of responses for preferred and null choices at zero disparity diverges only after 360 ms. This is similar to the time course of responses to preferred and null intermediate disparities, albeit slightly slower. Also note that there is no response during the delay period; thus, responses of these IT neurons do not represent “memory” of depth information per se. We also analyzed the time course of vergence eye position. Vergence angle was not different depending on the choice that the monkeys made (Fig. 5c). Moreover, vergence was constant throughout the trial and did not drift with time. This provides additional evidence that the large choice probabilities observed are not attributable to changes in vergence eye position.

Figure 5.

Time course of IT responses. Responses were binned into 20 ms bins and averaged across trial repetitions. For each neuron, responses for all disparities were then normalized to the peak-firing bin. Normalized firing rates were then averaged across the 28 neurons with choice probabilities significantly greater than 0.5. a, Mean normalized poststimulus time histograms (PSTHs) are shown for five different disparities. All trials at each disparity were included regardless of the monkey's choice. Stim, Stimulus. b, Mean normalized PSTHs are shown separately for preferred (red) and null (blue) choices at zero disparity. PSTHs for large disparities (solid trace, preferred; dashed trace, null) are shown as well.c, Time course of vergence angle sorted by choice (red, preferred; blue, null) for zero-disparity trials. The sign of vergence was corrected for each neuron so that positive values indicate convergence for near neurons and divergence for far neurons. Error bars indicate SEMs across the 28 neurons. deg, Degree.

Discussion

The distinction between the ventral and dorsal visual pathways (Ungerleider and Mishkin, 1982) cannot be based on the existence of disparity-selective neurons, because both pathways contain neurons sensitive for binocular disparity (Maunsell and Van Essen, 1983; Roy et al., 1992; Eifuku and Wurtz, 1999; Janssen et al., 1999; Taira et al., 2000; Uka et al., 2000; Hinkle and Connor, 2001, 2005; Watanabe et al., 2002; DeAngelis and Uka, 2003; Tanabe et al., 2005). The mere existence of these neurons does not reveal the functional role that these pathways play (Parker and Newsome, 1998). Therefore, it is important to investigate how disparity-selective neurons in the two pathways differ in the functional roles that they subserve. Here, we provided evidence for a link between neuronal activity in the IT and fine-depth discrimination, an ability to localize the depth of an object in space, by showing a trial-by-trial relationship between the activity of single neurons and behavioral performance. The average choice probability was substantially >0.5, and this effect could not be explained by vergence eye movements. Data are consistent with the interpretation that disparity-selective IT neurons convey relevant sensory information for fine discrimination of depth that is used for the execution of the subsequent saccade, although it is possible that responses are modulated depending on choice via signals from other areas after a decision is made.

Source of choice probability in this study

Before concluding that choice probabilities reflect a functional linkage between IT responses and behavioral choices, we consider some alternative explanations. One explanation is that spatial attention modulates both the activity of IT neurons and the monkey's choice and that this introduces a secondary correlation independent of a visual stimulus. As discussed by Uka and DeAngelis (2004), however, the sign of effect of choice as a result of spatial attention would depend on the polarity of stimulus disparity (preferred or null). For a stimulus closer to the preferred disparity, increased attention would tend to produce a larger neuronal response and a preferred choice (thus, choice probability >0.5). In contrast, increased spatial attention to a stimulus closer to the null disparity would tend to produce a larger neuronal response and a null choice (thus, choice probability <0.5). The data in Figure 3b clearly show that choice probabilities do not depend on the polarity of disparity, thus ruling against spatial attention as a source of choice probabilities.

Another possibility, as discussed by Uka and DeAngelis (2004), is that choice probabilities reflect a choice bias. For example, if the monkey expects a near disparity to appear, this might selectively enhance the activity of near-tuned neurons and increase the probability of an eventual near choice. The lack of significant choice probabilities in the prestimulus period, together with the slow rise in choice probability after stimulus onset (Fig. 5b), argues against an explanation based on choice bias. As discussed by Parker et al. (2002) and Uka and DeAngelis (2004), however, choice bias signals elsewhere in the brain could possibly give rise to choice probabilities in the form of featural attention by changing the response gain of IT neurons after stimulus onset.

Two basic models have been discussed previously as a source of choice probabilities. The “bottom-up” model assumes that neurons with high choice probability provide sensory input to a decision mechanism, such that trial-to-trial variability in IT responses drives trial-to-trial fluctuations in behavioral choices (Britten et al., 1996; Shadlen et al., 1996). This interpretation is favored for data from area MT in direction and depth discrimination tasks (Britten et al., 1996; Uka and DeAngelis, 2004) The “top-down” model assumes that the activity of decision-related neurons elsewhere in the brain is fed back to modulate IT responses. This interpretation is favored for data from area MT in a bi-stable structure-from-motion task (Dodd et al., 2001; Parker et al., 2002)

A critical piece of data in distinguishing the two possibilities involves the time course of choice probabilities. The data in Figure 5b show that choice probabilities appear 360 ms after onset of the visual stimulus, reaching a plateau ∼500 ms after stimulus onset. In interpreting these results, it is important to consider the time when the monkeys made their decisions. Although we do not know precisely when the monkeys made their decisions, it is possible that their decisions were made early in the trial, considering that the monkeys do not benefit much from integrating stimulus information over time (i.e., relevant stimulus information is prominent from the beginning of stimulus onset). Thus, if the monkeys were making their decisions early in the trial, choice probabilities could have emerged after the monkeys made their decisions.

The time course of choice probability in this study is considerably slower than what is found in area MT for direction and depth discrimination tasks (Britten et al., 1996; Uka and DeAngelis, 2004) and also for the bi-stable structure-from-motion task (Dodd et al., 2001): choice probabilities appear right after the start of stimulus response in these tasks, although there is a slight increase in choice probability over time in the bi-stable structure-from-motion task. This may possibly be attributable to a stronger contribution of top-down signals in our task compared with other studies in MT, although it could also be related to differences in local circuitry between area MT and the IT. Everything considered, the top-down model seems to be a more likely explanation for our results than the bottom-up model (in reference to other studies), but we cannot exclude the bottom-up model based on our data.

The role of disparity-selective neurons in the ventral visual pathway

Recent studies have shown the existence of disparity-selective neurons in the ventral visual pathway (Janssen et al., 1999; Uka et al., 2000; Hinkle and Connor, 2001, 2005; Watanabe et al., 2002; Tanabe et al., 2005). We have now shown the functional relevance of disparity-selective IT neurons in a depth discrimination task. It is also known that neurons in area MT (an area in the dorsal visual pathway) play an important role in stereopsis. Several studies have shown the involvement of area MT in disparity-related discrimination tasks (Bradley et al., 1998; DeAngelis et al., 1998; Dodd et al., 2001; Grunewald et al., 2002; Uka and DeAngelis, 2003, 2004).

Is there any difference between disparity processing in the ventral versus dorsal visual pathway? If there is, what is the critical difference? One thought is that the ventral pathway is involved in more perceptual aspects of disparity processing compared with the dorsal pathway (Neri, 2005). Two lines of evidence support this hypothesis. First, several studies have documented the link between neuronal activity and perception using anti-correlated random dot stereograms (anti-RDSs). These studies have shown that neurons in area V4 (Tanabe et al., 2004) and in the IT (Janssen et al., 2003) have less pronounced selectivity for anti-RDS compared with those in MT (Krug et al., 2004) and MST (Takemura et al., 2001). This is consistent with the idea that the representation of neurons in the ventral pathway is closer to perception. Second, there is preliminary evidence showing that V4 neurons encode relative disparity (Umeda et al., 2004), whereas MT neurons encode absolute disparity (Uka and DeAngelis, 2002). This has been confirmed by a functional magnetic resonance imaging study as well (Neri et al., 2004). The fact that both humans and monkeys are much more sensitive to relative compared with absolute disparity implies that the representation of neurons in the ventral pathway is closer to perception. The present study agrees with this emerging idea of the differential processing between the ventral and dorsal pathway in that it provides evidence for a link between fine disparity signals in the IT and behavioral discrimination.

There are potentially two ways to interpret what exactly the IT neurons were encoding. Although the monkeys were presumably basing their judgments on depth relative to the fixation point, it is unclear whether the IT neurons encoded the position of the stimulus in space or whether they encoded a particular three-dimensional structure (i.e., a structure consisting of two surfaces with one hovering over the other). It seems likely that IT neurons encode a particular three-dimensional structure, although we cannot rule out the possibility that IT neurons encode position in space.

Finally, although we propose that the IT is linked to discrimination of fine stereoscopic depth, we are not proposing that the IT is the only area involved. It is still unclear whether and how areas outside the IT are involved in fine-depth discrimination. We are also not proposing that disparity signals in the IT are important only for fine-depth discrimination. Disparity signals in the IT have been suggested to be useful for three-dimensional shape perception (Janssen et al., 1999). In fact, disparity signals in the IT may primarily be useful for three-dimensional object representation, but the same representations, in turn, are important for fine-depth discrimination.

In conclusion, the present study suggests that the IT, an area belonging to the ventral visual pathway, is functionally linked to fine stereoscopic depth discrimination, a perceptual task involving the localization of objects in depth. This could stimulate future experiments directed at resolving a more specific role of the ventral pathway in stereopsis.