Neurons in Dorsal Visual Area V5/MT Signal Relative Disparity

Abstract

Judgments of visual depth rely crucially on the relative binocular disparity between two visual features. While areas of ventral visual cortex contain neurons that signal the relative disparity between spatially adjacent visual features, the same tests in dorsal visual areas yield little evidence for relative disparity selectivity. We investigated the sensitivity of neurons in dorsal visual area V5/MT of macaque monkeys to relative disparity, using two superimposed, transparent planes composed of dots moving in opposite directions. The separation of the planes in depth specifies their relative disparity, while absolute disparity can be altered independently by changing the binocular depth of the two planes with respect to the monkey's fixation point. Many V5/MT neurons were tuned to relative disparity, independent of the absolute disparities of the individual planes. For the two plane stimulus, neuronal responses were often linearly related to responses to the absolute disparity of each component plane presented individually, but some aspects of relative disparity tuning were not explained by linear combination. Selectivity for relative disparity could not predict whether neuronal firing was related to the monkeys' perceptual reports of the rotation direction of structure-from-motion figures centered on the plane of fixation. In sum, V5/MT neurons are not just selective for absolute disparity, but also code for relative disparity between visual features. This selectivity may be important for segmentation and depth order of moving visual features, particularly the processing of three-dimensional information in scenes viewed by an actively moving observer.

Introduction

The relative disparity between pairs of visual features delivers our most accurate judgments of binocular depth (Westheimer, 1979). Moderate changes in vergence eye position do not alter relative disparity, whereas absolute disparity is important for vergence control mechanisms (Rashbass and Westheimer, 1961). Although relative disparity requires two binocular features, this requirement is consistent with a wide range of depth profiles. Choice of a specific spatial profile may be an important factor in experiments that probe for relative disparity selectivity (Parker, 2007).

Human fMRI experiments consistently show greater activation for disparity-defined stimuli in the dorsal stream (Tsao et al., 2003; Minini et al., 2010). However, activations specific to relative disparity predominate in ventral rather than dorsal regions (Neri et al., 2004; Parker, 2007). The dorsal visual regions generally (specifically V5/MT) have been regarded as contributing to coding of absolute disparity (Neri, 2005; Orban et al., 2006; Parker, 2007).

In macaque ventral visual areas, a subset of disparity-tuned neurons in V2 (Thomas et al., 2002) and a larger fraction in V4 (Umeda et al., 2007) signal the relative disparity between spatially adjacent surfaces; neurons in inferotemporal cortex (TEs) signal the depth profile of curved surfaces (Janssen et al., 1999). By contrast, such neuronal signals are absent in V1 and dorsal area V5/MT (Cumming and Parker, 1999; Uka and DeAngelis, 2006). For some V5/MT neurons, stimuli in the receptive field's surround suppress the response to stimuli in the center (Allman et al., 1985; Bradley and Andersen, 1998; Huang et al., 2007). This suppression weakens when the disparities of surround and center are different, but Uka and DeAngelis (2006) established that this did not amount to a consistent response to relative disparity. Further along the dorsal visual pathway, Eifuku and Wurtz (1999) identified a subset of MSTl neurons that might show such selectivity. But this proposal was based on the differences in disparity preference between center and surround when the disparity presented to either center or surround was fixed to zero, rather than a direct, systematic study of relative interactions.

Relative disparity may be used by the visual system to separate surfaces in depth when one surface is viewed transparently in front of another. Responses of V5/MT neurons to a single plane of moving dots are substantially altered by a second, superimposed plane containing dots moving in a different direction (Snowden et al., 1991). Like the center-surround studies mentioned above, the most common interaction is suppression when two planes are superimposed at the same depth and a release from that suppression as a difference in depth is introduced. It remains unclear whether these effects are consistent with signaling of relative disparity.

We set out to examine systematically whether V5/MT neurons signal relative disparity between superimposed, transparent planes independent of the absolute disparities of the individual planes. We also tested how far previously observed interaction effects in V5/MT neurons for such stimuli could be explained by a linear combination of the responses to the individual planes.

Materials and Methods

Animals

We recorded from three male rhesus macaque monkeys (Macaca mulatta), each of which was implanted with a recording chamber over occipital cortex that allowed access to V5/MT through V1/V2 from the back of the brain. Animals were also implanted with two scleral search coils to allow measurements of eye movements. They had previously been trained to fixate and discriminate cylinder rotation for fluid rewards (Dodd et al., 2001). V5/MT was identified using the following: (1) the sequence of gray matter/white matter stretches encountered on the approach to V5/MT; (2) the ordered receptive field map of V5/MT; and (3) the direction and disparity tuning of neurons in the target area. For one animal we subsequently also identified electrode tracks to V5/MT and lesions within V5/MT postmortem (Dodd et al., 2001). All procedures were carried out in accordance with Home Office (UK) Regulations. All animal experimentation was conducted in accordance with the Society for Neuroscience's “Policies on the Use of Animals and Humans in Neuroscience Research.”

Visual stimuli

Visual stimuli were displayed on two Eizo FlexScan monitors (Eizo) viewed via mirrors in a Wheatstone stereoscope configuration. Mean luminance was 42 cd/m². The maximum available contrast was 99%, and the frame rate was 72 Hz. The screens were positioned at a distance of 89 cm from the position of the eyes. Data from a few neurons were collected according to methods described in detail in Dodd et al. (2001). Where random dot stimuli were used, they consisted of black and white dots on a mid gray background. Dot size subtended 0.2 degrees of visual angle. The density of the random dot stimuli used was 25%. After each frame of the stimulus was presented, a fraction of dots were eliminated with a probability of 2% and replaced with new dots at randomly chosen locations in the display. Visual stimuli were matched to the receptive field properties of the neuron, in particular its size, direction, and velocity preferences (see below, Recording). Preferred speed of dots was qualitatively matched to the characteristics of the receptive field using dots that moved in the preferred direction for the neuron. When the stimulus was a pair of planes, with planes of dots moving simultaneously in opposite directions, the dots always had the same speed regardless of their direction of movement. Stimulus duration was 2 s.

Relative disparity.

To compare the tuning of V5/MT neurons to absolute and relative disparity, we presented stimuli made up of two transparent planes of black and white random dots that covered the minimum response field of the neuron. The dots that made up one plane always moved coherently in the neuron's preferred direction, and the dots in the other plane moved coherently in the neuron's null direction (Fig. 1a). The two planes were separated in depth by relative disparity between the sets of dots. The relative disparity was applied so that both planes were shifted in equal but opposite directions away from the fixation plane by the same disparity, i.e., near and far. On some trials, we also applied a pedestal disparity to both planes. This pedestal disparity changed the absolute disparity of both planes by the same amount in the same direction (Fig. 1a,b). In most experiments, we included three pedestal disparities for data collection (one near, one far, and zero disparity). For two cells from one monkey we collected only two pedestal disparities (near and far pedestal). For seven cells from two monkeys we applied five pedestal disparities (two near and two far disparities and zero disparity). All conditions were interleaved in a pseudo-random order. To be included in the analysis, neurons had to be selective to disparity when the two transparent planes were presented (one-way ANOVA, p < 0.05).

Figure 1.

Relative disparity stimuli. a, The relative disparity stimulus consisted of two transparent planes of dots moving in opposite directions. One set of dots always moved in the preferred direction of the recorded neuron, the other set moved in the opposite, null direction. The planes were separated in depth by the relative disparity d1 applied to the two sets of dots (red arrows). In these experiments, we also interleaved trials in which a pedestal disparity was applied to both planes. On the left is a set of planes with a pedestal disparity of zero. The line through the fixation point illustrates that the center of the two planes is at the fixation plane. A pedestal disparity −d2 was applied to the stimulus on the right, moving both planes forward. This pedestal disparity changed the absolute disparity of each of the two planes, but the relative disparity d1 between the two planes stayed constant. b, This diagram gives an example of a typical stimulus space explored in these experiments. Stimuli are described in terms of the absolute disparity for the preferred (PREF) (x-axis) and null (NULL) motion plane (y-axis) of the recorded neuron. Stimulus combinations along each green line have the same pedestal disparity (ped); stimuli along one red line have the same relative disparity (rel disp). This configuration was chosen as it allowed the investigation of a range of relative disparities across an equal number of preferred and null motion disparities in a reasonable number of trials.

One plane/two planes.

To compare responses to the absolute disparity of individual planes with the response to both planes combined, we presented planes of moving dots covering the minimum response field. When a single plane was shown, the dots in the plane were either coherently moving in the preferred direction (‘pref’) or in the null direction (‘null’) for this neuron (see Fig. 5a). Presentations of these single-plane stimuli at different absolute disparities were interleaved with presentations of both planes superimposed. The different stimuli were presented in a regular order of pref single plane followed by null single plane followed by the two-plane stimulus. This had the advantage that the expected effect of suppression with the introduction of the null plane in the two-plane stimulus (Snowden et al., 1991) could not be attributed to neuronal adaptation during the strong response to an immediately preceding pref stimulus.

Figure 5.

One- and two-plane stimulus. a, Sketch of a two-plane stimulus and its component planes. The two-plane stimulus was made up of a plane of dots moving in the preferred direction and another moving in the null direction. The two planes were always positioned symmetrically around the fixation point with opposite signs but the same magnitude of absolute disparity. In the example, the preferred direction plane (black) is at an absolute disparity of −0.2° (near), and the null direction plane (red) is at +0.2° (far). The relative disparity of the associated two-plane stimulus is −0.4°. In these experiments, the disparities of two-plane stimuli are identified by the absolute disparity of the preferred plane (−0.2°). Interleaved with the two-plane stimulus, individual planes were presented separately at the same disparities as in the two-plane stimulus (with reference to Fig. 1b, the two-plane stimulus set used here would fall along the green line in Fig. 1b labeled ped = 0 plus the individual preferred and null component planes that make up the two plane stimuli). The black cross and line depict the fixation cross and plane of fixation respectively. b, Neuronal tuning curves were plotted to compare for each disparity the response to the two-plane stimulus with the responses to the individual planes that made up this two-plane stimulus. The absolute disparity of the preferred plane is represented on the x-axis at the bottom of the graph, with the absolute disparity of the null plane at the top. The V5/MT neuron in this example was tuned to a plane of dots moving in the preferred direction presented at a negative disparity (near); there was little response to a plane of dots moving in the null direction at any of the disparities tested. Presenting both planes together did not change the sign of the disparity preference (negative), but the disparity tuning became sharper. c, Another V5/MT neuron from a different animal showed marked attenuation in the response to the pref stimulus when the null plane was added, again without altering the overall disparity preference of the neuron. In b and c, the green line represents the simple regression fit to the two plane responses based on the single plane responses for Equation 6, and the blue line represents the fit for Equation 7, including the interaction term.

The range of absolute disparities used was the same for individual planes presented alone and when presented as part of the two-plane stimulus. The stimulus was presented ‘as seen through a window,’ so the size of the stimulus stayed constant at the different disparities presented, which eliminates monocular cues to distance. A neuron's data were only included in the detailed analysis if the neuron showed a clear direction preference in the experimental measurements. Each included neuron was also required to show tuning to absolute disparity at the preferred motion direction (one-way ANOVA, p < 0.05).

Cylinder stimulus.

For a stereoscopic depth judgment task, we used a structure-from-motion cylinder made up of two sheets of random dots moving in opposite directions. This stimulus was similar to the two plane stimulus described above, but the dots that made up the cylinder surface had a sinusoidal velocity and disparity profile (see Dodd et al., 2001 for a detailed description).

Recording

Single units were recorded from area V5/MT with a polyimide-coated tungsten microelectrode (MicroProbe; impedance, 0.6–1.2 MOhm). In all experiments, monkeys were required to maintain fixation throughout each stimulus presentation. The fixation point was 0.2° of visual angle, and the fixation window had a diameter of between 1.2–2°. Monkeys received a fluid reward at the end of each successfully completed trial. The monkey was rewarded for careful fixation or, at the end of trials that required a discrimination judgment on the rotating cylinder stimulus, the monkey was rewarded for its correct perceptual report by making a controlled saccade to a new choice target after the main visual stimulus was extinguished.

For the relative disparity experiment, we recorded 80 neurons from three monkeys: 44 from monkey fle, 23 from monkey rin, and 13 from monkey bil. Three cells were excluded because fewer than four repeats were collected at each relative disparity and each pedestal condition. For the one-plane/two-planes experiment, we recorded 61 neurons from two monkeys (48 monkey fle; 13 monkey rin). One cell was excluded because its peak firing rate was <10 spikes/s, one because fewer than four trials per disparity per condition had been collected, and one because post-experiment inspection of the dataset revealed that the preferred direction of motion had been wrongly assigned. Fifty-three of the remaining 58 V5/MT neurons showed significant disparity tuning for a single plane of dots that moved in the preferred direction (‘pref plane’) (ANOVA, p < 0.05).

We always assessed first qualitatively the direction selectivity of each single unit. Then we mapped the receptive field with random dots moving at the preferred motion direction. Once the stimulus was matched to the minimum response field of the neuron, we collected quantitative direction tuning and absolute disparity tuning data, followed by tests for relative disparity tuning and/or the interleaved one-plane/two-planes experiment. All cells included in this study showed strong direction tuning and disparity tuning to a planar stimulus of dots moving in the preferred direction (assessed qualitatively when checking isolation of a single neuron from background firing). Only cells that were selective for disparity (either absolute or relative) with the two-plane stimulus were included in the relative disparity analysis and in the regression analysis for the one-plane/two-planes experiment (ANOVA, p < 0.05). To be included in the analysis of shift ratios (see below, Analysis, Calculating shift ratios), disparity tuning curves for each included pedestal had to be significant (ANOVA, p < 0.05).

For some cells, the experimental protocol continued with a measurement of choice probability in a stereoscopic depth judgment task that required the animal to report the direction of rotation of a moving cylinder defined by structure-from-motion (see Dodd et al., 2001; Krug et al., 2004). Like the two-plane stimuli, cylinder stimuli were matched to the motion and disparity preferences of the recorded neuron's receptive field. At the end of the trial, animals were required to make an eye movement to one of two choice targets. Choice targets were positioned to the left (for clockwise rotation) and right (counterclockwise rotation) of the fixation point. Correct responses were rewarded with fluid; on ambiguous trials, the animal was rewarded on 50% of trials selected randomly.

Analysis

Calculating shift ratios.

For 54 cells, at least one set of disparity curves collected at two different pedestal disparities could be fit with Gabor functions. For cells for which we collected disparity data with more than three pedestals, we only analyzed in detail data collected with the zero disparity pedestal and the largest near disparity and the largest far disparity, which is equivalent to the datasets where we collected three pedestal disparities. The fitted Gabor functions were allowed to differ only in their horizontal offset, which was taken as a measure of the ‘shift’ in the disparity function induced by the pedestal. To be included, the fitted curves had to explain >80% of the total variance of the data, and both curves had to show significant tuning to disparity when the two transparent planes were presented (ANOVA, p < 0.05). We included no more than two shift ratios per neuron. For each cell, we fit first the near-zero pedestal pair of disparity tuning and the zero-far pedestal pair. For four cells, we fitted the near-far pedestal pair, either because neither of the pairs including zero pedestal disparity could be adequately fit (n = 2) or because only the near and far pedestals had been collected (n = 2). When the near-far pair was analyzed, then it was the only pair of tuning curves to be fit for this neuron.

The shift ratio was calculated as follows:

where S is the lateral shift between a pair of tuning curves recorded with different pedestal disparities, V is the difference in mean vergence for all two-plane stimuli with those pedestal disparities, and D is the difference in stimulus disparity between the two pedestal disparities for which the shift was calculated.

It was evident that not all disparity tuning curves were well fitted with a Gabor function; therefore we also calculated shift ratios using a spline fit (spline, MATLAB) for all 68 neurons with at least two sets of disparity data collected at different pedestals that showed significant tuning to disparity when the two transparent planes were presented (ANOVA, p < 0.05). We quantified the peak shifts between pairs of fitted spline curves obtained with different pedestal disparities for the same cell. One neuron's data were excluded because the peak disparity could not be unambiguously defined. The shift ratio was calculated using the same equation (1) as shown for the Gabor fits.

Fitting 2D Gaussian surfaces.

To investigate the response profile to different combinations of disparities for the two transparent planes of preferred and null motion, we fitted three different Gaussian surfaces to the tuning curve data. The first is an isotropic 2D Gaussian (with five free parameters):

the second is a nonisotropic, x–y-separable, 2D Gaussian (with six free parameters):

and the third is a non-isotropic, rotated 2D Gaussian (with seven free parameters):

where x is the preferred motion disparity; y is the null motion disparity; x₀ and y₀ are the coordinates of the peak of the Gaussian; σ, σ_x and σ_y describe the spread of the Gaussian; θ is the rotational angle of the Gaussian; A is the amplitude of the Gaussian; and b is a parameter providing a vertical offset.

Included in this analysis were all 62 neurons for which significant tuning to disparity was found for the two transparent planes at three or more pedestal disparities (ANOVA, p < 0.05). Such disparity tuning (for one or two pairs of pedestal disparities) was also the entry criterion for the Gabor and spline fits. Data were plotted with firing rate (z) against null motion disparity (y) and preferred motion disparity (x) using the Matlab fit command, the MATLAB nonlinear fitting algorithm and the custom equations shown above. The peak of the Gaussian surface was constrained to lie between −2° and +2° degrees of disparity for both preferred and null motion. To compare goodness-of-fit, we compared the square of the correlation between response and predicted response values (r²) and carried out an F test on the summed square of residuals (p < 0.05).

Regression analysis.

The regression analysis was carried out in MATLAB using regstats, which performs a multilinear regression based on a model that is linear in its parameters.

Maximum slopes.

Maximum slopes were calculated from the square root of the mean firing rates at different neighboring disparities. The maximum slope is one potentially useful indicator of the ability of the neuron to discriminate between different disparities. An accurate indicator would take into account the variability of neuronal responses as well. It would be possible to allow for this neuronal variability by calculating a measure of difference in neuronal firing scaled by variance (Fisher information). In this dataset, estimation of variance directly from the data set is likely to be noisy, as accurate estimation of variance requires a large number of trials per data point. For this reason, we relied upon the well characterized relationship between variance of neuronal firing and mean firing rate in cortical neurons, specifically for neuronal activations in V5/MT. Each measure of maximum slope was therefore calculated by taking the square root of mean firing rate. To the extent that the variance of neuronal firing is proportional to the mean, which is exactly true for the Poisson model of neuronal firing, equal changes of square root of mean firing rate should be equally discriminable (Prince et al., 2002). Discrimination of disparity by each neuron was therefore characterized by the difference in square root of firing rates divided by the difference in disparity for the two neighboring points yielding the maximum slope. This was carried out separately for pref and two-plane stimuli.

Results

V5/MT cells are selective for the relative disparity between two transparent planes

We compared the relative and absolute disparity tuning of neurons in visual area V5/MT from three rhesus macaque monkeys. The responses of 80 single neurons were recorded to two superimposed, transparent surfaces of random dots that moved in opposite directions at the same speed. The dots of one plane moved coherently in the neuron's preferred direction, the dots forming the other plane moved into the opposite, null direction. The depth between the two surfaces (the relative disparity between the two planes) was systematically varied as well as the depth of both planes with respect to the fixation point (the pedestal disparity of the two planes). The pedestal disparity affected the absolute disparity of the two planes independently of the relative disparity between the two planes (Fig. 1). All stimuli in this study were carefully matched to the recorded neuron's receptive field size as well as the neuron's direction and velocity preference.

Neuronal responses were plotted either as a function of the absolute disparity of the plane that contained the dots moving in the neuron's preferred direction (Fig. 2a,d,g) or as a function of the relative disparity between the two planes (Fig. 2b,e,h). Data were separated according to changes in overall absolute disparity of the entire two-plane stimulus (the pedestal disparity).

Figure 2.

Neuronal responses to absolute and relative disparity for three example V5/MT cells [fle192.0 (a–c); rin252 (d–f); fle210.0 (g–i)]. a, The response of cell fle192.0 was plotted against the absolute disparity of the preferred (PREF) motion plane. Each of the three tuning curves for absolute (abs.) disparity was collected with a different pedestal (ped) disparity applied to the two planes. For different pedestal disparities, the absolute disparity preference of this cell changed. b, When the responses for this cell were plotted against the relative disparity between the two planes of the stimulus, all three tuning curves showed a consistent preference for a relative disparity close to 0°. c, Two-dimensional surface plot of the interpolated responses of this neuron (fle192.0) in the coordinate frame of Figure 1b. The structure in the responses running along the positive diagonal is indicative of selectivity for relative disparity. d, The same plot as in a for a second example cell, rin252.0. The preferred absolute disparity of this cell changed for different pedestal disparities. e, When plotted against the relative disparity between the preferred and null planes, this neuron showed very consistent tuning for the relative disparity between the two planes, regardless of the absolute disparity of the two planes. f, The two-dimensional interpolated surface plot of rin252.0 showing strong alignments of peak responses along the positive diagonal consistent with relative disparity tuning. g, The tuning curve of cell fle210.0 showed a consistent peak when the preferred plane was at an absolute disparity of zero regardless of the pedestal disparity and therefore of the relative disparity of the stimulus. h, When the responses of fle210.0 were plotted against the relative disparity between the two planes, there was no consistent relative disparity tuning. This cell was tuned to the absolute disparity of the preferred plane. i, Two-dimensional interpolated surface plot of fle210.0 showing primary alignment of response contours parallel to the y-axis, consistent with tuning for absolute disparity of the preferred motion direction.

When inspecting the tuning curves for each cell, we found that some cells consistently preferred the same absolute disparity for the preferred motion plane (Fig. 2g), even when changes of the pedestal disparity meant that the relative disparity between the two planes had changed (Fig. 2h). Such cells are exhibiting sensitivity to the absolute position of the preferred motion plane with respect to the fixation point, without responding to relative disparity. Other V5/MT neurons showed a strong and consistent response to a specific relative disparity between the two planes (Fig. 2b,e), while the absolute depth of the preferred plane yielded no consistent response under these conditions (Fig. 2a,d). Such neurons are primarily responsive to relative disparity. On qualitative inspection, a very small number of neurons appeared to show pronounced changes in the shape or the gain (or both properties) of the tuning curve under these stimulus manipulations (data not shown). The responses of such neurons were excluded from some of the subsequent data analyses, for example when data were fit with pairs of Gabor functions, using criteria described in detail later.

In the disparity stimulus space used for this study, tuning for absolute and relative disparity vary along straight lines at 45° to each other (see Fig. 1b). A neuron tuned to the absolute disparity of the preferred motion stimulus would have an elongated response profile with a vertical orientation in Figure 1b, whereas a neuron tuned to the relative disparity between preferred plane and null plane would have a diagonally elongated response profile with a slope of +1. We examined the pattern of responses in this stimulus space for all neurons for which we could collect disparity-tuned responses with at least three pedestal disparities (ANOVA, p < 0.05) (n = 62). When we qualitatively examined isocontour lines of the interpolated surfaces fitted to the data, elongated response profiles predominated (Figs. 2c,f,i). Diagonally oriented, elliptical response profiles characterize V5/MT cells with relative disparity tuning (Fig. 2c) and vertical ellipses V5/MT cells tuned to absolute disparity (Fig. 2i). In Figure 2f the responses of this neuron, which was broadly tuned to far disparity and selective for relative disparity, displayed elongated isodensity lines running diagonally.

To provide a quantitative estimate of relative disparity tuning across our sample of V5/MT cells, we fitted Gabor functions to the neuronal responses for absolute disparity of the preferred motion plane in the two-plane stimulus (Fig. 2a,d,g). Pairs of tuning curves for two different pedestal disparities were fit with a pair of Gabor functions that were only allowed to differ in their horizontal offset (Fig. 3a–c). This offset was measured as the ‘shift.’ For the chosen stimulus configuration, when spike rates are plotted against the absolute depth of the preferred motion plane (e.g., Fig. 2a), a change in pedestal disparity is associated with a change in relative disparity of the stimulus (Fig. 3a; see also Fig. 1b). The ‘shift ratio’ is the horizontal shift in degrees of the tuning curve, as the pedestal disparity is changed, divided by the difference in pedestal disparities in degrees. If a neuron were selective to the absolute disparity of the stimulus, we expect the tuning curves not to shift for different pedestal disparities (shift ratio = 0; example in Fig. 3c). If a neuron were selective to the relative disparity between the two planes, we would expect the tuning curves to shift by the same amount as the introduced pedestal (shift ratio = 1; example Figs. 3a,b). The underlying selectivity for relative disparity was revealed when we replotted the firing rate against the relative disparity between the two planes. In this case, the peak response remained at the same relative disparity despite changes in absolute disparity (Fig. 2b,e).

Figure 3.

Calculation of shifts. a, A pair of tuning curves from Figure 2a collected with a zero pedestal and at a positive (pos) pedestal disparity of 0.2° were fitted with Gabor functions. The two functions were only allowed to differ in their horizontal offset, called the ‘shift’, measured in degrees (abs., absolute). The shift ratio (after vergence correction) was 1.00, indicating relative disparity selectivity. b, This pair of tuning curves from Figure 2d was recorded with a zero pedestal and a positive pedestal of 0.2°. The shift ratio was 0.78 (after vergence correction). c, When similarly fitted with a pair of Gabor functions, the example cell from Figure 2g had a very small shift between the tuning curves for the negative (neg) near pedestal (−0.2°) and the zero pedestal. The shift ratio (after vergence correction) for this cell was 0.14, which indicates absolute disparity selectivity.

One factor that might have affected the measured shifts is a change in vergence eye position. If a monkey were to change systematically its state of vergence for different sets of stimuli, this would alter the absolute but not the relative disparity of the stimulus. We found no significant correlation between the vergence states of the animal and the absolute disparity of the front plane of the stimulus, but there was a slight influence of applied pedestal disparity on mean vergence. This is manifest as a weak but significant correlation between mean vergence and applied pedestal disparity (Spearman's r = 0.24, p < 0.01, n = 144). Therefore, we corrected all our measures of disparity shifts by the mean vergence measured for the trials with the same pedestal disparity in each experiment before calculating the shift ratio. This correction changed shift ratios on average by <8%.

For 54 V5/MT cells (28 monkey fle, 16 monkey rin, 10 monkey bil), disparity-tuning curves collected with at least two different pedestal disparities showed significant tuning (ANOVA, p < 0.05), and the Gabor fits accounted for >80% of the response variance. When we plotted the distribution of the 84 measured shift ratios from these 54 neurons, many neurons responded to the absolute disparity of the visual stimulus (shift ratio near 0), but a number were sensitive to the relative disparity between the two planes (shift ratio near 1), regardless of the absolute disparities of the planes (Fig. 4a). There were also quite a few cells with intermediate response properties, a pattern observed previously in other visual areas (Thomas et al., 2002; Umeda et al., 2007).

Figure 4.

Summary data shift ratios. a, The distribution of shift ratios for V5/MT neurons showed neurons with absolute disparity selectivity, neurons with relative disparity selectivity, and many with intermediate response patterns. Black bars indicate significant shifts (F test; p < 0.05); white bars, not significant. b, The distribution of shift ratios for V5/MT neurons as obtained with spline fits. c, For 30 V5/MT neurons we could fit both sets of tuning functions, i.e., near-zero pedestals and zero-far pedestals, with pairs of Gabor functions. When we compared the size of the shift induced by the two disparity pedestals of same magnitude but opposite depth sign, shift ratios were not significantly different (Wilcoxon rank sum, p = 0.25) and they were strongly correlated (Spearman's, r = 0.50, p < 0.01).

The median shift ratio was 0.46 for all 84 valid shifts from 54 cells. This shift ratio was significantly different from 0 (Wilcoxon sign rank, p < 0.0001), indicating selectivity within the population for relative disparity. This was also true for the median shift ratio for each of the three monkeys individually (fle: 0.39; rin: 0.48; bil: 0.61) (Wilcoxon sign rank, p < 0.01). For this calculation, the 84 valid shifts included data from 30 neurons with two valid shift ratios and 24 neurons with only one shift ratio from one pair of tuning curves. To check whether the inclusion of two measures for some cells biased our results, we recalculated the median shift ratio using only one shift ratio per cell. For the 30 cells with two valid shifts, we used the mean of the two shift ratios. This did not change the result (median 0.46, n = 54, p < 0.0001).

Not all tuning curves were fit adequately by pairs of Gabor functions. To assess shift ratios in a model-free way, spline curves (smooth polynomials constrained to pass through each data point) were fit independently to each tuning curve, and the peak was taken as a measure of the preferred disparity (Thomas et al., 2002). From the spline fits we estimated a median shift ratio of 0.52 (n = 138 shifts from 67 cells), which was significantly different from zero (Wilcoxon sign rank, p < 0.0001). For the 54 cells to which spline and Gabor analysis could be applied, the two shift ratios were correlated (Spearman's r = 0.61, p < 0.0001). The overall distribution of shift ratios was similar for both measures (Fig. 4a,b).

Using the spline-fitted curves, an estimate of the distribution of the disparity preferences for the two-plane stimulus was also derived. For 63 cells we could determine the peak of the fitted tuning function when no pedestal was applied. Forty cells showed a preference for the preferred motion stimulus to be in the near plane, 20 for the preferred motion stimulus to be in the far plane, and three functions had a peak when both planes were at zero disparity.

When interpreting the shift ratios based on tuning curves that take one-dimensional slices through the disparity space (see Fig. 1b), some caution is required. If the response interactions between the absolute disparity for preferred and null motion stimuli were described by an isotropic 2D Gaussian, a positive peak shift for different pedestals could be generated in some cases by an unfortunate choice of sampling in relation to the peak of the isotropic Gaussian, without any implication of relative disparity tuning. The two-dimensional shape of the responses to combinations of disparity for preferred and null motion directions therefore is critical.

It is already apparent from the examples in Figure 2, c and f, that the primary contours for some neurons are aligned elliptically along the diagonal, indicating tuning for relative disparity. To check quantitatively whether the shape of the response function might have affected our results when we assessed relative disparity tuning, data were fitted with an isotropic 2D Gaussian (Eq. 2), an anisotropic, x–y-separable 2D Gaussian (Eq. 3), and a rotated, anisotropic 2D Gaussian (Eq. 4). For 22 of the 62 neurons, a large proportion of their response variance was explained well by an isotropic 2D Gaussian (r² > 75%). But even so, for 10 of these 22 cells, fitting one or the other of the anisotropic Gaussian functions significantly improved the fit (F test, p < 0.05). Excluding the remaining 12 cells that appeared adequately fit by an isotropic Gaussian did not alter the estimate of the median shift ratio significantly (median = 0.43, neurons = 42; for comparison, all Gabor fits, median = 0.46, neurons = 54; Wilcoxon rank sum test, p = 0.68).

Shift ratios might also be misestimated in cases where the variation in firing due to the null plane disparity (y dimension in Fig. 1b) is relatively stronger than the influence of the preferred plane. This seems unlikely, given the way we screened the neurons for direction and disparity selectivity and also given the results of earlier studies (Snowden et al., 1991) and those in the next section below. Nevertheless, the fits of all neurons well described by either anisotropic Gaussian (Eq. 3, r² > 0.75; Eq. 4, r² > 0.75; if Eq. 4 is a significantly better fit than Eq. 3, F test, p < 0.05) were compared to fitting the same data when σ_y in the respective equations was set to infinity. When the dependence on variation in the y dimension was removed, 2D fits for data from only two neurons were significantly worse (F test, p < 0.05). Excluding these cells also from the analysis again did not significantly change the median shift ratio (median shift ratio = 0.43, neurons = 40; Wilcoxon rank sum test, p = 0.58). Thus, while for a small number of neurons an isotropic Gaussian might be an adequate description of the response profile to different disparity combinations, removing these cells from the analysis did not affect the general conclusion about relative disparity tuning in V5/MT. The 2D response profiles shown in Figure 2, c, f, and i, are representative of the population we sampled.

We also analyzed further the pairs of shift ratios for the 30 cells for which we could adequately fit more than one pair of tuning curves with a Gabor function. For those cells, shift ratios obtained with near and far pedestals were strongly correlated (Fig. 4c). This suggests that V5/MT neurons' ability to signal relative disparity was similar over the range of absolute disparities we tested. In summary, certain neurons in dorsal visual area V5/MT signal the relative disparity between two transparent planes over a range of absolute disparities.

The response of a V5/MT cell to the disparity of the individual planes can predict the response to the two-plane stimulus

The introduction of a second plane moving in the null direction modulates absolute disparity selectivity for a plane of dots moving in the preferred direction (Snowden et al., 1991). But it has remained unclear whether such changes in responsiveness could predict whether a cell is tuned to relative disparity. We investigated systematically whether the neuronal response to the two-plane stimulus could be predicted from the responses to separate presentations of the null and the preferred motion plane. We examined the goodness-of-fit of simple linear models (see Eqs. 5 and 6 below) and models including interaction terms (see Eqs. 7 and 8 below). Beyond the regression model, we also investigated whether the introduction of the null direction plane specifically changes the shape of the tuning function. Both measures were related back to relative disparity tuning for a subset of neurons.

We recorded from 53 V5/MT neurons, all of which had significant disparity tuning for a single plane of dots that moved in the preferred direction, the pref plane (ANOVA, p < 0.05), in two monkeys. When a single plane of dots moving in the opposite null direction, the null plane, was presented alone, 42 of 53 of these neurons were also tuned to changes in disparity of these dots (ANOVA, p < 0.05). For the two-plane stimulus, 48 of 53 of these neurons showed significant disparity tuning (ANOVA, p < 0.05; 38 monkey fle, 10 monkey rin) when a second transparent plane of dots moving in the null direction was presented simultaneously with the pref plane. In constructing the two-plane stimulus, the null plane was always placed at an absolute disparity that had the effect of arranging the pref and null planes symmetrically around the fixation plane. Hence, if the pref plane was at a disparity of −0.2°, the null plane was at a disparity of +0.2° (Fig. 5a).

Many neurons in V5/MT are known to have disparity tuning curves that are odd-symmetric with respect to the zero-disparity fixation plane (DeAngelis and Newsome, 1999; DeAngelis and Uka, 2003) when tested with single planes of moving dots. Such neurons prefer either near or far disparities. We compared the disparity preference for one- and two-plane stimuli for the 48 V5/MT cells that were tuned to the disparity of the pref plane and to the disparity of the two-plane stimulus (ANOVA p < 0.05). Forty-one of forty-eight cells exhibited a preference for the same disparity sign applied to the pref plane, regardless of whether the pref plane was presented alone or together with the null plane (see examples in Fig. 5b,c). For 4/48 cells, we could not determine whether the preferred disparity sign changed with the addition of the null plane to the stimulus, because these neurons were tuned to zero disparity (with a peak or a trough at zero) when the pref plane was presented alone. Only 3 of 48 cells showed a disparity preference reversal when the null plane was added to the pref plane. Thus, the preferred depth sign (‘near’ or ‘far’) generally stayed the same when a transparent null plane was added to the pref plane.

To analyze whether quantitative aspects of the neuronal responses to the disparity of the two planes could be predicted from the neuron's responses to the individual constituent planes, we performed linear regression analysis, testing the proposed function:

where R₂ was the response to the two-plane stimulus, R_P the response to a single plane of dots moving in the preferred direction, and R_N was the response to a plane of dots moving in the null direction. a, b, and c were fitted parameters, whereby b was the regression coefficient for the pref plane responses (R_P) and c was the regression coefficient for the null plane responses (R_N). For some cells, responses to the null plane stimuli were very low. To allow meaningful regression analysis, neurons were required to fire on average across all disparities at least one spike per trial to the null plane stimulus.

Regression analysis was performed for 37 cells with significant disparity tuning for all three conditions tested (ANOVA, p < 0.05), i.e., for the pref plane and the null plane, as well as for the two-plane stimulus. Using the above model, we could obtain a significant fit for 35 of 37 cells (F test, p < 0.05). The product of constant b and the pref plane response at a specific disparity (b * R_P) was strongly correlated with the corresponding two-plane response (Fig. 6a). The product of constant c and the null plane response (c * R_N) was more weakly but still significantly correlated with the two-plane response (Fig. 6b). Thirty of thirty-five neurons showed an individually significant regression coefficient b (pref plane response) (t test, p < 0.05) with a median of 0.84, significantly different from 0 (Wilcoxon rank sum, p < 0.0001) (Fig. 6c). Twenty-five of thirty-five neurons also had a significant regression coefficient c (null plane response) (t test, p < 0.05) with a median of 1.24 (Fig. 6d). The median value of c was significantly different from 0 (Wilcoxon sign rank, p < 0.02). For 21 of 35 neurons, constant a was individually significant (t test, p < 0.05) with a median of −17.31 (Fig. 6e). The median value of a was significantly different from 0 (Wilcoxon sign rank, p < 0.05). This negative value for the median of a indicates that the introduction of the null plane motion had a generally suppressive effect on the responses to motion in the preferred direction in the two-plane stimulus.

Figure 6.

Regression analysis. Using Equation 5, for 35 V5/MT cells the neuronal response to disparity in the two-plane stimulus could be predicted from the mean firing rates for absolute disparity of the one-plane stimuli with dots moving in the preferred direction (coefficient b) and the null direction (coefficient c) alone (see Results). a, Each data point is based on the mean responses for one disparity from one cell. Included cells were tuned to disparity for pref, null, and two plane stimuli that could be successfully fitted with the regression model and that had a significant regression coefficient. Coefficient b * R_P (response to pref) was strongly correlated with the corresponding two-plane response R₂ (Spearman's r = 0.78, p < 0.0001, n = 222). b, Coefficient c * R_N (response to null) was significantly but less strongly correlated with the corresponding two-plane response R₂ (Spearman's r = 0.43, p < 0.0001, n = 184). c, Thirty neurons had a significant regression coefficient b (pref plane response) with a median of 0.84. d, Twenty-five neurons had a significant regression coefficient c (null plane response) with a median of 1.24. e, For 21 neurons, constant a was significant; the median of constant a was −17.31. f, The distribution of r² had a median of 0.65 (n = 35).

The disparity-specific components of the interactions between motion in the preferred direction and motion in the null direction are delivered by the coefficients b and c. For many cells, the mean response to the absolute disparity of the individual pref plane (coefficient b) and the mean response to the absolute disparity of the individual null plane (coefficient c) were good predictors of the neuronal response to the same disparities in the two plane stimulus. Such a model could explain a large fraction of the firing rate variance (median, r² = 0.65, n = 35) (Fig. 6f).

The responses of 31 the 35 neurons that could be fit by the previous function could also be fit adequately (F test, p < 0.05) when the pref plane alone was used for predicting the responses to the two-plane stimulus with the following function:

Recall that the coefficient a reflects the general suppression effect of the null plane regardless of the disparity of the null plane. When we compared the 31 cells whose responses could be predicted by both regression Equations 5 and 6, the fraction of response variance accounted for increased significantly (from a median r² = 0.53 to r² = 0.71; n = 31; paired Wilcoxon sign rank, p < 0.0001) by including the responses to the null plane as a predictor. Therefore, in general, a better prediction of the neuron's response to the two-plane stimulus is provided when the individual responses to both the pref and null planes are taken into account. This implies that the disparity of the null plane also has a significant effect on firing to the two-plane stimulus.

We tested further whether the introduction of a nonlinear term improved the prediction of the two-plane response, first by adding an interaction term:

where d was the regression coefficient of the interaction term. When we compared the 35 cells whose responses could be predicted by the regression Equations 5 and 7, the fraction of response variance accounted for increased significantly with the introduction of the interaction term (from a median r² = 0.65 to r² = 0.73; n = 35; paired Wilcoxon sign rank, p < 0.0001). But only 19/35 neurons showed an individually significant regression coefficient d (interaction term) (t test, p < 0.05) with a median of −0.04, which was not significantly different from 0 (Wilcoxon sign rank, p = 0.44). So while there is an improvement with the interaction term, the effect is not consistent.

Using a model with motion in the preferred direction and only an interaction between preferred and null results in the following model:

For the 35 cells whose responses could be predicted by the regression Equations 5, 7, and 8, the fraction of response variance accounted for was median r² = 0.69. The coefficient d was significant for 25 cells with a median of 0.50, but the median was not significantly different from 0 (Wilcoxon sign rank, p = 0.88).

Thus, the introduction of an interaction term does generally improve the predictive power, suggesting a nonlinear component in the disparity-specific interaction between motion in preferred and null directions, but it is not clear how consistent the contribution is from cell to cell. Relative disparity tuning is a nonlinear interaction between the two surfaces. Therefore, one would expect that the goodness-of-fit of the simple linear models would be inferior for neurons that are tuned to relative disparity. For 24 neurons that were tuned to the pref, null, and two-plane stimuli (ANOVA, p < 0.05), we could collect shift ratios for relative disparity tuning and data for the regression analysis. The correlation coefficient between shift ratio and goodness-of-fit measure r² for the simplest regression model (Eq. 6) was −0.40 [Spearman's 95% confidence interval (−0.69, +0.007), p = 0.055]. This would be consistent with a strong trend of the simple linear regression model providing better fits (i.e., larger r² values) for cells with absolute rather than relative disparity tuning (smaller shift ratios), but more data are required.

Figure 5, b and c, also suggests that the introduction of the null plane might alter the shape of the disparity-tuning curve. A critical feature of a tuning curve is its steepness, because this parameter is relevant to the neuron's ability to contribute to sensory discriminations (Parker and Hawken, 1985). Therefore, we analyzed the maximum slopes of each tuning curve as the change in square root of the firing rate per unit change of disparity (see Materials and Methods for justification of this choice). Only those cells that showed significant disparity tuning to the stimulus, both with dots moving in the preferred plane and to the two-plane stimulus (ANOVA, p < 0.05), were included (n = 48). We found a systematic difference in tuning functions between the responses to disparity of the preferred motion plane alone and of the two planes together (Fig. 7a). Taking the square root (sqrt) of the firing rates, the median maximum slope for the disparity-tuning curve when both planes were present was 10.6 sqrt(spikes/s)/degree, but for the preferred plane alone it was 7.1 sqrt(spikes/s)/degree. This change in maximum slope between the two disparity tuning curves is significant (n = 48; paired Wilcoxon sign rank test, p = 0.01).

Figure 7.

Comparison of maximum slopes and shift ratio. a, Each data point represents a neuron with significant tuning to the pref plane and the two-plane stimulus (ANOVA, p < 0.05). For each neuron, we plotted the maximum slope for the pref motion plane (based on square root of firing rate divided by disparity to remove the dependence of variance of firing on mean firing rate; see Materials and Methods for detailed justification) against the maximum slope measured for the two planes (also based on square root of firing rate divided by disparity). The maximum slopes were highly correlated (Spearman's r = 0.62, p < 0.0001), but tuning curves tended to be steeper when both planes were presented (Wilcoxon paired sign rank test, p = 0.01). b, This plot compares for 30 neurons the maximum slope difference (again using square root of firing) between pref and two-plane stimuli and the shift ratio indicating relative disparity selectivity (see Results, V5/MT cells are selective for the relative disparity between two transparent planes). The two measures were significantly correlated (Spearman's r = 0.58, p < 0.001).

For 30 disparity selective neurons, we could compare the responses to the different planes and also collect relative disparity tuning functions (as described earlier). When we compared the difference in maximum slope with the shift ratio for relative disparity stimuli, we found that the two measures were systematically correlated (Fig. 7b). Neurons that showed an increase in steepness of the tuning curves when the null plane was introduced often tended to be tuned to relative disparities.

In summary, the sign of the disparity preference, as well as more specifically the tuning function to the two-plane stimulus, can be predicted to a large extent by the response to the absolute disparity of the constituent transparent planes alone. However, there is an indication of some nonlinear effects that correlate with selectivity for relative disparity. This further adds to the case that some aspects of these receptive fields are organized to deliver a specific selectivity for relative disparity.

Sensitivity to relative disparity is not necessary for decisions about structure-from-motion stimuli

It has been previously suggested that V5/MT has a role in surface segregation and therefore in the perception of structure-from-motion stimuli (Bradley et al., 1995, 1998). Information about the relative disparity between two transparent planes moving in opposite direction could potentially be used to determine the direction of rotation of a structure-from-motion cylinder (Treue et al., 1991). Therefore, for 24 V5/MT neurons from three monkeys, on occasions when we could acquire all the necessary data, we compared shift ratios for relative disparity and a measure of the relationship between the firing of this neuron and perceptual choices about a rotating structure-from-motion cylinder whose direction of rotation was ambiguous (disparity = 0°).

The correlation between single cell activity and perception for discriminating the direction of rotation of a bistable structure-from-motion cylinder is termed ‘choice probability’ (CP) (Dodd et al., 2001; Krug et al., 2004). It is the probability with which an independent observer could predict a monkey's perceptual choice on a given experimental trial based only on the firing rate of the recorded neuron and the distribution of firing rates and choices for that neuron. Choice probabilities range from 0 to 1, whereby a choice probability of 1 indicates a perfect prediction of perceptual choice by neuronal firing. A CP of 0.5 indicates no correlation between the two, and a significant CP below 0.5 would suggest that the relationship between neuronal firing and perceptual report is the opposite on ambiguous and unambiguous trials.

While many neurons in this sample showed significant CPs, shift ratios were not significantly correlated with CPs on the cylinder task (Spearman's r = −0.21, p = 0.31, n = 24) (Fig. 8). Neurons with higher shift ratios are more sensitive to relative disparity, so relative disparity was not associated with stronger CPs for cylinder judgments. Considering those neurons that were more selective for relative binocular disparity and those that were more selective for absolute binocular disparity, it emerged that both types could show significant CPs. It seems that relative disparity selectivity is not necessary to solve the cylinder task.

Figure 8.

For 24 neurons we also measured choice probability for judgments about direction of rotation for a bistable structure-from-motion cylinder. Choice probabilities were calculated for cylinder stimuli with both planes at 0° disparity. Significant (sig) choice probabilities can be measured for neurons tuned to absolute disparity (shift ratio = 0) as well as neurons sensitive to relative disparity (shift ratio = 1).

Choice probability and neurometric threshold have previously been shown to be correlated (Britten et al. 1996; Parker et al. 2002). Our data in this paper are consistent with a negative correlation between neurometric threshold and choice probabilities [Spearman's r = −0.39, 95% confidence intervals (−0.69, +0.01), p = 0.06, n = 24] as we have shown for a larger dataset with the same stimulus and task (Parker et al., 2002). But there is no evidence for a correlation between neurometric threshold and shift ratio (Spearman's r = 0.00; p = 1.00). Our earlier results calculating the maximum slopes suggested that introducing the null plane increases sensitivity. The lack of a correlation between neurometric threshold and shift ratio indicates that this does not necessarily mean that the cells tuned to relative disparity are the most sensitive neurons in the population with regard to disparity in the cylinder stimulus. However, to provide definite conclusions on this point, a considerable larger sample of V5/MT neurons would be needed.

The training for cylinder discrimination, which took place before all the data presented in this paper were collected, could theoretically have affected the disparity tuning properties of V5/MT neurons. This seems unlikely, as none of the training exposed the animals to a cylinder task in which a pedestal disparity of unpredictable size was added to the cylinders during the psychophysical task. Only cylinders with zero pedestal disparity were used throughout the training of the animals in this study. Moreover, the results on CP suggest that relative disparity selectivity of neurons was not relevant in terms of their contribution to performance in the cylinder task. All this evidence points to the view that the tuning properties of neurons have not been induced by the training history.

Discussion

Many neurons in dorsal visual area V5/MT are sensitive to the relative disparity between two superimposed, transparent planes independent of the absolute disparity of the individual planes. There are also neurons that are not purely sensitive to either relative or absolute disparity, but to combinations of the two. In general, we find that the disparity preference of a V5/MT neuron to the two planes can be predicted from the responses to the constituent planes, but linear interactions between the single-plane responses failed to explain fully the response to two planes. Relative disparity tuning of a V5/MT neuron does not predict its contribution to perceptual resolution of the rotation direction of zero-pedestal cylinders.

It has been long established that many V5/MT neurons are selective for absolute binocular disparity (Maunsell and van Essen, 1983). Our demonstration of relative disparity selectivity for transparent planes contrasts with earlier work, which found little selectivity for relative disparity within V5/MT neurons when the stimulus was a central disk and annular surround (Uka and DeAngelis, 2006) (Fig. 9; Table 1). In that study, the disk over the receptive field center and the annulus contained both dots moving coherently in the cell's preferred direction. Hence, there are two substantial differences between the stimuli used in their study and ours: the spatial configuration and the inclusion of motion opponency in our stimulus.

Figure 9.

Comparison of the distribution of shift ratios across visual cortical areas and with different relative disparity stimulus configurations (data replotted with permission from Cumming and Parker, 1999; Thomas et al., 2002; Uka and DeAngelis, 2006; Umeda et al., 2007).

Table 1.

Relative disparity tuning in different extra-striate visual areas

Cortical area	V1	V2	V4	V5/MT	V5/MT
Stimulus	Center surround	Center surround	Center surround	Center surround	Transparent planes
Median shift ratio	−0.03	0.12*	0.47*	0.05	0.46*

This table summarizes the median shift ratios we and other investigators (V1, Cumming and Parker, 1999; V2, Thomas et al., 2002; V5/MT center-surround, Uka and DeAngelis, 2006; V4, Umeda et al., 2007) have measured for absolute versus relative disparity tuning across different visual areas. Asterisk (*) indicates median significantly different from zero (Wilcoxon sign rank, p < 0.05).

When Uka and DeAngelis (2006) used dots that were rapidly refreshed but not moving in any particular direction, they reported a small shift ratio of 0.11, potentially indicating a limited sensitivity to relative disparity between center and surround. In some V5/MT neurons of owl and rhesus monkey, a surround stimulus modulates the response to a stimulus in the center of a receptive field (Allman et al., 1985; Bradley and Andersen, 1998; Huang et al., 2007). When suppression is induced by a surround moving in the same direction as the center, this suppression is weakened when the binocular disparity of the surround differs from the center (Bradley and Andersen, 1998). In those experiments, the surround just modulates the gain of the center response with no changes in selectivity (Bradley and Andersen, 1998; Huang et al., 2007).

Together, these earlier results suggest that the critical point for revealing selectivity for relative disparity in our experiments is the spatial configuration of the stimuli, specifically the use of superimposed, transparent planes. This conclusion is reinforced by earlier studies that examined responses in V5/MT to stimuli with motion transparency in the absence of differences in disparity (Qian and Andersen, 1994). However, in psychophysical experiments, motion contrast increases sensitivity to stereo cues for transparent planes (Lankheet and Palmen, 1998). To understand the contribution motion contrast makes to V5/MT responses, relative disparity tuning for transparent planes moving in the same direction needs to be investigated.

Some V5/MT neurons show a consistent response to the specific tilt of surfaces with large slants independent of mean disparity (Nguyenkim and DeAngelis, 2003). This would be another form of relative disparity, different from the one to center-surround found in the ventral stream. But the reported tilt selectivity is not very strong compared with absolute disparity tuning and could possibly be explained by local variations in selectivity to absolute disparity within different regions of the receptive field.

We compared relative disparity selectivity across different visual areas and stimulus configurations (Fig. 9; Table 1). Neurons in primary visual cortex (V1) did not show relative disparity selectivity for adjacent visual features (Cumming and Parker, 1999). The relative disparity selectivity found here in V5/MT with transparent planes was considerably stronger than that found in visual area V2 (Thomas et al., 2002). The median shift ratio we find in V5/MT is closer to that reported in visual area V4 (Umeda et al., 2007) with spatially adjacent features in the form of a center-surround stimulus as used in V1 (Cumming and Parker, 1999). This is interesting, as in models of visual processing, V4 often assumes a similar position in the hierarchy of the ventral stream as V5/MT does for the dorsal stream (Felleman and Van Essen, 1991). A role for V4 has also been proposed in 3D shape processing, which might rely on relative disparity cues (Hegdé and Van Essen, 2005). However, in contrast to V4, the sizes of shift ratios we found in V5/MT were generally consistent when different changes of pedestal disparity were applied to the same cell: V5/MT cells showed a significant correlation between shift ratios for each cell with different pedestal disparities (see Fig. 4b).

Further down the dorsal stream, in area MSTl, a small number of neurons appear to respond to relative disparity between adjacent surfaces (Eifuku and Wurtz, 1999). More detailed quantification of these neurons would be helpful. In MSTd, there are neurons that reverse their direction preference at different disparities (Roy et al., 1992), reminiscent of the stimulus used in this paper, but no systematic tests of relative disparity selectivity have been carried out. If neurons in either division of MST have similar selectivities to the V5/MT neurons studied here, it seems likely that V5/MT neurons are the source.

At first glance, it is surprising there is no association between relative disparity selectivity and choice probability for judgments about the direction of rotation of a bistable cylinder. However, the cylinder choice paradigm (Dodd et al., 2001) has been constructed with one plane of the cylinder in front and one behind the fixation plane. This means that the choice task presented to the monkey could be solved by relying on the combination of absolute disparity and direction of motion of the preferred plane as an alternative to the relative disparity between the two cylinder planes. To define the perceptual role of neurons sensitive to relative disparity, the contribution of these neurons to perceptual decisions that rely solely on relative disparity information needs to be assessed. Relative disparity signals might aid the depth segmentation of moving surfaces or objects and interact with the neural representation of motion parallax (Nadler et al., 2008).

The responses of V5/MT neurons to the disparity content of transparent two-plane stimuli could be predicted to a significant degree from responses to the absolute disparity of the individual component planes. Previously, for the center-surround case, relative disparity responses could be modeled by a variant of the disparity energy model (Ohzawa et al., 1990) in which inputs from neurons selective for absolute disparity of center and surround are multiplied (Thomas et al., 2002). In principle, a similar model could account for relative disparity selectivity between transparent planes, where (1) two inputs each with selectivity for a specific absolute disparity and opposite direction of motion are first multiplied and (2) the resultant responses for pairs with different absolute disparity selectivities are added. To test such a model, new experiments would need to investigate whether the relative disparity response at different pedestal disparities could also be predicted from the responses to the individual planes.

In sum, neurons in different parts of the visual pathways are selective for relative disparity, but they have different preferences for spatial profiles (Parker, 2007). This observation sits well with the traditional roles associated with dorsal and ventral streams (Ungerleider and Mishkin, 1982; Goodale and Milner, 1992). Support for this hypothesis is provided by a recent psychophysical study (Read et al., 2010) with the visually agnosic individual, DF, whose visual behavior formed the basis of Goodale and Milner's conclusions. This individual had no problems in responding to the disparity between the front and rear moving dots forming the rotating cylinder stimulus. DF made the judgment effortlessly the first time this stimulus was introduced. Indeed, DF's thresholds with the rotating cylinder were slightly superior to those of control participants. However, DF clearly had problems in responding to the disparity content of any configuration that contained spatially-adjacent differences in disparity, although some of the weak performance could be improved fleetingly by training on specific configurations.

As a broad conclusion, we propose that the dorsal and ventral stream areas both respond selectively to relative disparity, but this selectivity serves different visual functions in the two streams. The selectivity for relative disparity between adjacent visual features found in areas V4 and IT of the ventral stream is potentially useful for judgments about the shape of objects. By contrast, sensitivity for relative disparity found here in V5/MT could contribute to the segmentation and depth order of moving visual features in a scene. This sensitivity would aid navigation and other visuomotor functions within the complex visual environment. Specifically, as a moving binocular observer walks through the visual world (i.e., translatory motion) while maintaining gaze on a static target at an intermediate distance (by rotating the eyes) (Nadler et al., 2009), the combination of disparity and motion flow on the two retinas is similar to that delivered in the stimuli we have used to probe the response of V5/MT neurons.