Bridging the gap: global disparity processing in the human visual cortex

Abstract

The human stereoscopic system is remarkable in its ability to utilize widely separated features as references to support fine depth discrimination. In a search for possible neural substrates of this ability, we recorded high-density EEG and used a distributed inverse technique to estimate population-level disparity responses in five regions of interest (ROIs): V1, V3A, hMT+, V4, and lateral occipital complex (LOC). The stimulus was a central modulating disk surrounded by a correlated “reference” annulus presented in the fixation plane. We varied a gap separating the disk from the annulus parametrically from 0 to 5.5° as a test of long-range disparity integration. In the V1, LOC, and hMT+ ROIs, the responses with gaps >0.5° were equal to those obtained in a control condition where the surround was composed of uncorrelated noise (no reference). By contrast, in the V4 and V3A ROIs, responses with gaps as large as 5.5° were still significantly higher than the control. As a test of the spatial distribution of the disparity reference information, we manipulated the properties of the stimulus by placing noise between the center and the surround or throughout the surround. The V3A ROI was particularly sensitive to disparity noise between the center and annulus regions, suggesting an important contribution of disparity edge detectors in this ROI.

the general characteristics of disparity-tuned neurons in V1 are now well-established, thanks to more than three decades of single-cell studies in awake behaving primates (Cumming and DeAngelis 2001; Parker 2007; Poggio and Fischer 1977; Prince et al. 2002; Roe et al. 2007). Some V1 neurons are remarkably sensitive to small changes in disparity, certainly sensitive enough to account for primate stereoacuity (Prince et al. 2000). Yet surprisingly, awake behaving primates are unable to use the responses of these neurons to detect small changes in the disparity of an isolated target. Monkeys, like humans, require a second reference target in the visual field to respond precisely to changes in the disparity of a test target (McKee et al. 1990; Prince et al. 2000; Westheimer 1979). Prince et al. (2000) concluded that a reference target was needed to remove ambiguity about whether the change in disparity was produced by a stimulus change or a change in convergence.

If disambiguation were the primary function of reference targets, then any reference would suffice. However, psychophysical evidence shows that there are constraints on what serves as an optimum reference. For example, Farell (2006) found that stereoacuity for a Gabor target was best when the reference grating had the same orientation as the test. McKee et al. (1990) measured stereoacuity as a function of the distance separating test and reference lines and found that stereoacuity with a reference was better than unreferenced stereoacuity for distances up to 5°. Similarly, Read et al. (2010) found that stereoacuity for a referenced disk was superior to an unreferenced disk for distances as great as 10°. These psychophysical constraints suggest that the calculation of disparity differences depends on specialized neural wiring, rather than on a generalized differencing operation that occurs between any two targets in the visual field.

Here, we used high-density electroencephalogram (EEG) source-imaging to study how the neural response to changes in disparity was affected by the separation of the disparity-modulated stimulus from a fixed reference disparity. In human subjects, we measured population responses in five visual regions of interest (ROIs) [V1, V4, V3A, hMT+, and lateral occipital complex (LOC)]. Our previous work has shown that the presence of an abutting reference stimulus greatly enhances the response to disparity modulation in extrastriate areas (Cottereau et al. 2011a, 2012). How does this enhancement depend on the distance to the reference stimulus, and how does this dependence compare to comparable psychophysical measurements? Our measurements demonstrate that for large separations between center and surround (i.e., >0.5°), areas V1, LOC, and hMT+ have responses equivalent to those of the control condition where the surround was composed of uncorrelated noise. In contrast, areas V3A and V4 have significantly larger responses than the control for gaps up to 5.5°. These areas therefore contain neural populations that integrate disparity information over long distances, and these populations may support long-range perceptual judgments. In a second experiment, we introduced noise into the space between center and surround or throughout the surround. We showed that responses in area V3A are specifically affected by the presence of noise between the center and the surround. This result suggests that neurons acting as disparity edge detectors are present in this ROI.

MATERIALS AND METHODS

The basic stimulus (see Fig. 1) consisted of a central disk surrounded by an annulus; both were composed of dynamic random dots that changed positions every 47 ms (21.25 Hz). The diameter of the central disk was 4°. Its disparity alternated between 12.4 arcmin (uncrossed) and 0 arcmin every 470 ms (i.e., the disparity was driven by a square wave at 2.12 Hz) in a steady-state visual evoked potential (SSVEP) paradigm. Vergence cannot follow disparity changes at this temporal frequency, so it cannot null out the changes in stimulus disparity (Erkelens and Collewijn 1985; Rashbass and Westheimer 1961). The disparity of the annular surround, when binocularly correlated, remained constant during a trial. For all studies, the EEG stimulus presentation lasted 10.35 s (which is 22 cycles of the disk disparity modulation); the first 940 ms (i.e., the first 2 cycles) of the data record was discarded to avoid start-up transients, leading to 9.41-s trials (i.e., 20 cycles of the disk disparity modulation). Trials for all conditions were run interspersed in random order in 10-min blocks. The blocks were repeated four times, producing a total of 20 trials per condition.

Fig. 1.

Diagram of dynamic random dot stimulus configuration (dot update rate = 21.2 Hz). Central 4° disk modulated at 2.12 Hz from 0 to 12 arcmin uncrossed disparity. In a control condition, the surround consisted in a 16° circular annulus with uncorrelated dots. In the other conditions, the dots were correlated and a gap was introduced between center and surround. Its size varied from 0 to 5.5°. Viewing distance was 77 cm. A: left and right monocular images. B: front view. C: 3-dimensional (3D) view. D: top view.

We used two different types of stereoscopes for these studies: a mirror stereoscope and a beam splitter-based system. In the mirror stereoscope, the stimuli were presented on a pair of matched Sony monitors (Multiscan GS 220) viewed at a distance of 77 cm. At this viewing distance, the outer dimensions of the screens were 23° high by 18° wide. The monitors and mirrors were angled for natural convergence at this viewing distance. The contrast of the bright dots was 90%. In the interval between trials, the screen was filled with bright static dots at zero disparity to prevent changes in light adaptation. In the beam splitter system, orthogonally polarized images from two matched Sony Trinitron monitors (model 110GS) were combined via a beam splitter and viewed through appropriately oriented polarized filters placed immediately in front of the eyes. Each eye could see the image on only one screen; the viewing distance was 82 cm, and the contrast of the bright dots was 90%. Each separate experiment described below was, of course, conducted on a single stereoscope; the first experiment was performed in the mirror stereoscope and the second in the beam splitter system. We checked whether the difference in stereoscopes affected our most sensitive measure, namely, psychophysical stereoacuity thresholds. We found that the thresholds were not significantly different for the two types of stereoscopes. As will be apparent in the results below, the general pattern of EEG responses for comparable conditions was also unaffected by the change in stereoscopes.

Subjects

Twenty-one subjects (14 men, 7 women, age range 20–69 yr) participated in the recording sessions. Thirteen subjects (9 men, 4 women) were involved in the first EEG experiment, 12 subjects (8 men, 5 women) participated in the second, and 4 subjects (2 men, 2 women) did the psychophysical measurements. All the subjects were volunteers, with normal stereopsis and normal or corrected-to-normal visual acuity. All subjects were given instructions and detailed information about the experiments. They provided written informed consent before participating in the study in accordance with the Declaration of Helsinki; the human subjects review committee of Smith-Kettlewell Eye Research Institute approved the study.

Main Experiments

Different gaps.

In six interspersed conditions, we measured the SSVEP to uncrossed disparity modulation of the central disk. The outer ring of the surround had an eccentricity of 16° (dot density: 20 dots/°∧2). The distance between the outer ring of the center and the inner ring of the surround varied from 0 to 5.5° (see Fig. 1). In a control condition, the surround consisted of uncorrelated disparity noise (without a gap). This stimulus was first used by Prince et al. (2000) and was used in our previous studies (Cottereau et al. 2011a, 2012) to measure a response driven mainly by neurons tuned to absolute disparity.

For the six gap conditions, we presented a pair of nonius lines and a binocularly visible fixation point superimposed on the center of the disk-annulus stimulus. Subjects were asked to keep the nonius lines aligned during a recording trial. We eliminated the nonius lines and the fixation point for the uncorrelated surround condition to remove all immediate reference stimuli. In our last study (Cottereau et al. 2012), we tested whether the absence of nonius line and fixation point during the stimulation produced a fixation bias for an equivalent condition. A psychophysical nonius alignment test demonstrated that if this bias existed, it was negligible and not sufficient to influence our results. The direct comparison between the EEG responses obtained with and without fixation point for this same condition pointed toward the same conclusion (Cottereau et al. 2012).

Noise surround.

In a second experiment, we tested the influence of the nature of the space between center and surround on the EEG responses. The outer diameter of the surround annulus was diminished to 7.48° (dot density: 30 dots/°∧2). In the first two conditions, the distance between the outer ring of the center and the inner ring of the surround was 1°. The stimulus dimensions for these two conditions were chosen so that the area of the surround annulus and the area between center and surround were equal. The space between the center and surround was either empty (i.e., a gap) in condition 1 or filled with uncorrelated noise in condition 2. In condition 3, the surround touched the center but was filled with a mixture composed of 50% correlated dots and 50% uncorrelated noise. The noise amount was therefore the same in conditions 2 and 3. Nonius lines and a fixation point were displayed in the three conditions (see Different gaps).

Psychophysical thresholds.

We also measured disparity modulation thresholds psychophysically. Observers judged in which of two intervals the disparity of a 4° central disk was modulated by a tiny amount in an uncrossed direction. The stimuli were presented for 472 ms, equal to one period of our EEG presentations. The stimuli were presented in a mirror stereoscope and viewed at a distance of 86 cm. Four different modulation values were presented interspersed in a random sequence in a 100-trial block. We fitted a Weibull function to the percentages correct for these four values and estimated threshold as the disparity modulation corresponding to 81.6% correct. We measured thresholds for each subject and condition in two to four separate blocks (200–400 trials total); the thresholds were averaged from the blocks and are plotted as sensitivities (1/threshold) in Fig. 3.

Fig. 3.

Sensitivity (1/threshold) to different gaps (4 subjects). Bars provide standard error (SE). For subjects RF and SPM, dashed gray lines give the upper limit of confidence intervals (average + 1 SE) for sensitivity in the control condition (uncorrelated surround). Subjects ADP and TT were not able to reliably detect the disparity sign in this condition. This is shown by the 1/∞ for these subjects.

EEG Signal Acquisition and Preprocessing

The EEG data were collected with 128-sensor HydroCell Sensor Nets (Electrical Geodesics, Eugene, OR) and were band-pass filtered from 0.1 to 200 Hz. After each experimental session, the three-dimensional (3D) locations of all electrodes and three major fiducials (nasion, left and right periauricular points) were digitized with a 3Space FASTRACK 3D digitizer (Polhemus, Colchester, VT). For all observers, the 3D digitized locations were used to coregister the electrodes to their T1-weighted anatomical magnetic resonance imaging (MRI) scans and to construct the EEG forward model (see Forward Modeling of Current Source). Raw data were evaluated off-line according to a sample-by-sample thresholding procedure to remove noisy sensors that were replaced by the average of the six nearest spatial neighbors. On average, <5% of the electrodes were substituted. These electrodes were mainly localized in the front of the head or near the ears. The substitutions had a negligible impact on our results, as recordings at occipital and parietal locations mainly drive our estimates of the responses within the visual areas. After this operation, the EEG was re-referenced to the common average of all the sensors. Within each 9.41-s trial, the data were segmented into five 1.82-s-long epochs (i.e., each of these epochs was exactly 4 cycles of the disk disparity modulation). EEG epochs that contained a large percentage of data samples exceeding a noise threshold (depending on the subject and ranging between 25 and 50 μV) were excluded from the analysis on a sensor-by-sensor basis. This was typically the case for epochs containing artifacts such as blinks or eye movements. Finally, a Fourier analysis was applied on every remaining epoch, using a discrete Fourier transform with a rectangular window. For each frequency bin, the Fourier coefficients were then averaged across all the epochs and all the trials. These average Fourier coefficients were therefore obtained from up to 100 (5 epochs × 20 trials) values.

Structural and Functional Magnetic Resonance Imaging

Structural and functional MRI scanning was conducted at 3 T (Siemens Tim Trio, Erlangen, Germany) with a 12-channel head coil. We acquired a T1-weighted MRI data set (3D MP-RAGE sequence, 0.8 × 0.8 × 0.8 mm³) and a 3D T2-weighted data set (SE sequence at 1 × 1 × 1 mm³ resolution) for tissue segmentation and registration with the functional scans. For functional MRI (fMRI), we employed a single-shot, gradient-echo planar imaging (EPI) sequence (TR/TE = 2,000/28 ms, flip angle 80, 126 volumes per run) with a voxel size of 1.7 × 1.7 × 2 mm³ (128 × 128 acquisition matrix, 220-mm field of view, bandwidth 1,860 Hz/pixel, echo spacing 0.71 ms). We acquired 30 slices without gaps, positioned in the transverse-to-coronal plane approximately parallel to the corpus callosum and covering the whole cerebrum. Once per session, a 2D SE T1-weighted volume was acquired with the same slice specifications as the functional series in order to facilitate registration of the fMRI data to the anatomical scan. The general procedures for these scans (head stabilization, visual display system, etc.) are standard and have been described in detail elsewhere (Brewer et al. 2005). The FreeSurfer software package (http://surfer.nmr.mgh.harvard.edu) was used to extract both gray/white and gray/cerebrospinal fluid (CSF) boundaries. These surfaces can have different curvatures. In particular, the gray/white boundary has sharp gyri (the curvature changes rapidly) and smooth sulci (slowly changing surface curvature), while the gray/CSF boundary is the inverse, with smooth gyri and sharp sulci. To avoid these discontinuities, we generated a surface partway between these two boundaries that has gyri and sulci with approximately equal curvature. This “midgray” cortical surface consisted in a triangular tessellation of 20,484 regularly spaced vertices and was used to define the visual areas and the source space for the EEG current modeling (see next 2 sections).

Visual Area Definition

Retinotopic field mapping using rotating wedges and expanding/contracting rings produced ROIs defined by the visual cortical areas V1, V2v, V2d, V3v, V3d, V3A, and V4 in each hemisphere (Tootell and Hadjikhani 2004; Wade et al. 2002). ROIs corresponding to hMT+ were identified with low-contrast motion stimuli similar to those described by Huk and Heeger (2002).

The LOC was defined with a block-design fMRI localizer scan. During this scan, the observers viewed blocks of images depicting common objects (12 s/block) alternating with blocks containing scrambled versions of the same objects. The stimuli were those used in a previous study (Kourtzi and Kanwisher 2000). The regions activated by these scans included an area lying between the V1/V2/V3 foveal confluence and hMT+ that we identified as LOC. This definition covers almost all regions (e.g., V4d, LOC, LOp) that have previously been identified as lying within object-responsive lateral occipital cortex (Kourtzi and Kanwisher 2000; Tootell and Hadjikhani 2004).

Forward Modeling of Current Source

For each subject, the EEG source space was given by his/her “midgray” cortical surface tessellation (see above) and consisted in 20,484 regularly spaced vertices. The distance between connected vertices was on average 3.7 mm, with standard deviation of 1.5 mm and range 0.1–11 mm. Current dipoles were placed at each of these vertices. Their orientations were constrained to be orthogonal to the cortical surface to diminish the number of parameters to be estimated in the inverse procedure (Hämäläinen et al. 1993). The FSL toolbox (http://www.fmrib.ox.ac.uk/fsl/) was used to segment from the individual T1- and T2-weighted MRI scans contiguous volume regions for the inner skull, outer skull, and scalp. These MRI volumes were then converted into inner skull, outer skull, and scalp surfaces (Smith 2002; Smith et al. 2004) that defined the boundaries between the brain/CSF and the skull, the skull and the scalp, and the scalp and the air. The source space, the 3D electrode locations, and the individually defined boundaries were then combined with the MNE software package (http://www.nmr.mgh.harvard.edu/martinos/userInfo/data/sofMNE.php) to characterize the electric field propagation with a three-compartment boundary element method (BEM) (Hämäläinen and Sarvas 1989). The resulting forward model is linear and links the activity of the 20,484 cortical sources to the voltages recorded by our EEG electrodes.

Inverse Modeling Constrained by Visual ROIs

Cortical current density estimates of the neural responses were obtained from an L2 minimum-norm inverse of the forward model described above (Hämäläinen et al. 1993). We used the definition of the visual ROIs to constrain these estimates by modifying the source-covariance matrix. Our aim was to decrease the tendency of the minimum-norm procedure to smooth activity over very large surfaces and across different functional areas. Two modifications were applied: 1) We increased the variance allowed within the visual areas by a factor of 2 relative to other vertices, and 2) we enforced a local correlation constraint within each area using the first- and second-order neighborhoods on the cortical tessellation with a weighting function equal to 0.5 for the first order and 0.25 for the second. This correlation constraint therefore respects both retinotopy and areal boundaries and permitted us to dissociate the signals from different areas, unlike other smoothing methods such as LORETA that apply the same smoothing rule throughout cortex (Pascual-Marqui et al. 1994). The details of this approach can be found in Cottereau et al. (2011c).

ROI-Based Analysis of Steady-state Visual Evoked Potential

For each subject and condition, our inverse approach was applied to the average Fourier coefficients (see EEG Signal Acquisition and Preprocessing) and led to an estimation of these coefficients for each source of the cortical tessellation. Within each functionally defined ROI, the coefficients were then averaged across all the sources belonging to the area. We were particularly interested in their magnitude (i.e., their norm) at the odd (1st and 3rd) components of the steady-state frequency (2.12 Hz and 6.36 Hz). To take into account the difference of noise levels between the recordings from each of our subjects (Vialatte et al. 2010), we computed the signal-to-noise ratio (SNR) and divided these values by the root mean square of the associated noise, which is defined for a given frequency f by the average amplitude of the two neighbor frequencies (i.e., f − δf and f + δf, where δf gives the frequency resolution of the Fourier analysis). The corresponding SNRs are presented in decibels (20 × log₁₀).

Cross Talk

In a previous paper (Cottereau et al. 2011a), we described how we estimated the theoretical cross talk among visual areas for a specific EEG study. Cross talk refers to the neural activity generated in other areas that is attributed to a particular ROI, due to the smoothing of the electric field by the head volume. In brief, for each subject, we simulated the cross talk by placing sources in one ROI and estimating their contribution to other ROIs, using the same forward and inverse methods described above. The global cross talk matrix (i.e., averaged across all the subjects who participated in our EEG experiments) is shown in Fig. 2 for seven ROIs (V1, V2, V3, V4, LOC, V3A, and hMT+); the cross talk magnitude shown in the matrix is proportional to activity originating in the ROI where the cross talk is being estimated.

Fig. 2.

Simulation estimates of cross talk between source-imaged EEG signals in retinotopically defined visual areas. Grayscale values at row i and column j represent the relative contribution of area j to the cortical current density estimate in area i. The 5 visual regions of interest (ROIs) discussed in the present study are emphasized by the white dot square. LOC, lateral occipital complex.

From our simulations, it was apparent that there was significant cross talk in areas V2 and V3 (the last 2 rows of the matrix). For this reason, we excluded these two ROIs from our analysis and focused on V1, V3A, V4, hMT+, and the LOC. These areas are more widely separated, and their estimated activities are therefore more reliable. Within this subset of ROIs, the biggest cross talk comes from a V1 contribution to V3A (55%) and a V3A contribution to V4 (50%). We discuss the influence of cross talk coming from other regions that were not defined using fMRI in Effects of the Gap below.

Statistical Analysis

Repeated-measure analyses of variance (ANOVAs) were performed on the SNRs at the odd harmonics. We specifically tested effects of conditions and of ROIs and their interaction. Significant effects were followed up by post hoc pairwise Wilcoxon signed-rank tests. The validity of the ANOVAs (equal variance hypothesis) was controlled with Mauchly's sphericity test. The sphericity test showed no significant effect, so we did not correct the P values of the ANOVA.

RESULTS

Psychophysics

We measured stereoacuity thresholds for disparity modulation as a function of gap size in four subjects. The associated sensitivities (i.e., 1/threshold) are displayed in Fig. 3.

For all subjects, performance decreased when the gap was >1°. Interestingly, three subjects were better for small gap values than without any separation, suggesting that they were uncertain about the location of the central disk in the absence of the demarcation provided by the small gap. Indeed, the edge of the disk was invisible to most observers at behavioral threshold in the zero gap condition; instead, a central region of the disk appeared to dip slightly but at an unpredictable location from trial to trial. It is well known that uncertainty can reduce sensitivity (Green and Swets 1966; Pelli 1985) and that the addition of features that remove uncertainty about target location, such as the small gap surrounding the disk in this case, can improve sensitivity. For subjects RF and SPM, we were also able to measure a threshold for the control condition (uncorrelated surround). The two other subjects were unable to detect the change in disparity in the absence of the disparity reference. For all four subjects, the thresholds with a gap of 5.5° separating the disk from the correlated surround (the reference) were significantly better than the thresholds for the uncorrelated noise condition but were elevated from the best value by a factor of 3 to 9, as shown by the decline in sensitivity apparent in Fig. 3. The next section examines where these sensitivities are localized in visual cortex.

SSVEP Data

Figure 4 displays the amplitude spectrum for all the electrodes averaged across trials and subjects. Two conditions are presented, one in which no gap separated the center and surround (0°) and another with a 4° gap.

Fig. 4.

Amplitude spectrum for 0 and 4° gaps. Left: amplitude spectrum (averaged across 13 subjects) for the 128 electrodes (spectral resolution of 0.5 Hz). x-Axis is labeled with the harmonics of the stimulation frequency (i.e., f1 = 2.12 Hz for the modulation of the central disk and f2 = 21.25 Hz for the refresh rate of the local dots) to emphasize the precise time-locking of the steady-state visual evoked potential (SSVEP) to the stimulus. Colored boxes outline the amplitude at the first harmonic (1f1). Middle: topographic maps of the amplitude (in normalized units) of the 128 electrodes for the first harmonic. Right: estimate of the amplitude (in normalized units) of the first harmonic on 1 individual brain (back view). Data from all subjects were projected on this subject's anatomy and then averaged. A: results without gap (condition 1). B: results with a 4° gap (condition 5).

Over the-low frequency range, the spectrum has multiple, narrow peaks that occur at integer multiples of the stimulus frequency. Their amplitudes range from 3 to 20 times the noise level (defined here by the amplitude at nontagged frequencies). This result demonstrates that cortical responses are clearly entrained by the steady-state stimulation. The spectra also show a narrowband response at the dot refresh rate (i.e., f2 = 21.25 Hz). The brain is therefore also responding in a synchronous fashion to the refresh of the local elements in the stimulus. When the center and surround regions are abutted (no gap, Fig. 4A), the spectrum is dominated by first and third (odd) harmonics of the disparity update rate, which we refer to as 1f1 and 3f1. The introduction of a 4° gap strongly reduces the odd harmonic component of the response. Odd harmonics are produced by asymmetries in the responses to the two states of the stimulation, i.e., when the disk appears behind the surround and when it merges with it in the fixation plane. These asymmetries can be explained by the disparity tuning function in early visual areas. One of our previous EEG studies estimated these curves at the population level in V1, LOC, V4, hMT+, and V3A and found that in all these ROIs responses increase from 0 arcmin to 16 arcmin and then go back to their baseline level (Cottereau et al. 2011a). Responses at 12 arcmin are therefore stronger than those at 0 arcmin, leading to an asymmetry. In the 4° gap condition, the amplitudes at the odd harmonics and in the topography of the first harmonics suggest that, even though the cortical activity is diminished in this condition, the response is still mainly in the odd harmonics. The amplitude at the odd harmonics is therefore a good metric to characterize the cortical responses to the different conditions.

Effects of the Gap

The grand average topographic maps corresponding to the first harmonic suggest that the underlying neural activity arises mainly in the parietal and occipital cortex. This is confirmed by the surface-based average reconstruction (Fig. 4, right). Our question is a quantitative one—How does disparity processing in different cortical areas depend on the spatial separation of features? These cortical images are shown here only for visualization purposes; quantitative analyses will be performed in the five ROIs defined by fMRI retinotopy and standard localizers.

Figure 5 displays a composite measure of the odd-harmonic amplitudes in the five ROIs as a function of the gap size. The responses obtained when the surround was uncorrelated (control condition) are displayed by the filled areas in Fig. 5C (mean and standard error). In V1 as well as in V4, LOC, hMT+, and V3A, the amplitudes in the absence of a gap are larger than those obtained in the control condition, suggesting that responses in these ROIs are enhanced by the presence of the correlated surround. In our previous work (Cottereau et al. 2011a, 2012), the V4, LOC, hMT+, and V3A ROIs were each found to be influenced by the addition of a disparity reference, i.e., the surround. In light of our previous findings and single-cell studies (see e.g., Cumming and Parker 1999), we did not expect any surround (relative disparity) effects in V1. The activity observed with the 0° gap could arise from cross talk from all the areas surrounding V1, but particularly the V3A area (see Cross Talk above for more details), whose SNR for the no gap condition is particularly large.

Fig. 5.

Signal-to-noise ratios (SNRs) at the odd harmonics (i.e., 1f1 and 3f1) in 5 visual ROIs for different gap sizes. A: example of visual ROI definition on 1 single subject. B: SNR average values and associated SEs (13 subjects). Results in V1, ventral (V4 and LOC), and dorsal (hMT+ and V3A) ROIs are displayed in red, green, and blue respectively. Striped areas give the confidence interval for the responses with an uncorrelated surround (solid line is the average amplitude, and dashed lines are 1 SE above and below). C: temporal waveforms in normalized units (n.u.) estimated in areas V3A and V4 for 1 cycle (2.12 Hz) of the central disk disparity modulation. Responses with an uncorrelated surround (dotted lines) and with a 4° gap between center and surround (solid lines) are presented. The data were low-pass filtered at 20 Hz. Shaded contours give SEs. The first 236 ms where the disk is at 12° are emphasized by a gray box.

Taken across all the gap values, the presence of a correlated surround enhanced the SNRs at the odd harmonics compared with the activity obtained with the uncorrelated surround. This was confirmed by a repeated-measures (2 surround conditions) × (5 ROIs) ANOVA where the two surround conditions (correlated and uncorrelated) corresponded to the average SNRs across the six gap values on the one hand and the SNR for the uncorrelated surround on the other hand. This ANOVA supported a surround effect (P < 0.05) and also a significant interaction (P < 0.01) between ROI and the nature of the surround. Post hoc signed-rank tests showed that only V3A (P < 0.01) and V4 (P < 0.05) activity were significantly higher than the control when averaged across all gap conditions. The possibility that the responses in V4 are due to cross talk from V3A is ruled out by the results of the second experiment (see next section) that clearly demonstrate that the activity in V4 and V3A are different.

Generally, the effect of the gap is to decrease the SNRs. This was confirmed by a (6 gap conditions) × (5 ROIs) ANOVA that led to a main effect of gap size (P < 0.001) and also to an effect of the ROIs (P < 0.05). When a small gap (∼1°) is introduced, the amplitudes in V1, LOC, and hMT+ are not different from those obtained in the control condition. Therefore, V1, LOC, and hMT+ may not contain neurons capable of computing disparity differences over large distances. These results suggest that the disparity responses observed in LOC and hMT+ (Cottereau et al. 2012) arise from a limited spatial region of the stimulus near the edge of the modulating disk. However, the V3A ROI and, to a lesser degree, the V4 ROI have amplitudes at the odd harmonics that remain significantly above those from the control condition for gap values ranging up to 5.5°. This result is apparent in the temporal waveforms associated with the disk disparity modulation and is shown in Fig. 5C. The waveforms in areas V3A and V4 were obtained from an inverse Fourier transform of the Fourier coefficients and were normalized for each subject by dividing by the standard deviation of the response amplitudes across all conditions and ROIs. This normalization lessens interindividual differences in current. The responses were also low-pass filtered at 20 Hz. The dashed curves in Fig. 5C show the time courses with an uncorrelated surround and the solid curves the time courses with a 4° gap. The amplifications of the higher odd harmonics are clearly visible when the disk is at 12 arcmin (i.e., during the 1st half-cycle of the disparity modulation) and not aligned in depth with the surround. This is consistent with the observations made in our previous study (Cottereau et al. 2012). These results support the idea that V3A and V4 are able to process disparity information over large distances.

Spatial Resolution of ROI-Based Reconstruction Technique

As discussed in Cottereau et al. (2011a), electromagnetic imaging gives a blurry representation of the sources responsible for the measured scalp activity; the spatial resolution of this technique is poorer than fMRI measurements [see, however, Cottereau et al. 2011b for a good match between the retinotopic reconstructions obtained from the technique employed in the present study but applied to magnetoencephalography (MEG) imaging and fMRI estimates]. Thus the ROIs should be treated as a cortical region surrounding the designed label. For example, the SNR labeled “V3A” in Fig. 5B certainly represents source activity from V3A, but it may also include additional activity from adjacent areas, such as V3, V3B, or V7, as well. We have estimated the cross talk among the five ROIs (see materials and methods) using simulations, but we did not attempt to estimate cross talk from other adjacent anatomical regions, e.g., V3B or V7, since we had no rigorous way of defining them. Similarly, V4 activity may be partially dependent of other ventral regions such as VO-1 or VO-2 (Brewer et al. 2005). It is interesting that our simulations show that the V3A ROI receives a large amount of cross talk from V1 but our empirical results show, however, that activity estimated in the V4 and V3A ROIs does not resemble that attributed to the V1 ROI. Although our topographic maps of the cortical activity demonstrated that the disparity responses to our stimuli were mainly localized in occipito-parietal regions (see, e.g., Fig. 4), a limitation of our approach is that it cannot characterize the responses in areas not functionally defined in fMRI. Several regions located in more medial and anterior portions of the parietal cortex have been shown to respond to disparity stimulation. For example, areas V6 (Cardin and Smith 2011), dorsal intraparietal sulcus medial (DIPSM), or dorsal intraparietal sulcus anterior (DIPSA) (Durand et al. 2009) might have been activated in our experiments. It would be interesting to localize these areas and include them in the analysis in future studies of the disparity system.

Are Disparity Interactions Processed Through Spatial Integration?

The psychophysical measures demonstrate that stereoacuity is enhanced by a reference stimulus located as far as 5.5° away in the central visual field. Our EEG results also show that a surround that is located at 5.5° from the modulation stimulus enhances the response to disparity modulation in two extrastriate areas. It is thus possible that these two areas underlie the behavioral results, which demonstrate the effect of a reference disparity on sensitivity. How does this reference effect work? One could imagine a network of connections between any pair of disparities within a 5.5° region. These long-range connections could encode the disparity differences among all local stimuli, potentially without mutual interference by “jumping over” intervening stimuli to connect to other stimuli—something like an old-fashioned switchboard. It is, however, more likely that these reference effects depend on spatial integration within large receptive fields that integrate the responses across the surround.

To explore the nature of the disparity integration process, we performed another experiment that tested how the V3A and V4 ROIs were integrating disparity information across space. As before, the disparity of the central disk was modulated over time and was surrounded by an annulus of constant disparity. However, the annulus was smaller than in the main experiment (7.8° outer diameter) and was presented in one of three different configurations (see diagrams in Fig. 6A). In two conditions (C1 and C2), the annular surround was separated from the central disk by a 1° space that was either empty (i.e., a gap) or filled with uncorrelated noise. In a third condition (C3), the surround abutted the center and consisted of a mixture of 50% correlated and 50% uncorrelated dots. Because we had reduced the size of the annulus to make the area between center and surround equal to the area of the surround, the amount of noise in conditions C2 and C3 was proportionally the same. Our hypothesis was that if V3A and V4 were performing integration over space, their responses at the odd harmonics for conditions C2 (noise between center and surround) and C3 (mixed surround) should have been diminished because noise would be introduced in their disparity integration. Following the same idea, the comparison between odd harmonic responses to conditions C2 and C3 should provide information about the integration profile. Lower SNRs for C2 would imply that the area directly surrounding the center has a bigger impact on the global integration (i.e., the closer the surround, the bigger the interaction). On the other hand, equivalent SNRs for these two conditions would suggest uniform integration within the receptive fields of the neural population mediating these long-distance interactions. Figure 6 displays the SNRs obtained in the V3A and V4 ROIs in this experiment.

Fig. 6.

Results of the second experiment. A: schematic display of the 3 conditions. For the first 2 conditions, the gap was either empty (C1) or filled with uncorrelated noise (C2). In the third condition (C3), there was no gap but the surround was a mixture containing correlated and uncorrelated dots (50–50 ratio). B: SNRs at the global (odd harmonics) frequencies in the 2 visual areas that were the most responsive in the first experiment (i.e., V3A and V4 emphasized in the box). Barred areas correspond to the confidence interval for the control condition (uncorrelated surround). **Statistical difference with P values < 0.05 (post hoc signed-rank tests).

Both the V3A and V4 SNRs vary with condition, which was confirmed statistically with a 2 (ROIs) × 3 (Surround nature) repeated-measures ANOVA. There were main effects of condition (P < 0.01) and an interaction between condition and ROI (P < 0.05). There was also a marginal effect of the ROI (P = 0.062). In V4, post hoc signed-rank tests did not show any statistical difference in the SNR of the three conditions. In the V3A ROI, the SNR for condition C2 was significantly weaker than that obtained for conditions C1 and C3 (P < 0.01), but the SNRs in conditions C1 and C3 were not statistically different.

The results of this experiment show that the integration profiles are different for the neural populations of the V4 and V3A ROIs. The SNRs in the V4 ROI are little affected by adding noise to the surround no matter how it is distributed, although having an abutting surround with no noise would undoubtedly produce a better SNR than any of the three conditions tested (see 0 gap in Fig. 5). In the V3A ROI, introducing noise between the center and the surround reduces the SNR, but distributing the noise throughout an abutting surround has the same effect as introducing a 1° gap.

One likely explanation for these results is that the disparity-sensitive populations in these two ROIs are heterogeneous, consisting of neurons with variable-sized receptive fields and with different functionality. In the V4 ROI, there may be many neurons with relatively small receptive fields that are sensitive to the disparity difference between the center and the surround. This subset would respond most strongly when the surround is abutting the modulating center. Adding a gap or putting noise in the space between center and surround effectively removes this group from the population response, resulting in a smaller SNR in the V4 ROI. However, there must also be some neurons within this ROI with large receptive fields that can respond to the disparity difference over fairly substantial distances (up to 5.5°). This group may be less numerous or less sensitive than the group with relatively small receptive fields, resulting in weak, but significant, SNRs in the presence of a sizable distance between center and surround (>1°). The responses from this group may be intrinsically noisy, so that the addition of stimulus noise does not have much effect on the response. Alternatively, our recording from this area may be noisier than from other areas, so that again the added stimulus noise is not strong enough to exceed the inherent noise in the measurements.

To account for the different pattern in the V3A area, we speculate that many disparity-sensitive neurons, possibly including those with large receptive fields, are strongly responsive to the disparity “edge” in our stimuli (Anzai et al. 2011; Mendola et al. 1999). Disparity edge detectors have been observed in macaque area V2 (Bredfeldt and Cumming 2006), and cells in V3A may have inherited these properties. The gap removes the response from the group with small receptive fields, but leaves a sizable group of larger units active to “bridge the gap.” Adding noise between the center and the surround significantly degrades the strong response from the larger receptive fields. Distributing the noise uniformly across the surround not only reduces the average noise level per unit area, but it also introduces some signal dots near the center that can drive the response of small- and moderate-sized units so that the response returns to same level as that produced by the 1° gap.

DISCUSSION

Human observers can detect differences in disparity over substantial distances. Our psychophysical results show that our subjects' sensitivity was still better when a surround was displayed 5.5° away from the central test target than when it was surrounded by disparity noise. Our EEG recordings suggest that neurons in or near V4 and V3A respond to disparity differences over large enough distances to encode these differences for subsequent processing. Of the areas we studied, only these two areas retained odd-harmonic responses that were larger than the uncorrelated surround responses at large separations (5.5°) between center and surround regions. It is possible that the LOC and hMT+ ROIs also contain a subpopulation that responds to long-distance disparity differences. The response of this small population may disappear in the noise of our measurements, although it should be noted that the response of these two ROIs, in the absence of a gap, is about as robust as the V4 ROI (see Fig. 5 and Cottereau et al. 2012). We therefore conclude that neurons in the V4 and V3A ROIs are more likely to carry long-distance disparity information than LOC and hMT+.

A modulation of the evoked response that depends on the correlated surround is a form of relative disparity sensitivity. Of the early visual areas in macaque, area V4 has the largest proportion of cells that shift their disparity tuning with changes in the disparity of a surround, indicating that these cells encode relative disparity, invariant to changes in absolute disparity—at least over some range (Umeda et al. 2007). In humans, this result has been supported by an fMRI study using adaptation (Neri et al. 2004) that showed that V4 contains neurons responsive to relative disparity. There is strong evidence that fine stereoacuity depends on the ventral pathway. Uka, Tanabe, Watanabe, and Fujita (2005) found that binocularly tuned neurons in area IT, which receives much of its input from V4, showed significant choice probabilities to disparity differences well under 0.5 arcmin. Read et al. (2010) measured stereopsis in patient DF, who had experienced significant bilateral damage to her ventral visual pathway. Although DF could make coarse judgments of absolute disparity that were equal or better than normal subjects, her fine stereoacuity, based on relative disparity, was significantly degraded by the damage to her ventral pathway. Cells in the dorsal pathway are believed to be primarily driven by absolute disparity (Neri 2005). In primate V3A, recent single-cell recordings have supported this view (Anzai et al. 2011). However, the disparity-driven responses of ∼40% of these cells are modulated by the presence of a second disparity surrounding their receptive fields, which suggests a form of relative disparity processing (Cottereau 2011), even though they do not show the phase changes in disparity tuning observed in V4. In humans, population-level recordings in fMRI and EEG have shown that area V3A is strongly affected by the presence of a second reference in the visual field (Backus et al. 2001; Cottereau et al. 2011, 2012; Minini et al. 2010; Preston et al. 2008; Tsao et al. 2003). Moreover, Backus et al. (2001) showed that V3A was highly sensitive to small disparity differences (1 arcmin or less).

These observations are consistent with our results that suggest that V4 and V3A are implicated in the long-distance disparity judgments measured psychophysically. The idea that the dorsal and ventral pathways are both involved in 3D discrimination has been suggested in the electrophysiology literature (Orban et al. 2006) and in computational modeling (Chinellato and Del Pobil 2009). This connection was recently confirmed between areas IT and AIP in macaque (Verhoef et al. 2011), two areas that receive direct inputs from V4 and V3A (Nakamura et al. 2001; Orban et al. 2006; Sakata et al. 1997), respectively. Future studies should address more specifically the nature of the interactions between the two pathways during depth perception.

Stereopsis is not the only dimension in which sensitivity is enhanced by a reference target. Although there have been no neurophysiological studies on “relative motion,” human sensitivity for judging direction of target motion is also much better in the presence of a reference than when the target is viewed in isolation (Levi et al. 1984; McKee et al. 1990). Intriguingly, the distance over which the reference exerts a benefit on motion sensitivity is comparable to the long-range effect described here for stereopsis. However, not all human judgments in space and time show these long-range reference effects. For example, vernier acuity falls off markedly as the two target components are separated (Berry 1948; Levi and Klein 1982; Westheimer and McKee 1977; Whitaker 1993). Norcia et al. (1999) found a parallel difference between relative motion and vernier acuity in their visual evoked potential study. They observed that the amplitude of response components associated with vernier offsets in their grating target fell dramatically with even small gaps (10 arcmin or less) between the target components, while the amplitude of response components associated with relative motion were unaffected by these gaps. It may be that some neural mechanisms are particularly sensitive to local discontinuities in texture or lines, while other mechanisms are responsive to stimulus differences over substantial distances. For example, Xu et al. (2005) found strong surround suppression in V1 neurons that was contingent on the relative phases of gratings in center and surround; this phase-dependent suppression disappeared in the presence of a gap, but a residual “long-range” suppression effect remained that was independent of phase. Any mechanism that is involved in encoding figure-ground would function most effectively if driven by local discontinuities; figure-ground discrimination is superfluous if the figure is defined by a sizable gap between figure and ground. The results of our second experiment in V3A suggest that the population in this area may be composed of a mixture of neurons responsive to local disparity discontinuities and neurons responsive to long-range differences in stimulus components.

GRANTS

This work was supported by National Eye Institute Grant R01 EY-018875, the Smith-Kettlewell Eye Research Institute, and a Walt and Lilly Disney Amblyopia Research Award from Research to Prevent Blindness.

DISCLOSURES

No conflicts of interest, financial or otherwise, are declared by the author(s).

AUTHOR CONTRIBUTIONS

Author contributions: B.R.C., S.P.M., and A.M.N. conception and design of research; B.R.C. and S.P.M. performed experiments; B.R.C. analyzed data; B.R.C. and S.P.M. interpreted results of experiments; B.R.C. prepared figures; B.R.C. and S.P.M. drafted manuscript; B.R.C., S.P.M., and A.M.N. edited and revised manuscript; B.R.C., S.P.M., and A.M.N. approved final version of manuscript.