The mixed-polarity benefit of stereopsis arises in early visual cortex

Abstract

Depth perception is better when observers view stimuli containing a mixture of bright and dark visual features. It is currently unclear where in the visual system sensory processing benefits from the availability of different contrast polarity. To address this question, we applied transcranial magnetic stimulation to the visual cortex to modulate normal neural activity during processing of single- or mixed-polarity random-dot stereograms. In line with previous work, participants gave significantly better depth judgments for mixed-polarity stimuli. Stimulation of early visual cortex (V1/V2) significantly increased this benefit for mixed-polarity stimuli, and it did not affect performance for single-polarity stimuli. Stimulation of disparity responsive areas V3a and LO had no effect on perception. Our findings show that disparity processing in early visual cortex gives rise to the mixed-polarity benefit. This is consistent with computational models of stereopsis at the level of V1 that produce a mixed polarity benefit.

Introduction

Depth perception is better when observers view stimuli that contain a mixture of bright and dark visual features. Harris and Parker (1995) showed that a random-dot stereogram (RDS) with a noisy disparity profile allows for better depth judgments when it contains black and white dots (mixed polarity) compared to when it only contains one dot color (single polarity; see Figure 1).

Figure 1.

(A) Mixed- versus single-polarity stereograms. Single-polarity stereograms were either all dark or all bright. (B) The task was to discriminate the step arrangement of the stereogram. Stimulus disparity was comprised of a disparity step to which crossed and uncrossed disparity noise was added (sampled from Gaussian distribution centered at stimulus location). (C) Example anaglyphs that illustrate the mixed-polarity benefit (designed for red filter over left eye).

An unanswered question is where in the visual cortex disparity processing benefits from the availability of different contrast polarity. The stimulus in Figure 1 might challenge both early mechanisms that establish stereo correspondence and/or subsequent mechanisms of disparity discrimination. Read, Vaz, and Serrano-Pedraza (2011) were able to replicate the mixed-polarity benefit using a decorrelated RDS: Instead of binocular dot pairs having different horizontal offsets, in a decorrelated RDS some dots do not have a match in the other eye. This stimulus should challenge mechanisms of stereo correspondence more strongly, and indeed, they observed a stronger mixed-polarity benefit for decorrelated RDSs. This indicates that early mechanisms of disparity processing might benefit from mixed contrast polarity. However, it remains unclear where in the brain these mechanisms are located.

Here we sought to answer the question of where in the brain the mixed-polarity benefit arises. We applied transcranial magnetic stimulation (TMS) to early (V1) and higher (V3a and lateral occipital complex [LO]) visual brain areas, which have been shown to be involved in disparity processing (Goncalves et al., 2015; Patten & Welchman, 2015; Preston, Li, Kourtzi, & Welchman, 2008). We assume that the benefit is produced by a neural mechanism that extracts a more reliable disparity signal from mixed-polarity compared to single-polarity RDSs. By changing normal neural activity in this system through brain stimulation, we expect to differentially disrupt stereopsis for mixed- and single-polarity stimuli. This allows us to locate where in the visual cortex disparity processing benefits from the additional information carried by mixed contrast polarity.

We found that stimulation over V1 but not V3a or LO affected depth perception for mixed- but not single-polarity stimuli. This confirms that mechanisms of stereopsis in primary visual cortex, which are concerned with stereo correspondence, give rise to the mixed polarity benefit. Contrary to our expectation, TMS over V1 does not disrupt stereopsis. Instead, brain stimulation amplifies the mixed-polarity benefit by improving depth perception for mixed-polarity stimuli. We suggest two potential explanations for this surprising result: TMS might amplify disparity signals for mixed-polarity stimuli due to nonlinear processing in visual cortex. Alternatively, TMS might drive suppression of binocular contrast mismatches for mixed-polarity stimuli, which could improve the reliability of disparity signals.

Methods

Participants

For this study, we screened 83 naïve participants. All had normal or corrected-to-normal vision with good visual acuity (between −0.1 and 0.1 LogMAR). We screened participants with the demanding depth-discrimination task used in this experiment (see Experiment procedure). Twenty-two participants successfully passed the screening and were tested for this study. Before the experiment, participants provided written informed consent and were checked for contraindications to functional magnetic resonance imaging (fMRI) and TMS (Rossi, Hallett, Rossini, & Pascual-Leone, 2009; Wassermann, 1998). Procedures were approved by the University of Cambridge ethics committee and were performed in accordance with the ethical standards laid down in the 1964 Declaration of Helsinki. For this study, we initially tested seven participants for which all brain regions of interest received TMS. Another eight participants were tested in a shorter version of the experiment to confirm that the main finding holds for a larger sample size. Additionally, seven participants were tested in a replication without the acquisition of fMRI data. For this replication, we targeted the primary visual cortex and a control site with TMS which should be feasible without fMRI-based neuro-navigation.

Stimuli

Participants performed the experiment task with a haploscope in which the two eyes viewed separate 22-in. Samsung (2233) LCD displays through front-silvered mirrors. Both screens were gamma-corrected to linear luminance output. Viewing distance was 50 cm. Stimuli were displayed on 1,680 × 1,050 pixel screen at a vertical refresh rate of 60 Hz. Participants were instructed to maintain fixation on a square fixation cross at the center of the screen with horizontal and vertical nonius lines.

Stimuli were RDSs (dot radius: 0.068°, number of dots: 492, stimulus size: 4° × 4°) depicting a noisy disparity-based step function on a medium gray background (see Figure 1). Participants performed a two-alternative, forced-choice task and were asked to judge whether the top or bottom half of the stimulus appeared closer to them. Stimuli were surrounded by a correlated pink noise background to promote stable vergence.

Task difficulty was manipulated by changing the magnitude of the step function relative to the fixation point (i.e., we simultaneously varied the crossed and uncrossed disparities in tandem). Additionally, each dot in the RDS was randomly assigned crossed or uncrossed Gaussian disparity noise (see Figure 1B). Given individual differences in stereoscopic capabilities of naïve participants and the fact that discrimination performance with this type of task can improve substantially through training (Chang, Kourtzi, & Welchman, 2013), we varied both the step size and variance of the disparity noise during the training portion of the experiment. We thereby tailored the stimuli to the participant's discrimination capabilities. During the main experiment, only the step size was changed to manipulate task difficulty. Disparity at the left and right edges of the RDS was tapered to zero to avoid monocular cues of the relative position in depth.

Stimuli were displaced to the left or right of the fixation point to maximize the amount of information processed in one hemisphere and thereby increase the potential to reveal the effects of brain stimulation. Specifically, the stimulus was displaced 2° horizontally so that one edge of the stereogram became aligned with the center of the screen. Stimuli were presented to the left of fixation on two thirds of trials (corresponding to the stimulated right hemisphere) and to the right of fixation on one third of trials (corresponding to the unstimulated hemisphere). This imbalance represented the intersection between the need to acquire sufficient data in an individual condition and the limit of how many TMS pulses can be safely applied per day (Rossi et al., 2009; Wassermann, 1998). All testing conditions in this experiment had this same imbalance; we recorded eye movements to ensure that there was no significant left-side bias of version eye movements.

Stimuli were rendered for two different RDS conditions: In the single-polarity condition, all dots of the RDS were either white or black. In the mixed-polarity condition, 50% of the dots were white, 50% black. For a given experiment block, we always presented 50% mixed-polarity and 50% single-polarity stimuli. We also balanced the number of black and white single-polarity RDSs to ensure equal overall light exposure between the mixed- and single-polarity conditions.

Experiment procedure

Stimuli were presented for 300 ms, and observers had unlimited time to give a response. If no TMS was applied during the experiment, we kept an interstimulus interval (ISI) between response and stimulus onset of 800 ms. For trials that were accompanied by TMS, the ISI was a jittered period between 5 and 6 s. A longer average ISI was chosen to contain TMS effects within the trial duration (Kammer, Puls, Erb, & Grodd, 2005), and the timing was jittered to avoid stimulation effects that might build up through rhythmically applied TMS.

At the beginning of the experiment, participants performed a 1-hr training session to familiarize themselves with the task: Participants viewed single- and mixed-polarity RDSs with a range of step sizes of 0.2–20 arcmins. They received feedback on their judgments. During training, the Gaussian disparity noise was increased stepwise from σ = 1 to σ = 3 arcmins to define an optimal noise level at which a range of RDS step sizes would yield performance from near chance to near perfect performance. Participants trained for between 576 and 1,152 trials until a stable performance was reached. During the main experiment, RDSs always contained the disparity noise magnitude tailored to each participant based on the training results.

Next, we defined a psychometric function for each stimulus polarity condition using the method of constant stimuli (MOCS). For each polarity condition, we presented stimuli at seven disparity step sizes (between 0.2 and 20 arcmin) with 108 trials per stimulus intensity level (total number of trials 2,268). Trials of different difficulty and polarity were randomized. For each stimulus condition, a psychometric function was fitted to the data using psignifit [4.0] (Fründ, Haenel, & Wichmann, 2011). We also presented 360 catch trials for lapses (step size 10/20 arcmin, σ = 0 arcmin) to fix the lapse rate of the psychometric functions. Lapse rates were combined for different stimulus conditions. Eight participants were unable to fuse the largest disparity step size of the MOCS procedure (which was also used as lapse stimuli). For these participants, psychometric functions were fitted to only six difficulty steps, and a group-average lapse rate was used to define the upper asymptote.

From the psychometric function for mixed polarity, we estimated a threshold of 80% correct performance. Based on previous results, performance differences between single and mixed polarity should be largest at this performance range (Read et al., 2011). For the subsequent main experiment, stimuli were presented at the task difficulty that was estimated to yield 80% correct discrimination for mixed-polarity stimuli.

The experiment consisted of five different TMS conditions: In one condition, no TMS was applied; in the remaining conditions, TMS was applied over V1, V3a, LO, or Cz during stimulus presentation (see Supplementary Figure S1A). Each condition included a total of 456 trials.

Participants were tested on eight separate days so that we did not exceed the maximum number of stimulations that can be safely applied per day (Rossi et al., 2009; Wassermann, 1998). Each stimulation site was targeted on two different days. We only targeted one brain site per day to avoid confounding effects of TMS to different networks in the brain. The no-TMS condition was tested on the same days as TMS conditions prior to the application of brain stimulation to avoid carryover effects. The order of stimulation sites between testing days was randomized to average out potential training effects of disparity discrimination that might build up throughout the experiment.

TMS

We applied stimulation with a MagStim Rapid² stimulator (MagStim, Whitland, UK), using a figure-of-eight coil (70-mm outer diameter). The TMS coil was placed tangentially on the head, aiming at the defined region of interest in the brain (see Supplementary Figure S1B). A coil holder (Magic Arm, Manfrotto, Bassano del Grappa, Italy) retained the coil at its position on the head. Stimulation was applied to the right hemisphere. The right hemisphere was chosen based on previous success of right LO stimulation with a similar depth judgment task (Chang, Mevorach, Kourtzi, & Welchman, 2014). For stimulation targets V1 and V3a, we had no reason to expect any hemispheric differences.

Stimulation was applied at 10 Hz (five pulses, 0.4 s) synchronous with stimulus onset at a fixed intensity of 60% of maximum stimulator output (see Supplementary Figure S1A). For all stimulation targets, coil orientation was defined based on previous reports of successful TMS stimulation. Specifically, for V1, the coil handle was facing to the left (current direction medial to lateral; Mulckhuyse, Kelley, Theeuwes, Walsh, & Lavie, 2011). For V3a (McKeefry, Burton, Vakrou, Barrett, & Morland, 2008) and LO (Chang et al., 2014), the coil handle was facing upward (current direction superior to inferior). For control stimulations at Cz, the coil handle was facing from the front to the back of the head (current direction anterior to posterior; Chang et al., 2014). For participants with available MRI data, the orientation of the coil was subsequently adjusted based on the underlying anatomical structure of the brain. This was done to ensure that the induced electric current ran perpendicular to the underlying sulcus and thereby maximized the likelihood of neural activation through stimulation (Janssen, Oostendorp, & Stegeman, 2015; Laakso, Hirata, & Ugawa, 2014; Thielscher, Opitz, & Windhoff, 2011).

fMRI

fMRI data were collected on a 3T Siemens Prisma MRI scanner with a 32-channel head coil. Blood oxygen level–dependent signals were measured with an echo-planar imaging sequence (TE 29 ms; TR 2000 ms; 1.5 × 1.5 × 2 mm, 30 slices covering the visual cortex). For each participant, we acquired a high-resolution anatomical scan (1 mm³). fMRI data was analyzed with BrainVoyager QX [2.8] (Brain Innovation, Maastricht, The Netherlands; Goebel, Esposito, & Formisano, 2006). Functional data were preprocessed using three-dimensional motion correction, slice time correction, linear trend removal, and high-pass filtering. Retinotopic areas V1 and V3a were defined with standard retinotopic mapping procedures using rotating wedge stimuli and expanding ring stimuli. The borders of functional areas were defined by the resulting angular and eccentricity maps (Wandell, Dumoulin, & Brewer, 2007). The LO was mapped as the set of voxels that responded significantly (p < 0.01) stronger to intact than scrambled images of objects (Kourtzi, Betts, Sarkheil, & Welchman, 2005). Supplementary Figure S1C shows probability maps of V1, V3, and LO position for all participants in Talairach space.

Neuro-navigation

During the experiment, an anatomical scan was coregistered to the participant's head using anatomical landmarks. For all participants with MRI data, their individual structural scan was used; for all remaining participants, an average MNI 152 head was fitted using linear transformation and scaling (Brainsight 2.2.12; Rogue Research, Montreal, Quebec, Canada). During the experiment, we monitored the position of the TMS coil and the participant's head with an infrared camera and Brainsight 2.2.12 neuro-navigation software.

A normal vector originating in the center of the figure-of-eight TMS coil described the expected stimulation location in the brain (see Supplementary Figure S1B). For participants with fMRI data, stimulation targets were defined as the center of a region of interest (V1, V3a, or LO) in the brain. For participants without fMRI data, V1 was defined as a point 5 mm lateral of Oz (10–20 system). For each target, an ideal trajectory was defined approximately normal to the scalp surface. The precision of stimulation during the experiment is described by three coil position parameters (targeting error, angular error, and tilt error), which are described relative to this ideal trajectory (see Supplementary Figure S1B). During the experiment, coil-position parameters were monitored and recorded (see Supplementary Figure S1C).

Electric field simulation

For the seven participants for whom all areas of interest were stimulated, we created an electric-current model of TMS to investigate whether stimulation successfully targeted V1, V3a, and LO. We used simNIBS [2.0] (www.simnibs.org; Thielscher, Antunes, & Saturnino, 2015) to model current distributions with a finite-element method. We constructed detailed meshes (1.1 million tetrahedra) from the structural MRI scans and modeled electrical field spread. We assigned electrical conductivities to different tissue types as described by Windhoff, Opitz, and Thielscher (2013). Isotropic conductivity in the brain was assumed. A magnetic-dipole model for a MagStim 70 mm figure-eight coil was provided by simNIBS. We defined coil position and orientation in the simulation as the mean position and orientation recorded during the experiment with neuro-navigation. Rate of change of current flow in the stimulator coil for a given stimulator output was defined relative to the peak current at 100% stimulator output as provided by MagStim. Because the output is a sinusoidal waveform, current flow in the coil was calculated as the root mean square of the peak current for a pulse duration of 300 μs. For a stimulator output of 60% used in this study, this results in a rate of change of current flow of 20.08 A/μs.

Eye tracking

We recorded binocular eye movements with an Eyelink 1000 remote video tracker (SR Research). The system has a stated accuracy of 0.25° and resolution of 0.01° (root mean square). The tracker viewed the participant's eyes through infrared-transmitting cold mirrors. At the beginning of each experiment block, participants were instructed to keep fixating on a calibration marker, which was used to calibrate a 4° × 4° area on the screen in which stimuli were presented.

To analyze eye movement data, we converted raw gaze positions to degrees of visual angle. Trials during which tracking was lost in one or both eyes were excluded (average proportion of trials per participant 25.7%). This high proportion of lost trials was due to the challenge of tracking both eyes through the mirrors and eyeholes of the stereoscope. Time-series data were preprocessed by removing any data that corresponded to periods of blinks or saccades as identified by the EyeLink inbuilt detection functions. We removed an additional 50 ms of data before and after blinks to remove large gaze point offsets that were likely caused by eye rotation prior to blinks. Removed data was then linearly interpolated. Finally, eye tracking in a stereoscope sometimes led to erroneous tracking of interior parts of the stereoscope instead of participant's pupils (average proportion of trials per participant 7.9%). To remove all trials in which this occurred, we excluded all trials in which gaze position was located outside of a 4° × 4° window around fixation where stimuli were presented in this study.

We report vergence and horizontal version eye movements during stimulus presentation to check that brain stimulation and lateralized stimulus presentation did not interfere with vergence stability. To quantify changes of vergence through TMS, we fit a linear model to participants' average eye vergence during stimulus presentation. We quantify vergence changes on each trial in terms of the gradient (β) of the best fit (least-squares).

Analysis

Statistical analysis was conducted in SPSS (SPSS Inc., Chicago, IL). We analyzed raw proportion–correct values using repeated-measures ANOVAs and applied Greenhouse–Geiser correction where appropriate. For post hoc analysis, we used Bonferroni corrected t tests.

Results

Participants were tested for each polarity condition (black, white, or mixed dots) at a range of disparity differences and at a fixed disparity noise level. Figure 2 shows the psychometric functions that were fitted to the data. We found significant differences for thresholds of psychometric functions for black, white, and mixed stimuli, F(2, 36) = 16.97, p < 0.01. Participants had significantly better depth perception (lower disparity acuity thresholds) when a mixture of black and white dots was presented compared to only white, t(19) = −4.25, p < 0.01, or black, t(19) = −4.73, p < 0.01, dots (comparison of 80% correct performance thresholds). Depth discrimination was marginally better for white stimuli compared to black stimuli (t[18] = −2.36, p = 0.03). However, this difference was not significant after Bonferroni correction.

Figure 2.

Discrimination performance for RDSs with black dots, white dots, and a mixture of black and white dots. Task difficulty is defined by two parameters: the disparity offset in the RDS (x-axis) and the disparity noise assigned to each dot in the RDS (sampled from Gaussian with a SD (σ) in arcmin). Psychometric functions were fitted to mean proportion correct responses. For 14 participants, the upper asymptote was set to performance at lapse trials (see Methods). For eight participants no lapse data was available (marked with *) and the upper asymptote was set to the average group lapse rate. Vertical lines mark the threshold at which participants performed at 80% correct for each condition.

Next, we applied brain stimulation during the task to locate where in the visual cortex the mixed polarity benefit arises. For seven participants, brain stimulation was applied over all areas of interest in the visual cortex (V1, V3a, LO). The application of TMS did not significantly change disparity discrimination performance for all TMS conditions, F(4, 24) = 1.74, p = 0.17; however, a significant interaction between TMS site and stimulus dot polarity F(4, 24) = 5.8, p < 0.01, shows that TMS location is a critical factor in affecting the perceptual benefit between mixed and single polarity (see Figure 3A). Post hoc comparisons revealed significant improvements in disparity-discrimination performance through V1 stimulation compared to control stimulation for mixed-polarity stimuli, t(6) = 3.37, p = 0.015. For stimulation of higher visual areas V3a and LO, we did not observe a significant change in performance for mixed-polarity stimuli: V3a, t(6) = 1.86, p = 0.11; LO, t(6) = 0.66, p = 0.53.

Figure 3.

Mean discrimination performance for different stimulation conditions. Error bars depict 1 SEM. Results are shown for stimulus location left of fixation. Brain stimulation was applied to right hemisphere visual cortex. (A) Results for all stimulation conditions investigated in this study (n = 7). (B) Results for V1 stimulation investigated for a larger sample size (n = 15). (C) Replication of V1 stimulation effect of main experiment (n = 9). Additionally, task difficulty for mixed and single polarity was adjusted so that participants had similar proportions of correct responses for both stimulus conditions prior to brain stimulation. This was done to control whether TMS effects depend on task performance.

To assess which areas of the visual cortex were critically influenced by TMS, we simulated intensity and spread of the electric field induced by TMS for all participants that were stimulated over V1, V3a, and LO (n = 7). Figure 4 shows representative electric fields in one participant simulated for stimulation of all target areas. Overlaid are the boundaries of functional areas defined by retinotopy and localizer scans. For higher visual areas V3a and LO, mean electric field intensity was highest in the targeted area (see Supplementary Figure S2A). Stimulation over V1, on the other hand, may have induced a stronger electric field in area V2d (this was true for all participants). However, although electric field intensities in V2d were similar for stimulation over V1 and V3a, only TMS over V1 produced significant behavioral changes. We, therefore, calculated a measure of behaviorally relevant electric field intensity by subtracting electric field intensities for stimulation over V1 and V3a. Supplementary Figure S2B shows that this behaviorally relevant electric field component was greatest in V1 and suggests that behaviorally relevant changes of brain activity took place in V1. For Cz control stimulations, negligible electric field intensities were induced in all areas of interest.

Figure 4.

Representative electric-field intensity (V/m) simulations in one participant for stimulation over V1, V3a, and LO. Retinotopic areas and LO, defined from fMRI data, are superimposed. Electric-field intensity ranged from 0 to 35 V/m.

We were surprised by the observation that there were significant improvements in disparity discrimination under V1 stimulation. We sought to ensure that this was true for a larger sample size and, therefore, continued testing eight additional participants with stimulation over V1. In total, 15 participants were tested for V1 stimulation, Cz control stimulation, and a no stimulation condition (see Figure 3B). Again, there was no main effect of TMS, F(2, 28) = 1.13, p = 0.33, but there was a significant interaction between TMS and stimulus contrast polarity F(2, 28) = 4.32, p = 0.02. This suggests that it is critical where in the visual system TMS is applied. Stimulation over V1 significantly improved disparity-discrimination performance for mixed-polarity stimuli compared to control stimulation, t(14) = 3.15, p < 0.01. No such stimulation effect could be observed for single-polarity stimuli, t(14) = −0.76, p = 0.46. There was no significant difference in discrimination performance between control stimulation and no stimulation: single polarity, t(14) = 1.06, p = 0.31; mixed polarity, t(14) = −0.46, p = 0.65, suggesting that the side effects of TMS were not disruptive for task performance.

We also tested seven participants without neuro-navigation (no MRI data was collected), for which V1 stimulation was applied based on scalp landmarks (see Methods). We tested V1 stimulation, Cz control stimulation, and a no stimulation condition. Although TMS at the visual cortex appears to improve disparity-discrimination performance for mixed-polarity stimuli, this change was not significant, t(6) = 0.94, p = 0.39. This was mostly likely due to nonoptimal stimulation of V1 without neuro-navigation (see Discussion).

To resolve this conflict of the stimulation outcome with and without neuro-navigation and to decide whether there was an overall effect of stimulation, we performed a single-paper meta-analysis (Mcshane & Böckenholt, 2017). Results for the initial seven participants and subsequent eight participants with neuro-navigation as well as the seven participants without neuro-navigation were treated as three separate experiments. This meta-analysis confirmed that stimulation over V1 significantly increased observer disparity-discrimination performance compared to control stimulation for mixed-polarity stimuli (Z = 2.98, p < 0.001).

To test whether task difficulty affects the stimulation outcome, we retested nine participants in the main experiment. Stimulus properties for the mixed- and single-polarity conditions were adjusted so that participants had matched performance for both conditions prior to brain stimulation. Participants were tested for V1 stimulation and a no stimulation condition (see Figure 3C). Similar to the main experiment, stimulation over V1 significantly improved disparity-discrimination performance for mixed-polarity stimuli (Z = −2.31, p = 0.02). For single-polarity stimuli, TMS did slightly improve discrimination performance; however, this effect was not significant (Z = −1.36, p = 0.17).

TMS produces methodological challenges that might affect the experiment outcome. Due to the large size of the coil, stimulation cannot always be applied at an ideal location on the scalp. Additionally, participants will move their head relative to the stimulator coil, which makes it difficult to reliably target the same underlying population of neurons. In this study, we monitored coil position, orientation, and tilt relative to stimulation targets in the brain. Supplementary Figure S1C shows these control measures for each stimulation condition. For all our measures of coil precision, V1 stimulation was equal or less precise than for V3a and LO stimulation. This makes it unlikely that V1 stimulation effects can be explained by how easily an area can be reached with a TMS coil.

A useful predictor of the success of TMS in the brain is the distance between stimulation target and the center of the coil during stimulation (Stokes et al., 2013). Supplementary Figure S1C shows that for V1 this distance was lower compared to V3a and LO in our participants. In a previous study, we showed that stimulation of comparable intensity has reliable effects on V3a neural activity (Schaeffner & Welchman, 2016), suggesting that our null result for V3a in this paper is not due to insufficient stimulation intensity. Electric field simulations suggest that stimulation of V3a and LO was, in fact, more successful than V1 stimulation (see Supplementary Figure S2A). This makes it unlikely that the results of this study can be explained by stimulation efficacy based on the distance to the target brain region.

Another challenge of combining TMS with psychophysical tasks is the fact that stimulation produces muscle twitches on the scalp, which often lead to reflexive blinks. This effectively reduces the exposure time during stimulus presentation. Given that stimulation was applied at different locations on the head, this might systematically affect the behavioral outcome and favor coil locations at the back of the head (e.g., V1), which are furthest away from the eyes. Supplementary Figure S3A shows the number of blinks during stimulus presentation between different stimulation conditions. Participants had to blink more often when stimulation was applied during the task. However, this did not favor a specific stimulation condition and could, therefore, not have affected the experiment results in a systematic way. Also, it is conceivable that side effects of stimulation were more detrimental on behavior for a particular coil position. Supplementary Figure S3B shows the number of proportion of trials that participants missed for each condition based on catch trials for lapses (see Methods). Lapse rates were higher for V1 stimulation (with which depth perception improved through stimulation) compared to V3a and LO stimulation. This rules out the possibility that distraction, through stimulation, might have disproportionately affected observers during V3a and LO stimulation.

Finally, TMS stimulation might have affected the stability of eye vergence during the task. This could be the case for two reasons. First, depending on coil location relative to the eyes, stimulation might actively interfere with extraocular muscles. Second, it is possible that TMS affected brain areas involved in control of eye movements, such as parietal regions close to V3a (Pierrot-Deseilligny, Milea, & Müri, 2004). To control for this potential confound, we recorded pupil positions during stimulus presentation. Supplementary Figure S3C shows average vergence eye movements during stimulus presentation for different stimulation conditions. To quantify changes of vergence through TMS, we fitted a linear model to participants' eye vergence during stimulus presentation. Changes in vergence after stimulus onset did not differ significantly for different stimulation conditions, F(4, 24) = 0.76, p = 0.56. This confirms that stimulation of different brain regions did affect vergence during stimulus presentation.

In this study, we presented stimuli at a location lateral to fixation. This was done to maximize stimulus processing in one hemisphere and, therefore, increase the chances of TMS intervention, which we could only apply to one hemisphere at a time. We found that the lateral position of the stimulus did trigger horizontal version eye movements during stimulus presentation around 150 ms after stimulus onset (see Supplementary Figure S3D). Again, to quantify changes of version through TMS, we fitted a linear model to participants' gaze position during stimulus presentation. Horizontal eye movements toward stimulus position did not differ significantly between TMS conditions, F(4, 24) = 2.39, p = 0.08. This makes it unlikely that horizontal eye movements can explain the changes in stereopsis we observed.

Discussion

In this study, we investigated where in the visual cortex disparity processing benefits from the availability of a mixture of bright and dark visual features and allows for better depth perception. We found that stimulation over early visual cortex (V1) with TMS, during stimulus presentation, increased this perceptual benefit. Stimulation of higher visual areas V3a and LO did not change perception. Our findings show that disparity processing in early visual cortex gives rise to the mixed-polarity benefit. This is consistent with models of stereopsis at the level of V1 that produce a mixed-polarity benefit.

Where does the mixed-polarity benefit occur?

In this study, we applied TMS over V1, V3a, and LO to locate the disparity-processing mechanism that produces the mixed-polarity benefit. We found that stimulation over V1 significantly improved disparity discrimination for mixed-polarity but not single-polarity stimuli (see Figure 3). Stimulation of higher visual areas V3a and LO, which are responsive to binocular disparity (Goncalves et al., 2015; Patten & Welchman, 2015; Preston et al., 2008), did not significantly change perception.

In the main experiment of this study, we report results for 15 participants for which we applied neuro-navigated TMS. However, we also tested an additional seven participants for which the TMS coil was placed over primary visual cortex based on scalp landmarks because no MRI scan was available. With this approach, TMS over primary visual cortex did slightly improve observer depth perception for mixed-polarity stimuli, but the result was not significant. To resolve this conflict of the stimulation outcome with and without neuro-navigation, we performed a single-paper meta-analysis (see Results). This meta-analysis confirms that TMS over primary visual cortex did significantly improve depth perception for mixed-polarity stimuli. Moreover, we replicated the improved performance in a subsequent control condition that matched behavioral performance in the mixed- and single-polarity conditions.

There are good reasons why stimulation without neuro-navigation did not produce a similar outcome. It has been shown that coil placement based on scalp landmarks results in far less precise stimulation and that larger sample sizes are necessary to compensate for this imprecision (Sack et al., 2009). It is possible that with a larger sample size we would obtain a similar result compared to neuro-navigated TMS. Additionally, in the main experiment, neuro-navigation allowed us to optimally adjust the direction of the induced electric current for individual participants. This has been shown to optimize the stimulation outcome (Janssen et al., 2015; Laakso et al., 2014; Thielscher et al., 2011). Hence, without neuro-navigation, stimulation can be expected to be less successful, and this can explain the attenuated effect on observer performance that we found in this study.

Another limitation of TMS research is the unknown volume of brain tissue in which we are affecting neuronal behavior. Figure 3A shows that the effect of TMS starts to emerge as we move the coil from a distant control site (Cz) to areas LO and V3a, which are located closer to early visual cortex. It is conceivable that stimulation over V3a caused small changes of neural activity in early visual cortex, which were insufficient to significantly affect perception.

Additionally, TMS-related cell activation has been shown to propagate in neural networks and can reach interconnected areas (Bestmann, 2008). In previous research, we showed that TMS-related cell activation in V2d, V3d, and V3a propagates back to primary visual cortex (Schaeffner & Welchman, 2016). It is possible that in this study TMS induced activation of V3a, which propagated back to V1, producing marginal changes in perception.

It is difficult to confirm where in the brain TMS changes neural activity. In this study, we placed the TMS coil to maximize electric-field induction in a given target area. However, electric-field modeling reveals that, due to anatomical brain structure, TMS over V1 creates the strongest electric-field intensity in V2d (see Figure 4; Salminen-Vaparanta et al., 2014). To control whether stimulation of V2d played a role in this study, we calculated the differential between a behaviorally relevant (TMS over V1) and a behaviorally nonrelevant (TMS over V3a) electric-field intensity (for both coil positions, we found comparable electric-field intensities in V2d). This behaviorally relevant electric-field component was maximal in V1 (see Supplementary Figure S2B), suggesting that stimulation of V1 neurons underlies the changes in depth perception we observed in this study. However, we cannot rule out the possibility that stimulation of V2d or V3d also played a role.

How does the mixed-polarity benefit arise?

It is a long-standing observation that the presence of both black and white dots in RDSs improves disparity-based depth judgments (Harris & Parker, 1995; Read et al., 2011). Here we were able to replicate this effect, showing that discrimination thresholds were significantly lower for mixed-polarity stimuli compared to single-polarity stimuli. The benefit is present for a large range of disparity magnitudes (see Figure 2).

One concern about the contrast polarity effect in previous studies was that only small sample sizes of experienced psychophysical observers were tested. It is conceivable that this benefit only arises when the visual system has been trained to maximize the use of binocular disparity as a cue for depth perception. In this study, we specifically tried to test naïve observers who did not have a history of year-long exposure to RDSs. We show that the mixed-polarity benefit is present in a sample of naïve participants.

Different explanations have been proposed for how the mixed-polarity benefit arises. Harris and Parker (1995) suggest that separate “on” and “off” channels process bright and dark features in the RDS separately. This would reduce the number of potentially correct dot matches in a mixed-polarity RDS by half and double the number of correct dot matches a human observer can sample to get an optimal representation of the stimulus. Separate on and off channels are well established in the early visual system (Jiang, Purushothaman, & Casagrande, 2015; Schiller, 1992, 2010); however, our current understanding is that these separate channels converge in V1 onto simple cells (Schiller, 1992) and, therefore, cannot explain the mixed-polarity benefit. Additionally, separate on and off channels produce a doubling in observer performance for tasks that require a global correspondence solution (Edwards & Badcock, 1994). However, Read et al. (2011) showed that the mixed-polarity benefit is not fixed to a doubling of observer performance. This makes it unlikely that the benefit can be explained by separate on and off channels.

Alternatively, the benefit could be explained by different image statistics of mixed- and single-contrast polarity RDSs. Read and Cumming (2018) showed that stimuli used in this study and all previous studies contain higher interocular image correlation at the target disparity if they have mixed contrast polarity compared to a single contrast polarity. A more correlated input drives binocular cells in primary visual cortex more strongly and produces a more reliable binocular disparity signal. This stimulus artifact arises from the way that dots are placed to avoid dot overlap when the RDS is created (for details, see Read & Cumming, 2018) and could explain why the mixed-polarity benefit only arises when there is no dot overlap in the stimulus (Read et al., 2011). Read and Cumming (2018) showed that the standard binocular energy model produces larger energy peaks at preferred disparities for mixed-polarity RDSs. This could explain why the mixed-polarity benefit arises.

This difference in interocular image correlation was present in the stimuli used in this study (see Figure 5A). With increasing disparity noise, image correlation at target disparity decreases more strongly for single-polarity stimuli. From this, it follows that, in our experiment, the benefit should be larger for observers who can tolerate higher amounts of disparity noise in the stimulus. In this study, we tested people at one of three Gaussian disparity noise levels, depending on how much noise they could tolerate (see Methods). Figure 5B shows the depth-discrimination performance for increasing levels of disparity noise. We did find a trend that agrees with the prediction by Read and Cumming (2018): With greater disparity noise, observers required larger disparity signals to achieve 80% correct discrimination, and this effect is stronger for single-polarity stimuli compared to mixed-polarity stimuli. However, this trend was not significant, F(1, 19) = 2.32, p = 0.14, and so here we cannot conclude whether this observation is representative.

Figure 5.

(A) Binocular image correlation of mixed- and single-polarity RDSs at target disparity for different amounts of added disparity noise (1,000 RDSs per bar). (B) Mixed-polarity benefit with different amounts of disparity noise in the stimulus (for lower disparity noise levels, data was pooled). The data shown are RDS offsets in which observers achieved correct discrimination in 80% of trials and were taken from psychometric functions in Figure 2. Error bars depict 1 SEM.

Another explanation of the mixed-polarity benefit is offered by a recent augmentation of the binocular energy model. Read and Cumming (2007) proposed that the visual system might use opposite contrast polarity of features in the two retinal images to reject false matches and thereby find the true correspondence by a process of elimination. Recently, Goncalves and Welchman (2017) developed a binocular neural network, which is based on an advanced concept of the binocular energy model and uses proscription of unmatched image information to achieve stereo correspondence. This neural network produces a mixed-polarity benefit together with other phenomena of human vision, such as Da Vinci stereopsis. Additionally, the network only produces a perceptual benefit if dots did not overlap in the RDS as is true for human observers (Read et al., 2011).

Why does brain stimulation increase the benefit?

It is surprising that brain stimulation has the potential to improve stereopsis. The best explanation for why TMS amplifies the mixed-polarity benefit is that TMS changes neural activity in a sensory processing mechanism through which the benefit arises. We, therefore, discuss the different potential outcomes of stimulation and consider how they could explain an improvement of depth perception given the explanations of the mixed-polarity benefit that have been proposed above.

TMS has the potential to both drive excitation and increase suppression of neural activity in the brain (Rattay, 1999). Although it is possible for TMS to hyperpolarize cells, this only happens under very specific conditions (Rattay, 1999), and it is, therefore, assumed that cell suppression following TMS results from the activation of inhibitory connections (Moliadze, Zhao, Eysel, & Funke, 2003; Murphy, Palmer, Nyffeler, Müri, & Larkum, 2016). Electrical stimulation of animal neural tissue triggers initial brief excitation (Adrian & Moruzzi, 1939; Patton & Amassian, 1954), followed by two waves of GABA-ergic inhibition (Connors, Malenkat, & Silva, 1988) in the first 100 ms after stimulation. This general effect of initial excitation (Boroojerdi, Battaglia, Muellbacher, & Cohen, 2001; Devanne, Lavoie, & Capaday, 1997; Hess, Mills, & Murray, 1987; Ziemann, Lönnecker, Steinhoff, & Paulus, 1996) and subsequent GABA-ergic inhibition (Kujirai et al., 1993; Premoli et al., 2014) has been replicated in the human motor cortex. TMS to primary visual cortex in cats suggests that stimulation predominantly triggers suppression of simple and complex cells in a 100-ms window (Moliadze, Giannikopoulos, Eysel, & Funke, 2005; Moliadze et al., 2003).

Depending on the structure of a neural network, TMS will trigger a different ratio between activation and inhibition. Also, depending on the role of these networks, activation and inhibition of certain cell subpopulations will have different effects on the behavioral outcome. Accordingly, TMS has been shown to differently affect behavioral tasks and brain areas. TMS has been shown to impair sensory discrimination for visual features, such as motion direction (Pascual-Leone, Bartres-Faz, & Keenan, 1999), motion speed (McKeefry et al., 2008), object shape (Silson et al., 2013), and local orientation (Rahnev, Maniscalco, Luber, Lau, & Lisanby, 2012). It has been argued that brain stimulation results in random neural noise, which compromises cell populations that encode the relevant visual features. However, TMS-induced activation has also been shown to sum with sensory activation in a meaningful way to improve detection of elusive stimuli (Abrahamyan, Clifford, Arabzadeh, & Harris, 2011, 2015; Miniussi, Harris, & Ruzzoli, 2013; Schwarzkopf, Silvanto, & Rees, 2011).

So how can we explain an improvement in depth perception for mixed-polarity stimuli after TMS that we observe in this study? All variations of the binocular energy model, which describe disparity processing in primary visual cortex, contain a squaring nonlinear processing step to match the model output with the strong responses of complex cells to preferred disparities. Read and Cumming (2018) have shown that the binocular energy model produces a stronger population response for mixed- compared to single-polarity stimuli. This is because interocular image correlations are higher for mixed-polarity RDSs, and this difference is further amplified by nonlinear processing after summation of simple unit activity. If TMS activates neurons in V1 and thereby drives sensory responses, then we would predict a greater amplification for mixed polarity stimuli due to nonlinear processing. In this way, TMS would provide a stronger disparity signal boost to the visual system for mixed-polarity stimuli and could thereby improve depth perception.

To test this assumption, we retested nine participants and adjusted task difficulty until observers had similar proportions of correct responses for mixed- and single-polarity stimuli. These stimuli should produce comparable disparity signals for mixed- and single-polarity stimuli in primary visual cortex. Similarly to the finding in the main experiment, V1 stimulation significantly improved depth discrimination for mixed-polarity stimuli (see Figure 3C). For single-polarity stimuli, we observed a slight improvement in depth perception, but this increase was not significant. This suggests that the increase of the mixed-polarity benefit through TMS cannot be explained solely by an amplification of differentially strong disparity signals in primary visual cortex.

Alternatively, it is possible that TMS drives inhibitory connections, which lead to stronger suppression of binocular mismatches in primary visual cortex. The recently published binocular neural network (Goncalves & Welchman, 2017) makes use of the great number of potential feature mismatches in an RDS by inhibiting the resulting erroneous disparities to support correct stereopsis. This produces a mixed-polarity benefit because mixed-polarity images contain more contrast mismatch information than single-polarity images do. In this study, we applied stimulation in 100-ms intervals during stimulus presentations. Animal models of TMS effects in V1 suggest that stimulation in 100-ms intervals during stimulus presentation causes inhibition of simple and complex cells (Moliadze et al., 20053; Moliadze et al., 2003). Given that the visual system is presented with one global solution and far more potential false matches, which would signal incorrect disparities, general inhibition of the full population of simple and complex cells could have a net effect of predominantly suppressing binocular mismatches. This would support stereopsis and would increase the perceptual benefit of mixed-polarity stimuli. For single-polarity stimuli, this amplification would be less pronounced because less contrast mismatch information is available.

Conclusion

Our results show that a neural mechanism of stereopsis in early visual cortex benefits from the availability of a mixture of contrast polarity and improves depth discrimination. We found that stimulation over early visual cortex (V1) with TMS amplifies this mixed-polarity benefit, and stimulation of higher visual areas (V3a, LO) had no effect. This finding confirms that the mixed-polarity benefit arises during disparity processing in early visual cortex. This is consistent with computational models of stereopsis at the level of V1, which also produce a mixed-polarity benefit. The currently most promising explanations for the mixed-polarity benefit are (a) that higher interocular image correlation in mixed-polarity stereograms drives binocular cells in primary visual cortex more strongly and produces a more reliable binocular disparity signal or (b) that binocular contrast mismatches, which are available in mixed-polarity stimuli, are used to inhibit implausible correspondence solutions and thereby lead to a more reliable disparity signal. Brain stimulation might further increase this perceptual benefit by (a) amplifying responses of binocular cells to mixed-polarity stimuli or (b) by driving the inhibition of binocular contrast mismatches in primary visual cortex. Additional research is necessary to conclusively answer the question of how the mixed-polarity benefit arises in stereopsis.