Reliability-dependent contributions of visual orientation cues in parietal cortex

Significance

The world is 3D, but our eyes sense 2D projections. Constructing 3D spatial representations is consequently a complex problem the brain must solve for us to interact with the environment. Robust 3D representations can theoretically be created by combining distinct visual signals according to their reliabilities, which depend on factors such as an object’s orientation and distance. Here, we show that reliability constrains the integration of texture and disparity cues for 3D orientation in macaque parietal cortex. Consistent with human perceptual studies, the contribution of texture cues is found to increase as the object’s slant (i.e., depth variation) increases. This finding suggests that the parietal cortex is capable of combining multiple visual signals to perform statistical inference about the 3D world.

Abstract

Creating accurate 3D representations of the world from 2D retinal images is a fundamental task for the visual system. However, the reliability of different 3D visual signals depends inherently on viewing geometry, such as how much an object is slanted in depth. Human perceptual studies have correspondingly shown that texture and binocular disparity cues for object orientation are combined according to their slant-dependent reliabilities. Where and how this cue combination occurs in the brain is currently unknown. Here, we search for neural correlates of this property in the macaque caudal intraparietal area (CIP) by measuring slant tuning curves using mixed-cue (texture + disparity) and cue-isolated (texture or disparity) planar stimuli. We find that texture cues contribute more to the mixed-cue responses of CIP neurons that prefer larger slants, consistent with theoretical and psychophysical results showing that the reliability of texture relative to disparity cues increases with slant angle. By analyzing responses to binocularly viewed texture stimuli with conflicting texture and disparity information, some cells that are sensitive to both cues when presented in isolation are found to disregard one of the cues during cue conflict. Additionally, the similarity between texture and mixed-cue responses is found to be greater when this cue conflict is eliminated by presenting the texture stimuli monocularly. The present findings demonstrate reliability-dependent contributions of visual orientation cues at the level of the CIP, thus revealing a neural correlate of this property of human visual perception.

Transforming 2D retinal images into accurate 3D representations of the world is a fundamental, albeit complex, problem the brain must solve. The computation of 3D object orientation is essential to this process and necessary for a wide range of behaviors, including object recognition (1), reaching (2), and grasping (3, 4). Multiple signals, including texture (available monocularly) and binocular disparity, are used to determine 3D orientation (5, 6). Single-unit (7–10) and functional MRI (fMRI) (11–13) studies indicate that different orientation cues are combined in high-level cortical areas.

Object orientation is often described using angular variables called slant (rotation about an axis perpendicular to the line of sight) and tilt (rotation about an axis parallel to the line of sight) (14, 15) (Fig. S1). As a consequence of perspective geometry, which determines how a scene projects onto each retina (16), the reliability of texture cues for 3D orientation increases with slant (i.e., as depth variation increases) (17) (Fig. 1A). In contrast, the reliability of disparity cues is largely independent of slant (18). Thus, if robust orientation estimates are created by combining texture and disparity cues according to their reliabilities, the relative contributions of the cues will depend on the object’s slant. Human perceptual studies correspondingly show that as slant increases, texture contributes more (and hence disparity less) to judgments of surface orientation (5, 6) (Fig. 1B).

Fig. 1.

Perspective geometry constrains the reliability of texture cues. (A) Brick wall viewed at four slants: 0°, 20°, 45°, and 65°. Rotation by a fixed amount (e.g., ∆s = 20°) results in greater texture changes at larger slants (Bottom) compared to smaller slants (Top). The colored lines illustrate that the convergence of parallel lines in a 2D image due to perspective accelerates with slant, making texture cues more reliable at larger slants. This property of perspective geometry is highlighted in the rightmost parts of the diagrams, where the lines are reproduced on top of each other. Note the greater difference in slopes in the Bottom vs. Top diagrams. (B) Summary of human perceptual studies showing how the reliability and weighting of texture and disparity cues for 3D surface orientation depend on slant. Data with regression lines are plotted for five subjects from studies by Knill and Saunders (5) and Hillis et al. (6). (Left and Middle) Texture and disparity cue reliabilities computed from measured discrimination thresholds as a function of slant. Whereas texture reliability consistently increases with slant, disparity reliability is comparatively flat. (Right) The weight with which texture cues contribute to the perceived slant of a mixed-cue stimulus increases (hence, the disparity weight decreases) with slant, as predicted if texture and disparity cues are combined according to their reliabilities.

Here, we investigate how texture and disparity cues are combined at the single-cell level. Previous studies show that neurons in the caudal intraparietal area (CIP) of the macaque monkey jointly encode the slant and tilt of a planar object (14, 15), and that some CIP neurons are sensitive to both texture and disparity cues (7, 8). We conjectured that the contributions of these cues to CIP responses depend on the preferred slant; specifically, that cells preferring small slants would be less sensitive to texture cues than cells preferring large slants. To test this conjecture, we measured slant tuning curves from single neurons using mixed-cue (texture + disparity) and cue-isolated (texture or disparity) stimuli. Comparisons of the tuning curves reveal that the slant-dependent relative reliability of texture and disparity cues constrains their contributions to the responses of CIP neurons. This finding suggests that different visual cues may be combined according to their reliabilities in CIP to create a robust 3D representation of the world.

Results

Sensitivity of CIP Neurons to Texture and Disparity Cues.

Based on measurements of tilt tuning, previous work shows that about half of surface orientation-selective CIP neurons are sensitive to both texture and disparity cues (7), and have strongly correlated texture-defined and disparity-defined tilt preferences (8). However, because texture reliability does not depend on tilt, previous work could not examine if the contributions of texture and disparity cues to CIP responses depend on cue reliability. To investigate the reliability-dependent combination of these cues, it is essential to vary slant. Here, we measured CIP slant tuning curves using the following: (i) mixed-cue checkerboard stimuli with congruent texture and binocular disparity cues (CKB), (ii) checkerboard texture stimuli that could be viewed monocularly (mTXT) to assess texture sensitivity in the absence of disparity cues or binocularly (bTXT) to assess texture sensitivity in the presence of a conflicting disparity cue signaling zero slant (i.e., a frontoparallel plane), and (iii) random dot stereograms to assess sensitivity to binocular disparity cues (RDS) (Fig. 2A). Importantly, the stimuli were rendered in a 3D environment to ensure the equivalence of texture-defined and disparity-defined slants. These stimuli allow for direct comparison of responses across stimulus types but have the tradeoff of introducing a density gradient across the RDS stimuli (a potential texture cue). If this density gradient contributes to CIP slant selectivity, it could cause the contribution of texture cues to be underestimated at large slants, spuriously weakening support for our hypothesis. However, theoretical work (19), neural data (below), and human psychophysics (Fig. S2) all indicate that this density gradient is not a reliable slant cue, and is thus unlikely to contribute to CIP slant selectivity (SI Methods).

Fig. 2.

Visual stimuli and slant–tilt tuning. (A) Three sets of stimuli defining planar surfaces. Column 1 illustrates mixed-cue CKB stimuli. Column 2 illustrates texture stimuli with the same pattern as the CKB stimuli but zero disparity (TXT). The TXT stimuli could be presented monocularly (mTXT) or binocularly (bTXT). Column 3 illustrates RDS stimuli for assessing sensitivity to disparity cues. The yellow dot at the center of each stimulus is the fixation point (directly in front of the monkey). (B) Slant–tilt tuning curve of a CIP neuron measured with the CKB stimuli. Slant is the radial variable, and tilt is the angular variable. The firing rate is color coded, with red hues indicating a higher firing rate. The peak of the tuning curve lies in the upper left quadrant, indicating the cell preferred a planar surface with the upper left side closest to the monkey. The black line marks the slant tuning curve passing through the cell’s preferred surface orientation. Each tested tilt axis (e.g., the one specified by the black dot and line) is labeled and marked with a colored dot. (Inset) The preferred planar surface. (C) Slant tuning curve measured at each tilt axis is plotted in the colors indicated in B. Fits are von Mises functions. Mean responses are baseline-subtracted, and error bars are SEM.

For each cell, a joint slant–tilt tuning curve was first measured using the mixed-cue (CKB) stimuli. Slant and tilt are polar coordinates describing surface orientation in which slant (s) is the radial variable and tilt (t) is the angular variable (Fig. S1B). The origin corresponds to a frontoparallel plane (s = 0°), and greater radial distances correspond to planes with larger slants (more depth variation). The slant–tilt tuning curve of an example cell is shown in Fig. 2B. Slant tuning curves measured at fixed tilt axes are slices passing through the origin of the slant–tilt disk. For example, the black line in Fig. 2B marks the slant tuning curve passing through the cell’s preferred surface orientation. The slant tuning curve at each tested tilt axis is shown in Fig. 2C. Because RDS and TXT tilt preferences in CIP are similar (8), slant tuning curves were then measured using the mixed-cue and cue-isolated stimuli only at the tilt axis passing through the preferred surface orientation.

Slant tuning curves of four cells illustrating the range of observed responses are shown in Fig. 3. We first examined the percentage of RDS and bTXT responses that had significant slant tuning (ANOVA, P < 0.05). Nearly every slant tuning curve measured with the RDS stimuli (58 of 59 cells, 98%) and a smaller percentage measured with the bTXT stimuli (32 of 49 cells, 65%) were significantly tuned. Some cells (17 of 49, 35%) were therefore significantly tuned for the RDS stimuli but not the bTXT stimuli (Fig. 3 A, B, and D). Note that viewing a texture stimulus binocularly results in a cue conflict at nonzero slants because the disparity cues signal a slant of 0° (a frontoparallel plane) regardless of the texture-defined slant. If the responses of these 17 cells to the bTXT stimuli signaled the disparity-defined slant, then their RDS frontoparallel plane responses should be similar to the bTXT responses regardless of the texture-defined slant. To test this possibility, we calculated a discrimination index (DI) (20) assessing how well the responses of each neuron distinguished a RDS frontoparallel plane from the bTXT stimuli at each texture-defined slant (SI Methods). The average DI [DI = 0.22 ± 0.16 (SD)] was low, and the vast majority of them (132 of 153, 86%) were not statistically significant (Table S1), indicating that these cells responded to the bTXT stimuli as if they were frontoparallel. This finding is evident in Fig. 3 A, B, and D by comparing the frontoparallel RDS and CKB responses with each bTXT response. In the presence of the texture–disparity cue conflict occurring with bTXT presentations, these cells thus signaled the slant specified by the disparity, not the texture cues. Other cells behaved in a different fashion, having similar tuning for the CKB, RDS, and bTXT stimuli (Fig. 3C). In the presence of the texture–disparity cue conflict occurring with bTXT presentations, these cells signaled the texture, not the disparity cues.

Fig. 3.

Slant tuning curves. (A–D) Slant tuning curves of four cells illustrating the range of observed responses. Tuning curves were measured using the following stimuli: (i) CKB (black), (ii) RDS (blue), (iii) bTXT (green), and (iv) mTXT (magenta) for some cells. Fitted curves are von Mises functions (drawn for significantly tuned responses only; ANOVA, P < 0.05). Mean responses are baseline-subtracted, and error bars are SEM. (A) Cell that was tuned for the RDS stimuli but not the bTXT stimuli. (B) Cell that was tuned for the RDS stimuli but not the bTXT or mTXT stimuli. (C) Cell that was tuned for both the RDS and bTXT stimuli. (D) Cell that was tuned for both the RDS and mTXT stimuli but not the bTXT stimuli. Texture stimuli were not presented monocularly to the cells in A and C. Note that all of the bTXT responses in A, B, and D were similar in amplitude to the RDS and CKB responses to a frontoparallel plane (s = 0°).

Unlike previous studies (7, 8, 10), we also measured responses to mTXT stimuli (Fig. 3 B and D). The mTXT responses were significantly tuned (ANOVA, P < 0.05) in 15 of 22 measured tuning curves (68%) from 14 cells (the stimuli were presented separately to each eye for 8 cells and to just one eye for 6 cells). For 3 of the 8 cells tested with each eye, both the left and right eye responses were significantly tuned. For 13 of 22 tuning curves, the stimuli were presented to the eye contralateral to the recording hemisphere, and tuning was significant for 8 of 13 (62%) tuning curves. For the other 9 tuning curves, the stimuli were presented to the eye ipsilateral to the recording hemisphere, and tuning was significant for 7 of 9 (78%) tuning curves. Thus, significant tuning for texture stimuli could be elicited during both binocular and monocular viewing, and in the case of monocular viewing, significantly tuned responses could be elicited from either eye regardless of which anatomical hemisphere the cell was located in.

Importantly, measuring texture responses both monocularly and binocularly allowed us to examine how the cue conflict occurring with bTXT presentations affects the texture responses of CIP neurons. This examination revealed that texture responses could depend greatly on whether the stimuli were viewed monocularly or binocularly. For example, during presentation of bTXT stimuli (binocular viewing resulting in a cue conflict), the cell in Fig. 3D signaled the disparity-defined slant (s = 0°). However, when presented with mTXT stimuli (monocular viewing, no cue conflict), the same cell signaled the texture-defined slant. Comparing mTXT and bTXT responses further revealed that cells sensitive to both texture (mTXT) and disparity (RDS) cues can respond qualitatively differently when the cues conflict (bTXT). For example, the cell in Fig. 3C signaled the texture-defined slant during presentation of bTXT stimuli. In contrast, the cell in Fig. 3D signaled the disparity-defined slant.

Comparison of Mixed-Cue and Cue-Isolated Slant Tuning Curves.

To compare mixed-cue (CKB) and cue-isolated (RDS and TXT) responses quantitatively, each slant tuning curve with significant tuning (ANOVA, P < 0.05) was fit with a von Mises function (Fig. 3). The average fits were as follows: r = 0.96 ± 0.04 SD (CKB, n = 59), r = 0.94 ± 0.08 SD (RDS, n = 58), r = 0.88 ± 0.18 SD (bTXT, n = 32), and r = 0.87 ± 0.27 SD (mTXT, n = 15). Of these, 1 RDS, 4 bTXT, and 1 of the mTXT tuning curves were poorly fit, with an accounted variance (r²) ≤ 0.5, and thus removed from this analysis. The slant preferences, response amplitudes, and tuning bandwidths of the remaining tuning curves were then compared (Fig. 4 and Fig. S3).

Fig. 4.

Comparison of slant preferences measured with mixed-cue and cue-isolated stimuli. The histogram on the diagonal shows the difference in preferred slants, with crosses marking population averages. Values in the upper left portion of the histogram indicate larger cue-isolated than mixed-cue (CKB) slant preferences. Binocular disparity (RDS, n = 57), monocularly viewed texture (mTXT, n = 14), and binocularly viewed texture (bTXT, n = 28) slant preferences are shown. The unity line is black.

Across the population, the CKB and RDS slant preferences were highly correlated (r = 0.92, P < 0.001), whereas the CKB and TXT preferences were not (bTXT: r = 0.26, P = 0.2; mTXT: r = −0.07, P = 0.82) (Fig. 4). In this analysis, the preferred CKB slant was defined as positive, but the signs of the RDS and TXT slant preferences were unconstrained (SI Methods). Interestingly, we found that the RDS and TXT tuning curves always peaked at positive slants, indicating that the direction of the preferred slant (e.g., forward vs. backward) did not depend on the stimulus type. On average, the preferred bTXT slant was 14° greater than the preferred CKB slant (sign test, P < 0.0001), indicating that the bTXT responses systematically peaked at larger slants than the CKB responses. This result suggests that in the presence of the texture–disparity cue conflict occurring with bTXT presentations, stronger texture cues (i.e., larger slants) are generally required to drive responses signaling that the plane is not frontoparallel (the disparity-defined slant). In contrast, the mTXT and RDS responses, on average, peaked at slants 4° and 2° greater than the CKB responses, respectively, and were not significantly different from 0° (sign test, P ≥ 0.1). This result suggests that the variability between the mTXT and CKB slant preferences may reflect the computational difficulty of estimating slant from texture cues (19). Also note that unlike the CKB and RDS slant preferences, which spanned the entire range of slants, the TXT preferences were concentrated at larger slants (where texture cues are most reliable). This observation indicates that texture contributes most to CIP responses at large slants, consistent with fMRI results showing that texture-driven activity in the human anterior intraparietal area (which receives direct CIP input in monkeys) increases with slant (4, 21).

The high degree of similarity between CKB and RDS responses and poorer correspondence between CKB and TXT responses was also reflected in the accounted variance between the tuning curves (which were always positively correlated). On average, the disparity (RDS) responses accounted for 81 ± 15% SD (n = 59) of the variance in the CKB responses, the binocularly viewed texture (bTXT) responses accounted for 31 ± 31% SD (n = 49), and the monocularly viewed texture (mTXT) responses accounted for 44 ± 29% SD (n = 22). The finding that mTXT responses accounted for more variance than the bTXT responses in the CKB responses is consistent with pictorial depth cues eliciting stronger sensations of 3D space when they are viewed monocularly rather than binocularly (22).

These results suggest that CIP responses are determined more by disparity than texture cues. However, theoretical work shows that as slant increases over the range of values tested here, the reliability of texture cues increases over an order of magnitude (17), whereas the reliability of disparity cues remains largely constant (18). The reliability of texture relative to disparity thus increases with slant, and human slant judgments correspondingly show greater weighting of texture cues (decreased disparity weighting) as slant increases (5, 6) (Fig. 1B). If the contributions of texture and disparity cues to CIP responses depend on cue reliability, then they should reflect the slant-dependent reliability of texture cues. This possibility is examined next.

Contribution of 3D Visual Signals Depends on Cue Reliability.

To examine the contribution of texture and disparity cues to CIP responses, we computed Z-scored partial correlations between the CKB and TXT tuning curves as well as the CKB and RDS tuning curves (Z_TXT and Z_RDS, respectively) (23, 24). This analysis compares the similarity between the mixed-cue tuning curve and each cue-isolated tuning curve, controlling for the similarity between the cue-isolated responses. By Z-scoring the partial correlations, the mixed-cue responses can be statistically classified as texture-dominated or disparity-dominated (Fig. 5A). For comparisons made using the bTXT data, the mixed-cue responses were classified as disparity-dominated in 40 of 49 cells (82%). Of these 40 cells, 17 were not significantly tuned for the bTXT stimuli, defining a subpopulation that encoded the disparity-defined slant in the presence of a cue conflict. The other 23 cells were disparity-dominated but also significantly tuned for the bTXT stimuli. For these cells, both cues contributed to the responses when there was a cue conflict, but disparity played a significantly greater role. In contrast, 6 of 49 cells (12%) were texture-dominated. The remaining 3 of 49 (6%) cells were not classifiable because the texture and disparity tuning curves were highly similar. Together, the texture-dominated and unclassified cells (9 of 49 cells, 18%) define a subpopulation that signaled the texture-defined slant when there was a cue conflict. Importantly, the overall dominance of disparity cues was not a consequence of presenting the texture stimuli binocularly. For the 22 comparisons made using mTXT responses, the mixed-cue responses were classified as disparity-dominated in 16 of 22 (73%), 2 of 22 (9%) were texture-dominated, and 4 of 22 (18%) were unclassified.

Fig. 5.

Contributions of texture and disparity cues depend on cue reliability. (A) Z-scored partial correlations between mixed-cue (CKB) and cue-isolated (RDS and TXT) slant tuning curves were used to classify responses as dominated by disparity or texture cues (n = 71). ●, cell was significantly tuned for both cues; ○, cell was significantly tuned for one cue. (B) The difference in Z-scored partial correlations is plotted as a function of the preferred CKB slant (mTXT, n = 15; bTXT, n = 32). Type II regression lines (minimizing the perpendicular distance between the data points and the regression line; SI Methods) are plotted for both the mTXT and bTXT data. (C) Accounted variance (r²) between the CKB and RDS/TXT tuning curves as a function of the preferred CKB slant. There was no relationship for the RDS (n = 58) tuning curves, and there were significant positive relationships for both the mTXT (n = 15) and bTXT (n = 32) tuning curves. Type II regression lines are shown in the same colors. For the mTXT data in B and C, if responses were measured for both eyes, each data point is plotted (connected by a thin black line) but the average was used in the regression. (D) Accounted variance between CKB and texture tuning curves measured binocularly (bTXT) vs. monocularly (mTXT) (n = 22). Eliminating the cue conflict that exists when texture-only stimuli are viewed binocularly by presenting them monocularly increased the r² between the CKB and TXT responses. This result is also reflected in the vertical offset between the bTXT and mTXT regression lines in C.

If cue reliability constrains the contributions of texture and disparity cues to CIP responses, then cells preferring larger slants should be less dominated by disparity cues than cells preferring smaller slants. To test this hypothesis, we took the difference between Z-scored partial correlations, ∆Z = Z_TXT − Z_RDS for cells with significant texture and disparity tuning, as an index of the relative contributions of the two cues (23). Positive values of ∆Z indicate a greater contribution of texture cues, whereas negative values indicate a greater contribution of disparity cues (Methods). Consistent with theoretical work (18, 19) and human perceptual data (Fig. 1B, Right), ∆Z increased with the preferred slant (measured with the CKB stimuli) for both the mTXT and bTXT data, indicating that the contribution of texture cues increased with slant preference (Fig. 5B).

This finding could reflect two nonexclusive possibilities: (i) a decrease in the similarity of disparity and mixed-cue responses as the preferred slant increases or (ii) an increase in the similarity of texture and mixed-cue responses. To dissect these possibilities, we plotted the accounted variance between mixed-cue and cue-isolated tuning curves (for significantly tuned responses only) against the slant preference measured with the mixed-cue stimuli. Consistent with human perceptual data on the reliability of disparity cues (Fig. 1B, Middle), there was no correlation between the preferred slant and the CKB–RDS accounted variance (r = −0.01, P = 0.95) (Fig. 5C, blue data). This finding indicates that the similarity between mixed-cue and disparity responses is independent of the preferred slant. Consistent with human perceptual data on the reliability of texture cues (Fig. 1B, Left), there was a strong positive relationship between preferred slant and the CKB–bTXT accounted variance (r = 0.55, P = 0.001) (Fig. 5C, green data), which was enhanced for mTXT presentations (r = 0.84, P < 0.0001) (Fig. 5C, magenta data). This finding indicates that the similarity between mixed-cue and texture responses increased with slant preference. The stronger correlation for mTXT than bTXT data likely reflects that disparity cues signal a frontoparallel plane regardless of the texture-defined slant in bTXT presentations. When this cue conflict was present, the responses tended to better reflect the more reliable signal (disparity), but in the absence of the cue conflict (mTXT presentations), the responses tended to better reflect the only available signal (texture). Likewise, the mTXT responses, on average, accounted for 14% more variance in the CKB responses than did the bTXT responses (within cell comparison, n = 22) (Fig. 5D).

The weights with which texture and disparity cues are combined by CIP neurons were also examined by modeling the CKB responses as a weighted linear combination of the TXT and RDS responses (25, 26) (SI Methods). The CKB responses were well fit by this linear combination for both monocular and binocular presentations of the texture stimuli, with average fits of r = 0.93 ± 0.04 SD (mTXT, n = 22) and r = 0.92 ± 0.06 SD (bTXT, n = 49). To further test if the contribution of texture cues increases with slant preference, the texture weight was correlated with the preferred CKB slant (Fig. S4). Consistent with human perceptual studies (Fig. 1B) and the partial correlation analysis (Fig. 5B), the texture weight was found to increase with slant preference. Analogous to the accounted variance analysis (Fig. 5C), the mTXT weights increased with slant at a similar rate as the bTXT weights but were offset vertically, indicating that they were generally larger than the bTXT weights. Together, the present results show that CIP caries functionally useful information for creating a robust, multimodal 3D representation of the environment.

Discussion

As a consequence of perspective geometry and stereopsis, the reliability of 3D spatial information conveyed by different visual signals depends on viewing geometry (e.g., an object’s slant and distance) (16–18). Human perceptual studies show that the brain takes the reliabilities of these signals into account to create accurate 3D representations of the world from 2D retinal images (5, 6). In this study, we provided evidence for reliability-dependent integration of texture and disparity cues in macaque CIP. Across the population, the contribution of texture cues to neuronal responses was found to increase with preferred slant, as predicted from theoretical work showing that texture reliability increases with slant. A recent study also found that CIP neurons combine visual signals with an estimate of head–body orientation relative to gravity such that a gravity-centered representation of object tilt could be achieved from the population activity (15). Together, these findings suggest that area CIP is capable of combining multiple sensory signals, both within and across modalities, to perform statistical inference about the 3D world.

Previous work showed that some CIP neurons are sensitive to both texture and disparity cues (7, 8) but could not test whether this sensitivity was related to cue reliability because only tilt tuning was measured. Here, we examined how the integration of these cues depends on a cell’s preferred slant. Across the population, we found that cells representing larger slants are more sensitive to texture cues than cells representing smaller slants (Figs. 5B and Fig. S4). This finding is consistent with human judgments of surface slant, which show greater texture weighting (less disparity weighting) as slant increases (5, 6), thus revealing a neural correlate of this property of human visual perception. Consistent with the relationships between slant and texture/disparity reliabilities (18, 19) (Fig. 1B), we further found that the accounted variance between mixed-cue and texture responses increases with preferred slant, whereas the accounted variance between mixed-cue and disparity responses is constant (Fig. 5C).

In previous studies examining neuronal sensitivity to 3D orientation cues, either the texture-defined and disparity-defined slants could not be equated (9) or the texture stimuli were only presented binocularly (7, 8, 10). In contrast, we rendered the stimuli such that the texture-defined and disparity-defined slants were equivalent, and presented texture stimuli monocularly and binocularly. These differences enabled us to show that when texture stimuli are presented monocularly (i.e., with no cue conflict), the responses of CIP neurons generally reflect the texture-defined slant more closely (Fig. 5D) and that estimates of texture weights are greater (Fig. S4). Given this result, the finding that tilt-selective middle temporal neurons have weak texture sensitivity may, in part, reflect that the stimuli were presented binocularly (10). The present findings are also consistent with the sensation of depth being stronger when a texture stimulus (e.g., a painting with perspective) is viewed monocularly rather than binocularly (22), reflecting the elimination of the cue conflict occurring when pictorial depth cues are viewed binocularly. In fact, Leonardo da Vinci advised artists that they could more accurately reproduce visual perception of a scene by sketching it with one eye closed (27, 28). Our finding that the accounted variance between texture and mixed-cue responses increased if the texture stimuli were viewed monocularly rather than binocularly is consistent with this observation, and may reveal a neural basis of this perceptual effect.

Considering that disparity is generally a more reliable slant cue than texture for nearby objects (6), why do some cells (e.g., the one in Fig. 3C) signal the texture-defined slant when there is a cue conflict? Previous studies similarly found cells tuned for the tilt of a binocularly viewed texture-defined plane in both CIP (7, 8) and inferior temporal cortex, where tilt selectivity is greater for monocularly than binocularly viewed texture stimuli (9). The response properties of these neurons suggest that they may be important for perceiving depth from perspective in paintings or perspective-based illusions, such as the Ponzo illusion. Such illusions are perceived by several primate species as well as pigeons (29–31), suggesting that perspective-sensitive cells that disregard disparity may be widely found in visual animals. These cells may also be important for estimating the 3D orientation of distal objects (where disparity cues are less reliable) (6), as well as calibrating orientation estimates based on texture and disparity cues (32–34). Interestingly, some cells that were sensitive to both texture and disparity cues only signaled the texture-defined slant (e.g., Fig. 3C) or disparity-defined slant (e.g., Fig. 3D) when the cues conflicted. The existence of two populations of cells with these properties may enable us to perceive the 3D structure conveyed by pictorial depth cues in paintings (requiring cells like in Fig. 3C) without falsely interpreting that structure as real (requiring cells like in Fig. 3D to signal that the canvas is flat). Likewise, fluctuations in the activity of two such populations may be related to the bistability of 3D visual percepts resulting from large conflicts between binocular disparity and perspective cues (35, 36). Stimuli eliciting bistable 3D visual percepts may provide a useful tool for investigating the extent to which parallels between CIP responses and psychophysical results are a consequence of the percept produced (which is bistable, and thus variable) or reflect computations performed on retinal images (which are constant, and thus independent of the percept). For example, if CIP texture weights covary with perception on a trial-by-trial basis, it may suggest a closer link to perception than feedforward computation.

Whereas the reliability of texture cues increases with slant angle, the reliability of disparity cues decreases as the distance between the observer and a viewed object increases (6). In future work, it will be useful to examine if the contribution of disparity cues to CIP responses decreases as the distance from the object increases. Likewise, all cue conflicts examined in this study were between a variable texture-defined slant and a constant disparity-defined slant of 0°. It will be valuable to examine a broader range of conflict conditions in which texture-defined and disparity-defined slants both vary. Such a study would allow for a systematic test of cue integration at small, as well as large, conflicts to investigate how different internal models of priors may be used to integrate the cues robustly (37). Another open question is how neurons are able to combine sensory signals according to their reliability. Recent work suggests this ability requires a computation called divisive normalization in which the response of each neuron is normalized by a measure of the population activity (26, 34, 38, 39). Future experiments can be designed to test whether divisive normalization accounts for the integration of texture and disparity cues in CIP, and if individual neurons can reweight these cues to account for changes in viewing geometry (6, 25, 26). Lastly, visual distortions that transiently occur with changes in the power of optical lenses reflect a recalibration of the relationship between disparity cues and perceived slant (32). Together, these results suggest the existence of two mechanisms contributing to the visual encoding of 3D orientation: reliability-based cue weighting and calibration of different visual signals. In future studies, it will be important to determine how cue weighting and cue calibration (33, 34) together ensure that 3D representations of the environment are both reliable and accurate.

Methods

A detailed description of the experimental protocols and analyses is provided in the SI Methods. During experiments, a monkey sat 30 cm from a liquid crystal display screen on which planar stimuli were displayed as red/green anaglyphs. Slant tuning curves were measured between ±60°, sampled in 15° steps. The stimuli subtended 19° of visual angle and were centered on the fixation point (a yellow dot) directly in front of the monkey at screen distance. Fixation was maintained within a 2° version and 1° vergence window. Each stimulus presentation required 1,350 ms of fixation: 300 ms of black screen, followed by 1 s of a planar stimulus and then 50 ms of black screen. Single-unit extracellular action potentials were recorded using previously described methods, and stimulus-driven firing rates were calculated from the start of the visual response to the end of the 1-s stimulus presentation (14, 15).

Slant tuning curves measured at fixed tilt axes were fit with a $\pi$-periodic von Mises function, $F\left(s\right)=\mathrm{DC}+\mathrm{G}{e}^{k\left[\cos \left(2\left\{s-s_0\right\}\right)-1\right]}$. Here, $s$ is slant, DC is a baseline offset, G is the response gain, $k$ sets the tuning bandwidth, and $s_0$ is the preferred slant. The $-1$ makes the response amplitude independent of the bandwidth parameter. Tuning bandwidth was calculated as the full-width at half-height of the fitted von Mises function. To assess the contributions of texture and disparity cues to the mixed-cue responses of CIP neurons, we computed Z-scored partial correlations between the CKB and TXT tuning curves as well as the CKB and RDS tuning curves (Z_TXT and Z_RDS, respectively) (23, 24), and took their difference, ∆Z = Z_TXT − Z_RDS (23), as an index of the relative contributions of the two cues. A significance criterion of Z = 1.645 (P = 0.05) was used to classify the mixed-cue responses as texture-dominated or disparity-dominated.