Angle Alignment Evokes Perceived Depth and Illusory Surfaces

Abstract

There is a distinct visual process that triggers the perception of illusory surfaces and contours along the intersections of aligned, zigzag line patterns. Such illusory contours and surfaces are qualitatively different from illusory contours of the Kanizsa-type. The illusory contours and surfaces in this case are not the product of occlusion and do not imply occlusion of one surface by another. Rather the aligned angles in the patterns are combined by the visual system into the perception of a fold or a 3-d corner, as of stairs on a staircase or a wall ending on a floor. The depth impression is ambiguous and reversible like the Necker cube. Such patterns were used by American Indian artists of the Akimel O’odham (Pima) tribe in basketry, and also by modern European and American artists like Josef Albers, Bridget Riley, Victor Vasarely, and Frank Stella. Our research was aimed to find out what manipulations of the visual image affected perceived depth in such patterns in order to learn about the perceptual mechanisms. Using paired comparisons, we found that human observers perceive depth in such patterns if and only if lines in adjacent regions of the patterns joined to form angles, and also if and only if the angles were aligned precisely to be consistent with a fold or 3-d corner. The amount of perceived depth is graded, depending on the steepness and the density of angles in the aligned-angle pattern. The required precision of the alignment implies that early retinotopic visual cortical areas may be involved in this perceptual behavior but the linkage of form with perceived depth suggests involvement of higher cortical areas as well.

1. Introduction

Depth perception is an important part of visual perception. 3-d depth is used to judge the spatial layout of a scene (e.g. Sedgwick, 1986) as well as the solid shape of objects (e.g. Koenderink et al 1996). We are interested in this paper in depth perception in pictures, two dimensional images in which depth perception is possible even though stereopsis, optic flow, and motion parallax all contradict the picture’s apparent depth. Monocular cues for depth perception in pictures have been identified and studied such as: occlusion, texture density, and aerial perspective. In this paper we consider another monocular cue for depth: the alignment of angles in the picture. Angular alignment is a very different cue from occlusion. Depth from occlusion is a remarkable perceptual phenomenon, because sometimes it causes the observer to perceive occluding illusory contours and illusory surfaces, such as the Kanizsa triangle and related illusions (Kanizsa 1979; Anderson 2003), in order to derive a perceptually plausible interpretation of the scene. What we report here is that angular alignment triggers the perception of illusory surfaces and contours that are qualitatively different from Kanizsa-type illusory contours. This distinct perceptual mechanism for perceiving illusory surfaces is also associated with depth perception in pictures.

The phenomenon of depth from angle-alignment is illustrated in Figure 1, where horizontal illusory contours emerge in perception along the intersections of aligned, zigzag line patterns. The perceived contours coincide with the perception of 3-d corners, a perception that is evoked by the alignment of the vertices of the angles formed by the lines. Involuntarily we perceive Figure 1 as a staircase in depth. The depth impression is ambiguous and reversible like a Necker cube. Note that the illusory contours and surfaces in this case are not the product of occlusion and do not imply occlusion of one surface by another. Rather the aligned angles are combined by the visual system into the perception of a fold or in other words a 3-d corner, as of stairs on a staircase or a wall ending on a floor. It is difficult to avoid perceiving such a 3-d interpretation even though one knows that Figure 1 is a picture on a flat surface, from stereo, motion and accommodation cues that conflict with the 3-d interpretation derived from the pictorial cues. Previously, vision scientists recognized similar patterns that have similar illusory surfaces and contours (Kennedy 1975; Kennedy and Chatterway, 1975; Meyer and Petry, 1987; Day 1987). Also, Mamassian and Landy (2001) used aligned-angle patterns with brightness shading to test how well human observers could discern the direction of illumination. However, to our knowledge, there have been no scientific investigations of the perceptual and neural mechanisms that produce the perception of apparent depth in pictures with aligned-angle patterns. Implicitly, the connection between angle-defined illusory surfaces and apparent depth may have been suggested by Hans Wallach (Wallach 1935; translated into English in Wuerger et al 1996) who labeled a pattern of two gratings that formed aligned angles (resembling the upper two “stairs” in Figure 1) as a “roof line-pattern” (Dachlinienmuster, see Fig.41 in Wallach, 1935). Wallach may have been implying that his roof line-pattern caused a perception of depth, since the picture of a roof typically produces the visual perception of a solid object that protrudes (or recedes) in depth.

Figure 1.

Stimulus geometry. Parallel lines joining at corner angles induce illusory contours and the perception of a folded surface

Although there has been little or no scientific work before now on the illusory surfaces formed by aligned angles and the associated depth perception, the powerful perceptual effects have been used by artists. Surprisingly to us, one of the earliest uses of this technique can be traced to the artistry of the women of a Native Indian tribe in the American Southwest called Akimel O'odham (or Pima), in their weaving of trays and baskets. The flat tray in Figure 2a was made by Lucy Makil, an Akimel O'odham artist, around 1930 in Arizona USA (the tray was exhibited at the Museum of the American Indian in New York City, 2004–2006 in the Diker Collection; see Kuspit et al 2004). Though the woven tray is flat, the illusory surfaces formed by the aligned angles produce a strong perception of raised and lowered planes in depth. The perceived depth is ambiguous and reverses upon continuous viewing. This particular tray is an outstanding example of a long tradition of such weaving by Akimel O’odham and Tohono O’odham (also called Papago) peoples in the American Southwest (Breazeale, 1923; Robinson, 1954). Other examples of the technique, collected earlier, can be found in the American Museum of Natural History in New York City, such as the Akimel O’odham tray in Figure 2b from the American Museum collection. This flat tray is more coarsely woven than the Makil tray and so the weaving technique of the artist to produce the perceived 3-d effect is perhaps easier to see. Other patterns in baskets woven by the Akimel O'odham also evoke 3-d percepts, for instance the so-called “squash-blossom” pattern (Breazeale, 1923).

Figure 2.

a. Example of a Pima (Akimel O’odham) basket. This is a flat tray which induces the perception of a windmill whose wedges recede in depth relative to the ground. This is an image from the Charles and Valerie Diker Collection (Kuspit et al 2004), reproduced here with the permission of Mr. Charles Diker.

b. Another Pima (Akimel O’odham) basket tray, giving rise to a stellate pattern that either recedes or sticks out in depth. Catalogue number 50.1/4206, Courtesy of the Anthropology Department. American Museum of Natural History, New York City

Patterns resembling those in the Akimel O'odham baskets were employed in many different paintings by modern artists. Bridget Riley and Victor Vasarely are two famous artists who used aligned-angle patterns in their paintings. In Riley’s paintings the depth effect produced by aligned-angles is sometimes reinforced by the simultaneous presence of other pictorial depth cues like linear perspective or changes in texture density as in the example in Figure 3 (“Disfigured Circle”, see de Sausmarez, 1970). There are other striking examples of the use of angle-alignment for the creation of depth and illusory surfaces in the work of Bridget Riley, for example in the “Blaze” paintings and in the painting “Suspension”, among many others. The artist Victor Vasarely created patterns that look very similar to the Pima Indian tray patterns in Figure 2, in black-and-white paintings like Meandres (1959). Vasarely also painted nested-square patterns that look similar to the aligned-angle patterns studied later by Kennedy (1975). Vasarely used such patterns to evoke brightness illusions. Recently Troncoso et al (2005) reported experiments on brightness perception with patterns like Vasarely’s nested-squares. Troncoso et al pointed out the salience of the “corners” of the nested squares for apparent brightness but they did not dwell on the depth illusion and illusory contours evoked by the aligned right-angles of the squares. Before Vasarely and Riley, in the early 1940s Josef Albers had used aligned-angle patterns in his Graphic Tectonics drawings (Weber, 1984) and then later the painter Frank Stella used similar aligned-angle patterns in his hard-edged paintings such as Six Mile Bottom now in the Tate Gallery (Cooper and Luke, 2006).

Figure 3.

‘Depth from aligned angles’ in modern art. This is a copy of the painting Disfigured Circle by Bridget Riley. (reproduced with permission by Bridget Riley, from de Sausmarez, 1970).

The experiments reported in this paper are attempts to investigate by means of psychophysics the stimulus requirements for the perception of depth and illusory surfaces in patterns like those in Figures 1–3. In a series of experiments and demonstrations we studied how apparent depth varies as a function of the geometry of the pattern. There are many possible cues that have been studied before for depth and surface formation in pictures, such as luminance shading and contrast, orientation-contrast, and spatial-frequency-contrast. However, our results point to the importance for depth perception of the combination of oriented contours into angles, and the precise geometrical arrangement of the angles in the Aligned Angle pattern. In the Discussion we compare our results with earlier studies of shape from texture (reviewed in Todd, 2004) and with recent theoretical suggestions about the importance of curvature in shape perception (Ben-Shahar and Zucker, 2004).

2. Methods

2.1. Participants

Four observers, three of them female, with normal or corrected to normal vision, participated in the experiments. The participants included one of the authors and three other subjects who were naïve regarding the purpose of the experiment. Two of them were undergraduate students, who were paid $10 for each experimental session, and the third subject was our laboratory assistant. Participants' mean age was 24 years. All participants gave their written informed consent to perform the psychophysical experiments prior to the experiments.

2.2. Apparatus and Material

We had to simplify the stimulus in a way that would allow us to manipulate different stimulus aspects in isolation and so we devised the Aligned-Angle pattern depicted in Figure 4. As we show in Results, most of the experiments were done with aligned-angle patterns like those in Figure 4 rotated by 90° in global orientation so as to avoid the possible confounding effects of perceiving a ground plane defined by the endpoints of the lines in the flank region. The rationale for the use of 90° global orientation is presented below with the discussion of the results in the experiments on the magnitude of the roof-flank angle (Figure 5).

Figure 4.

Depiction of stimulus patterns used in the current experiments. Subjects were shown such patterns and asked to report which pattern in a pair looked more 3-d. Such paired comparisons were made with different standard patterns and different varying stimulus parameters such as, in this example comparison, the steepness of the angle between roof and flank.

Figure 5.

Perceived depth vs magnitude of the angles. Psychometric functions that show the relationship between flank orientation plotted along the horizontal axis (angle between roof and flank, in deg) and apparent depth (probability of the observer to judge the comparison c as looking ‘more 3d’ than the standard s) as a function of standard flank orientations (S1=40°, S2 = 60°) and global orientation. The patterns at 0 deg global orientation are designated “ground plane” and those at right angle, 90°. In the insets below, the patterns are illustrated at 90° global orientation. Each panel depicts the performance pattern of one observer; individual data points are proportions calculated on the basis of 20 observations.

Stimuli were generated using the Psychtoolbox in Matlab. Stimuli were presented on a 18'' monitor at a resolution of 1280x1024 pixels and a refresh rate of 75Hz. The observers viewed the stimuli binocularly at a viewing distance of 60cm. Individual stimuli (Aligned-Angle patterns) consisted of parallel lines of one orientation in the center (roof) that were flanked on both sides by parallel lines of another orientation (flanks; Fig. 4). The length of individual line elements was 2.4° visual angle and their width was 4.2arcmin (2 pixels). The global stimulus dimensions varied slightly as a function of the corner angle but on average it extended 5.2° x 6.2°. White stimuli (60% Michelson contrast) were presented in the center of the screen on a gray background with mean luminance of 35cd/m².

2.3. Procedure

Paired comparisons were performed for different stimulus dimensions. Unless otherwise noted, stimuli were presented in a two-interval forced-choice (2-IFC) procedure, meaning that a standard stimulus (s) of constant stimulus intensity was presented in one interval and a comparison stimulus (c) of varying intensity was presented in the other. The observer's task was to judge which of the two intervals contained the stimulus that looked ‘more 3d’. While this instruction to the observer was intentionally somewhat vague, it was also easily understood. The observers gave consistent comparisons and there was consistency across observers. At the beginning of each trial, the observer was asked to fixate a centrally presented cross that was displayed for 500ms. Then the first stimulus was presented for 300ms followed, after an inter-stimulus interval of 500ms, by the second stimulus presented again for 300ms. No time limit was imposed on the response. Standard and comparison stimuli were randomly assigned to one of the two intervals.

2.4. Data analysis

To capture the pattern of observers’ responses, psychometric functions were fit to the data. We calculated the proportion of responses indicating that the comparison had been perceived to ‘look more 3d’ than the standard stimulus, creating the dependent variable P(c > s). We plotted this proportion as a function of the comparison-stimulus strength (e.g. magnitude of flank orientation, or number of lines). Maximum likelihood fits of logistic psychometric functions were performed using the psignifit toolbox in Matlab (Wichmann and Hill, 2001a; see http://bootstrap-software.org/psignifit/).

3. Results

In order to investigate the stimulus attributes that are critical for the perception of apparent depth in the Aligned Angle pattern, we had to establish a performance metric with which to measure the effects of various manipulations of the pattern's geometry. Introspection suggested that the extent to which apparent depth is perceived varies systematically with the magnitude of the flank-roof angle (Fig. 4), as well as with the density of the constituent lines. We tested these introspections in the following pilot experiments and in this way tested the paired comparison method for gauging perceived depth in the basket patterns.

Pilot Experiment 1: Is apparent depth a monotonic function of the magnitude of corner angle?

We paired two standard stimuli of 40° and 60° flank orientation with four comparison stimuli each. In the comparison stimuli, flank orientations were linearly distributed across an interval of ± 30% of the standard flank orientation (e.g. when the standard s = 40°, c = 22, 34, 46, 58°). The line density of the roof elements of the Aligned-Angle pattern was constant during the experiment with a spatial frequency of 1.5 lines per degree (°) visual angle. We repeated the experiment for two different global orientations in order to test for any effects that might be attributable to the stimulus apparently standing on a ground plane. Therefore in this pilot experiment, in one condition the Aligned Angle pattern was at a global orientation of 0° and in the other condition it was rotated by 90° (Figure 5). Each combination of s and c strength was repeated 20 times in random sequence. The global orientation was changed in different experimental blocks so that observers performed a total of 160 trials for each global orientation.

Results

Psychometric functions for individual observers are shown in Figure 5. Observer order is kept constant across all graphs. All four observers showed a systematic relationship between their tendency to perceive a stimulus as having more depth, and the flank-roof angle. The steeper the angle between the roof and the flanks, the more depth was perceived in the Aligned-Angle pattern. This relationship was independent of global stimulus orientation, as is evident from the appearance of the globally-rotated patterns in Figure 5, and also from the psychometric functions (Fig. 5) which were comparable for Aligned-Angle patterns apparently standing on a ground plane and those that were rotated by 90°. The point of subjective equality (at 50% c > s), e.g. the values of roof-flank angle for which the observers judged the comparison to look more solid than the standard in 50% of the trials, was on the average 44° (sd ± 4) and 38° (±3) for a standard stimulus of 40° roof-flank angle for comparisons when standard and comparison were at 0° and 90° global orientation, respectively. The average comparison stimuli were 61° (±6) and 61° (±4) for a standard roof-flank angle of 60°. As can be seen in Figure 5, for the two observers' data in the right hand panels, the psychometric functions were generally steeper for the 90° global orientation, so in later experiments we used only the 90° global orientation, as depicted in the insets to the figures. (It is possible that the global orientation is a possible complication in many other “shape-from-X” experiments.)

The clean quantitative data of Figure 5 are good evidence that the paired comparison method allows us to estimate perceived depth in these patterns. There is a steep and unbiased estimation of perceived depth based on the roof-flank angle. It might be supposed that such a judgment could be based on other factors such as luminance shading after low-pass filtering since there was more total light in the flank region than the roof region in these experiments. In control experiments, the authors viewed patterns like those used in Figure 5 but with the line thickness reduced to eliminate any luminance shading, as in Todd and Reichel (1990), and we observed no difference in the effect of roof-flank angle. The experiments described below on gaps, phase-shift and misalignment also rule out luminance-shading and other non-geometric explanations of the perceived depth.

Pilot Experiment 2: Is apparent depth influenced by the density of constituent lines?

A second basic stimulus feature is line density or spatial frequency. The design of the experiment to measure the effect of line density was analogous to Pilot Experiment 1 on angle-magnitude. For the line density experiments, the roof-flank angle was fixed at 40° and stimuli varied in line density, as demonstrated in Figure 6. The standard spatial frequencies for the roof-lines were 1 and 2 lines/ ° ; comparison spatial frequencies were linearly spaced across an interval of ± 40% of each of the standards, centered at the standard (e.g. for standard line density s = 1, c = 0.4, 0.8, 1.2, 1.6 lines/ ° ).

Figure 6.

Psychometric functions that capture the relationship between density of constituent lines and apparent depth (probability of the observer to judge the comparison c as looking ‘more 3d’ than the standard s), as a function of standard line density (S1=1line/ ° , S2 = 2lines/ ° ) and global orientation. Each panel depicts the performance of one observer; individual data points are proportions calculated on the basis of 20 observations.

Results

Psychometric functions (Fig. 6) show that, for all four observers, perceived depth varied as a function of line density. Relative depth increased the more constituent lines were added. This relationship held for both global orientations tested here, though again the data were better for 90° global orientation. The 50% ‘thresholds’ for the 0° and 90° global orientations were 1.07 (±0.07) lines/° and 1.01 (±0.10) lines/ ° for a standard with a spatial frequency of 1 line/ °, and 2.07(±0.17) lines/ ° and 2.12 (±0.14) lines/ ° for a standard spatial frequency of 2 lines/ °, respectively. Note that an aligned-angle pattern of 1.6 lines/ ° always looked more 3-d than a standard of 1 line/ °, and almost always less 3-d than a standard of 2 lines/ °. This can be seen by comparing the 1.6 lines/ ° blue points with 1.6 lines/ ° red points in the graphs of Figure 6. These results suggest that there is some spatial limitation on the spatial integration of pattern information in the aligned-angle patterns. They also suggest, as did the results of the first pilot experiment, that we were measuring perceived depth accurately with the comparison method. Next we wanted to study the stimulus features that were necessary for evoking depth and surfaces in the aligned-angle patterns.

Experiment 1: Is it the angles or the line patterns that are critical for our judgment of apparent depth?

The question we wanted to address here is whether perceived depth depends on the continuity between the lines that form the angles, or whether only correctly-arranged parallel line patterns in the roof and flank regions might be sufficient for the perception of depth. For this purpose we introduced a condition in which central lines and flanks of some comparison stimuli were separated by a gap, and all other variables were kept the same. The standards had no gap. Line density and roof-flank angle of the standard stimuli were set to 1.5 lines/ ° and 40°, respectively. We used variation of line density of the comparison stimuli as a way of varying perceived depth, based on Pilot Experiment 2. Comparison stimuli varied in line density between ± 40% around the standard, as in Pilot Experiment 2. Two sets of comparison stimuli were used: in one set, corners were continuous (zero gap); in the other, flank and roof lines were separated by a gap of 0.4° visual angle (shown in the insets in Figure 7).

Figure 7.

The importance of the angles in the aligned-angle pattern. Psychometric functions of comparisons with 0.4° gap vs. zero gap. The standards always had zero gap. The vertical axis is probability that the comparison is more 3-d than the standard, and the horizontal axis is line density in lines/ °. Each panel depicts the performance pattern of one observer. Individual data points are proportions calculated on the basis of 20 observations.

Results

Psychometric functions are depicted in Figure 7. Introducing a gap had a large effect on depth perception in all four observers. There was a pronounced difference in observers’ sensitivity to apparent depth. Mean slopes at the point of subjective equality were 5.60(±5.3)(lines/ °)⁻¹ in the continuous condition and 0.52(±0.38)(lines/ °)⁻¹ in the gap condition. The point of subjective equality (at 50% c > s) was 1.3 (±.90) lines/ ° in the gap condition and 1.6(±.06) lines/ ° in the continuous condition. The results on the large drop in sensitivity suggest that the observers' perception of depth was disrupted by the presence of the gap. Observer 4 showed the biggest effect of gap as he never judged the stimulus with the gap as looking more 3d than the aligned angle pattern.

Later we discovered a picture by Bridget Riley indicating that she may have had a similar intuition about the importance of continuity. The picture copied in Figure 8 contains four vertically arranged surface areas of oriented lines that are separated by small gaps, and that appear perfectly flat, unlike the vivid depth in Riley’s paintings in which the lines intersect, producing aligned angles.

Figure 8.

Untitled artwork (‘Rothko Portfolio’) by Bridget Riley, illustrating the effect of disrupting the continuity of corners (reproduced with permission by Bridget Riley).

After the introduction of a relatively small gap had such a dramatic effect on apparent depth, we were interested in the question what would happen to apparent depth by deconstructing the angles by phase-shifting roof lines and flanks with respect to each other. A phase shift between parallel line patterns gives rise to an illusory contour along line ends. It was not clear a priori whether the percept of apparent depth would remain along this type of illusory contour, or disappear with the breaking-up of the angles. As is apparent when looking at the demonstration in Figure 9, the appearance of a phase-shifted aligned-angle pattern changes markedly, and the pattern loses a large amount if not all of its apparent 3d shape.

Figure 9.

Demonstration showing that a phase shift of the flanks reduces perceived depth and changes the perceptual organization.

Because the continuity of the angle elements seemed to be so important, we wished to make sure of this point, and so we constructed a demonstration that illustrates in another way the necessity for angle-continuity for the aligned-angle phenomenon. This is shown in Figure 10, where the angles are covered by occluders but the flank and roof lines are the same as in the full illusory pattern: occlusion of only the angles makes the picture appear less three-dimensional.

Figure 10.

Demonstration figure of the effect of occluding the angles on the depth perception of the figure.

The results of the gap experiments and the phase-shift and occluder demonstrations not only suggest the importance of the angles in the aligned-angle patterns but also they rule out many non-geometrical explanations of perceived depth and illusory surfaces in the perception of these patterns. For instance, Figures 7, 9–10 disprove the luminance shading conjecture that we already ruled out with the line thickness control (cf. Todd and Reichel 1990). But they also rule out explanations of perceived depth and illusory surfaces based on relative orientation or spatial frequency differences between roof and flank regions--because those differences are the same in the gap or no-gap patterns, for instance, yet the perception is markedly changed by the gap (and also by the phase shift, and also by the occluder). The results rather suggest that the geometry of angle formation is critical for the perception. But it is not the angles alone that matter but also the alignment of the angles, as we consider next.

Experiment 2: Is the alignment of the angles critical for our judgment of apparent depth?

The experiments so far established that the integrity of the angle elements is needed in order to see the aligned-angle type of illusory contour and its depth impression. Next we needed to investigate how much perception of depth in these patterns depended on the precise alignment of the angles. We misaligned the spatial locations of the angles with respect to one another as in Figure 11. We used a standard stimulus with 40° roof-flank angle, and density 1.5 lines/ °. This time the comparison stimuli varied in roof-flank angle between ± 30% around the standards’ angle, analogous to Pilot Experiment 1. Two sets of comparison stimuli were employed: the regular set in which corners are horizontally aligned, and a jittered set in which locations of the angles were misaligned by 10% of the elements’ line length; one such jittered set of angles is illustrated in Figure 11.

Figure 11.

Misalignment affects perceived depth. Psychometric functions that illustrate the relationship between flank orientation (in °) and apparent depth as a function of alignment of corner angles (aligned vs. 10% jitter). Each panel depicts the performance pattern of one observer, and individual data points are proportions calculated on the basis of 20 observations. Schematic stimuli are illustrated below the psychometric functions.

Results

The misalignment of corner angles by only 10% had a strong effect on the perception of apparent depth in the aligned-angle pattern in all four observers (Fig. 11). The point of subjective equality was on the average 45° (±8) for the aligned stimulus and 7° (±88) for the jittered stimulus. The relationship between flank orientation and apparent depth was abolished when neighboring corners were misaligned. Furthermore the introduction of the jittered condition – which was randomly interleaved with the aligned stimulus – seemed to have interfered in two of our observers with the perception of apparent depth in the aligned stimuli as well. The blue (Aligned) lines in Figure 11 for the middle two observers are shallower than for aligned stimuli shown previously such as in Figures 5–6.

In informal experiments with much smaller misalignments we found that jitter much less than 10% also could seriously degrade depth perception. It remains for future research to specify more precisely the misalignment threshold but it is <10% of the total line length.

Curvature

In the specific patterns we used, based on the Akimel O'odham (Pima) basket patterns (Breazeale 1923), the alignment of angles was necessary for 3-d perception and depth. However, these may be specific cases of a more general phenomenon where alignment of regions of high curvature produce strong perceptual effects (Ben-Shahar and Zucker 2004). As shown by pictures like the one in Figure 12, which is a detail from another painting by Bridget Riley (Arrest, 1967 reproduced in de Sausmarez, 1970) alignment of rounded curves will produce the perception of 3-d shape even without very sharp angles. There are many visual stimuli used in the scientific study of 3-d shape from pictures that resemble Figure 12 (e.g. Stevens and Brookes, 1987; Todd and Reichel, 1990; Li and Zaidi, 2001). This led us to the question, is alignment of curvature in a picture enough to evoke 3-d perception?

Figure 12.

Spatially aligned curves also evoke 3-d perception [cf. Stevens and Brookes (1987); Todd and Reichel (1990); Li and Zaidi (2001)].

Figure 13 is an attempt to answer this question by comparing a flank-roof-flank pattern with a pattern composed of circular arcs with a curvature chosen so that the arc could be inscribed within the flank-roof-flank pattern. A group of these arcs does not seem to generate the same amount of perceived depth even though the orientation change and curvature of each arc matches the average curvature of each element in the standard pattern. This suggests the importance of angles, or more generally, of regions within which curvature changes rapidly. The demonstration in Figure 12 evokes depth perception, we believe, because the curvature is not constant as in the right side of Figure 13, but rather is varying with spatial location across the pattern.

Figure 13.

Demonstration figure of the effect of removing the angles in the aligned-angle pattern while retaining the same average curvature. The pattern on the right is composed of circular arcs each of which could be inscribed in the flank-roof-flank pattern that is the elementary unit pattern on the left. Apparent depth is less for the right-hand pattern.

Upper-lower visual fields

In another experiment we presented the aligned–angle patterns in the upper and lower visual fields and compared strength of depth percept using the comparison technique. The reason for doing this is that an upper-lower field asymmetry was found for Kanizsa-type illusory contours and surfaces (Rubin et al 1995). In order to test for an analogous asymmetry in the processing of the aligned-angle patterns, we presented standard and comparison patterns above and below the horizontal meridian. This time stimuli had to be compared in a two-alternative forced choice (2-AFC single interval) task. Aligned-angle patterns were presented 2° above and below the center and we measured whether the angle – apparent depth relationship (as in Pilot Experiment 1) would differ between the upper and lower visual fields (VF). The results indicated that there was an effect of visual field location on the perception of depth in the Aligned-Angle patterns but that it was small. There is a systematic shift of the lower visual field psychometric functions to lower values of flank-slant (average points of subjective equality were 36°± 2 vs. 42°±3, in the lower vs. upper visual field), meaning that the lower visual field stimuli looked somewhat more ‘3-d’ than upper field stimuli at a smaller flank slant. However, there was not a systematic shift of sensitivity to flank-slant between upper and lower fields.

Contrast Polarity

Another issue related to physiological mechanisms, and also to comparisons with Kanizsa-type illusory figures, is dependence of the depth percept on contrast polarity. As has been known for a long time, (Prazdny, 1983; Shapley and Gordon 1987), Kanizsa and Ehrenstein figures can be formed by inducers that alternate in contrast polarity from black to white around gray. When we tested the aligned-angle pattern with alternating black and white elements, we found that contrast polarity had an effect. The process that generates depth from these patterns, and that generates illusory contours along the boundaries of illusory surfaces, does not seem to integrate black and white elements, as seen in the demonstration in Figure 14 where the mixed contrast pattern on the right looks much less 3-d than the all-black or all-white patterns on the left. The invariance of illusory contour formation with contrast polarity in Ehrenstein and Kanizsa figures was not observed in the aligned-angle patterns.

Figure 14.

Demonstration that contrast polarity has an effect on the integration of local signals to create depth perception and illusory contours and surfaces in aligned-angle patterns.

To this point, we only provided phenomenological evidence regarding the importance of angles compared to curves, and of the polarity of constituent lines, but clearly these notions require more comprehensive investigation in the future.

Discussion

Geometrical requirements for seeing depth in aligned-angle patterns

The aligned-angle patterns (for instance in the Akimel O'odham Indian baskets in Figure 2) evoke such a compelling perception of depth that one might believe them to be simply 2-d projections of 3-d objects. Then, as one of our Reviewers has suggested, the depth perception could perhaps be explained by saying that whatever allows us to see depth in 2-d projections explains the percept of the aligned angle patterns. For instance, the properties of 2-d junctions in the projected images could be used to tell the shape (as in Perkins' rules discussed in Kubovy, 1986). However, the belief in aligned-angle patterns as projections of objects is an illusion. Figure 1 is a spatially sampled version of a 2-d projection of a dihedral angle not the 2-d projection of the entire 3-d object. The perceived dihedral angle in Figure 1 is as illusory as a line made up of dots and not drawn in as a complete line. Or, in other words, {Figure 1:dihedral angle} is as {dotted line:line}. All our other Indian basket-like figures are just like Figure 1 in this respect; they are somewhat sparsely sampled representations of the 2-d projections of possible 3-d objects. The study in Pilot Experiment 2 shows that the sampling density influences the percept, which proves that the figures we used are only sampled versions of the true projected surfaces. The question still remains how the spatially-separated samples are integrated into the perception of a solid object. Similar questions of how perception combines fragmentary information into unified percepts arise in many other contexts (Wallach 1935; Meyer and Petry, 1987; Anderson 2003). This question was the starting point of our research.

Our experiments were aimed at finding what in the visual images formed by the aligned-angle patterns evoked depth perception. If the 3-d perception were a result of “shape-from-texture”, one possibility is that comparisons of spatial frequency between adjacent regions (spatial frequency contrast) would have been the explanation for the emergence of a depth percept. However, this does not explain the appearance of Figure 1 where there is no difference of spatial frequency across the illusory boundaries associated with an induced depth (there are many similar examples in Bridget Riley’s paintings (de Sausmarez 1970). Another possibility is orientation contrast between regions but this explanation does not account for the phase shift and gap experiments, and the misalignment experiment. Orientation and spatial frequency contrast do not depend on stimulus geometry , for instance, the integrity of angle- junctions, and alignment of angles in the image. Our experimental data show that disrupting the integrity of the angles in the aligned-angle pattern greatly weakened the depth percept (as shown by gap, phase-shift and occluder results) even when the orientation contrast and spatial frequency differences were the same in the disrupted as in the standard patterns. The experimental results on gaps and phase-shifts, and the occluder demonstration, point to the importance for the depth percept of the 2-d features, the angles (or 2-d corners), which are aligned in the patterns. Also consider Bridget Riley’s painting in Figure 8 where there is orientation contrast, but depth and illusory surfaces are not perceived because aligned-angles are absent.

The misalignment experiment tells us more about the crucial importance of stimulus geometry in perceiving depth in aligned-angle patterns. In the misaligned patterns there was the same kind of information about orientation contrast and spatial frequency differences between flank and roof regions as in the aligned patterns, but the perceived depth was considerably weaker in the misaligned. Furthermore, in the misaligned patterns (as in Figure 11) the angles were intact but not aligned precisely, and the misalignment severely weakened the depth percept. This argues for the importance of precise long-range alignment. We know from the art work of Bridget Riley, for instance in “Disfigured Circle” or in “Blaze”, that the illusory contours formed by the aligned angles do not have to be straight lines; they can curve and change direction but this change of direction must be smooth and graded, so that adjacent angles in the pattern are not displaced very far from each other. Then visual integration of the aligned 2-d features leads to the perception of surfaces that slant forward or backward in depth.

Relation to research on shape from texture

The patterns that inspired us (like Figures 2 and 3) and those we used in our experiments (like those in Figures 4–11) could be viewed as textures though they are very sparse textures. Therefore, it is useful to compare our results with some of the extensive scientific research on shape from texture, especially what is known about 3-d shape from texture (e.g. Todd and Reichel 1990; Li and Zaidi, 2001; Todd, 2004[review]).

Todd and Reichel (1990) showed that 3-d surface perception could be evoked by a wide range of contour patterns even when the contours were not parallel, even when contours were jagged along their length or randomized in position, and where luminance shading was removed by variations of contour thickness (as we emulated as mentioned above). What Todd and Reichel did not explore is what was central to our concern, namely the importance of stimulus geometrical alignment of surface features.

Li and Zaidi (2000, 2001a, 2001b) studied a very specific image situation: they constructed images which were 2-d perspective projections of a surface “painted” with a plaid pattern, and then corrugated (by computer) to produce a folded surface that was concave in one location and convex in another. The image on the display screen was how the pattern would look if viewed from a distance. They made the novel observation that the veridical perception of depth depended on the visibility of one particular plaid component on the uncorrugated surface: the component the orientation of which was perpendicular to the axis of the corrugation. Contours of this component would lie along lines of maximum 3-d curvature in the simulated object, and the orientation modulations of the component carry most information about the depth of the corrugated pattern. Li and Zaidi (2001a) showed the appearance of such a component in Figure 1c of that paper. The appearance of the component image resembles our Figure 12 and it generates a depth percept. Todd and Oomes (2002 Figure 2) show a similar image and remark that their observers could perceive it as bi-stable in depth. We believe that the observations of Li and Zaidi (2000, 2001a,b) and also Todd and Oomes (2002) are consistent with our observations but neither group explored the dependence of the 3-d perception on stimulus geometry that we believe is crucial for 3-d perception in such patterns.

One of the issues on which Li and Zaidi (2001a) and Todd and Oomes (2002) disagree is the requirement for perspective projection for 3-d perception from 2-d pictures. Li and Zaidi (2001a) found that perspective projection was required for 3-d perception with their corrugated plaid patterns. However, Todd and Oomes (2002) found that perspective projection was not necessary for the patterns they used to evoke 3-d percepts. Previously, Stevens and Brookes (1987) had shown that equal amounts of depth could be perceived in monocular textures whether in perspective or parallel projection (with stimuli resembling Riley's pattern in Figure 12 above). Our patterns do not have perspective projection in them and evoke strong 3-d percepts, so we agree with Stevens and Brookes (1987) and Todd and Oomes (2002) that perspective projection is not a requirement for seeing depth in pictures, in general. As stated above, one could interpret the aligned-angle patterns as discretely-sampled parallel projections of 3-d objects. A similar interpretation applies to the patterns used by, for instance, Todd and Reichel (1990). As we stated before, the interesting problem with such sampled patterns is, how does the visual system combine the contours and angles in the 2-d image to reach the perception of a 3-d solid object? As our experimental data show, the geometrical alignment of angles or regions of rapidly changing curvature seems to be one important requirement for the image to evoke a 3-d percept.

There is recent theoretical research and experiments on shape from texture that indicate the importance of curvature for seeing shapes in 2-d pictures of textures (Ben-Shahar and Zucker 2004). We believe Ben-Shahar and Zucker’s work may be related to our findings about the importance of angles, angle-alignment, and angle-density in the perception of 3-d shape in the aligned angle patterns. The demonstrations and experiments of Ben-Shahar and Zucker (2004) establish that it is not orientation contrast (and the computation of the orientation gradient) that is sufficient to explain shape from texture. Rather, the grouping of those points where curvature changes, when they are aligned along a smooth path, what Ben-Shahar and Zucker term the “flow” of local curvature, seems to define what shapes humans see when they observe texture patterns. In our experiments we are dealing with a sparse-texture-defined region where there are parallel contours. Rapid change of curvature is only present at and around the locations of the angles. The grouping of the angles into “flows” makes the aligned angle patterns evoke the perception of surfaces oriented and slanted in depth, in agreement with the theoretical concepts and suggestions of Ben-Shahar and Zucker (2004).

Visual areas in the brain and depth perception in aligned-angle patterns

Probably aligned-angle patterns activate many visual cortical areas, both lower and higher visual areas, as Kanizsa-type, illusory-contour figures do (Hirsch et al 1995; Mendola et al 1999; Stanley and Rubin 2003; Maertens and Pollmann, 2005). None of our experiments directly addresses the question of brain locus or mechanisms, but we can compare what we found with results of other studies to enable us to generate hypotheses that could be tested about the neural substrate of aligned-angle perception. What is very striking about the result on misalignment is that the depth percept depended on precise spatial alignment of the inducing elements. This by itself suggests some dependence on early visual cortical areas where precise retinotopic mapping is retained (e.g. Wandell 1999). However, the perception was of 3-d depth, and this presumably requires some activity in extra-striate cortex (cf. Tyler et al 2006). The brain locus of sensitivity to angles or 2-d corners probably begins in primary visual cortex, V1, according to the fMRI results of Troncoso et al (2007). Single visual neurons that combine signals from pairs of orientations have been found in area V2 of macaque monkeys (Anzai et al 2007). However, such V2 neurons may not be sensitive to angles per se since Ito and Komatsu (2004) found that, in macaque V2, the neurons that responded to angles also responded to individual line elements that composed the angles. Investigating macaque V4 neurons, Pasupathy and Conner (1999) found cells that were angle-specific, that only responded to a completed angle and not to the line elements in isolation. Such neurons could be the neuronal basis for sensitivity to the integrity of angles in the aligned-angle patterns, as in the gap and phase experiments, and in the occluder demonstration. However, up to now, there have been no neurophysiological studies about depth signals in aligned-angle patterns or curvature flows, and therefore no neuronal account of the sensitivity of depth perception to the alignment of angles in the misalignment experiment. This might be an important target for future research on visual perception of depth in pictures.