Natural-scene geometry predicts the perception of angles and line orientation

Abstract

Visual stimuli that entail the intersection of two or more straight lines elicit a variety of well known perceptual anomalies. Preeminent among these anomalies are the systematic overestimation of acute angles, the underestimation of obtuse angles, and the misperceptions of line orientation exemplified in the classical tilt, Zöllner, and Hering illusions. Here we show that the probability distributions of the possible real-world sources of projected lines and angles derived from a range-image database of natural scenes accurately predict each of these perceptual peculiarities. These findings imply that the perception of angles and oriented lines is determined by the statistical relationship between geometrical stimuli and their physical sources in typical visual environments.

The overestimation of the subtense of acute angles and the underestimation of obtuse ones was reported first by Wundt (1) and has been confirmed repeatedly by modern studies (2–7) (Fig. 1A). This subtle yet robust phenomenon is closely related to a series of more complex geometrical illusions elicited by intersecting lines. The simplest of these is the tilt illusion, in which the presentation of a vertical line in the context of an obliquely oriented line causes the vertical line to appear tilted slightly away from the oblique one (Fig. 1B). The direction of the illusory tilt is consistent with a perceptual enlargement of the acute angles in the stimulus and a reduction of the obtuse angles. The magnitude of this misperception is enhanced by an iteration of stimulus elements in the more elaborate Zöllner illusion (Fig. 1C). A further permutation is the Hering illusion, in which two straight lines appear curved or bowed in the context of intersecting lines with orientations that change progressively (Fig. 1D). Despite numerous studies (reviewed in ref. 8), the basis for these related phenomena is not known.

Fig. 1.

The misperception of angles and line orientation. (A) The overestimation of acute angles and the underestimation of obtuse ones (7). (B) Tilt illusion. The vertically orientated test line (gray) appears to be tilted slightly counterclockwise in the context of an obliquely oriented “inducing line” (black). (C) Zöllner illusion. The parallel vertical test lines (gray) appear tilted in the direction opposite that of the contextual lines (black). (D) Hering illusion. The two straight parallel test lines (gray) appear bowed.

The explanation of these effects that we test here is predicated on the fundamental ambiguity of geometrical stimuli. Because the size, distance, and orientation of physical objects are conflated in any projection of light, the real-world sources of the geometrical elements in an image cannot be derived from the retinal stimulus as such. Nonetheless, the observer must respond appropriately to the physical sources underlying the image on the retina. A growing body of evidence has suggested that the visual system copes with this biological quandary by generating percepts according to the probability distribution of the physical sources of the retinal images (see, for example, refs. 9–11). As a result, percepts do not necessarily accord with the physical measurements of the objects or conditions underlying the retinal image, giving rise to the variety of “optical illusions.” Motivated by the success of this probabilistic framework in explaining several other geometrical anomalies (12–14), we ask here whether the phenomena illustrated in Fig. 1 are predicted by the statistical relationship between the various stimuli and their possible real-world sources. To test this idea we used laser range scanning to obtain a database of natural scenes that included the location in 3D space of every point in the scenes, allowing us to relate a given pattern in an image to the spatial configuration of its real-world source.

Materials and Methods

The Range-Image Database. The details entailed in acquiring the range information used here have been described (12). Briefly, these data were obtained by using an LMS-Z210 3D laser scanner (Riegl USA, Orlando, FL), which provided digitized images with distance as well as luminance information for every pixel in the scene. The range-finding performance of the system is from 2 to ≈300 m, with an accuracy of ±25 mm over the full range at an angular resolution of 0.144°. Using this system, we collected 103 wide-field images (extending 333° horizontally and 80° vertically), including 25 fully natural scenes and 78 scenes that included buildings and other human constructions (Fig. 2A). Each of these wide-field range images is a spatial map of the 3D physical world in a spherical polar coordinate system, the origin of which is the origin of the laser beam.

Fig. 2.

Sampling the range-image database with geometrical templates. (A) Representative scene acquired by laser range scanning (only a small portion of the full wide-field image obtained by the scanner is shown here). A true color image (Upper) and the corresponding range image (Lower) are shown; the distance from the origin of laser beam to each point in the range image is indicated by color coding. Black areas (sky) are points in the scene from which no laser reflection was recorded. (B) The pixels in a 2D image projected from a region of the 3D scene in the database are represented diagrammatically by the grid squares. The black dots indicate the reference-line template, and the red dots indicate a series of templates for sampling a second line oriented at various angles with respect to the reference line. (C) The set of white dots indicates a valid sample for a reference line in this example (i.e., the set fulfills the geometrical criteria described in Materials and Methods). Blowups of the boxed area show examples of a second template that was successively overlaid on the same area of the image to sample for the presence of a second straight line in different orientations (the second template in each of these cases is also a valid sample).

From these images we generated a large number of projections approximating the central retinal images that would have arisen from viewing different regions of these scenes (see ref. 12 for details; each pixel in the images subtended 0.144° of visual angle). The result of each such projection was a 2D image of a portion of the original wide-field scene, together with complete information about its geometrical relationship to the corresponding part of the 3D world.

Determination of the Physical Sources of Angles. When considering angles formed by intersecting straight lines, it is natural to think of stimuli generated by luminance contrast in a scene. Luminance contrast boundaries (i.e., edges) have traditionally been the focus of vision research, primarily because edges typically correspond to object boundaries and are therefore considered “information-rich” (see, for example, ref. 15). Another reason for this focus in past research is that edges generally elicit much stronger responses in visual cortical neurons than do stimuli that lack contrast boundaries. Despite the obvious importance of edges in vision, from a purely geometrical point of view straight lines are not limited to those that happen to coincide with luminance contrast borders. For both geometrical and behavioral purposes, a straight line is simply a set of points with positions in space that conform to a linear progression. Because understanding the perception of geometry and its behavioral consequences is the goal here, we define straight lines in terms of geometry rather than contrast: the straight lines extracted from the database in what follows are sets of points that form straight lines by geometrical criteria (see below). Because this geometrical approach may seem counterintuitive to readers used to thinking about visual-scene analysis (and vision) primarily in terms of contrast, additional explanation is included in the supporting information, which is published on the PNAS web site.

Sampling with Geometrical Templates. To identify the physical sources of two intersecting lines forming particular angles, we first identified regions of scenes in the database that contained a valid physical source of one of the two lines (the reference line) by using a straight-line template (Fig. 2B). The template for the reference line, which corresponded to a visual angle of 1.6°, was oriented at 0° (horizontal), 45°, or 90° and applied to the 2D images transformed from the 3D scenes (Fig. 2C). The set of points underlying the reference-line template in the image was then screened to determine whether it corresponded to physical points that fell on a straight line on surfaces in the 3D world. For this purpose, a straight line was fitted to the relevant physical points by using the least-squares method. The set of points in the 3D scene was accepted as a valid sample of the physical source of the reference line if the average deviation of the points from the fitted line was less than an arbitrary standard (2% of the average distance of that set of points from the image plane; this standard was used because it was equally stringent for sets of image points with physical counterparts that were located at different distances). Applying this geometrical criterion ensured that the straight line in the image was generated by the projection of a bona fide straight line in 3D space.

After identifying a valid sample of the physical source of the reference line in this way, we then overlaid an additional straight-line template on the sample image to determine the probability of occurrence of a second line in the same region of the scene that formed an angle with the reference line (see Fig. 2B). The angle between the reference line and this second line was varied from 5° to 175° in 5° increments (the subtense of the angle was defined by the rotation required to move the reference line counterclockwise to superposition with the second line). The points underlying this second template then were examined to determine whether they also corresponded to a straight line in 3D space by using the criteria described above. If this further condition was met, the sample was counted as a valid physical source of the angle specified by the reference line and the line identified by the second template.

Each of the templates used in sampling the database consisted of 11 evenly spaced points (see Fig. 2B). Because the images were comprised of discrete pixels, the points in the template did not correspond precisely to the pixels in the images except when the templates were horizontal or vertical. The range (distance) information for each point in the templates was therefore interpolated according to (c₁R₁ + c₂R₂ + c₃R₃ + c₄R₄)/(c₁ + c₂ + c₃ + c₄), where R₁, R₂, R₃, and R₄ are the range values of the four pixels nearest to the point of interest, and the coefficients c₁, c₂, c₃, and c₄ are the reciprocals of the distances of the centers of these four pixels from the point of interest in the image plane.

This general sampling procedure was repeated for each of the ≈10⁶ 2D image projections generated from the 3D scenes. The total number of valid samples obtained in this way was ≈4.4 × 10⁶ (≈3 × 10⁴ to 2 × 10⁵ samples for each of the angles and orientations tested).

Determination of Probabilities. Finally, we accumulated the number of valid samples of the physical sources of angles identified in this way, thus obtaining a distribution of the frequency of occurrence of the physical sources of various angles in natural scenes. Normalizing each of the frequency distributions generated the corresponding probability distributions (P_A) of the number of occurrences of the physical sources of angles. Cumulative probabilities [i.e., F_A (A ≤ x), the summed probability of occurrence of the physical sources of all angles (A) less than or equal to a specific subtense x] were then obtained by calculating the area under the curves of P_A to the left of the line signifying A = x. These probabilities were used to predict the perceptual consequences of viewing angular stimuli.

Results

The Probabilistic Approach to Rationalizing Angle Perception. Given the fact that visual animals must respond appropriately to the physical source of a stimulus despite the uncertain significance of the retinal image, the hypothesis tested here is that visual perceptions of angles and line orientations are generated according to the probability distribution of the possible physical sources of the retinal stimulus. Each point in the probability distribution of the physical sources of angles obtained by using the method just described indicates the relative frequency of occurrence of all the physical conditions that generate an angle of a specific subtense. Such a distribution is, for all intents and purposes, an empirical scale that ranks the angle subtenses experienced by human observers: for any given angle in a projected image, the distribution indicates the percentage of all the physical sources experienced by observers that generated angle projections smaller than the subtense of that angle and the percentage that generated angle projections with larger subtenses. The apparent subtense of a particular angle, according to the hypothesis, should be predicted by the rank of that angle in the accumulated past experience with angles and their physical sources (i.e., according to the frequency of occurrence of the physical sources that projected as angles smaller than that angle, compared to the occurrence of sources that generated larger angle projections). In this conception, the higher the percentage of physical sources that in past experience generated angles with smaller subtenses than the angle at issue, the higher that angle ranks on the empirical scale defined by the probability distribution and thus the larger the perceived subtense of that angle.

Probability Distributions of the Physical Sources of Angles. Fig. 3 shows the probability distributions derived from the range-image database, showing the relative frequencies of occurrence of the physical sources of various angle projections. It is apparent that, regardless of the three different orientations of the reference line (horizontal, oblique, and vertical, indicated by the black lines in the icons under the graphs) or the type of scene from which the statistics were derived (natural scenes or scenes containing human artifacts), the distributions form a trough with lower values for angles at or near 90°. In other words, given any particular reference line in the image, the probability of occurrence of a second line forming an angle with it decreases as the second line becomes increasingly orthogonal to the reference line.

Fig. 3.

The normalized frequency of occurrence of the physical sources of angles with various subtenses, given the presence of the physical source of a horizontal (Left), oblique (Center), or vertical (Right) reference line. The distributions were obtained from the range-image database by applying appropriate geometrical templates as indicated in Fig. 2. The icons below the graphs indicate the orientations of the reference-line (black) and second-line (gray) templates. Note that results obtained from fully natural scenes (Upper) are not significantly different than those from “carpentered” environments (Lower).

The physical basis of these statistical observations can be understood by considering the sources of straight lines in the real world. Because any nonaccidental straight-line projection in the image plane arises from a straight line in 3D space (see Materials and Methods) and a straight line in 3D space belongs to a planar surface patch, the presence of two intersecting lines in the image typically signifies the presence of a plane in the corresponding region of the scene. A planar surface that contains the physical sources of two lines intersecting at 90° would have, on average, a larger area than a surface that contains the sources of two lines that are less orthogonal. The probability of occurrence of larger planar surfaces is naturally lower than the occurrence of smaller ones, because large surfaces include smaller ones. Accordingly, the occurrence of the physical sources of angles near 90° is statistically lower than the occurrence of the sources of angles near 0° or 180°.

Predicting the Perception of Acute and Obtuse Angles. Because the results derived from different types of scenes, as well as the results by using reference lines at different orientations, are similar, the probability distributions in Fig. 3 were pooled for additional analysis. Fig. 4A shows the cumulative probability distribution derived from this overall distribution. A cumulative probability value is the summed probability of occurrence of the sources of all angles less than or equal to a given subtense (see Materials and Methods). Thus, for any given angle subtense, the corresponding cumulative probability value indicates the percentage of the physical sources of all angle projections that would have projected as angles smaller than the given subtense and the percentage that would generate angle projections larger than the angle in question. For example, a 30° projected angle corresponds to a cumulative probability of 0.185 on the gray curve in Fig. 4A, meaning that ≈19% of the physical sources of angles generated projected angles ≤30° and ≈81% generated angles >30°. When compared to a cumulative distribution derived from a hypothetical probability distribution in which the probability is uniform for the physical sources of all angles (black line in Fig. 4A), the cumulative distribution of the physical sources of angles derived from the range-image database (gray curve) shows higher probability values for angles <90° and lower probability values for angles >90°.

Fig. 4.

Predicting the perceived angle subtense from the probability distributions of angle sources in the database. (A) The gray curve is the cumulative probability distribution of the physical sources of angles derived from the probability distribution pooled from the six distributions shown in Fig. 3. The black line indicates the cumulative probability distribution derived from a hypothetical distribution in which the probability of the sources of all possible angle projections is the same (see Inset). (B) The predicted perceptions of angle subtenses are shown by the gray curve; the dashed black line indicates percepts corresponding to the actual subtenses of the retinal stimuli. (C) The magnitude of angle misperception predicted by this analysis (gray curve) compared with psychophysical measurements of angle misperception (bars; data are from ref. 7).

If, as we hypothesize, the perception of any projected angle is generated probabilistically, then the angles seen should accord with their relative positions (i.e., ranks) on the empirical scale of angle subtenses defined by the cumulative probability distribution of the sources of angles. As indicated in Fig. 4A, if the probability of all angle subtenses were uniform, the spectrum of possible angles would have evenly distributed positions on this empirical scale, i.e., an angle of x° would have always corresponded to a cumulative probability of x/180. In this case, the position of an angle x on the empirical scale would always be the same as the position of the angle in the geometrical space of 0–180°, and thus the perceived subtense of any angle would always match its actual subtense.

The distribution of the occurrences of the physical sources of angles derived from the image database, however, is not uniform (see gray line in Fig. 4A). For any angle x between 0° and 90°, the cumulative probability is somewhat greater than x/180, meaning that the position of any such angle x on this empirical scale is shifted slightly in the direction of 180° compared to its position in the geometrical space of 0–180°. The opposite is true for any angle between 90° and 180°. To illustrate this point, consider again an angle of 30°. The position of this angle in geometrical space is 30/180. The cumulative probability corresponding to 30°, however, is 0.185 (≈33/180), which is larger than 30/180, meaning that a 30° angle ranks higher than 30/180 among all angle projections experienced by human observers. In contrast, any obtuse angle of x° would rank lower than x/180 on the empirical scale of angle subtenses. Thus, the positions of both acute and obtuse angles in the empirical space of angles are shifted systematically toward 90° compared to their positions in the geometrical space. Because in this framework the predicted subtenses of the perceived angles are given by the cumulative probability for any angle x multiplied by 180 (Fig. 4B), the subtense of any angle between 0° and 90° should be systematically enlarged in perception, whereas the subtense of angles between 90° and 180° should be reduced. A comparison of the angle misperceptions actually seen by subjects and those predicted by the present analysis shows good agreement (Fig. 4C).

Explanation of the Tilt, Zöllner, and Hering Illusions. The apparent tilt of a vertical line caused by the context of a tilted line (see Fig. 1B) can be explained in this same way. Consider, for example, the probability of occurrence of the physical sources of a second line oriented at various angles, given a reference line oriented at 60° from the horizontal (Fig. 5A). (This orientation of the reference line was chosen because it is frequently used to demonstrate the tilt effect; the argument that follows, however, applies to an inducing line at any orientation.) The probability distribution derived in this way shares the characteristics of the distributions shown in Fig. 3, in which the position of a vertical line (i.e., a line rotated 30° counterclockwise from the reference line) is indicated by the dashed line in Fig. 5A. The cumulative probability value associated with this angle [i.e., F_A (A ≤ 30)] is 0.184, which, when multiplied by 180, predicts that the perceived angle between the reference line and the vertical line should be 33.2°, or 3.2° greater than the actual angle between the two lines. Accordingly, the vertical line in the context of the line oriented at 60° should be perceived as being tilted away from the reference line slightly more than it actually is, thus appearing not quite vertical (Fig. 5B). This prediction is again consistent with what observers see in response to this sort of stimulus (see Fig. 1B and refs. 4, 16, and 17).

Fig. 5.

Statistical explanation of the tilt illusion. (A) Probability distribution of the physical sources of projected angles in the database, given a reference line oriented 60° from the horizontal (indicated by black in the icons below the graph). (B Left) Standard presentation of the tilt illusion, as shown in Fig. 1B; the gray line is vertical. (B Right) The direction (arrows) and magnitude of the tilt effect predicted by the distribution shown in A. The solid gray line is vertical; the dotted line indicates the predicted shift in the apparent position of the vertical line shown on the left.

The Zöllner illusion (see Fig. 1C) is essentially an iteration of such simple tilt stimuli, producing an overall effect in which the vertical test lines appear to be tilted away from the contextual lines more markedly. Similarly, the parallel lines in the Hering stimulus (Fig. 1D) appear bowed because of the concatenation of simple tilt stimuli. In this case, the upper and lower halves of the contextual lines tilt in opposite directions, causing the upper and lower halves of the test lines to appear progressively tilted in different directions, resulting in the apparent bowing.

Discussion

The basic quandary in visual perception is outlined in the Introduction: the physical sources of any given retinal image, to which the observer must respond, are not specified by the information in the retinal image. A plausible biological solution to this dilemma is to generate percepts that accord with the probability distribution of the possible physical sources of the retinal stimulus. The misperception of angles and the related geometrical illusions considered here are well predicted by this statistical relationship. This strategy of perception, which presumably is pertinent to all other geometrical stimuli, provides a way of maximizing the chance of successful visually guided behavior in the face of stimuli with significance that cannot be known directly (see the supporting information for an explanation of how this framework differs from Bayesian approaches).

Neural Mechanisms Underlying the Perception of Angles and Line Orientation. This explanation of the misperception of angles and line orientations also provides a framework for understanding the neural mechanisms underlying this aspect of geometrical processing. It has long been known that many neurons in the visual cortex respond selectively to lines or edges in particular orientations (18, 19). Moreover, orientation-selective cells tend to be spatially organized such that there is a systematic progression in the preferred orientation of the cells within any given patch of the visual cortex (20). Given the well documented existence of lateral inhibition among sensory elements in the input stages of the visual system (21, 22), a natural supposition is that similar lateral inhibitory effects among the cells in the visual cortex that selectively respond to lines and edges in various orientations might underlie the anomalous percepts illustrated in Fig. 1 (2, 4, 16, 23–25). In this conception, the orientation domains coactivated by the two angle arms would presumably inhibit each other, thus shifting the distribution of the cortical activity toward orientation domains further apart than would otherwise be the case.

Although this idea predicts the perceptual enlargement of acute angles, it does not provide a physiological basis for the underestimation of obtuse angles. Another difficulty is that, although some initial electrophysiological studies found lateral inhibitions among orientation-selective neurons in the visual cortex (26, 27), more recent work has shown that the interactions among visual cortical neurons is far more complex than the simple inhibitory effects originally envisioned (28–30). Thus, the effect of contextual line stimuli on the response to a target line segment can be inhibition, facilitation, or some combination thereof depending on a host of factors (e.g., the orientation and length of the stimuli and the other receptive properties of the neurons involved). As a result, there is presently no consensus about how such interactions should be interpreted or how they are related to the perception of angles and/or line orientation.

The alternative suggested by the present results is that these complex cortical interactions are a means of instantiating the empirical associations between intersecting line projections and their sources, the full spectrum of contextual interactions apparent in visual cortex reflecting the full range of possible image-source relationships. The pattern of cortical activity in response to two intersecting lines, in this conception, is determined by the empirical distribution of the real-world sources of intersecting lines, with the activity of individual neurons representing different points in the distribution. Because in the empirical space of angles defined by this distribution the angle between two intersecting lines is always shifted toward 90° compared to the actual angle in the retinal stimulus (see Fig. 4A), the peaks of the neuronal activity elicited by the angle would be shifted toward orientation domains more orthogonal in their selectivity than the activity peaks elicited by each line alone. In addition to providing a unified framework for understanding neuronal responses to both acute and obtuse angles, this prediction offers a different way of considering the relevant neurophysiology. Instead of being regarded as an epiphenomenon of cortical processing, the altered peaks of cortical activity in this interpretation signify the accumulated statistical information conveyed by the past experience of the species and the individual. The merits of this interpretation of the physiological responses elicited by angles and lines at different orientations can be tested with functional imaging techniques or other methods that reveal patterns of cortical activity.

Conclusion. Although the distortions that occur when viewing acute or obtuse angles in the absence of other contextual information may seem trivial with respect to the success or failure of human behavior, the visual strategy they signify lies at the core of vision. The advantage of the probabilistic strategy reflected in these perceptual anomalies is that the relationships among objects in the physical world are preserved in perceptual space, ensuring that the perceptions of the observer provide the most beneficial guide to action in the face of the inevitably uncertain meaning of retinal images.