Computer vision, camouflage breaking and countershading

Abstract

Camouflage is frequently used in the animal kingdom in order to conceal oneself from visual detection or surveillance. Many camouflage techniques are based on masking the familiar contours and texture of the subject by superposition of multiple edges on top of it. This work presents an operator, D_arg, for the detection of three-dimensional smooth convex (or, equivalently, concave) objects. It can be used to detect curved objects on a relatively flat background, regardless of image edges, contours and texture. We show that a typical camouflage found in some animal species seems to be a ‘countermeasure’ taken against detection that might be based on our method. Detection by D_arg is shown to be very robust, from both theoretical considerations and practical examples of real-life images.

1. Introduction

Visual camouflage is used by animals as well as humans in order to conceal or obscure their visual signature. In the field of computer vision, work related to camouflage can be roughly divided into two: camouflage assessment and design (e.g. Copeland & Trivedi 1997; Gretzmacher et al. 1998), and camouflage breaking. Despite the ongoing research, only little has been said in the computer vision literature on visual camouflage breaking (Marouani et al. 1995; Guilan & Shunqing 1997; McKee et al. 1997; Ternovskiy & Jannson 1997; Huimin et al. 1999).

This paper addresses the issue of camouflage breaking from a computer vision point of view. For this task, we present a mathematical operator which is based on the assumption that the concealed subject is a smooth three-dimensional convex object. Thus, the goal of the operator, called D_arg, is to detect three-dimensional convex or concave objects in two-dimensional representations (Tankus et al. 1997; Tankus & Yeshurun 1998). D_arg is applied directly to the grey-level function of the image. It responds to smooth three-dimensional convex or concave patches in objects and is not limited by any particular light source or reflectance function. It does not attempt to restore the three-dimensional scene and is a very robust operator that can detect subjects in highly cluttered scenes even under camouflages classified by human viewers as very hard to break. In contrast to the existing attempts to break camouflage (Marouani et al. 1995; Guilan & Shunqing 1997; McKee et al. 1997; Ternovskiy & Jannson 1997; Huimin et al. 1999), our operator is context free; its only a priori assumption about the target is its being three-dimensional and convex (or concave). In order to evaluate the performance of the operator in breaking camouflage, we juxtaposed D_arg with a representative edge-based operator. Only a small portion of the comparison can be provided in this paper (but see also Tankus et al. 1999 and Tankus & Yeshurun 2001).

We present biological evidence that detection of the convexity of the grey-level function may be employed by visual systems of predators to break the camouflage. This is based on Thayer's principle of countershading (Thayer 1896a,b, 1909; Poulton & Thayer 1902), which observes that some animals, whose body is three-dimensional convex, use apatetic coloration to prevent their image (under sunlight) from appearing as convex grey level to a viewer (see also Boynton 1952; Portmann 1959; Behrens 1978, 1988). This implies that other animals may break the camouflage based on the convexity of the grey levels they see (or else there was no need in such an apatetic coloration). For flat animals (e.g. moths), countershading is inappropriate, and other camouflage strategies (general resemblance and disruptive patterns) may be used. Hence, other breaking methods should be considered for these cases, but these are outside the scope of this paper.

The biological literature investigates the role of countershading in specific species (as individuals or colonies) and explains their usage for the camouflage (Stauffer et al. 1999; Braude et al. 2001). Several studies quantified the effectiveness of countershading as a camouflage showing that countershaded prey had significantly lower levels of predation than non-countershaded controls, and that countershading was effective against some species of prey birds but not others (Edmunds & Dewhirst 1994; Ruxton et al. 2004; Speed et al. 2005; Rowland et al. 2007). Luminescent countershading is used by fish, squid and shrimp in order to remain cryptic to silhouette scanning predators. The current research deals with the effects of lighting and gravity on countershading reflexes in these species (Ferguson et al. 1994; Latz 1996; Lindsay et al. 1999; Blake & Chan 2007). Recently, a new species of disc-winged bat (Thyroptera devivoi) has been described, and found to have a distinct countershading with dark brown dorsal fur, which is in contrast to pale brown ventral fur with frosted tips (Gregorin et al. 2006).

Section 2 defines the operator D_arg for convexity-based detection. Section 2a gives intuition for D_arg and is of particular importance for understanding its behaviour. Section 3 uses D_arg for camouflage breaking. Section 3a brings the biological evidence for camouflage breaking by the detection of grey-level convexity. Section 3b establishes the connection between the biological evidence and the specific convexity detector D_arg. Section 4 delineates a camouflage breaking comparison of an edge-based method with our convexity detector. Concluding remarks are given in §5.

2. Y_arg, D_arg: operators for the detection of convex domains

We next define an operator for the detection of three-dimensional objects with smooth convex and concave domains.

Let I(x,y) be an input image, and $\nabla I(x,y)=((\partial /\partial x)I(x,y),(\partial /\partial y)I(x,y))$ the Cartesian representation of the gradient map of I(x,y). Let us convert ▽I(x,y) into its polar representation (i.e. to the magnitude and direction of the gradient). The gradient argument is defined by

$$\theta (x,y)=\text{arg}(\nabla I(x,y))=\text{arctan}\left(\frac{\partial }{\partial y}I(x,y),\frac{\partial }{\partial x}I(x,y)\right),$$

where the two dimensional arctangent is

$$\text{arctan}(y,x)=\{\begin{array}{ll}\text{arctan}\left(\frac{y}{x}\right),\hfill & \text{if}x\ge 0\hfill \\ \text{arctan}\left(\frac{y}{x}\right)+\pi ,\hfill & \text{if}x<0,y\ge 0\hfill \\ \text{arctan}\left(\frac{y}{x}\right)-\pi ,\hfill & \text{if}x<0,y<0\hfill \end{array}$$

and the one-dimensional arctan(t) denotes the inverse function of tan(t) so that $\text{arctan}(t):[-\infty ,\infty ]\mapsto [(-\pi /2),(\pi /2)]$.

The proposed convexity detection mechanism, which we denote Y_arg, is simply the y-derivative of the argument map

$$Y_{\text{arg}}=\frac{\partial }{\partial y}\theta (x,y)=\frac{\partial }{\partial y}\text{arctan}\left(\frac{\partial }{\partial y}I(x,y),\frac{\partial }{\partial x}I(x,y)\right).$$

To obtain an isotropic operator based on Y_arg, we rotate the original image by 0°, 90°, 180° and 270°, operate Y_arg and rotate the results back to their original positions. The sum of the four responses is the response of an operator, which we name D_arg (the name was chosen to represent Differentiation of the ARGument (i.e. direction) of the gradient).

(a) Intuitive description of the operator

(i) What does Y_arg detect?

Y_arg detects the zero crossings of the gradient argument. This stems from the last step of the gradient argument calculation: the two-dimensional arctangent function. The arctangent function is discontinuous at the negative part of the x-axis; therefore, its y-derivative approaches infinity there. In other words, Y_arg approaches infinity at the negative part of the x-axis of the arctan, when this axis is being crossed. This limit reveals the zero crossings of the gradient argument (see Tankus et al. 1997 for more details).

(ii) Why detect zero crossings of the gradient argument?

Y_arg detects zero crossings of the gradient argument of the intensity function I(x,y). The existence of zero crossings of the gradient argument enforces a certain range of values on the gradient argument (trivially, values near zero). Considering the intensity function I(x,y) as a surface in three-dimensional space, the gradient argument ‘represents’ the direction of the normal to the surface. Therefore, a range of values of the gradient argument means a certain range of directions of the normal to the intensity surface. This enforces a certain structure on the intensity surface itself.

In Tankus et al. (1997), we have characterized the structure of the intensity surface as either a paraboloidal structure or any differentiable strongly monotonically increasing transformation of a paraboloidal structure (figure 1). Since paraboloids are arbitrarily curved surfaces, they can be used as a local approximation of three-dimensional convex or concave surfaces (recall that our input is discrete, and the continuous functions are only an approximation!). The detected intensity surface patches are therefore those exhibiting three-dimensional convex or concave structure. The convexity is three-dimensional, because it refers to the convexity of the intensity surface I(x,y), which is a two-dimensional surface in three-dimensional space (figure 1b), not convexity of contours (which is one-dimensional curve in the two-dimensional plane). The three-dimensional convexity of the intensity surface is characteristic of intensity surfaces emanating from smooth three-dimensional convex bodies.

Figure 1.

(a) Paraboloidal grey levels: I(x,y)=x²+5y². (b) The paraboloidal grey levels of (a), presented as a two-dimensional surface in three-dimensional space. (c) Gradient argument of (a). Discontinuity ray at the negative x-axis. (d) Y_arg of (a) (=(∂/∂y) of (c)). (e) Rotation of (a) (90° c.c.w.), calculation of gradient argument and inverse rotation. (f) Rotation of (a) (90° c.c.w.), calculation of Y_arg and inverse rotation. (g) Response of D_arg, the isotropic operator. (h) D_arg² (the square of (g)).

(iii) How to detect zero crossings of the gradient argument?

Zero crossings of the gradient argument can be detected in various methods. The trivial method would be to compute the gradient argument, and search for a change of sign in it. A more sophisticated method would be to smooth the gradient argument map beforehand (e.g. by a convolution with a Gaussian), in order to make the detection more robust. The suggested operator is even more robust to noise, due to the approach to infinity described above. In practice, this approach to infinity appears as a very strong response whenever zero crossing takes place. The approach is robust to scale changes in the detected subject, various lighting conditions and orientation (pose) of the subject (see Tankus & Yeshurun 2001).

(iv) Summary

We detect the zero crossings of the gradient argument by detecting the infinite response of Y_arg at the negative x-axis (of the arctan). These zero crossings occur where the intensity surface is three-dimensional convex or concave. Convex smooth three-dimensional objects usually produce three-dimensional convex intensity surfaces. Thus, detection of the infinite responses of Y_arg results in the detection of the domains of the intensity surface, which characterize three-dimensional smooth convex or concave subjects.

3. Camouflage breaking

The robustness of the operator under various conditions (illumination, scale, orientation, texture) has been thoroughly studied in Tankus et al. (1997). As a result, the smoothness condition of the detected three-dimensional convex objects can be relaxed (i.e. the surface may not be smooth and contain edges). In this paper, we further increase the robustness demands from the operator by introducing very strong camouflage.

(a) Biological evidence for camouflage breaking by convexity detection

Next, we exhibit evidence of biological camouflage breaking based on the detection of the convexity of the intensity function. This matches our idea of camouflage breaking by direct convexity estimation (using D_arg). We bring further evidence that not only can intensity convexity be used to break the camouflage, but also there are animals whose colouring is suited to prevent this specific kind of camouflage breaking.

It is well known that under directional light, a smooth three-dimensional convex object produces a convex intensity function. The biological meaning is that when the trunk of an animal (the convex subject) is exposed to top lighting (sunlight), a viewer sees shades (convex intensity function). As we shall see, these shades may reveal the animal, especially in the surroundings that break up shadows (e.g. woods) (see Portmann 1959). This supports the D_arg approach of camouflage breaking by detecting the convexity of the intensity function.

It has been suggested that the ability to trace an animal based on these shadow effects has led in many animals, during thousands of years of evolution, to coloration of animals that dissolves the shadow effects. This countershading coloration was first observed at the end of the nineteenth century (Thayer 1896a,b, 1909; Poulton & Thayer 1902), and is known as Thayer's principle. Portmann describes Thayer's principle: ‘If we paint a cylinder or sphere in graded tints of gray, the darkest part facing toward the source light, and the lightest away from it, the body's own shade so balances this color scheme that the outlines becomes dissolved. Such graded tints are typical of vertebrates and of many other animals.’ (Portmann 1959). When the animal is under top lighting (usually sunlight), the gradual change in albedo neutralizes the convexity of the intensity function. Had no countershading been used, the intensity function would have been convex, exposing the animal to convexity-based detectors (such as D_arg). Figure 2a (upper row) uses ray tracing to demonstrate Thayer's principle of countershading when applied to cylinders. It presents three cylinders: A, a cylinder of constant albedo under top lighting; B, a countershaded cylinder under ambient lighting (produced by mapping a convex texture); and C, a cylinder with the combined effect of countershading albedo and top lighting. While the first two cylinders produce convex grey levels (i.e. a gradual change in intensity), the countershaded one breaks up the shadow effect (equal to convex intensity function); its intensity map is flat.

Figure 2.

(a) Operation of D_arg² on a countershaded cylinder. Column A: a smooth cylinder under top lighting. Column B: the countershaded cylinder under ambient lighting. Column C: the countershaded cylinder under top lighting. The countershaded cylinder can barely be noted under top lighting, due to the camouflage. Under top lighting, the response of D_arg is much stronger when the cylinder is smooth than when it is countershaded, showing that this type of camouflage is effective against D_arg. (b) Cross sections (parallel to the y-axis, at the centre of the image) of (i) the intensity functions. Thayer's countershading yields a flat intensity function for a cylinder. (ii) D_arg². Dashed curve, column A; dotted curve, column B; solid curve, column A. Under top lighting, the flattened intensity function of the countershaded cylinder has a lower D_arg response than that of the convex intensity function of the smooth cylinder.

The existence of countermeasures to convexity-based detectors implies that there might exist predators that can use convexity-based detectors similar to D_arg.

(b) Thayer's countershading against D_arg-based detection

Let us demonstrate how Thayer's principle of countershading can be used to camouflage against D_arg-based detectors. In figure 2a (lower row), we operate D_arg on each of the images of the cylinders (of the upper row). As can be seen, the countershaded cylinder under top lighting (figure 2a, column C) attains much lower D_arg values than the smooth cylinder under the same lighting (figure 2a, column A). This is because countershading turns the intensity function from convex to (approximately) planar.

To see the transition from a convex intensity function to a planar one due to camouflage, we draw (figure 2b(i)) the vertical cross sections of the intensity functions of the cylinder images. The smooth cylinder under top lighting (column A) produces a convex cross section. The albedo, or the countershaded cylinder under ambient lighting (column B), consists of graded tints of grey (i.e. convex countershading). Finally, the countershaded cylinder under top lighting (column C) produces a flat intensity function, which means a lower probability of detection by D_arg.

We verify that the flat intensity function is indeed harder to detect using D_arg than the convex intensity function: we show that D_arg has a lower response to the countershaded cylinder under top lighting than it has to the smooth cylinder under the same lighting. This is obvious from figure 2b(ii), which shows the vertical cross sections of the responses of D_arg to the various images of the cylinder.

This demonstrates that Thayer's principle of countershading is an effective camouflage technique against convexity-based camouflage breakers and, more specifically, against D_arg. One can thus speculate that convexity-based camouflage breaking might also exist in nature (or else, the camouflage against it would be unnecessary).

(c) Neuronal implementation of D_arg

In order for an operator to be employed by a visual system of a predator, its neuronal implementation should be feasible. We next suggest a possible implementation for D_arg based on the hypercolumns of the primary visual cortex (V1) (Hubel & Wiesel 1974). A hypercolumn is a set of cortical columns, each of which is responsive to a certain orientation of lines in its visual field. The hypercolumn contains the full range of orientation preferences (0° to 180°) and is organized around pinwheels, with one set of preferences for each ocular dominance column (Levine 1985; Bressloff & Cowan 2003). If, while watching an input image, the output of cells in this hypercolumn is weighted according to the direction they represent (figure 3a), local differentiation of the outputs will implement Y_arg, yielding a high response at the negative x-axis as required. While differentiation can be implemented as a difference between neural outputs, summation of outputs is far more common in the cortex. Changing the negative weights in our model to positive only (figure 3b), and employing summation near the negative x-axis instead of difference, will preserve the qualitative results (i.e. high response at the negative x-axis) and provide a more implementable model. Finally, to implement the D_arg operator, one has to rotate the Y_arg operator to all orientations. The neuronal implementation of this process will result in a local summation of the outputs of the orientation-dependent cells (figure 3c). This simple neuronal implementation lends support to the idea that D_arg may serve in a biological vision system.

Figure 3.

Neuronal implementation of D_arg. (a) A cortical hypercolumn. Each circle represents a column, with the bar indicating the preferred direction of cells in this column (i.e. the direction of an input line to which they fire most). Local differentiation of the angle function near angles π and −π would yield a high response, similar to Y_arg. (b) The same hypercolumn, with only positive weights (i.e. angles). (c) Neural implementation of D_arg. Local summation of orientation-dependent cells may detect convexity.

4. Experimental results

In this section, we juxtapose the D_arg operator with a typical edge-based operator—the radial symmetry transform (Reisfeld et al. 1995)—as camouflage breakers. This operator seeks generalized symmetry in the edge map of an image around multiple central locations. It evaluates the contribution of edges around the point to symmetry from all sides of the point. The transform has been shown to generalize several edge-based attentional operators (e.g. detectors of high curvature, centre of gravity, corners) (Reisfeld et al. 1995). We compare D_arg with edge-based methods, because they have been suggested to be important in biological camouflage breaking, for example, through prey patterning that causes superexcitation of a predator's edge detectors (Osorio & Srinivasan 1991).

(a) Implementation

The first step in the computation of both D_arg and radial symmetry is the computation of the image gradient. This has been done by convolution with a Gaussian in one direction and with the derivative of a Gaussian on the other. The radii of the Gaussian were two pixels in each direction.

The radial symmetry operator is scale dependent, while the peaks of D_arg are not. Therefore, we have compared D_arg with radial symmetry of radii: 10 and 30 pixels (i.e. two radial symmetry transformations performed for each original image). In this paper, only one radius is introduced per original, but similar results were obtained for the other radius as well.

The gradient argument was computed in a neighbourhood of radius 30 pixels. A threshold of 65 per cent of the maximal value was applied to both D_arg² and radial symmetry maps to isolate regions of interest (marked by ‘plus’ signs in figures 4 and 5).

Figure 4.

Persian fallow deer (Dama dama mesopotamica) lying in a stony environment. (a) Detection by radial symmetry. The tones of the deer blend with the background, making the stones more prominent for edge-based methods. (b) Radial symmetry. (c) Detection by D_arg. D_arg detects the deer, breaking the camouflage. (d) D_arg².

Figure 5.

Nubian ibex (Capra ibex nubiana) in a rocky habitat. (a) Detection by radial symmetry. Edge-based methods fail to detect the ibex due to its apatetic coloration. (b) Radial symmetry. (c) Detection by D_arg. By contrast, D_arg responds to the convexity of the intensity function of the ibex, thus isolating it from the background. (d) D_arg².

(b) Apatetic coloration in animal

Animals use various types of camouflage to hide themselves, one of which is apatetic coloration (also known as background matching). In this type of camouflage, the colour, brightness or pattern of the animal matches one or several background types. Unlike countershading (also known as ‘self-shadow concealment’ when used for concealment), this type of camouflage does not account for the light falling on the animal. Figure 4 exhibits a natural camouflage of a Persian fallow deer (Dama dama mesopotamica) on stony ground. The camouflaged deer has few edges marked on its back, to prevent detection due to an abrupt disappearance of environmental edges. While these edges activate edge-based detectors to a small degree, they are not strong enough to be isolated from the environment. Indeed, the vast majority of the locations detected by radial symmetry concentrate outside the boundaries of the image of the deer. Thus, the deer would probably not be spotted by an edge-based detector. D_arg, however, produces three strong peaks, which match the trunk of the animal, being the most smooth three-dimensional convex region in the image (from a photographic viewpoint).

Figure 5 shows a Nubian ibex (Capra ibex nubiana) on a rocky hillside, under the shades of a tree (not seen in the picture). Owing to the apatetic colouring, the rocky background produces much stronger edges than the ibex, thus attracting edge-based methods. Radial symmetry specifies no single target, and the vast majority of detected locations are away from the subject. This is due to the subject being smooth and surrounded by edges formed by the rocks, which distract the radial symmetry transform from the ibex. D_arg detects the ibex as it appears smooth (from the photographic distance), and is three-dimensional and convex. Of note is that the ibex appears much smaller in figure 5 than the Persian fallow deer in figure 4. D_arg is able to detect both animals despite the difference in their scale using exactly the same settings (i.e. the same radii and thresholds were employed).

(c) Countershading: an effective camouflage against D_arg

Figure 6 demonstrates the effectiveness of countershading as a camouflage against convexity-based detectors such as D_arg. Two images of a caterpillar are juxtaposed: countershaded and non-countershaded. The response of D_arg to the countershaded caterpillar image is far lower than that to the non-countershaded one, as shown in figure 6b. This lends further support to the conclusion that detection of the convexity of the grey-level function may be employed by visual systems of predators to break the camouflage.

Figure 6.

D_arg fails to detect a countershaded caterpillar. Countershaded and non-countershaded caterpillars are present in the upper and lower parts of the image, respectively. (a) Detection by D_arg. Countershading flattens the grey-level function, so D_arg misses the countershaded caterpillar. (b) D_arg². Pay attention to the strong stripe for the non-countershaded caterpillar, and the weaker response to the countershaded caterpillar.

5. Conclusions

Thayer's principle states that various animals use countershading as a major basis for camouflage. The observation of such a countermeasure in animals implies that other animals might use convexity detection to break the camouflage (or otherwise, there was no need for the countermeasure). We therefore suggested an operator for convexity detection, D_arg, that might be employed in the visual system of predators. D_arg is basically intended for the detection of image domains emanating from smooth convex or concave three-dimensional objects, but the smoothness assumption can be relaxed. We speculate that the operator might be employed in biological vision systems because (i) it is highly effective in camouflage breaking, as was demonstrated in a comparison with an edge-based method (radial symmetry), (ii) there appears to be camouflages (i.e. countershading) developed especially against it (e.g. in caterpillars), and (iii) its implementation by a neural network is very simple.