Predator perception and the interrelation between different forms of protective coloration

Abstract

Animals possess a range of defensive markings to reduce the risk of predation, including warning colours, camouflage, eyespots and mimicry. These different strategies are frequently considered independently, and with little regard towards predator vision, even though they may be linked in various ways and can be fully understood only in terms of predator perception. For example, camouflage and warning coloration need not be mutually exclusive, and may frequently exploit similar features of visual perception. This paper outlines how different forms of protective markings can be understood from predator perception and illustrates how this is fundamental in determining the mechanisms underlying, and the interrelation between, different strategies. Suggestions are made for future work, and potential mechanisms discussed in relation to various forms of defensive coloration, including disruptive coloration, eyespots, dazzle markings, motion camouflage, aposematism and mimicry.

1. Introduction

Many animals possess protective coloration to reduce the risk of predator detection (camouflage), warn predators of the prey's unpalatability (aposematism) or fool a predator into mistaking the prey for something else (e.g. mimicry, masquerade). This paper investigates the interrelation between different forms of protective markings, and argues that an understanding of these strategies should be grounded in knowledge of how predator visual perception works (Hailman 1977; Guilford & Dawkins 1991; Stevens & Cuthill 2006), which can generate and test new predictions regarding how different forms of protective coloration function. Many different mechanisms underlying protective signals exploit related features of visual processing, and knowledge of this can help us understand how signals function. Effective conspicuousness, for example, can often be thought of as using opposing principles as those involved in concealment (figure 1). Various techniques have been developed to study the evolution of prey defensive coloration, including signal detection theory (cf. Pie 2005), and the related technique of Bayesian statistical decision theory, which can investigate the coevolution of predator perception and prey coloration and generate predictions about visual and information processing (e.g. Geisler & Diehl 2002, 2003). There is also great potential to use artificial neural networks in studies of perception (in particular, see Gurney 2007; Phelps 2007). However, the above methods rarely use explicit features of visual perception to understand different forms of protective markings. This is frequently important since signals may have a range of different, not necessarily mutually exclusive, functions, which can only be understood with an explicit knowledge of how the signal is processed. For example, the zigzag markings on some vipers are thought to function primarily as warning colours (Wüster et al. 2004; Niskanen & Mappes 2005), but other functions such as disruptive coloration, distance-dependent effects and flicker-fusion camouflage remain to be discounted (mentioned later). The frequent similarities between different species' visual systems allow models derived from computational neuroscience and visual perception to illustrate the mechanisms of how protective coloration functions (Rolls & Deco 2002; Stevens & Cuthill 2006), and how different strategies may be linked. This paper outlines, with examples, how various forms of protective coloration can be understood in terms of perception, and how alternative strategies exploit different, or similar, aspects of visual processing. Considering predator perception can help understand the form, function and evolution of different protective strategies.

Figure 1.

An example of how different forms of protective coloration use various aspects of visual perception, and how conspicuousness and concealment often exploit opposite principles in terms of visual processing. Note that the forms of coloration outlined are not necessarily mutually exclusive and may coexist on an animal (e.g. aposematism and disruptive coloration).

2. Preventing detection and recognition

(a) Crypsis and disruption

Many animals avoid visually hunting predators by concealing themselves, and the survival benefits of camouflage provided some of the earliest and most convincing examples of Darwinism. Simplistically, camouflage involves possessing markings that reduce the chance of detection, but this may be achieved in various ways (and strategies like masquerade prevent recognition rather than detection). Conventionally, camouflage is usually measured by how well an animal represents a random sample of the background, where the predation risk is highest (crypsis; Endler 1978, 1984). However, in heterogeneous environments, optimal concealment may involve a compromise in markings towards camouflage on all backgrounds, despite not specializing on any background fully (Thayer 1909; Merilaita et al. 1999, 2001; Ruxton et al. 2004a; Houston et al. in press). In addition, countershading, where animals are darker dorsally than ventrally, may eliminate an animal's three-dimensional form and cancel out the effects of shadows created on the underside of the body (Poulton 1890; Thayer 1896). While other explanations may account for countershading (Ruxton et al. 2004b), recent evidence of a concealing function has been shown in experiments with artificial prey and avian predators (Rowland et al. in press). Different light environments may promote various levels of countershading, whereby it may be more prevalent where light intensity is higher and scattering is lower (Hailman 1977). Evidence from comparative studies provides some support, since countershading appears more common in mammals inhabiting desert environments (Stoner et al. 2003). Additionally, Thayer (1909) argued that there are two forms of camouflage, including blending into the background (e.g. crypsis) and disruptive coloration, where the animal's appearance is broken up by strongly contrasting patterns that destroy the body outline (reviewed by Stevens et al. 2006a), which is crucial in preventing detection (Thayer 1909; Cott 1940). Disruptive theory predicts that markings (components matching elements of the background patterning) on an animal, used to break up the body outline, may be statistically more likely to be located peripherally than would be expected if the pattern distribution matched that found in the background (Stevens et al. 2006b). Disruptive theory has received significant recent support (Merilaita 1998; Cuthill et al. 2005, 2006; Merilaita & Lind 2005, 2006; Schaefer & Stobbe 2006; Stevens & Cuthill 2006; Stevens et al. 2006b; Fraser et al. 2007).

Mechanistically, disruptive camouflage works because the markings exploit edge detection mechanisms functioning in early visual processing at different spatial scales (figure 1; Osorio & Srinivasan 1991; Stevens & Cuthill 2006). Vertebrate visual systems contain receptive fields sensitive to lines and gratings, and process scenes by breaking down visual information into different spatial scales, while line detection mechanisms encode edge information via sharp changes in light intensity (Hubel & Wiesel 1962; De Valois & De Valois 1980; Shapley & Lennie 1985; Graham 1989; Bruce et al. 2003). This has a crucial role in object–background segmentation because changes in light intensity and composition frequently occur where one object ends and another begins (Bruce et al. 2003). Stevens & Cuthill (2006) used a model of vision which located the edges in images calibrated to avian cone sensitivities and potential colour pathways, containing artificial prey either cryptically or disruptively marked. The model successfully found more edges corresponding to the outline of the cryptic prey than it did for the disruptively marked targets, especially when the disruptive markings were highly contrasting. This verified a model of Osorio & Srinivasan (1991) who found that enhanced edge profiles were effective in preventing recognition of frogs by edge detection algorithms in garter snake, Thamnophis sirtalis, vision. These models indicate how disruptive coloration works in terms of predator perception (Endler 2006). Research into disruptive coloration in real animals, however, is lacking, and only one rigorous test has investigated the distribution of potentially disruptive markings on an animal (Merilaita 1998), and this did not explicitly measure the distribution of markings in the background (Stevens et al. 2006a). Various experiments have investigated the expression of potentially disruptive markings in animals, particularly in cuttlefish (e.g. Hanlon & Messenger 1988; Kelman et al. 2007). However, only one study to date has tested the potential survival benefits of disruptive coloration in a real animal (Silberglied et al. 1980), with respect to the apparently disruptive wing stripe in the butterfly Anartia fatima, but this failed to find any survival benefit. However, there is little evidence that the stripe is disruptive, and the experimental controls may have made the butterflies more similar to a co-occurring unpalatable species (Stevens et al. 2006a). Therefore, more experiments testing for the presence and survival value of disruptive markings in real animals are greatly needed.

(b) Dazzle

Related to disruptive coloration is Thayer's (1909) theory of ‘dazzle’ markings, which are thought to act like distractors, drawing the eye away from the outline of the body possessing them, preventing recognition. While dazzle and disruptive coloration are similar, and both involve high-contrast patterns, they are probably logically distinct. Disruptive colour patterns may work best when matching the background (Stevens et al. 2006b; Fraser et al. 2007), but dazzle markings may be optimal when they do not match the background. If dazzle markings work, there are two ways in which they may function. First, dazzle markings were effective in preventing estimates of the speed and trajectory of painted ships in war times by targeters (Behrens 1999), and so they may be found on active animals, as predicted by Thayer (1909). Such markings may function on some snakes, where the patterns make it difficult for predators to anticipate where to successfully direct a strike (see §3f below). Second, dazzle markings may also work on stationary prey, by producing a ‘crowding’ effect, whereby the perception of a visual stimulus (the prey) is affected by the presence of other non-overlapping stimuli (distractors; Chung et al. 2001). This is probably linked with contour interaction, and probably stems from interference of adjacent retinal receptors due to lateral inhibition from within a single detector, or from inhibitory influences from more distant neurons, where crowding occurs due to neuronal overlap between the target and the distractors (Polat & Sagi 1993; Wertheim et al. 2006). As such, dazzle markings may function on a purely physiological basis, without invoking attentional mechanisms. Interestingly, a crowding effect may be increased when the distractors have greater contrast than the target (Chung et al. 2001) as expected with dazzle coloration. While motion may also be involved, this form of dazzle markings may explain the function of zebra patterns, especially at low illumination levels (Ruxton 2002). The potential presence, function and mechanistic basis of dazzle markings deserve systematic investigation.

3. Conspicuousness and concealment

(a) Aposematism and signal composition

Aposematic animals advertise unpalatability or toxicity with bright contrasting signals or structural adaptations, so that predators avoid attacking them. There are a number of excellent and extensive accounts of such topics, particularly with respect to the initial evolution of warning coloration, which is not specifically discussed here (Edmunds 1974; Ruxton et al. 2004a; Mappes et al. 2005). However, the evolutionary form of aposematic signals has been neglected in terms of receiver psychology. Essentially, many of the principles of concealment may be reversed to give rise to effective conspicuousness (Hailman 1977). Aposematic markings may be highly detectable against a foliage background (Poulton 1890; Cott 1940), and the brightly coloured repeating patterns may evolve merely because this is very different from concealed, undefended species (Wallace 1889; Fisher 1930; Sherratt & Beatty 2003). Additionally, conspicuousness levels may be linked to the degree of toxicity. For example, Summers & Clough (2001) used human observers and colour photographs (although cameras generally need calibrating to obtain accurate data; Stevens et al. 2007) to show that there was a correlation between increased toxicity levels and brighter warning colours in Dendrobatid frogs (see also Speed & Ruxton 2007). Highly toxic animals may therefore have colours which increasingly diverge from the background attributes. In contrast, other work by Darst et al. (2006) found an opposite association in three Dendrobatid species, whereby brighter species had lower toxicity. Further work is needed to investigate this relationship in various aposematic groups.

Most studies of aposematism focus on the chromatic component of the signal, despite growing evidence that warning colours can be effective without chromatic information (Ham et al. 2006; Prudic et al. 2007). Furthermore, in many potentially conspicuous markings, the achromatic contrast may constitute a very high component of the signal. Lightness, or perceived intensity, arises from photoreceptor neural outputs, the long wave and medium wave cones in humans or single receptors in many other animals, and is also influenced by local contrast values (Kelber et al. 2003). In birds, the double cones appear to function in achromatic vision, and in tasks such as textural discriminations (Osorio et al. 1999b,c; Jones & Osorio 2004). This does not preclude a role for colour information, as chromatic and achromatic vision may be used in different ways; discrimination of large targets apparently uses chromatic information, whereas discrimination of small targets and texture requires achromatic contrast (Mullen 1985; Osorio et al. 1999b; Spaethe et al. 2001). Furthermore, evidence from domestic chicks, Gallus gallus, demonstrates that colour associations are more accurately learned and memorized than achromatic associations (Osorio et al. 1999a). Since warning colours are frequently learned, the chromatic component of warning signals may often be more important than the achromatic element. More information regarding the relative importance of colour and luminance in tasks such as prey detection and recognition are needed. It is also important to consider the visual environment, since while achromatic signals may be highly contrasting, colour signals may be particularly effective in heterogeneous environments, as chromatic signals are relatively resistant to the effects of shadows (Osorio & Vorobyev 2005).

Evidence indicates that markings highlighting the body outline and structures should facilitate predatory detection (figure 1; Cuthill et al. 2005; Endler 2006; Stevens & Cuthill 2006), and this may be further enhanced when some markings mismatch the background (Stevens et al. 2006b; Fraser et al. 2007). However, while enhanced contrast between adjacent patterns may enhance signal's conspicuousness, they may also generate a disruptive effect (Schaefer & Stobbe 2006; Stevens et al. 2006b). For conspicuousness, shape advertisement should therefore involve uniform patches running around the body edge (Hailman 1977). Therefore, there may be a fine line between edge disruption and enhancement, relating to the concentration of markings at the body margins (figure 1). Conspicuousness may also relate to body shape itself. Depth perception may make the detection of a solid three-dimensional animal easier than a flat body form, and so one may expect animals with flat body parts (e.g. wings) to be boarder enhanced (to have highlighted regions at the body edge), whereas since a solid object has ambiguous boarders, detection of the latter may favour uniform coloration (Hailman 1977). Furthermore, the principles of countershading may be reversed to enhance conspicuousness, whereby a lighter dorsal surface relative the ventrum should exaggerate the effects of illumination from above (Hailman 1977). The background form is also likely to play a key role in the efficacy of a given signal. For example, on regularly patterned backgrounds, conspicuousness may be achieved by either uniform patches of colour or barred patterns of the opposite orientation to the background markings (Hailman 1977). High-contrast backgrounds may also hinder detection since the predator's eyes are unable to accommodate rapidly to changes in brightness (Endler 1978). Bright colours may be favoured in aposematic signals in this type of environment, especially if this reduces the problems of shadows masking the achromatic signal (Osorio & Vorobyev 2005).

(b) Distance-dependent effects

Colour patterns of some animals may be distance dependent; conspicuous in close proximity and camouflaged from a distance (Edmunds 1974; Hailman 1977; Guilford & Dawkins 1991). Recently, in experiments with human observers and images of swallowtail butterfly caterpillars Papilio machaon, Tullberg et al. (2005) showed that the larvae did not maximize conspicuousness at a distance, but rather combined it with a level of camouflage. This experiment is a useful application of the theory; however, a true confirmation of such an effect needs to demonstrate similar results to the appropriate (non-human) predatory visual system. This is achievable with the data on an animal's spatial acuity, combined with probable viewing distances, the size and values of the colour patches, and the background properties. The minimum level of contrast that individuals can detect within gratings of different spatial frequencies gives rise to a contrast sensitivity function (CSF). The use of CSFs is illuminating since they show the interrelation between threshold contrast and spatial frequency resolution (Snowden et al. 2006). While many predators, specifically birds of prey, have a very good ability to resolve high-contrast small patterns (visual acuity), recent evidence indicates that birds may have lower-contrast sensitivities than humans and other mammals when a range of contrasts and spatial frequencies are considered (Ghim & Hodos 2006). As such, some birds may be less able to detect low-contrast patterns on their prey from a distance than other animals (Hailman 1977). Since contrast sensitivity optima differ between species, this may also allow a pattern to be conspicuous to one animal group, yet concealed to another at the same distance (Hailman 1977). Data on contrast sensitivities can be used to predict at what distance different patterns should become unresolvable, and therefore potentially concealed.

(c) Imperfect mimicry and camouflage

The presence of so-called ‘imperfect mimics’, which to human eyes do not closely resemble their hypothesized models, is still a contentious area, and various explanations exist to explain such species, including differences between human and predator vision, that mimics may be matching multiple models simultaneously, that mimicry need not be perfect if the model is highly toxic and that mimicry may be costly (reviewed by Gilbert 2005). Imperfect mimicry to humans could be a by-product of differences in perception between humans and other predators (such as in birds, which unlike humans can see ultraviolet light; Cuthill & Bennett 1993). However, as Gilbert (2005) points out, many hoverfly mimics and their models reflect minimal ultraviolet light, and so this would seem unlikely. Additionally, it is not apparent that a Batesian mimic should always rely completely on its mimicry for defence. Genuine aposematic animals have secondary defences to rely on if a predator attacks, whereas Batesian mimics have no such secondary defences, and so their primary protection may be to prevent detection in the first place. Only when detected is mimicry needed to prevent recognition as an edible species. Therefore, imperfect mimicry may be an adaptation to combine camouflage with some level of Batesian mimicry should concealment fail. Furthermore, it is usually assumed that an initial mutation towards mimicry will greatly increase conspicuousness. However, this assumption may be incorrect if Batesian mimics can retain some level of camouflage, reducing the costs of becoming mimetic. This need not necessarily involve a disruptive effect, which has been suggested with respect to aposematism (e.g. Gamberale-Stille 2001). Aposematic animals should therefore be easier to detect than their mimics from a greater distance. Conversely, given that many Batesian mimics may be smaller than their models, it is possible that a mimic may be constrained to be a relatively poor match in close proximity, in order to enhance mimicry from a distance.

(d) Wing/fin spots

The bodies of many insects and other animals bear a range of circular, highly contrasting features, often termed ‘eyespots’. These may intimidate or startle predators, preventing or halting an attack, and deflect predator strikes to non-vital body regions (reviewed by Stevens 2005). While evidence in favour of deflective spots is poor, there is good evidence that some butterfly wing spots can be startling, notably those of the peacock butterfly, Inachis io (Vallin et al. 2005). Recently, experiments have also demonstrated an intimidatory function of permanently visible spots (Stevens et al. in press). Historically, wing spots (and fin-spots in fish) have been thought to mimic the eyes of the predator's own enemies. This is in contrast to the proposal that eyespots work merely because they are highly salient signals towards predatory visual systems, and are highly effective in promoting dietary conservatism and neophobia (Stevens 2005). This hypothesis is grounded in visual perception, since many vertebrate retinae possess circular surround receptive fields with excitatory and inhibitory regions, sensitive to high-contrast patterns such as concentric spots, and provide information on the contrast in a visual stimulus (figure 1; Lythgoe 1979; Graham 1989; Bruce et al. 2003).

Despite a lack of evidence in favour of eye mimicry, results such as those of Vallin et al. (2005) are still being attributed to the effect of wing spots imitating a pair of large eyes. However, the only experiments designed to distinguish between the eye-mimicry and conspicuous signal hypotheses have shown that the only reliable feature in predicting the effectiveness of wing spots is the effect of local contrast, as proposed by the latter theory (Stevens et al. in press). Changing the level of eye mimicry, while holding contrast levels constant, had no influence on the effectiveness of the spots in preventing predation from birds (Stevens et al. in press). Future work should further investigate these theories, particularly with respect to eyespots invoked in startle displays, and with regards to the environmental conditions, as comparative studies indicate that simple body spots may also be linked with concealment in young animals in forest environments (Stoner et al. 2003).

4. Movement and protective coloration

Movement is one of the most important cues used in detection (Hailman 1977). It is well known in visual psychology that some spatio-chromatic arrangements create visual illusions, particularly with respect to motion, whereby either stationary stimuli appear to have moving parts, often associated with observer eye movement, or moving stimuli appear to have motion at the wrong speed or in the wrong direction (see examples in Snowden et al. 2006). Similar effects may work in nature (Forsman & Appelqvist 1998). As such, it is often important to consider the effects of prey behaviour on the effectiveness of a defensive strategy. Movement detectors in visual systems encode information about the direction and velocity of movement (Bruce et al. 2003; Snowden et al. 2006). Linked to motion detectors is the ‘movement after-effect’, whereby after staring at a moving object, motion appears to occur in the opposite direction owing to a temporary desensitization of direction-specific motion detectors (Bruce et al. 2003; Snowden et al. 2006). This can also produce the perception of movement slower than that which caused it (Bruce et al. 2003). It is possible to produce models of motion detectors, based on known properties of visual systems, which often combine two non-directional filters to form a directional one (e.g. Adelson & Bergen 1985; Bruce et al. 2003).

(a) Flicker-fusion camouflage

Some animals, particularly those covered with banded patterns, may be aposematic when motionless but cryptic when moving if they move faster than a predator's ‘flicker-fusion frequency’ or temporal acuity (Pough 1976; Endler 1978; Ruxton et al. 2004a). Two different hypotheses relate to the flicker-fusion effect. First, if a pattern moves across the visual field at a rate faster than the temporal acuity, the markings may blur into a monochrome appearance which may match the general background, conferring some level of camouflage. Second, patterns, particularly zigzag markings often found on snakes, may make it difficult for a predator to determine whether the animal is moving and how fast, similar to the idea of dazzle markings (see above; Jackson et al. 1976; Hailman 1977; Shine & Madsen 1994; Lindell & Forsman 1996). Pough (1976) found in the northern water snake, Natrix sipedon, that fast escape behaviour caused a blurring of the snakes' banding pattern, where the average colour generally matched some backgrounds. Since banding is generally narrower on small snakes, the effect may be greater in younger individuals (Pough 1976). Additionally, since visual temporal acuity is reduced when light levels are low (Jarvis et al. 2002), flicker-fusion effects may be more effective in environments with lower ambient light levels. Shine & Madsen (1994) argue that the flicker-fusion hypothesis is consistent with observations that bright colour patterns in snakes such as adders (Vipera) are mainly shown by active adult males during the mating season, supported by a mark-recapture study of the adder Vipera berus, where male morphs with zigzag patterns survived better than melanistic forms, whereas the situation was reversed in females (Lindell & Forsman 1996). However, several other factors could have produced such a result, and evidence that the zigzag patterns have a flicker-fusion effect on the predators' vision is lacking. As such, there is still no strong experimental support for the theory (Ruxton et al. 2004a). Tests should combine experimentation with knowledge of visual perception in predators. Calculations of whether an animal's patterns match the background when in motion would measure the background properties, the values of the colour pattern and the ambient light levels, and combine this with estimates of the predator's temporal acuity and the prey's movement speed. Demonstrating a flicker-fusion effect to humans is inadequate because temporal acuity varies across animals, and many species have higher acuities than we do. For example, birds, which are often among the main predators of snakes, may have a higher temporal acuity than humans (e.g. Jarvis et al. 2002). Finally, the speed of the animal's movement should correlate with the pattern spatial frequency, where a slower animal would need a higher spatial frequency for the effect to work, for which there is some evidence (Jackson et al. 1976). Again, the principles above can be reversed to consider how movement can lead to conspicuousness, whereby certain forms of motion may be highly visible, especially if they are quick enough to attract attention, yet not too fast to exceed the predator's critical fusion (Hailman 1977).

5. Conclusions

Many studies of protective coloration fail to consider the markings as perceived by the predator. This is crucial if we are to understand why a particular strategy is effective. There is a need for more studies of protective coloration in a range of environments, since the majority of work is done in standardized systems, where the ambient conditions are unnatural. Additionally, more research of adaptive coloration in real animals needs to be grounded in an understanding of natural history. Comparative studies such as those of Stoner et al. (2003) can offer insights into what forms of protective coloration are most effective under different conditions. Understanding the complexities of protective coloration is a far more attainable task today, thanks in part to a better understanding of visual perception and advances in methodologies for studying complex two-dimensional signals and their functions (Endler & Meilke 2005; Osorio & Vorobyev 2005; Stevens et al. 2007). Following Marr (1982), many models have aimed to understand visual perception in terms of algorithms, or processing rules which are implemented at various stages of vision (Bruce et al. 2003). This provides great potential to understand how protective coloration works in terms of predator vision because the algorithms are essentially mathematical forms, allowing computational models to be developed. One should not forget that there may be input from ‘top-down’ mechanisms in perception, but this approach can still be used to test theories of how different forms of coloration work (Stevens & Cuthill 2006). A greater understanding of how different forms of protective coloration function, and how they may be linked, will surely be strengthened by considering the visual and cognitive abilities of the animals concerned.