Biased generalization of salient traits drives the evolution of warning signals

Abstract

The importance of receiver biases in shaping the evolution of many signalling systems is widely acknowledged. Here, we show that receiver bias can explain which traits evolve to become warning signals. For warning coloration, a generalization bias for a signalling trait can result from predators learning to discriminate unprofitable from profitable prey. However, because the colour patterns of prey are complex traits with multiple components, it is crucial to understand which of the many aspects of prey appearance evolve into signals. We provide experimental evidence that the more salient differences in prey traits give rise to greater generalization bias, corresponding to stronger selection towards trait exaggeration. Our results are based on experiments with domestic chickens as predators in a Skinner-box-like setting, and imply that the difference in appearance between profitable and unprofitable prey that is most rapidly learnt produces the greatest generalization bias. As a consequence, certain salient traits of unprofitable prey are selected towards exaggeration to even higher salience, driving the evolution of warning coloration. This general idea may also help to explain the evolution of many other striking signalling traits found in nature.

1. Introduction

How warning signals evolve has been much discussed, including the possible importance of receiver biases [1–8]. It is often argued that signals evolve to be easily detectable, rapidly learnt and memorable [3,4,6,9–15], and also that signals could convey ‘honest’ information [6,16]. Around 30 years ago, two general explanations for warning signal evolution were put forward, one being that warning signals serve to increase the speed of discrimination learning [9,10] and the other that generalization bias, sometimes referred to as peak shift, is the driver of warning signal evolution [1,2]. Yet which of these ideas, if any, explains the evolution of warning signals remains unresolved.

Prey appearances are complex traits with multiple components [17,18], but, with few exceptions [19], this has not been taken into account in explanations of warning signal evolution. There has been a lack of specific hypotheses about the psychological and behavioural mechanisms that result in one trait becoming a signal rather than another. Here, we present one such mechanism, which is illustrated in figure 1. Predators learn to discriminate between profitable and unprofitable prey using differences in components of their appearance. As is generally the case for discrimination between complex stimuli (as well as for other kinds of learning), some aspects of stimuli play a greater role than others in the learning process [20,22–26]. Animals pay more attention and learn about salient differences between stimuli and disregard less salient differences. Salience can be defined as noticeability, in the sense of how much an individual's attention is directed towards a stimulus component in a given situation [26]. However, it is not the absolute intensity of a component that defines salience, though intensity can contribute. Instead, it is how discriminable a stimulus component is from the background or in comparison with components of other stimuli (see [26] for a discussion of the concept of salience in learning psychology). In effect, aspects of prey appearances with salient differences between profitable and unprofitable prey gain high associative strength (figure 1a,b). Also, a high-salience discrimination is learnt more rapidly [26]. This aspect of salience—that it corresponds to highly discriminable differences between stimuli—is the focus of our analysis here.

Figure 1.

Illustration of how the salience of stimulus components affects generalization bias. (a) Two stimuli (rewarded S+, unrewarded S−) shown in a two-dimensional space, representing two characteristics or components of compound stimuli. There is a larger difference between S+ and S− in the first component (dimension 1) compared with the second component (dimension 2), corresponding to higher salience. (b) After discrimination learning, the associative strength for the first component will be greater than for the second (the first overshadows the second [19,20]). (c,d) Generalization gradients around S− after discrimination learning. The gradients have Gaussian shapes, in agreement with previous knowledge [21], and represent attack probability as a function of variation around S− along dimensions 1 and 2. Our hypothesis is that the generalization bias and peak shift (arrows: min shifted from S−), as well as the variation in attack probabilities, is larger for the dimension with a higher salience difference between stimuli, which is dimension 1. (e) Two stimulus dimensions: a colour dimension (top), ranging from green to blue, C1–C7, and a greyscale dimension (bottom), from light to dark grey, G1–G7. (f) Examples of pairs of compound stimuli, S+ and S−, used in discrimination learning. In treatment 1 (left), there is a large colour and a small grey difference, and in treatment 2 (right) a small colour and large grey difference.

Our novel idea is that predators show a stronger bias in attack avoidance for components with higher-salience differences when generalizing [21] a learnt discrimination to novel prey appearances (figure 1c,d; see Discussion section on the importance of generalization bias and peak shift for signal evolution). Furthermore, there will be more variation in attack probabilities with stimulus variation for components that have acquired greater associative strength [20,22]. As a consequence, these components or traits will be under selection for exaggeration, and can evolve to become warning signals. An alternative to our hypothesis is that bias should be equally strong, or even stronger, for components with smaller differences between S− and S+ stimuli. The reason for such an alternative is that, when investigating a single stimulus dimension, larger peak shifts for smaller differences between S− and S+ have been found [21].

We tested our hypothesis, using domestic chickens as predators on artificial prey stimuli, in two experiments. In experiment 1, we examined generalization bias around the S− stimulus in figure 1f for two stimulus component dimensions, colour and grey, and we also examined how the respective component differences between S+ and S− influenced attack probabilities. We used a balanced experimental design, where in one treatment there was a large difference in the colour dimension and a small difference in the greyscale dimension between unrewarded and rewarded prey, and another treatment where the relative magnitude of the differences was reversed. With this approach, we can show that a generalization bias is elicited by the salience of the aspect of prey appearance that predators use in discriminating unprofitable from profitable prey, and not by some other property of the particular aspect of appearance (such as contrast against the background). In experiment 2, we then tested the salience of stimulus component differences, by measuring rates of discrimination learning for different stimulus pairs.

2. Material and methods

We used a Skinner-box-like setting for the experiments (electronic supplementary material (ESM), figure S1). We created prey stimuli consisting of two parts, or components (figure 1; ESM, figure S2) that each varied in one dimension. One component varied in a colour dimension from green to blue, and the other in a greyscale dimension from light to dark grey. We measured the reflectance spectra of the components (ESM, figure S3a,b) and used these, together with the irradiance in the experimental box (ESM, figure S3c) and the spectral sensitivities of chicken visual receptors (ESM, figure S3d), to place the colour components in a tetrachromatic visual space (ESM, figures S4 and S5a) and the greyscale components in a lightness space from black to white (ESM, figures S5b and S6). The latter was based on the chicken double cone receptors (ESM, figure S3d).

In order to interpret our results on generalization around the colour and greyscale components of the S−, we measured the distance between test stimulus components and the S− component using a unit derived from estimates of visual receptor noise thresholds for colour and greyscale perception [27–29]. The unit is referred to as ‘just noticeable difference’ (JND), and gives a rough estimate of the magnitude of distance in visual space that an individual can discriminate [28,29]. The unit is defined in terms of parameters, referred to as Weber fractions, according to a geometric analysis in [27]. For the Weber fractions, we obtained estimates for chicken from [28,29]. Our reason for this approach is to use a measure of distance between stimulus components that corresponds to the difference perceived by the chickens. See the ESM for a detailed description of our representation of the visual space of chickens, as well as of the experimental procedures and the handling of the animals.

(a). Experiment 1

Individuals in two treatment groups (n = 6 and n = 5) were subjected to discrimination learning trials, each consisting of 25 rewarded S+ and 25 unrewarded S− stimuli (figure 1; ESM, figure S2), presented sequentially and in random order. In treatment 1, subjects received a discrimination task with a large difference between S+ and S− in the colour dimension and a small difference in the grey dimension, and vice versa in treatment 2 (figure 1; ESM, figure S2). Initially, the birds tended to attack (peck on) all stimuli, so the learning amounted to avoiding attack on unrewarded stimuli. When the birds had learnt the discrimination, to a criterion of reaching avoidance of 16 or more of the 25 S− prey in a trial, they were subjected to generalization probe trials [30]. These trials examined variation around S−, in order to test for bias in the generalization of avoidance of S−, followed by probe trials to examine how the componentwise differences between S+ and S− influenced attack probabilities. For generalization around S−, all birds received the same set of seven non-reinforced probe stimuli (ESM, figure S2), intermixed with the regular discrimination stimuli (above), during 10 trials. The probe stimuli for generalization around S+ differed between treatments 1 and 2 because they had different S+ stimuli (figure 1; ESM, figure S2).

(b). Experiment 2

A new set of birds was used to investigate the salience of each of the component differences, i.e. small or big difference in the grey or colour dimension. A total of 24 hens were divided into four groups (6 per group) with the following discriminations (S+ – S−): C1G4 – C5G4 (big difference in colour), C4G1 – C5G1 (small difference in colour), C4G1 – C4G5 (big difference in grey) and C1G4 – C1G5 (small difference in grey). See figure 1e,f and ESM, figure S2 for illustrations of the pairs of stimuli used. The hens performed 10 trials with 100 stimulus presentations per trial, 50 each of S+ and S−.

(c). Statistical approach

We used the statistical software R version 3.4.2 [31] for the statistical analyses. Data from the first set of probe trials in experiment 1, investigating generalization bias around S−, were analysed by fitting generalized linear mixed-effect models using the glmer function in the lme4 package [32]. The binary response variable was attack (Y/N) on probe stimuli, with the probe stimulus as fixed effect, with levels C4G5, C5G5, C6G5, C7G5, C5G4, C5G6 and C5G7 as described above, and the individual bird as random effect. From the fit of this model to the data from treatments 1 and 2, we obtained the means and 95% confidence intervals shown in figure 2a,b and figure 2d,e. In order to test for biased generalization, we expressed the fixed effect in the model as six orthogonal contrasts, two of which correspond to biased generalization in the colour and greyscale dimensions, respectively. As an example, comparing the attack rates on C4G5 versus C6G5 tests if the generalization along the colour dimension is symmetric around C5G5 (S−).

Figure 2.

Generalization during probe trials in the asymptotic phase of discrimination learning. The top panels refer to treatment 1 (high-salience colour difference between S+ and S−) and the bottom panels to treatment 2 (high-salience grey difference between S+ and S−). The colour and grey dimensions are shown in figure 1e. In all cases, S− is C5G5 (figure 1f). (a,d) Proportion of attacks on probe stimuli varying along the colour dimension around S−. (b,e) Proportion of attacks on probe stimuli varying along the grey dimension around S−. (c,f) Proportion of attacks on probe stimuli deviating in colour or in grey from S+. The points and error bars show mean and 95% confidence intervals of the proportion attacks. Statistical tests for bias in generalization around S− are shown in panels (a) and (b), based on ESM, table S1, and (d) and (e) based on ESM, table S2. The light grey curves are fitted Gaussians that illustrate generalization gradients consistent with attacks on S− and neighbouring stimuli. The statistical tests of effects of variation around S+ in (c), based on ESM, table S3 and (f), based on ESM, table S4, show how components of S− elicit a decrease in attacks compared to attacks on S+. See text for further explanation.

For the data from experiment 2, comparing the rates of discrimination learning for large and small differences in the colour and greyscale dimensions, we fitted generalized linear mixed-effect models using the glmer function. The binomially distributed response variable was the number of times a bird avoided attacking the unrewarded stimulus (S−) out of the 50 presentations in a trial. The discrimination treatment and the centred trial number were the fixed effects, and midtrial intercepts and slopes for the individual birds were random effects. We used orthogonal contrasts of the treatment factor for planned comparisons of midtrial intercepts and slopes of the fitted learning curves. The three contrasts correspond to a comparison of the colour differences versus the greyscale differences, and then a comparison of large versus small difference within each of these two dimensions (table 1).

Table 1.

Planned comparisons for the mixed-effect generalized linear model analysis of the discrimination learning in the salience test illustrated in figure 3. The comparisons are between mid-trial intercepts and slopes of three orthogonal contrasts for the discrimination treatments: colour versus grey, high colour versus low colour difference and high grey versus low grey difference. See ESM, figure S1 for the stimuli used. Values in italics denote statistically significant differences.

comparison	intercept	s.e.	p-value	slope	s.e.	p-value
colour v. grey	0.170	0.200	0.395	0.038	0.045	0.394
high colour v. low colour	1.583	0.284	<0.0001	0.244	0.064	0.0001
high grey v. low grey	0.692	0.283	0.014	0.136	0.063	0.030

Data and R files used for the statistical and visual space analyses are available in the Dryad Digital Repository [33].

3. Results

The results from the probe trials supported our hypothesis. By comparing the proportion of attacks on stimuli on either side of S−, we found biased generalization around S− for the stimulus dimension with a large difference between S+ and S−, which is colour in treatment 1 (figure 2a; ESM, table S1) and grey in treatment 2 (figure 2e; ESM, table S2). For the dimension with small difference, there was no evidence of bias for treatment 1 (figure 2b; ESM, table S2), but a statistical tendency towards a bias in colour generalization in treatment 2 (figure 2d; ESM, table S2). We also compared attacks on S+ with attacks on stimuli with one component the same as S+ and the other the same as in S−. Figure 2c shows that the birds used colour, but not grey, to discriminate S+ from S− in treatment 1, whereas figure 2f indicates that they used grey and to some extent also colour for the discrimination in treatment 2. For birds, colour is a high-salience stimulus, consistent with the JND values in figure 2, which may explain why the small colour difference in treatment 2 gained some associative strength (figure 2f), and perhaps also some generalization bias (figure 2d).

In interpreting these results, we do not place the colour and grey stimulus components (C4 to C7 and G4 to G7) equidistantly along the x-axes in figure 2. Instead we express distances between a component of S− and a corresponding component of a test stimulus in units of JND. The size of this unit for the colour and greyscale dimension is illustrated in figure 2. Because the distances from C5 to C6 and G5 to G6 turned out to be greater than the distance from C5 to C4 and G5 to G4, respectively, our findings of statistically significant biases in generalization around S− are in fact conservative, compared to a situation of equidistantly placed components.

In a separate experiment, with different birds, we verified that the large and small differences in the colour and grey dimensions actually represent high and low discriminability (i.e. high and low salience). This is a prerequisite for our interpretation of the data in figure 2, namely that more salient differences give rise to greater generalization bias. Salience is positively related to the rate of discrimination learning [20,22–26], so to measure salience we performed discrimination learning trials for each component difference separately (ESM, figure S2). We found that the large colour difference was learnt more rapidly than the small colour difference and that the large grey difference was learnt more rapidly than the small grey difference (figure 3 and table 1), in accordance with our interpretation.

Figure 3.

Discrimination learning of large versus small difference between S+ and S− in the colour and the greyscale dimensions. The proportion of S− presentations which did not elicit an attack is shown, with bootstrap 95% confidence intervals. Solid dark line: large colour difference (C1G4 versus C5G4); solid light line: large grey difference (C4G1 versus C4G5); dashed dark line: small colour difference (C4G1 versus C5G1); and dashed light line: small grey difference (C1G4 versus C1G5). The stimuli are illustrated in ESM, figure S1. See table 1 for statistical comparisons between the learning curves.

4. Discussion

Our main result—that trait components with higher salience differences between unprofitable and profitable prey elicit greater generalization biases by predators—can explain how warning signals evolve. Thus, for an unprofitable prey species without warning coloration, predators may still learn to avoid its members, using salient components of its appearance. Generalization bias results in selection for exaggeration of those characteristics, and the process can continue until the species has acquired a distinct warning signal.

The alternative hypothesis, that there should be stronger or equal generalization bias for the component with smaller difference between S− and S+, is ruled out by our experiment (figure 2). When examining a single stimulus dimension, several studies have found larger peak shifts for treatments with smaller differences between S− and S+ [21,34], but there are also studies where the opposite was found [35]. At any rate, it is perhaps not surprising that effects observed when individuals learn to discriminate either small or large differences in a single dimension do not directly translate to a situation with discrimination between complex stimuli, with smaller differences in some components and larger in others.

A distinct question, but potentially related to the salience of differences in stimulus components, is when components differ in reliability [36] (e.g. in how well they predict the consequences of attacking prey). A predictively reliable aspect of a stimulus can, after learning, increase in salience, just as unreliable aspects can decrease. We did not examine that question here; for our discrimination learning, either component separately predicted reward. However, effects of changing salience through learning can sometimes be important (e.g. for mimicry evolution [25]).

Biased generalization is often discussed in terms of peak shift [1,3,5–8,11], where the maximum or minimum responding is shifted away from previously learned stimuli. It should be noted that we found evidence for biased generalization, but not direct evidence for peak shift. For instance, C6 in figure 2a was less attacked than C4, but not less attacked than the S− component C5. Because generalization gradients typically have a Gaussian shape [21], the fitted Gaussian curve in figure 1a indicates that there most likely is a peak shift (the minimum of the fitted curve is shifted away from C5). However, even if the generalization gradient would have a V-shaped minimum at C5, for instance, an exponential shape [21], our conclusion that biased generalization should give rise to natural selection for trait exaggeration still holds. The reason is that in nature, the trait probably has a quantitative genetic background, with additive genetic variation in a population. A realistic way of describing natural selection in such a situation is an approach that takes into account a distribution of trait values [37], for instance, a normal distribution. Corresponding to our case, predators will learn while being exposed to the trait variation of profitable and unprofitable prey. This will smooth out the generalization gradients from each individual predator–prey encounter. A bias from the build-up of positive and negative contributions to the overall gradient will then result in directional natural selection. Note that the bias in generalization around, for instance, C5 in figure 2a is the result of negative and positive contributions from encounters with S− and S+. In order to illustrate this general argument, the outcome of a situation where predators interact with prey populations with variable traits is shown in ESM, figure S7, using a theoretical model of reinforcement learning and either Gaussian or exponential contributions to generalization.

In most groups of animals with warning coloration, the appearance before and during signal evolution is not known. However, the rather small number of cases with such information is consistent with our idea of salience-driven generalization bias. In Delias butterflies areas of red colour on the ventral wing surface has evolved as a novel warning signal [38]. The entire genus has areas of ventral yellow, sometimes shifting towards orange, and predators appear to use this trait as a salient characteristic to discriminate unpalatable Delias butterflies from other palatable pierid butterflies [38]. Generalization bias towards orange/red thus appears as a reasonable driver of this warning signal evolution. The evolution of aposematism in Epipedobates poison frogs [39], as well as in red salamanders [40] and in Norwegian lemmings from an appearance like Siberian brown lemmings [41], also seem to fit this scheme of exaggeration of previously existing characteristics. Thus, aspects of brownish–greenish dorsal colour, disruptive markings or countershading, originally serving as camouflage, could be used by predators to discriminate unprofitable from profitable prey and then evolve to warning signals through salience-driven generalization bias.

Warning coloration often shows strong contrast against the background, with bright colours such as yellow, red and white in combination with black patterns [17]. For this reason, conspicuousness is often put forward as a major cost of warning signals. A loss of camouflage entails an increased risk of detection and attack by predators that have not learnt to avoid the warning signal. Thus, to fully understand the function and evolution of warning signals, it is necessary to understand how the cost of conspicuousness is balanced [15]. However, our results here do not support the idea that it is conspicuousness per se, in the sense of contrast against the background, that is selected for in warning signal evolution. Thus, warning signals ought to be salient, but not specifically contrasting against the background. Conspicuousness can be one way to increase salience, but it can also be a by-product of selection for salience in some other way. Our experiment here is an example where salience depends on a greater difference from palatable prey, and not on contrast against the background.

Our explanation shows some similarity to the idea that aposematic prey have been selected ‘in favor of looking as different as possible from those camouflaged forms for which predators are constantly hunting’ [42], for which there is experimental support [43]. An important distinction between that idea and our approach here is that we explicitly consider prey appearances as multidimensional, and we identify the perceived salience of trait differences as the crucial aspect deciding which traits evolve to become signals, which has not been done before.

Both biased generalization [1,2,5] and faster avoidance learning [9,10] have been put forward as crucial explanations for warning signal evolution. Our results show that these explanations represent two sides of the same coin, in the sense that high-salience trait differences are both learnt more quickly and give rise to stronger bias in generalization. The psychological phenomenon of salience-driven generalization bias has not been described previously, but might well be both widespread and important for signal evolution. An example where it has been found, but not identified as a general phenomenon of salience-driven bias, is a study where bumblebees showed greater generalization bias when flower colour hue was the only stimulus component to use in discrimination, compared with when both hue and a scent could be used [44].

The evolution of sexual ornaments through bias from sexual imprinting [8,11], as well as floral signal evolution guided by generalization bias of learnt preferences by pollinators [7,44], are examples of signalling systems where salience-driven bias could be a crucial mechanism that determines which characteristics evolve to become signals. More generally, it has been argued that most animal signals evolve through ritualization [45,46], starting from informative cues that are not yet signals. For instance, intention movements of fighting can act as cues that an opponent will initiate an attack. If responses to such cues are at least partly learnt, salience-driven generalization bias can then promote the evolution of threat signals. It is thus reasonable to think that salience-driven bias has played an important role for a wide range of biological signals.

Ethics

The experiments were carried out with ethics permission from Linköpings djurförsöksetiska nämnd (dnr 66–14).

Data accessibility

Data and R code are available from the Dryad Digital Repository: http://dx.doi.org/10.5061/dryad.b5151 [33].

Authors' contributions

G.G.-S. and O.L. designed the experiment and wrote the paper, with input from B.K. and A.B. B.K. and A.B. collected the data. O.L. performed the statistical analysis.

Competing interests

The authors declare no competing financial interest.

Funding

This work was supported by a grant from the Swedish Research Council (621-2010-5437) to O.L.