Biological auctions with multiple rewards

Abstract

The competition for resources among cells, individuals or species is a fundamental characteristic of evolution. Biological all-pay auctions have been used to model situations where multiple individuals compete for a single resource. However, in many situations multiple resources with various values exist and single reward auctions are not applicable. We generalize the model to multiple rewards and study the evolution of strategies. In biological all-pay auctions the bid of an individual corresponds to its strategy and is equivalent to its payment in the auction. The decreasingly ordered rewards are distributed according to the decreasingly ordered bids of the participating individuals. The reproductive success of an individual is proportional to its fitness given by the sum of the rewards won minus its payments. Hence, successful bidding strategies spread in the population. We find that the results for the multiple reward case are very different from the single reward case. While the mixed strategy equilibrium in the single reward case with more than two players consists of mostly low-bidding individuals, we show that the equilibrium can convert to many high-bidding individuals and a few low-bidding individuals in the multiple reward case. Some reward values lead to a specialization among the individuals where one subpopulation competes for the rewards and the other subpopulation largely avoids costly competitions. Whether the mixed strategy equilibrium is an evolutionarily stable strategy (ESS) depends on the specific values of the rewards.

1. Introduction

In the struggle for survival, most organisms need to compete for resources. In these competitions, individuals incur costs corresponding to the energy invested or the risk of a severe injury during the fight. If the individual wins the competition, it obtains a reward, for example, gaining access to a territory. The pay-off of the winning individual is the value of the reward minus the costs incurred during the competition. However, if many individuals compete for multiple resources, even the winners of the resources might incur a negative pay-off as the value of the resource won could be smaller than the total value of the invested time or energy throughout the competition. We can ask if there exists an evolutionarily stable strategy (ESS) in such competitions.

The war of attrition was one of the first models to study and predict animal behaviour [1,2]. In this game, two individuals compete for a resource. The strategy of the conflicting animals can correspond to the amount of time each animal is willing to stay in the fight. The individual that choses to fight longer wins the resource. Both individuals incur the same costs (proportional to the time in the fight) as the fight ends as soon as one of the individuals quits. Hence, the costs are equivalent to the strategy value of the loser. In a case study, the model has been used to predict the contest length of two male dung flies competing for a single female [3]. Over the years many different generalizations [4–6] have been proposed and used to model struggling firms [7] and forming coalitions in politics [8].

Many of these models have the structure of all-pay auctions [9–11]. The simplest examples are scotch auctions where two players compete for a single reward. The only difference to the war of attrition is that the costs of each player are identical to their own strategy [9]. Therefore, scotch auctions model scenarios where the investments always equal the costs (e.g. courtship feeding). Chatterjee et al. generalized scotch auctions to biological all-pay auctions as well as the war of attrition to biological second price all-pay auctions [12]. In contrast to classical auctions, where only the winning bidder has to pay, in all-pay auctions all participants have to pay [13,14]. In biological all-pay auctions, the payment of an individual is equivalent to its bid; in biological second price all-pay auctions all individuals other than the winner pay their own bid, and the winner pays the second-highest bid [12]. Biological auctions provide a natural generalization from 2 to n player models [10,12].

Biological all-pay auctions have been used to model the growth competition among plants for sunlight [9] (see [12] for more examples). Suppose that plants can decide (or during the course of evolution it has been decided) how much energy they should invest into growth. These investments are equivalent to a bidding strategy in the competition for sunlight and directly translate into their costs (payments). Clearly, the largest plant will get most (all) of the sunlight and will shade the other plants. According to this hypothetical model, the winner (i.e. the largest plant) takes all the sunlight and all the other plants are left with nothing, although they might have invested a lot of energy. However, in a more realistic scenario (e.g. plants in a forest), there is not a single winner but there are multiple winners; some of those may receive more sunlight than others and the remaining plants may not receive any direct sunlight.

Thus, a major limitation of the theoretical models of biological all-pay auctions is that they are only applicable in competitions for a single resource. In many situations (e.g. growth competitions, mating fights, courtship feeding [15,16]), there is not a single resource but multiple resources (maybe some less valuable than others) which could affect the bidding (fighting) strategy of the individuals. For example, consider three males courting for two females, where one of the females is preferred over the other (figure 1). Each of the males offers a nuptial gift. The preferred female can pick the male with the highest-valued nuptial gift and the second female can pick the male with the second-highest-valued nuptial gift. The male with the least valued gift will end up without a female despite his investments to offer a nuptial gift. Such a scenario cannot be modelled with biological all-pay auctions with a single resource.

Figure 1.

Courtship feeding of cedar waxwing modelled by biological auctions. (a) Male cedar waxwing provides a berry to the female during courting. Courtship feeding is common in many species [15,16]. Photo taken by Minette Layne (CC). (b) Three males are courting for two females. Green male offers a nuptial gift of value 0.7, orange male of 0.9 and brown male of 0.3. The left female (v₁ = 1) picks the orange male and the right female (v₂ = 0.6) picks the green male. Such a scenario can be modelled by biological auctions with multiple rewards. The pay-offs of the males are the rewards won minus their investments (here the value of the female minus the value of the nuptial gift). The green male obtains a negative pay-off of −0.1, the orange male obtains a pay-off of 0.1, and the brown male ends up without a reward and a pay-off of −0.3.

While auctions with multiple resources are well studied in classical auction theory [8,14] and also highly relevant in sponsored search auctions on Google and Facebook [17], perhaps surprisingly, they have not receive much attention in the evolutionary context. Haigh & Cannings [6] proposed some generalizations for the war of attrition where the number of competitors equals the number of resources. But so far, these models have not been studied in detail. In this work, we generalize the models of biological all-pay auctions to allow for multiple resources (thus also multiple winners) in the same competition. Repeating a single reward auction models a different type of competition as the same individual could win all of the rewards, whereas in a multiple reward auction each individual wins at most one reward. We investigate whether and how results for multiple and maybe differently valued resources differ from the classical results for single resource models.

We study these questions within the framework of evolutionary game theory [18–21]. We aim to identify evolutionarily stable strategies [1,22,23], which correspond to a bidding recipe in our generalized model of biological all-pay auctions. Once an ESS is adopted by a population, no mutant using another strategy can invade the local population by starting as a small fraction of the population. This stability condition of strategies and various other conditions have been studied in many different evolutionary games [18–21,23–31]. For a better understanding of our model, we also explore the stochastic evolutionary game dynamics in finite-sized populations [32–37]. In this work, we study both infinite populations (for analytical results) and finite populations (for computer simulation results).

We find that multiple rewards lead to qualitatively very different ESSs. While in the single reward case, the mixed strategy equilibrium either has a uniform shape or a negative exponential shape, in the multiple reward case the mixed strategy equilibrium can also be an increasing or a U-shaped function. The U-shaped mixed equilibria intuitively suggest the prevalence of two distinct types of bidding strategies in the population, namely, low- and high-bidding strategies. This specialization of subpopulations is unique to the multiple reward case and cannot be found in biological auctions with a single reward.

2. Results

In the model of biological all-pay auctions with multiple rewards, in each auction n individuals compete for m rewards with the values (where for and m ≤ n; figure 1b). Each individual has a fixed strategy value s_i () which corresponds to its bid in an auction. The highest bidder (i.e. the individual with the highest strategy value) in an auction receives the highest reward v₁, the second-highest bidder receives v₂ and so on. In general, the j-highest bidder receives reward v_j. In the case of k ≥ 2 individuals with the j-highest bid, it is decided randomly who receives v_j, v_j+1, … ,v_j+k−1. In each auction, a participating individual incurs costs equivalent to its bid (strategy value). The pay-off of an individual is the sum of the rewards won minus its costs (strategy value times the number of joined auctions) in a particular generation (figure 1b).

(a). The two strategy case

First we study scenarios with two distinct strategies s₁ and s₂ (s₁ > s₂). We assume an infinite population size. The individuals compete for two rewards with value v₁ and v₂. We denote by x the frequency of individuals with the smaller strategy value s₂ (we refer to them as s₂ strategists). Hence, 1 − x denotes the frequency of the s₁ strategists.

Lemma 2.1. —

The expected pay-off for the s₁ strategists is

The expected pay-off for the s₂ strategists is

In an evolutionary process with low mutation rates, we can assume that most of the time the population is dominated by a single strategy. Hence, if a mutant is generated, the mutant strategy either goes extinct or takes over the whole population before the next mutant is generated. We therefore consider the class of strategies s′ with the potential to invade and take over a population which is dominated by the strategy s (Theorem 1; proofs are given in the electronic supplementary material).

Theorem 2.2. —

In biological all-pay auctions with two rewards (v₁ ≥ v₂), a population of s strategists is invadable by s′ strategists if and only if

In figure 2, we show invasion plots for n = 3, 5 and 10. We observe that a mutant with strategy s + δ (δ → 0) can always invade a population since v₁ ≥ v₂ implies that (for n > 2). This observation implies that in biological all-pay auctions with two rewards a pure ESS cannot exist; if an ESS exists, it must be a mixed one. Also we see that the regions where a new strategy can invade a population dominated by another strategy becomes smaller for larger values of v₂ (figure 2). In contrast, for larger values of n, the invasion regions become larger; for , each strategy becomes invadable [12]. Note that we ignore strategies strictly greater than v₁ since their pay-off is always negative, independent of the outcome of the auction. A strategy strictly greater than v₁ implies that skipping the auction ensures a higher pay-off (zero) than participating in the auction.

Figure 2.

(a–c) Invasion plots for biological all-pay auctions with two rewards. The dark-shaded regions correspond to the regions where a mutant with such a strategy can invade (derived from Theorem 1). Whether a mutant can invade in a light-shaded region depends on the value of the second reward v₂. Note that the invasion regions become identical to the invasion regions in biological auctions with one reward if v₂ = 0. A larger value of v₂ decreases the size of the invasion region; a higher number of auction participants n increases the size of the invasion region. Value of highest reward v₁ = 1.

We note that in the case of n = 2 and two rewards, our model simplifies to the biological auction model with one reward [12] as both individuals would win at least v₂ and only compete for the remaining value of v₁ − v₂ [6]. Thus, our new results are for n > 2.

(b). The mixed strategy case

To study mixed strategies we let I be the strategy defined by the probabilistic density function p(x). Since there are m rewards ordered in decreasing values, the probability that a player bidding s wins reward v_j (1 ≤ j ≤ m) is

Summing over all rewards and subtracting the payment, we obtain for the expected pay-off E(s, I) of a player bidding s when all other n − 1 participants in the auction play according to I:

2.1

where Following Chatterjee et al. [12] to obtain an ESS given by p(x), we use the Bishop–Cannings theorem [4] which states that for any strategy s in the support of I, the expected pay-off E(s, I) has to be constant [2]. Therefore, for any possible ESS δE(s, I)/δs = 0 must hold. Note that this condition leads to a non-negative pay-off for any pure strategy in the support of I if the pure strategy zero (bids zero, pays nothing) is in the support of I. Hence, the expected sum of bids per auction (which equals the sum of payments) is in equilibrium upper bounded by the sum of reward values per auction. We differentiate equation (2.1) with respect to s, set E′(s, I) = 0, and get that

2.2

For the general case of m, we cannot obtain analytical solutions for p(s). However, in the special case of two rewards v₁ and v₂ (with v₁ ≥ v₂), equation (2.1) simplifies to

2.3

and analytical solutions become attainable.

In figure 3b, we show numerical solutions for p(s) of equation (2.3). These mixed equilibrium solutions correspond to the density of strategy values in a population where no individual can increase its benefit by solitary changing to a different strategy which is also in the support of the mixed strategy. The results for two rewards are qualitatively very different to the results for one reward (figure 3a) [12]; even if more individuals compete for the rewards and the ratio of rewards to participants is identical. The U-shaped form of the obtained mixed equilibria in the cases of n = 5 and n = 10 suggest that individuals might be able to specialize in low or high investments. This observed specialization helps to explain our initial example of the growth competition in a forest: a few individuals with similar high investments, many individuals with low investments, and rare incidences of plants with an intermediate growth investment. Although the height achieved differs greatly among the plants and the high-investing individuals win most of the rewards, in equilibrium the obtained pay-offs of the individuals are identical. Two case studies on Chenopodium album plants report data for the height distribution of many plants in a fixed area with a very similar pattern as the analytical results of our model [38,39]. Some plants were much taller than the majority of the plants. The density function of the heights in this plant species has a similar U-shaped form to our mixed equilibria functions (see bottom plots in Fig. 3 in [39]). As predicted by our results, when more plants are competing in a small area, fewer plants make high growth investments and the remaining plants abstain from high investments to save their energy (cf. our figure 3 and fig. 1 in [38]).

Figure 3.

An additional reward in biological auctions results in a qualitatively different equilibrium. The mixed strategy equilibrium changes with the number of participants, n, in an auction. (a) Mixed equilibrium functions have a negative exponential shape in biological all-pay auctions with one reward. (b) By contrast, in biological all-pay auctions with two rewards, the mixed equilibrium functions can also be increasing or U-shaped functions. The U-shaped mixed equilibria suggest some specialization among the individuals.

If the mixed equilibrium is also an ESS, then natural selection is sufficient to prevent any different strategy to successfully invade a population that adopted the ESS. We find significant differences to the single reward case [12]: (i) while for the single reward case an ESS always exists for n > 2, in the case of two rewards an ESS is not guaranteed to exist even if n > 2. (ii) For the single reward case, the mixed equilibrium either has a uniform shape or a negative exponential shape (always non-increasing), by contrast, in the case of multiple rewards, the mixed equilibrium can be a uniform, an increasing or a U-shaped function (Theorem 2; proofs are given in the electronic supplementary material). Note that in the case of v₂ = 0, the solutions simplify to the results for one reward [12]: (for n = 2) and (for n = 3). We note that for n = 2, P(0) = 0 and In other words, the strategy space is given by the interval [0, v₁ − v₂]. If v₁ = v₂, no bid should be made as one always obtains a reward of value v₁ independent of the bid.

Theorem 2.2. —

In biological all-pay auctions with two rewards (v₁ ≥ v₂), the following assertions hold:

1. For n = 2, the following probability density function is a mixed equilibrium but not an ESS:

2.4

2. For n = 3, the following probability density function is a mixed equilibrium:

2.5

Moreover, the mixed equilibrium is also an ESS iff v₁ > 2v₂.

In addition to our analytical results, we implemented a Wright–Fisher process [40] to study the evolutionary dynamics in our model. Our program simulates the evolution of pure bidding strategies in a population of size N starting from a population with random strategy values. The reproductive success of an individual is proportional to its fitness given by the sum of the rewards won minus its payments in K auctions per generation. Over time, successful strategies spread in the population. The details of the computer simulation are given in the Material and methods section.

We observe great agreement between the analytical strategy distribution and the simulated average strategy distribution (figure 4). The simulation results are less accurate for strategy values close to the maximum value v₁ which is because of the limited number of available strategies in the computer simulations. Nevertheless, the stochastic computer simulation results confirm the qualitative differences of the results for biological auctions with one and multiple rewards. In the case of two rewards, we find that the shape of the mixed equilibrium is mostly determined by the value of the second reward v₂ (figure 4). As described above, for v₂ → 0, the simulation results converge to the results of biological auctions with one reward.

Figure 4.

(a–c) Shapes of the mixed strategy equilibria in biological all-pay auctions with two rewards depend on the relative value of the second reward v₂ to the first reward v₁. Full lines are derived from equation (2.3). Dotted lines correspond to the average strategy distribution obtained through stochastic computer simulations of 10⁷ generations. Average strategy distributions converge very well to the calculated mixed equilibrium. Differences for high strategy values are because of the discrete strategies in the computer simulations. Parameter values: value of highest reward v₁ = 1, population size N = 1000, auctions per generations K = 1000, number of strategies ℓ = 101 in [0, v₁] and mutation probability u = 0.5%.

In figure 5, we show how our results extend to more rewards (m) and more auction participants (n). We observe that the general U-shape of the mixed equilibrium remains for larger values of m and n. The shape is determined by the values of the rewards and the ratio m/n. For very small m/n ratios, the shape of the mixed equilibrium shows similarities to the single reward case.

Figure 5.

(a–d) Biological all-pay auctions with many rewards and many participants. Lines correspond to the average strategy distribution obtained through stochastic computer simulations of 10⁷ generations. Shapes of the average strategy distribution depend on the values of all rewards (v_i) and on the number of participants per auction (n). Parameter values: population size N = 1000, auctions per generations K = 1000, number of strategies ℓ = 101 in [0, v₁] and mutation probability u = 0.5%.

We study the evolutionary dynamics in our model via time-averages over 25 generations of the strategy distribution in the population. We find that typically two subpopulations are present: low bidders and high bidders (figure 6). Rarely, we see intermediate bidding strategies invading these subpopulations for a few generations until again the low- and high-bidding subpopulations take over. These results demonstrate that there is no static equilibrium in the case of a finite population with pure strategies, similarly as in biological auctions with single rewards [12]. Although we observed a cyclic behaviour, low and high bidders existed throughout the simulation and can dominate the population for many generations (figure 6). By contrast, intermediate strategies only exist for a small number of generations and are quickly replaced by superior strategies.

Figure 6.

Evolutionary dynamics in biological all-pay auctions with multiple rewards. The dynamics show a clear distinction between low- and high-bidding subpopulations. Mutants with intermediate strategies are sometimes able to invade and disrupt these two subpopulations because of stochastic drift. After a few generations low- and/or high-bidding individuals again take over the population. No static equilibrium exists in a finite population with pure strategies. The strategy distribution snapshots are time-averages over 25 generations and were also taken in intervals of 25 generations after simulating 10⁴ generations initially. Parameter values: population size N = 1000, auctions per generations K = 1000, reward values v₁ = v₂ = v₃ = v₄ = v₅ = 1, number of strategies ℓ = 101 in [0, v₁] and mutation probability u = 1%.

3. Discussion

Most studies on models of competing individuals focus on one reward. The limitation to competitions with one reward restricts the applicability of those models and hinders us from understanding more realistic scenarios. In this work, we have generalized the model of biological all-pay auctions to multiple rewards. One might have expected that a generalization to multiple rewards does not change the shape of the ESSs and only results in an increased mean of the distribution. However, as we have shown above, the solutions for multiple rewards are qualitatively very different from the solutions for a single reward. While the mixed equilibria of the bidding strategies in the single reward case have the form of a negative exponential distribution, this can change to a U-shaped distribution in the case of multiple rewards.

These qualitatively very different results suggest that multiple rewards may allow for specialization of the individuals. The evolution of two distinct subpopulations, one that competes for the rewards and one that largely avoids the costs of the competitions, may also contribute to a speciation process. The size and the overlap of these subpopulations follow from the number of rewards, their values and the number of individuals competing for those rewards (figure 5). As we show in figure 6, the sizes of these subpopulations are subject to evolutionary dynamics determined by selection and mutation. However, the average strategy distribution converges to our analytical results. These evolving subpopulations of low bidders and high bidders also reveal a connection to the two-player and two-strategy Hawk–Dove game where a similar mixed equilibrium exists [22]. The hawks compete for the resource and risk being incurred, whereas the doves abstain from costly conflicts and, at best, share the resource. If we generalize this strategy space from hawks and doves to a continuum of bidding strategies, we obtain the war of attrition and the mixed equilibrium of high (hawks) and low (doves) bidders changes to a negative exponential distribution of bidding strategies [2].

A similar pay-off structure is used in sperm competition games where the strategy and costs correspond to the amount of ejaculated sperm [41,42]. In contrast to biological auctions, the reward (male fertilizing the female) is not always given to the males with the largest number of invested sperm. In fair and symmetric sperm competition games, the winning probability is given by the own strategy value divided by the sum of the strategy values of the competitors [41]. Another difference is that for each reward (female) a separate investment has to be made in sperm competition games whereas in biological auctions one bid is submitted to compete for multiple rewards. In agreement with our results, the amount of invested sperm of a male per mating decreases with the number of competing males. In the case of females with different reproductive potential, males may actively discriminate between the females (rewards) and use different strategies to maximize their own reproductive success [42,43].

The effects of supply and demand have been studied in the context of trading strategies in biological markets [44,45]. Noë & Hammerstein [44] show that trading strategies influence the selection of specific traits depending on the current demand and supply (e.g. sexual selection). In our model we show how a population evolves over time to meet a specific supply level. We study the effects of a fixed demand (number of participants per auction n) and supply (number of rewards m) on the investing strategy of the individuals. In the future it would be interesting to study both trading and investing strategies in the same environment. Such a study might provide new insights into the evolution of nuptial gifts in scorpionflies where the strategies of females and males evolve simultaneously [44,46,47].

Our results for multiple rewards on the expected growth investments in a competitive environment are very similar to the height distributions observed in C. album plants when there are many plants in a small area [38,39]. Models for single rewards fail to explain these data. We note that there are many more evolutionary models that try to explain the height distribution of plants [48,49]. Experimental biologists found various height distributions for different plant species. The key difference to the situation we are modelling here might often be the missing competition among the plants for sunlight. With only a few plants in a fixed area there is no competition for sunlight and hence our model is not applicable. However, there might be other kinds of competitions, e.g. for nutrients or water, where instead of the plant height the root length could matter. We believe that our model should also be relevant in various other competitive scenarios which we have not described here. An interesting direction to explore might be other trait distributions of species in situations of competitions for multiple resources like the height distribution of plants in a forest, which would help to further improve our model. Additionally, one could consider a structured population to account for relatedness effects.

4. Material and methods

(a). Stochastic computer simulations

The tool to simulate the biological auctions can be downloaded from https://github.com/johannesreiter/bioauctions (including the Python source code and additional documentation to execute the program). The program simulates the evolution of pure bidding strategies in a well-mixed population of constant size N according to a Wright–Fisher process [33,40]. Other input parameters are: (i) number of auctions per generation K, (ii) number of participants per auction n, (iii) number of rewards m, (iv) reward values v₁, … , v_m, (v) number of discrete strategies ℓ in the interval [0,1] and (vi) the mutation rate u.

Initially, the program creates a population of N individuals with random strategy values in the interval [0, v₁]. We do not need to consider strategy values greater than v₁ since individuals with such a strategy will incur a negative fitness in any environment of players and therefore, these strategies cannot be present in an equilibrium (Bishop–Cannings theorem [2,4]). Note that strategy zero always obtains a non-negative fitness. For efficiency reasons, we restricted our simulations from the beginning to the relevant interval. However, our tool does not restrict v₁ (except v₁ > 0) and hence various strategy spaces can be considered. In each generation, all individuals start with the same background fitness. The program then simulates K auctions and the fitness of the individuals is updated according to the rewards won and the payments in the auction in which they participated. The strategy values for the next generation of individuals are chosen proportional to the fitness values of the current population. Bidding strategies with higher fitness have a high probability to be chosen for reproduction and hence are more likely to spread in the population. With a mutation probability of u, a random strategy in [0, v₁] is assigned to an individual to ensure some diversity in the population. A well-mixed population is assumed (no population structure or relatedness effects). After the new generation has been created, the next round of K auctions is played and based on the achieved pay-offs in these auctions another generation will be created. The program simulates the evolution of strategies by sequentially creating strictly non-overlapping generations of individuals. The strategy distribution is computed via averaging over the number of individuals per strategy value over all generations.

To better understand the underlying evolutionary dynamics of our model, our tool can take snapshots of the strategy distribution over time. In the file settings.py, we can configure the frequency and the start and end generation of the snapshots.

(b). Mathematical proofs

The detailed proofs are provided in the electronic supplementary material.

Data accessibility

All our data are generated by a computer simulation. The source code and some additional documentation can be downloaded from https://github.com/johannesreiter/bioauctions.

Authors' contributions

J.G.R., M.A.N. and K.C. designed the study and wrote the manuscript. J.G.R. performed mathematical analysis and computer simulations. A.K., R.G. and K.C. provided input to the mathematical analysis.

Competing interests

We have no competing interests.

Funding

This work was supported by grants from the John Templeton Foundation, ERC Start Grant (279307: Graph Games), FWF NFN Grant (No S11407N23 RiSE/SHiNE), FWF Grant (No P23499N23) and a Microsoft faculty fellows award.