Modelling information exchange in worker–queen conflict over sex allocation

Abstract

We investigate the conflict between queen and worker over sex allocation, specifically the allocation of the queen's eggs between workers and reproductives and the allocation of the reproductive eggs between male and female. In contrast to previous models, we allow workers to observe and use information about the strategy of the queen. We consider three conflict models: simultaneous (no information exchange), sequential (a one-way information exchange) and negotiated (an iterated two-way information exchange). We find that the first model produces sex ratios intermediate between the classic queen (1 : 1) and worker (1 : 3) optima. The second model, in which the worker has information about the queen's decisions, produces a different result and one that is somewhat counter-intuitive in that the sex ratios are less female-biased than for the other two models, and in fact are often male-biased. The third model predicts sex ratios intermediate between the first two models. We discuss how these findings may shed new light on observed sex allocation patterns in social insects and we suggest some experimental tests.

1. Introduction

It was probably Hamilton (1967) who first drew attention to the effect that genetic asymmetries between males and females could have on the ‘unbeatable’ sex ratio. However, it was not until nine years later that Trivers & Hare (1976) applied these ideas to hymenopteran eusocial insects in which both queen and workers might have a say in colony sex allocation decisions. In these decisions, the queen and the workers have different genetic interests, the queen being equally related to sons and daughters, but the workers being more closely related to the queen's daughters than to her sons. This causes the workers to favour a higher proportion of females among the reproductives than the queen. Under the simplest of assumptions, such as the queen being singly mated, the queen's preference is a 1 : 1 male : female ratio and the workers prefer 1 : 3. Trivers and Hare felt that workers are more likely to win the conflict, and until recently many studies of hymenopteran sex allocation supported this view. However, several new studies give examples of sex ratios close to the queen's optimum (e.g. Helms 1999; Jemielity & Keller 2003; Duchateau et al. 2004). In order to understand these contradictory results, Mehdiabadi et al. (2003) call for more sophisticated shared-control conflict models to encompass the dynamic balance between queen and worker power over the colony sex ratio.

Previous models of worker–queen conflict that allow both parties some control over aspects of sex allocation typically conclude that a more or less ‘fair’ compromise between the workers' optimum and the queen's optimum is reached, the precise location of which depends on specific model assumptions (Bulmer 1981; Bulmer & Taylor 1981; Matessi & Eshel 1992; Reuter & Keller 2001).

A potentially important aspect of the worker–queen interaction that has not been addressed in models so far, concerns the flow of information between queen and workers. Indeed, Trivers & Hare (1976) argued that since workers in some sense might seem to have ‘the last say’, they should win the conflict and we might expect sex ratios to more closely match the workers' optimum than that of the queen. Similarly, Beekman et al. (2003) argue that ‘Because the workers act after the queen, this probably gives them greater power than the queen’. If it is true that the workers have ‘the last say’, the workers might be able to observe the queen's actions and use this information to determine their own actions. Here we present a model of shared worker–queen control which explicitly allows workers to do just that and adjust their allocation to the queen's allocation. As we shall see, this leads to some surprising predictions.

2. The Model

(a) Allocation decisions and control

The notation is given in table 1. We work with an outbred haplodiploid population, which is monogynous, monoandrous and has non-reproductive workers. We assume that the queen lays an arbitrary fixed number N of eggs, which are then raised by the workers. In this process there are three trade-off decisions to be made (figure 1). The first is the proportion x of males among the eggs and we give control of this to the queen. The second trade-off concerns the proportion y of female eggs made into workers (the rest to be gynes), and we give control of that to the workers (but see § 4). Finally there is the proportion z of worker resources allocated to the raising of males as opposed to gynes, and we also give control of that to the workers. Our models provide evolutionarily stable values of these three primary variables, x, y and z. Whenever possible we present analytical results, but most of our results are based on numerical calculations. In addition, we have used individual-based simulations to verify the analytical and numerical work. In these simulations, each decision variable is determined by a single genetic locus, unlinked to the other loci, and allelic values undergo small mutational modifications with a fixed small probability.

Table 1.

Notation.

notation	description
the mathematical analysis works with three primary variables
x	the proportion of males among the eggs (queen controlled)
y	the proportion of female eggs made into workers (worker controlled)
z	the proportion of workers allocated to the raising of males (worker controlled)
$\tilde{y}(x)$, $\tilde{z}(x)$	optimal values of y and z in response to queen's x value (model 2)
x1=x	proportion male eggs
x2=(1−x)(1−y)	proportion gyne eggs
r=x1/(x1+x2)	sex ratio among reproductive eggs
z1=zy(1−x)	proportion worker resources devoted to the raising of males
z2=(1−z)y(1−x)	proportion worker resources devoted to the raising of gynes
h(zi)	amount of resources devoted to raising sex i reproductives
ui=h(zi)/xi	amount of resources per egg devoted to the raising of sex i reproductives
si(u)	probability of survival to adulthood of sex i egg if allocated u (equation 1)
ki	the point at which sex i egg survival per worker, si(u)/u, is maximized
Xi=xisi(ui)	# sex i reproductive adults
R=X1/(X1+X2)	sex ratio among reproductive adults
$\hat{E}={\hat{X}}_2/{\hat{X}}_1$	population-wide reproductive adult female : male ratio. The ^ signifies a population-wide average
Q	inclusive fitness of queen
W	inclusive fitness of workers

Figure 1.

Allocation strategies. The queen controls the allocation x of eggs between haploid and diploid and the workers control the allocation y of diploid eggs between gynes and workers and the allocation z of workers between the raising of male and female reproductives.

(b) Consequences of allocation decisions

It is notationally and conceptually useful to work with the secondary variables x_i, the proportion of sex i reproductive eggs, and z_i, the proportion of resources allocated by workers to the rearing of sex i reproductive eggs, so that $x_1+x_2+z_1+z_2=1$. Expressions for these in terms of the primary variables are found in table 1. The total amount of resources invested by workers in the raising of sex i offspring is an increasing function h(z_i) with diminishing returns. This reflects an assumption that per capita productivity of workers declines when more workers are present (see also Reuter & Keller 2001). In examples we use $h(z_i)=z_i/(1+z_i)$.

We further suppose that the survival of the sex i eggs to adulthood is a function, s(u_i), of the amount of resources $u_i=h(z_i)/x_i$ invested per sex i egg. The function s is assumed to be increasing and sigmoidal. In our examples we use the functional form

$$s(u)=\frac{u^2}{u^2+k^2}.$$ 2.1

The parameter k is the value of u at which egg survival per unit investment, s(u)/u, is maximized. Thus the parameter k acts as a scaling factor for the u-axis. We consider only sufficiently small values of k, since for large values of k the survival function s(u) is concave-up (increasing marginal return) for a relatively large part of the u-domain with the result that selection favours specialization in either males (x<1 and z=1) or females (x=0 and z=0), as result obtained in the model of Roisin & Aron (2003). That this should happen is expected from Jensen's inequality $E(s(u)\ge s(E(u))$which holds for all convex functions s(u), and this behavior is confirmed by our simulations.

(c) Inclusive fitness

We suppose that the queen and workers choose the parameters under their control to maximize their inclusive fitness. Though they are engaged in the cooperative rearing of the same brood, their genetic interests are slightly different as the workers are more closely related than the queen to female reproductives and less closely related to males. Indeed, under outbreeding, the inclusive fitness of queen and worker are (Bulmer 1994 p. 222):

$$Q(x,y,z,\hat{x},\hat{y},\hat{z})=\frac{1}{2}{X}_2+\frac{1}{2}{X}_1\hat{E}$$ 2.2

$$W(x,y,z,\hat{x},\hat{y},\hat{z})=\frac{3}{4}{X}_2+\frac{1}{4}{X}_1\hat{E}$$ 2.3

Here X_i=x_is(u_i) is the number of sex i reproductive adults, and $\hat{E}={\hat{X}}_2/{\hat{X}}_1$ is the population-wide adult female : male ratio. The ^ signals that variables are to be given their population-wide values. Thus Q and W are functions of six variables: the individual values of the three primary variables, x, y and z (table 1) and, through $\hat{E}$, the population-wide values of these variables, $\hat{x},\hat{y}$ and $\hat{z}$.

Note that the inclusive fitness expressions do not contain terms that correspond to colony survival. Our models therefore apply to annual colonies, but also to perennial colonies as long as x, y and z do not affect colony survival. The latter assumption may sometimes be questionable, but we keep it for the sake of simplicity and because we see no reason why our main qualitative results would be changed by this complication.

(d) Information flow

We consider three models which embody different opportunities for the two parties to receive and respond to information about the other.

Model 1 Simultaneous moves. There is no chance for either party to get advance information about the other's moves.

Model 2 Sequential moves. The queen acts first; the workers can observe the queen's strategy and respond.

Model 3 Negotiation. The queen and workers negotiate with a sequence of offers and responses converging to a final strategy pair (McNamara et al. 1999; Taylor and Day 2004).

Mathematically, in model 1, we assume that each party acts to maximize its inclusive fitness given that the other acts according to the population. The conflict is resolved at Nash equilibrium for the game. All previous shared-control models of the worker–queen conflict are of this type (e.g. Bulmer 1981; Bulmer & Taylor 1981; Matessi & Eshel 1992; Reuter & Keller 2001). In model 2, the workers act to maximize their inclusive fitness given the behavior of the queen, and the queen acts to maximize her inclusive fitness given that the worker will respond optimally to her decisions. The conflict arrives at what is called a Stackelberg equilibrium (e.g. Fudenberg & Tirole 1991; Abe et al. 2003). In model 3, each party chooses, not its allocation strategy but its negotiation strategy, and as in model 1, each party acts to maximize its inclusive fitness given that the other acts according to the population. Here we arrive at a Nash equilibrium in the negotiation strategy variables.

(e) Equilibria and their stability

In each model, the three equilibrium conditions are obtained by setting to zero the partial derivatives of each actor's inclusive fitness with respect to her variables of control. The resulting equations can be found in Appendix A. The evolutionary and convergence stability of equilibria has been checked according to the criteria in Appendix B. We present results for models 1 and 2 only, as the results of model 3 were always intermediate.

3. Results

The main results are shown in figure 2 for a range of k-values. Sex ratio theory concerns the allocation of reproductive resources between the raising of male and female offspring. Since in our model, there are different types of allocation decisions made, there are different possible measures of sex ratio one might use. First of all there is the primary sex ratio r among the reproductive eggs, determined by the joint decisions of queen and workers. Second there is the proportion z of worker resources allocated to the raising of males and females, and third there is the resulting secondary ratio R among the reproductive adults. The latter is always intermediate between the first two, as the shift from the first (r) to the third (R) is produced by the relative bias in z.

Figure 2.

Primary allocation to males (r, solid line) and allocation of workers (z, small dashed line) and adult sex ratio (R, big dashed line). (a) Results of model 1 and (b) those of model 2. Results of the negotiation game (model 3) are intermediate.

In model 1 (simultaneous action) all sex ratios are female-biased and somewhere in the middle between the classic queen optimum of 50% male and the classic worker optimum of 25% male. The adult ratios are slightly more female-biased than the primary ratios, and this reflects the efforts of the workers to invest relatively more in females than would be the preference of the queen.

The striking thing about model 2 (sequential action) is that there are many more males than in model 1, and indeed the primary ratio r is often male-biased. In terms of figure 2, the curves in figure 2b are higher than the curves in figure 2a. For small k both the primary and the secondary sex ratios approach the classic queen optimum of 50% male. These calculations are supported by individual-based simulations in which we assume that the worker response functions $\tilde{y}(x)$ and $\tilde{z}(x)$ are linear functions of the queen's proportion of haploid eggs x.

Colony output, as measured by the number of emerging adult females is substantially higher for model 1 than model 2. The colony pays a cost for the information flow from queen to worker. The cause of this lower efficiency is the greater difference between the queen's allocation strategy r and the worker's allocation strategy z. This can be seen in the greater spread between the two curves in the right panel of figure 2 than in the left panel. This greater spread causes a greater difference between u₁ and u₂ and thus (since the s-graph is concave down) lower average survival.

4. Discussion

In terms of the classic sex ratio conflict between queen and worker, which has the queen favouring a 1 : 1 male : female ratio while the workers favour 1 : 3, model 1 predicts sex ratios close to the halfway point between the two parties' optima, so there is no clear ‘winner’ of the conflict. Earlier theoretical studies that, like our model 1, assume that workers do not take the queens' actions into account, usually reach the same conclusion. Bulmer & Taylor (1981) conclude that the queen may have considerable control over the investment ratio, but that the realistic possibility of variable investment in a new queen by the workers (comparable to our z) gives back much of that control to the workers. The analysis of Matessi & Eshel (1992) also shows that when conflict between queen and workers exist, ESSs are intermediate between 1 : 1 and 1 : 3. Reuter & Keller (2001) examine a model somewhat similar to our model 1. In their ‘mixed control’ model, the queen determines the proportion x of eggs that are male and the workers, not knowing the queen's decisions, determine the proportion y of diploid eggs that become workers. Their model stops there and assumes that the workers allocate equal resources per egg to males and gynes. In our models, we give the workers a chance to bias this allocation through determination of z. The model of Reuter & Keller (2001) also yields ratios between the classic queen and worker optima, but because their workers allocate equally to male and female eggs, allocation is less female-biased than in our model 1.

The unexpected outcome is that model 2 appears to be ‘won’ by the queen, even though the workers are the party with increased information and the ability to act upon that information. This result is at first counter-intuitive, as one might think that the information gained by the workers would give them more control, and that this should result in a greater female bias than in model 1. The key to understanding this result is to realize that natural selection acts on the queen's decision as if the queen knows that the workers will respond optimally, thus placing her in some sense in the driver's seat in being able to make the first move. This general phenomenon has been studied for some time in the game theory literature. Maynard Smith (1982) devotes a chapter to ‘honesty, bargaining and commitment,’ and gives a number of examples of simple games in which the player who is permitted to declare first his strategy can improve his payoff and reduce that of his opponent. Other examples show that the possibility of back-and-forth bargaining can lead to more cooperative outcomes (Taylor & Day 2004). In a standard economics text, Fudenberg & Tirole (1991) comment that ‘By committing himself to a given sequence of actions a player may be able to alter the play of his opponents.’ They call this effect ‘paradoxical’ and observe more generally that ‘A player can gain by reducing his action set or decreasing his payoff to some outcomes, provided his opponents are aware of the change…’ Elsewhere we study this phenomenon more generally in the context of models of conflict and cooperation in biology and derive conditions under which the party that makes the first move gains a certain advantage over the party with the second move (P. D. Taylor and I. Pen, personal observation). The argument of Trivers & Hare (1976) that workers should win the conflict because they have the ‘last say’ does not always work.

If this is paradoxical, then a similar paradox is found in a sex ratio model of Eshel & Sansone (1994) in which there is parent–offspring conflict over the sex ratio. If males are more costly than females, then the parent wants a more female-biased sex ratio than do the offspring. Suppose the offspring have control of their own sex, but the parent can manipulate the overall ratio by transforming males to females with success that depends on how receptive the individual is. Then the more information the parent has on the receptivity of different males to sex-change, the less control of the sex ratio she apparently gets. It turns out that an increase in parental discrimination causes an evolutionary decrease in receptivity, thus decreasing the manipulative ability of the parent.

A sex-ratio model in which this paradox does not occur is found in Pen & Weissing (2002). This study looks at the conflict between mother and father when male offspring have a different cost than female and where the father determines the sex ratio (say proportion sons r) and the mother the number n of eggs. The mother's investments per son and daughter are fixed at E_m and E_f and her survival depends negatively on her total investment E_T=n(rE_m+(1−r)E_f). The father favours r=1/2 whereas the mother favours equal allocation, rE_m=(1−r)E_f. If the mother cannot observe the r produced by the father, we get the father's optimum r=1/2, but if she can we get the mother's optimum of equal allocation.

Is our model 2 capable of explaining the cases where sex ratios close to the queen's optimum have been found (Mehdiabadi et al. 2003)? So far, very few species have been studied in sufficient detail to verify the assumptions of our models. However, there is one species of ant, Lasius niger, whose sex allocation and potential control mechanisms have received considerable study. This species seems to fit the assumptions of our model 2 quite closely. There is clear evidence that the workers of L. niger can detect the sex of individual eggs (Jemielity & Keller 2003): only either male or female larvae are placed in the uppermost chambers of the nest. It is therefore conceivable that workers could adjust their behavior (our variables y and z) to the fraction of haploid eggs (our variable x). Indeed, spatial segregation of the sexes seems a very effective way for workers to adjust how much they feed to each sex (our variable z). In agreement with the predictions of model 2, it has been found that L. niger has sex ratios very close to the ‘queen's optimum’ (Fjerdingstad et al. 2002), and that this relatively high proportion of males cannot be attributed to worker reproduction. What is unknown is whether L. niger workers have the crucial mechanism that is needed to make model 2 work: observing the queen's strategy and using this information to adjust their own decisions. It might be interesting to experimentally alter the number of haploid eggs in the nest and to observe whether this changes the workers' allocation decisions in the expected direction.

In our models we assumed that workers are in control of y, the fraction of diploid eggs that become workers. So far little is known about which party tends to be in control of y (Reuter & Keller 2001), but in the ant Lepthotorax acervorum it appears that workers can selectively bias the final caste of developing females (Hammond et al. 2001). Just to compare, we have examined the case in which the queen has control of both x and y and the workers can only determine z. This seems to make little difference in the outcome. All sex-ratio measures are close to their values in figure 2, (within 5% in model 1, and within 2% in model 2).

One might object to our models in that workers do not have the option of killing males as a means to achieve a more female-biased allocation. However, we did allow for this possibility but saw that there are no stable equilibria where workers kill a certain fraction of the males. The reason is that the queen could then increase her fitness by laying fewer haploid eggs and more diploid eggs. Of course, male-killing does occur sometimes (Sundström et al. 1996; Chapuisat et al. 1997; Foster & Ratnieks 2000), raising the question under what conditions (that are apparently not in our models) such behavior might be stable after all. An interesting possibility is that workers consuming haploid eggs actually gain some benefit from them, something, which we did not allow for. This has in fact been reported (Foster & Ratnieks 2001, and references therein).

Appendix A

Appendix A. Equilibrium conditions

In each model, the equilibrium equations are obtained by setting to zero the partial derivatives of each actor's inclusive fitness with respect to the variables of control. The queen has control variable x and the workers have the control vector y=(y,z)^T. For each equilibrium condition the variables x and y are to be evaluated at the population values $\hat{x}$ and $\hat{\mathit{y}}$. We use subscripted variables to denote derivatives with respect to those variables (using the same notiation for partial derivatives, gradient vectors and Hessian matrices).

Model 1. Simultaneous moves. The equilibrium conditions are

$$Q_x=0,$$ A1

$$W_{\mathbf{y}}=0.$$ A2

Model 2. Sequential moves. The worker equations are the same as model 1, but in the queen's conditions allowance has to be made for the response of the workers. In formulating these, we suppose that the workers are able to respond optimally (to maximize W). We define the function

$$\tilde{Q}(x,\hat{x})=Q(x,\tilde{\mathbf{y}}(x),\hat{x},\tilde{\mathbf{y}}(\hat{x})),$$ A3

where $\tilde{\mathbf{y}}(x)$ is the workers' response to an x-decision from the queen. Then the equilibrium conditions are

$${\tilde{Q}}_x=0,$$ A4

$$W_{\mathbf{y}}=0.$$ A5

Equation (A 5) defines implicitly the response function $\tilde{\mathbf{y}}(x)$. The derivative ${\tilde{\mathbf{y}}}_x$ is the rate of response of the workers to x-changes by the queen and it is obtained by differentiating (A 5) with respect to x and solving for ${\tilde{\mathbf{y}}}_x$:

$${\tilde{\mathbf{y}}}_x=-{(W_{\mathbf{y}\mathbf{y}})}^{-1}{W}_{x\mathbf{y}}.$$ A6

Appendix B

Appendix B. Stability of equilibria

(a) Evolutionary stability

For models 1 and 2 we have verified numerically that the following 3×3 matrix is a stability matrix (i.e. all eigenvalues have negative real part), evaluated in equilibrium:

$$\left[\begin{array}{cc}{Q}_{xx}& Q_{\mathbf{y}x}^T\\ W_{x\mathbf{y}}& W_{\mathbf{y}\mathbf{y}}\end{array}\right].$$ B1

For model 2, Q is replaced by $\tilde{Q}$ and its second derivatives are given by

$${\tilde{Q}}_{xx}=Q_{xx}+2Q_{x\mathbf{y}}^T\tilde{\mathbf{y}}+{\tilde{\mathbf{y}}}_x^T{Q}_{\mathbf{y}\mathbf{y}}{\tilde{\mathbf{y}}}_x+Q_{\mathbf{y}}^T{\tilde{\mathbf{y}}}_{xx},$$ B2

$${\tilde{Q}}_{\mathbf{y}x}=Q_{\mathbf{y}x}+Q_{\mathbf{y}\mathbf{y}}{\tilde{\mathbf{y}}}_x.$$ B3

The ${\tilde{\mathbf{y}}}_{xx}$ in (B 2) is obtained by differentiating (A 5) twice with respect to x and solving:

$${\tilde{\mathbf{y}}}_{xx}=-{(W_{\mathbf{y}\mathbf{y}})}^{-1}(W_{xx\mathbf{y}}+2W_{x\mathbf{y}\mathbf{y}}{\tilde{\mathbf{y}}}_x).$$ B4

(b) Convergence stability

For both models we have checked numerically that the standard evolutionary dynamic (Abrams et al. 1993; Dieckmann & Law 1996; Geritz et al. 1998; Hofbauer & Sigmund 1998) is stable, assuming no genetic correlation among the three variables. For example, for model 2, at population mean values $\hat{x}$ and $\hat{\mathbf{y}}=(\hat{y},\hat{z})$, the dynamic equations for the population means are:

$$\frac{\text{d}\hat{x}}{\text{d}t}={{\tilde{Q}}_x|}_{(x,\mathbf{y})=(\hat{x},\hat{\mathbf{y}})},$$ B5

$$\frac{\text{d}\hat{\mathbf{y}}}{\text{d}t}=W_{\mathbf{y}}{|}_{(x,\mathbf{y})=(\hat{x},\hat{\mathbf{y}})}.$$ B6

From (A 4) and (A 5), an equilibrium point is a stationary point of this system. Such a point is stable if the 3×3 matrix

$$\left[\begin{array}{cc}\frac{\partial }{\partial \hat{x}}\frac{\text{d}\hat{x}}{\text{d}t}& {\left(\frac{\partial }{\partial \hat{\mathbf{y}}}\frac{\text{d}\hat{x}}{\text{d}t}\right)}^T\\ \frac{\partial }{\partial \hat{x}}\frac{\text{d}\hat{\mathbf{y}}}{\text{d}t}& \frac{\partial }{\partial \hat{\mathbf{y}}}\frac{\text{d}\hat{\mathbf{y}}}{\text{d}t}\end{array}\right],$$ B7

is a stability matrix at that point. For model 1 we replace $\tilde{Q}$ in (B 5) with Q.