Joining or opting out of a Lotka–Volterra game between predators and prey: does the best strategy depend on modelling energy lost and gained?

Abstract

Apart from interacting, prey and predators may also avoid each other by moving into refuges where they lack food, yet survive by switching to an energy-saving physiological state. Lotka–Volterra models of predator–prey interactions ignore this option. Therefore, we have modelled this game of ‘joining versus opting out’ by extending Lotka–Volterra models to include portions of populations not in interaction and with different energy dynamics. Given this setting, the prey's decisions to join or to opt out influence those of the predator and vice versa, causing the set of possible strategies to be complex and large. However, using game theory, we analysed and published two models showing (i) which strategies are best for the prey population given the predator's strategy, and (ii) which are best for prey and predator populations simultaneously. The predicted best strategies appear to match empirical observations on plant-inhabiting predator and prey mites. Here, we consider a plausible third model that does not take energy dynamics into account, but appears to yield contrasting predictions. This supports our assumption to extend Lotka–Volterra models with ‘interaction-dependent’ energy dynamics, but more work is required to prove that it is essential and that what is best for the population is also best for the individual.

1. Introduction

Theory on the ecology of food webs is still founded to a considerable extent on the assumptions underlying a predator–prey model proposed independently by Lotka (1925) and Volterra (1926) [1–4]. One of these assumptions is that predators and prey are continuously exposed to each other and therefore interact. In reality, however, predators and prey also have the option to avoid interacting with each other. They may then move into refuges where there is no prey or food and where survival demands a switch to an energy-saving physiological state, such as diapause.

Diapause has always been thought to emerge solely to overcome the winter season and to emerge in response to abiotic factors signalling the onset of the winter season, but it may also emerge in response to food scarcity and/or the risk of being eaten, as recently shown for the case of predator mites and fruit-tree red spider mites (Acari: Phytoseiidae, Tetranychidae) [5–12]. Although empirical proof of these diapause-governing principles is limited, they may well hold generally in ecological interactions.

We have extended Lotka–Volterra predator–prey models in two ways [13,14]: (i) by splitting the predator population as well as the prey population in two portions, one that is joining the interactions and another that is not; and (ii) by including energy dynamics that differs between those joining the interactions and those opting out of the interactions. Using this extended Lotka–Volterra model, we ask how predators and prey should best allocate their time between joining the interactions and avoiding them. This question is not easy, because the best solution has to be determined from a large set of possible strategies, and because the strategies can be quite complex given that the prey's decisions to join or opt out influence those of the predators, and vice versa. However, using game theory, we have shown which strategy is best for the population of prey [14] and which is best for the population of prey and the population of predators simultaneously [13]. In this paper, we review these two models, their assumptions and their predictions, and additionally we present and analyse a third model that emerges as a plausible simplification of the earlier models and that differs from them in that it does not take energy dynamics into account. Hence, this new model allows us to ask whether including energy dynamics in the Lotka–Volterra models, extended as explained earlier, is essential to the predictions from these models. Moreover, we can compare the predictions of either of these models with empirical observations on opting-out strategies in natural predator–prey systems.

By making good use of optimal control theory and game theory, we are able to determine what is best for the populations of predator and prey. Whether these solutions are also best for the individual predator and prey is not analysed in this paper and therefore remains to be determined. In our view, both types of solutions are of scientific interest in their own right as they show how predominating selection levels determine what is the best strategy. If the predicted best strategy depends critically on the selection level taken into account in the models, then this may help us to infer which selection level dominates in natural systems by comparing the predicted and empirically observed strategies.

This paper is structured as follows. In §2, we briefly review what is empirically known about opting-out (diapause) strategies in a well-investigated predator–prey system involving predator mites (Acari: Phytoseiidae) and their prey, fruit-tree red spider mites (Acari: Tetranychidae). In §3, we first discuss the Lotka–Volterra models extended to include energy dynamics of predators and prey joining or opting out of the interactions. Then, we propose and analyse a new plausible model that lacks energy dynamics. Finally, in §4, we compare the predictions from the models presented and discuss whether including energy dynamics in the models is essential to their predictions. Moreover, we compare the strategies predicted with those empirically observed in plant-inhabiting predator and prey mites.

2. Observed behaviour of the predator mites and the fruit-tree red spider mites

Our models were inspired by studies on the use of predator mites (Acari: Phytoseiidae) for biological pest control of fruit-tree red spider mites (Acari: Tetranychidae) that feed on and damage leaves of apple trees [15,16]. Winters (covering six to seven months) are usually harsh and as such endanger the survival of prey [15] and (even more so) that of predators [15,17]. Predator and prey densities in the following summer season depend on their numbers entering a state of physiological rest (the so-called diapause state) during the previous year. The decision to enter diapause promotes the survival of the individual during winter and emerges from induction by a combination of sufficiently long night lengths and low temperatures [7,8]. However, using another similar spider mite species (more amenable to experimental treatment), it was shown that the decision to enter diapause also depends on predator density during summer [10–12]. From the point of view of the prey mite, this behaviour makes intuitive sense as it faces a grim future with increasing predator densities and thus an increased risk of death: it may then do better by giving up reproduction, moving away from leaves to twigs and branches (a refuge from predation, but without food) and by entering diapause earlier than indicated by the predictors of season length (night length and temperature). However, if too many prey mites would make the same decision, then this could create a negative feedback on the predator mite population, so that, at some point in time, the prey mites would profit from the decreased predation risk by terminating their diapause and returning to the leaves. This leads us to conclude that the prey's decision to enter diapause is part of a game where the predator is the leader, and the prey needs to find an optimal response to the predator.

Another complicating factor is that an early diapause raises the demands on the energy storage of the individual prey mite, which needs to cover a longer period before terminating diapause at the beginning of the next summer season—the energy level at diapause termination will determine the reproductive capacity of the prey mite [11]. Thus, the decision to enter diapause within a year will depend on the current internal energy store of the prey mite, as this will have far-reaching consequences for winter survival and reproduction in the summer season of the next year. Given the negative feedback between predator and prey and the complexity of the decisions that prey mites are faced with, it is virtually impossible to intuitively pinpoint the most likely strategies that will emerge from natural selection.

There is less information on the diapause behaviour of the predator mites. However, the predator mites are much more flexible in entering diapause or active states, and can switch among them multiple times during the season. Physiological decision variables depend on the predator and prey densities during summer, rather than only on reliable season indicators, such as night/day length and temperature [5,6].

3. Three models of the predator–prey interactions

In the remainder of this paper, we will focus on optimal control and game-theoretical models of interactions between predatory mites and fruit-tree red spider mites. Using these methods, we will seek optimal strategies for the populations. These strategies are supposed to be a result of evolutionary processes that take place at larger temporal and spatial scales (e.g. metapopulation scale) than considered in our time-bounded and spatially unstructured models. The results that are optimal may or may not be comparable with the results observed in reality. Throughout this paper, if we talk about the decisions of predatory mites and/or prey mites, then we are referring to the decisions taking place in the long-term evolutionary process and if we talk about optimal decisions, we mean the decisions that are the result of this evolutionary process.

In our previous work, we have developed two models of intra-seasonal interactions between predator mites (Acari: Phytoseiidae) and fruit-tree red spider mites (Acari: Tetranychidae). Both models extend the Lotka–Volterra equations [2,4] and include energy variables. We compare these two models with a new model proposed in this paper, which is also an extension of the Lotka–Volterra equations, but does not include an energy variable.

3.1. Optimal control model with energy dynamics

This model was introduced and analysed in our previous article [14]. The predator is assumed to be active the entire season; the goal is to find the optimal active/diapause ratio for the prey. The fitness function for the prey models its survival capability, and is related to the number of the individuals entering diapause during the summer. Therefore, the prey mites choose a u^R,*(t) ∈ [0,1] for t ∈ [0, T], where

3.1

In (3.1), the constant T denotes the length of the season. Moreover, with the function u^R we denote the strategy for the prey, namely u^R(t), t ∈ [0,T]. The decision variable u^R(t) indicates the portion of the prey population being active at time t : u^R(t) ∈ [0,1], for t ∈ [0,T_n]. R(t) represents the prey population at time t. Accordingly, the quantity (1 – u^R(t))R(t) represents the number of the prey individuals in diapause at time t. Furthermore, the variable E^R(t) ∈ [0,1] represents the (normalized) energy that is available to an average individual within the prey population: if E^R(t) = 0, then the average individual is dead, whereas E^R(t) = 1 represents maximal energy for the average individual.

The system dynamics within each summer season is modelled as follows (here P(t) denotes the predator population at time t):

3.2

3.3

3.4

The quantities α, β, γ > 0 and m,d > 0 are given parameters and except for d can be instantiated based on the field and laboratory observations of the mites and their interactions. As parameter d is difficult to estimate, it is kept free, and the results presented in the article [14] are valid for all values of d, unless stated differently. Both the number of predators P(t) and that of prey R(t) decrease at a rate α. In equation (3.3), the number of predators P(t) increases at a rate that is proportional to the rate of predation, represented by the product of the number of actual active prey u^R(t)R(t) and the number of predators P(t) with feeding rate βγ. Whenever active, the prey population in (3.4) decreases—owing to predation—proportionally to the number of active prey and number of predators (at rate β), whereas it increases—owing to feeding and reproduction—proportionally to the number of prey and the average internal energy (with rate γ). The energy of the prey in (3.2) varies as follows: whenever active (feeding), it increases proportionally to the distance to its maximum (1 − E^R(t)) with rate d; on the other hand, whenever in diapause, it decreases proportionally to the actual average energy of the prey (with rate m), as individuals in diapause slowly use their energy.

The optimal strategy of the prey in this model, which we found by using the Hamilton–Jacobi–Bellman (HJB) approach combined with the method of singular characteristics [18–20], follows the following rules (figures 1 and 2):

• — in the beginning of the summer season, the prey can be in any state (all active, all in diapause or anything in between), whereas at the end of the summer season, all prey individuals are in diapause;

• — if all prey individuals are active in early summer, then the prey will start entering diapause at a certain point in time and the proportion of diapausing individuals increases monotonically. Similarly, if only part of the prey population is active in early summer, then all prey end up being in diapause at one point in time and stay in diapause until the next year. Yet, if all prey individuals are in diapause in early summer, then they continue to stay in diapause until the next year;

• — the time (expressed in real time) of diapause onset depends on the energy of the prey, on predator population size, and on the rate of energy utilization, but it is independent of prey population size (i.e. timing of diapause does not require quorum sensing);

• — if predators are absent from the environment, then all prey individuals are in diapause later than if the predators are present (figure 3). Empirical observations on diapause of fruit-tree red spider mites on apple trees in the field (M. W. Sabelis & W. P. J. Overmeer 1987–88, unpublished data) reveal that virtually all individuals become active in early summer and starting from a certain point in time the population enters diapause, gradually. Moreover, experimental manipulation of the predator population in the field showed that the fruit-tree red spider mites enter diapause earlier in the presence of predator mites and once in diapause they stay in diapause. However, apart from an effect of predator presence, the density of fruit-tree red spider mites also had an effect on the time at which diapause was initiated, suggesting that some form of quorum sensing (possibly via spider–mite-induced plant volatiles) takes place; and

• — if more predators are present in the environment at the beginning of the season, then the prey individuals start entering diapause earlier, but the process of entering diapause is more gradual than if less predators are present; this effectively yields the previous observation (figure 3).

Figure 1.

Scheme of possible optimal active ratio u^R for the prey (u^R(t) ∈ [0,1] for each t ∈ [0, T]). Based on the proposed dynamics and the optimization problem, we have shown irreversibility and (largely) monotonicity of the strategy profile. Note that the optimal strategies do not need to be continuous corresponding to the singular events in the outcome of the optimization problem. (Adapted from [14].) (Online version in colour.)

Figure 2.

If the number of predators increases (while all other state variables and parameters stay the same), the prey individuals begin to enter diapause earlier, but more gradually, balancing between having enough energy to survive the diapause and escaping predation. Here, u^R indicates optimal active ratio for the prey (u^R(t) ∈ [0,1] for each t ∈ [0, T]). (Adapted from [14].) (Online version in colour.)

Figure 3.

The optimal strategy for the predator is to stay active during the entire summer season. Here, u^P indicates optimal active ratio for the predator (u^P(t) ∈ [0,1] for each t ∈ [0,T]). (Adapted from [13].) (Online version in colour.)

3.2. Game-theoretical model with energy dynamics

This model was introduced and analysed in Staňková et al. [13]. It extends the model presented in §3.1 as both predators and prey can make decisions to be active or in diapause. Therefore, the summer interactions between the predator mites and the prey mites can be formulated as a game played with a finite horizon [0,T] in which the predator mites select a u^P,*(t) ∈ [0,1] for t ∈ [0,T], where

3.5

whereas the prey mites choose a u^R,*(t) ∈ [0,1] for t ∈ [0,T], where

3.6

subject to the following system dynamics:

3.7

3.8

3.9

3.10

In (3.7), a > 0 is the energy decrease rate for the predator when active, ac (with c ∈ [0, 1)) is the energy decrease rate for the predator when in diapause, e is the energy increase rate for the predator when feeding (here the energy increase is proportional to the number of active fruit-tree red spider mites that are preyed upon and to the number of active predator mites).

In (3.8), d > 0 is the energy decrease rate for the prey when active, dh (with h ∈ [0,1)) is the energy decrease rate for the prey when in diapause, f(t) is a time-dependent function characterizing the presence of nutrients for the fruit-tree red spider mites in the environment , g(R_n) ∈ [0,1] is a non-increasing function of its variable, which represents competition among individual fruit-tree red spider mites—hence f(t)g(R_n)u^R is a term representing the increase of energy in the prey owing to its active state. The number of predator mites slowly decreases with rate α > 0 and increases proportionally to their energy and number of active individuals with rate βδ where β > 0, δ > 0. The number of fruit-tree red spider mites decreases with death rate (ɛ > 0) increases proportionally to their energy and number of active individuals with rate δ > 0 and decreases proportionally to the number of active predator mites and number of active fruit-tree red spider mites with rate γ > 0. As before, E^P and E^R refer to the energy levels of the average predator and prey individual, respectively.

The fitness function for the predator (3.11) reflects the fact that all predator individuals being alive at the end of the summer season (independently of whether they are active or in diapause) have a chance to survive the winter. As in the previous model, the fitness function for the prey (3.12) reflects the fact that only the prey individuals that are in diapause at the end of the summer season have chance to survive the winter, whereas the longer in diapause, they are and the more internal energy they have, the higher chance of survival they have.

The problem was solved as a Stackelberg game with the predator as the leader and the prey as the follower. The optimal behaviour for the predator is shown in figure 3. While it is optimal for the predator to stay active during the entire summer season, the behaviour of the prey is the same as in the optimal control model introduced in §3.1.

3.3. New model: game-theoretical model without energy dynamics

The similarity in predictions from the two models introduced in §§ 3.1 and 3.2 is striking. While we assumed that the energy variable is necessary in order to model the system in question realistically enough, we decided to validate this by introducing a game-theoretical model introduced in §3.2 simplified in that it does not take energy dynamics into account. This model is a straightforward extension of the classical Lotka–Volterra model, enriched by the decision variables for the predator and prey. Moreover, it naturally emerges from the models introduced in §§3.1 and 3.2. Can such a simpler model yield similar results as the optimal control model with energy and the game-theoretical model with energy?

In this new model, we again assume that the predator mites choose u^P,*(t) ∈ [0,1] for t ∈ [0,T], so that

3.11

whereas the prey mites choose a portion of individuals that are active (versus those in diapause) u^R,*(t) ∈ [0,1] for t ∈ [0,T], where

3.12

The system dynamics appears to be a trivial extension of the Lotka–Volterra model:

3.13

and

3.14

Here, α_P > 0 is the death rate of the predator, α_R > 0 is the death rate of the prey, β_P > 0 is the population increase rate for the predator based on feeding and β_R > 0 is the population decrease rate for the prey owing to predation. Note that if all predator mites are in diapause, then their number does not change. If they are active, then they need to feed on active prey mites in order to increase their number. Similarly, if all prey mites are in diapause, then the number of predator mites does not change. If some of the predator mites are active, the difference between the first and the second term in (3.14) indicates whether their number will decrease or increase. We assume that 0 < α_P ≤ β_P, 0 < α_R ≤ β_R, and 1 ≤ P(0), R(0).

Within a summer, the goal of both predator and prey (the players) is to maximize their chances of survival [21,22], which translates to the optimization problems defined by (3.11) and (3.12), subject to the dynamical constraints (3.13) and (3.14). We assume that the game between the predator mites and the prey mites has a Stackelberg structure, i.e. we assume that the predator can impose its decision on the prey.

Remark 3.1. —

The system of predatory mites and fruit-tree red spider mites is inter-seasonal, i.e. the summer season lasting for about five months is followed by the winter season, lasting for about seven months. Therefore, equations (3.13) and (3.14) apply to a period equal to a summer season, thus there are no long-term dynamics.

However, let us analyse the equilibrium dynamics obtained for various values of u^P and u^R. The equilibrium points are {P*^,1 = R*^,1 = 0} and where the latter point is well defined for u_R, u_P ≠ 0. The Jacobian of the system (3.13) and (3.14) is

Eigenvalues of J at equilibrium point {P*^,1 = R*^,1 = 0} are α_Ru_R and α_Pu_P. Eigenvalues of J at equilibrium point are and Therefore, if u^P ≡ 0 and u^R ≡ 0, both equilibria are marginally stable (namely related to periodic trajectories over the two populations). Otherwise, both equilibria are unstable.

As derived in appendix A of this paper, the optimal strategies of the predator and prey follow the pattern depicted in figure 4. As the number of switches in the strategies depend on the initial parameters α_P, α_R, β_P, β_R, season length T and initial values P(0) and R(0), results of the numerical case studies are shown in figures 6 and 7 of appendix B. Please bear in mind that these results are just approximations of the optimal results obtained by grid-based numerical techniques. That is why we have chosen an extremely small T which provides a relatively high precision of the outcome. For details about the numerical computations, see appendix B.

Figure 4.

Optimal behaviour for the predator and prey in the game-theoretic model without energy dynamics, with u^R and u^P denoting the portion of the predator mites and the prey mites being active, respectively. Behaviour at the end of the season (on the right-hand side of the dashed line): either the switches between being active and in diapause happen exactly at the same time for both predator and prey (see appendix A for discussion on this), but the predator may also switch before the prey does. This would suggest that the predator reacts to the behaviour of the prey in reverse time, as opposed to the reaction in the real time, which we would expect. Behaviour before the end of the season (before the dashed line): the behaviour here might vary as suggested by the outcomes of numerical case studies. See appendices A and B for derivation of this result and numerical case studies. (Online version in colour.)

Contrary to the models introduced in §§3.1 and 3.2, the optimal strategies of the predator and prey mites in apple orchards are much more versatile; the predator and prey clearly react to each other's decisions. However, note that the mechanism of their behaviour is much more clear when studying the problem in reverse time (either the predator or the prey switches their behaviour at the same time or the predator reacts to the behaviour of the prey). The possible frequent switching in the strategies (dependent on parameters α_P, α_R, β_P, β_R, season length T and initial conditions P(0) and R(0)) does not match empirical observations on the diapause behaviour of predator mites and fruit-tree red spider mites.

Our analytical computations (see appendix A) yield a prediction that is counterintuitive at first: the predator may opt out of the interaction before the prey. However, this is not surprising when it is realized that our analytical computations yield predictions for what is optimal for the population (rather than the individual). Indeed, the predator population may profit from opting out because it allows the prey population to grow and hence represent later a richer food source for the predator population. This option is known as ‘the milker–killer dilemma’ in the literature [23–25].

4. Discussion

In §§3.1–3.3, we introduced three models trying to capture diapause induction and termination behaviour of predator and prey mites. The first two models seem to correspond to the laboratory and field observations very well: it is expected that the prey reacts to the presence of the predator by changing the manner and speed of its diapause induction.

In most field observations, the predator indeed stays active the entire season. However, in rare cases, it might also happen that the predator enters and leaves diapause during the season, whereas the diapause induction in the prey is irreversible. This is due to the fact that the diapause in predator mites is much more flexible than that in the prey mites. In other case studies, it was observed that the predator mites enter diapause once the prey enter diapause; subsequently, the prey might become active when the predator is in diapause, followed by the predator becoming active as well. While repeated entering diapause/active state was an outcome of the game-theoretical model without energy introduced in §3.3, the way in which the predator and prey mites are predicted to enter the active/non-active (diapause) state is too versatile and differs from our empirical expectations. We conclude that models, including energy dynamics match the field and laboratory observations much better than the model without energy dynamics [13,14]. This is a very interesting observation as the model without energy is a trivial extension of the Lotka–Volterra model and the first step towards game-theoretical models from this widely used framework. Moreover, the model without energy is a special case of both models treated in our previous work (one would derive this model from the previous models by eliminating the energy dynamics). This model represents a simpler way of modelling the predator–prey interactions and a natural question to ask is whether extending it by energy variables is really necessary. The results in this paper suggest that one needs to include the energy dynamics in this model in order to model the system of interest with more realism. However, it remains to be seen whether there are no other models without energy dynamics which would be closer to the observed behaviour. If such a model is found, we would have falsified our hypothesis that including energy dynamics is essential to the predictions from §§3.1 and 3.2 models.

The system under consideration is multi-seasonal, i.e. each summer season is finitely long and it is followed by a winter season. For this reason, long-term analysis does not yield much insight into the behaviour of our model. If we, however, assume the summer season is infinitely long, then stability analysis shows that prey diapause stabilizes the predator–prey dynamics. This result has been reported earlier as the stabilizing effect of prey refuges on predator–prey dynamics [26,27].

For all models we have proposed so far, it is still to be shown that optimal summer behaviour of the predator and prey populations, as derived in this study, is resistant against invasion by mutant strategies and robust against structural modifications, such as the inclusion of predator decisions to enter diapause or not. Ultimately, we hope to explain winter dynamics of predator mites and fruit-tree red spider mites based on optimal timing of diapause induction in summer. The use of bifurcation analysis can help determine for which parameter domains the proposed optimal strategies are evolutionarily stable.

Appendix A

A.1. Derivation of the optimal strategies for the predator and prey in the third (new) model

First, we formulate the problem of the predator and the problem of the prey via HJB equations [28]. We will then study the reaction of the prey to the behaviour of the predator and subsequently compute the optimal behaviour of both of them. We assume here that a Stackelberg game is being played in which the predator can impose its decision on the prey. In the analysis, this is equivalent to analysing what behaviour is optimal for the prey with respect to the behaviour of the predator, and consequently checking what is optimal for the prey. However, most of the behaviour obtained from this analysis coincides with the outcomes of the Stackelberg game with the prey as the leader and outcomes of the Nash game, i.e. the game in which there is no hierarchy between the players.

The analysis is carried out in reverse time, i.e. proceeding from the end of the season towards its beginning, we will study the optimal behaviour of the predator mites and the prey mites.

A.2. Characteristic system for the prey

Let us introduce a reverse time and a value function for the prey

With the HJB equation has the form

A1

The characteristic system (in reverse time) is

A2

A3

A4

A5

with transversal conditions b_P(0) = 0, b_R(0) = 0, and additional initial conditions P(0) > 0, R(0) > 0. Optimal decision can then be derived as [18–20]

with

A6

Note that the sign of does not depend on R, as R is always positive if the prey is alive. As , u^R equals 0 for some interval while we cannot yet exclude the option that Note that if the season is very short, then u^R = 0 is optimal for the prey for the entire season. While the fact that u^R(0) = 0 is independent of the behaviour of the predator, the time when the strategy of the prey should change might be dependent on u^P.

This can be seen from the following characteristic system, obtained by substituting u^R = 0 into (A 2)–(A 5) and with initial conditions P(0), R(0) ≥ 1, b_P (0) = b_R(0) = 0:

A7

A8

A9

A10

The solution of (A 7)–(A 10) is

Substituting this solution into (A 6) leads to

Remark A.1. —

(Switching surface starting at R(0) = 0) Note that in our system it is impossible for R to reach value 0. If we allowed condition R(0) = 0, then there would be a switching surface starting from R(0) = 0 as would then equal to zero. In such a situation, prey would either start in diapause or u^R ∈ (0,1). The switching surface starting at R(0) = 0 would then have the following parametrization:

Trivially, these two equations are zero for R = 0 (i.e. there exists a switching surface for R = 0). However, they are also zero for

A11

A12

Note that this solution makes sense only if u^P≠0. Moreover, we know that b_P(0) = b_R(0) = 0 and P follows the dynamics in (A 2), with P(0) = 0. From the second time derivative of we can obtain expression for u^R along this switching surface. However, we need to know the expression for u^P (in case that u^P≠0) in order to be able to get an explicit expression for this intermediate value of u^R (as its expression contains u^P and its time derivatives). There is a switching surface present in the dynamics, but only if u^P ∈ (0,1]. The expression for u^R alongside this surface is then

with

Note that ψ₂ = 0 if u^P = 0. Therefore, we can proceed to the analysis of the characteristic system for the predator and only if u^P(0) ∈ (0,1].

Let us now investigate the optimal behaviour of the predator at the end of the season, taking into account that u^R,* = 0 at the end of the season.

A.3. Characteristic system for the predator

Adopting a similar analysis as the one for the prey, we can proceed as follows: we again consider reverse time and a value function for the predator

With the HJB equation has the following form:

A13

The characteristic system is

A14

A15

A16

A17

with initial conditions P(0), R(0) ≥ 1 and transversal conditions a_P(0) = a_R(0) = 0. The optimal decision can be expressed as

with

A18

Note that the sign of is independent of P, as P has to be positive for the predator to be alive. From the initial and transversal conditions, it follows that As u^R(0) = 0, Therefore, u^P(0) = 0.

Remark A.2. —

(Switching surface starting at P(0) = 0). Note that in our system it is impossible for P to reach value 0. If we allowed condition P(0) = 0, then there would be a switching surface starting from P(0) = 0. Then, depending on the initial value of R and value of u^R(0), the predator would either start in diapause or u^P ∈ (0,1). If we allow P(0) = 0, R(0) = 0 and u^R(0) ∈ (0,1), then there might be a switching surface for the predator starting at P(0) = 0, alongside which u^P ∈ (0,1), whereas the prey would act as described in Remark A.1. In such a case, if u^P ∈ (0,1) at the beginning of the season, the switching surface for the predator would have to satisfy the following conditions:

From equations one can compute that a_P = –1 and a_R = 0. As with a_P(0) = 0 and dynamics (A 16) a_P = –1 cannot be achieved, we can conclude that the intermediate strategy and switching surface starting at τ = 0 does not exist. This, however, implies, that also the switching surface discussed in remark A.1 does not exist.

Assuming that u^P(τ) = u^R(τ) = 0 on some interval [0, τ_x), and given that where and are the times in which the predator and prey change the strategy from u^P = 0 to u^P ∈ (0,1] and u^R = 0 to u^R ∈ (0,1] at the end of the season, respectively, we find on this interval that

A19

A20

A21

A22

Therefore, for τ ∈ [0, τ_x), P(τ) = P(0), R(τ) = R(0), a_P = 0, a_R = 0, and (from (A 18))

This means that as long as u^R = 0, u^P = 0 as well. Moreover, also if u^P = 0, while u^R = 1. Therefore, while we cannot exclude the option when and are arbitrarily close to each other.

As long as u^P = 0, the solution to the system (A 7)–(A 10) equals to P(τ) = 0, b_R(τ) = 1, P(τ) = P(0), R(τ) = R(0). Consequently,

This implies that if R(0)(τα_R–1) = 0, i.e. the time when u^R = 0 changes into another strategy is equal to (and is therefore independent of R and P.). Clearly,

If u^P = 0, while u^R ∈ (0,1), the characteristic system for the predator becomes

A23

A24

A25

A26

The switching surface of the predator will become where R^s solves (A 24) and solves (A 26), given that u^R = u^R,s∈(0,1).

A.4. Finding u^R,s if u^P=0

We can use the following relation, with {·,·} denoting Jacobi brackets [19]: with h_R being the Hamiltonian of (A 1). We assume for Then

A27

and

A28

In other words, as is a switching surface for the prey. Moreover, if there is an intermediate strategy u^R = u^R,s ∈ (0,1), implies [19]. Setting (A 27) and (A 28) to only leads to the trivial solution R = 0. Therefore, there is no singular strategy for the prey under the assumption that once u^R≠0, when u^P is still equal to zero.

Note that if u^R jumps at also co-states b_P and b_R might jump, as the HJB equation will change. However, as the expression for the switching surface is trivial, such a jump does not happen.

A.5. Finding optimal strategy for the predator when u^R=1

Let us now investigate optimal behaviour for the predator once u^R = 1, if the predator plays u^P = 0. The characteristic system for the predator (A 14)–(A 17) becomes

A29

A30

A31

A32

with initial conditions This implies that a_P(τ) = 0, a_R(τ) = 0 also for P(τ) = P(0), Then

which equals to at time Note that if R(0) = 1 and α_P = β_P, then otherwise, depending on values of R(0), α_P and β_P, the difference between and might be arbitrarily small or very high.

In order to find the strategy for the predator u^P,s ∈ (0,1), we have to solve the system of characteristic equations (A 14)–(A 17) with u^R = 1, u^P = u^P,s, leading to:

A33

A34

These two expressions can be equal to zero only for a_P = –1, a_R = 0 (not that this outcome coincides with the outcome found in remark A.2). As this is clearly impossible. The conclusion is that at time the predator switches to strategy u^P = 1 immediately.

A.6. With u^R=1, u^P=0, will u^R change to another value?

If u^R = 1 and u^P = 0, i.e. for time the system of characteristics for the prey becomes

A35

A36

A37

A38

with initial conditions Note that with u^P = 0 the switching surface can be expressed as

As b_R is increasing on and R is positive, the prey does not go into diapause from u^R = 1 if u^P = 0. Solving (A 35)–(A 38) yields P(τ) = P(0), b_P = 0,

A.7. If u^R=1, u^P=1, will predator and/or prey jump to another value?

Last but not least, the situation to be examined is when the prey is active and predator is active as well, i.e. when Behaviour of the prey can again be investigated by substituting u^R = 1, u^P = 1 into the characteristic system (A 2)–(A 5) and (A 16) and (A 17):

A39

A40

A41

A42

A43

A44

with

Note also that with u^P = 1, u^R = 1 the switching surfaces become

A45

and

A46

and they are both positive. Solving (A 40)–(A 44) analytically is impossible. One option is that as long as the prey is active, the predator stays active as well, if the decrease of is faster than the decrease of Then, in reverse time, the prey would enter diapause earlier than the predator, and the entire analysis could be repeated from the situation u^P = 0, u^R = 0. However, if at any moment the decrease of becomes slower than the decrease of then the predator would enter diapause later than the prey. Numerical studies in appendix B suggest that this situation can occur as well.

A.8. The expected behaviour of the predator/prey

Once both predator and prey are in diapause, we can repeat the analysis shown above. The typical optimal behaviour for the predator and prey is depicted in figure 5. However, numerical studies in appendix B suggest that the behaviour beforehand can look quite different.

Figure 5.

Typical behaviour for the predator and prey in the model dealt with in this article. Here, u^P and u^R refer to the portion of the predator and prey population being active, respectively. For certain initial values of parameters the times of the switches in the strategy for u^P and u^R are the same, for most initial values of the parameters they differ. Note that while the behaviour at the end of the season was found analytically (the behaviour on the right-hand side of the dashed line), the rest of this behaviour might vary as suggested by the outcomes of numerical case studies (cf. appendix B). (Online version in colour.)

Appendix B: numerical computations of the optimal strategies for the predator and prey in the third model

B.1. Setting of the numerical computations

The model of the intra-seasonal interaction between predatory mites and fruit-tree red spider mites was implemented in Fortran. In this program, the time interval [0,T] was discretized into n_t subintervals (commonly 5 · T and 10 · T subintervals). Moreover, the optimal decisions u^P and u^R were searched at n_u points (higher n_u increases the precision of the outcome, while n_t has to be divisible by n_u – 1), starting from (random) initial estimates. As we consider a game with Stackelberg structure, the constrained optimization for the prey (the follower) was embedded into the constrained optimization for the predator (the leader). The ordinary differential system (3.13) and (3.14) was discretized on n_t subintervals using the fourth-order Runge–Kutta method (with constant step τ = T/n_t) and subsequently the fitness functions for the predator and prey were approximated using the trapezoidal rule with the time step τ. Two cases were considered:

• — the optimal u^P and u^R are continuous, piecewise affine functions; and

• — the optimal u^P and u^R are piecewise affine functions with possible discontinuities in the internal nodal points. For the calculations of the ith subinterval of u^P and u^R, u_p and u_r are considered to be continuous (i.e. ).

Note that the results of the numerical computations strongly depend on the number of discretization points and on the chosen structure for u^P and u^R and that the results obtained here are just an approximation of the optimal strategies.

B.2. Results of the numerical computations

In table 1, we compare different algorithms in terms of P(T). The maximization criterion of the leader is and P(0) is given, therefore P(T) indicates the outcome for the leader well. Higher P(T) with the same values n_t and n_u indicates a better outcome for the predator. In all tested cases with the same n_t and n_i, the discontinuous strategies brought better outcome to the leader than the continuous ones, which indicates that indeed discontinuous strategies are optimal in this model. For the numerical case studies, we have considered a very short season in order to improve the precision of the algorithm. Parameters α_P, β_P, α_R and β_R were set to 0.05, 0.2, 0.25 and to 0.24. The graphs comparing different outcomes are shown in figures 6 and 7.

Table 1.

Comparison of different algorithms in terms of P(T).

strategy type	T	nt	ni	P(T)
continuous	10	100	50	1.282645
continuous	10	100	100	1.285299
discontinuous	10	100	50	1.321685
discontinuous	10	100	100	1.619314

Figure 6.

Comparison of the numerical outcomes of the game on interval [0,10] with continuous strategies: n_t = 100, n_i = 50 (a,b) and n_t = 100, n_i = 100 (c,d). Here, P(0) = R(0) = 1. (Online version in colour.)

Figure 7.

Comparison of the numerical outcomes of the game on interval [0,10] with discontinuous strategies: n_t = 100, n_i = 50 (a,b) and n_t = 100, n_i = 100 (c,d). Here, P(0) = R(0) = 1. (Online version in colour.)

While the behaviour before the last switch (when both predator and prey enter diapause after being active) varies among different numerical outcomes, the behaviour at the end of the season corresponds to our analytical results.