Adaptive Bet-Hedging Revisited: Considerations of Risk and Time Horizon

Abstract

Models of adaptive bet-hedging commonly adopt insights from Kelly’s famous work on optimal gambling strategies and the financial value of information. In particular, such models seek evolutionary solutions that maximize long-term average growth rate of lineages, even in the face of highly stochastic growth trajectories. Here, we argue for extensive departures from the standard approach to better account for evolutionary contingencies. Crucially, we incorporate considerations of volatility minimization, motivated by interim extinction risk in finite populations, within a finite time horizon approach to growth maximization. We find that a game-theoretic competitive optimality approach best captures these additional constraints and derive the equilibria solutions under straightforward fitness payoff functions and extinction risks. We show that for both maximal growth and minimal time relative payoffs, the log-optimal strategy is a unique pure strategy symmetric equilibrium, invariant with evolutionary time horizon and robust to low extinction risks.

Introduction

Kelly’s work on optimal gambling strategies and the value of side information was arguably the first convincing attempt at applying concepts from information theory for analysis in a different field Kelly (1956). This work was the precursor to growth-optimal portfolio theory which has extended the basic ideas to the realm of capital markets (Cover and Thomas 2006). There has recently been a resurge of interest in employing insights from optimal gambling theory in models of adaptive bet-hedging under fluctuating environments, where close analogies between the economic and biological setting have been convincingly made apparent (Bergstrom 2014; Rivoire and Leibler 2011; Donaldson-Matasci et al. 2010).

Biological bet-hedging was originally proposed to explain the observation of un-germinated seeds of annual plants (Cohen 1966). This strategy involves the variable phenotypic expression of a single genotype, rather than a result of genetic polymorphism, although it is difficult to empirically determine whether observed phenotypic diversity in a population arises from randomization by identical genomes or from an underlying polymorphism (Seger and Brockmann 1987). Indeed, evolutionary biologists have long acknowledged that in a stochastically variable environment, natural selection is likely to favor a gene that randomizes its phenotypic expression (Bergstrom 2014). Recent work has revealed a variety of potential instances of bet-hedging populations: delayed germination in desert winter annual plants that meets postulated criteria of adaptive bet-hedging in a variable environment (Gremer and Venable 2014), bacterial persistence in the presence of antibiotics that appears to constitute an adaptation tuned to the distribution of environmental change (Kussell et al. 2005), flowering times in Lobelia inflata which point to flowering being a conservative bet-hedging strategy (Simons and Johnston 2003), or even bet-hedging as a behavioral phenotype, such as the case of nut hoarding in squirrel populations in anticipation of short or long winters (Bergstrom 2014).

Notwithstanding these empirical findings, identifying actual cases of adaptive bet-hedging in the wild remains elusive. As Seger and Brockmann (1987) have noted more than three decades ago, it is in general difficult to determine whether observed diversity of behavior in a population arises from randomization by genetically identical individuals or from genetic heterogeneity within co-located individuals optimized for different environmental conditions. Moreover, phenotypic heterogeneity can arise within genetically homogenous populations as a form of specialization in a stable environment through stochastic gene expression, positive feedback loops, or asymmetrical cell division, all processes where bet-hedging is not at play (Rubin and Doebeli 2017). These difficulties provide further impetus for constructing better and more elaborate models to test against the data.

Of particular note in classic bet-hedging models is the adoption from economic theory of asymptotic growth rate optimality as the target function for fitness maximization strategies, where growth in wealth is analogous to growth in lineage size. Indeed, since evolution proceeds by shifting gene frequencies over generations, with frequency changes being multiplicative, long-term fitness is commonly measured by geometric mean fitness across generations (Hopper 2018). At the same time, it is also widely acknowledged that long-run growth rate is not a valid measure of fitness under fluctuating environments, such as in the case of bet-hedging populations (Lande 2007).

The resulting intrinsic unpredictability has led some researchers to formulate a probabilistic perspective for natural selection that integrates various effects of uncertainty on natural selection (Yoshimura et al. 2009). The applicability of geometric mean fitness has also come into question under finite-population models, where the probability of fixation provides additional and sometimes more suitable information than the geometric mean fitness (Proulx and Day 2001), and in periodically cycling selection regimes, where evolutionary success depends on the length of the cycle and the strength of selection (Ram et al. 2018). Moreover, both gambling and bet-hedging models targeting optimal growth rate implicitly assume an infinite time horizon in formulating the geometric average, thereby ignoring the finiteness of actual horizons over which both economic and evolutionary processes ultimately act. The problem is further amplified when interim extinction risk is taken into account, especially under finite-population models. Lineage growth trajectories which are highly stochastic are at risk of large “drawdowns,” which may pull the population below some extinction threshold, despite possessing a high asymptotic growth rate. Here we aim to incorporate considerations of finite evolutionary horizons and extinction risk in the search for adaptive optimality in bet-hedging models.

Background: The Standard Model

Most adaptive bet-hedging models are largely based on the classic horse race gambling model associated with Kelly (1956), where the biological counter-part is a lineage apportioning bets on several possible environments. Assume that k horses run in a race, and let horse $X_i$ win with probability $p_i$. If horse $X_i$ wins, the odds are $o_i$ for 1. A gambler wishes to apportion his bankroll among the horses $0<f_i\le 1$, such that $\sum f_i=1$ and participate in indefinitely repeated races $n\to \infty$. How to best apportion the bankroll each time? In this setting, wealth is a discrete-time stochastic process over n periods,

$$\begin{array}{c}\hfill W_n=\prod _{i=1}^n{W}_i(X)\end{array}$$

where $W(X)=f(X)O(X)$ is the random factor by which the gambler’s wealth is multiplied when horse X wins. More explicitly,

$$\begin{array}{c}\hfill W_n(f)=\prod _{i=1}^k{(f_i{o}_i)}^{H_i},\text{where,}H\sim \mathrm{Multinomial}(n,k,[p_1,\dots ,p_k]).\end{array}$$

Kelly’s first insight was that choosing to simply maximize expected wealth (for any time horizon n) gives ${\arg \max }_{f}E[W_n(f)]=1$, with the implication that one bets everything on a single horse (the one with the highest $p_i$) and a consequent chance of total ruin once that horse loses a race. Therefore, Kelly proposed maximizing the asymptotic growth rate [the rigorous justification provided by Breiman (1961)]. By the law of large numbers random wealth may be expressed as,

$$\begin{array}{c}\hfill W_n(f)\doteq 2^{nE[logW(X)]}\end{array}$$

where,

$$\begin{array}{c}\hfill E[logW(X)]=\sum _{i=1}^k{p}_{i}log{f}_i{o}_i\end{array}$$

is the asymptotic exponential growth rate. If the gambler stakes his entire wealth each time, i.e., $\sum f_i=1$, then

$$\begin{array}{c}\hfill E[logW(X)]=\sum _{i=1}^k{p}_{i}log{o}_i-H(p)-D(p||f)\end{array}$$

is maximized (convex nonlinear optimization) at “proportional gambling” $f=p$ where $D(p\Vert f)$ is minimized, without regard the actual odds provided by the bookie.

Indeed, the notion of proportional gambling, made famous by Kelly’s treatment, has found its way into classic models of diversified bet-hedging. In such models often assumed that “appropriate phenotypes are produced in proportion to the likelihood of each environment” (Hopper 2018) and that consequently “the classical bet-hedging prediction [is] that the optimum probability for employing a strategy is approximately equal to the probability that the strategy will be useful” (King and Masel 2007). Here we follow recent approaches that extend the standard model to non-lethal environments via a full-fitness matrix, such that this notion is no longer directly applicable.

Breiman (1961) was first to show that the Kelly solution is optimal in two convincing ways: [a] that given a Kelly strategy ${\varphi }^{\ast }$ and any other “essentially different” strategy $\varphi$ (not necessarily a fixed fractional betting strategy),

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}\frac{W_n({\varphi }^{\ast })}{W_n(\varphi )}=\infty a.s\end{array}$$

and [b] that it minimizes the expected time to reach asymptotically large wealth goals. Moreover, this strategy is myopic in the sense that at each iteration of the race one only needs to consider the presently given parameters (Hakansson 1971). However, Kelly strategies may also yield tremendous drawdowns a problem widely recognized in the gambling community, such that optimal Kelly is often viewed as “too risky”; in practice gamblers and investors use “fractional Kelly” which deviates from the optimal solution but reduces the effective variance of the stochastic growth (Fig. 1). In the biological framework, this can lead to abrupt extinction events in finite (especially small) populations with highly stochastic lineage growth trajectories. A further complication is that the underlying probability distributions are merely estimated from past data and model assumptions, leading often to over-betting and increased risk (MacLean et al. 2011).

Fig. 1.

The asymptotic exponential growth rate and its (finite-horizon) variance for either the two-horse racing model or the classic bet-hedging model (two environments). Note that the strategy (f on the x-axis) that maximizes the growth rate is far from the locus of minimal variance (Color figure online)

In this work, we extend the existing models to incorporate both interim extinction risk and finite evolutionary time horizons within a bet-hedging framework. This requires re-conceptualizing geometric mean fitness for such highly stochastic growth scenarios. We ultimately derive fitness functions that better account for such conditions where the fluctuating environment is strongly coupled to both long and short-term growth and locate optimal stable equilibria.

Methods

The Full-Fitness Matrix Model

We assume environments are i.i.d random events across generations, multinomially distributed (with some results generalized to non-identically distributed environments). Individuals within lineages have a static full-fitness matrix $[O_{\mathit{ij}}]$ in which non-lethal environments have low but generally nonzero fitness (Donaldson-Matasci et al. 2010; Rivoire and Leibler 2011). We adopt a finite-population model where lineages start off with some initial population size $W_0$, implicitly assumed higher than some bet-hedging evolutionary threshold (King and Masel 2007). Lineages then evolve strategies to randomize individual phenotypes toward maximizing growth across finite horizons in the face of interim extinction threats. More formally, with k environments and phenotypes,

1

the general model of lineage growth trajectory across n generations under strategy f is a random process,

$$\begin{array}{c}\hfill W_n=\prod _{i=1}^k(\sum _{j=1}^k{f}_j{o}_{\mathit{ij}}{)}^{H_i}\end{array}$$ 2

where,

$$\begin{array}{c}\hfill H\sim \mathrm{Multinomial}(n,k,[p_1,\dots ,p_k])\end{array}$$

with off-diagonal values reflecting the lower fitness for non-matching environments,

$$\begin{array}{c}\hfill o_{\mathit{ii}}>o_{\mathit{ij}}\ge 0\text{and}{o}_{\mathit{ii}}>1\end{array}$$

and where all individuals in a lineage are bet-hedging,

$$\begin{array}{c}\hfill \sum _{i=1}^k{f}_i=1.\end{array}$$

And finally, using a straightforward formulation of the growth rate, $W_n^{1/n}$, a random variable for any finite horizon.

We first derive the asymptotic growth rate optimal “Kelly” solution for this setting ($f^{\mathrm{Kelly}}$) with a corresponding bet-hedging region of the environment simplex (“Appendix A”). Relaxing the assumption of i.i.d environments, we derive the static Kelly solution for the case of nonstationary environments—where environments are independent but not identically distributed across generations (“Appendix B”). While under nonstationary environments an optimal growth rate is reached with a dynamic myopic strategy, we focus here on a static strategy since adaptations effectively stabilize across time spans much higher than single generations, such that from evolutionary considerations dynamic strategies are not likely to emerge. Alternative models of fluctuating environments such as Markov chains with underlying switching probabilities (e.g., Li et al. 2017) are not pursued here and left for future work. Finally, we identify a “reference” strategy that admits deterministic growth trajectories, namely the “Dutch book” solution (where the variance of the finite-time growth rate is zero) and characterize the consequent loss of growth incurred by exchanging opportunity for certainty (Appendix C).

Relative Fitness Payoff Function

We now wish to go beyond the standard approach of targeting the optimization of the asymptotic growth rate as undertaken in the previous section—to incorporate finite evolutionary horizons and extinction risk considerations. For the sake of simplicity, we confine our model here to the case of $k=2$ environments and phenotypes (so that the two environments occur with probability p and $1-p$). To motivate the shift to a finite-horizon framework, we first highlight an important property of our stochastic growth model, known also in portfolio theory (Markowitz 2006). We prove that for any two essentially different strategies, the maximal time $n_0$ one lineage “dominates” the other is finite for every realization of lineage trajectory pair (“Appendix D”). The exponentially diminishing histogram of last intersection times of given two growth strategies in Fig. 2b (with a single instance of two trajectories for illustration in Fig. 2a) demonstrates this phenomenon.

Fig. 2.

Two perspectives on growth trajectories $W_n$ and their last “intersection” point for two lineages with different strategies (a log-optimal Kelly and a suboptimal “fractional Kelly”). a The optimal Kelly strategy eventually departs from a suboptimal strategy without further intersections, “the last intersection.” | b A histogram demonstrating that the last intersection of the trajectories of any two growth strategies occurs at some finite time (here, somewhat above 500 generations) (Color figure online)

The sustained variance and high skewness of the growth rate distribution under any finite horizon necessitates a comparative approach in formulating a fitness payoff function (in fact, the growth rate is asymptotically log-normal as shown in “Appendix E”). Consider a relative fitness measure for two different lineage strategies f and g: The probability that a random trajectory of a lineage with strategy f exceeds the random trajectory of a lineage with strategy g (given time horizon n),

$$\begin{array}{c}\hfill h(f,g)=P(W_n(f)>W_n(g))\end{array}$$ 3

with an induced relation defined by,

$$\begin{array}{c}\hfill W_n(f)\ge W_n(g):P(W_n(f)>W_n(g))\ge P(W_n(f)<W_n(g)).\end{array}$$ 4

We may interpret this probabilistic relation between two strategies as relative fitness. Note that since realizations of $W_n(f)$ and $W_n(g)$ stem from the same underlying stochastic environmental sequence, they will generally be highly correlated (with the corresponding logarithmic growth rates in fact perfectly correlated, as shown in “Appendix F”). Consequently, the probability in Eq. (3) must be derived from their joint distribution rather than simply from marginal distributions. Figure 3 depicts realizations of the log growth rates of $W_n(f)$ and $W_n(g)$ as histogram distributions for some choice of strategies f and g, and some finite evolutionary horizon n. Asymptotically with time horizon n, such distributions approach normality with variance going to zero (“Appendix E”).

Fig. 3.

The distributions of log growth rates for two competing strategies here illustrated as histograms (the x-axis is the log growth rate). In this simple case shown, the strategy generating the higher growth rates right distribution (blue) is evidently of higher fitness than the one generating the left distribution (green). For clarity, only the marginal distributions are shown (the complete picture is in the joint distribution, due to the correlation between any two growth trajectories that share the same fluctuating environment regime) (Color figure online)

A few properties of the order induced by this relation are worth highlighting. [a] it is a complete order since any two $W_n$ are comparable under the relation, [b] it is transitive for any n and consequently a pre-order, and [c] its maximal element is $W_n^{\ast }(f^{\mathrm{Kelly}})$, such that both the order induced by $E[log{W}_n(f)]$ and the order induced by the payoff $P(W_n(f)>W_n(g))$ form complete preorders and have the same maximal element (“Appendix G”). Despite these beneficial properties, given any “vanilla” strategy g and time horizon n, the strategy that maximizes the payoff function,

$$\begin{array}{c}\hfill {\arg \max }_{f}P(W_n(f)>W_n(g))\end{array}$$

will vary as a function of g and n (demonstrated by counterexamples), and in particular will not necessarily be $f^{\mathrm{Kelly}}$. This implies that a wildtype lineage with strategy g different from $f^{\mathrm{Kelly}}$ will eventually be overtaken by some mutant invasive lineage with a strategy that maximizes this payoff function, a process that may potentially remain in recurrent flux, with invasive lineages replacing a wildtype lineage.

Competitive Optimality with Risk

To see whether evolutionary stable optima may also emerge, we develop a game-theoretic approach. Players are lineages with particular bet-hedging strategies and random initial population size. Lineages interact by competing over a common niche subject to the same environmental fluctuations. This setup is in some contrast to more standard evolutionary game theory settings, where agents are organisms rather than lineages and where the notion of an iterated strategy is prominent, but maintains the central aspect of interactions formalized in a payoff function (e.g., Stollmeier and Nagler 2018). A lineage survives the competitive encounter by avoiding extinction (defined in what follows) while exceeding its opponent in size over a given time horizon. This outcome is determined by a game-theoretic deterministic payoff function, modified from Eq. (3) to incorporate an extinction threshold and randomized initial lineage size. Ultimately, we are searching for Nash equilibria.

This approach is motivated by the classic work on time-invariant game-theoretic competitive optimality, within the scope of growth-optimal portfolio theory (Bell and Cover 1980, 1988). Bell and Cover consider a competitive setting for a stock portfolio model under any finite number of investment periods and prove that for any relative wealth payoff $E[\varphi (U{W}_1/V{W}_2)]$ and portfolio wealth $W_1$ and $W_2$, there are conditions on the function $\varphi$ such that the log-optimal Kelly portfolio is a solution to the game, given initial randomizations U and V (independent and of equal expectation). In particular, $\varphi (x)={\chi }_{[1,\infty )}(x)$ results in the payoff $\mathbb{P}(U{W}_1\le V{W}_2)$ with the log-optimal portfolio as a game-theoretic solution, given some initial fair randomizations. This additional fair randomization reduces the effect of small differences in end wealth, thus avoiding unwanted cases where the optimal strategy is beat by a small amount most of the time (Cover and Thomas 2006).

The Payoff Function in a Game-Theoretic Setting

For any time horizon n and extinction threshold d, we define a (deterministic) payoff function: The probability that a random trajectory of a lineage with strategy f exceeds the random trajectory of a lineage with strategy g without first going extinct (given time horizon n),

$$\begin{array}{cc}& M_n(f,g)=P(u_0{W}_n(f)>v_0{W}_n(g)|\text{extinction level}d):=\hfill \\ \hfill & P(u_0{W}_n(f)>v_0{W}_n(g)\wedge W_i,V_i>d,i:1,\dots ,n)+\hfill \\ \hfill & P(u_0{W}_i(f)>d,i:1,\dots ,n\wedge v_0{W}_i(g)\le d\text{some}i)\hfill \end{array}$$ 5

with initial population size independent randomizations $u_0$ and $v_0$, independent and of same mean but possibly of a different distribution class.

This payoff function induces a symmetric discrete-valued non-constant-sum game setting, although it is conceptually “zero-sum” $M_n(f,g)+M_n(g,f)<1$ (“Appendix H”). Crucially, our payoff matrix is finite since it reflects the finitely many strategies possible in a finite-population model—there can only be N different sized partitions of a population of size N in betting on two environments (under $k=2$ environments and phenotypes). A low-resolution toy-model instance of the payoff matrix is depicted in Fig. 4.

Fig. 4.

An example of a ($2\times 2$ strategies) evolutionary payoff matrix for a game of two lineages, with primary lineage payoff in bold (Color figure online)

Our goal would be to identify pure strategy Nash equilibria reflecting the evolutionary solutions to competitive bet-hedging. In particular, we would like to explore the conditions under which a bet-hedging setting admits a symmetric equilibrium and whether it is unique. In “Appendix I,” we prove that for an infinite-size payoff matrix (i.e., continuous strategies) the log-optimal strategy is the solution to this game, invariant with the choice of time horizon. Moreover, any finite matrix representing the N strategies possible for a lineage of finite size N necessarily also admits a solution, as illustrated in Fig. 5. This solution is the strategy closest to the log-optimal strategy under the finite resolution framework, such that it converges to it asymptotically with N (“Appendix L”). Finally, under a nonstationary environment model the log-optimal strategy again emerges as the equilibrium static strategy—even given short time horizons (“Appendix M”).

Fig. 5.

An example of payoff matrix simulations (a and c darker cells represent higher probabilities) and the resulting Nash equilibria in the maximal element matrices (b and d blue cells for column-maximal primary player payoffs, red for row-maximal opponent payoffs and yellow for symmetric Nash equilibria). Only portions of the matrices around the equilibrium are shown. a+b Low-resolution matrices corresponding to a small population with limited strategies | c+d Higher resolution matrices, which correspond to a larger population and subsequently higher number of strategies (multiple maxima in some adjacent cells of panel D is an effect due to a combination of using finite runs in the simulation of the payoff function, such that computed probabilities are rational values, along with high resolution in the range of strategies). The model uses a $2\times 2$ fitness matrix: $[o_{11}=3.0;o_{12}=0.2;o_{22}=1.8;o_{21}=0.1]$ with $p=0.594$ and a resulting log-optimal strategy $f=0.60$) (Color figure online)

The effect of lineage-size extinction thresholds on actual rates of extinction of random growth trajectories is illustrated in Fig. 6a. As would be expected, higher thresholds of extinction correspond to higher probabilities of extinction, with extinction rates that converge quickly to asymptotic values (“Appendix N”). Numerical simulations indicate that when incorporating low extinction thresholds that result in low extinction rates, the symmetric Nash equilibrium remains stable at the log-optimal strategy. Higher thresholds may result in a number of scenarios: a shift of the symmetric equilibrium away from the log-optimal solution, complete lack of equilibrium solution, or the emergence of multiple symmetric equilibria; in conjunction, multiple pairs of off-diagonal equilibria may appear (see Fig. 6b for one such scenario).

Fig. 6.

a The accumulative probability of extinction for log-optimal (Kelly) strategies for different lineage-size extinction thresholds. | b An instance of the maximal element matrix of a payoff matrix (a portion around the solutions) resulting from introducing an extinction threshold: two off-diagonal Nash equilibria (black) along with a symmetric Nash equilibrium shifted from the log-optimal strategy (yellow). The simulations use a $2\times 2$ fitness matrix: $[o_{11}=2.5;o_{12}=0.2;o_{22}=1.4;o_{21}=0.1]$ with $p=0.75$, and with $n=60$ in the simulation of panel B (Color figure online)

Minimum Time to Reach a Population Threshold Size

To gain further perspective on optimal strategies under highly stochastic growth, we consider evolutionary competition between lineages, where survival is determined by reaching a certain threshold of lineage size in minimal time [e.g., for $K-$selected species, see Reznick et al. (2002)]. In effect, the lineage with growth characteristics that minimize the time to reach a certain population size threshold “wins,” a setting with potential relevance in the context of competitively colonizing a limited niche, as in range expansion scenarios [see Villa Martin et al. (2019) for a bet-hedging population expanding into an unoccupied space]. We follow the classic results of Breiman 1961 on the log-optimal portfolio as the optimal strategy minimizing the expected time to reach an asymptotic target wealth, but instead of an infinite target we base the fitness payoff function on finite targets. Initial insight into the effect of strategy choice on the consequent distributions of minimal time (Fig. 7a) is provided by comparing their expectation, where the optimality of Kelly is already apparent (Fig. 7b).

Fig. 7.

a An example of the marginal distributions of the minimal time to reach a certain population size threshold, for two strategies | b A simulation of $(E[T(g,c)]-E[T(f^{\ast },c)])$ for several target values c, under the range of vanilla strategies g corresponding to positive asymptotic growth rates, where $f^{\ast }=0.8$ (Color figure online)

Instead of considering expectations of (highly correlated) minimal time distributions, we devise a more informative fitness payoff function based on the joint distribution. Crucially, this payoff will naturally be amenable to a game-theoretic approach, in line with the type of analysis in the previous section with payoff $M_n(f,g)$. As before, we condition the probability on avoiding an extinction threshold. The payoff captures the probability that a trajectory following strategy f reaches threshold c before a trajectory following strategy g, conditioned on avoiding an extinction threshold d. If both trajectories reach c at the same time (since time is in discrete generations), then the one which overshoots with a greater margin above c ‘wins’. Denote by T(f, c) the minimal time distribution given strategy f and target lineage size c,

$$\begin{array}{c}\hfill M_c(f,g)=P(T(f,c)<T(g,c)|\text{extinction level}d).\end{array}$$

More precisely, we denote new trajectories ${\{W_k^E\}}_{k=1}^n$ by

$$\begin{array}{c}\hfill W_0^E\in U[a,b],\end{array}$$

and for all $k=0,\dots ,n-1$

$$\begin{array}{c}\hfill W_{k+1}^E=\left\{\begin{array}{c}{W}_k^E{\overline{o}}_1{(f)}^{x_{k+1}}{\overline{o}}_2{(f)}^{1-x_{k+1}},\text{if}{W}_k^E\ge E\hfill \\ 0,\text{if}{W}_k^E<E.\hfill \end{array}\right)\end{array}$$

We denote also by

$$\begin{array}{c}\hfill T(f,c):=min\{n:W_n^E(f)\ge c\}\end{array}$$

the first time when the trajectory ${\{W_k^E\}}_{k=0}^n$ cut the threshold c. $T(f,C)=\infty$ if and only if this trajectory does not cut the threshold.

Then the payoff matrix $M_c(f,g)$ is defined by

$$\begin{array}{cc}\hfill M_c(f,g):=& \mathbb{P}(T(f,C)<T(g,C)|\text{given that at least one of them is finite})\hfill \\ \hfill & +\mathbb{P}(T(f,C)=T(g,C),W_{T(f,C)}^E>V_{T(g,C)}^E|\hfill \\ \hfill & \text{given that at least one of them is finite}).\hfill \end{array}$$ 6

We then identify pure strategy Nash equilibria reflecting the evolutionary solutions with the new relative payoff $M_c(f,g)$. In “Appendix J,” we prove that again Kelly is the solution to the game, invariant to the evolutionary “choice” of target population size c and that under a nonstationary environment regime Kelly emerges as the static equilibrium strategy. Finally, we highlight a deep mathematical link of this probabilistic perspective for minimal time optimality to the competitive optimality setting with payoff $M_n(f,g)$. Formally, $M_c(f,g)$ can be rewritten as a convex linear combination of $M_n(f,g)$: $M_c(f,g)={\sum }_{n=0}^{\infty }P(W_0{W}_n(f)>V_0{W}_n(g),T(f,c)=n)$ (see “Appendix J” for more details).

Discussion

In this work, we provide further support for the robustness of the expected log criterion as an optimality solution for biological bet-hedging. We develop a game-theoretic framework inherently invariant to the span of evolutionary horizons while incorporating considerations of interim extinction risk and use multiple optimality criteria to strengthen our results. This approach goes beyond standard models of bet-hedging, which focus on indefinite “long-term” growth rates and that ignore accounting for interim risk. Previous work generally upholds that “phenotypes with the greatest long-term average growth rate will dominate the entire population” as “the basic principle” used in optimization (Yoshimura and Jansen 1996), or that a proxy for the likely outcome of evolution is “to think of organisms as maximizing the long-term growth rate of their lineage” (Donaldson-Matasci et al. 2010).

Nevertheless, some authors have recently acknowledged the importance of accounting for finite time horizons. For instance, Rivoire and Leibler (2011) note in passing that in their model “the growth rate emerges as a unique measure of fitness when considering the long-term limit $T\to \infty$, but, if considering a finite ‘horizon’ there may be a different strategy that outperforms [it].” Indeed, as some evolutionists have argued, short-term fitness measures are also needed to achieve a full understanding of how evolution works in variable environments, as geometric mean fitness concerns the long-run evolutionary outcome (Okasha 2018). Moreover, long-term fitness metrics are typically formulated without regard to transient short-term population dynamics, in which lineages might come close to extinction. Under more inclusive models with extinction, selection in a fluctuating environment can also favor bet-hedging strategies that ultimately increase the risk of extinction (Libby and Ratcliff 2019). Given such considerations, the benefit of explicitly incorporating extinction considerations in stochastic growth models is clearly evident.

We have opted to focus on symmetric Nash equilibria rather than evolutionary stable strategies (ESS), which are strategies that cannot be beaten if the fraction of the rival invading mutants in the population is sufficiently small and are generally invoked in settings with iterative match-ups between individuals rather than lineages (Smith and Price 1973). Since the payoff in our game-theoretic setting pits one lineage against another (two different strategies), there is no explicit sense of invading mutants [but see Olofsson et al. (2009) for an ESS approach to bet-hedging]. Moreover, some of the classic aspects of Nash’s theorem do not directly apply within our setting. The theorem states that for every two-person zero-sum game with finitely many strategies, there exists a mixed strategy that solves the game (Nash 1951). While our framework is indeed “two-person”, it is not zero-sum and has finitely many strategies. Crucially, since an implicit goal of theoretical work such as ours may be toward predicting which strategies are likely to evolve, we focus on pure strategies rather than mixed ones, where the uniqueness of the equilibrium solution emerges as especially beneficial (echoing the classic approach of growth rate log-optimality where there is always a unique solution due to convexity).

We are not the first to attempt to model the expected minimal time to reach a finite asymptotic target, an extension of the seminal result of Breiman (1961) on properties of the log-optimal portfolio. Aucamp (1977) derived the first such analysis, given some basic assumptions that concern reaching a wealth target exactly vs. “overshooting” it. More recently, Kardaras and Platen (2010) find that in a continuous time or asset price model where a finite target can be exactly reached with no overshooting, the Kelly solution is still optimal; in a discrete time model Kelly is only approximately optimal, but if “time rebates” are introduced (to compensate overshooting the goal in the last investment period) it becomes exactly optimal. While these results on the expectation of the time distribution are in line with our analysis of stochastic lineage growth optimality, we obtain an even stronger result: Given finite population size targets, the log-optimal strategy emerges as a Nash equilibrium under a payoff function based on the joint distribution of minimal time trajectories.

Interestingly, Kelly (1956) has anticipated the application of his ideas in biological bet-hedging, writing “Although the model adopted here is drawn from the real-life situation of gambling, it is possible that it could apply to certain other economic situations...the essential requirements for the validity of the theory are the possibility of reinvestment of profits and the ability to control or vary the amount of money invested or bet in different categories.” It does not require a leap of the imagination to notice analogies of “economic situations” to evolutionary strategies, of “reinvestment of profits” to biological reproduction and growth, and of the “control” of invested money to evolved adaptive optimality. Of course, it is best appreciated with Shannon’s famous “bandwagon” warning in mind, cautioning over hasty attempts to apply insights from information theory to other fields (Shannon 1956).

Other Approaches to Optimization Under Finite Horizon and Risk

A seemingly straightforward way of introducing finite (albeit still arbitrary) horizons into optimization settings is by considering the expectation of a finite-horizon growth rate. This is the approach adopted in some recent stock portfolio models for finite horizons (Vince and Zhu 2013; Morgan 2015). Within our formalism from Eq. (2), this amounts to finding,

$$\begin{array}{c}\hfill {\arg \max }_{f}E[W_n{(f)}^{\frac{1}{n}}]={\arg \max }_f(\sum _{i=1}^k(\sum _{j=1}^k{f}_j{o}_{\mathit{ij}}{)}^{\frac{1}{n}}{p}_i{)}^n.\end{array}$$

However, this implicitly assumes some arbitrary utility function, in this case the n-th root, the maximization of which requiring some justification. In contrast, Kelly’s focus on ${\arg \max }_{f}E[log{W}_n]$ while implicitly assumes logarithmic utility, is equivalent the limit of the above expression, and leads to desired optimality properties as famously laid out by Breiman (1961).

A more convincing approach to maximizing wealth with risk management over finite horizons was proposed in Rujeerapaiboon et al. (2015) for portfolio construction. The authors consider the optimization of a minimum bound for finite-horizon growth,

$$\begin{array}{c}\hfill {\arg \max }_f\{{\arg \max }_c\mathbb{P}(\frac{1}{n}log{W}_n\ge c)\ge 1-\epsilon \}\end{array}$$

with a degree of freedom corresponding roughly to a risk aversion or a choice of certainty parameter.

The expression above allows deriving the portfolio giving the highest minimum bound for wealth for any level of certainty $\epsilon$. While choosing a particular horizon n and a risk aversion parameter is perfectly sensible in an investment setting, the translation to the biological framework is problematic: what would be evolution’s risk aversion in this setting? Or the appropriate time horizon for optimization? Any choice of these two parameters would inescapably be arbitrary in nature. In an alternative approach Rujeerapaiboon et al. (2018) reformulate the Kelly gambling setting in terms of the Conservative Expected Value (CEV), a risk-averse expectation for highly skewed distributions. This amounts essentially to devising a systematic way of constructing fractional Kelly strategies such that it is strongly coupled with the infimum of the finite-horizon growth rate. Here again, there is an implicit arbitrariness in the choice of horizon length if applied in the context of an evolutionary framework, which we seek to avoid.

Other authors have focused on incorporating risk to the standard Kelly gambling setting with an infinite time horizon. For instance, Busseti et al. (2016) develop a systematic way to trade-off growth rate and drawdown risk by formulating a risk-constrained Kelly gambling problem within the standard setting of growth rate maximization under asymptotic horizons. The additional risk constraint limits the probability of a drawdown to a specified level. Nevertheless, for our purposes, percentage drawdown is arguably not a natural metric for representing lineage extinction risks, as compared with explicit extinction thresholds, especially in scenarios of competing finite-size populations (Ashby et al. 2017). Still other approaches may seek to target risk minimization as a primary criterion. In an approach akin to our Dutch book analysis, Wolf et al. (2005) minimize the growth rate variance and consequently the probability of extinction due to “unlucky” environmental trajectories. However, this is at the inevitable expense of achieving high stochastic growth rates, a vital aspect of evolutionary fitness.

Game-Theoretic Competitive Optimality of Bell and Cover

The results presented here can also be seen as both a special case and an extension of the classic results of Bell and Cover (1980, 1988). There are several important distinctions: [a] their setting is formulated for continuous random variables whereas our environments are discrete events, [b] their payoff implies a zero-sum game whereas our game is nonzero-sum (more accurately, non-constant-sum) due to the effect of extinctions, and [c] their payoff function is a straightforward probability while our payoff is effectively a conditional probability (includes considerations of extinction risk). Moreover, implicit in Bell and Cover’s setting is an infinitely sized payoff matrix, whereas our payoff matrix is finite since it reflects a finite number of strategies possible in a finite population. These distinctions have enabled us to show that, at least given the particular payoff function and discrete framework, the emerging symmetric Nash equilibrium is in fact a strict and unique one.

Some authors have generalized or utilized other aspects of the classic competitive optimality results. Most recently, Garivaltis (2019) has shown that discrete-time results of Bell and Cover (1988) hold equally well for continuous-time rebalanced portfolios in a competitive setting between two investors, each aiming to maximize the expected ratio of one’s own wealth to the other. In an original use of evolutionary ideas in finance, Lo et al. (2017) and Orr (2017) consider a payoff function capturing relative wealth of two competing investors each with some set initial wealth, focusing on finite-period analysis. They analyze optimal strategies of a primary player against a given “vanilla” strategy, a framework consistent with our initial relative payoff non-game-theoretic setting. They find that the particular vanilla strategy chosen plays an important role in the optimal allocation, in conjunction with initial wealth of both players.

Finally, our game-theoretic analysis may hint at a solution to a “coincidence” pointed out in Bell and Cover (1980). They were left perplexed as to why competitive optimality for a finite horizon turned out, by “coincidence,” to have the same solution (namely, Kelly) as in the growth-optimal portfolio: “Finally, it is tantalizing that $b^{\ast }$ arises as the solution to such dissimilar problems [...] The underlying for this coincidence will be investigated.” Their follow-up 1988 paper suggests a “possible reason for the robustness of log-optimal portfolios” or why “log-optimal portfolios behave well in the competitive investment game”: namely that the wealth generated from any portfolio is always within “fair reach” of the wealth from the log-optimal portfolio. Indeed, the Kuhn–Tucker conditions and the consequent bound on the wealth ratio (Cover and Thomas 2006, Theorem 16.2.2) already imply that game-theoretic optimality is the driving force behind the asymptotic dominance. Fair randomization of initial wealth then leads to the game-theoretic solution for any increasing function of the wealth ratio. Our investigation of the payoff matrix suggests another perspective to this “coincidence.” Asymptotically with horizon n, the payoff matrix becomes maximally “contrasted,” with off-diagonal cells converging to probabilities of 0 or 1 (except those on “fault lines”), such that the Nash equilibrium emerges naturally. In effect, the “saddle-point” equilibrium, which has been established as invariant with n, asymptotically attains maximum curvature (“Appendix K”).

Conclusion

In this work, we have argued that under fluctuating environments and trait randomization geometric mean fitness should also encompass considerations of stochastic growth and extinction risk under finite evolutionary horizons. We show that for both the relative maximal growth payoff and the relative minimal time payoff there is a unique pure strategy symmetric equilibrium, which is invariant with evolutionary time horizon and robust to low extinction risk. Coinciding with the classic bet-hedging modeling approach, this is the Kelly log-optimal strategy. With higher thresholds of extinction, the equilibrium may shift away from Kelly and possibly branch out to multiple equilibria. Future work will be required to generalize the model to competitive optimality payoffs beyond pairwise lineages, Markovian environmental sequential transitions, random fitness matrices, and to more precisely capture the effect of high extinction thresholds on the optimal evolutionary solutions.

Appendix A: The Kelly Solution to the Full-Fitness Matrix Model

In this section, we derive the Kelly (log-optimal) solution for the full-fitness matrix model.

The case $k=2$: We have

$$\begin{array}{c}\hfill W_n(f)={(o_{11}f+o_{12}(1-f))}^H{(o_{21}f+o_{22}(1-f))}^{n-H}\end{array}$$

where $H\sim \mathrm{Binomial}(n,p)$. The Kelly solution is then defined by

$$\begin{array}{c}\hfill f^{\mathrm{Kelly}}:={\arg \max }_{f\in [0,1]}G(f)\end{array}$$

where $G(f):={lim}_{n\to \infty }{W}_n^{\frac{1}{n}}(f)$.

By denoting ${\overline{o}}_1(f):=o_{11}f+o_{12}(1-f),{\overline{o}}_2(f):=o_{21}f+o_{22}(1-f)$, we have

$$\begin{array}{cc}\hfill \underset{n\to \infty }{lim}\frac{1}{n}log{W}_n(f)=& \underset{n\to \infty }{lim}(\frac{H}{n}log{\overline{o}}_1(f)+\left(1,-,\frac{H}{n}\right)log{\overline{o}}_2(f))\hfill \\ \hfill =& plog{\overline{o}}_1(f)+(1-p)log{\overline{o}}_2(f).\hfill \end{array}$$

Therefore, by directed calculations, we obtain the Kelly solution which is dependent on p

$$\begin{array}{c}\hfill f^{\mathrm{Kelly}}(p)=\left\{\begin{array}{cc}0,\hfill & \text{if}p\in [0,p_{-}]\hfill \\ \frac{(1-p)o_{12}}{o_{12}-o_{11}}+\frac{p{o}_{22}}{o_{22}-o_{21}},\hfill & \text{if}p\in [p_{-},p_{+}]\hfill \\ 1,\hfill & \text{if}p\in [p_{+},1],\hfill \end{array}\right)\end{array}$$

where $p_{-}=\frac{o_{12}(o_{22}-o_{21})}{\mathrm{\Delta }}$ and $p_{+}=\frac{o_{11}(o_{22}-o_{21})}{\mathrm{\Delta }}$, and the corresponding optimal value is

$$\begin{array}{c}\hfill G(f^{\mathrm{Kelly}})=\left\{\begin{array}{cc}{o}_{12}^p{o}_{22}^{1-p},\hfill & \text{if}p\in [0,p_{-}]\hfill \\ (\frac{p(o_{11}{o}_{22}-o_{12}{o}_{21})}{o_{22}-o_{21}}{)}^p(\frac{(1-p)(o_{11}{o}_{22}-o_{12}{o}_{21})}{o_{11}-o_{12}}{)}^{1-p},\hfill & \text{if}p\in [p_{-},p_{+}]\hfill \\ o_{11}^p{o}_{21}^{1-p},\hfill & \text{if}p\in [p_{+},1].\hfill \end{array}\right)\end{array}$$

The case general k: By directed calculations, we obtain

$$\begin{array}{c}\hfill G(\mathbf{f})=\prod _{i=1}^k{\overline{o}}_i{(\mathbf{f})}^{p_i}\end{array}$$

where ${\overline{o}}_i(\mathbf{f}):={\sum }_{j=1}^k{o}_{\mathit{ij}}{f}_j$. This implies that for each $\mathbf{p}\in {\mathrm{\Delta }}_{k-1}:=\{(x_1,\dots ,x_k)\in {[0,1]}^k\text{such that}{x}_1+\cdots +x_k=1\}$, $G(\mathbf{f})$ is a continuous strict convex function in the compact convex domain ${\mathrm{\Delta }}_{k-1}$. Therefore there will always exist a unique Kelly solution $f^{\mathrm{Kelly}}\in {\mathrm{\Delta }}_{k-1}$ which is dependent on $\mathbf{p}$.

Remark:

• (i)If the fitness matrix is diagonal, i.e., $(o_{\mathit{ij}})=\mathrm{diag}\{o_1,\dots ,o_k\}$, then ${(f^{\mathrm{Kelly}})}_i=p_i$;
• (ii)
  $f^{\mathrm{Kelly}}$ solves the system

  $$\begin{array}{c}\hfill \sum _{i=1}^k\frac{p_i{o}_{\mathit{ij}}}{{\overline{o}}_i(\mathbf{f})}=1,\forall j=1,\dots ,k.\end{array}$$

Appendix B: The Solution to Nonstationary Environments

We model the environment probabilities on a parameterized Beta distribution, such that $p\sim B(\alpha ,\beta )$, and prove that the Kelly solution (a static f that maximizes the asymptotic growth rate) in the asymptotic framework corresponds to the solution of the i.i.d. environment case with a probability equaling the expectation of the Beta distribution.

For sake of simplicity, we consider only $k=2$. We have $W_n(f)={\overline{o}}_1{(f)}^H{\overline{o}}_2{(f)}^{n-H}$, where $H\sim GB(n,\{p_1,\dots ,p_n\}\sim Beta(\alpha ,\beta ))$, i.e., $H={\epsilon }_1+\cdots +{\epsilon }_n$ with ${\epsilon }_r\sim Bernoulli(p_r)$ and $p_r\sim Beta(\alpha ,\beta )$. Using the law of large numbers, we have

$$\begin{array}{cc}\hfill G(f)=& \underset{n\to \infty }{lim}(\frac{{\sum }_{i=1}^n{\epsilon }_i}{n}log{\overline{o}}_1(f)+(1-\frac{{\sum }_{i=1}^n{\epsilon }_i}{n})log{\overline{o}}_2(f))\hfill \\ \hfill =& \underset{n\to \infty }{lim}(\frac{{\sum }_{i=1}^n\mathbb{E}{\epsilon }_i}{n}log{\overline{o}}_1(f)+(1-\frac{{\sum }_{i=1}^n\mathbb{E}{\epsilon }_i}{n})log{\overline{o}}_2(f))\hfill \\ \hfill =& \underset{n\to \infty }{lim}((\frac{1}{n}\sum _{r=1}^n{p}_r)log{\overline{o}}_1(f)+(1-\frac{1}{n}\sum _{r=1}^n{p}_r)log{\overline{o}}_2(f))\hfill \\ \hfill =& plog{\overline{o}}_1(f)+(1-p)log{\overline{o}}_2(f)\text{a.s.}\hfill \end{array}$$

where $p={lim}_{n\to \infty }\frac{1}{n}{\sum }_{r=1}^n{p}_r=\frac{\alpha }{\alpha +\beta }$ is the expectation of the Beta distribution. Thus, the Kelly solution in this case is the same as the previous case.

Appendix C: The Dutch Book Solution and the Corresponding Loss of Growth

In this section, we derive the Dutch book solution for our model. By definition, the Dutch book solution $f^D$ satisfies ${\overline{o}}_1(f)={\overline{o}}_2(f)=\cdots ={\overline{o}}_k(f)$ with the positive growth, i.e., ${\overline{o}}_1(f)>1$.

The case $k=2$: The Dutch book solution satisfies

$$\begin{array}{c}\hfill o_{11}f+o_{12}(1-f)=o_{21}f+o_{22}(1-f)>1.\end{array}$$

Therefore, if $\mathrm{\Delta }:=o_{11}{o}_{22}-o_{12}{o}_{21}>o_{11}+o_{22}-o_{12}-o_{21}$ then we always have a unique Dutch book solution $f^D$

$$\begin{array}{c}\hfill f^D=\frac{o_{22}-o_{12}}{o_{22}-o_{12}+o_{11}-o_{21}}\end{array}$$

and

$$\begin{array}{c}\hfill G(f^D)=\frac{\mathrm{\Delta }}{o_{22}-o_{12}+o_{11}-o_{21}}>1\text{(which does not depend on}p).\end{array}$$

The general case k: We give out here some criteria to have a unique Dutch book solution in the general case k.

Lemma 1

Given a fitness matrix $O={(o_{i,j})}_{i,j=1}^k$. Denote by ${\alpha }_{i,j}=o_{i,j}-o_{k,j}$ for all $j=1,\dots ,k$ and $i=1,\dots ,k-1$. Denote by $\mathrm{\Lambda }={({\mathrm{\Lambda }}_{i,j})}_{i,j=1}^k$ such that

$$\begin{array}{c}\hfill \left(\begin{array}{cccc}{\alpha }_{1,1}& \cdots & {\alpha }_{1,k-1}& {\alpha }_{1,k}\\ ⋮& \ddots & ⋮& ⋮\\ {\alpha }_{k-1,1}& \cdots & {\alpha }_{k-1,k-1}& {\alpha }_{k-1,k}\\ 1& \cdots & 1& 1\end{array}\right)\left(\begin{array}{ccc}{\mathrm{\Lambda }}_{1,1}& \cdots & {\mathrm{\Lambda }}_{1,k}\\ ⋮& \ddots & ⋮\\ {\mathrm{\Lambda }}_{k,1}& \cdots & {\mathrm{\Lambda }}_{k,k}\end{array}\right)=I_k.\end{array}$$

If this fitness matrix O satisfies

• (i)$o_{\mathit{ii}}>o_{\mathit{ji}}\ge 0$ for all $i,j=1,\dots ,k$
• (ii)${\mathrm{\Lambda }}_{i,k}>0$ for all $i=1,\dots ,k$
• (iii)$\sum _{j=1}^k{o}_{i,j}{\mathrm{\Lambda }}_{j,k}>1$ for all $i=1,\dots ,k-1$

then there exists a Dutch book solution defined by $f_j^D={\mathrm{\Lambda }}_{j,k},j=1,\dots ,k$ and the corresponding deterministic wealth is

$$\begin{array}{c}\hfill G({\mathbf{f}}^D)=\prod _{i=1}^k(\sum _{j=1}^k{o}_{i,j}{f}_j^D{)}^{p_i}=\sum _{j=1}^k{o}_{i,j}{\mathrm{\Lambda }}_{j,k}.\end{array}$$

Proof

We have from Condition (iii)

$$\begin{array}{c}\hfill {\overline{o}}_1({\mathbf{f}}^D)=\sum _{j=1}^k{o}_{1,j}{f}_j^D=\sum _{j=1}^k{o}_{1,j}{\mathrm{\Lambda }}_{j,k}>1;\end{array}$$

Moreover from the definition of $\alpha$ and $\mathrm{\Lambda }$, we have $\overline{o_i}({\mathbf{f}}^D)={\overline{o}}_j({\mathbf{f}}^D)$ for all $i\ne j=1\dots ,k.$ $\square$

Corollary 1

In the case of a diagonal matrix, i.e., $o_{i,j}=\mathrm{diag}\{o_1,\dots ,o_k\}$, by direct calculation, we obtain ${\mathrm{\Lambda }}_{i,k}=\frac{o_i^{-1}}{\sum _{j=1}^k{o}_j^{-1}}.$

Conditions (i) and (ii) hold true iff $o_i>0$ and condition (iii) holds true iff $\sum _{j=1}^k{o}_j^{-1}<1$.

Corollary 2

For a finite n and assuming $min{\{o_{\mathit{ii}}\}}_{i=1}^k\gg max{\{o_{\mathit{ij}}\}}_{i\ne j}\ge 0,$ there exists a Dutch book solution ${\mathbf{f}}^D$.

Proof

The conclusion directly follows from the above Corollary 1 (for a diagonal fitness matrix). $\square$

Appendix D: Finite Last Intersection

In this section, we show that for a given pair of strategies (f, g) with $G(f)>G(g)$, there is a $T(f,g)<\infty$ such that $W_n(f,\mathbf{x})>W_n(g,\mathbf{x})$ for all $n\ge T(f,g)$ and for all $\mathbf{x}\in {\{0,1\}}^{\infty }$. This means that the last intersection between two random trajectories ${\{W_n(f,\mathbf{x})\}}_n$ and ${\{W_n(g,\mathbf{x})\}}_n$

$$\begin{array}{c}\hfill \tau (\mathbf{x}):=sup\{n:W_n(f,\mathbf{x})\le W_n(g,\mathbf{x})\}\end{array}$$

is bounded above by T(f, g) (a finite number depending only on f and g).

Proof

We first define the excess growth rate

$$\begin{array}{c}\hfill E_n(\mathbf{x}):=\frac{1}{n}log{W}_n(f,\mathbf{x})-\frac{1}{n}log{W}_n(g,\mathbf{x}).\end{array}$$

We note that for all $\mathbf{x}$

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}{E}_n(\mathbf{x})=logG(f)-logG(g)>0.\end{array}$$ 7

To this end, we need to prove that there is a $T(f,g)<\infty$ such that

$$\begin{array}{c}\hfill \underset{\mathbf{x}}{inf}{E}_n(\mathbf{x})>0\forall n\ge T(f,g).\end{array}$$

Otherwise, for each k there exist $n_k\ge k$ and ${\mathbf{x}}_k\in {\{0,1\}}^{\infty }$ such that $E_{n_k}({\mathbf{x}}_k)\le 0$. Now, there exists a subsequence of $\{{\mathbf{x}}_k\}$ which is convergent to some $\mathbf{x}\in {\{0,1\}}^{\infty }$. Therefore as $k\to \infty$ we have $n_k\to \infty$ and ${lim}_{n_k\to \infty }{E}_{n_k}(\mathbf{x})\le 0$, in contradiction to (7). $\square$

Appendix E: Asymptotic Log-Normality of the Growth Rate

In this section, we show that in our discrete model, the growth rate approaches log-normality with zero variance.

Proof

We rewrite $\frac{1}{n}log{W}_n(f)=\frac{1}{n}{\sum }_{i=1}^n{y}_i,$ where $y_i=x_{i}log{\overline{o}}_1(f)+(1-x_i)log{\overline{o}}_2(f)$ are independent discrete random variables with values: $log{\overline{o}}_1(f),log{\overline{o}}_2(f)$ and probabilities: $p,1-p$ correspondingly. Thus we have a sequence of i.i.d. random variables ${\{y_i\}}_i$ with expectation $\mu =E(y_i)=G(f)$ and variance ${\sigma }^2=var(y_i)=p(1-p){(log{\overline{o}}_1(f)-log{\overline{o}}_2(f))}^2$. By using the CLT, we have for a large n: $1/\sqrt{n}{\sum }_{i=1}^n(y_i-\mu )\sim N(0,{\sigma }^2)$ which is equivalent to

$$\begin{array}{c}\hfill \frac{1}{n}\sum _{i=1}^n{y}_i\sim N\left(\mu ,,,\frac{{\sigma }^2}{n}\right).\end{array}$$

$\square$

Appendix F: Fully Correlated Log Growth Rates for the Case $k=2$:

In this section, we show that for all $f,g\ne f^D$

$$\begin{array}{c}\hfill \mathrm{Cor}[log{W}_n(f),log{W}_n(g)]=\pm 1.\end{array}$$

Proof

Denote by

$$\begin{array}{c}\hfill W_n(f,\mathbf{x})={\overline{o}}_1{(f)}^{|\mathbf{x}|}{\overline{o}}_2{(f)}^{n-|\mathbf{x}|},\end{array}$$

where $\mathbf{x}=(x_1,\dots ,x_n)$ is a realization and $|x|=x_1+\cdots +x_n$. Because $f,g\ne f^D$ we have ${\overline{o}}_1(f)\ne {\overline{o}}_2(f)$ and ${\overline{o}}_1(g)\ne {\overline{o}}_2(g)$, therefore we can define

$$\begin{array}{c}\hfill \lambda =\frac{log\frac{{\overline{o}}_1(f)}{{\overline{o}}_2(f)}}{log\frac{{\overline{o}}_1(g)}{{\overline{o}}_2(g)}}\in \mathbb{R}\backslash \{0\}.\end{array}$$

We first prove that for any given m realizations ${\mathbf{x}}^{(1)},\dots ,{\mathbf{x}}^{(m)}$, we have

$$\begin{array}{cc}& log{W}_n(f,{\mathbf{x}}^{(i)})-\frac{1}{m}\sum _{k=1}^{m}log{W}_n(f,{\mathbf{x}}^{(k)})\hfill \\ \hfill & =\lambda (log{W}_n(g,{\mathbf{x}}^{(i)})-\frac{1}{m}\sum _{k=1}^{m}log{W}_n(g,{\mathbf{x}}^{(k)})).\hfill \end{array}$$ 8

Indeed, we note that

$$\begin{array}{cc}\hfill log{W}_n(f,{\mathbf{x}}^{(i)})-log{W}_n(f,{\mathbf{x}}^{(j)})=& log\frac{{\overline{o}}_1{(f)}^{|{\mathbf{x}}^{(i)}|}{\overline{o}}_2{(f)}^{n-|{\mathbf{x}}^{(i)}|}}{{\overline{o}}_1{(f)}^{|{\mathbf{x}}^{(j)}|}{\overline{o}}_2{(f)}^{n-|{\mathbf{x}}^{(j)}|}}\hfill \\ \hfill =& (|{\mathbf{x}}^{(i)}|-|{\mathbf{x}}^{(j)}|)log\frac{{\overline{o}}_1(f)}{{\overline{o}}_2(f)},\hfill \end{array}$$

and similarly for g. This implies (8). Therefore

$$\begin{array}{cc}& \mathrm{Cor}[log{W}_n(f),log{W}_n(g)]\hfill \\ \hfill & =\frac{\mathrm{Cov}[log{W}_n(f),log{W}_n(g)]}{\sqrt{\mathrm{Var}[log{W}_n(f)]}\sqrt{\mathrm{Var}[log{W}_n(g)]}}\hfill \\ \hfill & =\frac{\frac{1}{m}\sum _{i=1}^m(log{W}_n(f,{\mathbf{x}}^{(i)})-E[log{W}_n(f)])(log{W}_n(g,{\mathbf{x}}^{(i)})-E[log{W}_n(g)])}{\sqrt{\frac{1}{m}\sum _{i=1}^m(log{W}_n(f,{\mathbf{x}}^{(i)})-E[log{W}_n(f)]{)}^2}\sqrt{\frac{1}{m}\sum _{i=1}^m(log{W}_n(g,{\mathbf{x}}^{(i)})-E[log{W}_n(g)]){)}^2}}\hfill \\ \hfill & =\frac{\lambda }{|\lambda |}=\pm 1.\hfill \end{array}$$

Remark 1

Whether the correlation is $\pm 1$ depends on $\lambda >0$ or $\lambda <0$. For $f=f^{\mathrm{Kelly}}$ the growth factor with environment “1” > the growth factor with environment “0” implying $log\frac{{\overline{o}}_1(f)}{{\overline{o}}_2(f)}>0$. Similarly for g it implies $log\frac{{\overline{o}}_1(g)}{{\overline{o}}_2(g)}>0$, therefore $\lambda >0$. At $f^D$, $log\frac{{\overline{o}}_1(f^D)}{{\overline{o}}_2(f^D)}=0$, therefore it acts as a threshold. In most cases, the correlation will be $+1$ since both f and g induce a positive growth rate.

Appendix G: Kelly is the Maximal Element in the Fitness Payoff Relation

Here we assume lineage size initial randomization, i.e., $W_n(f)\gg W_n(g)$ iff

$$\begin{array}{c}\hfill M_n(f,g):=\mathbb{P}(W_0{W}_n(f)>V_0{W}_n(g))\ge \mathbb{P}(V_0{W}_n(g)>W_0{W}_n(f))\end{array}$$

where $W_0$ and $V_0$ are random, and show that the Kelly strategy is the maximal element in this relation.

Proof

As a direct consequence of Proposition 1 and Eq. (11), we have

$$\begin{array}{c}\hfill M_n(f^{\mathrm{Kelly}},f)\ge 1/2\ge M_n(f,f^{\mathrm{Kelly}})\forall f\in [0,1],\end{array}$$

and equality if and only if $f=f^{\mathrm{Kelly}}$. $\square$

Appendix H: Non-Constant-Sum Game, But Conceptually Zero-Sum

In this section, we show that

Proposition 1

• (i)For $d=0$, $M_n(f,g)+M_n(g,f)=1$ for all f, g.
• (ii)For $d>0$, $M_n(f,g)+M_n(g,f)<1$ for all f, g.

Moreover, the game is conceptually zero-sum, but not formally.

Proof

• (i)
  We have from Eq. (9)

  $$\begin{array}{cc}\hfill M_n(f,g)+M_n(g,f)=& \sum _{s=0}^n(\mathbb{P}(W_n(f,s)W_0>W_n(g,s)V_0)\hfill \\ \hfill & +\mathbb{P}(W_n(f,s)W_0<W_n(g,s)V_0))P(s)=\sum _{s=0}^{n}P(s)=1.\hfill \end{array}$$
• (ii)
  On the other hand, we have from Eq. (5) for all $f\ne g$

  $$\begin{array}{cc}\hfill M_n(f,g)+M_n(g,f)=& \mathbb{P}(CAB)+\mathbb{P}(A{B}^c)+\mathbb{P}(C^{c}AB)+\mathbb{P}(B{A}^c)\hfill \\ \hfill =& \mathbb{P}(AB)+\mathbb{P}(A{B}^c)+\mathbb{P}(B{A}^c)=\mathbb{P}(A\cup B)<1.\hfill \end{array}$$

  where $C=\{W_0{W}_n(f)>V_0{W}_n(g)\}$, $A=\{W_0{W}_i(f)>d\forall i=1,\dots ,n\}$, $B=\{V_0{W}_i(g)>d\forall i=1,\dots ,n\}$. For $f=g$ we also have

  $$\begin{array}{cc}\hfill M_n(f,f)& =\mathbb{P}(W_0>V_0,A_1,A_2)<\mathbb{P}(W_0>V_0)=\frac{1}{2}.\hfill \end{array}$$

  where $A_1=\{W_0{W}_i(f)>d\forall i=1,\dots ,n\}$, $A_2=\{V_0{W}_i(f)>d\forall i=1,\dots ,n\}$. Finally, numeric simulations demonstrate that if $M(W,V)>M(U,V)$ then $M(V,W)<M(V,U)$ for all W, V, U, i.e., changing to a strategy with a gain for one player always incurs a loss for the other player.$\square$

Appendix I: The Symmetric Nash Equilibrium Solution to Payoff $M_n(f,g)$

Proposition 2

We always have

$$\begin{array}{c}\hfill \mathbb{E}(\frac{W_n(f)}{W_n(f^{\mathrm{Kelly}})})\le 1\end{array}$$

and the equality happens if and only if $p_{-}<p<p_{+}$.

Proof

For given f, g, we denote by ${\alpha }_1=\frac{{\overline{o}}_1(f)}{{\overline{o}}_1(g)}$, ${\alpha }_2=\frac{{\overline{o}}_2(f)}{{\overline{o}}_2(g)}$. We have

$$\begin{array}{cc}\hfill \mathbb{E}(\frac{W_n(f)}{W_n(g)})=& \sum _{\mathbf{x}}\frac{W_n(f,\mathbf{x})}{W_n(g,\mathbf{x})}P(\mathbf{x})\hfill \\ \hfill =& \sum _{s=0}^n\frac{W_n(f,s)}{W_n(g,s)}P(s)\hfill \\ \hfill =& \sum _{s=0}^n{\alpha }_1^s{\alpha }_2^{n-s}\left(\begin{array}{c}n\\ s\end{array}\right)p^s{(1-p)}^{n-s}={(p{\alpha }_1+(1-p){\alpha }_2)}^n.\hfill \end{array}$$

On the other hand, from the formula $f^{\mathrm{Kelly}}=\left\{\begin{array}{cc}0,\hfill & \text{if}p\in [0,p_{-}]\hfill \\ \frac{(1-p)o_{12}}{o_{12}-o_{11}}+\frac{p{o}_{22}}{o_{22}-o_{21}},\hfill & \text{if}p\in [p_{-},p_{+}]\hfill \\ 1,\hfill & \text{if}p\in [p_{+},1],\hfill \end{array}\right)$ we have for any pair $(f,f^{\mathrm{Kelly}})$, $p{\alpha }_1+(1-p){\alpha }_2=1$ if $p\in [p_{-},p_{+}]$ and $p{\alpha }_1+(1-p){\alpha }_2<1$ if $p\notin [p_{-},p_{+}]$. $\square$

Proposition 3

We consider a game with payoff without extinction

$$\begin{array}{c}\hfill M_n(f,g):=\mathbb{P}(W_n(f)W_0>W_n(g)V_0),\end{array}$$

where $W_0,V_0$ have the same distribution. Then, in this game, $(f^{\mathrm{Kelly}},f^{\mathrm{Kelly}})$ is a strict Nash equilibrium.

Proof

First, we note that

$$\begin{array}{cc}\hfill M_n(f,g)=& \sum _{s=0}^n\mathbb{P}(W_n(f,s)W_0>W_n(g,s)V_0)P(s)\hfill \\ \hfill =& \sum _{s=0}^n\mathbb{P}({\alpha }_1^s{\alpha }_2^{n-s}{W}_0>V_0)P(s)\hfill \\ \hfill =& \sum _{s\in A_1}P(s)\frac{1}{2}{\alpha }_1^s{\alpha }_2^{n-s}+\sum _{s\in A_2}P(s)(1-\frac{1}{2}{\alpha }_1^{-s}{\alpha }_2^{-n+s}),\hfill \end{array}$$ 9

where $A_1=\{s\in \{0,\dots ,n\}:{\alpha }_1^s{\alpha }_2^{n-s}<1\}$ and $A_2=\{0,\dots ,n\}-A_1$. Therefore, for $f=g$ we have ${\alpha }_1={\alpha }_2=1$, which implies $A_1=\mathrm{\varnothing }$, $A_2=\{0,\dots ,n\}$ and

$$\begin{array}{c}\hfill M_n(f,f)=\sum _{s=0}^{n}P(s)(1-\frac{1}{2})=1/2.\end{array}$$ 10

For any $f\ne f^{\mathrm{Kelly}}$, by using the Cauchy inequality for the second term, we have

$$\begin{array}{cc}\hfill M_n(f,f^{\mathrm{Kelly}})<& \sum _{s\in A_1}P(s)\frac{1}{2}{\alpha }_1^s{\alpha }_2^{n-s}+\sum _{s\in A_2}P(s)\frac{1}{2}{\alpha }_1^s{\alpha }_2^{n-s}\hfill \\ \hfill =& \frac{1}{2}{(p{\alpha }_1+(1-p){\alpha }_2)}^n.\hfill \end{array}$$

From Proposition 2, we have

$$\begin{array}{c}\hfill M_n(f,f^{\mathrm{Kelly}})<1/2=M_n(f^{\mathrm{Kelly}},f^{\mathrm{Kelly}}),\forall f\ne f^{\mathrm{Kelly}}.\end{array}$$ 11

Therefore $(f^{\mathrm{Kelly}},f^{\mathrm{Kelly}})$ is a strict Nash equilibrium. $\square$

Proposition 4

The above Nash equilibrium is the unique one in the game.

Proof

Assume that $(f_0,g_0)\ne (f^{\mathrm{Kelly}},f^{\mathrm{Kelly}})$ is another Nash equilibrium. Without loss of generality, we assume that $g_0\ne f^{\mathrm{Kelly}}$. By definition of a Nash equilibrium, we have $M_n(f_0,g_0)\ge M_n(f,g_0)$ for all f and $M_n(g_0,f_0)\ge M_n(g,f_0)$ for all g. By choosing $f=g=f^{\mathrm{Kelly}}$ and using Proposition 1 we have $M_n(f_0,g_0)\ge M_n(f^{\mathrm{Kelly}},g_0)>\frac{1}{2}$ and $M_n(g_0,f_0)\ge M_n(f^{\mathrm{Kelly}},f_0)\ge \frac{1}{2}$. This implies that $M_n(f_0,g_0)+M_n(g_0,f_0)>1$ which is a contradiction to Proposition 1. Therefore $(f^{\mathrm{Kelly}},f^{\mathrm{Kelly}})$ is the unique Nash equilibrium (see Fig. 8 where the equilibrium lies at the saddle-point of the payoff landscape.) $\square$

Fig. 8.

A 3D graphical representation of a probability payoff matrix [Eq.(5)] with primary (blue) and opponent (orange) lineage payoffs as intersecting saddle-point surfaces, highlighting the equilibrium locus (Color figure online)

Appendix J: The Symmetric Nash Equilibrium Solution to Payoff $M_c(f,g)$

Proposition 5

We consider a game with payoff defined as (6) without extinction

$$\begin{array}{cc}\hfill M_c(f,g):=& \mathbb{P}(T(f,c)<T(g,c))+\mathbb{P}(T(f,c)\hfill \\ \hfill =& T(g,c),W_0{W}_{T(f,c)}(f)>V_0{W}_{T(g,c)}(g)).\hfill \end{array}$$

Then, in this game, $(f^{\mathrm{Kelly}},f^{\mathrm{Kelly}})$ is a strict Nash equilibrium.

Proof

First we note that

$$\begin{array}{cc}\hfill M_c(f,g)=& \sum _{n=1}^{\infty }\mathbb{P}(T(g,c)>n,T(f,c)=n)\hfill \\ \hfill & +\mathbb{P}(T(g,c)=n,W_0{W}_n(f)>V_0{W}_n(g),T(f,c)=n)\hfill \\ \hfill =& \sum _{n=1}^{\infty }\mathbb{P}(V_0{W}_n(g)<c,T(f,c)=n)\hfill \\ \hfill & +\mathbb{P}(V_0{W}_n(g)\ge c,W_0{W}_n(f)>V_0{W}_n(g),T(f,c)=n)\hfill \\ \hfill =& \sum _{n=1}^{\infty }\mathbb{P}(W_0{W}_n(f)>V_0{W}_n(g),T(f,c)=n).\hfill \\ \hfill \end{array}$$

Then, from Proposition 3 we have

$$\begin{array}{c}\hfill M_c(f,f)=\sum _{n=1}^{\infty }\mathbb{P}(W_0>V_0,T(f,c)=n)=\mathbb{P}(W_0>V_0)=\frac{1}{2}\forall f\end{array}$$

and

$$\begin{array}{c}\hfill M_c(f,f^{\mathrm{Kelly}})<\frac{1}{2}\sum _{n=1}^{\infty }\mathbb{P}(T(f,c)=n)=\frac{1}{2}\forall f.\end{array}$$

Therefore $(f^{\mathrm{Kelly}},f^{\mathrm{Kelly}})$ is a strict Nash equilibrium. $\square$

Proposition 6

$(f^{\mathrm{Kelly}},f^{\mathrm{Kelly}})$ is the unique Nash equilibrium.

Proof

We first note that for all f, g

$$\begin{array}{cc}\hfill M_c(f,g)+M_c(g,f)=& \sum _{n=1}^{\infty }\mathbb{P}(T(f,c)=n)=1.\hfill \end{array}$$

The remaining part of the proof is similar to the proof in Propposition 4. $\square$

It is worthwhile here to highlight a link between this payoff and $M_n(f,g)$. Formally, $M_c(f,g)$ can be rewritten as a convex linear combination of $M_n(f,g)$:

$$\begin{array}{c}\hfill M_c(f,g)=\sum _{n=0}^{\infty }P(W_0{W}_n(f)>V_0{W}_n(g),T(f,c)=n).\end{array}$$

This has a straightforward interpretation: for each event $(T(f,c)=n)$, [a] the event $(T(f,c)<T(g,c))$ is equivalent to the event $(T(g,c)>n)$ or $(V_0{W}_n(g)<c\le W_0{W}_n(f))$, and [b] the event $(T(f,c)=T(g,c),W_0{W}_{T(f,c)}>V_0{W}_{T(g,c)}(g))$ is equivalent to the event $(c\le V_0{W}_n(g)<W_0{W}_n(f))$. Consequently the combination of the two events $(T(f,c)<T(g,c))$ and $(T(f,c)=T(g,c),W_0{W}_{T(f,c)}>V_0{W}_{T(g,c)}(g))$ is equivalent to the event $(W_0{W}_n(f)>V_0{W}_n(g))$.

Appendix K: The Probability Payoff Matrix Converges with Horizon n to the Expected Log Matrix

Proposition 7

For any pair (f, g) with $G(f)\ne G(g)$, we have

$$\begin{array}{c}\hfill M_{\infty }(f,g):=\underset{n\to \infty }{lim}{M}_n(f,g)=\left\{\begin{array}{c}1,\text{if}G(f)>G(g)\hfill \\ 0,\text{if}G(f)<G(g).\hfill \end{array}\right)\end{array}$$

Proof

If $G(f)-G(g)=\epsilon >0$, then by a similar argumentation as Appendix D, there exists $n_0<\infty$ such that for all $n\ge n_0$ and all $\mathbf{x}$

$$\begin{array}{cc}& 0<\frac{1}{n}log{W}_0<\frac{\epsilon }{4},0<\frac{1}{n}log{V}_0<\frac{\epsilon }{4},\frac{1}{n}log{W}_n(f,\mathbf{x})>G(f)-\frac{\epsilon }{4},\hfill \\ \hfill & \frac{1}{n}log{W}_n(g,\mathbf{x})<G(g)+\frac{\epsilon }{4}.\hfill \end{array}$$

Therefore, for all $n\ge n_0$ and all $\mathbf{x}$

$$\begin{array}{c}\hfill \frac{1}{n}log{W}_0+\frac{1}{n}log{W}_n(f,\mathbf{x})-\frac{1}{n}log{V}_0-\frac{1}{n}log{W}_n(g,\mathbf{x})>\frac{\epsilon }{4}>0.\end{array}$$

This implies that

$$\begin{array}{cc}& M_n(f,g)\hfill \\ \hfill & =\mathbb{P}(\frac{1}{n}log{W}_0+\frac{1}{n}log{W}_n(f)>\frac{1}{n}log{V}_0+\frac{1}{n}log{W}_n(g))=1\text{for all}n\ge n_0.\hfill \end{array}$$

Therefore, $M_{\infty }(f,g)=1$. Similarly we obtain $M_{\infty }(f,g)=0$ if $G(f)<G(g)$. $\square$

Remark 2

For the case $G(f)=G(g)$ there are only two cases, $g=f$ or $g=\widehat{f}$. If $g=f$ we have $M_{\infty }(f,f)=\frac{1}{2}$. If $g=\widehat{f}$ we do not know the value of $M_{\infty }(f,\widehat{f})$.

See Fig. 9 for a graphical illustration of the convergence.

Fig. 9.

The convergence of the probability payoff matrix to the expected log matrix. a: a probability payoff matrix (primary lineage) with a very large time horizon n, where payoffs are highly contrasted such that off-diagonal cells are near 0 or 1 (excepting those corresponding to strategies that result in very similar asymptotic growth rates). | b: a payoff matrix with entries based on pairwise differences in E[logW]

Appendix L: Nash Equilibrium in Population Size N

In this section, we show that the Nash solution in population size N, denoted by $f_N^{\ast }$, will be the strategy closest to Kelly under the finite resolution regime, and such that it converges asymptotically with N to the Kelly strategy. Denote by $f_N^{\ast }$ the closest element to $f^{\mathrm{Kelly}}$ in $I_N:=\{0,\frac{1}{N},\dots ,1\}$, i.e., $f_N^{\ast }={\arg \min }_{f\in I_N}|f-f^{\mathrm{Kelly}}|$. We show that $(f_N^{\ast },f_N^{\ast })$ is the Nash solution for the game with strategies defined only on $I_N$. Due to the definition of $f_N^{\ast }$, we see that $|f_N^{\ast }-f^{\mathrm{Kelly}}|\le \frac{1}{N}\to 0$ as $N\to \infty$. To this end, we show that $M_n(f_N^{\ast },f_N^{\ast })\ge M_n(f,f_N^{\ast })$ for all $f\in I_N$. Indeed, we have already from (10) that $M_n(f_N^{\ast },f_N^{\ast })=\frac{1}{2}$. Moreover, we have $plog{\overline{o}}_1(f)+(1-p)log{\overline{o}}_2(f)<plog{\overline{o}}_1(f_N^{\ast })+(1-p)log{\overline{o}}_2(f_N^{\ast })$ for all $f\in I_N\backslash \{f_N^{\ast }\}$. Therefore there exists $\epsilon >0$ such that

$$\begin{array}{c}\hfill plog\frac{{\overline{o}}_1(f)}{{\overline{o}}_1(f_N^{\ast })}+(1-p)log\frac{{\overline{o}}_2(f)}{{\overline{o}}_2(f_N^{\ast })}<-\epsilon \forall f\in I_N\backslash \{f_N^{\ast }\}.\end{array}$$

Thus, for every $f\in I_N\backslash \{f_N^{\ast }\}$ we have ${\sum }_{s=0}^n\alpha (s)P(s)<-n\epsilon$ where $\alpha (s):=slog\frac{{\overline{o}}_1(f)}{{\overline{o}}_1(f_N^{\ast })}+(n-s)log\frac{{\overline{o}}_2(f)}{{\overline{o}}_2(f_N^{\ast })}$. We assume that $log{W}_0$ and $log{V}_0$ have the same distribution with $\mathrm{supp}log{W}_0\supset \{\alpha (0),\dots ,\alpha (n)\}$ and $|\mathrm{supp}log{W}_0|=|\mathrm{supp}log{V}_0|=r>2n\epsilon$. Denote by $A_1=\{s:\alpha (s)<0\}$, $A_2=\{s:\alpha (s)\ge 0\}$ and $\delta =\frac{\frac{1}{2}-\frac{n\epsilon }{r}}{\frac{3}{4}-\frac{n\epsilon }{r}}\in (0,\frac{1}{2})$. We have

$$\begin{array}{cc}\hfill M_n(f,f_N^{\ast })=& \sum _{s=0}^n\mathbb{P}(\alpha (s)+log{W}_0>log{V}_0)P(s)=\sum _{s\in A_1}\frac{\frac{1}{2}{(r+\alpha (s))}^2}{r^2}P(s)\hfill \\ \hfill & +\sum _{s\in A_2}(1-\frac{\frac{1}{2}{(r-\alpha (s))}^2}{r^2})P(s)\hfill \\ \hfill =& \frac{1}{2}+\sum _{s\in A_1}\frac{\alpha (s)}{r}(1+\frac{\alpha (s)}{2r})P(s)+\sum _{s\in A_2}\frac{\alpha (s)}{r}(1-\frac{\alpha (s)}{2r})P(s)\hfill \\ \hfill =& \frac{1}{2}+\sum _{s\in A_1}\frac{\alpha (s)}{r}(\delta +\frac{\alpha (s)}{2r})P(s)+\sum _{s\in A_2}\frac{\alpha (s)}{r}(\delta -\frac{\alpha (s)}{2r})P(s)\hfill \\ \hfill & +(1-\delta )\sum _{s=0}^n\frac{\alpha (s)}{r}P(s).\hfill \end{array}$$

Note that $\frac{\alpha (s)}{r}\in [-1,0]$ for $s\in A_1$ and $\frac{\alpha (s)}{r}\in [0,1]$ for $s\in A_2$. Moreover $x(\delta +x/2)\le \frac{1}{2}-\delta <\frac{1}{2}-\frac{3}{4}\delta$ for $x\in [-1,0]$; $x(\delta -x/2)\le \frac{{\delta }^2}{2}<\frac{1}{2}-\frac{3}{4}\delta$ for $x\in [-1,0]$. Therefore, for every $f\in I_N\backslash \{f_N^{\ast }\}$ we have

$$\begin{array}{cc}\hfill M_n(f,f_N^{\ast })<& \frac{1}{2}+(\frac{1}{2}-\frac{3}{4}\delta )\sum _{s=0}^{n}P(s)+(1-\delta )\sum _{s=0}^n\frac{\alpha (s)}{r}P(s)\hfill \\ \hfill <& \frac{1}{2}+(\frac{1}{2}-\frac{3}{4}\delta )+(1-\delta )\frac{-n\epsilon }{r}=\frac{1}{2}.\hfill \end{array}$$

Appendix M: Nash Equilibrium in Nonstationary Environments

Proposition 8

We consider also a game with payoff of players are

$$\begin{array}{c}\hfill M_n(f,g):=\mathbb{P}(W_n(f)W_0>W_n(g)V_0),\end{array}$$

where $W_0,V_0$ have the same distribution. Then, in this game, $(f^{\mathrm{Kelly}},f^{\mathrm{Kelly}})$ is the unique strict Nash equilibrium.

Proof

We note that in the nonstationary case we have

$$\begin{array}{c}\hfill M_n(f,g)=\sum _{H=0}^n\mathbb{P}({\alpha }_1^H{\alpha }_2^{n-H}{W}_0>V_0)P(H),\end{array}$$

where $H\sim GB(n,\{p_1,\dots ,p_n\}\sim Beta(\alpha ,\beta ))$ is a generalized binomial distribution. Therefore the proof is similar to the proof in Proposition 3 and is omitted. $\square$

Appendix N: Limit of the Extinction Rate

Proposition 9

Denote by

$$\begin{array}{c}\hfill Q_{n,d}(f):=\mathbb{P}(W_0{W}_1(f)>d,\dots ,W_0{W}_n(f)>d).\end{array}$$

the probability that the extinction does not occur until time n and $P_{n,d}(f)=1-Q_{n,d}(f)$ the probability of extinction until time n (also see Fig. 6). We prove that

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}{P}_{n,d}(f)=\left\{\begin{array}{cc}0,\hfill & \text{if}{\overline{o}}_1(f),{\overline{o}}_2(f)>1\hfill \\ 1,\hfill & \text{if}{\overline{o}}_1(f),{\overline{o}}_2(f)<1\hfill \\ c_d(f)\in [0,1],\hfill & \text{else.}\hfill \end{array}\right)\end{array}$$

Proof

For the sake of simplicity, we denote by

$$\begin{array}{c}\hfill {\beta }_{n,d}(x_1,\dots ,x_n):=\frac{d}{{\overline{o}}_1{(f)}^{x_1}{\overline{o}}_2{(f)}^{1-x_1}}\vee \cdots \vee \frac{d}{{\overline{o}}_1{(f)}^{x_1+\cdots +x_n}{\overline{o}}_2{(f)}^{n-x_1-\cdots -x_n}}.\end{array}$$

Then we rewrite the formula

$$\begin{array}{cc}\hfill Q_{n,d}(f)& =\sum _{x_1,\dots ,x_n}P(x_1,\dots ,x_n)\mathbb{P}(W_0>{\beta }_{n,d}(x_1,\dots ,x_n)).\hfill \end{array}$$

• (i)If ${\overline{o}}_1(f),{\overline{o}}_2(f)>1$: we have ${\beta }_{n,d}(x_1,\dots ,x_{n-1},1)={\beta }_{n,d}(x_1,\dots ,x_{n-1},0)={\beta }_{n-1,d}(x_1,\dots ,x_{n-1})$, therefore $Q_{n,d}(f)=Q_{n-1,d}(f)=\cdots =Q_{0,d}=\mathbb{P}(W_0>d)=1$ for all n. Therefore ${lim}_{n\to \infty }{P}_{n,d}(f)=0$.
• (ii)If ${\overline{o}}_1(f),{\overline{o}}_2(f)<1$: we have ${\beta }_{n,d}(x_1,\dots ,x_n)=\frac{d}{{\overline{o}}_1{(f)}^{x_1+\cdots +x_n}{\overline{o}}_2{(f)}^{n-x_1-\cdots -x_n}}$ which approaches infinity with n. Therefore for n large enough, $Q_{n,d}=0$. This implies ${lim}_{n\to \infty }{P}_{n,d}(f)=1$.
• (iii)If ${\overline{o}}_1(f)>1>{\overline{o}}_2(f)$: we have ${\beta }_{n,d}(x_1,\dots ,x_{n-1},1)={\beta }_{n-1,d}(x_1,\dots ,x_{n-1})$. Note that $Q_{n,d}$ is decreasing and bounded below by 0, therefore there exists the limit of $Q_{n,d}(f)$ which implies the limit of $P_{n,d}(f)$.$\square$

Remark 3

$c_d(f)$ is increasing with d (see Fig. 6).

Proof

If $d_1>d_2$ then ${\beta }_{n,d_1}(x_1,\dots ,x_n)>{\beta }_{n,d_2}(x_1,\dots ,x_n)$ therefore $Q_{n,d_1}(f)<Q_{n,d_2}(f)$ which implies that $c_{d_1}(f)\ge c_{d_2}(f)$. $\square$