Thermodynamic selection: mechanisms and scenarios

Abstract

Thermodynamic selection is an indirect competition between agents feeding on the same energy resource and obeying the laws of thermodynamics. We examine scenarios of this selection, where the agent is modeled as a heat-engine coupled to two thermal baths and extracting work from the high-temperature bath. The agents can apply different work-extracting, game-theoretical strategies, e.g. the maximum power or the maximum efficiency. They can also have a fixed structure or be adaptive. Depending on whether the resource (i.e. the high-temperature bath) is infinite or finite, the fitness of the agent relates to the work-power or the total extracted work. These two selection scenarios lead to increasing or decreasing efficiencies of the work-extraction, respectively. The scenarios are illustrated via plant competition for sunlight, and the competition between different ATP production pathways. We also show that certain general concepts of game-theory and ecology–the prisoner’s dilemma and the maximal power principle–emerge from the thermodynamics of competing agents. We emphasize the role of adaptation in developing efficient work-extraction mechanisms.

I. INTRODUCTION

Thermodynamics studies energy transfer, storage, and usage. It started as a theory of heat engines, drove the Industrial Revolution, and matured at nearly the same time when evolutionary biology emerged. Nowadays, thermodynamics is perhaps the most general phenomenological theory in all of science that applies to all types of systems at all levels of organization. Several attempts have been made to represent various aspects of evolutionary biology, in particular, evolution of populations and ecosystems, within the framework of thermodynamics [1-11].

Here we develop a thermodynamic approach to selection. Its main premise is that once organisms (agents) extract work (useful energy) and obey the laws of non-equilibrium thermodynamics in their metabolism [12], they can be modeled as heat engines. Agents interact (compete) indirectly, if the extraction goes from the same source. This competition can be represented via game theory, and hence its outcome depends on work-extraction strategies adopted by the agents. Such strategies depend on two parameters: efficiency and power. The second law of thermodynamics states that the efficiency of any heat engine—defined as the ratio of useful extracted energy (work) to the total energy input (heat)—is bound from above by Carnot’s efficiency [13-15]. But heat engines operating at the maximum efficiency yield zero work per unit of time (zero work power) resulting in the well-known power-efficiency tradeoff [16-19]: the most efficient regime is not powerful, whereas the most powerful regime is not efficient [20].

The energy budget of an organism can be described as three main energy currents: input, storage, and output (waste) [4, 21-23]. The relationship between these three currents are similar to that in a generalized heat engine: input heat, work (storage), and output heat. Similar to abiotic heat engines, organisms also face the power-efficiency (or speed-fidelity) trade-off. In particular, this trade-off is seen in molecular machines of cells [24-27], and also at the level of organism phenotypes [28-34]. The power-efficiency trade-off is subject to selection and depends on available energy resources.

Hence, our goal is to explore a physical model for the evolution of the metabolic power-efficiency trade-off, where agents are modeled as heat engines. We do not specify how the extracted work is utilized (reproduction, metabolism, defense, or other functions). Instead, we focus on different strategies (phenotypes) that are available to the agents to extract and store energy. The competition and selection emerge because at least two agents employ the same source (high-temperature bath). There are two general scenarios for such competition, for effectively infinite and for finite—and hence depletable—resources. The quantities relevant for evolution in these two situations are, respectively, the power of work extraction and the stored energy (=total extracted work).

Competition for an infinite resource is analogous to the competition of plants for light. Here the source, i.e. the Sun, acts as a thermal bath providing high temperature photons for the heat engine operation of the photosynthesis. It is not depletable, and yet, there is a competition for a limited energy current reaching the forest surface [63-73]. Plants can behave differently when facing such competition, from confrontation to avoidance of the competitor [71, 74, 75]. In section III we formalize and examine these situations that can have more general relevance in the context of nutrient allocation between cells in multicellular organisms. In particular, we show that the competition leads to increasing the efficiencies consistently with observations.

Exploitation of a finite source is a dynamical process, since this source is depleted due to the functioning of the agents themselves. We study this process in section IV and show that competition favors heat engines with lower efficiencies. An example of this is the fermentation (aerobic and anaerobic) and respiration pathways of ATP production in yeasts [31, 32, 34, 83, 84, 86] and in solid tumor cells [90-93]. Here the ATP production refers to work-extraction and storage [47]. Respiratory ATP production is far more efficient than fermentation, but the speed and hence the power of the fermentation path is greater [76, 83, 85]. Given the available resources and the presence of competition, cell choose one or the other pathway of ATP production [31, 83, 86].

Agents competing for a depletable resource alter the common environment similarly to what happens in niche construction theories [98-100]. Thereby they shape the selection process. Hence, we face a non-trivial game-theoretic situation, where the optimal values of power and efficiency under competition are not unique. However, the environmental changes caused by the behavior of competing agents are “myopic”, that is, the behavior of the agents is not based on perception of the global environmental state.

The common environment of competing agents changes due to the very engine functioning. This fact poses the problem of adaptive (i.e. structure adjusting) versus non-adaptive agents. This is analogous to the phenotype adaptation that is observed in organisms [28, 77-81, 94-97]. As seen below, adaptation plays an important role in selection process.

The rest of this paper is organized as follows. Next section defined the heat engine model we employ. Section III addresses competition for an infinite resources that amounts to sharing a fixed energy current. Section IV studies the competition for a finite resource. We explore this situation via studying two competing agents that can be adaptive or not. Sections III and IV can be read independently from each other. Both sections employ ideas and techniques from game theory, though no deep preliminary knowledge on this subject is assumed, since we provide the necessary background. We summarize in the last section. All technical derivations are relegated to Appendices.

II. THERMODYNAMIC AGENT (HEAT ENGINE)

A. Heat-engine model

To model energy extraction and storage, we focused on the minimal thermodynamically consistent model of a heat engine [18, 52]. For further clarity, we start with the explicit implementation of this model via three-energy level Markov systems attached to different heat baths at different temperatures; see Fig. 1 and Appendix A for details. Having an explicit model is essential for clarifying the nature of the involved parameters and the extracted work (stored energy). However, the model will be explored in the high-temperature (linear response) regime, where the implementation details are not essential, and where it is equivalent to linear thermodynamic models employed in biophysics [52].

FIG. 1:

A schematic representation of the heat-engine model (1-10), and related thermal baths that cause transitions between the three engine states with energies E_i, i = 1, 2, 3. Now ρ_i←j is the transition rate from state j to state i. The symbol ρ_i↔j in the figure means that both transitions ρ_i←j and ρ_j←i are allowed for. J_h, J_c, J are heat currents from the thermal baths with temperature T_h, T_c, T, respectively. Note that the thermal baths drive transitions between the energy levels.

The engine has three states i = 1, 2, 3. This is the minimal number of states a stationary heat engine can have, because it should be in a non-equilibrium state (i.e. to support one cyclic motion), and has to support three external objects, one work-source and two thermal baths. Each state i has energy E_i, such that

$$E_1=0<E_2<E_3.$$ (1)

Transitions between each pair of different states are caused by the different thermal baths having different temperatures (T_h, T_c, T) that accordingly provide or accept necessary energies; cf. Fig. 1. We assume that these thermal baths are in thermal equilibrium states, which means that the transition rates that drive Markov evolution of the engines obey the detailed-balance condition, for example the transition rates between states {1, 3} satisfy the following relation

$${\rho }_{1\leftarrow 3}{e}^{-{\beta }_h{E}_3}={\rho }_{3\leftarrow 1}{e}^{-{\beta }_h{E}_1},$$ (2)

where β_h = 1/T_h. Similar relation holds for the transition rates ρ_1←2 and ρ_2←3 caused by thermal baths with temperature T and T_c, respectively.

One temperature is assumed to be infinite [37]: β = 1/T = 0. This bath is then a work-source. This key point can be explained as follows. First, note that an infinite temperature thermal bath exchanges dE energy without changing its own entropy, dS = βdE = 0, which is a feature of mechanical device (a sources of work) [37]. Second, if the T = ∞-bath spontaneously interacts with any (positive) temperature bath, then the former bath always looses energy. Hence, its energy is freely convertible to any form of heat, as expected from work. Next, we assume T_h > T_c, as necessary for heat engine operation.

In the stationary (but generally non-equilibrium) state the average energy of the three-level system is constant

$$\sum _{i=1}^3\frac{d{p}_i}{dt}{E}_i=J_{\mathrm{h}}+J_{\mathrm{c}}+J=0,$$ (3)

where p_i is the probability of finding the system in energy level E_i. Here J and J_n with n = h, c are the average energy lost (for J, J_n > 0) or gain (J, J_n < 0) by each bath per unit of time. Eq. (3) is the first law of thermodynamics for a stationary state [44]. Now (3) indicates on a perfect coupling between the thermal baths and the three-level system: there is no an energy current standing for irreversible losses within the system; cf. [27, 32, 85, 87-89].

In the stationary state, the energy currents hold

$$J=\frac{E_2}{Ƶ}{\rho }_{2\leftarrow 1}{\rho }_{1\leftarrow 3}{\rho }_{3\leftarrow 2}\left(1-e^{({\beta }_c-{\beta }_h)E_3-{\beta }_c{E}_2}\right),$$ (4)

$$J_{\mathrm{h}}=-E_{3}J∕E_2,J_{\mathrm{c}}=(E_3-E_2)J∕E_2.$$ (5)

where 𝓩 is the normalization factor defined by transition rates (see Appendix A).

If the system functions as a heat engine, i.e. on average, pumps energy to the work-source, then

$$J<0,J_{\mathrm{h}}>0,J_{\mathrm{c}}<0,$$ (6)

Using Eq.(5,4) and the condition (6) one get the condition for the system to operate as a heat-engine

$$E_2[(1-\vartheta )E_3-E_2]>0,\vartheta \equiv T_{\mathrm{c}}∕T_{\mathrm{h}}={\beta }_{\mathrm{h}}∕{\beta }_{\mathrm{c}}.$$ (7)

The efficiency of any heat engine is defined as the result (the extracted work) divided over the resource (the energy coming from the hot bath). Recalling E₃ > E₂, the efficiency η reads from (7):

$$\eta \equiv \frac{-J}{J_{\mathrm{h}}}=\frac{E_2}{E_3}\le {\eta }_{\mathrm{C}}\equiv 1-\vartheta .$$ (8)

Hence, the efficiency η is bounded from the above by the Carnot efficiency n_C. Eq. (8) is the second law of thermodynamics for the heat engine efficiency [44].

Eqs. (4, 7, 8) demonstrate the power-efficiency trade-off: at the maximum efficiency η = η_C the power −J of the heat engine nullifies. This trade-off is also a general feature of heat engines [19]. A clear understanding of this trade-off is one pertinent reason for having an explicit microscopic model of a heat engine.

The work power −J depends on the specific form of the transition rates that enter the detailed balance condition (2) (see Appendix A); for example the Arrhenius form of transition rates applies in chemical reaction dynamics [43]. Here we work in the high-temperature limit, where the details of rates are not important provided they hold the detailed balance. Now E_i/T_c ≪ 1 and E_i/T_h ≪ 1, but 0 ≤ ϑ ≤ 1 in (7) can be arbitrary. In this limit the heat engine power reads via Eq.(4, 8):

$$-J=\rho {\beta }_{\mathrm{c}}{E}_3^2\eta (1-\vartheta -\eta ),$$ (9)

where $\rho =\frac{1}{Ƶ}({\rho }_{2\leftarrow 1}{\rho }_{1\leftarrow 3}{\rho }_{3\leftarrow 2}){\mid }_{{\beta }_h={\beta }_c=0}$ is a constant. Eq. (9) shows that for a fixed ϑ the maximum power of ∣J∣ of the engine is attained for

$$\eta =\frac{1-\vartheta }{2}=\frac{{\eta }_{\mathrm{C}}}{2}.$$ (10)

B. Relevance of heat-engines in biology

Below we shall heuristically apply the heat engine model to ATP production, where the two thermal baths refer to the e.g. glucose (the major reactant of the ATP production), while the work corresponds to the energy stored in ATP, which is metastable at physiological conditions. In this context let us discuss to which extent heat-engine models can be applied to transformations of chemical energy. The standard understanding of the chemical energy stored in certain molecular degrees of freedom is that it is isothermal and is described by the (Gibbs) free energy difference between reactants and products. This coarse-grained description does not tell where (in which degrees of freedom) the energy was stored and how it came out. Detailed mechanisms of such processes are still unclear, e.g. there is a long-standing and on-going debate on how precisely ATP delivers the stored energy during its hydrolysis and how this energy is employed for doing work; see e.g. [45-47]. However, it is clear that at sufficiently microscopic level all types of stored energy should be related to the fact that certain degrees of freedom are not in the thermal equilibrium with the environment [47]. Indeed, if all degrees of freedom would be thermalized at the same temperature, the second law will not allow any work-extraction ¹. It is known that frequently such non-thermalized degrees of freedom can be described by different effective temperatures [48]. Moreover, even a finite non-equilibrium system can (under certain conditions) play the role of a thermal bath, since the dynamics of its subsystem obeys the detailed balance condition [49]. Thus when describing work-extraction from chemical energy, it is meaningful to assume two different thermal baths, which is in fact the simplest situation of a non-equilibrium system. Modeling work-extraction through different chemical potentials (a situation closer to the standard understanding of the stored chemical energy) is in fact structurally similar to heat-engines [50, 51], also because we work in the high-temperature limit, where many implementation details are irrelevant. In this limit our model is fully consistent with linear equilibrium thermodynamics [18, 52]. Similar models have been widely employed in bioenergetics for modeling coupled chemical reactions, where the passage of heat from higher to lower temperatures corresponds to the down-hill reaction, whereas work extraction corresponds to the up-hill reaction [52].

III. COMPETITION FOR FIXED CURRENT

Two agents 1 and 2 competing for the same resource can be described as two heat engines attached to the same thermal baths [cf. (9)]. The sum of the currents coming from the high-temperature bath T_h to each heat engine is bound from above [cf. (6)]:

$$J_{1\mathrm{h}}+J_{2\mathrm{h}}\le A,A>0,$$ (11)

where A is a positive constant, and J_{1 h} and J_{2 h} are the currents coming from the same T_h-bath to each heat engine; cf. (6).

Using (9, 5) we write constraint (11) as

$${\eta }_1+ϵ{\eta }_2\ge a(1-\theta )(1+ϵ),ϵ\equiv E_{32}^2∕E_{31}^2,$$ (12)

$$a\equiv 1-\frac{1}{(1-\vartheta )(1+ϵ)}\frac{A}{E_{31}^2}.$$ (13)

Here E_{3 k} is the third (highest) energy level of agent k = 1, 2, ϵ is the ratio of energy scales of the engines that also determines the difference between 1 and 2. For ϵ = 1, the two agents are equivalent, i.e. the situation is symmetric.

We introduce scaled efficiencies x_k = η_k/η_C ∈ [0, 1] for agents k = 1 and k = 2. Then the work extracted by an agent reads

$$-J_k=\rho {\beta }_c{E}_{31}^2(1-\vartheta {)}^2{𝒲}_k,k=1,2,$$ (14)

$${𝒲}_1=x_1(1-x_1),{𝒲}_2=ϵx_2(1-x_2).$$ (15)

For a in (13) we assume

$$1∕2<a<1.$$ (16)

The first inequality in (16) ensures that the agents cannot simultaneously maximize 𝓦₁ and 𝓦₂ at η₁ = η₂ = (1 − ϑ)/2 (i.e. at x₁ = x₂ = 1/2), which refers to the maximum power regime (10) for both agents.

The second inequality in (16) means that (12) still allows some x₁ < 1 and x₂ < 1. Thus, Eq. (11) creates a non-trivial competition between the agents, which translates into the following game theoretical problem

$${𝒲}_1=x_1(1-x_1),{𝒲}_2=ϵx_2(1-x_2),$$ (17)

$$x_1+ϵx_2\ge a(1+ϵ),$$ (18)

where x_k and 𝓦_k are the actions and payoffs of the agents, respectively.

Thus, the present game has two dimensionless parameters: a and ϵ. Now a > 1/2 means that the constraint is non-trivial, while ϵ defines the payoff asymmetry between 1 and 2; cf. (15).

Game theory offers several concepts of equilibrium that formalize the ill-defined—due to (18)—notion of “joint” maximization for the utilities 𝓦₁ and 𝓦₂ [54-56]. For the game considered here, only the equilibrium concepts of Stackelberg and Pareto-Nash are non-trivial, and these are examined below.

A. Stackelberg’s equilibrium

Stackelberg’s competition model [53, 55, 57] is a sequential solution of the above game: the first agent (1) is the leader, since it has the advantage of the first move. The second agent (2) is the follower that responds to the first move by 1. Hence 1 chooses x₁ = 1/2, which is the unconstrained maximum of 𝓦₁. The best response of 2 to this action reads from the first inequality in (18)

$$x_2=\frac{1}{ϵ}\left[a(1+ϵ)-\frac{1}{2}\right]\le 1,$$ (19)

where the last inequality amounts to

$$a(1+ϵ)<\frac{1}{2}+ϵ.$$ (20)

There are cases where (20) does not hold even though the second inequality in (18) still holds. In such cases, no action by 2 is possible because x₂ > 1. Hence the co-existence of two agents is impossible. Satisfying (20) naturally becomes harder for a → 1 and easier for larger ϵ so that (20) always holds for ϵ → ∞. Thus, the follower tends to have a sufficiently large ϵ because otherwise, it is eliminated from the competition. Whenever (20) holds, 2 operates with a higher efficiency than 1, that is, invaders are more efficient [70, 71]. For ϵ < 1 the work extracted by 2 is smaller than the work extracted by 1: 𝓦₂ < 𝓦₁ = 1/4; cf. (17). However, 𝓦₂ > 𝓦₁ for ϵ > 1 and a sufficiently small a, that is, the second agent can extract a larger amount of work at a higher efficiency; cf. (17, (19).

B. Pareto line, Nash equilibria and evolutionary stability

A pair $(x_1^{[\mathrm{P}]},x_2^{[\mathrm{P}]})$ is a Pareto equilibrium of the game if for any allowed $x_1\ne x_1^{[\mathrm{P}]}$ and $x_2\ne x_2^{[\mathrm{P}]}$ it holds [54-56]

$$\text{If}{𝒲}_1(x_1,x_2^{[\mathrm{P}]})>{𝒲}_1(x_1^{[\mathrm{P}]},x_2^{[\mathrm{P}]}),$$ (21)

$$\text{then}{𝒲}_2(x_1,x_2^{[\mathrm{P}]})<{𝒲}_2(x_1^{[\mathrm{P}]},x_2^{[\mathrm{P}]}).$$ (22)

$$\text{If}{𝒲}_2(x_1^{[\mathrm{P}]},x_2)>{𝒲}_2(x_1^{[\mathrm{P}]},x_2^{[\mathrm{P}]}),$$ (23)

$$\text{then}{𝒲}_1(x_1^{[\mathrm{P}]},x_2)<{𝒲}_1(x_1^{[\mathrm{P}]},x_2^{[\mathrm{P}]}).$$ (24)

Eqs. (21-24) mean that if a change from $(x_1^{[\mathrm{P}]},x_2^{[\mathrm{P}]})$ benefits one agent, it implies losses for another agent. Now $(x_1^{[\mathrm{P}]},x_2^{[\mathrm{P}]})$ is an equilibrium, as far as there is a certain cooperation between the agents preventing any move $(x_1^{[\mathrm{P}]},x_2^{[\mathrm{P}]})\to (x_1,x_2^{[\mathrm{P}]})$ or $(x_1^{[\mathrm{P}]},x_2^{[\mathrm{P}]})\to (x_1^{[\mathrm{P}]},x_2)$ [54-56]. This contrasts the above Stackelberg’s solution, where agent 1 chooses the optimal condition for itself, irrespective of what happens to 2.

Eqs. (17, 18) show how to find the set of Pareto equilibria (i.e. the Pareto line). Note that both 𝓦₁ = x₁(1 − x₁) and 𝓦₂ = ϵx₂ (1 − x₂) achieve their absolute maxima at (resp.) x₁ = 1/2 and x₂ = 1/2. Hence the Pareto line is the line x₁ + ϵx₂ = a(1 + ϵ) possibly bound by lines x₁ = 1/2 and x₂ = 1/2:

$$x_1+ϵx_2=a(1+ϵ),0\le x_k\le 1,k=1,2,$$ (25)

$$\max [\frac{1}{2},a(1+\frac{1}{ϵ})-\frac{1}{ϵ}]\le x_2\le \text{min}[a(1+\frac{1}{ϵ})-\frac{1}{2ϵ},1].$$ (26)

Examples of Pareto lines (25, 26) are presented in Fig. 2 for different values of model parameters.

FIG. 2: The Pareto line (25, 26) of the game (17, 18) for different values of a, ϵ.

Blue (lower straight) line is x₁ + ϵx₂ = a(1 + ϵ) with a = 0.6, ϵ = 2. The allowed states are located above this line still confined by the square [0, 1] × [0, 1]. The Pareto line is the fraction [A, B] of this line that lies between intersections with x₁ = 1/2 and x₂ = 1/2; cf. (26). The worst outcomes for the players—defined by (28, 29, 30)—are ${𝒲}_2^{\ast }\ne 0$ and ${𝒲}_1^{\ast }=0$.

Red (upper straight) line is x₁ + ϵx₂ = a(1 + ϵ) with a = 0.8, ϵ = 0.8. Now the whole red line is the Pareto line. The worse outcomes for the players are ${𝒲}_2^{\ast }={𝒲}_1^{\ast }=0$.

Note that there is a continuum of Pareto equilibria so that additional reasoning is needed to select a unique outcome. Moreover, for this game, all Pareto equilibria are also Nash equilibria.

Let us discuss this point in more detail, and also outline relations with evolutionary stable strategies. Recall that a Nash equilibrium of the game with payoffs W_k(x₁, x₂) and strategies x₁ and x₂ for each agent k = 1, 2 is a pair of strategies $(x_1^{\ast },x_2^{\ast })$ such that for any allowed $x_1\ne x_1^{\ast }$ and $x_2\ne x_2^{\ast }$

$$W_1(x_1^{\ast },x_2^{\ast })\ge W_1(x_1,x_2^{\ast }),W_2(x_1^{\ast },x_2^{\ast })\ge W_2(x_1^{\ast },x_2).$$ (27)

Eqs. (27) mean that for each agent any unilaterial deviation from $(x_1^{\ast },x_2^{\ast })$ is not beneficial. If the inequalities in (27) are strict, the Nash equilibrium $(x_1^{\ast },x_2^{\ast })$ is said to be strict. Note that in our problem the payoff of the players depend on the opponent’s strategy via constraint (18). For 1 the best response strategy $x_1^{\ast }(x_2)$ to the given strategy x₂ of 2 reads: $x_1^{\ast }(x_2)=1∕2$ if 1/ϵ (a(1 + ϵ) − 1/2) ≤ x₂ ≤ 1; otherwise: $x_1^{\ast }(x_2)=a(1+ϵ)-x_2$. Similar relation holds for 2. The intersection of curves $x_1^{\ast }$ and $x_2^{\ast }$ coincides with Pareto line (25, 26).

The Nash equilibria in our situation are strict; hence they are also evolutionary stable [61], since no mutant strategy x₁ can obtain higher payoff than the resident strategy $x_1^{\ast }:W_1(x_1,x_2^{\ast })<W_1(x_1^{\ast },x_2^{\ast })$.

Note that a refinement of evolutionary stability for continuous strategies—known as continuously stable states [61]—cannot be applied to our situation. Indeed, the refinement demands from a strict Nash equilibrium to be also a local maximum of the payoffs. However, the notion of a local maximum cannot apply to our situation, since the payoff functions are discontinuous on the Pareto line [60], i.e. they are not defined bellow the Pareto line in Fig. 2. Further consideration of evolutionary stability demands implementing mixed strategies which goes out of our present scope.

C. Worst outcomes

For any game it is relevant to know what worst thing agent 2 can do for 1, at any expense for 2. Besides describing hostility, worst outcomes will be relevant for bargaining solutions below. The worst outcome ${𝒲}_k^{\ast }$ for agent k = 1, 2 read:

$${𝒲}_1^{\ast }={\max }_{x_1}{\text{min}}_{x_2}[{𝒲}_1],{𝒲}_2^{\ast }={\max }_{x_2}{\text{min}}_{x_1}[{𝒲}_2].$$ (28)

To find out ${𝒲}_1^{\ast }$, note from (18) that any choice x₁ of 1 should hold x₁ ≥ a(1 + ϵ) − ϵx₂. If 2 arranged its action x₂ such that a(1 + ϵ) − ϵx₂ ≥ 1, then there is no choice to be made by 1. For such cases we shall define its utility 𝓦₁ = 0, and this is clearly the worst outcome for 1. Now a(1 + ϵ) − ϵx₂ ≥ 1 or equivalently [a(1 + ϵ) − 1]/ϵ ≥ x₂ will work provided that a(1 + ϵ) − 1 ≥ 0. Hence, if the latter condition holds we get ${𝒲}_1^{\ast }=0$. Otherwise, if a(1 + ϵ) − 1 < 0, then 2 achieves min_x2[𝓦₁] when the bound x₁ ≥ a(1 + ϵ) − ϵx₂ is possibly tight, i.e. for x₂ = 0, and then x₁ = a(1 + ϵ) = argmax_x1 min_x2 [𝓦₁]. Calculating ${𝒲}_2^{\ast }$ in the same way we get

$${𝒲}_1^{\ast }=a(1+ϵ)\max [0,1-a(1+ϵ)],$$ (29)

$${𝒲}_2^{\ast }=a(1+{ϵ}^{-1})\max [0,ϵ-a(1+ϵ)].$$ (30)

Note that ${𝒲}_1^{\ast }{𝒲}_2^{\ast }=0$: (29, 30) imply ${𝒲}_1^{\ast }{𝒲}_2^{\ast }>0$ if $\frac{a}{1-a}<ϵ<\frac{1-a}{a}$, which is impossible since $1\ge a\ge \frac{1}{2}$.

The fact of e.g. ${𝒲}_1^{\ast }>0$ is important, since it means that 1 cannot be eliminated from the competition. Note that (29, 30) are not simply the separate global minima of (𝓦₁, 𝓦₂), i.e. generally $({𝒲}_1^{\ast },{𝒲}_2^{\ast })\ne (0,0)$. Still we get $({𝒲}_1^{\ast },{𝒲}_2^{\ast })=(0,0)$ for ϵ = 1.

D. Fair allocation and the Nash bargaining solution

Bargaining solutions assume additional cooperativity between the agents—that relates to fair division of resources—thereby looking for a unique choice within the Pareto line (25, 26) [54-56, 59]. We shall work with the Nash bargaining solution [54-56, 59]. An advantage of this over other bargaining solutions is the consistency of underlying axioms with thermodynamic processes [58].

The Nash bargaining solution $({\widehat{x}}_1,{\widehat{x}}_2)$ is found from (29, 30) by maximizing the geometric mean of ${𝒲}_1-{𝒲}_1^{\ast }$ and ${𝒲}_2-{𝒲}_2^{\ast }$ [56, 58, 59]:

$${\max }_{x_1,x_2}\left[\left({𝒲}_1(x_1)-{𝒲}_1^{\ast }\right)\left({𝒲}_2(x_2)-{𝒲}_2^{\ast }\right)\right].$$ (31)

The meaning of (31) is that both 1 and 2 (in a sense) simultaneously maximize their increments over the worst outcomes. Note that only one of ${𝒲}_1^{\ast }$ and ${𝒲}_2^{\ast }$ can be non-zero, that is, one of the agents will be competed out in the worst cases; see Appendix B.

Eq. (31) implies that for ϵ = 1 (the same energy scales) the maximizers ${\widehat{x}}_1$ and ${\widehat{x}}_2$ of (31) hold ${\widehat{x}}_1={\widehat{x}}_2=a$. Thus, the resources are allocated evenly between equivalent agents. More generally, (31) predicts $0.5<{\widehat{x}}_k<1$ and k = 1, 2, that is, none of the agents is eliminated, and none of them operates at the maximum power. Another implication of (31) is that

$$\text{sign}[{𝒲}_1-{𝒲}_2]=\text{sign}[1-ϵ],$$ (32)

i.e. for ϵ < 1, 2 extracts less work than 1; see Fig. 3, where 𝓦₁ − 𝓦₂ is presented for various ϵ and a. It is seen that the difference in the obtained resources of the agents increases with the available resource level (i.e. with decreasing a, cf. (18)). As shown by numerical solution of (31), no simple relation exists for ${\widehat{x}}_1-{\widehat{x}}_2$; see Fig.3. Indeed, acquiring more resource is possible either with higher efficiency level (left-top red region) or low efficiency level (left-bottom blue region).

FIG. 3: The Nash bargaining solution for the game (17, 18).

(a): 𝓦₁ − 𝓦₂ obtained from (31) for different a and ϵ. Red (blue) regions refer to 𝓦₁ − 𝓦₂ > 0 [𝓦₂ − 𝓦₁ > 0]. Darker red (darker blue) colors refer to larger 𝓦₁ − 𝓦₂ [larger 𝓦₂ − 𝓦₁]. Lines represent contours of fixed values for 𝓦₁ − 𝓦₂.

(b): ${\widehat{x}}_1-{\widehat{x}}_2$ for different a and ϵ, where $({\widehat{x}}_1,{\widehat{x}}_2)$ are found from argmax in (31). Colors follow the above logic.

E. Outlook

Plants compete for sunlight. They evolved various strategies for that: horizontal growth, vertical growth, shade tolerance etc) [63-73]. Photosynthes is a heat engine operating between two thermal baths, the hot photon bath generated by the Sun and the cold bath of the ambient environment. The sunlight is not an exhaustible source (in contrast to food), but plants growing on the same territory can appropriate different portions of the sunlight current depending on their competing abilities and strategies.

We can thus apply the heat-engine model to a single plant. The competition between two plants (i.e. two heat-engines) will be described via (11-15). Now (11, 18) correspond to constraints that determine the competition between two plants (agents): a larger a signifies stronger competition, while the minimal value a → 1/2 of a means no competition. Once ϵ determines the agents asymmetry between the work-currents [cf. (15)], it is natural to relate ϵ to the ratio of the full active leaf area of each plant: ϵ > 1 means that for agent 2 this area is larger.

Plants can have two basic competition strategies:

– (1) direct competition, where 1 (leader) is the native plant, and 2 (follower) invades the already established canopy.

–(2) Avoidance of competition, where some compromise on resource sharing is reached. This regime can refer to the fair solution discussed in (25-32).

Now (1) corresponds to Stackelberg’s solution (19). As follows from (20), the invader survives if its active leaf area is larger than that of the native plant (ϵ > 1), and it operates with a higher efficiency. Both these features are compatible with the observations on plant competition. Examples of invaders that hold these features are Ligustrum Sinense [67], and Schinus molle [68]. Also, there is a general agreement that a high resource environment (lower a) is more vulnerable to invasion than a lower resource one [63, 69-71], which is compatible with (19, 20). Note that more general models should, in particular, account for the fact that the payoff 𝓦_k of the agents in the competition non-linearly depend on the ratio of energy scales ϵ. Indeed, generally the light obtained by a plant does not depend linearly on its size [63-65, 67].

Another application of this formalism relates to resource sharing in a multi-cellular organisms, where different cells rely on constant nutrients supply. This assumes cooperation between cells and fair allocation of nutrients based on the resource demand of the cells; hence (2) applies. The expensive germ line hypothesis is an example of resource allocation problem between germ and somatic cells in multicellular organisms [39-42]. Germ cells require more nutrients, resulting in a clearly asymmetrical situation as in (17, 18). On the other hand, the lack of resources for somatic cells could cause senescence [41]. Hence, the resource allocation between germ and somatic cells can be considered as a bargaining process over the available resources. Under a shortage of resources, the organism would distribute the resources between germ and somatic cells more evenly, as can be seen in Fig.3, where ∣𝓦₁ − 𝓦₂∣ becomes smaller for larger a.

IV. COMPETITION FOR DEPLETABLE RESOURCES

A. Resource depletion

Above we assumed that the thermal baths are very large, hence their temperatures do not change during the heat engine functioning. Now we focus on finite baths that are still large enough so that their temperature(s) changes on time scales that are much longer than the relaxation time of the engine. Because the hotter bath is the source, we shall assume that its temperature T_h decreases as the result of the work-extraction. For simplicity, the other two baths will be held at a constant temperature.

Eq. (9) shows that once T_h decreases and gets closer to T_c the heat engine will stop functioning at 1 − ϑ = E₂/E₃. This is still compatible with ϑ < 1, i.e. the baths are still not in equilibrium, but this non-equilibrium cannot be employed by the engine for work-extraction.

The slow dynamics of T_h can be deduced from the formula of equilibrium heat-capacity, which governs the change of temperature for a body at equilibrium given the change of its internal energy [44] [cf. [35, 36]]:

$$\frac{d{T}_{\mathrm{h}}}{dt}\equiv {\stackrel{.}{T}}_{\mathrm{h}}=-C_{\mathrm{h}}^{-1}{J}_h$$ (33)

where C_h is the constant heat capacity of the bath. Indeed, C_h is large, since it scales with the number of the bath degrees of freedom. On the other hand, J_h ~ 1, hence $\frac{d{T}_{\mathrm{h}}}{dt}$ is small, i.e. T_h changes slowly.

B. The setup

For simplicity, we assume that E₃ and ρ are the same for both agents: E_{3 1} = E_{3 2} = E₃ and ρ₁ = ρ₂; cf. (9). Under these assumptions, the dimensionless power ${\widehat{J}}_k$ of each agent (k = 1, 2) and the time evolution of temperature ratio ϑ are given by [cf. (9, 8)]

$${\widehat{J}}_k\equiv \frac{J_k}{\mu C_{\mathrm{h}}}=-{\eta }_k(1-\vartheta -{\eta }_k),{\eta }_k=\frac{E_{2k}}{E_3},$$ (34)

$$\stackrel{.}{\vartheta }=2\mu {\vartheta }^2(1-\vartheta -\overline{\eta }),\overline{\eta }\equiv ({\eta }_1+{\eta }_2)∕2,$$ (35)

where (35) is found from (33), η_k are efficiencies of agents, and $\mu \equiv C_{\mathrm{h}}^{-1}{\beta }_{\mathrm{c}}^2\rho E_2^3$ is a constant. We will compare different types of heat engines that employ the same depletable source and compete with each other aiming to increase the stored energy (= extracted work). The heat engine power (34) and its time-integral (stored energy) describe (resp.) short- and long-term advantages of the agent.

We distinguish adaptive and non-adaptive agents in the competition for the same depletable source. The internal structure (difference in energy levels) of non-adaptive agents remains fixed during the time, i.e η_k = const, k = 1, 2 are time-independent. In contrast, adaptive agents continuously tune their time dependent efficiency to the source depletion:

$${\eta }_k={\alpha }_k{\eta }_{\mathrm{C}}={\alpha }_k(1-\vartheta (t)).$$ (36)

The rationale of choosing the specific adaptation protocol (36) comes from the locally equilibrium thermodynamics; e.g. α_k = 1/2 describes the local maximum power according to (10). Appendix C discusses in detail the energy stored by a single agent (adaptive or not).

Adaptation to environmental changes (phenotypic flexibility) is an intrinsic capacity of all organisms [28, 77-81, 94-97]. Adaptive processes are also involved in niche-construction [98-100], where competing agents shape the selection process by altering their own and competitors’ environment. In view of (36) we focus on myopic adaptation, where agents adapt to environmental changes caused by their activity, but do not shape the environment beforehand (that is, do not construct new niches).

We emphasize that (36) is mostly a phenomenological description of adaptation processes. We do not explore internal mechanisms of adaptation, and in particular, we do not account for the full energy cost of maintaining the adaptation; see [37] in this context. The main advantage of (36) (to be explained below) is that allows the heat engine to work till the final depletion of the source, i.e. till θ → 1; cf. the paragraph before (33).

C. Competition between non-adaptive agents

Let us now consider competition of two non-adaptive agents (1 and 2) attached to the same baths. We can take without loss of generality

$${\eta }_1\le {\eta }_2,$$ (37)

Hence 2 will have a finite working time determined from (35) via η₂ = 1 − ϑ(τ₂). We assume that for t > τ₂, 2 is detached from the baths. Otherwise, this agent will not function as a heat engine. Hence the competition for the same source brings in a finite consumption time of functioning as heat engine.

For a given initial value ϑ(0) = ϑ₀, define the stored energies for (resp.) 1 and 2:

$$W_1({\eta }_1,{\eta }_2)={\int }_0^{{\tau }_2}\mathrm{d}t\mid {\widehat{J}}_1\mid +{\int }_{{\tau }_2}^{\infty }\mathrm{d}t\mid {\widehat{J}}_1^{{}^{\prime }}\mid ,$$ (38)

$$W_2({\eta }_1,{\eta }_2)={\int }_0^{{\tau }_2}\mathrm{d}t\mid {\widehat{J}}_2\mid .$$ (39)

W₁(η₁, η₂) in (38) is composed of two terms. The first term in (38) is the stored energy for time interval τ₂, i.e. during the competition. The second term is evaluated once 2 stops work-extraction and 1 operates alone; see Appendix D. Note that the functions $\mid {\widehat{J}}_1(t)\mid$ and $\mid {\widehat{J}}_1^{{}^{\prime }}(t)\mid$ are different due to (35) and the finite working time τ₂. $\mid {\widehat{J}}_1^{{}^{\prime }}(t)\mid$ is given by (34), and found from (35) where evaluation starts at ϑ(τ₂) = 1 − η₂.

For a single agent in the absence of competition, the optimal efficiency of the heat engine is determined from maximizing the stored energy (time-integrated power) [see Appendix C]

$$\eta =1-\sqrt{{\vartheta }_0}.$$ (40)

Eq. (40) is formally similar to the Curzon-Ahlborn efficiency of heat engines [16-18], but the meaning of (40) is different, because (40) follows from the consideration of the full extracted work from a finite bath. We emphasize that (for a single non-adaptive agent) the optimal efficiency (40) is strictly smaller than the maximal (Carnot) efficiency 1 − ϑ₀; cf. (8).

Fig.4 shows the competition results described by (38, 39) for two non-adaptive agents 1 and 2. The energy-storing strategies of each agent are parametrized by their efficiencies η₁ and η₂; recall (37). It is seen that when competing with the efficiency ${\eta }_1=\frac{1-{\vartheta }_0}{2}$ at the maximal initial power [cf. (10)], it is never meaningful for 2 to employ a larger efficiency, i.e. ${\eta }_1=\frac{1-{\vartheta }_0}{2}$ wins over all ${\eta }_2>{\eta }_1=\frac{1-{\vartheta }_0}{2}$; see Fig.4. In particular, ${\eta }_2=1-\sqrt{{\vartheta }_0}$ is not anymore optimal in contrast to the single-agent case (40). However, when competing against ${\eta }_2=\frac{1-{\vartheta }_0}{2}$ it is beneficial for 1 to employ certain smaller efficiencies ${\eta }_1<{\eta }_2=\frac{1-{\vartheta }_0}{2}$.

FIG. 4:

Two non-adaptive agents 1 and 2 compete for a depletable source. Their stored energies (payoffs) are given (resp.) by (38) and (39), while the work-extracting strategies are parametrized by efficiencies η₁ and η₂. According to (8, 37), η₁ and η₂ hold 1 − θ₀ > η₂ > η₁. Three situations are shown on the same graph. For ϑ₀ = 0.25 blue-dotted lines delimit the domain 0.75 > η₂ > η₁ > 0, while the blue-shadowed domain indicates on W₂(η₁, η₂) > W₁(η₁, η₂). The rest of the domain delimited by blue-dotted curves refers to W₂(η₁, η₂) < W₁(η₁, η₂). Likewise, green and red lines and shadowed domains refer to ϑ₀ = 0.5 and ϑ₀ = 0.25, respectively. Thus on curves a, b and c we have W₂(η₁, η₂) = W₁(η₁, η₂) for (resp.) ϑ₀ = 0.25, 0.5, 0.75. The blue, red, green points refer to ${\eta }_1=\frac{1-{\vartheta }_0}{2}$ and ${\eta }_2=1-\sqrt{{\vartheta }_0}$ for (resp.) ϑ₀ = 0.25, 0.5, 0.75. The blue, red, green rectangles illustrate the outcomes of emergent prisoner’s dilemma subgame (43– 48) obtained through the best response analysis of the stored energies (38, 39).

D. Best responses and emergent prisoner’s dilemma

1. Best response cycle

Once W₁ and W₂ in (38, 39) depend on both η₁ and η₂, the notion of the optimal efficiency is to be studied via game theory. Here the stored energies W₁ (η₁, η₂) and W₂ (η₁, η₂) are payoff functions of (resp.) agents 1 and 2, and the choice of η₁ and η₂ (strategy profile) refers to their actions.

We can determine the best responses of this game that eventually leads to Nash equilibrium. Starting from an arbitrarily initial point ${\eta }_1^{[0]}$ an iterative process $({\eta }_1^{[0]},{\eta }_2^{[0]},{\eta }_1^{[1]},{\eta }_2^{[1]},\dots )$ of successive best responses is defined recalling (38, 39) and (37):

$${\eta }_2^{[k]}={\text{argmax}}_{\eta \ge {\eta }_1^{[k]}}[W_2({\eta }_1^{[k]},\eta )],k\ge 0,$$ (41)

$${\eta }_1^{[k+1]}={\text{argmax}}_{\eta \le {\eta }_2^{[k]}}[W_1(\eta ,{\eta }_2^{[k]})],k\ge 0.$$ (42)

This iteration converges to a 4-cycle that is independent from the initial point ${\eta }_1^{[0]}$:

$${\eta }_1^{\prime }\to {\eta }_2^{\prime }\to {\eta }_1^{″}\to {\eta }_2^{″}\to {\eta }_1^{\prime },$$ (43)

$${\eta }_1^{\prime }<{\eta }_2^{\prime }={\eta }_1^{″}<{\eta }_2^{″},$$ (44)

where ${\eta }_1^{\prime }\to {\eta }_2^{\prime }$ means that ${\eta }_2^{\prime }$ is the best response to ${\eta }_1^{\prime }$. Hence the agents can apply the following pairs of strategies:

$$({\eta }_1^{\prime },{\eta }_2^{\prime }),({\eta }_1^{″},{\eta }_2^{\prime }),({\eta }_1^{″},{\eta }_2^{″}),({\eta }_1^{\prime },{\eta }_2^{″}).$$ (45)

We emphasize that the efficiency values in (44) depend only on ϑ₀ and on the initial assumption η₂ ≥ η₁ in (38, 39). For example, at ϑ₀ = 0.5 we have ${\eta }_1^{\prime }=0.151$, ${\eta }_2^{\prime }={\eta }_1^{″}=0.161$, and ${\eta }_2^{″}=0.179$. Hence, the long-time best response in this game is not a unique Nash equilibrium, but rather the 4-cycle (44). For ϑ₀ → 0 (potentially large resources), we get that all efficiencies in (44) converge to 2/3. In contrast, for ϑ₀ → 1 they all converge to zero. Fig. 4 shows the location of (45) on the (η₁ ≤ η₂) plane for several values of ϑ₀; see rectangles in Fig. 4. It is seen that they are close to the η₁ = η₂ line.

Hence, the best response dynamics converges to the limit cycle. This is an asymptomatically stable state, because any deviation converges back to the cycle. Note that the above sequential best response algorithm does not ensure that all Nash equilibria of the game are found. Indeed, cycles of best response are widely observed in generic two-player dynamic games [62]. For adaptive agents studied below, there is no limit cycle due to the existence of global maxima of the payoff functions.

2. Emergent prisoner’s dilemma

Now note that

$$W_i({\eta }_1,{\eta }_2)<W_i\left(1-\sqrt{{\vartheta }_0},1-\sqrt{{\vartheta }_0}\right),i=1,2,$$ (46)

where (η₁, η₂) assumes any of 4 (i.e. any of the allowed) pair in (45). Eq. (46) is non-trivial, because it shows that the joint application of the best response strategies (44) loses to the joint application of the efficiency $1-\sqrt{{\vartheta }_0}$ that is optimal for a single agent; cf. (40).

However, this feature of $1-\sqrt{{\vartheta }_0}$ is unstable due to

$$W_1\left(\eta ,1-\sqrt{{\vartheta }_0}\right)>W_1\left(1-\sqrt{{\vartheta }_0},1-\sqrt{{\vartheta }_0}\right),$$ (47)

$$\eta ={\eta }_1^{\prime },{\eta }_1^{″}.$$ (48)

Eq. (47) means that switching from $1-\sqrt{{\vartheta }_0}$ to ${\eta }_1^{\prime }$ or ${\eta }_1^{″}$ is beneficial for 1. Now 2 responds to this switch in the best way, and the agents find themselves within actions (45) that are worst than $\left(1-\sqrt{{\vartheta }_0},1-\sqrt{{\vartheta }_0}\right)$ due to (46).

This situation does resemble the prisoner’s dilemma [31, 34, 53, 55, 82], where the agents following by best response strategies end up in the worse situation compared with the cooperative behavior, which for our case refers to (46). We emphasize that, in contrast to the standard prisoner’s dilemma, here the best-response strategies are not unique and amount to 4-cycle (43); see rectangles in Fig.4. Note that the 4-cycle also contains a step with equal efficiencies ${\eta }_2^{\prime }={\eta }_1^{″}$.

E. Adaptive agents

1. Competition between two adaptive agents

Let us consider now the competition between adaptive heat engines (36). In the absence of competition (i.e. for a single agent), the optimal value of α in (36) obtained from maximizing the total stored energy (extracted work) is close to the maximal Carnot efficiency (8), i.e. α → 1 in (36); see Appendix C. Indeed, the single adaptive agent (36) works as long as ϑ ≠ 1 (till the full depletion of the resource). Hence α → 1 minimizes the losses in the energy storage process. Note the difference with (40), where the optimal efficiency for the non-adaptive agent was strictly smaller than the Carnot efficiency. In this sense the usual premise of thermodynamics that the maximal efficiencies would be useful applies to the (single) adaptive agent, which implies advanced functioning mechanisms, as compared to the non-adaptive situation.

For the competition of two adaptive agents 1 and 2 with α₁ and α₂ in (36), we get for stored energies [see Appendix D]

$$\begin{gathered} W_k({\alpha }_1,{\alpha }_2)={\int }_0^{\infty }\mathrm{d}t\mid {\widehat{J}}_k(t)\mid \\ =\frac{{\alpha }_k(1-{\alpha }_k)}{2\mu (1-\overline{\alpha })}\left(\frac{1}{{\vartheta }_0}-1+\text{ln}{\vartheta }_0\right),k=1,2, \end{gathered}$$ (49)

where ϑ₀ is the initial ratio of temperatures, and $\overline{\alpha }\equiv ({\alpha }_1+{\alpha }_2)∕2$. Eq. (49) shows that the agent with α_k closer to 1/2 wins. Now (36) and (10) imply that α_k = 1/2 is (locally) the maximum power regime. Note that similar maximum-power regimes were proposed as an operating principle for ecological and biological systems (including living organisms) [3]. Here, we see that the optimality of maximum-power regime is closely related to the adaptation abilities of agents.

Adaptive agents do face the prisoner’s dilemma, but the situation here is simpler than that discussed in (43-(48). If α₁ = α₂ → 1, then $\frac{1-{\alpha }_i}{1-\overline{\alpha }}\to 1$ in (49), and each one will store more energy compared to the case when both operate at the maximum power α_k = 1/2. This situation, again, is unstable: if, for example, the first agent switches to the maximum power regime α₁ = 1/2 (with the second agent working at α₂ → 1), then the first agent will store more energy.

2. Adaptive agent competing with a non-adaptive one

Now we discuss the competition of an adaptive agent (36) against an agent with a time-independent efficiency η. Appendix D studies this situation in detail. Here we summarize the main results.

The adaptive engine operates till the full depletion of the resource. The non-adaptive agent will operate during a finite time τ (see Appendix D for details), i.e. the adaptive agent will still function alone for t > τ. Fig. 5 shows the competition results: the adaptive agent wins whenever it competes against sufficiently small or large values of η. Whenever η is around the initial maximal power value (1 − ϑ₀)/2 [cf. (10)], also α should be around the adaptive maximal power regime α = 1/2 for adaptive agent to win. In particular, $\alpha =\frac{1}{2}$ wins against any η, while α → 1 (i.e. the optimal α for a single agent without competition) looses to any η; see Fig.5. However, as Fig.6 shows, $\alpha =\frac{1}{2}$ is not the value, where the stored energy of the adaptive agent is maximized.

FIG. 5:

Competition between adaptive agent with efficiency η_ad = α(1 − ϑ(t)) for α ∈ [0, 1] and a non-adaptive agent with fixed efficiency η ∈ [0, 1 − ϑ₀]; cf. (36, 8). The initial temperature ratios are ϑ₀ = 0.1 (blue) and ϑ₀ = 0.25 (red). In shaded regions the adaptive agent extracts more total work than the fixed-η agent. Dotted curves show the line $\eta =1-\sqrt{{\theta }_0}$ for ϑ₀ = 0.1 (blue) and ϑ₀ = 0.25 (red).

FIG. 6:

Competition between adaptive agent with parameter α in (36) and a non-adaptive agent with a fixed efficiency η. The figure shows stored energies—blue and red for (resp.) adaptive and non-adaptive agents—versus α for a fixed η. Full (dashed) curves refer to $\eta =\frac{1-{\vartheta }_0}{2}(\eta =1-\sqrt{{\vartheta }_0})$, where the initial temperature ratio ϑ₀ = 0.1 and μ = 1 in (35).

Recall that $\eta =1-\sqrt{{\vartheta }_0}$ and α ≈ 1 determine the optimal efficiencies for the single (without competition) non-adaptive and adaptive agent, respectively. Fig.5 shows that if these values are kept under competition, then the non-adaptive agent wins.

F. Outlook

ATP stores energy: it is metastable at physiological conditions, and has the half life-time of several hours, after which it hydrolyses spontaneously dissipating the stored energy. Thermodynamically, ATP is similar to a high-temperature bath that stores energy at temperature T [47, 48]; cf. the heat-engine model (1-10).

ATP production in cells is an example of energy extraction and storage that occurs via fixed biochemical pathways [85]. Two such pathways of ATP production from glucose are well-known: fermentative (aerobic and anaerobic) and oxidative respiration. There is a form of power-efficiency dilemma here: oxidation pathway results to 18 times greater ATP molecules than by fermentation path, while the fermentation pathway is faster (100 times in muscle cells) and hence is more powerful [76, 83, 85]. These pathways have been extensively studied in competitive environments, both in yeasts [31, 32, 34, 83, 84, 86] and in solid tumor cells [90-93].

We suggest that ATP production paths refer to agents with different internal structure that eventually determines the difference in power and the overall stored energy. Our results in section IV C mean that competition favors lower efficiencies. This is indeed observed in yeasts and tumor cells [31, 32, 34, 83, 84, 86] and also in solid tumor cells [90-93]. Whether also higher powers are favored in our model is a more convoluted question. It does have a straightforward positive answer for the adaptive situation, where the maximal power regime is well-defined at all times.

Note that whenever the resources are shared under a centralized control—i.e. the competion is eliminated and we effectively have a single agent—the optimal efficiency is higher than the efficiency at the maximum power; see (40) and (10). Here the optimal efficiency is defined as the maximizer of the total extracted work. This finding can explain why the oxidative respiration is effectively ubiquitous in multicellular organisms, whereas when the control is lost (e.g. in tumors) cells may switch to fermentative ATP production.

Ref. [31] developed a dynamic model for several populations feeding on a common infinite (i.e. constantly renewable) source. The resource extracted by a population is equalized to its growth per capita. While the model does not hold the laws of thermodynamics (e.g. nothing prevents larger than one efficiencies), it demonstrates the exclusion rule: only the population with the largest power of extraction survives. The size of the survived population is determined by the efficiency of resource extraction. But the proper power-efficiency trade-off is absent [cf. section II], the populations anyhow strive for largest efficiency. It was proposed that this exclusion rule can apply to evolution of ATP production pathways and explain the emergence of multi-cellular organisms: alike cells gather together, thereby exclude competitors and develop less powerful, but more efficient mechanisms of resource extraction [31]. Some results of Ref. [31], e.g. on the relevance of power during the competition, broadly agree with our analysis. However, since our model explicitly agrees with thermodynamics, it provides a richer perspective, e.g. the power uniquely determines the competition outcome only for the adaptive situation. Also, the strict exclusion rule need not hold.

The standard prisoner’s dilemma game was also discussed in the context of agents operating by different ATP production paths [34, 83]. In our situation, an effective sub-game that resembles (but is more complicated than) the prisoner’s dilemma emerges out of the thermodynamic competition between two agents. It is likely that an effective prisoner’s dilemma situation is a reduced description for a more general class of models for agents exploiting the same finite source in accordance with the laws of thermodynamics. We shall explore this hypothesis elsewhere.

V. SUMMARY

The metabolism of living organisms obeys the laws of thermodynamics, while the energy extraction and storage mechanisms are subject to evolutionary pressure [12]. In this work we studied competition between energy-extracting thermodynamic agents modeled as heat engines.

Efficiency and power are two main characteristics of heat engines. They are complementary to each other [16–20]. This known power-efficiency tradeoff [16-20] is also observed in various biological systems [24-34]. The biological situation is however fundamentally more complex, since it involves adaptation to the source, various channels of using the extracted energy, their mutual feedback etc.

Models describing these processes in detail are yet to be developed. Here we treat competing agents as heat engines that extract energy (work) from the available source (high-temperature bath). We focus on the linear thermodynamic regime, where the implementation details of the engine are not essential. Two general scenarios for selection are considered: competition for a fixed energy current (the source is then effectively infinite) and for a depletable (finite) source. In both cases, agents compete indirectly, i.e. the interaction between agents is resource mediated. The optimization targets for these scenarios are (resp.) the power (= energy extracted per time unit) and the stored (= total extracted) energy. Both are relevant biologically.

Competition for the fixed energy current is considered under two known set-ups of game theory: Stackelberg equilibrium [53, 55, 57] and Pareto optimality [54-56]. Once photosynthesis is a heat engine operating between a hot thermal bath (photons generated by Sun) and the cold thermal bath (Earth environment), these set-ups are relevant for plants competing for light. We show that Stackelberg’s set-up reproduces features observed in invading plants [63-73]. Pareto optimality assumes weak cooperative behavior of agents, where the resource allocation is fair and is based on the demands of agents. We implemented the Nash bargaining program [54-56], as a solution for the fair allocation problem. We observed that the optimal efficiencies of competing agents are higher than the optimal efficiency of a single agent (without competition). This is in stark contrast to what happens in the competition for a finite (depletable) source; see below. The proposed scenario of fair resource allocation is relevant for multicellular organisms, e.g. the allocation between germline and soma cells [39-42].

We further examine adaptive and non-adaptive agents competing for depletable resources (finite high-temperature thermal bath). The difference between the two types of agents is the ability of the former to alter its internal structure adjusting it to changes in the exploitable source. Properties of the non-adaptive agent remain fixed in time. The fermentative and oxidative ATP production in cells, which are among the most fundamental, universal biochemical pathways in all life forms, are examples of energy production processes that proceed along fixed pathways [85]. When these alternative pathways of ATP production are considered as competing agents, the adaptation ability of agents is directly related to the phenotypic adaptation of organisms [28, 77-81, 94-97].

The optimal efficiencies of the work-extraction process differ for adaptive and non-adaptive agents. The adaptive agent wins over other agents if its efficiency is that of the maximal power. This however not the efficiency value that maximizes the stored energy in case of competition with non-adaptive agent. The optimal efficiency for the agent with a fixed structure has non-unique (and richer) optimal values. One reason behind this difference is that adaptive agents compete until the final depletion of the sources, whereas non-adaptive agents are unable to do so. We stress that our treatment of adaptation is to some extent formal, since it so far does not account for its full energy cost [37]: future models should show how to direct a part of the stored energy to the needs of adaptation [38].

One general outcome of our model is that agents competing for stored energy face analogues of the classical prisoner’s dilemma [31, 34, 53, 55, 82]. Here the cooperative behavior refers to efficiencies that are optimal in the absence of competition (i.e. for a single agent). The mutually cooperative behavior is still beneficial under competition, but it is not stable, when one agent employs a smaller efficiency (defection). This puts the agents in the best response cycle, where they store a smaller amount of energy. It is possible that prisoner’s dilemma is a general consequence of thermodynamic laws applied to exploitation of a depletable resource.

Overall, we found that simple models with no special assumption beyond the laws of thermodynamics can recapitulate certain features of biological evolution.

Appendix A: Markov model for heat engine: Structure, Power and Efficiency.

Here we use slightly different notations as compared to the main text. The notations are related as follows, ρ_i←j ≡ ρ_ij, and

$$\begin{gathered} T\equiv T_{21},T_h\equiv T_{31},T_c\equiv T_{32}, \\ \beta \equiv {\beta }_{21},{\beta }_h\equiv {\beta }_{31},{\beta }_{\mathrm{c}}\equiv {\beta }_{32}, \\ J\equiv J_{21},J_h\equiv J_{31},J_c\equiv J_{32}, \end{gathered}$$

Let us recall the simplest model of heat engine [37]. The model has three states i = 1, 2, 3, which is the minimal number of states a stationary operating heat engine can have, because it ought to be in a non-equilibrium state (i.e. to support one cyclic motion), and because it has to support three external objects: one work-source and two thermal baths. Each state i has energy E_i. Transitions between different states are caused by thermal baths that can provide or accept necessary energies. We assume that the resulting dynamics is described by a Markov master equation

$${\stackrel{.}{p}}_i\equiv \frac{d{p}_i}{dt}={\sum }_j[{\rho }_{ij}{p}_j-{\rho }_{ji}{p}_i],i,j=1,2,3,$$ (A1)

where p_i is the probability to find the system in state i at time t, and where ρ_ij is the transition rate from state j to state i.

We assume that each pair of transitions i ← j and j ← i is caused by thermal baths T_ij = T_ji = 1/β_ji, which are in thermal equilibrium (this point will be clarified latter, when we discuss temperature changing process). The equilibrium nature of thermal baths impose detailed balance condition on the transitions [43]

$${\rho }_{ij}{e}^{-{\beta }_{ij}{E}_j}={\rho }_{ji}{e}^{-{\beta }_{ij}{E}_i},{\beta }_{ij}={\beta }_{ji}.$$ (A2)

We take one temperature infinite [37]: β₂₁ = 0. This bath is then a work-source. This important point is explained via the following related arguments. First, note that an infinite temperature thermal bath exchanges energy without altering its own entropy, which is a feature of mechanical device (i.e. sources of work) [37]. Indeed, due to the equilibrium thermodynamic relation dS₂₁ = β₂₁dQ₂₁ = 0 it exchanges energy dQ₂₁ ≠ 0 at zero entropy change dS₂₁ = 0. Second, if an infinite temperature thermal bath interacts to any (positive) temperature bath, then the former bath always looses energy indicating on no additional costs for the transfer of energy that was stored at an infinite temperature. In that sense, the latter energy is freely convertible to any form of heat, as expected from work.

For the thermal bath with temperature T_ij, we define J_ij as the average energy lost by the bath (for J_ij > 0) or gain (J_ij < 0) per unit of time. Since each bath causes only one pair of transition we get

$$J_{ij}=J_{ji}=(E_i-E_j)({\rho }_{ij}{p}_j-{\rho }_{ji}{p}_i),$$ (A3)

In the stationary (but generally non-equilibrium) state the average energy of the three-levels system is constant:

$$\sum _{i=1}^3{\stackrel{.}{p}}_i{E}_i=0,$$ (A4)

and hence we get from (A2, A3, A4) that the sum of energy current nullifies:

$$J_{12}+J_{23}+J_{13}=0,$$ (A5)

which is the first law of thermodynamics in the stationary state [44].

The stationary probabilities p_i are found from (A1):

$$p_i=\frac{1}{Ƶ}[{\rho }_{ij}{\rho }_{ik}+{\rho }_{ij}{\rho }_{jk}+{\rho }_{ik}{\rho }_{kj}],$$ (A6)

where i ≠ j ≠ k, i, j, k = 1, 2, 3.

Using (A2,A3,A6) and noting that ρ₁₂ = ρ₂₁ due to the assumed condition β₁₂ = β₂₁ = 0, we obtain for the average energy currents

$$J_{21}=\frac{{\widehat{E}}_2}{Ƶ}{\rho }_{21}{\rho }_{13}{\rho }_{32}\left[1-e^{({\beta }_{32}-{\beta }_{31}){\widehat{E}}_3-{\beta }_{32}{\widehat{E}}_2}\right],$$ (A7)

$$J_{31}=-{\widehat{E}}_3{J}_{21}∕{\widehat{E}}_2,J_{32}=({\widehat{E}}_3-{\widehat{E}}_2)J_{21}∕{\widehat{E}}_2,$$ (A8)

$${\widehat{E}}_2\equiv E_2-E_1,{\widehat{E}}_3\equiv E_3-E_1.$$ (A9)

The heat engine functioning is defined as

$$0>J_{21}=-(E_2-E_1)(p_2-p_1){\rho }_{12},$$ (A10)

i.e. the infinite-temperature bath gains energy. Eq. (A7) implies that for the heat engine functioning it is necessary that

$${\widehat{E}}_2[(1-\vartheta ){\widehat{E}}_3-{\widehat{E}}_2]>0,\vartheta \equiv {\beta }_{31}∕{\beta }_{32}.$$ (A11)

We will assume that ϑ < 1, that is to say T₃₁ (T₃₂) is the temperature of the hot (cold) bath. Eq.(A11) shows that the system will operate as a heat engine for a given ϑ < 1 if ${\widehat{E}}_3>{\widehat{E}}_2$.

The efficiency of any heat engine is defined as the result (i.e. the extracted work) divided over the resource (i.e. the energy coming from the hot bath). Under ϑ < 1 and ${\widehat{E}}_3>{\widehat{E}}_2$ the efficiency η amounts to

$$\eta \equiv \frac{-J_{21}}{J_{31}}=\frac{{\widehat{E}}_2}{{\widehat{E}}_3}\le {\eta }_{\mathrm{C}}\equiv 1-\vartheta ,$$ (A12)

i.e. the efficiency is bounded from the above by the Carnot efficiency η_C. Hence, (A12) is the general message of the second law for the heat engine efficiency [44].

Another important message of (A12,A11,A7) is the power-efficiency trade-off: at the maximal efficiency the power −J₁₂ of the heat engine nullifies. This trade-off is also a general feature of heat engines [19], though it is frequently missed in phenomenological treatments of equilibrium thermodynamics [44]. This trade-off is one pertinent reason for having an explicit microscopic model of a heat engine [19].

a. High temperature regime.

The work power J₁₂ in (A7) depends on the specific form of the transition rates ρ_ij that enter the detailed balance condition (A2). The form of ρ_ij depends on the physical implementation of the model; e.g. the Arrhenius form of ρ_ij applies in chemical reaction dynamics [43]. We shall work in the high-temperature limit, where the details of ρ_ij are not important provided that it holds the detailed balance (A2). Now β₃₂ ≪ 1 and β₃₁ ≪ 1 are sufficiently small, i.e. E_iβ₃₂ ≪ 1 and E_iβ₃₁ ≪ 1, but 0 ≤ ϑ ≤ 1 in (A11) can be arbitrary. In this limit the power of heat engine reads from (A7):

$$J_{21}=-\rho {\beta }_{32}{\widehat{E}}_2((1-\vartheta ){\widehat{E}}_3-{\widehat{E}}_2),$$ (A13)

where we denoted $\rho =\frac{1}{Ƶ}({\rho }_{21}{\rho }_{13}{\rho }_{32}){\mid }_{{\beta }_{31}={\beta }_{32}=0}$. We shall assume that ρ is a constant. Note that for β₃₁ = β₃₂ = 0 we get ρ_ij = ρ_ji from (A2). Hence in this limit we get p_i = 1/3; cf. (A6). Now J₂₁ ≠ 0 in (A13) due to small deviations of p_i from 1/3 that exist due to a small but finite β₃₁ and β₃₂ = 0.

In the high-temperature regime our model is fully consistent with linear equilibrium thermodynamics [18, 52]. Structurally similar models were widely employed in bioenergetics for modeling coupled chemical reactions, where the passage of heat from higher to lower temperatures corresponds to the down-hill reaction, where the work-extraction refers to the up-hill reaction [52]. However, we emphasize that a microscopic model clarifies the status of the involved parameters, as well as demonstrates explicitly that the extracted work relates to the stored energy. These two important aspects are not clear within the phenomenological introduction of linear thermodynamics models.

Appendix B: Competition for a fixed energy current

Two agents competing for fixed resources can be described as two heat engines [cf. (A13)]:

$$J_{21}^{[k]}=-\rho {\beta }_{32}{\widehat{E}}_2^{[k]}((1-\vartheta ){\widehat{E}}_3^{[k]}-{\widehat{E}}_2^{[k]})$$ (B1)

$$=-\rho {\beta }_{32}{\widehat{E}}_3^{[k]2}{\eta }_k(1-\vartheta -{\eta }_k),k=1,2,$$ (B2)

where ${\eta }_k={\widehat{E}}_2^{[k]}∕{\widehat{E}}_3^{[k]}$ are the efficiencies [cf. (8)]. Note that for the photosynthesis θ ≃ 10⁻³ is a small number.

We write this constraint (11)as

$$\frac{\mid J_{21}^{[1]}\mid }{{\eta }_1}+\frac{\mid J_{21}^{[2]}\mid }{{\eta }_2}\le A^{\prime },\text{or},$$ (B3)

$${\widehat{E}}_3^{[1]2}(1-\vartheta -{\eta }_1)+{\widehat{E}}_3^{[2]2}(1-\vartheta -{\eta }_2)\le A^{″},$$ (B4)

where A′ and A″ are positive constants.

a. Game-theoretical approach: actions and utilities

$${\eta }_1+ϵ{\eta }_2\ge A,ϵ\equiv ({\widehat{E}}_3^{[2]}∕{\widehat{E}}_3^{[1]}{)}^2.$$ (B5)

To simplify the parametrization of the problem we denote

$$x_k={\eta }_k∕(1-\vartheta ),k=1,2,$$ (B6)

and write the extracted works as

$$-J_{21}^{[k]}=\rho {\beta }_{32}{\widehat{E}}_3^{[1]2}(1-\vartheta {)}^2{𝒲}_k,k=1,2,$$ (B7)

$${𝒲}_1=x_1(1-x_1),{𝒲}_2=ϵx_2(1-x_2).$$ (B8)

The problem described by (B5-B8) has a game-theoretic interpretation, where 𝓦₁ and 𝓦₂ refer to utilities of each agent, and (B5) ensures that their actions x₁ and x₂ are coupled. Hence, for A in (B5) we shall assume

$$\frac{(1+ϵ)(1-\vartheta )}{2}<A<(1+ϵ)(1-\vartheta ).$$ (B9)

The first inequality here means that the agent cannot simultaneously maximize 𝓦₁ and 𝓦₂ at η₁ = η₂ = (1 − ϑ)/2. The second inequality means that the constraint (B5) still allows some η₁ < 1 − ϑ and η₂ < 1 − ϑ. Introducing a new constant via A = a(1 + ϵ)(1 − ϑ) we conveniently write (B5) as

$$x_1+ϵx_2\ge a(1+ϵ),1\ge a\ge 1∕2,$$ (B10)

where the last inequalities follow from (B9).

b. Pareto line and the bargaining equilibrium

Pareto’s concept of equilibrium assumes a certain level of passive cooperation between 1 and 2. Now 1 does not increase its utility 𝓦₁ without decreasing the utility 𝓦₂ of 2, and likewise 2 does not increase 𝓦₂ without decreasing 𝓦₁ [54-56]. This contrasts to Stackelberg’s solution, where the first moving agent chooses the optimal condition for itself, irrespective of what happens to another agent. Not surprisingly, Pareto’s concept does not provide any unique choice of (x₁, x₂), and an additional reasoning is needed for selecting a unique outcome among Pareto solutions.

It is seen from (B8, B10) that the set of Pareto equilibria (i.e. the Pareto line) is given by the following three conditions

$$\begin{gathered} x_1+ϵx_2=a(1+ϵ),0\le x_k\le 1,k=1,2, \\ {x}_2\in [\max [\frac{1}{2},\frac{1}{ϵ}(a(1+ϵ)-1)]\phantom{]},\phantom{[} \\ \text{min}[\frac{1}{ϵ}(a(1+ϵ)-1),1]]. \end{gathered}$$ (B11)

Note that there is a continuum of Pareto equilibria. Moreover, for this game all Pareto equilibria are also Nash equilibria. Indeed, recall that the Nash equilibrium looks for a pair $(x_1^{\ast }(x_2),x_2^{\ast }(x_1))$ such that $x_1^{\ast }(x_2)$ is the best response to x₂ (i.e. the conditional maximum of 𝓦₁ given x₂), while $x_2^{\ast }(x_1)$ is the best response to x₁ [54-56].

Nash’s bargaining solution is one approach for making a unique choice within (B11) [54-56, 59]. For defining the bargaining target we need to find the worst utility for each agents:

$${𝒲}_1^{\ast }={\max }_{x_1}{\text{min}}_{x_2}[{𝒲}_1],{𝒲}_2^{\ast }={\max }_{x_2}{\text{min}}_{x_1}[{𝒲}_2].$$ (B12)

To find out ${𝒲}_1^{\ast }$, note from (B10) that any choice x₁ of 1 should hold x₁ ≥ a(1 + ϵ) − ϵx₂. If 2 arranged its action x₂ such that a(1 + ϵ) − ϵx₂ ≥ 1, then there is no choice to be made by 1. For such cases we shall define its utility 𝓦₁ = 0, and this is clearly the worst outcome for 1. Now a(1 + ϵ) − ϵx₂ ≥ 1 or equivalently [a(1 + ϵ) − 1]/ϵ ≥ x₂ will work provided that a(1 + ϵ) − 1 ≥ 0. Hence, if the latter condition holds we get 𝓦₁ = 0. Otherwise, if a(1 + ϵ) − 1 < 0, then 2 achieves min_x2 [𝓦₁] when the bound x₁ ≥ a(1 + ϵ) − ϵx₂ is possibly tight, i.e. for x₂ = 0, and then x₁ = a(1 + ϵ) = argmax_x1 min_x2[𝓦₁]. Altogether, we get

$${𝒲}_1^{\ast }=a(1+ϵ)\max [0,1-a(1+ϵ)].$$ (B13)

${𝒲}_2^{\ast }$ is calculated in the same way:

$${𝒲}_2^{\ast }=a(1+{ϵ}^{-1})\max [0,ϵ-a(1+ϵ)].$$ (B14)

Note that only one among ${𝒲}_1^{\ast }$ and ${𝒲}_2^{\ast }$ can be non-zero. Indeed, from (B13) and (B14) it follows that ${𝒲}_1^{\ast }$, ${𝒲}_2^{\ast }>0$ if $\frac{a}{1-a}<ϵ<\frac{1-a}{a}$ which is impossible since $1\ge a\ge \frac{1}{2}$. The fact of e.g. ${𝒲}_1^{\ast }>0$ is important, since it means that 1 cannot be eliminated from the competition. Now (B13, B14) are not simply the separate global minima of (𝓦₁, 𝓦₂), i.e. generally $({𝒲}_1^{\ast },{𝒲}_2^{\ast })\ne (0,0)$, e.g. because generally ${𝒲}_1^{\ast }=0$ is not reached for any action of 2. (Still we get $({𝒲}_1^{\ast },{𝒲}_2^{\ast })=(0,0)$ for ϵ = 1.)

Now the Nash bargaining solution $({\widehat{x}}_1,{\widehat{x}}_2)$ is found from maximizing the geometric mean of ${𝒲}_1-{𝒲}_1^{\ast }$ and ${𝒲}_2-{𝒲}_2^{\ast }$ [58]:

$${\max }_{x_1,x_2}[({𝒲}_1(x_1)-{𝒲}_1^{\ast })({𝒲}_2(x_2)-{𝒲}_2^{\ast })],$$ (B15)

where (x₁, x₂) should vary along all allowed values defined by x₁ + ϵx₂ ≥ a(1 + ϵ), 0 ≤ x_k ≤ 1, and by the values corresponding the worst outcomes of the players ${𝒲}_k^{\ast }$. As it is mentioned above, it is possible that in the worst case 1 will not have a choice for the given action of 2. That is, the action of 1 will be lie outside of the allowed region. Thus, we obtain the following region of maximization for (B15), where the worst outcomes are also take an account.

$$\max [0,(a(1+ϵ)-1)∕ϵ]<x_2<\text{min}[1,a(1+{ϵ}^{-1})].$$ (B16)

Note, that the last bounded region includes the Pareto line of the problem (B11). One may even maximize (B15) over the Pareto line, since the outcome of the maximization will lie on that line [58, 59].

Eq. (B15) maximizes a mean, since 1 and 2 are assumed to be equivalent, and it is the geometric mean, since generally dimensions of utilities 𝓦₁ and 𝓦₂ can be different. Ref. [58] discusses axioms of the bargaining solution and shows that they are consistent with thermodynamics.

Appendix C: Strategies of work-extraction. Depletable resources.

The dynamics of T₃₁ can be deduced from the formula of equilibrium heat-capacity, which governs the change of temperature for an equilibrium body given the change of its internal energy [44]. Once the changes of T₃₁ are assumed to be slow:

$$\frac{d{T}_{31}}{dt}=-C^{-1}{J}_{31}$$ (C1)

where C = C₃₁ is the heat capacity of thermal baths. Indeed, C is large, since it scales with the number of the bath degrees of freedom. On the other hand, J₃₁ ~ 1, hence $\frac{d{T}_{31}}{dt}$ is small, i.e. the change of T₃₁ is slow. We shall make a natural assumption that it is much slower than the relaxation of the engine to its stationary state (A6).

Taking into an account that β₃₂ = const, we find for ϑ = β₃₁/β₃₂ from (A13, A8):

$$\frac{d\vartheta }{dt}=C^{-1}{\beta }_{32}^2\rho {\vartheta }^2{\widehat{E}}_3((1-\vartheta ){\widehat{E}}_3-{\widehat{E}}_2).$$ (C2)

Different strategies of work-extraction refer to differences in its structural parameters ${\widehat{E}}_2$ and ${\widehat{E}}_3$.

a. Non-adaptive agent

Here we discuss the case, when the heat engine is adapted to the initial temperature ratio, i.e the efficiency η of the heat engine remains fixed in the course of time. Eq. (A13) shows that for a fixed ϑ the maximum power of heat engine (i.e. the maximum of ∣J₂₁∣) is attained for

$$\frac{{\widehat{E}}_2}{{\widehat{E}}_3}=\frac{1-\vartheta }{2}.$$ (C3)

Eq. (C3) implies that the maximal (Carnot) efficiency is attained for $\frac{{\widehat{E}}_2}{{\widehat{E}}_3}=1-\vartheta$. Note that in various comparisons between different engine structures it is sensible to keep the difference ${\widehat{E}}_3=E_3-E_1$ fixed and vary only ${\widehat{E}}_2$. Indeed, in the considered regime ${\widehat{E}}_3>{\widehat{E}}_2>0$, ${\widehat{E}}_3$ is the difference between the maximal and minimal energies of the heat engine. Hence fixing ${\widehat{E}}_3$ means to fix the global energy scale.

$${\widehat{J}}_{21}=-\eta (1-\vartheta -\eta ),{\widehat{J}}_{21}=J_{21}∕(\rho {\beta }_{32}{\widehat{E}}_3^2),$$ (C4)

$$\stackrel{.}{\vartheta }=\mu {\vartheta }^2(1-\vartheta -\eta ),\mu =C^{-1}{\beta }_{32}^2\rho {\widehat{E}}_3^2.$$ (C5)

For a constant efficiency of the heat engine η, which includes the cases where the engine is maximizes ∣J₁₂∣ for the initial resource, (C5) is solved as

$$\vartheta (t)=\frac{1-\eta }{1+W\left[\left(\frac{1-\eta }{{\vartheta }_0}-1\right)e^{\frac{1-\eta }{{\vartheta }_0}-1-\mu (1-\eta {)}^{2}t}\right]},$$ (C6)

where ϑ₀ = ϑ(0) is the initial value, and W[z] is Lambert’s function (or product logarithm), which solves equation z = We^W. Note that W[z] → z for z → 0, which determines the long-time behaviour ϑ(t) → (1 − η) of (C6), in this limit the power of of the heat engine nullifies since the efficiency of the heat engines becomes equal to the Carnot level η = η_C = 1 − ϑ(t).

The full dimensionless stored energy is found from (C4) and C6) as The full dimensionless stored energy is found from (C4) as

$$\begin{gathered} -{\int }_0^{\infty }\mathrm{d}t{\widehat{J}}_{21}(t)=\eta (1-\eta ){\int }_0^{\infty }\mathrm{d}t\left[1-{\left(1+W\left[\left(\frac{1-\eta }{{\vartheta }_0}-1\right)e^{\frac{1-\eta }{{\vartheta }_0}-1-\mu (1-\eta {)}^{2}t}\right]\right)}^{-1}\right] \\ =\frac{\eta }{\mu (1-\eta )}{\int }_0^1\frac{\mathrm{d}\widehat{t}}{\widehat{t}}\left[1-{\left(1+W\left[\left(\frac{1-\eta }{{\vartheta }_0}-1\right)e^{\frac{1-\eta }{{\vartheta }_0}-1}\widehat{t}\right]\right)}^{-1}\right] \\ =\frac{\eta }{\mu (1-\eta )}W\left[\left(\frac{1-\eta }{{\vartheta }_0}-1\right)e^{\frac{1-\eta }{{\vartheta }_0}-1}\right], \end{gathered}$$ (C7)

$$=\frac{\eta }{\mu (1-\eta )}\left(\frac{1-\eta }{{\vartheta }_0}-1\right),$$ (C8)

where (C7) follows from

$$W^{\prime }[z]=W[z]∕(z(1+W[z])),$$ (C9)

and (C8) holds due to the definition of W(z). When maximizing (C8) over η for a fixed ϑ₀ (i.e. for a fixed initial resource), we find that (C8) is maximized for $\eta =1-\sqrt{{\vartheta }_0}$, which is larger than the value η = (1 − ϑ₀)/2, at which the power ∣J₁₂∣ is maximized. Hence, for sufficiently sizable resources it pays to consume them with the efficiency larger than at the maximum of ∣J₁₂∣.

b. Adaptive agent

Next, let us consider the case of adaptive agent, where

$$\eta (t)=\alpha {\eta }_{\mathrm{C}}(t)=\alpha (1-\vartheta (t)),$$ (C10)

where 0 < α ≤ 1. This involves situations, where the engine is sufficiently complex to monitor the environment and adapt to time-dependent temperatures, i.e. (C10) involves the cases, where the engine maximizes ∣J₁₂∣ for any time-dependent ϑ(t). Now (C5) is solved as

$$\vartheta (t)={\left(1+W\left[\left(\frac{1}{{\vartheta }_0}-1\right)e^{\frac{1}{{\vartheta }_0}-1-\mu (1-\alpha )t}\right]\right)}^{-1}.$$ (C11)

Using the same method as for (C8), we get from (C10, C11):

$$-{\int }_0^{\infty }\mathrm{d}t{\widehat{J}}_{21}(t)=\frac{\alpha }{\mu }\left(W\left[\left(\frac{1}{{\vartheta }_0}-1\right)e^{\frac{1}{{\vartheta }_0}-1}\right]-\text{ln}\left\{1+W\left[\left(\frac{1}{{\vartheta }_0}-1\right)e^{\frac{1}{{\vartheta }_0}-1}\right]\right\}\right),$$ (C12)

$$=\frac{\alpha }{\mu }\left(\frac{1}{{\vartheta }_0}-1+\text{ln}{\vartheta }_0\right)$$ (C13)

which makes clear that the maximum of (C13) is reached for α → 1, i.e. once the adaptation (C10) is allowed, the maximal stored energy is reached under the most effective scenario of work-extraction. Note that reaching the α = 1 limit of (C13) demands an infinite time, since ∣J₁₂∣ → 0 for α → 1.

It is worth noting, that the advanced capability of adaptation is not always provide an advantage over the simpler scenarios of adaptation when the heat engine is operating alone, i.e. without competition. Indeed, comparing (C13) and (C8) in the case of the possible maximum efficiency $\eta =1-\sqrt{{\vartheta }_0}$ of the heat engine performing by fixed internal structure for the given initial resources ϑ₀ it becomes obvious that the stored energy of the heat engine operating by local adaptation scenario is greater than that of performing by fixed internal structure scenario when

$$\alpha \ge \frac{1-2\sqrt{{\vartheta }_0}+{\vartheta }_0}{1-{\vartheta }_0+{\vartheta }_0\text{ln}{\vartheta }_0}$$ (C14)

Obviously, the right hand-side of (C14) is always ≤ 1. Thus, the stored energy of the heat engine operating by the local adaptation scenario is greater than that of operating by the fixed internal structure for α → 1. Note that in the case of rich resources ϑ ≈ 0 the stored energies of both heat engines operating on the maximum possible efficiencies will be the same.

Appendix D: Two agents (heat engines) on the same resource

a. Competition of two non-adaptive agents.

Let us now consider two agents (1 and 2) recalling that they are taken to have the same value for ${\widehat{E}}_3$ and ρ and do interact with the same thermal baths. Instead of (C4, C5) we shall now have (k = 1, 2)

$${\widehat{J}}_{21}^{[k]}=-{\eta }_k(1-\vartheta -{\eta }_k),{\eta }_k={\widehat{E}}_2^{[k]}∕{\widehat{E}}_3,$$ (D1)

$$\stackrel{.}{\vartheta }=2\mu {\vartheta }^2(1-\vartheta -\overline{\eta }),\overline{\eta }=({\eta }_1+{\eta }_2)∕2.$$ (D2)

According to (D2), ϑ will relax two times faster [than in (C5)] to the rest point $1-\overline{\eta }$. If we assume η₁ < η₂ than the second agent will need to have a finite life-time determined via η₂ = 1 − ϑ(τ₂). Otherwise for t > τ₂ this agent will not anymore function as a heat engine. Hence the competition for the same resource brings in a finite life-time of functioning as heat engine.

By analogy to (C6) we solve (D2) as

$$\vartheta (t)=(1-\overline{\eta }){\left(1+W\left[\left(\frac{1-\overline{\eta }}{{\vartheta }_0}-1\right)e^{\frac{1-\overline{\eta }}{{\vartheta }_0}-1-2\mu (1-\overline{\eta }{)}^{2}t}\right]\right)}^{-1}.$$ (D3)

Hence the total extracted work by the second agent is to be calculated from (D1) as

$$-{\int }_0^{{\tau }_2}\mathrm{d}t{\widehat{J}}_{21}^{[2]}(t),\vartheta ({\tau }_2)=1-{\eta }_2,$$ (D4)

where τ₂ is the time at which the interaction of the second agent with thermal baths is to be switched off, since for t > τ₂ it will cease to function as a heat engine (i.e. extract work). Noting that W⁻¹(z) = ze^z, we find from (D3,D4):

$${\tau }_2=\frac{1}{2\mu (1-\overline{\eta }{)}^2}\text{ln}\left[\left(\frac{1-\overline{\eta }}{{\vartheta }_0}-1\right)e^{\frac{1-\overline{\eta }}{{\vartheta }_0}-1}\frac{1-{\eta }_2}{{\eta }_2-\overline{\eta }}{e}^{\frac{\overline{\eta }-{\eta }_2}{1-{\eta }_2}}\right]$$ (D5)

Using (D3, D4, D5) we obtain:

$$-{\int }_0^{{\tau }_2}\mathrm{d}t{\widehat{J}}_{21}^{[2]}(t)={\eta }_2(1-{\eta }_2){\tau }_2-{\eta }_2{\int }_0^{{\tau }_2}\mathrm{d}t\vartheta (t)={\eta }_2(1-{\eta }_2){\tau }_2-\frac{{\eta }_2}{2\mu (1-\overline{\eta })}\text{ln}\left[\frac{\frac{1-\overline{\eta }}{{\vartheta }_0}-1}{\frac{{\eta }_2-\overline{\eta }}{1-{\eta }_2}}\right].$$ (D6)

FIG. 7:

Implementation of best response algorithm for ϑ₀ = 0.1. Each path starts from different initial values (η₁, η₂) illustrated by rectangles connected to the path by one-side. An rectangle on each path represents the steps taken by algorithm.

Indeed, we have from (D3):

$${\int }_0^{{\tau }_2}\mathrm{d}t\vartheta (t)=\frac{1}{2\mu (1-\overline{\eta })}{\int }_{e^{-2\mu (1-\overline{\eta }{)}^2{\tau }_2}}^1\frac{\mathrm{d}\widehat{t}}{\widehat{t}}{\left(1+W\left[\left(\frac{1-\overline{\eta }}{{\vartheta }_0}-1\right)e^{\frac{1-\overline{\eta }}{{\vartheta }_0}-1}\widehat{t}\right]\right)}^{-1}$$ (D7)

$$=\frac{1}{2\mu (1-\overline{\eta })}\left(\text{ln}W\left[\left(\frac{1-\overline{\eta }}{{\vartheta }_0}-1\right)e^{\frac{1-\overline{\eta }}{{\vartheta }_0}-1}\right]-\text{ln}W\left[\left(\frac{1-\overline{\eta }}{{\vartheta }_0}-1\right)e^{\frac{1-\overline{\eta }}{{\vartheta }_0}-1}{e}^{-2\mu (1-\overline{\eta }{)}^2{\tau }_2}\right]\right)$$ (D8)

$$=\frac{1}{2\mu (1-\overline{\eta })}\text{ln}\left[\frac{\frac{1-\overline{\eta }}{{\vartheta }_0}-1}{\frac{{\eta }_2-\overline{\eta }}{1-{\eta }_2}}\right]$$ (D9)

The work extracted by the first agent reads

$$-{\int }_0^{{\tau }_2}\mathrm{d}t{\widehat{J}}_{21}^{[1]}(t)-{\int }_{{\tau }_2}^{\infty }\mathrm{d}t{\tilde{\widehat{J}}}_{21}^{[1]}(t),$$ (D10)

where in the first integral ϑ(t) is given by (D3). In the second integral θ(t) holds a different equation,

$$\stackrel{.}{\vartheta }=\mu {\vartheta }^2(1-\vartheta -\eta ),$$ (D11)

since for times t > τ₂ only the first agent couples to the baths.

The first integral in (D10) is calculated analogously to (D6):

$$-{\int }_0^{{\tau }_2}\mathrm{d}t{\widehat{J}}_{21}^{[1]}(t)={\eta }_1(1-{\eta }_1){\tau }_2-\frac{{\eta }_1}{2\mu (1-\overline{\eta })}\text{ln}\left[\frac{\frac{1-\overline{\eta }}{{\vartheta }_0}-1}{\frac{{\eta }_2-\overline{\eta }}{1-{\eta }_2}}\right].$$ (D12)

The second integral in (D10) is found from (C8), where we should change η → η₁ and ϑ₀ → 1 − η₂.

Thus the total stored energy of the first agent has the following form

$$\begin{gathered} -{\int }_0^{{\tau }_2}\mathrm{d}t{\widehat{J}}_{21}^{[1]}(t)-{\int }_{{\tau }_2}^{\infty }\mathrm{d}t{\tilde{\widehat{J}}}_{21}^{[1]}(t)= \\ {\eta }_1(1-{\eta }_1){\tau }_2-\frac{{\eta }_1}{2\mu (1-\overline{\eta })}\text{ln}\left[\frac{\frac{1-\overline{\eta }}{{\vartheta }_0}-1}{\frac{{\eta }_2-\overline{\eta }}{1-{\eta }_2}}\right]+ \\ +\frac{{\eta }_1}{\mu (1-{\eta }_1)}\left(\frac{1-{\eta }_1}{1-{\eta }_2}-1\right), \end{gathered}$$ (D13)

b. Game theoretical analysis. Emergence of 2 × 2 game.

Let us define the stored energies of each agent (D13) and (D6) by W₁(η₁, η₂) and W₂(η₁, η₂). The stored energies of each agent obviously depends on both efficiency levels {η₁, η₂}. Hence, both agents face game theoretical situation where its obtained payoff depends also on the chosen strategy of its opponent. Here, strategies of each players are the chosen efficiently levels.

Thus, for each initial temperature ratio having the set of possible strategies {η₁, η₂–satisfying η₁ < η₂ ∈ [0, 1 − ϑ₀], and payoffs [W₁, W₂] defined on the strategy profiles {η₁, η₂} one may try to find the pure Nash equilibrium of the game.

Here we employ the best response algorithm for search. The steps of the algorithm are as follows

• initialize the input as arbitrarily point {η₁, η₂} such that η₁ < η₂.

• Find the maximum of W₁ and W₂ keeping the opponents efficiency level fixed. update the values {η₁, η₂} from the maximization.

• Repeat until algorithm converges.

$$\begin{gathered} \text{initial}\text{input}\{{\eta }_1,{\eta }_2\}\in [0,1-{\vartheta }_0]\&\&{\eta }_1<{\eta }_2, \\ \text{repeat} \\ {\eta }_1^t={\text{argmax}}_{\widehat{\eta }}[W_1(\widehat{\eta },{\eta }_2^t),0\le \widehat{\eta }<{\eta }_2^t] \\ {\eta }_2^t={\text{argmax}}_{\widehat{\eta }}[W_2({\eta }_1^t,\widehat{\eta }),{\eta }_1^t<{\eta }_2\le 1-{\vartheta }_0] \\ \text{until}{\eta }_1^{t+n}={\eta }_1^t. \end{gathered}$$

A particular implementation of the algorithm is illustrated in Fig.7 for various initialization {η₁, η₂} and for ϑ₀ = 0.1.

The algorithm yields a pair of efficiency levels $({\eta }_1^{\prime },{\eta }_1^{″})$ and $({\eta }_2^{\prime },{\eta }_2^{″})$ respectively for 1 and 2 agents. The values of the obtained efficiency levels for different initial temperature ratio is given in tableI.

TABLE I:

The values of obtained optimal efficiency levels ${\eta }_1^{\prime }$, ${\eta }_1^{″}$, ${\eta }_2^{\prime }$, ${\eta }_2^{″}$ and the optimal efficiency level in the absence of competition $\eta =1-\sqrt{{\vartheta }_0}$ for various initial temperature ratio ϑ₀.

ϑ 0	$1-\sqrt{{\vartheta }_0}$	${\eta }_1^{\prime }$	${\eta }_1^{″}$	${\eta }_2^{\prime }$	${\eta }_2^{″}$
0.1	0.687	0.416	0.427	0.447	0.427
0.25	0.5	0.279	0.293	0.315	0.293
0.5	0.292	0.151	0.161	0.179	0.161
0.75	0.13	0.06	0.07	0.08	0.07
0.9	0.05	0.024	0.026	0.03	0.026

As it is seen from the Table.I, the values of optimal efficiencies slightly differ.

These pairs form a cyclic best response $\{{\eta }_1^{\prime },{\eta }_2^{\prime }\}\to \{{\eta }_1^{″},{\eta }_2^{\prime }\}\to \{{\eta }_1^{″},{\eta }_2^{″}\}\to \{{\eta }_1^{\prime },{\eta }_2^{″}\}\to \{{\eta }_1^{\prime },{\eta }_2^{\prime }\}$, i.e there is no mutually preferable outcome for the agents.

Thus, the best response program yields to 2 × 2 game characterized by the presence of cyclic best responses (known as coexistence games). TableII illustrates an example of the resulted 2 × 2 game.

TABLE II:

2 × 2 game for ϑ₀ = 0.1. The first/second number in each cell represents the payoffs of the first/second agent .

	${\eta }_2^{\prime }$	${\eta }_2^{″}$
${\eta }_1^{\prime }$	(1.843, 1.726)	(1.770, 1.724)
${\eta }_1^{″}$	(1.846, 1.761)	(1.764, 1.764)

Now, let us start from the cell $\{{\eta }_1^{\prime },{\eta }_2^{\prime }\}$. We see that for the first player it is better to change its strategy since $W_1({\eta }_1^{\prime },{\eta }_2^{\prime })<W_1({\eta }_1^{″},{\eta }_2^{\prime })$. Then, the second agent will change its strategy since $W_2({\eta }_1^{″},{\eta }_2^{\prime })<W_1({\eta }_1^{″},{\eta }_2^{″})$. In this way we will recover the whole best response cycle.

It has to be noted, that in the resulted 2 × 2 game each player receive less than if they would operate by $1-\sqrt{{\vartheta }_0}$, which is the optimal efficiency level in the absence of competition.

Indeed, the payoff of each player operating by $1-\sqrt{{\vartheta }_0}$ is obtained from (C8) by interchanging μ → 2μ. For the considered case $W_1\left(1-\sqrt{{\vartheta }_0},1-{\sqrt{\vartheta }}_0\right)=W_2\left(1-\sqrt{{\vartheta }_0},1-{\sqrt{\vartheta }}_0\right)=2.33$ which is greater than any payoff in Table.II.

However, playing by optimal strategies against $1-\sqrt{{\vartheta }_0}$ is preferable for each agent $W_1\left({\eta }_1^{\prime },1-\sqrt{{\vartheta }_0}\right)=2.73$ and $W_1\left({\eta }_1^{″},1-\sqrt{{\vartheta }_0}\right)=2.77$, while $W_1\left(1-\sqrt{{\vartheta }_0},1-\sqrt{{\vartheta }_0}\right)=2.33$

The latter situation holds for any initial temperature ratio ϑ₀.

Thus, both agents face the well-known prisoners dilemma game.

c. Competition of two adaptive agents.

Let us consider now the competition of locally adaptive heat engines. We will assume that η_k = α_k(1−ϑ(t)), k = 1, 2, i.e the efficiencies are varying according to the environmental changes. Putting the efficiencies back into the (D2) we obtain

$$\stackrel{.}{\vartheta }=2\mu (1-\overline{\alpha }){\vartheta }^2(1-\vartheta ),$$ (D14)

where $\overline{\alpha }=({\alpha }_1+{\alpha }_2)∕2$. The only rest point of (D14) is ϑ = 1. Thus, in contrast to the previous case, here both engines operate till the full consumption of the available resources. The solution is found from (C11) by changing μ → 2μ and $\alpha \to \overline{\alpha }$.

The stored energy of each agent (k = 1, 2) is found by analogy of (C13) and is given by the following expression

$$-{\int }_0^{\infty }\mathrm{d}t{\widehat{J}}_{21}^{[k]}(t)=\frac{{\alpha }_k(1-{\alpha }_k)}{2\mu (1-\overline{\alpha })}\left(\frac{1}{{\vartheta }_0}-1+\text{ln}{\vartheta }_0\right)$$ (D15)

From (D15) it follows that those engines for which α = 1/2 i.e $\eta (t)=\frac{1-\vartheta (t)}{2}$ will outperform others. Indeed, according to (A13) an engine operating by the α = 1/2 level is maximizing its power ${\widehat{J}}_{21}$ at each moment in time and eventually maximizes the stored energy due to the absence of any time constraints: since both engines are operating as long as ϑ(t) ≠ 1.

Let us discuss competition between two types of heat engines– operating close to the maximum efficiency α_me → 1 and operating at maximum power ${\alpha }_{\text{mp}}=\frac{1}{2}$, for the given initial temperature ratio ϑ₀.

From (D15) follows that the stored energy of each of the agents operating by near maximum efficiency level α_me → 1 will be proportional to $\frac{{\alpha }_{\text{me}}}{2\mu }$. While, the stored energy of the agents operating at maximum power regime is proportional to $\frac{{\alpha }_{\text{mp}}}{2\mu }$, note that the prefactor is the same for both cases. Due to α_me > α_mp it is better to operate at higher efficiency level if opponent does so.

However, it is better to operate at the maximum power regime α_mp when opponent is operating at α_me, since α_mp(1 − α_mp) > α_me(1 − α_me) (the denominator in both expressio is same $2\mu \left(1-\frac{{\alpha }_{\text{mp}}+{\alpha }_{\text{me}}}{2}\right))$.

Thus, adaptive agents are facing the same dilemma as non-adaptive agents.

d. Adaptive v.s. non-adaptive agent.

Here we discuss competition of two agents– one with fixed internal structure (non-adaptive) and adaptive agents. In contrast to the above discussed competition scenario, here, only the agent with fixed internal structure has a finite lifetime. Indeed, the temperature ratio dynamics has the following form

$$\stackrel{.}{\vartheta }=\mu {\vartheta }^2(1-\vartheta -\eta )+\mu {\vartheta }^2(1-\vartheta -\alpha (1-\vartheta ))$$ (D16)

where we have used (C10) for the efficiency of the adaptive agent, and η is an efficiency of the agent with fixed internal structure. The second term of (D16) nullifies only on the point ϑ = 1 , i.e. when the resources are fully depleted. While, the first term nullifies at the time ϑ(τ) = 1 − η.

Solving (D16) by the analogy of the above mentioned procedure we found

$$\begin{gathered} \vartheta (t)=a{\left(1+W\left[\left(\frac{a}{{\vartheta }_0}-1\right)e^{\frac{a}{{\vartheta }_0}-1-a\overline{\mu }t}\right]\right)}^{-1}, \\ a\equiv 1-\frac{\eta }{2-\alpha },\overline{\mu }\equiv (2-\alpha -\eta )\mu . \end{gathered}$$ (D17)

The lifetime τ of the agent with fixed internal structure is found from (D17) and the condition ϑ(τ) = 1 − η.

$$\tau =\frac{1}{a\overline{\mu }}\text{ln}\left[\left(\frac{a}{{\vartheta }_0}-1\right)e^{\frac{a}{{\vartheta }_0}-1}\frac{2-\alpha }{(1-\alpha )\eta }{e}^{-\frac{(1-\alpha )\eta }{2-\alpha }}\right]$$ (D18)

The stored energy of non-adaptive agent during its’ lifetime will be equal to

$${\int }_0^{\tau }\eta (1-\vartheta (t)-\eta )dt=\eta (1-\eta )\tau -\eta {\int }_0^{\tau }\vartheta (t)dt=\eta (1-\eta )\tau -\frac{\eta }{\overline{\mu }}\text{ln}\left[\frac{\frac{a}{{\vartheta }_0}-1}{\frac{1-\alpha }{2-\alpha }\eta }\right]$$ (D19)

where the integral is found from (D17) and (D18)

$$\begin{gathered} {\int }_0^{\tau }\vartheta (t)dt=a{\int }_0^{\tau }\frac{dt}{1+W\left[\left(\frac{a}{{\vartheta }_0}-1\right)e^{\frac{a}{{\vartheta }_0}-1-a\overline{\mu }t}\right]}= \\ =\frac{1}{\overline{\mu }}{\int }_{\widehat{t}(\tau )}^{\widehat{t}(0)}\frac{d\widehat{t}}{\widehat{t}\left(1+W\left[\left(\frac{a}{{\vartheta }_0}-1\right)e^{\frac{a}{{\vartheta }_0}-1}\widehat{t}\right]\right)}=\frac{1}{\overline{\mu }}\text{ln}\left[\frac{\frac{a}{{\vartheta }_0}-1}{\frac{1-\alpha }{2-\alpha }\eta }\right] \end{gathered}$$ (D20)

The stored energy of the adaptive agent is composed of two terms– the one in which another agent has been in the competition and after the turning of that agent. Note that the power of the adaptive agent has the following form $-{\widehat{J}}_{21}=\alpha (1-\vartheta )(1-\vartheta -\alpha (1-\vartheta ))$. The stored energy of the adaptive agent during the competition time τ is equal

$${\int }_0^{\tau }\alpha (1-\alpha )(1-\vartheta (t){)}^{2}dt=\frac{\alpha (1-\alpha )}{a\overline{\mu }}{\int }_{\widehat{t}(\tau )}^{\widehat{t}(0)}\frac{d\widehat{t}}{\widehat{t}}\frac{{\left(1-a+W\left[\left(\frac{a}{{\vartheta }_0}-1\right)e^{\frac{a}{{\vartheta }_0}-1}\right]\right)}^2}{{\left(1+W\left[\left(\frac{a}{{\vartheta }_0}-1\right)e^{\frac{a}{{\vartheta }_0}-1}\right]\right)}^2}=$$ (D21)

$$=\frac{\alpha (1-\alpha )}{a\overline{\mu }}\left((1-2\alpha )\text{ln}\left[\left(\frac{a}{{\vartheta }_0}-1\right)\frac{2-\alpha }{(1-\alpha )\eta }\right]+a^2\text{ln}\left[\frac{(1-\frac{{\vartheta }_0}{a})\zeta }{(1-\alpha )\eta }\right]+\frac{a}{{\vartheta }_0}-\frac{\zeta }{2-\alpha }\right),$$ (D22)

where we have denoted ζ ≡ 2 − α + (1 − α)η. After switching of the agent with fixed internal structure the time evolution of temperature ratio is given by (C11), where ϑ₀ is substituted by 1 − η. The stored energy of the adaptive agent will be found from (C13) by substituting ϑ₀ → 1 − η

$${\int }_{\tau }^{\infty }\alpha (1-\alpha )(1-\vartheta (t){)}^{2}dt=\frac{\alpha }{\mu }\left(\frac{1}{1-\eta }-1+\text{ln}1-\eta \right)$$ (D23)