Stackelberg Evolutionary Games of Cancer Treatment: What Treatment Strategy to Choose if Cancer Can be Stabilized?

Abstract

We present a game-theoretic model of a polymorphic cancer cell population where the treatment-induced resistance is a quantitative evolving trait. When stabilization of the tumor burden is possible, we expand the model into a Stackelberg evolutionary game, where the physician is the leader and the cancer cells are followers. The physician chooses a treatment dose to maximize an objective function that is a proxy of the patient’s quality of life. In response, the cancer cells evolve a resistance level that maximizes their proliferation and survival. Assuming that cancer is in its ecological equilibrium, we compare the outcomes of three different treatment strategies: giving the maximum tolerable dose throughout, corresponding to the standard of care for most metastatic cancers, an ecologically enlightened therapy, where the physician anticipates the short-run, ecological response of cancer cells to their treatment, but not the evolution of resistance to treatment, and an evolutionarily enlightened therapy, where the physician anticipates both ecological and evolutionary consequences of the treatment. Of the three therapeutic strategies, the evolutionarily enlightened therapy leads to the highest values of the objective function, the lowest treatment dose, and the lowest treatment-induced resistance. Conversely, in our model, the maximum tolerable dose leads to the worst values of the objective function, the highest treatment dose, and the highest treatment-induced resistance.

Introduction

Metastatic cancer, characterized by the spread of malignant cells from its original site to other parts of the body, remains largely incurable, with cancer death rates declining by only 1.5 percent per year between years 2001 and 2017  [37, 42, 60].

This lack of progress in metastatic cancers is, in part, due to the standard of care in metastatic cancers, which typically applies a drug or drug combination at maximum tolerable dose (MTD), either continuously or in predefined treatment cycles [23, 25, 27, 28, 80]. The same regimen continues until there is unacceptable toxicity, unambiguous evidence of tumor progression, or cure. However, in metastatic cancers, cure is rare [21, 68, 70].

The goal of killing as many cancer cells as fast as possible may be evolutionarily unwise [22, 24, 28, 74, 75, 86]. This is because MTD imposes a very strong selection pressure for the evolution of treatment-induced resistance, which subsequently leads to treatment failure [1, 22, 24, 36, 42, 53, 61, 84, 85, 92].

Evolutionary cancer therapies (also known as adaptive therapies) provide an alternative to the standard of care [28, 31, 48, 56, 74, 75]. Evolutionary therapies aim to manage treatment-induced resistance in cancer cells by anticipating and steering their ecological and evolutionary dynamics. Such therapies integrate mathematical models, known cancer biology, and patient-specific data to improve care. Different therapy goals lead to different types of evolutionary therapy:

• Delaying progression or tumor burden stabilization: In situations where curative therapies are too risky or unavailable, evolutionary therapies strive to prolong the time to progression (clinical trials NCT02415621, NCT03511196, NCT05393791, NCT03543969, and NCT03630120) or, when possible, to stabilize the tumor burden by maintaining a tumor burden that is viable for the patient (clinical trial NCT05080556). In all named trials, the goal of treatment has shifted from “treat to eradicate” to a less ambitious but more attainable “treat to delay progression” or “treat to contain” [2, 17, 51, 84, 86]. The strategy of stabilization of an incurable disease has been motivated by the success of similar strategies outside of oncology, for instance, when treating human immunodeficiency virus (HIV) [13, 18, 49] and diabetes [14, 87].

• Cure: Recently, there have been attempts to cure metastatic cancers through extinction (or first strike—second strike) evolutionary therapy (clinical trials NCT04388839, and NCT04343365). This therapy aims at cure by applying a “first strike” treatment to decrease the tumor burden below a critical threshold, followed by “second strike” treatments aiming at cancer eradication [29, 32, 33].

The first and most well-known trial based on evolutionary therapy is Zhang et al.’s trial (NCT02415621), which aimed at delaying cancer progression. In this trial, patients with metastatic castrate-Resistant Prostate Cancer (mCRPC) were given abiraterone until their prostate specific antigen (PSA, a blood biomarker of tumor burden) dropped below 50% of its initial value [92, 93]. At this point, the abiraterone treatment was stopped and re-administered only when the PSA returned to its initial value. Then a new therapy cycle started. This led to cycles on and off abiraterone in response to the patients’ PSA levels. While Zhang et al.’s protocol eventually fails and progression occurs, it happens much later than with the standard of care [92, 93]. As of the time of writing, 12 of 16 patients have progressed (median time to progression of about 30 months compared to about 14 months for the standard of care), though after six years, four patients continue to cycle without any indication of disease progression [93]. Zhang et al.’s adaptive protocol is currently repeated in trial NCT05393791, with nearly 200 patients.

Despite the success of Zhang et al.’s trial, theory tells us that one could do even better for this target group of patients. Cunningham et al. [17] demonstrated with a variant of Zhang et al.’s model [92] that stabilization of the tumor burden may be possible in mCRPC. Further, they showed that even when it is unclear at what tumor burden one should stabilize, it is often possible to reach this stable tumor burden by gradually decreasing the dose whenever the tumor burden is decreasing, starting from the maximum tolerable dose [28]. Such a dose reduction therapy is currently being tested in patients with relapsed Platinum-sensitive High-Grade Serous or High-Grade Endometrioid Ovarian Cancer (clinical trial NCT05080556). In [17] and [77] a potentially even more effective approach is explored, a dose titration protocol, where the treatment starts with a very low dose, which is gradually increased whenever the tumor burden grows and decreased whenever it decays. These theoretical results and the mentioned ongoing clinical trial in ovarian cancer provide motivation for our present work, where we analyze what the best constant treatment dose is if cancer cannot be cured but can be stabilized at a viable tumor burden.

To do so, we use an evolutionary version of the leader-follower games originally introduced in economics to conceptualize interactions with an imbalance in power between firms in oligopolistic markets by the German economist von Stackelberg [38, 73]. Indeed, the physician and the cancer cells engage in a type of leader-follower game: The physician chooses a treatment dose to maximize an objective function expressing the patient’s quality of life and the cancer cells adapt to this treatment, in a way that affects the physician’s objective [75]. The difference with standard leader-follower games is that cancer cells are not modeled as rational players but as playing an evolutionary game that determines the cell types that emerge, their population size, and their evolutionary traits [5, 11, 20, 39, 81].

The combination of these two levels of interaction (between the physician and cancer cells and between the cancer cells themselves) results in a game that some of us have recently termed a Stackelberg evolutionary game [45, 66, 76]. Stackelberg evolutionary game theory applies to situations with a rational player (the physician in the case of cancer treatment) and evolutionary followers, which can be a community of populations, species, or types that evolve by natural selection (cancer cells in the case of cancer treatment). In these games, the rational player can act as the leader and anticipate and steer the eco-evolutionary dynamics of the followers, who adapt to the actions of the leader, according to the principles of natural selection.

Here, we extend the cancer model of Pressley et al. [61], who considered a polymorphic cancer cell population consisting of sensitive cells that do not evolve and resistant cells whose treatment-induced resistance is a quantitative evolving trait. Pressley et al. [61] assumed only density-dependent selection of cancer cells. Motivated by in vitro and in vivo studies that demonstrated the cancer cells’ frequency-dependent selection [26, 43, 57, 59, 79, 93], here we assume direct competition between the two cancer cell types, leading to frequency dependence. Moreover, we assume that the sensitive cancer cells have a larger competitive effect on the resistant cells than vice versa. Furthermore, as Pressley et al. [61], we assume a cost of resistance in the growth rate of resistant cells, i.e., the growth rate of resistant cancer cells decreases with the resistance level.

Pressley et al. [61] demonstrated that Zhang et al.’s adaptive therapy protocol prolonged the time to progression when compared to continuous therapy at maximum tolerable dose also in models with the treatment-induced resistance evolving as a quantitative trait. Here, we focus on a situation when cure is impossible, but the tumor burden can be stabilized at a viable level for the patient. For this case, we determine the optimal constant treatment dose maximizing the objective function of the physician, which is a function depending on the tumor burden, treatment toxicity, and the cancer cells’ level of treatment-induced resistance. This objective function captures the patient’s quality of life.

In what follows, we: (1) develop a model for the cancer’s eco-evolutionary dynamics, (2) expand it into a Stackelberg evolutionary game of cancer treatment, and (3) compare three different strategies that the physician might use to treat the patient (maximum tolerable dose, an ecologically enlightened strategy, and an evolutionarily enlightened strategy), while assuming the tumor burden has reached its equilibrium size. We conclude by highlighting the main outcomes and their relevance for the field of (mathematical) oncology, discussing model limitations, and proposing future research directions.

Methods

Following [61], we consider a model with two cancer cell types: sensitive cells, whose resistance to treatment is minimal, and (potentially) resistant cells, which may adapt to evolutionary pressures by becoming more or less resistant to treatment. However, in contrast to Pressley et al. [61] we allow for differing magnitudes of inter- and intra-type competition between the two types of cancer cells. Furthermore, we determine for which treatment doses and treatment-induced resistance levels the tumor burden can be stabilized at levels viable for the patient. Stabilization in this context refers to employing a constant treatment dose that keeps the tumor burden at a progression-free/viable equilibrium size. When the tumor burden can be stabilized, we adopt a Stackelberg (leader-follower) evolutionary game theory approach [6, 66, 75, 76, 88]: The leader (physician) chooses a treatment dose with the aim of maximizing an objective function, which is a proxy for the patient’s quality of life and depends on the tumor burden, the resistance level of cancer cells, and the treatment dose.

The cancer cells adapt to treatment in two ways, which are easiest to conceptualize as a short-run and a long-run response (though, technically, our model does not assume a separation of time scales). In the short-run (ecological time scale), only the absolute and relative abundances of the two types of cancer cells evolve. In the longer-run (evolutionary time-scale), the resistance level of the resistant cells evolves as well. This resistance level may be seen as their (evolutionary) strategy. The function mapping a given treatment dose to the resistance level that evolves in response can be perceived as the cancer cells’ evolutionary response function, which is somewhat analogous to the best-response function of the follower in a standard Stackelberg game, in that it defines the response of the followers maximizing their objective with respect to their strategy. Drawing on this analogy, we call this function the cancer cells’ best-response function.

We consider three possible cases, leading to three potentially different therapeutic outcomes. In the first case, the physician just uses the maximum tolerable dose (MTD) of treatment. In the second case, the physician can observe the current resistance level of cancer cells. After a potential phase of “tatônnement”, where the physician adjusts the treatment dose based on the actual resistance level of cancer cells and the cancer cells evolve resistance in response to this updated treatment dose, this is expected to lead to a stable situation: the resistance level of resistant cells is a best response to the current treatment dose, and this treatment dose is a best response to the resistant cells’ current resistance level.

We refer to this stable situation as the (static) Nash equilibrium of the game. The corresponding strategy of the physician is called the Nash strategy.

In the third case, the physician also anticipates the evolutionary response of the cancer cells, meaning the physician knows in advance the best response of the cancer cells in terms of their resistance to all possible treatment doses that the physician can apply. This can lead to a stable outcome corresponding to a Stackelberg equilibrium of the leader-follower game, where the followers’ strategy is at an eco-evolutionary equilibrium, with respect to their strategy and population size. The corresponding strategy of the physician is their Stackelberg strategy, in accordance with the literature on Stackelberg games and our recent work [9, 45, 66, 69, 76]. The Stackelberg strategy fully exploits the leadership role of the physician and is, by construction, the leader’s best strategy [45, 75, 76].

When the tumor may be stabilized, we compare the outcomes corresponding to these three strategies: maximum tolerable dose, the ecologically enlightened/Nash strategy, and the evolutionarily enlightened/Stackelberg strategy.

Model of Cancer Eco-Evolutionary Dynamics

We consider two distinct cancer cell populations: sensitive and resistant. We introduce frequency-dependent interactions between these two cell types by varying the intra- and inter-type competition coefficients. The treatment-induced resistance is modeled as a quantitative trait, meaning that this resistance exists on a continuum, and the resistant cells can exhibit some level of resistance $u_R(t)\ge 0$ that evolves over time t in response to therapy. While we assume that the sensitive cells do not evolve resistance ($u_S(t)=0$ for all t), we retain both $u_S$ and $u_R$ in our model, even though $u_S=0$ in the current scenario. This is done to allow for future extensions of this model where $u_S$ may reach different values. We model the eco-evolutionary dynamics of the cancer cells using a fitness-generating function, also known as a G-function [83]. The G-function defines how the fitness of a focal cancer cell using strategy v is influenced by the environment and the strategies and population sizes of the resident types. In particular, the G-function is crucial for determining the evolutionary dynamics for how the resistant strategy evolves over time. The resistance level evolves in the direction of the fitness gradient $\frac{\partial G}{\partial v},$ with respect to the focal individual’s strategy v [83]. This derivative is then evaluated at the current resident strategy $u_R,$ leading to an equation defining the evolutionary dynamics for the resident strategy itself [61, 83]. The rate at which the resistance strategy changes is scaled by an evolutionary speed term $\sigma ,$ which we assume constant for simplicity. The exact value of $\sigma$ does not influence the equilibrium outcomes presented in this paper.

The eco-evolutionary dynamics of the two populations of cancer cells are as follows:

$$\begin{array}{cc}\hfill \frac{\text{d}{x}_R}{\text{d}t}& =x_{R}G\left(v,,,\mathbf{u},,,\mathbf{x},,,m\right){|}_{v=u_R},\hfill \end{array}$$ 1

$$\begin{array}{cc}\hfill \frac{\text{d}{u}_R}{\text{d}t}& =\sigma \frac{\partial G\left(v,,,\mathbf{u},,,\mathbf{x},,,m\right)}{\partial v}{|}_{v=u_R},\hfill \end{array}$$ 2

$$\begin{array}{cc}\hfill \frac{\text{d}{x}_S}{\text{d}t}& =x_S\left(r,(u_S),\left(1,-,\frac{{\alpha }_{\mathit{SS}}{x}_S+{\alpha }_{\mathit{SR}}{x}_R}{K}\right),-,d,-,\frac{m}{k+b{u}_S}\right),\hfill \end{array}$$ 3

$$\begin{array}{cc}\hfill u_S& =0,\hfill \end{array}$$ 4

where we assume that the G-function has the following form:

$$\begin{array}{c}\hfill G(v,\mathbf{u},\mathbf{x},m)=r(v)\left(1,-,\frac{{\alpha }_{\mathit{RS}}{x}_S+{\alpha }_{\mathit{RR}}{x}_R}{K}\right)-d-\frac{m}{k+bv},\end{array}$$ 5

and that the vectors $\mathbf{u}={(u_R,u_S)}^{\top }$ and $\mathbf{x}={(x_R,x_S)}^{\top }$ capture the resistance strategies and population sizes of the two cancer cell populations, respectively. Treatment-induced resistance may come at a cost [74], which may make resistant cells less fit in the absence of treatment. In (5), we assume a cost of resistance in the intrinsic growth rate. More specifically, $r(v)=r_{max}{e}^{-gv}$, where g determines the magnitude of the effect of resistance on the growth rate. The intra-type competition effects are given by ${\alpha }_{\mathit{SS}}$ and ${\alpha }_{\mathit{RR}}$, while ${\alpha }_{\mathit{RS}}$ and ${\alpha }_{\mathit{SR}}$ are the inter-type competition coefficients. Parameter K denotes the carrying capacity of cancer cells, while parameter d is the natural death rate of cancer cells. In our model, we assume that the physician applies a treatment dose $m(t)\in [0,1]$ at time $t\ge 0$, where $m(t)=0$ corresponds to no dose and $m(t)=1$ to MTD. We assume that drug efficacy increases with m and decreases with the focal cell’s resistance strategy v, its innate resistance k, and the benefit b of the resistance level.

Parameter values, except the cost of resistance’s magnitude g and inter-type competition coefficients ${\alpha }_{\mathit{RS}}$ and ${\alpha }_{\mathit{SR}}$, are taken from [61]. The ratio of the inter-type competition coefficients is taken from [93]. Parameters and their values are summarized in Table 1.

Table 1.

Variables and parameter values of the game [61, 93]

Variables	Meaning	Feasible values
x	Cancer cell population	[0, K]
$x_S$	Sensitive population	[0, K]
$x_R$	Resistant population	[0, K]
$u_S$	Resistance strategy of sensitive cells	0
$u_R$	Resistance strategy of resistant cells	${\mathbb{R}}_{+}$
v	Resistance strategy (focal individual)	${\mathbb{R}}_{+}$
m	Treatment dose	[0, 1]

Parameters		Values
$r_{max}$	Intrinsic growth rate of the cancer cells	0.45
d	Intrinsic death rate	0.01
K	Carrying capacity	10000
g	Magnitude of cost of resistance	0.8
k	Innate cell immunity	2
b	Magnitude of resistance benefit	10
$\sigma$	Evolutionary speed of resistant cells	1
$\delta$	Progression threshold (fraction of K)	0.7
${\alpha }_{\mathit{RS}}$	Inter-type competition coefficient	0.9
${\alpha }_{\mathit{SR}}$	Inter-type competition coefficient	0.15
${\alpha }_{\mathit{SS}}$	Intra-type competition coefficient	1
${\alpha }_{\mathit{RR}}$	Intra-type competition coefficient	1

Viability Analysis of the Eco-Evolutionary Equilibria of Cancer Dynamics

The time to progression corresponds to the moment when the tumor burden exceeds a predefined fraction  $\delta \in (0,1]$ of the carrying capacity. We set this $\delta$ to 0.7 as in [61]. Our first objective is to identify treatment doses and treatment-induced resistance levels for which it is possible to maintain the tumor burden below the progression threshold $\delta K$.

The sizes of sensitive and resistant cancer cell populations at the ecological equilibria $x_R^{\ast }(m,u_R)$ and $x_S^{\ast }(m,u_R)$ at any given resistance level are found by solving $\frac{\text{d}{x}_R}{\text{d}t}=0$ and $\frac{\text{d}{x}_S}{\text{d}t}=0$, respectively. The tumor burden $x^{\ast }(m,u_R)$ is defined as $x^{\ast }(m,u_R)=x_R^{\ast }(m,u_R)+x_S^{\ast }(m,u_R).$ The extinction, stabilization, and progression regions are defined as follows:

• Extinction region: all $(m,u_R)$-pairs for which the cancer population eventually goes extinct, i.e., where the tumor burden at equilibrium $x^{\ast }(m,u_R)$ is zero (shown in green in Fig. 1):

  $$\mathcal{G}=\{(m,u_R)\in [0,1]\times {\mathbb{R}}_{+}:x^{\ast }(m,u_R)=0\}.$$
• Stabilization region: all $(m,u_R)$-pairs for which the tumor burden at equilibrium $x^{\ast }(m,u_R)$ is positive, but does not exceed $\delta K$ (shown in yellow in Fig. 1):

  $${\mathcal{Y}}_{\delta }=\{(m,u_R)\in [0,1]\times {\mathbb{R}}_{+}:0<x^{\ast }(m,u_R)\le \delta K\}.$$
• Progression region: all $(m,u_R)$-pairs for which the tumor burden progresses, i.e., for which the tumor burden at equilibrium $x^{\ast }(m,u_R)$ is higher than the progression threshold $\delta K$ (shown in red in Fig. 1):

  $${\mathcal{R}}_{\delta }=\{(m,u_R)\in [0,1]\times {\mathbb{R}}_{+}:x^{\ast }(m,u_R)>\delta K\}.$$

Fig. 1 shows different values of the ecological equilibrium $x^{\ast }(m,u_R)$ for different m and $u_R$ values, determining the three regions $\mathcal{G},$ $Y_{\delta },$ and $R_{\delta }$ for the particular parametrization of the model.

Fig. 1.

The equilibrium population size. We identify three possible regions: The green area $\mathcal{G}$ corresponds to combinations of m and $u_R$ for which $x^{\ast }(m,u_R)=0$, i.e., the region where cancer is cured; the red area ${\mathcal{R}}_{\delta }$ corresponds to combinations of m and $u_R$ for which $x^{\ast }(m,u_R)>\delta K$, i.e., the region where the tumor burden is too high, and the disease progresses; the yellow area ${\mathcal{Y}}_{\delta }$ corresponds to combinations of m and $u_R$ for which the tumor burden stabilizes at a nonzero but viable level, and the corresponding values of the objective function will matter. Parametrization: $\delta =0.7$, $r_{max}=0.45$, $g=0.8$, $K=10000$, $d=0.01$, $k=2$, $b=10$, ${\alpha }_{\mathit{SS}}={\alpha }_{\mathit{RR}}=1$, ${\alpha }_{\mathit{SR}}=0.15$, ${\alpha }_{\mathit{RS}}=0.9$ (Color figure online)

According to (2), the treatment-induced resistance $u_R$ under a particular treatment dose m eventually evolves towards an evolutionarily stable strategy (ESS) $u_R^{\ast }=u_R^{\ast }(m)$. At eco-evolutionary equilibrium, the cancer population is at its ecological equilibrium and $u_R=u_R^{\ast }.$ By definition, when the population is at the eco-evolutionary equilibrium, the ESS strategy maximizes G  [3, 83] and the first order condition applies:

$$\begin{array}{c}\hfill \frac{\partial G(v,{\mathbf{u}}^{\ast },{\mathbf{x}}^{\ast },m)}{\partial v}{|}_{v=u_R^{\ast }}=0,\end{array}$$ 6

whenever this derivative exists.

We assume that it is possible to cure or contain the disease with a constant treatment dose if values of m exist for which the pair $(m,u_R^{\ast }(m))$ lies in the extinction or stabilization region, respectively. Otherwise, progression under a constant treatment dose is inevitable.

Expanding the Model into a Stackelberg Evolutionary Game

In the previous section, we introduced an evolutionary game between different types of cancer cells. Here, we extend this game into a Stackelberg evolutionary game, where the physician as the leader maximizes the patient’s quality of life through selecting a particular treatment dose. This quality of life is captured in an objective function, defined for treatment doses and resistance levels where cure is unachievable but stabilization is possible. By maximizing such an objective function with respect to the treatment dose, we determine which of the treatment strategies leading to a viable tumor burden is the most desirable for the patient. Our objective function depends on the cancer cell population, the toxicity due to the treatment dose, and the treatment-induced resistance of the cancer cells. We assume that the objective function decreases when the tumor burden increases, as the patient might experience pain or other side effects  [41, 52]. Similarly, the objective function decreases with the treatment dose, because of the increased treatment-induced toxicity. Moreover, we assume that the objective function declines with cancer’s resistance level, as more resistant cells might give rise to secondary tumors and side effects [54, 82], which decrease the patient’s quality of life. Besides, the resistance level could be associated with the Warburg effect, which may alter the micro-environment to be more suitable for cancer cell proliferation [47]. Here, we introduce the following objective function Q:

$$\begin{array}{c}\hfill Q(m,u_R,x^{\ast }(m,u_R))=\left\{\begin{array}{cc}{Q}^{max}-c_1(\frac{x^{\ast }(m,u_R)}{K}{)}^2-c_2{u}_R^2-c_3{m}^2,\hfill & (m,u_R)\in {\mathcal{Y}}_{\delta },\hfill \\ \text{undefined,}\hfill & \text{elsewhere}.\hfill \end{array}\right)\end{array}$$ 7

In (7), $Q^{max}$ represents the maximum value of the objective function, while the weights $c_1$, $c_2$, and $c_3$ (where $c_1+c_2+c_3=1$, $c_1,c_2,c_3\in [0,1]$) determine the impact of tumor burden, treatment-induced resistance level, and treatment toxicity on the patient’s quality of life, respectively.

Figure 2 depicts the objective function (7) for a particular parametrization.

Fig. 2.

The physician’s objective function as a function of the tumor burden, the treatment dose and the resistance level (7). The white space illustrates the region where the objective function is undefined. Parametrization: $\delta =0.7$, $r_{max}=0.45$, $g=0.8$, $K=10000$, $d=0.01$, $k=2$, $b=10$, ${\alpha }_{\mathit{SS}}={\alpha }_{\mathit{RR}}=1$, ${\alpha }_{\mathit{SR}}=0.15$, ${\alpha }_{\mathit{RS}}=0.9$, $\sigma =1$, $Q^{max}=1$, $c_1=0.5,$ $c_2=0.25,$ $c_3=0.25$

In those cases where the tumor burden can be stabilized, but cure is impossible, we compare the following treatment strategies and their effects on the physician’s objective function (see also Table 2):

• Ecologically enlightened strategy (or Nash strategy): The physician considers the ecological but not the evolutionary effects of treatment. For a fixed resistance level $u_R$, the best response of the physician would be the treatment dose:

  $$\begin{array}{c}\hfill m^{\ast }(u_R)=arg\underset{m}{max}Q(m,u_R,x^{\ast }(m,u_R)).\end{array}$$ 8

  Besides, for a given treatment dose m, the cancer cells’ resistance evolves to their ESS given by (6). We refer to the stable situation this may lead to as the Nash equilibrium of the game (meaning the Nash equilibrium of the underlying simultaneous-move game). It lies at the intersection of the cancer cells’ evolutionary response (ESS strategy) curve $u_R^{\ast }(m)$ and the physician’s best response curve $m^{\ast }(u_R)$. This Nash equilibrium is only meaningful if these curves intersect within the stabilization region. When this is the case, we denote by $m_N$ the physician’s Nash strategy and by $u_R^{\ast }(m_N)$ the cancer cells’ Nash strategy.
• Evolutionarily enlightened strategy (or Stackelberg strategy): The physician anticipates the ecological and evolutionary response of the cancer cells to therapy. It is defined as

  $$\begin{array}{c}\hfill m_S=arg\underset{m}{max}Q(m,u_R^{\ast }(m),x^{\ast }(m,u_R^{\ast }(m))).\end{array}$$ 9

  The cancer cells’ strategy at the Stackelberg equilibrium is given by their ESS $u_R^{\ast }(m_S)$.

For some parametrizations of our model, the MTD, i.e., $m=1,$ can also stabilize the tumor burden. In such a case, the physician’s objective function (7) is defined for $m=1$ as well. If that is the case, we can compare the outcomes of ecologically and evolutionarily enlightened treatment strategies with the physician’s objective function under MTD.

Table 2.

Four notions of equilibria utilized in this paper

Ecological equilibrium	The populations of sensitive and resistant cancer cells are not changing (${\dot{x}}_S=0$ and ${\dot{x}}_R=0$) given the current resistance level $u_R$ and the current treatment dose m.
Eco-evolutionary equilibrium	The cancer cells are at the ecological equilibrium, and the resistance level of the resistant cells is not changing (${\dot{u}}_R=0$).
(static) Nash equilibrium	The cancer cells are at an eco-evolutionary equilibrium while the treatment dose is at the best response to the current resistance level.
Stackelberg equilibrium	Cancer cells are at an eco-evolutionary equilibrium, and the physician maximizes the objective function knowing the evolutionary response of cancer cells.

Results

We first calculate the ecological equilibria of cancer cells. We then calculate the physician’s Nash and Stackelberg strategies and the game’s corresponding outcomes in terms of the objective function of the physician (7). When the equilibrium population size lies in the patient’s stabilization region, we compare the MTD, Stackelberg and Nash outcomes of the game. In Appendix A, we illustrate the basins of attraction of these equilibria. Competition coefficients ${\alpha }_{\mathit{SS}}$ and ${\alpha }_{\mathit{RR}}$ are set to 1 as in [61], which is a common assumption in the ecology literature [12, 26, 40, 83].

Ecological Equilibria of Cancer Cells

The ecological equilibria of the cancer cells can be found by setting $\frac{\text{d}{x}_R}{\text{d}t}$ and $\frac{\text{d}{x}_S}{\text{d}t}$ to zero. With

$$\begin{array}{cc}\hfill {\widehat{x}}_S(m,u_R)& \stackrel{\text{def}}{=}\frac{K}{r_{max}(1-{\alpha }_{\mathit{SR}}{\alpha }_{\mathit{RS}})}\left(\frac{{\alpha }_{\mathit{SR}}m{\text{e}}^{g{u}_R}}{k+b{u}_R},-,\frac{m}{k},+,{\alpha }_{\mathit{SR}},,d,,{\text{e}}^{g{u}_R}\right)\hfill \\ \hfill & +\frac{K\left(r_{max},-,d,-,{\alpha }_{\mathit{SR}},,r_{max}\right)}{r_{max}(1-{\alpha }_{\mathit{SR}}{\alpha }_{\mathit{RS}})},\hfill \\ \hfill {\widehat{x}}_R(m,u_R)& \stackrel{\text{def}}{=}\frac{K}{r_{max}(1-{\alpha }_{\mathit{SR}}{\alpha }_{\mathit{RS}})}\left(-,\frac{m{\text{e}}^{g{u}_R}}{k+b{u}_R},+,\frac{{\alpha }_{\mathit{RS}}m}{k},-,d,,{\text{e}}^{g{u}_R}\right)\hfill \\ \hfill & +\frac{K\left(r_{max},-,{\alpha }_{\mathit{RS}},r_{max},+,{\alpha }_{\mathit{RS}},,d\right)}{r_{max}(1-{\alpha }_{\mathit{SR}}{\alpha }_{\mathit{RS}})},\hfill \end{array}$$

we obtain the following ecological equilibria $(x_S^{\ast }(m,u_R),x_S^{\ast }(m,u_R))$:

$$\begin{array}{cc}\hfill (x_S^{\ast }(m,u_R),x_R^{\ast }(m,u_R))& \stackrel{\text{def}}{=}\left\{\begin{array}{c}(0,0),\text{if}{\widehat{x}}_S\le 0,{\widehat{x}}_R\le 0,\hfill \\ \left(max,\{,{\overline{x}}_S,,,0,\},,,0\right),\text{if}{\widehat{x}}_S\ge 0,{\widehat{x}}_R\le 0,\hfill \\ \left(0,,,max,\{,{\overline{x}}_R,,,0,\}\right),\text{if}{\widehat{x}}_S\le 0,{\widehat{x}}_R\ge 0,\hfill \\ \left({\widehat{x}}_S,,,{\widehat{x}}_R\right),\text{if}{\widehat{x}}_S>0,{\widehat{x}}_R>0,\hfill \end{array}\right)\hfill \end{array}$$ 10

with

$$\begin{array}{cc}\hfill {\overline{x}}_S& =K\left(1,-,\frac{d+\frac{m}{k}}{r_{max}}\right),\hfill \\ \hfill {\overline{x}}_R& =K\left(1,-,\frac{d+\frac{m}{k+b{u}_R}}{r_{max}},{\text{e}}^{g{u}_R}\right).\hfill \end{array}$$

In Fig. 3, we illustrate the areas in the $(m,u_R)$-plane with different types of ecological equilibria (10).

Fig. 3.

The $x_S^{\ast }(m,u_R)$ and $x_R^{\ast }(m,u_R)$ values corresponding to different m and $u_R$ values. We identify three possible regions: The black area corresponds to $(m,u_R)\in [0,1]\times [0,1]$ where $x_S^{\ast }(m,u_R)=x_R^{\ast }(m,u_R)=0$, the dashed area corresponds to $(m,u_R)\in [0,1]\times [0,1]$ where $x_S^{\ast }(m,u_R)=0,$ $x_R^{\ast }(m,u_R)>0$, and the gray area indicates the region $(m,u_R)\in [0,1]\times [0,1]$ where both populations coexist, i.e., $x_S^{\ast }(m,u_R)>0,$ $x_R^{\ast }(m,u_R)>0$. Parametrization: $\delta =0.7$, $r_{max}=0.45$, $g=0.8$, $K=10000$, $d=0.01$, $k=2$, $b=10$, ${\alpha }_{\mathit{SS}}={\alpha }_{\mathit{RR}}=1$, ${\alpha }_{\mathit{SR}}=0.15$, ${\alpha }_{\mathit{RS}}=0.9$

Extending the Pressley et al. [61] model to include competition coefficients creates a non-monotonic relation between the total population size at the ecological equilibrium and treatment dose. In Pressley et al. [61] model, the total equilibrium population size decreases as the treatment dose increases, which means that lower constant treatment doses will also fail if MTD fails. Therefore, we consider the extended model more realistic for many cancers and treatments. The cell population is considered extinct in areas where $x_S^{\ast }=0$ and $x_R^{\ast }=0$.

The Best Response Curves of Cancer and the Physician

The best response curve of the resistant cancer cell population (ESS curve) is determined using (6). With ${\displaystyle {\widehat{u}}_R(m)\stackrel{\text{def}}{=}-\frac{k}{b}-\frac{m}{2bd}+\sqrt{\frac{m^2}{4b^2{d}^2}+\frac{m}{\mathit{bdg}}},}$

$$\begin{array}{c}\hfill u_R^{\ast }(m)=\left\{\begin{array}{c}0,\text{if}{\widehat{u}}_R(m)<0,\hfill \\ {\widehat{u}}_R(m),\text{otherwise.}\hfill \end{array}\right)\end{array}$$ 11

Note that ${\widehat{u}}_R(m)$ increases with m. The best response curve $m^{\ast }(u_R)$ of the leader is determined by maximizing their objective function $Q(m,u_R^{\ast }(m),x^{\ast }(m,u_R^{\ast }(m)))$ with respect to m. Note that the objective function is differentiable and concave in m, and, therefore, $m^{\ast }(u_R)$ is calculated by setting the first derivative of the objective function to zero and projection on [0, 1] whenever the result falls outside of this interval. Letting $A=\frac{(1-{\alpha }_{\mathit{SR}}){\text{e}}^{g{u}_R}}{k+b{u}_R}+\frac{1-{\alpha }_{\mathit{RS}}}{k}$ and

$$\widehat{m}(u_R)\stackrel{\text{def}}{=}\frac{c_1\mathrm{A}\left({\mathrm{r}}_{max},(2-{\alpha }_{\mathrm{R}\mathrm{S}}-{\alpha }_{\mathrm{S}\mathrm{R}}),-,(1-{\alpha }_{\mathrm{R}\mathrm{S}}),\mathrm{d},-,(1-{\alpha }_{\mathrm{S}\mathrm{R}}),\mathrm{d},{\text{e}}^{{\mathrm{gu}}_{\mathrm{R}}}\right)}{{\displaystyle c_1{\mathrm{A}}^2+{\mathrm{c}}_3{\mathrm{r}}_{max}^2{(1-{\alpha }_{\mathrm{S}\mathrm{R}}{\alpha }_{\mathrm{R}\mathrm{S}})}^2}},$$

the leader’s best response then reads as

$$\begin{array}{c}\hfill m^{\ast }(u_R)=\left\{\begin{array}{c}0,\text{if}\widehat{m}(u_R)<0,\hfill \\ \widehat{m}(u_R),\text{if}\widehat{m}(u_R)\in [0,1],\hfill \\ 1,\text{otherwise.}\hfill \end{array}\right)\end{array}$$ 12

The parameter $c_2$ (determining the impact of treatment-induced resistance on the objective function) has no effect on the leader’s best response curve and, therefore, does not influence the Nash solution. However, the parameter $c_2$ affects the Stackelberg solution.

Identifying the Nash and Stackelberg Equilibria

The Nash equilibrium lies at the intersection of the best response curves of cancer cells and the physician, defined by (11) and (12), respectively. The Stackelberg equilibrium is calculated numerically through  (9). It corresponds to the point on the cancer’s best response curve that maximizes the physician’s objective function.

Figure 4A shows a particular parametrization of the model for which the MTD and the Nash and Stackelberg equilibria all lie in the stabilization region. The physician’s best response, calculated through (12), is shown as a dashed line. Figure 4B shows that for this parametrization, the evolutionarily enlightened (Stackelberg) strategy leads to the highest value of the objective function, followed by the ecologically enlightened (Nash) strategy, while MTD leads to the lowest value of the objective function. Moreover, the evolutionarily enlightened (Stackelberg) strategy corresponds to both a lower treatment dose/toxicity and lower treatment-induced resistance than the ecologically enlightened (Nash) strategy and MTD.

Fig. 4.

The outcomes of the MTD, ecologically enlightened (Nash), and evolutionarily enlightened (Stackelberg) strategies of the physician in the SEG against cancer: The yellow and red regions represent combinations of $u_R$ and m leading to tumor stabilization and progression, respectively. A Illustration of the outcomes of the Stackelberg, Nash, and MTD treatment strategies and corresponding values of the objective function. B The level curves of the objective function and outcomes of the Nash, Stackelberg, and MTD treatment strategies. Parametrization: $\delta =0.7$, $r_{max}=0.45$, $g=0.8$, $K=10000$, $d=0.01$, $k=2$, $b=10$, ${\alpha }_{\mathit{SS}}={\alpha }_{\mathit{RR}}=1$, ${\alpha }_{\mathit{SR}}=0.15$, ${\alpha }_{\mathit{RS}}=0.9$, $Q^{max}=1$; $c_1=0.5,$ $c_2=0.25,$ $c_3=0.25$ (Color figure online)

The Nash and Stackelberg equilibria of a Stackelberg evolutionary game can coincide under certain conditions, for instance, if the leader’s strategy does not affect the evolution of the resistance level of the cancer cells, as proven by  [76].1 Fig. 5 illustrates the case where the Nash and Stackelberg solutions coincide, due to the fact that the objective function does not include treatment-induced resistance [76]. However, even in this situation, the MTD results in a lower value of the objective function than the Nash and Stackelberg strategies. In appendix A, the local stability of the eco-evolutionary equilibria is determined through numerical analysis of the Jacobian matrix. Furthermore, in this appendix, we illustrate the domain of attraction of the Nash equilibrium.

Fig. 5.

The outcomes of the MTD, ecologically enlightened (Nash) and evolutionarily enlightened (Stackelberg) strategies of the physician in the SEG against cancer: The yellow and red regions represent tumor burden stabilization at a safe level and progression, respectively. A The Nash and Stackelberg strategies coincide. B The values of the physician’s objective function coincide with the Nash and Stackelberg strategies and are better than that of MTD. Parametrization: $\delta =0.7$, $r_{max}=0.45$, $g=0.8$, $K=10000$, $d=0.01$, $k=2$, $b=10$, ${\alpha }_{\mathit{SS}}={\alpha }_{\mathit{RR}}=1$, ${\alpha }_{\mathit{SR}}=0.15$, ${\alpha }_{\mathit{RS}}=0.9$, $Q^{max}=1$; $c_1=0.67,$ $c_2=0,$ $c_3=0.33$ (Color figure online)

Discussion

Cancer treatment is a Stackelberg (or leader-follower) evolutionary game. Recent works, including  [45, 75, 76, 88], suggested that physicians should exploit the advantages of their leadership role in this game. This is because the physician, unlike the cancer cells, can anticipate and steer the cancer’s eco-evolutionary response to treatment, while the cancer cells can only adapt to the current and past physician’s actions. Staňková et al. [75] proposed that in order to utilize their leadership role fully, the physician needs to (i) set the treatment goal, as different treatment goals will correspond to different treatment strategies, (ii) introduce a resistance management plan, and (iii) perform after-action reports, adjusting assumptions and parameters based on how different patients respond to the treatment (see also [93]).

In this work, we focused on a specific treatment goal, finding the constant treatment dose maximizing the physician’s objective function once the tumor burden can be stabilized at a level viable for the patient. We did so by utilizing the Stackelberg evolutionary game framework where cancer is modeled as an evolutionary game, extending the polymorphic cancer model by Pressley et al. [61]. In their paper, Pressley et al. compared the time to progression of Zhang et al. [92]’s protocol to that of the maximum tolerable dose. They demonstrated that, while the adaptive protocol always extended the time to progression, in some cases, this improvement was rather small.

Rather than analyzing the transient phase of controlling the tumor burden and prolonging the time to progression [16, 17, 35, 61], here we have focused on finding a constant treatment dose maximizing the physician’s objective function when the tumor burden can be stabilized [8]. This physician’s objective function depends on the tumor burden, the treatment toxicity, and the treatment-induced resistance in cancer cells. The weights in this objective function can be adjusted to capture the importance of the tumor burden, resistance level, and treatment toxicity for each patient, in line with patient-centered care [65]. Subsequently, we analyzed the impact of different treatment strategies in terms of this objective function: MTD, an ecologically enlightened therapy, and an evolutionarily enlightened therapy.

We have shown that the evolutionarily enlightened therapy leads to at least as high values of the objective function as the ecologically enlightened therapy, while the MTD leads to the lowest objective function values. For most parametrizations, the ecologically enlightened therapy leads to a higher treatment dose than the evolutionarily enlightened therapy, while both are less toxic than MTD.

For some parametrizations, the evolutionarily enlightened treatment corresponding to the Stackelberg strategy of the Stackelberg evolutionary game of cancer treatment leads to an outcome that is at the boundary of the progression region (Fig. 4). This means that a small deviation in estimating the cancer cells’ response would lead to growth of the tumor burden. However, upon observation of this cancer growth, the treatment dose could be increased a little, and the cancer would be stabilized again; thus, aiming at the Stackelberg strategy is still the best option.

In general, reaching any equilibrium requires frequent measurements of the tumor volume and its composition and depends on many factors, such as the speed of the cancer’s response to treatment. However, the physician may still be able to find the Stackelberg equilibrium by dose de-escalation, meaning starting from the MTD and applying small adjustments to the dose until the desired equilibrium is reached, as suggested by, among others, Enriquez-Navas et al. [23] and Cunningham et al. [17]. Another effective strategy may be starting from a minimal effective dose and gradually increasing it until an equilibrium is reached, which seems even more effective in a model similar to that of Zhang et al. [92]. This strategy has not yet been tested in clinical trials but may have potential [17, 78].

The model that we studied included treatment-induced resistance as a quantitative trait. Another option is to model resistance as a qualitative trait [4, 7, 10, 11, 35, 43, 55, 67, 71, 91]. Our model can be also extended to include several quantitative traits, evolving in response to multiple drugs and therapeutics [45, 64, 76]. Of interest in such models are cross-sensitivities [89, 90], co-resistance [58], and more possible effective treatment strategies available in a multi-drug setting, such as evolutionary double bind therapy [15, 30, 50].

In our model, the cost of resistance is associated with the growth rate of the resistant cancer cell type. Alternatively, one could consider models where either the carrying capacity [61] or the competition coefficients [83] explicitly depend on the resistance level.

In accordance with some parametrizations of the model presented here, there are cancers for which MTD can effectively stabilize tumors [6, 34, 44]. Also, there are metastatic cancers where we may be able to aim for a more ambitious treatment goal than tumor stabilization, namely for cure [22, 32]. For example, in multiple myeloma, there is a discussion on when to aim for a cure instead of containment. Strategies for these two goals differ substantially [19, 62, 63] and game-theoretic models fitted with patient data may help us to find a suitable treatment goal, corresponding evolutionary therapy, and predict the patient/tumor response to such therapy.

Our future research, therefore, will focus on the analysis of the properties of a wider class of game-theoretical models based on classical dynamics [34, 46], extended into a game-theoretic setting, and validating these models through in-vitro and in-vivo data, similarly to how it was done for other cancer models [44, 71, 72]. The ultimate goal is to analyze multiple dynamics that fit the data well, evaluate alternative hypotheses and models, and propose suitable treatment goals and evolutionary therapies to improve patient quality of life and survival.

Appendix A: Basins of Attraction of Eco-Evolutionary Equilibria

The determination of the local stability of the eco-evolutionary equilibria involves the analysis of the Jacobian matrix at these equilibria. We confirm the local stability of the equilibria for treatment dosage m within 0.6 to 0.8 numerically, through analysis of the eigenvalues of the corresponding Jacobian matrix. We employ numerical modeling to demonstrate the domain of attraction for the Stackelberg and Nash equilibria (Fig. 5). Similarly, we can investigate the basins of attraction for other derived equilibria, such as those in Fig. 4. The sensitive and resistant populations ($x_S$ and $x_R$) are normalized (assuming $K=1$), to make the visualizations more informative. We consider initial values $x_S(0)$, $x_R(0)$, $u_R(0)$ as points in the grid $\mathit{\ell }\times \mathit{\ell }\times \mathit{\ell }$ where $\mathit{\ell }=\{0.1,0.2,0.3,\dots ,1\},$ and $x_S(0)+x_R(0)<1$. The model converges to the eco-evolutionary equilibrium for all initial conditions within this grid. As an example, we illustrate this convergence to the equilibria from 3 different initial values. In Fig. 6, we illustrate how cancer eco-evolutionary dynamics starting in the stabilization region remain in the same region when reaching the Nash equilibrium. As we have not proven this for all possible parametrizations, there may be parametrizations with which the evolutionary dynamics leave the stabilization region before reaching equilibrium.

Fig. 6.

Eco-evolutionary dynamics of cancer while applying the Nash strategy dose ($m_N=0.723$), starting from 3 different initial conditions, with $(x_S(0),x_R(0),u_R(0))$ equal to (0.5, 0.1, 0.2),  (0.3, 0.4, 0.5),  and (0.2, 0.2, 0.7),  respectively. The eco-evolutionary dynamics starting from these three initial conditions are denoted in red, blue, and purple, respectively. We see how the eco-evolutionary dynamics starting at these initial conditions evolve in time and converge to the Nash equilibrium. A The eco-evolutionary dynamics. B The ecological dynamics. Parametrization: $\delta =0.7$, $r_{max}=0.45$, $g=0.8$, $K=10000$, $d=0.01$, $k=2$, $b=10$, ${\alpha }_{\mathit{SS}}={\alpha }_{\mathit{RR}}=1$, ${\alpha }_{\mathit{SR}}=0.15$, ${\alpha }_{\mathit{RS}}=0.9$, $\sigma =1$ (Color figure online)

Author Contributions

Monica Salvioli and Hasti Garjani: conceptualization, formal analysis, methodology, software, visualization, writing—original draft preparation, writing—reviewing and editing. Mohammadreza Satouri: Formal analysis, methodology, writing—reviewing and editing. Mark Broom: methodology, writing- reviewing and editing. Yannick Viossat: formal analysis, methodology, writing—reviewing and editing. Joel S. Brown: Conceptualization, methodology, writing—original draft preparation, writing—reviewing and editing. Johan Dubbeldam: Funding acquisition, methodology, supervision, writing—original draft preparation, writing—reviewing and editing. Kateřina Staňková: Conceptualization, formal analysis, funding acquisition, methodology, supervision, writing- original draft preparation, writing—reviewing and editing.

Funding

This research was supported by European Union’s Horizon 2020 research and innovation program under the Marie Skłodowska-Curie grant agreement No 955708 and the Dutch Research Council projects OCENW.KLEIN.277 and VI.Vidi.213.139.

Availability of data and material

Not applicable.

Code availability

https://github.com/HastiGarjani/Stackelberg-Evolutionary-Game-in-Cancer.git

Declarations

Conflict of interest

The authors declare no conflict of interest.

Ethical approval

Not applicable.

Consent to participate

Not applicable.

Consent for publication

Not applicable.