Optimal control of the SIR model with constrained policy, with an application to COVID-19

Abstract

This article considers the optimal control of the SIR model with both transmission and treatment uncertainty. It follows the model presented in Gatto and Schellhorn (2021). We make four significant improvements on the latter paper. First, we prove the existence of a solution to the model. Second, our interpretation of the control is more realistic: while in Gatto and Schellhorn (2021) the control $\alpha$ is the proportion of the population that takes a basic dose of treatment, so that $\alpha >1$ occurs only if some patients take more than a basic dose, in our paper, $\alpha$ is constrained between zero and one, and represents thus the proportion of the population undergoing treatment. Third, we provide a complete solution for the moderate infection regime (with constant treatment). Finally, we give a thorough interpretation of the control in the moderate infection regime, while Gatto and Schellhorn (2021) focused on the interpretation of the low infection regime. Finally, we compare the efficiency of our control to curb the COVID-19 epidemic to other types of control.

1. Introduction

This article extends the analysis of the model presented in [1]. In that article the authors gave analytical expressions for the optimal proportion of infected to undergo treatment in a pandemic. Analytical approaches allow to better understand the form of the solution compared to numerical approaches, which are currently more prevalent in this domain. It was possible for the authors to find formulae because of the type of objective function they chose. Rather than minimize an expected value of the number of infected or the cumulated number of infected (under a constraint on the dispersion of the results), we chose to minimize the expected value of a convex increasing function of the number of infected at the horizon. The quadratic utility function is a special case of the latter, and we indeed show results for a whole family of functions, called isoelastic functions. Each function in this family is characterized by a single parameter, called the risk-aversion parameter, and varying this parameter as we shall see enables us to better understand how the optimal control depends smoothly on the level of aversion to risk, where risk is understood as the probabilistic uncertainty of the result. In contradistinction, the optimal control of deterministic epidemiological models is often of the bang–bang type (see for instance [2], [3]). Several authors, such as [3], [4], [5], [6] use Pontryagin’s maximum principle. Some authors (e.g., [7]) use dynamic programming. Laarabi et al. [8] use the same framework, but add delay. The literature on the control of stochastic epidemiologic models tends to be more sparse and more recent. A variety of models and numerical methods has been proposed: Markov chain [9], backward SDE solved by the 4-step scheme [10], simulated annealing [11], stochastic programming [12], genetic algorithms [13]. Wang et al. [14] consider time-varying parameters.

The contributions of this article are fourfold. First, we prove existence of a solution. Second, whereas in [1] the optimal control $\alpha$ has the interpretation of the proportion of the population that takes a basic dose of treatment, so that $\alpha >1$ occurs only if a proportion of the population takes more than a basic dose of treatment. In the low infection regime part of our paper, $\alpha$ is constrained to be between zero and one, and represents thus the proportion of the population undergoing treatment. The latter interpretation is much more realistic, as it is uncommon to ration treatment. Third, we provide a complete solution for the moderate infection regime (with constant treatment). The final improvement is a thorough numerical analysis and sensitivity analysis of the moderate infection regime, while [1] focused exclusively on the interpretation of the control in the low infection regime. This enables us to discover some errors in the second-order term of the solution in [1], which we correct here. Finally, we compare the efficiency of our control to curb the COVID-19 pandemic to other types of control. Our optimal control is, as expected, superior to a full control (or no control), in terms of expected utility. It is clearly superior in the case of low infection, but the benefit is less pronounced in the case of moderate infection. There are two possible reasons for that. First, we included only few terms in the analytic series of the optimal control. Adding more terms would have yielded slightly better results. More importantly, the quality of the treatment available to COVID-19 patients in the US in 2020 was probably not sufficient to make a difference if the number of infected had climbed to more than 1%.

The structure of the article is as follows. In Section 2 we briefly introduce the model in [1], and provide a proof of existence of the solution. In Section 3, we show our results for the low infection regime. In Section 4, we extend and analyze the solution in the moderate infection regime. Section 5 shows our experimental results when applying our methodology to the COVID-19 in the US in 2020. We draw the conclusion in Section 6.

2. A stochastic SIR model with treatment uncertainty

Let $S$, $I$, $R$ be the proportions of susceptible, infected, and out of infection (recovered, and dead), respectively. Let $\beta$ be the transmission rate and $\mu$ be the death rate.

In the SIR model, the rate of decrease $\frac{dS}{dt}$ of the proportion of susceptible is equal to the constant transmission rate $\beta$ time $SI$. As in [1], we add a term ${\sigma }_S\sqrt{SI}\frac{d{B}_1}{dt}$, where $\frac{d{B}_1}{dt}$ is white noise, in order to model the error in the transmission rate:

$$\frac{dS}{dt}=-\beta SI+{\sigma }_S\sqrt{SI}\frac{d{B}_1}{dt}$$

Infected patients are either treated or not treated against the disease. In both cases they either recover or die. We denote by ${\mu }_0$ (${\mu }_1$) the constant death rate without (with) treatment and by $K\left(t\right)$ the recovery rate of the treatment. The optimal policy $\alpha$ is a progressively measurable process that represents the proportion of the infected population that receives treatment, thus $\alpha \left(t\right)\in [0,1]$. This constraint is an important addition to the model in [1]. Depending whether the individual is treated or not, there are then four different ways for an infected individual to exit the pool of infected:

• •not treated and recover
• •not treated and die
• •treated and recover
• •treated and died.

Thus, the “out of infection rate” will be:

$$\frac{dR\left(t\right)}{dt}=\underset{\text{not treated and recover}}{\underbrace{(1-\alpha \left(t\right))I\left(t\right)K_0}}+\underset{\text{not treated and die}}{\underbrace{(1-\alpha \left(t\right))I\left(t\right){\mu }_0}}+\underset{\text{treated and recover}}{\underbrace{\alpha \left(t\right)I\left(t\right)K_1\left(t\right)}}+\underset{\text{treated and die}}{\underbrace{\alpha \left(t\right)I\left(t\right){\mu }_1}}-\underset{\text{treatment measurement error}}{\underbrace{\alpha \left(t\right)I\left(t\right)\sigma \frac{d{B}_2}{dt}}}$$ (1)

For simplicity, we assume that the Brownian motion driving transmission uncertainty ($B_1$) is independent from the Brownian motion driving treatment uncertainty ($B_2$). Usually ${\mu }_0\ge {\mu }_1$ (people die faster without treatment than with treatment), but not necessarily. Most of the time $K_1\left(t\right)>K_0$ (treatment is better than no treatment), but not necessarily. We relax this requirement somewhat by requiring:

$$P(K_0<K_1\left(t\right))\text{ is close to one}$$ (2)

In order to keep the treatment rate within bounds, we model it as an Ornstein–Uhlenbeck process:

$$d{K}_1\left(t\right)={\lambda }_k({\overline{k}}_1-K_1\left(t\right))dt+{\sigma }_{k}d{B}_2\left(t\right)$$

with the mean-reversion rate ${\lambda }_k>0$ and the long run value of the treatment rate ${\overline{k}}_1$. It is well-known that $K_1$ is Gaussian, with variance equal to:

$$\mathbb{V}ar\left[K_1\left(t\right)\right]=\frac{{\sigma }_k^2}{2{\lambda }_k}(1-e^{-2{\lambda }_{k}t})$$

Thus, if mean-reversion is large compared to volatility ${\sigma }_k$, constraint (2) is satisfied. We simplify (1) by:

$$\frac{\frac{dR\left(t\right)}{dt}}{I\left(t\right)}=K_0+{\mu }_0+\alpha \left(t\right)(-K_0+K_1\left(t\right)-{\mu }_0+{\mu }_1)-\alpha \left(t\right)\sigma \frac{d{B}_2}{dt}$$

Putting everything together, the dynamics of the infected is:

$$\frac{dI\left(t\right)}{dt}=\beta S\left(t\right)I\left(t\right)-\frac{dR\left(t\right)}{dt}-{\sigma }_S\sqrt{S\left(t\right)I\left(t\right)}\frac{d{B}_1}{dt}$$

We try to minimize a measure of the infected over our horizon $T$. This article focuses on the solution of the Mayer problem, i.e., the objective is expressed as a measure of $I\left(T\right)$. Another possible control would have been the time-integral of the number of infected over the horizon (the Lagrange problem). Both figures have their merit,1 and we leave for future research the control of (a measure) of the Lagrange and Bolza problems. Rather than trying to minimize the expected value of the infected, namely $E\left[I\left(T\right)\right]$, we include in our objective the risk caused by the uncertainty of the model and its observations. Decision-makers are notoriously risk-averse. For this reason, Morgenstern and Von Neumann [15] introduced a class of utility functions $U$ that bears their names. A decision-maker in epidemiology that is averse to risk will thus minimize the expected utility of the infected, namely $E\left[U\left(I\left(T\right)\right)\right]$ where $U$ is increasing and convex. Alternately, one can maximize the negative thereof, i.e., maximize the expected value of a concave and decreasing function of $I\left(T\right)$. The policy obtained in maximizing the expected value of a concave utility function can be showed, under certain conditions, to maximize the expected value of the outcome (here $-I$) under a constraint on the dispersion of the outcome. Out of the universe of concave decreasing utility functions, we choose the power utility function:

$$U\left(I\right)=-\frac{I^{1-\gamma }}{1-\gamma }$$

The coefficient $\gamma$ is often called the risk-aversion parameter . When $\gamma =0$ the decision-maker is risk-neutral, meaning that the uncertainty does not have an influence on her decisions. It is straightforward to check that this power utility function is concave in $I$ when $\gamma <0$, which we will assume. The more negative $\gamma$ the more risk-averse is the decision-maker. Taking for instance $\gamma =-1$, we see that the objective is to

$$max\mathbb{E}\left[-\frac{I^2}{2}\right]$$

which returns the same policy as:

$$min\mathbb{E}\left[\frac{I^2}{2}\right]$$

The importance of analytic formulations is that other figures of interest in this model, like the expected number of deaths from treatment can be analytically calculated, and depend on $\gamma$. Thus, a decision-maker can calibrate its risk-aversion parameter $\gamma$ on other goals. Expected number of deaths is only one type of goal and economic factors that can be easily added. We define

$$\tau =min\{t>0|I\left(t\right)=0\text{ or }I\left(t\right)=1\}$$

Our controlled SIR model is thus:

$$\underset{0\le \alpha \left(t\right)\le 1}{sup}\mathbb{E}\left[-\frac{I{(min(\tau ,T))}^{1-\gamma }}{1-\gamma }\right]$$

$$dS\left(t\right)=-\beta S\left(t\right)I\left(t\right)dt+{\sigma }_S\sqrt{S\left(t\right)I\left(t\right)}d{B}_1\left(t\right)$$ (3)

$$dI\left(t\right)=\left(\beta S\left(t\right)-(K_0+{\mu }_0)+\alpha \left(t\right)(K_0-K_1\left(t\right)+{\mu }_0-{\mu }_1)\right)I\left(t\right)dt+\alpha \left(t\right)I\left(t\right)\sigma d{B}_2\left(t\right)-{\sigma }_S\sqrt{S\left(t\right)I\left(t\right)}d{B}_1\left(t\right)$$ (4)

$$dK\left(t\right)={\lambda }_k(\overline{K}-K\left(t\right))dt+{\sigma }_{k}d{B}_2\left(t\right)$$ (5)

The relative sign of our volatilities $\sigma$ and ${\sigma }_k$ is important. We will assume without loss of generality that $\sigma <0$. The sign of ${\sigma }_k$ is the sign of covariance between the measured value of today’s treatment rate and the change in value of the treatment rate between today and a future date. An example may help illustrate the difference. Suppose that over a week one performs daily measurements of the treatment recovery rate as well as daily forecasts of the evolution of the treatment recovery rate over the next day. The two quantities measured each day $t$ are proportional to the same white noise $B_2(t+$1 day$)-B_2\left(t\right)$. One then calculates weekly estimates $\hat{\sigma }$ of $\sigma$ and ${\hat{\sigma }}_k$ of ${\sigma }_k$ over these 7 daily observations. Since we arbitrarily choose $\sigma <0$, a negative ${\hat{\sigma }}_k$ shows a correlation of ＋1 between the measurement (of today’s treatment rate) and the forecast. Fig. 1 is a depiction of our model.

Fig. 1.

A stochastic SIR model.

Let $\tau$ be the time when the infection stops, i.e., the first time when either $I\left(\tau \right)$ or $S\left(\tau \right)$ or both are equal to zero. We naturally require the control to go down to zero at that time, for definiteness reasons. The next result considers an open-loop control.

Theorem 1

For a given progressively measurable control $0\le \alpha \le 1$ , with and given initial values $(S\left(0\right),I\left(0\right),R\left(0\right),K\left(0\right))$ there exists a unique solution of (3) (4) (5) up to time $\tau$ .

The proof of Theorem 1, included in Appendix A, follows the proof of a theorem of Yamada and Watanabe (1971), as exposed in the book by Karatzas and Shreve [16, Prop. 2.13, Sec. 5.2]. We showed in a companion document that the probability that either $I$ or $S$ is equal to zero over a finite interval is zero, following the method of proof in [17], [18]. It is thus unlikely that a simple discretization of our model results in negative values.2 For notational simplicity we define the impact of treatment risk $X$:

$$X\left(t\right)=\frac{K_0+{\mu }_0-{\mu }_1-K_1\left(t\right)}{\sigma }$$

as well as the long run impact of the treatment risk $\overline{X}$:

$$\overline{X}=\frac{K_0+{\mu }_0-{\mu }_1-{\overline{k}}_1}{\sigma }$$

We define ${\lambda }_x={\lambda }_k$ and ${\sigma }_x={\sigma }_k/\sigma$. For simplicity we write $\mu =K_0+{\mu }_0$.

In the absence of bounds on $\alpha$, Gatto and Schellhorn [1] show that, when a smooth optimal closed-loop control $\alpha$ exists, there exists a probability measure $\tilde{\mathbb{P}}$ 3 under which $I\left(t\right)e^{\mu t}$ is a martingale and $I$ is the optimal state. Moreover, $I$ has the explicit form:

$$I\left(t\right)=F(Z\left(t\right),X\left(t\right),S\left(t\right),t)$$ (6)

$${\left(Z\left(T\right)\right)}^{1/\gamma }=I\left(T\right)$$ (7)

and $F$ is twice continuously differentiable. In the sequel we will shorten this statement by saying that “$I$ solves the martingale problem (6)”.

3. Results in the low infection regime

We assume $S\left(t\right)$ close to one and $\sigma =0$. Thus the term:

$$r=\beta S\left(t\right)-(K_0+{\mu }_0)\simeq \beta -K_0-{\mu }_0$$

is assumed constant. With this simplification, we give an analytical solution to the constrained problem, i.e., the case where $0\le \alpha \left(t\right)\le 1$, a significant improvement over [1], who considered the unconstrained case.

We consider first the case where the treatment rate is constant, and then the case where it follows an Ornstein–Uhlenbeck process.

3.1. Constant treatment rate

Let $b=\beta -{\mu }_1-{\overline{k}}_1$. The problem is:

$$\underset{0\le \alpha \left(t\right)\le 1}{sup}\mathbb{E}\left[-\frac{I{\left(T\right)}^{1-\gamma }}{1-\gamma }\right]$$$$dI\left(t\right)=(r+\alpha \left(t\right)(b-r))I\left(t\right)dt+\alpha \left(t\right)\sigma I\left(t\right)d{B}_2\left(t\right)$$ (8)

Theorem 2

The following constant control is optimal:

$$\alpha =min\left(1,max\left(0,\frac{{\overline{k}}_1-K_0}{{\sigma }^2\left|\gamma \right|}\right)\right)$$

The proof is in Appendix B, and follows closely [19].

3.2. Treatment rate as Ornstein–Uhlenbeck process

The problem is

$$sup\mathbb{E}\left[-\frac{I{\left(T\right)}^{1-\gamma }}{1-\gamma }\right]$$$$dI\left(t\right)=(r+\alpha \left(t\right)\sigma X\left(t\right))I\left(t\right)dt+\alpha \left(t\right)\sigma I\left(t\right)d{B}_2\left(t\right)$$$$dX\left(t\right)={\lambda }_x(\overline{X}-X\left(t\right))dt-{\sigma }_{x}d{B}_2\left(t\right)$$ (9)

In the low infection regime our solution will depend on a kernel $H_0(X_t,\tau )$ with $\tau =T-t$, while in the moderate infection regime it will also depend on two other kernels $H_1(X_t,\tau )$ and $H_2(X_t,\tau )$ that are closely related. In order to unify notation we define the kernels. Define

$$H_0(X_t,\tau )=exp\left(\frac{1}{\gamma }\left(\frac{A_1(\tau ,\gamma )X_t^2}{2}+A_2(\tau ,\gamma )X_t+A_3(\tau ,\gamma )+(1-\gamma )(\mu +r)\tau \right)\right)$$ (10)

and, for $i>0$

$$H_i(X_t,\tau )=exp\left(\frac{i}{\gamma }\left(\frac{A_1(\tau ,\gamma /i)X_t^2}{2}+A_2(\tau ,\gamma /i)X_t+A_3(\tau ,\gamma /i)\right)\right)$$ (11)

where

$$A_1(\tau ,\gamma )=\frac{1-\gamma }{\gamma }\frac{2(1-exp(-\theta \left(\gamma \right)\tau ))}{2\theta \left(\gamma \right)-(b_2\left(\gamma \right)+\theta \left(\gamma \right))(1-exp(-\theta \left(\gamma \right)\tau ))}$$ (12)

$$A_2(\tau ,\gamma )=\frac{4{\lambda }_x\overline{X}{b}_1\left(\gamma \right){\left(1-exp\left(-\theta \left(\gamma \right)\tau /2\right)\right)}^2}{\theta \left(\gamma \right)\left(2\theta \left(\gamma \right)-(\theta \left(\gamma \right)+b_2\left(\gamma \right))(1-exp(-\theta \left(\gamma \right)\tau ))\right)}$$ (13)

$$A_3(\tau ,\gamma )={\int }_0^{\tau }\left(\frac{{\sigma }_x^2}{2\gamma }+{\lambda }_x\overline{X}\right)A_2^2(s,\gamma )+\frac{{\sigma }_x^2}{2}{A}_1(s,\gamma )+(\gamma -1)\mu ds$$ (14)

$$b_1\left(\gamma \right)=\frac{1-\gamma }{\gamma }{b}_2\left(\gamma \right)=2\left(\frac{\gamma -1}{\gamma }{\sigma }_x-{\lambda }_x\right)b_3\left(\gamma \right)=\frac{{\sigma }_x^2}{\gamma }$$

$$\theta \left(\gamma \right)=\sqrt{b_2^2\left(\gamma \right)-4b_1\left(\gamma \right)b_3\left(\gamma \right)}$$

We provide an explicit formula for $A_3(\tau ,\gamma )$ in Appendix C.

Gatto and Schellhorn [1, Prop. 1] provide an explicit solution to the PDE that $F$ in (6) satisfies, but with some typos in the expression of $H_0(X_t,\tau )$, which we correct here.

Theorem 3 Proposition 1 in [1] —

If ${\sigma }_x<0$ then $I$ solves the martingale problem (6) , where

$$I\left(t\right)={\left(Z\left(t\right)\right)}^{\frac{1}{\gamma }}{H}_0(X_t,T-t)$$

where

$$\frac{dZ\left(t\right)}{Z\left(t\right)}=(r+X^2\left(t\right))dt+X\left(t\right)d{B}_2\left(t\right)$$

$$Z\left(0\right)={\left(\frac{I\left(0\right)}{H_0(X\left(0\right),T)}\right)}^{\gamma }$$

The corresponding control ${\alpha }^{\ast }\left(t\right)$ is given by:

$${\alpha }_0\left(t\right)=\frac{X\left(t\right)}{\gamma \sigma }-\frac{{\sigma }_x}{\gamma \sigma }(A_{0,1}(T-t)X\left(t\right)+A_{0,2}(T-t))$$ (15)

This control has some very clear properties. It is decomposed into a myopic policy $\frac{X\left(t\right)}{\gamma \sigma }$ and a hedging policy, namely the second term of (15). Recall that $X$ is most often negative. Both myopic control and hedging policies are thus inversely proportional to the degree of risk aversion $\left|\gamma \right|$ and to the volatility $\sigma$ which corresponds to the contemporaneous transmission measurement error. The hedging policy gives protection against the risk of making decisions too soon. As expected, its magnitude decreases as time approaches the horizon $T$. This is a typical feature of the Mayer problem, which is usually attenuated in the Bolza problem.

4. Results in the moderate infection regime

We first handle the Ornstein–Uhlenbeck treatment rate case, which was presented in [1, Prop. 2].

4.1. Treatment rate as Ornstein–Uhlenbeck process

The problem is defined in Section 2. We rewrite here for convenience,

$$sup\mathbb{E}\left[-\frac{I{(min(\tau ,T))}^{1-\gamma }}{1-\gamma }\right]$$$$dS\left(t\right)=-\beta S\left(t\right)I\left(t\right)dt+\sigma \sqrt{S\left(t\right)I\left(t\right)}d{B}_1\left(t\right)$$$$dI\left(t\right)=\left(\beta S\left(t\right)-\mu +\alpha \left(t\right)\sigma X\left(t\right)\right)I\left(t\right)dt+\alpha \left(t\right)I\left(t\right)\sigma d{B}_2\left(t\right)$$$$-{\sigma }_S\sqrt{S\left(t\right)I\left(t\right)}d{B}_1\left(t\right)$$$$dX\left(t\right)={\lambda }_x(\overline{X}-X\left(t\right))dt-{\sigma }_{x}d{B}_2\left(t\right)$$ (16)

We further define

$$\tilde{M}(t,\tau )=\frac{2\theta \left(\gamma \right)e^{\frac{1}{2}(b_2(\gamma /2)-b_2\left(\gamma \right)-\theta \left(\gamma \right))(\tau -t)}}{2\theta \left(\gamma \right)-(b_2\left(\gamma \right)+\theta \left(\gamma \right))\left(1-e^{-\theta \left(\gamma \right)(\tau -t)}\right)}$$ (17)

$$m_Y(\tau ,x)=x\tilde{M}(t,\tau )+{\int }_{s=t}^{\tau }\tilde{M}(s,\tau )({\lambda }_x\overline{X}+\frac{{\sigma }_x^2}{{}^{\gamma }}{A}_2(\tau -s,\gamma ))ds+\frac{A_2(T-\tau ,\gamma )}{A_1(T-\tau ,\gamma )}{V}_Y(\tau ,x)={\sigma }_x^2{\int }_t^{\tau }{\tilde{M}}^2(s,\tau )dsg(X,t)={\int }_{\tau =t}^T{H}_2(X,\tau -t)\frac{1}{2}\frac{{\beta }^2}{{\sigma }_S^2\gamma }\frac{1}{\sqrt{1-2V_Y(\tau ,X)A_1(T-\tau ,\gamma )/\gamma }}\times exp(\frac{2}{\gamma }{A}_3(T-\tau ,\gamma )-\frac{A_2^2(T-\tau ,\gamma )}{\gamma A_1(T-\tau ,\gamma )}$$ (18)

$$+\frac{m_Y^2(\tau ,X)A_1(T-\tau ,\gamma )}{\gamma -2V_Y(\tau ,X)A_1(T-\tau ,\gamma )})d\tau$$

From this, we can calculate:

$$\frac{\partial g}{\partial X}={\int }_{\tau =t}^T{H}_2(X,\tau -t)\frac{1}{2}\frac{{\beta }^2}{{\sigma }_S^2\gamma }\frac{1}{\sqrt{1-2V_Y(\tau ,X)A_1(T-\tau ,\gamma )/\gamma }}\times exp\left(\frac{2}{\gamma }{A}_3(T-\tau ,\gamma )-\frac{A_2^2(T-\tau ,\gamma )}{\gamma A_1(T-\tau ,\gamma )}+\frac{m_Y^2(\tau ,X)A_1(T-\tau ,\gamma )}{\gamma -2V_Y(\tau ,X)A_1(T-\tau ,\gamma )}\right)\times \left(\frac{A_1(\tau -t,\gamma /2)X\left(t\right)+A_2(\tau -t,\gamma /2)}{\gamma /2}+\frac{2m_Y(\tau ,X)A_1(T-\tau ,\gamma )}{\gamma -2V_Y(\tau ,X)A_1(T-\tau ,\gamma )}\tilde{M}(t,\tau )\right)d\tau$$

Theorem 4

Let $I\left(0\right)=\varepsilon$ . If ${\sigma }_x<0$ then $I$ solves the martingale problem (6) , where:

$$I\left(t\right)=\varepsilon Z^{1/\gamma }\left(t\right)H_1(X\left(t\right),T-t)+{\varepsilon }^2{Z}^{2/\gamma }\left(t\right)S\left(t\right)g(X\left(t\right),t)+O\left({\varepsilon }^3\right)$$

where $Z\left(t\right)$ satisfies:

$$\frac{dZ}{Z}=\left(-\mu +X^2+\frac{{\beta }^{2}SI}{{\sigma }_S^2}\right)dt-\frac{\beta \sqrt{SI}}{{\sigma }_S}d{B}_1+Xd{B}_2$$

$$Z\left(0\right)={\left(\frac{-H_1(X\left(0\right),T)+\sqrt{H_1^2(X\left(0\right),T)-4\varepsilon S\left(0\right)g(X\left(0\right),0)(\mathcal{O}\left({\varepsilon }^2\right)-1)}}{2\varepsilon S\left(0\right)g(X\left(0\right),0)}\right)}^{\gamma }$$

The corresponding control ${\alpha }^{\ast }\left(t\right)={\alpha }_0\left(t\right)+\varepsilon {\alpha }_1\left(t\right)+\mathcal{O}\left({\varepsilon }^2\right)$ where:

$${\alpha }_0\left(t\right)=\frac{X\left(t\right)}{\gamma \sigma }-\frac{{\sigma }_x}{\gamma \sigma }\left(A_1(T-t,\gamma )X\left(t\right)+A_2(T-t,\gamma )\right)$$

$${\alpha }_1\left(t\right)=\frac{Z^{1/\gamma }\left(t\right)S\left(t\right)}{H_1(X\left(t\right),T-t)\sigma }\left(\frac{g(X\left(t\right),t)X\left(t\right)}{\gamma }-{\sigma }_x\frac{\partial g}{\partial X}+{\sigma }_x\frac{g(X\left(t\right),t)}{\gamma }\left(A_1(T-t,\gamma )X\left(t\right)+A_2(T-t,\gamma )\right)\right)$$

The proof is in Appendix D. We refer to [1] for a discussion of ${\alpha }_0$. In the moderate infection regime, the control ${\alpha }_0$ (which is the control in the low infection regime) dominates the corrections due to transmission of the disease. Our asymptotic expansion thus indicates how the optimal control should change as the pandemic progresses. The sign of ${\alpha }_1$ is determined by the signs of $\sigma$ and

$$\frac{g(X\left(t\right),t)}{\gamma }\left(X\left(t\right)+{\sigma }_x\left(A_1(T-t,\gamma )X\left(t\right)+A_2(T-t,\gamma )\right)\right)-{\sigma }_x\frac{\partial g}{\partial X}$$ (19)

More specifically, ${\alpha }_1$ is positive if $\sigma$ and (19) are both positive or negative. ${\alpha }_1$ is negative if one of them is positive and the other one is negative.

It is obvious that the magnitude of both $g(X\left(t\right),t)$ and $\frac{\partial g}{\partial X}$ decrease with time and are equal to zero when $t=T$. Therefore, the importance of ${\alpha }_1$ decreases as time increases.

To further discuss the sign of (19), we rewrite it by

$$\left|\frac{g(X\left(t\right),t)}{\gamma }\right|\left(\left(1+\left|{\sigma }_x{A}_1(T-t,\gamma )\right|\right)X\left(t\right)+\overline{X}\left|\frac{{\sigma }_x{A}_2(T-t,\gamma )}{\overline{X}}\right|\right)+\left|{\sigma }_x\right|\frac{\partial g}{\partial X}$$

Thus, suppose $\frac{\partial g}{\partial X}$, $X\left(t\right)$, and $\overline{X}$ are all positive, (19) is positive, and vice versa. In the following cases, we provide two simple cases that we can easily discuss the sign of ${\alpha }_1$:

• •if $\frac{\partial g}{\partial X}\sigma >0$, $\mu -{\mu }_1>max(K_1\left(t\right),{\overline{k}}_1)$, then ${\alpha }_1$ is positive.
• •if $\frac{\partial g}{\partial X}\sigma <0$, $\mu -{\mu }_1<min(K_1\left(t\right),{\overline{k}}_1)$, then ${\alpha }_1$ is negative.

In the following, we discuss the full expansion of the solution in Theorem 4. Consider equation (57) in [1]:

$$\left(\frac{\partial }{\partial t}+L_1+\varepsilon L_2\right)f=0$$

This time we use full asymptotic expansion:

$$f=f_1+\varepsilon f_2+\cdots =\sum _{i=1}^{\infty }{f}_i{\varepsilon }^{i-1}$$

and obtain:

$$0=\left(\frac{\partial }{\partial t}+L_1\right)f_1+\sum _{i=1}^{\infty }\left(\left(\frac{\partial }{\partial t}+L_1\right)f_{i+1}+L_2{f}_i\right){\varepsilon }^i$$

The terms of our asymptotic expansion are thus determined by:

$$\left(\frac{\partial }{\partial t}+L_1\right)f_1=0$$ (20)

$$\left(\frac{\partial }{\partial t}+L_1\right)f_{i+1}=-L_2{f}_{i}i=1,2,\dots$$ (21)

We use the Ansatz:

$$f_i(Z\left(t\right),X\left(t\right),t)=Z{\left(t\right)}^{2^{i-1}/\gamma }S{\left(t\right)}^{2^{i-1}-1}{g}_i(X\left(t\right),t)i=1,2,\dots$$

We have showed that $g_1=H_1$ and $g_2=g$ in the proof of Theorem 4. By the same process, we can also calculate the expressions for $g_3$, $g_4,\dots$ in the sequel.

4.2. Constant treatment rate

The problem is:

$$sup\mathbb{E}\left[-\frac{I{\left(T\right)}^{1-\gamma }}{1-\gamma }\right]$$$$dS\left(t\right)=-\beta S\left(t\right)I\left(t\right)dt+{\sigma }_S\sqrt{S\left(t\right)I\left(t\right)}d{B}_1\left(t\right)$$$$dI\left(t\right)=(r+\alpha \left(t\right)(b-r))I\left(t\right)dt+\alpha \left(t\right)\sigma I\left(t\right)d{B}_2\left(t\right)-{\sigma }_S\sqrt{S\left(t\right)I\left(t\right)}d{B}_1\left(t\right)$$ (22)

The HJB equation of this problem is obtained by simplifying (6), i.e., making the value function independent from $X$. Let $\tau =T-t$, the solution kernels $h_i\left(\tau \right)$ for $i=1,2,\dots$ are given by:

$$h_i\left(\tau \right)=exp\left(\frac{2^{i-1}}{\gamma }\left(\frac{a_{i,1}}{2}{\left(\frac{b-r}{\sigma }\right)}^2+a_{i,2}\right)\tau \right)$$ (23)

where

$$a_{i,1}=\frac{1-\gamma /2^{i-1}}{\gamma /2^{i-1}}$$ (24)

$$a_{i,2}=(\gamma /2^{i-1}-1)\mu$$ (25)

Theorem 5

Let $I\left(0\right)=\varepsilon$ , then $I$ solves the martingale problem (6) , where:

$$I\left(t\right)=\sum _{i=1}^{\infty }Z{\left(t\right)}^{2^{i-1}/\gamma }S{\left(t\right)}^{2^{i-1}-1}{g}_i\left(t\right){\varepsilon }^i$$

where $g_1\left(t\right)=h_1(T-t)$ , $g_i\left(t\right)$ , $i>1$ can be obtained by (47) , and $Z\left(t\right)$ satisfies:

$$\frac{dZ}{Z}=\left(-\mu +{\left(\frac{b-r}{\sigma }\right)}^2+\frac{{\beta }^{2}SI}{{\sigma }_S^2}\right)dt-\frac{\beta \sqrt{SI}}{{\sigma }_S}d{B}_1+\frac{b-r}{\sigma }d{B}_2$$

$$1=\sum _{i=1}^{\infty }Z{\left(0\right)}^{2^{i-1}/\gamma }S{\left(0\right)}^{2^{i-1}-1}{g}_i\left(0\right){\varepsilon }^{i-1}$$

The corresponding control ${\alpha }^{\ast }\left(t\right)$ is equal to ${\alpha }_0\left(t\right)+\varepsilon {\alpha }_1\left(t\right)+\mathcal{O}\left({\varepsilon }^2\right)$ , where ${\alpha }_0\left(t\right)$ and ${\alpha }_1\left(t\right)$ are equal to

$${\alpha }_0=\frac{b-r}{\gamma {\sigma }^2}{\alpha }_1=\frac{Z^{1/\gamma }\left(t\right)S\left(t\right)g_2\left(t\right)}{h_1(T-t)}\frac{b-r}{\gamma {\sigma }^2}$$

The proof is in Appendix E, where we also provide a formula for $g_3$. Observe that

$$g_2\left(t\right)=\frac{{\beta }^2}{2{\sigma }_S^2}\frac{h_2(T-t)-h_1^2(T-t)}{{\left(\frac{b-r}{\sigma }\right)}^2(a_{1,1}-a_{2,1})+2(a_{1,2}-a_{2,2})}=\frac{{\beta }^2}{2{\sigma }_S^2}\frac{h_2(T-t)-h_1^2(T-t)}{\gamma \mu -{\left(\frac{b-r}{\sigma }\right)}^2/\gamma }$$

is always positive because the signs of $h_2(T-t)-h_1^2(T-t)$ and $\gamma \mu -{\left(\frac{b-r}{\sigma }\right)}^2/\gamma$ are the same. The signs of ${\alpha }_0$ and ${\alpha }_1$ are determined by the sign of $\frac{r-b}{{\sigma }^2}$.

5. Application to COVID-19

We use the same weekly US COVID-19 data from June 7, 2020 to November 1, 2020 as in ([1] Sec. 5.1) and the parameters in Table 1 are estimated using the COVID-19 data set. In the following, we show both simulation plots and one scenario real data plots under low infection regime with constant treatment, low infection regime with OU treatment, moderate infection regime with constant treatment, moderate infection regime with OU treatment, respectively. We compare three types of treatment:

• •no control, i.e., $\alpha \left(t\right)=0$.
• •full control, i.e., $\alpha \left(t\right)=1$.
• •optimal control, i.e. the control from Theorems 2, 3, 4, 5.

The Github repository for implementing the models and generating the plots can be found at https://github.com/yujiading/optimal-control-sir-model.

Table 1.

Parameters.

Treatment parameter	Symbol	Value
Death rate/no treatment	${\mu }_0$	0.0575
Death rate	${\mu }_1$	0.0575
Recovery rate/no treatment	$K_0$	0.2559
Recovery rate at time 0	$K_1\left(0\right)$	0.2559
Long run value of recovery rate	${\overline{k}}_1$	0.4612
Volatility of the measurement of today’s recovery rate	$\sigma$	0.4418
Volatility of changes in the recovery rate	${\sigma }_k$	−1.1647
Speed of mean-reversion of the recovery rate	${\lambda }_k$	0.7692
Transmission rate	$\beta$	0.025
Proportion of infected at time 0	$\varepsilon$	0.01
Time step	$\Delta t$	0.001
Volatility of the measurement of today’s susceptible rate	${\sigma }_S$	2.17

5.1. Simulations

In Figs. 2, 3, 4, and 5 we use the Euler scheme to simulate $1,000,000$ scenarios using the parameters in Table 1 and calculate the expected values of $I\left(T\right)$ and utility of $I\left(T\right)$ for each regime. The risk-aversion parameters $\gamma$ that are considered are between $-0.5$ and $-5$. In all the regimes, the expected utility of our control is higher than full control and no control, as expected. This effect is more pronounced in the low infection than in the moderate infection regime. When $\gamma$ becomes more negative, the expected utility of no control is higher than that of full control. This is because when $\gamma$ becomes more negative, the decision-maker becomes more risk averse and trades off expected value of $I\left(T\right)$ against dispersion of $I\left(T\right)$. In the low infection regime, the expected number of infected is lower with full control than with no control, as expected, and, with the optimal control, it depends on the level of risk-aversion, as expected. When treatment is risky the more risk-averse a decision maker, the less he or she is likely to invest in treatment.

Fig. 2.

Expected infection and expected utility of optimal control, full control, and no control of low infection regime with constant treatment. Using $1,000,000$ simulation scenarios.

Fig. 3.

Expected infection and expected utility of optimal control, full control, and no control of low infection regime with OU treatment. Using $1,000,000$ simulation scenarios.

Fig. 4.

Expected infection and expected utility of optimal control, full control, and no control of moderate infection regime with constant treatment. Using $1,000,000$ simulation scenarios.

Fig. 5.

Expected infection and expected utility of optimal control, full control, and no control of moderate infection regime with OU treatment. Using $1,000,000$ simulation scenarios.

5.2. One scenario real data

In Fig. 6, we use the one scenario COVID-19 data as introduced above to plot the infections with optimal control, full control, and no control for each of the regimes. We can see that as gamma varies, some plots show contrafactual results. In fact, it is possible for a particular scenario to result in a better or worse outcome. This does not contradict our theoretical results, as in stochastic control the goal is not to optimize a single scenario.

Fig. 6.

Infection of optimal control, full control, and no control of low infection regime with constant treatment, low infection regime with OU treatment, moderate infection regime with constant treatment, and moderate infection regime with OU treatment. Using real COVID-19 data.

6. Conclusion

We showed that a stochastic optimal control approach enables to more efficiently use treatment in an pandemic such as COVID-19. On a theoretical level, we show that, in a first approximation, the control does not depend on the transmission of the pandemic, i.e., the control in the moderate infection regime resembles the control in the low infection regime. The influence of key parameters of the problems, namely risk-aversion and volatility, are clearly demonstrated in our formulas. On a practical level, we show that an optimal control would have been better than a full control, had it been available in the low infection regime which we experienced in summer 2020 in the US for COVID-19. However the relative poor efficiency of the treatment that we experienced then would have translated into a poor performance of any type of control, had the pandemic moved into a moderate infection regime. In that regime, the influence of the type of control would have turned out not to be significant.

In the Mayer problem, which we study here, the control depends significantly on the horizon. The study of the full Bolza problem remains to be done. Many other interesting problems remain to be solved. For instance, we showed optimality of the constrained control only in the constant, low infection regime case. Verification theorems need to be worked out in the multiple treatment case or the Ornstein–Uhlenbeck case. Optimal vaccination is another area where we believe a similar asymptotic approach can be used. Finally, Bertozzi et al. [20] use Hawkes processes to model COVID-19. The control of Hawkes processes remains a largely open problem that deserves attention, in particular for its application to epidemiology.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A. Proof of Theorem 1

We follow the proof in [16, Prop. 2.13, Sec. 5.2]. They consider the one-dimensional case. Let $h:[0,\infty )\to [0,\infty )$ be a strictly increasing function with $h\left(0\right)$ and

$${\int }_{(0,\varepsilon )}{h}^{-2}\left(u\right)=\infty \text{, }\forall \varepsilon >0$$ (26)

In our case, we take $h\left(x\right)=x$.  Because of (26), there exists a strictly decreasing sequence $\left\{a_n\right\}\subset (0,1]$ with $a_0$ and ${lim}_{n\to \infty }{a}_n=0$ such that ${\int }_{a_n}^{a_{n-1}}{h}^{-2}\left(u\right)du=n$. For every $n$ there exists continuous function ${\rho }_n$ on $\mathbb{R}$ with support on $(a_n,a_{n-1})$ so that

$$0\le {\rho }_n\left(x\right)\le \frac{2}{nh\left(x\right)}\text{, }x>0$$

and ${\int }_{a_{n-1}}^{a_n}{\rho }_n\left(u\right)du=1$. Then the function

$${\Psi }_n\left(x\right)={\int }_0^{\left|x\right|}{\int }_0^y{\rho }_n\left(u\right)dudy$$

is even and twice continuously differentiable, with $\left|{\Psi }_n^{\prime }\left(x\right)\right|\le 1$ and ${lim}_{n\to \infty }$ ${\Psi }_n\left(x\right)$ $=\left|x\right|$. Suppose there are two strong solutions $(I^{\left(1\right)},S^{\left(1\right)})$ and $(I^{\left(2\right)},S^{\left(2\right)})$,

$$d(I^{\left(1\right)}-I^{\left(2\right)}-\mathbb{E}[I^{\left(1\right)}-I^{\left(2\right)}])-\alpha \sigma (I^{\left(1\right)}-I^{\left(2\right)})d{B}_2=-{\sigma }_S\left(\sqrt{S^{\left(1\right)}{I}^{\left(1\right)}}-\sqrt{S^{\left(2\right)}{I}^{\left(2\right)}}\right)d{B}_1=-d(S^{\left(1\right)}-S^{\left(2\right)}-\mathbb{E}[S^{\left(1\right)}-S^{\left(2\right)}])$$

so that

$${\left(d(I^{\left(1\right)}-I^{\left(2\right)})\right)}^2<{\sigma }_S^2(S^{\left(1\right)}{I}^{\left(1\right)}-S^{\left(2\right)}{I}^{\left(2\right)})dt+{\left(\alpha \sigma \right)}^2{(I^{\left(1\right)}-I^{\left(2\right)})}^{2}dt{\left(d(S^{\left(1\right)}-S^{\left(2\right)})\right)}^2<{\sigma }_S^2(S^{\left(1\right)}{I}^{\left(1\right)}-S^{\left(2\right)}{I}^{\left(2\right)})dtd\left((I^{\left(1\right)}-I^{\left(2\right)})(S^{\left(1\right)}-S^{\left(2\right)})\right)<-{\sigma }_S^2(S^{\left(1\right)}{I}^{\left(1\right)}-S^{\left(2\right)}{I}^{\left(2\right)})dt$$

Thus, since $\left|{\Psi }_n^{\prime }\right|<1$,

$$\mathbb{E}[d{\Psi }_n(I_t^{\left(1\right)}-I_t^{\left(2\right)})+d{\Psi }_n(S_t^{\left(1\right)}-S_t^{\left(2\right)})]=\mathbb{E}[{\Psi }_n^{\prime }(I_t^{\left(1\right)}-I_t^{\left(2\right)})\left(d(I^{\left(1\right)}-I^{\left(2\right)})\right)+{\Psi }_n^{\prime }(S_t^{\left(1\right)}-S_t^{\left(2\right)})\left(d(S^{\left(1\right)}-S^{\left(2\right)})\right)]+\frac{1}{2}\mathbb{E}\left[{\Psi }_n^{″}(I_t^{\left(1\right)}-I_t^{\left(2\right)}){\left(d(I^{\left(1\right)}-I^{\left(2\right)})\right)}^2\right]+\frac{1}{2}\mathbb{E}\left[{\Psi }_n^{″}(S_t^{\left(1\right)}-S_t^{\left(2\right)}){\left(d(S^{\left(1\right)}-S^{\left(2\right)})\right)}^2\right]\le \mathbb{E}[2\left|\beta \right||S^{\left(1\right)}{I}^{\left(1\right)}-S^{\left(2\right)}{I}^{\left(2\right)}|dt+\left|D\right||I^{\left(1\right)}-I^{\left(2\right)}|dt]+\frac{1}{2}\mathbb{E}\left[{\Psi }_n^{″}(I_t^{\left(1\right)}-I_t^{\left(2\right)}){\sigma }_S^2(S^{\left(1\right)}{I}^{\left(1\right)}-S^{\left(2\right)}{I}^{\left(2\right)})dt\right]+\frac{1}{2}\mathbb{E}\left[{\Psi }_n^{″}(S_t^{\left(1\right)}-S_t^{\left(2\right)}){\sigma }_S^2(S^{\left(1\right)}{I}^{\left(1\right)}-S^{\left(2\right)}{I}^{\left(2\right)})dt\right]+\frac{1}{2}\mathbb{E}\left[{\Psi }_n^{″}(I_t^{\left(1\right)}-I_t^{\left(2\right)}){\left(\alpha \sigma \right)}^2{(I^{\left(1\right)}-I^{\left(2\right)})}^2\right]dt$$

where

$$D=-(K_0+{\mu }_0)+\alpha (K_0-K_1+{\mu }_0-{\mu }_1)$$

Observe that:

$$S^{\left(1\right)}{I}^{\left(1\right)}-S^{\left(2\right)}{I}^{\left(2\right)}=S^{\left(1\right)}(I^{\left(1\right)}-I^{\left(2\right)})+I^{\left(2\right)}(S^{\left(1\right)}-S^{\left(2\right)})<|I^{\left(1\right)}-I^{\left(2\right)}|+|S^{\left(1\right)}-S^{\left(2\right)}|$$

Since ${\Psi }_n^{″}<2/nh$ and $h$ is positive,

$$\left({\Psi }_n^{″}(I_t^{\left(1\right)}-I_t^{\left(2\right)})+{\Psi }_n^{″}(S_t^{\left(1\right)}-S_t^{\left(2\right)})\right)\left(S^{\left(1\right)}{I}^{\left(1\right)}-S^{\left(2\right)}{I}^{\left(2\right)}\right)<\frac{2}{n}\left(\frac{1}{h\left(|I^{\left(1\right)}-I^{\left(2\right)}|\right)}+\frac{1}{h\left(|S^{\left(1\right)}-S^{\left(2\right)}|\right)}\right)(|I^{\left(1\right)}-I^{\left(2\right)}|+|S^{\left(1\right)}-S^{\left(2\right)}|)<\frac{2}{n}\left(\frac{|I^{\left(1\right)}-I^{\left(2\right)}|}{h\left(|I^{\left(1\right)}-I^{\left(2\right)}|\right)}+\frac{|S^{\left(1\right)}-S^{\left(2\right)}|}{h\left(|S^{\left(1\right)}-S^{\left(2\right)}|\right)}\right)$$

Taking $h\left(x\right)=x$ results in

$$\mathbb{E}[d{\Psi }_n(I_t^{\left(1\right)}-I_t^{\left(2\right)})+d{\Psi }_n(S_t^{\left(1\right)}-S_t^{\left(2\right)})]<\left(\mathbb{E}\left[(2\left|\beta \right|+\left|D\right|)|I^{\left(1\right)}-I^{\left(2\right)}|\right]+\mathbb{E}\left[2\left|\beta \right||S^{\left(1\right)}-S^{\left(2\right)}|\right]+\frac{2{\sigma }_S^2}{n}+\frac{{\left(\alpha \sigma \right)}^2}{2}\mathbb{E}\left[|I^{\left(1\right)}-I^{\left(2\right)}|\right]\right)dt$$

Since ${lim}_{n\to \infty }{\Psi }_n\left(x\right)=\left|x\right|$,

$$\mathbb{E}[|I_t^{\left(1\right)}-I_t^{\left(2\right)}|+|S_t^{\left(1\right)}-S_t^{\left(2\right)}|]<{\int }_0^t\mathbb{E}\left[(2\left|\beta \right|+\left|D_s\right|)|I_s^{\left(1\right)}-I_s^{\left(2\right)}|\right]+\mathbb{E}\left[2\left|\beta \right||S_s^{\left(1\right)}-S_s^{\left(2\right)}|\right]+\frac{{\left({\alpha }_s\sigma \right)}^2}{2}\mathbb{E}\left[|I_s^{\left(1\right)}-I_s^{\left(2\right)}|\right]ds$$

But,

$$\mathbb{E}\left[(2\left|\beta \right|+\left|D_s\right|)|I_s^{\left(1\right)}-I_s^{\left(2\right)}|\right]<\sqrt{\mathbb{E}\left[{(2\left|\beta \right|+\left|D_s\right|)}^2\right]}\sqrt{\mathbb{E}\left[{|I_s^{\left(1\right)}-I_s^{\left(2\right)}|}^2\right]}$$

Since $|I_s^{\left(1\right)}-I_s^{\left(2\right)}|<1$, $\mathbb{E}\left[{(I_s^{\left(1\right)}-I_s^{\left(2\right)})}^2\right]<1$ and $\sqrt{\mathbb{E}\left[{(I_s^{\left(1\right)}-I_s^{\left(2\right)})}^2\right]}<\mathbb{E}\left[|I_s^{\left(1\right)}-I_s^{\left(2\right)}|\right]$ thus

$$\mathbb{E}[|I_t^{\left(1\right)}-I_t^{\left(2\right)}|+|S_t^{\left(1\right)}-S_t^{\left(2\right)}|]<{\int }_0^t\left(\sqrt{\mathbb{E}\left[{(2\left|\beta \right|+\left|D_s\right|)}^2\right]}+\frac{{\left(\alpha \sigma \right)}^2}{2}\right)\mathbb{E}\left[|I_s^{\left(1\right)}-I_s^{\left(2\right)}|\right]+2\left|\beta \right|\mathbb{E}\left[|S_s^{\left(1\right)}-S_s^{\left(2\right)}|\right]ds<{\int }_0^{t}max\left(\sqrt{\mathbb{E}\left[{(2\left|\beta \right|+\left|D_s\right|)}^2\right]}+\frac{{\left(\alpha \sigma \right)}^2}{2},2\left|\beta \right|\right)\times \mathbb{E}[|I_s^{\left(1\right)}-I_s^{\left(2\right)}|+|S_s^{\left(1\right)}-S_s^{\left(2\right)}|]ds$$

and local uniqueness follows by Gronwall’s inequality.

Appendix B. Proof of Theorem 2

We refer to the problem treated by Gatto and Schellhorn [1] as the unconstrained problem. Indeed, in that problem $\alpha$ was not constrained. We refer to our problem as the constrained problem. We follow the method of proof in [19], referred to hereafter as CK. They introduce auxiliary problems, which are unconstrained. They show that there exists an auxiliary problem which solution can be used to construct the solution of the original constrained problem. We follow the numbering of the sections in CK in order to ease understanding.

CK Section 2. The model.

To ease the correspondence with the CK paper, we define $b-r=K_0+{\mu }_0-{\mu }_1-{\overline{k}}_1$, $\theta ≔(b-r)/\sigma$, and

$$H^{\left(0\right)}\left(t\right)=exp(-rt)exp\left(-{\int }_0^t\theta d{B}_2\left(s\right)-\frac{1}{2}{\int }_0^t{\theta }^{2}ds\right)$$

Observe that $\mathbb{E}\left[{\int }_0^t{\theta }^{2}ds\right]<\infty$.

CK Section 3. Portfolio and consumption processes.

Define:

$$B_2^{\left(0\right)}\left(t\right)=B_2\left(t\right)+{\int }_0^t\theta ds$$

Denote by $I^{i,\alpha }$ the infected process subject to $I\left(0\right)=i$ and control $\alpha$. It is admissible if

$$0\le I^{i,\alpha }\left(t\right)\le 1\forall 0\le t\le T$$

The set of admissible $\alpha$ is denoted ${\mathcal{A}}_0\left(i\right)$. Note that (See (3.5) in CK)

$$H^{\left(0\right)}\left(t\right)I\left(t\right)=i+{\int }_0^t{H}^{\left(0\right)}\left(s\right)I\left(s\right)(\alpha \left(s\right)\sigma -\theta )d{B}_2\left(s\right)$$

CK Section 4. Convex sets and their support functions.

The difference between CK and this paper is that our objective is to minimize. This means that the key relation between our auxiliary infected and infected is reversed compared to the first equation in CK. Indeed if ${\alpha }_{\nu }$ solves the auxiliary problem and $\alpha$ the original problem, we must have:

$$I_{\nu }^{i,{\alpha }_{\nu }}\left(t\right)\le I^{i,\alpha }\left(t\right)$$

Define

$$\delta \left(\nu \right)=\left\{\begin{array}{cc}\hfill 0\hfill & \hfill \nu <0\hfill \\ \hfill \nu \hfill & \hfill \nu >0\hfill \end{array}\right)$$ (27)

It is subadditive:

$$\delta (\lambda +\nu )\le \delta \left(\lambda \right)+\delta \left(\nu \right)$$ (28)

CK Section 5. Utility functions.

The main difference between our utility functions and the utility functions in financial economics is that our utility functions are decreasing for positive arguments. Recall indeed that our utility function is, for $\gamma <0$:

$$U\left(i\right)=-\frac{i^{1-\gamma }}{1-\gamma }$$

Since

$$U^{\prime }\left(i\right)=-i^{-\gamma }$$

We have ${lim}_{i\to \infty }{U}^{\prime }\left(i\right)=-\infty$ and ${lim}_{i\to 0}{U}^{\prime }\left(i\right)=0$, again for $\gamma <0$. This is unlike CK and [21] who consider the case $0<\gamma <1$ with utility of wealth $U_2\left(x\right)=\frac{x^{1-\gamma }}{1-\gamma }$. In their case, ${lim}_{x\to \infty }{U}_2^{\prime }\left(x\right)=0$ and ${lim}_{x\to 0}{U}_2^{\prime }\left(x\right)=\infty$.

We define $I_2$ to be the inverse of $U^{\prime }$, with $I_2\left(y\right)$ on $y\le 0$. By straightforward calculations:

$$I_2\left(y\right)={(-y)}^{-1/\gamma }$$

We also define the Legendre–Fenchel dual

$$\tilde{U}\left(y\right)=\underset{x>0}{max}[U\left(x\right)-xy]=U\left(I_2\left(y\right)\right)-y{I}_2\left(y\right)$$

This function satisfies:

$${\tilde{U}}^{\prime }\left(y\right)=-I_2\left(y\right)y\le 0$$

CK Section 6. The constrained and unconstrained optimization problems.

We define:

$${\mathcal{A}}^{\prime }\left(i\right)=\{\alpha \in {\mathcal{A}}_0\left(i\right)|0\le \alpha \le 1\}$$

The supremum of the unconstrained problem is denoted by $V_0$, while the supremum of the constrained problem is denoted by $V$, namely:

$$V_0\left(i\right)=\underset{\alpha \in {\mathcal{A}}_0\left(i\right)}{sup}\mathbb{E}\left[U\left(I^{i,\alpha }\left(T\right)\right)\right|I\left(0\right)=i]$$

$$V\left(i\right)=\underset{\alpha \in {\mathcal{A}}^{\prime }\left(i\right)}{sup}\mathbb{E}\left[U\left(I^{i,\alpha }\left(T\right)\right)\right|I\left(0\right)=i]$$

CK Section 7. Solution of the unconstrained problem.

We note that the expectation

$${\mathcal{X}}_0\left(y\right)\equiv \mathbb{E}\left[H^{\left(0\right)}\left(T\right)I_2\left(y{H}^{\left(0\right)}\left(T\right)\right)\right]$$

is finite for every $y\in (-\infty ,0]$. We define its inverse ${\mathcal{Y}}_0$:

$${\mathcal{Y}}_0\left({\mathcal{X}}_0\left(y\right)\right)=y$$

The solution of the unconstrained problem is well-known, and equal to:

$$\alpha \left(s\right)=\frac{\theta }{\sigma \gamma }=\frac{r-b}{{\sigma }^2\left|\gamma \right|}$$

CK Section 8. Auxiliary unconstrained optimization problems.

Recall $\delta \left(\nu \right)$ in (27). It is easily seen that:

$$\alpha \nu -\delta \left(\nu \right)=\left\{\begin{array}{cc}\alpha \nu \hfill & \nu <0\hfill \\ (\alpha -1)\nu \hfill & \nu >0\hfill \end{array}\right\}\le 0$$

We introduce a new process $I^{\left(\nu \right)}$ by:

$$\frac{d{I}^{\left(\nu \right)}\left(t\right)}{I^{\left(\nu \right)}\left(t\right)}=(r+\alpha \left(t\right)\nu \left(t\right)-\delta \left(\nu \left(t\right)\right))dt+\alpha \left(t\right)\sigma d{B}_2^{\left(0\right)}\left(t\right)$$

Likewise we introduce

$${\theta }^{\left(\nu \right)}=\theta +\nu /\sigma$$

$$B^{\left(\nu \right)}\left(t\right)=B_2\left(t\right)+{\int }_0^t{\theta }^{\left(\nu \right)}\left(s\right)ds$$

$$H^{\left(\nu \right)}\left(t\right)=exp\left(-rt+{\int }_0^t\delta \left(\nu \left(s\right)\right)ds\right)\mathcal{E}\left(-{\int }_0^t{\theta }^{\left(\nu \right)}\left(s\right)d{B}_2\left(s\right)\right)$$

$$\mathcal{E}\left(-{\int }_0^t{\theta }^{\left(\nu \right)}\left(s\right)d{B}_2\left(s\right)\right)\equiv exp\left(-{\int }_0^t{\theta }^{\left(\nu \right)}\left(s\right)d{B}_2\left(s\right)-\frac{1}{2}{\int }_0^t{\left({\theta }^{\left(\nu \right)}\left(s\right)\right)}^{2}ds\right)$$

We denote by ${\mathcal{A}}_{\nu }^{\prime }\left(i\right)$ the class of $\alpha$ for which

$$I_{\nu }^{i,\alpha }\left(t\right)\le 1$$

Since the solution of our dual problem will have $\alpha \left(t\right)\nu \left(t\right)-\delta \left(\nu \left(t\right)\right)\le 0$, clearly ${\mathcal{A}}^{\prime }\left(i\right)\subset {\mathcal{A}}_{\nu }^{\prime }\left(i\right)$. We define:

$$V_{\nu }\left(i\right)=\underset{\pi \in {\mathcal{A}}_{\nu }^{\prime }\left(i\right)}{sup}\mathbb{E}\left[U\left(I^{i,\alpha }\left(T\right)\right)\right]$$

$${\mathcal{X}}_{\nu }\left(y\right)\equiv \mathbb{E}\left[H^{\left(\nu \right)}\left(T\right)I_2\left(y{H}^{\left(\nu \right)}\left(T\right)\right)\right]$$

We define a class of progressively measurable processes $\nu$ in $\mathbb{R}$ by:

$${\mathcal{D}}^{\prime }=\left\{\nu ;\mathbb{E}{\int }_0^T\delta \left(\nu \left(t\right)\right)dt\le \infty ,\mathbb{E}\int {\nu }^2\left(t\right)dt<\infty ,{\mathcal{X}}_{\nu }\left(y\right)<\infty ,y\in (-\infty ,0]\right\}$$

Proposition 8.3. in CK shows that, if for some $\lambda \in$ ${\mathcal{D}}^{\prime }$ the corresponding control ${\alpha }_{\lambda }$ is optimal for the auxiliary optimization problem and if

$$-\delta \left(\lambda \right)+{\alpha }_{\lambda }\left(t\right)\lambda \left(t\right)=0$$

then $\alpha \in {\mathcal{A}}^{\prime }\left(i\right)$ and is optimal for the constrained problem.

The solution of the unconstrained problem is:

$$\alpha \left(s\right)=\frac{{\theta }^{\left(\nu \right)}}{\sigma \gamma }=\frac{\theta +\nu /\sigma }{\sigma \gamma }=\frac{r-b-\nu }{{\sigma }^2\left|\gamma \right|}$$ (29)

CK Section 9. Contingent claims attainable by constrained portfolios.

We sketch the proof of Theorem 9.1 in CK, as the signs are different, and the structure of the control is slightly different.

CK 9.1 Theorem

Let $B$ be a positive ${\mathcal{F}}_T$ -measurable random variable and suppose there is a process $\lambda \in {\mathcal{D}}^{\prime }$ such that, for all $\nu \in {\mathcal{D}}^{\prime }$

$$\mathbb{E}\left[H^{\left(\nu \right)}\left(T\right)B\right]\le \mathbb{E}\left[H^{\left(\lambda \right)}\left(T\right)B\right]≔i$$ (30)

Then there exists a control $\alpha \in {\mathcal{A}}^{\prime }\left(i\right)$ such that $I^{i,\alpha }=B$ .

Sketch of Proof

See CK p.782 for a definition of the stopping time ${\tau }_n$. By (30) and subadditivity of $\delta$ (28):

$$0\le \underset{\varepsilon \downarrow 0}{lim}sup\frac{1}{\varepsilon }\mathbb{E}\left[(H^{\left(\lambda \right)\left(T\right)}-H^{(\lambda +\varepsilon (\nu -\lambda ))}\left(T\right))B\right]=\underset{\varepsilon \downarrow 0}{lim}sup\frac{1}{\varepsilon }\mathbb{E}\left[H^{\left(\lambda \right)}\left(T\right)B\left(1-exp\left({\int }_0^{T\wedge {\tau }_n}\left(\delta (\lambda \left(t\right)+\varepsilon (\nu \left(t\right)-\lambda \left(t\right)))-\delta \left(\lambda \left(t\right)\right)\right)dt\times \mathcal{E}\left({\int }_0^{T\wedge {\tau }_n}(-{\theta }^{\left(\lambda \right)}\left(t\right)+{\theta }^{(\lambda +\varepsilon (\nu -\lambda ))}\left(t\right))d{B}_2^{\left(\lambda \right)}\left(t\right)\right)\right)\right)\right]\le \underset{\varepsilon \downarrow 0}{lim}sup\mathbb{E}\left[H^{\left(\lambda \right)}\left(T\right)B(L_T+N_T)\right]$$

where

$${\breve{\delta }}^{\left(\nu \right)}\left(\lambda \left(t\right)\right)=\left\{\begin{array}{cc}\delta \left(\lambda \left(t\right)\right)\hfill & \nu =0\hfill \\ -\delta (\nu \left(t\right)-\lambda \left(t\right))\hfill & \text{otherwise}\hfill \end{array}\right)$$

$$L_T={\int }_0^{T\wedge {\tau }_n}{\breve{\delta }}^{\left(\nu \right)}\left(\lambda \left(t\right)\right)dt$$

$$N_T={\int }_0^{T\wedge {\tau }_n}\frac{\nu \left(t\right)-\lambda \left(t\right)}{\sigma }d{B}_2^{\left(\lambda \right)}\left(t\right)$$

By Ito’s lemma.

$$d\left[H^{\left(\lambda \right)}\left(t\right)I\left(t\right)(L_t+N_t)\right]=I\left(t\right)H^{\left(\lambda \right)}\left(t\right)d(L_t+N_t)+(L_t+N_t)H^{\left(\lambda \right)}\left(t\right)I\left(t\right)\alpha \left(t\right)\sigma d{B}_2^{\left(\lambda \right)}\left(t\right)+I\left(t\right)H^{\left(\lambda \right)}\left(t\right)\alpha \left(t\right)(\nu \left(t\right)-\lambda \left(t\right))dt$$

which implies

$$H^{\left(\lambda \right)}\left(T\right)I\left(T\right)(L_T+N_T)={\int }_0^{{\tau }_n}I\left(t\right)H^{\left(\lambda \right)}\left(t\right)\left(\frac{\nu \left(t\right)-\lambda \left(t\right)}{\sigma }+(L_t+N_t)\sigma \alpha \left(t\right)\right)d{B}_2^{\left(\lambda \right)}\left(t\right)+{\int }_0^{{\tau }_n}{H}^{\left(\lambda \right)}\left(t\right)I\left(t\right)\left(\alpha \left(t\right)(\nu \left(t\right)-\lambda \left(t\right))dt+d{L}_t\right)$$

Therefore,

$$0\le \mathbb{E}\left[H^{\left(\lambda \right)}\left(T\right)B(L_T+N_T)\right]=\mathbb{E}\left[{\int }_0^{{\tau }_n}{H}^{\left(\lambda \right)}\left(t\right)I\left(t\right)\left(\alpha \left(t\right)(\nu \left(t\right)-\lambda \left(t\right))dt+d{L}_t\right)\right]$$

It is easy to see that, for any $\rho \in {\mathcal{D}}^{\prime }$, take $\nu =\lambda +\rho$:

$$-\delta \left(\rho \left(t\right)\right)+\alpha \left(t\right)\rho \left(t\right)\ge 0$$ (31)

and, taking $\nu \left(t\right)=0$, we obtain:

$$-\delta \left(\lambda \left(t\right)\right)+\alpha \left(t\right)\lambda \left(t\right)\le 0$$

which together with (31) for $\rho =\lambda$ yields:

$$-\delta \left(\lambda \left(t\right)\right)+\alpha \left(t\right)\lambda \left(t\right)=0\square$$

CK Section 10. Equivalent optimality conditions.

The most important implication to prove is (D) $\Rightarrow$ (B) $\Rightarrow$ (A) in CK. It shows that the solution of the dual problem solves the auxiliary problem, and that, moreover, it is feasible and optimal for the original constrained problem. We make it more explicit here.

(Part of) CK 10.1 Theorem

Suppose that for every $\nu \in {\mathcal{D}}^{\prime }$ ,

$$\mathbb{E}\left[\tilde{U}\left({\mathcal{Y}}_{\lambda }\left(i\right)H^{\left(\lambda \right)}\left(T\right)\right)\right]\le \mathbb{E}\left[\tilde{U}\left({\mathcal{Y}}_{\lambda }\left(i\right)H^{\left(\nu \right)}\left(T\right)\right)\right]$$ (32)

then there exists a control ${\alpha }_{\lambda }\in [0,1]$ that is optimal for the constrained problem $V_{\lambda }\left(i\right)=\mathbb{E}\left[U\left(I^{i,{\alpha }_{\lambda }}\left(T\right)\right)\right]$ and such that

$$V_{\lambda }\left(i\right)=V\left(i\right)$$

Proof

$$\mathbb{E}\left[\tilde{U}\left({\mathcal{Y}}_{\lambda }\left(i\right)H^{\left(\lambda \right)}\left(T\right)\right)\right]\le \mathbb{E}\left[\tilde{U}\left({\mathcal{Y}}_{\lambda }\left(i\right)H^{(\lambda +\varepsilon (\nu -\lambda ))}\left(T\right)\right)\right]$$

Since ${\tilde{U}}^{\prime }\left(y\right)=-I_2\left(y\right)$,

$$0\le \underset{\varepsilon \downarrow 0}{lim}sup\frac{1}{\varepsilon }\mathbb{E}[\tilde{U}\left({\mathcal{Y}}_{\lambda }\left(i\right)H^{(\lambda +\varepsilon (\nu -\lambda ))}\left(T\right)\right)-\tilde{U}\left({\mathcal{Y}}_{\lambda }\left(i\right)H^{\left(\lambda \right)}\left(T\right)\right)]={\mathcal{Y}}_{\lambda }\left(i\right)\underset{\varepsilon \downarrow 0}{lim}sup\frac{1}{\varepsilon }\mathbb{E}\left[I_2\left({\mathcal{Y}}_{\lambda }\left(i\right)H^{\left(\lambda \right)}\left(T\right)\right)(H^{\left(\lambda \right)}\left(T\right)-H^{(\lambda +\varepsilon (\nu -\lambda ))}\left(T\right))\right]$$

By Theorem 9.1 there exists a control ${\alpha }_{\lambda }\in {\mathcal{A}}_{\lambda }^{\prime }\left(i\right)$ such that:

$$I^{i,{\alpha }_{\lambda }}\left(T\right)=I_2\left({\mathcal{Y}}_{\lambda }\left(i\right)H_{\lambda }\left(T\right)\right)$$

Clearly ${\alpha }_{\lambda }$ is optimal for the constrained problem, and

$$-\delta \left(\lambda \right)+{\alpha }_{\lambda }\left(t\right)\lambda \left(t\right)=0$$

Thus by proposition 8.3, ${\alpha }_{\lambda }$ is optimal for the constrained problem. □

CK Section 12. A dual problem.

Define:

$$\hat{V}\left(y\right)=\underset{\nu \in {\mathcal{D}}^{\prime }}{inf}\mathbb{E}\left[\tilde{U}\left(y{H}_{\nu }\left(T\right)\right)\right]$$

In our case,

$$\tilde{U}\left(y\right)=\underset{x>0}{max}\left[-\frac{x^{1-\gamma }}{1-\gamma }-xy\right]$$

Thus

$$y=U^{\prime }\left(x\right)=-x^{-\gamma }⟹I_2\left(y\right)={(-y)}^{-1/\gamma }$$

Let $\rho =(1-\gamma )/\gamma$. Then:

$$\tilde{U}\left(y\right)=-{(-y)}^{-\rho }/\rho$$

Typically, $\gamma =-1$, so that:

$$\tilde{U}\left(y\right)=y^2/2$$

The main problem in condition (32) is to find the optimal process $H^{\left(\lambda \right)}$ (across all $H^{\left(\nu \right)}$) but it depends on $y$ which depends on $\lambda$. Thus the dual must be fixed for a fixed but arbitrary real number $y$. The objective has the form

$$\mathbb{E}\left[\tilde{U}\left(y{H}^{\left(\nu \right)}\left(T\right)\right)\right]=\mathbb{E}[U\left(I_2\left(y{H}^{\left(\nu \right)}\left(T\right)\right)\right)-y{H}^{\left(\nu \right)}\left(T\right)I_2\left(y{H}^{\left(\nu \right)}\left(T\right)\right)]$$

The right hand-side of the equation (see [22, p.134]) is the maximum of the function $h(B,y)≔L(B,y)$ for all non-negative ${\mathcal{F}}_T$ measurable $B$ with $\mathbb{E}\left[H^{\left(\nu \right)}\left(T\right)B\right]\le i$. Thus a minimization over all positive numbers $y$ of $h(B,y)$ would yield the optimal utility of the unconstrained problem. We could thus first minimize $\mathbb{E}\left[\tilde{U}\left(y{H}^{\left(\nu \right)}\left(T\right)\right)\right]$ in $y$, and then minimize over $\nu$. However, the main idea is to first minimize over $\mu$, and then minimize over $y$, hoping that the two can be interchanged.

CK 12.1 Proposition

Suppose that for any $y$ there exists ${\lambda }_y$ such that $\hat{V}\left(y\right)=\mathbb{E}\left[\tilde{U}\left(y{H}^{{}_{\left({\lambda }_y\right)}}\left(T\right)\right)\right]$ . Then there exist an $\alpha \in {\mathcal{A}}^{\prime }\left(i\right)$ with $i={\mathcal{X}}_{{\lambda }_y\left(y\right)}$ which is optimal for the primal problem, and we have:

$$\hat{V}\left(y\right)=\underset{i}{sup}[V\left(i\right)-iy]$$

Proof

Write $\lambda$ for ${\lambda }_{{\mathcal{Y}}_{\lambda }\left(i\right)}$. Then

$$\mathbb{E}\left[\tilde{U}\left({\mathcal{Y}}_{\lambda }\left(i\right)H^{\left(\lambda \right)}\left(T\right)\right)\right]\le \mathbb{E}\left[\tilde{U}\left({\mathcal{Y}}_{\lambda }\left(i\right)H^{\left(\nu \right)}\left(T\right)\right)\right]$$

and we conclude by CK Theorem 10.1. □

CK Section 15. Deterministic coefficients and feedback formulae.

Define:

$$Q(y,t)=\mathbb{E}\left[\tilde{U}\left(y{H}^{\left(\nu \right)}\left(T\right)\right)\right|y{H}^{\left(\nu \right)}\left(t\right)=y]$$

Recall

$$\frac{d{H}^{\left(\nu \right)}}{H^{\left(\nu \right)}}=(-r+\delta \left(\nu \right))dt-(\theta +\nu /\sigma )d{B}_2$$

The HJB equation is:

$$\underset{\nu }{min}\frac{1}{2}{y}^2{(\theta +\nu /\sigma )}^2{Q}_{yy}+y(-r+\delta \left(\nu \right))Q_y+Q_t=0$$

$$Q(T,y)=\tilde{U}\left(y\right)=-\frac{{(-y)}^{-\rho }}{\rho }$$

Again, with $\rho =(1-\gamma )/\gamma <0$. We choose

$$Q(y,t)=-\frac{1}{\rho }{(-y)}^{-\rho }v\left(t\right)$$

Thus

$$\frac{1}{2}{y}^2{(\theta +\nu /\sigma )}^2{Q}_{yy}+y(-r+\delta \left(\nu \right))Q_y=-\frac{1}{2}(\rho +1){(-y)}^{-\rho }v\left(t\right){(\theta +\nu /\sigma )}^2+(-r+\delta \left(\nu \right)){(-y)}^{-\rho }v\left(t\right)$$

Dividing by ${(-y)}^{-\rho }v\left(t\right)$, the problem becomes:

$$\underset{\nu }{argmin}-\frac{1+\rho }{2}{(\theta +\nu /\sigma )}^2+\delta \left(\nu \right)$$ (33)

Recall that if $\nu$ is positive, then $\delta \left(\nu \right)=\nu$ thus we solve (33) and obtain

$$\nu =\frac{{\sigma }^2}{1+\rho }+r-b=-{\sigma }^2\left|\gamma \right|+r-b$$

since $1+\rho =1/\gamma$ and $\gamma$ is negative. If $\nu$ is negative, then $\delta \left(\nu \right)=0$, thus $\nu =r-b$.

From (29), the solution is

$$\alpha \left(s\right)=\frac{r-b-\nu }{{\sigma }^2\left|\gamma \right|}=min\left(1,max\left(0,\frac{r-b}{{\sigma }^2\left|\gamma \right|}\right)\right)$$

Suppose ${\mu }_0={\mu }_1$ and treatment is better than no treatment ${\overline{k}}_1>K_0$. Thus $r-b={\overline{k}}_1-K_0$ is positive. Thus

$$\alpha \left(s\right)=min\left(1,max\left(0,\frac{{\overline{k}}_1-K_0}{{\sigma }^2\left|\gamma \right|}\right)\right)$$

Appendix C. Explicit formula of $A_3(\tau ,\gamma )$ in (14)

$$A_3(\tau ,\gamma )={\int }_0^{\tau }\left(\frac{{\sigma }_x^2}{2\gamma }+{\lambda }_x\overline{X}\right)A_2^2(s,\gamma )+\frac{{\sigma }_x^2}{2}{A}_1(s,\gamma )+(\gamma -1)\mu ds=\left(\frac{{\sigma }_x^2}{2\gamma }+{\lambda }_x\overline{X}\right)\left(\frac{2{\lambda }_x\overline{X}{b}_2\left(\gamma \right)A_2(\tau ,\gamma )}{{\theta }^3\left(\gamma \right)b_3\left(\gamma \right)}+\frac{2{\overline{X}}^2{\lambda }_x^2}{{\theta }^3\left(\gamma \right)}\left(-\frac{A_1(\tau ,\gamma )}{b_3\left(\gamma \right)}+\frac{8b_1^2\left(\gamma \right)\tau log\left(\frac{2\theta \left(\gamma \right)-\left(\theta \left(\gamma \right)+b_2\left(\gamma \right)\right)\left(1-e^{-\theta \left(\gamma \right)\tau }\right)}{2\theta \left(\gamma \right)}\right)}{(b_2\left(\gamma \right)-\theta \left(\gamma \right))\sqrt{b_1\left(\gamma \right)b_3\left(\gamma \right)}}+\frac{b_2\left(\gamma \right)(\theta \left(\gamma \right)-2\sqrt{b_1\left(\gamma \right)b_3\left(\gamma \right)})}{\theta \left(\gamma \right)b_3^2\left(\gamma \right)}\times log\left(\frac{b_2\left(\gamma \right)-2\sqrt{b_1\left(\gamma \right)b_3\left(\gamma \right)}}{\theta \left(\gamma \right)}\left(\frac{2b_2\left(\gamma \right)+4\sqrt{b_1\left(\gamma \right)b_3\left(\gamma \right)}{e}^{-\theta \left(\gamma \right)\tau /2}}{2\theta \left(\gamma \right)-\left(b_2\left(\gamma \right)+\theta \left(\gamma \right)\right)\left(1-e^{-\theta \left(\gamma \right)\tau }\right)}\right)\right)\left(\left(-\frac{(b_2\left(\gamma \right)+\theta \left(\gamma \right))A_1(\tau ,\gamma )}{2b_1\left(\gamma \right)}\right)\right)+\frac{4b_1^2\left(\gamma \right)\tau }{{(b_2\left(\gamma \right)+\theta \left(\gamma \right))}^2}\right)\right)+\frac{{\sigma }_x^2}{2}\left(\frac{1}{b_3\left(\gamma \right)}log\left(\frac{2\theta \left(\gamma \right)e^{-\theta \left(\gamma \right)\tau }}{2\theta \left(\gamma \right)-\left(b_2\left(\gamma \right)+\theta \left(\gamma \right)\right)\left(1-e^{-\theta \left(\gamma \right)\tau }\right)}\right)-\frac{2b_1\left(\gamma \right)\tau }{b_2\left(\gamma \right)+\theta \left(\gamma \right)}\right)+(\gamma -1)\mu \tau$$

Appendix D. Proof of Theorem 4

We follow the proof of Proposition 2 in [1]. Recall the equations (58) (59) in [1] and following same notations:

$$\left(\frac{\partial }{\partial t}+L_1\right)f_1=0$$ (34)

$$\left(\frac{\partial }{\partial t}+L_1\right)f_2=-L_2{f}_1$$ (35)

where $L_1,L_2$ are equations (53) (54) in [1].

Solution of (34).

We postulate that:

$$f_1(Z,X,t)=Z^{1/\gamma }{H}_1(X,T-t)$$

Substitution in (34) shows that $H_1$ solves:

$$\left(\frac{\partial }{\partial t}+L^{\gamma }\right)H_1=0$$ (36)

$$H_1(X,0)=1$$

where the operator $L^{\gamma }$ is defined by:

$$L^{\gamma }H\equiv \frac{1}{2}{\sigma }_x^2\frac{{\partial }^{2}H}{\partial X^2}+\left(\left(\frac{\gamma -1}{\gamma }{\sigma }_x-{\lambda }_x\right)X+{\lambda }_x\overline{X}\right)\frac{\partial H}{\partial X}+\left(X^2\left(\frac{1}{2}\frac{1}{\gamma }\left(\frac{1}{\gamma }-1\right)\right)+\mu \left(1-\frac{1}{\gamma }\right)\right)H$$

Using the Ansatz (11), we can rewrite the LHS of (36) into:

$$(C_1\left(t\right)X^2+C_2\left(t\right)X+C_3\left(t\right))H_1/\gamma =0$$

Clearly all terms $C_1,C_2,C_3$ must be identically zero. Thus:

$$\frac{d{A}_1(t,\gamma )}{dt}=\frac{{\sigma }_x^2}{\gamma }{A}_1^2(t,\gamma )+2\left(\frac{\gamma -1}{\gamma }{\sigma }_x-{\lambda }_x\right)A_1(t,\gamma )+\frac{1-\gamma }{\gamma }$$

$$\frac{d{A}_2(t,\gamma )}{dt}=\frac{{\sigma }_x^2{A}_1(t,\gamma )}{\gamma }{A}_2(t,\gamma )+\left(\frac{\gamma -1}{\gamma }{\sigma }_x-{\lambda }_x\right)A_2(t,\gamma )+{\lambda }_x\overline{X}{A}_1(t,\gamma )$$

$$\frac{d{A}_3(t,\gamma )}{dt}=\frac{{\sigma }_x^2}{2}\left(A_1(t,\gamma )+\frac{A_2^2(t,\gamma )}{\gamma }\right)+{\lambda }_x\overline{X}{A}_2(t,\gamma )-\mu (1-\gamma )$$

which admit the solutions (12), (13), (14).

Solution of (35).

The second equation can be rewritten

$$\left(\frac{\partial }{\partial t}+L_1\right)f_2=\frac{1}{2}\frac{{\beta }^2}{\gamma {\sigma }_S^2}{Z}^{2/\gamma }S{H}_1{(X,T-t)}^2$$ (37)

We try the Ansatz:

$$f_2(Z\left(t\right),X\left(t\right),t)=Z{\left(t\right)}^{2/\gamma }S\left(t\right)g(X\left(t\right),t)$$ (38)

Thus

$$\left(\frac{\partial }{\partial t}+L^{\gamma /2}\right)g(X,t)=\frac{1}{2}\frac{{\beta }^2}{{\sigma }_S^2\gamma }{H}_1{(X,T-t)}^2$$

$$g(X,T)=0$$

We use Lemma 6 to obtain the $g(X,t)$ in (18).

The optimal policy is:

$${\alpha }^{\ast }=\frac{1}{\sigma F}\left(\frac{\partial F}{\partial Z}XZ-\frac{\partial F}{\partial X}{\sigma }_x\right)={\alpha }_0+\varepsilon {\alpha }_1+\mathcal{O}\left({\varepsilon }^2\right)$$

where

$${\alpha }_0=\frac{\partial f_1}{\partial Z}\frac{XZ}{\sigma f_1}-\frac{\partial f_1}{\partial X}\frac{{\sigma }_x}{\sigma f_1}=\frac{X\left(t\right)}{\gamma \sigma }-\frac{{\sigma }_x}{\gamma \sigma }\left(A_1(T-t,\gamma )X\left(t\right)+A_2(T-t,\gamma )\right)$$

$${\alpha }_1=\frac{XZ}{\sigma f_1}\left(\frac{\partial f_2}{\partial Z}-\frac{\partial f_1}{\partial Z}\frac{f_2}{f_1}\right)-\frac{{\sigma }_x}{\sigma f_1}\left(\frac{\partial f_2}{\partial X}-\frac{\partial f_1}{\partial X}\frac{f_2}{f_1}\right)=\frac{Z^{1/\gamma }\left(t\right)S\left(t\right)}{H_1(X,T-t)\sigma }\left(\frac{g(X\left(t\right),t)X\left(t\right)}{\gamma }-{\sigma }_x\frac{\partial g}{\partial X}\right)\left(+{\sigma }_x\frac{g(X\left(t\right),t)}{\gamma }\left(A_1(T-t,\gamma )X\left(t\right)+A_2(T-t,\gamma )\right)\right)$$

Lemma 6

Let $u(x,t)=\frac{1}{2}\frac{{\beta }^2}{{\sigma }_S^2\gamma }{H}_1{(x,T-t)}^2$ . The solution to

$$\frac{\partial g(x,t)}{\partial t}+L^{\gamma /2}g(x,t)=u(x,t)$$ (39)

$$g(x,T)=0$$

is in (18) .

Sketch of Proof

Define $m\left(x\right)$ and $r\left(x\right)$ to be such that:

$$L^{\gamma /2}f(x,t)=\frac{1}{2}{\sigma }_x^2\frac{{\partial }^{2}f(x,t)}{\partial x^2}+m\left(x\right)\frac{\partial f(x,t)}{\partial x}-r\left(x\right)f(x,t)$$

$$m\left(x\right)=\left(\frac{\gamma /2-1}{\gamma /2}{\sigma }_x-{\lambda }_x\right)x+{\lambda }_x\overline{X}$$

$$r\left(x\right)=-\left(x^2\frac{1}{\gamma }\left(\frac{2}{\gamma }-1\right)+\mu \left(1-\frac{2}{\gamma }\right)\right)$$

Let $f(x,t)$ be the solution of:

$$\frac{\partial f(x,t)}{\partial t}+\frac{1}{2}{\sigma }_x^2\frac{{\partial }^{2}f(x,t)}{\partial x^2}+m\left(x\right)\frac{\partial f(x,t)}{\partial x}=r\left(x\right)f(x,t)$$ (40)

Defining:

$$dX\left(t\right)=m\left(X\right)dt+{\sigma }_{x}dW\left(t\right)$$ (41)

we see that:

$$\frac{\partial f(x,t)}{\partial t}+\frac{1}{2}{\sigma }_x^2\frac{{\partial }^{2}f(x,t)}{\partial x^2}+m\left(x\right)\frac{\partial f(x,t)}{\partial x}=\mathbb{E}\left[df(X,t)\right|X\left(t\right)=x]/dt$$ (42)

Thus (40) can be rewritten:

$$\mathbb{E}[df(X\left(t\right),t)-r\left(X\left(t\right)\right)f(X\left(t\right),t)dt|X\left(t\right)]=0$$

Using the integrating factor $exp(-{\int }_0^{t}r\left(X\left(s\right)\right)ds)$, we have:

$$\mathbb{E}\left[d(exp(-{\int }_0^{t}r\left(X\left(s\right)\right)ds)f(X\left(t\right),t))\right|X\left(t\right)]=0$$

Under the boundary condition $f(X\left(T\right),T)=1$ the only possible solution is:

$$f(x,t;T)=\mathbb{E}[exp(-{\int }_t^{T}r\left(X\left(s\right)\right)ds)|X\left(t\right)=x]$$

Define $P(t,T)=f(X\left(t\right),t;T)=H_2(X\left(t\right),T-t)$. Clearly:

$$\frac{dP(t,T)}{P(t,T)}=r\left(X\left(t\right)\right)dt+v(t,T)dW\left(t\right)$$

where:

$$v(t,T)={\sigma }_x\frac{\frac{\partial f}{\partial x}}{f}$$

By Ito’s lemma, and for the exact same reason as (42):

$$\frac{\partial g(x,t)}{\partial t}+\frac{1}{2}{\sigma }_x^2\frac{{\partial }^{2}g(x,t)}{\partial x^2}+m\left(x\right)\frac{\partial g(x,t)}{\partial x}=\mathbb{E}\left[dg(X,t)\right|X\left(t\right)=x]/dt$$

The stochastic equivalent of (39) is:

$$\mathbb{E}[dg(X\left(t\right),t)-r\left(X\left(t\right)\right)g(x,t)dt|X\left(t\right)]=\mathbb{E}\left[u(X\left(t\right),t)dt\right|X\left(t\right)]$$

The solution is:

$$g(X\left(t\right),t)={\int }_{\tau =t}^{T}Q(t,\tau )d\tau$$

where:

$$Q(t,\tau )=\mathbb{E}[exp(-{\int }_t^{\tau }r\left(X\left(s\right)\right)ds)u(X\left(\tau \right),\tau )|X\left(t\right)]$$

Clearly, for some volatility ${\sigma }_Q(t,\tau )$

$$\frac{dQ(t,\tau )}{Q(t,\tau )}=r\left(X\left(t\right)\right)dt+{\sigma }_Q(t,\tau )dW\left(t\right)$$

We are now ready to define a change of numeraire. Let

$$d{W}^{\tau }=dW-v(t,\tau )dt$$

By Theorem 9.2.2 in [23], $Q(t,\tau )/P(t,\tau )$ is a ${\mathbb{P}}^{\tau }$-martingale, i.e., 

$$Q(t,\tau )=P(t,\tau ){\mathbb{E}}_t^{\tau }\left[u\left(X\left(\tau \right)\right)\right]$$

where

$$dX\left(t\right)=m\left(X\right)dt+{\sigma }_{x}dW\left(t\right)=m\left(X\right)dt+{\sigma }_x(d{W}^{\tau }\left(t\right)+v(t,\tau )dt)$$

From (11),

$$u(X\left(t\right),t)=\frac{1}{2}\frac{{\beta }^2}{{\sigma }_{S^2}\gamma }{e}^{\frac{2}{{}^{\gamma }}\left(\frac{A_1(T-t,\gamma )}{2}X{\left(t\right)}^2+A_2(T-t,\gamma )X\left(t\right)+A_3(T-t,\gamma )\right)}$$

Let us now take:

$$P(t,T)=exp\left(\frac{2}{{}^{\gamma }}\left(\frac{A_1(T-t,\gamma /2)}{2}{X}^2\left(t\right)+A_2(T-t,\gamma /2)X\left(t\right)+A_3(T-t,\gamma /2)\right)\right)$$

Thus:

$$v(t,\tau )=\frac{{\sigma }_x}{{}^{\gamma }}(A_1(\tau -t,\gamma /2)X\left(t\right)+A_2(\tau -t,\gamma /2))$$

$$dX\left(t\right)=[\left(\frac{\frac{\gamma }{2}-1}{\gamma /2}{\sigma }_x-{\lambda }_x+\frac{{\sigma }_x^2}{{}^{\gamma }}{A}_1(\tau -t,\gamma /2)\right)X\left(t\right)+{\lambda }_x\overline{X}+\frac{{\sigma }_x^2}{{}^{\gamma }}{A}_2(\tau -t,\gamma /2)]dt+{\sigma }_{x}d{W}^{\tau }\left(t\right)$$ (43)

Thus ${\mathbb{E}}_t^{\tau }\left[u\left(X\left(\tau \right)\right)\right]$ when (43) holds can be calculated exactly the same way as $\mathbb{E}\left[u\left(X\left(\tau \right)\right)\right]$ when (41) holds. The structure is also affine, and there will be a solution of the form:

$${\mathbb{E}}_t^{\tau }\left[u(X\left(\tau \right),\tau )\right]=\frac{1}{2}\frac{{\beta }^2}{{\sigma }_{S^2}\gamma }{\mathbb{E}}_t^{\tau }\left[e^{\frac{2}{\gamma }\left(\frac{A_1(T-\tau ,\gamma )}{2}{X}^2\left(\tau \right)+A_2(T-\tau ,\gamma )X\left(\tau \right)+A_3(T-\tau ,\gamma )\right)}\right]$$

To summarize, since $P(t,T)=H_2(X\left(t\right),T-t)$

$$g(x,t)={\int }_{\tau =t}^{T}P(t,\tau ){\mathbb{E}}_t^{\tau }\left[u(X\left(\tau \right),\tau )\right]d\tau ={\int }_{\tau =t}^T{H}_2(X,\tau -t)\frac{1}{2}\frac{{\beta }^2}{{\sigma }_{S^2}\gamma }$$$${\mathbb{E}}_t^{\tau }\left[e^{\frac{2}{\gamma }\left(\frac{A_1(T-\tau ,\gamma )}{2}{X}^2\left(\tau \right)+A_2(T-\tau ,\gamma )X\left(\tau \right)+A_3(T-\tau ,\gamma )\right)}\right]d\tau$$ (44)

Let $\tilde{M}(t,\tau )$ as in (17) and

$$Y\left(\tau \right)=X\left(\tau \right)+\frac{A_2(T-\tau ,\gamma )}{A_1(T-\tau ,\gamma )}$$

Clearly:

$${\mathbb{E}}^{\tau }\left[X\left(\tau \right)\right|X\left(t\right)=x]=x\tilde{M}(t,\tau )+{\int }_{s=t}^{\tau }\tilde{M}(s,\tau )({\lambda }_x\overline{X}+\frac{{\sigma }_x^2}{{}^{\gamma }}{A}_2(\tau -s,\gamma ))ds$$

$$\mathbb{V}a{r}^{\tau }\left[X\left(\tau \right)\right|X\left(t\right)=x]={\sigma }_x^2{\int }_t^{\tau }{\tilde{M}}^2(s,\tau )ds$$

Thus we can calculate:

$$m_Y(\tau ,x)={\mathbb{E}}^{\tau }\left[Y\left(\tau \right)\right|X\left(t\right)=x]={\mathbb{E}}^{\tau }\left[X\left(\tau \right)\right|X\left(t\right)=x]+\frac{A_2(T-\tau ,\gamma )}{A_1(T-\tau ,\gamma )}$$

$$V_Y(\tau ,x)=\mathbb{V}a{r}^{\tau }\left[Y\left(\tau \right)\right|X\left(t\right)=x]={\sigma }_x^2{\int }_t^{\tau }{\tilde{M}}^2(s,\tau )ds$$

We can further develop:

$${\mathbb{E}}_t^{\tau }\left[exp\left(\frac{2}{\gamma }\left(\frac{A_1(T-\tau ,\gamma )}{2}{X}^2\left(\tau \right)+A_2(T-\tau ,\gamma )X\left(\tau \right)+A_3(T-\tau ,\gamma )\right)\right)\right]={\mathbb{E}}_t^{\tau }\left[e^{\frac{2}{\gamma }{A}_3(T-\tau ,\gamma )+\frac{1}{\gamma }{A}_1(T-\tau ,\gamma ){\left(X\left(\tau \right)+\frac{A_2(T-\tau ,\gamma )}{A_1(T-\tau ,\gamma )}\right)}^2-\frac{A_2^2(T-\tau ,\gamma )}{\gamma A_1(T-\tau ,\gamma )}}\right]={\mathbb{E}}_t^{\tau }\left[e^{\frac{2}{\gamma }{A}_3(T-\tau ,\gamma )-\frac{A_2^2(T-\tau ,\gamma )}{\gamma A_1(T-\tau ,\gamma )}}{e}^{\frac{A_1(T-\tau ,\gamma )}{\gamma }{\left(X\left(\tau \right)+\frac{A_2(T-\tau ,\gamma )}{A_1(T-\tau ,\gamma )}\right)}^2}\right]=e^{\frac{2}{\gamma }{A}_3(T-\tau ,\gamma )-\frac{A_2^2(T-\tau ,\gamma )}{\gamma A_1(T-\tau ,\gamma )}}\frac{1}{\sqrt{2\pi V_Y(\tau ,x)}}\int e^{\frac{A_1(T-\tau ,\gamma )}{\gamma }{y}^2}{e}^{-\frac{{(y-m_Y(\tau ,x))}^2}{2V_Y(\tau ,x)}}dy=e^{\frac{2}{\gamma }{A}_3(T-\tau ,\gamma )-\frac{A_2^2(T-\tau ,\gamma )}{\gamma A_1(T-\tau ,\gamma )}+\frac{m_Y^2(\tau ,x)A_1(T-\tau ,\gamma )}{\gamma -2V_Y(\tau ,x)A_1(T-\tau ,\gamma )}}\frac{1}{\sqrt{1-2V_Y(\tau ,x)A_1(T-\tau ,\gamma )/\gamma }}$$

providing $\gamma <2A_1(T-\tau ,\gamma )V_Y(\tau ,x)$. Thus Eq. (18) follows from (44). □

Appendix E. Proof of Theorem 5

When $X\left(t\right)$ is a constant, equations (53) and (54) in [1] become

$$L_{1}F=\frac{1}{2}{Z}^2{\left(\frac{b-r}{\sigma }\right)}^2\frac{{\partial }^{2}F}{\partial Z^2}-\mu Z\frac{\partial F}{\partial Z}+\mu F$$

$$L_{2}F=-\frac{1}{2}\frac{{\beta }^2}{{\sigma }_S^2}ZSF\frac{\partial F}{\partial Z}$$

Use the Ansatz $f_1(Z\left(t\right),t)=Z^{1/\gamma }\left(t\right)h_1(T-t)$ and insert in (20) shows that $h_1$ solves:

$$\left(\frac{\partial }{\partial t}+L^{\gamma }\right)h_1=0h_1\left(0\right)=1$$ (45)

where the operator $L^{\gamma }$ is defined by:

$$L^{\gamma }H\equiv \left({\left(\frac{b-r}{\sigma }\right)}^2\left(\frac{1}{2}\frac{1}{\gamma }\left(\frac{1}{\gamma }-1\right)\right)+\mu \left(1-\frac{1}{\gamma }\right)\right)H$$

Using the Ansatz (23), we can rewrite (45) into:

$$\left(C_1\left(t\right){\left(\frac{b-r}{\sigma }\right)}^2+C_2\left(t\right)\right)h_1/\gamma =0$$

Clearly all terms $C_1,C_2$ must be identically zero. Thus

$$\frac{d{a}_{1,1}t}{dt}=\frac{1-\gamma }{\gamma }\frac{d{a}_{1,2}t}{dt}=\mu (\gamma -1)$$

which admit the solutions (24), (25) at $i=1$.

Now use $f_i(Z\left(t\right),t)=Z^{2^{i-1}/\gamma }\left(t\right)S^{2^{i-1}-1}\left(t\right)g_i\left(t\right)$. We can rewrite (21) by

$$\left(\frac{\partial }{\partial t}+L^{\gamma /2^i}\right)g_{i+1}\left(t\right)=\frac{1}{2}\frac{{\beta }^2{2}^{i-1}}{{\sigma }_S^2\gamma }{g}_i^2\left(t\right)g_{i+1}\left(T\right)=0$$ (46)

Let $u\left(t\right)=\frac{1}{2}\frac{{\beta }^2{2}^{i-1}}{{\sigma }_S^2\gamma }{g}_i^2\left(t\right)$ and

$$r_i=\frac{2^i}{\gamma }\left(\frac{1}{2}{\left(\frac{b-r}{\sigma }\right)}^2{a}_{i+1,1}+a_{i+1,2}\right)$$

Then

$$L^{\gamma /2^i}{g}_{i+1}\left(t\right)=r_i{g}_{i+1}\left(t\right)$$

and the stochastic equivalent of (46) is:

$$\frac{\partial g_{i+1}\left(t\right)}{\partial t}+r_i{g}_{i+1}\left(t\right)=u\left(t\right)g_{i+1}\left(T\right)=0$$

which admits

$$g_{i+1}\left(t\right)=-\frac{1}{2}\frac{{\beta }^2{2}^{i-1}}{{\sigma }_S^2\gamma }\frac{1}{h_{i+1}\left(t\right)}{\int }_t^T{g}_i^2\left(s\right)h_{i+1}\left(s\right)ds$$ (47)

We have showed that $g_1\left(t\right)=h_1(T-t)$. Here we also provide the $g_2$ and $g_3$ in the following:

$$g_2\left(t\right)=\frac{{\beta }^2}{2{\sigma }_S^2}\frac{h_2(T-t)-h_1^2(T-t)}{{\left(\frac{b-r}{\sigma }\right)}^2(a_{1,1}-a_{2,1})+2(a_{1,2}-a_{2,2})}$$

$$g_3\left(t\right)=-\frac{{\beta }^6}{16{\sigma }_S^6}{\left(\frac{1}{{\left(\frac{b-r}{\sigma }\right)}^2(a_{1,1}-a_{2,1})+2(a_{1,2}-a_{2,2})}\right)}^2\frac{1}{h_3\left(t\right)}\left(h_2^2\left(T\right)\frac{\frac{h_3\left(T\right)}{h_2^2\left(T\right)}-\frac{h_3\left(t\right)}{h_2^2\left(t\right)}}{\frac{a_{3,1}-a_{2,1}}{2}{\left(\frac{b-r}{\sigma }\right)}^2+a_{3,2}-a_{2,2}}\right)\left(+h_1^4\left(T\right)\frac{\frac{h_3\left(T\right)}{h_1^4\left(T\right)}-\frac{h_3\left(t\right)}{h_1^4\left(t\right)}}{\frac{a_{3,1}-a_{1,1}}{2}{\left(\frac{b-r}{\sigma }\right)}^2+a_{3,2}-a_{1,2}}\right)\left(-2\frac{h_2\left(T\right)}{h_1^2\left(T\right)}\frac{\frac{h_3\left(T\right)h_1^2\left(T\right)}{h_2\left(T\right)}-\frac{h_3\left(t\right)h_1^2\left(t\right)}{h_2\left(t\right)}}{\frac{a_{3,1}-\frac{a_{2,1}-a_{1,1}}{2}}{2}{\left(\frac{b-r}{\sigma }\right)}^2+a_{3,2}-\frac{a_{2,2}-a_{1,2}}{2}}\right)$$

Suppose we use the first two expansions, the optimal policy is given by:

$${\alpha }^{\ast }={\alpha }_0+\varepsilon {\alpha }_1+\mathcal{O}\left({\varepsilon }^2\right)$$

where

$${\alpha }_0=\frac{\partial f_1}{\partial Z}\frac{\frac{b-r}{\sigma }Z}{\sigma f_1}=\frac{\frac{b-r}{\sigma }}{\gamma \sigma }$$

$${\alpha }_1=\frac{\frac{b-r}{\sigma }Z}{\sigma f_1}\left(\frac{\partial f_2}{\partial Z}-\frac{\partial f_1}{\partial Z}\frac{f_2}{f_1}\right)=\frac{Z^{1/\gamma }\left(t\right)S\left(t\right)}{h_1(T-t)\sigma }\frac{g_2\left(t\right)\frac{b-r}{\sigma }}{\gamma }$$

Appendix F. Supplementary data

The following is the Supplementary material related to this article.

MMC S1

Online supplement for Optimal Control of the SIR model with constrained policy, with an application to COVID-19.