Optimal control of the SIR model in the presence of transmission and treatment uncertainty

Abstract

The COVID-19 pandemic illustrates the importance of treatment-related decision making in populations. This article considers the case where the transmission rate of the disease as well as the efficiency of treatments is subject to uncertainty. We consider two different regimes, or submodels, of the stochastic SIR model, where the population consists of three groups: susceptible, infected and recovered and dead. In the first regime the proportion of infected is very low, and the proportion of susceptible is very close to 100the proportion of infected is moderate, but not negligible. We show that the first regime corresponds almost exactly to a well-known problem in finance, the problem of portfolio and consumption decisions under mean-reverting returns (Wachter, JFQA 2002), for which the optimal control has an analytical solution. We develop a perturbative solution for the second problem. To our knowledge, this paper represents one of the first attempts to develop analytical/perturbative solutions, as opposed to numerical solutions to stochastic SIR models.

1. Introduction

In December 2019, a novel coronavirus later named severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) emerged in Wuhan, China, and during January and mid-March 2020 spread rapidly from its epicenter to other Chinese cities and to over 150 countries across all continents [1], [2]. On March 11 2020, the World Health Organization (WHO) declared COVID-19, the disease caused by SARS-CoV2, a pandemic [3]; six months after its emergence, the number of confirmed cases of COVID-19 globally exceeded 10 million, with over 500,000 deaths [1]. The pandemic has strained public health and medical systems internationally, caused global economic activity to stagnate, and disrupted normal patterns of life across societies [4], [5].

Epidemiologically, the rapid and explosive proliferation of SARS-CoV2 infection following its introduction to human populations is due the lack of pre-existing immunity to the new virus [6], [7]. SARS-CoV2 transmission primarily occurs through person-to-person contact when a person with COVID-19 coughs, sneezes or talks producing respiratory droplets containing the virus which reach the nose or mouth of another person in close proximity allowing for their inhalation [8]. Persons infected with SARS-CoV2 experience a wide range of clinical manifestations of illness, from asymptomatic to severe disease [9], [10]. While treatment guidelines recommend that patients with mild to moderate disease self-manage and monitor their illness at home and/or receive appropriate care to relieve symptoms, a proportion of patients with severe COVID-19 will seek medical attention and require hospitalization [11]. In the United States, in-patient care for COVID-19 currently involves supportive management of common complications of severe disease, as no specific FDA-approved drug is available to treat COVID-19 to date. A number of therapeutic options of unknown safety and efficacy are currently under investigation for COVID-19 and are being administered to hospitalized patients. As is true for the management of other diseases, the decision to treat COVID-19 involves the patient, their family and their health care provider, and weighs the potential benefits and risks of available treatment options. Until one or more vaccine is developed for COVID-19, drugs that shorten the infectious period and reduce transmission of SARS-CoV2 can contribute to controlling the epidemic within the population in addition to reducing morbidity and mortality in severely ill patients [12], [13], [14], [15].

Mathematical models have long been used in infectious disease epidemiology for understanding the dynamics of epidemics in populations and predicting outcomes of effective control strategies [16], [17], [18], [19]. In the classic SIR model, one of the most commonly implemented and the basis for other models, persons within a population move between three compartments, “Susceptible”, “Infected”, and “Recovered” as a pathogen spreads from person to person [20], [21]. Stochastic modeling approaches are important when there is uncertainty in model parameters as would be associated with variability in population demographics that can impact epidemic outcomes [22]. Here we use a simplified version of the stochastic SIRV (“V”: “Vaccinated”) model developed by Ishikawa [23] to examine the effect of implementing treatments of uncertain efficacy to control the COVID-19 epidemic. Whereas authors like Ishikawa implement a numerical method to determine the optimal control, our goal in simplifying his model was to obtain tractable solutions, either analytical or perturbative. There are two main methods in stochastic control: the maximum principle, and stochastic dynamic programming (DP). In epidemiology, the deterministic version of Pontryagin’s maximum principle was used for instance by Bolzoni et al. [24]. Ishikawa [23] used stochastic DP. Among others, Cox and Huang [25] pioneered the martingale approach as an alternative to stochastic DP. Roughly speaking, in stochastic DP one attempts to first determine the optimal control and then the optimal state dynamics. In the martingale approach, the order is reversed. We found it easier to use the martingale approach. In the low infection case, we show that our model is very similar to the financial model considered by Wachter [26], who uses the martingale approach. The same problem was generalized by Liu [27], who uses stochastic DP.

The main novelty of this article is to determine the optimal control in presence of uncertainty on the treatment recovery rate. We incorporate two forms of uncertainty in our model: (i) uncertainty on the contemporaneous value of the treatment recovery rate (which we will succinctly call treatment measurement error) and (ii) uncertainty on the future value of the treatment recovery rate.

We will consider two regimes of our SIR model. In the first regime the proportion of infected is very low, and the proportion of susceptible is very close to 100%.  This corresponds to a disease with few cases and deaths, and where recovered individuals do not acquire immunity. In a second regime, the proportion of infected is moderate, but not negligible. The main new mathematical result of this article is to develop a perturbative solution for the second regime. Remarkably, both regimes (in a first approximation) have the same optimal control policy, which is independent of both the proportion of infected and the proportion of susceptible. On a second approximation, the optimal policy in the second regime is influenced by the latter variables.

Another contribution of this paper is to import from finance to the epidemiologic literature two different measures which combine the expected recovery rate of treatment as well as its dispersion. The first one, the Sharpe ratio, is appropriate when only a single treatment is available. The second one, the beta of the treatment, extends this concept to multiple treatments. Indeed, some treatments taken together can have synergistic effects either in their mean and their dispersion of the combined recovery rate (positive correlation of the recovery rates), or both, or can have antagonistic effects in their dispersion of the combined recovery rate (negative correlation).

The structure of the article is as follows: Section 2 introduces the stochastic SIR model with treatment uncertainty. In Section 3 we present our results, both theoretical and numerical for the regime of low proportion of infected. In Section 4 we present our results for the regime of moderate proportion of infected. We briefly allude to the general case in our conclusion. We present in Section 5 an application to COVID-19. The proof of our main result, Proposition 2, is in Appendix.

2. A stochastic SIR model with treatment uncertainty

Notation

Let $S∕I∕R$ be the proportion of susceptible, infected, recovered. Let $\beta$ be the transmission rate and $\mu$ be the death rate.

In the SIR model, the rate of decrease $\frac{dS}{dt}$ of the proportion of susceptible is equal to the constant transmission rate $\beta$ time $SI$. In a stochastic model this remains true on average, that is,

$$E\left[\frac{dS\left(t\right)}{dt}\right]=-E\left[\beta S\left(t\right)I\left(t\right)\right]$$

We complete this model by adding a noise term $\sigma \sqrt{SI}\frac{d{B}_1}{dt}$, where $\frac{d{B}_1}{dt}$ is white noise. This is a simplified version of the model in Ishikawa [23]:

$$\frac{dS}{dt}=-\beta SI+{\sigma }_S\sqrt{SI}\frac{d{B}_1}{dt}$$ (1)

Our noise term is such that, as required, $S\left(t\right)$ remains in the interval $[0,1]$. Indeed, when $S\left(t\right)=0$, the rate $\frac{dS}{dt}$ is clearly zero, while when $S\left(t\right)=1$, we have $I\left(t\right)=0$, thus $\frac{dS}{dt}$ is also equal to zero.

For simplicity, we label the “no treatment case” by the subscript $i=0$, and the “treatment case” by the subscript $i=1$. We call $K_i$ ($i\in \{0,1\}$ ) the recovery rate of treatment $i$ and ${\mu }_i>0$ ($i\in \{0,1\}$) the death rate. In Section 3.2 we will generalize this model to the multiple treatment case, so that treatments recovery rates will be labeled $K_i$ for $i\in \{0,\dots ,n\}$.

The optimal policy $\alpha$ is referred to as the optimal allocation of the treatment. The product $\alpha \left(t\right)I\left(t\right)K_1\left(t\right)dt$ corresponds to the proportion of the population that recovers due to the treatment in period $[t,t+dt]$. The allocation $\alpha$ can have two different interpretations. In the first one $\alpha \left(t\right)$ represents the proportion of infected that undergo treatment, and thus $\alpha \left(t\right)\in [0,1]$. In the second interpretation, we assume (like in the AIDS epidemic) that treatment is very expensive, and that recovery depends linearly (in a first approximation) on how much one spends on the treatment. In this case $K_1$ corresponds then to the recovery rate of the basic dose of the treatment, while $\alpha \left(t\right)$ corresponds to how many doses the population purchases. The situation $\alpha \left(t\right)<0$ corresponds to the case where treatment is discovered to become harmful and necessitates an alternative treatment. For simplicity, we will describe our model as a function of the first interpretation thereafter but relax the constraint $\alpha \left(t\right)\in [0,1]$.1

Depending whether the individual is treated or not, there are then four different ways for an infected individual to exit the pool of infected:

• •not treated and recover
• •not treated and die
• •treated and recover
• •treated and die

Thus, the “out of infection rate” will be:

$$\frac{dR\left(t\right)}{dt}=\underset{\text{not treated and recover}}{\underbrace{(1-\alpha \left(t\right))I\left(t\right)K_0}}+\underset{\text{not treated and die}}{\underbrace{(1-\alpha \left(t\right))I\left(t\right){\mu }_0}}+\underset{\text{treated and recover}}{\underbrace{\alpha \left(t\right)I\left(t\right)K_1\left(t\right)}}+$$ (2)

$$\underset{\text{treated and die}}{\underbrace{\alpha \left(t\right)I\left(t\right){\mu }_1}}-\underset{\text{treatment measurement error}}{\underbrace{\alpha \left(t\right)I\left(t\right)\sigma \frac{d{B}_2}{dt}}}$$

For simplicity, we assume that the Brownian motion driving transmission uncertainty ($B_1$) is independent from the Brownian motion driving treatment uncertainty ($B_2$). We suppose that ${\mu }_0\ge {\mu }_1$ (people die faster without treatment than with treatment), but the reader will not lose any intuition by supposing that ${\mu }_0={\mu }_1$. Most of the time $K_1\left(t\right)>K_0$ (treatment is better than no treatment), but not necessarily. We relax this requirement somewhat by requiring:

$$P(K_0<K_1\left(t\right))\text{ is close to one}$$ (3)

We model the treatment rate as an Ornstein–Uhlenbeck process:

$$d{K}_1\left(t\right)={\lambda }_k({\overline{k}}_1-K_1\left(t\right))dt+{\sigma }_{k}d{B}_2\left(t\right)$$

with the mean-reversion rate ${\lambda }_k>0$ and the long run value of the treatment rate ${\overline{k}}_1$. It is well-known that $K_1$ is Gaussian, with variance equal to:

$$Var\left[K_1\left(t\right)\right]=\frac{{\sigma }_k^2}{2{\lambda }_k}(1-e^{-2{\lambda }_{k}t})$$

Thus, if mean-reversion is large compared to volatility ${\sigma }_k$, constraint (3) is satisfied. We simplify (2) by:

$$\frac{\frac{dR\left(t\right)}{dt}}{I\left(t\right)}=K_0+{\mu }_0+\alpha \left(t\right)(-K_0+K_1\left(t\right)-{\mu }_0+{\mu }_1)-\alpha \left(t\right)\sigma \frac{d{B}_2}{dt}$$ (4)

Putting everything together, the dynamics of the infected is:

$$\frac{dI\left(t\right)}{dt}=\beta S\left(t\right)I\left(t\right)-\frac{dR\left(t\right)}{dt}-{\sigma }_S\sqrt{S\left(t\right)I\left(t\right)}\frac{d{B}_1}{dt}$$

We try to minimize a measure of the infected over our horizon $T$. To model risk-aversion to unfavorable treatment decisions, the decision-maker (for instance, governmental biomedical and public health entities specifying treatment guidelines) is supposed to minimize the expected value of a convex and increasing function of $I\left(T\right)$. Alternately, one can maximize the negative thereof, i.e., maximize the expected value of a concave and decreasing function of $I\left(T\right)$. Such a function $u$ is called a utility function in financial economics. The policy obtained in maximizing the expected value of a concave utility function can be shown, under certain conditions, to maximize the expected value of the outcome (here $-I$ ) under a constraint on the dispersion of the outcome. Out of the universe of concave decreasing utility functions, we choose the power utility function

$$u\left(I\right)=-\frac{I^{1-\gamma }}{1-\gamma }$$

The coefficient $\gamma$ is often called the risk-aversion parameter. When $\gamma =0$, the decision-maker is risk-neutral, meaning that the uncertainty does not have an influence on her decisions. It is straightforward to check that this power utility function is concave in $I$ when $\gamma <0$,which we will assume. Taking for instance $\gamma =-1$, we see that the objective is to

$$\text{maximize }E[-\frac{I^2}{2}]$$

which returns the same policy as:

$$\text{minimize }E\left[\frac{I^2}{2}\right]$$

The importance of analytic formulations is that other figures of interest in this model, like the expected number of deaths from treatment can be analytically calculated, and depend on $\gamma$. Thus, a decision-maker can calibrate its risk-aversion parameter $\gamma$ on other goals. Expected number of deaths is only one type of goal and economic factors that can be easily added. Our controlled SIR model is thus:

$$\underset{\alpha \left(t\right)}{max}E[-\frac{I{\left(T\right)}^{1-\gamma }}{1-\gamma }]$$

$$dS\left(t\right)=-\beta S\left(t\right)I\left(t\right)dt+{\sigma }_S\sqrt{S\left(t\right)I\left(t\right)}d{B}_1\left(t\right)$$ (5)

$$dI\left(t\right)=\left(\beta S\left(t\right)-(K_0+{\mu }_0)+\alpha \left(t\right)(K_0-K_1\left(t\right)+{\mu }_0-{\mu }_1)\right)I\left(t\right)dt+\alpha \left(t\right)I\left(t\right)\sigma d{B}_2\left(t\right)-{\sigma }_S\sqrt{S\left(t\right)I\left(t\right)}d{B}_1\left(t\right)$$ (6)

$$dK\left(t\right)={\lambda }_k(\overline{K}-K\left(t\right))dt+{\sigma }_{k}d{B}_2\left(t\right)$$

Observation The relative sign of our volatilities $\sigma$ and ${\sigma }_k$ is important. We will assume without loss of generality that $\sigma <0$. The sign of ${\sigma }_k$ is the sign of covariance between the measured value of today’s treatment rate and the change in value of the treatment rate between today and a future date. An example may help illustrate the difference. Suppose that over a week one performs daily measurements of the treatment recovery rate as well as daily forecasts of the evolution of the treatment recovery rate over the next day. The two quantities measured each day $t$ are proportional to the same white noise $B_2(t+1\text{day})-B_2\left(t\right)$. One then calculates weekly estimates $\hat{\sigma }$ of $\sigma$ and ${\hat{\sigma }}_k$ of ${\sigma }_k$ over these 7 daily observations. Since we arbitrarily choose $\sigma >0$, a positive ${\hat{\sigma }}_k$ shows a correlation of +1 between the measurement (of today’s treatment rate) and the forecast.

Fig. 1 is a depiction of our model.

Fig. 1.

A stochastic SIR Model.

3. Results in the low infection regime

3.1. Single treatment case

We assume $S\left(t\right)$ close to one and ${\sigma }_S=0$. Thus the term:

$$r=\beta S\left(t\right)-(K_0+{\mu }_0)\simeq \beta -K_0-{\mu }_0$$

is almost constant. We call $r$ the risk-free infection rate. Indeed treatment is risky but, on average has beneficial effects. We also define the impact of treatment risk $X$:

$$X\left(t\right)=\frac{K_0+{\mu }_0-{\mu }_1-K_1\left(t\right)}{\sigma }$$ (7)

as well as the long run impact of the treatment risk $\overline{X}:$

$$\overline{X}=\frac{K_0+{\mu }_0-{\mu }_1-{\overline{k}}_1}{\sigma }$$ (8)

Defining ${\lambda }_x={\lambda }_k$, ${\sigma }_x={\sigma }_k∕\sigma$, it is straightforward to see that $X\left(t\right)$ is also an Ornstein–Uhlenbeck process, i.e. :

$$dX\left(t\right)={\lambda }_x(\overline{X}-X\left(t\right))dt-{\sigma }_{x}d{B}_2\left(t\right)$$

and condition (3) translates into:

$$P(K_0<K_1\left(t\right))\text{ =}$$

$$P(\frac{{\mu }_1-{\mu }_0}{\sigma }>X\left(t\right))\text{ is close to one}$$

In Appendix A we develop a comparison between our model and a model of optimal investment,

We restate our problem thus as:

$$\underset{I\left(T\right)}{max}E[-\frac{I{\left(T\right)}^{1-\gamma }}{1-\gamma }]$$ (9)

$$dI\left(t\right)=(r+\alpha \left(t\right)\sigma X\left(t\right))I\left(t\right)dt+\alpha \left(t\right)\sigma I\left(t\right)d{B}_2\left(t\right)$$ (10)

$$dX\left(t\right)={\lambda }_x(\overline{X}-X\left(t\right))dt-{\sigma }_{x}d{B}_2\left(t\right)$$ (11)

In this regime our solution will depend on a kernel $H_0(X_t,t)$, while in the second regime it will also depend on two other kernels $H_1(X_t,t)$ and $H_2(X_t,t)$ that are closely related. In order to unify notation we define the kernels $H_i(X_t,t)$ in a unique way.

Definition

the solution kernels $H_i(X_t,t)$ for $i=0,1,2$ are given by:

$$H_i(X_t,t)=exp\left(\frac{1}{\gamma }\left(\frac{A_{i,1}\left(\tau \right)X_t^2}{2}\right)+A_{i,2}\left(\tau \right)X_t+A_{i,3}\left(\tau \right)\right)$$ (12)

For the kernel $i=0$, we have:

$$A_{0,1}\left(\tau \right)=\frac{1-\gamma }{\gamma }\frac{2(1-exp(-{\theta }_0\tau ))}{2{\theta }_0-(b_{0,2}+{\theta }_0)(1-exp(-{\theta }_0\tau ))}$$ (13)

$$A_{0,2}\left(\tau \right)={\lambda }_x\overline{X}{\int }_0^{\tau }exp\left(b_{0,3}{A}_{0,1}(\tau -s)+\frac{b_{0,2}}{2}(\tau -s)\right)A_{0,1}\left(s\right)ds$$ (14)

$$A_{0,3}\left(\tau \right)={\int }_0^{\tau }\frac{{\sigma }_x^2}{2\gamma }{A}_{0,2}^2\left(s\right)+\left(\frac{{\sigma }_x^2}{2}+{\lambda }_x\overline{X}\right)A_{0,1}\left(s\right)+(1-\gamma )rds$$ (15)

where

$$b_{0,1}=\frac{1-\gamma }{\gamma }<0$$

$$b_{0,2}=2(\frac{\gamma -1}{\gamma }{\sigma }_x-{\lambda }_x)$$

$$b_{0,3}=\frac{1}{\gamma }{\sigma }_x^2<0$$

$${\theta }_0=\sqrt{b_{0,2}^2-4b_{0,1}{b}_{0,3}}$$

Proposition 1

If ${\sigma }_x<0$ then the problem (9) , (10) , (11) has a unique optimal solution given by:

$$I\left(t\right)={\left(Z\left(t\right)\right)}^{1∕\gamma }{H}_0(X_t,T-t)$$ (16)

where

$$\frac{dZ\left(t\right)}{Z\left(t\right)}=(r+X^2\left(t\right))dt+X\left(t\right)d{B}_2\left(t\right)$$ (17)

$$Z\left(0\right)={\left(\frac{I\left(0\right)}{H_0(X\left(0\right),T)}\right)}^{\gamma }$$ (18)

Moreover the optimal allocation of the treatment ${\alpha }^{\ast }\left(t\right)$ is equal to

$${\alpha }_0\left(t\right)=\frac{X\left(t\right)}{\gamma \sigma }-\frac{{\sigma }_x}{\gamma \sigma }\left(A_{0,1}(T-t)X\left(t\right)+A_{0,2}(T-t)\right)$$ (19)

Sketch of Proof

For existence and uniqueness of the solution, we refer to Wachter [26]. The key point is to verify that $A_{0,1}$ given in (13) is finite, which occurs if ${\theta }_0>b_{0,2}$. Whereas Wachter proved it in the case $\gamma >1$, in our case $\gamma <0$. We first rewrite ${\theta }_0^2$ as:

$$b_{0,2}^2-4b_{0,1}{b}_{0,3}=4{\left(\frac{\gamma -1}{\gamma }{\sigma }_x-{\lambda }_x\right)}^2-4{\sigma }_x^2\frac{1-\gamma }{{\gamma }^2}=4(\frac{\gamma -1}{\gamma }{\sigma }_x^2-2\frac{\gamma -1}{\gamma }{\sigma }_x{\lambda }_x+{\lambda }_x^2)$$

Recall that, since $\gamma <0$, $\frac{\gamma -1}{\gamma }>1$. Thus, if ${\sigma }_x<0$:

$$b_{0,2}^2-4b_{0,1}{b}_{0,3}>4{(\left|{\sigma }_x\right|+{\lambda }_x)}^2$$

Clearly, if ${\sigma }_x<0$, then $b_{0,2}<0$, so that:

$${\theta }_0>b_{0,2}$$

Thus show that $A_{0,1}$ is finite and thus differentiable. Clearly (14), (15) show that both $A_{0,2}$ and $A_{0,3}$ are finite and differentiable. Let the operator $L_0$ be defined by

$$L_{0}F=\frac{1}{2}{\sigma }_x^2\frac{{\partial }^{2}F}{\partial X^2}+\frac{1}{2}{Z}^2{X}^2\frac{{\partial }^{2}F}{\partial Z^2}-ZX{\sigma }_x\frac{{\partial }^{2}F}{\partial Z\partial X}-\mu Z\frac{\partial F}{\partial Z}+({\lambda }_x\overline{X}+({\sigma }_x-{\lambda }_x)X)\frac{\partial F}{\partial X}-rF$$

The martingale method then results in the Ansatz $I\left(t\right)=F(Z\left(t\right),X\left(t\right),t)$ where $F$ solves the PDE:

$$\left(\frac{\partial }{\partial t}+L_0\right)F=0$$

which solution is (16). ■

The advantage of this solution is the remarkably clear interpretation of (19). To borrow terminology from finance, the optimal $\alpha$ is the sum of:

• •the myopic allocation $\frac{X\left(t\right)}{\gamma \sigma }$
• •the hedging allocation $\frac{{\sigma }_x}{\left|\gamma \right|\sigma }(A_{0,1}(T-t)X\left(t\right)+A_{0,2}(T-t))$.

As shown in [28], the myopic allocation is the optimal $\alpha$ in a simpler model where the impact of treatment risk is constant, which means that, in our model the recovery rate $K_1$ is a constant ${\overline{k}}_1$. It coincides with the static allocation in a traditional mean–variance model of Markowitz [29]. Thus as expected, the myopic allocation can be decomposed into:

$$\frac{X\left(t\right)}{\gamma \sigma }=\frac{1}{\left|\gamma \right|\sigma }\underset{\text{Sharpe ratio}}{\underbrace{\frac{K_1\left(t\right)+{\mu }_1-(K_0+{\mu }_0)}{\sigma }}}$$ (20)

The Sharpe ratio of a security is a measure of its risk-adjusted return and characterizes the attractiveness of the security. Conversely in our model the Sharpe ratio characterizes the potential benefit of the treatment. The less uncertain the treatment ($\sigma$ small) the more the treatment should be recommended. Also, the higher the difference $K_1\left(t\right)-K_0$, i.e., the difference of recovery rates between treatment and no treatment, the more desirable the treatment. Eq. (20) also shows the importance of the term $\frac{1}{\gamma }$. The more risk-averse the decision maker $\gamma$, the less likely he or she is to opt for the risky treatment.

For a risk-neutral decision maker, $\gamma =0$. Thus the myopic solution is simple:

• •if ${\overline{k}}_1+{\mu }_1>K_0+{\mu }_0$:  treat everybody
• •if ${\overline{k}}_1+{\mu }_1<K_0+{\mu }_0$: treat nobody.

We note that the same bang–bang solution obtains in the case of perfect knowledge of the treatment quality ($\sigma =0$).

Whereas the myopic allocation is a best response to treatment measurement error, the hedging allocation responds to the (future) stochastic behavior of the treatment. It is easy to see that both $A_{0,1}$ and $A_{0,2}$ decrease with time (in absolute value) and are equal to zero at the horizon $t=T$. Moreover, $A_{0,1}$ is positive.

Thus the importance of the hedging allocation decreases with time. This is consistent with the meaning of hedging: hedging is important at the beginning of treatment, because its effects are felt over a long period, but when time is close to the horizon, its importance vanishes.

To get a better grasp of the hedging allocation, we rewrite it in two equivalent expressions. We replace $X\left(t\right)$ by (7) and $\overline{X}$ by (8) and write $A_{0,2}\left(\tau \right)=\overline{X}p\left(\tau \right)$, where:

$$p\left(\tau \right)={\lambda }_x{\int }_0^{\tau }exp\left(-b_{0,2}{A}_{0,1}(\tau -s)-\frac{b_{0,2}}{2}(\tau -s)\right)A_{0,1}\left(s\right)ds\ge 0$$

The first expression for the hedging allocation is then:

$$-\frac{{\sigma }_k}{\left|\gamma \right|{\sigma }^3}\left(\underset{\ge 0}{\underbrace{A_{0,1}(T-t)}}\underset{}{\left(K_1\left(t\right)+{\mu }_1-(K_0+{\mu }_0)\right)}+\underset{\ge 0}{\underbrace{p(T-t)}}({\overline{k}}_1+{\mu }_1-K_0-{\mu }_0)\right)$$

Suppose that ${\lambda }_x$ is very small so that $p$ is negligible. In that case, the influence of ${\overline{k}}_1$ is negligible. Then assuming treatment is beneficial (both $K_1\left(t\right)$ larger than $K_0$, and ${\mu }_1={\mu }_0$) the hedging allocation is:

• •positive if ${\sigma }_k∕\sigma <0$
• •negative otherwise.

This policy above is consistent with the meaning of hedging. Suppose that today’s measurements are negatively correlated with the forecast of the recovery rate, i.e., ${\sigma }_k∕\sigma <0$,  then the hedging allocation should be positive in anticipation of better treatment performance to come. Conversely, the hedging allocation should be negative when the forecast is worse than the measurement. To highlight the importance of the long run value of the treatment ${\overline{k}}_1$ when ${\lambda }_x$ is larger, we use our second expression for the hedging allocation:

$$-\frac{{\sigma }_k}{\left|\gamma \right|{\sigma }^3}\ast$$ (21)

$$+\left\{\underset{\ge 0}{\underbrace{p\left(t\right)}}({\overline{k}}_1-K_1\left(t\right))\right)$$ (22)

$$\left(+\underset{\ge 0}{\underbrace{\left(A_{0,1}(T-t)+p(T-t)\right)}}(K_1\left(t\right)+{\mu }_1-(K_0+{\mu }_0))\right\}$$ (23)

Consider the case when treatment improves with time, i.e., when ${\overline{k}}_1>K_1\left(t\right)$. There are now two subcases. If ${\sigma }_k$ has the opposite sign as $\sigma$, then the hedging allocation increases with $({\overline{k}}_1-K_1\left(t\right))$, in anticipation of even better results to come. Conversely, if ${\sigma }_k$ has the same sign as $\sigma$, then the hedging allocation decreases with $({\overline{k}}_1-K_1\left(t\right))$.

Like for the myopic allocation, the absolute value of the hedging allocation is inversely proportional to $\left|\gamma \right|{\sigma }^2$. The higher the risk aversion ($\left|\gamma \right|$ high) or the higher the imprecision (${\sigma }^2$ high), the smaller should be the magnitude of the hedging allocation.

Finally, it is remarkable that the value of $I\left(t\right)$ has no impact on the optimal treatment policy.

The following figures show how the optimal allocation varies as a function of the risk-aversion parameter $\gamma$. Fig. 2 shows the optimal allocation as a function of the Sharpe ratio for horizon $T=5$ months at time $t=0$.

Fig. 2.

Parameters are ${\mu }_0={\mu }_1=0.1$, $K_0=K_1\left(0\right)=1.8$, ${\overline{k}}_1=2$, $\sigma =1$, ${\sigma }_k=0.1$, ${\lambda }_k=1$.

Fig. 3, Fig. 4, Fig. 5, Fig. 6 report the expected valued and the standard deviation of the optimal allocation ${\alpha }_0$ as a function of time. As can be seen, since ${\alpha }_0$ is Gaussian, it is easy to reduce the probability that ${\alpha }_0$ is outside the interval $[0,1]$: one needs only select a lower $\gamma$. The parameters for the four cases below are given in Table 1. We assume that time is measured in months.

Fig. 3.

Optimal Allocation. See Table 1 for parameter values.

Fig. 4.

Optimal Allocation. See Table 1 for parameter values.

Fig. 5.

Optimal Allocation. See Table 1 for parameter values.

Fig. 6.

Optimal Allocation. See Table 1 for parameter values.

Table 1.

Parameters for Fig. 3, Fig. 4, Fig. 5, Fig. 6.

Treatment parameter	Symbol	Value
Death rate/no treatment	${\mu }_0$	0.1
Death rate	${\mu }_1$	0.1
Recovery rate/ no treatment	$K_0$	1.8
Recovery rate at time 0	$K_1\left(0\right)$	1.8
Long run value of recovery rate	${\overline{k}}_1$	2.5
Volatility of the measurement of today’s recovery rate	$\sigma$	1
Volatility of changes in the recovery rate	${\sigma }_k$	0.1
Speed of mean-reversion of the recovery rate	${\lambda }_k$	1

3.2. Multiple treatment case

Generalizing the model above to multiple treatments $i\in \{0,\dots ,n\}$ poses considerable technical difficulties. We refer the reader to Liu (2007) for a discussion. We consider instead in this section a useful simplification of the model. While the impact of each treatment is imprecise, each treatment recovery rate $K_i$ is constant, i.e.:

$$K_i={\overline{k}}_{i}i=0,\dots ,n$$

The allocation ${\alpha }_i\ge 0$, which as explained above can represent the proportion of the infected undergoing treatment $i$ satisfies:

$$\sum _{i=0}^n{\alpha }_i\left(t\right)=1$$

We suppose a normal model, whereby that the covariance between the treatment recovery rates $K_i$ and $K_j$ over a period of time equal to $\Delta t$ is given by $v_{ij}$ $\Delta t$. Let $\left({\sigma }_{i,j}\right)$ be a square root of the variance–covariance matrix $\mathbf{V}=\left(v_{ij}\right)$. The equation for the out-of infection rate (4) can thus be generalized into:

$$\frac{dR\left(t\right)}{I\left(t\right)}=K_0+{\mu }_0+\sum _{i=1}^n\alpha \left(t\right)(-k_0+{\overline{k}}_i-{\mu }_0+{\mu }_i)-\sum _{i=1}^n{\alpha }_i\left(t\right)\sum _{j=1}^n{\sigma }_{ij}d{B}_{j+1}$$ (24)

where $(B_2,\dots ,B_{n+1})$ are independent Brownian motions. The resulting problem is identical to the Merton [28] portfolio problem. We define the following vectors. Let $\mathbf{\alpha }=({\alpha }_1,\dots ,{\alpha }_n)$ be the allocation, $\overline{\mathbf{k}}=(k_1,\dots ,k_n)$ be the treatment recovery rate, $\mathbf{\mu }=({\mu }_1,\dots ,{\mu }_n)$ be the death rate of each treatment, and $\mathbf{e}$ be the vector of ones. Then the optimal allocation is:

$$\mathbf{\alpha \; =}\frac{1}{\left|\gamma \right|}{\mathbf{V}}^{-1}(\overline{\mathbf{k}}+\mathbf{\mu }-({\overline{k}}_0+{\mu }_0)\mathbf{e})$$

The attentive reader will realize that this is a multivariate generalization of the myopic allocation (20). While a good measure of the efficiency of the treatment is the Sharpe ratio in the single treatment case, we suggest that, for multiple treatments, a good measure to compare treatments would be the beta of each treatment recovery rate, especially if the number of treatments is large. The beta of a security is one of the main measures to pick stocks in a financial portfolio. In addition to the Sharpe ratio, the beta includes the correlation between the recovery rate of a single treatment and the recovery rate of a combination of optimal treatments. Since space is lacking to define beta properly, we refer the reader to a financial textbook such as Ingersoll [30].

4. Results in the moderate infection regime

For simplicity, we write:

$$\mu =K_0+{\mu }_0$$

We restate our problem thus as:

$$\underset{\alpha \left(t\right)}{max}E[-\frac{I{\left(T\right)}^{1-\gamma }}{1-\gamma }]$$ (25)

$$dS\left(t\right)=-\beta S\left(t\right)I\left(t\right)dt+{\sigma }_S\sqrt{S\left(t\right)I\left(t\right)}d{B}_1\left(t\right)$$ (26)

$$dI\left(t\right)=(\beta S\left(t\right)-\mu +\alpha \left(t\right)\sigma X\left(t\right))I\left(t\right)dt-{\sigma }_S\sqrt{S\left(t\right)I\left(t\right)}d{B}_1\left(t\right)+$$ (27)

$$\alpha \left(t\right)\sigma I\left(t\right)d{B}_2\left(t\right)$$ (28)

$$dX\left(t\right)={\lambda }_x(\overline{X}-X\left(t\right))dt-{\sigma }_{x}d{B}_2\left(t\right)$$ (29)

Our solutions are also written as a function of the kernels defined by the formula (12). For the first kernel $H_1$, we have:

$$A_{1,1}\left(\tau \right)=A_{0,1}\left(\tau \right)$$ (30)

$$A_{1,2}\left(\tau \right)=A_{0,2}\left(\tau \right)$$ (31)

$$A_{1,3}\left(\tau \right)=A_{0,3}\left(\tau \right)+(1-\gamma )(\mu -r)\tau$$ (32)

The second kernel $H_2$, is obtained by replacing $\gamma$ by $\gamma ∕2$ in the first kernel, that is:

$$A_{2,1}\left(\tau \right)=\frac{1-\gamma ∕2}{\gamma ∕2}\frac{2(1-exp(-{\theta }_2\tau ))}{2{\theta }_0-(b_{2,2}+{\theta }_2)(1-exp(-{\theta }_2\tau ))}$$ (33)

$$A_{2,2}\left(\tau \right)={\lambda }_x\overline{X}{\int }_0^{\tau }exp\left(b_{3,2}{A}_{2,1}(\tau -s)+\frac{b_{2,2}}{2}(\tau -s)\right)A_{2,1}\left(s\right)ds$$ (34)

$$A_{2,3}\left(\tau \right)={\int }_0^{\tau }\frac{{\sigma }_x^2}{2\gamma }{A}_{2,2}^2\left(s\right)+\left(\frac{{\sigma }_x^2}{2}+{\lambda }_x\overline{X}\right)A_{2,1}\left(s\right)-(1-\gamma ∕2)\mu ds$$ (35)

where

$$b_{2,1}=\frac{1-\gamma ∕2}{\gamma ∕2}$$

$$b_{2,2}=2(\frac{\gamma ∕2-1}{\gamma ∕2}{\sigma }_x-{\lambda }_x)$$

$$b_{2,3}=\frac{2}{\gamma }{\sigma }_x^2$$

$${\theta }_2=\sqrt{b_{2,2}^2-4b_{2,1}{b}_{2,3}}$$

Proposition 2

Let $I\left(0\right)=\epsilon$ . If ${\sigma }_x<0$ then the problem (25) to (29) has a solution such that

$$I\left(t\right)=\epsilon Z^{1∕\gamma }\left(t\right)H_1(X\left(t\right),T-t)+$$ (36)

$$+{\epsilon }^2\frac{{\beta }^2}{2\gamma {\sigma }_{S^2}}Z{\left(t\right)}^{2∕\gamma }S\left(t\right){\int }_{u=t}^T{H}_2(X\left(t\right),u-t)H_1{(X\left(t\right),u)}^{2}du+O\left({\epsilon }^3\right)$$

where $Z\left(t\right)$ satisfies:

$$\frac{dZ}{Z}=(-\mu +X^2+\frac{{\beta }^{2}SI}{{\sigma }_S^2})dt-\frac{\beta \sqrt{SI}}{{\sigma }_S}d{B}_1+Xd{B}_2$$ (37)

$$Z\left(0\right)={\left(\frac{I\left(0\right)∕\epsilon }{H_1(X\left(0\right),T)}\right)}^{\gamma }$$

Moreover the optimal proportion undergoing treatment ${\alpha }^{\ast }\left(t\right)$ is equal to ${\alpha }_0\left(t\right)+\epsilon {\alpha }_1\left(t\right)+O\left({\epsilon }^2\right)$ , where ${\alpha }_0\left(t\right)$ is given in (19) :

Observation: It is remarkable that, as a first order approximation, the optimal policy is the same in the low and moderate pandemic regimes. The term ${\alpha }_1\left(t\right)$ can be easily calculated by inserting (36) into (68). Both $S\left(t\right)$ and $I\left(t\right)$ have a non-negligible impact on ${\alpha }_1\left(t\right)$. We leave a more detailed analysis for future work.

5. Application to COVID-19

In this section, we assume a low infection regime. We calculated the optimal control and infected for two experiments:

• •experiment 1: US data set in 2020 with long run value of the recovery rate (${\overline{k}}_1$) estimated from the data
• •experiment 2: US data set in 2020 with improved long run value of the recovery rate (${\overline{k}}_1$).

The reason for considering the second dataset is clear, after observing the results. The value of ${\overline{k}}_1$ estimated from the data was very low, barely better than the no treatment recovery rate ($K_0$). With a constant value of $\beta$, the pandemic goes beyond control, and the low infection regime assumption does not hold any more, yielding absurd results. Multiplying ${\overline{k}}_1$ by a factor 10 makes us stay in the low infection regime in experiment 2.

Results in Section 3 show that for $\gamma >-5$ the probability that ${\alpha }_0\left(t\right)\notin [0,1]$ is significant. For this reason, we used lower values of $\gamma$ in this section.

In both experiments, we performed a Monte Carlo simulation using the Euler scheme and 10,000 scenarios, starting at $I\left(0\right)=0.003$, or about 1 million persons in the US.

5.1. Experiment 1: US DataSet in 2020

We calibrated our low infection regime model to weekly US Covid-19 data from April 12, 2020 to November 8, 2020. To simulate our model for the US population, we used publicly available data from the CDC [31] and the COVID Tracking Project [32] on COVID-19 cases and deaths by state over time for the period 4/12/20–11/8/20, mortality estimates from the Coronavirus Resource Center at Johns Hopkins University [33] and US Census data [34] to estimate the 2020 US population (i.e., denominator data). We supplemented these data with results from published studies of treated hospitalized COVID-19 patients [35], [36], [37], [38], statistics provided by the CDC for the purpose of COVID-19 pandemic planning [39], and referenced NIH COVID-19 Treatment Guidelines [40].

We assumed that there was no treatment before May 30, 2020, and an average treatment rate of 25% afterwards. The number of recovered in the period before May 30 was used to estimate $K_0$. The transmission rate $\beta$ was assumed to be constant over the period. Likewise, since the treatment did not show consistent benefit on reducing deaths for patients with COVID-19, we set ${\mu }_0={\mu }_1$ and chose the whole period to estimate it.

Taking the logarithm of the series and applying proper differencing, we obtained an ARMA(1,1) model for the period after May 30, 2020, which we estimated using the Econometric Toolbox in Matlab, which gave us all the other parameters. We set $K_1\left(0\right)=K_0$.

We obtained the following parameters shown in Table 2.

Fig. 7, Fig. 8 show the expected value and the standard deviation of the optimal allocation ${\alpha }_0$ as a function of time. As before, a lower $\gamma$ reduces the dispersion of ${\alpha }_0$ as well as its mean. The optimal control increases with time on average, since the average recovery rate $E\left[K_1\left(t\right)\right]$ increases with time. However, Fig. 9, Fig. 10 show that after about-15 10 weeks the pandemic leaves the low infection regime. Results are then absurd,2 and are showed only for the sake of completeness.

Table 2.

Parameters for Fig. 7, Fig. 8, Fig. 9, Fig. 10.

Treatment parameter	Symbol	Value
Death rate/no treatment	${\mu }_0$	0.0575
Death rate	${\mu }_1$	0.0575
Recovery rate/ no treatment	$K_0$	0.2559
Recovery rate at time 0	$K_1\left(0\right)$	0.2559
Long run value of recovery rate	${\overline{k}}_1$	0.4612
Volatility of the measurement of today’s recovery rate	$\sigma$	−0.4418
Volatility of changes in the recovery rate	${\sigma }_k$	−1.6623
Speed of mean-reversion of the recovery rate	${\lambda }_k$	0.7692

Fig. 7.

Optimal Allocation. See Table 2 for parameter values.

Fig. 8.

Optimal Allocation. See Table 2 for parameter values.

Fig. 9.

Optimal Infected. See Table 2 for parameter values.

Fig. 10.

Optimal Infected. See Table 2 for parameter values.

5.2. Experiment 2: US DataSet in 2020 with improved Treatment

We took the same parameters as in experiment 1, except for ${\overline{k}}_1$.

Fig. 10, Fig. 11 show the expected value and the standard deviation of the optimal allocation ${\alpha }_0$ as a function of time. As before, a lower $\gamma$ reduces the dispersion of ${\alpha }_0$ as well as its mean. The optimal control increases with time on average, since the average recovery rate $E\left[K_1\left(t\right)\right]$ increases with time. Compared to experiment 2, the long run value of the recovery rate is sufficient to keep the epidemic in check, and the allocation is larger, since the treatment is better. The results are relatively insensitive to the value of $\gamma$, for $-10>\gamma >-20$. For higher values of $\gamma$, the optimal allocation is often larger than 1.

Table 3.

Parameters for Fig. 11, Fig. 12, Fig. 13, Fig. 14.

Treatment parameter	Symbol	Value
Death rate/no treatment	${\mu }_0$	0.0575
Death rate	${\mu }_1$	0.0575
Recovery rate/ no treatment	$K_0$	0.2559
Recovery rate at time 0	$K_1\left(0\right)$	0.2559
Long run value of recovery rate	${\overline{k}}_1$	4
Volatility of the measurement of today’s recovery rate	$\sigma$	−0.4418
Volatility of changes in the recovery rate	${\sigma }_k$	−1.6623
Speed of mean-reversion of the recovery rate	${\lambda }_k$	0.7692

Fig. 11.

Optimal Allocation. See Table 3 for parameter values.

6. Conclusion

We obtained in this paper a series of analytical expressions for the optimal proportion of infected undergoing treatment in a pandemic. We analyzed the low infection regime, where the pandemic statistics and dynamics do not have an impact. We then analyzed the moderate infection regime, where pandemic statistics and dynamics have a second order impact on the optimal decision. The main technical result of this article is Proposition 2. It is indeed remarkable that, while the SIR model with treatment uncertainty has no clear analytical solution that we know of, the optimal policy is tractable.

Many important problems remain to be solved. The first one consists in delimiting the frontier between the moderate infection and the high infection regimes. The solution technique used from Proposition 2 can be expanded to higher orders, but one needs to verify whether the solution is meaningful, i.e., if $I\left(t\right)$ remains between zero and one. If not, then we reach the catastrophic high infection regime. Separating the differential operator acting on $I$ into two differential operators $L_1$ and $L_2$ (see (53), (54)) is qualitatively important. While the operator $L_1$ is a traditional semilinear parabolic operator, the operator $L_2$ is a quasilinear operator that resembles the operator in the nonlinear traffic equation. We may thus expect the catastrophe to arise from a shock wave, which would dominate the diffusive effects.

The multiple treatment situation deserves further attention. Indeed, our analysis in this article was restricted to the low infection regime with no uncertainty over the evolution of the treatment recovery rate. One should generalize our solution technique to the moderate infection regime, and possibly consider uncertainty over the forecast of the recovery rate.

Finally, we believe that the martingale approach of optimal control can be fruitfully applied to analytically characterize optimal vaccination schemes.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

CRediT authorship contribution statement

Nicole M. Gatto: Conceptualization, Methodology, Writing -original draft, Writing - review and editing, Visualization. Henry Schellhorn: Conceptualization, Methodology, Software, Formal Analysis, Writing - original draft, Writing - review & editing, Visualization.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A. Relation with the financial investment problem

The following table maps out the correspondence in variable names between the investment problem considered by Wachter [26] and our controlled stochastic SIR model.

Consumption/investment model	Controlled stochastic SIR model
Wealth $W\left(t\right):$ maximize	Infected $I\left(t\right)$: minimize
Risk-free interest rate $r>0$	Risk-free infection rate $r$
Risky asset allocation $\alpha$	Proportion undergoing risky treatment $\alpha$
Price of market risk $X$ (usually $\ge 0$)	Impact of treatment risk $X$ (usually $\le 0$)
Risk aversion coefficient $\gamma >0$	Risk aversion coefficient $\gamma <0$

Appendix B. Proof or Proposition 2

We introduce two Radon–Nikodym derivatives ${\tilde{Z}}_1$ and ${\tilde{Z}}_2$:

$$d{\tilde{Z}}_1=\frac{\beta \sqrt{SI}}{{\sigma }_S}{\tilde{Z}}_{1}d{B}_1$$

$$d{\tilde{Z}}_2=-{\tilde{Z}}_{2}Xd{B}_2$$

By Girsanov theorem, the measure $\tilde{\mathbb{P}}$ defined by:

$$\tilde{\mathbb{P}}\left(A\right)={\int }_A{\tilde{Z}}_1(T,\omega ){\tilde{Z}}_2(T,\omega )d\mathbb{P(\omega )}$$

for all $A$ in the filtration generated by $(B_1,B_2)$ is such that:

$${\tilde{B}}_1\left(t\right)\equiv B_1\left(t\right)-{\int }_0^t\frac{\beta \sqrt{S\left(s\right)I\left(s\right)}}{{\sigma }_S}ds$$ (38)

$${\tilde{B}}_2\left(t\right)\equiv B_2\left(t\right)+{\int }_0^{t}X\left(s\right)ds$$ (39)

are $\tilde{\mathbb{P}}$-Brownian motions. We defined a stochastic process $\varphi$ such that ${\tilde{Z}}_1\varphi I$ becomes a $\mathbb{P}$-martingale, with:

$$d\varphi =\mu \varphi dt-X\varphi d{B}_2$$

By Ito’s lemma:

$$d\left({\tilde{Z}}_1\varphi I\right)={\tilde{Z}}_1\varphi \left((I\frac{\beta \sqrt{SI}}{{\sigma }_S}-{\sigma }_S\sqrt{SI})d{B}_1+(\alpha \sigma I-XI)d{B}_2\right)-{\tilde{Z}}_1\varphi Ydt$$ (40)

Observe that ${\tilde{Z}}_{1}S$ is also a $\mathbb{P}$-martingale. Defining the Lagrange multipliers $l$ and $k$, the martingale method consist in first solving the following problem:

$$\underset{I\left(T\right)}{max}E[-\frac{I{\left(T\right)}^{1-\gamma }}{1-\gamma }+l\left({\tilde{Z}}_1\left(T\right)\varphi \left(T\right)I\left(T\right)-I_0\right)+k({\tilde{Z}}_1\left(T\right)S\left(T\right)-S\left(0\right))]$$

Since the last term does not contain $I\left(T\right)$, the optimal $I\left(T\right)$ satisfies $I\left(T\right)={\left(l{\tilde{Z}}_1\left(t\right)\varphi \left(t\right)\right)}^{-1∕\gamma }$. For convenience, we introduce a process $Z\equiv {\left(l{\tilde{Z}}_1\varphi \right)}^{-1}$, thus

$$I\left(T\right)={\left(Z\left(T\right)\right)}^{1∕\gamma }$$ (41)

By Ito’s lemma, the SDE (37) for $Z$ obtains. Since ${\tilde{Z}}_1\varphi I$ is a $\mathbb{P}$-martingale, and since $X_t,Z_t,S_t$ are sufficient statistics for the filtration ${\mathcal{F}}_t$

$$I\left(t\right)=Z\left(t\right)E\left[{\left(Z\left(T\right)\right)}^{1∕\gamma -1}\right|X_t,Z_t,S_t]$$ (42)

This, we posit a function $F$ such that the optimal $I$ satisfies:

$$I\left(t\right)=F(Z_t,X_t,S_t,t)$$

While $I$ is not a $\tilde{\mathbb{P}}$ - martingale, the process defined $W$ defined by:

$$W\left(t\right)\equiv I\left(t\right)e^{\mu t}$$ (43)

is a $\tilde{\mathbb{P}}$ - martingale. Indeed:

$$dW=e^{\mu t}(-{\sigma }_S\sqrt{SI}d{\tilde{B}}_1+\alpha \sigma Id{\tilde{B}}_2)$$ (44)

By Ito’s lemma applied to (43), we see that:

$$dW=\left(\frac{\partial F}{\partial S}{\sigma }_S\sqrt{SI}-\frac{\partial F}{\partial Z}\frac{\beta \sqrt{SI}Z}{{\sigma }_S}\right)e^{\mu t}d{\tilde{B}}_1-\left(\frac{\partial F}{\partial Z}XZ-\frac{\partial F}{\partial X}{\sigma }_x\right)e^{\mu t}d{\tilde{B}}_2$$ (45)

Comparing (27), (45), we see that:

$$-{\sigma }_S\sqrt{SI}=\left(\frac{\partial F}{\partial S}{\sigma }_S\sqrt{SI}-\frac{\partial F}{\partial Z}Z\frac{\beta \sqrt{SI}}{{\sigma }_S}\right)$$ (46)

$$\alpha \sigma I=\left(\frac{\partial F}{\partial Z}XZ-\frac{\partial F}{\partial X}{\sigma }_x\right)$$ (47)

Thus (27) becomes:

$$dI=E_t\left[dI\right]+\left(\frac{\partial F}{\partial S}{\sigma }_S\sqrt{SI}-\frac{\partial F}{\partial Z}Z\frac{\beta \sqrt{SI}}{{\sigma }_S}\right)d{B}_1+\left(\frac{\partial F}{\partial Z}XZ-\frac{\partial F}{\partial X}{\sigma }_x\right)d{B}_2$$ (48)

Since ${\tilde{E}}_t\left[d\left(I\left(t\right)e^{\mu t}\right)\right]=0$, substituting (38), (39) in (48) results in:

$$0={\tilde{E}}_t[\mu Idt+dI]=\mu Idt+E_t\left[dI\right]+\left(\frac{\partial F}{\partial S}\beta \sigma SI-\frac{\partial F}{\partial Z}\frac{{\beta }^{2}ZSI}{{\sigma }_S^2}\right)dt$$ (49)

$$-\left(\frac{\partial F}{\partial Z}{X}^{2}Z-\frac{\partial F}{\partial X}{\sigma }_{x}X\right)dt$$

Dividing (46) by $\sqrt{SI}$, we obtain:

$$\frac{\partial F}{\partial S}=\frac{\partial F}{\partial Z}Z\frac{\beta }{{\sigma }_S^2}-1$$

This relation will allow us to replace all the partial derivatives with respect to $S$ by derivatives with respect to $Z$:

$$\frac{{\partial }^{2}F}{\partial S^2}={\left(\frac{\beta }{{\sigma }_S^2}\right)}^2\left(\frac{{\partial }^{2}F}{\partial Z^2}{Z}^2+\frac{\partial F}{\partial Z}Z\right)$$

$$\frac{{\partial }^{2}F}{\partial Z\partial S}=\frac{\beta }{{\sigma }_S^2}\left(\frac{{\partial }^{2}F}{\partial Z^2}Z+\frac{\partial F}{\partial Z}\right)$$

$$\frac{{\partial }^{2}F}{\partial X\partial S}=\frac{\beta }{{\sigma }_{S^2}}\frac{{\partial }^{2}F}{\partial X\partial Z}Z$$

With these substitutions, the Dynkin operator $\mathcal{L}$ defined by $\mathcal{L}Fdt=E_t\left[dF\right]$ can thus be rewritten:

$$\mathcal{L}F=\frac{1}{2}\frac{{\partial }^{2}F}{\partial X^2}{\sigma }_x^2+\frac{1}{2}\frac{{\partial }^{2}F}{\partial Z^2}\left(\frac{{\beta }^{2}SF{Z}^2}{{\sigma }_S^2}+X^2{Z}^2\right)\times \underset{\frac{1}{2}\frac{{\partial }^{2}F}{\partial S^2}}{\underbrace{\frac{1}{2}\left(\frac{{\partial }^{2}F}{\partial Z^2}{Z}^2+\frac{\partial F}{\partial Z}Z\right)}}{\left(\frac{\beta }{{\sigma }_S^2}\right)}^2{\sigma }_S^{2}SF$$ (50)

$$-\frac{{\partial }^{2}F}{\partial X\partial Z}ZX{\sigma }_x-\underset{\frac{{\partial }^{2}F}{\partial Z\partial S}}{\underbrace{\frac{\beta }{{\sigma }_S^2}\left(\frac{{\partial }^{2}F}{\partial Z^2}Z+\frac{\partial F}{\partial Z}\right)}}\beta ZSF++\frac{\partial F}{\partial X}{\lambda }_x(\overline{X}-X)+\frac{\partial F}{\partial Z}Z\left[(-\mu +X^2+\frac{{\beta }^{2}SF}{{\sigma }_S^2})\right]+\underset{\frac{\partial F}{\partial S}}{\underbrace{\left(\frac{\partial F}{\partial Z}Z\frac{\beta }{{\sigma }_S^2}-1\right)}}(-\beta SF)$$ (51)

We can thus rewrite (49) as:

$$\frac{1}{2}\frac{{\partial }^{2}F}{\partial X^2}{\sigma }_x^2+\frac{1}{2}{Z}^2{X}^2\frac{{\partial }^{2}F}{\partial Z^2}-ZX{\sigma }_x\frac{{\partial }^{2}F}{\partial Z\partial X}$$ (52)

$$-\left(\frac{1}{2}{\left(\frac{\beta }{{\sigma }_{S^2}}\right)}^2{\sigma }_S^{2}SFZ+\mu Z\right)\frac{\partial F}{\partial Z}$$

$$+({\lambda }_x\overline{X}+({\sigma }_x-{\lambda }_x)X)\frac{\partial F}{\partial X}+\frac{\partial F}{\partial t}+\mu F=0$$

We do a perturbation expansion of (52) to the second order by defining two operators: $L_1$, which does not contain $S$ terms and $L_2$, which does. The operator $L_1$ will be more important than the operator $L_2$ in a moderate pandemic regime, as we shall see below. Defining:

$$L_{1}F=\frac{1}{2}{\sigma }_x^2\frac{{\partial }^{2}F}{\partial X^2}+\frac{1}{2}{Z}^2{X}^2\frac{{\partial }^{2}F}{\partial Z^2}-ZX{\sigma }_x\frac{{\partial }^{2}F}{\partial Z\partial X}$$ (53)

$$-\mu Z\frac{\partial F}{\partial Z}+({\lambda }_x\overline{X}+({\sigma }_x-{\lambda }_x)X)\frac{\partial F}{\partial X}+\mu F$$

$$L_{2}F=-\frac{1}{2}\frac{{\beta }^2}{{\sigma }_{S^2}}ZSF\frac{\partial F}{\partial Z}$$ (54)

Thus (52) can be written

$$\frac{\partial F}{\partial t}+\left(L_1+L_2\right)F=0$$ (55)

Writing $I\left(0\right)=\epsilon$, we will assume that $I\left(t\right)$ will remain of order $\epsilon$ in $L^2$. We define $f$ by:

$$f=\frac{1}{\epsilon }F$$ (56)

Since $L_1$ is linear and $\frac{\partial }{\partial t}$ are linear, equation (55) can be rewritten:

$$\frac{\partial f}{\partial t}+L_{1}f+\frac{1}{\epsilon }{L}_{2}F=0$$

However $L_2$ is quadratic, thus $L_{2}F=L_2\left(\epsilon f\right)={\epsilon }^2{L}_{2}f$, and (55) becomes:

$$(\frac{\partial }{\partial t}+L_1+\epsilon L_2)f=0$$ (57)

Our asymptotic expansion consists in:

$$f=f_1+\epsilon f_2+O\left({\epsilon }^2\right)$$

which we insert in (57) to find:

$$0=(\frac{\partial }{\partial t}+L_1+\epsilon L_2)(f_1+\epsilon f_2+O\left({\epsilon }^2\right))=(\frac{\partial }{\partial t}+L_1)f_1+\epsilon (L_2{f}_1+(\frac{\partial }{\partial t}+L_1)f_2)+O\left({\epsilon }^2\right)$$

The first two terms of our asymptotic expansion are thus determined by:

$$\left(\frac{\partial }{\partial t}+L_1\right)f_1=0$$ (58)

$$(\frac{\partial }{\partial t}+L_1)f_2=-L_2{f}_1$$ (59)

Solution of (58)

Fig. 12.

Optimal Allocation. See Table 3 for parameter values.

Fig. 13.

Optimal Infected. See Table 3 for parameter values.

Recall the differential operator $L_0$ which we defined in order to characterize the solution of the low pandemic mode. We remark that:

$$(L_0-L_1)F=-(r+\mu )F$$

Thus equation (58) has the same solution as Wachter [26], provided we do the substitution $r⟷-\mu$, and set $\rho =0$. Since we obtain slightly different results for $A_{0,2}$ from Wachter, we provide details of our solution. We postulate that the solution $f_1$ to (58) is separable:

$$f_1(Z,X,t)=Z^{1∕\gamma }{H}_1(X,T-t)$$ (60)

Substitution in (58) shows that $H_1$ solves:

$$\left(\frac{\partial }{\partial t}-L^{\gamma }\right)H_1=0$$ (61)

$$H_1(X,0)=1$$

where the operator $L^{\gamma }$ is defined by:

$$L^{\gamma }H\equiv \frac{1}{2}{\sigma }_x^2\frac{{\partial }^{2}H}{\partial X^2}+\left((\frac{\gamma -1}{\gamma }{\sigma }_x-{\lambda }_x)X+{\lambda }_x\overline{X}\right)\frac{\partial H}{\partial X}+\left(X^2\left(\frac{1}{2}\frac{1}{\gamma }(\frac{1}{\gamma }-1)\right)+\mu (1-\frac{1}{\gamma })\right)f$$

Using the Ansatz (12), we can rewrite the LHS of (61) into:

$$(C_1\left(t\right)X^2+C_2\left(t\right)X+C_3\left(t\right))H∕\gamma =0$$

Clearly all terms $C_1,C_2,C_3$ must be identically zero. The equation $C_1=0$ becomes:

$$\frac{d{A}_{1,1}}{dt}=\frac{{\sigma }_x^2}{\gamma }{A}_{1,1}^2+2(\frac{\gamma -1}{\gamma }{\sigma }_x-{\lambda }_x)A_{1,1}+\frac{1-\gamma }{\gamma }$$

The equation $C_2=0$ is:

$$\frac{d{A}_{1,2}}{dt}=\frac{{\sigma }_x^2{A}_{1,1}}{\gamma }{A}_{1,2}+\left(\frac{\gamma -1}{\gamma }{\sigma }_x-{\lambda }_x\right)A_{1,2}+{\lambda }_x\overline{X}{A}_{1,1}$$

The equation $C_3=0$ is:

$$\frac{d{A}_{1,3}}{dt}=\frac{{\sigma }_x^2}{2}\left(A_{1,1}+\frac{A_{1,2}^2}{\gamma }\right)+{\lambda }_x\overline{X}{A}_{1,2}-\mu (1-\gamma )$$

which admit the solutions (30),(31),(32).

Solution of (59)

The second equation can be rewritten

$$(\frac{\partial }{\partial t}+L_1)f_2=-L_2\left(Z^{1∕\gamma }{H}_1(X,T-t)\right)$$ (62)

$$=\frac{1}{2}\frac{{\beta }^2}{{\sigma }_{S^2}}ZS{Z}^{1∕\gamma }{H}_1(X,T-t)\frac{\partial Z^{1∕\gamma }}{\partial Z}{H}_1(X,T-t)$$ (63)

$$=-\frac{1}{2}\frac{{\beta }^2}{\gamma {\sigma }_{S^2}}{Z}^{2∕\gamma }S{H}_1{(X,T-t)}^2$$ (64)

The trick is to consider $\frac{\partial }{\partial t}+L_1$ to be a linear operator applied not to a function $f_2:{\mathbb{R}}^4\to \mathbb{R}$ but to a stochastic field:

$$f_2(Z(t,\omega ),X(t,\omega ),t,\omega )$$

We try the Ansatz:

$$f_2(Z\left(t\right),X\left(t\right),t)=Z{\left(t\right)}^{2∕\gamma }S\left(t\right)g(X\left(t\right),t)$$ (65)

By the same reasoning as before, the terms $Z{\left(t\right)}^{2∕\gamma }S\left(t\right)$ can be canceled out from (62) provided the terminal condition (41) holds:

$$\left(\frac{\partial }{\partial t}-L^{\gamma ∕2}\right)g(X,t)=-\frac{1}{2}\frac{{\beta }^2}{{\sigma }_{S^2}}{H}_1{(X,T-t)}^2$$

$$g(X,T)=0$$

Clearly, $f(X,t)\equiv H_2(X,T-t)$ given by (12),(33),(34), (35) solves

$$\left(\frac{\partial }{\partial t}-L^{\gamma ∕2}\right)f=0$$ (66)

$$f(X,T)=1$$

Thus:

$$g(X,t)=\frac{1}{2}\frac{{\beta }^2}{\gamma {\sigma }_{S^2}}{\int }_{u=t}^{T}f(t-u)H_1{(X,u)}^{2}du=\frac{1}{2}\frac{{\beta }^2}{\gamma {\sigma }_{S^2}}{\int }_{u=t}^T{H}_2(X,u-t)H_1{(X,u)}^{2}du$$ (67)

The optimal solution (36) results from assembling (56), (57), (65), and (67). The optimal policy is given by (47):

$${\alpha }^{\ast }=\frac{1}{\sigma F}\left(\frac{\partial F}{\partial Z}XZ-\frac{\partial F}{\partial X}{\sigma }_x\right)$$ (68)

Fig. 14.

Optimal Infected. See Table 3 for parameter values.