Taming the spread of an epidemic by lockdown policies

Abstract

We study the problem of a policymaker who aims at taming the spread of an epidemic while minimizing its associated social costs. The main feature of our model lies in the fact that the disease’s transmission rate is a diffusive stochastic process whose trend can be adjusted via costly confinement policies. We provide a complete theoretical analysis, as well as numerical experiments illustrating the structure of the optimal lockdown policy. In all our experiments the latter is characterized by three distinct periods: the epidemic is first let to freely evolve, then vigorously tamed, and finally a less stringent containment should be adopted. Moreover, the optimal containment policy is such that the product “reproduction number $\times$ percentage of susceptible” is kept after a certain date strictly below the critical level of one, although the reproduction number is let to oscillate above one in the last more relaxed phase of lockdown. Finally, an increase in the fluctuations of the transmission rate is shown to give rise to an earlier beginning of the optimal lockdown policy, which is also diluted over a longer period of time.

1. Introduction

During the current Covid-19 pandemic, policymakers are dealing with the trade-off between safeguarding public health and damming the negative economic impact of severe lockdowns. The fight against the virus is made especially hard by the absence of a vaccination and the consequent random horizon of any policy, as well as by the extraordinariness of the event. In particular, the lack of data from the past, the difficulty of rapidly and accurately tracking infected, and super-spreading events such as mass gatherings, give rise to a random behavior of the transmission rate/reproduction number of the virus (see, e.g., Hotz et al., 2020 1 ). In this paper we propose and study a model for the optimal containment of infections due to an epidemic in which both the time horizon and the transmission rate of the disease are stochastic.

In the last months, the scientific literature experienced an explosion in the number of works where the statistical analysis and the mathematical modeling of epidemic models is considered, as well as the economic and social impact of lockdown policies is investigated. A large bunch of papers provides numerical studies related to the Covid-19 epidemics in the setting of classical epidemic models or of generalization of them. Among many others, we refer to Alvarez et al. (2020), that studies numerically optimal containment policies in the context of a Susceptible–Infected–Recovered (SIR) model (cf. Kermack and McKendrick, 1927); Kantner (2020) which also allows for seasonal effects; Toda (2020), which estimates the transmission rate in various countries for a SIR model with given and fixed transmission rate; Aspri et al. (2020), which combines a careful numerical study with an elegant theoretical study of optimal lockdown policies in the SEAIRD (susceptible (S), exposed (E), asymptomatic (A), infected (I), recovered (R), deceased (D)) model; Bayraktar et al. (2020), where a detailed numerical analysis is developed for a SIR model of the Covid-19 pandemic in which herd immunity, behavior-dependent transmission rates, remote workers, and indirect externalities of lockdown are explicitly considered; Acemoglu et al. (2020), where – in the context of a multi-group SIR model – it is investigated the effect of lockdown policies which are targeted to different social groups (especially, the “young”, the “middle-aged” and the “old”); Gollier (2020), in which a multi-risk SIR model with heterogeneous citizens is calibrated on the Covid-19 pandemic in order to study the impact on incomes and mortality of age-specific confinements and Polymerase chain reaction (PCR) tests; Favero et al. (2020), which calibrates and tests a SEIRD model (susceptible (S), exposed (E), infected (I), recovered (R), deceased (D)) of the spread of Covid-19 in an heterogeneous economy where different age and sectors are related to distinct risks.

A theoretical study of the optimal confinement policies in epidemic models is usually challenging because of the nonlinear structure of the underlying dynamical system. The first results on a control-theoretic approach to confinement policies are perhaps those presented in Chapter 4 of Behncke (2000), where it is shown that the optimal policy depends only on the shadow price difference between infected and susceptible. In the context of an optimal timing problem, Huberts and Thijssen (2020) use a continuous-time Markov chain model to study the value and optimal exercise decision of two (sequential) options: the option to intervene on the epidemic and, after intervention has started, the option to end the containment policies. Control-theoretic analysis is also presented recently (Kruse and Strack, 2020, Miclo et al., 2020). In Miclo et al. (2020) the authors study a deterministic SIR model in which the social planner acts in order to keep the transmission rate below its natural level with the ultimate aim not to overwhelm the national health-care system. The minimization of a social cost functional is instead considered in Kruse and Strack (2020), in the context of a deterministic SIR model over a finite time-horizon. The resulting control problem is tackled via the Pontryagin maximum principle and then a thorough numerical illustration is also provided.

Inspired by the deterministic problems of Kruse and Strack (2020) and Miclo et al. (2020) (see also Acemoglu et al., 2020, Alvarez et al., 2020, among others), and motivated by the need of incorporating random fluctuations in the disease’s transmission rate, in this paper we consider a stochastic control-theoretic version of the classical SIR model of Kermack and McKendrick (1927). A population with finite size is divided into three different groups: healthy people who are susceptible to the disease, infected individuals, and people who have recovered (and are not anymore susceptible) or dead. However, differently to the classical SIR model, we suppose that disease’s transmission rate is time-dependent and stochastic. In particular, it evolves as a general diffusion process whose trend can be adjusted by a social planner through policies like social restrictions and lockdowns. The randomness in the transmission rate is modeled by a Wiener process representing all those factors affecting the transmission rate and that are not under the direct control of the regulator. The social planner faces the trade-off between the expected social and economic costs (e.g., drops in the gross domestic product) arising from severe restrictions and the expected costs induced by the number of infections that – if uncontrolled – might strongly impact the national health-care system and, more in general, on the social well-being. The social planner aims at minimizing those total expected costs up to the time at which a vaccination against the disease is discovered. In our model, such a time is also random and independent of the Wiener process.

We provide a complete theoretical study of our model by showing that the minimal cost function (value function) is a classical twice-continuously differentiable solution to its corresponding Hamilton–Jacobi–Bellman (HJB) equation, and by identifying an optimal control in feedback form.2 From a technical point of view, the main difference between the models in Acemoglu et al., 2020, Alvarez et al., 2020, Bayraktar et al., 2020, Kruse and Strack, 2020 and Miclo et al. (2020) and ours, is that we deal with a stochastic version of the SIR model, instead of a deterministic one. As a matter of fact, in the aforementioned works the transmission rate is a deterministic control variable, while it is a controlled stochastic state variable in our paper. Moreover, our formulation is also different from that of other stochastic SIR models where the random transmission rate is chosen in such a way that only the levels of infected and susceptible people become affected by noise, with the transmission rate itself not being a state variable (see, e.g., Jiang and Yu, 2011, Tornatore et al., 2005 and references therein). To the best of our knowledge, ours is the first work considering the transmission rate as a diffusive stochastic state variable and providing the complete theoretical analysis of the resulting control problem.

In addition to its theoretical value, the determination of an optimal control in feedback form allows us to perform numerical experiments aiming at showing some implications of our model. For the numerical analysis we specialize the dynamics of the transmission rate, that we take to be mean-reverting and bounded between $0$ and some $\gamma >0$ (cf. (4.1)). In this case study, the containment policies employed by the social planner have the effect of modifying the long-run mean of the transmission rate, towards which the process converges at an exponential rate. Moreover, we take a separable social cost function (cf. (4.2)). This is quadratic both in the regulator’s effort and in the percentage of infected people.

An interesting effect which is in fact common to all our numerical experiments is that the optimal lockdown policy is characterized by three distinct periods. In the first phase it is optimal to let the epidemic freely evolve, then the social restrictions should be stringent, and finally should be gradually relaxed in a third period. We also investigate which is the effect of the maximal level $L$ of allowed containment measures (i.e., the lockdown policy can take values in $[0,L]$) on the final percentage of recovered, which in fact turns out to be decreasing with respect to $L$. This then suggests that the case $L=1$ – which leads in a shorter period to the definitive containment of the disease with the smallest percentage of final recovered — might be thought of as optimal in the trade-off between social costs and final number of recovered.

We observe that if the epidemic spread is left uncontrolled, then its reproduction number ${\left({\mathcal{R}}_t\right)}_t$ fluctuates around 1.8 and the final percentage of recovered (i.e. the total percentage of infected during the disease) is approximately 72% of the society after circa 7 months (in all our simulations the initial infected were 1% of the population). On the other hand, when $L=1$, under the optimal policy we have a relative reduction of circa 30% of the total percentage of recovered individuals, and the reproduction number drops below 0.6 in the period of severe lockdown (circa 60 days). Moreover, the optimal containment is such that the so-called “herd immunity” is reached as the product ${\mathcal{R}}_t{S}_t$ (reproduction number $\times$ percentage of susceptible) becomes strictly smaller than the critical level of one, even if ${\mathcal{R}}_t$ oscillates at around 1.7 in the last more relaxed phase of lockdown. Finally, we observe that an increase of the fluctuations of the transmission rate $\beta$ have the effect of anticipating the beginning of the lockdown policies, of diluting the actions over a longer period, and of keeping a larger level of containment in the long run. This can be explained by thinking that an higher uncertainty in the transmission rate induces the policymaker to act earlier and over a longer period in order to prevent positive larger shocks of $\beta$.

The rest of the paper is organized as follows: In Section 2.1 we set up the model and the social planner problem. In Section 3 we develop the control-theoretic analysis and provide the regularity of the minimal cost function and an optimal control in feedback form. In Section 4 we present our numerical examples, while concluding remarks are made in Section 5. Finally, the Appendix collects the proof of some technical results needed in Section 3.

2. Problem formulation

2.1. The stochastic controlled SIR model

We model the spread of the infection by relying on a generalization of the classical SIR model that dates back to the work by Kermack and McKendrick (1927). The society has population $N$ and it consists of three different groups. The first group is formed by those people who are healthy, but susceptible to the disease; the second group contains those who are infected, while the last cohort consists of those who are recovered or dead. In line with the classical SIR model, we assume that, once recovered, an individual stays healthy for ever. We denote by $S_t$ the percentage of individuals who are susceptible at time $t\ge 0$, by $I_t$ the percentage of infected, and by $R_t$ the fraction of recovered or dead. Clearly, $S_t+I_t+R_t=1$ for all $t\ge 0$.

The fraction of infected people grows at a rate which is proportional to the fraction of society that it is still susceptible to the disease. In particular, letting ${\beta }_t$ be the instantaneous transmission rate of the disease, during an infinitesimal interval of time $\mathrm{d}t$, each infected individual generates ${\beta }_t{S}_t$ new infected individuals. It thus follows that the percentage of healthy individuals that get infected within $\mathrm{d}t$ units of time is $I_t{\beta }_t{S}_t$.

Notice that the instantaneous transmission rate ${\beta }_t$ measures the disease’s rate of infection, as well as the average number of contacts per person per time. In this regard, ${\beta }_t$ can be thus influenced by a social planner via policies that effectively cap the social interaction, like social distancing and lockdown.

During an infinitesimal interval of time $\mathrm{d}t$, the fraction of infected is reduced by $\alpha I_t$, since infected either recover from the disease, or die because of it at a rate $\alpha >0$.

According to the previous considerations, the dynamics of $S_t$ and $I_t$ can be thus written as

$$\mathrm{d}{S}_t=-{\beta }_t{S}_t{I}_t\mathrm{d}t,t>0,S_0=x,$$ (2.1)

and

$$\mathrm{d}{I}_t=\left({\beta }_t{S}_t{I}_t-\alpha I_t\right)\mathrm{d}t,t>0,I_0=y,$$ (2.2)

where $(x,y)\in {(0,1)}^2$ are given initial values such that3 $x+y\in (0,1)$.

Notice that for any $t\ge 0$, and for any choice of ${\left({\beta }_t\right)}_t$ we can write

$$S_t=x{e}^{-{\int }_0^t{\beta }_u{I}_u\mathrm{d}u}\text{and}{I}_t=y{e}^{-\alpha t+{\int }_0^t{\beta }_u{S}_u\mathrm{d}u},$$ (2.3)

and therefore $S_t>0$ and $I_t>0$ for all $t\ge 0$. Moreover, summing up (2.1), (2.2) we have $\mathrm{d}(S_t+I_t)=-\alpha I_t<0$ for all $t>0$, which then implies that $S_t+I_t<1$ for all $t\ge 0$.

We depart from the classical SIR model by assuming that the transmission rate ${\beta }_t$ is time-varying, stochastic, and may be controlled. More precisely, we let $(\Omega ,\mathcal{F},\mathbb{F}≔{\left({\mathcal{F}}_t\right)}_t,\mathsf{P})$ be a complete filtered probability space with filtration $\mathbb{F}$ satisfying the usual conditions, and we define on that a one-dimensional Brownian motion ${\left(W_t\right)}_t$. For a given and fixed $L\ge 0$, and for any ${\left({\xi }_t\right)}_t$ belonging to

$$\mathcal{A}≔\{\xi :\Omega \times [0,\infty )\to [0,L],{\left({\xi }_t\right)}_t$$

$$\mathbb{F}-\text{progressively measurable}\},$$

we assume that the transmission rate evolves according to the stochastic differential equation

$$\mathrm{d}{\beta }_t=b({\beta }_t,{\xi }_t)\mathrm{d}t+\sigma \left({\beta }_t\right)\mathrm{d}{W}_t,t>0,{\beta }_0=z>0.$$ (2.4)

The process ${\left({\xi }_t\right)}_t$ influences the trend of the transmission rate and it should be interpreted as any effort devoted by the social planner to the decrease of the transmission rate. In this sense, $\xi =0$ corresponds to the case of no effort done to decrease the disease, whereas the case $\xi =L$ corresponds to the maximal effort. To fix ideas, ${\xi }_t$ may represent a percentage of social/working lockdown at time $t$ and $L$ corresponds to the maximal implementable value of such lockdown (e.g. 60%, etc.). On the other hand, the Brownian motion ${\left(W_t\right)}_t$ models any shock affecting the transmission rate and which is not under the control of the social planner.

Regarding the dynamics of ${\left({\beta }_t\right)}_t$ we make the following standing assumption.

Assumption 2.1

• (i)For every $\xi \in \mathcal{A}$, there exists a unique strong solution to (2.4) and it lies in an open interval $\mathcal{I}\subseteq (0,\infty )$.
• (ii)
  $b:\mathcal{I}\times [0,L]\to \mathbb{R}$ is bounded, infinitely many times continuously differentiable with respect to its first argument, and has bounded derivatives of any order; that is, there exists $K_b>0$ such that

  $$\underset{n\in \mathbb{N}}{sup}\underset{(z,\xi )\in \mathcal{I}\times [0,L]}{sup}|\frac{{\partial }^n}{\partial z^n}b(z,\xi )|\le K_b.$$
• (iii)
  $\sigma :\mathcal{I}\to (0,\infty )$ is bounded, infinitely many times continuously differentiable with respect to its first argument, and has bounded derivatives of any order; that is, there exists $K_{\sigma }>0$ such that

  $$\underset{n\in \mathbb{N}}{sup}\underset{z\in \mathcal{I}}{sup}\left|{\sigma }^{\left(n\right)}\left(z\right)\right|\le K_{\sigma }.$$

A reasonable dynamics of the transmission rate ${\left({\beta }_t\right)}_t$ is the mean-reverting

$$\mathrm{d}{\beta }_t=\vartheta \left(\widehat{\beta }\left(L-{\xi }_t\right)-{\beta }_t\right)\mathrm{d}t+\sigma {\beta }_t(\gamma -{\beta }_t)\mathrm{d}{W}_t,t>0,{\beta }_0=z\in (0,\gamma ),$$ (2.5)

for some $\vartheta ,\gamma ,\sigma >0$, $\widehat{\beta }\in (0,\gamma )$. In this case, in can be shown that $0$ and $\gamma$ are unattainable by the diffusion ${\left({\beta }_t\right)}_t$, which then takes values in the interval $\mathcal{I}=(0,\gamma )$ for any $t\ge 0$. The level $\widehat{\beta }$ can be seen as the natural transmission rate of the disease, towards which the transmission rate reverts at rate $\vartheta$ when $\xi \equiv 0$. Finally, the level $\gamma$ is the maximal possible transmission rate of the disease, and $\sigma$ is a measure of the fluctuations of ${\left({\beta }_t\right)}_t$ around $\widehat{\beta }$. We will employ this dynamics in our numerical illustrations (cf. Section 4). Notice that dynamics (2.5) fulfills all the requirements of Assumption 2.1; this is shown, for the sake of completeness, in Proposition A.3 in the Appendix. Moreover, if ${\xi }_t\equiv L$, then the transmission rate defined through (2.5) reaches $0$ asymptotically, as its drift is negative and its diffusion coefficient stays bounded. Hence, under the maximal lockdown policy, the disease is asymptotically eradicated.

Remark 2.2

A modeling feature that needs some discussion regards the nature of the control rule in (2.5). In our formulation, the policymaker adjusts $\xi$ continuously over time with the aim of decreasing the trend of the transmission rate. However, motivated by the real-world strategies employed during the Covid-19 crisis, one can very well imagine a model where regulatory constraints are introduced once the reproduction number ${\mathcal{R}}_t={\beta }_t∕\alpha$ becomes larger than a certain value, say ${\mathcal{R}}^{\star }$. Within this setting, a natural question would be: which is the optimal ${\mathcal{R}}^{\star }$ and the optimal size of interventions? A possible answer to this question could be found by proposing a model where the policymaker instantaneously reduces the level of $\beta$ via lockdown policies and faces proportional and/or fixed costs for its actions. This would gives rise to a singular or impulsive stochastic control problem; see Alvarez (2001), Ferrari (2018) and Belak et al. (2017) and references therein. Given the underlying multi-dimensional setting, we expect that the optimal trigger level ${\mathcal{R}}^{\star }$ would be a function of the current values of $(S_t,I_t)$. However, the proof of such a conjecture would require the thorough study of acomplex (non convex) three-dimensional degenerate singular/impulse stochastic control problem that clearly requires techniques different from those employed in this work.

2.2. The social planner problem

The epidemic generates social costs, that we assume to be increasing with respect to the fraction of the population that is infected. These costs might arise because of lost gross domestic product (GDP) due to inability of working, because of an overstress of the national health-care system etc. The social planner thus employs policies ${\left({\xi }_t\right)}_t$ in the form, e.g., of social distancing or lockdown in order to adjust the growth rate of the transmission rate $\beta$, with the aim of effectively flattening the curve of the infected percentage of the society. Such actions however come with a cost, which increases with the amplitude of the effort. Assuming that a vaccination against the disease is discovered at a random time $\tau$ exponentially distributed with parameter ${\lambda }_o>0$ and independent of ${\left(W_t\right)}_t$ 4 (see also Remark 2.4), the social planner aims at solving

$$\underset{\xi \in \mathcal{A}}{inf}\mathsf{E}\left[{\int }_0^{\tau }{e}^{-\delta t}C\left(I_t,{\xi }_t\right)\mathrm{d}t\right].$$ (2.6)

Here, $\delta \ge 0$ measures the social planner’s time preferences, and $C:[0,1]\times [0,L]\to [0,\infty )$ is a running cost function measuring the negative impact of the disease on the public health as well as the economic/social costs induced by lockdown policies. The following requirements are satisfied by $C$.

Assumption 2.3

• (i)$(y,\xi )\mapsto C(y,\xi )$ is convex and continuous on $[0,1]\times [0,L]$.
• (ii)For any $y\in [0,1]$ we have that $\xi \mapsto C(y,\xi )$ is nondecreasing.
• (iii)For any $\xi \in [0,L]$ we have that $y\mapsto C(y,\xi )$ is nondecreasing.
• (iv)
  There exists $K>0$ such that for any $\xi \in [0,L]$ we have that

  $$|C(y,\xi )-C(y^{\prime },\xi )|\le K|y-y^{\prime }|,\forall (y,y^{\prime })\in {[0,1]}^2.$$
• (v)
  $y\mapsto C(y,\xi )$ is semiconcave5 on $[0,1]$, uniformly with respect to $\xi \in [0,L]$; that is, there exists $K>0$ such that for any $\xi \in [0,L]$ and any $\mu \in [0,1]$ one has

  $$\mu C(y,\xi )+(1-\mu )C(y^{\prime },\xi )-C\left(\mu y+(1-\mu )y^{\prime },\xi \right)$$

  $$\le K\mu (1-\mu ){|y-y^{\prime }|}^2,\forall (y,y^{\prime })\in {[0,1]}^2.$$

Without loss of generality, we also take $C(0,0)=0$. Convexity of $y\mapsto C(y,\xi )$ captures the fact that the social costs from the disease might be higher if a large share of the population is infected since, for example, the social health-care system is overwhelmed. The fact that $\xi \mapsto C(y,\xi )$ is convex describes that marginal costs of actions are increasing because, e.g., an additional lockdown policy might have a larger impact on an already stressed society. Finally, the Lipschitz and semiconcavity property of $C(\cdot ,\xi )$ are technical requirements that will be important in the next section.

An application of Fubini’s theorem, employing the independence of $\tau$ and ${\left(W_t\right)}_t$, allows to rewrite the problem defined in (2.6) as

$$\underset{\xi \in \mathcal{A}}{inf}\mathsf{E}\left[{\int }_0^{\tau }{e}^{-\delta t}C\left(I_t,{\xi }_t\right)\mathrm{d}t\right]=\underset{\xi \in \mathcal{A}}{inf}\mathsf{E}\left[{\int }_0^{\infty }{e}^{-\lambda t}C\left(I_t,{\xi }_t\right)\mathrm{d}t\right],$$ (2.7)

where $\lambda ≔{\lambda }_o+\delta$.

Remark 2.4

The assumption that a vaccination against the disease is discovered at an exponential random time $\tau$, independent of ${\left(W_t\right)}_t$, has the technical important effect of leading to a time-homogeneous social planner problem (cf. (2.7)). From a modeling point of view, such a requirement is clearly debatable, as it presupposes that the decision maker does not take into account the scientific progress in the epidemic’s treatment. In order to take care of this, we now propose an alternative more realistic formulation which, however, comes at the cost of substantially increasing the mathematical complexity of the social planner problem.

Suppose that the social planner has full information about the current technological level $Q_t$ achieved in the disease’s treatment and assume, for example, that this evolves according to the SDE:

$$\mathrm{d}{Q}_t=\mu \left(Q_t\right)\mathrm{d}t+\eta \left(Q_t\right)\mathrm{d}{B}_t,Q_0=q\in {\mathbb{R}}_{+},$$

for suitable $\mu$ and $\eta$, and for a standard Brownian motion ${\left(B_t\right)}_t$ independent of ${\left(W_t\right)}_t$. The process ${\left(B_t\right)}_t$ models all the exogenous shocks affecting the technological achievements (e.g., new scientific discoveries in related fields), while $\mu$ measures the instantaneous trend of the research. Define then a continuous-time Markov chain ${\left(M_t\right)}_t$ with two states, $0$ and $1$, where $0$ means that the vaccination is not available and $1$ that a treatment has been instead found. We assume that $1$ is an absorbing state and that the Markov chain has transition rate from state $0$ to state $1$ given by ${\left(\lambda (t,Q_t)\right)}_t$. Here, $\lambda :{\mathbb{R}}_{+}\times {\mathbb{R}}_{+}\mapsto {\mathbb{R}}_{+}$ is such that ${\Lambda }_t≔{\int }_0^t\lambda (s,Q_s)\mathrm{d}s<\infty$, a.s. for any $t\ge 0$, and it is nondecreasing in its second argument. This latter condition clearly means that the larger the technological level is, the faster the disease is treated.

Within this setting, the problem can be then still be written as

$$\underset{\xi \in \mathcal{A}}{inf}\mathsf{E}\left[{\int }_0^{\tau }{e}^{-\delta t}C\left(I_t,{\xi }_t\right)\mathrm{d}t\right],$$

where ${\left(I_t\right)}_t$ and ${\left({\xi }_t\right)}_t$ are as defined above in this section, but

$$\tau ≔inf\{t\ge 0:M_t=1\}.$$

The independence of $Q$ with respect to $W$ then leads to the equivalent formulation

$$V(x,y,z,q)≔\underset{\xi \in \mathcal{A}}{inf}\mathsf{E}\left[{\int }_0^{\infty }{e}^{-\delta t-{\Lambda }_t}C\left(I_t,{\xi }_t\right)\mathrm{d}t\right],$$

which defines a four-dimensional stochastic control problem. Clearly, this problem is much more challenging than (2.7) and its analysis, requiring different techniques and results, is left for future research.

Another interesting future work might concern an extension of the previous model in which the social planner can also increase the technological level $Q$ by supporting the research of a vaccination. Assuming that such an investment comes at proportional cost, this problem can be modeled in terms of an intricate stochastic control problem where the transition rate $\lambda$ of the Markov chain is controlled through a singular control.

In order to tackle Problem (2.7) with techniques from dynamic programming, it is convenient to keep track of the initial values of ${(S_t,I_t,{\beta }_t)}_t$. We therefore set

$$\mathcal{O}≔\left\{(x,y,z)\in {\mathbb{R}}^3:(x,y)\in {(0,1)}^2,x+y<1,z\in \mathcal{I}\right\},$$

and, when needed, we stress the dependency of $(S_t,I_t,{\beta }_t)$ with respect to $(x,y,z)\in \mathcal{O}$ and $\xi \in \mathcal{A}$ by writing $(S_t^{x,y,z;\xi },I_t^{x,y,z;\xi },{\beta }_t^{z;\xi })$. Indeed, due to (2.3) and the autonomous nature of (2.4), we have that $S_t$ and $I_t$ depend on $(x,y,z)$ and on $\xi$ through ${\beta }_t$, while ${\beta }_t$ depends only on $z$ and directly on $\xi$. We shall also simply set $(S_t^{x,y,z},I_t^{x,y,z},{\beta }_t^z)≔(S_t^{x,y,z;0},I_t^{x,y,z;0},{\beta }_t^{z;0})$ to denote the solutions to (2.1), (2.2), and (2.4) when $\xi \equiv 0$.

Then, for any $(x,y,z)\in \mathcal{O}$, we introduce the problem’s value function

$$V(x,y,z)≔\underset{\xi \in \mathcal{A}}{inf}\mathsf{E}\left[{\int }_0^{\infty }{e}^{-\lambda t}C\left(I_t^{x,y,z;\xi },{\xi }_t\right)\mathrm{d}t\right].$$ (2.8)

The latter is well defined given that $C$ is nonnegative. In the next section we will show that $V$ solves the corresponding dynamic programming equation in the classical sense, and we also provide an optimal control in feedback form.

3. The solution to the social planner problem

We introduce the differential operator $\mathcal{L}$ acting on functions belonging to the class $C^{1,1,2}\left({\mathbb{R}}^3\right)$:

$$\left(\mathcal{L}\phi \right)(x,y,z)≔xyz\left({\phi }_y-{\phi }_x\right)(x,y,z)-\alpha y{\phi }_y(x,y,z)+\frac{1}{2}{\sigma }^2\left(z\right){\phi }_{zz}(x,y,z).$$ (3.1)

Next, for any $(y,z,p)\in (0,1)\times \mathcal{I}\times \mathbb{R}$, define

$$C^{\star }(y,z,p)≔\underset{\xi \in [0,L]}{inf}\left(C(y,\xi )+b(z,\xi )p\right),$$ (3.2)

which is continuous on $[0,1]\times \mathcal{I}\times \mathbb{R}$. Indeed, byAssumptions 2.1-(ii) and 2.3-(iv), there exists a constant $\overline{K}>0$ such that

$$|C^{\star }(y^{\prime },z^{\prime },p^{\prime })-C^{\star }(y,z,p)|\le \underset{\xi \in [0,L]}{sup}(|C(y^{\prime },\xi )-C(y,\xi )|+|b(z^{\prime },\xi )-b(z,\xi )|\left|p^{\prime }\right|+\left|b(z,\xi )\right||p^{\prime }-p|)\le \overline{K}\left(|y^{\prime }-y|+|z^{\prime }-z|\left|p^{\prime }\right|+|p^{\prime }-p|\right).$$

By the dynamic programming principle, we expect that $V$ should solve (in a suitable sense) the Hamilton–Jacobi–Bellman (HJB) equation

$$\lambda v(x,y,z)=\left(\mathcal{L}v\right)(x,y,z)+C^{\star }(y,z,v_z(x,y,z)),(x,y,z)\in \mathcal{O}.$$ (3.3)

In order to show that $V$ indeed solves (3.3) in the classical sense, we start with the following important preliminary results. Their proofs are standard in the literature of stochastic control (see, e.g., Pham, 2009, Yong and Zhou, 1999), upon employing Assumption 2.1, Assumption 2.3.

Proposition 3.1

There exists $K>0$ such that, for each $\mathbf{q}≔(x,y,z),{\mathbf{q}}^{\prime }≔(x^{\prime },y^{\prime },z^{\prime })\in \mathcal{O}$

• (i)$0\le V\left(\mathbf{q}\right)\le K$and$|V\left(\mathbf{q}\right)-V\left({\mathbf{q}}^{\prime }\right)|\le K|\mathbf{q}-{\mathbf{q}}^{\prime }|$; i.e., $V$ is bounded and Lipschitz continuous on $\mathcal{O}$ ;
• (ii)
  for any$\mu \in [0,1]$and for some$K>0$

  $$\mu V\left(\mathbf{q}\right)+(1-\mu )V\left({\mathbf{q}}^{\prime }\right)-V\left(\mu \mathbf{q}+(1-\mu ){\mathbf{q}}^{\prime }\right)\le K\mu (1-\mu ){|\mathbf{q}-{\mathbf{q}}^{\prime }|}^2;$$

  i.e., $V$ is semiconcave on $\mathcal{O}$ .

Moreover, $V$ is a viscosity solution to the HJB equation (3.3) .

Proof

The first claim of (i) above follows from the fact that $C$ is nonnegative and bounded on ${[0,1]}^2$; the second claim of (i) is due to Proposition 3.1 in Yong and Zhou (1999), whose proof can be easily adapted to our stationary setting. Analogously, the semiconcavity property of (ii) can be obtained by arguing as in Proposition 4.5 of Yong and Zhou (1999). Finally, Theorem 5.2 of Yong and Zhou (1999) (again, easily adapted to our stationary setting) or Proposition 4.3.2-(2) of Pham (2009) leads to the viscosity property. □

The semiconcavity of $V$, together with the fact that $V$ solves the HJB equation (3.3) in the viscosity sense, yields the following directional regularity result.

Proposition 3.2

$V_z$ exists continuous on $\mathcal{O}$ .

Proof

Let $(x,y,z)\in \mathcal{O}$. By semiconcavity of $V$, there exists the left and right derivatives of $V$ along the direction $z$ at $(x,y,z)$ that we denote, respectively, by $V_z^{-}(x,y,z),V_z^{+}(x,y,z)$. Moreover, again by semiconcavity, we have the inequality $V_z^{-}(x,y,z)\ge V_z^{+}(x,y,z)$. Assuming, by contradiction, that $V$ is not differentiable with respect to $z$ at $(x,y,z)$ means assuming that $V_z^{-}(x,y,z)>V_z^{+}(x,y,z)$. Hence, we can apply Lemma A.1 in the Appendix and find a sequence of functions ${\left({\tilde{\phi }}^n\right)}_n\subset C^2\left(\mathcal{O}\right)$ such that

$$\left\{\begin{array}{c}{\tilde{\phi }}^n(\overline{x},\overline{y},\overline{z})=V(\overline{x},\overline{y},\overline{z}),\hfill \\ {\tilde{\phi }}^n\ge V\text{in a neighborhood of}(\overline{x},\overline{y},\overline{z}),\hfill \\ \left|D{\tilde{\phi }}^n(\overline{x},\overline{y},\overline{z})\right|\le \tilde{L}<\infty ,\hfill \\ {\tilde{\phi }}_{zz}^n(\overline{x},\overline{y},\overline{z})\stackrel{n\to \infty }{⟶}-\infty .\hfill \end{array}\right)$$ (3.4)

Then, the viscosity subsolution property of $V$ (cf. Proposition 3.1) yields

$$\lambda V(\overline{x},\overline{y},\overline{z})\le \left(\mathcal{L}{\tilde{\phi }}^n\right)(\overline{x},\overline{y},\overline{z})+C^{\star }(\overline{y},\overline{z},{\tilde{\phi }}_z^n(\overline{x},\overline{y},\overline{z})).$$

Taking the limit as $n\to \infty$ and using (3.4) we get a contradiction. We have thus proved that $V_z$ exists at each arbitrary $(x,y,z)\in \mathcal{O}$.

Now we show that $V_z$ is continuous. Take a sequence ${\left({\mathit{q}}^n\right)}_n\subset \mathcal{O}$ such that ${\mathit{q}}^n\to \mathit{q}\in \mathcal{O}$, and let ${\mathit{\eta }}^n=({\eta }_x^n,{\eta }_y^n,{\eta }_z^n)\in D^{+}V\left({\mathit{q}}^n\right)$, the latter being nonempty due to the semiconcavity of $V$. Since $V_z$ exists at each point of $\mathcal{O}$, we have ${\eta }_z^n=V_z\left({\mathit{q}}^n\right)$. Since $V$ is semiconcave, the supergradient $D^{+}V$ is locally bounded as a set-valued map, and therefore there exists a subsequence ${\left({\mathit{q}}^{n_k}\right)}_k$ such that ${\mathit{\eta }}^{n_k}\to \mathit{\eta }=({\eta }_x,{\eta }_y,{\eta }_z)$. By Cannarsa and Sinestrari (2014, Prop. 3.3.4-(a)), we have $\mathit{\eta }\in D^{+}V\left(\mathit{q}\right)$, and again, since $V_z$ exists, we have ${\eta }_z=V_z\left(\mathit{q}\right)$. Hence, we have proved that from any sequence ${\left({\mathit{q}}^n\right)}_n\subset \mathcal{O}$ converging to $\mathit{q}$, we can extract a subsequence ${\left({\mathit{q}}^{n_k}\right)}_k\subset \mathcal{O}$ such that $V_z\left({\mathit{q}}^{n_k}\right)\to V_z\left(\mathit{q}\right)$. By usual arguments on subsequences, the claim follows.  □

We can now prove the main theoretical result of our paper, which ensures that $V$ is actually a classical solution to the HJB equation (3.3). In turn, this provides a way to construct an optimal control in feedback form.6

Theorem 3.3

The following hold:

• (i) $V\in C^2\left(\mathcal{O}\right)$ and solves the HJB equation (3.3) in the classical sense.
• (ii)
  Let

  $$\widehat{\xi }(x,y,z)≔\underset{\xi \in [0,L]}{argmin}\left(C(y,\xi )-b(z,\xi )V_z(x,y,z)\right),(x,y,z)\in \mathcal{O}.$$ (3.5)

  If the system of equations 7

  $$\left\{\begin{array}{c}\mathrm{d}{S}_t=-{\beta }_t{S}_t{I}_t\mathrm{d}t,S_0=x,\hfill \\ \mathrm{d}{I}_t=\left({\beta }_t{S}_t{I}_t-\alpha I_t\right)\mathrm{d}t,I_0=y,\hfill \\ \mathrm{d}{\beta }_t=b({\beta }_t,\widehat{\xi }(S_t,I_t,{\beta }_t))\mathrm{d}t+\sigma \left({\beta }_t\right)\mathrm{d}{W}_t,{\beta }_0=z,\hfill \end{array}\right)$$ (3.6)

  admits a unique strong solution ${(S_t^{\star },I_t^{\star },{\beta }_t^{\star })}_t$ , then the control

  $${\xi }_t^{\star }≔\widehat{\xi }\left(S_t^{\star },I_t^{\star },{\beta }_t^{\star }\right),$$ (3.7)

  is optimal for (2.8) and ${\left({\beta }_t^{\star }\right)}_t$ is the optimally controlled transmission rate; that is,

  $$V(x,y,z)=\mathsf{E}\left[{\int }_0^{\infty }{e}^{-\lambda t}C\left(I_t^{\star },{\xi }_t^{\star }\right)\mathrm{d}t\right].$$

Proof

Proof of (i) — Step 1. Recall (3.2) and define $F(x,y,z)≔C^{\star }(y,z,V_z(x,y,z))$. Due to Proposition 3.2 and the continuity of $C^{\star }$ on $(0,1)\times \mathcal{I}\times \mathbb{R}$, we have that $F$ is continuous on $\mathcal{O}$. Moreover, since $C$ is bounded on $[0,1]\times [0,L]$, $V_z$ is bounded on $\mathcal{O}$ by Proposition 3.1-(i), and $b(\cdot ,\xi )$ is bounded (cf. Assumption 2.1-(ii)), there exists $K>0$ such that

$$\left|F(x,y,z)\right|\le K,\forall (x,y,z)\in \mathcal{O}.$$ (3.8)

Set now

$$v(x,y,z)≔\mathsf{E}\left[{\int }_0^{\infty }{e}^{-\lambda t}F\left(S_t^{x,y,z},I_t^{x,y,z},{\beta }_t^z\right)\mathrm{d}t\right],(x,y,z)\in \mathcal{O}.$$ (3.9)

Although not uniformly elliptic, the differential operator $\mathcal{L}$ defined in (3.1) is hypoelliptic, meaning that the so-called Hörmander’s condition is satisfied (cf. the proof of Proposition A.2 in the Appendix and Eq. (A.4) therein). In fact, by Proposition A.2 in the Appendix, for any $\mathbf{q}≔(x,y,z)\in \mathcal{O}$ the (uncontrolled) process ${\left({\mathbf{Q}}_t^{\mathbf{q}}\right)}_t≔(S_t^{x,y,z},I_t^{x,y,z},{\beta }_t^z)$ admits a transition density $p(t,\mathbf{q},\cdot )$, $t>0$, which is absolutely continuous with respect to the Lebesgue measure in ${\mathbb{R}}^3$, infinitely many times differentiable, and satisfying the Gaussian estimates (A.2), (A.3). As a consequence, by Fubini’s theorem we can write

$$v(x,y,z)={\int }_0^{\infty }{e}^{-\lambda t}({\int }_{\mathcal{O}}F\left(x^{\prime },y^{\prime },z^{\prime }\right)\times p(t,x,y,z;x^{\prime },y^{\prime },z^{\prime })\mathrm{d}{x}^{\prime }\mathrm{d}{y}^{\prime }\mathrm{d}{z}^{\prime })\mathrm{d}t,$$

and recalling (3.8), and applying the dominated convergence theorem, one shows that $v\in C^2\left(\mathcal{O}\right)$.

For $(x,y,z)\in \mathcal{O}$, let now ${\tau }_n≔inf\{t\ge 0:\left|(S_t^{x,y,z},I_t^{x,y,z},{\beta }_t^z)\right|\ge n\}$, $n\in \mathbb{N}$, and notice that the strong Markov property yields

$$e^{-\lambda (t\wedge {\tau }_n)}v(S_{t\wedge {\tau }_n}^{x,y,z},I_{t\wedge {\tau }_n}^{x,y,z},{\beta }_{t\wedge {\tau }_n}^z)+{\int }_0^{t\wedge {\tau }_n}F\left(S_u^{x,y,z},I_u^{x,y,z},{\beta }_u^z\right)\mathrm{d}u$$

$$=\mathsf{E}\left[{\int }_0^{\infty }{e}^{-\lambda t}F\left(S_t^{x,y,z},I_t^{x,y,z},{\beta }_t^z\right)\mathrm{d}t|{\mathcal{F}}_{t\wedge {\tau }_n}\right].$$

Since $v\in C^2\left(\mathcal{O}\right)$, we can apply Itô’s formula to the first addend on the left-hand side of the latter, take expectations, observe that the stochastic integral has zero mean (by definition of ${\tau }_n$ and the fact that $v_x$ is continuous), and finally find

$$\mathsf{E}\left[{\int }_0^{t\wedge {\tau }_n}{e}^{-\lambda u}\left(\mathcal{L}v+F-\lambda v\right)\left(S_u^{x,y,z},I_u^{x,y,z},{\beta }_u^z\right)\mathrm{d}u\right]+v(x,y,z)=\mathsf{E}\left[{\int }_0^{\infty }{e}^{-\lambda t}F\left(S_t^{x,y,z},I_t^{x,y,z},{\beta }_t^z\right)\mathrm{d}t\right];$$ (3.10)

that is, by (3.9),

$$\mathsf{E}\left[{\int }_0^{t\wedge {\tau }_n}{e}^{-\lambda u}\left(\mathcal{L}v+F-\lambda v\right)\left(S_u^{x,y,z},I_u^{x,y,z},{\beta }_u^z\right)\mathrm{d}u\right]=0.$$

Dividing now both left and right-hand sides of the latter by $t$, invoking the (integral) mean-value theorem, letting $t\downarrow 0$, and using that $t\mapsto (S_t^{x,y,z},I_t^{x,y,z},{\beta }_t^z)$ is continuous, we find that $v$ is a classical solution to

$$\lambda \phi =\mathcal{L}\phi +F\text{on}\mathcal{O}.$$ (3.11)

Proof of (i) — Step 2. Let $(x,y,z)\in \mathcal{O}$, and ${\left({\mathcal{K}}_n\right)}_n$ be an increasing sequence of open bounded subsets of $\mathcal{O}$ such that ${\bigcup }_{n\in \mathbb{N}}{\mathcal{K}}_n=\mathcal{O}$. Defining the stopping time

$${\rho }_n≔inf\{t\ge 0:\left(\underset{t}{\overset{x,y,z}{S}},I_t^{x,y,z},{\beta }_t^z\right)\notin {\mathcal{K}}_n\},n\in \mathbb{N},$$

we set

$${\widehat{v}}_n(x,y,z)≔\mathsf{E}[{\int }_0^{{\rho }_n}F\left(S_u^{x,y,z},I_u^{x,y,z},{\beta }_u^z\right)\mathrm{d}u+e^{-\lambda {\rho }_n}V\left(S_{{\rho }_n}^{x,y,z},I_{{\rho }_n}^{x,y,z},{\beta }_{{\rho }_n}^z\right)].$$ (3.12)

If $(x,y,z)\notin {\mathcal{K}}_n$, then ${\widehat{v}}_n(x,y,z)=V(x,y,z)$ as ${\rho }_n=0$ a.s. Take then $(x,y,z)\in {\mathcal{K}}_n$. By the same arguments as in Step 1 and considering that $V$ is continuous on ${\mathcal{K}}_n$, the function ${\widehat{v}}_n$ is a solution to

$$\lambda \phi =\mathcal{L}\phi +F,\text{on}{\mathcal{K}}_n,\phi =V\text{on}\partial {\mathcal{K}}_n.$$ (3.13)

Since also $V$ is a viscosity solution to the same equation and since uniqueness of viscosity solution holds for such a problem (cf., e.g., Crandall et al., 1992), we have ${\widehat{v}}_n=V$ on ${\overline{\mathcal{K}}}_n$. Because ${\rho }_n\uparrow \infty$ for $n\uparrow \infty$ (as the boundary of $\mathcal{O}$ is unattainable for $\left(S_t^{x,y,z},I_t^{x,y,z},{\beta }_t^z\right)$), by taking limits as $n\uparrow \infty$ in (3.12) we find that

$$V(x,y,z)=\underset{n\uparrow \infty }{lim}{\widehat{v}}_n(x,y,z)=v(x,y,z),(x,y,z)\in \mathcal{O},$$

where the last equality follows by dominated convergence upon recalling that $V$ is bounded. But then $V=v$ on $\mathcal{O}$, and therefore $V\in C^2\left(\mathcal{O}\right)$ and solves (3.11) by Step 1. That is, $V$ is a classical solution to the HJB equation (3.3).

Proof of (ii). The optimality of (3.7) follows by a standard verification theorem based on an application of Itô’s formula and the proved regularity of $V$ (see, e.g., Chapter 3.5 in Pham, 2009). □

4. A case study with numerical illustrations

In this section we illustrate numerically the results of our model, with the aim of providing qualitative properties of the optimal containment policies in a case study.

We use the mean-reverting model for the dynamics of $\beta$, i.e. 

$$\mathrm{d}{\beta }_t=\vartheta \left(\widehat{\beta }\left(L-{\xi }_t\right)-{\beta }_t\right)\mathrm{d}t+\sigma {\beta }_t(\gamma -{\beta }_t)\mathrm{d}{W}_t,t>0,{\beta }_0=z\in (0,\gamma ),$$ (4.1)

for some $L,\vartheta ,\gamma ,\sigma >0$, $\widehat{\beta }\in (0,\gamma )$. Notice that such a choice of the dynamics of $\beta$ fulfills all the requirements of Assumption 2.1 (see Proposition A.3 in the Appendix). Moreover, we assume that the social planner has a quadratic cost function of the form

$$C(y,\xi )={\left(\frac{y}{\overline{y}}\right)}^2+\frac{1}{2}{\xi }^2.$$ (4.2)

The latter can be interpreted as a Taylor approximation of any smooth, convex, separable cost function with global minimum in $(0,0)$. In (4.2), $\overline{y}\in (0,1)$ represents, e.g., the maximal percentage of infected people that the health-care system can handle.

Notice that in this case for any $(x,y,z)\in \mathcal{O}$ one has (cf. (3.5))

$$\widehat{\xi }(x,y,z)=\left\{\begin{array}{c}L,\text{if}{V}_z(x,y,z)>\frac{L}{\vartheta \widehat{\beta }},\hfill \\ \vartheta \widehat{\beta }{V}_z(x,y,z),\text{if}{V}_z(x,y,z)\in [0,\frac{L}{\vartheta \widehat{\beta }}],\hfill \\ 0,\text{if}{V}_z(x,y,z)<0.\hfill \end{array}\right)$$ (4.3)

Our numerics is based on a recursion on the nonlinear equation

$$\left(\lambda -\mathcal{L}\right)v(x,y,z)=C^{\star }(y,z,v_z(x,y,z)),(x,y,z)\in \mathcal{O},$$

which is solved by the value function in the classical sense (cf. Theorem 3.3). Namely, starting from $v^{\left[0\right]}\equiv 0$ we use the recursive algorithm:

$$\left(\lambda -\mathcal{L}\right)v^{[n+1]}=C^{\star }(y,z,v_z^{\left[n\right]}),n\ge 1$$

and those equations are solved by Monte Carlo methods based on the Feynman–Kac formula

$$v^{[n+1]}(x,y,z)$$

$$=\mathsf{E}\left[{\int }_0^{\infty }{e}^{-\lambda t}{C}^{\star }\left(I_t^{x,y,z},{\beta }_t^z,v_z^{\left[n\right]}(S_t^{x,y,z},I_t^{x,y,z},{\beta }_t^z)\right)\mathrm{d}t\right],$$

$$(x,y,z)\in \mathcal{O}.$$

Such an approach is needed because of the lack of appropriate boundary conditions on the HJB equation, as the boundary $\partial \mathcal{O}$ is unattainable for the underlying controlled dynamical system.

We take a day as a unit of time. In our experiments we assume that the average length of an infection equals $18$ days, so that $\alpha =\frac{1}{18}$ (see also Alvarez et al., 2020, Bayraktar et al., 2020, and Kruse and Strack, 2020), the level of the maximal possible transmission rate of the disease is $\gamma =0.16$, the natural transmission rate of the disease is $\widehat{\beta }=0.1$, towards which the transmission rate ${\left({\beta }_t\right)}_t$ reverts at rate $\vartheta =0.1$ when $\xi \equiv 0$, $\sigma =1$, so that the fluctuations of ${\left({\beta }_t\right)}_t$ are (at most) of order 10⁻². Furthermore, we set $\lambda =1∕365$,8 and we fix $\overline{y}=0.1$ in (4.2). Finally, in all simulations we assume that at day zero about 1% of the population is infected.

In all the subsequent pictures we show the mean paths of the considered quantities, with their 95% confidence interval. The Monte Carlo average has been performed by employing $6000$ independent simulations.

In Section 4.1, we compare the optimal social planner policy with the case of no restrictions; in Section 4.2 we consider strategies in which the containment measures are limited to a fixed percentage $L\in [0,1]$ and provide a comparison between them; in Section 4.3 we study the effect of the fluctuations of the transmission rate on the problem’s solution.

4.1. The optimal social planner policy

We compare the optimal social planner policy with the case of no restrictions (see Fig. 1). In the optimal social planner policy severe lockdown measures (larger than 40%) are imposed for a period of circa 63 days, starting on day 79; then, it follows a gradual reopening phase. The final percentage of recovered individuals is about 50%, in contrast to 72% which is the total percentage of recovered individuals in the case of no restrictions.

Fig. 1.

Comparison between the optimal social planner policy (upper panel) and the case of no restrictions (lower panel). The figures in the first column show the (average) evolution of the containment policy through the value of the optimal control ${\xi }_t$; the ones in the second column show the (average) evolution of the instantaneous reproduction number ${\mathcal{R}}_t=\frac{{\beta }_t}{\alpha }$; the ones in the third column show (average) evolution of the percentage of susceptible (in blue), infected (in red) and recovered (in green) individuals; the ones in the fourth column show the (average) evolution of the product ${\mathcal{R}}_t\cdot S_t$. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

Furthermore, the cases of optimal lockdown and no lockdown show a substantial difference in the evolution of the reproduction number ${\mathcal{R}}_t≔\frac{{\beta }_t}{\alpha }$: in the case of lockdown policies at work, in the most restrictive period, the latter is significantly decreased to values around 0.6. Another relevant quantity to analyze is ${\mathcal{R}}_t{S}_t$. Indeed, recalling (2.2), it is easy to see that the percentage of infected naturally decreases at exponential rate $\alpha ({\mathcal{R}}_t{S}_t-1)$ if ${\mathcal{R}}_t{S}_t$ is maintained strictly below $1$. We observe that, under the suboptimal action “no lockdown”, ${\mathcal{R}}_t{S}_t$ lies below one from day $85$ on. On the other hand, the optimal containment policy is such that ${\mathcal{R}}_t{S}_t<1$ from day $75$ on. As a consequence, ${\mathcal{R}}_t$ can let oscillate strictly above one (actually, around 1.7) during the final phase of partial reopening so that the negative impact of lockdowns on the economic growth can be partially dammed.

4.2. The optimal social planner policy with limited containment

In many countries, a vigorous lockdown could not always be feasible, especially for long periods. Further, as pointed out by recent literature (for instance see Aspri et al., 2020), gradual policies of longer duration but more moderate containment exhibit large welfare benefits comparable to the ones obtained by a drastic lockdown. For this reason, we consider a strategy in which the containment measures are limited to a fixed percentage $L\in [0,1]$. Notice that $L=0.7$ in Alvarez et al. (2020), $L=\{0.7,1\}$ in Acemoglu et al. (2020) and Bayraktar et al. (2020). A comparison of the optimal social planner policy with limited containment $L\in \{0.2,0.4,0.6,0.8\}$ is shown in Fig. 2 and a summary is contained in Table 1.

Fig. 2.

Comparison between the optimal social planner policy with limited containment $L$. The figures in the first row show the evolution of the (average) containment policy through the value of the optimal control ${\xi }_t$; the figures in the second row show the (average) evolution of the instantaneous reproduction number ${\mathcal{R}}_t=\frac{{\beta }_t}{\alpha }$; the figures in the third row show the evolution of the (average) percentage of susceptible (in blue), infected (in red) and recovered (in green) individuals; the figures in the fourth row show the (average) evolution of the product ${\mathcal{R}}_t\cdot S_t$. The limited level of containment varies with the columns: the first column treats the case $L=0.8$, the second column the case $L=0.6$, the third column the case $L=0.4$ and the last column the case $L=0.2$. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Table 1.

Optimal social planner policy with different values of limited containment $L$.

		$L=0.8$		$L=0.6$		$L=0.4$		$L=0.2$
First day of containment		54		54		54		54
Recovered		52%		58%		61%		68%

Clearly, the larger $L$ is, the smaller are the social costs (by definition of the value function). Our experiment shows that for $L=0.4,0.6,0.8,1$, the final percentage of recovered (hence of the total amount of infected) in average ranges from 52% (case $L=0.8$) up to 68% (case $L=0.2$). In all the cases, the optimal containment starts at the maximal rate and the first day of containment is substantially the same (around day $54$).

Different ceilings $L$ on the containment strategies also affect the values and the fluctuations’ size of the reproduction number $\frac{{\beta }_t}{\alpha }$: smaller values of $L$ correspond to milder variation of the reproduction number ${\mathcal{R}}_t$ of size 0.3, whereas larger values of $L$ lead to rapid changes of ${\mathcal{R}}_t$ which reaches levels smaller than $1$ (less than 0.8 for $L=0.8$ and less than 0.6 for $L=1$). In all the cases, ${\mathcal{R}}_t{S}_t$ lies strictly below $1$ after a certain date, which is decreasing with respect to $L$ (see the last column in Fig. 1, Fig. 2). Notice that without any containment policies, ${\mathcal{R}}_t{S}_t$ decreases on time due to a natural “herd-immunity” effect. On the other, when lockdowns are in place, we observe a faster decrease of ${\mathcal{R}}_t{S}_t$ which is forced by the initial vigorous policymaker’s actions. The final relaxation of the latter then allows for an increase of ${\mathcal{R}}_t{S}_t$, which is however constrained below the critical level of $1$. Such an effect is monotone decreasing with respect to $L$.

4.3. The role of uncertainty

The main new feature of our model is to consider a (controlled) stochastic transmission rate in the framework of the classical SIR model. In this section we study numerically how an increase of the fluctuations of the transmission rate affects the optimal solution. In particular, in Fig. 3 the volatility $\sigma$ takes values $1$, $5$ and $10$, thus leading to fluctuations of $\beta$ of order 10⁻², $5\times 10^{-2}$, and 10⁻¹, respectively (indeed, recall that $\sigma \left(\beta \right)=\sigma \beta (\gamma -\beta )$ attains its maximum at $\gamma ∕2$ and $\gamma =0.16$).

Fig. 3.

Comparison between the optimal social planner policy with different $\sigma$, when $L=1$. The figures in the first row show the evolution of the (average) containment policy through the value of the optimal control ${\xi }_t$; the figures in the second row show the evolution of the (average) percentage of susceptible (in blue), infected (in red) and recovered (in green) individuals. The level of $\sigma$ varies with the columns: the first column treats the case $\sigma =1$, the second column the case $\sigma =5$, the third column the case $\sigma =10$. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

We observe from Fig. 3 that larger fluctuations of $\beta$ have the effect of anticipating the beginning of the lockdown policies, and of diluting the actions over a longer period. Indeed, when $\sigma =5$ and $\sigma =10$, the optimal lockdown policy starts around day $46$ and $42$, respectively, in contrast to day $54$ of the case $\sigma =1$. Moreover, when $\sigma$ increases, the maximal employed lockdown intensity reduces and the level of containment stabilizes at a larger value in the long run. This can be explained by thinking that an increase in the fluctuations of the transmission rate induces the policymaker to act earlier and over a longer period in order to prevent positive large shocks of $\beta$. However, in order to dam the social costs resulting from a longer period of restrictions, the maximal intensity of the lockdown policy should be reduced.

Moreover, such a spread of the optimal lockdown policy gives rise to an increase of the final percentage of recovered (which is circa 58% and 60% when $\sigma =5$ and $\sigma =10$, respectively, and circa 50% when $\sigma =1$).

5. Conclusions

We have studied the problem of a policymaker which during an epidemic is challenged to optimally balance the safeguard of public health and the negative economic impact of severe lockdowns. The policymaker can implement containment policies in order to reduce the trend of the disease’s transmission rate, which evolves stochastically in continuous time. In the context of the SIR model, our theoretical analysis allows to identify the minimal social cost function as a classical solution to the corresponding dynamic programming equation, as well as to provide an optimal control in feedback form.

In a case study in which the transmission rate is a (controlled) mean-reverting diffusion process, numerical experiments show that the optimal lockdown policy is characterized by three distinct phases: the epidemic is first let freely evolve, then vigorously tamed, and finally a less stringent containment should be adopted. Interestingly, in the last period the epidemic’s reproduction number is to let oscillate strictly above one although the product “reproduction number $\times$ percentage of susceptible” is kept strictly below the critical level of one. Hence, under the optimal containment policy, the percentage of infected decreases naturally at an exponential rate and the social planner is then allowed to substantially relax the lockdown in order not to incur too heavy economic costs. Moreover, we show that an increase in the fluctuations of the transmission rate gives rise to an earlier beginning of the optimal lockdown policy, which is also diluted over a longer period of time.

We believe that our work is only a first step in enriching the SIR model of a stochastic controlled component and in understanding the policymaker’s problem of optimally balancing the safeguard of public health and social wealth. There is still much to be done in order to incorporate other features like the partial detectability of the transmission rate or the role of public investment on the discovery of a vaccination (see Remark 2.4 on this). We leave the analysis of the resulting challenging problems for future work.

Manuscript handled by Editor Aditya Goenka

Appendix. Technical results

Lemma A.1

Let ${\mathcal{O}}^{\prime }$ be an open neighborhood of $\mathit{0}=(0,0,0)\in {\mathbb{R}}^3$ . Let $W:{\mathcal{O}}^{\prime }\to \mathbb{R}$ be a semiconcave function such that $W_z^{-}\left(\mathit{0}\right)>W_z^{+}\left(\mathit{0}\right)$ . Then there exists a sequence of functions ${\left({\phi }^n\right)}_n\subset C^2\left({\mathcal{O}}^{\prime }\right)$ such that

$$\left\{\begin{array}{c}{\phi }^n\left(\mathbf{0}\right)=W\left(\mathit{0}\right)=0,\hfill \\ {\phi }^n\ge W\text{in a neighborhood of}\mathit{0},\hfill \\ \left|D{\phi }^n\left(\mathit{0}\right)\right|\le L<\infty ,\hfill \\ {\phi }_{zz}^n\left(\mathbf{0}\right)\stackrel{n\to \infty }{⟶}-\infty .\hfill \end{array}\right)$$ (A.1)

Proof

Since $W$ is semiconcave, there exists $C_0\ge 0$ such that

$$\widehat{W}:{\mathcal{O}}^{\prime }\to \mathbb{R},\widehat{W}(x,y,z)≔W(x,y,z)-C_0\left(x^2+y^2+z^2\right),$$

is concave. Fix such a $C_0$. Since $W_z^{-}\left(\mathit{0}\right)>W_z^{+}\left(\mathit{0}\right)$, also ${\widehat{W}}_z^{-}\left(\mathit{0}\right)>{\widehat{W}}_z^{+}\left(\mathit{0}\right)$ and it is clear that it is equivalent to show the claim for $\widehat{W}$. By Rockafellar (1970, Theorem 23.4), it follows that there exist

$$\mathit{\eta }=({\eta }_x,{\eta }_y,{\eta }_z),\mathit{\zeta }=({\zeta }_x,{\zeta }_y,{\zeta }_z)\in D^{+}W\left(\mathit{0}\right)\text{such that}{\eta }_z>{\zeta }_z.$$

Set

$$g\left(\mathit{q}\right)≔〈\mathit{\eta },\mathit{q}〉\wedge 〈\mathit{\zeta },\mathit{q}〉$$

and notice that $\widehat{W}\left(\mathit{0}\right)=0=g\left(\mathit{0}\right)$ and that, by concavity,

$$\widehat{W}\left(\mathit{q}\right)\le g\left(\mathit{q}\right)\forall \mathit{q}\in {\mathcal{O}}^{\prime }.$$

Define

$$A≔\text{Span}{\{\mathit{\eta }-\mathit{\zeta }\}}^{\perp },$$

and denote by $\Pi :{\mathbb{R}}^3\to A$ the orthogonal projection on $A$. Given $\mathbf{q}\in {\mathbb{R}}^3$ we then have the decomposition

$$\mathbf{q}=\Pi \mathbf{q}+\frac{\mathit{\eta }-\mathit{\zeta }}{|\mathit{\eta }-\mathit{\zeta }|}s,s=\frac{〈\mathbf{q},\mathit{\eta }-\mathit{\zeta }〉}{|\mathit{\eta }-\mathit{\zeta }|}.$$

Define, for $\mathit{q}\in {\mathcal{O}}^{\prime }$,

$${\phi }^n\left(\mathit{q}\right)≔g\left(\Pi \mathit{q}\right)+{\psi }^n\left(s\right),$$

where

$${\psi }^n:\mathbb{R}\to \mathbb{R},{\psi }^n\left(s\right)=-\frac{n}{2}{s}^2+\frac{1}{2}\frac{〈\mathit{\eta }+\mathit{\zeta },\mathit{\eta }-\mathit{\zeta }〉}{|\mathit{\eta }-\mathit{\zeta }|}s.$$

This sequence realizes (A.1). Indeed, the first two properties hold by construction; in particular the second one is due to the fact that we have

$$g\left(\mathit{q}\right)=g\left(\Pi \mathit{q}\right)+\left\{\begin{array}{c}\frac{〈\mathit{\zeta },\mathit{\eta }-\mathit{\zeta }〉}{|\mathit{\eta }-\mathit{\zeta }|}s\text{if}s\ge 0,\hfill \\ \hfill \\ \frac{〈\mathit{\eta },\mathit{\eta }-\mathit{\zeta }〉}{|\mathit{\eta }-\mathit{\zeta }|}s\text{if}s<0.\hfill \end{array}\right)$$

As for the last two properties, we notice that

$$D{\phi }^n\left(\mathit{q}\right)=\Pi \mathit{\eta }(=\Pi \mathit{\zeta })+\frac{\mathit{\eta }-\mathit{\zeta }}{|\mathit{\eta }-\mathit{\zeta }|}\frac{\mathrm{d}{\psi }^n}{\mathrm{d}s}\left(s\right),$$

so

$${\phi }_z^n\left(\mathit{q}\right)=〈\Pi \mathit{\eta },(0,0,1)〉+〈\frac{\mathit{\eta }-\mathit{\zeta }}{|\mathit{\eta }-\mathit{\zeta }|},(0,0,1)〉\frac{\mathrm{d}{\psi }^n}{\mathrm{d}s}\left(s\right)=〈\Pi \mathit{\eta },(0,0,1)〉+\frac{{\eta }_z-{\zeta }_z}{|\mathit{\eta }-\mathit{\zeta }|}\frac{\mathrm{d}{\psi }^n}{\mathrm{d}s}\left(s\right),$$

$${\phi }_{zz}^n\left(\mathit{q}\right)=\frac{{\eta }_z-{\zeta }_z}{|\mathit{\eta }-\mathit{\zeta }|}\frac{{\mathrm{d}}^2{\psi }^n}{\mathrm{d}{s}^2}\left(s\right),$$

which then imply them.  □

Denote by $\mathbf{q}=(q_1,q_2,q_3)≔(x,y,z)$ an arbitrary point of $\mathcal{O}$. For any multi-index $\alpha ≔({\alpha }_1,{\alpha }_2,{\alpha }_3)\in {\mathbb{N}}^3$ we denote by $\left|\alpha \right|={\sum }_{i=1}^3{\alpha }_i$ and $D_{\mathbf{q}}^{\alpha }={\partial }^{\left|\alpha \right|}∕{\partial }_{q_1}^{{\alpha }_1}\dots {\partial }_{q_3}^{{\alpha }_3}$, with the convention that ${\partial }^0$ is the identity.

Proposition A.2

For any $\mathbf{q}\in \mathcal{O}$ the (uncontrolled) process ${\left({\mathbf{Q}}_t^{\mathbf{q}}\right)}_t≔(S_t^{x,y,z},I_t^{x,y,z},{\beta }_t^z)$ admits a transition density $p$ which is absolutely continuous with respect to the Lebesgue measure in ${\mathbb{R}}^3$ , infinitely many times differentiable, and satisfies the Gaussian estimates

$$p(t,\mathbf{q};{\mathbf{q}}^{\prime })\le \frac{C_0\left(t\right){(1+\left|\mathbf{q}\right|)}^{m_0}}{t^{\frac{n_0}{2}}}{e}^{-\frac{D_0\left(t\right){|{\mathbf{q}}^{\prime }-\mathbf{q}|}^2}{t}},\forall t>0,{\mathbf{q}}^{\prime }=(x^{\prime },y^{\prime },z^{\prime })\in \mathcal{O},$$ (A.2)

$$\left|D_{\mathbf{q}}^{\alpha }p(t,\mathbf{q};{\mathbf{q}}^{\prime })\right|\le \frac{C_{\alpha }\left(t\right){(1+\left|\mathbf{q}\right|)}^{m_{\alpha }}}{t^{\frac{n_{\alpha }}{2}}}{e}^{-\frac{D_{\alpha }\left(t\right){|{\mathbf{q}}^{\prime }-\mathbf{q}|}^2}{t}},\forall t>0,{\mathbf{q}}^{\prime }=(x^{\prime },y^{\prime },z^{\prime })\in \mathcal{O}.$$ (A.3)

Here, $C_0$ , $D_0$ , $C_{\alpha }$ , and $D_{\alpha }$ are increasing functions of time.

Proof

Given $f,g\in C^1({\mathbb{R}}^3;{\mathbb{R}}^3)$, define the Lie bracket

$$[f,g]≔\sum _{j=1}^3\left(\frac{\partial g}{\partial q_j}f-\frac{\partial f}{\partial q_j}g\right).$$

Then, for any given and fixed $\mathbf{q}\in \mathcal{O}$, we set

$$\mu \left(\mathbf{q}\right)≔\left(\begin{array}{c}\hfill -xyz\hfill \\ \hfill xyz-\alpha y\hfill \\ \hfill b(z,0)\hfill \end{array}\right)\text{ and }\Sigma \left(\mathbf{q}\right)≔\left(\begin{array}{c}\hfill 000\hfill \\ \hfill 000\hfill \\ \hfill 00\sigma \left(z\right)\hfill \end{array}\right)$$

and denoting by ${\Sigma }_i$, $i=1,2,3$, the columns of the matrix $\Sigma$, we construct recursively the set of functions $L_0≔\{{\Sigma }_1,{\Sigma }_2,{\Sigma }_3\}$, $L_{k+1}≔\{[\mu ,\phi ],[{\Sigma }_1,\phi ],[{\Sigma }_2,\phi ],[{\Sigma }_3,\phi ]:\phi \in L_k\}$, $k\ge 0$. We also define $L_{\infty }≔{\cup }_{k\ge 0}{L}_k$. We say that the Hörmander condition holds true at $\mathbf{q}\in \mathcal{O}$ if

$$Span\left\{\phi \left(\mathbf{q}\right),\phi \in L_{\infty }\right\}={\mathbb{R}}^3.$$ (A.4)

Direct calculations show that

$$L_0\left(\mathbf{q}\right)=\left\{{\Sigma }_3\right\}\left(\mathbf{q}\right)=\left\{\left(\begin{array}{c}\hfill 0\hfill \\ \hfill 0\hfill \\ \hfill \sigma \left(z\right)\hfill \end{array}\right)\right\}$$

$$L_1\left(\mathbf{q}\right)=\left\{[\mu ,{\Sigma }_3]\right\}\left(\mathbf{q}\right)=\left\{\left(\begin{array}{c}\hfill xy\sigma \left(z\right)\hfill \\ \hfill -xy\sigma \left(z\right)\hfill \\ \hfill {\sigma }_z\left(z\right)b(z,0)-\sigma \left(z\right)b_z(z,0)\hfill \end{array}\right)\right\}$$

and

$$L_2\left(\mathbf{q}\right)=\left\{[\mu ,[\mu ,{\Sigma }_3]],[{\Sigma }_3,[\mu ,{\Sigma }_3]]\right\}\left(\mathbf{q}\right)=\left\{\left(\begin{array}{c}\hfill xy(2b(z,0){\sigma }_z\left(z\right)-\alpha \sigma \left(z\right)-\sigma \left(z\right)b_z(z,0))\hfill \\ \hfill xy(\sigma \left(z\right)b_z(z,0)-2b(z,0){\sigma }_z\left(z\right))\hfill \\ \hfill b{(z,0)}^2{\sigma }_{zz}\left(z\right)-\sigma \left(z\right)b(z,0)b_{zz}(z,0)+\sigma \left(z\right)b_z{(z,0)}^2-{\sigma }_z\left(z\right)b(z,0)b_z(z,0)\hfill \end{array}\right),\right)\left(\left(\begin{array}{c}\hfill xy\sigma \left(z\right){\sigma }_z\left(z\right)\hfill \\ \hfill -xy\sigma \left(z\right){\sigma }_z\left(z\right)\hfill \\ \hfill b(z,0)\sigma \left(z\right){\sigma }_{zz}\left(z\right)-\sigma {\left(z\right)}^2{b}_{zz}(z,0)-b(z,0){\sigma }_z{\left(z\right)}^2+\sigma \left(z\right)b_z(z,0){\sigma }_z\left(z\right)\hfill \end{array}\right)\right\}$$

Hence, the matrix associated to $(L_0\cup L_1\cup L_2)\left(\mathbf{q}\right)$ has the sub-matrix formed by all its rows and its first three columns with determinant $-\alpha x^2{y}^2{\sigma }^3\left(z\right)<0$. Hence, (A.4) holds true on $\mathcal{O}$ given the arbitrariness of $\mathbf{q}$.

Therefore, by Theorem 2.3.3 in Nualart (2006), for any $t>0$ the uncontrolled process ${(S_t^{x,y,z},I_t^{x,y,z},{\beta }_t^z)}_t$ admits a transition density $p$ which is absolutely continuous with respect to the Lebesgue measure in ${\mathbb{R}}^3$, and infinitely many times differentiable. Moreover, Theorem 9 and Remark 11 in Bally (2003) (see also Kusuoka and Stroock, 1985) show that $p$ satisfies the Gaussian estimates (A.2), (A.3). This completes the proof. □

Proposition A.3

The dynamics of $\beta$ as in (4.1) satisfy Assumption 2.1 .

Proof

For any $z\in \mathbb{R}$ and $\xi \in [0,L]$, recall that $b(z,\xi )=\vartheta (\widehat{\beta }(L-\xi )-z)$ and define

$$\tilde{\sigma }\left(z\right)=\left\{\begin{array}{c}0\text{if }z\in (-\infty ,0],\hfill \\ \sigma z(\gamma -z)\text{if }z\in (0,\gamma ),\hfill \\ 0\text{if }z\in [\gamma ,\infty ).\hfill \end{array}\right)$$ (A.5)

Then, for ${\left({\xi }_t\right)}_t\in \mathcal{A}$, introduce the stochastic differential equation

$$\mathrm{d}{\tilde{\beta }}_t=b({\tilde{\beta }}_t,{\xi }_t)\mathrm{d}t+\tilde{\sigma }\left({\tilde{\beta }}_t\right)\mathrm{d}{W}_t,{\tilde{\beta }}_0=z\in \mathbb{R}.$$ (A.6)

Because $b$ and $\tilde{\sigma }$ are Lipschitz-continuous and have sublinear growth, uniformly with respect to $\xi$, for any ${\left({\xi }_t\right)}_t\in \mathcal{A}$ there exists a unique strong solution to (A.6) starting at $z\in \mathbb{R}$ (see, e.g., Theorem 7 in Chapter V of Protter, 2004). We denote such a solution by ${\left({\tilde{\beta }}_t^{\xi ,z}\right)}_t$. Since $\xi \mapsto b(z,\xi )$ is decreasing, by Theorem 54 in Chapter V of Protter (2004), we have

$${\tilde{\beta }}_t^{L,z}\le {\tilde{\beta }}_t^{\xi ,z}\le {\tilde{\beta }}_t^{0,z},\forall t\ge 0\text{a.s.},$$ (A.7)

where ${\left({\tilde{\beta }}_t^{L,z}\right)}_t$ and ${\left({\tilde{\beta }}_t^{0,z}\right)}_t$) are, respectively, the solution to (A.6) with ${\xi }_t\equiv L$ and with ${\xi }_t\equiv 0$. On the other hand, by Feller’s test for explosion (cf. Proposition 5.22 in Chapter 5.5 of Karatzas and Shreve, 1991), it can be checked ${\tilde{\beta }}_t^{L,z}>0$ for all $t\ge 0$ a.s. and ${\tilde{\beta }}_t^{0,z}<\gamma$ for all $t\ge 0$ a.s. Hence, by (A.7), we get ${\tilde{\beta }}_t^{\xi ,z}\in (0,\gamma )$ for all $t\ge 0$ a.s. This proves that the SDE (4.1) admits a unique strong solution which lies within the open interval $\mathcal{I}=(0,\gamma )$ (cf. Assumption 2.1-(i)).

Given the boundedness of the interval $\mathcal{I}=(0,\gamma )$, using the expression of $b$ and (A.5) it is straightforward to verify that (ii) and (iii) of Assumption 2.1 are satisfied as well. □