Robust and optimal predictive control of the COVID-19 outbreak

Abstract

We investigate adaptive strategies to robustly and optimally control the COVID-19 pandemic via social distancing measures based on the example of Germany. Our goal is to minimize the number of fatalities over the course of two years without inducing excessive social costs. We consider a tailored model of the German COVID-19 outbreak with different parameter sets to design and validate our approach. Our analysis reveals that an open-loop optimal control policy can significantly decrease the number of fatalities when compared to simpler policies under the assumption of exact model knowledge. In a more realistic scenario with uncertain data and model mismatch, a feedback strategy that updates the policy weekly using model predictive control (MPC) leads to a reliable performance, even when applied to a validation model with deviant parameters. On top of that, we propose a robust MPC-based feedback policy using interval arithmetic that adapts the social distancing measures cautiously and safely, thus leading to a minimum number of fatalities even if measurements are inaccurate and the infection rates cannot be precisely specified by social distancing. Our theoretical findings support various recent studies by showing that (1) adaptive feedback strategies are required to reliably contain the COVID-19 outbreak, (2) well-designed policies can significantly reduce the number of fatalities compared to simpler ones while keeping the amount of social distancing measures on the same level, and (3) imposing stronger social distancing measures early on is more effective and cheaper in the long run than opening up too soon and restoring stricter measures at a later time.

1. Introduction

Social distancing is an effective way to contain the spread of a contagious disease, particularly when little is known about the virus and no vaccines or other pharmaceutical interventions are available. Social distancing and isolation (together with other non-pharmaceutical measures such as hygiene and face masks) have a direct influence on the infection rates and hence on the spread of the virus (Kissler et al., 2020, Maharaj and Kleczkowski, 2012, Maier and Brockmann, 2020). While this combination has proven effective during the last weeks, e.g. in the German outbreak of COVID-19, strict social distancing is also very costly in terms of economical and psychological damage, which naturally leads to a multi-objective decision problem.

There have been numerous approaches to model the COVID-19 outbreak and to predict future behavior for different distancing policies in simulation studies. The most commonly used modeling approaches are different extensions of the SIR (susceptible–infected–removed) model formulated either as system dynamics or as agent-based simulations (e.g. for Germany (Barbarossa et al., 2020, Dehning et al., 2020, German et al., 2020)). In many such studies, different policies are simulated and compared with respect to the goals that both the health care system is not overwhelmed such that every patient in need receives treatment and the mortality rate is kept low, and also such that the majority of people can resume social interaction as soon as possible. However, in line with Alleman, Torfs, and Nopens (2020) and others, we advocate to go from mere model predictions to (model predictive) control, since control generally offers the theory to develop and apply optimal or robust decision making under uncertainties.

While mathematical modeling and control of epidemics is a topic with rich history (see, e.g., the survey in Nowzari, Preciado, and Pappas (2016) and the references therein), there have also been numerous approaches to apply control theory to the COVID-19 spread. In Casella (2021), the author applies control theoretic principles and insights to a simple model of the outbreak to point out the difficulties of the system at hand: fast unstable dynamics with significant delays. In more recent literature, multiple works have addressed the problem of open-loop optimal control for the COVID-19 pandemic. In Bin et al., 2020, Tsay et al., 2020, for example, the authors argue in favor of ’on and off’-policies of the social distancing measures, yielding a bang–bang like optimal control strategy. Such ’on and off’-policies, where the control input switches between two states, however, could pose great challenges, amongst others, for the society, but also for production lines, supply chains and the economy in general.

In this paper, we propose optimal open-loop and feedback control strategies to handle the German COVID-19 outbreak. We employ the recently developed SIDARTHE model (Giordano et al., 2020) in order to design control policies which minimize the number of fatalities within a time horizon of two years, without using excessive social distancing measures. We also address robustness of our policies w.r.t. model and measurement uncertainties via a (robust) model predictive control (MPC) feedback strategy. Note that the following discussion and results are all based on the information and data available prior to the initial submission of this paper (May 2020).

Similar to the setup in this paper, the authors in Djidjou-Demasse, Michalakis, Choisy, Sofonea, and Alizon (2020) explore the best policy to implement while waiting for the availability of a vaccine. In their paper, they also distinguish between varying severity of symptoms (’mild’ or ‘severe’) and seek a solution to the multi-objective optimization problem of minimizing fatalities and costs due to the implementation of the control strategy itself. Their main outcome of the open-loop input strategy is qualitatively similar to our results in Section 3.2: Start with a loose strategy, soon increase all distancing measures such that the health care system capacities are never extorted and then relax the social distancing measures gradually and slowly. Another example for an open-loop optimal policy applied to the COVID-19 pandemic is presented in Kantner and Koprucki (2020) where the authors consider optimal control of the German outbreak using a slightly simpler model as the one chosen in the present paper (without distinguishing between detected and undetected individuals), which also includes an increased mortality rate if the ICU capacity is exceeded. Therein, the objective is not only to minimize the number of fatalities but also the number of susceptible individuals at the end of the time horizon, thus aiming for herd immunity. Our investigations in Section 3, however, indicate that herd immunity cannot be reached in a reasonable amount of time without overwhelming the hospital capacities. Therefore, our approach minimizes the number of fatalities after two years, with the underlying assumption that a vaccine will be available thereafter.

However, an open-loop optimal policy cannot suffice to control the COVID-19 pandemic given all the uncertainties in the spreading of the virus and the disease progression, as we will see in the numerical results. We argue, similar to Alleman et al. (2020), that an MPC-based feedback strategy is the right tool to develop optimal and robust social distancing policies, especially in the presence of model inaccuracies. By using online measurements of the current outbreak, feedback inherently introduces robustness with respect to uncertainties and disturbances to the policy. We also robustify the feedback mechanism by introducing a robust MPC-based feedback strategy for uncertain state measurements which is crucial in a situation where only a limited amount of data is available and, for example, the number of the currently infected persons can only be estimated roughly by applying different studies.

Our results are also in accordance with a very recently published joint strategy paper for Germany by authors from different German research institutions (Fraunhofer-Gesellschaft, Helmholtz Association, Leibniz Association and Max Planck Society) (Meyer-Hermann, Pigeot, Priesemann, & Schöbel, 2020). Firstly, they also state that reaching herd immunity without the availability of a vaccine would either exceed the health care capacities (with a resulting high mortality rate) or take several years (cf. our results in Section 3). Secondly, they state that the goal of wiping out the virus can only be a robust solution if this eradication would be a worldwide effort with very high social and economic costs (cf. our results in Section 3.1.1), which seems impossible to realize. Finally, they suggest an adaptive strategy for all policies influencing the infection rates with the goal to keep the spread of COVID-19 at bay while requiring the least possible restraints on the society and economy. With exactly this reasoning, we develop suitable control approaches in Section 3.2, Section 4, and Section 5 for such an ‘adaptive’ strategy.

To summarize, our key contributions are the following:

• •We extend the model in Giordano et al. (2020) by a mortality rate dependent on the state of the health care capacity and fit the parameters with data from Germany (Section 2).
• •We develop an optimization problem for finding the optimal input (in terms of setting infection rates) that minimizes the number of fatalities while keeping the costs occurring due to distancing measures low (Section 3). Moreover, we show that such an optimal input has significant advantages compared to simpler baseline policies.
• •We show that simply applying a precomputed (optimized) input is dangerous if the model is uncertain and explain why feedback is of utmost importance when dealing with such an unstable and uncertain system. Further, we demonstrate how such feedback can be incorporated via MPC, and we showcase the advantages of this control policy (Section 4).
• •We develop a robust MPC-based feedback strategy, which takes model inaccuracies, uncertain state measurements, and inexact inputs into account and can thus handle the COVID-19 outbreak cautiously and safely (Section 5).

Although based on a simple model fitted with limited data, we hope that these high-level insights inspire further investigations, possibly on more complex epidemiological models, and can ultimately help decision makers to improve and optimize their policies to mitigate the spread of epidemics while keeping the toll on the society and economy low.

2. Modeling of the COVID-19 epidemic

In this section, we describe the model of the COVID-19 epidemic that we use for our subsequent control approach. Our model is adapted from the SIDARTHE model proposed in Giordano et al. (2020) with the key differences that (i) we use more recent data to estimate new parameters to model the German COVID-19 outbreak (in Giordano et al. (2020), the Italian outbreak was considered) and (ii) we model the fact that the mortality rate increases if the number of critically ill patients exceeds the capacity of the German health care system. In Section 2.1, we describe the model of Giordano et al. (2020) and explain its ingredients. Thereafter, in Section 2.2, we provide details on our parameter estimation algorithm which fits the model to the German COVID-19 outbreak. Finally, we propose an extension of the model by increasing the mortality rate when the health care system is overwhelmed in Section 2.3.

2.1. The SIDARTHE model

The considered model based on Giordano et al. (2020) is shown in Fig. 1 and includes eight states: S - Susceptible, I - Infected (asymptomatic, undetected), D - Diagnosed (asymptomatic, detected), A - Ailing (symptomatic, undetected), R - Recognized (symptomatic, detected), T - Threatened (symptomatic with life-threatening symptoms, detected), H - Healed (immune after prior infection, detected or undetected), E - Extinct (dead, detected). In accordance with Fig. 1, the following differential equations describe the SIDARTHE model:

$$\dot{S}=-S(\alpha I+\beta D+\gamma A+\beta R),$$ (1a)

$$\dot{I}=S(\alpha I+\beta D+\gamma A+\beta R)-(ϵ+\zeta +\lambda )I,$$ (1b)

$$\dot{D}=ϵI-(\zeta +\lambda )D,$$ (1c)

$$\dot{A}=\zeta I-(\theta +\mu +\kappa )A,$$ (1d)

$$\dot{R}=\zeta D+\theta A-(\mu +\kappa )R,$$ (1e)

$$\dot{T}=\mu A+\mu R-(\sigma \left(T\right)+\tau \left(T\right))T,$$ (1f)

$$\dot{H}=\lambda I+\lambda D+\kappa A+\kappa R+\sigma \left(T\right)T,$$ (1g)

$$\dot{E}=\tau \left(T\right)T.$$ (1h)

In the Eqs. (1), capital letters describe fractions of the whole population that are currently in the respective state. Since the model represents the whole population, the states sum up to $1$, i.e., they must satisfy $S+I+D+A+R+T+H+E=1$ at all times. Therefore, one equation in (1) is redundant and hence, e.g., the state $H$ can be expressed via the algebraic relation $H=1-(S+I+D+A+R+T+E)$ instead of Eq. (1g), as it is common in the field of differential algebraic equations. In most parts of this section, we omit time arguments for simplicity. Further, Greek letters are the model parameters which are briefly summarized in the following:

Fig. 1.

Scheme of the considered model for the COVID-19 epidemic.

Adapted from [16].

• •$\alpha ,\beta ,\gamma$ describe the infection rates for susceptible individuals, i.e., the rate at which susceptible individuals are infected by the states $I$, $D$ or $R$, and $A$, respectively, and hence join the state $I$.
• •$ϵ,\theta$ describe the testing rate, i.e., at which rate (asymptomatic or symptomatic) infected individuals go from undetected to detected.
• •$\zeta$ describes the rate of asymptomatic (detected or undetected) infected individuals exhibiting symptoms, i.e., going from states $I$ or $D$ to $A$ or $R$, respectively.
• •$\mu$ is the rate at which infected individuals in $A$ or $R$ develop life-threatening symptoms, i.e., join the state $T$.
• •$\lambda ,\kappa ,\sigma \left(T\right)$ are recovery rates for individuals affected by COVID-19. The recovery rate for threatened individuals $\sigma \left(T\right)$ depends on $T$, compare Section 2.3.
• •$\tau \left(T\right)$ is the mortality rate, i.e., the rate at which individuals with life-threatening symptoms decease, and it depends on $T$, compare Section 2.3.

Key features of the considered model for the COVID-19 pandemic compared to simpler ones (e.g., SIR models, compare (Kermack & McKendrick, 1927)) are that it distinguishes between detected and undetected cases, symptomatic and asymptomatic individuals, and it includes a separate state $T$ for patients with life-threatening symptoms (compare [16] for a more detailed explanation of the key ingredients). The present model, i.e., Eqs. (1) as well as Fig. 1, is a mild modification of the model suggested in Giordano et al. (2020). First, we reduce the number of parameters by including the following assumptions. We assume that the rate for developing (severe) symptoms is the same for detected and undetected cases, since (to this day) no effective medication of COVID-19 is known. More precisely, the transitions from $I$ to $A$ and $D$ to $R$ have the same dynamics with rate $\zeta$, and similarly for the respective recovery rates as well as for the transitions from $A$ to $T$ and $R$ to $T$. Moreover, we assume that the rate $\beta$ at which susceptible individuals are infected is the same from states $D$ and $R$, since the state $D$ is neglected for the parameter identification step (compare Section 2.2). Finally, as a key difference to Giordano et al. (2020), we consider $T$-dependent rates $\tau \left(T\right)$ and $\sigma \left(T\right)$ for threatened patients, i.e., the mortality and recovery rates depend on the current number of threatened patients. Essentially, $\tau \left(T\right)$ increases and $\sigma \left(T\right)$ decreases if $T$ exceeds the capacity of the German health care system (see Section 2.3 for a detailed description of this model refinement).

2.2. Parameters for the German outbreak

In this section, we adjust the model parameters and the initial condition given in Giordano et al. (2020) to the COVID-19 outbreak in Germany. This is necessary, because the outbreaks in Germany and Italy evolved differently due to differences in the testing policy, the testing capacity, the health care system, the reaction of the governmental authorities, and the underlying counting method of confirmed cases.

In order to compute realistic parameters for Germany, we will use a pragmatic approach that enables us to easily include prior knowledge about relations between parameters. The approach is a least squares optimization of the available data, where prior knowledge is incorporated via hard constraints in the optimization problem. The available data is marked by a tilde and is given by:

• •the confirmed COVID-19 cases $\tilde{C}$, deaths $\tilde{E}$, and recoveries ${\tilde{H}}_c$ from Dong et al., 2020a, Dong et al., 2020b for the days $t\in [0,49]$ from February 28, 2020 ($t=0$) to April 21, 2020 ($t=53$). We filtered this data set using the Matlab function kaiser(7,3) with window length $7$ and shape factor $3$ to reduce the effect of noise corruption and having less confirmed cases during weekends. Further, we have to divide the data set by the total German population $N_{\mathrm{total}}=8.3\cdot 10^7$ to ensure all values are normalized and are in the range $[0,1]$.
• •the COVID-19 patients in ICU ${\tilde{T}}_2$ and how many of them died ${\tilde{E}}_2$ or recovered ${\tilde{H}}_2$ from DIVI-IntensivRegister (2020) for $t\in [24,53]$ from March 23 ($t=24$) to April 21 ($t=53$).1

This data set, however, is rather small compared to the complexity of the model (1) consisting of eight states and $13$ parameters. Therefore, we need to leverage additional prior knowledge in order to avoid over-fitting and ensure a realistic resulting parameter set and initial conditions. Based on other studies and the interpretation of our model states and parameters, we enforce the following assumptions.

• •The detection rate of asymptomatic cases is negligible, as the current German policy is to test only people showing symptoms (Jakob Simmank, 2020), i.e., $ϵ=0$.
• •At February 28, the initial date for our fit, there were $48$ confirmed cases, hence, we assume $R\left(0\right)=48∕N_{\mathrm{total}}$, $T\left(0\right)=D\left(0\right)=H\left(0\right)=E\left(0\right)=0$, and $I\left(0\right)$, $A\left(0\right)$, $S\left(0\right)$ appear as decision variables with $S\left(0\right)=1-R\left(0\right)-I\left(0\right)-A\left(0\right)$.
• •The test rate $\theta$ is approximately constant. Please note that this does not mean that the absolute number of tests is constant per day, as this value is rather proportional to $\theta A$ than to $\theta$.
• •The infection rates $\alpha$ and $\gamma$ were influenced by the countermeasures that the German authorities installed to fight the spread of the pandemic. According to Dehning et al. (2020), three main events changed the spreading rates: (1) March 9 — canceling large events, (2) March 16 — closing schools and non-essential stores, and (3) March 23 — contact ban (Kontaktsperre) that prohibits groups of more than two people and requires people to maintain a distance of at least $1.5\mathrm{m}$ in public. Hence, there are four different policies $u_i$, $i=1,2,3,4$ monotonically increasing from no countermeasures $u_1=0$ to full lockdown $u_4=1$ resulting in ${\alpha }_i={\alpha }_{\text{max}}+u_i({\alpha }_{\text{min}}-{\alpha }_{\text{max}})$ and ${\gamma }_i={\gamma }_{\text{max}}+u_i({\gamma }_{\text{min}}-{\gamma }_{\text{max}})$. This yields the following six decision variables ${\alpha }_{\text{min}}$, ${\alpha }_{\text{max}}$, ${\gamma }_{\text{min}}$, ${\gamma }_{\text{max}}$, $u_2$, and $u_3$.
• •One of the main reasons why the COVID-19 pandemic is spreading so fast is that infectiousness peaks even before the onset of symptoms (He et al., 2020). As asymptomatic individuals have no indication of their infection, they are on top of that also not as cautious as people with symptoms. Therefore, we require $\alpha \ge \gamma$ when searching for realistic parameters. Further, we want to ensure that people tested positive are significantly less contagious while in quarantine, such that we require $\gamma \ge 5\beta$.
• •The percentage of confirmed COVID-19 cases is estimated in the study (Bommer & Vollmer, 2020) as 27.32% in Germany. In our model, this value approaches the constant $\varphi =\frac{\zeta }{\lambda +\zeta }\frac{\theta +\mu }{\kappa +\theta +\mu }$, which is the proportion of people that develop symptoms ($I$ to $A$, $I$ to $D$ can be ignored as $ϵ=0$) and get detected ($A$ to $R$ or $T$); that is the percentage of confirmed accumulated cases in a steady state ($I=D=A=R=T=0$). To make sure our model coincides with the findings of Bommer and Vollmer (2020), we expect $\varphi$ to be slightly above of the estimated 27.32% as a steady state is not reached yet and the proportion of detected cases increases over time, i.e., we constrain $\varphi \in [0.3,0.45]$.
• •The percentage of asymptomatic disease progressions was estimated at 43% in a population screening study in Iceland (Gudbjartsson et al., 2020), at 43.2% in a comprehensive testing of the whole municipality of Vo’, Italy (Lavezzo et al., 2020) and at 17.9% (Mizumoto, Kagaya, Zarebski, & Chowell, 2020) in a study regarding the cruise ship Diamond Princess. To ensure that our model has a comparable ratio, we add the constraint $\frac{\lambda }{\lambda +\zeta }\in [0.18,0.43]$ to the optimization problem.
• •
  The (base) reproduction rate in the beginning of March was estimated as approximately $3$ (an der Heiden & Hamouda, 2020). Thus, for the parameters ${\alpha }_{\text{max}}$, ${\gamma }_{\text{max}}$ with no active countermeasures we require $R_0({\alpha }_{\text{max}},{\gamma }_{\text{max}})\in [2.5,3.5]$ where $R_0(\alpha ,\gamma )$ is given by (see Giordano et al. (2020) for details)

  $$R_0(\alpha ,\gamma )=\frac{1}{\zeta +\lambda }\left(\alpha +\frac{1}{\theta +\mu +\kappa }\left(\gamma \zeta +\frac{\beta \theta \zeta }{\mu +\kappa }\right)\right).$$ (2)
• •The median of the incubation time is $5$–$6$ days (Li et al., 2020, Linton et al., 2020, World Health Organization (WHO), 2020), which we identify as the half life period a person is in the state $I$, i.e. $log\left(2\right)∕(\lambda +\zeta )\in [5,6]$. Further, the median time from onset of symptoms until intensive care is $10$–$11$ days (Wang et al., 2020, Yang et al., 2020). Hence, we constrain the half life period of a ailing or recognized individuals to $log\left(2\right)∕(\kappa +\mu )\in [10,11]$.

In the state $H$ of (1) the confirmed recovered cases are not distinguished from the undetected ones, thus we define the number of confirmed recovered cases as $H_c$, with $H_c\left(0\right)=0$ and ${\dot{H}}_c=\lambda D+\kappa R+{\sigma }_1{T}_1+{\sigma }_2{T}_2$ and further the number of confirmed accumulated cases as $C=D+R+T+H_c+E$ in order to match the data ${\tilde{H}}_c$ and $\tilde{C}$. Considering the COVID-19 patients in intensive care ${\tilde{T}}_2$, a natural choice would be to identify them with threatened state $T$, however, all deaths in the model have been in $T$ before, but in reality only half to a third of the deaths happens in ICU (DIVI-IntensivRegister, 2020, Dong et al., 2020a). Hence, as the patients in ICU are only a part of $T$, we split $T$ into $T_1$ and $T_2$, where $T_2$ represents the number of people in intensive care and $T_1$ are all otherwise threatened COVID-19 cases. We assume that there are no transitions from $T_1$ to $T_2$ and vice versa, such that $T=T_1+T_2$ can be modeled as

$${\dot{T}}_1={\mu }_1(A+R)-({\sigma }_1+{\tau }_1)T_1$$ (3a)

$${\dot{T}}_2={\mu }_2(A+R)-({\sigma }_1\left(T_2\right)+{\tau }_2\left(T_2\right))$$ (3b)

with ${\mu }_1+{\mu }_2=\mu$. This more complex model with $T_1$ and $T_2$ will be approximated with a model of the form described in Section 2.1 as sketched in Fig. 2. Further, we define $H_2$ ($E_2$) to be the numbers of people that recovered (died) from $T_2$.

Fig. 2.

The threatened state $T$ is split up in ICU cases $T_2$ and non-ICU cases $T_1$ which is later approximated using a lumped model.

Finally, we perform the parameter optimization by solving a least squares problem via CasADi (Andersson, Gillis, Horn, Rawlings, & Diehl, 2019) to fit $C$, $E$, $H_c$, $T_2$, $E_2$, $H_2$ to the data $\tilde{C}$, $\tilde{E}$, ${\tilde{H}}_c$, ${\tilde{T}}_2$, ${\tilde{E}}_2$, ${\tilde{H}}_2$. The best fitting parameters are given in Table 1 and the resulting fit is shown in Fig. 3. Many of the constraints listed above are active at the optimal set of parameters, e.g., $\alpha =\gamma$, which is not surprising since we use the constraints to keep the parameters in a realistic range without further regularization.

Table 1.

The parameters of the model for Germany.

${\alpha }_{\text{min}}=$	$0.0422$	$u_1=$	$0.5816$	$\zeta =$	$0.0790$	${\mathrm{\mu }}_1=$	$0.0080$	${\mathrm{\mu }}_2=$	$0.0050$
${\alpha }_{\text{max}}=$	$0.3614$	$u_2=$	$0.7062$	$\lambda =$	$0.0596$	${\sigma }_1=$	$0.0370$	${\sigma }_2=$	$0.0552$
${\gamma }_{\text{min}}=$	$0.0422$	$\beta =$	$0.0084$	$\kappa =$	$0.0563$	${\tau }_1=$	$0.0159$	${\tau }_2=$	$0.0242$
${\gamma }_{\text{max}}=$	$0.3614$	$\theta =$	$0.1981$	$I_0=$	$500∕N_{\mathrm{total}}$		$A_0=304∕N_{\mathrm{total}}$

Fig. 3.

Simulation of the model (1) with the estimated parameters in Table 1 scaled by $N_{\mathrm{total}}$ compared to the actual data (DIVI-IntensivRegister, 2020, Dong et al., 2020a). The horizontal axis represents the time in days, where $t=1$ is February 28 and $t=53$ is April 21.

This fit further enables us to specify the full model state of today $x\left(53\right)≕x_0$, which will be used in the following sections as the initial condition where $t=53$ corresponds to April 21

$$x_0=\frac{1}{N_{\text{total}}}{\left[\begin{array}{cccccccc}\hfill 82636256\hfill & \hfill 20581\hfill & \hfill 0\hfill & \hfill 8041\hfill & \hfill 41931\hfill & \hfill 11469\hfill & \hfill 276911\hfill & \hfill 4810\hfill \end{array}\right]}^{\top }.$$ (4)

Please note that the model is quite sensitive to changes in the parameters and one obtains quite different parameter values if, e.g., the estimated range of unknown cases or the percentage of asymptomatic cases deviate from the assumptions.

2.3. Modeling of the mortality rate

It has been recognized as a key difficulty in handling the COVID-19 pandemic that the virus is highly contagious, thus infecting large numbers of individuals. In addition, since many elderly and ill people require hospitalization and/or intensive care (Verity et al., 2020), large waves of infections can quickly exceed the capacities of local health care systems (Grasselli, Pesenti, & Cecconi, 2020). Hence, ensuring that health care resources are sufficient is a key issue in handling the outbreak (IHME COVID-19 health service utilization forecasting team, C. J. L. Murray, 2020), given that an overwhelmed health care system even correlates positively with the mortality rate (Ji, Ma, Peppelenbosch, & Pan, 2020).

In this section, we describe how the mortality and recovery rates $\tau \left(T\right)$ and $\sigma \left(T\right)$ in (1) depend on the number of threatened patients $T$. The basic idea is that they are constant as long as the health care system’s capacity is not at its limit, and the mortality rate $\tau$ (the recovery rate $\sigma$) increases (decreases) significantly if it is overwhelmed. According to DIVI-IntensivRegister (2020), there are (on April 21) $2908$ COVID-19 patients in an ICU and $12623$ ICU spots are available. Hence, the overall ICU capacity currently available for COVID-19 patients is $2908+12623=15531$, and we define the relative ICU capacity as $T_{\text{ICU}}=\frac{15531}{N_{\mathrm{total}}}$, where $N_{\mathrm{total}}=8.3\cdot 10^7$. We consider a constant value of $T_{\text{ICU}}$ for simplicity, although it is likely that it will further increase in the future.

We assume that the mortality rate increases if the number of individuals requiring treatment in an ICU exceeds $T_{\text{ICU}}$, i.e., if $T_2>T_{\text{ICU}}$, with $T_2$ as in 2. More precisely, we assume that if a patient requiring intensive care does not receive it, then the patient deceases (i.e., for such patients, the mortality rate increases and the recovery rate is zero). According to data of deceased individuals from Italy, those who were not admitted to an ICU deceased in median within $4$ days (COVID-19 Surveillance Group, 2020). Hence, we model those individuals in $T_2$ which are not admitted to an ICU via decaying first order dynamics with a half-life period of $4$ days, i.e, the corresponding time constant ${\tau }_{\text{crit}}$ satisfies $e^{-4{\tau }_{\text{crit}}}=0.5$, thus leading to ${\tau }_{\text{crit}}=0.173$.

In a first approximation $T_2\approx \frac{{\mu }_2}{\mu }T$ and hence we only modify the mortality rate $\tau$ in case that $\frac{{\mu }_2}{\mu }T>T_{\text{ICU}}$. In the model (1), $\tau \left(T\right)$ and $\sigma \left(T\right)$ only occur jointly with $T$, which leads us to the following formula for $\tau \left(T\right)T$ and $\sigma \left(T\right)T$:

$$\tau \left(T\right)T=\frac{{\mu }_1}{\mu }{\tau }_{1}T+max\left\{\frac{{\mu }_2}{\mu }{\tau }_{2}T,{\tau }_2{T}_{\text{ICU}}+{\tau }_{\text{crit}}\left(\frac{{\mu }_2}{\mu }T-T_{\text{ICU}}\right)\right\},$$ (5a)

$$\sigma \left(T\right)T=\frac{{\mu }_1}{\mu }{\sigma }_{1}T+{\sigma }_{2}min\left\{\frac{{\mu }_2}{\mu }T,T_{\text{ICU}}\right\}.$$ (5b)

If $\frac{{\mu }_2}{\mu }T\le T_{\text{ICU}}$, then (5) implies $\tau \left(T\right)T=(\frac{{\mu }_1}{\mu }{\tau }_1+\frac{{\mu }_2}{\mu }{\tau }_2)T$ and $\sigma \left(T\right)T=(\frac{{\mu }_1}{\mu }{\sigma }_1+\frac{{\mu }_2}{\mu }{\sigma }_2)T$, i.e., a simple lumped model is recovered as long as the ICU capacity is not exceeded. If however $\frac{{\mu }_2}{\mu }T>T_{\text{ICU}}$, then the mortality rate increases to ${\tau }_{\text{crit}}$ for those $\frac{{\mu }_2}{\mu }T-T_{\text{ICU}}$ patients which require intensive care but do not receive it. Similarly, for this fraction, the recovery rate is set to zero implicitly in (5b). The individuals $T_1=\frac{{\mu }_1}{\mu }T$ not receiving intensive care are not affected by this mechanism.

Clearly, the modified rates in (5) are just a simple approximation of the effect that the mortality rate increases if hospitals are overwhelmed. This modification in the model is crucial when studying the effect of loosening quarantine measures and corresponding optimal policies, as done in the remainder of this paper. Since (fortunately) the German health care system has not been overwhelmed to this date, there are no quantitative data to validate the above modification and in particular, the exact value of ${\tau }_{\text{crit}}$. Nevertheless, the refinement is confirmed qualitatively by experiences in other countries (Grasselli et al., 2020, IHME COVID-19 health service utilization forecasting team, C. J. L. Murray, 2020, Ji et al., 2020). Moreover, even a substantial change of ${\tau }_{\text{crit}}$ has little effect on the overall dynamics since it only affects the exact number of fatalities. In particular, changing ${\tau }_{\text{crit}}$ does not lead to a qualitative change in an optimal policy to control the outbreak as long as ${\tau }_{\text{crit}}$ is sufficiently larger than ${\tau }_2$ and it is possible not to exceed the ICU capacity.

3. Open-loop optimal control of the COVID-19 outbreak

In this section, we discuss different policies that can be considered to keep the number of fatalities due to COVID-19 low while at the same time also impose as little constraints as possible on the public. The most significant degree of freedom currently is certainly influencing the infection rates $\alpha$ and $\gamma$. Measures for influencing the infection rates include hygienic measures, face masks, and different nuances of distancing policies, up to a mandated lockdown. Therefore, we define $u$ as introduced in Section 2.2 as our input, representing distancing policies or other measures that have a direct influence on the infection rates $\alpha$ and $\gamma$. We model this influence via

$$\alpha \left(t\right)={\alpha }_{\text{max}}+({\alpha }_{\text{min}}-{\alpha }_{\text{max}})u\left(t\right)$$ (6a)

$$\gamma \left(t\right)={\gamma }_{\text{max}}+({\gamma }_{\text{min}}-{\gamma }_{\text{max}})u\left(t\right),$$ (6b)

where a value of $u=1$ hence represents the policies in Germany as of mid April (lockdown) and $u=0$ represents no social distancing or other measures (i.e. corresponding to infection rates as in the beginning of March). Furthermore, we assume that the policies affecting the infection rates $\alpha$, $\gamma$ (i.e. $u$) stay constant for at least one week and can only be changed every seven days. In the first subsection, we will introduce different baseline policies which can give insights into the effects of different inputs $u$ and which will serve as a comparison to the optimal controller in the following subsection. More specifically, these baseline strategies will be used to define an upper bound on the social distancing measures that the optimal control in Section 3.2 and later on the feedback strategies in Sections 4, 5 can employ to minimize the fatalities.

Test capacity

In addition to varying the infection rates $\alpha$ and $\gamma$, another degree of freedom to influence the model (1) lies in adapting the testing policy. Testing individuals on COVID-19 is represented in the current model by parameters $\theta$ and $ϵ$ for symptomatic and asymptomatic individuals, respectively. In the following, we assume that only a fixed number of tests can be carried out every day. If we wish to only test symptomatic individuals, this includes both symptomatic individuals infected with COVID-19 (i.e., members of the state $A$) and individuals suffering from other illnesses with similar symptoms. In Robert Koch Institut (2017), the Robert Koch Institute estimates numbers on influenza-like illnesses (ILI) in Germany. While the numbers show clear seasonal differences, approximately 1.3% of the population become newly infected with ILI on average per week, and approximately 37% of them see a doctor (an indication for more severe symptoms). Moreover, influenza symptoms usually last 4–5 days leaving us with an approximate average of $p_{\text{sick}}=0.3\text{\%}$ of the population showing significant influenza-like symptoms at an arbitrary time of the year. When testing asymptomatic individuals, this includes infected persons without symptoms but also any other individual not known to be infected or healed and not showing any symptoms (i.e. $S+I+H-H_c-p_{\text{sick}}$, where $H_c$ are the confirmed healed cases, compare Section 2.2). The total amount of resources used for testing is then captured by the following cost:

$$c_{\text{test}}(ϵ,\theta ,A,S,I,H-H_c)=ϵ(S+I+H-H_c-p_{\text{sick}})+\theta \cdot (A+p_{\text{sick}})+A\mu .$$ (7)

Denoting the parameter $\theta$ from Table 1 as ${\theta }_n$, we assume a fixed bound $\overline{c}>0$ on the amount of resources for testing $c_{\text{test}}$ and that ${\theta }_n$ corresponds to the nominal value for $A=0$, i.e., $\overline{c}≔c_{\text{test}}(0,{\theta }_n,0)$. In the following, we assume that the current policy with respect to testing stays in place: all available tests are used on a daily basis for as many symptomatic people as tests are available. Then the testing policy used throughout this section reads $ϵ\left(t\right)=0$ (as is current practice) and

$$\theta \left(t\right)=({\theta }_n\cdot p_{\text{sick}}-\mu A\left(t\right))∕(p_{\text{sick}}+A\left(t\right)).$$ (8)

Note that this also implies that throughout this paper the state $D\equiv 0$. The allocation of tests (with the possibility of also saving test resources for later) can also be modeled as control inputs. However, in the present model the effects of temporarily saving tests (under the current resource constraints) are negligible compared to the effects of changes in the infection rates. Increasing the overall test capacity or improving the choice of test subjects (e.g. with tracing of cases), which corresponds to increasing values of ${\theta }_n$/$\overline{c}$, on the other hand, can potentially improve the evolution of the pandemic significantly, since detected individuals are less contagious than undetected ones. However, increasing test capacities or better allocated testing (especially with regard to $ϵ$, i.e. tracing of also asymptomatic infections) is at the current stage not included in our consideration but could be addressed in future work with the presented model by choosing $ϵ\ne 0$ and making ${\theta }_n$/$\overline{c}$ an increasing and time-varying variable.

Control goal

Given the introduced control input $u$, different control goals can be formulated. One such goal could be to obtain herd immunity. Herd immunity corresponds to the only stable equilibrium given no social distancing measures (i.e. with ${\alpha }_{\text{max}}$, ${\gamma }_{\text{max}}$) and requires a large part of the society to be immune. More precisely, herd immunity is reached if $S<S^{\star }$, where (Giordano et al., 2020) provide a formula for calculating $S^{\star }$ (see Section 3.1.1 for more details). Given our model, we can now calculate the minimum time that is needed to reach herd immunity. For this, we assume that we can choose a policy that utilizes the full health care capacity at all times. With $T\equiv \frac{\mu }{{\mu }_2}{T}_{\text{ICU}}$, $\dot{T}\equiv 0$, $\dot{A}+\dot{R}\equiv 0$, $\dot{I}+\dot{D}\equiv 0$, $S$ decreases each day by $\frac{(\zeta +\lambda )(\mu +\kappa )(\sigma +\tau )}{\zeta \mu }\frac{\mu }{{\mu }_2}{T}_{\text{ICU}}$. Hence,

$$t_{\text{herd}}=\frac{\zeta {\mu }_2(1-S^{\star })}{(\zeta +\lambda )(\mu +\kappa )(\sigma +\tau )T_{\text{ICU}}}$$

gives a lower bound on the time required to reach herd immunity without exceeding the health care capacity given the introduced model. The herein identified model parameters yield a time span $t_{\text{herd}}$ of more than six years. A stable steady state in the absence of a vaccine (i.e. herd immunity) can hence only be obtained either after many years or by overstraining the health care system and a corresponding significant rise in the number of fatalities. Therefore, our ongoing assumption throughout this section is that prior to herd immunity, a vaccine will be available and we assume the availability of the vaccine in approximately two years. Our goal is thus to find an optimal policy minimizing the number of fatalities for the next two years while imposing as little constraints as possible on the public and the economy.

In the next subsection, we simulate and discuss the following policies:

• 1.Keeping the social distancing measures in place (or even increasing the measures) until the virus is eradicated in Germany
• 2.Slowly (or more aggressively) loosening the distancing measures without overwhelming the health care capacities (while possibly risking a second wave).

In fact, the presented baseline policies are similar to the policies suggested by the German “Helmholtz-Initiative” in Helmholtz-Initiative ‘Systemische Epidemiologische Analyse der Covid-19-Epidemie’ (2020). We will discuss our conclusions in comparison to theirs at the end of the section.

In Section 3.2, we will then improve these baseline policies by applying optimal control techniques and we will discuss the importance and significance of the improvements.

3.1. Introducing different baseline policies

3.1.1. Consistent lockdown

In the following, we argue that a consistent lockdown strategy necessitates strong lockdown measures over a long time horizon to fully eradicate the virus as otherwise, dropping the social distancing measures too early leads to a second outbreak wave.

Based on the SIDARTHE model fitted to the German outbreak, described in Section 2.1, we simulate how long we would need to remain in lockdown and simply wait for the virus to disappear. We define the disappearance of the virus as follows: If – most probably – there is less than one active contagious case, i.e., $I+D+A+R+T<0.5∕N_{\mathrm{total}}$, the virus is eradicated. It takes $305$ days, which is almost one year, until this condition is fulfilled and clearly the economical and psychological damage caused by a lockdown period this long is excessive such that staying and waiting in lockdown is not an option. With even stricter measures, such as ${\alpha }_3=0.8{\alpha }_3$, ${\gamma }_3=0.8{\gamma }_3$, we could only marginally accelerate this process to $288$ days while increasing social distancing is costly, cf. the cost function in Section 3.2. Note that the equilibrium attained under the above lockdown policy is an unstable one that is not robust to uncertainties. In particular, if only one person remains infected when the measures are suspended they could cause a new outbreak. Also, the virus may be reimported from other countries or humans might be reinfected by an interim host.

Next, we simulate the following three scenarios in all of which the German population is kept in lockdown for a predefined period of time, followed by no measures at all. The only difference is the length of the lockdown period. In the first scenario, the measures are abolished immediately (April 21). The second one keeps the current strict measures for an additional $50$ days. The third variant simulates an even longer lockdown period, ending after $150$ days counting from April 21.

In Fig. 4, we compare the three scenarios. We clearly see that in all three cases the number of currently infected people $I+D+A+R+T$ rises drastically a few days after the measures are removed independent of how long the lockdown persisted before. In any case, we experience a second outbreak wave. Staying longer in lockdown slightly delays the following peak of the share of active cases $I+D+A+R+T$, yet the peak amplitude is almost the same in all three scenarios.

Fig. 4.

Simulation of the model (1) for a consistent lockdown of different lengths $T_{l,i}$ with days on the horizontal axis ($T_{l,1}=0$ - blue, solid; $T_{l,2}=50$ - red, dash-dotted; $T_{l,3}=150$ - yellow, dashed).

This behavior can be explained as follows. If there is no one who currently has the virus, i.e., $I_{\text{eq}}=D_{\text{eq}}=A_{\text{eq}}=R_{\text{eq}}=T_{\text{eq}}=0$, such that $S_{\text{eq}}+H_{\text{eq}}+E_{\text{eq}}=1$, an equilibrium point is attained. The stability of the equilibrium point depends on the value of $S_{\text{eq}}$ and the model parameters. In Giordano et al. (2020), the authors show that the IDART subsystem is asymptotically stable if and only if $S_{\text{eq}}<S^{\star }$, where $S_{\text{eq}}$ is the susceptible state at equilibrium for a given initial condition $x_0$ and the corresponding parameters, especially for $\alpha$ and $\gamma$ that are actively adjusted according to an underlying policy. The value of $S^{\star }$ follows from the stability analysis of the linearized IDART subsystem and has the following structure with respect to $\alpha$ and $\gamma$

$$S^{\star }=\frac{a_1}{\alpha a_2+\gamma a_3+a_4},$$ (9)

where $a_i,i=1\dots 4$ are constants, see Giordano et al. (2020) for details and the definition of $S^{\star }$. Note further that the commonly stated base reproduction rate (2) is directly linked to the value $S^{\star }$ via $R_0=1∕S^{\star }$. The stability of an equilibrium that depends on the parameters changes once we adjust $\alpha$ and $\gamma$. For strict measures (${\alpha }_{\min }$, ${\gamma }_{\min }$), the value $S^{\star }$ is high ($S^{\star }=2.242$), such that a stable equilibrium is attained for any $S$. This means that only a small number of people is infected by the virus before the equilibrium is attained. With no measures (${\alpha }_{\max }$, ${\gamma }_{\max }$), a stable equilibrium requires the share of susceptible people to be smaller than $S^{\star }=0.292$, i.e., herd immunity.

We can hence conclude that if $S_{\text{eq}}(x_0,{\alpha }_3,{\gamma }_3)>S^{\star }({\alpha }_0,{\gamma }_0)$, the equilibrium attained during lockdown is unstable with no measures. This means, there inevitably is a second outbreak wave once the lockdown ends. For the fitted model of the German outbreak, $S_{\text{eq}}(x_0,{\alpha }_3,{\gamma }_3)=0.9956$ is attained after the first wave. Hence, at least another 70.4% of the German population get infected in the second wave before a stable equilibrium is attained. Altogether, this leaves us with the following conclusion of two possible outcomes when choosing a consistent lockdown strategy:

• •Strong lockdown measures over long time horizons have to be taken to eradicate the virus in Germany. However, this takes a big toll on the public and any infected person, e.g. from abroad, could spark a second wave at any point.
• •Any lockdown strategy that does not fully wipe out the virus inevitably yields a second outbreak wave once all measures are suspended.

3.1.2. Iterative loosening of the distancing policies

As argued in the previous subsection, keeping the lockdown policy strict can never lead to a stable equilibrium when ending the lockdown, no matter how long it did take place before all measures were suspended. Hence, many countries are now discussing or have even already started to loosen the lockdown in very small steps. Indeed, experts consulting the German government (”Nationale Akademie der Wissenschaften Leopoldina”) have recently published their recommendations concerning a possible strategy for loosening the lockdown gradually in small steps (Leopoldina, 2020).

They name the following conditions for loosening the lockdown in small steps:

• (a)The number of new infections remains at a low value.
• (b)The capacity of the health care system must not be exceeded.
• (c)Precautions (such as hygienic measures, face masks, distancing) remain in force.

In the following, we try to translate these recommendations into a policy for our simplified model to first analyze the results and second, to use this as a baseline policy for the optimizer in the following subsection. We implement the conditions presented above via the following policy strategy:

• (a)$u$ can only be decreased if, over the last $n_{\text{stab}}$ days, the number of newly infected persons (i.e. $S(t-1)-S\left(t\right)$) is decreasing and $u$ has not been increased
• (b)$u$ can only be decreased if less than $X_{\text{lower}}$ of the ICUs are occupied
• (c)The decrease in $u$ can only be a small decrease at a time and therefore, the interval between $u_{max}=1$ (lockdown) and $u_{min}=0$ (no measures) is divided into $n_{\text{steps}}$ equidistant steps.

Additionally, we add that $u$ will be increased again (with the same step size as the decrease) if more than $X_{\text{upper}}$ of the ICUs are occupied and no decrease in the amount of necessary ICU is witnessed. This policy results in four ’tuning parameters’ of the policy: $n_{\text{stab}}$, $X_{\text{lower}}$, $X_{\text{upper}}$ and $n_{\text{steps}}$. In fact, it turns out that the outcome of the simulation is not very sensitive to the tuning parameters of the policy, but can be tuned to be slightly more careful or more aggressive. In the following subsection, we choose two different sets of parameters as baseline policies for the optimal control approach.

3.2. Optimal control strategy

In this section, we contrast the baseline policy from Section 3.1.2 with an optimal control policy, under idealized assumptions (exact model and state measurement). The purpose of this section is twofold: (a) Understand how an optimal policy differs qualitatively from the baseline policies. (b) Quantify the loss of performance (in terms of increased fatalities and/or unnecessary social policy $u$) resulting from using a suboptimal baseline policy. The degree of freedom is the input $u\in [0,1]$ affecting the social policy and we consider the fact that the policy can only be changed every $T_s=7$ days.

Multi-objective optimal control problem

In the following, we consider the problem only for the next $N=100$ weeks, assuming that thereafter a vaccine might be developed that would ideally prevent (almost all) further fatalities in the future. The overall control problem can be seen as a multi-objective optimal control problem, where we wish to simultaneously minimize the number of fatalities $E$ and the societal and economical cost of the social policy measures, which will be measured by the function $c_{\text{policy}}\left(u\right)=1∕\alpha \left(u\right)$. We point out that due to the parametrization (6) this cost also inherently considers the infection rate $\gamma$. This cost function is such that the social cost of achieving an arbitrarily small infection rate $\alpha$ grows unbounded, while for large values of $u$ incremental differences are less relevant. In order to suitably characterize an optimal solution to this multi-objective problem we use the baseline policies in Section 3.1. The resulting optimal control problem is given by (11) below, which will be explained in the following. In particular, our goal is to find an input policy that minimizes the number of accumulated fatalities, while using less resources than the baseline policy in terms of accumulated social impact of $c_{\text{policy}}$ (c.f. (11d)). We point out that similar “stabilization” problems subject to resource constraints for the control of epidemic outbreaks can be found in the literature, also using a fractional cost $c_{\text{policy}}$, compare e.g. Köhler et al., 2018, Preciado et al., 2014.

When minimizing the number of accumulated fatalities, it is important to consider not only the extinct individuals $E\left(N\cdot T_s\right)$ at the end of the two year horizon, but to account as well for the part of the already infected individuals that will decease after the two year horizon. The reason for this is, while the availability of a vaccine at the end of the horizon might prevent future infections, it cannot cure already infected people. Hence, if we do not account for the inevitable fatalities among the individuals infected at the end of the prediction horizon, the optimal controller does not take any efforts to keep them low and as a result a lot of people would die shortly after the two year horizon. Therefore, we propose an optimization objective that includes all past and inevitable future fatalities. Based on the model (1), we know that a total of $\frac{\zeta }{\zeta +\lambda }(I+D)$ of the infected people $I+D$ will develop symptoms in the future and further that a total of $\frac{\mu }{\mu +\kappa }\left(\frac{\zeta }{\zeta +\lambda }(I+D)+A+R\right)$ will become threatened. Thus, assuming the capacity $T_{\text{ICU}}$ is not exceeded afterwards, i.e., setting constant values $\tau =\tau \left(0\right)=\frac{{\mu }_1}{\mu }{\tau }_1+\frac{{\mu }_2}{\mu }{\tau }_2$ and $\sigma =\sigma \left(0\right)=\frac{{\mu }_1}{\mu }{\sigma }_1+\frac{{\mu }_2}{\mu }{\sigma }_2$, the amount of inevitable fatalities is exactly given by

$$F=E+\frac{\tau }{\tau +\sigma }\left(\frac{\mu }{\mu +\kappa }\left(\frac{\zeta }{\zeta +\lambda }(I+D)+A+R\right)+T\right).$$ (10)

Hence, given a baseline solution $u^b,x^b$ from Section 3.1.2, the corresponding optimal control problem reads as follows:

$$\underset{u\left(\cdot \right)}{min}F\left(N\cdot T_s\right)$$ (11a)

$$0\le u\left(k\cdot T_s\right)\in [0,1]k=0,\dots N-1$$ (11c)

$$\sum _{k=0}^{N-1}{c}_{\text{policy}}\left(u\left(k\cdot T_s\right)\right)\le \sum _{k=0}^{N-1}{c}_{\text{policy}}\left(u^b\left(k\cdot T_s\right)\right).$$ (11d)

Since we only change the policy every week, the index $k$ in (11) corresponds to weeks and $F\left(N\cdot T_s\right)$ corresponds to the objective function in (10), where the states result from simulating the system (1) with the parameters and the initial condition from Section 2 and the input $u\left(\cdot \right)$ over $k$-weeks. Condition (11d) ensures that the encountered social cost is smaller than the cost of the baseline policy. This optimal control problem is such that the baseline policy $u=u^b$ is a feasible solution and thus the resulting fatalities $F\left(N\cdot T_s\right)$ will always be lower than that of the baseline policy. We point out that it is possible to consider a more restrictive transient constraint on the policy cost instead of (11d), which is discussed in detail in Appendix B. The optimal control problem (11) can be formulated as a nonlinear program (NLP) and is subsequently solved using CasADi (Andersson et al., 2019).

Numerical results

For comparison and to implement the constraint (11d), we use the baseline policy from Section 3.1.2 with $X_{lower}=0.4$, $X_{upper}=0.7$, $n_{\text{steps}}=14$, $n_{\text{stab}}=14$, which overall is rather cautious and does not exceed the ICU capacity. The corresponding results for the baseline policy and the optimal control strategy can be seen in Fig. 5. Although the optimal control input yields initially (first $100$ days) a slightly larger number of infected individuals and thus slightly more fatalities in the first 200 days, the number of infected individuals is subsequently significantly lower and the overall number of fatalities is reduced to only 26%. The optimal controller allows for a smooth increase of the infection rate $\alpha$, while keeping the number of threatened individuals ($T$) consistently below the corresponding value of the baseline policy after the first 200 days, thus yielding a small number of fatalities. The rising number of infected individuals ($I$) at the end results from the finite-horizon and will be considered later in more detail.

Fig. 5.

Optimal control strategy (blue, solid), baseline policy (red, dashed) and ICU capacity (dotted, black).

We also consider a second baseline policy using $X_{lower}=0.6$, $X_{upper}=0.85$, $n_{\text{steps}}=12$, $n_{\text{stab}}=14$, which slightly relaxes the social policy, but also exceeds the ICU capacity. The result is shown in Fig. 6. We can see that in comparison to this second baseline policy the optimal policy significantly reduces the number of fatalities to only 39%. The optimal strategy is a lot more cautious in reducing the social policy, while the baseline is more aggressive and goes back and forth between increasing and decreasing $\alpha$, resulting in significant violations of the ICU capacity. Furthermore, the simple baseline policy results in a second wave as the restrictions are loosened too quickly, while the optimal strategy slowly but steadily increases $\alpha$ after the first 200 days, and thereby avoids a second wave.

Fig. 6.

More aggressive baseline: Optimal control strategy (blue, solid), baseline policy (red, dashed) and ICU capacity (dotted, black).

In both examples, a further observation should be highlighted. After an initial phase of containing the outbreak, the measures are slowly but steadily relaxed until a larger release at the end of the horizon. Similar behavior can be observed for many optimal control problems with finite horizons and is commonly referred to as “turnpike” behavior, which goes back to Dorfman, Samuelson, and Solow (1987). An explanation for this is that the consequences of decisions taken at later points in time mainly occur after the end of the horizon, such that a more aggressive policy towards the end is optimal, when only considering the finite two year horizon. Of course, one would not implement the “leaving arc” if the development of a vaccine would not be finalized after two years, since implementing such a policy may lead to an uncontrollable increase of infections towards the end of the time horizon in case that the model is inaccurate and should thus be avoided in practice. In Appendix A, we therefore discuss how adding “terminal constraints” to the optimization problem can prevent this turnpike behavior of the optimal solution, at the price of an increasing number of fatalities.

3.3. Discussion

If we compare the results in Section 3.2 with the consistent full lockdown from  Section 3.1.1, we can see that it is possible to appropriately increase $\alpha$ without exceeding the ICU capacities, while the consistent full lockdown strategy would require a lockdown that takes approximately a year to be effective. Hence, while a consistent lockdown can effectively minimize the number of deaths, this strategy is only viable in case this lockdown can be prolonged over the corresponding time horizon, unless a vaccine is developed earlier. On the other hand, both the optimal controller and the baseline controller allow for a significant relaxation of the lockdown (on average a doubling of $\alpha$), without significantly increasing the number of fatalities.

In comparison to the baseline policy suggested in Section 3.1.2, the optimal control policy results in a slower but smooth loosening of the distancing policies. Without increasing the social cost over the full time horizon, this optimal policy avoids any violation of the maximum ICU capacity and hence results in a significantly smaller fatality rate. We point out that the resulting optimal policy of slowly increasing $\alpha$ is qualitatively similar to the resulting optimal policy in Kantner and Koprucki (2020), albeit for a different control goal. It can be seen that an initially “stronger” lockdown (i.e., a smaller value of $\alpha$) over a longer time period with subsequent loosening leads to a better handling of the pandemic, compared to repeated tightening and loosening of distancing measures. Moreover, a smooth and monotone loosening of distancing policies is also desirable from an economic aspect since repeated lockdowns after interim-periods of relaxed distancing guidelines may be even more damaging to the economy, compared to an initially longer lockdown.

Comparing our results with the proposed scenarios by the Helmholtz Association (Helmholtz-Initiative ‘Systemische Epidemiologische Analyse der Covid-19-Epidemie’, 2020), we find that we agree that the goal of herd immunity without overwhelming the health care capacity would require years. Our results further agree with Helmholtz-Initiative ‘Systemische Epidemiologische Analyse der Covid-19-Epidemie’ (2020) that the contact restrictions can only be loosened slowly if the health care capacity must not be overwhelmed. However, since the authors in Helmholtz-Initiative ‘Systemische Epidemiologische Analyse der Covid-19-Epidemie’ (2020) do not consider the availability of a vaccine, their conclusion is keeping or even increasing the lockdown until the number of infected persons is small and all infections can be traced efficiently and effectively via strategically allocated (and increased) testing. With the assumption of a vaccine within the next two years, we argue that a slow and smooth loosening of the lockdown does not lead to many more fatalities while decreasing the social and economic cost significantly according to our model. Concurrently, increased and strategically better allocated testing, e.g., via a contact tracing mobile app (Oliver et al., 2020), is of course highly beneficial and greatly advisable to improve the performance (even if it this was not accounted for in our model).

To summarize, it seems possible to reduce the current restrictions and thus allow $\alpha$ to increase without exceeding the ICU capacity. Furthermore, optimized policies can significantly improve the outcome (in terms of fewer deaths and/or less social restrictions). However, the result is highly sensitive w.r.t. the change in the infection rate, while an accurate control of the infection rate $\alpha$ (e.g. $\pm 5\text{\%}$) through governmental policies seems difficult/unrealistic. In the next section, we will therefore deal with these issues by formulating a robust control strategy that takes uncertainty in our COVID-19 model into account and uses feedback based on uncertain state information.

4. Optimal feedback control of the COVID-19 outbreak

Section 3.2 shows that an optimal control policy can significantly reduce the number of fatalities compared to a baseline policy that allows for iterative loosening of social distancing measures. This optimal control policy is computed by optimizing over all possible policies to find the one minimizing the number of fatalities predicted by the model Eqs. (1) without using stronger shutdown measures than the baseline. Hence, the policy proposed in Section 3.2 strongly relies on the accuracy of the model identified in Section 2 and thus may fail to effectively control the outbreak in case of a model mismatch. However, such a model mismatch is inevitable in practice, especially since the model itself is a simplification of a much more complex reality and the identification outlined in Section 2.2 strongly depends on the (sparse) available data and the additional prior knowledge based e.g. on further studies concerning COVID-19, which also provide only estimates. In addition, the optimal control policy relies on exact knowledge of all states and on the assumption that values for $\alpha$ and $\gamma$ can be exactly imposed up to arbitrary precision via social distancing measures, both of which are unrealistic assumptions when applying the policy in practice.

In this section, we show how online measurements can be utilized via feedback to effectively and robustly control the German COVID-19 outbreak in the presence of uncertain parameters. More precisely, we illustrate that the optimal open-loop policy of Section 3.2 may lead to poor performance when applied to validation models with a different set of parameters, although these validation models result from adjusting only one prior assumption in the identification and still fit the past data well. On the other hand, we show that a model predictive control (MPC) feedback strategy, based on repeatedly computing an open-loop policy for the nominal model from Section 2, is inherently robust w.r.t. model inaccuracies and successfully handles the outbreak.

Basic idea

At each time step $k=1,\dots ,N$, where $k$ corresponds to weeks and $N=100$ as in Section 3.2, we solve the optimization problem (11) over the time horizon $k,\dots ,N$ using the current measurements as initial condition at week $k$. Then, we apply the computed optimal policy over one week before solving the problem for the new measurements again. In this way, since the initial conditions in the optimal control problem are updated, a feedback mechanism is included as is standard in MPC (Rawlings, Mayne, & Diehl, 2017). As a result, the prediction horizon $N-k$ of the optimization problem is shrinking with each time step $k$, such that it never exceeds the considered total time horizon of $N=100$ weeks, after which we assume the availability of a vaccine. Hence, since the constraint (11d) needs to hold over the whole time horizon $N$, we replace it by

$$\sum _{j=k}^{N-1}{c}_{\text{policy}}\left(u\left(j\cdot T_s\right)\right)\le c_{\text{policy}}^1-\sum _{j=0}^{k-1}{c}_{\text{policy}}\left(u\left(j\cdot T_s\right)\right),$$ (12)

with $c_{\text{policy}}^1={\sum }_{k=0}^{N-1}{c}_{\text{policy}}\left(u^b\left(k\cdot T_s\right)\right)$ being the cost of the first baseline policy in Section 3.1.2. As a second modification, we adapt the bound on the social distancing cost online, depending on the predicted states, as is detailed in the following.

Online adaptation of social policy constraint

In Section 3.2, we proposed an open-loop optimal control strategy, where the inputs were the infection rates $\alpha$ and $\gamma$. Loosely speaking, the control goal was to achieve a minimum number of fatalities without imposing stronger social distancing measures than a simple baseline policy (compare (11d)). Since this constraint heavily depends on the model to which the baseline is applied, a realistic setting with imperfect model knowledge should allow to adapt the constraints on the policy online in case that the nominal model is overly optimistic or pessimistic. Instead of simply requiring that the cost of the MPC-based feedback cannot exceed $c_{\text{policy}}^1$, we increase the maximum cost in case that the predicted number of patients requiring intensive care lies above 90% of the maximum capacity $T_{\text{ICU}}$ at least once during the horizon, and we decrease it in case that the number consistently lies below 10% of $T_{\text{ICU}}$. This adaption is natural, as in reality one would increase the efforts to contain the outbreak if the current measures are insufficient, and on the other hand, the population cannot be expected to accept strict measures when there are only few (severe) cases across the country. Therefore, the maximum cost in week $k$, denoted by $c_b\left(k\right)$, varies online with $k$ and is initialized as $c_b\left(0\right)=c_{\text{policy}}^1$. The amount by which we change $c_b$ online is $\pm {\Delta }_u\frac{N-k}{N}$, where ${\Delta }_u=\frac{1}{{\alpha }_{min}}-\frac{1}{{\alpha }_{max}}$ with ${\alpha }_{min}$ and ${\alpha }_{max}$ as in Section 2.2. If, for instance, the model predicts large numbers of future ICU patients, then the cost bound $c_b$ is increased by the difference between the minimum and the maximum social distancing cost, scaled by the remaining time horizon. This increase corresponds to the social cost of an additional week in full lockdown, scaled by the remaining time horizon via the factor $\frac{N-k}{N}$.

MPC-based feedback strategy

The proposed MPC-based feedback strategy is summarized in Algorithm 1. In the algorithm, $T(j\cdot T_s\mid k\cdot T_s)$ denotes the number of threatened individuals at time $j\cdot T_s$, predicted by the optimal solution of (11) at time $k\cdot T_s$. Essentially, the algorithm repeatedly applies the open-loop optimal control policy of Section 3.2 with the key difference that, at time $k$, all past measurements $j=1,\dots ,k$ are used in the optimization problem, thus including an online feedback. In addition, in Step 3 of the algorithm, the social policy constraints is adapted as described above.

Validation

To assess the improved robustness of Algorithm 1 compared to open-loop optimal control, we produce two validation models. More precisely, we identify two new sets of parameters A and B by proceeding exactly as in Section 2.2 with the only difference that we change the prior assumption that the stationary ratio of confirmed COVID-19 cases is in the interval $[0.3,0.45]$. Instead, we assume that this value is in $[0.3,0.6]$ for set A and in $[0.3,0.4]$ for set B. When performing parameter identification based on these modified prior assumptions, we also obtain sets of parameters that can accurately explain the existing past data on COVID-19 cases in Germany. However, the resulting models have different dynamics and different reproduction rates for the same lockdown policy. Increasing the above ratio as in parameter set A decreases the number of infected and undetected individuals resulting in a higher reproduction rate to explain the same amount of confirmed cases. Hence, if an open-loop policy based on the nominal model (i.e., with parameters described in Section 2.2) is applied to the validation model with parameters A, then the number of infections and thus the number of fatalities increases significantly.

To illustrate this effect, we apply the open-loop optimal control policy based on the model with parameters as in Section 2.2 to the new models with parameter sets A and B. The control effort of this policy, i.e., the amount of social distancing, is constrained as in (11d) by the cost of the baseline policy when applied to the nominal model identified in Section 2.2. The results for the model with parameters A can be seen in Fig. 7. Since this validation model has a higher reproduction rate for similar inputs as explained above, the number of fatalities after $N=100$ weeks increases significantly compared to the simulations in Section 3. This is due to the fact that the open-loop policy is only computed once, at the beginning of the time horizon, and is then applied over the whole time span of two years without any online adaption based on new measurements. Therefore, it cannot handle the model mismatch and thus has a significantly worse performance. In addition, Fig. 7 shows the evolution under the proposed MPC-based feedback, which leads to a significantly lower number of fatalities compared to the open-loop policy. We point out that the feedback (partially) compensates the fact that the control action is computed based on the nominal model parameters from Section 2.2, which differ significantly from parameter set A. Due to the larger number of infected individuals, the maximum social cost $c_b$ is increased at multiple time steps, which is indicated by the step-like increases of the input. Finally, an open-loop optimal control policy is computed which is allowed to use the same amount of resources as the MPC-based feedback (in hindsight), i.e., the adapted social and economical cost $c_b\left(N\right)$. While this policy performs better than the initial open-loop policy with fewer resources, it leads to a similar number of fatalities at the end of the horizon compared to the feedback controller. However, the number of threatened patients is very large at time $k=N$, which would lead to a significant increase in fatalities after the considered time period, even if a vaccine is available.

Fig. 7.

Different policies designed based on the nominal model identified in Section 2.2, applied to the validation model with parameter set A (top) and parameter set B (bottom): Open-loop optimal control strategy (‘OL’, blue dotted), Open-loop optimal control strategy with the same amount of resources as the MPC-based feedback (‘OL-resource’, red dashed), MPC-based feedback strategy as in Algorithm 1 (yellow solid), and ICU capacity (black dotted).

Fig. 7 also shows the same comparison for the model with parameter set B. In this case, since the reproduction rate is lower, the open-loop optimal policy leads to fewer fatalities than in Section 3.2. The MPC-based feedback leads to almost identical performance, but it reduces the cost budget at several time instants, i.e., it can handle the outbreak similarly well but with significantly lower social and economic costs. When restricting the budget of the open-loop optimal policy to the one of the feedback strategy, i.e., $c_b\left(N\right)$, then it leads to a dramatic increase in fatalities.

To conclude, the above discussion reveals that a combination of open-loop optimal control with feedback is inherently robust in the sense that it effectively controls the German COVID-19 outbreak even if the employed model is inaccurate. When comparing the result to an open-loop strategy, then the MPC-based feedback strategy can dramatically decrease the number of fatalities or the necessary amount of social distancing, respectively. Such robustness is an important property for applying any control strategy in a real-world scenario, where accurate model knowledge is rarely available. In the next section, we propose a more systematic robust MPC approach which explicitly takes model inaccuracies as well as uncertain state measurements and control inputs into account in order to safely and cautiously control the COVID-19 pandemic.

5. Robust and optimal feedback control of the COVID-19 outbreak

While the MPC-based feedback policy proposed in Section 4 is significantly more successful in handling the outbreak compared to a simple open-loop policy, it relies on the assumption that exact measurements of the state in (1) are available at each time step. In this section, we consider a more realistic scenario of uncertain measurements in terms of biased state estimates, and we analyze the impact on the closed-loop operation. In particular, we develop a robust MPC-based feedback strategy using interval arithmetic that takes the uncertainty into account during the predictions and thus leads to a safer policy minimizing the number of fatalities.

Biased state measurements

In the following, we consider the case where at each day $k$ instead of the true state $x\left(k\right)$ we only obtain an estimated state $\hat{x}\left(k\right)$, which is subject to an additional bias. In Table 2, we summarize the uncertainties in the states. For individuals in states $D$ and $R$, the disease COVID-19 was detected by tests. Hence, their values are well known, nevertheless, we assume that they can slightly differ from the true states by $\pm 1\text{\%}$, as there might be cases on the borderline between $D$ and $R$ that are hard to assign to either of the states. The number of people in ICUs is well documented. However, the state $T$ contains not only patients in ICUs ($T_{ICU}\pm 1\text{\%}$) but also other infected members of the risk group ($T_2\pm 5\text{\%}$), cf. Section 2.1, such that the uncertainty we use is $\pm (\frac{{\mu }_1}{\mu }\cdot 1\text{\%}+\frac{{\mu }_2}{\mu }\cdot 5\text{\%})$. We assume that the number of deaths is certain by $\pm 1\text{\%}$ as it includes some people that died of different causes. As the undetected cases can by definition not be measured, they must be estimated using random sampling or strategies like (Bommer & Vollmer, 2020). Therefore, the states $I$ and $A$ are much less certain, especially without symptoms ($I\pm 50\text{\%}$, $A\pm 20\text{\%}$). Recovering from the disease is a resulting state from both rather certain states, $D$ and $R$, and highly uncertain states, $I$ and $A$ such that overall it is uncertain itself ($H\pm 50\text{\%}$). The uncertainty of the state of susceptible persons $S$ results from the other states: $\hat{S}={\hat{x}}_1=1-{\sum }_{i=2}^8{\hat{x}}_i$.

Table 2.

Uncertainties in the states.

$I\pm 50\text{\%}$	$D\pm 1\text{\%}$	$A\pm 20\text{\%}$	$R\pm 1\text{\%}$	$H\pm 50\text{\%}$	$E\pm 1\text{\%}$
$T\pm (\frac{{\mathrm{\mu }}_1}{\mathrm{\mu }}\cdot 1\text{\%}+\frac{{\mathrm{\mu }}_2}{\mathrm{\mu }}\cdot 5\text{\%})$

It is possible to directly use this biased state estimate $\hat{x}\left(k\right)$ in Algorithm 1 and compensate the bias through the inherent robustness in the feedback implementation. In the following, we derive an alternative robust formulation that explicitly considers the uncertainty in the prediction.

Interval predictions

First, given a biased state estimate $\hat{x}\left(k\right)$ and known bounds on the bias (Table 2), it is possible to compute interval bounds $\underset{\_}{x}\left(k\right)$, $\overline{x}\left(k\right)$ such that the true state is guaranteed to lie in that interval, i.e., $x_i\left(k\right)\in [{\underset{\_}{x}}_i\left(k\right),{\overline{x}}_i\left(k\right)]$. The following formulation will predict the interval bounds ${\underset{\_}{x}}_i$ and ${\overline{x}}_i$ instead of using some nominal prediction. This methods is similar to interval arithmetic employed in robust MPC (Limon, Bravo, Alamo, & Camacho, 2005) and the robust moment enclosure for an SEIV epidemic model in Watkins, Nowzari, and Pappas (2019). Using the fact that the system is positive ($x_i$ and all the parameters are positive), it is possible to derive an interval prediction of the form

$$\dot{\overline{x}}=\overline{f}(\underset{\_}{x},\overline{x},u),$$ (14a)

$$\dot{\underset{\_}{x}}=\underset{\_}{f}(\underset{\_}{x},\overline{x},u),$$ (14b)

which ensures that $x\left(0\right)\in [\underset{\_}{x}\left(0\right),\overline{x}\left(0\right)]$ implies $x\left(t\right)\in [\underset{\_}{x}\left(t\right),\overline{x}\left(t\right)]$ for all $t\ge 0$, given suitable bounds on the uncertain parameters in the system model (1). The detailed derivation of the interval prediction model (14) can be found in Appendix C (more precisely, Eqs. (C.4)). Since deriving reliable bounds on all parameters in the model (1) is rather difficult or unnecessarily complex, we only focus on the uncertainty associated with the infection rate $\alpha$. In particular, we consider an uncertainty of $\pm 5\text{\%}$ on the infection rate $\alpha$. Thereby, we explicitly consider the problem that the infection rate cannot be precisely specified via social distancing measures. We will see later in the simulations that, although we do not account for all possible mismatches in the prediction model, we nevertheless obtain the desired properties in closed loop.

Given this interval prediction model, the proposed robust formulation now predicts an interval for the different state variables and minimizes the worst-case number of fatalities $\overline{F}$ based on $\overline{x}\left(N\cdot T_s\right)$. The overall procedure is summarized in Algorithm 2.

Numerical results

In the following simulations, we consider the extreme case where the number of estimated infected or previously infected individuals ($I$, $D$, $A$, $R$, $T$, $H$, $E$) is underestimated. The results for the robust MPC and the nominal MPC in comparison to open-loop optimal control strategies for the two validation parameter sets A and B (compare Section 4) can be seen in Fig. 8. Due to the worst-case prediction in the robust MPC, at $t=0$ the robust MPC already increases the resources two times, for both parameter sets, such that at $t=T_s$ the predictions satisfy $\frac{{\mu }_1}{{\mu }_2}\overline{T}\left(k\cdot T_s\right)\le 0.9T_{ICU}$.

Fig. 8.

Different policies designed based on the nominal model identified in Section 2.2, applied to the validation model with parameter set A (top) and parameter set B (bottom) with biased state measurements: Open-loop optimal control strategy (‘OL’, blue dotted), Open-loop optimal control strategy with the same amount of resources as the robust MPC-based feedback (‘OL-resource’, red dashed), nominal MPC-based feedback strategy as in Algorithm 1 with biased estimated state $\hat{x}\left(k\right)$ (yellow, dash-dot), robust MPC-based feedback strategy using Algorithm 2 (purple, solid), and ICU capacity (black dotted).

In the simulation with the model based on parameter set A, we can directly see that both the nominal MPC and the robust MPC reduce the number of fatalities compared to an open-loop optimal control strategy. Furthermore, if we compare the robust MPC and the nominal MPC, we can see that after $t=140$ days the nominal MPC implementation realizes that the spread is worse than initially assumed. This leads to a strong increase in social measures $u$ at $t=140$. With the robust formulation, $u$ decreases almost monotonically. Furthermore, the nominal implementation results in twice the number of fatalities, while the applied resources $c_{\text{policy}}$ over the two year horizon differ by less than ${\Delta }_u$, which corresponds to one week of lockdown. This indicates that the robust MPC, planning cautiously from the beginning, exploits its resources more efficiently by imposing stricter social distancing measures early on, which results to be beneficial in the long run.

For the second parameter set B, we can see that the worst-case robust formulation uses initially an unnecessarily high control effort. Nevertheless, overall the applied resources $c_{\text{policy}}$ differ by less than ${\Delta }_u$ compared to the nominal MPC, while the number of fatalities is still reduced by 33%. In comparison to the open-loop policies, both MPC policies require less or equally restrictive policy measures $u$, while the number of fatalities are significantly reduced.

6. Conclusions and discussion

In the following, we summarize our findings on a high-level and highlight the main take-away messages:

• •Our results in Section 3 confirm the conclusions in Meyer-Hermann et al. (2020) that neither eradication of the virus nor herd immunity without the availability of a vaccine are viable solutions to handle the current COVID-19 outbreak.
• •Applying an optimizer to the mathematical model describing the outbreak, one can significantly reduce the number of fatalities without increasing the costs associated to decreasing the infection rate (social distancing policies, closing schools, etc.), compare Section 3.2.
• •Since the proposed model can never exactly predict the COVID-19 pandemic, applying a nominal optimal policy introduces unnecessary conservatism, at best, up to posing a great danger (i.e. overwhelming the health care capacities risking high mortality rates). Therefore, our findings in Section 4 support (Meyer-Hermann et al., 2020) by showing that any policy to control the COVID-19 outbreak successfully has to be an adaptive strategy. This means we need to constantly measure, monitor and estimate the current numbers and adapt our policy accordingly, i.e., feedback is necessary for reliably handling the outbreak.
• •If we already a priori take into account that our model includes mismatches and that all measured and estimated numbers are not exact and can have a bias, we can further improve the outcome, as shown in Section 5. More specifically, we developed a robust MPC-based feedback strategy using interval arithmetic. The application of feedback without the robust description of the considered model can lead to intermediate increases in the number of new infections necessitating another period of lockdown. On the contrary, a robust feedback strategy can take these model mismatches and other uncertainties into account and is hence able to avoid such behavior, thus significantly reducing the number of fatalities.
• •When looking at the qualitative results the robust MPC-based feedback offers, one can see that, accounting for the instability and uncertainty of the spread of the virus, the controller suggests a rather strict policy at the beginning and only then allows for a gradual increase in the infection rate. Keeping this loosening slow at the beginning shows a beneficial effect in the long run. This qualitative result of the robust MPC underpins also the German policy and reaction to the outbreak of COVID-19 in Germany where initially strong measures (what we here refer to as lockdown) were applied. Only very recently the German government started to loosen these measures slowly and gradually.

There are also influences on the course of the outbreak that were not taken into account in the present paper but which are important in an overall strategy towards the spread of COVID-19 (e.g. increasing testing capacities, tracking of infections, as well as investigating which measures lead to the desired infection rate). However, controlling the infection rate is certainly one of the key factors and hence, this paper contributes towards mitigating the spread of COVID-19 under manageable societal and economic costs. We hope that the proposed feedback strategies inspire further investigations in this direction and offer qualitative and high-level insights that underpin the current policies or strategy papers.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A. Optimal control formulation using terminal constraints

In order to avoid artifacts of considering a finite-horizon problem (e.g. a lot of infected people at the end of the horizon), an alternative to considering the modified cost function F from (10) is the inclusion of additional terminal constraints for the contagious population $IDART=(I,D,A,R,T)\in {\mathbb{R}}^5$. In particular, we require that at the end of the control horizon $N$, the number of contagious individuals in each category ($I,D,A,R,T$) should be smaller than the corresponding number from the baseline policy (c.f. (A.1a)). In addition, at the end of the horizon the number of contagious individuals should be non-increasing, which is implemented as (A.1b).

Hence, in the following, we replace the cost $F$ by the number of fatalities $E$, and we add the following constraints to the optimal control problem (11) from Section 3.2:

$$IDART\left(N\cdot T_s\right)\le IDAR{T}^b\left(N\cdot T_s\right),$$ (A.1a)

$$IDART\left(N\cdot T_s\right)\le IDART\left((N-1)\cdot T_s\right).$$ (A.1b)

Again, the index $k$ in (11) corresponds to weeks and the states $IDART\left(k\cdot T_s\right)$ correspond to the result of simulating the system (1) with the parameters and initial condition from Section 2. These terminal conditions (A.1a)–(A.1b) (which should be interpreted element-wise) ensure that the final state after the finite horizon $N$ is “better” than the baseline solution (c.f. (A.1a)) and the outbreak can be contained (c.f. (A.1b)). The simulation results with the two baseline policies shown in Fig. A.9, Fig. A.10 demonstrate that the terminal constraints indeed effectively prevent the turnpike behavior. However, the additional constraints also lead to a slight increase in the number of fatalities.

Fig. A.9.

Cautious baseline: Optimal control strategy with terminal constraints (blue, solid), baseline policy (red, dashed) and ICU capacity (dotted, black).

Fig. A.10.

Aggressive baseline: Optimal control strategy with terminal constraints (blue, solid), baseline policy (red, dashed) and ICU capacity (dotted, black).

Appendix B. Alternative average constraint formulation

We wish to briefly mention a stronger restriction on the societal cost of the optimal control strategy. In particular, instead of only restricting the cost over the considered horizon of $N=100$ weeks, a stronger property is to ensure that at any time $t$, the previously accumulated policy cost is smaller than the corresponding cost of the baseline policy. This can be done by replacing condition (11d) with the following transient constraint:

$$\sum _{k=0}^{i-1}{c}_{\text{policy}}\left(u_1\left(k\right)\right)\le \sum _{k=0}^{i-1}{c}_{\text{policy}}\left(u_1^b\left(k\right)\right),$$ (B.1)

$$i=1,\dots ,N.$$ (B.2)

The corresponding results for both baselines considered in Section 3.2 can be seen in Fig. B.11. In this case the number of fatalities are reduced by 33% and 37%, respectively. Thus, also for this more restrictive setting, the optimal controller can significantly reduce the number of fatalities. In addition, in the comparison to the more aggressive baseline we also see that early measures are absolutely crucial, since the two policies are essentially equivalent in the period from 110 days–150 days, i.e., including the critical time period where the ICU capacity is exceeded, but differ significantly over 30 weeks prior to the violation of the ICU capacity.

Fig. B.11.

Transient constraint on social policy: Top: cautious baseline; Bottom: aggressive baseline. Optimal control strategy (blue, solid), baseline policy (red, dashed) and ICU capacity (dotted, black).

Appendix C. Interval predictions

In the following, we derive the dynamics of the interval predictions $\overline{f},\underset{\_}{f}$ used in (14). Note that the following property holds for any scalars $a\in [\underset{\_}{a},\overline{a}]$, $b\in [\underset{\_}{b},\overline{b}]$:

$$a\cdot b\in [\underset{\_}{a}\underset{\_}{b},\overline{a}\overline{b}],$$ (C.1)

if $\underset{\_}{a},\underset{\_}{b}\ge 0$. Furthermore, to ensure that $x\left(t\right)\in [\underset{\_}{x}\left(t\right),\overline{x}\left(t\right)]$ given $x\left(0\right)\in [\underset{\_}{x}\left(0\right),\overline{x}\left(0\right)]$, we only require

$${\dot{\overline{x}}}_i\ge {\dot{x}}_i\text{if}{\overline{x}}_i=x_i,$$ (C.2)

and similarly

$${\dot{\underset{\_}{x}}}_i\le {\dot{x}}_i\text{if}{\underset{\_}{x}}_i=x_i,$$ (C.3)

for all $i=1,\dots ,8$. Essentially, we use the property (C.1) together with the fact that $x_i$ and the parameters are positive to ensure that (C.2), (C.3) hold in order to derive the differential equations for the interval. More precisely, assuming that every parameter is uncertain in some bounds (e.g. $\beta \in [\underset{\_}{\beta },\overline{\beta }]$) yields the following $2\cdot 8$ ODEs:

$$\dot{\overline{S}}=-\overline{S}(\underset{\_}{\alpha }\underset{\_}{I}+\underset{\_}{\beta }\underset{\_}{D}+\underset{\_}{\gamma }\underset{\_}{A}+\underset{\_}{\beta }\underset{\_}{R}),$$ (C.4a)

$$\dot{\underset{\_}{S}}=-\underset{\_}{S}(\overline{\alpha }\overline{I}+\overline{\beta }\overline{D}+\overline{\gamma }\overline{A}+\overline{\beta }\overline{R}),$$ (C.4b)

$$\dot{\overline{I}}=\overline{S}(\overline{\alpha }\overline{I}+\overline{\beta }\overline{D}+\overline{\gamma }\overline{A}+\overline{\beta }\overline{R})-(\underset{\_}{ϵ}+\underset{\_}{\zeta }+\underset{\_}{\lambda })\overline{I},$$ (C.4c)

$$\dot{\underset{\_}{I}}=\underset{\_}{S}(\underset{\_}{\alpha }\underset{\_}{I}+\underset{\_}{\beta }\underset{\_}{D}+\underset{\_}{\gamma }\underset{\_}{A}+\underset{\_}{\beta }\underset{\_}{R})-(\overline{ϵ}+\overline{\zeta }+\overline{\lambda })\underset{\_}{I},$$ (C.4d)

$$\dot{\overline{D}}=\overline{ϵ}\overline{I}-(\underset{\_}{\zeta }+\underset{\_}{\lambda })\overline{D},$$ (C.4e)

$$\dot{\underset{\_}{D}}=\underset{\_}{ϵ}\underset{\_}{I}-(\overline{\zeta }+\overline{\lambda })\underset{\_}{D},$$ (C.4f)

$$\dot{\overline{A}}=\overline{\zeta }\overline{I}-(\underset{\_}{\theta }+\underset{\_}{\mu }+\underset{\_}{\kappa })\overline{A},$$ (C.4g)

$$\dot{\underset{\_}{A}}=\underset{\_}{\zeta }\underset{\_}{I}-(\overline{\theta }+\overline{\mu }+\overline{\kappa })\underset{\_}{A},$$ (C.4h)

$$\dot{\overline{R}}=\overline{\zeta }\overline{D}+\overline{\theta }\overline{A}-(\underset{\_}{\mu }+\underset{\_}{\kappa })\overline{R},$$ (C.4i)

$$\dot{\underset{\_}{R}}=\underset{\_}{\zeta }\underset{\_}{D}+\underset{\_}{\theta }\underset{\_}{A}-(\overline{\mu }+\overline{\kappa })\underset{\_}{R},$$ (C.4j)

$$\dot{\overline{T}}=\overline{\mu }\overline{A}+\overline{\mu }\overline{R}-(\underset{\_}{\sigma }\left(\overline{T}\right)+\underset{\_}{\tau }\left(\overline{T}\right))\overline{T},$$ (C.4k)

$$\dot{\underset{\_}{T}}=\underset{\_}{\mu }\underset{\_}{A}+\underset{\_}{\mu }\underset{\_}{R}-(\overline{\sigma }\left(\underset{\_}{T}\right)+\overline{\tau }\left(\underset{\_}{T}\right))\underset{\_}{T},$$ (C.4l)

$$\dot{\overline{H}}=\overline{\lambda }\overline{I}+\overline{\lambda }\overline{D}+\overline{\kappa }\overline{A}+\overline{\kappa }\overline{R}+\overline{\sigma }\left(\overline{T}\right)\overline{T},$$ (C.4m)

$$\dot{\underset{\_}{H}}=\underset{\_}{\lambda }\underset{\_}{I}+\underset{\_}{\lambda }\underset{\_}{D}+\underset{\_}{\kappa }\underset{\_}{A}+\underset{\_}{\kappa }\underset{\_}{R}+\underset{\_}{\sigma }\left(\underset{\_}{T}\right)\underset{\_}{T},$$ (C.4n)

$$\dot{\overline{E}}=\overline{\tau }\left(\overline{T}\right)\overline{T},$$ (C.4o)

$$\dot{\underset{\_}{E}}=\underset{\_}{\tau }\left(\underset{\_}{T}\right)\underset{\_}{T}.$$ (C.4p)

Since these dynamics only correspond to possibly conservative overapproximations, we can use ${\sum }_{i=1}^8{x}_i=1$ to possibly improve the resulting bounds for $S$ using the following projections: $\overline{S}\le 1-{\sum }_{i=2}^8{\underset{\_}{x}}_i$ and $\underset{\_}{S}\ge 1-{\sum }_{i=2}^8{\overline{x}}_i$. In principle it would also be possible to directly set $\overline{S},\underset{\_}{S}$ using the other states ${\overline{x}}_i,{\underset{\_}{x}}_i$ instead of simulating (C.4a)–(C.4b), but this may not necessarily ensure $\overline{S}\le 1$ and $\underset{\_}{S}\ge 0$.