Optimized numerical solutions of SIRDVW multiage model controlling SARS-CoV-2 vaccine roll out: An application to the Italian scenario

Abstract

In the context of SARS-CoV-2 pandemic, mathematical modelling has played a fundamental role for making forecasts, simulating scenarios and evaluating the impact of preventive political, social and pharmaceutical measures. Optimal control theory represents a useful mathematical tool to plan the vaccination campaign aimed at eradicating the pandemic as fast as possible. The aim of this work is to explore the optimal prioritisation order for planning vaccination campaigns able to achieve specific goals, as the reduction of the amount of infected, deceased and hospitalized in a given time frame, among age classes. For this purpose, we introduce an age stratified SIR-like epidemic compartmental model settled in an abstract framework for modelling two-doses vaccination campaigns and conceived with the description of COVID19 disease. Compared to other recent works, our model incorporates all stages of the COVID-19 disease, including death or recovery, without accounting for additional specific compartments that would increase computational complexity and that are not relevant for our purposes. Moreover, we introduce an optimal control framework where the model is the state problem while the vaccine doses administered are the control variables. An extensive campaign of numerical tests, featured in the Italian scenario and calibrated on available data from Dipartimento di Protezione Civile Italiana, proves that the presented framework can be a valuable tool to support the planning of vaccination campaigns. Indeed, in each considered scenario, our optimization framework guarantees noticeable improvements in terms of reducing deceased, infected or hospitalized individuals with respect to the baseline vaccination policy.

1. Introduction

In January 2020 the tremendous SARS-CoV-2 virus (the agent pathogen of the COVID19 disease) outbroke in the Chinese province of Hubei, with epicenter the city of Wuhan. The first detected infections in Italy date back to the 21st February, when two distinct cases have been detected in Veneto and Lombardy regions. From this point onward up to September 2022 more than 602 millions cases and 6.4 millions deaths have been recorded around the world according to the weekly reports available from the World Health Organization (WHO) (World Health Organization – Emergency Response Team, 2022), which has declared the pandemic alert since March 2020. Since the severe symptomatology and highly–transmissible nature of the disease, Public Health Authorities of many countries have reacted tempestively in order to minimize the infectious risk introducing Non-Pharmaceutical Interventions (so-called NPIs) such as the compulsory adoption of face-masks, hygienic precautions, several measures for minimizing contacts, imposing smart-working whenever possible and sometimes with different kinds of lockdown. In Italy, starting from March 2020 to June 2020 a strict lockdown has been imposed at all levels. Non-pharmaceutical Interventions have been a fundamental tool in order to minimize the spread of the virus and the generation of more severe variants. However, the virus has overtaken such preventive measures, letting many different variants arising, especially some of those particularly dangerous to be catalogued as Variants of Concern, VOC, by WHO: B.1.617.2 (Delta), B.1.117 (Alpha), P.1 (Gamma) and B.1.1.529 (Omicron). NPIs have represented the sole tool for facing the pandemic until December 2021, when the first vaccine has been approved by the American Food and Drugs administration (FDA) and by the European Medical Agency (EMA). Starting from January 2021 until September 2022 six different vaccines developed with different pharmaceutical techniques have been approved and employed in the Italian vaccination campaign: Pfizer mRNABNT162b2 (Comirnaty), COVID-19 Vaccine Moderna mRNA–1273 (Spikevax), Vaxzevria, Jcovden, Nuvaxovid (Novavax) and Valneva. Some of these vaccines require two administrations in order to acquire effectiveness against transmission and severe symptoms. One of the main difficulties that Italian Public Health authorities had to face during the planning stage of the vaccination campaign was the prioritization order across ages (at least for those for which the vaccine administrations had been safely tested) and working categories, alongside with the choice of the suitable elapsing time between subsequent administrations.

In this complex social and medical scenario, we set this work aiming at contributing to the knowledge acquired in the mathematical epidemiology field. Mathematicians have been very responsive, e.g. (Bertuzzo et al., 2020; Chinazzi et al., 2020; Gatto et al., 2020; Kuhl, 2021; Parolini et al., 2021). Typical mathematical approaches for modelling and forecasting scenarios consider Compartmental (Bertaglia and Pareschi, 2021; Bertozzi et al., 2020; Capistrán et al., 2022; Parolini et al., 2021; Rozhnova et al., 2021; Viana et al., 2021) and Agent-Based models (Gharakhanlou and Hooshangi, 2020; Kerr et al., 2021; Shamil et al., 2021; Wolfram, 2020). The former approach adopts dynamical systems for modelling all the necessary features of the disease, and can be easily adapted to available data for the calibration of the possibly many parameters involved. The latter, instead, is particularly focused on capturing the behavior of and interactions among individuals in specific exposure contacts (e.g. schools). In this work, we recast the main questions regarding the planning of the vaccination campaign in the framework of the Optimal Control Theory (Kirk, 2004). Some recent works have already investigated the use of optimal control techniques based on compartmental models, to act on the levels of NPIs to be implemented for minimizing infected (Araz, 2021; Lemecha Obsu and Balcha, 2020; Tsay et al., 2020; Zamir et al., 2020) or deceased individuals (Perkins and Guido, 2020; Richard et al., 2021), sometimes coupling the evolution of the states of infectiousness model with other opinion models as in (J Silva et al., 2021). Other works have dealt with the optimal allocation of vaccines, based on SIR-like models as in the case of (Libotte et al., 2020; Ziarelli, 2021) or taking into account the spatial heterogeneity (Lemaitre et al., 2022), as well as age-dependencies (Shim, 2021). However, a detailed classification and comparison of vaccination policies reducing infections, deaths or hospitalisations dependently on age is missing in the state-of-the-art. To the best of our knowledge, this is the first work providing an accurate analysis on the optimization of age-specific vaccination campaigns for SARS-CoV-2 vaccines and considering a state model that can be straight-forwardly calibrated with data that are commonly available in many countries. This model has been developed in order to describe the main and fundamental features of the disease and the main characteristics of the vaccination campaign, yet with a very moderate computational complexity. This is fundamental for the interpretation of results and for tackling the main numerical issues. Indeed, the numerical complexity of our approach is similar to that of an age-stratified SIR, with the advantage however of being completely adapted to the progression of COVID19 in Italy.

In particular, the present work displays optimally controlled vaccination policies under state constraints of an age-stratified compartmental model in an abstract framework which is then applied to the specific Italian case study during the first half of 2021, using real data available daily from the Dipartimento di Protezione Civile Italiana (Open acces DPC dataset, 2020). An extensive campaign of numerical tests shows that the presented optimal control framework can represent a valuable tool to support, starting from real epidemiological data, the planning of vaccination campaigns aimed at fulfilling specific goals (e.g. reduction of deceased, infected, hospitalized).

The outline of the paper is as follows. In Section 2 we illustrate the rationales behind the chosen compartmental model, detailing the specific features that allow to describe the COVID19 context. Moreover, we introduce the mathematical formulation of the optimal control problem and the main numerical technicalities. In Section 3 we apply the abstract framework to the Italian scenario highlighting the versatility and the potentialities of the implemented computational architecture. Lastly, in Section 4 the main conclusions deriving from the analysis are drawn.

2. Methods

In this section, we present the mathematical description of an optimal control problem aiming at the planning of a vaccination campaign which is able to achieve specific goals. In general, an optimal control problem is built upon three components: (1) the state problem, in our case the epidemiological model governing the transmission dynamics with the daily vaccination rates in each as control variables; 2) a cost function to be minimized, in our case the number of deceased, infected or hospitalized individuals; 3) the set of constraints that the control must satisfy, in our case the limitations on vaccine administration rate and the total vaccine stock. More precisely, in Section 2.1 we introduce the epidemiological differential model governing the control problem, while in Section 2.2 we introduce the optimal control problem. Finally, we provide details about the implemented numerical scheme in Section 2.3.

2.1. Mathematical model

We introduce a deterministic age-stratified compartmental model embodying the main features of SARS-CoV-2 transmission and vaccination campaign, cf. (Ivorra et al., 2020; Parolini et al., 2022; Ram and Schaposnik, 2021). More precisely, we consider (see also the flowchart in Fig. 1) the following system of DDEs on the time interval I = (0, T_f]

$$\begin{gathered} \{\begin{array}{l}\dot{S_i}=-\beta r_i\sum _{k\in N_A}\frac{S_i{C}_{ik}{I}_k}{N_i}-U_{1,i}\frac{S_i}{S_i+I_{u,i}}+{\mu }_R{R}_i, \\ \dot{I_i}=\beta r_i\sum _{k\in N_A}\frac{\left(S_i+{\sigma }_V{V}_i+{\sigma }_W{W}_i\right)C_{ik}{I}_k}{N_i}-\gamma I_i, \\ \dot{R_i}=\left(1-f_i\left(S_i,V_i,W_i\right)\right)\gamma I_i-U_{R,i}-{\mu }_R{R}_i, \\ \dot{D_i}=f_i\left(S_i,V_i,W_i\right)\gamma I_i, \\ \dot{V_i}=-\beta r_i\sum _{k\in N_A}\frac{{\sigma }_V{V}_i{C}_{ik}{I}_k}{N_i}+U_{1,i}\frac{S_i}{S_i+I_{u,i}}-U_{2,i}, \\ \dot{W_i}=-\beta r_i\sum _{k\in N_A}\frac{{\sigma }_W{W}_i{C}_{ik}{I}_k}{N_i}+U_{2,i}+U_{R,i}, \\ {S}_i\left(0\right)=S_{i,0},I_i\left(0\right)=I_{i,0},R_i\left(0\right)=R_{i,0},\\ D_i\left(0\right)=D_{i,0},V_i\left(0\right)=V_{i,0},W_i\left(0\right)=W_{i,0},\end{array}\forall t\in \mathrm{I},i\in N_A, \end{gathered}$$ (1)

where, as usual, each time dependent variable accounts for the number of individuals in different conditions with respect to the disease: Susceptible (S), Infectious (I), Recovered (R), Deceased (D) due to complications related to SARS-CoV-2, Vaccinated with a first dose administered (V) and Vaccinated who have completed the vaccination cycle (W). Actually, each state is split according to the index i ∈ N_A, where N_A is the set of indexes identifying the different age-classes. The parameters involved in the model are described in the following.

• ●T_f (days): final time of the simulation frame;
• ●β ∈ (0, 1): transmission rate, depending on the implemented Non-Pharmaceutical Interventions (NPIs) and virus transmissibility. It is assumed to be constant across all ages as in (Marziano et al., 2021);
• ●σ_V, σ_W ∈ (0, 1): vaccine effectiveness on transmissibility after administration of first dose (the former) or completing the cycle (the latter). It can be interpreted as the ratio of transmissibility between vaccinated individuals and unvaccinated ones. The value 0 means that the vaccine is fully effective, 1 totally ineffective;
• ●θ_V, θ_W ∈ (0, 1): vaccine effect on mortality after administration of first dose (the former) or completing the cycle (the latter). It can be interpreted as the ratio of probability of getting severe symptoms between vaccinated individuals and unvaccinated ones. The value 0 means that the vaccine is fully effective, 1 totally ineffective;
• ●IFR_i: age-dependant Infection Fatality Rate;
• ●The fatality function f_i(S_i, V_i, W_i) changes accordingly to the immunological profile of the population, i.e. taking into account the vaccination effect in reducing mortality:

$$f_i\left(S_i,V_i,W_i\right)=\left\{\begin{array}{ll}IF{R}_i\left(\frac{S_i\left(t-t_a\right)+{\theta }_V{\sigma }_V{V}_i\left(t-t_a\right)+{\theta }_W{\sigma }_W{W}_i\left(t-t_a\right)}{S_i\left(t-t_a\right)+{\sigma }_V{V}_i\left(t-t_a\right)+{\sigma }_W{W}_i\left(t-t_a\right)}\right)& t>t_a\\ IF{R}_i& 0\le t\le t_a\end{array}\right)$$ (2)

where t_a is the amount of days after the inoculation that we consider for reaching the complete vaccine effectiveness. Heuristically, the fatality function reduces the fatality rate IFR_i of a proper factor that takes into account the reduced probability of getting severe symptoms when vaccines have been inoculated (this effectiveness is ruled by the parameters θ_V, θ_W).

• ●C_ik: i, k-th entry of the contact matrix, tracing back contacts between ages starting from the POLYMOD surveys (Mossong et al., 2008);
• ●r_i: susceptibilities to infection depending on age;
• ●γ: recovery rate from the disease infection, which is maintained constant across ages. Since infectious individuals are supposed to exit from the correspondent compartment with flux γI_i, the parameter γ is interpreted as the inverse of the average time of recovery t_R;
• ●N_i: Number of individuals in the i-th age stratification;
• ●μ_R: natural waning immunity rate, taking into account plausible reinfections coming from previously-recovered individuals;
• ●I_u,i(t) = (1 − δ(t)) I_i(t), t ∈ I: approximate number of undetected individuals whose age falls in the i-th stratification at time t;
• ●U_1,i, U_2,i, U_R,i: daily amount of administered first doses, second doses and doses administered to the i-th age-class, respectively. The choice of these variables is coherent with actual implementation of the Italian vaccination campaign: two consecutive doses to be administered for completing the vaccination cycle to Susceptible individuals, one single administration to Recovered ones. To reduce the computational complexity of the model we assume that the functions U_1,i(t), U_2,i(t), U_R,i(t) are piecewise constant (constant on each week) and the weekly value of administrations is supposed to be equally distributed among each day of the week.

Remark

Since the horizon of interest of our simulations is reasonably short, we neglect migratory effects, births and deaths which are not COVID-related. Moreover, we neglect the possible presence of comorbidities fostering the onset of severe symptoms leading to death. Finally, there are other implicit assumptions that is worth underlining: there is no genetic mutation of the virus during the period of interest; vaccine effectiveness waning is not contemplated in the considered time interval.

Fig. 1.

SIRDVW model flowchart. Schematic flowchart of each age-stratification of the adopted SIRDVW multi-age model. The colored fluxes account for new infections and they explicitly embed interactions among age-classes.

The differential problem (1) has been endowed with proper initial conditions for each of the considered age-state compartment. Throughout this work, we consider time-invariant parameters except for the transmission rate β and the detection rate δ, which are assumed to depend on time. Especially for the transmission rate, which embodies many different effects, carrying out a calibration process is of paramount importance when dealing with realistic scenarios. In Section S2 of the Supporting Information one can find the calibration procedure employed for the Italian setting during the first half of 2021. On the other hand, more details about the choices of the parameters and the initial conditions can be retrieved in Section S1 (Supporting Information).

2.2. The optimal control problem

In this section we assume the perspective of Public Health Authorities asking whether it is possible to plan an optimal vaccination campaign in order to minimize some specific goals (total number of infected, deceased or hospitalized) and understand possible priority orders among age–classes. Let us specify the main features of the Optimal Control Problem:

Control Variables. We assume that we could control the weekly amount of doses to be distributed among susceptibles and individuals who already got a first dose, and each day of the same week we assume to distribute an equal amount of doses. The control variables are

$$0\le U_{1,i}\left(t\right),U_{2,i}\left(t\right)\le N_{week}\left(t\right),\forall i\in N_A,t\in \mathrm{I}.$$ (3)

In particular,

$${\left\{U_{1,i}\left(t\right)\right\}}_i,{\left\{U_{2,i}\left(t\right)\right\}}_i,N_{week}\left(t\right)\in {\mathbb{P}}_c^0≔\left\{f\left(t\right)=\sum _{k=0}^{N_{int}}{a}_k{\mathbb{1}}_{\left[T_{k-1},T_k\right]}\left(t\right),{\left\{T_k\right\}}_{k\le N_{int}}\in \mathbb{R},a_k\in \mathbb{R},\forall t\in \mathrm{I}\right\},$$ (4)

letting $N_{int}\in \mathbb{N}$ the number of weeks and ${\left\{T_k\right\}}_{k\le N_{int}}$ the set of first days of each week. In Section 3, unless otherwise specified the piecewise constant function N_week(t) has been obtained summing per-weeks the amount of actually administered doses in Italy during the considered period. Moreover, for reducing the computational complexity of the control problem and to adhere as much as possible with typical realistic scenarios (e.g. the Italian one), we assume that the numbers of administered second doses are set equal to the number of first administrations with a time delay imposed by pharmaceutical properties of the vaccine, i.e.

$$U_{2,i}\left(t\right)=U_{1,i}\left(t-{\delta }_w\right),\forall t\in \left({\delta }_w,T_f\right),i\in N_A,$$ (5)

where δ_w is the elapsing time among subsequent administrations.

Finally, the control variables need to fulfill a budget constraint imposed by the limited amount of available doses and due to other potential sanitarian restrictions, such as the limited personnel and infrastructures. In particular, this constraint reads as: ∀j = 0, …, N_int − 1

$$\sum _{i\in N_A}{\int }_{T_j}^{T_{j+1}}\left(U_{1,i}\left(t\right)+U_{2,i}\left(t\right)+U_{R,i}\left(t\right)\right)dt\le \min \left(N_S,N_{week}\left(T_j\right)\right),$$ (6)

with N_S is the budget limit due to sanitarian capabilities.

State Problem. The state problem is a revised version of (1) under assumption (5) on the administrations of first and second doses. For each age class i ∈ N_A and given initial conditions, the problem at each time instant t ∈ I reads as (1) in which (5) has been plugged in. In a compact way, the state problem can be rewritten as

$$\begin{array}{cc}{\dot{\mathbf{x}}}_i\left(t\right)& ={\mathbf{F}}_i\left(\mathbf{x}\left(t\right),\mathbf{x}\left(t-t_a\right),U_{1,i}\left(t\right),U_{1,i}\left(t-{\delta }_m\right)\right)={\mathbf{f}}_i\left(x\left(t\right)\mathit{,}x\left(t\mathit{-}{t}_a\right)\right)+\hat{\mathbf{b}}{U}_{\mathit{1}\mathit{,}i}\left(t\right)+\stackrel{\mathbf{\sim }}{\mathbf{b}}{U}_{\mathit{1}\mathit{,}i}\left(t\mathit{-}{\delta }_m\right)\mathit{\forall }t\mathit{\in }\mathit{(}\mathit{0}\mathit{,}{T}_f\mathit{]}\mathit{,}i\mathit{\in }{N}_A\mathit{,}\\ & \end{array}$$ (7)

with

$${\mathbf{x}}_i\left(t\right)=\left[\begin{array}{c}{S}_i\left(t\right)\\ I_i\left(t\right)\\ R_i\left(t\right)\\ D_i\left(t\right)\\ V_i\left(t\right)\\ W_i\left(t\right)\end{array}\right],\mathbf{x}\left(t\right)=\left[\begin{array}{c}{\mathbf{x}}_1\left(t\right)\\ {\mathbf{x}}_2\left(t\right)\\ {\mathbf{x}}_3\left(t\right)\\ {\mathbf{x}}_4\left(t\right)\\ {\mathbf{x}}_5\left(t\right)\end{array}\right],\widehat{\mathbf{b}}=\left[\begin{array}{c}-1\\ 0\\ 0\\ 0\\ 1\\ 0\end{array}\right]\mathrm{a}\mathrm{n}\mathrm{d}\tilde{\mathbf{b}}=\left[\begin{array}{c}0\\ 0\\ 0\\ 0\\ -1\\ 1\end{array}\right].$$ (8)

This compact form will be useful for the computation of the gradient to be adopted in the numerical optimization process.

Cost functionals. Assuming the perspective of policy makers, we consider three different cases.

• 1.
  $${\mathcal{J}}_D\left(\mathbf{x}\right)=\sum _{i\in N_A}{\int }_0^{T_f}{D}_i{\left(t\right)}^{2}dt,$$ (9)

i.e. the number of total deceased individuals during the whole process.

• 2.
  $${\mathcal{J}}_I\left(\mathbf{x}\right)=\sum _{i\in N_A}{\int }_0^{T_f}{I}_i{\left(t\right)}^{2}dt,$$ (10)

i.e. the number of total infected individuals during the whole process.

• 3.
  $${\mathcal{J}}_H\left(\mathbf{x}\right)=\sum _{i\in N_A}{\int }_0^{T_f}{H}_i{\left(t\right)}^{2}dt,$$ (11)

i.e. the number of total hospitalized individuals during the whole process. Indeed, one of the main issues to be faced during the first waves of the pandemic was dictated by the limited amount of beds and medical equipment for individuals affected by COVID19 exhibiting severe symptoms, especially in Intensive Care Units (ICU). However, we need an explicit computation of the hospitalized individuals as in (Marziano et al., 2021), since the SIRDVW does not provide directly the amount of hospitalized individuals as in (Parolini et al., 2022). In our case, for each age class we determine the amount of hospitalized individuals as

$$H_i\left(t\right)=h{\kappa }_i{I}_i\left(t\right)\left(\frac{S_i\left(t-t_a\right)+{\theta }_V{\sigma }_V{V}_i\left(t-t_a\right)+{\theta }_W{\sigma }_W{W}_i\left(t-t_a\right)}{S_i\left(t-t_a\right)+{\sigma }_V{V}_i\left(t-t_a\right)+{\sigma }_W{W}_i\left(t-t_a\right)}\right),$$ (12)

where κ_i is the age-dependent propensity for severe respiratory symptoms and h is an estimated fraction of hospitalized individuals with respect to the amount of infected (values retrieved from (Marziano et al., 2021)).

We are now ready to formulate the problem of the optimal vaccination campaign as an optimal control problem.

Optimal Control Problem. Find the doses administrations ${\widehat{U}}_{1,i}\in {\mathbb{P}}_c^0\forall i\in N_A$ such that it minimizes the chosen cost functional ${\mathcal{J}}_X$, i.e.

$${\left\{{\hat{U}}_{1,i}\right\}}_{i\in N_A}=\underset{{\left\{U_{1,i}\right\}}_{i\in N_A}\in {\mathrm{\mathbb{P}}}_c^0}{\mathrm{arg\; min}}{\mathcal{J}}_X\left(\mathbf{x}\right),$$ (13)

subject to the state problem (7) under the constraints (5) and (6).

2.3. Numerical procedure

To solve the optimal control problem (13) we employ an Inexact Projected Gradient Descent algorithm (Calamai and Moré, 1987), where the inexactness stems from employing an inexact adjoint system. More precisely, following (Abraha et al., 2021; Göllmann et al., 2009; Rodrigues et al., 2018) we define the Hamiltonian function of the optimal control problems with the aforementioned cost functionals, under the state problem 7, as

$${\mathcal{H}}_X\left(t\right)≔\sum _{i\in N_A}{X}_i^2\left(t\right)+{\mathbf{p}}_i{\left(t\right)}^T{\mathbf{F}}_i\left(\mathbf{x}\left(t\right),\mathbf{x}\left(t-t_a\right),U_{1,i}\left(t\right),U_{1,i}\left(t-{\delta }_m\right)\right),$$ (14)

where X ∈ {D, I, H} and p_i is the adjoint vector referring to the i-th age class. This function can be explicitly rewritten:

$$\begin{gathered} {\mathcal{H}}_X\left(t\right)=\sum _{i\in N_A}{X}_i^2\left(t\right)+p_i^1\left(t\right)\left(-\beta r_i\sum _{k\in N_A}\frac{S_i{C}_{ik}{I}_k}{N_i}-U_{1,i}\frac{S_i}{S_i+I_{u,i}}+{\mu }_R{R}_i\right) \\ +p_i^2\left(t\right)\left(\beta r_i\sum _{k\in N_A}\frac{\left(S_i+{\sigma }_V{V}_i+{\sigma }_W{W}_i\right)C_{ik}{I}_k}{N_i}-\gamma I_i\right) \\ +p_i^3\left(t\right)\left(\left(1-f_i\left(S_i,V_i,W_i\right)\right)\gamma I_i-U_{R,i}-{\mu }_R{R}_i\right) \\ +p_i^4\left(t\right)\left(f_i\left(S_i,V_i,W_i\right)\gamma I_i\right) \\ +p_i^5\left(t\right)\left(-\beta r_i\sum _{k\in N_A}\frac{{\sigma }_V{V}_i{C}_{ik}{I}_k}{N_i}+U_{1,i}\frac{S_i}{S_i+I_{u,i}}-U_{2,i}\right) \\ +p_i^6\left(t\right)\left(-\beta r_i\sum _{k\in N_A}\frac{{\sigma }_W{W}_i{C}_{ik}{I}_k}{N_i}+U_{2,i}+U_{R,i}\right). \end{gathered}$$ (15)

Consider t ≥ t_a and define the following auxiliary variables:

$$Z_i\left(t\right)≔S_i\left(t-t_a\right),O_i\left(t\right)≔V_i\left(t-t_a\right),P_i\left(t\right)≔W_i\left(t-t_a\right).$$ (16)

Plugging it into (15) we obtain:

$$\begin{gathered} {\mathcal{H}}_X\left(t\right)=\sum _{i\in N_A}{X}_i^2\left(t\right)+p_i^1\left(t\right)\left(-\beta r_i\sum _{k\in N_A}\frac{S_i{C}_{ik}{I}_k}{N_i}-U_{1,i}\frac{S_i}{S_i+I_{u,i}}+{\mu }_R{R}_i\right) \\ +p_i^2\left(t\right)\left(\beta r_i\sum _{k\in N_A}\frac{\left(S_i+{\sigma }_V{V}_i+{\sigma }_W{W}_i\right)C_{ik}{I}_k}{N_i}-\gamma I_i\right) \\ +p_i^3\left(t\right)\left(\left(1-IF{R}_i\frac{Z_i+{\sigma }_V{\theta }_V{O}_i+{\sigma }_W{\theta }_W{P}_i}{Z_i+{\sigma }_V{O}_i+{\sigma }_W{P}_i}\right)\gamma I_i-U_{R,i}-{\mu }_R{R}_i\right) \\ +p_i^4\left(t\right)\left(IF{R}_i\frac{Z_i+{\sigma }_V{\theta }_V{O}_i+{\sigma }_W{\theta }_W{P}_i}{Z_i+{\sigma }_V{O}_i+{\sigma }_W{P}_i}\gamma I_i\right) \\ +p_i^5\left(t\right)\left(-\beta r_i\sum _{k\in N_A}\frac{{\sigma }_V{V}_i{C}_{ik}{I}_k}{N_i}+U_{1,i}\frac{S_i}{S_i+I_{u,i}}-U_{2,i}\right) \\ +p_i^6\left(t\right)\left(-\beta r_i\sum _{k\in N_A}\frac{{\sigma }_W{W}_i{C}_{ik}{I}_k}{N_i}+U_{2,i}+U_{R,i}\right). \end{gathered}$$ (17)

Hence, ∀ i ∈ N_A the adjoint backward system (as illustrated in (Rodrigues et al., 2018)) reads as follows:

$$\begin{gathered} \{\begin{array}{l}\dot{p_i^1}\left(t\right)=-\frac{\partial {\mathcal{H}}_X}{\partial S_i}\left(t\right)-{\mathbb{I}}_{\left[0;T_f-t_a\right]}\frac{\partial {\mathcal{H}}_X}{\partial Z_i}\left(t+t_a\right), \\ \dot{p_i^2}\left(t\right)=-\frac{\partial {\mathcal{H}}_X}{\partial I_i}\left(t\right), \\ \dot{p_i^3}\left(t\right)=-\frac{\partial {\mathcal{H}}_X}{\partial R_i}\left(t\right), \\ \dot{p_i^4}\left(t\right)=-\frac{\partial {\mathcal{H}}_X}{\partial D_i}\left(t\right), \\ \dot{p_i^5}\left(t\right)=-\frac{\partial {\mathcal{H}}_X}{\partial V_i}\left(t\right)-{\mathbb{I}}_{\left[0;T_f-t_a\right]}\frac{\partial {\mathcal{H}}_X}{\partial O_i}\left(t+t_a\right), \\ \dot{p_i^6}\left(t\right)=-\frac{\partial {\mathcal{H}}_X}{\partial W_i}\left(t\right)-{\mathbb{I}}_{\left[0;T_f-t_a\right]}\frac{\partial {\mathcal{H}}_X}{\partial P_i}\left(t+t_a\right).\\ \end{array} \end{gathered}$$ (18)

with p_i(T_f)|_k = −2 X_i(T_f) δ_k,X as final conditions, and

$$\begin{array}{l}\frac{\partial {\mathcal{H}}_X}{\partial Z_i}=-IF{R}_i\gamma I_i{p}_i^3{\phi }_z+IF{R}_i\gamma I_i{p}_i^4{\phi }_z-{\kappa }_{i}h{I}_i{\phi }_z{\delta }_X^H,\\ {\phi }_z=\frac{\left(Z_i+{\sigma }_V{O}_i+{\sigma }_W{P}_i\right)-\left(Z_i+{\sigma }_V{\theta }_V{O}_i+{\sigma }_W{\theta }_W{P}_i\right)}{{\left(Z_i+{\sigma }_V{O}_i+{\sigma }_W{P}_i\right)}^2}\end{array}$$ (19)

$$\begin{array}{l}\frac{\partial {\mathcal{H}}_X}{\partial O_i}=-IF{R}_i\gamma I_i{p}_i^3{\phi }_o+IF{R}_i\gamma I_i{p}_i^4{\phi }_o-{\kappa }_{i}h{I}_i{\phi }_o{\delta }_X^H,\\ {\phi }_o=\frac{{\sigma }_V{\theta }_V\left(Z_i+{\sigma }_V{O}_i+{\sigma }_W{P}_i\right)-{\sigma }_V\left(Z_i+{\sigma }_V{\theta }_V{O}_i+{\sigma }_W{\theta }_W{P}_i\right)}{{\left(Z_i+{\sigma }_V{O}_i+{\sigma }_W{P}_i\right)}^2},\end{array}$$ (20)

$$\begin{array}{l}\frac{\partial {\mathcal{H}}_X}{\partial P_i}=-IF{R}_i\gamma I_i{p}_i^3{\phi }_p++IF{R}_i\gamma I_i{p}_i^4{\phi }_p-{\kappa }_{i}h{I}_i{\phi }_p{\delta }_X^H,\\ {\phi }_p=\frac{{\sigma }_W{\theta }_W\left(Z_i+{\sigma }_V{O}_i+{\sigma }_W{P}_i\right)-{\sigma }_W\left(Z_i+{\sigma }_V{\theta }_V{O}_i+{\sigma }_W{\theta }_W{P}_i\right)}{{\left(Z_i+{\sigma }_V{O}_i+{\sigma }_W{P}_i\right)}^2}.\end{array}$$ (21)

In light of the numerical evidence (see Section S4 in the Supporting Information) and in order to reduce the computational complexity of the problem involving all age stratification, we employ an inexact version of the adjoint problem 18, which is obtained from setting

$$\frac{\partial {\mathcal{H}}_X}{\partial Z_i},\frac{\partial {\mathcal{H}}_X}{\partial O_i},\frac{\partial {\mathcal{H}}_X}{\partial P_i}=0.$$ (22)

More precisely, the inexact adjoint system reads as follows

$${\dot{\stackrel{\sim }{\mathbf{p}}}}_i\left(t\right)=-\frac{\partial {\mathcal{H}}_X}{\partial {\mathbf{x}}_i}\left(t\right),{\stackrel{\sim }{\mathbf{p}}}_i\left(T_f\right){|}_k=-2X_i\left(T_f\right){\delta }_{k,X}\forall i\in N_A$$ (23)

where X ∈ {D, I, H} and ${\tilde{\mathbf{p}}}_i$ is the inexact adjoint vector referring to the i-th age class. To lighten the notation and with a slight abuse of notation, in the following we are going to denote by p_i the solution of the inexact adjoint problem.

In view of the above discussion, the main steps of the Inexact Projected Gradient Descent algorithm are hereafter summarized.

• -Choice of the initial guess. Set the initial guess of the control functions.
• -Optimization cycle. Fix a number of maximum iterations and at each iteration perform the following steps.
• 1.Solve the direct problem with the current control variable;
• 2.Solve the inexact time-reverse inexact adjoint problem (23);
• 3.Compute the descending direction for the numerical optimization scheme:

$$\delta U_{1,D_i}\left(t\right)=-\left({\mathbf{p}}_i{\left(t\right)}^T\widehat{\mathbf{b}}+{\mathbf{p}}_i{\left(t+{\delta }_m\right)}^T\tilde{\mathbf{b}}{\mathbb{1}}_{\left[0;T-{\delta }_m\right]}\right),\forall t\in \mathrm{I}.$$ (24)

In the sequel we briefly motivate this choice. Following the KKT conditions (Wright and Nocedal, 1999), define the Lagrangian function

$${\mathcal{L}}_h\left(\mathbf{x},\dot{\mathbf{x}},{\left\{U_{1,i}\right\}}_{i\in N_A},\mathbf{p}\right)=\sum _{i\in N_A}{\mathcal{L}}_{h,i}\left({\mathbf{x}}_i,{\dot{\mathbf{x}}}_i,U_{1,i},{\mathbf{p}}_i\right),$$ (25)

where,

$${\mathcal{L}}_{h,i}\left({\mathbf{x}}_i,{\dot{\mathbf{x}}}_i,U_{1,i},{\mathbf{p}}_i\right)={\int }_0^T{X}_i{\left(t\right)}^{2}dt+{\int }_0^T{\mathbf{p}}_i^T\left(t\right)\left({\mathbf{f}}_i\left(\mathbf{x},t\right)+\hat{\mathbf{b}}{U}_{1,i}\left(t\right)+\stackrel{\sim }{\mathbf{b}}{U}_{1,i}\left(t-{\delta }_m\right)-\dot{\mathbf{x}}\right)dt.$$ (26)

Compute the gradient of the Lagrangian at the continuous level in order to compute a descending direction for the optimal control algorithm. We obtain

(27)

Indeed,

(28)

Plugging (24) into (27) ensures $D_{U_{1,i}}{\mathcal{L}}_{h,i}\left(\delta U_{1,D_i}\right)<0$ which implies, in view of (7), that (24) is a descent direction for the cost functional ${\mathcal{J}}_X$.

• 4.Update the control variables at the current step as in the Projected Gradient Descent method with Armijo adaptive learning step:

$$U_{1,i}^{new}=\mathrm{\Pi }\left(U_{1,i}^{old}-\alpha \delta U_{1,i}\right)$$ (29)

where Π stands for the projection operator on the space of admissible controls (i.e. satisfying the constraints). At the discrete level, all the constraints are linear, therefore we applied the Shalev-Schwarz method (Shalev-Shwartz and Singer, 2006) for linear projection over a d-dimensional simplex.

Stopping criterion. The algorithm stops when the difference between two successive values of the cost functional is lower than a fixed tolerance tol.

3. Results

In this section we present and discuss the numerical results of the solution of the optimal control problem (13) in the context of the Italian third epidemic wave of SARS-CoV-2 (first half of 2021) corresponding to the beginning of the vaccination campaign. We are aware of the main limitations of the considered model (see Section 2.1), hence we do not intend to propose a critical retrospective analysis of the actual implemented vaccination policy. More realistically, we aim at: (1) extracting the main differences between the optimal solutions obtained minimizing different cost functionals (2) highlighting the structural features (e.g. in terms of age stratification); of the obtained vaccination strategies. In the following, the population is split in five compartments depending on age:

$$N_A=\left\{\left(0÷19\right),\left(20÷39\right),\left(40÷59\right),\left(60÷79\right),\left(80+\right)\right\}.$$ (30)

we set the optimization process in the period from February 12th, 2021, to June 1st, 2021, i.e. we consider 15 weeks. However, we carried out a calibration process of the model parameters in the six months starting on January 1st, 2021. The first month of the Italian vaccination campaign has been neglected in the optimization framework since it has been devoted to the immunisation of the sanitarian personnel. In this period the vaccination priority order has been dictated by a very specific political choice, impossible to be embodied in our age-stratified compartmental model that does not take into account for working classes. All the choices on the parameters, the results of the calibration process and the sensitivity analyses on the reproduction number can be found in Sections S1 and S2 (Supporting Information). We recover the initial conditions of each compartment by running a direct simulation of (7) starting on January 1st, 2021 and ending on February 12th, 2021. Indeed, the assumption of initializing to 0 all the vaccination states is reasonable on January 1st, 2021, and it helps in limiting uncertainty on the initial conditions to the other states (see Section S2). The implemented code is available in the dedicated GitHub repository (MOX Laboratory at Politecnico di Milano epiMOX research Group, 2022). Both the state and the adjoint problems have been solved employing a Runge-Kutta method of order 4, with a time step of 1 day. The gradient of the Hamiltonian function with respect to each state variable, necessary in (23), has been retrieved through automatic differentiation. In all our simulations the tolerance of the optimization iterative scheme has been fixed to 5e-4.

Each analysis is supported by quantitative results as the evolution of the cost functional, the optimized control variables by age-stratification and the discrepancies in terms of deceased, infected and hospitalized between the simulated optimal solution and the simulated solution with the starting policy. The discrepancies (with sign) are measured as follows.

• 1.Daily Variation of Infected. ${\mathrm{\Lambda }}_I:\mathbb{N}\to \mathbb{R}$

$${\mathrm{\Lambda }}_I\left(d\right)=\sum _{i\in N_A}{\left[{\mathbf{I}}_{i,IC}\right]}_d-{\left[{\mathbf{I}}_{i,OC}\right]}_d,\forall d\le N_{days},d\in \mathbb{N}.$$ (31)

where I_i,IC is the vector of daily infected belonging to the i-th age class, obtained from the solution of the state problem (7) where the daily amount of administered doses is set equal to the value actually administered in Italy. The vector I_i,OC contains the same type of informations computed by solving (7) with the optimal vaccination campaign coming from the solution of (13).

• 2.Daily Variation of Hospitalized. ${\mathrm{\Lambda }}_H:\mathbb{N}\to \mathbb{R}$

$${\mathrm{\Lambda }}_H\left(d\right)=\sum _{i\in N_A}{\left[{\mathbf{H}}_{i,IC}\right]}_d-{\left[{\mathbf{H}}_{i,OC}\right]}_d,\forall d\le N_{days},d\in \mathbb{N}.$$ (32)

where H_i,IC is the vector of daily hospitalized of the i-th age class of the solution of the state problem (7) with vaccination imposed as the vaccinations actually implemented in Italy, and H_i,OC is the amount of hospitalized computed with the optimal control variables.

• 3.Daily Variation of Deceased. ${\mathrm{\Lambda }}_D:\mathbb{N}\to \mathbb{R}$

$${\mathrm{\Lambda }}_D\left(d\right)=\sum _{i\in N_A}{\left[{\mathbf{D}}_{i,IC}\right]}_d-{\left[{\mathbf{D}}_{i,OC}\right]}_d,\forall d\le N_{days},d\in \mathbb{N}.$$ (33)

where D_i,IC represents the vector of daily deaths of the i-th age class of the solution of the state problem (23) with vaccination imposed as the vaccinations actually implemented in Italy, D_i,OC is the respective counterpart computed with the optimal control variables;

The results are organized as follows. In Sections 1, 2, 3 we present and discuss the optimal solutions obtained by minimizing deceased, infected and hospitalized, respectively. In Section 3.4 we explore the impact on the optimization of the vaccination policy of different choices for the initial guess, while in Section 3.5 we explore the impact of different constraints on the available amount of doses to be administered during the period of interest.

3.1. Minimization of deceased

In this section we optimize the vaccination campaign (i.e. the administration doses by ages) with respect to the total amount of deceased caused by SARS-CoV-2 infections, cf. (9):

$${\mathcal{J}}_D\left(\mathbf{x}\right)=\sum _{i\in N_A}{\int }_0^{T_f}{D}_i{\left(t\right)}^{2}dt.$$

The initial guess is chosen equal to the vaccination campaign actually implemented at the national level. In particular, from the DPC data (Open access DPC dataset vaccinations, 2021) we reconstruct the vaccination policies of first doses applied in Italy split by age-classes and we average the respective values on a weekly basis as shown in Fig SF.3 (Supporting Information). From Fig. 2 we notice that during the optimization process the cost functional has a descending behaviour corresponding to a reduction in terms of deceased of nearly 498 individuals at the final time. The algorithm stops after 951 iterations reaching the desired tolerance.Fig. 3 compares the amount of first doses administered in the optimal solution with the ones administered in the initial policy, while Fig. 4 shows the percentage repartition of doses across ages in the initial policy and in the optimal solution. In Fig. 3 we report the weekly total initial doses assigned to each age-stratification retrieved by data available from Dipartimento di Protezione Civile Italiana (dashed line) and obtained from the solution of the optimal problem (solid line). As one can notice from both Figures, the optimal strategy to minimize deceased suggests to increase the amount of doses to the over 80s while reducing the amount of doses to the other age classes. In particular, the younger is the age class, the higher is the relative amount of the reduction of doses. These results are not completely unexpected since they point out how the strategy to minimize deceased is to administer vaccinations to the elderly as much as possible (the highest increment in a week is of more than 150 thousands doses). Indeed, the over 80s represent the most fragile in terms of probability to contract the infection in a severe and, consequently, fatal way (see the values of the IFR_i in Table ST1 in the Supporting Information). Moreover, Fig. 4 compares the percentage repartition across age-classes of the vaccinations of the initial with the ones of the optimal policy. Notice that the optimal policy addresses to the over 60s from the 69% to the 88% of doses in each week, whilst the same quantity ranges from 43% to 81% with the initial policy.

Fig. 2.

Cost functional evolution for the case of minimization of deceased. Evolution of the cost functional during the optimization process. X-axis reports the iterations of the scheme.

Fig. 3.

Optimal control variables minimizing deceased. Weekly amount of doses delivered for each age-stratification.

Fig. 4.

Age-distribution of doses in the optimal policy minimizing deceased. Percentage repartition of doses across ages in the optimal solution (left) and in the initial policy (right).

However, as one can expect, this is not the best solution in terms of other quantities of interest such as infected and hospitalized. Indeed, from Fig. 5, Fig. 6, Fig. 7 it turns out that the optimal solution obtained from the minimization of deceased produces an increase in terms of infected and hospitalized especially at the final time (red curves assess a negative Variation of Infected and Hospitalized, while green curve assess a positive Variation of Deceased, cf. (31), (32), (33).

Fig. 5.

Absolute value of the Variation of Deceased measure (Λ_D).

Fig. 6.

Absolute value of the Variation of Infected measure (Λ_I).

Fig. 7.

Absolute value of the Variation of Hospitalized measure (Λ_H).

3.2. Minimization of infected

In this section we present the results of the optimal vaccination policy minimizing infected individuals during the whole time frame (cf. (10)):

$${\mathcal{J}}_I\left(\mathbf{x}\right)=\sum _{i\in N_A}{\int }_0^{T_f}{I}_i{\left(t\right)}^{2}dt.$$

In Fig. 8 we report the history of the cost functional in terms of the number of iterations: we observe a reduction of the number of infected individuals from 58.5 thousands (initial value) to 55.5 thousands (final value). The optimization process stops after 405 iterations.

Fig. 8.

Cost functional evolution for the case of minimization of infected. Evolution of the cost functional during the optimization process. X-axis reports the iterations of the scheme.

From Fig. 9, Fig. 10 we can extract the main features, in terms of age-class dose repartition, of the optimal vaccination campaign for minimizing infected. The reduction implied by the cost functional is guided by an unattended decrease in the amount of doses to the youngest and to the oldest age-classes against an increase of the amount of doses in the (20 ÷ 59) age-class. This result can be interpreted in light of the contact matrix weighted by the age-dependant susceptibility to the virus (Fig. 11). This matrix has been computed multiplying each column of the POLYMOD contact matrix by the age-dependant susceptibilities r_i. In this way each row takes into account the absolute amount of high-infection-risk contacts that one individual belonging to a specific age class has with individuals belonging to the others. Then, its values has been normalized through the maximum norm. In particular, the two rows corresponding to the age classes (20 ÷ 39) and (40 ÷ 59) achieve the highest values, meaning that individuals ageing (20 ÷ 59) are the ones having the highest probability of transmitting the virus heterogeneously across ages. Fig. 10 highlights how almost the totality of the delivered doses has to be administered at these age classes to the detriment of (0 ÷ 19), (60 ÷ 79) and (80+) classes. One may notice that the entry ((0 ÷ 19), (0 ÷ 19)) in the weighted contact matrix is also relatively high with respect to the others, even though vaccinations to this age-stratification are not promoted by the optimal controlled solution.

Fig. 9.

Optimal control variables minimizing infected. Weekly amount of doses delivered for each age-stratification.

Fig. 10.

Age-distribution of doses in the optimal policy minimizing infected. Percentage repartition of doses across ages in the optimal solution (left) and in the initial policy (right).

Fig. 11.

Normalized contact matrix weighted by age-dependant susceptibilities r_i.

Finally, let us notice that, similarly to the previous section, also in this case the optimal solution obtained by minimizing infected does not necessarily imply the minimization of deceased. Indeed, without protecting from the illness those who are more prone to severe outcomes, this optimal solution prescribes an increase in terms of deaths of more than 1750 units. On the other hand, hospitalized, which are strictly linked to the infected individuals, are correspondingly reduced even though their reduction was not directly contemplated directly in the definition of the cost functional (see Fig. 12, Fig. 13, Fig. 14, Fig. 15).

Fig. 12.

Absolute value of the Variation of Deceased measure (Λ_D).

Fig. 13.

Absolute value of the Variation of Infected measure (Λ_I).

Fig. 14.

Absolute value of the Variation of Hospitalized measure (Λ_H).

Fig. 15.

Cost functional evolution for the case of minimization of hospitalized. Evolution of the cost functional during the optimization process. X-axis reports the iterations of the scheme.

3.3. Minimization of hospitalized

In this section we collect the results of the optimal vaccination policy obtained by minimizing hospitalized individuals during the period of interest (cf. (11)):

$${\mathcal{J}}_H\left(\mathbf{x}\right)=\sum _{i\in N_A}{\int }_0^{T_f}{H}_i{\left(t\right)}^{2}dt.$$

After 1947 iterations the algorithm stops satisfying the prescribed stopping criterion. During the whole time horizon, the amount of hospitalized is reduced by 55.3 thousands individuals. This is an important remark, since during the first two waves of COVID19 epidemic one of the greatest issues was dealing with the limited amount of ICUs (Intensive Care Units) in hospitals and the extremely high level of beds occupancy due to people with respiratory symptoms linked to SARS-CoV-2. Reducing hospitalized individuals is one of the main objectives to be taken into account assuming a social perspective. Fig. 16 shows which is the optimal strategy to reduce the amount of hospitalized individuals. It is suggested to increase the amount of doses to administer to the age stratification (20 ÷ 59), without neglecting the administrations to the over 60s, but reducing the administrations to the (0 ÷ 19). Indeed, the age-classes to which an higher absolute amount of doses has to be provided are the ones more prone to host the virus in a more severe way. This policy is confirmed by Fig. 17 representing the age repartition of doses.

Fig. 16.

Optimal control variables minimizing hospitalized. Weekly amount of doses delivered for each age-stratification.

Fig. 17.

Age-distribution of doses in the optimal policy minimizing hospitalized. Percentage repartition of doses across ages in the optimal solution (left) and in the initial policy (right).

Lastly, we note that the optimal solution that reduces the most the amount of hospitalized individuals during the whole time frame is actually a solution increasing the amount of deceased (see Fig. 18, Fig. 19, Fig. 20). Indeed, in the optimal policy the amount of doses delivered to the most fragile people (the elderly) is slightly reduced with respect to the ones administered in the initial guess. Moreover, even though we have reduced the total amount of hospitalized, this solution does not improve the amount of infected with respect to the solution reducing the infectious in the interval of interest (Fig. 9). Although the total amount of infected in this solution is increased with respect to the one minimizing infectious, the value of hospitalized is lower due to the different repartition of infected across ages (notice that in (11) the infected belonging to different age-classes are weighted differently by age).

Fig. 18.

Absolute value of the Variation of Deceased measure (Λ_D).

Fig. 19.

Absolute value of the Variation of Infected measure (Λ_I).

Fig. 20.

Absolute value of the Variation of Hospitalized measure (Λ_H).

3.4. Optimal solution with different initial guesses

As the optimal control problem (13) is solved via a Projected Gradient Descent method, the solution is typically a local minimum and it is influenced by the specific choice of the initial policy. In view of this remark, in the present section we investigate the dependence on the chosen initial policy of the solution of (13). Particularly, we compare the results obtained in the previous sections (where the initial guess was set equal to the actual implemented national vaccination campaign) with the ones obtained starting from a different initial guess. The new initial guess that has been considered as starting policy is the homogeneous allocation of doses proportionally to the populosity of each age stratification. Specifically, let N_week(d) be the total amount of doses that have been distributed during the week at which day d belongs. Then, the initial guess of the amount of doses to be distributed each day of the week correspondent to d can be computed as

$$U_{1,i}^{\left(0\right)}\left(d\right)=\frac{N_{week}\left(d\right)}{7\left(\mathrm{d}\mathrm{a}\mathrm{y}\mathrm{s}\right)}\frac{N_i}{N}=\frac{N_{week}\left(d\right)}{7}\frac{N_i}{\sum _{k\in N_A}{N}_k},\forall d\in \mathrm{I},i\in N_A.$$ (34)

Fig. 21 shows the comparison between infected individuals generated by the optimal policy starting from the DPC-oriented policy (DPC-IG) and the one from the homogeneous doses initial guess (Homogeneous-IG). Instead, Fig. 22 displays the comparison of deceased between the two optimal solutions minimizing deceased, and lastly, Fig. 23 the trends of hospitalized when optimizing hospitalizations. The states have been simulated in a supplemental interval of 40 days, considering the calibrated values of the transmission rates and assuming that no first-dose administrations are allocated to each age class after the June 1st, 2021, this in order to have fair comparison among the solutions. The optimal policies retrieved in each case, together with their specific initial guesses (dashed lines), are presented in Fig. 24, Fig. 25, Fig. 26. For the solution optimizing the number of infected individuals, one can notice from Fig. 21 that the two solutions overlap almost completely during the whole time interval. However, the respective policies in Fig. 24 do not coincide: they both tend to allocate more than the 80% of the weekly available doses to individuals of age comprised between 20 and 59, neglecting completely the doses to administer to the older and the younger age-classes. However, the two solutions are actually different starting from April to the end of the simulation considering the doses to allocate to the (60 ÷ 79) years old. Hence, the solution starting with the DPC-based initial guess stops at a local minimum with non-zero administrations to this age-class; this is probably due to the influence of the chosen initial guess. On the other hand, the solutions minimizing deceased agree in administering more doses to the (80+) category, as we previously commented in Section 3.1 (see Fig. 25), although the curves of the optimal policies keep the same trend of the initial guesses which are different from the two simulations. The (Homogeneous-IG) optimal solution starts from a value of total deceased of 106244 and reduces deceased to the optimal value of 102657, whilst the other starts with an IG value of 103000 up to 102213 deceased. Therefore, the trend suggested by the initial policy proportional to the populosity of the compartment generate an higher reduction of deceased with respect to the initial guess, although the DPC-oriented one reaches an improved optimal value in terms of absolute amount of deceased. Finally, the solutions for the minimization of hospitalized (Fig. 23, Fig. 26) concur in increasing the amount of doses to the (20 ÷ 59) accordingly to the specific trend of the initial guess, and decreasing allocations corresponding to the other age-classes (see Fig. 19, Fig. 20 ).

Fig. 21.

Evolution of infected of the two solutions obtained minimizing infected starting from the two distinct initial guesses.

Fig. 22.

Evolution of deceased of the two solutions obtained minimizing deceased starting from the two distinct initial guesses.

Fig. 23.

Evolution of hospitalized of the two solutions obtained minimizing hospitalized starting from the two distinct initial guesses.

Fig. 24.

Weekly amount of doses delivered for each age-stratification in the solutions minimizing infected starting from DPC-IG and Homogeneous-IG.

Fig. 25.

Weekly amount of doses delivered for each age-stratification in the solutions minimizing deceased starting from DPC-IG and Homogeneous-IG.

Fig. 26.

Weekly amount of doses delivered for each age-stratification in the solutions minimizing hospitalized starting from DPC-IG and Homogeneous-IG.

However, the optimal policy starting from homogeneous initial guess allocates more doses to the over 80 with respect to the initial guess. This is not unexpected since individuals belonging to this age-class are the ones more prone to contract the disease in its most severe form, and so they require hospitalization more often than the other age-states. Hence, in the hospitalized-reduction case the social interactions responsible for the spread of infections and age-dependent frailties are two fundamental components that do not prevail with each other and have to be both taken into account during the administration process.

Summarizing, starting from different initial guesses, the optimal solutions return the same minimization values when minimizing infected and hospitalized, while there is a significant difference of about 444 deaths in the case of minimizing deceased. On the other hand, in all cases the optimal policies are different confirming the local nature of the optimization method used (PGD) and the impact of the initial guess. However, the increasing and decreasing trends by age of the vaccine doses in the optimal solutions are preserved in the case of minimization of the infected and the deceased, while for the hospitalized the doses to the elderly are different between the two solutions (increasing doses for the homogeneous solution with respect to the initial policy and decreasing the same amount for the solution starting with the DPC-oriented policy).

3.5. Optimal solution subject to different initial reproduction numbers

In this section we consider different scenarios where the constraints on the specific amount of doses administered in Italy in the first half of 2021 has been relaxed. In addition, we fixed the transmission rate during the whole time frame, the value having been selected so to deal with paradigmatic values of the initial reproduction numbers (i.e. smaller, slightly larger, larger than 1). Specifically, the right-hand side in constraint (6) is a constant function of time, imposed at the constant value of N_week = 2.1 millions doses per week, which is plausible approximation of the average amount of doses administered in Italy when the sanitarian capacity has reached its maximum. Moreover, following the scenario-analyses proposed in (Giordano et al., 2021), we fix the transmission rate during the whole time frame at three different values in order to achieve three different initial reproduction numbers, i.e. ${\mathcal{R}}_0=\left\{0.72,1.01,1.30\right\}$. In this way we consider optimal policies in presence of a minor epidemic (Case 1, ${\mathcal{R}}_0\simeq 0.7$), at the bifurcation value (Case 2, ${\mathcal{R}}_0,\simeq 1$) and in presence of a major outbreak (Case 3, ${\mathcal{R}}_0,\simeq 1.3$). The values of the transmission rate corresponding to the three reproduction numbers have been computed from the definition of the initial reproduction number for the SIRDVW model, that is

$${\mathcal{R}}_0≔\frac{\beta }{\gamma }.$$ (35)

Each scenario has been simulated starting from three different initial guesses, representative of possible different approaches to the vaccination campaign.

• ●IG1: each age class receives the proportion of total doses correspondent to the proportion of population of the same age-class on the total, i.e.

$$U_{1,i}^{\left(0\right)}\left(d\right)=\frac{1}{2}\frac{N_{week}}{7\left(\mathrm{d}\mathrm{a}\mathrm{y}\mathrm{s}\right)}\frac{N_i}{N_t}=\frac{N_{week}}{14}\frac{N_i}{\sum _{k\in N_A}{N}_k},\forall d\in I,i\in N_A.$$ (36)

The total amount of doses are halved between first and second administrations. This explains the coefficient $\frac{1}{2}$ appearing in (36) (and in (37)).

• •IG2: the amount of doses assigned to each age-class are proportional to the correspondent Infectious Fatality Rate, i.e.

$$U_{1,i}^{\left(0\right)}\left(d\right)=\frac{1}{2}\frac{N_{week}}{7\left(\mathrm{d}\mathrm{a}\mathrm{y}\mathrm{s}\right)}\frac{IF{R}_i}{IF{R}_t}=\frac{N_{week}}{14}\frac{IF{R}_i}{\sum _{k\in N_A}IF{R}_k},\forall d\in I,i\in N_A;$$ (37)

• ●IG3: we deal with a total amount of first doses administrations as a square wave with phase equal to three weeks and peak equal to the total weekly amount of doses. In each age class we distribute the amount of doses proportionally to the populosity as in IG1, i.e.

$$\begin{gathered} {N}_{sw,doses}\left(t\right)=\left\{\begin{array}{ll}{N}_{week}& t\in [21\left(2k\right),21\left(2k+1\right))\forall k\in \mathrm{\mathbb{N}},0\le t\le T_f,\\ 0& t\in [21\left(2k+1\right),21\left(2\left(k+1\right)\right))\forall k\in \mathrm{\mathbb{N}},0\le t\le T_f;\end{array}\right) \\ {U}_{1,i}^{\left(0\right)}\left(d\right)=N_{sw,doses}\left(d\right)\frac{N_i}{N_t},\forall d\in I,i\in N_A. \end{gathered}$$ (38)

The amount of second doses is analogously a square wave in counter phase with respect to the amount of first doses.

Firstly, we compare the solutions employing the initial guess IG1. As for the minimization of infected (refer to Fig. 27, Fig. 30) the optimal solutions concur in reducing the amount of administrations to the oldest (over sixties) and to the youngest (under twenties) age-classes. Moreover, as far as the initial reproduction number increases, the amount of administrations decreases. However, the amount of doses tends to increase with time as far as the outbreak runs out. In presence of a severe outbreak (Case 3, ${\mathcal{R}}_0=1.3$), the optimal vaccination roll-out indicates to administer over the 60% of doses to the (20 ÷ 39) age-classes, and the remaining part to the (40 ÷ 59), leading to the same conclusions of Subsection 3.2. However, the amount of doses allocated to the (20 ÷ 39) decreases with the depletion of the epidemic. In this case, with the optimal administrations the solution reaches a peak of infected which is dampened of approximately the half with respect to the initial guess, and it is advanced of nearly 23 days (see Fig. 27). Instead, considering the minimization of deceased (Fig. 28, Fig. 31) the optimal roll-out suggests to decrease the administrations to the (0 ÷ 19) and increase the amount of administrations to the oldest ones. However, in case of a sever outbreak a significant amount of allocations is associated to the (20 ÷ 39) and to the (40 ÷ 59), meaning that in this case it is difficult to contain the amount of deaths without reducing the spread of the disease carried out by the socially active population. This result is unexpected, since the amount of doses distributed to those age-stratifications deficits the allocations to the most fragile, i.e. the over eighties. The optimal solutions related to the severe outbreak allows to reduce deaths in the initial solution of nearly 23 thousands units in the major outbreak case, 8 thousands in the bifurcation one and almost 1.8 thousands in the case of a minor outbreak. Concerning the minimization of hospitalized (Fig. 29, Fig. 32) in the three cases the optimal vaccination campaigns agree in allocating the majority of available doses to the (20 ÷ 59) and to the elderly as in Subsection 3.3. However, the administrations to the most socially active population (20 ÷ 59) increases with the initial reproduction number. With the optimal solution the evident peak of hospitalizations of the initial guess in the case of a major outbreak is anticipated of nearly twenty days, whilst in the case of a minor outbreak the different vaccination policy does not lead to significant reduction in the curve of hospitalizations.

Fig. 27.

Evolution of infected prescribing the initial and the optimal vaccination policies minimizing infected starting from IG1 with three different ${\mathit{R}}_{\mathit{0}}$.

Fig. 30.

Weekly amount of doses delivered for each age-stratification in the solutions minimizing infected starting from IG1 and considering the three different outbreaks.

Fig. 28.

Evolution of deceased prescribing the initial and the optimal vaccination policies minimizing deceased starting from IG1 with three different ${\mathit{R}}_{\mathit{0}}$.

Fig. 31.

Weekly amount of doses delivered for each age-stratification in the solutions minimizing deceased starting from starting from IG1 and considering the three different outbreaks.

Fig. 29.

Evolution of hospitalized prescribing the initial and the optimal vaccination policies minimizing hospitalized starting from IG1 with three different ${\mathit{R}}_{\mathit{0}}$.

Fig. 32.

Weekly amount of doses delivered for each age-stratification in the solutions minimizing hospitalized starting from starting from IG1 and considering the three different outbreaks.

We now consider the initial guess IG2. As a general remark, we expect that policies maximizing the amount of doses to those who are more likely to contract the disease with severe symptoms, as the IFR-based initial guess IG2, is closer to the solution minimizing deceased with respect to the former. We first focus on the case of minimization of infected (see Fig. 33, Fig. 36), and we notice that the optimal policies consists in diminishing the amount of doses to the over sixties, increasing the (20 ÷ 59)'s allocations, and that the growth (respectively reduction) amounts depends on the respective value of the initial reproduction number. Indeed, administrations to the (20 ÷ 59) age classes increase as far as the reproduction number grows. In the case of a severe outbreak with the optimal solution we attain an early peak of nearly 20 days, and also the value at the peak has been lowered up to 451 thousands. On the other hand, the optimal allocation of doses for minimizing deaths tends to increase the amount of first doses to the (20 ÷ 59) to the detriment of the over sixties. Indeed, in the initial policy administrations to individuals in the (20 ÷ 39) and (40 ÷ 59) age-classes have been almost completely neglected due to the proportionality of doses with the ${\left\{IF{R}_i\right\}}_i$. The only optimal solution which significantly decreases the amount of deceased is the one related to the severe outbreak (reduction of nearly 4330 deaths). Lastly, for what concerns the optimal administrations for minimizing hospitalizations (see Fig. 34, Fig. 35, Fig. 36, Fig. 37, Fig. 38) the solution confirms that a sufficient amount of doses (almost 68%) has to be allocated to the (20 ÷ 59). In this way, for the case of a major outbreak there is the possibility of anticipating and reducing the peak of nearly two months. Actually, in the case of a minor outbreak the algorithm hardly moves from the initial guess, which seems to be almost optimal.

Fig. 33.

Evolution of infected prescribing the initial and the optimal vaccination policies minimizing infected starting from IG2 with three different ${\mathit{R}}_{\mathit{0}}$.

Fig. 36.

Weekly amount of doses delivered for each age-stratification in the solutions minimizing infected starting from IG2 and considering the three different outbreaks.

Fig. 34.

Evolution of deceased prescribing the initial and the optimal vaccination policies minimizing deceased starting from IG2 with three different ${\mathit{R}}_{\mathit{0}}$.

Fig. 35.

Evolution of hospitalized prescribing the initial and the optimal vaccination policies minimizing hospitalized starting from IG2 with three different ${\mathit{R}}_{\mathit{0}}$.

Fig. 37.

Weekly amount of doses delivered for each age-stratification in the solutions minimizing deceased starting from IG2 and considering the three different outbreaks.

Fig. 38.

Weekly amount of doses delivered for each age-stratification in the solutions minimizing hospitalized starting from IG2 and considering the three different outbreaks.

Finally, we run the numerical tests employing the initial guess IG3. From Fig. 39, Fig. 40, Fig. 41 we notice that all the optimal vaccination campaigns regardless of cost functionals tend to retain the square wave behavior of the initial guess. However, the absolute amount of doses is different depending on the cost functional and on the value of the initial reproduction number. Specifically, the optimal policy obtained from the minimization of infected suggests to allocate more doses to the (20 ÷ 59), and in the case of a severe outbreak to privilege administrations to the (20 ÷ 39) with almost 700 thousands doses weekly (see Fig. 39). On the other hand, the optimal solution from the minimization of deceased increases the amount of administrations to the most fragile, and even to the (20 ÷ 39) in the case of a major outbreak, in agreement with what has been previously remarked (Fig. 40). Finally, the optimal vaccination campaign for minimizing hospitalisations counsels to administer more doses to the over eighties and to the (20 ÷ 59), as far as the initial reproduction number grows. The solution minimizing hospitalized individuals is the one closer to the initial policy with respect to the other cases starting with the initial square wave policy (see Fig. 42, Fig. 43, Fig. 44 ).

Fig. 39.

Evolution of infected prescribing the initial and the optimal vaccination policies minimizing infected starting from IG3 with three different ${\mathit{R}}_{\mathit{0}}$.

Fig. 40.

Evolution of deceased prescribing the initial and the optimal vaccination policies minimizing deceased starting from IG3 with three different ${\mathit{R}}_{\mathit{0}}$.

Fig. 41.

Evolution of hospitalized prescribing the initial and the optimal vaccination policies minimizing hospitalized starting from IG3 with three different ${\mathit{R}}_{\mathit{0}}$.

Fig. 42.

Weekly amount of doses delivered for each age-stratification in the solutions minimizing infected starting from IG3 and considering the three different outbreaks.

Fig. 43.

Weekly amount of doses delivered for each age-stratification in the solutions minimizing deceased starting from IG3 and considering the three different outbreaks.

Fig. 44.

Weekly amount of doses delivered for each age-stratification in the solutions minimizing hospitalized starting from IG3 and considering the three different outbreaks.

In Table 1 we report, for different values of ${\mathcal{R}}_0$ and for different choice of the initial guess, the reductions of infected, deceased and hospitalized individuals with respect to the corresponding value obtained from the initial guess. We notice that as far as the reproduction number increases the projected gradient method is able to retrieve optimal policies corresponding to relevant improvement in the compartments to be minimized.

Table 1.

Amount of saved infected, deceased and hospitalized individuals with respect to the corresponding value obtained from the initial guess (for different values of the initial reproduction number and for different choice of the initial guess).

		IG1	IG2	IG3
Minimizing ${\mathcal{J}}_I$Average infected saved	${\mathcal{R}}_0=0.72$	5623	8711	2003
	${\mathcal{R}}_0=1.01$	29902	53140	20080
	${\mathcal{R}}_0=1.30$	158184	388583	147602
Minimizing ${\mathcal{J}}_D$Total Deceased saved	${\mathcal{R}}_0=0.72$	2175	30	250
	${\mathcal{R}}_0=1.01$	6051	469	1732
	${\mathcal{R}}_0=1.30$	22671	4330	21156
Minimizing ${\mathcal{J}}_H$Average hospitalized saved	${\mathcal{R}}_0=0.72$	1767	325	57
	${\mathcal{R}}_0=1.01$	7602	11242	1423
	${\mathcal{R}}_0=1.30$	42444	116299	39598

4. Discussion and conclusions

During epidemic waves, similar to the COVID19 pandemic, preventive medical and non-pharmaceutical interventions can significantly improve the epidemiological scenario, e.g. reducing the amount of infections or the associated deaths in case of severe illness. In particular, vaccinations for SARS-CoV-2 have confirmed their fundamental role played both in alleviating transmission effects and in easing severe symptoms. When the resources are stockpiled, it is of paramount importance to consider the complexity of the scenario where the problem is set, for instance embodying spatial or social heterogeneities. In this work, we presented an epidemic age-stratified compartmental model and formulated an optimal control problem for obtaining the vaccine distributions across ages, with the aim of minimizing specific goals. The model governing the optimal control problem is an age-stratified model, where the six compartments are split into five age-classes. It incorporates all the necessary features to effectively deal with the COVID19-vaccination campaign, and it can consistently incorporate the available data from DPC (Open acces DPC dataset, 2020, 2021). We decided to model the age dependency since the different reactions to the disease and the vaccine efficacy are mainly dependent on age for several diseases, including COVID19. Hence, it is natural to establish a vaccination plan on the basis of age. For instance, the Italian vaccination plan in 2021 was geared towards age distribution, giving special priority to the frailest individuals, the elderly, and to those presenting comorbidities. Following the Pontryagin-KKT approach, we derived the optimal control formulation in order to minimize infectious, deceased or hospitalized caused by the disease. Finally, we detailed the iterative numerical strategy (the projected gradient method) employed for the solution of the minimization problem.

After the introduction to the abstract mathematical framework, we adapted the optimal control problem to the Italian national level during the first six months of 2021, in order to provide guidelines to orient the vaccination campaign in this particular setting. It is worth remarking that the present study is not meant as a retrospective analysis of the actually implemented vaccination planning during COVID19 emergency, rather as a validation of a mathematical support tool to be possibly adopted in future emergency scenarios. The underlying assumption over the whole study is that there is no vaccine hesitancy, i.e. people are not mistrustful of the vaccine.

A first set of optimal vaccination problems has been solved by employing as initial guess for the minimization algorithm the vaccination campaign actually implemented in Italy during the first six months of 2021. The resulting optimal vaccination policies for reducing the amount of infected individuals suggest to administer more doses to people ageing (20 ÷ 59), reducing the other age-classes. This result can be explained in light of the contact matrix weighted by age-dependent susceptibility, and assessing that individuals in these age-classes are the ones more active from a social perspective. However, this strategy differs from the one aimed at minimizing the number of deceased individuals, which indeed requires to increase the amount of doses for the over eighties, who are the ones more prone to contract the disease in a more severe way. Finally, the solution minimizing the hospitalized individuals tends to increase the amount of doses to the age-class (20 ÷ 59) as in the minimizing-infected case.

The second set of simulations aimed at studying the impact of the initial guess for the minimization scheme on the results of the optimal control problem. In particular, we compared the vaccination policies from the previous set of experiments with the policies obtained by employing an initial guess that allocates to each age class an amount of doses proportionally to the numerousness of each age-compartment. In this case, we qualitatively obtained the same guidelines for the vaccination policy that we obtained in the first set of experiments, though the dependence of the solution on the initial guess (typical of gradient-based methods) clearly stands.

The last set of simulations explored the optimal vaccination policy under different levels of severity of the outbreak. In particular we assumed to deal with constant total amount of delivered doses in each week and constant value of the transmission rate, this latter dictating the severity of the outbreak. In this context, the introduced optimization framework was able to achieve significant improvements with respect to the initial policies especially in the major outbreak case, i.e. with a higher value of the initial reproduction number. Moreover, the optimally controlled solutions prescribe different optimal policies considering the different reproduction rates, though, as expected from the non-convex nature of the minimization problem, the output of the optimization algorithm depends on the choice of the initial guess.

The present work is not devoid of limitations. First, the employed optimization framework can be adopted only once the story of the total amount of vaccinations has been previously fixed. A possible approach could be to leverage on Model Predictive Control theory (Rawlings and Risbeck, 2017), allowing to adapt the optimal trajectory with new updates coming directly from new available data. A second limitation is related to our epidemiological model that only considers age-dependency, neglecting working categories and without distinguishing the different exposure contexts in which contacts happen and that present different transmission risks. This reflects in the contact matrix and in the transmission rates, that could be consequently adapted and modified. Another weak point relies in the amount of assumptions that has been made in terms of the control variables, such as the imposition of second doses on the basis of the firsts after a fixed amount of time. This hypothesis can be easily relaxed, at the expense of increasing the computational complexity of the algorithm, by allowing second doses to be independent control variables, and by adding an additional timing constraint between the control variables. Anyway, our assumption is in accordance with the vaccination policies implemented in several countries which decided to prescribe the second doses after a fixed number of days from the first inoculation. Finally, we focus on the uncertainty of the data adopted as a reference during the optimization process or during the calibration phase. Despite the data provided by the Italian Dipartimento di Protezione Civile can be straightforwardly analyzed and interpreted, and are available on a daily basis, they are obviously affected by uncertainty arising from the Italian narrow testing campaign, from reporting delays, the diffuse general mistrust in testing and other common sources of errors. In particular, data have no specifications on the characteristics about ages and comorbidities of most severe cases in hospitals, such as in ICUs and they do not specify on the prevalence of asymptomatic and symptomatic cases. The latter information could be fundamental in order to keep uncertainty under control, since the pattern of asymptomatic infections is the one that mainly contributes to herd immunity as well as to transmission of infections (Subramanian et al., 2021). One last cause of the uncertainty on the parameters is the difficulty in unquestionably attributing the broad spectrum of symptoms caused by COVID-19 to the disease itself. As a consequence of the previous discussion, wide uncertainty regions for both deceased and infected come out after the Bayesian MCMC calibration stage (see Section S1 for quantitative measures). We try to maintain under control the uncertainty propagating on the parameters by calibrating the model only on the amount of certified deaths ascribed to COVID19, which is, in our opinion, the most reliable datum. One possible strategy to address the issue of uncertainty is to handle model parameters using the posterior distributions reconstructed through the MCMC and to treat the optimization problem as a Stochastic Optimal Control problem. From a numerical perspective, optimization can be managed using the robust versions SA or SAA algorithms introduced in (Nemirovski et al., 2009).

In conclusion, this work introduced an optimal control framework to deal with the optimal allocations of vaccines against SARS-CoV-2 under different scenarios of epidemic transmission and vaccine availability. We have tested the proposed framework both in a realistic scenario (vaccination campaign in Italy in the period January 1st, 2021 to June 1st, 2021) and in artificial ones, paradigmatic of different level of severity of the outbreak. In all these contexts we analyzed the mechanisms interlacing our optimal age-prioritization allocation, the social interaction between age classes and the susceptibilities to infection depending on age. On one hand, we concluded that to achieve the maximum effectiveness (in terms of minimization of deceased, infected or hospitalized) requires non-trivial age prioritization vaccination strategies. On the other hand, besides inherent limitations, the proposed optimal control framework represents a useful tool to provide a benchmark for comparing different vaccine roll-out strategies.

Author contributions

Giovanni Ziarelli: Conceptualization, Data Curation, Formal Analysis, Investigation, Methodology, Software, Visualization, Writing – Original Draft Preparation;

Luca Dede’: Conceptualization, Writing – Review & Editing;

Nicola Parolini: Conceptualization, Data Curation, Validation, Writing – Original Draft Preparation;

Marco Verani: Conceptualization, Methodology, Validation, Writing – Original Draft Preparation;

Alfio Quarteroni: Conceptualization, Writing – Review & Editing.

Handling Editor: Dr He Daihai

Appendix A. Supplementary data

The following is the Supplementary data to this article.