Vaccination Control in a Stochastic SVIR Epidemic Model

Abstract

For a stochastic differential equation SVIR epidemic model with vaccination, we prove almost sure exponential stability of the disease-free equilibrium for ℛ ₀ < 1, where ℛ ₀ denotes the basic reproduction number of the underlying deterministic model. We study an optimal control problem for the stochastic model as well as for the underlying deterministic model. In order to solve the stochastic problem numerically, we use an approximation based on the solution of the deterministic model.

1. Introduction

The discovery of the first vaccine marked a major breakthrough in the battle against infectious diseases. The subsequent development of vaccines for various diseases has brought about remarkable results. Vaccination gained increasing popularity and success after it eradicated the smallpox outbreak of 1976 [1].

Vaccination is a commonly used method to control diseases such as measles, polio, and tuberculosis. Usually there are different schedules of dosage for different diseases and vaccines. For some diseases, doses should be taken by vaccinees several times and there must be some fixed time interval between two doses (see for instance Gabbuti et al. [2]). In a given population, the proportion of susceptibles who goes on to vaccination depends on different factors, one of which is the availability of the necessary resources.

There are numerous examples of vaccination models in the literature; see, for instance, the book of Brauer and Castillo-Chávez [3] or the journal papers [4–10]. With vaccination models one would be interested in the extent to which a vaccination program would reduce the basic reproduction number or how one can optimally roll out a vaccination program over time, in order to reach a certain target. In the latter case, optimal control theory is the obvious candidate to employ in the analysis [11–14]. It is usually the proportion of susceptibles who are admitted into vaccination, which is used as the control variable.

As a way of accommodating randomness in a compartmental epidemic model, several authors have proposed models with stochastic perturbation. This means that one modifies a system of ordinary differential equations (ODEs) by adding stochastic noise or stochastic perturbations, giving rise to a system of stochastic differential equations (SDEs) [7, 8, 15–18]. We shall refer to the original system of ODEs as the underlying deterministic model.

It is known (see the book of Mao [19], for instance) that the stability of a system can be improved by adding stochastic perturbations. SDE epidemic models have been studied by various authors until it was proved in research articles [9, 15, 20, 21] that stochastic perturbation improves stability of the disease-free equilibrium in the given models. SDE models with vaccination have been studied by Tornatore et al. [7, 8]. In particular in [8] it is shown that the model permits solutions that are almost surely positive, and a theorem on exponential stability in mean square stability is proved. The model of Tornatore et al. [8] will be further analyzed in this paper.

For SDE models in epidemiology, optimal control has not been studied (or at least not published) extensively. One of the reasons for this could very well be the difficulty with high dimensionality of the resulting partial differential equation (PDE) for the value function; see the paper of Sulem and Tapiero [22], for instance. A four-compartmental SIVR model such as in [7, 8] could easily lead to a PDE having the time variable together with three state variables. In this contribution we study the exponential stability of the stochastic model of Tornatore et al. [8] and control of vaccination both in the underlying deterministic model and in the stochastic model. In control problems, our goal is to characterize the vaccination rate (as control variable) which, on a finite time interval, minimizes the number of infected individuals balanced against the cost of vaccination. In the stochastic case we minimize an expected value. We use standard methods for the deterministic case and the Hamilton-Jacobi-Bellman equation in the stochastic case.

In Section 2 we study exponential stability of the disease-free equilibrium of the stochastic SIVR model that was introduced in [8]. We prove that almost sure exponential stability of the disease-free equilibrium prevails whenever ℛ ₀ < 1, ℛ ₀ being the basic reproduction number of the underlying deterministic model. The deterministic control problem is treated in Section 3, and we find a numerical solution for the optimal control problem by using the fourth-order Runge-Kutta method. The optimal control of the SDE model is presented in Section 4. We observe a similarity in the form of the control in the two cases, stochastic versus deterministic. On this basis, in Section 5 we propose that the numerical solution of the control problem of the underlying deterministic model can be used to compute an approximate solution to the stochastic control problem, assuming the perturbation parameter to be small. We present computational examples to illustrate our findings.

2. The Stochastic Model and Stability

In this section we introduce the SVIR model and we analyse the stability of the disease-free equilibrium. Firstly, we formulate the necessary assumptions for modeling with SDEs. Let us assume having a filtered complete probability space (Ω, ℱ, {ℱ _t}_t≥t0, ℙ) and let W(t) be a one-dimensional Wiener process defined on this probability space.

We consider a stochastic SVIR model similar to the stochastic SIVR model of [8]. As in [8], the population is subdivided into four compartments/classes. These classes consist of all the individuals who are susceptible to the disease (S), under vaccination (V), infected with the disease (I), and removed (R).

It is assumed that births and deaths occur at the same constant rate μ and that all newborns enter the susceptible class. New infections occur at a rate βSI, for a constant β which is called the contact rate. A fraction α(t) of the susceptible class is being vaccinated at time t. The vaccination may reduce but not completely eliminate susceptibility to the disease. Therefore the model includes a factor ρ in the contact rate of vaccinated members, such that 0 ≤ ρ ≤ 1. If ρ = 0 the vaccination is perfectly effective while ρ = 1 means the vaccination has no effect at all. Furthermore, immunity to the disease is assumed to be permanent so that a fraction γ of infectives goes into the removed class.

The total population is constant and the variables are normalized so that

$$\begin{array}{c}S\left(t\right)+V\left(t\right)+I\left(t\right)+R\left(t\right)=\mathrm{1}\forall t\ge \mathrm{0}.\end{array}$$ (1)

The stochastic SVIR model is given as follows:

$$\begin{array}{c}dS\left(t\right)\\ =\left(\mu -\mu S\left(t\right)-\beta I\left(t\right)S\left(t\right)-\alpha \left(t\right)S\left(t\right)\right)dt\\ -\sigma S\left(t\right)I\left(t\right)dW\left(t\right),\\ dV\left(t\right)\\ =\left(\alpha \left(t\right)S\left(t\right)-\rho \beta V\left(t\right)I\left(t\right)-\mu V\left(t\right)\right)dt\\ -\rho \sigma V\left(t\right)I\left(t\right)dW\left(t\right),\\ dI\left(t\right)\\ =\left(\beta S\left(t\right)I\left(t\right)+\rho \beta V\left(t\right)I\left(t\right)-\gamma I\left(t\right)-\mu I\left(t\right)\right)dt\\ +\sigma \left(S\left(t\right)+\rho V\left(t\right)\right)I\left(t\right)dW\left(t\right),\\ dR\left(t\right)=\left(\gamma I\left(t\right)-\mu R\left(t\right)\right)dt.\end{array}$$ (2)

Of course the parameters μ, γ, and β are positive constants. The parameter σ determines the intensity of the stochastic perturbation. If σ = 0, then the model reduces to the underlying deterministic model.

If α is constant then the model above is identical to that in [8], and the point

$$\begin{array}{c}{E}_{\mathrm{0}}=\left(S_{\mathrm{0}},I_{\mathrm{0}},V_{\mathrm{0}},R_{\mathrm{0}}\right)=\left(m,\mathrm{0,1}-m,\mathrm{0}\right)\\ \text{with}\hspace{0.17em}\hspace{0.17em}m=\frac{\mu }{\mu +\alpha }\end{array}$$ (3)

is the disease-free equilibrium point of the underlying deterministic model and is the only equilibrium point of the stochastic model.

For some n ∈ ℕ, some x ₀ ∈ ℝ ⁿ, and an n-dimensional Wiener process B(t), consider the general n-dimensional stochastic differential equation

$$\begin{array}{c}dx\left(t\right)=F\left(x\left(t\right),t\right)dt+G\left(x\left(t\right),t\right)dB\left(t\right),\\ x\left(\mathrm{0}\right)=x_{\mathrm{0}}.\end{array}$$ (4)

A solution to the above equation is denoted by x(t, x ₀). We assume that F(t, 0) = G(t, 0) = 0 for all t ≥ 0, so that the origin point is an equilibrium of (4).

Definition 1 . —

The equilibrium x = 0 of the system equation (4) is said to be almost surely exponentially stable if, for all x ₀ ∈ ℝ ⁿ,

$$\begin{array}{c}\underset{t\to \infty }{\lim }\sup \frac{\mathrm{1}}{t}\ln \left|x\left(t,x_{\mathrm{0}}\right)\right|<\mathrm{0}\text{a.s.}\end{array}$$ (5)

Let us denote by ℒ the differential operator associated with the function displayed in (4), defined for a function U(t, x) ∈ C ^1,2(ℝ × ℝ ⁿ) by

$$\begin{array}{c}\mathcal{L}U=\frac{\partial U}{\partial t}+F^{\text{trp}}\frac{\partial U}{\partial x}+\frac{\mathrm{1}}{\mathrm{2}}\text{Trc}\left[G^{\text{trp}}\frac{{\partial }^{\mathrm{2}}U}{\partial x^{\mathrm{2}}}G\right].\end{array}$$ (6)

Here Trc means trace and trp denotes the transpose of a matrix.

For the remainder of this section we shall study stability and for this purpose we regard α(t) to be a positive constant, α(t) = α for all t > 0. In [8] it was shown that the system of SDEs has unique solutions which are almost surely positive. Thus we shall restrict ourselves to sample paths w ∈ Ω for which the coordinates are positive for all t ≥ 0. The following observation is towards the proof of the stability theorem. Recall that we have introduced the number m = μ(μ + α)⁻¹ in expression (3). Let us define, for any constants a > 0 and c > 0, the stochastic processes:

$$\begin{array}{c}z\left(t\right)a{\left(S\left(t\right)-m\right)}^{\mathrm{2}}+I\left(t\right)+cR\left(t\right),\\ \\ u_{\mathrm{1}}\left(t\right)=-\frac{\mathrm{2}a\left(S\left(t\right)-m\right)\sigma S\left(t\right)I\left(t\right)}{z\left(t\right)},\\ u_{\mathrm{2}}\left(t\right)=\frac{\sigma \left(S\left(t\right)+\rho V\left(t\right)\right)I\left(t\right)}{z\left(t\right)}.\end{array}$$ (7)

Note that I(t)/z(t) ≤ 1 and S(t) + ρV(t) ≤ S(t) + V(t) ≤ 1. Now we calculate

$$\begin{array}{c}\underset{\mathrm{0}}{\overset{t}{{\displaystyle \int }}}{u}_{\mathrm{1}}^{\mathrm{2}}\left(s\right)ds\underset{\mathrm{0}}{\overset{t}{{\displaystyle \int }}}\mathrm{4}{a}^{\mathrm{2}}{\left[S\left(s\right)-m\right]}^{\mathrm{2}}{\sigma }^{\mathrm{2}}{S}^{\mathrm{2}}\left(s\right)ds\le \mathrm{4}{a}^{\mathrm{2}}{\sigma }^{\mathrm{2}}t,\\ \\ \underset{\mathrm{0}}{\overset{t}{{\displaystyle \int }}}{u}_{\mathrm{2}}^{\mathrm{2}}\left(s\right)ds\underset{\mathrm{0}}{\overset{t}{{\displaystyle \int }}}{\sigma }^{\mathrm{2}}{\left[S\left(s\right)+\rho V\left(s\right)\right]}^{\mathrm{2}}ds\le {\sigma }^{\mathrm{2}}t.\\ \end{array}$$ (8)

Therefore by the strong law of large numbers for martingales (see [19]) it follows that

$$\begin{array}{c}\underset{t\to \infty }{\lim }\sup \frac{\mathrm{1}}{t}\underset{\mathrm{0}}{\overset{t}{{\displaystyle \int }}}\left[u_{\mathrm{1}}\left(s\right)+u_{\mathrm{2}}\left(s\right)\right]dW\left(s\right)=\mathrm{0}.\end{array}$$ (9)

Proposition 2 . —

For constants a > 0 and c > 0, let z(t) = a(S(t) − m)² + I(t) + cR(t), and let U(t) = ln⁡z(t). Then z(t) almost surely (a.s.) converges exponentially to 0 if

$$\begin{array}{c}\underset{t\to \infty }{\lim }\sup \mathcal{L}U\left(t\right)<\mathrm{0}\left(a.s.\right).\end{array}$$ (10)

Proof —

Using the Itô formula and with u ₁(t) and u ₂(t) defined as above, we can express U(t) as

$$\begin{array}{c}U\left(t\right)=U\left(\mathrm{0}\right)+\underset{\mathrm{0}}{\overset{t}{{\displaystyle \int }}}\mathcal{L}U\left(s\right)ds\\ +\underset{\mathrm{0}}{\overset{t}{{\displaystyle \int }}}\left(u_{\mathrm{1}}\left(s\right)+u_{\mathrm{2}}\left(s\right)\right)dW\left(s\right).\end{array}$$ (11)

We have shown that

$$\begin{array}{c}\underset{t\to \infty }{\lim }\sup \frac{\mathrm{1}}{t}\underset{\mathrm{0}}{\overset{t}{{\displaystyle \int }}}\left(u_{\mathrm{1}}\left(s\right)+u_{\mathrm{2}}\left(s\right)\right)dW\left(s\right)=\mathrm{0}\left(\text{a.s.}\right),\end{array}$$ (12)

and hence the claim of the proposition follows readily.

The following observation from [9] is useful in exponential stability analysis.

Lemma 3 . —

For k ∈ ℕ, let X(t) = (X ₁(t), X ₂(t),…, X _k(t)) be a bounded ℝ ^k-valued function. Let (t _0,n) be any increasing unbounded sequence of positive real numbers. Then there is a family of sequences (t _l,n) such that, for each l ∈ {1,2,…, k}, (t _l,n) is a subsequence of (t _l−1,n) and the sequence X _l(t _l,n) converges to the largest limit point of the sequence X _l(t _l−1,n).

Now we present our stability theorem. Recall that ℛ ₀ is the basic reproduction number of the underlying deterministic model, and ℛ ₀ = β/(γ + μ). Also recall from expression (3) that m = μ(μ + α)⁻¹.

Theorem 4 . —

If ℛ ₀ < 1, then the disease-free equilibrium point (S ₀, I ₀, V ₀, R ₀) = (m, 0,1 − m, 0) is almost surely exponentially stable.

Proof —

The condition ℛ ₀ < 1 is equivalent to β − (γ + μ) < 0. Choose any number c > 0 such that β − (γ + μ) + γc < 0. Choose a number a > 0 such that

$$\begin{array}{c}\beta -\left(\gamma +\mu \right)+\gamma c+a\left(\mathrm{2}\beta +\mathrm{3}{\sigma }^{\mathrm{2}}\right)<\mathrm{0}.\end{array}$$ (13)

Now let z(t) = a(S(t) − m)² + I(t) + cR(t) and U(t) = ln⁡z(t). It suffices to prove that z(t) converges to 0 exponentially (a.s.). To this end, by Proposition 2 it suffices to prove that

$$\begin{array}{c}\underset{t\to \infty }{\lim }\sup \mathcal{L}U\left(t\right)<\mathrm{0}\left(\text{a.s.}\right).\end{array}$$ (14)

Now we calculate ℒU(t). The latter can be expressed in the form

$$\begin{array}{c}\mathcal{L}U\left(t\right)=A\left(t\right)+B_{\mathrm{1}}\left(t\right)+B_{\mathrm{2}}\left(t\right)+B_{\mathrm{3}}\left(t\right),\end{array}$$ (15)

with A(t), B ₁(t), B ₂(t), and B ₃(t) as follows:

$$\begin{array}{c}A\left(t\right)\\ =\frac{\mathrm{2}a\left[m-S\left(t\right)\right]}{z\left(t\right)}\left[\left(\mu +\alpha \right)\left[S\left(t\right)-m\right]+\beta I\left(t\right)S\left(t\right)\right]\\ +\frac{I\left(t\right)}{z\left(t\right)}\left[\beta S\left(t\right)+\rho \beta V\left(t\right)-\gamma -\mu \right]\\ +\frac{c}{z\left(t\right)}\left[\gamma I\left(t\right)-\mu R\left(t\right)\right],\\ B_{\mathrm{1}}\left(t\right)=\frac{\mathrm{2}az\left(t\right)-\mathrm{4}{a}^{\mathrm{2}}{\left(S\left(t\right)-m\right)}^{\mathrm{2}}}{\mathrm{2}{z}^{\mathrm{2}}\left(t\right)}{\left(\sigma S\left(t\right)I\left(t\right)\right)}^{\mathrm{2}},\\ B_{\mathrm{2}}\left(t\right)\\ =-\frac{\mathrm{2}a\left(S\left(t\right)-m\right)}{z^{\mathrm{2}}\left(t\right)}\sigma \left(S\left(t\right)+\rho V\left(t\right)\right)I\left(t\right)\sigma S\left(t\right)I\left(t\right),\\ B_{\mathrm{3}}\left(t\right)=-\frac{\mathrm{1}}{\mathrm{2}{z}^{\mathrm{2}}\left(t\right)}{\left[\sigma \left(S\left(t\right)+\rho V\left(t\right)\right)I\left(t\right)\right]}^{\mathrm{2}}.\end{array}$$ (16)

With reference to A(t) we note that since ρ ≤ 1, we have

$$\begin{array}{c}\beta S\left(t\right)+\rho \beta V\left(t\right)\beta \left[S\left(t\right)+\rho V\left(t\right)\right]\\ \le \beta \left[S\left(t\right)+V\left(t\right)\right]\le \beta ,\end{array}$$ (17)

and therefore we obtain the inequality:

$$\begin{array}{c}A\left(t\right)\\ \le \frac{\mathrm{2}a\left[m-S\left(t\right)\right]}{z\left(t\right)}\left[\left(\mu +\alpha \right)\left[S\left(t\right)-m\right]+\beta I\left(t\right)S\left(t\right)\right]\\ +\frac{I\left(t\right)}{z\left(t\right)}\left[\beta -\gamma -\mu \right]+\frac{c}{z\left(t\right)}\left[\gamma I\left(t\right)-\mu R\left(t\right)\right].\end{array}$$ (18)

In view of Lemma 3 we can define the following limits for a suitable increasing, unbounded sequence (t _n):

$$\begin{array}{c}q=\underset{n\to \infty }{\lim }\frac{{\left(S\left(t_n\right)-m\right)}^{\mathrm{2}}}{z\left(t_n\right)},\\ \\ i=\underset{n\to \infty }{\lim }\frac{I\left(t_n\right)}{z\left(t_n\right)},\\ \\ r=\underset{n\to \infty }{\lim }\frac{R\left(t_n\right)}{z\left(t_n\right)},\\ \\ s=\underset{n\to \infty }{\lim }S\left(t_n\right),\\ \\ v=\underset{n\to \infty }{\lim }V\left(t_n\right),\\ \end{array}$$ (19)

and with

$$\begin{array}{c}\underset{t\to \infty }{\mathrm{limsup}}\mathcal{L}U\left(t\right)=\underset{n\to \infty }{\lim }\mathcal{L}U\left(t_n\right).\end{array}$$ (20)

In particular then we have

$$\begin{array}{c}aq+i+cr=\mathrm{1},\mathrm{0}\le s\le \mathrm{1},\hspace{0.17em}\hspace{0.17em}\mathrm{0}\le v\le \mathrm{1}.\end{array}$$ (21)

Let

$$\begin{array}{c}\mathrm{\Lambda }=\underset{n\to \infty }{\lim }\mathcal{L}U\left(t_n\right).\end{array}$$ (22)

We find an upper bound for B ₁(t) + B ₂(t) + B ₃(t). Since S ² I ≤ 1 we have

$$\begin{array}{c}{B}_{\mathrm{1}}\left(t\right)\frac{\mathrm{1}}{\mathrm{2}}\frac{\mathrm{2}az\left(t\right)}{z^{\mathrm{2}}\left(t\right)}{\left(\sigma S\left(t\right)I\left(t\right)\right)}^{\mathrm{2}}\le a{\sigma }^{\mathrm{2}}\frac{I\left(t\right)}{z\left(t\right)},\\ \\ \left|B_{\mathrm{2}}\left(t\right)\right|\mathrm{2}a{\sigma }^{\mathrm{2}}{\left(\frac{I\left(t\right)}{z\left(t\right)}\right)}^{\mathrm{2}}\\ \end{array}$$ (23)

since |S(t) − m | < 1. Noting that B ₃(t) ≤ 0 and i < 1, we obtain

$$\begin{array}{c}\underset{n\to \infty }{\lim }{B}_{\mathrm{1}}\left(t_n\right)+B_{\mathrm{2}}\left(t_n\right)+B_{\mathrm{3}}\left(t_n\right)\le \mathrm{3}{\sigma }^{\mathrm{2}}ai.\end{array}$$ (24)

Therefore Λ satisfies the following inequality:

$$\begin{array}{c}\mathrm{\Lambda }\le -\mathrm{2}\left(\mu +\alpha \right)aq\\ +i\left[-\mathrm{2}a\beta s\left(s-m\right)+\beta s+\rho \beta v-\left(\gamma +\mu \right)\right]+c\gamma i-c\mu r+\mathrm{3}a{\sigma }^{\mathrm{2}}i.\end{array}$$ (25)

Now we note that |2aβs(s − m)| ≤ 2aβ, and so we can write

$$\begin{array}{c}\mathrm{\Lambda }\le -\mathrm{2}\left(\mu +\alpha \right)aq\\ +i\left[\beta -\left(\gamma +\mu \right)+\gamma c+\mathrm{2}a\beta +\mathrm{3}a{\sigma }^{\mathrm{2}}\right]-rc\mu .\end{array}$$ (26)

The coefficients of q, i, and r are negative (see (13)). Furthermore, q, i, and r cannot all be zero since

$$\begin{array}{c}aq+i+cr=\mathrm{1}.\end{array}$$ (27)

Therefore Λ < 0, and the proof is complete.

We run two sets of simulations of the deterministic and stochastic S and I-trajectories over T = 300 years. In the simulations, we use small perturbation parameters and small values of V. All parameter values in the computations except β and γ are the same in both scenarios and we also consider the same initial conditions. The common parameter values used in the computations are

•   μ = 0.016, ρ = 0.1, α = 0.3 and σ = 0.65

while the initial conditions are

•   S ₀ = 0.7, V ₀ = 0.05 and I ₀ = 0.15.

In the first run (see Figures 1 and 2) we use β = 0.55 and γ = 0.6, and we obtain ℛ ₀ = 0.893 < 1. In the second run (see Figures 3 and 4) we choose β = 0.2 and γ = 0.1, which yields ℛ ₀ = 1.724 > 1.

Figure 1.

A simulation of S for the deterministic and stochastic models with ℛ ₀ = 0.893.

Figure 2.

A simulation of I for the deterministic and stochastic models with ℛ ₀ = 0.893.

Figure 3.

A simulation of S for the deterministic and stochastic models with ℛ ₀ = 1.724.

Figure 4.

A simulation of I for the deterministic and stochastic models with ℛ ₀ = 1.724.

For ℛ ₀ < 1, Theorem 4 guarantees the disease-free equilibrium to be almost surely exponentially stable and indeed the I-curves in Figure 2 appear to converge to 0. For ℛ ₀ > 1 the theorem did not predict almost sure exponential stability and in fact the I-curves in Figure 4 (and many simulations with ℛ ₀ = 1.724 not shown) certainly do not show any clear intention of converging to 0.

3. The Deterministic Optimal Control Problem

We now formulate and solve the deterministic version of the control problem. Recall from Section 2 that in our SVIR model, α(t) represents the fraction of the susceptible class being vaccinated at time t. We wish to design an optimal vaccination schedule, α ^∗(t), which minimizes a combination of the number of infectives on the one hand and the overall cost of the vaccination on the other hand, over a certain time horizon [0, T]. The cost of the vaccination is assumed to be proportional to the square of α(t).

For the purposes of optimization we introduce the functions f ₁(t), f ₂(t), and f ₃(t) appearing in the SDE system (2) as follows:

$$\begin{array}{c}{f}_{\mathrm{1}}\left(t\right)\mu -\mu S\left(t\right)-\beta I\left(t\right)S\left(t\right)-\alpha \left(t\right)S\left(t\right),\\ \\ f_{\mathrm{2}}\left(t\right)=\alpha \left(t\right)S\left(t\right)-\rho \beta V\left(t\right)I\left(t\right)-\mu V\left(t\right),\\ f_{\mathrm{3}}\left(t\right)=\beta S\left(t\right)I\left(t\right)+\rho \beta V\left(t\right)I\left(t\right)-\gamma I\left(t\right)-\mu I\left(t\right).\end{array}$$ (28)

Now we can formulate the optimization problem.

Problem 5 . —

Minimize the objective function

$$\begin{array}{c}D\left(\alpha \left(·\right)\right)=\underset{\mathrm{0}}{\overset{T}{{\displaystyle \int }}}\left({\alpha }^{\mathrm{2}}\left(t\right)+cI\left(t\right)\right)dt\end{array}$$ (29)

subject to

•   $\dot{S}(t)=f_{\mathrm{1}}(t)$, $\dot{V}(t)=f_{\mathrm{2}}(t)$, $\dot{I}(t)=f_{\mathrm{3}}(t)$,
•   S(0) = S ₀ ≥ 0, V(0) = V ₀ ≥ 0, I(0) = I ₀ ≥ 0.

The control variable α(t) is assumed to be a measurable function of time and bounded: $\mathrm{0}\le \alpha (t)\le \overline{\alpha }\le \mathrm{1}$.

We solve Problem 5 using well-established control theory such as in the book [23] of Lenhart and Workman. We construct the Hamiltonian function, and to this end we introduce Lagrange multipliers λ ₁(t), λ ₂(t), and λ ₃(t), also referred to as the costate variables. In the theorem below, the control variable, the state variables, and the costate variables are functions of time, but this dependence is suppressed except where required explicitly. The Hamiltonian of Problem 5 has the form

$$\begin{array}{c}H\left(t,S,V,I,\alpha \left(t\right)\right)={\alpha }^{\mathrm{2}}+cI+{\lambda }_{\mathrm{1}}{f}_{\mathrm{1}}\left(t,S,V,I,\alpha \right)\\ +{\lambda }_{\mathrm{2}}{f}_{\mathrm{2}}\left(t,S,V,I,\alpha \right)+{\lambda }_{\mathrm{3}}{f}_{\mathrm{3}}\left(t,S,V,I,\alpha \right).\end{array}$$ (30)

Theorem 6 . —

An optimal solution for Problem 5 exists and satisfies the following system of differentiable equations:

$$\begin{array}{c}\dot{{\lambda }_{\mathrm{1}}}{\lambda }_{\mathrm{1}}\left(\mu +\beta I+\alpha \right)-{\lambda }_{\mathrm{2}}\alpha -{\lambda }_{\mathrm{3}}\beta I,\\ \\ \dot{{\lambda }_{\mathrm{2}}}={\lambda }_{\mathrm{2}}\left(\rho \beta I+\mu \right)-{\lambda }_{\mathrm{3}}\rho \beta I,\\ \dot{{\lambda }_{\mathrm{3}}}=-c+{\lambda }_{\mathrm{1}}\beta S+{\lambda }_{\mathrm{2}}\rho \beta V-{\lambda }_{\mathrm{3}}\left(\beta S+\rho \beta V-\gamma -\mu \right),\end{array}$$ (31)

with transversality conditions λ ₁(T) = 0, λ ₂(T) = 0, and λ ₃(T) = 0. Furthermore, the optimal vaccination rate is given by

$$\begin{array}{c}{\alpha }^{\ast }\left(t\right)=\min \left[\max \left(\mathrm{0},\frac{\mathrm{1}}{\mathrm{2}}S\left(t\right)\left[{\lambda }_{\mathrm{1}}\left(t\right)-{\lambda }_{\mathrm{2}}\left(t\right)\right]\right),\overline{\alpha }\right].\end{array}$$ (32)

Proof —

In particular the Hamiltonian is convex with respect to α(t) and the existence of a solution follows. We check the first-order conditions for this optimization problem. We calculate the partial derivatives of the Hamiltonian with respect to the different state variables to obtain the time derivatives $\dot{{\lambda }_i}(t)$ of the costate variables. The calculation of

•   $\dot{{\lambda }_{\mathrm{1}}}(t)=-\partial H/\partial S$, $\dot{{\lambda }_{\mathrm{2}}}(t)=-\partial H/\partial V$, and $\dot{{\lambda }_{\mathrm{3}}}(t)=-\partial H/\partial I$

leads to the equations asserted in the theorem. We now turn to the final part of the proof, which is about the form of the optimal control, α ^∗(t). Since the function α ^∗(t) must optimize the Hamiltonian, we calculate

$$\begin{array}{c}\frac{\partial H}{\partial \alpha }=\mathrm{2}\alpha -S\left({\lambda }_{\mathrm{1}}-{\lambda }_{\mathrm{2}}\right).\end{array}$$ (33)

Consider now a fixed value of t. If 2α − S(λ ₁ − λ ₂) is zero for some value α(t) in the interval $[\mathrm{0},\overline{\alpha }]$, then the given value of α(t) is optimal. If for every $\overline{\overline{\alpha }}\in [\mathrm{0},\overline{\alpha }]$ we have $\mathrm{2}\overline{\overline{\alpha }}-S({\lambda }_{\mathrm{1}}-{\lambda }_{\mathrm{2}})\ge \mathrm{0}$ (resp., $\mathrm{2}\overline{\overline{\alpha }}-S({\lambda }_{\mathrm{1}}-{\lambda }_{\mathrm{2}})\le \mathrm{0}$), then we must choose α(t) = 0 (resp., $\alpha (t)=\overline{\alpha }$). Thus we must have

$$\begin{array}{c}{\alpha }^{\ast }\left(t\right)=\min \left[\max \left(\mathrm{0},\frac{\mathrm{1}}{\mathrm{2}}S\left(t\right)\left[{\lambda }_{\mathrm{1}}\left(t\right)-{\lambda }_{\mathrm{2}}\left(t\right)\right]\right),\overline{\alpha }\right].\end{array}$$ (34)

In Figure 5 we plot for T = 3 years the S, V, and I-trajectories of the deterministic model subject to the optimal control α ^∗. We simulate α ^∗ and α ^∗/S ^∗ for the deterministic model in Figure 6. In the simulations, we consider the parameters c = 0.3, μ = 0.016, β = 0.55, ρ = 0.1, γ = 0.45, and $\overline{\alpha }=\mathrm{0.8}$ and initial conditions S ₀ = 0.7, V ₀ = 0.05, and I ₀ = 0.2.

Figure 5.

A simulation of S, V, and I for the deterministic model subject to the optimal control α ^∗.

Figure 6.

A simulation of α ^∗ and α ^∗/S ^∗ for the deterministic model.

In Figure 6 the dashed curve represents α ^∗(t), that is, the optimal vaccination rate. In this illustration, the optimal vaccination roll-out starts with initial vaccination of approximately 0.11 and decreases gradually over the next 3 years.

4. The Stochastic Optimal Control Problem

In this section we formulate the stochastic version of the optimization problem and describe its solution. For stochastic control theory we refer to the book [24] of Øksendal. Our objective is to find an optimal vaccination rate α ^∗(t) that minimizes the objective functional which for an initial state x ₀ is defined by

$$\begin{array}{c}{\mathbb{E}}_{\mathrm{0},x_{\mathrm{0}}}\left[\underset{\mathrm{0}}{\overset{T}{{\displaystyle \int }}}\left({\alpha }^{\mathrm{2}}\left(s\right)+cI\left(s\right)\right)ds\right].\end{array}$$ (35)

Here the expectation is obtained on the condition that the initial state (at time t = 0) of the system is x ₀. In step with the deterministic problem of earlier, we assume that there is a fixed constant $\overline{\alpha }\le \mathrm{1}$ with $\alpha (t)\le \overline{\alpha }$ (a.s.). The class of admissible control laws is

$$\begin{array}{c}\mathcal{A}=\left\{\alpha \left(·\right):\alpha \hspace{0.17em}\hspace{0.17em}\text{is adapted},\text{and}\hspace{0.17em}\hspace{0.17em}\mathrm{0}\le \alpha \le \overline{\alpha }\hspace{0.17em}\hspace{0.17em}\text{a.s.}\right\}.\end{array}$$ (36)

To solve this stochastic control problem, we define the performance criterion as follows:

$$\begin{array}{c}J\left(t,x;\alpha \right)={\mathbb{E}}_{t,x}\left[\underset{t}{\overset{T}{{\displaystyle \int }}}\left({\alpha }^{\mathrm{2}}\left(s\right)+cI\left(s\right)\right)ds\right],\end{array}$$ (37)

where the expectation is conditional on the state of the system being a fixed value x at time t. We define the value function as

$$\begin{array}{c}U\left(t,x\right)=\underset{\alpha \left(·\right)\in \mathcal{A}}{\inf }J\left(t,x;\alpha \right)=J\left(t,x;{\alpha }^{\ast }\right).\end{array}$$ (38)

We determine a control law that minimizes the expected value J : 𝒜 → ℝ ₊ given by (37). We now formulate the stochastic analogue of the optimal control problem, subsequent to which we present the solution formulae.

Problem 7 . —

Given the system (4) and given 𝒜 as in (36) with J as in (37), find the value function

$$\begin{array}{c}U\left(t,x\right)=\underset{\alpha \in \mathcal{A}}{\inf }J\left(t,x;\alpha \right),\end{array}$$ (39)

and an optimal control function

$$\begin{array}{c}{\alpha }^{\ast }\left(t\right)=\arg \underset{\alpha \in \mathcal{A}}{\inf }J\left(x;\alpha \left(t\right)\right)\in \mathcal{A}.\end{array}$$ (40)

We can find an expression for the optimal vaccination strategy α ^∗ through the following theorem.

Theorem 8 . —

A solution to the optimal vaccination problem stated in Problem (36) is of the form

$$\begin{array}{c}{\alpha }^{\ast }\left(t\right)=\min \left[\max \left(\mathrm{0},\frac{\mathrm{1}}{\mathrm{2}}S\left(t\right)\left[U_S\left(t\right)-U_V\left(t\right)\right]\right),\overline{\alpha }\right].\end{array}$$ (41)

Proof —

We determine (41) via the dynamic programming approach. First we calculate ℒU(t):

$$\begin{array}{c}\mathcal{L}U\left(t\right)=f_{\mathrm{1}}\left(t\right)U_S\left(t\right)+f_{\mathrm{2}}\left(t\right)U_V\left(t\right)+f_{\mathrm{3}}\left(t\right)U_I\left(t\right)\\ +\frac{\mathrm{1}}{\mathrm{2}}{\left(\sigma S\left(t\right)I\left(t\right)\right)}^{\mathrm{2}}{U}_{SS}\left(t\right)\\ +\frac{\mathrm{1}}{\mathrm{2}}{\left(\rho \sigma S\left(t\right)I\left(t\right)\right)}^{\mathrm{2}}{U}_{VV}\left(t\right)\\ +\frac{\mathrm{1}}{\mathrm{2}}{\left(\sigma \left(S\left(t\right)+\rho V\left(t\right)\right)I\left(t\right)\right)}^{\mathrm{2}}{U}_{II}\left(t\right)\\ +\rho {\left(\sigma I\left(t\right)\right)}^{\mathrm{2}}S\left(t\right)V\left(t\right)U_{SV}\left(t\right)\\ -{\left(\sigma I\left(t\right)\right)}^{\mathrm{2}}S\left(t\right)\left(S\left(t\right)+\rho V\left(t\right)\right)U_{SI}\left(t\right)-\rho {\left(\sigma I\left(t\right)\right)}^{\mathrm{2}}V\left(t\right)\left(S\left(t\right)+\rho V\left(t\right)\right)U_{VI}\left(t\right).\end{array}$$ (42)

Applying the Hamilton-Jacobi-Bellman theory (see, for instance, [24]) we must find the infimum:

$$\begin{array}{c}\underset{\alpha \in \mathcal{A}}{\inf }\left[{\alpha }^{\mathrm{2}}\left(t\right)+cI\left(t\right)+\mathcal{L}U\left(t\right)\right].\end{array}$$ (43)

For this purpose, we need to find the partial derivative of the expression

$$\begin{array}{c}{\alpha }^{\mathrm{2}}\left(t\right)+cI\left(t\right)+\mathcal{L}U\left(t\right)\end{array}$$ (44)

with respect to α, and this derivative should vanish. This leads to the equation:

$$\begin{array}{c}\mathrm{2}\alpha \left(t\right)-S\left(t\right)U_S\left(t\right)+S\left(t\right)U_V\left(t\right)=\mathrm{0}.\end{array}$$ (45)

We consider the bounds on α and by an argument similar to that in the proof of the deterministic case; the asserted expression for α ^∗(t) emerges.

5. Numerical Example

In the discussion that follows, the computations are done for T = 3 years and we consider three different values of the perturbation parameter, σ. In particular we consider

•  the σ-values: σ ∈ {0.15,0.30,0.45}

together with the following parameter values and initial conditions:

•   c = 0.3, μ = 0.016, β = 0.55, ρ = 0.1, γ = 0.45, and $\overline{\alpha }=\mathrm{0.8}$,
•   S ₀ = 0.7, V ₀ = 0.05, and I ₀ = 0.2.

For each value of σ above, we use the results of the deterministic control problem to find an approximate numerical solution for the stochastic control problem. In particular, we use λ ₁ − λ ₂ as a proxy for U _S − U _V in the calculation of α ^∗ in this case. We note that the presence of S(t) makes U into a stochastic variable even with the said proxy (in the stochastic case).

For each value of σ, we show in Table 1 the U(0)-values obtained as the average over 3000 runs made for different candidates for $\widehat{\alpha }$ as we take $\alpha =S\widehat{\alpha }$. In these runs we take λ ₁ and λ ₂ of the underlying deterministic model.

Table 1.

U(0) for different candidates of $\widehat{\alpha }$ and σ
${\widehat{\alpha }}_i$	σ = 0.15	σ = 0.3	σ = 0.45
${\widehat{\alpha }}_{-\mathrm{3}}$	29.84672	34.22981	36.21729
${\widehat{\alpha }}_{-\mathrm{2}}$	29.26219	34.09944	36.15703
${\widehat{\alpha }}_{-\mathrm{1}}$	28.96307	34.07355	36.14670
${\widehat{\alpha }}_{\mathrm{0}}$	28.91028	33.93111	35.95781
${\widehat{\alpha }}_{\mathrm{1}}$	29.52337	34.34593	36.26951
${\widehat{\alpha }}_{\mathrm{2}}$	30.15975	34.56046	36.36480
${\widehat{\alpha }}_{\mathrm{3}}$	31.01890	34.80816	36.47420
${\widehat{\alpha }}_{\mathrm{4}}$	29.03129	34.07210	36.05649
${\widehat{\alpha }}_{\mathrm{5}}$	30.71396	35.24172	36.82271

Using a contact rate β ₀ = β − (σ ²/2) instead of β, we choose the first candidate

$$\begin{array}{c}{\widehat{\alpha }}_{\mathrm{0}}\left(t\right)=\frac{\mathrm{1}}{\mathrm{2}}\left[{\lambda }_{\mathrm{1}}\left(t\right)-{\lambda }_{\mathrm{2}}\left(t\right)\right].\end{array}$$ (46)

The adjustment on the contact rate is motivated by the general stabilizing effect of this type of stochastic perturbation and the perturbation being associated with the parameter β.

We also take

$$\begin{array}{c}{\widehat{\alpha }}_i\left(t\right)={\widehat{\alpha }}_{\mathrm{0}}\left(t\right)+ϵi-\frac{ϵi}{\mathrm{3}}t,\text{for}\hspace{0.17em}\hspace{0.17em}i=-\mathrm{3},-\mathrm{2},-\mathrm{1,1},\mathrm{2,3}\end{array}$$ (47)

corresponding to the rate β and with ϵ = 0.001.

Furthermore, we consider

$$\begin{array}{c}{\widehat{\alpha }}_{\mathrm{4}}\left(t\right)=\frac{\mathrm{1}}{\mathrm{2}}\left[{\lambda }_{\mathrm{1}}\left(t\right)-{\lambda }_{\mathrm{2}}\left(t\right)\right],\end{array}$$ (48)

corresponding to the contact rate β.

Just for further comparison we choose a linear curve candidate ${\widehat{\alpha }}_{\mathrm{5}}(t)$, say,

$$\begin{array}{c}{\widehat{\alpha }}_{\mathrm{5}}\left(t\right)=\eta +\phi t\end{array}$$ (49)

with η = 0.0001 and φ = 0.00015, also corresponding to a contact rate of β.

Let us use the notation U _i(0) for the value of U(0) corresponding to the control $S{\widehat{\alpha }}_i$. We notice that U ₄(0) and U ₅(0) are bigger than U ₀(0). Even more interesting is the pattern U ₋₃(0) > U ₋₂(0) > U ₋₁(0) > U ₀(0) and U ₀(0) < U ₁(0) < U ₂(0) < U ₃(0). This creates the impression that U ₀(0) is close to being a minimum. At least, with all the difficulties of a more precise solution, the choice of ${\widehat{\alpha }}_{\mathrm{0}}$ seems like a viable option in a real application. The inequalities  U ₋₃(0) > U ₋₂(0) > U ₋₁(0) > U ₀(0) and U ₀(0) < U ₁(0) < U ₂(0) < U ₃(0) has been tested for other parameter choices too, with σ small. The simulations tested all revealed the same behaviour.

In Figures 7–9 we simulate the optimal solutions S ^∗, V ^∗, and I ^∗ of the deterministic and stochastic (σ = 0.3) models. We use the same values for the parameters c, μ, β, ρ, γ, σ, $\overline{\alpha }$, and initial conditions S ₀, V ₀, and I ₀ as in the discussion preceding these simulations.

Figure 7.

A simulation of S ^∗(t) for the deterministic and stochastic models.

Figure 9.

A simulation of I ^∗(t) for the deterministic and stochastic models.

An important point to note about our approximation is that it fully accommodates the stochasticity (embodied and concentrated in the factor S of the expression for α ^∗(t)). Therefore it gives a very good approximation, at least in the sense that in Table 1, the minimum value of U(0) consistently corresponds to ${\widehat{\alpha }}_{\mathrm{0}}$.

6. Conclusion

This paper presents some further insights into a stochastic vaccination model introduced in the paper of Tornatore et al. [8]. Our investigations covered two important aspects: exponential stability of the disease-free equilibrium and optimal control of vaccination.

Regarding stability, the main result of our paper has a particularly simple formulation. Essentially it says that we have almost sure exponential stability whenever the basic reproduction number of the underlying deterministic model is below unity. It will be good to know how an increase in vaccination rate would perhaps lead to better stability of the stochastic model. Nevertheless, the result as it stands is a good assurance.

As for the control problem, on the basis of a popular type of objective functional, we designed an efficient strategy for the roll-out of vaccination. It roughly amounts to minimizing the infections, balanced against the cost of vaccination. We exploit a similarity between the forms of the optimal controls for the stochastic model and the underlying deterministic model; then we use the relative simplicity of the latter to find approximate numerical solutions for the stochastic optimal control. Numerical simulation enables us to assess the feasibility of the option we followed, for a specific example. A more formal approach to the numerical solution of the optimal control problem is far more intricate and labour-intensive, and our method is a workable alternative. This could be the starting point for more sophisticated approximation methods.

Figure 8.

A simulation of V ^∗(t) for the deterministic and stochastic models.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.