Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2012 Apr 1.
Published in final edited form as: Math Biosci. 2011 Jan 28;230(2):67–78. doi: 10.1016/j.mbs.2011.01.003

A general approach for population games with application to vaccination

Timothy C Reluga 1,, Alison P Galvani 2
PMCID: PMC3063328  NIHMSID: NIHMS275546  PMID: 21277314

Abstract

Reconciling the interests of individuals with the interests of communities is a major challenges in designing and implementing health policies. In this paper, we present a technique based on a combination of mechanistic population-scale models from Markov decision process theory and game theory that facilitates the evaluation of game theoretic decisions at both individual and community scales. We demonstrate how nonlinear population models can be combined with game theory to inform the management of infectious diseases. To illustrate our technique, we provide solutions to several variants of the simple vaccination game including imperfect vaccine efficacy and differential waning of natural and vaccine immunity. In addition, we show how path-integral approaches can be applied to the study of models in which strategies are fixed waiting times rather than exponential random variables. These methods can be applied to a wide variety of decision problems with population-dynamic feedbacks.

Keywords: population games, epidemiology, vaccination, infectious diseases, Markov decision processes

1 Introduction

There has been a long-standing dichotomy in approaches regarding disease intervention. Public health interventions to control disease have focused on the best interests of communities and populations, implicitly assuming that the interests of individuals parallel those of their communities. By contrast, medical interventions to treat disease has focused on the well-being of the individual with limited attention to the relationships between individual and community health. Often health impacts upon the two spheres are complementary. For example, improvements in individual sanitation are good for a community because they reduce the risks of disease for everyone in the community. Similarly, an inexpensive treatment that cures a patient of an infectious disease is good for both the patient and the community because a cured patient no longer poses a transmission risk to others.

One consequence of the dichotomy of perspectives between public health and medicine is that mathematical and quantitative methods in epidemiology and medicine have developed in independent directions. Epidemiology models focus on aggregate behaviors to help us understand the transmission and control of epidemic and endemic infectious disease [Ross and Hudson, 1917, Kermack and McKendrick, 1927]. In epidemiology models, individuals are typically treated as entities that obey prescribed laws rather than as humans with free-will. In contrast, medical models favor decision theory approaches that focus on determining the best medical treatments from the persepctive of the individual. Such methodologies assist the patient and the doctor in weighing the risks and benefits for various treatment modalities. For instance, the Markov decision process theory (MDPT) uses a stochastic process to describe the possible events in an individual’s future [Sonnenberg and Beck, 1993]. MDPT associates values with events, and calculates the expected value conditional on each choice. The expected value can then be optimized over the different choices available to the individual. Although the risks an individual faces may depend on the overall state of the community, this form of analysis does not account for any externalities posed to the community by the individual’s choices.

Modern public health and medical practices in wealthy nations have pushed back the tide of many historically devastating diseases, for example through the development of vaccines as well as prophylatic and therapeutic drugs. However, as we have solved old problems, we have unveiled new problems. For some of these new problems, the interests of the individual now diverge from the interests of the community; medical planning and epidemiological planning can act in opposition. Such conflicts are known as “market failures” in economics, and “policy resistance” in public health [Sterman, 2006].

The maintenance of herd immunity to infectious diseases through mass vaccination is an example of this discrepancy between interests. In the past, the high prevalence of many vaccine-preventable infectious diseases made the choice of whether or not to vaccinate a simple one. When the risks from infection were significant, vaccination was a worth while measure for the community and for an individual. The dramatic success of vaccines has greatly reduced the risk of infection for many diseases, and individuals are having more difficulty deciding whether or not to be vaccinating against some diseases despite potential benefit to their communities.

One approach to the study of individual interests in communities is rational expectation population game theory. If we understand the epidemiological processes, and assume that individuals have broad and reliable information about the state of the world and their own health risks, we can derive functional relationships to resolve the balance of different competing interests. This approach does not capture the full complexity of the decision process, but does give a useful reference point. An early attempt to use game theory for studying vaccination was motivated by concerns over the safety of pertussis vaccines [Fine and Clarkson, 1986]. This was followed by studies of the economics of compulsory vaccination programs [Brito et al., 1991] and disease eradication efforts [Geoffard and Philipson, 1997, Philipson, 2000, Barrett, 2003]. In 2003, Bauch et al. [2003] published a model in which individual risk and population-scale dynamics were explicitly modeled [Bauch et al., 2003]. Generalization of that work has lead to the class of models studied here. A growing number of papers expand on these ideas, and investigate how various factors can favorably or unfavorably influence vaccination practices and health outcomes [Francis, 2004, Chen, 2004, Bauch and Earn, 2004, Bauch, 2005, Chen, 2006a,b, Kremer et al., 2006, Chen, 2009].

Here, we describe in detail the mathematics of a population game approach to address problems in health-care. Our approach has been used in our previous research [Galvani et al., 2007, Reluga et al., 2007], but without the detailed exposition or generality appropriate to facilitate broader usage. We demonstrate how population-scale dynamic models can be coupled to individual-scale decision models. This approach allows us to formulate population games that describe the utility of decisions to the individual and to the population using closed-form algebraic results. We also review standard conditions for equilibria in population games. We use simple vaccination problems illustrate the application of these methods. Where other methods often require approximations or numerical solution, our method frequently yields exact analytic solutions that can be easily interpreted in the context of biological and economic parameters. As proof-of-principle, we derive analytic solutions to two versions of the vaccination game. The first version is a compartmental model in which individuals’ strategies are randomized daily. This model is closely related to previously studied models [Bauch and Earn, 2004, Chen, 2006a]. The second version uses McKendrick-von Foerster-like partial differential equations to describe a vaccination game where strategies are based on fixed waiting times. Despite differences in the mathematics, both versions lead to similar results. In basic scenarios, there is a unique Nash equilibrium for behavior with global invasion potential that converges to the community’s preferred policy for vaccines that are either very inexpensive or very expensive. When generalizations of the compartmental model are considered, biological complications, such as limited vaccine efficacy and differences in immunity waning, can lead to more complex equilibrium structures.

2 Methods

Our approach has three parts: a model of the value of choices that is analyzed using decision theory and game theory, a model of changes in an individual state that is analyzed using stochastic process theory, and a model of the population’s state that is analyzed using dynamic systems theory.

2.1 General formulation of population games

The theory we present incorporates expected utility theory with additive payoffs and hyperbolic discounting. There is no universal answer as to how a rational individual should interpret the distribution of their potential payoffs when choosing their best strategies [Rockafellar, 2007]. In some situations, a stochastic ordering of outcomes suffices to identify the best strategies uniquely, but in many situations, stronger assumptions are needed. Individuals might maximize their worst case payoff, minimize their risk given some constraints on acceptable payoffs, or be guided by a variety of other optimization goals. Our choice of expected utility theory is a common [Arrow and Kurz, 1970] and mathematically convenient assumption.

Under expected utility theory, individuals have a set of strategic actions (π) and aim to maximize the utility of their actions, where the utility is defined as the sum of all future payoffs. Suppose we know an individual’s entire future, in the sense that there is a function x(t, π) that, given a individual’s strategy π, returns an individual’s state at time t. If time is measured continuously, the utility of the strategy π over the course of a lifetime is

t0tfeh(tt0)Φ(x(t),x(t),t,π)dt+φ(x(tf)), (2.1)

where t0 is the start time, tf is the terminal time, φ(x(tf)) is the value of the final state, and Φ(x, x′, t, π) is the rate of utility gain per unit time as a function of the state x, the change in state x′ at time t, time, and the strategy. The parameter h represents the rate at which future gains are discounted relative to current gains1.

However, the future of each individual is uncertain. We can use observational data to infer the likelihoods of certain events occurring at specific points in an individual’s life, but no particular individual’s future can be predicted with certainty. The path x(t) of a person’s life can only be known in a probabilistic sense, so the utility of a given strategy π is a random variable that depends on the likelihood of different paths. According to the postulates of expected utility theory, an individual accounts for this uncertainty by maximizing their average or “expected” utility. We can calculate an individual’s expected utility by determining the probability distribution of all possible paths x(t) and integrating over this distribution. The expected utility

U=(x0,t0)t0tfehtΦ(x(t),x(t),t,π)dt+φ(x(tf))Dxπdx0, (2.2)

where Inline graphicxπ denotes the infinitesimal probability, depending on the strategy π, of each possible state path starting at x0 at time t = t0, and dx0 is the probability of each initial state x0 at time t0.

To make this approach practical, we must provide a specific description of the state paths an individual can experience over their life, including the dependency of these paths on external factors. In the models we present, the probabilities of paths depend on the overall states of the population, the environment, and the decisions an individual makes. The state of an individual at time t is represented by a probability distribution p(t) over the individual’s possible states. The state distribution p changes according to a continuous time Markov process. A Markov process uses a system of linear differential equations to describe changes in the probability that an individual will be found in any given state. The transition rates in this Markov process may depend on many factors including time and the individual’s strategy, the population’s state (n) and the environment’s state (e). Thus,

dpdt=Q(n,e,t,π)p, (2.3)

where Q is a matrix of transition rates. To take into account the finiteness of an individual’s life, we allow the possibility of a “dishonest” Markov process [Kingman, 1967]. Usually, Markov processes are defined such that eQ is a stochastic operator with columns that sum to 1 to preserve probability mass. Dishonest Markov processes are defined in the same way as Markov processes, except that eQ is a substochastic operator with at least one column sum less than 1. For a strictly dishonest Markov process, all the column sums of eQ are strictly less than 1. Every dishonest Markov process can be extended to a regular Markov process by adding an absorbing state for death, but it will be useful to avoid this complication. For an honest Markov process, the dominate eigenvalue is λ0(Q) = 0, whereas for a dishonest Markov process, the dominate eigenvalue is λ0(Q) ≤ 0, and for a strictly dishonest process, the dominate eigenvalue is λ0(Q) < 0.

In order to use Eq. (2.3) in evaluating Eq. (2.2), we must also specify a description of the dynamics of the population and environment (n and e, respectively). The population-scale and individual-scale dynamics are driven by separate but related processes. A convenient approach is to postulate that the population-scale dynamics satisfy a system of differential equations where the rates of change in the states of the population depend on the average behavior of individuals in the population π̄, such that

dndt=Gn(n,e,t,π¯), (2.4a)
dedt=Ge(n,e,t,π¯). (2.4b)

The vector π̄ representing the effective average behavior of individuals is often referred to as the resident strategy. We assume almost all individuals in a population use strategy π̄, and that the populations are so large that the population dynamics are not significantly affected by the few individuals with invading strategies that differ from the resident strategy.

Using Eq. (2.4) in combination with appropriate initial conditions, we can determine n(t, π̄) and e(t, π̄), and thus the transition rates of Eq. (2.3) can be interpreted as functions of the resident strategy π̄ and an individual’s strategy π. According to Markov decision process theory [Howard, 1960], the instantaneous utility gains for a finite state Markov process can be represented in terms of a vector (f) of gains per unit time for residence of each state and a matrix (F) of instantaneous utility gains associated with each transition, with Fij representing the instantaneous gain for a transition from state j to state i. Let the vector v(t) represent the expected present value of the utility of each state at time t > t0. Generalizing the Bellman equation for a continuous time Markov process to the case of non-autonomous processes,

dvTdt=vT(hIQ)[fT+1T(FQ)], (2.5)

where ◦ represents the Hadamard product of two matrices2. Eq. (2.5) is a linear inhomogeneous system. It is interpreted backwards from terminal time tf to initial time t0 rather than forwards from t0 to tf, as the expected utility today is calculated based on the expected utility tomorrow, not the expected utility yesterday. For a contemporary reference on the theory of Markov decision processes, see Puterman [2005].

Taking the solution of Eq. (2.3) as a Greens function, Eq. (2.5) can be integrated and yields the expected utility

U(π,π¯)=vT(t0)p(t0)=t0tf[fT+1T(FQ)]eh(tt0)p(t)dt+vT(tf)p(tf)eh(tft0). (2.6)

This can be shown to be equivalent to Eq. (2.2) when the state space is finite by expanding Eq. (2.2) with respect to small perturbations of t0 and using the history-independence property of Markov processes to simplify the resulting integration. We write the expected utility as a function of the resident strategy π̄ and an invading strategy π to emphasize that it is specifying an aggregate game between competing strategies. Readers familiar with optimal control theory may alternatively recognize v as the vector of present value shadow prices from optimization of Eq (2.6) subject to constraints given by Eq. (2.3) (see Appendix A).

Although population games can be formulated and studied in the form we have so far presented (for example, [Reluga, 2010]), it is often useful to focus on cases where the population-scale dynamics have a simple attractor. If all dynamics are autonomous and (n*, e*) is a stationary solution of the population-scale dynamics and strategies are independent of time, then the transition rate matrix

Q:=Q(n,e,π) (2.7)

is also stationary. As the time horizon of the utility calculation becomes infinitely long (tf → ∞), there is a positive discount rate h > 0 or the Markov process is strictly dishonest, then the expected utility has closed form

U=[fT+1T(FQ)](hIQ)1p(t0). (2.8)

In special cases where the Markov process is honest and there is no discounting (h = 0), Eq. (2.5) will never converge to a steady state and Eq. (2.8) cannot be calculated, because the required matrix inversion is singular. In these cases, it is more appropriate to study the asymptotic growth rate of the expected utility. We can show that if is the unique limiting stationary distribution of the Markov process (Q* = 0, Σjj = 1, p(t) → ), then

U:=limtfUtf=[fT+1T(FQ)]p^. (2.9)

In practice, the observation that

U=limh0hU(π,π¯), (2.10)

is particularly useful for simplifying theoretical work, as illustrated in the following examples.

3 Vaccination Games

We now apply these mathematical methods on the analysis of vaccination behavior in some simple epidemic models. We focus on mathematical solutionsto illustrate the population-game framework in action as applied to a simple problem of practical importance.

One of the biggest questions in the formulation of population games is how to model the strategy space of individual decisions. In general, individuals can choose mixed strategies with arbitrary time dependence that is conditional on any subset of the system state information. The complexity of such strategy space models is usually unsolvable and may mask general properties of the games in questions. We provide examples that allow for strategies that can be represented in terms of a single parameter. In the first model, we assume that individuals chose the time they wait before vaccination from an exponential distribution. We refer to this as a “hazard-strategy” because it is equivalent to individuals having a constant hazard per unit time of vaccination. In the second model, individuals choose a fixed time to wait between the loss of immunity and their next vaccination. We refer to this as a “delay-strategy” model because the resulting mathematics is closely related to that of delayed differential equations. The population-game framework described above can be employed for both of these models. Although the solution techniques have technical differences, the final results are similar.

3.1 Hazard-strategies in a vaccination game

We consider a homogeneous population of size N afflicted by a influenza-like illness for which there is a costly and risky vaccine available. We describe this system with a compartment model where individuals in the population can occupy susceptible (S), infected (I), resistant (R), and vaccinated (V) states. Individuals die and are replaced at rate μ, recover from infection at rate γ, and lose immunity at rate a. The population-scale dynamics are described by

dSdt=μN+a(R+V)λ(I)S(π¯+μ)S, (3.1a)
dIdt=λ(I)S(γ+μ)I, (3.1b)
dRdt=γI(a+μ)R, (3.1c)
dVdt=π¯S(a+μ)V, (3.1d)

where λ(I) is the force of infection, π̄ is the population’s resident vaccination rate, and the population size

N=S+I+R+V (3.1e)

is constant. The force of infection λ(I) is assumed to be an increasing concave function of the number of infections I. Initially, we assume that infection and vaccination immunity are complete and wane at equal rates. This model includes special cases of life-long immunity with demographic turnover (a = 0, μ > 0) and temporary immunity without demographic turnover (a > 0, μ = 0).

Although the population’s state may evolve deterministically, each individual’s state distribution evolves according to the Markov process described by Eq. (2.3). At steady state, the stationary forward transition rate matrix is

Q=[λ(I)πμ0aaλ(I)γμ000γaμ0π00aμ], (3.2)

where π is the individual’s chosen strategy of a daily probability whether or not to get vaccinated. Q* represents an honest Markov process if μ = 0, and a strictly dishonest process if μ > 0.

The source term μN in Eq. (3.1a) represents birth and immigration of new individuals into the population at a constant rate, and corresponds to the initial condition

p(0)=[1000]T. (3.3)

When the states of all individuals are combined into one system, the mean-field equations described by System (3.1) are confirmed.

We are interested in the typical vaccination rate of the population π̄, and its relation to the individual’s risk from vaccination per unit time, π. Individuals incur costs from both infection and vaccination. The vaccination cost to the individual occurs in the transitions between vaccination and infection states. The infection cost accumulates as individuals reside in the infected state. As these are the only two costs, and there are no explicit gains,

f=[0cI00],F=[000000000000cV000]. (3.4)

The expected change in utility per unit time for residence in each vaccination and infection state is given by the vector

fT+1T(FQ)=[πcVcI00.] (3.5)

It follows from Eq. (2.8) that the utility of strategy π to an individual in a population at equilibrium with strategy π̄ is

U(π,π¯)=[fT+1T(FQ)](hIQ)1p(0) (3.6a)
=(a+μ+h)[cVπ(γ+μ+h)+cIλ(I)](μ+h)[λ(I)(γ+a+μ+h)+(γ+μ+h)(π+a+μ+h)], (3.6b)

where I* is determined by the steady-state condition

λ(I)=I/N(γ+μ)(π¯+a+μ)(a+μ)I/N(γ+a+μ). (3.7)

Since λ(I) is increasing and concave, I*(π̄) must be a decreasing function of the resident strategy π̄.

We can now study Eq. (3.6) as a game, looking for equilibrium strategies that are Nash equilibria and have invasion potential. For a review of the relevant material, see Appendix B. If an individual is never vaccinated,

U(0,π¯)=(a+μ+h)cIλ(I)(μ+h)[λ(I)(γ+a+μ+h)+(γ+μ+h)(a+μ+h)]. (3.8)

In the limit of infinitely fast vaccination,

U(,π¯)=(a+μ+h)cV(μ+h). (3.9)

Inspection of Eq. (3.6) shows that the utility is a Cartesian hyperbola (linear fractional transform, more generally) as a function of the individual behavior π with a vertical asymptote for negative argument. In this case, a positive finite value of π can maximize the utility only when the hyperbola is actually a line with zero slope. Critical strategies can be determined by setting utilities at the extremes and all intermediate individual strategies equal to each other.

U(0,π)=U(π,π)=U(,π), (3.10)

so at the critical strategy,

cV=cIa+γ+μ+h+(γ+μ+h)(a+μ+h)λ(I). (3.11)

In some simple models, in which individuals have perfect knowledge about their state, changes in transition rates are proportional to changes in behavior, and all behavior costs are indirect, boundary conditions such as U(0, π*) = U(∞, π*) can be used to identify equilibria [Reluga, 2009, Appendix B]. However, this condition does not generally suffice for the identification of equilibria [Reluga et al., 2007].

Solving Eq. (3.11) for the force of infection,

λ(I(π¯=π))=(γ+μ+h)(a+μ+h)cIcV(γ+a+μ+h). (3.12)

This condition may not have a solution, depending on the costs of infection and vaccination. For the critical strategy to be a positive vaccination rate, it is required that cI/cV − (γ + a + μ + h) > 0. If the cost of vaccination is so high that even when there is no vaccination (π̄ = 0)

cV>cIa+γ+μ+h+(γ+μ+h)(a+μ+h)λ(I(π¯=0)), (3.13)

then the critical strategy is for no one to be vaccinated, π* = 0. If the cost of vaccination cV is very small relative to the minimal force of infection (π̄ = ∞),

cV<cIa+γ+μ+h+(γ+μ+h)(a+μ+h)λ(I(π¯=0)), (3.14)

then the critical strategy is for no one to be vaccinated as quickly as possible, π* = ∞. In each scenario, the respective extreme point is a Nash equilibrium because no alternative strategy generates greater utility for the individual (see Appendix B).

In the intermediate state between every individual being vaccinated and no one being vaccinated, I*(π̄) is uniquely determined by Eq. (3.7) provided the force of infection is a continuously increasing function of the incidence. This implies that there is a unique critical strategy. The critical strategy satisfies the condition for a Nash equilibrium, because U(π, π*) is a flat line. The condition for a strict Nash equilibrium is not satisfied, but we can now show invasion potential by proving that

d2U(π,π¯)dπdπ¯=(ddπUλ)(λIdIdπ¯)+Uλ×ddπ(λIdIdπ¯)<0. (3.15)

Both λ and I* are independent of the individual strategy π, so their derivative with respect to the individual strategy is 0. From our assumptions about the force of infection, we know λI>0 and dIdπ¯<0. Finally,

ddπUλ=cV(h+μ+a)[cIcV(γ+a+μ+h)]3(h+μ+γ)(h+μ)[π(μ+a+h+γ)cIcV(a+h+π+μ)]2>0 (3.16)

provided that

cIγ+a+μ+h>cV. (3.17)

Substituting, we find Eq. (B.7) holds. The critical point is a Nash equilibrium with invasion potential.

Once the equilibria have been identified, we can evaluate their sensitivity to the parameters. The larger the cost of infection per unit time cI, the lower the force of infection allowed at the Nash equilibrium. The larger the cost of vaccination, the higher the force of infection allowed at the Nash equilibrium. By inspection of Eq. (3.6), we see that as the discount rate h increases, the effective cost of infection decreases as vaccination decreases. A higher force of infection is allowed at equilibrium. The effects of the other parameters are less obvious because the endemic disease prevalence I* is not only a function of the average vaccination rate π̄, but also an implicit function of the recovery, waning, and mortality parameters γ, a, and μ.

3.1.1 Exact solutions in the case of standard incidence

To examine the relationship between parameter values and the Nash equilibrium vaccination rate, we consider a specific model. We assume that the force of infection is modeled using standard incidence

λ(I)=βI/N, (3.18)

and that demographic turnover and discounting are sufficiently slow to be neglected (μ = 0 and h = 0).

Under those conditions, the reproductive number at the disease-free equilibrium is

R=(βγ)(aa+π¯). (3.19)

If Inline graphic < 1, the disease-free equilibrium is stable. An average vaccination rate of

π¯π^=a(βγ1) (3.20)

is sufficient to achieve herd immunity and thus to ensure eradication.

The endemic-disease equilibrium solution when Inline graphic > 1 is

S=Nγβ,I=Nβaγaπ¯γβ(a+γ),R=Nγaβaγaπ¯γβ(a+γ),V=Nπ¯γβa. (3.21)

In the absence of discounting or mortality, we apply Eq. (2.10), and find the asymptotic expected utility gain per unit time

U=a(cIβI/N+cVπγ)(a+γ)βI/N+γ(π+a) (3.22)

Using equilibrium infection prevalence I*,

U(π,π¯)=a[(βaγaπ¯γ)cI+γπ(a+γ)cV](a+γ)(π¯γπγβa) (3.23)

The shape of U′ is illustrated in Figures 1 and 2.

Figure 1.

Figure 1

Asymptotic expected utility gain per unit time U′ (π̄, π̄) for an individual using the population’s average strategy π̄. There is a corner in the utility gain at vaccination rates of about π̄ = .25 that is barely sufficient to ensure eradications. This is the social optimum, as faster or slower rates yield slower gains to the average individual. Parameter values: γ = 1, β = 6, a = .05, cI = 4, cV = 2.

Figure 2.

Figure 2

An individual’s asymptotic expected utility loss U′ (π, π̄) as a function of the individual’s strategy and the population’s average vaccination rate for the same parameters as presented in Figure 1. The absolute utility loss (left) is minimized for different individual choices, depending on the population’s average behavior. The relative utility (right), calculated as the ratio of the absolute utility loss divided by the average utility loss, shows that vaccination rates around .195 are self-consistent in the sense that no one can unilaterally improve on their situation by deviating from the average behavior. All contour values are negative; smaller values represent higher utilities.

For a given population behavior π̄, individuals wish to choose their own vaccination rate π to maximize Eq. (3.23). The best response correspondence3 πbest(π̄) is a set-valued mapping that returns the set of optimal replies to a population strategy π̄. Thus, the best response correspondence is

πBest(π¯)={0ifπ¯>aβγ+acγ+ca1[0,)ifπ¯=aβγ+acγ+ca1ifπ¯<aβγ+acγ+ca1 (3.24)

where c = cV/cI expresses the relative cost of vaccination compared to infection. We find that there is a unique Nash equilibrium

π={0ifcβγβ(a+γ),aβγa1c(γ+a)if0c<βγβ(a+γ), (3.25)

that is an improvement on every alternative strategy the population may adopt, but that cannot be improved upon when it is adopted by the population. Only when c = 0 does the equilibrium strategy eradicate disease (π* = π̂). For all positive costs c, the equilibrium vaccination hazard is less than that needed for eradication (π* < π̂). When the population adopts π* and β > γ, the disease incidence

I=Naβmin{cγ1c(a+γ),βγa+γ} (3.26)

and the individual’s expected asymptotic utility gain is

U(π,π)=max{acV,cIa(βγ)β(a+γ).} (3.27)

For comparison, at the herd-immunity threshold,

U(π^,π^)=(1γβ)cVa, (3.28)

which is strictly greater than the expected asymptotic utility gain at the Nash equilibrium.

Overall our analyses indicate that the recovery rate and the waning rate affect the vaccination rate at the Nash equilibrium. Faster waning decreases the vaccination rate at the Nash equilibrium. In general, the longer the duration of natural resistance, the lower the vaccination rate is at Nash equilibrium. Higher vaccination costs decrease vaccination rate. More rapid recovery also decreases vaccination rate. An infinitely fast vaccination rate is never an equilibrium in this special case [Bauch et al., 2003], because all risks to the individual vanish under the standard incidence assumption (Eq. (3.18)) as the vaccination rate approaches the eradiction threshold given in Eq. (3.20). Infinitely fast vaccination rates may be equilibria if there is an external source of risk [Reluga et al., 2006]. The inclusion of discounting h should slow vaccination, but the inclusion of demographic turnover (μ > 0) may increase or decrease the vaccination rate, depending on the other parameter values.

3.1.2 Differential waning of immunity

These results can easily be extended to a case in which vaccine based resistance wanes at rate aV, whereas naturally acquired resistance wanes at rate aR. The population-scale dynamics satisfy

dSdt=aRR+aVVλ(I)Sπ¯S, (3.29a)
dIdt=λ(I)SγI, (3.29b)
dRdt=γIaRR, (3.29c)
dVdt=π¯SaVV,. (3.29d)

Using the corresponding individual-scale model and the initial condition and costs previously described, Nash equilibrium behavior is given by

π={0ifc(βγ)aRβaV(aR+γ),βaVγaVaRaRcaV(aR+γ)otherwise. (3.30)

When aV = aR = a, we recover Eq. (3.25). The algebraic calculations needed to reach this result are analogous to those of equal waning rates.

The response to changes in the waning rate of vaccine-induced immunity is not monotone. When vaccine waning is very slow, small increases in the waning rate also increase the critical vaccination rate. However, near the critical cost threshold, faster waning decreases the vaccination rate (Figure 3). Vaccination is only useful if the cost per time of vaccination cV aV is less than that of natural immunity cI(1γβ)aRaR+γ.

Figure 3.

Figure 3

The equilibrium vaccination rate π* as a function of the waning rates aR and aV. The shaded region indicates parameter values where π* = 0. Increases in the waning rate of natural immunity aR increase Nash equilibrium vaccination. However, the response to changes in the waning rate of vaccine immunity aV may either increase or decrease equilibrium vaccination. If natural immunity lasts longer than vaccine immunity, the equilibrium is to refuse vaccination. Parameter values β = 4, γ = 1, c = .1.

3.1.3 Imperfect immunity

We can extend this analysis to the more interesting case of imperfect immunity. Imperfect immunity has been studied previously by Kremer, Snyder, and Williams (Snyder, personal communication) and by Chen [2006a]. The following analysis provides an alternative mathematical approach to some of their results. Our analysis indicates that the parameter region in which multiple Nash equilibria exist is narrow, and perhaps rarely encountered in practice.

Assuming that vaccination confers partial immunity, that infection confers complete immunity, and that both vaccination mediated and infection mediated immunity wane at the same rate, then System (3.1) becomes

dSdt=aRR+aVVλSπ¯S, (3.31a)
dIdt=λ(S+σV)γI, (3.31b)
dRdt=γIaRR, (3.31c)
dVdt=π¯SσλVaVV. (3.31d)

When we construct the corresponding individual-scale model and solve for the utility, we find that the utility of an individual strategy is

U(π,π¯)=aRλ(λσ+aV+πσ)cIaRπγ(aV+λσ)cVσ(γ+aR)λ2+λ[(γ+aR)(πσ+aV)+aRσγ]=aRγ(aV+π). (3.32)

If λ = βI/N, then the steady-state force of infection λ* is either 0 or solves the quadratic polynomial

o=σ(aR+γ)λ2+(γaRσ+γπ¯σ+aRaV+aRπ¯σ+aVγβaRσ)λ+aVγaRβaRaV+π¯aRγβaRπ¯σ. (3.33)

If σβ/γ > 1, even the immediate vaccination of every individual will not eradicate the disease.

The utility is still a hyperbola in terms of the individual strategy π, so we expect all strategies to yield the same utilities at Nash equilibria. Local Nash equilibria can be identified in the standard manner (see Figure 4). No vaccination (π* = 0) is a Nash equilibrium if

Figure 4.

Figure 4

Dependence of the Nash equilibria vaccination rates π* upon the relative cost of vaccination (c = cV/cI). The Nash equilibrium is unique when c < .393 or c > .426. If .393 < c < .426, there are three Nash equilibria. Parameter values γ = 1, β = 6, aR = aV = 0.05, σ = 0.15.

c>aR(βγ)(1σ)β(aRβσ+aRaV+γaVσγaR,) (3.34)

where the relative vaccine cost is c = cV/cI. If everyone is vaccinated as soon as immunity is lost (π̄ = ∞), the equilibrium force of infection is

λ=max{0,βIN}=max{0,β(1γσβ)aRaR+γ.} (3.35)

Instant vaccination of susceptible individuals (π* = ∞) is a Nash equilibrium if σβ/γ > 1 and

c<aR(βσγ)(1σ)(aRaV+σβaR+aVγaRγ)(βσ+σγγ). (3.36)

When aR = aV = a, if Inline graphic = β/γ > 2 and 0 < a < Inline graphic(Inline graphic − 2) and R0+aR0(R01)<σ<1, then there is an interval of relative costs c for which both vaccination and no vaccination are Nash equilibria (Figure 4). This interval can be bounded in terms of Eq. (3.34), Eq. (3.36), and the condition in which

(γσγ+a)2c22(1σ)(a+γ+σγ)c+(1σ)2=0 (3.37)

represents the location of a fold bifurcation in the utility function (Figure 5). Further study of the utility function shows that both of these equilibria have local invasion potential, and are separated by a third Nash equilibrium that never has invasion potential. Plots and numerical experiments suggest that the regions of non-uniqueness are small, and probably more narrow than the variances in both population heterogeneity and parameter estimates when waning rates are equal. This suggests that the multiple-equilibrium phenomena identified by Chen and Kremer et al. may be of limited practical importance, but further systematic analysis is needed, particularly in immunological contexts [Reluga et al., 2008, Heffernan and Keeling, 2009].

Figure 5.

Figure 5

Parameter-space diagrams of the bifurcation structure in the Nash equilibria of Eq. (3.32) as functions of the relative probability of infection σ and the relative cost of vaccine c = cV/cI. The right plot is a magnification of the intersecting region of the left plot. The curves represent sets of parameters where bifurcations occur. The plot is divided into 5 regions. If the cost of vaccine is very high, no vaccination (π* = 0) is the only equilibrium. If the cost of vaccine is very small and the relative probability of infection under vaccination is small, there is a unique equilibrium 0 < π* < ∞. If the relative probability of infection under vaccination is larger, individuals get vaccinated instantly on entering the susceptible compartment (π* = ∞). For vaccine costs near the threshold for no vaccination, there may be two locally evolutionarily stable equilibria: No or instant vaccination (π* = 0 and π* = ∞) if the relative probability of infection is sufficiently high, and no or a finite vaccination rate if the relative probability of infection is small enough. Parameter values γ = 1, β = 6, aR = aV = 0.05.

3.2 Delay strategies

A strategy space composed of constant hazards may not appeal to all researchers. Individuals probably do not choose waiting times independent of exponential distributions. An alternative formulation might be based on a decision to delay the next vaccination for a fixed time after the loss of immunity. The model we described here can easily be adapted to study this problem as well. Although the analysis is more difficult, the results are very similar to those for the constant hazard strategy space.

The switch from hazard to delay strategies is equivalent to a switch from exponential distributions to delta function distributions for the waiting time prior to vaccination. To formulate an epidemic model with fixed waiting times, we must switch gears from ordinary differential equations to integro-differential equations. The susceptible population will now be structured in terms of the amount of time that the individual has waited after becoming susceptible. This can be represented mathematically with a McKendrick--von Foerster style partial differential equation where the “age” dimension is interpreted as the waiting time. After individuals have been susceptible for a certain period of time, they are vaccinated. The longer individuals remain susceptible, the less often they must pay vaccination costs but the greater their accumulated risk of infection. This factor can be introduced as a removal hazard proportional to the force of infection. The number of newly infected individuals per unit time equals the total number of susceptible individuals becoming infected over all ages. Therefore, individuals optimize the trade-off between vaccine costs and infection risks.

Excepting the adjustments described above, the dynamics of the infected, vaccinated, and resistant states are the same as those described in System 3.1 when the mortality rate μ = 0. At the population scale, the dynamics satisfy

dSdt+dSdw=λ(I)S,S(0,t)=aR(t)=aV(t),w[0,w¯), (3.38a)
dIdt=λ(I)0w¯S(t,w)dwγI, (3.38b)
dRdt=γIaR, (3.38c)
dVdt=S(t,w¯)aV, (3.38d)

with the total population size

N=0w¯Sdw+I+R+V (3.38e)

staying constant. Here, the waiting time before vaccination, , is the population’s resident strategy.

The steady-state analysis of System (3.38) shows that there is always a disease-free stationary solution with I* = 0, V* = N/(1 + aw̄), and S*(a) = aN/(1 + aw̄). There is also an endemic disease stationary solution with I* > 0 if

limI0λ(I)γIaw¯N1+aw¯>1. (3.39)

When λ(I) = βI/N, the critical vaccination delay

w^=γaβ(βγ). (3.40)

Analogous to Eq. (3.7), the endemic disease incidence is the unique positive solution of the transcendental equation

λ(I)=aγI(1eλ(I)w¯)a(NI)(1eλ(I)w¯)γI (3.41)

This solution corresponds closely with (3.7) in the absence of population turnover.

Calculation of an individual’s utility is significantly more complicated in the delay strategy model than in the hazard-strategy model. We use a recursive decomposition of the path integral formulation of the expected utility given by Eq. (2.2) to reduce the necessary calculations to matrix arithmetic.

The instantaneous utility gain of each state, except the infected state, is 0. Upon infection, individuals lose utility at a rate of cI per unit time. In addition, there is an instantaneous cost cV paid every time an individual enters the vaccinated class. From these observations, we can construct recursive formulas for evaluating the expected utility of various initial states. Let UI, US, UV, and UR be the utilities conditional on initial infected, susceptible, vaccinated, and resistant states respectively. If an individual is initially in the infected state, then under the current model, there is a constant hazard γ per unit time of leaving the infected state and entering the recovered class. The total time t1 that the individual is in the infected state has distribution γeγt1. The individual loses utility at rate Φ(I) = −cI for every day of infection. Thus, expected utility should be the expectation of −cI t1 plus the expected utility of entering the recovered class at time t1, appropriately discounting the future relative to the present.

Mathematically, the expected utility is evaluated as follows:

UI=I0ehtΦ(x(t))dtDx (3.42a)
=0γeγt1[0t1ehtΦ(I)dt+Rt1ehtΦ(x(t))dtDx]dt1 (3.42b)
=0γeγt1[(1eht1h)(cI)+eht1R0ehtΦ(x(t))dtDx]dt1 (3.42c)
=0γeγt1[(eht11h)cI+eht1UR]dt1 (3.42d)
=cI+γURγ+h (3.42e)

The expected utilities of the other initial states are calculated in a similar fashion.

US=S(0)0ehtΦ(x(t))dtDx (3.43a)
=0wλ(I)eλ(I)t1[0t1ehtΦ(I)dt+It1ehtΦ(x(t))dtDx]dt1+eλ(I)wVwehtΦ(x(t))dtDx (3.43b)
=0wλ(I)eλ(I)t1[(1eht1h)Φ(S)+eht1UI]dt1+e(λ(I)+h)wUV (3.43c)
=0wλ(I)e(λ(I)+h)t1U1dt1+e(λ(I)+h)wUV (3.43d)
=λ(I)(1e(λ(I)+h)wλ(I)+h)UI+e(λ(I)+h)wUV (3.43e)
UV=cVV0ehtΦ(x(t))dtDx (3.44a)
=cV+0aeat1[0t1ehtΦ(V)dt+St1ehtΦ(x(t))dtDx]dt1 (3.44b)
=cV+aa+hUS (3.44c)
UR=R0ehtΦ(x(t))dtDx (3.45a)
=0aeat1[0t1ehtΦ(R)dt+St1ehtΦ(x(t))dtDx]dt1 (3.45b)
=aa+hUS (3.45c)

Thus, the conditional expected utility for each of the four initial conditions can be expressed as a linear equation in terms of the other conditional expected utilities. In matrix form,

[USUIURUV]=[0λ(I)(1e(λ(I)+h)wλ(I)+h)0e(λ(I)+h)w00γγ+h0aa+h000aa+h000][USUIURUV]+[0cIγ+h0cV] (3.46)

This matrix equation can be solved to determine the conditional expected utilities. The unconditional expected utility is then determined by weighing each conditional expected utility and the probability of the corresponding initial condition.

However, in the case of slow discounting (small h), it is useful to study the asymptotic behavior. Let the asymptotic utility gain rate

U+=limh0hUx(0)=aλ(I)[cI(eλ(I)w1)+cVγ][(γ+a)λ(I)+aγ]eλ(I)wa[γ+λ(I)]. (3.47)

Note that U+ is independent of the initial condition x(0).

An example of U+ is plotted in Figure 6. If an individual chooses a very long (w = ∞) or a very short (w = 0) delay, the asymptotic utility gain

Figure 6.

Figure 6

An individual’s asymptotic expected absolute utility losses (left) and relative utility losses (right) calculated from Eq. (3.47) as a function of the individual’s strategy and the population’s resident strategy. Parameter values are the same as those used in Figures 1 and 2.

U+(,w¯)=aλ(I)cI(γ+a)λ(I)+aγ,andU+(0,w¯)=acV. (3.48)

As in the hazard strategy example, ∂U+/∂w vanishes for all individual strategies w when the force of infection λ satisfies

cV=cIa+γ+aγλ(I). (3.49)

Therefore, if there is a non-negative vaccination strategy w* in which the force of infection satisfies Eq. (3.49) and (3.41) when = w*, then w* is a Nash equilibrium. When

cIa+γ+aγλ(I(w¯=0))<cV<cIa+γ+aγλ(I(w¯=)), (3.50)

Eq. (B.7) is negative because d2U+/dw2 = 0 and

d2U+dw¯dw(w=w,w¯=w)=aγcV2eλw[cV(γ+a)cI][cI(eλw1)+cVγ]2dλdw¯<0, (3.51)

because cV (γ + a) < cI by Eq. (3.50). Thus, the critical strategy also has local invasion potential. If

cIa+γ+aγλ(I(w¯=))<cV, (3.52)

then the equilibrium shifts to no vaccination. This condition is automatically satisfied when cV > cI/(γ + a). If

cV<cIa+γ+aγλ(I(w¯=0)), (3.53)

then the equilibrium would be vaccination as soon as the individual loses immunity. But λ(I*( = 0)) = 0, so the right hand side diverges and this inequality is never satisfied under the current model.

Whereas the force of infection is the same in both models for the equilibrium strategy, the individual utility gain at the critical delay-strategy is

U+(w,w)=max{acV,acIa+γ+aγλ(I)}. (3.54)

This is consistent with the hazard strategy model.

4 Discussion

We have described a general method for calculating the utilities of strategies to individuals in population games. By combining deterministic models of population dynamics with stochastic models of an individual’s dynamics in an optimal control setting, we have obtained a powerful reformulation of population games for the study of decision making.

Our method is widely applicable to aggregate games with explicit population dynamics. For example, our method facilitates the integration of models for social and biological systems, game theory and public policy. Our method also permits the calculation of utilities from relatively complex population games. Consequently, our method facilitates increased complexity of population game models in order to take into account realism that could not be incorporated previously, including population heterogeneity, such as with regard to risk aversion or age [Galvani et al., 2007].

By applying our method to two vaccination policies that differ in vaccination timing, we showed that the utilities of vaccination decisions can be determined using Markov decision process theory. We demonstrated the application of these utilities for determining the invisibility of strategies and for calculating Nash equilibria. We also used our method to characterize the utility landscape surrounding critical strategies, thereby revealing that the relationship between the waning of vaccine-induced immunity and the level of vaccination at the Nash equilibrium is not monotone. When vaccine waning is very slow, small increases in the rate of waning elevate expected vaccination demand. Around the critical cost threshold, however, more rapid waning decreases vaccination.

Ongoing research includes the analysis of differential games where population dynamics are not stationary, the analysis of trembling hand equilibria conditions, and extensions to populations including age, space, and contact network structures. Previous work has demonstrated the potential for vaccination behavior to generate unstable dynamics [Muller, 1997, Reluga et al., 2006], but these have not yet been studied as games. There are naturally occurring problems where the attractor of the population-scale dynamics is periodic, leading to cases where the transition rate matrix Q* is also periodic. In elementary cases, calculation of the expected utility over long horizons under periodic oscillations can be reduced to the solution of Eq. (2.5) with the periodic boundary condition:

v(t0)=v(t0+tp)ehtp, (4.1)

where tp is the period of oscillation. One particular issue of interest is determining if approximation methods are appropriate in cases where the population-scale dynamics are quasiperiodic or chaotic. It is also important to determine alternative methods to specify the coupling between an individual’s actions and the population dynamics. In cases where individuals have perfect information, clear motives, and act independently, game-theory approaches such as that studied here are justifiable. However, human behavior frequently diverges from these assumptions, and many alternative approaches have yet to be fully explored [Gigerenzer and Todd, 2000, Kahneman, 2003].

In conclusion, we described a general method for the calculation of the utilities of strategies to individuals in population games. This method offers a framework for integrating individual and aggregate behaviors. We exemplified its applicability by employing our method to determine the invisibility of strategies and to calculate Nash equilibria for vaccination decisions against seasonal influenza. We showed that the utilities of vaccination decisions can be determined using Markov decision process theory. Our method can be widely applied for population games and permits calculation of utilities from relatively complex systems.

Acknowledgments

This research was supported by NIH grant R01AI072706 (APG and TCR), and Bill and Melinda Gates Foundation Grant Number 49276 (TCR). We thank Jing Li for her suggestions and an anonymous reviewer for robust comments, including simplifications to the justification of Eq. (3.11).

A A Hamiltonian Formulation of Population Games

Using an optimal-control framework, individuals wish to maximize

U(π)=t0tfeh(tt0)[fT+1T(FQ)]pdt

subject to the dynamic constraints

p.=Q(n,t,π(t))p,p(t0)=p0,n.=Gn(n,e,t,π¯(t)),n(t0)=n0,e.=Ge(n,e,t,π¯(t)),e(t0)=e0.

The Hamiltonian

H=eh(tt0)[fT+1T(FQ)]p+vtQp+knTGn+keTGe

where v, kn, and ke are vectors of Lagrange multipliers. A strategy π* is a Nash equilibrium if it solves

π=argmaxπH(π,π)

subject to the constraint equations

dpdt=Hv,dndt=HknT,dedt=HkeT,dvTdt=Hp=eh(tt0)[fT+1T(FQ)]vTQ,dknTdt=Hn=vTQnpknTGnnkeTGen,dkeTdt=He,=vTQepknTGnekeTGee.

A change of variables puts these equations in the present-value form used in Section 2.1. Note that the optimization is independent of the population-scale adjoint variables kn and ke, so the last two of these equations can be ignored within the scope of this paper.

B A Review of Equilibria Concepts for Population Games

For introductions to the field of game theory, see Osborne [2004], Rasmusen [2006], Hofbauer and Sigmund [1998]. The game-theoretic aspect of our analysis focuses on the properties of the expected utility U(π, π̄) as a function of the individual’s strategy π and the effective average of the population’s aggregated strategy π̄, referred to as the resident strategy.

Strategies used in a population game are commonly classified in terms of their ability to replace and to be replaced by neighboring strategies [Geritz et al., 1998, Christiansen and Loeschcke, 1980]. Strategies with special properties in terms of replacement are called equilibria. The challenges of identifying equilibria are closely analogs of the challenges of identifying the global maximums of a function: there are general rules that define the properties of equilibria, and there are differentiation rules that greatly facilitate their identification.

A strategy π* is a global Nash equilibrium if no other alternative strategy has a higher utility than the strategy itself when invading a population using the strategy π*; i.e.

π,U(π,π)U(π,π). (B.1)

A strategy π* is a strict Nash equilibrium if every alternative strategy has less utility than the strategy π*; i.e.

ππ,U(π,π)<U(π,π). (B.2)

Equally important is whether a strategy can successfully invade populations with differing resident strategies. A strategy π* has global invasion potential if it never does worse than any resident strategy played against itself; i.e.

π¯,U(π,π¯)U(π¯,π¯). (B.3)

A strategy π* has strict global invasion potential if equality only holds when the strategy π* is itself the resident strategy, i.e.

π¯π,U(π,π¯)>U(π¯,π¯). (B.4)

Those Nash equilibria with invasion potential where at least one of the conditions is strict are evolutionarily stable strategies (ESS’s) of the population game [Thomas, 1985, Mesterton-Gibbons, 1992].

The definitions above place no restrictions on any set of strategies or on the utility function being studied. They can be applied in general. However, for nicely behaved utility functions, there are additional rules that facilitate the efficient identification of equilibria. In the applications presented here, the strategy space is composed of real numbers, and the utility function is continuous and differentiable almost everywhere. Therefore, it is convenient to define local forms of the Nash and invasion conditions that can be tested for using differential calculus and Taylor series. In a single population with a 1-parameter strategy space, a strategy π* is a critical strategy of a utility function U(π, π̄) if the following equation holds

dUdπ(π,π)=0. (B.5)

A critical strategy is a strict local Nash equilibrium if

d2Udπ2(π,π)<0. (B.6)

A critical strategy has strict local invasion potential if

d2Udπ2(π,π)+2d2Udπdπ¯(π,π)<0. (B.7)

If we define the relative utility function

U(π,π¯)=U(π,π¯)U(π¯,π¯), (B.8)

the local invasion potential condition may be expressed in the simplified form

d2Udπ¯2(π,π)>0. (B.9)

A local Nash equilibrium with local invasion potential is a local ESS.

Note that Eq. (B.7) differs from the more-common convergent-stability condition [Eshel, 1983]

d2Udπ2(π,π)+d2Udπdπ¯(π,π)<0, (B.10)

which implies that small improvements in strategy will lead to the critical point. A local ESS is convergently stable. However, a critical point need not have local invasion potential to be convergently stable.

These differential conditions can be generalized to vector strategies. For a discussion of generalizations to multiple types of actors, see Reluga [2009].

Footnotes

1

Notation: Throughout this paper, bold-face lower-case letters represent vectors, while bold-face upper case letters represent matrices or linear operators

2

The components of the Hadamard product are the product of the components of F and Q, (FQ)jk = FjkQjk.

3

A correspondence is different from a function in that there may be many different outputs for a given input. Mathematically, correspondences are often replaced by functions mapping to the domain power set of the correspondences target. However, such mathematical consideration obscures the important fact that best-responses are not always unique.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Contributor Information

Timothy C. Reluga, Email: timothy@reluga.org.

Alison P. Galvani, Email: alison.galvani@yale.edu.

References

  1. Kenneth J. Public Investment, the Rate of Return, and Optimal Fiscal Policy. Johns Hopkins Press; Baltimore, MD: 1970. Arrow and Mordecai Kurz. [Google Scholar]
  2. Barrett S. Global disease eradication. Journal of the European Economic Association. 2003;1(2):591–600. [Google Scholar]
  3. Bauch CT, Earn DJD. Vaccination and the theory of games. Proceedings of the National Academy of Sciences. 2004;101:13391–13394. doi: 10.1073/pnas.0403823101. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Bauch Chris T. Dynamic games with imitation predict vaccinating behavior. Proceedings of the Royal Society of London, Series B. 2005;272(1573):1669–1675. doi: 10.1098/rspb.2005.3153. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Bauch Chris T, Galvani Alison P, Earn David JD. Group interest versus self-interest in smallpox vaccination policy. Proceedings of the National Academy of Sciences. 2003 September 2;100(18):10564–10567. doi: 10.1073/pnas.1731324100. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Brito DL, Sheshinski E, Intriligator MD. Externalities and compulsory vaccinations. Journal of Public Economics. 1991;45:69–90. [Google Scholar]
  7. Chen FH. Rational behavioral response and the transmission of stds. Theoretical Population Biology. 2004;66(1):307–316. doi: 10.1016/j.tpb.2004.07.004. [DOI] [PubMed] [Google Scholar]
  8. Chen FH. A susceptible-infected epidemic model with voluntary vaccinations. Journal of Mathematical Biology. 2006a;53(1):253–272. doi: 10.1007/s00285-006-0006-1. [DOI] [PubMed] [Google Scholar]
  9. Chen FH. On the transmission of hiv with self-protective behavior and preferred mixing. Mathematical Biosciences. 2006b;199(1):141–159. doi: 10.1016/j.mbs.2005.12.004. [DOI] [PubMed] [Google Scholar]
  10. Chen Frederick H. Modeling the effect of information quality on risk behavior change and the transmission of infectious diseases. Mathematical Biosciences. 2009;217:125–133. doi: 10.1016/j.mbs.2008.11.005. [DOI] [PubMed] [Google Scholar]
  11. Christiansen FB, Loeschcke V. Evolution and intraspecific exploitative competition I. One-locus theory for small additive gene effects. Theoretical Population Biology. 1980;18(3):297–313. doi: 10.1016/0040-5809(84)90031-5. [DOI] [PubMed] [Google Scholar]
  12. Eshel I. Evolutionary and continuous stability. Journal of Theoretical Biology. 1983;103(1):99–111. doi: 10.1006/jtbi.1996.0312. [DOI] [PubMed] [Google Scholar]
  13. Fine PEM, Clarkson JA. Individual versus public priorities in the determination of optimal vaccination policies. American Journal of Epidemiology. 1986;124(6):1012–1020. doi: 10.1093/oxfordjournals.aje.a114471. [DOI] [PubMed] [Google Scholar]
  14. Francis PJ. Optimal tax/subsidy combinations for the flu season. Journal of Economic Dynamics and Control. 2004 September;28(10):2037–2054. [Google Scholar]
  15. Galvani AP, Reluga TC, Chapman G. Long-standing influenza vaccination policy is in accord with individual self-interest but not with the utilitarian optimum. Proceedings of the National Academy of Sciences. 2007;104(3):5692–5697. doi: 10.1073/pnas.0606774104. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Geoffard Pierre-Yves, Philipson Tomas. Disease eradication: Private versus public vaccination. The American Economic Review. 1997 March;87(1):222–230. [Google Scholar]
  17. Geritz SAH, Kisdi E, Meszena G, Metz JAJ. Evolutionary singular strategies and the adaptive growth and branching of the evolutionary tree. Evolutionary Ecology. 1998;12:35–57. [Google Scholar]
  18. Gigerenzer Gerd, Todd Peter M. Simple Heuristics That Make Us Smart. Oxford University Press; 2000. [DOI] [PubMed] [Google Scholar]
  19. Heffernan JM, Keeling MJ. Implications of vaccination and waning immunity. Proceedings of the Royal Society B. 2009;276:2071–2080. doi: 10.1098/rspb.2009.0057. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Hofbauer J, Sigmund K. Evolutionary Games and Population Dynamics. Cambridge University Press; Cambridge, UK: 1998. [Google Scholar]
  21. Howard Ronald A. Dynamic Programming and Markov Processes. MIT Press; Cambridge, MA: 1960. [Google Scholar]
  22. Kahneman D. Maps of Bounded Rationality: Psychology for Behavioral Economics. American economic review. 2003;93(5):1449–1475. [Google Scholar]
  23. Kermack WO, McKendrick AG. Contributions to the mathematical-theory of epidemics. Proceedings of the Royal Society of London. 1927;115:700–721. [Google Scholar]
  24. Kingman JFC. Markov transition probabilities. I. Probability Theory and Related Fields. 1967;7(4):248–270. [Google Scholar]
  25. Kremer Michael, Snyder Christopher M, Williams Heidi. Working paper. 2006. Which vaccines deserve the largest subsides? an integrated economic and epidemiological model. [Google Scholar]
  26. Mesterton-Gibbons Michael. An Introduction to Game-Theoretic Modelling. Addison-Wesley; Redwood City, CA: 1992. [Google Scholar]
  27. Muller J. Optimal vaccination strategies - for whom? Mathematical Biosciences. 1997 January 15;139(2):133–154. doi: 10.1016/s0025-5564(96)00140-x. [DOI] [PubMed] [Google Scholar]
  28. Osborne MJ. An introduction to game theory. Oxford University Press; New York, NY: 2004. [Google Scholar]
  29. Philipson T. Economic epidemiology of infectious disease. In: Culyer Anthony J, Newhouse Joseph P., editors. Handbook of health economics. 1B. Elsevier; New York, NY: 2000. pp. 1761–1797. [Google Scholar]
  30. Puterman Martin L. Markov decision processes: discrete stochastic dynamic programming. Wiley; Hoboken, NJ: 2005. [Google Scholar]
  31. Rasmusen E. Games and Information: An Introduction to Game Theory. Blackwell Publishing; 2006. [Google Scholar]
  32. Reluga T. Game theory of social distancing in response to an epidemic. PLOS Computational Biology. 2010;6(5):e1000793. doi: 10.1371/journal.pcbi.1000793. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Reluga TC. An SIS game with two subpopulations. Journal of Biological Dynamics. 2009;3(5):515–531. doi: 10.1080/17513750802638399. [DOI] [PubMed] [Google Scholar]
  34. Reluga TC, Bauch CT, Galvani AP. Evolving public perceptions and stability in vaccine uptake. Mathematical Biosciences. 2006;204:185–198. doi: 10.1016/j.mbs.2006.08.015. [DOI] [PubMed] [Google Scholar]
  35. Reluga TC, Medlock J, Poolman E, Galvani AP. Optimal timing of disease transmission in an age-structured population. Bulletin of Mathematical Biology. 2007;69(8):2711–2722. doi: 10.1007/s11538-007-9238-5. [DOI] [PubMed] [Google Scholar]
  36. Reluga TC, Medlock J, Perelson A. Backward bifurcations and multiple equilibria in epidemic models with structured immunity. Journal of Theoretical Biology. 2008;252:155–165. doi: 10.1016/j.jtbi.2008.01.014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Rockafellar RT. Coherent approaches to risk in optimization under uncertainty. Tutorials in Operations Research INFORMS. 2007:38–61. [Google Scholar]
  38. Ross R, Hudson HP. An application of the theory of probabilities to the study of a priori pathometry - part II. Proceedings of the Royal Society London A. 1917;43:212–225. [Google Scholar]
  39. Sonnenberg FA, Beck JR. Markov models in medical decision making: a practical guide. Medical Decision Making. 1993;13(4):322. doi: 10.1177/0272989X9301300409. [DOI] [PubMed] [Google Scholar]
  40. Sterman John D. Learning from evidence in a complex world. American Journal of Public Health. 2006;96:505–514. doi: 10.2105/AJPH.2005.066043. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Thomas Bernhard. On evolutionarily stable sets. Journal of Mathematical Biology. 1985;22:105–115. [Google Scholar]

RESOURCES