Fast estimation of time-varying infectious disease transmission rates

Abstract

Compartmental epidemic models have been used extensively to study the historical spread of infectious diseases and to inform strategies for future control. A critical parameter of any such model is the transmission rate. Temporal variation in the transmission rate has a profound influence on disease spread. For this reason, estimation of time-varying transmission rates is an important step in identifying mechanisms that underlie patterns in observed disease incidence and mortality. Here, we present and test fast methods for reconstructing transmission rates from time series of reported incidence. Using simulated data, we quantify the sensitivity of these methods to parameters of the data-generating process and to mis-specification of input parameters by the user. We show that sensitivity to the user’s estimate of the initial number of susceptible individuals—considered to be a major limitation of similar methods—can be eliminated by an efficient, “peak-to-peak” iterative technique, which we propose. The method of transmission rate estimation that we advocate is extremely fast, for even the longest infectious disease time series that exist. It can be used independently or as a fast way to obtain better starting conditions for computationally expensive methods, such as iterated filtering and generalized profiling.

Author summary

Many pathogens cause recurrent epidemics. Patterns of recurrence are strongly affected by seasonality of the transmission rate, which can arise from seasonal changes in weather and host population behaviour (e.g., aggregation of children in schools). To understand and predict recurrent epidemic patterns, it is essential to reconstruct the time-varying transmission rate, which is never observed directly. Existing transmission rate estimation methods tend to fall into one of two categories: accurate but too slow to apply to long time series of reported incidence, or fast but inaccurate unless the number of individuals initially susceptible to infection is known precisely. Here, we introduce and compare fast methods inspired by the algorithm that Fine and Clarkson pioneered in the early 1980s. The method that we suggest accurately reconstructs seasonally varying transmission rates, even with crude information about the initial size of the susceptible population.

1 Introduction

The transmission rate of an infectious disease is a salient quantity in the study of epidemics. Changes in the transmission rate over time greatly influence the spread of infection [1, 2]. Quantifying how it changes over time can elucidate factors governing disease spread (e.g., weather [3], contact patterns [4]), inform epidemic forecasts, and suggest strategies for epidemic control [5].

In practice, we do not observe transmission directly. Instead, we observe the number of cases of infection (disease incidence) or number of deaths from infection (disease mortality) reported over time, and must reconstruct time-varying transmission rates from these data [6–13]. Utilizing historical mortality records, it is possible to identify patterns in transmission dating far back in time. Most notably, the London Bills of Mortality and the Registrar General’s Weekly Returns enable investigation of transmission patterns continuously from the mid-17th century to the present, for a number of infectious diseases including cholera [14] and smallpox [15].

A mechanistic understanding of long infectious disease time series—three centuries of weekly data in the case of smallpox [15]—requires methods of transmission rate estimation that are both accurate and fast, and therefore tractable for long time scales. Simulation-based methods of transmission rate estimation from reported incidence or mortality have been developed using the susceptible-infected-removed (SIR) model for infectious disease dynamics [16]. Markov chain Monte Carlo (MCMC [17, 18]) and sequential Monte Carlo (as in iterated filtering [8, 19, 20]) methods are statistically rigorous, but not tractable for long time scales owing to high computational cost. Generalized profiling [21, 22], which combines trajectory and gradient matching, is faster, but still too slow for convenient exploration of time series spanning hundreds of years. (Several CPU hours were required to apply generalized profiling to just 26 years of weekly data [22].)

In comparison, Finkenstädt and Grenfell’s popular “time series SIR” (tSIR) method [7, 23] is extremely fast, using a simple discretization of a continuous-time SIR model to reduce transmission rate estimation to a local regression problem. However, the tSIR method assumes that the duration of infection is equal to the time step, that natural death of susceptible individuals can be ignored, and that cumulative incidence approximates cumulative births. The latter two assumptions are reasonable for pre-vaccination measles, when most susceptible individuals were eventually infected [6]. However, in many contexts (e.g., with pathogens less transmissible than measles), susceptible mortality over time scales of interest and the difference between incidence and births are non-negligible.

In their unpublished PhD and MSc theses, Krylova (Ch. 4 in [24]) and deJonge [25] separately modified a fast discretization method originally proposed by Fine and Clarkson [6]. Krylova’s approach has been employed to estimate the amplitude of seasonal variation in measles transmission in 20th century New York City [9]. Unlike the tSIR method and unlike Fine and Clarkson, Krylova’s and deJonge’s methods do not place constraints on the infectious period or ignore susceptible mortality.

Here, we present a new algorithm based on deJonge’s method and compare its performance to the methods of Fine and Clarkson and Krylova. Our main investigative approach is to apply each method to simulated reported incidence data with known underlying transmission rate, so that error in transmission rate estimates can be quantified exactly.

Our analysis of the methods reveals a shared sensitivity to process and observation error. We mitigate this issue by introducing a smoothing step. The methods are additionally sensitive to error in the user’s estimate of the initial number of susceptible individuals, which is rarely known with any precision. We propose a fast, iterative technique for estimating this parameter from time series of incidence, births, and natural mortality, eliminating a long-standing barrier to the use of fast methods of transmission rate reconstruction.

2 Methods

In §§2.1 and 2.2 below, we present three fast methods for estimating time-varying transmission rates, based on a mechanistic model of disease spread. In §§2.3–2.7, we outline our systematic analysis of the sensitivity of the methods to parameters of the data-generating process and to error in the user-specified values of input parameters. Finally, in §2.8, we introduce peak-to-peak iteration (PTPI), a technique for estimating the initial number of susceptible individuals. Essential notation is summarized in Table 1.

Table 1. Notation.

Unless otherwise stated, simulations of reported incidence time series use the reference values listed here. If a symbol is to be interpreted differently in relation to disease incidence and disease mortality data, then the correct definition is indicated by (I) and (M), respectively.

Symbol	Name	Definition	Ref. val.	Unit	Notes
tk	kth observation time	Time of the kth observation in time series data, for k = 0, …, n.	t0 + kΔt	years
t0	Transient period	Duration of the transient in system (1) that is ignored in simulations of reported incidence, before observations are recorded.	2000	years	System (1) is numerically integrated between t = 0 years and t = t0, and observed starting at t = t0. This is done so that simulations reflect dynamics near the attractor of system (1).
Δt	Observation interval	Time between successive observations in time series data.	1	weeks	Disease mortality is reported weekly in the London Bills of Mortality.
n	Time series length	Time between the initial and final observations in time series data, in units Δt, given by (tn − t0)/Δt.	1042	—	If Δt = 1 week, then 1042Δt = ⌊20 × 365/7⌋Δt ≃ 20 years.
[·]Δt	Nearest kΔt rounding	For time lengths t, $[t{]}_{\Delta t}=\left[\frac{t}{\Delta t}\right]\Delta t$, where [⋅] denotes nearest integer rounding.	—	—
〈⋅〉	Long-term averaging	For functions x(t), $\langle x\rangle ={\text{lim}}_{t\to \infty }\frac{1}{t}{\int }_0^{t}x\left(s\right)\text{d}s$. For sequences xk, $\langle x\rangle ={\text{lim}}_{n\to \infty }\frac{1}{n}{\sum }_{k=0}^n{x}_k$.	—	—
(S(t), I(t), R(t))	State	Number of (susceptible, infected, removed) individuals in the population at time t.	—	—	“Removed” individuals have either recovered from the disease and gained permanent immunity or died from the disease.
N(t)	Population size	S(t) + I(t) + R(t).	—	—
B(t)	Births	Number of births that occur during the time interval [t − Δt, t).	—	—
Q(t)	Cumulative incidence	Number of susceptibles who become infected during the time interval [t0, t).	—	—
Z(t)	Incidence	Number of susceptibles who become infected during the time interval [t − Δt, t).	Q(t) − Q(t − Δt)	—
C(t)	Reported disease (I) incidence or (M) mortality	Number of (I) infections or (M) disease-induced deaths reported during the time interval [t − Δt, t).	Eq (31)	—	C is an abbreviation of “cases”, which are reported as infections or as deaths.
(S0, I0, R0)	Initial state (t = t0)	(S(t0), I(t0), R(t0)).	(S*, I*, R*)	—	(S*, I*, R*) denotes the state of system (1) after numerical integration between t = 0 years and t = t0 with seasonally forced transmission rate β(t) (Eq (27)), constant vital rates νc and μc, and initial state (t = 0 years) $(\widehat{S},\widehat{I},\widehat{R})$ (Eq (32); see below).
N0	Initial population size (t = t0)	N(t0).	S* + I* + R*	—
$(\widehat{S},\widehat{I},\widehat{R})$	Endemic equilibrium	Endemic equilibrium of system (1) with constant transmission rate (β ≡ 〈β〉) and constant vital rates (ν ≡ μ ≡ μc).	Eq (32)	—
${\widehat{N}}_0$	Initial population size (t = 0 years)	N(0).	106	—
xk	Estimation input/output	Within an estimation algorithm (Boxes 1–3), the supplied or estimated value of x(tk) (x = C, B, μ, Z, S, I, β).	—	varies
ν(t)	Birth rate	Number of births per unit time relative to ${\widehat{N}}_0$, at time t.	νc	year−1	In simulations of reported incidence, ν(t) is modeled as a constant νc. In general, estimation of β(t) from data does not require the underlying ν(t) to be constant.
νc	Birth rate (constant)	See ν(t) above.	0.04	year−1
μ(t)	Natural mortality rate	Number of natural deaths per unit time per capita, at time t.	μc	year−1	In simulations of reported incidence, μ(t) is modeled as a constant μc. In general, estimation of β(t) from data does not require the underlying μ(t) to be constant.
μc	Natural mortality rate (constant)	See μ(t) above.	0.04	year−1
tgen	Mean generation interval	Mean time between onset of infection (in infector) and subsequent transmission of infection (by infector) [26, 27].	13	days	The reference value is the sum of the observed mean latent and infectious periods, which for measles are 8 days and 5 days, respectively [16].
γ	Removal rate	Number of removals (recoveries or deaths from disease) per unit time per infected.	1/tgen	day−1
prep	Case reporting probability	(I) Probability that an infection is reported, or (M) the case fatality ratio times the probability that a death from disease is reported.	0.25 or 1	—	If we simulate data with under-reporting, then we use prep = 0.25 as a reference value. Otherwise, we set prep = 1.
trep	Mean case reporting delay	Mean time between infection and reporting of (I) infection or (M) disease-induced death.	2 or 0	weeks	If we simulate data with reporting delays, then we use trep = 2 weeks as a reference value. Otherwise, we set trep = 0 weeks.
β(t)	Transmission rate	Number of infections per unit time per susceptible per infected, at time t.	Eq (27)	year−1	In simulations of reported incidence, β(t) is modeled by the seasonal forcing function defined in Eq (27). In general, estimation of β(t) from data does not require it vary seasonally or even periodically.
〈β〉	Mean transmission rate	Continuous-time average of the seasonally forced β(t), equal to ${\text{lim}}_{t\to \infty }\frac{1}{t}{\int }_0^t\beta \left(s\right)\text{d}s$.	β*	year−1	β* ≈ 5.6 × 10−4year−1 is the value of 〈β〉 that satisfies Eq (2) with ${\mathcal{R}}_0=20$, νc = μc = 0.04 year−1, tgen = γ−1 = 13 days, and ${\widehat{N}}_0={10}^6$.
${\mathcal{R}}_0$	Basic reproduction number	Number of individuals that a typical infected person is expected to infect in an otherwise completely susceptible population.	Eq (2)	—	For measles in the 20th century, ${\mathcal{R}}_0\approx 20$ [16].
α	Seasonal amplitude	Amplitude of the seasonally forced β(t) relative to 〈β〉.	0.08	—	For measles, α ≈ 0.08 [2]. We require α ∈ [0, 1] to ensure that the seasonal forcing function defined in Eq (27) is non-negative.
ϕ(t; ϵ)	Environmental noise (realized)	Phase shift in the seasonally forced β(t), at time t.	Normal(0, ϵ2)	—	ϕ is a realization of a continuous-time stochastic process defined by a set {Φ(t; ϵ)} of independent and Normal(0, ϵ2)-distributed random variables.
ϵ	Standard deviation of environmental noise	See ϕ(t; ϵ) above.	0.5	—
q	Loess smoothing parameter	Rough number of nearest neighbours weighted in local regression (i.e., when fitting loess curves to time series), determining the degree of smoothing.	—	—	See §2.2.6 for an exact definition.

2.1 Model of disease transmission

We assume that the principal mechanisms of disease spread in the focal population are captured by the SIR model [16], formulated with time-varying rates of birth, death, and transmission. Expressing the model as a system of ordinary differential equations, we write

$$\frac{\text{d}S}{\text{d}t}=\nu \left(t\right){\widehat{N}}_0-\beta \left(t\right)SI-\mu \left(t\right)S,$$ (1a)

$$\frac{\text{d}I}{\text{d}t}=\beta \left(t\right)SI-\gamma I-\mu \left(t\right)I,$$ (1b)

$$\frac{\text{d}R}{\text{d}t}=\gamma I-\mu \left(t\right)R,$$ (1c)

where S, I, and R are the numbers of individuals who are susceptible, infected, and removed, respectively; N = S + I + R is the population size; and ${\widehat{N}}_0=N\left(0\right)$ is the population size at an initial time, defined to be 0 years for simplicity. (We reserve the notation N₀ for N(t₀), where t₀ > 0 years is the length of a transient; see Table 1.)

The time-varying parameters are

• ν(t) birth rate, the number of births per unit time relative to ${\widehat{N}}_0$;

• μ(t) natural mortality rate, the number of natural deaths per unit time per capita (i.e., relative to N); and

• β(t) transmission rate, the number of infections per unit time per susceptible per infected.

The constant parameter γ is the rate of removal from the infected compartment (due to recovery or death from disease) per infected individual.

In Eq (1a) and Eq (1b), we use mass action incidence β(t)SI rather than standard incidence β(t)SI/N. Mass action incidence is essential for reproducing transitions in epidemic patterns resulting from changes in the birth rate [2, 28]. In Eq (1a), we write the net birth rate as $\nu \left(t\right){\widehat{N}}_0$ rather than ν(t). This formulation is for convenience: the factor of ${\widehat{N}}_0$ does not affect dynamics, but ensures that ν(t) and μ(t) have the same scale.

The SIR model (1) assumes that the focal population is homogeneously mixed and subject to the mass action principle, which states that incidence is proportional to the product of the densities of susceptibles and infecteds [16]. The model further assumes that the latent period (time from infection to onset of infectiousness) can be ignored and that the infectious period (time from onset of infectiousness to recovery or death from disease) is exponentially distributed [29]. The distributions of the latent and infectious periods affect disease dynamics [28, 30, 31], but Krylova and Earn [28] showed that the effect on long-term dynamical structure is typically small when the mean generation interval is fixed (see Fig 11 in [28]). For this reason, we assign the mean generation interval implied by the SIR model (1) (t_gen = γ⁻¹) the value of the sum of the observed mean latent and infectious periods. This sum is the true mean generation interval if the latent and infectious periods are both exponentially distributed, and is a good estimate of the true mean generation interval more generally [28].

Transmissibility of infection is typically measured by the basic reproduction number ${\mathcal{R}}_0$, defined as the number of individuals that a typical infected person is expected to infect in an otherwise completely susceptible population [16]. If the birth and death rates are constant (ν ≡ ν_c and μ ≡ μ_c), and if the transmission rate has a well-defined average 〈β〉 [32], then the basic reproduction number for the SIR model (1) can be written as [28]

$$\begin{array}{c}\hfill {\mathcal{R}}_0=\frac{{\nu }_{\text{c}}{\widehat{N}}_0}{{\mu }_{\text{c}}}·\frac{⟨\beta ⟩}{\gamma +{\mu }_{\text{c}}}.\end{array}$$ (2)

2.2 Estimating β(t) from time series data

Here, we examine three fast methods for estimating time-varying transmission rates β(t). The methods take as input (i) time series of reported disease incidence or disease mortality, (ii) time series of births and natural mortality, and (iii) values for input parameters, such as the mean generation interval t_gen. By assumption, the time series data are available at discrete, equally spaced time points

$$\begin{array}{c}\hfill t_k=t_0+k\Delta t,k=0,\dots ,n,\end{array}$$ (3)

where Δt is the observation interval. The methods return as output a time series estimate of β(t), denoted by ${\left\{(t_k,{\beta }_k)\right\}}_{k=0}^n$ or simply β_k, which can be averaged (§2.2.5) or smoothed (§2.2.6) to distill temporal patterns of interest.

Missing data must be imputed: the three methods are recursive, so they break down as soon as they encounter a missing value. Imputation can be accomplished most simply via linear interpolation between available data. More sophisticated techniques accounting for uncertainty in missing values are described in [33].

2.2.1 FC method

We review the method first described by Fine and Clarkson [6], referred to here as the “FC method”. Let S(t) and I(t) be the number of susceptibles and infecteds in the population at time t. S decreases when susceptibles become infected or die and increases when susceptibles are born. Let Z(t) and B(t) be the number of infections and births, respectively, that occur during the time interval [t − Δt, t). Assuming that natural mortality was negligible, Fine and Clarkson reconstructed S from Z and B with the recursion

$$\begin{array}{c}\hfill S(t+\Delta t)\approx S\left(t\right)+B(t+\Delta t)-Z(t+\Delta t).\end{array}$$ (4)

Fine and Clarkson further assumed that the observation interval Δt was equal to the mean generation interval t_gen, so that prevalence could be approximated by incidence. That is,

$$\begin{array}{c}\hfill I\left(t\right)\approx Z\left(t\right)\end{array}$$ (5)

for all t. They derived an expression for Z(t + Δt) via the mass action principle

$$\begin{array}{c}\hfill Z(t+\Delta t)\approx \beta \left(t\right)S\left(t\right)I\left(t\right)\Delta t.\end{array}$$ (6)

Rearranging Eq (6), they obtained an estimate of β(t), given by

$$\begin{array}{c}\hfill \beta \left(t\right)\approx \frac{Z(t+\Delta t)}{S\left(t\right)I\left(t\right)\Delta t}.\end{array}$$ (7)

Fine and Clarkson applied Eqs (4), (5), and (7) to estimate S(t_k), I(t_k), and β(t_k) (for k = 0, …, n), after specifying (i) the initial number of susceptibles S₀ = S(t₀), and (ii) values of Z(t_k) and B(t_k) from incidence and birth data, respectively.

A limitation of the FC method is the constraint requiring Δt = t_gen. For some diseases, this is a minor issue, because incidence and birth data can be aggregated so that the time between successive aggregates is approximately equal to t_gen. For example, the mean generation interval of measles is approximately two weeks, so Fine and Clarkson [6] aggregated pairs of weekly observations. A second, more serious limitation is the assumption, implicit in Eqs (4) and (5), that natural mortality is negligible. We discuss this issue in §3.1.

2.2.2 S method

Krylova (Ch. 4 in [24]) generalized the FC method in order to eliminate the constraint requiring Δt = t_gen and account for natural mortality. Her approach is based on the SEIR model, which distinguishes “exposed” individuals in the latent stage of infection from infectious individuals. Here, we adapt Krylova’s approach to the SIR model (1) and refer to our approach as the “S method.”

We define S, I, Z, and B as in the FC method. Let μ(t) be the per capita natural mortality rate at time t, and let Q(t) be the total number of infections occurring between the initial observation time t₀ and current time t (i.e., cumulative incidence). The observation interval Δt is no longer constrained to be equal to the mean generation interval t_gen.

We reconstruct S recursively by discretizing Eq (1a):

$$\begin{array}{c}\hfill S(t+\Delta t)\approx S\left(t\right)+B(t+\Delta t)-Z(t+\Delta t)-\mu \left(t\right)S\left(t\right)\Delta t.\end{array}$$ (8)

Eq (8) is the result of applying the forward Euler method for numerical integration to Eq (1a), and replacing the incidence and birth terms with Z(t + Δt) and B(t + Δt), respectively. Eq (8) is identical to Eq (4) of the FC method, except that it includes a natural mortality term.

In order to estimate β(t), we note that, by definition, dQ/dt is the rate at which individuals enter the infected compartment. From Eq (1b), this is

$$\begin{array}{c}\hfill \frac{\text{d}Q}{\text{d}t}=\beta \left(t\right)S\left(t\right)I\left(t\right).\end{array}$$ (9)

If the mean generation interval t_gen is short enough that I and μ are roughly constant between times t and t + t_gen, then dI/dt ≈ 0 in that interval, and using Eq (1b) we can write

$$\begin{array}{c}\hfill \beta \left(t\right)S\left(t\right)I\left(t\right)\approx (\gamma +\mu (t\left)\right)I\left(t\right)\approx (\gamma +\mu (t+t_{\text{gen}}\left)\right)I(t+t_{\text{gen}}).\end{array}$$ (10)

In this case, dQ/dt is also (approximately) the rate at which individuals leave the infected compartment, t_gen time after infection:

$$\begin{array}{c}\hfill \frac{\text{d}Q}{\text{d}t}\approx (\gamma +\mu (t+t_{\text{gen}}\left)\right)I(t+t_{\text{gen}}).\end{array}$$ (11)

Note that Eq (11) is also valid if the generation interval is narrowly distributed around its mean t_gen (even if t_gen is long).

Discretizing Eqs (9) and (11) using forward Euler, we obtain two approximations of Z(t + Δt):

$$\begin{array}{c}\hfill Z(t+\Delta t)=Q(t+\Delta t)-Q\left(t\right)\approx \{\begin{array}{cc}\beta \left(t\right)S\left(t\right)I\left(t\right)\Delta t\hfill & \text{from}\text{Eq}\left(9\right),\hfill \\ (\gamma +\mu (t+t_{\text{gen}}\left)\right)I(t+t_{\text{gen}})\Delta t\hfill & \text{from}\text{Eq}\left(11\right).\hfill \end{array}\end{array}$$ (12)

Rearranging Eq (12) yields an estimate of β(t), given by

$$\begin{array}{c}\hfill \beta \left(t\right)\approx \frac{Z(t+\Delta t)}{S\left(t\right)I\left(t\right)\Delta t},\end{array}$$ (13)

and an estimate of I(t), given by

$$\begin{array}{c}\hfill I\left(t\right)\approx \frac{Z(t+\Delta t-t_{\text{gen}})}{(\gamma +\mu (t\left)\right)\Delta t}.\end{array}$$ (14)

Since data are available only at the observation times t_k (Eq (3)), the value of Z(t + Δt − t_gen) in Eq (14) will be observed only if t_gen is an integer multiple of Δt. In general, t_gen is not divisible by Δt. Therefore, in practice, we replace t_gen in Z(t + Δt − t_gen) with the nearest integer multiple of Δt, denoted here by [t_gen]_Δt:

$$\begin{array}{c}\hfill I\left(t\right)\approx \frac{Z(t+\Delta t-t_{\text{gen}})}{(\gamma +\mu (t\left)\right)\Delta t}.\end{array}$$ (15)

Thus, the S method is defined by Eq (13), coupled with Eqs (8) and (15) for the reconstruction of S and I. The S method requires users to specify (i) input parameters S₀ = S(t₀) and t_gen = γ⁻¹, and (ii) values of Z(t_k), B(t_k), and μ(t_k) from incidence, birth, and natural mortality data, respectively.

The FC method is a special case of the S method, obtained by setting Δt = t_gen and μ(t) ≡ 0.

2.2.3 SI method

DeJonge [25] improved Krylova’s method (Ch. 4 in [24]) by reconstructing I directly from Eq (1b) instead of relying on the approximation in Eq (11). Here, we improve deJonge’s discretization, which employs the forward Euler method, by instead combining forward and backward Euler. One way to do this is to use the trapezoidal method: whereas forward and backward Euler take f ′(t)Δt and f ′(t + Δt)Δt, respectively, to approximate integrals ${\int }_t^{t+\Delta t}{f}^{\prime }\left(\tau \right)\text{d}\tau$, the trapezoidal method takes the average $\frac{1}{2}[f^{\prime }\left(t\right)+f^{\prime }(t+\Delta t)]\Delta t$, which is less prone to error. Our discretization, which we call the “SI method”, is consistently more accurate than deJonge’s and others (see §S9 of S1 Text for a comparison of nine possible algorithms). Numerically integrating Eq (1a) and Eq (1b) using the trapezoidal method, and replacing the incidence and birth terms with Z(t + Δt) and B(t + Δt), respectively, we obtain

$$\begin{array}{c}\hfill S(t+\Delta t)\approx \frac{[1-\frac{1}{2}\mu \left(t\right)\Delta t]S\left(t\right)+B(t+\Delta t)-Z(t+\Delta t)}{1+\frac{1}{2}\mu (t+\Delta t)\Delta t}\end{array}$$ (16)

and

$$\begin{array}{c}\hfill I(t+\Delta t)\approx \frac{[1-\frac{1}{2}(\gamma +\mu \left(t\right))\Delta t]I\left(t\right)+Z(t+\Delta t)}{1+\frac{1}{2}(\gamma +\mu (t+\Delta t\left)\right)\Delta t}.\end{array}$$ (17)

Eq (17) eliminates an important problem with Eq (15) of the S method, which estimates I(t) ≈ 0 if Z(t + Δt − [t_gen]_Δt) = 0, leading to division by zero in Eq (13).

Discretizing Eq (9) using forward and backward Euler, we obtain two approximations of Z(t + Δt):

$$\begin{array}{c}\hfill Z(t+\Delta t)=Q(t+\Delta t)-Q\left(t\right)\approx \{\begin{array}{cc}\beta \left(t\right)S\left(t\right)I\left(t\right)\Delta t\hfill & \text{from}\text{forward}\text{Euler},\hfill \\ \beta (t+\Delta t)S(t+\Delta t)I(t+\Delta t)\Delta t\hfill & \text{from}\text{backward}\text{Euler}.\hfill \end{array}\end{array}$$ (18)

Rearranging Eq (18) yields two estimates of β(t),

$$\begin{array}{c}\hfill \beta \left(t\right)\approx \{\begin{array}{cc}\frac{Z(t+\Delta t)}{S\left(t\right)I\left(t\right)\Delta t}\hfill & \text{from}\text{forward}\text{Euler},\hfill \\ \frac{Z\left(t\right)}{S\left(t\right)I\left(t\right)\Delta t}\hfill & \text{from}\text{backward}\text{Euler},\hfill \end{array}\end{array}$$ (19)

whose average supplies a more accurate estimate (see §S9 of S1 Text), given by

$$\begin{array}{c}\hfill \beta \left(t\right)\approx \frac{Z\left(t\right)+Z(t+\Delta t)}{2S\left(t\right)I\left(t\right)\Delta t}.\end{array}$$ (20)

Thus, the SI method is defined by Eq (20), coupled with Eqs (16) and (17) for the reconstruction of S and I. Compared to the S method, the SI method, in principle, requires one additional input parameter, namely the initial number of infecteds I₀ = I(t₀). In §3.6, we show that, in practice, this additional information is not necessary.

2.2.4 Estimating true incidence from reported incidence

Let C(t) be the number of infections reported during the time interval [t − Δt, t). We estimate true incidence Z from reported incidence C via

$$\begin{array}{c}\hfill Z\left(t\right)\approx \frac{1}{p_{\text{rep}}}C(t+[t_{\text{rep}}{]}_{\Delta t}),\end{array}$$ (21)

where p_rep is the probability that an infection is reported and [t_rep]_Δt is the mean time between infection and reporting, rounded to the nearest integer multiple of the observation interval Δt.

Eq (21) has the limitation that multiplying by $p_{\text{rep}}^{-1}$ does not correct for under-reporting if, by chance, C(t + [t_rep]_Δt) = 0. In this situation, not only is the result Z(t) ≈ 0 incorrect, but we divide by zero in the FC and S methods when we substitute Eqs (5) and (15) in Eqs (7) and (13), respectively. If C(t + [t_rep]_Δt) = C(t + [t_rep]_Δt + Δt) = 0, then the SI method also suffers: Eq (20) gives β(t) ≈ 0. To circumvent these issues, we replace zeros in reported incidence time series by linearly interpolating between nonzero values prior to estimating true incidence using Eq (21). We do not replace leading and trailing zeros.

If what we observe is deaths from disease, rather than infections, then we have the complication that only a fraction of infections end in death. In this situation, we can still use Eq (21) to estimate Z, provided we interpret (i) C as reported disease mortality, (ii) p_rep as the case fatality ratio times the probability that a death from disease is reported, and (iii) t_rep as the mean time between infection and reporting of disease-induced death.

A more sophisticated method of inferring true incidence from reported data is described in [34].

2.2.5 Averaging raw estimates of β(t)

Given fixed time series data and input parameters, the FC, S, and SI methods return estimates of β(t) that are entirely determined (not random). In the absence of additional data observed from the same population, it is difficult to assign confidence to the output.

However, if an estimate $\tilde{\beta }\left(t\right)$ is approximately periodic (with apparent period T) and contains m complete cycles, and if we assume β(t) is truly periodic, then we can view $\tilde{\beta }\left(t\right)$ as containing a sample of m estimates of the true cycle, with some variance, and use its mean as an estimator instead of any one of the m cycles. For such an estimate $\tilde{\beta }\left(t\right)$ defined on the interval [t₀, t₀ + mT), the mean and variance are given by

$$\overline{x}\left(t\right)=\frac{1}{m}\sum _{i=0}^{m-1}\tilde{\beta }(t+iT),t\in [t_0,t_0+T),$$ (22a)

$$s^2\left(t\right)=\frac{1}{m-1}\sum _{i=0}^{m-1}{[\tilde{\beta }(t+iT)-\overline{x}\left(t\right)]}^2,t\in [t_0,t_0+T).$$ (22b)

In §3.3, we apply the S and SI methods to simulated data to estimate an underlying, seasonally forced β(t) (Eq (27)), which has a period of 1 year. We linearly interpolate the raw time series estimate β_k and compute the average 1-year cycle in the interpolant β_int(t) using Eq (22a). Comparing this average to the true 1-year cycle, we are able to assess bias in the two methods.

Note that $\overline{x}\left(t\right)$ and s²(t) can be used to obtain a formal, likelihood-based measure of confidence in estimates $\tilde{\beta }\left(t\right)$ (see §2.3.4 in [35]).

2.2.6 Smoothing raw estimates of β(t)

Process and observation error introduce random fluctuations in reported incidence on top of longer-term (e.g., seasonal) variation. In §3.2, we show that noise in reported incidence is spuriously amplified in β_k, the raw time series estimate of β(t).

To distill temporal patterns of interest from the noise, we fit a smooth loess (short for local regression; see Ch. 8.1 in [36]) curve β_loess(t; q) to the points ${\left\{(t_k,{\beta }_k)\right\}}_{k=0}^n$ and use β_loess(t; q) as our final estimate of β(t). Here, q ∈ {5, …, n + 1} is an integer-valued parameter controlling the degree of smoothing. At times t ∈ [t₀, t_n], the fitted value β_loess(t; q) is obtained as follows:

1. Order the distances d_k = |t_k − t| of the time points t_k (Eq (3)) from t, letting $d_{k_i}$ denote the ith smallest distance (for i = 1, …, n + 1).

2. Fit a quadratic polynomial p₂(t) to the points ${\left\{(t_k,{\beta }_k)\right\}}_{k=0}^n$. This is done by weighted least squares using tricube weights

   $$\begin{array}{c}\hfill w_k=\{\begin{array}{cc}{(1-{\left(\frac{d_k}{d_{k_q}}\right)}^3)}^3\hfill & \text{if}0\le d_k<d_{k_q},\hfill \\ 0\hfill & \text{if}{d}_k\ge d_{k_q}.\hfill \end{array}\end{array}$$ (23)

   Hence only time points t_k nearer to t than the qth nearest time point are weighted in the fit.

3. Define β_loess(t; q) = p₂(t).

Typically, smoother fits are obtained with greater q [36, 37].

The optimal value of q for a given time series β_k, denoted by q_opt, is that which minimizes error in β_loess(t; q) relative to β(t). In §3.4, we estimate β(t) from simulated data, smooth β_k using each value of q on a grid, and use our knowledge of β(t) to determine q_opt. We show that it is possible for smoothing to eliminate much of the error in β_k attributable to process and observation error. Thus, in §2.2.7, we explicitly define the FC, S, and SI methods with loess smoothing as a final step.

In practice, β(t) is not known, so we cannot determine q_opt. In this case, q_opt can be estimated using statistical methods, such as time series cross-validation [38]. However, reasonable results can be obtained much more quickly by inspecting β_loess(t; q) directly and increasing q from 4 until a desirable degree of smoothing is achieved (e.g., until noise on the time scale of weeks is visibly reduced, and patterns on the time scale of months are easier to discern).

2.2.7 Summary

In Boxes 1–3 below, we summarize the three methods derived in §§2.2.1–2.2.6 for estimating time-varying transmission rates β(t) from time series data with observation times t_k (Eq (3)). We use the notation x_k to refer to the value supplied or computed for x(t_k) within the estimation algorithms (x = C, B, μ, Z, S, I, β).

Box 1. FC method (Fine & Clarkson 1982 [6])

$$Z_k\leftarrow \frac{1}{p_{\text{rep}}}{C}_{k+r},\text{where}r=\frac{{[t_{\text{rep}}]}_{\Delta t}}{\Delta t},$$ (24a)

$$S_k\leftarrow S_{k-1}+B_k-Z_k,$$ (24b)

$$I_k\leftarrow Z_k,$$ (24c)

$${\beta }_k\leftarrow \frac{Z_{k+1}}{S_k{I}_k\Delta t},$$ (24d)

where Δt is assumed to be roughly equal to t_gen, and natural mortality is assumed to be negligible. Users must specify:

• a time series ${\left\{(t_k,C_k)\right\}}_{k=0}^n$ of reported incidence or reported disease mortality, with zeros replaced via linear interpolation between nonzero values;

• a time series ${\left\{(t_k,B_k)\right\}}_{k=0}^n$ of births;

• input parameters S₀, t_gen, p_rep, and t_rep.

Box 3. SI method (adapted from deJonge 2014 [25])

$$Z_k\leftarrow \frac{1}{p_{\text{rep}}}{C}_{k+r},\text{where}r=\frac{{[t_{\text{rep}}]}_{\Delta t}}{\Delta t},$$ (26a)

$$S_k\leftarrow \frac{[1-\frac{1}{2}{\mu }_{k-1}\Delta t]S_{k-1}+B_k-Z_k}{1+\frac{1}{2}{\mu }_k\Delta t},$$ (26b)

$$I_k\leftarrow \frac{[1-\frac{1}{2}(\gamma +{\mu }_{k-1})\Delta t]I_{k-1}+Z_k}{1+\frac{1}{2}(\gamma +{\mu }_k)\Delta t},$$ (26c)

$${\beta }_k\leftarrow \frac{Z_k+Z_{k+1}}{2S_k{I}_k\Delta t},$$ (26d)

$${\beta }_{\text{loess}}(t;q)\leftarrow \text{loess}\text{curve}\text{fit}\text{to}\text{time}\text{series}{\left\{(t_k,{\beta }_k)\right\}}_{k=0}^n.$$ (26e)

Users must specify:

• a time series ${\left\{(t_k,C_k)\right\}}_{k=0}^n$ of reported incidence or reported disease mortality, with zeros replaced via linear interpolation between nonzero values;

• a time series ${\left\{(t_k,B_k)\right\}}_{k=0}^n$ of births;

• a time series ${\left\{(t_k,{\mu }_k)\right\}}_{k=0}^n$ of the per capita natural mortality rate;

• input parameters S₀, I₀, t_gen = γ⁻¹, p_rep, and t_rep;

• loess smoothing parameter q.

Box 2. S method (adapted from Krylova 2011 [24])

$$Z_k\leftarrow \frac{1}{p_{\text{rep}}}{C}_{k+r},\text{where}r=\frac{{[t_{\text{rep}}]}_{\Delta t}}{\Delta t},$$ (25a)

$$S_k\leftarrow S_{k-1}+B_k-Z_k-{\mu }_{k-1}{S}_{k-1}\Delta t,$$ (25b)

$$I_k\leftarrow \frac{Z_{k+1-g}}{(\gamma +{\mu }_k)\Delta t},\text{where}g=\frac{{[t_{\text{gen}}]}_{\Delta t}}{\Delta t},$$ (25c)

$${\beta }_k\leftarrow \frac{Z_{k+1}}{S_k{I}_k\Delta t},$$ (25d)

$${\beta }_{\text{loess}}(t;q)\leftarrow \text{loess}\text{curve}\text{fit}\text{to}\text{time}\text{series}{\left\{(t_k,{\beta }_k)\right\}}_{k=0}^n.$$ (25e)

Users must specify:

• a time series ${\left\{(t_k,C_k)\right\}}_{k=0}^n$ of reported incidence or reported disease mortality, with zeros replaced via linear interpolation between nonzero values;

• a time series ${\left\{(t_k,B_k)\right\}}_{k=0}^n$ of births;

• a time series ${\left\{(t_k,{\mu }_k)\right\}}_{k=0}^n$ of the per capita natural mortality rate;

• input parameters S₀, t_gen = γ⁻¹, p_rep, and t_rep;

• loess smoothing parameter q.

In Box 4, we provide instructions for input specification based on our analysis of the methods.

Box 4. Instructions for input specification

• β_k is sensitive to mis-specification of S₀, but not I₀ (cf. §3.6.1). If the user’s estimate of S₀ is uncertain, and if the incidence time series Z_k is roughly periodic, then a more accurate estimate of S₀ may be obtained via peak-to-peak iteration (PTPI; cf. §3.7).

• If S_k is negative for any k, then it is likely that the case reporting probability p_rep was underestimated or that births were systematically under-reported by B_k. This can be resolved by correcting the estimate of p_rep or correcting B_k, then restarting the algorithm. Users should apply close to the minimal correction necessary to prevent negative S_k.

• q must be tuned to the β_k time series. An estimate of q_opt can be obtained using statistical methods, such as time series cross-validation [38]. However, q can be tuned quickly through visual inspection of β_loess(t; q): one can increase q from 5 until a desirable degree of smoothing is achieved (e.g., until noise on the time scale of weeks is visibly reduced, and patterns on the time scale of months are easier to discern).

2.3 Simulating reported incidence data

In order to compare the performance of the FC, S, and SI methods in estimating β(t), we apply the methods to simulated reported incidence data with known underlying β(t). Here, we outline our methods for simulating these data using the SIR model (1).

2.3.1 Seasonal forcing of β(t) with environmental stochasticity

We reproduce seasonal fluctuation in the transmission rate by modeling β(t) in Eq (1) as a sinusoidal forcing function with period equal to one year:

$$\begin{array}{c}\hfill \beta \left(t\right)=⟨\beta ⟩(1+\alpha \text{cos}(\frac{2\pi t}{\text{1}\text{year}}\left)\right).\end{array}$$ (27)

Here, α ∈ [0, 1] is the amplitude of seasonal forcing relative to the mean 〈β〉. We introduce stochastic fluctuation by adding a randomly generated phase shift:

$$\begin{array}{c}\hfill {\beta }_{\varphi }\left(t\right)=⟨\beta ⟩(1+\alpha \text{cos}(\frac{2\pi t}{\text{1}\text{year}}+\varphi (t;ϵ)\left)\right).\end{array}$$ (28)

ϕ is a realization of a continuous-time stochastic process consisting of independent, Normal(0, ϵ²)-distributed random variables. It models environmental stochasticity leading to random noise in the transmission rate. Modeling environmental stochasticity with a random phase shift rather than additive noise conveniently avoids negative β_ϕ(t): β_ϕ(t) oscillates between 〈β〉(1 − α) and 〈β〉(1 + α) regardless of the distribution of the noise. In practice, we take the values of ϕ at times t_k (Eq (3)) and linearly interpolate to obtain values in between. This helps to make simulations of Eqs (1) and (9) with adaptive time steps (cf. §2.3.2) reproducible.

2.3.2 Generating incidence time series with demographic stochasticity

We supplement Eq (1) with Eq (9), so that trajectories of the resulting system record changes in cumulative incidence Q. In this system, we employ the noisy transmission rate β_ϕ(t) (Eq (28)) and constant vital rates ν_c and μ_c. We then either (i) numerically integrate the differential equations to approximate their solution, or (ii) treat the system more realistically as an event-driven, continuous-time Markov process (with event rates specified by terms in the differential equations) and use the adaptive tau-leaping algorithm for stochastic simulation [39, 40]. The latter approach accounts for demographic stochasticity in disease dynamics. We prevent disease fadeout in simulations with demographic stochasticity by setting the rates of infected recovery and death to zero whenever I = 1.

In both methods of simulation, we record the state of the system at times t_k (Eq (3)), choosing initial state

$$\begin{array}{c}\hfill \left(\begin{array}{c}S\left(t_0\right)\\ I\left(t_0\right)\\ R\left(t_0\right)\\ Q\left(t_0\right)\end{array}\right)=\left(\begin{array}{c}{S}_0\\ I_0\\ R_0\\ 0\end{array}\right),\end{array}$$ (29)

where S₀ + I₀ + R₀ = N₀ = N(t₀). Finally, we derive incidence Z from Q via first differences:

$$\begin{array}{c}\hfill Z\left(t\right)=Q\left(t\right)-Q(t-\Delta t).\end{array}$$ (30)

2.3.3 Introducing observation error

Observation error due to under-reporting (p_rep < 1) and reporting delays (t_rep > 0 weeks) creates discrepancies between true incidence Z and reported incidence C. We introduce random observation error to simulated incidence time series with delayed binomial sampling:

$$\begin{array}{c}\hfill C(t+[t_{\text{rep}}{]}_{\Delta t})\sim \text{Binomial}(Z\left(t\right),p_{\text{rep}}).\end{array}$$ (31)

For simulations without observation error, we set p_rep < 1 and t_rep > 0 weeks.

2.3.4 Parametrization

The simulation method outlined in §§2.3.1–2.3.3 is parametrized by

$$\begin{array}{cc}\text{disease}\text{parameters}\hfill & ⟨\beta ⟩,\alpha ,ϵ,t_{\text{gen}};\hfill \\ \text{population}\text{parameters}\hfill & {\widehat{N}}_0=N\left(0\right),S_0=S\left(t_0\right),I_0=I\left(t_0\right),{\nu }_{\text{c}},{\mu }_{\text{c}},;\text{and}\hfill \\ \text{reporting}\text{parameters}\hfill & p_{\text{rep}},t_{\text{rep}},t_0,\Delta t,n.\hfill \end{array}$$

For most simulations, we assign parameters the reference values listed in Table 1. We consider different values when we investigate the sensitivity of β(t) estimates to data-generating parameters (cf. §2.6.1).

We bypass transient dynamics by choosing t₀ = 2000 years and numerically integrating system (1) between 0 years and t₀ in order to obtain a point (S*, I*, R*) very near the attractor. For this step, we exclude environmental noise, defining β(t) as in Eq (27), and take the initial state to be the endemic equilibrium of the unforced system (system (1) with β ≡ 〈β〉 and ν ≡ μ ≡ μ_c):

$$\begin{array}{c}\hfill \left(\begin{array}{c}S\left(0\right)\\ I\left(0\right)\\ R\left(0\right)\end{array}\right)=\left(\begin{array}{c}\widehat{S}\\ \widehat{I}\\ \widehat{R}\end{array}\right)=\left(\begin{array}{c}\frac{{\widehat{N}}_0}{{\mathcal{R}}_0}\\ {\widehat{N}}_0(1-\frac{1}{{\mathcal{R}}_0})\left(\frac{{\mu }_{\text{c}}}{\gamma +{\mu }_{\text{c}}}\right)\\ {\widehat{N}}_0(1-\frac{1}{{\mathcal{R}}_0})\left(\frac{\gamma }{\gamma +{\mu }_{\text{c}}}\right)\end{array}\right).\end{array}$$ (32)

2.4 Creating mock birth and natural mortality time series

In addition to reported incidence data, the FC, S, and SI methods require time series of births and the per capita natural mortality rate. For simplicity, we create mock time series by (i) choosing constant vital rates ${\nu }_{\text{c}}^{\prime }$ and ${\mu }_{\text{c}}^{\prime }$, then (ii) setting $B_k={\nu }_{\text{c}}^{\prime }{\widehat{N}}_0\Delta t$ and ${\mu }_k={\mu }_{\text{c}}^{\prime }$ for all k. Note that ${\nu }_{\text{c}}^{\prime }{\widehat{N}}_0\Delta t$ is the result of integrating the net birth rate in the SIR model (1), given by $\nu \left(t\right){\widehat{N}}_0$, between successive observation times using $\nu \equiv {\nu }_{\text{c}}^{\prime }$.

We specify ${\nu }_{\text{c}}^{\prime }={\nu }_{\text{c}}$ and ${\mu }_{\text{c}}^{\prime }={\mu }_{\text{c}}$, where ν_c and μ_c are the data-generating vital rates (cf. §2.3.4), except when we investigate the sensitivity of β(t) estimates to incorrect vital data (cf. §2.6.2). For example, to model under-reporting of births, we simply set ${\nu }_{\text{c}}^{\prime }<{\nu }_{\text{c}}$.

2.5 Measuring β(t) estimation error

When we simulate reported incidence data, the underlying transmission rate β(t) is defined beforehand via Eq (27) and known for all t. We use this knowledge to quantify the error in estimates of β(t) obtained from the data. Specifically, given an estimate $\tilde{\beta }\left(t\right)$ defined at time points t_k (Eq (3)), we compute the relative root mean square error (RRMSE), defined as

$$\begin{array}{c}\hfill \text{RRMSE}(\beta ,\tilde{\beta })≔\sqrt{\frac{1}{n+1}\sum _{k=0}^n{\left(\frac{\beta \left(t_k\right)-\tilde{\beta }\left(t_k\right)}{\overline{\beta }}\right)}^2},\end{array}$$ (33)

where

$$\begin{array}{c}\hfill \overline{\beta }≔\frac{1}{n+1}\sum _{k=0}^n\beta \left(t_k\right).\end{array}$$ (34)

Note that by “underlying transmission rate” we mean the transmission rate excluding environmental noise. Although we simulate data using the noisy β_ϕ(t), defined in Eq (28), our aim is to reconstruct the noiseless β(t), defined in Eq (27).

2.6 Sensitivity analysis

Error in β(t) estimation from reported incidence data depends on how the data were generated. The number of cases reported over time is influenced by features of the disease (e.g., the natural history of infection), population (e.g., contact patterns), and case reporting (e.g., the frequency and accuracy of reports). In our simulations of reported incidence, there are 14 data-generating parameters (cf. §2.3.4), whose values are summarized in the vector

$$\begin{array}{c}\hfill \mathit{\theta }=(⟨\beta ⟩,\alpha ,ϵ,{\widehat{N}}_0,S_0,I_0,{\nu }_{\text{c}},{\mu }_{\text{c}},t_{\text{gen}},p_{\text{rep}},t_{\text{rep}},t_0,\Delta t,n).\end{array}$$ (35)

Estimation error also depends on how accurately certain data-generating parameters are specified by users of the FC, S, and SI methods. The initial observation time t₀, observation interval Δt, and time series length n are always known exactly. Other parameters (〈β〉, α, ϵ, ${\widehat{N}}_0$, ν_c, and μ_c) influence our simulations of reported incidence, but in practice are not parameters of the FC, S, and SI methods. In practice, users are required to specify only S₀, t_gen, p_rep, t_rep, and (with the SI method) I₀. However, when we test the methods here, we do specify vital rates ν_c and μ_c in order to create mock (constant) birth and natural mortality time series (cf. §2.4). The specified values of these 7 input parameters are summarized in the vector

$$\begin{array}{c}\hfill {\mathit{\theta }}^{\prime }=(S_0^{\prime },I_0^{\prime },{\nu }_{\text{c}}^{\prime },{\mu }_{\text{c}}^{\prime },t_{\text{gen}}^{\prime },p_{\text{rep}}^{\prime },t_{\text{rep}}^{\prime }).\end{array}$$ (36)

First, we investigate the sensitivity of the methods to the data-generating parameter values θ. Then, we examine their sensitivity to error in the user’s specification θ′ of the input parameters. Here, we describe our analysis using the notation $\tilde{\beta }(t;\mathit{\theta },{\mathit{\theta }}^{\prime })$ to refer to transmission rate estimates constructed with user input θ′, from data generated by parameter values θ.

2.6.1 Sensitivity to data-generating parameters

In §3.5, we consider the ideal situation in which the input θ′ corresponds exactly to the data-generating θ. In this case, how sensitive is error in $\tilde{\beta }(t;\mathit{\theta },{\mathit{\theta }}^{\prime })$ to θ? For example, is β(t) estimated more accurately for diseases with longer mean generation interval t_gen, etc.? To answer these questions, we perform the following steps on a grid of data-generating parameter values θ:

1. Simulate 1000 reported incidence time series using θ.

2. Create corresponding mock (constant) birth and natural mortality time series (cf. §2.4), specifying ${\nu }_{\text{c}}^{\prime }={\nu }_{\text{c}}$ and ${\mu }_{\text{c}}^{\prime }={\mu }_{\text{c}}$ in the input θ′.

3. Estimate β(t) from the simulated data, specifying $S_0^{\prime }=S_0$, $I_0^{\prime }=I_0$, $t_{\text{gen}}^{\prime }=t_{\text{gen}}$, $p_{\text{rep}}^{\prime }=p_{\text{rep}}$, and $t_{\text{rep}}^{\prime }=t_{\text{rep}}$ in the input θ′.

4. Compute the median RRMSE in the estimates $\tilde{\beta }(t_k;\mathit{\theta },{\mathit{\theta }}^{\prime })$ (1000 estimates corresponding to 1000 simulations).

We repeat this analysis 6 times, corresponding to 2 methods of β(t) estimation (S or SI) and 3 methods of data simulation:

• without demographic stochasticity and without observation error (fixing p_rep = 1, t_rep = 0 weeks),

• with demographic stochasticity but without observation error (fixing p_rep = 1, t_rep = 0 weeks), or

• with demographic stochasticity and with observation error (fixing p_rep = 0.25 unless sensitivity to p_rep is considered, t_rep = 2 weeks).

Environmental stochasticity (ϵ = 0.5) is included in all simulations.

2.6.2 Sensitivity to mis-specification of input parameters

In §3.6, we fix the data-generating θ and consider the more realistic situation in which components of the input θ′ differ from the corresponding components of θ by a potentially large factor. In this case, how sensitive is error in $\tilde{\beta }(t;\mathit{\theta },{\mathit{\theta }}^{\prime })$ to error in θ′? For example, how important is having an accurate estimate of t_gen, etc.? To answer these questions, we perform the following steps:

1. Simulate 1000 reported incidence time series using fixed data-generating parameter values θ. (We assign the reference values listed in Table 1.)

2. For each point on a grid of input parameter values θ′:

   1. Create mock (constant) birth and natural mortality time series, taking ${\nu }_{\text{c}}^{\prime }$ and ${\mu }_{\text{c}}^{\prime }$ from the input θ′.

   2. Estimate β(t) from the simulated data, taking $S_0^{\prime }$, $I_0^{\prime }$, $t_{\text{gen}}^{\prime }$, $p_{\text{rep}}^{\prime }$, and $t_{\text{rep}}^{\prime }$ from the input θ′.

   3. Compute the median RRMSE in the estimates $\tilde{\beta }(t_k;\mathit{\theta },{\mathit{\theta }}^{\prime })$ (1000 estimates corresponding to 1000 simulations).

We repeat this analysis 6 times, as outlined at the end of §2.6.1.

2.7 Asymptotic analysis

Here, we examine analytically the propagation of input error to the output of the SI method. (Similar expressions for propagated errors are obtained by analyzing the S method.) Our analysis here supports numerical results presented in §3.6 concerning the sensitivity of β(t) estimation error to mis-specification of input parameters.

2.7.1 Explicit solutions of the (S_k, I_k) difference equations

The SI method uses Eq (26a) to Eq (26c) to recursively reconstruct S(t) and I(t) from time series of reported incidence, births, and natural mortality. After substitution of Eq (26a), Eq (26b) and Eq (26c) can be written as

$$\begin{array}{cc}\hfill S_{k+1}& =\frac{1-\frac{1}{2}{\mu }_k\Delta t}{1+\frac{1}{2}{\mu }_{k+1}\Delta t}{S}_k+\frac{B_{k+1}-\frac{1}{p_{\text{rep}}}{C}_{k+1+r}}{1+\frac{1}{2}{\mu }_{k+1}\Delta t},k=0,1,\dots ,\end{array}$$ (37a)

$$I_{k+1}=\frac{1-\frac{1}{2}(\gamma +{\mu }_k)\Delta t}{1+\frac{1}{2}(\gamma +{\mu }_{k+1})\Delta t}{I}_k+\frac{\frac{1}{p_{\text{rep}}}{C}_{k+1+r}}{1+\frac{1}{2}(\gamma +{\mu }_{k+1})\Delta t},k=0,1,\dots ,$$ (37b)

where r = [t_rep]_Δt/Δt is the mean case reporting delay in units of the observation interval, rounded to the nearest integer. Eq (37) are linear, first order difference equations, whose explicit solutions are obtained using standard algebraic techniques (see Eq 1.2.4 in [41]) and given by

$$\begin{array}{cc}\hfill S_k& =S_0\prod _{j=0}^{k-1}\frac{1-\frac{1}{2}{\mu }_j\Delta t}{1+\frac{1}{2}{\mu }_{j+1}\Delta t}+\sum _{i=0}^{k-1}(B_{i+1}-\frac{1}{p_{\text{rep}}}{C}_{i+1+r})\prod _{j=i+1}^{k-1}\frac{1-\frac{1}{2}{\mu }_j\Delta t}{1+\frac{1}{2}{\mu }_{j+1}\Delta t},k=0,1,\dots ,\end{array}$$ (38a)

$$\begin{array}{cc}\hfill I_k& =I_0\prod _{j=0}^{k-1}\frac{1-\frac{1}{2}(\gamma +{\mu }_j)\Delta t}{1+\frac{1}{2}(\gamma +{\mu }_{j+1})\Delta t}+\sum _{i=0}^{k-1}\frac{1}{p_{\text{rep}}}{C}_{i+1+r}\prod _{j=i+1}^{k-1}\frac{1-\frac{1}{2}(\gamma +{\mu }_j)\Delta t}{1+\frac{1}{2}(\gamma +{\mu }_{j+1})\Delta t},k=0,1,\dots ,\end{array}$$ (38b)

with the conventions ${\displaystyle \sum _{i=b}^a}{x}_i=0$ and ${\displaystyle \prod _{i=b}^a}{x}_i=1$ if a < b. As we show in §2.7.2, explicit solutions of Eq (37) facilitate asymptotic analysis.

2.7.2 Propagation of input error to (S_k, I_k)

We consider the special case in which the vital rates are constant and set $B_k={\nu }_{\text{c}}{\widehat{N}}_0\Delta t$ and μ_k = μ_c for all k (cf. §2.4). Then Eq (38) simplify to

$$\begin{array}{c}\hfill \begin{array}{cc}& S_k(S_0,{\nu }_{\text{c}},{\mu }_{\text{c}},p_{\text{rep}})\hfill \\ & =S_0{\left(\frac{1-\frac{1}{2}{\mu }_{\text{c}}\Delta t}{1+\frac{1}{2}{\mu }_{\text{c}}\Delta t}\right)}^k+\sum _{i=0}^{k-1}\frac{{\nu }_{\text{c}}{\widehat{N}}_0\Delta t-\frac{1}{p_{\text{rep}}}{C}_{i+1+r}}{1+\frac{1}{2}{\mu }_{\text{c}}\Delta t}{\left(\frac{1-\frac{1}{2}{\mu }_{\text{c}}\Delta t}{1+\frac{1}{2}{\mu }_{\text{c}}\Delta t}\right)}^{k-1-i}\hfill \end{array}\end{array}$$ (39a)

$$\begin{array}{c}\hfill \begin{array}{cc}& I_k(I_0,{\mu }_{\text{c}},t_{\text{gen}},p_{\text{rep}})\hfill \\ & =I_0{\left(\frac{1-\frac{1}{2}(\gamma +{\mu }_{\text{c}})\Delta t}{1+\frac{1}{2}(\gamma +{\mu }_{\text{c}})\Delta t}\right)}^k+\sum _{i=0}^{k-1}\frac{\frac{1}{p_{\text{rep}}}{C}_{i+1+r}}{1+\frac{1}{2}(\gamma +{\mu }_{\text{c}})\Delta t}{\left(\frac{1-\frac{1}{2}(\gamma +{\mu }_{\text{c}})\Delta t}{1+\frac{1}{2}(\gamma +{\mu }_{\text{c}})\Delta t}\right)}^{k-1-i}\hfill \end{array}\end{array}$$ (39b)

where we have made explicit the dependence of S_k and I_k on input parameters S₀, I₀, ν_c, μ_c, t_gen = γ⁻¹, and p_rep. Using Eq (39), we can derive exact expressions for the error propagated to S_k and I_k in the SI method as a result of assigning an incorrect value to an input parameter.

If the initial number of susceptibles is truly S₀, but we specify $S_0^{\prime }=\omega S_0$, where ω > 0, then the error propagated to S_k is

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \text{Err}(S_k,S_0\leftarrow \omega S_0)& =S_k(\omega S_0,{\nu }_{\text{c}},{\mu }_{\text{c}},p_{\text{rep}})-S_k(S_0,{\nu }_{\text{c}},{\mu }_{\text{c}},p_{\text{rep}})\hfill \\ & =(\omega -1)S_0{\left(\frac{1-\frac{1}{2}{\mu }_{\text{c}}\Delta t}{1+\frac{1}{2}{\mu }_{\text{c}}\Delta t}\right)}^k\hfill \\ & =(\omega -1)S_0{\left(\frac{1-\frac{\Delta t}{2t_{\text{life}}}}{1+\frac{\Delta t}{2t_{\text{life}}}}\right)}^k\stackrel{k\to \infty }{\to }0,\hfill \end{array}\end{array}$$ (40)

where $t_{\text{life}}={\mu }_{\text{c}}^{-1}$ is the life expectancy in the population. Similarly, specifying $I_0^{\prime }=\omega I_0$ for I₀ yields an error

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \text{Err}(I_k,I_0\leftarrow \omega I_0)& =I_k(\omega I_0,{\mu }_{\text{c}},t_{\text{gen}},p_{\text{rep}})-I_k(I_0,{\mu }_{\text{c}},t_{\text{gen}},p_{\text{rep}})\hfill \\ & =(\omega -1)I_0{\left(\frac{1-\frac{1}{2}(\gamma +{\mu }_{\text{c}})\Delta t}{1+\frac{1}{2}(\gamma +{\mu }_{\text{c}})\Delta t}\right)}^k\hfill \\ & =(\omega -1)I_0{\left(\frac{1-\frac{\Delta t}{2t_{\text{inf}}}}{1+\frac{\Delta t}{2t_{\text{inf}}}}\right)}^k\stackrel{k\to \infty }{\to }0\hfill \end{array}\end{array}$$ (41)

in I_k, where t_inf = (γ + μ_c)⁻¹ is the mean time between infection and removal from the infected compartment, accounting for the possibility of natural death during infection. Eqs (40) and (41) show that the errors propagated to S_k and I_k vanish as k → ∞; we exploit this fact to improve susceptible reconstruction (cf. §2.8).

Mis-specifying ν_c by assigning a value ${\nu }_{\text{c}}^{\prime }=\omega {\nu }_{\text{c}}$ creates an error in S_k that increases in magnitude over time and converges to a limit:

$$\begin{array}{c}\hfill \begin{array}{cc}& \text{Err}(S_k,{\nu }_{\text{c}}\leftarrow \omega {\nu }_{\text{c}})\hfill \\ & =S_k(S_0,\omega {\nu }_{\text{c}},{\mu }_{\text{c}},p_{\text{rep}})-S_k(S_0,{\nu }_{\text{c}},{\mu }_{\text{c}},p_{\text{rep}})\hfill \\ & =\sum _{i=0}^{k-1}\frac{(\omega -1){\nu }_{\text{c}}{\widehat{N}}_0\Delta t}{1+\frac{1}{2}{\mu }_{\text{c}}\Delta t}{\left(\frac{1-\frac{1}{2}{\mu }_{\text{c}}\Delta t}{1+\frac{1}{2}{\mu }_{\text{c}}\Delta t}\right)}^{k-1-i}\hfill \\ & =\frac{(\omega -1){\nu }_{\text{c}}{\widehat{N}}_0\Delta t}{1+\frac{1}{2}{\mu }_{\text{c}}\Delta t}\sum _{i=0}^{k-1}{\left(\frac{1-\frac{1}{2}{\mu }_{\text{c}}\Delta t}{1+\frac{1}{2}{\mu }_{\text{c}}\Delta t}\right)}^i\hfill \\ & =\frac{(\omega -1){\nu }_{\text{c}}{\widehat{N}}_0}{{\mu }_{\text{c}}}[1-{\left(\frac{1-\frac{1}{2}{\mu }_{\text{c}}\Delta t}{1+\frac{1}{2}{\mu }_{\text{c}}\Delta t}\right)}^k]\stackrel{k\to \infty }{\to }(\omega -1){\nu }_{\text{c}}{\widehat{N}}_0{t}_{\text{life}}.\hfill \end{array}\end{array}$$ (42)

Unlike Eq (42), the exact expression for Err(S_k, μ_c ← ωμ_c) is not readily simplified and is difficult to interpret:

$$\begin{array}{c}\hfill \begin{array}{cc}& \text{Err}(S_k,{\mu }_{\text{c}}\leftarrow \omega {\mu }_{\text{c}})\hfill \\ & =S_k(S_0,{\nu }_{\text{c}},\omega {\mu }_{\text{c}},p_{\text{rep}})-S_k(S_0,{\nu }_{\text{c}},{\mu }_{\text{c}},p_{\text{rep}})\hfill \\ & =S_0{\left(\frac{1-\frac{1}{2}\omega {\mu }_{\text{c}}\Delta t}{1+\frac{1}{2}\omega {\mu }_{\text{c}}\Delta t}\right)}^k+\sum _{i=0}^{k-1}\frac{{\nu }_{\text{c}}{\widehat{N}}_0\Delta t-\frac{1}{p_{\text{rep}}}{C}_{i+1+r}}{1+\frac{1}{2}\omega {\mu }_c\Delta t}{\left(\frac{1-\frac{1}{2}\omega {\mu }_{\text{c}}\Delta t}{1+\frac{1}{2}\omega {\mu }_{\text{c}}\Delta t}\right)}^{k-1-i}\hfill \\ & -S_0{\left(\frac{1-\frac{1}{2}\omega {\mu }_{\text{c}}\Delta t}{1+\frac{1}{2}\omega {\mu }_{\text{c}}\Delta t}\right)}^k-\sum _{i=0}^{k-1}\frac{{\nu }_{\text{c}}{\widehat{N}}_0\Delta t-\frac{1}{p_{\text{rep}}}{C}_{i+1+r}}{1+\frac{1}{2}{\mu }_c\Delta t}{\left(\frac{1-\frac{1}{2}{\mu }_{\text{c}}\Delta t}{1+\frac{1}{2}{\mu }_{\text{c}}\Delta t}\right)}^{k-1-i}.\hfill \end{array}\end{array}$$ (43)

However, if C_k has a well-defined long-term average 〈C〉 (this will be true if, for instance, C_k is periodic), then Err(S_k, μ_c ← ωμ_c) has a well-defined long-term average 〈Err(S_k, μ_c ← ωμ_c)〉 with a simple form. Replacing C_i+1+r in Eq (43) with 〈C〉, simplifying the resulting expression, then taking the limit as k → ∞, we obtain

$$\begin{array}{c}\hfill \begin{array}{cc}& ⟨\text{Err}(S_k,{\mu }_{\text{c}}\leftarrow \omega {\mu }_{\text{c}})⟩\hfill \\ & =\underset{k\to \infty }{\text{lim}}\{S_0{\left(\frac{1-\frac{1}{2}\omega {\mu }_{\text{c}}\Delta t}{1+\frac{1}{2}\omega {\mu }_{\text{c}}\Delta t}\right)}^k+\frac{{\nu }_{\text{c}}{\widehat{N}}_0\Delta t-\frac{1}{p_{\text{rep}}}⟨C⟩}{\omega {\mu }_{\text{c}}\Delta t}[1-{\left(\frac{1-\frac{1}{2}\omega {\mu }_{\text{c}}\Delta t}{1+\frac{1}{2}\omega {\mu }_{\text{c}}\Delta t}\right)}^k]\hfill \\ & -S_0{\left(\frac{1-\frac{1}{2}{\mu }_{\text{c}}\Delta t}{1+\frac{1}{2}{\mu }_{\text{c}}\Delta t}\right)}^k-\frac{{\nu }_{\text{c}}{\widehat{N}}_0\Delta t-\frac{1}{p_{\text{rep}}}⟨C⟩}{{\mu }_{\text{c}}\Delta t}[1-{\left(\frac{1-\frac{1}{2}{\mu }_{\text{c}}\Delta t}{1+\frac{1}{2}{\mu }_{\text{c}}\Delta t}\right)}^k]\}\hfill \\ & =(\frac{1}{\omega }-1)({\nu }_{\text{c}}{\widehat{N}}_0\Delta t-\frac{1}{p_{\text{rep}}}⟨C⟩)\frac{t_{\text{life}}}{\Delta t},\hfill \end{array}\end{array}$$ (44)

We can similarly show the following, still assuming that 〈C〉 is well-defined:

$$⟨\text{Err}(I_k,{\mu }_{\text{c}}\leftarrow \omega {\mu }_{\text{c}})⟩=\frac{⟨C⟩(t_{\text{inf}}^{\prime }-t_{\text{inf}})}{p_{\text{rep}}\Delta t},t_{\text{inf}}^{\prime }={(\gamma +\omega {\mu }_{\text{c}})}^{-1},$$ (45)

$$⟨\text{Err}(I_k,t_{\text{gen}}\leftarrow \omega t_{\text{gen}})⟩=\frac{⟨C⟩(t_{\text{inf}}^{\prime }-t_{\text{inf}})}{p_{\text{rep}}\Delta t},t_{\text{inf}}^{\prime }={(\frac{1}{\omega }\gamma +{\mu }_{\text{c}})}^{-1},$$ (46)

$$⟨\text{Err}(S_k,p_{\text{rep}}\leftarrow \omega p_{\text{rep}})⟩=(1-\frac{1}{\omega })\frac{⟨C⟩t_{\text{life}}}{p_{\text{rep}}\Delta t},$$ (47)

$$⟨\text{Err}(I_k,p_{\text{rep}}\leftarrow \omega p_{\text{rep}})⟩=(\frac{1}{\omega }-1)\frac{⟨C⟩t_{\text{inf}}}{p_{\text{rep}}\Delta t}.$$ (48)

Here, $t_{\text{inf}}^{\prime }$ is the (incorrect) mean time spent infected that results when ωμ_c is incorrectly specified for μ_c (Eq (45)) or ωt_gen is incorrectly specified for t_gen (Eq (46)).

2.7.3 Propagation of error in (S_k, I_k) to β_k

Let β_k(Z_k, Z_k+1, S_k, I_k) be the raw SI method estimate of β(t_k), given by the right hand side of Eq (26d). Suppose that, due to propagated error (cf. §2.7.2), the arguments are incorrect by a factor, so that

$$\begin{array}{c}\hfill Z_k={\omega }_{Z}Z\left(t_k\right),Z_{k+1}={\omega }_{Z}Z\left(t_{k+1}\right),S_k={\omega }_{S}S\left(t_k\right),I_k={\omega }_{I}I\left(t_k\right),\end{array}$$ (49)

where ω_Z, ω_S, ω_I > 0. Then the computed β_k will have relative error

$$\begin{array}{c}\hfill \frac{{\beta }_k(Z_k,Z_{k+1},S_k,I_k)-{\beta }_k(Z\left(t_k\right),Z\left(t_{k+1}\right),S\left(t_k\right),I\left(t_k\right))}{{\beta }_k(Z\left(t_k\right),Z\left(t_{k+1}\right),S\left(t_k\right),I\left(t_k\right))}=\frac{{\omega }_Z}{{\omega }_S{\omega }_I}-1.\end{array}$$ (50)

Hence severe underestimation of S_k or I_k (ω_S ≪ 1 or ω_I ≪ 1) causes the relative error in β_k to blow up.

2.8 Estimating S₀ via peak-to-peak iteration

Reconstruction of susceptibles S(t) is a necessary step in the reconstruction of β(t) using the FC, S, and SI methods. In §3.6, we show that susceptible reconstruction requires accurate specification of the initial number of susceptibles S₀ = S(t₀). However, reliable estimates of S₀ have, to this point, been difficult to obtain in practice.

We propose a technique for iteratively improving estimates of S₀, requiring only incidence, birth, and natural mortality data at times t_k (Eq (3)). Crucially, our technique, which we call “peak-to-peak iteration” (PTPI), enables accurate susceptible reconstruction without direct observation of the susceptible population size at the initial time.

Our approach is motivated by application of the SI method to simulated data. When we incorrectly guessed the value of S₀ and attempted to reconstruct S(t) via Eq (26b), the absolute error in the reconstruction ${\left\{(t_k,S_k)\right\}}_{k=0}^n$ decreased monotonically over time (k). (Eq (40) shows that the error propagated from S₀ to S_k vanishes as k → ∞.) Consequently, if the underlying dynamics are at least approximately periodic, and if t₀ and t_n occur at the same phase of the cycle, then S_n is actually a better estimate of S₀ than our initial guess. In this situation, instead of reconstructing β(t) directly, we can use S_n as an updated estimate of S₀, and reconstruct S(t) more accurately. This procedure can be repeated any number of times, and, with simulated data, we observe rapid convergence to an accurate estimate of S₀ (cf. §3.7).

The key point is that the reconstructed final state can be used as an improved estimate of the initial state only if the initial and final states occur at the same phase of the cycle. This will not be true unless the observation period (the time between the first and last observations in time series data) is an integer multiple of the period of the underlying dynamics. We can ensure this by choosing appropriate times at which to start and stop S(t) reconstruction. In noisy periodic data, the points in a cycle that are easiest to identify robustly are the peaks. Consequently, we ignore observations (i) prior to the time t_a of the first peak in the incidence time series and (ii) after the time t_b of the last peak that occurs near an integer multiple of the apparent period after the first peak. For the truncated time series, the iterations converge to an accurate estimate of S(t_a) starting from an initial guess, and we recover the corresponding accurate estimate of S₀ by solving Eq (26b) backwards in time, from t_a to t₀:

$$\begin{array}{c}\hfill S_{k-1}\approx \frac{[1+\frac{1}{2}{\mu }_k\Delta t]S_k-B_k+Z_k}{1-\frac{1}{2}{\mu }_{k-1}\Delta t}.\end{array}$$ (51)

The complete PTPI algorithm, which consists of finding t_a and t_b (truncation step) and estimating S₀ (iteration step), is outlined in Boxes 5 and 6 below. In §3.7, we assess the performance of PTPI by applying the technique to simulated data with known underlying S₀, starting from an incorrect initial estimate of S₀.

Box 5. Peak-to-peak iteration: Truncation step

Goal: Given a roughly periodic time series ${\left\{(t_k,Z_k)\right\}}_{k=0}^n$ of incidence, we want to find the time t_a of the first peak and the time t_b of the last peak occurring at the same phase of the cycle. These times are necessary for the iteration step (Box 6).

Algorithm:

1. Smooth the raw incidence time series Z_k by applying a (2ℓ₁ + 1)-point central moving average, computed via

   $$\begin{array}{c}\hfill {\overline{Z}}_k=\frac{1}{2{\ell }_1+1}\sum _{i=-{\ell }_1}^{{\ell }_1}{Z}_{k+i},k={\ell }_1,\dots ,n-{\ell }_1.\end{array}$$ (52)

   Choose minimal ℓ₁ large enough to remove spurious peaks in Z_k caused by noise, while retaining true peaks.

2. Identify the period T of the smoothed incidence time series ${\left\{(t_k,{\overline{Z}}_k)\right\}}_{k={\ell }_1}^{n-{\ell }_1}$ from its power spectrum, and calculate the number of embedded cycles, given by $m=\lfloor \frac{t_{n-{\ell }_1}-t_{{\ell }_1}}{T}\rfloor$.

3. Construct the set $\mathcal{I}$ indexing peaks in ${\left\{(t_k,{\overline{Z}}_k)\right\}}_{k={\ell }_1}^{n-{\ell }_1}$:

   $$\begin{array}{c}\hfill \mathcal{I}=\{k\in \{{\ell }_1+{\ell }_2,\dots ,n-{\ell }_1-{\ell }_2\}:{\overline{Z}}_k>{\overline{Z}}_{k\pm i}\text{for}\text{all}i=1,\dots ,{\ell }_2\}.\end{array}$$ (53)

   Choose minimal ℓ₂ large enough to ensure that $\mathcal{I}$ indexes true peaks in ${\overline{Z}}_k$, but not spurious peaks caused by noise (any that remain after smoothing).

4. Define $\mathcal{T}=\{t_k:k\in \mathcal{I}\}$, the set of times of peaks in ${\overline{Z}}_k$, and record the time of the first peak, given by $t_a=min\left(\mathcal{T}\right)$.

5. For i = 0, …, m, define τ_i = t_a+ iT and find the element of $\mathcal{T}$ nearest τ_i, namely $\underset{\tau \in \mathcal{T}}{\text{arg}\text{min}}|{\tau }_i-\tau |$. The resulting subset ${\mathcal{T}}_{\text{phase}}$ should contain successive time points that are roughly one period apart, i.e., the corresponding peaks in ${\overline{Z}}_k$ should occur at the same phase of the cycle.

6. Record the time of the last such peak, given by $t_b=\text{max}\left({\mathcal{T}}_{\text{phase}}\right)$.

Box 6. Peak-to-peak iteration: Iteration step

Goal: We want to produce an accurate estimate of the initial number of susceptibles S₀ = S(t₀), given

• a roughly periodic time series ${\left\{(t_k,Z_k)\right\}}_{k=0}^n$ of incidence,

• a time series ${\left\{(t_k,B_k)\right\}}_{k=0}^n$ of births,

• a time series ${\left\{(t_k,{\mu }_k)\right\}}_{k=0}^n$ of the per capita natural mortality rate,

• times t_a and t_b as defined in the truncation step (Box 5), and

• an initial estimate of S₀.

Algorithm:

1. Define an initial estimate of S(t_a). (We use the initial estimate of S₀.)

2. Reconstruct S(t) between times t_a and t_b using Eq (26b), starting with the current estimate of S(t_a).

3. Update the estimate of S(t_a) with the estimate of S(t_b) obtained in (ii).

4. Repeat (ii) and (iii) until the sequence of estimates of S(t_a) converges (to within a desirable tolerance).

5. Reconstruct S(t) between times t₀ and t_a using Eq (51), starting with the final estimate of S(t_a) obtained in (iv). The reconstruction is performed backwards in time, from t_a to t₀.

6. Record the estimate of S₀ = S(t₀) computed in (v). This value can be passed back to Eq (26b), allowing for reconstruction of S(t) between times t₀ and t_n, as usual.

3 Results

In §3.1, we compare the performance of the FC, S, and SI methods in estimating β(t) from an idealized reported incidence time series. In §3.2, we show how process and observation error create spurious noise in estimates of β(t). In §§3.3 and 3.4, we examine averaging and smoothing as ways to distill temporal patterns of interest from noisy estimates of β(t). In §§3.5 and 3.6, we summarize our systematic analysis of the sensitivity of β(t) estimation error to data-generating parameters and to mis-specification of input parameters by the user. In §3.7, addressing apparent sensitivity to mis-specification of the initial number of susceptibles S₀, we assess the performance of PTPI as a method of estimating S₀. Finally, in §3.8, we report the run times of the S and SI methods and PTPI.

The results reported here are entirely reproducible using the annotated R code available in S1 File.

3.1 Example of β(t) estimation using the FC, S, and SI methods

We applied the FC, S, and SI methods without input error to estimate S(t) and β(t) from an idealized reported incidence time series, simulated without process or observation error. The time series estimates S_k and β_k are shown in Fig 1. The S and SI methods estimated S(t) and β(t) accurately at every time point, whereas the FC method captured seasonality but failed otherwise. In the FC method, S_k neglects natural mortality (Eq (24b)), so it increases without bound while β_k decays to zero due to division by S_k (Eq (24d)).

Fig 1. Example of S(t) and β(t) estimation using the FC, S, and SI methods.

Plotted are the susceptible population size S(t) and seasonally forced transmission rate β(t) (Eq (27)) underlying 20 years of weekly reported incidence, together with time series estimates S_k and β_k obtained from the data by the FC [blue], S [green], and SI [red] methods. The reported incidence time series (Δt = 1 week, n = ⌊20 × 365/7⌋ = 1042) was simulated without process or observation error (ϵ = 0, p_rep = 1), using reference values (Table 1) for all other data-generating parameters. The three estimation methods were applied without input error, i.e., all input parameters were assigned their true (data-generating) values. [Panel A] S(t) scaled by 1/N₀, describing the number of susceptibles as a proportion of the initial population size. Grey lines show that the absolute error in the FC method estimate of S(t) increases linearly as μ_c 〈S〉t, where μ_c is the constant per capita natural mortality rate and 〈S〉 is the continuous-time average of S(t). [Panel B] β(t) scaled by 1/〈β〉, describing the transmission rate relative to its mean. RRMSE (Eq (33)) in the β_k time series generated by the (FC, S, SI) method is roughly (0.3355, 0.0240, 0.0021).

Fig 1A confirms that the absolute error in the FC method estimate of S(t) increases linearly as μ_c 〈S〉t, where μ_c is the constant per capita natural mortality rate and 〈S〉 is the continuous-time average of S(t). In practice, the FC method fails whenever natural mortality in the underlying population is non-negligible. Since the S and SI methods address this limitation at effectively no computational cost, we do not present further analysis of the FC method.

In Fig 1B, the SI method estimate of β(t) was very accurate (RRMSE ≈ 0.2%), whereas the S method estimate peaked too early and too high (RRMSE ≈ 2.4%).

3.2 Effects of process and observation error

We applied the S and SI methods without input error to four reported incidence time series C_k, simulated using the same parameter values but with different levels of process and observation noise. The first simulation was purely deterministic, while the remaining three included (i) environmental stochasticity [ES], (ii) ES and demographic stochasticity [ES+DS], or (iii) ES, DS, and observation error [ES+DS+OE]. Fig 2 shows the resulting estimates Z_k, I_k, and β_k of true incidence Z(t), prevalence I(t), and the seasonally forced transmission rate β(t).

Fig 2. Effects of process and observation error on the S and SI methods.

Plotted are the estimates [Row A] Z_k, [Row B] I_k, and [Row C] β_k of true incidence Z(t), prevalence I(t), and the seasonally forced transmission rate β(t) (Eq (27)) obtained by applying the [Left] S and [Right] SI methods without input error to each of four simulated reported incidence time series (indicated by the legend; Δt = 1 week, n = ⌊3 × 365/7⌋ = 156). The first simulation was purely deterministic [dark grey] (ϵ = 0, p_rep = 1), while the remaining three accounted for (i) environmental stochasticity [ES, light grey] (ϵ = 0.5, p_rep = 1), (ii) ES and demographic stochasticity [ES+DS, blue] (ϵ = 0.5, p_rep = 1), or (iii) ES, DS, and observation error [ES+DS+OE, red] (ϵ = 0.5, p_rep = 0.25). Reference values (Table 1) were assigned to all other data-generating parameters, in all four simulations. The left and right panels in Row A are identical, because the S and SI methods compute Z_k identically (compare Eqs (25a) and (26a)). RRMSE in the β_k time series is (0.0239, 0.0375, 0.1126, 0.1432) with the S method and (0.0021, 0.0153, 0.0494, 0.0591) with the SI method (order follows the legend). Note that the underlying β(t) was the same in all simulations; it is not plotted in Row C, but is close to perfectly represented by the dark grey curve in the right panel (RRMSE ≈ 0.2%). Due to process error, the underlying Z(t) and I(t) (also not shown) varied between the deterministic, ES, and ES+DS simulations.

Noise of any type introduces random fluctuations in C_k on top of longer-term (e.g., seasonal) variation. Noise in C_k is propagated to Z_k (Fig 2A) and I_k (Fig 2B), because (i) in both the S and SI methods, we scale C_k+r by a constant factor of $p_{\text{rep}}^{-1}\ge 1$ to compute Z_k (Eqs (25a) and (26a)); (ii) in the S method, we scale C_k+1−g+r by a constant factor of [p_rep(γ + μ_k)Δt]⁻¹ to compute I_k (Eq (25c) after substitution of Eq (25a)); and (iii) in the SI method, I_k contains a weighted sum of C_i terms (Eq (38b)).

Noise in Z_k and I_k is amplified in β_k (Fig 2C), distorting the correct temporal pattern, for the following reason. When Z and I are close to zero, small absolute changes in either yield large relative changes in the ratio Z/I and in turn β_k, which contains a factor of Z_k+1/I_k in the S method (Eq (25d)) and (Z_k + Z_k+1)/(2I_k) in the SI method (Eq (26d)). Hence low amplitude noise in Z_k and I_k appears as spurious, higher amplitude noise in β_k. This is an important issue in practice, because the incidence of endemic diseases is typically very small relative to the population size, and periodic fluctuations bringing incidence even closer to zero are common for many diseases [4, 14, 42].

Fig 2 shows that the SI method is much better than the S method at resisting noise propagation. One reason is the effective smoothing of incidence in the SI method, which replaces Z_k+1 with (Z_k + Z_k+1)/2 in the computation of β_k (compare Eqs (25d) and (26d)). We expose a second reason in §3.2.1 below by comparing the variance in I_k induced by observation error, between the two methods. (We expect similar results for process error.)

3.2.1 Propagation of noise from C_k to I_k

Consider the S and SI method estimates of prevalence I(t_k),

$$I_k^{\text{[S]}}=\frac{C_{k+1-g+r}}{p_{\text{rep}}(\gamma +{\mu }_{\text{c}})\Delta t},$$ (54a)

$$I_k^{\text{[SI]}}=I_0{\left(\frac{1-\frac{1}{2}(\gamma +{\mu }_{\text{c}})\Delta t}{1+\frac{1}{2}(\gamma +{\mu }_{\text{c}})\Delta t}\right)}^k+\sum _{i=0}^{k-1}\frac{C_{i+1+r}}{p_{\text{rep}}[1+\frac{1}{2}(\gamma +{\mu }_{\text{c}})\Delta t]}{\left(\frac{1-\frac{1}{2}(\gamma +{\mu }_{\text{c}})\Delta t}{1+\frac{1}{2}(\gamma +{\mu }_{\text{c}})\Delta t}\right)}^{k-1-i}.$$ (54b)

Here, g = [t_gen]_Δt/Δt and r = [t_rep]_Δt/Δt are the mean generation interval and case reporting delay in units of the observation interval, rounded to the nearest integer. These estimates are obtained from Eq (25c) (after substitution of Eqs (25a)) and (38b) when we assume a constant natural mortality rate μ_c. Following §2.3.3, suppose reported incidence is generated from true incidence Z(t_k) via $C_{k+r}\stackrel{\text{ind.}}{\sim }\text{Binomial}(Z\left(t_k\right),p_{\text{rep}})$. Then the variance of C_k+r is

$$\begin{array}{c}\hfill \text{Var}\left(C_{k+r}\right)=Z\left(t_k\right)p_{\text{rep}}(1-p_{\text{rep}}).\end{array}$$ (55)

It follows from Eqs (54) and (55) and the identity Var(aX) = a² Var(X) that

$$\text{Var}\left(I_k^{[\text{S}]}\right)=\frac{\left(1-p_{\text{rep}}\right)Z\left(t_{k+1-g}\right)}{p_{\text{rep}}{\left[\left(\gamma +{\mu }_{\text{c}}\right)\Delta t\right]}^2},$$ (56a)

$$Var\left(I_k^{\text{[SI]}}\right)=\sum _{i=0}^{k-1}\frac{(1-p_{\text{rep}})Z\left(t_{i+1}\right)}{p_{\text{rep}}{[1+\frac{1}{2}(\gamma +{\mu }_{\text{c}})\Delta t]}^2}{\left(\frac{1-\frac{1}{2}(\gamma +{\mu }_{\text{c}})\Delta t}{1+\frac{1}{2}(\gamma +{\mu }_{\text{c}})\Delta t}\right)}^{2(k-1-i)}.$$ (56b)

If Z(t) has a well-defined average 〈Z〉, then replacing instances of Z in Eq (56) with 〈Z〉 and taking the limit as k → ∞, we obtain the average variances

$$\langle Var\left(I_k^{\text{[S]}}\right)\rangle =\frac{\left(1-p_{\text{rep}}\right)\langle Z\rangle }{p_{\text{rep}}{\left[\left(\gamma +{\mu }_{\text{c}}\right)\Delta t\right]}^2},$$ (57a)

$$\begin{array}{cc}⟨Var\left(I_k^{\text{[SI]}}\right)⟩\hfill & =\underset{k\to \infty }{\text{lim}}\left\{\frac{(1-p_{\text{rep}})⟨Z⟩}{p_{\text{rep}}{[1+\frac{1}{2}(\gamma +{\mu }_{\text{c}})\Delta t]}^2}\sum _{i=0}^{k-1}{\left(\frac{1-\frac{1}{2}(\gamma +{\mu }_{\text{c}})\Delta t}{1+\frac{1}{2}(\gamma +{\mu }_{\text{c}})\Delta t}\right)}^{2i}\right\}\hfill \\ & =\underset{k\to \infty }{\text{lim}}\left\{\frac{(1-p_{\text{rep}})⟨Z⟩}{2p_{\text{rep}}(\gamma +{\mu }_{\text{c}})\Delta t}\right[1-{\left(\frac{1-\frac{1}{2}(\gamma +{\mu }_{\text{c}})\Delta t}{1+\frac{1}{2}(\gamma +{\mu }_{\text{c}})\Delta t}\right)}^{2k}\left]\right\}\hfill \\ & =\frac{(1-p_{\text{rep}})⟨Z⟩}{2p_{\text{rep}}(\gamma +{\mu }_{\text{c}})\Delta t}.\hfill \end{array}$$ (57b)

Comparing these with 〈Var(C_k)〉 = 〈Z〉p_rep(1 − p_rep) using reference parameter values t_gen = γ⁻¹ = 13 days, μ_c = 0.04year⁻¹, and Δt = 1 week, we obtain

$$\frac{⟨\text{Var}\left(I_k^{[\text{S}]}\right)⟩}{⟨\text{Var}(C_k\text{)}⟩}=\frac{1}{p_{\text{rep}}^2{\left[\left(\gamma +{\mu }_{\text{c}}\right)\Delta t\right]}^2}=\frac{1}{p_{\text{rep}}^2}{\left(\frac{t_{\text{inf}}}{\Delta t}\right)}^2\approx \frac{3.44}{p_{\text{rep}}^2},$$ (58a)

$$\frac{⟨\text{Var}\left(I_k^{[\text{S}]}\right)⟩}{⟨\text{Var}(C_k\text{)}⟩}=\frac{1}{2p_{\text{rep}}^2{\left[\left(\gamma +{\mu }_{\text{c}}\right)\Delta t\right]}^2}=\frac{1}{2p_{\text{rep}}^2}{\left(\frac{t_{\text{inf}}}{\Delta t}\right)}^2\approx \frac{0.93}{p_{\text{rep}}^2},$$ (58b)

where t_inf = (γ + μ_c)⁻¹ is the mean time spent infected. Hence, while both the S and SI methods suffer from propagation of noise from reported incidence C_k to estimated prevalence I_k, particularly for p_rep ≪ 1, the S method tends to be much worse (by a factor of 3.44/0.93 ≈ 3.7 in this example). Comparative resistance to noise propagation is a distinct advantage of the SI method over the S method.

3.3 Averaging the raw estimate of β(t)

Fig 3A displays two raw estimates β_k (S and SI methods, applied without input error) of a seasonally forced β(t), each spanning 1000 years (only the first 10 years are shown). The estimates embed 1000 1-year cycles, which are displayed in Fig 3B and 3C together with their 1-year average (cf. §2.2.5).

Fig 3. Bias and variance in 1-year cycles embedded in three estimates of a seasonally forced β(t).

[Panel A] In black, the seasonally forced β(t) (Eq (27)) underlying 1000 years of simulated reported incidence data. In (transparent) colour, raw estimates β_k obtained from the data by the S [green] and SI [red] methods, both applied without input error. Only the first 10 of 1000 years are shown. [Panels B and C] In black, the true 1-year cycle in the seasonally forced β(t). In light (transparent) colour, the 1000 1-year cycles embedded in the linear interpolant β_int(t) of β_k. In dark colour, the average 1-year cycle (Eq (22a)) in β_int(t). Results are shown for both the S [Panel B, green] and SI [Panel C, red] methods. [Panel D] Like Panel C, except for a smooth loess curve β_loess(t; q) (q = 53) fit to β_k, instead of the interpolant β_int(t). [Details] A reported incidence time series with 1000 years of weekly observations (Δt = 1 week, n = 52153) was simulated with environmental noise in transmission (ϵ = 0.5), demographic stochasticity, and random under-reporting of cases (p_rep = 0.25), using reference values (Table 1) for the remaining parameters.

Both estimates suffered from spurious noise distorting the correct seasonal pattern, caused by process and observation error in the data-generating process (cf. §3.2). As in Fig 2C, the variance was markedly smaller with the SI method. Averaging the embedded 1-year cycles recovered the true 1-year cycle from the noise. In the absence of input error, the S method appears to carry a slight bias (peaking early and too high, as in Fig 1), whereas the SI method is nearly unbiased.

While some existing infectious disease time series span several centuries [15], in practice, averaging as in Fig 3B and 3C is sensible only over time intervals during which the underlying seasonal pattern in transmission is roughly stationary.

3.4 Smoothing the raw estimate of β(t)

Regardless of whether averaging is employed, comparison of Fig 3C and 3D shows that it is helpful to smooth the β_k time series by fitting a loess curve β_loess(t; q) (cf. §2.2.6). An appropriate degree of smoothing (i.e., choice of loess smoothing parameter q) eliminated spurious noise without significantly increasing bias.

Fig 4A quantifies the effect of smoothing β_k using the optimal value q_opt for parameter q (cf. §2.2.6). It plots RRMSE before and after smoothing as a function of the amount of noise in the simulated reported incidence data, which was modulated by varying the case reporting probability p_rep between 0.01 and 1 (more noise for smaller p_rep; see Eq (31)).

Fig 4. Reduction in β(t) estimation error with optimal loess smoothing.

The horizontal axis measures the case reporting probability p_rep, for which 41 values equally spaced on a logarithmic scale between 0.01 and 1 were considered. Using each value of p_rep and reference values (Table 1) for all other parameters, 100 reported incidence time series (Δt = 1 week, n = 1042) were simulated accounting for environmental noise in transmission (ϵ = 0.5), demographic stochasticity, and random under-reporting of cases (measured by p_rep). The underlying seasonally forced β(t) (Eq (27)) was estimated from reported incidence using the S and SI methods, both applied without input error, yielding two raw estimates β_k per simulation. Smooth loess curves β_loess(t; q) (q = 10, …, 110; cf. §2.2.6) were fit to each β_k time series. The optimal q for a given time series, denoted by q_opt, was defined as the value that minimized RRMSE (Eq (33)) in β_loess(t_k; q). Overall, for each value of p_rep and each β(t) estimation method (S and SI), 100 values of q_opt were obtained corresponding to 100 β_k time series. Plotted on the vertical axis as functions of p_rep are the median and 5th and 95th percentiles of [Panel A] RRMSE in the raw estimates β_k [dashed lines] and optimal loess estimates β_loess(t_k; q_opt) [solid lines] and [Panel B] q_opt. Lines and bands indicate the median and 5th–95th percentile range, respectively. Results for the S and SI methods are shown in green and red, respectively.

Using the optimal loess estimate β_loess(t_k; q_opt) instead of the raw estimate β_k significantly reduced RRMSE—by at least 46% for the S method and 17% for the SI method across all simulations. Although raw estimates generated by the SI method were consistently more accurate (expected in light of Fig 3B and 3C), optimal loess estimates were comparable between the S and SI methods for p_rep > 0.2 (RRMSE ≈ 3%). For p_rep < 0.2 (severe under-reporting of cases), optimal smoothing failed to an increasing extent to recover the underlying β(t) from noise in β_k. In this setting, the S method was greatly outperformed by the SI method, which is more resilient to noise in reported incidence (cf. §3.2).

Fig 4B shows that median q_opt was roughly constant for p_rep > 0.1, with

$$\begin{array}{c}\hfill \text{median}{q}_{\text{opt}}\approx \{\begin{array}{cc}65\hfill & \text{for}\text{the}\text{S}\text{method},\hfill \\ 53\hfill & \text{for}\text{the}\text{SI}\text{method}.\hfill \end{array}\end{array}$$ (59)

More smoothing (greater q) was required to minimize RRMSE for p_rep > 0.1. More generally, Fig 4 indicates that the S and SI methods should always include a smoothing step. Hence, in the remaining analysis, we always smooth β_k.

3.5 Sensitivity to data-generating parameters

Here, we characterize the sensitivity of β(t) estimation error to parameters of the data-generating process. As in §§3.1–3.4, we consider the ideal case in which the user-specified values of all input parameters are equal to the true (data-generating) values. The details of our analysis are outlined in §2.6.1.

Fig 5 plots the median RRMSE in estimates of a seasonally forced β(t) (Eq (27)) from 1000 realizations of a reported incidence time series, as a bivariate function of the mean 〈β〉 and amplitude α of seasonal forcing. To aid interpretation, the 〈β〉 axis was scaled to measure the basic reproduction number ${\mathcal{R}}_0$ (Eq (2)).

Fig 5. Sensitivity of β(t) estimation error to the mean 〈β〉 and amplitude α of seasonal forcing.

Contained in each panel are heatmaps of median RRMSE (Eq (33)) in estimates of a seasonally forced β(t) (Eq (27)) from simulated reported incidence time series, as a bivariate function of the mean 〈β〉 and amplitude α of seasonal forcing. The 〈β〉 axis has been scaled to measure the basic reproduction number ${\mathcal{R}}_0$ (Eq (2)). When simulating reported incidence, reference values (Table 1) were assigned to all data-generating parameters except 〈β〉 and α. A grid of $({\mathcal{R}}_0,\alpha )$ pairs with levels ${\mathcal{R}}_0=2,3,\dots ,32$ and α = 0, 0.01, …, 0.2 was considered, with 〈β〉 defined for each value of ${\mathcal{R}}_0$ via Eq (2). For each parametrization, 1000 simulations were performed with environmental stochasticity [ES] (ϵ = 0.5) and with or without demographic stochasticity [DS] and observation error [OE], as indicated by row: [Row A] without DS or OE (p_rep = 1, t_rep = 0 weeks), [Row B] with DS but without OE (p_rep = 1, t_rep = 0 weeks), [Row C] with DS and OE (p_rep = 0.25, t_rep = 2 weeks). Corresponding mock birth and natural mortality time series were created, then β(t) was estimated from the data using [Left] the S method and [Right] the SI method, all without input error. For each set of estimates of β(t) (1000 estimates per parametrization, per simulation method, per estimation method), the median RRMSE was calculated (after smoothing with fixed q; see Eq (59)) and displayed as one point in the appropriate heatmap, coloured according to the logarithmic scale on the right. The darkest blue indicates median RRMSE less than 0.01.

Fig 6 plots median RRMSE as a univariate function of each of 6 additional parameters—the initial states S₀ and I₀, vital rates ν_c and μ_c, mean generation interval t_gen, and case reporting probability p_rep—with the focal parameter assigned values between $\frac{1}{4}$ and 4 times its reference value (Table 1). The horizontal axis measures the ratio of the focal parameter’s data-generating value to its reference value, so that commensurate deviations from the reference case can be compared across the 6 parameters.

Fig 6. Sensitivity of β(t) estimation error to data-generating parameters other than 〈β〉 and α.

Plotted in each panel is the median RRMSE (Eq (33)) in estimates of a seasonally forced β(t) (Eq (27)) from simulated reported incidence time series (Δt = 1 week, n = 1042), as a univariate function of each of 5 or 6 data-generating parameters (indicated by the legend). When simulating reported incidence, reference values (Table 1) were assigned to all but the focal parameter, which was assigned 41 values logarithmically spaced between $\frac{1}{4}$ and 4 times its reference value. The horizontal axis (logarithmic scale) measures the ratio of the focal parameter’s true value to its reference value, so that commensurate deviations from the reference case can be compared across parameters. For each parametrization, 1000 simulations were performed with environmental stochasticity [ES] (ϵ = 0.5) and with or without demographic stochasticity [DS] and observation error [OE], as indicated by row: [Row A] without DS or OE (p_rep = 1, t_rep = 0 weeks), [Row B] with DS but without OE (p_rep = 1, t_rep = 0 weeks), or [Row C] with DS and OE (p_rep = 0.25 except when p_rep is the focal parameter, t_rep = 2 weeks). Corresponding mock birth and natural mortality time series were created, then β(t) was estimated from the data using [Left] the S method and [Right] the SI method, all without input error. For each set of estimates of β(t) (1000 estimates per parametrization, per simulation method, per estimation method), the median RRMSE was calculated (after smoothing with fixed q; see Eq (59)) and displayed as one point in the appropriate panel and graph.

In order to produce Figs 5 and 6, we assigned reference values (Table 1) to all but the focal data-generating parameter(s) (e.g., all except 〈β〉 and α in Fig 5). We fit loess curves β_loess(t; q) to all raw estimates β_k of β(t), and recorded the RRMSE in β_loess(t_k; q). Motivated by Fig 4B and Eq (59), we fixed q = q*, taking q* = 65 with the S method and q* = 53 with the SI method.

A pattern in our interpretation of Figs 5 and 6 below is that error in β(t) estimation is sensitive to a parameter if changes in that parameter (i) cause incidence Z(t) or prevalence I(t) to approach zero more frequently or more closely, or (ii) increase noise in estimated incidence Z_k or estimated prevalence I_k. Both outcomes incorrectly increase noise in β_k (cf. §3.2).

When the noise in β_k is extreme, setting q = q* can undersmooth the time series (q* < q_opt). In this case, smaller RRMSE is attainable by determining q_opt and setting q = q_opt. Nevertheless, we did not find q_opt for each of the 5 × 10⁶ time series considered by Figs 5 and 6, which would have increased the total computation time by a factor of 100. Consequently, Figs 5 and 6 may overestimate the sensitivity of β(t) estimation error to data-generating parameters. (In §S5.3 of S1 Text, we show that the quantitative effect of choosing q* over q_opt is likely to be small.)

3.5.1 Sensitivity to the basic reproduction number ${\mathcal{R}}_0$ and seasonal amplitude α (Fig 5)

For fixed α, median RRMSE was a non-monotonic function of ${\mathcal{R}}_0$. The reason is that changes in (effective) ${\mathcal{R}}_0$ are responsible for dynamical transitions that alter the structure of solutions of the SIR model (1) [28, 42, 43]. Specifically, as ${\mathcal{R}}_0$ is increased from 2 to 32, minimum incidence Z_min and minimum prevalence I_min on the attractor varies non-monotonically (see Fig 2 in [28]). Smaller Z_min and I_min yield more noise in β_k, and correspondingly greater RRMSE. For fixed ${\mathcal{R}}_0$, I_min decreases monotonically as α is increased from 0 to 1 (see Fig 11 in [43]), so we expect median RRMSE to increase monotonically with α, as observed in Fig 5.

3.5.2 Sensitivity to the initial state (S₀, I₀) (Fig 6)

RRMSE is sensitive to the data-generating S₀, but not I₀. The reference values of S₀ and I₀ are taken from a point (S*, I*, R*) on the attractor of the SIR model (1) with seasonally forced β(t) and constant vital rates ν_c and μ_c (cf. §2.3.4). When S₀ is far from S*, the solution of system (1) undergoes extreme fluctuation before relaxing to the attractor, and both Z and I approach zero during the transient, generating spurious noise at the start of the β_k time series.

Note that I₀ differing from I* has a much smaller effect on dynamics than S₀ differing from S* by the same factor. Since I* ≪ S*, the perturbation of (S₀, I₀, R₀) from the attractor is much smaller.

3.5.3 Sensitivity to vital rates ν_c and μ_c (Fig 6)

Median RRMSE was a non-monotonic function of the data-generating birth rate ν_c. This behaviour arises because scaling ν_c is dynamically equivalent to scaling ${\mathcal{R}}_0$ by the same factor [2, 28], and median RRMSE is a non-monotonic function of ${\mathcal{R}}_0$ (cf. §3.5.1 above).

Changing the data-generating natural mortality rate μ_c had a negligible effect on RRMSE. This is unsurprising, because natural death is dominated by recovery and disease-induced death in governing the rate of infected decrease. That is, γ ≫ μ(t) in Eq (1b), so changes in μ_c by up to a factor of 4 have little effect on dynamics.

3.5.4 Sensitivity to the mean generation interval t_gen (Fig 6)

Median RRMSE increased rapidly as the data-generating t_gen was made smaller than 2^−4/5 (roughly 0.57) times its reference value of 13 days. A period-doubling bifurcation occurs near this value of t_gen, and the attractor of the SIR model (1) acquires a 2-year cycle with much smaller Z_min and I_min (see §S5.3.1 of S1 Text). Propagation of noise to β_k intensifies, resulting in greater RRMSE.

The performance of the S method fluctuates more as a function of t_gen than that of the SI method. This occurs because the S method rounds t_gen in the numerator of Eq (25c) to the nearest integer multiple of Δt, and the rounding error oscillates as a function of t_gen. The SI method does not require rounding, so these fluctuations are not observed.

3.5.5 Sensitivity to the case reporting probability p_rep (Fig 6)

When the reported incidence data contain observation error (Fig 6C), RRMSE is additionally sensitive to the case reporting probability p_rep. Decreasing p_rep increases noise in reported incidence C_k (Eq (31)), which is propagated to estimated incidence Z_k, estimated prevalence I_k, and in turn β_k (cf. §3.2).

Fig 6 suggests weak sensitivity to p_rep. However, noise in Z_k and I_k is amplified in β_k to the extent that Z and I are close to zero (cf. §3.2). Hence, for example, if the data-generating t_gen were assigned a value smaller than half its reference value of 13 days, then we would have observed more acute sensitivity to p_rep as a result of closer approaches to zero by Z and I (cf. §3.5.4 above).

3.5.6 S method versus SI method (Figs 5 and 6)

Both the S and SI methods performed well, estimating β(t) with median RRMSE less than 10% across most parametrizations. However, by resisting noise propagation (cf. §3.2), the SI method was significantly less sensitive to the data-generating parameters and to the addition of demographic stochasticity and observation error.

3.6 Sensitivity to mis-specification of input parameters

In §3.5, we considered the ideal situation in which the user knows the true (data-generating) values of the input parameters. Here, we examine the more realistic situation in which the user specifies input parameters with some error. The effect of mis-specification is particularly important for parameters that are difficult to estimate accurately, such as the case reporting probability p_rep. The details of our analysis are outlined in §2.6.2.

We restrict our attention to application of the SI method to reported incidence data simulated with process and observation error. Differences in RRMSE between methods of data simulation and β(t) estimation are dominated (by an order of magnitude) by the increase in RRMSE resulting from mis-specified input parameters.

Fig 7A plots the median RRMSE in estimates of β(t) from 1000 realizations of a reported incidence time series, as a univariate function of the factor by which an input parameter—one of the initial states S₀ and I₀, mean generation interval t_gen, vital rates ν_c and μ_c, and case reporting parameters p_rep and t_rep—was mis-specified. The specified value of the focal parameter was varied between $\frac{1}{4}$ and 4 times its true (data-generating) value, and the remaining parameters were specified without error.

Fig 7. Sensitivity of β(t) estimation error to the user-specified values of input parameters.

[Panel A] Median RRMSE (Eq (33)) in estimates of β(t) from simulated reported incidence time series (Δt = 1 week, n = 1042), as a univariate function of the factor by which an input parameter was mis-specified. One thousand simulations were performed using fixed values (Table 1) for all data-generating parameters. The simulations accounted for environmental stochasticity [ES] (ϵ = 0.5), demographic stochasticity [DS], and observation error [OE] (p_rep = 0.25, t_rep = 2 weeks). For each simulation, corresponding mock birth and natural mortality time series were created, and β(t) was estimated from the data using the SI method. True (data-generating) values were specified for all input parameters except the focal parameter (indicated by the legend), for which 41 values logarithmically spaced between $\frac{1}{4}$ and 4 times the true value were specified in turn. Each input parametrization yielded 1000 estimates of β(t), whose median RRMSE was calculated (after smoothing with fixed q; see Eq (59)) and displayed as one point in the appropriate graph. [Panel B] Result of repeating the analysis from Panel A in which S₀ was specified with varying amounts of error, but with the initially erroneous value of S₀ updated using the method of peak-to-peak iteration (PTPI; 25 iterations) prior to β(t) estimation. The original result, obtained without PTPI, is presented for comparison.

3.6.1 Sensitivity to error in the specified initial state $(S_0^{\prime },I_0^{\prime })$

Fig 7 shows that error in the specified value of S₀ is propagated non-negligibly to estimates of β(t), while mis-specification of I₀ has practically no effect on β(t) estimation error. Eqs (40) and (41) show that specifying incorrect values $S_0^{\prime }$ and $I_0^{\prime }$ for S₀ and I₀ creates errors in S_k and I_k that vanish geometrically as k → ∞. However, since t_life ≫ t_inf, the decay is significantly slower in S_k. Indeed, with reference values μ_c = 0.04 year⁻¹, t_gen = γ⁻¹ = 13 days, and Δt = 1 week, we find that a factor of 10 reduction in error between times t_k and t_k+i requires just i = 5 in the infected time series, compared to i = 3002 in the susceptible time series (roughly 58 years with Δt = 1 week). Hence, in practice, accurate reconstruction of S(t), I(t), and in turn β(t) relies on accurate specification of S₀, but not I₀. We address sensitivity to mis-specification of S₀ in §3.7 below.

3.6.2 Sensitivity to error in the specified birth rate ${\nu }_{\text{c}}^{\prime }$ and case reporting probability $p_{\text{rep}}^{\prime }$

Mis-specifying ν_c or p_rep by a factor of 2^1/10 (7.2%) yielded median RRMSE greater than 30%. Mis-specifying by a factor of 2^−1/10 (−6.7%) led to even worse estimates of β(t), with median RRMSE exceeding 100% (not visible in Fig 7A). Eqs (42) and (47) show that specifying incorrect values ${\nu }_{\text{c}}^{\prime }$ and $p_{\text{rep}}^{\prime }$ for ν_c and p_rep generates absolute errors in S_k that tend to increase over time (k) to a limit. In practice, systematic underestimation of births by the B_k time series (modeled here by ${\nu }_{\text{c}}^{\prime }<{\nu }_{\text{c}}$) and overestimation of incidence by the Z_k time series ($p_{\text{rep}}^{\prime }<p_{\text{rep}}$) can cause S_k to eventually take negative values. Once this happens, attempts by the S and SI methods to reconstruct β(t) fail completely.

While this failure may seem concerning, it should be viewed as a tool for diagnosing incorrect birth and case reporting rates: if the S or SI method yields negative S_k for any k, then one should speculate that births were underestimated or that incidence was overestimated, and retry the algorithm with a scaled up B_k time series and/or with greater p_rep (as Z_k is computed by scaling reported incidence by a factor of $p_{\text{rep}}^{-1}$; see Eqs (25a) and (26a)). Of course, overcorrection is also undesirable (cf. right half of Fig 7A). In our work, we have found that a brief exploration of possible adjustments—factors by which to increase B_k and/or p_rep—suffices to identify ones that prevent both negative S_k and pronounced transient dynamics at the start of the susceptible time series (indicating under- or overcorrection).

3.7 Solution of the S₀ estimation problem using PTPI

In §3.6, we showed that the performance of the S and SI methods is highly sensitive to mis-specification of the initial number of susceptibles S₀. Here, we assess PTPI as a way to iteratively improve initially poor estimates of S₀ prior to reconstruction of S(t) and β(t).

Fig 8 demonstrates PTPI for an example in which S₀ was overestimated by a factor of 4 by a user of the SI method. PTPI yielded increasingly accurate estimates of S₀ and correspondingly more accurate reconstructions of S(t) (Fig 8B) and β(t) (Fig 8C). Fig 7B repeats our analysis from §3.6, except using PTPI (25 iterations) to update the incorrect estimate of S₀ prior to reconstructing β(t). We see that application of PTPI in conjunction with the SI method enables accurate β(t) reconstruction independently of errors in the initial estimate of S₀. This result is unsurprising in light of Fig 9, which shows that PTPI converges rapidly (in fewer than 10 iterations) to an accurate estimate of S₀ independently of the initial guess. Due to process error in the underlying dynamics, the relative error in the limiting estimate of S₀ varied between the 1000 realizations of reported incidence considered (5th–95th percentile range [−11.9, 12.5]%, median 0.9%). Process error creates variance in the time between peaks in incidence (see Fig 8A), violating the periodicity assumption of PTPI (the theoretical basis of the technique; cf. §2.8). Nevertheless, Figs 7–9 demonstrate that PTPI can significantly improve S(t) and β(t) reconstruction from roughly periodic incidence data.

Fig 8. Example of S(t) and β(t) reconstruction with an overestimate of S₀ corrected by peak-to-peak iteration (PTPI).

[Panel A] Truncation step of PTPI (Box 5). Plotted is a reconstruction of true incidence Z(t) from a simulated reported incidence time series, before [Z_k, black] and after [${\overline{Z}}_k$, yellow] smoothing with a 13-point central moving average. Vertical lines indicate peaks in ${\overline{Z}}_k$. The times of the first peak in ${\overline{Z}}_k$ and the last peak occurring at the same phase of the cycle (in this case, the last peak) are denoted by t_a and t_b. [Panel B] Iteration step of PTPI (Box 6), where the initial estimates of both S₀ = S(0) and S(t_a) were taken to be 4 times the true (data-generating) value of S₀. Plotted in grey are successive reconstructions of S(t) between times t_a and t_b, generated by updating the estimate of S(t_a) with the estimate of S(t_b) obtained in the previous iteration. Dashed continuations to the left of t_a display estimation of S₀ backwards in time from estimates of S(t_a). Plotted in black is the result of reconstructing S(t) starting from the final estimate of S₀, which was obtained after 25 iterations and had a relative error of roughly 1.4% (compared to 300% in the initial estimate). [Panel C] The sequence of reconstructions of β(t) corresponding to the estimates of S₀ shown in Panel B. [Details] Twenty years of weekly reported incidence (Δt = 1 week, n = 1042) were simulated with environmental noise in transmission (ϵ = 0.5), demographic stochasticity, and random under-reporting of cases (p_rep = 0.25), using reference values (Table 1) for the remaining parameters. Z(t), S(t) and β(t) were reconstructed from reported incidence using the SI method without input error (apart from mis-specification of S₀).

Fig 9. Convergence of estimates of S₀ obtained using peak-to-peak iteration (PTPI).

S₀ was estimated by applying PTPI (25 iterations) to 1000 incidence time series (i.e., 1000 realizations of a reported incidence time series, scaled by $p_{\text{rep}}^{-1}$). An initial guess for S₀ was taken to be $\frac{1}{4}$ or 4 times the true (data-generating) value. For each initial guess, this process generated 1000 sequences of 26 estimates of S₀. Plotted are the median [black lines] and 5th–95th percentile range [grey bands] of the estimate of S₀ at each iteration, for the first 10 iterations. The vertical axis measures (on a logarithmic scale) the ratio of the estimated and true values of S₀, hence convergence close to 1 [dashed green line] represents convergence of the estimates close to the true value. [Details] One thousand reported incidence time series (Δt = 1 week, n = 1042) were simulated with environmental noise in transmission (ϵ = 0.5), demographic stochasticity, and random under-reporting of cases (p_rep = 0.25), using reference values (Table 1) for the remaining parameters, including S₀ (hence S₀ was the same in all simulations). True incidence was estimated from reported incidence via Eq (26a) (with reporting parameters p_rep and t_rep correctly specified), yielding 1000 time series of estimated incidence. Corresponding mock (constant) birth and natural mortality time series were created (with vital rates ν_c and μ_c correctly specified), and these data (estimated incidence, births, natural mortality) were passed to the PTPI algorithm, allowing for iterative re-estimation of S₀.

3.8 Run time

We implemented the S and SI methods and PTPI in R and ran them on a MacBook Pro with a 2.4 GHz Quad-Core Intel Core i5 chip. The S and SI methods are both extremely fast, requiring a total of 0.124 and 0.376 seconds, respectively, to generate a reconstruction of β(t) from 1000 years of weekly reported incidence (Δt = 1 week, n = 52142). Application of PTPI in conjunction with either method increases the run time with each iteration, but the total run time remains inconsequential due to the rate of convergence of the iterations to a limiting estimate of S₀. For example, when we applied PTPI to the same simulated data, the truncation step (Box 5) added 0.094 seconds to the total run time, while the iteration step (Box 5) added 1.01 seconds per iteration on average.

4 Discussion

We have compared three fast methods of estimating the time-varying transmission rate β(t) from reported incidence time series, all based on discretizations of the SIR model (1). Fine and Clarkson’s method [6], referred to here as the FC method, fails rapidly in practice, because it treats natural mortality in the susceptible population as negligible. Although Krylova’s method [24], adapted here as the S method, corrects this limitation of the FC method and is accurate for certain simulated data, her method suffers from extreme sensitivity to process and observation error. Specifically, noise in reported incidence is spuriously propagated to its estimates of β(t). Our algorithm for transmission rate estimation, referred to here as the SI method and based on deJonge’s method [25], is much more resilient to noise in reported incidence and therefore superior to the S method.

Like its predecessors, the SI method is sensitive to (i) certain input parameters: the initial number of susceptible individuals S₀, the case reporting probability p_rep, and the mean generation interval t_gen; as well as (ii) vital data: times series of births and natural mortality without substantial systematic errors.

The requirement of a good estimate of S₀ has been a major barrier to use of existing fast methods of β(t) estimation (including those presented in [6, 24, 25]). We have proposed and demonstrated PTPI as a valid and fast technique for obtaining accurate estimates of S₀ from poor initial guesses, conditional on periodic dynamics (epidemic recurrence with a fixed period). Use of the SI method in conjunction with PTPI represents a major advance over the existing fast methods.

Estimation of the case reporting probability p_rep is possible using maximum likelihood approaches, including trajectory matching. However, a fast way to obtain a crude estimate of p_rep is to divide cumulative reported incidence over the time interval [t₀, t_n], by the cumulative incidence that is expected from the unforced SIR model (system (1) with β ≡ 〈β〉, ν ≡ ν_c, and μ ≡ μ_c) at equilibrium:

$$\begin{array}{c}\hfill p_{\text{rep}}\approx \frac{{\displaystyle \sum _{k=1}^n}{C}_k}{{\nu }_{\text{c}}{\widehat{N}}_0(1-\frac{1}{{\mathcal{R}}_0})(t_n-t_0)}.\end{array}$$ (60)

This approximation can be made in temporal subintervals to obtain a time-varying reporting rate, which would replace the constant p_rep in Eq (26a). Sensitivity of the SI method to mis-specification of the mean generation interval (t_gen) may be of greater concern, though if the distribution of the incubation period (time from infection to onset of symptoms) is narrow, then t_gen will be well approximated by the (observable) mean serial interval [44].

Overall, the SI method, in conjunction with PTPI, represents a highly tractable approach to reconstructing susceptibles and β(t) from infectious disease time series that span decades or centuries. It makes fewer assumptions about the disease and population of interest than the regression-based tSIR method [7, 23] (i.e., it does not require an infectious period equal to the observation interval, ignore susceptible mortality, or assume that cumulative incidence approximates cumulative births). Moreover, it is significantly less complex and much less computationally demanding than simulation-based methods of inference, such as iterated filtering [8, 19, 20] and generalized profiling [21, 22].

Even when the observed infectious disease time series is short enough that simulation-based methods are tractable, the approach to transmission rate reconstruction that we promote here can be usefully employed to provide better starting conditions at negligible computational cost.

Supporting information

S1 Text. Text supplement.

A .pdf document containing annotated R code, making the results reported here completely reproducible by the reader.

(PDF)

S1 File. Source files.

A .zip archive containing all of the source files needed to compile S1 Text.

(ZIP)

Data Availability

All relevant data are found within the manuscript and Supporting Information files. Specifically, all of the data are simulated and can be generated reproducibly by running the R scripts contained in the Supporting Information files.

Funding Statement

MJ was supported by an Undergraduate Student Research Award from the Natural Sciences and Engineering Research Council of Canada (NSERC) and a Student Fellowship from the M. G. DeGroote Institute for Infectious Disease Research. OK was supported by a Postgraduate Scholarship from NSERC. DJDE was supported by a Discovery Grant from NSERC. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.