Flexible Bayesian inference on partially observed epidemics

Abstract

Individual-based models of contagious processes are useful for predicting epidemic trajectories and informing intervention strategies. In such models, the incorporation of contact network information can capture the non-randomness and heterogeneity of realistic contact dynamics. In this article, we consider Bayesian inference on the spreading parameters of an SIR contagion on a known, static network, where information regarding individual disease status is known only from a series of tests (positive or negative disease status). When the contagion model is complex or information such as infection and removal times is missing, the posterior distribution can be difficult to sample from. Previous work has considered the use of Approximate Bayesian Computation (ABC), which allows for simulation-based Bayesian inference on complex models. However, ABC methods usually require the user to select reasonable summary statistics. Here, we consider an inference scheme based on the Mixture Density Network compressed ABC, which minimizes the expected posterior entropy in order to learn informative summary statistics. This allows us to conduct Bayesian inference on the parameters of a partially observed contagious process while also circumventing the need for manual summary statistic selection. This methodology can be extended to incorporate additional simulation complexities, including behavioural change after positive tests or false test results.

1 Introduction

In the study of infectious diseases, mathematical models are useful for predicting trajectories of outbreaks or evaluating the efficacy of intervention strategies. The spread of disease is dependent on non-random, heterogeneous patterns of human contact. To capture the realistic dynamics of interpersonal interactions, it is sometimes necessary to move beyond traditional fully mixed models to individual-based or agent-based models and incorporate contact network information. Contact networks provide a natural representation of human mixing patterns, where a population of individuals are represented as nodes and potential transmission contacts are represented as edges. By leveraging contact network structure, it is possible to infer transmission paths of past epidemics [1], identify individuals vital to the spread of contagion [2, 3] or propose strategies that modify contact network topology in order to control disease [4]. Contact networks have also been extended to non-disease contagions, such as the spread of behaviour and misinformation [5]. Although network data have traditionally been difficult to obtain, emerging technologies, such as Bluetooth proximity sensing, make it increasingly feasible to obtain this type of information at scale.

When considering inference on parameters governing spreading processes on networks, it is appealing to consider statistical inference from a Bayesian perspective. Under this framework, parameters of interest are treated as random variables with an initial prior distribution and the target of inference is the distribution of these parameters conditioned on the observed data. Bayesian inference allows for the incorporation of prior information on uncertain parameters from domain experts, while also providing transparent and interpretable results in the form of posterior distributions. However, in the study of infectious disease, data are commonly missing. Oftentimes, the complete history describing the evolution of an epidemic is unavailable. Infection and recovery times may not be precisely recorded and need to be inferred solely from observations of disease status of individuals at specific time points. Thus, there exists a need for network methods that incorporate the uncertainty in real-world data.

Previous work on Bayesian inference for network epidemics has often focused on methods that sample directly from an analytical posterior via Monte Carlo Markov Chain (MCMC) algorithms [6–9]. In these methods, uncertainty in the data is accounted for via data augmentation, where missing information is treated as a set of latent variables that are then jointly inferred upon along with parameters of interest. However, when large amounts of data are missing, data-augmentation methods can become computationally infeasible due to the high dimensionality of the latent space. Furthermore, in order to maintain analytical tractability, it may be necessary to limit the expressiveness and flexibility of the model. As models become more complex, MCMC methods often require the design of specialized, problem-specific algorithms.

Alternatively, Approximate Bayesian Computation (ABC) describes a set of simulation-guided methods that allow for direct sampling from an approximate posterior distribution without the need to specify a likelihood. Such methods were first developed in the study of genetics [10–12] and have since found use in applications such as ecology and physics [13, 14]. In the study of epidemics, ABC has been used to conduct inference on contagion dynamics in both homogeneously mixed populations [15–17] and on contact networks [18–20]. Under an ABC framework, parameter values are sampled from the prior distribution and applied to a model. Parameter values that lead to simulated data deemed suitably similar to the observed data are accepted as samples from the approximate posterior. Because no likelihood evaluation is required, ABC allows for inferences on parameters of complicated models that may otherwise be intractable.

As ABC methods can suffer from the curse of dimensionality, it is often necessary to summarize the observed data with low-dimensional summary statistics [21]. Outside of exponential likelihood families, Bayes sufficient statistics are usually unavailable. Thus, dimension-reducing summary statistics must be provided by the user, often based on scientific or intuitive understanding of what metrics of epidemic evolution are relevant to the problem. There are numerous techniques for identifying summary statistics that are both low-dimensional and highly informative. One group of such techniques aims to choose a best subset from an existing set of summary statistics [22–24], though such methods typically require a proposed set of user-defined summary statistics to select from. Other methods transform a given set of statistics to construct lower-dimensional statistics [25–28]. Many such methods have been shown to be special cases or large-sample limits of, or equivalent to, an information-theoretic approach that minimizes expected posterior entropy [29].

In this article, we investigate the use of the Mixture Density Network-compressed ABC (MDN-ABC) for disease parameter inference on an incompletely observed epidemic. The MDN-ABC uses a conditional mixture density network to learn low-dimensional but informative summary statistics by minimizing the expected posterior entropy [29]. Unlike the MDN itself, however, the MDN-ABC makes no parametric assumptions about the true posterior. Thus, this framework enjoys the modelling flexibility and asymptotic guarantees of ABC methods while also eliminating the need to define or select summary statistics.

In Section 2 of this article, we will define the notation used to describe a contagious process on a network. We will also discuss the basics of ABC algorithms and the architecture used for the MDN-ABC. In Section 3, we will investigate the performance of MDN-ABC on a simple, fully observed SI epidemic and demonstrate its fidelity to existing closed-form solutions. In Section 4, we will demonstrate the use of MDN-ABC on a partially observed SIR process where disease state of individual nodes is known only from asynchronous tests. We will consider a range of synthetic networks as well as an empirical social network from Karnataka, India [30].

2 Methods

2.1 Approximate Bayesian computation

In Bayesian inference, parameters are treated as random variables. Inferences are based on the posterior distributions: the probability distribution of parameters conditioned on the observed data. This posterior distribution takes the form of:

$$\begin{array}{c}\pi (\theta |Y)\propto L(Y|\theta )\pi (\theta ).\hfill \end{array}$$ (2.1)

Here, $L(Y|\theta )$ is the likelihood of the data and $\pi (\theta )$ is the prior distribution of the parameter. When both the likelihood and prior are tractable, there exist various methods for sampling from the posterior distribution. However, in complex models, these expressions may be difficult to specify in closed form. This includes many stochastic simulation-based problems, as even simple mechanistic models can have complex likelihoods.

ABC is a likelihood-free method first named by [12], though similar methods had previously been applied to problems in population genetics [10, 31]. Crucially, ABC does not require a tractable likelihood. It requires only that given a proposed parameter value, a dataset can be forward-simulated from the model. The simulated data are then compared to the observed data, and parameter values that produce data that are deemed similar enough to the observed data are accepted as posterior samples. The typical rejection ABC follows a simple rejection sampling scheme:

1. Sample a candidate parameter value ${\theta }^{\prime }$ from prior $\pi (\theta )$.

2. Forward simulate the model using ${\theta }^{\prime }$ to obtain a simulated dataset $Y^{\prime }$.

3. Given distance function d, compute distance between simulated and observed datasets: $d(Y_{\text{obs}},Y^{\prime })$.

4. Consider acceptance threshold $ϵ$. If $d(Y_{\text{obs}},Y^{\prime })\le \epsilon$, accept ${\theta }^{\prime }$ as a sample from the posterior.

5. Return to Step 1 until a predetermined number of samples are obtained.

ABC is approximate in two aspects. First, as $ϵ\to 0$, the ABC posterior converges to the target posterior. However, even when data are discrete, setting $ϵ=0$ usually leads to prohibitively large numbers of simulations to draw a desired number of posterior samples. In the continuous case, $P(Y_{\text{obs}}=Y)=0$, so the probability of generating an accepted sample would be 0. Thus, it is necessary to set $ϵ>0$. In such situations, the ABC algorithm is not drawing samples from the proper posterior, but an approximation.

In addition, the observed raw data $Y_{\text{obs}}$ is often high-dimensional. As the kernel smoothing implicit in ABC is subject to performance degradation in high-dimensional settings, it is often necessary to instead calculate summary statistics $S(Y_{\text{obs}})$ and $S(Y^{\prime })$. Steps 3 and 4 are then replaced with:

3a. Given distance function d, compute distance between simulated and observed datasets: $d(S(Y_{\text{obs}}),S(Y^{\prime }))$.

4a. Consider acceptance threshold $ϵ$. If $d(S(Y_{\text{obs}}),S(Y^{\prime }))<ϵ$, accept ${\theta }^{\prime }$ as a sample from the approximate posterior.

When both of these approximations are applied, the distribution being approximated by ABC is

$$\begin{array}{c}p(\theta |s_{\text{obs}})=\int p(\theta ,s|s_{\text{obs}})\mathrm{d}s\propto \int K_{ϵ}(||s-s_{\text{obs}}||)p(s|\theta )p(\theta )\mathrm{d}s.\hfill \end{array}$$ (2.2)

Here, $s_{\text{obs}}=S(Y_{\text{obs}})$, ‖u‖ is the Euclidean norm of u, and $K_{ϵ}(|\left|u\right||)=K\left(\frac{\left|\right|u\left|\right|}{ϵ}\right)/ϵ$ is a smoothing parameter [21]. The rejection ABC draws samples from the joint conditional distribution $p(\theta ,s|s_{\text{obs}})$, which serves as a good approximation to the true posterior when $ϵ$ is small and the summary statistic $s_{\text{obs}}$ is highly informative.

There exists a rich body of literature on other ABC sampling algorithms that can improve computational performance. Examples include the Monte Carlo Markov Chain ABC [32] and the Sequential Monte Carlo ABC [33]. Such methods typically allow for more efficient sampling; instead of sampling repeatedly from a potentially uninformative prior, the proposal region from which the parameter values are drawn is gradually narrowed down. In this article, we consider only the basic rejection ABC algorithm for the sake of simplicity. However, our methodology affects only the definition of summary statistics $S(Y)$, so extensions to other ABC algorithms are straightforward.

2.2 MDN-compressed ABC

In order to avoid the curse of dimensionality, it is usually necessary to compress the output data of our model via some statistic $s(Y)$. However, in general, Bayes sufficient statistics, such that $p(\theta |s(Y_{\text{obs}}))=p(\theta |Y_{\text{obs}})$, are not available. Thus, it is necessary instead to opt for optimal summary statistics that minimize some loss functional. Hoffmann and Onnela [29] proposed minimization of the expected posterior entropy (EPE)

$$\begin{array}{c}\mathcal{H}=-\int p(s,\theta )\hspace{0.17em}\text{log}\hspace{0.17em}p(\theta |s)\mathrm{d}s\mathrm{d}\theta .\hfill \end{array}$$ (2.3)

Typically, the loss function employed for neural network training is the Monte Carlo estimate of the expected posterior entropy:

$$\begin{array}{c}\widehat{\mathcal{H}}=-m^{-1}\sum _{i=1}^m\hspace{0.17em}\text{log}\hspace{0.17em}f({\theta }_i,s(Y_i)),\hfill \end{array}$$ (2.4)

where $f(\theta ,t)$ is a conditional density estimator that approximates the posterior, ${\theta }_i$ and $Y_i$ are joint samples from $p(\theta ,Y)$, and m is the number of samples in the minibatch. The minimization of the EPE is equivalent to other information theoretic approaches, including minimizing expected Kullback–Leibler divergence, maximizing mutual information, and maximizing expected surprise [29].

Conceptually, related work has proposed the use of mixture density networks (MDNs) [34] to learn a conditional posterior density estimation that minimizes the EPE [35]. The MDN itself is a combination of a neural network and a mixture density model. Using the Monte Carlo estimate of the EPE as the loss function, the neural network is trained to learn the weights and the parameters for the mixture distribution that approximates the posterior. For example, if the mixture distribution takes the form of K Gaussian components, the neural network of the MDN would learn the K mixture weights, means, and variances associated with the mixture distribution. Compared to ABC, MDN methods have the advantage of directly learning an approximate distribution to the posterior, instead of simply sampling from an $ϵ$-ball centred on the observed data [35]. However, such methods rely on parametric assumptions about the posterior distribution and do not enjoy the same asymptotic guarantees as ABC.

Hoffmann and Onnela [29] introduced a method dubbed MDN-compressed ABC, or MDN-ABC. This framework combines MDN and ABC methods using two simple neural networks combined in series. The overall network is trained to minimize EPE with a mixture density network. Instead of using the output of the MDN, the ABC summary statistics are extracted as the output of the first neural network (known as the compressor network). The second neural network (known as the output network) is utilized only for the learning of the conditional density estimator. A schematic of this method is shown in Fig. 1.

Figure 1:

The MDN-ABC utilizes two simple neural networks in series. The combined neural network inside the dashed rectangle is trained to generate a conditional mixture density that minimizes the expected posterior entropy (right). However, the estimated mixture density relies on parametric assumptions about the posterior distribution. Instead, we extract a single layer of the full neural network (the output layer of the compressor network) and treat it as our summary statistic for ABC. This schematic is a simplified visualization, and the depth and width of the neural networks utilized may not correspond to those pictured.

MDN-ABC learns highly informative summary statistics and compares well with the MDN posterior. Furthermore, in some scenarios, the posterior may be multimodal or otherwise complex. While the conditional density estimate can theoretically approximate the posterior arbitrarily closely given infinite computational budget, the parametric assumptions may be too restrictive to capture the true posterior in practical settings. Other methods that estimate the posterior mean of the parameters as the summary [26] also have difficulty with multimodal posteriors. Under these situations, the MDN-ABC can still perform well, due to the lack of parametric assumptions. In this article, we will focus on the MDN-ABC; however, MDN-ABC yields a conditional density estimator as a byproduct, so our framework can be easily extended to a pure MDN approach.

2.3 Epidemics on networks

We consider a network $\mathcal{G}$. $\mathcal{G}$ consists of a set of nodes $\mathcal{N}=\{1,\dots ,n\}$, representing individuals, which are connected by edges $\mathcal{E}\subseteq \mathcal{N}\times \mathcal{N}$. In studies of epidemics, such edges represent potential paths of transmission. For this article, we focus on static, undirected, unweighted networks. However, the ABC framework allows for flexible implementation of more complex network dynamics as well. As long as the model can be easily forward-simulated, such complexity does not affect the asymptotic properties of ABC.

We consider two types of contagion, expressed through compartmental models [36]. In an SI process, we assume that nodes can take on one of two disease states: susceptible (S) and infectious (I). Nodes begin in the susceptible state, and progress to the infectious state with a per-contact rate of $\beta$ in continuous time. This parameter is directly translatable to the definition of transmissibility in [37]. Once infected, nodes are capable of transmitting the infection to susceptible contacts, and they remain the infectious state for the remainder of the epidemic. In an SIR process, nodes can take on an additional state: recovered (R). Under this model, infected nodes progress to the recovered state with rate $\gamma$. Once recovered, nodes can no longer infect susceptible nodes or be infected by infected nodes.

We consider continuous SIR epidemics similar to those defined in [38]. At time t each node $i\in \mathcal{N}$ has a disease state $X_t^i$. For SI models, $X_t^i\in \{S,I\}$, and for SIR models, $X_t^i\in \{S,I,R\}$. The instantaneous transition rates are defined as follows:

$$\begin{array}{cc}\gamma \hfill & =\underset{\Delta t\to 0}{\lim }\frac{P(X_{t+\Delta t}^i=R|X_t^i=I)}{\Delta t}.\hfill \end{array}$$ (2.5)

$$\begin{array}{cc}\beta \hfill & =\underset{\Delta t\to 0}{\lim }\frac{P(X_{t+\Delta t}^i=I \text{via} j|X_t^i=S,X_t^j=I)}{\Delta t}.\hfill \end{array}$$ (2.6)

Here, $\{X_{t+\Delta t}^i=I \text{via} j\}$ is the event that node i is infected by infected neighbour j. Note that for SI epidemics, $\gamma =0$. In this article, we will primarily focus on the posterior distributions of $\beta$ and $\gamma$ as the targets of inference.

It has been shown that overly coarse discretization of continuous-time processes can lead to misleading conclusions [38]. Thus, we implemented a continuous-time Gillespie simulation of an SIR epidemic.

1. Begin with $B<\left|\mathcal{N}\right|$ nodes in the infected state. All other nodes are considered susceptible. Initiate at $t=0$.

2. While $t<t_{\text{max}}$:

   • 2.1 For each node i, calculate a transition coefficient ${\alpha }_i$ based on node i’s current status.

     1. Recovered nodes have transition rate 0.

     2. Infected nodes have transition rate $\gamma$.

     3. Susceptible nodes have transition rate $m_i\beta$, where $m_i$ is the number of node i’s neighbours that are currently in the infected state.

   • 2.2 Increment t by random variable $\tau$, where $\tau \sim \text{Exponential}\left(\frac{1}{\sum _i{\alpha }_i}\right)$.

   • 2.3 Select a single node j to transition. The probability of selection for each node i is $\frac{{\alpha }_i}{\sum _i{\alpha }_i}$.

   • 2.4 If node j is susceptible, it becomes infected. If node j is infected, it becomes recovered.

   • 2.5 If there are no remaining infected nodes, end simulation. Otherwise, return to Step 2.

For this simple infection process, if infection and recovery times are fully observed for all nodes and gamma-distributed priors are assumed for $\beta$ and $\gamma$, closed-form solutions are available for the maximum likelihood estimates and the posterior distributions of both parameters [8]. However, stochastic network epidemics, even when mechanistically simple, can have complex likelihoods. The complexity is further compounded when data for infection and recovery times are missing. Thus, this specific problem is a good candidate for ABC methods.

3 Simple example: fully observed SI epidemic

In this section, we will consider a completely observed SI epidemic and compare the results of the MDN-compressed ABC to closed-form solutions. Here, $\beta$ is the only parameter we wish to infer on, as $\gamma =0$. Since all infection times are observed, the scenario is simple enough for closed-form solutions to be available. Derived in [8], the maximum likelihood estimate for $\beta$ is:

$$\begin{array}{c}{\widehat{\beta }}_{\text{MLE}}=\frac{n_I-1}{\sum _{j=1}^n{e}_{SI}(t_j)(t_j-t_{j-1})}.\hfill \end{array}$$ (3.1)

Here, $n_I$ is the total number of infected individuals at the end of the epidemic. $e_{SI}(t)$ is the number of edges existing between susceptible and infected individuals at time t, and $t_1<t_2<\cdots <t_n$ are ordered infection times for all nodes.

Next, given a Gamma-distributed prior for $\beta$ such that $\beta \sim \text{Gamma}(a,b)$, the posterior distribution of $\beta$ given a vector of ordered event times $\mathit{t}$ is:

$$\begin{array}{c}\beta |\mathit{t}\sim \text{Gamma}(a+(n_I-1),b+\frac{(n_I-1)}{{\widehat{\beta }}_{\text{MLE}}}).\hfill \end{array}$$ (3.2)

In order to diagnose sensitivity of our method to network topology, we varied two aspects of the underlying contagion network: degree distribution (Poisson or log-normal) and mean degree (2, 4 or 8). This led to six scenarios, for all of which MDNs were trained using an identical neural network architecture, optimizer and training regime. Poisson-distributed networks were generated as Erdös-Rényi random graphs [39] and log-normal distributed networks were generated with a Chung–Lu model [40] with expected degrees drawn from a log-normal distribution with ${\sigma }^2$ fixed at 0.5. We also ensured that each graph consisted of a single connected component. For each component outside of the largest connected component (LCC), we selected a random unconnected dyad between the LCC and the smaller component and added one additional edge. This addition of edges changes the asymptotic degree distribution of the networks, and we will refer to the two new degree distributions as ‘Connected Erdös–Rényi’ (CER) and ‘Connected Log-Normal’ (CLN). All networks consisted of 100 nodes. We chose the prior for $\beta$ to be $\text{Gamma}(2,4)$, a distribution that is much more disperse (variance of 0.125) than the true posterior and has a mean (0.5) different than the true value of $\beta$. We set the true value of $\beta$ to be 0.15 and chose a single origin node to be infected at time $t=0$. The simulation continued until all nodes reach the infected state.

For each network scenario, a total of $5\times {10}^6$ realizations of the continuous-time SI simulation were generated for the training set, and $2.5\times {10}^6$ realizations were generated for the validation set. Each simulation was run by drawing a proposal $\beta$ from the Gamma prior and forward-simulating the epidemic to obtain the infection time of each node. Every simulation used the same underlying network and began at the same origin node selected in the original epidemic. Methods for inferences on an unknown source are discussed in [19]. For the MDN-ABC, the raw data were the infection time $t_i$ for each node. The MDN was trained by minimizing the expected posterior entropy with a similar method to [29]. We utilized two gamma components for our conditional posterior density, and a 15-dimensional feature space. Our compressor was a simple feed-forward neural network with four hidden layers. The total number of nodes per layer was: [100, 80, 60, 30, 20, 15]. All layers were fully connected with hyperbolic tangent activation functions. The output network consisted of three fully connected neural networks with two hidden layers (nodes per layer: [15, 10, 10, 2]) that learned the mixing coefficients, shapes and rates of the two gamma-distributed components for the MDN. The entire neural network was trained with stochastic gradient descent with a minibatch size of 500. We used an Adam optimizer with the learning rate initialized at $5\times {10}^{-5}$. After each epoch (full pass through the training data), validation loss was evaluated by calculating the Monte Carlo estimate of EPE on the validation set. If 10 epochs elapsed without an improvement in validation loss, training was terminated.

For our ABC, we reused the $5\times {10}^6$ training realizations as our pool of simulations. Using the summary statistics learned by our MDN-ABC architecture, we calculated the Euclidean distance between the 15-dimensional summary statistic vector of each simulation output and the 15-dimensional summary statistic calculated from the original ‘observed’ epidemic. We then selected the best $0.02\%$ of the training set, resulting in approximately 1000 posterior samples. In addition, we also drew posterior samples using the raw data itself, $(t_1,\dots ,t_{100})$, as the statistic of interest, terming this method the ‘uncompressed ABC’. For a single network scenario (log-normal degree distribution with mean degree 8), the Gaussian kernel estimates of the posteriors drawn from the compressed MDN-ABC and the uncompressed ABC are shown in Fig. 2(left), where they are compared with the closed-form posterior from [8].

Figure 2:

The left figure shows the Gaussian kernel density estimate (used for visualization purposes only) of the posterior draws from the MDN-ABC, the analytical solution, and rejection ABC from the uncompressed raw data itself. The right figure shows violin plots comparing the MDN-ABC posteriors to the analytical solution [8], across 10 different instances of the epidemic. The network is Connected Log-Normal with a mean degree of 8.

In addition, the epidemic trajectory of the original ‘true’ epidemic is stochastic. In order to examine the variance in posteriors introduced by this stochasticity, we regenerated 10 instances of ‘original’ epidemics and re-drew 1000 MDN-ABC posterior samples without retraining the MDN. In Fig. 2 (right), we compare these 10 MDN-ABC posteriors to the gold standard closed-form solutions.

Similarly, for every network scenario, we generated 10 instances of original epidemics and used MDN-ABC to obtain 1000 posterior samples. In Fig. 3, we show $95\%$ credible intervals obtained from the MDN-ABC samples, compared with 5000 samples drawn from the closed-form solution for the posterior distribution.

Figure 3:

Comparison of 95% credible intervals for MDN-ABC posterior samples to samples from the analytical solution [8], over six network scenarios: (a) Connected Erdös-Rényi (CER) with mean degree 2, (b) Connected Log-Normal (CLN) with mean degree 2, (c) CER with mean degree 4, (d) CLN with mean degree 4, (e) CER with mean degree 8 and (f) CLN with mean degree 8, each with 10 instances of an original epidemic.

In this example, we can observe that the MDN-ABC can generate posterior samples that match closely with the analytical solution. However, in more complex situations, such as scenarios where the infection and recovery times are missing, closed-form solutions are not available. In addition, due to the missing event times, many summary statistics such as those employed in [20] and [19], including the time of the peak of the epidemic curve, the mean of infection times, and the length of epidemic, cannot be calculated. In the next section, we will consider the use of MDN-ABC in this case of the partially observed epidemic.

4 SIR epidemic with missing outcome data

Previous literature regarding inferences on network epidemics have typically assumed that event times are precisely known [19, 20], or are aggregated over coarse time intervals [8, 18]. In previous ABC applications to network epidemics, the chosen summary statistics, such as the time of the peak of the epidemic [20] or the proportion of nodes infected at each time step [19], are dependent on exact knowledge of event times.

However, in most real-world epidemics, the exact times of infection and recovery are often unobserved. Instead, information regarding disease status is known from observations of individual disease status. These observations may not be synchronous, as the disease status of individuals may be sampled at different times. Some individuals may not be observed at all. As the test status of individuals is not necessarily observed at the time of their transition between states, coarse aggregations of outcomes (e.g. number of infections per day) may also be misleading. Similar work by [17] has also considered ABC for applications in which case-detection times of an epidemic are observed; however, this work makes the assumption that case-detection corresponds to removal time, and all such removal times are detected. In our example, we do not assume that individuals who have received a positive test proceed to follow any quarantine procedure, but our model can easily be extended to accommodate test-dependent contact avoidance.

Similar to the fully observed SI epidemic, for the SIR epidemic with missing outcome data, we chose the prior for $\beta$ and $\gamma$ both to be $\text{Gamma}(2,4)$. We set the true value of $\beta$ to be 0.15 as before and the true value of $\gamma$ to be 0.1. We initialized $5\%$ of the population to be infected at time $t=0$. The simulation continued until time $t=50$. Unlike the fully observed scenario, exact infection and recovery times were considered to be unavailable. Instead, each node was randomly assigned a time $0<t<7$ to begin testing and was then tested every 7 time steps. If each time step is considered to be a day, this would correspond to a testing cadence of 1 week. A test was said to have returned positive if the node was infected, and returned negative if the node was susceptible or recovered. In this simulation, we considered mandatory tests on individuals, such that an individual’s probability of being tested or adhering to a testing schedule was independent of their disease status. Tests were also assumed to have perfect sensitivity and specificity, though false positives and negatives can easily be incorporated into the model. All contact networks consisted of 100 nodes in one connected component. Over the simulation spanning $t=50$ time steps, the 100 nodes yielded a total of 720 test results.

A total of $5\times {10}^6$ realizations of the continuous-time SIR simulation were generated for the training set, and $2.5\times {10}^6$ realizations were generated for the validation set. The raw data consisted of the sequence of test results (positive or negative) obtained for each node. Again, the MDN was trained to learn two gamma components for our conditional posterior density. Our compressor consisted of six hidden layers ([720, 300, 200, 100, 60, 40, 30, 15]), and the output network had two hidden layers ([15, 15, 15, 2] for the mixing coefficients of the mixture of gamma distributions, and [15, 15, 15, 4] for the parameters of the mixture of gamma distributions). We extracted a total of 15 summary features. The MDN was trained with the same training regime as the fully observed SI model. MDN-ABC posterior samples were again drawn by picking the best $0.02\%$ of the training data, minimizing the Euclidean distance in the 15-dimensional summary statistic vector.

Again, to examine sensitivity of our method to varying degree distributions, we plot 95% credible intervals obtained across six different scenarios for the underlying contact network (Connected Erdös–Rényi and Connected Log-Normal degree distributions, with mean degrees of 2, 4 and 8) in Fig. 4. For mass action models, the intensity of an epidemic is often summarized with the basic reproduction rate $R_0$, which describes the expected number of secondary infections arising from a single infected individual introduced to a completely naive homogeneous population. Such calculations are more complicated in network models with heterogeneous contact structures. To develop intuition about the expected severity of epidemics, we consider the expected number of cases arising from a single infection introduced into a susceptible network. As derived in [41], if each edge has a uniform probability of transmission $\tau$ and the initial infected node is randomly selected from the population, the expected number of infections arising from that node is $\tau C_1$ and the expected number of infections arising from infected nodes other than the first is $\tau \frac{C_2}{C_1}$. Here, if $k=\{0,\dots ,n-1\}$, and $p_k$ is the degree distribution of the network,

$$\begin{array}{c}{C}_1=\sum k{p}_k.\hfill \end{array}$$ (4.1)

$$\begin{array}{c}{C}_2=\sum (k-1)k{p}_k.\hfill \end{array}$$ (4.2)

Figure 4:

Visualization of 95% credible intervals for posterior samples from the MDN-ABC for a partially observed SIR epidemic for 10 realizations of the original epidemic, over six network scenarios: (a) Connected Erdös-Rényi (CER) with mean degree 2, (b) Connected Log-Normal (CLN) with mean degree 2, (c) CER with mean degree 4, (d) CLN with mean degree 4, (e) CER with mean degree 8 and (f) CLN with mean degree 8.

We calculated these values for the empirical degree distributions of the synthetic networks, each consisting of 100 nodes, generated for our six network scenarios. For the Connected Erdös-Rényi (CER) network with approximate mean degree 2, $C_1=2.30$ and $C_2=4.18$. For the CER network with approximate mean degree 4, $C_1=4.24$ and $C_2=16.92$. For the CRT network with approximate mean degree 8, $C_1=7.80$ and $C_2=59.96$. For the Connected Log-Normal (CLN) network with approximate mean degree 2, $C_1=2.30$ and $C_2=5.52$. For the CLN network with approximate mean degree 4, $C_1=3.98$ and $C_2=20.50$. For the CLN network with approximate mean degree 8, $C_1=8.38$ and $C_2=106.72$. As one example, setting $\tau =1$, a randomly selected initially infected node in the CER network with mean degree 4 would lead to $C_1=4.24$ expected secondary infections and subsequent infected nodes would each lead to $\frac{C_2}{C_1}=\frac{16.92}{4.24}=3.99$ expected secondary infections.

In order to validate our posterior samples, we also evaluated the coverage properties of the credible intervals derived from MDN-ABC samples [42–44]. For each network scenario, we drew tuples $({\beta }_{(1)},{\gamma }_{(1)}),\dots ,({\beta }_{(n)},{\gamma }_{(n)})$ from the prior distributions of $\beta$ and $\gamma$. Using these values of $\beta$ and $\gamma$, we simulated $n=5000$ epidemics and used MDN-ABC to obtain 5000 sets of posterior samples. Following from [27], we define an $\alpha \%$ credible interval as the interval I such that $\text{Pr}(\theta \in I|Y_{\text{obs}})=\alpha /100$. If the MDN-ABC posterior samples are a good approximation to the true posterior, the $\alpha \%$ credible intervals estimated from the MDN-ABC posterior samples should contain the “true” values of $\beta$ and $\gamma$ in $\alpha \%$ of the simulations. Thus, for each of the 5000 simulated epidemics, we used the credible intervals estimated from each set of MDN-ABC posterior samples to calculate the empirical coverage for for $\alpha \%$ intervals ranging from $\alpha =0$ to 100. The empirical coverage plots are shown in Fig. 5. Significant departures from linearity would indicate a poor fit, while linearity indicates that that MDN-ABC provides a good approximation to the true posterior.

Figure 5:

Coverage properties for MDN-ABC posteriors for a partially observed SIR epidemic, over six network scenarios: (a) Connected Erdös-Rényi (CER) with mean degree 2, (b) Connected Log-Normal (CLN) with mean degree 2, (c) CER with mean degree 4, (d) CLN with mean degree 4, (e) CER with mean degree 8 and (f) CLN with mean degree 8.

4.1 Empirical network: Karnataka

To demonstrate the use of MDN-ABC on a real-world network, we simulated an SIR process on a social network sampled from a village in Karnataka, India [30]. This empirical network consists of 354 nodes representing individual people of whom 346 are part of the largest connected component (LCC). Edges represent relationships between individuals and were obtained from surveys in which individuals named others with whom they interacted with. Individuals were specifically questioned regarding a number of social interaction types, including transaction of money, exchange of advice and home visits. The network is assumed to be undirected, as relationships are assumed to be reciprocal. The mean degree is approximately 8.7, and the maximum degree was 35. The simulated epidemic was identical to the one described in the previous section. The true value of $\beta$ was set to 0.15 and the true value of $\gamma$ to 0.1. To seed the epidemic, 5% of the population was initialized in the infected state. All individuals were assigned a day of the week to begin testing, and infection statuses were known only from weekly tests that returned a positive or negative status. The prior for each parameter was a Gamma(2,4) distribution. For the MDN, we employed a similar neural network architecture as the partially observed SIR case, though for this example, we extracted 20 summary statistics as opposed to 15.

In Fig. 6, we display the results for MDN-ABC on a the simulated epidemic on the Karnataka network. We also once again extracted the 95% credible intervals across 10 different realizations of the original epidemic and examined the coverage properties by drawing independent realizations of $(\beta ,\gamma )$ from the prior and evaluating the empirical coverage probabilities of the MDN-ABC posteriors. Lastly, we re-simulated epidemics using values of $\beta$ and $\gamma$ drawn from the MDN-ABC posterior, sampling from the posterior predictive of epidemic trajectories.

Figure 6:

MDN-ABC results for simulated epidemic on Karnataka village network: (a) MDN-ABC posterior samples, (b) 95% credible intervals for $\beta$ across 10 instances of original epidemic, (c) 95% credible intervals for $\gamma$ across 10 instances of original epidemic, (d) empirical coverage for $\beta$, (e) empirical coverage for $\gamma$ and (f) 150 epidemic trajectories drawn from the posterior predictive.

In the examples presented in Section 4, we have focused on simulated SIR epidemics with recovery times that follow an exponential distribution. This model is computationally convenient and commonly employed, but is often unrealistic for modelling most real-world epidemics. Fortunately, the flexibility of MDN-ABC allows for easy adjustments to the epidemic model, as inference is possible as long as forward-simulation of the epidemic is computationally inexpensive. In Section 1 of the Appendix, we present the MDN-ABC results for a simulated SIR epidemic with non-Markovian recovery times on the Karnataka network.

4.2 Interpretability of features

While MDN-ABC allows users to skip the subjective summary statistic selection phase of ABC, this task is delegated to a black-box neural network. The individual summary statistics are a layer of neurons within the neural network trained on simulated datasets. Unfortunately, the meaning of each individual statistic produced by the MDN often lacks the interpretability of more intuitive measures of epidemic progression, such as statistics describing the epidemic curves and subgraphs induced by infected individuals.

In the field of neural networks, especially in image recognition, rich literature exists on interpretability for neural network behaviour. Such methods include saliency maps [45] and activation minimization [46]. These techniques can be applied to qualitatively interpret MDN-ABC features. However, optimization-based visualization may struggle in this specific example. Given a particular epidemic model and a fixed network, not all epidemic trajectories are possible. For example, in our model, nodes receive the contagion from infected neighbours (there are no outside sources of infection); thus, no node can become infected before its neighbours have been infected, or after all of its neighbours have recovered.

For one example of feature visualization, we created a synthetic network similar to a Cayley tree. The origin node of the epidemic at the center of the tree was parent to six subtrees, where each subtree was a complete binary tree with four layers. We again generated a partially observed SIR epidemic, this time using $\beta =0.35$ and $\gamma =0.07$ for a more aggressive epidemic. The testing cadence was once again $t=7$ time steps, and the simulation proceeded for a total of 50 time steps. We employed a similar architecture from the partially observed SIR epidemic simulation example, and trained an MDN with 2 gamma components and 15 features. We then extracted the training samples that generated the maximum activation in each of the 15 neurons that serve as the MDN-ABC features. In Figure 7, we display the infected nodes and times of infection for the six training epidemics that contribute to the greatest activation of the ninth neuron out of 15 (recovery times are not pictured). By using such representations, it becomes possible to visualize what epidemic trajectories are generally captured by each feature.

Figure 7:

Six training samples that yield the maximum activation in feature neuron 9 of 15. Shades of nodes denote time of infection. Grey nodes remained uninfected during the epidemic.

Notably, the data used to train the neural network did not represent the entire epidemic trajectory; the only information available for training was the series of tests results, positive and negative, observed during the epidemic. However, the epidemics represented by these training samples share topological features that are readily interpretable by humans. For example, the epidemics that led to the maximum activation of the ninth MDN-ABC feature exhibited relatively rapid spread, indicating that the underlying $\beta$ is large. In addition, these epidemics were largely contained to a single subtree in the network, suggesting that the infectious period is also short (ie the origin node recovered before passing the contagion onto the remaining subtrees).

5 Discussion

This article presents MDN-ABC as a method for conducting Bayesian inference for network epidemics in situations where information for outcomes comes only from individual reports of disease status. As summary statistics are no longer defined by the user but learned by an MDN via EPE minimization, MDN-ABC offers increased flexibility in handling diverse data types and allows for increased model expressiveness in simulation-based inference.

Our article invites several directions for extension. Our work primarily focuses on relatively simple compartmental disease models with the SI and SIR contagion types. Potential extensions to more expressive compartmental models such as SEIR may better describe specific diseases of interest, such as COVID-19 and influenza. Some diseases may also follow more complex models of transmission, where infected individuals may differ in their ability to transmit to susceptible individuals, due to viral load or other factors. In addition, most networks are not closed systems, and infection can enter the network from an outside community. For such situations, the incorporation of an additional “spark term”, an instantaneous contact-independent rate of infection, may be more realistic. These additional layers of complexity can all be parameterized and added to the simulated model to better reflect a particular epidemic under study.

Within the current scope of our article, MDN-ABC can accommodate a wide range of types of observed data. We focus here on binary disease outcomes, but it is straightforward to extend this work to multiple outcomes (e.g. the presence or absence of multiple symptoms) or continuous outcomes (e.g. viral load). A primary limitation, however, is that the dimensionality of the observed data must be independent of the disease parameters. Essentially, our current model is limited to cases where an individual’s test/observation schedule is independent of their disease status. This may be a reasonable assumption for cases where tests are mandated, which was a common intervention against COVID-19 [47–49]. This assumption may also hold for diseases observed in wildlife [50] or livestock, as well as for environmental testing such as testing on wastewater [48]. However, in many realistic situations, individuals may be more likely to seek out testing if they experience symptoms or if their close contacts are known to have been infected. Similarly, individuals who are aware of their positive disease status may temporarily cease contact with others, effectively removing them from the study population [51, 52]. In such cases, the MDN architecture must be adjusted to allow for variable input sizes. We leave this extension for future work.

The underlying contact networks in our simulations are assumed to be unweighted, undirected, static and perfectly observed. The likelihood-free nature of ABC allows for the inclusion of more nuanced contagion-network interactions, so implementation of weighted edges or dynamic networks is straightforward. Another avenue for further work is in the incorporation of uncertainty in the observed network itself. Typically, networks must be imputed from observed data, such as contact diaries and surveys [53, 54] or close-proximity events detected by wearable RFID devices [54, 55] or Bluetooth [56, 57]. To account for the uncertainty associated with the underlying contact network, methods similar to [58, 59] may be combined with MDN-ABC to sample from the joint posterior of the epidemic parameters of interest and parameters that model the missingness of network data.

One drawback of the MDN-ABC is its significant computational cost compared to other ABC approaches. It requires the training of a mixture density network, so sufficiently large datasets for training and validation must be simulated. However, these simulations represent independent realizations of epidemics, so this problem is embarrassingly parallel. In addition, once training of the neural network is complete, more advanced ABC sampling algorithms can be applied for increased sampling efficiency while utilizing the summary statistics defined by the MDN. A further limitation of MDN-ABC, as well as most other ABC techniques, is difficulty in inferences on high-dimensional parameters, as the size of the MDN must expand to account for mixture densities of higher dimension.

For the purposes of this article, we utilized a simple feedforward neural network architecture for our MDN. However, recent advances in neural network models may be employed for more efficient learning of the mixture density parameters. For example, developments in graph neural networks (GNNs) allows users to leverage the underlying relationships within the data [60]. Typically, GNNs employ a message passing scheme that allows nodes to gather information about neighbouring nodes via a chosen aggregation operator. For network epidemics, test results or event times may be represented as node-level attributes. Extensions to GNNs include heterogeneous graph neural networks [61], which allow for the inclusion of different types of nodes with different attributes. In this article, GNNs were not strictly necessary to obtain reasonable results. Thus, we chose to use a relatively generic feedforward neural network, which was intuitively simple and easily applicable to both epidemic settings that we discussed (fully observed SI process and partially observed SIR process). This simpler neural network architecture is also more easily generalizable to epidemic models that may not necessarily have network-structured data, such as compartmental models or agent-based models. However, in real-world applications of MDN-ABC, it can be beneficial to explore more specialized neural network architectures for the specific type of epidemic model and available data. The exact architecture of the feed-forward neural network in MDN-ABC may be changed without major perturbations in the inferential conclusions. While there may be slight differences in the resultant approximate posteriors due to the differences in the learned summary statistics or the choice of layer from which the summary statistics are drawn, conclusions are consistent over a range of architectures. We show MDN-ABC posterior distributions from a variety of neural network architectures in the Appendix.

While network data have traditionally been difficult to obtain, emerging technologies, such as Bluetooth proximity sensing, make it increasingly feasible to obtain this type of information at scale. The availability of contact network information can be utilized to study infectious diseases with realistic and high-resolution models. However, there remains a need to incorporate real-world uncertainties in inferences on contagion parameters. By minimizing EPE, it is possible to employ an MDN to learn informative summary statistics for ABC settings, while allowing for uncertainty associated with partial observations. MDN-ABC can thus be used as a flexible method to develop a more nuanced understanding of infectious disease spread and contribute to individual-level risk exposure assessments and targeted interventions.

A Appendix

A.1 Non-Markovian recovery times

Due to the simplicity of compartmental models, it is common in epidemic modelling to assume that the infectious period is exponentially distributed. However, this is sometimes unrealistic; for example, an exponential infectious period would imply that the most probable infectious period is 0. Thus, it is often necessary to account for non-exponential recovery times when seeking to capture infectious disease dynamics [1, 62, 63]. Compared to analytical methods, the flexibility of the MDN-ABC allows for straightforward adjustment of the epidemic model. We chose to use a gamma distribution to model the duration of infectivity. The underlying network was once again chosen to be the network from the Karnataka study.

For computational convenience, the simulation proceeded in discrete time, with a discrete time-step of 0.2. In order to maintain a transmission rate roughly equivalent to the continuous-time examples in Section 4, we set $\tilde{\beta }=0.03$. Though similar to the transition rate given in Section 2.3, this $\tilde{\beta }$ coefficient captures the discrete-time per-contact transmission probability, as opposed to a continuous-time rate. Due to the low transition probability, this discretized simulation approximates the continuous SIR process reasonably well [38]. The time to recovery was modeled as a gamma distribution with shape parameter ${\gamma }_a$ and rate parameter ${\gamma }_b$. The true values of these parameters were set to ${\gamma }_a=100$ and ${\gamma }_b=10$, such that the mean infectious period was equivalent to the mean of the exponentially distributed infectious period in Section 4.1. Our inference targeted the three parameters $(\tilde{\beta },{\gamma }_a,{\gamma }_b)$. Once again, the simulation initiated with 5% of the population randomly chosen for infection. The prior was set to be Gamma(0.5,0.8) for $\tilde{\beta }$, Gamma(10,8) for ${\gamma }_a$ and Gamma(2,5) for ${\gamma }_b$. The MDN was trained with the same architecture and optimizer as Section 4.1. Again, we used 10 stochastic realizations of the epidemic to obtain 95% credible intervals of the MDN-ABC posteriors. We also evaluated coverage properties by drawing 5000 parameter values from the prior and obtaining the empirical coverage of the credible intervals. All results are shown in Fig. A1.

A.2 Sensitivity to neural network structure

In this article, we chose to employ a simple feed-forward neural network. The number of layers and number of nodes in each layer can often be varied without major changes to inferential conclusions drawn from the MDN-ABC. However, there may exist some differences in the approximate posteriors drawn from summary statistics learned by neural networks of differing architectures due to the black-box nature of the neural network. In Fig. A2, we show the 95% credible intervals of MDN-ABC posteriors drawn from an array of neural network layouts given in Table A1, for the partially observed SIR epidemic on a Connected Erdös–Rényi network with mean degree 8 (this was one of the six network scenarios discussed in Sections 3 and 4). For each architecture, we used the same training and validation datasets, with the same optimizer and training schedule as Section 4.

Figure A1:

MDN-ABC results for simulated epidemic on Karnataka village network, using non-Markovian recovery times: (a) empirical coverage for $\tilde{\beta }$, (b) empirical coverage for ${\gamma }_a$, (c) empirical coverage for ${\gamma }_b$, (d) 95% credible intervals for $\tilde{\beta }$ across 10 instances of original epidemic, (e) 95% credible intervals for ${\gamma }_a$ across 10 instances of original epidemic, (f) 95% credible intervals for ${\gamma }_b$ across 10 instances of original epidemic and (g) 150 epidemic trajectories drawn from the posterior predictive.

Figure A2:

Results for SIR epidemic on Connected Erdös-Rényi network with mean degree 8, using neural network architectures listed in Table A1: (a) 95% MDN-ABC credible intervals for $\beta$, (b) Gaussian kernel density estimates for $\beta$, (c) 95% MDN-ABC credible intervals for $\gamma$ and (d) Gaussian kernel density estimates for $\gamma$.

Table A1:

Neural network architectures used for sensitivity testing. All layers are fully connected. Note that the full output network contains three fully connected networks that represent the weights and the two parameters of the gamma components. When inferring on two epidemic parameters, the output network that represents the weights has half as many nodes in the output layer as the output networks that represent the gamma distribution parameters. Architecture 3 is the same architecture used in Section 4.

	Compressor network	Output network
1	[720, 100, 50, 15]	[15, 4]
2	[720, 200, 100, 50, 15]	[15, 15, 4]
3	[720, 300, 200, 100, 60, 40, 30, 15]	[15, 15, 15, 4]
4	[720, 300, 200, 100, 80, 60, 40, 30, 30, 15]	[15, 15, 15, 15, 4]

Funding

This work was supported by the National Institutes of Health [T32AI007358, R01 AI138901].