From viral evolution to spatial contagion: a biologically modulated Hawkes model

Abstract

Summary

Mutations sometimes increase contagiousness for evolving pathogens. During an epidemic, scientists use viral genome data to infer a shared evolutionary history and connect this history to geographic spread. We propose a model that directly relates a pathogen’s evolution to its spatial contagion dynamics—effectively combining the two epidemiological paradigms of phylogenetic inference and self-exciting process modeling—and apply this phylogenetic Hawkes process to a Bayesian analysis of 23 421 viral cases from the 2014 to 2016 Ebola outbreak in West Africa. The proposed model is able to detect individual viruses with significantly elevated rates of spatiotemporal propagation for a subset of 1610 samples that provide genome data. Finally, to facilitate model application in big data settings, we develop massively parallel implementations for the gradient and Hessian of the log-likelihood and apply our high-performance computing framework within an adaptively pre-conditioned Hamiltonian Monte Carlo routine.

Supplementary information

Supplementary data are available at Bioinformatics online.

1 Introduction

The coronavirus disease 2019 (COVID-19) pandemic has demonstrated the need for new scientific tools for the analysis and prediction of viral contagion across human landscapes. The mathematical characterization of the complex relationships underlying pathogen genetics and spatial contagion stands as a central challenge of 21st century epidemiology. We approach this task by unifying two distinct probabilistic approaches to viral modeling. On the one hand, Bayesian phylogenetics (Mau et al., 1999; Sinsheimer et al., 1996; Suchard et al., 2001; Yang and Rannala, 1997) uses genetic sequences from a limited collection of viral samples to integrate over high-probability shared evolutionary histories in the form of phylogenies or family trees. On the other hand, self-exciting, spatiotemporal Hawkes processes (Reinhart, 2018) model spatial contagion by allowing an observed event to increase the probability of additional observations nearby and in the immediate future.

Both modeling paradigms come with their own advantages. For Bayesian phylogenetics, the past twenty years have witnessed a slew of high-impact scientific studies in viral epidemiology (Boni et al., 2020; Dudas et al., 2017; Faria et al., 2014; Gire et al., 2014; Rambaut et al., 2008; Smith et al., 2009) and the rise of powerful computing tools facilitating inference from expressive, hierarchical models of phylogenies and evolving traits (Ronquist et al., 2012; Suchard et al., 2018). Unfortunately, the number of evolutionary trees to integrate over explodes with the number of viral samples analyzed (Felsenstein, 1978), so Bayesian phylogenetic analyses typically restrict to a relatively small number of viral samples, at most totaling a few thousand. The fact that viral cases that undergo genetic sequencing usually represent a small subset of the total case count exacerbates this issue. Thus, failure to detect phylogenetic clades that represent novel strains on account of computational and surveillance limitations always remain a possibility. Until now, these weaknesses have also held for the sub-discipline of Bayesian phylogeography, which attempts to relate viral evolutionary histories to geographic spread as represented by (typically Brownian) phylogenetic diffusions. These models describe viral spread through either discretized (Lemey et al., 2009) or continuous (Lemey et al., 2010) space, but both approaches induce their own form of bias (Holbrook et al., 2021a). In the face of these shortcomings, Bayesian phylogeography needs new tools for directly modeling spatial contagion (Table 1).

Table 1.

Comparison of two probabilistic modeling paradigms within viral epidemiology, the combination of which represents a new tool for Bayesian phylogeography

	Traditional Bayesian phylogenetics	Hawkes processes
Observational limit	N in low thousands	N in high tens-of-thousands
Biological insight	Evolutionary history	None
Genetic sequencing	Required	Not required
Spatiotemporal data	Not required	Required
Geographic spread	Not modeled	Modeled
Large-scale transport	Does not induce bias	Induces bias

Hawkes processes (Hawkes, 1971a,b, 1972, 2018; Hawkes and Adamopoulos, 1973) are widely applicable point process models for generally viral or contagious phenomena, such as earthquakes and aftershocks (Fox et al., 2016; Hawkes and Adamopoulos, 1973; Ogata, 1988; Zhuang et al., 2004), financial stock trading (Bacry et al., 2015; Hawkes, 2018), viral content on social media (Kobayashi and Lambiotte, 2016; Rizoiu et al., 2017), gang violence (Holbrook et al., 2021b; Loeffler and Flaxman, 2018; Mohler, 2013, 2014; Park et al., 2019) and wildfires (Schoenberg, 2004). Unsurprisingly, Hawkes processes are natural models for the contagion dynamics of biological viruses as well. Kim (2011) uses spatiotemporal Hawkes processes (Reinhart, 2018), which model viral cases as unmarked events in space and time, to model the spread of avian influenza virus (H5N1). Meyer and Held (2014) incorporate power laws to describe spatial contagion dynamics and model meningococcal disease in Germany from 2001 to 2008. Although Rizoiu et al. (2018) do not model epidemiological data, they do draw connections between epidemiological susceptible, infectious or recovered (SIR) models and Hawkes processes, showing that the rate of events in the SIR model is equal to that of a finite-population Hawkes model. Kelly et al. (2019) apply a temporal nonparametric Hawkes process to the 2018–2019 Ebola outbreak in the Democratic Republic of the Congo and successfully generate accurate disease prevalence forecasts. Chiang et al. (2020) model COVID-19 cases and deaths in the USA at the county level using spatially indexed mobility and population data to modify the process conditional intensity. Most recently, Bertozzi et al. (2020) compare the performance of a temporal Hawkes process model with temporally evolving conditional intensity to that of SIR and susceptible, exposed, infectious or recovered (SEIR) models for modeling regional COVID-19 case dynamics.

Because such Hawkes processes do not involve genetic information, one may apply the model to a much larger collection of cases, i.e. those for which a timestamp and spatial coordinates are available. Moreover, recent successes in scaling Hawkes process inference to a big data setting enable inference from observations numbered in the high tens-of-thousands (Holbrook et al., 2021b; Yuan et al., 2021). This ability to interface with an order of magnitude more cases represents a major benefit of the Hawkes process in comparison to the Bayesian phylogenetic paradigm, but the tradeoff is that conclusions drawn from a Hawkes process analysis are devoid of explicit biological insight. It is possible for the model to attribute self-exciting dynamics to nearby viral cases that are only distantly related to the phylogenetic tree. Finally, these processes do not immediately account for viral spread through large-scale transportation networks (Brockmann and Helbing, 2013; Holbrook et al., 2021a) but attribute events resulting from such contagion to a ‘background’ process.

In the following, we construct a Bayesian hierarchical model that allows both modeling approaches to support each other. This model (Fig. 1) learns phylogenetic trees that describe the evolutionary history of the subset of observations that yield genetic sequencing and uses this history to inform the distribution of a latent relative rate or productivity (Schoenberg, 2020; Schoenberg et al., 2019) for each virus in this limited set. In turn, these virus-specific rates modify the rate of self-excitation of a spatiotemporal Hawkes process describing the contagion of all viruses, sequenced or not. We use a Metropolis-within-Gibbs strategy to jointly infer all parameters and latent variables of our phylogenetic Hawkes process and overcome the $O(N^2)$ computational complexity of the Hawkes process likelihood by incorporating the modified likelihood in the hpHawkes open-source, high-performance computing library (Holbrook et al., 2021b) available at https://github.com/suchard-group/hawkes. Within the same library, we also develop multiple parallel computing algorithms for the log-likelihood gradient and Hessian with respect to the model’s virus-specific rates. Graphics processing units (GPU)-based implementations of these gradient and Hessian calculations score 100-fold speedups over single-core computing and help overcome quadratic complexity in the context of an adaptively preconditioned Hamiltonian Monte Carlo (HMC; Neal, 2011). These speedups prove useful in our analysis of 23 421 viral cases from the 2014 to 2016 Ebola outbreak in West Africa.

Fig. 1.

The phylogenetic Hawkes process relates the evolutionary history of a virus to the rate at which subsequent viral cases occur nearby. (Left) A phylogenetic tree characterizes the evolution of a viral strain. (Middle) A 1D Brownian motion ‘along the tree’ describes the evolution of infinitesimal rates as a function of branch lengths and tree topology (Section 2.2). Branch lengths influence this evolution insofar as they dictate the length of time over which the individual rates evolve; tree topology influences this evolution insofar as the end of a parent’s trajectory fixes the beginning of the child’s trajectory. (Right) Virus-specific rates additively contribute to the Hawkes process’ rate function for future cases (Section 2.1)

2 Materials and methods

We develop the phylogenetic Hawkes process and its efficient inference in the following sections. Importantly, our proposed hierarchical model integrates both sequenced and unsequenced viral case data, representing a significant and clear contribution insofar as:

1. the percentage of confirmed viral cases sequenced during an epidemic is often in the single digits (Wadman, 2021); and

2. previous phylogeographic models have failed to leverage additional information provided by geolocated, unsequenced case data.

We address this shortcoming by constructing a new hierarchical model that both models all spatiotemporal data with a Hawkes process (Section 2.1) and allows an inferred evolutionary history in the form of a phylogenetic tree to influence dependencies between relative rates of contagion (Section 2.2) for the small subset of viral cases for which genome data are available. We believe that this approach is altogether novel.

2.1 Spatiotemporal Hawkes process for viral contagion

Hawkes processes (Hawkes, 1971a,b, 1972, 2018) constitute a useful class of inhomgeneous Poisson point processes (Daley and Jones, 2003) for which individual events contribute to an increased rate of future events. Spatiotemporal Hawkes processes (Reinhart, 2018) are marked Hawkes processes with spatial coordinates for marks (Daley and Jones, 2003). We are interested in spatiotemporal Hawkes processes with infinitesimal rate:

$$\lambda (\mathbf{x},t)=\mu (\mathbf{x})+\xi (\mathbf{x},t)=\mu (\mathbf{x})+\sum _{t_n<t}{g}_n(\mathbf{x}-{\mathbf{x}}_n,t-t_n)\hspace{0.17em},$$

where $\mathbf{x}\in {ℝ}^D,\hspace{0.17em}t\in {ℝ}^{+}$ and the subscript n indicates that the usual triggering function $g(·,·)$ takes on different forms depending on some characteristic associated with event n. These non-negative, monotonically non-increasing, event-indexed triggering functions additively contribute to $\xi (·,·)$, the self-excitatory rate component, and encourage this rate to increase after each observed event. Here, $\mu (·)$ is the background rate and only depends on spatial position $\mathbf{x}$. Conditioned on observations $({\mathbf{x}}_n,t_n)$, $n=1,\dots ,N$, we specify the rate components:

$$\mu (\mathbf{x})=\frac{{\mu }_0}{{\tau }_x^D}\sum _{n=1}^N\varphi \left(\frac{\mathbf{x}-{\mathbf{x}}_n}{{\tau }_x}\right){\mathcal{I}}_{[x\ne x_n]}\hspace{0.17em}\hspace{1em}\text{and}\hspace{1em}\xi (\mathbf{x},t)=\frac{{\theta }_0\omega }{h^D}\sum _{t_n<t}{\theta }_n\hspace{0.17em}{e}^{-\omega (t-t_n)}\varphi \left(\frac{\mathbf{x}-{\mathbf{x}}_n}{h}\right)\hspace{0.17em},$$

where ${\tau }_x>0$ and h > 0 are the background and self-excitatory spatial lengthscales, ${\mu }_0>0$ and ${\theta }_0>0$ are the background and self-excitatory weights, $1/\omega >0$ is the self-excitatory temporal lengthscale, $\varphi (·)$ is the D-dimensional standard normal probability density function, and the background rate’s indicator function prevents a trivial maximum at ${\tau }_x\to 0$ (Habbema et al., 1974; Robert, 1976). The inclusion of ${\theta }_n>0$ for $n=1,\dots ,N$ within the self-excitatory rate marks a major departure from similar model specifications in Holbrook et al. (2021b) and Loeffler and Flaxman (2018). These ‘degrees of contagion’ or ‘productivities’ (Schoenberg, 2020; Schoenberg et al., 2019) allow different events to contribute differently to the overall self-excitatory rate of the process: the larger the θ_n, the higher the rate directly following event n (Fig. 1, right). Following the connection of the Hawkes process with exponential triggering function to a discret-time SIR model (Rizoiu et al., 2018), Bertozzi et al. (2020) refer to these quantities as a reproduction number. In the following, we refer to θ_n as the event-specific, case- or virus-specific rate for the nth event, case or viral observation.

Denoting $\mathit{\theta }={({\theta }_1,\dots ,{\theta }_N)}^T$, the likelihood of observing data $(\mathbf{X},\mathbf{t})={(({\mathbf{x}}_1,t_1),\dots ,({\mathbf{x}}_N,t_N))}^T$ is (Daley and Jones, 2003):

$$\mathcal{L}(\mathbf{X},\mathbf{t}|{\mu }_0,{\tau }_x,{\theta }_0,\mathit{\theta },\omega ,h)=\text{exp}\hspace{0.17em}\left(-{\int }_{{ℝ}^D}{\int }_0^{t_N}\lambda (\mathbf{x},t)\hspace{0.17em}\mathrm{d}t\hspace{0.17em}\mathrm{d}\mathbf{x}\right)\prod _{n=1}^N\lambda ({\mathbf{x}}_n,t_n):=e^{-\Lambda (t_N)}·\prod _{n=1}^N{\lambda }_n\hspace{0.17em}.$$

The choice of ${ℝ}^D$ for integration domain is popular and often necessary but assumes complete observation over the entirety of ${ℝ}^D$ (Schoenberg, 2013). The resulting integral may be written (Appendix A):

$$\Lambda (t_N)={\mu }_0{t}_N-{\theta }_0\sum _{n=1}^N{\theta }_n(e^{-\omega (t_N-t_n)}-1):=\sum _{n=1}^N{\Lambda }_n\hspace{0.17em},$$

leading to a log-likelihood of

$$\ell (\mathbf{X},\mathbf{t}|{\mu }_0,{\tau }_x,{\theta }_0,\mathit{\theta },\omega ,h)=-\Lambda (t_N)+\sum _{n=1}^N\hspace{0.17em}\text{log}\hspace{0.17em}{\lambda }_n=\sum _{n=1}^N\{\hspace{0.17em}\text{log}\hspace{0.17em}[\sum _{n\prime =1}^N(\frac{{\mu }_0\hspace{0.17em}{\mathcal{I}}_{[{\mathbf{x}}_n\ne {\mathbf{x}}_{n\prime }]}}{{\tau }_x^D}\varphi \left(\frac{{\mathbf{x}}_n-{\mathbf{x}}_{n\prime }}{{\tau }_x}\right)+\frac{{\theta }_0{\theta }_{n\prime }\omega \hspace{0.17em}{\mathcal{I}}_{[t_{n\prime }<t_n]}}{h^D}{e}^{-\omega (t_n-t_{n\prime })}\varphi \left(\frac{{\mathbf{x}}_n-{\mathbf{x}}_{n\prime }}{h}\right))]-{\Lambda }_n\}:=\sum _{n=1}^N\left[\text{log}\hspace{0.17em}\left(\sum _{n\prime =1}^N{\lambda }_{nn\prime }\right)-{\Lambda }_n\right]:=\sum _{n=1}^N{\ell }_n\hspace{0.17em}.$$ (1)

Here, we use the following shorthand notations: Λ_n is the additive contribution of the nth event to the likelihood’s integration term; ${\lambda }_{nn\prime }$ refers to the additive contribution of the $n\prime$th event to the rate function evaluated at the nth event ${\lambda }_n=\lambda ({\mathbf{x}}_n,t_n)$; and ${\ell }_n$ is the overall additive contribution of the nth event to the log-likelihood. We reference these formulas while outlining our inference strategy in Section 2.3 and detailing our massively parallel algorithms for calculating the log-likelihood gradient and Hessian with respect to event-specific rates $\mathit{\theta }$ in Appendix B. We describe our biologically modulated joint prior on event-specific rates ${\theta }_1,\dots ,{\theta }_N$ in Section 2.2.

2.2 Phylogenetic Brownian process prior on rates

We use standard Bayesian phylogenetics hierarchical approaches (Suchard et al., 2003) to model a single molecular sequence alignment $\mathbf{S}$ containing sequences from $M\le N$ evolutionarily related viruses. Let $\mathcal{M}$ denote the ordered index set with cardinality $\left|\mathcal{M}\right|=M$ containing every number within the set $\{1,\dots ,N\}$ that corresponds to an observed virus for which genome data are present. In the following, we number the elements within $\mathcal{M}$ as $m_1,m_2,\dots ,m_M$. Moreover, we make use of the set ${\mathcal{M}}^{+}$ with cardinality $\left|{\mathcal{M}}^{+}\right|=2M-1$, satisfying $\mathcal{M}\subset {\mathcal{M}}^{+}$ and containing elements $m_1,\dots ,m_{2M-1}$. Our primary object of interest is the phylogenetic tree $\mathcal{G}$ (Fig. 1, left) defined as a bifurcating, directed graph with M terminal degree-1 nodes $({\nu }_{m_1},\dots ,{\nu }_{m_M})$ that correspond to the tips of the tree (or sequenced observations), $M-2$ internal degree-3 nodes $({\nu }_{m_{M+1}},\dots ,{\nu }_{m_{2M-2}})$, a root degree-2 node ${\nu }_{m_{2M-1}}$ and edge weights $(w_{m_1},\dots ,w_{m_{2M-2}})$ that encode the elapsed evolutionary time between nodes. Here, each w_m communicates the expected number of molecular substitutions per site, which is itself the product between the real-time duration and the evolutionary rate arising from a molecular clock model. For example, we use a relaxed molecular clock model (Drummond et al., 2006) that allows for substitution rates to flexibly vary across branches (Section 3.2). One may either know $\mathcal{G}$  a priori or endow it with a prior distribution parameterized by some vector $\varphi$. Suchard et al. (2001, 2018) develop the joint distribution $p(\mathbf{S},\varphi ,\mathcal{G})$ in detail.

We assume that the event-specific rates $\mathit{\theta }$ defined within our Hawkes model take the form (Fig. 1, middle):

$$\{\begin{array}{cc}{\theta }_n={\theta }_n(z_n)=\text{exp}\hspace{0.17em}(z_n+{\mathit{\beta }}^T\mathbf{f}(t_n))\hfill & z_n\in ℝ\hspace{0.17em},\hspace{1em}n\in \mathcal{M}\hfill \\ {\theta }_n=1\hfill & \hspace{1em}n\notin \mathcal{M}\hspace{0.17em},\hfill \end{array}$$

and that the elements of vector $\mathbf{z}={(z_{m_1},\dots ,z_{m_M})}^T$ follow a Brownian diffusion process along the branches of $\mathcal{G}$ (Cavalli-Sforza and Edwards, 1967; Felsenstein, 1985; Lemey et al., 2010). Here, $\mathbf{f}(·)$ is some fixed vector function and the inclusion of the linear term ${\mathit{\beta }}^T\mathbf{f}(t_n)$ helps control for global trends resulting from extrinsic events such as mass quarantine or travel restrictions. Under the Brownian process, the latent value of a child node ν_c in tree $\mathcal{G}$ is normally distributed about the value of its parent node ${\nu }_{\text{pa}(c)}$ with variance $w_c\times {\sigma }^2$, where ${\sigma }^2$ gives the dispersal rate after controlling for correlation in values that are shared by descent through the phylogenetic tree $\mathcal{G}$. We further posit that the latent value of the root node ${\nu }_{m_{2M-1}}$ is a priori normally distributed with mean 0 and variance ${\tau }_0\times {\sigma }^2$. The vector $\mathbf{z}$ is then multivariate normally distributed Cybis et al. (2015) and has probability density function:

$$p(\mathbf{z}\hspace{0.17em}|\hspace{0.17em}{\mathbf{V}}_{\mathcal{G}},{\sigma }^2,{\tau }_0)={\left(2\pi {\sigma }^2\right)}^{-M/2}{\left|{\mathbf{V}}_{\mathcal{G}}\right|}^{-1/2}\hspace{0.17em}\text{exp}\left(-\frac{1}{2{\sigma }^2}{\mathbf{z}}^T{\mathbf{V}}_{\mathcal{G}}^{-1}\mathbf{z}\right)\hspace{0.17em},$$ (2)

where ${\mathbf{V}}_{\mathcal{G}}=\left\{v_{nm}\right\}$ is a symmetric, positive definite, block-diagonal M × M matrix with structure dictated by $\mathcal{G}$. Defining $d_F(u,v)$ to be the sum of edge-weights along the shortest path between nodes u and v in tree $\mathcal{G}$, the diagonal elements $v_{mm}={\tau }_0+d_F({\nu }_{m_{2M-1}},{\nu }_{m_m})$ are the elapsed evolutionary time between the root node and tip node m_m, and off-diagonal elements $v_{nm}={\tau }_0+[d_F({\nu }_{m_{2M-1}},{\nu }_{m_n})+d_F({\nu }_{m_{2M-1}},{\nu }_{m_m})-d_F({\nu }_{m_n},{\nu }_{m_m})]/2$ are the evolutionary time period between the root node and the most recent common ancestor of tip nodes m_n and m_m.

2.3 Inference

Due to the complexity of the phylogenetic Hawkes process and the large number of viruses we seek to model, we must use advanced statistical, algorithmic and computational tools to infer the posterior distribution:

$$p({\sigma }^2,\mathcal{G},\varphi ,{\mu }_0,{\tau }_x,{\theta }_0,\mathit{\theta },\omega ,h,\mathit{\beta }\hspace{0.17em}|\hspace{0.17em}\mathbf{X},\mathbf{t},\mathbf{S})\propto \mathcal{L}(\mathbf{X},\mathbf{t}|{\mu }_0,{\tau }_x,{\theta }_0,\mathit{\theta }(\mathbf{z}),\omega ,h)\times p(\mathbf{z}\hspace{0.17em}|\hspace{0.17em}{\sigma }^2,\mathcal{G})\times p({\mu }_0)\times p({\tau }_x)\times p({\theta }_0)\times p(\omega )\times p(h)\times p(\mathit{\beta })\times p({\sigma }^2)\times p(\mathbf{S},\varphi ,\mathcal{G})\hspace{0.17em}.$$ (3)

We do so using a random-scan Metropolis-with-Gibbs scheme, in which we compute key quantities with the help of adaptively preconditioned HMC (Neal, 2011), dynamic programming and parallel computing on cutting-edge GPUs.

2.3.1 Dynamic programming for phylogenetic diffusion quantities

We must evaluate $p(\mathbf{z}\hspace{0.17em}|\hspace{0.17em}{\sigma }^2,\mathcal{G})$ to sample $\mathcal{G}$. The bottleneck within the evaluation of Equation (2) is the ostensibly $\mathcal{O}(M^3)$ matrix inverse ${\mathbf{V}}_{\mathcal{G}}^{-1}$, but Pybus et al. (2012) develop a dynamic programming algorithm to perform the requisite computations in $\mathcal{O}(M)$ with parallelized post-order traversals of $\mathcal{G}$. We use this algorithm, which is closely related to the linear-time algorithms of Freckleton (2012) and Ho and Ané (2014), as all are examples of message passing on a directed, acyclic graph (Cavalli-Sforza and Edwards, 1967; Pearl, 1982). Similar tricks render inference for $\varphi$ linear in M, and Fisher et al. (2021) extend Pybus et al. (2012) to compute gradients with respect to $\varphi$. Finally, implementing these algorithms on GPUs would lead to additional speedups (Suchard and Rambaut, 2009), but the computational bottleneck we face when applying the phylogenetic Hawkes process arises from the Hawkes process likelihood and its gradients.

2.3.2 Massive parallelization for Hawkes model quantities

Sampling the Hawkes process parameters ${\mu }_0,{\tau }_x,{\theta }_0,\omega ,h,\beta$ and event-specific rates $\mathit{\theta }$ requires evaluation of the likelihood $\ell (\mathbf{X},\mathbf{t}|{\mu }_0,{\tau }_x,{\theta }_0,\mathit{\theta },\omega ,h)$ or its logarithm. Unfortunately, the double summation of Equation (1) results in an $\mathcal{O}(N^2)$ computational complexity that makes repeated likelihood evaluations all but impossible for the number of observations considered in this paper. We therefore use the high-performance computing framework of Holbrook et al. (2021b) to massively parallelize likelihood evaluations in the context of univariate, adaptive Metropolis-Hastings proposals for parameters ${\mu }_0,{\tau }_x,{\theta }_0,\omega$, h and β. On the other hand, inference for the M-vector $\mathbf{z}$ requires more than fast univariate proposals, so we opt for HMC to sample from its high-dimensional posterior. Even in high dimensions, HMC efficiently generates proposal states by simulating a physical Hamiltonian system that renders the target posterior distribution invariant. Here, we follow standard procedure and specify the system with total energy:

$$H(\mathbf{z},\mathbf{p})=-\text{log}\hspace{0.17em}(\pi (\mathbf{z})\hspace{0.17em}\xi (\mathbf{p}|\mathbf{M}))\propto -\text{log}\hspace{0.17em}\pi (\mathbf{z})+\frac{1}{2}{\mathbf{p}}^T{\mathbf{M}}^{-1}\mathbf{p}\hspace{0.17em},$$

where $\pi (\mathbf{z})$ is the density of the marginal posterior for $\mathbf{z},\hspace{0.17em}\mathbf{p}$ is a Gaussian distributed ‘momentum’ variable with density $\xi (\mathbf{p}|\mathbf{M})$, and $\mathbf{M}$ is the system mass matrix and the covariance of $\mathbf{p}$. Within this Newtonian framework, simulating from the posterior distribution $\pi (·)$ is analogous to simulating Hamiltonian dynamics that satisfy the equations:

$$\stackrel{.}{z}=\frac{\partial }{\partial \mathbf{p}}H(\mathbf{z},\mathbf{p})=\frac{1}{2}{\mathbf{M}}^{-1}\mathbf{p}\hspace{0.17em},\hspace{1em}\stackrel{.}{p}=-\frac{\partial }{\partial \mathbf{z}}H(\mathbf{z},\mathbf{p})=\nabla \hspace{0.17em}\text{log}\hspace{0.17em}\pi (\mathbf{z})\hspace{0.17em}.$$

Here, we interpret $\mathbf{z}$ as the position of a physical object with velocity ż proportional to ${\mathbf{M}}^{-1}\mathbf{p}$, the momentum divided by mass. In the same way, the instantaneous change in momentum &pdot; is proportional to the gradient of the potential energy (or the negative log-posterior) with respect to the ‘position’ $\mathbf{z}$. But, there is no free lunch: simulation of the physical system requires repeated evaluations of the log-likelihood gradient, and these evaluations may become burdensome in big data contexts. We again follow standard HMC procedure and use the leapfrog algorithm (Leimkuhler and Reich, 2004) to integrate Hamilton’s equations. Setting $\mathrm{ϵ}>0$ small, at any time step s within the numerical discretization scheme we update position and momentum according to the following rules:

$$\begin{array}{cc}\mathbf{p}(s+\mathrm{ϵ}/2)\hfill & =\mathbf{p}(s)+\frac{\mathrm{ϵ}}{2}\nabla \hspace{0.17em}\text{log}\hspace{0.17em}\pi (\mathbf{z}(s))\hfill \\ \hfill \mathbf{z}(s+\mathrm{ϵ})& =\mathbf{z}(s)+\mathrm{ϵ}\hspace{0.17em}{\mathbf{M}}^{-1}\mathbf{p}(s+\mathrm{ϵ}/2)\hfill \\ \hfill \mathbf{p}(s+\mathrm{ϵ})& =\mathbf{p}(s+\mathrm{ϵ})+\frac{\mathrm{ϵ}}{2}\nabla \hspace{0.17em}\text{log}\hspace{0.17em}\pi (\mathbf{z}(s+\mathrm{ϵ}))\hspace{0.17em}.\hfill \end{array}$$

In practice, the step size ϵ is a crucially important tuning parameter, but we use standard adaptation techniques (Rosenthal, 2011) to avoid tedious hand-tuning. Unfortunately, the gradient of the Hawkes process log-likelihood of Equation (1) with respect to $\mathit{\theta }$—a key term when calculating $\nabla \hspace{0.17em}\text{log}\hspace{0.17em}\pi (\mathbf{z})$—becomes computationally onerous for large N and M. Recalling that Λ_m is the additive contribution of the mth event to the likelihood’s integration term and that λ_nm is the mth event’s additive contribution to the rate function evaluated at the nth event, the gradient with respect to a single event-specific rate θ_m takes the form:

$$\begin{array}{c}\frac{\partial \ell }{\partial {\theta }_m}=\frac{\partial }{\partial {\theta }_m}\left(-{\Lambda }_m+\sum _{n=1}^N\hspace{0.17em}\text{log}\hspace{0.17em}{\lambda }_n\right)=-\frac{\partial {\Lambda }_m}{\partial {\theta }_m}+\sum _{t_m<t_n}\frac{1}{{\lambda }_n}\frac{\partial {\lambda }_{nm}}{\partial {\theta }_m}\\ ={\theta }_0\left(e^{-\omega (t_N-t_m)}-1\right)+\sum _{t_m<t_n}\frac{1}{{\lambda }_n}\frac{{\theta }_0\omega }{h^D}{e}^{-\omega (t_n-t_m)}\varphi \left(\frac{{\mathbf{x}}_n-{\mathbf{x}}_m}{h}\right)\hspace{0.17em}.\end{array}$$ (4)

Given all observations and model parameters, one may compute the gradient using this formula, but the summation and λ_n are both of complexity $\mathcal{O}(N)$. Thus, computing the entire vector $\partial \ell /\partial \mathit{\theta }={(\partial \ell /\partial {\theta }_1,\dots ,\partial \ell /\partial {\theta }_M)}^T$ requires time $\mathcal{O}(NM)$. Worse still, due to the multiscale nature of the posterior for the relative rates (Fig. 4), we find it necessary to precondition the Hamiltonian dynamics by specifying a diagonal mass matrix with elements:

$${\mathbf{M}}_{mm}^{-1}\approx -\frac{{\partial }^2\ell }{\partial {\theta }_m^2}=\sum _{t_m<t_n}\frac{1}{{\lambda }_n^2}\frac{{\theta }_0^{\mathrm{2}}{\omega }^2}{h^{2D}}{e}^{-2\omega (t_n-t_m)}{\varphi }^2\left(\frac{{\mathbf{x}}_n-{\mathbf{x}}_m}{h}\right)\hspace{0.17em}.$$ (5)

Fig. 4.

The posterior distribution presents multiple challenges: it is high-dimensional; it takes on different scales for different parameters; it is multimodal in some parameters; and it exhibits complex correlation structures between parameters. (Left) Histogram and quartiles from 100 million MCMC samples for the ESS of all 1377 model parameters. (Middle) Multimodal marginal posterior for a single relative rate. (Right) Posterior correlations between 21 relative rates

Specifically, we maintain a running average of Hessians calculated at a fixed interval and use this as our preconditioner $\mathbf{M}$, thus maintaining asymptotic unbiasedness of Monte Carlo estimates (Haario et al., 2001). Just as with the gradient, the summation and λ_n are both of complexity $\mathcal{O}(N)$, and the resulting complexity for the entire Hessian is $\mathcal{O}(NM)$. To overcome these rate-limiting steps, we develop massively parallel central processing unit (CPU) and GPU implementations of both the gradient and the Hessian. In Section B, Algorithms 1 and 2 detail both parallel implementations of the gradient. Although our GPU-based implementations are fastest (Section 3.1), our CPU implementations are competitive, making use of both multi-core processing and SIMD (single instruction, multiple data) vectorization (Holbrook et al., 2021a). Regardless of implementation, all of our high-performance software remains freely available for public use.

2.4 Software availability

We use the Bayesian evolutionary analysis by sampling trees (BEAST) software package (Suchard et al., 2018), a popular tool for viral phylogenetic inference that implements Markov chain Monte Carlo (MCMC) methods to explore $p(\mathbf{S},\varphi ,\mathcal{G})$ and $p(\mathbf{z}\hspace{0.17em}|\hspace{0.17em}{\sigma }^2,\mathcal{G})$ (Cybis et al., 2015) under a range of evolutionary models. In writing this article, we have contributed to the open-source, stand-alone library hpHawkes http://github.com/suchard-group/hawkes for computing the spatiotemporal Hawkes process log-likelihood (Equation 1), its gradient (Equation 4), and its Hessian (Equation 5). hpHawkes integrates into BEAST with the help of an application programming interface. Within hpHawkes, we combine C++ code with which standard compilers generate vectorized CPU-specific instructions and OpenCL kernels that allow for GPU-specific optimization. Finally, we have used the Rcpp package (Eddelbuettel and François, 2011) to make the same massive parallelization speedups available to users of the R programming language.

3 Demonstration

3.1 Massive parallelization

Figure 2 shows benchmarking results for evaluating the Hawkes log-likelihood gradient with respect to event-specific rates $\mathit{\theta }$ (Equation 4). For the GPU results, we use an NVIDIA Quadro GV100, which has 5120 CUDA cores (at 1.13 GHz) and reaches an (unboosted) 2.9 teraflops peak double-precision floating-point performance (or 5.8 teraflops for fused operations such as fused multiply-add). We use a Linux machine with two 26-core Intel Xeon Gold processors (2.1 GHz) for CPU results. Each physical core supports two threads or logical cores, and the machine achieves a peak performance of 874 gigaflops with double-precision floating point enhanced with advanced vector extensions (AVX) vectorization (again, double this for fused operations). Based on peak double-precision floating-point operations, our a priori expectation is for fully parallelized GPU-based gradient evaluations to be roughly 3.3 times faster than 104-threaded AVX evaluations on the CPU.

Fig. 2.

Spatiotemporal Hawkes process log-likelihood gradient and Hessian calculations with respect to event-specific rates $\mathit{\theta }$ with CPUs and GPUs. (Left) Multiplicative speedups over single-threaded AVX vectorization for single-threaded non-vectorized and SSE, multi-threaded AVX and many-core GPU processing for 25 000 randomly generated data points. (Right) Seconds per gradient and Hessian calculations for multi-threaded AVX and GPU implementations from 10 000 to 90 000 data points

On the left of Figure 2, we compare relative efficiency for GPU and various CPU implementations of the log-likelihood gradient and Hessian for 25000 simulated data points using single-threaded AVX computing (15.7 and 16.3 s per gradient and Hessian evaluations) as baseline. Using streaming SIMD extension (SSE) or non-vectorized single-threaded computing results in 1.5- and 2.2-fold slowdowns for the gradient and 1.3- and 2.1-fold slowdowns for the Hessian. Sticking with AVX processing, we see diminishing returns as we increase the number of threads. For both the gradient and the Hessian, the 14-, 54- and 104-thread AVX implementations are roughly 13, 33 and 41 times faster than single-threaded AVX. Agreeing with our a priori expectations, the GPU implementation is 140.4 times faster than single-threaded AVX and 3.5 times faster than 104-threaded AVX for the gradient and 120.0 times faster than single-threaded AVX and 2.9 times faster than 104-threaded AVX for the Hessian. The right of Figure 2 demonstrates the $\mathcal{O}(N^2)$ computational complexity for the same gradient and Hessian evaluations by varying the number of data points from 10000 to 90000. Although parallelization does not overcome this quadratic scaling, it does reduce computational costs for finite observation counts.

3.2 2014–2016 Ebola outbreak in West Africa

During the 2014–2016 outbreak in Guinea, Sierra Leone and Liberia, Ebola viral fever resulted in over 28 000 known cases and 11000 known deaths (World Health Organization, 2015). First reports of the virus in Guinea emerged during March of 2014 (Baize et al., 2014). At around the same time, viral cases with the same Guinean origin (Gire et al., 2014) emerged in Sierra Leone and Liberia. In May 2014, the virus crossed from Guinea to Kailahun, Sierra Leone. From there, it spread to multiple counties of Liberia and Guinea (Dudas et al., 2017), and the same strain reached Freetown, the capital of Sierra Leone, by July 2014. In the fall of 2014, Sierra Leone and Liberia were detecting 500 and 700 new cases a week. Only by the end of 2014 did case numbers begin to abate in most areas due to control measures. By March of 2015 sustained transmission of the virus only continued in western Guinea and western Sierra Leone (Dudas et al., 2017). Figure 3 shows the spatiotemporal distribution of the majority of known Ebola virus cases during the epidemic. The right-hand side of Figure 3 is a stacked histogram, displaying the relative contribution of sequenced and unsequenced viral cases to the total case count.

Fig. 3.

Spatiotemporal distribution of 23 178 viral cases during the 2014–2016 Ebola outbreak in Guinea, Sierra Leone and Liberia. On the one hand, this number consists of 1367 viral samples that yield RNA sequence data and interface directly with the prior over phylogenetic trees. On the other hand, all 23 178 cases for which spatiotemporal data are available—including 21 811 unsequenced cases—interface with the Hawkes process likelihood. This leaves 243 sequenced cases for which spatiotemporal data are not available that interface with the phylogenetic but not the Hawkes process model

Using our high-performance computing framework, we apply the phylogenetic Hawkes process to the analysis of 23 421 viral cases. Dudas et al. (2017) provide a total of 1610 cases furnishing genomic sequencing, 1367 of which come with date and location data (https://github.com/ebov/space-time). We supplement this sequenced data with 21811 date and location pairs from unsequenced cases documented by the World Health Organization (https://apps.who.int/gho/data/node.ebola-sitrep). The precision of the spatial data is district or county level. To leverage spatial information as much as practically possible within our Hawkes model, we assume the locations follow a Gaussian distribution at district population centroids and with variance guaranteeing a 95% probability of the case occurring within the circle of equal area to the district and centered at the population centroid. We then integrate over uncertainty with respect to these locations by periodically sampling new locations according to the assumed Gaussian distribution throughout the MCMC run and with a period of roughly 100 iterations. That said, sensitivity analyses show that model inference is robust to fixing randomly generated locations for the entire MCMC chain. We make the combined data and documentation for our entire BEAST analysis available within the single file Final.xml and place this as well as other project scripts together at the repository https://github.com/suchard-group/EBOVPhyloHawkes. In addition to the software mentioned in the previous section, we make use of the ggplot2 and ggmap R packages for data and results visualization (Kahle and Wickham, 2013; Wickham, 2016).

For the phylogenetic prior specification $p(\mathbf{S},\varphi ,\mathcal{G})$, we follow the phylogeographic analysis of Dudas et al. (2017) and use a mixture of 1000 phylogenetic trees obtained as high-probability posterior samples from their purely phylogenetic analysis of the 1610 sequenced viral samples. In that preceding Bayesian analysis, Dudas et al. (2017) combine an HKY$+{\Gamma }_4$ substitution model prior for molecular evolution (Hasegawa et al., 1985; Yang, 1994), a relaxed molecular clock prior on rates (Drummond et al., 2006), a non-parametric coalescent ‘Skygrid’ prior on effective population size dynamics (Gill et al., 2013) and a continuous time Markov chain reference prior for overall rate (Ferreira and Suchard, 2008). We assume a priori that the background lengthscale τ_x follows a diffuse inverse gamma distribution with Shape 1 and Scale 10, where distance units are latitudinal and longitudinal degrees. An inverse gamma distribution with shape and scale parameters equal to 2 and 0.5 for both h and $1/\omega$ encodes our beliefs that self-excitatory dynamics occur at finer spatiotemporal scales, where years are the temporal units. We upweight self-excitatory dynamics by giving θ₀ and μ₀ gamma priors with shape parameters 1 and 2 and scale parameters 0.001 and 2, respectively. We absorb τ₀ into σ and place a tight inverse gamma prior on $1/\sigma$ with shape and scale parameters of 2 and 0.5. Finally, we set $\mathbf{f}(t_n)=t_n$ and place a normal prior on the univariate coefficient β with mean 0 and SD of 10. We find all parameters robust to prior specification due to the large number of observations considered.

We generate 100 million MCMC samples according to the routine outlined in Section 2.3 and discard the first 500000 as burn-in. Using our parallel computing algorithms and a single NVIDIA GV100 GPU (Section 3.1), the routine requires 6.77 h to generate 1 million samples and 28 days to generate all 100 million samples. Figure 4 shows the distribution of effective samples sizes (ESS) across all model parameters and illustrates some of the challenges facing any MCMC routine for the phylognetic Hawkes model. Namely, the posterior distribution is high-dimensional, multimodal, multiscale and has complex correlation structures.

Table 2 shows posterior means and 95% highest posterior density (HPD) credible intervals for the phylogenetic Hawkes process parameters. The posterior mean for the spatial bandwidth τ_x of the Hawkes background process is 194 km (147, 243), allowing the model to incorporate and adapt to large scale geographic movement. On the other hand, the Hawkes process self-excitatory spatial bandwidth h has a posterior mean of 7.4 km (7.1, 7.6), indicating the smaller local scale for which the model attributes viral contagion. The self-excitatory temporal bandwidth $1/\omega$ has a posterior mean of 29.8 days (28.5, 30.9), indicating the timescale for which the model attributes the same viral contagion. The normalized self-excitatory weight ${\theta }_0/({\theta }_0+{\mu }_0)$ indicates the proportion of events the model attributes to self-excitatory (compared with background) dynamics and has a posterior mean of 0.96 (0.95, 0.97). The posterior mean of the self-excitatory rate’s temporal trend coefficient β is −2.22 (−2.37, −2.06) indicates that, for every additional year and ceteris paribus, one should expect a multiplicative decrease of $1-\text{exp}\hspace{0.17em}(-2.22)\times 100\approx 90\%$ to the process self-excitatory rate. In this way, the model adjusts for downward trends arising from epidemiological control (e.g. mass quarantine and travel restrictions) and controls for these factors when inferring virus-specific relative rates.

Table 2.

Posterior means and 95% HPD credible intervals from the application of the phylogenetic Hawkes process to the 2014–2016 Ebola outbreak in Guinea, Sierra Leone and Liberia

			Posterior mean
Hierarchical model module	Model parameter	Symbol	(95% HPD credible intervals)	Unit
Hawkes process	Background spatial lengthscale	τx	194 (147, 243)	km
	Self-excitatory temporal lengthscale	$1/\omega$	29.8 (28.5, 30.9)	days
	Self-excitatory spatial lengthscale	h	7.37 (7.13, 7.62)	km
	Normalized self-excitatory weight	${\theta }_0/({\theta }_0+{\mu }_0)$	0.96 (0.95, 0.97)	—
	Temporal trend coefficient	β	−2.22 (−2.37, −2.06)	—
Phylogenetic diffusion	SD	σ	51.0 (46.4, 55.7)	log rate

Next, we consider posterior inference of the virus-specific rates $\mathit{\theta }$ for those viral observations that provide RNA sequences. When interpreting these results, it is important to understand that the phylogenetic Hawkes process implicitly assumes that such samples spark nearby contagion as described by spatial and temporal bandwidths h and $1/\omega$. Recall that the posteriors for these two parameters concentrate at over 7 km and 4 weeks, respectively. Figure 5 depicts the relationship between posterior mean values of $\mathit{\theta }$ and the spatiotemporal distribution of corresponding viruses. Since these rates represent multiplicative factors of the global self-excitatory weight θ₀, a null value would be 1. Posterior means range from approximately −2.5 to 7.5 and increasingly vary as a function of time. As one might expect, the highest rates appear near or within larger clusters. Thanks to the negative temporal trend coefficient β and the increase of uncertainty with time, larger rate values do obtain for some viral cases occurring in 2015, despite following after peak epidemic. Figure 6 features posterior means and 95% intervals for the same virus-specific rates. Only a small subset of 183 rate intervals does not include 1. Of these, 177 have lower bound >1. We interpret all 183 of the corresponding viruses as having statistically significantly increased or decreased contagiousness.

Fig. 5.

Hawkes model posterior mean rates $\mathit{\theta }$ for the 1367 (of 1610) RNA-sequenced viral samples for which date/location data are available. Unsurprisingly, the largest relative rates occur within or nearby major clusters of events. Adjusting for downward trends in case data with a negative coefficient β (Table 2) allows detection of higher relative rates after peak outbreak (late 2014) including a jump in infections mid-2015 (Figure 3)

Fig. 6.

95% credible intervals and posterior means for virus-specific rates $\mathit{\theta }$ corresponding to the 1367 sequenced viruses that come with date/location data and therefore appear in the Hawkes process module. We call those 183 intervals which do not include 1 ‘significant’ and color the 177 intervals that are above 1 red

Finally, Figure 7 shows how these posterior rates organize as a function of the inferred posterior maximum clade credibility tree $\mathcal{G}$. Generally speaking, shorter branch lengths indicate larger effective populations of viruses, whereas larger branch lengths indicate smaller. For example, the structure of the bottom subtree reflects this intuition as branches are short with many splits during peak outbreak in the second half of 2014 but become mostly long in late 2014 and the remainder of 2015. According to the phylogenetic Brownian process model outlined in Section 2.2, virus-specific rate values are more highly correlated to one another when closely located to one-another on the phylogenetic tree. It is plausible that these correlations allow the phylogenetic Hawkes model to infer higher rates for some strains that survive late into the epidemic despite dropping case counts. The model attributes some of the highest values to strains appearing in Coyah, Conakry and Kindia, Guinea, in late 2014 and early 2015. Interestingly, the model also attributes its lowest values to cases in Kono, Sierra Leone, in early 2015. Taken together, Figures 6 and 7 may provide helpful leads for epidemiologists searching for variants with heightened relative rates of contagion.

Fig. 7.

Posterior mean virus-specific relative rates color the posterior maximum clade credibility tree of phylogeny $\mathcal{G}$, a few subtrees of which potentially demonstrate elevated contagiousness

4 Discussion

We propose the phylogenetic Hawkes process, a Bayesian hierarchical model that relates viral spatial contagion to molecular evolution by uniting the two epidemiological paradigms of self-exciting point process and phylogenetic modeling. Due to difficulties in scaling the model to larger numbers of observations, we advance a computing strategy that combines HMC, dynamic programming and massive parallelization for key inferential bottlenecks. Finally, we apply our novel model and high-performance computing framework to the analysis of over 23000 viral cases arising from the 2014 to 2016 Ebola outbreak in West Africa, and Ebola strains and subtrees with plausibly higher degrees of contagiousness reveal themselves.

Unfortunately, the current model will fail when applied to the analysis of a global pandemic due to the dominant role of non-local, large-scale transportation networks in propagating viral spread (Holbrook et al., 2020, 2021a). We are particularly interested in developing extensions to the phylogenetic Hawkes process that leverage recent advances in scaling high-dimensional multivariate Hawkes processes (Nickel and Le, 2020) and applying the resulting multivariate phylogenetic Hawkes process to the analysis of global pandemics. In this context, each additional dimension will represent an additional country or province. Prodigious computational challenges are inevitable, and we suspect that non-trivial GPU implementations will be necessary for big data applications.

Moving beyond inference, a major question is whether the phylogenetic Hawkes process can be useful for prediction of spatial contagion and dynamics. Here, recent neural network extensions of the Hawkes process might prove useful (Mei and Eisner, 2017; Zhang et al., 2020; Zuo et al., 2020), but it is unclear what forward simulation of phylogenetic branching dynamics would look like in the context of a Hawkes process. Moreover, generating samples from the posterior predictive distribution of a Hawkes process would be extremely time consuming when one is conditioning on millions of posterior samples. To work around this, one could perhaps parallelize over fixed parameter settings the algorithm of Dassios and Zhao (2011) for simulating Hawkes processes when the temporal triggering function is exponential. Such an implementation would require efficient use of parallel pseudo-random number generators (Salmon et al., 2011).

More broadly, the phylogenetic Hawkes process is a single contribution to the immense scientific project surrounding surveillance of viral populations, inference of viral evolutionary histories and prediction of viral spread. Due to the inherent difficulty of this challenge, we imagine our model will be most useful when used in conjunction with other epidemiological tools, mathematical models and expert intensive lab work. That said, we believe the phylogenetic Hawkes process represents a major step toward automated, holistic and data-centered epidemiological modeling insofar as it fully leverages genomic and spatiotemporal data obtained from both sequenced and unsequenced viral cases. This joint modeling approach stands in contrast to other phylogenetic modeling approaches that also attempt to recover phylogenetically localized measures of contagiousness but do not leverage unsequenced case data. For example, Łuksza et al. (2014) combine summary statistics of clade growth with birth–death models to infer clade-specific reproduction numbers but analyze only 81 sequenced cases and no unsequenced cases from the early 2014 Ebola outbreak. On the one hand, the birth–death model allows one to obtain readily interpretable clade-specific reproduction numbers. On the other hand, the phylogenetic Hawkes model allows rates to vary continuously between and within clades. Similarly, Stadler et al. (2014) and Volz and Pond (2014) apply phylogenetic compartmental models to analyze <80 sequenced cases and no unsequenced cases. The fact that these analyses (Łuksza et al., 2014; Stadler et al., 2014; Volz and Pond, 2014) do not take spatial information into account makes it difficult to compare certain results with those from the phylogenetic Hawkes process analysis. For temporal measures, however, we do infer a self-excitatory lengthscale $1/\omega$ with 95% HPD credible interval of (28.5, 30.9) days, which is marginally consistent with Stadler et al.’s (2014) 95% HPD credible intervals of (2.11, 23.20) days for the incubation period and (1.24, 6.98) days for the infectious period.

These comparisons raise a broader theoretical question: can the phylogenetic Hawkes process provide phylogenetically localized basic or effective reproduction numbers and, as a result, become more readily interpretable in broader science? Rizoiu et al. (2018) show that the rate of events in an epidemiological SIR model is equal to that of their finite-population Hawkes model. Accordingly, crucial next steps in the development of the phylogenetic Hawkes process framework are the incorporation of finite-population dynamics as well as the extension of the temporal model from Rizoiu et al. (2018) to the spatiotemporal regime.

Appendix A

A1 The likelihood integral with event-specific rates

Without loss of generality, we consider the temporal Hawkes process with constant background rate. To compute the likelihood (Equation 1), we must calculate the integral:

$$\begin{array}{cc}\Lambda (t_N)\hfill & ={\int }_0^{t_N}\lambda (t)\hspace{0.17em}d\mathrm{t}={\int }_0^{{\mathrm{t}}_{\mathrm{N}}}\left(\mu +{\theta }_0\sum _{{\mathrm{t}}_{\mathrm{n}}<\mathrm{t}}{\theta }_{\mathrm{n}}\omega {\mathrm{e}}^{-\omega ({\mathrm{t}-\mathrm{t}}_{\mathrm{n}})}\right)\text{dt}\hfill \\ \hfill & ={\int }_0^{t_1}\mu \hspace{0.17em}\mathrm{d}t+\sum _{n=1}^{N-1}{\int }_{t_n}^{t_{n+1}}\left(\mu +{\theta }_0\sum _{t_n<t}{\theta }_n\omega e^{-\omega (t-t_n)}\right)\mathrm{d}t\hfill \\ \hfill & =\mu \hspace{0.17em}{t}_N+{\theta }_0\omega \sum _{n=1}^{N-1}{\int }_{t_n}^{t_{n+1}}\sum _{n\prime =1}^n{\theta }_{n\prime }{e}^{-\omega (t-t_{n\prime })}\hspace{0.17em}\mathrm{d}t\hfill \\ \hfill & =\mu \hspace{0.17em}{t}_N+{\theta }_0\omega \sum _{n=1}^{N-1}\sum _{n\prime =1}^n{\theta }_{n\prime }{\int }_{t_n}^{t_{n+1}}{e}^{-\omega (t-t_{n\prime })}\hspace{0.17em}\mathrm{d}t\hfill \\ \hfill & =\mu \hspace{0.17em}{t}_N-{\theta }_0\sum _{n=1}^{N-1}\sum _{n\prime =1}^n{\theta }_{n\prime }(e^{-\omega (t_{n+1}-t_{n\prime })}-e^{-\omega (t_n-t_{n\prime })})\hspace{0.17em},\hfill \end{array}$$

and we further simplify the double summation in the following.

Claim 1. The temporal Hawkes process with rate function:  

$$\lambda (t)=\mu +{\theta }_0\sum _{t_n<t}{\theta }_n\omega e^{-\omega (t-t_n)}$$ (6)

admits the integral  

$$\Lambda (t_N)={\int }_0^{t_N}\lambda (t)\hspace{0.17em}d\mathrm{t}=\mu \hspace{0.17em}{\mathrm{t}}_{\mathrm{N}}-{\theta }_0\sum _{\mathrm{n}=1}^{\mathrm{N}-1}{\theta }_{\mathrm{n}}({\mathrm{e}}^{-\omega ({\mathrm{t}}_{\mathrm{N}}{-\mathrm{t}}_{\mathrm{n}})}-1)\hspace{0.17em}.$$ (7)

Proof. Proceeding by induction, the assertion is trivial for N = 1. If it is true for some N > 0, this implies that:

$$\sum _{n=1}^{N-1}\sum _{n\prime =1}^n{\theta }_{n\prime }(e^{-\omega (t_{n+1}-t_{n\prime })}-e^{-\omega (t_n-t_{n\prime })})=\sum _{n=1}^{N-1}{\theta }_n(e^{-\omega (t_N-t_n)}-1)\hspace{0.17em}.$$

It follows that:

$$\begin{array}{ll}\Lambda (t_{N+1})\hfill & =\mu t_{N+1}-{\theta }_0{\displaystyle \sum _{n=1}^N{\displaystyle \sum _{n\prime =1}^n{\theta }_{n\prime }}}\left(e^{-\omega (t_{n+1}-t_{n\prime })}-e^{-\omega (t_n-t_{n\prime })}\right)\hfill \\ \hfill & =\mu (t_{N+1}-t_N)-{\theta }_0{\displaystyle \sum _{n\prime =1}^N{\theta }_{n\prime }}\left(e^{-\omega (t_{N+1}-t_{n\prime })}-e^{-\omega (t_N-t_{n\prime })}\right)\hfill \\ \hfill & +\mu t_N-{\theta }_0{\displaystyle \sum _{n=1}^{N-1}{\displaystyle \sum _{n\prime =1}^n{\theta }_{n\prime }}}\left(e^{-\omega (t_{n+1}-t_{n\prime })}-e^{-\omega (t_n-t_{n\prime })}\right)\hfill \\ \hfill & =\mu (t_{N+1}-t_N)-{\theta }_0{\displaystyle \sum _{n\prime =1}^N{\theta }_{n\prime }}\left(e^{-\omega (t_{N+1}-t_{n\prime })}-e^{-\omega (t_N-t_{n\prime })}\right)+\Lambda (t_n)\hfill \\ \hfill & =\mu (t_{N+1}-t_N)-{\theta }_0{\displaystyle \sum _{n\prime =1}^N{\theta }_{n\prime }}\left(e^{-\omega (t_{N+1}-t_{n\prime })}-e^{-\omega (t_N-t_{n\prime })}\right)\hfill \\ \hfill & +\mu t_N-{\theta }_0{\displaystyle \sum _{n=1}^{N-1}{\theta }_n}\left(e^{-\omega (t_N-t_n)}-1\right)\hfill \\ \hfill & =\mu t_{N+1}-{\theta }_0{\displaystyle \sum _{n=1}^N{\theta }_n}\left(e^{-\omega (t_{N+1}-t_n)}-e^{-\omega (t_N-t_n)}\right)\hfill \\ \hfill & -{\theta }_0{\displaystyle \sum _{n=1}^{N-1}{\theta }_n}\left(e^{-\omega (t_N-t_n)}-1\right)\hfill \\ \hfill & =\mu t_{N+1}-{\theta }_0{\displaystyle \sum _{n=1}^N{\theta }_n}\left(e^{-\omega (t_{N+1}-t_n)}-1\right),\hfill \end{array}$$

thus completing the proof.

Appendix B

B1 Parallelized gradient algorithms

Algorithms 1 and 2 present instructions for computing the Hawkes process log-likelihood gradient (Equation 4) with respect to the M-vector $\mathit{\theta }$ of event-specific rates. Figure 2 shows resulting speedups for both Algorithms (1) and (2) on a CPU and GPU, respectively.

Algorithm 1 .

Parallel evaluation of Hawkes process log-likelihood gradient: Makes use of multiple CPU cores and loop vectorization to calculate Hawkes process log-likelihood gradient with respect to event-specific rates $\mathit{\theta }$. When using double-precision floating point, this algorithm may use either SSE or AVX vectorization to make J = 2- or 4-long jumps. We denote the number of CPU cores as B. Symbols $\ell$, λ and Λ appear in Equations (1) and (4).

1: Compute rates ${\lambda }_1,\dots ,{\lambda }_N$:

 a:  parfor  $b\in \{1,\dots ,B\}$  do

 b:   if  $b\ne B$  then

 c:     $\mathit{Upper}\leftarrow b\lfloor N/B\rfloor$

 d:   else

 e:    $\mathit{Upper}\leftarrow \lceil N/B\rceil$

 f:   end if

 g:   for  $n\in \{(b-1)\lfloor N/B\rfloor +1,\dots ,\mathit{Upper}\}$  do

 h:    copy ${\mathbf{x}}_n$, t_n to cache

 i:    $\mathit{\lambda }{n}_n\leftarrow 0$    ▹ vector of length J

 j:    $n\prime \leftarrow 1$

 k:    while  $n\prime <N$  do

 l:     $J\leftarrow \min (J,N-n\prime )$

 m:     copy ${\mathbf{x}}_{n\prime :(n\prime +J)}$, $t_{n\prime :(n\prime +J)}$ to cache

 n:      ${\Delta }_{nn\prime }:{\Delta }_{nn\prime :(n\prime +J-1)}\leftarrow ({\mathbf{x}}_n-{\mathbf{x}}_{n\prime }):({\mathbf{x}}_n-{\mathbf{x}}_{n\prime +J-1})$ ▹ vectorized subtraction

 o:  calculate ${\delta }_{nn\prime }:{\delta }_{n(n\prime +J-1)}$ ▹ vectorized multiplication

 p:  calculate ${\lambda }_{nn\prime }:{\lambda }_{n(n\prime +J-1)}$ ▹ vectorized evaluation

 q:   $\mathit{\lambda }{n}_n\leftarrow \mathit{\lambda }{n}_n+{\lambda }_{nn\prime }:{\lambda }_{n(n\prime +J-1)}$ ▹ vectorized addition

 r:     $n\prime \leftarrow n\prime +J$

 s:    end while

 t:   end for

 u:  end parfor

2: Compute M gradients $\frac{\partial \ell }{\partial {\theta }_n}$:

 a:  parfor  $b\in \{1,\dots ,B\}$  do

 b:   if  $b\ne B$  then

 c:    $\mathit{Upper}\leftarrow b\lfloor M/B\rfloor$

 d:    else

 e:     $\mathit{Upper}\leftarrow \lceil M/B\rceil$

 f:    end if

 g:    for  $n\in \{(b-1)\lfloor M/B\rfloor +1,\dots ,\mathit{Upper}\}$  do

 h:     copy ${\mathbf{x}}_n$, t_n to cache

 i:      $\frac{\partial \ell }{\partial {\theta }_n}\leftarrow 0$

 j:     $n\prime \leftarrow 1$

 k:      while  $n\prime <N$  do

 l:     $J\leftarrow \min (J,N-n\prime )$

 m:     copy ${\mathbf{x}}_{n\prime :(n\prime +J)},\hspace{0.17em}{t}_{n\prime :(n\prime +J)}$ to cache

 n:     ${\Delta }_{nn\prime }:{\Delta }_{nn\prime :(n\prime +J-1)}\leftarrow ({\mathbf{x}}_n-{\mathbf{x}}_{n\prime }):({\mathbf{x}}_n-{\mathbf{x}}_{n\prime +J-1})$    ▹ vectorized subtraction

 o:     calculate ${\delta }_{nn\prime }:{\delta }_{n(n\prime +J-1)}$    ▹ vectorized multiplication

 p:     calculate $e^{-\omega (t_{n\prime }-t_n)}\varphi \left(\frac{{\delta }_{nn\prime }}{h}\right):e^{-\omega (t_{n\prime +J-1}-t_n)}\varphi \left(\frac{{\delta }_{n(n\prime +J-1)}}{h}\right)$    ▹ vectorized evaluation

 q:    for  $j\in n\prime ,\dots ,n\prime +J-1$  do

 r:    $\frac{\partial \ell }{\partial {\theta }_n}\leftarrow \frac{\partial \ell }{\partial {\theta }_n}+{\mathcal{I}}_{[t_n<t_j]}\frac{1}{{\lambda }_j}\frac{\partial {\lambda }_{jn}}{\partial {\theta }_n}$

 s:    end for

 t:    $n\prime \leftarrow n\prime +J$

 u:    end while

 v:    $\frac{\partial \ell }{\partial {\theta }_n}\leftarrow \frac{\partial \ell }{\partial {\theta }_n}+{\theta }_0\left(e^{-\omega (t_N-t_n)}-1\right)$

 w:    end for

 x:   end parfor

Algorithm 2 .

Parallel evaluation of Hawkes process log-likelihood gradient: Computes the log-likelihood gradient with respect to event-specific rates $\mathit{\theta }$ using multiple levels of parallelization on a GPU. In this article, we specify B = 128 for the size of the GPU work groups. Symbols $\ell$, λ and Λ appear in Equations (1) and (4).

1: Compute rates ${\lambda }_1,\dots ,{\lambda }_N$:

a:   parfor  $n\in \{1,\dots ,N\}$  do

b:  copy ${\mathbf{x}}_n$, t_n to local    ▹ B threads

c:   parfor  $N\prime \in \{1,\dots ,\lfloor N/B\rfloor \}$  do

d:    $n\prime \leftarrow N\prime$

e:    ${\lambda }_{nN\prime }\leftarrow 0$

f:    while  $n\prime <N$  do

g:     copy ${\mathbf{x}}_{n\prime },\hspace{0.17em}{t}_{n\prime }$ to local    ▹ B threads

h:     ${\Delta }_{nn\prime }\leftarrow {\mathbf{x}}_n-{\mathbf{x}}_{n\prime }$    ▹ vectorized subtraction

i:     calculate ${\delta }_{nn\prime }=\sqrt{\sum {\Delta }_{nn\prime }°{\Delta }_{nn\prime }}$    ▹ vectorized multiplication

j:     ${\lambda }_{nN\prime }\leftarrow {\lambda }_{nN\prime }+{\lambda }_{nn\prime }$    ▹ ${\lambda }_{nn\prime }$ a function of ${\delta }_{nn\prime }$, t_n and $t_{n\prime }$

k:     $n\prime \leftarrow n\prime +B$

l:    end while

m:   end parfor

n:   ${\lambda }_n\leftarrow \sum _{N\prime }{\lambda }_{nN\prime }$       ▹ binary tree reduction on chip

o:   end parfor

2: Compute M gradients $\frac{\partial \ell }{\partial {\theta }_n}$:

a:   parfor  $n\in \{1,\dots ,M\}$  do

b:   copy ${\mathbf{x}}_n$, t_n to local    ▹ B threads

c:   parfor  $N\prime \in \{1,\dots ,\lfloor N/B\rfloor \}$  do

d:   $n\prime \leftarrow N\prime$

e:   ${\left(\frac{\partial \ell }{\partial {\theta }_n}\right)}_{N\prime }\leftarrow 0$

f:   while  $n\prime <N$  do

g:    copy ${\mathbf{x}}_{n\prime },\hspace{0.17em}{t}_{n\prime }$ to local    ▹ B threads

h:    ${\Delta }_{nn\prime }\leftarrow {\mathbf{x}}_n-{\mathbf{x}}_{n\prime }$    ▹ vectorized subtraction

i:    calculate ${\delta }_{nn\prime }=\sqrt{\sum {\Delta }_{nn\prime }°{\Delta }_{nn\prime }}$    ▹ vectorized multiplication

j:     ${\left(\frac{\partial \ell }{\partial {\theta }_n}\right)}_{N\prime }\leftarrow {\left(\frac{\partial \ell }{\partial {\theta }_n}\right)}_{N\prime }+{\mathcal{I}}_{[t_n<t_{n\prime }]}\frac{1}{{\lambda }_{n\prime }}\frac{\partial {\lambda }_{n\prime n}}{\partial {\theta }_n}$

k:     $n\prime \leftarrow n\prime +B$

l:   end while

m:   end parfor

n:    $\frac{\partial \ell }{\partial {\theta }_n}\leftarrow \sum _{N\prime }{\left(\frac{\partial \ell }{\partial {\theta }_n}\right)}_{N\prime }$    ▹ binary tree reduction on chip

o:  $\frac{\partial \ell }{\partial {\theta }_n}\leftarrow \frac{\partial \ell }{\partial {\theta }_n}+{\theta }_0\left(e^{-\omega (t_N-t_n)}-1\right)$

p: end parfor

Appendix C

C1 Data and modules

Figure 8 shows the number of observations that interface with the phylogenetic and Hawkes modules of the phylogenetic Hawkes process model for the Ebola virus analysis of Section 3.2.

Fig. 8.

The modules of the hierarchical model and the distribution of the data that interface with them