Nonparametric Bayesian Segmentation of a Multivariate Inhomogeneous Space-Time Poisson Process

Abstract

A nonparametric Bayesian model is proposed for segmenting time-evolving multivariate spatial point process data. An inhomogeneous Poisson process is assumed, with a logistic stick-breaking process (LSBP) used to encourage piecewise-constant spatial Poisson intensities. The LSBP explicitly favors spatially contiguous segments, and infers the number of segments based on the observed data. The temporal dynamics of the segmentation and of the Poisson intensities are modeled with exponential correlation in time, implemented in the form of a first-order autoregressive model for uniformly sampled discrete data, and via a Gaussian process with an exponential kernel for general temporal sampling. We consider and compare two different inference techniques: a Markov chain Monte Carlo sampler, which has relatively high computational complexity; and an approximate and efficient variational Bayesian analysis. The model is demonstrated with a simulated example and a real example of space-time crime events in Cincinnati, Ohio, USA.

1 Introduction

1.1 Motivating application

Assume access to the locations of various types of crimes occurring in a given city, as a function of time. As a motivating example, in Figure 1(a) data are shown for 3090 crimes (of 17 crime types) in Cincinnati in Jan 2008. Our focus is on obtaining a spatial segmentation, such as that shown in Figure 1(b). In addition to the spatial dependence of point process data, we wish to simultaneously explore time dynamics. For example, in the crime data analysis, the crime intensity in summer may be different statistically from that in winter, and this intensity may change smoothly over seasons; consequently, the spatial segmentation of the city may also vary smoothly over time.

Figure 1.

Crime events and the segmentation of the city. In (a) 3090 crime events are shown as black dots; in (b) each color indexes a segment with associated crime intensities in 17 crime types (see Section 4 for details).

The analysis of time dynamics helps to discover the temporal pattern of the events and to predict the spatial segmentation at an unobserved time instance or in the future. We desire that the analysis provide a simple summary that is useful to police forces and city planners in targeting resources, as well as to researchers in studying crime trends. We would like to obtain this space-time segmentation quickly, utilizing data from different types of events, while allowing temporal interpolation and forecasting.

1.2 Summary of proposed model

Consider the data 𝒟 = {s_i, v_it}_{i=1, …, M, t=1, …, T}, where v_it is a d-dimensional vector of the counts of d types of events, occurring in a (small) spatial region Δ(s_i), with the center of the region being s_i ∈ ℝ². In the context of Figure 1, we are interested in d types of crime. The contiguous grid of spatial regions Δ(·) is fixed in advance, and the size of Δ(·) is very small relative to the size of the entire spatial domain, providing justification for an approximation in which we index regions by the center point and assume homogeneity within regions (using the model developed below, in the limit Δ → 0 we have a Poisson process). There are T time points at which data are observed, not necessarily uniformly spaced in time. Although not done here, one may envision aligning the grid Δ(·) with the geometry of the terrain (e.g., roads).

The proposed space-time model may be summarized as

$${\mathit{v}}_{\mathit{\text{it}}}~{\displaystyle \prod _{j=1}^d}\text{Poisson}({\lambda }_{\mathit{\text{ijt}}}),{\mathit{\lambda }}_{\mathit{\text{it}}}~{\displaystyle \sum _{k=1}^K}{w}_k({\mathit{s}}_i;{\mathit{\theta }}_{\mathit{\text{kt}}}){\delta }_{{\mathit{\lambda }}_{\mathit{\text{kt}}}^{*}}$$ (1)

where w_k(s_i; θ_kt) ≥ 0, ${\displaystyle {\sum }_{k=1}^K}{w}_k({\mathit{s}}_i;{\mathit{\theta }}_{\mathit{\text{kt}}})=1$ for all s_i, ${\delta }_{{\mathit{\lambda }}_{\mathit{\text{kt}}}^{*}}$ is a unit measure concentrated at ${\mathit{\lambda }}_{\mathit{\text{kt}}}^{*}$, and λ_ijt is the jth component of λ_it. This corresponds to a mixture model, with space-time varying mixture weights w_k(s_i; θ_kt) and time-varying atoms ${\delta }_{{\mathit{\lambda }}_{\mathit{\text{kt}}}^{*}}$.

Expression w_k(s; θ_kt) represents a general parametric function capable of modeling the probability of cluster k at spatial location s. In the details of the proposed model, one of the {w_k(s; θ_kt)}_k=1,K is likely to be dominant (large probability) over a contiguous region, yielding a segmentation. Since the parameters θ_kt change in general with time t, a probabilistic space-time segmentation is manifested. Within the proposed model, the prior encourages that {θ_kt} and ${\mathit{\lambda }}_{\mathit{\text{kt}}}^{*}$ vary smoothly as a function of time, and hence the model imposes smooth space-time variation in the shape/form of the segments, and smooth temporal variation of the Poisson rates associated with a given segment.

Two methods are considered for imposing temporal smoothness, representing two perspectives on imposing the same temporal structure. For discrete-time data with uniform temporal spacing, it is natural to consider the first-order autoregressive model, i.e., AR(1), as ${\theta }_{\mathit{\text{kpt}}}~𝒩(\zeta {\theta }_{\mathit{\text{kp}}(t-1)},{\alpha }_0^{-1})$, with θ_kpt the pth component of θ_kt, ζ the AR(1) coefficient (with |ζ| < 1), and α₀ a precision to be inferred (ζ and α₀ could also be extended to depend on k and p). The log of each component of ${\mathit{\lambda }}_{\mathit{\text{kt}}}^{*}$ may be similarly modeled.

We also consider a Gaussian process (GP) model (Rasmussen and Willams 2006) in time for each component θ_kpt, and for the log of each component of ${\mathit{\lambda }}_{\mathit{\text{kt}}}^{*}$, thus allowing non-uniform temporal sampling. To make the AR(1) and GP models consistent, we assume an exponential model for the GP covariance between times t_i and t_l, c₀$c_1^{|t_i-t_l|}$, with c₁ playing a role analogous to ζ in the AR(1) model, and the variance c₀ corresponding to [(1 − ζ²)α₀]⁻¹ from the AR(1) model. The AR(1) and chosen GP representations are therefore essentially different means of imposing the same temporal prior, with the former restricted to uniform temporal sampling.

In addition to developing a new model for multivariate inhomogeneous space-time Poisson process data, a contribution of this paper concerns computations, in the form of a detailed comparison of Markov chain Monte Carlo (MCMC) and variational Bayesian (VB) inference for this class of models. The former is widely used, but it can be computationally prohibitive for the motivating large-scale problems considered here. Computations based on VB are attractive for large-scale modeling studies, but many simplifying assumptions must be made.

1.3 Related research

A natural model for exploiting spatial information, and to model point process data, is the inhomogeneous Poisson process (Diggle 2003),(Møller and Waagepetersen 2004). Researchers have recently studied nonparametric Bayesian approaches for such applications. One of these approaches models the Poisson intensity function by a variation of a Gaussian process (GP) (Adams et al. 2009; Rathbun and Cressie 1994; Møller et al. 1998). The log-Gaussian Cox process (Møller et al. 1998), corresponding to an intensity function modeled as an exponentiated GP, has proven highly successful in point process (Hossain and Lawson 2009) and geostatistical modeling (Diggle et al. 2010; Pati et al. 2010). Mixture models provide another approach to representing the Poisson intensity function (Wolpert and Ickstadt 1998). Kottas and Sansó (2007) proposed a Dirichlet process (DP) mixture model of bivariate beta densities to model heterogeneity in intensity functions. Dirichlet process mixture models of multivariate normal densities can be also found in (Ji et al. 2009; Chakraborty and Gelfand 2010).

In Taddy (2008, 2010); Taddy and Kottas (2012) a dynamic model was proposed for Poisson point processes, based on a novel version of the dependent Dirichlet process. Models of this type have been applied to the data considered in Figure 1, although the problem of segmentation was not considered. In Achcar et al. (2011) a time inhomogeneous Poisson model was proposed, with change-points to estimate the number of times that a given environmental standard is violated in a time interval of interest.

Rather than modeling the Poisson intensity via a GP or a DP mixture model, the model in (1) constitutes a mixture model with space-time mixture weights, and the spatial locations {s_i} of the grid are modeled as covariates. The details of how w_k(s; θ_kt) is modeled encourages contiguous regions in space and time for which a single component (cluster) dominates, encouraging a piecewise-constant Poisson intensity function. In Heikkinen and Arjas (1998) the authors similarly build a piecewise constant prior model for spatial Poisson intensities, using Voronoi tessellations. We model w_k(s; θ_kt) via an extension of the logistic stick-breaking process (LSBP) (Ren et al. 2011). The region of interest is partitioned into a set of contiguous small square cells, with related ideas considered in Hossain and Lawson (2009). Within the context of the aforementioned GP construction for the temporal dependence of θ_kt, related ideas were presented in the context of factor analysis (Luttinen and Ilin 2009), where GPs were used to describe the smoothness of both spatial locations and time. An AR model for temporal dynamics was considered in Taddy (2008, 2010).

2 Model Details

2.1 Basic construction

The proposed space-time model for data 𝒟 = {s_i, v_it}_{i=1, …, M, t=1, …, T} is summarized as

$${\mathit{v}}_{\mathit{\text{it}}}~{\displaystyle \prod _{j=1}^d}\text{Poisson}({\lambda }_{\mathit{\text{ijt}}}),{\mathit{\lambda }}_{\mathit{\text{it}}}~{\displaystyle \sum _{k=1}^K}{w}_k({\mathit{s}}_{\mathit{\text{it}}}){\delta }_{{\mathit{\lambda }}_{\mathit{\text{kt}}}^{*}}$$ (2)

$$w_k({\mathit{s}}_{\mathit{\text{it}}})=p_k({\mathit{s}}_{\mathit{\text{it}}}){\displaystyle \prod _{h=1}^{k-1}}[1-p_h({\mathit{s}}_{\mathit{\text{it}}})]$$ (3)

$$p_k({\mathit{s}}_{\mathit{\text{it}}})=\sigma (g_k({\mathit{s}}_{\mathit{\text{it}}})),\text{ for }k=1,\dots ,K-1,p_K({\mathit{s}}_{\mathit{\text{it}}})=1$$ (4)

$$g_k({\mathit{s}}_{\mathit{\text{it}}})={\displaystyle \sum _{j=1}}{\beta }_{\mathit{\text{kjt}}}𝒦({\mathit{s}}_{\mathit{\text{it}}},{\mathit{s̃}}_j;{\psi }_k)+{\beta }_{k0t}$$ (5)

where (2) is repeated here from (1), for convenience. Below we explain and motivate each term in this construction. Parameters θ_kt from the Introduction correspond here to {β_kjt}j=0,J and ψ_k. In what follows, the notation s_it is meant to assign statistics to spatial location s_i at time t; for example, w_k(s_it) is the kth mixture weight as observed at s_i and time t. The spatial grid defining the regions {Δ(s_i)}_i=1,M is not changing with time.

The expression in (3), with p_k(s_it) ∈ [0, 1] for all s_it, is suggestive of the stick-breaking representation of the Dirichlet process (Sethuraman 1994). The function σ(x) = exp(x)/(1 + exp(x)) is associated with a logistic model, and p_K(s_it) = 1 such that ${\displaystyle {\sum }_{k=1}^K}{w}_k({\mathit{s}}_{\mathit{\text{it}}})=1$ for all s_it. By the construction of g_k(s_it) in (5), the probabilities p_k(s_it) have space-time variation, with such variation transferred to the mixture weights w_k(s_it) via (3). Therefore, via mixture weights w_k(s_it) in (2) we constitute a multivariate Poisson mixture model, with weights that vary as a function of s_it.

Function 𝒦(s, s̃j ; ψ_k) denotes a kernel with parameter ψ_k. Here we employ the radial basis function $𝒦(\mathit{s},{\mathit{s̃}}_j;{\psi }_k)=\text{exp}(-{\Vert \mathit{s}-{\mathit{s̃}}_j\Vert }_2^2/{\psi }_k)$, with J predefined kernel centers {s̃j}_j=1,J; for convenience these J centers are here aligned with the centers of the spatial grid defined by Δ(s̃j) (recall discussion in the Introduction). The appropriate kernel parameters {ψ_k} will be inferred. To ease computations, we assume a discrete set of parameters $\{{\psi }_1^{*},\dots ,{\psi }_L^{*}\}$ over which a uniform prior is placed; each kernel parameter ψ_k is assumed drawn from this finite library of parameters.

The space-time dependence of the model is manifested in how {β_kjt}_j=0,J and $\{{\mathit{\lambda }}_{\mathit{\text{kt}}}^{*}\}$ are modeled.

2.2 Temporal modeling

When the data are sampled uniformly in time, an autoregressive (AR) temporal model is natural. Specifically, we consider

$${\beta }_{\mathit{\text{kjt}}}~𝒩(\zeta {\beta }_{\mathit{\text{kj}}(t-1)},{\alpha }_{\beta }^{-1}),j=0,\dots ,J$$ (6)

$$\text{log }{\lambda }_{\mathit{\text{kjt}}}^{*}~𝒩(\xi \text{ log }{\lambda }_{\mathit{\text{kj}}(t-1)}^{*},{\alpha }_{\lambda }^{-1}),j=1,\dots ,J$$ (7)

with ${\beta }_{\mathit{\text{kj}}0}=\text{log }{\lambda }_{\mathit{\text{kj}}0}^{*}=0$. Gamma priors are placed on α_β and α_λ. Further, ζ and ξ are drawn from a truncated normal 𝒩_(0,1) (0, 1) with 0 < ζ, ξ < 1.

The collection of data may be expensive, and there may be situations for which nonuniform temporal sampling is desired (e.g., to provide fine-scale sampling in particular regions – seasons – of time that may be interesting). This suggests using a Gaussian process (GP) model (Rasmussen and Willams 2006) for the temporal variation of β_kjt and log ${\mathit{\lambda }}_{\mathit{\text{kjt}}}^{*}$.

For the kth mixture component, we let

$${\mathit{B}}_k~𝒩({\mathit{B}}_k|\mathbf{0},{\mathbf{\Omega }}_k)={\displaystyle \prod _{j=0}^J}𝒩({\mathit{\beta }}_{\mathit{\text{kj}}:}|\mathbf{0},{\mathbf{\Sigma }}_{\mathit{\text{kj}}}),{[{\mathbf{\Sigma }}_{\mathit{\text{kj}}}]}_{\mathit{\text{il}}}=c_0{c_1}^{|t_i-t_l|}$$ (8)

where β_kj: = [β_kj1, …, β_kjT]^T, and B_k ∈ ℝ^{T(J + 1)} denotes a vector formed by concatenating β_kj: for j = 0, …, J. The covariance Ω_k is a block-diagonal matrix of size T(J + 1) × T(J + 1), and each block Σ_kj is a T × T covariance matrix; the entry at row i and column l, denoted as [Σ_kj]_il, is evaluated using the GP covariance function with the hyperparameters {c₀, c₁}. A gamma prior is placed on c₀. Since c₁ plays the same role as ζ, we also draw c₁ from the truncated normal 𝒩_(0,1) (0, 1) with 0 < c₁ < 1.

The Gaussian process priors are also placed on log ${\mathit{\lambda }}_{\mathit{\text{kjt}}}^{*}$. For mixture component k

$$\text{log}({\mathit{\lambda }}_{\mathit{\text{kj}}:}^{*})~𝒩(\mathbf{0},{\mathbf{\Gamma }}_{\mathit{\text{kj}}}),{[{\mathbf{\Gamma }}_{\mathit{\text{kj}}}]}_{\mathit{\text{il}}}=d_0{d_1}^{|t_i-t_l|}$$ (9)

where $\text{log}({\mathit{\lambda }}_{\mathit{\text{kj}}:}^{*})={[\text{log}({\lambda }_{\mathit{\text{kj}}1}^{*}),\dots ,\text{log}({\lambda }_{\mathit{\text{kjT}}}^{*})]}^T$, and the covariance matrix Γ_kj ∈ ℝ^T×T, with the entries defined by the GP covariance function with the hyperparameters {d₀, d₁}. A gamma prior and truncated normal prior are placed on d₀ and d₁. As discussed in the Introduction, the considered AR(1) and GP priors are consistent, and provide different modeling strategies for the same imposed temporal dynamics.

2.3 Model interpretation

Equations (3)–(5) are of the form of the logistic stick-breaking process (LSBP) introduced in Ren et al. (2011); however, that paper did not consider Poisson data, and space-time processes were not addressed. Recall that σ(x) ≈ 1 for x > 4; we refer to this as the “clipping” property of the logistic, as all x larger than about 4 contribute effectively in the same manner to σ(x); one may alternatively use a probit model, to achieve the same end. If β_kjt > 4, then p_k(s) ≈ 1 for ${\Vert \mathit{s}-{\mathit{s̃}}_j\Vert }_2^2<{\psi }_k$. This implies via (3) that within region ${\Vert \mathit{s}-{\mathit{s̃}}_j\Vert }_2^2<{\psi }_k$, if β_kjt > 4 mixture component k is highly probable (assuming that other clusters k′ ≠ k do not have large p_k′ (s) in the vicinity of s̃_j). The “clipping” nature of the logistic function, and large values of β_kjt > 4, encourage contiguous regions for which a given cluster k has high space-time probability of being manifested (all locations s at which g_k(s) > 4 have similarly high probability of being associated with cluster k, regardless of the exact value of g_k(s)). The weights {β_kjt} play the role of assigning which regions in space-time are most likely to be associated with a given cluster k, and ψ_k defines the size scale of the cluster. Note that while we truncate the model to K mixture components, this does not mean that all components need actually be used to represent the data. For example, if a given β_k0t is large and negative, then the kth mixture component is unlikely to be utilized at all spatial locations at time t; K is simply an upper bound on the number of mixture components (segment types).

3 Posterior inference

The posterior distribution of the model parameters is inferred via an MCMC sampler and via variational Bayesian (VB) inference (Beal 2003). The VB inference typically converges quickly and is computationally efficient; by contrast, MCMC convergence may be difficult to diagnose, and a large number of iterations is required to collect samples representing the joint posterior distribution. The detailed MCMC and VB update equations are provided in the Appendix (we provide equations for the GP model, with minor changes manifested for the AR case). Since VB analysis is not as widely used in the statistics literature, for completeness we provide details on its modeling assumptions.

Let Θ represent a vector of all model parameters; the goal is to infer the posterior p(Θ|𝒟). The likelihood of the data is represented by p(𝒟|Θ) and the prior on the model parameters is denoted by p(Θ). Let q(Θ; Γ) be a parametric distribution with hyperparameters Γ, and consider the variational expression

$$ℱ(\mathbf{\Gamma })={\displaystyle \int d\mathbf{\Theta }q(\mathbf{\Theta };\mathbf{\Gamma })\text{ln}\frac{q(\mathbf{\Theta };\mathbf{\Gamma })}{p(𝒟|\mathbf{\Theta })p(\mathbf{\Theta })}=D_{\mathit{\text{KL}}}[q(\mathbf{\Theta };\mathbf{\Gamma })\Vert p(\mathbf{\Theta }|𝒟)]-\text{ln}p(𝒟)}.$$ (10)

In VB analysis the goal is to optimize the hyperparameters Γ to minimize the Kullback-Leibler divergence between q(Θ; Γ) and the true posterior p(Θ|𝒟); this corresponds to adjusting Γ in q(Θ; Γ) such that ℱ(Γ) is minimized. Note that $\int d\mathbf{\Theta }q(\mathbf{\Theta };\mathbf{\Gamma })\text{ln}\frac{q(\mathbf{\Theta };\mathbf{\Gamma })}{p(𝒟|\mathbf{\Theta })p(\mathbf{\Theta })}$ is only a function of the likelihood p(𝒟|Θ) and the prior p(Θ), and not the unknown posterior; with careful selection of q(Θ; Γ), numerical techniques akin to expectation-maximization (EM) (Beal 2003) can be employed to minimize (Γ), with assurance of convergence to a local-optimal solution.

Focusing on the GP temporal model (the AR case is very similar), the model parameters are

$$\mathbf{\Theta }=\{{\{{\mathit{\lambda }}_{\mathit{\text{kj}}:}^{*}\}}_{\begin{array}{c}j=1,\dots ,d,\hfill \\ k=1,\dots ,K\hfill \end{array}},{\{{\mathit{B}}_k\}}_{k=1,\dots ,K},{\{Z_k({\mathit{s}}_{\mathit{\text{it}}})\}}_{\begin{array}{c}t=1,\dots ,T,\hfill \\ i=1,\dots ,M,\hfill \\ k=1,\dots ,K\hfill \end{array}},c_0,c_1,d_0,d_1\}$$ (11)

where Z_k(s_it) ~ Bernoulli(p_k(s_it)), with p_k(s_it) defined in (4). Completing the generative process, $v_{\mathit{\text{it}}}~{\displaystyle {\prod }_{j=1}^d}\text{Poisson}({\lambda }_{\mathrm{k̂}\mathit{\text{jt}}}^{*})$ if Z_k(s_it) = 0 for k < k̂ and Z_k̂(s_it) = 1; ${\lambda }_{\mathrm{k̂}\mathit{\text{jt}}}^{*}$ is the jth component of vector ${\mathit{\lambda }}_{\mathrm{k̂}\mathit{\text{jt}}}^{*}$.

In VB one typically assumes a factorized form for q(Θ; Γ), i.e., q(Θ; Γ) = ∏_l q_l(Θ_l; Γ_l), where Θ_l represents the lth set of model parameters and q_l(Θ_l; Γ_l) is a parametric density function with hyperparameters Γ_l; the union of all Θ_l corresponds to Θ. Through careful selection of q_l(Θ_l; Γ_l) one may iteratively optimize the variational expression ℱ(Θ).

For the proposed model, q(B_k) is a multivariate normal distribution, q(Z_k(s_it)) is Bernoulli (with Bernoulli probability defined by a logistic function), q(ψ_k) is multinomial based upon a finite library of possible parameters ${\{{\psi }_l^{*}\}}_{l=1,L}$, and q(c₀) and q(d₀) are gamma distributions. It is not possible to define a $q({\mathit{\lambda }}_{\mathit{\text{kj}}:}^{*})$ that yields closed-form updates. Therefore, the parameters ${\mathit{\lambda }}_{\mathit{\text{kj}}:}^{*}$ within the VB analysis are also approximated at each iteration via a point estimate that maximizes the functional ℱ(Γ). Similarly, q(c₁) and q(d₁) cannot be obtained in closed form. The parameters c₁ and d₁ are updated on each VB iteration by defining parameters that maximize the functional ℱ(Γ).

4 Example Results

While the proposed model may appear relatively complicated, the number of hyperparameters that need be set is actually modest. We compare the AR-LSBP and GP-LSBP models for imposing a prior on the temporal dependence with a simpler model in which the priors for each time point t are independent. In the context of this independent LSBP (ind-LSBP), we impose

$${\beta }_{\mathit{\text{kjt}}}~𝒩(0,{\alpha }_{\mathit{\text{kjt}}}^{-1}),{\alpha }_{\mathit{\text{kjt}}}~\text{Gamma}(a_0,b_0)$$ (12)

and we set a₀ = b₀ = 10⁻⁶ as in the relevance vector machine (RVM) (Tipping 2001). The same gamma priors are placed on α_β and α_λ for the AR-LSBP model, and on c₀ and c₁ for the GP-LSBP model. In all examples the truncation level on the LSBP was set at K = 20, and the results are insensitive to this parameter, as long as it is large relative to the actual number of clusters/segments inferred by the model. Finally, we must specify the library for kernel parameters {ψ_k}_k=1,K; the manner in which these are specified is discussed when presenting the specific examples.

For uniform temporal sampling, the AR(1) and GP imposition of temporal dynamics are theoretically identical, for the imposed GP covariance. Nevertheless, even for uniform temporal sampling we show results for both of these implementations, because the details of the numerics dictates that the two models are slightly different in practice. Specifically, within the GP model a point estimate is employed for the kernel hyperparameters, with this obviously unnecessary for the direct AR(1) model. The comparison allows examination of the accuracy of this approximation within the GP inference, relative to the direct AR(1) implementation; this sheds light on the quality of the computations for non-uniform temporal sampling, where the GP implementation is required.

4.1 Simulation Example

We assume the data are constructed by a total of 9 equally spaced time instances, t = 1, 2, …, 9. At each time we randomly draw 50 spatial locations in one-dimensional space from a uniform distribution with support [0, 20], denoted as s_it ~ Uniform[0, 20], i = 1, …, 50, t = 1, …9. For each location, we draw an event count v_it from a Poisson distribution with the intensity parameter λ_it. To represent the time dynamics, we let λ_it = 20 when $5+\frac{5}{8}(t-1)\le {\mathit{s}}_{\mathit{\text{it}}}\le 10+\frac{5}{8}(t-1)$, and λ_it = 1 otherwise. By this setting the high-intensity window moves gradually from [5, 10] to [10, 15] when time t increases. Note that here s_it ∈ ℝ¹ and v_it ∈ ℝ¹. The kernel centers are defined as s̃_j = 0.5(j − 1) for j = 1, …, J. The data are depicted in Figure 2. Within the analysis, the library of kernel parameters is the union of the following two sets: {0.05, 0.1, 0.05, …, 0.5} and {0.5, 1, 1.5, …, 5}.

Figure 2.

Simulation example. The high-intensity window moves gradually from [5, 10] to [10, 15] when time increases.

The mean results from VB are shown in Figure 3, in which the inferred Poisson rate is constituted; for these and all VB results the computations were stopped when the change in the variational bound changed by 10⁻⁴. Further, all VB results are initialized at random. The VB results presented below represent a local-optimal solution, which forms one source of error, and this is compounded by the factorized approximation to the posterior. Nevertheless, the VB implementation of the GP-LSBP and AR-LSBP model yields results comparable to that of the MCMC implementation. When implementing MCMC, a total of 10,000 iterations are run, with the first 1000 discarded as burnin. On the same PC (and both codes written in Matlab), the VB GP-LSBP and ARLSBP results required approximately 158 seconds of CPU time, while the VB ind-LSBP results required approximately 96 seconds. In contrast, the GP-LSBP and AR-LSBP results based on the MCMC sampler required 6517 seconds, and ind-LSBP required 2913 seconds (109 and 48 minutes, respectively). The software was not optimized, and these numbers therefore represent a relative view of computational expense of the VB and MCMC solutions.

Figure 3.

Segmentation and latent intensity inferred by VB: Comparison between GP-LSBP and ind-LSBP, considering the simulated-data example. The AR-LSBP results are similar to the GP-LSBP results, and are omitted for brevity.

From Figure 3 it is observed that, for the VB solution, incorporation of temporal smoothness in the GP-LSBP model yields significant improvements in the inferred Poisson rate, as compared to the VB ind-LSBP solution (with temporal dependence not accounted for in the prior); the AR-LSBP model performed similarly to GP-LSBP. It appears that the prior constraint imposed by GP/AR within the VB solution plays an important role in mitigating the underlying VB approximations. By contrast, for the MCMC results improvements are manifested via GP-LSBP and AR-LSBP relative to ind-LSBP, but in this case the differences are less dramatic (plots of MCMC results are not shown, for brevity).

We next examine the generative performance of the proposed model. After the model has been learned, either via VB or MCMC, we randomly generate 100 new test data, following the same procedure that generated the training data. We then compute the average log-likelihood and the accuracy rate of segmentation from the learned GP-LSBP, AR-LSBP and ind-LSBP models. The accuracy rate of segmentation is defined as the number of test data points segmented correctly as a fraction of the total number of test data points. The results are summarized in Table 1. We find that the GP-LSBP and AR-LSBP models achieve a higher likelihood and accuracy of segmentation compared to the ind-LSBP. Note that the differences between GP-LSBP, AR-LSBP and ind-LSBP are relatively modest for the MCMC solution, while there are again marked advantages in the GP-LSBP and AR-LSBP solutions relative to ind-LSBP when employing VB inference.

Table 1.

Comparison of generative performance between AR-LSBP, GP-LSBP and ind-LSBP, on simulated data.

Method	Average log-likelihood		Accuracy rate of segmentation
	VB	MCMC	VB	MCMC
AR-LSBP	−3.702	−1.749	0.9796	0.9801
GP-LSBP	−3.882	−2.082	0.9765	0.9757
ind-LSBP	−15.544	−2.274	0.9478	0.9741

Finally we test the prediction performance of the model. We first generate data 𝒟 = {s_i, v_it}_{i=1,…, 50, t=1, …, 9} as discussed above, and then randomly select N_miss time instances t̂₁, …, t̂N_miss from t = 1, …, 9, and this constructs our test data 𝒟_tst; the training data 𝒟_trn is composed of the data in 𝒟 but not in 𝒟_tst. We learn the model based on VB or MCMC analysis with 𝒟_trn, and predict the kernel weights β̂_kjt̂ and Poisson intensities ${\mathrm{\lambda ̂}}_{k\mathrm{t̂}}^{*}$ at time t̂. The average log-likelihood and accuracy of segmentation are evaluated based on the prediction results of 𝒟_tst, given only the spatial locations s_it̂. We perform 100 trials, and at each trial N_miss time instances are selected randomly to construct 𝒟_tst. The average results are shown in Table 2.

Table 2.

Comparison of prediction performance between AR-LSBP, GP-LSBP and ind-LSBP.

Nmiss	Average log-likelihood
	AR-LSBP		GP-LSBP		ind-LSBP
	VB	MCMC	VB	MCMC	VB	MCMC
1	−3.948	−1.975	−4.102	−2.123	−21.194	−2.641
2	−4.211	−2.241	−4.526	−2.473	−27.195	−3.077
3	−4.468	−2.573	−4.718	−2.652	−27.776	−3.507
4	−4.882	−2.740	−5.133	−3.108	−26.682	−3.963
5	−5.801	−3.014	−5.987	−3.521	−31.217	−4.316
Nmiss	Accuracy rate of segmentation
	AR-LSBP		GP-LSBP		ind-LSBP
	VB	MCMC	VB	MCMC	VB	MCMC
1	0.9792	0.9794	0.9767	0.9758	0.7165	0.9545
2	0.9787	0.9786	0.9761	0.9754	0.6669	0.9581
3	0.9787	0.9785	0.9763	0.9752	0.6458	0.9379
4	0.9780	0.9783	0.9752	0.9740	0.6647	0.9274
5	0.9763	0.9770	0.9741	0.9633	0.6131	0.9066

Only the GP-LSBP results are fully principled in this analysis, where we use the learned parameters of the GP covariance matrix to interpolate to new time points (Rasmussen and Willams 2006). The AR model implicitly assumes that the data are sampled uniformly in time, while the ind-LSBP has no principled means of interpolating to missing time points. Nevertheless, as a comparison, for the AR-LSBP computations in this test the AR component was simply applied to consecutive observed time points, essentially assuming that the temporal variation was smooth, even if not sampled uniformly. To interpolate to new points using the learned AR-LSBP and ind-LSBP results, to obtain model parameters at any new point t̂, we average the learned model parameters from the two closest observed points, before and after t̂. From Table 2 it is observed that again for the VB solution, there is a marked advantage manifested via the GP-LSBP and AR-LSBP priors, as compared to ind-LSBP. For the MCMC solution, there is also a noticeable advantage manifested via the GP-LSBP and AR-LSBP solutions, particularly for segmentation accuracy for relatively large N_miss. Based upon the average log-likelihood, we note a small but consistent advantage of the AR-LSBP model over the GP-LSBP counterpart, for both VB and MCMC computations. This observation on simulated data will carry over to the analysis of real data.

4.2 Crime Data

We investigate crime events in Cincinnati, Ohio, USA; the data are available online at http://www.cincinnati-oh.gov. The data include the date, time, location and other information of all reported crimes in Cincinnati since 2006. This data set was first studied in Taddy (2008, 2010), where a mixture of beta distributions was employed to model the event density ν(s), and to discover the evolution of the density with time. In our problem we seek to segment the city into contiguous regions, with crime events at each region characterized by a common constant Poisson intensity vector.

We consider 117,314 crime events within the city, reported from January 2006 to December 2008. Each crime is assigned a uniform crime reporting (UCR) code. In total more than 170 different UCR codes describe a variety of crimes. These crime events can be categorized into 17 different crime types, based on the prefix of their UCR codes. They are: 1) murder, 2) rape, 3) robbery, 4) assault with weapon, 5) burglary, 6) nonvehicle theft, 7) vehicle theft, 8) general assault, 9) arson, 10) forgery, 11) fraud, 12) receiving stolen property 13) vandalism, 14) weapons related but no physical harm, 15) sexual crime, 16) children related, 17) general harassment. As an example, the locations (latitude and longitude coordinates) of the 3090 crime events in January 2008 are shown in Figure 1(a). Based on the locations of all the 117,314 crime events, the observation window is considered within a rectangular region of [39.06°, 39.24°] latitude and [−84.70°, −84.35°] longitude.

We construct the data 𝒟 = {s_i, v_it}_{i=1, …, M, t=1, …, T} as follows. The total crime events within one month are considered as one time instance, and therefore there are in total 36 time points. At each time, the observation window is divided into 15,750 small square grids (90 rows by 175 columns) of size 0.002° × 0.002°, and the event location s_it is defined as the center of each small square area, with this denoted as Δ(s_i). The count v_ijt is then the number of Type j crimes within Δ(s_i) over the corresponding month indexed by t. This produces a 17-dimensional count vector v_it at s_i for i = 1, …, 15750 and t = 1, …, 36. Related research in Taddy (2008, 2010) applied marked Poisson processes to address the crime types, regarding each crime type at s_it as a random mark. Here we attempt to segment the city by considering all the crime types within a local region Δ(s_it) as a correlated variable (a vector), instead of treating each event as a random type.

The proposed GP-LSBP, AR-LSBP and ind-LSBP models are inferred via VB and MCMC, with truncation level K = 20. The kernel centers are uniformly spaced every 0.04° (latitude and longitude) in the observation window, with a total of 60 kernel centers defined. The library of kernel parameters ${\{{\psi }_l^{*}\}}_{l=1,L}$ is the union of the following sets: {0.006°, 0.012°, 0.018°, …, 0.06°} and {0.06°, 0.12°, 0.18°, …, 0.6°}. On the same PC, the VB GP-LSBP and AR-LSBP results require approximately 2.8 hours of CPU time, while the VB ind-LSBP results required approximately 1.3 hours. By contrast, due to the large size of the data, 3000 MCMC samples are employed, with 1000 discarded as burn-in. With the same PC, the MCMC GP-LSBP and AR-LSBP results required approximately 47.5 hours. We also considered 10,000 MCMC samples, with 1000 discarded as burn-in (at very significant computational cost), with little change in the results relative to those presented below.

Figure 4(a) shows the VB-based segmentation of the entire spatial observation window at 36 time instances, using GP-LSBP (similar results were found using AR-LSBP, omitted for brevity). The city is segmented into 4 regions (inferred by the model), and the segmentation changes smoothly with time. For comparison, Figure 4(b) shows the segmentation results obtained by applying an independent LSBP (VB computations) at each time instance. It is observed that with GP priors the proposed model presents a spatial segmentation more consistently over time and spatially more contiguously than ind-LSBP.

Figure 4.

Comparison of spatial segmentation for crime data in Cincinnati, Ohio from January 2006 to December 2008 (VB results). Each color represents a segment with an associated intensity vector ${\mathit{\lambda }}_{\mathit{\text{kt}}}^{*}$, and there are a total of four segments inferred: 1 - dark blue, 2 - light blue, 3 - yellow, and 4 - dark red. (a) GP-LSBP, (b) ind-LSBP.

We are also interested in examining the clustering manifested by the MCMC computations, with this complicated by label switching between samples. We compute an MCMC clustering that may be compared to the VB results as follows. We consider one spatial location from Segment 1 in Figure 4, denoted ${\mathit{s}}_1^{*}$. Based upon the MCMC collection samples, for each other spatial location in the scene $\mathit{s}\ne {\mathit{s}}_1^{*}$, we compute the probability that position s and ${\mathit{s}}_1^{*}$ are in the same cluster. All positions s with high probability of such clustering should (ideally) constitute a spatial region similar to Segment 1 inferred via VB. In Figure 5(a) we show MCMC results for Segment 1, and the high-probability regions (red) do indeed align well with the VB results in Figure 4. In Figure 5(b) we compute similar MCMC results for Segment 2, and in this case the high-probability spatial locations are aligned well with the VB results for Segment 2 in Figure 4. We found in general good agreement between the VB and MCMC segmentation results for GP-LSBP and AR-LSBP for these data.

Figure 5.

Comparison of spatial segmentation for crime data in Cincinnati, Ohio from January 2006 to December 2008 (MCMC results). (a) Segment 1, (b) Segment 2, where these segments are related to the results in Figure 4(a). The color scale is the same in (a) and (b).

Figures 6(a)–(d) show the dynamic change of the VB-inferred Poisson intensities for each segment. To make the figure easier to read, we only plot components 3, 5 and 6 from the 17-dimensional vector ${\mathit{\lambda }}_{\mathit{\text{kt}}}^{*}$; these components correspond to crime types “robbery”, “burglary”, and “nonvehicle theft”, respectively. From these figures we observed that in all segments the crime intensities fluctuated periodically over season. Generally in summer there were more crime events of all types than than in winter. The overall crime intensities varied with regions. Segment 4 was in the downtown region, and had much more crime events compared to other regions. In all four regions Type 6 crime (nonvehicle theft) was dominant. In addition, the crime patterns were different in different regions. For example, Segment 4 had relatively less Type 5 crime (burglary), while in other 3 segments, the intensity of Type 5 crime was almost half of Type 6 crime. In Segment 4, Type 3 crime (robbery) was prevalent, while Segment 1 had relatively less Type 3 crime. For a comparison, we also present the MCMC-inferred Poisson intensities of Segment 3, as a representative (typical) example. It is observed that the MCMC and VB results are in generally good agreement for the GP-LSBP and AR-LSBP models.

Figure 6.

Inferred intensity vector ${\mathit{\lambda }}_{\mathit{\text{kt}}}^{*}$ associated with the segments shown in Figure 4(a). Only 3 crime types are shown here to make the figure easy to read.

These results may be used by police to assign resources (personnel) to segmented regions in a consistent manner, to address varying levels of crimes. The segments typically change with season, and the spatial distribution of resources may be temporally adjusted as well. By relating the demographics of regions to the spatial segments (we didn’t have access to such demographics), one may deduce relationships between types of crimes and the types of people living and working in given regions, of interest to criminologists and city planners.

Following the same procedure as in the simulated example, we now examine the prediction performance of our model for the crime data. We randomly select N_miss time instances to construct a test set, and let the remaining data be the training set. Ten random trials are performed and the comparison of average log-likelihood between GP-LSBP, AR-LSBP and ind-LSBP inferred by VB is shown in Table 3. Since in this real application there is no ground truth, we cannot evaluate the accuracy rate of segmentation as done in the simulated example. From Table 3 GP-LSBP and ARLSBP consistently achieve higher likelihood than the independent LSBP for various N_miss values. Note also that for these real data there is less of a difference between the AR/GP-LSBP and ind-LSBP results for the VB solution, as compared to the synthetic data considered above. We do not perform this experiment for MCMC inference, as the computational requirements needed to perform this many experiments are prohibitive with this large data set (however, in isolated tests, the results were slightly better than the VB-based GP-LSBP and AR-LSBP models, consistent with the simulated example above).

Table 3.

Comparison of average log-likelihood in the prediction for the crime data (VB inference).

Nmiss	1	2	3	4	5	6
AR-LSBP	−6.131	−6.352	−7.204	−7.631	−7.957	−8.338
GP-LSBP	−6.570	−6.762	−7.713	−7.965	−8.426	−8.721
ind-LSBP	−8.666	−9.247	−9.595	−8.840	−9.848	−8.762

4.3 Pearson residuals

Following Taddy (2010), we check model quality via computation of Pearson residuals (see Turner et al. (2005) for a detailed discussion of residuals for spatial point processes). For the modeling framework considered here, the Pearson residual reduces to

$$R(\mathrm{\Delta }({\mathit{s}}_{\mathit{\text{it}}}),{\mathrm{\lambda ̂}}_{\mathit{\text{it}}})=\frac{n_{\mathit{\text{it}}}}{\sqrt{{\mathrm{\lambda ̂}}_{\mathit{\text{it}}}}}=-\sqrt{{\mathrm{\lambda ̂}}_{\mathit{\text{it}}}}$$ (13)

where n_it is the number of events in region Δ(s_it) and λ̂_it is the inferred Poisson rate parameter in small region Δ(s_it). Ideally the residual should be close to zero, if the underlying Poisson assumption is valid. Note that within the proposed model we have a vector of counts v_it, and therefore we may compute the residual for each of the different types of crimes.

From Figure 7, which is based upon VB inference, we observe that the Pearson residuals tend to decrease substantially based upon a model that explicitly imposes temporal smoothness (note that the residuals are significantly lower for GP-LSBP and AR-LSBP, relative to ind-LSBP). Further, the AR-LSBP residuals are smaller than those of the GP-LSBP. Although we omit the MCMC results for brevity, similar phenomena were observed in that case. The residuals tend to be small, in the range [−2,2], with the larger values manifested on the edges of segments, as might be expected (segment interfaces are characterized typically by abrupt changes in statistical properties).

Figure 7.

Pearson residuals for “nonvehicle theft,” using VB inference; best viewed electriconically, zoomed in. (a) ind-LSBP, (b) GP-LSBP, (c) AR-LSBP.

5 Conclusions

A Bayesian hierarchical model has been presented for segmenting time-evolving point process data, when the events are in vector form. The spatial-dependent point process is modeled using a generalization of a Poisson process, with piecewise constant Poisson intensities defined within the observation window. The logistic stick-breaking process is employed to favor spatially contiguous segments, and GP and AR models are considered for imposition of temporal smoothness of the segmentation and the Poisson intensity.

In addition to developing the model, a contribution of this paper concerns a detailed comparison between MCMC sampling and a VB approximation. For both the synthetic and real data, it was found that the GP-LSBP and AR-LSBP results computed via VB and MCMC were in close agreement, and the imposition of temporal smoothness manifested via GP/AR (compared to treating the different temporal samples independently) yielded significant improvements in the VB results. While the VB results are approximate, and are subject to local-optimal solutions (although the GP/AR models seemed to mitigate this to some extent), the VB approach provides significant advantages with regard to computations. For the large crime data set considered, while the MCMC results are in principle convergent, if run for enough samples, this attractiveness is tempered by the very significant computation time required to realize a number of collection samples to assure that we are indeed sampling from the posterior. Given that computational requirements will in practice mitigate the ability to collect as many MCMC samples as desired (and therefore MCMC is also an approximation), the VB solution appears to be an attractive option. However, the results presented here indicate that imposition of as much information as possible (here smoothness via GP/AR) is desirable. In future research it is of interest to consider online VB analysis (Hoffman et al. 2010), which provides further acceleration for large datasets, and it is appropriate for time-dependent data observed in an online/sequential manner, like the time-evolving crime data considered here.

Appendix: MCMC and VB Update Equations

A.1 MCMC Inference

The MCMC computations are performed using Gibbs sampling where the conditional density functions are analytic, and samples are drawn from the conditional density functions via Metropolis-Hastings when not analytic. The update equations are summarized as follows.

• Sample ${\mathit{\lambda }}_{\mathit{\text{kj}}:}^{*}$ from their respective posteriors conditional on {Z_k (s_it)} and {ν_ijt}:

  $$p({\mathit{\lambda }}_{\mathit{\text{kj}}:}^{*}|-)\propto {\displaystyle \prod _{t=1}^T}{\displaystyle \prod _{i=1}^M}\text{Poisson }{({\nu }_{\mathit{\text{ijt}}}|{\lambda }_{\mathit{\text{kjt}}}^{*})}^{\mathbf{I}(c_i=k)}\text{ ln}𝒩({\mathit{\lambda }}_{\mathit{\text{kj}}:}^{*}|0,{\mathbf{\Gamma }}_{\mathit{\text{kj}}}).$$ (14)

  It is not possible to sample ${\mathit{\lambda }}_{\mathit{\text{kj}}:}^{*}$ from the full conditionals. We update each ${\mathit{\lambda }}_{\mathit{\text{kj}}:}^{*}$ by the Metropolis-Hastings algorithm. When updating ${\mathit{\lambda }}_{\mathit{\text{kj}}:}^{*}$, the proposed ${\mathit{\lambda }}_{\mathit{\text{kj}}:}^{*(\tau +1)}$ is generated from the following distribution

  $$q(\text{ln }{\mathit{\lambda }}_{\mathit{\text{kj}}:}^{*(\tau +1)}|\text{ ln }{\mathit{\lambda }}_{\mathit{\text{kj}}:}^{*(\tau )})=𝒩(\text{ln }{\mathit{\lambda }}_{\mathit{\text{kj}}:}^{*(\tau )},(d_0+d_2){\mathbf{I}}_{\mathbf{T}})$$ (15)

  where I_T is the T × T identity matrix, and T denotes the number of time points. The acceptance probability for the proposed ${\mathit{\lambda }}_{\mathit{\text{kj}}:}^{*(\tau +1)}$ is min $(1,\alpha ({\mathit{\lambda }}_{\mathit{\text{kj}}:}^{*(\tau +1)},{\mathit{\lambda }}_{\mathit{\text{kj}}:}^{*(\tau )}\left)\right)$, where

  $$\alpha ({\mathit{\lambda }}_{\mathit{\text{kj}}:}^{*(\tau +1)},{\mathit{\lambda }}_{\mathit{\text{kj}}:}^{*(\tau )})=\text{exp}(-\frac{1}{2}{({\mathit{\lambda }}_{\mathit{\text{kj}}:}^{*(\tau +1)})}^T{\mathbf{\Gamma }}_{\mathit{\text{kj}}}^{-1}{\mathit{\lambda }}_{\mathit{\text{kj}}:}^{*(\tau +1)}+\frac{1}{2}{\mathit{\lambda }}_{\mathit{\text{kj}}:}^{*(\tau )T}{\mathbf{\Gamma }}_{\mathit{\text{kj}}}^{-1}{\mathit{\lambda }}_{\mathit{\text{kj}}:}^{*(\tau )})·{\displaystyle \prod _{t=1}^T}{\left[\right(\frac{{\lambda }_{\mathit{\text{kjt}}}^{*(t+1)}}{{\lambda }_{\mathit{\text{kjt}}}^{*(t)}})}^{{\displaystyle \sum _{i=1}^M}{w}_k({\mathit{s}}_{\mathit{\text{it}}}){\upsilon }_{\mathit{\text{ij}}1}-1}\text{exp}[{\displaystyle \sum _{i=1}^M}{w}_k({\mathit{s}}_{\mathit{\text{it}}})({\lambda }_{\mathit{\text{kjt}}}^{*(\tau +1)}{\lambda }_{\mathit{\text{kjt}}}^{*(\tau )}\left)\right]].$$ (16)
• Sample β_k:i from their respective posteriors conditional on {Z_k (s_it)}:

  $$p({\mathit{B}}_k|-)\propto {\displaystyle \prod _{t=1}^T}{\displaystyle \prod _{i=1}^M}p(Z_k({\mathit{s}}_{\mathit{\text{it}}})|B_k){\displaystyle \prod _{j=1}^J}𝒩({\beta }_{\mathit{\text{kj}}:}|\mathbf{0},{\mathrm{\Sigma }}_{\mathit{\text{kj}}}).$$ (17)

  Reorder the entries of B_k (and the associated Ω_k) in (8) such that B_k = [β_k:1, …, β_k:T]^T, then we obtain

  $$p({\mathit{B}}_k|-)\propto \text{exp}\{-{\displaystyle \sum _{t=1}^T}{\displaystyle \sum _{i=1}^M}f({\eta }_{\mathit{\text{kit}}}){\mathit{\beta }}_{k:t}^T{\phi }_{\mathit{\text{kit}}}{\phi }_{\mathit{\text{kit}}}^T{\mathit{\beta }}_{k:t}\}\times \text{ exp}\{-\frac{1}{2}{\mathit{B}}_k^T{\mathrm{\Omega }}_K^{-1}{\mathit{B}}_k+{\displaystyle \sum _{t=1}^T}{\displaystyle \sum _{i=1}^M}(2Z_k({\mathit{s}}_{\mathit{\text{it}}})-1){\phi }_{\mathit{\text{kit}}}^T{\mathit{\beta }}_{k:t}\}.$$ (18)

  So, B_k can be drawn from a normal distribution as

  $$p({\mathit{B}}_k|-)=𝒟({\mathit{B}}_k;{({\mathbf{\Omega }}_k^{-1}+{\mathit{U}}_k)}^{-1}{\mathit{Y}}_k,{({\mathbf{\Omega }}_k^{-1}+{\mathit{U}}_k)}^{-1}),$$ (19)

  where U_k is a (J + 1) T × (J + 1) T block-diagonal matrix with the t-th (J + 1) × (J + 1) block expressed as ${\mathit{u}}_{\mathit{\text{kt}}}=2{\displaystyle \sum _{i=1}^M}f({\eta }_{\mathit{\text{kit}}}){\mathit{\varphi }}_{\mathit{\text{kit}}}{\mathit{\varphi }}_{\mathit{\text{kit}}}^T$ and Y_k is a (J + 1) T × 1 vector formed by concatenating the T vectors ${\mathit{y}}_{\mathit{\text{kt}}}={\displaystyle \sum _{i=1}^M}(Z_k({\mathit{s}}_{\mathit{\text{it}}})-\frac{1}{2}){\mathit{\varphi }}_{\mathit{\text{kit}}}$, t = 1, ⋯, T. In these expressions ϕ_kit = [1, 𝒟 (s_it, s̃₁; ψ_k), ⋯, 𝒟 (s_it, s̃_J; ψ_k)]^T. The parameter $f({\eta }_{\mathit{\text{kit}}})={\phi }_{\mathit{\text{kit}}}^T{\mathit{\beta }}_{k:t}$.
• Sample Z_k (s_it) from their respective posteriors conditional on B_k and {ν_ijt}. According to the definition of LSBP,

  $$p(Z_k({\mathit{s}}_{\mathit{\text{it}}})=1|-)=\{\begin{array}{c}\hfill \frac{\sigma (g_k({\mathit{s}}_{\mathit{\text{it}}}))p({\nu }_{\mathit{\text{it}}}|{\mathit{\lambda }}_{\mathit{\text{kt}}}^{*})}{\sigma (g_k({\mathit{s}}_{\mathit{\text{it}}}))p({\upsilon }_{\mathit{\text{it}}}|{\mathit{\lambda }}_{\mathit{\text{kt}}}^{*})+\sigma (-g_k({\mathit{s}}_{\mathit{\text{it}}}))p({\nu }_{\mathit{\text{it}}}|{\mathit{\lambda }}_{k\prime t}^{*})},\text{ if }{Z}_l({\mathit{s}}_{\mathit{\text{it}}})=0\text{ for }l<k\hfill \\ \hfill \sigma (g_k({\mathit{s}}_{\mathit{\text{it}}})),\text{ if }\exists l<k,\text{ such that }{Z}_l({\mathit{s}}_{\mathit{\text{it}}})=1\hfill \end{array}$$ (20)

  where k′ is the first integer larger than k, associated with non-zero indicator. The equation can be expressed as

  $$p(Z_k({\mathit{s}}_{\mathit{\text{it}}})=1|-)=\frac{1}{1+\text{exp }(-{\rho }_{\mathit{\text{kit}}})},$$ (21)

  with

  $${\rho }_{\mathit{\text{kit}}}={\displaystyle \prod _{l<k}}(1-Z_l({\mathit{s}}_{\mathit{\text{it}}}))\text{ log }p({\nu }_{\mathit{\text{it}}}|{\mathit{\lambda }}_{\mathit{\text{kt}}}^{*})-{\displaystyle \sum _{k\prime >k}}{Z}_l({\mathit{s}}_{\mathit{\text{it}}}){\displaystyle \prod _{\begin{array}{c}l<k\prime \hfill \\ l\ne k\hfill \end{array}}}(1-Z_l({\mathit{s}}_{\mathit{\text{it}}}))\text{ log }p(({\nu }_{\mathit{\text{it}}}|{\mathit{\lambda }}_{k\prime t}^{*}))+{\phi }_{\mathit{\text{kit}}}^T{\mathit{\beta }}_{k:t}.$$ (22)
• With a uniform prior assumed on the kernel parameter library (a predefined finite set), the posterior distribution for each ψ_k can be represented as

  $$p({\psi }_k={\psi }_l^{*})\propto {\displaystyle \prod _{t=1}^T}{\displaystyle \prod _{i=1}^M}\sigma {(g_k^l({\mathit{s}}_{\mathit{\text{it}}}))}^{w_k({\mathit{s}}_{\mathit{\text{it}}})}{\displaystyle \prod _{t=1}^T}{\displaystyle \prod _{i=1}^M}{\displaystyle \prod _{k\prime >k}}(1-\sigma {(g_k^l({\mathit{s}}_{\mathit{\text{it}}})))}^{w_{k\prime }({\mathit{s}}_{\mathit{\text{it}}})}.$$ (23)

  For each specific k from k = 1, …, K, we have the following update equation

  $${\psi }_k={\psi }_{r_k}^{*},r_k~\text{Mult }(p_{k1},\dots ,p_{\mathit{\text{kL}}}),p_{\mathit{\text{kj}}}=\frac{p({\psi }_k={\psi }_j^{*})}{{\displaystyle {\sum }_{l=1}^L}p({\psi }_k={\psi }_l^{*})}.$$ (24)

  We sample the kernel parameters based on the multinomial distributions from a given discrete set in each MCMC iteration.
• Sample c₀ from its posteriors conditional on {B_k} and {a₀, b₀}:

  $$p(c_0)\propto \text{Gamma}(c_0;a_0,b_0){\displaystyle \prod _{k=1}^K}𝒩({\mathit{B}}_k;\mathbf{0},{\mathbf{\Omega }}_k).$$ (25)

  Therefore, c₀ can be drawn from a Gamma distribution

  $$p(c_0)=\text{Gamma}(c_0;{ã}_0,{\mathrm{b̃}}_0),$$ (26)

  where ã₀ = a₀+0.5KT(J + 1) and ${\mathrm{b̃}}_0=b_0+0.5{\displaystyle \sum _{k=1}^K}{\displaystyle \sum _{j=0}^J}{\mathit{\beta }}_{\mathit{\text{kj}}:}^T{\displaystyle {\sum ̃}_{\mathit{\text{kj}}}^{-1}}{\mathit{\beta }}_{\mathit{\text{kj}}:}$ with [Σ̃_kj]_il = c₁^|t_i−t_l|.
• Sample c₁ from its posterior conditional on {B_k}:

  $$p(c_1)\propto {𝒩}_{(0,1)}(c_1;0,1){\displaystyle \prod _{k=1}^K}𝒩({\mathit{B}}_k;\mathbf{0},{\mathbf{\Omega }}_k).$$ (27)

  When updating c₁, the proposed $c_1^{(\tau +1)}$ is generated from the following distribution:

  $$q(c_1^{(\tau +1)}|c_1^{\tau })={𝒩}_{(0,1)}(c_1^{(\tau +1)};c_1^{\tau },1).$$ (28)

  The acceptance probability for the proposed $c_1^{(\tau +1)}$ is min $(1,\alpha (c_1^{(\tau +1)},c_1^{\tau }))$ , where

  $$\alpha (c_1^{(\tau +1)},c_1^{\tau })=\frac{{|{\mathbf{\Sigma }}_{\mathit{\text{kj}}}^{-1}(c_1^{\tau })|}^{\frac{K(J+1)}{2}}}{{|{\mathbf{\Sigma }}_{\mathit{\text{kj}}}^{-1}(c_1^{(\tau +1)})|}^{\frac{K(J+1)}{2}}}\text{exp}\left\{\frac{1}{2}\right({c_1^{(\tau +1)}}^2-{c_1^{(\tau +1)}}^2\left)\right\}\times \text{ exp }\left\{\frac{1}{2}\right({\displaystyle \sum _{k=1}^K}{\displaystyle \sum _{j=0}^J}{\mathit{\beta }}_{\mathit{\text{kj}}:}^T{\mathbf{\Sigma }}_{\mathit{\text{kj}}}^{-1}(c_1^{\tau }){\mathit{\beta }}_{\mathit{\text{kj}}:}-{\displaystyle \sum _{k=1}^K}{\displaystyle \sum _{j=0}^J}{\mathit{\beta }}_{\mathit{\text{kj}}:}^T{\mathbf{\Sigma }}_{\mathit{\text{kj}}}^{-1}(c_1^{(\tau +1)}){\mathit{\beta }}_{\mathit{\text{kj}}:}\left)\right\}.$$ (29)
• Similarly, d₀ can be drawn from a Gamma distribution

  $$p(d_0)=\text{Gamma}(d_0;{ã}_0,{\mathrm{b̃}}_0),$$ (30)

  where ã₀ = a₀ + 0.5dKT and ${\mathrm{b̃}}_0=b_0+0.5{\displaystyle \sum _{k=1}^K}{\displaystyle \sum _{j=1}^d}\text{ln}{\mathit{\lambda }}_{\mathit{\text{kj}}:}^{*T}{\mathbf{\Gamma ̃}}_{\mathit{\text{kj}}}^{-1}\text{ln}{\mathit{\lambda }}_{\mathit{\text{kj}}:}^{*}$ with [Γ̃_kj]_il = d₁^|t_i−t_l|.
• Similar to c₁, we update d₁ by the Metropolis-Hastings algorithm. The proposed $d_1^{(\tau +1)}$ is generated from the following distribution:

  $$q(d_1^{(\tau +1)}|d_1^{\tau })={𝒩}_{(0,1)}(d_1^{(\tau +1)};d_1^{\tau },1).$$ (31)

  The acceptance probability for the proposed $d_1^{(\tau +1)}$ is min $(1,\alpha (d_1^{(\tau +1)},d_1^{\tau }))$, where

  $$\alpha (d_1^{(\tau +1)},d_1^{\tau })=\frac{{|{\mathbf{\Gamma }}_{\mathit{\text{kj}}}^{-1}(c_1^{\tau })|}^{\frac{\mathit{\text{dK}}}{2}}}{{|{\mathbf{\Gamma }}_{\mathit{\text{kj}}}^{-1}(c_1^{(\tau +1)})|}^{\frac{\mathit{\text{dK}}}{2}}}\text{exp}\left\{\frac{1}{2}\right(d_1^{{(\tau +1)}^2}-d_1^{{(\tau +1)}^2}\left)\right\}\times \text{exp}\left\{\frac{1}{2}\right({\displaystyle \sum _{k=1}^K}{\displaystyle \sum _{j=1}^d}\text{ln}{\mathit{\lambda }}_{\mathit{\text{kj}}:}^{*T}{\mathbf{\Gamma }}_{\mathit{\text{kj}}}^{-1}(d_1^{\tau })\text{ln}{\mathit{\lambda }}_{\mathit{\text{kj}}:}-{\displaystyle \sum _{k=1}^K}{\displaystyle \sum _{j=1}^d}\text{ln}{\mathit{\lambda }}_{\mathit{\text{kj}}:}^{*T}{\mathbf{\Sigma }}_{\mathit{\text{kj}}}^{-1}(d_1^{(\tau +1)})\text{ln}{\mathit{\lambda }}_{\mathit{\text{kj}}:}\left)\right\}.$$ (32)

A2. VB inference

The log-normal priors placed on the Poisson intensities introduce non-conjugacy, which results in difficulty for VB inference. Therefore, we employ a point estimate for the Poisson intensities, by maximizing the lower bound ℱ. For the GP hyperparameters c₁ and d₁, the truncated normal prior also introduces non-conjugacy. Their posteriors are also inferred from point estimation by maximizing the VB lower bound. The update equations of the posterior inference of Θ are summarized below. In our model,

$$\mathbf{\Theta }=\{{\{{\mathit{\lambda }}_{\mathit{\text{kj}}:}^{*}\}}_{\begin{array}{c}j=1,\dots ,d,\hfill \\ k=1,\dots ,K\hfill \end{array}},{\{{\mathit{B}}_k\}}_{k=1,\dots ,K},{\{Z_k({\mathit{s}}_{i,t})\}}_{\begin{array}{c}t=1,\dots ,T,\hfill \\ i=1,\dots ,M,\hfill \\ k=1,\dots ,K\hfill \end{array}},c_0,c_1,d_0,d_1\}.$$

• The lower bound for the Poisson intensity ${\mathit{\lambda }}_{\mathit{\text{kj}}:}^{*}$ may be derived as

  $$ℱ({\mathit{\lambda }}_{\mathit{\text{kj}}:}^{*})\propto -\frac{1}{2}{\mathbf{\Lambda }}_{k,j}^T{\mathbf{\Gamma }}_{\mathit{\text{kj}}}^{-1}{\mathbf{\Lambda }}_{\mathit{\text{kj}}}-{\mathit{Q}}_{\mathit{\text{kj}}}^T{e}^{{\mathbf{\Lambda }}_{\mathit{\text{kj}}}}+{\mathit{R}}_{\mathit{\text{kj}}}^T{\mathbf{\Lambda }}_{\mathit{\text{kj}}}+\text{constant}$$ (33)

  where ${\mathbf{\Lambda }}_{\mathit{\text{kj}}}=\text{log}({\mathit{\lambda }}_{\mathit{\text{kj}}:}^{*}),{\mathit{R}}_{\mathit{\text{kj}}}={[{\displaystyle {\sum }_{i=1}^{M_1}}\langle w_k({\mathit{s}}_{i1})\rangle {\nu }_{\mathit{\text{ij}}1}-1,\cdots ,{\displaystyle {\sum }_{i=1}^M}\langle w_k({\mathit{s}}_{\mathit{\text{iT}}})\rangle {\nu }_{\mathit{\text{ijT}}}-1]}^T,\text{ and }{\mathit{Q}}_{\mathit{\text{kj}}}={[{\displaystyle {\sum }_{i=1}^{M_1}}\langle w_k({\mathit{s}}_{i1})\rangle ,\cdots ,{\displaystyle {\sum }_{i=1}^M}\langle w_k({\mathit{s}}_{i1})\rangle ]}^T$, with 〈;·〉 denoting the expectation such that 〈w_k(s_it)〉 = q(w_k(s_it) = 1) (see Section 2 for detail of w_k(s_it)). The point estimate for ${\mathit{\lambda }}_{\mathit{\text{kj}}:}^{*}$ can be updated at each VB iteration by maximizing the lower bound $ℱ({\mathit{\lambda }}_{\mathit{\text{kj}}:}^{*})$. One may easily examine that $ℱ({\mathit{\lambda }}_{\mathit{\text{kj}}:}^{*})$ is a concave function, and therefore a global maximum can be obtained by any appropriate convex optimization method. Note that if ${\mathbf{\Gamma }}_{\mathit{\text{kj}}}^{-1}\to 0$ (setting large variance for the prior distribution), by taking the derivative of (33) and setting it to zero, we have ${\mathit{\lambda }}_{\mathit{\text{kj}}:}^{*}=e^{{\mathbf{\Lambda }}_{\mathit{\text{kj}}}}\to {\mathit{R}}_{\mathit{\text{kj}}}/{\mathit{Q}}_{\mathit{\text{kj}}}$, which is consistent with the update equation if independent gamma priors are placed on ${\mathit{\lambda }}_{\mathit{\text{kjt}}:}^{*}$ for t = 1, …, T. Therefore, the GP priors represented in Γ_kj introduce the correlation among the components of ${\mathit{\lambda }}_{\mathit{\text{kj}}:}^{*}$.
• To update the variational distribution for the kernel weights β_kjt, note that the logistic link function σ(·) is not within the exponential family and therefore introduces the nonconjugacy. We here follow Jaakkola and Jordan (1998) by introducing a variational bound using the inequality

  $$\sigma {(y)}^z{[1-\sigma (y)]}^{1-z}=\sigma (x)\ge \sigma (\eta )\text{ exp}(\frac{x-\eta }{2}-f(\eta )(x^2-{\eta }^2))$$

  where x = (2z − 1)y, $f(\eta )=\frac{\text{tanh}(\eta /2)}{4\eta }$, and η is a variational parameter. An exact bound is achieved as η = ±x. If we reorder the entries of B_k (and the associated Ω_k) in (8) such that B_k = [β_k:1, …, β_k:T]^T, the update equation for B_k can be expressed as

  $$q({\mathit{B}}_k)=𝒩({({\mathbf{\Omega }}_k^{-1}+{\mathit{U}}_k)}^{-1}{\mathit{Y}}_k,{({\mathbf{\Omega }}_k^{-1}+{\mathit{U}}_k)}^{-1})$$ (34)

  where U_k is a (J + 1)T×(J + 1)T block-diagonal matrix with the tth (J + 1)×(J + 1) block expressed as

  $${\mathit{u}}_{\mathit{\text{kt}}}=2{\displaystyle \sum _{i=1}^M}f({\eta }_{\mathit{\text{kit}}}){\mathit{\varphi }}_{\mathit{\text{kit}}}{\mathit{\varphi }}_{\mathit{\text{kit}}}^T$$

  and Y_k is a (J + 1)T × 1 vector formed by concatenating the T vectors

  $${\mathit{y}}_{\mathit{\text{kt}}}={\displaystyle \sum _{i=1}^M}(\langle Z_k({\mathit{s}}_{\mathit{\text{it}}})\rangle -\frac{1}{2}){\mathit{\varphi }}_{\mathit{\text{kit}}},t=1,\dots ,T.$$

  In the above expressions ϕ_kit = [1, 𝒦 (s_it, s̃₁; ψ_k), …, 𝒦 (s_it, s̃_J; ψ_k)]^T. The variational parameters η_kit are then updated as

  $${\eta }_{\mathit{\text{kit}}}^2={\mathit{\varphi }}_{\mathit{\text{kit}}}^T\langle {\mathit{\beta }}_{k:t}^T{\mathit{\beta }}_{k:t}\rangle {\mathit{\varphi }}_{\mathit{\text{kit}}}$$ (35)

  where $\langle {\mathit{\beta }}_{k:t}^T{\mathit{\beta }}_{k:t}\rangle =\mathit{\text{COV}}({\mathit{\beta }}_{k:t},{\mathit{\beta }}_{k:t})+\langle {\mathit{\beta }}_{k:t}\rangle {\langle {\mathit{\beta }}_{k:t}\rangle }^T$ and it may be evaluated from q(B_k) with the mean and variance associated with time t.
• The variational distribution for the binary indicator Z_k(s_it) may be updated as

  $$q(Z_k({\mathit{s}}_{\mathit{\text{it}}})=1)=\frac{1}{1+\text{exp}(-{\rho }_{\mathit{\text{kit}}})}$$ (36)

  with

  $${\rho }_{\mathit{\text{kit}}}={\displaystyle \prod _{l<k}}(1-\langle Z_l({\mathit{s}}_{\mathit{\text{it}}})\rangle )\text{log}p({\nu }_{\mathit{\text{it}}}|{\mathit{\lambda }}_{\mathit{\text{kt}}}^{*})-{\displaystyle \sum _{k\prime >k}}\langle Z_{k\prime }({\mathit{s}}_{\mathit{\text{it}}})\rangle {\displaystyle \prod _{\begin{array}{c}l<k\prime \hfill \\ l\ne k\hfill \end{array}}}(1-\langle Z_l({\mathit{s}}_{\mathit{\text{it}}})\rangle )\text{log}p({\nu }_{\mathit{\text{it}}}|{\mathit{\lambda }}_{k\prime t}^{*})+{\displaystyle \sum _{j=1}^J}\langle {\beta }_{\mathit{\text{kjt}}}\rangle 𝒦({\mathit{s}}_{\mathit{\text{it}}},{\mathit{s̃}}_{\mathit{j}};{\psi }_k)+\langle {\beta }_{k0t}\rangle$$

  where $\text{log }p({\nu }_{\mathit{\text{it}}}|{\mathit{\lambda }}_{\mathit{\text{kt}}}^{*})$ is the data log-likelihood from the Poisson distribution such that $\text{log }p({\mathit{v}}_{\mathit{\text{it}}}|{\mathit{\lambda }}_{\mathit{\text{kt}}}^{*})=\text{log }({\displaystyle {\prod }_{j=1}^d}\text{poisson}({\mathit{\nu }}_{\mathit{\text{ijt}}}|{\lambda }_{\mathit{\text{kjt}}}^{*}))$, and the expectation 〈β_kjt〉 can be obtained from q(B_k).
• Due to the non-conjugacy of the sigmoid function, we cannot acquire a variational distribution for ψ_k. However, we can sample it from its posterior distribution by establishing a discrete set of potential kernel widths ${\{{\psi }_l^{*}\}}_{l=1,\cdots ,L}$. The posterior distribution for each ψ_k is represented as

  $$p({\psi }_k={\psi }_l^{*})\propto \text{exp}\{{\displaystyle \sum _{t=1}^T}{\displaystyle \sum _{i=1}^M}\langle w_k({\mathit{s}}_{\mathit{\text{it}}})\rangle \langle \text{log }\sigma (g_k^l({\mathit{s}}_{\mathit{\text{it}}}))\rangle \}\times \text{exp}\{{\displaystyle \sum _{t=1}^T}{\displaystyle \sum _{i=1}^M}{\displaystyle \sum _{k\prime >k}}\langle w_{k\prime }({\mathit{s}}_{\mathit{\text{it}}})\rangle \langle \text{log}(1-\sigma (g_k^l({\mathit{s}}_{\mathit{\text{it}}})))\rangle \},$$ (37)

  where $g_k^l({\mathit{s}}_{\mathit{\text{it}}})={\displaystyle {\sum }_{j=1}^J}{\beta }_{\mathit{\text{kjt}}}𝒦({\mathit{s}}_{\mathit{\text{it}}},{\mathit{s̃}}_j;{\psi }_l^{*})+{\beta }_{k0t}$. The detailed calculations of $\langle \text{log }\sigma (g_k^l({\mathit{s}}_{\mathit{\text{it}}}))\rangle \text{ and }\langle \text{log }(1-\sigma (g_k^l({\mathit{s}}_{\mathit{\text{it}}})\left)\right)\rangle$ can be found in Ren et al. (2011).
• The variational distribution for c₀ may be updated as

  $$q(c_0)=\text{Gamma }(c_0;{ã}_0,{\mathrm{b̃}}_0),$$ (38)

  with ã₀ = a₀+0.5KT(J + 1) and ${\mathrm{b̃}}_0=b_0+0.5{\displaystyle \sum _{k=1}^K}{\displaystyle \sum _{j=0}^J}{\displaystyle \sum _{i=1}^T}{\displaystyle \sum _{l=1}^T}{[{\displaystyle {\sum ̃}_{\mathit{\text{kj}}}^{-1}}]}_{\mathit{\text{il}}}\langle {\mathit{\beta }}_{\mathit{\text{kji}}}{\mathit{\beta }}_{\mathit{\text{kjl}}}\rangle$ with [Σ̃_kj]_il = c₁^|t_i−t_l|.
• The VB lower bound for c₁ may be derived as

  $$ℱ(c_1)=\text{log}{𝒩}_{(0,1)}(c_1;0,1)+{\displaystyle \sum _{k=1}^K}\text{log}𝒩({\mathit{B}}_k;\mathbf{0},{\mathbf{\Omega }}_k)+\text{constant}.$$ (39)

  The point estimate for c₁ can be updated at each VB iteration by maximizing the lower bound ℱ(c₁).
• Since a point estimate of λ_kj: is employed at each VB iteration, the variational distribution for d₀ may be the same as (30)

  $$q(d_0)=\text{Gamma }(d_0;{ã}_0,{\mathrm{b̃}}_0),$$ (40)

  where ã₀ = a₀ + 0.5dKT and ${\mathrm{b̃}}_0=b_0+0.5{\displaystyle \sum _{k=1}^K}{\displaystyle \sum _{j=1}^d}\text{ln}{\mathit{\lambda }}_{\mathit{\text{kj}}:}^{*T}{\mathbf{\Gamma ̃}}_{\mathit{\text{kj}}}^{-1}\text{ln}{\mathit{\lambda }}_{\mathit{\text{kj}}:}^{*}$.
• Similarly, the lower bound for d₁ is

  $$ℱ(d_1)=\text{log}{𝒩}_{(0,1)}(d_1;0,1)+{\displaystyle \sum _{k=1}^K}{\displaystyle \sum _{j=1}^d}\text{log}𝒩({\mathbf{\Lambda }}_{\mathit{\text{kj}}};\mathbf{0},{\mathbf{\Gamma }}_{\mathit{\text{kj}}})+\text{constant}$$ (41)

  and the point estimation for d₁ is obtained by maximizing ℱ(d₁).

By following (33)–(41), the model parameters and GP hyperparameters can be updated iteratively until convergence. In our experiment, we observed fast convergence; typically the relative change of the lower bound reduces to 10⁻⁴ within 100 iterations.