Zigzag path connects two Monte Carlo samplers: Hamiltonian counterpart to a piecewise deterministic Markov process

Abstract

Zigzag and other piecewise deterministic Markov process samplers have attracted significant interest for their non-reversibility and other appealing properties for Bayesian posterior computation. Hamiltonian Monte Carlo is another state-of-the-art sampler, exploiting fictitious momentum to guide Markov chains through complex target distributions. We establish an important connection between the zigzag sampler and a variant of Hamiltonian Monte Carlo based on Laplace-distributed momentum. The position and velocity component of the corresponding Hamiltonian dynamics travels along a zigzag path paralleling the Markovian zigzag process; however, the dynamics is non-Markovian in this position-velocity space as the momentum component encodes non-immediate pasts. This information is partially lost during a momentum refreshment step, in which we preserve its direction but re-sample magnitude. In the limit of increasingly frequent momentum refreshments, we prove that Hamiltonian zigzag converges strongly to its Markovian counterpart. This theoretical insight suggests that, when retaining full momentum information, Hamiltonian zigzag can better explore target distributions with highly correlated parameters by suppressing the diffusive behavior of Markovian zigzag. We corroborate this intuition by comparing performance of the two zigzag cousins on high-dimensional truncated multivariate Gaussians, including a 11,235-dimensional target arising from a Bayesian phylogenetic multivariate probit modeling of HIV virus data.

1. Introduction

Emergence of Monte Carlo methods based on continuous-time, non-reversible processes has been hailed as fundamentally new development (Fearnhead et al. 2018) and has attracted an explosion of interest (Dunson & Johndrow 2020). The two most prominent among such algorithms are the bouncy particle sampler (Bouchard-Côté et al. 2018) and zigzag sampler (Bierkens et al. 2019a), both having roots in the computational physics literature (Peters & de With 2012, Turitsyn et al. 2011). These algorithms draw samples from target distributions by simulating piecewise deterministic Markov processes (PDMP), which move along linear trajectories with instantaneous changes in their velocities occurring at random times according to inhomogeneous Poisson processes.

One known issue with the bouncy particle sampler is its near-reducible behavior in the absence of frequent velocity refreshment (Bouchard-Côté et al. 2018, Fearnhead et al. 2018). In case of a high-dimensional i.i.d. Gaussian, Bierkens et al. (2022) show that the bouncy particle sampler’s optimal performance is achieved when refreshment accounts for as much as 78% of all the velocity changes. Such frequent velocity refreshment can lead to “random-walk behavior,” hurting computational efficiency of the sampler (Neal 2010, Fearnhead et al. 2018, Andrieu & Livingstone 2021).

The zigzag sampler on the other hand is provably ergodic without velocity refreshment (Bierkens et al. 2019b) and appears to have a competitive edge in high-dimensions (Bierkens et al. 2022). Much remains unknown, however, about how these samplers perform relative one another and against other classes of algorithms. Early empirical results, while informative, remain limited by their focus on low-dimensional or synthetic examples. For example, the simulated logistic regression examples in Bierkens et al. (2019a) only have 16 regression coefficients and the most complex examples of Bierkens et al. (2022) comprise 256-dimensional multi-variate Gaussians and spherically symmetric t-distributions. Generating further theoretical and empirical insight on the practical performance of these algorithms thus stands out as one of the most critical research areas (Fearnhead et al. 2018, Bierkens et al. 2019a, Dunson & Johndrow 2020).

This article brings novel insight into the zigzag sampler’s performance by revealing its intimate connection to a version of Hamiltonian Monte Carlo (HMC) (Duane et al. 1987, Neal 2010), another state-of-the-art paradigm for Bayesian computation. HMC exploits an auxiliary momentum variable and simulates Hamiltonian dynamics to guide exploration of the original parameter space. This notion of “guidance” provided by momentum has historically inspired the earliest examples of non-reversible Monte Carlo algorithms (Diaconis et al. 2000). Beyond this analogy and heuristic, however, the precise nature of relation between HMC and non-reversible methods has never been explored. In fact, the PDMP sampler literature has so far mentioned HMC only in passing or used it merely as a computational benchmark (Bierkens et al. 2019a, Bouchard-Côté et al. 2018, Sherlock & Thiery 2021). A faint link between the two paradigms appears in Deligiannidis et al. (n.d.), who observe a univariate randomized HMC to emerge as the first-coordinate marginal of the bouncy particle sampler in a high-dimensional limit. Their weak convergence result, however, only concerns the univariate marginal and offers little insight as to why a randomized HMC may appear as the limit.

We study a less explored version of HMC, based on the momentum variable components $p_i$ having independent Laplace distributions. The corresponding Hamiltonian dynamics follows a zigzag path akin to that of the Markovian zigzag process. We thus call the resulting algorithm zigzag HMC and, to differentiate the dynamics underlying the two samplers, refer to them as Hamiltonian and Markovian zigzag. In other words, except when we explicitly invoke partial momentum refreshment as a theoretical tool, Hamiltonian zigzag refers to the deterministic dynamics that constitutes a proposal generation mechanism for zigzag HMC. Hamiltonian zigzag in a sense is also “Markovian,” although this term is rarely applied to a deterministic process, in the position-momentum space as its future trajectory depends solely on the current state. Hamiltonian zigzag becomes non-Markovian, however, when viewed as dynamics in the position-velocity space, velocity being the time derivative of position.

We establish that Markovian zigzag is essentially Hamiltonian zigzag with “less momentum,” thereby providing a unified perspective to compare the two Monte Carlo paradigms. We consider a partial momentum refreshment for Hamiltonian zigzag, in which $p_i’\text{s}$ retain their signs but have their magnitudes resampled from the exponential distributions. With this refreshment step inserted at every $\text{Δ}t>0$ time interval, Hamiltonian zigzag converges strongly to its Markovian counterpart as $\text{Δ}t\to 0$.

This result has significant implications in the relative performance of the two zigzag algorithms. Markovian zigzag to some extent avoids random-walk behavior by retaining its direction from previous moments; the inertia induced by retaining full momentum information, however, may allow Hamiltonian zigzag to better explore the space when parameters exhibit strong dependency. The intuition is as follows. Along each coordinate, the partial derivative of the log-density depends on other coordinates and its sign can flip back and forth as the dynamics evolves. Such fluctuation leads to inconsistent guidance from the derivative, “pushing” the dynamics to one direction at one moment and to the opposite at the next. However, momentum can help each coordinate of the dynamics keeps traveling in an effective direction without being affected by small fluctuation in the derivative.

The advantage of having full momentum is visually illustrated in Figure 1. After comparable amounts of computation, Hamiltonian zigzag has traversed a high-density region while Markovian zigzag is still slowly diffusing away from the initial position. This difference in behavior also manifests itself in the overall distance traveled by the dynamics (Figure 2). Our observation here suggests that, with its ability to make larger transitions under comparable computational efforts, Hamiltonian zigzag constitutes a more effective transition kernel.

Figure 1:

Trajectories of the first two position coordinates of Hamiltonian zigzag without momentum refreshment (left) and Markovian zigzag (right). The target is a 2¹⁰ = 1,024-dimensional Gaussian, corresponding to a stationary lag-one auto-regressive process with auto-correlation 0.99 and unit marginal variances. Both dynamics are simulated for 10⁵ linear segments, starting from the same position $x_i=-1$ for all $i$ and same random velocity. The line segment colors change from darkest to lightest as the dynamics evolve.

Figure 2:

Squared distance ${\Vert \mathit{x}(t)-{\mathit{x}}_0\Vert }^2$ of the two zigzag dynamics from the initial position, plotted as function of the number of velocity change events. The experimental setup is identical to that of Figure 1. The dashed line indicates the expected squared distance between the initial position and an independent sample from the target, as a benchmark of the distance traveled by an efficient transition kernel.

We empirically quantify the superior performance of zigzag HMC over Markovian zigzag on a range of truncated Gaussians, a special yet practically relevant class of targets on which we can efficiently simulate both zigzags. One of our examples arises from a Bayesian phylogenetic multivariate probit model for studying correlation structure among binary biological traits of HIV viruses (Zhang et al. 2021). This is a real-world high-dimensional problem in which the super-linear scaling property of Markovian zigzag established by Bierkens et al. (2019a) does not hold — with the number of parameters growing proportional to that of observations, their sub-sampling and control variate techniques cannot be applied here. The same reasons hamper a use of the sub-sampling approach by Bouchard-Côté et al. (2018) based on the local bouncy particle sampler (Bardenet et al. 2017).

The deterministic nature of its dynamics endows Hamiltonian zigzag with a couple of additional advantages over its Markovian counterpart as a building block for sampling algorithms for truncated Gaussians. First, we can combine Hamiltonian zigzag with the no-U-turn algorithm to yield an effectively tuning-free sampler (Section 4.1). Hamiltonian zigzag is further applicable whenever a subset of parameters is conditionally distributed as truncated Gaussian; the split HMC framework (Shahbaba et al. 2014, Nishimura et al. 2020) allows us to combine Hamiltonian zigzag with other integrators to jointly update all the parameters. The split HMC extension is successfully deployed by Zhang et al. (2023), who additionally demonstrates zigzag HMC’S advantage over the bouncy particle sampler. These samplers based on Hamiltonian zigzag are provided as a part of the Bayesian phylogenetic software beast (Suchard et al. 2018) and are also available via the R package hdtg (Zhang et al. 2022). Finally, while Hamiltonian zigzag’s use outside truncated Gaussians is beyond the scope of this article, we can in principle apply zigzag HMC to any targets by approximating the dynamics through the coordinate-wise integrator of Nishimura et al. (2020) or through the mid-point integrator of Chin & Nishimura (2024) (Supplement Section S4).

While this work focuses on the two zigzags, their relationship as presented here seems to hint at HMC’S more general connection to PDMP and other non-reversible methods. We explore, and to some extent substantiate, this conjecture in the Discussion section. In particular, we construct a discrete space analogue of Hamiltonian dynamics which, when combined with partial momentum refreshments, recovers the non-reversible algorithm of Diaconis et al. (2000). Our result on the two zigzags, therefore, portends broader implications in advancing our understanding of HMC and other non-reversible methods.

2. Zigzag Hamiltonian Monte Carlo

2.1. Hamiltonian dynamics based on Laplace momentum

In order to sample from the parameter of interest $\mathit{x}$, HMC introduces an auxiliary momentum variable $\mathit{p}$ and targets the augmented distribution $\pi (\mathit{x},\mathit{p}):={\pi }_X(\mathit{x}){\pi }_P(\mathit{p})$. HMC explores the augmented space by simulating Hamiltonian dynamics, whose evolution is governed by the differential equation known as Hamilton’s equation:

$$\frac{\text{d}\mathit{x}}{\text{d}t}=\nabla K(\mathit{p}),\frac{\text{d}\mathit{p}}{\text{d}t}=-\nabla U(\mathit{x}),$$ (1)

where $U(\mathit{x})=-\text{log}{\pi }_X(\mathit{x})$ and $K(\mathit{p})=-\text{log}{\pi }_P(\mathit{p})$ are referred to as potential and kinetic energy. The solution of (1) preserves the target $\pi (\mathit{x},\mathit{p})$ and, if it can be simulated exactly, can be deployed as a transition kernel. Such exact simulation is infeasible in most applications, but a numerical approximation by a reversible integrator constitutes a valid Metropolis-Hastings proposal (Fang et al. 2014). To this day, the original version of HMC by Duane et al. (1987), based on Gaussian-distributed momentum and the leapfrog integrator, dominates practice (Salvatier et al. 2016, Carpenter et al. 2017).

For the purpose of dealing with discontinuous target densities, Nishimura et al. (2020) proposes the use of Laplace momentum ${\pi }_P(\mathit{p})\propto \text{exp}\left(-\sum _i\left|p_i\right|\right)$ in HMC. The authors then proceed to develop a reversible integrator that qualitatively approximates the corresponding Hamiltonian dynamics, noting only in passing the similarity of its trajectories to those of Markovian zigzag. Under Laplace momentum, Hamilton’s equation becomes

$$\frac{\text{d}\mathit{x}}{\text{d}t}=\text{sign}(\mathit{p}),\frac{\text{d}p}{\text{d}t}=-\nabla U(\mathit{x}),$$ (2)

in which the velocity $\mathit{v}:=\text{d}\mathit{x}/\text{d}t\in {\{\pm 1\}}^d$ depends only on the sign of $\mathit{p}$ and thus remains constant except when one of the $p_i’\text{s}$ undergoes a sign change. This property yields a piecewise linear trajectory as follows.

We momentarily ignore a mathematical technicality that arises from the discontinuity of the sign function in (2). Starting from the state $(\mathit{x}(\tau )$, $\mathit{p}(\tau ))$ with ${\mathit{p}}_i(\tau )\ne 0$ for all $i$ and $\mathit{v}=\text{sign}(\mathit{p})$, the dynamics according to (2) evolves as

$$\mathit{x}(\tau +t)=\mathit{x}(\tau )+t\mathit{v}(\tau ),$$ (3)

$$p_i(\tau +t)=p_i(\tau )-{\int }_0^t{\partial }_{i}U(\mathit{x}(\tau +s))\text{d}s\text{for all}i,$$ (4)

where (3) holds as long as the signs of $p_i$, and hence $\mathit{v}$, remain constant on $[\tau ,\tau +t)$. During this time, the momentum magnitude evolves as

$$\left|p_i\right|(\tau +t)=\left|p_i\right|(\tau )-{\int }_0^t{v}_i(\tau ){\partial }_{i}U(\mathit{x}(\tau )+s\mathit{v}(\tau ))\text{d}s\text{for all}i.$$ (5)

From (4) and (5), we see that the sign change in $p_i$ occurs at time $\tau +t_i$ where

$$t_i=\underset{t>0}{\text{inf}}\left\{\left|p_i(\tau )\right|={\int }_0^t{v}_i(\tau ){\partial }_{i}U(\mathit{x}(\tau )+s\mathit{v}(\tau ))\text{d}s\right\}.$$ (6)

Let $i^{\ast }={\text{argmin}}_i{t}_i$ denote the first coordinate to experience a sign change. As $p_{i^{\ast }}$ changes its sign at time $\tau +t_{i^{\ast }}$, the velocity $\mathit{v}=\text{d}\mathit{x}/\text{d}t$ undergoes an instantaneous change

$$v_{i^{\ast }}\left(\tau +t_{i^{\ast }}\right)=-v_{i^{\ast }}(\tau ),v_j\left(\tau +t_{i^{\ast }}\right)=v_j(\tau )\text{for}j\ne i^{\ast }.$$ (7)

Afterward, the position component proceeds along a new linear path

$$\mathit{x}(\tau +t)=\mathit{x}\left(\tau +t_{i^{\ast }}\right)+\left(t-t_{i^{\ast }}\right)\mathit{v}\left(\tau +t_{i^{\ast }}\right)\text{for}t\ge t_{i^{\ast }}$$ (8)

until the next sign change event. All the while, the momentum component continues evolving according to (4). We summarize the properties (3)–(8) in the algorithmic description of Hamiltonian zigzag in Section 2.2.

Under mild conditions, which we quantify momentarily, the trajectory of $(\mathit{x}(t),\mathit{p}(t))$ as described satisfies the equation (2) except at the instantaneous moments of sign changes in $p_i’\text{s}$. For a differential equation with discontinuous right-hand side, what constitutes a solution and whether it is unique are delicate and complex questions. A study of existence and uniqueness typically starts by interpreting the equation as a differential inclusion problem (Filippov 1988). More precisely, given a differential equation $\text{d}\mathit{z}/\text{d}t=\mathit{f}(\mathit{z})$ and discontinuity points of $\mathit{f}$, the equality requirement is relaxed to an inclusion of the form $\text{d}\mathit{z}/\text{d}t\in \mathit{F}(\mathit{z})$ for a suitable set $\mathit{F}(\mathit{z})$, such as a convex polytope whose extreme points consist of ${\text{lim}}_{{\mathit{z}}^{\prime }\to \mathit{z}}\mathit{f}\left({\mathit{z}}^{\prime }\right)$. The theory remains incomplete, however, despite years of research effort (Fetecau et al. 2003, Khulief 2013).

As current theory falls short, we will directly establish an existence and uniqueness result for Hamilton’s equation with Laplace momentum. For continuously differentiable $U$, the process (3)–(8) defines a unique trajectory consistent with (2) as long as it stays away from the sets

$$S_i=\left\{(\mathit{x},\mathit{p}):{\partial }_{i}U(\mathit{x})=0,p_i=0\right\}.$$ (9)

If ${\partial }_{i}U\left(\mathit{x}\left(\tau +t_i\right)\right)\ne 0$ at the moment of sign change $p_i\left(\tau +t_i\right)=0$, the relation (4) dictates

$$\underset{t\to t_i^{+}}{\text{lim}}\text{sign}\left(p_i(\tau +t)\right)=-\underset{t\to t_i^{-}}{\text{lim}}\text{sign}\left(p_i(\tau +t)\right),$$

where $t_i^{+}$ and $t_i^{-}$ indicate the right and left limit. Hence, the velocity change and subsequent evolution according to (7) and (8) define a unique trajectory that satisfies Equation (2) for almost every $t\ge 0$ and has its position components continuous in $t$.

In Theorem 2.1 below, we make a convenient assumption to ensure that a trajectory avoids the problematic sets (9) from almost every initial state. While identifying more general conditions is beyond the scope of this work, we believe the dynamics to be well-defined under a much weaker assumption on $U(\mathit{x})$ satisfied by most practical situations. The theorem also establishes the time-reversibility and symplecticity of the dynamics, which together imply that Hamiltonian zigzag preserves the target distribution and thus constitutes a valid transition kernel (Neal 2010, Fang et al. 2014).

Theorem 2.1.

Suppose that $U(\mathit{x})$ is twice continuously differentiable and that the sets $\left\{x:{\partial }_{i}U(\mathit{x})=0\right\}$ comprise differentiable manifolds of dimension at most $d-1$. Then Eq (2) defines a unique dynamics on ${\text{ℝ}}^{2d}$ away from a set of Lebesgue measure zero. More precisely, there is a measure zero set $\text{Ω}\subset {\text{ℝ}}^{2d}$ such that, for all initial conditions $(\mathit{x}(0),\mathit{p}(0))\in {\text{ℝ}}^{2d}\backslash \text{Ω}$, there exists a unique solution that satisfies Eq (2) at almost every $t\ge 0$, that remains in ${\text{ℝ}}^{2d}\backslash \text{Ω}$ for all $t\ge 0$, and whose position component is continuous in $t$. Moreover, the dynamics is time-reversible and symplectic on ${\text{ℝ}}^{2d}\backslash \text{Ω}$.

The assumption of Theorem 2.1 in particular holds for strongly convex $U(\mathit{x})$ since it would imply that $\nabla {\partial }_{i}U={\left({\partial }_1{\partial }_{i}U,\dots ,{\partial }_d{\partial }_{i}U\right)}^{\top }\ne \mathbf{0}$ and, by the implicit function theorem, that $\left\{\mathit{x}:{\partial }_{i}U(\mathit{x})=0\right\}$ is a differentiable manifold of dimension $d-1$ (Spivak 1965). We thus have the following corollary:

Corollary 2.2.

For a twice continuously differentiable, strongly log-concave target ${\pi }_X(\mathit{x})$ and the corresponding potential energy $U(\mathit{x})=-\text{log}{\pi }_X(x)$, the Hamiltonian dynamics based on Laplace momentum is well-defined, time-reversible, and symplectic away from a set of Lebesgue measure zero.

We note that, the exact nature of Hamiltonian zigzag on smooth targets being auxiliary to their work, Nishimura et al. (2020) establish corresponding results only under a piecewise constant $U$ with piecewise linear discontinuity set. Given the incomplete theory behind discontinuous differential equations and non-smooth Hamiltonian mechanics, Theorem 2.1 is significant on its own and of independent interest. We defer the proof to Supplement Section S1, however, to keep the article’s focus on the connection between Hamiltonian and Markovian zigzag and its implication on Monte Carlo simulation. We similarly defer to Supplement Section S2 a discussion of the theory’s extension to accommodate constraints on the parameter space, which justifies our application of Hamiltonian zigzag to truncated Gaussian targets (Section 4).

2.2. Simulation of Hamiltonian zigzag dynamics

To summarize the above discussion and prepare for the subsequent discussion of Hamiltonian zigzag’s connection to Markovian one, we describe the evolution of Hamiltonian zigzag in the space $(\mathit{x},\mathit{v},|\mathit{p}|)$ with $p_i=v_i\left|p_i\right|$ as follows. We denote the $k$-th event time by ${\tau }^{(k)}$ for $k\ge 1$ and the corresponding state at the moment by $\left({\mathit{x}}^{(k)},{\mathit{v}}^{(k)},|\mathit{p}{|}^{(k)}\right)=\left(\mathit{x}\left({\tau }^{(k)}\right),\mathit{v}\left({\tau }^{(k)}\right),|\mathit{p}|\left({\tau }^{(k)}\right)\right)$. From a given initial condition at time ${\tau }^{(0)}$, the position coordinate follows a piecewise linear path segmented by the event times ${\tau }^{(k)}$, in-between which the dynamics evolves according to

$$\begin{array}{c}\mathit{x}\left({\tau }^{(k)}+t\right)={\mathit{x}}^{(k)}+t{\mathit{v}}^{(k)},\mathit{v}\left({\tau }^{(k)}+t\right)=v^{(k)},\text{and}\\ \left|p_i\right|\left({\tau }^{(k)}+t\right)={\left|p_i\right|}^{(k)}-{\int }_0^t{v}_i^{(k)}{\partial }_{i}U\left({\mathit{x}}^{(k)}+s{\mathit{v}}^{(k)}\right)\text{d}s.\end{array}$$

The $(k+1)$-th event occurs at time

$${\tau }^{(k+1)}={\tau }^{(k)}+\underset{i}{\text{min}}{t}_i^{(k)}\text{where}{t}_i^{(k)}=\underset{t>0}{\text{inf}}\left\{{\left|p_i\right|}^{(k)}={\int }_0^t{v}_i^{(k)}{\partial }_{i}U\left(x^{(k)}+s{v}^{(k)}\right)\text{d}s\right\},$$

resulting in an instantaneous change in the $i^{\ast }$-th component of velocity for $i^{\ast }(k)={\text{argmin}}_i{t}_i^{(k)}$:

$$v_{i^{\ast }}^{(k+1)}=-v_{i^{\ast }}^{(k)}\text{and}{v}_j^{(k+1)}=v_j^{(k)}\text{for}j\ne i^{\ast }.$$

The position and momentum magnitude at time ${\tau }^{(k+1)}$ are given by

$$\begin{array}{c}{\mathit{x}}^{(k+1)}={\mathit{x}}^{(k)}+\left({\tau }^{(k+1)}-{\tau }^{(k)}\right){\mathit{v}}^{(k)}\text{and}\\ {\left|p_i\right|}^{(k+1)}={\left|p_i\right|}^{(k)}-{\int }_0^{{\tau }^{(k+1)}-{\tau }^{(k)}}{v}_i^{(k)}{\partial }_{i}U\left({\mathit{x}}^{(k)}+s{\mathit{v}}^{(k)}\right)\text{d}s.\end{array}$$

The dynamics then continues in the same manner for the next interval $\left[{\tau }^{(k+1)},{\tau }^{(k+2)}\right)$.

Algorithm 1 summarizes the trajectory simulation process as pseudo-code. For the moment, we do not concern ourselves with either how we would solve for $t_i$ in Line 7 or how to evaluate the integrals in Line 12 and 16. As we demonstrate in Section 4, we can exploit the analytical solutions available under (truncated) multivariate Gaussians for a highly efficient implementation (Supplement Section S5).

Algorithm 1.

Hamiltonian zigzag trajectory simulation for $t\in [0,T]$

1:	function HamiltonianZigzag $(\mathit{x},\mathit{p},T)$
2:	$\tau \leftarrow 0$
3:	$\mathit{v}\leftarrow \text{sign}(\mathit{p})$
4:	while $\tau <T$ do
5:	for $i=1,\dots ,d$ do
6:
7:	$\begin{array}{c}{t}_i=\underset{t>0}{\text{inf}}\{\left|p_i\right|=\\ {\int }_0^t{v}_i{\partial }_{i}U(x+sv)\text{d}s\}\end{array}$
8:	$t^{\ast }\leftarrow {\text{min}}_i{t}_i$
9:	if $\tau +t^{\ast }>T$ then
10:	# No further event occurred
11:	$\mathit{x}\leftarrow \mathit{x}+(T-\tau )\mathit{v}$
12:	$\mathit{p}\leftarrow \mathit{p}-{\int }_0^{T-\tau }\nabla U(\mathit{x}+s\mathit{v})\text{d}s$.
13:	$\tau \leftarrow T$
14:	else
15:	$\mathit{x}\leftarrow \mathit{x}+t^{\ast }\mathit{v}$
16:	$\mathit{p}\leftarrow \mathit{p}-{\int }_0^{t^{\ast }}\nabla U(\mathit{x}+s\mathit{v})\text{d}s$.
17:	$i^{\ast }\leftarrow {\text{argmin}}_i{t}_i$
18:	$v_{i^{\ast }}\leftarrow -v_{i^{\ast }}$
19:	$\tau \leftarrow \tau +t^{\ast }$
20:	return $(\mathit{x},\mathit{p})$

Algorithm 2.

Markovian zigzag trajectory simulation for $t\in [0,T]$

1:	function MarkovianZigzag $(x,v,T)$
2:	$\tau \leftarrow 0$
3:
4:	while $\tau <T$ do
5:	for $i=1,\dots ,d$ do
6:	$u_i\sim \text{Unif}(0,1)$
7:	$\begin{array}{c}{t}_i=\underset{t>0}{\text{inf}}\{-\text{log}{u}_i=\\ {\int }_0^t{\left[v_i{\partial }_{i}U(\mathit{x}+s\mathit{v})\right]}^{+}\text{d}s\}\end{array}$
8:	$t^{\ast }\leftarrow {\text{min}}_i{t}_i$
9:	if $\tau +t^{\ast }>T$ then
10:	# No further event occurred
11:	$\mathit{x}\leftarrow \mathit{x}+(T-\tau )\mathit{v}$
12:
13:	$\tau \leftarrow T$
14:	else
15:	$\mathit{x}\leftarrow \mathit{x}+t^{\ast }\mathit{v}$
16:
17:	$i^{\ast }\leftarrow {\text{argmin}}_i{t}_i$
18:	$v_{i^{\ast }}\leftarrow -v_{i^{\ast }}$
19:	$\tau \leftarrow \tau +t^{\ast }$
20:	return $(\mathit{x},\mathit{v})$

3. Link between Hamiltonian and Markovian zigzags

3.1. Hamiltonian zigzag’s apparent similarity to Markovian zigzag

The Markovian zigzag process by Bierkens et al. (2019a) follows a piecewise linear trajectory similar to Hamiltonian zigzag, but without any apparent concept of momentum. Starting from the state $(\mathit{x}(\tau ),\mathit{v}(\tau ))$ with $\mathit{v}\in {\{\pm 1\}}^d$, Markovian zigzag follows a linear path

$$\mathit{x}(\tau +t)=\mathit{x}(\tau )+t\mathit{v}(\tau ),\mathit{v}(\tau +t)=\mathit{v}(\tau )$$

for $t\ge 0$, until the next velocity switch event $v_i\leftarrow -v_i$ that occurs with Poisson rate

$${\text{λ}}_i(\mathit{x},\mathit{v})={\left[v_i{\partial }_{i}U(\mathit{x})\right]}^{+}:=\text{max}\left\{0,v_i{\partial }_{i}U(\mathit{x})\right\}{.}^1$$ (10)

In particular, the next event time $\tau +t^{\ast }$ can be simulated by setting $t^{\ast }={\text{min}}_i{t}_i$ where

$$t_i=\underset{t>0}{\text{inf}}\left\{-\text{log}{u}_i={\int }_0^t{\left[v_i(\tau ){\partial }_{i}U(\mathit{x}(\tau )+s\mathit{v}(\tau ))\right]}^{+}\text{d}s\right\}\text{for}{u}_i\sim \text{Unif}(0,1).$$ (11)

At time $\tau +t^{\ast }$, the velocity undergoes an instantaneous change

$$v_{i^{\ast }}\left(\tau +t_{i^{\ast }}\right)=-v_{i^{\ast }}(\tau ),v_j\left(\tau +t_{i^{\ast }}\right)=v_j(\tau )\text{for}j\ne i^{\ast }=\underset{i}{\text{argmin}}{t}_i.$$

The position component then proceeds along a new linear path

$$\mathit{x}(\tau +t)=\mathit{x}\left(\tau +t_{i^{\ast }}\right)+\left(t-t_{i^{\ast }}\right)\mathit{v}\left(\tau +t_{i^{\ast }}\right)\text{for}t\ge t_{i^{\ast }}$$

until the next sign change event. The Markovian zigzag process as described has the stationary distribution $\pi (\mathit{x},\mathit{v}):=2^{-d}{\pi }_X(\mathit{x})$, with $\mathit{v}$ distributed uniformly on ${\{\pm 1\}}^d$.

Algorithm 2 describes the dynamics of Markovian zigzag in pseudo-code with empty lines inserted as appropriate to facilitate comparison with the dynamics of Hamiltonian zigzag. The similarity between the two zigzags is striking. The main difference lies in Lines 6 and 7 of the algorithms, reflecting the formulae (11) and (6) for their respective velocity switch event times. For one thing, Markovian zigzag’s event time depends on the random quantity $u_i$ while Hamiltonian zigzag’s is deterministic. On the other hand, when combining Hamiltonian zigzag with momentum refreshment, the distributional equality $-\text{log}{u}_i\stackrel{d}{=}\left|p_i\right|\sim \text{Exp}(\text{scale}=1)$ makes the quantities $-\text{log}{u}_i$ and $\left|p_i\right|$ comparable in a sense and, as we will show in Section 3.2, is a key element connecting the two zigzags.

The quantities $-\text{log}{u}_i$ and $\left|p_i\right|$ being comparable, the only remaining difference between (11) and (6) is the presence and absence of the positive part operator ${[\cdot ]}^{+}$ in the integrands. These presence and absence of ${[\cdot ]}^{+}$ are manifestations of the fact that Markovian zigzag is memory-less while Hamiltonian zigzag transfers energy between the potential and kinetic parts and encodes this information in momentum. The etiology and consequence of this difference is most easily seen in the case of a one-dimensional unimodal target $U(x)$, as visually illustrated in Figure 3. Before a velocity switch event, a zigzag trajectory from the initial position $x_0$ and velocity $v_0$ satisfies

$$\begin{array}{l}{\int }_0^t{\left[v_0{\partial }_{i}U\left(x_0+s{v}_0\right)\right]}^{+}\text{d}s=\{\begin{array}{l}0\text{for}0\le t\le t_{\text{min}}\hfill \\ U\left(x_0+t{v}_0\right)-U_{\text{min}}\text{for}t>t_{\text{min}},\hfill \end{array}\\ {\int }_0^t{v}_0{\partial }_{i}U\left(x_0+s{v}_0\right)\text{d}s=U\left(x_0+t{v}_0\right)-U\left(x_0\right)\\ =U\left(x_0+t{v}_0\right)-U_{\text{min}}-\left[U\left(x_0\right)-U_{\text{min}}\right],\\ \text{where}{t}_{\text{min}}:=\underset{t\ge 0}{\text{argmin}}U\left(x_0+t{v}_0\right)\text{and}{U}_{\text{min}}:=U\left(x_0+t_{\text{min}}{v}_0\right).\\ \end{array}$$

Figure 3:

Comparison of Markovian (left) and Hamiltonian (right) zigzag trajectories under the one-dimensional potential $U(x)$. Neither zigzag is affected by velocity switch events while going down the potential energy hill, during which the velocity and gradient point in the opposite directions and the relation $v(t)\partial U(x(t))<0$ holds. During this time, Hamiltonian zigzag stores up kinetic energy converted from potential energy, while Markovian zigzag remains memory-less. Once the trajectories reaches the potential energy minimum $U_{\text{min}}$ at time $t_{\text{min}}:={\text{argmin}}_{t\ge 0}U\left(x_0+t{v}_0\right)$ and start going “uphill,” the accumulated momentum $\left|p\left(t_{\text{min}}\right)\right|=\left|p_0\right|+U\left(x_0\right)-U_{\text{min}}$ keeps Hamiltonian zigzag traveling in the same direction longer than Markovian zigzag. The last statement technically holds only “on average” due to randomness in the realized values of $\left|p_0\right|\stackrel{d}{=}-\log u$.

The velocity switch event formulae (11) and (6) therefore simplify to

$$\begin{array}{c}{t}^{\text{M}}=\underset{t\ge t_{\text{min}}}{\text{inf}}\left\{-\text{log}u=U\left(x_0+t{v}_0\right)-U_{\text{min}}\right\},\\ t^{\text{H}}=\underset{t\ge t_{\text{min}}}{\text{inf}}\left\{U\left(x_0\right)-U_{\text{min}}+\left|p_0\right|=U\left(x_0+t{v}_0\right)-U_{\text{min}}\right\}.\end{array}$$ (12)

From (12), we see that the Markovian event necessarily precedes the Hamiltonian one (i.e. $t^{\text{M}}\le t^{\text{H}}$) when $\left|p_0\right|=-\text{log}u$ and hence $\left|p\left(t_{\text{min}}\right)\right|=U\left(x_0\right)-U_{\text{min}}+\left|p_0\right|\ge -\text{log}u$.

In higher dimensions, the same reasoning applies to the relative behavior of the two zigzags along each coordinate. On average, the memory-less property as manifested by the presence of ${[\cdot ]}^{+}$ in (11) causes Markovian zigzag to experience more frequent velocity switch events and travel shorter distances along each linear segment. In contrast, when a coordinate of Hamiltonian zigzag is going down potential energy hills (i.e. $v_i{\partial }_{i}U(\mathit{x})<0$), the decrease in potential energy causes an equivalent increase in kinetic energy and in momentum magnitude as dictated by the relation (5). Hamiltonian zigzag can then use this stored kinetic energy to continue traveling in the same direction for longer distances.

3.2. Markovian zigzag as an infinite momentum refreshment limit

We now consider a version of Hamiltonian zigzag in which we periodically refresh the momentum by resampling their magnitudes $\left|p_i(\tau )\right|\sim \text{Exp}(1)$ while keeping their signs. This process follows a zigzag path as before, but its inter-event times are now random. We see from the formula of (6) that, following a momentum magnitude refreshment at time $\tau$, the velocity flip $v_i\leftarrow -v_i$ occurs during the interval $[\tau ,\tau +\text{Δ}t]$ if and only if

$$\left|p_i(\tau )\right|\le \underset{0\le t\le \text{Δ}t}{\text{max}}\left[{\int }_0^t{v}_i(\tau ){\partial }_{i}U(\mathit{x}(\tau +s))\text{d}s\right].$$

Provided that ${\partial }_{i}U$ is continuous and ${\partial }_{i}U(x(\tau ))\ne 0$, the sign of ${\partial }_{i}U(\mathit{x}(\tau +s))$ stays constant on the interval $s\in [\tau ,\tau +\text{Δ}t]$ for sufficiently small $\text{Δ}t$, so that

$$\underset{0\le t\le \text{Δ}t}{\text{max}}\left[{\int }_0^t{v}_i(\tau ){\partial }_{i}U(\mathit{x}(\tau +s))\text{d}s\right]<0\text{or}={\int }_0^{\text{Δ}t}{\left[v_i(\tau ){\partial }_{i}U(\mathit{x}(\tau +s))\right]}^{+}\text{d}s.$$

Under these conditions, the probability of the $i$-th velocity flip is therefore

$$\begin{array}{c}\text{ℙ}\{\left|p_i(\tau )\right|\le \underset{0\le t\le \text{Δ}t}{\text{max}}\left[{\int }_0^t{v}_i(\tau ){\partial }_{i}U(\mathit{x}(\tau +s))\text{d}s\right]\}\\ =\text{ℙ}\left\{\left|p_i(\tau )\right|\le {\int }_0^{\text{Δ}t}{\left[v_i(\tau ){\partial }_{i}U(\mathit{x}(\tau +s))\right]}^{+}\text{d}s\right\}\\ =1-\text{exp}\left\{-{\int }_0^{\text{Δ}t}{\left[v_i(\tau ){\partial }_{i}U(\mathit{x}(\tau +s))\right]}^{+}\text{d}s\right\}\\ ={\left[v_i(\tau ){\partial }_{i}U(\mathit{x}(\tau ))\right]}^{+}\text{Δ}t+O\left(\text{Δ}{t}^2\right).\end{array}$$ (13)

Equation (13) shows that, immediately following the momentum magnitude refreshment, an i-th velocity switch event for Hamiltonian zigzag happens at a rate essentially identical to that of Markovian zigzag as given in (10).

Now consider resampling the momentum magnitude after every time interval of size $\text{Δ}t$ and letting $\text{Δ}t\to 0$. Our analysis above suggests that, under this limit, the rate of coordinate-wise velocity switch events for Hamiltonian zigzag converges to ${\text{λ}}_i(\mathit{x},\mathit{v})={\left[v_i{\partial }_{i}U(\mathit{x})\right]}^{+}$. That is, Hamiltonian zigzag becomes equivalent to Markovian zigzag under this infinite momentum refreshment limit.

We now turn the above intuition into a rigorous argument. In Theorem 3.1 below, $D[0,\infty )$ denotes the space of right-continuous-with-left-limit functions from $[0,\infty )$ to ${\text{ℝ}}^d\times {\text{ℝ}}^d$ endowed with Skorokhod topology, the canonical space to study convergence of stochastic processes with jumps (Billingsley 1999, Ethier & Kurtz 2005). In particular, the convergence $\left({\mathit{x}}_{\text{Δ}t},{\mathit{v}}_{\text{Δ}t}\right)\to (\mathit{x},\mathit{v})$ in this space implies the convergence of the ergodic average $T^{-1}{\int }_0^{T}f\left({\mathit{x}}_{\text{Δ}t}(t)\right)\text{d}t\to T^{-1}{\int }_0^{T}f(\mathit{x}(t))\text{d}t$ for any continuous real-valued function $f$.

Theorem 3.1 (Weak convergence).

Given an initial position $\mathit{x}(0)$ and velocity $\mathit{v}(0)\in {\{\pm 1\}}^d$, consider Hamiltonian zigzag dynamics with $p_i(0)=v_i(0)\left|p_i(0)\right|$ with $\left|p_i(0)\right|\sim \text{Exp}(1)$ and with momentum magnitude resampling at every $\text{Δ}t$ interval, i.e. $\left|p_i(n\text{Δ}t)\right|\sim \text{Exp}(1)$ for $n\in {\text{ℤ}}^{+}$. For each $\text{Δ}t$, let $\left({\mathit{x}}_{\text{Δ}t},{\mathit{p}}_{\text{Δ}t}\right)$ denote the corresponding dynamics and $\left({\mathit{v}}_{\text{Δ}t}\right)$ the right-continuous modification of velocity $\text{d}{\mathit{x}}_{\text{Δ}t}/\text{d}t=\text{sign}({\mathit{p}}_{\text{Δ}t})$. Then, as $\text{Δ}t\to 0$, the dynamics $\left({\mathit{x}}_{\text{Δ}t},{\mathit{v}}_{\text{Δ}t}\right)$ converges weakly to the Markovian zigzag process in $D[0,\infty )$.

This weak convergence result characterizes Markovian zigzag as a special case, albeit only in the limit, of Hamiltonian zigzag. As such, it almost guarantees that Hamiltonian zigzag, if combined with optimal momentum refreshment schedule, will outperform Markovian zigzag. At a more practical level, the interpretation of Markovian zigzag as Hamiltonian zigzag with less momentum pinpoints the cause of dramatic differences in their efficiency observed in our numerical examples (Section 4).

En route to the weak convergence result, we in fact establish a stronger convergence in probability via explicit coupling of Hamiltonian and Markovian zigzag processes:

Theorem 3.2 (Strong convergence).

The Hamiltonian zigzags with momentum magnitude refreshments, as described in Theorem 3.1, can be constructed so that their position-velocity components $\left({\mathit{x}}_{\text{Δ}t},{\mathit{v}}_{\text{Δ}t}\right)$ converge strongly to the Markovian zigzag in $D[0,T]$ for all $T<\infty$. More precisely, there exists a family of Hamiltonian zigzags $\left({\mathit{x}}_{\text{Δ}t}^{\text{H}},{\mathit{p}}_{\text{Δ}t}^{\text{H}}\right)$ and Markovian zigzags $\left({\mathit{x}}_{\text{Δ}t}^{\text{M}},{\mathit{v}}_{\text{Δ}t}^{\text{M}}\right)$ on the same probability space such that, for any $ϵ>0$ and $T>0$,

$$\underset{\text{Δ}t\to 0}{\lim }\text{ℙ}\left\{{\rho }_T\left[\left({\mathit{x}}_{\text{Δ}t}^{\text{H}},{\mathit{v}}_{\text{Δ}t}^{\text{H}}\right),\left({\mathit{x}}_{\text{Δ}t}^{\text{M}},{\mathit{v}}_{\text{Δ}t}^{\text{M}}\right)\right]>ϵ\right\}=0,$$ (14)

where ${\mathit{v}}_{\text{Δ}t}^{\text{H}}$ is the right-continuous modification of $\text{sign}\left({\mathit{p}}_{\text{Δ}t}^{\text{H}}\right)$ and ${\rho }_T(\cdot ,\cdot )$ is the Skorokhod metric on $[0,T]$. In fact, the two zigzags can be constructed so that

$$\underset{\text{Δ}t\to 0}{\text{lim}}\text{ℙ}\left\{\left({\mathit{x}}_{\text{Δ}t}^{\text{H}},{\mathit{v}}_{\text{Δ}t}^{\text{H}}\right)\equiv \left({\mathit{x}}_{\text{Δ}t}^{\text{M}},{\mathit{v}}_{\text{Δ}t}^{\text{M}}\right)on[0,T]\right\}=1.$$

In the statement above, the distributions of the Hamiltonian zigzags $\left({\mathit{x}}_{\text{Δ}t}^{\text{H}},{\mathit{v}}_{\text{Δ}t}^{\text{H}}\right)$ depend on $\text{Δ}t$ due to momentum magnitude refreshments, but their Markovian counterparts all have the same distribution $\left({\mathit{x}}_{\text{Δ}t}^{\text{M}},{\mathit{v}}_{\text{Δ}t}^{\text{M}}\right)\stackrel{d}{=}\left({\mathit{x}}^{\text{M}},{\mathit{v}}^{\text{M}}\right)$. By Theorem 3.1 of Billingsley (1999), convergence in the sense of (14) implies weak convergence $\left({\mathit{x}}_{\text{Δ}t}^{\text{H}},{\mathit{v}}_{\text{Δ}t}^{\text{H}}\right)\to \left({\mathit{x}}^{\text{M}},{\mathit{v}}^{\text{M}}\right)$ in $D[0,T]$ for any $T>0$ and hence, by Theorem 16.7 of Billingsley (1999), in $D[0,\infty )$. In particular, our Theorem 3.1 follows from our Theorem 3.2, whose proof is in Supplement Section S3.

4. Numerical study: two zigzags duel over truncated multivariate Gaussians

Our theoretical result of Section 3.2 shows Markovian zigzag as essentially equivalent to Hamiltonian zigzag with constant refreshment of momentum magnitude. As we heuristically argue in Section 1 and 3.1, the loss of full momentum information can make Markovian zigzag more prone to random-walk behavior in the presence of strong dependency among parameters. We validate this intuition empirically in this section.

We have so far put aside the issue of numerically simulating zigzag trajectories in practice. The coordinate-wise integrator of Nishimura et al. (2020) provides one way to qualitatively approximate Hamiltonian zigzag. With a suitable modification (Supplement Section S4), the mid-point integrator of Chin & Nishimura (2024) provides another option. For exact simulations, however, both zigzags require computing the times of velocity switch events (Line 7 in Algorithm 1 and 2). Hamiltonian zigzag additionally requires computing the integrals of Line 12 and 16 for updating momentum. Being a Markovian process, Markovian zigzag allows the use of Poisson thinning in determining event times (Bierkens et al. 2022). This fact makes it somewhat easier to simulate Markovian zigzag, while an efficient implementation remains challenging except for a limited class of models (Vanetti et al. 2017).

Here we focus on sampling from a truncated multivariate Gaussian, a special yet practically relevant class of targets, on which we can efficiently simulate both zigzags. Besides simple element-wise multiplications and additions, simulating each linear segment of the zigzags only requires solving $d$ quadratic equations and extracting a column of the Gaussian precision matrix $\mathbf{\text{Φ}}$ (Supplement Section S5). This in particular gives the zigzags a major potential advantage, depending on the structure of $\mathbf{\text{Φ}}$, over other state-of-the-art algorithms for truncated Gaussians that require computationally expensive pre-processing operations involving $\mathbf{\text{Φ}}$ (Pakman & Paninski 2014, Botev 2017). In fact, the numerical results of Zhang et al. (2022) indicate zigzag HMC as a preferred choice over the algorithms of Pakman & Paninski (2014) and Botev (2017) in many high-dimensional applications. Supplement Section S7 provides more detailed discussion of how these algorithms compare in their algorithmic complexities and complement the benchmark of Zhang et al. (2022) with additional numerical results.

We compare performances of the two zigzags on a range of truncated Gaussians, consisting of both synthetic and real-data posteriors. As predicted, Hamiltonian zigzag emerges as a clear winner as dependency among parameters increases.

4.1. Zigzag-Nuts: Hamiltonian zigzag with no-U-turn algorithm

Given the availability of analytical solutions in simulating zigzag trajectories, Markovian zigzag is completely tuning-free in the truncated Gaussian case. Hamiltonian zigzag requires periodic momentum refreshments $p_i\sim$ Laplace(scale = 1) for ergodicity, so the integration time $T$ in-between refreshments remains a user-specified input. On the other hand, being a reversible dynamics, Hamiltonian zigzag can take advantage of the no-U-turn algorithm (nuts) of Hoffman & Gelman (2014) to automatically determine an effective integration time. This way, we only need to supply a base integration time $\text{Δ}T$ to Hamiltonian zigzag — the no-U-turn algorithm will then identify an appropriate integration time $T=2^k\text{Δ}T$, where $k\ge 0$ is the height of a binary trajectory tree at which the trajectory exhibits a U-turn behavior for the first time. We provide in Supplement Section S8 the details of how to combine the no-U-turn algorithm with reversible dynamics in general.

With the automatic multiplicative adjustment of the total integration time, the combined Zigzag-Nuts algorithm only requires us to set $\text{Δ}T$ as a reasonable underestimate of an optimal integration time. Based on the intuition that the integration time should be proportional to a width of the target in the least constrained direction (Neal 2010), we choose $\text{Δ}T$ for Zigzag-Nuts as follows. In the absence of truncation, this width of the target is proportional to ${\nu }_{\text{max}}^{1/2}\left({\mathbf{\text{Φ}}}^{-1}\right)={\nu }_{\text{min}}^{-1/2}(\mathbf{\text{Φ}})$ where ${\nu }_{\text{max}}$ and ${\nu }_{\text{min}}$ denote the largest and smallest eigenvalues, both of which can be computed quickly via a small number of matrix-vector operations $\mathit{w}\to {\mathbf{\text{Φ}}}^{-1}\mathit{w}$ or $\mathit{w}\to \mathbf{\text{Φ}}\mathit{w}$ using the Lanzcos algorithm (Meurant 2006). For the standard HMC based on Gaussian momentum, an optimal integration time on multivariate Gaussian targets is $T=\varpi {\nu }_{\text{min}}^{-1/2}(\mathbf{\text{Φ}})/2$ (Bou-Rabee & Sanz-Serna 2018), where $\varpi \approx$ 3.14 denotes Archimedes’s constant. This suggests that $\text{Δ}T\approx {\nu }_{\text{min}}^{-1/2}(\mathbf{\text{Φ}})$ should be close to the upper end of reasonable base integration times. We hence propose a choice $\text{Δ}T={\nu }_{\text{min}}^{-1/2}(\mathbf{\text{Φ}})\text{Δ}{T}_{\text{rel}}$ for $\text{Δ}{T}_{\text{rel}}\le 1$, where $\text{Δ}{T}_{\text{rel}}$ represents a base integration time relative to the target’s width ${\nu }_{\text{min}}^{-1/2}(\mathbf{\text{Φ}})$.

In our numerical results, we find that $\text{Δ}{T}_{\text{rel}}=0.1$ works well in a broad range of problems. We observe further performance gains from a larger value, i.e. $\text{Δ}{T}_{\text{rel}}>0.1$, if the target is highly constrained. We use $\text{Δ}{T}_{\text{rel}}=0.1$ in this section for simplicity’s sake, but additional numerical results are available in Supplement Section S9.

4.2. Study set-up and efficiency metrics

The existing empirical evaluations of Markovian zigzag rely on simple low-dimensional target distributions; consequently, there is great interest in having its performance tested on more challenging higher-dimensional problems (Dunson & Johndrow 2020). We start from where Bierkens et al. (2022) left off — 256-dimensional correlated Gaussians (without truncation) — and first test the two zigzags on synthetic truncated Gaussians of dimension up to 4,096. We then proceed to a real-world application, comparing the performances of the two zigzags on a 11,235-dimensional truncated Gaussian posterior.

We compare the two zigzags’ performances using effective sample sizes (ESS), a well-established metric for quantifying efficiency of Markov chain Monte Carlo (MCMC) algorithms (Geyer 2011). When assessing a relative performance of two MCMC algorithms, we also need to account for their per-iteration computational costs. We therefore report ESS per unit time, as is commonly done in the literature, where “time” refers to the actual time it takes for our code to run and is not to be confused with the time scales of zigzag dynamics. We note that relative computational speed can vary significantly from one computing environment to another due to various performance optimization strategies used by modern hardware, such as instruction-level parallelism and multi-tiered memory cache (Guntheroth 2016, Nishimura & Suchard 2023, Holbrook et al. 2020). For this reason, while ESS per time is arguably the most practically relevant metric, we consider an alternative platform-independent performance metric in Supplement Section S10.

We run Zigzag-Nuts for 25,000 iterations on each synthetic posterior of Section 4.3. Each iteration of Zigzag-Nuts is more computationally intensive on the real-data posterior of Section 4.4, but also mixes more efficiently than on the hardest synthetic posterior. We hence use a shorter chain of 1,500 iterations on the real-data one. With these chain lengths, we obtain at least 100 ESS along each coordinate in all our examples.

For each Markovian zigzag simulation, we collect MCMC samples spaced at time intervals of size $\text{Δ}T$, the base integration time for Zigzag-Nuts. This way, we sample Markovian zigzag at least as frequently as Hamiltonian zigzag along their respective trajectories. We thus ensure a fair comparison between the two zigzags and, if any, tilt the comparison in favor of Markovian zigzag. To obtain at least 100 ESS along each coordinate, we simulate Markovian zigzag for $T=250,000\times \text{Δ}T$ on each synthetic posterior and $T=1,500\times \text{Δ}T$ on the real-data posterior, generating 250,000 and 1,500 samples respectively.

Note that, while both zigzags can in theory utilize entire trajectories to estimate posterior quantities of interest (Bierkens et al. 2019a, Nishimura & Dunson 2020), such approaches are often impractical in high-dimensional settings due to memory constraints. In the 11,235-dimensional example of Section 4.4, for example, 1,500 iterations of Markovian (Hamiltonian) zigzag undergoes 1.6 × 10⁸ (2.1 × 10⁸) velocity switch events. Storing all these event locations would require 1.8 (2.4) TB in 64-bit double precision, while providing little practical benefit because of their high auto-correlations.

We implement both zigzags (Algorithm S2 and S3 in Supplement Section S5) in the Java programming language as part of the Bayesian phylogenetic software beast; the code and instruction to reproduce the results are available at https://github.com/aki-nishimura/code-for-hamiltonian-zigzag-2024. We run each MCMC on a c5.2xlarge instance in Amazon Elastic Compute Cloud, equipped with 4 Intel Xeon Platinum 8124M processors and 16 GB of memory. For each target, we repeat the simulation 5 times with different seeds and report ESS averaged over these 5 independent replicates. ESS’s are computed using the R CODA package (Plummer et al. 2006).

4.3. Threshold model posteriors under correlated Gaussian priors

Bierkens et al. (2022) consider Gaussian targets with compound symmetric covariance

$$\text{Var}\left(x_i\right)=1,\text{Cov}\left(x_i,x_j\right)=\rho \in [0,1)\text{for}i\ne j.$$ (15)

Such a distribution can be interpreted as a prior induced by a model $x_i={\rho }^{1/2}z+{(1-\rho )}^{1/2}{ϵ}_i$, with shared latent factor $z\sim 𝒩(0,1)$ and individual variations ${ϵ}_i\sim 𝒩(0,1)$. We construct truncated Gaussian posteriors by assuming a threshold model

$$y_i=\mathrm{𝟙}\left\{x_i>0\right\}-\mathrm{𝟙}\left\{x_i\le 0\right\}.$$

An arbitrary thresholding would make it difficult to get any feel of the geometric structure behind the resulting truncated Gaussian posterior. We hence assume a simple thresholding with $y_i=1$ for all $i$, inducing posteriors constrained to the positive orthant $\left\{x_i>0\right\}$. We investigate the effect of degree of correlation on zigzags’ performance by varying the correlation coefficient, using the values $\rho =1-{(0.1)}^k$ for $k=0,1,2$.

The numerical result for the compound symmetric posteriors, including the i.i.d. case $\rho =0$, is summarized in Table 1. Since $x_i$ are exchangeable, we calculate ESS only along the first coordinate. We also calculate ESS along the principal eigenvector of ${\mathbf{\text{Φ}}}^{-1}$ since HMC typically struggles most in sampling from the least constrained direction (Neal 2010). Markovian zigzag face a similar challenge as evidenced by the visual comparison of the two zigzag samples projected onto the principal component (Figure 4).

Table 1:

ESS per computing time — relative to that of Markovian zigzag — under the compound symmetric posteriors. We test the algorithms under three correlation parameter values ($\rho =0$, 0.9, and 0.99) and two varying dimensions ($d=256$ and 1,024). ESS’S are calculated along the first coordinate and along the principal eigenvector of ${\mathbf{\text{Φ}}}^{-1}$, each shown under the labels $"x_1"$ and “PC.”

	Relative Ess per time
Compound symmetric	$\rho =0$	$\rho =0.9$		$\rho =0.99$
	$x_1$	$x_1$	PC	$x_1$	PC
Case: $d=256$
Markovian	1	1	1	1	1
Zigzag-Nuts	0.64	4.5	4.6	41	40
Zigzag-Hmc $\left(T_{\text{rel}}=\sqrt{2}\right)$	5.5	46	66	180	180
Case: $d=1$,024
Zigzag-Nuts	0.57	4.7	4.5	54	54
Zigzag-Hmc $\left(T_{\text{rel}}=\sqrt{2}\right)$	5.6	56	85	300	300

Figure 4:

Traceplot of the zigzag samples from the 1,024-dimensional compound symmetric posterior (15) with $\rho =0.99$, projected onto the principal component $\mathit{u}=(1,\dots ,1)/\sqrt{d}$ via a map $\mathit{x}\to \langle \mathit{x},\mathit{u}\rangle$. The horizontal axis is scaled to represent the number of velocity switch events.

As predicted, Hamiltonian zigzag demonstrates increasingly superior performance over its Markovian counterpart as the correlation increases, delivering 4.5 to 4.7-fold gains in relative ESS at $\rho =0.9$ and 40 to 54-fold gains at $\rho =0.99$. The efficiency gain is generally greater at the higher dimension $d=1,024$. For the i.i.d. case, Hamiltonian zigzag seems to have no advantage. This is in a sense expected since, on an i.i.d. target, both zigzags become equivalent to running $d$ independent one-dimensional dynamics and have no interactions among the coordinates. In other words, Hamiltonian zigzag’s additional momentum plays no role when parameters are independent. Also, for a univariate Gaussian target, Markovian zigzag has been shown to induce negative auto-correlations and thus achieve sampling efficiency above that of independent Monte Carlo (Bierkens & Duncan 2017).

The compound symmetric targets here have a particularly simple correlation structure; the covariance matrix can be written as ${\mathbf{\text{Φ}}}^{-1}=(1-\rho )\mathit{I}+\rho \mathbf{1}{\mathbf{1}}^{\top }$ for $\mathbf{1}=(1,\dots ,1)$, meaning that the probability is tightly concentrated along the principal component and is otherwise distributed symmetrically in all the other directions. This simple structure in particular allows us to manually identify an effective integration time $T$ for Hamiltonian zigzag without too much troubles. We therefore use this synthetic example to investigate how Zigzag-Nuts perform relative to manually-tuned Hamiltonian zigzag, which we denote as “Zigzag-HMC” in Table 1.

For each of the three targets, we try $T={\nu }_{\text{min}}^{-1/2}(\mathbf{\text{Φ}})T_{\text{rel}}$ with $T_{\text{rel}}=2^{k/2}$ for $k=-2,-1,0,1,2$. We report the ESS’s based on $T_{\text{rel}}=2^{1/2}$ in Table 1 as we find this choice to yield the optimal ESS in the majority of cases. We see that manually-optimized Hamiltonian zigzag delivers substantial increases in ESS compared to Zigzag-Nuts. Such efficiency gains are also observed by the authors who proposed alternative methods for tuning HMC (Wang et al. 2013, Wu et al. 2018). The results here indicate that their tuning approaches may be worthy alternatives to Zigzag-Nuts and may further reinforce Hamiltonian zigzag’s advantage over Markovian zigzag.

Finally, given that the zigzags’s motions are restricted to the discrete set of directions $\mathit{v}\in {\{\pm 1\}}^d$ and that the compound symmetric posteriors happen to be concentrated along one of these directions, one may wonder whether this coincidental structure affect the above numerical results. To answer this question, we conduct additional simulations with rotated versions of the compound symmetric posterior, corresponding to the covariance matrices ${\mathbf{\text{Φ}}}^{-1}=(1-\rho )\mathit{I}+\rho \mathit{u}{\mathit{u}}^{\top }$ with the principal components $\mathit{u}\in {ℝ}^d$ drawn uniformly from the $(d-1)$-dimensional sphere. These additional simulations indicate that essentially the same pattern holds in the relative performance of two zigzags and that absolute ESS per time changes little across different rotations of the target (Supplement Section S11). The latter finding reinforces the theoretical results of Bierkens et al. (2023) who show that, under a class of anistropic Gaussian targets, any deviation from diagonal covariance results in diffusive behavior in Markovian zigzag. On the other hand, the inertia provided by full momentum information appears to endow Hamiltonian zigzag with fundamentally different behavior.

4.4. Posterior from phylogenetic multivariate probit model

We now consider a 11,235-dimensional target arising from the phylogenetic multivariate probit model of Zhang et al. (2021). For simplicity’s sake, here we describe the model with some simplifications and refer interested readers to the original work for full details.

The goal of Zhang et al. (2021) is to learn correlation structure among $m=21$ binary biological traits across $n=535$ HIV viruses while accounting for their shared evolutionary history. Their model assumes that, conditional on the bifurcating phylogenetic tree $𝒯$ informed by the pathogen genome sequences, latent continuous biological traits ${\mathit{x}}_1,\dots ,{\mathit{x}}_n\in {ℝ}^m$ follows Brownian motion along the tree with an unknown $m\times m$ diffusion covariance $\mathbf{\text{Γ}}$ (Figure 5). The latent traits $\mathit{x}=\left[{\mathit{x}}_1,\dots ,{\mathit{x}}_n\right]$ map to the binary observation $\mathit{y}\in {ℝ}^{m\times n}$ via the threshold model $y_{ij}=\mathrm{𝟙}\left\{x_{ij}>0\right\}-\mathrm{𝟙}\left\{x_{ij}\le 0\right\}$.

Figure 5:

Example paths of latent biological traits following the phylogenetic diffusion. The traits of two distinct organisms evolve together until a branching event. Beyond that point, the traits evolve independently but with the same diffusion covariance induced by a shared bio-molecular mechanism.

The model gives rise to a posterior distribution on the joint space $(\mathit{x},\mathbf{\text{Γ}},𝒯)$ of latent biological traits, diffusion covariance, and phylogenetic tree. To deal with this complex space, Zhang et al. (2021) deploys a Gibbs sampler, the computational bottleneck of which is updating $\mathit{x}$ from its full conditional — a 11,235 = 21 × 535 dimensional truncated multivariate Gaussian. The $mn\times mn$ precision matrix $\mathbf{\text{Φ}}=\mathbf{\text{Φ}}(\mathbf{\text{Γ}},𝒯)$ of $\mathit{x}\mid \mathbf{\text{Γ}},𝒯$ induced by the phylogenetic Brownian diffusion model changes at each Gibbs iteration, precluding use of sampling algorithms that require expensive pre-processings of $\mathbf{\text{Φ}}$. For the purpose of our simulation, we fix $\mathbf{\text{Γ}}$ at the highest posterior probability sample and $𝒯$ at the maximum clade credibility tree obtained from the prior analysis by Zhang et al. (2021). Our target then is the distribution $\mathit{x}\mid \mathbf{\text{Γ}},𝒯\sim 𝒩\left(\mathit{\mu }(\mathbf{\text{Γ}},𝒯),{\mathbf{\text{Φ}}}^{-1}(\mathbf{\text{Γ}},𝒯)\right)$ truncated to $\{\text{sign}(\mathit{x})=\mathit{y}$}. There are 404 (3.6%) missing entries in $\mathit{y}$ and the target remains unconstrained along these coordinates. The model is parametrized in such a way that the marginal variances of $\mathit{x}\mid \mathbf{\text{Γ}},𝒯$ are comparable across the coordinates; the diagonal preconditioning technique for HMC (Stan Development Team 2018, Nishimura et al. 2020), therefore, plays little role in this application (Supplement Section S12). The target distribution parameters $\mathit{\mu }$, $\mathbf{\text{Φ}}$, and $\mathit{y}$ are available on a Zenodo repository at http://doi.org/10.5281/zenodo.4679720.

On this real-world posterior, Hamiltonian zigzag again outperforms its Markovian counterpart, delivering 6.5-fold increase in the minimum ESS across the coordinates and 19-fold increase in the ESS along the principal eigenvector of ${\mathbf{\text{Φ}}}^{-1}(\mathbf{\text{Γ}},𝒯)$ (Table 2). There is no simple way to characterize the underlying correlation structure for this non-synthetic posterior, so we provide a histogram of the pairwise correlations in Figure 6 as a crude descriptive summary. We in particular find that only 0.00156% of correlations have magnitudes above 0.9. However, the joint structure, truncation, and high-dimensionality apparently make for a complex target, which Hamiltonian zigzag can explore more efficiently by virtue of its full momentum.

Table 2:

Relative ESS per computing time under the phylogenetic probit posterior $(d=\mathrm{11,235})$. The “min” label indicates the minimum ESS across all the coordinates.

Phylogenetic probit	Relative Ess per time
	min	PC
Markovian	1	1
Zigzag-Nuts	6.5	19

Figure 6:

Histogram of the pairwise correlations, i.e. the upper off-diagonal entries of ${\mathbf{\text{Φ}}}^{-1}(\mathbf{\text{Γ}},𝒯)$, in the phylogenetic probit posterior. The vertical axis is in log₁₀ scale. No correlation falls outside the horizontal plot range [−0.5,1.0], with the smallest and largest correlation being −0.416 and 0.989. Out of $mn(mn-1)/2\approx$ 63.1 × 10⁶ correlations, 55.3×10⁶ (87.6%) lie within [−0.2,0.2] and 987 (0.00156%) within [0.9,1].

5. Discussion

In recent years, both piecewise deterministic Markov process (PDMP) samplers and HMC have garnered intense research interests as potential game changers for Bayesian computation. In this article, we established that one of the most prominent PDMP is actually a close cousin of HMC, differing only in the amount of “momentum” they are born with. This revelation provided novel insights into relative performance of the two samplers, demonstrated via the practically relevant case of truncated multivariate Gaussian targets.

The uncovered kinship between two zigzags begs the question: is there a more general relationship between PDMP and HMC? Searching for an affirmative answer to this question would require us to go beyond the current HMC framework based on classical Hamiltonian dynamics. Discontinuous changes in velocity seen in PDMP cannot be imitated by smooth Hamiltonian dynamics; such behavior is possible under Hamiltonian zigzag only because of its non-differentiable momentum distribution.

In a sense, our result shows that HMC’S momentum is really made of two components: direction and magnitude of inertia. PDMP’S velocity on the other hand consists only of direction. We suspect it is possible to introduce a notion of inertia magnitudes to PDMP in the form of auxiliary parameters — these parameters can interact with the main parameters so as to emulate the behavior of HMC, preserving the total log-density and storing inertia obtained from potential energy downhills for later use.

While we leave more thorough investigations for future research, we now demonstrate how the above idea translates into a novel non-reversible algorithm based on Hamiltonian-like dynamics in a discrete space. We illustrate the approach on a cyclic graph $G$ with $n$ vertices, i.e. $n$ discrete states placed along a circle. For $x\in G$, we use $x+1$ to denote the neighbor in the clockwise direction and $x-1$ in the counter-clockwise direction.

We first augment the position space $\{x\in G\}$ with “direction” $v\in \{-1,+1\}$ and “inertia” $|p|\ge 0-$ there is no native notion of “momentum $p$” in this setting, but we use the notation and terminology to draw parallels with Hamiltonian zigzag. On this augmented space, we define discrete Hamiltonian dynamics with associated “potential energy” $U(x)$, whose trajectory $\{x(t),v(t),|p|(t)\}$ evolves as follows. At each time $t\in \text{ℤ}$, the next state is given by

$$\begin{array}{c}x(t+1)=x(t)+v(t),v(t+1)=v(t),\\ \left|p\right|(t+1)=\left|p\right|(t)-U(x(t)+v(t))+U(x(t))\end{array}$$

if $|p|(t)\ge U(x(t)+v(t))-U(x(t))$; otherwise,

$$x(t+1)=x(t),v(t+1)=-v(t),|p|(t+1)=|p|(t).$$

Figure 7 visually illustrates behavior of this discrete dynamics, which has much conceptual similarity to one-dimensional Hamiltonian zigzag as studies in Section 3.1 and illustrated in Figure 3. In particular, the dynamics by design conserves the sum $U(x)+|p|$, with the “inertia” $|p|$ playing the role analogous to that of the kinetic energy $K(p)$ in the continuous case. It is straightforward to verify that the dynamics is reversible and volume-preserving, admits $\pi (x,v,|p|)=\frac{1}{2}{\pi }_X(x)\exp (-|p|)$ for ${\pi }_X(x)\propto \exp (-U(x))$ as an invariant distribution, and thus constitutes a valid transition kernel (Fang et al. 2014, Nishimura et al. 2020).

Figure 7:

Example trajectory of discrete Hamiltonian dynamics on a cyclic graph $G$ along with associated potential energy ${\{U(x)\}}_{x\in G}$. The trajectory initially proceeds counter-clockwise as directed by the velocity $v_0=-1$. It does not have sufficient inertia to reach $x_0-2$, however, and hence reverses its course during the second step. The conservation of $U(x)+|p|$ implies that the trajectory then continues clockwise at least to $x_0+4$.

Now consider adding a “partial momentum refreshment” to the above dynamics, drawing $|p|(t)\sim \text{Exp}(1)$ and flipping the sign of $v(t)$ with probability $r\in [0,1]$ at the beginning of each time step. Remarkably, the position-velocity marginal of this dynamics coincides with the general form of non-reversible algorithms presented in Section 5 of Diaconis et al. (2000). Under a “full momentum refreshment” of drawing $|p|(t)\sim \text{Exp}(1)$ and $v(t)\sim$ Unif $(\{-1,+1\})$, the dynamics reduces to a Metropolis algorithm with symmetric proposals to the neighbors. We can analogously construct Hamiltonian-like dynamics on a more general discrete space, adding partial momentum refreshment to which recovers the fiber algorithm of Diaconis et al. (2000) and its generalization by Herschlag et al. (2020).

Despite using HMC as an inspiration, Diaconis et al. (2000) and subsequent work have failed to recognize the actual HMC analogues lurking behind their non-reversible methods. This is understandable — the notion of momentum from smooth Hamiltonian dynamics does not easily transfer to discrete spaces or non-differentiable paths of PDMP. On the other hand, having established the explicit link between the two zigzags and having identified the dual role of momentum in providing both direction and inertia, we only need a bit of outside-the-box thinking to extend the idea to discrete spaces.

A unified framework for the HMC and other non-reversible paradigms would present a number of opportunities. For example, the introduction of momentum to Herschlag et al. (2020)’s flow-based algorithm could improve its performance by reducing random walk behavior along each flow. There could also be cross-fertilization of ideas between continuous- and discrete-space methods. The elaborate procedures developed in the PDMP literature for choosing the next direction at an event time (Fearnhead et al. 2018, Wu & Robert 2020), for example, is likely applicable to the flow-based algorithm in discrete spaces. All in all, our revelation of the two zigzags’ kinship — and the insights generated from it — will likely catalyze further novel developments in Monte Carlo methods.