Faster Hamiltonian Monte Carlo by learning leapfrog scale: an offline randomized solution

Abstract

We introduce a Hamiltonian Monte Carlo (HMC) methodology based on an offline empirical calibration of randomized leapfrog parameters. The approach, referred to as eHMC, where e stands for empirical, leverages importance sampling to construct an empirical distribution on discretization parameters, thereby eliminating the need for manual burn-in diagnostics and online adaptation. The proposal distribution used in the calibration stage is obtained via a Population Monte Carlo scheme with tempering and relies on flexible parametric variational families such as normalizing flows. Once the calibration stage complete, the resulting algorithm defines a homogeneous Markov chain via a mixture of HMC kernels with a fixed mixing distribution, and hence preserves the target distribution. Numerical experiments indicate that eHMC can achieve competitive or improved sampling efficiency compared to the No-U-Turn Sampler (NUTS) in the case useful integration times can be summarized by the offline distribution. The comparison is assessed by standard efficiency metrics normalized by the number of leapfrog steps during the post-calibration sampling phase.

Introduction

Hamiltonian Monte Carlo (HMC, Duane et al. 1987; Neal 2011) has emerged as an efficient Markov Chain Monte Carlo (MCMC) sampling method, which is particularly adept at dealing with high-dimensional target distributions. The method relies on a deterministic differential flow stemming from Hamiltonian mechanics to produce transitions across the parameter space of an augmented distribution, referred to as Hamiltonian. Such time-continuous dynamics leave the augmented distribution invariant; however, in practice, the latter must be discretized. The transitions are hence computed using a gradient-based symplectic integrator, the most popular of which is the second-order Störmer-Verlet also known as the leapfrog integrator (Hairer et al. 2003). A Metropolis-Hastings acceptance ratio must furthermore be added towards correcting the approximate numerical solution and hence preserving the intended target measure.

Whilst the algorithm theoretically benefits from a fast exploration of the parameter space by accepting large transitions with high probability, this efficiency is marred by its high sensitivity to hand-tuned parameters. These include the step size $\epsilon$ of the discretization scheme, the number of steps L of the integrator, and the covariance matrix M of the auxiliary variables. Indeed, the discretization error resulting from a poorly tuned step size often leads to a low acceptance rate, since the latter is governed by the variation of the Hamiltonian between the initial and final states of the leapfrog path, which is of order ${\epsilon }^2$. Furthermore, calibrating the number of steps of the integrator is paramount to avoid slow mixing chains or to save some computation cost. More specifically, if the value of L is not large enough, HMC may exhibit a random walk behaviour similar to that of standard Metropolis Hastings algorithms, and may thus struggle to properly explore the support of the target. Conversely, if the value of L is too large the associated dynamic may retrace its steps back to a neighbourhood of the initial state (Betancourt et al. 2015). This may result in the algorithm wasting computation efforts and diminishing mixing. Finally, the choice of a well-suited covariance matrix for the auxiliary variable can enhance both the speed and the mixing of HMC. One proposal in this regard is the Riemann manifold approach of Girolami and Calderhead (2011) that adapts the covariance matrix of the auxiliary variables to the curvature of the target distribution, followed by the automatic tuning strategy of Wang et al. (2013).

In terms of calibration concerns, the popular No-U-Turn Sampler (NUTS, Hoffman and Gelman 2014) automatically adjusts both the step size and the number of integration steps. This algorithm achieves at least the same efficiency in terms of effective sample size and expected squared jump distance as compared to the standard HMC, albeit at the cost of increased computational complexity. After selecting the step size $\epsilon$ by primal-dual averaging (Nesterov 2009) during a burn-in phase, NUTS dynamically adapts L at each iteration to determine an integration length adapted to the local Hamiltonian dynamics. This trajectory is obtained through a recursive doubling scheme that follows the Hamiltonian flow forward and backward until a no-U-turn condition is satisfied. The symmetric trajectory construction and proposal random selection yields a valid reversible transition.

Although NUTS dynamically constructs a trajectory whose length is adapted to the local geometry, the proposal is not obtained by deterministically selecting the endpoint of this trajectory. Instead, the next state is sampled from the set of candidate states generated along the trajectory using slice sampling in the original formulation (Hoffman and Gelman 2014) and multinomial sampling in the current Stan implementation. Consequently, the principal computational cost of NUTS lies in constructing the trajectory online at every iteration in order to determine an appropriate integration length. Recent work by Bou-Rabee et al. (2025) has also investigated adaptive calibration within NUTS. They introduce local step-size adaptation steps into NUTS using a Gibbs self-tuning mechanism. Their approach performs local adaptation during the construction of the Hamiltonian trajectory while preserving the reversibility property.

The present work explores the opportunity of learning offline a distribution of integration lengths and exploiting it to define a randomized HMC scheme that recovers part of the benefits of dynamic trajectory adaptation while avoiding the systematic construction of extended trajectories during the sampling phase. Therefore, the proposal potentially reduces the online computational cost, whenever a suitable distribution of integration lengths can be learned from a preliminary, exploratory stage. However, it is not intended to provide a universal replacement for NUTS but rather stands as a mostly automated offline calibration mechanism for randomized HMC in some settings. These cover targets where a reusable distribution of integration parameters offers an adequate summary of the trajectory lengths encountered across the typical set.

The idea of randomizing elements of the Hamiltonian dynamics has already been investigated in the literature. For instance, Bou-Rabee and Sanz-Serna (2017) propose to randomize the integration time according to a continuous distribution, yielding a Markov process with ergodicity properties and avoiding pathological periodic trajectories of standard Hamiltonian-based samplers. Their convergence results establish that randomization of the integration time can fundamentally alter the long-time behavior of the sampler. While they focus on theoretical guarantees for continuous-time randomization mechanisms, our approach targets a practical, offline, calibration of randomized discrete-time HMCs.

In this paper, we construct an empirical joint distribution on the discretization parameters $(\epsilon ,L)$. Our finite calibration phase is exploiting the U-turn criterion of NUTS on the level sets associated with a sample of points that are approximately distributed according to the target distribution. This sample is then used in a second phase to jointly simulate $(\epsilon ,L)$ at each iteration of a regular HMC scheme. This strategy allows us to exploit information on admissible trajectory lengths to randomize the integration parameters directly, rather than targetting the Hamiltonian flow primarily as an intermediate object for proposal selection. Finally, as a by-product of the initial sample points and the ones obtained when visiting the level sets, we can derive a matrix M adapted to the target distribution.

Unlike calibration schemes that leverage a Markovian algorithmic structure (see, e.g., Bou-Rabee et al. 2026) the empirical distribution used in our method is based on independent weighted samples obtained through a Population Monte Carlo method (Cappé et al. 2004), a particular instance of adaptive importance sampling. As a result, the calibration stage is embarrassingly parallel and can readily be distributed across CPU cores. Furthermore, it does not require finding the scale of a burn-in period. The proposals used along this algorithm are adapted to align with an annealing sequence of distributions following Koblents and Míguez (2015). Most crucially, the temperature schedule of this annealing sequence is itself determined automatically by relying on an effective sample size criterion, which avoids the delicate tuning of additional annealing parameters. Our method shares a common feature with other adaptive importance sampling approaches, namely an exponentiation of the importance weights, which enables the transition from an initial proposal distribution to a distribution well fitted to the problem at hand. It therefore relates to annealed importance sampling (Neal 2001), sequential Monte Carlo samplers (Del Moral et al. 2006), and more recent Entropic Mirror Descent approaches (Korba and Portier 2022; Chopin et al. 2024; Cherradi et al. 2026). These methods generally pursue the more ambitious objective of constructing highly efficient proposal distributions. In contrast, the Population Monte Carlo stage in eHMC does not aim at optimizing the proposal as accurately. Our method only needs to produce a sufficiently representative weighted sample from which informative initial points and a pertinent empirical distribution over $(ϵ,L)$ can be learned for the subsequent randomized HMC algorithm.

The plan of the paper is as follows. We begin with a brief description of HMC and NUTS in Sect. 3. We then proceed to expose our method in Sect. 4. We first detail the construction of the joint empirical distribution on $(\epsilon ,L)$ in Sect. 4.1, before introducing a Population Monte Carlo solution to generate the initial weighted sample paramount to our solution. We illustrate the efficiency of the proposed algorithm compared to the NUTS version implemented in Stan Development Team (2024a) on several benchmark examples in Sect. 5.

Notations and conventions

In what follows, we assume that the random variables take values in ${\mathbb{R}}^d$, possibly after a reparametrization, and admit a density with respect to the Lebesgue measure $\text{Leb}$. We denote by ${\mathbb{M}}_1$ the set of probability measures on ${\mathbb{R}}^d$ and by ${\mathbb{M}}_1^{+}$ the subset of those that are positive almost everywhere. Given a probability measure $\pi \in {\mathbb{M}}_1$, we denote as ${\mathbb{M}}_{\pi }$ the set of probability measures that dominates $\pi$. Given two measures $\mu$ and $\nu$ in ${\mathbb{M}}_1$, the product measure is denoted by $\mu \otimes \nu$. We use the same notation to refer to a measure and its associated density, meaning that if $\pi$ is absolutely continuous with respect to $\text{Leb}$, $\pi (\text{d}x)=\pi (x)\text{Leb}(\text{d}x)$. We denote ${\delta }_{\theta }$ the Dirac measure centered on a point $\theta \in {\mathbb{R}}^d$.

Hamiltonian Monte Carlo

Consider a probability measure $\pi \in {\mathbb{M}}_1^{+}$ with continuously differentiable log-density on ${\mathbb{R}}^d$, and let $\tilde{\pi }$ denote an unnormalized and tractable version of $\pi$. HMC (Neal 2011) generates a Markov chain on an augmented parameter space $(\theta ,v)$ where $v\in {\mathbb{R}}^d$ is an auxiliary momentum variable (as opposed to $\theta$ being called the position) independent of $\theta$ and distributed according to a $d-$dimensional normal distribution, $\mathcal{N}(0,M)$. The positive definite matrix $M\in {\mathcal{S}}^{++}({\mathbb{R}}^d)$ is referred to as the mass matrix. We refer the reader to Betancourt (2018) for a discussion on the choice of other momentum distributions. The Hamiltonian is then defined as the unnormalized negative log-version of the joint $\pi \otimes \mathcal{N}(0,M)$, namely

$$\begin{array}{c}\hfill H(\theta ,v)=\frac{1}{2}{v}^T{M}^{-1}v-log\tilde{\pi }(\theta ),\end{array}$$

and the transitions of the chain derive from Hamilton’s equations applied to the latter, namely

$$\begin{array}{c}\hfill \frac{\text{d}\theta }{\text{d}t}=\frac{\partial H}{\partial v}=M^{-1}v,\frac{\text{d}v}{\text{d}t}=-\frac{\partial H}{\partial \theta }=\mathrm{\nabla }log\tilde{\pi }(\theta ).\end{array}$$ 1

Such a mechanism aims at efficiently exploring the parameter space ${\mathbb{R}}^d\times {\mathbb{R}}^d$ as compared to standard random-walk Metropolis-Hastings proposals while yielding a marginal chain in $\theta$ with the distribution of interest. Since the dynamic preserves the Hamiltonian, the momentum and the target density increase together and conversely, facilitating moves between low and high probability regions.

In practice, the above differential equations cannot be solved analytically and HMC samplers resort to time reversible and symplectic numerical integrators. Among these, the leapfrog integrator is commonly used for its trade-off between second order accuracy and computational cost. Given a discretization time-step $\epsilon$, it yields a mapping $F_{\epsilon }:(\theta ,v)\mapsto ({\theta }^{\star },v^{\star })$ defined by

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{\theta }^{\star }=\theta +\epsilon M^{-1}r,\hfill \\ v^{\star }=r+\raisebox{1ex}{$\epsilon $}\!\left/ \!\raisebox{-1ex}{$2$}\right.\mathrm{\nabla }log\tilde{\pi }({\theta }^{\star }),\hfill \end{array}\right)\text{where}r=v+\raisebox{1ex}{$\epsilon $}\!\left/ \!\raisebox{-1ex}{$2$}\right.\mathrm{\nabla }log\tilde{\pi }(\theta ).\end{array}$$

In order to approximate the solution $\left({\theta }^{\star },,,v^{\star }\right)$ at time t, the integrator applies the mapping $L=\lfloor \raisebox{1ex}{$t$}\!\left/ \!\raisebox{-1ex}{$\epsilon $}\right.\rfloor$ times. L is referred to as the number of leapfrog steps. This discrete scheme serves as a proposal kernel for $(\theta ,v)$ but no longer leaves the measure $\pi \otimes \mathcal{N}(0,M)$ invariant. To account for the discretization bias and to preserve the target measure, a Metropolis-Hastings correction is introduced. A transition from $(\theta ,v)$ to the proposal $\left({\theta }^{\star },,,-,v^{\star }\right)$ is accepted with probability

$$\begin{array}{c}\hfill \rho \left(\theta ,,,v,,,{\theta }^{\star },,,v^{\star }\right)=1\wedge exp\left\{H,(\theta ,v),-,H,\left({\theta }^{\star },,,-,v^{\star }\right)\right\}.\end{array}$$ 2

which implies that detailed balance is satisfied for the target $\pi \otimes \mathcal{N}(0,M)$. The solution of (1) keeps the Hamiltonian constant, meaning proposals are always accepted for the exact dynamic. After discretization, the deviation to $H(\theta ,v)$ can be bounded (Leimkuhler and Reich 2005),

$$\begin{array}{c}\hfill \left|H,(\theta ,v),-,H,\left({\theta }^{\star },,,-,v^{\star }\right)\right|<C{\epsilon }^2.\end{array}$$

Hence, the acceptance rate (2) still tends to be high even for a proposal $\left({\theta }^{\star },,,-,v^{\star }\right)$ quite far from $(\theta ,v)$, provided $\epsilon$ remains moderate. In practice, the choice of $\epsilon$ can be done on the basis of the global acceptance rate of the sampler using the primal-dual averaging method (Nesterov 2009). The latter provides an adaptive MCMC scheme which aims at a targeted acceptance probability $p_0\in (0,1)$. Under mild conditions, Beskos et al. (2013) have shown that the optimal scaling for HMC yields an acceptance probability of around 0.65. Though it is not necessarily a suitable choice for $p_0$ in complex models, it can serve as a useful benchmark value.

When it comes to choosing the number L of integration steps, NUTS (Hoffman and Gelman 2014) is arguably the most commonly used version of HMC sampler. It eliminates the need to specify the number L of integration steps by adaptively choosing the locally largest value at each iteration of the algorithm. More precisely, given a step size $ϵ$, the current value of $\theta$ and a momentum v, an iteration of NUTS begins with mimicking the Hamiltonian dynamics by recursively doubling the leapfrog path, either forward or backward with equal probability, until the path begins to retrace towards the starting point. For the Euclidean version, this means that the backward and forward end points of the path, $({\theta }^{-},v^{-})$ and $({\theta }^{+},v^{+})$, satisfy

$$\begin{array}{c}\hfill {({\theta }^{+}-{\theta }^{-})}^{\top }{M}^{-1}{v}^{-}<0\text{or}{({\theta }^{+}-{\theta }^{-})}^{\top }{M}^{-1}{v}^{+}<0.\end{array}$$ 3

For an extension of the aforementioned termination criterion to the non-Euclidean case, we refer the reader to Betancourt (2018). The second step of NUTS consists in randomly picking one of the position-momentum pairs visited along the forward and backward leapfrog path, each pair being weighted according to the target density. In its original version, the latter was done by a slice sampling move, but most recent versions implemented in Stan (Stan Development Team 2024b) rely on a multinomial sampling strategy. The overall process allows for detailed balance to hold and thus validates NUTS.

Compared with the standard HMC sampler, which uses a fixed length L across iterations, NUTS relies on an adaptive scheme while requiring an evaluation of the Hamiltonian along its entire leapfrog path. Even though this leapfrog path is locally the longest possible in terms of Equation (3), the final proposal is not obtained by deterministically taking the last point of the trajectory, but is picked at random among the states generated along the trajectory using slice-sampling in the original version or multinomial sampling in current Stan implementation. Progressive sampling mechanisms increase the probability of selecting states from recently built subtrees, but the distance between the proposed position and the current one may be significantly smaller than the maximal one achieved during the construction, so part of the computational effort does not translate directly into transitions of the Markov chain. This online construction is essential in targets where useful integration times vary strongly across the typical set, but may be unnecessary when a reusable distribution of integration times can be learned offline.

Empirical Hamiltonian Monte Carlo

Following the observation that the systematic construction of the longest admissible Hamiltonian trajectories may be unnecessary, we propose a two-stage procedure, referred to as the eHMC sampler (see Algorithm 1), that (i) learns a distribution over the corresponding integration parameters and (ii) reuses the latter to define a randomized version of the Hamiltonian kernel.

Calibration stage

The first stage of eHMC relies on importance sampling (Kahn 1949; Kahn and Harris 1951) to approximate the target distribution $\pi$. Given N independent samples $({\vartheta }_1,\cdots ,{\vartheta }_N)$ sampled from a proposal distribution $\mu \in {\mathbb{M}}_{\pi }$, the target measure $\pi$ is estimated by the empirical measure associated with the weighted sample $\{({\vartheta }_1,{\omega }_1),\dots ,({\vartheta }_N,{\omega }_N)\}$, where the weights are derived from the Radon–Nikodym derivative of $\pi$ with respect to the proposal distribution $\mu$, namely

$$\begin{array}{c}\hfill {\widehat{\pi }}_N=\sum _{i=1}^N{\omega }_i{\delta }_{{\vartheta }_i},{\omega }_i\propto \frac{\tilde{\pi }({\vartheta }_i)}{\mu ({\vartheta }_i)}(i=1,\dots ,N).\end{array}$$

Empirical distribution of the integration time The randomization of parameters L and $\epsilon$ in eHMC is based on the empirical distribution of the longest integration time associated with each position. Given an augmented parameter $(\theta ,v)$ and a step size $\epsilon >0$, the maximal integration length in the direction v is given by the first occurrence of a U-turn, analogous to Equation (3), along the discretized Hamiltonian trajectory

$$\begin{array}{c}\hfill L(\theta ,v,\epsilon )=\underset{\ell \in \mathbb{N}}{inf}\left\{({\theta }_{\ell }^{\star },v_{\ell }^{\star }),=,F_{\epsilon }^{\ell },(\theta ,v),:,{({\theta }_{\ell }^{\star }-\theta )}^{\top },M^{-1},v_{\ell }^{\star },<,0\right\}.\end{array}$$ 4

Lemma 1

Assume that the potential $U:\theta \mapsto -log\tilde{\pi }(\theta )$ is coercive, i.e., ${\Vert U(\theta )\Vert }_2\to \infty$ as ${\Vert \theta \Vert }_2\to \infty$. Then, for all positive definite matrices M, there exists ${\epsilon }^{\star }>0$, such that for all $0<\epsilon \le {\epsilon }^{\star }$, $L(\theta ,v,\epsilon )$ is finite $\pi \otimes \mathcal{N}(0,M)$-almost surely.

We refer the reader to Appendix A for details. Considering a set of local step sizes ${\epsilon }_i\le {\epsilon }^{\star }$ associated with each pair $({\vartheta }_i,v_i)$, $i=1,\dots ,N$, we thus define an empirical measure on the induced set of maximal numbers of leapfrog steps

$$\begin{array}{cc}\hfill \mathcal{L}=\{& L({\vartheta }_1,v_1,{\epsilon }_1),\dots ,L({\vartheta }_N,v_N,{\epsilon }_N),\hfill \\ \hfill & L({\vartheta }_1,-v_1,{\epsilon }_1),\dots ,L({\vartheta }_N,-v_N,{\epsilon }_N)\},\hfill \end{array}$$

namely,

$$\begin{array}{c}\hfill {\widehat{\mu }}_{\mathcal{L}}=\frac{1}{2}\sum _{i=1}^N{\omega }_i{\delta }_{L({\vartheta }_i,v_i,{\epsilon }_i)}+\frac{1}{2}\sum _{i=1}^N{\omega }_i{\delta }_{L({\vartheta }_i,-v_i,{\epsilon }_i)}.\end{array}$$

However, our aim is to further select the step size at random within the HMC scheme. We thus build an empirical distribution that accounts for a local adaptation of the step size for each realisation ${\vartheta }_i$.

Consider an excursion on the discretized level set starting from a point $(\theta ,v)$. The leapfrog integrator generates a sequence of pairs $({\theta }_{\ell },v_{\ell })=F_{\epsilon }^{\ell }(\theta ,v)$, $\ell \in {\mathbb{N}}^{\ast }$. Each of these defines a valid Metropolis–Hastings proposal for the augmented target distribution based on the acceptance probability $\rho (\theta ,v,{\theta }_{\ell },v_{\ell })$ as defined in Equation (2). Hence, we can locally adapt the step size by monitoring the average acceptance probability along the leapfrog path. Namely, given a user-specified target probability $p_0\in (0,1)$, we calibrate the step size from

$$\begin{array}{cc}\hfill \overline{\epsilon }(\theta ,v,p_0)=sup\{& 0<\epsilon \le {\epsilon }^{\star }:\sum _{\ell =1}^{L(\theta ,v,\epsilon )}\frac{\rho \left(\theta ,,,v,,,{\theta }_{\ell },,,v_{\ell }\right)}{L(\theta ,v,\epsilon )}\ge p_0\}.\hfill \end{array}$$ 5

The probability $p_0$ plays a role similar to target acceptance probability in the primal-dual averaging algorithm. The latter set is non empty. Indeed, for any time interval, the deviation to $H(\theta ,v)$ is of order $\mathcal{O}({\epsilon }^2)$ (Leimkuhler and Reich 2005), which ensures that for $\epsilon$ small enough the acceptance probability (2) becomes larger than $p_0$. Setting ${\epsilon }_i^{+}=\overline{\epsilon }({\vartheta }_i,v_i,p_0)$ and ${\epsilon }_i^{-}=\overline{\epsilon }({\vartheta }_i,-v_i,p_0)$, the empirical measure on the discretization parameters is then defined as

$$\begin{array}{c}\hfill {\widehat{\mu }}_{\mathcal{T}}=\frac{1}{2}\sum _{i=1}^N{\omega }_i{\delta }_{({\epsilon }_i^{+},L({\vartheta }_i,v_i,{\epsilon }_i^{+}))}+\frac{1}{2}\sum _{i=1}^N{\omega }_i{\delta }_{({\epsilon }_i^{-},L({\vartheta }_i,-v_i,{\epsilon }_i^{-}))}.\end{array}$$ 6

The U-turn criterion in (3) is chosen here as a practical device for constructing the empirical distribution (6). Its validity is theoretically motivated by Lemma 1, which establishes that the corresponding stopping time is almost surely finite and therefore induces a well-defined empirical distribution over $(\epsilon ,L)$. Note that the U-turn criterion is not essential to the proposed methodology. Any alternative offline stopping criterion could be used, provided an analogous result holds, namely that this criterion defines an almost surely finite stopping time and hence induces a well-defined distribution over $(\epsilon ,L)$. Once such a distribution is obtained, the validity of the final sampler no longer depends on the particular criterion at its core (see Sect. 4.2 for the stationarity argument). Alternative criteria satisfying the above requirement may then prove useful, especially in settings where the first U-turn time is not the most informative proxy for a suitable integration length, such as highly oscillatory Hamiltonian dynamics. They may further bring an assessment of convergence by comparing the resulting outcomes.

Adapting the mass matrix The Hamiltonian dynamic and consequently the empirical distribution ${\widehat{\mu }}_{\mathcal{T}}$ depend on the choice of the mass matrix M. It is well known that tuning M so that $M^{-1}$ accounts for the covariance structure of the target distribution $\pi$ can substantially improve the efficiency of the sampler. Accordingly, we first estimate $M^{-1}$ by the sample covariance of the weighted particles $\{({\vartheta }_1,{\omega }_1),\dots ,({\vartheta }_N,{\omega }_N)\}$, namely

$$\begin{array}{c}\hfill M^{-1}=\sum _{i=1}^N{\omega }_i\left({\vartheta }_i,-,\overline{\vartheta }\right){\left({\vartheta }_i,-,\overline{\vartheta }\right)}^{\top },\overline{\vartheta }=\sum _{i=1}^N{\omega }_i{\vartheta }_i,\end{array}$$

which corresponds to the minimizer of the Kullback–Leibler divergence over Gaussian momentum distributions (Girolami and Calderhead 2011). The latter can then be refined by exploiting the additional information provided by all the positions traced out by the integrator during the construction of the empirical measure (6). Indeed, for each initial pair $({\vartheta }_i,v_i)$, consider the sequences of position-momentum $({\vartheta }_{i,\ell }^{+},v_{i,\ell }^{+})=F_{{\epsilon }_i^{+}}^{\ell }({\vartheta }_i,v_i)$ and $({\vartheta }_{i,\ell }^{-},v_{i,\ell }^{-})=F_{{\epsilon }_i^{-}}^{\ell }({\vartheta }_i,-v_i)$ generated along the leapfrog trajectory. In the continuous-time, ideal, setting, these intermediate positions are distributed according to the same marginal distribution as ${\vartheta }_i$. In the discretized setting, we account for integration error by weighting each position according to its corresponding Metropolis–Hastings acceptance probability (2). The discrete measure

$$\begin{array}{cc}\hfill \nu \propto \sum _{i=1}^N& {\omega }_i[\sum _{\ell =1}^{L({\vartheta }_i,v_i,{\epsilon }_i^{+})}\left\{{\delta }_{{\vartheta }_{i,\ell -1}^{+}},+,\rho ,({\vartheta }_i,v_i,{\vartheta }_{i,\ell }^{+},v_{i,\ell }^{+}),\left({\delta }_{{\vartheta }_{i,\ell }^{+}},-,{\delta }_{{\vartheta }_{i,\ell -1}^{+}}\right)\right\}\hfill \\ \hfill & +\sum _{\ell =1}^{L({\vartheta }_i,-v_i,{\epsilon }_i^{-})}\left\{{\delta }_{{\vartheta }_{i,\ell -1}^{-}},+,\rho ,({\vartheta }_i,-v_i,{\vartheta }_{i,\ell }^{-},v_{i,\ell }^{-}),\left({\delta }_{{\vartheta }_{i,\ell }^{-}},-,{\delta }_{{\vartheta }_{i,\ell -1}^{-}}\right)\right\}].\hfill \end{array}$$ 7

yields a Rao–Blackwellized empirical covariance associated with the Metropolis–Hastings transitions along each trajectory that we used to estimate $M^{-1}$ (see Appendix B).

The calibration scheme thus consists of a series of adaptation epochs that alternate between updating first the mass matrix and then constructing the empirical measure (6) of the discretization parameters. For the sake of simplicity, we have described the Rao–Blackwellization based on the positions of the last excursion of each trajectory. In practice, the computation of ${\epsilon }_i$, as defined in Equation (5) typically involves multiple excursions on the same level set, and all intermediate positions can be incorporated in the estimator in the same manner.

Sampling stage

In standard MCMC methods, a preliminary burn-in phase is typically required to ensure that the chain has moved sufficiently far from its initial distribution toward its invariant distribution. This transient regime is often difficult to diagnose and may incur a non-negligible computational cost. In standard NUTS implementations, this warmup phase is used to tune algorithmic parameters such as the step size and mass matrix. In eHMC, this role is transferred to the offline calibration stage. The latter provides an importance-weighted sample that yields an empirical approximation of $\pi$, removing the reliance on a transient Markov-chain phase for parameter adaptation and initialization.

Once both ${\widehat{\mu }}_T$ and M are set, they are used to define a homogeneous Markov chain. At each iteration, say k, a pair $({\epsilon }_k,L_k)$ is sampled from ${\widehat{\mu }}_{\mathcal{T}}$ and an HMC proposal is generated using the leapfrog integrator with mass matrix M and integration length $L_k{\epsilon }_k$ (see Algorithm 1). The resulting algorithm yields a randomized HMC kernel that does not belong to the class of adaptive MCMC methods with online parameter updates. Instead, it defines a mixture of Markov transition kernels with fixed mixing distribution ${\widehat{\mu }}_T$. Since, for any fixed $(\epsilon ,L)$, the corresponding HMC kernel leaves the target distribution $\pi$ invariant (Neal 2011), it follows that the mixture kernel defined by sampling $(\epsilon ,L)$ from ${\widehat{\mu }}_{\mathcal{T}}$ also preserves $\pi$ (Tierney 1994; Roberts and Rosenthal 2007).

Construction of the importance sampling proposal

The eHMC sampler relies on having an initial proposal distribution $\mu$ to derive a weighted sample approximately distributed according to $\pi$. Finding a well suited proposal $\mu$ is paramount to guarantee good statistical properties of importance sampling estimators (e.g., Agapiou et al. 2017; Chatterjee and Diaconis 2018, ), but this may prove challenging. Numerous adaptive strategies have been proposed over the past two decades to design efficient proposal distributions, and we refer the reader to Elvira and Martino (2021) for a broad overview. However the objective here differs from standard importance sampling. Rather than seeking highly accurate approximations of the target distribution in order to control bias and variance, our goal is to obtain a weighted sample that captures sufficient diversity in regions of non-negligible probability mass.

In what follows, we rely on a Population Monte Carlo (PMC) scheme (Cappé et al. 2004) to construct the proposal $\mu$ (see Algorithm 2). PMC is an adaptive importance sampling framework that generates a sequence ${({\mu }_t)}_{t\ge 0}$ in ${\mathbb{M}}_{\pi }$ by iteratively updating the proposal ${\mu }_{t+1}$ using weighted samples drawn from ${\mu }_t$. We adopt a tempering strategy inspired by Koblents and Míguez (2015) and Aufort et al. (2022), which improves robustness in complex or multimodal settings. The PMC procedure described below should be viewed as one possible solution for the offline calibration stage rather than as an essential component of the proposed methodology. More generally, any procedure yielding a representative weighted sample of the target distribution could be used to estimate the empirical distribution (6). The purpose of the calibration stage is to transfer the adaptation effort outside the final Markov chain, not to advocate a particular adaptive importance sampling method. This does not, however, eliminate all calibration choices: the quality of the initial proposal, the PMC settings (or those of any alternative calibration procedure), and the size of the calibration sample remain important practical considerations, as it is generally the case for adaptive importance sampling methods.

Proposal adaptation The initial proposal ${\mu }_0$ is set by the user. In a Bayesian context, a natural choice is the prior distribution. Subsequent proposals are chosen from a parametric family $\mathcal{F}\subset {\mathbb{M}}_{\pi }$ from which we can sample and evaluate the density. To progressively bridge the gap between ${\mu }_t$ and $\pi$, we introduce a sequence ${({\pi }_t)}_{t\ge 1}$ of annealed target distributions defined, for all $t\ge 0$, as

$$\begin{array}{c}\hfill {\pi }_{t+1}\propto {\tilde{\pi }}^{{\beta }_{t+1}}{\mu }_t^{1-{\beta }_{t+1}},\end{array}$$ 8

where ${({\beta }_t)}_{t\ge 1}\in {[0,1]}^{\mathbb{N}}$ is a non-decreasing sequence such that ${\beta }_1=0$ and ${lim}_{t\to \infty }{\beta }_t=1$. This construction defines a continuum of intermediate distributions connecting ${\mu }_0$ to $\pi$ that has been extensively used in the tempering literature (Chopin and Papaspiliopoulos 2021) and helps to uncover regions of ${\mathbb{R}}^d$ where $\pi$ has non-negligible mass. At iteration $t\ge 0$, given a sample $({\vartheta }_{t,1},\dots ,{\vartheta }_{t,N_0})$ from ${\mu }_t$, the annealed target ${\pi }_{t+1}$ is approximated by the self-normalized importance sampling measure

$$\begin{array}{c}\hfill {\widehat{\pi }}_{t+1}\propto \sum _{i=1}^{N_0}{\tilde{\omega }}_{t,i}^{t+1}{\delta }_{{\vartheta }_{t,i}},{\tilde{\omega }}_{t,i}=\frac{\tilde{\pi }({\vartheta }_{t,i})}{{\mu }_t({\vartheta }_{t,i})}.\end{array}$$ 9

The next proposal is then obtained as the projection of ${\widehat{\pi }}_{t+1}$ onto $\mathcal{F}$

$$\begin{array}{cc}\hfill {\mu }_{t+1}& =\underset{\mu \in \mathcal{F}}{\text{argmin}}\text{KL}({\widehat{\pi }}_{t+1}\Vert \mu )\hfill \\ \hfill & =\underset{\mu \in \mathcal{F}}{\text{argmin}}\left\{-,\sum _{i=1}^{N_0},{\tilde{\omega }}_{t,i}^{t+1},log,\mu ,({\vartheta }_{t,i})\right\}.\hfill \end{array}$$ 10

When $\mathcal{F}$ is parametric, this corresponds to computing a maximum likelihood estimate of the family parameter based on the weighted sample.

Temperature scheduling A standard strategy to define the sequence ${({\beta }_t)}_{t\ge 1}$ consists in selecting ${\beta }_{t+1}$ to control the effective sample size (ESS) of the importance weights (Gramacy et al. 2010; Beskos et al. 2016; Elvira and Martino 2021). Assume that at iteration t, for all $\beta \in [0,1]$, $\tilde{\pi }/{\mu }_t\in L^{2\beta }({\mu }_t)$. Then, the ESS associated with proposal ${\mu }_t$ and unnormalized target ${\mu }_t^{1-\beta }{\tilde{\pi }}^{\beta }$ is

$$\begin{array}{cc}& \text{ESS}\left({\tilde{\pi }}^{\beta },{\mu }_t^{1-\beta },,,{\mu }_t\right)=N\frac{{\left\{{\int }_{{\mathbb{R}}^d},{\tilde{\omega }}_t,{(\vartheta )}^{\beta },{\mu }_t,(\text{d}\vartheta )\right\}}^2}{{\int }_{{\mathbb{R}}^d}{\tilde{\omega }}_t{(\vartheta )}^{2\beta }{\mu }_t(\text{d}\vartheta )},\hfill \\ \hfill & {\tilde{\omega }}_t(·)=\frac{\tilde{\pi }(·)}{{\mu }_t(·)}.\hfill \end{array}$$

Its estimate

$$\begin{array}{c}\hfill \widehat{\text{ESS}}\left({\tilde{\pi }}^{\beta },{\mu }_t^{1-\beta },,,{\mu }_t\right)=\frac{{\left({\sum }_{i=1}^0,{\tilde{\omega }}_{t,i}^{\beta }\right)}^2}{{\sum }_{i=1}^0{\tilde{\omega }}_{t,i}^{2\beta }},\end{array}$$ 11

quantifies the discrepancy between the weighted and unweighted empirical measures, or, equivalently, the degeneracy of the importance weights: a higher effective sample size indicates a better empirical approximation of the target distribution. In our setting, ${\beta }_{t+1}$ is chosen to ensure that the reweighted sample provides a sufficiently accurate approximation of the annealed target ${\pi }_{t+1}$, and avoids overfitting to the current sample. More precisely, we set

$$\begin{array}{c}\hfill {\beta }_{t+1}=sup\left\{\beta ,\in ,[0,1],:,\widehat{\text{ESS}},\left({\tilde{\pi }}^{\beta },{\mu }_t^{1-\beta },,,{\mu }_t\right),\ge ,{\lambda }_0,N_0\right\},\end{array}$$ 12

where ${\lambda }_0\in (0,1)$ is a user-specified parameter. Under the assumption $\tilde{\pi }/{\mu }_t\in L^{2\beta }({\mu }_t)$, the function $\beta \mapsto \text{ESS}({\pi }^{\beta }{\mu }_t^{1-\beta },{\mu }_t)$ is continuous and non-increasing (see Appendix D), which ensures that this update is well-defined. The adaptation stops when we obtain $\widehat{\text{ESS}}\left(\tilde{\pi },,,{\mu }_t\right)\ge \mathrm{\Delta }$, where $\mathrm{\Delta }\le {\lambda }_0{N}_0$ is a user-specified threshold.

Practical implementation Unlike standard adaptive importance sampling methods, where the effective sample size is controlled to ensure that a large proportion of particles carry significant weight, our objective is more modest. The threshold $\mathrm{\Delta }$ is chosen only to guarantee a minimal level of effectiveness, sufficient to capture regions of non-negligible probability mass. The importance weights may then exhibit substantial variability. To mitigate the resulting weight degeneracy, we can perform a resampling step from the empirical measure ${\widehat{\pi }}_{t+1}$, possibly followed by a Markov kernel perturbation that preserves ${\pi }_{t+1}$ in order to prevent sample impoverishment (see Algorithm 2). This resample-move strategy, commonly used in sequential Monte Carlo methods (Chopin and Papaspiliopoulos 2021), produces a set of unweighted particles while preserving consistency with respect to the target ${\pi }_{t+1}$. Consequently, the proposal update step (10) consists in fitting ${\mu }_{t+1}$ to the sample that has undergone this transformation, rather than to the original weighted sample.

In practice, this procedure can also be applied at the final iteration of the adaptation, that is once we get $\mu$ such that $\widehat{\text{ESS}}\left(\tilde{\pi },,,\mu \right)\ge \mathrm{\Delta }$, so that the sample used in the calibration phase consists of independent and equally weighted particles, namely

$$\begin{array}{c}\hfill {\widehat{\pi }}_N=\frac{1}{N}\sum _{i=1}^N{\delta }_{{\vartheta }_i}.\end{array}$$

Numerical experiments

In this section, we assess the performance of the proposed eHMC algorithm, in comparison to NUTS, on four target distributions chosen to illustrate different challenges, including strong nonlinearity, correlation structure and multiscale geometry. All experiments were implemented using a combination of R and Python, with Stan models used to define the target distributions. We used the version of NUTS implemented in Rstan (Stan Development Team 2024a). The codes for eHMC and the experiments are available on https://github.com/jstoehr/eHMC.

Models

Banana-shaped distribution(banana) We first consider a two-dimensional nonlinear target distribution defined by

$$\begin{array}{c}\hfill {\theta }_1\sim \mathcal{N}(0,{\sigma }_1^2),{\theta }_2\mid {\theta }_1\sim \mathcal{N}\left(\lambda ,({\theta }_1^2-\ell ),,,{\sigma }_2^2\right),\end{array}$$

with ${\sigma }_1=10$, $\lambda =0.03$, $\ell =100$ and ${\sigma }_2=1$. This distribution is commonly used as a benchmark for testing the robustness of sampling algorithms to nonlinear dependencies (Naderi et al. 2026).

Within the eHMC calibration stage, the initial proposal is set to a multivariate Gaussian distribution with mean ${(0,-\lambda \ell )}^{\top }$ and diagonal covariance matrix ${\sigma }_1^2{I}_2$. The latter centers the proposal along the main curvature of the target while maintaining isotropic variability.

Multivariate Normal distribution(MVNorm) We next consider a 100-dimensional Gaussian target distribution with density

$$\begin{array}{cc}& \pi (\theta )\propto exp\left(-,\frac{1}{2},{\theta }^{\top },A^{-1},\theta \right),\hfill \\ \hfill & A_{i,j}=0.{99}^{|i-j|},i,j=1,\cdots ,100.\hfill \end{array}$$

The covariance matrix is taken as the AR(1) correlation matrix with autoregressive parameter $\rho =0.99$. The resulting matrix exhibits a condition number of approximately $7.6\times {10}^3$, indicating a moderately ill-conditioned geometry with strong anisotropy due to strong dependence between neighboring coordinates.

The initial proposal for eHMC is set to a mixture of two Gaussian distributions centered at zero,

$$\begin{array}{c}\hfill {\mu }_0=\alpha \mathcal{N}(0,{\mathrm{\Sigma }}_1)+(1-\alpha )\mathcal{N}(0,{\mathrm{\Sigma }}_2),\alpha =0.1,\end{array}$$

where the covariance matrices ${\mathrm{\Sigma }}_1$ and ${\mathrm{\Sigma }}_2$ are constructed from the eigendecomposition $A=QD{Q}^{\top }$ of the target covariance matrix. Specifically, ${\mathrm{\Sigma }}_1$ and ${\mathrm{\Sigma }}_2$ are defined as

$$\begin{array}{cc}\hfill {\mathrm{\Sigma }}_1& =Qdiag(d_1,\dots ,d_{100})Q^{\top },\hfill \\ \hfill {\mathrm{\Sigma }}_2& =Qdiag({\lambda }_1,\dots ,{\lambda }_{100})Q^{\top },\hfill \end{array}$$

with for $j=1,\dots ,100$

$$\begin{array}{c}\hfill d_j=\left\{\begin{array}{cc}log(D_{\mathit{jj}}),\hfill & D_{\mathit{jj}}>1,\hfill \\ D_{\mathit{jj}},\hfill & D_{\mathit{jj}}\le 1,\hfill \end{array}\right){\lambda }_j=\left\{\begin{array}{cc}0.6D_{\mathit{jj}},\hfill & D_{\mathit{jj}}>1,\hfill \\ 1.1D_{\mathit{jj}},\hfill & D_{\mathit{jj}}\le 1.\hfill \end{array}\right)\end{array}$$

This construction yields a dispersed proposal that captures different scales of the target distribution.

Bayesian logistic regression(BLP) We also consider a Bayesian logistic regression model with

$$\begin{array}{cc}\hfill \theta & \sim \mathcal{N}(0,I),\hfill \\ \hfill y_i& \sim \text{Bernoulli}\left({\text{logit}}^{-1},(x_i^{\top }\theta )\right),i=1,\dots ,n.\hfill \end{array}$$

This model yields a posterior distribution that is log-concave but may exhibit strong correlations between parameters depending on the design matrix. The inference is based on the German Credit dataset from the UCI Machine Learning Repository (Dua and Graff 2019), which consists of $n=1000$ observations. The design matrix includes an intercept term and 24 covariates.

The initial proposal distribution is constructed as a Gaussian approximation centered around a ridge-regularized least-squares estimate of the posterior mode. Specifically, we define the approximate precision matrix as $A=X^{\top }X+{\sigma }^{-2}{I}_K$. The proposal mean is then given by $A^{-1}{X}^{\top }y$, with covariance matrix $c{A}^{-1}$, where $c>1$ is an inflation factor introduced to broaden the proposal and improve coverage of the posterior tails. In what follows, we set $c=4$. This construction provides a computationally inexpensive Laplace-style initialization that is substantially better adapted to the posterior than an isotropic prior proposal.

Neal’s funnel distribution(funnel) To assess limitations of the eHMC solution, we consider Neal’s funnel distribution. For dimension $d=10$, the target is defined by

$$\begin{array}{cc}\hfill y& \sim \mathcal{N}(0,3^2),\hfill \\ \hfill x_i\mid y& \sim \mathcal{N}(0,exp(y)),i=1,\dots ,d-1.\hfill \end{array}$$

Equivalently, up to an additive constant,

$$\begin{array}{c}\hfill log\pi (y,x)\propto -\frac{1}{2}\frac{y^2}{9}-\frac{d-1}{2}y-\frac{1}{2}exp(-y)\sum _{i=1}^{d-1}{x}_i^2.\end{array}$$

This target is a standard challenge for randomized HMC methods since the curvature varies strongly with the value of y. For some parts of the distribution, small changes in x require small integration time, whereas for other parts much larger moves are possible. We therefore expect dynamic trajectory construction to be more important on this target than on the previous benchmarks.

The initial proposal distribution uses the correct conditional structure but inflated conditional variance,

$$\begin{array}{c}\hfill y\sim \mathcal{N}(0,3^2),x_i\mid y\sim \mathcal{N}(0,4exp(y)).\end{array}$$

Although this proposal captures the characteristic funnel geometry, it remains a poor approximation of the target distribution. In particular, the variance inflation produces a substantial mismatch with the target, resulting in an effective sample size close to one particle in some experiments.).

Experimental design

For each model, the experiments were repeated over 20 seeds. For each seed, we ran 50 parallel chains and set the target acceptance probability $p_0\in \{0.651,0.80\}$ for both methods.

NUTS setting We use the NUTS implementation in Stan. Following the default choice, we used a diagonal mass matrix M. Each chain then consists of 4000 iterations in total, including 2000 warmup iterations during which we used the default dual averaging adaptation for the step size, and $T=2000$ kept samples. The computational cost of NUTS is measured by the number of gradient evaluations, or equivalently, up to a constant factor, by the total number of leapfrog steps performed during the sampling phase. The warmup phase is not included in this cost.

eHMC setting For each seed, the calibration phase of eHMC was performed using the PMC procedure described in Sect. 4.3 with an initial sample size $N_0=50000$, ${\lambda }_0=0.8$ and an effective sample size threshold $\mathrm{\Delta }\in \{0.05N_0,0.1N_0\}$. The parametric family $\mathcal{F}$ was defined through a normalizing flow, which provides a flexible class of tractable distributions based on an invertible and differentiable transformation that maps a simple base distribution to the distribution of $({\vartheta }_{t,1},\dots ,{\vartheta }_{t,N_0})$. More precisely, let ${\{f_{\xi }:{\mathbb{R}}^d\to {\mathbb{R}}^d\}}_{\eta \in \mathrm{\Xi }}$ be a family of bijective and differentiable mappings and p be a base distribution. The family $\mathcal{F}$ is defined as

$$\begin{array}{c}\hfill \mathcal{F}=\left\{\theta ,\mapsto ,p,\left(f_{\xi }^{-1},(\theta )\right),\left|det,D,f_{\xi }^{-1},(\theta )\right|,;,,\xi ,\in ,\mathrm{\Xi }\right\},\end{array}$$

where $D{f}_{\xi }^{-1}$ is the Jacobian of $f_{\xi }^{-1}$. In our setting, the base distribution was set to a multivariate Gaussian $\mathcal{N}(0,{\mathrm{\Sigma }}_t)$, where ${\mathrm{\Sigma }}_t$ denotes the covariance matrix associated with the empirical measure (9). The normalizing flow was implemented as a Masked Autoregressive Flow (MAF, Papamakarios et al. 2017, ) with 5 transformations, each parameterized by neural networks with 2 hidden layers of 32 units. The model was trained using mini-batches of size 2048 and a learning rate of ${10}^{-3}$. Early stopping was used with a patience of 20 epochs without validation improvement, reserving $20\%$ of the data for validation. The procedure was implemented using zuko v1.4.0 (Rozet et al. 2022).

We stress anew that the fitted normalizing flow is not intended to serve as a transport map for running HMC on a transformed target density. Within the PMC tempering scheme, the flow is successively fitted to a sequence of annealed target distributions to improve the efficiency of the importance sampler. However, the calibration procedure stops as soon as the proposal yields a sufficiently large importance sampling ESS with respect to the target $\pi$. Consequently, the annealing sequence does not need to reach $\beta =1$, and the fitted flow approximates the last annealed distribution visited by the algorithm rather than the target distribution $\pi$.

After the proposal adaptation step, we resampled $N\in \{2000,5000\}$ particles, without applying a Markov kernel perturbation. This equally weighted sample was then used to construct the empirical distribution ${\widehat{\mu }}_{\mathcal{T}}$, as defined in (6), with $K=1$ adaptation epoch. To match the default choice in Rstan, we used a diagonal mass matrix obtained from the diagonal of the empirical covariance estimate based on the empirical measure (7). The final eHMC chains are then run for $T=2000$ iterations without further online adaptation. The computational cost of eHMC is measured by the number of gradient evaluations, or equivalently, up to a constant factor, by the total number of leapfrog steps performed during the sampling phase. The offline calibration stage is not included in this cost.

Performance metrics

For each model, we compare eHMC and NUTS in terms of the effective sample size associated with the Markov chain and the expected squared jump distance. To account for differences in computational cost across samplers, performances are assessed by normalizing both metrics by the number of gradient evaluations of the log-target density, or equivalently the total number of leapfrog steps performed. This normalization reflects the fact that the evaluation of $\mathrm{\nabla }log\tilde{\pi }$ generally dominates the overall computational cost.

Markov chain effective sample size(${\mathrm{ESS}}_{\mathrm{MC}}$) We recall that the standard effective sample size associated with a Markov chain $\{{\theta }^{(1)},\dots ,{\theta }^{(T)}\}$ and a measurable function f is defined as

$$\begin{array}{c}\hfill {\mathrm{ESS}}_{\mathrm{MC}}(f)=T/\left(1,+,2,\sum _{k=1}^{\infty },{\rho }_k,(f)\right),\end{array}$$

where ${\rho }_k(f)$ is the lag k auto-correlation of the Markov chain $\{f({\theta }^{(1)}),\dots ,f({\theta }^{(T)})\}$. Informally, the ${\mathrm{ESS}}_{\mathrm{MC}}$ of an MCMC output is interpretable as providing the equivalent number of independent simulations from the target distribution, where equivalent means with equivalent variance, as a function of the target acceptance probability $p_0$. Hence, the more efficient a sampler is in its approximation to i.i.d. sampling from the target, the larger the associated ${\mathrm{ESS}}_{\mathrm{MC}}$ should be. In the following examples, we focus on the ${\mathrm{ESS}}_{\mathrm{MC}}$ for the estimation of the mean and the variance, i.e., ${\mathrm{ESS}}_{\mathrm{MC}}(\theta )$ and ${\mathrm{ESS}}_{\mathrm{MC}}({\theta }^2)$, using the monitor function from the Rstan package.

Figure 1 displays, for all configurations, the distribution of ${\mathrm{ESS}}_{\mathrm{MC}}(\theta )$ and ${\mathrm{ESS}}_{\mathrm{MC}}({\theta }^2)$ per gradient evaluation across dimensions and runs. Figure 2 reports, over runs, the distribution of their minimum taken over all components of the parameter of the model. This metric captures the least efficient direction of the sampler (see Tables 2 and 3 in Appendix E for numerical summaries).

Fig. 1.

Distribution of the Markov chain effective sample size per number of leapfrog steps for mean (left panel) and variance (right panel) estimation, aggregated across dimensions and 20 independent runs of 50 parallel chains of length $T=2000$. For eHMC, the proposal is trained using ${\lambda }_0=0.8$, $N_0=50000$, from which $N\in \{2000,5000\}$ resampled particles are used to construct the empirical measure on the integration parameters. Results are shown for $\mathrm{\Delta }/N_0\in \{0.05,0.10\}$ in the x-axis

Fig. 2.

Distribution of the minimum across dimensions of the Markov chain effective sample size per number of leapfrog steps for mean (left panel) and variance (right panel) estimation, aggregated over 20 independent runs of 50 parallel chains of length $T=2000$. For eHMC, the proposal is trained using ${\lambda }_0=0.8$, $N_0=50000$, from which $N\in \{2000,5000\}$ resampled particles are used to construct the empirical measure on the integration parameters. Results are shown for $\mathrm{\Delta }/N_0\in \{0.05,0.10\}$ in the x-axis

Table 2.

Minimum across dimensions of the Markov chain effective sample size per leapfrog step, averaged over 20 independent runs of 50 parallel chains of length $T=2000$, for the estimation of the mean. For eHMC, the proposal is trained using ${\lambda }_0=0.8$, $N_0=50000$, from which N resampled particles are used to construct the empirical measure on the integration parameters. Values are reported as mean $\pm 2$ standard errors, scaled by a factor ${10}^4$. Values in bold indicate the best performance across the methods and experimental configurations

		eHMC
Model	NUTS	$\mathrm{\Delta }/N_0=5\%$		$\mathrm{\Delta }/N_0=10\%$
		$N=2000$	$N=5000$	$N=2000$	$N=5000$
(a) Results for a target Metropolis–Hastings acceptance probability $p_0=0.651$.
Banana	$5.26\pm 8.7$	$\mathbf{13.6}\pm \mathbf{18}$	$13.1\pm 18.4$	$11.7\pm 17$	$12.8\pm 16$
MVNorm	$7.81\pm 0.67$	$\mathbf{11.6}\pm \mathbf{0.91}$	$11.5\pm 0.87$	$\mathbf{11.6}\pm \mathbf{0.91}$	$11.5\pm 0.87$
BLP	$151\pm 3.3$	$\mathbf{202}\pm \mathbf{8.0}$	$202\pm 8.1$	$202\pm 10$	$201\pm 7.9$
Funnel	$0.902\pm 0.38$	$2.15\pm 2.8$	$2.35\pm 2.7$	$1.86\pm 1.9$	$\mathbf{2.63}\pm \mathbf{2.9}$
(b) Results for a target Metropolis–Hastings acceptance probability $p_0=0.8$.
Banana	$10.3\pm 16$	$15.2\pm 16$	$\mathbf{18.5}\pm \mathbf{19}$	$17.3\pm 18$	$19\pm 19$
MVNorm	$6.73\pm 0.42$	$11.3\pm 0.81$	$\mathbf{11.4}\pm \mathbf{0.42}$	$11.3\pm 0.81$	$\mathbf{11.4}\pm \mathbf{0.42}$
BLP	$116\pm 5.4$	$232\pm 10$	$232\pm 12$	$\mathbf{234}\pm \mathbf{11}$	$232\pm 9.3$
Funnel	$0.887\pm 0.63$	$1.83\pm 2.2$	$1.89\pm 2.8$	$\mathbf{2.85}\pm \mathbf{3.6}$	$2.04\pm 3.1$

Table 3.

Minimum across dimensions of the Markov chain effective sample size per leapfrog step, averaged over 20 independent runs of 50 parallel chains of length $T=2000$, for the estimation of the variance. For eHMC, the proposal is trained using ${\lambda }_0=0.8$, $N_0=50000$, from which N resampled particles are used to construct the empirical measure on the integration parameters. Values are reported as mean $\pm 2$ standard errors, scaled by a factor ${10}^4$. Values in bold indicate the best performance across the methods and experimental configurations

		eHMC
Model	NUTS	$\mathrm{\Delta }/N_0=5\%$		$\mathrm{\Delta }/N_0=10\%$
		$N=2000$	$N=5000$	$N=2000$	$N=5000$
(a) Results for a target Metropolis–Hastings acceptance probability $p_0=0.651$.
Banana	$3.39\pm 6.3$	$7.29\pm 10$	$7.46\pm 11$	$6.36\pm 9.8$	$\mathbf{7.36}\pm \mathbf{10}$
MVNorm	$10.9\pm 0.81$	$12.5\pm 1.6$	$12.5\pm 1.4$	$12.5\pm 1.3$	$\mathbf{12.5}\pm \mathbf{1.4}$
BLP	$\mathbf{97.9}\pm \mathbf{5.5}$	$70.3\pm 4.8$	$69.8\pm 5.1$	$70.5\pm 5.1$	$70.0\pm 4.8$
Funnel	$\mathbf{6.09}\pm \mathbf{5.4}$	$2.14\pm 3.0$	$1.9\pm 1.9$	$1.83\pm 2.4$	$2.29\pm 3.0$
(b) Results for a target Metropolis–Hastings acceptance probability $p_0=0.8$
Banana	$6.61\pm 13$	$7.79\pm 9.4$	$\mathbf{10.1}\pm \mathbf{11}$	$9.28\pm 11$	$10.3\pm 11$
MVNorm	$8.28\pm 0.69$	$12.1\pm 1.2$	$\mathbf{12.2}\pm \mathbf{1.2}$	$12.1\pm 1.2$	$\mathbf{12.2}\pm \mathbf{1.2}$
BLP	$\mathbf{85.7}\pm \mathbf{3.2}$	$75.4\pm 5.7$	$75.0\pm 5.3$	$75.7\pm 6.0$	$73.7\pm 5.0$
Funnel	$\mathbf{6.77}\pm \mathbf{4.7}$	$1.53\pm 2.0$	$1.57\pm 2.0$	$2.27\pm 2.6$	$1.92\pm 3.0$

For the banana, MVNorm, and BLP targets, eHMC generally yields larger ${\mathrm{ESS}}_{\mathrm{MC}}(\theta )$ per gradient evaluation than NUTS across the configurations considered, with only modest sensitivity to N and $\mathrm{\Delta }/N_0$. The largest improvements are observed for the banana distribution, where eHMC achieves a median ${\mathrm{ESS}}_{\mathrm{MC}}(\theta )$ approximately 2 times larger than NUTS, while it is around 1.5 times larger for both BLP and MVNorm targets. On these model, between the configurations, eHMC generally provides a modest but consistent additional gain when $p_0$ increases. Furthermore, Fig. 2 shows that for these models eHMC exhibits improved sampling efficiency even along the least efficient direction of the sampler, although we observe a higher variability for the banana target. The funnel distribution exhibits a more nuanced pattern. Although eHMC retains a larger minimum coordinate-wise ${\mathrm{ESS}}_{\mathrm{MC}}(\theta )$ for mean estimation than NUTS (see Table 2), this metric does not capture movement between narrower and broader parts of the target (see results on ESJD). Overall, NUTS outperforms eHMC in terms of ${\mathrm{ESS}}_{\mathrm{MC}}(\theta )$ on this example. Increasing $p_0$ produces only modest and non-uniform changes across the funnel experiments.

Results for variance estimation are slightly more nuanced. For the banana, MVNorm, and BLP targets, although eHMC achieves competitive or improved efficiency compared to NUTS in terms of the median ${\mathrm{ESS}}_{\mathrm{MC}}({\theta }^2)$ when accounting for the number of leapfrog steps performed, performance in the least efficient direction of the sampler deteriorates for the BLP target, as illustrated in Fig. 2. The contrast is stronger for the funnel example: NUTS yields a minimum variance ESS approximately three to four times larger than eHMC, depending on $p_0$ and the calibration configuration. Overall NUTS provides an ${\mathrm{ESS}}_{\mathrm{MC}}({\theta }^2)$ that overseeds eHMC performance on this multiscale target.

The expected squared jump distance (ESJD) Furthermore, considering that the ${\mathrm{ESS}}_{\mathrm{MC}}$ criterion only reflects on the marginal efficiency of a sampler, we report in addition the standard expected squared jump distance. While depending on the measure of the jump, this quantity appears as a global mixing assessment that expresses another aspect of the efficiency of a sampler. We recall that, given generated samples $\{{\theta }^{(1)},\dots ,{\theta }^{(T)}\}$, its estimated ESJD is defined as

$$\begin{array}{c}\hfill \text{EJSD}=\frac{1}{T-1}\sum _{t=1}^{T-1}{∥{\theta }^{(t+1)},-,{\theta }^{(t)}∥}_2^2.\end{array}$$

Table 1 summarizes the ESJD per gradient evaluation averaged over 20 independent runs (see Fig. 3 in  Appendix E for the full distribution). For the banana, MVNorm, and BLP targets, eHMC yields larger ESJD per gradient evaluation than NUTS. The funnel shows the opposite behavior: NUTS has an ESJD approximately 54 times larger for $p_0=0.651$ and 19 times larger for $p_0=0.8$ than the best-performing eHMC configuration. The funnel example shows a substantially larger ESJD for NUTS, approximately 54 times larger for $p_0=0.651$ and 19 times larger for $p_0=0.8$, than the best-performing eHMC configuration. This indicates that eHMC makes much shorter effective moves through the state space. This behavior is consistent with the strong spatial heterogeneity of the funnel geometry, where the integration lengths required for efficient exploration vary considerably across the distribution support. Consequently, a single offline distribution of integration lengths is less able to accommodate these local changes than the online adaptation performed by NUTS.

Table 1.

Expected squared jump distance averaged over 20 independent runs of 50 parallel chains of length $T=2000$. For eHMC, the proposal is trained using ${\lambda }_0=0.8$, $N_0=50000$, from which N resampled particles are used to construct the empirical measure on the integration parameters. Values are reported as mean $\pm 2$ standard errors, scaled by a factor ${10}^6$. Values in bold indicate the best performance across the methods and experimental configurations

		eHMC
Model	NUTS	$\mathrm{\Delta }/N_0=5\%$		$\mathrm{\Delta }/N_0=10\%$
		$N=2000$	$N=5000$	$N=2000$	$N=5000$
(a) Results for a target Metropolis–Hastings acceptance probability $p_0=0.651$.
Banana	$44.2\pm 2.7$	$54.6\pm 2.9$	$54.4\pm 2.6$	$\mathbf{54.9}\pm \mathbf{2.5}$	$54.7\pm 2.1$
MVNorm	$3.88\pm 0.062$	$5.59\pm 0.094$	$\mathbf{5.60}\pm \mathbf{0.12}$	$5.59\pm 0.094$	$\mathbf{5.60}\pm \mathbf{0.12}$
BLP	$0.742\pm 0.0053$	$1.15\pm 0.026$	$1.15\pm 0.019$	$\mathbf{1.16}\pm \mathbf{0.016}$	$1.15\pm 0.018$
Funnel	$\mathbf{1610}\pm \mathbf{930}$	$30.1\pm 42$	$27.8\pm 47$	$28.3\pm 40$	$27\pm 50$
(b) Results for a target Metropolis–Hastings acceptance probability $p_0=0.8$.
Banana	$37.7\pm 2.9$	$\mathbf{58.6}\pm \mathbf{3.5}$	$58.2\pm 2.9$	$58.3\pm 2.6$	$58.0\pm 2.1$
MVNorm	$3.2\pm 0.064$	$5.27\pm 0.093$	$\mathbf{5.27}\pm \mathbf{0.080}$	$5.27\pm 0.093$	$\mathbf{5.27}\pm \mathbf{0.080}$
BLP	$0.648\pm 0.012$	$1.19\pm 0.024$	$\mathbf{1.19}\pm \mathbf{0.015}$	$1.19\pm 0.017$	$1.19\pm 0.016$
Funnel	$\mathbf{599}\pm \mathbf{310}$	$31.6\pm 44$	$28.1\pm 49$	$27.8\pm 39$	$25.1\pm 46$

Fig. 3.

Distribution of the expected squared jump distance aggregated over 20 independent runs of 50 parallel chains of length $T=2000$. For eHMC, the proposal is trained using ${\lambda }_0=0.8$, $N_0=50000$, from which $N\in \{2000,5000\}$ resampled particles are used to construct the empirical measure on the integration parameters. Results are shown for $\mathrm{\Delta }/N_0\in \{0.05,0.10\}$ in the x-axis

Taken together, the experiments expose settings in which our approach is useful. On the banana, MVNorm, and BLP examples, our empirical distribution of integration parameters yields higher sampling efficiency than NUTS for several of the considered metrics. The funnel provides a contrasting result. Because the appropriate integration scale varies strongly with position, the empirical distribution cannot reproduce the benefits of local trajectory adaptation, and NUTS performs substantially better in terms of both sampling efficiency and global movement of the Markov chain. In conclusion, and rather unsurprisingly, eHMC constitutes a relevant, automated, approach to randomized HMC that is well suited to targets whose relevant integration scales can be learned and reused offline. However, it is not intended to replace locally adaptive methods such as NUTS in settings where the appropriate integration time varies substantially across the state space.

Extensions

While the proposed approach focuses on learning a distribution over integration parameters to improve sampling efficiency, it can be naturally extended in several directions. In particular, we could introduce a partial refreshment of the momentum variable v, as commonly considered in persistent HMC methods (Neal 2001). Rather than fully resampling the momentum at each iteration, the idea is to retain it with probability $\eta \in [0,1]$, thereby inducing correlation between successive trajectories and potentially improving exploration efficiency. This partial refreshment also enables trajectory reuse and can be combined with Rao–Blackwellization strategies. This modification can be exploited in the empirical distribution of integration parameters. The resulting algorithm is referred to as prHMC.

More formally, for any $(\epsilon ,L)$, let $P_{\epsilon ,L}$ denote the HMC transition kernel with step size $\epsilon$ and L leapfrog steps, and let ${\widehat{\mu }}_{\mathcal{T}}$ be a probability measure on $(0,\infty )\times \mathbb{N}$ such as (6). We then introduce a refreshment kernel that updates the auxiliary variables $(v,\epsilon ,L)$. At each iteration, the momentum is refreshed with probability $\eta \in [0,1]$. Two variants can be considered. In the first variant, the integration parameters $(\epsilon ,L)$ are kept fixed as long as the momentum is not refreshed. The corresponding kernel writes as

$$\begin{array}{cc}& R_{\eta }^{(1)}\left((v,\epsilon ,L),,,\text{d},v^{\prime },\text{d},{\epsilon }^{\prime },\text{d},L^{\prime }\right)\hfill \\ \hfill & =(1-\eta ){\delta }_{(v,\epsilon ,L)}(\text{d}{v}^{\prime }\text{d}{\epsilon }^{\prime }\text{d}{L}^{\prime })+\eta \mathcal{N}(0,M)(\text{d}{v}^{\prime }){\widehat{\mu }}_{\mathcal{T}}(\text{d}{\epsilon }^{\prime },\text{d}{L}^{\prime }).\hfill \end{array}$$

In the second variant, the step size $\epsilon$ is kept fixed when the momentum is not refreshed, while the number of leapfrog steps L is resampled at each iteration according to the marginal empirical distribution

$$\begin{array}{c}\hfill {\widehat{\mu }}_{\mathcal{L}}^{\star }=\frac{1}{2}\sum _{i=1}^N{\omega }_i{\delta }_{L({\vartheta }_i,v_i,{\epsilon }_i^{+})}+\frac{1}{2}\sum _{i=1}^N{\omega }_i{\delta }_{L({\vartheta }_i,-v_i,{\epsilon }_i^{-})},\end{array}$$

while the step size $\epsilon$ is resampled only when the momentum is refreshed. The corresponding kernel then writes as

$$\begin{array}{cc}& R_{\eta }^{(2)}\left((v,\epsilon ,L),,,\text{d},v^{\prime },\text{d},{\epsilon }^{\prime },\text{d},L^{\prime }\right)\hfill \\ \hfill & =(1-\eta ){\delta }_v(\text{d}{v}^{\prime }){\delta }_{\epsilon }(\text{d}{\epsilon }^{\prime }){\widehat{\mu }}_{\mathcal{L}}^{\star }(\text{d}{L}^{\prime })+\eta \mathcal{N}(0,M)(\text{d}{v}^{\prime }){\widehat{\mu }}_{\mathcal{T}}(\text{d}{\epsilon }^{\prime },\text{d}{L}^{\prime }).\hfill \end{array}$$

Keeping the step size fixed when the momentum is not refreshed enables the reuse of points previously computed along the discretized trajectory. This is achieved thanks to a flipping mechanism: the momentum is negated when a HMC proposal is rejected and left unchanged otherwise.

$$\begin{array}{c}\hfill Q\left((\theta ,v),,,\text{d},{\theta }^{\prime },\text{d},v^{\prime }\right)={\delta }_{\theta }(\text{d}{\theta }^{\prime }){\delta }_{-v}(\text{d}{v}^{\prime }).\end{array}$$

As a result, the sampler can explore a given level set both forward and backward, thereby reusing previously generated points when the direction of the momentum flips.

The prHMC kernel is then defined for any $\eta \in [0,1]$ as the mixture kernel obtained by composing the refreshment kernel and the flipping kernel with the HMC kernel, i.e.,

$$\begin{array}{cc}& P_{\eta }^{(i)}((\theta ,v,\epsilon ,L),\text{d}{\theta }^{\prime }\text{d}{v}^{\prime }\text{d}{\epsilon }^{\prime }\text{d}{L}^{\prime })\hfill \\ \hfill & =\int Q((\overline{\theta },\overline{v}),\text{d}{\theta }^{\prime }\text{d}{v}^{\prime })P_{{\epsilon }^{\prime },L^{\prime }}\left((,\theta ,,,\tilde{v},),,,\text{d},\overline{\theta },\text{d},\overline{v}\right)\hfill \\ \hfill & R_{\eta }^{(i)}\left((v,\epsilon ,L),,,\text{d},\tilde{v},\text{d},{\epsilon }^{\prime },\text{d},L^{\prime }\right).\hfill \end{array}$$

Since the Gaussian momentum distribution is invariant under the map $v\mapsto -v$, the flipping kernel Q preserves the augmented target $\pi \otimes \mathcal{N}(0,M)$. Moreover, the refreshment kernel $R_{\eta }^{(i)}$ preserves $\mathcal{N}(0,M)\otimes {\widehat{\mu }}_{\mathcal{T}}$ by construction. In conclusion, since the HMC kernel leaves $\pi \otimes \mathcal{N}(0,M)$ invariant, the prHMC kernel leaves the augmented measure $\pi \otimes \mathcal{N}(0,M)\otimes {\widehat{\mu }}_{\mathcal{T}}$ invariant.

While this extension is theoretically appealing and naturally complements the proposed framework by leveraging intermediate proposals along the leapfrog path to reduce the computational cost, the empirical results obtained on the set of examples considered in this paper remain inconclusive in terms of sampling efficiency (Figs. 4 and 5 in Appendix E). In particular, significant improvements are observed only on the banana distribution (Fig. 4). In terms of ESJD, performances slightly deteriorate compared with eHMC, although it remains superior to NUTS (Fig. 6 in Appendix E). This suggests that the benefits of partial refreshment may depend sensitively on the geometry of the target distribution and the interaction with the learned integration parameters. Further investigation is therefore required to better understand the regimes in which this approach is most effective.

Fig. 4.

Distribution of the Markov chain effective sample size per number of leapfrog steps for mean (left panel) and variance (right panel) estimation, aggregated across dimensions and 20 independent runs of 50 parallel chains of length $T=2000$. prHMC denotes the variant $P_{\eta }^{(1)}$, where $(v,\epsilon ,L)$ are partially refreshed with probability $\eta =0.75$. The same proposal is used for both eHMC and prHMC, trained with ${\lambda }_0=0.8$, $N_0=50000$ samples, from which $N\in \{2000,5000\}$ particles are resampled to construct the empirical distribution on integration parameters. Results are shown for $\mathrm{\Delta }/N_0\in \{0.05,0.10\}$ in the x-axis

Fig. 5.

Distribution of the Markov chain effective sample size per number of leapfrog steps for mean (left panel) and variance (right panel) estimation, aggregated across dimensions and 20 independent runs of 50 parallel chains of length $T=2000$. prHMC denotes the variant $P_{\eta }^{(2)}$, where $(v,\epsilon )$ are partially refreshed with probability $\eta =0.75$. The same proposal is used for both eHMC and prHMC, trained with ${\lambda }_0=0.8$, $N_0=50000$ samples, from which $N\in \{2000,5000\}$ particles are resampled to construct the empirical distribution on integration parameters. Results are shown for $\mathrm{\Delta }/N_0\in \{0.05,0.10\}$ in the x-axis

Fig. 6.

Distribution of the expected squared jumping distance over 20 independent runs of 50 parallel chains of length $T=2000$. The left panel shows the prHMC variant $P_{\eta }^{(1)}$, where $(v,\epsilon ,L)$ are partially refreshed, and the right panel shows $P_{\eta }^{(2)}$, where $(v,\epsilon )$ are partially refreshed, with $\eta =0.75$. Both methods use the same proposal, trained with ${\lambda }_0=0.8$, $N_0=50000$ samples, from which $N\in \{2000,5000\}$ particles are resampled to form the empirical distribution on integration parameters. Results are shown for $\mathrm{\Delta }/N_0\in \{0.05,0.10\}$ in the x-axis

Finitness of the U-turn condition

Notation 1

In what follows, let denote $(\theta (t),v(t))$ the solution of the Hamilton’s equations (1) at time t.

Preliminary result

Lemma 2

Assume that the target distribution $\pi$ admits a continuously differentiable density on ${\mathbb{R}}^d$, and that the potential $U:\theta \mapsto -log\tilde{\pi }(\theta )$. Then, given $(\theta (0),v(0))\sim \pi \otimes \mathcal{N}(0,M)$, there exists $t^{\star }>0$ such that the continuous-time Hamiltonian trajectory at time $t^{\star }$ satisfies

$$\begin{array}{c}\hfill {(\theta (t^{\star })-\theta (0))}^{\top }{M}^{-1}v(t^{\star })<0,\end{array}$$

almost surely.

Proof

In continuous-time Hamiltonian dynamics, the conservation of the Hamiltonian and coercivity of U yield that the level sets $\{(\theta ,v):H(\theta ,v)\le c\}$ are compact. Otherwise, given $v\in {\mathbb{R}}^d$, we could find a sequence ${({\theta }_n)}_{n\ge 1}$ such that ${lim}_{n\to \infty }{\theta }_n=\infty$ and for all $n\ge 1$, $U({\theta }_n)\le c$, which contradicts the coercivity assumption. Consequently, for all $t\ge 0$, the trajectory $(\theta (t),v(t))$ remains in a compact set $\mathcal{C}$.

Let define for all $t\ge 0$, $g(t)={\Vert \theta (t)-\theta (0)\Vert }_2^2$. Then, using Hamiltonian’s equations,

$$\begin{array}{c}\hfill g^{\prime }(t)=2{(\theta (t)-\theta (0))}^{\top }{M}^{-1}v(t).\end{array}$$

Hence, to prove the result, we establish that g is a decreasing function.

Assume that, for all $t\ge 0$, $g^{\prime }(t)\ge 0$. Then g is non-decreasing, and, since $(\theta (t),v(t))$ remain in a compact set, g(t) is bounded. It follows that ${lim}_{t\to \infty }g(t)$ is finite. Since the Hamiltonian flow preserves the Lebesgue measure and the Lebesgue measure is finite on the compact set $\mathcal{C}$, the Poincaré recurrence theorem implies that, for almost every initial condition $(\theta (0),v(0))$, the trajectory returns arbitrarily close to its starting point infinitely often, i.e., there exists a sequence ${(t_n)}_{n\ge 0}$ such that

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}{t}_n=\infty ,\underset{n\to \infty }{lim}(\theta (t_n),v(t_n))=(\theta (0),v(0)).\end{array}$$

It follows, in particular, that ${lim}_{n\to \infty }g(t_n)=0$.

Since g is non-decreasing and $g(0)=0$, this implies that for all $t\ge 0$, $g(t)=0$. Thus, for all $t\ge 0$, we have $\theta (t)=\theta (0)$. By the first Hamiltonian equation, we get that for all $t\ge 0$, $v(t)=0$. Therefore, the trajectories for which $(\theta (t),v(t))$ remains constant belong to the set of trajectories such that $v(0)=0$, a set with zero measure under $\pi \otimes \mathcal{N}(0,M)$.

In conclusion, for almost every initial condition, there exists $t^{\star }>0$ such that

$$\begin{array}{c}\hfill \frac{1}{2}{g}^{\prime }(t^{\star })={(\theta (t^{\star })-\theta (0))}^{\top }{M}^{-1}v(t^{\star })<0.\end{array}$$

$\square$

Proof of Lemma 1

Let define for all $(\theta ,v)$, and all $t\ge 0$,

$$\begin{array}{c}\hfill g(t)={(\theta (t)-\theta )}^{\top }{M}^{-1}v(t^{\star }).\end{array}$$

From Lemma 2, for $\pi \otimes \mathcal{N}(0,M)$-almost every initial condition $(\theta ,v)$, there exists a finite time $t^{\star }>0$ such that $g(t^{\star })<0$. Since g is continuous, for all $\alpha <0$, there exists $\eta >0$ such that

$$\begin{array}{c}\hfill \forall t\in (t^{\star }-\eta ,t^{\star }+\eta ),g(t)\le \alpha .\end{array}$$ A1

Let $\alpha <0$. Given $\epsilon >0$, the leapfrog integrator approximates the true solution with error $\mathcal{O}({\epsilon }^2)$ on any fixed finite time interval (Leimkuhler and Reich 2005), i.e., there exists a constant $C_1>0$ such that for all $\ell$ with $\epsilon \ell \le t^{\star }+\eta$

$$\begin{array}{c}\hfill {∥({\theta }_{\ell },v_{\ell }),-,(\theta (\epsilon \ell ),v(\epsilon \ell ))∥}_2\le C_1{\epsilon }^2,({\theta }_{\ell },v_{\ell })=F_{\epsilon }^{\ell }(\theta ,v).\end{array}$$

Moreover, in the proof of Lemma 2 we have seen that the level sets of the Hamiltonian $\{(\theta ,v):H(\theta ,v)\le c\}$ are compact. From the approximation error of the leapfrog integrator, there exists $C_2>0$ such that for all $\ell$ with $\epsilon \ell \le t^{\star }+\eta$,

$$\begin{array}{c}\hfill \left|H,({\theta }_{\ell },v_{\ell }),-,H,(\theta ,v)\right|\le C_2{\epsilon }^2.\end{array}$$

It follows that

$$\begin{array}{c}\hfill ({\theta }_{\ell },v_{\ell })\in \left\{({\theta }^{\prime },v^{\prime }),:,H,({\theta }^{\prime },v^{\prime }),\le ,H,(\theta ,v),+,C_2,{\epsilon }^2\right\}.\end{array}$$

which is compact. The function $\psi :(\tilde{\theta },v)\mapsto {(\tilde{\theta }-\theta )}^{\top }{M}^{-1}v$ is infinitely differentiable on ${\mathbb{R}}^{2d}$ and therefore Lipschitz on any compact set. We thus have that there exists a constant $C_3$ such that for all $\ell$ with $\epsilon \ell \le t^{\star }+\eta$

$$\begin{array}{cc}& \left|\psi ,({\theta }_{\ell },v_{\ell }),-,g,(\epsilon \ell )\right|\hfill \\ \hfill & \le C_3{∥({\theta }_{\ell },v_{\ell }),-,(\theta (\epsilon \ell ),v(\epsilon \ell ))∥}_2\le C_1{C}_3{\epsilon }^2.\hfill \end{array}$$ A2

Let set

$$\begin{array}{c}\hfill {\epsilon }_0=\sqrt{\frac{-\alpha }{C_1{C}_3}}.\end{array}$$

For all $\epsilon \le {\epsilon }_0$, it follows from (A1) and (A2) that for all $\ell$ such that $|\epsilon \ell -t^{\star }|\le \eta$

$$\begin{array}{cc}\hfill \psi ({\theta }_{\ell },v_{\ell })& ={({\theta }_{\ell }-\theta )}^{\top }{M}^{-1}{v}_{\ell }\hfill \\ \hfill & \le C_1{C}_3{\epsilon }^2+g(\epsilon \ell )\le 2\alpha <0.\hfill \end{array}$$ A3

To conclude, there exists ${\epsilon }^{\star }\le {\epsilon }_0$ such that $\{\ell \in \mathbb{N}:|{\epsilon }^{\star }\ell -t^{\star }|\le \eta \}$ is non empty. Thus, for all $\epsilon \le {\epsilon }^{\star }$, there exists $\ell \in \mathbb{N}$ that satisfies (A3), which implies that $L(\theta ,v,\epsilon )<\infty$.

Formula for the mass matrix update

Using the same notations as in Sect. 4.1, let denote for $1\le i\le N$

$$\begin{array}{cccc}\hfill L_i^{+}& =L({\vartheta }_i,v_i,{\epsilon }_i^{+}),\hfill & \hfill {\rho }_{i,\ell }^{+}& =\rho ({\vartheta }_i,v_i,{\vartheta }_{i,\ell }^{+},v_{i,\ell }^{+}),\hfill \\ \hfill L_i^{-}& =L({\vartheta }_i,-v_i,{\epsilon }_i^{-}),\hfill & \hfill {\rho }_{i,\ell }^{-}& =\rho ({\vartheta }_i,-v_i,{\vartheta }_{i,\ell }^{-},v_{i,\ell }^{-}).\hfill \end{array}$$

The normalizing constant of the measure $\nu$ as defined in (7) is

$$\begin{array}{c}\hfill Z=\sum _{i=1}^N{\omega }_i\left(L_i^{+},+,L_i^{-}\right).\end{array}$$

Mean of the measure $\nu$

$$\begin{array}{cc}& m=\frac{1}{Z}\sum _{i=1}^N{\omega }_i\left[\sum _{\ell =1}^{L_i^{+}},\left\{(1-{\rho }_{i,\ell }^{+}),{\theta }_{i,\ell -1}^{+},+,{\rho }_{i,\ell }^{+},{\theta }_{i,\ell }^{+}\right\}\right)\hfill \\ \hfill & \left(+,\sum _{\ell =1}^{L_i^{-}},\left\{(1-{\rho }_{i,\ell }^{-}),{\theta }_{i,\ell -1}^{-},+,{\rho }_{i,\ell }^{-},{\theta }_{i,\ell }^{-}\right\}\right],\hfill \end{array}$$

Variance of the measure $\nu$ Let denote

$$\begin{array}{cc}& V_i^{+}=\sum _{\ell =1}^{L_i^{+}}(1-{\rho }_{i,\ell }^{+})({\theta }_{i,\ell -1}^{+}-m){({\theta }_{i,\ell -1}^{+}-m)}^{\top },\hfill \\ \hfill & S_i^{+}=\sum _{\ell =1}^{L_i^{+}}{\rho }_{i,\ell }^{+}({\theta }_{i,\ell }^{+}-m){({\theta }_{i,\ell }^{+}-m)}^{\top },\hfill \end{array}$$ B4

$$\begin{array}{cc}& V_i^{-}=\sum _{\ell =1}^{L_i^{-}}(1-{\rho }_{i,\ell }^{-})({\theta }_{i,\ell -1}^{-}-m){({\theta }_{i,\ell -1}^{-}-m)}^{\top },\hfill \\ \hfill & S_i^{-}=\sum _{\ell =1}^{L_i^{-}}{\rho }_{i,\ell }^{-}({\theta }_{i,\ell }^{-}-m){({\theta }_{i,\ell }^{+}-m)}^{\top }.\hfill \end{array}$$ B5

Then the variance is given by

$$\begin{array}{c}\hfill {\text{Var}}_{\nu }(\theta )=\frac{1}{Z}\sum _{i=1}^N{\omega }_i\left(V_i^{+},+,S_i^{+},+,V_i^{-},+,S_i^{-}\right).\end{array}$$

Invariance of the randomized kernel

Lemma 3

Let ${\widehat{\mu }}_{\mathcal{T}}$ be a probability measure on $(0,\infty )\times \mathbb{N}$ constructed during the calibration stage and kept fixed during the sampling phase. For any $(\epsilon ,L)$, let $P_{\epsilon ,L}$ denote the Hamiltonian Monte Carlo transition kernel with step size $\epsilon$ and L leapfrog steps. Define the Markov transition kernel

$$\begin{array}{c}\hfill P(x,\text{d}{x}^{\prime })=\int P_{\epsilon ,L}(x,\text{d}{x}^{\prime }){\widehat{\mu }}_{\mathcal{T}}(\text{d}\epsilon ,\text{d}L).\end{array}$$

Then P leaves $\pi$ invariant.

Proof

For any fixed $(\epsilon ,L)$, the Hamiltonian Monte Carlo kernel $P_{\epsilon ,L}$ preserves the target distribution $\pi$, that is

$$\begin{array}{c}\hfill \int \pi (\text{d}x)P_{\epsilon ,L}(x,\text{d}{x}^{\prime })=\pi (\text{d}{x}^{\prime }).\end{array}$$

Then, by Fubini’s theorem,

$$\begin{array}{cc}\hfill \int \pi (\text{d}x)P(x,\text{d}{x}^{\prime })& =\int \pi (\text{d}x)\int P_{\epsilon ,L}(x,\text{d}{x}^{\prime }){\widehat{\mu }}_{\mathcal{T}}(\text{d}\epsilon ,\text{d}L)\hfill \\ \hfill & =\int \left(\int ,\pi ,(\text{d}x),P_{\epsilon ,L},(x,\text{d}{x}^{\prime })\right){\widehat{\mu }}_{\mathcal{T}}(\text{d}\epsilon ,\text{d}L)\hfill \\ \hfill & =\int \pi (\text{d}{x}^{\prime }){\widehat{\mu }}_{\mathcal{T}}(\text{d}\epsilon ,\text{d}L)\hfill \\ \hfill & =\pi (\text{d}{x}^{\prime }).\hfill \end{array}$$

Therefore $\pi$ is invariant for P. $\square$

Monotonicity of the effective sample size

Lemma 4

Let $\mu \in {\mathbb{M}}_{\pi }$. Assume that for all $\beta \in [0,1]$, $\pi /\mu \in L^{2\beta }(\mu )$. Then, the function $\beta \mapsto \text{ESS}\left({\tilde{\pi }}^{\beta },{\mu }^{1-\beta },,,\mu \right)$ is continuous and non-increasing on [0, 1].

Proof

Assume $\vartheta \sim \mu$. Let define the random variable $X=log(\pi (\vartheta )/\mu (\vartheta ))$. Then we can write

$$\begin{array}{c}\hfill \text{ESS}\left({\tilde{\pi }}^{\beta },{\mu }^{1-\beta },,,\mu \right)=\frac{{\mathbb{E}}_{\mu }{[exp(\beta X)]}^2}{{\mathbb{E}}_{\mu }[exp(2\beta X)]}.\end{array}$$

Continuity The mapping $\beta \mapsto exp(\beta X)$ is continuous on [0, 2]. Moreover, for all $\beta \in [0,2]$, $exp(\beta X)\le 1+exp(2X)$. Since, by assumption, ${\mathbb{E}}_{\mu }[exp(2X)]<\infty$, we have that $1+exp(2X)$ is integrable with respect to $\mu$. Therefore, by the dominated convergence theorem, $\beta \mapsto {\mathbb{E}}_{\mu }[exp(\beta X)]$ is continuous on [0, 2]. It follows that $\beta \mapsto \text{ESS}\left({\tilde{\pi }}^{\beta },{\mu }^{1-\beta },,,\mu \right)$ is continuous on [0, 1] as ratio of positive and continuous functions.

Monotonicity Define for all $\beta \in [0,1]$

$$\begin{array}{cc}& g(\beta )=log\text{ESS}\left({\tilde{\pi }}^{\beta },{\mu }^{1-\beta },,,\mu \right)=2\kappa (b)-\kappa (2\beta ),\hfill \\ \hfill & \kappa :\beta \mapsto log{\mathbb{E}}_{\mu }[exp(\beta X)].\hfill \end{array}$$

The function $\kappa$ is the cumulant generating function associated with X. It is twice differentiable on the interior of the set $\mathcal{D}=\{\beta \in \mathbb{R}:{\mathbb{E}}_{\mu }[exp(\beta X)]<\infty \}$. Since, for all $\beta \in [0,1]$, $\pi /\mu \in L^{2\beta }(\mu )$, we have that $[0,2]\subseteq \mathcal{D}$. Thus $\kappa$ is twice differentiable on (0, 2). If, for all $\beta \in (0,2)$, we define the exponentially tilted measure

$$\begin{array}{c}\hfill {\mu }_{\beta }(\text{d}\vartheta )=\frac{exp(\beta X)}{{\mathbb{E}}_{\mu }[exp(\beta X)}\mu (\text{d}\vartheta ),\end{array}$$

we have

$$\begin{array}{c}\hfill {\kappa }^{\prime }(\beta )={\mathbb{E}}_{{\mu }_{\beta }}[X],{\kappa }^{\prime \prime }(\beta )=\mathbb{V}{\text{ar}}_{{\mu }_{\beta }}[X]\ge 0.\end{array}$$

It follows that ${\kappa }^{\prime }$ is non-decreasing on (0, 2) and for all $\beta \in (0,1)$,

$$\begin{array}{c}\hfill g^{\prime }(\beta )=2{\kappa }^{\prime }(b)-2{\kappa }^{\prime }(2\beta )\le 0.\end{array}$$

Therefore, g is non-increasing on (0, 1). Since g is continuous on [0, 1], we get that g, and consequently $\beta \mapsto \text{ESS}\left({\tilde{\pi }}^{\beta },{\mu }^{1-\beta },,,\mu \right)$, is non-increasing on [0, 1]. $\square$

Additional numerical results

prHMC denotes the variant $P_{\eta }^{(1)}$, where $(v,\epsilon ,L)$ are partially refreshed with probability $\eta =0.75$. The same proposal is used for both eHMC and prHMC, trained with $N_0=50000$ samples, from which $N\in \{2000,5000\}$ particles are resampled to construct the empirical distribution on integration parameters. Results are shown for $\mathrm{\Delta }/N_0\in \{0.05,0.10\}$ in the x-axis.

Author Contributions

Changye Wu, Julien Stoehr and Christian P Robert contributed to the creation and development of the method and to the redaction of the paper. Changye Wu then Julien Stoehr wrote successive versions of the code behind the experiments. Pierre Pudlo suggested a significant improvement to the approach that led to the current version and reviewed the manuscript.

Data Availability

The code used in this article is available at https://github.com/jstoehr/eHMC

Code availability

The code used in this article is available at https://github.com/jstoehr/eHMC.