Using Perturbed Underdamped Langevin Dynamics to Efficiently Sample from Probability Distributions

Abstract

In this paper we introduce and analyse Langevin samplers that consist of perturbations of the standard underdamped Langevin dynamics. The perturbed dynamics is such that its invariant measure is the same as that of the unperturbed dynamics. We show that appropriate choices of the perturbations can lead to samplers that have improved properties, at least in terms of reducing the asymptotic variance. We present a detailed analysis of the new Langevin sampler for Gaussian target distributions. Our theoretical results are supported by numerical experiments with non-Gaussian target measures.

Introduction and Motivation

Sampling from probability measures in high-dimensional spaces is a problem that appears frequently in applications, e.g. in computational statistical mechanics and in Bayesian statistics. In particular, we are faced with the problem of computing expectations with respect to a probability measure $\mathit{\pi }$ on ${\mathbb{R}}^d$, i.e. we wish to evaluate integrals of the form:

$$\begin{array}{c}\hfill \mathit{\pi }(f):={\int }_{{\mathbb{R}}^d}f(x)\mathit{\pi }(\mathrm{d}x).\end{array}$$ 1

As is typical in many applications, particularly in molecular dynamics and Bayesian inference, the density (for convenience denoted by the same symbol $\mathit{\pi }$) is known only up to a normalization constant; furthermore, the dimension of the underlying space is quite often large enough to render deterministic quadrature schemes computationally infeasible.

A standard approach to approximating such integrals is Markov Chain Monte Carlo (MCMC) techniques [19, 32, 52], where a Markov process ${(X_t)}_{t\ge 0}$ is constructed which is ergodic with respect to the probability measure $\mathit{\pi }$. Then, defining the long-time average

$$\begin{array}{c}\hfill {\mathit{\pi }}_T(f):=\frac{1}{T}{\int }_0^{T}f(X_s)\mathrm{d}s\end{array}$$ 2

for $f\in L^1(\mathit{\pi })$, the ergodic theorem guarantees almost sure convergence of the long-time average ${\mathit{\pi }}_T(f)$ to $\mathit{\pi }(f)$.

There are infinitely many Markov, and, for the purposes of this paper diffusion, processes that can be constructed in such a way that they are ergodic with respect to the target distribution. A natural question is then how to choose the ergodic diffusion process ${(X_t)}_{t\ge 0}$. Naturally the choice should be dictated by the requirement that the computational cost of (approximately) calculating (1) is minimized. A standard example is given by the overdamped Langevin dynamics defined to be the unique (strong) solution ${(X_t)}_{t\ge 0}$ of the following stochastic differential equation (SDE):

$$\begin{array}{c}\hfill \mathrm{d}{X}_t=-\mathrm{\nabla }V(X_t)\mathrm{d}t+\sqrt{2}\mathrm{d}{W}_t,\end{array}$$ 3

where $V=-log\mathit{\pi }$ is the potential associated with the smooth positive density $\mathit{\pi }$. Under appropriate assumptions on V, i.e. on the measure $\mathit{\pi }(\mathrm{d}x)$, the process ${(X_t)}_{t\ge 0}$ is ergodic and in fact reversible with respect to the target distribution.

Another well-known example is the underdamped Langevin dynamics given by ${(X_t)}_{t\ge 0}={(q_t,p_t)}_{t\ge 0}$ defined on the extended space (phase space) ${\mathbb{R}}^d\times {\mathbb{R}}^d$ by the following pair of coupled SDEs:

$$\begin{array}{c}\hfill \mathrm{d}{q}_t=M^{-1}{p}_t\mathrm{d}t,\mathrm{d}{p}_t=-\mathrm{\nabla }V(q_t)\mathrm{d}t-\mathit{\Gamma }{M}^{-1}{p}_t\mathrm{d}t+\sqrt{2\mathit{\Gamma }}\mathrm{d}{W}_t,\end{array}$$ 4

where the mass and friction tensors M and $\mathit{\Gamma }$ are assumed to be symmetric positive definite matrices. It is well-known [36, 46] that ${(q_t,p_t)}_{t\ge 0}$ is ergodic with respect to the measure $\widehat{\mathit{\pi }}:=\mathit{\pi }\otimes \mathcal{N}(0,M)$, having density with respect to the Lebesgue measure on ${\mathbb{R}}^{2d}$ given by

$$\begin{array}{c}\hfill \widehat{\mathit{\pi }}(q,p)=\frac{1}{\widehat{Z}}exp\left(-V(q)-\frac{1}{2}p·M^{-1}p\right),\end{array}$$ 5

where $\widehat{Z}$ is a normalization constant. Note that $\widehat{\mathit{\pi }}$ has marginal $\mathit{\pi }$ with respect to p and thus for functions $f\in L^1(\mathit{\pi })$, we have that $\frac{1}{t}{\int }_0^{t}f(q_t)\mathrm{d}t\to \mathit{\pi }(f)$ almost surely. Notice also that the dynamics restricted to the q-variables is no longer Markovian. The p-variables can thus be interpreted as giving some instantaneous memory to the system, facilitating efficient exploration of the state space. Higher order Markovian models, based on a finite dimensional (Markovian) approximation of the generalized Langevin equation can also be used [12].

As there is a lot of freedom in choosing the dynamics in (2), see the discussion in Sect. 2, it is desirable to choose the diffusion process ${(X_t)}_{t\ge 0}$ in such a way that ${\mathit{\pi }}_T(f)$ can provide a good estimation of $\mathit{\pi }(f)$. The performance of the estimator (2) can be quantified in various manners. The ultimate goal, of course, is to choose the dynamics as well as the numerical discretization in such a way that the computational cost of the longtime-average estimator is minimized, for a given tolerance. The minimization of the computational cost consists of three steps: bias correction, variance reduction and choice of an appropriate discretization scheme. For the latter step see Sect. 5 and [14, Sect. 6].

Under appropriate conditions on the potential V it can be shown that both (3) and (4) converge to equilibrium exponentially fast, e.g. in relative entropy. One performance objective would then be to choose the process ${(X_t)}_{t\ge 0}$ so that this rate of convergence is maximised. Conditions on the potential V which guarantee exponential convergence to equilibrium, both in $L^2(\mathit{\pi })$ and in relative entropy can be found in [7, 39, 54]. In the case when the target measure $\mathit{\pi }$ is Gaussian, both the overdamped  (3) and the underdamped (4) dynamics become generalized Ornstein–Uhlenbeck processes. For such processes the entire spectrum of the generator—or, equivalently, the Fokker–Planck operator—can be computed analytically and, in particular, an explicit formula for the $L^2$-spectral gap can be obtained [38, 43, 44]. A detailed analysis of the convergence to equilibrium in relative entropy for stochastic differential equations with linear drift, i.e. generalized Ornstein–Uhlenbeck processes, has been carried out in [1, 2].

In addition to speeding up convergence to equilibrium, i.e. reducing the bias of the estimator (2), one is naturally also interested in reducing the asymptotic variance. Under appropriate conditions on the target measure $\mathit{\pi }$ and the observable f, the estimator ${\mathit{\pi }}_T(f)$ satisfies a central limit theorem (CLT) [31], that is,

$$\begin{array}{c}\hfill \frac{1}{\sqrt{T}}\left({\mathit{\pi }}_T(f)-\mathit{\pi }(f)\right)\underset{T\to \infty }{\overset{d}{\to }}\mathcal{N}\left(0,2{\mathit{\sigma }}_f^2\right),\end{array}$$

where ${\mathit{\sigma }}_f^2<\infty$ is the asymptotic variance of the estimator ${\mathit{\pi }}_T(f)$. The asymptotic variance characterises the magnitude of fluctuations of ${\mathit{\pi }}_T(f)$ around $\mathit{\pi }(f)$. Consequently, another natural objective is to choose the process ${(X_t)}_{t\ge 0}$ such that ${\mathit{\sigma }}_f^2$ is as small as possible. It is well known that the asymptotic variance can be expressed in terms of the solution to an appropriate Poisson equation for the generator of the dynamics [31]

$$\begin{array}{c}\hfill -\mathcal{L}\mathit{\varphi }=f-\mathit{\pi }(f),{\mathit{\sigma }}_f^2={\int }_{{\mathbb{R}}^d}\mathit{\varphi }(-\mathcal{L}\mathit{\varphi })\mathit{\pi }(\mathrm{d}x).\end{array}$$ 6

Techniques from the theory of partial differential equations can then be used in order to study the problem of minimizing the asymptotic variance. This is the approach that was taken in [14], see also [23], and it will also be used in this paper.

Other measures of performance have also been considered. For example, in [50, 51], performance of the estimator is quantified in terms of the rate functional of the ensemble measure $\frac{1}{t}{\int }_0^t{\mathit{\delta }}_{X(t)}(dx)$. See also [28] for a study of the nonasymptotic behaviour of MCMC techniques, including the case of overdamped Langevin dynamics.

Similar analyses have been carried out for various modifications of (3). Of particular interest to us are the Riemannian manifold MCMC [18] (see the discussion in Sect. 2) and the nonreversible Langevin samplers [20, 21]. As a particular example of the general framework that was introduced in [18], we mention the preconditioned overdamped Langevin dynamics $\mathrm{d}{X}_t=-P\mathrm{\nabla }V(X_t)\mathrm{d}t+\sqrt{2P}\mathrm{d}{W}_t,$ that was presented in [4]. There, the long-time behaviour of as well as the asymptotic variance of the corresponding estimator ${\mathit{\pi }}_T(f)$ are studied and applied to equilibrium sampling in molecular dynamics. A variant of the standard underdamped Langevin dynamics that can be thought of as a form of preconditioning and that has been used by practitioners is the mass-tensor molecular dynamics [6].

The nonreversible overdamped Langevin dynamics

$$\begin{array}{c}\hfill \mathrm{d}{X}_t=-\left(\mathrm{\nabla }V(X_t)-\mathit{\gamma }(X_t)\right)\mathrm{d}t+\sqrt{2}\mathrm{d}{W}_t,\end{array}$$ 7

where the vector field $\mathit{\gamma }$ satisfies $\mathrm{\nabla }·(\mathit{\pi }\mathit{\gamma })=0$ is ergodic (but not reversible) with respect to the target measure $\mathit{\pi }$ for all choices of the divergence-free vector field $\mathit{\gamma }$. The asymptotic behaviour of this process was considered for Gaussian diffusions in [20], where the rate of convergence of the covariance to equilibrium was quantified in terms of the choice of $\mathit{\gamma }$. This work was extended to the case of non-Gaussian target densities, and consequently for nonlinear SDEs of the form (7) in [21]. The problem of constructing the optimal nonreversible perturbation, in terms of the $L^2(\mathit{\pi })$ spectral gap for Gaussian target densities was studied in  [34] see also [55]. Optimal nonreversible perturbations with respect to miniziming the asymptotic variance were studied in [14, 23]. In all these works it was shown that, in theory [i.e. without taking into account the computational cost of the discretization of the dynamics (7)], the nonreversible Langevin sampler (7) is never worse that its reversible counterpart (3), both in terms of converging faster to the target distribution as well as in terms of having a lower asymptotic variance. It should be emphasized that the two optimality criteria, maximizing the spectral gap and minimizing the asymptotic variance, lead to different choices for the nonreversible drift $\mathit{\gamma }(x)$.

The goal of this paper is to extend the analysis presented in [14, 34] by introducing the following modification of the standard underdamped Langevin dynamics:

$$\begin{array}{cc}\hfill \mathrm{d}{q}_t=& M^{-1}{p}_t\mathrm{d}t-\mathit{\mu }{J}_1\mathrm{\nabla }V(q_t)\mathrm{d}t,\hfill \\ \hfill \mathrm{d}{p}_t=& -\mathrm{\nabla }V(q_t)\mathrm{d}t-\mathit{\nu }{J}_2{M}^{-1}{p}_t\mathrm{d}t-\mathit{\Gamma }{M}^{-1}{p}_t\mathrm{d}t+\sqrt{2\mathit{\Gamma }}\mathrm{d}{W}_t,\hfill \end{array}$$ 8

where $M,\mathit{\Gamma }\in {\mathbb{R}}^{d\times d}$ are constant strictly positive definite matrices, $\mathit{\mu }$ and $\mathit{\nu }$ are scalar constants and $J_1,J_2\in {\mathbb{R}}^{d\times d}$ are constant skew-symmetric matrices. As demonstrated in Sect. 2, the process defined by (8) will be ergodic with respect to the Gibbs measure $\widehat{\mathit{\pi }}$ defined in (5).

Our objective is to investigate the use of these dynamics for computing ergodic averages of the form (2). To this end, we study the long time behaviour of (8) and, using hypocoercivity techniques, prove that the process converges exponentially fast to equilibrium. This perturbed underdamped Langevin process introduces a number of parameters in addition to the mass and friction tensors which must be tuned to ensure that the process is an efficient sampler. For Gaussian target densities, we derive estimates for the spectral gap and the asymptotic variance, valid in certain parameter regimes. Moreover, for certain classes of observables, we are able to identify the choices of parameters which lead to the optimal performance in terms of asymptotic variance. While these results are valid for Gaussian target densities, we advocate these particular parameter choices also for more complex target densities. To demonstrate their efficacy, we perform a number of numerical experiments on more complex, multimodal distributions. In particular, we use the Langevin sampler (8) in order to study the problem of diffusion bridge sampling.

The rest of the paper is organized as follows. In Sect. 2 we present some background material on Langevin dynamics, we construct general classes of Langevin samplers and we introduce criteria for assessing the performance of the samplers. In Sect. 3 we study qualitative properties of the perturbed underdamped Langevin dynamics (8) including exponentially fast convergence to equilibrium and the overdamped limit. In Sect. 4 we study in detail the performance of the Langevin sampler (8) for the case of Gaussian target distributions. In Sect. 5 we introduce a numerical scheme for simulating the perturbed dynamics (8) and we present numerical experiments on the implementation of the proposed samplers for the problem of diffusion bridge sampling. Section 6 is reserved for conclusions and suggestions for further work.

Construction of General Langevin Samplers

Background and Preliminaries

In this section we consider estimators of the form (2) where ${(X_t)}_{t\ge 0}$ is a diffusion process given by the solution of the following Itô SDE:

$$\begin{array}{c}\hfill \mathrm{d}{X}_t=a(X_t)\mathrm{d}t+\sqrt{2}b(X_t)\mathrm{d}{W}_t,\end{array}$$ 9

with drift coefficient $a:{\mathbb{R}}^d\to {\mathbb{R}}^d$ and diffusion coefficient $b:{\mathbb{R}}^d\to {\mathbb{R}}^{d\times m}$ both having smooth components, and where ${(W_t)}_{t\ge 0}$ is a standard ${\mathbb{R}}^m$–valued Brownian motion. Associated with (9) is the infinitesimal generator $\mathcal{L}$ formally given by

$$\begin{array}{c}\hfill \mathcal{L}f=a·\mathrm{\nabla }f+\mathit{\Sigma }:\mathrm{\nabla }\mathrm{\nabla }f,f\in C_c^2({\mathbb{R}}^d)\end{array}$$ 10

where $\mathit{\Sigma }=b{b}^{\top }$, $\mathrm{\nabla }\mathrm{\nabla }f$ denotes the Hessian of the function f and $:$ denotes the Frobenius inner product. In general, $\mathit{\Sigma }$ is nonnegative definite, and could possibly be degenerate. In particular, the infinitesimal generator (10) need not be uniformly elliptic. To ensure that the corresponding semigroup exhibits sufficient smoothing behaviour, we shall require that the process (9) is hypoelliptic in the sense of Hörmander. If this condition holds, then irreducibility of the process ${(X_t)}_{t\ge 0}$ will be an immediate consequence of the existence of a strictly positive invariant distribution $\mathit{\pi }(x)\mathrm{d}x$, see [30].

Suppose that ${(X_t)}_{t\ge 0}$ is nonexplosive. It follows from the hypoellipticity assumption that the process ${(X_t)}_{t\ge 0}$ possesses a smooth transition density p(t, x, y) which is defined for all $t\ge 0$ and $x,y\in {\mathbb{R}}^d$, [5, Theorem VII.5.6]. The associated strongly continuous Markov semigroup ${(P_t)}_{t\ge 0}$ is defined by $P_{t}f(x)={\int }_{{\mathbb{R}}^d}p(t,x,y)f(y)\mathrm{d}y.$ Suppose that ${(P_t)}_{t\ge 0}$ is invariant with respect to the target measure $\mathit{\pi }$, i.e. ${\int }_{{\mathbb{R}}^d}{P}_{t}f(x)\mathit{\pi }(\mathrm{d}x)={\int }_{{\mathbb{R}}^d}f(x)\mathit{\pi }(\mathrm{d}x)$ for $t\ge 0$ and all bounded continuous functions f. Then ${(P_t)}_{t\ge 0}$ can be extended to a positivity preserving contraction semigroup on $L^2(\mathit{\pi })$ which is strongly continuous. Moreover, the infinitesimal generator corresponding to ${(P_t)}_{t\ge 0}$ is given by an extension of $(\mathcal{L},C_c^2({\mathbb{R}}^d))$, also denoted by $\mathcal{L}$.

Due to hypoellipticity and invariance with respect to ${(P_t)}_{t\ge 0}$, the probability measure $\mathit{\pi }$ on ${\mathbb{R}}^d$ has a smooth density with respect to the Lebesgue measure. If this density is strictly positive, it follows that $\mathit{\pi }$ is necessarily the unique invariant distribution. Slightly abusing the notation, we will denote both the measure and its density by $\mathit{\pi }$. Furthermore, we will denote by $L^2(\mathit{\pi })$ be the Hilbert space of $\mathit{\pi }$-square integrable functions equipped with inner product ${〈·,·〉}_{L^2(\mathit{\pi })}$ and norm $\left||·\right|{|}_{L^2(\mathit{\pi })}$.

A General Characterisation of Ergodic Diffusions

A natural question is what conditions on the coefficients a and b of (9) are required to ensure that ${(X_t)}_{t\ge 0}$ is invariant with respect to the distribution $\mathit{\pi }(x)\mathrm{d}x$. The following result provides a necessary and sufficient condition for a diffusion process to be invariant with respect to a given target distribution.

Theorem 1

Consider a diffusion process ${(X_t)}_{t\ge 0}$ on ${\mathbb{R}}^d$ defined by the unique, non-explosive solution to the Itô SDE (9) with drift $a\in C^1({\mathbb{R}}^d;{\mathbb{R}}^d)$ and diffusion coefficient $b\in C^1({\mathbb{R}}^d;{\mathbb{R}}^{d\times m})$. Then ${(X_t)}_{t\ge 0}$ is invariant with respect to $\mathit{\pi }$ if and only if

$$\begin{array}{c}\hfill a=\mathit{\Sigma }\mathrm{\nabla }log\mathit{\pi }+\mathrm{\nabla }·\mathit{\Sigma }+\mathit{\gamma },\end{array}$$ 11

where $\mathit{\Sigma }=b{b}^{\top }$ and $\mathit{\gamma }:{\mathbb{R}}^D\to {\mathbb{R}}^D$ is a continuously differentiable vector field satisfying

$$\begin{array}{c}\hfill \mathrm{\nabla }·\left(\mathit{\pi }\mathit{\gamma }\right)=0.\end{array}$$ 12

If additionally $\mathit{\gamma }\in L^1(\mathit{\pi })$, then there exists a skew-symmetric matrix function $C:{\mathbb{R}}^d\to {\mathbb{R}}^{d\times d}$ such that $\mathit{\gamma }=\frac{1}{\mathit{\pi }}\mathrm{\nabla }·\left(\mathit{\pi }C\right).$ In this case the infinitesimal generator can be written as an $L^2(\mathit{\pi })$-extension of

$$\begin{array}{c}\hfill \mathcal{L}f=\frac{1}{\mathit{\pi }}\mathrm{\nabla }·\left((\mathit{\Sigma }+C)\mathit{\pi }\mathrm{\nabla }f\right),f\in C_c^2({\mathbb{R}}^d).\end{array}$$

The proof of the first part of this result can be found in [46, Chap. 4]; similar versions of this characterisation can be found in [54] and [21]. For the existence of the skew-symmetric matrix C see, e.g., [16, Sec.4, Prop. 1]. See also [37].

Remark 1

If (11) holds and $\mathcal{L}$ is hypoelliptic it follows immediately that ${(X_t)}_{t\ge 0}$ is ergodic with unique invariant distribution $\mathit{\pi }(x)\mathrm{d}x$ (see [30]).

More generally, we can consider Itô diffusions in an extended phase space:

$$\begin{array}{c}\hfill \mathrm{d}{Z}_t=b(Z_t)\mathrm{d}t+\sqrt{2}\mathit{\sigma }(Z_t)\mathrm{d}{W}_t,\end{array}$$ 13

where ${(W_t)}_{t\ge 0}$ is a standard Brownian motion in ${\mathbb{R}}^N$, $N\ge d$. This is a Markov process with generator

$$\begin{array}{c}\hfill \mathcal{L}=b(z)·{\mathrm{\nabla }}_z+\mathit{\Sigma }(z):{\mathrm{\nabla }}_z{\mathrm{\nabla }}_z,\end{array}$$ 14

where $\mathit{\Sigma }(z)=(\mathit{\sigma }{\mathit{\sigma }}^T)(z)$. We will consider dynamics ${(Z_t)}_{t\ge 0}$ that is ergodic with respect to ${\mathit{\pi }}_z(z)\mathrm{d}z$ such that ${\int }_{{\mathbb{R}}^m}{\mathit{\pi }}_z(x,y)\mathrm{d}y=\mathit{\pi }(x),$ where $z=(x,y),x\in {\mathbb{R}}^d,y\in {\mathbb{R}}^m,d+m=N$.

There are various well-known choices of dynamics which are invariant (and indeed ergodic) with respect to the target distribution $\mathit{\pi }(x)\mathrm{d}x$.

1. Choosing $b=I$ and $\mathit{\gamma }=0$ we immediately recover the overdamped Langevin dynamics (3).

2. Choosing $b=I$, and $\mathit{\gamma }\ne 0$ such that (12) holds gives rise to the nonreversible overdamped equation defined by (7). As it satisfies the conditions of Theorem 1, it is ergodic with respect to $\mathit{\pi }$. In particular choosing $\mathit{\gamma }(x)=J\mathrm{\nabla }V(x)$ for a constant skew-symmetric matrix J we obtain

   $$\begin{array}{c}\hfill \mathrm{d}{X}_t=-(I+J)\mathrm{\nabla }V(X_t)\mathrm{d}t+\sqrt{2}\mathrm{d}{W}_t,\end{array}$$ 15

   which has been studied in previous works.
3. Given a target density $\mathit{\pi }>0$ on ${\mathbb{R}}^d$, if we consider the augmented target density $\widehat{\mathit{\pi }}$ on ${\mathbb{R}}^{2d}$ given in (5), then choosing

   $$\begin{array}{c}\hfill \mathit{\gamma }((q,p))=\left(\begin{array}{c}{M}^{-1}p\\ -\mathrm{\nabla }V(q)\end{array}\right)\text{and}b=\left(\begin{array}{c}\mathbf{0}\\ \sqrt{\mathit{\Gamma }}\end{array}\right)\in {\mathbb{R}}^{2d\times d},\end{array}$$ 16

   where M and $\mathit{\Gamma }$ are positive definite symmetric matrices, the conditions of Theorem 1 are satisfied for the target density $\widehat{\mathit{\pi }}$. The resulting dynamics ${(q_t,p_t)}_{t\ge 0}$ is determined by the underdamped Langevin equation (4). It is straightforward to verify that the generator is hypoelliptic, [35, Sec 2.2.3.1], and thus ${(q_t,p_t)}_{t\ge 0}$ is ergodic.
4. More generally, consider the augmented target density $\widehat{\mathit{\pi }}$ on ${\mathbb{R}}^{2d}$ as above, and choose

   $$\begin{array}{c}\hfill \mathit{\gamma }((q,p))=\left(\begin{array}{c}{M}^{-1}p-\mathit{\mu }{J}_1\mathrm{\nabla }V(q)\\ -\mathrm{\nabla }V(q)-\mathit{\nu }{J}_2{M}^{-1}p\end{array}\right)\text{and}b=\left(\begin{array}{c}\mathbf{0}\\ \sqrt{\mathit{\Gamma }}\end{array}\right)\in {\mathbb{R}}^{2d\times d},\end{array}$$ 17

   where $\mathit{\mu }$ and $\mathit{\nu }$ are scalar constants and $J_1,J_2\in {\mathbb{R}}^{d\times d}$ are constant skew-symmetric matrices. With this choice we recover the perturbed Langevin dynamics (8). It is straightforward to check that (17) satisfies the invariance condition (12), and thus Theorem 1 guarantees that (8) is invariant with respect to $\widehat{\mathit{\pi }}$.
5. In a similar fashion, one can introduce an augmented target density on ${\mathbb{R}}^{(m+2)d}$, with

   $$\begin{array}{c}\hfill \widehat{\widehat{\mathit{\pi }}}(q,p,u_1,\dots ,u_m)\propto e^{-\left(\frac{{|p|}^2}{2}+\frac{|u_1{|}^2+\dots +{|u_m|}^2}{2}+V(q)\right)},\end{array}$$

   where $p,q,u_i\in {\mathbb{R}}^d$, for $i=1,\dots ,m$. Clearly ${\int }_{{\mathbb{R}}^d\times {\mathbb{R}}^{md}}\widehat{\widehat{\mathit{\pi }}}(q,p,u_1,\dots ,u_m)\mathrm{d}p\mathrm{d}{u}_1\dots \mathrm{d}{u}_m=\mathit{\pi }(q)$. We now define $\mathit{\gamma }:{\mathbb{R}}^{(m+2)d}\to {\mathbb{R}}^{(m+2)d}$ by

   $$\begin{array}{c}\hfill \mathit{\gamma }(q,p,u_1,\dots ,u_m)={\left(p-{\mathrm{\nabla }}_{q}V(q)+\sum _{j=1}^m{\mathit{\lambda }}_j{u}_j-{\mathit{\lambda }}_{1}p\cdots -{\mathit{\lambda }}_{m}p\right)}^T\end{array}$$

   and $b:{\mathbb{R}}^{(m+2)d}\to {\mathbb{R}}^{(m+2)d\times (m+2)d}$ by

   $$\begin{array}{c}\hfill b(q,p,u_1,\dots ,u_m)=\left(\begin{array}{cccccc}\mathbf{0}& \mathbf{0}& \mathbf{0}& \mathbf{0}& \dots & \mathbf{0}\\ \mathbf{0}& \mathbf{0}& \mathbf{0}& \mathbf{0}& \dots & \mathbf{0}\\ \mathbf{0}& \mathbf{0}& \sqrt{{\mathit{\alpha }}_1}{I}_{d\times d}& \mathbf{0}& \dots & \mathbf{0}\\ \mathbf{0}& \mathbf{0}& \mathbf{0}& \sqrt{{\mathit{\alpha }}_2}{I}_{d\times d}& \dots & \mathbf{0}\\ ⋮& ⋮& ⋮& ⋮& \ddots & ⋮\\ \mathbf{0}& \mathbf{0}& \mathbf{0}& \mathbf{0}& \dots & \sqrt{{\mathit{\alpha }}_m}{I}_{d\times d}\end{array}\right),\end{array}$$

   where ${\mathit{\lambda }}_i\in \mathbb{R}$ and ${\mathit{\alpha }}_i>0$, for $i=1,\dots ,m$. The resulting process (9) is given by

   $$\begin{array}{c}\hfill \begin{array}{cc}\hfill \mathrm{d}{q}_t& =p_t\mathrm{d}t,\mathrm{d}{p}_t=-{\mathrm{\nabla }}_{q}V(q_t)\mathrm{d}t+\sum _{j=1}^d{\mathit{\lambda }}_j{u}^j(t)\mathrm{d}t,\hfill \\ \hfill \mathrm{d}{u}_t^1& =-{\mathit{\lambda }}_1{p}_t\mathrm{d}t-{\mathit{\alpha }}_1{u}_t^1\mathrm{d}t+\sqrt{2{\mathit{\alpha }}_1}\mathrm{d}{W}_t^1,\hfill \\ \hfill ⋮& \\ \hfill \mathrm{d}{u}_t^m& =-{\mathit{\lambda }}_m{p}_t\mathrm{d}t-{\mathit{\alpha }}_m{u}_t^m\mathrm{d}t+\sqrt{2{\mathit{\alpha }}_m}\mathrm{d}{W}_t^m,\hfill \end{array}\end{array}$$ 18

   where ${(W_t^j)}_{t\ge 0,j=1,\dots ,m}$ are independent ${\mathbb{R}}^d$–valued Brownian motions. This process is ergodic with unique invariant distribution $\widehat{\widehat{\mathit{\pi }}}$, and under appropriate conditions on V, converges exponentially fast to equilibrium in relative entropy [42]. Equation (18) is a Markovian representation of a generalised Langevin equation of the form

   $$\begin{array}{c}\hfill \mathrm{d}{q}_t=p_t\mathrm{d}t,\mathrm{d}{p}_t=-{\mathrm{\nabla }}_{q}V(q_t)\mathrm{d}t-{\int }_0^{t}F(t-s)p_s\mathrm{d}s+N(t),\end{array}$$

   where N(t) is a mean-zero stationary Gaussian process with autocorrelation function F(t), i.e. $\mathbb{E}\left[N(t)\otimes N(s)\right]=F(t-s)I_{d\times d}$ and $F(t)={\sum }_{i=1}^m{\mathit{\lambda }}_i^2{e}^{-{\mathit{\alpha }}_i|t|}.$
6. Let $\tilde{\mathit{\pi }}(z)\propto exp(-\mathrm{\Phi }(z))$ be a positive density on ${\mathbb{R}}^N$ where $N>d$ such that $\mathit{\pi }(x)={\int }_{{\mathbb{R}}^{N-d}}\tilde{\mathit{\pi }}(x,z)\mathrm{d}z,$ where $(x,y)\in {\mathbb{R}}^d\times {\mathbb{R}}^{N-d}$. Then choosing $b=I_{D\times D}$ and $\mathit{\gamma }=0$ we obtain the dynamics

   $$\begin{array}{c}\hfill \mathrm{d}{X}_t=-{\mathrm{\nabla }}_x\mathrm{\Phi }(X_t,Y_t)\mathrm{d}t+\sqrt{2}\mathrm{d}{W}_t^1,\mathrm{d}{Y}_t=-{\mathrm{\nabla }}_y\mathrm{\Phi }(X_t,Y_t)\mathrm{d}t+\sqrt{2}\mathrm{d}{W}_t^2,\end{array}$$

   then ${(X_t,Y_t)}_{t\ge 0}$ is immediately ergodic with respect to $\tilde{\mathit{\pi }}$.

Comparison Criteria

For a fixed observable f, a natural measure of accuracy of the estimator ${\mathit{\pi }}_T(f)=t^{-1}{\int }_0^{t}f(X_s)\mathrm{d}s$ is the mean square error (MSE) defined by

$$\begin{array}{c}\hfill MSE(f,T):={\mathbb{E}}_x{\left|{\mathit{\pi }}_T(f)-\mathit{\pi }(f)\right|}^2,\end{array}$$ 19

where ${\mathbb{E}}_x$ denotes the expectation conditioned on the process ${(X_t)}_{t\ge 0}$ starting at x. It is instructive to introduce the decomposition $MSE(f,T)={\mathit{\mu }}^2(f,T)+{\mathit{\sigma }}^2(f,T)$, where

$$\begin{array}{cc}\hfill \mathit{\mu }(f,T)=& \left|{\mathbb{E}}_x[{\mathit{\pi }}_T(f)]-\mathit{\pi }(f)\right|\text{and}{\mathit{\sigma }}^2(f,T)={\mathbb{E}}_x{\left|{\mathit{\pi }}_T(f)-{\mathbb{E}}_x{\mathit{\pi }}_T(f)\right|}^2\hfill \\ \hfill =& \text{Var}[{\mathit{\pi }}_T(f)].\hfill \end{array}$$ 20

Here $\mathit{\mu }(f,T)$ measures the bias of the estimator ${\mathit{\pi }}_T(f)$ and ${\mathit{\sigma }}^2(f,T)$ measures the variance of fluctuations of ${\mathit{\pi }}_T(f)$ around the mean.

The speed of convergence to equilibrium of the process ${(X_t)}_{t\ge 0}$ will control both the bias term $\mathit{\mu }(f,T)$ and the variance ${\mathit{\sigma }}^2(f,T)$. To make this claim more precise, suppose that the semigroup ${(P_t)}_{t\ge 0}$ associated with ${(X_t)}_{t\ge 0}$ decays exponentially fast in $L^2(\mathit{\pi })$, i.e. there exist constants $\mathit{\lambda }>0$ and $C\ge 1$ such that

$$\begin{array}{c}\hfill \left||P_{t}g-\mathit{\pi }(g)\right|{{|}_{L^2(\mathit{\pi })}\le C{e}^{-\mathit{\lambda }t}\left||g-\mathit{\pi }(g)\right||}_{L^2(\mathit{\pi })},g\in L^2(\mathit{\pi }).\end{array}$$ 21

Remark 2

If (21) holds with $C=1$, this estimate is equivalent to $-\mathcal{L}$ having a spectral gap in $L^2(\mathit{\pi })$. Allowing for a constant $C>1$ is essential for our purposes though in order to treat nonreversible and degenerate diffusion processes by the theory of hypocoercivity as outlined in [54].

The following lemma characterises the decay of the bias $\mathit{\mu }(f,T)$ as $T\to \infty$ in terms of $\mathit{\lambda }$ and C. The proof can be found in [41].

Lemma 1

Let ${(X_t)}_{t\ge 0}$ be the unique, non-explosive solution of (9), such that $X_0\sim {\mathit{\pi }}_0\ll \mathit{\pi }$ and $\frac{d{\mathit{\pi }}_0}{d\mathit{\pi }}\in L^2(\mathit{\pi })$, where $\frac{d{\mathit{\pi }}_0}{d\mathit{\pi }}$ denotes the Radon-Nikodym derivative of ${\mathit{\pi }}_0$ with respect to $\mathit{\pi }$. Suppose that the process is ergodic with respect to $\mathit{\pi }$ such that the Markov semigroup ${(P_t)}_{t\ge 0}$ satisfies (21). Then for $f\in L^{\infty }(\mathit{\pi })$,

$$\begin{array}{c}\hfill \mathit{\mu }(f,T)\le \frac{C}{\mathit{\lambda }T}\left(1-e^{-\mathit{\lambda }T}\right){||f||}_{L^{\infty }}\text{Var}{}_{\mathit{\pi }}{\left[\frac{d{\mathit{\pi }}_0}{d\mathit{\pi }}\right]}^{\frac{1}{2}}.\end{array}$$

The study of the long time behaviour of the variance ${\mathit{\sigma }}^2(f,T)$ involves deriving a central limit theorem for the additive functional ${\int }_0^{t}f(X_t)-\mathit{\pi }(f)\mathrm{d}t$. As discussed in [13], we reduce this problem to proving well-posedness of the Poisson equation

$$\begin{array}{c}\hfill -\mathcal{L}\mathit{\chi }=f-\mathit{\pi }(f),\mathit{\pi }(\mathit{\chi })=0.\end{array}$$ 22

The only complications in this approach arise from the fact that the generator $\mathcal{L}$ need not be symmetric in $L^2(\mathit{\pi })$ nor uniformly elliptic. The following result summarises conditions for the well-posedness of the Poisson equation and it also provides with us with a formula for the asymptotic variance. Again, the proof can be found in [41].

Lemma 2

Let ${(X_t)}_{t\ge 0}$ be the unique, non-explosive solution of (9) with smooth drift and diffusion coefficients, such that the corresponding infinitesimal generator is hypoelliptic. Syppose that ${(X_t)}_{t\ge 0}$ is ergodic with respect to $\mathit{\pi }$ and moreover, ${(P_t)}_{t\ge 0}$ decays exponentially fast in $L^2(\mathit{\pi })$ as in (21). Then for all $f\in L^2(\mathit{\pi })$, there exists a unique mean zero solution $\mathit{\chi }$ to the Poisson equation (22). If $X_0\sim \mathit{\pi }$, then for all $f\in C^{\infty }({\mathbb{R}}^d)\cap L^2(\mathit{\pi })$

$$\begin{array}{c}\hfill \sqrt{T}\left({\mathit{\pi }}_T(f)-\mathit{\pi }(f)\right)\underset{T\to \infty }{\overset{d}{\to }}\mathcal{N}(0,2{\mathit{\sigma }}_f^2),\end{array}$$ 23

where ${\mathit{\sigma }}_f^2$ is the asymptotic variance defined by

$$\begin{array}{c}\hfill {\mathit{\sigma }}_f^2={〈\mathit{\chi },(-\mathcal{L})\mathit{\chi }〉}_{L^2(\mathit{\pi })}={〈\mathrm{\nabla }\mathit{\chi },\mathit{\Sigma }\mathrm{\nabla }\mathit{\chi }〉}_{L^2(\mathit{\pi })}.\end{array}$$ 24

Moreover, if $X_0\sim {\mathit{\pi }}_0$ where ${\mathit{\pi }}_0\ll \mathit{\pi }$ and $\frac{d{\mathit{\pi }}_0}{d\mathit{\pi }}\in L^2(\mathit{\pi })$ then (23) holds for all $f\in C^{\infty }({\mathbb{R}}^d)\cap L^{\infty }({\mathbb{R}}^d)$.1

Clearly, observables that only differ by a constant have the same asymptotic variance. In the sequel, we will hence restrict our attention to observables $f\in L^2(\mathit{\pi })$ satisfying $\mathit{\pi }(f)=0$, simplifying expressions (22) and (23). The corresponding subspace of $L^2(\mathit{\pi })$ will be denoted by $L_0^2(\mathit{\pi })$. If the exponential decay estimate (21) is satisfied, then Lemma 2 shows that $-\mathcal{L}$ is invertible on $L_0^2(\mathit{\pi })$, so we can express the asymptoptic variance as

$$\begin{array}{c}\hfill {\mathit{\sigma }}_f^2={⟨f,{(-\mathcal{L})}^{-1}f⟩}_{L^2(\mathit{\pi })},f\in L_0^2(\mathit{\pi }).\end{array}$$ 25

We note that the constants C and $\mathit{\lambda }$ appearing in the exponential decay estimate (21) also control the speed of convergence of ${\mathit{\sigma }}^2(f,T)$ to zero. Indeed, it is straightforward to show that if (21) is satisfied, then the solution $\mathit{\chi }$ of (22) satisfies

$$\begin{array}{c}\hfill {\mathit{\sigma }}_f^2={〈\mathit{\chi },f-\mathit{\pi }(f)〉}_{L^2(\mathit{\pi })}\le \frac{C}{\mathit{\lambda }}\left||f\right|{|}_{L^2(\mathit{\pi })}^2.\end{array}$$ 26

Lemmas 1 and 2 would suggest that choosing the coefficients $\mathit{\Sigma }$ and $\mathit{\gamma }$ to optimize the constants C and $\mathit{\lambda }$ in (26) would be an effective means of improving the performance of the estimator ${\mathit{\pi }}_T(f)$, especially since the improvement in performance would be uniform over an entire class of observables. When this is possible, this is indeed the case. However, as has been observed in [20, 21, 34], maximising the speed of convergence to equilibrium is a delicate task. As the leading order term in MSE(f, T), it is typically sufficient to focus specifically on the asymptotic variance ${\mathit{\sigma }}_f^2$ and study how the parameters of the SDE (9) can be chosen to minimise ${\mathit{\sigma }}_f^2$. This study was undertaken in [14] for processes of the form (7).

Perturbation of Underdamped Langevin Dynamics

The primary objective of this work is to compare the performances of the perturbed underdamped Langevin dynamics (8) and the unperturbed dynamics (4) according to the criteria outlined in Sect. 2.3 and to find suitable choices for the matrices $J_1$, $J_2$, M and $\mathit{\Gamma }$ that improve the performance of the sampler. We begin our investigations of (8) by establishing ergodicity and exponentially fast return to equilibrium, and by studying the overdamped limit of (8). As the latter turns out to be nonreversible and therefore in principle superior to the usual overdamped limit (3), e.g. [21], this calculation provides us with further motivation to study the proposed dynamics.

For the bulk of this work, we focus on the particular case when the target measure is Gaussian, i.e. when the potential is given by $V(q)=\frac{1}{2}{q}^{T}Sq$ with a symmetric and positive definite precision matrix S (i.e. the covariance matrix is given by $S^{-1}$). In this case, we advocate the following conditions for the choice of parameters:

$$\begin{array}{c}\hfill M=S,\mathit{\Gamma }=\mathit{\gamma }S,S{J}_{1}S=J_2,\mathit{\mu }=\mathit{\nu }.\end{array}$$ 27

Under the above choices (27), we show that the large perturbation limit ${lim}_{\mathit{\mu }\to \infty }{\mathit{\sigma }}_f^2$ exists and is finite and we provide an explicit expression for it (see Corollary 4). From this expression, we derive an algorithm for finding optimal choices for $J_1$ in the case of quadratic observables (see Algorithm 2).

If the friction coefficient is not too small ($\mathit{\gamma }>\sqrt{2}$), and under certain mild nondegeneracy conditions, we prove that adding a small perturbation will always decrease the asymptotic variance for observables of the form $f(q)=q·Kq+l·q+C$:

$$\begin{array}{c}\hfill {\left.\frac{\mathrm{d}}{\mathrm{d}\mathit{\mu }}{\mathit{\sigma }}_f^2\right|}_{\mathit{\mu }=0}=0\text{and}{\left.\frac{{\mathrm{d}}^2}{\mathrm{d}{\mathit{\mu }}^2}{\mathit{\sigma }}_f^2\right|}_{\mathit{\mu }=0}<0,\end{array}$$

see Theorem 3. In fact, we conjecture that this statement is true for arbitrary observables $f\in L^2(\mathit{\pi })$, but we have not been able to prove this. The dynamics (8) [used in conjunction with the conditions (27)] proves to be especially effective when the observable is antisymmetric (i.e. when it is invariant under the substitution $q\mapsto -q$) or when it has a significant antisymmetric part. In particular, in Proposition 3 we show that under certain conditions on the spectrum of $J_1$, for any antisymmetric observable $f\in L^2(\mathit{\pi })$ it holds that ${lim}_{\mathit{\mu }\to \infty }{\mathit{\sigma }}_f^2=0$.

Numerical experiments and analysis show that departing significantly from (27) in fact possibly decreases the performance of the sampler. This is in stark contrast to (7), where it is not possible to increase the asymptotic variance by any perturbation. For that reason, until now it seems practical to use (8) as a sampler only when a reasonable estimate of the global covariance of the target distribution is available. In the case of Bayesian inverse problems and diffusion bridge sampling, the target measure $\mathit{\pi }$ is given with respect to a Gaussian prior. We demonstrate the effectiveness of our approach in these applications, taking the prior Gaussian covariance as S in (27).

Remark 3

In [34, Rem. 3] another modification of (4) was suggested (albeit with the simplifications $\mathit{\Gamma }=\mathit{\gamma }·I$ and $M=I$):

$$\begin{array}{c}\hfill \mathrm{d}{q}_t=(1-J)M^{-1}{p}_t\mathrm{d}t,\mathrm{d}{p}_t=-(1+J)\mathrm{\nabla }V(q_t)\mathrm{d}t-\mathit{\Gamma }{M}^{-1}{p}_t\mathrm{d}t+\sqrt{2\mathit{\Gamma }}\mathrm{d}{W}_t,\end{array}$$ 28

J again denoting an antisymmetric matrix. However, under the change of variables $p\mapsto (1+J)\tilde{p}$ the above equations transform into

$$\begin{array}{c}\hfill \mathrm{d}{q}_t={\tilde{M}}^{-1}{p}_t\mathrm{d}t,\mathrm{d}{\tilde{p}}_t=-\mathrm{\nabla }V(q_t)\mathrm{d}t-\tilde{\mathit{\Gamma }}{\tilde{M}}^{-1}{\tilde{p}}_t\mathrm{d}t+\sqrt{2\tilde{\mathit{\Gamma }}}\mathrm{d}{\tilde{W}}_t,\end{array}$$ 29

where $\tilde{M}={(1+J)}^{-1}M{(1-J)}^{-1}$ and $\tilde{\mathit{\Gamma }}={(1+J)}^{-1}\mathit{\Gamma }{(1-J)}^{-1}$. Since any observable f depends only on q (the p-variables are merely auxiliary), the estimator ${\mathit{\pi }}_T(f)$ as well as its associated convergence characteristics (i.e. asymptotic variance and speed of convergence to equilibrium) are invariant under this transformation. Therefore, (28) reduces to the underdamped Langevin dynamics (4) and does not represent an independent approach to sampling. Suitable choices of M and $\mathit{\Gamma }$ will be discussed in Sect. 4.5.

Properties of Perturbed Underdamped Langevin Dynamics

In this section we study some of the properties of the perturbed underdamped dynamics (8). First, note that its generator is given by

$$\begin{array}{c}\hfill \mathcal{L}=\underset{{\mathcal{L}}_0}{\underbrace{\underset{{\mathcal{L}}_{ham}}{\underbrace{M^{-1}p·{\mathrm{\nabla }}_q-{\mathrm{\nabla }}_{q}V·{\mathrm{\nabla }}_p}}\underset{{\mathcal{L}}_{therm}}{\underbrace{-\mathit{\Gamma }{M}^{-1}p·{\mathrm{\nabla }}_p+\mathit{\Gamma }:D_p^2}}}}\underset{{\mathcal{L}}_{pert}}{\underbrace{-\mathit{\mu }{J}_1\mathrm{\nabla }V·{\mathrm{\nabla }}_q-\mathit{\nu }J{M}^{-1}p·{\mathrm{\nabla }}_p}},\end{array}$$ 30

decomposed into the perturbation ${\mathcal{L}}_{pert}$ and the unperturbed operator ${\mathcal{L}}_0$, which can be further split into the Hamiltonian part ${\mathcal{L}}_{ham}$ and the thermostat (Ornstein–Uhlenbeck) part ${\mathcal{L}}_{therm}$, see [35, 36, 46].

Lemma 3

The infinitesimal generator $\mathcal{L}$ (30) is hypoelliptic.

Proof

The proof consists of verifying the conditions of Hörmander’s Theorem for the generator (30) and can be found in [41]. $\square$

An immediate corollary of this result and of Theorem 1 is that the perturbed underdamped Langevin process (8) is ergodic with unique invariant distribution $\widehat{\mathit{\pi }}$ given by (5).

As explained in Sect. 2.3, the exponential decay estimate (21) is crucial for our approach, as in particular it guarantees the well-posedness of the Poisson equation (22). From now on, we will therefore make the following assumption on the potential V,  required to prove exponential decay in $L^2(\mathit{\pi })$:

Assumption 1

Assume that the Hessian of V is bounded and that the target measure $\mathit{\pi }(\mathrm{d}q)=\frac{1}{Z}{e}^{-V}\mathrm{d}q$ satisfies a Poincare inequality, i.e. there exists a constant $\mathit{\rho }>0$ such that

$$\begin{array}{c}\hfill {\int }_{{\mathbb{R}}^d}{\mathit{\varphi }}^2\mathrm{d}\mathit{\pi }\le \mathit{\rho }{\int }_{{\mathbb{R}}^d}{|\mathrm{\nabla }\mathit{\varphi }|}^2\mathrm{d}\mathit{\pi },\mathit{\varphi }\in L_0^2(\mathit{\pi })\cap H^1(\mathit{\pi }).\end{array}$$ 31

Sufficient conditions on the potential so that Poincaré’s inequality holds, e.g. the Bakry-Emery criterion, are presented in [7].

Theorem 2

Under Assumption 1 there exist constants $C\ge 1$ and $\mathit{\lambda }>0$ such that the semigroup ${(P_t)}_{t\ge 0}$ generated by $\mathcal{L}$ satisfies exponential decay in $L^2(\mathit{\pi })$ as in (21).

Proof

The proof uses the machinery of hypocoercivity developed in [54] and can be found in [41]. Using the framework of [15], we conjecture that the assumption on the boundedness of the Hessian of V can be substantially weakened and more quantitative decay estimates (in particular with respect to $\mathit{\mu }$ and $\mathit{\nu }$) can be obtained. This approach has recently been successfully applied to equilibrium and nonequilibirum Langevin dynamics, see [27, 53]. We leave this work track for future study. $\square$

The Overdamped Limit

In this section we develop a connection between the perturbed underdamped Langevin dynamics (8) and the nonreversible overdamped Langevin dynamics (7). The analysis is very similar to the one presented in [35, Sect. 2.2.2] and we will be brief. For convenience in this section we will perform the analysis on the d-dimensional torus ${\mathbb{T}}^d\cong {(\mathbb{R}/\mathbb{Z})}^d$, i.e. we will assume $q\in {\mathbb{T}}^d$. Consider the following scaling of (8):

$$\begin{array}{cc}\hfill \mathrm{d}{q}_t^{\mathit{ϵ}}=& \frac{1}{\mathit{ϵ}}{M}^{-1}{p}_t^{\mathit{ϵ}},\mathrm{d}t-\mathit{\mu }{J}_1{\mathrm{\nabla }}_{q}V(q_t)\mathrm{d}t,\hfill \end{array}$$ 32a

$$\begin{array}{cc}\hfill \mathrm{d}{p}_t^{\mathit{ϵ}}=& -\frac{1}{\mathit{ϵ}}{\mathrm{\nabla }}_{q}V(q_t^{\mathit{ϵ}})\mathrm{d}t-\frac{1}{{\mathit{ϵ}}^2}\mathit{\nu }{J}_2{M}^{-1}{p}_t^{\mathit{ϵ}}\mathrm{d}t-\frac{1}{{\mathit{ϵ}}^2}\mathit{\Gamma }{M}^{-1}{p}_t^{\mathit{ϵ}}\mathrm{d}t+\frac{1}{\mathit{ϵ}}\sqrt{2\mathit{\Gamma }}\mathrm{d}{W}_t,\hfill \end{array}$$ 32b

valid for the small mass/small momentum regime $M\to {\mathit{ϵ}}^{2}M,p_t\to \mathit{ϵ}{p}_t.$ Equivalently, those modifications can be obtained from subsituting $\mathit{\Gamma }\to {\mathit{ϵ}}^{-1}\mathit{\Gamma }$ and $t\mapsto {\mathit{ϵ}}^{-1}t$, and so in the limit as $\mathit{ϵ}\to 0$ the dynamics (32) describes the limit of large friction with rescaled time. It turns out that as $\mathit{ϵ}\to 0$, the dynamics (32) converges to the limiting SDE

$$\begin{array}{c}\hfill \mathrm{d}{q}_t=-{(\mathit{\nu }{J}_2+\mathit{\Gamma })}^{-1}{\mathrm{\nabla }}_{q}V(q_t)\mathrm{d}t-\mathit{\mu }{J}_1{\mathrm{\nabla }}_{q}V(q_t)\mathrm{d}t+{(\mathit{\nu }{J}_2+\mathit{\Gamma })}^{-1}\sqrt{2\mathit{\Gamma }}\mathrm{d}{W}_t.\end{array}$$ 33

The following proposition makes this statement precise.

Proposition 1

Denote by $(q_t^{\mathit{ϵ}},p_t^{\mathit{ϵ}})$ the solution to (32) with (deterministic) initial conditions $(q_0^{\mathit{ϵ}},p_0^{\mathit{ϵ}})=(q_{init},p_{init})$ and by $q_t^0$ the solution to (33) with initial condition $q_0^0=q_{init}.$ For any $T>0$, ${(q_t^{\mathit{ϵ}})}_{0\le t\le T}$ converges to ${(q_t^0)}_{0\le t\le T}$ in $L^2(\mathrm{\Omega },C([0,T]),{\mathbb{T}}^d)$ as $\mathit{ϵ}\to 0$, i.e. ${lim}_{\mathit{ϵ}\to 0}\mathbb{E}({sup}_{0\le t\le T}{|q_t^{\mathit{ϵ}}-q_t^0|}^2)=0.$

Proof

The proof follows standard arguments (see for instance [46]) and can be found in [41]. By a more refined analysis, it is also possible to get information on the rate of convergence; see, e.g. [48, 49]. $\square$

Remark 4

The overdamped limit (33) respects the invariant distribution, in the sense that it is ergodic with respect to $\mathit{\pi }(\mathrm{d}q)=\frac{1}{Z}{e}^{-V}\mathrm{d}q$.

The limiting SDE (33) is nonreversible due to the term $-\mathit{\mu }{J}_1{\mathrm{\nabla }}_{q}V(q_t)\mathrm{d}t$ and also because the matrix ${(\mathit{\nu }{J}_2+\mathit{\Gamma })}^{-1}$ is in general neither symmetric nor antisymmetric. This result, together with the fact that nonreversible perturbations of overdamped Langevin dynamics of the form (7) are by now well-known to have improved performance properties, motivates further investigation of the dynamics (8).

Sampling from a Gaussian Distribution

In this section we study in detail the performance of the Langevin sampler (8) for Gaussian target densities, first considering the case of unit covariance. In particular, we study the optimal choice for the parameters in the sampler, the exponential decay rate and the asymptotic variance. We then extend our results to Gaussian target densities with arbitrary covariance matrices.

Unit Covariance: Small Perturbations

In our study of the dynamics given by (8) we first consider the simple case when $V(q)=\frac{1}{2}{|q|}^2$, i.e. the task of sampling from a Gaussian measure with unit covariance. We will assume $M=I$, $\mathit{\Gamma }=\mathit{\gamma }I$ and $J_1=J_2=:J$ (so that the $q-$ and $p-$dynamics are perturbed in the same way, albeit posssibly with different strengths $\mathit{\mu }$ and $\mathit{\nu }$). Our first result concerns the asymptotic variance for linear and quadratic observables for small perturbations of equal strength ($\mathit{\mu }=\mathit{\nu }$). For sufficiently strong damping $(\mathit{\gamma }>\sqrt{2}$) always leads to an improvement in asymptotic variance under the nondegeneracy conditions $[J,K]\ne 0$ and $l\notin kerJ$:

Theorem 3

Consider the dynamics

$$\begin{array}{cc}\hfill \mathrm{d}{q}_t=& p_t\mathrm{d}t-\mathit{\mu }J{q}_t\mathrm{d}t,\hfill \\ \hfill \mathrm{d}{p}_t=& -q_t\mathrm{d}t-\mathit{\mu }J{p}_t\mathrm{d}t-\mathit{\gamma }{p}_t\mathrm{d}t+\sqrt{2\mathit{\gamma }}\mathrm{d}{W}_t,\hfill \end{array}$$ 34

with $\mathit{\gamma }>\sqrt{2}$ and an observable of the form $f(q)=q·Kq+l·q+C$, where $K\in {\mathbb{R}}_{sym}^{d\times d}$, $l\in {\mathbb{R}}^d$ and $C\in \mathbb{R}$. If at least one of the conditions $[J,K]\ne 0$ and $l\notin kerJ$ is satisfied, then the asymptotic variance of the unperturbed sampler is at a local maximum independently of K and J (and $\mathit{\gamma }$, as long as $\mathit{\gamma }>\sqrt{2}$), i.e.

$$\begin{array}{c}\hfill {\left.{\mathit{\partial }}_{\mathit{\mu }}{\mathit{\sigma }}_f^2\right|}_{\mathit{\mu }=0}=0\text{and}{\left.{\mathit{\partial }}_{\mathit{\mu }}^2{\mathit{\sigma }}_f^2\right|}_{\mathit{\mu }=0}<0.\end{array}$$

Proof

The dynamics (34) are of Ornstein–Uhlenbeck type, i.e. we can write

$$\begin{array}{c}\hfill \mathrm{d}{X}_t=-B{X}_t\mathrm{d}t+\sqrt{2Q}\mathrm{d}{\overline{W}}_t,B=\left(\begin{array}{cc}\mathit{\mu }J& -I\\ I& \mathit{\gamma }I+\mathit{\mu }J\end{array}\right),Q=\left(\begin{array}{cc}\mathbf{0}& \mathbf{0}\\ \mathbf{0}& \mathit{\gamma }I\end{array}\right)\end{array}$$ 35

with $X={(q,p)}^T$, and ${({\overline{W}}_t)}_{t\ge 0}$ denoting a standard Wiener process on ${\mathbb{R}}^{2d}$. The generator of (35) is then given by

$$\begin{array}{c}\hfill \mathcal{L}=-Bx·\mathrm{\nabla }+{\mathrm{\nabla }}^{T}Q\mathrm{\nabla }.\end{array}$$ 36

According to Lemma 2, the asymptotic variance can be expressed as

$$\begin{array}{c}\hfill {\mathit{\sigma }}_f^2={⟨\mathit{\chi },f⟩}_{L^2(\widehat{\mathit{\pi }})},\text{where}\mathit{\chi }\text{is}\text{the}\text{solution}\text{to}-\mathcal{L}\mathit{\chi }=f,\widehat{\mathit{\pi }}(\mathit{\chi })=0.\end{array}$$ 37

By calculations similar to those in [14, Sect. 4], $\mathit{\chi }$ is given by $\mathit{\chi }(x)=x·Cx+D·x-\mathrm{Tr}C$, where

$$\begin{array}{c}\hfill BC+C{B}^T=\overline{K},B^{T}D=\overline{l},\end{array}$$ 38

using the notations

$$\begin{array}{c}\hfill \overline{K}=\left(\begin{array}{cc}K& \mathbf{0}\\ \mathbf{0}& \mathbf{0}\end{array}\right)\in {\mathbb{R}}^{2d\times 2d}\text{and}\overline{l}=\left(\begin{array}{c}l\\ \mathbf{0}\end{array}\right)\in {\mathbb{R}}^{2d}.\end{array}$$ 39

The asymptotic variance is then given by

$$\begin{array}{c}\hfill {\mathit{\sigma }}_f^2=2\mathrm{Tr}(C\overline{K})+D·\overline{l}.\end{array}$$ 40

Taking derivatives of 38 and solving the ensuing matrix equations, it is possible to obtain explicit expressions for ${\mathit{\partial }}_{\mathit{\mu }}{C|}_{\mathit{\mu }=0}$, ${\mathit{\partial }}_{\mathit{\mu }}^2{C|}_{\mathit{\mu }=0}$, ${\mathit{\partial }}_{\mathit{\mu }}{D|}_{\mathit{\mu }=0}$ and ${\mathit{\partial }}_{\mathit{\mu }}^2{C|}_{\mathit{\mu }=0}$ as detailed in [41]. We obtain

$$\begin{array}{cc}\hfill {\left.{\mathit{\partial }}_{\mathit{\mu }}{\mathit{\sigma }}_f^2\right|}_{\mathit{\mu }=0}=& 0\text{and}{\left.{\mathit{\partial }}_{\mathit{\mu }}^2{\mathit{\sigma }}_f^2\right|}_{\mathit{\mu }=0}=(-2{\mathit{\gamma }}^3+4\mathit{\gamma }){|Jl|}^2\hfill \\ \hfill & +\left(\mathit{\gamma }-\frac{4}{{\mathit{\gamma }}^3}-{\mathit{\gamma }}^3-\frac{1}{\mathit{\gamma }}\right)·\left(\mathrm{Tr}(JKJK)-\mathrm{Tr}(J^2{K}^2)\right).\hfill \end{array}$$

Notice that $\mathrm{Tr}(JKJK)-\mathrm{Tr}(J^2{K}^2)=\frac{1}{2}\mathrm{Tr}({[J,K]}^2)$ and that [J, K] is symmetric. It follows that $\mathrm{Tr}(JKJK)-\mathrm{Tr}(J^2{K}^2)\ge 0$ with equality if and only if $[J,K]=0$. Together with $-2{\mathit{\gamma }}^3+4\mathit{\gamma }<0$ for $\mathit{\gamma }>\sqrt{2}$ and $\mathit{\gamma }-\frac{4}{{\mathit{\gamma }}^3}-{\mathit{\gamma }}^3-\frac{1}{\mathit{\gamma }}<0$ for $\mathit{\gamma }>0$, the claim follows. $\square$

Remark 5

As we will see in Sect. 4.3, Example 1, if $[J,K]=0$ and $l\in kerJ$, the asymptotic variance is constant as a function of $\mathit{\mu }$, i.e. the perturbation has no effect.

Numerical examples show that the conditions $\mathit{\gamma }>\sqrt{2}$ and $\mathit{\mu }=\mathit{\nu }$ are indeed necessary for the conclusions of Theorem 3 to hold (an explanation in terms of the spectrum of the generator will be provided in Sect. 4.2). In particular, an unfortunate choice of the perturbations will actually increase the asymptotic variance of the dynamics.

Let us illustrate this by plotting the asymptotic variance as a function of the perturbation strength $\mathit{\mu }$ (see Fig. 1), making the choices $d=2$, $l={(1,1)}^T$,

$$\begin{array}{c}\hfill K=\left(\begin{array}{cc}2& 0\\ 0& 1\end{array}\right)\text{and}J=\left(\begin{array}{cc}0& 1\\ -1& 0\end{array}\right).\end{array}$$ 41

The asymptotic variance has been computed according to (38) and (40). Going beyond the results of this section, the graphs give an impression of the behaviour of the asymptotic variace for large values of $\mathit{\mu }$, discussed further in Sect. 4.3.

Fig. 1.

Asymptotic variance for linear and quadratic observables, depending on relative perturbation and friction strengths. a Equal perturbations: $\mathit{\mu }=\mathit{\nu }$. b Approximately equal perturbations: $\mathit{\mu }=0.9\mathit{\nu }$. c Opposing perturbations: $\mathit{\mu }=-\mathit{\nu }$. d Equal perturbations: $\mathit{\mu }=\mathit{\nu }$ (sufficiently large friction $\mathit{\gamma }$). e Equal perturbations: $\mathit{\mu }=\mathit{\nu }$ (small friction $\mathit{\gamma }$)

Figure 1a, b, c show the asymptotic variance associated with the quadratic observable $f(q)=q·Kq$. In accordance with Theorem 3, the asymptotic variance is at a local maximum at zero perturbation in the case $\mathit{\mu }=\mathit{\nu }$ (see Fig. 1a). For increasing perturbation strength, the graph shows monotone decay for $\mathit{\mu }\to \infty$ (this limiting behaviour will be explored analytically in Sect. 4.3). If the condition $\mathit{\mu }=\mathit{\nu }$ is only approximately satisfied (Fig. 1b), our numerical examples still exhibits decaying asymptotic variance in the neighbourhood of the critical point. In this case, however, the asymptotic variance diverges for growing values of the perturbation $\mathit{\mu }$. If the perturbations are opposed ($\mathit{\mu }=-\mathit{\nu }$), it is possible for certain observables that the unperturbed dynamics represents a global minimum. Such a case is observed in Fig. 1c. In Fig. 1d, e the observable $f(q)=l·q$ is considered. If the damping is sufficiently strong ($\mathit{\gamma }>\sqrt{2}$), the unperturbed dynamics is at a local maximum of the asymptotic variance (Fig. 1d). Furthermore, the asymptotic variance approaches zero as $\mathit{\mu }\to \infty$ (for a theoretical explanation see again Sect. 4.3). The graph in Fig. 1e shows that the assumption of $\mathit{\gamma }$ not being too small cannot be dropped from Theorem 3. Even in this case though the example shows decay of the asymptotic variance for large values of $\mathit{\mu }$.

Exponential Decay Rate

Let us denote by ${\mathit{\lambda }}^{\ast }$ the optimal exponential decay rate in (21), i.e.

$$\begin{array}{c}\hfill {\mathit{\lambda }}^{\ast }=sup\{\mathit{\lambda }>0|\text{There exists}C\ge 1\text{such that}(21)\text{holds}\}.\end{array}$$ 42

Note that ${\mathit{\lambda }}^{\ast }$ is well-defined and positive by Theorem 2. We also define the spectral bound of the generator $\mathcal{L}$ by

$$\begin{array}{c}\hfill s(\mathcal{L})=inf(\text{Re}\mathit{\sigma }(-\mathcal{L})\backslash \{0\}).\end{array}$$ 43

In [38] it is proven that the Ornstein–Uhlenbeck semigroup ${(P_t)}_{t\ge 0}$ considered in this section is differentiable (see Proposition 2.1). In this case (see Corollary 3.12 of [17]), it is known that the exponential decay rate and the spectral bound coincide, i.e. ${\mathit{\lambda }}^{\ast }=s(\mathcal{L})$, whereas in general only ${\mathit{\lambda }}^{\ast }\le s(\mathcal{L})$ holds. In this section we will therefore analyse the spectral properties of the generator $\mathcal{L}$ in the Gaussian case. In particular, this leads to some intuition of why choosing equal perturbations ($\mathit{\mu }=\mathit{\nu }$) is crucial for the performance of the sampler.

In [38] (see also [43]), it was proven that the spectrum of $\mathcal{L}$ as in (36) in $L^2(\widehat{\mathit{\pi }})$ is given by

$$\begin{array}{c}\hfill \mathit{\sigma }(\mathcal{L})=\left\{-\sum _{j=1}^r{n}_j{\mathit{\lambda }}_j:n_j\in \mathbb{N},{\mathit{\lambda }}_j\in \mathit{\sigma }(B)\right\}.\end{array}$$ 44

Note that $\mathit{\sigma }(\mathcal{L})$ only depends on the drift matrix B. In the case where $\mathit{\mu }=\mathit{\nu }$, the spectrum of B can be computed explicitly.

Lemma 4

Assume $\mathit{\mu }=\mathit{\nu }$. Then the spectrum of B is given by2

$$\begin{array}{c}\hfill \mathit{\sigma }(B)=\left\{\mathit{\mu }\mathit{\lambda }+\sqrt{(\frac{\mathit{\gamma }}{2}{)}^2-1}+\frac{\mathit{\gamma }}{2}|\mathit{\lambda }\in \mathit{\sigma }(J)\right\}\cup \left\{\mathit{\mu }\mathit{\lambda }-\sqrt{(\frac{\mathit{\gamma }}{2}{)}^2-1}+\frac{\mathit{\gamma }}{2}|\mathit{\lambda }\in \mathit{\sigma }(J)\right\}.\end{array}$$ 45

Proof

We will compute $\mathit{\sigma }(B-\frac{\mathit{\gamma }}{2}I)$ and then use the identity $\mathit{\sigma }(B)=\left\{\mathit{\lambda }+\frac{\mathit{\gamma }}{2}|\mathit{\lambda }\in \mathit{\sigma }\left(B-\frac{\mathit{\gamma }}{2}I\right)\right\}.$ We have

$$\begin{array}{cc}\hfill det\left(B-\frac{\mathit{\gamma }}{2}I-\mathit{\lambda }I\right)& =det\left(\left(\mathit{\mu }J-\frac{\mathit{\gamma }}{2}I-\mathit{\lambda }I\right)\left(\mathit{\mu }J+\frac{\mathit{\gamma }}{2}I-\mathit{\lambda }I\right)+I\right)\hfill \\ \hfill & =det\left(\mathit{\mu }J-\mathit{\lambda }I+\sqrt{{\left(\frac{\mathit{\gamma }}{2}\right)}^2-1}I\right)·det\left(\mathit{\mu }J-\mathit{\lambda }I-\sqrt{{\left(\frac{\mathit{\gamma }}{2}\right)}^2-1}I\right),\hfill \end{array}$$

where I is understood to denote the identity matrix of appropriate dimension. The above quantity is zero if and only if

$$\begin{array}{c}\hfill \mathit{\lambda }-\sqrt{{\left(\frac{\mathit{\gamma }}{2}\right)}^2-1}\in \mathit{\sigma }(\mathit{\mu }J)\text{or}\mathit{\lambda }+\sqrt{{\left(\frac{\mathit{\gamma }}{2}\right)}^2-1}\in \mathit{\sigma }(\mathit{\mu }J).\end{array}$$

Together with (4.2), the claim follows. $\square$

Using formula (45), in Fig. 2a we show a sketch of the spectrum $\mathit{\sigma }(-\mathcal{L}$) for the case of equal perturbations ($\mathit{\mu }=\mathit{\nu })$ with the convenient choices $n=1$ and $\mathit{\gamma }=2.$ Of course, the eigenvalue at 0 is associated to the invariant measure since ${\mathcal{L}}^{†}\widehat{\mathit{\pi }}=0$. The arrows indicate the movement of the eigenvalues as the perturbation $\mathit{\mu }$ increases in accordance with Lemma 4. Clearly, the spectral bound of $\mathcal{L}$ is not affected by the perturbation. Note that the eigenvalues on the real axis stay invariant under the perturbation. The subspace of $L_0^2(\widehat{\mathit{\pi }})$ associated to those will turn out to be crucial for the characterisation of the limiting asymptotic variance as $\mathit{\mu }\to \infty$ (see Remark 10).

Fig. 2.

Effects of the perturbation on the spectra of $-\mathcal{L}$ and B. a $\mathit{\sigma }(-\mathcal{L})$ in the case $\mathit{\mu }=\mathit{\nu }$. The arrows indicate the movement of the spectrum as the perturbation strength $\mathit{\mu }$ increases. b $\mathit{\sigma }(B)$ in the case $J_1=0$, i.e. the dynamics is only perturbed by $-\mathit{\nu }{J}_{2}p\mathrm{d}t$. The arrows indicate the movement of the eigenvalues as $\mathit{\nu }$ increases

To illustrate the suboptimal properties of the perturbed dynamics when the perturbations are not equal, we plot the spectrum of the drift matrix $\mathit{\sigma }(B)$ in the case when the dynamics is only perturbed by the term $\mathit{\nu }{J}_{2}p\mathrm{d}t$ (i.e. $\mathit{\mu }=0$) for $n=2$, $\mathit{\gamma }=2$ and

$$\begin{array}{c}\hfill J_2=\left(\begin{array}{cc}0& -1\\ 1& 0\end{array}\right),\end{array}$$ 46

(see Fig. 2b). Note that the full spectrum $\mathit{\sigma }(-\mathcal{L})$ can be inferred from (44). For $\mathit{\nu }=0$ we have that the spectrum $\mathit{\sigma }(B)$ only consists of the (degenerate) eigenvalue 1. For increasing $\mathit{\nu }$, the figure shows that the degenerate eigenvalue splits up into four eigenvalues, two of which get closer to the imaginary axis as $\mathit{\nu }$ increases, leading to a smaller spectral bound and therefore to a decrease in the speed of convergence to equilibrium. Figure 2a, b give an intuitive explanation of why the fine-tuning of the perturbation strengths is crucial.

We close this subsection by providing autocorrelation plots (see Fig. 3) for the linear observable considered in Fig. 1d (with a friction coefficient of $\mathit{\gamma }=2.5$). It is well-known that the asymptotic variance is given by the integrated autocorrelation function (see e.g. Proposition IV 1.3 in [3]),

$$\begin{array}{c}\hfill {\mathit{\sigma }}_f^2={\int }_0^{\infty }{\mathbb{E}}_{\mathit{\pi }}\left[\left(f(q_0)-\mathit{\pi }(f)\right)\left(f(q_t)-\mathit{\pi }(f)\right)\right]\mathrm{d}t.\end{array}$$ 47

Comparing Fig. 3a, b yields additional insight into the mechanics of the variance reduction: the increase of the imaginary part of the eigenvalues of $\mathcal{L}$ (as indicated in Fig. 2a) leads to oscillations in the autocorrelation function and therefore to cancellations in (47). A similar effect has already been observed in [50] for the nonreversible overdamped Langevin dynamics (15).

Fig. 3.

Autocorrelation plots for the perturbed and unperturbed dynamics. a Unperturbed Langevin dynamics. b Perturbed Langevin dynamics

Unit Covariance: Large Perturbations

In the previous subsection we observed that for the particular perturbation $J_1=J_2$ and $\mathit{\mu }=\mathit{\nu }$ [see equation (34)] the perturbed Langevin dynamics demonstrated an improvement in performance for $\mathit{\mu }$ in a neighbourhood of 0, when the observable is linear or quadratic. Recall that this dynamics is ergodic with respect to a standard Gaussian measure $\widehat{\mathit{\pi }}$ on ${\mathbb{R}}^{2d}$ with marginal $\mathit{\pi }$ with respect to the q-variable. As before, we shall consider only observables that do not depend on p. Moreover, we assume without loss of generality that $\mathit{\pi }(f)=0$. For such observables we will write $f\in L_0^2(\mathit{\pi })$ and consider the canonical embedding $L_0^2(\mathit{\pi })\subset L^2(\widehat{\mathit{\pi }})$. We emphasize that $L_0^2(\mathit{\pi })$ consists of functions that only depend on q, whereas functions in $L^2(\widehat{\mathit{\pi }})$ may depend on both q and p.

In this subsection will analyse the asymptotic variance for large values of $\mathit{\mu }$. The infinitesimal generator of (34) can be written as

$$\begin{array}{c}\hfill \mathcal{L}=\underset{{\mathcal{L}}_0}{\underbrace{p·{\mathrm{\nabla }}_q-q·{\mathrm{\nabla }}_p+\mathit{\gamma }(-p·{\mathrm{\nabla }}_p+{\mathit{\Delta }}_p)}}+\mathit{\mu }\underset{\mathcal{A}}{\underbrace{(-Jq·{\mathrm{\nabla }}_q-Jp·{\mathrm{\nabla }}_p)}}=:{\mathcal{L}}_0+\mathit{\mu }\mathcal{A},\end{array}$$ 48

where we have introduced the notation ${\mathcal{L}}_{pert}=\mathit{\mu }\mathcal{A}$. In the sequel, the adjoint of an operator $\mathcal{B}$ in $L^2(\widehat{\mathit{\pi }})$ will be denoted by ${\mathcal{B}}^{\ast }$. In the rest of this section we will make repeated use of the Hermite polynomials

$$\begin{array}{c}\hfill g_{\mathit{\alpha }}(x)={(-1)}^{|\mathit{\alpha }|}{e}^{\frac{{|x|}^2}{2}}{\mathrm{\nabla }}^{\mathit{\alpha }}{e}^{-\frac{{|x|}^2}{2}},\mathit{\alpha }\in {\mathbb{N}}^{2d},\end{array}$$ 49

invoking the notation $x=(q,p)\in {\mathbb{R}}^{2d}$. For $m\in {\mathbb{N}}_0$ define the Hilbert spaces

$$\begin{array}{c}\hfill H_m=\mathrm{span}\{g_{\mathit{\alpha }}{:|\mathit{\alpha }|=m\},⟨f,g⟩}_m:={⟨f,g⟩}_{L^2(\widehat{\mathit{\pi }})},f,g\in H_m.\end{array}$$

The following result (Theorem 4) holds for operators of the form (36) providing an orthogonal decomposition of $L^2(\widehat{\mathit{\pi }})$ into invariant subspaces. The drift and diffusion matrices B and Q are assumed to be such that $\mathcal{L}$ is the generator of an ergodic stochastic process (see [2, Definition 2.1] for precise conditions).

Theorem 4

[2, Sect. 5]. The following holds:

1. The space $L^2(\widehat{\mathit{\pi }})$ has a decomposition into mutually orthogonal subspaces:

   $$\begin{array}{c}\hfill L^2(\widehat{\mathit{\pi }})=\underset{m\in {\mathbb{N}}_0}{⨁}{H}_m.\end{array}$$

2. For all $m\in {\mathbb{N}}_0$, $H_m$ is invariant under $\mathcal{L}$ as well as under the semigroup ${(e^{t\mathcal{L}})}_{t\ge 0}$.

3. The spectrum of $\mathcal{L}$ has the following decomposition:

   $$\begin{array}{c}\hfill \mathit{\sigma }(\mathcal{L})=\bigcup _{m\in {\mathbb{N}}_0}\mathit{\sigma }(\mathcal{L}{|}_{H_m}),\mathit{\sigma }(\mathcal{L}{|}_{H_m})=\left\{\sum _{j=1}^{2d}{\mathit{\alpha }}_j{\mathit{\lambda }}_j:|\mathit{\alpha }|=m,{\mathit{\lambda }}_j\in \mathit{\sigma }(B)\right\}.\end{array}$$

Remark 6

Note that by the ergodicity of the dynamics, $ker\mathcal{L}$ consists of constant functions and so $ker\mathcal{L}=H_0$. Therefore, $L_0^2(\widehat{\mathit{\pi }})$ has the decomposition

$$\begin{array}{c}\hfill L_0^2(\widehat{\mathit{\pi }})=L^2(\widehat{\mathit{\pi }})/ker\mathcal{L}=\underset{m\ge 1}{⨁}{H}_m.\end{array}$$

Our first main result of this section is an expression for the asymptotic variance in terms of the unperturbed operator ${\mathcal{L}}_0$ and the perturbation $\mathcal{A}$:

Proposition 2

Let $f\in L_0^2(\mathit{\pi })$ (so in particular $f=f(q)$). Then the associated asymptotic variance is given by

$$\begin{array}{c}\hfill {\mathit{\sigma }}_f^2={⟨f,-{\mathcal{L}}_0{({\mathcal{L}}_0^2+{\mathit{\mu }}^2{\mathcal{A}}^{\ast }\mathcal{A})}^{-1}f⟩}_{L^2(\widehat{\mathit{\pi }})}.\end{array}$$ 50

Remark 7

The proof of the preceding Proposition will show that ${\mathcal{L}}_0^2+{\mathit{\mu }}^2{\mathcal{A}}^{\ast }\mathcal{A}$ is invertible on $L_0^2(\widehat{\mathit{\pi }})$ and that ${({\mathcal{L}}_0^2+{\mathit{\mu }}^2{\mathcal{A}}^{\ast }\mathcal{A})}^{-1}f\in \mathcal{D}({\mathcal{L}}_0)$ for all $f\in L_0^2(\widehat{\mathit{\pi }})$.

To prove Proposition 2 we will make use of the generator with reversed perturbation

$$\begin{array}{c}\hfill {\mathcal{L}}_{-}={\mathcal{L}}_0-\mathit{\mu }\mathcal{A}\end{array}$$

and the momentum flip operator

$$\begin{array}{c}\hfill P:L_0^2(\widehat{\mathit{\pi }})\to L_0^2(\widehat{\mathit{\pi }}),\mathit{\varphi }(q,p)\mapsto \mathit{\varphi }(q,-p).\end{array}$$ 51

Clearly, $P^2=I$ and $P^{\ast }=P$. Further properties of ${\mathcal{L}}_0$, $\mathcal{A}$ and the auxiliary operators ${\mathcal{L}}_{-}$ and P are gathered in the following lemma:

Lemma 5

For all $\mathit{\varphi },\mathit{\psi }\in C^{\infty }({\mathbb{R}}^{2d})\cap L^2(\widehat{\mathit{\pi }})$ the following holds:

1. The generator ${\mathcal{L}}_0$ is symmetric in $L^2(\widehat{\mathit{\pi }})$ with respect to P:

   $$\begin{array}{c}\hfill {⟨\mathit{\varphi },P{\mathcal{L}}_{0}P\mathit{\psi }⟩}_{L^2(\widehat{\mathit{\pi }})}={⟨{\mathcal{L}}_0\mathit{\varphi },\mathit{\psi }⟩}_{L^2(\widehat{\mathit{\pi }})}.\end{array}$$
2. The perturbation $\mathcal{A}$ is skewadjoint in $L^2(\widehat{\mathit{\pi }})$:

   $$\begin{array}{c}\hfill {\mathcal{A}}^{\ast }=-\mathcal{A}.\end{array}$$
3. The operators ${\mathcal{L}}_0$ and $\mathcal{A}$ commute:

   $$\begin{array}{c}\hfill [{\mathcal{L}}_0,\mathcal{A}]\mathit{\varphi }=0.\end{array}$$
4. The perturbation $\mathcal{A}$ satisfies

   $$\begin{array}{c}\hfill P\mathcal{A}P\mathit{\varphi }=\mathcal{A}\mathit{\varphi }.\end{array}$$
5. $\mathcal{L}$ and ${\mathcal{L}}_{-}$ commute,

   $$\begin{array}{c}\hfill [\mathcal{L},{\mathcal{L}}_{-}]\mathit{\varphi }=0,\end{array}$$
   and the following relation holds:

   $$\begin{array}{c}\hfill {⟨\mathit{\varphi },P\mathcal{L}P\mathit{\psi }⟩}_{L^2(\widehat{\mathit{\pi }})}={⟨{\mathcal{L}}_{-}\mathit{\varphi },\mathit{\psi }⟩}_{L^2(\widehat{\mathit{\pi }})}.\end{array}$$ 52

6. The operators $\mathcal{L}$, ${\mathcal{L}}_0$, ${\mathcal{L}}_{-}$, $\mathcal{A}$ and P leave the Hermite spaces $H_m$ invariant.

Remark 8

The claim (c) in the above lemma is crucial for our approach, which itself rests heavily on the fact that the $q-$ and $p-$perturbations match ($J_1=J_2$).

Proof of Lemma 5

The statement (a) is well-known and its proof can be found in [35, Sect. 2.2.3.1] for instance. The claim (b) follows by noting that the flow vector field $b(q,p)=(-Jq,-Jp)$ associated to $\mathcal{A}$ is divergence-free with respect to $\widehat{\mathit{\pi }}$, i.e. $\mathrm{\nabla }·(\widehat{\mathit{\pi }}b)=0$. Therefore, $\mathcal{A}$ is the generator of a strongly continuous unitary semigroup on $L^2(\widehat{\mathit{\pi }})$ and hence skewadjoint by Stone’s Theorem. The claims (c), (d) and (e) follow by direct computations which can be found in [41]. To prove (f) first notice that $\mathcal{L}$, ${\mathcal{L}}_0$ and ${\mathcal{L}}_{-}$ are of the form (36) and therefore leave the spaces $H_m$ invariant by Theorem 4. It follows immediately that also $\mathcal{A}$ leaves those spaces invariant. The fact that P leaves the spaces $H_m$ invariant follows directly by inspection of (49) and (51). $\square$

Now we proceed with the proof of Proposition 2:

Proof of Proposition 2

Since the potential V is quadratic, Assumption 1 clearly holds and thus Lemma 2 ensures that $\mathcal{L}$ and ${\mathcal{L}}_{-}$ are invertible on $L_0^2(\widehat{\mathit{\pi }})$ with

$$\begin{array}{c}\hfill {(-\mathcal{L})}^{-1}={\int }_0^{\infty }{e}^{t\mathcal{L}}\mathrm{d}t,\end{array}$$ 53

and analogously for ${\mathcal{L}}_{-}^{-1}$. In particular, the asymptotic variance can be written as ${\mathit{\sigma }}_f^2={⟨f,{(-\mathcal{L})}^{-1}f⟩}_{L^2(\widehat{\mathit{\pi }})}.$ Due to the respresentation (53) and Theorem 4, the inverses of $\mathcal{L}$ and ${\mathcal{L}}_{-}$ leave the Hermite spaces $H_m$ invariant. We will prove the claim from Proposition 2 under the assumption that $Pf=f$ which includes the case $f=f(q)$. For the following calculations we will assume $f\in H_m$ for fixed $m\ge 1$. Combining statement (f) with (a) and (e) of Lemma 5 (and noting that $H_m\subset C^{\infty }({\mathbb{R}}^{2d})\cap L^2(\widehat{\mathit{\pi }})$) we see that

$$\begin{array}{c}\hfill P\mathcal{L}P={\mathcal{L}}_{-}^{\ast }\text{and}P{\mathcal{L}}_{0}P={\mathcal{L}}_0^{\ast }\end{array}$$ 54

when restricted to $H_m$. Therefore, the following calculations are justified:

$$\begin{array}{cc}\hfill {⟨f,{(-\mathcal{L})}^{-1}f⟩}_{L^2(\widehat{\mathit{\pi }})}& =\frac{1}{2}{⟨f,{(-\mathcal{L})}^{-1}f⟩}_{L^2(\widehat{\mathit{\pi }})}+{⟨f,{(-{\mathcal{L}}^{\ast })}^{-1}f⟩}_{L^2(\widehat{\mathit{\pi }})}\hfill \\ \hfill & =\frac{1}{2}{⟨f,{(-\mathcal{L})}^{-1}f⟩}_{L^2(\widehat{\mathit{\pi }})}+{⟨Pf,{(-{\mathcal{L}}^{\ast })}^{-1}Pf⟩}_{L^2(\widehat{\mathit{\pi }})}\hfill \\ \hfill & =\frac{1}{2}{⟨f,{(-\mathcal{L})}^{-1}f⟩}_{L^2(\widehat{\mathit{\pi }})}+{⟨f,{(-{\mathcal{L}}_{-})}^{-1}f⟩}_{L^2(\widehat{\mathit{\pi }})}\hfill \\ \hfill & =\frac{1}{2}{⟨f,\left({(-\mathcal{L})}^{-1}+{(-{\mathcal{L}}_{-})}^{-1}\right)f⟩}_{L^2(\widehat{\mathit{\pi }})},\hfill \end{array}$$

where in the second line we have used the assumption $Pf=f$ and in the third line the properties $P^2=I$, $P^{\ast }=P$ and Eq. (54). Since $\mathcal{L}$ and ${\mathcal{L}}_{-}$ commute on $H_m$ according to Lemma 5(e),(f) we can write

$$\begin{array}{c}\hfill {(-\mathcal{L})}^{-1}+{(-{\mathcal{L}}_{-})}^{-1}={\mathcal{L}}_{-}{(-\mathcal{L}{\mathcal{L}}_{-})}^{-1}+\mathcal{L}{(-\mathcal{L}{\mathcal{L}}_{-})}^{-1}=-2{\mathcal{L}}_0{(\mathcal{L}{\mathcal{L}}_{-})}^{-1}\end{array}$$

for the restrictions on $H_m$, using $\mathcal{L}+{\mathcal{L}}_{-}=2{\mathcal{L}}_0$. We also have $\mathcal{L}{\mathcal{L}}_{-}=({\mathcal{L}}_0+\mathit{\mu }\mathcal{A})({\mathcal{L}}_0-\mathit{\mu }\mathcal{A})={\mathcal{L}}_0^2+{\mathit{\mu }}^2{\mathcal{A}}^{\ast }\mathcal{A},$ since ${\mathcal{L}}_0$ and $\mathcal{A}$ commute. We thus arrive at the formula

$$\begin{array}{c}\hfill {\mathit{\sigma }}_f^2={⟨f,-{\mathcal{L}}_0{({\mathcal{L}}_0^2+{\mathit{\mu }}^2{\mathcal{A}}^{\ast }\mathcal{A})}^{-1}f⟩}_{L^2(\widehat{\mathit{\pi }})},f\in H_m.\end{array}$$ 55

Now since ${({\mathcal{L}}_0^2+{\mathit{\mu }}^2{\mathcal{A}}^{\ast }\mathcal{A})}^{-1}f={(\mathcal{L}{\mathcal{L}}_{-})}^{-1}f\in \mathcal{D}({\mathcal{L}}_0)$ for all $f\in L^2(\widehat{\mathit{\pi }})$, it follows that the operator $-{\mathcal{L}}_0{({\mathcal{L}}_0^2+{\mathit{\mu }}^2{\mathcal{A}}^{\ast }\mathcal{A})}^{-1}$ is bounded. We can therefore extend formula (55) to the whole of $L^2(\widehat{\mathit{\pi }})$ by continuity, using the fact that $L_0^2(\widehat{\mathit{\pi }})={⨁}_{m\ge 1}{H}_m$. $\square$

Applying Proposition 2 we can analyse the behaviour of ${\mathit{\sigma }}_f^2$ in the limit of large perturbation strength $\mathit{\mu }\to \infty$. To this end, we introduce the orthogonal decomposition

$$\begin{array}{c}\hfill L_0^2(\mathit{\pi })=ker(Jq·{\mathrm{\nabla }}_q)\oplus ker{(Jq·{\mathrm{\nabla }}_q)}^{\perp },\end{array}$$ 56

where $Jq·{\mathrm{\nabla }}_q$ is understood as an unbounded operator acting on $L_0^2(\mathit{\pi })$, obtained as the smallest closed extension of $Jq·{\mathrm{\nabla }}_q$ acting on $C_c^{\infty }({\mathbb{R}}^d)$. In particular, $ker(Jq·{\mathrm{\nabla }}_q)$ is a closed linear subspace of $L_0^2(\mathit{\pi })$. Let $\mathit{\Pi }$ denote the $L_0^2(\mathit{\pi })$-orthogonal projection onto $ker(Jq·{\mathrm{\nabla }}_q)$. We will write ${\mathit{\sigma }}_f^2(\mathit{\mu })$ to stress the dependence of the asymptotic variance on the perturbation strength. The following result shows that for large perturbations, the limiting asymptotic variance is always smaller than the asymptotic variance in the unperturbed case. Furthermore, the limit is given as the asymptotic variance of the projected observable $\mathit{\Pi }f$ for the unperturbed dynamics.

Theorem 5

Let $f\in L_0^2(\mathit{\pi })$ (so in particular $f=f(q)$). Then ${lim}_{\mathit{\mu }\to \infty }{\mathit{\sigma }}_f^2(\mathit{\mu })={\mathit{\sigma }}_{\mathit{\Pi }f}^2(0)\le {\mathit{\sigma }}_f^2(0).$

Remark 9

Note that the fact that the limit exists and is finite is nontrivial. In particular, as Fig. 1b, c demonstrate, it is often the case that ${lim}_{\mathit{\mu }\to \infty }{\mathit{\sigma }}_f^2(\mathit{\mu })=\infty$ if the condition $\mathit{\mu }=\mathit{\nu }$ is not satisfied.

Remark 10

The decomposition (56) can be interpreted in terms of the spectrum $\mathit{\sigma }(\mathcal{L})$ as follows: First observe that for functions f that only depend on q, $f\in ker(Jq·{\mathrm{\nabla }}_q)$ is equivalent to $f\in ker\mathcal{A}$. Let us denote by $\overline{\mathit{\sigma }}={\bigcap }_{\mathit{\mu }\in \mathbb{R}}\mathit{\sigma }({\mathcal{L}}_0+\mathit{\mu }\mathcal{A})$ the part of $\mathit{\sigma }({\mathcal{L}}_0)$ that is not affected by the perturbation and by

$$\begin{array}{c}\hfill \overline{E}=\overline{\mathrm{span}\{f\in L^2(\widehat{\mathit{\pi }}):\text{there exists}\mathit{\lambda }\in \overline{\mathit{\sigma }}\text{such that}{\mathcal{L}}_{0}f=\mathit{\lambda }f\}}\subset L^2(\widehat{\mathit{\pi }})\end{array}$$

the corresponding subspace. Then it is straightforward to see that $ker(\mathcal{A})=\overline{E}$.3 In Fig. 2a, $\overline{\mathit{\sigma }}$ has been highlighted by diamonds.

Proof of Theorem 5

Note that ${\mathcal{L}}_0$ and ${\mathcal{A}}^{\ast }\mathcal{A}$ leave the Hermite spaces $H_m$ invariant and their restrictions to those spaces commute (see Lemma 5, (b), (c) and (f)). Furthermore, as the Hermite spaces $H_m$ are finite-dimensional, those operators have discrete spectrum. As ${\mathcal{A}}^{\ast }\mathcal{A}$ is nonnegative self-adjoint, there exists an orthogonal decomposition $L_0^2(\mathit{\pi })={⨁}_i{W}_i$ into eigenspaces of the operator $-{\mathcal{L}}_0{({\mathcal{L}}_0^2+{\mathit{\mu }}^2{\mathcal{A}}^{\ast }\mathcal{A})}^{-1}$, the decomposition $⨁W_i$ being finer then $⨁H_m$ in the sense that every $W_i$ is a subspace of some $H_m$. Moreover, $-{\mathcal{L}}_0{({\mathcal{L}}_0^2+{\mathit{\mu }}^2{\mathcal{A}}^{\ast }\mathcal{A})}^{-1}{{|}_{W_i}=-{\mathcal{L}}_0{({\mathcal{L}}_0^2+{\mathit{\mu }}^2{\mathit{\lambda }}_i)}^{-1}|}_{W_i},$ where ${\mathit{\lambda }}_i\ge 0$ is the eigenvalue of ${\mathcal{A}}^{\ast }\mathcal{A}$ associated to the subspace $W_i$. Consequently, formula (50) can be written as

$$\begin{array}{c}\hfill {\mathit{\sigma }}_f^2=\sum _i{⟨f_i,-{\mathcal{L}}_0{({\mathcal{L}}_0^2+{\mathit{\mu }}^2{\mathit{\lambda }}_i)}^{-1}{f}_i⟩}_{L^2(\widehat{\mathit{\pi }})},\end{array}$$ 57

where $f={\sum }_i{f}_i$ and $f_i\in W_i$. Let us assume now without loss of generality that $W_0=ker{\mathcal{A}}^{\ast }\mathcal{A}$, so in particular ${\mathit{\lambda }}_0=0$. Then clearly

$$\begin{array}{c}\hfill \underset{\mathit{\mu }\to \infty }{lim}{\mathit{\sigma }}_f^2=2{⟨f_0,-{\mathcal{L}}_0{({\mathcal{L}}_0^2)}^{-1}{f}_0⟩}_{L^2(\widehat{\mathit{\pi }})}=2{⟨f_0,{(-{\mathcal{L}}_0)}^{-1}{f}_0⟩}_{L^2(\widehat{\mathit{\pi }})}={\mathit{\sigma }}_{f_0}^2(0).\end{array}$$

Now notice that $W_0=ker{\mathcal{A}}^{\ast }\mathcal{A}=ker\mathcal{A}$, showing the equality in the claim. It remains to show that ${\mathit{\sigma }}_{\mathit{\Pi }f}^2(0)\le {\mathit{\sigma }}_f^2(0)$. To see this, we write

$$\begin{array}{cc}\hfill {\mathit{\sigma }}_f^2(0)& =2{〈f,{(-{\mathcal{L}}_0)}^{-1}f{⟩}_{L^2(\widehat{\mathit{\pi }})}=2⟨\mathit{\Pi }f+(1-\mathit{\Pi })f,{(-{\mathcal{L}}_0)}^{-1}(\mathit{\Pi }f+(1-\mathit{\Pi })f)〉}_{L^2(\widehat{\mathit{\pi }})}\hfill \\ \hfill & ={\mathit{\sigma }}_{\mathit{\Pi }f}^2(0)+{\mathit{\sigma }}_{(1-\mathit{\Pi })f}^2(0)+R,\hfill \end{array}$$

where

$$\begin{array}{c}\hfill R=2{⟨\mathit{\Pi }f,{(-{\mathcal{L}}_0)}^{-1}(1-\mathit{\Pi })f⟩}_{L^2(\widehat{\mathit{\pi }})}+2{⟨(1-\mathit{\Pi })f,{(-{\mathcal{L}}_0)}^{-1}\mathit{\Pi }f⟩}_{L^2(\widehat{\mathit{\pi }})}.\end{array}$$

Note that since we only consider observables that do not depend on p, $\mathit{\Pi }f\in ker(Jq·{\mathrm{\nabla }}_q)$ and $(1-\mathit{\Pi })f\in {⨁}_{i\ge 1}{W}_i$. Since ${\mathcal{L}}_0$ commutes with $\mathcal{A}$, it follows that ${(-{\mathcal{L}}_0)}^{-1}$ leaves both $W_0$ and ${⨁}_{i\ge 1}{W}_i$ invariant. Therefore, as the latter spaces are orthogonal to each other, it follows that $R=0$, from which the result follows. $\square$

From Theorem 5 it follows that in the limit as $\mathit{\mu }\to \infty$, the asymptotic variance ${\mathit{\sigma }}_f^2(\mathit{\mu })$ is not decreased by the perturbation if $f\in ker(Jq·{\mathrm{\nabla }}_q)$. In fact, this result also holds true non-asymptotically, i.e. observables in $ker(Jq·{\mathrm{\nabla }}_q)$ are not affected at all by the perturbation:

Lemma 6

Let $f\in ker(Jq·{\mathrm{\nabla }}_q)$. Then ${\mathit{\sigma }}_f^2(\mathit{\mu })={\mathit{\sigma }}_f^2(0)$ for all $\mathit{\mu }\in \mathbb{R}$.

Proof

From $f\in ker(Jq·{\mathrm{\nabla }}_q)$ it follows immediately that $f\in ker{\mathcal{A}}^{\ast }\mathcal{A}$. Then the claim follows from the expression (57). $\square$

Example 1

Recall the case of observables of the form $f(q)=q·Kq+l·q+C$ with $K\in {\mathbb{R}}_{sym}^{d\times d}$, $l\in {\mathbb{R}}^d$ and $C\in \mathbb{R}$ from Sect. 4.1. If $[J,K]=0$ and $l\in kerJ$, then $f\in ker(Jq·{\mathrm{\nabla }}_q)$ as

$$\begin{array}{c}\hfill Jq·{\mathrm{\nabla }}_q(q·Kq+l·q+C)=2Jq·Kq+Jq·l=q·(KJ-JK)q-q·Jl=0.\end{array}$$

From the preceding lemma it follows that ${\mathit{\sigma }}_f^2(\mathit{\mu })={\mathit{\sigma }}_f^2(0)$ for all $\mathit{\mu }\in \mathbb{R},$ showing that the assumption in Theorem 3 does not exclude nontrivial cases.

The following result shows that the dynamics (34) is particularly effective for antisymmetric observables (at least in the limit of large perturbations):

Proposition 3

Let $f\in L_0^2(\mathit{\pi })$ satisfy $f(-q)=-f(q)$ and assume that $kerJ=\{0\}$. Furthermore, assume that the eigenvalues of J are rationally independent, i.e.

$$\begin{array}{c}\hfill \mathit{\sigma }(J)=\{\pm i{\mathit{\lambda }}_1,\pm i{\mathit{\lambda }}_2,\dots ,\pm i{\mathit{\lambda }}_d\}\end{array}$$ 58

with ${\mathit{\lambda }}_i\in {\mathbb{R}}_{>0}$ and ${\sum }_i{k}_i{\mathit{\lambda }}_i\ne 0$ for all $(k_1,\dots ,k_d)\in {\mathbb{Z}}^d\backslash (0,\dots ,0)$. Then ${lim}_{\mathit{\mu }\to \infty }{\mathit{\sigma }}_f^2(\mathit{\mu })=0$.

Proof of Proposition 3

The claim would immediately follow from $f\in ker{(Jq·\mathrm{\nabla })}^{\perp }$ according to Theorem 5, but that does not seem to be so easy to prove directly. Instead, we again make use of the Hermite polynomials.

Recall from the proof of Proposition 2 that $\mathcal{L}$ is invertible on $L_0^2(\widehat{\mathit{\pi }})$ and its inverse leaves the Hermite spaces $H_m$ invariant. Consequently, the asymptotic variance of an observable $f\in L_0^2(\widehat{\mathit{\pi }})$ can be written as

$$\begin{array}{c}\hfill {\mathit{\sigma }}_f^2={⟨f,{(-\mathcal{L})}^{-1}f⟩}_{L^2(\widehat{\mathit{\pi }})}=\sum _{m=1}^{\infty }{⟨{\mathit{\Pi }}_{m}f,{(-\mathcal{L}{|}_{H_m})}^{-1}{\mathit{\Pi }}_{m}f⟩}_{L^2(\widehat{\mathit{\pi }})},\end{array}$$ 59

where ${\mathit{\Pi }}_m:L_0^2(\widehat{\mathit{\pi }})\to H_m$ denotes the orthogonal projection onto $H_m$. From (49) it is clear that $g_a$ is symmetric for $|\mathit{\alpha }|$ even and antisymmetric for $|\mathit{\alpha }|$ odd. Therefore, from f being antisymmetric it follows that $f\in {⨁}_{m\ge 1,m\text{odd}}{H}_m.$ In view of (45), ((c)) and (58) the spectrum of ${\mathcal{L}}_{|H_m}$ can be written as

$$\begin{array}{cc}\hfill \mathit{\sigma }(\mathcal{L}{|}_{H_m})=& \left\{\mathit{\mu }\sum _{j=1}^{2d}{\mathit{\alpha }}_j{\mathit{\beta }}_j+C_{\mathit{\alpha },\mathit{\gamma }}:|\mathit{\alpha }|=m,{\mathit{\beta }}_j\in \mathit{\sigma }(J)\right\}\hfill \\ \hfill =& \left\{i\mathit{\mu }\sum _{j=1}^d({\mathit{\alpha }}_j-{\mathit{\alpha }}_{j+d}){\mathit{\lambda }}_j+C_{\mathit{\alpha },\mathit{\gamma }}:|\mathit{\alpha }|=m\right\}\hfill \end{array}$$ 60

with appropriate real constants $C_{\mathit{\alpha },\mathit{\gamma }}\in \mathbb{R}$ that depend on $\mathit{\alpha }$ and $\mathit{\gamma }$, but not on $\mathit{\mu }$. For $|\mathit{\alpha }|={\sum }_{j=1}^{2d}{\mathit{\alpha }}_j=m$ odd, we have that

$$\begin{array}{c}\hfill \sum _{j=1}^d({\mathit{\alpha }}_j-{\mathit{\alpha }}_{j+d}){\mathit{\lambda }}_j\ne 0.\end{array}$$ 61

Indeed, assume to the contrary that the above expression is zero. Then it would follow that ${\mathit{\alpha }}_j={\mathit{\alpha }}_{j+d}$ for all $j=1,\dots ,d$ by rational independence of ${\mathit{\lambda }}_1,\dots ,{\mathit{\lambda }}_d$ and $|m|$ would have to be even. From (60) and (61) it is clear that

$$\begin{array}{c}\hfill sup\left\{r>0:B(0,r)\cap \mathit{\sigma }(\mathcal{L}{|}_{H_m})=\mathrm{\varnothing }\right\}\stackrel{\mathit{\mu }\to \infty }{\to }\infty ,\end{array}$$

where B(0, r) denotes the ball of radius r centered at the origin in $\mathbb{C}$. Consequently, the spectral radius of ${(-\mathcal{L}{|}_{H_m})}^{-1}$ and hence ${(-\mathcal{L}{|}_{H_m})}^{-1}$ itself converges to zero as $\mathit{\mu }\to \infty$. The result then follows from (59). $\square$

Remark 11

The idea of the preceding proof can be explained using Fig. 2a and Remark 10. The eigenvalues in the fixed spectrum $\overline{E}$ (on the real axis, highlighted by diamonds) correspond to Hermite polynomials of even order. The independence condition on the eigenvalues of J prevents cancellations that would lead to fixed eigenvalues associated to Hermite polynomials of odd order. Therefore, antisymmetric observables are orthogonal to $\overline{E}=ker\mathcal{A}$.

The following corollary gives a version of the converse of Proposition 3 and provides further intuition into the mechanics of the variance reduction achieved by the perturbation.

Corollary 1

Let $f\in L_0^2(\mathit{\pi })$ and assume that $li{m}_{\mathit{\mu }\to \infty }{\mathit{\sigma }}_f^2(\mathit{\mu })=0$. Then

$$\begin{array}{c}\hfill {\int }_{B(0,r)}f\mathrm{dq}=0\end{array}$$

for all $r\in (0,\infty )$, where B(0, r) denotes the ball centered at 0 with radius r.

Proof

According to Theorem 5, ${lim}_{\mathit{\mu }\to \infty }{\mathit{\sigma }}_f^2(\mathit{\mu })=0$ implies ${\mathit{\sigma }}_{\mathit{\Pi }f}^2(0)=0$. We can write

$$\begin{array}{c}\hfill {\mathit{\sigma }}_{\mathit{\Pi }f}^2(0)={⟨\mathit{\Pi }f,{(-{\mathcal{L}}_0)}^{-1}\mathit{\Pi }f⟩}_{L^2(\widehat{\mathit{\pi }})}=\frac{1}{2}{⟨\mathit{\Pi }f,\left({(-{\mathcal{L}}_0)}^{-1}+{(-{\mathcal{L}}_0^{\ast })}^{-1}\right)\mathit{\Pi }f⟩}_{L^2(\widehat{\mathit{\pi }})}\end{array}$$ 62

and recall from the proof of Proposition 2 that ${(-{\mathcal{L}}_0)}^{-1}$ and ${(-{\mathcal{L}}_0^{\ast })}^{-1}$ leave the Hermite spaces $H_m$ invariant. Therefore $ker\left({(-{\mathcal{L}}_0)}^{-1}+{(-{\mathcal{L}}_0^{\ast })}^{-1}\right)=0$ in $L_0^2(\widehat{\mathit{\pi }})$, and in particular ${\mathit{\sigma }}_{\mathit{\Pi }f}^2(0)=0$ implies $\mathit{\Pi }f=0$, which in turn shows that $f\in ker{(Jq·\mathrm{\nabla })}^{\perp }$. From ${(Jq·\mathrm{\nabla })}^{\ast }=-Jq·\mathrm{\nabla }$, it follows that

$$\begin{array}{c}\hfill ker{(Jq·\mathrm{\nabla })}^{\perp }=\overline{\mathrm{im}{(Jq·\mathrm{\nabla })}^{\ast }}=\overline{\mathrm{im}(Jq·\mathrm{\nabla })}.\end{array}$$ 63

Hence, there exists a sequence ${({\mathit{\varphi }}_n)}_n\in C_c^{\infty }({\mathbb{R}}^d)$ such that $Jq·\mathrm{\nabla }{\mathit{\varphi }}_n\to f$ in $L^2(\mathit{\pi })$. Taking a subsequence if necessary, we can assume that the convergence is pointwise $\mathit{\pi }$-almost everywhere and that the sequence is pointwise bounded by a function in $L^1(\mathit{\pi })$. Since J is antisymmetric, we have that $Jq·\mathrm{\nabla }{\mathit{\varphi }}_n=\mathrm{\nabla }·({\mathit{\varphi }}_{n}Jq)$. Now Gauss’s theorem yields

$$\begin{array}{c}\hfill {\int }_{B(0,r)}f\mathrm{d}q={\int }_{B(0,r)}\mathrm{\nabla }·(\mathit{\varphi }Jq)\mathrm{d}q={\int }_{\mathit{\partial }B(0,r)}\mathit{\varphi }Jq·\mathrm{d}n,\end{array}$$

where n denotes the outward normal to the sphere $\mathit{\partial }B(0,r)$. This quantity is zero due to the orthogonality of Jq and n, and so the result follows from Lebesgue’s dominated convergence theorem. $\square$

Optimal Choices of J for Quadratic Observables

Assume $f\in L_0^2(\mathit{\pi })$ is given by $f(q)=q·Kq+l·q-\mathrm{Tr}K$, with $K\in {\mathbb{R}}_{sym}^{d\times d}$ and $l\in {\mathbb{R}}^d$ (note that the constant term is chosen such that $\mathit{\pi }(f)=0$). Our objective is to choose J in such a way that ${lim}_{\mathit{\mu }\to \infty }{\mathit{\sigma }}_f^2(\mathit{\mu })$ becomes as small as possible. To stress the dependence on the choice of J, we introduce the notation ${\mathit{\sigma }}_f^2(\mathit{\mu },J)$. Also, we denote the orthogonal projection onto ${(kerJ)}^{\perp }$ by ${\mathit{\Pi }}_{kerJ}^{\perp }$.

Lemma 7

(Zero variance limit for linear observables). Assume $K=0$ and ${\mathit{\Pi }}_{kerJ}^{\perp }l=0$. Then

$$\begin{array}{c}\hfill \underset{\mathit{\mu }\to \infty }{lim}{\mathit{\sigma }}_f^2(\mathit{\mu },J)=0.\end{array}$$

Proof

According to Theorem 5, we have to show that $\mathit{\Pi }f=0$, where $\mathit{\Pi }$ is the $L^2(\mathit{\pi })$-orthogonal projection onto $ker(Jq·\mathrm{\nabla })$. Let us thus use (63) and prove that $f\in \overline{\mathrm{im}(Jq·\mathrm{\nabla })}.$ Indeed, since ${\mathit{\Pi }}_{kerJ}^{\perp }l=0$, by Fredholm’s alternative there exists $u\in {\mathbb{R}}^d$ such that $Ju=l$. Now define $\mathit{\varphi }\in L_0^2(\mathit{\pi })$ by $\mathit{\varphi }(q)=-u·q,$ leading to $f=Jq·\mathrm{\nabla }\mathit{\varphi },$ so the result follows. $\square$

Lemma 8

(Zero variance limit for purely quadratic observables.) Let $l=0$ and consider the decomposition $K=K_0+K_1$ into the traceless part $K_0=K-\frac{\mathrm{Tr}K}{d}·I$ and the trace-part $K_1=\frac{\mathrm{Tr}K}{d}·I.$ For the corresponding decomposition of the observable

$$\begin{array}{c}\hfill f(q)=f_0(q)+f_1(q)=q·K_{0}q+q·K_{1}q-\mathrm{Tr}K\end{array}$$

the following holds:

1. There exists an antisymmetric matrix J such that ${lim}_{\mathit{\mu }\to \infty }{\mathit{\sigma }}_{f_0}^2(\mathit{\mu },J)=0,$ and there is an algorithmic way (see Algorithm 1) to compute an appropriate J in terms of K.

2. The trace-part is not effected by the perturbation, i.e. ${\mathit{\sigma }}_{f_1}^2(\mathit{\mu },J)={\mathit{\sigma }}_{f_1}^2(0)$ for all $\mathit{\mu }\in \mathbb{R}$.

Proof

To prove the first claim, according to Theorem 5 it is sufficient to show that $f_0\in ker{(Jq·\mathrm{\nabla })}^{\perp }=\overline{\mathrm{im}(Jq·\mathrm{\nabla })}$. Let us consider the function $\mathit{\varphi }(q)=q·Aq$, with $A\in {\mathbb{R}}_{sym}^{d\times d}$. It holds that $Jq·\mathrm{\nabla }\mathit{\varphi }=2q·(J^{T}Aq)=q·[A,J]q.$ The task of finding an antisymmetric matrix J such that ${lim}_{\mathit{\mu }\to \infty }{\mathit{\sigma }}_{f_0}^2(\mathit{\mu },J)=0$ can therefore be accomplished by constructing an antisymmetric matrix J such that there exists a symmetric matrix A with the property $K_0=[A,J]$. Given any traceless matrix $K_0$ there exists an orthogonal matrix $U\in O({\mathbb{R}}^d)$ such that $U{K}_0{U}^T$ has zero entries on the diagonal, and that U can be obtained in an algorithmic manner (see for example [29] or [22, Chap. 2, Sect. 2, Problem 3]) Assume thus that such a matrix $U\in O({\mathbb{R}}^d)$ has been found and choose real numbers $a_1,\dots ,a_d\in \mathbb{R}$ such that $a_i\ne a_j$ if $i\ne j$. We now set $\overline{A}=\mathrm{diag}(a_1,\dots ,a_n),$ and

$$\begin{array}{c}\hfill {\overline{J}}_{ij}=\left\{\begin{array}{cc}\frac{{(U{K}_0{U}^T)}_{ij}}{a_i-a_j}\hfill & \text{if}i\ne j,\hfill \\ 0\hfill & \text{if}i=j.\hfill \end{array}\right.\end{array}$$ 64

Observe that since $U{K}_0{U}^T$ is symmetric, $\overline{J}$ is antisymmetric. A short calculation shows that $[\overline{A},\overline{J}]=U{K}_0{U}^T$. We can thus define $A=U^T\overline{A}U$ and $J=U^T\overline{J}U$ to obtain $[A,J]=K_0$. Therefore, the J constructed in this way indeed satisfies (4.4). For the second claim, note that $f_1\in ker(Jq·\mathrm{\nabla })$, since $Jq·\mathrm{\nabla }\left(q·\frac{\mathrm{Tr}K}{d}q\right)=2\frac{\mathrm{Tr}K}{d}q·Jq=0$ due to the antisymmetry of J. The result then follows from Lemma 6. $\square$

We would like to stress that the perturbation J constructed in the previous lemma is far from unique due to the freedom of choice of U and $a_1,\dots ,a_d\in \mathbb{R}$ in its proof. However, it is asymptotically optimal:

Corollary 2

In the setting of Lemma 8 the following holds:

$$\begin{array}{c}\hfill \underset{J^T=-J}{min}\left(\underset{\mathit{\mu }\to \infty }{lim}{\mathit{\sigma }}_f^2(\mathit{\mu },J)\right)={\mathit{\sigma }}_{f_1}^2(0).\end{array}$$

Proof

The claim follows immediately since $f_1\in ker(Jq·\mathrm{\nabla })$ for arbitrary antisymmetric J as shown in (4.4), and therefore the contribution of the trace part $f_1$ to the asymptotic variance cannot be reduced by any choice of J according to Lemma 6. $\square$

As the proof of Lemma 8 is constructive, we obtain the following algorithm for determining optimal perturbations for quadratic observables:

Algorithm 1

Given $K\in {\mathbb{R}}_{sym}^{d\times d}$, determine an optimal antisymmetric perturbation J as follows:

1. Set $K_0=K-\frac{\mathrm{Tr}K}{d}·I.$

2. Find $U\in O({\mathbb{R}}^d)$ such that $U{K}_0{U}^T$ has zero entries on the diagonal.

3. Choose $a_i\in \mathbb{R},$ $i=1,\dots d$ such that $a_i\ne a_j$ for $i\ne j$ and set

   $$\begin{array}{c}\hfill {\overline{J}}_{ij}=\frac{{(U{K}_0{U}^T)}_{ij}}{a_i-a_j}\end{array}$$

   for $i\ne j$ and ${\overline{J}}_{ii}=0$ otherwise.

4. Set $J=U^T\overline{J}U$.

Remark 12

In [14], the authors consider the task of finding optimal perturbations J for the nonreversible overdamped Langevin dynamics given in (15). In the Gaussian case this optimization problem turns out be equivalent to the one considered in this section. Indeed, equation (39) of [14] can be rephrased as $f\in ker{(Jq·\mathrm{\nabla })}^{\perp }.$ Therefore, Algorithm 1 and its generalization Algorithm 2 (described in Sect. 4.5) can be used without modifications to find optimal perturbations of overdamped Langevin dynamics.

Gaussians with Arbitrary Covariance and Preconditioning

In this section we extend the results of the preceding sections to the case when the target measure $\mathit{\pi }$ is given by a Gaussian with arbitrary covariance, i.e. $V(q)=\frac{1}{2}q·Sq$ with $S\in {\mathbb{R}}_{sym}^{d\times d}$ symmetric and positive definite. The dynamics (8) then takes the form

$$\begin{array}{c}\hfill \mathrm{d}{q}_t=M^{-1}{p}_t\mathrm{d}t-\mathit{\mu }{J}_{1}S{q}_t\mathrm{d}t,\mathrm{d}{p}_t=-S{q}_t\mathrm{d}t-\mathit{\nu }{J}_2{M}^{-1}{p}_t\mathrm{d}t-\mathit{\Gamma }{M}^{-1}{p}_t\mathrm{d}t+\sqrt{2\mathit{\Gamma }}\mathrm{d}{W}_t.\end{array}$$ 65

The key observation is now that the choices $M=S$ and $\mathit{\Gamma }=\mathit{\gamma }S$ together with the transformation $\tilde{q}=S^{1/2}q$ and $\tilde{p}=S^{-1/2}p$ lead to the dynamics

$$\begin{array}{cc}\hfill \mathrm{d}{\tilde{q}}_t& ={\tilde{p}}_t\mathrm{d}t-\mathit{\mu }{S}^{1/2}{J}_1{S}^{1/2}{\tilde{q}}_t\mathrm{d}t,\hfill \\ \hfill \mathrm{d}{\tilde{p}}_t& =-{\tilde{q}}_t\mathrm{d}t-\mathit{\mu }{S}^{-1/2}{J}_2{S}^{-1/2}{\tilde{p}}_t\mathrm{d}t-\mathit{\gamma }{\tilde{p}}_t\mathrm{d}t+\sqrt{2\mathit{\gamma }}\mathrm{d}{W}_t,\hfill \end{array}$$ 66

which is of the form (34) if $J_1$ and $J_2$ obey the condition $S{J}_{1}S=J_2$. Clearly the dynamics (66) is ergodic with respect to a Gaussian measure with unit covariance, in the following denoted by $\tilde{\mathit{\pi }}$. The connection between the asymptotic variances associated to (65) and (66) is as follows:

For an observable $f\in L_0^2(\mathit{\pi })$ we can write

$$\begin{array}{c}\hfill \sqrt{T}(\frac{1}{T}{\int }_0^{T}f(q_s)\mathrm{d}s-\mathit{\pi }(f))=\sqrt{T}(\frac{1}{T}{\int }_0^T\tilde{f}({\tilde{q}}_s)\mathrm{d}s-\tilde{\mathit{\pi }}(\tilde{f})),\end{array}$$

where $\tilde{f}(q)=f(S^{-1/2}q)$. Therefore, the asymptotic variances satisfy ${\mathit{\sigma }}_f^2={\tilde{\mathit{\sigma }}}_{\tilde{f}}^2,$ where ${\tilde{\mathit{\sigma }}}_{\tilde{f}}^2$ denotes the asymptotic variance of the process ${({\tilde{q}}_t)}_{t\ge 0}$. Because of this, the results from the previous sections generalise to (65), subject to the condition that the choices $M=S$, $\mathit{\Gamma }=\mathit{\gamma }S$ and $S{J}_{1}S=J_2$ are made. We formulate our results in this general setting as corollaries:

Corollary 3

Consider the dynamics

$$\begin{array}{cc}\hfill \mathrm{d}{q}_t=& M^{-1}{p}_t\mathrm{d}t-\mathit{\mu }{J}_1\mathrm{\nabla }V(q_t)\mathrm{d}t,\hfill \\ \hfill \mathrm{d}{p}_t=& -\mathrm{\nabla }V(q_t)\mathrm{d}t-\mathit{\mu }{J}_2{M}^{-1}{p}_t\mathrm{d}t-\mathit{\Gamma }{M}^{-1}{p}_t\mathrm{d}t+\sqrt{2\mathit{\Gamma }}\mathrm{d}{W}_t,\hfill \end{array}$$ 67

with $V(q)=\frac{1}{2}q·Sq$. Assume that $M=S$, $\mathit{\Gamma }=\mathit{\gamma }S$ with $\mathit{\gamma }>\sqrt{2}$ and $S{J}_{1}S=J_2$. Let $f\in L^2(\mathit{\pi })$ be an observable of the form

$$\begin{array}{c}\hfill f(q)=q·Kq+l·q+C\end{array}$$ 68

with $K\in {\mathbb{R}}_{sym}^{d\times d}$, $l\in {\mathbb{R}}^d$ and $C\in \mathbb{R}$. If at least one of the conditions $K{J}_{1}S\ne S{J}_{1}K$ and $l\notin kerJ$ is satisfied, then the asymptotic variance is at a local maximum for the unperturbed sampler, i.e.

$$\begin{array}{c}\hfill {\left.{\mathit{\partial }}_{\mathit{\mu }}{\mathit{\sigma }}_f^2\right|}_{\mathit{\mu }=0}=0\text{and}{\left.{\mathit{\partial }}_{\mathit{\mu }}^2{\mathit{\sigma }}_f^2\right|}_{\mathit{\mu }=0}<0.\end{array}$$

Proof

Note that

$$\begin{array}{c}\hfill \tilde{f}(q)=f(S^{-1/2}q)=q·S^{-1/2}K{S}^{-1/2}q+S^{-1/2}l·q+C=q·\tilde{K}q+\tilde{l}·q+C\end{array}$$

is again of the form (68) (where in the last equality, $\tilde{K}=S^{-1/2}K{S}^{-1/2}$ and $\tilde{l}=S^{-1/2}l$ have been defined). From (66), (4.5) and Theorem 3 the claim follows if at least one of the conditions $[\tilde{K},S^{1/2}{J}_1{S}^{1/2}]\ne 0$ and $\tilde{l}\notin ker(S^{1/2}{J}_1{S}^{1/2})$ is satisfied. The first of those can easily seen to be equivalent to $S^{-1/2}(KJS-SJK)S^{-1/2}\ne 0,$ which is equivalent to $K{J}_{1}S\ne S{J}_{1}K$ since S is nondegenerate. The second condition is equivalent to $S^{1/2}{J}_{1}l\ne 0,$ which is equivalent to $J_{1}l\ne 0,$ again by nondegeneracy of S. $\square$

Corollary 4

Assume the setting from the previous corollary and denote by $\mathit{\Pi }$ the orthogonal projection onto $ker(J_{1}Sq·\mathrm{\nabla })$. For $f\in L^2(\mathit{\pi })$ it holds that ${lim}_{\mathit{\mu }\to \infty }{\mathit{\sigma }}_f^2(\mathit{\mu })={\mathit{\sigma }}_{\mathit{\Pi }f}^2(0)\le {\mathit{\sigma }}_f^2(0).$

Proof

Theorem 5 implies ${lim}_{\mathit{\mu }\to \infty }{\tilde{\mathit{\sigma }}}_{\tilde{f}}^2(\mathit{\mu })={\tilde{\mathit{\sigma }}}_{\tilde{\mathit{\Pi }}\tilde{f}}^2(0)\le {\tilde{\mathit{\sigma }}}_{\tilde{f}}^2(0)$ for the transformed system (66). Here $\tilde{f}(q)=f(S^{-1/2}q)$ is the transformed observable and $\tilde{\mathit{\Pi }}$ denotes $L^2(\mathit{\pi })$-orthogonal projection onto $ker(S^{1/2}{J}_1{S}^{1/2}q·\mathrm{\nabla })$. According to (4.5), it is sufficient to show that $(\mathit{\Pi }f)\circ S^{-1/2}=\tilde{\mathit{\Pi }}\tilde{f}$. This however follows directly from the fact that the linear transformation $\mathit{\varphi }\mapsto \mathit{\varphi }\circ S^{1/2}$ maps $ker(S^{1/2}{J}_1{S}^{1/2}q·\mathrm{\nabla })$ bijectively onto $ker(J_{1}Sq·\mathrm{\nabla })$. $\square$

Let us also reformulate Algorithm 1 for the case of a Gaussian with arbitrary covariance.

Algorithm 2

Given $K,S\in {\mathbb{R}}_{sym}^{d\times d}$ with $f(q)=q·Kq$ and $V(q)=\frac{1}{2}q·Sq$ (assuming S is nondegenerate), determine optimal perturbations $J_1$ and $J_2$ as follows:

1. Set $\tilde{K}=S^{-1/2}K{S}^{-1/2}$ and ${\tilde{K}}_0=\tilde{K}-\frac{\mathrm{Tr}\tilde{K}}{d}·I$.

2. Find $U\in O({\mathbb{R}}^d)$ such that $U{\tilde{K}}_0{U}^T$ has zero entries on the diagonal.

3. Choose $a_i\in \mathbb{R}$, $i=1,\dots ,d$ such that $a_i\ne a_j$ for $i\ne j$ and set

   $$\begin{array}{c}\hfill {\overline{J}}_{ij}=\frac{{(U{\tilde{K}}_0{U}^T)}_{ij}}{a_i-a_j}.\end{array}$$

4. Set $\tilde{J}=U^T\overline{J}U$.

5. Put $J_1=S^{-1/2}\tilde{J}{S}^{-1/2}$ and $J_2=S^{1/2}J{S}^{1/2}$.

Finally, we obtain the following optimality result from Lemma 7 and Corollary 2.

Corollary 5

Let $f(q)=q·Kq+l·q-\mathrm{Tr}K$ and assume that ${\mathit{\Pi }}_{kerJ}^{\perp }l=0$. Then

$$\begin{array}{c}\hfill \underset{J_1^T=-J_1,J_2=S{J}_{1}S}{min}\left(\underset{\mathit{\mu }\to \infty }{lim}{\mathit{\sigma }}_f^2(\mathit{\mu },J_1,J_2)\right)={\mathit{\sigma }}_{f_1}^2(0),\end{array}$$

where $f_1(q)=q·K_{1}q$, $K_1=\frac{\mathrm{Tr}(S^{-1}K)}{d}S$. Optimal choices for $J_1$ and $J_2$ can be obtained using Algorithm 2.

Remark 13

Since in Sect. 4.1 we analysed the case where $J_1$ and $J_2$ are proportional, we are not able to drop the restriction $J_2=S{J}_{1}S$ from the above optimality result. Analysis of completely arbitrary perturbations will be the subject of future work.

Remark 14

The choices $M=S$ and $\mathit{\Gamma }=\mathit{\gamma }S$ have been introduced to make the perturbations considered in this article lead to samplers that perform well in terms of reducing the asymptotic variance. However, adjusting the mass and friction matrices according to the target covariance in this way (i.e. $M=S$ and $\mathit{\Gamma }=\mathit{\gamma }S$) is a popular way of preconditioning the dynamics, see for instance [18] and, in particular mass-tensor molecular dynamics [6]. Here we will present an argument why such a preconditioning is indeed beneficial in terms of the convergence rate of the dynamics. Let us first assume that S is diagonal, i.e. $S=\mathrm{diag}(s^{(1)},\dots ,s^{(d)})$ and that $M=\mathrm{diag}(m^{(d)},\dots ,m^{(d)})$ and $\mathit{\Gamma }=\mathrm{diag}({\mathit{\gamma }}^{(d)},\dots ,{\mathit{\gamma }}^{(d)})$ are chosen diagonally as well. Then (65) decouples into one-dimensional SDEs of the following form:

$$\begin{array}{c}\hfill \mathrm{d}{q}_t^{(i)}=\frac{1}{m^{(i)}}{p}_t^{(i)}\mathrm{d}t,\mathrm{d}{p}_t^{(i)}=-s^{(i)}{q}_t^{(i)}\mathrm{d}t-\frac{{\mathit{\gamma }}^{(i)}}{m^{(i)}}{p}_t^{(i)}\mathrm{d}t+\sqrt{2{\mathit{\gamma }}^{(i)}}\mathrm{d}{W}_t,i=1,\dots ,d.\end{array}$$ 69

Let us write those Ornstein–Uhlenbeck processes as

$$\begin{array}{c}\hfill \mathrm{d}{X}_t^{(i)}=-B^{(i)}{X}_t^{(i)}\mathrm{d}t+\sqrt{2Q^{(i)}}\mathrm{d}{W}_t^{(i)},B^{(i)}=\left(\begin{array}{cc}0& -\frac{1}{m^{(i)}}\\ s^{(i)}& \frac{{\mathit{\gamma }}^{(i)}}{m^{(i)}}\end{array}\right),Q^{(i)}=\left(\begin{array}{cc}0& 0\\ 0& {\mathit{\gamma }}^{(i)}\end{array}\right).\end{array}$$ 70

As in Sect. 4.2, the rate of the exponential decay of (70) is equal to $min\text{Re}\mathit{\sigma }(B^{(i)})$. A short calculation shows that the eigenvalues of $B^{(i)}$ are given by

$$\begin{array}{c}\hfill {\mathit{\lambda }}_{1,2}^{(i)}=\frac{{\mathit{\gamma }}^{(i)}}{2m^{(i)}}\pm \sqrt{(\frac{{\mathit{\gamma }}^{(i)}}{2m^{(i)}}{)}^2-\frac{s^{(i)}}{m^{(i)}}}.\end{array}$$

Therefore, the rate of exponential decay is maximal when

$$\begin{array}{c}\hfill (\frac{{\mathit{\gamma }}^{(i)}}{2m^{(i)}}{)}^2-\frac{s^{(i)}}{m^{(i)}}=0,\end{array}$$ 71

in which case it is given by

$$\begin{array}{c}\hfill {({\mathit{\lambda }}^{(i)})}^{\ast }=\sqrt{\frac{s^{(i)}}{m^{(i)}}}.\end{array}$$

Naturally, it is reasonable to choose $m^{(i)}$ in such a way that the exponential rate ${({\mathit{\lambda }}^{(i)})}^{\ast }$ is the same for all i, leading to the restriction $M=cS$ with $c>0$. Choosing c small will result in fast convergence to equilibrium, but also make the dynamics (69) quite stiff, requiring a very small timestep $\mathit{\Delta }t$ in a discretisation scheme. The choice of c will therefore need to strike a balance between those two competing effects. The constraint (71) then implies $\mathit{\Gamma }=2cS$. By a coordinate transformation, the preceding argument also applies if S, M and $\mathit{\Gamma }$ are diagonal in the same basis, and of course M and $\mathit{\Gamma }$ can always be chosen that way. Numerical experiments show that it is possible to increase the rate of convergence to equilibrium even further by choosing M and $\mathit{\Gamma }$ nondiagonally with respect to S (although only by a small margin). A clearer understanding of this is a topic of further investigation.

Numerical Experiments: Diffusion Bridge Sampling

Numerical Scheme

In this section we introduce a splitting scheme for simulating the perturbed underdamped Langevin dynamics given by Eq. (8). In the unpertubed case, i.e. when $J_1=J_2=0$, the BAOAB scheme (see [33] and references therein) has proven to be efficient for computing long time ergodic averages with respect to q-dependent observables. Motivated by this, we introduce the following perturbed scheme, introducing additional Runge-Kutta integration steps:

$$\begin{array}{cc}\hfill p_{n+1/2}=& p_n-\frac{1}{2}\mathit{\Delta }t\mathrm{\nabla }V(q_n),\hfill \end{array}$$ 72a

$$\begin{array}{cc}\hfill q_{n+1/2}=& q_n+\frac{1}{2}\mathit{\Delta }t·M^{-1}{p}_{n+1/2},\hfill \end{array}$$ 72b

$$\begin{array}{cc}\hfill q_{n+1/2}^{\prime }=& R{K}_4\left(\frac{1}{2}\mathit{\Delta }t,q_{n+1/2}\right),\hfill \end{array}$$ 72c

$$\begin{array}{cc}\hfill \widehat{p}=& exp(-\mathit{\Delta }t(\mathit{\Gamma }{M}^{-1}+\mathit{\nu }{J}_2{M}^{-1}))p_{n+1/2}+\sqrt{I-e^{-2\mathit{\Gamma }\mathit{\Delta }t}}\mathcal{N}(0,1),\hfill \end{array}$$ 72d

$$\begin{array}{cc}\hfill q_{n+1/2}^{\prime \prime }=& R{K}_4\left(\frac{1}{2}\mathit{\Delta }t,q_{n+1/2}^{\prime }\right),\hfill \end{array}$$ 72e

$$\begin{array}{cc}\hfill q_{n+1}=& q_{n+1/2}^{\prime \prime }+\frac{1}{2}\mathit{\Delta }t·M^{-1}\widehat{p},\hfill \end{array}$$ 72f

$$\begin{array}{cc}\hfill p_{n+1}=& \widehat{p}-\frac{1}{2}\mathit{\Delta }t·\mathrm{\nabla }V(q_{n+1}),\hfill \end{array}$$ 72g

where $R{K}_4(\mathit{\Delta }t,q_0)$ refers to fourth order Runge-Kutta integration of the ODE

$$\begin{array}{c}\hfill \dot{q}=-J_1\mathrm{\nabla }V(q),q(0)=q_0\end{array}$$ 73

up until time $\mathit{\Delta }t$. We remark that the $J_2$-perturbation is linear and can therefore be included in the O-part without much computational overhead. We emphasize the fact that many different splitting schemes could be investigated: although the BAOAB-scheme works well for unperturbed Langevin dynamics, it is not clear whether this remains true for the perturbed dynamics. Moreover, the perturbations introduced by $J_1$ and $J_2$ can be added in various places. Other discretisation schemes for the ODE (73) could be useful as well, for instance one could use a symplectic integrator, using the Hamiltonian structure of (73). However, since V as the Hamiltonian for (73) is not separable in general, such a symplectic integrator would have to be implicit. Note that (72c) and (72e) could be merged since (72e) commutes with (72d). In this paper, we content ourselves with the above scheme for our numerical experiments. Investigation of optimal numerical schemes for perturbed Langevin dynamics is an interesting problem for further research.

Remark 15

The aformentioned schemes lead to an error in the approximation for $\mathit{\pi }(f)$, since the invariant measure $\mathit{\pi }$ is not preserved exactly by the numerical scheme. In practice, the BAOAB-scheme can therefore be accompanied by an accept-reject Metropolis step as in [40], leading to an unbiased estimate of $\mathit{\pi }(f)$, albeit with an inflated variance. In this case, after every rejection the momentum variable has to be flipped ($p\mapsto -p$) in order to keep the correct invariant measure. We note here that our perturbed scheme can be ’Metropolized’ in a similar way by ’flipping’ the matrices $J_1$ and $J_2$ after every rejection ($J_1\mapsto -J_1$ and $J_2\mapsto -J_2)$ and using an appropriate (volume-preserving and time-reversible) integrator for the dynamics given by (73). Implementations of this idea are the subject of ongoing work. See [47] for a similar approach to nonreversible overdamped Langevin dynamics.

Diffusion Bridge Sampling

To numerically test our analytical results, we will apply the dynamics (8) to sample a measure on path space associated to a diffusion bridge. Specifically, consider the SDE

$$\begin{array}{c}\hfill \mathrm{d}{X}_s=-\mathrm{\nabla }U(X_s)\mathrm{d}s+\sqrt{2{\mathit{\beta }}^{-1}}\mathrm{d}{W}_s,\end{array}$$

with $X_s\in {\mathbb{R}}^n$, $\mathit{\beta }>0$ and the potential $U:{\mathbb{R}}^n\to \mathbb{R}$ obeying adequate growth and smoothness conditions (see [24], Sect. 5 for precise statements). The law of the solution to this SDE conditioned on the events $X(0)=x_{-}$ and $X(s_{+})=x_{+}$ is a probability measure $\mathit{\pi }$ on $L^2([0,s_{+}],{\mathbb{R}}^n)$ which poses a challenging and important sampling problem, especially if U is multimodal. This setting has been used as a test case for sampling probability measures in high dimensions (see for example [9] and [45]). For a more detailed introduction (including applications) see [11] and for a rigorous theoretical treatment the papers [11, 24–26].

In the case $U\equiv 0$, it can be shown that the law of the conditioned process is given by a Gaussian measure ${\mathit{\pi }}_0$ with mean zero and precision operator $\mathcal{S}=-\frac{\mathit{\beta }}{2}\mathit{\Delta }$ on the Sobolev space $H^1([0,s_{+}],{\mathbb{R}}^d)$ equipped with appropriate boundary conditions. The general case can then be understood as a perturbation thereof: The measure $\mathit{\pi }$ is absolutely continuous with respect to ${\mathit{\pi }}_0$ with Radon-Nikodym derivative

$$\begin{array}{c}\hfill \frac{\mathrm{d}\mathit{\pi }}{\mathrm{d}{\mathit{\pi }}_0}\propto exp(-\mathit{\Psi }),\end{array}$$ 74

where

$$\begin{array}{c}\hfill \mathit{\Psi }(x)=\frac{\mathit{\beta }}{2}{\int }_0^{s_{+}}G(x(s),\mathit{\beta })\mathrm{d}s\text{and}G(x,\mathit{\beta })=\frac{1}{2}{|\mathrm{\nabla }U(x)|}^2-\frac{1}{\mathit{\beta }}\mathit{\Delta }U(x).\end{array}$$

We will make the choice $x_{-}=x_{+}=0$, which is possible without loss of generality as explained in [10, Remark 3.1], leading to Dirichlet boundary conditions on $[0,s_{+}]$ for the precision operator $\mathcal{S}$. Furthermore, we choose $s_{+}=1$ and discretise the ensuing s-interval [0, 1] according to

$$\begin{array}{c}\hfill [0,1]=[0,s_1)\cup [s_1,s_2)\cup \dots \cup [s_{n-1},s_n)\cup [s_n,1]\end{array}$$

in an equidistant way with stespize $s_{j+1}-s_j\equiv \mathit{\delta }=\frac{1}{d+1}$. Functions on this grid are determined by the values $x(s_1)=x_1,\dots ,x(s_n)=x_n$, recalling that $x(0)=x(1)=0$ by the Dirichlet boundary conditions. We discretise the functional $\mathit{\Psi }$ as

$$\begin{array}{c}\hfill \tilde{\mathit{\Psi }}(x_1,\dots ,x_n)=\frac{\mathit{\beta }}{2}\mathit{\delta }\sum _{i=1}^{d}G(x_i,\mathit{\beta })=\frac{\mathit{\beta }}{2}\mathit{\delta }\sum _{i=1}^d(\frac{1}{2}(U^{\prime }{(x_i)}^2-\frac{1}{\mathit{\beta }}{U}^{\prime \prime }(x_i)),\end{array}$$

such that its gradient is given by

$$\begin{array}{c}\hfill {(\mathrm{\nabla }\tilde{\mathit{\Psi }})}_i=\frac{\mathit{\beta }}{2}\mathit{\delta }(2U^{\prime }(x_i)U^{\prime \prime }(x_i)-\frac{1}{\mathit{\beta }}{U}^{\prime \prime \prime }(x_i)),i=1,\dots ,d.\end{array}$$

We denote by $A_{\mathit{\delta }}$ the discretised Dirichlet Laplacian on [0, 1] with stepsize $\mathit{\delta }$. Following (74), the discretised target measure $\widehat{\mathit{\pi }}$ has the form

$$\begin{array}{c}\hfill \widehat{\mathit{\pi }}=\frac{1}{Z}{e}^{-V}\mathrm{d}x\text{with}V(x)=\tilde{\mathit{\Psi }}(x)-\frac{\mathit{\beta }\mathit{\delta }}{4}x·A_{\mathit{\delta }}x,x\in {\mathbb{R}}^d.\end{array}$$

In the following we will consider the case $n=1$ with potential $U:\mathbb{R}\to \mathbb{R}$ given by $U(x)=\frac{1}{2}{(x^2-1)}^2$ and set $\mathit{\beta }=1$. To test our algorithm we adjust the parameters M, $\mathit{\Gamma }$, $J_1$ and $J_2$ according to the recommended choice in the Gaussian case, (27), where we take $S=\frac{\mathit{\beta }}{2}\mathit{\delta }·A_{\mathit{\delta }}$ as the precision operator of the Gaussian target. We will consider the linear observable $f_1(x)=l·x$ with $l=(1,\dots ,1)$ and the quadratic observable $f_2(x)={|x|}^2$. In a first experiment we adjust the perturbations $J_1$ and $J_2$ to the observable $f_2$ according to Algorithm 2. The dynamics (8) is integrated using the splitting scheme introduced in Sect. 5.1 with a stepsize of $\mathit{\Delta }t={10}^{-4}$ over the time interval [0, T] with $T={10}^2$. Furthermore, we choose initial conditions $q_0=(1,\dots ,1)$, $p_0=(0,\dots ,0)$ and introduce a burn-in time $T_0=1$, i.e. we take the estimator to be $\widehat{\mathit{\pi }}(f)\approx \frac{1}{T-T_0}{\int }_{T_0}^{T}f(q_t)\mathrm{d}t.$ We compute the variance of the above estimator from $N=500$ realisations and compare the results for different choices of the friction coefficient $\mathit{\gamma }$ and of the perturbation strength $\mathit{\mu }$.

The numerical experiments show that the perturbed dynamics generally outperform the unperturbed dynamics independently of the choice of $\mathit{\mu }$ and $\mathit{\gamma }$, both for linear and quadratic observables. One notable exception is the behaviour of the linear observable for small friction $\mathit{\gamma }={10}^{-3}$ (see Fig. 4a), where the asymptotic variance initially increases for small perturbation strengths $\mathit{\mu }$. This does not contradict our analytical results, since the small perturbation results Theorem 3 and Corollary 3 are only valid if $\mathit{\gamma }>\sqrt{2}$. We remark here that the condition $\mathit{\gamma }>\sqrt{2}$, while necessary for the theoretical results from Sect. 4.1, is not a very advisable choice in practice (at least in this experiment), since Figs. 4b and 5b clearly indicate that the optimal friction is around $\mathit{\gamma }\approx {10}^{-1}$. Interestingly, the problem of choosing a suitable value for the friction coefficient coefficient $\mathit{\gamma }$ becomes mitigated by the introduction of the perturbation: While the performance of the unperturbed sampler depends quite sensitively on $\mathit{\gamma }$, the asymptotic variance of the perturbed dynamics is a lot more stable with respect to variations of $\mathit{\gamma }$. A somewhat surprising phenomenon is that the standard deviation $\mathit{\sigma }$ associated to the linear observable decays in the range $\mathit{\gamma }\in [10,100]$ for the unperturbed sampler (see Fig. 4b). We confirmed this behaviour by further numerical experiments and remark that as the target measure $\widehat{\mathit{\pi }}$ is fairly complicated, convexity of the function $\mathit{\gamma }\mapsto \mathit{\sigma }$ should not be expected.

Fig. 4.

Standard deviation of $\widehat{\mathit{\pi }}(f)$ for a linear observable as a function of friction $\mathit{\gamma }$ and perturbation strength $\mathit{\mu }$

Fig. 5.

Standard deviation of $\widehat{\mathit{\pi }}(f)$ for a quadratic observable as a function of friction $\mathit{\gamma }$ and perturbation strength $\mathit{\mu }$

In the regime of growing values of $\mathit{\mu }$, the experiments confirm the results from Sect. 4.3, i.e. the asymptotic variance approaches a limit that is smaller than the asymptotic variance of the unperturbed dynamics.

As a final remark we report our finding that the performance of the sampler for the linear observable is qualitatively independent of the choice of $J_1$ [as long as $J_2$ is adjusted according to (27)]. This result is in alignment with Propostion 3 which predicts good properties of the sampler for antisymmetric observables. In contrast to this, a judicious choice of $J_1$ is critical for quadratic observables. In particular, applying Algorithm 2 significantly improves the performance of the perturbed sampler in comparison to choosing $J_1$ arbitrarily.

Outlook and Future Work

A new family of Langevin samplers was introduced in this paper. These new SDE samplers consist of perturbations of the underdamped Langevin dynamics (that is known to be ergodic with respect to the canonical measure), where auxiliary drift terms in the equations for both the position and the momentum are added, in a way that the perturbed family of dynamics is ergodic with respect to the same (canonical) distribution. These new Langevin samplers were studied in detail for Gaussian target distributions where it was shown, using tools from spectral theory for differential operators, that an appropriate choice of the perturbations in the equations for the position and momentum can improve the performance of the Langvin sampler, at least in terms of reducing the asymptotic variance. The performance of the perturbed Langevin sampler to non-Gaussian target densities was tested numerically on the problem of diffusion bridge sampling.

The work presented in this paper can be improved and extended in several directions. First, a rigorous analysis of the new family of Langevin samplers for non-Gaussian target densities is needed. The analytical tools developed in [14] can be used as a starting point. Furthermore, the study of the actual computational cost and its minimization by an appropriate choice of the numerical scheme and of the perturbations in position and momentum would be of interest to practitioners. In addition, the analysis of our proposed samplers can be facilitated by using tools from symplectic and differential geometry. Finally, combining the new Langevin samplers with existing variance reduction techniques such as zero variance MCMC, preconditioning/Riemannian manifold MCMC can lead to sampling schemes that can be of interest to practitioners, in particular in molecular dynamics simulations. All these topics are currently under investigation.