Hamiltonian Monte Carlo acceleration using surrogate functions with random bases

Abstract

For big data analysis, high computational cost for Bayesian methods often limits their applications in practice. In recent years, there have been many attempts to improve computational efficiency of Bayesian inference. Here we propose an efficient and scalable computational technique for a state-of-the-art Markov chain Monte Carlo methods, namely, Hamiltonian Monte Carlo. The key idea is to explore and exploit the structure and regularity in parameter space for the underlying probabilistic model to construct an effective approximation of its geometric properties. To this end, we build a surrogate function to approximate the target distribution using properly chosen random bases and an efficient optimization process. The resulting method provides a flexible, scalable, and efficient sampling algorithm, which converges to the correct target distribution. We show that by choosing the basis functions and optimization process differently, our method can be related to other approaches for the construction of surrogate functions such as generalized additive models or Gaussian process models. Experiments based on simulated and real data show that our approach leads to substantially more efficient sampling algorithms compared to existing state-of-the-art methods.

1 Introduction

Bayesian Statistics has provided a principled and robust framework to create many important and powerful data analysis methods over the past several decades. Given a probabilistic model for the underlying mechanism of observed data, Bayesian methods properly quantify uncertainty and reveal the landscape or global structure of the parameter space. However, these methods tend to be computationally intensive since Bayesian inference usually requires the use of Markov chain Monte Carlo (MCMC) algorithms to simulate samples from intractable distributions. Although the simple Metropolis et al. algorithm (1953) is often effective at exploring low-dimensional distributions, it can be very inefficient for complex, high-dimensional distributions: successive states may exhibit high autocorrelation, due to the random walk nature of the movement. As a result, the effective sample size (ESS) tends to be quite low and the convergence to the true distribution is usually very slow. The celebrated Hamiltonian Monte Carlo (HMC; Duane et al. 1987; Neal 2011) reduces the random walk behavior of Metropolis by Hamiltonian dynamics, which uses gradient information to propose states that are distant from the current state, but nevertheless have a high probability of acceptance.

Although HMC explores the parameter space more efficiently than random walk Metropolis does, it does not fully exploit the structure (i.e., geometric properties) of parameter space (Girolami and Calderhead 2011) since dynamics are defined over Euclidean space. To address this issue, Girolami and Calderhead (2011) proposed a new method, called Riemannian manifold HMC (RMHMC), that uses the Riemannian geometry of the parameter space (Amari and Nagaoka 2000) to improve standard HMC’s efficiency by automatically adapting to local structures.

To make such geometrically motivated methods practical for big data analysis, one needs to combine them with efficient and scalable computational techniques. A common bottleneck for using such sampling algorithms for big data analysis is repetitive evaluations of functions, their derivatives, geometric and statistical quantities that involve the whole observed data and maybe a complicated model. A natural question is how to construct effective approximation of these quantities that provides a good balance between accuracy and computation cost. One common approach is subsampling (see, for example, Welling and Teh 2011; Hoffman et al. 2010; Shahbaba et al. 2014; Chen et al. 2014), which restricts the computation to a subset of the observed data. This is based on the idea that big datasets contain a large amount of redundancy so the overall information can be retrieved from a small subset. However, in general applications, we cannot simply use random subsets for this purpose: the amount of information we lose as a result of random sampling leads to non-ignorable loss of accuracy, which in turn has a substantially negative impact on computational efficiency (Betancourt 2015). Therefore, in practice, it is a challenge to find good criteria and strategies for an appropriate and effective subsampling.

Another approach is to exploit smoothness or regularity in parameter space, which is true for most statistical models. This way, one could find computationally cheaper surrogate functions to substitute the expensive target (or potential energy) functions (Liu 2001; Rasmussen 2003; Welling and Teh 2011; Meeds and Welling 2014; Lan et al. 2014; Strathmann et al. 2015). However, the usefulness of these methods is often limited to moderate dimensional problems because of the computational cost needed to achieve desired approximation accuracy.

In this work, our objective is to develop a faster alternative to the method of Rasmussen (2003). To this end, we propose to use random non-linear bases along with efficient learning algorithms to construct a surrogate functions that provides effective approximation of the probabilistic model based on the collective behavior of the large data. The randomized non-linear basis functions combined with the computationally efficient learning process can incorporate correct criteria for an efficient implicit subsampling resulting in both flexible and scalable approximation (Huang et al. 2006a, b; Rahimi and Recht 2007, 2008). Because our method can be presented as a special case of shallow random networks implemented in HMC, we refer to it as random network surrogate function (RNS); however, we will show that our proposed method is related to (and can be extended to) other surrogate functions such as generalized additive models (GAMs) and Gaussian process models by constructing the surrogate functions using different bases and optimization process.

Our proposed method provides a natural framework to incorporate surrogate functions in the sampling algorithms such as HMC, and it can be easily extended to geometrically motivated methods such as RMHMC. Further, for problems with a limited time budget, we propose an adaptive version of our method that substantially reduces the required number of training points. This way, the random bases surrogate function could be utilized earlier and its approximation accuracy could be improved adaptively as more training points become available. We show that theoretically the learning procedure for our surrogate function is asymptotically equivalent to potential matching, which is itself a novel distribution matching strategy similar to the score matching method discussed in Hyvärinen (2005) and Strathmann et al. (2015).

Finally, we should emphasize that our method is used to generate high quality proposals at low computational cost. However, when calculating the acceptance probability of these proposals, we use the original Hamiltonian (used in standard HMC) to ensure that the stationary distribution of the Markov chain will remain the correct target distribution.

Our paper is organized as follows. An overview of HMC and RMHMC is given in Sect. 2. Our RNS–HMC is explained in detail in Sect. 3. The adaptive model is developed in Sect. 4. Experimental results based on simulated and real data are presented in Sect. 5. Code for these examples is available at github.com/chengzhang-uci/RNSHMC. Finally, Sect. 6 is devoted to discussion and future work. As an interesting supplement, an overall description of the potential matching procedure is presented in the Appendix 2.

2 Preliminaries

2.1 Hamiltonian Monte Carlo

In Bayesian Statistics, we are interested in sampling from the posterior distribution of the model parameters q given the observed data, Y = (y₁, y₂, …, y_N)^T,

$$P(q|Y)\propto \text{exp}(-U(q)),$$ (1)

where the potential energy function U is defined as

$$U(q)=-\sum _{i=1}^N\text{log }P(y_i|q)-\text{log }P(q).$$ (2)

The posterior distribution is almost always analytically intractable. Therefore, MCMC algorithms are typically used for sampling from the posterior distribution to perform statistical inference. As the number of parameters increases, however, simple methods such as random walk (Metropolis et al. 1953) and Gibbs sampling (Geman and Geman 1984) may require a long time to converge to the target distribution. Moreover, their explorations of parameter space become slow due to inefficient random walk proposal-generating mechanisms, especially when there exist strong dependencies between parameters in the target distribution. By inclusion of geometric information from the target distribution, HMC (Duane et al. 1987; Neal 2011) introduces a Hamiltonian dynamics system with auxiliary momentum variables p to propose samples of q in a Metropolis framework that explores the parameter space more efficiently compared to standard random walk proposals. More specifically, HMC generates proposals jointly for q and p using the following system of differential equations:

$$\frac{\mathrm{d}{q}_i}{\mathrm{d}t}=\frac{\partial H}{\partial p_i},$$ (3)

$$\frac{\mathrm{d}{p}_i}{\mathrm{d}t}=-\frac{\partial H}{\partial q_i},$$ (4)

where the Hamiltonian function is defined as $H(q,p)=U(q)+\frac{1}{2}{p}^{\mathrm{T}}{M}^{-1}p$. The quadratic kinetic energy function $K(p)=\frac{1}{2}{p}^{\mathrm{T}}{M}^{-1}p$ corresponds to the negative log-density of a zero-mean multivariate Gaussian distribution with the covariance M. Here, M is known as the mass matrix, which is often set to the identity matrix, I. Starting from the current state (q, p), the Hamiltonian dynamics system (3), (4) is simulated for L steps using the leapfrog method, with a stepsize of ε. The proposed state, (q*, p*), which is at the end of the trajectory, is accepted with probability min(1, exp[−H(q*, p*) + H(q, p)]). By simulating the Hamiltonian dynamics system together with the correction step, HMC generates samples from a joint distribution

$$P(q,p)\propto \text{exp }(-U(q)-\frac{1}{2}{p}^{\mathrm{T}}{M}^{-1}p).$$

Notice that q and p are separated, the marginal distribution of q then follows the target distribution. These steps are presented in Algorithm 1.

Following the dynamics of the assumed Hamiltonian system, HMC can generate distant proposals with high acceptance probability which allows an efficient exploration of parameter space.

2.2 Riemannian manifold HMC

Although HMC explores the target distribution more efficiently than random walk Metropolis, it does not fully exploit the geometric structures of the underlying probabilistic model since a flat metric (i.e., M = I) is used. Using more geometrically motivated methods could substantially improve sampling algorithms’ efficiency. Recently, Girolami and Calderhead (2011) proposed a new method, called RMHMC, that exploits the Riemannian geometry of the target distribution to improve standard HMC’s efficiency by automatically adapting to local structures. To this end, instead of the identity mass matrix commonly used in standard HMC, they use a position-specific mass matrix M = G(q). More specifically, they set G(q) to the Fisher information matrix, and define Hamiltonian as follows:

Algorithm 1.

Hamiltonian Monte Carlo

Input: Starting position q(1) and step size ε
for t = 1, 2, … do
	Resample momentum p
	p(t) ~ 𝒩(0, M), (q0, p0) = (q(t), p(t))
	Simulate discretization of Hamiltonian dynamics:
	for l = 1 to L do
		$p_{l-1}\leftarrow p_{l-1}-\frac{\epsilon }{2}\frac{\partial U}{\partial q}(q_{l-1})$
		ql ← ql−1 + εM−1 pl−1
		$p_l\leftarrow p_l-\frac{\epsilon }{2}\frac{\partial U}{\partial q}(q_l)$
	(q*, p*) = (qL, pL)
	Metropolis–Hasting correction:
	u ~ Uniform[0, 1]
	ρ = exp[H(q(t), p(t)) − H(q*, p*)]
	if u < min(1, ρ), then q(t+1) = q*

$$H(q,p)=U(q)+\frac{1}{2}\text{log det }G(q)+\frac{1}{2}{p}^{\mathrm{T}}G{(q)}^{-1}p=\varphi (q)+\frac{1}{2}{p}^{T}G{(q)}^{-1}p,$$ (5)

where $\varphi (q)≔U(q)+\frac{1}{2}\text{log det }G(q)$. Note that standard HMC is a special case of RMHMC with G(q) = I. Based on this dynamic, they propose the following HMC on Riemannian manifold:

$$\dot{q}={\nabla }_{p}H(q,p)=G{(q)}^{-1}p,$$ (6)

$$ṗ=-{\nabla }_{q}H(q,p)=-{\nabla }_q\varphi (q)+\frac{1}{2}\nu (q,p),$$

where the ith element of the vector ν(q, p) is

$${(\nu (q,p))}_i=-p^{\mathrm{T}}{\partial }_i\left(G{(q)}^{-1}\right)$$

$$p={\left(G{(q)}^{-1}p\right)}^{\mathrm{T}}{\partial }_{i}G(q)G{(q)}^{-1}p,$$

with the shorthand notation ∂_i = ∂/∂q_i for partial derivative.

The above dynamic is non-separable (it contains products of q and p), and the resulting proposal-generating mechanism based on the standard leapfrog method is neither time reversible nor symplectic. Therefore, the standard leapfrog algorithm cannot be used for the above dynamic (Girolami and Calderhead 2011). Instead, we can use the Stömer–Verlet (1967) method, known as generalized leapfrog (Leimkuler and Reich 2004),

$$p^{(t+1/2)}=p^{(t)}-\frac{\epsilon }{2}\left[{\nabla }_q\varphi \right(q^{(t)})-\frac{1}{2}\nu (q^{(t)},p^{(t+1/2)}\left)\right],$$ (7)

$$q^{(t+1)}=q^{(t)}+\frac{\epsilon }{2}\left[G^{-1}\right(q^{(t)})+G^{-1}(q^{(t+1)}\left)\right]p^{(t+1/2)},$$ (8)

$$p^{(t+1)}=p^{(t+1/2)}-\frac{\epsilon }{2}\left[{\nabla }_q\varphi \right(q^{(t+1)})-\frac{1}{2}\nu (q^{(t+1)},p^{(t+1/2)}\left)\right].$$ (9)

The resulting map is (1) deterministic, (2) reversible, and (3) volume preserving. However, it requires solving two computationally intensive implicit equations [Eqs. (7) and (8)] at each leapfrog step.

3 Random network surrogate–HMC (RNS–HMC)

For HMC, the Hamiltonian dynamics contains the information from the target distribution through the potential energy U and its gradient. For RMHMC, more geometric structure (i.e., the Fisher information) is included through the mass matrix for kinetic energy. It is the inclusion of such information in the Hamiltonian dynamics that allows HMC and RMHMC to improve upon random walk Metropolis. However, one common computational bottleneck for HMC and other Bayesian models for big data is repetitive evaluations of functions, their derivatives, geometric and statistical quantities. Typically, each evaluation involves the whole observed data. For example, one has to compute the potential U and its gradient from the Eq. (2) for HMC and mass matrix M, its inverse and the involved partial derivatives for RMHMC at every time step or MH correction step. When N is large, this can be extremely expensive to compute. In some problems, each evaluation may involve solving a computationally expensive problem. (See the inverse problem and Remark in Sect. 5.2.)

To alleviate this issue, in recent years several methods have been proposed to construct surrogate Hamiltonians. For relatively low-dimensional spaces (sparse) grid-based piecewise interpolative approximation using precomputing strategy was developed in Welling and Teh (2011). Such grid-based methods, however, are difficult to extend to high-dimensional spaces due to the use of structured grids. Alternatively, we can use Gaussian process model, which are commonly used as surrogate models for emulating expensive-to-evaluate functions, to learn the target functions from early evaluations (training data; Rasmussen 2003; Lan et al. 2014; Meeds and Welling 2014). However, naive (but commonly used) implementations of Gaussian process models have high computation cost associated with inverting the covariance matrix, which grows cubically as the size of the training set increases. This is especially crucial in high-dimensional spaces, where we need large training sets in order to achieve a reasonable level of accuracy. Recently, scalable Gaussian processes using inducing point methods (Snelson and Ghahramani 2006; Quinonero-Candela and Rasmussen 2005) have been introduced to scale up GPs to larger datasets. While these methods have been quite successful in reducing computation cost, the tuning of inducing points could still be problematic in high-dimensional spaces. (See a more detailed discussion in Sect. 3.3.)

The key in developing surrogate functions is to develop a method that can effectively capture the collective properties of large datasets with scalability, flexibility, and efficiency. In this work, we propose to construct surrogate functions using proper random non-linear bases and efficient optimization process on training data. More specifically, we present our method as a special case of shallow neural networks; although, we show that it is related to (and can be extended to) other surrogate functions such as GAMs and Gaussian process models. Random networks of non-linear functions prove capable of approximating a rich class of functions arbitrarily well (Huang et al. 2006b; Rahimi and Recht 2008). Using random non-linear networks and algebraic learning algorithms can also be viewed as an effective implicit subsampling with desired criteria. The choice of hidden units (basis functions) and the fast learning process can be easily adapted to be problem specific and scalable. Unlike typical (naive) Gaussian process models, our random network scales linearly with the number of training points. In fact, a random non-linear network can be considered as a standard regression model with randomly mapped features. For such shallow random networks, the computational cost for inference is cubic in the number of hidden nodes. Those differences in scaling allow us to explicitly trade off computational efficiency and approximation accuracy and construct more efficient surrogate in certain applications. As our empirical results suggest, with appropriate training data good approximation of smooth functions in high-dimensional space can be achieved using a moderate and scalable number of hidden units. Therefore, our proposed method has the potential to scale up to large data sizes and provide effective and scalable surrogate Hamiltonians that balance accuracy and efficiency well.

3.1 Shallow random network approximation

A typical shallow network architecture (i.e., a single-hidden layer feedforward scalar-output neural network) with s hidden units, a non-linear activation function a, and a scalar (for simplicity) output z for a given d-dimensional input q is defined as

$$z(q)=\sum _{i=1}^s{\upsilon }_{i}a(q;{\gamma }_i)+b,$$ (10)

where γ_i is the ith hidden node parameter, υ_i is the output weight for the ith hidden node, and b is the output bias. Given a training dataset

$$𝒯=\left\{\right(q^{(j)},t^{(j)})|q^{(j)}\in {ℝ}^d,t^{(j)}\in ℝ,j=1,\dots ,N\},$$

the neural network can be trained by finding the optimal model parameters W = {γ_i, υ_i, i = 1, …, s, b} to minimize the mean square error cost function,

$$C(\mathit{W}|𝒯)=\frac{1}{N}\sum _{j=1}^N{\Vert z\left(q^{(j)}\right)-t^{(j)}\Vert }^2.$$ (11)

The most popular algorithm in machine learning to optimize (11) is backpropagation (Rumelhart et al. 1986). However, as a gradient descent-based iterative algorithm, backpropagation is usually quite slow and can be trapped at some local minimum since the cost function is non-linear, and for most cases, non-convex. Motivated by the fact that randomization is computationally cheaper than optimization, alternative methods based on random non-linear bases have been proposed (Huang et al. 2006a; Rahimi and Recht 2007). These methods drastically decrease the computation cost while maintaining a reasonable level of approximation accuracy. The key feature of random networks is that they reduce the full optimization problem into standard linear regression by mapping the input data to a randomized feature space and then apply existing fast algebraic training methods (e.g., by minimizing squared error) to find the output weight. Given the design objective, algebraic training can achieve exact or approximate matching of the data at the training points. Compared to the gradient descent-based techniques, algebraic training methods can reduce computational complexity and provide better generalization properties. A typical algebraic approach for single-hidden layer feedforward random networks is extreme learning machine (ELM; Huang et al. 2006a), which is summarized in Algorithm 2. Using randomized non-linear features, ELM estimates the output weight by finding the least squares solution to the resulting linear equations system Hυ = T. Note that presented this way, our method can also be regarded as a random version of GAM. In practice, people could add regularization to improve stability and generalizability.

3.2 Choice of non-linearity

There are many choices for non-linear activation functions in random networks. Different types of activation functions can be used for different learning tasks. In this paper, we focus on random networks with two typical types of non-linear nodes:

Algorithm 2.

Extreme Learning Machine

Given a training set

𝒯 = {(Ij, tj)|Ij ∈ ℝd, tj ∈ ℝm, j = 1, …, N}, activation

function a(x; γ) and hidden node number s

Step 1: Randomly assign hidden node parameters

γi, i = 1, …, s

Step 2: Calculate the hidden layer output matrix H

Hji = a(Ij; γi), i = 1, …, s, j = 1, …, N

Step 3: Calculate the output weight υ

υ = H†T, T = [t1, t2, …, tN]T,

where H† is the Moore–Penrose generalized inverse of matrix H

• Additive nodes

  $$a(q;\gamma )=a(\mathit{w}·q+d),\mathit{w}\in {ℝ}^d,d\in ℝ,\gamma =\{\mathit{w},d\},$$

  where w and d are the weight vector and the bias of the hidden node.
• Radial basis functions (RBFs) nodes

  $$a(q;\gamma )=a(-\frac{{\Vert q-c\Vert }^2}{2{\ell }^2}),c\in {ℝ}^d,\ell \in {ℝ}^{+},\gamma =\{c,\ell \},$$

  where c and ℓ are the center and width of the hidden node.

Both random networks can approximate a rich class of functions arbitrarily well (Huang et al. 2006b; Rahimi and Recht 2008). With randomly assigned input weights and biases composed linearly inside the non-linear activation function, additive nodes form a set of basis functions, whose level sets are hyperplanes orientated by w_i and shifted by d_i, respectively. Random networks with additive nodes tend to reflect the global structure of the target function. On the other hand, RBF nodes are almost compactly supported (can be adjusted by the width ℓ) rendering good local approximation for the corresponding random networks.

3.3 Connection to GPs and sparse GPs

It is worth noting the connection between networks with RBF nodes and Gaussian processes models (Rasmussen 1996; Neal 1999). Given a training dataset

$$𝒯=\left\{\right(q^{(j)},t^{(j)})|q^{(j)}\in {ℝ}^d,t^{(j)}\in ℝ,j=1,\dots ,N\},$$

and using squared-exponential covariance function

$$K(q^{(j)},q^{(j\prime )})={\sigma }_{\mathrm{f}}^2\text{exp}(-\frac{{\Vert q^{(j)}-q^{(j\prime )}\Vert }^2}{2{\ell }^2}),\theta =\{{\sigma }_{\mathrm{f}},\ell \},$$

the standard GP regression with a Gaussian noise model has the following marginal likelihood

$$p(\mathit{t}|\mathit{Q},\theta )=𝒩(\mathit{t}|\mathbf{0},{\mathit{K}}_N+{\sigma }^2\mathit{I}),$$

where $\mathit{Q}={\{q^{(j)}\}}_{j=1}^N,\mathit{t}={\{t^{(j)}\}}_{j=1}^N,{[{\mathit{K}}_N]}_{jj\prime }=K(q^{(j)},q^{(j\prime )})$ is the covariance matrix and σ is the noise parameter. Prediction on a new observation q* is made according to the conditional distribution

$$p(t^{*}|q^{*},𝒯,\theta )=𝒩{(t^{*}|{\mathit{k}}_{*}^{\mathrm{T}}({\mathit{K}}_N+{\sigma }^2\mathit{I})}^{-1}\mathit{t},K_{**}-{\mathit{k}}_{*}^{\mathrm{T}}{({\mathit{K}}_N+{\sigma }^2\mathit{I})}^{-1}{\mathit{k}}_{*}+{\sigma }^2),$$

where [k_*]_j = K(q^(j), q*) and K_** = K(q*, q*).

On the other hand, if we use K(q^(j), ·) as the j th hidden node, j = 1, 2, …, N, the output matrix becomes H = K_N, and the output weight learned by algebraic approach to a regularized least square problem is

$$\hat{\upsilon }=\text{arg }\underset{\upsilon }{\text{min}}{\Vert \mathit{H}\upsilon -\mathit{t}\Vert }^2+{\sigma }^2{\upsilon }^{\mathrm{T}}{\mathit{K}}_N\upsilon ={({\mathit{K}}_N+{\sigma }^2\mathit{I})}^{-1}\mathit{t}.$$

Therefore, such a network provides the same prediction point estimate as the above full GP models. This way, a Gaussian process model can be interpreted as a self-organizing RBF network where new hidden nodes are added adaptively with new observations. This is also an alternative point of view to Neal (1996) where GP models were shown to be equivalent to single-hidden layer neural networks with infinite many hidden nodes.

Notice that the above GP model scales cubically with the number of data points N which limits the application of GPs to relatively small datasets. To derive a GP model that is computationally tractable for large datasets, sparse GPs based on inducing point methods (Snelson and Ghahramani 2006; Quinonero-Candela and Rasmussen 2005) have been previously proposed. These sparse models introduce a set of inducing points $\overline{\mathit{Q}}={\{{\overline{q}}^{(i)}\}}_{i=1}^M$ and approximate the exact kernel K(q, q′) by an approximation K̃(q, q′) for fast computation. For example, the fully independent training conditional (FITC; Snelson and Ghahramani 2006) method uses the approximate kernel

$${\tilde{K}}_{\mathrm{F}\mathrm{I}\mathrm{T}\mathrm{C}}(q,q\prime )={\mathit{k}}_q^{\mathrm{T}}{\mathit{K}}_M^{-1}{\mathit{k}}_{q\prime }+{\delta }_{qq\prime }(K(q,q\prime )-{\mathit{k}}_q^{\mathrm{T}}{\mathit{K}}_M^{-1}{\mathit{k}}_{q\prime }),$$

where K_M is the exact covariance matrix for the M inducing points and k_q, k_q′ are the exact covariance matrices between q, q′ and the inducing points. Given the same training dataset 𝒯, the marginal likelihood is

$$p(\mathit{t}|\mathit{Q},\theta )=𝒩(\mathit{t}|\mathbf{0},{\mathit{K}}_{NM}{\mathit{K}}_M^{-1}{\mathit{K}}_{MN}+\mathit{\Lambda }+{\sigma }^2\mathit{I}).$$

Here, Λ is a diagonal matrix with ${\mathit{\Lambda }}_{jj}=K_{jj}-{\mathit{k}}_j^{\mathrm{T}}{\mathit{K}}_M^{-1}{\mathit{k}}_j$ that adjusts the diagonal elements to the ones from the exact covariance matrix K_N. Similarly, predictions can be made according to the conditional distribution

$$p(t^{*}|q^{*},𝒯,\theta )=𝒩(t^{*}|{\mathit{k}}_{*}^{\mathrm{T}}{\mathit{\Sigma }}_M^{-1}{\mathit{K}}_{MN}{(\mathit{\Lambda }+{\sigma }^2\mathit{I})}^{-1}\mathit{t},K_{**}-{\mathit{k}}_{*}^{\mathrm{T}}({\mathit{K}}_M^{-1}-{\mathit{\Sigma }}_M^{-1}){\mathit{k}}_{*}+{\sigma }^2),$$

where Σ_M = K_M + K_MN(Λ + σ²I)⁻¹K_NM. The computation cost is reduced to 𝒪(M²N) for learning and after that the predictive mean costs 𝒪(M) per new observation. The hyperparameters θ, σ and inducing points Q̅ can be tuned to maximize the marginal likelihood. However, in high dimension the tuning of Q̅ becomes infeasible.

On the other hand, if we use the inducing points Q̅ as the centers of an RBF network, the output matrix H = K_NM. Given the diagonal matrix D = Λ + σ²I, the output weight estimated by the algebraic approach to a weighted least square problem plus a regularization term is

$$\hat{\upsilon }=\text{arg }\underset{\upsilon }{\text{min}}{\Vert {\mathit{D}}^{-\frac{1}{2}}(\mathit{H}\upsilon -\mathit{t})\Vert }^2+{\upsilon }^{\mathrm{T}}{\mathit{K}}_M\upsilon ={\mathit{\Sigma }}_M^{-1}{\mathit{K}}_{MN}{(\mathit{\Lambda }+{\sigma }^2\mathit{I})}^{-1}\mathit{t}.$$

Therefore, the same predictive mean can be reproduced if we use the inducing points as centers and use the same hyperparameter configuration in our random network with RBF nodes and optimize with respect to the above cost function. Moreover, those hyperparameters in each basis (or hidden nodes) can also be trained (typically by gradient decent) if we abandon the use of random bases and simple linear regression.

Figure 1 compares random networks with different node types and related GP methods on fitting a simple function y = x²/2 which corresponds to the negative log-density of the standard normal distributions. We used softplus function σ(x) = log(1 + exp(x)) in additive nodes and exponential square kernels in RBF nodes and GP methods. As we can see from the graph, random networks generally perform better than GP methods when the number of observations is small. The randomness in the configuration of hidden nodes force networks to learn more globally. In contrast, GP models are more local and need more data to generalize well. By introducing sparsity, FITC tends to generalize better than full GP, especially on small datasizes. Since our goal is to fit negative log-posterior density function in (2) and U(q) → ∞ as q moves away from the high density domain, using softplus basis functions in random networks are more capable to capture this far field feature by striking a better balance between flexibility and generalization while being less demanding on the datasize. Also, the number of hidden neurons (bases) can be used to regularize the approximation to mitigate overfitting issue.

Fig. 1.

Comparing different surrogate approximations with an increasing number of observations N = 10, 20, 40 on target function y = x²/2. The observation points are nested samples from the standard normal distribution. For FITC and random networks, we choose five inducing points and five hidden neurons, respectively. FITC and random networks are all run 100 times and averaged to reduce the random effects on the results

3.4 Surrogate-induced Hamiltonian flow

As mentioned in the previous sections, repetitive computation of Hamiltonian, its gradient and other quantities that involve all dataset undermine the overall exploration efficiency of HMC. To alleviate this issue, we exploit the smoothness or regularity in parameter space, which is true for most statistical models. As discussed in Neal (1996), Liu (2001), and Rasmussen (2003), one can improve computational efficiency of HMC by approximating the energy function and using the resulting approximation to device a surrogate transition mechanism while still converging to the correct target distribution. To this end, Rasmussen (2003) proposed to use pre-convergence samples (which are discarded during the burn-in period) to approximate the energy function using a Gaussian process model. Here, we define an alternative surrogate-induced Hamiltonian as follows:

$$\tilde{H}(q,p)=Ũ(q)+\frac{1}{2}{p}^{\mathrm{T}}{M}^{-1}p,$$

where Ũ(q) is the neural network surrogate function. H̃ (q, p) now defines a surrogate-induced Hamiltonian flow, parametrized by a trajectory length t, which is a map ϕ̃_t: (q, p) → (q*, p*). Here, (q*, p*) being the end-point of the trajectory governed by the following equations

$$\frac{\mathrm{d}q}{\mathrm{d}t}=\frac{\partial \tilde{H}}{\partial p}=M^{-1}p,\frac{\mathrm{d}p}{\mathrm{d}t}=-\frac{\partial \tilde{H}}{\partial q}=-\frac{\partial Ũ}{\partial q}.$$

When the original potential U(q) is computationally costly, simulating the surrogate-induced Hamiltonian system provides a more efficient proposing mechanism for our HMC sampler. The introduced bias along with discretization error from the leapfrog integrator are all naturally corrected in the MH step where we use the original Hamiltonian in the computation of acceptance probability. As a result, the stationary distribution of the Markov chain will remain the correct target distribution (see Appendix 1 for a detailed proof). Note that by controlling the approximation quality of the surrogate function, we can maintain a relatively high acceptance probability. This is illustrated in Fig. 2 for a two-dimensional banana-shaped distribution (Girolami and Calderhead 2011).

Fig. 2.

Comparing HMC and NNS–HMC based on a two-dimensional banana-shaped distribution. The left panel shows the gradient fields (force map) for the original Hamiltonian flow (red) and the surrogate-induced Hamiltonian flow (blue). The middle and right panel show the trajectories for HMC and NNS–HMC samplers. Both samplers start from the same point (red square) with same initial momentums. Blue points at the end of the trajectories are the proposals. The overall acceptance probability drops from 0.97 using HMC to 0.88 using NNS–HMC. (Color figure online)

To construct such a surrogate, the early evaluations of the target function during the early iterations of MCMC will be used as the training set based on which we can train a shallow random network using fast algebraic approaches, such as ELM (Algorithm 2). The gradient of the scalar output z [see (10)] for a network with additive hidden nodes, for example, then can be computed as

$$\frac{\partial z}{\partial q}=\sum _{i=1}^s{\upsilon }_{i}a\prime ({\mathit{w}}_i·q+d_i){\mathit{w}}_i,$$ (12)

which costs only 𝒪(s) computations. To balance the efficiency in computation and flexibility in approximation, and to reduce the possibility of overfitting, the number of hidden nodes s need to be small as long as a reasonable level of accuracy can be achieved. In practice, this can be done by monitoring the resulting acceptance rate using an initial chain.

Following Rasmussen (2003), we propose to run our method, henceforth called RNS–HMC (see Algorithm 3), in two phases: exploration phase and exploitation phase. During the exploration phase, we initialize the training dataset D with an empty set or some samples from the prior distribution of parameters. We then run the standard HMC algorithm for some iterations and collect information from the new states (i.e., accepted proposals). When we have sufficiently explored the high density domain in parameter space and collected enough training data (during the burn-in period), a shallow random network is trained based on the collected training set D to form a surrogate for the potential energy function. The surrogate function will be used to approximate the gradient information needed for HMC simulations later in the exploitation phase.

As an illustrative example, we compare the performance of different surrogate HMC methods on a challenging Gaussian target density in 32 dimensions (a lower dimensional case was used in Rasmussen 2003). The target density has 31 confined directions and a main direction that is 10 times wider, and all variables are correlated. Both the full GPs and FITC methods are implemented using GPML package (Rasmussen and Nickisch 2010). The results are presented in Fig. 3. Compared to the full GPs, FITC and random networks (with additive and RBF nodes) all scales linearly with the number of observations. Both RNSs can start with fewer training data. We also compare the efficiency of the surrogate induced Hamiltonian flows in terms of time-normalized mean ESSs. The efficiency of FITC and random networks all increases as the number of observations increase until no more approximation gain can be obtained (see the acceptance probability in the middle panel). However, the efficiency of full GP begin to drop before the model reaches its full capacity. That is because its predictive complexity also grows with the number of observations, which in turn diminishes the overall efficiency. Overall, the random network with additive nodes outperform other methods based on these example.

Fig. 3.

Comparing the efficiency of our random network surrogates and Gaussian process surrogates on a challenging 32 dimensional Gaussian target whose covariance matrix has an eigenvector (1, 1, …, 1)^T with a corresponding eigenvalue of 1.0, and all other eigenvalues are 0.01. We set the step size to keep the acceptance probability around 70% for HMC and use the same step size in all surrogate methods. For FITC and random networks, the number of inducing points and hidden neurons are all set to be 1000 to allow reasonably accurate approximation. We ran each algorithm ten times and plot the medians and 80% error bars

Our proposed method provides a natural framework to incorporate surrogate functions in HMC. Moreover, it can be easily extended to RMHMC. To this end, the Hessian matrix of the surrogate function can be used to construct a metric in parameter space and the third order derivatives can be used to simulate the corresponding modified Hamiltonian flow. We refer to this extended version of our method as RNS–RMHMC.

Note that the approximation quality of the neural network surrogate function depends on several factors including the

Algorithm 3.

Random Network Surrogate HMC

Input: Starting position q(1), step size ε and number of hidden         units s
Initialize the training dataset: D = ∅ or several random samples from the prior
for t = 1, 2, …, B do
	Resample momentum p
	p(t) ~ 𝒩(0, M), (q0, p0) = (q(t), p(t))
	Simulate discretization of Hamiltonian dynamics and propose (q*, p*)
	Metropolis–Hasting correction:
	u ~ Uniform[0, 1], ρ = exp[H(q(t), p(t)) − H(q*, p*)]
	if u < min(1, ρ), then
	q(t+1) = q*, D = D ∪ {(q*, U(q*))}
Train a neural network with s hidden units via ELM on D to form the surrogate function z
for t = B + 1, B + 2, … do
	Resample momentum p
	p(t) ~ 𝒩(0, M), (q0, p0) = (q(t), p(t))
	Simulate discretization of a new Hamiltonian dynamics using z:
	for l = 1 to L do
		$p_{l-1}\leftarrow p_{l-1}-\frac{\epsilon }{2}\frac{\partial z}{\partial q}(q_{l-1})$
		ql ← ql−1 + εM−1 pl−1
		$p_l\leftarrow p_l-\frac{\epsilon }{2}\frac{\partial z}{\partial q}(q_l)$
	(q*, p*) = (qL, pL)
	Metropolis–Hasting correction:
	u ~ Uniform[0, 1], ρ = exp[H(q(t), p(t)) − H(q*, p*)]
	if u < min(1, ρ), then q(t+1) = q*

dimension of parameter space, d, the number of hidden neurons, s and the training size, N. Here, we assume that N is sufficiently large enough, and investigate the efficiency of RNS–HMC in terms of its acceptance probability for different values of d and s based on a standard logistic regression model with simulated data. Similar to the results presented in Strathmann et al. (2015), Fig. 4 shows the acceptance rate (over 10 MCMC runs) as a function of d and s. For dimensions up to d ≈ 50, RNS–HMC maintains a relatively high acceptance probability with a shallow random network trained in a few seconds on a laptop computer. Appendix 2 provides a theoretical justification for our method.

Fig. 4.

Acceptance probability of surrogate-induced Hamiltonian flow on simulated logistic regression models for different number of parameters, d, and hidden neurons, s. The step size is chosen to keep the acceptance probability around 70% for HMC. Left acceptance probability as a function of s (x-axis) and d (y-axis). Middle acceptance probability as a function of s for a fixed dimension d = 32. Right acceptance probability as a function of d for a fixed s = 2000

4 Adaptive RNS–HMC

So far, we have assumed that the neural network model in our method is trained using a sufficiently large enough number of training points after waiting for an adequate number of iterations to allow the sampler explore the parameter space. This, however, could be very time consuming in practice: waiting for a long time to collect a large number of training points could undermine the benefits of using the surrogate Hamiltonian function.

Figure 5 shows the average acceptance probabilities (over 10 MCMC chains) as a function of the number of training points, N, and the number hidden neurons, s, on a simulated logistic regression model for a fixed number of parameters, d = 32. While it takes around 2000 training data points to fulfill the network’s capability and reach a high acceptance comparable to HMC, only 500 training points are enough to provide an acceptable surrogate Hamiltonian flow (around 0.1 acceptance probability). Therefore, we can start training the neural network surrogate earlier and adapting it as more training points become available. Although adapting a Markov chain based on its history may undermine its ergodicity and consequently its convergence to the target distribution (Andrieu and Thoms 2008), we can enforce a vanishing adaption rate a_t such that a_t → 0 and ${\sum }_{t=1}^{\infty }{a}_t=\infty$ and update the surrogate function with probability a_t at iteration t. By Theorem 5 of Roberts and Rosenthal (2007), the resulting algorithm is ergodic and converges to the right target distribution.

Fig. 5.

Acceptance probability of the surrogate-induced Hamiltonian flow based on a simulated logistic regression models with dimension d = 32. Left acceptance probability as a function of the number of hidden neurons s (x-axis) and the number of training points N (y-axis). Middle acceptance probability as a function of N for a fixed s = 1000. Right acceptance probability as a function of s for a fixed N = 1600

It is straightforward to adapt the neural network surrogate from the history of Markov chain. However, the estimator in Algorithm 2 costs 𝒪(Nds + Ns² + s³) computation and 𝒪(Ns) storage, where N is the number of training data (e.g., the number of rows in the output matrix H). As N increases, finding the right weight and bias for the output neuron becomes increasingly difficult. In 1960, Greville shows that H^† can be learned incrementally in real time as new data points become available. Based on Greville’s method, online and adaptive pseudoinverse solutions for updating ELM weights has been proposed in Schaik and Tapson (2015) which can be readily employed here to develop an adaptive version of RNS–HMC. To be more efficient, only the estimator is updated.

Proposition 1

Suppose the current output matrix is H_k and the target vector is T_k. At time k+1, a new sample q_k+1 and the target (potential) t_k+1 are collected. Denote the output vector from the hidden layer at q_k+1 as h_k+1. The adaptive updating formula for the empirical potential matching estimator is given by

$${\upsilon }_{k+1}={\upsilon }_k+(t_{k+1}-h_{k+1}^T{\upsilon }_k)b_{k+1},$$

where b_k+1 and auxiliary matrices Φ_k+1, Θ_k+1 are updated according to c_k+1 = Φ_kh_k+1.

1. c_k+1 = 0

   $$b_{k+1}=\frac{{\Theta }_k{h}_{k+1}}{1+h_{k+1}^{\mathrm{T}}{\Theta }_k{h}_{k+1}},{\Phi }_{k+1}={\Phi }_k,$$

   $${\Theta }_{k+1}={\Theta }_k-{\Theta }_k{h}_{k+1}{b}_{k+1}^{\mathrm{T}}.$$
2. c_k+1 ≠ 0

   $$b_{k+1}=\frac{c_{k+1}}{{\Vert c_{k+1}\Vert }^2},{\Phi }_{k+1}={\Phi }_k-{\Phi }_k{h}_{k+1}{b}_{k+1}^{\mathrm{T}},$$

   $${\Theta }_{k+1}={\Theta }_k-{\Theta }_k{h}_{k+1}{b}_{k+1}^{\mathrm{T}}+(1+h_{k+1}^{\mathrm{T}}{\Theta }_k{h}_{k+1})b_{k+1}{b}_{k+1}^{\mathrm{T}}-b_{k+1}{h}_{k+1}^{\mathrm{T}}{\Theta }_k^{\mathrm{T}}.$$

At time k, the estimator takes a one-off 𝒪(kds+ks²+s³) computation and 𝒪(s²) storage (only need to store Φ_k and Θ_k, not $H_k^{†}$). Starting at a previously computed solution ${\upsilon }_K=H_K^{†}{T}_K$, and two auxiliary matrices ${\Phi }_K=I-H_K^{\mathrm{T}}{(H_K^{†})}^{\mathrm{T}},{\Theta }_K=H_K^{†}{(H_K^{†})}^{\mathrm{T}}$, this adaptive updating costs 𝒪(ds + s²) computation and 𝒪(s²) storage, independent of the training data size k. Further details are provided in Appendix 3. We refer to this extended version of our method as adaptive RNS–HMC (ARNS–HMC).

5 Experiments

In this section, we use several experiments based on logistic regression models and inverse problem for elliptic partial differential equation (PDE) to compare our proposed methods to standard HMC and RMHMC in terms of sampling efficiency defined as time-normalized ESS. Given B MCMC

Algorithm 4.

ARNS–HMC

Input: Initial estimator υ0 and auxiliary matrices Φ0, Θ0,         adaption schedule at,         step size ε, number of hidden units s
Initialize the surrogate z0 = z(q, υ0)
for t = 0, 1, … do
	Resample momentum p
	p(t) ~ 𝒩(0, M), (q0, p0) = (q(t), p(t))
	Propose (q*, p*) by simulating discretization of Hamiltonian dynamics with $\frac{\partial U}{\partial q}=\frac{\partial z_t}{\partial q}$
	Metropolis–Hasting correction:
	u ~ Uniform[0, 1], ρ = exp[H(q(t), p(t)) − H(q*, p*)]
	if u < min(1, ρ), then q(t+1) = q*; else q(t+1) = q(t)
	update the estimator and auxiliary matrices to
	υt+1, Φt+1, Θt+1 using (q(t+1), U(q(t+1)))
	u ~ Uniform[0, 1],
	if u < at, then zt+1 = z(q, υt+1); else zt+1 = zt

samples for each parameter, $\mathrm{E}\mathrm{S}\mathrm{S}=B{[1+2{\Sigma }_{k=1}^K\gamma (k)]}^{-1}$, where ${\Sigma }_{k=1}^K\gamma (k)$ is the sum of K monotone sample autocorrelations (Geyer 1992). We use the minimum ESS over all parameters normalized by the CPU time, s (in seconds), as the overall measure of efficiency: min(ESS)/s. The corresponding step sizes ε and number of leapfrog steps L for HMC and RMHMC are chosen to make them stable and efficient (e.g., reasonably high acceptance probability and fast mixing). The same settings are used for our methods. Note that while the acceptance rates are similar in the first two examples, they drop a little bit for the last two examples, which is mainly due to the constraints we imposed on our surrogate functions. To prevent non-ergodicity and ensure high ESS for both HMC and RNS–HMC, we follow the suggestion by Neal (2011) to uniformly sample L from {1, …, L} in each iteration. The number of hidden nodes s in RNSs are not tuned too much and better results could be obtained by more careful tunings.

In what follows, we first compare different methods in terms of their time-normalized ESS after the burin-period. To this end, we collect 5000 samples after a reasonable large number of iterations (5000) of burn-in to make sure the chains have reached their stationary states. For our methods, the accepted proposals during the burn-in period after a short warming-up session (the first 1000 iterations) are used as a training set for a shallow random network. Later, we show the advantages of our adaptive algorithm.

5.1 Logistic regression model

As our first example, we compare HMC, RMHMC, RNS–HMC, and RNS–RMHMC using a simulation study based on a logistic regression model with 50 parameters and N = 10⁵ observations. The design matrix is $X=(\frac{1}{10}\mathbf{1},X_1)$ and true parameters β are uniformly sampled from [0, 1]⁵⁰, where $X_1~{𝒩}_{49}(\mathbf{0},\frac{1}{100}{I}_{49})$. The binary responses Y = (y₁, y₂, …, y_N)^T are sampled independently from Bernoulli distributions with probabilities $p_i=1/(1+\text{exp}(-x_i^{\mathrm{T}}\beta ))$. We assume β ~ 𝒩₅₀(0, 100I₅₀), and sample from the corresponding posterior distribution.

Notice that the potential energy function U is now a convex function, the Hessian matrix is positive semi-definite everywhere. Therefore, we use the Hessian matrix of the surrogate as a local metric in RNS–RMHMC. For HMC and RNS–HMC, we set the step size and leapfrog steps ε = 0.045, L = 6. For RMHMC and RNS–RMHMC, we set the step size and leapfrog steps ε = 0.54, L = 2.

To illustrate that our method indeed converges to the right target distribution, Fig. 6 provides the one- and two-dimensional posterior marginals of some selected parameters obtained by HMC and RNS–HMC. Table 1 compares the performance of the four algorithms. As we can see, RNS–HMC has substantially improved the sampling efficiency in terms of time-normalized min(ESS).

Fig. 6.

HMC versus RNS–HMC: comparing one- and two-dimensional posterior marginals of β₁, β₁₁, β₂₁, β₃₁, β₄₁ based on the logistic regression model with simulated data

Table 1.

Comparing the algorithms using logistic regression models and an elliptic PDE inverse problem

Experiments	Methods	AP	ESS	s/Iter	min(ESS)/s	spdup
LR (simulation)	HMC	0.76	(4351, 5000, 5000)	0.061	14.17	1
	RMHMC	0.80	(1182, 1496, 1655)	3.794	0.06	0.004
s = 2000	RNS–HMC	0.76	(4449, 4999, 5000)	0.007	123.56	8.72
	RNS–RMHMC	0.82	(1116, 1471, 1662)	0.103	2.17	0.15
LR (Bank Marketing)	HMC	0.70	(2005, 2454, 3368)	0.061	6.52	1
	RMHMC	0.92	(1769, 2128, 2428)	0.631	0.56	0.09
s = 1000	RNS–HMC	0.70	(1761, 2358, 3378)	0.007	52.22	8.01
	RNS–RMHMC	0.90	(1974, 2254, 2457)	0.027	14.41	2.21
LR (a9a 60 dimension)	HMC	0.72	(1996, 2959, 3564)	0.033	11.96	1
	RMHMC	0.82	(5000, 5000, 5000)	3.492	0.29	0.02
s = 2500	RNS–HMC	0.68	(1835, 2650, 3203)	0.005	81.80	6.84
	RNS–RMHMC	0.79	(4957, 5000, 5000)	0.370	2.68	0.22
Elliptic PDE	HMC	0.91	(4533, 5000, 5000)	0.775	1.17	1
	RMHMC	0.80	(5000, 5000, 5000)	4.388	0.23	0.20
s = 1000	RNS–HMC	0.75	(2306, 3034, 3516)	0.066	7.10	6.07
	RNS–RMHMC	0.66	(2126, 4052, 5000)	0.097	4.38	3.74

For each method, we provide the acceptance probability (AP), the CPU time (s) for each iteration and the time-normalized ESS

The speed up for the best method is identified in bold

Next, we apply our method to two real datasets: Bank Marketing and the a9a dataset Lin et al. (2008). The Bank Marketing dataset (40,197 observations and 24 features) is collected based on direct marketing campaigns of a Portuguese banking institution aiming at predicting if a client will subscribe to a term deposit (Moro et al. 2014). We set the step size and number of leapfrog steps ε = 0.012, L = 45 for HMC and RNS–HMC; ε = 0.4, L = 6 for RMHMC and RNS–RMHMC. The a9a dataset (32,561 features and 123 features) is complied from the UCI adult dataset (Bache and Lichman 2013) which has been used to determine whether a person makes over 50K a year. We reduce the number of features to 60 by random projection (increasing the dimension to 100 results in a substantial drop in the acceptance probability). We set the step size and number of leapfrog steps ε = 0.012, L = 10 for HMC and RNS–HMC; ε = 0.5, L = 4 for RMHMC and RNS–RMHMC. All datasets are normalized to have zero-mean and unit standard deviation. The priors are the same as before. The results for the two datasets are summarized in Table 1. As before, both RNS–HMC and RNS–RMHMC significantly outperform their counterpart algorithms.

5.2 Elliptic PDE inverse problem

Another computationally intensive model is the elliptic PDE inverse problem discussed in Dashti and Stuart (2011) and Conard et al. (2014). This classical inverse problem involves inference of the diffusion coefficient in an elliptic PDE which is usually used to model isothermal steady flow in porous media. Let c be the unknown diffusion coefficient and u be the pressure field, the forward model is governed by the elliptic PDE

$${\nabla }_{\mathit{x}}·(c(\mathit{x},\theta ){\nabla }_{\mathit{x}}u(\mathit{x},\theta ))=0,$$ (13)

where x = (x₁, x₂) ∈ [0, 1]² is the spatial coordinate. The boundary conditions are

$${u(\mathit{x},\theta )|}_{x_2=0}=x_1,{u(\mathit{x},\theta )|}_{x_2=1}=1-x_1,$$

$${\frac{\partial u(\mathit{x},0)}{\partial x_1}|}_{x_1=0}=0,{\frac{\partial u(\mathit{x},0)}{\partial x_1}|}_{x_1=1}=0.$$

In our numerical simulation, (13) is solved using standard continuous GFEM with bilinear basis functions on a uniform 30 × 30 quadrilateral mesh.

A log-Gaussian process prior is used for c(x) with mean zero and an isotropic squared-exponential covariance kernel:

$$C({\mathit{x}}_1,{\mathit{x}}_2)={\sigma }^2\text{exp}(-\frac{{\Vert {\mathit{x}}_1-{\mathit{x}}_2\Vert }_2^2}{2l^2}),$$

for which we set the variance σ² = 1 and the length scale l = 0.2. Now, the diffusivity field can be easily parameterized with a Karhunen–Loeve (K–L) expansion:

$$c(\mathit{x},\theta )\approx \text{exp}(\sum _{i=1}^d{\theta }_i\sqrt{{\lambda }_i}{\upsilon }_i(\mathit{x})),$$

where λ_i and υ_i(x) are the eigenvalues and eigenfunctions of the integral operator defined by the kernel C, and the parameter θ_i are endowed with independent standard normal priors, θ_i ~ 𝒩(0, 0.5²), which are the targets of inference. In particular, we truncate the K–L expansion at d = 20 modes and condition the corresponding mode weights on data. Data are generated by adding independent Gaussian noise to observations of the solution field on a uniform 11 × 11 grid covering the unit square.

$$y_j=u({\mathit{x}}_j,\theta )+{\epsilon }_j,{\epsilon }_j~𝒩(0,{0.1}^2),j=1,2,\dots ,N.$$

The number of leapfrog steps and step sizes are set to be L = 10, ε = 0.16 for both HMC and NNS–HMC. Note that the potential energy function is no longer convex; therefore, we can not construct a local metric from the Hessian matrix directly. However, the diagonal elements

$$\frac{{\partial }^{2}U}{\partial {\theta }_i^2}=\frac{1}{{\sigma }_{\theta }^2}+\sum _{j=1}^N\frac{1}{{\sigma }_y^2}{\left(\frac{\partial u_j}{\partial {\theta }_i}\right)}^2-\sum _{j=1}^N\frac{{\epsilon }_j}{{\sigma }_y^2}\frac{{\partial }^2{u}_j}{\partial {\theta }_i^2},$$

$${\sigma }_{\theta }=0.5,{\sigma }_y=0.1,i=1,2,\dots ,d,$$

are highly likely to be positive in that the deterministic part (first two terms) is always positive and the noise part (last term) tends to cancel out. The diagonals of the Hessian matrix of surrogate therefore induce an effective local metric which can be used in RNS–RMHMC. A comparison of the results of all algorithms are presented in Table 1. As before, RNS–HMC provides a substantial improvement in the sampling efficiency. For the RMHMC methods, we set L = 3, ε = 0.8. As seen in the table, RMHMC is less efficient than HMC mainly due to the slow computation speed. However, RNS–RMHMC improves RMHMC substantially and outperforms HMC. Although the metric induced by the diagonals of the Hessian matrix of surrogate may not be as effective as Fisher information, it is much cheaper to compute and provide a good approximation.

Remark

In addition to the usual computational bottleneck as in previous examples, e.g., large amount of data, there is another challenge on top of that for this example due to the complicated forward model. Instead of a simple explicit probabilistic model that prescribes the likelihood of data given the parameter of interest, a PDE (13) is involved in the probabilistic model. The evaluation of geometrical and statistical quantities, therefore, involves solving a PDE similar to (13) in each iteration of HMC and RMHMC. This is a preventive factor in practice. Using our methods based on neural network, surrogates provide a huge advantage. Numerical experiments show a gain of efficiency by more than 20 times. More improvement is expected as the amount of data increases.

5.3 Adaptive learning

Next, using the above four examples we show that ARNS–HMC can start with far fewer training points and quickly reach the same level of performance as that of RNS– HMC. Figure 7 shows that as the number of training points (from initial MCMC iterations) increases, ARNS–HMC fully achieves the network’s capability and reaches a comparable acceptance rate to that of HMC.

Fig. 7.

Median acceptance rate of ANNS–HMC along with the corresponding 90% interval (shaded area). The red line shows the average acceptance rate of standard HMC. (Color figure online)

We also compare ARNS–HMC to HMC and RNS–HMC in terms of the relative error of mean (REM) which is defined as ${\Vert \overline{q(t)}-E(q)\Vert }_2/{\Vert E(q)\Vert }_2$, where $\overline{q(t)}$ means sample mean up to time t. Figure 8 shows the results using the four examples discussed above. Note that before training the neural network models, both RNS–HMC and ARNS–HMC are simply standard HMC so the three algorithms have similar performance. As we can see, ARNS–HMC has the best overall performance: it tends to provide lower REM at early iterations. This could be useful if we have limited time budget to fit a model.

Fig. 8.

Relative error of mean as a function of running time

6 Discussion and future work

In this paper, we have proposed an efficient and scalable computational method for Bayesian inference by exploring and exploiting regularity of probability models in parameter space. Our method is based on training surrogate function of the potential energy after exploring the parameter space sufficiently well. For situations where it is not practical to wait for a thorough exploration of parameter space, we have proposed an adaptive version of our method that can start with fewer training points and can quickly reach its full potential.

As an example, we used random networks and efficient learning algorithms to construct effective surrogate functions. These random bases surrogate functions provide good approximations of collective information of the full dataset while striking a good balance between accuracy and computation cost for efficient computation. Random networks combined with the optimized learning process can provide flexibility, accuracy, and scalability. Note that in general the overall performance could be sensitive to the architecture of the random network. Our proposed RNS method scales differently than GP emulators because of the specific constraints we imposed on its architecture. As our experimental results show, this approach could improve the performance of HMC in some applications.

In its current form, our method is more effective in problems with costly likelihood and a moderate number of parameters. In spite of improvements we have made to standard HMC, dealing with high-dimensional and complex distributions still remains quite challenging. For multimodal distributions, for example, our method’s effectiveness largely depends on the quality of training samples. If these samples are collected from one mode only, the surrogate function will miss the remaining modes and the sampler might not be able to explore them (especially if they are isolated modes). A surrogate function based on Gaussian processes might have a better chance at finding these modes in the tails of the approximate distribution since it tends to go to zero gradually. To address this issue, we can utilize mode searching and mode exploring ideas such as those proposed by Ahn et al. (2013) and Lan et al. (2014). For constrained target distributions, we can employ the method of Lan et al. (2014) based on spherical HMC.

For HMC, gradient of the potential function is an important driving force in the Hamiltonian dynamics. Although accurate approximation of a well-sampled smooth function automatically leads to accurate approximation of its gradient, this is not the case when the sampling is not well distributed. For example, when dense and well-sampled training datasets are difficult to obtain in very high dimensions, one can incorporate the gradient information in the training process. In future, we will study more effective way to utilize this information in the training process. As a common practice in adaptive MCMC methods, one may also learn the mass matrix M adaptively together with the surrogate in ARNS–HMC.

Appendix 1: Convergence to the correct target distribution

In order to prove that the equilibrium distribution remains the same, it suffices to show that the detailed balance condition still holds. Denote θ = (p, q), θ′ = ϕ̃_t(θ). In the Metropolis–Hasting step, we use the original Hamiltonian to compute the acceptance probability

$$\alpha (\theta ,\theta \prime )=\text{min}(1,\text{ exp}[-H(\theta \prime )+H(\theta )]),$$

therefore,

$$\alpha (\theta ,\theta \prime )\mathbb{P}(\mathrm{d}\theta )=\alpha (\theta ,\theta \prime )\text{ exp}[-H(\theta )]\mathrm{d}\theta \stackrel{\theta ={\tilde{\varphi }}_t^{-1}(\theta \prime )}{=}\text{min}(\text{exp}[-H(\theta )],\text{ exp}[-H(\theta \prime )])\left|\frac{\mathrm{d}\theta }{\mathrm{d}\theta \prime }\right|\mathrm{d}\theta \prime =\alpha (\theta \prime ,\theta )\text{ exp}[-H(\theta \prime )]\mathrm{d}\theta \prime =\alpha (\theta \prime ,\theta )\mathbb{P}(\mathrm{d}\theta \prime ),$$

since $|\frac{\mathrm{d}\theta }{\mathrm{d}\theta \prime }|=1$ due to the volume conservation property of the surrogate-induced Hamiltonian flow ϕ̃_t. Now that we showed the detailed balance condition is satisfied, along with the reversibility of the surrogate-induced Hamiltonian flow, the modified Markov chain will converge to the correct target distribution.

Appendix 2: Potential matching

In the paper, training data collected from the history of Markov chain are used without a detailed explanation. Here, we will analyze the asymptotical behavior of surrogate-induced distribution and try to explain why the history of the Markov chain is a reasonable choice for training data. Recall that we find our neural network surrogate function by minimizing the mean square error (11). Similarly to Hyvärinen (2005), it turns out that minimizing (11) is asymptotically equivalent to minimizing a new distance between the surrogate-induced distribution and the underlying target distribution, independent of their corresponding normalizing constants.

Suppose we know the density of the underlying intractable target distribution up to a constant

$$P(q|Y)=\frac{1}{Z}\text{exp}(-U(q)),$$

where Z is the unknown normalizing constant. Our goal is to approximate this distribution using a parametrized density model, also known up to a constant,

$$Q(q,\theta )=\frac{1}{Z(\theta )}\text{exp}(-V(q,\theta )).$$

Ignoring the multiplicative constant, the corresponding potential energy functions are U(q) and V(q, θ), respectively. The straightforward square distance between the two potentials will not be a well-defined measure between the two distributions because of the unknown normalizing constants. Therefore, we use the following distance instead:

$$K(\theta )=\underset{d}{\text{min}}\int {\Vert V(q,\theta )-U(q)-d\Vert }^{2}P(q|Y)\mathrm{d}q=\int {\Vert V(q,\theta )-U(q)\Vert }^{2}P(q|Y)\mathrm{d}q-[E_q{(V(\theta )-U)]}^2=𝕍\mathrm{a}{\mathrm{r}}_q(V(\theta )-U).$$ (14)

Because of its similarity to score matching (Hyvärinen 2005), we refer to the approximation method based on this new distance as potential matching; the corresponding potential matching estimator of θ is given by

$$\hat{\theta }=\text{arg }\underset{\theta }{\text{min}}K(\theta ).$$

It is easy to verify that K(θ) = 0 ⇒ V(θ) = U + constant ⇒ Q(q, θ) = P(q|Y), so K(θ) is a well-defined squared distance. Exact evaluation of (14) is analytically intractable. In practice, given N samples from the target distribution q₁, q₂, …, q_N, we minimize the empirical version of (14)

$$\tilde{K}(\theta )=\underset{d}{\text{min}}\frac{1}{N}\sum _{n=1}^N{\Vert V(q_n,\theta )-U(q_n)-d\Vert }^2=\frac{1}{N}\sum _{n=1}^N{\Vert V(q_n,\theta )-U(q_n)\Vert }^2-{(\frac{1}{N}\sum _{n=1}^{N}V(q_n,\theta )-U(q_n))}^2,$$ (15)

which is asymptotically equivalent to K by law of large numbers. (15) could be more concise if we allow a shift term in the parametrized model (V(q, θ) = V(q, θ_−d)+θ_d). In that case, the empirical potential matching estimator is

$$\tilde{\theta }=\text{arg }\underset{\theta }{\text{min}}\tilde{K}(\theta )=\text{arg }\underset{\theta }{\text{min}}\underset{d}{\text{min}}\frac{1}{N}\sum _{n=1}^N{\Vert V(q_n,{\theta }_{-d})+({\theta }_d-d)-U(q_n)\Vert }^2=\text{arg }\underset{\theta }{\text{min}}\frac{1}{N}\sum _{n=1}^N{\Vert V(q_n,{\theta }_{-d})+{\theta }_d-U(q_n)\Vert }^2=\text{arg }\underset{\theta }{\text{min}}\frac{1}{N}\sum _{n=1}^N{\Vert V(q_n,\theta )-U(q_n)\Vert }^2.$$

In particular, if we adopt an additive model for V(q, θ) with a shift term

$$V(q,\theta )=\sum _{i=1}^s{\upsilon }_i\sigma ({\mathit{w}}_{\mathit{i}}·q+d_i)+b,\theta =(\mathit{\upsilon },b),$$

where w_i, d_i, and activation function σ are chosen according to Algorithm 2 and collect early evaluations from the history of Markov chain

$${𝒯}_N=\left\{\right(q^{(1)},U\left(q^{(1)}\right)),(q^{(2)},U\left(q^{(2)}\right)),\dots ,\times (q^{(N)},U\left(q^{(N)}\right)\left)\right\},$$

as training data; this way, the estimated parameters (i.e., weights for the output neuron) are asymptotically the potential matching estimates

$$\underset{N\to \infty }{\text{lim}}{\hat{\theta }}_{\mathrm{E}\mathrm{L}\mathrm{M},{𝒯}_N}=\text{arg }\underset{\theta }{\text{min}}\underset{N\to \infty }{\text{lim}}C(\theta |{𝒯}_N)=\hat{\theta },$$

since the Markov chain will eventually converge to the target distribution. When truncated at a finite N, the estimated parameters are almost the empirical potential matching estimates except that the samples from the history of the Markov chain are not exactly (but quite close) from the target distribution.

Appendix 3: Adaptive learning

Theorem 1

(Greville) If a matrix A, with k columns, is denoted by A_k and partitioned as A_k = [A_k−1a_k], with A_k−1 a matrix having k − 1 columns, then the Moore–Penrose generalized inverse of A_k is

$$A_k^{†}=\left[\begin{array}{c}\hfill A_{k-1}^{†}(I-a_k{b}_k^{\mathrm{T}})\hfill \\ \hfill b_k^{\mathrm{T}}\hfill \end{array}\right],$$

where

$$c_k=(I-A_{k-1}{A}_{k-1}^{†})a_k,b_k=\{\begin{array}{cc}\frac{{(A_{k-1}^{†})}^{\mathrm{T}}{A}_{k-1}^{†}{a}_k}{1+{\Vert A_{k-1}^{†}{a}_k\Vert }^2},\hfill & c_k=0,\hfill \\ \frac{c_k}{{\Vert c_k\Vert }^2},\hfill & c_k\ne 0.\hfill \end{array}$$

Proof of Proposition 1

To save computational cost, we only update the estimator. Suppose the current output matrix is $H_{k+1}=\left[\begin{array}{c}\hfill H_k\hfill \\ \hfill k_{k+1}^{\mathrm{T}}\hfill \end{array}\right]$ and the target vector is $T_{k+1}=\left[\begin{array}{c}\hfill T_k\hfill \\ \hfill t_{k+1}\hfill \end{array}\right]$, then

$${\upsilon }_{k+1}^{\mathrm{T}}{H}_{k+1}^{\mathrm{T}}=T_{k+1}^{\mathrm{T}}\Rightarrow {\upsilon }_{k+1}^{\mathrm{T}}=T_{k+1}^{\mathrm{T}}{\left(H_{k+1}^{\mathrm{T}}\right)}^{†}=\left[T_k^{\mathrm{T}}{t}_{k+1}\right]{\left(\right[H_k^{\mathrm{T}}{h}_{k+1}\left]\right)}^{†}=\left[T_k^{\mathrm{T}}{t}_{k+1}\right]\left[\begin{array}{c}\hfill {(H_k^{\mathrm{T}})}^{†}(I-h_{k+1}{b}_{k+1}^{\mathrm{T}})\hfill \\ \hfill b_{k+1}^{\mathrm{T}}\hfill \end{array}\right]=T_k^{\mathrm{T}}{\left(H_k^{\mathrm{T}}\right)}^{†}(I-h_{k+1}{b}_{k+1}^{\mathrm{T}})+t_{k+1}{b}_{k+1}^{\mathrm{T}}={\upsilon }_k^{\mathrm{T}}+(t_{k+1}-{\upsilon }_k^{\mathrm{T}}{h}_{k+1})b_{k+1}^{\mathrm{T}},$$

where b_k+1 is obtained according to Theorem 1. Note that the computation of b_k+1 and c_k+1 still involves H_k and $H_k^{†}$ whose sizes increase as data size grows. Following Kohonen (1988), Kovanic (1979), and Schaik and Tapson (2015), we introduce two auxiliary matrices here

$${\Phi }_k=I-H_k^{\mathrm{T}}{\left(H_k^{\mathrm{T}}\right)}^{†}\in {ℝ}^{s\times s},$$

$${\Theta }_k={\left({\left(H_k^{\mathrm{T}}\right)}^{†}\right)}^{\mathrm{T}}{\left(H_k^{\mathrm{T}}\right)}^{†}=H_k^{†}{\left(H_k^{\mathrm{T}}\right)}^{†}\in {ℝ}^{s\times s},$$

and rewrite b_k+1 and c_k+1 as

$$c_{k+1}={\Phi }_k{h}_{k+1},b_{k+1}=\{\begin{array}{cc}\frac{{\Theta }_k{h}_{k+1}}{1+h_{k+1}^{\mathrm{T}}{\Theta }_k{h}_{k+1}},\hfill & c_{k+1}=0,\hfill \\ \frac{c_{k+1}}{{\Vert c_{k+1}\Vert }^2},\hfill & c_{k+1}\ne 0.\hfill \end{array}$$

Updating of the two auxiliary matrices can also be done adaptively

$${\Phi }_{k+1}=I-H_{k+1}^{\mathrm{T}}{\left(H_{k+1}^{\mathrm{T}}\right)}^{†}=I-\left[H_k^{\mathrm{T}}{h}_{k+1}\right]\left[\begin{array}{c}\hfill {(H_k^{\mathrm{T}})}^{†}(I-h_{k+1}{b}_{k+1}^{\mathrm{T}})\hfill \\ \hfill b_{k+1}^{\mathrm{T}}\hfill \end{array}\right]={\Phi }_k-{\Phi }_k{h}_{k+1}{b}_{k+1}^{\mathrm{T}},$$

$${\Theta }_{k+1}=H_{k+1}^{†}{\left(H_{k+1}^{\mathrm{T}}\right)}^{†}=\left[\right(I-b_{k+1}{h}_{k+1}^{\mathrm{T}}\left)H_k^{†}{b}_{k+1}\right]\left[\begin{array}{c}\hfill {(H_k^{\mathrm{T}})}^{†}(I-h_{k+1}{b}_{k+1}^{\mathrm{T}})\hfill \\ \hfill b_{k+1}^{\mathrm{T}}\hfill \end{array}\right]=(I-b_{k+1}{h}_{k+1}^{\mathrm{T}}){\Theta }_k(I-h_{k+1}{b}_{k+1}^{\mathrm{T}})+b_{k+1}{b}_{k+1}^{\mathrm{T}},$$

and if c_k+1 = 0, these formulas can be simplified as below

$${\Phi }_{k+1}={\Phi }_k,{\Theta }_{k+1}={\Theta }_k-{\Theta }_k{h}_{k+1}{b}_{k+1}^{\mathrm{T}}.$$