Extended Stochastic Gradient MCMC for Large-Scale Bayesian Variable Selection

Summary

Stochastic gradient Markov chain Monte Carlo (MCMC) algorithms have received much attention in Bayesian computing for big data problems, but they are only applicable to a small class of problems for which the parameter space has a fixed dimension and the log-posterior density is differentiable with respect to the parameters. This paper proposes an extended stochastic gradient MCMC algorithm which, by introducing appropriate latent variables, can be applied to more general large-scale Bayesian computing problems, such as those involving dimension jumping and missing data. Numerical studies show that the proposed algorithm is highly scalable and much more efficient than traditional MCMC algorithms. The proposed algorithms have much alleviated the pain of Bayesian methods in big data computing.

1. Introduction

After six decades of continual development, MCMC has proven to be a powerful and typically unique computational tool for analyzing data of complex structures. However, for large datasets, its computational cost can be prohibitive as it requires all of the data to be processed at each iteration. To tackle this difficulty, a variety of scalable algorithms have been proposed in the recent literature. According to the strategies they employed, these algorithms can be grouped into a few categories, including stochastic gradient MCMC algorithms (Welling & Teh, 2011; Ding et al., 2014; Ahn et al., 2012; Chen et al., 2014; Betancourt, 2015; Ma et al., 2015; Nemeth & Fearnhead, 2019), split-and-merge algorithms (Scott et al., 2016; Srivastava et al., 2018; Xue & Liang, 2019), mini-batch Metropolis-Hastings algorithms (Chen et al., 2016; Korattikara et al., 2014; Bardenet et al., 2014; Maclaurin & Adams, 2014; Bardenet et al., 2017), nonreversibleMarkov process-based algorithms(Bierkens et al., 2019; Bouchard Coté et al., 2018), and some discrete sampling algorithms based on the multi-armed bandit (Chen & Ghahramani, 2016).

Although scalable algorithms have been developed for both continuous and discrete sampling problems, they are hard to be applied to dimension jumping problems. Dimension jumping is characterized by variable selection where the number of parameters changes from iteration to iteration in MCMC simulations. Under their current settings, the stochastic gradient MCMC and nonreversible Markov process-based algorithms are only applicable to problems for which the parameter space has a fixed dimension and the log-posterior density is differentiable with respect to the parameters. For the split-and-merge algorithms, it is unclear how to aggregate samples of different dimensions drawn from the posterior distributions based on different subset data. The multi-armed bandit algorithms are only applicable to problems with a small discrete domain and can be extremely inefficient for high-dimensional variable selection problems. The mini-batch Metropolis-Hastings algorithms are in principle applicable to dimension jumping problems. However, they are generally difficult to use. For example, the algorithms by Chen et al. (2016), Korattikara et al. (2014), and Bardenet et al. (2014) perform approximate acceptance tests using subset data. The amount of data consumed for each test varies significantly from one iteration to another, which compromise their scalability. The algorithms by Maclaurin & Adams (2014) and Bardenet et al. (2017) perform exact tests but require a lower bound on the parameter distribution across its domain. Unfortunately, the lower bound is usually difficult to obtain.

This paper proposes an extended stochastic gradient Langevin dynamics algorithm which, by introducing appropriate latent variables, extends the stochastic gradient Langevin dynamics algorithm to more general large-scale Bayesian computing problems such as variable selection and missing data. The extended stochastic gradient Langevin dynamics algorithm is highly scalable and much more efficient than traditional MCMC algorithms. Compared to the mini-batch Metropolis-Hastings algorithms, the proposed algorithm is much easier to use, which involves only a fixed amount of data at each iteration and does not require any lower bound on the parameter distribution.

2. A Brief Review of Stochastic Gradient Langevin Dynamics

Let X_N = (X₁,X₂, …, X_N) denote a set of N independent and identically distributed samples drawn from the distribution f(x|θ), where N is the sample size and θ is the parameter. Let $p\left(X_N\mid \theta \right)={\prod }_{i=1}^{N}f\left(X_i\mid \theta \right)$ denote the likelihood function, let π(θ) denote the prior distribution of θ, and let log π(θ|X_N) = logp(X_N|θ) + log π(θ) denote the log-posterior density function. If θ has a fixed dimension and log π(θ|X_N) is differentiable with respect to θ, then the stochastic gradient Langevin dynamics algorithm (Welling & Teh, 2011) can be applied to simulate from the posterior, which iterates by

$${\theta }_{t+1}={\theta }_t+\frac{{ϵ}_{t+1}}{2}{\widehat{\nabla }}_{\theta }\text{log}\pi \left({\theta }_t\mid X_N\right)+\sqrt{\left({ϵ}_{t+1}\tau \right){\eta }_{t+1}},{\eta }_{t+1}~N\left(0,I_d\right),$$ (1)

where d is the dimension of θ, I_d is an d × d-identity matrix, ϵ_t+1 is the step size (also known as the learning rate), τ is the temperature, and ${\widehat{\nabla }}_{\theta }\text{log}\pi \left({\theta }_t\mid X_N\right)$ denotes an estimate of ∇_θ log π(θ_t|X_N) based on a mini-batch of samples. The learning rate can be decreasing or kept as a constant. For the former, the convergence of the algorithm was studied in Teh et al. (2016). For the latter, the convergence of the algorithm was studied in Sato & Nakagawa (2014) and Dalalyan & Karagulyan (2017). Refer to Nemeth & Fearnhead (2019) for more discussions on the theory, implementation and variants of this algorithm.

3. An Extended Stochastic Gradient Langevin Dynamics Algorithm

To extend the applications of the stochastic gradient Langevin dynamics algorithm to varying-dimensional problems such as variable selection and missing data, we first establish an identity for evaluating ∇_θ log π(θ|X_N) in presence of latent variables. As illustrated below, the latent variables can be the model indicator in the variable selection problems or missing values in the missing data problems.

Lemma 1.

For any latent variable ϑ,

$${\nabla }_{\theta }\text{log}\pi \left(\theta \mid X_N\right)=\int {\nabla }_{\theta }\text{log}\pi \left(\theta \mid \vartheta ,X_N\right)\pi \left(\vartheta \mid \theta ,X_N\right)d\vartheta ,$$ (2)

where π(ϑ | θ, X_N) and π(θ | ϑ, X_N) denote the conditional distribution of ϑ and θ, respectively.

Lemma 1 provides us a Monte Carlo estimator for ∇_θ log π(θ | X_N) by averaging over the samples drawn from the conditional distribution π(ϑ|θ, X_N). The identity (2) is similar to Fisher’s identity. The latter has been used in evaluating the gradient of the log-likelihood function in presence of latent variables, see e.g. Cappé et al. (2005). When N is large, the computation can be accelerated by subsampling. Let X_n denote a subsample, where n denotes the subsample size. Without loss of generality, we assume that N is a multiple of n, i.e., N/n is an integer. Let X_n,N = {X_n, … , X_n} denote a duplicated dataset with the subsample, whose total sample size is also N. Following from (2), we have

$${\nabla }_{\theta }\text{log}\pi \left(\theta \mid X_{n,N}\right)=\int {\nabla }_{\theta }\text{log}\pi \left(\theta \mid \vartheta ,X_{n,N}\right)\pi \left(\vartheta \mid \theta ,X_{n,N}\right)d\vartheta .$$ (3)

Since ∇_θ log π(θ | X_n,N) = ∇_θ log p(X_n,N|θ) + ∇_θ log π(θ) is true and log p(X_n,N|θ) is unbiased for log p(X_N|θ), ∇_θ log π(θ | X_n,N) forms an unbiased estimator of ∇_θ log π(θ | X_N). Sampling from π(γ_S|θ, X_n,N) can be much faster than sampling from π(γ_S|θ, X_N) as for the former the likelihood only needs to be evaluated on a mini-batch of samples.

3.1. Bayesian Variable Selection

As an illustrative example, we consider the problem of variable selection for linear regression

$$Y=z^T\beta +\epsilon ,$$ (4)

where ε is a zero-mean Gaussian random error with variance σ², $\beta \in {ℝ}^p$ is the vector of regression coefficients, and z = (z₁,z₂, … , z_p) is the vector of explanatory variables. Let ${\gamma }_S=\left({\gamma }_S^1,\dots ,{\gamma }_S^p\right)$ be a binary vector indicating the variables included in model S, and let β_S be the vector of regression coefficients associated with the model S. From the perspective of Bayesian statistics, we are interested in estimating the posterior probability π(γ_S|X_N)·for each model $S\in \mathcal{S}$ and the posterior mean π(ρ) = ∫ ρ(β)π(β|X_N) for some integrable function ρ(·), where $\mathcal{S}$ comprises 2^p models. Both quantities can be estimated using the reversible jump Metropolis-Hastings algorithm (Green, 1995) by sampling from the posterior distribution π(γ_S, β_S|X_N). However, when N is large, the algorithm can be extremely slow due to repeated scans of the full dataset in simulations.

As aforementioned, the existing stochastic gradient MCMC algorithms cannot be directly applied to simulate of π(γ_S, β_S|X_N) due to the dimension jumping issue involved in model transition. To address this issue, we introduce an auxiliary variable θ = (θ¹,θ², … , θ^p), which links γ_S and β_S through

$${\beta }_S=\theta *{\gamma }_S=\left({\theta }^1{\gamma }_S^1,{\theta }^2{\gamma }_S^2,\dots ,{\theta }^p{\gamma }_S^p\right),$$ (5)

where * denotes elementwise multiplication. Let ${\theta }_{\left[S\right]}=\left\{{\theta }^i:{\gamma }_S^i=1,i=1,2,\dots ,p\right\}$ and ${\theta }_{\left[-S\right]}=\left\{{\theta }^i:{\gamma }_S^i=0,i=1,2,\dots ,p\right\}$ be subvectors of θ corresponding to the nonzero and zero elements of γ_S, respectively. Note that β_S is sparse with all elements in θ_[−S] being zero, while θ can be dense. Based on the relation (5), we suggest to simulate from π(θ|X_N) using the stochastic gradient Langevin dynamic algorithm, for which the gradient ∇_θ log π(θ|X_N) can be evaluated using Lemma 1 by treating γ_S as the latent variable. Let π(θ) denote the prior of θ. To simplify the computation of ∇_θ log π(θ | γ_S, X_N), we further assume the a priori independence that π(θ|γ_S) = π(θ_[S]|γ_S)π(θ_[−S]|γ_S). Then it is easy to derive

$${\nabla }_{\theta }\text{log}\pi \left(\theta \mid {\gamma }_S,X_N\right)=\{\begin{array}{ll}{\nabla }_{{\theta }_{\left[S\right]}}\text{log}p\left(X_N\mid {\theta }_{\left[S\right]},{\gamma }_S\right)+{\nabla }_{{\theta }_{\left[S\right]}}\pi \left({\theta }_{\left[S\right]}\mid {\gamma }_S\right),\hfill & \text{for component }{\theta }_{\left[S\right]},\hfill \\ {\nabla }_{{\theta }_{\left[-S\right]}}\text{log}\pi \left({\theta }_{\left[-S\right]}\mid {\gamma }_S\right),\hfill & \text{for component }{\theta }_{\left[-S\right]},\hfill \end{array}$$

which can be used in evaluating ∇log π(θ|X_N) by Lemma 1. If a mini-batch of data is used, the gradient can be evaluated based on (3). This leads to an extended stochastic gradient langevin dynamics algorithm.

Algorithm 1.

[Extended Stochastic Gradient Langevin Dynamics for Bayesian Variable Selection]

(Subsampling) Draw a subsample of size n (with or without replacement) from the full dataset XN at random, and denote the subsample by $X_n^{\left(t\right)}$, where t indexes the iteration. (Simulating models) Simulate models ${\gamma }_{S_1,n}^{\left(t\right)},\dots ,{\gamma }_{S_m,n}^{\left(t\right)}$ from the conditional posterior $\pi \left({\gamma }_S\mid {\theta }^{\left(t\right)},X_{n,N}^{\left(t\right)}\right)$ by running a short Markov chain, where $X_{n,N}^{(t)}=\left\{X_n^{(t)},\dots ,X_n^{(t)}\right\}$ and θ(t) is the sample of θ at iteration t. (Updating θ) Update θ(t) by setting ${\theta }^{(t+1)}={\theta }^{(t)}+{\left(2m\right)}^{-1}{ϵ}_{t+1}{\sum }_{k=1}^m{\nabla }_{\theta }\text{log}\pi \left({\theta }^{(t)}\mid {\gamma }_{S_k,n}^{(t)},X_{n,N}^{(t)}\right)+\sqrt{\left({ϵ}_{t+1}\tau \right){\eta }_{t+1}}$, where ϵt+1 is the learning rate, ηt+1 ~ N(0, Ip), τ is the temperature, and p is the dimension of θ.

Theorem 1 justifies the validity of this algorithm with the proof given in the Appendix.

Theorem 1.

Assume that the conditions (A.1)-(A.3) (given in Appendix) hold, m, p, n are increasing with N such that N ≥ n ≻ p, m ≻ p^1/2, and a constant learning rate ϵ ≺ 1/N is used. Then, as N → ∞,

1. W₂(π_t,π_*) → 0 as t → ∞, where π_t denotes the distribution of θ^(t), π_* = π(θ|X_N), and W₂(·,·) denotes the second order Wasserstein distance between two distributions.

2. If ρ(θ) is α-Lipschitz for some constant α > 0, then ${\sum }_{t=1}^T\rho \left({\theta }^{(t)}\right)/T\stackrel{p}{\to }{\pi }_{*}\left(\rho \right)$ as T → ∞, where $\stackrel{p}{\to }$ denotes convergence in probability and π_*(ρ) = ∫_Θ ρ(θ)π(θ|X_N)dθ.

3. If (A.4) further holds, ${\sum }_{t=1}^T{\sum }_{i=1}^{m}I\left({\gamma }_{S_i,n}^{(t)}={\gamma }_S\right)/\left(mT\right)-\pi \left({\gamma }_S\mid X_N\right)\stackrel{p}{\to }0$ as T → ∞

Part (i) establishes the weak convergence of θ_t; that is, if the total sample size N and the iteration number t are sufficiently large, and the subsample size n and the number of models m simulated at each iteration are reasonably large, then π(θ_t|X_N) will converge to the true posterior π(θ|X_N) in 2-Wasserstein distance. Refer to Gibbs & Su (2002) for discussions on the relation between Wasserstein distance and other probability metrics. Parts (ii) & (iii) address our general interests on how to estimate the posterior mean and posterior probability, respectively, based the samples simulated by Algorithm 1. For parts (i), (ii) and (iii), the explicit convergence rates are given in equations (3), (5) and (10), respectively.

For the choice of m ≻ p^1/2, p can be approximately treated as the maximum size of the models under consideration, which is of the same order as the true model. Therefore, m can be pretty small under the model sparsity assumption. Theorem 1 is established with a constant learning rate. In practice, one may use a decaying learning rate, see e.g. Teh et al. (2016), where it is suggested to set ϵ_t = O(1/t^κ) for some 0 < κ ≤ 1. For the decaying learning rate, Teh et al. (2016) recommended some weighted averaging estimators for π_*(ρ). Theorem 2 shows that the unweighted averaging estimators used above still work if the learning rate slowly decays at a rate of ϵ_t = O(1/t^κ) for 0 < κ < 1. However, if κ = 1, the weighted averaging estimators are still needed. The proof of Theorem 2 is given in the supplementary material.

Theorem 2.

Assume the conditions of Theorem 1 hold. If a decaying learning rate ϵ_t = O(1/t^κ) is used for some 0 < κ < 1, then parts (i), (ii) and (iii) of Theorem 1 are still valid.

3.2. Missing Data

Missing data are ubiquitous over all fields from science to technology. However, under the big data scenario, how to conduct Bayesian analysis in presence of missing data is still unclear. The existing data-augmentation algorithm (Tanner & Wong, 1987) is full data based and thus can be extremely slow. In this context, we let X_N denote the incomplete data and let θ denote the model parameters. If we treat the missing values as latent variables, then Lemma 1 can be used for evaluating the gradient ∇_θ log π(θ|X_N). However, Algorithm 1 cannot be directly applied to missing data problems, since the imputation of the missing data might depend on the subsample only. To address this issue, we propose Algorithm S1 (given in the Supplementarymaterial), where the missing values ϑ are imputed from π(ϑ|θ, X_n) at each iteration. Theorem 1 and Theorem 2 are still applicable to this algorithm.

4. An Illustrative Example

This section illustrates the performance of Algorithm 1 using a simulated example. More numerical examples are presented in the supplementary material. Ten synthetic datasets were generated from the model (4) with N = 50,000, p = 2001, σ² = 1, β₁ = ⋯ = β₅ = 1, β₆ = β₇ = β₈ = −1, and β₀ = β₉ = ⋯ = β_p = 0, where σ² is assumed to be known, and the explanatory variables are normally distributed with a mutual correlation coefficient of 0.5. A hierarchical prior was assumed for the model and parameters with the detail given in the supplementary material. For each dataset, Algorithm 1 was run for 5000 iterations with n = 200, m = 10, and the learning rate ϵ_t ≡ 10⁻⁶, where the first 2000 iterations were discarded for the burn-in process and the samples generated from the remaining iterations were used for inference. At each iteration, the reversible jump Metropolis-Hastings algorithm (Green, 1995) was used for simulating the models ${\gamma }_{S_i,n}^{(t)}$, i = 1, 2, …, m with the detail given in the supplementary material.

Table 1 summarizes the performance of the algorithm, where the false selection rate (FSR), negative selection rate (NSR), mean squared errors for false predictors (MSE₀) and mean squared errors for true predictors (MSE₁) are defined in the supplementary material. The variables were selected according to the median posterior probability rule (Barbieri & Berger, 2004), which selects only the variables with the marginal inclusion probability greater than 0.5. The Bayesian estimates of parameters were obtained by averaging over a set of thinned (by a factor of 10) posterior samples. For comparison, some existing algorithms were applied to this example with the results given in Table 1 and the implementation details given in the supplementary material. The comparison show that the proposed algorithm has much alleviated the pain of Bayesian methods in big data analysis.

Table 1.

Bayesian variable selection with the extended stochastic gradient Langevin dynamics (eSGLD), reversible jump Metropolis-Hastings (RJMH), split-and-merge (SaM) and Bayesian Lasso (B-Lasso) algorithms, where FSR, NSR, MSE₁ and MSE₀ are reported in averages over 10 datasets with standard deviations given in the parentheses, and the CPU time (in minutes) was recorded for one dataset on a Linux machine with Intel® Core™i7–3770 CPU@3.40GHz.

Algorithm	FSR	NSR	MSE1	MSE0	CPU(m)
eSGLD	0(0)	0(0)	2.91 × 10−3(1.90 × 10−3)	1.26 × 10−7(1.18 × 10−8)	3.3
RJMH	0.50(0.10)	0.16(0.042)	1.60 × 10−1(3.89 × 10−2)	2.64 × 10−5(8.75 × 10−6)	180.1
SaM	0.05(0.05)	0.013(0.013)	1.29 × 10−2(1.27 × 10−2)	1.01 × 10−6(1.00 × 10−6)	150.4
B-Lasso	0(0)	0(0)	2.32 × 10−4(3.58 × 10−5)	1.40 × 10−7(5.08 × 10−9)	32.8

5. Discussion

This paper has extended the stochastic gradient Langevin dynamics algorithm to general large-scale Bayesian computing problems, such as those involving dimension jumping and missing data. To the best of our knowledge, this paper provides the first Bayesian method and theory for high-dimensional discrete parameter estimation with mini-batch samples, while the existing methods work for continuous parameters or very low dimensional discrete problems only. Other than generalized linear models, the proposed algorithm can have many applications in data science. For example, it can be used for sparse deep learning and accelerating computation for statistical models/problems where latent variables are involved, such as hidden Markov models, random coefficient models, and model-based clustering problems.

Algorithm 1 can be further extended by updating θ using a variant of stochastic gradient Langevin dynamics, such as stochastic gradient Hamiltonian Monte Carlo (Chen et al., 2014), stochastic gradient thermostats (Ding et al., 2014), stochastic gradient Fisher scoring (Ahn et al., 2012), or preconditioned stochastic gradient Langevin dynamics (Li et al., 2016). We expect that the advantages of these variants (over stochastic gradient Langevin dynamics) can be carried over to the extension.

Appendix

A.1. Proof of Lemma 1

Proof. Let π(θ) denote the prior density of θ, and let π(ϑ) denote the density of ϑ. Then

$$\begin{gathered} {\nabla }_{\theta }\text{log}\pi \left(\theta \mid X_N\right)={\nabla }_{\theta }\text{log}p\left(X_N\mid \theta \right)+{\nabla }_{\theta }\text{log}\pi \left(\theta \right)=\frac{1}{p\left(X_N\mid \theta \right)}{\nabla }_{\theta }\int p\left(X_N,\vartheta \mid \theta \right)d\vartheta +{\nabla }_{\theta }\text{log}\pi \left(\theta \right) \\ =\int \frac{p\left(X_N,\vartheta \mid \theta \right)}{p\left(X_N\mid \theta \right)}{\nabla }_{\theta }\text{log}p\left(X_N,\vartheta \mid \theta \right)d\vartheta +{\nabla }_{\theta }\text{log}\pi \left(\theta \right) \\ =\int \pi \left(\vartheta \mid \theta ,X_N\right){\nabla }_{\theta }\left[\text{log}p\left(X_N\mid \theta ,\vartheta \right)+\text{log}\pi \left(\theta \mid \vartheta \right)+\text{log}\pi \left(\vartheta \right)-\text{log}\pi \left(\theta \right)\right]d\vartheta +{\nabla }_{\theta }\text{log}\pi \left(\theta \right) \\ =\int {\nabla }_{\theta }\text{log}\pi \left(\theta \mid \vartheta ,X_N\right)\pi \left(\vartheta \mid \theta ,X_N\right)d\vartheta , \end{gathered}$$

where the second and third equalities follow from the relation ∇_θ log(g(θ)) = ∇_θg(θ)/g(θ) (for an appropriate function g(θ)), and the fourth and fifth equalities are by direct calculations of the conditional distributions. □

A.2. Proof of Theorem 1

Let π_* = π(θ|X_N) denote the posterior density function of θ, and let π_t = π(θ^(t)|X_N) denote the density of θ^(t) generated by Algorithm 1 at iteration t. We are interested in studying the discrepancy between π_* and π_t in the 2nd order Wasserstein distance. The following conditions are assumed.

• (A.1)
  The posterior π_* is strongly log-concave and gradient-Lipschitz:

  $$f\left(\theta \right)-f\left(\theta \prime \right)-\nabla f{\left(\theta \prime \right)}^T\left(\theta -\theta \prime \right)\ge \frac{q_N}{2}{\Vert \theta -\theta \prime \Vert }_2^2,\forall \theta ,\theta \prime \in \Theta ,$$ (1)

  $${\Vert \nabla f\left(\theta \right)-\nabla f\left(\theta \prime \right)\Vert }_2\le Q_N{\Vert \theta -\theta \prime \Vert }_2,\forall \theta ,\theta \prime \in \Theta ,$$ (2)

  where f (θ) = − log π(θ|X_N), and $c_0^{\prime }N\le q_N\le Q_N\le c_{0}N$ for some positive constants c₀ and $c_0^{\prime }$.
• (A.2)The posterior π_* has bounded second moment: ∫_Θ θ^T θπ_*(θ)dθ = O(p).
• (A.3)${\text{max}}_{S\in \mathcal{S}}{E}_{X_N}\left[{\Vert {\nabla }_{\theta }\text{log}\pi \left(\theta \mid {\gamma }_S,X_N\right)\Vert }^2\mid \theta \right]=O\left(N^2\left({\Vert \theta \Vert }^2+p\right)\right)$, where E_XN denotes expectation with respect to the distribution of X_N, and $\mathcal{S}$ denotes the set of all possible models.
• (A.4)Let L_N(γ_S, θ) = log p(X_N|γ_S, θ)/N and let $\left\{L_N^{\left(i\right)}\left(\theta \right):i=1,2,\dots ,\left|\mathcal{S}\right|\right\}$ be the descending order statistics of $\left\{L_N\left({\gamma }_S,\theta \right):S\in \mathcal{S}\right\}$. Assume that there exists a constant δ > 0 such that ${\text{inf}}_{\theta \in \Theta }\left(L_N^{\left(1\right)}(\theta )-L_N^{\left(2\right)}\left(\theta \right)\right)\ge \delta$.

Proof. Part (i). In Algorithm 1, the gradient ∇log π(θ^(t)|X_N) is estimated by running a short Markov chain with a mini-batch of data. Since the initial distribution of the Markov chain might not coincide with its equilibrium distribution, the resulting gradient estimate can be biased. Let ${\zeta }^{\left(t\right)}=\frac{1}{m}{\sum }_{k=1}^m{\nabla }_{\theta }\text{log}\pi \left({\theta }^{\left(t\right)}\mid {\gamma }_{S_k,n}^{\left(t\right)},X_{n,N}^{\left(t\right)}\right)-\nabla \text{log}\pi \left({\theta }^{\left(t\right)}\mid X_N\right)$. Following from (A.3), we have

$${\Vert E\left({\zeta }^{\left(t\right)}\mid {\theta }^{\left(t\right)}\right)\Vert }^2=O\left[\frac{N^2\left({\Vert {\theta }^{\left(t\right)}\Vert }^2+p\right)}{m^2}\right],E{\Vert {\zeta }^{\left(t\right)}-E\left({\zeta }^{\left(t\right)}\mid {\theta }^{\left(t\right)}\right)\Vert }^2=O\left[\frac{N^2\left({\Vert {\theta }^{\left(t\right)}\Vert }^2+p\right)}{mn}\right].$$

Following from Lemma S2 in the supplementary material, if m ≻ p^1/2, ϵ ≺ 1/N ≺ (mn)/(Np), and V = O(p) holds, then

$$W_2\left({\pi }_t,{\pi }_{*}\right)={\left(1-\omega \right)}^t{W}_2\left({\pi }_0,{\pi }_{*}\right)+O\left(\frac{p^{1/2}}{m}\right)+O\left({\left(ϵp\right)}^{1/2}\right)+O\left({\left(\frac{ϵNp}{mn}\right)}^{1/2}\right)\to 0,\text{ as }t\to \infty ,$$ (3)

for some ω > 0, since q_N ≍ N and Q_N ≍ N hold by conditions (A.1) and (A.2).

Part (ii). Since ρ(θ) is α-Lipschitz, we have |ρ(θ)| ≤ α‖θ‖ + C′ for some constant C′. Further, π_* is strongly log-concave, so π_*(|ρ|) < ∞, i.e., ρ is π_*-integrable. On the other hand,

$$\begin{gathered} \Vert \int \rho \left(\theta \right)d{\pi }_{*}\left(\theta \right)-\int \rho \left(\tilde{\theta }\right)d{\pi }_t\left(\tilde{\theta }\right)\Vert =\Vert E\rho (\theta )-E\rho \left(\tilde{\theta }\right)\Vert \le E\Vert \rho \left(\theta \right)-\rho \left(\tilde{\theta }\right)\Vert \\ \le \alpha E{\Vert \theta -\tilde{\theta }\Vert }_2\le \alpha {\left\{E{\Vert \theta -\tilde{\theta }\Vert }_2^2\right\}}^{1/2}=\alpha W_2\left({\pi }_{*},{\pi }_t\right)=o\left(1\right),\left(\text{due to eq}.\left(\text{3}\right)\right). \end{gathered}$$ (4)

where θ and $\tilde{\theta }$ are two random variables whose marginal distributions follow π_* and π_t respectively, E(·) denotes expectation with respect to the joint distribution of θ and $\tilde{\theta }$, and ${\left(E{\Vert \theta -{\theta }_t\Vert }_2^2\right)}^{1/2}=W_2\left({\pi }_{*},{\pi }_t\right)$. This implies that ρ is also π_t-integrable and $\int \rho \left(\tilde{\theta }\right)d{\pi }_t\left(\tilde{\theta }\right)\to \int \rho \left(\theta \right)d{\pi }_{*}\left(\theta \right)$ as t → ∞.

Further, by the property of Markov chain, WLLN applies and thus ${\sum }_{t=1}^T\rho \left({\theta }^{\left(t\right)}\right)/T-{\sum }_{t=1}^T\int \rho \left(\tilde{\theta }\right)d{\pi }_t\left(\tilde{\theta }\right)/T=O_p\left(T^{-1/2}\right)$. Combining it with the above result leads to

$$\sum _{t=1}^T\rho \left({\theta }^{\left(t\right)}\right)/T-{\pi }_{*}\left(\rho \right)=O_p\left(T^{-1/2}\right)+\alpha \sum _{t=1}^T{W}_2\left({\pi }_{*},{\pi }_t\right)/T\to 0.$$ (5)

Part (iii). To establish the convergenceof $\widehat{\pi }\left({\gamma }_S\mid X_N\right)$, we define L_N(γ_S, θ^(t)) = log p(X_N|γ_S, θ^(t))/N, $L_n\left({\gamma }_S,{\theta }^{\left(t\right)}\right)=\text{log}p\left(X_n^{\left(t\right)}\mid {\gamma }_S,{\theta }^{\left(t\right)}\right)/n$, and ${\xi }_{n,S}^{\left(t\right)}=L_n\left({\gamma }_S,{\theta }^{\left(t\right)}\right)-L_N\left({\gamma }_S,{\theta }^{\left(t\right)}\right)$ for any $S\in \mathcal{S}$ For each S, ${\xi }_{n,S}^{\left(t\right)}$ is approximately Gaussian with $E\left({\xi }_{n,S}^{\left(t\right)}\right)=0$ and $\text{Var}\left({\xi }_{n,S}^{\left(t\right)}\right)=O\left(1/n\right)$. Therefore, for any positive ν, with probability $1-|\mathcal{S}{|}^{-\nu }$, max_S |ξ_{n, S}| is bounded by ${\delta }_n:={\left\{\left(2\nu +2\right)\text{log}\left|\mathcal{S}\right|/n\right\}}^{1/2}=O\left[{\left\{\left(v+1\right)p/n\right\}}^{1/2}\right]$ according to the tail probability of the Gaussian. It implies, with high probability, that if S is the most likely model, i.e., $L_N^{\left(t\right)}\left({\gamma }_S\right)=L_N^{\left(1\right)}\left({\theta }^{\left(t\right)}\right)$, then

$$\begin{gathered} \left|\pi \left({\gamma }_S\mid X_{n,N}^{\left(t\right)},{\theta }^{\left(t\right)}\right)-\pi \left({\gamma }_S\mid X_N,{\theta }^{\left(t\right)}\right)\right| \\ =\left|\frac{1}{1+{\sum }_{S\prime \ne S}{e}^{N\left(L_N\left({\gamma }_{S\prime },{\theta }^{(t)}\right)-L_N\left({\gamma }_S,{\theta }^{(t)}\right)+{\xi }_{n,S\prime }-{\xi }_{n,S}\right)}}-\frac{1}{1+{\sum }_{S\prime \ne S}{e}^{N\left(L_N\left(X_N\mid {\gamma }_{S\prime },{\theta }^{(t)}\right)-L_N\left(X_N\mid {\gamma }_S,{\theta }^{(t)}\right)\right)}}\right| \\ =\frac{{\sum }_{S\prime \ne S}{e}^{N\left(L_N\left(X_N\mid {\gamma }_{S\prime },{\theta }^{(t)}\right)-L_N\left(X_N\mid {\gamma }_S,{\theta }^{(t)}\right)\right)+b_{S\prime }})}{{\left[1+{\sum }_{S\prime \ne S}{e}^{N\left(L_N\left(X_N\mid {\gamma }_{S\prime },{\theta }^{(t)}\right)-L_N\left(X_N\mid {\gamma }_S,{\theta }^{(t)}\right)+b_{S\prime }\right)}\right]}^2}N\left|{\xi }_{n,S\prime }-{\xi }_{n,S}\right| \\ \le \left(2^p-1\right)e^{-N\left(\delta -2{\delta }_n\right)}N2{\delta }_n\le e^{-N\delta /2}\to 0, \end{gathered}$$

if νp ≺ n (i.e., δ_n ≺ δ) and N ≻ p, where the second equality follows from the mean-value theorem by viewing N(L_N(X_N|γ_S′, θ^(t)) − L_N(X_N|γ_S, θ^(t)))’s as the arguments of π(γ_S|X_N, θ^(t)), and b_S′ denotes a value between 0 and (ξ_n,S′ − ξ_{n, S}). Similarly, if S is not the most likely model, then we denote S* as the most likely model and, by the mean-value theorem,

$$\begin{gathered} \left|\pi \left({\gamma }_S\mid X_{n,N}^{(t)},{\theta }^{(t)}\right)-\pi \left({\gamma }_S\mid X_N,{\theta }^{(t)}\right)\right| \\ =\left|\frac{e^{N\left(L_N\left({\gamma }_S,{\theta }^{(t)}\right)-L_N\left({\gamma }_{S^{*}},{\theta }^{(t)}\right)+{\xi }_{n,S}-{\xi }_{n,S^{*}}\right)}}{1+{\sum }_{S\prime \ne S}{e}^{N(L_N\left({\gamma }_{S\prime },{\theta }^{(t)}\right)-L_N\left({\gamma }_{S^{*}},{\theta }^{(t)}\right)+{\xi }_{n,S^{\prime }}-{\xi }_{n,S^{*}}})}-\frac{e^{N\left(L_N\left({\gamma }_S,{\theta }^{(t)}\right)-L_N\left({\gamma }_{S^{*}},{\theta }^{(t)}\right)\right)}}{1+{\sum }_{S\prime \ne S}{e}^{N(L_N\left({\gamma }_{S\prime },{\theta }^{(t)}\right)-L_N\left({\gamma }_{S^{*}},{\theta }^{(t)}\right)}}\right| \\ \le \left[1+\left(2^p-1\right)e^{-N\left(\delta -2{\delta }_n\right)}+e^{2N{\delta }_n}\right]e^{-N\left(\delta -2{\delta }_n\right)}N2{\delta }_n\le e^{-N\delta /2}\to 0. \end{gathered}$$

In conclusion, with probability $1-1/{\left|\mathcal{S}\right|}^{\nu },\left|\pi \left({\gamma }_S\mid X_{n,N}^{\left(t\right)},{\theta }^{\left(t\right)}\right)-\pi \left({\gamma }_S\mid X_N,{\theta }^{\left(t\right)}\right)\right|<\text{exp}\left(-N\delta /2\right)$ for all S, any iteration t and any θ^(t) ∈ Θ. Then, one could choose some ν = (n/p)^1/2 → ∞, such that $\pi \left({\gamma }_S\mid X_{n,N}^{\left(t\right)},{\theta }^{\left(t\right)}\right)-\pi \left({\gamma }_S\mid X_N,{\theta }^{\left(t\right)}\right)$ is bounded by

$$\underset{S}{\text{max}}E\left|\pi \left({\gamma }_S\mid X_{n,N}^{(t)},{\theta }^{(t)}\right)-\pi \left({\gamma }_S\mid X_N,{\theta }^{(t)}\right)\right|\le \text{exp}\left(-N\delta /2\right)+1/{\left|\mathcal{S}\right|}^{\nu }\to 0,$$ (6)

for any iteration t. Conditioned on {θ^(t) : t = 1,2, …}, [π(γ_S|X_n,N, θ^(t)) − π(γ_S|X_N, θ^(t))]’s are independent and each is bounded by 1, so WLLN applies. Therefore, for any $S\in \mathcal{S}$, by WLLN,

$$\frac{1}{T}\sum _{t=1}^T\pi \left({\gamma }_S\mid X_{n,N}^{(t)},{\theta }^{(t)}\right)-\frac{1}{T}\sum _{t=1}^T\pi \left({\gamma }_S\mid X_N,{\theta }^{(t)}\right)=O_p\left(T^{-1/2}\right)+\text{exp}\left(-N\delta /2\right)+1/{\left|\mathcal{S}\right|}^{\nu }\to 0,$$ (7)

provided p ≺ n ≤ N. Since {θ^(t) : t = 1,2, …} forms a time-homogeneous Markov chain, whose convergence is measured by (3), and the function π(γ_S|X_N, θ) is bounded and continuous in θ,

$$\frac{1}{T}\sum _{t=1}^T\pi \left({\gamma }_S\mid X_N,{\theta }^{(t)}\right)-\pi \left({\gamma }_S\mid X_N\right)=O_p\left(T^{-1/2}\right),$$ (8)

holds for any $S\in \mathcal{S}$. Combining (8) with (7) leads to

$$\frac{1}{T}\sum _{t=1}^T\pi \left({\gamma }_S\mid X_{n,N}^{(t)},{\theta }^{(t)}\right)-\pi \left({\gamma }_S\mid X_N\right)=O_p\left(T^{-1/2}\right)+\text{exp}(-N\delta /2)+1/{\left|\mathcal{S}\right|}^{\nu }\to 0.$$ (9)

Conditioned on $X_{n,N}^{\left(t\right)}$ and θ^(t) by the standard theory of MCMC, $m^{-1}{\sum }_{i=1}^{m}I\left({\gamma }_S^{\left(t,i\right)}={\gamma }_S\right)$ forms a consistent estimator of $\pi \left({\gamma }_S\mid X_{n,N}^{\left(t\right)},{\theta }^{\left(t\right)}\right)$ with an asymptotic bias of O(1/m). Since m is increasing with p and N, the estimator is asymptotically unbiased. Combining this result with (9) leads to

$$\frac{1}{mT}\sum _{t=1}^T\sum _{i=1}^{m}I\left({\gamma }_{S_i,n}^{(t)}={\gamma }_S\right)-\pi \left({\gamma }_S\mid X_N\right)=O_p\left(T^{-1/2}\right)+\text{exp}\left(-N\delta /2\right)+1/{\left|\mathcal{S}\right|}^{\nu }+O_p\left(m^{-1/2}\right),$$ (10)

which converges to 0 as T → ∞ and N → ∞. □