PAC-Bayes Bounds on Variational Tempered Posteriors for Markov Models

Abstract

Datasets displaying temporal dependencies abound in science and engineering applications, with Markov models representing a simplified and popular view of the temporal dependence structure. In this paper, we consider Bayesian settings that place prior distributions over the parameters of the transition kernel of a Markov model, and seek to characterize the resulting, typically intractable, posterior distributions. We present a Probably Approximately Correct (PAC)-Bayesian analysis of variational Bayes (VB) approximations to tempered Bayesian posterior distributions, bounding the model risk of the VB approximations. Tempered posteriors are known to be robust to model misspecification, and their variational approximations do not suffer the usual problems of over confident approximations. Our results tie the risk bounds to the mixing and ergodic properties of the Markov data generating model. We illustrate the PAC-Bayes bounds through a number of example Markov models, and also consider the situation where the Markov model is misspecified.

1. Introduction

This paper presents probably approximately correct (PAC)-Bayesian bounds on variational Bayesian (VB) approximations of fractional or tempered posterior distributions for Markov data generation models. Exact computation of either standard or tempered posterior distributions is a hard problem that has, broadly speaking, spawned two classes of computational methods. The first, Markov chain Monte Carlo (MCMC), constructs ergodic Markov chains to approximately sample from the posterior distribution. MCMC is known to suffer from high variance and complex diagnostics, leading to the development of variational Bayesian (VB) [1] methods as an alternative in recent years. VB methods pose posterior computation as a variational optimization problem, approximating the posterior distribution of interest by the ‘closest’ element of an appropriately defined class of ‘simple’ probability measures. Typically, the measure of closeness used by VB methods is the Kullback–Leibler (KL) divergence. Excellent introductions to this so-called KL-VB method can be found in [2,3,4]. More recently, there has also been interest in alternative divergence measures, particularly the $\alpha$-Rényi divergence [5,6,7], though in this paper, we focus on the KL-VB setting.

Theoretical properties of VB approximations, and in particular asymptotic frequentist consistency, have been studied extensively under the assumption of an independent and identically distributed (i.i.d.) data generation model [4,8,9]. On the other hand, the common setting where data sets display temporal dependencies presents unique challenges. In this paper, we focus on homogeneous Markov chains with parameterized transition kernels, representing a parsimonious class of data generation models with a wide range of applications. We work in the Bayesian framework, focusing on the posterior distribution over the unknown parameters of the transition kernel. Our theory develops PAC bounds that link the ergodic and mixing properties of the data generating Markov chain to the Bayes risk associated with approximate posterior distributions.

Frequentist consistency of Bayesian methods, in the sense of concentration of the posterior distribution around neighborhoods of the ‘true’ data generating distribution, have been established in significant generality, in both the i.i.d. [10,11,12] and in the non-i.i.d. data generation setting [13,14]. More recent work [14,15,16] has studied fractional or tempered posteriors, a class of generalized Bayesian posteriors obtained by combining the likelihood function raised to a fractional power with an appropriate prior distribution using Bayes’ theorem. Tempered posteriors are known to be robust against model misspecification: in the Markov setting we consider, the associated stationary distribution as well as mixing properties are sensitive to model parameterization. Further, tempered posteriors are known to be much simpler to analyze theoretically [14,16]. Therefore, following [14,15,16] we focus on tempered posterior distributions on the transition kernel parameters, and study the rate of concentration of variational approximations to the tempered posterior. Equivalently, as shown in [16] and discussed in Section 1.1, our results also apply to so-called $\alpha$-variational approximations to standard posterior distributions over kernel parameters. The latter are modifications of the standard KL-VB algorithm to address the well-known problem of overconfident posterior approximations.

While there have been a number of recent papers studying the consistency of approximate variational posteriors [5,8,15] in the large sample limit, rates of convergence have received less attention. Exceptions include [9,15,17], where an i.i.d. data generation model is assumed. [15] establishes PAC-Bayes bounds on the convergence of a variational tempered posterior with fractional powers in the range $[0, 1)$, while [9] considers the standard variational posterior case (where the fractional power equals 1). [17], on the other hand, establishes PAC-Bayes bounds for risk-sensitive Bayesian decision making problems in the standard variational posterior setting. The setting in [15] allows for model misspecification and the analysis is generally more straightforward than that in [9,17]. Our work extends [15] to the setting of a discrete-time Markov data generation model.

Our first results in Theorem 1 and Corollary 1 of Section 2 establish PAC-Bayes bounds for sequences with arbitrary temporal dependence. Our results generalize [15], [Theorem 2.4] to the non-i.i.d. data setting in a straightforward manner. Note that Theorem 1 also recovers ([16], [Theorem 3.3]), which is established under different ‘existence of test’ conditions. Our objective in this paper is to explicate how the ergodic and mixing properties of the Markov data generating process influences the PAC-Bayes bound. The sufficient conditions of our theorem, bounding the mean and variance of the log-likelihood ratio of the data, allows for developing this understanding, without the technicalities of proving the existence of test conditions intruding on the insights.

In Section 3, we study the setting where the data generating model is a stationary $\alpha$-mixing Markov chain. Stationarity means that the Markov chain is initialized with the invariant distribution corresponding to the parameterized transition kernel, implying all subsequent states also follow this marginal distribution. The $\alpha$-mixing condition ensures that the variance of the likelihood ratio of the Markov data does not grow faster than linear in the sample size. Our main results in this setting are applicable when the state space of the Markov chain is either continuous or discrete. The primary requirement on the class of data generating Markov models is for the log-likelihood ratio of the parameterized transition kernel and invariant distribution to satisfy a Lipschitz property. This condition implies a decoupling between the model parameters and the random samples, affording a straightforward verification of the mean and variance bounds. We highlight this main result by demonstrating that it is satisfied by a finite state Markov chain, a birth-death Markov chain on the positive integers, and a one-dimensional Gaussian linear model.

In practice, the assumption that the data generating model is stationary is unlikely to be satisfied. Typically, the initial distribution is arbitrary, with the state distribution of the Markov sequence converging weakly to the stationary distribution. In this setting, we must further assume that the class of data generating Markov chains are geometrically ergodic. We show that this implies the boundedness of the mean and variance of the log-likelihood ratio of the data generating Markov chain. Alternatively, in Theorem 4 we directly impose a drift condition on random variables that bound the log-likelihood ratio. Again, in this more general nonstationary setting, we illustrate the main results by showing that the PAC-Bayes bound is satisfied by a finite state Markov chain, a birth-death Markov chain on the positive integers, and a one-dimensional Gaussian linear model.

In preparation for our main technical results starting in Section 2 we first note relevant notations and definitions in the next section.

1.1. Notations and Definitions

We broadly adopt the notation in [15]. Let the sequence of random variables $X^n=(X_0,\dots ,X_n)\subset {\mathbb{R}}^{m\times (n+1)}$ represent a dataset of $n+1$ observations drawn from a joint distribution $P_{{\theta }_0}^{\left(n\right)}$, where ${\theta }_0\in \mathsf{\Theta }\subseteq {\mathbb{R}}^d$ is the ‘true’ parameter underlying the data generation process. We assume the state space $S\subseteq {\mathbb{R}}^m$ of the random variables $X_i$ is either discrete-valued or continuous, and write $\{x_0,\cdots ,x_n\}$ for a realization of the dataset. We also adopt the convention that $0log(0/0)=0$.

For each $\theta \in \mathsf{\Theta }$, we will write $p_{\theta }^{\left(n\right)}$ as the probability density of $P_{\theta }^{\left(n\right)}$ with respect to some measure $Q^{\left(n\right)}$, i.e., $p_{\theta }^{\left(n\right)}:=\frac{d{P}_{\theta }^{\left(n\right)}}{d{Q}^{\left(n\right)}}$, where $Q^{\left(n\right)}$ is either Lebesgue measure or the counting measure. Unless stated otherwise, all probabilities, expectations and variances, which we represent as P, $\mathrm{E}\left[X\right]$ and $\mathrm{Var}\left[X\right]$, are with respect to the true distribution $P_{{\theta }_0}^{\left(n\right)}$.

Let $\pi \left(\theta \right)$ be a prior distribution with support $\mathsf{\Theta }$. The ${\alpha }^{te}$-fractional posterior is defined as

$$\begin{array}{cc}\hfill {\pi }_{n,{\alpha }^{te}|X^n}\left(d\theta \right)& :=\frac{e^{-{\alpha }^{te}{r}_n(\theta ,{\theta }_0)\left(X^n\right)}\pi \left(d\theta \right)}{\int e^{-{\alpha }^{te}{r}_n(\theta ,{\theta }_0)\left(X^n\right)}\pi \left(d\theta \right)},\hfill \end{array}$$ (1)

where, for ${\theta }_0,\theta \in \mathsf{\Theta }$, $r_n(\theta ,{\theta }_0)(·):=log\left(\frac{p_{{\theta }_0}^{\left(n\right)}(·)}{p_{\theta }^{\left(n\right)}(·)}\right)$, is the log-likelihood ratio of the corresponding density functions, and ${\alpha }^{te}\in (0,\mathsf{\infty })$ is a tempering coefficient. Setting ${\alpha }^{te}=1$ recovers the standard Bayesian posterior. Note that we will use superscripts to distinguish different quantities that are referred to just as $\alpha$ in the literature.

The Kullback–Leibler (KL) divergence between distributions $P,Q$ is defined as

$$\begin{array}{c}\hfill \mathcal{K}(P,Q):={\int }_{\mathcal{X}}log\left(\frac{p\left(x\right)}{q\left(x\right)}\right)p\left(x\right)dx,\end{array}$$

where $p,q$ are the densities corresponding to $P,Q$ on some sample space $\mathcal{X}$. In particular, the KL divergence between the distributions parameterized by ${\theta }_0$ and $\theta$ is

$$\begin{array}{cc}\hfill \mathcal{K}(P_{{\theta }_0}^{\left(n\right)},P_{\theta }^{\left(n\right)})& :=\int log\left(\frac{p_{{\theta }_0}^{\left(n\right)}(x_0,\cdots ,x_n)}{p_{\theta }^{\left(n\right)}(x_0,\cdots ,x_n)}\right)p_{{\theta }_0}^{\left(n\right)}(x_0,\cdots ,x_n)d{x}_0\cdots d{x}_n\hfill \\ \hfill & =\int r_n(\theta ,{\theta }_0)(x_0,\cdots ,x_n)p_{{\theta }_0}^n(x_0,\cdots ,x_n)d{x}_0\cdots d{x}_n.\hfill \end{array}$$ (2)

The ${\alpha }^{re}$-Rényi divergence$D_{{\alpha }^{re}}(P_{\theta }^{\left(n\right)},P_{{\theta }_0}^{\left(n\right)})$ is defined as

$$\begin{array}{cc}\hfill D_{{\alpha }^{re}}(P_{\theta }^{\left(n\right)},P_{{\theta }_0}^{\left(n\right)})& :=\frac{1}{{\alpha }^{re}-1}log\int exp\left(-{\alpha }^{re}{r}_n(\theta ,{\theta }_0)(x_0,\cdots ,x_n)\right)p_{{\theta }_0}^{\left(n\right)}(x_0,\cdots ,x_n)d{x}_0\cdots d{x}_n,\hfill \end{array}$$ (3)

where ${\alpha }^{re}\in (0,1)$. As ${\alpha }^{re}\to 1$, the ${\alpha }^{re}$-Rényi divergence recovers the KL divergence.

Let $\mathcal{F}$ be some class of distributions with support in ${\mathbb{R}}^d$ and such that any distribution P in $\mathcal{F}$ is absolutely continuous with respect to the tempered posterior: $P\ll {\pi }_{n,{\alpha }^{te}|X^n}$.

Many choices of $\mathcal{F}$ exist; for instance (see also [15]), $\mathcal{F}$ can be the set of Gaussian measures, denoted ${\mathcal{F}}_{id}^{\mathsf{\Phi }}$:

$$\begin{array}{cc}\hfill {\mathcal{F}}_{id}^{\mathsf{\Phi }}& =\{\mathsf{\Phi }(d\theta ;\mu ,\mathsf{\Sigma }):\mu \in {\mathbb{R}}^d,{\mathsf{\Sigma }}_{d\times d}\in \mathrm{P}.\mathrm{D}.\},\hfill \end{array}$$ (4)

where P.D. references the class of positive definite matrices. Alternately, $\mathcal{F}$ can be the family of mean-field or factored distributions where the components ${\theta }_i$ of $\theta$ are independent of each other. Let ${\tilde{\pi }}_{n,{\alpha }^{te}|X^n}$ be the variational approximation to the tempered posterior, defined as

$$\begin{array}{cc}\hfill {\tilde{\pi }}_{n,{\alpha }^{te}|X^n}& :=\underset{\rho \in \mathcal{F}}{argmin}\mathcal{K}(\rho ,{\pi }_{n,{\alpha }^{te}|X^n})\hfill \end{array}$$ (5)

It is easy to see that finding ${\tilde{\pi }}_{n,{\alpha }^{te}|X^n}$ in Equation (5) is equivalent to the following optimization problem:

$$\begin{array}{cc}\hfill {\tilde{\pi }}_{n,{\alpha }^{te}|X^n}& :=\underset{\rho \in \mathcal{F}}{argmax}\left[\int r_n(\theta ,{\theta }_0)(x_0,\cdots ,x_n)\rho \left(d\theta \right)-{\left({\alpha }^{te}\right)}^{-1}\mathcal{K}(\rho ,\pi )\right].\hfill \end{array}$$ (6)

Setting ${\alpha }^{te}=1$ again recovers the usual variational solution that seeks to approximate the posterior distribution with the closest element of $\mathcal{F}$ (the right-hand side above is called the evidence lower bound (ELBO)). Other settings of ${\alpha }^{te}$ constitute ${\alpha }^{te}$-variational inference [16], which seeks to regularize the ‘overconfident’ approximate posteriors that standard variational methods tend to produce.

Our results in this paper focus on parametrized Markov chains. We term a Markov chain as ‘parameterized’ if the transition kernel $p_{\theta }(·|·)$ is parametrized by some $\theta \in \mathsf{\Theta }\subseteq {\mathbb{R}}^d$. Let $q^{\left(0\right)}(·)$ be the initial density (defined with respect to the Lebesgue measure over ${\mathbb{R}}^m$) or initial probability mass function. Then, the joint density is $p_{\theta }^{\left(n\right)}(x_0,\cdots ,x_n)=q^{\left(0\right)}\left(x_0\right){\prod }_{i=0}^{n-1}{p}_{\theta }\left(x_{i+1}\right|x_i)$; recall, this joint density $p_{\theta }^{\left(n\right)}(x_0,\cdots ,x_n)$ corresponds to the walk probability of a time-homogeneous Markov chain. We assume that corresponding to each transition kernel $p_{\theta },\theta \in \mathsf{\Theta },$ there exists an invariant distribution $q_{\theta }^{\left(\mathsf{\infty }\right)}\equiv q_{\theta }$ that satisfies

$$q_{\theta }\left(x\right)=\int p_{\theta }\left(x\right|y)q_{\theta }\left(dy\right)\forall x\in {\mathbb{R}}^m,\theta \in \mathsf{\Theta }.$$

We will also use $q_{\theta }$ to designate the density of the invariant measure (as before, this is with respect to the Lebesgue or counting measure for continuous or discrete state spaces, respectively). A Markov chain is stationary if its initial distribution is the invariant probability distribution, that is, $X_0\sim q_{\theta }$.

Our results in the ensuing sections will be established under strong mixing conditions [18] on the Markov chain. Specifically, recall the definition of the $\alpha$-mixing coefficients of a Markov chain $\left\{X_n\right\}$:

Definition 1

($\alpha$-mixing coefficient). Let ${\mathcal{M}}_i^j$ denote the σ-field generated by the Markov chain $\{X_k:i\le k\le j\}$ parameterized by $\theta \in \mathsf{\Theta }$. Then, the α-mixing coefficient is defined as

$$\begin{array}{c}\hfill {\alpha }_k=\underset{t>0}{sup}\underset{(A,B)\in {\mathcal{M}}_{-\mathsf{\infty }}^t\times {\mathcal{M}}_{t+k}^{\mathsf{\infty }}}{sup}\left|P_{\theta }(A\cap B)-P_{\theta }\left(A\right)P_{\theta }\left(B\right)\right|.\end{array}$$ (7)

Informally speaking, the $\alpha$-mixing coefficients $\left\{{\alpha }_k\right\}$ measure the dependence between any two events A (in the ‘history’ $\sigma$-algebra) and B (in the ‘future’ $\sigma$-algebra) with a time lag k. We note that we do not use superscripts to identify these $\alpha$ parameters, since they are the only ones with subscripts, and can be identified through this.

2. A Concentration Bound for the ${\mathbf{\alpha }}^{\mathbf{re}}$-Rényi Divergence

The object of analysis in what follows is the probability measure ${\tilde{\pi }}_{n,{\alpha }^{te}|X^n}\left(\theta \right)$, the variational approximation to the tempered posterior. Our main result establishes a bound on the Bayes risk of this distribution; in particular, given a sequence of loss functions ${\ell }_n(\theta ,{\theta }_0)$, we bound $\int {\ell }_n(\theta ,{\theta }_0){\tilde{\pi }}_{n,{\alpha }^{te}|X^n}\left(\theta \right)d\theta$. Following recent work in both the i.i.d. and dependent sequence settings [14,15,16], we will use ${\ell }_n(\theta ,{\theta }_0)=D_{{\alpha }^{re}}(P_{\theta }^{\left(n\right)},P_{{\theta }_0}^{\left(n\right)})$, the ${\alpha }^{re}$-Rényi divergence between $P_{\theta }^{\left(n\right)}$ and $P_{{\theta }_0}^{\left(n\right)}$ as our loss function. Unlike loss functions like Euclidean distance, Rényi divergence compares $\theta$ and ${\theta }_0$ through their effect on observed sequences, so that issues like parameter identifiability no longer arise. Our first result generalizes [15], [Theorem 2.1] to a general non-i.i.d. data setting.

Proposition 1.

Let $\mathcal{F}$ be a subset of all probability distributions on Θ. For any ${\alpha }^{re}\in (0,1)$, $ϵ\in (0,1)$ and $n\ge 1$, the following probabilistic uniform upper bound on the expected ${\alpha }^{re}$-Rényi divergence holds:

$$\begin{array}{c}\hfill P\left[\underset{\rho \in \mathcal{F}}{sup}\int D_{{\alpha }^{re}}(P_{\theta }^{\left(n\right)},P_{{\theta }_0}^{\left(n\right)})\rho \left(d\theta \right)\le \frac{{\alpha }^{re}}{1-{\alpha }^{re}}\int r_n(\theta ,{\theta }_0)\rho \left(d\theta \right)+\frac{\mathcal{K}(\rho ,\pi )+log\left(\frac{1}{ϵ}\right)}{1-{\alpha }^{re}}\right]\ge 1-ϵ.\end{array}$$ (8)

The proof of Proposition 1 follows easily from [15], and we include it in Appendix B.1.1 for completeness. Mirroring the comments in [15], when $\rho ={\tilde{\pi }}_{n,{\alpha }^{te}}$ this result is precisely [14, Theorem 3.4]. We also note from [14] that $\forall {\alpha }^{re},\beta \in (0,1]$${\alpha }^{re}$-Rényi divergences are all equivalent through the following inequality $\frac{{\alpha }^{re}(1-\beta )}{\beta (1-{\alpha }^{re})}{D}_{\beta }\le D_{{\alpha }^{re}}\le D_{\beta }\forall {\alpha }^{re}\le \beta$. Hence, for the subsequent results, we simplify by assuming that ${\alpha }^{te}={\alpha }^{re}$. This probabilistic bound implies the following PAC-Bayesian concentration bound on the model risk computed with respect to the fractional variational posterior:

Theorem 1.

Let $\mathcal{F}$ be a subset of all probability distributions parameterized by Θ, and assume there exist ${ϵ}_n>0$ and ${\rho }_n\in \mathcal{F}$ such that

• i.  $\int \mathcal{K}(P_{{\theta }_0}^{\left(n\right)},P_{\theta }^{\left(n\right)}){\rho }_n\left(d\theta \right)=\int \mathrm{E}\left[r_n(\theta ,{\theta }_0)\right]{\rho }_n\left(d\theta \right)\le n{ϵ}_n$,
• ii.  $\int \mathrm{Var}\left(r_n(\theta ,{\theta }_0)\right){\rho }_n\left(d\theta \right)\le n{ϵ}_n$, and
• iii.  $\mathcal{K}({\rho }_n,\pi )\le n{ϵ}_n$.

Then, for any ${\alpha }^{re}\in (0,1)$ and $(ϵ,\eta )\in (0,1)\times (0,1)$,

$$\begin{array}{c}\hfill P\left[\int D_{{\alpha }^{re}}(P_{\theta }^{\left(n\right)},P_{{\theta }_0}^{\left(n\right)}){\tilde{\pi }}_{n,{\alpha }^{re}}\left(d\theta \right|X^{\left(n\right)})\le \frac{({\alpha }^{re}+1)n{ϵ}_n+{\alpha }^{re}\sqrt{\frac{n{ϵ}_n}{\eta }}-log\left(ϵ\right)}{1-{\alpha }^{re}}\right]\ge 1-ϵ-\eta .\end{array}$$ (9)

The proof of Theorem 1 is a generalization of [15] (Theorem 2.4) to the non-i.i.d. setting, and a special case of [16] (Theorem 3.1), where the problem setting includes latent variables. We include a proof for completeness. As noted in [15], the sufficient conditions follow closely from [13] and we will show that they hold for a variety of Markov chain models.

A direct corollary of Theorem 1 follows by setting $\eta =\frac{1}{n{ϵ}_n}$, $ϵ=e^{-n{ϵ}_n}$ and using the fact that $e^{-n{ϵ}_n}\ge \frac{1}{n{ϵ}_n}$. Note that Equation (9) is vacuous if $\eta +ϵ>1$. Therefore, without loss of generality, we restrict ourselves to the condition $\frac{2}{n{ϵ}_n}<1$.

Corollary 1.

Assume $\exists {ϵ}_n>0$, ${\rho }_n\in \mathcal{F}$ such that the following conditions hold:

• i.  $\int \mathcal{K}(P_{{\theta }_0}^{\left(n\right)},P_{\theta }^{\left(n\right)}){\rho }_n\left(d\theta \right)=\int E\left[r_n(\theta ,{\theta }_0)\right]{\rho }_n\left(d\theta \right)\le n{ϵ}_n$,
• ii.  $\int \mathrm{Var}\left(r_n(\theta ,{\theta }_0)\right){\rho }_n\left(d\theta \right)\le n{ϵ}_n$, and
• iii.  $\mathcal{K}({\rho }_n,\pi )\le n{ϵ}_n$.

Then, for any ${\alpha }^{re}\in (0,1)$,

$$\begin{array}{c}\hfill P\left[\int D_{{\alpha }^{re}}(P_{\theta }^{\left(n\right)},P_{{\theta }_0}^{\left(n\right)}){\tilde{\pi }}_{n,{\alpha }^{re}}\left(d\theta \right|X^{\left(n\right)})\le \frac{2({\alpha }^{re}+1){ϵ}_n}{1-{\alpha }^{re}}\right]\ge 1-\frac{2}{n{ϵ}_n}.\end{array}$$ (10)

We observe that Theorem 1 and Corollary 1 place no assumptions on the nature of the statistical dependence between data points. However, verification of the sufficient conditions is quite hard, in general. One of our key contributions is to verify that under reasonable assumptions on the smoothness of the transition kernel, the sufficient conditions of Theorem 1 and Corollary 1 are satisfied by ergodic Markov chains.

Observe that the first two conditions in Corollary 1 ensure that the distribution ${\rho }_n$ concentrates on parameters $\theta \in \mathsf{\Theta }$ around the true parameter ${\theta }_0$, while the third condition requires that ${\rho }_n$ not diverge from the prior $\pi$ rapidly as a function of the sample size n. In general, verifying the first and third conditions is relatively straightforward. The second condition, on the other hand, is significantly more complicated in the current setting of dependent data, as the variance of $r_n(\theta ,{\theta }_0)$ includes correlations between the observations $\{X_0,\cdots ,X_n\}$. In the next section, we will make assumptions on the transition kernels (and corresponding invariant densities) that ’decouple’ the temporal correlations and the model parameters in the setting of strongly mixing and ergodic Markov chain models, and allow for the verification of the conditions in Corollary 1. Towards this, Propositions 2 and 3 below characterize the expectation and variance of the log-likelihood ratio $r_n(·,·)$ in terms of the one-step transition kernels of the Markov chain. First, consider the expectation of $r_n(·,·)$ in condition (i).

Proposition 2.

Fix ${\theta }_1,{\theta }_2\in \mathsf{\Theta }$ and consider the parameterized Markov transition kernels $p_{{\theta }_1}$ and $p_{{\theta }_2}$, and initial distributions $q_{{\theta }_1}^{\left(0\right)}$ and $q_{{\theta }_2}^{\left(0\right)}$. Let $p_{{\theta }_1}^{\left(n\right)}$ and $p_{{\theta }_2}^{\left(n\right)}$ be the corresponding joint probability densities; that is,

$$p_{{\theta }_j}^{\left(n\right)}(x_0,\dots ,x_n)=q_{{\theta }_j}^{\left(0\right)}\left(x_0\right)\prod _{i=1}^n{p}_{{\theta }_i}\left(x_i\right|x_{i-1})$$ (11)

for $j\in \{1,2\}$. Then, for any $n\ge 1$, the log-likelihood ratio $r_n({\theta }_2,{\theta }_1)$ satisfies

$$\begin{array}{cc}\hfill {\mathrm{E}}_{{\theta }_1}\left[r_n({\theta }_2,{\theta }_1)\right]& =\sum _{i=1}^n{\mathrm{E}}_{{\theta }_1}\left[log\left(\frac{p_{{\theta }_1}\left(X_i\right|X_{i-1})}{p_{{\theta }_2}\left(X_i\right|X_{i-1})}\right)\right]+{\mathrm{E}}_{{\theta }_1}\left[Z_0\right],\hfill \end{array}$$ (12)

where $Z_0:=log\left(\frac{q_{{\theta }_1}^{\left(0\right)}\left(X_0\right)}{q_{{\theta }_2}^{\left(0\right)}\left(X_0\right)}\right)$. The expectation in the first term is with respect to the joint density function $p_{{\theta }_1}(y,x)=p_{{\theta }_1}\left(y\right|x)q_{{\theta }_1}^{(i-1)}\left(x\right)$ where the marginal density satisfies

$$\begin{array}{c}\hfill q_{{\theta }_1}^{(i-1)}\left(x\right)=\left\{\begin{array}{cc}\int p_{{\theta }_1}^{(i-1)}(x_0,\dots ,x_{i-2},x)d{x}_0\cdots d{x}_{i-2}\hfill & fori>1,and\hfill \\ q_{{\theta }_1}^{\left(0\right)}\left(x\right)\hfill & fori=1.\hfill \end{array}\right.\end{array}$$

If the Markov chain is also stationary under ${\theta }_1$, then Equation (12) simplifies to

$$\begin{array}{c}\hfill {\mathrm{E}}_{{\theta }_1}\left[r_n({\theta }_2,{\theta }_1)\right]=n{\mathrm{E}}_{{\theta }_1}\left[log\left(\frac{p_{{\theta }_1}\left(X_1\right|X_0)}{p_{{\theta }_2}\left(X_1\right|X_0)}\right)\right]+{\mathrm{E}}_{{\theta }_1}\left[Z_0\right].\end{array}$$ (13)

Notice that ${\mathrm{E}}_{{\theta }_1}\left[r_n({\theta }_2,{\theta }_1)\right]$ is precisely the KL divergence, $\mathcal{K}(P_{{\theta }_1}^{\left(n\right)},P_{{\theta }_2}^{\left(n\right)})$. Next, the following proposition uses [19] (Lemma 1.3) to upper bound the variance of the log-likelihood ratio.

Proposition 3.

Fix ${\theta }_1,{\theta }_2\in \mathsf{\Theta }$ and consider parameterized Markov transition kernels $p_{{\theta }_1}$ and $p_{{\theta }_2}$, with initial distributions $q_{{\theta }_1}^{\left(0\right)}$ and $q_{{\theta }_2}^{\left(0\right)}$. Let $p_{{\theta }_1}^{\left(n\right)}$ and $p_{{\theta }_2}^{\left(n\right)}$ be the corresponding joint probability densities of the sequence $(x_0,\cdots ,x_n)$, and $q_{{\theta }_j}^{\left(i\right)}$ the marginal density for $i\in \{1,\dots ,n\}$ and $j\in \{1,2\}$. Fix $\delta >0$ and, for each $i\in \{1,\cdots ,n\}$, define

$$\begin{array}{cc}\hfill C_{{\theta }_1,{\theta }_2}^{\left(i\right)}& :=\int {\left|log\left(\frac{p_{{\theta }_1}\left(x_i\right|x_{i-1})}{p_{{\theta }_2}\left(x_i\right|x_{i-1})}\right)\right|}^{2+\delta }{p}_{{\theta }_1}\left(x_i\right|x_{i-1})q_{{\theta }_1}^{(i-1)}\left(x_{i-1}\right)d{x}_{i}d{x}_{i-1}.\hfill \end{array}$$

Similarly, define $Z_0:=log\left(\frac{q_{{\theta }_1}^{\left(0\right)}\left(X_0\right)}{q_{{\theta }_2}^{\left(0\right)}\left(X_0\right)}\right)$, and $D_{1,2}:={\mathrm{E}}_{{\theta }_1}{\left|Z_0\right|}^{2+\delta }.$ Suppose the Markov chain corresponding to ${\theta }_1$ is α-mixing with coefficients $\left\{{\alpha }_k\right\}$. Then,

$$\begin{array}{ccc}\hfill \mathrm{Var}\left(r_n({\theta }_1,{\theta }_2)\right)& <& \sum _{i,j=1}^n\left(\frac{4}{n}+2n^{\delta /2}(C_{{\theta }_1,{\theta }_2}^{\left(i\right)}+C_{{\theta }_1,{\theta }_2}^{\left(j\right)}+\sqrt{C_{{\theta }_1,{\theta }_2}^{\left(i\right)}{C}_{{\theta }_1,{\theta }_2}^{\left(j\right)}})\right)\left({\alpha }_{|i-j|-1}^{\delta /(2+\delta )}\right)\hfill \\ \mathrm{(14)}& & & +\sum _{i=1}^n\left(\frac{4}{n}+2n^{\delta /2}(C_{{\theta }_1,{\theta }_2}^{\left(i\right)}+D_{1,2}+\sqrt{C_{{\theta }_1,{\theta }_2}^{\left(i\right)}{D}_{1,2}})\right)\left({\alpha }_{i-1}^{\delta /(2+\delta )}\right)\hfill \mathrm{(15)}& & & +\mathrm{Cov}(Z_0,Z_0).\hfill \end{array}$$

Note that this result holds for any parameterized Markov chain. In particular, when the Markov chain is stationary, $C_{{\theta }_1,{\theta }_2}^{\left(i\right)}=C_{{\theta }_1,{\theta }_2}^{\left(1\right)}\forall i$ and $\forall \theta \in \mathsf{\Theta }$, and Equation (14) simplifies to

$$\begin{array}{cc}\hfill \mathrm{Var}\left(r_n({\theta }_1,{\theta }_2)\right)& <n\left(\frac{4}{n}+6n^{\delta /2}{C}_{{\theta }_1,{\theta }_2}^{\left(1\right)}\right)\left(\sum _{k\ge 0}{\alpha }_k^{\delta /(2+\delta )}\right)\hfill \\ \hfill & +\left(\frac{4}{n}+2n^{\delta /2}(C_{{\theta }_1,{\theta }_2}^{\left(1\right)}+D_{1,2}+\sqrt{C_{{\theta }_1,{\theta }_2}^{\left(1\right)}{D}_{1,2}})\right)\left(\sum _{k\ge 1}{\alpha }_k^{\delta /(2+\delta )}\right)\hfill \\ \hfill & +\mathrm{Cov}(Z_0,Z_0).\hfill \end{array}$$ (16)

If the sum ${\sum }_{k\ge 0}{\alpha }_k^{\delta /(2+\delta )}$ is infinite, the bound is trivially true. For it to be finite, of course, the coefficients ${\alpha }_k$ must decay to zero sufficiently quickly. For instance, Theorem A.1.2 shows that if the Markov chain is geometrically ergodic, then the $\alpha$-mixing coefficients are geometrically decreasing. We will use this fact when the Markov chain is non-stationary, as in Section 4. In the next section, however, we first consider the simpler stationary Markov chain setting where geometric ergodic conditions are not explicitly imposed. We also note that unless only a finite number of ${\alpha }_k$ are nonzero, the sum ${\sum }_{k\ge 0}{\alpha }_k^{\delta /(2+\delta )}$ is infinite when $\delta =0$, and our results will typically require $\delta >0$.

3. Stationary Markov Data-Generating Models

Observe that the PAC-Bayesian concentration bound in Corollary 1 specifically requires bounding the mean and variance of the log-likelihood ratio $r_n(\theta ,{\theta }_0)$. We ensure this by imposing regularity conditions on the log-ratio of the one-step transition kernels and the corresponding invariant densities. Specifically, we assume the following conditions that decouple the model parameters from the random samples, allowing us to verify the bounds in Corollary 1.

Assumption 1.

There exist positive functions $M_k^{\left(1\right)}(·,·)$ and $M_k^{\left(2\right)}(·)$, $k\in \{1,2,\cdots ,m\}$ such that for any parameters ${\theta }_1,{\theta }_2\in \mathsf{\Theta }$, the log of the ratio of one-step transition kernels and the log of the ratio of the invariant distributions satisfy, respectively,

$$\begin{array}{c}\hfill |log{p}_{{\theta }_1}\left(x_1\right|x_0)-log{p}_{{\theta }_2}\left(x_1\right|x_0)|\le \sum _{k=1}^m{M}_k^{\left(1\right)}(x_1,x_0)\left|f_k^{\left(1\right)}({\theta }_2,{\theta }_1)\right|\forall (x_0,x_1),and\end{array}$$ (17)

$$\begin{array}{c}\hfill |log{q}_{{\theta }_1}\left(x\right)-log{q}_{{\theta }_2}\left(x\right)|\le \sum _{k=1}^m{M}_k^{\left(2\right)}\left(x\right)\left|f_k^{\left(2\right)}({\theta }_2,{\theta }_1)\right|\forall x.\end{array}$$ (18)

We further assume that for some $\delta >0$, the functions $f_k^{\left(1\right)},f_k^{\left(2\right)}$ and $M_k^{\left(1\right)}$ satisfy the following:

• i  there exist constants $C_k^{\left(t\right)}$ and measures ${\rho }_n\in \mathcal{F}$ such that $\int |f_k^{\left(t\right)}(\theta ,{\theta }_0){|}^{2+\delta }{\rho }_n\left(d\theta \right)<\frac{C_k^{\left(t\right)}}{n}$ for $t\in \{1,2\}$, $n\ge 1$ and $k\in \{1,2,\cdots ,m\}$, and
• ii  there exists a constant B such that $\int M_k^{\left(1\right)}{(x_1,x_0)}^{2+\delta }{p}_{{\theta }_j}\left(x_1\right|x_0)q_{{\theta }_j}^{\left(0\right)}\left(x_0\right)d{x}_{1}d{x}_0<B,k\in \{1,\cdots ,m\}$ and $j\in \{1,2\}$.

The following examples illustrate Equations (17) and (18) for discrete and continuous state Markov chains.

Example 1.

Suppose $\{X_0,\cdots ,X_n\}$ is generated by the birth-death chain with parameterized transition probability mass function,

$$\begin{array}{c}\hfill p_{\theta }\left(j\right|i)=\left\{\begin{array}{cc}\theta \hfill & ifj=i-1,\hfill \\ 1-\theta \hfill & ifj=i+1.\hfill \end{array}\right.\end{array}$$

In this example, the parameter θ denotes the probability of birth. We shall see that, $m=3$: $M_1^{\left(1\right)}(X_1,X_0)=I_{[X_1=X_0+1]}$, $M_2^{\left(1\right)}(X_1,X_0)=I_{[X_1=X_0-1]}$, and $M_3^{\left(1\right)}(X_1,X_0)=1$. We also define $M_1^{\left(2\right)}\left(X_0\right)=1$, and set $M_2^{\left(2\right)}\left(X_0\right)$ and $M_3^{\left(2\right)}\left(X_0\right)$ both to $X_0-1$. Let $f_1^{\left(1\right)}(\theta ,{\theta }_0)=log\left[\frac{{\theta }_0}{\theta }\right]$, $f_2^{\left(1\right)}(\theta ,{\theta }_0)=log\left[\frac{1-{\theta }_0}{1-\theta }\right]$, $f_3^{\left(1\right)}(\theta ,{\theta }_0)=0$, $f_1^{\left(2\right)}(\theta ,{\theta }_0)=-f_3^{\left(2\right)}(\theta ,{\theta }_0)=log\left[\frac{1-{\theta }_0}{1-\theta }\right]$, and $f_2^{\left(2\right)}(\theta ,{\theta }_0)=log\left[\frac{{\theta }_0}{\theta }\right]$. The derivation of these terms and that they satisfy the conditions of Assumption 1 is provided in the proof of Proposition 6.

Example 2.

Suppose $\{X_0,\cdots ,X_n\}$ is generated by the ‘simple linear’ Gauss–Markov model

$$X_n=\theta X_{n-1}+W_n,$$

where $\left\{W_n\right\}$ is a sequence of i.i.d. standard Gaussian random variables. Then, $m=2$, with $M_1^{\left(1\right)}(X_n,X_{n-1})=\left|X_n{X}_{n-1}\right|$, $M_2^{\left(1\right)}(X_n,X_{n-1})=X_n^2,M_1^{\left(2\right)}\left(x\right)=\frac{x^2}{2}$ and $M_2^{\left(2\right)}\left(X\right)=0$. Corresponding to these, we have $f_1^{\left(1\right)}(\theta ,{\theta }_0)=(\theta -{\theta }_0),f_2^{\left(1\right)}(\theta ,{\theta }_0)=({\theta }_0^2-{\theta }^2),f_1^{\left(2\right)}({\theta }_0,{\theta }_0)=({\theta }_0^2-{\theta }^2)$ and $f_2^{\left(2\right)}({\theta }_0,{\theta }_0)=0$. The derivation of these quantities and that these satisfy the conditions of Assumption 1 under appropriate choice of ${\rho }_n$ is shown in the proof of Proposition 10.

Note that assuming the same number m of $M_k^{\left(1\right)}$ and $M_k^{\left(2\right)}$ involves no loss of generality, since these functions can be set to 0. Both Equations (17) and (18) can be viewed as generalized Lipschitz-smoothness conditions, recovering the usual Lipschitz-smoothness when $m=1$ and when $f_k^{\left(t\right)}$ is Euclidean distance. Our generalized conditions are useful for distributions like the Gaussian, where Lipschitz smoothness does not apply. From Jensen’s inequality we have $\int |f_k^{\left(t\right)}(\theta ,{\theta }_0)\left|{\rho }_n\left(d\theta \right)\right|\le {\left[\int |f_k^{\left(t\right)}(\theta ,{\theta }_0){|}^{2+\delta }{\rho }_n\left(d\theta \right)\right]}^{\frac{1}{2+\delta }}$, and Assumption 1(i) above implies that for some constant $C>0$ and $k\in \{1,2,\cdots ,m\},t\in \{1,2\}$,

$$\begin{array}{cc}\hfill \int |f_k^{\left(t\right)}(\theta ,{\theta }_0)|{\rho }_n\left(d\theta \right)& \le \frac{C}{n^{1/(2+\delta )}}<\frac{C}{\sqrt{n}}.\hfill \end{array}$$ (19)

Assumption 1(i) is satisfied in a variety of scenarios, for example, under mild assumptions on the partial derivatives of the functions $f_k^{\left(t\right)}$. To illustrate this, we present the following proposition.

Proposition 4.

Let $f(\theta ,{\theta }_0)$ be a function on a bounded domain with bounded partial derivatives with $f({\theta }_0,{\theta }_0)=0$. Let $\left\{{\rho }_n(·)\right\}$ be a sequence of probability densities on θ such that ${\mathrm{E}}_{{\rho }_n}\left[\theta \right]={\theta }_0$ and ${\mathrm{Var}}_{{\rho }_n}\left[\theta \right]=\frac{{\sigma }^2}{n}$ for some $\sigma >0$. Then, for some $C>0$,

$$\int |f(\theta ,{\theta }_0){|}^{2+\delta }{\rho }_n\left(d\theta \right)<\frac{C}{n}.$$ (20)

Proof. 

Define ${\partial }_{\theta }f(\theta ,{\theta }_0):=\frac{\partial f(\theta ,{\theta }_0)}{\partial \theta }$ as the partial derivative of the function f. By the mean value theorem, $|f(\theta ,{\theta }_0)|=|\theta -{\theta }_0\left|\right|{\partial }_{\theta }f({\theta }^{*},{\theta }_0)|$, for some ${\theta }^{*}\in [min\{\theta ,{\theta }_0\},max\{\theta ,{\theta }_0\}]$. Since the partial derivatives are bounded, there exists $L\in \mathbb{R}$ such that ${\partial }_{\theta }f({\theta }^{*},{\theta }_0)<L$, and $\int |f(\theta ,{\theta }_0){|}^{2+\delta }{\rho }_n\left(d\theta \right)<L^{2+\delta }\int {|\theta -{\theta }_0|}^{2+\delta }{\rho }_n\left(d\theta \right)$. Choose $G>0$ be such that $\left|\theta \right|<G$, then ${\left|\frac{\theta -{\theta }_0}{2G}\right|}^{2+\delta }<{\left|\frac{\theta -{\theta }_0}{2G}\right|}^2$. Therefore, $\int |\theta -{\theta }_0{|}^{2+\delta }{\rho }_n\left(d\theta \right)<{\left(2G\right)}^{2+\delta }\mathrm{Var}\left[\frac{\theta }{2G}\right]<{\left(2G\right)}^{\delta }\frac{{\sigma }^2}{n}$. Now choosing ${\left(2G\right)}^{\delta }{\sigma }^2$ as C completes the proof. □

If ${\partial }_{\theta }{f}_k^{\left(t\right)}$ is continuous and $\mathsf{\Theta }$ is compact, then ${\partial }_{\theta }{f}_k^{\left(t\right)}$ is always bounded. Furthermore, observe that if $\mathrm{E}\left[M_k^{\left(1\right)}{(X_1,X_0)}^{2+\delta }\right]<B$, without loss of generality we can use Jensen’s inequality to conclude that, for all $0<a<2+\delta$, $\mathrm{E}\left[M_k^{\left(1\right)}{(X_1,X_0)}^a\right]<B^{\frac{a}{2+\delta }}<B$.

We can now state the main theorem of this section.

Theorem 2.

Let $\{X_0,\dots ,X_n\}$ be generated by a stationary, α-mixing Markov chain parametrized by ${\theta }_0\in \mathsf{\Theta }$. Suppose that Assumption 1 holds and that the α-mixing coefficients satisfy ${\sum }_{k\ge 1}{\alpha }_k^{\delta /(2+\delta )}<+\mathsf{\infty }$. Furthermore, assume that $\mathcal{K}({\rho }_n,\pi )\le \sqrt{n}C$ for some constant $C>0$. Then, the conditions of Corollary 1 are satisfied with ${ϵ}_n=\mathrm{O}\left(max(\frac{1}{\sqrt{n}},\frac{n^{\delta /2}}{n})\right)$.

Theorem 2 is satisfied by a large class of Markov chains, including chains with countable and continuous state spaces. In particular, if the Markov chain is geometrically ergodic, then it follows from Equation (A4) (in the appendix) that ${\sum }_{k\ge 1}{\alpha }_k^{\delta /(2+\delta )}<+\mathsf{\infty }$. Observe that in order to achieve $O\left(\frac{1}{\sqrt{n}}\right)$ convergence, we need $\delta \le 1$. Key to the proof of Theorem 2 is the fact that the variance of the log-likelihood ratio can be controlled via the application of Assumption 1 and Proposition 3. Note also that as $\delta$ decreases, satisfying the condition ${\sum }_{k\ge 1}{\alpha }_k^{\delta /(2+\delta )}$ requires the Markov chain to be faster mixing.

We now illustrate Theorem 2 for a number of Markov chain models. First, consider a birth-death Markov chain on a finite state space.

Proposition 5.

Suppose the data-generating process is a birth-death Markov chain, with one-step transition kernel parametrized by the birth probability ${\theta }_0\in \mathsf{\Theta }$. Let $\mathcal{F}$ be the set of all Beta distributions. We choose the prior to be a Beta distribution. Then, the conditions of Theorem 2 are satisfied and ${ϵ}_n=\mathrm{O}\left(\frac{1}{\sqrt{n}}\right)$.

Proof. 

The proof of Proposition 5 follows from the more general Proposition 8, by fixing the initial distribution to the invariant distribution under ${\theta }_0$. Therefore it has been omitted. We simply refer to the proof of Proposition 8 under a more general setup in Appendix B.3. □

The birth-death chain on the finite state space is, of course, geometrically ergodic and the $\alpha$-mixing coefficients ${\alpha }_k$ decay geometrically. Note that the invariant distribution of this Markov chain is uniform over the state space, and consequently this is a particularly simple example. A more complicated and more realistic example is a birth-death Markov chain on the nonnegative integers. We note that if the probability of birth $\theta$ in a birth-death Markov chain on positive integers is greater than $0.5$, then the Markov chain is transient, and consequently, not ergodic. Hence, our prior should be chosen to have support within $(0,0.5)$. For that purpose, we define the class of scaled beta distributions.

Definition 2

(Scaled Beta). If X is a beta distribution on with parameters a and b, then Y is said to be a scaled beta distribution with same parameters on the interval $(c,m+c)$ if

$$\begin{array}{cc}\hfill Y& =mx+c;(m,c)\in {\mathbb{R}}^2\hfill \end{array}$$

and in that case, the pdf of Y is obtained as

$$\begin{array}{c}\hfill f\left(y\right)=\left\{\begin{array}{cc}\frac{1}{m\mathrm{Beta}(a,b)}{\left(\frac{y-c}{m}\right)}^{a-1}{\left(1-\frac{y-c}{m}\right)}^{b-1}\hfill & \mathrm{if}y\in (c,m+c),\hfill \\ 0\hfill & \mathrm{otherwise}.\hfill \end{array}\right..\end{array}$$

Here, $\mathrm{E}\left[Y\right]=m\frac{a}{a+b}+c$ and $\mathrm{Var}\left[Y\right]=m^2\frac{ab}{{(a+b)}^2(a+b+1)}$. For the birth-death chain, we set $m=0.5$ and $c=0$ giving it support on $(0,\frac{1}{2})$. Setting $m=2$ and $c=-1$ gives a beta distribution rescaled to have support on $(-1,1)$.

Proposition 6.

Suppose the data-generating process is a positive recurrent birth-death Markov chain on the positive integers parameterized by the birth probability ${\theta }_0\in (0,\frac{1}{2})$. Further let $\mathcal{F}$ be the set of all Beta distributions rescaled to have support $(0,\frac{1}{2})$. We choose the prior to be a scaled Beta distribution on $(0,1/2)$ with parameters a and b. Then, the conditions of Theorem 2 are satisfied with ${ϵ}_n=\mathrm{O}\left(\frac{1}{\sqrt{n}}\right)$.

Proof. 

The proof of Proposition 6 (for the stationary case) follows from the more general Proposition 9 (the nonstationary case) by fixing the initial distribution to the invariant distribution under ${\theta }_0$. We omit the proof and simply refer to the proof of Proposition 9 under a more general setup in Appendix B.3. □

Unlike with the finite state-space, the invariant distribution now depends on the parameter $\theta \in \mathsf{\Theta }$, and verification of the conditions of the proposition is more involved. In Appendix A.2, we prove that the class of scaled beta distributions satisfy the condition $\mathcal{K}({\rho }_n,\pi )\le n{ϵ}_n$ when the prior $\pi$ is a beta or an uniform distribution. This fact will allow us to prove the above propositions.

Both Proposition 5 and Proposition 6 assume a discrete state space. The next example considers a strictly stationary simple linear model (as defined in Example 2), which has a continuous, unbounded state space.

Proposition 7.

Suppose the data-generating model is a stationary simple linear model:

$$\begin{array}{cc}\hfill X_n& ={\theta }_0{X}_{n-1}+W_n,\hfill \end{array}$$ (21)

where $\left\{W_n\right\}$ are i.i.d. standard Gaussian random variables and $|{\theta }_0|<1$. Suppose that $\mathcal{F}$ is the class of all beta distributions rescaled to have the support $(-1,1)$. Then, the conditions of Theorem 2 are satisfied with ${ϵ}_n=\mathrm{O}\left(\frac{1}{\sqrt{n}}\right)$.

Proof. 

This is a special case of the more general non-stationary simple linear model which is detailed in Proposition 10. Therefore, the proof of the fact that the simple linear model satisfies Assumption 1 when starting from stationarity is deferred to the proof of Proposition 10. The simple linear model with $|{\theta }_0|<1$ has geometrically decreasing (and therefore summable) $\alpha$-mixing coefficients as a consequence of [20] (eq. (15.49)) and Theorem A.1.2. Combining these two facts, it follows that the conditions of Theorem 2 are satisfied. □

Observe that Theorem 1 (and Corollary 1) are general, and hold for any dependent data-generating process. Therefore, there can be Markov chains that satisfy these, but do not satisfy Assumption 1 which entails some loss of generality. However, as our examples demonstrate, common Markov chain models do indeed satisfy the latter assumption.

4. Non-Stationary, Ergodic Markov Data-Generating Models

We call a time-homogeneous Markov chain non-stationary if the initial distribution $q^{\left(0\right)}$ is not the invariant distribution. There are two sets of results in this setting: in Theorem 3 and Theorem 4 we explicitly impose the $\alpha$-mixing condition, while in Theorem 5 we impose a f-geometric ergodicity condition (Definition A.1.2 in the appendix). As seen in Equation (A4) (in the appendix) if the Markov chain is also geometrically ergodic, then $\forall \delta >0$, $\sum {\alpha }_k^{\delta /(2+\delta )}<\mathsf{\infty }$. This condition can be relaxed, albeit at the risk of more complicated calculations that, nonetheless, mirror those in the geometrically ergodic setting. A common thread through these results is that we must impose some integrability or regularity conditions on the functions $M_k^{\left(1\right)}$.

First, in Theorem 3 we assume that the $M_k^{\left(1\right)}$ functions in Assumption 1 are uniformly bounded and that the $\alpha$-mixing condition is satisfied. This result holds for both discrete and continuous state space settings.

Theorem 3.

Let $\{X_0,\cdots ,X_n\}$ be generated by an α-mixing Markov chain parametrized by ${\theta }_0\in \mathsf{\Theta }$ with transition probabilities satisfying Assumption 1 and with known initial distribution $q^{\left(0\right)}$. Let $\left\{{\alpha }_k\right\}$ be the α-mixing coefficients under ${\theta }_0$, and assume that ${\sum }_{k\ge 1}{\alpha }_k^{\delta /(2+\delta )}<+\mathsf{\infty }$. Suppose that there exists $B\in \mathbb{R}$ such that ${sup}_{x,y}\left|M_k^{\left(1\right)}(x,y)\right|<B$ for all $k\in \{1,2,\cdots ,m\}$ in Assumption 1. Furthermore, assume that there exists ${\rho }_n\in \mathcal{F}$ such that $\mathcal{K}({\rho }_n,\pi )\le \sqrt{n}C$ for some constant $C>0$. If the initial distribution $q^{\left(0\right)}$ satisfies $E_{q^{\left(0\right)}}{\left|M_k^{\left(2\right)}\left(X_0\right)\right|}^2<+\mathsf{\infty }$ for all $k\in \{1,2,\cdots ,m\}$, then the conditions of Corollary 1 are satisfied with ${ϵ}_n=\mathrm{O}\left(max(\frac{1}{\sqrt{n}},\frac{n^{\delta /2}}{n})\right)$.

The following result in Proposition 8 illustrates Theorem 3 in the setting of a finite state birth-death Markov chain.

Proposition 8.

Suppose the data-generating process is a finite state birth-death Markov chain, with one-step transition kernel parametrized by the birth probability ${\theta }_0$. Let $\mathcal{F}$ be the set of all Beta distributions. We choose the prior on ${\theta }_0$ to be a Beta distribution. Then, the conditions of Theorem 3 are satisfied with ${ϵ}_n=\mathrm{O}\left(\frac{1}{\sqrt{n}}\right)$ for any initial distribution $q^{\left(0\right)}$.

Theorem 3 also applies to data generated by Markov chains with countably infinite state spaces, so long as the class of data-generating Markov chains is strongly ergodic and the initial distribution has finite second moments. The following example demonstrates this in the setting of a birth-death Markov chain on the positive integers, where the initial distribution is assumed to have finite second moments.

Proposition 9.

Suppose the data-generating process is a birth-death Markov chain on the non-negative integers, parameterized by the probability of birth ${\theta }_0\in (0,\frac{1}{2})$. Further let $\mathcal{F}$ be the set of all Beta distributions rescaled upon the support $(0,\frac{1}{2})$. Let $q^{\left(0\right)}$ be a probability mass function on non-negative integers such that ${\sum }_{i=1}^{\mathsf{\infty }}{i}^2{q}^{\left(0\right)}\left(i\right)<+\mathsf{\infty }$. We choose the prior to be a scaled Beta distribution on $(0,1/2)$ with parameters a and b. Then, the conditions of Theorem 3 are satisfied with ${ϵ}_n=\mathrm{O}\left(\frac{1}{\sqrt{n}}\right)$.

Since continuous functions on a compact domain are bounded, we have the following (easy) corollary (stated without proof).

Corollary 2.

Let $\{X_0,\cdots ,X_n\}$ be generated by an α-mixing Markov chain parametrized by ${\theta }_0\in \mathsf{\Theta }$ on a compact state space, and with initial distribution $q^{\left(0\right)}$. Suppose the α-mixing coefficients satisfy ${\sum }_{k\ge 1}{\alpha }_k^{\delta /(2+\delta )}<+\mathsf{\infty }$, and that Assumption 1 holds with continuous functions $M_k^{\left(1\right)}(·,·)$, $k\in \{1,2,\cdots ,m\}$. Furthermore, assume that there exists ${\rho }_n$ such that $\mathcal{K}({\rho }_n,\pi )\le \sqrt{n}C$ for some constant C. Then, Theorem 3 is satisfied with ${ϵ}_n=\mathrm{O}\left(max(\frac{1}{\sqrt{n}},\frac{n^{\delta /2}}{n})\right)$.

In general, the $M_k^{\left(1\right)}$ functions will not be uniformly bounded (consider the case of the Gauss–Markov simple linear model in Example 2), and stronger conditions must be imposed on the data-generating Markov chain itself. The following assumption imposes a ‘drift’ condition from [21]. Specifically, [21] (Theorem 2.3) shows that under the conditions of Assumption 2, the moment generating function of an aperiodic Markov chain $\left\{X_n\right\}$ can be upper bounded by a function of the moment generating function of $X_0$. Together with the $\alpha$-mixing condition, Assumption 2 implies that this Markov data generating process satisfies Corollary 1.

Assumption 2.

Consider a Markov chain $\left\{X_n\right\}$ parameterized by ${\theta }_0\in \mathsf{\Theta }$. Let ${\mathcal{M}}_{-\mathsf{\infty }}^n$ denote the σ-field generated by $\{X_{-\mathsf{\infty }},\cdots ,X_{n-1},X_n\}$. Denote the stochastic process $\left\{M_n^k\right\}:=\left\{M_k^{\left(1\right)}(X_n,X_{n-1})\right\}$; recall $M_k^{\left(1\right)}$, for each $k=1,\dots ,m_1$, are defined in Assumption 1. For each $k=1,\dots ,m$, assume the process $\left\{M_n^k\right\}$ satisfies the following conditions:

• The drift condition holds for $\left\{M_n^k\right\}$, i.e., $\mathrm{E}\left[M_n^k-M_{n-1}^k|{\mathcal{M}}_{-\mathsf{\infty }}^{n-1},M_{n-1}^k>a\right]\le -ϵ$ for some $ϵ,a>0$.

• For some $\lambda >0$ and $\mathcal{D}>0$, $\mathrm{E}\left[e^{\lambda (M_n^k-M_{n-1}^k)}|{\mathcal{M}}_{-\mathsf{\infty }}^{n-1}\right]\le \mathcal{D}$.

Under this drift condition, the next theorem shows that Corollary 1 is satisfied.

Theorem 4.

Let $\{X_0,\dots ,X_n\}$ be generated by an aperiodic α-mixing Markov chain parametrized by ${\theta }_0\in \mathsf{\Theta }$ and initial distribution $q^{\left(0\right)}$. Suppose that Assumption 1 and Assumption 2 hold, and that the α-mixing coefficients satisfy ${\sum }_{k\ge 1}{\alpha }_k^{\delta /(2+\delta )}<+\mathsf{\infty }$. Furthermore, assume $\mathcal{K}({\rho }_n,\pi )\le \sqrt{n}C$ for some constant $C>0$. If $\int e^{\lambda M_k^{\left(1\right)}(y,x)}{p}_{{\theta }_0}\left(y\right|x)q_1^{\left(0\right)}\left(x\right)dx<+\mathsf{\infty }$ for all $k=1,\dots ,m_1$, then the conditions of Corollary 1 are satisfied with ${ϵ}_n=\mathrm{O}\left(max(\frac{1}{\sqrt{n}},\frac{n^{\delta /2}}{n})\right)$.

Verifying the conditions in Theorem 4 can be quite challenging. Instead, we suggest a different approach that requires f-geometric ergodicity. Unlike the drift condition in Assumption 2, f-geometric ergodicity additionally requires the existence of a petite set. As noted before, geometric ergodicity implies $\alpha$-mixing with geometrically decaying mixing coefficients. As with Theorem 4, we assume for simplicity that the Markov chain is aperiodic.

Theorem 5.

Let $\{X_0,\cdots ,X_n\}$ be generated by an aperiodic Markov chain parametrized by ${\theta }_0\in \mathsf{\Theta }$ with known initial distribution $q^{\left(0\right)}$, and assumed to be V-geometrically ergodic for some $V:{\mathbb{R}}^m\to [1,\mathsf{\infty })$. Suppose that Assumption 1 holds and $\int M_k^{\left(1\right)}{(y,x)}^{2+\delta }{p}_{{\theta }_0}\left(y\right|x)dy<V\left(x\right)\forall k,x\mathrm{and}\mathrm{some}\delta >0$. Furthermore, assume that $\mathcal{K}({\rho }_n,\pi )\le \sqrt{n}C$ for some constant $C>0$. If the initial distribution $q^{\left(0\right)}$ satisfies $E_{q^{\left(0\right)}}\left[V\left(X_0\right)\right]<+\mathsf{\infty }$, then the conditions of Corollary 1 are satisfied with ${ϵ}_n=\mathrm{O}\left(max(\frac{1}{\sqrt{n}},\frac{n^{\delta /2}}{n})\right)$.

The following Proposition 10 shows, the simple linear model satisfies Theorem 5 when the parameter ${\theta }_0$ is suitably restricted.

Proposition 10.

Consider the simple linear model satisfying the equation

$$\begin{array}{cc}\hfill X_n& ={\theta }_0{X}_{n-1}+W_n,\hfill \end{array}$$ (22)

where $\left\{W_n\right\}$ are i.i.d. standard Gaussian random variables and $|{\theta }_0|<2^{\frac{1}{4+2\delta }-1}$ for $\delta >0$. Let $\mathcal{F}$ be the space of all scaled Beta distributions on $(-1,1)$ and suppose the prior π is a uniform distribution on $(-1,1)$. Then, the conditions of Theorem 5 are satisfied with ${ϵ}_n=\mathrm{O}\left(max(\frac{1}{\sqrt{n}},\frac{n^{\delta /2}}{n})\right)$, if the initial distribution $q^{\left(0\right)}$ satisfies $E_{q^{\left(0\right)}}\left[X_0^{4+2\delta }\right]<+\mathsf{\infty }$.

5. Misspecified Models

We show next how our results can be extended to the misspecified model setting. Assume that the true data generating distribution is parametrized by ${\theta }_0\notin \mathsf{\Theta }$. Let ${\theta }_n^{*}:=arg{min}_{\theta \in \mathsf{\Theta }}\mathcal{K}(P_{{\theta }_0}^{\left(n\right)},P_{\theta }^{\left(n\right)})$ represent the closest parametrized distribution in the variational family to the data-generating distribution. Further, assume our usual conditions:

• i. $\int \mathrm{E}\left[r_n(\theta ,{\theta }_n^{*})\right]{\rho }_n\left(d\theta \right)\le n{ϵ}_n$,
• ii. $\int \mathrm{Var}\left(r_n(\theta ,{\theta }_n^{*})\right){\rho }_n\left(d\theta \right)\le n{ϵ}_n$.

Now, since $r_n(\theta ,{\theta }_0)=r_n(\theta ,{\theta }_n^{*})+r_n({\theta }_n^{*},{\theta }_0)$, we have

$$\int \mathcal{K}(P_{{\theta }_0}^{\left(n\right)},P_{\theta }^{\left(n\right)}){\rho }_n\left(d\theta \right)\le \mathrm{E}\left[r_n({\theta }_0,{\theta }_n^{*})\right]+n{ϵ}_n.$$ (23)

Similarly, decomposing the variance it follows that

$$\begin{array}{cc}\hfill \mathrm{Var}\left[r_n(\theta ,{\theta }_0)\right]& =\mathrm{Var}\left[r_n(\theta ,{\theta }_n^{*})\right]+\mathrm{Var}\left[r_n({\theta }_n^{*},{\theta }_0)\right]+2\mathrm{Cov}[r_n(\theta ,{\theta }_n^{*}),r_n({\theta }_n^{*},{\theta }_0)].\hfill \end{array}$$ (24)

Using the fact that $2ab\le a^2+b^2$ on the covariance term $2\mathrm{Cov}[r_n(\theta ,{\theta }_n^{*}),r_n({\theta }_n^{*},{\theta }_0)]=2\mathrm{E}\left[\left(r_n(\theta ,{\theta }_n^{*})-\mathrm{E}\left[r_n(\theta ,{\theta }_n^{*})\right]\right)\left(r_n({\theta }_n^{*},{\theta }_0)-\mathrm{E}\left[r_n({\theta }_n^{*},{\theta }_0)\right]\right)\right]$, we have

$$\begin{array}{cc}\hfill \mathrm{Var}\left[r_n(\theta ,{\theta }_0)\right]& \le 2\mathrm{Var}\left[r_n(\theta ,{\theta }_n^{*})\right]+2\mathrm{Var}\left[r_n({\theta }_n^{*},{\theta }_0)\right].\hfill \end{array}$$ (25)

Integrating both sides with respect to ${\rho }_n\left(d\theta \right)$ we get

$$\begin{array}{cc}\hfill \int \mathrm{Var}\left[r_n(\theta ,{\theta }_0)\right]{\rho }_n\left(d\theta \right)& \le 2\int \mathrm{Var}\left[r_n(\theta ,{\theta }_n^{*})\right]{\rho }_n\left(d\theta \right)+2\int \mathrm{Var}\left[r_n({\theta }_n^{*},{\theta }_0)\right]{\rho }_n\left(d\theta \right)\hfill \\ \hfill & \le 2n{ϵ}_n+2\mathrm{Var}\left[r_n({\theta }_n^{*},{\theta }_0)\right].\hfill \end{array}$$ (26)

Consequently, we arrive at the following result:

Theorem 6.

Let $\mathcal{F}$ be a subset of all probability distributions parameterized by Θ. Let ${\theta }_n^{*}=arg{min}_{\theta \in \mathsf{\Theta }}\mathcal{K}(P_{{\theta }_0}^{\left(n\right)},P_{\theta }^{\left(n\right)})$ and assume there exist ${ϵ}_n>0$ and ${\rho }_n\in \mathcal{F}$ such that

• i.  $\int \mathrm{E}\left[r_n(\theta ,{\theta }_n^{*})\right]{\rho }_n\left(d\theta \right)\le n{ϵ}_n$,
• ii.  $\int \mathrm{Var}\left(r_n(\theta ,{\theta }_n^{*})\right){\rho }_n\left(d\theta \right)\le n{ϵ}_n$, and
• iii.  $\mathcal{K}({\rho }_n,\pi )\le n{ϵ}_n$.

Then, for any ${\alpha }^{re}\in (0,1)$ and $(ϵ,\eta )\in (0,1)\times (0,1)$,

$$\begin{array}{cc}\hfill P[\int D_{{\alpha }^{re}}& (P_{\theta }^{\left(n\right)},P_{{\theta }_0}^{\left(n\right)}){\tilde{\pi }}_{n,{\alpha }^{re}}\left(d\theta \right|X^{\left(n\right)})\le \hfill \\ \hfill & \frac{({\alpha }^{re}+1)n{ϵ}_n+\mathrm{E}\left[r_n({\theta }_0,{\theta }_n^{*})\right]+{\alpha }^{re}\sqrt{\frac{2n{ϵ}_n+2\mathrm{Var}\left[r_n({\theta }_n^{*},{\theta }_0)\right]}{\eta }}-log\left(ϵ\right)}{1-{\alpha }^{re}}]\ge 1-ϵ-\eta .\hfill \end{array}$$ (27)

The proof of this theorem is straightforward and follows from the proof of Theorem 1 by plugging in the upper bounds for KL-divergence from Equation (23), and variance from Equation (26) to (A13). A sketch of the proof is presented in the appendix.

6. Conclusions

Concentration of the KL-VB model risk, in terms of the expected ${\alpha }^{re}$-Rényi divergence, is well established under the i.i.d. data generating model assumption. Here, we extended this to the setting of Markov data generating models, linking the concentration rate to the mixing and ergodic properties of the Markov model. Our results apply to both stationary and non-stationary Markov chains, as well as to the situation with misspecified models. There remain a number of open questions. An immediate one is to extend the current analysis to continuous-time Markov chains and Markov jump processes, possibly using uniformization of the continuous time model. Another direction is to extend this to the setting of non-homogeneous Markov chains, where analogues of notions such as stationarity are less straightforward. Further, as noted in the introduction, [14] establish PAC-Bayes bounds under slightly weaker ‘existence of test functions’ conditions, while our results are established under the stronger conditions used by [15] for the i.i.d. setting. Weakening the conditions in our analysis is important, but complicated. A possible path is to build on results from [22], who provides conditions form the existence of exponentially powerful test functions exist for distinguishing between two Markov chains. It is also known that there exists a likelihood ratio test separating any two ergodic measures [23]. However, leveraging these to establish the PAC-Bayes bounds for the KL-VB posterior is a challenging effort that we leave to future papers. Finally it is of interest to generalize our PAC-bounds to posterior approximations beyond KL-variational inference, such as ${\alpha }^{re}$-Rényi posterior approximations [6], and loss-calibrated posterior approximations [24,25].

Appendix A. Technical Desiderata

Appendix A.1. Definitions Related to Markov Chains

As noted before, ergodicity plays an acute role in establishing our results. We consolidate various definitions used throughout the paper in this appendix. Recall that we assume the parameterized Markov chain possesses an invariant probability density or mass function $q_{\theta }$ under parameter $\theta \in \mathsf{\Theta }$. Our results in Section 4 also rely on the ergodic properties of the Markov chain, and we assume that the Markov chain is f-geometrically ergodic [20] (Chapter 15). First, refer to the definition of the functional norm ${\parallel ·\parallel }_f$, from Definition A.1.1,

Definition A.1.1

(f-norm). The functional norm in f-metric of a measure v, or the f-norm of v is

$${\parallel v\parallel }_f=\underset{g:\left|g\right|<f}{sup}\left|\int gdv\right|,$$ (A1)

where f and g are any two functions.

An immediate consequence of this definition is that if $f_1,f_2$ are two functions such that $f_1<f_2$ (i.e., for all points in the support of the functions), then

$${\parallel v\parallel }_{f_1}\le {\parallel v\parallel }_{f_2}.$$ (A2)

Now that we have defined the ${\parallel ·\parallel }_f$ norm, we can now define f-geometric ergodicity. In the following, we assume the Markov chain is positive Harris; see [20] for a definition. This is a mild and fairly standard assumption in Markov chain theory.

Definition A.1.2

(f-geometric ergodicity). For any function f, Markov chain $\left\{X_n\right\}$ parameterized by $\theta \in \mathsf{\Theta }$ is said to be f-geometrically ergodic if it is positive Harris and there exists a constant $r_f>1$, that depends on f, such that for any $A\in \mathcal{B}\left(X\right)$,

$$\sum _{n=1}^n{r}_f^n{∥P_{\theta }(X_n\in A|X_0=x)-{\int }_A{q}_{\theta }\left(y\right)dy∥}_f<\mathsf{\infty }.$$ (A3)

It is straightforward to see that this is equivalent to

$${∥P_{\theta }(X_n\in A|X_0=x)-\int q_{\theta }\left(y\right)dy∥}_f\le C{r}_f^{-n}$$

for an appropriate constant C (which may depend on the state x), that is, the Markov chain approaches steady state at a geometrically fast rate. If a Markov chain is f-geometrically ergodic for $f\equiv 1$, then, it is simply termed as geometrically ergodic. It is straightforward to see (via Theorem A.1.2 in the Appendix) that a geometrically ergodic Markov chain is also $\alpha$-mixing, with mixing coefficients satisfying

$$\sum _{k\ge 0}{\alpha }_k^{\upsilon }<\mathsf{\infty }\forall \upsilon >0,$$ (A4)

showing that, under geometric ergodicity, the $\alpha$-mixing coefficients raised to any positive power $\upsilon$ are finitely summable. We note here that the most standard procedure to establish f-geometric ergodicity for any Markov chain is through the verification of the drift condition. The drift condition is a sufficient condition for a Markov chain to be f-geometrically ergodic, as long as there exists a set (called petite set) towards which the Markov chain drifts to (see Assumption A.1.1 in the appendix). If a Markov chain is f-geometrically ergodic with $f\equiv V$, for some particular function V, then we call it V-geometrically ergodic.

We defined V-geometric ergodicity in the previous sections. In this section, we provide a sufficient condition for a Markov chain to be V-geometrically ergodic. First, we recall the definition of resolvent from [20] (Chapter 5).

Definition A.1.3

(Resolvent). Let $n\in \{0,1,2,\cdots \}$ and $q_n$ be such that $q_n\ge 0\forall n$ and ${\sum }_{n=1}^{\mathsf{\infty }}{q}_n=1$. Note that $q_n$ can be thought of being a probability mass function for a random variable "q" taking values on non-negative integers. Then, the resolvent of a Markov chain with respect to q is given by $K_q(x,A)$ where,

$$\begin{array}{cc}\hfill K_q(x,A)& =\sum _{n=0}^{\mathsf{\infty }}{q}_{n}P(X_n\in A|X_0=x).\hfill \end{array}$$ (A5)

Then, the definition of petite sets follows (see, for Reference, [20] (Chapter 5)).

Definition A.1.4

(Petite Sets). Let $X_0,\cdots ,X_n$ be n samples from a Markov chain taking values on the state space $\mathcal{X}$. Let C be a set. We shall call C to be $v_q$ petite if

$$\begin{array}{c}\hfill K_q(x,B)\ge {\upsilon }_q\left(B\right)\end{array}$$

for all $x\in C$ and $B\in \mathcal{B}\left(\mathcal{X}\right)$, and a non-trivial measure ${\upsilon }_q$ on $\mathcal{B}\left(\mathcal{X}\right)$, and a probability mass function q on $\{1,2,3,\cdots \}$

Now, let $\Delta V\left(x\right):=E\left[V\left(X_n\right)\right|X_{n-1}=x]-V\left(x\right)$ for $V:S\to [1,\mathsf{\infty })$.

Assumption A.1.1

(Drift condition). [20] (Chapter 5) Suppose the chain $\left\{X_n\right\}$ is, aperiodic and ψ-irreducible. Let there exists a petite set C, constants $b<\mathsf{\infty },\beta >0$, and a non-trivial function $V:S\to [1,\mathsf{\infty })$ satisfying

$$\begin{array}{c}\hfill \Delta V\left(x\right)\le -\beta V\left(x\right)+b{I}_{x\in C}\forall x\in S.\end{array}$$ (A6)

If a Markov chain drifts towards a petite set then it is V-geometrically ergodic. Suppose, for simplicity, that $V\left(x\right)=\left|X\right|$. Then, the drift condition becomes $E\left[\right|X_n\parallel X_{x-1}]-|X_{n-1}|=-\beta |X_n|+b{I}_{X_n\in C}$. The left hand side of this equation represents the change in the state of the Markov chain in one time epoch. Thus, the condition in Assumption A.1.1 essentially states that the Markov chain drifts towards a petite set C and then, once it reaches that set, moves to any point in the state space with at least some probability independent of C.

Theorem A.1.1

(Geometrically ergodic theorem). Suppose that $\left\{X_n\right\}$ is satisfies Assumption A.1.1. Then, the set $S_V=\{x:V\left(x\right)<\mathsf{\infty }\}$ is absorbing, i.e., $P_{\theta }(X_1\in S_V|X_0=x)=1$$\forall x\in S_V$, and full, i.e., $\psi \left(S_V^c\right)=0$. Furthermore, $\exists \mathrm{constants}r>1,R<\mathsf{\infty }$ such that, for any $A\in \mathcal{B}\left(S\right)$,

$${∥P_{\theta }(X_n\in A|X_0=x)-{\int }_A{q}_{\theta }\left(y\right)dy∥}_V\le R{r}^{-n}V\left(x\right).$$ (A7)

Any aperiodic and $\psi$-irreducible Markov chain satisfying the drift condition is geometrically ergodic. A consequence of Equation (A2) is that if, $\left\{X_n\right\}$ is V-geometrically ergodic, then for any other function $U,\mathrm{such}\mathrm{that}\left|U\right|<V$, it is also U-geometrically ergodic. In essence, a geometrically ergodic Markov chain is asymptotically uncorrelated in a precise sense. Recall $\rho$-mixing coefficients defined as follows. Let $\mathcal{A}$ be a sigma field and ${\mathcal{L}}^2\left(A\right)$ be the set of square integrable, real valued, $\mathcal{A}$ measurable functions.

Definition A.1.5

($\rho$-mixing coefficient). Let ${\mathcal{M}}_i^j$ denote the sigma field generated by the measures $X_k,\mathrm{where}i\le k\le j$. Then,

$${\rho }_k=\underset{t>0}{sup}\underset{(f,g)\in {\mathcal{L}}^2\left({\mathcal{M}}_{-\mathsf{\infty }}^t\right)\times {\mathcal{L}}^2\left({\mathcal{M}}_{t+k}^{\mathsf{\infty }}\right)}{sup}\left|\mathrm{Corr}(f,g)\right|,$$ (A8)

where $\mathrm{Corr}$ is the correlation function.

Theorem A.1.2.

If $X_n$ is geometrically ergodic, then it is α-mixing. That is, there exists a constant $c>0$ such that ${\alpha }_k=\mathrm{O}\left(e^{-ck}\right)$.

Proof. 

By [26] (Theorem 2) it follows that a geometrically ergodic Markov chain is asymptotically uncorrelated with $\rho$-mixing coefficients (see Definition A.1.5) that satisfy ${\rho }_k=\mathrm{O}\left(e^{-ck}\right)$. Furthermore, it is well known that [18,26] ${\alpha }_k\le \frac{1}{4}{\rho }_k$, implying ${\alpha }_k=\mathrm{O}\left(e^{-ck}\right)$. □

Appendix A.2. Bounding the KL-Divergence between Beta Distributions

The following results will be utilized in the proofs of Propositions 8–10.

Lemma A.2.1.

Let ${\theta }_0\in (0,1)$. Let, ${\rho }_n$ be a sequence of Beta distributions with parameters $a_n=n{\theta }_0$ and $b_n=n(1-{\theta }_0)$. Let π denote an uniform distribution, $U(0,1)$. Then, $\mathcal{K}({\rho }_n,\pi )<C+\frac{1}{2}log\left(n\right)$, for some constant $C>0$.

Proof. 

Without loss of generality, we can assume $a_n>1$ and $b_n>1$. The same form of the result can be obtained in all the other cases, by appropriate use of the bounds presented in the proof. We write the KL divergence $\mathcal{K}({\rho }_n,\pi )$ as $\int log\left(\frac{{\rho }_n}{\pi }\right){\rho }_n\left(d\theta \right)$. Since $\pi$ is uniform, $\pi \left(\theta \right)=1$ whenever $\theta \in (0,1)$. Hence, the KL-divergence can be written as the negative of the entropy of ${\rho }_n$ ${\int }_0^{1}log\left({\rho }_n\left(\theta \right)\right){\rho }_n\left(d\theta \right),$ which can be written as

$$\begin{array}{c}\hfill \mathcal{K}({\rho }_n,\pi )=(a_n-1)\psi \left(a_n\right)+(b_n-1)\psi \left(b_n\right)-(a_n+b_n-2)\psi (a_n+b_n)\\ \hfill -log\mathrm{Beta}(a_n,b_n),\end{array}$$ (A9)

where $\psi$ is the digamma function. Using Stirling’s approximation on $\mathrm{Beta}(a_n,b_n)$ yields,

$$\begin{array}{c}\hfill \mathrm{Beta}(a_n,b_n)=\sqrt{2\pi }\frac{a_n^{a_n-1/2}{b}_n^{b_n-1/2}}{{(a_n+b_n)}^{a_n+b_n-1/2}}(1+o\left(1\right)).\end{array}$$

Hence, setting $C_1=log\left(2\sqrt{\pi }\right)$, we can write $-log\mathrm{Beta}(a_n,b_n)$ as,

$$\begin{array}{cc}\hfill -log\mathrm{Beta}(a_n,b_n)& =C_1-(a_n-\frac{1}{2})log\left(a_n\right)-(b_n-\frac{1}{2})log\left(b_n\right)\hfill \\ \hfill & +(a_n+b_n-\frac{1}{2})log(a_n+b_n)+log(1+o\left(1\right)).\hfill \end{array}$$

From [27] we have that $log\left(x\right)-\frac{1}{x}<\psi \left(x\right)<log\left(x\right)-\frac{1}{2x}\forall x>0$. Since we assumed $a_n>1$ and $b_n>1$, the fact that $\psi \left(x\right)<log\left(x\right)-\frac{1}{2x}$ implies

$$\begin{array}{cc}\hfill (a_n-1)\psi \left(a_n\right)& <(a_n-1)log\left(a_n\right)-\frac{a_n-1}{2a_n}\mathrm{and},\hfill \\ \hfill (b_n-1)\psi \left(b_n\right)& <(b_n-1)log\left(b_n\right)-\frac{b_n-1}{2b_n}.\hfill \end{array}$$

Finally, using the fact that $log\left(x\right)-\frac{1}{x}<\psi \left(x\right)$, we get,

$$\begin{array}{c}\hfill -(a_n+b_n-2)\psi (a_n+b_n)<-(a_n+b_n-2)log(a_n+b_n)+\frac{a_n+b_n-2}{a_n+b_n}.\end{array}$$

Therefore, after much cancellation, the KL-divergence

$$\begin{array}{c}\hfill (a_n-1)\psi \left(a_n\right)+(b_n-1)\psi \left(b_n\right)-(a_n+b_n-2)\psi (a_n+b_n)-log\mathrm{Beta}(a_n,b_n)\end{array}$$

can be upper bounded by

$$\begin{array}{c}\hfill -\frac{1}{2}log\left(a_n\right)-\frac{1}{2}log\left(b_n\right)+\frac{3}{2}log(a_n+b_n)+\frac{a_n+b_n-2}{a_n+b_n}-\frac{a_n-1}{2a_n}-\frac{b_n-1}{2b_n}.\end{array}$$

Now, plugging in the values of $a_n$ and $b_n$, we get Plugging in the values of $a_n$ and $b_n$, we get as upper bound for the KL-divergence as,

$$\begin{array}{cc}\hfill \mathcal{K}({\rho }_n,\pi )& <-\frac{1}{2}log\left(n{\theta }_0\right)-\frac{1}{2}log\left(n(1-{\theta }_0)\right)+\frac{3}{2}log\left(n\right)+\frac{n-2}{n}-\frac{n{\theta }_0-1}{2n{\theta }_0}-\frac{n(1-{\theta }_0)-1}{2n(1-{\theta }_0)}\hfill \\ \hfill & =\frac{1}{2}log\left(n\right)-\frac{1}{2}\left(log\left({\theta }_0\right)+log(1-{\theta }_0)\right)+3-\frac{2}{n}-\frac{1}{2n{\theta }_0}-\frac{1}{2n(1-{\theta }_0)}\hfill \\ \hfill & <C+\frac{1}{2}log\left(n\right),\hfill \end{array}$$

for some large enough positive constant C. This completes our proof. □

Proposition A.2.1.

Let ${\theta }_0\in (0,1)$. Let, ${\rho }_n$ be a sequence of Beta distributions with parameters $a_n=n{\theta }_0$ and $b_n=n(1-{\theta }_0)$. Let π denote an Beta distribution, with parameters $(a,b)$. Then, $\mathcal{K}({\rho }_n,\pi )<C+\frac{1}{2}log\left(n\right)$, for some constant $C>0$.

Proof. 

Without loss of generality, we assume $a>1$ and $b>1$. As mentioned in the proof of Lemma A.2.1, the other cases follows similarly. We write the KL-divergence between ${\rho }_n$ and $\pi$ as,

$$\begin{array}{cc}\hfill \mathcal{K}({\rho }_n,\pi )=& \int log\left(\frac{{\rho }_n}{\pi }\right){\rho }_n\left(d\theta \right)=\int log\left(\frac{{\rho }_n}{U}\right){\rho }_n\left(d\theta \right)+\int log\left(\frac{U}{\pi }\right){\rho }_n\left(d\theta \right),\hfill \end{array}$$

where, U is an uniform distribution on $(0,1)$. We analyze the second term in the above expression. The second term can be written as,

$$\begin{array}{cc}\hfill \int log\left(\frac{U}{\pi }\right){\rho }_n\left(d\theta \right)& =\int log\left(\frac{1}{\frac{1}{\mathrm{Beta}(a,b)}{\theta }^{a-1}{(1-\theta )}^{b-1}}\right){\rho }_n\left(d\theta \right)\hfill \\ \hfill & =C_1-(a-1)\int log\left(\theta \right){\rho }_n\left(d\theta \right)-(b-1)\int log(1-\theta ){\rho }_n\left(d\theta \right),\hfill \end{array}$$

where $C_1$ is $log\left(\mathrm{Beta}\right(a,b\left)\right)$. Since, ${\rho }_n$ follows a Beta distribution with parameters $a_n=n{\theta }_0$ and $b_n=n(1-{\theta }_0)$, we get that,

$$\begin{array}{cc}\hfill \int log\left(\frac{U}{\pi }\right){\rho }_n\left(d\theta \right)& =C_1-(a-1)\left[\psi \left(a_n\right)-\psi (a_n+b_n)\right]-(b-1)\left[\psi \left(b_n\right)-\psi (a_n+b_n)\right]\hfill \end{array}$$

Since, $log\left(x\right)-\frac{1}{x}<\psi \left(x\right)<log\left(x\right)-\frac{1}{2x}$, looking at the term $\left[\psi \left(a_n\right)-\psi (a_n+b_n)\right]$, we get that,

$$\begin{array}{cc}\hfill -\left[\psi \left(a_n\right)-\psi (a_n+b_n)\right]& =-\left[\psi \left(n{\theta }_0\right)-\psi (n{\theta }_0+n(1-{\theta }_0))\right]\hfill \\ \hfill & =-\left[\psi \left(n{\theta }_0\right)-\psi \left(n\right)\right].\hfill \end{array}$$

Using the lower bound on $\psi \left(n{\theta }_0\right)$ and the upper bound on $\psi \left(n\right)$, we get

$$\begin{array}{cc}\hfill -\left[\psi \left(a_n\right)-\psi (a_n+b_n)\right]& <-log\left(n{\theta }_0\right)+\frac{1}{n{\theta }_0}+log\left(n\right)-\frac{1}{2n}\hfill \\ & =-log\left({\theta }_0\right)+\frac{2-{\theta }_0}{2n{\theta }_0}.\hfill \end{array}$$

Furthermore, similarly, we get that,

$$\begin{array}{cc}\hfill -\left[\psi \left(b_n\right)-\psi (a_n+b_n)\right]& <-log(1-{\theta }_0)+\frac{2-(1-{\theta }_0)}{2n(1-{\theta }_0)}.\hfill \end{array}$$

Therefore it follows that

$$\begin{array}{cc}\hfill max& \left\{-(a-1)\left[\psi \left(a_n\right)-\psi (a_n+b_n)\right],-(b-1)\left[\psi \left(b_n\right)-\psi (a_n+b_n)\right]\right\}\hfill \\ \hfill & <max\left\{(a-1)\left[-log\left({\theta }_0\right)+\frac{2-{\theta }_0}{2n{\theta }_0}\right],(b-1)\left[-log(1-{\theta }_0)+\frac{2-(1-{\theta }_0)}{2n(1-{\theta }_0)}\right]\right\}\hfill \\ \hfill & <C,\hfill \end{array}$$

for a large positive constant C. Using the above bounds, we finally show that,

$$\begin{array}{c}\hfill C_1-(a-1)\left[\psi \left(a_n\right)-\psi (a_n+b_n)\right]-(b-1)\left[\psi \left(b_n\right)-\psi (a_n+b_n)\right]\\ \hfill <C_1+2C,\end{array}$$

which can be upper bounded by $C^{\prime }$ for some large constant $C^{\prime }$. Finally, we upper bound $\int log\left(\frac{{\rho }_n}{U}\right){\rho }_n\left(d\theta \right)$ by Lemma A.2.1 thereby completing the proof. □

Appendix B. Proofs of Main Results

Appendix B.1. Proofs for A Concentration Bound for the α^re-Rényi Divergence

Appendix B.1.1. Proof of Proposition 1

We start by recalling the variational formula of Donsker and Varadhan [28].

Lemma B.1.1

(Donsker-Varadhan)b. For any probability distribution function π on Θ, and for any measurable function $h:\mathsf{\Theta }\to \mathbb{R}$, if $\int e^{h}d\pi <\mathsf{\infty }$, then

$$\begin{array}{cc}\hfill log\int e^{h}d\pi & =\underset{\rho \in {\mathcal{M}}^{+}\left(\mathsf{\Theta }\right)}{sup}\left\{\int hd\rho -\mathcal{K}(\rho ,\pi )\right\}\hfill \end{array}$$ (A10)

Now, fix ${\alpha }^{re}\in (0,1)$, and $\theta \in \mathsf{\Theta }$. First, observe that by the definition of the ${\alpha }^{re}$-Rényi divergence we have

$$\begin{array}{cc}\hfill & {\mathrm{E}}_{{\theta }_0}^{\left(n\right)}[exp(-{\alpha }^{re}{r}_n(\theta ,{\theta }_0))]=exp[-(1-{\alpha }^{re})D_{{\alpha }^{re}}(P_{\theta }^{\left(n\right)},P_{{\theta }_0}^{\left(n\right)})]\hfill \end{array}$$

Multiplying both sides of the equation by $exp[(1-{\alpha }^{re})D_{{\alpha }^{re}}(P_{\theta }^{\left(n\right)},P_{{\theta }_0}^{\left(n\right)})$ and integrating with respect to (w.r.t.) $\pi \left(\theta \right)$ it follows that

$$\begin{array}{c}\int {\mathrm{E}}_{{\theta }_0}^{\left(n\right)}\left[exp\left(-{\alpha }^{re}{r}_n(\theta ,{\theta }_0)+(1-{\alpha }^{re})D_{{\alpha }^{re}}(P_{\theta }^{\left(n\right)},P_{{\theta }_0}^{\left(n\right)})\right)\right]\pi \left(d\theta \right)=1,\mathrm{or}\hfill \end{array}$$

$$\begin{array}{cc}\hfill & {\mathrm{E}}_{{\theta }_0}^{\left(n\right)}\left[\int exp\left(-{\alpha }^{re}{r}_n(\theta ,{\theta }_0)+(1-{\alpha }^{re})D_{{\alpha }^{re}}(P_{\theta }^{\left(n\right)},P_{{\theta }_0}^{\left(n\right)})\right)\pi \left(d\theta \right)\right]=1.\hfill \end{array}$$

Define $h\left(\theta \right):=-{\alpha }^{re}{r}_n(\theta ,{\theta }_0)+(1-{\alpha }^{re})D_{{\alpha }^{re}}(P_{\theta }^{\left(n\right)},P_{{\theta }_0}^{\left(n\right)})$. Then, applying Lemma B.1.1 to the integrand on the left hand side (l.h.s.) above, it follows that

$$\begin{array}{c}\hfill {\mathrm{E}}_{{\theta }_0}^{\left(n\right)}\left[exp\left(\underset{\rho \in {\mathcal{M}}^{+}\left(\mathsf{\Theta }\right)}{sup}\left[\int h\left(\theta \right)\rho \left(d\theta \right)-\mathcal{K}(\rho ,\pi )\right]\right)\right]=1.\end{array}$$

Multiply both sides of this equation by $ϵ>0$ to obtain

$$\begin{array}{c}\hfill {\mathrm{E}}_{{\theta }_0}^{\left(n\right)}\left[exp\left(\underset{\rho \in {\mathcal{M}}^{+}\left(\mathsf{\Theta }\right)}{sup}\left[\int h\left(\theta \right)\rho \left(d\theta \right)-\mathcal{K}(\rho ,\pi )+log\left(ϵ\right)\right]\right)\right]=ϵ.\end{array}$$

Now, by Markov’s inequality, we have

$$P_{{\theta }_0}^{\left(n\right)}\left[\underset{\rho \in {\mathcal{M}}^{+}\left(\mathsf{\Theta }\right)}{sup}\int (-{\alpha }^{re}{r}_n(\theta ,{\theta }_0)+(1-{\alpha }^{re})D_{{\alpha }^{re}}(P_{\theta }^{\left(n\right)},P_{{\theta }_0}^{\left(n\right)}))\rho \left(d\theta \right)-\mathcal{K}(\rho ,\pi )+log\left(ϵ\right)\ge 0\right]\le ϵ.$$ (A11)

Thus, it follows via complementation that

$$\begin{array}{cc}\hfill P_{{\theta }_0}^{\left(n\right)}[\forall \rho \in \mathcal{F}\left(\mathsf{\Theta }\right)\int D_{{\alpha }^{re}}(P_{\theta }^{\left(n\right)},P_{{\theta }_0}^{\left(n\right)})\rho \left(d\theta \right)\le \frac{{\alpha }^{re}}{(1-{\alpha }^{re})}\int r_n(\theta ,{\theta }_0)\rho \left(d\theta \right)+& \frac{\mathcal{K}(\rho ,\pi )-log\left(ϵ\right)}{1-{\alpha }^{re}}]\hfill \\ \hfill & \ge 1-ϵ,\hfill \end{array}$$

thereby completing the proof.

Appendix B.1.2. Proof of Theorem 1

Recall the definition of the fractional posterior and the VB approximation,

$$\begin{array}{c}\hfill {\pi }_{n,{\alpha }^{re}|X^n}=\frac{{exp}^{-{\alpha }^{re}{r}_n(\theta ,{\theta }_0)\left(X^n\right)}\pi \left(d\theta \right)}{\int {exp}^{-{\alpha }^{re}{r}_n(\gamma ,{\theta }_0)\left(X^n\right)}\pi \left(d\gamma \right)},{\tilde{\pi }}_{n,{\alpha }^{re}|X^n}=\underset{\rho \in \mathcal{F}}{argmin}\mathcal{K}(\rho ,{\pi }_{n,{\alpha }^{re}|X^{\left(n\right)}}).\end{array}$$

It follows by definition of the KL divergence that

$$\begin{array}{cc}\hfill {\tilde{\pi }}_{n,{\alpha }^{re}|X^n}& =\underset{\rho \in \mathcal{F}}{argmin}\left\{-{\alpha }^{re}\int r_n(\theta ,{\theta }_0)\rho \left(d\theta \right)+\mathcal{K}(\rho ,\pi )\right\},\hfill \end{array}$$ (A12)

where $\pi$ is the prior distribution. Following Proposition 1 it follows that for any $ϵ>0$

$$\int D_{{\alpha }^{re}}(P_{\theta }^{\left(n\right)},P_{{\theta }_0}^{\left(n\right)})\tilde{\pi }\left(d\theta \right|X^n)\le \frac{{\alpha }^{re}}{(1-{\alpha }^{re})}\int r_n(\theta ,{\theta }_0)\rho \left(d\theta \right)+\frac{\mathcal{K}(\rho ,\pi )-log\left(ϵ\right)}{1-{\alpha }^{re}},$$

with probability $1-ϵ$. We fix an $\eta \in (0,1)$. Using Chebychev’s inequality, we have

$$\begin{array}{cc}\hfill & P_{{\theta }_0}^{\left(n\right)}[\frac{{\alpha }^{re}}{1-{\alpha }^{re}}\int r_n(\theta ,{\theta }_0){\rho }_n\left(d\theta \right)\ge \frac{{\alpha }^{re}}{1-{\alpha }^{re}}\int \mathrm{E}\left[r_n(\theta ,{\theta }_0)\right]{\rho }_n\left(d\theta \right)\hfill \\ \hfill & +\frac{{\alpha }^{re}}{1-{\alpha }^{re}}\sqrt{\frac{\mathrm{Var}[\int r_n(\theta ,{\theta }_0){\rho }_n\left(d\theta \right)]}{\eta }}+\frac{\mathcal{K}({\rho }_n,\pi )}{1-{\alpha }^{re}}]\hfill \\ \hfill & =P_{{\theta }_0}^{\left(n\right)}[\frac{{\alpha }^{re}}{1-{\alpha }^{re}}\int r_n(\theta ,{\theta }_0){\rho }_n\left(d\theta \right)-\frac{{\alpha }^{re}}{1-{\alpha }^{re}}\int \mathrm{E}\left[r_n(\theta ,{\theta }_0)\right]{\rho }_n\left(d\theta \right)-\frac{\mathcal{K}({\rho }_n,\pi )}{1-{\alpha }^{re}}\hfill \\ \hfill & \ge \frac{{\alpha }^{re}}{1-{\alpha }^{re}}\sqrt{\frac{\mathrm{Var}[\int r_n(\theta ,{\theta }_0){\rho }_n\left(d\theta \right)]}{\eta }}]\hfill \\ \hfill & \le \frac{\mathrm{Var}\left[\frac{{\alpha }^{re}}{1-{\alpha }^{re}}\int r_n(\theta ,{\theta }_0){\rho }_n\left(d\theta \right)-\frac{{\alpha }^{re}}{1-{\alpha }^{re}}\int \mathrm{E}\left[r_n(\theta ,{\theta }_0)\right]{\rho }_n\left(d\theta \right)-\frac{\mathcal{K}({\rho }_n,\pi )}{1-{\alpha }^{re}}\right]}{\frac{{\left({\alpha }^{re}\right)}^2}{{\left(1-{\alpha }^{re}\right)}^2}\frac{\mathrm{Var}[\int r_n(\theta ,{\theta }_0){\rho }_n\left(d\theta \right)]}{\eta }}.\hfill \end{array}$$

Note that $\frac{{\alpha }^{re}}{1-{\alpha }^{re}}\int E\left(r_n(\theta ,{\theta }_0)\right){\rho }_n\left(d\theta \right)$ and $\frac{\mathcal{K}({\rho }_n,\pi )}{1-{\alpha }^{re}}$ are constants with respect to the data, implying

$$\begin{array}{cc}\hfill \mathrm{Var}[\frac{{\alpha }^{re}}{1-{\alpha }^{re}}& \int r_n(\theta ,{\theta }_0){\rho }_n\left(d\theta \right)-\frac{{\alpha }^{re}}{1-{\alpha }^{re}}\int \mathrm{E}\left[r_n(\theta ,{\theta }_0)\right]{\rho }_n\left(d\theta \right)-\frac{\mathcal{K}({\rho }_n,\pi )}{1-{\alpha }^{re}}]\hfill \\ & =\frac{{\left({\alpha }^{re}\right)}^2}{{(1-{\alpha }^{re})}^2}\mathrm{Var}\left[\int r_n(\theta ,{\theta }_0){\rho }_n\left(d\theta \right)\right].\hfill \end{array}$$

Therefore, we have

$$\begin{array}{cc}\hfill P_{{\theta }_0}^{\left(n\right)}[\frac{{\alpha }^{re}}{1-{\alpha }^{re}}\int r_n(\theta ,{\theta }_0){\rho }_n\left(d\theta \right)\ge \frac{{\alpha }^{re}}{1-{\alpha }^{re}}& \int \mathrm{E}\left[r_n(\theta ,{\theta }_0)\right]{\rho }_n\left(d\theta \right)\hfill \\ & +\frac{{\alpha }^{re}}{1-{\alpha }^{re}}\sqrt{\frac{\mathrm{Var}[\int r_n(\theta ,{\theta }_0){\rho }_n\left(d\theta \right)]}{\eta }}+\frac{\mathcal{K}({\rho }_n,\pi )}{1-\alpha }]\le \eta .\hfill \end{array}$$

From Proposition 1, with probability $1-ϵ$ the following holds

$$\int D_{{\alpha }^{re}}(P_{\theta }^{\left(n\right)},P_{{\theta }_0}^{\left(n\right)}){\tilde{\pi }}_{n,{\alpha }^{re}|X^n}\left(d\theta \right)\le \frac{{\alpha }^{re}\int r_n(\theta ,{\theta }_0){\rho }_n\left(d\theta \right)+\mathcal{K}({\rho }_n,\pi )-log\left(ϵ\right)}{1-{\alpha }^{re}}.$$

Therefore, with probability $1-\eta -ϵ$ the following statement holds

$$\begin{array}{cc}\hfill \int D_{{\alpha }^{re}}(P_{\theta }^{\left(n\right)},P_{{\theta }_0}^{\left(n\right)}){\tilde{\pi }}_{n,{\alpha }^{re}|X^n}\left(d\theta \right)& \le \frac{{\alpha }^{re}}{1-{\alpha }^{re}}\int \mathcal{K}(P_{{\theta }_0}^{\left(n\right)},P_{\theta }^{\left(n\right)}){\rho }_n\left(d\theta \right)\hfill \\ & +\frac{{\alpha }^{re}}{1-{\alpha }^{re}}\sqrt{\frac{\mathrm{Var}[\int r_n(\theta ,{\theta }_0){\rho }_n\left(d\theta \right)]}{\eta }}\hfill \\ & +\frac{\mathcal{K}({\rho }_n,\pi )-log\left(ϵ\right)}{1-{\alpha }^{re}}.\hfill \end{array}$$ (A13)

Next, we observe that

$$\begin{array}{cc}\hfill \mathrm{Var}\left[\int r_n(\theta ,{\theta }_0){\rho }_n\left(d\theta \right)\right]& =E_{{\theta }_0}^{\left(n\right)}\left[{\left|\int r_n(\theta ,{\theta }_0){\rho }_n\left(d\theta \right)-E\left[\int r_n(\theta ,{\theta }_0){\rho }_n\left(d\theta \right)\right]\right|}^2\right]\hfill \\ \hfill & \le \int \mathrm{Var}\left[r_n(\theta ,{\theta }_0)\right]{\rho }_n\left(d\theta \right),\hfill \end{array}$$

by a straightforward application of Jensen’s inequality to the inner integral on the left hand side. Finally, following the hypotheses (i), (ii) and (iii), we have,

$$\begin{array}{cc}\hfill \int D_{{\alpha }^{re}}(P_{\theta }^{\left(n\right)},P_{{\theta }_0}^{\left(n\right)}){\tilde{\pi }}_{n,{\alpha }^{re}|X^n}\left(d\theta \right)& \le \frac{{\alpha }^{re}}{1-{\alpha }^{re}}\int \left(\mathcal{K}(P_{{\theta }_0}^{\left(n\right)},P_{\theta }^{\left(n\right)})+\sqrt{\frac{\int \mathrm{Var}\left[r_n(\theta ,{\theta }_0)\right]{\rho }_n\left(d\theta \right)}{\eta }}\right){\rho }_n\left(d\theta \right)\hfill \\ & +\frac{1}{{\alpha }^{re}}\left(\mathcal{K}({\rho }_n,\pi )-log\left(ϵ\right)\right)\hfill \\ \hfill & \le \frac{{\alpha }^{re}({ϵ}_n+\sqrt{\frac{n{ϵ}_n}{\eta }})}{1-{\alpha }^{re}}+\frac{n{ϵ}_n-log\left(ϵ\right)}{1-{\alpha }^{re}},\hfill \end{array}$$

thereby concluding the proof. □

Appendix B.1.3. Proof of Proposition 2

We define $Y_i:=log\left(\frac{p_{{\theta }_1}\left(X_i\right|X_{i-1})}{p_{{\theta }_2}\left(X_i\right|X_{i-1})}\right)$ for $i=1,\dots ,n$, and $Z_0=log\left(\frac{q_1^{\left(0\right)}\left(X_0\right)}{q_2^{\left(0\right)}\left(X_0\right)}\right)$. Then, using the Markov property we can see that the Kullback–Leibler divergence between the joint distributions $P_{{\theta }_1}^{\left(n\right)}$ and $P_{{\theta }_2}^{\left(n\right)}$ satisfies $\mathcal{K}\left(P_{{\theta }_1}^{\left(n\right)},P_{{\theta }_2}^{\left(n\right)}\right)={\sum }_{i=1}^n{\mathrm{E}}_{{\theta }_1}\left[Y_i\right]+{\mathrm{E}}_{{\theta }_1}\left[Z_0\right].$ If the Markov chain $\left\{X_i\right\}$ is stationary under ${\theta }_1$, so is $\left\{Y_i\right\}$. Hence $Y_i\stackrel{d}{=}{Y}_1$ and the above equation reduces to,

$$\begin{array}{c}\hfill \mathcal{K}\left(P_{{\theta }_1}^{\left(n\right)},P_{{\theta }_2}^{\left(n\right)}\right)=n{\mathrm{E}}_{{\theta }_1}\left[Y_1\right]+{\mathrm{E}}_{{\theta }_1}\left[Z_0\right].\end{array}$$ (A14)

□

Appendix B.1.4. Proof of Proposition 3

First, recall the following result from [19].

Lemma B.1.2.

[19] (Lemma 1.2) Let $X_{-\mathsf{\infty }},\cdots ,X_1,X_2,\cdots$ be an α-mixing Markov chain with α-mixing coefficients given by ${\alpha }_k$. Let ${\mathcal{M}}_a^b$ be the sigma-field generated by the subsequence $(X_a,X_{a+1},\cdots ,X_b)$. Let ${\eta }_t\in {\mathcal{M}}_{-\mathsf{\infty }}^t$ and ${\tau }_t\in {\mathcal{M}}_{t+k}^{\mathsf{\infty }}$ be adapted random variables such that $|{\eta }_t|\le 1,|{\tau }_t|\le 1$. Then,

$$\begin{array}{c}\hfill \underset{t}{sup}\underset{{\eta }_t,{\tau }_t}{sup}|\mathrm{E}\left[{\eta }_t{\tau }_t\right]-\mathrm{E}\left[{\eta }_t\right]\mathrm{E}\left[{\tau }_t\right]|\le 4{\alpha }_k.\end{array}$$ (A15)

This lemma provides an upper bound on the covariance of events $\eta \mathrm{and}\tau$, as shown next.

Lemma B.1.3.

Let $\eta \in {\mathcal{M}}_{-\mathsf{\infty }}^t$$\tau \in {\mathcal{M}}_{t+k}^{\mathsf{\infty }}$ be such that, ${E\left|\eta \right|}^{2+\delta }\le C_1,E{\left|\tau \right|}^{2+\delta }\le C_2\mathrm{for}\mathrm{some}\delta >0$. Then, for a fixed $n<+\mathsf{\infty }$, we have

$$|\mathrm{E}\eta \tau -\mathrm{E}\eta \mathrm{E}\tau |\le \left(\frac{4}{n}+2n^{\delta /2}(C_1+C_2)+2n^{\delta /2}\sqrt{C_1{C}_2}\right){\alpha }_k^{2\delta /(2+\delta )}.$$ (A16)

Proof. 

Let $N<+\mathsf{\infty }$ be a fixed number. We get from the triangle inequality that

$$\begin{array}{cc}\hfill |\mathrm{E}\eta \tau -\mathrm{E}\eta \mathrm{E}\tau |& \le |\mathrm{E}\eta \tau I_{\left[\right|\eta |\le N,|\tau |\le N]}-\mathrm{E}\eta I_{\left[\right|\eta |\le N]}\mathrm{E}\tau I_{\left[\right|\tau |\le N]}|\hfill \\ \hfill & +|\mathrm{E}\eta \tau I_{\left[\right|\eta |\ge N,|\tau |\le N]}-\mathrm{E}\eta I_{\left[\right|\eta |\ge N]}\mathrm{E}\tau I_{\left[\right|\tau |\le N]}|\hfill \\ \hfill & +|\mathrm{E}\eta \tau I_{\left[\right|\eta |\le N,|\tau |\ge N]}-\mathrm{E}\eta I_{\left[\right|\eta |\le N]}\mathrm{E}\tau I_{\left[\right|\tau |\ge N]}|\hfill \\ \hfill & +|\mathrm{E}\eta \tau I_{\left[\right|\eta |\ge N,|\tau |\ge N]}-\mathrm{E}\eta I_{\left[\right|\eta |\ge N]}\mathrm{E}\tau I_{\left[\right|\tau |\ge N]}|.\hfill \end{array}$$ (A17)

Multiplying and dividing the first term by $N^2$ and applying Lemma B.1.2, we get $|\mathrm{E}\eta \tau I_{\left[\right|\eta |\le N,|\tau |\le N]}-\mathrm{E}\eta I_{\left[\right|\eta |\le N]}\mathrm{E}\tau I_{\left[\right|\tau |\le N]}|\le 4N^2{\alpha }_k$. For the second term, if $\left|\tau \right|\le N$, then $\tau \le N$ and $\tau \ge -N$. Plugging this in the second term we get,

$$\begin{array}{}\mathrm{(A18)}& \hfill |\mathrm{E}\eta \tau I_{\left[\right|\eta |\ge N,|\tau |\le N]}-& \mathrm{E}\eta I_{\left[\right|\eta |\ge N]}\mathrm{E}\tau I_{\left[\right|\tau |\le N]}|\le \left|N\mathrm{E}\eta I_{\left[\right|\eta |\ge N}+N\left[\mathrm{E}\eta I_{\left[\right|\eta |\ge N]}\right]\right|\hfill \mathrm{(A19)}& \hfill & =2N|\mathrm{E}\eta I_{\left[\right|\eta |\ge N]}|.\hfill \end{array}$$

Since $\left|\eta \right|\ge N$, we have $1\le \frac{{\left|\eta \right|}^{1+\delta }}{N^{1+\delta }}$. Following this,

$$\begin{array}{}\mathrm{(A20)}& \hfill |2N\mathrm{E}\eta I_{\left[\right|\eta |\ge N]}|& \le 2N\left|\mathrm{E}\left[\frac{{\left|\eta \right|}^{2+\delta }}{N^{1+\delta }}{I}_{\left[\right|\eta |\ge N]}\right]\right|\hfill \mathrm{(A21)}& \hfill & \le 2N\frac{1}{N^{1+\delta }}\left|\mathrm{E}{\eta }^{2+\delta }\right|\le 2\frac{C_1}{N^{\delta }}.\hfill \end{array}$$

Similarly, we can also write for the third term, $|\mathrm{E}\eta \tau I_{\left[\right|\eta |\le N,|\tau |\ge N]}-\mathrm{E}\eta I_{\left[\right|\eta |\le N]}\mathrm{E}\tau I_{\left[\right|\tau |\ge N]}|\le 2\frac{C_2}{N^{\delta }}$. Finally, for the last term we get that by Cauchy-Schwarz inequality,

$$\begin{array}{cc}\hfill |\mathrm{E}\eta \tau I_{\left[\right|\eta |\ge N,|\tau |\ge N]}-\mathrm{E}\eta I_{\left[\right|\eta |\ge N]}\mathrm{E}\tau I_{\left[\right|\tau |\ge N]}|& \le \sqrt{\mathrm{Var}\left[\eta I_{\left[\right|\eta |\ge N]}\right]\mathrm{Var}\left[\tau I_{\left[\right|\tau |\ge N]}\right]}\hfill \end{array}$$ (A22)

$$\begin{array}{cc}\hfill & <2\sqrt{\mathrm{Var}\left[\eta I_{\left[\right|\eta |\ge N]}\right]\mathrm{Var}\left[\tau I_{\left[\right|\tau |\ge N]}\right]}\hfill \end{array}$$ (A23)

$$\begin{array}{cc}\hfill & \le 2\sqrt{\mathrm{E}\left[{\eta }^2{I}_{\left[\right|\eta |\ge N]}\right]\mathrm{E}\left[{\tau }^2{I}_{\left[\right|\tau |\ge N]}\right]}.\hfill \end{array}$$ (A24)

Since $\left|\eta \right|>N$, $1<\frac{{\left|\eta \right|}^{\delta }}{N^{\delta }}$. Similarly, $1<\frac{{\left|\tau \right|}^{\delta }}{N^{\delta }}$. Plugging these in the previous equation, we get,

$$\begin{array}{cc}\hfill \sqrt{\mathrm{E}\left[{\eta }^2{I}_{\left[\right|\eta |\ge N]}\right]\mathrm{E}\left[{\tau }^2{I}_{\left[\right|\tau |\ge N]}\right]}& \le \sqrt{\frac{1}{N^{2\delta }}\mathrm{E}\left[{\left|\eta \right|}^{2+\delta }{I}_{\left[\right|\eta |\ge N]}\right]\mathrm{E}\left[{\left|\tau \right|}^{2+\delta }{I}_{\left[\right|\tau |\ge N]}\right]}\hfill \end{array}$$ (A25)

$$\begin{array}{cc}\hfill & \le \frac{1}{N^{\delta }}\sqrt{C_1{C}_2}.\hfill \end{array}$$ (A26)

Combining the four upper bounds above, we get,

$$\begin{array}{cc}\hfill |\mathrm{E}\eta \tau -\mathrm{E}\eta \mathrm{E}\tau |& \le 4N^2{\alpha }_k+\frac{2}{N^{\delta }}(C_1+C_2)+\frac{2}{N^{\delta }}\sqrt{C_1{C}_2}.\hfill \end{array}$$ (A27)

Now, in particular, setting $N=n^{-1/2}{\alpha }_k^{-1/(2+\delta )}$ it follows that

$$\begin{array}{cc}\hfill |\mathrm{E}\eta \tau -\mathrm{E}\eta \mathrm{E}\tau |& \le \frac{4}{n}{\alpha }_k^{\delta /(2+\delta )}+2n^{\delta /2}{\alpha }_k^{\delta /(2+\delta )}(C_1+C_2)+2n^{\delta /2}{\alpha }_k^{\delta /(2+\delta )}\sqrt{C_1{C}_2}\hfill \end{array}$$ (A28)

$$\begin{array}{cc}\hfill & =\left(\frac{4}{n}+2n^{\delta /2}(C_1+C_2)+2n^{\delta /2}\sqrt{C_1{C}_2}\right){\alpha }_k^{\delta /(2+\delta )}.\hfill \end{array}$$ (A29)

□

Lemma B.1.4.

Let $\left\{X_t\right\}$ be an α-mixing Markov chain with mixing coefficient ${\alpha }_k$. Further assume that $\mathrm{E}|X_t{|}^{2+\delta }\le C_1\mathrm{and}\mathrm{E}{\left|X_{t+k}\right|}^{2+\delta }\le C_2$ for some $\delta >0$. Then, for any t and any $n>0$

$$|\mathrm{Cov}(X_t,X_{t+k})|\le \left(\frac{4}{n}+2n^{\delta /2}(C_1+C_2)+2n^{\delta /2}\sqrt{C_1{C}_2}\right){\alpha }_k^{\delta /(2+\delta )}.$$ (A30)

Proof. 

Set $\eta =X_t,\tau =X_{t+k}$ Lemma B.1.3. □

We also need to establish the following technical lemma.

Lemma B.1.5.

Let $\left\{X_t\right\}$ be an α-mixing Markov Chain with mixing coefficients $\left\{{\alpha }_t\right\}$. Then the process $\left\{Y_t\right\}$ where $Y_t:=log\left(\frac{p_{{\theta }_0}\left(X_t\right|X_{t-1})}{p_{\theta }\left(X_t\right|X_{t-1})}\right)$ is also α-mixing with mixing coefficients $\left\{{\tilde{\alpha }}_t\right\}$ where ${\tilde{\alpha }}_t={\alpha }_{t-1}$.

Proof. 

By $Z_i$ denote the paired random measure $(X_i,X_{i-1})$. Let ${\mathcal{M}}_i^j$ denote the sigma field generated by the measures $X_k,\mathrm{where}i\le k\le j$. By ${\mathcal{G}}_i^j$ denote the sigma field generated by the measures $Z_k,\mathrm{where}i\le k\le j$. Let $C\in {\mathcal{M}}_{i-1}^j$. Then, C can be expressed as $(C_{i-1}\times C_i\times \cdots \times C_j)$. for $C_{i-1}\in {\mathcal{M}}_{i-1}^{i-1},C_i\in {\mathcal{M}}_i^i\cdots$ and so on. Now, consider a map. $T_i^j:(C_{i-1}\times C_i\times \cdots \times C_j)\stackrel{}{\to }(C_{i-1}\times C_i\times C_i\times \cdots \times C_{j-1}\times C_{j-1}\times C_j)$. Note that, $T_i^j\left(C\right)\in {\mathcal{G}}_i^j$. It is easy to see that ${\mathcal{G}}_i^j=T_i^j\left({\mathcal{M}}_{i-1}^j\right)\cup {\mathcal{M}}_{i-1}^{*j}$, where $T_i^j\left({\mathcal{M}}_{i-1}^j\right)$ is obtained by applying the map $T_i^j$ to each element of ${\mathcal{M}}_{i-1}^j$. If we assume this latter set to be the range and ${\mathcal{M}}_{i-1}^j$ to be the domain, then, by construction, $T_i^j$ is a bijection. Furthermore, the two classes are made of disjoint sets, i.e., if $A\in T_i^j\left({\mathcal{M}}_{i-1}^j\right)\mathrm{and}{A}^{*}\in {\mathcal{M}}_{i-1}^{*j}$, then $A\cap A^{*}=\varphi$. Furthermore, note that ${\mathcal{M}}_{i-1}^{j*}$ is made of impossible sets. i.e., $P\left(A^{*}\right)=0\forall A^{*}\in {\mathcal{M}}_{i-1}^{j*}$. Now consider the $\alpha$-mixing coefficients for $Z_i$. By definition, it is given by

$$\begin{array}{cc}\hfill {\alpha }_k^z& =\underset{i}{sup}\underset{A\in {\mathcal{G}}_{-\mathsf{\infty }}^i,B\in {\mathcal{G}}_{i+k}^{\mathsf{\infty }}}{sup}|P(A\cap B)-P\left(A\right)P\left(B\right)|\hfill \\ \hfill & =\underset{i}{sup}\underset{A\in {\mathcal{G}}_{-\mathsf{\infty }}^i,B\in {\mathcal{G}}_{i+k}^{\mathsf{\infty }}}{sup}|P((A^o\cup A^{*})\cap (B^o\cup B^{*}))-P\left((A^o\cup A^{*})\right)P\left((B^o\cup B^{*})\right)|.\hfill \end{array}$$

where,

$A=(A^o\cup A^{*})$	$B=(B^o\cup B^{*})$
$A^o\in {\mathcal{T}}_{-\mathsf{\infty }}^i\left(M_{-\mathsf{\infty }}^i\right)$	$A^{*}\in {\mathcal{M}}_{-\mathsf{\infty }}^{*i}$
$B^o\in T_{i+k-1}^{\mathsf{\infty }}\left({\mathcal{M}}_{j+k-1}^{\mathsf{\infty }}\right)$	$B^{*}\in {\mathcal{M}}_{j+k-1}^{*\mathsf{\infty }}.$

Then, the expression for the $\alpha$-mixing coefficient can be reduced into

$$\begin{array}{cc}\hfill {\alpha }_k^z& =\underset{i}{sup}\underset{A^o\in T_{-\mathsf{\infty }}^i\left({\mathcal{M}}_{-\mathsf{\infty }}^i\right),B^o\in T_{i+k-1}^{\mathsf{\infty }}\left({\mathcal{M}}_{i+k-1}^{\mathsf{\infty }}\right)}{sup}|P(A^o\cap B^o)-P\left(A^o\right)P\left(B^o\right)|.\hfill \end{array}$$

Note that, by bijection property of $T_i^j$, we can find $A^{\prime }\in {\mathcal{M}}_{-\mathsf{\infty }}^i$ and $B^{\prime }\in {\mathcal{M}}_{i+k-1}^{\mathsf{\infty }}$ such that

$$\begin{array}{cc}\hfill {\alpha }_k^z& =\underset{i}{sup}\underset{A^{\prime }\in {\mathcal{M}}_{-\mathsf{\infty }}^i,B^{\prime }\in {\mathcal{M}}_{i+k-1}^{\mathsf{\infty }}}{sup}|P(T_{-\mathsf{\infty }}^i\left(A^{\prime }\right)\cap T_{i+k-1}^{\mathsf{\infty }}\left(B^{\prime }\right))-P\left(T_{-\mathsf{\infty }}^i\left(A^{\prime }\right)\right)P\left(T_{i+k-1}^{\mathsf{\infty }}\left(B^{\prime }\right)\right)|.\hfill \\ \hfill & ={\alpha }_{k-1}.\hfill \end{array}$$

Now, $log\left(\frac{p_{{\theta }_0}\left(X_n\right|X_{n-1})}{p_{\theta }\left(X_n\right|X_{n-1})}\right)$ is just a function of the paired Markov chain $Z_i$, therefore it has $\alpha$-mixing coefficient ${\alpha }_{k-1}$. □

We now proceed to the proof of Proposition 3. Let $\left\{X_k\right\}$ be a stationary $\alpha$-mixing Markov chain under ${\theta }_1$ with mixing coefficients $\left\{{\alpha }_k\right\}$. Observe that the log-likelihood can be expressed as

$$\begin{array}{cc}\hfill r_n({\theta }_2,{\theta }_1)& =\sum _{i=1}^{n}log\left(\frac{p_{{\theta }_1}\left(X_i\right|X_{i-1})}{p_{{\theta }_2}\left(X_i\right|X_{i-1})}\right)+log\left(\frac{q_1^{\left(0\right)}\left(X_0\right)}{q_2^{\left(0\right)}\left(X_0\right)}\right)\hfill \\ \hfill & \equiv \sum _{i=1}^n{Y}_i+Z_0.\hfill \end{array}$$

Therefore, the variance of the log-likelihood ratio is simply

$$\begin{array}{cc}\hfill {\mathrm{Var}}_{{\theta }_1}\left[r_n({\theta }_2,{\theta }_1)\right]& ={\mathrm{Var}}_{{\theta }_1}\left[\sum _{i=1}^n{Y}_i+Z_0\right]\hfill \\ \hfill & =\sum _{i,j=1}^n{\mathrm{Cov}}_{{\theta }_1}(Y_i,Y_j)+\sum _{i,j=1}^n{\mathrm{Cov}}_{{\theta }_1}(Y_i,Z_0)+{\mathrm{Cov}}_{{\theta }_1}(Z_0,Z_0).\hfill \end{array}$$

It follows from Lemma B.1.5 that $\left\{Y_k\right\}$ is a stochastic process with $\alpha$-mixing coefficients ${\alpha }_{k-1}$. Therefore, using Lemma B.1.4 we have

$$\begin{array}{cc}\hfill |{\mathrm{Cov}}_{{\theta }_1}(Y_i,Y_j)|& =|{\mathrm{E}}_{{\theta }_1}{Y}_i{Y}_j-{\mathrm{E}}_{{\theta }_1}{Y}_i{\mathrm{E}}_{{\theta }_1}{Y}_j|\hfill \\ \hfill & <(\frac{4}{n}+2n^{\delta /2}({\mathrm{E}}_{{\theta }_1}|Y_i{|}^{2+\delta }+{\mathrm{E}}_{{\theta }_1}{\left|Y_j\right|}^{2+\delta }\hfill \\ \hfill & +\sqrt{{\mathrm{E}}_{{\theta }_1}|Y_i{|}^{2+\delta }{\mathrm{E}}_{{\theta }_1}{\left|Y_j\right|}^{2+\delta }}\left)\right){\alpha }_{|j-i|-1}^{\delta /(2+\delta )}\hfill \\ \hfill & =\left(\frac{4}{n}+2n^{\delta /2}(C_{{\theta }_1,{\theta }_2}^{\left(i\right)}+C_{{\theta }_1,{\theta }_2}^{\left(j\right)}+\sqrt{C_{{\theta }_1,{\theta }_2}^{\left(i\right)}{C}_{{\theta }_1,{\theta }_2}^{\left(j\right)}})\right){\alpha }_{|j-i|-1}^{\delta /(2+\delta )}.\hfill \end{array}$$

Similarly, as above we can also say

$$\begin{array}{cc}\hfill |{\mathrm{Cov}}_{{\theta }_1}(Y_i,Z_0)|<& \left(\frac{4}{n}+2n^{\delta /2}(C_{{\theta }_1,{\theta }_2}^{\left(i\right)}+D_{1,2}+\sqrt{C_{{\theta }_1,{\theta }_2}^{\left(i\right)}{D}_{1,2}})\right)\left({\alpha }_{i-1}^{\delta /(2+\delta )}\right)\hfill \end{array}$$

Combining, the two upper bounds above, we get the first result:

$$\begin{array}{cc}\hfill {\mathrm{Var}}_{{\theta }_1}\left[r_n({\theta }_2,{\theta }_1)\right]& <\sum _{i,j=1}^n\left(\frac{4}{n}+2n^{\delta /2}(C_{{\theta }_1,{\theta }_2}^{\left(i\right)}+C_{{\theta }_1,{\theta }_2}^{\left(j\right)}+\sqrt{C_{{\theta }_1,{\theta }_2}^{\left(i\right)}{C}_{{\theta }_1,{\theta }_2}^{\left(j\right)}})\right)\left({\alpha }_{|i-j|-1}^{\delta /(2+\delta )}\right)\hfill \\ \hfill & +\sum _{i=1}^n\left(\frac{4}{n^2}+2n^{\delta /2}(C_{{\theta }_1,{\theta }_2}^{\left(i\right)}+D_{1,2}+\sqrt{C_{{\theta }_1,{\theta }_2}^{\left(i\right)}{D}_{1,2}})\right)\left({\alpha }_{i-1}^{\delta /(2+\delta )}\right)\hfill \\ \hfill & +\mathrm{Var}[Z_0,Z_0].\hfill \end{array}$$

If $\left\{X_i\right\}$ is stationary under ${\theta }_1$, so is $\left\{Y_i\right\}$. Therefore, ${\mathrm{E}}_{{\theta }_1}|Y_i{|}^{2+\delta }={\mathrm{E}}_{{\theta }_1}{\left|Y_1\right|}^{2+\delta }=C_{{\theta }_1,{\theta }_2}^{\left(1\right)}\forall i$, and

$$\begin{array}{cc}\hfill \sum _{i,j=1}^n{\mathrm{Cov}}_{{\theta }_1}(Y_i,Y_j)& \le \sum _{i,j=1}^n\left(\frac{4}{n}+6n^{\delta /2}{C}_{{\theta }_1,{\theta }_2}^{\left(1\right)}\right){\alpha }_{|j-i|-1}^{\delta /(2+\delta )}\hfill \\ \hfill & \le n\left(\frac{4}{n}+6n^{\delta /2}{C}_{{\theta }_1,{\theta }_2}^{\left(1\right)}\right)\left(\sum _{h\ge 1}{\alpha }_{h-1}^{\delta /(2+\delta )}\right).\hfill \end{array}$$ (A31)

Again, using Lemma B.1.4 on ${\mathrm{Cov}}_{{\theta }_1}(Y_i,Z_0)$, yields

$$\sum _{i=1}^n{\mathrm{Cov}}_{{\theta }_1}(Y_i,Z_0)\le \left(\frac{4}{n}+2n^{\delta /2}(C_{\theta }+D_{1,2}+\sqrt{C_{\theta }{D}_{1,2}})\right)\left(\sum _{h\ge 1}{\alpha }_h^{\delta /(2+\delta )}\right).$$ (A32)

Finally, using Equations (A31) and (A32) we have

$$\begin{array}{cc}\hfill {\mathrm{Var}}_{{\theta }_1}\left[r_n({\theta }_2,{\theta }_1)\right]& \le n\left(\frac{4}{n}+6n^{\delta /2}{C}_{{\theta }_1,{\theta }_2}^{\left(1\right)}\right)\left(\sum _{h\ge 1}{\alpha }_{h-1}^{\delta /(2+\delta )}\right)+\hfill \\ \hfill & \left(\frac{4}{n}+2n^{\delta /2}(C_{{\theta }_1,{\theta }_2}^{\left(1\right)}+D_{1,2}+\sqrt{C_{{\theta }_1,{\theta }_2}^{\left(1\right)}{D}_{1,2}})\right)\left(\sum _{h\ge 1}{\alpha }_h^{\delta /(2+\delta )}\right)\hfill \\ \hfill & +{\mathrm{Cov}}_{{\theta }_1}(Z_0,Z_0).\hfill \end{array}$$

□

Appendix B.2. Proofs for Stationary Markov Data-Generating Models

Proof of Theorem 2

Part 1: Verifying condition (i) of Corollary 1.

We substitute the true parameter ${\theta }_0$ for ${\theta }_1$ and $\theta$ for ${\theta }_2$. We also set $q_1^{\left(0\right)}$ to be the invariant distribution of the Markov chain under ${\theta }_0$, $q_0$, and $q_2^{\left(0\right)}$ as the invariant distribution of the Markov chain under $\theta$, $q_{\theta }$. Applying the fact that these Markov chains are stationary to Proposition 2, we have

$$\begin{array}{cc}\hfill \mathcal{K}(P_{{\theta }_0}^{\left(n\right)},P_{\theta }^{\left(n\right)})& =n\mathrm{E}\left[log\left(\frac{p_{{\theta }_0}\left(X_1\right|X_0)}{p_{\theta }\left(X_1\right|X_0)}\right)\right]+\mathrm{E}\left[Z_0\right],\hfill \\ \hfill & \le n\sum _{j=1}^m\mathrm{E}\left[M_j^{\left(1\right)}(X_1,X_0)\right]\left|f_j^{\left(1\right)}(\theta ,{\theta }_0)\right|+\sum _{k=1}^m\mathrm{E}\left[M_k^{\left(2\right)}\left(X_0\right)\right]\left|f_k^{\left(2\right)}(\theta ,{\theta }_0)\right|,\hfill \end{array}$$ (A33)

where the inequality follows from Assumption 1. Therefore, it follows that

$$\begin{array}{cc}\hfill \int \mathcal{K}(P_{{\theta }_0}^{\left(n\right)},P_{\theta }^{\left(n\right)}){\rho }_n\left(d\theta \right)& \le n\sum _{j=1}^m\mathrm{E}\left[M_j^{\left(1\right)}(X_1,X_0)\right]\int \left|f_j^{\left(1\right)}(\theta ,{\theta }_0)\right|{\rho }_n\left(d\theta \right)\hfill \\ \hfill & +\sum _{k=1}^m\mathrm{E}\left[M_k^{\left(2\right)}\left(X_0\right)\right]|\int f_k^{\left(2\right)}(\theta ,{\theta }_0)|{\rho }_n\left(d\theta \right).\hfill \end{array}$$

By Assumption 1(i), it follows that

$$\begin{array}{cc}\hfill \int \mathcal{K}(P_{{\theta }_0}^{\left(n\right)},P_{\theta }^{\left(n\right)}){\rho }_n\left(d\theta \right)& \le n\sum _{j=1}^m\mathrm{E}\left[M_j^{\left(1\right)}(X_1,X_0)\right]\frac{C}{\sqrt{n}}+\sum _{k=1}^m\mathrm{E}\left[M_k^{\left(2\right)}\left(X_0\right)\right]\frac{C}{\sqrt{n}}\le n{ϵ}_n^{\left(1\right)},\hfill \end{array}$$

where ${ϵ}_n^{\left(1\right)}=\mathrm{O}\left(\frac{1}{\sqrt{n}}\right)$.

Part 2: Verifying condition (ii) of 1. Again, using Proposition 3 along with the fact that the Markov chain is stationary we have

$$\begin{array}{cc}\hfill \mathrm{Var}\left[r_n(\theta ,{\theta }_0)\right]& \le n\left(\frac{4}{n}+6n^{\delta /2}{C}_{{\theta }_0,\theta }^{\left(1\right)}\right)\left(\sum _{k\ge 0}{\alpha }_k^{\delta /(2+\delta )}\right)\hfill \\ \hfill & +\left(\frac{4}{n^2}+2n^{\delta /2}(C_{{\theta }_0,\theta }^{\left(1\right)}+D_{{\theta }_0,\theta }+\sqrt{C_{{\theta }_0,\theta }^{\left(1\right)}{D}_{{\theta }_0,\theta }})\right)\left(\sum _{k\ge 1}{\alpha }_k^{\delta /(2+\delta )}\right)\hfill \\ \hfill & +\mathrm{Var}\left[Z_0\right].\hfill \end{array}$$

It then follows that

$$\begin{array}{cc}\hfill \int \mathrm{Var}\left[r_n(\theta ,{\theta }_0)\right]{\rho }_n\left(d\theta \right)& \le n\left(\frac{4}{n}+6n^{\delta /2}\int C_{{\theta }_0,\theta }^{\left(1\right)}{\rho }_n\left(d\theta \right)\right)\left(\sum _{k\ge 1}{\alpha }_{k-1}^{\delta /(2+\delta )}\right)+\int \mathrm{Var}\left[Z_0\right]{\rho }_n\left(d\theta \right)\hfill \\ \hfill & +(\frac{4}{n^2}+2n^{\delta /2}(\int C_{{\theta }_0,\theta }^{\left(1\right)}{\rho }_n\left(d\theta \right)\hfill \\ \hfill & +\int D_{{\theta }_0,\theta }{\rho }_n\left(d\theta \right)+\int \sqrt{C_{{\theta }_0,\theta }^{\left(1\right)}{D}_{{\theta }_0,\theta }}{\rho }_n\left(d\theta \right)\left)\right)\left(\sum _{k\ge 1}{\alpha }_k^{\delta /(2+\delta )}\right).\hfill \end{array}$$

First, consider the term $\int C_{{\theta }_0,\theta }^{\left(1\right)}{\rho }_n\left(\theta \right)$, and observe that

$$\begin{array}{cc}\hfill \int C_{{\theta }_0,\theta }^{\left(1\right)}{\rho }_n\left(d\theta \right)& =\int \mathrm{E}log{\left|\frac{p_{{\theta }_0}\left(X_1\right|X_0)}{p_{\theta }\left(X_1\right|X_0)}\right|}^{2+\delta }{\rho }_n\left(d\theta \right).\hfill \end{array}$$

By Assumption 1, we have

$$\begin{array}{cc}\hfill \int \mathrm{E}log{\left|\frac{p_{{\theta }_0}\left(X_1\right|X_0)}{p_{\theta }\left(X_1\right|X_0)}\right|}^{2+\delta }{\rho }_n\left(d\theta \right)& \le \int \mathrm{E}{\left[\sum _{j=1}^m{M}_j^{\left(1\right)}(X_1,X_0)\left|f_k^{\left(1\right)}(\theta ,{\theta }_0)\right|\right]}^{2+\delta }{\rho }_n\left(d\theta \right).\hfill \end{array}$$

Since the function $x\mapsto x^{2+\delta }$ is convex, we can apply Jensen’s inequality to obtain,

$$\begin{array}{cc}\hfill {\left(\sum _{j=1}^m{M}_j^{\left(1\right)}(X_1,X_0)\left|f_k^{\left(1\right)}(\theta ,{\theta }_0)\right|\right)}^{2+\delta }& \le m^{1+\delta }\sum _{k=1}^m{M}_j^{\left(1\right)}{(X_1,X_0)}^{2+\delta }{\left|f_k^{\left(1\right)}(\theta ,{\theta }_0)\right|}^{2+\delta }.\hfill \end{array}$$

Therefore, it follows that

$$\begin{array}{cc}\hfill \int \mathrm{E}log{\left|\frac{p_{{\theta }_0}\left(X_1\right|X_0)}{p_{\theta }\left(X_1\right|X_0)}\right|}^{2+\delta }{\rho }_n\left(d\theta \right)& \le m^{1+\delta }\sum _{k=1}^m\mathrm{E}\left[M_k^{\left(1\right)}{(X_1,X_0)}^{2+\delta }\right]\hfill \\ \hfill & \times \int |f_k^{\left(1\right)}(\theta ,{\theta }_0){|}^{2+\delta }{\rho }_n\left(d\theta \right).\hfill \end{array}$$

By Assumption 1, $\int |f_k(\theta ,{\theta }_0){|}^{2+\delta }{\rho }_n\left(d\theta \right)<\frac{C}{n}$ and $\mathrm{E}\left[M_k^{\left(1\right)}{(X_1,X_0)}^{2+\delta }\right]<B$, implying that

$$\begin{array}{cc}\hfill \int C_{{\theta }_0,\theta }^{\left(1\right)}{\rho }_n\left(d\theta \right)& \le m^{1+\delta }\sum _{k=1}^{m}B\frac{C}{n}=m^{2+\delta }\frac{BC}{n}.\hfill \end{array}$$

Since $\left({\sum }_{k\ge 0}{\alpha }_k^{\delta /(2+\delta )}\right)<\mathsf{\infty }$, it follows that $\left(\frac{4}{n}+6n^{\delta /2}\int C_{{\theta }_0,\theta }^{\left(1\right)}{\rho }_n\left(d\theta \right)\right)\left({\sum }_{k\ge 1}{\alpha }_{k-1}^{\delta /(2+\delta )}\right)=\mathrm{O}\left(\frac{n^{\delta /2}}{n}\right)$. Similarly, we can show that $\int D_{{\theta }_0,\theta }{\rho }_n\left(d\theta \right)=\mathrm{O}\left(\frac{1}{n}\right)$, and $\int \mathrm{Var}\left[Z_0\right]{\rho }_n\left(d\theta \right)=\mathrm{O}\left(\frac{1}{n}\right)$.

For the final term $\int \sqrt{C_{{\theta }_0,\theta }^{\left(1\right)}{D}_{{\theta }_0,\theta }}{\rho }_n\left(d\theta \right)$, use the Cauchy-Schwarz inequality to obtain the upper bound ${\left(\int C_{{\theta }_0,\theta }^{\left(1\right)}{\rho }_n\left(d\theta \right)\int D_{{\theta }_0,\theta }{\rho }_n\left(d\theta \right)\right)}^{1/2}$ which is also of order $O\left(\frac{1}{n}\right)$. Combining all of these together we have

$$\begin{array}{c}\hfill \int \mathrm{Var}\left[r_n(\theta ,{\theta }_0)\right]{\rho }_n\left(d\theta \right)\le n{ϵ}_n^{\left(2\right)},\end{array}$$

for some ${ϵ}_n^{\left(2\right)}=\mathrm{O}\left(\frac{n^{\delta /2}}{n}\right)$.

Since $\mathcal{K}({\rho }_n,\pi )<\sqrt{n}C=n\frac{C}{\sqrt{n}},$ it follows that $\mathcal{K}({\rho }_n,\pi )<n{ϵ}_n^{\left(3\right)},$ where ${ϵ}_n^{\left(3\right)}=\mathrm{O}(1/\sqrt{n})$ as before. Finally, by choosing ${ϵ}_n=max({ϵ}_n^{\left(1\right)},{ϵ}_n^{\left(2\right)},{ϵ}_n^{\left(3\right)})$, our theorem is proved. □

Appendix B.3. Proofs for Non-Stationary, Ergodic Markov Data-Generating Models

Appendix B.3.1. Proof of Theorem 3

Part 1: Verifying condition (i) of Corollary 1: As in the proof of Theorem 2 substitute the true parameter ${\theta }_0$ for ${\theta }_1$ and $\theta$ for ${\theta }_2$ in. We also set $q_1^{\left(0\right)}$ and $q_2^{\left(0\right)}$ to the distribution $q^{\left(0\right)}$. Applying Proposition 2 to the corresponding transition kernels and initial distribution we have,

$$\begin{array}{cc}\hfill \mathcal{K}(P_{{\theta }_0}^{\left(n\right)},P_{\theta }^{\left(n\right)})& =\sum _{i=1}^n\mathrm{E}\left[log\left(\frac{p_{{\theta }_0}\left(X_i\right|X_{i-1})}{p_{\theta }\left(X_i\right|X_{i-1})}\right)\right]+\mathrm{E}\left[log\left(\frac{D\left(X_0\right)}{D\left(X_0\right)}\right)\right]\hfill \\ \hfill & =\sum _{i=1}^n\mathrm{E}\left[log\left(\frac{p_{{\theta }_0}\left(X_i\right|X_{i-1})}{p_{\theta }\left(X_i\right|X_{i-1})}\right)\right].\hfill \end{array}$$ (A34)

Now, applying Assumption 1, we can bound the previous equation as follows,

$$\begin{array}{cc}\hfill \mathcal{K}(P_{{\theta }_0}^{\left(n\right)},P_{\theta }^{\left(n\right)})& \le \sum _{i=1}^n\mathrm{E}\left[\sum _{k=1}^m{M}_k^{\left(1\right)}(X_i,X_{i-1})\left|f_k^{\left(1\right)}(\theta ,{\theta }_0)\right|\right]\hfill \\ \hfill & =\sum _{i=1}^n\sum _{k=1}^m\mathrm{E}\left[M_k^{\left(1\right)}(X_i,X_{i-1})\right]\left|f_k^{\left(1\right)}(\theta ,{\theta }_0)\right|.\hfill \end{array}$$ (A35)

Since $M_k^{\left(1\right)}$’s are bounded there exists a constant Q so that,

$$\begin{array}{cc}\hfill \int \mathcal{K}(P_{{\theta }_0}^{\left(n\right)},P_{\theta }^{\left(n\right)}){\rho }_n\left(d\theta \right)& \le Q\int \sum _{i=1}^n\sum _{k=1}^m\left|f_k^{\left(1\right)}(\theta ,{\theta }_0)\right|{\rho }_n\left(d\theta \right)\hfill \\ \hfill & =Qn\sum _{k=1}^m\int \left|f_k^{\left(1\right)}(\theta ,{\theta }_0)\right|{\rho }_n\left(d\theta \right).\hfill \end{array}$$

By Assumption 19 in Assumption 1, it follows that

$$\begin{array}{cc}\hfill \int \mathcal{K}(P_{{\theta }_0}^{\left(n\right)},P_{\theta }^{\left(n\right)}){\rho }_n\left(d\theta \right)& \le Qn\sum _{k=1}^m\frac{C}{\sqrt{n}}=nmQ\frac{C}{\sqrt{n}}=n{ϵ}_n^{\left(1\right)},\hfill \end{array}$$

for some ${ϵ}_n^{\left(1\right)}=\mathrm{O}\left(\frac{1}{\sqrt{n}}\right)$.

Part 2: Verifying condition (ii) of Corollary 1: As in the previous part, $Z_0=0$, implying that $D_{\theta ,{\theta }_0}$. Applying Proposition 3 and integrating with respect to ${\rho }_n$, we obtain

$$\begin{array}{cc}\hfill \int \mathrm{Var}\left[r_n(\theta ,{\theta }_0)\right]{\rho }_n\left(d\theta \right)& \le \sum _{i=1}^n\left(\frac{4}{n}+2n^{\delta /2}\int C_{{\theta }_0,\theta }^{\left(i\right)}{\rho }_n\left(d\theta \right)\right)\left({\alpha }_{i-1}^{\delta /(2+\delta )}\right)\hfill \\ \hfill & +\sum _{i,j=1}^n\left(\frac{4}{n}+2n^{\delta /2}(\int C_{{\theta }_0,\theta }^{\left(i\right)}{\rho }_n\left(d\theta \right)+\int C_{{\theta }_0,\theta }^{\left(j\right)}{\rho }_n\left(d\theta \right)+\int \sqrt{C_{{\theta }_0,\theta }^{\left(i\right)}{C}_{{\theta }_0,\theta }^{\left(j\right)}}{\rho }_n\left(d\theta \right))\right)\hfill \\ \hfill & \times \left({\alpha }_{|i-j|-1}^{\delta /(2+\delta )}\right).\hfill \end{array}$$ (A36)

First, consider the term $\int C_{{\theta }_0,\theta }^{\left(i\right)}{\rho }_n\left(d\theta \right)$. Using Assumption 1, we can upper bound $C_{{\theta }_0,\theta }^{\left(i\right)}$ as,

$$\begin{array}{cc}\hfill C_{{\theta }_0,\theta }^{\left(i\right)}& \le \mathrm{E}{\left[\sum _{k=1}^m{M}_k^{\left(1\right)}(X_i,X_{i-1})\left|f_k^{\left(1\right)}(\theta ,{\theta }_0)\right|\right]}^{2+\delta }\hfill \\ \hfill & \le \sum _{k=1}^m{m}^{1+\delta }\mathrm{E}\left[{\left(M_k^{\left(1\right)}(X_i,X_{i-1})\left|f_k^{\left(1\right)}(\theta ,{\theta }_0)\right|\right)}^{2+\delta }\right](\mathrm{by}\mathrm{Jensen}’\mathrm{s}\mathrm{inequality})\hfill \\ \hfill & =\sum _{k=1}^m{m}^{1+\delta }\mathrm{E}\left[M_k^{\left(1\right)}{(X_i,X_{i-1})}^{2+\delta }\right]{\left|f_k^{\left(1\right)}(\theta ,{\theta }_0)\right|}^{2+\delta }.\hfill \end{array}$$

Since $M_k^{\left(1\right)}$’s are upper bounded by Q, it follows from the previous expression that, $C_{{\theta }_0,\theta }^{\left(i\right)}\le {\sum }_{k=1}^m{m}^{1+\delta }{Q}^{2+\delta }{\left|f_k^{\left(1\right)}(\theta ,{\theta }_0)\right|}^{2+\delta }$.

Hence, from Assumption 1, we get,

$$\begin{array}{cc}\hfill \int C_{{\theta }_0,\theta }^{\left(i\right)}{\rho }_n\left(d\theta \right)& \le \sum _{k=1}^m{m}^{1+\delta }{Q}^{2+\delta }\int {\left|f_k^{\left(1\right)}(\theta ,{\theta }_0)\right|}^{2+\delta }{\rho }_n\left(d\theta \right)\le {\left(mQ\right)}^{2+\delta }\frac{C}{n}.\hfill \end{array}$$

Using the upper bound above, we can say for an L large enough, $\int C_{{\theta }_0,\theta }^{\left(i\right)}{\rho }_n\left(d\theta \right)\le \frac{L}{n}$. Next, by the Cauchy-Schwarz inequality, we have that $\int \sqrt{C_{{\theta }_0,\theta }^{\left(i\right)}{C}_{{\theta }_0,\theta }^{\left(j\right)}{\rho }_n\left(d\theta \right))}<\sqrt{\int C_{{\theta }_0,\theta }^{\left(i\right)}{\rho }_n\left(d\theta \right)\int C_{{\theta }_0,\theta }^{\left(j\right)}{\rho }_n\left(d\theta \right))}\le \frac{L}{n}$. Thus, we have the following upper bound.

$$\begin{array}{cc}\hfill \int \mathrm{Var}\left[r_n(\theta ,{\theta }_0)\right]{\rho }_n\left(d\theta \right)& \le \sum _{i=1}^n\left(\frac{4}{n}+2n^{\delta /2}\frac{L}{n}\right)\left({\alpha }_{i-1}^{\delta /(2+\delta )}\right)\hfill \\ \hfill & +\sum _{i,j=1}^n\left(\frac{4}{n}+2n^{\delta /2}(\frac{L}{n}+\frac{L}{n}+\frac{L}{n})\right)\left({\alpha }_{|i-j|-1}^{\delta /(2+\delta )}\right)\hfill \\ \hfill & =\left(\frac{4}{n}+2n^{\delta /2}\frac{L}{n}\right)\left(\sum _{i=1}^n{\alpha }_{i-1}^{\delta /(2+\delta )}\right)\hfill \\ \hfill & +\left(\frac{4}{n}+6n^{\delta /2}\frac{L}{n}\right)\left(\sum _{i,j=1}^n{\alpha }_{|i-j|-1}^{\delta /(2+\delta )}\right).\hfill \end{array}$$

Since ${\sum }_{i,j=1}^n{\alpha }_{|i-j|-1}^{\delta /(2+\delta )}<n{\sum }_{k\ge 1}{\alpha }_{k-1}^{\delta /(2+\delta )}<\mathsf{\infty }$, we have that for some ${ϵ}_n^{\left(2\right)}=\mathrm{O}\left(\frac{n^{\delta /2}}{n}\right)$,

$$\begin{array}{c}\hfill \int \mathrm{Var}\left[r_n(\theta ,{\theta }_0)\right]{\rho }_n\left(d\theta \right)<n{ϵ}_n^{\left(2\right)}.\end{array}$$

Since $\mathcal{K}({\rho }_n,\pi )\le \sqrt{n}C$, following the concluding argument in Theorem 2 completes the proof. □

Appendix B.3.2. Proof of Proposition 8

We verify Assumption 1 and the proof follows from Theorem 3. For $i\in \{1,2,\cdots ,K-1\}$,

$$\begin{array}{c}\hfill p_{\theta }\left(j\right|i)=\left\{\begin{array}{cc}\theta \hfill & \mathrm{if}j=i-1,\hfill \\ 1-\theta \hfill & \mathrm{if}j=i+1.\hfill \end{array}\right.\end{array}$$

If $i=0$ or $i=K$, then the Markov chain goes back to 1 or $K-1$, respectively, with probability 1. With the convention $log\frac{0}{0}=0$, the log ratio of the transition probabilities becomes,

$$\begin{array}{c}\hfill |log{p}_{{\theta }_0}\left(X_1\right|X_0)-log{p}_{\theta }\left(X_1\right|X_0)|=I_{[X_1=X_0+1]}log\left(\frac{{\theta }_0}{\theta }\right)+I_{[X_1=X_0-1]}log\left(\frac{1-{\theta }_0}{1-\theta }\right).\end{array}$$

In this case, $m=2$. $M_1^{\left(1\right)}(X_1,X_0)=I_{[X_1=X_0+1]}$ and $M_2^{\left(1\right)}(X_1,X_0)=I_{[X_1=X_0-1]}$, both of which are bounded. Let $f_1^{\left(1\right)}(\theta ,{\theta }_0):=log\left(\frac{{\theta }_0}{\theta }\right)$ suppose $f_2^{\left(1\right)}(\theta ,{\theta }_0):=log\left(\frac{1-{\theta }_0}{1-\theta }\right)$.

The stationary distribution $q_{\theta }\left(i\right)=\frac{1}{K}\forall i\in 1,2,\cdots ,K$. Hence the log of the ratio of the invariant distribution becomes

$$\begin{array}{cc}\hfill log{q}_0\left(x\right)-log{q}_{\theta }\left(x\right)& =0,\hfill \end{array}$$ (A37)

and we can set $M_i^{\left(2\right)}(·):=1$ and $f_i^{\left(2\right)}(·,·):=0$ for $i\in \{1,2\}$. Thus, to prove the concentration bound for this Markov chain it is enough to assume that $\delta =1$ and show that $\int {\left[f_1^{\left(1\right)}(\theta ,{\theta }_0)\right]}^3{\rho }_n\left(d\theta \right)<\frac{C}{n}$ and $\int {\left[f_2^{\left(1\right)}(\theta ,{\theta }_0)\right]}^3{\rho }_n\left(d\theta \right)<\frac{C}{n}$ for some constant $C>0$.

As given, $\left\{{\rho }_n\right\}$ is a sequence of beta probability distribution functions, with parameters $a_n,b_n$ that satisfy the constraint $\frac{a_n}{a_n+b_n}={\theta }_0$. Specifically, we choose $a_n=n{\theta }_0$ and (therefore) $b_n=n(1-{\theta }_0)$. Thus, we get the following,

$$\begin{array}{cc}\hfill \int |f_1^{\left(1\right)}(\theta ,{\theta }_0){|}^3{\rho }_n\left(d\theta \right)& =\int {\left|log\left(\frac{{\theta }_0}{\theta }\right)\right|}^3{\rho }_n\left(d\theta \right)\hfill \\ \hfill & <\int {\left|\frac{{\theta }_0}{\theta }-1\right|}^3{\rho }_n\left(d\theta \right)\hfill \\ \hfill & =\frac{1}{\mathrm{Beta}(a_n,b_n)}{\int }_0^1{\left|\frac{{\theta }_0-\theta }{\theta }\right|}^3{\theta }^{a_n-1}{(1-\theta )}^{b_n-1}d\theta .\hfill \end{array}$$

Since ${\theta }_0,\theta \in (0,1)$, so is $\frac{|{\theta }_0-\theta |}{2}$, giving $|{\theta }_0{-\theta |}^3<2{({\theta }_0-\theta )}^2$. We use that fact to arrive at

$$\begin{array}{cc}\hfill \int |f_1^{\left(1\right)}(\theta ,{\theta }_0){|}^3{\rho }_n\left(d\theta \right)& \le \frac{2}{\mathrm{Beta}(a_n,b_n)}{\int }_0^1{({\theta }_0-\theta )}^2{\theta }^{a_n-4}{(1-\theta )}^{b_n-1}d\theta \hfill \\ \hfill & =\frac{2\mathrm{Beta}(a_n-3,b_n)}{\mathrm{Beta}(a_n,b_n)}\frac{(a_n-3)\left(b_n\right)}{{(a_n+b_n-3)}^2(a_n+b_n-2)}.\hfill \end{array}$$

From our choice of $a_n$ and $b_n$, $\frac{2\mathrm{Beta}(a_n-3,b_n)}{\mathrm{Beta}(a_n,b_n)}=O\left(1\right)$, and plugging the values of $a_n$ and $b_n$ into $\frac{(a_n-3)\left(b_n\right)}{{(a_n+b_n-3)}^2(a_n+b_n-2)}$, we get $\frac{(a_n-3)\left(b_n\right)}{{(a_n+b_n-3)}^2(a_n+b_n-2)}=\frac{1}{n}\frac{({\theta }_0-\frac{3}{n})(1-{\theta }_0)}{{(1-\frac{3}{n})}^2(1-\frac{2}{n})}$, which is upper bounded by $\frac{C_1}{n}$ for some constant $C_1>0$. Hence,

$$\begin{array}{cc}\hfill \int |f_1^{\left(1\right)}(\theta ,{\theta }_0){|}^3{\rho }_n\left(d\theta \right)& <\frac{C_1}{n}.\hfill \end{array}$$

Similarly, we can also show that,

$$\begin{array}{cc}\hfill \int |f_2^{\left(1\right)}(\theta ,{\theta }_0){|}^3{\rho }_n\left(d\theta \right)& <\frac{C_2}{n}.\hfill \end{array}$$

Finally, from Proposition A.2.1, we get that $\mathcal{K}({\rho }_n,\pi )<C+\frac{1}{2}log\left(n\right)$ for some large constant C. Hence, $\mathcal{K}({\rho }_n,\pi )<C_3\sqrt{n}$ for some constant $C_3>0$. Choosing $C=max(C_1,C_2,C_3)$, we satisfy all the conditions of Assumption 1 and Theorem 3. □

Appendix B.3.3. Proof of Proposition 9

For the purpose of this proof, we choose ${\rho }_n$’s with scaled Beta distribution with parameters $a_n=n({\theta }_0/2)$ and $b_n=n(1-{\theta }_0/2)$. Since, ${\rho }_n$ is a scaled Beta distribution with the scaling factors $m=0.5$ and $c=0$, the pdf of ${\rho }_n$ is given by

$$\begin{array}{cc}\hfill {\rho }_n\left(\theta \right)& =\frac{2}{\mathrm{Beta}(a_n,b_n)}{\left(2\theta \right)}^{a_n}{\left(1-2\theta \right)}^{b_n}\hfill \end{array}$$

Since this is a scaled distribution, $E_{{\rho }_n}\left[\theta \right]=2\frac{a_n}{a_n+b_n}={\theta }_0$ and there exists a constant $\sigma >0$, ${\mathrm{Var}}_{{\rho }_n}\left[\theta \right]=\frac{{\sigma }^2}{n}$. Now, we analyse the transition probabilities. For $i\in \{1,2,\cdots \}$, the Birth-Death process has transition probabilities

$$\begin{array}{c}\hfill p_{\theta }\left(j\right|i)=\left\{\begin{array}{cc}\theta \hfill & \mathrm{if}j=i-1,\hfill \\ 1-\theta \hfill & \mathrm{if}j=i+1.\hfill \end{array}\right.\end{array}$$

If $i=0$, then the Markov chain goes to 1 with probability 1. Hence with the convention $log\frac{0}{0}=0$ the ratio of the log of the transition probabilities becomes,

$$\begin{array}{c}\hfill |log{p}_{{\theta }_0}\left(X_1\right|X_0)-log{p}_{\theta }\left(X_1\right|X_0)|=I_{[X_1=X_0+1]}log\left[\frac{{\theta }_0}{\theta }\right]+I_{[X_1=X_0-1]}log\left[\frac{1-{\theta }_0}{1-\theta }\right].\end{array}$$

In this case, $m=3$. $M_1^{\left(1\right)}(X_1,X_0)=I_{[X_1=X_0+1]}$ and $M_2^{\left(1\right)}(X_1,X_0)=I_{[X_1=X_0-1]}$. Define $M_3^{\left(1\right)}(X_1,X_0):=1$. All these random variables are bounded. Define $f_1^{\left(1\right)}(\theta ,{\theta }_0):=log\left[\frac{{\theta }_0}{\theta }\right],f_2^{\left(1\right)}(\theta ,{\theta }_0):=log\left[\frac{1-{\theta }_0}{1-\theta }\right]$ and $f_3^{\left(1\right)}(\theta ,{\theta }_0):=0$. Similarly as in the proof on Proposition 8,

$$\begin{array}{cc}\hfill \int {\left[f_1^{\left(1\right)}(\theta ,{\theta }_0)\right]}^3{\rho }_n\left(d\theta \right)& <\frac{C_1}{n},\mathrm{and}\hfill \\ \hfill \int {\left[f_2^{\left(1\right)}(\theta ,{\theta }_0)\right]}^3{\rho }_n\left(d\theta \right)& <\frac{C_2}{n}.\hfill \end{array}$$

The stationary distribution is given by $q_{\theta }\left(i\right)={\left(\frac{\theta }{1-\theta }\right)}^{i-1}{q}_{\theta }\left(1\right)\forall i\in 1,2,\cdots$, so that $q_{\theta }\left(i\right)=(1-\theta ){\left(\frac{\theta }{1-\theta }\right)}^{i-1}$ Hence the log of the ratio of the invariant distribution becomes

$$\begin{array}{cc}\hfill log{q}_0\left(i\right)-log{q}_{\theta }\left(i\right)& =log\left[\frac{1-{\theta }_0}{1-\theta }\right]+(i-1)log\left[\frac{{\theta }_0}{\theta }\right]-(i-1)log\left[\frac{1-{\theta }_0}{1-\theta }\right]\hfill \end{array}$$ (A38)

We define $M_1^{\left(2\right)}\left(X_0\right):=1$, and $M_2^{\left(2\right)}\left(X_0\right)=M_3^{\left(2\right)}\left(X_0\right):=X_0-1$. We can write ${\mathrm{E}}_{q^{\left(0\right)}}{\left[M_2^{\left(2\right)}\left(X_0\right)\right]}^2={\sum }_{i=1}^{\mathsf{\infty }}{(i-1)}^2{q}^{\left(0\right)}\left(i\right)<{\sum }_{i=1}^{\mathsf{\infty }}{i}^2{q}^{\left(0\right)}\left(i\right)$. We have chosen $q^{\left(0\right)}$ such that ${\sum }_{i=1}^{\mathsf{\infty }}{i}^2{q}^{\left(0\right)}\left(i\right)$ is bounded. Hence, ${\mathrm{E}}_{q^{\left(0\right)}}{\left[M_2^{\left(2\right)}\left(X_0\right)\right]}^2<\mathsf{\infty }$. To verify Assumption i define, $f_1^{\left(2\right)}(\theta ,{\theta }_0)=-f_3^{\left(2\right)}(\theta ,{\theta }_0):=log\left[\frac{1-{\theta }_0}{1-\theta }\right]$, and define $f_2^{\left(2\right)}(\theta ,{\theta }_0):=log\left[\frac{{\theta }_0}{\theta }\right]$. Therefore following the proof of Proposition 8,

$$\begin{array}{cc}\hfill \int |f_1^{\left(2\right)}(\theta ,{\theta }_0){|}^3{\rho }_n\left(d\theta \right)=\int {\left|f_3^{\left(2\right)}(\theta ,{\theta }_0)\right|}^3{\rho }_n\left(d\theta \right)=& \int |f_2^{\left(1\right)}(\theta ,{\theta }_0){|}^3{\rho }_n\left(d\theta \right)<\frac{C_2}{n},\mathrm{and},\hfill \\ \hfill \int |f_2^{\left(2\right)}(\theta ,{\theta }_0){|}^3{\rho }_n\left(d\theta \right)=& \int |f_1^{\left(1\right)}(\theta ,{\theta }_0){|}^3{\rho }_n\left(d\theta \right)<\frac{C_1}{n}.\hfill \end{array}$$

Finally, we take the KL-divergence $\mathcal{K}({\rho }_n,\pi )$. ${\rho }_n$ follows a scaled Beta distribution on $(0,1/2)$ with parameters $a_n=n({\theta }_0/2)$ and $b_n=n(1-{\theta }_0/2)$, while $\pi$ follows a scaled Beta distribution on $(0,1/2)$ with parameters a and b. Thus,

$$\begin{array}{cc}\hfill \mathcal{K}({\rho }_n,\pi )& ={\int }_0^{\frac{1}{2}}log\left(\frac{{\rho }_n\left(\theta \right)}{\pi \left(\theta \right)}\right){\rho }_n\left(d\theta \right),\hfill \end{array}$$

which, by substituting $t=2\theta$, we get,

$$\begin{array}{cc}\hfill \mathcal{K}({\rho }_n,\pi )& =2{\int }_0^{1}log\left(\frac{{\rho }_n\left(t\right)}{\pi \left(t\right)}\right){\rho }_n\left(dt\right).\hfill \end{array}$$

${\int }_0^{1}log\left(\frac{{\rho }_n\left(t\right)}{\pi \left(t\right)}\right){\rho }_n\left(dt\right)$ is the KL-divergence between a Beta distribution with parameters $a_n$ and $b_n$ and a Beta distribution with parameters a and b. An application of Proposition A.2.1 gives us for a constant $C_1>0$,

$$\begin{array}{c}\hfill {\int }_0^{1}log\left(\frac{{\rho }_n\left(t\right)}{\pi \left(t\right)}\right){\rho }_n\left(dt\right)<C_1+\frac{1}{2}log\left(n\right).\end{array}$$

Hence we can say, $\mathcal{K}({\rho }_n,\pi )<2\left[C_1+\frac{1}{2}log\left(n\right)\right]$. Thus, we now get that for some constant $C_3>0$,

$$\begin{array}{c}\hfill \mathcal{K}({\rho }_n,\pi )<C_3\sqrt{n}.\end{array}$$

Choosing $C=max(C_1,C_2,C_3)$ we satisfy all of the conditions of Assumption 1 and thus by Theorem 3, we are complete the proof. □

Appendix B.3.4. Proof of Theorem 4

Part 1: Verifying condition (i) of Corollary 1 As in the proof of Theorem 2 substitute the true parameter ${\theta }_0$ for ${\theta }_1$ and $\theta$ for ${\theta }_2$. We also set our initial distributions $q_1^{\left(0\right)}$ and $q_2^{\left(0\right)}$ to the known initial distribution $q^{\left(0\right)}$. A method similar to Equation (A35), yields

$$\begin{array}{cc}\hfill \mathcal{K}(P_{{\theta }_0}^{\left(n\right)},P_{\theta }^{\left(n\right)})& \le \sum _{i=1}^n\sum _{k=1}^m\mathrm{E}\left[M_k^{\left(1\right)}(X_i,X_{i-1})\right]\left|f_k^{\left(1\right)}(\theta ,{\theta }_0)\right|.\hfill \end{array}$$

Because $M_k^{\left(1\right)}$s satisfy Assumption 2, it follows by the application of Theorem 2.3, [21] that $\exists \lambda >0$ such that for any $0<\kappa \le \lambda$, and for some $\zeta \in (0,1)$ possibly depending upon $\lambda$,

$$\begin{array}{c}\hfill \mathrm{E}\left[e^{\kappa M_k^{\left(1\right)}(X_i,X_{i-1})}\right|X_1,X_0]\le {\zeta }^{i-1}{e}^{\kappa M_k^{\left(1\right)}(X_1,X_0)}+\frac{1-{\zeta }^i}{1-\zeta }\mathcal{D}{e}^{\kappa a}\mathrm{for}\mathrm{all}i>1.\end{array}$$

We rewrite $\mathrm{E}\left[M_k^{\left(1\right)}(X_i,X_{i-1})|X_1,X_0\right]$ as follows:

$$\begin{array}{cc}\hfill \mathrm{E}\left[M_k^{\left(1\right)}(X_i,X_{i-1})|X_1,X_0\right]& =\frac{\mathrm{E}\left[\kappa M_k^{\left(1\right)}(X_i,X_{i-1})\right|X_1,X_0]}{\kappa }\hfill \\ \hfill & \le \frac{\mathrm{E}\left[e^{\kappa M_k^{\left(1\right)}(X_i,X_{i-1})}\right|X_1,X_0]}{\kappa }.\hfill \end{array}$$

Therefore, ${\sum }_{i=1}^n\mathrm{E}\left[M_k^{\left(1\right)}(X_i,X_{i-1})\right]$ can be upper bounded as,

$$\begin{array}{cc}\hfill \sum _{i=1}^n\mathrm{E}\left[M_k^{\left(1\right)}(X_i,X_{i-1})\right]& =\sum _{i=1}^n\mathrm{E}\left[\kappa M_k^{\left(1\right)}(X_i,X_{i-1})|X_1,X_0\right]{\kappa }^{-1}\hfill \\ \hfill & \le \sum _{i=1}^n\left[{\zeta }^{i-1}\mathrm{E}{e}^{\kappa M_k^{\left(1\right)}(X_1,X_0)}+\frac{1-{\zeta }^i}{1-\zeta }\mathcal{D}{e}^{\kappa a}\right]{\kappa }^{-1}.\hfill \end{array}$$

Since, $\zeta \in (0,1)$, ${\zeta }^i<1$. Hence, we can write that,

$$\begin{array}{cc}\hfill \sum _{i=1}^n\left[{\zeta }^{i-1}\mathrm{E}{e}^{\kappa M_k^{\left(1\right)}(X_1,X_0)}+\frac{1-{\zeta }^i}{1-\zeta }\mathcal{D}{e}^{\kappa a}\right]{\kappa }^{-1}& \le \sum _{i=1}^n\left[{\zeta }^{i-1}\mathrm{E}{e}^{\kappa M_k^{\left(1\right)}(X_1,X_0)}+\frac{1}{1-\zeta }\mathcal{D}{e}^{\kappa a}\right]{\kappa }^{-1}\hfill \\ \hfill & =\left[\frac{1-{\zeta }^n}{1-\zeta }\mathrm{E}{e}^{\kappa M_k^{\left(1\right)}(X_1,X_0)}+\frac{n}{1-\zeta }\mathcal{D}{e}^{\kappa a}\right]{\kappa }^{-1}\hfill \\ \hfill & \le nL,\hfill \end{array}$$

for a large constant L. Therefore $\int \mathcal{K}(P_{{\theta }_0}^{\left(n\right)},P_{\theta }^{\left(n\right)}){\rho }_n\left(d\theta \right)$ can be upper bounded as follows,

$$\begin{array}{cc}\hfill \int \mathcal{K}(P_{{\theta }_0}^{\left(n\right)},P_{\theta }^{\left(n\right)}){\rho }_n\left(d\theta \right)& \le \int \sum _{k=1}^{m}nL\left|f_k^{\left(1\right)}(\theta ,{\theta }_0)\right|{\rho }_n\left(d\theta \right)\hfill \\ \hfill & =\sum _{k=1}^{m}nL\int \left|f_k^{\left(1\right)}(\theta ,{\theta }_0)\right|{\rho }_n\left(d\theta \right).\hfill \end{array}$$

By Assumption 1, $\int |f_k^{\left(1\right)}(\theta ,{\theta }_0)|{\rho }_n\left(d\theta \right)<\frac{C}{n}$, hence,

$$\begin{array}{cc}\hfill \int \mathcal{K}(P_{{\theta }_0}^{\left(n\right)},P_{\theta }^{\left(n\right)}){\rho }_n\left(d\theta \right)& \le nL\frac{C}{\sqrt{n}}.\hfill \end{array}$$

Hence, for some ${ϵ}_n^{\left(1\right)}=\mathrm{O}\left(\frac{1}{\sqrt{n}}\right)$, we have obtained that, $\int \mathcal{K}(P_{{\theta }_0}^{\left(n\right)},P_{\theta }^{\left(n\right)}){\rho }_n\left(d\theta \right)\le n{ϵ}_n^{\left(1\right)}$.

Part 2: Verifying condition (ii) of Corollary 1: Similar to as in the proof of Theorem 3, we upper bound $\int \mathrm{Var}\left[r_n(\theta ,{\theta }_0)\right]{\rho }_n\left(d\theta \right)$ by

$$\begin{array}{cc}\hfill \int \mathrm{Var}\left[r_n(\theta ,{\theta }_0)\right]{\rho }_n\left(d\theta \right)& \le \sum _{i,j=1}^n(\frac{4}{n}+2n^{\delta /2}(\int C_{{\theta }_0,\theta }^{\left(i\right)}{\rho }_n\left(d\theta \right)+\int C_{{\theta }_0,\theta }^{\left(j\right)}{\rho }_n\left(d\theta \right)\hfill \end{array}$$ (A39)

$$\begin{array}{cc}\hfill & +\int \sqrt{C_{{\theta }_0,\theta }^{\left(i\right)}{C}_{{\theta }_0,\theta }^{\left(j\right)}}{\rho }_n\left(d\theta \right)\left)\right)\left({\alpha }_{|i-j|-1}^{\delta /(2+\delta )}\right)\hfill \\ \hfill & +\sum _{i=1}^n\left(\frac{4}{n}+2n^{\delta /2}\int C_{{\theta }_0,\theta }^{\left(i\right)}{\rho }_n\left(d\theta \right)\right)\left({\alpha }_{i-1}^{\delta /(2+\delta )}\right),\hfill \end{array}$$ (A40)

where $C_{{\theta }_0,\theta }$ is upper bounded as

$$\begin{array}{cc}\hfill C_{{\theta }_0,\theta }^{\left(i\right)}& \le \sum _{k=1}^m{m}^{1+\delta }\mathrm{E}{\left[M_k^{\left(1\right)}(X_i,X_{i-1})\right]}^{2+\delta }{\left|f_k^{\left(1\right)}(\theta ,{\theta }_0)\right|}^{2+\delta }.\hfill \end{array}$$

There exists a constant $C_{\delta }$ depending upon $\delta$ such that,

$$\begin{array}{cc}\hfill {\left[M_k^{\left(1\right)}\right]}^{2+\delta }(X_i,X_{i-1})& =\frac{{\kappa }^{2+\delta }{\left[M_k^{\left(1\right)}\right]}^{2+\delta }{(X_i,X_{i-1})}^{2+\delta }}{{\kappa }^{2+\delta }}\hfill \\ \hfill & \le \frac{e^{\kappa M_k^{\left(1\right)}(X_i,X_{i-1})}+C_{\delta }}{{\kappa }^{2+\delta }}.\hfill \end{array}$$

By expressing $\mathrm{E}\left[M_k^{\left(1\right)}{(X_i,X_{i-1})}^{2+\delta }\right]=\mathrm{E}\left[\mathrm{E}\left[M_k^{\left(1\right)}{(X_i,X_{i-1})}^{2+\delta }|X_1,X_0\right]\right]$ and following a method similar to the previous part, we get,

$$\begin{array}{cc}\hfill \mathrm{E}\left[M_k^{\left(1\right)}{(X_i,X_{i-1})}^{2+\delta }\right]& \le \frac{\left[{\zeta }^i\mathrm{E}{e}^{\kappa M_k^{\left(1\right)}(X_1,X_0)}+\frac{1-{\zeta }^i}{1-\zeta }\mathcal{D}{e}^{\kappa a}\right]+C_{\delta }}{{\kappa }^{2+\delta }}.\hfill \end{array}$$

The fact that $0<\zeta <1$ implies that $0<{\zeta }^i<\zeta$. This gives us the following,

$$\begin{array}{cc}\hfill \mathrm{E}\left[M_k^{\left(1\right)}{(X_i,X_{i-1})}^{2+\delta }\right]& \le \frac{\left[\zeta \mathrm{E}{e}^{\kappa M_k^{\left(1\right)}(X_1,X_0)}+\frac{1}{1-\zeta }\mathcal{D}{e}^{\kappa a}\right]+C_{\delta }}{{\kappa }^{2+\delta }}.\hfill \end{array}$$

Since $\kappa <\lambda$, by the application of Jensen’s inequality, we get

$$\begin{array}{cc}\hfill \mathrm{E}\left[M_k^{\left(1\right)}{(X_i,X_{i-1})}^{2+\delta }\right]& \le \frac{\left[\zeta \mathrm{E}{e}^{\lambda M_k^{\left(1\right)}(X_1,X_0)}+\frac{1}{1-\zeta }\mathcal{D}{e}^{\kappa a}\right]+C_{\delta }}{{\kappa }^{2+\delta }}\hfill \\ \hfill & =\frac{\left[\zeta \int e^{\lambda M_k^{\left(1\right)}(x_1,x_0)}{p}_{{\theta }_0}\left(x_1\right|x_0)D\left(x_0\right)d{x}_{1}d{x}_0+\frac{1}{1-\zeta }\mathcal{D}{e}^{\kappa a}\right]+C_{\delta }}{{\kappa }^{2+\delta }}.\hfill \end{array}$$

We know that $\int |f_k^{\left(1\right)}(\theta ,{\theta }_0){|}^{2+\delta }{\rho }_n\left(d\theta \right)<\frac{C}{n}$. Thus, following Assumption 1 we can say that, for a large constant L, $\int C_{{\theta }_0,\theta }^{\left(i\right)}{\rho }_n\left(d\theta \right)\le \frac{L}{n}$. The rest of the proof follows similarly as in the proof of Theorem 3, and we obtain an ${ϵ}_n^{\left(2\right)}=\mathrm{O}\left(\frac{n^{\delta /2}}{n}\right)$, such that,

$$\begin{array}{c}\hfill \int \mathrm{Var}\left[r_n(\theta ,{\theta }_0)\right]{\rho }_n\left(d\theta \right)<n{ϵ}_n^{\left(2\right)}.\end{array}$$

Since, $\mathcal{K}({\rho }_n,\pi )\le \sqrt{n}C$, similar arguments as in the proof of Theorem 2 holds. The theorem is thus proved.

Appendix B.3.5. Proof of Theorem 5

Part 1: Verifying condition (i) of Corollary 1 As in the proof of Theorem 2 substitute the true parameter ${\theta }_0$ for ${\theta }_1$ and $\theta$ for ${\theta }_2$. We also set $q_1^{\left(0\right)}$ and $q_2^{\left(0\right)}$ to the known initial distribution $q^{\left(0\right)}$. Similar to the steps leading to Equation (A35), we get

$$\begin{array}{cc}\hfill \mathcal{K}(P_{{\theta }_0}^{\left(n\right)},P_{\theta }^{\left(n\right)})& \le \sum _{i=1}^n\sum _{k=1}^m\mathrm{E}\left[M_k^{\left(1\right)}(X_i,X_{i-1})\right]\left|f_k^{\left(1\right)}(\theta ,{\theta }_0)\right|.\hfill \end{array}$$

Consider the term $\mathrm{E}\left[M_k^{\left(1\right)}(X_i,X_{i-1})\right]$. With $q_{{\theta }_0}^{(i-1)}$ the marginal distribution of $X_{i-1}$, we have

$$\begin{array}{cc}\hfill \mathrm{E}\left[M_k^{\left(1\right)}(X_i,X_{i-1})\right]& =\int M_k^{\left(1\right)}(x_i,x_{i-1})p_{{\theta }_0}\left(x_i\right|x_{i-1})q_{{\theta }_0}^{(i-1)}\left(x_{i-1}\right)d{x}_{i}d{x}_{i-1}.\hfill \\ \hfill \mathrm{E}\left[M_k^{\left(1\right)}(X_i,X_{i-1})\right]& =\int M_k^{\left(1\right)}(x_i,x_{i-1})p_{{\theta }_0}\left(x_i\right|x_{i-1})p_{{\theta }_0}^{i-1}\left(x_{i-1}\right|x_0)q_{{\theta }_0}^{\left(0\right)}\left(x_0\right)d{x}_{0}d{x}_{i}d{x}_{i-1}\hfill \end{array}$$

Recall that the marginal density satisfies $q_{{\theta }_0}^{(i-1)}\left(x_{i-1}\right)=\int p_{{\theta }_0}^{i-1}\left(x_{i-1}\right|x_0)q_{{\theta }_0}^{\left(0\right)}\left(x_0\right)d\left(x_0\right)$, where $p_{{\theta }_0}^i(·|x_0)$ is the i-step transition probability. Then

$$\begin{array}{cc}\hfill \mathrm{E}\left[M_k^{\left(1\right)}(X_i,X_{i-1})\right]& =\int \mathrm{E}\left[M_k^{\left(1\right)}(X_i,x_{i-1})|x_{i-1}\right]p_{{\theta }_0}^{i-1}\left(x_{i-1}\right|x_0)q_{{\theta }_0}^{\left(0\right)}\left(x_0\right)d{x}_{0}d{x}_{i-1}.\hfill \end{array}$$

Since the Markov chain $\left\{X_n\right\}$ satisfies Assumption A.1.1, we know by the application of Theorem A.1.1 that $\left\{X_n\right\}$ is V-geometrically ergodic. Hence, $\exists \tau <1$, $R<\mathsf{\infty }$ such that $\forall \left|f\right|<V$

$$\begin{array}{cc}\hfill |\int f\left(x_{i-1}\right)p_{{\theta }_0}^{i-1}\left(x_{i-1}\right|x_0)d{x}_{i-1}& -\int f\left(x_{i-1}\right)q_{{\theta }_0}\left(x_{i-1}\right)d{x}_{i-1}|<RV\left(x_0\right){\tau }^{i-1},\hfill \end{array}$$

where $q_{{\theta }_0}$ is the stationary distribution, implying that

$$\begin{array}{cc}\hfill \int f\left(x_{i-1}\right)p_{{\theta }_0}^{i-1}\left(x_{i-1}\right|x_0)d{x}_{i-1}& <\int f\left(x_{i-1}\right)q_{{\theta }_0}\left(x_{i-1}\right)d{x}_{i-1}+RV\left(x_0\right){\tau }^{i-1}.\hfill \end{array}$$

By the application of Jensen’s inequality we get ${\left(\mathrm{E}\left[M_k^{\left(1\right)}(X_i,X_{i-1})|X_{i-1}\right]\right)}^{2+\delta }\le \mathrm{E}\left[M_k^{\left(1\right)}{(X_i,X_{i-1})}^{2+\delta }|X_{i-1}\right]<V\left(X_{i-1}\right)$. Since $V(·)\ge 1$, it follows from the previous expression that $\mathrm{E}\left[M_k^{\left(1\right)}(X_i,X_{i-1})|X_{i-1}\right]<V{\left(X_{i-1}\right)}^{1/(2+\delta )}\le V\left(X_{i-1}\right)$. Thus, setting $f\left(x\right)=\mathrm{E}\left[M_k^{\left(1\right)}(X_i,X_{i-1})|X_{i-1}=x\right]$, we obtain

$$\begin{array}{cc}\hfill \mathrm{E}\left[M_k^{\left(1\right)}(X_i,X_{i-1})\right]& <\int \left[\mathrm{E}\left[M_k^{\left(1\right)}(X_i,X_{i-1})|X_{i-1}\right]q_{{\theta }_0}\left(x_i\right)d{x}_{i-1}+RV\left(x_0\right){\tau }^{i-1}\right]q^{\left(0\right)}\left(x_0\right)d{x}_0\hfill \\ \hfill & =\mathrm{E}\left[M_k^{\left(1\right)}(X_1,X_0)\right]+{\tau }^{i-1}\int RV\left(x_0\right)q^{\left(0\right)}\left(x_0\right)d{x}_0.\hfill \end{array}$$

Summing from $i=1$ to n, we get

$$\begin{array}{cc}\hfill \sum _{i=1}^n\mathrm{E}\left[M_k^{\left(1\right)}(x_i,x_{i-1})\right]& <n\mathrm{E}\left[M_k^{\left(1\right)}(X_1,X_0)\right]+\sum _{i=1}^n{\tau }^{i-1}\int RV\left(x_0\right)q^{\left(0\right)}\left(x_0\right)d{x}_0\hfill \end{array}$$

$$\begin{array}{cc}\hfill & =n\mathrm{E}\left[M_k^{\left(1\right)}(X_1,X_0)\right]+\frac{1-{\tau }^n}{1-\tau }\int RV\left(x_0\right)q^{\left(0\right)}\left(x_0\right)d{x}_0.\hfill \end{array}$$

This gives us the following bound on $\int \mathcal{K}(P_{{\theta }_0}^{\left(n\right)},P_{\theta }^{\left(n\right)}){\rho }_n\left(d\theta \right)$:

$$\begin{array}{cc}\hfill \int \mathcal{K}(P_{{\theta }_0}^{\left(n\right)},P_{\theta }^{\left(n\right)}){\rho }_n\left(d\theta \right)& \le \sum _{k=1}^m\left[n\mathrm{E}\left[M_k^{\left(1\right)}(X_1,X_0)\right]+\frac{1-{\tau }^n}{1-\tau }\int RV\left(x_0\right)D\left(x_0\right)d{x}_0\right]\hfill \\ \hfill & \times \int |f_k^{\left(1\right)}(\theta ,{\theta }_0)|{\rho }_n\left(d\theta \right).\hfill \end{array}$$

By Assumption 1, $\int |f_k^{\left(1\right)}(\theta ,{\theta }_0)|{\rho }_n\left(d\theta \right)<\frac{C}{\sqrt{n}}$. Hence, we can rewrite the previous expression as

$$\begin{array}{cc}\hfill \int \mathcal{K}(P_{{\theta }_0}^{\left(n\right)},P_{\theta }^{\left(n\right)}){\rho }_n\left(d\theta \right)& \le \sum _{k=1}^m\left[n\mathrm{E}\left[M_k^{\left(1\right)}(X_1,X_0)\right]+\frac{1-{\tau }^n}{1-\tau }\int RV\left(x_1\right)D\left(x_1\right)d{x}_1\right]\frac{C}{\sqrt{n}}\hfill \\ \hfill & =nm\left[\mathrm{E}\left[M_k^{\left(1\right)}(X_1,X_0)\right]+\frac{1-{\tau }^n}{n(1-\tau )}\int RV\left(x_0\right)D\left(x_0\right)d{x}_0\right]\frac{C}{\sqrt{n}}.\hfill \end{array}$$

Since, $\tau <1$, $0<1-{\tau }^n<1$, and we rewrite the previous equation as,

$$\begin{array}{cc}\hfill \int \mathcal{K}(P_{{\theta }_0}^{\left(n\right)},P_{\theta }^{\left(n\right)}){\rho }_n\left(d\theta \right)& \le nm\left[\mathrm{E}\left[M_k^{\left(1\right)}(X_1,X_0)\right]+\frac{1}{n(1-\tau )}\int RV\left(x_0\right)D\left(x_0\right)d{x}_0\right]\frac{C}{\sqrt{n}}.\hfill \end{array}$$

Hence, there exists an ${ϵ}_n^{\left(1\right)}=\mathrm{O}\left(\frac{1}{\sqrt{n}}\right)$ such that $\int \mathcal{K}(P_{{\theta }_0}^{\left(n\right)},P_{\theta }^{\left(n\right)}){\rho }_n\left(d\theta \right)\le n{ϵ}_n^{\left(1\right)}$.

Part 2: Verifying condition (ii) of Corollary 1: Similar to as in the proof of Theorem 3, we upper bound $\int \mathrm{Var}\left[r_n(\theta ,{\theta }_0)\right]{\rho }_n\left(d\theta \right)$ by

$$\begin{array}{cc}\hfill \int \mathrm{Var}\left[r_n(\theta ,{\theta }_0)\right]{\rho }_n\left(d\theta \right)& \le \sum _{i,j=1}^n(\frac{4}{n}+2n^{\delta /2}(\int C_{{\theta }_0,\theta }^{\left(i\right)}{\rho }_n\left(d\theta \right)+\int C_{{\theta }_0,\theta }^{\left(j\right)}{\rho }_n\left(d\theta \right)\hfill \end{array}$$ (A41)

$$\begin{array}{cc}\hfill & +\int \sqrt{C_{{\theta }_0,\theta }^{\left(i\right)}{C}_{{\theta }_0,\theta }^{\left(j\right)}}{\rho }_n\left(d\theta \right)\left)\right)\left({\alpha }_{|i-j|-1}^{\delta /(2+\delta )}\right)\hfill \\ \hfill & +\sum _{i=1}^n\left(\frac{4}{n}+2n^{\delta /2}\int C_{{\theta }_0,\theta }^{\left(i\right)}{\rho }_n\left(d\theta \right)\right)\left({\alpha }_{i-1}^{\delta /(2+\delta )}\right),\hfill \end{array}$$ (A42)

where $C_{{\theta }_0,\theta }$ is upper bounded as

$$\begin{array}{cc}\hfill C_{{\theta }_0,\theta }^{\left(i\right)}& \le \sum _{k=1}^m{m}^{1+\delta }\mathrm{E}{\left[M_k^{\left(1\right)}(X_i,X_{i-1})\right]}^{2+\delta }{\left|f_k^{\left(1\right)}(\theta ,{\theta }_0)\right|}^{2+\delta }.\hfill \end{array}$$

Since $\mathrm{E}\left[M_k^{\left(1\right)}{(X_i,X_{i-1})}^{2+\delta }|X_{i-1}\right]<V\left(X_{i-1}\right)$, by a similar application of V-geometric ergodicity, we can say that, $\exists 0<\tau <1$, such that

$$\begin{array}{cc}\hfill \mathrm{E}{\left[M_k^{\left(1\right)}(X_i,X_{i-1})\right]}^{2+\delta }& \le n\mathrm{E}{\left[M_k^{\left(1\right)}(X_1,X_0)\right]}^{2+\delta }+{\tau }^{i-1}\int RV\left(x_0\right)D\left(x_0\right)d{x}_0,\hfill \end{array}$$

which, by the fact that ${\tau }^{i-1}<\tau$, gives us,

$$\begin{array}{cc}\hfill \mathrm{E}{\left[M_k^{\left(1\right)}(X_i,X_{i-1})\right]}^{2+\delta }& \le \mathrm{E}{\left[M_k^{\left(1\right)}(X_1,X_0)\right]}^{2+\delta }+\tau \int RV\left(x_0\right)D\left(x_0\right)d{x}_0.\hfill \end{array}$$

By Assumption 1, we know that, $\int |f_k^{\left(1\right)}(\theta ,{\theta }_0){|}^{2+\delta }{\rho }_n\left(d\theta \right)<\frac{C}{n}$. Hence, for a large constant L, $\int C_{{\theta }_0,\theta }^{\left(i\right)}{\rho }_n\left(d\theta \right)\le \frac{L}{n}$. We also see that since the chain is geometrically ergodic, by the application of Equation (A4), ${\sum }_{k\ge 1}{\alpha }_k^{\delta /(2+\delta )}<+\mathsf{\infty }$. The rest of the proof follows similarly as in the proof of Theorem 3, and we obtain an ${ϵ}_n^{\left(2\right)}=\mathrm{O}\left(\frac{n^{\delta /2}}{n}\right)$, such that,

$$\begin{array}{c}\hfill \int \mathrm{Var}\left[r_n(\theta ,{\theta }_0)\right]{\rho }_n\left(d\theta \right)<n{ϵ}_n^{\left(2\right)}.\end{array}$$

Since, $\mathcal{K}({\rho }_n,\pi )\le \sqrt{n}C$, similar arguments as in the proof of Theorem 2 holds. The theorem is thus proved. □

Appendix B.3.6. Proof of Proposition 10

For the purpose of the proof, we choose ${\rho }_n$’s with scaled Beta distribution with parameters $a_n=n\frac{1+{\theta }_0}{2}$ and $b_n=n\frac{1-{\theta }_0}{2}$. Since, ${\rho }_n$ is a scaled Beta distribution with the scaling factors $m=2$ and $c=-1$, the pdf of ${\rho }_n$ is given by

$$\begin{array}{cc}\hfill {\rho }_n\left(\theta \right)& =\frac{1}{2\mathrm{Beta}(a_n,b_n)}{\left(\frac{1+\theta }{2}\right)}^{a_n}{\left(\frac{1-\theta }{2}\right)}^{b_n}\hfill \end{array}$$

Since this is a scaled distribution, $E_{{\rho }_n}\left[\theta \right]=2\frac{a_n}{a_n+b_n}-1={\theta }_0$ and there exists a constant $\sigma >0$, ${\mathrm{Var}}_{{\rho }_n}\left[\theta \right]=\frac{{\sigma }^2}{n}$. We now analyse the log-ratio of the transition probabilities for the Markov chain,

$$\begin{array}{c}\hfill log{p}_{{\theta }_0}\left(X_n\right|X_{n-1})-log{p}_{\theta }\left(X_n\right|X_{n-1})=2X_n{X}_{n-1}(\theta -{\theta }_0)+X_{n-1}^2({\theta }_0^2-{\theta }^2).\end{array}$$

Observe that in this setting, $M_1^{\left(1\right)}(X_n,X_{n-1})=\left|X_n{X}_{n-1}\right|$ and $M_2^{\left(1\right)}(X_n,X_{n-1})=X_n^2$. Next, using the fact that

$$\begin{array}{cc}\hfill \mathrm{E}\left[\right|X_n{|}^{2+\delta }\left|X_{n-1}\right]& =\mathrm{E}\left[\right|X_n-{\theta }_0{X}_{n-1}+{\theta }_0{X}_{n-1}{|}^{2+\delta }\left|X_{n-1}\right],\hfill \end{array}$$

and by an application of triangle inequality, we obtain

$$\begin{array}{cc}\hfill \mathrm{E}\left[\right|X_n{|}^{2+\delta }\left|X_{n-1}\right]& \le \mathrm{E}\left[{\left(|X_n-{\theta }_0{X}_{n-1}|+|{\theta }_0{X}_{n-1}|\right)}^{2+\delta }|X_{n-1}\right]\hfill \\ \hfill & =\mathrm{E}\left[{\left(2\frac{|X_n-{\theta }_0{X}_{n-1}|+|{\theta }_0{X}_{n-1}|}{2}\right)}^{2+\delta }|X_{n-1}\right]\hfill \\ \hfill & =\mathrm{E}\left[2^{2+\delta }{\left(\frac{|X_n-{\theta }_0{X}_{n-1}|+|{\theta }_0{X}_{n-1}|}{2}\right)}^{2+\delta }|X_{n-1}\right].\hfill \end{array}$$

Now by using Jensen’s inequality we get,

$$\begin{array}{cc}\hfill \mathrm{E}\left[\right|X_n{|}^{2+\delta }\left|X_{n-1}\right]& \le \mathrm{E}\left[2^{2+\delta }\left(\frac{|X_n-{\theta }_0{X}_{n-1}{|}^{2+\delta }+{\left|{\theta }_0{X}_{n-1}\right|}^{2+\delta }}{2}\right)|X_{n-1}\right]\hfill \\ \hfill & =2^{1+\delta }\mathrm{E}\left[|X_n-{\theta }_0{X}_{n-1}{|}^{2+\delta }|X_{n-1}\right]+2^{1+\delta }\left|{\theta }_0{X}_{n-1}\right|.\hfill \end{array}$$

We know if $Y\sim N(\mu ,{\sigma }^2)$, then ${\mathrm{E}|Y-\mu |}^p={\sigma }^p\frac{2^{\frac{p}{2}\Gamma \left(\frac{p+1}{2}\right)}}{\sqrt{\pi }}$. Consequently,

$$\begin{array}{cc}\hfill \mathrm{E}\left[\right|X_n{|}^{2+\delta }\left|X_{n-1}\right]& \le 2^{1+\delta }\left[\frac{2^{\frac{2+\delta }{2}\Gamma \left(\frac{3+\delta }{2}\right)}}{\sqrt{\pi }}\right]+2^{1+\delta }{\left|{\theta }_0{X}_{n-1}\right|}^{2+\delta }.\hfill \end{array}$$ (A43)

It follows that,

$$\begin{array}{cc}\hfill \mathrm{E}\left[M_1^{\left(1\right)}{(X_n,X_{n-1})}^{2+\delta }\right|X_{n-1}]& \le 2^{1+\delta }\left[\frac{2^{\frac{2+\delta }{2}\Gamma \left(\frac{3+\delta }{2}\right)}}{\sqrt{\pi }}\right]|X_{n-1}{|}^{2+\delta }+2^{1+\delta }|{\theta }_0{|}^{2+\delta }{\left|X_{n-1}\right|}^{4+2\delta }\hfill \\ \hfill & \le \left(2^{1+\delta }\left[\frac{2^{\frac{2+\delta }{2}\Gamma \left(\frac{3+\delta }{2}\right)}}{\sqrt{\pi }}\right]+2^{1+\delta }{\left|{\theta }_0\right|}^{2+\delta }\right)\left(\right|X_{n-1}{|}^{4+2\delta }+1).\hfill \end{array}$$

Since ${\theta }_0<1$, we can say,

$$\begin{array}{cc}\hfill \mathrm{E}\left[M_1^{\left(1\right)}{(X_n,X_{n-1})}^{2+\delta }\right|X_{n-1}]& \le \left(2^{1+\delta }\left[\frac{2^{\frac{2+\delta }{2}\Gamma \left(\frac{3+\delta }{2}\right)}}{\sqrt{\pi }}\right]+2^{1+\delta }\right)\left(\right|X_{n-1}{|}^{4+2\delta }+1).\hfill \end{array}$$

Define a constant $C_{\delta }:=\left(2^{1+\delta }\left[\frac{2^{\frac{2+\delta }{2}\Gamma \left(\frac{3+\delta }{2}\right)}}{\sqrt{\pi }}\right]+2^{1+\delta }\right)$. The above term then becomes,

$$\begin{array}{cc}\hfill \mathrm{E}\left[M_1^{\left(1\right)}{(X_n,X_{n-1})}^{2+\delta }\right|X_{n-1}]& \le C_{\delta }\left(\right|X_{n-1}{|}^{4+2\delta }+1).\hfill \end{array}$$

Next we analyse the term $M_2^{\left(1\right)}(X_n,X_{n-1})$.

$$\begin{array}{cc}\hfill \mathrm{E}\left[M_2^{\left(1\right)}{(X_n,X_{n-1})}^{2+\delta }|X_{n-1}\right]& =\mathrm{E}\left[X_{n-1}^{4+2\delta }\right|X_{n-1}]\hfill \\ \hfill & =X_{n-1}^{4+2\delta }\hfill \\ \hfill & \le C_{\delta }(X_{n-1}^{4+2\delta }+1).\hfill \end{array}$$

Then, defining $V\left(x\right):=C_{\delta }(x^{4+2\delta }+1)$ it follows that,

$$\begin{array}{cc}\hfill \mathrm{E}\left[V\left(X_n\right)|X_{n-1}\right]& =\mathrm{E}\left[C_{\delta }(X_n^{4+2\delta }+1)|X_{n-1}\right].\hfill \end{array}$$

Using a technique similar to Equation (A43) we get,

$$\begin{array}{cc}\hfill \mathrm{E}\left[C_{\delta }(X_n^{4+2\delta }+1)|X_{n-1}\right]& \le \left[C_{\delta }(2^{3+2\delta }\left[\frac{2^{\frac{4+2\delta }{2}\mathsf{\Gamma }\left(\frac{5+2\delta }{2}\right)}}{\sqrt{\pi }}\right]+2^{3+2\delta }|{\theta }_0{X}_{n-1}{|}^{4+2\delta }+1)\right].\hfill \end{array}$$

Define another constant $C_{\delta }^{\prime }:=C_{\delta }\left(2^{3+2\delta }\left[\frac{2^{\frac{4+2\delta }{2}\mathsf{\Gamma }\left(\frac{5+2\delta }{2}\right)}}{\sqrt{\pi }}\right]-2^{3+2\delta }{\left|{\theta }_0\right|}^{4+2\delta }+1\right)$. Since $\delta >0$, $\frac{2^{\frac{4+2\delta }{2}\mathsf{\Gamma }\left(\frac{5+2\delta }{2}\right)}}{\sqrt{\pi }}>1$. Furthermore, since $|{\theta }_0|<1$, so is $|{\theta }_0{|}^{4+2\delta }$. Hence,

$$2^{3+2\delta }\left[\frac{2^{\frac{4+2\delta }{2}\mathsf{\Gamma }\left(\frac{5+2\delta }{2}\right)}}{\sqrt{\pi }}\right]-2^{3+2\delta }{\left|{\theta }_0\right|}^{4+2\delta }>0.$$

Hence, we have shown that,

$$\begin{array}{cc}\hfill \mathrm{E}\left[V\left(X_n\right)|X_{n-1}\right]& \le (2^{3+2\delta }|{\theta }_0{|}^{4+2\delta })C_{\delta }(X_{n-1}^{4+2\delta }+1)+C_{\delta }^{\prime }.\hfill \end{array}$$

Since $|{\theta }_0|<2^{\frac{1}{4+2\delta }-1}$, $2^{3+2\delta }{\left|{\theta }_0\right|}^{4+2\delta }<1$, and we can express the above equation as,

$$\begin{array}{cc}\hfill \mathrm{E}\left[V\left(X_n\right)|X_{n-1}\right]& \le V\left(X_{n-1}\right)+C_{\delta }^{\prime }.\hfill \end{array}$$

Define the set $C\left(m\right):=\{x:|x{|}^{4+2\delta }+1\le m\}$. From Proposition 11.4.2, [20], for a large enough m, $C\left(m\right)$ forms a petite set. Thus, we have proved that $V\left(x\right)$ as defined in this example satisfies Assumption A.1.1, and $\left\{X_n\right\}$ is V-geometrically ergodic. The $f_j^{\left(1\right)}$’s corresponding to Assumption 1 are given by $f_1^{\left(1\right)}(\theta ,{\theta }_0)=(\theta -{\theta }_0)$ and $f_2^{\left(1\right)}(\theta ,{\theta }_0)=({\theta }_0^2-{\theta }^2)$. Therefore, it follows that,

$$\begin{array}{cc}\hfill {\partial }_{\theta }{f}_1^{\left(1\right)}& =1,\hfill \\ \hfill {\partial }_{\theta }{f}_2^{\left(1\right)}& =-2\theta \mathrm{and}\hfill \\ \hfill -2& <-2\theta <2.\hfill \end{array}$$

Since $f_1^{\left(1\right)}({\theta }_0,{\theta }_0)=f_2^{\left(1\right)}({\theta }_0,{\theta }_0)=0$, We just showed that they also have bounded partial derivatives. We also know that $\left|\theta \right|<1$. Hence, by Proposition 4$f_j^{\left(1\right)}$’s satisfy the conditions of Assumption 1.

The invariant distribution for the simple linear model Markov-chain under parameter $\theta$ is given by a gaussian distribution with mean 0 and variance $\frac{1}{1-{\theta }^2}$. In other words,

$$\begin{array}{c}\hfill q_{\theta }\left(x\right)=\frac{1}{\sqrt{2\pi }}{e}^{-\frac{1-{\theta }^2}{2}{x}^2}.\end{array}$$

Analyzing the log likelihood yields,

$$\begin{array}{cc}\hfill log{q}_0\left(x\right)-log{q}_{\theta }\left(x\right)& =-\frac{x^2}{2}(1-{\theta }_0^2)+\frac{x^2}{2}(1-{\theta }^2)\hfill \\ \hfill & =\frac{x^2}{2}({\theta }_0^2-{\theta }^2).\hfill \end{array}$$

Let $f_1^{\left(2\right)}({\theta }_0,{\theta }_0)=({\theta }_0^2-{\theta }^2)$ and $f_1^{\left(2\right)}({\theta }_0,{\theta }_0)=0$. Since $f_1^{\left(2\right)}({\theta }_0,{\theta }_0)=f_2^{\left(1\right)}({\theta }_0,{\theta }_0)$, by following arguments similar as before, can conclude that $f_1^{\left(2\right)}({\theta }_0,{\theta }_0)$ also satisfies the requirements of Assumption 1. Let $M_1^{\left(2\right)}\left(x\right)=\frac{x^2}{2}$ and define $M_2^{\left(2\right)}\left(x\right):=1$. Let $X_0\sim q_1^{\left(0\right)}$. As long as $\int x^{4+2\delta }{q}_1^{\left(0\right)}\left(x\right)dx<\mathsf{\infty }$, we satisfy all the conditions required for Theorem 5. Finally we need to verify the condition that $\mathcal{K}({\rho }_n,\pi )<C\sqrt{n}$ for some constant $C>0$. The KL-divergence $\int log\left(\frac{{\rho }_n\left(\theta \right)}{\pi \left(\theta \right)}\right){\rho }_n\left(d\theta \right)$ becomes,

$$\begin{array}{cc}\hfill \mathcal{K}({\rho }_n,\pi )& ={\int }_{-1}^{1}log\left(\frac{1}{2\mathrm{Beta}(a_n,b_n)}{\left(\frac{1+\theta }{2}\right)}^{a_n}{\left(\frac{1-\theta }{2}\right)}^{b_n}\right)\hfill \\ \hfill & \times \frac{1}{2\mathrm{Beta}(a_n,b_n)}{\left(\frac{1+\theta }{2}\right)}^{a_n}{\left(\frac{1-\theta }{2}\right)}^{b_n}d\theta .\hfill \end{array}$$

Substituting, $y=\frac{1+\theta }{2}$, we get,

$$\begin{array}{cc}\hfill \mathcal{K}({\rho }_n,\pi )& ={\int }_0^{1}log\left(\frac{1}{2\mathrm{Beta}(a_n,b_n)}{\left(y\right)}^{a_n}{\left(1-y\right)}^{b_n}\right)\frac{1}{2\mathrm{Beta}(a_n,b_n)}{\left(y\right)}^{a_n}{\left(1-y\right)}^{b_n}dy\hfill \\ \hfill & ={\int }_0^{1}log\left(\frac{1}{2}\right)\frac{1}{\mathrm{Beta}(a_n,b_n)}{\left(y\right)}^{a_n}{\left(1-y\right)}^{b_n}dy\hfill \\ \hfill & +{\int }_0^{1}log\left(\frac{1}{\mathrm{Beta}(a_n,b_n)}{\left(y\right)}^{a_n}{\left(1-y\right)}^{b_n}\right)\frac{1}{\mathrm{Beta}(a_n,b_n)}{\left(y\right)}^{a_n}{\left(1-y\right)}^{b_n}.\hfill \end{array}$$

The first term integrates up to $log(1/2)$. The second term is the KL-divergence between a Uniform and Beta distribution with parameters $a_n=n\frac{1+{\theta }_0}{2}$ and $b_n=n(1-\frac{1+{\theta }_0}{2})$ and support $[0,1]$. Following Lemma A.2.1 it follows that $\mathcal{K}({\rho }_n,\pi )$ is upper bounded by,

$$\begin{array}{cc}\hfill \mathcal{K}({\rho }_n,\pi )& <log(1/2)+C_1+\frac{1}{2}log\left(n\right)<C\sqrt{n},\hfill \end{array}$$

for some large constant C. This completes the proof. □

Appendix B.4. Proofs for Misspecified Models

Proof of Theorem 6

As in the proof of Theorem 1, following Equation (A13), we note that,

$$\begin{array}{cc}\hfill \int D_{{\alpha }^{re}}(P_{\theta }^{\left(n\right)},P_{{\theta }_0}^{\left(n\right)}){\tilde{\pi }}_{n,{\alpha }^{re}|X^n}\left(d\theta \right)& \le \frac{{\alpha }^{re}}{1-{\alpha }^{re}}\int \mathcal{K}(P_{{\theta }_0}^{\left(n\right)},P_{\theta }^{\left(n\right)}){\rho }_n\left(d\theta \right)\hfill \\ & +\frac{{\alpha }^{re}}{1-{\alpha }^{re}}\sqrt{\frac{\mathrm{Var}[\int r_n(\theta ,{\theta }_0){\rho }_n\left(d\theta \right)]}{\eta }}+\frac{\mathcal{K}({\rho }_n,\pi )-log\left(ϵ\right)}{1-{\alpha }^{re}}.\hfill \end{array}$$ (A44)

Following from Equations (23) and (26), we get that,

$$\begin{array}{cc}\hfill \int \mathcal{K}(P_{{\theta }_0}^{\left(n\right)},P_{\theta }^{\left(n\right)}){\rho }_n\left(d\theta \right)& \le \mathrm{E}\left[r_n({\theta }_0,{\theta }_n^{*})\right]+n{ϵ}_n,\hfill \end{array}$$

and

$$\begin{array}{cc}\hfill \int \mathrm{Var}\left[r_n(\theta ,{\theta }_0)\right]{\rho }_n\left(d\theta \right)& \le 2n{ϵ}_n+2\mathrm{Var}\left[r_n({\theta }_n^{*},{\theta }_0)\right].\hfill \end{array}$$

Plugging these into Equation (A44), we are done. □

Author Contributions

Formal analysis, I.B.; Investigation, I.B.; Methodology, I.B., V.A.R. and H.H.; Resources, V.A.R. and H.H.; Validation, V.A.R. and H.H. All authors have read and agreed to the published version of the manuscript.

Funding

National Science Foundation: IIS-1816499; DMS-1812197.

Conflicts of Interest

The authors declare no conflict of interest.