Skip to main content
Philosophical transactions. Series A, Mathematical, physical, and engineering sciences logoLink to Philosophical transactions. Series A, Mathematical, physical, and engineering sciences
. 2023 Mar 27;381(2247):20220143. doi: 10.1098/rsta.2022.0143

Statistical inference with exchangeability and martingales

Chris C Holmes 1,✉, Stephen G Walker 2,3
PMCID: PMC10041353  PMID: 36970832

Abstract

In this paper, we start by reviewing exchangeability and its relevance to the Bayesian approach. We highlight the predictive nature of Bayesian models and the symmetry assumptions implied by beliefs of an underlying exchangeable sequence of observations. By taking a closer look at the Bayesian bootstrap, the parametric bootstrap of Efron and a version of Bayesian thinking about inference uncovered by Doob based on martingales, we introduce a parametric Bayesian bootstrap. Martingales play a fundamental role. Illustrations are presented as is the relevant theory.

This article is part of the theme issue ‘Bayesian inference: challenges, perspectives, and prospects’.

Keywords: bootstrap, parametric bootstrap, predictive inference, score function

1. Introduction

Exchangeability forms one of the key starting points for the Bayesian approach, with many articles commencing along the lines ‘Assume the observations to beexchangeable’. The existence of a prior distribution, following such an assumption, has provided a foundation for the Bayesian paradigm (see ch. 4, Bernardo & Smith [1]). The existence of the prior from the representation theorem was established originally by de Finetti [2], for binary outcomes, and later for more general outcomes by Hewitt & Savage [3].

The exchangeable assumption on variables has the implication that the order of the labels becomes irrelevant. Consequently, (x1:∞) forms an infinite exchangeable sequence if, and only if the joint distribution of (x1,…,xn) is invariant under permutations, i.e.

p(x1,…,xn)=p(xσ(1),…,xσ(n)), 1.1

for all permutations σ on {1,…,n} and for all n. Hewitt & Savage [3] referred to this as the joint density being symmetric, in contrast to de Finetti who talked about the exchangeability of the variables. Under the exchangeable assumption, de Finetti [2] showed, for {0,1} variables, that there is a representation of the joint density having the form

p(x1,…,xn)=∫∏i=1np(xi∣θ) Π(dθ),

for some (prior) distribution function Π. For Bayesians, the prior forms the model, and so the sequence is set by the corresponding density function π(θ), assumed to exist, and defined on (0,1). Then p(x∣θ)=θx(1−θ)1−x; i.e. the (xi) given θ are independent Bernoulli random variables. Hewitt & Savage [3] extended this result to more general types of symmetric density.

The existence of the prior π from the representation theorem was of great importance to the Bayesian argument. See, for example, Cifarelli & Regazzini [4] and Bernardo & Smith [1]. It motivated Bayesian analysis through the simple assumption of exchangeable observables. The breakthrough paper of Lindley & Smith [5] was based on the idea of exchangeability, as noted in the opening line from the abstract, 'The usual linear statistical model is reanalyzed using Bayesian methods and the concept of exchangeability'.

Exchangeability is seen to be equivalent to saying that the order in which the data arrives makes no difference to inference; indeed, the probability model for the observations is symmetric. The predictive density follows under the usual conditioning process, i.e.

p(xn+1∣x1,…,xn)=p(x1,…,xn,xn+1)p(x1,…,xn).

Clearly, from the definition of exchangeability, the arrangement/ordering of the conditioned (x1,…,xn) becomes irrelevant.

It is also possible to view the predictive using an update of the prior π(θ), that is

p(xn+1∣x1,…,xn)=∫f(xn+1∣θ) π(θ∣x1,…,xn) dθ, 1.2

where

π(θ∣x1,…,xn)=∏i=1nf(xi∣θ) π(θ)p(x1,…,xn), 1.3

is the posterior distribution, with f(xi∣θ) the normalized likelihood function for data point (xi).

Exchangeability has been viewed as a property on the data; creating a dependence structure where in reality there could be none. To see the argument, suppose the experimenter is to predict the first outcome x1; in order to make a decision, for example. Write this predictive as p0(x). Once x1 has been viewed, the experimenter might then wish to predict x2. It would be an obstinate experimenter who persisted to predict with p0(x), since it would normally be assumed that witnessing x1 would somehow revise the prediction for x2. Indeed, the rationale for a statistical analysis and learning from data is that the observation x1 provides information about the underlying process or population leading to improved prediction.

So write the revised predictive as px1(x). It is easy to see how such a procedure progresses; one eventually predicts xn using the revised density px1,…,xn−1(x). This results in a joint predictive density for the forthcoming observations to be

p(x1,…,xn)=p0(x1) ∏i=2npx1,…,xi−1(xi).

If indeed the order is assumed not to matter; reflecting the notion that the observations could be independent and identically distributed (i.i.d.), then it would be required that p(x1,…,xn)=p(xσ(1),…,xσ(n)) be symmetric in the arguments and hence the existence of the prior-likelihood representation and the Bayesian approach. We write ‘could be’ here to reflect the notion that the data do not explicitly yield how they arose and the leap from the mathematics to the day to day practical data analysis relies on assumptions.

The above motivation de-mystifies Bayes. It is seen as a learning approach, updating beliefs as data arrive. It is no less and no more about the px1,…,xi(x) modelling a dependence between the observables arising as a consequence of revising predictives given observations. The crucial point is that it is the sequence of predictive distribution functions that depend on the observations rather than the next observation having some physical dependence on what has previously been seen. By physical dependence between variables x and y it would be assumed there is a form y=f(x) for some function f. In short, the exchangeable model is an appropriate learning model for i.i.d. data structures.

An early application of the predictive viewpoint for finite populations was developed in Roberts [6], Ericson [7] and later by Geisser [8,9]. This was extended to non-parametric predictive models in Lo [10] and Ghosh & Meeden [11]. Fortini et al. [12] explored concepts of predictive sufficiency and Fortini & Petrone [13,14] discuss the construction of a range of popular exchangeable non-parametric priors through a sequence of predictive distributions, motivated through a predictive de Finetti’s representation theorem.

There are a number of articles that focus on the construction of predictive distributions derived from representation theorems and relying on notions of exchangeability and partial exchangeability. Here, we mention Lijoi et al. [15,16], Favaro et al. [17] and De Blasi [18]. In such cases, and when the posterior is more complicated to derive, then full Bayesian inference can be achieved through these predictive distributions.

(a) . Prediction and model evidence

The predictive viewpoint is especially useful for understanding the plausibility of two or more competing theories (models) for the data, and the updating of such beliefs. We have seen how a hypothesis, such as a posited model M, for the data is characterized by the joint predictive pM(x1,…,xn), which provides an empirical model score when evaluated on the observed data, as seen through the 1-step predictions, and updates, on a sequence of held-out data

log⁡pM(x1,…,xn)=log⁡p0(x1)+∑i=2nlog⁡px1:i−1(xi).

Comparing two hypotheses (models), pM1 versus pM2, can be done by considering the Bayes factor (Kass & Raftery [19])

BF=pM1(x1,…,xn)pM2(x1,…,xn),

obtained by the relative predictive scores on the observed sequence x1:n. This highlights that Bayesian model choice is in essence a comparison of predictive performance on observables.

With exchangeability, the total score for any model is invariant to the n! possible ways of constructing the held-out predictive sequence

log⁡pM(x1,…,xn)=log⁡p0(xσ(1))+∑i=2nlog⁡pσ(x1:i−1)(xσ(i)),

noting the kth element in the sum on the right-hand side has the form p(xσ(k)∣xσ(1:k−1)). It is then straightforward to show [20,21] that the predictive score is equivalent to exhaustive cross-validation averaged over all n-choose-k possible held-out datasets of size k, for all k=1,…,n−1, when using the log predictive, log⁡p(x∣x1:i), as the cross-validation scoring rule. Moreover, Fong & Holmes [20] show this to be the unique scoring rule that ensures order coherency in the model evaluation, and this is irrespective of whether the model is true. Taken together, this highlights the usefulness of the predictive viewpoint and its focus on observables.

(b) . Generalizations

A concern could well be to what extent does a representation theorem cover different and more complex data structures than i.i.d. sequences, such as stationary sequences. A sequence x1,x2,… is stationary if

px1,…,xn(y1,…,yn)=px1+k,…,xn+k(y1,…,yn),

for any n and any k. The representation of a stationary sequence due to Mitra [22] establishes the existence of a prior π(θ) for which it is possible to write

p(x1,…,xn)=∫∏i=1np(xi∣x1,…,xi−1,θ) π(θ) dθ.

See also Aldous [23] and Lijoi et al. [15,16]. The connection between stationary sequences and exchangeable sequences has been considered by Berti et al. [24], where the link involves conditionally identically distributed (c.i.d.) sequences. A c.i.d. sequence is such that

pxn+k(x∣x1,…,xn−1)=pxn(x∣x1,…,xn−1),

for all k≥1. That is, marginally, xn+k has the same distribution as xn for any k≥1.

For more complex data structures, the typical case in the modern data science era, there will be no representation theorem. Hence, some guiding principles are needed. The usual approach is to assign prior distributions to parameters of a likelihood function; to see Bayes as nothing more than likelihood times prior; see Robert [25]. While this is a simple plan, it is not motivated in a de Finetti sense. It is unclear that statistical uncertainty is being reflected in any meaningful way if the majority of prior distributions are constructed for convenience and attempted to be set up for objective reasons.

One attempt to cover more types of data structures is to use the notion of partial exchangeability. See for example, Camerlenghi et al. [26] who look at the problem from a predictive perspective. An interesting question is to what extent partial exchangeability extends to real data, such as covariate-driven non-homogeneous Markov time series with unknown order.

2. A new look at the Bayesian approach

Fong et al. [27] describe how Bayesian uncertainty can be interpreted through missing data. To make this notion concrete, in the case of i.i.d. observations from an infinite population, the missing data when a sample of size n has been observed; i.e. x1:n, is the xn+1:∞. Our interpretation of the Bayesian approach is to deal with this uncertainty by providing a conditional distribution for p(xn+1:∞∣x1:n). More generally, the assumption behind most, if not all, statistical problems is that there is an amount of data, let us label it as xcomp, which if observed, would yield the problem solved. That is, the parameter to be estimated is done so exactly, or the decision to be made can be done so correctly. For example, in a clinical trial if the recruited population was so large that any resulting uncertainty of the treatment effect estimate was considered negligible. So for an observed dataset, say xobs, the Bayesian approach is to construct a predictive distribution p(xmis∣xobs) such that xcomp=(xmis,xobs).

The traditional Bayesian approach can be understood from this perspective. To illustrate this, let us return to the i.i.d. observations and for any sample of size n, there is a predictive density as in (1.2) and derived from the posterior (1.3). Then

p(xn+1:∞∣x1:n)=∏i=n+1∞p(xi∣x1,…,xi−1).

Now use this predictive to generate a large sample (xn+1:N,x1:n) which we will regard as xcomp for some suitably large N. Once this has been obtained, we define θN=θ(x1:N), which is some functional of the data of interest. For example, it could be the sample mean. Doob [28], in his paper on the applications of martingales, establishes under extremely mild conditions, that if θN represents the posterior mean, then θN→θ a.s. and θ is a sample from the posterior derived from the sample of size n.

A less rigorous but intuitive construction is by noting the conventional Bayesian posterior on parameters of interest can be written as

π(θ∣xobs)=∫π(θ,xmis∣xobs) dxmis,

leading to

π(θ∣xobs)=∫π(θ∣xcomp) p(xmis∣xobs) dxmis, 2.1

where xcomp can be taken so large so as to yield no relevant uncertainty in the parameter of interest, θ, meaning in the context of a particular analysis for all practical purposes the conditional posterior can be replaced with a point estimate π(θ∣xcomp)=1θ(xcomp). This highlights the source of the uncertainty in the Bayesian posterior, i.e. the left-hand side of (2.1), which arises from the missing data in the population. The conventional Bayesian posterior is recovered by taking θ(xcomp) as the posterior mean. A numerical solution to (2.1) follows a data-augmentation Monte Carlo approach by simulating xmis∼p(xmis∣xobs) using the sequence of 1-step predictives and then picking off the resulting estimate θ(xcomp). As shown in Fong et al. [27], the computational solution is trivially parallel and often much faster to run than Markov chain Monte Carlo (MCMC). This missing data view also points to extensions of traditional Bayes involving more general predictive machines for use in p(xmis∣xobs).

The Bayes posterior (2.1) has two components,

π(θ∣xobs)=∫π(θ∣xcomp)⏟Bayes estimate (large sample)  p(xmis∣xobs)⏟predictive (missing info.)  dxmis.

The usual Bayesian approach is to construct the predictive, p(xmis∣xobs), using a likelihood-prior mixture (1.2), that is fixed a priori, with inference focused on parameters of the likelihood. This ensures exchangeability and provides for a simple update via Bayes rule. The approach in Fong et al. [27] supports the replacement of π(θ∣xcomp) with an estimate, θ(xcomp), using an appropriate functional targeting any statistic of interest, not necessarily indexing a likelihood function, and a predictive p(xmis∣xobs) built using all available information at time of inference, not necessarily involving a prior. For example, in this regard, it is perfectly valid and reasonable to consider uncertainty in estimates arising from the Bayes linear model (e.g. Lindley & Smith [5]) obtained in the limit of large data, without assuming that the true data generating process is a linear model. The conventional Bayesian approach conflates these two issues, that the estimate refers to parameters in the predictive model assumed to be true (see §3).

(a) . Martingales

In fact, martingales are important for this view of the Bayesian methodology. Consider the posterior mean conditional on x1:n+1, i.e.

θn+1=∫θ π(θ∣x1:n+1) dθ=∫θ f(xn+1∣θ) π(θ∣x1:n) dθp(xn+1∣x1:n),

where xn+1 has been taken from the predictive p(xn+1∣x1:n). Then, clearly, E (θn+1∣x1:n)=θn and so for m>n, the sequence of posterior means forms a martingale. Indeed, any mean functional would form a martingale as would the sequence of posterior distributions itself. Martingales are the important feature to ensure convergence of the parameter of interest.

Our argument, and promoted in Fong et al. [27], is that it is the martingale property which should be the determining criterion for the choice of p(xn+1:∞∣x1:n) and, more generally, for p(xmis∣xobs). That is, if we define θm+1=θ(x1:m+1) and xm+1 has been taken from a ‘predictive’ p(xm+1∣x1:m), then we require that E(θm+1∣x1:m)=θm.

Fong et al. [27] focus on non-parametric models including a sequence of non-parametric predictives using copulas. Specifically, they take

Pm+1(x)=(1−am)Pm(x)+am H(Pm(x),Pm(xm+1)), 2.2

where the weights are of the form am=1/(a+m)→0 and H is a partial derivative of a Gaussian copula function with correlation parameter ρ. That is H(u,v)=∂Cρ(u,v)/∂v, which yields

H(u,v)=Φ(Φ−1(u)−ρΦ−1(v)1−ρ2).

The sequence (Pm)m>n forms a martingale when the xm+1 comes from Pm(⋅). The sequence commences at the distribution estimator Pn from the observed sample x1:n. Convergence of (Pm) to a random distribution function follows from Berti et al. [29]. The limit P∞ is regarded as a random draw from the martingale posterior distribution.

The use of copulas here should not come as any great surprise. Consider the Bayesian predictive density given by

p(x∣x1:n+1)=∫f(x∣θ) f(xn+1∣θ)π(θ∣x1:n) dθp(xn+1∣x1:n),

which can be written as

p(x∣x1:n+1)=p(x∣x1:n) ∫f(x∣θ) f(xn+1∣θ)π(θ∣x1:n) dθp(x∣x1:n) p(xn+1∣x1:n).

The fraction part on the right is a copula density function, which can be written as

cn(Pn(x) Pn(xn+1)),

for some copula density cn(⋅,⋅). In most cases, this copula itself depends on the x1:n; though in some conjugate Bayesian models, it only depends on the sample size n. Indeed, the complexity of prior to posterior procedures can be rendered difficult due to the nature of an intractable copula when it does depend on x1:n. This motivated Hahn et al. [30] to adopt copula densities which only depended on the sample size and took the non-parametric copula distribution function

C(u,v)=(1−am) uv+am Cρ(u,v),

i.e. a sample size weighted combination of the independence copula and the Gaussian copula with correlation parameter ρ. The non-parametric credentials are valid here since the Dirichlet process sequence of predictives arise when, instead of Cρ, we use the maximum copula, C~(u,v)=min{u,v}, which arises as ρ→1.

In order to better understand the martingale posterior and to investigate the parametric counterparts, we reconsider the bootstrap techniques, notably the bootstrap and Bayesian bootstrap. This will help us better understand the separate dealings with statistical uncertainty.

3. The bootstrap and Bayesian bootstrap

The bootstraps are very insightful tools with which to understand how the frequentist and Bayesian approaches to dealing with uncertainty differ. Both start with the empirical distribution function Pn, which puts mass 1/n at each of the observed data points.

The frequentist regards the sample of size n; i.e. x1:n as being the source of uncertainty, in that it is but one of many possible observed samples of size n. The uncertainty in this sample, and the consequences of this, could be ascertained if multiple independent samples of size n could be gathered. This is not possible as the true distribution is unknown; but the empirical distribution function is available. Sampling alternative datasets of size n is therefore taken from the empirical distribution function Pn. Multiple bootstrap samples can be obtained, (x1:n(b))∼Pn for b=1,…,B. From each of these bootstrap samples, a statistic of interest can be computed, T(b)=T(x1:n(b)) and consequently the sampling variance of T=T(x1:n) can be estimated.

The Bayesian bootstrap was introduced in Rubin [31] and was seen as a Bayesian version of the bootstrap introduced in Efron [32]. The reason why this comparison was made is because both generate random distributions of the form

P=∑i=1nqi 1xi.

For the Efron bootstrap, the random weights (qi) are based on multi-nomial sampling; q=Mn(n;1/n,…,1/n)/n. On the other hand, the Bayesian credentials for the Bayesian bootstrap is due to its connection with the Dirichlet process [33]. The Bayesian bootstrap generates random probability distributions which attach random weights to the set of observed data x1:n; i.e.

P=∑i=1nwi 1xi, 3.1

where the (w1:n) have a Dirichlet distribution with all the parameters set to 1. This version of the bootstrap is very clear and that it belongs to the Bayesian non-parametric set of tools is also very clear. However, based on work done by Blackwell & MacQueen [34], which can be seen as a follow-up to the Doob [28] result, the Bayesian bootstrap can be viewed in an alternative and illuminating way.

Take a sample of size n and construct the empirical distribution function, Pn. Then sample xn+1∼Pn and update the empirical distribution with the new sample. That is, if xn+1 is taken from Pn then construct the new empirical predictive distribution Pn+1 given by

Pn+1=n Pn+1xn+1n+1.

Continue in this fashion; to get xn+1 and Pn+2 and so on. The sequence (Pm)m>n forms a martingale and some elementary theory for martingales on distribution functions [24] indicates that with probability one Pn converges to a random distribution P∞. Since P∞ can only be of the form (3.1) it is only a matter of finding the weights attached to the (wi). It is not difficult to establish the correct Dirichlet weights for them; see Blackwell & MacQueen [34] and Sethuraman [35].

Thus we see the Bayesian bootstrap is constructing a predictive distribution P(xn+1:∞∣x1:n) and it is easy to see the sequence is given by

Pm+1(x)=(1−am) Pm(x)+am 1(xm+1≤x), 3.2

with xm+1 coming from Pm, and am=1/(1+m). The sequence forms a martingale and this is easy to check. The limit P∞ can be sampled according to (3.1). Then (3.1) can be viewed as a ‘short-cut’ to avoid the sampling of the (Pm). The non-trivial differences between the two types of bootstrap is quite clear.

Given P∞, and the missing data view from §2, we can proceed to make inference by picking off an estimate of any particular statistic of interest, θ(P∞). One particularly interesting statistic is the estimate of the parameters in a statistical model, f(x∣θ), minimizing the self-information loss (log-likelihood), i.e.

θ(P∞)=arg⁡minθ[∫−log⁡f(x∣θ) P∞(dx)].

With P∞ as the Bayesian bootstrap leads to the weighted-likelihood bootstrap and extensions thereof [36,–41]. As noted in §2, we see it as perfectly valid to think about the Bayesian uncertainty in the value of θ arising from a statistical model, f(x∣θ), fit in the limit of large data, without assuming that the data were generated by this statistical model for some unknown setting of θ.

It is to be noted that the sequence in (2.2) can be seen as a smoothed version of (3.2), where the indicator function has been smoothed to the Gaussian copula. Also note that as ρ→1 it is that Hρ(u,v)→1(u=v).

The frequentist is replicating statistics of sample size n. The Bayesian is obtaining ‘true’ statistics via T=T(P∞) and acknowledging the uncertainty in any single T(P) through the posterior it generates. It is this latter view of the Bayesian handling of uncertainty that is of interest to us. It is our intention to extend the family of Bayesian bootstraps and to use these to construct posterior distributions and adopting this as the fundamental Bayesian paradigm. That is, the construction of p(xn+1:∞∣x1:n), namely p(xmis∣xobs), is the fundamental task while ensuring convergence of the statistics of interest, θ(xcomp), and the sequence of predictive distribution. This can be guaranteed via the use of martingales.

Our understanding of the construction of p(xn+1:∞∣x1:n) is that it is prior free; it starts at the sample size n. Instead of generating multiple replica datasets of size n from this, along the lines of the frequentist bootstrap, we use it to generate the missing data along the line of the Bayesian bootstrap and the construction of a posterior distribution. To acknowledge the fusion here,we label the final output as a form of frequentist posterior; as we can take p(xn+1:∞∣x1:n) to be the product of f(xm+1∣θ^m) for m=n,…,∞, where the θ^m would be the maximum-likelihood estimator based on x1:m.

4. A parametric Bayesian bootstrap

Consider the data model f(⋅∣θ). The parametric bootstrap introduced in Efron [42] replaces the empirical distribution function with the plug-in density estimator, f(⋅∣θ^). The reasons for this become straightforward if the model is assumed to be correct, or is approximately so. Here, the θ^=θ(x1:n) is some functional of the data, such as the mle. In all other respects, the parametric bootstrap mirrors that of the non-parametric one. So a bootstrap sample (x1:n(b)) is taken i.i.d. from f(⋅∣θ^). For each sample, a new mle estimator is obtained as θ^(b) and the collection of such can be used to estimate the sampling distribution and specifically the variance of θ^.

Efron observed that the distribution of the (θ^(b)) resembles closely a possible posterior distribution and advocated it could be used in some importance sampling procedure to obtain samples from a posterior. It cannot be regarded as a posterior itself as it is not derivable from a prior and the more important point is that it is a distribution for the estimator, not for the true parameter value.

The connection and differences between the bootstrap and Bayesian bootstrap have shown us that a different sampling strategy from f(⋅∣θ^) implements our Bayesian approach. The Bayesian version assumes further the θ^ is an unbiased estimator for the true value θ. We then sample xn+1 from the data model using the plug-in θ^ which is then updated to θ^n+1 using xn+1, and so on. In short, given x1:m with estimator θ^m, we perform

xm+1∼f(⋅∣θ^m)andθ^m+1=θ(x1:m+1),m≥n.

The (θ^m≥n) form a martingale and provided the variances do not grow but converge then a random θ∞ exists and can be seen as a sample from a ‘parametric martingale’ posterior distribution π(⋅∣x1:n).

It is worth at this point considering a simple example. Suppose we have a normal location parameter model for which the data model is N(⋅∣θ,σ2) where θ is unknown and σ is known. Then from a sample of size n we get the θ^n=x¯n which is an unbiased estimator for θ. We then sample xn+1 from N(⋅∣x¯n,σ2) and get

θ^n+1=x¯n+1=nθ^n+xn+1n+1,

and so on. This is a martingale for (θ^)m>n and to investigate convergence, we consider the τm2=Var θ^m. Now writing θ^m+1=θ^m+σ z/(m+1), where z is a standard normal random variable, we get

τm+12=τm2+σ2(m+1)2⟹τm+12=σ2∑k=n+1m1k2,

resulting in θ^∞ being a normal random variable with mean θ^n and variance cn σ2 where cn=∑k>n1/k2≈1/n. This leads to the frequentist posterior as a probability measure directly on the unknown θ^∞ as approximately N(θ^n,σ2/n).

In general, we assume we can write

θ^m+1=θ^m+sm(xm+1,θ^m),

for some function sm, where xm+1 comes from f(⋅∣θ^m). Hence, for the martingale, we require

E sm(xm+1,θ^m)=0,

and so the sm resembles or acts similarly to a score function. Note that it is easy to compute the variance of θ∞ since it would be equal to

τ∞2=∑k>nVar sk−1(xk,θ^k−1).

This finding motivates a potentially simpler idea, which is to use the score function from the data model; i.e. define s(x,θ)=∂log⁡f(x∣θ)/∂θ. Then we can use the sequence of estimators as

θm+1=θm+ϵm s(xm+1,θm),

where xm+1 comes from f(⋅∣θm). Here, ϵm would act as a step size and the updating here resembles a gradient descent algorithm. The variance of θ∞ would now be

τ∞2=∑k>nϵk−12 Var s(xk,θk−1),

which if ϵk is of order 1/(k+n) then τ∞2 will be of order 1/n, which is usual for parametric posterior distributions.

The idea here for updating the parameter is as follows. If there is a procedure for updating the parameter estimator as real data arrive; e.g. the mle or gradient descent for more complex problems, then this procedure continues post data where the data are now replaced by sample data from the model using the current parameter estimate. This is done in such a way so that the ensuing sequence of parameter estimators forms a martingale.

Here, it is worth pointing out about the form for p(xmis∣xobs), constructed sequentially. Unless the model is based on exchangeability, the ordering of the xn+1:∞ will matter. For example, without exchangeability, the joint density of (xn+2,xn+1) will not share the same density as (xn+3,xn+2). We are not aware of any practical consequence of this. As well, if the sequence is c.i.d. the proximity to an exchangeable sequence depends on the sample size and asymptotically a c.i.d. is exchangeable. On the other hand, if exchangeability is regarded as an essential component to Bayesian uncertainty quantification, then the prior to posterior will be the best choice.

5. Illustrations

Example 5.1. —

The first and obvious example is the normal location parameter; the martingale here is θm+1=(mθm+ym+1)/(m+1) for m≥n with ym+1∼N(θm,σ2). So θm+1=θm+σ zm/(m+1), which is based on the θm being the sample mean at each iteration. Here, the (zm) are independent standard normal random variables. The frequentist posterior is normal with mean θn, the sample mean from the observed data, and with variance σn2=σ2∑m=n∞1/(m+1)2. The sum is approximately 1/n.

If the variance is also unknown, we set up two martingales, one for the mean and one for the variance. Now we take the same martingale for the mean and take ym+1∼N(θm,σm2), where we take

σm2=∑i=1m(yi−θm)2κmandκm=∑i=1mii+1.

The martingale for (σm2) is explained as

κm+1σm+12=∑i=1myi2+ym+12−(mθm+ym+1)2/(m+1),

which can be tidied up to give

κm+1σm+12=κmσm2+mm+1(θm−ym+1)2.

Taking a conditional expectation keeping y1:m fixed,

κm+1E (σm+12∣y1:m)=σm2κm+1,

which demonstrates the martingale.

We take a worked illustration with n=100 independent samples from a standard normal distribution. The value of N we use to stop the martingales is 1000. This value appears large enough for the martingales to converge. A histogram of the 100 samples from the 2×100 martingales we ran are presented in figure 1. The sample mean and sample variance from the data were 0.048 and 0.848, respectively.

Figure 1.

Figure 1.

Marginal frequentist posterior distribution for θ and σ2.

Example 5.2. —

Here, we consider the Poisson model; with density function

f(y,θ)=θy/y! e−θ,y∈{0,1,2…}.

Hence, l′(y,θ)=−1+y/θ and l″(y,θ)=−y/θ2. Taking ym+1∼Pois(θm), the martingale becomes

θm+1=θm+(ym+1/θm−1)∑i=1myi/θm2=θm+(ym+1−θm)/m.

In this case, it is easy to compute the limit values of the mean and variance. So E θ∞=θn, due to the martingale, and

Var θ∞=θn ∑m≥n1m2≈θnn.

Note that the Bayesian objective posterior using the prior π(θ)∝1/θ is gamma with mean θn and variance θn/n.

We ran an illustration using n=100 and taking independent samples from the Poisson distribution with mean 1. We collected 100 frequentist posterior samples by running 100 martingales each to a length of N=2000. The posterior is presented as the histogram and the objective Bayesian gamma posterior with prior π(θ)∝1/θ is shown alongside in figure 2.

Figure 2.

Figure 2.

Frequentist posterior (histogram) from the Poisson example, alongside the Bayesian gamma objective posterior (line).

Example 5.3. —

In this part, we consider a two parameter gamma model, f(y,θ)=baya−1e−yb/Γ(a). So l(y,a,b)=alog⁡b−log⁡Γ(a)+alog⁡y−by and so

∂l∂a=log⁡b−ψ(a)+log⁡yand∂l∂b=ab−y,

where ψ is the digamma function, with

∂2l∂a2=−ψ′(a),∂2l/∂b2=1/b2and∂2l∂a∂b=−ab2.

Defining the matrix and vector

Hm=m(ψ(am)−1/bm−1/bmam/bm2)andvm=(log⁡bm−ψ(am)+log⁡ym+1am/bm−ym+1),

where ym+1∼Ga(am,bm). Then the martingale is θm+1=θm+Hm−1 vm.

We took n=100 samples from the gamma distribution with a=3 and b=2 and 500 martingales were run each with a run length of 1000. The frequentist distribution for (a,b) is presented in figure 3.

Figure 3.

Figure 3.

(a and b) Frequentist posteriors from the gamma example.

Example 5.4. —

This example looks at the standard linear regression model,

yi=xi′β+σεi,i=1,…,n,

where xi is a p×1 vector of regression variables and β is a p×1 vector of unknown coefficients. We assume the σ is known and the (εi) are independent standard normal errors. The estimator based on the observed sample of size n is βn=(X′X)−1X′y where X is the n×p matrix from (xi)i=1:n.

We construct the martingale using

∂l∂βj=∑i=1n(yi−xi′β) xijand∂2l∂βjβk=−∑i=1nxijxik,

where l(β)=12∑i=1:n(yi−xi′β)2. The martingale becomes, for m≥n,

βm+1=βm+(∑i=1mxi xi′)−1 xm+1 (ym+1−xm+1′βm),

where ym+1∼N(xm+1′βm,σ2). Here, the (xi)m>n can be arbitrarily chosen or sampled from the observed values (xi)i=1:n. The clear choice would be to sample from the empirical distribution.

For the specific illustration, we took n=100 samples with the true β=(−2,3) and all the (xij) sampled independently from a standard normal distribution, with σ=1. We ran 500 martingales and each was run for a length of size 1000. The 500 frequentist posterior samples are presented in figure 4.

Figure 4.

Figure 4.

Frequentist posteriors from the linear model example.

Example 5.5. —

A further example is a Markov-driven time series with standard normal marginals. So

yi=ρ yi−1+1−ρ2εi,i=2,…,n,

where the (εi) are independent standard normal random variables. Then ρn=∑i=2nyi yi−1/(n−1). The log-density is given by

l(y,ρ)=−12log⁡(1−ρ2)−12(y2−ρy1)2/(1−ρ2).

The first and second derivatives with respect to ρ are easy to obtain, e.g.

∂l∂ρ=ρ(1−ρ2)−ρ(y2−ρy1)2(1−ρ2)+y1(y2−ρy1)(1−ρ2)2.

Hence, the martingale is

ρm+1=ρm−∂l/∂ρ(ym+1,ρm)∑i=1m∂2l/∂ρ2(yi,ρm),

where ym+1 is normal with mean ρym and variance 1−ρm2.

By way of an illustration, we took y1=0 and then the (yi) for i=1,…,n, with n=100, according to the Markov model with ρ=0.7. The estimator turned out to be 0.758. We ran 100 martingales and each was run for a length of 500 iterations. The histogram of the 100 samples from the frequentist posterior is provided in figure 5.

Figure 5.

Figure 5.

Frequentist posterior from the Markov time series model example.

6. Theory

The theory we present here is about martingales and the convergence of such to limits. For general information on martingales, see Williams [43]. The first result is for the sequence of random distributions (Pm)m>n, which form a martingale in the sense that

E(Pm+1(y)∣y1:m)=Pm(y)for all y.

Since a probability distribution is bounded the Pm(y) converges almost surely to a P∞(y) for each y. That the sequence (Pm)m=n+1∞ converges with probability one to a random distribution function is a consequence of theorem 2.2 appearing in Berti et al. [29]. A modified version of the theorem relevant to the present paper is provided here.

Theorem 6.1. —

Given Pn, if Pm(y), is a sequence of random distribution functions, for m>n, and converges almost surely to P∞(y) for each y with E Pm=Pn, and Pn is a tight distribution function, then Pm converges weakly to the random distribution function P∞ almost surely.

From the construction of Pm(y) for m>n, it is clearly evident that Pm(y) is a martingale for each y. From the martingale convergence theorem, there exists a P∞(y) for each y for which Pm(y)→P∞(y) almost surely. Now Pn is a fixed distribution function and hence for any ϵ>0 there will be a compact set K such that Pn(K)>1−ϵ. Hence, as such, Pn is a tight distribution function. For more on tightness, see Billingsley [44]. Hence, from the theorem, Pi converges weakly to the random distribution function P∞ almost surely.

For a general martingale for the parameter (θm)m>n, we have the following martingale convergence theorem:

Theorem 6.2. —

If (θm)m>n is a martingale and supmE|θm|2<∞ then there exists a random variable θ∞ such that θm→θ∞ a.s. and E(θ∞)=θn.

In practice, we consider Var(θm).

One of the martingales on which we rely is the stochastic gradient descent, or ascent in our case, algorithm. If (yn) are i.i.d. from f(⋅∣θ∗) and

θn=θn−1+σn2 s(yn,θn−1), 6.1

for some suitable sequence (σn2) and starting point θ0, where s(y,θ)=∂log⁡f(y∣θ)/∂θ, then we require that θn→θ∗ as n→∞ in the sense that E[(θn−θ∗)2]→0. See, for example, Murphy [45].

The essence of the argument for convergence is as follows:

(θn+1−θ∗)2 =(θn+σn+12 s(yn+1,θn)−θ∗)2 =(θn−θ∗)2+σn+14s2(yn+1,θn)+2σn+12s(yn+1,θn) (θn−θ∗).

The last term on the right side is the important part and write the expectation as

2σn+12(θn−θ∗)2 s(θn,θ∗)θn−θ∗,

where

s(θ,θ∗)=∫s(y,θ) f(y∣θ∗) dy,

and the assumption is that for θ sufficiently close to θ∗, it is that s(θ,θ∗)>0 if θ<θ∗, s(θ,θ∗)<0 if θ>θ∗ and s(θ∗,θ∗)=0, and further that s(θ,θ∗)/(θ−θ∗)<−L within a neighbourhood for some finite L>0. Hence, we can write

E [(θn+1−θ∗)2∣y1:n]≤(θn−θ∗)2{1−2Lσn+12}+σn+14s2(yn+1,θn).

Proceeding, for some constants c1>0 and c2>0, we have

dn+1≤(1−c1σn+12) dn+c2σn+14,dn=E [(θn−θ∗)2],

and recall σn+12 is of order 1/n. It is now standard mathematics to demonstrate that dn→0.

This supports (6.1) as a Bayesian style learning algorithm, updating the Bayesian estimator θ0 with data. Once we reach the end of the data, we revert to a martingale, generating the data with the current point estimator and proceeding with (6.1) to infinity. The θ∞ exists and has expected value θn and the variance, just as with usual Bayesian updating, is O(1/n). Hence, the ‘posterior’ is available and which we regard as well motivated as the traditional Bayesian approach.

In order for the martingale to converge, we need to look at Var(θm). Now if

θm+1=θm+σm2 s(ym+1,θm), 6.2

then

Var(θm+1)=Var(θm)+σm4 E Var{ s(ym+1,θm)},

where the variance applies to the ym+1 given θm and the expectation with respect to θm. Hence,

Var(θ∞)=∑m>nσm4 E Var{ s(ym+1,θm)},

which we need to ensure is finite. This can be achieved through a focus on σm, the interpretation of which is to be found in (6.2) and illustrated in the examples we have presented, specifically how it is related to posterior variances involving the Fisher information.

7. Discussion

From the understanding of Bayesian inference through the construction of p(xmis∣xobs) in order to obtain xcomp we relax the strict assumptions which take this predictive to start with a prior distribution on a parameter. We argue it is straightforward to construct such a predictive, using density estimators, for example, even based on maximum-likelihood estimators, which in the non-parametric case could be represented by the empirical distribution function.

A key to the construction of p(xmis∣xobs) is the martingale, indeed, it is a key component of the Bayesian framework, though often concealed from view as the prior to posterior update puts it firmly into the background. The martingale ensures convergence; i.e. the existence of what we have named the frequentist posterior. On the other hand, it is the step-by-step density estimators, or predictives, p(xm+1∣x1:m), which provide the accuracy.

After the data are exhausted, it is the xmis which creates the uncertainty; rather than the notion of an alternative dataset of size n that could have been seen in the Frequentist approach. Simulating this xmis from the predictive to form xcomp=(xmis,xobs) generates random outcomes of parameter values using the estimate θ(xcomp), and these form the basis of the frequentist posterior distribution. Each sampled θ∞ represents a potentially true value and the collection quantifies the associated uncertainty.

For readers remaining unclear on the ideas, let us return to the standard Bayesian approach understood from the missing data perspective. Given x1:n the Bayesian derives what is regarded as the ‘best’ density estimator for xn+1 via the posterior predictive (1.2). This is sampled, the model is updated using the sampled value and the procedure moves on to xn+2, and so on. In the limit, the object of interest, let us assume it is θ, constructed from x1:∞, is a sample from the posterior based on the sample of size n. The key is the martingale sequence of estimators of θ as the samples are obtained.

The extended view of this strategy promoted in the paper is what exactly is the best density estimator for xm+1 given x1:m for any m. Many density estimators are available. However, the Bayesian procedure tells us how to perform posterior analysis using any density estimator, including classical ones, such as those based on maximum-likelihood estimators.

Acknowledgements

The authors are grateful for the comments and suggestions of three reviewers which have allowed us to improve the presentation of the paper.

Data accessibility

This article has no additional data.

Authors' contributions

C.C.H.: conceptualization, formal analysis, investigation, methodology, writing–original draft, writing–review and editing; S.G.W.: conceptualization, formal analysis, investigation, methodology, writing–original draft, writing–review and editing.

Both authors gave final approval for publication and agreed to be held accountable for the work performed therein.

Conflict of interest declaration

We declare we have no competing interests.

Funding

No funding has been received for this article.

References

  • 1.Bernardo JM, Smith AFM. 2009. Bayesian theory, vol. 405. Chichester, UK: John Wiley & Sons. [Google Scholar]
  • 2.de Finetti B. 1937. La prevision: ses lois logiques, ses sources subjectives. Ann. Inst. H. Poincare 7, 1-68. [English translation in Studies in Subjective Probability (1980) (H. E. Kyburg and H. E. Smokler, eds.) 53-118. Krieger, Malabar, FL. [Google Scholar]
  • 3.Hewitt E, Savage LJ. 1955. Symmetric measures on Cartesian products. Trans. Am. Math. Soc. 80, 470-501. ( 10.1090/S0002-9947-1955-0076206-8) [DOI] [Google Scholar]
  • 4.Cifarelli DM, Regazzini E. 1996. De Finetti’s contribution to probability and statistics. Stat. Sci. 11, 253-282. ( 10.1214/ss/1032280303) [DOI] [Google Scholar]
  • 5.Lindley DV, Smith AF. 1972. Bayes estimates for the linear model. J. R. Stat. Soc. B (Methodological), with discussion 34, 1-41. ( 10.1111/j.2517-6161.1972.tb00885.x) [DOI] [Google Scholar]
  • 6.Roberts HV. 1965. Probabilistic prediction. J. Am. Stat. Assoc. 60, 50-62. ( 10.1080/01621459.1965.10480774) [DOI] [Google Scholar]
  • 7.Ericson WA. 1969. Subjective Bayesian models in sampling finite populations. J. R. Stat. Soc. B (Statistical Methodology) 31, 195-224. ( 10.1111/j.2517-6161.1969.tb00782.x) [DOI] [Google Scholar]
  • 8.Geisser S. 1982. Aspects of the predictive and estimative approaches in the determination of probabilities. Biometrics 38, 75-85. ( 10.2307/2529856) [DOI] [Google Scholar]
  • 9.Geisser S. 1983. On the prediction of observables: a selective update. Technical report, University of Minnesota.
  • 10.Lo AY. 1988. A Bayesian bootstrap for a finite population. Annals of Statistics 16, 1684-1695. [Google Scholar]
  • 11.Ghosh M, Marsden G. 1997. Bayesian Methods for Finite Population Sampling. New York, USA: Routledge. [Google Scholar]
  • 12.Fortini S, Ladelli L, Regazzini E. 2000. Exchangeability, predictive distributions and parametric models. Sankhya A Indian J. Stat. A. 62, 86-109. [Google Scholar]
  • 13.Fortini S, Petrone S. 2012. Predictive construction of priors in Bayesian nonparametrics. Braz. J. Probab. Stat. 26, 423-449. ( 10.1214/11-BJPS176) [DOI] [Google Scholar]
  • 14.Fortini S, Petrone S. 2014. Predictive distribution (de Finetti’s view). Wiley StatsRef: Statistics Reference Online, pp. 1–9.
  • 15.Lijoi A, Pruenster I, Walker SG. 2007. Bayesian consistency for stationary models. Econometr. Theory 23, 749-759. ( 10.1017/S0266466607070314) [DOI] [Google Scholar]
  • 16.Lijoi A, Mena RH, Pruenster I. 2007. Bayesian nonparametric estimation of the probability of discovering a new species. Biometrika 94, 769-786. ( 10.1093/biomet/asm061) [DOI] [Google Scholar]
  • 17.Favaro S, Pruenster I. 2009. Bayesian nonparametric inference for species variety with a two parameter Poisson-Dirichlet process prior. J. R. Stat. Soc. B 71, 993-1008. ( 10.1111/j.1467-9868.2009.00717.x) [DOI] [Google Scholar]
  • 18.De Blasi P, Favaro S, Lijoi A, Mena R, Pruenster I, Ruggiero M. 2015. Are Gibbs–type priors the most natural generalization of the Dirichlet process? IEEE Trans. Pattern Anal. Mach. Intell. 37, 212-229. ( 10.1109/TPAMI.2013.217) [DOI] [PubMed] [Google Scholar]
  • 19.Kass RE, Raftery AE. 1995. Bayes factors. J. Am. Stat. Assoc. 90, 773-795. ( 10.1080/01621459.1995.10476572) [DOI] [Google Scholar]
  • 20.Fong E, Holmes CC. 2020. On the marginal likelihood and cross-validation. Biometrika 107, 489-496. ( 10.1093/biomet/asz077) [DOI] [Google Scholar]
  • 21.Gneiting T, Raftery AE. 2007. Strictly proper scoring rules, prediction, and estimation. J. Am. Stat. Assoc. 102, 359-378. [Google Scholar]
  • 22.Maitra A. 1977. Integral representations of invariant measures. Trans. Am. Math. Soc. 229, 209-225. ( 10.1090/S0002-9947-1977-0442197-5) [DOI] [Google Scholar]
  • 23.Aldous DJ 1985. Exchangeability and related topics. Lectures notes in Mathematics, 1117. New York, NY: Springer. [Google Scholar]
  • 24.Berti P, Pratelli L, Rigo P. 2004. Limit theorems for a class of identically distributed random variables. Ann. Probab. 32, 2029-2052. ( 10.1214/009117904000000676) [DOI] [Google Scholar]
  • 25.Robert CP. 2007. The Bayesian choice. New York, NY: Springer. [Google Scholar]
  • 26.Camerlenghi F, Lijoi A, Pruenster I. 2017. Bayesian prediction with multiple–samples information. J. Multivariate Anal. 156, 18-28. ( 10.1016/j.jmva.2017.01.010) [DOI] [Google Scholar]
  • 27.Fong E, Holmes C, Walker SG. 2023. Martingale posterior distributions. J. R. Stat. Soc. B. In press. [Google Scholar]
  • 28.Doob JL. 1949. Application of the theory of martingales. Actes du Colloque International Le Calcul des Probabilites et ses applications (Lyon, 28 Juin–3 Juillet 1948), Paris CNRS, 23–27.
  • 29.Berti P, Pratelli L, Rigo P. 2006. Almost sure weak convergence of random probabilty measures. Stoch. Int. J. Probab. Stoch. Process. 78, 91-97. ( 10.1080/17442500600745359) [DOI] [Google Scholar]
  • 30.Hahn PR, Martin R, Walker SG. 2018. On recursive Bayesian predictive distributions. J. Am. Stat. Assoc. 113, 1085-1093. ( 10.1080/01621459.2017.1304219) [DOI] [Google Scholar]
  • 31.Rubin DB. 1981. The Bayesian bootstrap. Ann. Stat. 9, 130-134. ( 10.1214/aos/1176345338) [DOI] [Google Scholar]
  • 32.Efron B. 1979. Bootstrap methods: another look at the jackknife. Ann. Stat. 7, 1-26. ( 10.1214/aos/1176344552) [DOI] [Google Scholar]
  • 33.Ferguson TS. 1973. A Bayesian analysis of some nonparametric problems. Ann. Stat. 1, 209-230. ( 10.1214/aos/1176342360) [DOI] [Google Scholar]
  • 34.Blackwell D, MacQueen JB. 1973. Ferguson distributions via Polya urn schemes. Ann. Stat. 1, 353-355. ( 10.1214/aos/1176342372) [DOI] [Google Scholar]
  • 35.Sethuraman J. 1984. A constructive definition of Dirichlet priors. Statistica Sinica 4, 639-650. [Google Scholar]
  • 36.Fong E, Lyddon S, Holmes C. 2019. Scalable nonparametric sampling from multimodal posteriors with the posterior bootstrap. In Int. Conf. on Machine Learning (pp. 1952–1962). PMLR.
  • 37.Lyddon S, Walker S, Holmes CC. 2018. Nonparametric learning from Bayesian models with randomized objective functions. Adv. Neural Inf. Process. Syst. 31, 2075-2085. [Google Scholar]
  • 38.Lyddon SP, Holmes CC, Walker SG. 2019. General Bayesian updating and the loss-likelihood bootstrap. Biometrika 106, 465-478. ( 10.1093/biomet/asz006) [DOI] [Google Scholar]
  • 39.Newton MA, Polson NG, Xu J. 2021. Weighted Bayesian bootstrap for scalable posterior distributions. Can. J. Stat. 49, 421-437. ( 10.1002/cjs.11570) [DOI] [Google Scholar]
  • 40.Newton MA, Raftery AE. 1994. Approximate Bayesian inference with the weighted likelihood bootstrap. J. R. Stat. Soc. B (Methodological) 56, 3-26. ( 10.1111/j.2517-6161.1994.tb01956.x) [DOI] [Google Scholar]
  • 41.Nie L, Ročková V. 2023. Bayesian bootstrap spike-and-slab LASSO. J. Am. Stat. Assoc. In press. ( 10.1080/01621459.2022.2025815) [DOI] [Google Scholar]
  • 42.Efron B. 2012. Bayesian inference and the parametric bootstrap. Ann. Appl. Probab. 6, 1971-1997. ( 10.1214/12-AOAS571) [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Williams D. 1991. Probability with martingales. Cambridge, UK: Cambridge University Press. [Google Scholar]
  • 44.Billingsley P. 1999. Convergence of probability measures. New York, NY: John Wiley & Sons, Inc. [Google Scholar]
  • 45.Murphy K. 2021. Probabilistic machine learning: an introduction. Cambridge, MA: MIT Press. [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Data Availability Statement

This article has no additional data.


Articles from Philosophical transactions. Series A, Mathematical, physical, and engineering sciences are provided here courtesy of The Royal Society

RESOURCES