Harnessing the Bethe free energy

ABSTRACT

A wide class of problems in combinatorics, computer science and physics can be described along the following lines. There are a large number of variables ranging over a finite domain that interact through constraints that each bind a few variables and either encourage or discourage certain value combinations. Examples include the k‐SAT problem or the Ising model. Such models naturally induce a Gibbs measure on the set of assignments, which is characterised by its partition function. The present paper deals with the partition function of problems where the interactions between variables and constraints are induced by a sparse random (hyper)graph. According to physics predictions, a generic recipe called the “replica symmetric cavity method” yields the correct value of the partition function if the underlying model enjoys certain properties [Krzkala et al., PNAS (2007) 10318–10323]. Guided by this conjecture, we prove general sufficient conditions for the success of the cavity method. The proofs are based on a “regularity lemma” for probability measures on sets of the form ${\Omega }^n$ for a finite Ω and a large n that may be of independent interest. © 2016 Wiley Periodicals, Inc. Random Struct. Alg., 49, 694–741, 2016

1. INTRODUCTION

Despite their simplicity, or perhaps because thereof, the first and the second moment method are the most widely used techniques in probabilistic combinatorics. Erdős employed the first moment method famously to lower‐bound the Ramsey number as well as to establish the existence of graphs of high girth and high chromatic number 28, 29. Even a half‐century on, deterministic constructions cannot hold a candle to these probabilistic results 14, 41. Moreover, the second moment method has been used to count prime factors 51 and Hamilton cycles 47 as well as to determine the two possible values of the chromatic number of a sparse random graph 3.

Yet there are quite a few problems for which the standard first and the second moment methods are too simplistic. The random k‐SAT model is a case in point. There are n Boolean variables $x_1,\dots ,x_n$ and m clauses $a_1,\dots ,a_m$, where $m=\lceil \alpha n\rceil$ (the real αn rounded up to the next integer) for some fixed $\alpha >0$. Each clause binds k variables, which are chosen independently and uniformly, and discourages them from taking precisely one of the $2^k$ possible truth value combinations. The forbidden combination is chosen uniformly and independently for each clause.

The random k‐SAT instance $\Phi ={\Phi }_k(n,m)$ gives rise to a probability measure on the set ${\{0,1\}}^n$ of all Boolean assignments naturally. Indeed, for a given parameter $\beta \ge 0$ the Gibbs measure ${\mu }_{\Phi ,\beta }$ is defined by letting

$${\mu }_{\Phi ,\beta }(\sigma )=\frac{1}{Z_{\beta }(\Phi )}{\displaystyle \prod _{i=1}^m\exp }(-\beta \mathit{1}\left\{\sigma \hspace{0.17em}\text{violates}\hspace{0.17em}{a}_i\right\})$$ (1.1)

for every assignment $\sigma \in {\{0,1\}}^n$, where

$$Z_{\beta }(\Phi )={\displaystyle \sum _{\sigma \in {\left\{0,1\right\}}^n}{\displaystyle \prod _{i=1}^m\exp }}(-\beta \mathit{1}\left\{\sigma \hspace{0.17em}\text{violates}\hspace{0.17em}{a}_i\right\})$$ (1.2)

is called the partition function. Thus, the Gibbs measure weighs assignments according to the number of clauses that they violate. In effect, by tuning β we can interpolate between just the uniform distribution on ${\{0,1\}}^n$ (β = 0) and a measure that strongly favours satisfying assignments ($\beta \to \infty$). Hence, if we think of $\Phi$ as inducing a “height function” $\sigma \mapsto \#\{\text{clauses}\hspace{0.17em}\text{of}\hspace{0.17em}$ $\Phi$ violated by $\sigma \}$ on the set of assignments, then varying β allows us to explore the resulting landscape. Apart from its intrinsic combinatorial interest, the shape of the height function, the so‐called “Hamiltonian”, governs the performance of algorithms such as the Metropolis process or Simulated Annealing.

To understand the Gibbs measure it is key to get a handle on the partition function $Z_{\beta }(\Phi )$. Of course, the default approach to this kind of problem would be to apply the first and second moment methods. However, upon closer inspection it emerges that $Z_{\beta }(\Phi )<\exp (-\Omega (n))\mathrm{E}[Z_{\beta }(\Phi )]$ with high probability for any $\alpha ,\beta >0$ 5. In other words, the first moment over‐estimates the partition function of a typical random formula by an exponential factor. The reason for this is a “lottery effect”: a tiny minority of formulas render an exceptionally high contribution to $\mathrm{E}[Z_{\beta }(\Phi )]$. Unsurprisingly, going to the second moment only exacerbates the problem and thus for any $\alpha ,\beta >0$ we find $\mathrm{E}[Z_{\beta }{(\Phi )}^2]\ge \exp (\Omega (n))\mathrm{E}{[Z_{\beta }(\Phi )]}^2$. In other words, the second moment method fails rather spectacularly for all possible parameter combinations.

The first and the second moment method fall victim to similar large deviations effects in many alike “random constraint satisfaction problems”. These problems, ubiquitous in combinatorics, information theory, computer science and physics 4, 36, 46, can be described along the following lines. A random factor graph, chosen either from a uniform distribution (like the random k‐SAT model above) or from a suitable configuration model, induces interactions between the variables and the constraints. The variables range over a fixed finite domain Ω and each constraint binds a few variables. The constraints come with “weight functions” that either encourage or discourage certain value combinations of the incident variables. Multiplying up the weight functions of all the contraints just like in ((1.1), (1.2)), we obtain the Gibbs measure and the partition function.

With the standard first and second moment method drawing a blank, we seem to be at a loss as far as calculating the partition function is concerned. However, physicists have put forward an ingenious albeit non‐rigorous alternative called the cavity method 36. This technique, which applies almost mechanically to any problem that can be described in the language of sparse random factor graphs, yields an explicit conjecture as to the value of the partition function. More specifically, the cavity method comes in several installments. In this paper, we are concerned with the simplest, so‐called “replica symmetric” version.

In one of their key papers 34 physicists hypothesized abstract conditions under which the replica symmetric cavity method yields the correct value of the partition function. The thrust of this paper is to prove corresponding rigorous results. Specifically, according to 34 the replica symmetric cavity method gives the correct answer if the Gibbs measure satisfies certain correlation decay properties. For example, the Gibbs uniqueness condition requires that under the Gibbs measure the value assigned to a variable x is asymptotically independent of the values assigned to the variables at a large distance from x in the factor graph. In Corollary 4.6 below we prove that this condition is indeed sufficient to guarantee the success of the cavity method. Additionally, Theorems 4.4 and 4.5 yield rigorous sufficient conditions in terms of substantially weaker conditions, namely a symmetry property and the non‐reconstruction property.

A key feature of the paper is that we establish these results not for specific examples but generically for a very wide class of factor graph models. Of course, stating and proving general results requires a degree of abstraction. In particular, we resort to the framework of local weak convergence of graph sequences 8, 35. This framework suits the physics predictions well, which come in terms of the “limiting tree” that describes the local structure of a large random factor graph. To be precise, the replica symmetric prediction is given by a functional called the Bethe free energy applied to an (infinite) random tree.

The principal tool to prove these results is a theorem about the structure of probability measures on sets of the form ${\Omega }^n$ for some fixed finite set Ω and a large integer n, Theorem 2.1 below. We expect that this result, which is inspired by Szemerédi's regularity lemma 49, will be of independent interest. To prove our results about random factor graphs, we combine Theorem 2.1 with the theory of local weak convergence to carry out completely generically “smart” first and second moment arguments that avoid the lottery effects that the standard arguments fall victim to.

In Section 2 we begin with the abstract results about probability measures on cubes. Subsequently, in Section 3 we set the stage by introducing the formalism of factor graphs and local weak convergence. Further, in Section 4 we state and prove the main results about Gibbs measures on random factor graphs. Finally, Section 5 contains the proof of a technical result that enables us to control the local structure of random factor graphs.

1.1. Related Work

A detailed (non‐rigorous) discussion of the cavity method can be found in 36. It is known that the replica symmetric version of the cavity method does not always yield the correct value of the partition function. For instance, in some factor graph models there occurs a “condensation phase transition” beyond which the replica symmetric prediction is off 20, 34. The more complex “1‐step replica symmetry breaking (1RSB)” version of the cavity method 37 is expected to yield the correct value of the partition function some way beyond condensation. However, another phase transition called “full replica symmetry breaking” spells doom on even the 1RSB cavity method 36.

The replica symmetric cavity method has been vindicated rigorously in various special cases. For instance, Montanari and Shah 38 proved that in the random k‐SAT model the replica symmetric prediction is correct up to the Gibbs uniqueness threshold. A similar result was obtained by Bandyopadhyay and Gamarnik 9 for graph colorings and independent sets. Furthermore, Dembo, Montanari and Sun 23 proved the replica symmetric conjecture on a class of models with specific types of constraints. A strength of 23 is that the result applies even to sequences of non‐random factor graphs under a local weak convergence assumption. But both 23, 38 are based on the “interpolation method” 30, 32, 42, which entails substantial restrictions on the types of models that can be handled. By contrast, the present proof method is based on a completely different approach centered around the abstract classification of measures on cubes that we present in Section 2.

Since the “vanilla” second moment method fails on the random k‐SAT model, more sophisticated variants have been proposed. The basic idea is to apply the second moment method not to the partition function itself but to a tweaked random variable. For instance, Achlioptas and Moore 2 applied the second moment method to NAE‐satisfying assignments, i.e., both the assignment and its binary inverse satisfy all clauses. However, the number of NAE‐satisfying assignments is exponentially smaller than the total number of satisfying assignments and thus this type of argument cannot yield the typical value of the partition function. The same is true of the more subtle random variable of Achlioptas and Peres 6. Furthermore, the work of Ding, Sly and Sun 26 that yields the precise k‐SAT threshold for large k is based on applying the second moment method to a random variable whose construction is guided by the 1RSB cavity method. Among other things, the random variable from 26 incorporates conditioning on the local structure of the factor graph, an idea that will be fundamental to our arguments as well.

The material of Section 2 of the present paper has recently been investigated from the more analytic viewpoint of the theory of graph limits 18. This leads to a general notion of limits of probability measures on discrete cubes. The article 18 also discusses the connection with the Aldous‐Hoover representation of exchangeable arrays, which has long been known to be related to the theory of graph limits, and Panchenko's notion of asymptotic Gibbs measures 7, 33, 44, 45. A further recent application of the methods of the present paper to two special classes of random factor graph models can be found in 19.

1.2. Notation

If $\mathcal{X}$ is a finite set, then we denote by $\mathcal{P}(\mathcal{X})$ the set of probability measures on $\mathcal{X}$. Moreover, ${\Vert ·\Vert }_{\text{TV}}$ signifies the total variation norm. If μ is a probability measure on a product space ${\mathcal{X}}^V$ for finite sets $\mathcal{X}$, V and $U\subset V$, then ${\mu }_{\downarrow U}\in \mathcal{P}({\mathcal{X}}^U)$ denotes the marginal distribution of μ on U. That is, if ${(x_u)}_{u\in U}\in {\mathcal{X}}^U$, then

$${\mu }_{\downarrow U}({(x_u)}_{u\in U})={\displaystyle \sum _{{(x_u)}_{u\in V\setminus U}\in {\mathcal{X}}^{V\setminus U}}\mu }({(x_v)}_{v\in V}).$$

If $U=\left\{u\right\}$ for some $u\in V$, then we briefly write ${\mu }_{\downarrow u}$ rather than ${\mu }_{\downarrow \left\{u\right\}}$. Further, if $S\subset {\mathcal{X}}^V$ is an event with $\mu (S)>0$, than $\mu [·|S]$ is the conditional distribution given S. That is, for any event $Q\subset {\mathcal{X}}^V$ we have $\mu [Q|S]=\mu [Q\cap S]/\mu [S]$.

The entropy of a probability measure $\mu \in \mathcal{P}(\mathcal{X})$ is denoted by $H(\mu )$. Thus, with the convention that $0\ln 0=0$ we have $H(\mu )=-{\displaystyle \sum _{x\in \mathcal{X}}\mu }(x)\ln \mu (x)$. Further, agreeing that $0\ln \frac{0}{0}=0$ as well, we recall that the Kullback‐Leibler divergence of $\mu ,\nu \in \mathcal{P}(\mathcal{X})$ is

$$D\left(\nu \left|\right|\mu \right)={\displaystyle \sum _{x\in \mathcal{X}}\nu }(x)\ln \frac{\nu (x)}{\mu (x)}\in [0,\infty ].$$

We are going to work with probability measures on sets ${\Omega }^n$ for a (small) finite Ω and a large integer n a lot. If $\mu \in \mathcal{P}({\Omega }^n)$, then we write ${\mathit{\sigma }}_{\mu },{\mathit{\tau }}_{\mu }$ for two independent samples from μ. Where μ is obvious from the context we just write $\mathit{\sigma },\mathit{\tau }$. Additionally, if $X(\mathit{\sigma })$ is a random variable, then ${〈X(\mathit{\sigma })〉}_{\mu }={\displaystyle \sum _{\sigma \in {\Omega }^n}\mu }(\sigma )X(\sigma )$ stands for the expectation of X with respect to μ. Further, if $\sigma \in {\Omega }^n,\hspace{0.17em}\varnothing \ne U\subset [n]$ and $\omega \in \Omega$, then we let

$$\sigma [\omega |U]=|{\sigma }^{-1}(\omega )\cap U|/|U|.$$

Thus, $\sigma [·|U]$ is a probability distribution on Ω, namely the distribution of $\sigma (x)$ for a random $x\in U$. If $U=\left\{x\right\}$ for some $x\in [n]$, then we just write $\sigma [\omega |x]$ rather than $\sigma [\omega |\left\{x\right\}]$. Clearly, $\sigma [\omega |x]=\mathit{1}\{\sigma (x)=\omega \}$.

We use the ${〈·〉}_{\mu }$ notation for averages over $\mu \in \mathcal{P}({\Omega }^n)$ to avoid confusion with averages over other, additional random quantities, for which we reserve the common symbols $\mathrm{E}[·],\hspace{0.17em}\mathrm{P}[·]$. Furthermore, we frequently work with conditional expectations. Hence, let us recall that for a probability space $(\mathcal{X},\mathcal{A},\mathrm{P})$, a random variable $X:\mathcal{X}\to ℝ$ and a σ‐algebra $\mathcal{F}\subset \mathcal{A}$ the conditional expectation $\mathrm{E}[X|\mathcal{F}]$ is a $\mathcal{F}$‐measurable random variable on $\mathcal{X}\to ℝ$ such that for every $\mathcal{F}$‐measurable event F we have $\mathrm{E}[\mathit{1}\{F\}\mathrm{E}[X|\mathcal{F}]]=\mathrm{E}[\mathit{1}\left\{F\right\}X]$. Moreover, recall that the conditional variance is defined as $\text{Var}[X|\mathcal{F}]=\mathrm{E}[X^2|\mathcal{F}]-\mathrm{E}{[X|\mathcal{F}]}^2$.

In line with the two previous paragraphs, if $Y:{\Omega }^n\to ℝ$ is a random variable, $\mu \in \mathcal{P}({\Omega }^n)$ and $\mathcal{F}$ is a σ‐algebra on ${\Omega }^n$, then we write ${〈Y|\mathcal{F}〉}_{\mu }$ for the conditional expectation, which is a $\mathcal{F}$‐measurable random variable $\sigma \in {\Omega }^n\mapsto {〈Y|\mathcal{F}〉}_{\mu }(\sigma )$. Accordingly, for an event $A\subset {\Omega }^n$ with $\mu (A)>0$ we write ${〈Y|A〉}_{\mu }={〈Y\mathit{1}\left\{A\right\}〉}_{\mu }/\mu (A)\in ℝ$ for the expectation of Y given A.

Finally, we need the Paley‐Zygmund inequality 43, Lemma 19]: if $X\ge 0$ is a random variable with a finite second moment, then

$$\mathrm{P}\left[X\ge t\mathrm{E}[X]\right]\ge {(1-t)}^2\frac{\mathrm{E}{[X]}^2}{\mathrm{E}[X^2]}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\text{for}\hspace{0.17em}\text{any}0<t<1.$$ (1.3)

2. PROBABILITY MEASURES ON THE CUBE

In this section we present a general “regularity lemma” for probability measures on sets ${\Omega }^n$ for some finite set Ω and a large integer n (Theorem 2.1 below).

2.1. Examples

Needless to say, probability distributions on sets ${\Omega }^n$ for a small finite Ω and a large integer n are ubiquitous. To get an idea of what we might hope to prove about them in general, let us look at a few examples.

The simplest case certainly is a product measure $\mu =p^{\otimes n}$ with $p\in \mathcal{P}(\Omega )$. By the Chernoff bound, for any fixed $\epsilon >0$ there is $n_0=n_0(\epsilon ,\Omega )>0$ such that for $n>n_0$ we have

$${〈{\Vert \mathit{\sigma }[·|U]-p\Vert }_{\text{TV}}〉}_{\mu }<\epsilon \hspace{0.17em}\text{for}\hspace{0.17em}\text{every}\hspace{0.17em}U\subset [n]\hspace{0.17em}\text{such}\hspace{0.17em}\text{that}\hspace{0.17em}\left|U\right|\ge \epsilon n.$$ (2.1)

In words, if we fix a large enough set U of coordinates and then choose $\mathit{\sigma }$ randomly, then with probability close to one the empirical distribution on U will be close to p.

As a twist on the previous example, let $p\in \mathcal{P}(\Omega )$, assume that n is a square and define a measure μ by letting

$$\mu ({\omega }_1,\dots ,{\omega }_n)={\displaystyle \prod _{i=0}^{\sqrt{n}-1}\left[p({\omega }_{1+i\sqrt{n}})\mathit{1}\{\forall j\in [\sqrt{n}]:{\omega }_{j+i\sqrt{n}}={\omega }_{1+i\sqrt{n}}\}\right]}.$$

In words, the coordinates come in blocks of size $\sqrt{n}$. While the values of all the coordinates in one block coincide and have distribution p, the coordinates in different blocks are independent. Although μ is not a product distribution, (2.1) is satisfied for any fixed $\epsilon >0$ and large enough n. Furthermore, if for a fixed k > 1 we choose $x_1,\dots ,x_k\in [n]$ uniformly and independently, then

$$\mathrm{E}{\Vert {\mu }_{\downarrow \{x_1,\dots ,x_k\}}-{\mu }_{\downarrow x_1}\otimes \cdots \otimes {\mu }_{\downarrow x_k}\Vert }_{\text{TV}}<\epsilon ,$$ (2.2)

provided that $n>n_1(\epsilon ,k,\Omega )$ is sufficiently large. This is because for large enough n it is unlikely that two of the randomly chosen $x_1,\dots ,x_k$ belong to the same block.

As a third example, consider the set $\Omega =\{0,1\}$ and the measure μ defined by

$${\mu }^{(0)}({\omega }_1,\dots ,{\omega }_n)={\left(\frac{1}{3}\right)}^{{\displaystyle \sum _{i=1}^n{\omega }_i}}{\left(\frac{2}{3}\right)}^{n-{\displaystyle \sum _{i=1}^n{\omega }_i}},$$

$${\mu }^{(1)}({\omega }_1,\dots ,{\omega }_n)={\left(\frac{2}{3}\right)}^{{\displaystyle \sum _{i=1}^n{\omega }_i}}{\left(\frac{1}{2}\right)}^{n-{\displaystyle \sum _{i=1}^n{\omega }_i}},\hspace{1em}\mu =\frac{1}{2}({\mu }^{(0)}+{\mu }^{(1)}).$$

All the marginals ${\mu }_{\downarrow i},\hspace{0.17em}i\in [n]$, are equal to the uniform distribution on {0, 1}. But of course the uniform distribution on ${\Omega }^n$ is a horrible approximation to μ. Indeed, by the Chernoff bound with overwhelming probability a point $({\omega }_1,\dots ,{\omega }_n)$ drawn from μ either satisfies $\frac{1}{n}{\displaystyle \sum _{i=1}^n{\omega }_i}\sim 1/3$ or $\frac{1}{n}{\displaystyle \sum _{i=1}^n{\omega }_i}\sim 2/3$. However, the conditional distribution given, say, $\frac{1}{n}{\displaystyle \sum _{i=1}^n{\omega }_i}\le 1/2$, is close to a product measure. Thus, μ induces a decomposition of ${\Omega }^n$ into two “states” $S_0=\{\frac{1}{n}{\displaystyle \sum _{i=1}^n{\omega }_i}\le 1/2\},\hspace{0.17em}{S}_1=\{\frac{1}{n}{\displaystyle \sum _{i=1}^n{\omega }_i}>1/2\}$ such that $\mu [·|S_0],\hspace{0.17em}\mu [·|S_1]$ are close to product measures.

As a final example, consider $\Omega =\{0,1\}$, assume that n is even and define $\mu \in \mathcal{P}({\Omega }^n)$ by letting

$$\mu ({\omega }_1,\dots ,{\omega }_n)={\left(\frac{1}{2}\right)}^{n/2}{\left(\frac{1}{3}\right)}^{{\displaystyle \sum _{i>n/2}{\omega }_i}}{\left(\frac{2}{3}\right)}^{n/2-{\displaystyle \sum _{i>n/2}{\omega }_i}}.$$

In words, μ is a product measure with marginal distribution $\text{Be}(1/2)$ on the first $n/2$ coordinates and $\text{Be}(1/3)$ on the other coordinates. Clearly, μ satisfies (2.1) with $p=\text{Be}(1/2)$ for sets $U\subset [n/2]$ and with $p=\text{Be}(1/3)$ for sets $U\subset [n]\setminus [n/2]$, provided that n is large.

In summary, the following picture emerges. The conditions (2.1) and (2.2) are proxies for saying that a given measure μ resembles a product measure. Furthermore, in order to obtain from a given μ measures that satisfy (2.1) or (2.2) it may be necessary to decompose the space ${\Omega }^n$ into “states” so that the conditional distributions have these properties. In addition, because different coordinates may have different marginal distributions, for (2.1) to hold it may be necessary to partition the set $[n]$ of coordinates.

2.2. Homogeneity

The main result of this section shows that by partitioning the space ${\Omega }^n$ and/or the set $[n]$ of coordinates it is always possible to “approximate” a given measure μ by measures that satisfy (2.1) for some suitable p as well as (2.2). In fact, the number of parts that we have to partition $[n]$ and ${\Omega }^n$ into is bounded only in terms of the desired accuracy but independently of n.

Let us introduce some terminology. If $V=(V_1,\dots ,V_k)$ is a partition of some set V, then we call $\#V=k$ the size of $V$. Furthermore, a partition $W=(W_1,\dots ,W_l)$ refines another partition $V=(V_1,\dots ,V_k)$ if for each $i\in [l]$ there is $j\in [k]$ such that $W_i\subset V_j$.

For $\epsilon >0$ we say that $\mu \in \mathcal{P}({\Omega }^n)$ is $\epsilon$ ‐regular on a set $U\subset [n]$ if for every subset $W\subset U$ of size $\left|W\right|\ge \epsilon \left|U\right|$ we have

$${〈{\Vert \mathit{\sigma }[·|W]-\mathit{\sigma }[·|U]\Vert }_{\text{TV}}〉}_{\mu }<\epsilon .$$

Further, μ is $\epsilon$ ‐regular with respect to a partition $V$ if there is a set $J\subset [\#\mathit{V}]$ such that ${\displaystyle \sum _{i\in [\#\mathit{V}]\setminus J}|}{V}_i|<\epsilon n$ and such that μ is $\epsilon$‐regular on V _i for all $i\in J$. Additionally, if $V$ is a partition of $[n]$ and $S$ is a partition of ${\Omega }^n$, then we say that μ is $\epsilon$ ‐homogeneous with respect to $(V,S)$ if there is a subset $I\subset [\#S]$ such that the following is true.

• 1 HM1: We have $\mu (S_i)>0$ for all $i\in I$ and ${\displaystyle \sum _{i\in [\#\mathit{S}]\setminus I}\mu }(S_i)<\epsilon$.
• 2 HM2: for all $i\in [\#S]$ and $j\in [\#V]$ we have ${\max }_{\sigma ,\sigma \prime \in S_i}{\Vert \sigma [·|V_j]-\sigma \prime [·|V_j]\Vert }_{\text{TV}}<\epsilon .$
• 3 HM3: for all $i\in I$ the measure $\mu [·|S_i]$ is $\epsilon$‐regular with respect to $V$.
• 4 HM4: μ is $\epsilon$‐regular with respect to $\mathit{V}$.

Theorem 2.1

For any

$\epsilon >0$

there exists

$N=N(\epsilon ,\Omega )>0$

such that for every n > N, any measure

$\mu \in \mathcal{P}({\Omega }^n)$

and any partition

${\mathit{V}}_0$

of

$[n]$

of size

$\#{\mathit{V}}_0\le 1/\epsilon$

the following is true. There exist a refinement

$\mathit{V}$

of

${\mathit{V}}_0$

and a partition

$\mathit{S}$

of

${\Omega }^n$

such that

$\#V+\#S\le N$

and such that μ is

$\epsilon$

‐homogeneous with respect to

$(V,S)$

Informally speaking, Theorem 2.1 shows that any probability measure $\mu \in \mathcal{P}({\Omega }^n)$ admits a partition $(V,S)$ such that the following is true. Almost the entire probability mass of μ belongs to parts S _i such that the conditional measure $\mu [·|S_i]$ is $\epsilon$‐regular w.r.t. $V$. This means that almost every coordinate $x\in [n]$ belongs to a class V _j such that for every “large” $U\subset V_j$ for $\mathit{\sigma }$ chosen from $\mu [·|S_i]$ very likely the empirical distribution $\mathit{\sigma }[·|U]$ is close to the marginal distribution ${〈\mathit{\sigma }[·|V_j]〉}_{\mu [·S_i]}$ of the entire class.

Theorem 2.1 and its proof, which we defer to Section 2.3, are inspired by Szemerédi's regularity lemma 49. Let us proceed to state a few consequences of Theorem 2.1.

A $(\epsilon ,k)$ ‐state of μ is a set $S\subset {\Omega }^n$ such that $\mu (S)>0$ and

$$\frac{1}{n^k}{\displaystyle \sum _{x_1,\dots ,x_k\in [n]}{\Vert {\mu }_{\downarrow \{x_1,\dots ,x_k\}}[·|S]-{\mu }_{\downarrow x_1}[·|S]\otimes \cdots \otimes {\mu }_{\downarrow x_k}[·|S]\Vert }_{\text{TV}}}<\epsilon .$$

In other words, if we choose $x_1,\dots ,x_k\in [n]$ independently and uniformly at random, then the expected total variation distance between the joint distribution ${\mu }_{\downarrow \{{\overrightarrow{x}}_1,\dots ,{\overrightarrow{x}}_k\}}[·|S]$ of $x_1,\dots ,x_k$ and the product ${\mu }_{\downarrow {\overrightarrow{x}}_1}[·|S]\otimes \cdots \otimes {\mu }_{\downarrow {\overrightarrow{x}}_k}[·|S]$ of the marginal distributions is small.

Corollary 2.2

For any

$\epsilon >0,\hspace{0.17em}k\ge 2$

there exists

$\eta =\eta (\epsilon ,k,\Omega )>0$

such that for every

$n>1/\eta$

any measure

$\mu \in \mathcal{P}({\Omega }^n)$

has pairwise disjoint

$(\epsilon ,k)$

‐states

$S_1,\dots ,S_N$

such that

$\mu (S_i)\ge \eta$

for all

$i\in [N]$

and

${\displaystyle \sum _{i=1}^N\mu }(S_i)\ge 1-\epsilon$

Thus, we can chop the space ${\Omega }^n$ into subsets $S_1,\dots ,S_N,\hspace{0.17em}N\le 1/\eta$, that capture almost the entire probability mass such that $\mu [·|S_i]$ “resembles a product measure” for each $i\in [N]$. We prove Corollary 2.2 in Section 2.4.

Let us call μ $(\epsilon ,k)$ ‐symmetric if $S={\Omega }^n$ itself is an $(\epsilon ,k)$‐state.

Corollary 2.3

For any $\epsilon ,k$ there exists δ such that for any $\xi >0$ there is $\eta >0$ such that for all $n>1/\eta$ and all $\mu \in \mathcal{P}({\Omega }^n)$ the following is true. If for any two $(\xi ,2)$ ‐states S₁, S₂ with $\mu (S_1),\mu (S_2)\ge \eta$ we have

$$\frac{1}{n}{\displaystyle \sum _{x\in [n]}{\Vert {\mu }_{\downarrow x}[·|S_1]-{\mu }_{\downarrow x}[·|S_2]\Vert }_{\text{TV}}}<\delta ,$$ (2.3)

then μ is $(\epsilon ,k)$ ‐symmetric.

Thus, the entire measure μ “resembles a product measure” if extensive states have similar marginal distributions. Conversely, we have the following.

Corollary 2.4

For any

$\epsilon >0,\hspace{0.17em}\eta >0$

there exists

$\delta >0$

such that for all

$n>1/\delta$

and all

$\mu \in \mathcal{P}({\Omega }^n)$

the following is true. If μ is

$(\delta ,2)$

‐symmetric, then for any S with

$\mu (S)\ge \eta$

we have

$$\frac{1}{n}{\displaystyle \sum _{x\in [n]}{\Vert {\mu }_{\downarrow x}[·|S]-{\mu }_{\downarrow x}\Vert }_{\text{TV}}}<\epsilon .$$

The proofs of Corollaries 2.3 and 2.4 can be found in Sections 2.5 and 2.6, respectively. Finally, in Section 2.7 we prove the following fact that will be useful in Section 4.

Proposition 2.5

For any $\epsilon >0$ there exist $\delta >0$ such that for large enough n the following is true. If $\mu \in \mathcal{P}({\Omega }^n)$ is $(\delta ,2)$ ‐symmetric, then $\mu \otimes \mu \in \mathcal{P}({\Omega }^n\times {\Omega }^n)$ is $(\epsilon ,2)$ ‐symmetric.

2.3. Proof of Theorem 2.1

Throughout this section we assume that n is sufficiently large. To prove Theorem 2.1 and guided by 49, we define the index of μ with respect to a partition $V$ of $[n]$ as

$${\text{ind}}_{\mu }(V)=\frac{1}{|\Omega |n}{\displaystyle \sum _{\omega \in \Omega }{\displaystyle \sum _{j\in [\#\overrightarrow{V}]}{\displaystyle \sum _{x\in V_j}{〈{(\mathit{\sigma }[\omega |x]-\mathit{\sigma }[\omega |V_j])}^2〉}_{\mu }}}}.$$

The index can be viewed as a conditional variance (cf. 50). Indeed, choose $x\in [n]$ uniformly and independently of $\mathit{\sigma }$. Furthermore, let ${\mathcal{F}}_{\mathit{V}}$ be the σ‐algebra generated by the events $\{x\in V_i\}$ for $i\in [\#\mathit{V}]$. Writing $\mathrm{E}[·]$ and $\text{Var}[·]$ for the expectation and variance with respect to the choice of $x$ only, we see that

$${\text{ind}}_{\mu }(\mathit{V})=\frac{1}{|\Omega |}{\displaystyle \sum _{\omega \in \Omega }\mathrm{E}}{〈\text{Var}[\mathit{\sigma }[\omega |\overrightarrow{x}]|{\mathcal{F}}_{\mathit{V}}]〉}_{\mu }.$$

Lemma 2.6

For any partition $V$ of $[n]$ we have ${\text{ind}}_{\mu }(V)\in [0,1]$. If $W$ is a refinement of $V$, then ${\text{ind}}_{\mu }(W)\le {\text{ind}}_{\mu }(V)$

The fact that ${\text{ind}}_{\mu }(V)\in [0,1]$ is immediate from the definition. Moreover, if $\mathit{W}$ refines $\mathit{V}$, then ${\mathcal{F}}_{\mathit{V}}\subset {\mathcal{F}}_{\mathit{W}}$. Consequently, $\mathrm{E}{〈\text{Var}[\mathit{\sigma }[\omega |x]|{\mathcal{F}}_{\mathit{W}}]〉}_{\mu }\le \mathrm{E}{〈\text{Var}[\mathit{\sigma }[\omega |x]|{\mathcal{F}}_{\mathit{V}}]〉}_{\mu }$. Averaging over $\omega \in \Omega$ yields ${\text{ind}}_{\mu }(W)\le {\text{ind}}_{\mu }(V)$.

Lemma 2.7

If $\mu \in \mathcal{P}({\Omega }^n)$ fails to be $\epsilon$ ‐regular with respect to $V$, then there is a refinement $W$ of $V$ such that $\#W\le 2\#V$ and ${\text{ind}}_{\mu }(W)\le {\text{ind}}_{\mu }(V)-{\epsilon }^4/(4|\Omega {|}^3).$

Let $\overline{J}$ be the set of all indices $j\in [\#V]$ such that there exists $U\subset V_j$ of size $\left|U\right|\ge \epsilon \left|V_j\right|$ such that

$${〈{\Vert \mathit{\sigma }[·|U]-\mathit{\sigma }[·|V_j]\Vert }_{\text{TV}}〉}_{\mu }\ge \epsilon .$$ (2.4)

Since μ fails to be $\epsilon$‐regular with respect to $\mathit{V}$ we have

$${\displaystyle \sum _{j\in \overline{J}}|}{V}_j|\ge \epsilon n.$$ (2.5)

For each $j\in \overline{J}$ pick a set $U_j\subset V_j,\hspace{0.17em}\left|U_j\right|\ge \epsilon \left|V_j\right|$ such that (2.4) is satisfied. Then there exists ${\omega }_j\in \Omega$ such that

$${〈\left|\mathit{\sigma }[{\omega }_j|U_j]-\mathit{\sigma }[{\omega }_j|V_j]\right|〉}_{\mu }\ge \epsilon /(2|\Omega |).$$ (2.6)

Let $\mathit{W}$ be the partition obtained from $\mathit{V}$ by splitting each class V _j, $j\in \overline{J}$, into the sub‐classes $U_j,V_j\setminus U_j$. Clearly, $\#\mathit{W}\le 2\#\mathit{V}$. Furthermore,

$$\begin{array}{l}{\text{ind}}_{\mu }(\mathit{V})=\frac{1}{\left|\Omega \right|}{\displaystyle \sum _{\omega \in \Omega }\mathrm{E}}{〈\text{Var}[\mathit{\sigma }[\omega |x]|{ℱ}_{\mathit{V}}]〉}_{\mu }\\ =\frac{1}{\left|\Omega \right|}{\displaystyle \sum _{\omega \in \Omega }\left(\mathrm{E}{〈\text{Var}[\mathit{\sigma }[\omega |x]|{ℱ}_{\mathit{W}}]〉}_{\mu }+\mathrm{E}{〈\text{Var}[\mathrm{E}[\mathit{\sigma }[\omega |x]|{ℱ}_{\mathit{W}}]|{ℱ}_{\mathit{V}}]〉}_{\mu }\right)}\\ ={\text{ind}}_{\mu }(\mathit{W})+\frac{1}{\left|\Omega \right|}{\displaystyle \sum _{\omega \in \Omega }\mathrm{E}}{〈\text{Var}[\mathrm{E}[\mathit{\sigma }[\omega |x]|{ℱ}_{\mathit{W}}]|{ℱ}_{\mathit{V}}]〉}_{\mu }.\end{array}$$ (2.7)

$$$$

$$$$

If $j\in \overline{J}$ then (2.6) implies that on V _j we have

$${〈\text{Var}[\mathrm{E}[\mathit{\sigma }[{\omega }_j|x]|{ℱ}_{\mathit{W}}]|{ℱ}_{\mathit{V}}]〉}_{\mu }\ge \frac{\left|U_j\right|}{\left|V_j\right|}{〈{(\mathit{\sigma }[{\omega }_j|U_j]-\mathit{\sigma }[{\omega }_j|V_j])}^2〉}_{\mu }\ge \frac{{\epsilon }^3}{4|\Omega {|}^2}.$$ (2.8)

Hence, combining (2.5) and (2.8), we find

$$\frac{1}{|\Omega |}{\displaystyle \sum _{\omega \in \Omega }\mathrm{E}}{〈\text{Var}[\mathrm{E}[\mathit{\sigma }[\omega |x]|{\mathcal{F}}_{\mathit{W}}]|{\mathcal{F}}_{\mathit{V}}]〉}_{\mu }\ge \frac{{\epsilon }^4}{4|\Omega {|}^3}.$$ (2.9)

Finally, the assertion follows from (2.7) and (2.9).

Proof of Theorem 2.1

The set $\mathcal{P}(\Omega )$ is compact. Therefore, there exists a partition $Q=(Q_1,\dots ,Q_K)$ of $\mathcal{P}(\Omega )$ into pairwise disjoint sets such that for all $i\in [K]$ and any two measures $\mu ,\mu \prime \in Q_i$ we have ${\Vert \mu -\mu \prime \Vert }_{\text{TV}}<\epsilon$.

Given any partition $W$ of $[n]$, we can construct a corresponding decomposition $S(W)$ of ${\Omega }^n$ as follows. Call $\sigma ,\sigma \prime \in {\Omega }^n$ $W$‐equivalent if for every $i\in [\#W]$ there exists $j\in [\#Q]$ such that $\sigma [·|W_i],\sigma \prime [·|W_i]\in Q_j$. Then $S(W)$ comprises of the equivalence classes.

We construct the desired partition $V$ of $[n]$ inductively, starting from any given partition $V(0)$ of size at most $1/\epsilon$. The construction stops once μ is $\epsilon$‐homogeneous with respect to $(V(t),S(V(t)))$. Assuming that this is not the case, we obtain $V(t+1)$ from $V(t)$ as follows. If μ fails to be $\epsilon$‐regular with respect to $\mathit{V}(t)$, then we let $\mathit{V}(t+1)$ be the partition promised by Lemma 2.7, which guarantees that

$$\#\mathit{V}(t+1)\le 2\#\mathit{V}(t)\hspace{1em}\hspace{0.17em}\text{and}\hspace{0.17em}\hspace{1em}{\text{ind}}_{\mu }(\mathit{V}(t+1))\le {\text{ind}}_{\mu }(V(t))-{\epsilon }^4/(4|\Omega {|}^3).$$ (2.10)

Otherwise let $S(t)=S(V(t))$ and $s(t)=\#S(t)$ for the sake of brevity. Further, let ${\mu }_{i,t}=\mu [·|S_i(t)]$ for $i\in [s(t)]$ with $\mu [S_i(t)]>0$. Moreover, let $\overline{I}(t)$ be the set of all $i\in [s(t)]$ such that $\mu [S_i(t)]>0$ and ${\mu }_{i,t}$ fails to be $\epsilon$‐regular with respect to $V(t)$. If μ fails to be $\epsilon$‐homogeneous with respect to $(V(t),S(t))$ but μ is $\epsilon$‐regular w.r.t. $\mathit{V}(t)$, then

$${\displaystyle \sum _{i\in \overline{I}(t)}\mu }[S_i(t)]\ge \epsilon .$$ (2.11)

Lemma 2.7 shows that for any $i\in \overline{I}(t)$ there exists a refinement $W(t,i)$ of $V(t)$ such that

$${\text{ind}}_{{\mu }_{i,t}}(W(t,i))\le {\text{ind}}_{{\mu }_{i,t}}(V(t))-{\epsilon }^4/(4|\Omega {|}^3).$$ (2.12)

Let $V(t+1)$ be the coarsest common refinement of all the partitions ${(W(t,i))}_{i\in \overline{I}(t)}$. Then

$$\#V(t+1)\le \#V(t)·2^{\#Q^{\#V(t)}}.$$ (2.13)

In addition, (2.12) and Lemma 2.6 imply

$${\text{ind}}_{{\mu }_{i,t}}(V(t+1))\le {\text{ind}}_{{\mu }_{i,t}}(V(t))-\mathit{1}\{i\in \overline{I}(t)\}{\epsilon }^4/(4|\Omega {|}^3).$$ (2.14)

Therefore, by (2.11), (2.14) and Bayes’ rule

$$\begin{array}{l}{\text{ind}}_{\mu }(V(t+1))=\frac{1}{n\left|\Omega \right|}{\displaystyle \sum _{\omega \in \Omega }{\displaystyle \sum _{j\in [\#V(t+1)]}{\displaystyle \sum _{x\in V_j(t+1)}{〈{(\mathit{\sigma }[\omega |x]-\mathit{\sigma }[\omega |V_j(t+1)])}^2〉}_{\mu }}}}\\ =\frac{1}{n\left|\Omega \right|}{\displaystyle \sum _{\omega ,j,x}{\displaystyle \sum _{i\in [s(t)]:\mu [S_i(t)]>0}\mu }}[S_i(t)]{〈{(\mathit{\sigma }[\omega |x]-\mathit{\sigma }[\omega |V_j(t+1)])}^2〉}_{{\mu }_{i,t}}\\ ={\displaystyle \sum _{i:\mu [S_i(t)]>0}\mu }[S_i(t)]{\text{ind}}_{{\mu }_{i,t}}(V(t+1))\\ \le -{\epsilon }^5/(4|\Omega {|}^3)+{\displaystyle \sum _{i:\mu [S_i(t)]>0}\mu }[S_i(t)]{\text{ind}}_{{\mu }_{i,t}}(V(t))\\ ={\text{ind}}_{\mu }(V(t))-{\epsilon }^5/(4|\Omega {|}^3).\end{array}$$ (2.15)

$$$$

$$$$

$$$$

$$$$

Combining (2.10), (2.15) and Lemma 2.6, we conclude that μ is $\epsilon$‐homogeneous with respect to $(V(T),S(T))$ for some $T\le 4|\Omega {|}^3/{\epsilon }^5$. Finally, (2.13) entails that $\#V(T),\#S(T)$ are bounded in terms of $\epsilon ,\Omega$ only.

2.4. Proof of Corollary 2.2

To derive Corollary 2.2 from Theorem 2.1 we use the following handy sufficient condition for $(\epsilon ,k)$‐symmetry.

Lemma 2.8

For any $k\ge 2,\hspace{0.17em}\epsilon >0$ there is $\delta =\delta (\epsilon ,k,\Omega )$ such that for large enough n the following is true. Assume that $\mu \in \mathcal{P}({\Omega }^n)$ is δ‐regular with respect to a partition $V$ and set ${\overline{\mu }}_i(·)={〈\mathit{\sigma }[·|V_i]〉}_{\mu }$ for $i\in [\#\mathit{V}]$. If

$${\displaystyle \sum _{i\in [\#\overrightarrow{V}]}\frac{\left|V_i\right|}{n}}{〈{\Vert \mathit{\sigma }[·|V_i]-{\overline{\mu }}_i\Vert }_{\text{TV}}〉}_{\mu }<\delta ,\frac{\pi }{2}$$ (2.16)

then μ is $(\epsilon ,k)$ ‐symmetric.

Choose a small $\xi =\xi (\epsilon ,k,\Omega )>0$ and a smaller $\delta =\delta (\xi )>0$. Then (2.16) implies that there is $J\subset [\#V]$ satisfying

$${\displaystyle \sum _{j\in J}|}{V}_j|\ge (1-\xi )n$$ (2.17)

such that for all $j\in J,\hspace{0.17em}U\subset V_j,\hspace{0.17em}\left|U\right|\ge \xi \left|V_j\right|$ we have

$${〈{\Vert \mathit{\sigma }[·|U]-{\overline{\mu }}_j\Vert }_{\text{TV}}〉}_{\mu }\le \xi .$$ (2.18)

In particular, we claim that (2.18) implies the following (if ξ is small enough):

$$\begin{array}{l}\forall \omega \in \Omega ,j\in J,\Sigma \subset {\Omega }^n:\mu (\Sigma )\ge {\xi }^{1/4}\Rightarrow \left|\left\{x\in V_j:\left|{〈\mathit{\sigma }[\omega |x]|\Sigma 〉}_{\mu }-{\overline{\mu }}_j(\omega )\right|>{\xi }^{1/4}\right\}\right|\\ \le {\xi }^{1/4}\left|V_j\right|.\end{array}$$ (2.19)

$$$$

Indeed, assume that ${〈\mathit{1}\{\mathit{\sigma }\in \Sigma \}〉}_{\mu }\ge {\xi }^{1/4}$ and $\left|\{x\in V_j:|{〈\mathit{\sigma }[{\omega }_0|x]|\Sigma 〉}_{\mu }-{\overline{\mu }}_j({\omega }_0)>{\xi }^{1/4}\left\}\right|\right|>{\xi }^{1/4}\left|V_j\right|$ for some ${\omega }_0\in \Omega$. Then because ${〈\mathit{\sigma }[·|x]|\Sigma 〉}_{\mu }$ is a probability measure on Ω for every x, there exists $\omega \in \Omega$ such that the set $U=\left\{x\in V_j:{〈\mathit{\sigma }[\omega |x]|\Sigma 〉}_{\mu }<{\overline{\mu }}_j(\omega )-{\xi }^{1/4}/|\Omega |\right\}$ has size $\left|U\right|>{\xi }^{1/4}\left|V_j\right|/(2|\Omega |)$. In particular, ${〈\mathit{\sigma }[\omega |U]|\Sigma 〉}_{\mu }\le {\overline{\mu }}_j(\omega )-{\xi }^{1/4}/|\Omega |.$ Therefore, by Markov's inequality

$${〈\mathit{1}\{\mathit{\sigma }[\omega |U]\ge {\overline{\mu }}_j(\omega )-{\xi }^{1/3}\left\}\right|\Sigma 〉}_{\mu }\le \frac{{\overline{\mu }}_j(\omega )-{\xi }^{1/4}/|\Omega |}{{\overline{\mu }}_j(\omega )-{\xi }^{1/3}}\le \frac{1-{\xi }^{1/4}/|\Omega |}{1-{\xi }^{1/3}}\le 1-{\xi }^{1/4}/(2|\Omega |).$$

Consequently, we obtain

$${〈{\Vert \mathit{\sigma }[·|U]-{\overline{\mu }}_j\Vert }_{\text{TV}}〉}_{\mu }\ge {\xi }^{1/3+1/4}{〈\mathit{1}\{\mathit{\sigma }\in \Sigma \}〉}_{\mu }/(2|\Omega |)\ge {\xi }^{7/8}.$$

Since $\left|U\right|>{\xi }^{1/4}\left|V_j\right|/(2|\Omega |)>\xi \left|V_j\right|$, this is a contradiction to (2.18).

Now, fix any ${\omega }_1,\dots ,{\omega }_k\in \Omega$ and let $x_1,\dots ,x_k\in [n]$ be chosen independently and uniformly at random. Let ${\Sigma }_h={\Sigma }_h(x_1,\dots ,x_h)\subset {\Omega }^n$ be the event that $\mathit{\sigma }(x_i)={\omega }_i$ for all $i\le h$. We are going to show that for $0\le h<k$,

$$\mathrm{E}\left[\mu ({\Sigma }_h)\left|{〈\mathit{\sigma }[{\omega }_{h+1}|x_{h+1}]|{\Sigma }_h〉}_{\mu }-{〈\mathit{\sigma }[{\omega }_{h+1}|x_{h+1}]〉}_{\mu }\right|\right]<{\xi }^{1/5}.$$ (2.20)

In the case h = 0 there is nothing to show. As for the inductive step, condition on $x_1,\dots ,x_h$.

• 1
  Case 1: $\mu ({\Sigma }_h)\le {\xi }^{1/4}$: regardless of the choice of ${\overrightarrow{x}}_{h+1}$ we have

  $$\mu ({\Sigma }_h)\left|{〈\mathit{\sigma }[{\omega }_{h+1}|x_{h+1}]|{\Sigma }_h〉}_{\mu }-{〈\mathit{\sigma }[{\omega }_{h+1}|x_{h+1}〉}_{\mu }\right|\le {\xi }^{1/4}.$$
• 2
  Case 2:

  $$\mu ({\Sigma }_h)>{\xi }^{1/4}$$

  : due to (2.17) with probability at least $1-2\xi$ we have $x_{h+1}\in V_j\setminus \{x_1,\dots ,x_h\}$ for some $j\in J$. Hence, (2.19) implies ${\mathrm{E}}_{{\overrightarrow{x}}_{h+1}}\left[\left|{〈\mathit{\sigma }[{\omega }_{h+1}|x_{h+1}]|{\Sigma }_h〉}_{\mu }-{〈\mathit{\sigma }[{\omega }_{h+1}|x_{h+1}]〉}_{\mu }\right|\right]\le {\xi }^{1/4}.$

Hence, (2.20) follows.

To complete the proof, we are going to show by induction on $h\in [k]$ that

$$\mathrm{E}\left|{〈{\displaystyle \prod _{i=1}^h\mathit{\sigma }}[{\omega }_i|x_i]〉}_{\mu }-{\displaystyle \prod _{i=1}^h{〈\mathit{\sigma }[{\omega }_i|x_i]〉}_{\mu }}\right|\le h{\xi }^{1/5}.$$ (2.21)

For h = 1 there is nothing to show. To proceed from h to h + 1 we use the triangle inequality to write

$$\mathrm{E}\left[\left|{〈{\displaystyle \prod _{i=1}^{h+1}\mathit{\sigma }}[{\omega }_i|x_i]〉}_{\mu }-{\displaystyle \prod _{i=1}^{h+1}{〈\mathit{\sigma }[{\omega }_i|x_i]〉}_{\mu }}\right|\right]$$

$$\hspace{1em}\le \mathrm{E}\left[\mu ({\Sigma }_h)\left|{〈\mathit{\sigma }[{\omega }_{h+1}|x_{h+1}]|{\Sigma }_h〉}_{\mu }-{〈\mathit{\sigma }[{\omega }_{h+1}|x_{h+1}]〉}_{\mu }\right|\right]$$

$$\hspace{1em}\hspace{1em}+\mathrm{E}\left[{〈\mathit{\sigma }[{\omega }_{h+1}|x_{h+1}]〉}_{\mu }\left|{〈{\displaystyle \prod _{i=1}^h\mathit{\sigma }}[{\omega }_i|x_i]〉}_{\mu }-{\displaystyle \prod _{i=1}^h{〈\mathit{\sigma }[{\omega }_i|x_i]〉}_{\mu }}\right|\right].$$

Invoking the induction hypothesis and (2.20) completes the proof.

Proof of Corollary 2.2

For a small enough $\delta =\delta (\epsilon ,k)>0$ let $(\mathit{V},\mathit{S})$ be a pair of partitions of size at most $N=N(\delta ,\Omega )$ such that μ is $\delta /2$‐homogeneous with respect to $(\mathit{V},\mathit{S})$ as guaranteed by Theorem 2.1. Let $\eta =\epsilon /(2N)$ and let J be the set of all $j\in [\#\mathit{S}]$ such that $\mu (S_j)\ge \eta$ and such that $\mu [·|S_j]$ is δ‐regular with respect to $\mathit{V}$. Then

$${\displaystyle \sum _{j\in [\#\mathit{S}]\setminus J}\mu }(S_j)\le \delta +\epsilon /2<\epsilon .$$

Furthermore, for every $j\in J$ the measure $\mu [·|S_j]$ satisfies (2.16) due to HM2. Therefore, Lemma 2.8 implies that $\mu [·|S_j]$ is $(\epsilon ,k)$‐symmetric. Consequently, the sets ${(S_j)}_{j\in J}$ are pairwise disjoint $(\epsilon ,k)$‐states with $\mu (S_j)\ge \eta$ for all $j\in J$ and ${\displaystyle \sum _{j\in J}\mu }(S_j)\ge 1-\epsilon$.

2.5. Proof of Corollary 2.3

Pick small enough $\delta =\delta (\epsilon ,k,\Omega ),\gamma =\gamma (\delta ,\xi ),\eta (\gamma )>0$. Then by Theorem 2.1 μ is γ‐homogeneous with respect to $(\mathit{V},\mathit{S})$ for partitions that satisfy $\#\mathit{V}+\#\mathit{S}\le N=N(\gamma )$. Let $J\subset [\#\mathit{S}]$ contain all j such that $\mu [·|S_j]$ is γ‐regular with respect to $\mathit{V}$ and such that $\mu (S_j)\ge \eta$. Let ${\overline{\mu }}_{i,j}={〈\mathit{\sigma }[·|V_i]〉}_{\mu [·|S_j]}$. Then by HM2 for every $j\in J$ we have

$$\frac{1}{n}{\displaystyle \sum _{i\in [\#V]}|}{V}_i|{〈{\Vert \mathit{\sigma }[·|V_i]-{\overline{\mu }}_{i,j}\Vert }_{\text{TV}}〉}_{\mu [ ·|S_j]}<3\gamma .$$

Therefore, Lemma 2.8 implies that S _j is a $(\xi ,2)$‐state. Consequently, our assumption (2.3) and the triangle inequality entail that for all $j,j\prime \in J$,

$${\displaystyle \sum _{i\in [\#\mathit{V}]}\frac{\left|V_i\right|}{n}}{\Vert {〈\mathit{\sigma }[·|V_i]〉}_{\mu [ ·|S_j]}-{〈\mathit{\sigma }[·|V_i]〉}_{\mu [·|S_{j\prime }]}\Vert }_{\text{TV}}<\delta .$$ (2.22)

Choosing η small, we can ensure that ${\displaystyle \sum _{j\cancel{\in }J}\mu }(S_j)\le \delta$. Therefore, letting ${\overline{\mu }}_i={〈\mathit{\sigma }[·|V_i]〉}_{\mu }$, we obtain from (2.22)

$$\begin{array}{l}{\displaystyle \sum _{i\in [\#\mathit{V}]}\frac{\left|V_i\right|}{n}}{〈{\Vert \mathit{\sigma }[·|V_i]-{\overline{\mu }}_i\Vert }_{\text{TV}}〉}_{\mu }\le \delta +{\displaystyle \sum _{i\in [\#\mathit{V}]}\frac{\left|V_i\right|}{n}}{\displaystyle \sum _{j\in J}\mu }(S_j){〈{\Vert \mathit{\sigma }[·|V_i]-{\overline{\mu }}_i\Vert }_{\text{TV}}〉}_{\mu [ · |S_j]}\\ \le 2\delta +{\displaystyle \sum _{i\in [\#\mathit{V}]}\frac{\left|V_i\right|}{n}}{\displaystyle \sum _{j\in J}\mu }(S_j){\Vert {〈\mathit{\sigma }[·|V_i]〉}_{\mu [ · |S_j]}-{\overline{\mu }}_i\Vert }_{\text{TV}}\\ [\text{by}\hspace{0.17em}\hspace{0.17em}\mathbf{H}\mathbf{M}\mathbf{2}]\\ \le 5\delta .\end{array}$$ (2.23)

$$$$

$$$$

$$$$

Since μ is γ‐regular and thus $5\delta$‐regular w.r.t. $\mathit{V}$ by HM4, (2.23) and Lemma 2.8 imply that μ is $(\epsilon ,k)$‐symmetric.

2.6. Proof of Corollary 2.4

Choose $\delta =\delta (\gamma ,\eta )$ small enough, assume that $S\subset {\Omega }^n$ satisfies $\mu (S)\ge \eta$ and that μ is $(\delta ,2)$‐symmetric. Assume for contradiction that

$$\frac{1}{n}{\displaystyle \sum _{x\in [n]}{\Vert {\mu }_{\downarrow x}[·|S]-{\mu }_{\downarrow x}\Vert }_{\text{TV}}}>\epsilon .$$ (2.24)

Let

$$$$

$$$$

Then (2.24) entails that $\left|W\right|\ge \epsilon n/2$. Therefore, there is $\omega \in \Omega$ such that $|W_s(\omega )|\ge \epsilon n/(4|\Omega |)$ for either $s=+1$ or s = – 1. Let $W\prime =W_s(\omega )$ for the sake of brevity. Of course, by the definition of $W\prime$,

$${\left({〈\mathit{\sigma }[\omega |W\prime ]〉}_{\mu [ · |S]}-{〈\mathit{\sigma }[\omega |W\prime ]〉}_{\mu }\right)}^2\ge \frac{{\epsilon }^2}{16|\Omega {|}^2}.$$ (2.25)

Since μ is $(\delta ,2)$‐symmetric,

$$\begin{array}{l}{〈{\left(\mathit{\sigma }[\omega |W\prime ]-{〈\mathit{\tau }[\omega |W\prime ]〉}_{\mu }\right)}^2〉}_{\mu }=\frac{1}{|W\prime {|}^2}{\displaystyle \sum _{x,y\in W\prime }\left[{〈\mathit{\sigma }[\omega |x]\mathit{\sigma }[\omega |y]〉}_{\mu }-{〈\mathit{\tau }[\omega |x]〉}_{\mu }{〈\mathit{\tau }[\omega |y]〉}_{\mu }\right]}\\ \le \frac{16\delta |\Omega {|}^2}{{\epsilon }^2}.\end{array}$$ (2.26)

$$$$

On the other hand we have

$${〈{\left(\mathit{\sigma }[\omega |W\prime ]-{〈\mathit{\tau }[\omega |W\prime ]〉}_{\mu }\right)}^2〉}_{\mu }\ge \mu (S){\left({〈\mathit{\tau }[\omega |W\prime ]〉}_{\mu [ · |S]}-{〈\mathit{\tau }[\omega |W\prime ]〉}_{\mu }\right)}^2.$$ (2.27)

Finally, plugging (2.25) and (2.26) into (2.27), we find $\frac{16\delta |\Omega {|}^2}{{\epsilon }^2}\ge \frac{\eta {\epsilon }^2}{32|\Omega {|}^2}$, which is a contradiction if δ is small enough.

2.7. Proof of Proposition 2.5

Choose small enough $\alpha =\alpha (\epsilon ,\Omega ),\hspace{0.17em}\gamma =\gamma (\alpha )>0,\hspace{0.17em}\chi =\chi (\gamma )>0$ and an even smaller $\delta =\delta (\gamma ,\chi )>0$ and assume that μ is $(\delta ,2)$‐symmetric. Suppose that μ is χ‐homogeneous with respect to a partition $(\mathit{V},\mathit{S})$ such that $\#\mathit{V}+\#\mathit{S}\le N=N(\gamma )$ as promised by Theorem 2.1. Let J be the set of all $j\in [\#\mathit{S}]$ such that $\mu (S_j)\ge {\gamma }^2/N$. Moreover, let I be the set of all $i\in [\#\mathit{V}]$ such that μ is χ‐regular on V _i and $\left|V_i\right|\ge \gamma n/N$. By Corollary 2.4 we have

$$\frac{1}{\left|V_i\right|}{\displaystyle \sum _{x\in V_i}{\Vert {\mu }_{\downarrow x}[·|S_j]-{\mu }_{\downarrow x}[·]\Vert }_{\text{TV}}}<\gamma \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\text{for}\hspace{0.17em}\text{all}\hspace{0.17em}i\in I,j\in J,$$

provided that δ is chosen small enough. Therefore, letting ${\overline{\mu }}_i={〈\mathit{\sigma }[·|V_i]〉}_{\mu }$, for all $i\in I$ we have

$${〈{\Vert \mathit{\sigma }[·|V_i]-{\overline{\mu }}_i\Vert }_{\text{TV}}〉}_{\mu }<2\gamma .$$ (2.28)

Fix some $i\in I$. We claim that $\mu \otimes \mu$ is α‐regular on V _i. Hence, let $U\subset V_i$ be a set of size $\left|U\right|\ge \alpha \left|V_i\right|$ and let

$$ℰ=\left\{{\Vert \mathit{\sigma }[·|U]-{\overline{\mu }}_i\Vert }_{\text{TV}}\le {\gamma }^{1/3}\right\}.$$

Then (2.28) implies that ${〈\mathit{1}\{\mathit{\sigma }\cancel{\in }ℰ\}〉}_{\mu }<{\gamma }^{1/3}$, because μ is γ‐regular on V _i. Now, fix some $\sigma \in ℰ$. For $\omega \in \Omega$ let $U(\sigma ,\omega )=\{x\in U:\sigma (x)=\omega \}$. Let

$$ℰ\prime (\sigma ,\omega )=\left\{{\Vert \mathit{\tau }[·|U(\sigma ,\omega )]-{\overline{\mu }}_i\Vert }_{\text{TV}}\le {\gamma }^{1/3}\right\}.$$

If $|U(\sigma ,\omega )|\ge {\gamma }^{1/2}\left|U\right|$, then due to (2.28) and γ‐regularity we obtain, by a similar token as previously, ${〈\mathit{1}\{\mathit{\tau }\notin ℰ\prime (\sigma ,\omega )\}〉}_{\mu }\le {\gamma }^{1/3}$. Consequently, the event $ℰ\prime (\sigma )$ that $ℰ\prime (\sigma ,\omega )$ occurs for all ω satisfying $|U(\sigma ,\omega )|\ge {\gamma }^{1/2}\left|U\right|$ has probability at least $1-|\Omega |{\gamma }^{1/3}$. Therefore, for any $\omega ,\omega \prime \in \Omega$ we obtain

$${〈\left|\frac{1}{\left|U\right|}{\displaystyle \sum _{x\in U}\mathit{1}}\{\mathit{\sigma }(x)=\omega \}\mathit{1}\{\mathit{\tau }(x)=\omega \prime \}-{\mu }_i(\omega ){\mu }_i(\omega \prime )\right|〉}_{\mu }$$

$$\hspace{1em}\le (|\Omega |+1){\gamma }^{1/3}+〈\left|\frac{1}{\left|U\right|}{\displaystyle \sum _{x\in U}\mathit{1}}\right\{\mathit{\sigma }(x)=\omega \left\}\mathit{1}\right\{\mathit{\tau }(x)=\omega \prime \}-{\mu }_i(\omega ){\mu }_i(\omega \prime )|$$

$$\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}|\mathit{\sigma }\in ℰ,\mathit{\tau }\in ℰ\prime (\mathit{\sigma })〉\mu$$

$$\hspace{1em}\le {\gamma }^{1/7}+{〈{\max }_{\omega :|U(\mathit{\sigma },\omega )|\ge {\gamma }^{1/2}\left|U\right|}|\mathit{\tau }[\omega \prime |U(\mathit{\sigma },\omega )]-{\mu }_i(\omega \prime )\left|\right|\mathit{\sigma }\in ℰ,\mathit{\tau }\in ℰ\prime (\mathit{\sigma })〉}_{\mu }\le {\gamma }^{1/8}.$$

Summing over all $\omega ,\omega \prime$ and choosing γ small enough, we conclude that $\mu \otimes \mu$ is α‐regular on V _i.

Finally, (2.28) implies that $\mu \otimes \mu$ satisfies

$${〈{\Vert (\mathit{\sigma }\otimes \mathit{\tau })[·|V_i]-{\overline{\mu }}_i\otimes {\overline{\mu }}_i\Vert }_{\text{TV}}〉}_{\mu \otimes \mu }<\alpha .$$

Therefore, picking α small enough, we can apply Lemma 2.8 to conclude that $\mu \otimes \mu$ is $(\epsilon ,2)$‐symmetric.

3. FACTOR GRAPHS

3.1. Examples

The aim in this section is to set up a comprehensive framework for the study of “random factor graphs” and their corresponding Gibbs measures. To get started let us ponder a few concrete examples.

In the Ising model on a graph $G=(V,E)$ the variables of the problem are just the vertices of the graph. The values available for each variable are ±1. Thus, an assignment is simply a map $\sigma :V\to \{\pm 1\}$. Moreover, each edge of G gives rise to a constraint. Specifically, given a parameter $\beta >0$ we define a weight function ψ _e corresponding to the edge $e=\{v,w\}$ by letting ${\psi }_e(\sigma )=\exp (\beta \sigma (v)\sigma (w)).$ Thus, edges $e=\{v,w\}$ give larger weight to assignments σ such that $\sigma (v)=\sigma (w)$ than in the case $\sigma (v)\ne \sigma (w)$. The corresponding partition function reads

$$Z_{\beta }(G)={\displaystyle \sum _{\sigma :V\to \{\pm 1\}}{\displaystyle \prod _{e\in E}{\psi }_e}}(\sigma )={\displaystyle \sum _{\sigma :V\to \{\pm 1\}}\exp }\left[\beta {\displaystyle \sum _{\{v,w\}\in E}\sigma }(v)\sigma (w)\right].$$

Further, the Gibbs distribution ${\mu }_{G,\beta }$ induced by G, β is the probability measure on ${\{\pm 1\}}^V$ defined by

$${\mu }_{G,\beta }(\sigma )=\frac{1}{Z_{\beta }(G)}{\displaystyle \prod _{e\in E}{\psi }_e}(\sigma )=\frac{1}{Z_{\beta }(G)}\exp \left[\beta {\displaystyle \sum _{\{v,w\}\in E}\sigma }(v)\sigma (w)\right].$$

Thus, ${\mu }_{G,\beta }$ weighs assignments according to the number of edges $e=\{v,w\}$ such that $\sigma (v)=\sigma (w)$.

The Ising model has been studied extensively in the mathematical physics literature on various classes of graphs, including and particularly random graphs. For instance, if $\mathit{G}(n,d)$ is a random regular graph of degree d on n vertices, then $Z_{\beta }(\mathit{G}(n,d))$ is known to “converge” to the value predicted by the cavity method 22. Formally, the cavity method yields a certain number $F(\beta ,d)$ such that

$$\underset{n\to \infty }{\lim }\frac{1}{n}\mathrm{E}[\ln Z_{\beta }(\mathit{G}(n,d))]=F(\beta ,d).$$ (3.1)

Because $Z_{\beta }(\mathit{G}(n,d))$ is exponential in n with high probability, the scaling applied in (3.1) is the appropriate one to obtain a finite limit. Furthermore, by Azuma's inequality $\ln Z_{\beta }(\mathit{G}(n,d))$ is concentrated about its expectation. Therefore, (3.1) implies that $\frac{1}{n}\ln Z_{\beta }(\mathit{G}(n,d))$ converges to $F(\beta ,d)$ in probability.

The Potts antiferromagnet on a graph $G=(V,E)$ can be viewed as a twist on the Ising model. In this case we look at assignments $\sigma :V\to [k]$ for some number $k\ge 3$. The weight functions associated with the edges are defined by ${\psi }_e(\sigma )=\exp (-\beta \mathit{1}\{\sigma (v)=\sigma (w)\})$ for some $\beta >0$. Thus, this time the edges prefer that the incident vertices receive different values. The Gibbs measure and the partition function read

$${\mu }_{G,\beta }(\sigma )=\frac{1}{Z_{\beta }(G)}\exp \left[-\beta {\displaystyle \sum _{\{v,w\}\in E}1}\{\sigma (v)=\sigma (w)\}\right],$$

$$Z_{\beta }(G)={\displaystyle \sum _{\sigma :V\to [k]}\exp }\left[-\beta {\displaystyle \sum _{\{v,w\}\in E}1}\{\sigma (v)=\sigma (w)\}\right].$$

While it is known that $\underset{n\to \infty }{\lim }\frac{1}{n}\mathrm{E}[\ln Z_{\beta }(\mathit{G}(n,d))]$ exists and that $\ln Z_{\beta }(\mathit{G}(n,d))$ is concentrated about its expectation 15, the precise value remains elusive for a wide range of $d,\beta$ (in contrast ferromagnetic version of the model 24). However, it is not difficult to see that for sufficiently large values of $d,\beta$ we have 12

$$\underset{n\to \infty }{\lim }\frac{1}{n}\mathrm{E}[\ln Z_{\beta }(\mathit{G}(n,d))]<\underset{n\to \infty }{\lim }\frac{1}{n}\ln \mathrm{E}[Z_{\beta }(\mathit{G}(n,d))].$$

Hence, just like in the random k‐SAT model the first moment overshoots the actual value of the partition function by an exponential factor. The Potts model is closely related to the k‐colorability problem. Indeed, if we think of the k possible values as colors, then for large β the Gibbs measure concentrates on colorings with few monochromatic edges.

As a third example let us consider the following version of the random k‐SAT model. Let $k\ge 3,\hspace{0.17em}\Delta >1$ be fixed integers, let $V_n=\{x_1,\dots ,x_n\}$ be a set of Boolean variables and let $d_n:V_n\times \{\pm 1\}\to [\Delta ]$ be a map such that

$$m={\displaystyle \sum _{x\in V_n}(d_n(}x,1)+d_n(x,-1))/k$$

is an integer. Then we let $\Phi (n,k,d_n)$ be a random k‐CNF formula with m clauses in which each variable $x\in V_n$ appears precisely $d_n(x,1)$ times as a positive literal and precisely $d_n(x,-1)$ times as a negative literal. As in Section 1, for a clause a and a truth assignment $\sigma :V\to \{0,1\}$ we let ${\psi }_a(\sigma )=\exp (-\beta \mathit{1}\{\sigma \hspace{0.17em}\text{violates}\hspace{0.17em}a\}).$ Then for a given parameter $\beta >0$ we obtain a Gibbs measure that weighs assignments by the number of clauses that they violate and a corresponding partition function $Z_{\beta }(\Phi (n,k,d_n))$, cf. ((1.1), (1.2)). Hence, for given $\beta >0,\hspace{0.17em}k\ge 3$ and degree assignments ${(d_n)}_n$ the problem of determining $\underset{n\to \infty }{\lim }\frac{1}{n}\mathrm{E}[\ln Z_{\beta }(\Phi (n,k,d_n))]$ arises. This question is anything but straightforward even in the special case that $d_n(x,\pm 1)=d_0$ is the same for all x. In 11 we show how the results of the present paper can be put to work to tackle this case.

3.2. Random Factor Graphs

The following definition encompasses a variety of concrete models.

Definition 3.1

Let $\Delta >0$ be an integer, let $\Omega ,\Theta$ be finite sets and let $\Psi =\left\{{\psi }_1,\dots ,{\psi }_l\right\}$ be a finite set of functions ${\psi }_i:{\Omega }^{h_i}\to (0,\infty )$ of arity $h_i\in [\Delta ]$. A $(\Delta ,\Omega ,\Psi ,\Theta )$ ‐model $\mathcal{M}=(V,F,d,t,{({\psi }_a)}_{a\in F})$ consists of

• 1 M1: a countable set V of variable nodes,
• 2 M2: a countable set F of constraint nodes,
• 3
  M3: a map $d:V\cup F\to [\Delta ]$ such that

  $${\displaystyle \sum _{x\in V}d}(x)={\displaystyle \sum _{a\in F}d}(a),$$ (3.2)
• 4
  M4: a map $t:C_V\cup C_F\to \Theta$, where we let

  $$C_V={\displaystyle \underset{x\in V}{\cup }\left\{x\right\}}\times [d(x)],C_F={\displaystyle \underset{a\in F}{\cup }\left\{a\right\}}\times [d(a)],$$

  such that

  $$\left|t^{-1}(\theta )\cap C_V\right|=\left|t^{-1}(\theta )\cap C_F\right|\hspace{1em}\hspace{0.17em}for\hspace{0.17em}each\hspace{0.17em}\theta \in \Theta ,$$ (3.3)
• 5 M5: a map $F\to \Psi ,\hspace{0.17em}a\mapsto {\psi }_a$ such that ${\psi }_a:{\Omega }^{d(a)}\to (0,\infty )$ for all $a\in F$

The size of the model is $\#\mathcal{M}=\left|V\right|$. Furthermore, a $\mathcal{M}$ ‐factor graph is a bijection $G:C_V\to C_F,\hspace{0.17em}(x,i)\mapsto G(x,i)$ such that $t(G(x,i))=t(x,i)$ for all $(x,i)\in C_V$

Of course, (3.2) and (3.3) require that either both quantities are infinite or both are finite.

The semantics is that Δ is the maximum degree of a factor graph. Moreover, Ω is the set of possible values that the variables of the model range over, e.g., the set $\{\pm 1\}$ in the Ising model. Further, Θ is a set of “types”. For instance, in the random k‐SAT model the types can be used to specify the signs of the literals. Additionally, $\Psi$ is a set of possible weight functions.

A model $\mathcal{M}$ comes with a set V of variable nodes and a set F of contraint nodes. The degrees of these nodes are prescribed by the map d. Just like in the “configuration model” of graphs with a given degree sequence we create d(v) “clones” of each node v. The sets C_V, C _F contain the clones of the variable and constraint nodes, respectively. Further, the map t assigns a type to each “clone” of either a constraint or variable node and each constraint node a comes with a weight function ψ _a.

A $\mathcal{M}$‐factor graph is a type‐preserving matching G of the variable and constraint clones. Let $\mathcal{G}(\mathcal{M})$ be the set of all $\mathcal{M}$‐factor graphs and write $\mathit{G}=\mathit{G}(\mathcal{M})$ for a uniformly random sample from $\mathcal{G}(\mathcal{M})$. Contracting the clones of each node, we obtain a bipartite (multi‐)graph with variable nodes V and constraint nodes F. We often identify $\mathit{G}$ with this multi‐graph. For instance, if we speak of the distance of two vertices in $\mathit{G}$ we mean the length of a shortest path in this multi‐graph.

For a clone $(x,i)\in C_V$ we denote by $\partial (G,x,i)=G(x,i)$ the clone that G matches (x, i) to. Similarly, for $(a,j)\in C_F$ we write $\partial (G,a,j)$ for the variable clone (x, i) such that $\partial (G,x,i)=(a,j)$. Moreover, for a variable x we let $\partial (G,x)=\{\partial (G,x,i):i\in [d(x)]\}$ and analogously for $a\in F$ we set $\partial (G,a)=\{\partial (G,a,j):j\in [d(a)]\}$. To economise notation we sometimes identify a clone (x, i) with the underlying variable x. For instance, if $\sigma :V\to \Omega$ is an assignment, then we take the liberty of writing $\sigma (x,i)=\sigma (x)$. Additionally, where convenient we view $\partial (G,x)$ as the set of all constraint nodes $a\in F$ such that there exist $i\in [d(x)],\hspace{0.17em}j\in [d(a)]$ such that $(a,j)=G(x,i)$. The corresponding convention applies to $\partial (G,a)$.

A $\mathcal{M}$ ‐assignment is a map $\sigma :V\to \Omega$ and we define

$${\psi }_{G,a}(\sigma )={\psi }_a(\sigma ({\partial }_G(a,1)),\dots ,\sigma ({\partial }_G(a,d(a))))\hspace{1em}\hspace{1em}\hspace{0.17em}\text{for}\hspace{0.17em}a\in F,\hspace{1em}\hspace{0.17em}\text{and}\hspace{0.17em}$$

$${\psi }_G(\sigma )={\displaystyle \prod _{a\in F}{\psi }_a}(\sigma ).$$

Further, the Gibbs distribution and the partition function of G are

$${\mu }_G(\sigma )={\psi }_G(\sigma )/Z_G,\hspace{1em}\hspace{0.17em}\text{where}\hspace{0.17em}Z(G)={\displaystyle \sum _{\sigma :V\to \Omega }{\psi }_G}(\sigma ).$$ (3.4)

We denote expectations with respect to the Gibbs measure by ${〈·〉}_G={〈·〉}_{{\mu }_G}$.

The fundamental problem that arises is the study of the random variable $\ln Z(\mathit{G})$. As mentioned in Section 1, this random variable holds the key to getting a handle the Gibbs measure and thus the combinatorics of the problem. The following proposition establishes concentration about the expectation. For two factor graphs $G,G\prime \in \mathcal{G}(\mathcal{M})$ let

$$\text{dist}(G,G\prime )=\left|\left\{(x,i)\in C_V:\partial (G,x,i)\ne \partial (G\prime ,x,i)\right\}\right|.$$ (3.5)

Proposition 3.2

For any

$\Delta ,\Omega ,\Theta ,\Psi$

there exists

$\eta =\eta (\Delta ,\Omega ,\Theta ,\Psi )>0$

such that for any

$(\Delta ,\Omega ,\Psi ,\Theta )$

‐model

$\mathcal{M}$

of size

$n=\#\mathcal{M}\ge 1/\eta$

and any

$\epsilon >0$

we have

$\mathrm{P}\left[\left|\ln Z(\mathit{G})-\mathrm{E}[\ln Z(\mathit{G})]\right|>\epsilon n\right]\le \exp (-\eta {\epsilon }^{2}n)$

There exists a number $\rho >0$ that depends on $\Delta ,\Omega ,\Psi ,\Theta$ only such that for any two factor graphs $G,G\prime \in \mathcal{G}(\mathcal{M})$ we have $|\ln Z(G)-\ln Z(G\prime )|\le \rho ·\text{dist}(G,G\prime )$. Therefore, the assertion follows from Azuma's inequality.

Thus, Proposition 3.2 reduces our task to calculating the expectation $\mathrm{E}[\ln Z(\mathit{G})]$. Generally, the standard first and second moment method do not suffice to tackle this problem because the logarithm sits inside the expectation. While, of course, Jensen's inequality guarantees that

$$\mathrm{E}[\ln Z(\mathit{G})]\le \ln \mathrm{E}[Z(\mathit{G})],$$ (3.6)

equality does not typically hold. In fact, we saw examples where $\ln \mathrm{E}[Z(\mathit{G})]-\mathrm{E}[\ln Z(\mathit{G})]$ is linear in the size $\#\mathcal{M}$ of the model already. If so, then the Paley‐Zygmund inequality (1.3) entails that $\ln (\mathrm{E}[Z{(\mathit{G})}^2]/\mathrm{E}{[Z(\mathit{G})]}^2)$ is linear in $\#\mathcal{M}$ as well, dooming the second moment method. Furthermore, even if $\mathrm{E}[\ln Z(\mathit{G})]\sim \ln \mathrm{E}[Z(\mathit{G})]$ the second moment method does not generally succeed 20. Let us now revisit the examples from Section 3.1.

Example 3.3

((the Ising model on the random d‐regular graph)). Suppose that $d\ge 2,\beta >0$. Let $\Delta =d,\hspace{0.17em}\Omega =\left\{\pm 1\right\},\hspace{0.17em}\Psi =\{\psi \}$, where $\psi :{\{\pm 1\}}^2\to (0,\infty ),\hspace{0.17em}({\sigma }_1,{\sigma }_2)\mapsto \exp (\beta {\sigma }_1{\sigma }_2)$, and set $\Theta =\left\{0\right\}$. Further, given $n\ge 1$ such that dn is even we define a $(\Delta ,\Omega ,\Psi ,\Theta )$‐model $\mathcal{M}(d,n)$ by letting $V=\{x_1,\dots ,x_n\},\hspace{0.17em}F=\{a_1,\dots ,a_{dn/2}\}$, d(x) = d for all $x\in V$, d(a) = 2 for all $a\in F$, $t(x,i)=t(f,j)=0$ for all $(x,i)\in C_V,\hspace{0.17em}(f,j)\in C_F$, and ${\psi }_a=\psi$ for all $a\in F$. Thus, all clones have the same “type” and all constraint nodes have arity two and the same weight function. Hence, the random graph $\mathit{G}(\mathcal{M})$ is obtained by matching the dn variable clones randomly to the dn constraint clones. If we simply replace the constraint nodes, which have degree two, by edges joining the two adjacent variable nodes, then the resulting random multigraph is contiguous to the uniformly random d‐regular graph on n vertices. In the model $\mathcal{M}$ (3.6) holds with (asymptotic) equality for all $d,\beta$ 22.

Example 3.4

((the Potts antiferromagnet on the random d‐regular graph)). The construction is similar to the previous example, except that $\Omega =[k]$ is the set of colors and $\psi ({\sigma }_1,{\sigma }_2)=\exp (-\beta \mathit{1}\{{\sigma }_1={\sigma }_2\})$. In this example (3.6) holds with asymptotic equality if either $d\le d_0(k)$ or $d>d_0(k)$ and $\beta \le {\beta }_0(d,k)$ for certain critical values $d_0(k),\hspace{0.17em}{\beta }_0(d,k)$. However, for sufficiently large $d,\beta$ there occurs a linear gap 12, 21.

Example 3.5

((random k‐SAT)). To capture the random k‐SAT model we let $\Delta >0$ be a maximum degree and $\Omega =\Theta =\{\pm 1\}$. Further, each $s\in {\{\pm 1\}}^k$ gives rise to a function

$${\psi }_s:{\{\pm 1\}}^k\to (0,\infty ),\hspace{1em}\hspace{1em}\sigma \mapsto \exp (-\beta \mathit{1}\{\sigma =-s\})$$

and we let $\Psi =\{{\psi }_s:s\in {\{\pm 1\}}^k\}$. The idea is that s is the “sign pattern” of a k‐clause, with $s_i=\pm 1$ indicating that the ith literal is positive/negative. Then a truth assignment σ of the k variables is satisfying unless ${\sigma }_i=-s_i$ for all i. The corresponding model $\mathcal{M}$ has a set $V=\{x_1,\dots ,x_n\}$ of Boolean variables and a set $F=\{a_1,\dots ,a_m\}$ of clauses. Moreover, the map $d:V\to [\Delta ]$ prescribes the degree of each variable, while of course each clause has degree k. Additionally, the map $t:C_V\cup C_F\to \Theta =\{\pm 1\}$ prescribes the positive/negative occurrences of the variables and the sign patterns of the clauses. Thus, a variable x occurs $\left|\right\{i\in [d(v)]:t(x,i)=\pm 1\left\}\right|$ times positively/negatively and the jth literal of a clause a is positive iff $t(a,j)=1$. Finally, the weight function of clause a is ${\psi }_{(t(a,1),\dots ,t(a,k))}$. The bound (3.6) does not generally hold with equality 5, 11.

While Definition 3.1 encompasses many problems of interest, there are two restrictions. First, because all weight functions $\psi \in \Psi$ take strictly positive values, Definition 3.1 does not allow for “hard” constraints. For instance, Definition 3.1 does not accommodate the graph coloring problem, which imposes the strict requirement that no single edge be monochromatic. However, for some purposes hard constraints can be approximated by soft ones, e.g., by choosing a very large value of β in the Potts antiferromagnet. Moreover, many of the arguments in the following sections do extend to hard constraints with a bit of care. However, the assumption that all ψ are strictly positive saves us many case distinctions as it ensures that $Z(\mathit{G})$ is strictly positive and that therefore the Gibbs measure is well‐defined.

The second restriction is that we prescribe a fixed maximum degree Δ. Thus, if we consider a sequence $\underset{\_}{\mathcal{M}}={({\mathcal{M}}_n)}_n$ of $(\Delta ,\Omega ,\Psi ,\Theta )$‐models with $\#{\mathcal{M}}_n=n$, then all factor graphs have a bounded degree. By comparison, if we choose a k‐SAT formula with n variables and $m=\alpha n/k$ clauses uniformly at random for fixed $k\ge 3,\alpha >0$, then the maximum variable degree will be of order $\ln n/\ln \ln n$. Yet this case can be approximated well by a sequence of models with a large enough maximum degree Δ. In fact, if we calculate $\mathrm{E}[\ln Z]$ for any fixed Δ, then the $\Delta \to \infty$ limit is easily seen to yield the answer in the case of uniformly random formulas. Nevertheless, the bounded degree assumption is technically convenient because it facilitates the use of local weak convergence, as we will discuss next.

Remark 3.6

For the sake of simplicity in (3.4) we definied the partition function as the sum over all $\sigma :V\to \Omega$. However, the results stated in the following carry over to the cases where Z is defined as the sum over all configurations of a subset of $\varnothing \ne {\mathcal{C}}_{\mathcal{M}}\subset {\Omega }^V$, e.g., all σ that have Hamming distance at most αn from some reference assignment σ ₀ for a fixed $\alpha >0$. Of course, in this case the Gibbs measure is defined such that its support is equal to ${\mathcal{C}}_{\mathcal{M}}$.

3.3. Local Weak Convergence

Suppose that we fix $\Delta ,\Omega ,\Psi ,\Theta$ as in Definition 3.1 and that $\underset{\_}{\mathcal{M}}={({\mathcal{M}}_n)}_n$ is a sequence of $(\Delta ,\Omega ,\Psi ,\Theta )$‐models such that ${\mathcal{M}}_n=(V_n,F_n,d_n,t_n,{({\psi }_a)}_{a\in F_n})$ has size n. Let us write $\mathit{G}=\mathit{G}({\mathcal{M}}_n)$ for the sake of brevity. According to the cavity method, $\underset{n\to \infty }{\lim }\frac{1}{n}\mathrm{E}[\ln Z(\mathit{G})]$ is determined by the “limiting local structure” of the random factor graph $\mathit{G}$. To formalise this concept, we adapt the concept of local weak convergence of graph sequences 8, 35 to our current setup, thereby generalising the approach taken in 23.

Definition 3.7

A $(\Delta ,\Omega ,\Psi ,\Theta )$ ‐template consists of a $(\Delta ,\Omega ,\Psi ,\Theta )$ ‐model $\mathcal{M}$, a connected factor graph $H\in \mathcal{G}(\mathcal{M})$ and a root r_H, which is a variable or factor node. Its size is $\#\mathcal{M}$. Moreover, two templates $H,H\prime$ with models $\mathcal{M}=(V,F,d,t,({\psi }_a))$, $\mathcal{M}\prime =(V\prime ,F\prime ,d\prime ,t\prime ,({\psi }_a\prime ))$ are isomorphic if there exists a bijection $\pi :V\cup F\to V\prime \cup F\prime$ such that

• 1 ISM1: $\pi (r_H)=r_H\prime$,
• 2 ISM2: $\pi (V)=V\prime$ and $\pi (F)=F\prime$,
• 3 ISM3: $d(v)=d\prime (\pi (v))$ for all $v\in V\cup F$,
• 4 ISM4: $t(v,i)=t\prime (\pi (v),i)$ for all $(v,i)\in C_V\cup C_F$,
• 5 ISM5: ${\psi }_a={\psi }_{\pi (a)}$ for all $a\in F$, and
• 6 ISM6: if $(v,i)\in C_V,(a,j)\in C_F$ satisfy $\partial (G,x,i)=(a,j)$, then $\partial (G\prime ,\pi (x),i)=(\pi (a),j)$

Thus, a template is, basically, a finite or countably infinite connected factor graph with a distinguished root. Moreover, an isomorphism preserves the root as well as degrees, types, weight functions and adjacencies.

Let us write $[H]$ for the isomorphism class of a template and let $\mathfrak{G}=\mathfrak{G}(\Delta ,\Omega ,\Theta ,\Psi )$ be the set of all isomorphism classes of $(\Delta ,\Omega ,\Psi ,\Theta )$‐templates. For each $[H]\in \mathfrak{G}$ and $\ell \ge 1$ let ${\partial }^{\ell }[H]$ be the isomorphism class of the template obtained by removing all vertices at a distance greater than $\ell$ from the root. We endow $\mathfrak{G}$ with the coarsest topology that makes all the functions

$$\Gamma \in \mathfrak{G}\mapsto \mathit{1}\{{\partial }^{\ell }[\Gamma ]={\partial }^{\ell }[{\Gamma }_0]\}\in \{0,1\}\hspace{1em}\hspace{1em}\hspace{0.17em}\text{for}\hspace{0.17em}\ell \ge 1,{\Gamma }_0\in \mathfrak{G}$$

continuous. Moreover, the space $\mathcal{P}(\mathfrak{G})$ of probability measures on $\mathfrak{G}$ carries the weak topology. So does the space ${\mathcal{P}}^2(\mathfrak{G})$ of probability measures on $\mathcal{P}(\mathfrak{G})$. For $\Gamma \in \mathfrak{G}$ we write ${\delta }_{\Gamma }\in \mathcal{P}(\mathfrak{G})$ for the Dirac measure that puts mass one on the single point Γ. Similarly, for $\lambda \in \mathcal{P}(\mathfrak{G})$ we let ${\delta }_{\lambda }\in {\mathcal{P}}^2(\mathfrak{G})$ be the Dirac measure on λ. Our assumption that the maximum degree is bounded by a fixed number Δ ensures that $\mathfrak{G},\hspace{0.17em}\mathcal{P}(\mathfrak{G}),\hspace{0.17em}{\mathcal{P}}^2(\mathfrak{G})$ are compact Polish spaces.

For a factor graph $G\in \mathcal{G}({\mathcal{M}}_n)$ and a variable or constraint node v we write $[G,v]$ for the isomorphism class of the connected component of v in G rooted at v. Then each factor graph $G\in \mathcal{G}({\mathcal{M}}_n)$ gives rise to the empirical distribution

$${\lambda }_G=\frac{1}{\left|V_n\right|+\left|F_n\right|}{\displaystyle \sum _{v\in V_n\cup F_n}{\delta }_{[G,v]}}\in \mathcal{P}(\mathfrak{G}).$$

We say that $\underset{\_}{\mathcal{M}}$ converges locally to $\vartheta \in \mathcal{P}(\mathfrak{G})$ if

$$\underset{n\to \infty }{\lim }\mathrm{E}[{\delta }_{{\lambda }_{\mathit{G}}}]={\delta }_{\vartheta }.$$ (3.7)

Denote a random isomorphism class chosen from the distribution $\vartheta$ by $\mathit{T}={\mathit{T}}_{\vartheta }$. Unravelling the definitions, we see that (3.7) holds iff for every integer $\ell >0$ and every $[H]\in \mathfrak{G}$ we have

$$\frac{1}{\left|V_n\right|+\left|F_n\right|}{\displaystyle \sum _{v\in V_n\cup F_n}\mathit{1}}\{{\partial }^{\ell }[\mathit{G},v]={\partial }^{\ell }[H]\}\hspace{0.17em}\stackrel{n\to \infty }{\to }\hspace{0.17em}\mathrm{P}\left[{\partial }^{\ell }{\mathit{T}}_{\vartheta }={\partial }^{\ell }[H]\right]\hspace{1em}\hspace{0.17em}\text{in}\hspace{0.17em}\text{probability}\hspace{0.17em}.$$ (3.8)

We are going to be interested in the case that $\underset{\_}{\mathcal{M}}$ converges locally to a distribution $\vartheta$ on acyclic templates. Thus, let $\mathfrak{T}$ be the set of all acyclic templates. Further, we write $\mathcal{V}$ for the set of all templates whose root is a variable node and $\mathcal{F}$ for the set of all templates whose root is a constraint node. Additionally, for a template $[H]$ we write $r_{[H]}$ for the root vertex, $d_{[H]}$ for its degree and ${\psi }_{[H]}$ for the weight function of the root vertex if $[H]\in \mathcal{F}$. Moreover, for $j\in [d_{[H]}]$ we write $[H]\uparrow j$ for the template obtained from $[H]$ by re‐rooting the template at the jth neighbor of $r_{[H]}$. (This makes sense because condition ISM6 from Definition 3.7 preserves the order of the neighbors.)

We will frequently condition on the depth‐$\ell$ neighborhoods of the random factor graph $\mathit{G}$ for some finite $\ell$. Hence, for $G,G\prime \in \mathcal{G}({\mathcal{M}}_n)$ and $\ell \ge 1$ we write $G{\cong }_{\ell }G\prime$ if ${\partial }^{\ell }[G,x]={\partial }^{\ell }[G\prime ,x]$ for all variable nodes $x\in V_n$ and ${\partial }^{\ell +1}[G,a]={\partial }^{\ell +1}[G\prime ,a]$ for all constraint nodes $a\in F_n$. Let ${\mathcal{T}}_{\ell }={\mathcal{T}}_{\ell ,{\mathcal{M}}_n}$ be the σ‐algebra on $\mathcal{G}({\mathcal{M}}_n)$ generated by the equivalence classes of the relation ${\cong }_{\ell }$. Additionally, for $G\in \mathcal{G}({\mathcal{M}}_n)$ and $\ell \ge 0$ we let

$${\lambda }_{G,\ell }=\frac{1}{\left|V_n\right|+\left|F_n\right|}\left[{\displaystyle \sum _{x\in V_n}{\delta }_{{\partial }^{\ell }[G,x]}}+{\displaystyle \sum _{a\in F_n}{\delta }_{{\partial }^{\ell +1}[G,a]}}\right]$$

be the empirical distribution of the depth‐$\ell$ neighborhood structure.

Furthermore, let

$${\mathfrak{T}}_{\ell }=\left\{{\partial }^{\ell }T:T\in \mathfrak{T}\cap \mathcal{V}\right\}\cup \left\{{\partial }^{\ell +1}T:T\in \mathfrak{T}\cap \mathcal{F}\right\}.$$

Then for a probability measure $\vartheta \in \mathcal{P}(\mathfrak{T})$ we denote by ${\vartheta }_{\ell }$ the image of $\vartheta$ under the map

$$\mathfrak{T}\to {\mathfrak{T}}_{\ell },\hspace{1em}\hspace{1em}T\mapsto \{\begin{array}{ll}{\partial }^{\ell }T\hfill & \hspace{0.17em}\text{if}\hspace{0.17em}T\in \mathfrak{T}\cap \mathcal{V},\hfill \\ {\partial }^{\ell +1}T\hfill & \hspace{0.17em}\text{if}\hspace{0.17em}T\in \mathfrak{T}\cap \mathcal{F}.\hfill \end{array}$$

Because all degrees are bounded by Δ, the set ${\mathfrak{T}}_{\ell }$ is finite for every $\ell \ge 1$. Hence, (3.8) entails that $\underset{\_}{\mathcal{M}}$ converges locally to $\vartheta \in \mathcal{P}(\mathfrak{T})$ iff

$$\underset{n\to \infty }{\lim }\mathrm{E}{\Vert {\lambda }_{\mathit{G},\ell }-{\vartheta }_{\ell }\Vert }_{\text{TV}}=0\hspace{1em}\hspace{1em}\hspace{0.17em}\text{for}\hspace{0.17em}\text{every}\hspace{0.17em}\ell \ge 1.$$ (3.9)

3.4. The Planted Distribution

While $\mathit{G}$ is chosen uniformly at random (from the configuration model), we need to consider another distribution that weighs factor graphs by their partition function. Specifically, given $\ell \ge 0$ let ${\widehat{\mathit{G}}}_{\ell }={\widehat{\mathit{G}}}_{\ell ,{\mathcal{M}}_n}$ be a random graph chosen according to the distribution

$$\mathrm{P}\left[{\widehat{\mathit{G}}}_{\ell }=G\right]=Z(G)·\mathrm{E}\left[\frac{\mathit{1}\{\mathit{G}=G\}}{\mathrm{E}[Z|{\mathcal{T}}_{\ell }]}\right]\hspace{1em}\hspace{1em}(G\in \mathcal{G}({\mathcal{M}}_n)),$$ (3.10)

which we call the planted distribution. The definition (3.10) ensures that the distribution of the “depth‐$\ell$ neighborhood structure” of ${\widehat{\mathit{G}}}_{\ell }$ coincides with that of $\mathit{G}$.

Perhaps more intuitively, the planted distribution can be described by the following experiment. First, choose a random factor graph $\mathit{G}$. Then, given $\mathit{G}$, choose the factor graph ${\widehat{\mathit{G}}}_{\ell }$ randomly such that a graph $G{\cong }_{\ell }\mathit{G}$ comes up with a probability that is proportional to Z(G). Perhaps despite appearances, the planted distribution is reasonably easy to work with in many cases. For instance, it has been employed successfully to study random k‐SAT as well as random graph or hypergraph coloring problems 1, 11, 13, 20, 26.

3.5. Short Cycles

In most cases of interest the random factor graph is unlikely to contain many short cycles, and it will be convenient for us to exploit this fact. Hence, let us call a factor graph G l‐acyclic if it does not contain a cycle of length at most l. We say that the sequence $\underset{\_}{\mathcal{M}}$ of models has high girth if for any $\ell ,l>0$ we have

$$\underset{n\to \infty }{\lim \inf }\mathrm{P}\left[\mathit{G}\text{is}l-\text{acyclic}\right]>0,\underset{n\to \infty }{\lim \inf }\mathrm{P}\left[{\widehat{\mathit{G}}}_{\ell }\text{is}l–\text{acyclic}\right]>0.$$ (3.11)

Thus, there is a non‐vanishing probability that the random factor graph $\mathit{G}$ is l‐acyclic. Moreover, short cycles do not have too heavy an impact on the partition function as the graph chosen from the planted distribution has a non‐vanishing probability of being l‐acyclic as well.

In the following, we are going to denote the event that a random factor graph is l‐acyclic by ${\mathcal{A}}_l$. Let us highlight the following consequence of the high girth condition and the construction of the planted distribution.

Proposition 3.8

Assume that $\underset{\_}{\mathcal{M}}$ is a sequence of $(\Delta ,\Omega ,\Psi ,\Theta )$ ‐models of high girth. Let $\ell \ge 1$ be an integer and suppose that $ℬ$ is an event such that $\underset{n\to \infty }{\lim }\mathrm{P}\left[{\widehat{\mathit{G}}}_{\ell }\in ℬ\right]=1$. If b is a real and $l\ge 0$ is an integer such that

$$\underset{n\to \infty }{\lim }\mathrm{P}\left[\ln \mathrm{E}[Z(\mathit{G})|{\mathcal{T}}_{\ell }]\ge bn|{\mathcal{A}}_l\right]=1,$$ (3.12)

then $\underset{n\to \infty }{\lim }\frac{1}{n}\ln \mathrm{E}\left[\mathit{1}\{ℬ\cap {\mathcal{A}}_l\}Z(\mathit{G})\right]\ge b$

Since $\underset{n\to \infty }{\lim }\mathrm{P}\left[{\widehat{\mathit{G}}}_{\ell }\in ℬ\right]=1$ the high girth condition (3.11) implies that $\underset{n\to \infty }{\lim }\mathrm{P}\left[{\widehat{\mathit{G}}}_{\ell }\in ℬ|{\mathcal{A}}_l\right]=1$ for every l. Set ${ℬ}_l={\mathcal{A}}_l\cap ℬ$. Then by the definition (3.10) of the planted distribution,

$$1-o(1)=\mathrm{P}\left[{\widehat{\mathit{G}}}_{\ell }\in ℬ|{\mathcal{A}}_l\right]={\displaystyle \sum _{G\in {ℬ}_l}Z}(G)\mathrm{E}\left[\frac{\mathit{1}\{\mathit{G}=G\}}{\mathrm{E}[Z|{\mathcal{T}}_{\ell }]}|{\mathcal{A}}_l\right]=\mathrm{E}\left[\frac{\mathit{1}\{\mathit{G}\in {ℬ}_l\}Z(\mathit{G})}{\mathrm{E}[Z|{\mathcal{T}}_{\ell }]}|{\mathcal{A}}_l\right]$$

$$=\mathrm{E}\left[\frac{\mathrm{E}[\mathit{1}\{\mathit{G}\in {ℬ}_l\left\}Z\right|{\mathcal{T}}_{\ell }]}{\mathrm{E}[Z|{\mathcal{T}}_{\ell }]}|{\mathcal{A}}_l\right].$$

Consequently, $\mathrm{P}\left[\mathrm{E}[\mathit{1}\{\mathit{G}\in {ℬ}_l\}Z]|{\mathcal{T}}_{\ell }]\ge \mathrm{E}[Z|{\mathcal{T}}_{\ell }]/2|{\mathcal{A}}_l\right]=1-o(1)$. Hence, (3.12) yields

$$\mathrm{P}\left[\ln \mathrm{E}[\mathit{1}\{\mathit{G}\in {ℬ}_l\}Z]|{\mathcal{T}}_{\ell }]\ge bn-1|{\mathcal{A}}_l\right]=1-o(1).$$

Therefore, the assertion follows from (3.11).

Remark 3.9

Strictly speaking, the first condition in (3.11) is superfluous as it is implied by the second one.

From here on out we assume that $\underset{\_}{\mathcal{M}}$ is a sequence of $(\Delta ,\Omega ,\Psi ,\Theta )$ ‐models of high girth that converges locally to $\vartheta \in \mathcal{P}(\mathfrak{T})$ and we fix $\Delta ,\Omega ,\Theta ,\Psi$ for the rest of the paper.

4. THE BETHE FREE ENERGY

In this section we present the main results of the paper. The thrust is that certain basic properties of the Gibbs measure entail an asymptotic formula for $\mathrm{E}[\ln Z(\mathit{G})]$. The results are guided by the physics predictions from 34.

4.1. An Educated Guess

The formula for $\mathrm{E}[\ln Z(\mathit{G})]$ that the cavity method predicts, the so‐called “replica symmetric solution”, comes in terms of the distribution $\vartheta$ to which $\underset{\_}{\mathcal{M}}$ converges locally. Thus, the cavity method claims that in order to calculate $\mathrm{E}[\ln Z(\mathit{G})]$ it is not necessary to deal with the mind‐boggling complexity of the random factor graph with its expansion properties, long cycles etc. Instead, it suffices to think about the random tree $\mathit{T}={\mathit{T}}_{\vartheta }$, a dramatically simpler object. The following definition will help us formalise this notion.

Definition 4.1

A marginal assignment is a measurable map

$p:\mathfrak{T}\to {\displaystyle \underset{j=1}{\overset{\Delta }{\cup }}\mathcal{P}}({\Omega }^j),\hspace{0.17em}T\mapsto p_T$

such that

• 1 MA1: $p_T\in \mathcal{P}(\Omega )$ for all $T\in \mathcal{V}$,
• 2 MA2: $p_T\in \mathcal{P}({\Omega }^{d_T})$ and $p_{T\downarrow j}=p_{T\uparrow j}$ for all $T\in \mathcal{F},j\in [d_T]$,
• 3
  MA3: For all $T\in \mathcal{F}$ we have

  $$\begin{array}{l}H(p_T)+{〈\ln {\psi }_T(\mathit{\sigma })〉}_{p_T}=\max \{H(\nu )+{〈\ln {\psi }_T(\mathit{\sigma })〉}_{\nu }:\nu \in 𝒫({\Omega }^{d_T})\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{0.17em}\mathrm{s}.\mathrm{t}.\hspace{0.17em}{\nu }_{\downarrow j}=p_{T\uparrow j}\hspace{0.17em}for\hspace{0.17em}all\hspace{0.17em}j\in [d_T]\}.\end{array}$$

  $$$$ (4.1)
  Further, the Bethe free energy of p with respect to $\vartheta$ is

  $${ℬ}_{\vartheta }(p)=\mathrm{E}\left[(1-d_{\mathit{T}})H(p_{\mathit{T}})|\mathcal{V}\right]+\frac{\mathrm{P}\left[\mathit{T}\in \mathcal{F}\right]}{\mathrm{P}\left[\mathit{T}\in \mathcal{V}\right]}\mathrm{E}\left[H(p_{\mathit{T}})+{〈\ln {\psi }_{\mathit{T}}(\mathit{\sigma })〉}_{p_{\mathit{T}}}|\mathcal{F}\right],$$ (4.2)

  where, of course, $\mathrm{E}[·],\mathrm{P}[·]\mathrm{E}[·],\mathrm{P}[·]$ refer to the choice of the random tree $\mathit{T}={\mathit{T}}_{\vartheta }$

Thus, a marginal assignment provides a probability distribution p _T on Ω for each tree whose root is a variable node. Furthermore, for trees T rooted at a contraint node p _T is a distribution on ${\Omega }^{d_T}$, which we think of as the joint distribution of the variables involved in the constraint. The distributions assigned to T rooted at a constraint node must satisfy a consistency condition: the jth marginal of p _T has to coincide with the distribution assigned to the tree $T\uparrow j$ rooted at the jth child of the root of T for every $j\in [d_T]$; of course, $T\uparrow j$ is a tree rooted at a variable node. In addition, MA3 requires that for $T\in \mathcal{F}$ the distribution p _T maximises the functional $H(\nu )+{〈{\psi }_T(\mathit{\sigma })〉}_{\nu }$ amongst all distribution ν with the same marginal distributions as p _T. Furthermore, the Bethe free energy is a functional that maps each marginal assignment p to a real number. For a detailed derivation of this formula based on physics intuition we refer to 36.

The basic idea behind Definition 4.1 is to capture the limiting distribution of the marginals of the variables of the random factor graph $\mathit{G}$. More specifically, Definition 4.1 aims to provide the “limiting object” of the following combinatorial construction: for a fixed $\ell$ take a random factor graph $\mathit{G}$ on a large enough number n of variable nodes and for each possible tree $T\in {\mathfrak{T}}_{\ell }$ record the empirical distribution of the marginals of the nodes whose neighborhood is isomorphic to T. In the simplest possible case (which here we confine ourselves to), in the limit of large $\ell ,n$ we expect to obtain distributions that satisfy MA2. That is, in the limit of large $\ell$ the empirical distribution of the marginals of variable nodes converges to a deterministic limit; going from $\ell$ to $\ell +1$ (the depth up to which constraint nodes can “see”) does not make much of a difference. Moreover, in the proof of Corollary 4.8 we are going to see that MA3 is the “correct” way of linking the constraint/variable distributions.

Given a distribution $\vartheta$ on trees, the cavity method provides a plausible recipe for constructing marginal assignments. Roughly speaking, the idea is to identify fixed points of an operator called Belief Propagation on the random infinite tree 36. However, this procedure is difficult to formalise mathematically because generally there are several Belief Propagation fixed points and model‐dependent considerations are necessary to identify the “correct” one. To keep matters as simple as possible we are therefore going to assume that a marginal assignment is given.

Remark 4.2

Because the entropy is concave, conditions MA2 and MA3 specify the distributions p _T for $T\in \mathcal{F}$ uniquely. In other words, a marginal assignment is actually determined completely by the distributions p _T for $T\in \mathcal{V}$.

For a marginal assignment p, an integer $\ell$ and a tree $T\in {\mathfrak{T}}_{\ell }\cap \mathcal{V}$ we define

$$p_{\ell ,T}=\mathrm{E}[p_{\mathit{T}}|{\partial }^{\ell }\mathit{T}=T].$$

Thus, $p_{\ell }$ is the conditional expectation of p given the first $\ell$ layers of the tree. To avoid notational hazards we let $p_T,p_{\ell ,T}$ be the uniform distribution on Ω for all $T\in \mathfrak{G}\setminus \mathcal{T}$.

Lemma 4.3

For any

$\epsilon >0$

there is

${\ell }_0>0$

such that for all

$\ell >{\ell }_0$

we have

$\mathrm{E}[{\Vert p_{\ell ,{\partial }^{\ell }\mathit{T}}-p_{\mathit{T}}\Vert }_{\text{TV}}|\mathit{T}\in \mathcal{V}]<\epsilon .$

Define an equivalence relation ${\equiv }_{\ell }$ on $\mathfrak{T}\cap \mathcal{V}$ by letting $T{\equiv }_{\ell }T\prime$ iff ${\partial }^{\ell }T={\partial }^{\ell }T\prime$. Then for any $\omega \in \Omega$ the sequence of random variables $X_{\ell }(\mathit{T})=p_{\ell ,{\partial }^{\ell }\mathit{T}}(\omega )$ is a martingale with respect to the filtration generated by the equivalence classes of ${\equiv }_{\ell }$. By the martingale convergence theorem 27, Theorem 5.7], ${(p_{\ell })}_{\ell }$ converges $\vartheta$‐almost surely to p.

Unless specified otherwise, in the rest of this section p is understood to be a marginal assignment.

4.2. Symmetry

In the terminology of Section 2, the cavity method claims that $\frac{1}{n}\mathrm{E}[\ln Z(\mathit{G})]$ converges to the Bethe free energy of a suitable marginal assignment iff

$$\underset{n\to \infty }{\lim }\mathrm{P}\left[{\mu }_{\mathit{G}}\hspace{0.17em}\text{is}\hspace{0.17em}(\epsilon ,2)-\text{symmetric}\right]=1\hspace{1em}\hspace{1em}\hspace{0.17em}\text{for}\hspace{0.17em}\text{any}\hspace{0.17em}\epsilon >0\hspace{0.17em}(\text{see}\hspace{0.17em}[34]).$$ (4.3)

This claim is, of course, based on bold non‐rigorous deliberations. Nonetheless, we aim to prove a rigorous statement that comes reasonably close.

To this end, let p be a marginal assignment. We say that $\underset{\_}{\mathcal{M}}$ is p‐symmetric if for every $\epsilon >0$ there is ${\ell }_0>0$ such that for all $\ell >{\ell }_0$ we have

$$\underset{n\to \infty }{\lim }\hspace{0.17em}\mathrm{P}\left[\frac{1}{n^2}{\displaystyle \sum _{x,y\in V_n}{\Vert {\mu }_{\mathit{G}\downarrow \{x,y\}}-p_{\ell ,{\partial }^{\ell }[\mathit{G},x]}\otimes p_{\ell ,{\partial }^{\ell }[\mathit{G},y]}\Vert }_{\text{TV}}}>\epsilon \right]=0.$$ (4.4)

In other words, for any $\epsilon >0$ for $\ell$ sufficiently large the random factor graph $\mathit{G}$ enjoys the following property with high probability. If we pick two variable nodes x, y of $\mathit{G}$ uniformly and independently, then the joint distribution ${\mu }_{\mathit{G}\downarrow \{x,y\}}$ is close to the product distribution $p_{\ell ,{\partial }^{\ell }[\mathit{G},x]}\otimes p_{\ell ,{\partial }^{\ell }[\mathit{G},y]}$ determined by the depth‐$\ell$ neighborhoods of x, y. Of course, as $\mathit{G}$ has bounded maximum degree the distance between randomly chosen x, y is going to be greater than, say, $\ln \ln n$ with high probability. Thus, similar in spirit to (4.3), (4.4) provides that far‐apart variables typically decorrelate and that p captures the Gibbs marginals.

In analogy to (4.4), we say that the planted distribution of $\underset{\_}{\mathcal{M}}$ is p‐symmetric if for every $\epsilon >0$ there is ${\ell }_0>0$ such that for all $\ell >{\ell }_0$ we have

$$\underset{n\to \infty }{\lim }\hspace{0.17em}\mathrm{P}\left[\frac{1}{n^2}{\displaystyle \sum _{x,y\in V_n}{\Vert {\mu }_{{\widehat{\mathit{G}}}_{\ell }\downarrow \{x,y\}}-p_{\ell ,{\partial }^{\ell }[{\widehat{\mathit{G}}}_{\ell },x]}\otimes p_{\ell ,{\partial }^{\ell }[{\widehat{\mathit{G}}}_{\ell },y]}\Vert }_{\text{TV}}}>\epsilon \right]=0\hspace{1em}\hspace{1em}\hspace{0.17em}\text{for}\hspace{0.17em}\text{any}\hspace{0.17em}\epsilon >0.$$

The main result of this paper is

Theorem 4.4

If

$\underset{\_}{\mathcal{M}}$

is p‐symmetric, then

$$\underset{n\to \infty }{\lim \sup }\frac{1}{n}\mathrm{E}[\ln Z(\mathit{G})]\le {ℬ}_{\vartheta }(p).$$

If the planted distribution of

$\underset{\_}{\mathcal{M}}$

is p‐symmetric as well, then

$$\underset{n\to \infty }{\lim }\frac{1}{n}\mathrm{E}[\ln Z(\mathit{G})]={ℬ}_{\vartheta }(p).$$

Thus, the basic symmetry assumption (4.4) implies that ${ℬ}_{\vartheta }(p)$ is an upper bound on $\frac{1}{n}\mathrm{E}[\ln Z(\mathit{G})]$. If, additionally, the symmetry condition holds in the planted model, then this upper bound is tight. In particular, in this case $\frac{1}{n}\mathrm{E}[\ln Z(\mathit{G})]$ is completely determined by the limiting local structure $\vartheta$ and p.

The proof of Theorem 4.4, which can be found in Section 4.6, is based on Theorem 2.1, the decomposition theorem for probability measures on cubes. More precisely, we combine Theorem 2.1 with a conditional first and a second moment argument given the local structure of the factor graph, i.e., given ${\mathcal{T}}_{\ell }$ for a large $\ell$. The fact that it is necessary to condition on the local structure in order to cope with “lottery effects” has been noticed in prior work 6, 17, 22, 23. Most prominently, such a conditioning was crucial in order to obtain the precise k‐SAT threshold for large enough k 26. But here the key insight is that Theorem 2.1 enables us to carry out conditional moment calculations in a fairly elegant and generic way.

The obvious question that arises from Theorem 4.4 is whether there is a simple way to show that $\underset{\_}{\mathcal{M}}$ is p‐symmetric (and that the same is true of the planted distribution). In Sections 4.3 and 4.4 we provide two sufficient conditions called non‐reconstruction and Gibbs uniqueness. That these two conditions entail symmetry was predicted in 34, and Theorem 2.1 enables us to prove it.

While the present paper deals with a very general class of factor graphs, the methods give somewhat stronger results in special classes, e.g., models with only one type and variable all nodes of the same degrees or variable nodes with Poisson degrees. The details have been worked out in 18, 19.

4.3. Non‐reconstruction

Following 34 we define a correlation decay condition, the “non‐reconstruction” condition, on factor graphs and show that it implies symmetry. The basic idea is to formalise the following. Given $\epsilon >0$ pick a large $\ell =\ell (\epsilon )>1$, choose a random factor graph $\mathit{G}$ for some large n and pick a variable node x uniformly at random. Further, sample an assignment $\mathit{\sigma }$ randomly from the Gibbs measure ${\mu }_{\mathit{G}}$. Now, sample a second assignment $\mathit{\tau }$ from ${\mu }_{\mathit{G}}$ subject to the condition that $\mathit{\tau }(y)=\mathit{\sigma }(y)$ for all variable nodes y at distance at least $\ell$ from x. Then the non‐reconstruction condition asks whether the distribution of $\mathit{\tau }(x)$ is markedly different from the unconditional marginal ${\mu }_{\mathit{G}\downarrow x}$. More precisely, non‐reconstruction occurs if for any $\epsilon$ there is $\ell (\epsilon )$ such that with high probability $\mathit{G}$ is such that the shift that a random “boundary condition” $\mathit{\sigma }$ induces does not exceed $\epsilon$ in total variation distance.

Of course, instead of conditioning on the values of all variables at distance at least $\ell$ from x, we might as well just condition on the variables at distance either $\ell$ or $\ell +1$ from x, depending on the parity of $\ell$. This is immediate from the definition (3.4) of the Gibbs measure.

As for the formal definition, suppose that $G\in \mathcal{G}({\mathcal{M}}_n)$ is a factor graph, let $x\in V_n$ and let $\ell \ge 1$. Let ${\nabla }_{\ell }(G,x)$ signify the σ‐algebra on ${\Omega }^n$ generated by the events $\mathit{1}\{\mathit{\sigma }(y)=\omega \}$ for $\omega \in \Omega$ and $y\in V_n$ at distance either $\ell$ or $\ell +1$ from x. Thus, ${\nabla }_{\ell }(G,x)$ pins down all $\mathit{\sigma }(y)$ for y at distance $\ell$ from x if $\ell$ is even and $\ell +1$ otherwise. Then we say that $\underset{\_}{\mathcal{M}}$ has non‐reconstruction with respect to a marginal assignment p if for any $\epsilon >0$ there is $\ell >0$ such that

$$\underset{n\to \infty }{\lim }\mathrm{P}\left[\frac{1}{n}{\displaystyle \sum _{x\in V_n}{〈{\Vert {〈\mathit{\tau }[·|x]|{\nabla }_{\ell }(\mathit{G},x)〉}_{\mathit{G}}-p_{\ell ,{\partial }^{\ell }[\mathit{G},x]}\Vert }_{\text{TV}}〉}_{\mathit{G}}}>\epsilon \right]=0.$$

To parse the above, the outer $\mathrm{P}[·]$ refers to the choice of $\mathit{G}$. The big ${〈·〉}_{\mathit{G}}$ is the choice of the boundary condition called $\mathit{\sigma }$ above. Finally, ${〈·|{\nabla }_{\ell }(\mathit{G},x)〉}_{\mathit{G}}$ is the random choice given the boundary condition.

Analogously, the planted distribution of $\underset{\_}{\mathcal{M}}$ has non‐reconstruction with respect to p if for any $\epsilon >0$ there exists $\ell >0$ such that

$$\underset{n\to \infty }{\lim }\mathrm{P}\left[\frac{1}{n}{\displaystyle \sum _{x\in V_n}{〈\Vert {〈\mathit{\sigma }[·|x]|{\nabla }_{\ell }({\widehat{\mathit{G}}}_{\ell },x)〉}_{{\widehat{\mathit{G}}}_{\ell }}-p_{\ell ,{\partial }^{\ell }[{\widehat{\mathit{G}}}_{\ell },x]}\Vert 〉}_{{\widehat{\mathit{G}}}_{\ell }}}>\epsilon \right]=0.$$

Theorem 4.5

If $\underset{\_}{\mathcal{M}}$ has non‐reconstruction with respect to p, then $\underset{\_}{\mathcal{M}}$ is p‐symmetric. If the planted distribution of $\underset{\_}{\mathcal{M}}$ has non‐reconstruction with respect to p, then it is p‐symmetric.

In concrete applications the non‐reconstruction condition is typically reasonably easy to verify. For instance, in 11 we determine the precise location of the so‐called “condensation phase transition” in the regular k‐SAT model via Theorems 4.4 and 4.5. The proof of Theorem 4.5 can be found in Section 4.7.

4.4. Gibbs Uniqueness

Although the non‐reconstruction condition is reasonably handy, to verify it we still need to “touch” the complex random graph $\mathit{G}$. Ideally, we might hope for a condition that can be stated solely in terms of the limiting distribution $\vartheta$ on trees, which is conceptually far more accessible. The “Gibbs uniqueness” condition as put forward in 34 fills this order.

Specifically, suppose that T is a finite acyclic template whose root r _T is a variable node. Then we say that T is $(\epsilon ,\ell )$ ‐unique with respect to a marginal assignment p if

$${\Vert {〈\mathit{\sigma }[·|r_T]|{\nabla }_{\ell }T〉}_T-p_T\Vert }_{\text{TV}}<\epsilon .$$ (4.5)

To parse (4.5), we observe that ${〈\mathit{\sigma }[·|r_T]|{\nabla }_{\ell }T〉}_T$ is a random variable, namely the average of the value $\mathit{\sigma }[·|r_T]$ assigned to the root variable under the Gibbs measure μ _T given the values of the variables at distance at least $\ell$ from r _T. Hence, (4.5) requires that ${〈\mathit{\sigma }[·|r_T]|{\nabla }_{\ell }T〉}_T$ is at total variation distance less than $\epsilon$ for every possible assignment of the variables at distance at least $\ell$ from r _T, i.e., for every “boundary condition”.

More generally, we say that $T\in \mathfrak{T}\cap \mathcal{V}$ is $(\epsilon ,\ell )$‐unique with respect to p if the finite template ${\partial }^{\ell +1}T$ has this property. (That ${\partial }^{\ell +1}T$ is finite follows once more from the fact that all degrees are bounded by Δ.) Further, we call the measure $\vartheta \in \mathcal{P}(\mathfrak{T})$ Gibbs‐unique with respect to p if for any $\epsilon >0$ we have

$$\underset{\ell \to \infty }{\lim }\mathrm{P}\left[\mathit{T}\hspace{0.17em}\text{is}\hspace{0.17em}(\epsilon ,\ell )-\text{unique}\hspace{0.17em}\mathrm{w}.\mathrm{r}.\mathrm{t}.\hspace{0.17em}p\right]=1.$$

Corollary 4.6

If

$\vartheta \in \mathcal{P}(\mathfrak{T})$

is Gibbs‐unique with respect to p, then

$\underset{n\to \infty }{\lim }\frac{1}{n}\mathrm{E}[\ln Z(\mathit{G})]={ℬ}_{\vartheta }(p)$

If $\vartheta$ is Gibbs‐unique with respect to p, then (3.9) guarantees that $\underset{\_}{\mathcal{M}}$ has non‐reconstruction with respect to p. Indeed, given $\epsilon >0,\ell >0$ and a graph G let $ℰ(G,\epsilon ,\ell )$ denote the set of vertices $x\in V_n$ for which ${\partial }^{\ell }[G,x]$ is acyclic and $(\epsilon ,\ell )$‐unique. Then we have

$$\begin{array}{l}\frac{1}{n}{\displaystyle \sum _{x\in V_n}}{〈{\Vert {〈\mathit{\sigma }[·|x]|{\nabla }_{\ell }(\mathit{G},x)〉}_{\mathit{G}}-p_{\ell ,{\partial }^{\ell }[\mathit{G},x]}\Vert }_{\text{TV}}〉}_{\mathit{G}}\le \frac{1}{n}{\displaystyle \sum _{x\in V_n}}{\Vert {\Vert {〈\mathit{\sigma }[·|x]|{\nabla }_{\ell }(\mathit{G},x)〉}_{\mathit{G}}-p_{\ell ,{\partial }^{\ell }[\mathit{G},x]}\Vert }_{\text{TV}}\Vert }_{\infty }\\ \le \epsilon +\left(1-\frac{|ℰ(\overrightarrow{G},\epsilon ,\ell )|}{n}\right),\end{array}$$

$$\le \epsilon +\left(1-\frac{|ℰ(\overrightarrow{G},\epsilon ,\ell )|}{n}\right),$$

and by (3.9) $\mathrm{P}\left[|ℰ(\overrightarrow{G},\epsilon ,\ell )|\le (1-\epsilon )n\right]$ tends to 0 as $n\to \infty$. Similarly, because the distribution of the depth‐$\ell$ neighborhood structure in the planted distribution ${\widehat{\mathit{G}}}_{\ell }$ coincides with ${\vartheta }_{\ell }$, Gibbs‐uniqueness implies that the planted model has non‐reconstruction with respect to p as well. Therefore, the assertion follows from Theorems 4.4 and 4.5.

In problems such as the random k‐SAT model, the Ising model or the Potts antiferromagnet that come with an “inverse temperature” parameter $\beta \ge 0$, Gibbs uniqueness is always satisfied for sufficiently small values of β. Consequently, Corollary 4.6 shows that the cavity method always yields the correct value of $\underset{n\to \infty }{\lim }\frac{1}{n}\mathrm{E}[\ln Z(\mathit{G})]$ in the case of small β, the so‐called “high temperature” case in physics jargon. Furthermore, if the Gibbs uniqueness condition is satisfied then there is a canonical way of constructing the marginal assignment p by means of the Belief Propagation algorithm 36, Chapter 14]. Hence, Corollary 4.6 provides a full comprehensive answer in this case.

4.5. Meet the Expectation

In this section we lay the groundwork for proving Theorem 4.4. In particular, the conditions MA1–MA3 will be used in the proofs of Corollaries 4.8 and 4.9 in this section, which will be vital to the proof of Theorem 4.4 in Section 4.6. To proceed, we need to get a handle on the conditional expectation of Z given ${\mathcal{T}}_{\ell }$ and for this purpose we need to study the possible empirical distributions of the values assigned to the variables of a concrete factor graph $G\in \mathcal{G}({\mathcal{M}}_n)$. Specifically, by a $(G,\ell )$‐marginal sequence we mean a map $q:{\mathfrak{T}}_{\ell }\to {\displaystyle \underset{j=1}{\overset{\Delta }{\cup }}\mathcal{P}}({\Omega }^j),\hspace{0.17em}T\mapsto q_T$ such that

• 1 MS1: $q_T\in \mathcal{P}(\Omega )$ if $T\in \mathcal{V}\cap {\mathfrak{T}}_{\ell }$,
• 2 MS2: $q_T\in \mathcal{P}({\Omega }^{d_T})$ if $T\in \mathcal{F}\cap {\mathfrak{T}}_{\ell }$,
• 3
  MS3: for all $T\in {\mathfrak{T}}_{\ell }\cap \mathcal{V}$ we have

  $${\displaystyle \sum _{T\prime \in {\mathfrak{T}}_{\ell }\cap \mathcal{F}}{\displaystyle \sum _{j\in [d_{T\prime }]}{\lambda }_{G,\ell }}}(T\prime )1\{{\partial }^{\ell }[T\prime \uparrow j]=T\}(q_{T\prime \downarrow j}-q_T)=0.$$ (4.6)

Thus, q assigns each tree $T\in {\mathfrak{T}}_{\ell }$ rooted at a variable node a distribution on Ω and each tree $T\in {\mathfrak{T}}_{\ell }$ rooted at a constraint node a distribution on ${\Omega }^{d_T}$, just like in Definition 4.1. Furthermore, the consistency condition (4.6) provides that for a given T rooted at a variable the average marginal distribution over all $T\prime ,j$ such that ${\partial }^{\ell }[T\prime \uparrow j]=T$ is equal to q _T. However, in contrast to condition MA2 from Definition 4.1 MS3 does not require this marginalisation to work out for every $T\prime ,j$ individually.

Suppose now that $U\subset F_n$ is a set of constraint nodes such that $d(a)=d_0$ for all $a\in U$. Then for $\sigma :V_n\to \Omega$ we let

$$\sigma [({\omega }_1,\dots ,{\omega }_{d_0})|U]=\frac{1}{\left|U\right|}{\displaystyle \sum _{a\in U}{\displaystyle \prod _{j=1}^{d_0}\mathit{1}}}\{\sigma (\partial (G,a,j))={\omega }_j)\}.$$

Thus, $\sigma [·|U]\in 𝒫({\Omega }^{d_0})$ is the empirical distribution of the sequences $\{(\sigma (\partial (G,a,1)),\dots ,\sigma (\partial (G,a,d_0))):a\in U\}$. A factor graph G and $\sigma :V_n\to \Omega$ induce a $(G,\ell )$‐marginal sequence $q_{G,\sigma ,\ell }$ canonically, namely the empirical distributions

$$q_{G,\sigma ,\ell ,T}=\sigma [·|\{x\in V_n:{\partial }^{\ell }[G,x]=T\}]\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\text{for}\hspace{0.17em}T\in {\mathcal{T}}_{\ell }\cap \mathcal{V},$$

$$q_{G,\sigma ,\ell ,T}=\sigma [·|\{a\in F_n:{\partial }^{\ell +1}[G,a]=T\}]\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\text{for}\hspace{0.17em}T\in {\mathcal{T}}_{\ell }\cap ℱ.$$

Conversely, given a $(G,\ell )$‐marginal sequence q let $\Sigma (G,\ell ,q,\delta )$ be the set of all $\sigma :V_n\to \Omega$ such that for all $T\in {\mathfrak{T}}_{\ell }\cap \mathcal{V},\hspace{0.17em}T\prime \in {\mathfrak{T}}_{\ell }\cap \mathcal{F}$ we have

$${\Vert q_{G,\sigma ,\ell ,T}-q_T\Vert }_{\text{TV}}\le \delta ,{\Vert q_{G,\sigma ,\ell ,T\prime }-q_{T\prime }\Vert }_{\text{TV}}\le \delta .$$ (4.7)

Moreover, let

$$Z_{\ell ,q,\delta }(G)=Z(G){〈\mathit{1}\{\mathit{\sigma }\in \Sigma (G,\ell ,q,\delta )\}〉}_G.$$

Finally, define

$${ℬ}_{G,\ell }(q)={\displaystyle \sum _{T\in {\mathfrak{T}}_{\ell }\cap \mathcal{V}}}(1-d_T)H(q_T){\lambda }_{G,\ell }(T|\mathcal{V})+\frac{\left|F_n\right|}{\left|V_n\right|}{\displaystyle \sum _{T\in {\mathfrak{T}}_{\ell }\cap \mathcal{F}}}\left[H(q_T)+{〈\ln {\psi }_T(\mathit{\sigma })〉}_{q_T}\right]{\lambda }_{G,\ell }(T|\mathcal{F}).$$

In Section 5 we are going to prove the following formula for the expectation of $Z_{\ell ,q,\delta }(G)$.

Proposition 4.7

For any

$\epsilon >0,\hspace{0.17em}\ell >0$

there is

$\delta >0$

such that for large enough n the following is true. Assume that

$G\in \mathcal{G}({\mathcal{M}}_n)$

is

$100\ell$

‐acyclic and let q be a

$(G,\ell )$

‐marginal sequence. Then

$$\left|n^{-1}\ln \mathrm{E}[\mathit{1}\{{\mathcal{A}}_{2\ell +5}\}{Z}_{\ell ,q,\delta }(\mathit{G})|\mathit{G}{\cong }_{\ell }G]-{ℬ}_{G,\ell }(q)\right|<\epsilon .$$

We are going to be particularly interested in the expectation of $Z_{\ell ,q,\delta }(\mathit{G})$ for q “close” to a specific marginal assignment p (in the sense of Definition 4.1). Formally, a $(G,\ell )$‐marginal sequence q is $(\epsilon ,\ell )$‐judicious with respect to p if

$${\displaystyle \sum _{T\in {\mathfrak{T}}_{\ell }\cap \mathcal{V}}{\lambda }_{G,\ell }}[T|\mathcal{V}]{\Vert q_T-p_{\ell ,T}\Vert }_{\text{TV}}+{\displaystyle \sum _{T\in {\mathfrak{T}}_{\ell }\cap \mathcal{F}}{\displaystyle \sum _{j\in [d_T]}{\lambda }_{G,\ell }}}[T|\mathcal{F}]{\Vert q_{T\downarrow j}-p_{\ell ,{\partial }^{\ell }[T\uparrow j]}\Vert }_{\text{TV}}<\epsilon .$$

We say that $(G,\sigma )$ is $(\epsilon ,\ell )$‐judicious with respect to p if the empirical distribution $q_{G,\sigma ,\ell }$ is $(\epsilon ,\ell )$‐judicious w.r.t. p.

Let us explain this definition briefly. Suppose we are given a factor graph G and a certain “depth” $\ell$. The for an assignment σ we can jot the empirical distribution of the values assigned to the variables x with ${\partial }^{\ell }[G,x]\cong T$ for each tree $T\in {\mathfrak{T}}_{\ell }\cap \mathcal{V}$. The “judicious” condition essentially provides that these empirical distributions are fairly “homogeneous”. That is, if we refine our classification of variable nodes according to the depth‐$(\ell +1)$ structure ${\partial }^{\ell +1}[G,x]$, then the resulting empirical distributions are close to the coarser ones obtained at level $\ell$. Clearly, this condition is closely related to the MA2 condition from Definition 4.1.

Before we prove Theorem 4.4 we deduce two corollaries to Proposition 4.7. The first one gives an upper bound on the “judicious part” of $\mathrm{E}[Z(\mathit{G})]$. The second one yields a lower bound.

Corollary 4.8

Suppose that p is a marginal assignment. For any $\alpha >0$ there exist $\epsilon >0,\ell >0$ such that for all $0<\beta ,\gamma <\epsilon$ and all $l\ge \ell$ the following is true. Let $ℒ(\gamma ,l)$ be the event that ${\Vert {\lambda }_{\mathit{G},l}-{\vartheta }_l\Vert }_{\text{TV}}<\gamma$. Then

$$\underset{n\to \infty }{\lim \sup }\frac{1}{n}\ln \mathrm{E}\left[\mathit{1}\{\mathit{G}\in ℒ(\gamma ,l)\cap {\mathcal{A}}_{100l}\}Z(\mathit{G}){〈\mathit{1}\{(\mathit{G},\mathit{\sigma })\hspace{0.17em}\text{is}\hspace{0.17em}(\beta ,l)\text{-}judicious\hspace{0.17em}w.r.t.\hspace{0.17em}p\}〉}_{\mathit{G}}\right]$$

$$\hspace{1em}\le {ℬ}_{\vartheta }(p)+\alpha .$$

Pick a small enough $\epsilon =\epsilon (\alpha )>0$. By Lemma 4.3 there exists $\ell$ such that $\mathrm{E}[{\Vert p_{l,{\partial }^l\mathit{T}}-p_{\mathit{T}}\Vert }_{\text{TV}}|\mathcal{V}]<\epsilon$ for all $l\ge \ell$. Now, fix any $0<\beta ,\gamma <\epsilon ,\hspace{0.17em}l\ge \ell$, pick $\xi =\xi (\beta ,l)$ small enough and assume that n is big enough. Let Q(G) be the set of all (G, l)‐marginal sequences that are $(\beta ,l)$‐judicious w.r.t. p. Because ${\mathfrak{T}}_l$ is a finite set, there exists a number $N=N(\xi )$ such that for every factor graph G there is a subset $Q_{*}(G)\subset Q(G)$ of size $|Q_{*}(G)|\le N$ such that the following is true. If $(G,\sigma )$ is $(\beta ,l)$‐judicious w.r.t. p, then $\sigma \in {\displaystyle \underset{q\in Q_{*}(G)}{\cup }\Sigma }(G,l,q,\xi )$. Therefore, for all G we have

$$Z(G){〈\mathit{1}\{(G,\mathit{\sigma })\hspace{0.17em}\text{is}\hspace{0.17em}(\epsilon ,l)\text{-judicious}\hspace{0.17em}\mathrm{w}.\mathrm{r}.\mathrm{t}.\hspace{0.17em}p\}〉}_G\le N{\max }_{q\in Q(G)}{Z}_{\ell ,q,\xi }(G).$$ (4.8)

Proposition 4.7 and (4.8) imply that for ξ small enough and n large enough for any factor graph $G\in {\mathcal{A}}_{100\ell }$ there is $q^G\in Q(G)$ such that

$$\begin{array}{l}{n}^{-1}\ln \mathrm{E}[\mathit{1}\{{\mathcal{A}}_{100\ell }\}Z(\mathit{G}){〈\mathit{1}\{(\mathit{G},\mathit{\sigma })\hspace{0.17em}\text{is}\hspace{0.17em}(\epsilon ,l)\text{-judicious}\hspace{0.17em}\mathrm{w}.\mathrm{r}.\mathrm{t}.\hspace{0.17em}p\}〉}_{\mathit{G}}|\mathit{G}{\cong }_{\ell }G]\\ \hspace{1em}\le {ℬ}_{G,l}(q^G)+\alpha /2\le \alpha /2+{\displaystyle \sum _{T\in {\mathfrak{T}}_{\ell }\cap \mathcal{V}}(1-d_T)}H(q_T^G){\lambda }_G(T|\mathcal{V})\\ \hspace{1em}\hspace{1em}+\frac{\left|F_n\right|}{\left|V_n\right|}{\displaystyle \sum _{T\in {\mathcal{T}}_{\ell }\cap ℱ}\left[H(q_T^G)+{〈\ln {\psi }_T(\mathit{\sigma })〉}_{q_T^G}\right]}{\lambda }_G(T|ℱ).\end{array}$$ (4.9)

$$$$

$$$$

Further, for any $j\in [\Delta ]$ the function $\nu \in \mathcal{P}({\Omega }^j)\mapsto H(\nu )$ is uniformly continuous because $\mathcal{P}({\Omega }^j)$ is compact. By the same token, $\nu \mapsto {〈\ln \psi (\mathit{\sigma })〉}_{\nu }$ is uniformly continuous for any $\psi \in \Psi$. Consequently, if $G\in ℒ(\gamma ,l)$ for some $\gamma <\epsilon$ and ε is chosen small enough, then we obtain

$${\displaystyle \sum _{T\in {\mathfrak{T}}_{\ell }\cap \mathcal{V}}(1-d_T)}H(q_T^G){\lambda }_G(T|\mathcal{V})<\mathrm{E}[(1-d_{\mathit{T}})H(p_{\mathit{T}})|\mathcal{V}]+\alpha /4.$$ (4.10)

Similarly, because our choice of l ensures that $\mathrm{E}[{\Vert p_{l,{\partial }^l\mathit{T}}-p_{\mathit{T}}\Vert }_{\text{TV}}|\mathcal{V}]<\epsilon$ and because of the uniform degree bound Δ, condition MA2 from Definition 4.1 implies that

$$\mathrm{E}\left[{\displaystyle \sum _{j=1}^{d_{\mathit{T}}}{\Vert p_{l,{\partial }^l[\mathit{T}\uparrow j]}-p_{\mathit{T}\downarrow j}\Vert }_{\text{TV}}}|\mathcal{F}\right]<{\epsilon }^{1/4}.$$

Moreover, for any $\psi \in \Psi$ and any ${\nu }_1,\dots ,{\nu }_{d_{\psi }}\in \mathcal{P}(\Omega )$ there is a unique distribution $\widehat{\nu }\in \mathcal{P}({\Omega }^{d_{\psi }})$ with marginals ${\widehat{\nu }}_{\downarrow j}={\nu }_j$ that maximises $H(\widehat{\nu })+{〈\psi (\mathit{\sigma })〉}_{\widehat{\nu }}$ because the entropy is concanve and the map $\mu \mapsto {〈\psi (\mathit{\sigma })〉}_{\mu }$ is linear. In fact, the map $({\nu }_1,\dots ,{\nu }_{d_{\psi }})\mapsto \widehat{\nu }$ is uniformly continuous. Therefore, MA2 and MA3 show that for $G\in ℒ(\gamma ,l)$,

$$\frac{\left|F_n\right|}{\left|V_n\right|}{\displaystyle \sum _{T\in {\mathcal{T}}_{\ell }\cap \mathcal{F}}\left[H(q_T^G)+{〈\ln {\psi }_T(\mathit{\sigma })〉}_{q_T^G}\right]}{\lambda }_G(T|\mathcal{F})<\mathrm{E}[H(p_{\mathit{T}})+{〈{\psi }_{\mathit{T}}(\mathit{\sigma })〉}_{p_{\mathit{T}}}|\mathcal{F}]+\alpha /4.$$ (4.11)

Combining (4.9), (4.10) and (4.11), we get

$$\ln \mathrm{E}[1\{{\mathcal{A}}_{100\ell }\}Z(\mathit{G}){〈\mathit{1}\{(\mathit{G},\mathit{\sigma })\hspace{0.17em}\text{is}\hspace{0.17em}(\epsilon ,l)\text{-judicious}\hspace{0.17em}\mathrm{w}.\mathrm{r}.\mathrm{t}.\hspace{0.17em}p\}〉}_{\mathit{G}}|\mathit{G}{\cong }_{\ell }G]\le {ℬ}_{\vartheta }(p)+\alpha n.$$ (4.12)

Finally, the assertion follows from (4.12) and Bayes’ rule.

Corollary 4.9

Suppose that p is a marginal assignment. For any $\alpha >0$ there exists $\ell >0$ such that for all $l\ge \ell$,

$$\underset{n\to \infty }{\lim }\mathrm{P}\left[\frac{1}{n}\ln \mathrm{E}\left[Z(\mathit{G})|{\mathcal{T}}_l\right]\le {ℬ}_{\vartheta }(p)-\alpha |{\mathcal{A}}_{100l}\right]=0.$$

Choose a small $\epsilon =\epsilon (\alpha )>0$. By Lemma 4.3 there exists $\ell$ such that $\mathrm{E}[{\Vert p_{l,{\partial }^l\mathit{T}}-p_{\mathit{T}}\Vert }_{\text{TV}}|\mathcal{V}]<\epsilon$ for all $l\ge \ell$. Hence, fix some $l\ge \ell$ and define

$$q:T\in {\mathfrak{T}}_l\cap \mathcal{V}\to \mathcal{P}(\Omega ),\hspace{1em}T\mapsto p_{l,{\partial }^{l}T}.$$

Moreover, for $T\in {\mathfrak{T}}_l\cap \mathcal{F}$ let $q_T\in \mathcal{P}({\Omega }^{d_T})$ be the (unique) distribution that maximises $H(q_T)+{〈\ln {\psi }_T(\mathit{\sigma })〉}_{q_T}$ subject to the condition that $q_{T\downarrow j}=q_{{\partial }^{l}T\uparrow j}$ for all $j\in [d_T]$ (cf. (4.1)). Then q is a marginal sequence. Indeed, MS1, MS2 are trivially satisfied and MS3 holds because $q_{T\downarrow j}=q_{{\partial }^{l}T\uparrow j}$ for all $T\in {\mathfrak{T}}_l\cap \mathcal{F},\hspace{0.17em}j\in [d_T]$. Further, if we pick $\delta =\delta (\epsilon ,l)>0$ small enough, then Proposition 4.7 implies that for large n and any $G\in {\mathcal{A}}_{100l}$

$$\begin{array}{l}{n}^{-1}\ln \mathrm{E}[Z_{l,q,\delta }(\mathit{G})|\mathit{G}{\cong }_{\ell }G]\ge {ℬ}_{G,\ell }(q)-\alpha /2=-\alpha /2+{\displaystyle \sum _{T\in {\mathfrak{T}}_{\ell }\cap \mathcal{V}}(1-d_T)}H(q_T){\lambda }_{G,l}(T|\mathcal{V})\\ \hspace{1em}+\frac{\left|F_n\right|}{\left|V_n\right|}{\displaystyle \sum _{T\in {\mathcal{T}}_{\ell }\cap ℱ}\left[H(q_T)+{〈\ln {\psi }_T(\mathit{\sigma })〉}_{q_T}\right]}{\lambda }_{G,l}(T|ℱ).\end{array}$$ (4.13)

$$$$

To complete the proof, we need to compare the r.h.s. of (4.13) with ${ℬ}_{\vartheta }(p)$. Thus, let us write

$$\beta (G)={\displaystyle \sum _{T\in {\mathfrak{T}}_{\ell }\cap \mathcal{V}}(1-d_T)}H(q_T){\lambda }_{G,l}(T|\mathcal{V}),$$

$$\beta \prime (G)={\displaystyle \sum _{T\in {\mathcal{T}}_{\ell }\cap \mathcal{F}}\left[H(q_T)+{〈\ln {\psi }_T(\mathit{\sigma })〉}_{q_T}\right]}{\lambda }_{G,l}(T|\mathcal{F}).$$

Because ${\Vert {\vartheta }_l-{\lambda }_{\mathit{G},l}\Vert }_{\text{TV}}<\epsilon$ w.h.p. by (3.9), $\mathrm{E}[{\Vert p_{l,{\partial }^l\mathit{T}}-p_{\mathit{T}}\Vert }_{\text{TV}}|\mathcal{V}]<\epsilon$ by our choice of l, the entropy is a uniformly continuous on $\mathcal{P}(\Omega )$ and $d_T\le \Delta$ is uniformly bounded for all T, (3.11) ensures that we can make $\epsilon$ so small that

$$\underset{n\to \infty }{\lim }\mathrm{P}\left[\mathrm{E}\left[\left|\beta (G)-\mathrm{E}[(1-d_{\mathit{T}})H(p_{\mathit{T}})|\mathcal{V}]\right||{\mathcal{T}}_l\right]>\alpha /4|{\mathcal{A}}_{100l}\right]=0.$$ (4.14)

A similar argument applies to $\beta \prime (G)$. Indeed, since $\mathrm{E}[{\Vert p_{l,{\partial }^l\mathit{T}}-p_{\mathit{T}}\Vert }_{\text{TV}}|\mathcal{V}]<\epsilon$ and because all degrees are bounded by Δ, condition MA2 from Definition 4.1 implies that

$$\mathrm{E}\left[{\displaystyle \sum _{j=1}^{d_{\mathit{T}}}{\Vert p_{l,{\partial }^l[\mathit{T}\uparrow j]}-p_{\mathit{T}\downarrow j}\Vert }_{\text{TV}}}|\mathcal{F}\right]<{\epsilon }^{1/4}$$

provided $\epsilon$ is small enough. In effect, because for any $\psi \in \Psi$ the function $\mu \in \mathcal{P}({\Omega }^{d_{\psi }})\mapsto H(\mu )+{〈\psi (\mathit{\sigma })〉}_{\mu }$ is uniformly continuous, MA3 and the construction of q _T for $T\in {\mathfrak{T}}_l\cap \mathcal{F}$ ensure that

$$\mathrm{E}\left[\left|H(q_{l,{\partial }^{l+1}\mathit{T}})+{〈{\psi }_{\mathit{T}}(\mathit{\sigma })〉}_{q_{l,{\partial }^{l+1}\mathit{T}}}-H(p_{\mathit{T}})-{〈{\psi }_{\mathit{T}}(\mathit{\sigma })〉}_{p_{\mathit{T}}}\right||\mathcal{F}\right]<\alpha /8.$$ (4.15)

Since ${\Vert {\vartheta }_l-{\lambda }_{\mathit{G},l}\Vert }_{\text{TV}}<\epsilon$ w.h.p. by (3.9), (4.15) implies that

$$\underset{n\to \infty }{\lim }\mathrm{P}\left[\mathrm{E}\left[\left|\beta \prime (G)-\mathrm{E}[H(p_{\mathit{T}})+{〈{\psi }_{\mathit{T}}(\mathit{\sigma })〉}_{p_{\mathit{T}}}|\mathcal{F}]\right||{\mathcal{T}}_l\right]>\alpha /4|{\mathcal{A}}_{100l}\right]=0.$$ (4.16)

Finally, the assertion follows from (4.13), (4.14) and (4.16).

4.6. Proof of Theorem 4.4

We begin by spelling out the following consequence of the symmetry assumption. Let p be a marginal assignment.

Lemma 4.10

If

$\underset{\_}{\mathcal{M}}$

is p‐symmetric, then for any

$\epsilon >0$

for all sufficiently large

$\ell$

we have

$$\begin{array}{l}\underset{n\to \infty }{\lim }\mathrm{P}\left[{\displaystyle \sum _{x\in V_n}{\Vert {\mu }_{\mathit{G}\downarrow x}-p_{\ell ,{\partial }^{\ell }[\mathit{G},x]}\Vert }_{\text{TV}}}>\epsilon n\right]\\ \hspace{1em}=\underset{n\to \infty }{\lim }\mathrm{P}\left[{\mu }_{\mathit{G}}\hspace{0.17em}fails\hspace{0.17em}to\hspace{0.17em}be\hspace{0.17em}(\epsilon ,2)\text{-}symmetric\right]=0\hspace{1em}\hspace{0.17em}and\hspace{0.17em}\end{array}$$ (4.17)

$$$$

$$\begin{array}{l}\underset{n\to \infty }{\lim }\mathrm{P}\left[{\displaystyle \sum _{x\in V_n}{\Vert {\mu }_{{\widehat{\mathit{G}}}_{\ell }\downarrow x}-p_{\ell ,{\partial }^{\ell }[{\widehat{\mathit{G}}}_{\ell },x]}\Vert }_{\text{TV}}}>\epsilon n\right]\\ \hspace{1em}=\underset{n\to \infty }{\lim }\mathrm{P}\left[{\mu }_{{\widehat{\mathit{G}}}_{\ell }}\hspace{0.17em}fails\hspace{0.17em}to\hspace{0.17em}be\hspace{0.17em}(\epsilon ,2)\text{-}symmetric\right]=0.\end{array}$$ (4.18)

$$$$

Choose $\eta =\eta (\epsilon )>0$ small enough. For an integer $\ell >0$ consider the event

$${ℰ}_{\ell }=\left\{{\displaystyle \sum _{x,y\in V_n}{\Vert {\mu }_{\mathit{G}\downarrow \{x,y\}}-p_{\ell ,{\partial }^{\ell }[\mathit{G},x]}\otimes p_{\ell ,{\partial }^{\ell }[\mathit{G},y]}\Vert }_{\text{TV}}}<{\eta }^2{n}^2\right\}$$

If $\mathcal{M}$ is p‐symmetric, then $\underset{n\to \infty }{\lim }\mathrm{P}\left[\mathit{G}\in {ℰ}_{\ell }\right]=1$ for sufficiently large $\ell$. Similarly, if the planted distribution is p‐symmetric, then $\underset{n\to \infty }{\lim }\mathrm{P}\left[{\widehat{\mathit{G}}}_{\ell }\in {ℰ}_{\ell }\right]=1$ for large $\ell$.

Hence, assume that $G\in {ℰ}_{\ell }$. Then by the triangle inequality, for any $\omega \in \Omega$,

$$\frac{1}{n}{\displaystyle \sum _{x\in V_n}\left|p_{\ell ,{\partial }^{\ell }[G,x]}(\omega )-{\mu }_{G\downarrow x}(\omega )\right|}$$

$$\hspace{1em}=\frac{1}{n^2}{\displaystyle \sum _{x\in V_n}\left|\left[{\displaystyle \sum _{y\in V_n}{\displaystyle \sum _{\omega \prime \in \Omega }{p}_{\ell ,{\partial }^{\ell }[G,x]}}}(\omega )p_{\ell ,{\partial }^{\ell }[G,y]}(\omega \prime )\right]-\left[{\displaystyle \sum _{y\in V_n}{\displaystyle \sum _{\omega \prime \in \Omega }{\mu }_{G\downarrow x,y}}}(\omega ,\omega \prime )\right]\right|}\le {\eta }^2.$$

Therefore,

$$\frac{1}{n}{\displaystyle \sum _{x\in V_n}{\Vert p_{\ell ,{\partial }^{\ell }[G,x]}-{\mu }_{G\downarrow x}\Vert }_{\text{TV}}}\le {\eta }^2|\Omega |<\eta .$$ (4.19)

Furthermore, by (4.19) and the triangle inequality,

$$\frac{1}{n^2}{\displaystyle \sum _{x,y\in V_n}{\Vert {\mu }_{G\downarrow x}\otimes {\mu }_{G\downarrow y}-p_{\ell ,{\partial }^{\ell }[G,x]}\otimes p_{\ell ,{\partial }^{\ell }[G,y]}\Vert }_{\text{TV}}}\le 2\eta .$$ (4.20)

Since $G\in {ℰ}_{\ell }$, (4.20) entails that

$$\frac{1}{n^2}{\displaystyle \sum _{x,y\in V_n}{\Vert {\mu }_{G\downarrow x}\otimes {\mu }_{G\downarrow y}-{\mu }_{G\downarrow \{x,y\}}\Vert }_{\text{TV}}}\le 3\eta <\epsilon ,$$

i.e., G is $(\epsilon ,2)$‐symmetric.

Together with Lemma 4.10 the following lemma shows that under the assumptions of Theorem 4.4 the partition function is dominated by its judicious part.

Lemma 4.11

There is a number

${\epsilon }_0={\epsilon }_0(\Delta ,\Omega ,\Psi ,\Theta )$

such that for all

$0<\epsilon <{\epsilon }_0,\hspace{0.17em}\ell >0$

there exists

$\chi >0$

such that for large enough n the following is true. If

$G\in \mathcal{G}({\mathcal{M}}_n)$

is a

$(2\ell +5)$

‐acyclic factor graph such that

$${\displaystyle \sum _{x\in V_n}{\Vert {\mu }_{G\downarrow x}-p_{\ell ,{\partial }^{\ell }[G,x]}\Vert }_{\text{TV}}}<{\epsilon }^{3}n$$ (4.21)

and μ_G is

$(\chi ,2)$

‐symmetric, then

${〈\mathit{1}\{(G,\mathit{\sigma })\hspace{0.17em}\text{is}\hspace{0.17em}(\epsilon ,\ell )\text{-}judicious\hspace{0.17em}w.r.t.\hspace{0.17em}p\}〉}_G\ge 1/2$

Pick $\delta =\delta (\ell ,\epsilon )>0$ small, $\beta =\beta (\delta )$ and $\gamma =\gamma (\beta )$ smaller and $\chi =\chi (\gamma )>0$ smaller still and assume that $n>n_0(\chi )$. Let $V_0$ be the partition of V _n such that $x,y\in V_n$ belong to the same class iff ${\partial }^{\ell +2}[G,x]={\partial }^{\ell +2}[G,y]$. By Theorem 2.1 there exists a refinement $V$ of $V_0$ such that μ _G is γ‐homogeneous with respect to $(\mathit{V},\mathit{S})$ for some partition $\mathit{S}$ of ${\Omega }^n$ such that $\#\mathit{V}+\#\mathit{S}\le N=N(\gamma )$. We may index the classes of $V$ as $V_{T,i}$ with $T={\partial }^{\ell +2}[G,x]$ for all x in the class and $i\in [N_T]$ for some integer N _T.

Let J be the set of all $j\in [\#\mathit{S}]$ such that $\mu (S_j)\ge {\delta }^6/N$ and $\mu [·|S_j]$ is γ‐regular. Then

$${\displaystyle \sum _{j\in J}\mu }(S_j)\ge 1-{\delta }^6.$$ (4.22)

Choosing χ small enough, we obtain from Corollary 2.4 that

$$\frac{1}{n}{\displaystyle \sum _{x\in V_n}{\Vert {\mu }_{G\downarrow x}[·|S_j]-{\mu }_{G\downarrow x}\Vert }_{\text{TV}}}<{\delta }^7\hspace{1em}\hspace{0.17em}\text{for}\hspace{0.17em}\text{all}\hspace{0.17em}j\in J.$$

Therefore, by (4.21) and the triangle inequality, for $j\in J$ we get

$$\begin{array}{l}\frac{1}{n}{\displaystyle \sum _{T,i}|}{V}_{T,i}|{〈{\Vert \mathit{\sigma }[·|V_{T,i}]-p_{\ell ,T}\Vert }_{\text{TV}}〉}_G\\ \hspace{1em}=\frac{1}{n}{\displaystyle \sum _{T,i}{\displaystyle \sum _{j\in [\#\mathit{S}]}|}}{V}_{T,i}|{\mu }_G(S_j){〈{\Vert \mathit{\sigma }[·|V_{T,i}]-p_{\ell ,T}\Vert }_{\text{TV}}|S_j〉}_G\\ \hspace{1em}\le {\delta }^7+\frac{1}{n}{\displaystyle \sum _{T,i}{\displaystyle \sum _{j\in [\#\mathit{S}]}|}}{V}_{T,i}|{\mu }_G(S_j){\Vert {〈\mathit{\sigma }[·|V_{T,i}]|S_j〉}_G-p_{\ell ,T}\Vert }_{\text{TV}}\hspace{1em}\hspace{1em}[\text{by}\hspace{0.17em}\hspace{0.17em}\mathbf{H}\mathbf{M}\mathbf{2}]\\ \hspace{1em}\le {\delta }^7+\frac{1}{n}{\displaystyle \sum _{T,i}{\displaystyle \sum _{x\in V_{T,i}}{\displaystyle \sum _{j\in [\#\mathit{S}]}{\mu }_G}}}(S_j){\Vert {\mu }_{G\downarrow x}[·|S_j]-p_{\ell ,{\partial }^{\ell }[G,x]}\Vert }_{\text{TV}}<3{\epsilon }^3.\end{array}$$

Consequently, by (4.22), Bayes' rule and the triangle inequality, summing on $$ and $$ we get

$$$$ (4.23)

$$$$

$$$$

$$$$

Applying the triangle inequality once more, we find

$${\displaystyle \sum _{T\in {\mathfrak{T}}_{\ell }\cap \mathcal{V}}{\lambda }_{G,\ell }}[T|\mathcal{V}]〈{\Vert q_{G,\mathit{\sigma },\ell ,T}-p_{\ell ,T}\Vert }_{\text{TV}}〉\le \frac{1}{n}{\displaystyle \sum _{T,i}|}{V}_{T,i}|{〈{\Vert \mathit{\sigma }[·|V_{T,i}]-p_{\ell ,T}\Vert }_{\text{TV}}〉}_G<3{\epsilon }^3.$$ (4.24)

Further, consider $T\in {\mathfrak{T}}_{\ell }\cap \mathcal{F}$ such that ${\lambda }_{G,\ell }[T|\mathcal{F}]>0$ and let $j\in [d_T]$. Because G is $(2\ell +5)$‐acyclic, there exists a set $\Gamma (T,j)\subset {\mathfrak{T}}_{\ell +2}\cap \mathcal{V}$ with the following two properties. First, for every constraint node a with ${\partial }^{\ell +1}[G,a]=T$ the variable node $x=\partial (G,a,j)$ satisfies ${\partial }^{\ell +2}[G,x]\in \Gamma (T,j)$. Second, for every variable node x with ${\partial }^{\ell +2}[G,x]\in \Gamma (T,j)$ there is a constraint node a with ${\partial }^{\ell +1}[G,a]=T$ such that $\partial (G,a,j)=x$. For $R\in \Gamma (T,j)$ let $m_{R,T,i,j}$ be the number of constraint nodes a with ${\partial }^{\ell +1}[G,a]=T$ such that $x=\partial (G,a,j)$ belongs to $V_{R,i}$. Then by the triangle inequality,

$$\begin{array}{l}{\displaystyle \sum _{T\in {\mathfrak{T}}_{\ell }\cap ℱ}{\displaystyle \sum _{j\in [d_T]}{\lambda }_{G,\ell }}}[T|ℱ]{〈{\Vert q_{G,\mathit{\sigma },\ell ,T\downarrow j}-p_{\ell ,{\partial }^{\ell }[T\uparrow j]}\Vert }_{\text{TV}}〉}_G\\ \hspace{1em}\le {\displaystyle \sum _{T\in {\mathfrak{T}}_{\ell }\cap ℱ}{\displaystyle \sum _{j\in [d_T]}{\displaystyle \sum _{R\in \Gamma (T,j)}{\displaystyle \sum _{i\in [N_R]}\frac{m_{R,T,i,j}}{\left|F_n\right|}}}}}{〈{\Vert \mathit{\sigma }[·|V_{R,i}]-p_{\ell ,R}\Vert }_{\text{TV}}〉}_G\\ \hspace{1em}\le \frac{{\Delta }^2}{n}{\displaystyle \sum _{R\in {\mathfrak{T}}_{\ell +2}\cap \mathcal{V}}{\displaystyle \sum _{i\in [N_R]}|}}{V}_{R,i}|{〈{\Vert \mathit{\sigma }[·|V_{R,i}]-p_{\ell ,R}\Vert }_{\text{TV}}〉}_G;\end{array}$$ (4.25)

$$$$

$$$$

the last inequality follows because all degrees are between one and Δ. Finally, the assertion follows from (4.24) and (4.25).

We proceed by proving the upper bound and the lower bound statement from Theorem 4.4 separately. Strictly speaking, the proof of the lower bound implies the upper bound as well. But presenting the arguments separately makes them slightly easier to follow.

Proof of Theorem 4.4

upper bound. We assume that $\underset{\_}{\mathcal{M}}$ is p‐symmetric. Pick and fix a number $\alpha >0$ ; we aim to show that for large enough n,

$$\frac{1}{n}\mathrm{E}[\ln Z(\mathit{G})]\le {ℬ}_{\vartheta }(p)+4\alpha .$$ (4.26)

For $\epsilon ,l>0$ let

$$ℰ(\epsilon ,l)=\left\{{\displaystyle \sum _{x\in V_n}{\Vert {\mu }_{\mathit{G}\downarrow x}-p_{l,{\partial }^l[\mathit{G},x]}\Vert }_{\text{TV}}}<\epsilon n\right\}.$$

Additionally, let $\mathcal{S}(\chi )$ be the event that ${\mu }_{\mathit{G}}$ is $(\chi ,2)$‐symmetric and let $ℒ(\epsilon ,l)$ be the event that ${\Vert {\lambda }_{\mathit{G},l}-{\vartheta }_l\Vert }_{\text{TV}}<\epsilon$. Corollary 4.8 shows that for some small enough $\epsilon >0$ and large enough $\ell$ (both dependent on α) for all $l\ge \ell$ for large enough n we have

$$\frac{1}{n}\ln \mathrm{E}\left[\mathit{1}\{\mathit{G}\in ℒ({\epsilon }^4,l)\cap {\mathcal{A}}_{100l}\}Z(\mathit{G}){〈\mathit{1}\{(\mathit{G},\mathit{\sigma })\hspace{0.17em}\text{is}\hspace{0.17em}(\epsilon ,l)\text{-judicious}\hspace{0.17em}\mathrm{w}.\mathrm{r}.\mathrm{t}.\hspace{0.17em}p\}〉}_{\mathit{G}}\right]\le {ℬ}_{\vartheta }(p)+\alpha .$$ (4.27)

To apply this bound we are going to argue that $Z(\mathit{G})〈\mathit{1}\{(\mathit{G},\mathit{\sigma })\hspace{0.17em}\text{is}\hspace{0.17em}\hspace{0.17em}(\epsilon ,l)\text{-judicious}\hspace{0.17em}\mathrm{w}.\mathrm{r}.\mathrm{t}.\hspace{0.17em}p\}〉$ is not much smaller than $Z(\mathit{G})$ for most $\mathit{G}$.

The proof of this fact is based on Lemma 4.11. To apply it, we need to pick and fix some specific, large enough ${\ell }_{*}>\ell$ (upon which the value of χ provided by Lemma 4.11 will depend). By Lemma 4.3 there is ${\ell }_{*}\prime$ such that

$$\mathrm{E}\left[{\Vert p_{l,{\partial }^l\mathit{T}}-p_{\mathit{T}}\Vert }_{\text{TV}}|\mathcal{V}\right]<{\epsilon }^4\hspace{1em}\hspace{1em}\hspace{0.17em}\text{for}\hspace{0.17em}\text{all}\hspace{0.17em}l\ge {\ell }_{*}\prime .$$ (4.28)

Further, by Lemma 4.10 there is ${\ell }_{*}\prime \prime$ such that

$$\underset{n\to \infty }{\lim }\mathrm{P}\left[\mathit{G}\in ℰ({\epsilon }^3,l)\right]=1\hspace{1em}\hspace{1em}\hspace{0.17em}\text{for}\hspace{0.17em}\text{all}\hspace{0.17em}l\ge {\ell }_{*}\prime \prime .$$

Let ${\ell }_{*}=\ell +{\ell }_{*}\prime +{\ell }_{*}\prime \prime$. Now, Lemma 4.11 yields $\chi =\chi (\epsilon ,{\ell }_{*})>0$ such that the following is true. Consider the event $\mathcal{U}=\mathcal{S}(\chi )\cap ℒ({\epsilon }^4,{\ell }_{*})\cap ℰ({\epsilon }^3,{\ell }_{*})$. Then

$$\mathit{1}\{\mathit{G}\in \mathcal{U}\cap {\mathcal{A}}_{100{\ell }_{*}}\}Z(\mathit{G})\le 2\mathit{1}\{\mathit{G}\in \mathcal{U}\cap {\mathcal{A}}_{100{\ell }_{*}}\}Z(\mathit{G}){〈\mathit{1}\{(\mathit{G},\mathit{\sigma })\hspace{0.17em}\text{is}\hspace{0.17em}(\epsilon ,{\ell }_{*})\text{-judicious}\hspace{0.17em}\mathrm{w}.\mathrm{r}.\mathrm{t}.\hspace{0.17em}p\}〉}_{\mathit{G}}.$$ (4.29)

Combining (4.27) and (4.29) we obtain

$$\mathrm{E}[1\{\mathit{G}\in \mathcal{U}\cap {\mathcal{A}}_{100{\ell }_{*}}\}Z(\mathit{G})]\le 2\exp (n({ℬ}_{\vartheta }(p)+\alpha )).$$ (4.30)

Hence, we are left to estimate the probability of the event $\mathcal{U}\cap {\mathcal{A}}_{100{\ell }_{*}}$. With respect to $\mathcal{U}$ we obtain from Lemma 4.10 that $\underset{n\to \infty }{\lim }\mathrm{P}\left[\mathit{G}\in \mathcal{S}(\chi )\right]=1$ and $\underset{n\to \infty }{\lim }\mathrm{P}\left[\mathit{G}\in ℰ({\epsilon }^3,{\ell }_{*})\right]=1$. Moreover, the local convergence assumption (3.9) implies that $\underset{n\to \infty }{\lim }\mathrm{P}\left[\mathit{G}\in ℒ({\epsilon }^4,{\ell }_{*})\right]=1$. Consequently,

$$\underset{n\to \infty }{\lim }\mathrm{P}\left[\mathit{G}\in \mathcal{U}\right]=1.$$ (4.31)

Hence, the high girth assumption (3.11) yields

$$\underset{n\to \infty }{\lim \inf }\mathrm{P}\left[\mathit{G}\in \mathcal{U}\cap {\mathcal{A}}_{100{\ell }_{*}}\right]>0.$$ (4.32)

Finally, combining (4.30) and (4.32) and using Markov's inequality, we obtain

$$\underset{n\to \infty }{\lim }\mathrm{P}\left[Z(\mathit{G})>\exp (n({ℬ}_{\vartheta }(p)+2\alpha ))|\mathcal{U}\cap {\mathcal{A}}_{100{\ell }_{*}}\right]=0.$$

Further, since (4.32) shows that the probability of the event $\mathcal{U}\cap {\mathcal{A}}_{100{\ell }_{*}}$ is bounded away from 0, Proposition 3.2 yields

$$\underset{n\to \infty }{\lim }\mathrm{P}\left[Z(\mathit{G})>\exp (n({ℬ}_{\vartheta }(p)+3\alpha ))\right]=0.$$ (4.33)

Because $|n^{-1}\ln Z(\mathit{G})|$ is bounded by some number $C=C(\Delta ,\Omega ,\Psi ,\Theta )>0$ by the definition (3.4) of Z, (4.26) follows from (4.33).

To establish the lower bound we introduce a construction reminiscent of those used in 24, 25, 31, 39, 48. Namely, starting from the sequence $\underset{\_}{\mathcal{M}}$ of $(\Delta ,\Omega ,\Psi ,\Theta )$‐models, we define another sequence ${\underset{\_}{\mathcal{M}}}^{\otimes }={({\mathcal{M}}_n^{\otimes })}_n$ of models as follows. Let ${\Omega }^{\otimes }=\Omega \times \Omega$ and let us denote pairs $(\omega ,\omega \prime )\in {\Omega }^{\otimes }$ by $\omega \otimes \omega \prime$. Further, for any $\psi :{\Omega }^h\to (0,\infty )$ we define a function

$${\psi }^{\otimes }:{({\Omega }^{\otimes })}^h\to (0,\infty ),\hspace{1em}\hspace{1em}({\omega }_1\otimes {\omega }_1\prime ,\dots ,{\omega }_h\otimes {\omega }_h\prime )\mapsto \psi ({\omega }_1,\dots ,{\omega }_h)·\psi ({\omega }_1\prime ,\dots ,{\omega }_h\prime ).$$

Let ${\Psi }^{\otimes }=\{{\psi }^{\otimes }:\psi \in \Psi \}$. Then the $(\Delta ,\Omega ,\Psi ,\Theta )$‐model ${\mathcal{M}}_n=(V_n,F_n,d_n,t_n,{({\psi }_a)}_{a\in F_n})$ gives rise to the $(\Delta ,{\Omega }^{\otimes },{\Psi }^{\otimes },\Theta )$‐model ${\mathcal{M}}_n^{\otimes }=(V_n,F_n,d,t,{({\psi }_a^{\otimes })}_{a\in F_n})$.

Clearly, there is a canonical bijection $\mathcal{G}(\mathcal{M})\to \mathcal{G}({\mathcal{M}}^{\otimes }),\hspace{0.17em}G\mapsto G^{\otimes }$. Moreover, the construction ensures that the Gibbs measure ${\mu }_{G^{\otimes }}\in \mathcal{P}({\Omega }^{\otimes n})$ equals ${\mu }_G\otimes {\mu }_G$. Explicitly, for all ${\omega }_1,{\omega }_1\prime ,\dots ,{\omega }_n,{\omega }_n\prime \in \Omega$,

$${\mu }_{G^{\otimes }}({\omega }_1\otimes {\omega }_1\prime ,\dots ,{\omega }_n\otimes {\omega }_n\prime )={\mu }_G({\omega }_1,\dots ,{\omega }_n){\mu }_G({\omega }_1\prime ,\dots ,{\omega }_n\prime ).$$ (4.34)

In effect, we obtain

$$Z(G^{\otimes })=Z{(G)}^2.$$ (4.35)

Further, writing ${\mathfrak{G}}^{\otimes },{\mathfrak{T}}^{\otimes }$ for the $(\Delta ,{\Omega }^{\otimes },{\Psi }^{\otimes },\Theta )$‐templates and the acyclic $(\Delta ,{\Omega }^{\otimes },{\Psi }^{\otimes },\Theta )$‐templates, we can lift the marginal assignment p from $\mathfrak{T}$ to ${\mathfrak{T}}^{\otimes }$ by letting $p_{T^{\otimes }}^{\otimes }=p_T\otimes p_T$ for all T. Additionally, let ${\vartheta }^{\otimes }\in \mathcal{P}({\mathfrak{T}}^{\otimes })$ be the image of $\vartheta$ under the map $T\in \mathfrak{T}\mapsto T^{\otimes }\in {\mathfrak{T}}^{\otimes }$ so that

$${ℬ}_{{\vartheta }^{\otimes }}(p^{\otimes })=2{ℬ}_{\vartheta }(p).$$ (4.36)

Proof of Theorem 4.4

lower bound. We assume that $\underset{\_}{\mathcal{M}}$ is p‐symmetric and that the same is true of the planted distribution. For $\epsilon ,l>0$ consider the event

$${ℰ}^{\otimes }(\epsilon ,l)=\left\{\frac{1}{n}{\displaystyle \sum _{x\in V_n}{\Vert {\mu }_{{\mathit{G}}^{\otimes }\downarrow x}-p_{{\partial }^l[{\mathit{G}}^{\otimes },x]}^{\otimes }\Vert }_{\text{TV}}}<\epsilon \right\}.$$ (4.37)

and let ${\mathcal{S}}^{\otimes }(\chi )$ be the event that ${\mu }_{{\mathit{G}}^{\otimes }}$ is $(\chi ,2)$‐symmetric. Moreover, as before let $ℒ(\epsilon ,\ell )=\{{\Vert {\lambda }_{\mathit{G},\ell }-{\vartheta }_{\ell }\Vert }_{\text{TV}}<\epsilon \}$. Basically, we are going to apply the same argument as in the proof of the upper bound to the random factor graph ${\mathit{G}}^{\otimes }$ and to ${\widehat{\mathit{G}}}_l$ for a large enough l.

Hence, let $\alpha >0$. Then Corollary 4.8 applied to ${\underset{\_}{\mathcal{M}}}^{\otimes }$ yields a small $\epsilon =\epsilon (\alpha )>0$ and a large $\ell =\ell (\alpha )>0$ such that for all $l\ge \ell$ and large enough n we have

$$\frac{1}{n}\ln \mathrm{E}\left[\mathit{1}\{\mathit{G}\in ℒ({\epsilon }^4,l)\cap {\mathcal{A}}_{100l}\}Z({\mathit{G}}^{\otimes }){〈\mathit{1}\{({\mathit{G}}^{\otimes },\mathit{\sigma })\hspace{0.17em}\text{is}\hspace{0.17em}(\epsilon ,l)\text{-judicious}\hspace{0.17em}\mathrm{w}.\mathrm{r}.\mathrm{t}.\hspace{0.17em}p\}〉}_{{\mathit{G}}^{\otimes }}\right]$$

$$\hspace{1em}\le {ℬ}_{{\vartheta }^{\otimes }}(p^{\otimes })+\alpha .$$ (4.38)

Further, by Lemma 4.3 and (4.34) there exists ${\ell }_{*}\prime$ such that

$$\mathrm{E}\left[{\Vert p_{l,{\partial }^l\mathit{T}}-p_{\mathit{T}}\Vert }_{\text{TV}}|\mathcal{V}\right]+\mathrm{E}\left[{\Vert p_{l,{\partial }^l{\mathit{T}}^{\otimes }}^{\otimes }-p_{{\mathit{T}}^{\otimes }}^{\otimes }\Vert }_{\text{TV}}|\mathcal{V}\right]<{\epsilon }^4\hspace{1em}\hspace{1em}\hspace{0.17em}\text{for}\hspace{0.17em}\text{all}\hspace{0.17em}l\ge {\ell }_{*}\prime .$$ (4.39)

Moreover, Lemma 4.10 shows that for some ${\ell }_{*}\prime \prime$ we have

$$\underset{n\to \infty }{\lim }\mathrm{P}\left[\mathit{G}\in {ℰ}^{\otimes }({\epsilon }^3,l)\right]=1\hspace{1em}\hspace{1em}\text{for}\hspace{0.17em}\text{all}\hspace{0.17em}l\ge {\ell }_{*}\prime \prime .$$ (4.40)

Similarly, (4.34), (4.39) the planted p‐symmetry assumption and Lemma 4.10 imply that there is ${\ell }_{*}\prime \prime \prime$ such that for any $l>{\ell }_{*}\prime \prime \prime$ for large enough $\widehat{l}$ we have

$$\underset{n\to \infty }{\lim }\mathrm{P}\left[{\widehat{\mathit{G}}}_{\widehat{l}}\in {ℰ}^{\otimes }({\epsilon }^3,l)\right]=1.$$ (4.41)

Additionally, Corollary 4.9 shows that for a certain ${\ell }_{*}\prime \prime \prime \prime$ we have

$$\underset{n\to \infty }{\lim }\mathrm{P}\left[\frac{1}{n}\ln \mathrm{E}\left[Z(\mathit{G})|{\mathcal{T}}_l\right]\le {ℬ}_{\vartheta }(p)-\alpha |{\mathcal{A}}_{100l}\right]=0\hspace{1em}\hspace{1em}\hspace{0.17em}\text{for}\hspace{0.17em}\text{all}\hspace{0.17em}l\ge {\ell }_{*}\prime \prime \prime \prime .$$ (4.42)

Let ${\ell }_{*}={\ell }_{*}\prime +{\ell }_{*}\prime \prime +{\ell }_{*}\prime \prime \prime +{\ell }_{*}\prime \prime \prime \prime$.

Applying Lemma 4.11 to ${\underset{\_}{\mathcal{M}}}^{\otimes }$, we obtain ${\chi }_{*}={\chi }_{*}(\epsilon ,{\ell }_{*})>0$ such that the following is true: let ${\mathcal{U}}^{\otimes }={\mathcal{S}}^{\otimes }({\chi }_{*})\cap ℒ({\epsilon }^4,{\ell }_{*})\cap {ℰ}^{\otimes }({\epsilon }^3,{\ell }_{*})$ and define $\mathcal{Z}(\mathit{G})=\mathit{1}\{\mathit{G}\in {\mathcal{U}}^{\otimes }\cap {\mathcal{A}}_{100{\ell }_{*}}\}Z(\mathit{G})$. Then (using (4.35))

$$\mathcal{Z}{(\mathit{G})}^2=\mathit{1}\{\mathit{G}\in {\mathcal{U}}^{\otimes }\cap {\mathcal{A}}_{100{\ell }_{*}}\}Z({\mathit{G}}^{\otimes })$$

$$\le 2\mathit{1}\{\mathit{G}\in {\mathcal{U}}^{\otimes }\cap {\mathcal{A}}_{100{\ell }_{*}}\}Z({\mathit{G}}^{\otimes }){〈\mathit{1}\{({\mathit{G}}^{\otimes },\mathit{\sigma })\hspace{0.17em}\text{is}\hspace{0.17em}(\epsilon ,{\ell }_{*})\text{-judicious}\hspace{0.17em}\mathrm{w}.\mathrm{r}.\mathrm{t}.\hspace{0.17em}p\}〉}_{{\mathit{G}}^{\otimes }}.$$ (4.43)

Further, Proposition 2.5 and Lemma 4.10 imply that

$$\underset{n\to \infty }{\lim }\mathrm{P}\left[\mathit{G}\in {\mathcal{S}}^{\otimes }({\chi }_{*})\right]=1.$$ (4.44)

Combining (3.9), (4.40) and (4.44), we get

$$\underset{n\to \infty }{\lim }\mathrm{P}\left[\mathit{G}\in {\mathcal{U}}^{\otimes }\right]=1.$$ (4.45)

Now, (4.36), (4.38) and (4.43) give an upper bound on the second moment of $\mathcal{Z}$, namely

$$\begin{array}{l}\mathrm{E}[\mathcal{Z}{(\mathit{G})}^2]\le 2\mathrm{E}[\mathit{1}\{\mathit{G}\in ℒ({\epsilon }^4,{\ell }_{*})\cap {\mathcal{A}}_{100{\ell }_{*}}\}\\ \hspace{1em}\hspace{1em}\hspace{1em}{〈\mathit{1}\{({\mathit{G}}^{\otimes },\mathit{\sigma })\hspace{0.17em}\text{is}\hspace{0.17em}(\epsilon ,{\ell }_{*})\text{-judicious}\hspace{0.17em}\mathrm{w}.\mathrm{r}.\mathrm{t}.\hspace{0.17em}{p}^{\otimes }\}〉}_{{\mathit{G}}^{\otimes }}Z({\mathit{G}}^{\otimes })]\\ \le 2\exp (n(2{ℬ}_{\vartheta }(p)+\alpha )).\end{array}$$ (4.46)

$$$$

$$$$

As a next step, we are going to show that

$$\mathrm{E}[\mathcal{Z}(\mathit{G})]\ge \exp (n({ℬ}_{\vartheta }(p^{\otimes })-2\alpha )).$$ (4.47)

Indeed, by Proposition 2.5 and Lemma 4.10 we have

$$\underset{n\to \infty }{\lim }\mathrm{P}\left[{\widehat{\mathit{G}}}_l\in {\mathcal{S}}^{\otimes }({\chi }_{*})\right]=1$$ (4.48)

for large enough l. Further, the local convergence assumption (3.9) and the construction (3.10) of the planted distribution ensures that for large enough l,

$$\underset{n\to \infty }{\lim }\mathrm{P}\left[{\widehat{\mathit{G}}}_l\in ℒ({\epsilon }^4,{\ell }_{*})\right]=1.$$

Hence, (4.41) and (4.48), show that for l large enough

$$\underset{n\to \infty }{\lim }\mathrm{P}\left[{\widehat{\mathit{G}}}_l\in {\mathcal{U}}^{\otimes }\right]=1.$$ (4.49)

Thus, (4.42), (4.49) and Proposition 3.8 yield (4.47).

Finally, combining (4.46) and (4.47) and applying the Paley‐Zygmund inequality (1.3), we obtain for large n,

$$\mathrm{P}\left[Z(\mathit{G})\ge \exp (n({ℬ}_{\vartheta }(p)-4\alpha ))\right]\ge \mathrm{P}\left[\mathcal{Z}(\mathit{G})\ge \exp (n({ℬ}_{\vartheta }(p)-4\alpha ))\right]$$

$$\ge \frac{\mathrm{E}{[\mathcal{Z}(\mathit{G})]}^2}{2\mathrm{E}[\mathcal{Z}{(\mathit{G})}^2]}\ge \exp (-10\alpha n).$$

Because this holds for any $\alpha >0$, the assertion follows from Proposition 3.2.

4.7. Proof of Theorem 4.5

The key step of the proof is to establish the following statement.

Lemma 4.12

For any

$\epsilon >0$

there exists

$\delta >0$

such that for any

$\ell >0$

there exists n₀ such that for all

$n>n_0$

the following is true. Assume that

$G\in \mathcal{G}({\mathcal{M}}_n)$

satisfies

$$\frac{1}{n}{\displaystyle \sum _{x\in V_n}{〈{\Vert {〈\mathit{\sigma }[·|x]|{\nabla }_{\ell }(G,x)〉}_{\mu }-p_{\ell ,{\partial }^{\ell }[G,x]}\Vert }_{\text{TV}}〉}_{\mu }}<{\delta }^9.$$ (4.50)

Then G is

$(\epsilon ,2)$

‐symmetric and

${\displaystyle \sum _{x\in V_n}{\Vert {\mu }_{G\downarrow x}-p_{\ell ,{\partial }^{\ell }[G,x]}\Vert }_{\text{TV}}}<\epsilon n$

Before we prove Lemma 4.12 let us show how it implies Theorem 4.5.

Proof of Theorem 4.5

If $G\in \mathcal{G}({\mathcal{M}}_n)$ satisfies is $(\epsilon ,2)$‐symmetric and ${{\displaystyle \sum _{x\in V_n}\Vert {\mu }_{G\downarrow x}-p_{\ell ,{\partial }^{\ell }[G,x]}\Vert }}_{\text{TV}}<\epsilon n$, then by the triangle inequality

$${\displaystyle \sum _{x,y\in V_n}{\Vert {\mu }_{G\downarrow \{x,y\}}-p_{\ell ,{\partial }^{\ell }[G,x]}\otimes p_{\ell ,{\partial }^{\ell }[G,y]}\Vert }_{\text{TV}}}\le {\displaystyle \sum _{x,y\in V_n}{\Vert {\mu }_{G\downarrow \{x,y\}}-{\mu }_{G\downarrow x}\otimes {\mu }_{G\downarrow y}\Vert }_{\text{TV}}}$$

$$\hspace{1em}+{\Vert {\mu }_{G\downarrow x}\otimes {\mu }_{G\downarrow y}-p_{\ell ,{\partial }^{\ell }[G,x]}\otimes p_{\ell ,{\partial }^{\ell }[G,y]}\Vert }_{\text{TV}}$$

$$\le 4\epsilon n^2.$$

Therefore, the theorem follows by applying Lemma 4.12 either to the random factor graph $\mathit{G}$ or to the random factor graph ${\widehat{\mathit{G}}}_{\ell }$ chosen from the planted model.

Proof of Lemma 4.12

The proof is morally similar to one for the special case of the “stochastic block model” from 40. Let $\gamma =\gamma (\epsilon )>0$ be sufficiently small. By Theorem 2.1 we can pick $\delta =\delta (\gamma )>0$ small enough so that there exists a partition $(\overrightarrow{V},\overrightarrow{S})$ with $\#\overrightarrow{V}+\#\overrightarrow{S}<{\delta }^{-1}$ with respect to which μ _G is ${\gamma }^4$‐homogeneous. Suppose that V _i, S _j are classes such that $\left|V_i\right|\ge {\delta }^{3/2}n,\hspace{0.17em}{\mu }_G(S_j)\ge {\delta }^{3/2}$ and such that $\mu [·|S_j]$ is ${\gamma }^4$‐regular on V _i. We claim that

$$\frac{1}{\left|V_i\right|}{\displaystyle \sum _{x\in V_i}{\Vert {\mu }_{G\downarrow x}[·|S_j]-p_{\ell ,{\partial }^{\ell }[G,x]}\Vert }_{\text{TV}}}<3\gamma .$$ (4.51)

The assertion is immediate from this inequality. Indeed, suppose that (4.51) is true for all i, j such that $\left|V_i\right|\ge {\delta }^{3/2}n,\hspace{0.17em}{\mu }_G(S_j)\ge {\delta }^{3/2}$ such that $\left|V_i\right|\ge {\delta }^{3/2}n,{\mu }_G(S_j)\ge {\delta }^{3/2}$ is ${\gamma }^4$‐regular on V _i. Then because $\#\mathit{V}+\#\mathit{S}\le 1/\delta$

$${\displaystyle \sum _x{\Vert {\mu }_{G\downarrow x}[·|S_j]-p_{\ell ,{\partial }^{\ell }[G,x]}\Vert }_{\text{TV}}}<4\gamma n.$$ (4.52)

Hence, by HM1 and Bayes’ rule, ${\displaystyle \sum _x{\Vert {\mu }_{G\downarrow x}-p_{\ell ,{\partial }^{\ell }[G,x]}\Vert }_{\text{TV}}}<5\gamma n<\epsilon n$. Further, (4.52) and Lemma 2.8 imply that μ _G is $(\epsilon ,2)$‐regular (provided that we pick γ small enough). Thus, we are left to prove (4.51).

Assume for contradiction that (4.51) is violated for V _i, S _j such that $\left|V_i\right|\ge {\delta }^{3/2}n,\hspace{0.17em}{\mu }_G(S_j)\ge {\delta }^{3/2}$. Then by the triangle inequality there is a set $W\subset V_i$ of size at least $\gamma \left|V_i\right|$ such that for all $x\in W$ we have

$${\Vert {\mu }_{G\downarrow x}[·|S_j]-p_{\ell ,{\partial }^{\ell }[G,x]}\Vert }_{\text{TV}}\ge \gamma .$$

For $x\in W$ pick ${\omega }_x\in \Omega$ such that $|{\mu }_{G\downarrow x}[{\omega }_x|S_j]-p_{\ell ,{\partial }^{\ell }[G,x]}|\ge \gamma$ is maximum. Then by the pigeonhole principle there exist $\omega \in \Omega$ and $W\prime \subset W,\hspace{0.17em}|W\prime |\ge \left|W\right|/(2|\Omega |)$, such that either

$$\forall x\in W\prime :{\mu }_{G\downarrow x}[\omega |S_j]\ge p_{\ell ,{\partial }^{\ell }[G,x]}(\omega )+\gamma \hspace{1em}\hspace{1em}\hspace{0.17em}\text{or}\hspace{0.17em}$$ (4.53)

$$\forall x\in W\prime :{\mu }_{G\downarrow x}[\omega |S_j]\le p_{\ell ,{\partial }^{\ell }[G,x]}(\omega )-\gamma$$ (4.54)

In particular, for some ω we have

$$\forall x\in W\prime :{\mu }_{G\downarrow x}[\omega |S_j]\ge p_{\ell ,{\partial }^{\ell }[G,x]}(\omega )+\gamma /|\Omega |$$ (4.55)

We claim that there is a set $L\subset W\prime$ of size $\left|L\right|=\lceil 1/\delta \rceil$ with the following properties.

• (i)the pairwise distance between any two $x,y\in L$ is at least $10(\ell +1)$.
• (ii)
  for all $x\in L$ we have

  $${〈{\Vert {〈\mathit{\sigma }[·|x]|{\nabla }_{\ell }(G,x)〉}_G-p_{\ell ,{\partial }^{\ell }[G,x]}\Vert }_{\text{TV}}〉}_{{\mu }_G}<{\delta }^4.$$ (4.56)

Indeed, because $\left|V_i\right|\ge {\delta }^{2}n$ and $\mu (S_j)\ge {\delta }^2$ the assumption (4.50) implies that

$${\displaystyle \sum _{x\in V_i}{〈{\Vert {〈\mathit{\sigma }[·|x]|{\nabla }_{\ell }(G,x)〉}_G-p_{\ell ,{\partial }^{\ell }[G,x]}\Vert }_{\text{TV}}〉}_{{\mu }_G[ · |S_j]}}<{\delta }^5\left|V_i\right|.$$ (4.57)

Since $|W\prime |\ge \gamma \left|V_i\right|/|\Omega |\ge \delta \left|V_i\right|$, (4.57) implies that there is a set $W\prime \prime \subset W\prime$ of size $|W\prime \prime |\ge |W\prime |/2$ such that (4.56) holds for all $x\in W\prime \prime$. Now, construct a sequence $W\prime \prime =W_0\prime \prime \supset W_1\prime \prime \cdots$ inductively as follows. In step $i\ge 1$ pick some $x_i\in W_{i-1}\prime \prime$. Then $W_i\prime \prime$ contains x _i and all $y\in W_{i-1}\prime \prime \setminus \left\{x_i\right\}$ whose distance from x _i is greater than $10(\ell +1)$. Since for each x _i the total number of variable nodes at distance at most $10(\ell +1)$ is bounded by ${\Delta }^{10(\ell +1)}$ and $|W_0\prime \prime |\ge \delta \left|V_i\right|/2\ge {\delta }^{3}n/2$, the set ${\displaystyle \underset{i\ge 1}{\cap }{W}_i}\prime \prime$ has size at least ${\delta }^3{\Delta }^{-10(\ell +1)}n/2>1/\delta$, provided that n is large enough. Finally, simply pick any subset $L\subset {\displaystyle \underset{i\ge 1}{\cap }{W}_i}\prime \prime$ of size $\left|L\right|=\lceil 1/\delta \rceil$.

Consider the event $ℰ=\left\{\mathit{\sigma }[\omega |L]\ge |L{|}^{-1}{\displaystyle \sum _{x\in L}{p}_{\ell ,{\partial }^{\ell }[G,x]}}+{\gamma }^3\right\}.$ We claim that

$${\mu }_G[ℰ|S_j]\le 2{\delta }^2.$$ (4.58)

Indeed, by (4.56) and the union bound we have

$$\begin{array}{l}{〈\mathit{1}\left\{\forall x\in L:{\Vert {〈\mathit{\sigma }[·|x]|{\nabla }_{\ell }(G,x)〉}_G-p_{\ell ,{\partial }^{\ell }[G,x]}\Vert }_{\text{TV}}\le \delta \right\}〉}_{{\mu }_G}\\ \hspace{1em}\ge 1-{\displaystyle \sum _{x\in L}{〈\mathit{1}\left\{{\Vert {〈\mathit{\sigma }[·|x]|{\nabla }_{\ell }(G,x)〉}_G-p_{\ell ,{\partial }^{\ell }[G,x]}\Vert }_{\text{TV}}>\delta \right\}〉}_{{\mu }_G}}\ge 1-{\delta }^2.\end{array}$$ (4.59)

$$$$

Now, let $\mathfrak{L}$ be the coarsest σ‐algebra such that $\mathfrak{L}\supset {\nabla }_{\ell }(G,x)$ for all $x\in L$. Suppose that $\sigma \in S_j$ is such that

$${\Vert {〈\mathit{\sigma }[·|x]|{\nabla }_{\ell }(G,x)〉}_G(\sigma )-p_{\ell ,{\partial }^{\ell }[G,x]}\Vert }_{\text{TV}}\le \delta \hspace{1em}\hspace{0.17em}\text{for}\hspace{0.17em}\text{all}\hspace{0.17em}x\in L.$$ (4.60)

We claim that (4.60) implies

$${〈\mathit{1}\{\mathit{\sigma }\in ℰ\}|\mathfrak{L}〉}_G(\sigma )<{\delta }^3.$$ (4.61)

Indeed, let $X={\displaystyle \sum _{x\in L}\mathit{1}}\{\mathit{\sigma }(x)=\omega \}$. Then (4.60) implies that

$$〈X(\mathit{\sigma })|\mathfrak{L}〉(\sigma )\le 2\delta \left|L\right|+{\displaystyle \sum _{x\in L}{p}_{\ell ,{\partial }^{\ell }[G,x]}}(\omega ).$$ (4.62)

Furthermore, the pairwise distance of the variables in L is at least $2(\ell +1)$ and given $\mathfrak{L}$ the values of the variables at distance either $\ell$ or $\ell +1$ from each $x\in L$ are fixed. Therefore, given $\mathfrak{L}$ the events $\{\mathit{\sigma }(x)=\omega \}$ are mutually independent. In effect, X is stochastically dominated by a sum of independent random variables. Hence, recalling that γ is much smaller than δ, we see that (4.61) follows from (4.62) and the Chernoff bound. Finally, combining (4.59) and (4.61) we obtain (4.58).

But (4.58) does not sit well with (4.55). In fact, (4.55) entails that ${\mu }_G[ℰ|S_j]\ge {\gamma }^2$; for consider the random variable $Y={\displaystyle \sum _{x\in L}\mathit{1}}\{\mathit{\sigma }(x)\ne \omega \}$. Then (4.55) yields ${〈Y〉}_{\mu [ · |S_j]}\le {\displaystyle \sum _{x\in L}(}1-{\mu }_{G\downarrow x}[\omega |S_j])\le \left|L\right|(1-\gamma /|\Omega |)-{\displaystyle \sum _{x\in L}{p}_{\ell ,{\partial }^{\ell }[G,x]}}(\omega )$. Hence, by Markov's inequality

$$1-{\mu }_G[ℰ|S_j]\le \frac{{〈Y〉}_{\mu [ · |S_j]}}{\left|L\right|(1-{\gamma }^3)-{\displaystyle \sum _{x\in L}{p}_{\ell ,{\partial }^{\ell }[G,x]}}(\omega )}\le \frac{\left|L\right|(1-\gamma /|\Omega |)-{\displaystyle \sum _{x\in L}{p}_{\ell ,{\partial }^{\ell }[G,x]}}(\omega )}{\left|L\right|(1-{\gamma }^3)-{\displaystyle \sum _{x\in L}{p}_{\ell ,{\partial }^{\ell }[G,x]}}(\omega )}$$

$$\le \frac{1-\gamma /|\Omega |}{1-{\gamma }^3}\le 1-{\gamma }^2.$$

Combining this bound with (4.58), we obtain ${\gamma }^2\le {\mu }_G(ℰ)/{\mu }_G(S_j)\le 2{\delta }^2/{\mu }_G(S_j)$. Thus, choosing δ much smaller than γ, we conclude that ${\mu }_G(S_j)<{\delta }^{3/2}$, which is a contradiction. Thus, we have established that (4.51).

5. CONDITIONING ON THE LOCAL STRUCTURE

5.1. A Generalised Configuration Model

The aim in this section is to prove Proposition 4.7. The obvious problem is the conditioning on the σ‐algebra ${\mathcal{T}}_{\ell }$ that fixes the depth‐$\ell$ neighborhoods of all variable nodes and the depth‐$\ell +1$ neighborhoods of all constraint nodes. Following 16, we deal with this conditioning by setting up a generalised configuration model.

Recall that ${\mathfrak{T}}_{\ell }$ is the (finite) set of all isomorphism classes ${\partial }^{\ell }T$ for $T\in \mathfrak{T}\cap \mathcal{V}$ and ${\partial }^{\ell +1}T$ for $T\in \mathfrak{T}\cap \mathcal{F}$. Let $\ell ,n>0$ be integers and let $\mathcal{M}=(V,F,d,t,{({\psi }_a)}_{a\in F})$ be a $(\Delta ,\Omega ,\Psi ,\Theta )$‐model of size n. Moreover, let $G\in \mathcal{G}(\mathcal{M})$ be a $100\ell$‐acyclic factor graph. Then we define an enhanced $(\Delta ,\Omega ,\Psi ,{\Theta }_{\ell })$‐model $\mathcal{M}(G,\ell )$ with type set ${\Theta }_{\ell }=({\mathfrak{T}}_{\ell }\cap \mathcal{V})\times [\Delta ]$ as follows. The set of variable nodes is V, the set of constraint nodes is F, the degrees are given by d and the weight function associated with each constraint a is ψ _a just as in $\mathcal{M}$. Moreover, the type of a variable clone (x, i) is $t_{G,\ell }(x,i)=({\partial }^{\ell }[G,x],i)$. Further, the type of a constraint clone (a, j) such that $\partial (G,a,j)=(x,i)$ is $t_{G,\ell }(a,j)=({\partial }^{\ell }[G,x],i)$. Clearly, $\mathcal{G}(\mathcal{M}(G,\ell ))\subset \mathcal{G}(\mathcal{M})$. The following lemma shows that the model $\mathcal{M}(G,\ell )$ can be used to generate factor graphs whose local structure coincides with that of G.

Lemma 5.1

Assume that

$\ell \ge 0$

and that

$G\prime \in \mathcal{G}(\mathcal{M}(G,\ell ))$

is

$2\ell +4$

‐acyclic. Then

$G\prime$

viewed as a

$\mathcal{M}$

‐factor graph satisfies

$G{\cong }_{\ell }G\prime$

We are going to show inductively for $l\in [\ell ]$ that $G{\cong }_{l}G\prime$. The case l = 0 is immediate from the construction. Thus, assume that l > 0, let $(x,i)\in C_V$ and let B be the set of all clones that have distance precisely l – 1 from (x, i). Since $G\prime$ is $(2\ell +2)$‐acyclic, the pairwise distance of any two clones in B is at least 2. Moreover, by induction we know that $t_{G,1}(w,j)=t_{G\prime ,1}(w,j)$ for all $(w,j)\in B$. Therefore, $t_{G,l}(x,i)=t_{G\prime ,l}(x,i)$.

In order to prove Proposition 4.7 we need to enhance the model $\mathcal{M}(G,\ell )$ further to accommodate an assignment that provides a value from Ω for each clone. Thus, let $\widehat{\sigma }:C_V\cup C_F\to \Omega$ be a map. We call $\widehat{\sigma }$ valid if $\widehat{\sigma }(x,i)=\widehat{\sigma }(x,j)$ for all $x\in V,\hspace{0.17em}i,j\in [d(x)]$ and if for all $\theta \in {\Theta }_{\ell }$ we have

$$\forall \omega \in \Omega :\left|\left\{(x,i)\in C_V:\widehat{\sigma }(x,i)=\omega ,t_{G,\ell }(x,i)=\theta \right\}\right|$$

$$\hspace{1em}=\left|\left\{(a,j)\in C_F:\widehat{\sigma }(a,j)=\omega ,t_{G,\ell }(a,j)=\theta \right\}\right|.$$

Of course, we can extend a valid $\widehat{\sigma }$ to a map $V\to \Omega ,\hspace{0.17em}x\mapsto \widehat{\sigma }(x,1)$. Given a valid $\widehat{\sigma }$ we define a $(\Delta ,\Omega ,\Psi ,{\Theta }_{\ell }\times \Omega )$‐model $\mathcal{M}(G,\widehat{\sigma },\ell )$ with variable nodes V, constraint nodes F, degrees d and weight functions ${({\psi }_a)}_{a\in F}$ such that the type $t_{G,\widehat{\sigma },\ell }(x,i)$ of a variable clone (x, i) is $({\partial }^{\ell }[G,x],i,\widehat{\sigma }(x,i))$ and such that the type $t_{G,\widehat{\sigma },\ell }(a,j)$ of a constraint clone (a, j) with $\partial (G,a,j)=(x,i)$ is $({\partial }^{\ell }[G,x],i,\widehat{\sigma }(a,j))$. By construction, $\mathcal{G}(\mathcal{M}(G,\widehat{\sigma },\ell ))\subset \mathcal{G}(\mathcal{M}(G,\ell ))\subset \mathcal{G}(\mathcal{M})$. Let us recall the definition of the distance from (3.5). Further, for two maps $\widehat{\sigma },\widehat{\sigma }\prime :C_V\cup C_F\to \Omega$ let $\text{dist}(\widehat{\sigma },\widehat{\sigma }\prime )=\left|\left\{(v,i)\in C_V\cup C_F:\widehat{\sigma }(v,i)\ne \widehat{\sigma }\prime (v,i))\right\}\right|$. In Section 5.2 we are going to establish the following.

Lemma 5.2

For any $\epsilon ,\ell >0$ there is $n_0=n_0(\epsilon ,\ell ,\Delta ,\Omega ,\Psi ,\Theta )$ such that for $n>n_0$ the following holds. If $\mathcal{M}$ is a $(\Delta ,\Omega ,\Psi ,\Theta )$ ‐model of size n, $G\in \mathcal{G}(\mathcal{M})$ is $100\ell$ ‐acyclic and $\widehat{\sigma }$ is valid, then with probability at least $1-\epsilon$ the random factor graph $\mathit{G}(\mathcal{M}(G,\widehat{\sigma },\ell ))$ has the following property. There exist a valid $\widehat{\sigma }\prime$ and a $4\ell$ ‐acyclic $G\prime \in \mathcal{G}(\mathcal{M}(G,\widehat{\sigma }\prime ,\ell ))$ such that $dist(\widehat{\sigma },\widehat{\sigma }\prime )+\text{dist}(G\prime ,\mathit{G}(\mathcal{M}(G,\widehat{\sigma },\ell )))\le n^{0.9}.$

To proceed consider a $(G,\ell )$‐marginal sequence q. We call $\widehat{\sigma }$ q‐valid if the following two conditions hold.

• 1
  V1: For all $T\in {\mathfrak{T}}_{\ell }\cap \mathcal{V},\omega \in \Omega$ we have

  $$\left|\left\{x\in V:{\partial }^{\ell }[G,x]=T,\widehat{\sigma }(x)=\omega \right\}\right|=q_T(\omega )\left|\left\{x\in V:{\partial }^{\ell }[G,x]=T\right\}\right|.$$
• 2
  V2: For all $T\in {\mathfrak{T}}_{\ell }\cap \mathcal{F},{\omega }_1,\dots ,{\omega }_{d_T}\in \Omega$ we have

  $$\left|\left\{a\in F:{\partial }^{\ell +1}[G,a]=T,\forall j\in [d_T]:\widehat{\sigma }(a,j)={\omega }_j\right\}\right|$$

  $$\hspace{1em}=q_T({\omega }_1,\dots ,{\omega }_{d_T})\left|\left\{a\in F:{\partial }^{\ell +1}[G,a]=T\right\}\right|.$$

Lemma 5.3

For any $\epsilon ,\ell >0$ there is $n_0=n_0(\epsilon ,\ell ,\Delta ,\Omega ,\Psi ,\Theta )$ such that for $n>n_0$ the following holds. Assume that $\mathcal{M}$ is a $(\Delta ,\Omega ,\Psi ,\Theta )$ ‐model of size n, $G\in \mathcal{G}(\mathcal{M})$ is $100\ell$ ‐acyclic and q is a $(G,\ell )$ ‐marginal sequence such that there exists a q‐valid $\widehat{\sigma }$. Then with the sum ranging over all q‐valid $\widehat{\sigma }$ we have

$$\exp \left(n{ℬ}_G(\ell ,q)-\sqrt{n}\right)\le {\displaystyle \sum _{\widehat{\sigma }}\frac{|\mathcal{G}(\mathcal{M}(G,\widehat{\sigma },\ell ))|}{|\mathcal{G}(\mathcal{M}(G,\ell ))|}}\le \exp \left(n{ℬ}_G(\ell ,q)+\sqrt{n}\right).$$

We defer the proof of Lemma 5.3 to Section 5.3.

Proof of Proposition 4.7

We claim that

$$\left|\left\{G\prime \in \mathcal{G}(\mathcal{M}(n)):G\prime {\cong }_{\ell }G\right\}\right|\ge |\mathcal{G}(\mathcal{M}(G,\ell ))|\exp (-n^{0.91}).$$ (5.1)

To see this, apply Lemma 5.2 to the constant map $\widehat{\sigma }:(v,j)\in C_V\cup C_F\mapsto {\omega }_0$ for some fixed ${\omega }_0\in \Omega$. Then we conclude that with probability at least 1/2 the random graph $\mathit{G}(\mathcal{M}(G,\ell ))=\mathit{G}(\mathcal{M}(G,\widehat{\sigma },\ell ))$ is at distance at most $n^{0.9}$ from a $4\ell$‐acyclic $G\prime \in \mathit{G}(\mathcal{M}(G,\ell ))\subset \mathcal{G}(\mathcal{M})$. Furthermore, by Lemma 5.1 this factor graph $G\prime$, viewed as an element of $\mathcal{G}(\mathcal{M})$, satisfies $G{\cong }_{\ell }G\prime$. Finally, since the total number of factor graphs at distance at most $n^{0.9}$ from $G\prime$ is bounded by $\exp (n^{0.91})$ because all degrees are bounded, we obtain (5.1).

Let $\delta >0$ be small enough. If $\sigma \in \Sigma (G,\ell ,q,\delta )$, then by (4.7) there exists a $(G,\ell )$‐marginal sequence $q\prime$ such that $\sigma \in \Sigma (G,\ell ,q\prime ,0)$ such that ${\Vert q_T-q_T\prime \Vert }_{\text{TV}}<\delta$ for all $T\in {\mathfrak{T}}_{\ell }$. Because ${\mathfrak{T}}_{\ell }$ is finite and $\Sigma (G,\ell ,q\prime ,0)\ne \varnothing$, the total number of such $q\prime$ is bounded by a polynomial in n. Moreover, due to the continuity of ${ℬ}_{G,\ell }(·)$ we can choose $\delta =\delta (\ell )$ small enough so that $|{ℬ}_{G,\ell }(q\prime )-{ℬ}_{G,\ell }(q)|<\epsilon /2$ for all such $q\prime$. Hence, summing over all $\widehat{\sigma }$ corresponding to $\sigma \in \Sigma (G,\ell ,q,\delta )$, we obtain from (5.1) and Lemma 5.3 that

$$\mathrm{E}[Z_{\ell ,q}(\mathit{G})|\mathit{G}{\cong }_{\ell }G]\le {\displaystyle \sum _{\widehat{\sigma }}\frac{|\mathcal{G}(\mathcal{M}(G,\widehat{\sigma },\ell ))|}{\left|\left\{G\prime \in \mathcal{G}(\mathcal{M}(n)):G\prime {\cong }_{\ell }G\right\}\right|}}\le \exp (n{ℬ}_G(\ell ,q)+\epsilon n).$$

Conversely, by Lemma 5.2 with probability at least 1/2 the graph $\mathit{G}(\mathcal{M}(G,\widehat{\sigma },\ell ))$ is within distance at most $n^{0.9}$ of a $4\ell$‐acyclic $G\prime$, which satisfies $G\prime {\cong }_{\ell }G$ by Lemma 5.1. As before, the total number of graphs at distance at most $n^{0.9}$ off $G\prime$ is bounded by $\exp (n^{0.91})$. Similarly, the total number of $\widehat{\sigma }\prime$ at distance at most $n^{0.9}$ off $\widehat{\sigma }$ is bounded by $\exp (n^{0.91})$. Therefore, by Lemma 5.1

$$\mathrm{E}[\mathit{1}\{{\mathcal{A}}_{2\ell +1}\}{Z}_{\ell ,q}(\mathit{G})|\mathit{G}{\cong }_{\ell }G]\ge \frac{\exp (-2n^{0.98})}{2}{\displaystyle \sum _{\widehat{\sigma }}\frac{|\mathcal{G}(\mathcal{M}(G,\widehat{\sigma },\ell ))|}{\left|\mathcal{G}(\mathcal{M}(G,\ell ))\right|}}\ge \exp (n{ℬ}_G(\ell ,q)-\epsilon n),$$

as desired.

5.2. Proof of Lemma 5.2

Let ${\Theta }_{*}=\left\{t_{G,\widehat{\sigma },\ell }(x,i):(x,i)\in C_V\right\}$ be the set of all possible types. For each $\tau \in {\Theta }_{*}$ let $n_{\tau }$ be the number of clones $(x,i)\in C_V$ with $t_{G,\widehat{\sigma },\ell }(x,i)=\tau$. Throughout this section we assume that $n>n_0(\epsilon ,\ell ,\Delta ,\Omega ,\Psi ,\Theta )$ is sufficiently large.

Lemma 5.4

There exists

$\beta >0$

such that the following is true. For any

$G,\widehat{\sigma }$

there exists

$3/4<\gamma <7/8$

such that for every

$\tau \in {\Theta }_{*}$

either

$n_{\tau }\le n^{\gamma }$

or

$n_{\tau }>n^{\gamma +\beta }$

The number of possible types is bounded independently of n. Hence, choosing β small enough, we can ensure that there exists an integer j > 0 such that $3/4+j\beta <7/8$ such that $[n^{3/4+j\beta },n^{3/4+(j+1)\beta }]\cap \left\{n_{\tau }:\tau \in T\right\}=\varnothing$.

Fix $\beta ,\gamma$ as in the previous lemma. Call τ rare if $n_{\tau }\le n^{\gamma }$ and common otherwise. Let Y be the number of variable clones that belong to cycles of length at most $10\ell$ in $\mathit{G}(\mathcal{M}(G,\widehat{\sigma },\ell ))$.

Lemma 5.5

For large enough n we have

$\mathrm{E}[Y]\le n^{\gamma }\ln n$

Let R be the set of variable clones (v, i) of a rare type and let U be the set of all variable clones whose distance from R in G does not exceed $100\ell$. Since the maximum degree as well as the total number of types are bounded, we have $\left|U\right|\le \left|R\right|\ln \ln n\le n^{\gamma }\sqrt{\ln n}$, provided that n is big enough. Thus, to get the desired bound on $\mathrm{E}[Y]$ we merely need to consider the set W of common clones that are at distance more than $100\ell$ from R.

More specifically, let (v, i) be a common clone. We are going to bound the probability that $(v,i)\in W$ and that (v, i) lies on a cycle of length at most $10\ell$. To this end, we are going to explore the (random) factor graph from (v, i) via the principle of deferred decisions. Let $i_1=i,\dots ,i_l\in [\Delta ]$ be a sequence of $l\le 10\ell$ indices. If (v, i) lies on a cycle of length at most $10\ell$, then there exists such a sequence $(i_1,\dots ,i_l)$ that corresponds to this cycle. Namely, with $v_1=v$ the cycle comprises of the clones $(v_1,i_1),\dots ,(v_l,i_l)$ such that $\partial (\mathit{G}(\mathcal{M}(G,\widehat{\sigma },\ell )),v_j,i_j)=(v_{j+1},i_{j+1})$. In particular, $v_l=v_1$. Clearly, the total number of sequences $(i_1,\dots ,i_l)$ is bounded. Furthermore, given that (v _l, i _l) is common, the probability that $v_l=v_0$ is bounded by $2n^{-\gamma }$. Since $\gamma >3/4$, the linearity of expectation implies that $\mathrm{E}[Y]\le \left|U\right|+2n^{1-\gamma }\ln n\le n^{\gamma }\ln n$.

Lemma 5.6

Assume that $G\prime \prime \in \mathcal{G}(\mathcal{M}(G,\widehat{\sigma },\ell ))$ satisfies $Y(G\prime \prime )\le n^{\gamma }{\ln }^{2}n$. Then there is a $4\ell$ ‐acyclic $G\prime \in \mathcal{G}(\mathcal{M}(G,\ell ))$ such that $\text{dist}(G\prime ,G\prime \prime )\le n^{0.9}$

Let R be the set of variable clones (v, i) of a rare type and let U be the set of all variable clones whose distance from R in G does not exceed $10\ell$. Moreover, let $G\prime \prime \prime \in \mathcal{G}(\mathcal{M}(G,\ell ))$ minimise $\text{dist}(G\prime \prime ,G\prime \prime \prime )$ subject to the condition that $\partial (G\prime \prime \prime ,v,i)=\partial (G,v,i)$ for all $(v,i)\in U$. Then $\text{dist}(G\prime \prime ,G\prime \prime \prime )\le n^{\gamma }\ln n$ because the total number of types is bounded. Therefore, the assumption $Y(G\prime \prime )\le n^{\gamma }{\ln }^{2}n$ implies that $Y(G\prime \prime \prime )\le n^{\gamma }{\ln }^{3}n$, say. In addition, because G is $100\ell$‐acyclic, none of the clones in R lies on a cycle of length at most $4\ell$ in $G\prime \prime \prime$.

Altering only a bounded number of edges in each step, we are now going to remove the short cycles of $G\prime \prime \prime$ one by one. Let C be the set of common clones. The construction of $G\prime \prime \prime$ ensures that only common clones lie on cycles of length at most $4\ell$. Consider one such clone (v, i) and let N be the set of all variable clones that can be reached from (v, i) by traversing precisely two edges of $G\prime \prime \prime$; thus, N contains all clones (w, j) such that w has distance two from v and all clones (v, j) that are incident to the same constraint node as (v, i). Once more by the construction of $G\prime \prime \prime$ we have $N\subset C$. Furthermore, $\left|N\right|\le {\Delta }^2$.

We claim that there exists $N\prime \subset C$ and a bijection $\xi :N\to N\prime$ such that the following conditions are satisfied.

• (i) $t_{G,\widehat{\sigma },\ell }(w,j)=t_{G,\widehat{\sigma },\ell }(\xi (w,j))$ for all $(w,j)\in N$
• (ii)the pairwise distance in $G\prime \prime \prime$ between any two clones in $N\prime$ is at least $100\ell$.
• (iii)the distance in $G\prime \prime \prime$ between $N\cup \{(v,i)\}$ and $N\prime$ is at least $100\ell$.
• (iv)the distance between R and $N\prime$ is at least $100\ell$.
• (v)any $(w,j)\in N\prime$ is at distance at least $100\ell$ from any clone that belongs to a cycle of $G\prime \prime \prime$ of length at most $4\ell$.

Since the maximum degree of $G\prime \prime \prime$ is bounded by Δ, there are no more than $n^{\gamma }{\ln }^{4}n$ clones violate condition (iii), (iv) or (v). By comparison, there are at least $n^{\gamma +\beta }$ clones of any common type. Hence, the existence of ξ follows.

Now, obtain $G\prime \prime \prime \prime$ from $G\prime \prime \prime$ as follows.

• let $G\prime \prime \prime \prime (\xi (w,j))=G\prime \prime \prime (w,j)$ and $G\prime \prime \prime \prime (w,j)=G\prime \prime \prime (\xi (w,j))$ for all $(w,j)\in N$.

• let $G\prime \prime \prime \prime (w,j)=G\prime \prime \prime (w,j)$ for all $(w,j)\cancel{\in }N\cup N\prime$.

It is immediate from the construction that any clone on a cycle of length at most $4\ell$ in $G\prime \prime \prime \prime$ also lies on such a cycle of $G\prime \prime \prime$. Moreover, (v, i) does not lie on a cycle of length at most $4\ell$ in $G\prime \prime \prime \prime$. Hence, $Y(G\prime \prime \prime \prime )<Y(G\prime \prime \prime )$. In addition, all clones on cycles of length at most $4\ell$ and their neighbours are common. Hence, the construction can be repeated on $G\prime \prime \prime \prime$. Since $Y(G\prime \prime \prime )\le n^{\gamma }{\ln }^{3}n$, we ultimately obtain a $4\ell$‐acyclic $G\prime \prime$ with $\text{dist}(G\prime ,G\prime \prime )\le n^{\gamma }{\ln }^{4}n<n^{0.9}$.

Proof of Lemma 5.2

The assertion is immediate from Lemmas 5.5 and 5.6 and Markov's inequality.

5.3. Proof of Lemma 5.3

Let ${\mathcal{V}}_{\ell }={\mathfrak{T}}_{\ell }\cap \mathcal{V}$ and for $T\in {\mathcal{V}}_{\ell }$ let n _T be the number of variable nodes x such that ${\partial }^{\ell }[G,x]=T$. By Stirling's formula the number $|\Sigma (G,\ell ,q,0)|$ of assignments $\sigma :V_n\to \Omega$ with marginals as prescribed by q satisfies

$$\left|\ln |\Sigma |-{\displaystyle \sum _{T\in {\mathcal{V}}_{\ell }}{n}_T}H(q_T)\right|\le {\ln }^{2}n.$$ (5.2)

Further, for $T\in {\mathcal{V}}_{\ell }$ and $i\in [d_T]$ let $C_V(T,i)$ be the set of all clones $(x,i)\in C_V$ such that $t_{G,\ell }(x,i)=(T,i)$. Moreover, let $C_F(T,i)$ be the set of all clones $(a,j)\in C_F$ such that $t_{G,\ell }(a,j)=(T,i)$. Additionally, let ${\mathcal{F}}_{\ell }(T,i)$ be the set of all pairs $(T\prime ,j)$ with $T\prime \in {\mathfrak{T}}_{\ell }\cap \mathcal{F},\hspace{0.17em}j\in [d_{T\prime }]$ such that there is $(a,j)\in C_F(T,i)$ such that ${\partial }^{\ell +1}[G,a]=T\prime$. Of course, the total number of perfect matchings between $C_V(T,i)$ and $C_F(T,i)$ equals $n_T!$. If we fix $\sigma \in \Sigma (G,\ell ,q,0)$, then any such perfect matching induces an assignment $\widehat{\sigma }:C_F(T,i)\to \Omega$ by mapping a clone $(a,j)\in C_F(T,i)$ matched to (x, i) to the value $\sigma (x)$. Let $B_{T,i}$ be the event that in a such random matching for all $(T\prime ,j)\in {\mathcal{F}}_{\ell }(T,i)$ and all ω we have

$$\left|\left\{(a,j)\in C_F:{\partial }^{\ell +1}[G,a]=T\prime ,\widehat{\sigma }(a,j)=\omega \right\}\right|=q_{T\prime \downarrow j}(\omega )\left|\left\{(a,j)\in C_F:{\partial }^{\ell +1}[G,a]=T\prime \right\}\right|$$

Moreover, for $(T\prime ,j)\in {\mathcal{F}}_{\ell }(T,i)$ let $m_{T\prime }$ be the number of $a\in F$ such that ${\partial }^{\ell +1}[G,a]=T\prime$. Then

$$\mathrm{P}\left[B_{T,i}\right]=\frac{1}{n_T!}\left[{\displaystyle \prod _{\omega \in \Omega }\left(\begin{array}{c}{q}_T(\omega )n_T\\ {(q_{T\prime \downarrow j}(\omega )m_{T\prime })}_{(T\prime ,j)\in {\mathcal{F}}_{\ell }(T,i)}\end{array}\right)}\right]$$

$$\hspace{1em}\times \left[{\displaystyle \prod _{(T\prime ,j)\in {\mathcal{F}}_{\ell }(T,i)}\left(\begin{array}{c}{m}_{T\prime }\\ {(q_{T\prime \downarrow j}(\omega )m_{T\prime })}_{\omega \in \Omega }\end{array}\right)}\right]{\displaystyle \prod _{(T\prime ,j)\in {\mathcal{F}}_{\ell }(T,i),\omega \in \Omega }(}{q}_{T\prime \downarrow j}(\omega )m_{T\prime })!$$

$$={\left(\begin{array}{c}{n}_T\\ {(q_T(\omega )n_T)}_{\omega \in \Omega }\end{array}\right)}^{-1}{\displaystyle \prod _{(T\prime ,j)\in {\mathcal{F}}_{\ell }(T,i)}\left(\begin{array}{c}{m}_{T\prime }\\ {(q_{T\prime \downarrow j}(\omega )m_{T\prime })}_{\omega \in \Omega }\end{array}\right)}$$

$$=\exp \left[O(\ln n)-{\displaystyle \sum _{(T\prime ,j)\in {\mathcal{F}}_{\ell }(T,i)}{m}_{T\prime }}D\left(q_{T\prime \downarrow j}\left|\right|q_T\right)\right].$$

Let ${\mathcal{F}}_{\ell }={\mathfrak{T}}_{\ell }\cap \mathcal{F}$. Multiplying up over all (T, i), we obtain for $B=\cap B_{T,i}$

$$\mathrm{P}\left[B\right]={\displaystyle \prod _{T\in {\mathcal{V}}_{\ell }}{\displaystyle \prod _{i\in [d_T]}\mathrm{P}}}\left[B_{T,i}\right]=\exp \left[O(\ln n)-{\displaystyle \sum _{T\prime \in {\mathcal{F}}_{\ell }}{\displaystyle \sum _{j\in [d_{T\prime }]}{m}_{T\prime }}}D\left(q_{T\prime \downarrow j}\left|\right|q_{{\partial }^{\ell }[T\prime \uparrow j]}\right)\right],$$ (5.3)

where the constant hidden in the $O(·)$ depends on $\Delta ,\Omega ,\Psi ,\Theta ,\ell$ only.

Further, for $T\prime \in {\mathcal{F}}_{\ell }$ let $S_{T\prime }$ be the event that for every $({\omega }_1,\dots ,{\omega }_{d_{T\prime }})\in {\Omega }^{d_{T\prime }}$ we have

$$\left|\left\{a\in F:{\partial }^{\ell +1}[G,a]=T\prime ,\forall j\in [d_{T\prime }]:\widehat{\sigma }(a,j)={\omega }_j\right\}\right|$$

$$\hspace{1em}=q_{T\prime }({\omega }_1,\dots ,{\omega }_{d_{T\prime }})\left|\left\{a\in F:{\partial }^{\ell +1}[G,a]=T\prime \right\}\right|.$$

Then

$$\begin{array}{l}\mathrm{P}\left[S_{T\prime }|B\right]=\left(\begin{array}{c}{m}_{T\prime }\\ m_{T\prime }{q}_{T\prime }\end{array}\right){\displaystyle \prod _{j\in [d_{T\prime }]}{\left(\begin{array}{c}{m}_{T\prime }\\ m_{T\prime }{q}_{T\prime \downarrow j}\end{array}\right)}^{-1}}\\ =\exp \left[O(\ln n)-m_{T\prime }D\left(q_{T\prime }\left|\right|q_{T\prime \downarrow 1}\otimes \cdots \otimes q_{T\prime \downarrow d_{T\prime }}\right)\right].\end{array}$$ (5.4)

$$$$

Moreover,

$$\begin{array}{l}-D\left(q_{T\prime }\left|\right|q_{T\prime \downarrow 1}\otimes \cdots \otimes q_{T\prime \downarrow d_{T\prime }}\right)=H(q_{T\prime })-{\displaystyle \sum _{j\in [d_{T\prime }]}H}(q_{T\prime \downarrow j})\\ =H(q_{T\prime })-{\displaystyle \sum _{j\in [d_{T\prime }]}H}(q_{{\partial }^{\ell }[T\prime \uparrow j]})\\ \hspace{1em}+{\displaystyle \sum _{j\in [d_{T\prime }]}(H(}{q}_{{\partial }^{\ell }[T\prime \uparrow j]})-H(q_{T\prime \downarrow j}))\end{array}$$ (5.5)

$$$$

$$$$

In addition,

$$\begin{array}{l}H(q_{{\partial }^{\ell }[T\prime \uparrow j]})-H(q_{T\prime \downarrow j})\\ \hspace{1em}={\displaystyle \sum _{\omega \in \Omega }-}{q}_{{\partial }^{\ell }[T\prime \uparrow j]}(\omega )\ln q_{{\partial }^{\ell }[T\prime \uparrow j]}(\omega )+q_{T\prime \downarrow j}(\omega )\ln q_{T\prime \downarrow j}(\omega )\\ \hspace{1em}={\displaystyle \sum _{\omega \in \Omega }(}{q}_{T\prime \downarrow j}(\omega )-q_{{\partial }^{\ell }[T\prime \uparrow j]}(\omega ))\ln q_{{\partial }^{\ell }[T\prime \uparrow j]}(\omega )+q_{T\prime \downarrow j}(\omega )\ln \frac{q_{T\prime \downarrow j}(\omega )}{q_{{\partial }^{\ell }[T\prime \uparrow j]}(\omega )}\\ \hspace{1em}=D\left(q_{T\prime \downarrow j}\left|\right|q_{{\partial }^{\ell }[T\prime \uparrow j]}\right)+{\displaystyle \sum _{\omega \in \Omega }(}{q}_{T\prime \downarrow j}(\omega )-q_{{\partial }^{\ell }[T\prime \uparrow j]}(\omega ))\ln q_{{\partial }^{\ell }[T\prime \uparrow j]}(\omega ).\end{array}$$ (5.6)

$$$$

$$$$

$$$$

Further, because q is a $(G,\ell )$‐marginal sequence, condition MS3 guarantees that

$${\displaystyle \sum _{T\prime \in {\mathcal{F}}_{\ell }}{m}_{T\prime }}{\displaystyle \sum _{j\in [d_{T\prime }]}{\displaystyle \sum _{\omega \in \Omega }(}}{q}_{T\prime \downarrow j}(\omega )-q_{{\partial }^{\ell }[T\prime \uparrow j]}(\omega ))\ln q_{{\partial }^{\ell }[T\prime \uparrow j]}(\omega )=0.$$ (5.7)

Hence, letting $S=\cap S_{T\prime }$, we obtain from ((5.4), (5.5))

$$\mathrm{P}\left[S|B\right]=\exp \left[O(\ln n)+{\displaystyle \sum _{T\prime \in {\mathcal{F}}_{\ell }}{m}_{T\prime }}\left[H(q_{T\prime })+{\displaystyle \sum _{j\in [d_{T\prime }]}}\left[D\left(q_{T\prime \downarrow j}\left|\right|q_{{\partial }^{\ell }[T\prime \uparrow j]}\right)-H(q_{{\partial }^{\ell }[T\prime \uparrow j]})\right]\right]\right].$$ (5.8)

Once more the constant hidden in the $O(·)$ depends on $\Delta ,\Omega ,\Psi ,\Theta ,\ell$ only. Further, given $S\cap B$ we have

$${\displaystyle \prod _{a\in F}{\psi }_a}(\sigma )=\exp \left[{\displaystyle \sum _{T\prime \in {\mathcal{F}}_{\ell }}{m}_{T\prime }}{〈\ln {\psi }_{T\prime }(\mathit{\sigma })〉}_{q_{T\prime }}\right].$$ (5.9)

Finally, the assertion follows from (5.2), (5.3), (5.8) and (5.9).

ACKNOWLEDGEMENTS

The second author thanks Dimitris Achlioptas for inspiring discussions. We also thank two anonymous reviewers for their careful reading and their invaluable comments, which led to an improved version of Corollary 2.4, among other things.