A local maximal inequality under uniform entropy

Abstract

We derive an upper bound for the mean of the supremum of the empirical process indexed by a class of functions that are known to have variance bounded by a small constant δ. The bound is expressed in the uniform entropy integral of the class at δ. The bound yields a rate of convergence of minimum contrast estimators when applied to the modulus of continuity of the contrast functions.

1. Introduction

The empirical measure ${\mathbb{P}}_n$ and empirical process ${\mathbb{G}}_n$ of a sample of observations X₁, … , X_n from a probability measure P on a measurable space ($\mathcal{X},\mathcal{A}$) attach to a given measurable function $f:\mathcal{X}\to \mathbb{R}$ the numbers

$${\mathbb{P}}_{n}f=\frac{1}{n}\sum _{i=1}^{n}f\left(X_i\right),{\mathbb{G}}_{n}f=\frac{1}{\sqrt{n}}\sum _{i=1}^n(f\left(X_i\right)-Pf).$$

It is often useful to study the suprema of these stochastic processes over a given class $\mathcal{F}$ of measurable functions. The distribution of the supremum

$${\parallel {\mathbb{G}}_n\parallel }_{{\mathcal{F}}^{:}}=\underset{f\in \mathcal{F}}{\text{sup}}\mid \mathbb{n}f\mid$$

is known to concentrate near its mean value, at a rate depending on the size of the envelope function of the class $\mathcal{F}$, but irrespective of its complexity. On the other hand, the mean value of ${\parallel {\mathbb{G}}_n\parallel }_{\mathcal{F}}$ depends on the size of the class $\mathcal{F}$. Entropy integrals, of which there are two basic versions, are useful tools to bound this mean value.

The uniform entropy integral was introduced in [9] and [5], following [3], in their study of the abstract version of Donsker’s theorem. We define an L_r-version of it as

$$J(\delta ,\mathcal{F},L_r)=\underset{Q}{\text{sup}}{\int }_0^{\delta }\sqrt{1+\log N(\epsilon {\parallel F\parallel }_{Q,r},\mathcal{F},L_r\left(Q\right))}d\epsilon .$$

Here the supremum is taken over all finitely discrete probability distributions Q on ($\mathcal{X},\mathcal{A}$), the covering number $N(\epsilon ,\mathcal{F},L_r\left(Q\right))$ is the minimal number of balls of radius ε in L_r(Q) needed to cover $\mathcal{F}$, F is an envelope function of $\mathcal{F}$, and ${\parallel f\parallel }_{Q,r}$ denotes the norm of a function f in L_r(Q). The integral is defined relative to an envelope function, which need not be the minimal one, but can be any measurable function $F:\mathcal{X}\to \mathbb{R}$ such that |f| ≤ F for every $f\in \mathcal{F}$. If multiple envelope functions are under consideration, then we write $J(\delta ,\mathcal{F}\mid F,L_r)$ to stress this dependence. An inequality, due to Pollard (also see [12], 2.14.1), says, under some measurability assumptions, that

$${\text{E}}_P^{\ast }{\parallel {\mathbb{G}}_n\parallel }_{\mathcal{F}}\lesssim J(1,\mathcal{F},L_2){\parallel F\parallel }_{P,2}.$$ (1.1)

Here ≲ means smaller than up to a universal constant. This shows that for a class $\mathcal{F}$ with finite uniform entropy integral, the supremum ${\parallel G_n\parallel }_{\mathcal{F}}$ is not essentially bigger than a multiple of the empirical process ${\mathbb{G}}_{n}F$ at the envelope function F. The inequality is particularly useful if this envelope function is small.

The bracketing entropy integral has its roots in the Donsker theorem of [8], again following initial work by Dudley. For a given norm it can be defined as

$$J_{\left[\right]}(\delta ,\mathcal{F},\parallel \cdot \parallel )={\int }_0^{\delta }\sqrt{1+\log N_{\left[\right]}(\epsilon \parallel F\parallel ,\mathcal{F},\parallel \cdot \parallel )}d\epsilon .$$

Here the bracketing number $N_{\left[\right]}(\epsilon ,\mathcal{F},\parallel \cdot \parallel )$ is the minimal number of brackets $[l,u]=\{f:\mathcal{X}\to \mathbb{R}:l\le f\le u\}$ of size $\parallel u-l\parallel$ smaller than ε needed to cover $\mathcal{F}$. A useful inequality, due to Pollard (also see [12], 2.14.2), is

$${\text{E}}_P^{\ast }{\parallel {\mathbb{G}}_n\parallel }_{\mathcal{F}}\lesssim J_{\left[\right]}(1,\mathcal{F},L_2\left(P\right)){\parallel F\parallel }_{P,2}.$$ (1.2)

Bracketing numbers are bigger than covering numbers (at twice the size), and hence the bracketing integral is bigger than a multiple of the corresponding entropy integral. However, the bracketing integral involves only the single distribution P, whereas the uniform entropy integral takes a supremum over all (discrete) distributions, making the two integrals incomparable in general. Apart from this difference the two maximal inequalities have the same message.

The two inequalities (1.1) and (1.2) involve the size of the envelope function, but not the sizes of the individual functions in the class $\mathcal{F}$. They also exploit finiteness of the entropy integrals only, roughly requiring that the entropy grows at smaller order than ε⁻² as ε ↓ 0, and not the precise size of the entropy. In the case of the bracketing integral this is remedied in the equality (see [12], 3.4.2), valid for any class of functions $f:\mathcal{X}\to [-1,1]$ with Pf² ≤ δ²PF² and any δ ∈ (0, 1),

$${\text{E}}_P^{\ast }{\parallel {\mathbb{G}}_n\parallel }_{\mathcal{F}}\lesssim J_{\left[\right]}(\delta ,\mathcal{F},L_2\left(P\right)){\parallel F\parallel }_{P,2}\left(1+\frac{J_{\left[\right]}(\delta ,\mathcal{F},L_2\left(P\right))}{{\delta }^2\sqrt{n}{\parallel F\parallel }_{P,2}}\right).$$ (1.3)

Here the assumption that the class of functions is uniformly bounded is too restrictive for some applications, but can be removed if the entropy integral is computed relative to the stronger “norm”

$${\parallel f\parallel }_{P,B}={\left(2P(e^{\mid f\mid }-1-\mid f\mid )\right)}^{1∕2}.$$

Although it is not a norm, this quantity can be used to define the size of brackets and hence bracketing numbers. Inequality (1.3) is valid for an arbitrary class of functions with ${\parallel f\parallel }_{P,B}\le \delta {\parallel F\parallel }_{P,B}$ if the L₂(P)-norm is replaced by ${\parallel \cdot \parallel }_{P,B}$ in its right side (at four appearances) (see Theorem 3.4.3 of [12]). The “norm” ${\parallel \cdot \parallel }_{P,B}$ derives from the refined version of Bernstein’s inequality, which was first used in the literature on rates of convergence of minimum contrast estimators in [1] (also see [11]).

Maximal inequalities of type (1.3) using uniform entropy are thus far unavailable. In this note we derive an exact parallel of (1.3) for uniformly bounded functions, and investigate similar inequalities for unbounded functions. The validity of these results seems unexpected, as the stronger control given by bracketing has often been thought necessary for estimates of moduli of continuity. It was suggested to us by Theorem 3.1 and its proof in [4].

1.1. Application to minimum contrast estimators

Inequalities involving the sizes of the functions f are of particular interest in the investigation of empirical minimum contrast estimators. Suppose that ${\widehat{\theta }}_n$ mimimizes a criterion of the type

$$\theta \mapsto {\mathbb{P}}_n{m}_{\theta },$$

for given measurable functions $m_{\theta }:\mathcal{X}\to \mathbb{R}$ indexed by a parameter θ, and that the population contrast satisfies, for a “true” parameter θ₀ and some metric d on the parameter set,

$$P{m}_{\theta }-P{m}_{{\theta }_0}\gtrsim d^2(\theta ,{\theta }_0).$$

A bound on the rate of convergence of ${\widehat{\theta }}_n$ to θ₀ can then be derived from the modulus of continuity of the empirical process ${\mathbb{G}}_n{m}_{\theta }$ indexed by the functions m_θ. Specifically (see e.g. [12], 3.2.5) if ϕ_n is a function such that δ ↦ ϕ_n(δ)/δ^α is decreasing for some α < 2 and

$$\text{E}\underset{\theta :d(\theta ,{\theta }_0)<\delta }{\text{sup}}\mid {\mathbb{G}}_n(m_{\theta }-m_{{\theta }_0})\mid \lesssim {\varphi }_n\left(\delta \right)$$ (1.4)

then $d({\widehat{\theta }}_n,{\theta }_0)=O_P\left({\delta }_n\right)$, for δ_n any solution to

$${\varphi }_n\left({\delta }_n\right)\le \sqrt{n}{\delta }_n^2.$$ (1.5)

Inequality (1.4) involves the empirical process indexed by the class of functions ${\mathcal{M}}_{\delta }=\{m_{\theta }-m_{{\theta }_0}:d(\theta ,{\theta }_0)<\delta \}$. If d dominates the L₂(P)-norm, or another norm ∥ ⋅ ∥ that can be used in an equality of the type (1.3), such as the Bernstein norm, and the norms of the envelopes of the classes M_δ are bounded in δ, then we can choose

$${\varphi }_n\left(\delta \right)=J(\delta ,{\mathcal{M}}_{\delta },\parallel \cdot \parallel )\left(1+\frac{J(\delta ,{\mathcal{M}}_{delta},\parallel \cdot \parallel )}{{\delta }^2\sqrt{n}}\right),$$

where J is an appropriate entropy integral. For this choice the inequality (1.5) is equivalent to

$$J({\delta }_n,{\mathcal{M}}_{{\delta }_n},\parallel \cdot \parallel )\le \sqrt{n}{\delta }_n^2.$$ (1.6)

Thus a rate of convergence can be read off directly from the entropy integral.

We note that an inequality of type (1.3) is unattractive for very small δ, as the bound may even increase to infinity as δ ↓ 0. However, it is accurate for the range of δ that are important in the application to moduli of continuity.

Moduli of continuity also play an important role in model selection theorems. See for instance [7].

Inequalities involving uniform entropy permit for instance the immediate derivation of rates of convergence for minimum contrast functions that form VC-classes. Furthermore, uniform entropy is preserved under various (combinatorial) operations to make new classes of functions. This makes uniform entropy integrals a useful tool in situations where bracketing numbers may be difficult to handle. Equation (1.6) gives an elegant characterization of rates of convergence in these situations, where thus far ad-hoc arguments were necessary.

2. Uniformly bounded classes

Call the class $\mathcal{F}$ of functions P-measurable if the map

$$(X_1,\dots ,X_n)\mapsto \underset{f\in \mathcal{F}}{\text{sup}}\mid \sum _{i=1}^n{e}_{i}f\left(X_i\right)\mid$$

on the completion of the probability space (${\mathcal{X}}^n,{\mathcal{A}}^n,P^n$) is measurable, for every sequence e₁, e₂, … , e_n ∈ {−1, 1}.

Theorem 2.1. Let $\mathcal{F}$ be a P-measurable class of measurable functions with envelope function F ≤ 1 and such that ${\mathcal{F}}^2$ is P-measurable. If Pf² < δ²PF², for every f and some δ ∈ (0, 1), then

$${\text{E}}_P^{\ast }{\parallel {\mathbb{G}}_n\parallel }_{\mathcal{F}}\lesssim J(\delta ,\mathcal{F},L_2)\left(1+\frac{J(\delta ,\mathcal{F},L_2)}{{\delta }^2\sqrt{n}{\parallel F\parallel }_{P,2}}\right){\parallel F\parallel }_{P,2}.$$

Proof. We use the following refinement of (1.1) (see e.g. [12], 2.14.1): for any P-measurable class $\mathcal{F}$,

$${\text{E}}_P^{\ast }{\parallel {\mathbb{G}}_n\parallel }_{\mathcal{F}}\lesssim E_P^{\ast }J\left(\frac{{\text{sup}}_f{\left({\mathbb{P}}_n{f}^2\right)}^{1∕2}}{{\left({\mathbb{P}}_n{F}^2\right)}^{1∕2}},\mathcal{F},L_2\right){\left({\mathbb{P}}_n{F}^2\right)}^{1∕2}.$$ (2.1)

Because $\delta \mapsto J(\delta ,\mathcal{F},L_2)$ is the integral of a nonincreasing nonnegative function, it is a concave function such that the map t ↦ J(t)/t, which is the average of its derivative over [0, t], is nonincreasing. The concavity shows that its perspective $(x,t)\mapsto tJ(x∕t,\mathcal{F},L_2)$ is a concave function of its two arguments (cf. [2], page 89). Furthermore, the “extended-value extension” of this function (which by definition is −∞ if x ≤ 0 or t ≤ 0) is obviously nondecreasing in its first argument and was noted to be nondecreasing in its second argument. Therefore, by the vector composition rules for concave functions ([2], pages 83–87, especially lines -2 and -1 of page 86), the function $(x,y)\mapsto H(x,y)≔J(\sqrt{x∕y},\mathcal{F},L_2)\sqrt{y}$ is concave. We have that $E_P^{\ast }{\mathbb{P}}_n{F}_2={\parallel F\parallel }_{P,2}^2$. Therefore, by an application of Jensen’s inequality to the right side of the preceding display we obtain, for ${\sigma }_n^2={\text{sup}}_f{\mathbb{P}}_n{f}^2$,

$${\text{E}}_P^{\ast }{\parallel {\mathbb{G}}_n\parallel }_{\mathcal{F}}\lesssim J\left(\frac{\sqrt{{\text{E}}_P^{\ast }{\sigma }_n^2}}{{\parallel F\parallel }_{P,2}},\mathcal{F},L_2\right){\parallel F\parallel }_{P,2}.$$ (2.2)

The application of Jensen’s inequality with outer expectations can be justified here by the monotonicity of the function H, which shows that the measurable majorant of a variable H(U, V) is bounded above by H(U*, V*), for U* and V* measurable majorants of U and V. Thus E*H(U, V) ≤ EH(U*, V*), after which Jensen’s inequality can be applied in its usual (measurable) form.

The second step of the proof is to bound ${\text{E}}_P^{\ast }{\sigma }_n^2$. Because ${\mathbb{P}}_n{f}^2=P{f}^2+n^{-1∕2}{\mathbb{G}}_n{f}^2$ and Pf² ≤ δ²PF² for every f, we have

$${\text{E}}_P^{\ast }{\sigma }_n^2\le {\delta }^2{\parallel F\parallel }_{P,2}^2+\frac{1}{\sqrt{n}}{\text{E}}_P^{\ast }{\parallel {\mathbb{G}}_n\parallel }_{{\mathcal{F}}^2}.$$ (2.3)

Here the empirical process in the second term can be replaced by the symmetrized empirical process ${\mathbb{G}}_n^o$ (defined as ${\mathbb{G}}_n^{o}f=n^{-1∕2}{\sum }_{i=1}^n{\epsilon }_{i}f\left(X_i\right)$ for independent Rademacher variables ε₁, ε₂, … , ε_n) at the cost of adding a multiplicative factor 2 (e.g. [12], 2.3.1). The expectation can be factorized as the expectation on the Rademacher variables ε followed by the expectation on X₁, …, X_n, and ${\text{E}}_{\epsilon }{\parallel {\mathbb{G}}_n^o\parallel }_{{\mathcal{F}}^2}\le 2E_{\epsilon }{\parallel {\mathbb{G}}_n^o\parallel }_{\mathcal{F}}$ by the contraction principle for Rademacher variables ([6], Theorem 4.12), and the fact that F ≤ 1 by assumption. Taking the expectation on X₁, … , X_n, we obtain that ${\text{E}}_P^{\ast }{\parallel {\mathbb{G}}_n\parallel }_{{\mathcal{F}}^2}\le 4{\text{E}}_P^{\ast }{\parallel {\mathbb{G}}_n^o\parallel }_{\mathcal{F}}$, which in turn is bounded above by $8{\text{E}}_P^{\ast }{\parallel {\mathbb{G}}_n\parallel }_{\mathcal{F}}$ by the desymmetrization inequality (e.g. 2.36 in [12]).

Thus ${\mathcal{F}}^2$ in the last term of (2.3) can be replaced by $\mathcal{F}$, at the cost of inserting a constant. Next we apply (2.2) to this term, and conclude that $z^2≔{\text{E}}_P^{\ast }{\sigma }_n^2∕{\parallel F\parallel }_{P,2}^2$ satisfies the inequality

$$z^2\lesssim {\delta }^2+\frac{J(z,\mathcal{F},L_2)}{\sqrt{n}{\parallel F\parallel }_{P,2}}.$$ (2.4)

We apply Lemma 2.1 with r = 1, A = δ and $B^2=1∕\left(\sqrt{n}{\parallel F\parallel }_{P,2}\right)$ to see that

$$J(z,\mathcal{F},L_2)\lesssim J(\delta ,\mathcal{F},L_2)+\frac{J^2(\delta ,\mathcal{F},L_2)}{{\delta }^2\sqrt{n}{\parallel F\parallel }_{P,2}}.$$

We insert this in (2.2) to complete the Proof.

Lemma 2.1. Let $J:(0,\infty )\to \mathbb{R}$ be a concave, nondecreasing function with J(0) = 0. If z² ≤ A² + B²J(z^r) for some r ∈ (0, 2) and A,B > 0, then

$$J\left(z\right)\lesssim J\left(A\right){\left[1+J\left(A^r\right){\left(\frac{B}{A}\right)}^2\right]}^{1∕(2-r)}.$$

Proof. For t > s > 0 we can write s as the convex combination s = (s/t)t+ (1−s/t)0 of t and 0. Since J(0) = 0, the concavity of J gives that J(s) ≥ (s/t)J(t). Thus the function t ↦ J(t)/t is decreasing, which implies that J(Ct) ≤ CJ(t) for C ≥ 1 and any t > 0.

By the monotonicity of J and the assumption on z it follows that

$$J\left(z^r\right)\le J\left({(A^2+B^{2}J\left(z^r\right))}^{r∕2}\right)\le J\left(A^r\right){\left(1+{\left(\frac{B}{A}\right)}^{2}J\left(z^r\right)\right)}^{r∕2}.$$

This implies that J(z^r) is bounded by a multiple of the maximum of J(A^r) and J(A^r)(B/A)^rJ(z^r)^r/2. If it is bounded by the second one, then $J{\left(z^r\right)}^{1-r∕2}\lesssim J\left(A^r\right){(B∕A)}^r$. We conclude that

$$J\left(z^r\right)\lesssim J{\left(A^r\right)}^{2∕(2-r)}{\left(\frac{B}{A}\right)}^{2r∕(2-r)}.$$

Next again by the monotonicity of J,

$$\begin{array}{cc}\hfill J\left(z\right)& \le J\left(\sqrt{A^2+B^{2}J\left(z^r\right)}\right)\le J\left(A\right)\sqrt{1+{\left(\frac{B}{A}\right)}^{2}J\left(z^r\right)}\hfill \\ \hfill & \lesssim J\left(A\right){\left[1+{\left(\frac{B}{A}\right)}^2\left(J\left(A^r\right)+J{\left(A^r\right)}^{2∕(2-r)}{\left(\frac{B}{A}\right)}^{2r∕(2-r)}\right)\right]}^{1∕2}\hfill \\ \hfill & \lesssim J\left(A\right)\left[1+\sqrt{J\left(A^r\right)}\left(\frac{B}{A}\right)+{\left(\frac{B}{A}\right)}^{1∕(2-r)}\right].\hfill \end{array}$$

The middle term on the right side is bounded by a multiple of the sum of the first and third terms, since $x\lesssim 1^p+x^q$ for any conjugate pair (p, q) and any x > 0, in particular $x=\sqrt{J\left(A^r\right)}B∕A$.

For values of δ such that $\delta {\parallel F\parallel }_{P,2}⪡1∕\sqrt{n}$ Theorem 2.1 can be improved. (This seems not to be of prime interest for statistical applications.) Its bound can be written in the form $J(\delta ,\mathcal{F},L_2){\parallel F\parallel }_{P,2}+J^2(\delta ,\mathcal{F},L_2)∕\left({\delta }^2\sqrt{n}\right)$. In the second term δ can be replaced by $1∕\left({\parallel F\parallel }_{P,2}\sqrt{n}\right)$, which is better if δ is smaller than the latter number, as the function $\delta \mapsto J(\delta ,\mathcal{F},L_2)∕\delta$ is decreasing.

Lemma 2.2. Under the conditions of Theorem 2.1,

$${\text{E}}_P^{\ast }{\parallel {\mathbb{G}}_n\parallel }_{\mathcal{F}}\lesssim J(\delta ,\mathcal{F},L_2){\parallel F\parallel }_{P,2}+J^2\left(\frac{1}{\sqrt{n}{\parallel F\parallel }_{P,2}},\mathcal{F},L_2\right)\sqrt{n}{\parallel F\parallel }_{P,2}^2.$$

Proof. We follow the proof of Theorem 2.1 up to (2.4), but next use the alternative bounds

$$\begin{array}{cc}\hfill J(z,\mathcal{F},L_2)& \lesssim J\left({\delta }^2+\frac{J(z,\mathcal{F},L_2)}{\sqrt{n}{\parallel F\parallel }_{P,2}},\mathcal{F},L_2\right)\hfill \\ \hfill & \le J(\delta ,\mathcal{F},L_2)+J\left(\frac{J(z,\mathcal{F},L_2)}{\sqrt{n}{\parallel F\parallel }_{P,2}},\mathcal{F},L_2\right)\hfill \\ \hfill & \le J(\delta ,\mathcal{F},L_2)+J({\delta }_n,\mathcal{F},L_2)\sqrt{\frac{J(z,\mathcal{F},L_2)}{{\delta }_n}\vee 1,}\hfill \end{array}$$

for $1∕{\delta }_n=\sqrt{n}{\parallel F\parallel }_{P,2}$. Here we have used the subadditivity of the map $\delta \mapsto J(\delta ,\mathcal{F},L_2)$, and the inequality $J(C\delta ,\mathcal{F},l_2)\le CJ(\delta ,\mathcal{F},L_2)$ for C ≥ 1 in the last step. We can bound the sum of the three terms on the right side by a multiple of the maximum of these terms and conclude that the left side is smaller than at least one of the three terms. Solving next yields that

$$J(z,\mathcal{F},L_2)\lesssim J(z,\mathcal{F},L_2)\vee \frac{J^2({\delta }_n,\mathcal{F},L_2)}{{\delta }_n}\vee J({\delta }_n,\mathcal{F},L_2).$$

Because $J({\delta }_n,\mathcal{F},L_2)\ge {\delta }_n$ for every δ_n > 0, by the definition of the entropy integral, the third term on the right is bounded by the second term. We substitute the bound in (2.2) to finish the proof.

3. Unbounded classes

In this section we investigate relaxations of the assumption that the class $\mathcal{F}$ of functions is uniformly bounded, made in Theorem 2.1. We start with a moment bound on the envelope.

Theorem 3.1. Let $\mathcal{F}$ be a P-measurable class of measurable functions with envelope function F such that PF^{(4p−2)/(p−1)} < ∞ for some p > 1 and such that ${\mathcal{F}}^2$ and ${\mathcal{F}}^4$ are P-measurable. If Pf² < δ²PF² for every f and some δ ∈ (0, 1), then

$${\text{E}}_P^{\ast }{\parallel {\mathbb{G}}_n\parallel }_{\mathcal{F}}\lesssim J(\delta ,\mathcal{F},L_2){\left(1+\frac{J({\delta }^{1∕p},\mathcal{F},L_2)}{{\delta }^2\sqrt{n}}\frac{{\parallel F\parallel }_{P,(4p-2)∕(p-1)}^{2-1∕p}}{{\parallel F\parallel }_{P,2}^{2-1∕p}}\right)}^{p∕(2p-1)}{\parallel F\parallel }_{P,2}.$$

Proof. Application of (2.1) to the functions f², forming the class ${\mathcal{F}}^2$ with envelope function F², yields

$${\text{E}}_P^{\ast }{\parallel {\mathbb{G}}_n\parallel }_{{\mathcal{F}}^2}\lesssim {\text{E}}_P^{\ast }J\left(\frac{{\sigma }_{n,4}^2}{{\left({\mathbb{P}}_n{F}^4\right)}^{1∕2}},{\mathcal{F}}^2\mid F^2,L_2\right){\left({\mathbb{P}}_n{F}^4\right)}^{1∕2},$$ (3.1)

for σ_n,r the diameter of $\mathcal{F}$ in $L_r\left({\mathbb{P}}_n\right)$, i.e.

$${\sigma }_{n,r}^r=\underset{f}{\text{sup}}{\mathbb{P}}_n{\mid f\mid }^r.$$ (3.2)

Preservation properties of uniform entropy (see [10], or [12], 2.10.20, where the supremum over Q can also be moved outside the integral to match our current definition of entropy integral, applied to ϕ(f) = f² with L = 2F) show that $J(\delta ,{\mathcal{F}}^2\mid F^2,L_2)\lesssim J(\delta ,\mathcal{F}\mid F,L_2)$, for every δ > 0. Because ${\mathbb{P}}_n{f}^2=P{f}^2+n^{-1∕2}{\mathbb{G}}_n{f}^2$ and Pf² ≤ δ²PF² by assumption, we fing that

$${\text{E}}_P^{\ast }{\sigma }_{n,2}^2\lesssim {\delta }^{2}P{F}^2+\frac{1}{\sqrt{n}}{\text{E}}_P^{\ast }J\left(\frac{{\sigma }_{n,4}^2}{{\left({\mathbb{P}}_n{F}^4\right)}^{1∕2}},\mathcal{F},L_2\right){\left({\mathbb{P}}_n{F}^4\right)}^{1∕2}.$$ (3.3)

The next step is to bound σ_n,4 in terms of σ_n,2.

By Hölder’s inequality, for any conjugate pair (p, q) and any 0 < s < 4,

$${\mathbb{P}}_n{f}^4\le {\mathbb{P}}_n{\mid f\mid }^{4-s}{F}^s\le {\left({\mathbb{P}}^n{\mid f\mid }^{(4-s)p}\right)}^{1∕p}{\left({\mathbb{P}}_n{F}^{sq}\right)}^{1∕q}.$$

Choosing s such that (4 − s)p = 2 (and hence sq = (4p − 2)/(p − 1)), we find that

$${\sigma }_{n,4}^4\le {\sigma }_{n,2}^{2∕p}{\left({\mathbb{P}}_n{F}^{sq}\right)}^{1∕q}.$$

We insert this bound in (3.3). The function (x, y) ↦ x^1/py^1/q is concave, and hence the function $(x,y,z)\mapsto J(\sqrt{x^{1∕p}{y}^{1∕q}∕z},\mathcal{F},L_2)\sqrt{z}$ can be seen to be concave by the same arguments as in the proof of Theorem 2.1. Therefore, we can apply Jensen’s inequality to see that

$${\text{E}}_P^{\ast }{\sigma }_{n,2}^2\lesssim {\delta }^{2}P{F}^2+\frac{1}{\sqrt{n}}J\left(\frac{{\left({\text{E}}_P^{\ast }{\sigma }_{n,2}^2\right)}^{1∕\left(2p\right)}\left(P{F}^{sq}\right)1∕\left(2q\right)}{{\left(P{F}^4\right)}^{1∕2}},\mathcal{F},L_2\right){\left(P{F}^4\right)}^{1∕2}.$$

We conclude that $z≔{\left(E_P^{\ast }{\sigma }_{n,2}^2\right)}^{1∕2}∕{\parallel F\parallel }_{P,2}$ satisfies

$$\begin{array}{cc}\hfill z^2& \lesssim {\delta }^2+\frac{1}{\sqrt{n}}J\left(z^{1∕p}\frac{{\left(P{F}^2\right)}^{1∕\left(2p\right)}{\left(P{F}^{sq}\right)}^{1∕\left(2q\right)}}{{\left(P{F}^4\right)}^{1∕2}},\mathcal{F},L_2\right)\frac{{\left(P{F}^4\right)}^{1∕2}}{P{F}^2}\hfill \\ \hfill & \lesssim {\delta }^2+J(z^{1∕p},\mathcal{F},L^2)\frac{{\left(P{F}^{sq}\right)}^{1∕\left(2q\right)}}{\sqrt{n}{\left(P{F}^2\right)}^{1-1∕\left(2p\right)}}.\hfill \end{array}$$

In the last step we use that $J(C\delta ,\mathcal{F},L_2)\le CJ(\delta ,\mathcal{F},L_2)$ for C ≥ 1, and Hölder’s inequality as previously to see that the present C satisfies this condition. We next apply Lemma 2.1 (with r = 1/p) to obtain a bound on $J(z,\mathcal{F},L_2)$, and conclude the proof by substituting this bound in (2.2).

The preceding theorem assumes only a finite moment of the envelope function, but in comparison to Theorem 2.1 substitutes $J({\delta }^{1∕p},\mathcal{F},L_2)$ in the correction term of the upper bound, where p > 1 and hence δ1/p ≫ δ for small δ. In applications to moduli of continuity of minimum contrast criteria this is sufficient to obtain consistency with a rate, but typically the rate will be suboptimal. The rate improves as p ↓ 1, which requires finite moments of the envelope function of order increasing to infinity, the limiting case p = 1 corresponding to a bounded envelope, as in Theorem 2.1. The following theorem interpolates between finite moments of any order and a bounded envelope function. If applied to obtaining rates of convergence it gives rates that are optimal up to a logarithmic factor.

Theorem 3.2. Let $\mathcal{F}$ be a P-measurable class of measurable functions with envelope function F such that P exp(F^p+ρ) < ∞ for some p, ρ > 0 and such that ${\mathcal{F}}^2$ and ${\mathcal{F}}^4$ are P-measurable. If Pf² < δ²PF² for every f and some δ ∈ (0, 1/2), then for a constant c dependent on p, PF², PF⁴ and P exp(F^p+ρ),

$${\text{E}}_P^{\ast }{\parallel {\mathbb{G}}_n\parallel }_{\mathcal{F}}\le cJ(\delta ,\mathcal{F},L_2)\left(1+\frac{J(\delta {\left(\log (1∕\delta )\right)}^{1∕p},\mathcal{F},L_2)}{{\delta }^2\sqrt{n}}\right).$$

Proof. Fix r = 2/p. The functions ψ, $\overline{\psi }:[0,\infty )\mapsto [0,\infty )$ defined by

$$\psi \left(f\right)={\log }^r(1+f),\overline{\psi }\left(f\right)=e^{f^{1∕r}}-1,$$

are each other’s inverses, and are increasing from $\psi \left(0\right)=\overline{\psi }\left(0\right)=0$ to infinity. Thus their primitive functions $\Psi \left(f\right)={\int }_0^f\psi \left(s\right)ds$ and $\overline{\Psi }\left(f\right)={\int }_0^f\overline{\psi }\left(s\right)$ satisfy Young’s inequality $fg\le \Psi \left(f\right)+\overline{\Psi }\left(g\right)$, for every f, g ≥ 0 (e.g. [2], page 120, 3.38).

The function t ↦ t log^r(1/t) is concave in a neighbourhood of 0 (specifically: on the interval (0, e^1−r ∧ 1)), with limit from the right equal to 0 at 0, and derivative tending to infinity at this point. Therefore, there exists a concave, increasing function k: (0, ∞) → (0, ∞) that is identical to t ↦ t log^r(1/t) near 0 and bounded below and above by a positive constant times the identity throughout its domain. (E.g. extend t ↦ t log^r(1/t) linearly with slope 1 from the point where the derivative of the latter function has decreased to 1.) Write k(t) = tl^r(t), so that l^r is bounded below by a constant and l(t) = log(1/t) near 0. Then, for every t > 0,

$$\frac{\log (2+t∕C)}{\mathcal{l}\left(C\right)}\lesssim \log (2+t).$$ (3.4)

(The constant in ≲ may depend on r.) To see this, note that for C > c the left side is bounded by a multiple of log(2 + t/c), whereas for small C the left side is bounded by a multiple of $[\log (2+t)+\log (1+1∕C)]∕l\left(C\right)\lesssim \log (2+t)+1$.

From the inequality Ψ(f) ≤ fψ(f), we obtain that, for f > 0,

$$\Psi \left(\frac{f}{{\log }^r(2+f)}\right)\lesssim f.$$

Therefore, by (3.4) followed by Young’s inequality,

$$\begin{array}{cc}\hfill \frac{f^4}{k\left(C^2\right)}& =\frac{f^2∕C^2}{{\log }^r(2+f^2∕C^2)}\frac{f^2{\log }^r(2+f^2∕C^2)}{l^r\left(C^2\right)}\hfill \\ \hfill & \lesssim \frac{f^2}{C^2}+\overline{\Psi }\left(F^2{\log }^r(2+F^2)\right).\hfill \end{array}$$

On integrating this with respect to the empirical measure, with $C^2={\mathbb{P}}_n{f}^2$, we, see that, with $G=\overline{\Psi }\left(F^2{\log }^r(2+F^2)\right)$,

$${\mathbb{P}}_n{f}^4\lesssim k\left({\mathbb{P}}_n{f}^2\right)(1+{\mathbb{P}}_{n}G).$$

We take the supremum over f to bound ${\sigma }_{n,4}^4$ as in (3.2) in terms of $k\left({\sigma }_{n,2}^2\right)$, and next substitute this bound in (3.3) to find that

$$\begin{array}{cc}\hfill {\text{E}}_P^{\ast }{\sigma }_{n,2}^2& \le {\delta }^{2}P{F}^2+\frac{1}{\sqrt{n}}{\text{E}}_P^{\ast }J\left(\frac{\sqrt{k\left({\sigma }_{n,2}^2\right)}\sqrt{1+{\mathbb{P}}_{n}G}}{{\left({\mathbb{P}}_n{F}^4\right)}^{1∕2}},\mathcal{F},L_2\right){\left({\mathbb{P}}_n{F}^4\right)}^{1∕2}\hfill \\ \hfill & \le {\delta }^{2}P{F}^2+\frac{1}{\sqrt{n}}J\left(\frac{\sqrt{k\left({\text{E}}_P^{\ast }{\sigma }_{n,2}^2\right)}\sqrt{1+PG}}{{\left(P{F}^4\right)}^{1∕2}},\mathcal{F},L^2\right)\left(P{F}^4\right)1∕2,\hfill \end{array}$$

where we have used the concavity of k, and the concavity of the other maps, as previously. By assumption the expected value PG is finite for r = 2/p. It follows that $z^2={\text{E}}_P^{\ast }{\sigma }_{n,2}^2∕P{F}^2$ satisfies, for suitable constants a, b, c depending on r, PF², PF⁴ and PG,

$$z^2\lesssim {\delta }^2+\frac{a}{\sqrt{n}}J(\sqrt{k\left(z^{2}b\right)}c,\mathcal{F},L_2).$$

By concavity and the fact that k(0) = 0, we have k(Cz) ≤ Ck(z), for C ≥ 1 and z > 0. The function $z\mapsto \sqrt{k\left(z^{2}b\right)}c$ inherits this property. Therefore we can apply Lemma 3.1, with k of the lemma equal to the present function $z\mapsto \sqrt{k\left(z^{2}b\right)}c$, to obtain a bound on $J(z,\mathcal{F},L_2)$ in terms of $J(\delta ,\mathcal{F},L_2)$ and $J(\sqrt{k\left({\delta }^{2}b\right)c},\mathcal{F},L_2)$, which we substitute in (2.2). Here k(δ²) = δ²log^r(1/δ) for suffciently small δ > 0 and $k\left({\delta }^2\right)\lesssim {\delta }^2\lesssim {\delta }^2{\log }^r(1∕\delta )$ for δ < 1/2 and bounded away from 0. Thus we can simplify the bound to the one in the statement of the theorem, possibly after increasing the constants a, b, c to be at least 1, to complete the proof.

Lemma 3.1. Let $J:(0,\infty )\to \mathbb{R}$ be a concave, nondecreasing function with J(0) = 0, and let k: (0, ∞) → (0, ∞) be nondecreasing and satisfy k(Cz) ≤ Ck(z) for C ≥ 1 and z > 0. If z² ≤ A²+B² J(k(z)) for some A, B > 0, then

$$J\left(z\right)\lesssim J\left(A\right)\left[1+J\left(k\left(A\right)\right){\left(\frac{B}{A}\right)}^2\right].$$

Proof. As noted in the proof of Lemma 2.1 the properties of J imply that J(Cz) ≤ CJ(z) for C ≥ 1 and any z > 0. In view of the assumed property of k and the monotonicity of J it follows that $Jok\left(Cz\right)\le CJok\left(z\right)$ for every C ≥ 1 and z > 0. Therefore, by the monotonicity of J and k, and the assumption on z,

$$Jok\left(z\right)\le Jok\left(\sqrt{A^2+B^{2}Jok\left(z\right)}\right)\le Jok\left(A\right)\sqrt{1+{(B∕A)}^{2}Jok\left(z\right)}.$$

As in the proof of Lemma 2.1 we can solve this for J ο k(z) to find that

$$Jok\left(z\right)\lesssim Jok\left(A\right)+Jok{\left(A\right)}^2{\left(\frac{B}{A}\right)}^2.$$

Next again by the monotonicity of J,

$$\begin{array}{cc}\hfill J\left(z\right)& \le J\left(\sqrt{A^2+B^{2}Jok\left(z\right)}\right)\le J\left(A\right)\sqrt{1+{(B∕A)}^{2}Jok\left(z\right)}\hfill \\ \hfill & \lesssim J\left(A\right)\left[1+\left(\frac{B}{A}\right)\sqrt{Jok\left(A\right)}+{\left(\frac{B}{A}\right)}^{2}Jok\left(A\right)\right].\hfill \end{array}$$

The middle term on the right side is bounded by the sum of the first and third terms.